N0x0n

YouTube link: https://youtu.be/wVyu7NB7W6Y

Invidious link: https://inv.nadeko.net/watch?v=wVyu7NB7W6Y

Sorry for the formatting... Tried to remove the URL for better readability, but there seems some kind of bug.

TLDW

• hack phones remotely just knowing it's phone number
• Intercept 2FA sms
• Intercept phone calls
• Reroute phone calls
• Geolocation of a target

I dunno if it has already been posted/discussed here but this kinda blew my mind ! Sorry there's a lot of clickbait but the general subject is interesting...

I never heard of SS7 and have actually no idea how the whole phone system communication works but that's kinda scary...

Yes we are probably not the first target with this "hack" nor is it as easy as exposed in this video and nor do we have 14k $ to spend on this, but that's not out of reach for some people. I mean it's not as expensive as Pegasus and people with the mean and some good stable income can probably misuse this system for targeting specific vulnerable people (example in the video).

Hello everyone :).

Trying to keep it short, cause after 2 days of troubleshooting I'm a bit tired and really confused on what happened here... Maybe it's my lack of understanding about Legacy BIOS/UEFI/EFI/Bootloader/GRUB... But that was really an odd "issue" that resolved by itself?

Intro

My 15 years old laptop, an Asus N76V, is still going strong, though It's purpose is not the same as a few years back. It's working great as a mini-server to host docker containers, DNS server, firewall, wireguard tunnel...

Space left on my volume group (LVM) was getting tight so I decided to install a new Samsung 1TB SSD into it's second slot. As easy as it is, I though this would not take more than 30 minutes...

Old SSD just vanished from boot option in bios.

After booting into BIOS to see if the new SSD is recognized, everything seems okay, however my primary SSD containing the Bootloader and system just vanished as boot option?? I was not to angry about it because if something strange had happened and everything got wiped for whatever reason, I still have my daily backups, so the troubleshooting begins...

Things I have tried to bring my primary SSD back as boot option in BIOS

Though my secondary and new SSD shows as boot option, it's empty and has no Bootloader or system installed.

1. Change a lot of things in the BIOS

From AHCI to password protection and disable anti-theft, secure boot... I think I changed every option possible (related to SSD) in the BIOS and reverted back to default without my primary SSD showing again as boot option.

2. Clear CMOS

Opening the case and taking out the little battery to clear the CMOS had not effect at all either.

3. boot into Debian rescue mode, boot-repair, chroot session

From a Live session USB tried a few things in rescue mode and even in the Live session used the boot-repair tool and tried to manually fix the EFI/Bootloader in a chrooted environment, because Yes even if I couldn't boot into my system my data was still there an safe !!

4. Disconnect new SSD, swap bay position.

Even when I removed the new SSD from it's SATA connection or swaped SATA bay connections, the primary SSD didn't showed back into the BIOS....

Guess I have to reinstall Debian on my new SSD?

So I gave up on trying to fix the Bootloader and primary SSD and just went for a fresh Debian install on my new SSD.

Nothing uncommon during the installation process except that during the partition all my volume groups and logical volumes from my primary SSD are visible, so I left them alone and created a new VG and LVs. I do as usual a LVM manual partioning with EXT4 filesystem with separate /boot partition, however I forgot to set the ESP partition (was getting a bit late and got a bit on my nerves...). Installation went without issues.

And then It came back....

So booting into my BIOS to see if my new SSD boot position is okay... Ohhhhh and what a surprise to see my primary SSD back as boot choice... however my new SSD isn't there anymore (expected as I forgot to set an ESP partition... And Bootloader is from my primary SSD).

Booting into my system I'm greeted by the Bootloader? GRUB? With 2 choices

• My old linux OS
• My new linux OS

Some kind of relief and happy moment after 2 days...

Kinda curious what happens If set the /boot/EFI partition on secondary SSD

I got kinda confused on what happened here so to further confuse myself I reformatted and reinstalled Debian on the secondary disk with the correct /boot/EFI partition.

And I got even more confused... The boot priority in BIOS only shows the secondary SSD as boot option...!?

Questions

What happend here?

1. Why did my primary SSD (which had a proper Bootloader and clean system) disappeared as boot option in the BIOS as soon as I installed a new blank SSD and didn't came back even when the new one was unplugged from SATA?

2. Why does my BIOS only shows 1 disk as possible boot device when both have proper Bootloader and system?

3. Does 1 external Bootloader suffice to make both system work?!

Not sure about the last question... so maybe I'm looking more or less to sharpen my understanding of a Bootloader/EFI/UEFI/GRUB... And any hint to a good resource, book, eBook to get a better understanding is really appreciated :)

I know those are a lot of words (and I said I will keep it short...) but I think the context is important here, and I'm not able to express correctly my issue/thoughts without it.

Thanks in advance for those who beared with me and read the whole text and can hint me to the correct direction to a better understanding on what happened.

Edit

My question was very badly written but the new title reflect the actual question. Thanks to 3 very friendly and dedicated users (@harsh3466 @tuna @learnbyexample) I was able to find a solution for my files, so thank you guys !!!

For those who will randomly come across this post here are 3 possible ways to achieve the desired results.

Solution 1 (https://lemmy.ml/post/25346014/16383487)

#! /bin/bash
files="/home/USER/projects/test.md"

mdlinks="$(grep -Po ']\((?!https).*\)' "$files")"
mdlinks2="$(grep -Po '#.*' <<<$mdlinks)"

while IFS= read -r line; do
	#Converts 1.2 to 1-2 (For a third level heading needs to add a supplementary [0-9]) 
	dashlink="$(echo "$line" | sed -r 's|(.+[0-9]+)\.([0-9]+.+\))|\1-\2|')"
	sed -i "s/$line/${dashlink}/" "$files"

	#Puts everything to lowercase after a hashtag
	lowercaselink="$(echo "$dashlink" | sed -r 's|#.+\)|\L&|')"
	sed -i "s/$dashlink/${lowercaselink}/" "$files"

	#Removes spaces (%20) from markdown links after a hashtag
	spacelink="$(echo "$lowercaselink" | sed 's|%20|-|g')"
	sed -i "s/$lowercaselink/${spacelink}/" "$files"

done <<<"$mdlinks2"

Solution 2 (https://lemmy.ml/post/25346014/16453351)

sed -E ':l;s/(\[[^]]*\]\()([^)#]*#[^)]*\))/\1\n\2/;Te;H;g;s/\n//;s/\n.*//;x;s/.*\n//;/^https?:/!{:h;s/^([^#]*#[^)]*)(%20|\.)([^)]*\))/\1-\3/;th;s/(#[^)]*\))/\L\1/;};tl;:e;H;z;x;s/\n//;'

Solution 3 (https://lemmy.ml/post/25346014/16453161)

perl -pe 's/\[[^]]+\]\((?!https?)[^#]*#\K[^)]+(?=\))/lc $&=~s:%20|\d\K\.(?=\d):-:gr/ge'

Relevant links

https://mike.bailey.net.au/notes/software/apps/obsidian/issues/markdown-heading-anchors/#background

Hi everyone !

I'm in need for some assistance for string manipulation with sed and regex. I tried a whole day to trial & error and look around the web to find a solution however it's way over my capabilities and maybe here are some sed/regex gurus who are willing to give me a helping hand !

With everything I gathered around the web, It seems it's rather a complicated regex and sed substitution, here we go !

What Am I trying to achieve?

I have a lot of markdown guides I want to host on a self-hosted forgejo based git markdown. However the classic markdown links are not the same as one github/forgejo...

Convert the following string:

[Some text](#Header%20Linking%20MARKDOWN.md)

Into

[Some text](#header-linking-markdown.md)

As you can see those are the following requirement:

• Pattern: [Some text](#link%20to%20header.md)
• Only edit what's between parentheses
• Replace space (%20) with -
• Everything as lowercase
• Links are sometimes in nested parentheses
  • e.g. (look here [Some text](#link%20to%20header.md))
• Do not change a line that begins with https (external links)

While everything is probably a bit complex as a whole the trickiest part is probably the nested parentheses :/

What I tried

The furthest I got was the following:

sed -Ei 's|\(([^\)]+)\)|\L&|g' test3.md #make everything between parentheses lowercase

sed -i '/https/ ! s/%20/-/g' test3.md #change every %20 occurrence to -

These sed/regx substitution are what I put together while roaming the web, but it has a lot a flaws and doesn't work with nested parentheses. Also this would change every %20 occurrence in the file.

The closest solution I found on stackoverflow looks similar but wasn't able to fit to my needs. Actually my lack of regex/sed understanding makes it impossible to adapt to my requirements.

I would appreciate any help even if a change of tool is needed, however I'm more into a learning processes, so a script or CLI alternative is very appreciated :) actually any help is appreciated :D !

Thanks in advance.

Hello :)

There isn't any community about note taking where I could post my question and no this is not a "What's the best note taking app" question...

I'm getting tired of maintaining my Obsidian vaults... Somehow I'm fighting to get it right and obsidian seems to fight back. I've got 4 vaults of the same subject and I always end to make a mess out of it and make a fresh one... Also my notes a scattered in all direction and the more my knowledge base grows the less I seem to be able to find something...

This is probably a me problem rather than Obsidian issue. The way I'm taking notes are not compatible with Obsidian. IMO Obsidian's defaults configuration are bad and visually not appealing. Sure customization in Obsidian is "endless" but digging in the HTML code to change the style or adding plugins to somehow get something visually appealing seems more like a chore than actually taking notes.

Here I'm again roaming the web for a Note taking app the could fit my needs and after trying a lot of different apps (please don't suggest the already well known apps... I have probably already tried it...) I couldn't find something that fits my workflow.

The only one that looked great and simple was osmosnote but it isn't maintained anymore. There's also dendron but it's in maintenance mode. So there goes the only ones that looked promising from my perspective.

After giving it more thoughts, I was looking for something that could:

• Keep my scripts updated
• Simple markdown text
• No database
• Local first
• Open source
• If webapp self-hostable
• Back-linking
• Keep track of changes

Except for back-linking, a self-hosted Forgejo with git seems to fit all my needs, however I'm not sure if this is the right tool and I'm scared that in the long run I will mess it up the same way I did with Obsidian.

Does anyone here has some experience and is taking notes that way? I'm really curious on your experience and maybe your thoughts if it's feasible ? Practical ?

Please don't suggest Org.mode or Emacs ! They look very cool and very promising but they are WAY to much overkill ! And they also implement a totally new way of taking notes... Relearning on how to take notes will probably give me the last hit on abandoning to document anything !

Thank you for any helpful input !

cross-posted from: https://lemmy.ml/post/23615167

For better visibility I cross-post my question in this community.

Heyha ! I just came across a very odd issue/bug that somehow resolved by itself without knowing who or what was the culprit.

For context, YouTube doing his thing making nearly all public instances obsolete, I'm self-hosting a Piped instance in my homelab via Docker.

Everything is going smoothly, self-signed certs, traefik, accessible via Wireguard outside of my network, and and and !! LibreTube connects without any issues to my Piped instance on my Android phone and so does RiMusic.

However, in RiMusic when I was trying to access my synced Piped playlists, RiMusic went crazy and my playlist seemed to be in a query loop were I was unable to play any songs and was flickering alot.

• Reboot the phone => Same behavior
• Reboot the piped instance => Same behavior
• Uninstall RiMusic/New docker piped instance => Same behavior
• Flush everything from cache/playlist/configuration/data... => Same behavior

Nothing seems to resolve the issue software wise, next step check the logs (Interesting part):

My piped-nginx showed A HUGE amount of requests coming from my phone when accessing a Piped playlist:

"GET /playlists/d0e2c698-f3f4-435f-b2c9-96c6d3a88781 HTTP/1.1" 200 4161 "-" "ktor-client" "10.XXX.XXX.XXX"

Traefik also showed a lot of loadbalacing debug notifications something that never happens, because I'm the only user in my homelab setup !

My first though was that this is probably a RiMusic bug, but before reporting a report to GitHub, I did other debugging stuff.

• Create an account and connect to a public piped instance
• Create playlist/add some songs
• Connect with RiMusic

The exact same behavior EXCEPT it stopped the loop after a few requests and made RiMusic usable again and was able to play my playlist without issues. Try again on my own instance but again, infinite loop, a lot of requests on Traefik and Piped-nginx. It even broke my Piped instance...

The only logical explanation is that the public piped instances have some request rate limiting (Yeah I know this is common practice and even mandatory on public instances). So here I go rate limiting my own requests to see if this could work as a temporary workaround while writing a GitHub bug report to RiMusic.

Adding some basic traefik labels just to give it a try:

labels:
  - "traefik.http.middlewares.test-ratelimit.ratelimit.average=10"
  - "traefik.http.middlewares.test-ratelimit.ratelimit.burst=20"

At first nothing happened but after a few docker compose -f down/up I was able to access my playlist from my own instance without any issues/bug/strangeness. Cool It works? So just out of curiosity I commented out the new traefik middelwares and restarted both container (Traefik/Piped). And .... RiMusic playlist connected to my piped instance works without the ratelimite lines... Wait what ??

What just happend ? I have absolutely no idea... I don't even know if the mentioned labels did anything... But everything works... No loading loop, No Traefik container overflown with loadblancer logs, No Piped-nginx with thousand request... It just vanished as it never existed in the first place.

I'm totally clueless except that somehow when accessing a playlist in private or public piped instance with RiMusic my phone went crazy with an infinite loop of api requests (Dunno if that's the correct term :/). Here Am I with no idea what actually happend...

And yes my phone is Heavely debloated and firewalled (Magisk,rethinkDNS) so those are not unknown requests from the web or any open source application, whats so ever !

Sorry for the long write up I hope It's readable and comprehensible. I just wanted to share my experience with you and If you also encountered some strange and inexplicable bug/issue that resolved by itself, feel free to share :).

PS: If someone has any good lead on what happened or some good insight where I should look next to get more out of this experience, I'm open to every good read !

Heyha ! I just came across a very odd issue/bug that somehow resolved by itself without knowing who or what was the culprit.

For context, YouTube doing his thing making nearly all public instances obsolete, I'm self-hosting a Piped instance in my homelab via Docker.

Everything is going smoothly, self-signed certs, traefik, accessible via Wireguard outside of my network, and and and !! LibreTube connects without any issues to my Piped instance on my Android phone and so does RiMusic.

However, in RiMusic when I was trying to access my synced Piped playlists, RiMusic went crazy and my playlist seemed to be in a query loop were I was unable to play any songs and was flickering alot.

• Reboot the phone => Same behavior
• Reboot the piped instance => Same behavior
• Uninstall RiMusic/New docker piped instance => Same behavior
• Flush everything from cache/playlist/configuration/data... => Same behavior

Nothing seems to resolve the issue software wise, next step check the logs (Interesting part):

My piped-nginx showed A HUGE amount of requests coming from my phone when accessing a Piped playlist:

"GET /playlists/d0e2c698-f3f4-435f-b2c9-96c6d3a88781 HTTP/1.1" 200 4161 "-" "ktor-client" "10.XXX.XXX.XXX"

Traefik also showed a lot of loadbalacing debug notifications something that never happens, because I'm the only user in my homelab setup !

My first though was that this is probably a RiMusic bug, but before reporting a report to GitHub, I did other debugging stuff.

• Create an account and connect to a public piped instance
• Create playlist/add some songs
• Connect with RiMusic

The exact same behavior EXCEPT it stopped the loop after a few requests and made RiMusic usable again and was able to play my playlist without issues. Try again on my own instance but again, infinite loop, a lot of requests on Traefik and Piped-nginx. It even broke my Piped instance...

The only logical explanation is that the public piped instances have some request rate limiting (Yeah I know this is common practice and even mandatory on public instances). So here I go rate limiting my own requests to see if this could work as a temporary workaround while writing a GitHub bug report to RiMusic.

Adding some basic traefik labels just to give it a try:

labels:
  - "traefik.http.middlewares.test-ratelimit.ratelimit.average=10"
  - "traefik.http.middlewares.test-ratelimit.ratelimit.burst=20"

At first nothing happened but after a few docker compose -f down/up I was able to access my playlist from my own instance without any issues/bug/strangeness. Cool It works? So just out of curiosity I commented out the new traefik middelwares and restarted both container (Traefik/Piped). And .... RiMusic playlist connected to my piped instance works without the ratelimite lines... Wait what ??

What just happend ? I have absolutely no idea... I don't even know if the mentioned labels did anything... But everything works... No loading loop, No Traefik container overflown with loadblancer logs, No Piped-nginx with thousand request... It just vanished as it never existed in the first place.

I'm totally clueless except that somehow when accessing a playlist in private or public piped instance with RiMusic my phone went crazy with an infinite loop of api requests (Dunno if that's the correct term :/). Here Am I with no idea what actually happend...

And yes my phone is Heavely debloated and firewalled (Magisk,rethinkDNS) so those are not unknown requests from the web or any open source application, whats so ever !

Sorry for the long write up I hope It's readable and comprehensible. I just wanted to share my experience with you and If you also encountered some strange and inexplicable bug/issue that resolved by itself, feel free to share :).

PS: If someone has any good lead on what happened or some good insight where I should look next to get more out of this experience, I'm open to every good read !

Oupsii !

TIL something new... My hate for MacOS took over common logic. 2.8GB, 3 seconds file transfer on USB was to beautiful to be true. After some further investigation and hints from @JonnyRobbie@lemmy.world @nanook@friendica.eskimo.com I learned that Linux writes to cache before writing it to the device, to see whats happening in the background: sync & watch -n 1 grep -e Dirty: /proc/meminfo.

Still, the transfer speed on Linux was slightly faster than on MacOS. My rant was unjustified, It just my fault for being clueless on some more advanced Linux stuff. But I learned something new today, so this post was actually helpful !

Howerver, I still hate MacOS and will probably give Asahi remix a try.

Thanks to everyone !

Hey guys ! I'm getting tired/bored of MacOS' shenanigans... Yesterday was the last drop that make me think of trying an alternative.

While trying to upload a 2.8 GB file over to an USB-C stick it took like 8 minutes? Okay that's "good" enough if you only do it from time to time... But 25 files takes literally 1h30min... Are we in 2001?

I mean the exact same 2.8GB file, with the exact same USB-C stick took FU***** 3 seconds on Linux !!

Ohh and don't think I didn't tried to "fix" the issue, after a long search on the web I came across a lot of people having similar issues that aren't fixed since 2 major updates? With a total radio silence from the shiny poisonous Apple...

Among other things I tried:

• Disable Spotlight indexing sudo mdutil -a -i off
• Reformat the USB stick from Mac
• All available filesystem FAT32, exFAT...(yes even MacOS native APFS)
• Another USB stick
• ....

Enough is enough. I was willing to learn their way of thinking for my personal experience and somehow always got my way around to reproduce what I learned on Linux to Mac. But now that there is an alternative OS, I think I'm ready to get back home.

So does anyone here already gave Asahi Remix a try? If so what was your experience with it?

I read their FAQ and most of their documentation and it seems good enough for daily drive (except for some quirks here and there) but I wanted to hear from people who already made the jump and how was their personal feeling.

PS: I got that MacOS for my birthday from a family member with good intentions. That wasn't a personal choice. While I'm more than happy and thankful for the gift, I totally hate it more and more... Especially because MOST of my self-hosted services, applications, scripts, are open source.

Hi everyone !

Intro

Was a long ride since 3 years ago I started my first docker container. Learned a lot from how to build my custom image with a Dockerfile, loading my own configurations files into the container, getting along with docker-compose, traefik and YAML syntax... and and and !

However while tinkering with vaultwarden's config and changing to postgresSQL there's something that's really bugging me...

Questions

• How do you/devs choose which database to use for your/their application? Are there any specific things to take into account before choosing one over another?

• Does consistency in database containers makes sense? I mean, changing all my containers to ONLY postgres (or mariaDB whatever)?

• Does it make sense to update the database image regularly? Or is the application bound to a specific version and will break after any update?

• Can I switch between one over another even if you/devs choose to use e.g. MariaDB ? Or is it baked/hardcoded into the application image and switching to another database requires extra programming skills?

Maybe not directly related to databases but that one is also bugging me for some time now:

• What's redis role into all of this? I can't the hell of me understand what is does and how it's linked between the application and database. I know it's supposed to give faster access to resources, but If I remember correctly, while playing around with Nextcloud, the redis container logs were dead silent, It seemed very "useless" or not active from my perspective. I'm always wondering "Humm redis... what are you doing here?".

Thanks :)

Edit

After a long process of roaming the web, re-runs and troubleshoot the script with this wonderful community, the script is functional and does what it's intended to do. The script itself is probably even further improvable in terms of efficiency/logic, but I lack the necessary skills/knowledge to do so, feel free to copy, edit or even propose a more efficient way of doing the same thing.

I'm greatly thankful to @AernaLingus@hexbear.net, @GenderNeutralBro@lemmy.sdf.org, @hydroptic@sopuli.xyz and Phil Harvey (exiftool) for their help, time and all the great idea's (and spoon-feeding me with simple and comprehensive examples ! )

How to use

Prerequisites:

• parallel package installed on your distribution

Copy/past the below script in a file and make it executable. Change the start_range/end_range to your needs and install the parallel package depending on your OS and run the following command:

time find /path/to/your/image/directory/ -type f | parallel ./script-name.sh

This will order only the pictures from your specified time range into the following structure YEAR/MONTH in your current directory from 5 different time tag/timestamps (DateTimeOriginal, CreateDate, FileModifyDate, ModifyDate, DateAcquired).

You may want to swap ModifyDate and FileModifyDate in the script, because ModifyDate is more accurate in a sense that FileModifyDate is easily changeable (as soon as you make some modification to the pictures, this will change to your current date). I needed that order for my specific use case.

From: '-directory<$DateAcquired/' '-directory<$ModifyDate/' '-directory<$FileModifyDate/' '-directory<$CreateDate/' '-directory<$DateTimeOriginal/'

To: '-directory<$DateAcquired/' '-directory<$FileModifyDate/' '-directory<$ModifyDate/' '-directory<$CreateDate/' '-directory<$DateTimeOriginal/'

As per exfitool's documentation:

ExifTool evaluates the command-line arguments left to right, and latter assignments to the same tag override earlier ones.

#!/bin/bash

if [ $# -eq 0 ]; then
    echo "Usage: $0 <filename>"
    exit 1
fi

# Concatenate all arguments into one string for the filename, so calling "./script.sh /path/with spaces.jpg" should work without quoting
filename="$*"

start_range=20170101
end_range=20201230

FIRST_DATE=$(exiftool -m -d '%Y%m%d' -T -DateTimeOriginal -CreateDate -FileModifyDate -DateAcquired -ModifyDate "$filename" | tr -d '-' | awk '{print $1}')

if [[ "$FIRST_DATE" != '' ]] && [[ "$FIRST_DATE" -gt $start_range ]] && [[ "$FIRST_DATE" -lt $end_range ]]; then
        exiftool -api QuickTimeUTC -d %Y/%B '-directory<$DateAcquired/' '-directory<$ModifyDate/' '-directory<$FileModifyDate/' '-directory<$CreateDate/' '-directory<$DateTimeOriginal/' '-FileName=%f%-c.%e' "$filename"

else
        echo "Not in the specified time range"

fi

Hi everyone !

Please no bash-shaming, I did my outmost best to somehow put everything together and make it somehow work without any prior bash programming knowledge. It took me a lot of effort and time.

While I'm pretty happy with the result, I find the execution time very slow: 16min for 2288 files.

On a big folder with approximately 50,062 files, this would take over 6 hours !!!

If someone could have a look and give me some easy to understand hints, I would greatly appreciate it.

What Am I trying to achieve ?

Create a bash script that use exiftool to stripe the date from images in a readable format (20240101) and compare it with an end_range to order only images from that specific date range (ex: 2020-01-01 -> 2020-12-30).

Also, some images lost some EXIF data, so I have to loop through specific time fields:

• DateTimeOriginal
• CreateDate
• FileModifyDate
• DateAcquired

The script in question

#!/bin/bash

shopt -s globstar

folder_name=/home/user/Pictures
start_range=20170101
end_range=20180130


for filename in $folder_name/**/*; do

	if [[ $(/usr/bin/vendor_perl/exiftool -m -d '%Y%m%d' -T -DateTimeOriginal "$filename") =~ ^[0-9]+$ ]]; then
		DateTimeOriginal=$(/usr/bin/vendor_perl/exiftool -d '%Y%m%d' -T -DateTimeOriginal "$filename")
	        if  [ "$DateTimeOriginal" -gt $start_range ] && [ "$DateTimeOriginal" -lt $end_range ]; then
			/usr/bin/vendor_perl/exiftool -api QuickTimeUTC -r -d %Y/%B '-directory<$DateTimeOriginal/' '-FileName=%f%-c.%e' "$filename"
			echo "Found a value"
		echo "Okay its $(tput setab 22)DateTimeOriginal$(tput sgr0)"

		fi

        elif [[ $(/usr/bin/vendor_perl/exiftool -m -d '%Y%m%d' -T -CreateDate "$filename") =~ ^[0-9]+$ ]]; then
                CreateDate=$(/usr/bin/vendor_perl/exiftool -d '%Y%m%d' -T -CreateDate "$filename")
                if  [ "$CreateDate" -gt $start_range ] && [ "$CreateDate" -lt $end_range ]; then
                        /usr/bin/vendor_perl/exiftool -api QuickTimeUTC -r -d %Y/%B '-directory<$CreateDate/' '-FileName=%f%-c.%e' "$filename"
                        echo "Found a value"
                echo "Okay its $(tput setab 27)CreateDate$(tput sgr0)"
                fi

        elif [[ $(/usr/bin/vendor_perl/exiftool -m -d '%Y%m%d' -T -FileModifyDate "$filename") =~ ^[0-9]+$ ]]; then
                FileModifyDate=$(/usr/bin/vendor_perl/exiftool -d '%Y%m%d' -T -FileModifyDate "$filename")
                if  [ "$FileModifyDate" -gt $start_range ] && [ "$FileModifyDate" -lt $end_range ]; then
                        /usr/bin/vendor_perl/exiftool -api QuickTimeUTC -r -d %Y/%B '-directory<$FileModifyDate/' '-FileName=%f%-c.%e' "$filename"
                        echo "Found a value"
                echo "Okay its $(tput setab 202)FileModifyDate$(tput sgr0)"
                fi


        elif [[ $(/usr/bin/vendor_perl/exiftool -m -d '%Y%m%d' -T -DateAcquired "$filename") =~ ^[0-9]+$ ]]; then
                DateAcquired=$(/usr/bin/vendor_perl/exiftool -d '%Y%m%d' -T -DateAcquired "$filename")
                if  [ "$DateAcquired" -gt $start_range ] && [ "$DateAcquired" -lt $end_range ]; then
                        /usr/bin/vendor_perl/exiftool -api QuickTimeUTC -r -d %Y/%B '-directory<$DateAcquired/' '-FileName=%f%-c.%e' "$filename"
                        echo "Found a value"
                echo "Okay its $(tput setab 172)DateAcquired(tput sgr0)"
                fi

        elif [[ $(/usr/bin/vendor_perl/exiftool -m -d '%Y%m%d' -T -ModifyDate "$filename") =~ ^[0-9]+$ ]]; then
                ModifyDate=$(/usr/bin/vendor_perl/exiftool -d '%Y%m%d' -T -ModifyDate "$filename")
                if  [ "$ModifyDate" -gt $start_range ] && [ "$ModifyDate" -lt $end_range ]; then
                        /usr/bin/vendor_perl/exiftool -api QuickTimeUTC -r -d %Y/%B '-directory<$ModifyDate/' '-FileName=%f%-c.%e' "$filename"
                        echo "Found a value"
                echo "Okay its $(tput setab 135)ModifyDate(tput sgr0)"
                fi

        else
                echo "No EXIF field found"

done

Things I have tried

1. Reducing the number of if calls

But it didn't much improve the execution time (maybe a few ms?). The syntax looks way less readable but what I did, was to add a lot of or ( || ) in the syntax to reduce to a single if call. It's not finished, I just gave it a test drive with 2 EXIF fields (DateTimeOriginal and CreateDate) to see if it could somehow improve time. But meeeh :/.

#!/bin/bash

shopt -s globstar

folder_name=/home/user/Pictures
start_range=20170101
end_range=20201230

for filename in $folder_name/**/*; do

        if [[ $(/usr/bin/vendor_perl/exiftool -m -d '%Y%m%d' -T -DateTimeOriginal "$filename") =~ ^[0-9]+$ ]] || [[ $(/usr/bin/vendor_perl/exiftool -m -d '%Y%m%d' -T -CreateDate "$filename") =~ ^[0-9]+$ ]]; then
                DateTimeOriginal=$(/usr/bin/vendor_perl/exiftool -d '%Y%m%d' -T -DateTimeOriginal "$filename")
		CreateDate=$(/usr/bin/vendor_perl/exiftool -d '%Y%m%d' -T -CreateDate "$filename")
                if  [ "$DateTimeOriginal" -gt $start_range ] && [ "$DateTimeOriginal" -lt $end_range ] || [ "$CreateDate" -gt $start_range ] && [ "$CreateDate" -lt $end_range ]; then
                        /usr/bin/vendor_perl/exiftool -api QuickTimeUTC -r -d %Y/%B '-directory<$DateTimeOriginal/' '-directory<$CreateDate/' '-FileName=%f%-c.%e' "$filename"
                        echo "Found a value"
                echo "Okay its $(tput setab 22)DateTimeOriginal$(tput sgr0)"

                else
			echo "FINISH YOUR SYNTAX !!"
		fi

	fi
done

2. Playing around with find

To recursively find my image files in all my folders I first tried the find function, but that gave me a lot of headaches... When my image file name had some spaces in it, it just broke the image path strangely... And all answers I found on the web were gibberish, and I couldn't make it work in my script properly... Lost over 4 yours only on that specific issue !

To overcome the hurdle someone suggest to use shopt -s globstar with for filename in $folder_name/**/* and this works perfectly. But I have no idea If this could be the culprit of slow execution time?

3. Changing all [ ] into [[ ]]

That also didn't do the trick.

How to Improve the processing time ?

I have no Idea if it's related to my script or the exiftool call that makes the script so slow. This isn't that much of a complicated script, I mean, it's a comparison between 2 integers not a hashing of complex numbers.

I hope someone could guide me in the right direction :)

Thanks !

cross-posted from: https://lemmy.ml/post/15968883

Hello everyone ! Nobody seems to have an answer on !networking@sh.itjust.works (or maybe they are not interested because it's an enteprise network community?) and !homenetworking@selfhosted.forum seems dead?

Anyway, If anyone could guide me or direct me to the right direction, I would really appreciate it !

TL:DR

What is encapsulated into the frame that makes everyone understand: "OHHH that’s for 10.0.0.8, your docker container on bridge network br-b1de on the veth2b interface !!! "

Hi everyone !

I'm scratching my head in finding an actual answer on how virtual networking in docker actually works (mostly on the packets/frame level) or some good documentation to improve my understanding on how everything fits together.

Because I'm probably lacking the correct network terminology I made a simple network topology of my network. Don't hesitate to correct any network mistake.

In my scenario, my docker container with the virtual interface veth2b22c98 and the following ip (10.0.0.8) connects to bridge network br-b1de95b5ea89. When I curl, from my conntainer, lemmy.ml the packets/frame is send to my enp4s0 and goes through my wireguard tunnel to my VPN provider which sends back the packet/frame/handshake...

I probed every interface with tcpdump (enp4s0, wg0, br-b1,veth2b):

• enp4s0: Every packet/frame is encapsulated into the wireguard protocol with my physical interface's IP (192.168.1.30) and no DNS is visible on that interface (like expected) and sends it out to my ISP's public IP.

• wg0: Shows every packet/frame with the actual protocol with my wireguard's interface IP (192.168.2.1) with the destination IP of lemmy.ml (Dst: 54.36.178.108)

• br-b1: Shows every packet/frame with the actual protocol with my containers IP (10.0.0.8) with the destination IP of lemmy.ml (Dst: 54.36.178.108)

I know there is a mix of 2 different concepts in my scenario (wireguard tunnel and virtual networking) but I really do not understand how the frame gets back to my docker container. When I look at the frames on wg0, there is no mention of either the MacAddress of my container or the actual IP of my container.

How/when/what ? is exactly happening to my frame so that it gets to the correct target between my physical interface, virtual interface, bridge ? I mean with VLAN's there's a VLAN tag on the frame, so you can easily identify with Wireshark where it should go. But here, I cannot find any clue who or what is doing the magic so the frame finds it's way back to my docker container.

What is encapsulated into the frame that makes everyone understand: "OHHH that's for 10.0.0.8, your docker container on bridge network br-b1de on the veth2b interface !!! "

Sorry for my broken English and lack of networking terminology and thank you for those who beared with me and are willing the give me some hints/proper networking lesson.

Edit: Whoops I just read that networking@sh.itjust.works is for enterprise networks? I hope my small homelab question doesn't break the rules? If so I will redirect my question.

Hi everyone !

I'm scratching my head in finding an actual answer on how virtual networking in docker actually works (mostly on the packets/frame level) or some good documentation to improve my understanding on how everything fits together.

Because I'm probably lacking the correct network terminology I made a simple network topology of my network. Don't hesitate to correct any network mistake.

In my scenario, my docker container with the virtual interface veth2b22c98 and the following ip (10.0.0.8) connects to bridge network br-b1de95b5ea89. When I curl, from my conntainer, lemmy.ml the packets/frame is send to my enp4s0 and goes through my wireguard tunnel to my VPN provider which sends back the packet/frame/handshake...

I probed every interface with tcpdump (enp4s0, wg0, br-b1,veth2b):

• enp4s0: Every packet/frame is encapsulated into the wireguard protocol with my physical interface's IP (192.168.1.30) and no DNS is visible on that interface (like expected) and sends it out to my ISP's public IP.

• wg0: Shows every packet/frame with the actual protocol with my wireguard's interface IP (192.168.2.1) with the destination IP of lemmy.ml (Dst: 54.36.178.108)

• br-b1: Shows every packet/frame with the actual protocol with my containers IP (10.0.0.8) with the destination IP of lemmy.ml (Dst: 54.36.178.108)

I know there is a mix of 2 different concepts in my scenario (wireguard tunnel and virtual networking) but I really do not understand how the frame gets back to my docker container. When I look at the frames on wg0, there is no mention of either the MacAddress of my container or the actual IP of my container.

How/when/what ? is exactly happening to my frame so that it gets to the correct target between my physical interface, virtual interface, bridge ? I mean with VLAN's there's a VLAN tag on the frame, so you can easily identify with Wireshark where it should go. But here, I cannot find any clue who or what is doing the magic so the frame finds it's way back to my docker container.

What is encapsulated into the frame that makes everyone understand: "OHHH that's for 10.0.0.8, your docker container on bridge network br-b1de on the veth2b interface !!! "

Sorry for my broken English and lack of networking terminology and thank you for those who beared with me and are willing the give me some hints/proper networking lesson.

Edit: Changed something on my network diagram (wireguard is not in a container it's bare bone on the server) and some typo.