Returning the MD5-hash of all files in a directory to a .txt file with file name

Question

I have a directory of pictures.

I need the MD5-hash of each file in that directory and the name of the file placed into a .txt document. This file is to be read at a later time to reference the MD5 and file name together.

Note:
I would like this to just pull all directory files with me specifying them.
I have tried playing with tar and find and I just cannot seem to find a good solution...

This is a directory example:

/Desktop/testing
RandomFilename1.png
RandomFilename2.png
RandomFilename3.png

The .txt output is:

RandomFilename1,da39a3ee5e6b4b0d3255bfef95601890afd80709
RandomFilename2,da39a3ee5e6b4b0d3255bfef95601890afd80709
RandomFilename3,da39a3ee5e6b4b0d3255bfef95601890afd80709

I've looked everywhere online with no luck.
This will be on a Linux terminal.

Solution 1

You can use the following bash command:

for file in testing/*; do md5sum $file; done > res.txt

Of course you have to adjust the destination directory of res.txt.
The result could look similar to this:

8b1500ea6fe42927891fd1831bb955ba  testing/Pic1.gif
73f3222889a14411a3b78675ed0bc269  testing/Pic2.gif
c5b18ef1ea1346f13f668a7ead4e4c92  testing/Pic3.gif

So the MD5 hash is followed by the filename and path.

Solution 2

For what you want to do, execute md5sum and use sed or awk to transform the output

md5sum *.png | sed 's/^\([0-9a-f]\+\) \+\(.\+\)/\2,\1/' >images.md5sums
md5sum *.png | awk '{print $2","$1}' >image.md5sums

However, the plain output of md5sum is hash filename and is equivalent to your format. Using that standard md5sum output format is often more convenient, since you can use it directly with md5sum -c to check the hash of the files (after a copy or network transfer). Some other utilities may also expect this particular format.

Comments

[PerlDuck]

The benefit of this solution is that you can later check the md5's with md5sum -c res.txt. I'd also suggest a simple md5sum *.gif > res.txt instead of a loop.

[muru]

Also, you should move the redirect outside the loop: for file ... done > res.txt, instead of opening and closing that file for each run of the loop.

[Sergiy Kolodyazhnyy]

Nice, although I'd suggest couple improvements. Use for F in ./* instead of plain * there. Filenames with leading dashes will be interpreted as flags to command and not filename. I literally had spend 5 minutes figuring out why for f in * loop was hanging, and that was because I had - file , so expansion made md5sum - read from stdin. Second, you could do echo "$F", $(md5sum < "$F" | tr -d '-' ) and that also reduces complexity in parsing md5sum output.

[Sergiy Kolodyazhnyy]

What you can also do is md5sum "$F" | awk '{printf "%s,%s\n",substr($0,length($1)+3),$1 }' This is just 2 pipelines and should be safe against filenames with spaces.

[Kulfy]

@SergiyKolodyazhnyy Yeah you are right. It is unresponsive with filenames starting with special characters. I tried using ./* instead of simply * but I'm getting errors like: sed: -e expression #1, char 14: unknown option to 's' ./test.cpp. I don't know why. While the second suggestion echo "$F", $(md5sum < "$F" | tr -d '-' ) is working fine. Again with third suggestion md5sum "$F" | awk '{printf "%s,%s\n",substr($0,length($1)+3),$1 }' error is being generated invalid option with filenames having - in starting.

[Sergiy Kolodyazhnyy]

@Kulfy It probably complains about the / part because / is separator in sed, so you're doing something like sed -e 's/./foobar//g'. Not a big deal, GNU sed can use different separators so something like sed -e 's#./test##g' could work

[Kulfy]

@SergiyKolodyazhnyy Thank you for letting me know this. But the output is like ./<filename>, <mda5sum> ./ and sometimes ./<filename>, <mda5sum>