Dump md5 and sha1 checksums with a single command!

Question

I am looking for command or utilities for calculating md5,sha1 hash value by one command.
Right now ubuntu has sha1sum and md5sum command for calculating hash value.

Solution 1

You can achieve this with some proper bash ninja-fu. :)

You know the procedure to calculate one at a time:

$ echo abc | md5sum
0bee89b07a248e27c83fc3d5951213c1  -
$ echo abc | sha1sum
03cfd743661f07975fa2f1220c5194cbaff48451  -

Edit: as @gertvdijk suggested, and reading the info pages a bit more, this can be done directly with tee and Process Substitution supported by modern shells, without redirects. This way you can pass your data to two processes and one file using tee:

$ echo abc | tee >(md5sum) >(sha1sum) > output.txt

It's also possible to chain if you need more, but you have to take care of STDOUT from all the subprocesses. This will NOT give you the expected result, but mixes the first two checksums together with the data in output.txt:

$ echo abc | tee >(md5sum) >(sha1sum) | tee >(sha256sum) >(sha512sum) > output.txt

If you redirect the checksums to a file inside the substituted processes, you can chain these as you like:

$ echo abc | tee >(md5sum > /tmp/md5.txt) >(sha1sum > /tmp/sha1.txt) | tee >(sha256sum > /tmp/sha256.txt) >(sha512sum > /tmp/sha512.txt) > output.txt

Here's my initial suggestion without process substitution, but which allows chaining/recursive use without mixing the data and the output:

$ echo abc | tee -a /proc/self/fd/2 2> >(sha1sum) > >(md5sum)
0bee89b07a248e27c83fc3d5951213c1  -
03cfd743661f07975fa2f1220c5194cbaff48451  -

The trick here is to use tee, which duplicates the data to STDOUT and a file. We're being clever by telling it to write the data to the file /proc/self/fd/2, which happens to always be the current process' STDERR file descriptor. And with the > >(program) syntax we can redirect each file descriptor to a program's STDIN instead of a file. Just like |, but with more control. > >(md5sum) redirects STDOUT to the md5sum program, while 2> >(sha1sum) redirects STDERR to the sha1sum program.

Note that the order of 2> and > seems to matter, I have to put 2> first on the command line. These are evaluated from right to left, but I'm not sure why this makes a difference.

To do this on a file or a hard drive, you should replace "echo abc" with a cat or a dd, eg:

dd if=/dev/sda bs=8k | tee -a /proc/self/fd/2 2> >(sha1sum) > >(md5sum)

The nifty thing about this is that you can actually recurse and run several at the same time, not just two. The syntax gets hairy, but this works:

echo abc | tee -a /proc/self/fd/2 2> >(tee -a /proc/self/fd/2 2> >(sha256sum) > >(sha384sum) ) > >(sha512sum)

If you want to capture the result and use it in a script, that works too:

A=$(echo abc | tee -a /proc/self/fd/2 2> >(sha1sum) > >(md5sum))

Now $A is a string containing all the output, including newlines. You can parse the values out later too:

echo "checksum=[$(echo "$A" | head -1 | cut -d " " -f 1)]"

I'm not sure you have any guarantees regarding ordering of the output though.

Solution 2

Cant help you with command line but I know a GUI tool named as quickhash.

You can download that tool from Quickhash

Description:

A Linux and Windows GUI to enable the rapid selection and subsequent hashing of files (individually or recursively throughout a folder structure) text and (on Linux) disks. Designed for Linux, but also available for Windows. MD5, SHA1, SHA256, SHA512 available. Output copied to clipboard or saved as CSV\HTML file.

Comments

[Lekensteyn]

Why do you want it? Usually you want to be able to verify hashes after generating them. For example, to generate a hash: md5sum hosts. Then, to verify this result: echo "b9adfb2e2022a3a84ba06b55eeb2dc64 hosts" | md5sum --check (should give: hosts: OK)

[gertvdijk]

+1. tee and clever use of output redirection in the shell is the way to go. This saves a lot of resources, especially when reading large files.

[gertvdijk]

By the way, I think you don't need to redirect to stderr to duplicate the output of the stream. The use of a subshell will also do the trick, maintaining stderr. See my example here in a blog post.

[ketil]

@gertvdijk Right, process substitution is cleaner, and easier to chain (you don't need to recurse). I'll update my response.

[gertvdijk]

Nice. I'd give you another upvote if I could. :-)

[EkriirkE]

While these work well for nice small files, you're doubling the efforts and processing time for larger files which I'm looking to avoid...

[ketil]

@EkriirkE, what do you mean by doubling the efforts? Yes, the data is duplicated for each checksum, but this is still only done in one pass. You could optimize this a bit more by writing a single application that does everything, and I'm sure there are tools that do this already, but for large amounts of data the IO tends to be the bottleneck. The above works great for huge files or disk images. Have you tried timing this vs just reading the data?