Pass xargs argument as a string to another command with '>'?

Question

Suppose I have a bunch of files in /tmp/.

I do

find . -type f | xargs -iFILES

And I want to do a md5sum in each file, outputting to a file with the same name, but with .md5 extension.

find . -type f | xargs -iFILES md5sum FILES > FILES.md5

This is supposed to create a md5 file for each file found by find command. Instead, it creates a single FILES.md5 file on disk with checksums from all files.

How do I say to md5sum command that the FILES represent the current filename and not a FILES literal string?

Solution 1

You need some way to say that you want to send the output of md5sum to a file. Since find (or xargs) doesn't have this functionality built-in, and md5sum only knows how to print to standard output, a shell redirection is the most straightforward way.

Note that your command won't work in the general case for another reason: the output format of find is not the input format of xargs, they differ with file names containing whitespace or \"'. Use find -exec instead.

find . -type f -exec sh -c 'md5sum "$0" >"$0.md5"' {} \;

Solution 2

You need to use a subshell to handle the IO redirection:

find . -type f | xargs -iFILES sh -c 'md5sum FILES > FILES.md5'

Solution 3

With GNU Parallel you can do:

find . -type f | parallel md5sum {} \> {}.md5

You get the added benefit of running md5sum in parallel and that files like:

My brother's 12" records.txt

will not cause your command to crash. Watch the intro video to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ