Best way run a command on each file in a directory tree

Question

There appear be a number of ways to do this including loops in shell script, find and xargs. Which of these is best, and which is most portable?

Solution 1

find . -exec <command> {} \;

for the current directory as the root of the tree or in general:

find <directory> -exec <command> {} \;

Solution 2

Use the -print0 option to find and the -0 option to xargs if you have file or directory names with spaces:

find {dir} -print0 | xargs -0r {command}

The -print0 option to find prints out the filenames as a NUL-separated rather than whitespace separated list, while the -0 option to xargs instructs it to break its input on NUL rather than whitespace. Since NUL is one of the characters that is not allowed in Unix filenames, there is no way for it to be mistaken for part of a filename.

The -r option is there so that xargs won't give you an error if it has no input.

Solution 3

If portability is an issue I would stay away from shell-specific ways of doing it (there are lots of people using lots of different shells, but in general find and xargs are really basic tools that no one would dare change in a radical way).

Using basic arguments of find and xargs should give you a more portable solution.

Solution 4

If grouping arguments together is acceptable, find | xargs will probably give better performance, since it will execute the command a lot fewer times than find -exec. If you want to execute the command each time for each file or execute in the subdirectory of the file you should probably use find -exec or -execdir.

As a rule, it's preferable to stay away from shell-specific loops; find & xargs are enough for most scenarios.

Solution 5

Make sure the version of the command you're using doesn't already have a flag for recursive operation. :)

View more solutions

Comments

[Hemant]

on some systems find dont have -exec option there you can use find . | xargs <Command>

[Admin]

Indeed, and if you need to filter by something, try -name "*.c" or for example -type d for directories...

[Lucas Jones]

Find can do surprisingly complex things - you might want to check the man page - unixhelp.ed.ac.uk/CGI/man-cgi?find - for the full option list.

[Shawn Chin]

You might also want to watch out for filenames with spaces, which may break the -exec and xargs approach. See tinyurl.com/ygvgmw5

[ElBel]

-1 Won't work for filenames containing spaces. Better use find -print0 | xargs -0.

[vonbrand]

If possible, use the find ... | xargs ... idiom, it processes files in batches, and this will usually be faster (less processes launched).