Using find and tar with files with special characters in the name

Question

I want to archive all .ctl files in a folder, recursively.

tar -cf ctlfiles.tar `find /home/db -name "*.ctl" -print`

The error message :

tar: Removing leading `/' from member names
tar: /home/db/dunn/j: Cannot stat: No such file or directory
tar: 74.ctl: Cannot stat: No such file or directory

I have these files: /home/db/dunn/j 74.ctl and j 75. Notice the extra space. What if the files have other special characters? How do I archive these files recursively?

Solution 1

Use the -T feature of tar to tell it to read the list of files from another file (tar treats each line as a separate file).

You can then use <() notation to have your shell generate a pseudo-file from the output of a command:

tar cf ctlfiles.tar -T <(find /home/db -name "*.ctl")

If your shell does not support <() notation, you can use a temporary file:

find /home/db -name "*.ctl" > ctlfile-list
tar cf ctlfiles.tar -T ctlfile-list
rm ctlfile-list

Solution 2

You can use the -print0 feature of find with the -0 feature of xargs, like this:

find /home/db -name '*.ctl' -print0 | xargs -0 tar -cf ctlfiles.tar

-print0 (that's hyphen-print-zero) tells find to use a null as the delimiter between paths instead of spaces, and -0 (that's hyphen zero) tells xargs to expect the same.

Edited to add:

If you have a large number of files, xargs may invoke tar more than once. See comments for ways to deal with that, or make find invoke tar directly, like this, which works with any number of files, even if they have spaces or newlines in their names:

rm -f ctlfiles.tar
find /home/db -name '*.ctl' -exec tar -rf ctlfiles.tar {} +

Solution 3

When the argument following "-T" is "-", the list of files is taken from stdin. Recent versions of tar typically support the "-null" option, which indicates that the files given in the source specified by the "-T" option are null-separated.

Hence the following works with an arbitrary number of files, possibly containing newline characters:

find /home/db -name '*.ctl' -print0 | tar --null -T - -cf ctlfiles.tar

Comments

[R Samuel Klatchko]

That will fail if xargs decides that it has too many arguments are wants to run the command multiple times.

[Andrew]

True, but I believe adding the -r option to tar should fix that.

[Andrew]

In fact if you use -r, you could probably dispense with xargs and let find invoke tar directly, like this: find /home/db -name '*.ctl' -exec tar -rf ctlfiles.tar {} \; Although in practice, the original answer would likely work -- if the number of files isn't in the thousands -- and would invoke tar once instead of N times.

[Sophie Alpert]

You could also set the number with xargs -0 -n 1000000000000 and then as long as you have less than a trillion items, you'll be set.

[John]

This fails if a filename contains a newline. Rare but possible.

[R Samuel Klatchko]

@davisre - good point

[akira]

the first part of the answer does not work because you pipe "names" as content to tar and not "filenames which tar has to put into the tarball". the xargs thing and executing tar for each found file (as suggested in the 2nd part) is unneeded.

[akira]

this is the right answer, it tells one instance of tar to read the filenames needed to get the content from stdin. @davisre: i tried to create a filename in zsh with ctrl-v-enter, this creates a \r and this solution still works. maybe with a \n it does not work but i think that is pretty rare to have such filenames, it is more probable to have unicode chars in the filenames than a \n.

[wchargin]

+1; this is the only safe and portable answer.

[wchargin]

You can fix the newline problem with --null; see David Bartlett’s answer. (This also works with arbitrary Unicode characters, as long as they use an extended ASCII encoding like UTF-8 or ISO-8859-x.)

[wchargin]

“since it’ll invoke tar once for each file”—just change {} \; to {} +.

[Rob Davis]

Thanks, wchargin. Updated.