Recursive copy of specific files in Unix/Linux?

Question

I need to copy all *.jar files from directory and all its subdirectories. How can I do it in UNIX/Linux terminal? Command cp -r *.jar /destination_dir doesn't work.

Solution 1

rsync is useful for local file copying as well as between machines. This will do what you want:

rsync -avm --include='*.jar' -f 'hide,! */' . /destination_dir

The entire directory structure from . is copied to /destination_dir, but only the .jar files are copied. The -a ensures all permissions and times on files are unchanged. The -m will omit empty directories. -v is for verbose output.

For a dry run add a -n, it will tell you what it would do but not actually copy anything.

Solution 2

If you don't need the directory structure only the jar files, you can use:

shopt -s globstar
cp **/*.jar destination_dir

If you want the directory structure you can check cp's --parents option.

Solution 3

If your find has an -exec switch, and cp an -t option:

find . -name "*.jar" -exec cp -t /destination_dir {} +

If you find doesn't provide the "+" for parallel invocation, you can use ";" but then you can omit the -t:

find . -name "*.jar" -exec cp {} /destination_dir ";"

Solution 4

cp --parents `find -name \*.jar` destination/

from man cp:

--parents
       use full source file name under DIRECTORY

Solution 5

tar -cf - `find . -name "*.jar" -print` | ( cd /destination_dir && tar xBf - )

View more solutions

Comments

[ghoti]

The OP tagged this "unix", not Linux. AFAIK, the --parents option to cp is Linux-only.

[glenn jackman]

If the user's not using GNU find, there will be -exec but not with the + command terminator.

[user unknown]

Thanks, @glennjackman. Added a plusless solution.

[jordanm]

@glennjackman -1. The + command terminator is defined by POSIX. pubs.opengroup.org/onlinepubs/009604599/utilities/find.html

[glenn jackman]

@jordanm, hmm. On a Solaris 8 box at work, the first find in my path is GNU find 4.1, which doesn't have +. Guess I shouldn't take a 10 year old program as gospel.

[user unknown]

@glennjackman :) 10 years old? But mine is only 4.4.2 - doesn't look so dramatically.

[Eduardo Ivanec]

Why do you use the B flag for the destination tar? The man page says only "reblock as we read". I haven't ever run across obvious issues without it in similar settings, but I'm curious.

[Pavan Manjunath]

@EduardoIvanec Reblock will make sure that the copy has happened correctly. linuxdevcenter.com/pub/a/linux/lpt/18_16.html. Also, from tar GNU docs, If --read-full-records (-B) is used, tar will not panic if an attempt to read a record from the archive does not return a full record. Instead, tar will keep reading until it has obtained a full record But as you observed, it should run fine in most of the cases.

[Sean]

+1 for shopt globstar, I didn't know about either of those things, useful.

[halloleo]

Thanks for pointing out the --parents option. It does something slightly different to the posters requirement: It creates the directory structure as far as needed for the files to copy, not the whole directory structure. - Was exactly what I needed! :-)

[Matteo]

Would you mind explaining me this part: -f 'hide,! */' ? Thanks in advance for your help!

[Sean]

Sure. The -f means it is a filter rule (short for --filter). The rule says hide all non-directories, ie files. The '*/' pattern matches a directory, then the ! negates it to mean anything that is not a directory (so, a file). The '--include=*.jar' has precedence over the filter so .jar files (only) are included.

[Matteo]

That was very useful, thks!Just to make sure I got it, you are filtering the command by saying to match everything that is not a directory, which means all the files. And the --include takes care of making sure that only .m files are considered. A last question, what if I were interested in copying only the files in that directory, without copying the entire directory structure?

[Sean]

Yes, match all files and hide them, except for .jar files. Not sure I understand your last question: do you mean flattening the tree structure? If so see: stackoverflow.com/questions/9800989/… Or do you mean copy the .jar files from directory A to dir B? Plain old 'cp' will do that for you, or 'rsync *.jar /path/to/new/dir'

[Matteo]

Cool, thks again for your explanation. I mean copy all (and only) the .jar files in a directory A to a directory B, without going in the sub-directories of A.

[lreeder]

This doesn't recursively copy subdirectories.

[user unknown]

@lreeder: It does for me. Gnu-find v. 4.4.2 - just tested.

[lreeder]

Hmm, still doesn't work recursively for me. find -version says find (GNU findutils) 4.4.2. find . -name "*.jar" -exec cp -t /destination_dir {} + copies all files flatly to /destination_dir, but not their parent directories.

[user unknown]

@lreeder: No, it doesn't copy the directories which wasn't part of the problem specification but to copy all *.jar files from directory and all its subdirectories.

[Ryan]

This is clearly not the correct answer to this question, I'm not sure why it was marked as accepted.

[CivFan]

@Ryan It's not so clear to at least 44 upvoters. Why is this not the correct answer to this question?

[Stuart Cardall]

cp -rf --parents destination_dir works perfectly

[alexey]

@CivFan I think what Ryan wanted is the flattening of the directories (i.e. copy of all jar files from subdirectories to one folder). This is ambiguous in OP's question as he doesn't say if destination_dir should contain subfolders or be flat. I need it to be flat and question suggested by Sean above has a nice answer by Kaz that worked for me.

[David]

I had an issue with the first tar quitting early due to signal 13 (broken pipe) with this method. Only some of the files were copied.

[IsMakeFire]

I had a permission issue, which I initially fixed by doing a sudo, then I realized where "/destination_dir" was in my file system :P