How can I recursively copy files by file extension, preserving directory structure?

Question

At the Linux command line, I'd like to copy a (very large) set of .txt files from one directory (and its subdirectories) to another.

I need the directory structure to stay intact, and I need to ignore files except those ending in .txt.

Solution 1

You can use find and cpio to do this

cd /top/level/to/copy
find . -name '*.txt' | cpio -pdm /path/to/destdir

(-updm for overwrite destination content.)

Solution 2

cd /source/path
find -type f -name \*.txt -exec install -D {} /dest/path/{} \;

Solution 3

Easiest way that worked for me:

cp --parents -R jobs/**/*.xml ./backup/

one catch is you have to navigate to the "desired" directory before so the "parent path" is correct.

Also make sure that you enabled recursive globs in bash:

shopt -s globstar

Solution 4

Another approach

find . -name '*.txt' -exec rsync -R {} path/to/dext \;

Solution 5

I was trying to do the same thing on macOS, but none of the options really worked for me. Until i discovered ditto.

I had to copy many .wav files, and have it skip Video files... So here is what I came up with:

find . -type f -iname "*.wav" -ls -exec ditto {} /destination/folder/{} \;

• find . - Runs find in current folder. make sure you cd /source/folder before you start

• -type f - Specifies to only look for files

• -iname "*.wav" - This tells it to look for case insensitive *.wav
• -ls - This shows you the file that it is working on. Otherwase it shows nothing.
• -exec ditto {} /destination/folder/{} \; - Does all the work of copying and creating the files with the same directory tree.

View more solutions

Comments

[unclaimedbaggage]

nods Cheers - this would work, but without filtering to .txt I'm looking at a few million files (coming out at a few hundred GB). If need be I may have to, but I'd love to filter while copying if possible

[unclaimedbaggage]

Cheers, edited version works if I remove the '0' from -pmd0

[G-Man Says 'Reinstate Monica']

You should keep the 0 in -pmd0 and add -print0 to the end of the find command (just before the |).

[Iain J. Reid]

This method isn't recursive, meaning that for large directories you could be doing this for quite a while...

[Mubashar]

why m? i thought its just to retain the file modify date.

[grim]

You're missing a . after find. Also on macOS 10.13.1, this worked: find . -type f -name "*.txt" -exec install -v {} /dest/path/{} \;

[Aspartame_Xu]

I like this solution. I used find . -iname '*.txt' -exec rsync -Rptgon {} path/to/dext \; to do a case insensitive match and to preserver ownership and permissions.

[itapadar]

why? (this is classic of the voodoo involved in *sh command line invocations)