How to I copy the directory and subfolders but ignore certain files in the subfolder?

Question

I believe the question is best asked with an example.

/home
   test1.txt
   test2.txt
   /home/my-folder
      test3.txt
      test4.txt

1. test1.txt, test2.txt and my-folder folder are inside /home.
2. test3.txt and text4.txt are inside /home/my-folder.

I want to copy all the contents of /home folder but exclude the 2 files (test3.txt and test4.txt) inside my-folder.

How can I do it using cp?

I know it's possible with rsync as I just tried it but there are times when rsync is not installed in a server and I don't have rights to install software.

Solution 1

You can do it with find(1) and cpio(1):

find /home -path './my-folder/test[34].txt' -prune -o \( -type f -print \) | \
    cpio -pdamv /some/other/dir

Solution 2

You can't do this with cp alone, short of listing the files to copy. Making partial copies goes beyond cp's capabilities.

Rsync is the obvious tool for the job and it's very widespread.

If you only have POSIX tools, you can use pax. You can omit files by rewriting their path to an empty string.

cd /home && pax -rw -pe -s'~^\./my-folder/test[34]\.txt$~~' . /path/to/destination

If you have only a minimal Linux server that lacks pax, see if its traditional equivalents cpio or tar are available. See lcd047's answer for a cpio example. With GNU tar, you can do

mkdir /path/to/destination
tar -cf - -C /home --exclude='./my-folder/test[34].txt' . |
  tar -xf - -C /path/to/destination

Solution 3

Rsync

I was in need of something like this and dive a bit on the forums. As mentioned from Gilles, I find that the best way is to use RSYNC. Two things that I like about it:

• You can use an external file like .gitignore as an input to exclude the files and folders you don't want to copy over.
• it's efficiency when you need to do repeat the same backup(copy) of the same source directory(you don't need to copy existing files or folders) as I will mention below in Example 4.

Example 1: Exclude a specific file:

rsync -a --exclude 'file.txt' originalDirectory/ backupDirectory/

Example 2: Exclude a specific folder(e.g. named dirName):

rsync -a --exclude 'dirName' originalDirectory/ backupDirectory/

Example 3: Exclude multiple files and folders:

rsync -a --exclude={'file1.txt', 'file2.json','dir1/*','dir2'} originalDirectory/ backupDirectory/

Example 4: Exclude multiple files and folders:

rsync -a --exclude-from='exclude.txt' originalDirectory/ backupDirectory/

exclude.txt contains the following:

file1.txt
file2.json
file3.pdf
dir1
dir2
*.png

Comments

[lcd047]

This (1) assumes bash, (2) doesn't descend into subdirectories, and (3) omits my-folder completely, not just the two files.

[lcd047]

Care to explain the downvotes? Thank you!

[mikeserv]

Looks like there's only the one, and some people just downvote a pipe - which might be why. But I'm curious - I though you could only prune a directory - does your use of the -path primitive enable you to also -prune explicit or even globbed filenames as well? Cause that's pretty damn cool, if so.

[mrjayviper]

what if I want to exclude all files with the txt extension? or perhaps a combo of .txt and .tmp? Thanks again :)

[Gilles 'SO- stop being evil']

@mrjayviper Then make the pattern *.txt. To ignore *.tmp as well, add a second -s or --exclude argument, you can have as many as you like.

[lcd047]

@mikeserv My understanding is -prune removes selected nodes from a tree, it doesn't matter if the nodes are terminals or not. Pruning file globs certainly works fine with both GNU find and BSD find.

[mikeserv]

So... are you not going to do the getting to know you thing? It's cool if you don't.

[mikeserv]

You know, this isn't the first exchange of yours where I've you explicitly referred to yourself as a human. Most people tend to take that kind of thing for granted. I consider those that don't very suspicious. So out w/ it already - are you a robot?