Recursively copying hidden files - Linux

Question

Is there an easy way to recursively copy all hidden files in a directory to another directory? I would like to back up just all the settings files in a home directory, not the normal files. I tried:

cp -R .* directory

but it recognizes . and .. and recursively copies all the non-hidden files too. Is there a way to get cp to ignore . and ..?

Solution 1

My favorite to move dirs in general has been:

tar cvf - . | (cd /dest/dir; tar xvf -)

which tars up the current directory to stdout then pipes it to a subshell that first cd's to the destination directory before untarring stdin. Simple, direct, extensible - consider what happens when you replace the () with an ssh to another machine. Or to answer your question you might do:

tar cvf - .* --exclude=\. --exclude=\.\. | (cd /dest/dir; tar xvf -)

Solution 2

Almost every time this can be solved just with:

cp -R .[a-zA-Z0-9]* directory

It's pretty unusual to have a hidden file that doesn't start with one of those characters.

Other pattern matches are available (.??*, .[^.]*) - see the comments

Solution 3

You could use rsync.

rsync -a ./ /some/other/directory/

that will copy the contents of the current directory (including dot files, but not including ..)

Solution 4

I implore you, step away from plain shell expansion on the cp command line - shell expansion has all sorts of ahem "interesting" corner cases (unwanted recursion caused by . and .., spaces, non-printable stuff, hardlinks, symbolic links, and so on.) Use find instead (it comes in the findutils package, in case you don't have it installed - which would be weird, all distributions install it by default):

find -H /path/to/toplevel/dir/ -maxdepth 1 -name '.*' -a \( -type d -o -type f -o -type l \) -exec cp -a '{}' /path/to/destination/dir/ \;

Step by step explanation:

• -H will cause find not to follow symlinks (except if the actual toplevel directory name you gave it is a symlink; that it will follow.)
• /path/to/toplevel/dir/ is, obviously, supposed to be replaced by you with the path do the directory which hosts the settings files and directories you want to back up.
• -maxdepth 1 will stop find from recursively descending into any directories whose name starts with a dot. We don't need it to recurse, cp will do that for us, we just need the names at this level.
• -name '.*' tells find that we want all names that start with a dot. This won't work correctly if the environment variable POSIXLY_CORRECT is set, but it rarely (if ever) is. This is the first match condition we have specified so far.
• a \( ....... \) is an and followed by a more complex condition in parentheses (I've used ..... to replace it, it's explained below.) We need to escape the parentheses since they'll otherwise be (mis)interpreted by the shell, hence the backslash in front of them,
• -type d -o -type f -o -type l are three conditions with an or between them. -type d matches directories, -type f matches regular files, and -type l matches symlinks. You can select what you want - for example, if you don't want to backup settings directories, omit -type d (and the -o right behind it, obviously.)
• -exec ..... \; tells find to execute a command every time a match is encountered. The end of the command is marked by a semicolon, which we again need to escape with a backslash to avoid shell interpretation. Within that command line, you need to use {} where you want the name of the currently encountered match to end up. Since shells might also misinterpret the curly braces, you should place them in apostrophes, as in '{}'. The command we want to execute in this case is cp -a '{}' /path/to/destination/dir/ (-a means archive, which recurses in subdirectories, copies symlinks as links, and preserves permissions and extended attributes, and /path/to/destination/dir/ is obviously the name of the destination directory - replace it.)

So, in plain English, this find command line says this:

Start at /path/to/toplevel/dir/. Do not descend into any subdirectories. Find all directories, files and symlinks whose name starts with a dot. For each of those you have found found, copy it to /path/to/destination/dir/ preserving nature, permissions, and extended attributes.

Solution 5

I've always used .??* to find hidden files without getting "." and "..". It might miss ".a" or something, though, but I never have one of those.

View more solutions

Comments

[Chaminda Bandara]

Is there any difference between -r and -R ?

[Hamish Downer]

That will only ignore .-xyz - the stuff in the [] only matches the first character after the dot.

[rclanan]

.[^.]* would catch everything

[rclanan]

... except stuff beginning with two periods.

[rclanan]

definately nice and neat :) +1

[Alnitak]

@roe - that's a nice version too

[Alnitak]

@mish - yes, I did say as much in the answer.

[Zifre]

I don't want it to mysteriously fail if there are some ".a"s

[EricMinick]

I like it because if you know you don't have any single character hidden files, it's really fast to type.

[lYriCAlsSH]

find is the most useful thing ever.

[Zifre]

Thanks, the exclude option is what I needed. I have already done it the other way, but I'll keep this for future reference. This seems really flexible (although maybe slower).

[Naveed Abbas]

Nice answer. Although I prefer to -exec ... ';' Avoiding backslash means this command will work both in CLI and in the crontab :))

[user59183]

Simple, and works for hidden files in hidden folders too.

[Naman Bansal]

Good advice. I have a couple of improvements: 1) Use cd /dest/dir && tar xvf -. The && will stop you from blatting over the source directory if you have a typo in the destination. 2) You only need the tar v flag on one of the tar commands (or neither).

[pjz]

Fair enough. I've also been known to group things the other way: to copy a remote srcdir to here, do (cd /src/dir && tar cf - .) | tar xvf -

[palerdot]

for me this is the less confusing and most effective of the solutions provided

[Ibn Saeed]

This has to be the best answer. It should be upvoted to the top.

[theruss]

Nice solution. I also needed it to not copy across .git dirs. Adding -C does this and so the command becomes: rsync -aC ./ /some/other/directory/

[Liquidgenius]

The above solution copies all files, including hidden files, not only hidden files for clarity.