cp behaves weirdly when . (dot) or .. (dot dot) are the source directory

Question

This answer reveals that one can copy all files - including hidden ones - from directory src into directory dest like so:

mkdir dest
cp -r src/. dest

There is no explanation in the answer or its comments as to why this actually works, and nobody seems to find documentation on this either.

I tried out a few things. First, the normal case:

$ mkdir src src/src_dir dest && touch src/src_file src/.dotfile dest/dest_file
$ cp -r src dest
$ ls -A dest
dest_file  src

Then, with /. at the end:

$ mkdir src src/src_dir dest && touch src/src_file src/.dotfile dest/dest_file
$ cp -r src/. dest
$ ls -A dest
dest_file  .dotfile  src_dir  src_file

So, this behaves simlarly to *, but also copies hidden files.

$ mkdir src src/src_dir dest && touch src/src_file src/.dotfile dest/dest_file
$ cp -r src/* dest
$ ls -A dest
dest_file  src_dir  src_file

. and .. are proper hard-links as explained here, just like the directory entry itself.

Where does this behaviour come from, and where is it documented?

Solution 1

The behaviour is a logical result of the documented algorithm for cp -R. See POSIX, step 2f:

The files in the directory source_file shall be copied to the directory dest_file, taking the four steps (1 to 4) listed here with the files as source_files.

. and .. are directories, respectively the current directory, and the parent directory. Neither are special as far as the shell is concerned, so neither are concerned by expansion, and the directory will be copied including hidden files. *, on the other hand, will be expanded to a list of files, and this is where hidden files are filtered out.

src/. is the current directory inside src, which is src itself; src/src_dir/.. is src_dir’s parent directory, which is again src. So from outside src, if src is a directory, specifying src/. or src/src_dir/.. as the source file for cp are equivalent, and copy the contents of src, including hidden files.

The point of specifying src/. is that it will fail if src is not a directory (or symbolic link to a directory), whereas src wouldn’t. It will also copy the contents of src only, without copying src itself; this matches the documentation too:

If target exists and names an existing directory, the name of the corresponding destination path for each file in the file hierarchy shall be the concatenation of target, a single slash character if target did not end in a slash, and the pathname of the file relative to the directory containing source_file.

So cp -R src/. dest copies the contents of src to dest/. (the source file is . in src), whereas cp -R src dest copies the contents of src to dest/src (the source file is src).

Another way to think of this is to compare copying src/src_dir and src/., rather than comparing src/. and src. . behaves just like src_dir in the former case.

Solution 2

When you run cp -R src/foo dest, you'll get dest/foo. So if directory dest/foo does not exist, cp will create it, and then copy the contents of src/foo to dest/foo.

When you run cp -R src/. dest, cp sees that dest/. exists, and then it's just the matter of copying the contents of src/. to dest/..

When you think of it as copying a directory named . from src and merging its contents with the existing directory dest/., it will make sense.

Comments

[AlexP]

What do you mean nobody can find documentation? The cp reference clearly explains how cp -R works. . and .. are directories just like any other directories, there is nothing magical or mysterious about them.

[roaima]

I have tried to explain why it works at How to copy a folder recursively in an idempotent way using cp

[iFreilicht]

But it doesn't behave the same way. Specifying src will copy the directory into dest, src/. will copy the contents. I'll try to make that clearer in the question.

[Stephen Kitt]

There, I think that answers your underlying question.

[Stéphane Chazelas]

Not sure what you mean by so neither are concerned by expansion (which avoids issues with the shell expanding hidden files or not). cp -r src/.* dest/ is a problem in shells like bash that don't remove . and .. from their globs (fixed in zsh, fish, pdksh (from the Forsyth shell) and derivatives).

[Stephen Kitt]

@Stéphane the OP compares copying src/. and src/* (note, not src/.*); src/* doesn’t include hidden files if globbing ignores them...

[Stéphane Chazelas]

Ah OK. Note that yash -o dotglob -c 'echo *' does include . and ... * can include . and .. as well in ksh depending on how you configure FIGNORE.

[Stephen Kitt]

@Stéphane indeed — I didn’t particulary want to go into globbing subtleties in this answer ;-).

[ilkkachu]

Hmm, "directory containing source_file". Well, obviously src contains src/. but it does mean that the containing directory of a directory depends on how you name the directory. Of course the existence of the . links in a way means that all directories contain themselves, but that might not be intuitive for all. Instead of this behaviour, one might also be tempted to assume that "the directory containing directory foo" would be determined by foo/.., in which case it wouldn't matter if we refer to foo or foo/.: the resulting containing directory would be the same.

[ilkkachu]

Which is to say that the distinction between foo and foo/. seems a bit delicate, but I don't mind, I also find it slightly amusing.

[Mark Amery]

Not convinced this reading of the spec makes sense. If "the directory containing" src/. is src, then logically shouldn't "the directory containing" src/src_dir/.. be src/src_dir? In which case, shouldn't the pathname of the file relative to the directory containing source_file, for the file src_dir, be ../src_dir? And thus, shouldn't we expect cp -R src/src_dir/.. target to create a copy of src_dir as a sibling of target, not a child? Obviously, that's not what cp really does, but it seems like it should if we accept your interpretation here. What am I missing?

[Stephen Kitt]

@Mark the construction applies to files (and directories) inside the source, being copied to the target. Let’s say src/src_dir contains a file named a. cp -r src/src_dir/.. target copies everything inside src/src_dir/.., i.e. inside src; so src_dir and src_dir/a, which are (after construction based on the given source directory) src/src_dir/../src_dir and src/src_dir/../src_dir/a. Their paths relative to the source directory are src_dir and src_dir/a, and the result is target/src_dir and target/src_dir/a.

[Mark Amery]

@StephenKitt "Their paths relative to the source directory are src_dir and src_dir/a" - but how can this be consistent with the handling of .? If the relative path of src_dir when you do cp -r src is src/src_dir, and the relative path of src_dir when you do cp -r src/. is src_dir, then when you do cp -r src/src_dir/.. shouldn't the relative path be ../src_dir? Concluding otherwise seems to mean we should treat src as "containing" src/., but not treat src/src_dir as "containing" src/src_dir/... But... why? What justification for that is there in spec, if any?

[Admin]

I think this code is relevant to the discussion github.com/coreutils/coreutils/blob/…