Linux: copy and create destination dir if it does not exist

Question

I want a command (or probably an option to cp) that creates the destination directory if it does not exist.

Example:

cp -? file /path/to/copy/file/to/is/very/deep/there

Solution 1

mkdir -p "$d" && cp file "$d"

(there's no such option for cp).

Solution 2

If both of the following are true:

1. You are using the GNU version of cp (and not, for instance, the Mac version), and
2. You are copying from some existing directory structure and you just need it recreated

then you can do this with the --parents flag of cp. From the info page (viewable at http://www.gnu.org/software/coreutils/manual/html_node/cp-invocation.html#cp-invocation or with info cp or man cp):

--parents
     Form the name of each destination file by appending to the target
     directory a slash and the specified name of the source file.  The
     last argument given to `cp' must be the name of an existing
     directory.  For example, the command:

          cp --parents a/b/c existing_dir

     copies the file `a/b/c' to `existing_dir/a/b/c', creating any
     missing intermediate directories.

Example:

/tmp $ mkdir foo
/tmp $ mkdir foo/foo
/tmp $ touch foo/foo/foo.txt
/tmp $ mkdir bar
/tmp $ cp --parents foo/foo/foo.txt bar
/tmp $ ls bar/foo/foo
foo.txt

Solution 3

Short Answer

To copy myfile.txt to /foo/bar/myfile.txt, use:

mkdir -p /foo/bar && cp myfile.txt $_

How does this work?

There's a few components to this, so I'll cover all the syntax step by step.

The mkdir utility, as specified in the POSIX standard, makes directories. The -p argument, per the docs, will cause mkdir to

Create any missing intermediate pathname components

meaning that when calling mkdir -p /foo/bar, mkdir will create /foo and /foo/bar if /foo doesn't already exist. (Without -p, it will instead throw an error.

The && list operator, as documented in the POSIX standard (or the Bash manual if you prefer), has the effect that cp myfile.txt $_ only gets executed if mkdir -p /foo/bar executes successfully. This means the cp command won't try to execute if mkdir fails for one of the many reasons it might fail.

Finally, the $_ we pass as the second argument to cp is a "special parameter" which can be handy for avoiding repeating long arguments (like file paths) without having to store them in a variable. Per the Bash manual, it:

expands to the last argument to the previous command

In this case, that's the /foo/bar we passed to mkdir. So the cp command expands to cp myfile.txt /foo/bar, which copies myfile.txt into the newly created /foo/bar directory.

Note that $_ is not part of the POSIX standard, so theoretically a Unix variant might have a shell that doesn't support this construct. However, I don't know of any modern shells that don't support $_; certainly Bash, Dash, and zsh all do.

---

A final note: the command I've given at the start of this answer assumes that your directory names don't have spaces in. If you're dealing with names with spaces, you'll need to quote them so that the different words aren't treated as different arguments to mkdir or cp. So your command would actually look like:

mkdir -p "/my directory/name with/spaces" && cp "my filename with spaces.txt" "$_"

Solution 4

Such an old question, but maybe I can propose an alternative solution.

You can use the install programme to copy your file and create the destination path "on the fly".

install -D file /path/to/copy/file/to/is/very/deep/there/file

---

There are some aspects to take in consideration, though:

1. you need to specify also the destination file name, not only the destination path
2. the destination file will be executable (at least, as far as I saw from my tests)

You can easily amend the #2 by adding the -m option to set permissions on the destination file (example: -m 664 will create the destination file with permissions rw-rw-r--, just like creating a new file with touch).

---

And here it is the shameless link to the answer I was inspired by =)

Solution 5

Shell function that does what you want, calling it a "bury" copy because it digs a hole for the file to live in:

bury_copy() { mkdir -p `dirname $2` && cp "$1" "$2"; }

View more solutions

Comments

[ephemient]

I don't think you need the test -d: mkdir -p still succeeds even if the directory already exists.

[Michael Krelin - hacker]

oops, right. But then, test may be a bash builtin (especially if written as [[ -d "$d" ]]) and mkdir can not ;-)

[ephemient]

mkdir is a builtin in BusyBox ;)

[Tarnay Kálmán]

You should have quotation marks around the dirname $2 too

[olt]

This doesn't work on Mac OS X, so I guess it's Linux specific.

[Michael Krelin - hacker]

@holms, $d is a variable that presumably contains the directory in question. I mean, if you have to repeat it thrice you're likely to assign it to variable first.

[Michael Krelin - hacker]

I'd say gnu-specific. If you have macports you can install coreutils and its gcp.

[Michael Krelin - hacker]

@Kalmi, for proper quotation you'd also want to quote $2 as an argument to dirname. Like mkdir -p "$(dirname "$2")". Without this quoting $2 in cp is useless ;)

[user2918201]

Man, linux has everything

[user467257]

Note that this answer assumes $2 is not already the target directory, otherwise you'd just want "mkdir -p "$2" && cp "$1" "$2" as the function body

[David Winiecki]

I feel like this is the answer to a slightly different question, even though it is the answer I was looking for (and therefore the one I upvoted). I guess this is the solution assuming you want the destination parent directories to be the same as the origin parent directories, which is probably the use case that most people reading this are interesting in.

[Mark Amery]

Aside: I'm usually disappointed in the lack of detail in questions about Bash or Unix shell commands on Stack Overflow, which frequently throw out a complicated and extremely dense one-liner that's hard to unpick if you're not already a Unix wizard. I've tried to go for the opposite extreme here and both explain every detail of the syntax and link to the appropriate places to find documentation on how this stuff works. I hope this helps at least some folks. Also, credit where due: I nicked this approach from this answer to a duplicate question: stackoverflow.com/a/14085147/1709587

[Rich]

This doesn't work for me -- I got "No such file or directory" on the dest dir

[Rich]

I think this has a bug, as @user467257 points out. I don't think this is fixable, as $2 is ambiguous between the target dir and the target file in this scenario. I have posted an alternative answer which addresses this.

[thdoan]

This assumes directory 'bar' already exists, but the original question implies how to automatically create 'bar' when it's provided as the destination and it doesn't already exist. The answer to this is provided by @MichaelKrelin-hacker.

[Spongman]

actually, it's not the same at all. his requires you to pass the filename twice (hence the '-t' flag) and it set the execute bits on the file (hence the '-m' example). his should not be the top answer, since it doesn't actually answer the question - whereas mine does.

[MediaVince]

WOW that is a great resource, i can even copy many files at once provided the destination exists: cp --parents foo/bar/z.txt foo/baz/r.csv bar/foo/h.doc /previously/madedir/

[MediaVince]

better yet it can copy dirs with cp -R --parents my/needed/dir/ /previously/madedir/ then check it with tree !$

[thdoan]

--parents has its uses, but it won't work if, say, you want to base the directory creation only on the destination path (e.g., copy file c in directory a/b into directory d/e/f and create the destination path if necessary).

[bfontaine]

As others commented on the other answer you don’t need to test if the directory exists before calling mkdir -p. It’ll succeed even if it already exists.

[thebunnyrules]

I tried it and go this result: rsync -a bull /some/hoop/ti/doop/ploop RESULT rsync: mkdir "/some/hoop/ti/doop/ploop" failed: No such file or directory (2) rsync error: error in file IO (code 11) at main.c(675) [Receiver=3.1.2]

[thebunnyrules]

I've never seen $_ before. what does it refer to? the folder used in the last command?

[Mark Amery]

@thebunnyrules But... but... there's a "How does this work?" section in my answer that literally contains multiple paragraphs specifically devoted to the question you just asked. It feels ignored and unloved. :(

[thebunnyrules]

Sorry about that. You're absolutely right. I already knew everything else in your answer so I just quickly scanned through it. I should have read it more carefully.

[thebunnyrules]

Wow, you really did put alot of work into that answer... Thanks for that. It was nice to learn about: $_. I can see myself using it quite a bit from now on. I wrote a script to automate the whole process and put up in this thread. Give it a go, maybe you'll like: stackoverflow.com/questions/1529946/…

[Yury]

@Rich, I disagree that this is not fixable. The following works just fine for both files and directories: mkdir -p dirname "$2" && cp -r "$1" "$2"; Note that the backticks around dirname $2 don't show up because SO parses them as code markers.

[Rich]

@Yury, what I mean is that if you do "cp x a/b" and there is not yet a directory "b", then the invocation is ambiguous between renaming the file to "b" inside directory "a", or moving the file to a new directory "b" inside "a" (i.e. "a/b/x"). See my answer for more info.

[marcdahan]

@thebunnyrules the you have to check the path with pwd command (look at :[link ] (access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Li‌​nux/… )) and insert correctly it into your instruction

[thebunnyrules]

If I understand your suggestion correctly, you're proposing that I specify a complete path to the source aka use: "$(pwd)/bull" in my command instead of just the filename: bull? The thing is, the error is with rsync creating destination dir, not the source (bull). Seems to me rsync uses a simple mkdir and not a mkdir -p.

[Bernhard Döbler]

I just did mkdir -p /foo/bar && touch $_/myfile.txt to create a new file in a non existent directory. Worked like a charm.

[kvantour]

This works for a single file. Just take into account that there is the final file and not the final subdirectory.

[törzsmókus]

note that this command creates the destination files with 755 (rwxr-xr-x) permissions, which is probably not desired. you can specify something else with the -m switch, but I could not find a way to just keep the file permissions :(

[Xedret]

This is perhaps an overkill, but pretty efficient, works as expected, thank you good sir.

[borracciaBlu]

Agree. In man cp: -R If source_file designates a directory, cp copies the directory and the entire subtree connected at that point.

[Yordan Georgiev]

minor detail : one could also: mkdir -p /foo/bar && cp ../from/another/dir/myfile.txt $_/my-file-with-new-name

[Arnie97]

The man pages for --parents is, IMHO, too brief and confusing: use full source file name under DIRECTORY. The full documentation via info cp is quite nice, though.

[JoeAC]

rsync is more predictable than cp. For example, if I wish to cp /from/somedir /to-dir then cp behaves differently if /to-dir exists already: If /to-dir exists, then cp will create /to-dir/some-dir, otherwise just the contents of /from/somedir are placed into /to-dir . However, rsync behaves the same in case, i.e, /to-dir/some-dir will always be created.

[xbmono]

Thanks! This is what I needed. In my case I don't know whether the destination has directory or not and whether the directory exists or not so this code gives me the parent directory which then I can create it safely if it doesn't exist

[Shital Shah]

It doesn't. The '/path/to/dst/' must exist beforehand.

[T S]

Should be pointed out that ditto does not ship with most Linux distributions.

[ssasi]

how about cp -r file "$d" ?

[Michael Krelin - hacker]

@ssasi won't work, will say "directory doesn't exist" if you supply trailing / or copy as an intended directory if you don't and parent directory exists. It's to recursively copy directory, not to create directories.

[sEver]

Overengineering at it's finest.

[Mr Mayur]

This is bad practice and should not be use.

[cokedude]

@MichaelKrelin-hacker What does the $d mean?

[cokedude]

@MarkAmery have you tried this in AIX? It does not seem to be working. Usage: cp [-fhipHILPU][-d|-e] [-r|-R] [-E{force|ignore|warn}] [--] src target or: cp [-fhipHILPU] [-d|-e] [-r|-R] [-E{force|ignore|warn}] [--] src1 ... srcN directory

[Michael Krelin - hacker]

@cokedude, environment variable containing directory name. You can either set it to your directory or just substitute yourself. Like mkdir -p dir && cp file dir.

[Michael Krelin - hacker]

@cokedude, but looks like I answered that before…

[Thomas W]

For those looking to automate, the cp --parents and rarchive answers may likely be better. This is because the mkdir -p part of this answer takes just a directory, requiring you to script an extra step to parse the directory out from the full filename.

[2682562]

I don't know why this response has only 3 votes atm, rather than 500+ thank you @ulf-gjerdingen for the simple and straight forward answer.

[Pavman]

to ensure proper file ownership/mask: install -D /path/to/source /path/to/dest/dir -o some-user -m 0754

[cokedude]

Is this part creating 2 directories ` dirname /path/to/copy/file/to/is/very/deep/there `?

[Dario Seidl]

Only looking at the last argument and then doing cp "$@" is a great idea, that way other options passed to the function are passed to cp. And you're right that without trailing slash it's ambiguous, but you could treat it as a dir with trailing slash and as a file name without, i.e. in the else branch use mkdir -p "$(dirname "$last")" && cp "$@".