How do I merge one directory into another using Bash?

Question

I'm looking for shell script that merge files from one directory into another.

Sample:

html/
  a/
    b.html
  index.html

html_new/
  a/
    b2.html
    b.html

Usage:

./mergedirs.sh html html_new

Result:

html/
  a/
    b.html
    b2.html
  index.html

html/a/b.html was replaced by html_new/a/b.html
html/a/b2.html was copied from html_new/a/b2.html
html/index.html was kept untouched

Solution 1

You probably just want cp -R $1/* $2/ — that's a recursive copy.

(If there might be hidden files (those whose names begin with a dot), you should prefix that command with shopt -s dotglob; to be sure they get matched.)

Solution 2

cp -RT source/ destination/

All files and directories in source will end up in destination. For example, source/file1 will be copied to destination/file1.

The -T flag stops source/file1 from being copied to destination/source/file1 instead. (Unfortunately, cp on macOS does not support the -T flag.)

Solution 3

Take a look at rsync

rsync --recursive html/ html_new/

Rsync got alot of flags to set so look at rsync manpage for details

Solution 4

Just use rsync - it's a great tool for local file copy and merging in addition to remote copying.

rsync -av /path/to/source_folder/ /path/to/destination_folder/

Note that the trailing slash on the source folder is necessary to copy only the contents of source_folder to the destination. If you leave it off, it will copy the source_folder and it's contents, which is probably not what you are looking for since you want to merge folders.

Solution 5

Even though this question and its accepted answer are ancient, I am adding my answer because the presently existing ones using cp either don't handle some edge-cases or require working interactively. Often edge-cases/scriptability/portability/multiple-sources don't matter though, in which case simplicity wins, and it is better to use cp directly with less flags (as in other answers) to reduce cognitive load - but for those other times (or for a robustly reusable function) this invocation/function is useful, and incidentally isn't bash-specific (I realise this question was about bash though, so that's just a bonus in this case). Some flags can be abbreviated (e.g. with -a), but I have included all explicitly in long-form (except for -R, see below) for the sake of explanation. Obviously just remove any flags if there is some feature you specifically don't want (or you are on a non-posix OS, or your version of cp doesn't process that flag - I tested this on GNU coreutils 8.25's cp):

mergedirs() {
    _retval=0
    _dest="$1"
    shift
    yes | \
        for _src do
            cp -R --no-dereference --preserve=all --force --one-file-system \
                  --no-target-directory "${_src}/" "$_dest" || { _retval=1; break; }
        done 2>/dev/null
    return $_retval
}

mergedirs destination source-1 [source-2 source-3 ...]

Explanation:

• -R: has subtly different semantics from -r/--recursive on some systems (particularly with respect to special files in source dirs) as explained in this answer
• --no-dereference: never follow symbolic links in SOURCE
• --preserve=all: preserve the specified attributes (default: mode,ownership,timestamps), if possible additional attributes: context, links, xattr, all
• --force: if an existing destination file cannot be opened, remove it and try again
• --one-file-system: stay on this file system
• --no-target-directory: treat DEST as a normal file (explained in in this answer, namely: If you do a recursive copy and the source is a directory, then cp -T copies the content of the source into the destination, rather than copying the source itself.)
• [piped input from yes]: even with --force, in this particular recursive mode cp still asks before clobbering each file, so we achieve non-interactiveness by piping output from yes to it
• [piped output to /dev/null]: this is to silence the messy string of questions along the lines of cp: overwrite 'xx'?
• [return-val & early exit]: this ensures the loop exits as soon as there is a failed copy, and returns 1 if there was an error

BTW:

• A funky new flag which I also use with this on my system is --reflink=auto for doing so-called "light copies" (copy-on-write, with the same speed benefits as hard-linking, and the same size benefits until and in inverse proportion to how much the files diverge in the future). This flag is accepted in recent GNU cp, and does more than a no-op with compatible filesystems on recent Linux kernels. YMWV-a-lot on other systems.

View more solutions

Comments

[Tomasz Gandor]

When "merging" multiple directories etc. consider also -n (no clobber) or -f (force) option to either keep existing files or overwrite them.

[nwgat]

works nicely, it copies and merges /folder/data to folder2/data

[fnkr]

I pefer this solution because it works "as is" with .dotfiles.

[vim]

there is small typo, extra hyphen, so it should be rsync -av /path/to/source_folder/ /path/to/destination_folder/

[Ron Wertlen]

On Mac OS X / darwin you'll want to use cp -r source/ destination -- it's tricky you have to use the / exactly as indicated.

[Hadrien TOMA]

Proud to be the one upvoting this answer and so making it now equals to the current accepted one 😄.

[brianegge]

Please note is will overwrite any files in destination which have the same name as the source