How to cp remaining files after failed attempt

Question

I'm trying to copy a large directory from one drive to another. I mistakenly logged out before it was finished so only about 80% of the files copied over. Is there away to copy the remaining files without starting from scratch?

Solution 1

I would try,

rsync -a /from/file /dest/file

you can use other options like --append, -P (--partial --progress). See man rsync for more info.

Or if you are using cp then use cp -u.

from man cp:

-u, --update
     copy only when the SOURCE file is newer than the destination file or when the destination file is missing.

Solution 2

For partial file transfers, cp might not be what you need.

I suggest using rsync instead. Even when transfer fails, you can simply re-run the command at a later time, and it'll copy only the missing files.

Example:

$ rsync -aPEmivvz from/ to/

(will copy from/ into to/ directory)

Since rsync does syncing and checking after each transfer, ensuring files have been copied correctly is trivial.

The swich set -aPEmivvz is my standard go-to selection, whenever copying files over networks or external drives, including plug-in devices, like SD-Cards, etc.

These are the switches I use almost always:

-a: "archive", includes -rlptgo (recursive, symlinks as symlinks, permissions, times, group, owner)

-P: "partial progress" shows nice progress bar for each file

-E: "executability" preserve whether executable or not

-m: "noempty" prunes empty dirs

-i: "summary" print change summary for all updates

-vv: "more verbose"

-z: "compression"

Solution 3

What you probably want rather than cp in this case is rsync.

Just recursively copying one directory structure to a new location:

rsync -av /path/to/source /path/to/destination 

Solution 4

You can also copy all files again, ask for confirmation for each overwrite, then automatically answer no to the question:

yes n | cp -i /source /dest

(you have probably used more options, like -r, so just add them like cp -pri /source /dest ...)

Solution 5

You should not use cp for this. As others have stated it's not the right tool for the job. The correct tool is going to depend on your goals.

If you just want a straight copy:

rsync /source /destination with some level of flags. See the other answers for some really good examples.

If you want to have bi-directional updates

unison is the way to go. Rsync doesn't do very well at conflict resolution with bidirectional syncs. Specially deletes. So you you want bi-directional copies fire up unison

If you want backups and the files are text based, then you can try using git. Git can be run locally, there's no need for a "server". In fact, git servers are only file sharing methods. You could totally have a git repo on the backup drive and "push" to it. Again this only really works if the files are text based and your trying to get a one way backup.

Again if your going for a backup (and not just one time) then look at deja-dupe, BackIntime, or similar options. They take snapshots of the data at a point in time. They all run rsync under the hood.

View more solutions

Comments

[barro32]

I'll have to give it to Rahul cus he just beat you to it, but thanks and everyone gets an upvote!

[ilkkachu]

If the first copy ended abruptly, one of the files may be copied only partially, and cp -u might not care, since it only checks timestamps, not the size... So you'd need to touch the source files first if you have to use it. (Or just use rsync.)

[Rahul]

@ilkkachu that's right. I will update it soon.

[Trevor]

Does cp ever leave a single file only partially transfered? If it does, then -u seems like it would leave you with a directory that was only partially working.