Use rsync for backup without overwrite

Question

I know about --ignore-existing option and wonder if there's something slightly different.

My situation is: I have a webserver with large amount of user-uploaded pictures. It's about 2TB now. Usually these files are not modified at all. I use rsync and backup them to other server.

But what if some file becomes broken? Suppose, a half of a file disappeared. It will copied to backup and overwrites previous correct file, so backup becomes broken as well.

I may use --ignore-existing. But what if some file was still changed (that's very rarely but possible). With this option backup will not have this new file.

I would like to have all versions of a file on backup if it changes. Is it possible? That is: if rsync defines that file already exists, it moves existing old file to some ../previous-TIMESTAMP/oldFileHere and after that puts the new file on the place of an old one.

Is there a way to achieve this?

Solution 1

sorry this is the wrong answer, see my other answer

I think you want to use rsnapshot rather than rsync. It's built specifically for recovering older versions of files. It provides versioning of files at an hourly, weekly and monthly basis in the directory ~/.snapshots.

If your files are small you might even consider versioning them on a modification-basis rather than time-basis by putting them in a git repo. That's what I do with my databases, dot files and personal notes.

Solution 2

--backup --suffix=

If you don't want to use rsnapshot you can use destination files with suffixes like this:

rsync --backup --suffix=`date +'.%F_%H-%M'` ~/life_story.txt /media/myusername/backupdrive/life_story.txt

Note:

• The copies will be "smart" - if the source and the destination are the same, no new timestamped backup will be generated (which is good)
• There's an option to use a subdirectory instead of a file suffix (--backup-dir) but I haven't tried that myself.

Comments

[cxrodgers]

great answer! I modified to rsync --backup-dir=backup_`date +%F_%H-%M-%S` src dst. This syncs src to dst, and creates a dated backup directory with copies only of the files that are about to be overwritten in dst. The backup directory preserves the subdirectory structure