rsync only files created or modified after a date and time

Question

The goal is to rsync (or using any other tools that give the desired result) only files that are created or modified after a date. It should work with filenames containing spaces (and/or other characters that needs to escaped).

I succeeded using this command on OS X 10.6.5: find . -newermt "2010-11-23 16:52:50" -type f -exec rsync -vuptn '{}' ../rsync_test/ ';' I don't think -newermt is working on ohter OS than OS X though.

The problem with this command is that it runs rsync once per file found and that doesn't seems to be the proper way to do it. I tried to change the last ; with a + so it waits for the entire file list, but it brakes (I did try "+",\+ and other way to escape).

Background: I am trying to backup my files at a remote location. The files being a lot photos in raw and other format (photographs work. about 30GB per day), rsync'ing though the network would take unusable time due to small upload bandwidth (ADSL sucks). So the idea is to have a lot of disk space at a remote location and transporting new/modified files on a small external hard drive once a week. rsync can then easily be used to add the new/modified files at the remote location. For that I need to sync the new/modified files on an external hard drive during the week so I always have 2 copies of every files either at a remote location for files older than a week or on an external hard drive for new/modified files since the last transport.

I was going to make this command run every 10 minutes (cron) with the date and time from the last time it was run.

Can you fix my command? Or do you have an even better solution?

Solution

I'm a little bit confused about what you're trying to do. Rsync, all by itself, will only transfer new/modified files. So if you transport data on a hard drive once/week, you can run rsync periodically and it will no more bandwidth than (and possibly less than) the solution you're proposing using find.

All this work with find seems like you're just duplicating features that are already there.

If you really want to use find, you don't need xargs or xargs-like features; just do this:

find ... -print | rsync ... --files-from=-

Where ... is "whatever other options you feel are appropriate". This causes rsync to read a list of files from stdin.

Comments

[Admin]

Recent versions of GNU find support -newermt.

[Yanik]

The idea is to empty the "transport" hard drive every time it's data has been synchronized with the remote storage hard drives. So rsync (as I understand) will copy the files already on the remote hard drive since it only has access to the transport drive.

[user2751502]

Ah, I see. So the find | rsync solution here will do what you want. You're not getting much benefit from rsync here (since you're always copying to an empty destination rsync doesn't get to do any of its fancy bandwidth-reducing work); you may also want to look into something like the GNU version of tar, which has good support for incremental backups (which is what you're doing). See, for example, gnu.org/software/tar/manual/html_section/Incremental-Dumps.h‌​tml.

[Yanik]

Using find . -newermt "2010-11-23 16:52:50" -print0 | rsync -vuptn -0 --files-from=- ../rsync_test/ gives me a syntax or usage error. Same as if I use -print

[Yanik]

I already tried with xargs, but can't seem to find out how I should use it. Can you give me an example?

[Yanik]

+ was taken from the man on my machine :) but didn't work.

[Ignacio Vazquez-Abrams]

find ... -print0 | xargs -0 -I{} ... rsync {} <destination>

[Yanik]

I found out. You still need to specify a source for rsync. The working command is then: find . -newermt "2010-11-23 16:52:50" -print0 | rsync -vuptn -0 --files-from=- . ../rsync_test/