How do I make rsync delete files that have been deleted from the source folder?
Solution 1
To delete files in the target, add the --delete
option to your command. For example:
rsync -avh source/ dest/ --delete
NB: -avh
is for --archive --verbose --human-readable
Solution 2
The rsync command wont delete any file while you use some of its options delete
in that command. So if any file or folder added in source, it'll be synced to target without any deletion.
I suggest you to use rsync
for make backup from source files and use find ... rm
for deletion files for period of time or size of files:
rsync [options] SOURCE TARGET
find TARGET -maxdepth 1 -type f -mtime +60 -exec rm -f {} \;
The above code block, make a backup from source and then delete every files which last modified time are more than 2 month.
UPDATE
As I find that the delete
options are just for TARGET that if some files are removed from source, rsync --delete
remove them from TARGET. And the delete
option by after
and before
, as mentioned in its man page:
--delete-before receiver deletes before transfer, not during
Means that:
- rsync delete the file from TARGET which are removed from SOURCE.
- rsync start syncing files.
--delete-after receiver deletes after transfer, not during
Means that:
- rsync start syncing files.
- rsync delete the file from TARGET which are removed from SOURCE after syncing.
NOTE: The --delete-{before/after}
implement just in TARGET.
Solution 3
The command
$ rsync -avhn --delete local/ remote/
takes care to always sync back changes done locally to the remote. That means it takes care to synchronize local and remote such that
- files added in local are added to remote
- files removed from local are removed from remote
- files added in remote are removed
- files removed from remote are restored from local if they exist, else ignored
The parameters explained:
-a
archive mode-v
increase verbosity-h
output numbers in a human-readable format-n
dry run, perform a trial run with no changes made. Always use this flag initially to prevent data loss. When you are happy, remove it.
Let the results speak for themselves:
Create 2 directories
$ mkdir local/ remote/
Create files in them
$ touch local/local_only remote/remote_only local/exists_locally_and_remotely remote/exists_locally_and_remotely`
See what has been created (before rsync):
$ ls local/ remote/ local/: exists_locally_and_remotely local_only remote/: exists_locally_and_remotely remote_only
calling rsync:
$ rsync -avh --delete local/ remote/ sending incremental file list deleting remote_only local_only sent 160 bytes received 50 bytes 420.00 bytes/sec total size is 0 speedup is 0.00
See the result (after rsync):
$ ls local/ remote/ local/: exists_locally_and_remotely local_only remote/: exists_locally_and_remotely local_only
As you can see, the file remote/remote_only has been deleted, the file local/local_only has been synchronized.
Solution 4
This command will copy increment data and keep it in sync with remote server.
- It will copy only incremental data.
- It will delete if any data deleted from source.
- It will copy again from source if any data deleted at destination.
- basically this command will keep the both environment in sync.
rsync -avWe ssh --delete-before (source) root@localhost:(destination)
rsync -avW --delete-before -e ssh (source) root@localhost:(destination)
Example:
rsync -avWe ssh --delete-before /data [email protected]:/backup
rsync -avW --delete-before -e ssh /data [email protected]:/backup
Solution 5
If there are any errors during an rsync scync, rsync will not properly delete the files it should have, even if you used --delete
, --delete-after
, or --delete-before
.
This is why it is important to address rsync errors.
Most of my errors were due to using the --perms
option while syncing with a Non-Linux file system. When I replaced --perms
with --no-perms
, those errors went away and then deleting worked.
--perms
is ok when you are syncing from a Linux file system to a another Linux file system, but if you're syncing from Linux to a Non-Linux file system (like NTFS, FAT), --perms
causes errors because rsync cannot set Linux permissions on a non-linux file systems. Again, errors = no delete.
-- When syncing to a Non-Linux partition, I use --no-perms
to avoid those errors that sabotage --delete
, --delete-after
, or --delete-before
.
If you still get errors after that, and cannot figure out how to make those errors go way, you can run a command that is exclusively dedicated to deleting the out-of-sync files:
sudo rsync -r --delete --existing --ignore-existing --ignore-errors --progress /path/to/source/ /path/to/destination
The command above will delete stuff that's out of sync, but won't sync any files. So, you should sync again after this. That command is based off of this answer, except that I added the --ignore-errors
argument also, so it would delete even if there are errors.
Related videos on Youtube
user254251
Updated on September 18, 2022Comments
-
user254251 over 1 year
I recently set up a machine with Ubuntu Server to host game servers. I installed a backup plugin for each game server that creates frequent backups of game world files in a particular folder on the machine. I also established a
cron
task to automatically copy those backups to my Dropbox folder every night using rsync with the-a
option.After a few months my Dropbox account reached its storage limit and I realized I would not be able to keep so many backups, so I configured the game server backup plugin to not retain so many backups, then waited a few days to see if it would delete the older backups as it is scheduled to do on a weekly basis. The backup plugin eventually did its job and deleted the older backups, so I was expecting the
rsync
cron task to subsequently delete the older backups from my Dropbox folder to match the source folder, but it has not done so. So I have a couple of questions:By default, does
rsync
only add files to the destination folder that have been added to the source folder and change files that have been changed in the source folder but NOT delete files that were deleted from the source folder?If that is the case, what is the best way to make
rsync
do this? I want the destination folder to perfectly reflect the source folder, and that means deleting any files that have been deleted from the source folder.
I see some options listed in the manual page for
rsync
that might do the trick, but since I'm not familiar with.-
GDP2 over 3 yearsSee also these related and important considerations from other StackExchanges: unix.stackexchange.com/questions/18564/…, superuser.com/questions/156664/…
-
user254251 almost 10 yearsThanks for the reply!! Are you referring to a "delete" option for rsync? Why can't I just use the "delete" option for rsync?
-
sighrobot almost 10 years@user254251, If you use delete, the rsync command immediately delete files. But in this case there is more time for any errors.
-
user254251 almost 10 yearsIf I understand correctly, you're saying I should separate the deletion task from the rsync task to avoid errors. I have a question, though. The manual page for rsync lists some options that appear to serve the purpose of separating the deletion task by running it before or after the copying task. For example, I see two options called --delete-before and --delete-after. Would rsync with these options have the same effect as the method you described? I read the full description of each option on the manual page but there is some information in the descriptions that I don't understand.
-
sighrobot almost 10 years@user254251, Answer was updated. I don't know if there is a way to delete files from TARGET by time limitation in rsync.
-
user254251 almost 10 yearsThanks! So do you think I am safe from errors if I simply use the --delete-before option? I am not specifying a timeout so I shouldn't have to worry if the rsync deletion stage delays the transfer stage. rsync runs once per day and that is plenty of time to sync the backup files before the next sync, so I shouldn't need to time-limit the sync either.
-
Tulains Córdova over 8 yearsI think -W makes it not to copy only incremental data but to always copy whole files.
-
Jeff Tian over 8 yearsI got this error by executing this command:
rsync: Failed to exec --delete-before: No such file or directory (2)
-
Subhamoy S. about 8 yearsPerhaps using
-aqr
instead would also be an idea :) -
Tom Saleeba over 7 yearsMake sure the source is a directory. Using
source/* dest/
won't work. -
MHT over 7 yearsI found that event with
--delete
or--delete-after
it won't delete because of some errors: "IO error encountered -- skipping file deletion". To resolve this add--ignore-errors
option and debug the errors separately -
ivanleoncz almost 7 yearsWorks perfectly! Even though it is working, I would recommend to ALWAYS use
-n, --dry-run
option, before runningrsync
, specially when it comes to options like these (--delete
). It will avoid any possible headache :). -
x-yuri about 5 years
-
user254251 almost 5 yearsThanks for the tip! I am the original author of this question 5 years ago. I'm really glad you posted this, cuz I have been planning to set up syncing to an NTFS drive in the near future, and I probably would have run into the permissions error because I was planning to use the "-a" option for rsync, which syncs permissions (among other things). So I added "--no-perms" to my notes. I now plan to use rsync -a --no-perms --delete-before . I have a question: Wouldn't --delete-before avoid problems with file deletion by running the deletion before the sync? Thought that was its purpose.
-
Lonnie Best almost 5 years@user254251 - I'm not certain. I guess it would depend on how early it really "deletes before". If it some how encounters one error before the deleting begins, your screwed. My advice to you is to avoid NTFS anytime you can unless you like sharp pains in your ass :). NTFS to NTFS works fine. NTFS to Linix works fine. Avoid "Linux file system" to "Non-Linux File System"; it is too much of a pain to ensure deleting works right. You CAN accomplish this, but I decided it isn't worth the trouble versus just formatting the destination drive as EXT4.
-
Dipin over 4 years@oemb1905 bud, your sentiment is good but this is the land of man deep digging your flags
-avWeqr
are not right: - uses bothvq
, this is verbose being cancelled by quiet and not sane. -a
is shorthand for-rlptgoD
... therefore your second use ofr
is redundant. -e
is used twice also summary: your use-avWeqr
==-aW
to my reading. -
oemb1905 over 4 years@Williams yes, correct it should be
rsync -av --delete -e 'ssh -p 59333 -i /home/user/.ssh/id_rsa' /home/source/ [email protected]:/home/destination/
... theW
is optional and will vary according to people's needs ... (deleted old comment so people would not use) ... -
Michael about 4 years@MHT is it safe to always use this without debugging the errors? because in my case it's usually stuff i can't fix, like the system writing files i can't touch but don't care about
-
PeterCo over 3 yearsAre you sure that you don't need two
--
in front of--delete
? And why do you think the position of this delete option has to appear after the directories? -
Jonathan over 2 yearsIs it safe to use
--delete
with multiple sources? -
oemb1905 over 2 yearsThis poster is incorrect - why so many upvotes when there are at least two correct answers?
-
sighrobot over 2 years@oemb1905, There is no answer selected and this answer contain more detail. You can like or unlike it.
-
oemb1905 over 2 years@shgnInc yep, and I did ... but it's also important to let passer'bys know when something is incorrect.
-
oemb1905 over 2 yearsSpecifically, your entire opening paragraph is 100% not accurate.
-
B. Shea over 2 yearsYes safe. Only affects destination @Jonathan .. examine the help/man file next time. > man rsync ==== "--delete delete extraneous files from dest dirs" ---- It does NOT say 'from source dir(s)'.