Delete files from rdiff-backup

Question

We have a system running that uses rdiff-backup for its backups. However, the backup partition is out of space, so we'll need to re-examine the situation. We do need a permanent solution, but until that time, we simply need to free up some space on the backup partition.

I have located a number of files that we can remove*, but I wonder how I should do this. The first step would of course be to remove the files from the actual file system. However, next I would need to delete it from the backups (that's where we need the space), but I have no idea how to go about that. Does anybody have an idea how to do this?

*someone - long out of office - who had games and personal photos in his home directory, multiple copies of virtual hard disks

Solution 1

The process for dealing with deletions when you've run out of space is described in the unofficial FAQ under How do I remove files from the backup set. Repeating here just for completeness.

---

This method is very dangerous and shouldn't be used, unless the files that you want to remove are causing your backup drive to run out of space and your only alternative to removing those files is removing entire increments.

IMPORTANT: Properly speaking, you should do step 4 for every increment of mirror_metadata. Rdiff-backup prior to 1.1.1 does not mind having extra mirror_metadata entries for files that are removed from the backup set this way, except in the most recent version of mirror_metadata. However, at 1.1.1 the mirror_metadata handling changed -- rdiff-backup now diffs the metadata files -- and it's unknown whether having extra entries in these diff'd files will affect restore operations. (Technical note: the mirror_metadata diffs are NOT using the same method as file diffs. They are not rdiff delta files, but plain text files (and no, they are not ordinary text diffs either). Because of this, it is safe to hand-edit them, so if you need to you can do step 4 on these diffs.)

1. Check the time -- make sure it is not close to time for a scheduled run of rdiff-backup. Also make sure rdiff-backup is not running.

2. Go into your mirror target directory and delete the file or directory there.

3. Go into rdiff-backup-data/increments on the target and delete all traces of the file/directory there. Important! If you are removing a directory, make sure you find and remove all of the *.dir files for it as well! If it's a file, make sure you find and remove all of the *.missing files (if there are any). Be careful not to remove anything that isn't related to the thing you're trying to remove, or you might lose the ability to restore other files.

4. Important step! (and WARNING this is untested with rdiff-backup 1.1.1 or later) Go back up into rdiff-backup-data and gunzip the latest mirror-metadata file. Edit the mirror_metadata file in a well-behaved text editor (WARNING! Do not use pico or nano or anything else that might automatically do line wrapping!) and remove all references to the file or directory you deleted. Be very careful not to mess up the format of the file.

Solution 2

The official github repository for rdiff-backup now has a script to do this:

https://github.com/rdiff-backup/rdiff-backup/blob/master/src/rdiff-backup-delete

The script referenced in the other answers does not seem to be available any more.

Solution 3

it appears somebody made a script doing the above steps https://www.eugenemakerspace.com/wiki/Sites/Rdiff-backup-delete

I didn't test it, but choose a 'saver' way by deleting all files which are already gone voor 2 weeks, by running

rdiff-backup --remove-older-than 2W host.net::/remote-dir

more example http://www.nongnu.org/rdiff-backup/examples.html

Comments

[Admin]

Many changes since OP and accepted answer: when installed (on Rocky Linux at least) rdiff-backup now includes the script rdiff-backup-delete mentioned by medoc below although its not mentioned in the man page.

[Admin]

@Bob If a good answer on that script is written, I'll be happy to accept it. I just didn't think that a link to a github file was a good answer...

[Admin]

I wouldn't worry about it - I commented in several places so any passers-by will see the info - I'm sure the accepted answer was right at the time, its just a very old question.

[Admin]

The rdiff-backup-delete script prints help info

[Admin]

@Bob I still would. No reason to willfully keep the answer out of date...

[Admin]

not sure you can unaccept an accepted answer but if you can then I'd suggest you accept medoc's answer - it is a link to the official github source from the rdiff-backup project and the script that addresses the OP - that's good enough in my book. Any other info will be in the help output from that script when you try to run it. A new answer would just be a duplicate of medoc's - possibly with an example but a duplicate nonetheless.

[Admin]

@Bob You can (AFAIK). But an answer that needs a link (that can stop working) to answer the question isn't a good answer. Also, "there's a help function in that script" is not good enough of an explanation on how to use it. And then there's the fact it's unclear if this file is something that is generally provided with an install or something that's just in the git repo. It can be an edit of the existing answer if you really want to prevent duplication, but I wouldn't say that a proper answer with the same idea as a thoroughly incomplete answer is a duplicate.

[kynan]

The unofficial FAQ is down, glad you reproduced the instructions.

[Halfgaar]

Forcing a regression might also be useful, because if you can get away with just deleting the last increment, it's the safer approach.

[varta]

it appears somebody made a script doing the above steps eugenemakerspace.com/wiki/Sites/Rdiff-backup-delete

[Admin]

script included in dnf install on Rocky Linux though not in man page

[Admin]

rdiff-backup-delete script installed with repo install of rdiff-backup on Rocky Linux (not in man page though)