Updating tar.gz daily only with changed files

Question

Possible Duplicate:
Add/update a file to an existing tar.gz archive?

I have some game files I wish to update daily and they create a tar.gz of the folder, but instead of making a full tar.gz of all the files over again each day I'm trying to find a way to have it check the directory for files changed/added/deleted and update the archive to save time and CPU instead of recreating the archive from scratch every day. How can I do this?

I was trying to use --listed-incremented with no luck so far.

Solution

You must create a level 0 backup first:

$ tar --create --verbose --listed-incremental ./game.snar --gzip \
    --file game_`date +%F`.tar.gz game/

and the next day, this command compress only files changed since the creation of the ./game.snar:

$ tar --create --verbose --listed-incremental ./game.snar --gzip \
    --file game_`date +%F`.tar.gz game/

This archive is called a level 1 backup.

When you want to restore, put all archive files into a folder and extract each in order of their creation using the --incremental option, something like this:

$ for t in game_2011-10-2*.tar.gz; \
    do tar --verbose --extract --incremental --gzip --file $t; done

Comments

[trr]

Note that zip allows for adding new files or replacing/deleting files in an existing archive, while .tar.gz doesn't, because the compression is applied to the archive as a whole. This is why you have to do it with multiple files for .tar.gz. That said, tar can do this pretty intelligently whereas with zip you probably need your own logic to determine which files have changed. Either way, the overall archive size won't shrink for deleted/replaced files so it's a good idea to re-do the entire archive semi-regularly, such as when it's taking up too much space.

[schily]

Please do not advertize software that does not work. GNU tar can do incremental backups, but this does not help you since GNU tar is unable to do incremental restores in case of non-trivial differences between two incrementals. The related bugs have been reported in 2004, 2011, 2016 and 2018 to no avail.