deja-dup folder is eating disk space, problem with backup in 16.04

Question

I have 16.04 installed and it runs Backup periodically. I noticed recently that a surprising amount of disk space on my system was used, roughly 530GB out of 700GB total. Looking into the problem with the command ncdu and Disk Usage Analyzer, I saw that out of a total of 530GB used 377GB was accounted for by the folder /deja-dup! This seems way too high to me (how can the backups be so much larger than the thing they are backing up?), but I admit a thorough ignorance about how this is supposed to work. Is this abnormal?

Worryingly, I have the impression that /deja-dup is growing. My temptation is to just delete it, but I would like to have a backup program running properly. Not sure whether it is pertinent but I enter a password each time backup starts.

Any help in diagnosing this problem, or pointing me towards a solution would be greatly appreciated.

Solution 1

DejaDup has a loosely configurable scheduling policy accessible under System Settings -> Backups -> Scheduling which defaults to keeping backups forever. Depending on your use case you might find it sufficient to simply change the frequency and retention settings as shown below.

Note that older backups will be apparently be deleted earlier if DejaDup determines that the storage location is low on space. It appears that this doesn't always work as expected.

Solution 2

Its perfectly normal and have that same situation with my backup. That appears to be its way of backing up your system.

My solution is to set up what I call a fail safe evironment for my computer where I have:

1. un-interrupted power supply,

2. Little or no human activities around my pc

3. Zero need for doing any work on it, then

4. I delete the old backup and start afresh.

It creates different back ups based on the difference between new and old ones making it possible to restore to a particular point in your back up history, which as you have seen leads to massive external HDD usage.

it's features include:

1. Support for local, remote, or cloud backup location (including services like Amazon S3 and Rackspace Cloud Files)

2. Built-in encryption and compression support

3. Incremental backups (so you can restore from any particular point)

4. Scheduled backups

5. Integrates into Nautilus and other file managers

Feature 3 means your storage device gets filled over time.

Comments

[Organic Marble]

Deja-dup tries to keep at least 2 full backups around. In practice this means 2x the size of whatever you are backing up, plus incremental updates. I learned to allot at least 4x the size of what I am backing up. It makes more sense for me to back up to an external device, else deja-dup eats your drive (and it's not very safe either, if that drive fails).

[Elder Geek]

Defining "Periodically" might be helpful for those of us attempting to answer your question.

[gfgm]

Good point re "periodically". Per the settings it runs weekly. The only two options in the gui seem to be weekly or daily.

[SDsolar]

I got so fed up with this filling up my SSD boot drive that I just put a 1 TB hard drive in my main Ubuntu box and set up rsync on cron to back it up that way. Now it is like a robocopy /MIR and only keeps the latest files - automatically, 4 times a day as data is collected.. Plus, I periodically use sudo apt-get autoremove to keep the system clean. I can also run versions of rsync from the Ubuntu laptop but it prompts for the password. Same with the multitude of Raspberry Pi systems that monitor the solar plant. It all works but is very manual because of the password requirement.

[D-unit]

I notice the same thing. Every time i backup the folder size increases. I wonder why there are no options to manage this.

[George Udosen]

@D-unit, I think the trade off is we have that ability to restore to a given point in time.

[gfgm]

thanks for your response @George. I think this is a little crazy, and I suppose whether it's a feature or a bug depends on your use case. For the average user, gradually cannibalizing all your disk space seems a bug. This thread recommended the duplicity remove-all-but-n-full command, which I used to cut the size of backups in half.