Incremental (delta) backup of a encrypted data

Question

I have several files that I need to keep encrypted on my home hard disk.

I currently use a TrueCrypt file container, currently around 100MB in size. I need to backup this file daily, and I'm looking for an incremental backup solution.

I read that on encrypted volumes even a small change in a file would change the whole volume and prevent the use of an efficient incremental backup solution. Any ideas how to backup in this case in an efficient way which would allow incremental solution?

I currently use TrueCrypt as my backup solution and CrashPlan as my backup utility, but I'm willing to experiment with alternatives.

Solution 1

First, not the whole container is changed when you edit a small file. This often comes up in connection with Dropbox. Dropbox only uploades changed blocks (TrueCrypt encryption is block cipher) from the Container. Here is a thread in the dropbox forum talking about that. But I don't know if there are incremental backup solutions sophisticated enough to look for changed blocks in files.

Second, you could take a backup of the mounted container on another encrypted drive. Standard backup software should suffice.

Solution 2

You said you're using Crashplan. It does byte-level analysis of the changes within files and stores only the changes within each file, so you're already getting the benefit of an incremental backup.

If you're worried that adding a 14 byte file within a TrueCrypt container changes 22K bytes in the container, that's because very small files will produce disproportionately large changes to the filesystem. For example, the filesystem might be using 16K clusters, so the smallest file will affect a 16K block. In addition, the metadata and journaling adds more overhead. Try adding a 200K file, and the overhead will be proportionately much smaller.

Solution 3

As changes in the Truecrypt volume only occur to regions where file changes has been written to, an efficient backup tool that detects binary differences for incremental backups should do the job.

What tool in detail depends on your operating system.

Comments

[Robert]

What operating system?

[kobaltz]

Could you do an incremental backup into another container? So if you have 7 containers, Mon, Tues, Wed, Thurs, Fri, Sat, Sun then you can mount the appropriate drive each day to drive X. Use SyncBACK or similar to do an incremental backup to these containers. That way you have a rolling 7 day backup. It's just like having a Tape Back Up but encrypted and all digital.

[Mosh]

Robert, the OS is Currently windows. kobaltz, I'm using an automated backup solution (If I had to trust myself, I would backup for a week and then forget about it.

[Mosh]

Robert, as I mentioned below, adding a 14 bytes file ("hello world" of course) changes 22K bytes in the container. Adding a 1MB word file sends my diff application to 100% CPU hell, so I guess the change is a major one.

[Mosh]

Gareth, can you help me edit some other stuff? I'm writing an introduction for an article and I can use some help. ;)

[Mosh]

Thank you for your quick reply. See my comment to Robert. I use automated backup tool and I can't keep my container mounted all the time, since it defeats the whole purpose of encryption.

[Mosh]

Thanks for the reply. I tried to add a small text file ("Hello world") to an encrypted container and it changed 22K bytes. Not such a small change for a 14 bytes file.

[Robert]

Don't forget that also the MFT containing the file attributes will change, the NTFS journal has changed and Windows may also change data in the SystemVolumeInformation directory.

[Mosh]

As I mentioned in the comments, adding a 1MB Word file sends my computer into the 100% CPU hell when I'm trying to diff the containers. The problem is with the encryption, where every small change make a disproportional huge change. I was wondering if there is a better encryption method which will be easier on incremental backup.

[Mike Rowave]

Crashplan's diff algorithm probably performs a lot better than the diff utility you used. Like other similar backup services, they not only have a well-tuned algorithm, but they maintain metadata to help subsequent updates run fast. Did you notice any actual performance problems with Crashplan and small file updates? I think you're unnecessarily worried about performance.