Is btrfs suitable as backup filesystem?

Question

Right now I have a pretty traditional backup filesystem structure on top of ext4. Every time a backup is made, a new folder backup-DATE is created to which files are rsync'ed (with hardlinks made using rsync's --link-dest option).

Since I have read about bitrot, I would like to have a checksum for all files, transparently. Apparently ext4 cannot do that, but btrfs does offer support for data checksums (and even a built-in RAID1 mode). For a start, I would like to use btrfs as a "dumb" filesystem which supports data checksums without using its advanced features such as RAID, subvolume snapshots, send/receive, etc.

However, their wiki do not really inspire confidence in the filesystem for backups purposes:

"While many people use it reliably, there are still problems being found. You should keep and test backups of your data, and be prepared to use them." - Getting Started

"Is btrfs stable? Long answer: [..] Whatever you do, we recommend keeping good, tested, off-system (and off-site) backups." - FAQ.

My use case is to have an offline backup. For that reason the disk will see very little use (as in hours) and will be frequently plugged/unplugged (eSATA or USB 3.0). Having a reliable filesystem is a must. It must not be worse than ext4 wrt. power failures, unclean shutdowns, etc.

Is it actually recommended to use btrfs as filesystem for backup purposes? Are there other properties of btrfs which may make it less (or more) suitable?

Solution 1

I will just provide a short answer because I think this is being overthought.

If you read the main kernel wiki about the btrfs (sub-)commands, you will find that there are two commands for:

1. making a "backup": btrfs-send
2. and restore: btrfs-restore

Just in case, this means that it is not (designed to be) a backup, but to be an snapshot filesystem, with the idea of rolling back if needed, not as a backup but as "flexible".

Therefore — no, do not use it as backup — use it as a versioned filesystem where you can test things and go back. Don't rely on it.

Solution 2

I recently had problems with a btrfs file system on an up-to-date kernel 4.10.0. The file system got destroyed in a virtualbox VM because TRIM doesn't seem to be correctly implemented somewhere, and AFAIK it had something to do with index numbers of sub volumes. After switching to VMware, the file system was still corrupt and surprisingly btrfs check wasn't able to find and fix the error. Finally I switched back to ext4.

The good thing: I didn't lose data. btrfs seems to be always consistent at least for reading, but it showed me it's still far away from production readiness.

Anyway, on a server I'm still using it as backup volume because there I need the feature of cow-copying for deduplication (exactly your use case). The data is just too much in size for a traditional file system.

Update

I still have the filesystem on my server (see above), but it broke right after I posted this here. Now, I have a big read-only backup volume of 700G which would expand to ~7TB on ext4 if I try to copy away everything using tar|tar. Due to lack of time, I didn't yet check if newer kernel versions can handle it. The actual problem is a "transaction abort" which occours ~2 seconds after mounting writeable and which remounts the volume read-only. The original cause is probably a broken version of btrfs-convert, which I used years ago when I created this volume, and still a limited feature set of the current btrfs check which at least should be able to find all damages on a volume which reproducibly lead to a transaction abort or any other problems, instead of just saying that my filesystem is healthy.

Update 2

I was finally able to solve this by copying over everything to a new volume using a python script (based on an oss deduplication tool). It works with a file pool on the target filesystem based on checksums and makes cow-links from there. This semi-efficient process took 2 days, but then I had a new clean btrfs-based filesystem with all my data recovered. Could publish that code if wanted, but don't expect a perfect, easy-to-use and error-proof tool.

Comments

[StrongBad]

See unix.stackexchange.com/questions/140360/…. With BTRFS you can use subvolume snapshots instead of hardlinks.

[Lekensteyn]

@StrongBad a reliable free space indicator is definitely something where btrfs does not really excel. The question is however not about replacing hardlinks with subvolume snapshots, but rather the reliability of btrfs as filesystem for a backup disk.

[Lekensteyn]

@kirill-a I did consider ZFS, but since it is not mainlined I am hesitant to use it.

[StrongBad]

@Lekensteyn I think subvolume snapshots qualifies as "Are there other properties of btrfs which may make it less (or more) suitable as backup filesystem?"

[Lekensteyn]

@StrongBad I do not disagree that subvolumes are an alternative worth to be considered, but replacing one (hardlinks) for the other (subvolumes) is no use if data can easily be lost. I have read that btrfs does a lot in the background. Will this make cause corruption issues when the machine locks up during a transfer, when the power/data cable breaks, etc. Are there certain options/pitfalls which make it more likely to lose data compared to ext4?

[kirill-a]

@Lekensteyn ZFS is designed with a focus on data integrity. Data integrity is achieved by using a (Fletcher-based) checksum or a (SHA-256) hash throughout the file system tree. That's what you need. You can read more: wiki

[Konstantinos]

You can also use compression on btrfs which makes it more convenient!

[wurtel]

You can run rsync with --checksum now and again which will compare the files based on checksums; any bitrot should be detectable that way. However any bitrot on the source side will be backupped as well... so your source filesystem is at least as important as your backup filesystem.

[Lekensteyn]

@wurtel The problem with --checksum is that you cannot really detect bitrot. The number of metadata changes (mtime, permissions, ...) and actual data changes will by far exceed that single file so manual inspection is not an option. So data checksumming (as provided by btrfs) is really necessary here.

[JDługosz]

Use ZFS on a dedicated computer running OpenBSD; e.g. FreeNAS or NAS4Free. (I'm running a FreeNAS file/media server)

[Lekensteyn]

@JDługosz That does not help if the disk is directly attached to a Linux machine.

[JDługosz]

It is possible to run a ZFS NAS appliance in a virtual machine, though that reduces the safety; given that you are directly attached, that's true about anything. For house/small-office planning purposes, consider a hand-me-down board or tiny microATX computer to make a maximally robust NAS, if that's the desire. You said it's a backup, so it should be on a different machine than the one being backed up, right? ZFS is used with a RAID, so corrupted objects can be recovered through redundancy. Otherwise, knowing it's bad doesn't help you get it back!

[Lekensteyn]

@JDługosz While it is possible to run a VM for ZFS, it is not desirable due to the resource overhead (RAM). It is indeed for backup purposes and therefore it is a separate, external disk. A NAS would not help if the house burtnt down, so this external approach allows for offsite backups too. By the way, btrfs also has support for various RAID modes. Please do not try to sell me on ZFS, I have a friend who tries to do that ;)

[JDługosz]

For off-site backups, I use a bare disk(set) in a Pelikan case. I'm currently looking into UDF file system as being universal and non propietary, and has excellent recovery options: the filesystem knfo can be duplicated, and the file control block is self-contained and not dependant on a central structure. For RAID alternative on a single drive, par2 with recovery blocks not contiguous with the file.

[Jarek]

I would like to recommend storeBackup (open source) as a solution that seems to suite the OP's requirements. See: storebackup.org

[schaiba]

@JDługosz : since when does OpenBSD support ZFS?

[JDługosz]

@schaiba see the FreeNAS and NAS4Free images I mentioned.

[schaiba]

@JDługosz : both FreeNAS and NAS4Free are FreeBSD-based, not OpenBSD.