Which filesystem should I use to format my external backup HDD? btrfs?

Question

I have a new external USB drive I plan to use for storing backups only. (It is a WD Red 2TB NAS Hard Drive SATA III model WD20EFRX mounted in a USB enclosure.)

The drive came without a partition table. I partitioned it with gdisk (GPT).

Should I use the btrfs file system? I'm using btrfs on my internal HDD and it has been trouble-free since I first installed Kubuntu 12.04 last year.

But I'm wondering if it would be smarter to partition this backup storage drive with ext4.

FYI, it seems like the arguments that btrfs is not production ready are old. Here's some info I found.

Matthias Eckermann, senior product manager at SUSE Enterprise Linux, says that Btrfs is ready for production systems, and as of SUSE Linux Enterprise 11 SP2 Btrfs is officially supported.

Also, according to the btrfs wiki, "since February 2012 there are two vendors who support btrfs in their distributions, Oracle and SUSE."

Solution

According to the principle designer of BTRFS, BTRFS still has some issues as filesystems become full. The send/receive functionality to send/receive snapshot differences for offsite backup also is not fully functional yet, and online deduplication (useful for backing up virtual machine images) is going to happen maybe for the 3.11 kernel (not yet released). raidz support was new for 3.10 and I haven't had a chance to test it yet, I may do that this evening. All in all, BTRFS is still under very active development, and I prefer to wait until it's finished (or at least not getting major functionality improvements with each kernel release!) before actually using it in production.

The advantage of using BTRFS or ZFS on an external enclosure used for rsync-based backups is that you can make snapshots via e.g. a daily cron job, and then do time travel to retrieve old data if necessary (if, say, files have disappeared from your hard drive for unknown reasons and you need to get them back from past backups). I use a USB3 enclosure with ZFSonLinux for that purpose because I need de-duplication support for virtual machines (since the big .img file is always different from the perspective of rsync, de-duplication means that only the actual changed blocks in the .img file get changed on the backup rather than multiple copies of huge 30gb files). Hopefully when the 3.12 kernel comes out the BTRFS deduplication support will be mature enough that I can migrate away from ZFS for this application -- ZFS is cool and all, but the fact that it is not integrated with the Linux kernel causes issues (e.g., I use a Centos 6.4 virtual machine to do the backups because ZFS won't compile against the 3.10 kernel).

For backing up Linux filesystems, create a snapshot on your BTRFS (or LVM) and mount the snapshot (if LVM) and back it up via rsync. That assures that you'll have a consistent backup as of the snapshot time. Then when finished with the snapshot, delete it. (More important with LVM since snapshots have a significant performance impact there). My cron script that fires off the backup job also does the rotation of the snapshots (daily, weekly, monthly) on my ZFS backup filesystem before it starts actually backing up, so that I can time travel if needed.

As far as reliability, hoary old ext4 is probably the most reliable filesystem because of the way it statically allocates structures on disk, meaning you can always find them and at least get most of your data back if things crash badly. The downside is poor performance on edge cases of very large files (where the way inode chain blocks work make random access in those files very slow), issues with large filesystems (which are very slow to create and fsck), or the edge case of large numbers of small files (which exhaust the inode table). I personally continue to run ext4 on top of LVM on top of RAID for my root filesystems and use other filesystems either for performance reasons or for functionality reasons as required.

Comments

[Rinzwind]

You are not giving us any reason to favor any file system. Stability is not an issue it seems. So the only correct answer is: whatever you feel like. You seem to favor btrfs so use that.

[MountainX]

@Rinzwind. You say, "Stability is not an issue it seems." What do you mean?

[MountainX]

Your answer makes sense, but I already chose btrfs between the time I asked the question and you answered it. I have had good luck with btrfs on my internal HDD, but in hindsight it would have been smarter to use ext4 on my external backup drive.

[ThorSummoner]

Can we get a 2016 update?

[princebillyGK]

Can we get a 2021 update? @ThorSummoner

[ThorSummoner]

@princebillyGK up to you, check out btrfs.wiki.kernel.org/index.php/Status - some big names have bitten the bullet btrfs.wiki.kernel.org/index.php/Production_Users links from wiki.debian.org/Btrfs - in my feelings, the list of warnings and gotchas on the debian page is too long for me to choose to use btrfs as my daily driver, but I've also never had a real issue with my non-critical filesystems when using btrfs and the features (filesystem snapshots) are very nice to have