btrfs vs. ZFS on a Ubuntu file server

Question

I have a server which is running Ubuntu (Server Edition), and its main job is serving files through FTP (vsftpd) and AFP (netatalk). My /srv directory is starting to fill up, and I need to add more hard drives. Because its technology is outdated for what I need, RAID is out of consideration.

For what it's worth, right now, should I go for the more mature ZFS, using the kernel module or the FUSE extension, or give the newer but more Linux-like btrfs a spin? What are the pros and cons of each, in this situation?

TIA.

Solution 1

The big, big negative for both options is that neither is ready for prime-time on the Linux kernel. I would strongly recommend against using either of them at this time, their maturity just isn't there yet.

That said, if you have to use one or the other I lean towards btrfs. It fits better in the OS, and should perform better than ZFS due to it being in the kernel rather than userspace.

Solution 2

If RAID is "outdated" for you I'm assuming you want something with end to end checksumming (i.e. bit-rot protection). As was mentioned BTRFS is not ready for any kind of non-development environment, it's still under development.

ZFS under fuse does work to a certain extent, but performance (speed) is poor, and reliability long term is a big unknown.

If this is for personal use than you can use Solaris 11 Express, but it's it's commercial (well, not "development" or "evaluation") you will need to purchase a license to use it. It does have the newest ZFS pool version (31 I believe). Based on SVN 151a build.

OpenIndiana is the most up to date freely available Solaris based build, so probably the most "robust" solution out there for free ZFS hosting. It is using ZFS pool version 28. Based on SVN 147.

Nexenta - two editions, a core edition (CLI only) and a community edition (free up to 18TB of storage) (well, and then paid editions). This is based on SVN 134 with ZFS pool version 22, but has lots of fixes from up to 147 backported.

FreeBSD has the most mature non solaris ZFS implementation (and the only other one I would consider using). The 8.2 Stable release is running ZFS pool verison 15. FreeBSD 9-Current has support up to ZFS pool version 28.

A note on the zpool versions, you can migrate upwards, but not backwards - if you have a ZPOOL on 10 disks at version 15, you can import it into any OS that supports version 15 or higher, but a version 28 pool must be imported into an OS that supports version 28 or higher.

Between ZFS version 15 and 28 probably the notable features are triple parity (Raid-Z3) and deduplication. If you aren't using those then it really doesn't matter (dedup is nice but very memory hungry - 1GB per TB of deduped data is a good rule of thumb, and if you don't have the RAM then performance grinds to a halt. So only use it if you really need it.

Also, if you go with a solaris based distro you should check out napp-it as a web interface to support configuring and managing ZFS - NAPP-it Website.

If you go with a BSD based setup you might want to check out the ZFS-Guru distribution - ZFS Guru FreeBSD distribution

You could probably also setup a VM and run one of the distro's in a virtual machine, if you do that I would suggest something like ESXi which suports Vt-d directed-io passthrough. Whatever your SATA/SAS interface is, pass complete control of it through to the ZFS host os for best performance. If you need extra ports a LSI1068E based card is the best bet (just in HBA mode). They are available for ~70USD.

Solution 3

If you want to use ZFS, your best options would be to use OpenIndiana (http://www.openindiana.org) or Solaris 11 Express from Oracle. Your second best option for ZFS would be the latest 8.2 release of FreeBSD (http://www.freebsd.org).

BTRFS is still under heavy development, and I wouldn't trust it with any data that I cared about, and running ZFS under Linux is going to be a road fraught with potential issues. If it is really the filesystem you are caring about, then you should pick the OS that best supports the filesystem. The same would go for using NTFS. While it's possible to read/write to NTFS under Linux, if that was the filesystem I wanted to use, I would only do it under a Windows OS.

Comments

[sciurus]

What do you mean by "its technology is outdated for what I need"?

[Ryan Bair]

Some crazy kids have ported ZFS to the Linux kernel: zfsonlinux.org

[Deb]

Nifty, I'd still trust btrfs over that simply because more eyeballs have been on btrfs.

[osij2is]

+1 Great round up explanation of all the different distros that support ZFS

[osij2is]

About the VM idea, is ESXi the only way to go? Any thoughts on XenServer or Hyper-V? I currently have an LSI1068E HBA for a ZFS implementation, but I've debated on virtualizing ZFS (nexenta) or not (solaris express or openindiana).

[basic6]

ZFSonLinux is actually very stable. The same can not (yet) be said about btrfs, which still crashes from now and then (though usually not corrupting data).