ZFS with two disks: does this make sense? what is the safest configuration?

Question

I am in the process of planning a file server for private usage (small amounts of various kinds of data, but mainly photos, videos, music). I will probably use NAS4free/FreeNAS and want to give ZFS a try.

My focus is not speed, it is reliability and data safety. Nevertheless, I want to keep things 'cheap', so I want to run ZFS mirrored among two disks ('raid 1' fashion). I will buy a consumer board, so no hardware raid controller. So far, I am thinking to use the ZFS 'mirror' command for this or whatever corresponding option FreeNAS provides.

When reading about this topic, I mainly see people using three disks and then using RAIDZ-1 which renders them immune in case one disk fails. What I do not quite understand so far is what is the problem with having ZFS mirrored among two disks only? How will I notice when one disk fails? What is the failover strategy? Do I just get a message that the file system is not usable, because disk A is failing and that I should replace it now? I hope that one can achieve a real benefit when running two mirrored disks than just using one.

Given two equivalent disks, which ZFS mode would you run in?

Solution 1

For two disks, you want mirror mode. This will cause the drives to be exact images of each other. If ZFS tries to read a block of data from the drive and it fails verification, then ZFS will automatically try to repair it. If the repair fails, it'll read it from the other drive and mark the bad drive as failed. You should get a notification that a drive failed, but the filesystem will still be usable. You get some read speed improvements, but no write speed improvements. As long as you have at least one drive from a mirror set, you have all of your data.

With raidz, you need at least three drives. They function similar to RAID5 where (effectively) one drive stores recovery information, and the other drives store data. It can handle any drive failing, as long as only one drive fails at a time.

raidz2 and raidz3 are the same as raidz, except that they can handle two or three drives failing, respectively. They require more drives to operate and reduce effective capacity, though.

Solution 2

ZFS mirror is the way to go if you have two similar disks and look for reliability and data safety.

RAIDZ is a cheaper solution as a lesser percentage of storage is dedicated to data security but mirroring is faster.

About failure detection and handling, you need to monitor your pool (zpool status) to know if errors are present.

You might regularly scrub your pool to check its health (zpool scrub poolname)

ZFS will automatically self-heal those it can but should a whole disk fails, you would need to add a new disk to the pool to replace the broken one. The pool will then automatically resilver the new disk (i.e. copy the data from the healthy mirror side).

I'm not familiar with NAS4Free so only suggest the CLI commands to use. The GUI should provides a front end for most or all of them.

Note that NAS4Free Raid levels (0/1/5 and combination) are different than ZFS stripes/mirror/raidz and combination. I would suggest to only use the latter, i.e. there is no point to create a ZFS pool on a volume composed of a software raid 1 (mirror). You would lose many features ZFS provides.

Comments

[Jan-Philip Gehrcke]

Update: got two WD30EFRX disks on the day after I asked this question, 7.4 years ago. smartctl -a /dev/adaX shows that as of today both have ~50000 Power_On_Hours: 5.7 years, no failures (yet :D). twitter.com/gehrcke/status/1344286972597198850

[jlliagre]

Hopefully it does. I was only warning you not to use traditional mirroring instead: wiki.nas4free.org/…

[user1984103]

@Jan-PhilipGehrcke Yes. Assuming the drives are all the same size, a mirror array has the same usable size as 1 drive (no matter how many disks you use), whereas RAIDZ has N-1 drives of usable space. (Three drives gives you two drives of usable space, five drives gives you 4 drives of usable space, etc.).

[Poulsbo]

"With raidz, you need at least three drives." -- Not true for raidz (single parity). Per the ZFS docs, "You need at least two disks for a single-parity RAID-Z configuration and at least three disks for a double-parity RAID-Z configuration."