Huge storage penalty on FreeNAS with ZFS, RAIDZ, and different size disks

zfs truenas

6,905

A RAID-Z group within a ZFS pool will always lock the size to the smallest disk within the pool. So, currently, you have what is essentially a RAID-Z of 3x 40GB drives. One disk worth is dedicated to parity bits, so you've got 2x 40GB, which is 76.29 GiB.

The way that you can work around this limitation is by not using RAID-Z at all. ZFS also lets you independently set that data should be stored in at least X locations throughout the pool, preferring different disks for the extra copies when possible. Add each disk to the pool separately, then run zfs set copies=2 poolname; this will direct ZFS to store all data in at least two places.

6,905

Bryan Mills

I write stuff Languages: HTML, CSS, JS, PHP, Java, Bash, Windows Batch, some ColdFusion, and some NSIS Learning: C#, C++ Databases: MySQL, some MSSQL Certifications: CompTIA A+, Network+, and Security+ Certifications Projects: PircBotX - A simple, easy to use, Java IRC Bot Framework forked from the popular PircBot framework, bringing many new up-to-date features and bug fixes in an official alternative distribution. Quackbot - A fully abstracted, fully pluggable, and fully extendable IRC bot based off of PircBotX, aiming for simplicity and power. It abstracts plugins to the point where they can be written in javascript or any other language that a PluginLoader exists for.

Updated on September 18, 2022

Comments

Bryan Mills almost 2 years
I'm setting up a FreeNAS server with the following 3 storage drives
- 2 80 GB drives
- 1 40 GB Drive
I choose ZFS with RAIDZ since it seemed like the only free RAID that supported multiple disks. However when I setup FreeNAS I was surprised that I only had 76 GB of usable space. Where did the other 124 GB go?

What am I missing here? Isn't ZFS supposed to allow you to use multi-size disks and still have redundancy? How can I configure FreeNAS (I would prefer to stay away from the command line, but will go there if its absolutely necessary) to give me the most storage with protection?
Bryan Mills about 13 years

I recreated the volume (selected stripped), then ran your command, but the size of the volume doesn't change. Should it since its duplicating the data?
MShoubaki about 13 years

You probably don't want striped, as Shane says "add each disk to the pool separately". The volume should then be 40+(80*2)GB in size. When you write, it will write 2 copies, meaning that you don't see the space available change, but your used space grows twice as fast (write 1GB, the allocated space shows 2GB). That's the unfortunate thing, a RAIDZ would have only increased consumption by 50% in a 3 drive array.
Bryan Mills about 13 years

@Sean Ah, okay. Added it, ran the command, and saw the effects with dd. For a temporary solution this works just fine. Long term though I'm probably going to scrounge around for more hds and/or a better system
Dan Pritts almost 12 years

You could also have partitioned the 80GB drives, and made a RAIDZ out of 40GB partitions; then make a mirror out of the remaining space on the two 80GB drives. better HDs clearly a better solution.