ZFS raidz2 performance issues read/write speeds on Freenas

Question

I'm having some performance issues on my home backup FreeNas box using a raidz2 configuration and I am wondering what might be causing it (not enough processing power, eco drives?).

It's running:

• Freenas 8.3.1
• ASUS E35M1-I (Hudson M1)
• 8GB ram (DDR3 1066)
• 6 x 2TB drives (mix WD Cavier green 5900s, Seagate Green 5900s)
• raidz2

Write tests:

dd if=/dev/zero of=/mnt/beast/test/ddfile bs=2048k count=10000

20971520000 bytes transferred in 146.161146 secs (143482181 bytes/sec)
20971520000 bytes transferred in 166.954361 secs (125612292 bytes/sec)

Read tests:

dd of=/dev/null if=/mnt/beast/test/ddfile bs=2048k count=10000

20971520000 bytes transferred in 85.232895 secs (246049603 bytes/sec)
20971520000 bytes transferred in 73.342644 secs (285938968 bytes/sec)

It seems 2 to 3x slower than other benchmarks I've seen posted for Raidz2. Any ideas why or how I might improve performance?

Solution

CPU should not be a bottleneck unless you have set SHA256 checksumming on your filesystems (or gzip compression).

Start by running "top -SH" and "gstat". First one will show you where CPU spends time and gstat will show you what your disks are doing. This should let you narrow down what bottlenecks your performance -- CPU or disks.

With RAIDZ your performance is limited by the slowest drive. All it takes is one flaky drive with few bad sectors to slow things down to a crawl, so check gstat output for drives that have unusually high response time.

In general, I would recommend upgrading to FreeNAS 9 which is based on FreeBSD-9 as there are a lot of ZFS improvements that didn't make it into FreeBSD-8 (and thus into FreeNAS)

Comments

[Doktoro Reichard]

Can you develop more about your answer? Just presenting values doesn't help, as you aren't actually answering the question.

[Sacha K]

There's no way you're really writing at 3 GB/s with your setup. You must have compression enabled. 20 GB of zeros compresses to pretty much nothing so it looks fast.

[Sacha K]

Your first result could be real, for the second one there is some cache or compression involved.

[Sacha K]

Your first result could be real. For the second one there is cache or compression involved.