freenas zfs read performance sucks .. why?

Question

I just set up a freenas zfs raid-z2 with 4 drives sata enterprise drives and doing some performance tests. Right now I'm pushing and pulling linux images into the storage. My notebook has a samsung 840pro ssd with 400MB/s local read write speed. Samba4 is used.

I can write with avg 105 MB/s in an continuous stream. I'm impressed, this is is really fine thinking of a 1Gb/s lan.

However reading is pretty slow and network io is jumping from a few kB to 30MB/s probably in avg. about 10MB/s. Adding a l2arc doesn't help.

Any ideas, why the reading performance is so poor? is this normal?

Solution

You may want to read this.

Essentially, in a single RAID Z group, read performance is equal to the performance of a single disk. RAID Z is great for write performance and poor for read performance. Given the slow low-end disks you're using, the numbers you've posted seem reasonable.

If you want to use RAID Z and still have reasonable read performance, you'll have to create multiple RAID Z groups (which you don't have disks for) and stripe across them.

With four disks, you may be best off creating two mirrors and striping data across them.

Comments

[Manuel]

My sata drives seem to be not to bad. just run xdd on the server. I can write up to 2000MB/s random access. Reading single files I end up with 600MB/s. Multiple reads up to 1200MB/s My problem was my local virus scanner, checking incoming traffic. Deactivating it I ended up with 80MB/s incoming over samba. Reading your link raidz2 should rise performance, however adding more devices wouldn't increase performance much. Keeping in mind that most files are >1MB and my lan is limited to 2Gb/s mirroring wouldn't increase performance a lot. If I had small junks I'd agree to mirroring.

[Manuel]

Do you have any experiences with l2arc?