ZFS on KVM virtualised server. Should I run ZFS on the host or the guest?

Question

I'm planning to run ZFS on Linux (latest Ubuntu stable ppa). The hardware is a xeon E3-1270 with 24 GB of RAM, 6 SAS ports on the motherboard, and a supermicro HBA with 8 SAS ports. I plan to have 4 x 3TB on 5400rpm SATA drives and 4 x 2TB on 7200rpm SATA drives in RAIDZ1 for a start. This will run 2 file servers in virtual machines. The file servers have different performance needs, so one will use the storage from the 5400rpm drives, and the other will use the 7200 rpm drives. The OS will run on a separate SSD.

The question is: how should I configure this (for performance, reliability, etc)?

a. Should I run ZFS on the VM host, and export the logical drive to the VM guest? Or should I export the raw disks to the VM guest and run ZFS on the guest?

b. For the file server with higher performance requirements, should I attach the 4 x 7200rpm drives to the HBA and attach the entire HBA to the VM guest?

Solution 1

If you want performance forget RAIDZ1 (that seems to be a common knowledge among people using ZFS for a while). Instead use mirrors as your disk are not fast being high capacity and low rpm.

I would keep ZFS closer to hardware. There is no much penalty in giving virtual disks to VMs I would not run ZFS there unless there is a compelling reason (no raid in virtual servers).

---

As others said you should consider if you really need virtualization. My answer does not deal with that part.

Solution 2

I don't see any need to virtualize anything in this instance. A file server doesn't need that level of separation... But of the options given, running ZFS natively on the host makes more sense. Don't bother with passthrough to your guests in this setup...

Comments

[ewwhite]

Why bother separating or virtualizing the file servers?

[Michael Hampton]

Because we can? Though I'm not seeing a need to virtualize here.

[Anon]

Besides having different performance needs, the file servers also serve different groups of users. ie, one will be exposed to the internet via ftp,http,rsync,cvsup; the other is for the local network and mainly runs cifs. They're virtualized to enforce a clearer distinction in their trust boundaries.

[Michael Hampton]

Virtualization for that reason is extreme overkill...

[Anon]

Thanks for the tip. Is the recommendation to reduce complexity (since you say not to bother with passthrough)? Or is running natively nearly equivalent or better for performance/reliability?

[ewwhite]

Reduce complexity and simplify management. If performance were the the most important consideration, you wouldn't be using virtual machines for this purpose.

[Marianne]

I completely agree. If you need something that looks and smells like separate machines, look into LXC or chroot. Or FreeBSD jails/Solaris zones if you like learning new things.