Linux-on-Linux KVM: Which filesystem should I use for guest and host?

Question

I'm in the process on playing around with KVM and setting up a linux on a linux-server, so I can

• reboot the (guest)server
• encrypt the rootFS of the guest without having to do initramfs-tricks to get remote reboot to work.

I'm thinking about using btrfs as filesystem, since it's a single ssd-disk and no RAID.

Following thoughts:

• If I use btrfs on the host, i have excellent data-integrity, all the checksums etc.
• So I can use something fast and simple, maybe even ext2 on the guest?

Or should I do it the other way round?

Solution 1

Running BTRFS on the host for disk images (qcow2, etc) is a really bad idea. The style of disk writes to an image file is the worst IO pattern for btrfs, the tuning KVM page does not contain a lot of tips:

http://www.linux-kvm.org/page/Tuning_KVM

But on this one they are clear:

"Don't use the linux filesystem btrfs on the host for the image files. It will result in low IO performance. The kvm guest may even freeze when high IO traffic is done on the guest."

I use ext4 for local files and a nfs store via ZFS on solaris for remote disk images. We will be switching to zfs based iscsi shares in the near future. If you don't need a nas / central file store and are going to have a relatively stable number of VMs DukeLion's suggestion is best.

We use the cluster for software testing, so we are creating 1000s of VMs each day using qcow2 overlay files. LVM does not support doing that.

Solution 2

It all depends on what you want to achieve.

I would not recommend use ext2 for root fs in any server - virtual or physical. You may get slower performance in some cases and filesystem inconsistencies is still possible, making reboots incredibly slow.

If you want simplicity and performance - I'd suggest you to put guest image on lvm logical volume and use btrfs/ext4/xfs in guest.

Comments

[Fabian Zeindl]

and what on the host?

[sartis]

raw lvm logical volume maybe with encryption

[David Corsalini]

+1 for the LVM approach

[Fabian Zeindl]

Hm, that way sounds really nice, but as far as I know you can to the volume management with btrfs as well. So maybe just btrfs instead of the lvm?

[sartis]

btrfs is a filesystem approach, having more unneeded features and functionality. LVM is dealing with plain block devices. There's nothing particularly bad with btfs, but with LVM it will be more simple and efficient.

[Fabian Zeindl]

A follow-up question: When using encryption, should i use raw-crypt-lvm-diskImage, or raw-lvm-crypt diskImage or raw-lvm-diskImage-crypt. I don't care if all vms share one password, i merely ask because of performance etc.

[n8whnp]

I have not needed to do encryption, so can't say a ton about that. We use it for a internal testing lab, no need to encrypt testing VMs.

[Dan Pritts]

Be aware that if you put your VMs on logical volumes, you lose the ability to snapshot the disks. This is huge.