How to convert a raw disk image to a copy-on-write image based on another image for use with kvm and virt-manager?

Question

I have a virtual Windows machine running on kvm. Presently it has a 90GB raw disk image. I would like to clone this VM without having to keep two copies of the 90GB raw disk image around.

It seems like a good approach for doing this is to make two new qcow or qcow2 images based on the original. First I converted the raw image to a qcow2 image:

qemu-img convert -O qcow2 basewindowsxp.img basewindowsxp.qcow2

Then I tried creating a new image backed by this:

qemu-img create -F qcow2 -f qcow2 -b `pwd`/basewindowsxp.qcow2 windowsxp-1.qcow2

Then I used virt-manager to point the original VM at windowsxp-1.qcow2. However, when I try to start up the VM in this new configuration, virt-manager reports an error:

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/engine.py", line 588, in run_domain
    vm.startup()
  File "/usr/share/virt-manager/virtManager/domain.py", line 150, in startup
    self._backend.create()
  File "/usr/lib/python2.6/dist-packages/libvirt.py", line 300, in create
    if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirtError: internal error unable to start guest: qemu: could not open disk image /var/lib/libvirt/images/windowsxp-1.qcow2

The error suggests that the filename was misspecified or that the filesystem permissions are too restrictive, but neither of these is the case:

$ ls -l /var/lib/libvirt/images/windowsxp-1.qcow2 
-rwxrwxrwx 1 root root 262144 2010-05-27 08:32 /var/lib/libvirt/images/windowsxp-1.qcow2

Why won't virt-manager start this vm?

Solution

This issue was caused by the way libvirt uses apparmor.

The default behavior is to provide some protection for the host against the guest by restricting which files the virtualization process on the host is allowed to access. libvirt knows that the virtualization process (kvm in this case) needs the disk image in order to operate properly, so it creates an apparmor profile which allows access to windowsxp-1.qcow2. However, it doesn't know that windowsxp-1.qcow2 is backed by basewindowsxp.qcow2, so the apparmor profile doesn't allow access to that file.

It's unfortunate that the error reporting from kvm is so minimal. The underlying failure was almost certainly an EPERM when opening basewindowsxp.qcow, but apparently this error gets flattened out and the useful information lost.

However, reading the system logs will reveal that apparmor is doing something. For example,

May 28 13:12:28 hostname kernel: [ 5338.835932] type=1503 audit(1275066748.269:42): operation="open" pid=10601 parent=1 profile="libvirt-b1a29fd0-698c-11df-9c21-f78cb972735d" requested_mask="::w" denied_mask="::w" fsuid=0 ouid=1001 name="/var/lib/libvirt/images/basewindowsxp.img"

This shows what happens when an apparmor profile denies write access to a file for a process. Each time the vm startup failed because of this misconfiguration, this log message appeared in /var/log/messages.

There are several possible solutions to the problem.

1) Disable apparmor protection. This is controllable via the virt-manager GUI. In the overview section, security subsection, apparmor can be disabled.

2) Manually allow access to the extra file. This is controlled by modifying the apparmor files in /etc/apparmor.d/libvirt/ directory. Adding a line like:

"/var/lib/libvirt/images/basewindowsxp.img" r,

to the file matching the uuid of the vm in question will grant read access to the filename in quotes.

3) Upgrade to a newer version of apparmor/libvirt/the base platform and re-create the VMs. Apparently this misconfiguration has been noticed and is addressed automatically in new enough versions of the software in question.