XenServer Converting HVM to Paravirtualised

Question

Recently I have been tasked with the daunting process of converting a setup of HVM enabled VMs (running on Citrix XenServer 5.6.0) into PV (paravirtualised) containers.

The constraints of the project was that:

1. The operating system must be functionally identical after the migration.
2. minimal modification to the operating system (with exception of kernel / drive mapping)

I also was allowed to change the bootloader(ie, grub) in what ever way I see fit.

However, I have attempted this, I will firstly like to show you my steps I took.

This at the moment is CentOS5.5 specific:

Steps:

1. yum install kernel-xen

   This installed: 2.6.18-194.32.1.el5xen

2. edited: /boot/grub/menu.lst changed my specs to match:

   title CentOS (2.6.18-194.32.1.el5xen)    
   root (hd0,0)
   kernel /vmlinuz-2.6.18-194.32.1.el5xen ro root=/dev/VolGroup00/LogVol00 console=xvc0
   initrd /initrd-2.6.18-194.32.1.el5xen.img
   

   Then I changed my xenserver parameters to match:

   xe vm-param-set uuid=[vm uuid] PV-bootloader-args="--kernel /vmlinuz-2.6.18-194.32.1.el5xen --ramdisk /initrd-2.6.18-194.32.1.el5xen.img"
   xe vm-param-set uuid=[vm uuid] HVM-boot-policy=""
   xe vm-param-set uuid=[vm uuid] PV-bootloader=pygrub 
   xe vbd-param-set uuid==[Virtual Block Device/VBD uuid] bootable=true
   

Some things to note, I am running a VolGroup LVM ;)

Anyways, after all these steps (which aren't much!) I boot the VM and it boots initial kernel just fine, however I am presented with this error:

Boot Screen:

device-mapper: dm-raid45: initialized v0.2594l
Waiting for driver initialization.
Scanning and configuring dmraid supported devices
Scanning logical volumes
  Reading all physical volumes.  This may take a while...
Activating logical volumes
  Volume group "VolGroup00" not found
Creating root device.
Mounting root filesystem.
mount: could not find filesystem '/dev/root'
Setting up other filesystems.
Setting up new root fs
setuproot: moving /dev failed: No such file or directory
no fstab.sys, mounting internal defaults
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
switchroot: mount failed: No such file or directory

Now my hints are that it cannot detect / because of the fact that when you change from HVM mode to PV it does something (not that obvious)

When you make a SR (storage) on a HVM, you get it mounted to the guest os as /dev/hda.

However in PV mode, this presents itself as /dev/xvda...

Could this be the answer? and if so, how the heck to I implement it??

Update:
So I have gotten a bit further in my quest, as it now detects the LVM's...

To do this, I required to recompile the xen-kernel initrd image.

Command: mkinitrd -v --builtin=xen_vbd --preload=xenblk initrd-2.6.18-194.32.1.el5xen.img 2.6.18-194.32.1.el5xen

Now when I boot I get this:

Boot Screen:

Loading dm-raid45.ko module
device-mapper: dm-raid45: initialized v0.2594l
Scanning and configuring dmraid supported devices
Scanning logical volumes
  Reading all physical volumes.  This may take a while...
  Found volume group "VolGroup00" using metadata type lvm2
Activating logical volumes
  3 logical volume(s) in volume group "VolGroup00" now active
Creating root device.
Mounting root filesystem.
mount: error mounting /dev/root on /sysroot as ext3: Device or resource busy
Setting up other filesystems.
Setting up new root fs
setuproot: moving /dev failed: No such file or directory
no fstab.sys, mounting internal defaults
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
switchroot: mount failed: No such file or directory
Kernel panic - not syncing: Attempted to kill init!

Solution

Try: xe vm-param-set uuid=[vm uuid] PV-args="root=/dev/VolGroup00/LogVol00"

Comments

[user1111]

Update: so it appears that that after recompiling the initrd, I have managed to now find the logvolgroup.

[user1111]

The command was: mkinitrd -v --builtin=xen_vbd --preload=xenblk initrd-2.6.18-194.32.1.el5xen.img 2.6.18-194.32.1.el5xen

[user1111]

Now I am stuck with: Found volume group "VolGroup00" using metadata type lvm2 Activating logical volumes 3 logical volume(s) in volume group "VolGroup00" now active Creating root device. Mounting root filesystem. mount: error mounting /dev/root on /sysroot as ext3: Device or resource busy Setting up other filesystems. Setting up new root fs setuproot: moving /dev failed: No such file or directory no fstab.sys, mounting internal defaults

[pepoluan]

Can you post the contents of fstab? PS: prefix the outputs with 4 spaces, and remove the empty lines in-between. The code will look nicer.

[user1111]

Hi Everyone, I have worked out all this and got it working some time ago... I will be writing an extensive blog on this and exact steps on how to do it for a whole set of flavors. On that thought, paravirtualization is great, especially for density requirements, we run close to ten thousand VM's so it's important for us to make it as dense as possible without impeding performance :) I do like vmware unfortunately it's not my call on what they implement... I do however very much love Xen ;)

[user1111]

Hi Everyone, I have worked out all this and got it working some time ago... I will be writing an extensive blog on this and exact steps on how to do it for a whole set of flavors. On that thought, paravirtualization is great, especially for density requirements, we run close to ten thousand VM's so it's important for us to make it as dense as possible without impeding performance :) I do like vmware unfortunately it's not my call on what they implement... I do however very much love Xen ;)

[hookenz]

Cool, where is the blog? it's only useful to comment like this if you've got the goods.