Clone kvm/qemu VM to a different server
Solution 1
Okay so the way I was doing it actually did work just fine. The problem is just that I didn't have enough resources to run that VM. So just to answer my own question... here are the details to how I did the VM duplication across different servers without shared disk.
Because you don't have a shared disk, you can't do a typical 'clone' and then 'migration'. Instead you do a typical clone
Here's the command to do the cloning (/local/vm/ is the path to your VM images, usually /var/something/):
virt-clone --original=vm-to-clone --name=cloned-vm -f /local/vm/cloned-vm.img --mac=xx:xx:xx:xx:xx:xx
Now copy that img file from one server to the other... my servers can't talk directly to eachother, so I use this little SSH redirect to do the trick:
ssh -n server1 '(cd /local/vm/; cat cloned-vm.img)' | ssh server2 '(cd /local/vm/; cat > cloned-vm.img)'
Then copy the config for that VM over:
ssh -n server1 '(cd /etc/libvirt/qemu/; cat cloned-vm.xml)' | ssh server2 '(cd /etc/libvirt/qemu/; cat > cloned-vm.xml)'
Update the config with any new changes. In my case (and this was what was causing my problem), I needed to lower the "memory" and "currentMemory" attributes.
Add the new VM to libvirt:
virsh define /etc/libvirt/qemu/cloned-vm.xml
Run it:
virsh create /etc/libvirt/qemu/cloned-vm.xml
Solution 2
And here is what I do where $VM is the VM name, and DEST is the destination hypervisor hostname. It does this on running VMs using dd and snapshot LVM disks (It is assuming that the LVM Group is called HypGroup00).
I just threw this together, so it isn't necessarily the prettiest, but does the job, I'm using it to migrate some VMs from a CentOS 5.9 hypervisor to CentOS 6 with minimal downtime.
This is for CentOS 5.9, tested on going to CentOS 5 and 6 as a destination.
VM=webserver
DEST=hyp5
Figure out which disks to copy, since most of our VM disks are referenced via the /dev/mapper/ path rather than the /dev/volgroup/volname path. So we need to translate between the two.
DISKS=`cat /etc/libvirt/qemu/$VM.xml | grep HypGroup00 | sed "s|.*/\(HypGroup00-.*\)'/>|\1|"`
LVS=""
for disk in $DISKS; do
echo VM Disk $disk
LV=`lvdisplay /dev/mapper/$disk | grep "LV Name" | awk '{print $3}'`
LVS="$LVS $LV"
done
Copy over the VM definition and register it
virsh dumpxml $VM > /tmp/$VM.xml
scp /tmp/$VM.xml $DEST:/tmp/
ssh $DEST virsh undefine $VM > /dev/null 2>&1
ssh $DEST virsh define /tmp/$VM.xml
Now create the LV on the remote server
for lv in $LVS; do
ssh $DEST lvremove --force $lv
SIZE=`lvdisplay $lv | grep "LV Size" | awk '{print $3}'`
SHORTNAME=`basename $lv`
ssh $DEST lvcreate -L"$SIZE"G -n$SHORTNAME HypGroup00
done
Now create the snapshot LVs and start copying over the data
virsh suspend $VM
for lv in $LVS; do
lvcreate -L10G -s -n $lv-snapshot $lv
done
virsh resume $VM
Copy the disks across
for lv in $LVS; do
echo Copying LV $lv, this will take a while...
time dd bs=1M if=$lv-snapshot | gzip --fast | ssh $DEST "gzip -d | dd of=$lv"
done
A more complex version of the above that shows progress if required from dd, no good for multiple copies because of /tmp/pid, but you can change to to include $$ if you wish.
(dd bs=1M if=$lv-snapshot & echo $! >&3 ) 3>/tmp/pid 2> >(grep 'copied' 1>&2) | ssh $DEST "dd bs=1M of=$lv" &
# Need this sleep to give the above time to run
sleep 1
PID=$(</tmp/pid)
while kill -0 $PID; do
kill -USR1 $PID
sleep 5
done
Clean up
for lv in $LVS; do
lvremove --force $lv-snapshot
done
Related videos on Youtube
Kish
I am a computer nerd currently working at NYU. Social: @UnixTools @andrewcase google+
Updated on September 18, 2022Comments
-
Kish almost 2 years
What's the easiest/quickest way to clone a VM from one server onto another if you don't have shared storage between the two servers (so you can't do a standard migration)?
I have a production ready VM installed on one server, and I want to clone it onto another system. I don't have shared storage between the two hosts, but I've copied the disk image between the two hosts and added a config for it (virsh defined it). When I try to start it however it doesn't take:
# virsh create /etc/libvirt/qemu/cloned-vm.xml error: Failed to create domain from /etc/libvirt/qemu/cloned-vm.xml error: Unable to read from monitor: Connection reset by peer
I'm using KVM on RHEL6. Here is the duplicated config
<!-- WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE OVERWRITTEN AND LOST. Changes to this xml configuration should be made using: virsh edit or other application using the libvirt API. --> <domain type='kvm'> <name>cloned-vm</name> <uuid>NEW_UUID_HERE</uuid> <memory>15360000</memory> <currentMemory>15360000</currentMemory> <vcpu>7</vcpu> <os> <type arch='x86_64' machine='rhel6.2.0'>hvm</type> <boot dev='hd'/> </os> <features> <acpi/> <apic/> <pae/> </features> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='none'/> <source file='/local/vm/cloned-vm.img'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> <interface type='bridge'> <mac address='NE:W_:MA:C_:AD:DR'/> <source bridge='br2'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target type='serial' port='0'/> </console> <input type='tablet' bus='usb'/> <input type='mouse' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes'/> <video> <model type='cirrus' vram='9216' heads='1'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <memballoon model='virtio'> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </memballoon> </devices> </domain>
-
mgorven about 12 yearsDo you want to do offline migration, or just shutdown and boot?
-
Kish about 12 yearsThe VM to duplicate isn't in production, so it doesn't matter to me, I just need it duplicated across servers.
-
-
reox almost 6 yearsyou could pipe
dd
throughpv
which gives you a status bar, instead of calling USR1 all the time.