How to clone a single partition from a HD to a qcow2 disk image for use in qemu-kvm?

Question

Edit (1&2):

I cloned my working OS (powered off, HD mounted on USB drive,

> dd if=/dev/sdc1 of=/data/system.img

The original SSD is as follows:

> sfdisk -d /dev/sdc
# partition table of /dev/sdc
unit: sectors

/dev/sdc1 : start=       63, size= 92164842, Id=83
/dev/sdc2 : start=        0, size=        0, Id= 0
/dev/sdc3 : start=        0, size=        0, Id= 0
/dev/sdc4 : start= 92164905, size=884603160, Id= 5
/dev/sdc5 : start= 92164968, size= 33559722, Id=82
/dev/sdc6 : start=125724753, size=851043312, Id=83

To create a VM with only the first partition, I got inspiration from the following article: Technical notes: convert a partition image to a bootable disk image.

I reconstructed a full disk image:

> dd if=/dev/zero of=d.img count=1 bs=1MiB

To start with a normal modern system header (starting with a blank slate with 2048 blocks of 512 bytes) - but unlike my old system which only had one 512-byte block.

> pv system.img >> d.img # to paste sdc1 onto that header
> file d.img 
d.img: data
> fdisk d.img # to initialise the header suitably (create the partition)

re-create the partition (n (+all defaults, said no to removing the existing ext4 signature), a (make bootable), w)

> file d.img
d.img: DOS/MBR boot sector; partition 1 : ID=0x83, active, start-CHS \
(0x0,32,33), end-CHS (0x275,145,28), startsector 2048, 92364800 sectors
> cp d.img e.img # take a backup
> file e.img 
e.img: DOS/MBR boot sector; partition 1 : ID=0x83, active, start-CHS \
(0x0,32,33), end-CHS (0x275,145,28), startsector 2048, 92364800 sectors

> losetup -f -P d.img
> losetup -l
/dev/loop4          0      0         0  0 /data/d.img         0     512
> blkid
/dev/loop4: PTUUID="55733c83" PTTYPE="dos"
/dev/loop4p1: UUID="437b9924-b81d-4054-b89e-b1ce0cf2a2c7" TYPE="ext4" PTTYPE="dos" PARTUUID="55733c83-01"
> mkdir d
> mount /dev/loop4p1 d/
> ll d
> mount --bind /dev d/dev
> mount --bind /sys d/sys
> mount --bind /proc d/proc
> chroot d
> ls -al /boot/grub
> less /boot/grub/grub.cfg
<...>
insmod ext2
set root='(hd0,1)'
search --no-floppy --fs-uuid --set 437b9924-b81d-4054-b89e-b1ce0cf2a2c7
<...>
> grub-install /dev/loop4
Installation finished. No error reported.
> vi etc/fstab
# uncomment swap and /data, just keep the root partition (which includes /boot)
> exit # come out of the chroot environment
> umount d/sys
> umount d/proc
> umount d/dev
> umount d/
> losetup -d /dev/loop4
> qemu-img convert -f raw -O qcow2 d.img d.qcow2

So I can definitely see the image, but somehow GRUB in qemu fails to see the partition:

> qemu-system-x86_64 d.qcow2

Stops under grub_rescue, stating that it does not recognise the UUID as above, and what I can see is:

> ls
(hd0) (fd0)

One thing I did differently to the article I got inspired from is that I used the local (old) grub-install (in the chroot), instead of the host's system. And checking 'file d.img' before and after shows a difference, so maybe that is where it breaks, even though I can mount it again and again & fdisk shows the same data before and after:

Before: d.img: DOS/MBR boot sector; partition 1 : ID=0x83, active, start-CHS (0x0,32,33), end-CHS (0x275,145,28), startsector 2048, 92364800 sectors
After: d.img: DOS/MBR boot sector

I have no idea why the valid partition is not seen.

Original question:

Context:
I cloned my current OS (Ubuntu10.04) from a single partition on an unmounted SSD.

dd if=/dev/sdc1 conv=sync,noerror bs=100M of=/data/system.img

Converted the raw image to qcow2 on my new U18.04 system (both images reside on an ext4 data partition separate from the OS):

qemu-img convert -f raw -O qcow2 system.img system.qcow2

This obviously fails to start (qemu-kvm says geom error when attempting to boot).

I do not want to copy the whole disk as it is too large (very slow and useless for my purpose).
However it became obvious to me that this method fails to copy essential disk sectors - amongst which the MBR with the GRUB boot.

So I also copied the first 512 bytes into a separate file:

dd if=/dev/sdc of=/data/sdc-512B.img bs=512 count=1 conv=sync,noerror
cat sdc-512B.img system.img > system2.img
qemu-img convert -f raw -O qcow2 system2.img system2.qcow2

Now the system looks like it is going to start but freezes forever during the boot. It say Booting from hard disk... but hangs there.

---

SW Versions:

> qemu-system-x86_64 --version
QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.14)
> uname -a
Linux <hostname> 4.18.0-18-generic #19~18.04.1-Ubuntu SMP Fri Apr 5 10:22:13 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
> lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.2 LTS
Release:        18.04
Codename:       bionic

---

Problem:

This image does not boot under kvm:

qemu-system-x86_64-spice -hda system2.qcow2 -m 4096

or

qemu-system-x86_64 -hda system2.qcow2 -m 4096

or

qemu-system-x86_64 -hda system2.qcow2 -m 4096 -no-acpi

Result is similar: the qemu window hangs during the boot process with one CPU working at 90-100% until I terminate it.

What am I doing wrong & what guide do I need to read to make this work without having to copy the whole disk?

Solution

/dev/sdc1 : start=       63, size= 92164842, Id=83
/dev/sdc2 : start=        0, size=        0, Id= 0
/dev/sdc3 : start=        0, size=        0, Id= 0
/dev/sdc4 : start= 92164905, size=884603160, Id= 5
/dev/sdc5 : start= 92164968, size= 33559722, Id=82
/dev/sdc6 : start=125724753, size=851043312, Id=83

Try this:

$ dd if=/dev/null of=disk.img seek=$((125724753 + 851043312))
    # create a big sparse file, the same size as /dev/sdc

# dd if=/dev/sdc of=disk.img conv=notrunc count=$((63 + 92164842))
    # copy through the mbr + gap + 1st partition into place

# sfdisk -d /dev/sdc | sfdisk disk.img
    # replicate the complete partitioning of /dev/sdc onto disk.img

$ qemu img convert -O qcow2 disk.img disk.qcow2
    # convert the raw image to qcow2

The commands with the # prompt need read-only access to your disk (ie you should run them as root). They should only write to the sparse disk.img file, but you better check twice ;-)

The resulted image may not be directly bootable if grub or other bootloader kept its 2nd or 3rd stage on a partitition you didn't copy through (sdc5 or sdc6).

You should not run grub-install. If you want to check if the partitioning did fine, you can do (before deleting disk.img):

# kpartx -l disk.img

# kpartx -a disk.img  # even try to attach ...
# mount /dev/mapper/loop0p1 /mnt/tmp  # and mount it

Comments

[asoundmove]

Would the person who down-voted me please provide some valuable insight as to what I can do to make it work?

[Amitav Pajni]

Consider just moving your data off the obsolete system and onto a new system. Also I've never had any luck converting a raw image to qcow2; if you must run a VM, consider running the raw image, or better yet, using advanced storage such as ZFS.

[asoundmove]

I have indeed moved all the data onto a NAS. But it is the apps, configuration and set-up that i still cling to and want to keep for the time being, while migrating. I just thought it would be easy to fire-up a VM from the partition image. Seems like it is not quite that straight forward.

[mosvy]

@MichaelHampton converting a raw image to qcow2 is just a matter of qemu-img convert -O qcow2 disk.img disk.qcow2. I did it countless times. To the OP: please include the output of sfdisk -d /dev/sdc in your question.

[asoundmove]

I edited my response to show the results of fsdisk -d, and more: I suspect my method breaks at grub-install.

[asoundmove]

Great suggestion, thank you. I tried but it fails: it hangs at Booting from the HD. Although this is my first brush with qemu, I believe I need GRUB in the MBR and I have neither copied the relevant sectors from sda, nor have I re-installed the grub configuration. Why is it bad to run grub-install? Though I can guestmount -a G.qcow2 -m /dev/sda1 G and see the contents correctly.

[asoundmove]

We have advanced a step further. It is now booting past grub, into the Ubuntu splash screen, but then it freezes - now, maybe I need to play with qemu settings, such as RAM adn CPUs.

[asoundmove]

It works: I started a VM & logged in. Now is configuration time: vga, network...