Converting an EC2 AMI to vmdk image

Question

I've come quite close to getting Amazon Linux to boot inside VirtualBox, thanks to this answer and these websites. A quick overview of the steps I've taken:

1. Launch EC2 instance with Amazon Linux 2011.09 64-bit AMI
2. dd the contents of the EBS volume over ssh to a local image file.
3. Mount the image file as a loopback device and then to a local mount point.
4. Create a new empty disk image file, partition with an offset for a bootloader, and create an ext4 filesystem.
5. Mount the new image's partition and copy everything from the EC2 image.
6. Install grub (using Ubuntu's grub-legacy-ec2 package, not grub2).
7. Convert the image file to vmdk using qemu-img.
8. Create a new VirtualBox VM with the vmdk.

Now the VM boots, grub loads, and the kernel is found. But it fails when it tries to mount the root device:

dracut Warning: No root device "block:/dev/xvda1" found
 
dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernel command line.
 
 
dracut Warning: Signal caught!
 
dracut Warning: Boot has failed. To debug this issue add "rdshell" to the kernel command line.
Kernel panic - not syncing: Attempted to kill init!
Pid: 1, comm: init Not tainted 2.6.35.14-107.1.39.amzn1.x86_64 #1

I have tried changing /boot/grub/menu.lst to find the root device by label and UUID, but nothing works. I'm guessing the xen kernel is not compatible with VirtualBox.

The reasoning behind all this effort is to make a Vagrant box that is as close to possible as the production enviroment, so deploys can be tested locally. I know it's cheap to do test runs on EC2, but poor connectivity often ruins the experience. Plus it would be really nice to have a virtual machine with the production environment so that co-workers don't have to install everything under the sun just to get up and running with app development.

If I were to try running a different kernel, what kernel could I get to be as close as possible to Amazon Linux 2011.09?

Solution 1

I tried so much and I was giving up, but at the end I succeed!

I've managed to boot with an Amazon Linux AMI ported under vmware!

I'm almost a newby, but I did it, after hours of trials, substituting the original amazon kernel with a vmlinuz-2.6.32-38-generic taken from an ubuntu 64 10.04 distro.

I just get a lot of errors because the amazon configuration inside the AMI is looking for something in an internal address that obviously it doesn't find. It's the /usr/bin/cloud-init, I think.

But I can login inside, and it seems that everything is working!

Thanks for your help!!

I've written this blogpost with step by step instructions to migrate from AMI to Vmware.

Hope this helps!

Solution 2

The official kernel Amazon derived theirs in their 2011.09 distribution, ie 2.6.35.14.

You can probably get the source package for the Amazon one with get_reference_source, but I wouldn't expect it to work outside of Amazon anyway.

Solution 3

The people over at Cloudify have shared how they create a VirtualBox ready Vagrant box, without access to bare metal machines - they use AWS.

9 steps to heaven:

• Create a source image (AMI) [...] using Packer.
• Launch a worker instance in AWS with the snapshot or source image as one of its disks.
• On the worker image: Create a raw image volume as a file and create an ext4 partition on it.
• Copy over the data from the source image disk to the previously created ext4 partition.
• Install the bootloader (extlinux) on the ext4 partition.
• Convert the raw image into a VMDK.
• Bundle the VMDK using an OVF descriptor and Vagrant metadata and create a tar file with the content and .box extension.
• Upload to S3 [or whatever you like].
• Clean up.

Source: Blog post at http://www.developer.com/design/creating-a-reproducible-and-portable-development-environment.html with the above 9 steps

Code: https://github.com/cloudify-cosmo/cloudify-packager/tree/3.2m4/image-builder

Comments

[Pratap]

Was doing work similar to you and found the solution on other link .. sharing with you as well. Tested this and it worked for me ... blog.zorangagic.com/2015/03/…

[Emanuele Preda]

BTW, Reed, I arrived exactly at the point you stated (dracut telling "no root device found"... did you move any further??

[Reed G. Law]

I came to basically the same solution you did: copying the kernel from Ubuntu. It works, but at that point I don't know how similar the system will perform to the Amazon Linux AMI.

[Codebling]

Please also see my answer here, which provides 2 alternative updated methods for exporting to VirtualBox/VMWare VMDK, as well as instructions on using your newly created VMDK with Vagrant