How to convert flat raw disk image to vmdk for virtualbox or vmplayer?

Question

I have some old images of old Linux filesystems in flat file format. they can be used by Bochs, but I need to run them with Virtual Box. Virtual Box cannot use images in this format, so I need to convert these images from flat file to .vmdk file format. Is there any way to do this?

Solution 1

First, install QEMU. On Debian-based distributions like Ubuntu, run:

$ apt-get install qemu

Then run the following command:

$ qemu-img convert -O vmdk imagefile.dd vmdkname.vmdk

I’m assuming a flat disk image is a dd-style image. The convert operation also handles numerous other formats.

For more information about the qemu-img command, see the output of

$ qemu-img -h

Solution 2

Since the question mentions VirtualBox, this one works currently:

VBoxManage convertfromraw imagefile.dd vmdkname.vmdk --format VMDK

Run it without arguments for a few interesting details (notably the --variant flag):

VBoxManage convertfromraw

Solution 3

To answer TJJ: But is it also possible to do this without copying the whole file? So, just to somehow create an additional vmdk-metafile, that references the raw dd-image.

Yes, it's possible. Here's how to use a flat disk image in VirtualBox:

First you create an image with dd in the usual way:

dd bs=512 count=60000 if=/dev/zero of=usbdrv.img

Then you can create a file for VirtualBox that references this image:

VBoxManage internalcommands createrawvmdk -filename "usbdrv.vmdk" -rawdisk "usbdrv.img"

You can use this image in VirtualBox as is, but depending on the guest OS it might not be visible immediately. For example, I experimented on using this method with a Windows guest OS and I had to do the following to give it a drive letter:

• Go to the Control Panel.
• Go to Administrative Tools.
• Go to Computer Management.
• Go to Storage\Disk Management in the left side panel.
• You'll see your disk here. Create a partition on it and format it. Use FAT for small volumes, FAT32 or NTFS for large volumes.

You might want to access your files on Linux. First dismount it from the guest OS to be sure and remove it from the virtual machine. Now we need to create a virtual device that references the partition.

sfdisk -d usbdrv.img

Response:

label: dos
label-id: 0xd367a714
device: usbdrv.img
unit: sectors

usbdrv.img1 : start=          63, size=       48132, type=4

Take note of the start position of the partition: 63. In the command below I used loop4 because it was the first available loop device in my case.

sudo losetup -o $((63*512)) loop4 usbdrv.img
mkdir usbdrv
sudo mount /dev/loop4 usbdrv
ls usbdrv -l

Response:

total 0
-rwxr-xr-x. 1 root root 0 Apr  5 17:13 'Test file.txt'

Yay!

Solution 4

On windows, use https://github.com/Zapotek/raw2vmdk to convert raw files created by dd or winhex to vmdk. raw2vmdk v0.1.3.2 has a bug - once the vmdk file is created, edit the vmdk file and fix the path to the raw file (in my case instead of D:\Temp\flash_16gb.raw (created by winhex) the generated path was D:Tempflash_16gb.raw). Then, open it in a vmware virtual machine version 6.5-7 (5.1 was refusing to attach the vmdk harddrive). howgh!

Solution 5

krosenvold's answer inspired the following script which does the following:

• get the dd dump via ssh from a remote server (as gz file)
• unzip the dump
• convert it to vmware

the script is restartable and checks the existence of the intermediate files. It also uses pv and qemu-img -p to show the progress of each step.

In my environment 2 x Ubuntu 12.04 LTS the steps took:

• 3 hours to get a 47 GByte disk dump of a 60 GByte partition
• 20 minutes to unpack to a 60 GByte dd file
• 45 minutes to create the vmware file

#!/bin/bash
# get a dd disk dump and convert it to vmware
#  see http://stackoverflow.com/questions/454899/how-to-convert-flat-raw-disk-image-to-vmdk-for-virtualbox-or-vmplayer
#  Author: wf  2014-10-1919

#
# get a dd dump from the given host's given disk and create a compressed
#   image at the given target 
#
#  1: host e.g. somehost.somedomain
#  2: disk e.g. sda
#  3: target e.g. image.gz
#
# http://unix.stackexchange.com/questions/132797/how-to-use-ssh-to-make-a-dd-copy-of-disk-a-from-host-b-and-save-on-disk-b
getdump() {
  local l_host="$1"
  local l_disk="$2"
  local l_target="$3"
  echo "getting disk dump of $l_disk from $l_host"
  ssh $l_host sudo fdisk -l  | egrep "^/dev/$l_disk"
  if [ $? -ne 0 ]
  then
    echo "device $l_disk does not exist on host $l_host" 1>&2
    exit 1
  else
    if [ ! -f $l_target ]
    then
      ssh $l_host "sudo dd if=/dev/$disk bs=1M | gzip -1 -" | pv | dd of=$l_target
    else
      echo "$l_target already exists"
    fi
  fi
}

#
# optionally install command from package if it is not available yet
# 1: command
# 2: package
#
opt_install() {
  l_command="$1"
  l_package="$2"
  echo "checking that $l_command from package $l_package  is installed ..."
  which $l_command
  if [ $? -ne 0 ]
  then
    echo "installing $l_package to make $l_command available ..."
    sudo apt-get install $l_package 
  fi
}

#
# convert the given image to vmware
#  1: the dd dump image
#  2: the vmware image file to convert to
#
vmware_convert() {
  local l_ddimage="$1"
  local l_vmwareimage="$2"
  echo "converting dd image $l_image to vmware $l_vmwareimage"
  #  convert to VMware disk format showing progess
  # see http://manpages.ubuntu.com/manpages/precise/man1/qemu-img.1.html
  qemu-img convert -p -O vmdk "$l_ddimage" "$l_vmwareimage"
}

#
# show usage
#
usage() {
  echo "usage: $0 host device"
  echo "      host: the host to get the disk dump from e.g. frodo.lotr.org"  
  echo "            you need ssh and sudo privileges on that host"
  echo "
  echo "    device: the disk to dump from e.g. sda"
  echo ""
  echo "  examples:
  echo "       $0 frodo.lotr.org sda"
  echo "       $0 gandalf.lotr.org sdb"
  echo ""
  echo "  the needed packages pv and qemu-utils will be installed if not available"
  echo "  you need local sudo rights for this to work"
  exit 1
}

# check arguments
if [ $# -lt 2 ]
then
  usage
fi

# get the command line parameters
host="$1"
disk="$2"

# calculate the names of the image files
ts=`date "+%Y-%m-%d"`
# prefix of all images
#   .gz the zipped dd
#   .dd the disk dump file
#   .vmware - the vmware disk file
image="${host}_${disk}_image_$ts"

echo "$0 $host/$disk ->  $image"

# first check/install necessary packages
opt_install qemu-img qemu-utils
opt_install pv pv

# check if dd files was already loaded
#  we don't want to start this tedious process twice if avoidable
if [ ! -f $image.gz ]
then
  getdump $host $disk $image.gz
else
  echo "$image.gz already downloaded"
fi

# check if the dd file was already uncompressed
# we don't want to start this tedious process twice if avoidable
if [ ! -f $image.dd ]
then
  echo "uncompressing $image.gz"
  zcat $image.gz | pv -cN zcat > $image.dd
else
  echo "image $image.dd already uncompressed"
fi
# check if the vmdk file was already converted
# we don't want to start this tedious process twice if avoidable
if [ ! -f $image.vmdk ]
then
  vmware_convert $image.dd $image.vmdk
else
  echo "vmware image $image.vmdk already converted"
fi

View more solutions

Comments

[Admin]

Should it be disk image or partition image could be used as well ?

[akw]

It's sufficient to install qemu-utils (apt-get install qemu-utils). This solution works nice (albeit a little bit slowish..)

[Giuseppe Guerrini]

IMHO this is a more clean solution since it only needs Vbox's tools, and vbox is supposed to be already there.

[karmakaze]

+1 and VBoxManage can also be used to convert the other way (virtual-image to raw) VBoxManage clonehd --format raw vmdiskimage.vmdk imagefile.dd

[Wolfgang Fahl]

I created a bash script to automated the process - see my answer @krosenvold - thanx this worked great

[Wolfgang Fahl]

for MacOS you might want to change apt-get install with sudo port install and instead of qemu-utils you need qemu then ...

[MikeKulls]

+1 because using virtual box tools is more likely to make it work with virtual box (in case there are any oddities about the way it does things)

[eyeshield21]

For anyone browsing for an explicit answer this also works with .img files: vboxmanage convertdd w1.img w1.vdi

[TJJ]

But is it also possible to do this without copying the whole file? So, just to somehow create an additional vmdk-metafile, that references the raw dd-image.

[Lloeki]

There is VBoxManage internalcommands createrawvmdk -filename "my_raw_disk.vmdk" -rawdisk "/dev/sda42" but I really don't know if it can point to a file instead of a block device.

[Max Vollmer]

Judging from your username and the name of the author of that project, it seems that this is your work. Please disclose your affiliation, as otherwise this is considered spam. Read How to not be a spammer.

[pdwalker]

this solution is cross platform - win/osx/linux.