booting dd image on virtualbox

Question

I have been through a lot of questions regarding backing up your data, most recommended method is Clonezilla live CD but that wont work for me due to the server being remotely hosted by a company so I do not have a physical access

According to fdisk -l sda1 is the boot partition but after using dd to make a byte by byte copy of the data and downloaded to my computer, I cannot seem to boot from it on virtualbox after converting the output file to VDI, a quick search says that the partition does not have the boot files and I would need to make an image of the whole drive

I cannot make a dd image of the whole drive as the drive is 107GB and over 80GB is empty, I will end up with my drive fully consumed and no image

Is there a way to make a dd image with the boot files without cloning the whole drive? or maybe another way other than dd?

Solution

I have managed to make a virtual machine by doing the following

1. Create a Virtual Machine with a large HDD (500GB was enough for me).
2. Install Ubuntu server 12.04 (matching my remote server).
3. run this command ssh user@remoteserverip "dd if=/dev/sda" | dd of=/backup/ubuntu.bin it will make a copy of sda and save it to the virtual machine.
4. After the ubuntu.bin is fully downloaded to the virtual machine, install virtualbox using the following command sudo apt-get install virtualbox.
5. Run this command VBoxManage convertdd /backup/ubuntu.bin /backup/ubuntu.vdi --format VDI it will convert the image to a virtual hard disk.
6. Download ubunti.vdi form the virtual machine to the host computer, the host was a windows system so I used WinSCP to download the file.
7. Start a new virtual machine using the ubuntu.vdi file and you should be a running virtual machine of your remote server.

Comments

[sudodus]

A dd image is cloned, which means that every byte is copied as it is. You can compress the image, which works best, if you zeroise all free space in the partition(s). But you must expand from the compressed image in order to run from the image. There are other ways to backup, and I think you should select one of those in this case. After all, your personal data are most important to back up. It is rather easy to reinstall Ubuntu. See this link: BackupYourSystem

[Faez abdulkarem]

I have no problems with the image size, 20GB is no problem so no need for compression. The issue is running it on virtualbox as I have no knowledge on that I took a look at the link you provided, and tried some of the methods but most of them I either did wrongly or it did not work Tar gives me an error and rsync loops itself and makes a backup of the backup

[Faez abdulkarem]

I have tried TAR again and at the end, it gives the following Exiting with failure status due to previous errors on the previous lines all I see was socket ignored and no other issues

[sudodus]

You can create a new virtual drive, that is big enough for the dd image (at least 20 GB), and then clone from the dd image to this new virtual drive. Then try to boot from the new virtual drive. (If the original system, the source of the dd image, was running in the virtual machine, the drivers should be correct, and the cloned copy on the new virtual drive should work too.)

[sudodus]

Beware, you should not boot with with both the original drive and the cloned copy connected. So please 'disconnect' the original virtual drive before booting into the new drive with the cloned copy. Otherwise the system will be confused, and can destroy both systems.

[Faez abdulkarem]

The virtualbox is running on local computer, we want to terminate the remote server but before we do that, we need to make sure it is running on our local server I have made a bin image of the drive, converted it into vdi using virtualbox on the remote server, then downloaded it to local server and ran it on virtualbox but the boot files were not on the partition that I cloned I am trying again with TAR and hopefully this time it will backup everything, if it doesn't then I am not sure what to do

[sudodus]

It could be a problem, if there are two different systems for virtual machines. The installed system may not be portable between them. And in that case, you should save your personal files, which should be easy, and install a fresh Ubuntu system in your local virtual machine, which should also be easy. But you will have to redo the tweaks and install your custom program packages again.

[Faez abdulkarem]

honestly I am a little bit confused, and I am not sure if I delivered my question in the right way As stated above, I have a remote server and local server, I want all the data on the remote server except sda3 as it just empty space of almost 80GB, I used it as a path to create the images I have made a dd image in iso, it runs on the virtualbox but no boot files, I made another image in bin then converted it into VDI but still the same output So what I am trying to do would be a complete image of the drive, TAR seems to be the best option but for some reason it is not working, then use it

[sudodus]

I think it will be easier and faster to save your personal files (including 'work' files) and install a fresh Ubuntu system in your local virtual machine.

[Faez abdulkarem]

Honestly I do not know the important files on the remote server, as it has OpenERP, custom scripts and lots of other stuff installed by my manager before I started working on the remote server Hence why he wants a complete image TAR is still failing, I am not sure why though, is there a log somewhere that could point what the issue is?

[sudodus]

I think your manager is the best person to help you with those details.

[Faez abdulkarem]

The image on TAR keeps failing on files in the /tmp path another quick google says that file is not important should I exclude it?

[sudodus]

Yes. Temporary files are not important.

[Faez abdulkarem]

Still getting an error on the tmp path Here is the code that I am running sudo tar -cpzf backup-12-04-2018.tar.gz --one-file-system --exclude=/home2 --exclude=./tmp / is this the right syntax to exclude multiple directories?

[sudodus]

I think you should exclude /tmp, not ./tmp; Remove the dot.

[Faez abdulkarem]

Giving it a try right now, hope it works out Thank you @sudodus for the support

[Faez abdulkarem]

It is still not working, TAR is failing without giving a clue on what the issue is

[sudodus]

Did you discuss the problem with your manager? That way you can get important information about the system that is hard to guess or find. Maybe you can also discuss the alternative with a fresh installation locally (and what is really necessary to copy from the original system to a new local system).

[Faez abdulkarem]

I have tried talking with him about this solution but he still wants a full image of the remote system While this might be a long shot, would it be possible to unmount sda3 and mount is as sdb1? that would actually let me make a full copy of sda and save it to sdb1

[sudodus]

Please describe the details: In which computer/system do you intend to unmount sda3 and mount is as sdb1? How would you boot that computer/system? It should be from another drive for cloning to work.

[Faez abdulkarem]

I have got it working now, but the issue is eth0 is not working. guess I will be posting another question for that. Thanks for you help

[sudodus]

Congratulations and thanks for sharing your solution :-)

[Ricardo Ortega Magaña]

Typo in "converdd" is "convertdd"

[John Manko]

Perfect solution. Thanks!

[Mikolasan]

In VirtualBox 6 instead of convertdd use convertfromraw docs