Is there a way to DD multiple partitions into one raw image file in such a way that it boots?

Question

I have made great progress with my USB project so far, However I was wondering if there would be a way to DD multiple partitions into one Raw image file for redistribution. The following are the commands I have been using to achieve a full clone from usb to usb:

  DD if=/dev/sd[x] status=progress | gzip > newredhat.raw.gz

however, the only issue with this, is that it copies the entirety of the USB stick, (meaning it would copy the volume as 28gb) and in some cases it doesn't work when trying to go to a smaller USB stick. I had tried doing the following to work around this :

dd if=/dev/sdb1 | dd if=/dev/sdb2 | dd if=/dev/sdb3 | gzip > newredhat.raw.gz

Which to my surprise worked, however I don't believe it is saving the file the way I would imagine it is. Is there any way that I can avoid copying the entire disk drive (SDB) and only copy the necesarry partitions SDB1, SDB2, and SDB3, in such a way that I save them to one image file, and then zcat that file to a new USB of variable size in order for it to then run?

Thanks for any and all help in advance!

Solution

If you are only trying to avoid copying blank space at the end then the following should work.

1. Run fdisk on the device.
2. Make sure the units are set to sectors (they are by default in recent versions of fdisk)
3. Press p to print the partition table.
4. Read the end sector of the last partition.
5. Add 1
6. Divide by 2048 to convert to megabytes
7. Round up to the nearest whole number (better to copy an unneeded sector than not copy sectors that are needed)
8. Use this command:

---

DD if=/dev/sd[x] bs=1M count=<size in megabytes you calculated> status=progress | gzip > newredhat.raw.gz

Comments

[iBug]

I'd recommend file-based backup instead. You dd the partition table (usually the first 1 MB of the disk) and tar contents individual partitions. The downside is, obviously, being impossible to store the backup in one file.

[RickwhoPrograms]

Right, I thought of that approach as well however it doesn't preserve the one file structure, because eventually the whole package gets zipped down into a gz, and the compression ratio is actually quite great.. I did 32gb gzipped into like 8gb! Also I know there must be a way to accomplish this, I had to struggle to find the other method as well but I did eventually haha!

[Kamil Maciorowski]

dd if=… | dd if=… | dd if=… | … makes no sense. Only the last dd can do something useful.

[RickwhoPrograms]

the first 2 (sdb1 and sdb2) are 200m and 1.1g respectively, so if the command isn't working why do I end up with a 3.6gb file? Is there some replication going on?

[Kamil Maciorowski]

Please post the output of gdisk -l /dev/sdb. Add this information to your question by editing.

[Cadoiz]

You could/should use conv=noerror,sync for error safety for error safety: unix.stackexchange.com/a/144178/318461 - also think about the partition table, which you miss on the second command - consider this question: superuser.com/questions/738738/restoring-a-disk-image-with-d‌​d

[RickwhoPrograms]

let me give this a shot and see if it works as intended! thanks for the detailed response!!

[RickwhoPrograms]

Would this effectively save the file into a smaller sized bootable image though? also what If I just wanted to capture say the first 7gb of bootable media ?

[Alex]

dd ... bs=7G count=1 ?

[RickwhoPrograms]

but would that copy it in such a way that it is preserved as a bootable raw image?

[Alex]

I'm not sure. If you want to optimize image, you can use clonezilla that keep in its image only actual data. I don't know if it would work for you if going to distribute your solution, but actually you can customize/automate clonezilla or give receiver a link to clonezilla and instruction how to restore image.

[RickwhoPrograms]

Hi Alex, I tried a few things to no avail, I'm think the only optimal or close solution would be to do similar to what plugwash has suggested, I'm going to give it a shot now to see if it works. If it does ill upvote his answer and resolve the issue based on that!

[RickwhoPrograms]

Unfortunately projected solution doesn't seem to work either, RHEL hangs when loading OS and never fully boots.. The search continues! The fdisk produced 18,500mb's which I copied 19,000 just to be safe but still hangs!

[Cadoiz]

@Rick attention as to this only copies the partition table. Did you still use the second variant of your code to also backup the partitions? You could also try to do separate imaging for each partition.

[Cadoiz]

Probably you could also automate the @plugwash's procedure by scanning for the end sector of the last partition automatically. Alternatively, you can also consider using sudo sfdisk -d /dev/sdb > backup.sfdpart (the .extension is arbitrary) to backup and sudo sfdisk --force /dev/sdc < backup.sfdpart to (re-)load.