disk image without free space

Question

I've an image of a bootable 16GB SD card. I've created the image with:

cat /dev/sdd | gzip >sdcard.img.gz

And I was happy because

$ du -h sdcard.img.gz
482M    sdcard.img.gz

482MB instead of 16GB, yay!

Here're the details of the (uncompressed) image:

$ du -h sdcard.img
15G     sdcard.img
$ partx -s sdcard.img
NR START     END SECTORS SIZE NAME UUID
 1 16384   81919   65536  32M      6e1be81b-01
 2 81920 3588095 3506176 1.7G      6e1be81b-02

However, now I need to write this image back to the SD card but I don't want to write 14GB of trailing zeros/junk! That'd take ages.

1. How can I create image without copying what's after the last partition?
2. When I already created image of whole SD card, how can I truncate it to not include useless junk?

The point is, I don't care about the size the image is taking in the backup, but I care about the size that's transferred back to SD card, because copying to SD card is slow and copying 14GB of useless data is pointless. So compressing the disk image or copying to a sparse aware filesystem as other answers on Internet suggest is not what I'm looking for.

Solution 1

Answering your first question: given you have an MBR there, I suggest you do something like dd'ing the first megabyte of the original drive (that contains the boot record and possibly the boot loader), then iterating over the partitions contained therein: dev=/dev/sda fdisk -l "$dev" | sed -ne '/^\//s,\(^[^ ]*\) .*,\1,p' | while read part do dd "if=$part" "of=$(basename "$part")" done And after you record the first megabyte to the target drive, ask the kernel to read the partition table with partprobe or kpartx. After this you should be able to dd the corresponding images contents to your new partitions.

Solution 2

Write to the end of the last partition (and a little more, for example a total of 2G (gibibytes) in this case).

You can use dd for this purpose. Check and doublecheck that you will write to the correct SD card (and not another drive) before you press the Enter key, because dd does what you tell it to do without any question.

This command line works from an uncompressed image file

$ sudo dd if=sdcard.img of=/dev/sdx bs=1M count=2K

where x is the drive letter.

But in this case there is a compressed image file. Run as root (for example via sudo -s and a corresponding command line can look like this, where zcat extracts from a gzip compressed file.

# < sdcard.img.gz zcat | dd of=/dev/sdx bs=4096 count=524288
524288+0 records in
524288+0 records out
2147483648 bytes (2,1 GB, 2,0 GiB) copied, 56,6886 s, 37,9 MB/s

Comments

[woky]

I don't think you understand what I wrote.

[sudodus]

@woky, you can solve the the first problem using dd with bs and count, but it is risky. I have no solution to the second problem, but if you create a new image, you can use dd with bs and count to clone only the relevant part of the drive.

[woky]

Thank you @sudosus. What you suggested is certainly better than nothing, though it's not exactly... exact. I thought there'd be some standard way to read only required part of the disk (i.e. without me adding GBs and MBs and ceiling in head). Also, I'd favor head -c2G /dev/sdb | gzip >sdcard.img.gz.

[sudodus]

You are right that my command line with dd was not correct/exact. I overlooked that the image is compressed, and should be piped via gzip as you do with your command line with head.

[sudodus]

@woky, I wanted to make a working command line (with dd), and added it to the answer. Your command line with head works to create an image (it looks good to me but I have not tested it). A corresponding extracting command line could look like this, < sdcard.img.gz zcat | pv | head -c2G > /dev/sdx where pv keeps you informed about the progress.