How do I shrink the partition of an img file made from dd?

Question

I've copied the USB drive to an .img file using dd:

dd if=/dev/sdc of=myimage.img

I want to reduce the size of the partition in the image. I've tried several methods and always end up with a loopback-mounted image whose partition is still the full size of the USB.

• How do I modify myimage.img to have a smaller partition once loopback-mounted?

• Do I need to copy zeros into the empty part of the partition before doing so?

• Do I need to defragment so that when I reduce the image size, I'm deleting empty bytes? (From what I've read, Linux spreads out into the whole partition so I have no expectation that all of the data at the end of the image are zero bytes. Even writing all zeros will just consume the empty bytes wherever they lie.)

NOTE: I'm not trying to save disk space, so zipping doesn't help me.

BACKGROUND

I have Linux installed on a USB drive using ext4. I intend to duplicate the install for multiple devices. I've done so successfully but would like to create on the same USB drive a read-only partition with the system and a small partition that allows persistent storage. Rather than break my USB, I'm trying to modify a copy of the USB. I hope we don't get distracted by this background.

In short, I've done the following:

# Create mount point in current directory
sudo mkdir mnt
# Loopback mount the image
fdisk -l myimage.img
sudo mount -o offset=<partion_block_start * block_size> myimage.img mnt
# Copy all zeros to remaining space of the image
cd mnt
sudo dd if=/dev/zero of=filler conv=fsync bs=1M
rm filler
cd ..

First, I tried to use parted as described in this SuperUser answer and qemu-img as described in this other SuperUser answer.

sudo umount mnt
parted myimage.img
# At parted command prompt
(parted) resizepart 1
# Entered my end <target size>. Note that parted uses zero-based 
# indexing. This could be your final image size. In my case, the way the
# Linux installer worked, the partition started at 1M.
(parted) print
# I see that the partition is now sized as I expect
(parted) exit
# Just another sanity check
sudo parted -m esp3_007.img unit B print
# I see that the partition is now sized as I expect

When I try to mount the image again, it works fine but df still shows the partition as the same size. So then I tried

qemu-img resize myimage.img <target_size>

And now, when I try to mount the image, I get the "mount: wrong fs type, bad option, bad superblock..." error message.

Then I tried using gparted as described in this off-site post. The GUI showed a full partition unless I ran parted in the first place. Even then, the GUI wouldn't let me resize the partition.

For trying to force the partition size to be smaller, and starting with a fresh copy of myimage.img I tried fdisk described in this AskUbuntu answer

sudo fdisk myimage.img
Command (m for help): d
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4, default 1): 1
# defaults on the rest seemed to be correct in my case.

When I mounted the partition, it still showed as the same size.

Solution

On loop images

First of all, forget offset=, use losetup --partscan and just mount the partition via /dev/loop0p1.

# losetup --partscan /dev/loop0 myimage.img
# lsblk
# mount /dev/loop0p1 /mnt

To efficiently clear empty space within a partition, run fstrim on the loop-mounted filesystem, just like you would on an SSD. (This will actually make the image file sparse.)

# fstrim -v /mnt

On resizing partitions

But for now, you don't need to clear empty space or do anything like that. Whether the about-to-be-truncated area is filled with zeros or with chunks of old data is completely irrelevant.

Instead, you need to do exactly the same as you would do with ext4 on a real disk – you need to shrink each layer from the inside out. You cannot skip steps just because it's an image.

To shrink a partition that contains a filesystem, you must first tell the filesystem to shrink itself. For ext2/3/4 this is done using resize2fs. This will relocate data that might be living in the area you're about to chop off, and will store the new boundaries as part of the filesystem's metadata. (I suppose that's what you meant by "defragmenting" it.)

Only once the filesystem has been shrunk you can also shrink the partition containing it. This can be done via parted or fdisk by just changing the partition's end address.

Side note: You should be able to use GParted to resize the filesystem and the partition in one step – if it supports with working on loop devices, that is. It might depend on GParted's version. (However, the CLI parted cannot shrink filesystems, it just truncates the partition.)

Finally, once both the filesystem and the partition have been resized, you can truncate the whole image containing them. To do that, first detach the loop device and use truncate --size=... on your image file.

(To do it safely without having to do careful calculations, I would shrink the filesystem slightly more than needed to create some 'buffer' space; e.g. if I wanted a 4 GB image, I'd shrink the filesystem to 3 GB, the partition to 3.5 GB, and then truncate the image to 4 GB. Then grow everything in the opposite order to fill the 'buffer' space.)

Comments

[Xen2050]

NOTE: I'm not trying to save disk space - then why bother? Just copying files works A-OK for linux, then add a bootloader

[MrMas]

As I stated, I need to change the partition size to add another partition without reinstalling Linux.

[Xen2050]

You don't have to reinstall, just copy the files + grub/etc

[MrMas]

BTW, gparted wasn't working for similar reasons the other commands weren't working: gparted doesn't work directly on image files. Looping back, using losetup as described in this answer and then running gparted on the /dev/loopx allows for resizing.