How to make ext4 filesystem sparse?

Question

I am using Linux as guest OS in VirtualBox. I deleted huge number of files from its filesystem. Now i want to shrink the filesystem image file (vdi). The shrinking works by compressing filesystem image wherever it has "null" value in disk.

It seems an application called zerofree can write "null" into free space of filesystem in such a way that it becomes sparse. But the instructions say it works only on ext2/ext3. I have ext4 on my guest OS.

1. Why won't it work on ext 4 (reason cited is "extents", but can someone shed more light on it) ?

2. Will it work, If i mount the ext 4 as ext 3 and then remount as ext 4 ?

3. Any other tools that can do similiar thing as zerofree on ext ?

Solution 1

The page you reference (http://intgat.tigress.co.uk/rmy/uml/index.html) states:

The utility also works on ext3 or ext4 filesystems.

So I'm not sure where you're getting that it doesn't work on ext4 filesystems.

Note that the zerofree utility is different from the zerofree kernel patch that is mentioned on the same page (which indeed does not seem to have a version for ext4).

Update: At least in the case of VirtualBox, I don't think you need this utility at all. In my testing, on a stock Ubuntu 10.04 install on ext4, you can just zero out the filesystem like so:

$ dd if=/dev/zero of=test.file

...wait for the virtual disk to fill, then

$ rm test.file

and shut the VM down. Then on your VirtualBox host do:

$ VBoxManage modifyhd --compact yourImage.vdi

and you'll recover all the unused space.

Solution 2

You could use e2fsprogs, there is a version that is usable with ext4.

Make yourself superuser:

# su

Unmount your drive:

# umount /dev/sda1

Remove the journal of your drive:

# tune2fs -O ^has_journal /dev/sda1

And then resize your file system:

# resize2fs /dev/sda1 30G

Comments

[psusi]

According to the link there, zerofree works on ext4.

[phuclv]

I'm not sure if it works at your time or not, but the manpage said that it supports ext4

[rajaganesh87]

Thanks for ur reply, but I don't want to shrink the partition but the filesystem image(vdi) of the guest.

[rajaganesh87]

Thanks for your reply. The manpage of zerofree did not specify ext4, thats why i needed an alternative.

[Ronny K]

The alternative proposed (dd if=/dev/zero of=test.file) worked for me! Much simpler than other solutions.

[RMuesi]

Do a sync after the dd.

[Stefan]

dd idea worked for me too. Nicely done!

[CMCDragonkai]

Would you do this for ZFS? It seems to keep going forever.

[Geoff]

@CMCDragonkai If you've got compression or de-duplication turned on in ZFS filling the disk up with zeroes will likely never complete. Otherwise, it should work. If you've got compression turned on, you'll have to use incompressible data to fill the disk, change if=/dev/zero to if=/dev/random (or other randomness device as appropriate for your system). I'm not sure this will work for the purposes of shrinking the disk image afterwards though. I'll have to test it and get back to you.

[CMCDragonkai]

That makes sense using /dev/random. But I wonder if it does reclaim the disk space from deleted stuff.

[CMCDragonkai]

@pdo wouldn't make more sense to just add "1" to the file instead of calling random all the time? Would this be faster?

[Geoff]

@CMCDragonkai you could always just turn compression off, fill the disk with zeroes, delete and shrink, then turn compression back on :)

[CMCDragonkai]

Yea I was thinking about that. I wonder if it will work? I'll try and report back.

[robtot]

I tried this on my Lubuntu virtual machine which has 2 partitions. It only filled the sda partition with zeros (although that worked great). How do I modify the if= parameter to use a different partition?

[Geoff]

@OrganicMarble you need to change the of= parameter, not the if=. The example of=test.file just means put the output in a file called "test.file" in the current working directory. To zero out a different partition, you'll need to specify the output file in that different partition, for example: of=/path/to/other/partition/test.file

[Nabi K.A.Z.]

I test this solution for ext4 and xfs partition, and works well. Thanks.

[nzifnab]

One advantage of using zerofree over dd is that it only writes zeroes over the non-zero unused sectors of your disk - making it much faster and meaning it won't grow your virtual disk to its maximum size before it can be compacted.