fstrim trims more than half of partition size even though partition mounted with discard

Question

When I installed my SSD I just mounted with discard and didn't sweat it. However today I was reading about the pros and cons of using fstrim instead and decided to run the program to get an idea of how long it would actually take (still with my partitions mounted with discard). The command took several minutes on both my root and home partitions. For my home partition I used -v and got this:

$ sudo fstrim -v /home
/home: 137494052864 bytes were trimmed

This is more than the amount of free space on the partition!

$ df -h /home
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2       206G   78G  118G  40% /home

Subsequent runs finish in less than a second, eg:

$ sudo fstrim -v /home
/home: 0 bytes were trimmed

Surely if I have always had the partition mounted with discard, fstrim should not trim a large amount of data like that? The discard option is definitely enabled, here are the relevant fstab lines:

UUID=xxxxxxxx...    /          ext4   noatime,discard,errors=remount-ro  0      1
UUID=xxxxxxxx...    /home      ext4   noatime,discard,errors=remount-ro  0      2

And mount output lines:

/dev/disk/by-uuid/xxxxxxxx... on / type ext4 (rw,noatime,discard,errors=remount-ro,stripe=128,data=ordered)
/dev/sda2 on /home type ext4 (rw,noatime,discard,errors=remount-ro,stripe=128,data=ordered)

The SSD is a TOSHIBA THNSNS256GMCP. Why does this happen?

Solution

Two things here:

1. fstrim trims all the data that is unallocated in the filesystem (well, not really all the data, only the data blocks that are not allocated, I don't think the unused parts of the inode table or the parts of not-completely used blocks are trimmed), regardless of whether discard is in used or not. fstrim cannot know which of those unallocated blocks have been "trimmed" or not already in the past, but it (actually the kernel, all the fstrim work is done in the FITRIM ioctl) does however keep track of which block group have been trimmed and will not trim them again if there hasn't been any unallocation in that block group since then, unless you're requesting a FITRIM with a smaller minimum extent length (from checking the ext4 code, it may be different for other file systems) which explains why you get 0 on the next run.

   Note that it doesn't harm to trim a block that has already been trimmed. That's just telling the SSD again that it can do whatever it wants with it (like erase it so it can be ready to use again for something else).

2. In df output, the "available" value doesn't take into account the space that is "reserved" for root, you'll notice that 206 - 76 is 130G, not 118G. 12G (about 5%) are reserved. See tunefs -m to change how much is reserved.

Comments

[Graeme]

So if fstrim doesn't know what has been trimmed already, why does it report 0 bytes the second time? Surely this must come from the disk, but then why would it report such a large trim the first time? Surely the disk would be agnostic to whether discard or trim had been used.

[Stéphane Chazelas]

@Graeme, argh, good point. fstrim, uses the FITRIM ioctl, and it's the kernel doing all the work and reporting the result to fstrim. I suppose the kernel keeps track of what has already been trimmed, but it can only do that since it's booted. Will investigate and update the answer.

[Graeme]

Ok, yes the kernel must track what has been trimmed since boot. If I reboot and do another fstrim, I get roughly the same output.

[Graeme]

Ok, thanks. Understand this trim stuff much better now.

[user3188445]

While some may feel ext4 should keep track of what is trimmed, the current design has an advantage. You can do a raw copy of a partition from one SSD to another, then mount the new SSD and call fstrim on it, and ensure everything is discarded. If instead the ext4 file system remembered what had been trimmed, you would copy this information over and it wouldn't be correct any more.