fstrim doesn't seem to trim a partition that uses lvm and dm-crypt

Question

I try to setup Fedora 25 with dm-crypt and LVM, but struggle to make TRIM work.

$ sudo fstrim -av                                                                            
/boot: 28.6 MiB (30003200 bytes) trimmed
/: 56.5 GiB (60672704512 bytes) trimmed

$ sudo fstrim -av                                                                            
/boot: 0 B (0 bytes) trimmed
/: 56.5 GiB (60671877120 bytes) trimmed

As you can see from the above output, repeatedly running fstrim works on unencrypted ext4 /boot, but seems to have no effect on / (which is on the same disk).

The setup is SSD -> dm-crypt -> LVM -> XFS

$ lsblk -D                                                                                                                                                                                     
NAME                                          DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sda                                                  0      512B       2G         0
├─sda2                                               0      512B       2G         0
│ └─luks-dd5ce54a-34c9-540c-a4cf-2a712b8a3a5e        0      512B       2G         0
│   └─fedora-root                                    0      512B       2G         0
└─sda1                                               0      512B       2G         0

According to this question, DISC-ZERO == 0 should not be the problem

# cat /etc/crypttab
luks-dd... UUID=dd.. none discard

# cat /etc/lvm/lvm.conf
devices {
...
     issue_discards = 1
...
}

I've also added rd.luks.options=discard option to /etc/default/grub, and updated initramfs and grub.cfg:

# grub2-mkconfig -o /boot/grub2/grub.cfg
# dracut -f

The discard option did correctly propagate:

# dmsetup table luks-d...                                                                                                                        
0 233385984 crypt aes-xts-plain64 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0 8:2 4096 1 allow_discards

I've tried variations of the above setup that can be found around the web, but this seems to follow manual pages. What I did not try was using different file system, but XFS should be supported.

Solution

I think your testing does not match the documentation (man fstrim).

-v, --verbose

Verbose execution. With this option fstrim will output the number of bytes passed from the filesystem down the block stack to the device for potential discard. This number is a maximum discard amount from the storage device's perspective, because FITRIM ioctl called repeated will keep sending the same sectors for discard repeatedly.

fstrim will report the same potential discard bytes each time, but only sectors which had been written to between the discards would actually be discarded by the storage device. Further, the kernel block layer reserves the right to adjust the discard ranges to fit raid stripe geometry, non-trim capable devices in a LVM setup, etc. These reductions would not be reflected in fstrim_range.len (the --length option).

I suggest looking for discard requests using blktrace instead, i.e. at the same time as you run fstrim. Hopefully it will show whether discard requests are being submitted to the block device on the bottom of the stack. You can compare the results between sda1 and sda2 (after a fresh boot, to avoid the undocumented behaviour on sda1).

btrace -a discard $DEV

Comments

[NefariousOctopus]

Just for reference you can runblktrace -d $DEV -a discard -o - | blkparse -i - to get live output. You might want to add it to your answer :) Thanks