Possible to get SSD TRIM (discard) working on ext4 + LVM + software RAID in Linux?

Question

We use RAID1+0 with md on Linux (currently 2.6.37) to create an md device, then use LVM to provide volume management on top of the device, and then use ext4 as our filesystem on the LVM volume groups.

With SSDs as the drives, we'd like to see the TRIM commands propagate through the layers (ext4 -> LVM -> md -> SSD) to the devices.

It looks like recent 2.6.3x kernels have had a lot of new SSD-related TRIM support added, including lots more coverage of Device Mapper scenarios, but we still can't seem to get it to cascade down properly.

Is this possible yet? If so, how? If not, is any progress being made?

Solution 1

^{UPD. 2020-06-17}

Looking back through commits history from 2020:

• 2010-08: Allow discards to be passed through to linear mappings if at least one underlying device supports it.

2 years later there're commits in regards for md (LSR), the one among them:

• 2012-10: md: raid 10 supports TRIM

— Basically in a few months since I edited my answer previously, Linux kernel became able to support block discards in the setup.

---

Previous versions of the answer:

^{UPD. 2012-07-17} UPD.: Thanks to Wodin for letting me know — according to lkml.org/lkml/2012/3/11/261 this functional has been addedproposed recently. proposed != accepted, though.

^{UPD. 2011-02-01} Not possible, cause Linux Soft RAID doesn't support this (yet?).

Solution 2

As of 2.6.37, it should be present (source). The kernel doesn't do it in the background, the block discard process is currently designed to be run on demand (cron script!). Dm-crypt support doesn't exist yet.

On January 13th, 2011 a patch was merged into dm-raid1.c that reads:

dm raid1: support discard

Enable discard support in the DM mirror target.
Also change an existing use of 'bvec' to 'addr' in the union.

I'm not 100%, but I think that's the merge-window for 2.6.38.

EXT4 added support a while ago, as did LVM. RAID is the one key that doesn't have support. As of 1/13/2011, it appears support has been added. Look for it in 2.6.38 or maybe 2.6.39.

---

Time has passed and TRIM support is definitely included in the 3.7 kernel. The commit for RAID10 reads:

This makes md raid 10 support TRIM. If one disk supports discard and another not, or one has discard_zero_data and another not, there could be inconsistent between data from such disks. But this should not matter, discarded data is useless. This will add extra copy in rebuild though.

Solution 3

Mdtrim may need more working:

Cyberax-mdtrim-0a40e8d# ./mdtrim.py -m /dev/md4 -s /home
Scratch directory is /home, trimmer file size is 0 GB 790 MB
Found slave sdc2 on /dev/sdc with MD offset 0 and partition offset 249856
Creating trimmer file
252,2: device not found in /dev
Traceback (most recent call last):
  File "./mdtrim.py", line 120, in <module>
    if lines[2].find("assuming %d byte sectors" % sector_size) == -1:
IndexError: list index out of range

Solution 4

As a summary, here are current recommended settings for performance and data safety on Linux.

Because of some bugs/risks, Debian does not enable LVM discard by default: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=717313

Gentoo Wiki proposes a one-time sequence to discard blocks in VG free space (as root):

lvcreate -l100%FREE -n trim yourvg
blkdiscard /dev/your/trim
lvremove yourvg/trim

About filesystems, it is recommended not to enable discard mount option in /etc/fstab but to only enable fstrim.timer generally provided in systemd-based distributions.

sudo systemctl enable fstrim.timer

Underlying layers like software raid should handle/forward discard/trim events from FS or LVM.

View more solutions

Comments

[poige]

See updated serverfault.com/a/229486/67675 :)

[poige]

and serverfault.com/questions/227918/…

[Don MacAskill]

Saw that thread, and the related commits, but like I said in the question - does this mean it'll properly pass through not only LVM (Device Mapper) but also md (Software RAID)?

[Deb]

@DonMacAskill The 'raid1.c' file doesn't have any commits referencing TRIM, FITRIM, or 'dispose'. So, looks like mdadm --create RAID support isn't there yet after all.

[Wodin]

It does now. lkml.org/lkml/2012/3/11/261

[poige]

@Wodin, I can't find the proposed changes even in 3.5. They don't seem to be accepted.

[Michael Hampton]

@sysadmin1138 I think you mean discard, which is what it's called within the Linux kernel. And that patch does seem to refer to discard.

[Konrad Gajewski]

How about software raid from btrfs?

[poige]

"dm-raid1.c that reads" — and dm's RAID isn't about having LVM-2 built on top of Linux Software RAID as originally given in the subject. I mean it's totally unrelated to the q-n altogether.

[poige]

@KonradGajewski, I think Btrfs can do that as well as it does in its plain layout.