How can I format an SD card with a more robust Linux-usable filesystem with a specific cluster size for better write performace?

Question

Goal: microSD card formatted...

• for best write performance
• for use only with embedded Linux
• for better reliability (random power failures may occur)
• using an 64kB cluster size

I'm using an 8GB microSD card for data storage inside an embedded Linux/ARM device. The SD card is not removable. I've been using ext3 instead of the pre-installed FAT32 because it seems to better handle random power failures during writes. However, I kept noticing that my write performance is always best with the pre-installed FAT32 from Kingston. If I reformat the card with FAT32, the performance still suffers. After browsing wikipedia, I stumbled upon the following comment saying that some cards are optimized for specific cluster sizes. In my case, the Kingston comes pre-formatted for an 64kB cluster size.

Risks of reformatting

Reformatting an SD card with a different file system, or even with the same one, may make the card slower, or shorten its lifespan. Some cards use wear leveling, in which frequently modified blocks are mapped to different portions of memory at different times, and some wear-leveling algorithms are designed for the access patterns typical of the file allocation table on a FAT16 or FAT32 device.[60] In addition, the preformatted file system may use a cluster size that matches the erase region of the physical memory on the card; reformatting may change the cluster size and make writes less efficient.

Solution 1

Most SD cards (as well as USB drives) out there are optimised for use with FAT32. Using them with any other FS will cause your performance to suffer.

More info at LWN.

Solution 2

Note: The SD Card Association formatting tool does the following for you.

I don't have the full answer yet for all devices. However, here's the answer for 8GiB cards using FAT32.

Partition the card such that the FAT32 partition starts at sector 8,192 which is 4MiB from the start of the disk. Next, format the FAT32 partition with 32kiB clusters and set the number of "reserved sectors" such that the data area, cluster 2, of the FAT32 partition starts at logical sector 8,192 (or 8MiB from the start of the disk (4MiB from the start of the partition).

Use 32kiB clusters
| 4MiB      | 4MiB                                | remainder of disk    |  
| -------   | ---------------------------------   | ------------------   |  
| MBR       | First Partition Header + FAT tables | First Partition Data |  

Summary reasoning: the SD Card Association has recommended these 4MiB boundaries as optimized zones and recommends FAT32 as the file system for these sized cards. The controller chip in the card is optimized for small reads/writes in that second 4MiB area.

There's a lot more to it than that, but that's the quick and dirty answer. The above method only works for disks < 16GiB for 32kiB clusters and < 32GiB for 64kiB clusters. I suspect that at larger disk sizes, the data area is pushed out to logical 8MiB, but I don't have a disk larger than 8GiB to use.

Comments

[Admin]

If ext2 or ext3 allows setting cluster size (I don't know enough to know whether this is possible), this would be my suggestion.

[Harvey]

@Randolph: It allows you to set a block size of 1, 2, or 4kB. I ended up just trying them all and 1kB is significantly faster than the others.

[Admin]

Please post your solution as an answer so that we can close the question.

[hookenz]

64KB block size on flash would give bad write performance.

[hookenz]

I would recommend you use a 1KB sector size and ext2. If you're putting linux on there make use of tmpfs and minimise writes to the flash drive as much as possible. Also do not put a swap file on it.

[Harvey]

@MattH: I have several class 10 8GiB microSDs and they all have 8kiB or 16kiB page sizes. Using a 1kiB block size could cause much more churn of the drive than necessary. The key to flash is always writing in blocks equal to or larger than the page size. Ideally, most modern disks can write multiple pages at once in a stream, so you really want that number as your minimum write size if optimizing additionally for speed.

[hookenz]

That's not what I've found in practice. But don't use a journalled file system on them or your performance will seem to suffer.

[Hubert Kario]

And my experience is the same as Arnd Bergmann and I was using top of the line SanDisk SD and CF cards as well as Kingston's USB drives. btrfs in SSD mode, ext2, ext4 without journal... they all did deliver sub 100KiB/s performance after partially filling up the device.

[Hanzaplastique]

Would be interesting to find some "official" documentation on this, or a procedure how to investigate this.

[Harvey]

@Hanzaplastique: I'll try to track it down. The 4MB boundaries are in active use on most cards and I remember finding the recommendation for this in the SD Card specifications buried somewhere on their site.

[Hi-Angel]

Sorry, but the answer is misleading and unhelpful — apart from the link from which I came here btw. Such cards are optimized not for FAT32 but for specific page/segment sizes, which just happen to usually be on FAT32 (though not by chance). But it doesn't mean that a EXT can't be formatted to fit those sizes, nor does it mean that FAT32 can't be formatted to misfit them (which happened with OP btw). You may want to change the formulation to swap the definite will cause your performance to suffer with a bit more elaboration.

[Hubert Kario]

@Hi-Angel: maybe my mkfs.ext3 man page is incomplete, but I don't see any option there to put the inodes/metadata where would FAT32 allocation tables end up normally. Same for XFS, the best you can do is put them in the first 2^32 sectors

[Hi-Angel]

Fair enough, but removing/creation of files is less often a thing than modifiyng existing ones, like logging, or changing config/code/etc. E.g. if you copy a music library to a card — sizes of files are big enough anyway that most of the time would take copying the content, and you didn't notice the inode creation time. That is, if you format a FS to fit the erasure block size, overhead for inodes most of the usage time would either be non-existent, or small enough to get lost as a statistical noise. You can only notice something at all for a bunch of really small files.

[Hubert Kario]

@Hi-Angel I'd suggest you actually try doing this. This is not my experience and that is not behaviour described in the LWN article

[Hi-Angel]

@HubertKario I will, when I get a preformatted stick, so that I wouldn't do time attacks. Some of that I knew from other articles, like this one. And how is that not the behavior described? Can you quote a contradicting part?

[Hubert Kario]

@Hi-Angel the erasure block is a given with working with Flash. What I'm talking about is this: lwn.net/Articles/428836. that 4 to 8 MiB range in which FAT32 places File Allocation Table. That range is optimised to be constantly opened and with random access. Apart from it, cards support just one more opened block. So unless your metadata is in the 4-8 block or the same block as data, your performance will suffer.