Formatting an ext4 partition with a 16KB block, possible?

Question

I've been looking all over for this. I need to format a 6TB drive I have with a 16KB AUS in order to achieve maximum dedup. Is this possible?

I tried this with mkfs and got the following warning:

mkfs.ext4: 16384-byte blocks too big for system (max 4096)
Proceed anyway? (y,n) n

Is it suppose to be safe?

Solution

You need to use mkfs.ext4 with -C 16384 and also -O bigalloc. From man mkfs.ext4:

-C cluster-size Specify the size of cluster in bytes for filesystems using the bigalloc feature. Valid cluster-size values are from 2048 to 256M bytes per cluster. This can only be specified if the bigalloc feature is enabled. (See the ext4 (5) man page for more details about bigalloc.) The default cluster size if bigalloc is enabled is 16 times the block size.

The default cluster size would be 64KiB (16x4) if using 4096 byte blocks, as found in /etc/mke2fs.conf. This only applies if bigalloc is enabled at all; otherwise the cluster size is the block size.

Depending on the number of files you expect, you may also wish to set -i inode_ratio. The default is 16KiB+ on most systems, so you won't run out; but it may be inefficient if the average file is much larger.

The version of ext4 in your system's kernel must support the bigalloc feature, added in Linux 3.2.

Comments

[muru]

Probably of interest: unix.stackexchange.com/a/73568/70524

[JustAGuy]

well, I eventually found out I need to re-compile the kernel in order to use 16k blocks.

[sudodus]

@JustAGuy, Is it a feasible option for you to re-compile the kernel?

[JustAGuy]

Unfortunately not.

[sudodus]

@JustAGuy, Would a FAT file system work for your application, and in that case, would sector size = 16384 serve the purpose you wanted to implement with ext4? According to man mkfs.vfat you can try the option -S 16384; at least you can test the performance of such a file system for a huge partition.

[Bogdan Mart]

this should be marked as correct answer

[Siu Ching Pong -Asuka Kenji-]

Here is a reference to Bigalloc from Linux Kernel Wiki: ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#Bigalloc