Disadvantages of ntfs-3g `big_writes` mount option

Question

man ntfs-3g makes the big_writes mount option seem like a Good Thing:

   big_writes
          This option prevents fuse from splitting write buffers into 4K
          chunks, enabling big write buffers to be transferred from the
          application in a single step (up to some system limit, generally
          128K bytes).

No down sides are listed... when would I want to avoid mounting with big_writes?

Solution

It's a good question, really. From what I can see here there's probably no downsides to it.

Back in 2008 when big_writes was first added to FUSE Miklos said:

Prior to 2.6.26 fuse only supported single page write requests. In theory all fuse filesystem should be able support bigger than 4k writes, as there's nothing in the API to prevent it. Unfortunately there's a known case in NTFS-3G where big writes cause filesystem corruption. There could also be other filesystems, where the lack of testing with big write requests would result in bugs.

To prevent such problems on a kernel upgrade, disable big writes by default, but let filesystems set a flag to turn it on.

The specific problem is documented in NTFS-3G's param.h as:

/*
 *        Use of big write buffers
 *
 *    With small volumes, the cluster allocator may fail to allocate
 *    enough clusters when the volume is nearly full. At most a run
 *    can be allocated per bitmap chunk. So, there is a danger when the
 *    number of chunks (capacity/(32768*clsiz)) is less than the number
 *    of clusters in the biggest write buffer (131072/clsiz). Hence
 *    a safe minimal capacity is 4GB
 */

#define SAFE_CAPACITY_FOR_BIG_WRITES 0x100000000LL

And currently it does defend against this limit by verifying that the fs capacity is at least 4GB before requesting big_writes. So it is safe.

In fact, Tuxera recommends turning it on if NTFS-3G is eating up your CPU:

Workaround: using the mount option “big_writes” generally reduces the CPU usage, provided the software requesting the writes supports big blocks.

The future

Since libfuse 3.0.0 (2016) big_writes is always on:

The -o big_writes mount option has been removed. It is now always active. File systems that want to limit the size of write requests should use the -o max_write=<N> option instead. commit

However, libfuse3 is a whole new generation, and ntfs-3g does not support building against it at all. I did inadvently backport the new 1MiB write buffer (libfuse 3.6.0) into ntfs-3g's internal mini libfuse2, so if you want to use that you can grab it from here:

1. https://github.com/AOSC-Dev/ntfs-3g/commit/96f6fbedaf7aad70b4a4c39d8dd4a637a63beec1
2. https://github.com/AOSC-Dev/ntfs-3g/commit/6aa576ae2a13fb636bff65bcb2701e731d2f5a94

Yes, I did remember to refactor the write size / volume size limit. And yes, it is faster with big file writes even on modern Ryzen.

PS: I am planning to add libfuse3 support to ntfs-3g, but there's no real deadline for it. Tuxera is demonstrably not interested in such a change, and coming up with a name for a fork is a bit too much for me.

Comments

[bodqhrohro]

>coming up with a name for a fork is a bit too much for me How about ntfs-lte? ;)