How to flush writes to disk aggressively?

Question

I'm using Ubuntu 18.04 on x86_64 devices.

I have been haunted for months by a mysterious issue that I couldn't reproduce myself in which our application's configuration files were getting corrupted. Yesterday, I finally saw it happen twice and determined the cause of the issue: the user was pulling and reinserting the device's power to restart it after uploading new configuration files over SFTP.

I believe that the OS is buffering disk writes and then later choosing when to commit them to the disk. If the device is powered off before the buffer is flushed, then the write is lost. This presents as the file getting corrupted - it has a size of zero.

I know that the obvious solution is to stop this (horrifying) practice and encourage users to safely reboot the devices (since a reboot will flush the writes). I will try to encourage this, but unfortunately the users are used to being able to pull the power to restart the device; as bad as this is, it won't be easy to tell everyone to start doing it the right way.

So, my first question is if there's a way to aggressively flush writes, such that writes are more likely to be flushed to a disk quickly. I imagine there is some way to tune a kernel parameter that will affect this behavior.

I did realize that almost all of the important file uploads are going to be in one application-specific directory, so I also wonder if it is possible to use a program that listens for writes to the directory and forces a disk synchronization after each write. This might be preferable since it enforces quick flushing only for that directory, instead of the entire OS. Similarly, if there is a solution that affects only a single partition of a disk, that would be sufficient, since it would be straightforward to move the application-specific files to a dedicated partition.

What is the best way to tell Linux to aggressively flush writes to disk? Is there a solution that effectively does this for only a particular directory, or for a disk partition?

Solution

You can sync just a file with fsync or (sans metadata changes) with fdatasync, just a file system with syncfs, or everything with sync. But just a directory? sync /var/log/syslog is an example of syncing just a directory.

Comments

[millinon]

I don't have a program that's writing the files - they're just being uploaded via SFTP. This would mean that the SFTP daemon would have to be responsible for calling fsync. I'm thinking about just having a Python program using pyinotify to listen for any changes to the directory and responding to any change with a call to os.sync.