Flush disk write cache

Question

When the policy for a disk in Windows XP and Vista is set to enable write caching on the hard disk, is there a way to flush a file that has just been written, and ensure that it has been committed to disk?

I want to do this programmatically in C++.

Closing the file does perform a flush at the application level, but not at the operating system level. If the power is removed from the PC after closing the file, but before the operating system has flushed the disk write cache, the file is lost, even though it was closed.

Solution 1

You haven't specified the development environment, so:

.Net

IO streams have a .Flush method that does what you want.

Win32 API

There is the FlushFileBuffers call, which takes a file handle as argument.

EDIT (based on a comment from the OA): FlushFileBuffers does not need administrative privileges; it does only if the handle passed to it is the handle for a volume, not for a single file.

Solution 2

.NET FileStream.Flush() will NOT flush the Windows cache for that file content; Flush() only flushes the .NET internal file buffer. In .NET 4.0, Microsoft fixed the problem by adding an optional parameter to Flush() which if set true causes FlushFileSystemBuffers to be called. In .NET 3.5 and below your only choice is to call FlushFileBuffers via pinvoke. See MSDN'sFileStream.Flush community comment for how to do this.

Solution 3

You should not fix this at the time you close the file. Windows will cache, unless you open the file passing FILE_FLAG_WRITE_THROUGH to CreateFile().

You may also want to pass FILE_FLAG_NO_BUFFERING; this tells Windows not to keep a copy of the bytes in cache.

This is more efficient than FlushFileBuffers(), according to the CreateFile documentation on MSDN.

See also file buffering and file caching on MSDN.

Solution 4

You should also note, that your data might not get flushed to the actual disk, even when invoking a flush method of your frameworks API.

Calling the flush method will only tell the kernel to flush its pages to disk. However, if you have the disk write-cache turned on, it is allowed to delay the actual writing process indefinitely.

In order to ensure that your data gets written to the physical layer you have to turn of the write cache in your operating system. This most often comes with a performance penalty up to one or two orders of magnitude when dealing with a lot of small io-operations. Battery based support (UPS) or disks that accept commands to flush the disk write-cache are another option to deal with this problem.

Solution 5

From the microsoft documents you would use _flushall and link in COMMODE.OBJ to ensure that all buffers were committed to disk.

View more solutions

Comments

[selwyn]

It appears that this would work. Unfortunately, the application must be able to run without administrative rights.

[Will Dean]

I couldn't find this .NET method. As far as I can see, the only place the framework uses FlushFileBuffers is in SerialStream. I suspect that p/invoking FlushFileBuffers is safe bet.

[Zan Lynx]

Windows will force disk cache flushes for important data unless the disk policy has been set to "Turn off Windows write-cache buffer flushing"

[Philipp]

Of course it will. However, If the harddisk's write-cache is turned on the disk can stall the write-operation. Read, for example: support.microsoft.com/kb/259716/EN-US Quote: "By enabling write caching, file system corruption and/or data loss could occur if the machine experiences a power, device or system failure and cannot be shutdown properly."

[Vladislav Vaintroub]

Actually, Windows will force flush caching on FlushFileBuffers() no matter if write cache buffering was turned on, and even if write cache flushing (earlier known as "advanced performance checkbox") was turned on.

[Eric]

I found that calling FlushFileBuffers via pInvoke can cause an exception (stackoverflow.com/q/9195807/4540). Under .NET 4, it's easier and safer to just call FileStream.Flush(true) as @jimvfr suggests (stackoverflow.com/a/3992428/4540).

[jimvfr]

WARNING: .Net 4.0 Flush(true) doesn't fix it!!!! MS bug report here says file.Flush(true) is broken, then fixed, but doesn't say what version or service pack it was fixed in! Sounds like bug was if internal .NET FileStream buffer is empty, the Flush(true) did nothing??

[Artie Leech]

We also had to add the "c" mode option during fopen()