Concurrent file write

Question

how to write to a text file that can be accessed by multiple sources (possibly in a concurrent way) ensuring that no write operation gets lost?

Like, if two different processes are writing in the same moment to the file, this can lead to problems. The simples solution (not very fast and not very elegant) would be locking the file while beginning the process (create a .lock file or similar) and release it (delete the lock) while the writing is done.

When beginning to write, i would check if the .lock file exists and delay the writing till the file is released.

What is the recommended pattern to follow for this kind of situation?

Thanks

EDIT I mean processes, like different programs from different clients, different users and so on, not threads within the same program

Solution 1

The fastest way of synchronizing access between processes is to use Mutexes / Semaphores. This thread answers how to use them, to simulate read-writer lock pattern: Is there a global named reader/writer lock?

Solution 2

Consider using a simple database. You will get all this built-in safety relatively easy.

Solution 3

I would look at something like the Command Pattern for doing this. Concurrent writing would be a nightmare for data integrity.

Essentially you use this pattern to queue your write commands so that they are done in order of request.

You should also use the ReaderWriterLock to ensure that you can read from the file while writing occurs. This would be a second line of defense behind the command pattern so that only one thread could write to the file at a given time.

Solution 4

I suggest using the ReaderWriterLock. It's designed for multiple readers but ensures only a single writer can writer data at any one time MSDN.

View more solutions

Comments

[Matt Howells]

Don't use ReaderWriterLock, it has poor performance. Use ReaderWriterLockSlim or one of Jeffrey Richter's locks in the Wintellect PowerThreading library.

[Marcin Deptuła]

Don't use neither ;). ReadWriter locks, Monitors (locks) etc. are not capable of inter process locks, which is what author meant I think (Like, if two different processes are writing in the same moment to the file)?

[Efren Narvaez]

+1 for recommending using a queue, but I almost dinged you for the RW-lock comment afterwards. The Queue manages it's own locks... what else are you locking on?

[Kieveli]

Don't use interprocess locking. Switch to an OS that only runs one process at a time.

[Kieveli]

There's a way to use a named Mutex that can be referenced from multiple processes. If all file access is from within a single process (but multiple threads), then they can all point to the same mutex to acquire the lock.

[joshlrogers]

I meant it more for a second line of defense. Which in hind sight I think there is a better way to do it than using the RWL, but I would hate to have him prematurely optimize despite the performance downfalls of RWL. However I am starting to wonder if he meant multiple different processes/apps accessing and writing to a file, and if so then both of these would be worthless.

[Marcin Deptuła]

Kieveli - The author meant different processes, therefore - Mutexes / Semaphoers are the most obvious choices, and that's why I wrote about them, if we wold be talking about threads, both mutexes and simpler constructs like ReadWriteLock's would work.

[Marcin Deptuła]

Yeah, let's all start using MS-DOS

[Ian]

Ravadre, indeed they don't work across processes, however this answer was in well before the edit. I'll leave it here though in case anyone else stumbles across it looking for same process solutions.