Performing atomic write operations in a file in bash

Question

After going through the bash documentation, this question and this one it's still not clear to me how can I perform atomic write (append) operations to a file in bash. I have a script that is run in multiple instances and at some point must write data to a file:

echo "$RESULT" >> `pwd`/$TEMP_DIR/$OUT_FILE

How is it possible to make all write operations from all concurrently running scripts to that file atomic (so that data from one instance doesn't overlap data from another)?

Solution

flock is one of the ways of interlocking operations. The utility is part of the util-linux toolset and is only available for Linux. Other utilities, available across a wider range of platforms, are based around Daniel J. Bernstein's setlock utility from his daemontools package:

• setlock from daemontools
• setlock from Bruce Guenter's daemontools-encore
• s6-setlock from Laurent Bercot's s6
• chpst from Gerrit Pape's runit
• runlock from Wayne Marshall's perp
• setlock from my nosh toolset

These tools operate with a slightly different paradigm to that used in M. Kurenkov's answer (one that flock can also employ, but does not in that answer). One invokes the setlock program to chain load to the command that must be interlocked. setlock itself opens and locks the lock file, and leaves a file descriptor for it open in its process. The lock persists for as long as that process does (unless the subsequent command chained to explicitly releases the lock by finding and closing the open file descriptor).

For the case in the question one must interlock the command that produces the output line, being aware that this invokes an external echo in place of a shell built-in echo command:

setlock mylockfile echo "$RESULT" >> ./$TEMP_DIR/$OUT_FILE

In this case it is not necessary to interlock opening the output file in append mode. If it were, one would have to open that file within the lock, which necessitates either using programs like fdredir/redirfd:

setlock mylockfile fdredir --append 1 "./$TEMP_DIR/$OUT_FILE" echo "$RESULT"

outfile() { setlock mylockfile fdredir --append 1 "./$TEMP_DIR/$OUT_FILE" "$@" ; }
[…]
outfile echo "$RESULT"

setlock mylockfile sh -c 'echo '"$RESULT"' >> "./'$TEMP_DIR'/'$OUT_FILE'"'

This of course generalizes to things other than writing to output files:

setlock mylockfile sh -c '… interlocked ; stuff …'