How to atomically rename a file in Java, even if the dest file already exists?

Question

I have a cluster of machines, each running a Java app.

These Java apps need to access a unique resource.txt file concurrently.

I need to atomically rename a temp.txt file to resource.txt in Java, even if resource.txt already exist.

Deleting resource.txt and renaming temp.txt doesn't work, as it's not atomic (it creates a small timeframe where resource.txt doesn't exist).

And it should be cross-platform...

Thanks !

Solution 1

For Java 1.7+, use java.nio.file.Files.move(Path source, Path target, CopyOption... options) with CopyOptions "REPLACE_EXISTING" and "ATOMIC_MOVE".

See API documentation for more information.

For example:

Files.move(src, dst, StandardCopyOption.ATOMIC_MOVE);

Solution 2

On Linux (and I believe Solaris and other UNIX operating systems), Java's File.renameTo() method will overwrite the destination file if it exists, but this is not the case under Windows.

To be cross platform, I think you'd have to use file locking on resource.txt and then overwrite the data.

The behavior of the file lock is platform-dependent. On some platforms, the file lock is advisory, which means that unless an application checks for a file lock, it will not be prevented from accessing the file. On other platforms, the file lock is mandatory, which means that a file lock prevents any application from accessing the file.

try {
    // Get a file channel for the file
    File file = new File("filename");
    FileChannel channel = new RandomAccessFile(file, "rw").getChannel();

    // Use the file channel to create a lock on the file.
    // This method blocks until it can retrieve the lock.
    FileLock lock = channel.lock();

    // Try acquiring the lock without blocking. This method returns
    // null or throws an exception if the file is already locked.
    try {
        lock = channel.tryLock();
    } catch (OverlappingFileLockException e) {
        // File is already locked in this thread or virtual machine
    }

    // Release the lock
    lock.release();

    // Close the file
    channel.close();
} catch (Exception e) {
}

Linux, by default, uses voluntary locking, while Windows enforces it. Maybe you could detect the OS, and use renameTo() under UNIX with some locking code for Windows?

There's also a way to turn on mandatory locking under Linux for specific files, but it's kind of obscure. You have to set the mode bits just right.

Linux, following System V (see System V Interface Definition (SVID) Version 3), lets the sgid bit for files without group execute permission mark the file for mandatory locking

Solution 3

Here is a discussion that relates: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4017593

Solution 4

As stated here, it looks like the Windows OS doesn't even support atomic file rename for older versions. It's very likely you have to use some manual locking mechanisms or some kind of transactions. For that, you might want to take a look into the apache commons transaction package.

Solution 5

You might get some traction by establishing a filechannel lock on the file before renaming it (and deleting the file you're going to overwrite once you have the lock). -r

View more solutions

Comments

[TofuBeer]

What about the user deleting the file by hand outside of Java? You cannot control that all file access goes through the Java service.

[Martin McNulty]

If you specify ATOMIC_MOVE, then "all other options are ignored" and "if the target file exists then it is implementation specific if the existing file is replaced or this method fails by throwing an IOException". However, I've tested passing ATOMIC_MOVE on Windows 7, Solaris 10 and RHEL Server 6.3 and they all perform the rename atomically, replacing the destination file.

[Martin McNulty]

I should mention that I only tested renaming files (not directories) within the same directory (not cross-filesystem) on local (not networked) filesystems. That's enough for my purposes, but YMMV.

[user1052080]

this should be marked as deprecated, as the new way should be java.nio!

[Thomas Mueller]

This only works if the file is so small that it can be overwritten in one write operation.

[user3151902]

Please remember that using a control file is not sufficient to implement safe file locking!

[Leliel]

@user1052080 Unfortunately NIO only goes partway to solving this problem. NIO doesn't support sync of file operations other than writing from a stream. This is sometimes a critical issue.

[vlp]

Commons Transaction buyer beware (see project web page): We have decided to move the project to dormant as we are convinced that the main advertised feature transactional file access can not be implemented reliably. We are convinced that no such implementation can be possible on top of an ordinary file system. Although there are other useful parts (as multi level locking including deadlock detection) the transactional file system is the main reason people use this library for. As it simply can not be made fully transactional, it does not work as advertised.