Is mv atomic on my fs?

Question

How can I check if mv is atomic on my fs (ext4)?

The OS is Red Hat Enterprise Linux Server release 6.8.

In general, how can I check this? I have looked around, and didn't find if my OS is standard POSIX.

Solution

Interestingly enough, it seems the answer may be, "It depends".

To be clear, mv is specified to

The mv utility shall perform actions equivalent to the rename() function

The rename function specification states:

This rename() function is equivalent for regular files to that defined by the ISO C standard. Its inclusion here expands that definition to include actions on directories and specifies behavior when the new parameter names a file that already exists. That specification requires that the action of the function be atomic.

But the latest the ISO C specification for rename() states:

7.21.4.2 The rename function

Synopsis

#include <stdio.h>
int rename(const char *old, const char *new);

Description

The rename function causes the file whose name is the string pointed to by old to be henceforth known by the name given by the string pointed to by new. The file named old is no longer accessible by that name. If a file named by the string pointed to by new exists prior to the call to the rename function, the behavior is implementation-defined.

Returns

The rename function returns zero if the operation succeeds, nonzero if it fails, in which case if the file existed previously it is still known by its original name.

Surprisingly, note that there is no explicit requirement for atomicity. It may be required somewhere else in the latest publicly-available C Standard, but I haven't been able to find it. If anyone can find such a requirement, edits and comments are more than welcome.

See also Is rename() atomic?

Per the Linux man page:

If newpath already exists, it will be atomically replaced, so that there is no point at which another process attempting to access newpath will find it missing. However, there will probably be a window in which both oldpath and newpath refer to the file being renamed.

The Linux man page claims the replacement of the file will be atomic.

Testing and verifying that atomicity might be very difficult, though, if that is how far you need to go. You're not clear as to what you mean in your use of "How can I check if mv is atomic". Do you want requirements/specification/documentation that it's atomic, or do you need to actually test it?

Note also, the above assumes the two operand file names are in the same file system. I can find no standard restriction on the mv utility to enforce that.

Comments

[Tizianoreica]

I have to ensure that move is atomic. is Testing enought to accept this? I can't tell. Yes, I'm working on a same fs (ext4 to ext4).

[Gilles 'SO- stop being evil']

POSIX doesn't guarantee atomicity either, but Linux, like most unix variants, does, for “native” filesystems such as ext4.

[Andrew Henle]

@Gilles The POSIX standard states "This rename() function is equivalent for regular files to that defined by the ISO C standard. ... That specification requires that the action of the function be atomic." But I can't find that atomicity requirement in the C standard. Is it there implicitly? Is it stated somewhere else in the standard? Because it's not explicitly in the rename() specification that I can see. And I quoted the entire ISO C rename spec.

[Gilles 'SO- stop being evil']

Given that ISO C only defines the behavior of one program and not of a whole system, it would be weird for it to say anything about rename atomicity.

[Andrew Henle]

@Gilles True. But we're left with the POSIX standard saying that ISO C requires that rename() be atomic. Perhaps a good language lawyer question on Stackoverflow...

[McN]

I read "that specification" as referring to the previous sentence ("Its inclusion here... specifies behavior when the new parameter names a file that already exists"), which refers earlier parts of the POSIX doc ("a link named new shall remain visible to other threads throughout... and refer either to the file referred to by new or old..."). In other words, POSIX promises to implement the ISO C standard, and makes additional guarantees beyond what ISO C provides. Does that interpretation help?

[Wodin]

@Tizianoreica I know this is an ancient post, but I just saw your comment and thought I should clarify: The actual filesystem has to be the same for the rename to be atomic. Not just the same type of filesystem. e.g. if you have / as an ext4 fs and /tmp as a different ext4 fs then you cannot atomically mv from one to the other.

[David Lopez]

I think that "rename() function" in mv manual you cited refers to the OS system call, not stdio library function.

[Andrew Henle]

@DavidLopez First, the strange fixation that attempts to differentiate between "system calls" and "library functions" makes absolutely no sense to me. It's meaningless. Unless you're directly writing assembly code, everything is a library function. POSIX defines the interface and how it behaves, not implementation details such as "Is this characterized as 'system call' or a 'library function'". There doesn't even have to be an actual rename "system call" - in fact, on most modern systems there isn't.

[David Lopez]

@AndrewHenle: thanks for the clarification. What I tried to address is the difference between the POSIX rename() function that mv manual refers to, and the ISO C standard rename() function which is specified independently of the OS (I mean, also for non-POSIX OS's such as WIndows).

[akinomyoga]

"Returns: The rename function returns zero if the operation succeeds, nonzero if it fails, in which case if the file existed previously it is still known by its original name." This part in the C standard implies that the operation is atomic. Maybe this is not the atomicity you expect, but this is the definition of the atomicity for the filesystem.

[Andrew Henle]

@akinomyoga No, it is not. That definition does not address what is observable from outside of the calling thread. Just because the calling thread can only see either success or a failed rename() that leaves the original file unchanged does not mean that an outside observer would never see something like both files existing at once, or even neither file existing, or some other combination. So no, the part you quoted doesn't even imply file system atomicity.

[akinomyoga]

@AndrewHenle Oh, thank you for your reply! I see. You are right. Does that mean before the introduction of the thread in C11 it was effectively atomic within the program, but later it became non-atomic at all because of the threads?