When to use volatile with multi threading?

Question

If there are two threads accessing a global variable then many tutorials say make the variable volatile to prevent the compiler caching the variable in a register and it thus not getting updated correctly. However two threads both accessing a shared variable is something which calls for protection via a mutex isn't it? But in that case, between the thread locking and releasing the mutex the code is in a critical section where only that one thread can access the variable, in which case the variable doesn't need to be volatile?

So therefore what is the use/purpose of volatile in a multi-threaded program?

Solution 1

Short & quick answer: volatile is (nearly) useless for platform-agnostic, multithreaded application programming. It does not provide any synchronization, it does not create memory fences, nor does it ensure the order of execution of operations. It does not make operations atomic. It does not make your code magically thread safe. volatile may be the single-most misunderstood facility in all of C++. See this, this and this for more information about volatile

On the other hand, volatile does have some use that may not be so obvious. It can be used much in the same way one would use const to help the compiler show you where you might be making a mistake in accessing some shared resource in a non-protected way. This use is discussed by Alexandrescu in this article. However, this is basically using the C++ type system in a way that is often viewed as a contrivance and can evoke Undefined Behavior.

volatile was specifically intended to be used when interfacing with memory-mapped hardware, signal handlers, and the setjmp machine code instruction. This makes volatile directly applicable to systems-level programming rather than normal applications-level programming.

The 2003 C++ Standard does not say that volatile applies any kind of Acquire or Release semantics on variables. In fact, the Standard is completely silent on all matters of multithreading. However, specific platforms do apply Acquire and Release semantics on volatile variables.

[Update for C++11]

The C++11 Standard now does acknowledge multithreading directly in the memory model and the language, and it provides library facilities to deal with it in a platform-independent way. However the semantics of volatile still have not changed. volatile is still not a synchronization mechanism. Bjarne Stroustrup says as much in TCPPPL4E:

Do not use volatile except in low-level code that deals directly with hardware.

Do not assume volatile has special meaning in the memory model. It does not. It is not -- as in some later languages -- a synchronization mechanism. To get synchronization, use atomic, a mutex, or a condition_variable.

[/End update]

The above all applies to the C++ language itself, as defined by the 2003 Standard (and now the 2011 Standard). Some specific platforms however do add additional functionality or restrictions to what volatile does. For example, in MSVC 2010 (at least) Acquire and Release semantics do apply to certain operations on volatile variables. From the MSDN:

When optimizing, the compiler must maintain ordering among references to volatile objects as well as references to other global objects. In particular,

A write to a volatile object (volatile write) has Release semantics; a reference to a global or static object that occurs before a write to a volatile object in the instruction sequence will occur before that volatile write in the compiled binary.

A read of a volatile object (volatile read) has Acquire semantics; a reference to a global or static object that occurs after a read of volatile memory in the instruction sequence will occur after that volatile read in the compiled binary.

However, you might take note of the fact that if you follow the above link, there is some debate in the comments as to whether or not acquire/release semantics actually apply in this case.

Solution 2

In C++11, don't use volatile for threading, only for MMIO

But TL:DR, it does "work" sort of like atomic with mo_relaxed on hardware with coherent caches (i.e. everything); it is sufficient to stop compilers keeping vars in registers. atomic doesn't need memory barriers to create atomicity or inter-thread visibility, only to make the current thread wait before/after an operation to create ordering between this thread's accesses to different variables. mo_relaxed never needs any barriers, just load, store, or RMW.

For roll-your-own atomics with volatile (and inline-asm for barriers) in the bad old days before C++11 std::atomic, volatile was the only good way to get some things to work. But it depended on a lot of assumptions about how implementations worked and was never guaranteed by any standard.

For example the Linux kernel still uses its own hand-rolled atomics with volatile, but only supports a few specific C implementations (GNU C, clang, and maybe ICC). Partly that's because of GNU C extensions and inline asm syntax and semantics, but also because it depends on some assumptions about how compilers work.

It's almost always the wrong choice for new projects; you can use std::atomic (with std::memory_order_relaxed) to get a compiler to emit the same efficient machine code you could with volatile. std::atomic with mo_relaxed obsoletes volatile for threading purposes. (except maybe to work around missed-optimization bugs with atomic<double> on some compilers.)

The internal implementation of std::atomic on mainstream compilers (like gcc and clang) does not just use volatile internally; compilers directly expose atomic load, store and RMW builtin functions. (e.g. GNU C __atomic builtins which operate on "plain" objects.)

---

Volatile is usable in practice (but don't do it)

That said, volatile is usable in practice for things like an exit_now flag on all(?) existing C++ implementations on real CPUs, because of how CPUs work (coherent caches) and shared assumptions about how volatile should work. But not much else, and is not recommended. The purpose of this answer is to explain how existing CPUs and C++ implementations actually work. If you don't care about that, all you need to know is that std::atomic with mo_relaxed obsoletes volatile for threading.

(The ISO C++ standard is pretty vague on it, just saying that volatile accesses should be evaluated strictly according to the rules of the C++ abstract machine, not optimized away. Given that real implementations use the machine's memory address-space to model C++ address space, this means volatile reads and assignments have to compile to load/store instructions to access the object-representation in memory.)

---

As another answer points out, an exit_now flag is a simple case of inter-thread communication that doesn't need any synchronization: it's not publishing that array contents are ready or anything like that. Just a store that's noticed promptly by a not-optimized-away load in another thread.

    // global
    bool exit_now = false;

    // in one thread
    while (!exit_now) { do_stuff; }

    // in another thread, or signal handler in this thread
    exit_now = true;

Without volatile or atomic, the as-if rule and assumption of no data-race UB allows a compiler to optimize it into asm that only checks the flag once, before entering (or not) an infinite loop. This is exactly what happens in real life for real compilers. (And usually optimize away much of do_stuff because the loop never exits, so any later code that might have used the result is not reachable if we enter the loop).

 // Optimizing compilers transform the loop into asm like this
    if (!exit_now) {        // check once before entering loop
        while(1) do_stuff;  // infinite loop
    }

Multithreading program stuck in optimized mode but runs normally in -O0 is an example (with description of GCC's asm output) of how exactly this happens with GCC on x86-64. Also MCU programming - C++ O2 optimization breaks while loop on electronics.SE shows another example.

We normally want aggressive optimizations that CSE and hoist loads out of loops, including for global variables.

Before C++11, volatile bool exit_now was one way to make this work as intended (on normal C++ implementations). But in C++11, data-race UB still applies to volatile so it's not actually guaranteed by the ISO standard to work everywhere, even assuming HW coherent caches.

Note that for wider types, volatile gives no guarantee of lack of tearing. I ignored that distinction here for bool because it's a non-issue on normal implementations. But that's also part of why volatile is still subject to data-race UB instead of being equivalent to relaxed atomic.

Note that "as intended" doesn't mean the thread doing exit_now waits for the other thread to actually exit. Or even that it waits for the volatile exit_now=true store to even be globally visible before continuing to later operations in this thread. (atomic<bool> with the default mo_seq_cst would make it wait before any later seq_cst loads at least. On many ISAs you'd just get a full barrier after the store).

C++11 provides a non-UB way that compiles the same

A "keep running" or "exit now" flag should use std::atomic<bool> flag with mo_relaxed

Using

• flag.store(true, std::memory_order_relaxed)
• while( !flag.load(std::memory_order_relaxed) ) { ... }

will give you the exact same asm (with no expensive barrier instructions) that you'd get from volatile flag.

As well as no-tearing, atomic also gives you the ability to store in one thread and load in another without UB, so the compiler can't hoist the load out of a loop. (The assumption of no data-race UB is what allows the aggressive optimizations we want for non-atomic non-volatile objects.) This feature of atomic<T> is pretty much the same as what volatile does for pure loads and pure stores.

atomic<T> also make += and so on into atomic RMW operations (significantly more expensive than an atomic load into a temporary, operate, then a separate atomic store. If you don't want an atomic RMW, write your code with a local temporary).

With the default seq_cst ordering you'd get from while(!flag), it also adds ordering guarantees wrt. non-atomic accesses, and to other atomic accesses.

(In theory, the ISO C++ standard doesn't rule out compile-time optimization of atomics. But in practice compilers don't because there's no way to control when that wouldn't be ok. There are a few cases where even volatile atomic<T> might not be enough control over optimization of atomics if compilers did optimize, so for now compilers don't. See Why don't compilers merge redundant std::atomic writes? Note that wg21/p0062 recommends against using volatile atomic in current code to guard against optimization of atomics.)

---

volatile does actually work for this on real CPUs (but still don't use it)

even with weakly-ordered memory models (non-x86). But don't actually use it, use atomic<T> with mo_relaxed instead!! The point of this section is to address misconceptions about how real CPUs work, not to justify volatile. If you're writing lockless code, you probably care about performance. Understanding caches and the costs of inter-thread communication is usually important for good performance.

Real CPUs have coherent caches / shared memory: after a store from one core becomes globally visible, no other core can load a stale value. (See also Myths Programmers Believe about CPU Caches which talks some about Java volatiles, equivalent to C++ atomic<T> with seq_cst memory order.)

When I say load, I mean an asm instruction that accesses memory. That's what a volatile access ensures, and is not the same thing as lvalue-to-rvalue conversion of a non-atomic / non-volatile C++ variable. (e.g. local_tmp = flag or while(!flag)).

The only thing you need to defeat is compile-time optimizations that don't reload at all after the first check. Any load+check on each iteration is sufficient, without any ordering. Without synchronization between this thread and the main thread, it's not meaningful to talk about when exactly the store happened, or ordering of the load wrt. other operations in the loop. Only when it's visible to this thread is what matters. When you see the exit_now flag set, you exit. Inter-core latency on a typical x86 Xeon can be something like 40ns between separate physical cores.

---

In theory: C++ threads on hardware without coherent caches

I don't see any way this could be remotely efficient, with just pure ISO C++ without requiring the programmer to do explicit flushes in the source code.

In theory you could have a C++ implementation on a machine that wasn't like this, requiring compiler-generated explicit flushes to make things visible to other threads on other cores. (Or for reads to not use a maybe-stale copy). The C++ standard doesn't make this impossible, but C++'s memory model is designed around being efficient on coherent shared-memory machines. E.g. the C++ standard even talks about "read-read coherence", "write-read coherence", etc. One note in the standard even points the connection to hardware:

http://eel.is/c++draft/intro.races#19

[ Note: The four preceding coherence requirements effectively disallow compiler reordering of atomic operations to a single object, even if both operations are relaxed loads. This effectively makes the cache coherence guarantee provided by most hardware available to C++ atomic operations. — end note ]

There's no mechanism for a release store to only flush itself and a few select address-ranges: it would have to sync everything because it wouldn't know what other threads might want to read if their acquire-load saw this release-store (forming a release-sequence that establishes a happens-before relationship across threads, guaranteeing that earlier non-atomic operations done by the writing thread are now safe to read. Unless it did further writes to them after the release store...) Or compilers would have to be really smart to prove that only a few cache lines needed flushing.

Related: my answer on Is mov + mfence safe on NUMA? goes into detail about the non-existence of x86 systems without coherent shared memory. Also related: Loads and stores reordering on ARM for more about loads/stores to the same location.

There are I think clusters with non-coherent shared memory, but they're not single-system-image machines. Each coherency domain runs a separate kernel, so you can't run threads of a single C++ program across it. Instead you run separate instances of the program (each with their own address space: pointers in one instance aren't valid in the other).

To get them to communicate with each other via explicit flushes, you'd typically use MPI or other message-passing API to make the program specify which address ranges need flushing.

---

Real hardware doesn't run std::thread across cache coherency boundaries:

Some asymmetric ARM chips exist, with shared physical address space but not inner-shareable cache domains. So not coherent. (e.g. comment thread an A8 core and an Cortex-M3 like TI Sitara AM335x).

But different kernels would run on those cores, not a single system image that could run threads across both cores. I'm not aware of any C++ implementations that run std::thread threads across CPU cores without coherent caches.

For ARM specifically, GCC and clang generate code assuming all threads run in the same inner-shareable domain. In fact, the ARMv7 ISA manual says

This architecture (ARMv7) is written with an expectation that all processors using the same operating system or hypervisor are in the same Inner Shareable shareability domain

So non-coherent shared memory between separate domains is only a thing for explicit system-specific use of shared memory regions for communication between different processes under different kernels.

See also this CoreCLR discussion about code-gen using dmb ish (Inner Shareable barrier) vs. dmb sy (System) memory barriers in that compiler.

I make the assertion that no C++ implementation for other any other ISA runs std::thread across cores with non-coherent caches. I don't have proof that no such implementation exists, but it seems highly unlikely. Unless you're targeting a specific exotic piece of HW that works that way, your thinking about performance should assume MESI-like cache coherency between all threads. (Preferably use atomic<T> in ways that guarantees correctness, though!)

---

Coherent caches makes it simple

But on a multi-core system with coherent caches, implementing a release-store just means ordering commit into cache for this thread's stores, not doing any explicit flushing. (https://preshing.com/20120913/acquire-and-release-semantics/ and https://preshing.com/20120710/memory-barriers-are-like-source-control-operations/). (And an acquire-load means ordering access to cache in the other core).

A memory barrier instruction just blocks the current thread's loads and/or stores until the store buffer drains; that always happens as fast as possible on its own. (Or for LoadLoad / LoadStore barriers, block until previous loads have completed.) (Does a memory barrier ensure that the cache coherence has been completed? addresses this misconception). So if you don't need ordering, just prompt visibility in other threads, mo_relaxed is fine. (And so is volatile, but don't do that.)

See also C/C++11 mappings to processors

Fun fact: on x86, every asm store is a release-store because the x86 memory model is basically seq-cst plus a store buffer (with store forwarding).

---

Semi-related re: store buffer, global visibility, and coherency: C++11 guarantees very little. Most real ISAs (except PowerPC) do guarantee that all threads can agree on the order of a appearance of two stores by two other threads. (In formal computer-architecture memory model terminology, they're "multi-copy atomic").

• Will two atomic writes to different locations in different threads always be seen in the same order by other threads?
• Concurrent stores seen in a consistent order

Another misconception is that memory fence asm instructions are needed to flush the store buffer for other cores to see our stores at all. Actually the store buffer is always trying to drain itself (commit to L1d cache) as fast as possible, otherwise it would fill up and stall execution. What a full barrier / fence does is stall the current thread until the store buffer is drained, so our later loads appear in the global order after our earlier stores.

• Are loads and stores the only instructions that gets reordered?
• x86 mfence and C++ memory barrier
• Globally Invisible load instructions

(x86's strongly ordered asm memory model means that volatile on x86 may end up giving you closer to mo_acq_rel, except that compile-time reordering with non-atomic variables can still happen. But most non-x86 have weakly-ordered memory models so volatile and relaxed are about as weak as mo_relaxed allows.)

Solution 3

(Editor's note: in C++11 volatile is not the right tool for this job and still has data-race UB. Use std::atomic<bool> with std::memory_order_relaxed loads/stores to do this without UB. On real implementations it will compile to the same asm as volatile. I added an answer with more detail, and also addressing the misconceptions in comments that weakly-ordered memory might be a problem for this use-case: all real-world CPUs have coherent shared memory so volatile will work for this on real C++ implementations. But still don't do it.

Some discussion in comments seems to be talking about other use-cases where you would need something stronger than relaxed atomics. This answer already points out that volatile gives you no ordering.)

---

Volatile is occasionally useful for the following reason: this code:

/* global */ bool flag = false;

while (!flag) {}

is optimized by gcc to:

if (!flag) { while (true) {} }

Which is obviously incorrect if the flag is written to by the other thread. Note that without this optimization the synchronization mechanism probably works (depending on the other code some memory barriers may be needed) - there is no need for a mutex in 1 producer - 1 consumer scenario.

Otherwise the volatile keyword is too weird to be useable - it does not provide any memory ordering guarantees wrt both volatile and non-volatile accesses and does not provide any atomic operations - i.e. you get no help from the compiler with volatile keyword except disabled register caching.

Comments

[Stefan Mai]

In some cases, you don't want/need protection by the mutex.

[David Heffernan]

Sometimes its fine to have a race condition, sometimes it isn't. How are you using this variable?

[John Dibling]

@David: An example of when it is "fine" to have a race, please?

[David Heffernan]

@John Here goes. Imagine you have a worker thread which is processing a number of tasks. The worker thread increments a counter whenever it finishes a task. The master thread periodically reads this counter and updates the user with news of the progress. So long as the counter is properly aligned to avoid tearing there is no need to synchronise access. Although there is a race, it is benign.

[John Dibling]

@David: It would be difficult to evaluate the safety of such a device without a complete examination of the code. Even if an examination concluded that the writes were atomic (questionable) and fully written-through the cache (difficult to tell), I would still reject this as "bad code." It's safetey would be extremely tenuous, and easily broken by the smallest changes to the code. Maintennance programmers would break this device easily, and the problems may not show up in testing.

[David Heffernan]

@John The hardware on which this code runs guarantees that aligned variables cannot suffer from tearing. If the worker is updating n to n+1 as the reader reads, the reader doesn't care whether they get n or n+1. No important decisions will be taken since it is only used for progress reporting.

[John Dibling]

@David: I guess I don't know what you mean by "tearing."

[David Heffernan]

@John re tearing, I offer you the following from Joe Duffy: msdn.microsoft.com/en-us/magazine/cc817398.aspx

[John Dibling]

@David: Wow, wall of text. :) But thanks, I haven't read this yet. I will when I get a chance.

[David Heffernan]

@John It's all excellent stuff but the bit on tearing is only a couple of paragraphs.

[Jonathan Wakely]

isvolatileusefulwiththreads.com (also @DavidHeffernan, software.intel.com/en-us/blogs/2013/01/06/… is the must-read piece on "benign" data races)

[alecov]

When to use: never.

[curiousguy]

@JohnDibling "An example of when it is "fine" to have a race" Whenever it's fine to use a mutex. Or an atomic. Pretty much all non trivial MT programs have harmless race conditions.

[Gabriel Staples]

Related: Electrical Engineering Stack Exchange: Using volatile in embedded C development. This says volatile is required in 2 places: 1) for memory-mapped registers, 2) when sharing global variables between an ISR context and your main context.

[David Heffernan]

If I recall, C++0x atomic, is meant to do properly what a lot of people believe (incorrectly) is done by volatile.

[zeuxcg]

Yeah. It's funny, I've added a line about C++0x atomic facilities, but then removed it since it seemed not related to the exact question :) C++0x atomic is as it should have been - concrete load/store semantics on each access are very useful.

[Admin]

+1 Very good links. The semantics of volatile should not be confused among languages, including what might be found in a dictionary :-) (It carries very strict memory-model semantics in Java, for instance, but that's a different language/environment).

[Michael]

Some compilers do provide additional semantics to volatile that is useful for multithreaded development, but this is definitely not part of the standard.

[Ben Jackson]

Part of me wants to downvote this because of the condescending tone of the answer and the first comment. "volatile is useless" is akin to "manual memory allocation is useless". If you can write a multithreaded program without volatile it is because you stood on the shoulders of people who used volatile to implement threading libraries.

[David Heffernan]

@Ben just because something challenges your beliefs doesn't make it condescending

[Ben Jackson]

@John: I think the first comment stating "if you downvote you don't understand" affected my interpretation of the tone. Other than that I read it as a strong statement against the use of volatile which is probably correct for most people writing application level code.

[josesuero]

@Ben: no, read up on what volatile actually does in C++. What @John said is correct, end of story. It has nothing to do with application code vs library code, or "ordinary" vs "god-like omniscient programmers" for that matter. volatile is unnecessary and useless for synchronization between threads. Threading libraries can't be implemented in terms of volatile; it has to rely on platform-specific details anyway, and when you rely on those, you no longer need volatile.

[josesuero]

volatile doesn't prevent memory accesses from being reordered. volatile accesses won't be reordered with respect to each others, but they provide no guarantee about reordering with respect to non-volatile objects, and so, they're basically useless as flags as well.

[Ben Jackson]

I think most of the "volatile is useless" crowd are relying on the fact that more likely code such as while (!global_flag) { sleep(1); } will work without volatile because their compilers don't look into sleep() when optimizing and thus assume that sleep() may modify global_flag and thus the right code is produced for the wrong reasons. Perhaps a suitable example could be constructed with LLVM link-time optimizations?

[David Preston]

But isn't this, and the same example in the other response, busy waiting and thus something that should be avoided? If this is a contrived example, are there any real life examples that aren't contrived?

[David Preston]

But that code is using busy waiting - something to be avoided.

[ctrl-alt-delor]

Ok yes. You realy need to do something in the braces (I will edit post). Busy waiting is usualy a bad idea. You may be processing something (a list) until another thread signals you to stop. Without the valatile it will continue forever, and for this example no lock is needed, bool is atomic.

[josesuero]

@Ben: I think you've got it upside down. The "volatile is useless" crowd relies on the simple fact that volatile does not protect against reordering, which means it is utterly useless for synchronization. Other approaches might be equally useless (as you mention, link-time code optimization might allow the compiler to peek into code you assumed the compiler would treat as a black box), but that doesn't fix the deficiencies of volatile.

[josesuero]

@Chris: Busy waiting is occasionally a good solution. In particular, if you expect to only have to wait for a couple of clock cycles, it carries far less overhead than the much more heavyweight approach of suspending the thread. Of course, as I've mentioned in other comments, examples such as this one are flawed because they assume reads/writes to the flag won't be reordered with respect to the code it protects, and no such guarantee is given, and so, volatile isn't really useful even in this case. But busy waiting is an occasionally useful technique.

[Admin]

@jalf: "volatile is unnecessary and useless for synchronization between threads" (which is what you said) is not the same thing as "volatile is useless for multithreaded programming" (which is what John said in the answer). You are 100% correct, but I disagree with John (partially) - volatile can still be used for multithreaded programming (for a very limited set of tasks)

[Admin]

eg, see the tenth response (by "Spud") in the comments here for a legitimate use of volatile: software.intel.com/en-us/blogs/2007/11/30/… (although this IS somewhat x86 specific as other platforms may require cores to flush data to each other, which volatile obviously won't do)

[Admin]

@jalf: some uses of flags do not require that they are not reordered (though most do, so while there is a very very niche use case where using volatile is perfectly fine, there are lots and lots of people who misuse volatile in unsafe ways)

[josesuero]

@Dan: name a situation where reordering is not a problem. The flag is used to indicate that some event has occurred, and that only works if the event has actually occurred when the flag is set. Which other cases do you have in mind?

[Admin]

@jalf: See the article by Arch Robinson (linked elsewhere on this page), 10th comment (by "Spud"). Basically, the reordering does not change the logic of the code. The posted code uses the flag to cancel a task (rather than to signal the task is done), so it doesn't matter if the task is cancelled before or after the code (eg: while (work_left) { do_piece_of_work(); if (cancel) break;}, if the cancel is reordered within the loop, the logic is still valid. I had a piece of code which worked similarly: if the main thread wants to terminate, it sets the flag for other threads, but it doesn't...

[Admin]

...matter if the other threads do an extra few iterations of their work loops before they terminate, as long as it happens reasonably soon after the flag is set. Of course, this is the ONLY use that I can think of and its rather niche (and may not work on platforms where writing to a volatile variable does not make the change visible to other threads, though on at least x86 and x86-64 this works). I certainly wouldn't advise anybody to actually do that without a very good reason, I'm just saying that a blanket statement like "volatile is NEVER useful in multithreaded code" is not 100% correct.

[GManNickG]

@Dan: So you've basically said "it is useful! ...assuming this, and ignoring this and that, and...". Anything can be true if you make enough assumptions; and you've certainly ditched C++ as a language with those.

[Admin]

@John: well, you never said it was useless for everything and I never said that you said that either. You still said that its useless for multithreaded programming, which is what I'm disputing. Its not particularly useful, but its not entirely useless either.

[Admin]

@GMan: Everything that is useful is only useful under a certain set of requirements or conditions. Volatile is useful for multithreaded programming under a strict set of conditions (and in some cases, may even be better (for some definition of better) than alternatives). You say "ignoring this that and.." but the case when volatile is useful for multithreading doesn't ignore anything. You made up something which I never claimed. Yes, the usefulness of volatile is limited, but it does exist - but we can all agree that it is NOT useful for synchronization.

[GManNickG]

@Dan: My point was volatile is a C++ language concept, not an implementation concept. Yet you said it was useful because of some implementation details. That has no implications on it's usefulness in C++.

[Martin Ba]

@JohnDib : I think you should consider adding to your answer that volatile has release&aquire [semantics on Visual C++ ](msdn.microsoft.com/en-us/library/12a04hfd%28v=vs.80%29.asp‌​x) (also see my answer here: stackoverflow.com/questions/6995310/…)

[John Dibling]

@Martin: I have recently made an extensive elaborative edit to this answer. I included some platform-specific details.

[ed9w2in6]

I can hardly think of an real world example to use volatile in normal applications-level programming, is there a one?

[Voo]

Since I was looking for a good answer I could link to why volatile is useless, I can't let these comments here stand. What Dan is saying here only works for x86 because of its strong underlying memory model. What he's proposing here is equally broken as every other usecase of volatile on many other platforms (e.g. there's no guarantee that you don't read a stale value from the cache). So yes if you want a program that doesn't just work under x86, volatile is really never useful..

[ctrl-alt-delor]

@jalf my understanding of things is that volatile tells to compiler that the variable can be read/written asynchronously to the program (by another thread or my hardware) it is supposed to give the semantic of “if I say read or write then read or write, and do it when I tell you to.”. If the CPU is re-ordering instruction that change these semantics, then the compiler has to defeat this optimisation as well. Am I missing something.

[josesuero]

@richard Yes and no. The first half is correct. But this only means that the CPU and compiler are not allowed to reorder volatile variables with respect to each others. If I read a volatile variable A, and then read a volatile variable B, then the compiler must emit code that is guaranteed (even with CPU reordering) to read A before B. But it makes no guarantees about all the non-volatile variable accesses. They can be reordered around your volatile read/write just fine. So unless you make every variable in your program volatile, it won't give you the guarantee you're interested in

[fredoverflow]

What is "Windows 2010"?

[John Dibling]

@FredOverflow: A typo.

[Mgetz]

This needs an update to mention std::atomic<T>

[John Dibling]

@Mgetz: the Jan 20 edit included mention of atomic and other devices.

[Trevor Boyd Smith]

My summary of all of this is: (1.) there exist examples where volatile is correct/useful for multithreading (@Dan's example: use a volatile bool to stop a thread's loop. another example) (1.addendum) the examples where volatile is correct/useful for multithreading are compiler/implementation dependent (one compiler may have atomic-like behaviour for volatile while another may not) AND hardware dependent (arm vs x86) (3.) the statement "volatile is NEVER correct/useful in multithreaded code" is wrong because there exist at least one counter example

[supercat]

@Voo: There are many platforms where it's possible to cheaply ensure that a write to a volatile write-once variable will eventually be seen on all other threads (within a few seconds); indeed, optimization could be assisted if there were a qualifier which were looser than volatile so that a compiler would be free to optimize out a bounded number of consecutive accesses [e.g. if a compiler unrolls a loop 8 times, it would only have to check the variable once in the unrolled loop, rather than checking eight times]. The overhead imposed by a mechanism with loose semantics...

[supercat]

...could be much less than would be necessary when using a mechanism with more rigid semantics.

[Voo]

@supercat which is exactly what std::atomic does with the memory models weaker than acquire/release.

[Voo]

@Trevor So you also think that "referencing one past the end of an array" is not always wrong because there's one example where it "works"? That's not how c works and particularly dangerous thinking if concurrency is involved.

[Johann Gerell]

@Voo if by "referencing one past the end of an array" you mean "point one past the end of an array", then you're sloppy with terminology as that doesn't reference the memory it points at.

[Voo]

@Johann Are you just nitpicking or really confused? I would have thought any C++ programmer would know that accessing memory outside an array is undefined behavior, but still works often enough - which is the whole point of the argument there. But yes I lost "memory" there and no I didn't mean pointing one past the end of an array because that's not undefined behavior.

[Johann Gerell]

@Voo No, I genuinely did not know if you meant pointing at when you said referencing - I don't consider it nitpicking to point (sic!) out when those two terms are misused, since they are so completely different. Now that you clarified that you meant that a memory access by referencing "one past the array end" can seemingly succeed, I perfectly understand your statement.

[Johann Gerell]

@Voo Just to clarify why I replied in the first place; you wrote "I would have thought any C++ programmer would know that accessing memory outside an array is undefined behavior" but I would argue that the vast majority of C++ developers don't know what UB means and don't know the difference between point at and reference. That majority is like black matter and unlikely to be found on SO or conferences or reading C++ blogs - they are still the majority.

[David Schwartz]

@jalf That is not true. There is no requirement that volatile prevent CPU reordering and on most modern platforms, it does not actually do so.

[underscore_d]

@DavidSchwartz Do you just go around posting false things about volatile? Do you have a different Standard than these people? stackoverflow.com/questions/2535148/…

[David Schwartz]

@underscore_d Read the comments to the two answers that appear to disagree with me and you'll see that they do not. You will not find accesses to volatiles emitting memory barriers, so reordering is not prevented.

[underscore_d]

@DavidSchwartz We mean reordering of reads/writes to that specific volatile, which is guaranteed not to occur - not reordering of any said read/write relative to other statements (whether or not they're also volatile), which is allowed to happen. Is the latter what you mean?

[David Schwartz]

@underscore_d Again, you will see that on pretty much every compiler, reads and writes to volatile variables do not emit memory barriers, even on platforms that require them to prevent reads and writes from being reordered and/or coalesced. Try it. (Do you agree that the CPU can coalesce writes to ordinary memory? Do you see the compiler doing anything to stop it when two writes occur to the same volatile? What about writes to two adjacent volatile shorts, or an "ABA" write to them? Do you see anything emitted to stop write combining?)

[curiousguy]

"Do not use volatile except in low-level code that deals directly with hardware." Strongly disagree. Volatile is useful of signal handlers, for some cases of multithread code, for testing... It just isn't a replacement for atomics in most cases.

[David Schwartz]

What does this have to do with volatile? Yes, this code is UB -- but it's UB with volatile as well.

[curiousguy]

@DavidSchwartz "Do you agree that the CPU can coalesce writes to ordinary memory?" Who cares?

[David Schwartz]

@curiousguy Well, because if it can coalesce two writes, it can't possibly preserve the ordering of them. The point is that there is absolutely no restrictions on the CPUs ability to reorder, modify, or merge the writes. Nothing imposes any such restriction and actual CPUs do that.

[ctrl-alt-delor]

Should not the C compiler, issue boundary instructions between volatile read/writes, to ensure that volatile is honoured?, Or are there CPUs that do not allow volatile to be implemented?

[curiousguy]

@DavidSchwartz What is the definition of "coalesce two writes"?

[David Schwartz]

@curiousguy The code says to make two or three writes and the CPU instead does one or two. For example, you do i=1; j=2; k=3; and the CPU does i=1; k=3; in a single, atomic operation. Nothing prevents a CPU from doing this even if i, j, and k are volatile and real CPUs do this. CPUs can even optimize out writes to volatiles and real-world CPUs actually do this. So i=1; j=2; i=3; can result in the i=1; write getting optimized out, again, even if i is volatile. This is why many CPUs have memory barrier operations and volatile doesn't use them nor is it required to.

[curiousguy]

Indeed with all variables starting with 0, if i=1; j=2; i=3; is turned into j=2; i=3; that would break another thread expecting (incorrectly) that j==2 can only happen if i>0.

[Maxim Egorushkin]

@curiousguy Unix self-pipe trick is more useful in signal handlers than volatile and allows for much more robust code.

[Peter Cordes]

@ctrl-alt-delor: That's not what volatile's "no reordering" means. You're hoping it means that the stores will become globally visible (to other threads) in program order. That's what atomic<T> with memory_order_release or seq_cst gives you. But volatile only gives you a guarantee of no compile-time reordering: each access will appear in the asm in program order. Useful for a device driver. And useful for interaction with an interrupt handler, debugger, or signal handler on the current core/thread, but not for interacting with other cores.

[Peter Cordes]

volatile in practice is sufficient for checking a keep_running flag like you're doing here: Real CPUs always have coherent caches that don't require manual flushing. But there's no reason to recommend volatile over atomic<T> with mo_relaxed; you'll get the same asm.

[Peter Cordes]

@Voo, re your earlier comment about reading stale values from cache: Everything uses coherent data caches (MESI) software.rajivprab.com/2018/04/29/…; an asm store in one core will quickly be seen by asm loads in other cores. volatile does work in practice for a keep_running flag that doesn't need ordering wrt. anything else, but atomic<bool> with mo_relaxed will give you the same asm as volatile and doesn't have UB.

[Peter Cordes]

@Voo: posted my own answer, and updated this one with a disclaimer. This comment thread seems to be a mess, with some people arguing against volatile for because it gives no ordering even though the use case in this answer doesn't need any. (Assuming it's a keep_running flag, not a data_ready flag.)

[Voo]

@Peter "Everything uses coherent data caches ", everything? I mean sure your run of the mill x64 CPU will, but every ISA ever invented now and in the future? There's some very weird architectures out there. I can't imagine that all those DSPs, or mixed CPU SOCs all guarantee anything close to MESI concurrency.

[Peter Cordes]

@Voo: I don't just mean x86-64. I mean PowerPC, MIPS, ARM, RISC-V, etc. etc. All SMP systems that a C++ implementation would want to start std::threads across, or that can run a single system image kernel. Mixed SOCs with non-coherent shared memory apparently exist, but not AFAIK as compiler targets for ISO C++ compliant compilers. (Discussion in comments about them, which I did link in my answer on this question.)

[Voo]

@Peter But that's my point: Now we already weakened an already weak claim to "no compiler I currently know exists for these architectures I didn't know 4 hours ago existed". But why go through all this trouble when there's a simple solution that is actually guaranteed to work perfectly fine? (Granted I'm reasonably sure all those DSPs do break several rules of the standard, but do all future ones also?)

[Peter Cordes]

But why go through all this trouble when there's a simple solution that is actually guaranteed to work perfectly fine? FFS, why do people keep assuming I'm advocating for the use of volatile??? This is really frustrating. I'm trying to promote understanding of caches in mainstream CPUs so people can make correct decisions about performance when using atomic<T> correctly (perhaps with mo_relaxed). Did you read my edit to this answer? Or any of the bolded or ##Heading text in my answer? They're all carefully worded to say don't actually use volatile, but here's how CPUs work.

[Samuel Liew]

Comments are not for extended discussion; this conversation has been moved to chat.

[curiousguy]

@Voo You are missing the point. It isn't that weird CPU don't exist, or uncommon ways to make common CPU to work on the same motherboard. It's that no sane thread implementation strategy is imaginable on these. Any C or C++ compilers tries to have predictable performance for at least common code patterns. That you can emulate the Turing machine on some CPU does not mean you can have a reasonable compiler. You are like ppl arguing that vtables are not std. All compilers use them.

[Voo]

@curiousguy "It's that no sane thread implementation strategy is imaginable on these". You are aware that those weird CPUs do having threading implementations (just not standard conform ones) right? Lack of imagination is not the same as impossible.

[curiousguy]

@Voo Of course weird CPU might have some extremely weird threads. But it doesn't matter from the POV of portable code as these special compilers don't try to support normal code, need specific barriers and specific synchronization devices, etc. Any system where you need to purge caches, at great cost, to get memory visibility isn't going to run portable MT code. Code is going to be written specifically under those constraints.

[curiousguy]

Just imagine a shared_ptr on such system w/o coherent caches. On any destruction of an instance where count>1 you would have to write your modified cache content to main memory, and when count==1 you even need to write back the cache and then purge it to get fresh data? Could you even have dynamic allocation on a shared domain on that system?

[bernie]

Great write-up. This is exactly what I was looking for (giving all the facts) instead of a blanket statement that just says "use atomic instead of volatile for a single global shared boolean flag".

[Peter Cordes]

@bernie: I wrote this after getting frustrated by repeated claims that not using atomic could lead to different threads having different values for the same variable in cache. /facepalm. In cache, no, in CPU registers yes (with non-atomic variables); CPUs use coherent cache. I wish other questions on SO weren't full of explanations for atomic that spread misconceptions about how CPUs work. (Because that's a useful thing to understand for performance reasons, and also helps to explain why the ISO C++ atomic rules are written as they are.)

[Daniel Nitzan]

@PeterCordes With the default seq_cst ordering you'd get from while(!flag), it also adds ordering guarantees wrt. non-atomic accesses are you saying that mo_seq_cst forbids reordering of non-mo_seq_cst around mo_seq_cst?

[Peter Cordes]

@DanielNitzan: yes, a seq_cst load can synchronize-with a release or seq-cst store in another thread, so any loads in the source after that spin-wait had better be after it in the asm as well. Because ISO C++ says it's safe to read non-atomic variables that were written before that release-store (as long as they aren't still being written by other later stores). It's not a 2-way barrier, though; in theory a seq_cst load could happen earlier than it appears in source order. In practice IDK if gcc/clang will combine earlier accesses with later across a seq_cst load. (rough descriptions...)

[Daniel Nitzan]

@PeterCordes oh, right, seq_cst operations have strel/ldacq semantics.

[Daniel Nitzan]

@PeterCordes And an acquire-load means ordering access to cache in the other core; here you claimed that the LoadLoad part of the read-acquire semantics is guaranteed by the memory order machine clear. This is cheaper than blindly ordering load accesses to the cache.

[Daniel Nitzan]

@PeterCordes A memory barrier instruction just blocks the current thread's loads and/or stores until the store buffer drains; should be a full memory barrier? or at least specifically a StoreLoad barrier?

[Daniel Nitzan]

What a full barrier / fence does is stall the current thread until the store buffer is drained Not a complete stall, OoO execution can still proceed, i.e. arithmetic instructions on registers, speculative execution of stores and loads etc

[Peter Cordes]

@DanielNitzan: yes, sometimes I simplify to get the basic concept across, glossing over details like that. Note that MFENCE on Skylake unfortunately does stall the whole thread, as an implementation detail post microcode update. Apparently Intel wanted MFENCE to order NT loads from WC memory, but it previously didn't, and the only way they could do that with just a microcode update was to basically add LFENCE to it. But fortunately they didn't slow down locked instructions, so they're the preferred barrier.

[Peter Cordes]

@DanielNitzan: "blocks the current thread's loads and/or stores until the store buffer drains" - you're right that doesn't apply to a LoadLoad or LoadStore barrier. For StoreStore, it blocks commit of younger stores until older stores have committed, i.e the order in which the store buffer drains. That's effectively like putting a divider on a grocery-store conveyor belt, and does amount to draining the SB before any younger stores (can become visible). Anyway, edited to fix that, thanks; that was specific enough to actually be a problem.

[Emmef]

What about non-cache coherent architectures like Intel SCC (Single Chip Cloud)? I can imagine that a language running on such a platform will stick to the strictest and minimal guarantees that are allowed in its memory model. The guarantees provided by (cross-thread) cache coherence are not a part of that memory model. That would mean that relying on caching coherence is strictly writing non-portable code. Please tell me where am I wrong (the same might apply for C++ code that runs as bytecode on a JVM; need research here)

[Peter Cordes]

@Emmef: You don't run threads of a single process (instance of a C++ program) across the cores of such a machine. Instead, each coherency-domain runs a separate instance of the program, with message passing such as MPI potentially using shared memory with explicit flushing of those regions, like I mentioned in my answer. As the SCC wikipedia says, "you get a functional processor that is fast ... with a framework resembling a network of cloud computers". Notably not resembling an SMP system.

[Peter Cordes]

@Emmef: I think this answer does mention the hypothetical possibility of a C++ implementation that does run std::thread across cores that aren't cache-coherent. It's not impossible, the data-race UB rules make that possible as long as every release operation does explicit flushing on all previous non-atomic stores, or some more sophisticated mechanism to just sync the actually-shared data. That was the point of the "In theory: C++ threads on hardware without coherent caches" section of my answer, that it's unlikely but possible (and would break any assumptions based on cache coherence).

[Peter Cordes]

@Emmef: If you have a suggestion to make that part of my answer clearer, let me know.