std::thread::join() hangs if called after main() exits when using VS2012 RC

Question

The following example runs successfully (i.e. doesn't hang) if compiled using Clang 3.2 or GCC 4.7 on Ubuntu 12.04, but hangs if I compile using VS11 Beta or VS2012 RC.

#include <iostream>
#include <string>
#include <thread>
#include "boost/thread/thread.hpp"

void SleepFor(int ms) {
  std::this_thread::sleep_for(std::chrono::milliseconds(ms));
}

template<typename T>
class ThreadTest {
 public:
  ThreadTest() : thread_([] { SleepFor(10); }) {}
  ~ThreadTest() {
    std::cout << "About to join\t" << id() << '\n';
    thread_.join();
    std::cout << "Joined\t\t" << id() << '\n';
  }
 private:
  std::string id() const { return typeid(decltype(thread_)).name(); }
  T thread_;
};

int main() {
  static ThreadTest<std::thread> std_test;
  static ThreadTest<boost::thread> boost_test;
//  SleepFor(100);
}

The issue appears to be that std::thread::join() never returns if it is invoked after main has exited. It is blocked at WaitForSingleObject in _Thrd_join defined in cthread.c.

Uncommenting SleepFor(100); at the end of main allows the program to exit properly, as does making std_test non-static. Using boost::thread also avoids the issue.

So I'd like to know if I'm invoking undefined behaviour here (seems unlikely to me), or if I should be filing a bug against VS2012?

Solution 1

Tracing through Fraser's sample code in his connect bug (https://connect.microsoft.com/VisualStudio/feedback/details/747145) with VS2012 RTM seems to show a fairly straightforward case of deadlocking. This likely isn't specific to std::thread - likely _beginthreadex suffers the same fate.

What I see in the debugger is the following:

On the main thread, the main() function has completed, the process cleanup code has acquired a critical section called _EXIT_LOCK1, called the destructor of ThreadTest, and is waiting (indefinitely) on the second thread to exit (via the call to join()).

The second thread's anonymous function completed and is in the thread cleanup code waiting to acquire the _EXIT_LOCK1 critical section. Unfortunately, due to the timing of things (whereby the second thread's anonymous function's lifetime exceeds that of the main() function) the main thread already owns that critical section.

DEADLOCK.

Anything that extends the lifetime of main() such that the second thread can acquire _EXIT_LOCK1 before the main thread avoids the deadlock situation. That's why the uncommenting the sleep in main() results in a clean shutdown.

Alternatively if you remove the static keyword from the ThreadTest local variable, the destructor call is moved up to the end of the main() function (instead of in the process cleanup code) which then blocks until the second thread has exited - avoiding the deadlock situation.

Or you could add a function to ThreadTest that calls join() and call that function at the end of main() - again avoiding the deadlock situation.

Solution 2

I realize this is an old question regarding VS2012, but the bug is still present in VS2013. For those who are stuck on VS2013, perhaps due to Microsoft's refusal to provide upgrade pricing for VS2015, I offer the following analysis and workaround.

The problem is that the mutex (at_thread_exit_mutex) used by _Cnd_do_broadcast_at_thread_exit() is either not yet initialized, or has already been destroyed, depending on the exact circumstances. In the former case, _Cnd_do_broadcast_at_thread_exit() tries to initialize the mutex during shutdown, causing a deadlock. In the latter case, where the mutex has already been destroyed via the atexit stack, the program will crash on the way out.

The solution I found is to explicitly call _Cnd_do_broadcast_at_thread_exit() (which thankfully is declared publicly) early during program startup. This has the effect of creating the mutex before anyone else tries to access it, as well as ensuring that the mutex continues to exist until the last possible moment.

So, to fix the problem, insert the following code at the bottom of a source module, for instance somewhere below main().

#pragma warning(disable:4073) // initializers put in library initialization area
#pragma init_seg(lib)

#if _MSC_VER < 1900
struct VS2013_threading_fix
{
    VS2013_threading_fix()
    {
        _Cnd_do_broadcast_at_thread_exit();
    }
} threading_fix;
#endif

Solution 3

I believe your threads have already been terminated and their resources freed following the termination of your main function and before static destruction. This is the behavior of the VC runtimes dating back to at least VC6.

Do child threads exit when the parent thread terminates

boost thread and process cleanup on windows

Comments

[Fraser]

@Plexico thread_.joinable() does return true.

[TractorPulledPork]

This answer is specific to windows, but I assumed since you are using VS.

[Fraser]

I am using Windows, but I'm not sure your answer is right unfortunately. The main thread is the one which is executing join, so even though the function main has exited, the main thread has not terminated. Thanks anyway.

[TractorPulledPork]

More relevant content: stackoverflow.com/questions/8001989/…

[TractorPulledPork]

Perhaps the code that kills the threads is somewhere in the runtime on the outside of the main call but I can assure you your threads are gone by the time your destructors are called.

[Fraser]

I can pause the execution while the program is hanging and see the stacks for both the main thread trying to complete the join call, and the child thread in _lock of mlock.c, so the threads haven't terminated. Also, your answer doesn't really answer my question. Is this hanging a result of undefined behaviour or a bug in VS2012?

[Fraser]

@CWoods - can you stick the link to my MS Connect bug report into your answer? I'd like to mark yours as correct and delete mine.

[CWoods]

@Frazer - Done! Glad to have helped!

[Florian Winter]

Unbelievable that this problem still exists and affects C++11 threads. I have seen similar behavior with a Windows port of pthreads 10 years ago...

[leetNightshade]

This workaround doesn't work for me in Visual Studio 2013 Ultimate.

[Serge Rogatch]

Yes, the bug still exists in MSVC++2013 for _beginthreadex() too: I've just refactored from C++11 thread to MSVC++ _beginthreadex(), no way. MSDN also says that CreateThread cannot be used for threads which use CRT (and I need FILE* operations): msdn.microsoft.com/en-us/library/windows/desktop/… . There is also no option to refactor to non-static variable, as the threading happens in a logging singleton. So MSVC++ 2013 is broken on threading, and CUDA Toolkit does not yet support MSVC++2015. %(

[Serge Rogatch]

Doesn't work for me either for a singleton active object in a DLL, MSVC++2013 Update 5.

[TonySalimi]

@CWoods the MS link bug report is broken. Do you have an updated link?

[CWoods]

@hsalimi The landing page of the broken link suggests you search the C++ Developer Community Site (developercommunity.visualstudio.com/spaces/62/index.html) with the old ConnectID (747145) but as I type this it seems their search functionality is down unfortunately. You'll have to try it yourself later...

[TonySalimi]

The system is up again, but cannot find the updated link with the ID.

[CWoods]

@hsalimi I'm not having any luck either I'm afraid. The only thing that I can suggest is that you try reaching out directly to Microsoft (it's a long shot). As far as I can remember the issue was never actually fixed in any VS2012 or VS2013 version/patch. I don't think it was until VS2015 when the fix was actually released.

[CWoods]

@jwd Thanks for thinking of using the wayback machine to make the link working again!