When to use std::async vs std::threads?

Question

Can anybody give a high level intuition about when to use each of them?

References:

• Is it smart to replace boost::thread and boost::mutex with c++11 equivalents?
• When is it a good idea to use std::promise over the other std::thread mechanisms?

Solution 1

It's not really an either-or thing - you can use futures (together with promises) with manually created std::threads. Using std::async is a convenient way to fire off a thread for some asynchronous computation and marshal the result back via a future but std::async is rather limited in the current standard. It will become more useful if the suggested extensions to incorporate some of the ideas from Microsoft's PPL are accepted.

Currently, std::async is probably best suited to handling either very long running computations or long running IO for fairly simple programs. It doesn't guarantee low overhead though (and in fact the way it is specified makes it difficult to implement with a thread pool behind the scenes), so it's not well suited for finer grained workloads. For that you either need to roll your own thread pools using std::thread or use something like Microsoft's PPL or Intel's TBB.

You can also use std::thread for 'traditional' POSIX thread style code written in a more modern and portable way.

Bartosz Milewski discusses some of the limitations of the way std::async is currently specified in his article Async Tasks in C++11: Not Quite There Yet

Solution 2

One simple reason I've found is the case when you want a way to detect (via polling) whether an asynchronous job is done. With std::thread, you have to manage it yourself. With std::async you can query std::future::valid() (or use std::future::wait_for/wait_until(...)) to know when it is done.

Solution 3

One use-case of using std::futrue over std::thread is you want to call a function which returns a value. When you want return value of the function, you can call get() method of future.

std::thread doesn't provide a direct way to get the return value of the function.

Solution 4

I realise it has been 8 years since this question was posed. The C++ concurrency landscape has shifted quite a bit since then. Recently I too had to wander this landscape, wondering which path to take moving forward. I'd like share some of my thoughts and may be get it validated. I would slightly modify the original question to std::async vs thread pool, instead of just std::thread.

Since 2011 I have been heavily using boost::thread_group and boost::asio::io_service for thread pooling and event looping respectively. My every application starts like this:

int noOfCores = boost::thread::hardware_concurrency();
for (int i = 0; i < noOfCores; i++)
{
    _threadPool.create_thread(boost::bind(&pri_queue::run, &_taskQueue));
}

The task queue _taskQueue is of type pri_queue somewhat similar to this boost example, except my run() function waits on io_service.run_one(). Therefore, I also control the priority in which the tasks are executed, by assigning priority while queuing.

After this, I can throw any function (bound with parameters using boost::bind) at this queue using post() for execution, or schedule it with a delay using boost::asio::deadline_timer::async_wait().

Since everything in my framework is event driven, I am comfortable in dividing any functionality into multiple function objects while awaiting the events like in this boost example of async http client. This model is very time tested, has no thread creation cost since every thread is created upfront.

However, C++ standard has been updated 3 times (14, 17, 20) since I adopted this model across all the products in the company. So you could say I am suffering a bit of FOMO, when I look at all the new changes bandied around. Pardon me, after looking at std::async & coroutines, I don't see how they are helping someone already comfortable using io_service + thread pool model like me. It appears more expensive, and I have no control over priority or thread creation, the implementation differs across compilers.

I see that it is making the functions appear synchronous and structured (all pieces in one place), compared to asynchronous functionality spilt into multiple function objects.

For C++ veterans, I would say thread-pooling is better than std::async or even coroutines. Of course, if the application is not event driven, or if you are new to asynchronous programming, std::async would be easier to deal with.

View more solutions

Comments

[Javi]

So in case I want to write efficient code (or threads being created and joined frequently) it is better to stick to classical threads?

[mattnewport]

Well threads being created and joined frequently is what you want to avoid for efficient code :) That's the issue with std::async - it is quite likely to be implemented by firing off a new thread rather than with a thread pool. If you want efficient fine grained threaded workloads you generally want something like a thread pool.

[Ben Voigt]

@mattnewport: Actually, std::async most often does use a threadpool. But it isn't required to.

[mattnewport]

Arguably though Microsoft isn't following the standard by using a threadpool. The article I added a link to at the end of my answer discusses some of the reasons why. Microsoft is pushing extensions to std::future to make it more compatible with a threadpool / task based parallelism approach in a future revision of the standard.

[Javi]

@mattnewport note the "or" in the parenthesis hehe. I am not experienced with thread pools yet but it would be worthy to explore them as well. Thank you guys.

[Mert Mertce]

Here is what Bjarne Stroustroup says about async: ...don't even think of using async() to launch tasks that do I/O, manipulate mutexes, or in other ways interact with other tasks. The idea behind async() is the same as the idea behind the range-for statement: Provide a simple way to handle the simplest, rather common, case and leave the more complex examples to the fully general mechanism.

[Admin]

I think Apple's GCD should also be listed as an option, as it is freely available for other platforms.

[Rom098]

Polling is possible in case you use std::thread with std::packaged_task.

[chmielot]

I find this approach very interesting and I'd be interested in a complete example of that event driven framework. Would you mind sharing the code?

[Sharath]

An example with priority queue would be too big. That is why I described the process, while pointing to various boost examples.