If async-await doesn't create any additional threads, then how does it make applications responsive?

Question

Time and time again, I see it said that using async-await doesn't create any additional threads. That doesn't make sense because the only ways that a computer can appear to be doing more than 1 thing at a time is

• Actually doing more than 1 thing at a time (executing in parallel, making use of multiple processors)
• Simulating it by scheduling tasks and switching between them (do a little bit of A, a little bit of B, a little bit of A, etc.)

So if async-await does neither of those, then how can it make an application responsive? If there is only 1 thread, then calling any method means waiting for the method to complete before doing anything else, and the methods inside that method have to wait for the result before proceeding, and so forth.

Solution 1

Actually, async/await is not that magical. The full topic is quite broad but for a quick yet complete enough answer to your question I think we can manage.

Let's tackle a simple button click event in a Windows Forms application:

public async void button1_Click(object sender, EventArgs e)
{
    Console.WriteLine("before awaiting");
    await GetSomethingAsync();
    Console.WriteLine("after awaiting");
}

I'm going to explicitly not talk about whatever it is GetSomethingAsync is returning for now. Let's just say this is something that will complete after, say, 2 seconds.

In a traditional, non-asynchronous, world, your button click event handler would look something like this:

public void button1_Click(object sender, EventArgs e)
{
    Console.WriteLine("before waiting");
    DoSomethingThatTakes2Seconds();
    Console.WriteLine("after waiting");
}

When you click the button in the form, the application will appear to freeze for around 2 seconds, while we wait for this method to complete. What happens is that the "message pump", basically a loop, is blocked.

This loop continuously asks windows "Has anyone done something, like moved the mouse, clicked on something? Do I need to repaint something? If so, tell me!" and then processes that "something". This loop got a message that the user clicked on "button1" (or the equivalent type of message from Windows), and ended up calling our button1_Click method above. Until this method returns, this loop is now stuck waiting. This takes 2 seconds and during this, no messages are being processed.

Most things that deal with windows are done using messages, which means that if the message loop stops pumping messages, even for just a second, it is quickly noticeable by the user. For instance, if you move notepad or any other program on top of your own program, and then away again, a flurry of paint messages are sent to your program indicating which region of the window that now suddenly became visible again. If the message loop that processes these messages is waiting for something, blocked, then no painting is done.

So, if in the first example, async/await doesn't create new threads, how does it do it?

Well, what happens is that your method is split into two. This is one of those broad topic type of things so I won't go into too much detail but suffice to say the method is split into these two things:

1. All the code leading up to await, including the call to GetSomethingAsync
2. All the code following await

Illustration:

code... code... code... await X(); ... code... code... code...

Rearranged:

code... code... code... var x = X(); await X; code... code... code...
^                                  ^          ^                     ^
+---- portion 1 -------------------+          +---- portion 2 ------+

Basically the method executes like this:

1. It executes everything up to await

2. It calls the GetSomethingAsync method, which does its thing, and returns something that will complete 2 seconds in the future

   So far we're still inside the original call to button1_Click, happening on the main thread, called from the message loop. If the code leading up to await takes a lot of time, the UI will still freeze. In our example, not so much

3. What the await keyword, together with some clever compiler magic, does is that it basically something like "Ok, you know what, I'm going to simply return from the button click event handler here. When you (as in, the thing we're waiting for) get around to completing, let me know because I still have some code left to execute".

   Actually it will let the SynchronizationContext class know that it is done, which, depending on the actual synchronization context that is in play right now, will queue up for execution. The context class used in a Windows Forms program will queue it using the queue that the message loop is pumping.

4. So it returns back to the message loop, which is now free to continue pumping messages, like moving the window, resizing it, or clicking other buttons.

   For the user, the UI is now responsive again, processing other button clicks, resizing and most importantly, redrawing, so it doesn't appear to freeze.

5. 2 seconds later, the thing we're waiting for completes and what happens now is that it (well, the synchronization context) places a message into the queue that the message loop is looking at, saying "Hey, I got some more code for you to execute", and this code is all the code after the await.
6. When the message loop gets to that message, it will basically "re-enter" that method where it left off, just after await and continue executing the rest of the method. Note that this code is again called from the message loop so if this code happens to do something lengthy without using async/await properly, it will again block the message loop

There are many moving parts under the hood here so here are some links to more information, I was going to say "should you need it", but this topic is quite broad and it is fairly important to know some of those moving parts. Invariably you're going to understand that async/await is still a leaky concept. Some of the underlying limitations and problems still leak up into the surrounding code, and if they don't, you usually end up having to debug an application that breaks randomly for seemingly no good reason.

• Asynchronous Programming with Async and Await (C# and Visual Basic)
• SynchronizationContext Class
• Stephen Cleary - There is no thread well worth a read!
• Channel 9 - Mads Torgersen: Inside C# Async well worth a watch!

---

OK, so what if GetSomethingAsync spins up a thread that will complete in 2 seconds? Yes, then obviously there is a new thread in play. This thread, however, is not because of the async-ness of this method, it is because the programmer of this method chose a thread to implement asynchronous code. Almost all asynchronous I/O don't use a thread, they use different things. async/await by themselves do not spin up new threads but obviously the "things we wait for" may be implemented using threads.

There are many things in .NET that do not necessarily spin up a thread on their own but are still asynchronous:

• Web requests (and many other network related things that takes time)
• Asynchronous file reading and writing
• and many more, a good sign is if the class/interface in question has methods named SomethingSomethingAsync or BeginSomething and EndSomething and there's an IAsyncResult involved.

Usually these things do not use a thread under the hood.

---

OK, so you want some of that "broad topic stuff"?

Well, let's ask Try Roslyn about our button click:

Try Roslyn

I'm not going to link in the full generated class here but it's pretty gory stuff.

Solution 2

I explain it in full in my blog post There Is No Thread.

In summary, modern I/O systems make heavy use of DMA (Direct Memory Access). There are special, dedicated processors on network cards, video cards, HDD controllers, serial/parallel ports, etc. These processors have direct access to the memory bus, and handle reading/writing completely independently of the CPU. The CPU just needs to notify the device of the location in memory containing the data, and then can do its own thing until the device raises an interrupt notifying the CPU that the read/write is complete.

Once the operation is in flight, there is no work for the CPU to do, and thus no thread.

Solution 3

the only ways that a computer can appear to be doing more than 1 thing at a time is (1) Actually doing more than 1 thing at a time, (2) simulating it by scheduling tasks and switching between them. So if async-await does neither of those

It's not that await does neither of those. Remember, the purpose of await is not to make synchronous code magically asynchronous. It's to enable using the same techniques we use for writing synchronous code when calling into asynchronous code. Await is about making the code that uses high latency operations look like code that uses low latency operations. Those high latency operations might be on threads, they might be on special purpose hardware, they might be tearing their work up into little pieces and putting it in the message queue for processing by the UI thread later. They're doing something to achieve asynchrony, but they are the ones that are doing it. Await just lets you take advantage of that asynchrony.

Also, I think you are missing a third option. We old people -- kids today with their rap music should get off my lawn, etc -- remember the world of Windows in the early 1990s. There were no multi-CPU machines and no thread schedulers. You wanted to run two Windows apps at the same time, you had to yield. Multitasking was cooperative. The OS tells a process that it gets to run, and if it is ill-behaved, it starves all the other processes from being served. It runs until it yields, and somehow it has to know how to pick up where it left off the next time the OS hands control back to it. Single-threaded asynchronous code is a lot like that, with "await" instead of "yield". Awaiting means "I'm going to remember where I left off here, and let someone else run for a while; call me back when the task I'm waiting on is complete, and I'll pick up where I left off." I think you can see how that makes apps more responsive, just as it did in the Windows 3 days.

calling any method means waiting for the method to complete

There is the key that you are missing. A method can return before its work is complete. That is the essence of asynchrony right there. A method returns, it returns a task that means "this work is in progress; tell me what to do when it is complete". The work of the method is not done, even though it has returned.

Before the await operator, you had to write code that looked like spaghetti threaded through swiss cheese to deal with the fact that we have work to do after completion, but with the return and the completion desynchronized. Await allows you to write code that looks like the return and the completion are synchronized, without them actually being synchronized.

Solution 4

I am really glad someone asked this question, because for the longest time I also believed threads were necessary to concurrency. When I first saw event loops, I thought they were a lie. I thought to myself "there's no way this code can be concurrent if it runs in a single thread". Keep in mind this is after I already had gone through the struggle of understanding the difference between concurrency and parallelism.

After research of my own, I finally found the missing piece: select(). Specifically, IO multiplexing, implemented by various kernels under different names: select(), poll(), epoll(), kqueue(). These are system calls that, while the implementation details differ, allow you to pass in a set of file descriptors to watch. Then you can make another call that blocks until the one of the watched file descriptors changes.

Thus, one can wait on a set of IO events (the main event loop), handle the first event that completes, and then yield control back to the event loop. Rinse and repeat.

How does this work? Well, the short answer is that it's kernel and hardware-level magic. There are many components in a computer besides the CPU, and these components can work in parallel. The kernel can control these devices and communicate directly with them to receive certain signals.

These IO multiplexing system calls are the fundamental building block of single-threaded event loops like node.js or Tornado. When you await a function, you are watching for a certain event (that function's completion), and then yielding control back to the main event loop. When the event you are watching completes, the function (eventually) picks up from where it left off. Functions that allow you to suspend and resume computation like this are called coroutines.

Solution 5

await and async use Tasks not Threads.

The framework has a pool of threads ready to execute some work in the form of Task objects; submitting a Task to the pool means selecting a free, already existing¹, thread to call the task action method.
Creating a Task is matter of creating a new object, far way faster than creating a new thread.

Given a Task is possible to attach a Continuation to it, it is a new Task object to be executed once the thread ends.

Since async/await use Tasks they don't create a new thread.

---

While interrupt programming technique are used widely in every modern OS, I don't think they are relevant here.
You can have two CPU bonded tasks executing in parallel (interleaved actually) in a single CPU using aysnc/await.
That could not be explained simply with the fact that the OS support queuing IORP.

---

Last time I checked the compiler transformed async methods into DFA, the work is divided into steps, each one terminating with an await instruction.
The await starts its Task and attach it a continuation to execute the next step.

As a concept example, here is a pseudo-code example.
Things are being simplified for the sake of clarity and because I don't remember all the details exactly.

method:
   instr1                  
   instr2
   await task1
   instr3
   instr4
   await task2
   instr5
   return value

It get transformed into something like this

int state = 0;

Task nextStep()
{
  switch (state)
  {
     case 0:
        instr1;
        instr2;
        state = 1;

        task1.addContinuation(nextStep());
        task1.start();

        return task1;

     case 1:
        instr3;
        instr4;
        state = 2;

        task2.addContinuation(nextStep());
        task2.start();

        return task2;

     case 2:
        instr5;
        state = 0;

        task3 = new Task();
        task3.setResult(value);
        task3.setCompleted();

        return task3;
   }
}

method:
   nextStep();

---

¹ Actually a pool can have its task creation policy.

View more solutions