Do asynchronous operations in ASP.NET MVC use a thread from ThreadPool on .NET 4

Question

After this question, it makes me comfortable when using async operations in ASP.NET MVC. So, I wrote two blog posts on that:

• My Take on Task-based Asynchronous Programming in C# 5.0 and ASP.NET MVC Web Applications
• Asynchronous Database Calls With Task-based Asynchronous Programming Model (TAP) in ASP.NET MVC 4

I have too many misunderstandings in my mind about asynchronous operations on ASP.NET MVC.

I always hear this sentence: Application can scale better if operations run asynchronously

And I heard this kind of sentences a lot as well: if you have a huge volume of traffic, you may be better off not performing your queries asynchronously - consuming 2 extra threads to service one request takes resources away from other incoming requests.

I think those two sentences are inconsistent.

I do not have much information about how threadpool works on ASP.NET but I know that threadpool has a limited size for threads. So, the second sentence has to be related to this issue.

And I would like to know if asynchronous operations in ASP.NET MVC uses a thread from ThreadPool on .NET 4?

For example, when we implement a AsyncController, how does the app structures? If I get huge traffic, is it a good idea to implement AsyncController?

Is there anybody out there who can take this black curtain away in front of my eyes and explain me the deal about asynchrony on ASP.NET MVC 3 (NET 4)?

Edit:

I have read this below document nearly hundreds of times and I understand the main deal but still I have confusion because there are too much inconsistent comment out there.

Using an Asynchronous Controller in ASP.NET MVC

Edit:

Let's assume I have controller action like below (not an implementation of AsyncController though):

public ViewResult Index() { 

    Task.Factory.StartNew(() => { 
        //Do an advanced looging here which takes a while
    });

    return View();
}

As you see here, I fire an operation and forget about it. Then, I return immediately without waiting it be completed.

In this case, does this have to use a thread from threadpool? If so, after it completes, what happens to that thread? Does GC comes in and clean up just after it completes?

Edit:

For the @Darin's answer, here is a sample of async code which talks to database:

public class FooController : AsyncController {

    //EF 4.2 DbContext instance
    MyContext _context = new MyContext();

    public void IndexAsync() { 

        AsyncManager.OutstandingOperations.Increment(3);

        Task<IEnumerable<Foo>>.Factory.StartNew(() => { 

           return 
                _context.Foos;
        }).ContinueWith(t => {

            AsyncManager.Parameters["foos"] = t.Result;
            AsyncManager.OutstandingOperations.Decrement();
        });

        Task<IEnumerable<Bars>>.Factory.StartNew(() => { 

           return 
                _context.Bars;
        }).ContinueWith(t => {

            AsyncManager.Parameters["bars"] = t.Result;
            AsyncManager.OutstandingOperations.Decrement();
        });

        Task<IEnumerable<FooBar>>.Factory.StartNew(() => { 

           return 
                _context.FooBars;
        }).ContinueWith(t => {

            AsyncManager.Parameters["foobars"] = t.Result;
            AsyncManager.OutstandingOperations.Decrement();
        });
    }

    public ViewResult IndexCompleted(
        IEnumerable<Foo> foos, 
        IEnumerable<Bar> bars,
        IEnumerable<FooBar> foobars) {

        //Do the regular stuff and return

    }
}

Solution 1

Here's an excellent article I would recommend you reading to better understand asynchronous processing in ASP.NET (which is what asynchronous controllers basically represent).

Let's first consider a standard synchronous action:

public ActionResult Index()
{
    // some processing
    return View();
}

When a request is made to this action a thread is drawn from the thread pool and the body of this action is executed on this thread. So if the processing inside this action is slow you are blocking this thread for the entire processing, so this thread cannot be reused to process other requests. At the end of the request execution, the thread is returned to the thread pool.

Now let's take an example of the asynchronous pattern:

public void IndexAsync()
{
    // perform some processing
}

public ActionResult IndexCompleted(object result)
{
    return View();
}

When a request is sent to the Index action, a thread is drawn from the thread pool and the body of the IndexAsync method is executed. Once the body of this method finishes executing, the thread is returned to the thread pool. Then, using the standard AsyncManager.OutstandingOperations, once you signal the completion of the async operation, another thread is drawn from the thread pool and the body of the IndexCompleted action is executed on it and the result rendered to the client.

So what we can see in this pattern is that a single client HTTP request could be executed by two different threads.

Now the interesting part happens inside the IndexAsync method. If you have a blocking operation inside it, you are totally wasting the whole purpose of the asynchronous controllers because you are blocking the worker thread (remember that the body of this action is executed on a thread drawn from the thread pool).

So when can we take real advantage of asynchronous controllers you might ask?

IMHO we can gain most when we have I/O intensive operations (such as database and network calls to remote services). If you have a CPU intensive operation, asynchronous actions won't bring you much benefit.

So why can we gain benefit from I/O intensive operations? Because we could use I/O Completion Ports. IOCP are extremely powerful because you do not consume any threads or resources on the server during the execution of the entire operation.

How do they work?

Suppose that we want to download the contents of a remote web page using the WebClient.DownloadStringAsync method. You call this method which will register an IOCP within the operating system and return immediately. During the processing of the entire request, no threads are consumed on your server. Everything happens on the remote server. This could take lots of time but you don't care as you are not jeopardizing your worker threads. Once a response is received the IOCP is signaled, a thread is drawn from the thread pool and the callback is executed on this thread. But as you can see, during the entire process, we have not monopolized any threads.

The same stands true with methods such as FileStream.BeginRead, SqlCommand.BeginExecute, ...

What about parallelizing multiple database calls? Suppose that you had a synchronous controller action in which you performed 4 blocking database calls in sequence. It's easy to calculate that if each database call takes 200ms, your controller action will take roughly 800ms to execute.

If you don't need to run those calls sequentially, would parallelizing them improve performance?

That's the big question, which is not easy to answer. Maybe yes, maybe no. It will entirely depend on how you implement those database calls. If you use async controllers and I/O Completion Ports as discussed previously you will boost the performance of this controller action and of other actions as well, as you won't be monopolizing worker threads.

On the other hand if you implement them poorly (with a blocking database call performed on a thread from the thread pool), you will basically lower the total time of execution of this action to roughly 200ms but you would have consumed 4 worker threads so you might have degraded the performance of other requests which might become starving because of missing threads in the pool to process them.

So it is very difficult and if you don't feel ready to perform extensive tests on your application, do not implement asynchronous controllers, as chances are that you will do more damage than benefit. Implement them only if you have a reason to do so: for example you have identified that standard synchronous controller actions are a bottleneck to your application (after performing extensive load tests and measurements of course).

Now let's consider your example:

public ViewResult Index() { 

    Task.Factory.StartNew(() => { 
        //Do an advanced looging here which takes a while
    });

    return View();
}

When a request is received for the Index action a thread is drawn from the thread pool to execute its body, but its body only schedules a new task using TPL. So the action execution ends and the thread is returned to the thread pool. Except that, TPL uses threads from the thread pool to perform their processing. So even if the original thread was returned to the thread pool, you have drawn another thread from this pool to execute the body of the task. So you have jeopardized 2 threads from your precious pool.

Now let's consider the following:

public ViewResult Index() { 

    new Thread(() => { 
        //Do an advanced looging here which takes a while
    }).Start();

    return View();
}

In this case we are manually spawning a thread. In this case the execution of the body of the Index action might take slightly longer (because spawning a new thread is more expensive than drawing one from an existing pool). But the execution of the advanced logging operation will be done on a thread which is not part of the pool. So we are not jeopardizing threads from the pool which remain free for serving another requests.

Solution 2

Yes - all threads come from the thread-pool. Your MVC app is already multi-threaded, when a request comes in a new thread will be taken from the pool and used to service the request. That thread will be 'locked' (from other requests) until the request is fully serviced and completed. If there is no thread available in the pool the request will have to wait until one is available.

If you have async controllers they still get a thread from the pool but while servicing the request they can give up the thread, while waiting for something to happen (and that thread can be given to another request) and when the original request needs a thread again it gets one from the pool.

The difference is that if you have a lot of long-running requests (where the thread is waiting for a response from something) you might run out of threads from the the pool to service even basic requests. If you have async controllers, you don't have any more threads but those threads that are waiting are returned to the pool and can service other requests.

A nearly real life example... Think of it like getting on a bus, there's five people waiting to get on, the first gets on, pays and sits down (the driver serviced their request), you get on (the driver is servicing your request) but you can't find your money; as you fumble in your pockets the driver gives up on you and gets the next two people on (servicing their requests), when you find your money the driver starts dealing with you again (completing your request) - the fifth person has to wait until you are done but the third and fourth people got served while you were half way through getting served. This means that the driver is the one and only thread from the pool and the passengers are the requests. It was too complicated to write how it would work if there was two drivers but you can imagine...

Without an async controller, the passengers behind you would have to wait ages while you looked for your money, meanwhile the bus driver would be doing no work.

So the conclusion is, if lots of people don't know where their money is (i.e. require a long time to respond to something the driver has asked) async controllers could well help throughput of requests, speeding up the process from some. Without an aysnc controller everyone waits until the person in front has been completely dealt with. BUT don't forget that in MVC you have a lot of bus drivers on a single bus so async is not an automatic choice.

Solution 3

There are two concepts at play here. First of all we can make our code run in parallel to execute faster or schedule code on another thread to avoid making the user wait. The example you had

public ViewResult Index() { 

    Task.Factory.StartNew(() => { 
        //Do an advanced looging here which takes a while
    });

    return View();
}

belongs to the second category. The user will get a faster response but the total workload on the server is higher because it has to do the same work + handle the threading.

Another example of this would be:

public ViewResult Index() { 

    Task.Factory.StartNew(() => { 
        //Make async web request to twitter with WebClient.DownloadString()
    });

    Task.Factory.StartNew(() => { 
        //Make async web request to facebook with WebClient.DownloadString()
    });


    //wait for both to be ready and merge the results

    return View();
}

Because the requests run in parallel the user won't have to wait as long as if they where done in serial. But you should realize that we use up more resources here than if we ran in serial because we run the code at many threads while we have on thread waiting too.

This is perfectly fine in a client scenario. And it is quite common there to wrap synchronous long running code in a new task(run it on another thread) too keep the ui responsive or parallize to make it faster. A thread is still used for the whole duration though. On a server with high load this could backfire because you actually use more resources. This is what people have warned you about

Async controllers in MVC has another goal though. The point here is to avoid having threads sittings around doing nothing(which can hurt scalability). It really only matters if the API's you are calling have async methods. Like WebClient.DowloadStringAsync().

The point is that you can let your thread be returned to handle new requests untill the web request is finished where it will call you callback which gets the same or a new thread and finish the request.

I hope you understand the difference between asynchronous and parallel. Think of parallel code as code where your thread sits around and wait for the result. While asynchronous code is code where you will be notified when the code is done and you can get back working at it, in the meantime the thread can do other work.

Solution 4

Applications can scale better if operations run asynchronously, but only if there are resources available to service the additional operations.

Asynchronous operations ensure that you're never blocking an action because an existing one is in progress. ASP.NET has an asynchronous model that allows multiple requests to execute side-by-side. It would be possible to queue the requests up and processes them FIFO, but this would not scale well when you have hundreds of requests queued up and each request takes 100ms to process.

If you have a huge volume of traffic, you may be better off not performing your queries asynchronously, as there may be no additional resources to service the requests. If there are no spare resources, your requests are forced to queue up, take exponentially longer or outright fail, in which case the asynchronous overhead (mutexes and context-switching operations) isn't giving you anything.

As far as ASP.NET goes, you don't have a choice - it's uses an asynchronous model, because that's what makes sense for the server-client model. If you were to be writing your own code internally that uses an async pattern to attempt to scale better, unless you're trying to manage a resource that's shared between all requests, you won't actually see any improvements because they're already wrapped in an asynchronous process that doesn't block anything else.

Ultimately, it's all subjective until you actually look at what's causing a bottleneck in your system. Sometimes it's obvious where an asynchronous pattern will help (by preventing a queued resource blocking). Ultimately only measuring and analysing a system can indicate where you can gain efficiencies.

Edit:

In your example, the Task.Factory.StartNew call will queue up an operation on the .NET thread-pool. The nature of Thread Pool threads is to be re-used (to avoid the cost of creating/destroying lots of threads). Once the operation completes, the thread is released back to the pool to be re-used by another request (the Garbage Collector doesn't actually get involved unless you created some objects in your operations, in which case they're collected as per normal scoping).

As far as ASP.NET is concerned, there is no special operation here. The ASP.NET request completes without respect to the asynchronous task. The only concern might be if your thread pool is saturated (i.e. there are no threads available to service the request right now and the pool's settings don't allow more threads to be created), in which case the request is blocked waiting to start the task until a pool thread becomes available.

Solution 5

Yes, they use a thread from the thread pool. There is actually a pretty excellent guide from MSDN that will tackle all of your questions and more. I have found it to be quite useful in the past. Check it out!

http://msdn.microsoft.com/en-us/library/ee728598.aspx

Meanwhile, the comments + suggestions that you hear about asynchronous code should be taken with a grain of salt. For starters, just making something async doesn't necessarily make it scale better, and in some cases can make your application scale worse. The other comment you posted about "a huge volume of traffic..." is also only correct in certain contexts. It really depends on what your operations are doing, and how they interact with other parts of the system.

In short, lots of people have lots of opinions about async, but they may not be correct out of context. I'd say focus on your exact problems, and do basic performance testing to see what async controllers, etc. actually handle with your application.

View more solutions

Comments

[Chris Chilvers]

Not sure on the answer, but its worth noting asynchronous and multi-threading are different things. So it would be possible to have a fixed number of threads with asynchronous handling. What would happen is when one page has to block for say, I/O, another page would get a chance to run on the same thread. Which is how both of those statements can be true, async can make things faster but too many threads is a problem.

[tugberk]

@ChrisChilvers Yep, multithreading is not always necessary on asynchronous operation. I already figured that but I think I have no controller over that as far as I can tell. AsyncController spin up how many threads it wants from my point of view but not sure on that either. Is there a notion of threadpool on desktop apps like WPF as well? I think number of threads is not a issue on those kind of apps.

[oleksii]

Take a look at video Threading with Jeff Richter

[Chris Chilvers]

I think the problem (and thus inconsistency) is the second statement uses asynchronous when it means many threads. This could be because that is how asp.net has implemented async pages and thus the specific implementation has confused the issue (since the name of the feature that causes the problem would be async pages), but I'm not sure about the specific implemntation. So either they mean "many threads", or they mean "async pages within asp.net version X" since future versions may change the implementation. Or they just mean using the thread pool to perform async within a page.

[tugberk]

@ChrisChilvers oh, man! I am more confused after these comments :s

[tugberk]

@oleksii thanks, I will have a look!

[Chris Chilvers]

What I mean is asynchronous and multi-threading are different things. The second statement is more referring to having lots of threads being a problem rather than asynchronous being a problem, thus the phrase "performing queries asynchronously" is incorrect there. They actually mean in parallel using multiple threads. That's not to say that too many requests (async or not) cause a problem, as the server still has limited memory, i/o and cpu time to service the requests. So having 1,000 threads would probably cause issues, but you can have only a single thread and still be using async.

[tugberk]

I have read that document maybe hundreds of times and still I have so much confusion (maybe the problem is me, who knows). When you look around, you see so much inconsistent comments about asynchrony on ASP.NET MVC as you can see on my question.

[tugberk]

Thanks! After I read your answer, I edited the question with a code sample. Can you have a look?

[tugberk]

for the last sentence: inside a controller action, I was querying a database 5 times separately (I had to) and it was all taking approx 400 ms. Then, I implemented AsyncController and run them in parallel. The response time reduced dramatically to approx. 200 ms. But I have no idea how many thread it creates, what happens to those threads after I am done with them, does GC comes and clean them right after I am done so that my app won't have a memory leak, so on so forth. Any idea on that part.

[tugberk]

You have a magic sentence there for me: Task.Factory.StartNew call will queue up an operation on the .NET thread-pool.. In this context, which one is correct here: 1-) It creates a new thread and when it is done, that thread goes back to threadpool and wait there to be reused again. 2-) It gets a thread from threadpool and that thread goes back to threadpool and wait there to be reused again. 3-) It takes the most efficient approach and can do either of them.

[Paul Turner]

The thread-pool creates threads as they are required and recycles threads when they are not being used. It's exact behaviour has varied across CLR versions. You can find specific information about it here msdn.microsoft.com/en-us/library/0ka9477y.aspx

[tugberk]

It starts to shape up in my mind now. So, CLR owns the thread-pool, right? For instance, a WPF application also has a notion of thread-pool and it is deals with there-pool as well.

[tugberk]

Really detailed, thanks! Let's assume that, we have 4 async Tasks (System.Threading.Task) running inside IndexAsync method. Inside those operations, we are making db calls to a server. So, all of them are I/O intensive operations, right? In that case, do we create 4 separate threads (or gets 4 separate threads from thread-pool)? Assuming I have a multi-core machine they are going to run in parallel as well, right?

[Paul Turner]

The Thread Pool is its own thing within the CLR. Other components which are "aware" of the pool are indicating that they use Thread Pool threads (where appropriate) rather than creating and destroying their own ones. Creating or destroying a thread is a relatively expensive operation, so a using the pool is a big efficiency gain on short-running operations.

[Darin Dimitrov]

@tugberk, database calls are I/O operations, but it will all depend on how you implement them. If you use a blocking database call such as SqlCommand.ExecuteReader you are wasting everything since this is a blocking call. You are blocking the thread on which this call executes and if this thread happens to be a thread from the pool it's very bad. You will benefit only if you use I/O Completion Ports: SqlCommand.BeginExecuteReader. If you don't use IOCP no matter what you do, don't use async controllers as you will do more damage than benefit on the overall performance of your application.

[tugberk]

Well, most of the time I use EF code first. I am not sure if it fits. I put up a sample which shows what I generally do. I updated the question, can you have a look?

[Darin Dimitrov]

@tugberk, what you have shown are blocking calls. They do not use IOCP. Also you seem to execute those calls using TPL so you are basically using threads from the thread pool to execute them. So you are basically in the situation it depends. You are parallelizing your database calls, so you are improving the performance of this particular action but you might be degrading performance of other actions since you are jeopardizing threads from the pool. Might not be the case. As I said in my answer: MEASURE. It's the only way. And if you don't want to bother, then don't use async actions.

[A.R.]

attach a debugger and find out.

[tugberk]

I see. So, basically I am running them in parallel and I assume it has nothing to do with asyncrony?

[Darin Dimitrov]

@tugberk, you are running them in parallel, so the total time of execution is less compared to if you run them sequentially. But in order to run them you use worker threads. Well actually EF is lazy so when you do _context.Foo you are actually not executing anything. You are just building an expression tree. Be extremely careful with this. The query execution is deferred only when you start enumerating over the resultset. And if this happens in the view this might be catastrophic to the performance. To eagerly execute an EF query append .ToList() at the end.

[tugberk]

Good tips! So, to sum it up in one word, I gain response time reduction with the above example code by running them in parallel. Do you approve this approach? Y/N? Do I need to investigate further?

[Darin Dimitrov]

@tugberk, yes, you gain response time in this specific example by running the queries in parallel. But, and it's the but that's more important: you might degrade the overall processing capabilities of your entire application because you are consuming worker threads. And you might not. You will have to measure this. What you have to understand is that asynchronous processing and multithreading is an extremely complex matter. There are no straight answers. If there were it would be simple. So go measure if you have doubts. It's the only way.

[tugberk]

Last one then I will leave you alone:D you have a good word here: MEASURE. I used Mini Profiler to measure the response time. What do you suggest me to use to measure the overall perf? For more specifically, where should I be looking at to measure those kind of perf?

[Darin Dimitrov]

@tugberk, you will need a load testing tool in order to simulate multiple users in parallel on your web site to see how it behaves under heavy load. Mini Profiler cannot simulate load on your website. It might help you see and optimize your ADO.NET queries and profile a single request which is useless when you need to see how your site behaves in a real world situation when lots of users are hitting it.

[Pittsburgh DBA]

Very nice analogy. Thank you.

[Omer Cansizoglu]

I liked the description. Thanks

[Anand Vyas]

Excellent way to explain it.Thank you,

[Christian Davén]

That "excellent article" has been removed from the MSDN site, and can (for now, at least) be found at the Internet Archive.

[Nirman]

Your answer in combination of Darin's answer sums up the whole mechanism behind async controllers, what it is, and more importantly what it is not!

[Tomuke]

This is a nice analogy, but my only question is this: the guy fumbling in his pockets in your analogy would be some kind of work/processing in our application... so what processes that work once we release the thread? Surly it is another thread? So what do we gain here?