Long running task in WebAPI

Question

Here's my problem: I need to call multiple 3rd party methods inside an ApiController. The signature for those methods is Task DoSomethingAsync(SomeClass someData, SomeOtherClass moreData). I want those calls to continue running in the background, after the ApiController has sent the data back to the client. When DoSomethingAsync completes I want to do some logging and maybe save some data to the file system. How can I do that? I'd prefer to use the asyny/await syntax.

Solution 1

Stephen described why starting essentially long running fire-and-forget tasks inside an ApiController is a bad idea.

Perhaps you should create a separate service to execute those fire-and-forget tasks. That service could be a different ApiController, a worker behind a queue, anything that can be hosted on its own and have an independent lifetime.

This would make management of the different task lifetimes much easier and separate the concerns of the long-running tasks from the ApiController's core responsibilities.

Solution 2

Great news, there is a new solution in .NET 4.5.2 called the QueueBackgroundWorkItem API. It's really simple to use:

HostingEnvironment.QueueBackgroundWorkItem(ct => DoSomething(a, b, c));

Here's an article that describes it in detail.

https://blogs.msdn.microsoft.com/webdev/2014/06/04/queuebackgroundworkitem-to-reliably-schedule-and-run-background-processes-in-asp-net/

And here's anohter article that mentions a few other approaches not mentioned in this thread. http://www.hanselman.com/blog/HowToRunBackgroundTasksInASPNET.aspx

Solution 3

You almost never want to do this. It is almost always a big mistake.

ASP.NET (and most other servers) work on the assumption that it's safe to tear down your service once all requests have completed. So you have no guarantee that your logging will be done, or that your data will be written to disk. Particularly with the disk writes, it's entirely possible that your writes will be corrupted.

That said, if you are absolutely sure that you want to implement this extremely dangerous design, you can use the BackgroundTaskManager from my blog.

Update: I've written a blog series that goes into detail on a proper solution for request-extrinsic code. In summary, what you really want to do is move the request-extrinsic code out of ASP.NET. Introduce a durable queue and an independent processor; the ASP.NET controller action will place a request onto the queue, and the independent processor will read requests and execute them. This "processor" can be an Azure Function/WebJob, Win32 Service, etc.

Solution 4

As pointed out by others, it is not recommended. However, whenever there is a need there is a way, so take a look at IRegisteredObject

See also

http://haacked.com/archive/2011/10/16/the-dangers-of-implementing-recurring-background-tasks-in-asp-net.aspx/

Solution 5

Though the question is several years old, best possible solution now is to use Singal R in this case.

https://github.com/Myrmex/signalr-notify-progress

View more solutions

Comments

[RickAndMSFT]

Correct before Azure-WebJobs. Now you can do this with Azure-WebJobs. A worker role is the most robust approach.

[aaddesso]

I find this is an acceptable trade-of for things like: debug-logging, performance recording, event tracking (using an external service). In all of these cases a "fire-and-forget" semantic is acceptable and we don't really care if individual events may get lost in the case of a service tear down. Or is there something I'm missing?

[Stephen Cleary]

@JohannesRudolph: I would say the primary use case is cache updates. It's an acceptable way to do logging, if you accept that your logs may not have all data. I would interpret "event tracking" as a business requirement, so not a good use case for this. If "event tracking" isn't important, then "debug logging", "performance recording", and "event tracking" are just three different kinds of logging. Note that ASP.NET 4.5.2 has something similar built-in now.

[jt000]

Another reason why it is better to have a separate service is to make it easier to scale out. For example, maybe you find that you need more servers to run your long running task, but not more servers to run your website. By splitting these up at the beginning you can better customize your scale-out solution.

[Torben Junker Kjær]

QueueBackgroundWorkItem can be a solution if the long-running task is not really that long-running, i.e. if it can finish in less than 90 seconds. If it really is long-running another solution is needed (like handing it over to a Windows service)

[William T. Mallard]

I couldn't get this to work in a self-hosted WebAPI within a Windows Service (backend data layer).

[Kiran Ahir]

How to implement it?

[Panagiotis Kanavos]

@KiranAhir it's explained in the docs: Background tasks with hosted services in ASP.NET Core. In a web app, you can create a BackgroundService to take care of long tasks. You can also create a separate Web API project with a Background service to receive requests from the front-end web app. There's even a Worker Service template that creates a standalone background service

[Sedat Kapanoglu]

QueueBackgroundWorkItem doesn't exist in .NET Core (.NET 5+), fyi.