Run a background task from a controller action in ASP.NET Core

Question

I am developing a web application with a REST API using C# with ASP.NET Core 2.0.

What I want to achieve is when the client send a request to an endpoint I will run a background task separated from the client request context which will be ended if the task started successfully.

I know there is HostedService but the problem is that the HostedService starts when the server starts, and as far as I know there is no way to start the HostedService manually from a controller.

Here is a simple code that demonstrates the question.

[Authorize(AuthenticationSchemes = "UsersScheme")]
public class UsersController : Controller
{
    [HttpPost]
    public async Task<JsonResult> StartJob([FromForm] string UserId, [FromServices] IBackgroundJobService backgroundService)
    {
        // check user account
        (bool isStarted, string data) result = backgroundService.Start();

        return JsonResult(result);
    }
}

Solution 1

You still can use IHostedService as base for background tasks in combination with BlockingCollection.

Create wrapper for BlockingCollection so you can inject it as singleton.

public class TasksToRun
{
    private readonly BlockingCollection<TaskSettings> _tasks;

    public TasksToRun() => _tasks = new BlockingCollection<TaskSettings>();

    public void Enqueue(TaskSettings settings) => _tasks.Add(settings);

    public TaskSettings Dequeue(CancellationToken token) => _tasks.Take(token);
}

Then in implementation of IHostedService "listen" for tasks and when tasks "arrive" execute it.
BlockingCollection will stop execution if collection is empty - so your while loop will not consume processor time.
.Take method accept cancellationToken as argument. With token you can cancel "waiting" for next task when application stops.

public class BackgroundService : IHostedService
{
    private readonly TasksToRun _tasks;

    private CancellationTokenSource _tokenSource;

    private Task _currentTask;

    public BackgroundService(TasksToRun tasks) => _tasks = tasks;

    public async Task StartAsync(CancellationToken cancellationToken)
    {
        _tokenSource = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
        while (cancellationToken.IsCancellationRequested == false)
        {
            try
            {
                var taskToRun = _tasks.Dequeue(_tokenSource.Token);

                // We need to save executable task, 
                // so we can gratefully wait for it's completion in Stop method
                _currentTask = ExecuteTask(taskToRun);               
                await _currentTask;
            }
            catch (OperationCanceledException)
            {
                // execution cancelled
            }
        }
    }

    public async Task StopAsync(CancellationToken cancellationToken)
    {
        _tokenSource.Cancel(); // cancel "waiting" for task in blocking collection

        if (_currentTask == null) return;

        // wait when _currentTask is complete
        await Task.WhenAny(_currentTask, Task.Delay(-1, cancellationToken));
    }
}

And in the controller you simply add task you want to run to our collection

public class JobController : Controller
{
    private readonly TasksToRun _tasks;

    public JobController(TasksToRun tasks) => _tasks = tasks;

    public IActionResult PostJob()
    {
        var settings = CreateTaskSettings();

        _tasks.Enqueue(settings);

        return Ok();
    }
}

Wrapper for blocking collection should be registered for dependency injection as singleton

services.AddSingleton<TasksToRun, TasksToRun>();

Register background service

services.AddHostedService<BackgroundService>();

Solution 2

Microsoft has documented the same at https://docs.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-2.1

It accomplishes using BackgroundTaskQueue, which gets work assigned from Controller and the work is performed by QueueHostedService which derives from BackgroundService.

Solution 3

This is heavily inspired from the documentation linked in skjagini's answer, with a few improvements.

I figured that it may help to reiterate the entire example here, in case the link breaks at some point. I have made some adjustments; most notably, I inject an IServiceScopeFactory, to allow the background processes to safely request services themselves. I explain my reasoning at the end of this answer.

---

The core idea is creating a task queue, which the user can inject into their controller and then assign tasks to. The same task queue is present in a long-running hosted service, which dequeues one task at a time and executes it.

Task queue:

public interface IBackgroundTaskQueue
{
    // Enqueues the given task.
    void EnqueueTask(Func<IServiceScopeFactory, CancellationToken, Task> task);

    // Dequeues and returns one task. This method blocks until a task becomes available.
    Task<Func<IServiceScopeFactory, CancellationToken, Task>> DequeueAsync(CancellationToken cancellationToken);
}

public class BackgroundTaskQueue : IBackgroundTaskQueue
{
    private readonly ConcurrentQueue<Func<IServiceScopeFactory, CancellationToken, Task>> _items = new();

    // Holds the current count of tasks in the queue.
    private readonly SemaphoreSlim _signal = new SemaphoreSlim(0);

    public void EnqueueTask(Func<IServiceScopeFactory, CancellationToken, Task> task)
    {
        if(task == null)
            throw new ArgumentNullException(nameof(task));

        _items.Enqueue(task);
        _signal.Release();
    }

    public async Task<Func<IServiceScopeFactory, CancellationToken, Task>> DequeueAsync(CancellationToken cancellationToken)
    {
        // Wait for task to become available
        await _signal.WaitAsync(cancellationToken);

        _items.TryDequeue(out var task);
        return task;
    }
}

At the heart of the task queue, we have a thread-safe ConcurrentQueue<>. Since we don't want to poll the queue until a new task becomes available, we use a SemaphoreSlim object to keep track of the current number of tasks in the queue. Each time we call Release, the internal counter is incremented. The WaitAsync method blocks until the internal counter becomes greater than 0, and subsequently decrements it.

For dequeuing and executing the tasks, we create a background service:

public class BackgroundQueueHostedService : BackgroundService
{
    private readonly IBackgroundTaskQueue _taskQueue;
    private readonly IServiceScopeFactory _serviceScopeFactory;
    private readonly ILogger<BackgroundQueueHostedService> _logger;

    public BackgroundQueueHostedService(IBackgroundTaskQueue taskQueue, IServiceScopeFactory serviceScopeFactory, ILogger<BackgroundQueueHostedService> logger)
    {
        _taskQueue = taskQueue ?? throw new ArgumentNullException(nameof(taskQueue));
        _serviceScopeFactory = serviceScopeFactory ?? throw new ArgumentNullException(nameof(serviceScopeFactory));
        _logger = logger ?? throw new ArgumentNullException(nameof(logger));
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        // Dequeue and execute tasks until the application is stopped
        while(!stoppingToken.IsCancellationRequested)
        {
            // Get next task
            // This blocks until a task becomes available
            var task = await _taskQueue.DequeueAsync(stoppingToken);

            try
            {
                // Run task
                await task(_serviceScopeFactory, stoppingToken);
            }
            catch(Exception ex)
            {
                _logger.LogError(ex, "An error occured during execution of a background task");
            }
        }
    }
}

Finally, we need to make our task queue available for dependency injection, and start our background service:

public void ConfigureServices(IServiceCollection services)
{
    // ...
    
    services.AddSingleton<IBackgroundTaskQueue, BackgroundTaskQueue>();
    services.AddHostedService<BackgroundQueueHostedService>();
    
    // ...
}

We can now inject the background task queue into our controller and enqueue tasks:

public class ExampleController : Controller
{
    private readonly IBackgroundTaskQueue _backgroundTaskQueue;

    public ExampleController(IBackgroundTaskQueue backgroundTaskQueue)
    {
        _backgroundTaskQueue = backgroundTaskQueue ?? throw new ArgumentNullException(nameof(backgroundTaskQueue));
    }

    public IActionResult Index()
    {
        _backgroundTaskQueue.EnqueueTask(async (serviceScopeFactory, cancellationToken) =>
        {
            // Get services
            using var scope = serviceScopeFactory.CreateScope();
            var myService = scope.ServiceProvider.GetRequiredService<IMyService>();
            var logger = scope.ServiceProvider.GetRequiredService<ILogger<ExampleController>>();
            
            try
            {
                // Do something expensive
                await myService.DoSomethingAsync(cancellationToken);
            }
            catch(Exception ex)
            {
                logger.LogError(ex, "Could not do something expensive");
            }
        });

        return Ok();
    }
}

---

Why use an IServiceScopeFactory?

In theory, we could directly use the service objects which we have injected into our controller. This will probably work well with singleton services, and also with most scoped services.

However, for scoped services which implement IDisposable (e.g., DbContext), this will likely break: After enqueuing the task, the controller method returns and the request is completed. The framework then cleans up the injected services. If our background task is sufficiently slow or delayed, it may try to call a method of a disposed service, and will then run into an error.

To avoid this, our queued tasks should always create their own service scope, and should not make use of service instances from the surrounding controller.