ASP.NET Web API OperationCanceledException when browser cancels the request

Question

When a user loads a page, it makes one or more ajax requests, which hit ASP.NET Web API 2 controllers. If the user navigates to another page, before these ajax requests complete, the requests are canceled by the browser. Our ELMAH HttpModule then logs two errors for each canceled request:

Error 1:

System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at System.Web.Http.Controllers.ExceptionFilterResult.<ExecuteAsync>d__0.MoveNext()

Error 2:

System.OperationCanceledException: The operation was canceled.
   at System.Threading.CancellationToken.ThrowIfCancellationRequested()
   at System.Web.Http.WebHost.HttpControllerHandler.<WriteBufferedResponseContentAsync>d__1b.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.WebHost.HttpControllerHandler.<CopyResponseAsync>d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.WebHost.HttpControllerHandler.<ProcessRequestAsyncCore>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.TaskAsyncHelper.EndTask(IAsyncResult ar)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Looking at the stacktrace, I see that the exception is being thrown from here: https://github.com/ASP-NET-MVC/aspnetwebstack/blob/master/src/System.Web.Http.WebHost/HttpControllerHandler.cs#L413

My question is: How can I handle and ignore these exceptions?

It appears to be outside of user code...

Notes:

• I am using ASP.NET Web API 2
• The Web API endpoints are a mix of async and non-async methods.
• No matter where I add error logging, I am unable to catch the exception in user code
  • Global.asax Applicaiton_Error
  • TaskScheduler.UnobservedTaskException
  • ELMAH Error Filtering void ErrorLog_Filtering (https://code.google.com/p/elmah/wiki/ErrorFiltering)

Solution 1

This is a bug in ASP.NET Web API 2 and unfortunately, I don't think there's a workaround that will always succeed. We filed a bug to fix it on our side.

Ultimately, the problem is that we return a cancelled task to ASP.NET in this case, and ASP.NET treats a cancelled task like an unhandled exception (it logs the problem in the Application event log).

In the meantime, you could try something like the code below. It adds a top-level message handler that removes the content when the cancellation token fires. If the response has no content, the bug shouldn't be triggered. There's still a small possibility it could happen, because the client could disconnect right after the message handler checks the cancellation token but before the higher-level Web API code does the same check. But I think it will help in most cases.

David

config.MessageHandlers.Add(new CancelledTaskBugWorkaroundMessageHandler());

class CancelledTaskBugWorkaroundMessageHandler : DelegatingHandler
{
    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        HttpResponseMessage response = await base.SendAsync(request, cancellationToken);

        // Try to suppress response content when the cancellation token has fired; ASP.NET will log to the Application event log if there's content in this case.
        if (cancellationToken.IsCancellationRequested)
        {
            return new HttpResponseMessage(HttpStatusCode.InternalServerError);
        }

        return response;
    }
}

Solution 2

When implementing an exception logger for WebApi, it is recommend to extend the System.Web.Http.ExceptionHandling.ExceptionLogger class rather than creating an ExceptionFilter. The WebApi internals will not call the Log method of ExceptionLoggers for canceled requests (however, exception filters will get them). This is by design.

HttpConfiguration.Services.Add(typeof(IExceptionLogger), myWebApiExceptionLogger); 

Solution 3

Here's an other workaround for this issue. Just add a custom OWIN middleware at the beginning of the OWIN pipeline that catches the OperationCanceledException:

#if !DEBUG
app.Use(async (ctx, next) =>
{
    try
    {
        await next();
    }
    catch (OperationCanceledException)
    {
    }
});
#endif

Solution 4

I have found a bit more details on this error. There are 2 possible exceptions that can happen:

1. OperationCanceledException
2. TaskCanceledException

The first one happens if connection is dropped while your code in controller executes (or possibly some system code around that as well). While the second one happens if the connection is dropped while the execution is inside an attribute (e.g. AuthorizeAttribute).

So the provided workaround helps to mitigate partially the first exception, it does nothing to help with the second. In the latter case the TaskCanceledException occurs during base.SendAsync call itself rather that cancellation token being set to true.

I can see two ways of solving these:

1. Just ignoring both exceptions in global.asax. Then comes the question if it's possible to suddenly ignore something important instead?
2. Doing an additional try/catch in the handler (though it's not bulletproof + there is still possibility that TaskCanceledException that we ignore will be a one we want to log.

---

config.MessageHandlers.Add(new CancelledTaskBugWorkaroundMessageHandler());

class CancelledTaskBugWorkaroundMessageHandler : DelegatingHandler
{
    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        try
        {
            HttpResponseMessage response = await base.SendAsync(request, cancellationToken);

            // Try to suppress response content when the cancellation token has fired; ASP.NET will log to the Application event log if there's content in this case.
            if (cancellationToken.IsCancellationRequested)
            {
                return new HttpResponseMessage(HttpStatusCode.InternalServerError);
            }
        }
        catch (TaskCancellationException)
        {
            // Ignore
        }

        return response;
    }
}

The only way I figured out we can try to pinpoint the wrong exceptions is by checking if stacktrace contains some Asp.Net stuff. Does not seem very robust though.

P.S. This is how I filter these errors out:

private static bool IsAspNetBugException(Exception exception)
{
    return
        (exception is TaskCanceledException || exception is OperationCanceledException) 
        &&
        exception.StackTrace.Contains("System.Web.HttpApplication.ExecuteStep");
}

Solution 5

You could try changing the default TPL task exception handling behavior through web.config:

<configuration> 
    <runtime> 
        <ThrowUnobservedTaskExceptions enabled="true"/> 
    </runtime> 
</configuration>

Then have a static class (with a static constructor) in your web app, which would handle AppDomain.UnhandledException.

However, it appears that this exception is actually getting handled somewhere inside ASP.NET Web API runtime, before you even have a chance to handle it with your code.

In this case, you should be able to catch it as a 1st chance exception, with AppDomain.CurrentDomain.FirstChanceException, here is how. I understand this may not be what you are looking for.

View more solutions

Comments

[Bates Westmoreland]

Neither of those allowed me to handle the exception either.

[noseratio]

@BatesWestmoreland, not even FirstChanceException? Have you tried handling it with a static class which persists across HTTP requests?

[Bates Westmoreland]

The problem I am trying to solve it to catch, and ignore these exceptions. Using AppDomain.UnhandledException or AppDomain.CurrentDomain.FirstChanceException may allow me to inspect the exception, but not catch and ignore. I didn't see a way to mark these exceptions as handled using either of these approaches. Correct me if I am wrong.

[Bates Westmoreland]

In my case, these exceptions occur because the browser cancels the request as the user navigates to a new url.

[Gabriel S.]

I see. In my case, the requests are issued through the WinHTTP API, not from a browser.

[Bates Westmoreland]

As an update, this does capture some of the requests. We still see quite a few in our logs. Thanks for the work-around. Looking forward to a fix.

[Ken Smith]

I used this workaround for quite a while, and it seemed to work great. Recently - I think associated with an upgrade to ASP.NET MVC 5.2 - this error started showing up in my logs again.

[Kiran]

@KenSmith: Do you still see this error after upgrading to 5.2.2?

[KnightFox]

@KiranChalla - I can confirm that the upgrade to 5.2.2 still has these errors.

[Nate Barbettini]

@KnightFox I am also seeing these errors in my application sporadically, even though I have the latest version of Web API 2.

[Ravi]

As a temporary fix and in order to stop business from panicking can I add below code into my ElmahExceptionFilter?? public override void OnException(HttpActionExecutedContext context) { if (null != context.Exception && !(context.Exception is TaskCanceledException || context.Exception is OperationCanceledException)) { Elmah.ErrorSignal.FromCurrentContext().Raise(context.Excepti‌​on); } base.OnException(context); }

[M2012]

Still I'm getting the error. I've used above suggestion, any other clues.

[Mohan Gopi]

I updated ASP.NET MVC 5.2 and i added workaround also still i am getting "Task Cancelled Exception".

[Bill.Zhuang]

after patch this workaround, still meet those exceptions under webapi 5.2.2.

[Shaddy Zeineddine]

@MohanGopi please see my solution below. I ran into issues using the accepted answer and I found out it stops working in later versions of WebAPI

[sepehr]

In my case ,client is retrofit (okhttp) , an android network library , not a browser . Can you guide me to find the root of this problem please?

[seangwright]

When I tried the recommendation above, I still received Exceptions when the request had been canceled even before it was passed to SendAsync (you can simulate this by holding down F5 in the browser on a url that makes requests to your Api. I solved this issue by also adding the if (cancellationToken.IsCancellationRequested) check above the call to SendAsync. Now the exceptions no longer show up when the browser quickly cancels requests.

[NitinSingh]

I was getting this error primarily from the OWIN context and this one targets it better

[JobaDiniz]

Is there a link to the github bug? Anyone knows if this is already fixed in 5.2.6?

[vsevolod]

@JobaDiniz Bug on gihub I believe they are not going to fix it.

[Ilya Chernomordik]

It seems that the problem with this approach is that the error still pops up in the Global.asax error handling... Event though it's not sent to the exception handler

[Ilya Chernomordik]

I have found some more details on when both exception happen and figured out that this workaround only works against one of them. Here are some details: stackoverflow.com/a/51514604/1671558

[Schoof]

In your suggested code, you create the response variable inside the try and return it outside of the try. That can't possible work can it? Also where do you use the IsAspNetBugException?

[Schoof]

Shouldn't the if be before you call the base.SendAsync? This doesn't solve the issue for us at all.

[Ilya Chernomordik]

No, that can't work, it's just has to be declared outside of the try/catch block of course and initialized with something like completed task. It's just an example of the solution that is not bulletproof anyway. As for the other question you use it in the Global.Asax OnError handler. If you don't use that one to log your messages, you don't need to worry anyway. If you do, this is an example of how you filter out "non errors" from system.

[Ciaran Gallagher]

With this in place, will my requests still be processed correctly in case of where a cancellation token is sent?

[Ciaran Gallagher]

I tried the suggestion by @seangwright, but now my code doesn't throw exceptions but also doesn't process the request. Is there any way to ignore the cancellation request?

[jsgoupil]

For those who want to trigger this error to see how it behaves, the best way to do so is the following. Click Pause while debugging. Make a request to your API then abort it. Resume your Web API app. You will get the OperationCanceledException.

[Triynko]

Throwing unobserved task exceptions is a bad idea; there's a reason it's off by default. Every method with an async void signature would throw one, as well as any Tasks at all that get garbage collected without having observed their result. Furthermore, that's not even related to unhandled TaskCancelledExceptions. If a TaskCancelledException is being thrown, the result of the Task (cancelled) has already been observed.