Authorization header is lost on redirect

Question

Below is the code that does authentication, generates the Authorization header, and calls the API.

Unfortunately, I get a 401 Unauthorized error following the GET request on the API.

However, when I capture the traffic in Fiddler and replay it, the call to the API is successful and I can see the desired 200 OK status code.

[Test]
public void RedirectTest()
{
    HttpResponseMessage response;
    var client = new HttpClient();
    using (var authString = new StringContent(@"{username: ""theUser"", password: ""password""}", Encoding.UTF8, "application/json"))
    {
        response = client.PostAsync("http://host/api/authenticate", authString).Result;
    }

    string result = response.Content.ReadAsStringAsync().Result;
    var authorization = JsonConvert.DeserializeObject<CustomAutorization>(result);
    client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(authorization.Scheme, authorization.Token);
    client.DefaultRequestHeaders.Add("Accept", "application/vnd.host+json;version=1");

    response =
        client.GetAsync("http://host/api/getSomething").Result;
    Assert.True(response.StatusCode == HttpStatusCode.OK);
}

When I run this code the Authorization header is lost.

However, in Fiddler that header is passed successfully.

Any idea what I'm doing wrong?

Solution 1

The reason you are experiencing this behavior is that it is by design.

Most HTTP clients (by default) strip out authorization headers when following a redirect.

One reason is security. The client could be redirected to an untrusted third party server, one that you would not want to disclose your authorization token to.

What you can do is detect that the redirect has occurred and reissue the request directly to the correct location.

Your API is returning 401 Unauthorized to indicate that the authorization header is missing (or incomplete). I will assume that the same API returns 403 Forbidden if the authorization information is present in the request but is simply incorrect (wrong username / password).

If this is the case, you can detect the 'redirect / missing authorization header' combination and resend the request.

---

Here is the code from the question rewritten to do this:

[Test]
public void RedirectTest()
{
    // These lines are not relevant to the problem, but are included for completeness.
    HttpResponseMessage response;
    var client = new HttpClient();
    using (var authString = new StringContent(@"{username: ""theUser"", password: ""password""}", Encoding.UTF8, "application/json"))
    {
        response = client.PostAsync("http://host/api/authenticate", authString).Result;
    }

    string result = response.Content.ReadAsStringAsync().Result;
    var authorization = JsonConvert.DeserializeObject<CustomAutorization>(result);

    // Relevant from this point on.
    client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(authorization.Scheme, authorization.Token);
    client.DefaultRequestHeaders.Add("Accept", "application/vnd.host+json;version=1");

    var requestUri = new Uri("http://host/api/getSomething");
    response = client.GetAsync(requestUri).Result;

    if (response.StatusCode == HttpStatusCode.Unauthorized)
    {
        // Authorization header has been set, but the server reports that it is missing.
        // It was probably stripped out due to a redirect.

        var finalRequestUri = response.RequestMessage.RequestUri; // contains the final location after following the redirect.

        if (finalRequestUri != requestUri) // detect that a redirect actually did occur.
        {
            if (IsHostTrusted(finalRequestUri)) // check that we can trust the host we were redirected to.
            {
               response = client.GetAsync(finalRequestUri).Result; // Reissue the request. The DefaultRequestHeaders configured on the client will be used, so we don't have to set them again.
            }
        }
    }

    Assert.True(response.StatusCode == HttpStatusCode.OK);
}


private bool IsHostTrusted(Uri uri)
{
    // Do whatever checks you need to do here
    // to make sure that the host
    // is trusted and you are happy to send it
    // your authorization token.

    if (uri.Host == "host")
    {
        return true;
    }

    return false;
}

---

Note that you could save the value of finalRequestUri and use it for future requests to avoid the extra request involved in the retry. However as this is a temporary redirect you should probably issue the request to the original location each time.

Solution 2

I would turn off the automatic redirect behavior and create a client hander that hides the code dealing with the temporary redirect. The HttpClient class allows you to install DelegatingHandlers from which you can modify the request of response.

public class TemporaryRedirectHandler : DelegatingHandler
{
    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        var response = await base.SendAsync(request, cancellationToken);
        if (response.StatusCode == HttpStatusCode.TemporaryRedirect)
        {
            var location = response.Headers.Location;
            if (location == null)
            {
                return response;
            }

            using (var clone = await CloneRequest(request, location))
            {
                response = await base.SendAsync(clone, cancellationToken);
            }
        }
        return response;
    }


    private async Task<HttpRequestMessage> CloneRequest(HttpRequestMessage request, Uri location)
    {
        var clone = new HttpRequestMessage(request.Method, location);

        if (request.Content != null)
        {
            clone.Content = await CloneContent(request);
            if (request.Content.Headers != null)
            {
                CloneHeaders(clone, request);
            }
        }

        clone.Version = request.Version;
        CloneProperties(clone, request);
        CloneKeyValuePairs(clone, request);
        return clone;
    }

    private async Task<StreamContent> CloneContent(HttpRequestMessage request)
    {
        var memstrm = new MemoryStream();
        await request.Content.CopyToAsync(memstrm).ConfigureAwait(false);
        memstrm.Position = 0;
        return new StreamContent(memstrm);
    }

    private void CloneHeaders(HttpRequestMessage clone, HttpRequestMessage request)
    {
        foreach (var header in request.Content.Headers)
        {
            clone.Content.Headers.Add(header.Key, header.Value);
        }
    }

    private void CloneProperties(HttpRequestMessage clone, HttpRequestMessage request)
    {
        foreach (KeyValuePair<string, object> prop in request.Properties)
        {
            clone.Properties.Add(prop);
        }
    }

    private void CloneKeyValuePairs(HttpRequestMessage clone, HttpRequestMessage request)
    {
        foreach (KeyValuePair<string, IEnumerable<string>> header in request.Headers)
        {
            clone.Headers.TryAddWithoutValidation(header.Key, header.Value);
        }
    }
}

You would instantiate the HttpClient like this:

var handler = new TemporaryRedirectHandler()
{
    InnerHandler = new HttpClientHandler()
    {
        AllowAutoRedirect = false
    }
};

HttpClient client = new HttpClient(handler);

Comments

[MvdD]

This solves a different problem though (token refresh).

[Mark Seemann]

@MvdD That depends on how you implement refreshAuth.

[Mark Seemann]

Why would you turn off automatic redirects?

[MvdD]

@MarkSeemann So I can handle them myself in the client handler I install.

[MvdD]

Sure, but typically you do not want to refresh the token unless it is expired. In the OP case, the token wasn't expired, but omitted from the redirected request.

[MvdD]

Plus, you are still making a call to the redirected location that will fail because of a missing Authorization header. Seems wasteful.

[Foole]

Another potential problem here is that HttpRequestMessage can't be reused. Is response.RequestMessage usable?

[Mark Seemann]

@Foole It works in the ad-hoc tests I've made so far... but it's possible that cloning the message will turn out to be necessary.

[Michael Welch]

Hey @MvdD, you didn't happen to wrap up your class in a NuGet Package did you? Mind if I do? I need to tweak it just a little (I want to make sure the host names of the original request and the redirect match in my use case) and then make it available to my customers.

[Khawaja Asim]

Ah you explained it beautifully, just one thing. what if when the Authorization token is wrong it will send the call again. so double call for unauthorized users.

[Christian Findlay]

This seems to be a massive flaw in the way HttpClient works. Obviously it should strip sensitive data from redirects, but the fact that it redirects without telling you is a massive security flaw anyway. What if you send some other sensitive header with the assumption that HttpClient won't just ferry that to some other server? But, by the same token, are we supposed to write retry code like this all over the place? HttpClient needs a redirect callback method so we've got a chance to add/remove headers as needed.

[Jeanno]

Thanks so much for this, i had to change the if statement in SendAsync response.StatusCode == HttpStatusCode.TemporaryRedirect || response.StatusCode == HttpStatusCode.Found , but worked great otherwise

[Mike]

@MelbourneDeveloper - Yes, I agree. It strips the header when a site redirects from http to https, even though the rest of the url matches (ie http://www.test.com to https://www.test.com. However, it will call other sites that you don't trust(?)

[Sergei Sirik]

Anybody knows in which version of java this default behavior was introduced? Seems like on 1.8.0_231 it behaves like you describe, but on 1.8.0_92 it doesn't drop Authorization header after redirect.