Force retry on specific http status code

Question

Dealing with a specific website, sometimes I receive a http response with status code 403. I wanted to re-execute the request in such cases (because, in my specific situation, this server throws a 403 when it is actually overloaded). I tried to use a ResponseHandler together with a StandardHttpRequestRetryHandler, but it didn't work the way I hoped; I expected that throwing an exception in the ResponseHandler would trigger the StandardHttpRequestRetryHandler, but it does not seems to be the case. How can I achieve the desired functionality?

Here is a sample code that illustrates my situation:

import java.io.IOException;

import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.ResponseHandler;
import org.apache.http.client.methods.HttpUriRequest;
import org.apache.http.client.methods.RequestBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.client.StandardHttpRequestRetryHandler;
import org.apache.http.protocol.HttpContext;
import org.apache.http.util.EntityUtils;


public class Main {

  public static void main(String[] args) throws Exception {

    // a response handler that throws an exception if status is not 200
    ResponseHandler<String> responseHandler = new ResponseHandler<String> () {

      @Override
      public String handleResponse(HttpResponse response) throws
        ClientProtocolException, IOException
      {
        System.out.println("-> Handling response");

        if (response.getStatusLine().getStatusCode() != 200){
          // I expected this to trigger the retryHandler 
          throw new ClientProtocolException("Status code not supported");
        }
        return EntityUtils.toString(response.getEntity());
      }

    };

    StandardHttpRequestRetryHandler retryHandler = 
        new StandardHttpRequestRetryHandler(5, true)
    {
      @Override
      public boolean retryRequest(
          final IOException exception,
          final int executionCount,
          final HttpContext context)
      {
        System.out.println("-> Retrying request");
        return super.retryRequest(exception, executionCount, context);
      }
    };

    // my client with my retry handler
    HttpClient client = HttpClients
        .custom()
        .setRetryHandler(retryHandler)
        .build();

    // my request
    HttpUriRequest request = RequestBuilder
        .create("GET")
        .setUri("http://httpstat.us/403")         //always returns 403
        .build();

    String contents = client.execute(request, responseHandler);

    System.out.println(contents);
  }


}

Solution 1

Try using a custom ServiceUnavailableRetryStrategy

CloseableHttpClient client = HttpClients.custom()
        .setServiceUnavailableRetryStrategy(new ServiceUnavailableRetryStrategy() {
            @Override
            public boolean retryRequest(
                    final HttpResponse response, final int executionCount, final HttpContext context) {
                int statusCode = response.getStatusLine().getStatusCode();
                return statusCode == 403 && executionCount < 5;
            }

            @Override
            public long getRetryInterval() {
                return 0;
            }
        })
        .build();

Solution 2

You can do it by manually checking the status code, something like this:

CloseableHttpResponse response = null;
boolean success = false;
while(!success) {
    response = client.execute(httpGet);
    int status = response.getStatusLine().getStatusCode();
    success = (status == 200);
    if (!success) {
        if(status == 403) {
            Thread.sleep(2000); // wait 2 seconds before retrying
        } else {
            throw new RuntimeException("Something went wrong: HTTP status: " + status);
        }
    }
}
String contents = EntityUtils.toString(response.getEntity());
response.close();

// ....

System.out.println(contents);

You would need to add some things like retrying a predefined number of times before throwing a final exception and catching some checked exceptions (like the InterruptedException thrown by Thread.sleep()), but basically the code shows the main idea.

Comments

[Drunix]

And don't forget to add some kind of retry limit, just in case the 403 really means "forbidden". Anyway, if there is any chance, get the broken server fixed.

[Cacovsky]

Thank you for your answer. In fact, since there is a RetryHandler, I would expect some kind of integration between the RetryHandler and the ResponseHandler, avoiding the kind of solution you've proposed. But it seems that it is not possible using HttpComponents pluggable stuff. I'll keep trying, though.

[Cacovsky]

This is exactly what we came up with. I was just posting this very same answer, thanks!

[Cacovsky]

Please, see oleg's answer, it was exactly what I was looking for.

[one stevy boi]

This is great for the case of a 503 "Service Unavailable", it even has the word "Unavailable" right in the name. Unfortunately I came here looking for the answer to the title question, which is: how to retry a request for a given status code, or other condition. I have a use-case where I need to re-authenticate on receiving a 401 and retry a request, and while I could override the ServiceUnavailableRetryStrategy, it really seems like a hack given that the interface seems to be meant specifically for cases of the server being unavailable. If only they'd called it something else

[kjkszpj]

@one stevy boi - Agree to you - stuck at the exact same place, I wonder if you found a better solution?