Server returned HTTP response code: 503 for URL

Question

Getting "Exception in thread "main" java.io.IOException: Server returned HTTP response code: 503 for URL" error when I make multiple calls to Amazon Product Advertising API.

Is the reason overloading of the service? One Possible solution is to use Thread.Sleep(milliseconds) method.

But is there any other more sophisticated solution? Like proxies or something?

Here is the code used to make the connection:

URL amazon = new URL(url);
        URLConnection yc = amazon.openConnection();
        BufferedReader in = new BufferedReader(
                                new InputStreamReader(
                                yc.getInputStream()));

I am using Java!

Solution 1

From http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html, which defines these status codes:

10.5.4 503 Service Unavailable

The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay. If known, the length of the delay MAY be indicated in a Retry-After header. If no Retry-After is given, the client SHOULD handle the response as it would for a 500 response.

Note: The existence of the 503 status code does not imply that a server must use it when becoming overloaded. Some servers may wish to simply refuse the connection.

So yes, server overload is a likely cause. To figure out how to deal with this you probably want to discuss it with whoever operates the service, to understand what they can tolerate. Then, if you search for "throttle web requests" on SO, you'll find a number of discussions of how to implement throttling once you know your requirement.

Edit

Now that I see you're talking about Amazon Advertising API, a quick search got me to this page: http://docs.aws.amazon.com/AWSECommerceService/latest/DG/TroubleshootingApplications.html

The Efficiency Guidelines on that page pretty clearly state what will trigger your problem, and the request rates that are acceptable.

Solution 2

HTTP 503 means Requested Service is Unavailable.

It might be due to,

1) Server might be busy

2) Server might be down for maintenance

If you are accessing a service and getting this error, then try to contact the service owner to find out what's going on.

If you are the owner of the service, then check the errors in the log to find out the problem.

If you are just making lot's of connections to the service from client class, and getting this error, then you need to think about why you need to create so many connections.

Comments

[DataMiningEnthusiast]

Thank you. I am using Amazon Product Advertising API. Since its a service owned by Amazon they do not allow continuous access of the service. I mean it works when I give sleep time of two seconds, but I was thinking is there another way so that I don't have to wait 2 seconds after each API call? To make the connection I am using URLConnection. I have added the code.

[K139]

@DataMiningEnthusiast If the Amazon enforces the 2 seconds rule, then it is better to obey it, and make the call after 2 seconds with sleep.

[DataMiningEnthusiast]

Okay. They didn't mention it particularly that it has to be two seconds. But I tried 1 second and it gave error. It seems to be working with 2 seconds until now. Thanks for the frequent reply.

[DataMiningEnthusiast]

Wow! Thanks, I don't how I skipped that part in their documentation. I already knew that the API has maximum limit to it, but didn't read it officially anywhere. Now I have.

[DataMiningEnthusiast]

Hi, after setting the Thread.sleep() to 3 seconds also, I am getting the same error. I am using Jsoup.connect(url).timeout(300*1000).get(); to get the data from url. I used timeout as well and after every call to url, i call thread.sleep(3000). This is the error HTTP error fetching URL. Status=503