Status code 500 not treated as exception

Question

Making a request to the server, as in code below, I've got status code 500, which was not caught as an exception. The output was "500", but I need for all 500 codes to result in sys.exit(). Does requests.exceptions.RequestException not treat 500 as an exception or is it something else? The requests module docs http://docs.python-requests.org/en/latest/user/quickstart/#errors-and-exceptions are not very clear on what falls under this class. How do I make sure that all 500 codes result in sys.exit()?

import requests
import json
import sys

url = http://www.XXXXXXXX.com
headers = {'user':'me'}
try:
    r = requests.post(url, headers=headers)
    status = r.status_code
    response = json.dumps(r.json(), sort_keys=True, separators=(',', ': '))
    print status
except requests.exceptions.RequestException as e:
    print "- ERROR - Web service exception, msg = {}".format(e)
    if r.status_code < 500:
        print r.status_code
    else:
        sys.exit(-1)

Solution 1

A status code 500 is not an exception. There was a server error when processing the request and the server returned a 500; more of a problem with the server than the request.

You can therefore do away with the try-except:

r = requests.post(url, headers=headers)
status = r.status_code
response = json.dumps(r.json(), sort_keys=True, separators=(',', ': '))

if str(status).startswith('5'):
    ...

Solution 2

From the Requests documentation:

If we made a bad request (a 4XX client error or 5XX server error response), we can raise it with Response.raise_for_status():

>>> bad_r = requests.get('http://httpbin.org/status/404')
>>> bad_r.status_code
404

>>> bad_r.raise_for_status()
Traceback (most recent call last):
  File "requests/models.py", line 832, in raise_for_status
    raise http_error
requests.exceptions.HTTPError: 404 Client Error

So, use

r = requests.post(url, headers=headers)
try:
    r.raise_for_status()
except requests.exceptions.HTTPError:
    # Gave a 500 or 404
else:
    # Move on with your life! Yay!

Solution 3

If you want a successful request, but "non-OK" response to raise an error, call response.raise_for_status(). You can then catch that error and handle it appropriately. It will raise a requests.exceptions.HTTPError that has the response object hung onto the error.

Comments

[jarcobi889]

Except in this case if it's a 501 or any other code, you'd have to code in every number above 500. Only useful if you know what response code the server will return, guaranteed

[jarcobi889]

True, but "all 500 codes" could mean 501 or 503 or whatever. This catches those too without hard-coding every number between 500 and 600

[jarcobi889]

Good fix! I hadn't thought of that

[postoronnim]

I think the server is configured to only return "500" for all of 500 family. All good points, guys - thanks. Accepting Moses' for being most direct.

[Nick T]

Rather than str(status).startswith('5') I'd recommend the simpler integer comparison: 500 <= status < 600 (or just status >= 500 as there are no larger error codes)