Get HTTP response header using Python requests module

Question

I am using 'requests' module in Python to query a RESTful API endpoint. Sometimes, the endpoint returns an HTTP Error 500. I realize I can get the status code using requests.status_code but when I get error 500, I'd like to see the HTTP "response text" (I'm not sure what it's called, examples below). So far I've been able to get some of the headers using response.headers. However, the info I'm looking for is still not there.

Using "curl -vvv", I can see the HTTP response that I'm after (some output omitted for clarity):

< HTTP/1.1 200 OK <---------------------this is what I'm after)
* Server nginx/1.4.1 is not blacklisted
< Server: nginx/1.4.1
< Date: Wed, 05 Feb 2014 16:13:25 GMT
< Content-Type: application/octet-stream
< Connection: close
< Set-Cookie: webapp.session.id="mYzk5NTc0MDZkYjcxZjU4NmM=|1391616805|f83c47a363194c1ae18e"; expires=Fri, 07 Mar 2014 16:13:25 GMT; Path=/
< Content-Disposition: attachment; filename = "download_2014161325.pdf"
< Cache-Control: public

Again, that's from curl. Now, when I use Python's request module and ask for headers, this is all I get:

CaseInsensitiveDict(
 {
  'date': 'Tue, 04 Feb 2014 21:56:45 GMT',
  'set-cookie': 'webapp.session.id="xODgzNThlODkzZ2U0ZTg=|1391551005|a11ca2ad11195351f636fef"; expires=Thu, 06 Mar 2014 21:56:45 GMT; Path=/, 
  'connection': 'close',
  'content-type': 'application/json',
  'server': 'nginx/1.4.1'
 }
)

Notice the curl response includes "HTTP/1.1 200 OK" but the requests.headers does not. Nearly everything else in the response headers are there. The requests.status_code gives me the 200. In this example, all I'm after is the "OK". In other scenarios, our nginx server returns more detailed messages, like "HTTP/1.1 500 search unavailable" or "HTTP/1.1 500 bad parameters", etc. I'd like to get this text. Is there a way or could I hack something with Popen and curl? Requests.content and requests.text don't help.

Solution 1

You are looking for the Response.reason attribute:

>>> import requests
>>> r = requests.get('http://httpbin.org/get')
>>> r.status_code
200
>>> r.reason
'OK'
>>> r = requests.get('http://httpbin.org/status/500')
>>> r.reason
'INTERNAL SERVER ERROR'

Solution 2

That is an excellent answer BUT please keep in mind that for certain applications, you need to retrieve the response headers. This is often the case in paginated REST apis. Those can be retrieved with:

r.headers

And iterate the keys with:

[x for x in r.headers]

Happy coding! [R]

Comments

[harperville]

That's exactly what I needed. Thank you. In which doc set can I find this? I have searched through docs.python-requests.org and can't find it.

[Martijn Pieters]

The documentation for this was missing; I added a pull request to remedy that.