How to extract the HTTP error text from a requests response?

python python-requests

29,087

You can access the response's status code using responsedata.status_code and its textual description via responsedata.reason (see more in http://docs.python-requests.org/en/master/api/)

29,087

Author by

Programmer120

Updated on July 09, 2022

Comments

Programmer120 almost 2 years

I have the following code:

    tries = 10
    for n in range(tries):
        try:
            ....
            responsedata = requests.get(url, data=data, headers=self.hed, verify=False)
            responsedata.raise_for_status()
            ..
            if .... : 
                break   #exit loop condition

        except (ChunkedEncodingError, requests.exceptions.HTTPError) as e:
            print ("page #{0} run #{1} failed. Returned status code {2}. Msg: {3}. Retry.".format(page, n, responsedata.status_code, sys.exc_info()[0]))
            if n == tries - 1:
               raise e  # exit the process

The prints I see are:

page #53 run #0 failed. Returned status code 502. Msg: <class 'requests.exceptions.HTTPError'>. Retry.
page #1 run #1 failed. Returned status code 500. Msg: <class 'requests.exceptions.ChunkedEncodingError'>. Retry.

While this is Ok it doesn't give me actual information about the problem. The message just tell me the exception title.

If I print the: responsedata.text when exception happens I see:

 Returned status code 502. Message is: ...
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>
<title>502 - Web server received an invalid response while acting as a gateway or proxy server.</title>
<style type="text/css">
<!--
body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}
fieldset{padding:0 15px 10px 15px;}
h1{font-size:2.4em;margin:0;color:#FFF;}
h2{font-size:1.7em;margin:0;color:#CC0000;}
h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;}
#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS", Verdana, sans-serif;color:#FFF;
background-color:#555555;}
#content{margin:0 0 0 2%;position:relative;}
.content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;position:relative;}
-->
</style>
</head>
<body>
...

This is a giant message most of it is garbage but it also says: 502 - Web server received an invalid response while acting as a gateway or proxy server. can I access this message and also print it to my log?