SSLError (Read operation timed out) in Python requests

Question

I have a python API script and my script sometimes gets terminated on this line despite using try/except. Here is the code:

    try:
            r = requests.post(URL, data=params, headers=headers, timeout=self.request_timeout)
            try:
                response = r.json()
            except Exception, e:
                message = "ERROR_0104! Unexpected error occured. The error is: "
                message += str(e)
                print message
                aux_func.write_log(message)
                return 'Switch'
    except requests.exceptions.RequestException:
            print "Exception occurred on 'API requests post' procedure."
            counter += 1
            continue
    ...

The error occurs on the second line of above shown code. This is the error:

     r = requests.post(URL, data=params, headers=headers, timeout=self.request_timeout)
      File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 88, in post
        return request('post', url, data=data, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
        return session.request(method=method, url=url, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 383, in request
        resp = self.send(prep, **send_kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 486, in send
        r = adapter.send(request, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 394, in send
        r.content
      File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 679, in content
        self._content = bytes().join(self.iter_content(CONTENT_CHUNK_SIZE)) or bytes()
      File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 616, in generate
        decode_content=True):
      File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/response.py", line 236, in stream
        data = self.read(amt=amt, decode_content=decode_content)
      File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/response.py", line 183, in read
        data = self._fp.read(amt)
      File "/usr/lib/python2.7/httplib.py", line 543, in read
        return self._read_chunked(amt)
      File "/usr/lib/python2.7/httplib.py", line 585, in _read_chunked
        line = self.fp.readline(_MAXLINE + 1)
      File "/usr/lib/python2.7/socket.py", line 476, in readline
        data = self._sock.recv(self._rbufsize)
      File "/usr/lib/python2.7/ssl.py", line 305, in recv
        return self.read(buflen)
      File "/usr/lib/python2.7/ssl.py", line 224, in read
        return self._sslobj.read(len)
    ssl.SSLError: The read operation timed out

I presume something within the Requests module is causing this, but I don't know what.

Solution 1

The read operation has timed out, as it says.

It times out, however, with an ssl.SSLError. This is not what your except is catching. If you want to catch and retry, you need to catch the right error.

Solution 2

I saw that there was some confusion here regarding what the solution is because of lack of enough details. I posted the answer on the comment to the top post but the formatting is not great in the comment section and I will post a properly formatted answer here.

The problem, as Veedrac has mentioned is that I was not catching all the possible exceptions in the code that I posted in the question. My code only catches "requests.exceptions.RequestException", and any other exception will cause the code to exit abruptly.

Instead, I'm gonna re-write the code like this:

try:
        r = requests.post(URL, data=params, headers=headers, timeout=self.request_timeout)
        try:
            response = r.json()
        except Exception, e:
            message = "ERROR_0104! Unexpected error occured. The error is: "
            message += str(e)
            print message
            aux_func.write_log(message)
            return 'Switch'
except requests.exceptions.RequestException:
        print "Exception occurred on 'API requests post' procedure."
        counter += 1
        continue
except Exception, e:
        print "Exception {0} occurred".format(e)
        continue

All I did was add an extra generic exception catcher at the end which will catch all other unaccounted for exceptions.

I hope this helps.

Thanks.

Comments

[rrlamichhane]

ah, that makes sense. Now, I did it like this, I'll see if it works: try: r = requests.post(URL, data=params, headers=headers, timeout=self.request_timeout) except request.exceptions.RequestException: ... except Exception, e: ... Thanks for the tip!

[MrSynAckSter]

It would be very helpful to write an answer with code that states what the right error to catch is. He is clearly confused.

[SkyNT]

A little late? Additional people will come here after googling this specific error (including me) and miss out on a potential solution.

[Veedrac]

@SkyNT I didn't expect this to get so many views. Feel free to suggest an edit.

[Toren]

How to catch the ssl.SSLError ? What is the right way to capture this exception ?

[Veedrac]

@Toren Does the obvious way not work? import ssl, except ssl.SSLError?