Is it possible to refresh a session in requests?
Either make your B()
a singleton, or make the session a class attribute (and thereby effectively a global).
For example, making it a class attribute only when you create at least one instance could look like this:
class B:
def __init__(self):
if not hasattr(type(self), '_session'):
self._create_session()
@classmethod
def _create_session(cls):
cls._session = requests.Session()
def get(self):
self._session.get('some_url')
If using the session can raise an exception because the server is not closing a session connection correctly, just re-create the session at that time:
def __init__(self):
if not hasattr(type(self), '_session'):
self._create_session()
@classmethod
def _create_session(cls):
cls._session = requests.Session()
def get(self):
retries = 5
while retries:
try:
return self._session.get('some_url')
except requests.ConnectionException as e:
last_connection_exception = e
retries -= 1
raise last_connection_exception
The above example retries up to 5 times. You do not need to re-create the session each time. If a connection has been closed, even with an exception, the session object will just create a new TCP/IP connection for the next request.
If you find that the session object is somehow shot and no longer capable of creating new connections, while a fresh session does work, then that'd be a bug. Please report that to the project with a suitable MCVE.
Comments
-
laike9m almost 2 years
I create a session using
requests.Session()
. For some reason the server side closes this connection, so I have to reconnect. The problem is, this session is used in many places, so I'm wondering is it possible to rebuild a TCP connection but keep the session object so that I can still use it?Example:
s = requests.Session() class B: def __init__(self, session): self._session = session def get(self): self._session.get('some_url') b1 = B(s) b2 = B(s) b3 = B(s) # some get calls ... # then connection is closed # some get calls ...
If I could keep the seesion object, there's no need to replace every
_session
in everyB
instance.Error log:
Traceback (most recent call last): File "/usr/local/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 376, in _make_request httplib_response = conn.getresponse(buffering=True) TypeError: getresponse() got an unexpected keyword argument 'buffering' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen body=body, headers=headers) File "/usr/local/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 378, in _make_request httplib_response = conn.getresponse() File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1174, in getresponse response.begin() File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 282, in begin version, status, reason = self._read_status() File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 243, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 571, in readinto return self._sock.recv_into(b) File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 924, in recv_into return self.read(nbytes, buffer) File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 786, in read return self._sslobj.read(len, buffer) File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 570, in read v = self._sslobj.read(len, buffer) ConnectionResetError: [Errno 54] Connection reset by peer During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.5/site-packages/requests/adapters.py", line 376, in send timeout=timeout File "/usr/local/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 609, in urlopen _stacktrace=sys.exc_info()[2]) File "/usr/local/lib/python3.5/site-packages/requests/packages/urllib3/util/retry.py", line 247, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/lib/python3.5/site-packages/requests/packages/urllib3/packages/six.py", line 309, in reraise raise value.with_traceback(tb) File "/usr/local/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen body=body, headers=headers) File "/usr/local/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 378, in _make_request httplib_response = conn.getresponse() File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1174, in getresponse response.begin() File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 282, in begin version, status, reason = self._read_status() File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 243, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 571, in readinto return self._sock.recv_into(b) File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 924, in recv_into return self.read(nbytes, buffer) File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 786, in read return self._sslobj.read(len, buffer) File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ssl.py", line 570, in read v = self._sslobj.read(len, buffer) requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/laike9m/ICT/zhihu-analysis/dynamic/main.py", line 108, in <module> main() File "/Users/laike9m/ICT/zhihu-analysis/dynamic/main.py", line 89, in main m.detect_new_question() File "/Users/laike9m/ICT/zhihu-analysis/dynamic/monitor.py", line 32, in detect_new_question question = latest_question = next(it) File "/usr/local/lib/python3.5/site-packages/zhihu/topic.py", line 269, in questions res = self._session.get(question_url, params=params) File "/usr/local/lib/python3.5/site-packages/requests/sessions.py", line 480, in get return self.request('GET', url, **kwargs) File "/usr/local/lib/python3.5/site-packages/requests/sessions.py", line 468, in request resp = self.send(prep, **send_kwargs) File "/usr/local/lib/python3.5/site-packages/requests/sessions.py", line 576, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python3.5/site-packages/requests/adapters.py", line 426, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))
This is a very common issue: Python handling socket.error: [Errno 104] Connection reset by peer. I don't have control over the server so I don't know why or how this happens.
The server does support keep-alive cause I'm able to make hundreds of requests(the period lasts for an hour or more).
-
Torxed over 8 yearsYep, I just verified on my own server that the session will be re-created automatically with
requests
lib. There's no need to recreate or even bother monitoring for a broken pipe/session. I do however start to wonder what the OP's actual problem is. -
laike9m over 8 years@Torxed A common issue: stackoverflow.com/questions/1434451/…. requests can't deal with it but raise an exception.
-
Torxed over 8 years@laike9m This is what we're trying to tell you. The server might not honor the
keep-alive
and thus dropping the TCP session. Other reasons for this might be network spikes, wifi connection being iffy or otherwise non-recoverable fatal errors in the network. Either way,requests
library will handle these, if not, please state your actual error message and not what you're experiencing.