How to use cookies in Python Requests
Solution 1
You can use a session object. It stores the cookies so you can make requests, and it handles the cookies for you
s = requests.Session()
# all cookies received will be stored in the session object
s.post('http://www...',data=payload)
s.get('http://www...')
Docs: https://requests.readthedocs.io/en/master/user/advanced/#session-objects
You can also save the cookie data to an external file, and then reload them to keep session persistent without having to login every time you run the script:
How to save requests (python) cookies to a file?
Solution 2
From the documentation:
-
get cookie from response
url = 'http://example.com/some/cookie/setting/url' r = requests.get(url) r.cookies
{'example_cookie_name': 'example_cookie_value'}
-
give cookie back to server on subsequent request
url = 'http://httpbin.org/cookies' cookies = {'cookies_are': 'working'} r = requests.get(url, cookies=cookies)`
Solution 3
Summary (@Freek Wiekmeijer, @gtalarico) other's answer:
Logic of Login
- Many resource(pages, api) need
authentication
, then can access, otherwise405 Not Allowed
- Common
authentication
=grant access
method are:cookie
-
auth header
Basic xxx
Authorization xxx
How use cookie
in requests
to auth
- first get/generate cookie
- send cookie for following request
- manual set
cookie
inheaders
- auto process
cookie
byrequests
's-
session
to auto manage cookies -
response.cookies
to manually set cookies
-
use requests
's session
auto manage cookies
curSession = requests.Session()
# all cookies received will be stored in the session object
payload={'username': "yourName",'password': "yourPassword"}
curSession.post(firstUrl, data=payload)
# internally return your expected cookies, can use for following auth
# internally use previously generated cookies, can access the resources
curSession.get(secondUrl)
curSession.get(thirdUrl)
manually control requests
's response.cookies
payload={'username': "yourName",'password': "yourPassword"}
resp1 = requests.post(firstUrl, data=payload)
# manually pass previously returned cookies into following request
resp2 = requests.get(secondUrl, cookies= resp1.cookies)
resp3 = requests.get(thirdUrl, cookies= resp2.cookies)
user1474157
Updated on November 07, 2021Comments
-
user1474157 over 2 years
I am trying to login to a page and access another link in the page.
payload={'username'=<username>,'password'=<password>} with session() as s: r = c.post(<URL>, data=payload) print r print r.content
This is giving me a "405 Not Allowed" error. I checked the post method details using chrome developer tools and could see an api (URL/api/auth). I posted to that URL with the payload and it was working and i was getting a response similar to what i could see in the developer.
Unfortunately when trying to 'get' another url after login, i am still getting the content from the login page. Why is the login not sticking? Should i use cookies? I am a newbie, so i don't really know how to work with cookies.
-
user1474157 almost 9 yearsThanks. Actually, it seems there are no cookies that getting created. I checked the request headers and could not see any 'cookies'. Meanwhile there is one cookie created in response headers. How can i make my login stick if there is no cookie.
-
Freek Wiekmeijer almost 9 yearsThe normal flow for an authentication cookie is: (1) when you submit a login form, you receive a cookie in the response headers. (2) on subsequent page requests, you add the cookie to the request headers.
-
Aurelius almost 6 years:O This is amazing! I wish I had known about this feature before. Is it storing the cookies on disk like a browser, or just holding them in memory?
-
gtalarico almost 6 yearsIt just keeps them in memory while the application is running. To save/load cookies from disk see this: stackoverflow.com/a/30441145/4411196
-
yashas123 almost 4 yearsI instead of saving them, made a global variable assigned it a session and made a function so it logins if the cookie is not there and/or returns the session
-
Jitin over 3 yearshow do you add multiple cookies ?
-
Freek Wiekmeijer over 3 yearsThe kwarg “cookies” is a dict, you can add as many items as you want.
-
Andrzej Martyna over 2 yearsfor me print(s.cookies) shows that cookies are in the session but are not always passed in subsequent requests (in fact the same code works against localhost but against a test server does not)