Submitting to a web form using python
Solution 1
If you want to pass q
as a parameter in the URL using requests
, use the params
argument, not data
(see Passing Parameters In URLs):
r = requests.get('http://stackoverflow.com', params=data)
This will request https://stackoverflow.com/?q=%5Bpython%5D , which isn't what you are looking for.
You really want to POST
to a form. Try this:
r = requests.post('https://stackoverflow.com/search', data=data)
This is essentially the same as GET
-ting https://stackoverflow.com/questions/tagged/python , but I think you'll get the idea from this.
Solution 2
import urllib
import urllib2
url = 'http://www.someserver.com/cgi-bin/register.cgi'
values = {'name' : 'Michael Foord',
'location' : 'Northampton',
'language' : 'Python' }
data = urllib.urlencode(values)
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
the_page = response.read()
This makes a POST request with the data specified in the values. we need urllib to encode the url and then urllib2 to send a request.
Comments
-
Serial almost 4 years
I have seen questions like this asked many many times but none are helpful
Im trying to submit data to a form on the web ive tried requests, and urllib and none have worked
for example here is code that should search for the [python] tag on SO:
import urllib import urllib2 url = 'http://stackoverflow.com/' # Prepare the data values = {'q' : '[python]'} data = urllib.urlencode(values) # Send HTTP POST request req = urllib2.Request(url, data) response = urllib2.urlopen(req) html = response.read() # Print the result print html
yet when i run it i get the html soure of the home page
here is an example of using requests:
import requests data= { 'q': '[python]' } r = requests.get('http://stackoverflow.com', data=data) print r.text
same result! i dont understand why these methods arent working i've tried them on various sites with no success so if anyone has successfully done this please show me how!
Thanks so much!