How to click submit button in a form?
14,740
- You need to use "data", not "params". "params" is used for GET parameters encoded into the URL, but "data" is sent in the body.
- The correct url is
https://www.nysaves.org/nytpl/fundperform/fundHistorySearch.do
, nothttps://www.nysaves.org/nytpl/fundperform/fundHistory.do?
. - You don't need the
submit
keyword. But adding it won't hurt.
This code will work:
startDate = datetime.datetime(2016,1,1).strftime('%m/%d/%Y')
endDate = datetime.datetime(2016,2,20).strftime('%m/%d/%Y')
serviceurl = 'https://www.nysaves.org/nytpl/fundperform/fundHistorySearch.do'
payload = {'fundid': 1003022, 'startDate': startDate, 'endDate': endDate}
r = requests.post(serviceurl, data=payload)
print(r.text)
Author by
vkc
Updated on June 26, 2022Comments
-
vkc almost 2 years
I am trying to scrap this website for fund price history, by providing the start date, end date and click the 'Get Prices' button via POST method.
However the page requests.post() return does not contain the results, as if the "Get Price" button was never pressed. This is the URL I put together through the code:
I read other posts on Stackoverflow about submitting form via POST in Python and I couldn't make it work. Could you please help? Many thanks!
import requests import datetime startDate = datetime.datetime(2016,1,1).strftime('%m/%d/%Y') endDate = datetime.datetime(2016,2,20).strftime('%m/%d/%Y') serviceurl = 'https://www.nysaves.org/nytpl/fundperform/fundHistory.do?' payload = {'fundid':1003022, 'startDate':startDate, 'endDate': endDate, 'submit':'Get Prices'} r = requests.post(serviceurl, params=payload) #from IPython.core.display import HTML #HTML(r.content.decode('utf-8'))