Simulating ajax POST call using Python Requests
As Martijn Pieters suggested, I tried headers one by one and found that this combination is working now:
import requests
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0',
'Accept': 'application/json, text/javascript, */*; q=0.01',
'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
'X-Requested-With': 'XMLHttpRequest'
}
s = requests.Session()
s.post(login_url, data=login_data)
content = s.post(url, data=data_video, headers=headers)
I thank everyone and especially Martijn Pieters.
Related videos on Youtube
TomT
Updated on September 17, 2022Comments
-
TomT over 1 year
I'm doing a project where my parserstealsgets data about every video on the specific site and save it to my database. I have accomplished everything except full link to the video which is hidden.
There is a player, which automaticaly starts on page load. I have found the JavaScript code which starts the player:function getVidData(resolution, init) { << some code here >> jQuery.ajax({type: 'POST', url: '/ajaxdata.php', dataType: 'json', data: 'mod=videodata&vid=48902&res=' + resolution, success: function (response) { if (response.error != '' && response.error != undefined) { << error handling code here >> } else { StartPlayer(response.width, response.height, response.filename); } } }); }
So after a call if no error found it starts a player using filename from response. That is what I need.
I rechecked a call in Live HTTP Headers:http://<< SITE_URL >>/ajaxdata.php POST /ajaxdata.php HTTP/1.1 Host: << SITE_URL >> User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0 Accept: application/json, text/javascript, */*; q=0.01 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Content-Type: application/x-www-form-urlencoded; charset=UTF-8 X-Requested-With: XMLHttpRequest Referer: << VIDEO_PAGE >> Content-Length: 31 Cookie: << COOKIE VALUES >> DNT: 1 Connection: keep-alive Pragma: no-cache Cache-Control: no-cache mod=videodata&vid=48901&res=640 HTTP/1.1 200 OK Server: nginx/1.5.9 Date: Tue, 22 Apr 2014 16:30:06 GMT Content-Type: text/html Transfer-Encoding: chunked Connection: keep-alive Expires: Tue, 22 Apr 2014 16:30:05 GMT Cache-Control: no-cache Pragma: no-cache Content-Encoding: gzip
So it calls ajaxdata.php with specific params and in response i should find the filename.
However this Python code returns absolutely nothing to me (neither content nor errors)import requests url = "http://LALLALAA/ajaxdata.php" data_video = {"mod": "videodata", "vid": "48901", 'res': '640'} s = requests.Session() s.post(login_url, data=login_data) # Authentication content = s.post(url, data=data_video) print content.content
Variable content prints only "Response [200]"
Now I'm completely stuck and would be grateful if anyone could point to errors I done or solutions i could try.
Thanks-
Martijn Pieters about 10 yearsBetter start experimenting with the headers then;
s.post(url, data=data_video, headers={'X-Requested-With', 'XMLHttpRequest'})
, etc.
-
-
user1788736 over 8 yearsIf the page is password protected and ajax call needs cookies how we can simulate the ajax request?