Simulating ajax POST call using Python Requests

17,559

As Martijn Pieters suggested, I tried headers one by one and found that this combination is working now:

import requests

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0',
    'Accept': 'application/json, text/javascript, */*; q=0.01',
    'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
    'X-Requested-With': 'XMLHttpRequest'
}

s = requests.Session()
s.post(login_url, data=login_data)

content = s.post(url, data=data_video, headers=headers)

I thank everyone and especially Martijn Pieters.

Share:
17,559

Related videos on Youtube

TomT
Author by

TomT

Updated on September 17, 2022

Comments

  • TomT
    TomT over 1 year


    I'm doing a project where my parser steals gets data about every video on the specific site and save it to my database. I have accomplished everything except full link to the video which is hidden.
    There is a player, which automaticaly starts on page load. I have found the JavaScript code which starts the player:

    function getVidData(resolution, init) {
        << some code here >>
        jQuery.ajax({type: 'POST', url: '/ajaxdata.php', dataType: 'json', data: 'mod=videodata&vid=48902&res=' + resolution, success: function (response) {
            if (response.error != '' && response.error != undefined) {
            << error handling code here >>
            } else {
                StartPlayer(response.width, response.height, response.filename);
            }
        }  });
    }
    

    So after a call if no error found it starts a player using filename from response. That is what I need.
    I rechecked a call in Live HTTP Headers:

    http://<< SITE_URL >>/ajaxdata.php
    POST /ajaxdata.php HTTP/1.1
    Host: << SITE_URL >>
    User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0
    Accept: application/json, text/javascript, */*; q=0.01
    Accept-Language: en-US,en;q=0.5
    Accept-Encoding: gzip, deflate
    Content-Type: application/x-www-form-urlencoded; charset=UTF-8
    X-Requested-With: XMLHttpRequest
    Referer: << VIDEO_PAGE >>
    Content-Length: 31
    Cookie: << COOKIE VALUES >>
    DNT: 1
    Connection: keep-alive
    Pragma: no-cache
    Cache-Control: no-cache
        mod=videodata&vid=48901&res=640
    
    HTTP/1.1 200 OK
    Server: nginx/1.5.9
    Date: Tue, 22 Apr 2014 16:30:06 GMT
    Content-Type: text/html
    Transfer-Encoding: chunked
    Connection: keep-alive
    Expires: Tue, 22 Apr 2014 16:30:05 GMT
    Cache-Control: no-cache
    Pragma: no-cache
    Content-Encoding: gzip
    

    So it calls ajaxdata.php with specific params and in response i should find the filename.
    However this Python code returns absolutely nothing to me (neither content nor errors)

    import requests
    
    url = "http://LALLALAA/ajaxdata.php"
    data_video = {"mod": "videodata", "vid": "48901", 'res': '640'}
    
    s = requests.Session()
    s.post(login_url, data=login_data) # Authentication
    
    content = s.post(url, data=data_video)
    print content.content
    

    Variable content prints only "Response [200]"
    Now I'm completely stuck and would be grateful if anyone could point to errors I done or solutions i could try.

    Thanks

    • Martijn Pieters
      Martijn Pieters about 10 years
      Better start experimenting with the headers then; s.post(url, data=data_video, headers={'X-Requested-With', 'XMLHttpRequest'}), etc.
  • user1788736
    user1788736 over 8 years
    If the page is password protected and ajax call needs cookies how we can simulate the ajax request?