Using POST and urllib2 to access web API

Question

I am trying to access a web API using a POST technique. I AM able to access it using a GET technique, but the API owners tell me that certain functionality only works with POST. Unfortunately I can't seem to get POST working.

Here's what works with GET:

API_URL = "http://example.com/api/"

def call_api(method, **kwargs):
    url = API_URL + method
    if kwargs:
        url += '?' + urllib.urlencode(kwargs)
    req = urllib2.Request(url)
    auth = 'Basic ' + base64.urlsafe_b64encode("%s:%s" % (USER, PASS))
    req.add_header('Authorization', auth)
    return urllib2.urlopen(req)

Here's what does NOT work with POST (causes HTTP 400 error):

API_URL = "http://example.com/api/"

def call_api(method, **kwargs):
    url = API_URL + method
    data=''
    if kwargs:
        data=urllib.urlencode(kwargs)
    req = urllib2.Request(url, data)
    auth = 'Basic ' + base64.urlsafe_b64encode("%s:%s" % (USER, PASS))
    req.add_header('Authorization', auth)
    return urllib2.urlopen(req)

Does anything jump out at anyone as being inherently incorrect in the POST code? I've never done a POST call before but everything I've read seems to suggest that my code is reasonable. Is there some different way I'm supposed to do the add_header thing for the authorization if I'm using POST?

Solution 1

With urllib2 you need to add the data to the POST body:

def call_api(method, **kwargs):
    url = API_URL + method
    req = urllib2.Request(url)

    if kwargs:
        req.add_data(urllib.urlencode(kwargs))

    auth = 'Basic ' + base64.urlsafe_b64encode("%s:%s" % (USER, PASS))
    req.add_header('Authorization', auth)

    # req.get_method() -> 'POST'

    return urllib2.urlopen(req)

Solution 2

As @sneeu notes above, it's the act of adding the data to be posted to the request that converts the request from a GET into a POST.

However, this still assumes that what the API is expecting to receive in the POST body is form-encoded data. Many more recent APIs that I've worked with are expecting something else in there (XML or JSON, most commonly).

Can you verify what that API is expecting to receive as a data payload?

Solution 3

I have faced the same problem, I want to send data with the POST method of HTTP, but after dir(req) I found get_method, but no set_method, and I also found there is a property called data, so try this out:

>>> req.data={"todototry":"123456"}
>>> req.get_method()
'POST'
>>>

Thanks @sneeu.

Comments

[Daniel Roseman]

Your POST code appears to be exactly the same as your GET code. Did you copy and paste something incorrectly?

[neomech]

Doh! Yes, there was a pasting error. Fixed now.

[neomech]

I had an error in my question where I didn't paste my POST code, but as you can now see it's almost identical to what you have here. I'm going to just go and assume that the problem relates to the API and what it's expecting, and not with my code.

[neomech]

Based on the code I've seen so far, I think you're right...the error is a result of the API not expecting whatever I'm trying to post.