Python3: JSON POST Request WITHOUT requests library

Question

I want to send JSON encoded data to a server using only native Python libraries. I love requests but I simply can't use it because I can't use it on the machine which runs the script. I need to do it without.

newConditions = {"con1":40, "con2":20, "con3":99, "con4":40, "password":"1234"} 
params = urllib.parse.urlencode(newConditions)
params = params.encode('utf-8')

req = urllib.request.Request(conditionsSetURL, data=params)
urllib.request.urlopen(req)        

My server is a local WAMP server. I always get an

urllib.error.HTTPError: HTTP Error 500: Internal Server Error

I am 100% sure that this is NOT a server issue, because the same data, with the same url, on the same machine, with the same server works with the requests library and Postman.

Solution

You are not posting JSON, you are posting a application/x-www-form-urlencoded request.

Encode to JSON and set the right headers:

import json

newConditions = {"con1":40, "con2":20, "con3":99, "con4":40, "password":"1234"} 
params = json.dumps(newConditions).encode('utf8')
req = urllib.request.Request(conditionsSetURL, data=params,
                             headers={'content-type': 'application/json'})
response = urllib.request.urlopen(req)

Demo:

>>> import json
>>> import urllib.request
>>> conditionsSetURL = 'http://httpbin.org/post'
>>> newConditions = {"con1":40, "con2":20, "con3":99, "con4":40, "password":"1234"} 
>>> params = json.dumps(newConditions).encode('utf8')
>>> req = urllib.request.Request(conditionsSetURL, data=params,
...                              headers={'content-type': 'application/json'})
>>> response = urllib.request.urlopen(req)
>>> print(response.read().decode('utf8'))
{
  "args": {}, 
  "data": "{\"con4\": 40, \"con2\": 20, \"con1\": 40, \"password\": \"1234\", \"con3\": 99}", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept-Encoding": "identity", 
    "Connection": "close", 
    "Content-Length": "68", 
    "Content-Type": "application/json", 
    "Host": "httpbin.org", 
    "User-Agent": "Python-urllib/3.4", 
    "X-Request-Id": "411fbb7c-1aa0-457e-95f9-1af15b77c2d8"
  }, 
  "json": {
    "con1": 40, 
    "con2": 20, 
    "con3": 99, 
    "con4": 40, 
    "password": "1234"
  }, 
  "origin": "84.92.98.170", 
  "url": "http://httpbin.org/post"
}

Comments

[Startec]

Where is the type of request (i.e. post) identified?

[Martijn Pieters]

@Startec: by using the data parameter the Request object switches from GET to POST.

[Startec]

@MartijnPieters Ahh, I see, and does Python (without requests) support any other methods (such as delete or trace)?

[Martijn Pieters]

@Startec: You can't. See How to make HTTP DELETE method using urllib2? for work-arounds.

[Startec]

@MartijnPieters why do you want to use dumps to make a string out of the json? In apps like Postman you send the raw json file, (where curly braces would not be commented out)

[Martijn Pieters]

@Startec: newConditions is a Python dictionary, not a JSON string.

[Startec]

@MartijnPieters The use of json.dumps makes all newConditions strings: Including parameters that used to be ints (such as con1) Is this up to the server to decode into the correct types?

[Martijn Pieters]

@Startec: no, json.dumps() produces one JSON encoded string, in which the integers are still (JSON) integers. Any compliant JSON decoder will give you integers again.