How do I access the data sent to my server using BaseHTTPRequestHandler?

Question

I'm a newbie to Python (using v3.3) and web programing and I've been struggling with a problem all night. I'm issuing a POST call to my server and sending it some data as follows:

DATA = {"listName":"Test list","listDesc":"A test list with test stuff in it.","refreshMode":"Replace","DBKey":"1","UserDisplaySeq":"1"}
DATA = json.dumps(DATA)
METHOD = "POST"
DATA = DATA.encode("utf-8")
params = "account_id=acct 2"
try:
    URL = "http://localhost:8080/lists?" + quote_plus(params)
    request = urllib.request.Request(url=URL,data=DATA,method=METHOD)
    response = urllib.request.urlopen(request)
...

I also have a request handler coded as follows (there are lot of print statements in here for debugging purposes):

class MyHandler(BaseHTTPRequestHandler):
...
def do_POST(self):
    length = int(self.headers['Content-Length'])
    print("HEADERS: ", self.headers)
    print (str(length))
    print(self.rfile)
    post_data = urllib.parse.parse_qs(self.rfile.read(length).decode('utf-8'))
    print(post_data)

This prints the following result to the console:

Starting thread
started httpserver...
HEADERS:  Accept-Encoding: identity
User-Agent: Python-urllib/3.3
Content-Length: 138
Content-Type: application/x-www-form-urlencoded
Host: localhost:8080
Connection: close


138
<_io.BufferedReader name=404>
{}

My questions:
1) In the server (do_POST), how do I access the data I think I'm sending with my request (i.e. {"listName":"Test list","listDesc":"A test...)?

2) Is my request even sending the data in the first place?

3) Is there a place where this is documented in novice-accessible terms?

Solution 1

Give this a try. I stole it from an answer to another question

def do_POST(self):
    ctype, pdict = cgi.parse_header(self.headers.getheader('content-type'))
    if ctype == 'multipart/form-data':
        postvars = cgi.parse_multipart(self.rfile, pdict)
    elif ctype == 'application/x-www-form-urlencoded':
        length = int(self.headers.getheader('content-length'))
        postvars = cgi.parse_qs(self.rfile.read(length), keep_blank_values=1)
    else:
        postvars = {}

    print(postvars.get("listName", "didn't find it"))

Solution 2

1) In the server (do_POST), how do I access the data I think I'm sending with my request (i.e. {"listName":"Test list","listDesc":"A test...)?

you can access the data just by:

print self.rfile.read(length).

after make sure this is working. you can do other parse work. I suggest use simplejson to decode the json string. urllib.parse.parse_qs seems unnecessary.

2) Is my request even sending the data in the first place?

the code looks fine. to make sure it works, just try:

    curl -d "asdf" http://yourhost:yourport

to see if the server have same response. 
so you can know whether the server side or client side goes wrong.

3) Is there a place where this is documented in novice-accessible terms?

the official document is always a good choice:
http://docs.python.org/2/library/basehttpserver.html