Python socket client Post parameters

Question

Fir let me clear I don't want to to use higher level APIs, I only want to use socket programming

I have wrote following program to connect to server using POST request.

import socket
import binascii

host = "localhost"
port = 9000
message = "POST /auth HTTP/1.1\r\n"
parameters = "userName=Ganesh&password=pass\r\n"
contentLength = "Content-Length: " + str(len(parameters))
contentType = "Content-Type: application/x-www-form-urlencoded\r\n"

finalMessage = message + contentLength + contentType + "\r\n"
finalMessage = finalMessage + parameters
finalMessage = binascii.a2b_qp(finalMessage)


s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
s.sendall(finalMessage)

print(s.recv(1024))

I checked online how POST request is created.

Somehow Paramters are not getting passed to the server. Do I have to add or remove "\r\n" in between the request?

Thanks in advance, Regards, Ganesh.

Solution

This line finalMessage = binascii.a2b_qp(finalMessage) is certainly wrong, so you should remove the line completely, another problem is that there is no new-line missing after Content-Length. In this case the request sent to the socket is (I am showing the CR and LF characters here as \r\n, but also splitting lines for clarity):

POST /auth HTTP/1.1\r\n
Content-Length: 31Content-Type: application/x-www-form-urlencoded\r\n
\r\n
userName=Ganesh&password=pass\r\n

So obviously this does not make much sense to the web server.

---

But even after adding a newline and removing a2b_qp, there is still the problem is that you are not talking HTTP/1.1 there; the request must have a Host header for HTTP/1.1 (RFC 2616 14.23):

A client MUST include a Host header field in all HTTP/1.1 request messages . If the requested URI does not include an Internet host name for the service being requested, then the Host header field MUST be given with an empty value. An HTTP/1.1 proxy MUST ensure that any request message it forwards does contain an appropriate Host header field that identifies the service being requested by the proxy. All Internet-based HTTP/1.1 servers MUST respond with a 400 (Bad Request) status code to any HTTP/1.1 request message which lacks a Host header field.

Also you do not support chunked requests and persistent connections, keepalives or anything, so you must do Connection: close (RFC 2616 14.10):

HTTP/1.1 applications that do not support persistent connections MUST include the "close" connection option in every message.

Thus, any HTTP/1.1 server that would still respond normally to your messages without Host: header is also broken.

This the data that you should send to the socket with that request:

POST /auth HTTP/1.1\r\n
Content-Type: application/x-www-form-urlencoded\r\n
Content-Length: 29\r\n
Host: localhost:9000\r\n
Connection: close\r\n
\r\n
userName=Ganesh&password=pass

Note that you'd not add the \r\n in the body anymore (thus the length of body 29). Also you should read the response to find out whatever the error is that you're getting.

---

On Python 3 the working code would say:

host = "localhost"
port = 9000

headers = """\
POST /auth HTTP/1.1\r
Content-Type: {content_type}\r
Content-Length: {content_length}\r
Host: {host}\r
Connection: close\r
\r\n"""

body = 'userName=Ganesh&password=pass'                                 
body_bytes = body.encode('ascii')
header_bytes = headers.format(
    content_type="application/x-www-form-urlencoded",
    content_length=len(body_bytes),
    host=str(host) + ":" + str(port)
).encode('iso-8859-1')

payload = header_bytes + body_bytes

# ...

socket.sendall(payload)

Comments

[Ganesh Satpute]

@Antti Haapala, Thank you very much. This solved my problem. And pardon my HTTP knowledge. One more thing, that call binascii.a2b_qp() is necessary as socket does not receive String as input.