HTTP POST query how to calculate content length

Question

I am making a HTTP POST query to a server and I am creating the post body manually. I think I am making some mistake with the content-length header because on the server side when I get the http response at the beginning I see the headers with http response 200 and then when in my php script I print the post parameters and file names I get the correct values but together with some junk bytes. Here is the body of my http post:

StringBuffer str = new StringBuffer();
str.append("POST /tst/query.php HTTP/1.1\r\n"
        + "Host: myhost.com\r\n"
        + "User-Agent: sampleAgent\r\n"
        + "Content-type: multipart/form-data, boundary=AaB03x\r\n" 
        + "Content-Length: 172\r\n\r\n"
        + "--AaB03x\r\n"
        + "content-disposition: form-data; name=\"asd\"\r\n\r\n123\r\n--AaB03x\r\n"
        + "content-disposition: form-data; name=\"pics\"; filename=\"file1.txt\"\r\n"
        + "Content-Type: text/plain\r\n\r\n555\r\n"
        + "--AaB03x--"
);

Here is the output from the server(ignore [0.0] - it comes from the console where I print the result)

[0.0] HTTP/1.1 200 OK

[0.0] Date: Sat, 10 Dec 2011 11:53:11 GMT

[0.0] Server: Apache

[0.0] Transfer-Encoding: chunked

[0.0] Content-Type: text/html

[0.0] 

[0.0] 6

[0.0] Array
[0.0] 

[0.0] 2

[0.0] (
[0.0] 

[0.0] 1

[0.0]  

[0.0] 1

[0.0]  

[0.0] 1

[0.0]  

[0.0] 1

[0.0]  

[0.0] 1

[0.0] [

[0.0] 3

[0.0] asd

[0.0] 5

[0.0] ] => 

3
123
1
2
)
0

And the php script on the server which is as simple as you can think of:

<?php 
    print_r($_POST) ;
?>

Solution 1

String boundary = "AaB03x";
String body = "--" + boundary + "\r\n"
            + "Content-Disposition: form-data; name=\"asd\"\r\n"
            + "\r\n"
            + "123\r\n"
            + "--" + boundary + "\r\n"
            + "Content-Disposition: form-data; name=\"pics\"; filename=\"file1.txt\"\r\n"
            + "Content-Type: text/plain\r\n"
            + "\r\n"
            + "555\r\n"
            + "--" + boundary + "--";

StringBuffer str = new StringBuffer();
str.append("POST /tst/query.php HTTP/1.1\r\n"
         + "Host: myhost.com\r\n"
         + "User-Agent: sampleAgent\r\n"
         + "Content-type: multipart/form-data, boundary=\"" + boundary + "\"\r\n" 
         + "Content-Length: " + body.length() + "\r\n"
         + "\r\n"
         + body
);

...I would say is the way it should be done

Solution 2

From the HTTP RFC (RFC2626, 14.3)

The Content-Length entity-header field indicates the size of the entity-body, in decimal number of OCTETs, sent to the recipient or, in the case of the HEAD method, the size of the entity-body that would have been sent had the request been a GET.

In other words you should count the number of bytes (octets), therefor \r\n should be considered to be 2 octets/bytes.

Comments

[gop]

OK so in that case the content-length is 181 i that query but the server still returns something strange: int the php script I say print_r($_POST) and i get the params together with some byte here and there which make no sense :(

[gop]

I tried and here is what I get: response: HTTP/1.1 200 OK Date: Sat, 10 Dec 2011 11:53:11 GMT Server: Apache Transfer-Encoding: chunked Content-Type: text/html 6 Array 2 ( 1 1 1 1 1 [ 3 asd 5] => 3 123 1 2 ) 0 After each line there is \r\n but I removed it for readability. So if its not the content length why that happens? Is it ok the server to return all that bytes through the socket?

[DaveRandom]

I think it is the Transfer-Encoding: chunked that is messing you up, you need to "un-chunk" it - read this

[DaveRandom]

Please edit the question with a literal version of the output (in a code section) and the source of your PHP script

[gop]

I updated the question. You are right about the chunking. Is there a way to tell the server to return it unchunked or it depends on server configuration and I have to parse it and un-chunk it manually?

[DaveRandom]

I don't know of a way to control this from PHP, you would have to modify the server configuration. This may help you find an acceptable solution to your problem.

[DaveRandom]

Having said all this, I suspect you might do better to just use the Apache provided HTTP related packages which will handle a lot of this stuff for you.

[gop]

Thanks a lot! This is sufficient to solve my problem. I will talk to the guy who manages the server and we will decide what to do.

[Moshe Rubin]

IMHO '\r\n' sequences count as one (1) octet, not two. Please see my post stackoverflow.com/questions/31406022/….