How to handle "100 continue" HTTP message?

Question

I'm writing a simplistic HTTP server that will accept PUT requests mostly from cURL as client and I'm having a bit of an issue with handling the Expect: 100-continue header.

As I understand it, the server is supposed to read the header, send back a HTTP/1.1 100 Continue response on the connection, read the stream up to the value on Content-Length and then send back the real response code (Usually HTTP/1.1 200 OK but any other valid HTTP answer should do).

Well, that's exactly what my server does. The problem is that, apparently, if I send a 100 Continue answer, cURL fails to report any subsequent HTTP error code and assumes the upload was a success. For instance, if the upload is rejected due to the nature of the content (there is a basic data check happening), I want the calling client to detect the problem and act accordingly.

Am I missing something obvious ?

edit: here is a sample output from cURL with a secondary header containing an error:

> PUT /test1%2Epdf HTTP/1.1
> Authorization: Basic xxxx
> User-Agent: curl/7.20.0 (i386-pc-win32) libcurl/7.20.0 OpenSSL/0.9.8l zlib/1.2.3
> Host: localhost
> Accept: */*
> Content-Length: 24
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
< HTTP/1.1 415 Unsupported Media Type
< Connection: close
< Content-Type: text/xml
< Content-Length: 289
<

Solution 1

If you are using libcURL to write your client-side program, make sure you set the CURLOPT_FAILONERROR option to 1. For example, in C, you would do something like:

curl_easy_setopt (curl_handle, CURLOPT_FAILONERROR, 1L);

According to the libcURL documentation, this option "tells the library to fail silently if the HTTP code returned is equal to or larger than 400."

Furthermore, the documentation makes it clear that "the default action would be to return the page normally, ignoring that code."

If you are using the curl command line tool, simply adding -f or --fail into your curl command will cause similar behaviour as described above. This is also described in the curl man page.

Note that both these methods are not fail-safe, as is clearly stated in the docs:

"This method is not fail-safe and there are occasions where non-successful response codes will slip through, especially when authentication is involved (response codes 401 and 407)."

Solution 2

I know this is old, but here is my understanding of "100 Continue"

Your server is supposed to validate the request based on header alone from the client i.e. if the request is invalid, don't send "100 Continue" but actual http error instead e.g. 403. This should prevent the client from posting the data which I understand is the whole point of roundtrip to the server (i.e. client waiting for "100 Continue") in the first place.

If you are validating actual posted data, then you need to apply higher-level protocol here i.e. send your error wrapped in valid HTTP response content. Yes, it seems like limitation and I'm not assuming it's protocol limitation; more likely client confusion having to handle server response more than once.

Solution 3

Actually there should be real header after 100 Continue header

So, I normally do like this on client side.

$contents=curl_exec($ch);

list( $header, $contents ) = explode( "\r\n\r\n", $contents , 2);
if(strpos($header," 100 Continue")!==false){
    list( $header, $contents) = explode( "\r\n\r\n", $contents , 2);
}

Solution 4

Elaborating on YOU's answer, and still using PHP as an example:

It is possible that multiple 100 Continue headers could be received. I use the following to slowly work through the headers and remove each of the 100 Continue responses if they exist:

<?php
// a little setup first
$ch = curl_init();
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch,CURLOPT_HEADER,1);
// etc...
$str = curl_exec($ch);

// the goods
$delimiter = "\r\n\r\n"; // HTTP header delimiter
// check if the 100 Continue header exists
while ( preg_match('#^HTTP/[0-9\\.]+\s+100\s+Continue#i',$str) ) {
    $tmp = explode($delimiter,$str,2); // grab the 100 Continue header
    $str = $tmp[1]; // update the response, purging the most recent 100 Continue header
} // repeat

// now we just have the normal header and the body
$parts = explode($delimiter,$str,2);
$header = $parts[0];
$body = $parts[1];
?>

Solution 5

Try adding an empty line (CRLF) after the 100 Continue line (see RFC 2616, Section 6),

View more solutions

Comments

[YOU]

Isn't you need blank line after HTTP/1.1 100 Continue ?

[Stephane]

There is one. The fact that it's not being logged seems to be a display issue with cURL.

[bishop]

Just to clarify, send back a completely valid HTTP response (HTTP/1.1 100 Continue\r\n\r\n) not simply the string "HTTP/1.1 100 Continue". cURL client will wait until it receives those two <CR><LF> sequences, and if it gives up, it'll output (in verbose mode) the message "Done waiting for 100-continue".

[akostadinov]

FYI a gist with 100-continue example I just found: gist.github.com/trevorrowe/c2353ab959c6852a2bd7

[Stephane]

Well, I do send a real header after the code 100 answer. It seems that cURL doesn't care, though.

[Stephane]

It seems the missing line is a bug. I've checked the code and there is an empty line after the "100 continue"

[Stephane]

I'm using the command-line tool but the -f flag doesn't seem to affect how the tool behaves. I've since settled for writing my own client that handles these failed upload properly. Thanks for the answer, though

[Stephane]

Thanks for the answer. You're right: it's an old question but you get it pretty much correct. my real issue is that, well, "100" isn't a success error code: the client should always check the return code contained in the actual response. Apparently, that's not what cURL is doing.

[Piotr Kołaczkowski]

"This should prevent the client from posting the data" - this probably the most misunderstood part of the RFC ever. Lack of 100-Continue does not prevent the client to continue, RFC exlicitly states the client may continue to sent the request body. Actually the only valid way of handling of a response is sending the body or disconnecting.

[paperclip]

Additional helpful background on this misunderstanding by curl / web servers if people are interested on the curl mailing list: More on POST and PUT with 100-continue