What is the maximum chunk size in HTTP response with Transfer-Encoding chunked?

Question

The w3.org (RFC2616) seems not to define a maximum size for chunks. But without a maximum chunk-size there is no space for the chunk-extension. There must be a maximum chunk-size, else I can't ignore the chunk-extension as I'm advised to do if it can't be understood (Quote:"MUST ignore chunk-extension extensions they do not understand").

Solution 1

Each chunk extension must begin with a semi-colon and the list of chunk extensions must end with a CRLF. When parsing the chunk-size, stop at either a semi-colon or a CRLF. If you stopped at a semi-colon, ignore everything up to the next CRLF. There is no need for a maximum chunk-size.

chunk          = chunk-size [ chunk-extension ] CRLF
                 chunk-data CRLF

chunk-size     = 1*HEX

chunk-extension= *( ";" chunk-ext-name [ "=" chunk-ext-val ] )

Solution 2

The HTTP specification is pretty clear about the syntax of the HTTP messages.

The chunk size is always given as a hexadecimal number. If that number is not directly followed by a CRLF, but a ; instead, you know that there is an extension. This extension is identified by its name (chunk-ext-name). If you never heard of that particular name, you MUST ignore it.

So what exactly is your problem?

• Read a hexadecimal number
• Ignore everything up to the next CRLF
• Be happy

Comments

[Yahia]

why do you think you need a maximum size ? are you implementing a server ? a client ? a proxy ?

[smRaj]

I would like to ask what would you suggest to do when the server is corrupt and sending never ending hexadecimal number ? Be a victim and read the never ending hexadecimal number forever or fix a limit that suits your application and throw out a warning when that happens?

[smRaj]

Request your thoughts for the same question I posted in Roland's answer.

[David Schwartz]

@smRaj Whatever makes the most sense in your application, probably setting a reasonable limit.

[smRaj]

@DavidSchwartz: That helps.

[puchu]

You can't just ignore, this is the potential security leak. You have to limit chunk metadata (size and list of extensions) by reasonable length.

[puchu]

PS that's why you shouldn't use abandoned nodejs http parser. Please look here. It doesn't make overflow check it is possible to hang this parser forever by simple attack.