How does TCP handle multiple requests targeted to one port?

Question

As a server may get many requests from clients at the same time to a particular port, for example port 80 for HTTP objects, how does the server handle simultaneous requests?

Solution

TCP doesn't really define what a "request" looks like; as a transport protocol it only provides a duplex data stream (no different from data transfer over a serial port) and leaves it up to the application to work out the rest.

This also means that requests aren't targeted to a port; they're targeted to sent over a specific connection (data stream). Individual TCP packets have no meaning on their own; they're always reassembled to a continuous stream of data. You've already noted that each packet carries destination and source port numbers, identifying it as belonging to a specific connection. (Note: They don't exactly distinguish a specific client, as a single client may open multiple connections; instead they distinguish a specific stream of data.)

So the question really has nothing to do with ports anymore – the only thing left to ask is about how protocols running over TCP distinguish multiple requests over the same connection.

---

Some protocols aren't even request-oriented to begin with – they might be command-oriented, like FTP or SMTP; or they might have a mix of requests with and without a response, like SSH; or they might transfer unstructured data that arguably doesn't look anything like made out of distinct requests at all, like simple Telnet.

Some protocols, like HTTP/1.0 or Gopher or WHOIS, only allow one request per connection. Once the server has sent the response it just closes the connection, and the client has to connect again (with a new TCP src/dst port pair).

HTTP/1.1 supports long-lived connections, but still only one request at a time. The client has to wait for the first response to finish before sending a second request. (And now there's a need to distinguish multiple responses: for regular resources the HTTP client uses the 'Content-Length' header to know when the response is done, and "indefinite length" responses are sent using a 'chunked' format.)

There is also an HTTP/1.1 extension called "pipelining" that allows multiple requests to be stacked and sent at once. Its mechanism is simple: responses arrive in exactly the same order as requests. (But note that although most HTTP clients use long-lived connections, they do not use HTTP pipelining as it turned out to offer more problems than solutions.)

Some protocols add their own multiplexing on top of the TCP-provided stream. The most well-known example by now is HTTP/2, which has a system of "streams" over a single TCP connection – each request and response is assigned its own stream, divided into small chunks, and the chunks carry their length and the stream ID. The receiver can distinguish multiple requests by reassembling them based on stream ID ... which is practically mirroring how TCP works.

Other protocols like DNS are simpler but have the same general concepts; a DNS request or response over TCP is prefixed with its own length in bytes, so the receiver knows exactly how many bytes the 1st request is, and therefore where the 2nd request starts. Responses again carry a "request ID" and the client can know which response belongs to which request.

---

So in general, there are two common formats for delimiting multiple requests over a single TCP connection or other transport stream:

• Line-based, where each packet is terminated or delimited with a line break or another special character. IRC, FTP (control), SMTP are mostly line-based.

• Length-based, where each packet is prefixed with its own length in bytes. SSH, DNS, git://, HTTP/2 are length-based.

• HTTP/1.x is an unfortunate mix of both: the request/response header is line-based, but the body is length-based.

  There are other weirdos, e.g. IMAP is line-based except when it isn't.

And there are two common methods for processing multiple requests and sorting out the corresponding responses (if the protocol allows that):

• Pipelining – the client can submit multiple requests in a specific order, and the responses arrive in exactly the same order.

  So in HTTP/1.1, if the client requests page A, page B, page C, then the server responds with page A, page B, and page C always in that order.

• Multiplexing – the client can submit multiple requests with individual "request IDs", and the responses may arrive in any order, as each response carries the original "request ID" that prompted it.

  For example, in DNS, the client can request google.com/A (req#1234) and facebook.com/AAAA (req#3456), and the server may respond to req#3456 before it ever gets around to handling req#1234.

  IMAP has tagged requests/responses, but it also has unsolicited responses that the server may send unrelated to any particular request (e.g. push notifications).

• Multiplexing can also be used with "channel IDs", as is the case in HTTP/2 and SSHv2. The difference is that "channel IDs" are long-lived and indeed work much like the ports in TCP header, whereas "request IDs" are short-lived.

  In HTTP/2, each request is assigned a stream and the responses may be interleaved – the client may receive some data tagged "stream A", some data tagged "stream B", and then more data for stream A. Although streams only carry a single request/response, they're still made out of multiple different messages and so act more like channel IDs.

• There may even be a mix of both – e.g. a protocol may have multiplexed channels, but over each channel it may carry multiple tagged or pipelined requests.

  SSHv2 works like that – the same connection may have multiple interactive shell channels (unstructured), SFTP channels (pipelined request/response), TCP forwarding channels (unstructured), an ssh-agent forwarding channel (pipelined request/response), and so on.

Comments

[OOOO]

Perhaps the following link can help you.stackoverflow.com/questions/3329641/…

[Steffen Ullrich]

It is unclear for me what exactly you are specifically asking about. Is it how the server can distinguish the different clients (they have different source IP and port) or how the server can handle multiple requests at once (multiple processes, multiple threads or event based data handling) or what exactly? Please narrow down your question.

[Richard]

@SteffenUllrich No, not how it can distinguish the different clients. Destination and source ports are defined in the header already. Yes, I'm asking how the server handles multiple requests at once.

[OOOO]

@RichardW It is my pleasure.

[user1686]

@RichardW: Your edit kinda contradicts your own comment about destination and source ports, because the linked thread also just talks about ports... I guess the key point is that ports don't distinguish clients, they distinguish connections?

[Richard]

@grawity Yes, sorry, I meant that they distinguish connections and not clients.

[I say Reinstate Monica]

Please do not edit your original question to include an answer. Instead, post an answer on your question.