TCP: can two different sockets share a port?

Question

This might be a very basic question but it confuses me.

Can two different connected sockets share a port? I'm writing an application server that should be able to handle more than 100k concurrent connections, and we know that the number of ports available on a system is around 60k (16bit). A connected socket is assigned to a new (dedicated) port, so it means that the number of concurrent connections is limited by the number of ports, unless multiple sockets can share the same port. So the question.

Solution 1

A server socket listens on a single port. All established client connections on that server are associated with that same listening port on the server side of the connection. An established connection is uniquely identified by the combination of client-side and server-side IP/Port pairs. Multiple connections on the same server can share the same server-side IP/Port pair as long as they are associated with different client-side IP/Port pairs, and the server would be able to handle as many clients as available system resources allow it to.

On the client-side, it is common practice for new outbound connections to use a random client-side port, in which case it is possible to run out of available ports if you make a lot of connections in a short amount of time.

Solution 2

TCP / HTTP Listening On Ports: How Can Many Users Share the Same Port

So, what happens when a server listen for incoming connections on a TCP port? For example, let's say you have a web-server on port 80. Let's assume that your computer has the public IP address of 24.14.181.229 and the person that tries to connect to you has IP address 10.1.2.3. This person can connect to you by opening a TCP socket to 24.14.181.229:80. Simple enough.

Intuitively (and wrongly), most people assume that it looks something like this:

    Local Computer    | Remote Computer
    --------------------------------
    <local_ip>:80     | <foreign_ip>:80

    ^^ not actually what happens, but this is the conceptual model a lot of people have in mind.

This is intuitive, because from the standpoint of the client, he has an IP address, and connects to a server at IP:PORT. Since the client connects to port 80, then his port must be 80 too? This is a sensible thing to think, but actually not what happens. If that were to be correct, we could only serve one user per foreign IP address. Once a remote computer connects, then he would hog the port 80 to port 80 connection, and no one else could connect.

Three things must be understood:

1.) On a server, a process is listening on a port. Once it gets a connection, it hands it off to another thread. The communication never hogs the listening port.

2.) Connections are uniquely identified by the OS by the following 5-tuple: (local-IP, local-port, remote-IP, remote-port, protocol). If any element in the tuple is different, then this is a completely independent connection.

3.) When a client connects to a server, it picks a random, unused high-order source port. This way, a single client can have up to ~64k connections to the server for the same destination port.

So, this is really what gets created when a client connects to a server:

    Local Computer   | Remote Computer           | Role
    -----------------------------------------------------------
    0.0.0.0:80       | <none>                    | LISTENING
    127.0.0.1:80     | 10.1.2.3:<random_port>    | ESTABLISHED

Looking at What Actually Happens

First, let's use netstat to see what is happening on this computer. We will use port 500 instead of 80 (because a whole bunch of stuff is happening on port 80 as it is a common port, but functionally it does not make a difference).

    netstat -atnp | grep -i ":500 "

As expected, the output is blank. Now let's start a web server:

    sudo python3 -m http.server 500

Now, here is the output of running netstat again:

    Proto Recv-Q Send-Q Local Address           Foreign Address         State  
    tcp        0      0 0.0.0.0:500             0.0.0.0:*               LISTEN      - 

So now there is one process that is actively listening (State: LISTEN) on port 500. The local address is 0.0.0.0, which is code for "listening for all ip addresses". An easy mistake to make is to only listen on port 127.0.0.1, which will only accept connections from the current computer. So this is not a connection, this just means that a process requested to bind() to port IP, and that process is responsible for handling all connections to that port. This hints to the limitation that there can only be one process per computer listening on a port (there are ways to get around that using multiplexing, but this is a much more complicated topic). If a web-server is listening on port 80, it cannot share that port with other web-servers.

So now, let's connect a user to our machine:

    quicknet -m tcp -t localhost:500 -p Test payload.

This is a simple script (https://github.com/grokit/quickweb) that opens a TCP socket, sends the payload ("Test payload." in this case), waits a few seconds and disconnects. Doing netstat again while this is happening displays the following:

    Proto Recv-Q Send-Q Local Address           Foreign Address         State  
    tcp        0      0 0.0.0.0:500             0.0.0.0:*               LISTEN      -
    tcp        0      0 192.168.1.10:500        192.168.1.13:54240      ESTABLISHED -

If you connect with another client and do netstat again, you will see the following:

    Proto Recv-Q Send-Q Local Address           Foreign Address         State  
    tcp        0      0 0.0.0.0:500             0.0.0.0:*               LISTEN      -
    tcp        0      0 192.168.1.10:500        192.168.1.13:26813      ESTABLISHED -

... that is, the client used another random port for the connection. So there is never confusion between the IP addresses.

Solution 3

A connected socket is assigned to a new (dedicated) port

That's a common intuition, but it's incorrect. A connected socket is not assigned to a new/dedicated port. The only actual constraint that the TCP stack must satisfy is that the tuple of (local_address, local_port, remote_address, remote_port) must be unique for each socket connection. Thus the server can have many TCP sockets using the same local port, as long as each of the sockets on the port is connected to a different remote location.

See the "Socket Pair" paragraph at: http://books.google.com/books?id=ptSC4LpwGA0C&lpg=PA52&dq=socket%20pair%20tuple&pg=PA52#v=onepage&q=socket%20pair%20tuple&f=false

Solution 4

Theoretically, yes. Practice, not. Most kernels (incl. linux) doesn't allow you a second bind() to an already allocated port. It weren't a really big patch to make this allowed.

Conceptionally, we should differentiate between socket and port. Sockets are bidirectional communication endpoints, i.e. "things" where we can send and receive bytes. It is a conceptional thing, there is no such field in a packet header named "socket".

Port is an identifier which is capable to identify a socket. In case of the TCP, a port is a 16 bit integer, but there are other protocols as well (for example, on unix sockets, a "port" is essentially a string).

The main problem is the following: if an incoming packet arrives, the kernel can identify its socket by its destination port number. It is a most common way, but it is not the only possibility:

• Sockets can be identified by the destination IP of the incoming packets. This is the case, for example, if we have a server using two IPs simultanously. Then we can run, for example, different webservers on the same ports, but on the different IPs.
• Sockets can be identified by their source port and ip as well. This is the case in many load balancing configurations.

Because you are working on an application server, it will be able to do that.

Solution 5

No. It is not possible to share the same port at a particular instant. But you can make your application such a way that it will make the port access at different instant.

View more solutions

Comments

[K J]

Thanks for the answer, Remy! Your answer is everything I was curious about. ;)

[K J]

Thank you for the perfect answer, Jeremy!

[user207421]

What you say is entirely true of the server side. However the structure of the BSD Sockets API means that outgoing client-side ports must be unique in practice, because the bind() operation precedes the connect() operation, even implicitly.

[Sam YC]

@EJP Hi, I thought bind() is only used at server side before accept()? So client side will bind the particular port as well?

[Ondrej Peterka]

@Remy Connections are discriminated not only by source/destination port/IP, but also by a protocol (TCP, UDP etc.), if I am not mistaken.

[Remy Lebeau]

@GMsoF: bind() can be used on the client side before connect().

[Remy Lebeau]

@OndraPeterka: yes, but not all platforms restrict on that. For instance, Windows happily allows separate IPv4 and IPv6 server sockets to listen on the same local IP:Port without jumping through hoops, but *Nix systems (including Linux and Android) do not.

[user2268997]

@RemyLebeau is there a distinction between client-side and server-side sockets? I ask this because I want to connect to multiple servers using a single Socket.AFIK there is no difference between them after the connection has been estabilished. But I can't find the API to initiate a connection to several host, port pairs using a single Socket.

[Remy Lebeau]

@user2268997: You cannot use a single socket to connect to multiple servers. You must create a separate socket for each connection.

[Jobs]

This is the best answer I've ever seen on SO.

[Fernando Gonzalez Sanchez]

In Windows by default two servers can connect to the same address/port, unless you use SO_EXCLUSIVEADDRUSE, see for details msdn.microsoft.com/en-us/library/windows/desktop/…

[Remy Lebeau]

@FernandoGonzalezSanchez: A single client can have multiple TCP sockets bound to the same local IP/Port pair as long as they are connected to different remote IP/Port pairs. That is not specific to Windows, that is part of how TCP works in general.

[Fernando Gonzalez Sanchez]

@Remy Lebeau: Sorry, wasnt clear enough, two servers (not two clients) can listen to same ip/port in windows by default without multicasting (which is known issue). Your comment is for client, which is obviously true by design.

[Remy Lebeau]

@FernandoGonzalezSanchez: Despite what MSDN documentation says, if two TCP listening sockets try to bind to the same IP/Port (wildcard or otherwise) with SO_REUSEADDR enabled on both sockets, the second bind() still fails with WSAEADDRINUSE. I just tried it. It does not make sense to have two TCP servers listening on the same IP/Port pair, and even the documentation warns against that.

[Fernando Gonzalez Sanchez]

@RemyLebeau. It's indeed possible (the purpose of SO_EXCLUSIVEADDRUSE is to avoid it), but they don't actually "listen", is a kind of port hijacking. A dysfunctional condition.

[Remy Lebeau]

@FernandoGonzalezSanchez: I don't know what to tell you then, because like I said earlier, I actually tried it and it failed to bind successfully. I could see how UDP and maybe multicast sockets might allow it, but not unicast TCP sockets.

[John Strood]

@RemyLebeau Is it not possible, even theoretically, to create two client TCP sockets on the same port and connect to the same server?

[Remy Lebeau]

@Djack the only way that can work is if the client sockets are bound to different local IP addresses on the same port, or are using different transport protocols.

[Prem KTiw]

@RemyLebeau With respect to your previous reply to Djack, I want to know how servers are different than client, in the sense that they are capable of accepting multiple socket connections through one port, but client is unable to do so . Because it is this strange behavior that allows the server to have lot more connections, but restricting client to have only limited no. of connections. Thanks

[Remy Lebeau]

@premktiw a given socket connection is uniquely identified by a combination of transport protocol, client IP+port, and server IP+port. Multiple clients can connect to the same server IP+port only if their client IP+port are different from each other. On the server side, every connection has the same server IP+port, so the client IP+port must be different. On the client side, it can connect multiple times to the same server if it binds to a different IP+port for each connection.

[Prem KTiw]

Okay , what I say is that, use a single port to connect to multiple servers. Shouldn't that be possible ? Because in that case I have different sockets for all the connections. And if that is true then a browser must have been managing the connections to different servers through same port ?

[Prem KTiw]

@RemyLebeau Thanks for reply. Please read the previous comment also . And this also implies that different ports are only needed when there is need to connect to same server concurrently ? And if that is true then a client should not run out of connections, if he do not connects to a server concurrently.

[Prem KTiw]

@N0thing "This way, a single client can have up to ~64k connections to the server for the same destination port." So in practice , if a client do not connects to same server and port, twice or multiple times concurrently, then a client can have even more than ~64K connections. Is that true. If yes then that implies that from a single port on client side it can have connection to many different server processes (such that socket connection is different). So in all, multiple client sockets can reside on same port on client machine ? Please read my comment to "Remey Lebeau" answer. Thanks :D

[Remy Lebeau]

@premktiw yes, technically a client can use the same local port to connect to multiple different servers concurrently. But typically a client lets the OS pick a random available port for each new outbound connection. A client should bind to a specific local port only when required to do so by protocol or network routing restrictions.

[Prem KTiw]

So you are saying that it is not in the hand of client programmer who is writing a client program to bind multiple socket connections to same port. Why OS always picks a random port for client's each new connection, and the OS for server does not do that ? Thanks

[Remy Lebeau]

@premktiw: the client programmer is in full control of how sockets are bound locally. If the programmer wants to use the same port for multiple connections, (s)he must explicitly bind to that port. However, it is customary to let the OS choose a random port instead (by explicitly binding to port 0, or by not binding at all so connect() binds implicitly) to reduce the possibility of any conflicts. A server can bind to a random port, but it is not useful to do so unless the server has a way of announcing the chosen port so clients can then connect to it.

[Remy Lebeau]

@premktiw: you really need to stop asking questions here and start doing some research on your own. There is plenty of material on how IP connections work over a network and how ports work. Get yourself some good books on TCP/UDP and general socket programming.

[Remy Lebeau]

@premktiw: Yes, multiple client sockets can be bound to the same local IP/port pair at the same time, if they are connected to different server IP/Port pairs so the tuples of local+remote pairs are unique. And yes, it is possible for a client to have more than 64K concurrent connections total. From a single port, it can be connected to a potentially infinite number of servers (limited by available OS resources, available router ports, etc) as long as the server IP/port pairs are unique.

[Prem KTiw]

@RemyLebeau Satisfied . Thank You very very much :D

[Prem KTiw]

Would the "Harold's" book on network programming in java will be better ? Please suggest some books . How about Stevens UNIX Network Programming ? I know C, C++ and JAVA

[Remy Lebeau]

"Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam."

[user207421]

He didn't ask about making a second bind().

[PatrykG]

@bibstha How does firewall deal with random ports, when all incoming connections are rejecting?

[Bibek Shrestha]

@PatrykG random ports are for client and firewalls do not look into connecting client port, rather the destination port.

[mnj]

Isn't there a mistake under "So, this is really what gets created when a client connects to a server:". I think that client connects to port 80 of a server, not to some random port. The client establishes the connection from some random port, not 80. Am i right?

[peterh]

@user207421 Did you ever seen an OS where listening sockets aren't set up by bind()? I can imagine it, yes it is quite possible, but fact is that both WinSock and the Posix API uses the bind() call for that, even their parametrization is practically the same. Even if an API doesn't have this call, somehow you need to say it, from where do you want to read the incoming bytes.

[peterh]

@user207421 Of course 100k or more TCP connections can be handled with the same ports, listen()/accept() API calls can create the sockets on a way that the kernel will differentiate them by their incoming ports. The question of the OP can be interpreted on the way that he asks essentially for it. I think, it is quite realistic, but not this is what his question literally means.

[TechNyquist]

I had to scroll the way up the text to put a +1. Well done.

[joe]

In the most voted answer, you can only have one socket listening on the same port. Is it better to say Thus the server can have many TCP **connections** using the same local port... connection and socket are different things right?

[Jeremy Friesner]

You can only have one TCP socket listening on a port, but you can have multiple connected TCP sockets simultaneously using that port for sending and receiving data. (these would be the sockets returned by accept() on the listening socket)

[Ajak6]

@N0thing You said that If a server listens to a port(e.g. 80), when a client connects to it, the requested is handed off to another thread but then how does the server response work with multiple connections at port 80 from different clients? There must me some magic happening at some layer?

[absuu]

What an answer! Adequate personal opinions plus authoritative references!