Why is my port closed when accessing from the network?

Question

I have read so many pages, and tried so many things, but am stuck here.

I have a freshly installed Ubuntu 20.04 server. And on it I run a Django test server (just to try it out). It is running and listening on port 8000.

I can see that:

$ sudo netstat -tulpen
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode      PID/Program name    
tcp        0      0 0.0.0.0:5432            0.0.0.0:*               LISTEN      112        62240      47748/postgres      
tcp        0      0 127.0.0.1:8000          0.0.0.0:*               LISTEN      1000       65495      48618/python        
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      101        23068      755/systemd-resolve 
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      0          27689      850/sshd: /usr/sbin 
tcp6       0      0 :::5432                 :::*                    LISTEN      112        62241      47748/postgres      
tcp6       0      0 :::22                   :::*                    LISTEN      0          27691      850/sshd: /usr/sbin 
udp        0      0 127.0.0.53:53           0.0.0.0:*                           101        23067      755/systemd-resolve 
udp        0      0 192.168.0.15:68         0.0.0.0:*                           100        23071      753/systemd-network 
udp6       0      0 fe80::224:e8ff:fe22:546 :::*                                100        23086      753/systemd-network 

And it responds just fine. I can on the server do:

wget localhost:8000

and I get a response.

So now I move to a desktop machine on my lan. And I try wget:

$ wget server.lan:8000
--2020-10-11 11:42:18--  http://server.lan:8000/
Resolving server.lan (server.lan)... 192.168.0.15
Connecting to server.lan (server.lan)|192.168.0.15|:8000... failed: Connection refused.

Sure enough:

$ nmap server.lan
Starting Nmap 7.80 ( https://nmap.org ) at 2020-10-11 22:43 AEDT
Nmap scan report for server.lan (192.168.0.15)
Host is up (0.00025s latency).
Not shown: 998 closed ports
PORT     STATE SERVICE
22/tcp   open  ssh
5432/tcp open  postgresql

Nmap done: 1 IP address (1 host up) scanned in 0.04 seconds

$ nmap serverlan -p 8000
Starting Nmap 7.80 ( https://nmap.org ) at 2020-10-11 22:44 AEDT
Nmap scan report for server.lan (192.168.0.15)
Host is up (0.00068s latency).

PORT     STATE  SERVICE
8000/tcp closed http-alt

Nmap done: 1 IP address (1 host up) scanned in 0.03 seconds

OK, standard checks on the server:

$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:8000

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination        

Because I enabled port 8000 with:

$ sudo iptables -A INPUT -p tcp --dport 8000 -j ACCEPT

Following instructions here: https://www.e2enetworks.com/help/knowledge-base/how-to-open-ports-on-iptables-in-a-linux-server/

and:

$ sudo ufw status verbose
Status: inactive

Yet from a desktop on the lan port 8000 is closed!

I should note port 22 is open (as in I am doing this using an ssh login to the server, and I can see the database on port 5432 as well and manage it with pgadmin4 from the desktop just fine, so ports 22 and 5432 are open, and yet I cannot see on iptables -L that they are open.

What mystical magic is Ubuntu 20.04 working that none of many pages I've read hit at here.

I can even ping it fine:

$ ping arachne.lan
PING arachne.lan (192.168.0.15) 56(84) bytes of data.
64 bytes from 192.168.0.15 (192.168.0.15): icmp_seq=1 ttl=64 time=0.269 ms
64 bytes from 192.168.0.15 (192.168.0.15): icmp_seq=2 ttl=64 time=0.267 ms
64 bytes from 192.168.0.15 (192.168.0.15): icmp_seq=3 ttl=64 time=0.530 ms
64 bytes from 192.168.0.15 (192.168.0.15): icmp_seq=4 ttl=64 time=0.284 ms
64 bytes from 192.168.0.15 (192.168.0.15): icmp_seq=5 ttl=64 time=0.280 ms
64 bytes from 192.168.0.15 (192.168.0.15): icmp_seq=6 ttl=64 time=0.280 ms

So because I like it and use I installed cockpit and it runs of port 9090. It installs and works from a desktop.

Now:

$ netstat -tulpen
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode      PID/Program name    
tcp        0      0 0.0.0.0:5432            0.0.0.0:*               LISTEN      112        62240      -                   
tcp        0      0 127.0.0.1:8000          0.0.0.0:*               LISTEN      1000       65495      48618/python        
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      101        23068      -                   
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      0          27689      -                   
tcp6       0      0 :::5432                 :::*                    LISTEN      112        62241      -                   
tcp6       0      0 :::9090                 :::*                    LISTEN      0          71091      -                   
tcp6       0      0 :::22                   :::*                    LISTEN      0          27691      -                   
udp        0      0 127.0.0.53:53           0.0.0.0:*                           101        23067      -                   
udp        0      0 192.168.0.15:68         0.0.0.0:*                           100        23071      -                   
udp6       0      0 fe80::224:e8ff:fe22:546 :::*                                100        23086      -  

But iptables is indifferent:

$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:8000

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination      

but nmap from a desktop machine can see it:

$ nmap server.lan
Starting Nmap 7.80 ( https://nmap.org ) at 2020-10-11 22:55 AEDT
Nmap scan report for server.lan (192.168.0.15)
Host is up (0.00024s latency).
Not shown: 997 closed ports
PORT     STATE SERVICE
22/tcp   open  ssh
5432/tcp open  postgresql
9090/tcp open  zeus-admin

Nmap done: 1 IP address (1 host up) scanned in 0.05 seconds

So cockpit and postgresql and ssh all manage somehow when installing to tell Ubuntu 20.04 to open a port.

So looking on the server at an introspective nmap:

$ nmap localhost
Starting Nmap 7.80 ( https://nmap.org ) at 2020-10-11 11:58 UTC
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00021s latency).
Not shown: 996 closed ports
PORT     STATE SERVICE
22/tcp   open  ssh
5432/tcp open  postgresql
8000/tcp open  http-alt
9090/tcp open  zeus-admin

Nmap done: 1 IP address (1 host up) scanned in 0.09 seconds

Port 8000 looks open.

There is no firewall between the desktop and server, just a standard LAN switch.

How is it, that cockpit and postgresql and ssh can all achieve something and it be so hard to work out what and how they have achieved it? They don't use iptables it seems. Nor ufw. What magic am I missing? How can I open port 8000?

Solution

Unfortunately you went down the wrong rabbit hole (but well done for posting a thouroughly researched question!)

The issue is not the port, it's the interface that your service is listening on, shown in the Local Address column of the netstat report: 127.0.0.1 is the localhost (or "loopback") interface - it only accepts connections originating from the same host.

You can generally configure services to listen on a particular interface or on all external interfaces. So for example

udp        0      0 192.168.0.15:68         0.0.0.0:*                           100        23071      753/systemd-network     

is listening on port 68 on whatever interface is assigned the IPv4 address 192.168.0.15, whereas your SSH service

tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      0          27689      850/sshd: /usr/sbin 

is listening on all external interfaces, as denoted by netstat's 0.0.0.0 local address value.

I don't know anything about Django, but this [so] Q&A suggests some methods to configure it to listen on external interfaces:

• How to access the local Django webserver from outside world

Alternatively (for the security conscious) you could leave it listening on the loopback interface, and set up an SSH tunnel for web traffic.

Comments

[Admin]

In my case, there was an additional firewall on my cloud provider page, so u could try to check if there is one

[Bernd Wechner]

The wrong rabbit hole indeed! Thanks so much for pointing it out. Part of me wishes there was maybe just one more little cue on the netstat manpage under Local Address warning of just this. Listing some of the standard Local Address forms and their caveats, like 127.0.0.1 and 0.0.0.0 and 192.168.?.? ,,,,I did read the manual, but it doesn't hint to a learner (and ironically would you believe this is a learner who's worked in networking for years and with *nix systems for decades but still had a blind moment) the true meaning and gravity of that field!