How to debug ssh incoming connections?

Question

I'm having troubles ssh-ing to a server (ubuntu 16.04) I have root access to from my machine (macOS 10.12). The situation is weird: I checked that sshd is running on the server and that the port 22 is open (sudo netstat -anp | grep sshd) and traffic isn't blocked (sudo iptables -L | grep ssh and sudo ufw verbose); hosts.deny does not have my IP address either.

Weirdest thing - I can login from another machine with no problem. When I run nmap <server-ip> on my machine it shows only port 80 is open; running the same command on another machine gives only one open port - 22. I tried logging in from third machine - no luck again. Not sure of what's going on.

I need some way (tail -f logs, probably) where I can see what is actually happening on the server when I try to connect from different machines. Maybe this will help me to troubleshoot. Any advices on how shall I debug this problem?

Solution 1

Command man 5 sshd_config will give you plenty of options.

Setting LogLevel VERBOSE in file /etc/ssh/sshd_config is probably what you need, although there are higher levels.

The messages will end ip in /var/log/auth.log by default, so you will have to tail -f /var/log/auth.log in a separate terminal to see what is going on.

Solution 2

The problem turned out to be other machine with same IP address as the server. Very trivial but not obvious (sigh). Thanks for the input for those who answered.

I was desperate to start using Wireshark, when I checked arp tables on clients and noticed that clients that have intermittent problems sometimes get different mac addresses for the same server IP address. This made me think that there might be a different machine in the network. That was the problem.

Comments

[peetonn]

does not help - doesn't log anything when trying to connect

[peetonn]

checked local firewall, it's disabled

[peetonn]

weird thing is - from the machine where it does connect - sometimes it also gives connection refused

[sмurf]

Actually it does help - it shows that no packet reaches the sshd process. Your next step is to find out which firewall is blocking it - the server itself or some router on the way. Try running this command on the server: sudo tcpdump dst x.x.x.x and dst port 22, replacing x.x.x.x with the actual IP address, try to connect from your macOs machine and check if you have anything printed on the screen. If yes - check the servers firewall rules. If not the problem is either in your macOs or in the routers/firewalls between it and the server.

[peetonn]

I'll mark this post as answer, because the question was about debugging ssh connection (not about solving my particular problem) and it did help a little. I posted below on how I resolved my problem.

[Bidyut]

it is network issue. two systems may be assigned with the same IP, because of that the connection refused you are getting for server from where it can connect.