ssh_exchange_identification: Connection closed by remote host (not using hosts.deny)

Question

I'm not using hosts.allow or hosts.deny, furthermore SSH works from my windows-machine (same laptop, different hard drive) but not my Linux machine.

ssh -vvv root@host -p port gives:

OpenSSH_6.6, OpenSSL 1.0.1f 6 Jan 2014
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to host [host] port <port>.
debug1: Connection established.
debug1: identity file /home/torxed/.ssh/id_dsa type -1
debug1: identity file /home/torxed/.ssh/id_dsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6
ssh_exchange_identification: read: Connection reset by peer

On the windows machine, everything works fine, so I checked the security logs and the lines in there are identical, the server treats the two different "machines" no different and they are both allowed via public-key authentication.

So that leads to the conclusion that this must be an issue with my local ArchLinux laptop.. but what?

[torxed@archie ~]$ cat .ssh/known_hosts 
[torxed@archie ~]$ 

So that's not the problem...

[torxed@archie ~]$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination 

No conflicts with the firewall settings (for now)..

[torxed@archie ~]$ ls -la .ssh/
total 20
drwx------  2 torxed users 4096 Sep  3  2013 .
drwx------ 51 torxed users 4096 May 11 11:11 ..
-rw-------  1 torxed users 1679 Sep  3  2013 id_rsa
-rw-r--r--  1 torxed users  403 Sep  3  2013 id_rsa.pub
-rw-r--r--  1 torxed users  170 May 11 11:21 known_hosts

Permissions appear to be fine (same on the server).. Also tried without configuring /etc/ssh/ssh_config with the same result except for a lot of auto-configuration going on in the client which ends up with the same error.

Solution 1

Originally posted on Ask Ubuntu

If you have ruled out any "external" factors, the following set of steps usually helps to narrow it down. So while this doesn't directly answer your question, it may help tracking down the error cause.

Troubleshooting sshd

What I find generally very useful in any such cases is to start sshd without letting it daemonize. The problem in my case was that neither syslog nor auth.log showed anything meaningful.

When I started it from the terminal I got:

# $(which sshd) -Ddp 10222
/etc/ssh/sshd_config line 8: address family must be specified before ListenAddress.

Much better! This error message allowed me to see what's wrong and fix it. Neither of the log files contained this output.

NB: at least on Ubuntu the $(which sshd) is the best method to satisfy sshd requirement of an absolute path. Otherwise you'll get the following error: sshd re-exec requires execution with an absolute path. The -p 10222 makes sshd listen on that alternative port, overriding the configuration file - this is so that it doesn't clash with potentially running sshd instances. Make sure to choose a free port here.

Finally: connect to the alternative port (ssh -p 10222 user@server).

This method has helped me many many times in finding issues, be it authentication issues or other types. To get really verbose output to stdout, use $(which sshd) -Ddddp 10222 (note the added dd to increase verbosity). For more debugging goodness check man sshd.

---

The main advantage of this method is that it allows you to check the sshd configuration without having to restart the sshd on the default port. Normally this should not interfere with existing SSH-connections, but I've seen it. So this allows one to validate the configuration file prior to - potentially - cutting off ones access to a remote server (for example I have that for some VPS and even for physical servers where I need to pay extra to get out-of-band access to the machine).

Solution 2

You can also have a host who's memory is so badly fragmented that it can't allocate a page a contiguous memory to fork the process for hosting an SSH session.

In such a case, you can get either of the messages:

ssh_exchange_identification: read: Connection reset by peer

or:

Connection closed by aaa.bbb.ccc.ddd

depending on how far the host gets before it bails out.

If memory fragmenting is the apparent cause, the solution is to access the server via other means and to restart some of the pertinent services. I have found Apache and MySQL to be the culprit on VM's since VM's don't have a swap partition. Failing that, reboot the host.

Solution 3

Just in case, because this happened to me. Get sure you have sshd running in the host!

It's a stupid failure, but might be really be your problem.

Solution 4

I found that this error was due to the exceeded the ssh sessions to the server. I found the hosts trying to connect and killed all sessions from all clients. The issue solved after clearing up all sessions.

Solution 5

I ran across the ssh_exchange_identification: read: Connection reset by peer problem in a script that starts 16 or more ssh sessions in a loop. sshd apparently can't keep up; adding a short sleep (clearly a workaround ... :D downvoters) solved my problem:

for i in $(seq 32)
do
    ssh -f root@$HOST "./test_server -p $(expr $BASE_PORT + $i)" > svr${i}.out
    # for > 8 connections, ssh has ssh_exchange_identification issues
    sleep 0.1
done

View more solutions

Comments

[0xC0000022L]

please give the output of iptables-save|grep -v '^#', that'll include the other tables (e.g. nat and mangle). If they're empty, simply state that. Your iptables output above is by default limited to the filter table. Also, on the SSH server run SSH on an alternative port like this and give the debug output.

[Torxed]

@0xC0000022L gist.github.com/Torxed/d7a5a556c527ffbb609d and gist.github.com/Torxed/1fd9b5b0c276629caf30 and regarding the firewall, SSH is working for my windows drive (again, same laptop ergo mac and IP) but not for my linux disk.

[Torxed]

@0xC0000022L I'm extremely sorry. I connected to the wrong IP.. Running SSH on port 8080 that's why i recieved this issue when connecting to a host running a web-cache on port 8080 >_<

[0xC0000022L]

Hehe ... what we just did is called: en.wikipedia.org/wiki/Rubber_duck_debugging ... ;)

[Torxed]

@0xC0000022L Haha, yepp.. I tried to myself why my connection attempts didn't show up in the logs, anywhere.. Which led me to double check my IP from running ip addr instead of just reading through my notes.. Aaaand facepalm occurred..

[Andrew Hows]

This happened to me intermittently while my server was being hit by some random attacker trying to brute-force sshd. Fixed by adding firewall rules to drop connections from the attacker.

[Anthon]

If sshd was not running the connection would not be closed but refused (try ssh -p someportwithoutsshd localhost).

[Serenkiy]

Well, my case wasn't a direct connection. I created a Reverse Tunnel, to a non-listening machine, and that was the output in the ssh client connection.

[aborted]

How did you do that?

[knocte]

how did you do that? ping...

[Flatron]

One way would be finding the open sessions using who and killing the user processes.

[Jeff Schaller]

In that case, you’d get a “connection refused” like another answer implied, not “Connection establishes” followed by “Connection reset by peer”

[Bryan Estrito]

stupid me also don't know that I don't have any sshd running, fixed it by installing openssh-server

[Tejas Kale]

How to kill ssh sessions : unix.stackexchange.com/questions/127571/…

[aIKid]

Happened to me for unknown reasons as well! I was primariliy using VSCode, but I wasn't able to SSH through powershell in general (WIndows 10). Restarted my PC and it starts working again.