SSH logs i dont understand: maximum authentication attempts exceeded

Question

I'm trying to decrypt all of my ssh logs (in order to give them reasonable tags with logstash). But I have found one case that I dont really understand:

Oct 23 07:43:47 sshd[59830]: Connection from 74.194.6.5 port 60126 on 213.67.100.148 port 22
Oct 23 07:43:51 sshd[59830]: error: maximum authentication attempts exceeded for root from 74.194.6.5 port 60126 ssh2 [preauth]
Oct 23 07:43:51 sshd[59830]: Disconnecting authenticating user root 74.194.6.5 port 60126: Too many authentication failures [preauth]

How can it reach the maximum authentication attempts without giving any sign of a authentication attempt?

Usually I get things like before the "maximum auth..." row appears.

Oct 23 08:54:06 sshd[62392]: Failed keyboard-interactive/pam for [...]
Oct 23 08:52:41 sshd[49690]: Failed publickey for [...]

But not always.

Please note, I dont have problems logging in..

Solution 1

That error message gets triggered, among others, when the ssh client attempts a key-based login and offers more than MaxAuthTries invalid keys. The SSH server will then break off the connection. That can either be caused by a (malicious) client that has no valid keys at all, or by valid users who simply have many different key-pairs and the MaxAuthTries number is reached before the valid key can get exchanged. When that happens the connection will be terminated and won't even reach the stage where alternative login methods are offered/attempted.

(At the default log level) the ssh server doesn't record the failed keys get are exchanged and therefor the error message "error: maximum authentication attempts exceeded for ... ssh2 [preauth]" seems to appear without any prior authentication attempts in the log file.

You can easily simulate that with:

for n in $(seq 1 10 ) ; do ssh-keygen -b 2048 -t rsa -f /tmp/sshkey-$n -q -N "" ; done
ssh  -v -i /tmp/sshkey-1  -i /tmp/sshkey-2  -i /tmp/sshkey-3  ... user@host

debug1: Next authentication method: publickey
debug1: Offering RSA public key: /tmp/sshkey-1
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Offering RSA public key: /tmp/sshkey-2
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Offering RSA public key: /tmp/sshkey-3
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Offering RSA public key: /tmp/sshkey-4
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Offering RSA public key: /tmp/sshkey-5
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Offering RSA public key: /tmp/sshkey-6
debug2: we sent a publickey packet, wait for reply
Received disconnect from hostn port 22:2: Too many authentication failures
Authentication failed

The default value for MaxAuthTries is 6.

sshd[19032]: error: maximum authentication attempts exceeded for login from 10.9.8.7 port 54956 ssh2 [preauth]
sshd[19032]: Disconnecting: Too many authentication failures [preauth]

Increasing the sshd_config LogLevel to VERBOSE will generate the extra log events that make slightly more sense:

sshd[19271]: Connection from 10.9.8.7 port 58823 on 10.9.8.8 port 22
sshd[19271]: Failed publickey for login from 10.9.8.7 port 58823 ssh2: RSA SHA256:QGnu...fpY
sshd[19271]: Failed publickey for login from 10.9.8.7 port 58823 ssh2: RSA SHA256:cjje...dDo
sshd[19271]: Failed publickey for login from 10.9.8.7 port 58823 ssh2: RSA SHA256:IIWe...d1M
sshd[19271]: Failed publickey for login from 10.9.8.7 port 58823 ssh2: RSA SHA256:xrQs...Et0
sshd[19271]: Failed publickey for login from 10.9.8.7 port 58823 ssh2: RSA SHA256:0Zln...UI4
sshd[19271]: Failed publickey for login from 10.9.8.7 port 58823 ssh2: RSA SHA256:hhsj...7Q4
sshd[19271]: error: maximum authentication attempts exceeded for login from 10.9.8.7 port 58823 ssh2 [preauth]
sshd[19271]: Disconnecting: Too many authentication failures [preauth]

Solution 2

According with the sshd config man page

MaxAuthTries
Specifies the maximum number of authentication attempts permitted per connection. Once the number of failures reaches half this value, additional failures are logged. The default is 6.

As you can see, the limit is valid in a per connection basis and not all the attempts are logged. You can also choose how many information do you want in logs

LogLevel
Gives the verbosity level that is used when logging messages from sshd(8). The possible values are: QUIET, FATAL, ERROR, INFO, VERBOSE, DEBUG, DEBUG1, DEBUG2, and DEBUG3. The default is INFO. DEBUG and DEBUG1 are equivalent. DEBUG2 and DEBUG3 each specify higher levels of debugging output. Logging with a DEBUG level violates the privacy of users and is not recommended.

In OpenSSH/Logging and Troubleshooting you can see examples of logs in which you can see that the rejecting part is similar to the one you have shown:

...
Mar 19 11:11:10 server sshd[54798]: Failed password for root from 122.121.51.193 port 59928 ssh2
Mar 19 11:11:10 server sshd[54798]: error: maximum authentication attempts exceeded for root from 122.121.51.193 port 59928 ssh2 [preauth]
Mar 19 11:11:10 server sshd[54798]: Disconnecting authenticating user root 122.121.51.193 port 59928: Too many authentication failures [preauth]

Summing it up, failed authentication attempts are not always sent to logs. The befaviour can be configured in the conf file for httpd.

And now, from my not very happy experience exposing ssh to the internet, let me give some general recomendations:

• Please, do not allow user / password access. You should use only private / public pairs of keys. There is a lot of bad guys out there.
• In any case, root should not be allowed to enter by means of SSH.
• Think installing fail2ban or something similar that can ban IPs for accessing to your system

Comments

[Zip]

This is just a guess, but maybe that is because root login is disabled? Maybe it internally just sets maximum logins to 0...

[Peter]

Makes sense! But thats not the case. But on the other hand, I have only seen this log-pattern with malicious connections. So maybe Its just a strange corner case with ill behaved clients.

[Nehal Dattani]

Do you see that time difference of 4 seconds between connection initialization and first error ? That's where something is happening and increasing log verbosity to debug would tell you what's happening. set "LogLevel" to debug and restart ssh. Then try to connect again and see if you find anything useful in the logs.

[Peter]

Im not the one connecting. This is pattern show up very seldom and only with malicious connections.

[Peter]

Hi!, Im not very happy with this answer either. Because I stated in my original post that EVERY attempt is logged in almost all connections done. Also in the link you posted it clearly states "Every failed login attempt is recorded" Even in the exact example your are posting. My question is WHY my server is not logging these attempts? As to your recommendations. I only accept SSH keys + CERTIFICATES signed by a trusted CA. About root, thats a matter of debate. Facebook for example allows root logins over ssh. I have overload rules in my firewall. So please try to answer the question instead.

[Mircea Vutcovici]

@Peter This seems a bit harsh to say to someone making an effort to help you.

[Peter]

I have loglevel verbose and I dont get any "Failed [...] for" rows. (sometimes)

[Peter]

But what use is a answer that even contradicts itself?

[J.M. Robles]

Mircea, thank you very much for your kind words. @Peter Best luck. I sincerely wish you solved all your problems, including the trace logs