SSH slow at starting session

linux shell ssh openssh

10,944

Solution 1

pam_krb5.so was configured to acquire AFS tokens for a non-existent shell which had a 30 second timeout halting any authentication using that module, not just SSH. Removed that and authentication happens much quicker.

Solution 2

In a very similar case, it was one of the update-motd scripts.

The following did the trick:

sudo rm /etc/update-motd.d/90-updates-available

Here is a little helper which measures the time of each script:

$ for f in /etc/update-motd.d/*;do echo $f;time $f;done
/etc/update-motd.d/00-header            0m0.007s
/etc/update-motd.d/10-help-text         0m0.005s
/etc/update-motd.d/90-updates-available 0m49.163s
/etc/update-motd.d/91-release-upgrade   0m0.152s
/etc/update-motd.d/98-fsck-at-reboot    0m0.015s
/etc/update-motd.d/98-reboot-required   0m0.003s
(output reduced to the relevant parts)

Solution 3

If your ssh server has reverse DNS mapping activated, it could be the cause of the delay, look for VerifyReverseMapping in the /etc/ssh/sshd_config file of the server.

10,944

penguin359

Updated on September 18, 2022

Comments

penguin359 almost 2 years
Starting an Interactive shell over SSH is slow to one of my servers. Everything leading up to it including negotiating encryption is fast, but then it hangs for 45 seconds. After that, it finishes and I have a shell. How do I identify what it's hanging on? I tried clearing the environment and disabling all forwarding in case that was slowing it down but it didn't help. Here's my test command:
```
env -i ssh -x -a -vvv server
```
and here's the output from SSH:
```
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug1: Requesting [email protected]
debug1: Entering interactive session.
*(hangs for 45 seconds here)*
debug3: Wrote 128 bytes for a total of 3191
debug2: callback start
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug1: Sending environment.
```
- forcefsck about 13 years
  
  unix.stackexchange.com/questions/10476/…
- penguin359 about 13 years
  
  @forcefsck I don't think that post is relevant. My initial connection is fast, keys get exchange, authentication is fast, no waiting on a password prompt like in the above post. Also, all forward and reverse DNS should be finished by this point I'm pretty certain. Also, this problem is not intermittent like in the above post, it's every single time. The verbose output I posted above starts just after the authentication succeeded message. I think it's more along the lines of bash or a pam session module being slow to start.
- Gilles 'SO- stop being evil' about 13 years
  
  Is there a delay for non-interactive sessions as well? Is there a delay when you start a second connection while one is active? What authentication method do you use? Is your home directory automounted on login in some way (remote or encrypted)? A pam module could be the culprint, what do you have in /etc/pam.d/sshd (or however it's called on your system)? If you have access to the server logs, is there anything relevant?
- penguin359 about 13 years
  
  @Gilles Yes, it seems to affect remote commands, scp, and sftp. If you mean two separate SSH connections, both are slow. If you mean starting a second session through the same control socket, I just tried that and it's fast, but I do get a warning about xauth failure to lock .Xauthority. I use anything from keyboard-interactive, publickey, host-based, to gssapi-keyex. Authentication is always fast regardless, though speed varies slightly. My home directory is local to the server and not encrypted. PAM is essentially Ubuntu default + pam_krb5.so installed.
- Alen Milakovic about 13 years
  
  You say "slow to one of my servers". Does it behave normally when sshing to other machines? And this is from your home machine, which you are logged into locally? What is the OS on local and remote (including version)? Also, is sshing from the server to your home machine normal? Does sshing as another user (both local and remote) make any difference?
- penguin359 about 13 years
  
  Shoot! It was PAM, one of the admins configured pam_krb5.so to attempt to acquire AFS tokens for a non-existent cell. I wasn't even thinking PAM when I originally wrote the question.
penguin359 about 13 years

I thought reverse DNS mapping happened before authentication was allowed to succeed, but I'll test that.
penguin359 about 13 years

I think you mean UseDNS, I got a depreciated warning on VerifyReverseMapping. Regardless, I set both to no and it didn't help. I do put care into making sure my reverse mapping is correct and host can map my client's hostname to an A record and back again with no problem.
Rennex about 8 years

Thanks! Most people just advise to turn off DNS, but my ssh login was still being slow. I started digging into the login process, and your answer pointed me to /etc/update-motd.d/50-landscape-sysinfo, which seemed to be the slow one.
Eaten by a Grue almost 6 years

this is really helpful - for me the killer was 50-landscape-sysinfo. And fyi, you can simply chmod -x /etc/update-motd.d/90-updates-available to prevent it from executing at login instead of removing it altogether.
TecHunter over 2 years

not really a solution as you will deactivate completely PAM. solving an issue and potentially creating many others