SSH 7.4 prolonged pause at "pledge: network"

Question

I have a machine recently updated to Fedora 25, running openSSH 7.4. Ever since then, logging in via ssh takes 25-30 seconds on a LAN where it normally takes no more than 1 second.

Running the client with -vvv, using public key authentication, the pause occurs here.

debug1: Authentication succeeded (publickey).
Authenticated to crystalline.kodiak ([192.168.0.22]:127).
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Requesting [email protected]
debug3: send packet: type 80
debug1: Entering interactive session.
debug1: pledge: network

This looks identical to output to other (Fedora 23, openSSH 7.2) machines on the same network which do not have any problem.

Watching top on the server side during the login, systemd flares up briefly -- a few seconds -- at the beginning of the pause, something not noticeable on the other machines. After that the system is completely idle. Likewise, there is no unusual activity on the client side.

Once logged in everything is fine.

I have watched the exchange from the client with Wireshark and during the pause there are no packets exchanged. The client and server are on Ethernet through a Router, so I am also able to watch the server address for any traffic. There's nothing going on.

Here's the sshd_config:

Port 127
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
IgnoreRhosts yes
SyslogFacility AUTHPRIV
LogLevel INFO
TCPKeepAlive yes
ClientAliveInterval 120
ClientAliveCountMax 15
PermitRootLogin yes
StrictModes yes
PubkeyAuthentication yes
AuthorizedKeysFile  .ssh/authorized_keys
PasswordAuthentication no
ChallengeResponseAuthentication no
KerberosAuthentication no
GSSAPIAuthentication no
UsePAM yes
X11Forwarding no
UsePrivilegeSeparation sandbox
AcceptEnv LANG LC_*
Subsystem   sftp    /usr/libexec/openssh/sftp-server  

As per Sato Katsura's suggestion in comments, I have tried with UseDNS no; this did not make any difference.

Solution 1

Turns out this is quite the corner case.

The machine is a Raspberry Pi, running the stock Pi kernel, but with a generic armhf Fedora 25 userland. It was also set up headless and never used otherwise, but when plugged into a monitor and keyboard there was an obvious problem with systemd-logind.service. I traced it down to this issue, which was introduced last year when core parts of systemd became dependent on seccomp, which for whatever reason is not included in the stock Pi kernel, but possibly via a misconfiguration which make it seem that it is.

The solution was fairly simple; service file options which require seccomp need to be removed. There's a handful of these in /usr/lib/systemd/system, including systemd-logind.service.

It would also likely leave networking disabled on a stock system, but I use my own service for this and that was not affected (i.e., the chance of someone else running into this problem this way is slim).

Anyway, I commented out the following lines in all of them:

MemoryDenyWriteExecute=yes
SystemCallFilter=...

Rebooted, no more problems.

Solution 2

In my case the reason was a crashed rsyslogd. I found this out because there were no more log entries in /var/log/secure.

So I restarted service rsyslog. That resolved the problem for us.