Desperate: statd timed out, lockd cannot monitor / unmonitor

linux nfs debian-squeeze rpc

17,924

Solution 1

Here comes couple of suggestions:

I once managed to break the loopback interface (lo) and thanks to it several services, such as NFS, stopped working properly. See with ifconfig if you still have your beloved lo interface up and running. If it's not, go see /etc/network/interfaces and see what's going on.

Also as some people already mentioned, check the commands pgrep -v statd and netstat -tlnpu to see if statd is running.

Or perhaps someone has changed something under /etc at the server side? If you do not have /etc under version control, see if any files have been recently modified: find /etc -mtime -14 would show files changed during last 14 days, for example.

Solution 2

Have a look here : http://sophiedogg.com/lockd-and-statd-nfs-errors/

Try :

# /etc/init.d/nfs-common stop
# /etc/init.d/nfs-kernel-server stop
# rm -rf /var/lib/nfs/statd/sm/*
# rm -rf /var/lib/nfs/statd/sm.bak/*
# /etc/init.d/nfs-common start
# /etc/init.d/nfs-kernel-server start

I had the same problem, and this solved it... but for only one month. I don't know why for now. I had to delete the files again today.

17,924

Emanuel V

Updated on September 18, 2022

Comments

Emanuel V almost 2 years
Since this afternoon something is wrong with the server. On the server side I see messages in dmesg as follows:
```
statd: server rpc.statd not responding, timed out
lockd: cannot unmonitor <client>
statd: server rpc.statd not responding, timed out
lockd: cannot monitor <client>
```
On the client side I see in dmesg:
```
lockd: server <server> not responding, still trying
lockd: server <server> OK
```
This is paralysing the entire network! I have tried this solution suggested by Xian, but it makes no difference.

Server, Debian Linux, Squeeze 64-bit:
```
>> uname -a
Linux <server> 2.6.32-5-amd64 #1 SMP Fri May 10 08:43:19 UTC 2013 x86_64 GNU/Linux
```
Clients, Linux Mint 13-64bit:
```
>> uname -a
Linux <client> 3.2.0-49-generic #75-Ubuntu SMP Tue Jun 18 17:39:32 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
```
I have not run an update on the server, so I don't know what could have changed. I did upgrade one of our client machines, but can't see why that would mess with the server, since all machines seem affected. Any ideas on how to fix this?

UPDATE 1

The server stalls for a while at
```
Starting portmap deamon
Starting NFS common utilities: statd idmapd
```
This takes about 2 minutes until boot continues...

UPDATE 2

It is indeed the client machine that was upgraded that caused this. It seems it somehow stalled statd on the server, causing all other machines to have issues. I rebooted the entire network, leaving that one machine off and I did not encountered any problems. Not really a fix, but I have since downgraded that machine again, and everything seems to be stable.
- David Schwartz almost 11 years
  
  Are you sure the physical network is okay? Switches? Cables?
- user16081-JoeT almost 11 years
  
  What does the server use for storage, is it mounting an external nfs mount, or is it serving nfs from local storage? check your available devices on the server with df, if it hangs your storage is probably offline / dead / unplugged.
- Emanuel V almost 11 years
  
  @user16081 The storage is internal raid drive. df comes back normal.
- Emanuel V almost 11 years
  
  @DavidSchwartz I believe the network is ok. I can still log into the server remotely, without disconnects. Not sure how else to test the network, as it is all locked away somewhere.
- user16081-JoeT almost 11 years
  
  noticing you upgraded 1 client, check this, last comment, to see if it might apply to your case? forums.gentoo.org/viewtopic-t-850532-start-0.html
- user16081-JoeT almost 11 years
  
  another possibility, broken rdns for a client machine's ip? bugs.launchpad.net/ubuntu/+source/nfs-utils/+bug/1089941
- Emanuel V almost 11 years
  
  @user16081 Both solutions seem promising! I am giving the second one a try, but I am unsure how to apply the first one?
- Emanuel V almost 11 years
  
  @user16081 Fix 1: removing the mdns4 from nsswitch.conf did not fix it, sadly.
- gokva almost 11 years
  
  is statd running? ps -ef | grep -v grep | grep statd, if so, which interfaces/ports does it have? netstat -lnp | grep rpc.statd, if not, start it, and paste back the output /sbin/rpc.statd -F -d
- gokva almost 11 years
  
  also grep for statd in whichever files your OS keeps it's system log; grep statd /var/log/messages*
- djhaskin987 over 9 years
  
  Having this issue too. From what I gather, I cannot rule network issues out. @DavidSchwartz is right -- "Not responding" Simply means "I mailed the server a letter and it hasn't replied back." It could be any number of network issues.