NFS4 permission denied when userid does not match (even though idmap is working)

Question

I have NFS4 setup with idmapd working correctly. ls -l from the client shows the correct user names, even though the user ids differ between the machines.

However, when the user ids do not match, I get 'permission denied' errors trying access files, even though ls -l shows the correct username. When the user ids do happen to match by coincidence, everything works fine.

sudo sysctl -w sunrpc.nfsd_debug=1023 gives the following output in the server syslog for the failed file access:

nfsd_dispatch: vers 4 proc 1
nfsv4 compound op #1/3: 22 (OP_PUTFH)
nfsd: fh_verify(28: 00070001 015c0001 00000000 9853d400 2a4892a5 4918a0ba)
nfsv4 compound op ffff88003d0f5078 opcnt 3 #1: 22: status 0
nfsv4 compound op #2/3: 3 (OP_ACCESS)
nfsd: fh_verify(28: 00070001 015c0001 00000000 9853d400 2a4892a5 4918a0ba)
nfsd: fh_verify - just checking
nfsv4 compound op ffff88003d0f5078 opcnt 3 #2: 3: status 0
nfsv4 compound op #3/3: 9 (OP_GETATTR)
nfsd: fh_verify(28: 00070001 015c0001 00000000 9853d400 2a4892a5 4918a0ba)
nfsd: fh_verify - just checking
nfsv4 compound op ffff88003d0f5078 opcnt 3 #3: 9: status 0
nfsv4 compound returned 0
nfsd_dispatch: vers 4 proc 1
nfsv4 compound op #1/7: 22 (OP_PUTFH)
nfsd: fh_verify(28: 00070001 015c0001 00000000 9853d400 2a4892a5 4918a0ba)
nfsv4 compound op ffff88003d0f5078 opcnt 7 #1: 22: status 0
nfsv4 compound op #2/7: 32 (OP_SAVEFH)
nfsv4 compound op ffff88003d0f5078 opcnt 7 #2: 32: status 0
nfsv4 compound op #3/7: 18 (OP_OPEN)
NFSD: nfsd4_open filename dom_file op_stateowner (null)
renewing client (clientid 4f96587d/0000000e)
nfsd: nfsd_lookup(fh 28: 00070001 015c0001 00000000 9853d400 2a4892a5 4918a0ba, dom_file)
nfsd: fh_verify(28: 00070001 015c0001 00000000 9853d400 2a4892a5 4918a0ba)
nfsd: fh_verify - just checking
nfsd: fh_lock(28: 00070001 015c0001 00000000 9853d400 2a4892a5 4918a0ba) locked = 0 
nfsd: fh_compose(exp 08:01/22806529 srv/dom_file, ino=22809724)
nfsd: fh_verify(36: 01070001 015c0001 00000000 9853d400 2a4892a5 4918a0ba)
nfsd: fh_verify - just checking
fh_verify: srv/dom_file permission failure, acc=804, error=13
nfsv4 compound op ffff88003d0f5078 opcnt 7 #3: 18: status 13
nfsv4 compound returned 13

Is that useful to anyone? Any hints on to debug this would be greatly appreciated.

Server kernel: 2.6.32-40-server (Ubuntu 10.04)
Client kernel: 3.2.0-27-generic (Ubuntu 12.04)

Same problem with my new server running 3.2.0-27-generic (Ubuntu 12.04).

Thanks.

Solution

NFSv4 uses utf8 string principals between client and server. As a result, it's sufficient to use the same user names and nfs4 domain on client and server. The uids can be differ. BUT.... If you use AUTH_SYS (mount with sec=sys, which is default) your RPC requests instead of kerberos principal will use uid and gids from the client host. In this case, if uids on client and server differ, you will get access deny. In case of RPCGSS_SEC, your kerberos principal is send and on the server will use the same idmapd to map it to local uid and gids. ls -l works as expected as it uses idmapd to map files owner and group ids to principals which later on send to the client.

Comments

[SystemParadox]

thread.gmane.org/gmane.linux.nfsv4/7103/focus=7105 seems to imply that idmapd doesn't serve the purpose most people think it does, but I'm not convinced

[Tim]

It's really incredible, but idmapd really doesn't do that! It took me 2 days to find that on the web (ie your link to gmane, or this one) and another 2 days to believe that someone could have created such a surprisingly confusing mess. no, talking about uid mapping, having ls -l print out mapped uids but letting it fail when accessing those files... jeee, tststs, headbang, no way!