NFS: access denied by server while mounting. client side is a LXC proxmox(!)

Question

sorry if this topic duplicate some other topics but I can't find answer in them.

what do I have:

server is Ubuntu 14.04.5 LTS

client is Ubuntu 14.04.2 LTS - that is proxmox LXC

#cat /etc/exports

/nfs        192.168.88.0/24(ro,fsid=0,insecure,no_root_squash,no_subtree_check,sync)
/nfs/keycdn 192.168.88.0/24(rw,nohide,insecure,no_root_squash,no_subtree_check,sync)

If I run this command from server:

mount 192.168.88.185:/nfs/keycdn /mnt

It's mounted, and work - I can create a file

If I run on client side, I get an error:

root@kafka-02:/# mount 192.168.88.185:/nfs/keycdn /mnt
mount.nfs: access denied by server while mounting 192.168.88.185:/nfs/keycdn

I have such rights on a folder: server

#ll /nfs/
drwxrwxrwx  3 nobody nogroup 4096 Mar 27 13:49 ./
drwxr-xr-x 24 root   root    4096 Mar 27 12:41 ../
drwxrwxrwx  2 nobody nogroup 4096 Mar 27 15:17 keycdn/

To make effect after modifying /etc/exports I ran exportfs -rav and service nfs-kernel-server restart

I tried define vers of nfs in /etc/fstab on client side. but no effect

There are verbose on running mount command on server and client:

Server

mount -vvvvvv 192.168.88.185:/nfs/keycdn /mnt
mount: fstab path: "/etc/fstab"
mount: mtab path:  "/etc/mtab"
mount: lock path:  "/etc/mtab~"
mount: temp path:  "/etc/mtab.tmp"
mount: UID:        0
mount: eUID:       0
mount: no type was given - I'll assume nfs because of the colon
mount: spec:  "192.168.88.185:/nfs/keycdn"
mount: node:  "/mnt"
mount: types: "nfs"
mount: opts:  "(null)"
mount: external mount: argv[0] = "/sbin/mount.nfs"
mount: external mount: argv[1] = "192.168.88.185:/nfs/keycdn"
mount: external mount: argv[2] = "/mnt"
mount: external mount: argv[3] = "-v"
mount: external mount: argv[4] = "-o"
mount: external mount: argv[5] = "rw"
mount.nfs: timeout set for Mon Mar 27 15:51:33 2017
mount.nfs: trying text-based options 'vers=4,addr=192.168.88.185,clientaddr=192.168.88.185'
mount.nfs: mount(2): No such file or directory
mount.nfs: trying text-based options 'addr=192.168.88.185'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying 192.168.88.185 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=17
mount.nfs: trying 192.168.88.185 prog 100005 vers 3 prot UDP port 34598

Client vers=3

mount -o 'vers=3' -vvvvvvv 192.168.88.185:/nfs/keycdn /mnt
mount: fstab path: "/etc/fstab"
mount: mtab path:  "/etc/mtab"
mount: lock path:  "/etc/mtab~"
mount: temp path:  "/etc/mtab.tmp"
mount: UID:        0
mount: eUID:       0
mount: no type was given - I'll assume nfs because of the colon
mount: spec:  "192.168.88.185:/nfs/keycdn"
mount: node:  "/mnt"
mount: types: "nfs"
mount: opts:  "vers=3"
mount: external mount: argv[0] = "/sbin/mount.nfs"
mount: external mount: argv[1] = "192.168.88.185:/nfs/keycdn"
mount: external mount: argv[2] = "/mnt"
mount: external mount: argv[3] = "-v"
mount: external mount: argv[4] = "-o"
mount: external mount: argv[5] = "rw,vers=3"
mount.nfs: timeout set for Mon Mar 27 15:52:32 2017
mount.nfs: trying text-based options 'vers=3,addr=192.168.88.185'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying 192.168.88.185 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=17
mount.nfs: trying 192.168.88.185 prog 100005 vers 3 prot UDP port 34598
mount.nfs: mount(2): Permission denied
mount.nfs: access denied by server while mounting 192.168.88.185:/nfs/keycdn

vers=4

mount -o 'vers=4' -vvvvvvv 192.168.88.185:/nfs/keycdn /mnt
mount: fstab path: "/etc/fstab"
mount: mtab path:  "/etc/mtab"
mount: lock path:  "/etc/mtab~"
mount: temp path:  "/etc/mtab.tmp"
mount: UID:        0
mount: eUID:       0
mount: no type was given - I'll assume nfs because of the colon
mount: spec:  "192.168.88.185:/nfs/keycdn"
mount: node:  "/mnt"
mount: types: "nfs"
mount: opts:  "vers=4"
mount: external mount: argv[0] = "/sbin/mount.nfs"
mount: external mount: argv[1] = "192.168.88.185:/nfs/keycdn"
mount: external mount: argv[2] = "/mnt"
mount: external mount: argv[3] = "-v"
mount: external mount: argv[4] = "-o"
mount: external mount: argv[5] = "rw,vers=4"
mount.nfs: timeout set for Mon Mar 27 15:56:55 2017
mount.nfs: trying text-based options 'vers=4,addr=192.168.88.185,clientaddr=192.168.88.252'
mount.nfs: mount(2): Permission denied
mount.nfs: access denied by server while mounting 192.168.88.185:/nfs/keycdn

Solution

This answer help only if you have use LXC proxmox, but i think you should try his in other case too

The problem was on proxmox side. Proxmox have apparmor and that thing is blocking everything.

To stop apparmor and make him not work, do this steps

# /etc/init.d/apparmor stop
# /etc/init.d/apparmor teardown
# update-rc.d apparmor disable

after this steps everything start work like a charm