Systemd fails to start openvpn in lxd managed 16.04 container

Question

openvpn starts fine from the command line using the exact ExecStart= call from the systemd unit file:

/usr/sbin/openvpn --daemon ovpn-server --status /run/openvpn/server.status 10 --cd /etc/openvpn --script-security 2 --config /etc/openvpn/server.conf /run/openvpn/server.pid

ps ax confirms that the process is there:

1634 ?        Ss     0:00 /usr/sbin/openvpn --daemon ovpn-server --status /run/openvpn/server.status 10 --cd /etc/openvpn --script-security 2 --config /etc/openvpn/server.conf --writepid /run/openvpn/

But systemctl start openvpn@server is not successful:

● [email protected] - OpenVPN connection to server
   Loaded: loaded (/lib/systemd/system/[email protected]; disabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 2016-03-17 09:54:52 UTC; 4s ago
     Docs: man:openvpn(8)
           https://community.openvpn.net/openvpn/wiki/Openvpn23ManPage
           https://community.openvpn.net/openvpn/wiki/HOWTO
  Process: 1679 ExecStart=/usr/sbin/openvpn --daemon ovpn-server --status /run/openvpn/server.status 10 --cd /etc/openvpn --script-security 2 --config /etc/openvpn/server.conf --writepid /run/openvpn/s
 Main PID: 819 (code=exited, status=1/FAILURE)

Mar 17 09:54:52 vpn ovpn-server[1679]:   push_ifconfig_ipv6_remote = ::
Mar 17 09:54:52 vpn ovpn-server[1679]:   enable_c2c = DISABLED
Mar 17 09:54:52 vpn ovpn-server[1679]:   duplicate_cn = DISABLED
Mar 17 09:54:52 vpn ovpn-server[1679]:   cf_max = 0
Mar 17 09:54:52 vpn ovpn-server[1679]:   cf_per = 0
Mar 17 09:54:52 vpn ovpn-server[1679]:   max_clients = 1024
Mar 17 09:54:52 vpn systemd[1]: [email protected]: Control process exited, code=exited status=1
Mar 17 09:54:52 vpn systemd[1]: Failed to start OpenVPN connection to server.
Mar 17 09:54:52 vpn systemd[1]: [email protected]: Unit entered failed state.
Mar 17 09:54:52 vpn systemd[1]: [email protected]: Failed with result 'exit-code'.

I interpret journalctl | grep ovpn-server | tail -n 100 that it failed to fork:

Mar 17 09:57:44 vpn ovpn-server[1693]: OpenVPN 2.3.10 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [MH] [IPv6] built on Feb  2 2016
Mar 17 09:57:44 vpn ovpn-server[1693]: library versions: OpenSSL 1.0.2g  1 Mar 2016, LZO 2.08
Mar 17 09:57:44 vpn ovpn-server[1693]: daemon() failed or unsupported: Resource temporarily unavailable (errno=11)
Mar 17 09:57:44 vpn ovpn-server[1693]: Exiting due to fatal error

Solution 1

I've been looking for a fix for this also. What I have found to work is to comment out the LimitNPROC line in /lib/systemd/system/[email protected].

Don't forget to run systemctl daemon-reload after that.

Solution 2

It's better to avoid modyfying systemd units originating from system packages. Just use systemd override drop-in:

systemctl edit openvpn@

Unit name for openvpn server might be different, eg. for package version 2.4.5-xenial0 it will be

systemctl edit openvpn-server@

Then put into editor:

[Service]
LimitNPROC=infinity

It should create /etc/systemd/system/[email protected]/override.conf file (or similar for corresponding unit name). To activate it you may want to reload systemd with

systemctl daemon-reload

If starting the unit is successful with LimitNPROC=infinity then go back and change the limit to a higher value than the default of 10. The limit is important to prevent the service from consuming all available pids which can cause a denial of service.

Credits:
override: https://unix.stackexchange.com/a/398541/218321
"unlimited limit": https://unix.stackexchange.com/a/345596/218321

Solution 3

I also had to add those bellow their respective tun brother

DeviceAllow=/dev/net/tap rw
DeviceAllow=/dev/net/tap1 rw

to run it on L2 level.

Comments

[Piotr Kula]

This issue happened for me on OpenVZ with Ububtu 16.4 - By commenting out LimitNPROC as in the answer, the service started fine.

[Christian David]

Thank you! How did you diagnose the issue? Even knowing the answer, googling for LimitNPROC lxd wouldn't have returned a hit that is immediately helpful to me.

[Iain]

I pulled a 15.10 lxd instance and started comparing the systemd config. The LimitNPROC wasn't suspicious at first but it didn't exist in the older config so I tried removing it and it solved it. I'd been tearing my hair out over it for a day by that point.

[Quentin Skousen]

Thanks, this fixed my problem too! I did have to run systemctl daemon-reload before it took effect.

[user1338062]

Has this been reported?

[Raghavendra P]

Someone reported this as github.com/lxc/lxd/issues/3336

[David Foerster]

Welcome to Ask Ubuntu! I recommend to edit this answer to expand it with specific details about how to do this. (See also How do I write a good answer? for general advice about what sorts of answers are considered most valuable on Ask Ubuntu.)

[Stuart Cardall]

this also fixes openvpn inside openvz containers

[WoJ]

This is a better solution than the accepted one. I just upgraded my container and openvpn stopped to work. This was because I updated the stock systemd file instead of the override.

[Jaime Hablutzel]

Note that the usage of openvpn@ is deprecated as indicated in community.openvpn.net/openvpn/wiki/…

[sergtech]

This is the method that seems to work on my lxc (proxmox) (unprivileged) (18.04.5) container as the editing of the openvpn@ did not solve the problem