HAProxy works fine ran manually, but as a Systemd-service fails to load SSL pem-file

Question

I'm trying to set up SSL on a Fedora 22 -server running HAProxy. Here's the config (/etc/haproxy/haproxy.cfg):

global
    log         127.0.0.1 local2
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon
    tune.ssl.default-dh-param 2048

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    option  http-server-close
    option  forwardfor
    option  redispatch
    retries 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s

frontend main
    bind *:80
    bind *:443 ssl crt /etc/haproxy/certificate.pem
    redirect scheme https if !{ ssl_fc }
    default_backend app

backend app
    balance roundrobin
    server app1 127.0.0.1:8000 check

Now, when I run systemctl restart haproxy && journalctl -u haproxy.service -f, I get this error:

Sep 13 15:39:31 fedora-server systemd[1]: Started HAProxy Load Balancer.
Sep 13 15:39:31 fedora-server systemd[1]: Starting HAProxy Load Balancer...
Sep 13 15:39:31 fedora-server haproxy-systemd-wrapper[15620]: haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
Sep 13 15:39:31 fedora-server haproxy-systemd-wrapper[15620]: [ALERT] 255/153931 (15621) : parsing [/etc/haproxy/haproxy.cfg:30] : 'bind *:443' : unable to load SSL private key from PEM file '/etc/haproxy/certificate.pem'.
Sep 13 15:39:31 fedora-server haproxy-systemd-wrapper[15620]: [ALERT] 255/153931 (15621) : Error(s) found in configuration file : /etc/haproxy/haproxy.cfg
Sep 13 15:39:31 fedora-server haproxy-systemd-wrapper[15620]: [ALERT] 255/153931 (15621) : Proxy 'main': no SSL certificate specified for bind '*:443' at [/etc/haproxy/haproxy.cfg:30] (use 'crt').
Sep 13 15:39:31 fedora-server haproxy-systemd-wrapper[15620]: [ALERT] 255/153931 (15621) : Fatal errors found in configuration.
Sep 13 15:39:31 fedora-server haproxy-systemd-wrapper[15620]: haproxy-systemd-wrapper: exit, haproxy RC=256

Here's the service config:

# cat /usr/lib/systemd/system/haproxy.service
[Unit]
Description=HAProxy Load Balancer
After=syslog.target network.target

[Service]
EnvironmentFile=/etc/sysconfig/haproxy
ExecStart=/usr/sbin/haproxy-systemd-wrapper -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid $OPTIONS
ExecReload=/bin/kill -USR2 $MAINPID

[Install]
WantedBy=multi-user.target

However, I can copy the command that the service attempts to run, and it works fine.

First, this works, when I run it manually (taken from service config):

# whoami
root
# /usr/sbin/haproxy-systemd-wrapper -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid

This also works (taken from service logs, apparently haproxy-systemd-wrapper runs this:

# whoami
root
# /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds

To clarify: when I run either of these commands manually as root, they work and I can access my site via SSL.

So, I'm assuming when ran as a service HAProxy isn't able to read the certificate.

Here's what I've tried so far:

• chown haproxy:haproxy /etc/haproxy/certificate.pem
• Switching user and group to root in haproxy.cfg
• Adding User=root and Group=root to service configuration under [Service]
• sudo -u haproxy /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds gives a different error (regarding ports), so most likely user haproxy isn't used by the service, otherwise would get further

I can access the file as user haproxy, sudo -u haproxy cat /etc/haproxy/certificate.pem.

Edit:

Here's the updated service config based on your answer:

[Unit]
Description=HAProxy Load Balancer
After=syslog.target network.target
SELinuxContext=unconfined_u:object_r:var_t:s0

What happens now:

Sep 13 17:51:21 ServiceName systemd[1]: Starting HAProxy Load Balancer...
Sep 13 17:51:21 ServiceName systemd[1]: haproxy.service: main process exited, code=exited, status=203/EXEC
Sep 13 17:51:21 ServiceName systemd[1]: Unit haproxy.service entered failed state.
Sep 13 17:51:21 ServiceName systemd[1]: haproxy.service failed.

How I got the value for SELinuxContext:

# ls -lZ /etc/haproxy/certificate.pem
-rw-r--r--. 1 root root unconfined_u:object_r:var_t:s0 9245 Sep 13 17:43 /etc/haproxy/certificate.pem

Solution

You're on Fedora, which uses SELinux. Systemd services are run in a clean environment – they're not spawned by systemctl directly – so among other things they start in a different SELinux context (I think the default is system_u:system_r:init_t:s0?).

Make sure that the correct SELinux context is set, both for the certificate file (using ls -lZ and chcon) and for the haproxy process (possibly using SELinuxContext= in the systemd unit).

Try running id to see your (or the service's) current context.

Comments

[Martti Laine]

Thanks for the reply! I tried it out, with no luck, see the edit.

[Martti Laine]

Actually, might have got it by doing something (last thing I did was restorecon for the certificate). Can test properly after a DNS update, so will comment after that.

[user1686]

SELinuxContext= belongs to the [Service] section, by the way.