Cannot use `systemctl --user` due to "Failed to get D-bus connection: permission denied"

Question

I'm trying to set up user-level services, using this answer to a similar question. I have create the required files and rebooted.

I'm making progress because I now get "Failed to get D-bus connection: permission denied" when it was "Failed to get D-bus connection: connection refused", but I'm stumped because I don't know what object it is trying to access (file? socket?) and so cannot even check current permissions. Any ideas?

So far I have added:

loginctl enable-linger userservice

/usr/lib/systemd/user/dbus.service (-rw-r--r-- root root)

[Unit]
Description=D-Bus User Message Bus
Requires=dbus.socket

[Service]
ExecStart=/usr/bin/dbus-daemon --session --address=systemd: --nofork --nopidfile --systemd-activation
ExecReload=/usr/bin/dbus-send --print-reply --session --type=method_call --dest=org.freedesktop.DBus / org.freedesktop.DBus.ReloadConfig

[Install]
Also=dbus.socket

/usr/lib/systemd/user/dbus.socket (-rw-r--r-- root root)

[Unit]
Description=D-Bus User Message Bus Socket

[Socket]
ListenStream=%t/bus
ExecStartPost=-/bin/systemctl --user set-environment DBUS_SESSION_BUS_ADDRESS=unix:path=%t/bus

[Install]
WantedBy=sockets.target
Also=dbus.service

/home/userservice/.config/systemd/user/userservice.service

[Unit]
Description=Test user-level service

[Service]
Type=dbus
BusName=com.wtf.service
ExecStart=/home/userservice/userservice.py
Restart=on-failure

[Install]
WantedBy=default.target

Not added any links elsewhere...

To make it fail:

systemctl --user status

Edit 2018-10-25:

Added export XDG_RUNTIME_DIR=/run/user/$(id -u) to .bashrc. The variable is set and now I get: Failed to get D-us connection: no such file or directory. Strangely, neither man systemctl nor systemctl --help mention the --user option, while both mention --system and specify that this is the default (so what are the other options).

Using RHEL 7.4 (with systemd 219 as reported by systemctl --version) with SELinux.

Solution 1

I've noticed the tableau server uses --user systemd services - they even have a note about this in their docs: https://help.tableau.com/current/server-linux/en-us/systemd_user_service_error.htm

The systemd user service is not used as commonly as the normal systemd process manager. Red Hat disabled the systemd user service in RHEL 7 (and thereby all distros that come from RHEL, like CentOS, Oracle Linux 7, Amazon Linux 2). However, RedHat has assured Tableau that running the systemd user service is supported as long as the service is re-enabled.

How they do it (example is with a userid 29575)

# cat /etc/systemd/system/[email protected]
[Unit]
Description=User Manager for UID %i
After=systemd-user-sessions.service
# These are present in the RHEL8 version of this file except that the unit is Requires, not Wants.
# It's listed as Wants here so that if this file is used in a RHEL7 settings, it will not fail.
# If a user upgrades from RHEL7 to RHEL8, this unit file will continue to work until it's
# deleted the next time they upgrade Tableau Server itself.
After=user-runtime-dir@%i.service
Wants=user-runtime-dir@%i.service

[Service]
LimitNOFILE=infinity
LimitNPROC=infinity
User=%i
PAMName=systemd-user
Type=notify
# PermissionsStartOnly is deprecated and will be removed in future versions of systemd
# This is required for all systemd versions prior to version 231
PermissionsStartOnly=true
ExecStartPre=/bin/loginctl enable-linger %i
ExecStart=-/lib/systemd/systemd --user
Slice=user-%i.slice
KillMode=mixed
Delegate=yes
TasksMax=infinity
Restart=always
RestartSec=15

[Install]
WantedBy=default.target

After you create that file:

systemctl daemon-reload
systemctl enable [email protected]
systemctl start [email protected]

And you'll need to set XDG_RUNTIME_DIR in env of that user via bashrc or similar:

[ -z "${XDG_RUNTIME_DIR}" ] && export XDG_RUNTIME_DIR=/run/user/$(id -ru)

I've tested in on a recent RHEL 7.8 and it works as expected, I can run "systemctl --user status" as my user after doing this.

Solution 2

So there's a long standing issue where the XDG_RUNTIME_DIR environment variable doesn't get set properly, or at all, when users log in, and therefore can't access the user D-Bus. This happens when the user logs in via some other method than the local graphical console.

You can work around this by adding to the user's $HOME/.bashrc:

export XDG_RUNTIME_DIR=/run/user/$(id -u)

Then log out and back in.

Solution 3

You should be aware of how the PAM works.

If you login in the system using either of

1. graphical session
2. login on terminal (username and password)
3. ssh

then the PAM machinery will call pam_systemd, and this will setup all needed hooks to use systemctl; if you switch user using sudo or su, this will not happen.

This is deliberate , see https://github.com/systemd/systemd/issues/7451#issuecomment-346787237

What's weird is that actually /etc/pam.d/su includes /etc/pam.d/common-session that contains an invocation of pam_systemd but this fails, indeed the /var/log/auth.log says:

pam_systemd(su:session): Cannot create session: Already running in a session

Currently, a good way is to use ssh to connect to the same machine but as another user, something like ssh user@localhost.

Comments

[Michael Hampton]

Can you update the system? RHEL 7.5 is current. And remember that RHEL requires a subscription; do not attempt to use it without one.

[xenoid]

Stuck on 7.4 for the time being ('recently updated from 7.2). The behemoth organization the server belongs to has all the required licenses, but is somewhat upgrade-shy.

[Michael Hampton]

You've enabled linger for a user named userservice. You need to actually log in to this account to run systemctl --user commands. If these services are going to be managed by the admin, rather than by the user, you should create normal system services, not user services.

[xenoid]

I'm using systemctl --user as the userservice user...

[Michael Hampton]

While logged in as userservice, check the value of the XDG_RUNTIME_DIR environment variable: echo $XDG_RUNTIME_DIR

[xenoid]

Not set for userservice... I see that when I'm root, it points to /run/user/0 and there is indeed a /run/user/10010 (which is the uid of userservice...)

[Giraffe]

For me sudo systemctl --user status xx failed with "unable to connect to bus" because it was sudo but attempting to examine user services. Needed to not run it as sudo.

[Beko Pharm]

bashrc is afair not parsed by user systemd. That's login shell specific. You have to set environment for systemd services with EnvironmentFile or Environment. It's designed that you start with an untainted env. See freedesktop.org/software/systemd/man/systemd.exec.html

[kkm]

To answer your question where is the dbus user socket, look at user's dbus.socket file in your sample: ListenStream=%t/bus. %t stands for the aame value as $XDG_RUNTIME_DIR, but I think it's recognized even without the variable being set. If I'm mistaken, no luck: %t/bus turns into /bus, and the user hardly has a permission to create sockets in /.

[xenoid]

Doesn't seem to be sufficient, but I'm starting to wonder if systemd 219 is enough to support user services. Please see augmented question.

[Michael Hampton]

@xenoid Just to be clear, you have rebooted the system, right? Or at least restarted systemd-logind?

[xenoid]

Yes, rebooted several times since. But I wonder if we are looking at the right things, since /run/user/10010 remains empty (and /run/user/0 is empty too).

[Mikhail T.]

Thank you very much, @Klaas - this is exactly, what I was looking for. Pulling my hairs out trying to understand, how to get systemd --user to run on RHEL7. I'd add, that using of numeric IDs is not necessary -- you can use the actual user name and the %i inside will still be substituted accordingly. Numeric IDs may be faster -- or safer -- if your user-mapping is network based, but they are ugly :(

[Klaas]

for completness: I am not sure this is actually supported: access.redhat.com/solutions/3461241 bugzilla.redhat.com/show_bug.cgi?id=1173278#c1

[Joseph Van Riper]

This also works with Amazon Linux instances.