How can I make my user services wait till the network is online?

Question

I have written a couple of systemd user service files which I want users to enable and which need a working network connection. I thought that would be as easy as:

Wants=network-online.target
After=network-online.target

However, the services seem to start too early, and in journalctl I see:

network-online.target: Cannot add dependency job, ignoring: Unit network-online.target failed to load: No such file or directory.

Then I searched more and tried

Wants=network.target
After=network.target

and did sudo systemctl enable systemd-networkd-wait-online.service.

Now I have in journalctl:

network.target: Cannot add dependency job, ignoring: Unit network.target failed to load: No such file or directory.

And again the service starts too early.

Is that message supposed to be there? How can I debug my problem?

---

EDIT: the reason is simple and specifically stated in the Arch Wiki:

systemd --user runs as a separate process from the systemd --system process. User units can not reference or depend on system units.

This forum post seems to suggest a simple solution: I should link the necessary system unit as a user, thus creating a symlink to it available on the unit search path.

After doing that, I don't see any No such file or directory messages, however, I still can't make the services actually run after the network is online. I have tried linking network.target, network-online.target and systemd-networkd-wait-online.service, setting my units to depend on each of them, with no success. When I check the linked unit's status in user mode, they are all some of them are dead, e.g.:

$ systemctl --user status network.target
● network.target - Network
   Loaded: loaded (/usr/lib/systemd/system/network.target; linked; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:systemd.special(7)
           http://www.freedesktop.org/wiki/Software/systemd/NetworkTarget
$ systemctl status network.target
● network.target - Network
   Loaded: loaded (/usr/lib/systemd/system/network.target; static; vendor preset: disabled)
   Active: active since Sat 2015-07-18 19:20:11 MSK; 3h 35min ago
     Docs: man:systemd.special(7)
           http://www.freedesktop.org/wiki/Software/systemd/NetworkTarget

Jul 18 19:20:11 calc-server systemd[1]: Reached target Network.
Jul 18 19:20:11 calc-server systemd[1]: Starting Network.

However, I can see network-online.target active in user mode after linking it:

$ systemctl --user status network-online.target
● network-online.target - Network is Online
   Loaded: loaded (/usr/lib/systemd/system/network-online.target; linked; vendor preset: enabled)
   Active: active since Sun 2015-07-19 00:35:38 MSK; 2min 48s ago
     Docs: man:systemd.special(7)
           http://www.freedesktop.org/wiki/Software/systemd/NetworkTarget

Jul 19 00:35:38 calc-server systemd[469]: Reached target Network is Online.
Jul 19 00:35:38 calc-server systemd[469]: Starting Network is Online.

Solution 1

As this topic is No1 in google search results I share an alternate solution for all who will face the same problem.

In my system I added a simplified equivalent to /lib/systemd/system/systemd-networkd-wait-online.service (which is WantedBy=network-online.target) and stored it in user services directory:

[Unit]
Description=User Wait for Network to be Configured

[Service]
Type=oneshot
ExecStart=/lib/systemd/systemd-networkd-wait-online
RemainAfterExit=yes

[Install]
WantedBy=default.target

Then I made my user services to depend on this new one:

Wants=networkd-wait-online.service
After=networkd-wait-online.service

However, Wants= does not seem to be required due to WantedBy=default.target. You can remove WantedBy=default.target so this makes network-wait-online.service optional, but you certainly need Wants=network-wait-online.service then.

Original service systemd-networkd-wait-online.service contains more dependencies, but they do not seem to be working in user context. Someone may provide user running equivalents for all of these dependencies, e.g. by using systemctl --user link for network.target and network-online.target as in other suggestions here, but I decided to keep it simple ;)

Note that despite of making a symlink to system service you do NOT tell your systemd --user manager to use some events from systemd --system manager to watch on a system service state, but make it to run another instance of this service from user context.

Solution 2

Since you cannot depend on a system service, your only solution is to provide a user service that detects whether the network is online. (Or make your services system services.) Details for a "detect-online" user service would depend on your definition of "online". It could wait for a ping to 8.8.8.8 to succeed, for instance. Or for a DNS name resolution to succeed. For instance, in a similar situation with vpnc, I wait for a ping to a vpn IP to succeed.

Then you can make your user services depend on (After=) your detect-online user service.

#!/bin/sh

host="${1:-8.8.8.8}"

pingcheck() {
  ping -n -c 1 -w 5 $1 >/dev/null 2>&1
}

# Do you want a timeout ?
while :; do
  pingcheck ${host} && exit 0
  sleep 10
done

Solution 3

I would recommend to test something like this:

# /etc/systemd/system/foo.service
[Unit]
After=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/bin/logger -t foo "testing online target"

[Install]
WantedBy=multi-user.target

Followed by:

# systemctl daemon-reload && systemctl enable foo.service

Comments

[Admin]

I hope you'll have an answer.

[Lev Levitsky]

@Deleteme Thanks, I think I found the cause of this (see update), but not how to solve the problem.

[Sparhawk]

Did you every find a solution to this problem? Also, I notice you emphasise that only some linked units are dead. Which are the ones that worked, and how do they differ?

[Lev Levitsky]

@Sparhawk unfortunately not. As a workaround, I just use timers set to several seconds after boot. The question has an example: network-online target is active after linking, and network target is not.

[Sparhawk]

Perhaps I misunderstand, but I'm looking for something to fire every time the network resumes, to check my email after resuming from suspend. I thought this would be the way to do it. I see the examples now. I thought you meant some custom services fired and some did not. However, I see now you are talking about the user versions of the network targets.

[Lev Levitsky]

@TCB13 I think that's pretty much what I did, except on my system network.target is at /usr/lib/systemd/system/network.target.

[TCB13]

@LevLevitsky Something seems to be wrong here, because with cat I get the file on the path, however systemd can't read it, look at this: pastebin.com/RpsbP3qg

[Lev Levitsky]

@TCB13 You are running systemctl --user as root, I'm not sure how that should work. Running it as user works for me (creates a symlink).

[TCB13]

@LevLevitsky that's a good point.

[Louis]

On arch: systemctl --user link /usr/lib/systemd/system/network-online.target

[Lev Levitsky]

But this is a system service, and I am trying to set up a user service.

[Lev Levitsky]

Thank you, this seems reasonable and simple enough. But if 10 users enable my user service, there will be 10 instances pinging the same server simultaneously, which seems a bit redundant. In practice this is probably not at issue, but I was asking this in the hope of getting a single "online flag" available to all user services. No better solutions have yet been suggested, though.