How do I override or configure systemd services?

Question

Many sysv init scripts used a corresponding file in /etc/default to allow the administrator to configure it. Upstart jobs can be modified using .override files. How do I override or configure systemd units, now that systemd is the default in Ubuntu?

Solution

systemd units need not obey files in /etc/default. systemd is easily configurable, but requires that you know the syntax of systemd unit files.

Packages ship unit files typically in /lib/systemd/system/. These are not to be edited. Instead, systemd allows you to override these files by creating appropriate files in /etc/systemd/system/.

For a given service foo, the package would provide /lib/systemd/system/foo.service. You can check its status using systemctl status foo, or view its logs using journalctl -u foo. To override something in the definition of foo, do:

sudo systemctl edit foo

This creates a directory in /etc/systemd/system named after the unit, and an override.conf file in that directory (/etc/systemd/system/foo.service.d/override.conf). You can add or override settings using this file (or other .conf files in /etc/systemd/system/foo.service.d/). This is also applicable to non-service units - you could do systemctl edit foo.mount, systemctl edit foo.timer, etc.

Overriding command arguments

Take the getty service for example. Say I want to have TTY2 autologin to my user (this is not advisable, but just an example). TTY2 is run by the getty@tty2 service (tty2 being an instance of the template /lib/systemd/system/getty@service). To do this, I have to modify the getty@tty2 service.

$ systemctl cat getty@tty2
# /lib/systemd/system/[email protected]
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Unit]
Description=Getty on %I
Documentation=man:agetty(8) man:systemd-getty-generator(8)
Documentation=http://0pointer.de/blog/projects/serial-console.html
After=systemd-user-sessions.service plymouth-quit-wait.service
After=rc-local.service

# If additional gettys are spawned during boot then we should make
# sure that this is synchronized before getty.target, even though
# getty.target didn't actually pull it in.
Before=getty.target
IgnoreOnIsolate=yes

# On systems without virtual consoles, don't start any getty. Note
# that serial gettys are covered by [email protected], not this
# unit.
ConditionPathExists=/dev/tty0

[Service]
# the VT is cleared by TTYVTDisallocate
ExecStart=-/sbin/agetty --noclear %I $TERM
Type=idle
Restart=always
RestartSec=0
UtmpIdentifier=%I
TTYPath=/dev/%I
TTYReset=yes
TTYVHangup=yes
TTYVTDisallocate=yes
KillMode=process
IgnoreSIGPIPE=no
SendSIGHUP=yes

# Unset locale for the console getty since the console has problems
# displaying some internationalized messages.
Environment=LANG= LANGUAGE= LC_CTYPE= LC_NUMERIC= LC_TIME= LC_COLLATE= LC_MONETARY= LC_MESSAGES= LC_PAPER= LC_NAME= LC_ADDRESS= LC_TELEPHONE= LC_MEASUREMENT= LC_IDENTIFICATION=

[Install]
WantedBy=getty.target
DefaultInstance=tty1

In particular, I have to change the ExecStart line, which currently is:

$ systemctl cat getty@tty2 | grep Exec     
ExecStart=-/sbin/agetty --noclear %I $TERM

To override this, do:

sudo systemctl edit getty@tty2

And add:

[Service]
ExecStart=
ExecStart=-/sbin/agetty -a muru --noclear %I $TERM

Note that:

1. I had to explicitly clear ExecStart before setting it again, as it is an additive setting, similar to other lists like Environment (as a whole, not per-variable) and EnvironmentFile; and opposed to overriding settings like RestartSec or Type. ExecStart can have multiple entries only for Type=oneshot services. Note that dependency settings like Before, After, Wants, etc. are also lists, but cannot be cleared using this way. You'll have to override/replace the entire service for that (see below).
2. I had to use the proper section header. In the original file, ExecStart is in the [Service] section, so my override has to put ExecStart in the [Service] section as well. Often, having a look at the actual service file using systemctl cat will tell you what you need to override and which section it is in.

Usually, if you edit a systemd unit file, for it to take effect, you need to run:

sudo systemctl daemon-reload

However, systemctl edit automatically does this for you.

Now:

$ systemctl cat getty@tty2 | grep Exec
ExecStart=-/sbin/agetty --noclear %I $TERM
ExecStart=
ExecStart=-/sbin/agetty -a muru --noclear %I $TERM

$ systemctl show getty@tty2 | grep ExecS
ExecStart={ path=/sbin/agetty ; argv[]=/sbin/agetty -a muru --noclear %I $TERM ; ... }

And if I do:

sudo systemctl restart getty@tty2

and press CtrlAltF2, presto! I'll be logged into my account on that TTY.

As I said before, getty@tty2 is an instance of a template. So, what if I wanted to override all instances of that template? That can be done by editing the template itself (removing the instance identifier - in this case tty2):

systemctl edit getty@

Overriding the environment

A common use case of /etc/default files is setting environment variables. Usually, /etc/default is a shell script, so you could use shell language constructs in it. With systemd, however, this is not the case. You can specify environment variables in two ways:

Via a file

Say you have set the environment variables in a file:

$ cat /path/to/some/file
FOO=bar

Then, you can add to the override:

[Service]
EnvironmentFile=/path/to/some/file

In particular, if your /etc/default/grub contains only assignments and no shell syntax, you could use it as the EnvironmentFile.

Via Environment entries

The above could also be accomplished using the following override:

[Service]
Environment=FOO=bar

However, this can get tricky with multiple variables, spaces, etc. Have a look at one of my other answers for an example of such an instance.

Variations in editing

Replacing the existing unit entirely

If you want to make massive changes to the existing unit, such that you're effectively replacing it altogether, you could just do:

systemctl edit --full foo

Temporary edits

In the systemd file hierarchy, /run takes precedence over /etc, which in turn takes precedence over /lib. Everything said so far also applies to using /run/systemd/system instead of /etc/systemd/system. Typically /run is a transient filesystem whose contents are lost on reboot, so if you want to override a unit only until reboot, you can do:

systemctl edit --runtime foo

Undoing changes

You can simply remove the corresponding override file, and do systemctl daemon-reload to have systemd read the updated unit definition.

You can also revert all changes:

systemctl revert foo

Further Reading

Via this mechanism, it becomes very easy to override systemd units, as well as to undo such changes (by simply removing the override file). These are not the only settings which can be modified.

The following links would be useful:

• Arch Wiki entry on systemd
• systemd for Administrators, Part IX: On /etc/sysconfig and /etc/default (by the lead developer of systemd, Lennart Poettering)
• The systemd manpages, in particular, the manpages of systemd.unit and systemd.service
• Ubuntu Wiki entry on Systemd for Upstart users

Comments

[tysik]

Note that when clearing the ExecStart= with a blank entry you cannot put a comment after it like: ExecStart= # Empty line to clear previous entries. This will be taken as another ExecStart= entry and added to the list. PS. I could not add comment to muru's answer because of my low reputation.

[Colin]

You have to clear the variable before setting it for services not of the type oneshot. This solved my problem.

[Mark Edington]

Can anyone point me to where in the systemd docs I can find information on "clearing" the inherited value with ExecStart= as shown in this answer?

[muru]

@MarkEdington from the systemd.service(5) manpage, section on ExecStart: "Unless Type= is oneshot, exactly one command must be given. When Type=oneshot is used, zero or more commands may be specified. Commands may be specified by providing multiple command lines in the same directive, or alternatively, this directive may be specified more than once with the same effect. If the empty string is assigned to this option, the list of commands to start is reset, prior assignments of this option will have no effect."

[Tomilov Anatoliy]

How to remove override using systemctl? I can't just empty file.

[muru]

@Orient you can sudo rm the override file and then systemctl daemon-reload, or you can systemctl edit and replace everything in the override with comments. Comments in service files begin with #.

[Ayell]

@Orient systemctl revert foo

[Nikolaos Kakouros]

What is the order of precedence for the three methods (override file, environment file, environment variable)? Ie, for a variable defined in all three, which value will be the effective one?

[CMCDragonkai]

If I want to override ExecStart by adding/changing some of the command line parameters. How do I do this if I don't know the path to the executable? Do I have to copy the path to the executable or is there some way of preserving it? Also if it is an additive setting, it seems to mean that one should create a "default" command that can have additional settings applied simply be appending more options that override earlier options?

[muru]

@CMCDragonkai yes, you have to know the path - which you can get easily enough by using systemctl cat or systemctl show. It's additive in the sense that multiple instances are added to a list. They don't add to an item in that list.

[xenoterracide]

is the double ExecStart in the override actually necessary, or is that just to hide the previous value in systemctl show?

[muru]

@xenoterracide depends on if you want to replace the previous value or just add another command to be executed.

[Krisztián Szegi]

Excellent answer, except one part - it makes you believe you can reset After, based on the line "I had to explicitly clear ExecStart before setting it again, as it is an additive setting, similar to After..." Although it is a list/additive setting - as I had to found out from elsewhere - you cannot reset dependency type settings...

[muru]

@KrisztiánSzegi ah, noted, and I have updated the text accordingly. I now see that this is specified explicitly in the last para of Ex 2 In man systemd.unit

[Krisztián Szegi]

@muru Nice! Thanks for the time clearing this up!