Have SystemD spawn N processes?

11,945

Solution 1

What Munir mentioned is exactly how you do this. Basically you create a service file, and start it 30 times. Now that may seem a little unweildy, but it has advantages, such as being able to shut one of them down if it's misbehaving, and not have to shut all of them down. There are also some things you can do to make management easier.

First, the unit file. Create a file, such as /etc/systemd/system/[email protected]. The important bit is the @ symbol.

It's contents might look like:

[Service]
ExecStart=/bin/sleep 600 %I

[Install]
WantedBy=multi-user.target

Then start it with systemctl start [email protected], systemctl start [email protected].
The processes that get launched will look like:

root     17222  19   0  0.0  0.0 Ss         00:05 /bin/sleep 600 1
root     17233  19   0  0.0  0.0 Ss         00:02 /bin/sleep 600 2

Notice that the %I got substituted with whatever you put after the @ when you started it.

You can start all 30 with a little shell-fu:

systemctl start test@{1..30}.service

You can also enable them at boot like any normal service: systemctl enable [email protected].

 

Now, what I meant by things you can do to make management easier: Maybe you don't want to have to use test@{1..30}.service for managing them all. It is a little unwieldy. You can instead create a new target for your service.

Create /etc/systemd/system/test.target with:

[Install]
WantedBy=multi-user.target

Then adjust the /etc/systemd/system/[email protected] so that it looks like:

[Unit]
StopWhenUnneeded=true

[Service]
ExecStart=/bin/sleep 600 %I

[Install]
WantedBy=test.target

Reload systemd with systemctl daemon-reload (only necessary if you are modifying the unit file, and didn't skip the earlier version of it). And now enable all the services you want to be managed by doing systemctl enable test@{1..30}.service.
(If you had previously enabled the service while it had WantedBy=multi-user.target, disable it first to clear out the dependency)

You can now do systemctl start test.target and systemctl stop test.target, and it will start/stop all 30 processes.
And again, you can enable at boot like any other unit file: systemctl enable test.target.

Solution 2

Here is my example using a python script that runs in a virtualenv:

/etc/systemd/system/[email protected]

[Unit]
Description=manages my worker service, instance %i
After=multi-user.target

[Service]
PermissionsStartOnly=true
Type=idle
User=root
ExecStart=/usr/local/virtualenvs/bin/python /path/to/my/script.py
Restart=always
TimeoutStartSec=10
RestartSec=10

Disable: sudo systemctl disable my-worker\@{1..30}.service

Enable N workers: sudo systemctl enable my-worker\@{1..2}.service

Reload: sudo systemctl daemon-reload

Start: sudo systemctl start [email protected]

Check status: sudo systemctl status my-worker@1

Share:
11,945

Related videos on Youtube

Naftuli Kay
Author by

Naftuli Kay

Updated on September 18, 2022

Comments

  • Naftuli Kay
    Naftuli Kay about 1 year

    At my organization, we have a number of queue consuming worker processes. We're currently using SupervisorD to manage them, but would like to use SystemD if possible for certain advantages therein. I'm fairly experienced with writing custom units, but I don't immediately have an analog in SystemD land for this.

    In the SupervisorD documentation a parameter called numprocs is detailed which allows one to set the number of processes they'd like to be started with the service. If I want 30 processes started, it's a one-line change.

    Is there a setting in SystemD units that will allow me to specify how many of these processes I'd like started?

  • bastian
    bastian almost 6 years
    Do you know if that will restart processes that fail?
  • siride
    siride over 5 years
    @bastian: you can use Restart=on-failure. Read the systemd.service man page for more.
  • Ray Oei
    Ray Oei almost 3 years
    Very helpful but replacing "StopWhenUnneeded=true" by PartOf=test.target is a better solution today in 2021. See how to stop units of the same target as it solves issues with restarting the service for instance.
  • Majal
    Majal about 2 years
    @phemmer, you're a systemd guru!