How best to start my systemd service to run multiple apps

Question

I have a one-shot service that calls a script in /usr/

Here my .service file:

[Unit]
Description = Start apps
After = network.target

[Service]
Type=simple
ExecStart=/usr/start_apps.sh

[Install]
WantedBy=multi-user.target

My script does somthing like:

cd /home/user/apps
# Run apps one at a time in background
./app1 &
./app2 &

Now this appears to try to run these apps, but when I do pgrep app there is nothing running.

So, I assuemed this is because the service has ended. So I tried a different approach:

cd /home/user/apps
# Run app1 in back ground 
./app1 &
# Run app2 in forground so the service does not stop
./app2

Now this works (pgrep shows me the apps are running) - but when I start the service it never returns I have to do ctrl+c and then I can get back to the bash prompt and the apps are both still running.

What I want is to start the service, have both apps run and then return to the command line all by doing just:

systemctl start my-service.service

What is the best way to do this?

update I am installing my service like this:

systemctl stop my-service.service
 ... copy the service/script files in place...
systemctl daemon-reload
systemctl enable my-service.service
systemctl start my-service.service

not sure... but it might be relavant?

Solution 1

Starting all the processes in background should work, but then you need to use Type=forking and you'll probably want to use a few more options, in particular RemainAfterExit=yes is probably important.

That use case is similar to the handling of an rc.local script, which is handled in systemd through this unit template, so perhaps using all those options would be a good idea:

[Service]
Type=forking
ExecStart=/usr/start_apps.sh
TimeoutSec=0
RemainAfterExit=yes
GuessMainPID=no

I'm not sure why you're getting blocked and having to hit Ctrl+C on the case where you have the last process in foreground... Do you have to hit Ctrl+C on the systemctl start command itself? I'd say that's a bit odd...

Please note that starting apps this way (many of them in background, from a script) is not really how systemd is intended to be used and you'll be missing out on features. For instance systemctl stop or systemctl restart will most likely not work at all on that unit.

These days, running apps in background is seen as a "hack" (not only by systemd but by many of its precursors and most other modern service managers.) While systemd is still able to do that, that support is there mostly for backwards compatibility and is not intended to be used when better alternatives exist.

My recommendation would be that you create separate services for each app, or perhaps use template units if you're starting many instances of the same process. Also look into directives to specify dependencies between units so they start and stop in the correct order. In particular, take a look at the PartOf= directive, which allows you to manage a group of services as a single unit, allowing you to start, stop or restart them all together with a single command.

Solution 2

If these applications can crash and would need to get restarted or if they would ever need to get stopped or restarted individually consider checking out supervisorctl.

You can start supervisorctl from systemd as a first class service, and then be able to have it crank up any number of dependent processes and manage them giving you one place to start or stop your services (supervisorctl stop all, supervisorctl start all) and similar auditing capabilities to systemd (supervisorctl status myapp).

The docs for supervisorctl are quite good and it sounds like your use case might be such that a simple shell script, like you posted, might not be enough to pull this off without significant headache.

Comments

[code_fodder]

@PhilipCouling updated - sorry, should have thought of that

[filbranden]

systemd is made to run each app in a separate service and will work best that way. Can you expand on why you want to manage multiple apps from the same service? There are ways to connect multiple services together (e.g. PartOf=) that might do what you need, but you need to explain why you think you need what you're doing for such a recommendation to make sense...

[code_fodder]

@filbranden for convinience. My system is made up of ~10-15 applications which all communicate with each other so my simple start script (which I use to manaully start the apps) makes it easy to start them all. I was hoping that I could just use the simple script within systemd...? I "could" make loads of services - but that would be an extra pain, but not un-doable! :)

[GManProgram]

@code_fodder The "correct" way to do that would be to make a unit file for each application, defining which services they depend on using Requires= etc. If you then have a service that requires everything to be running, starting that will start everything. If that is not the case, you can define a target unit which will start/stop all of the services when the target is started or stopped. Definitely check out the systemd.unit, systemd.service and systemd.target man pages. They are very well written and can point you in the right direction :)

[code_fodder]

Thanks very much for this - the template appears to work : ) .... As for the correct way to do it, yes I will probably add this to my todo list - I assume if I create a chain of dependant services I can start/stop all of them in one command?. But anyway, for now I can go started using the hacky approach until I get time to sort that lot out :o

[filbranden]

My recommendation for a group is create a master service that does nothing (no ExecStart= is possible in fairly modern versions of systemd) and has RemainAfterExit=true, then use PartOf= that one service on all the others. That one dummy service becomes the master controller for all others then. Glad your immediate problem is solved. Good luck!

[code_fodder]

ah, yes - that sounds spot on : )

[ThatsRightJack]

@filbranden I like your answer as it's given me some good direction, but now I'm wondering what would be the benefit of creating a "ghost" service vs creating a target?