Restart all services in Linux without rebooting kernel?

Question

I have remote machine that has encrypted drives thus each full reboot requires physical access to machine. Machine just experienced issues with lack of disk space so part of running services crashed / entered weird malfunction states. I'd like to "soft reboot" machine without actually rebooting entire kernel to avoid remounting drives and necessity to enter passphrase.

OS is using systemd for services management and kernel 4.9.

Solution

I have used with mixed success (these shortcuts are "available for compatibility only")

init 1; init 3

Which, nowadays typically translates to this:

systemctl isolate rescue; systemctl isolate default

Which kills everything except services that are deemed essential or part of the targets specified - systemd still provides targets that roughly mimic part of what runlevels used to to.

systemctl isolate UNIT

Start the unit specified on the command line and its dependencies and stop all others, unless they have IgnoreOnIsolate=yes (see systemd.unit(5)). If a unit name with no extension is given, an extension of ".target" will be assumed. - man 1 systemctl

The reason I do not quite like to recommend this, is exactly the killing part:

This is similar to changing the runlevel in a traditional init system. The isolate command will immediately stop processes that are not enabled in the new unit, possibly including the graphical environment or terminal you are currently using. - man 1 systemctl

I tried it on Ubuntu and figured i preferred if ssh.service and network.target were part of my rescue.target. Because it sure is safer if only services except ssh and its networking dependencies were restarted.

Because if something goes wrong while activating rescue.target, the command above might otherwise both kill my session and fail to bring everything back up.

Comments

[anx]

see also unix.stackexchange.com/q/123678

[anx]

If systemd locks up during the operation (I actually witnessed that before), screen surviving may be of very little help to you. Because, well, ssh sessions do not survive certain network interface changes.