Difference between nice level and systemctl CPUShares property

Question

As I understand it (please correct me if I'm wrong),

systemctl set-property ... CPUShares=some_value

limits the cpu time for an entire cgroup unit. If we instead want to limit cpu time for a process within that unit, we can run the process with

systemd-run ... nice=some_value

I'm wondering if there's some intrinsic difference between the concepts of the nice-value of a process and the CPUShares-value of a group of processes? Can we limit CPUShares of a process, or set the nice-value of a cgroup unit? When would we want to do one or the other?

Solution 1

From reading man systemd-run, it will create a service and thus a cgroup on the fly. From reading systemd.exec, the Nice= directive will apply to all executed processes, so the way that systmd handles the concepts of Nice= and CPUShares= are very similar.

My understanding of the relationship is that it has to do with history. nice has existed for a listed a couple of decades and always applied to specific processes.

On the other hand, the concept of cgroups and the idea of applying CPUShares= to a process group is much a newer concept but accomplishes the same goal.

I expect systemd supports Nice= for historical compatibility.

I would use whichever one you are comfortable with, but not both to avoid confusion. If you have no preference, I would use the newer CPUShares= concept.

Solution 2

On Linux,

• a nice value applies to a task, that is a process or thread (see link for disambiguation),
• a "CPU shares" value applies to a task group (for example: a cgroup).

The default non-realtime Linux' task scheduler (CFQ), distributes CPU time "fairly" among the different cgroups. It will use the cpu.shares value of each cgroup (by default 1024), relative to other cgroups' cpu.shares values, to grant more or less CPU time to each cgroup (more shares = more CPU bandwith).

Now, within each cgroup, a task's nice value will be used to grant it more or less CPU time, relative to other tasks' nice values within the same cgroup.

sched(7) says:

Under group scheduling, a thread's nice value has an effect for scheduling decisions only relative to other threads in the same task group.

---

Side notes:

• a task group is a cgroup or an autogroup (see sched(7)),
• autogrouping may be disabled by defaut on your Linux distribution, check with cat /proc/sys/kernel/sched_autogroup_enabled,
• task groups can have hierarchical relationships (cgroups within a cgroup) that I guess the CFQ task scheduler considers to distribute CPU time,
• "recent" distributions may be using systemd to put tasks in cgroups by default (try systemd-cgtop and systemd-cgls).

Comments

[Totor]

Beware, using CPU shares and nice values are likely to result in very different behaviors on "recent" distributions using systemd default cgroups (Debian 9 is one of those). See sched(7) : "Under group scheduling, a thread's nice value has an effect for scheduling decisions only relative to other threads in the same task group." A cgroup being such a task group. See my answer for more details.