Run linux process at very very low priority?

Question

I have regular process that not so important but will consume so much CPU power, and I have another process which is really important, but it spend most of the time idle, but when it got a job it really need high computing power

I tried running with nice -20 ./low_priority_process and nice --20 ./high_priority_process but still the lower priority consume significant amount of CPU when high priority process is in need, how can I run a process that will really yield or even auto-suspend when another process using CPU power

Solution 1

Have a look at cgroups, it should provide exactly what you need - CPU reservations (and more). I'd suggest reading controlling priority of applications using cgroups.

That said, put the important yet often idle processes into group with allocated 95% of CPU and your other applications into another one with allocated 5% - you'll get (almost) all of the power for your jobs when needed, while the constantly power hungry process will only get 5% at most at those times. When the computational surges disappear all CPU performance will be thrown at the remaining processes. As a benefit, if you create a special cgroup (with minimal performance requirements) for processes like sshd, you'll be able to log in no matter what is trying to get all CPU it can - some CPU time will be reserved for sshd.

Solution 2

If the process priority (nice value) is low then it will not be interrupting a higher priority process. The reason you're seeing the low priority process still consuming a significant amount of CPU when the higher priority process is running is because the higher priority process is not that busy. Probably waiting on IO. Use chrt -p -i 0 $PID to run the process at an even lower priority than nice 19 -p $PID (assuming we're talking about Linux here).

chrt -p -i 0 $PID puts the process into the "true" idle scheduler.

http://linux.die.net/man/1/chrt

Solution 3

For future-comers, here is a full example of nice with stress.

1. The test machine has 2 CPUs

$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              2 
On-line CPU(s) list: 0,1
Thread(s) per core:  2
...

2. Install stress: apt-get install stress
3. Make the 2 CPUs busy with a low-priority call to stress: nice -20 stress --cpu 2
4. Check CPU usage with top:

                                                 v
                                                 v
                                                 v
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                   
15894 ubuntu    39  19    8240     96      0 R  99.7  0.0   0:06.43 stress                                                                                    
15895 ubuntu    39  19    8240     96      0 R  99.7  0.0   0:06.42 stress                                                                                    

This shows that both CPUs are fully occupied.

6. Launch a single-cpu stress process with high priority: nice --20 stress --cpu 1
7. Check cpu usage again with top

                                                 v
                                                 v
                                                 v
                                                 v
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                   
15928 ubuntu    20   0    8240    100      0 R  99.7  0.0   0:24.02 stress                                                                                    
15894 ubuntu    39  19    8240     96      0 R  51.2  0.0   1:12.46 stress                                                                                    
15895 ubuntu    39  19    8240     96      0 R  48.8  0.0   1:12.35 stress                                                                                    

This shows that the single-core stress process gets its full CPU, whereas the lower-priority processes both share the remaining 1 cpu

8. On the other hand, killing all the above stress calls and just triggering a single 3-process stress --cpu 3 would give 66% CPU to each

Comments

[uray]

yeah -20 is "low priority" and --20 is "high priority" look at double dash for nice command. and yes I am understand completely about nice value, but my question is, is there a way to tell scheduler other than using nice value

[muru]

@user77710 are you sure? Neither the Ubuntu manpage nor the POSIX manpage specify this syntax. Both use -n -20, -n 20 etc.

[uray]

yeah whatever, the syntax is not important, my point is settng nice value of very low and very high for both processes does not give the result of what i wanted

[muru]

@user77710 WHAT? "the syntax is not important"? How can you be sure that you're setting the niceness values then?

[uray]

because i can see the process nice value on htop or top or ps or whatever

[uray]

is there a command like cpulimit or nice to modify the cgroup of process? because i if cgroup is an API call, i am in position can't recompile any of those app to use cgroup

[peterph]

No, you just mount the cgroup hierarchy somewhere, create per-group directories and write PIDs of processes into some files. All this has to be done as root (or, more precisely with appropriate privileges). Some init systems (namely systemd, at least in some cases) "steal" the cgroups interface from users who have to use the init system interface (usually a special command). Read the linked answer and wikipedia article. Really. :)

[peterph]

Oh, and there is also libcgroup package, which comes with utilities for working with cgroups including a daemon (cgrulesengd) that can actually sort processes into groups depending on some conditions.

[Wycofane]

chrt -p -i 0 $PID

[unixmiah]

@Ken Shark, try my new answer.

[soger]

chrt is still not enough. What I'm trying to do is to watch videos while reencode other videos in the background. Using chrt helped with youtube videos but very high quality mkvs still skip. What I wished is to playback my video normally but use every bit of CPU power remaining for reencoding because I have a huge batch to do.

[starbeamrainbowlabs]

@soger Is your video encoding process GPU accelerated? it could potentially be a resource contention issue with your GPU?

[soger]

@starbeamrainbowlabs No, it is not.

[Ken Sharp]

Is your hard disk busy? See ionice.

[starbeamrainbowlabs]

@KenSharp @soger sudo ionice -c Idle -p PID_HERE :D

[Ken Sharp]

ionice -c3 command There is no reason to use sudo.

[soger]

No, it's not a hard disk problem either. Youtube uses no hard disk and mplayer would complain if it were running out of buffer.