How can I automatically kill the process with the highest CPU load?

Question

Sometimes programs will lock up in the background and cause high CPU usage. Is there any way I can programmatically ascertain which process is causing the highest CPU load at the moment and kill it?

Solution 1

There are a family of Unix commands that will probably serve you better if you're aware of them for this type of work.

• pgrep
• pkill
• killall

You can use these tools to make your "attacks" more targeted, especially in situations where you know the misbehaving process by name(s).

killall

I have a recurring issue with Chrome where it eventually needs to be dealt with by killing it. I usually do this command to eradicate all of them.

$ killall chrome

pgrep & pkill

But I could do this as well, to deal with only the newest process:

# to list
$ pgrep -n chrome
23108

# to kill
$ pkill -n chrome

Killing based on your command line

You can also add the -f switch to reach those processes that have long path arguments that you'd rather match on, instead of just their executable's name.

For example, say I had these processes:

$ ps -eaf | grep some
saml     26624 26575  0 22:51 pts/44   00:00:00 some weird command
saml     26673 26624  0 22:51 pts/44   00:00:00 some weird command's friend
saml     26911 26673  8 22:54 pts/44   00:00:00 some weird command's friend

They're just Bash shells with their ARGV0 set to those names. Incidentally I made those processes using this trick:

$ (exec -a "some weird command name's friend" bash)

Going after friends

But say I have a lot of them, and I only want to go after a particular set of them because they have "friend" in their command lines. I could do this:

$ pgrep -f friend
26673
26911

Going after the youngest friend

And if there were a couple of them and I wanted to go after the newest, add the -n switch back into the mix:

$ pgrep -fn friend
26911

You can also use regular expressions when enlisting the -f switch, so these would work, for example:

$ pgrep -f "weird.*friend"
26673
26911

Displaying their names

You can double check the processes names using the -l switch:

$ pgrep -f "weird.*friend" -l
26673 some weird command's friend
26911 some weird command's friend

Controlling the output

Or tell pgrep to list the process IDs delimited using a comma (,):

$ pgrep -f "weird.*friend" -d,
26673,26911

You can do cool things like this:

$ ps -fp $(pgrep -f weird -d,)
UID        PID  PPID  C STIME TTY          TIME CMD
saml     26624 26575  0 22:51 pts/44   00:00:00 some weird command
saml     26673 26624  0 22:51 pts/44   00:00:00 some weird command's friend
saml     26911 26673  0 22:54 pts/44   00:00:00 some weird command's friend

So how do I kill the high CPU process?

I would use the above to be more selective in going after a high CPU process. You could use the approach of killing using these methods:

# newest guys
$ pkill -nf vlc ; pkill -nf opensnap

# kill all of these
$ killall vlc; killall opensnap

Look at their CPU loads:

$ top -b -n 1 | grep -E $(pgrep -f "weird.*friend" -d\|) | grep -v grep
26911  0.1  112m 106m 6408  848 4900 1512    0    0 S  20   0  0.0 some weird command's friend                                     
26673  0.1  112m 106m 6392  848 5020 1504    0    0 S  20   0  0.0 some weird command's friend 

Here I've changed the delimiter from a comma (,) aka. this switch -d,, to a pipe (|) aka. this switch -d\|, so that I can use it in a grep. Doing this will return the process IDs like this:

$ pgrep -f "weird.*friend" -d\|
26673|26911

We then insert these into the grep -E ... command so we can filter the output from top based on certain process IDs.

This might seem like a lot of bending backwards, but we now know with certainty that the process IDs we're using are only the ones related to a give process named "weird.*friend".

From here you can find the process with the highest CPU and kill it, if you really want to go that way.

A more targeted approach to high CPU

$ top -b -n 1 | grep -E $(pgrep -f "weird.*friend" -d\|) | \
    grep -v grep | sort -nk14,14 | tail -1
26911  0.1  112m 106m 6408  848 4900 1512    0    0 S  20   0  0.0 some weird command's friend                                     

The above shows the sorted output from top by the CPU column (14th). It's sorted from lowest to highest, so we take the last line (tail -1) which would be the highest CPU process of the "weird.*friend" processes.

Solution 2

The program "and", Auto-Nice Daemon can be set up to do something to that effect.

You set up a list with several specific known troublemakers and three levels of re-nicing (so that you can re-nice progressively harshly), but any of which could be to kill the process already, even though that would more usually be reserved as the last resort.

It's not always easy to regulate things so that you achieve a certain desired levels of CPU load, but there are other tools that may help (and help avoiding killing offending process as well, by preventing the offense), such as cpulimit, and already starting programs with nice and ionice.

Solution 3

Edited script

@msw, @sim and other users have raised valid concerns as to the concept of my script. Inspired by @sim's answer and msw's remarks I have sat down and rethought how to solve my particular issue (couple of known processes causing trouble now and then). Here is what I came up with:

#!/bin/bash

# tries to kill process with highest CPU load
# (if it is part of a specified list of troublemakers)

TROUBLEMAKERS="vlc opensnap glxgears stress"


sleep 1 # wait a few seconds (just as a precaution)

TOPPROCESS=$(top -b -n 1 | sed 1,6d | sed -n 2p)
TOPPID=$(echo "$TOPPROCESS" | awk '{print $1}')
TOPNAME=$(echo "$TOPPROCESS" | awk '{print $12}')

if [[ "$TROUBLEMAKERS" == *"$TOPNAME"* ]]
  then
      echo "Cause of high CPU load: "$TOPNAME" ("$TOPPID")"
      echo "In troublemaker list. Killing..."
      kill -9 $TOPPID
  else
      echo "Cause of high CPU load: "$TOPNAME" ("$TOPPID")"
      echo "Not in troublemaker list. Exiting..."
      exit 1
fi

exit 0

In contrast to the previous script this one will only kill the process with the highest CPU load if its name matches a number of known troublemakers (processes that tend to lock up on your system).

---

Original script

Here's a simple script that will determine the process causing the highest momentary CPU load on the system and kill it (with the exception of Xorg, which would cause the GUI to crash):

#!/bin/bash

# tries to kill process with highest CPU load
# (if it isn't Xorg)

sleep 1 # wait a few seconds (just as a precaution)

TOPPROCESS=$(top -b -n 1 | sed 1,6d | sed -n 2p)
TOPPID=$(echo "$TOPPROCESS" | awk '{print $1}')
TOPNAME=$(echo "$TOPPROCESS" | awk '{print $12}')

if [ "$TOPNAME" != "Xorg" ]
  then
      kill -9 $TOPPID
  else
      echo "CPU load caused by Xorg. Exiting."
      exit 1
fi

exit 0

The TOPPROCESS snippet is based on this entry at commandlinefu.com.

Comments

[Admin]

If you don't care what process you kill, why not just power down the machine and eliminate the guesswork?

[Admin]

@msw In my particular case I have two processes (vlc and opensnap) that will sometimes lock up and cause high system loads under specific and predictable circumstances (e.g. closing a vlc window, switching display layouts). When I see the system temperature going up under these conditions I can be pretty confident that it's one of those processes causing it. And even if I do first check the system monitor for the actual cause hitting a hotkey to immediately eliminate it is still faster than going through right click → kill.

[Admin]

@msw but as much fun as it is to discuss why I deem this script useful for my particular case I don't think that's why we are here for. If there any issues with the code aside from its application I would love to know them. I am not very experienced with bash and right now the downvote on my answer tells me that there are faults in my script. But what are they?

[Admin]

The biggest flaw (although the downvote wasn't me) is a conceptual flaw: highest momentary CPU load is a horrible indicator of "badness" and your script is already aware of this with the "don't kill Xorg" bit. If vlc and opensnap are known troubles killall vlc opensnap won't give you the false positives that any script wrapped around top will.

[Admin]

msw brings up a terrific point, I too did not DV you but have been watching this Q and you have to be careful with assuming that high CPU load in the moment, is the culprit.

[Admin]

@msw and sim, thank you for your comments. I see your point now and I agree that the momentary CPU load is probably a bad indicator for system impact (especially given that my old script would terminate a process if it were to momentarily spike through the load of the actual "troublemaker"). I edited my answer to include a more targeted variant of my original code.

[Matt]

Nice detail! Your top fields differ from what I get as defaults though and the sort only really works properly when the field you are sorting on doesn't exist in the top headers. top -b -n 1 | tail -n +8 will start after the header on linux.

[Matt]

Also separating out the pgrep command deals with no pgrep results more readily pids=$(pgrep "vlc|opensnap" -d\|) && top...

[Matt]

Also also, assuming linux top, a comma delimited list of pid's can be supplied directly and with the default order of highest cpu first awk can do the header cut and print the first pid pids=$(pgrep "vlc|opensnap" -d,) && top -b -n 1 -p $pids| awk '/^\ *PID/ { getline; print $1; exit; }'

[sebasth]

please explain how your answer works

[sayeed99]

well I hard coded the possibility of having a process running and taking up more than 90% cpu pid gets the output of ps and has the top most element output is like : %cpu PID 7.5 2148 3rd word is cpu usage 4rth is pid since bash only uses integer, used bc to compare 90 with kcpu if you want to forcefully fill the top most (max cpu usage) process remove the inner if