What's the best way to clean up after a fork bomb?

Question

$ ls
bash: no more processes

Uh oh. Looks like someone made a fork bomb. Where I used to work, this pretty much meant that the shared server would need to be power-cycled, since even the sysadmins with root often couldn't get the problem cleaned up. Often, they couldn't even get a prompt.

I've heard a few tricks (notably, to send STOP signals rather than KILL signals, since the latter would allow the remaining threads to immediately replace the killed ones), but I've never seen a comprehensive guide entitled So, You Have Yourself a Fork Bomb?

Let's make one.

Solution 1

Prevent the fork bomb from exhausting the process limit with a reasonable per user process limit using ulimit.

That way, a single user will exhaust their process quota long before the system limit is reached.

Solution 2

The first thing to try would be to get users that are logged in to logout. It's possible their shell may be the parent process of the process doing all the forking and that might end the problem.

If that doesn't work, you could try running kill -STOP -2 as root to freeze all processes running as any user other than root. If that works, you can then use kill -CONT <pid> to unfreeze some known processes that are unrelated to the fork bomb and kill them to eliminate the full process table issue and give you some breathing room to track down and kill the original source of the problem. Sendmail would be a good example of a system process to kill as it would be easy to identify by using the .pid file to identify the pid. For example, kill -CONT $(< /var/run/sendmail.pid); kill $(< /var/run/sendmail.pid).

Solution 3

Not sure how you could even send a STOP signal, since spawning kill would require an available process handle. Besides, in my experience systems become overloaded and unusable long before running out of processes.

Have you considered simply enforcing per-user process limits with ulimit? That would prevent your users from launching fork bombs (accidentally or not).

Solution 4

Some BSD systems have the ability to reserve the last 5 or so processes for root. Maybe your system has that ability.

View more solutions

Comments

[raldi]

kill is a shell built-in, at least in bash.

[Admin]

I think that's a key component - identify the builtins for your shell of choice.

[raldi]

What OS do you see a "-2" option for kill? I don't see it in the man page on Linux.

[starshine531]

This should work in most OS's as you are specifying a negative value for the pid. If <pid> is less than -1, then kill is sent to every process in the process group -<pid>. Sending a sig of STOP to the pid -2 it should stop all processes that are not special system processes or root owned processes.

[ephemient]

See the kill(2) manpage for killing a "negative pid", but I still don't believe this works. Why would all non-init processes be in group 2? I understand that you'd like to avoid init, since results of stopping it are often quite fatal, but...

[Stephen Darlington]

If it's not a built-in you can run "exec kill PID", which doesn't fork. But it's risky since, if it doesn't work, you may not be able to get another shell. Think of it as the bee sting approach to system administration!

[Nik Reiman]

How do you actually configure the system to do this?

[joshudson]

@ephemient, 2 is too low to be a process group id so maybe it's another special value.

[ephemient]

@Joshua There are no special values beside 0 and -1, according to opengroup.org/onlinepubs/009695399/functions/kill.html opengroup.org/onlinepubs/000095399/utilities/kill.html