How to stop Fork Bomb out of memory error - RHEL 6

Question

I set up test VM to test the effect of fork bombs. So I edited the limits.conf as follows for root user:

root    hard         nproc  512

Now I drop a fork bomb like so:

:(){ :|:& };:

After this, after a while (which I believe it takes to reach the 512 limit), the following error shows up:

This continues without stopping. Is there anyway to stop this without rebooting the machine?

Solution 1

While developing the answer for this question, titled: Where is the fork() on the fork bomb :(){ :|: & };:?, I put together what I called a fuse delayed fork bomb, which was easier to kill.

Additionally while developing that answer I regularly was able to halt a fork bomb by killing all the processes. It was easier and more repeatable than I would've expected.

Methods used

It's been a while since I wrote that answer so I'm not 100% sure now but off the top of my head I think I was using this method:

$ pkill -f :

It would stall for a bit waiting for a process but eventually it was able to run. Also I would note the parent process ID before starting the fork bomb and would do this too:

$ pkill -P <PPID>

That's the parent process ID (PPID) from where the fork bomb was run. That method would take all the child processes down which would cause them all to cascade and die.

Solution 2

Is there anyway to stop this without rebooting the machine?

It's not quite impossible, and you can do it via luck -- i.e., you manage to kill all the processes before another one is spawned.¹ But you have to get very very lucky, so it is not a reliable or worthwhile effort [maybe slm is luckier than me here, lol -- TBH I haven't tried that hard]. If you play around with priorities, your chances could improve (see man nice), although I suspect this will also mess with the efficacy of the fork bomb.

A better idea might be to use one that times out. For an example in C, see footnote number 5 to my answer here.² You can do the same thing with a shell script, albeit would not be as short as :(){ :|:& };::

#!/bin/bash

export fbomb_duration=$1
export fbomb_start=$(date +%s)

go () {
    now=$(date +%s)
    if [[ $(($now-$fbomb_start)) -gt $fbomb_duration ]]
        then exit 0;
    fi
    go &
}

while ((1)); do
    go
done           

Execute that with one argument, a number of seconds. All forks will die after that time.

¹ In fact, it can happen all on its own, eventually, if the kernel OOM killer gets lucky. But don't hold your breath.

² The method used there to hamstring that particular bomb (by setting vm.overcommit_memory=2) will almost certainly not work in general, but you could try. I'm not since I'd like to leave my system running for now ;)

Solution 3

I believe you could do as suggested in the answer from here assuming you have access to shell.

killall -STOP -u user1
killall -KILL -u user1

Comments

[goldilocks]

+1 if you give a hint about how to optimally kill all the processes ;)

[slm]

@goldilocks - see updates.

[slm]

I think you could simplify this by using sleeps instead. Maybe?

[goldilocks]

@slm The best you can do with a normal kernel in terms of sleep granularity is 10ms, and that is too long -- it would be more of a fizzling fork pie than a fork bomb.

[goldilocks]

:) If you put the fork in a script which calls itself by name, you could also do something like while ((1)); do killall myscript.sh; done.

[slm]

@goldilocks - good idea.