PHP script error 'exec(): Unable to fork...' on Ubuntu 14.04.3 LTS

Question

I'm running unittest scripts and for test case setup I need to run shell command from time to time. As a result shell command is run around 50 times during unittest execution.

It works ok on CentOS release 6.6 (Final) server but doesn't work on Ubuntu 14.04.3 LTS.

The problem is that after around 20th time shell script execution I get the following error:

exec(): Unable to fork [$(which mysql) -u user -ppwd test_db < "/opt/jenkins/workspace/some - project/application/../tests/Application/assets/sql/test_db.sql" 2> /dev/null]

I've checked all the topics around the internet and no one helped me. I've increased process number and open files limit to enormous values

# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 3885
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 100000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 6553500
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

PHP reports similar numbers. (file test.php contains <?php echo system("ulimit -a");)

# php test.php
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        8192
coredump(blocks)     0
memory(kbytes)       unlimited
locked memory(kbytes) 64
process              6553500
nofiles              100000
vmemory(kbytes)      unlimited
locks                unlimited
locks                unlimited

I run scripts as a non-root user from CLI.

I've increased php memory size to 2Gb

memory_limit => 2048M => 2048M

I've tried to use system, passthrough, shel_exec and it makes not difference.

PHP version is 5.6

PHP 5.6.14-1+deb.sury.org~trusty+1 (cli) 

Does anybody know how to solve the problem?

Solution

Are you using apache / nginx with php or php from the shell?

In any case, the problems may be a process leak or too much memory used.

You can keep open a root shell to the server and try to reproduce this problem from another shell (or browser).

Then when it happens again have a look at the process tree ecc from the root shell and let us know

Comments

[Stepashka]

I run script from CLI. Will try to do what you suggest.

[Stepashka]

Thanks for your advice. I found an issue. The problem was in the virtual machine's memory. There was only 1Gb and tests use more than that. I do not understand why it did not report memory issue but instead reported 'unable to fork'. We've increased memory up to 4Gb and it works ok now.