server reached pm.max_children setting (280), consider raising it

Question

PHP-FPM process getting crashed in every few days due to reached pm.max_children, I have to manually start it to make the site up.

Server Configuration:

Total Memory: 7986M 
CPU: 2 Core
1 PHP Process Taking: 22M

PHP-FPM Configuration:

pm = dynamic
pm.max_children = 280
pm.start_servers = 15
pm.min_spare_servers = 15
pm.max_spare_servers = 35
pm.max_requests = 1000

Check the PHP-FPM log and found the following error.

[02-Aug-2017 23:23:54] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 10 idle, and 41 total children
[02-Aug-2017 23:24:04] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 55 total children
[02-Aug-2017 23:24:05] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 16 children, there are 0 idle, and 63 total children
[02-Aug-2017 23:24:06] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 78 total children
[02-Aug-2017 23:24:07] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 93 total children
[02-Aug-2017 23:24:08] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 108 total children
[02-Aug-2017 23:24:09] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 123 total children
[02-Aug-2017 23:24:10] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 138 total children
[02-Aug-2017 23:24:11] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 153 total children
[02-Aug-2017 23:24:12] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 168 total children
[02-Aug-2017 23:24:13] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 183 total children
[02-Aug-2017 23:24:14] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 198 total children
[02-Aug-2017 23:24:15] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 213 total children
[02-Aug-2017 23:24:16] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 228 total children
[02-Aug-2017 23:24:17] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 243 total children
[02-Aug-2017 23:24:18] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 258 total children
[02-Aug-2017 23:24:19] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 273 total children
[02-Aug-2017 23:24:20] WARNING: [pool www] server reached pm.max_children setting (280), consider raising it

I can someone please suggest the best-optimized configuration for the PHP-FPM, Which can help to avoid the crashed.

What does this below line means:

[02-Aug-2017 23:23:54] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 10 idle, and 41 total children

Solution

Here's one suggestion for calculating the right number:

https://myshell.co.uk/blog/2012/07/adjusting-child-processes-for-php-fpm-nginx/

the following command will help us to determine the memory used by each (PHP-FPM) child process:

ps -ylC php-fpm --sort:rss

Then you take your server's available RAM and calculate:

pm.max_children = Total RAM dedicated to the web server / Max child process size

Also see:

php5-fpm: server reached pm.max_children

Comments

[Tim]

How busy is your server? Do you really have 280 simultaneous requests to your server? That would probably represent thousands of users using your website at one time, and would use a LOT of CPU - two cores would probably be 100% used. I wonder if you have a PHP bug or some weird configuration that's not releasing PHP workers. I also suggest you consider page caching for anonymous users, that can increase performance significantly and reduce PHP load.

[ceejayoz]

Going from 31 active workers to 280 in a 30 second period sounds like a DDoS or something to me. That's one hell of a traffic spike.