MaxClients in apache. How to know the size of my proccess?

Question

From http://httpd.apache.org/docs/2.2/misc/perf-tuning.html

The single biggest hardware issue affecting webserver performance is RAM. A webserver should never ever have to swap, as swapping increases the latency of each request beyond a point that users consider "fast enough". This causes users to hit stop and reload, further increasing the load. You can, and should, control the MaxClients setting so that your server does not spawn so many children it starts swapping. This procedure for doing this is simple: determine the size of your average Apache process, by looking at your process list via a tool such as top, and divide this into your total available memory, leaving some room for other processes.

The main issue is that I can't understand how to know the size, because, well i have the size of httpd on no more of 3888

But, if we need to determine the number for MaxClients, and I have 4GB of RAM, so I get: 972, so I should use like 900 in the MaxClients?

Solution 1

First, determine the PID of one of your Apache processes.

Then you can do something like this:

cat /proc/PIDHERE/status | grep VmRSS

This will yield the (current) resident-set-size of that particular process, similar to:

VmRSS: 304456 kB

This value is as it sounds, it is the size of the process resident in RAM.

Then normalize your unit of measure (4GB * 1024 * 1024 = 4,194,304 KB). Divide:

4194304 KB / 304456 KB = 13.77 processes

Consider that you probably have other processes running on your system that will consume memory too, and ideally you want to minimize swapping, therefore you would not likely want 13 Apache MaxClients configured (using my numbers), you want some amount less (at your discretion).

This is a crude estimate; the size of your Apache processes may grow over time depending on load.

Solution 2

Predicting the maxClients from test scenarios is a starting point - but to solve the problem properly you need to start measuring how your application is behaving with real traffic.

Assuming your apache is running pre-fork....

Set up a cron job to count the number of httpd processes and the output of 'free'. Note that if your webserver is serving up any content from local files (and in a lot of cases, even when it's not) the amount of memory available for cache/buffers will have a big impact on performance. i.e. if you get to the point of swapping, your web performance is probably horrible!

Once you've got some data, plot it on a chart and do a least squares regression on it - extrapolate to find the number of clients at which you reach your target limit for httpd memory usage. A starting point for memory target would be the lesser of 80% of the physical memory / 80% of the size of the content.

(note if you've got MinSpareServers set to a very high value, results may not be accurate)

#!/bin/bash

LOGFILE='/var/log/httpd/memusage'
PIDS = `ps -ef | grep httpd | grep -v grep | wc -l`
MEM = `free | grep 'buffers/cache'`
DAY = `date '%Y-%m-%d %H:%M:%S'`
echo ${DAY} ${PIDS} ${MEM} >>LOGFILE

In an ideal world, you'd also measure the URL response time in the same log file - but that's getting a lot more complex.

Comments

[symcbean]

While RSS does not include shared pages, it does include pages flagged as copy-on-write - i.e. there is space on a machine for more processes than (sum of rss)/(physical memory). See also my answer elsewhere - free space is essential for good performance.

[symcbean]

Try running 'free' and you'll see what I expect (2 numbers). If it's true that there's little variation in these numbers with different numbers of processes then either you are running the event based server or your maxspareservers setting is silly. If you'd posted a sample of the output and the relevant parts of your httpd.conf then maybe we would have a better picture?

[symcbean]

....and "crash"? What crash?

[PeterB]

Sorry, see pastebin.com/aHZCagVn for httpd.conf settings and sample output. By 'crash' I mean too many clients, the server runs out of memory and locks up. Please see pastebin.com/fnXzBfQL for more.

[symcbean]

It's not crashing - the OOM killer is kicking in. The numbers show something strange happening above 90 or so clients, but even at low levels the amount of memory usage is very high - I don't know what's running in your website but I've had machines happily serving up 250+ concurrent PHP requests with a quarter of the memory you've got. Your MaxRequestsPerChild is rather low - try 3000. Try a maxclients of 60 until you've sorted the memory problem. Also if this is a real traffic profile then you need to increase your min/maxspareservers a lot (15/30?)

[PeterB]

Thanks symcbean, and yes this is a real traffic profile (though the spikes are caused by bots). Here is the output from top: pastebin.com/hiDY1xMz Jetty/solr and MySQL are comparitively large, but it doesn't add up to near the memory usage (even allowing for OS?)

[loopforever]

@PeterB it seems more likely that MySQLd is causing this issue. When the kernel's OOM killer kicks in kills the worst offenders first based on the value of /proc/PID/oom_score_adj. As evidenced in your output, mysqld went first. You may need to adjust settings in my.cnf more appropriately to fit within the constraints of your VM. As a /temporary/ workaround, just to see if you can avoid OOM killer kicking, try adding a swapfile for additional virtual memory; then get rid of it once you've identified your real issue.