What is bottleneck of my Apache server?

Question

$netstat -anp | grep :80 | grep TIME_WAIT | wc -l  
840 
$netstat -anp |grep :80 | grep ESTABLISHED | wc -l 
50

memory usage : 850MB / 1000MB

apache2.conf contains..

<IfModule mpm_prefork_module>
    StartServers          5
    MinSpareServers       5
    MaxSpareServers      10
    MaxClients          150
    MaxRequestsPerChild   0
</IfModule>

<IfModule mpm_worker_module>
    StartServers          2
    MinSpareThreads      25
    MaxSpareThreads      75
    ThreadLimit          64
    ThreadsPerChild      25
    MaxClients          150
    MaxRequestsPerChild   0
</IfModule>

<IfModule mpm_event_module>
    StartServers          2
    MaxClients          150
    MinSpareThreads      25
    MaxSpareThreads      75
    ThreadLimit          64
    ThreadsPerChild      25
    MaxRequestsPerChild   0
</IfModule>

Are there any configuration changes that can help me or its just my RAM the bottleneck here?
Urgent help needed..!!

Solution

Without more information on your server, configuration, and application it is difficult to give a detailed answer but here are a few general things to consider:

• 150 MaxClients with 1GB of RAM might be too many, especially with an Apache server running with PHP. What may happen is that Apache will use up all the RAM and begin to hit the swap memory which will instantly kill your performance. The best way to check this is during peak hours check the output of "top" and "free" and ensure the swap memory usage is near 0. If your free memory goes to zero and swap memory begins to increase try reducing MaxClients. The tricky part is that the server will seem to run just fine with a high MaxClients setting until you happen to hit a traffic peak that begins to use the swap.
• You can reduce the amount of memory Apache uses by disabling any modules you don't use. Try disabling a few at a time and testing to make sure you don't accidentally disable a module you need (speaking from experience).
• Having a lot of connections in TIME_WAIT is normal. This is the state that occurs after the socket is finished and is waiting to closed/destroyed. Having a lot of connections in ESTABLISHED may be due just a high server load, serving files that take a while, or a large KeepAlive setting (though I'm not sure about what state connections are kept in the latter). For my application I found the KeepAlive setting actually hurt performance so I disabled it. The default KeepAlive setting (15 seconds I think) is actually pretty large for most applications with a value of 1 or 2 seconds sometimes working better.
• The next most likely bottleneck will be CPU. Check the server's load with "uptime" or "top". Generally a load less than one is good, around one is fine, and as values increase it indicates a more potential issue. There's no load value that is "bad" (at least until you get very high values): typically I see loads less than 0.5 with occasional spikes up to 5 on my servers when they're running fine. Checking the CPU idle rate in "top" is another measure of how busy the CPU is.
• If your CPU is a bottleneck consider installing a PHP opcode cache like eAccelerator or APC. This can immediately reduce your CPU usage by one half (or so). Other things to consider would be adding a caching layer to reduce the amount of requests the web server has to serve and using a light weight server like lighttpd/nginx to serve static content.
• You can more accurately test your bottlenecks (hardware or configuration) using benchmark programs like ab or siege. Do a few baseline measurements with the current configuration, change settings, and try again. If you don't measure the effect of your changes to configuration you may end up spending a lot of time changing the wrong thing.

Comments

[Victor Matheus Alves Ramos]

What is this web server serving? While you listed the different mpms, you don't tell us which one is being used. Need more information to provide assistance.

[rrh]

This is a simple web server with php scripting. I don't know what mpms means. How can I check what mpms I am using ? (sorry for my ignorance)

[Patrick R]

spend some time here: httpd.apache.org/docs/2.0/misc/perf-tuning.html. This will help you understand how to fashion your question. Note the "Choosing an MPM" section.