httpd memory usage

Question

Having some problems with httpd (Apache/2.2.29) memory usage.

Over time, memory usage in the httpd processes creep up until it's eventually at 100%.

Last time I restarted httpd was about 24 hours ago. Output from free -m is:

[ec2-user@www ~]$ free -m
             total       used       free     shared    buffers     cached
Mem:          1655       1415        239          0        202        424
-/+ buffers/cache:        788        866
Swap:         1023          4       1019

To prove that it's definitely httpd, I restarted httpd and ran free -m again:

[ec2-user@www ~]$ sudo service httpd restart
Stopping httpd:                                            [  OK  ]
Starting httpd:                                            [  OK  ]
[ec2-user@www ~]$ free -m
             total       used       free     shared    buffers     cached
Mem:          1655        760        894          0        202        360
-/+ buffers/cache:        197       1457
Swap:         1023          4       1019

So, restarting Apache takes free memory from 239 Mb to 894 Mb - which seems like a big leap.

I've been going through the list of currently enabled Apache modules (there's quite a lot) and disabled/removed mod_wsgi and mod_perl (neither of which are required for this server, which is running a PHP-based web application - Magento, specifically).

Based on https://servercheck.in/blog/3-small-tweaks-make-apache-fly, I've run ps aux | grep 'httpd' | awk '{print $6/1024 " MB";}' and get the following output:

[root@www ~]# ps aux | grep 'httpd' | awk '{print $6/1024 " MB";}' 15.1328 MB 118.09 MB 127.449 MB 129.059 MB 117.734 MB 113.824 MB 125.062 MB 123.922 MB 119.855 MB 108.066 MB 136.23 MB 114.031 MB 113.27 MB 110.695 MB 102.113 MB 113.234 MB 186.816 MB 118.602 MB 0.835938 MB

Running the other suggested diagnosis tool for MaxClients which is ps aux | grep 'httpd' | awk '{print $6/1024;}' | awk '{avg += ($1 - avg) / NR;} END {print avg " MB";}' returns the following:

[root@www ~]# ps aux | grep 'httpd' | awk '{print $6/1024;}' | awk '{avg += ($1 - avg) / NR;} END {print avg " MB";}'
110.212 MB

This server (Amazon AWS m1.small instance) has 1.7 Gb of RAM. So, therefore:

Any further pointers/suggestions on how best to tweak the httpd settings or how to diagnose what exactly might be causing this?

Solution 1

Here's what I've done to 'solve' it:

1. Set MaxClients 7 (based on (1740.8Mb Memory on server - 900Mb for MySQL + other stuff) / 111Mb average usage per httpd process = 7.5747747747747747747747747747748)

Therefore:

<IfModule prefork.c>
StartServers       8
MinSpareServers    5
MaxSpareServers   20
ServerLimit      256
MaxClients         7
MaxRequestsPerChild  4000
</IfModule>

2. Disable all Apache modules except for authz_host_module, log_config_module, expires_module, deflate_module, setenvif_module, mime_module, autoindex_module, negotiation_module, dir_module, alias_module, rewrite_module, php5_module

3. Remove the mod_ssl package since the client isn't using https:// whatsoever.

I'll report back once this new configuration has been running a while to see if this solves it.

Some inspiration here was borrowed from: http://www.activoinc.com/blog/2009/08/31/performance-optimized-httpd-conf-for-magento-ecommerce/ and http://www.activoinc.com/downloads/httpd.conf-magento

Solution 2

I'm afraid option MaxRequestsPerChild helped in your case, as it's enables process recycling after defined number of requests, so memory leak is there, but not visible anymore.

Additionally: MaxClients = ServerLimit * ThreadsPerChild

In your case if you need only 7 concurrent users (MaxClients=7) it's totally enough with 2 process (just in case if one will fail to minimize downtime), so config can be:

<IfModule prefork.c>
StartServers       2
MinSpareServers    2
MaxSpareServers   20
ServerLimit        2
MaxClients         8
ThreadsPerChild    4
MaxRequestsPerChild  4000
</IfModule>

I use MaxClients 8, just to make more equal request distribution between 2 processes.

Comments

[James Spittal]

Above configuration has definitely helped massively.

[James Spittal]

Couple days later, 'memory leak' problems in httpd seem to be gone completely.

[baptx]

In my case, I did not need this configuration if I wait a bit less than 24 hours, the memory will be freed with the default Apache configuration (checked with htop command after viewing analytics "Visits Over Time" for several years in Piwik / Matomo). But do you know where the memory leak could come from? Apache server or the PHP code running with mod_php? I guess it is better to fix the memory leak than trying to hide it with a specific configuration.