How To Tune Apache on Ubuntu 14.04 Server

Question

Currently on my Apache 2 (Apache 2.4.7 to be exact) on Ubuntu 14.04, I have this setting:

/etc/apache2/mods-enabled/mpm_prefork.conf

<IfModule mpm_prefork_module>
StartServers                    20
MinSpareServers                 100
MaxSpareServers                 250
MaxRequestWorkers               150
MaxConnectionsPerChild          0
</IfModule>

The server is an 8GB (RAM) Amazon server that does nothing more than load up a three-page signup form for some Google ad campaigns.

I found a script called apachetuneit.sh on the web, but then after awhile the Apache was reporting this error:

[Tue Apr 21 16:45:42.227935 2015] [mpm_prefork:error] [pid 1134] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting

How can I judge how to set these settings?

I am asking specifically only for how to tune Apache 2.4 and nothing else. This is why this question is different than this question.

Solution 1

1. Recognize that Ubuntu 14.04 uses Apache 2 with PHP running through an mpm_prefork module, of which an editable file is in /etc/apache2/mods-enabled/mpm_prefork.conf. Also, recognize that starting in Apache 2.4, MaxClients is now renamed as MaxRequestWorkers, and so any documentation regarding MaxClients needs to be switched to MaxRequestWorkers.

2. Stop the Apache web service with the following command, temporarily:

    sudo service apache2 stop

3. Wait 5 seconds and then run the following command to find out how much virtual memory you have on the server that is free:

    sudo free -ht

Read the Mem: line and look at the free column. Consider this as the amount of RAM that you can dedicate to Apache, although I usually like to deduct 2GB on a beefier server (as in > 4GB), or 1GB on a lighter server. So, if the free column said I had 13GB free, I would recommend giving Apache 11GB. That's a baseline. If we encounter any database issue in the logs occasionally (as in like 3 times in the logs over a 3 day period) that it needs more memory, then we might consider that we only had 10GB to play with instead of 11GB (in this case). If we encounter in the Apache logs that the server needs more MaxRequestWorkers, then that's a separate issue I'll address below.

4. Start the Apache web server.

    sudo service apache2 start

5. Open like 10 browser tabs, connect to some of your longer or slower-loading pages from your website, and refresh like 3-4 times on each tab.

6. After doing that, rapidly now run the following command:

    sudo ps -ylC apache2 | awk '{x += $8;y += 1} END {print "Apache Memory Usage (MB): "x/1024; print "Average Process Size (MB): "x/((y-1)*1024)}'

Run it like 5 times rapidly.

Look at the Average Process Size value and average that value out among the 5 times you ran that.

Now do the following math, and be sure to convert GB to MB as necessary so that all the numbers are in MB values. So, either multiply times 1024 or divide by 1024, depending on which way you need to go.

MaxRequestWorkers = Baseline Free (with buffer space) / Avg Process Size

For example, I had a 14GB server, but when Apache was stopped the server showed it used 1GB RAM in idle. I then provide another 1GB in some extra buffer space for the OS in case it needs it. That means I would have a Baseline Free of 12GB. Now I must convert it from GB to MB, and so I multiply 12 x 1024 and get 12288. The 12288 MB is my Baseline Free value. In my case I saw that the Average Process Size was 21MB. So, I take 12288 / 21 and I get approximately 585. Now, it's common that sysops round down this value, and so I got 580.

7. Edit the file /etc/apache2/mods-enabled/mpm_prefork.conf and consider setting it to the following defaults, replacing XXX with your MaxRequestWorkers calculation:

`<IfModule mpm_prefork_module>`
  StartServers                    2
  MinSpareServers                 2
  MaxSpareServers                 5
  MaxRequestWorkers               XXX
  ServerLimit                     XXX
  MaxConnectionsPerChild          0
</IfModule>

Note that you may not see the ServerLimit parameter there. Add it. This parameter defaults to 256 if not present, but needs to be the same value as MaxRequestWorkers or you'll get an error.

8. Another critical factor in your Apache configuration is the /etc/apache2/apache2.conf file with the Timeout variable and is measured in seconds. This is how long you can send or receive from the server before it times out. You have to also keep in mind a file upload or file download, such as if you have a website where people can upload or download CSV or other large files, for instance. And you need to keep in mind a busy database server and where you might need to provide some time before pages timeout. The smaller you make that Timeout variable, the more available the web server is to receive new connections. Note, however, that setting this value too low may cause havoc with PHP session variables, although not with browser session-based cookies. So, for instance, a value of 300 (5 minutes) might be good for a web server that relies on PHP session variables for web app workflow instead of browser session cookies. A value of 45 might be good for a web server that serves up nothing more than static advertising landing pages, but would be terrible for a server that needs to use PHP session variables a great deal. So, edit the Timeout parameter in this file to the amount you need. This may take some testing with all your web pages to see if the value is too low. It's probably a good idea, however, to not set it higher than 300 unless you're seeing problems in large file uploads or large file downloads.

9. Now restart your Apache web service. If you did something wrong, Apache will likely tell you about it the moment you start it again, and you can rectify it.

    sudo service apache2 restart

10. Now repeat the 10 tab browser trick that you did previously, and see if you encounter Apache configuration errors in the Apache web server error log:

    sudo tail -f /var/log/apache2/error.log

...press CTRL+C to get out of that, should you want.

Look for a complaint about needing MaxRequestWorkers (and recently since you restarted the web server). If you see that even with an optimal MaxRequestWorkers setting, then you're likely needing more firepower for your websites or web applications. Consider these options:

• Using a CDN for large file downloads, images, and scripts.
• Using a caching service like CloudFlare or others.
• Redoing your website or web application strategy to use multiple web servers acting as one "web app" behind a load balancer.
• Adding more RAM to the server, and thus doing this calculation all over again.

---

11. Now that the Apache server is tuned, it's sort of baseline tuned. You'll need to check on it over the course of 2-3 weeks and look for MaxRequestWorker issues in the Apache error logs. From that, you can make a decision on optimization (see step 10). You can also install Munin with apt on Ubuntu and look at the Apache performance over time and plot an idea of growth before you decide you need to do anything about the amount of traffic the web server is handling.

Solution 2

What you can raise MaxRequestWorkers to depends on how much RAM each of your httpd / apache processes takes up. If each one takes up 50MB (just picking a random number), then every 20 request workers you have in use at once (i.e. concurrent connections) will take up 1GB. So in that example, you could have at most 8GB * 20 = 160 MaxRequestWorkers. However, you can't use up all the RAM for apache, you need to reserve some for other things running on there. Also leaving some spare RAM can be helpful for performance as the OS will use it to cache things and speed up performance (though it may be preferable to not run out of connections as compared to the server being extra quick as the site will appear slow while users' browsers wait for a connection to come available if they get all used up).

Comments

[Jenny D]

possible duplicate of How do you do load testing and capacity planning for web sites?

[ServerChecker]

@JennyD That article is far too broad for this specific need.

[Jenny D]

The way that you judge these specific settings is you do load testing.

[ServerChecker]

@JennyD It's even easier than that. Just use ap.sh script under regular load multiple times in the day and average, do the math per the cloudinservice link below, and set MaxRequestWorkers to that value. Then, watch the Apache logs for troubles with MaxRequestWorkers and either increase, add more RAM, add CloudFlare, or add another web node. No need to do extremely intense load testing.

[Jenny D]

You could write an answer containing information about that script, and relevant stuff from the links you included in your own answer, to the question I pointed to as a possible duplicate.

[ServerChecker]

What's a good way to profile this? What commands can I type to see how much RAM each apache process uses under load? We also need RAM room for MySQL (a WordPress site) and normal system processes too.

[g491]

See how many processes are running and run the free command to get a memory baseline. Then increase the number of processes running by increasing MinSpareServers, restarting apache, running free again, and doing the math. Check out moreofless.co.uk/apache-memory-usage for more details and explanation.

[ServerChecker]

I now learned that there's a super fast way to do this. It's outlined in the cloudinservice.com link in my answer below. The command is ap.sh that someone figured out. Also, I learned that MaxClients has now been renamed as MaxRequestWorkers in Apache 2.4+.

[Jenny D]

“While these links may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes.”

[ServerChecker]

@JennyD Good point! I'll do that.

[ServerChecker]

@Moderator I need formatting help on this answer. Step 7 was supposed to be conf file with indents. Step 11 wasn't supposed to be indented. Code commands that start with sudo need to show as code, not as normal text. I followed the documentation on formatting, but still struggle here.

[user1928596]

for centos newbies like me : ps -lyU apache instead of ps -ylC apache2