How to figure out how much RAM each prefork thread requires for maximum Wordpress performance on an EC2 small instance

Question

Just read Making WordPress Stable on EC2-Micro

In the "Tuning Apache" section, I can't quite figure out how he comes up with his numbers for his prefork config.

He explains how to get the numbers for an average process, which I get. But then:

Or roughly 53MB per process...In this case, ten threads should be safe. This means that if we receive more than ten simultaneous requests, the other requests will be queued until a worker thread is available. In order to maximize performance, we will also configure the system to have this number of threads available all of the time.

From 53MB per process, with 613MB of RAM, he somehow gets this config, which I don't get:

<IfModule prefork.c>
 StartServers       10
 MinSpareServers    10
 MaxSpareServers   10
 MaxClients       10
 MaxRequestsPerChild  4000
</IfModule>

How exactly does he get this from 53MB per process, with 613MB limit?

Bonus question

From the below, on a small instance (1.7 GB memory), what would good settings be?

bitnami@ip-10-203-39-166:~$ ps xav |grep httpd
 1411 ?        Ss     0:00      2     0 114928 15436  0.8 /opt/bitnami/apache2/bin/httpd -f /opt/bitnami/apache2/conf/httpd.conf
 1415 ?        S      0:06     10     0 125860 55900  3.1 /opt/bitnami/apache2/bin/httpd -f /opt/bitnami/apache2/conf/httpd.conf
 1426 ?        S      0:08     19     0 127000 62996  3.5 /opt/bitnami/apache2/bin/httpd -f /opt/bitnami/apache2/conf/httpd.conf
 1446 ?        S      0:05     48     0 131932 72792  4.1 /opt/bitnami/apache2/bin/httpd -f /opt/bitnami/apache2/conf/httpd.conf
 1513 ?        S      0:05      7     0 125672 54840  3.1 /opt/bitnami/apache2/bin/httpd -f /opt/bitnami/apache2/conf/httpd.conf
 1516 ?        S      0:02      2     0 125228 48680  2.7 /opt/bitnami/apache2/bin/httpd -f /opt/bitnami/apache2/conf/httpd.conf
 1517 ?        S      0:06      2     0 127004 55796  3.1 /opt/bitnami/apache2/bin/httpd -f /opt/bitnami/apache2/conf/httpd.conf
 1518 ?        S      0:03      1     0 127196 54208  3.0 /opt/bitnami/apache2/bin/httpd -f /opt/bitnami/apache2/conf/httpd.conf
 1531 ?        R      0:04      0     0 127500 54236  3.0 /opt/bitnami/apache2/bin/httpd -f /opt/bitnami/apache2/conf/httpd.conf

Solution 1

The idea behind the calculation is that you need to keep some memory for other things - the operating system and other services running on the instance. After that, whatever you have left, you divide by the average memory used (resident set size - RSS column) to get the maximum number of threads your server can handle without running out of memory.

That said, I like to do my calculation slightly differently, but I'll address that later.

From httpd.conf:

prefork MPM

• StartServers: number of server processes to start
• MinSpareServers: minimum number of server processes which are kept spare
• MaxSpareServers: maximum number of server processes which are kept spare
• ServerLimit: maximum value for MaxClients for the lifetime of the server
• MaxClients: maximum number of server processes allowed to start
• MaxRequestsPerChild: maximum number of requests a server process serves

The config you have shown will keep a constant 10 processes running: you start 10 processes (StartServers) and you have a maximum of 10 server processes you allow (MaxClients). In this case, the MinSpareServers and MaxSpareServers shouldn't come into play. After a server process has served 4000 requests, it will be terminated and a new one created in its place (to try and limit memory leaks, etc)

So, the assumption being made is that everything else on your server will use about 80MB of memory - leaving you with 530MB. Divide that by your average process size (53MB) to get 10 processes.

I'd suggest that the assumption is off a bit. If you are running anything else (e.g. MySQL) on the server, you definitely need to keep more memory in reserve. Additionally, there is no room for variation factored in. For estimation, I would leave at least 120MB for the operating system and other services and then reduce the remainder by 10% for variation. So, about 8 apache processes max.

<IfModule prefork.c>
 StartServers       2
 MinSpareServers    1
 MaxSpareServers   3
 MaxClients       8
 MaxRequestsPerChild  1000
</IfModule>

So, with the above - there are a maximum of 8 processes. We are only starting 2. We would like at least one idle at any given time - but no more than 3 idle. Also, since memory is an issue, the MaxRequestsPerChild has been reduced.

Now, I prefer to calculate the number by actually running the server for a bit, then stopping apache and seeing how much memory the rest of the system is using. Subtract that number from your total, take 90%, and divide by your average apache process size. Especially for more complex setups, this gives a more realistic value.

As a point of mention, on a memory constrained box, Nginx is likely preferable to Apache as a webserver. Also, you definitely want some swap space on a t1.micro (although, you should not find yourself using it much).

Solution 2

Setting MinSpareServers to the same value as MaxSpareServers doesn't make a lot of sense - it rather defeats the purpose of supporting hysteresis for demand management.

I don't see a 'tuning apache' section in the post you reference. What I can see doesn't seem to provide any explanation of how the author measured the footprint of the webserver. See this question and answers.

A keepAliveTimeout of 5 seems ridiculously high - and no explanation is offered for that either.

I don't see the logic behind setting the maxKeepAliveRequests to "about two-thirds of MaxClients"

Assuming an avg 53Mb process (which seems rather large to me) and 613Mb of free memory, I'd start with something like:

keepAliveTimeout 1
maxKeepAliveRequests 15

<IfModule prefork.c>
   StartServers       5
   MinSpareServers    2
   MaxSpareServers    4
   MaxClients        10
   MaxRequestsPerChild  4000
</IfModule>

But if you've got a lot of static files referenced from each page then you might want to force more servers to be running.

The maxrequestsperchild setting of 1000 is probably OK.

Comments

[Admin]

lampe2e.blogspot.com/2015/03/…

[Abdulkadir Dalga]

"you definitely want some swap space on a t1.micro" - but I'm using a small...

[Abdulkadir Dalga]

And do you have a recommendation for my specifc numbers?

[Abdulkadir Dalga]

I fixed the link. Sorry! based on your logic, any hits for my small instance?

[symcbean]

Thanks - your post makes more sense - but the linked article is still strange in a number of respects. Dropping idle processes will free up more memory for I/O caching (i.e. don't set MaxSpareServers=MaxClients). s/hits/hints/ for running your own blog on a small server? Use a different CMS!

[cyberx86]

Looking at your processes, I'd say 60MB/process average. So, keeping 200MB for O/S and other services (increase if you are running MySQL), you have 1.5GB - let's leave about 10% spare - which would give you about 23 processes max. So: StartServers: 5; MinSpareServers: 2; MaxSpareServers: 5; MaxClients: 23; MaxRequestsPerChild: 2000