Apache consuming too much CPU and memory

Question

I'm having some troubles with CPU and memory usage with an Apache Web Server.

We're running an Ubuntu Server 12.04 LTS on a Virtual Machine. Our server has following specs:

• 8GB RAM;
• 4 vCPUs (12ghz);

We configured the server to run a Drupal (7.23) based website. So we installed Apache, PHP, MySQL... The versions are below:

• Apache 2.2.22;
• PHP 5.3.10 (The PHP are running as Apache Module.);
• APC 3.1.7;
• MySQL 5.5.31 (all innodb tables);

I am running some apache modules too. Take a look (apachectl -M):

• core_module (static)
• log_config_module (static)
• logio_module (static)
• mpm_prefork_module (static)
• http_module (static)
• so_module (static)
• actions_module (shared)
• alias_module (shared)
• authz_host_module (shared)
• deflate_module (shared)
• dir_module (shared)
• env_module (shared)
• include_module (shared)
• mime_module (shared)
• php5_module (shared)
• proxy_module (shared)
• proxy_http_module (shared)
• reqtimeout_module (shared)
• rewrite_module (shared)
• setenvif_module (shared)
• ssl_module (shared)
• status_module (shared)

On apache2.conf, we have this config:

    Timeout 90
    KeepAlive On
    MaxKeepAliveRequests 80
    KeepAliveTimeout 5
    HostnameLookups Off
    LogLevel warn

    <IfModule mpm_prefork_module>
        StartServers          10
        MinSpareServers       10
        MaxSpareServers       30
        MaxClients           120
        MaxRequestsPerChild 1000
    </IfModule>

The Virtual Host of my site:

    <VirtualHost *:80>
        ServerName blabla.bla.bla
        ServerAdmin [email protected]
        DocumentRoot /l/disk0/site/public_html

        <Directory />
            AllowOverride None
        </Directory>

        <Directory /l/disk0/site/public_html>
            Options MultiViews Indexes Includes FollowSymLinks ExecCGI
            AllowOverride All
            Order allow,deny
            allow from all
        </Directory>

        LogLevel warn
        ErrorLog "/l/disk0/site/logs/apache/site/error.log"
        CustomLog "/l/disk0/site/logs/apache/sit/access.log" combined
        SSLProxyEngine on
        RewriteEngine   on
        RewriteLog     logs/rewrite_www_log
        RewriteLogLevel        0

        Include rewrites-www.conf
</VirtualHost>

Drupal Modules:

• ACL 7.x-1.0
• APC - Alternative PHP Cache 7.x-1.0-beta4
• Boost 7.x-1.0-beta2
• Cache Expiration 7.x-2.0-beta2
• CAPTCHA 7.x-1.0
• Chaos tool suite (ctools) 7.x-1.3
• Date 7.x-2.6
• Domain Access 7.x-3.10
• Domain Blocks 7.x-2.0
• Domain CTools 7.x-1.3
• Domain Locale 7.x-1.0-beta3
• Domain Taxonomy 7.x-3.x-dev (2012-abr-29)
• Domain Views 7.x-1.5
• Embed Views Display 7.x-1.2
• Entity API 7.x-1.2
• Entity reference 7.x-1.0
• IMCE 7.x-1.7
• IMCE Mkdir 7.x-1.0
• Internationalization 7.x-1.10
• Link 7.x-1.1
• Localization update 7.x-1.0-beta3
• Media 7.x-1.3
• Meta tags quick 7.x-2.7
• Newsletter 7.x-1.0-beta9
• Options Element 7.x-1.9
• Page Style 7.x-1.0
• Panels 7.x-3.3
• Pathauto 7.x-1.2
• pathologic 7.x-2.11
• profile2 7.x-1.3+0-dev (2013-mai-24)
• select_or_other 7.x-2.19
• sheetnode 7.x-1.0-beta4+3-dev (2013-mai-25)
• SMTP Authentication Support 7.x-1.0
• Token 7.x-1.5
• Transliteration 7.x-3.1
• Variable 7.x-2.3
• Views 7.x-3.7
• Vocabulary Permissions Per Role 7.x-1.0
• Webform 7.x-3.19
• Webform Validation 7.x-1.2
• workbench 7.x-1.2
• workbench_access 7.x-1.2
• workbench_media 7.x-1.1
• workbench_profile 7.x-1.1
• xmlsitemap 7.x-2.0-rc2

My site is simple and don't have many visitors. I am talking about 500 visitors a day maybe. Can Drupal use so much CPU? Or is it a module?

Other problem is memory usage. When a process is created, 80M is allocated for apache2. I think is too much.

My problem is that CPU (all cores) have a high utilisation. Most of time, it's hitting between 90% and 100% usage! The offending process is the apache2. Memory is also consumed without pity. Of a 8GB total, the consumed memory is about 6.5GB to 7.5GB. I don't know if my apache configuration is wrong or if I'm really need more hardware (I guess not). Drupal can cause high CPU load?

When the CPU usage hits 100%, the site goes down and we have to restart apache. I did a workaround solution with Drupal using APC and installing Boost. had some effectiveness, but CPU utilisation still high. Very high.

If you need more information, like Drupal modules and PHP extensions. Please let me know.

Solution

Other problem is memory usage. When a process is created, 80M is allocated for apache2. I think is too much.

Is that real or virtual memory? Honestly, it's not very much; more to the point, you should focus on fixing things that are causing problems, not just things that you "think" should be different.

If you want Apache processes to take up less memory, you should disable modules (as each one is more code that needs to be in memory). But if you need all the modules you've got enabled, then, well, that's that.

One approach I've used when administrating a memory-constrained machine was to move certain tasks out of Apache and into other servers, so I could tune them separately.

But a far simpler approach is to change

MaxClients           120

to something more reasonable for your workload:

If we consider the rewrite accesses and the main site traffic, we have about 70 requests per minute. Right now, we have 33 incoming connections.

I'll get back to that momentarily, but if you're only dealing with 33 concurrent requests, you don't need 120 workers!

MaxClients           40

And you should probably tune down MinSpareServers and MaxSpareServer to something like 5 and 10, respectively. There's no need to have 30 workers sitting around doing nothing.

Now, getting back to

If we consider the rewrite accesses and the main site traffic, we have about 70 requests per minute. Right now, we have 33 incoming connections.

If you've got 33 concurrent requests, but you're only doing 70 a minute, there are a couple of possibilities:

1. Your requests are taking around 30 seconds each to serve!
2. Your request rate isn't very stable, and most of the minute you're doing nothing.

If #1 is the case, I don't really know how to help - something is incredibly wrong, so wrong I don't even know where to tell you to start looking.

If it's #2, my guess is you're serving all your static assets (images, js, css, fonts) from your server. It's best to put these on a CDN, but if you really can't do that, you can set super-long cache times on them and turn Varnish back on. If you're using Apache processes with PHP and a whole host of other things just to serve static files, you're wasting resources - do that with something simpler!

My problem is that CPU (all cores) have a high load. Most of time, hitting between 90% and 100% load! The offending process is the apache2.

Is this a constant number, or only when you're serving requests?

How does disk I/O look (iostat -mhx 2)? What is MySQL doing (show processlist;)?

---

Your server is vastly overpowered for what you've described. This is good news, because it means you should be able to fix this problem.

Comments

[humungs]

Thanks for your reply! The root of the problem is the badly software coded. Anyway, I applied your suggestion about MaxClients, MinSpareServer and MaxSpareServer. Thanks again!

[humungs]

The real problem was in Drupal (and one module). I answered my own question here: drupal.stackexchange.com/questions/87687/…