CentOS7 kswapd0 process eating 99% CPU

Question

When I run top on centos7, I found kswapd0 eating 99% CPU, as shown in the screenshot above.

Below is some other information

top - 11:38:04 up 32 days, 16:14,  1 user,  load average: 0.06, 0.13, 0.19
Tasks: 142 total,   2 running, 140 sleeping,   0 stopped,   0 zombie
%Cpu(s): 12.0 us,  2.0 sy,  0.0 ni, 79.0 id,  6.7 wa,  0.0 hi,  0.0 si,  0.3 st
KiB Mem :  3618860 total,   199684 free,  1130200 used,  2288976 buff/cache
KiB Swap:        0 total,        0 free,        0 used.  2152124 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
12493 mysql     20   0 1200596 402140   6868 S  5.3 11.1  46:54.22 mysqld
13460 apache    20   0 1089628  43932  22264 S  3.0  1.2   1:16.40 php-fpm
13462 apache    20   0 1088772  42704  21672 S  1.3  1.2   1:14.47 php-fpm
 9391 root      20   0  124508  11588   4852 S  0.7  0.3   8:38.31 AliYunDun
13474 apache    20   0 1073892  40976  19884 S  0.7  1.1   1:15.93 php-fpm
12609 apache    20   0 1088832  44976  24000 S  0.3  1.2   1:15.94 php-fpm
12615 apache    20   0 1077948  42776  21764 S  0.3  1.2   1:16.34 php-fpm
12616 apache    20   0 1077568  43116  22344 S  0.3  1.2   1:14.59 php-fpm
12619 apache    20   0 1085672  44144  22380 S  0.3  1.2   1:14.62 php-fpm
12663 nginx     20   0   66928   7604   2276 S  0.3  0.2   4:55.48 nginx
12664 nginx     20   0   68064   9124   2708 S  0.3  0.3   5:17.61 nginx
13451 apache    20   0 1077360  41092  20452 S  0.3  1.1   1:15.16 php-fpm
20987 root      20   0  143572   5788   4492 S  0.3  0.2   0:00.75 sshd
27671 root      20   0  157676   2208   1544 R  0.3  0.1   0:00.12 top
28370 apache    20   0 1071564  34900  15676 S  0.3  1.0   1:01.53 php-fpm
    1 root      20   0  191276   5412   1464 S  0.0  0.1   2:03.59 systemd
    2 root      20   0       0      0      0 S  0.0  0.0   0:00.06 kthreadd
    3 root      20   0       0      0      0 S  0.0  0.0   0:46.51 ksoftirqd/0
    5 root       0 -20       0      0      0 S  0.0  0.0   0:00.00 kworker/0:0H
    7 root      rt   0       0      0      0 S  0.0  0.0   0:00.00 migration/0
    8 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcu_bh
    9 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/0
   10 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/1
   11 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/2
   12 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/3
   13 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/4
   14 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/5
   15 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/6
   16 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/7
   17 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/8
   18 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/9
   19 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/10
   20 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/11
   21 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/12
   22 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/13
   23 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/14

I dont know why

[root@fl basic]#free -lh
              total        used        free      shared  buff/cache   available
Mem:           3.5G        1.1G        197M         87M        2.2G        2.1G
Low:           3.5G        3.3G        197M
High:            0B          0B          0B
Swap:            0B          0B          0B

Solution 1

Well, root don't eat your CPU, it do kswapd. There is known bug in linux kernel. We have same issues in CentOS 7.1 after upgrading to 7.2 it has gone. May be kernel update would be solution for you.

Solution 2

As stated by AlexanderT, it can be a kernel bug and so you should update your kernel.

If you can't do that, a possible workaround is to periodically drop the buffer/pagecache by issuing the following command: echo 3 > /proc/sys/vm/drop_caches

Comments

[Mario7]

shell run rpm --query centos-release output centos-release-7-2.1511.el7.centos.2.10.x86_64

[Alexander Tolkachev]

@Mario7 what kernel version?

[Mario7]

uname -r 3.10.0-327.36.3.el7.x86_64

[Alexander Tolkachev]

@Mario7 we don't have problems with same kernel.