ps shows that migration/* processes consume a lot of CPU, but top keep silence about them

Question

here is the result of ps command, shown only first 10 lines

ps aufx | head
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         2  0.0  0.0      0     0 ?        S    May08   0:00 [kthreadd]
root         3  0.0  0.0      0     0 ?        S    May08   0:04  \_ [ksoftirqd/0]
**root         6 98.5  0.0      0     0 ?        S    May08 6841:08  \_ [migration/0]**
**root         7 99.9  0.0      0     0 ?        S    May08 6933:53  \_ [migration/1]**
root         9  0.0  0.0      0     0 ?        S    May08   0:04  \_ [ksoftirqd/1]
**root        11 95.3  0.0      0     0 ?        S    May08 6618:59  \_ [migration/2]**
root        12  0.0  0.0      0     0 ?        S    May08   0:00  \_ [kworker/2:0]
root        13  0.0  0.0      0     0 ?        S    May08   0:14  \_ [ksoftirqd/2]
**root        14 94.6  0.0      0     0 ?        S    May08 6569:27  \_ [migration/3]**

but top command shows nothing special about migtration processes/threads.

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 5377 root      13  -7 64304  45m  11m S    2  1.5 169:28.81 Xorg
 7368 user2    16  -4  457m  98m  34m S    2  3.3  95:06.16 application
 5555 root      10 -10 64212  20m 2820 S    1  0.7  97:04.17 application
 7400 user  16  -4  416m  96m  30m S    1  3.2  93:15.77 application
 7903 user1    16  -4  402m  90m  32m S    1  3.0  59:56.47 application
 7396 user  16  -4  433m  98m  30m S    1  3.3  93:20.15 application
 7401 user  16  -4  417m  98m  30m S    1  3.3  93:05.87 application
 7403 user     16  -4  416m  96m  30m S    1  3.2  93:01.16 application
  500 root      20   0     0    0    0 S    0  0.0   0:46.80 jbd2/sda3-8
 1017 root      20   0  4536  696  476 S    0  0.0  17:38.05 cgrulesengd
 6703 user      12  -8 55240  17m  16m S    0  0.6   3:27.26 application
 7269 root      18  -2 10652 4892 2388 S    0  0.2   6:46.83 python2.5
...

OS: debian squeeze
kernel: 2.6.32

Solution

edit: I think its likely that the OP was not in threads mode in top, and that was the reason for the difference in listings between top and ps aufx | head. For example I see the migration threads in both top and ps with these options;

$ /bin/top -H -b -n 1 | head -25
top - 07:43:24 up 5 days,  5:07, 18 users,  load average: 1.36, 1.03, 0.76
Threads: 1165 total,   1 running, 1164 sleeping,   0 stopped,   0 zombie

...snip

   11 root      20   0       0      0      0 S  0.0  0.0   0:00.00 rcuob/0
   12 root      rt   0       0      0      0 S  0.0  0.0   0:45.92 migration/0

and ps aufx

$ ps aufx | head
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         2  0.0  0.0      0     0 ?        S    Jan30   0:00 [kthreadd]

... snip

root        11  0.0  0.0      0     0 ?        S    Jan30   0:00  \_ [rcuob/0]
root        12  0.0  0.0      0     0 ?        S    Jan30   0:45  \_ [migration/0]

---

according to this question on askubuntu.com , it suggests that the CPU% you see attributed to migration/N threads (with no associated process) are "the threads you are seeing are kernel threads responsible for moving threads between CPUs."

There is a thread here which references a kernel issue relating to migration/N threads thrashing under no load, if you are seeing associated performance issues.

Comments

[ckujau]

This doesn't answer the question why "top" cannot see it, or does it?

[gokva]

In "threads" mode, I see the migration/N threads in "top -H", the same as "ps aufx | head". So I guess the OP needed to provide the "-H" option to top, or alternative press "H" when in interactive mode. Reading back, I agree, I don't think what I wrote answers the question very well. :-(

[ckujau]

While "top -H" shows individual threads, I came across the same disconnect between "ps" and "top" and thus came across this question on Serverfault: "ps" reported a very high CPU usage for (kernel) threads but they did not show up in "top" or "top -H". This comment may get too long to explain in detail, two pointers though: Gentoo #394487 and upstream #47341. So, either it's a kernel problem or a problem with an old procps version.