Ubuntu 16.04 kworker using high CPU constantly

Question

On Ubuntu 16.04 computer I saw kworker take up 80% of a CPU, consistently.

I generated a perf report:

• sudo apt-get install linux-tools-common linux-tools-$(uname -r)
• Record some 10 seconds of backtraces on all CPUs sudo perf record -g -a sleep 10
• View it with sudo perf report.

Top few results are:

Samples: 47K of event 'cycles:ppp', Event count (approx.): 38282161158
  Children      Self  Command          Shared Object                  Symbol
+   77.04%     0.00%  kworker/3:3      [kernel.kallsyms]              [k] ret_from_fork
+   77.04%     0.00%  kworker/3:3      [kernel.kallsyms]              [k] kthread
+   77.04%     0.00%  kworker/3:3      [kernel.kallsyms]              [k] worker_thread
+   77.00%     0.13%  kworker/3:3      [kernel.kallsyms]              [k] process_one_work
+   76.26%     0.04%  kworker/3:3      [kernel.kallsyms]              [k] rpm_idle
+   75.98%     0.06%  kworker/3:3      [kernel.kallsyms]              [k] rpm_suspend
+   75.97%     0.03%  kworker/3:3      [kernel.kallsyms]              [k] pm_runtime_work
+   75.84%     0.03%  kworker/3:3      [kernel.kallsyms]              [k] __rpm_callback
+   75.82%     0.01%  kworker/3:3      [kernel.kallsyms]              [k] usb_runtime_idle
+   75.76%     0.00%  kworker/3:3      [kernel.kallsyms]              [k] __pm_runtime_suspend
+   75.64%     0.00%  kworker/3:3      [kernel.kallsyms]              [k] rpm_callback
+   75.61%     0.04%  kworker/3:3      [kernel.kallsyms]              [k] usb_runtime_suspend
+   75.51%     0.03%  kworker/3:3      [kernel.kallsyms]              [k] usb_suspend_both
+   73.51%     0.15%  kworker/3:3      [kernel.kallsyms]              [k] usb_control_msg
+   71.97%     0.14%  kworker/3:3      [kernel.kallsyms]              [k] usb_start_wait_urb
+   68.38%     0.05%  kworker/3:3      [kernel.kallsyms]              [k] usb_submit_urb
+   68.33%     0.30%  kworker/3:3      [kernel.kallsyms]              [k] usb_submit_urb.part.8
+   67.75%     0.82%  kworker/3:3      [kernel.kallsyms]              [k] usb_hcd_submit_urb
+   60.96%    60.41%  kworker/3:3      [kernel.kallsyms]              [k] xhci_hub_control
+   55.99%     0.12%  kworker/3:3      [kernel.kallsyms]              [k] hub_suspend
+   54.97%     0.04%  kworker/3:3      [kernel.kallsyms]              [k] set_port_feature
+   19.36%     0.01%  kworker/3:3      [kernel.kallsyms]              [k] usb_resume_interface.isra.8
+   19.35%     0.02%  kworker/3:3      [kernel.kallsyms]              [k] hub_resume
+   19.32%     0.14%  kworker/3:3      [kernel.kallsyms]              [k] hub_activate
+   18.78%     0.08%  kworker/3:3      [kernel.kallsyms]              [k] hub_ext_port_status
+   11.13%     0.00%  ksoftirqd/3      [kernel.kallsyms]              [k] ret_from_fork
+   11.13%     0.00%  ksoftirqd/3      [kernel.kallsyms]              [k] kthread
+   10.44%     0.51%  ksoftirqd/3      [kernel.kallsyms]              [k] smpboot_thread_fn

I do not know how to reproduce. It goes away after a reboot. What is going on?

Solution

Same problem here on a Dell XPS laptop and kernel 4.4.0-130

I fixed it by disabling dynamic USB power management in kernel

I added usbcore.autosuspend=-1 to my kernel's boot command line:

GRUB_CMDLINE_LINUX_DEFAULT="<existing stuff> usbcore.autosuspend=-1"

• To do that permanently, open the grub configuration file

  sudo nano /etc/default/grub
  

and add the flag described above.

• Next you have to update your grub via

  sudo update-grub
  

• After reboot, everything should be fine, the busy CPU is now free for doing something useful :)

Disabling dynamic PM might impact battery life when you have USB devices connected, but few USB devices are able to handle this mode....

Comments

[sunapi386]

Thanks for your reply, however what you proposed are not actionable items for me. - Can't update the kernel because some drivers don't work well, but this remains a possible fix. - Can't update to Ubuntu 18 yet as there are program build issues, mainly needing cuda 8. - Not a good idea to limit the cpu because we need the compute resource. - Haven't found a fix yet. I'll post if I find something.

[Fergie]

Thanks- seems to work for me!

[vanadium]

Seems to work for me with Xubuntu 19.10 on Dell XPS 7390 although too early to say definitively

[Alexis Wilke]

That worked for me with a Dell PowerEdge T40 Server, BTX (Intel Xeon E-2224G). Same 100% CPU usage by a kworker even though it was a tower and not a laptop.

[Louis Gagnon]

this worked for me with Ubuntu 20.04.1 LTS where the problem had never showed up before the last two weeks. Thanks, you made my day!