How to get isolcpus kernel parameter working with Precise 12.04 amd64?
Solution 1
Disabled SMT (Hyper-Threading) in BIOS, power cycled, tested again with HT off and isolcpus works as expected.
Re-enabled SMT in BIOS, power cycled, still works as expected.
Definitely not the best answer you could expect, but it worked for me nevertheless.
Solution 2
isolcpus only isloate from ceratin user-space activity and will not isolate the CPU from the kernel activity (watchdogs, kworkers, linux stack ...). Moreover, you should not isolate cpu 0 (which has numerous dedicated activities like usb discovery, acpi timer setup, wrmsr/rdmsr dispatching .... which tend to amount to 2% with standard distro where all drivers are enabled by default)
You can verify which processes are running on which cores, by displaying all the threads sorted by core number.
ps -aFeL | cut -c 48- | sort -n
You also need to verify and modify the affinities in /proc/irq/* , in order to try to change the interrupt affinities.
You might dynamically spare cpus and remove "most" of linux kernel task using cpuset tool.
instead of
taskset -c 3-7,11-15 program args
try this
sudo apt install cpuset
cset set --list
cset shield -c 3-7,11-15
cset set --list
cset shield -e program -- args
And then, you can appreciate the difference
ps -aFeL | cut -c 48- | sort -n
When running these 2 commands
cset shield -e stress -- -c 16
stress -c 16
Related videos on Youtube
user3669605
Updated on September 18, 2022Comments
-
user3669605 over 1 year
I've been testing this stuff trying to reserve some CPUs in a host with two Intel E5645 but I can't get it working for some reason. Steps I followed:
-
Edit /etc/default/grub and added isolcpus=0,1 to GRUB_CMDLINE_LINUX_DEFAULT
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash isolcpus=0,1"
Run update-grub
- Reboot
After that, cat /proc/cmdline reveals:
BOOT_IMAGE=/boot/vmlinuz-3.2.0-26-generic root=UUID=52cfedad-40be-41b9-9f88-c282a7ae3f24 ro quiet splash isolcpus=0,1 vt.handoff=7
Tested using stress:
apt-get install stress && stress -c 24
Monitored using top and pressing 1 to display individual CPU stats. So far no CPUs are isolated from the scheduler and all of them are busted by stress.
Cpu0 : 99.7%us, 0.3%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu1 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu2 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu3 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu4 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu5 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu6 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu7 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu8 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu9 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu10 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu11 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu12 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu13 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu14 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu15 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu16 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu17 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu18 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu19 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu20 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu21 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu22 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa Cpu23 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa
Tested the same stuff in CentOS 6 x86_64 and it works as expected.
Searched ubuntu and linux bugs database in launchpad but did not find anything so far.
Is it me being dumb or am I missing something? Hints?
Thanks!
Refs:
http://www.kernel.org/doc/Documentation/kernel-parameters.txt
-
-
MonoThreaded over 9 yearsAre you saying that both reboots gave the same result regardless of the Hyper-Threading settings? In which case, maybe rebooting was the work around(?)