Understanding RCU when Configuring the Linux Kernel

Question

I am configuring the Linux kernel version 3.9.4. I am being asked questions about RCU (seen below). Specifically, what are each of these and what are the advantages and disadvantages of enabling or disabling some of these?

Consider userspace as in RCU extended quiescent state (RCU_USER_QS) [N/y/?]
Tree-based hierarchical RCU fanout value (RCU_FANOUT) [64]
Disable tree-based hierarchical RCU auto-balancing (RCU_FANOUT_EXACT) [N/y/?]
Accelerate last non-dyntick-idle CPU's grace periods (RCU_FAST_NO_HZ) [Y/n/?]
Offload RCU callback processing from boot-selected CPUs (RCU_NOCB_CPU) [N/y/?] 

Solution

There are some details about these options over on the LTTng Project site. RCU's are (read-copy-update). These are data structures in the kernel which allow for the same data to be replicated across cores in a multi-core CPU and they guarantee that the data will be kept in sync across the copies.

excerpt

liburcu is a LGPLv2.1 userspace RCU (read-copy-update) library. This data synchronization library provides read-side access which scales linearly with the number of cores. It does so by allowing multiples copies of a given data structure to live at the same time, and by monitoring the data structure accesses to detect grace periods after which memory reclamation is possible.

Resources

• There is a good reference to what RCU's are and how they work over on lwn.net titled: What is RCU, Fundamentally?.
• There's also this resource by the same title as lwn.net but it's different content.
• There is also the Wikipedia entry on the RCU topic too.
• Finally there's the Linux kernel documentation available here: rcu.txt.

So what are these options?

• RCU_USER_QS

This option sets hooks on kernel / userspace boundaries and puts RCU in extended quiescent state when the CPU runs in userspace. It means that when a CPU runs in userspace, it is excluded from the global RCU state machine and thus doesn't try to keep the timer tick on for RCU.

Unless you want to hack and help the development of the full dynticks mode, you shouldn't enable this option. It also adds unnecessary overhead.

If unsure say N

• RCU Fanout

This option controls the fanout of hierarchical implementations of RCU, allowing RCU to work efficiently on machines with large numbers of CPUs. This value must be at least the fourth root of NR_CPUS, which allows NR_CPUS to be insanely large. The default value of RCU_FANOUT should be used for production systems, but if you are stress-testing the RCU implementation itself, small RCU_FANOUT values allow you to test large-system code paths on small(er) systems.

Select a specific number if testing RCU itself. Take the default if unsure.

• RCU_FANOUT_EXACT

This option forces use of the exact RCU_FANOUT value specified, regardless of imbalances in the hierarchy. This is useful for testing RCU itself, and might one day be useful on systems with strong NUMA behavior.

Without RCU_FANOUT_EXACT, the code will balance the hierarchy.

Say N if unsure.

• RCU_FAST_NO_HZ

This option permits CPUs to enter dynticks-idle state even if they have RCU callbacks queued, and prevents RCU from waking these CPUs up more than roughly once every four jiffies (by default, you can adjust this using the rcutree.rcu_idle_gp_delay parameter), thus improving energy efficiency. On the other hand, this option increases the duration of RCU grace periods, for example, slowing down synchronize_rcu().

Say Y if energy efficiency is critically important, and you don't care about increased grace-period durations.

Say N if you are unsure.

• RCU_NOCB_CPU

Use this option to reduce OS jitter for aggressive HPC or real-time workloads. It can also be used to offload RCU callback invocation to energy-efficient CPUs in battery-powered asymmetric multiprocessors.

This option offloads callback invocation from the set of CPUs specified at boot time by the rcu_nocbs parameter. For each such CPU, a kthread ("rcuox/N") will be created to invoke callbacks, where the "N" is the CPU being offloaded, and where the "x" is "b" for RCU-bh, "p" for RCU-preempt, and "s" for RCU-sched. Nothing prevents this kthread from running on the specified CPUs, but (1) the kthreads may be preempted between each callback, and (2) affinity or cgroups can be used to force the kthreads to run on whatever set of CPUs is desired.

Say Y here if you want to help to debug reduced OS jitter. Say N here if you are unsure.

So do you need it?

I would say if you don't know what a particular option does when compiling the kernel then it's probably a safe bet that you can live without it. So I'd say no to those questions.

Also when doing this type of work I usually get the config file for the kernel I'm using with my distro and do a comparison to see if I'm missing any features. This is probably your best resource in terms of learning what all the features are about.

For example in Fedora there are sample configs included that you can refer to. Take a look at this page for more details: Building a custom kernel.