rcu: Add callback-free CPUs

RCU callback execution can add significant OS jitter and also can
degrade both scheduling latency and, in asymmetric multiprocessors,
energy efficiency.  This commit therefore adds the ability for selected
CPUs ("rcu_nocbs=" boot parameter) to have their callbacks offloaded
to kthreads.  If the "rcu_nocb_poll" boot parameter is also specified,
these kthreads will do polling, removing the need for the offloaded
CPUs to do wakeups.  At least one CPU must be doing normal callback
processing: currently CPU 0 cannot be selected as a no-CBs CPU.
In addition, attempts to offline the last normal-CBs CPU will fail.

This feature was inspired by Jim Houston's and Joe Korty's JRCU, and
this commit includes fixes to problems located by Fengguang Wu's
kbuild test robot.

[ paulmck: Added gfp.h include file as suggested by Fengguang Wu. ]

Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

1 files changed, 22 insertions, 0 deletions

diff --git a/init/Kconfig b/init/Kconfig
index ec62139207d3..5ac6ee094225 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -654,6 +654,28 @@ config RCU_BOOST_DELAY
 
           Accept the default if unsure.
 
+config RCU_NOCB_CPU
+        bool "Offload RCU callback processing from boot-selected CPUs"
+        depends on TREE_RCU || TREE_PREEMPT_RCU
+        default n
+        help
+          Use this option to reduce OS jitter for aggressive HPC or
+          real-time workloads.  It can also be used to offload RCU
+          callback invocation to energy-efficient CPUs in battery-powered
+          asymmetric multiprocessors.
+
+          This option offloads callback invocation from the set of
+          CPUs specified at boot time by the rcu_nocbs parameter.
+          For each such CPU, a kthread ("rcuoN") will be created to
+          invoke callbacks, where the "N" is the CPU being offloaded.
+          Nothing prevents this kthread from running on the specified
+          CPUs, but (1) the kthreads may be preempted between each
+          callback, and (2) affinity or cgroups can be used to force
+          the kthreads to run on whatever set of CPUs is desired.
+
+          Say Y here if you want reduced OS jitter on selected CPUs.
+          Say N here if you are unsure.
+
 endmenu # "RCU Subsystem"
 
 config IKCONFIG