Merge branch 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull RCU updates from Ingo Molnar:
 "The main RCU changes in this cycle are:

   - Idle entry/exit changes, to throttle callback execution and other
     refinements to speed up kbuild, primarily to address performance
     issues located by Tibor Billes.

   - Grace-period related changes, primarily to aid in debugging,
     inspired by an -rt debugging session.

   - Code reorganization moving RCU's source files into its own
     kernel/rcu/ directory.

   - RCU documentation updates

   - Miscellaneous fixes.

  Note, the following commit:

    5c889690aa08 mm: Place preemption point in do_mlockall() loop

  is identical to the commit already in your tree via email:

    22356f447ceb mm: Place preemption point in do_mlockall() loop

  [ Your version of the changelog nicely demonstrates it how kernel oops
    messages should be trimmed properly :-/ ]"

* 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (30 commits)
  rcu: Move RCU-related source code to kernel/rcu directory
  rcu: Fix occurrence of "the the" in checklist.txt
  kthread: Add pointer to vmstat-avoidance patch
  rcu: Update stall-warning documentation
  rcu: Consistent rcu_is_watching() naming
  rcu: Change EXPORT_SYMBOL() to EXPORT_SYMBOL_GPL()
  rcu: Is it safe to enter an RCU read-side critical section?
  rcu: Throttle invoke_rcu_core() invocations due to non-lazy callbacks
  rcu: Throttle rcu_try_advance_all_cbs() execution
  rcu: Remove redundant code from rcu_cleanup_after_idle()
  rcu: Fix CONFIG_RCU_NOCB_CPU_ALL panic on machines with sparse CPU mask
  rcu: Avoid sparse warnings in rcu_nocb_wake trace event
  rcu: Track rcu_nocb_kthread()'s sleeping and awakening
  rcu: Distinguish between NOCB and non-NOCB rcu_callback trace events
  rcu: Add tracing for rcuo no-CBs CPU wakeup handshake
  rcu: Add tracing of normal (non-NOCB) grace-period requests
  rcu: Add tracing to rcu_gp_kthread()
  rcu: Flag lockless access to ->gp_flags with ACCESS_ONCE()
  rcu: Prevent spurious-wakeup DoS attack on rcu_gp_kthread()
  rcu: Improve grace-period start logic
  ...

5 files changed, 89 insertions, 54 deletions

diff --git a/Documentation/DocBook/device-drivers.tmpl b/Documentation/DocBook/device-drivers.tmpl
index fe397f90a34f..6c9d9d37c83a 100644
--- a/Documentation/DocBook/device-drivers.tmpl
+++ b/Documentation/DocBook/device-drivers.tmpl
@@ -87,7 +87,10 @@ X!Iinclude/linux/kobject.h
 !Ekernel/printk/printk.c
 !Ekernel/panic.c
 !Ekernel/sys.c
-!Ekernel/rcupdate.c
+!Ekernel/rcu/srcu.c
+!Ekernel/rcu/tree.c
+!Ekernel/rcu/tree_plugin.h
+!Ekernel/rcu/update.c
      </sect1>
 
      <sect1><title>Device Resource Management</title>
diff --git a/Documentation/RCU/checklist.txt b/Documentation/RCU/checklist.txt
index 7703ec73a9bb..91266193b8f4 100644
--- a/Documentation/RCU/checklist.txt
+++ b/Documentation/RCU/checklist.txt
@@ -202,8 +202,8 @@ over a rather long period of time, but improvements are always welcome!
         updater uses call_rcu_sched() or synchronize_sched(), then
         the corresponding readers must disable preemption, possibly
         by calling rcu_read_lock_sched() and rcu_read_unlock_sched().
-        If the updater uses synchronize_srcu() or call_srcu(),
-        the the corresponding readers must use srcu_read_lock() and
+        If the updater uses synchronize_srcu() or call_srcu(), then
+        the corresponding readers must use srcu_read_lock() and
         srcu_read_unlock(), and with the same srcu_struct.  The rules for
         the expedited primitives are the same as for their non-expedited
         counterparts.  Mixing things up will result in confusion and
diff --git a/Documentation/RCU/stallwarn.txt b/Documentation/RCU/stallwarn.txt
index 8e9359de1d28..6f3a0057548e 100644
--- a/Documentation/RCU/stallwarn.txt
+++ b/Documentation/RCU/stallwarn.txt
@@ -12,12 +12,12 @@ CONFIG_RCU_CPU_STALL_TIMEOUT
         This kernel configuration parameter defines the period of time
         that RCU will wait from the beginning of a grace period until it
         issues an RCU CPU stall warning.  This time period is normally
-        sixty seconds.
+        21 seconds.
 
         This configuration parameter may be changed at runtime via the
         /sys/module/rcutree/parameters/rcu_cpu_stall_timeout, however
         this parameter is checked only at the beginning of a cycle.
-        So if you are 30 seconds into a 70-second stall, setting this
+        So if you are 10 seconds into a 40-second stall, setting this
         sysfs parameter to (say) five will shorten the timeout for the
         -next- stall, or the following warning for the current stall
         (assuming the stall lasts long enough).  It will not affect the
@@ -32,7 +32,7 @@ CONFIG_RCU_CPU_STALL_VERBOSE
         also dump the stacks of any tasks that are blocking the current
         RCU-preempt grace period.
 
-RCU_CPU_STALL_INFO
+CONFIG_RCU_CPU_STALL_INFO
 
         This kernel configuration parameter causes the stall warning to
         print out additional per-CPU diagnostic information, including
@@ -43,7 +43,8 @@ RCU_STALL_DELAY_DELTA
         Although the lockdep facility is extremely useful, it does add
         some overhead.  Therefore, under CONFIG_PROVE_RCU, the
         RCU_STALL_DELAY_DELTA macro allows five extra seconds before
-        giving an RCU CPU stall warning message.
+        giving an RCU CPU stall warning message.  (This is a cpp
+        macro, not a kernel configuration parameter.)
 
 RCU_STALL_RAT_DELAY
 
@@ -52,7 +53,8 @@ RCU_STALL_RAT_DELAY
         However, if the offending CPU does not detect its own stall in
         the number of jiffies specified by RCU_STALL_RAT_DELAY, then
         some other CPU will complain.  This delay is normally set to
-        two jiffies.
+        two jiffies.  (This is a cpp macro, not a kernel configuration
+        parameter.)
 
 When a CPU detects that it is stalling, it will print a message similar
 to the following:
@@ -86,7 +88,12 @@ printing, there will be a spurious stall-warning message:
 
 INFO: rcu_bh_state detected stalls on CPUs/tasks: { } (detected by 4, 2502 jiffies)
 
-This is rare, but does happen from time to time in real life.
+This is rare, but does happen from time to time in real life.  It is also
+possible for a zero-jiffy stall to be flagged in this case, depending
+on how the stall warning and the grace-period initialization happen to
+interact.  Please note that it is not possible to entirely eliminate this
+sort of false positive without resorting to things like stop_machine(),
+which is overkill for this sort of problem.
 
 If the CONFIG_RCU_CPU_STALL_INFO kernel configuration parameter is set,
 more information is printed with the stall-warning message, for example:
@@ -216,4 +223,5 @@ that portion of the stack which remains the same from trace to trace.
 If you can reliably trigger the stall, ftrace can be quite helpful.
 
 RCU bugs can often be debugged with the help of CONFIG_RCU_TRACE
-and with RCU's event tracing.
+and with RCU's event tracing.  For information on RCU's event tracing,
+see include/trace/events/rcu.h.
diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index fcbb736d55fe..203f4a9d9efe 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -2599,7 +2599,7 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
         ramdisk_size=   [RAM] Sizes of RAM disks in kilobytes
                         See Documentation/blockdev/ramdisk.txt.
 
-        rcu_nocbs=      [KNL,BOOT]
+        rcu_nocbs=      [KNL]
                         In kernels built with CONFIG_RCU_NOCB_CPU=y, set
                         the specified list of CPUs to be no-callback CPUs.
                         Invocation of these CPUs' RCU callbacks will
@@ -2612,7 +2612,7 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
                         real-time workloads.  It can also improve energy
                         efficiency for asymmetric multiprocessors.
 
-        rcu_nocb_poll   [KNL,BOOT]
+        rcu_nocb_poll   [KNL]
                         Rather than requiring that offloaded CPUs
                         (specified by rcu_nocbs= above) explicitly
                         awaken the corresponding "rcuoN" kthreads,
@@ -2623,126 +2623,145 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
                         energy efficiency by requiring that the kthreads
                         periodically wake up to do the polling.
 
-        rcutree.blimit= [KNL,BOOT]
+        rcutree.blimit= [KNL]
                         Set maximum number of finished RCU callbacks to process
                         in one batch.
 
-        rcutree.fanout_leaf=    [KNL,BOOT]
+        rcutree.rcu_fanout_leaf= [KNL]
                         Increase the number of CPUs assigned to each
                         leaf rcu_node structure.  Useful for very large
                         systems.
 
-        rcutree.jiffies_till_first_fqs= [KNL,BOOT]
+        rcutree.jiffies_till_first_fqs= [KNL]
                         Set delay from grace-period initialization to
                         first attempt to force quiescent states.
                         Units are jiffies, minimum value is zero,
                         and maximum value is HZ.
 
-        rcutree.jiffies_till_next_fqs= [KNL,BOOT]
+        rcutree.jiffies_till_next_fqs= [KNL]
                         Set delay between subsequent attempts to force
                         quiescent states.  Units are jiffies, minimum
                         value is one, and maximum value is HZ.
 
-        rcutree.qhimark=        [KNL,BOOT]
+        rcutree.qhimark= [KNL]
                         Set threshold of queued
                         RCU callbacks over which batch limiting is disabled.
 
-        rcutree.qlowmark=       [KNL,BOOT]
+        rcutree.qlowmark= [KNL]
                         Set threshold of queued RCU callbacks below which
                         batch limiting is re-enabled.
 
-        rcutree.rcu_cpu_stall_suppress= [KNL,BOOT]
-                        Suppress RCU CPU stall warning messages.
-
-        rcutree.rcu_cpu_stall_timeout= [KNL,BOOT]
-                        Set timeout for RCU CPU stall warning messages.
-
-        rcutree.rcu_idle_gp_delay=      [KNL,BOOT]
+        rcutree.rcu_idle_gp_delay= [KNL]
                         Set wakeup interval for idle CPUs that have
                         RCU callbacks (RCU_FAST_NO_HZ=y).
 
-        rcutree.rcu_idle_lazy_gp_delay= [KNL,BOOT]
+        rcutree.rcu_idle_lazy_gp_delay= [KNL]
                         Set wakeup interval for idle CPUs that have
                         only "lazy" RCU callbacks (RCU_FAST_NO_HZ=y).
                         Lazy RCU callbacks are those which RCU can
                         prove do nothing more than free memory.
 
-        rcutorture.fqs_duration= [KNL,BOOT]
+        rcutorture.fqs_duration= [KNL]
                         Set duration of force_quiescent_state bursts.
 
-        rcutorture.fqs_holdoff= [KNL,BOOT]
+        rcutorture.fqs_holdoff= [KNL]
                         Set holdoff time within force_quiescent_state bursts.
 
-        rcutorture.fqs_stutter= [KNL,BOOT]
+        rcutorture.fqs_stutter= [KNL]
                         Set wait time between force_quiescent_state bursts.
 
-        rcutorture.irqreader= [KNL,BOOT]
-                        Test RCU readers from irq handlers.
+        rcutorture.gp_exp= [KNL]
+                        Use expedited update-side primitives.
+
+        rcutorture.gp_normal= [KNL]
+                        Use normal (non-expedited) update-side primitives.
+                        If both gp_exp and gp_normal are set, do both.
+                        If neither gp_exp nor gp_normal are set, still
+                        do both.
 
-        rcutorture.n_barrier_cbs= [KNL,BOOT]
+        rcutorture.n_barrier_cbs= [KNL]
                         Set callbacks/threads for rcu_barrier() testing.
 
-        rcutorture.nfakewriters= [KNL,BOOT]
+        rcutorture.nfakewriters= [KNL]
                         Set number of concurrent RCU writers.  These just
                         stress RCU, they don't participate in the actual
                         test, hence the "fake".
 
-        rcutorture.nreaders= [KNL,BOOT]
+        rcutorture.nreaders= [KNL]
                         Set number of RCU readers.
 
-        rcutorture.onoff_holdoff= [KNL,BOOT]
+        rcutorture.object_debug= [KNL]
+                        Enable debug-object double-call_rcu() testing.
+
+        rcutorture.onoff_holdoff= [KNL]
                         Set time (s) after boot for CPU-hotplug testing.
 
-        rcutorture.onoff_interval= [KNL,BOOT]
+        rcutorture.onoff_interval= [KNL]
                         Set time (s) between CPU-hotplug operations, or
                         zero to disable CPU-hotplug testing.
 
-        rcutorture.shuffle_interval= [KNL,BOOT]
+        rcutorture.rcutorture_runnable= [BOOT]
+                        Start rcutorture running at boot time.
+
+        rcutorture.shuffle_interval= [KNL]
                         Set task-shuffle interval (s).  Shuffling tasks
                         allows some CPUs to go into dyntick-idle mode
                         during the rcutorture test.
 
-        rcutorture.shutdown_secs= [KNL,BOOT]
+        rcutorture.shutdown_secs= [KNL]
                         Set time (s) after boot system shutdown.  This
                         is useful for hands-off automated testing.
 
-        rcutorture.stall_cpu= [KNL,BOOT]
+        rcutorture.stall_cpu= [KNL]
                         Duration of CPU stall (s) to test RCU CPU stall
                         warnings, zero to disable.
 
-        rcutorture.stall_cpu_holdoff= [KNL,BOOT]
+        rcutorture.stall_cpu_holdoff= [KNL]
                         Time to wait (s) after boot before inducing stall.
 
-        rcutorture.stat_interval= [KNL,BOOT]
+        rcutorture.stat_interval= [KNL]
                         Time (s) between statistics printk()s.
 
-        rcutorture.stutter= [KNL,BOOT]
+        rcutorture.stutter= [KNL]
                         Time (s) to stutter testing, for example, specifying
                         five seconds causes the test to run for five seconds,
                         wait for five seconds, and so on.  This tests RCU's
                         ability to transition abruptly to and from idle.
 
-        rcutorture.test_boost= [KNL,BOOT]
+        rcutorture.test_boost= [KNL]
                         Test RCU priority boosting?  0=no, 1=maybe, 2=yes.
                         "Maybe" means test if the RCU implementation
                         under test support RCU priority boosting.
 
-        rcutorture.test_boost_duration= [KNL,BOOT]
+        rcutorture.test_boost_duration= [KNL]
                         Duration (s) of each individual boost test.
 
-        rcutorture.test_boost_interval= [KNL,BOOT]
+        rcutorture.test_boost_interval= [KNL]
                         Interval (s) between each boost test.
 
-        rcutorture.test_no_idle_hz= [KNL,BOOT]
+        rcutorture.test_no_idle_hz= [KNL]
                         Test RCU's dyntick-idle handling.  See also the
                         rcutorture.shuffle_interval parameter.
 
-        rcutorture.torture_type= [KNL,BOOT]
+        rcutorture.torture_type= [KNL]
                         Specify the RCU implementation to test.
 
-        rcutorture.verbose= [KNL,BOOT]
+        rcutorture.verbose= [KNL]
                         Enable additional printk() statements.
 
+        rcupdate.rcu_expedited= [KNL]
+                        Use expedited grace-period primitives, for
+                        example, synchronize_rcu_expedited() instead
+                        of synchronize_rcu().  This reduces latency,
+                        but can increase CPU utilization, degrade
+                        real-time latency, and degrade energy efficiency.
+
+        rcupdate.rcu_cpu_stall_suppress= [KNL]
+                        Suppress RCU CPU stall warning messages.
+
+        rcupdate.rcu_cpu_stall_timeout= [KNL]
+                        Set timeout for RCU CPU stall warning messages.
+
         rdinit=         [KNL]
                         Format: <full_path>
                         Run specified binary instead of /init from the ramdisk,
diff --git a/Documentation/kernel-per-CPU-kthreads.txt b/Documentation/kernel-per-CPU-kthreads.txt
index 32351bfabf20..827104fb9364 100644
--- a/Documentation/kernel-per-CPU-kthreads.txt
+++ b/Documentation/kernel-per-CPU-kthreads.txt
@@ -181,12 +181,17 @@ To reduce its OS jitter, do any of the following:
                 make sure that this is safe on your particular system.
         d.      It is not possible to entirely get rid of OS jitter
                 from vmstat_update() on CONFIG_SMP=y systems, but you
-                can decrease its frequency by writing a large value to
-                /proc/sys/vm/stat_interval.  The default value is HZ,
-                for an interval of one second.  Of course, larger values
-                will make your virtual-memory statistics update more
-                slowly.  Of course, you can also run your workload at
-                a real-time priority, thus preempting vmstat_update().
+                can decrease its frequency by writing a large value
+                to /proc/sys/vm/stat_interval.  The default value is
+                HZ, for an interval of one second.  Of course, larger
+                values will make your virtual-memory statistics update
+                more slowly.  Of course, you can also run your workload
+                at a real-time priority, thus preempting vmstat_update(),
+                but if your workload is CPU-bound, this is a bad idea.
+                However, there is an RFC patch from Christoph Lameter
+                (based on an earlier one from Gilad Ben-Yossef) that
+                reduces or even eliminates vmstat overhead for some
+                workloads at https://lkml.org/lkml/2013/9/4/379.
         e.      If running on high-end powerpc servers, build with
                 CONFIG_PPC_RTAS_DAEMON=n.  This prevents the RTAS
                 daemon from running on each CPU every second or so.