aboutsummaryrefslogtreecommitdiffstats
path: root/kernel/rcutree.c
diff options
context:
space:
mode:
authorPaul E. McKenney <paulmck@linux.vnet.ibm.com>2010-02-22 20:04:59 -0500
committerIngo Molnar <mingo@elte.hu>2010-02-25 04:34:55 -0500
commit8bd93a2c5d4cab2ae17d06350daa7dbf546a4634 (patch)
tree3facbdbfbcc1b169fad20f456b0a2521adadfb25 /kernel/rcutree.c
parent998f2ac3fea93bfa8b55c279fff68f7c5b9ab93d (diff)
rcu: Accelerate grace period if last non-dynticked CPU
Currently, rcu_needs_cpu() simply checks whether the current CPU has an outstanding RCU callback, which means that the last CPU to go into dyntick-idle mode might wait a few ticks for the relevant grace periods to complete. However, if all the other CPUs are in dyntick-idle mode, and if this CPU is in a quiescent state (which it is for RCU-bh and RCU-sched any time that we are considering going into dyntick-idle mode), then the grace period is instantly complete. This patch therefore repeatedly invokes the RCU grace-period machinery in order to force any needed grace periods to complete quickly. It does so a limited number of times in order to prevent starvation by an RCU callback function that might pass itself to call_rcu(). However, if any CPU other than the current one is not in dyntick-idle mode, fall back to simply checking (with fix to bug noted by Lai Jiangshan). Also, take advantage of last grace-period forcing, the opportunity to do so noted by Steve Rostedt. And apply simplified #ifdef condition suggested by Frederic Weisbecker. Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: laijs@cn.fujitsu.com Cc: dipankar@in.ibm.com Cc: mathieu.desnoyers@polymtl.ca Cc: josh@joshtriplett.org Cc: dvhltc@us.ibm.com Cc: niv@us.ibm.com Cc: peterz@infradead.org Cc: rostedt@goodmis.org Cc: Valdis.Kletnieks@vt.edu Cc: dhowells@redhat.com LKML-Reference: <1266887105-1528-15-git-send-email-paulmck@linux.vnet.ibm.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
Diffstat (limited to 'kernel/rcutree.c')
-rw-r--r--kernel/rcutree.c5
1 files changed, 2 insertions, 3 deletions
diff --git a/kernel/rcutree.c b/kernel/rcutree.c
index 099a255ede4..29d88c08d87 100644
--- a/kernel/rcutree.c
+++ b/kernel/rcutree.c
@@ -1550,10 +1550,9 @@ static int rcu_pending(int cpu)
1550/* 1550/*
1551 * Check to see if any future RCU-related work will need to be done 1551 * Check to see if any future RCU-related work will need to be done
1552 * by the current CPU, even if none need be done immediately, returning 1552 * by the current CPU, even if none need be done immediately, returning
1553 * 1 if so. This function is part of the RCU implementation; it is -not- 1553 * 1 if so.
1554 * an exported member of the RCU API.
1555 */ 1554 */
1556int rcu_needs_cpu(int cpu) 1555static int rcu_needs_cpu_quick_check(int cpu)
1557{ 1556{
1558 /* RCU callbacks either ready or pending? */ 1557 /* RCU callbacks either ready or pending? */
1559 return per_cpu(rcu_sched_data, cpu).nxtlist || 1558 return per_cpu(rcu_sched_data, cpu).nxtlist ||