diff options
| author | Paul E. McKenney <paulmck@linux.vnet.ibm.com> | 2014-03-17 00:36:25 -0400 |
|---|---|---|
| committer | Paul E. McKenney <paulmck@linux.vnet.ibm.com> | 2014-05-14 12:46:11 -0400 |
| commit | ac1bea85781e9004da9b3e8a4b097c18492d857c (patch) | |
| tree | e28ea65bf56d1624371885954a46ab64cab1524b /kernel/rcu | |
| parent | 0e980234c97f98be6619b9281d83777f725b94ff (diff) | |
sched,rcu: Make cond_resched() report RCU quiescent states
Given a CPU running a loop containing cond_resched(), with no
other tasks runnable on that CPU, RCU will eventually report RCU
CPU stall warnings due to lack of quiescent states. Fortunately,
every call to cond_resched() is a perfectly good quiescent state.
Unfortunately, invoking rcu_note_context_switch() is a bit heavyweight
for cond_resched(), especially given the need to disable preemption,
and, for RCU-preempt, interrupts as well.
This commit therefore maintains a per-CPU counter that causes
cond_resched(), cond_resched_lock(), and cond_resched_softirq() to call
rcu_note_context_switch(), but only about once per 256 invocations.
This ratio was chosen in keeping with the relative time constants of
RCU grace periods.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Diffstat (limited to 'kernel/rcu')
| -rw-r--r-- | kernel/rcu/update.c | 18 |
1 files changed, 18 insertions, 0 deletions
diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c index 4c0a9b0af469..ed7a0d72562c 100644 --- a/kernel/rcu/update.c +++ b/kernel/rcu/update.c | |||
| @@ -338,3 +338,21 @@ static int __init check_cpu_stall_init(void) | |||
| 338 | early_initcall(check_cpu_stall_init); | 338 | early_initcall(check_cpu_stall_init); |
| 339 | 339 | ||
| 340 | #endif /* #ifdef CONFIG_RCU_STALL_COMMON */ | 340 | #endif /* #ifdef CONFIG_RCU_STALL_COMMON */ |
| 341 | |||
| 342 | /* | ||
| 343 | * Hooks for cond_resched() and friends to avoid RCU CPU stall warnings. | ||
| 344 | */ | ||
| 345 | |||
| 346 | DEFINE_PER_CPU(int, rcu_cond_resched_count); | ||
| 347 | |||
| 348 | /* | ||
| 349 | * Report a set of RCU quiescent states, for use by cond_resched() | ||
| 350 | * and friends. Out of line due to being called infrequently. | ||
| 351 | */ | ||
| 352 | void rcu_resched(void) | ||
| 353 | { | ||
| 354 | preempt_disable(); | ||
| 355 | __this_cpu_write(rcu_cond_resched_count, 0); | ||
| 356 | rcu_note_context_switch(smp_processor_id()); | ||
| 357 | preempt_enable(); | ||
| 358 | } | ||