rcu: Avoid losing ->need_future_gp[] values due to GP start/end races

The rcu_cbs_completed() function provides the value of ->completed
at which new callbacks can safely be invoked.  This is recorded in
two-element ->need_future_gp[] arrays in the rcu_node structure, and
the elements of these arrays corresponding to the just-completed grace
period are zeroed at the end of that grace period.  However, the
rcu_cbs_completed() function can return the current ->completed value
plus either one or two, so it is possible for the corresponding
->need_future_gp[] entry to be cleared just after it was set, thus
losing a request for a future grace period.

This commit avoids this race by expanding ->need_future_gp[] to four
elements.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>

1 files changed, 1 insertions, 2 deletions

diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h
index bd1103763551..952cd0c223fe 100644
--- a/kernel/rcu/tree.h
+++ b/kernel/rcu/tree.h
@@ -150,8 +150,7 @@ struct rcu_node {
         struct swait_queue_head nocb_gp_wq[2];
                                 /* Place for rcu_nocb_kthread() to wait GP. */
 #endif /* #ifdef CONFIG_RCU_NOCB_CPU */
-        int need_future_gp[2];
-                                /* Counts of upcoming no-CB GP requests. */
+        int need_future_gp[4];  /* Counts of upcoming no-CB GP requests. */
         raw_spinlock_t fqslock ____cacheline_internodealigned_in_smp;
 
         spinlock_t exp_lock ____cacheline_internodealigned_in_smp;