rcu: Fix and comment ordering around wait_event()

It is all too easy to forget that wait_event() does not necessarily
imply a full memory barrier.  The case where it does not is where the
condition transitions to true just as wait_event() starts execution.
This is actually a feature: The standard use of wait_event() involves
locking, in which case the locks provide the needed ordering (you hold a
lock across the wake_up() and acquire that same lock after wait_event()
returns).

Given that I did forget that wait_event() does not necessarily imply a
full memory barrier in one case, this commit fixes that case.  This commit
also adds comments calling out the placement of existing memory barriers
relied on by wait_event() calls.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

1 files changed, 5 insertions, 3 deletions

diff --git a/kernel/rcu/torture.c b/kernel/rcu/torture.c
index 3929cd451511..69a4ec80a788 100644
--- a/kernel/rcu/torture.c
+++ b/kernel/rcu/torture.c
@@ -1578,6 +1578,7 @@ static int rcu_torture_barrier_cbs(void *arg)
 {
         long myid = (long)arg;
         bool lastphase = 0;
+        bool newphase;
         struct rcu_head rcu;
 
         init_rcu_head_on_stack(&rcu);
@@ -1585,10 +1586,11 @@ static int rcu_torture_barrier_cbs(void *arg)
         set_user_nice(current, 19);
         do {
                 wait_event(barrier_cbs_wq[myid],
-                           barrier_phase != lastphase ||
+                           (newphase =
+                            ACCESS_ONCE(barrier_phase)) != lastphase ||
                            kthread_should_stop() ||
                            fullstop != FULLSTOP_DONTSTOP);
-                lastphase = barrier_phase;
+                lastphase = newphase;
                 smp_mb(); /* ensure barrier_phase load before ->call(). */
                 if (kthread_should_stop() || fullstop != FULLSTOP_DONTSTOP)
                         break;
@@ -1625,7 +1627,7 @@ static int rcu_torture_barrier(void *arg)
                 if (kthread_should_stop() || fullstop != FULLSTOP_DONTSTOP)
                         break;
                 n_barrier_attempts++;
-                cur_ops->cb_barrier();
+                cur_ops->cb_barrier(); /* Implies smp_mb() for wait_event(). */
                 if (atomic_read(&barrier_cbs_invoked) != n_barrier_cbs) {
                         n_rcu_torture_barrier_error++;
                         WARN_ON_ONCE(1);