locking/barriers: Replace smp_cond_acquire() with smp_cond_load_acquire()

This new form allows using hardware assisted waiting.

Some hardware (ARM64 and x86) allow monitoring an address for changes,
so by providing a pointer we can use this to replace the cpu_relax()
with hardware optimized methods in the future.

Requested-by: Will Deacon <will.deacon@arm.com>
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>

1 files changed, 6 insertions, 6 deletions

diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c
index 2f9153b183c9..1b8dda90ebfa 100644
--- a/kernel/locking/qspinlock.c
+++ b/kernel/locking/qspinlock.c
@@ -475,7 +475,7 @@ void queued_spin_lock_slowpath(struct qspinlock *lock, u32 val)
          * sequentiality; this is because not all clear_pending_set_locked()
          * implementations imply full barriers.
          */
-        smp_cond_acquire(!(atomic_read(&lock->val) & _Q_LOCKED_MASK));
+        smp_cond_load_acquire(&lock->val.counter, !(VAL & _Q_LOCKED_MASK));
 
         /*
          * take ownership and clear the pending bit.
@@ -562,7 +562,7 @@ queue:
          *
          * The PV pv_wait_head_or_lock function, if active, will acquire
          * the lock and return a non-zero value. So we have to skip the
-         * smp_cond_acquire() call. As the next PV queue head hasn't been
+         * smp_cond_load_acquire() call. As the next PV queue head hasn't been
          * designated yet, there is no way for the locked value to become
          * _Q_SLOW_VAL. So both the set_locked() and the
          * atomic_cmpxchg_relaxed() calls will be safe.
@@ -573,7 +573,7 @@ queue:
         if ((val = pv_wait_head_or_lock(lock, node)))
                 goto locked;
 
-        smp_cond_acquire(!((val = atomic_read(&lock->val)) & _Q_LOCKED_PENDING_MASK));
+        val = smp_cond_load_acquire(&lock->val.counter, !(VAL & _Q_LOCKED_PENDING_MASK));
 
 locked:
         /*
@@ -593,9 +593,9 @@ locked:
                         break;
                 }
                 /*
-                 * The smp_cond_acquire() call above has provided the necessary
-                 * acquire semantics required for locking. At most two
-                 * iterations of this loop may be ran.
+                 * The smp_cond_load_acquire() call above has provided the
+                 * necessary acquire semantics required for locking. At most
+                 * two iterations of this loop may be ran.
                  */
                 old = atomic_cmpxchg_relaxed(&lock->val, val, _Q_LOCKED_VAL);
                 if (old == val)