mmiowb: Hook up mmiowb helpers to spinlocks and generic I/O accessors

Removing explicit calls to mmiowb() from driver code means that we must
now call into the generic mmiowb_spin_{lock,unlock}() functions from the
core spinlock code. In order to elide barriers following critical
sections without any I/O writes, we also hook into the asm-generic I/O
routines.

Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Will Deacon <will.deacon@arm.com>

1 files changed, 5 insertions, 1 deletions

diff --git a/kernel/locking/spinlock_debug.c b/kernel/locking/spinlock_debug.c
index 9aa0fccd5d43..399669f7eba8 100644
--- a/kernel/locking/spinlock_debug.c
+++ b/kernel/locking/spinlock_debug.c
@@ -111,6 +111,7 @@ void do_raw_spin_lock(raw_spinlock_t *lock)
 {
         debug_spin_lock_before(lock);
         arch_spin_lock(&lock->raw_lock);
+        mmiowb_spin_lock();
         debug_spin_lock_after(lock);
 }
 
@@ -118,8 +119,10 @@ int do_raw_spin_trylock(raw_spinlock_t *lock)
 {
         int ret = arch_spin_trylock(&lock->raw_lock);
 
-        if (ret)
+        if (ret) {
+                mmiowb_spin_lock();
                 debug_spin_lock_after(lock);
+        }
 #ifndef CONFIG_SMP
         /*
          * Must not happen on UP:
@@ -131,6 +134,7 @@ int do_raw_spin_trylock(raw_spinlock_t *lock)
 
 void do_raw_spin_unlock(raw_spinlock_t *lock)
 {
+        mmiowb_spin_unlock();
         debug_spin_unlock(lock);
         arch_spin_unlock(&lock->raw_lock);
 }