[ARM] barriers: improve xchg, bitops and atomic SMP barriers

Mathieu Desnoyers pointed out that the ARM barriers were lacking:

- cmpxchg, xchg and atomic add return need memory barriers on
  architectures which can reorder the relative order in which memory
  read/writes can be seen between CPUs, which seems to include recent
  ARM architectures. Those barriers are currently missing on ARM.

- test_and_xxx_bit were missing SMP barriers.

So put these barriers in.  Provide separate atomic_add/atomic_sub
operations which do not require barriers.

Reported-Reviewed-and-Acked-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>

1 files changed, 2 insertions, 0 deletions

diff --git a/arch/arm/lib/bitops.h b/arch/arm/lib/bitops.h
index 2e787d40d599..c7f2627385e7 100644
--- a/arch/arm/lib/bitops.h
+++ b/arch/arm/lib/bitops.h
@@ -18,12 +18,14 @@
         mov     r2, #1
         add     r1, r1, r0, lsr #3      @ Get byte offset
         mov     r3, r2, lsl r3          @ create mask
+        smp_dmb
 1:      ldrexb  r2, [r1]
         ands    r0, r2, r3              @ save old value of bit
         \instr  r2, r2, r3                      @ toggle bit
         strexb  ip, r2, [r1]
         cmp     ip, #0
         bne     1b
+        smp_dmb
         cmp     r0, #0
         movne   r0, #1
 2:      mov     pc, lr