x86, rwsem: Stay on fast path when count > 0 in __up_write()

When count > 0 there is no need to take the call_rwsem_wake path.  If
we did take that path, it would just return without doing anything due
to the active count not being zero.

Signed-off-by: Michel Lespinasse <walken@google.com>
LKML-Reference: <201007202219.o6KMJj9x021042@imap1.linux-foundation.org>
Acked-by: David Howells <dhowells@redhat.com>
Cc: Mike Waychison <mikew@google.com>
Cc: Suleiman Souhlal <suleiman@google.com>
Cc: Ying Han <yinghan@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>

1 files changed, 2 insertions, 3 deletions

diff --git a/arch/x86/include/asm/rwsem.h b/arch/x86/include/asm/rwsem.h
index 606ede126972..5bf5e04e497f 100644
--- a/arch/x86/include/asm/rwsem.h
+++ b/arch/x86/include/asm/rwsem.h
@@ -216,9 +216,8 @@ static inline void __up_write(struct rw_semaphore *sem)
         rwsem_count_t tmp;
         asm volatile("# beginning __up_write\n\t"
                      LOCK_PREFIX "  xadd      %1,(%2)\n\t"
-                     /* tries to transition
-                        0xffff0001 -> 0x00000000 */
-                     "  jz       1f\n"
+                     /* subtracts 0xffff0001, returns the old value */
+                     "  jns        1f\n\t"
                      "  call call_rwsem_wake\n"
                      "1:\n\t"
                      "# ending __up_write\n"