wait: introduce wait_event_exclusive_cmd

It's just a variant of wait_event_cmd(), with exclusive flag being set.

For cases like RAID5, which puts many processes to sleep until 1/4
resources are free, a wake_up wakes up all processes to run, but
there is one process being able to get the resource as it's protected
by a spin lock. That ends up introducing heavy lock contentions, and
hurts performance badly.

Here introduce wait_event_exclusive_cmd to relieve the lock contention
naturally by letting wake_up just wake up one process.

Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
v2: its assumed that wait*() and __wait*() have the same arguments - peterz

Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>

1 files changed, 13 insertions, 0 deletions

diff --git a/include/linux/wait.h b/include/linux/wait.h
index 2db83349865b..db78c7204947 100644
--- a/include/linux/wait.h
+++ b/include/linux/wait.h
@@ -358,6 +358,19 @@ do {									\
         __ret;                                                          \
 })
 
+#define __wait_event_exclusive_cmd(wq, condition, cmd1, cmd2)           \
+        (void)___wait_event(wq, condition, TASK_UNINTERRUPTIBLE, 1, 0,  \
+                            cmd1; schedule(); cmd2)
+/*
+ * Just like wait_event_cmd(), except it sets exclusive flag
+ */
+#define wait_event_exclusive_cmd(wq, condition, cmd1, cmd2)             \
+do {                                                                    \
+        if (condition)                                                  \
+                break;                                                  \
+        __wait_event_exclusive_cmd(wq, condition, cmd1, cmd2);          \
+} while (0)
+
 #define __wait_event_cmd(wq, condition, cmd1, cmd2)                     \
         (void)___wait_event(wq, condition, TASK_UNINTERRUPTIBLE, 0, 0,  \
                             cmd1; schedule(); cmd2)