workqueue: fix how cpu number is stored in work->data

Once a work starts execution, its data contains the cpu number it was
on instead of pointing to cwq.  This is added by commit 7a22ad75
(workqueue: carry cpu number in work data once execution starts) to
reliably determine the work was last on even if the workqueue itself
was destroyed inbetween.

Whether data points to a cwq or contains a cpu number was
distinguished by comparing the value against PAGE_OFFSET.  The
assumption was that a cpu number should be below PAGE_OFFSET while a
pointer to cwq should be above it.  However, on architectures which
use separate address spaces for user and kernel spaces, this doesn't
hold as PAGE_OFFSET is zero.

Fix it by using an explicit flag, WORK_STRUCT_CWQ, to mark what the
data field contains.  If the flag is set, it's pointing to a cwq;
otherwise, it contains a cpu number.

Reported on s390 and microblaze during linux-next testing.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Sachin Sant <sachinp@in.ibm.com>
Reported-by: Michal Simek <michal.simek@petalogix.com>
Reported-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Tested-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Tested-by: Michal Simek <monstr@monstr.eu>

1 files changed, 8 insertions, 6 deletions

diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h
index d74a529ed13e..5f76001c4e6d 100644
--- a/include/linux/workqueue.h
+++ b/include/linux/workqueue.h
@@ -25,17 +25,19 @@ typedef void (*work_func_t)(struct work_struct *work);
 
 enum {
         WORK_STRUCT_PENDING_BIT = 0,    /* work item is pending execution */
-        WORK_STRUCT_LINKED_BIT  = 1,    /* next work is linked to this one */
+        WORK_STRUCT_CWQ_BIT     = 1,    /* data points to cwq */
+        WORK_STRUCT_LINKED_BIT  = 2,    /* next work is linked to this one */
 #ifdef CONFIG_DEBUG_OBJECTS_WORK
-        WORK_STRUCT_STATIC_BIT  = 2,    /* static initializer (debugobjects) */
-        WORK_STRUCT_COLOR_SHIFT = 3,    /* color for workqueue flushing */
+        WORK_STRUCT_STATIC_BIT  = 3,    /* static initializer (debugobjects) */
+        WORK_STRUCT_COLOR_SHIFT = 4,    /* color for workqueue flushing */
 #else
-        WORK_STRUCT_COLOR_SHIFT = 2,    /* color for workqueue flushing */
+        WORK_STRUCT_COLOR_SHIFT = 3,    /* color for workqueue flushing */
 #endif
 
         WORK_STRUCT_COLOR_BITS  = 4,
 
         WORK_STRUCT_PENDING     = 1 << WORK_STRUCT_PENDING_BIT,
+        WORK_STRUCT_CWQ         = 1 << WORK_STRUCT_CWQ_BIT,
         WORK_STRUCT_LINKED      = 1 << WORK_STRUCT_LINKED_BIT,
 #ifdef CONFIG_DEBUG_OBJECTS_WORK
         WORK_STRUCT_STATIC      = 1 << WORK_STRUCT_STATIC_BIT,
@@ -56,8 +58,8 @@ enum {
         WORK_CPU_LAST           = WORK_CPU_NONE,
 
         /*
-         * Reserve 6 bits off of cwq pointer w/ debugobjects turned
-         * off.  This makes cwqs aligned to 64 bytes which isn't too
+         * Reserve 7 bits off of cwq pointer w/ debugobjects turned
+         * off.  This makes cwqs aligned to 128 bytes which isn't too
          * excessive while allowing 15 workqueue flush colors.
          */
         WORK_STRUCT_FLAG_BITS   = WORK_STRUCT_COLOR_SHIFT +