sched: adjust when cpu_active and cpuset configurations are updated during cpu on/offlining

Currently, when a cpu goes down, cpu_active is cleared before
CPU_DOWN_PREPARE starts and cpuset configuration is updated from a
default priority cpu notifier.  When a cpu is coming up, it's set
before CPU_ONLINE but cpuset configuration again is updated from the
same cpu notifier.

For cpu notifiers, this presents an inconsistent state.  Threads which
a CPU_DOWN_PREPARE notifier expects to be bound to the CPU can be
migrated to other cpus because the cpu is no more inactive.

Fix it by updating cpu_active in the highest priority cpu notifier and
cpuset configuration in the second highest when a cpu is coming up.
Down path is updated similarly.  This guarantees that all other cpu
notifiers see consistent cpu_active and cpuset configuration.

cpuset_track_online_cpus() notifier is converted to
cpuset_update_active_cpus() which just updates the configuration and
now called from cpuset_cpu_[in]active() notifiers registered from
sched_init_smp().  If cpuset is disabled, cpuset_update_active_cpus()
degenerates into partition_sched_domains() making separate notifier
for !CONFIG_CPUSETS unnecessary.

This problem is triggered by cmwq.  During CPU_DOWN_PREPARE, hotplug
callback creates a kthread and kthread_bind()s it to the target cpu,
and the thread is expected to run on that cpu.

* Ingo's test discovered __cpuinit/exit markups were incorrect.
  Fixed.

Signed-off-by: Tejun Heo <tj@kernel.org>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Paul Menage <menage@google.com>

1 files changed, 16 insertions, 0 deletions

diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index 2d9073883ea9..de6b1722cdca 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -52,6 +52,22 @@ struct notifier_block;
  * CPU notifier priorities.
  */
 enum {
+        /*
+         * SCHED_ACTIVE marks a cpu which is coming up active during
+         * CPU_ONLINE and CPU_DOWN_FAILED and must be the first
+         * notifier.  CPUSET_ACTIVE adjusts cpuset according to
+         * cpu_active mask right after SCHED_ACTIVE.  During
+         * CPU_DOWN_PREPARE, SCHED_INACTIVE and CPUSET_INACTIVE are
+         * ordered in the similar way.
+         *
+         * This ordering guarantees consistent cpu_active mask and
+         * migration behavior to all cpu notifiers.
+         */
+        CPU_PRI_SCHED_ACTIVE    = INT_MAX,
+        CPU_PRI_CPUSET_ACTIVE   = INT_MAX - 1,
+        CPU_PRI_SCHED_INACTIVE  = INT_MIN + 1,
+        CPU_PRI_CPUSET_INACTIVE = INT_MIN,
+
         /* migration should happen before other stuff but after perf */
         CPU_PRI_PERF            = 20,
         CPU_PRI_MIGRATION       = 10,