sched: dynticks idle load balancing

Fix the process idle load balancing in the presence of dynticks.  cpus for
which ticks are stopped will sleep till the next event wakes it up.
Potentially these sleeps can be for large durations and during which today,
there is no periodic idle load balancing being done.

This patch nominates an owner among the idle cpus, which does the idle load
balancing on behalf of the other idle cpus.  And once all the cpus are
completely idle, then we can stop this idle load balancing too.  Checks added
in fast path are minimized.  Whenever there are busy cpus in the system, there
will be an owner(idle cpu) doing the system wide idle load balancing.

Open items:
1. Intelligent owner selection (like an idle core in a busy package).
2. Merge with rcu's nohz_cpu_mask?

Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Acked-by: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

1 files changed, 9 insertions, 0 deletions

diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index f4fc867f467..3483e6cb954 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -217,6 +217,14 @@ void tick_nohz_stop_sched_tick(void)
                  * the scheduler tick in nohz_restart_sched_tick.
                  */
                 if (!ts->tick_stopped) {
+                        if (select_nohz_load_balancer(1)) {
+                                /*
+                                 * sched tick not stopped!
+                                 */
+                                cpu_clear(cpu, nohz_cpu_mask);
+                                goto out;
+                        }
+
                         ts->idle_tick = ts->sched_timer.expires;
                         ts->tick_stopped = 1;
                         ts->idle_jiffies = last_jiffies;
@@ -285,6 +293,7 @@ void tick_nohz_restart_sched_tick(void)
         now = ktime_get();
 
         local_irq_disable();
+        select_nohz_load_balancer(0);
         tick_do_update_jiffies64(now);
         cpu_clear(cpu, nohz_cpu_mask);