sched/fair: Propagate an effective runnable_load_avg

The load balancer uses runnable_load_avg as load indicator. For
!cgroup this is:

  runnable_load_avg = \Sum se->avg.load_avg ; where se->on_rq

That is, a direct sum of all runnable tasks on that runqueue. As
opposed to load_avg, which is a sum of all tasks on the runqueue,
which includes a blocked component.

However, in the cgroup case, this comes apart since the group entities
are always runnable, even if most of their constituent entities are
blocked.

Therefore introduce a runnable_weight which for task entities is the
same as the regular weight, but for group entities is a fraction of
the entity weight and represents the runnable part of the group
runqueue.

Then propagate this load through the PELT hierarchy to arrive at an
effective runnable load avgerage -- which we should not confuse with
the canonical runnable load average.

Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>

1 files changed, 3 insertions, 0 deletions

diff --git a/include/linux/sched.h b/include/linux/sched.h
index 26a7df4e558c..bdd6ad6fcce1 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -331,9 +331,11 @@ struct load_weight {
 struct sched_avg {
         u64                             last_update_time;
         u64                             load_sum;
+        u64                             runnable_load_sum;
         u32                             util_sum;
         u32                             period_contrib;
         unsigned long                   load_avg;
+        unsigned long                   runnable_load_avg;
         unsigned long                   util_avg;
 };
 
@@ -376,6 +378,7 @@ struct sched_statistics {
 struct sched_entity {
         /* For load-balancing: */
         struct load_weight              load;
+        unsigned long                   runnable_weight;
         struct rb_node                  run_node;
         struct list_head                group_node;
         unsigned int                    on_rq;