sched: fix SMP migration latencies

fix SMP migration latencies: the vruntimes of different CPUs are
at incompatible offsets so they have to be fixed up when migrating
a task across CPUs.

Signed-off-by: Mike Galbraith <efault@gmx.de>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>

1 files changed, 3 insertions, 0 deletions

diff --git a/kernel/sched.c b/kernel/sched.c
index 0f0cf374c775..4ad789d268fe 100644
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -992,6 +992,9 @@ void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
         if (p->se.block_start)
                 p->se.block_start -= clock_offset;
 #endif
+        if (likely(new_rq->cfs.min_vruntime))
+                p->se.vruntime -= old_rq->cfs.min_vruntime -
+                                                new_rq->cfs.min_vruntime;
 
         __set_task_cpu(p, new_cpu);
 }