Merge branch 'timers/nohz-irq-work-v7' of git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks into timers/nohz

Pull nohz updates from Frederic Weisbecker:

 " This set moves the nohz kick, used to notify a full dynticks CPU when
   events require tick rescheduling, out of the scheduler tick to a
   dedicated IPI.

   This debloats a bit the scheduler IPI from off-topic work that was
   abusing that scheduler fast path for its convenient asynchronous
   properties. Now the nohz kick uses irq-work for its own needs.

   Of course this implied quite some background infrastructure rework,
   including:

     * Clean up some irq-work internals
     * Implement remote irq-work
     * Implement nohz kick on top of remote irq-work
     * Move full dynticks timer enqueue notification to new kick
     * Move multi-task notification to new kick
     * Remove unecessary barriers on multi-task notification
 "

Signed-off-by: Ingo Molnar <mingo@kernel.org>

1 files changed, 9 insertions, 3 deletions

diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 31cc02ebc54e..eb8567610295 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1221,9 +1221,15 @@ static inline void add_nr_running(struct rq *rq, unsigned count)
 #ifdef CONFIG_NO_HZ_FULL
         if (prev_nr < 2 && rq->nr_running >= 2) {
                 if (tick_nohz_full_cpu(rq->cpu)) {
-                        /* Order rq->nr_running write against the IPI */
-                        smp_wmb();
-                        smp_send_reschedule(rq->cpu);
+                        /*
+                         * Tick is needed if more than one task runs on a CPU.
+                         * Send the target an IPI to kick it out of nohz mode.
+                         *
+                         * We assume that IPI implies full memory barrier and the
+                         * new value of rq->nr_running is visible on reception
+                         * from the target.
+                         */
+                        tick_nohz_full_kick_cpu(rq->cpu);
                 }
        }
 #endif