[PATCH] Optimize calc_load()

calc_load() is called by timer interrupt to update avenrun[].  It currently
calls nr_active() at each timer tick (HZ per second), while the update of
avenrun[] is done only once every 5 seconds.  (LOAD_FREQ=5 Hz)

nr_active() is quite expensive on SMP machines, since it has to sum up
nr_running and nr_uninterruptible of all online CPUS, bringing foreign
dirty cache lines.

This patch is an optimization of calc_load() so that nr_active() is called
only if we need it.

The use of unlikely() is welcome since the condition is true only once every
5*HZ time.

Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>
Cc: Ingo Molnar <mingo@elte.hu>
Acked-by: "Siddha, Suresh B" <suresh.b.siddha@intel.com>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

1 files changed, 9 insertions, 5 deletions

diff --git a/kernel/timer.c b/kernel/timer.c
index 0256ab443d8a..feddf817baa5 100644
--- a/kernel/timer.c
+++ b/kernel/timer.c
@@ -1146,11 +1146,15 @@ static inline void calc_load(unsigned long ticks)
         unsigned long active_tasks; /* fixed-point */
         static int count = LOAD_FREQ;
 
-        active_tasks = count_active_tasks();
-        for (count -= ticks; count < 0; count += LOAD_FREQ) {
-                CALC_LOAD(avenrun[0], EXP_1, active_tasks);
-                CALC_LOAD(avenrun[1], EXP_5, active_tasks);
-                CALC_LOAD(avenrun[2], EXP_15, active_tasks);
+        count -= ticks;
+        if (unlikely(count < 0)) {
+                active_tasks = count_active_tasks();
+                do {
+                        CALC_LOAD(avenrun[0], EXP_1, active_tasks);
+                        CALC_LOAD(avenrun[1], EXP_5, active_tasks);
+                        CALC_LOAD(avenrun[2], EXP_15, active_tasks);
+                        count += LOAD_FREQ;
+                } while (count < 0);
         }
 }