net_sched: move TCQ_F_THROTTLED flag

In commit 371121057607e (net: QDISC_STATE_RUNNING dont need atomic bit
ops) I moved QDISC_STATE_RUNNING flag to __state container, located in
the cache line containing qdisc lock and often dirtied fields.

I now move TCQ_F_THROTTLED bit too, so that we let first cache line read
mostly, and shared by all cpus. This should speedup HTB/CBQ for example.

Not using test_bit()/__clear_bit()/__test_and_set_bit allows to use an
"unsigned int" for __state container, reducing by 8 bytes Qdisc size.

Introduce helpers to hide implementation details.

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
CC: Patrick McHardy <kaber@trash.net>
CC: Jesper Dangaard Brouer <hawk@diku.dk>
CC: Jarek Poplawski <jarkao2@gmail.com>
CC: Jamal Hadi Salim <hadi@cyberus.ca>
CC: Stephen Hemminger <shemminger@vyatta.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

1 files changed, 3 insertions, 3 deletions

diff --git a/net/sched/sch_cbq.c b/net/sched/sch_cbq.c
index 4aaf44c95c52..25ed522b2891 100644
--- a/net/sched/sch_cbq.c
+++ b/net/sched/sch_cbq.c
@@ -351,7 +351,7 @@ cbq_mark_toplevel(struct cbq_sched_data *q, struct cbq_class *cl)
 {
         int toplevel = q->toplevel;
 
-        if (toplevel > cl->level && !(cl->q->flags & TCQ_F_THROTTLED)) {
+        if (toplevel > cl->level && !(qdisc_is_throttled(cl->q))) {
                 psched_time_t now;
                 psched_tdiff_t incr;
 
@@ -625,7 +625,7 @@ static enum hrtimer_restart cbq_undelay(struct hrtimer *timer)
                 hrtimer_start(&q->delay_timer, time, HRTIMER_MODE_ABS);
         }
 
-        sch->flags &= ~TCQ_F_THROTTLED;
+        qdisc_unthrottled(sch);
         __netif_schedule(qdisc_root(sch));
         return HRTIMER_NORESTART;
 }
@@ -974,7 +974,7 @@ cbq_dequeue(struct Qdisc *sch)
                 skb = cbq_dequeue_1(sch);
                 if (skb) {
                         sch->q.qlen--;
-                        sch->flags &= ~TCQ_F_THROTTLED;
+                        qdisc_unthrottled(sch);
                         return skb;
                 }