net: gro: selective flush of packets

Current GRO can hold packets in gro_list for almost unlimited
time, in case napi->poll() handler consumes its budget over and over.

In this case, napi_complete()/napi_gro_flush() are not called.

Another problem is that gro_list is flushed in non friendly way :
We scan the list and complete packets in the reverse order.
(youngest packets first, oldest packets last)
This defeats priorities that sender could have cooked.

Since GRO currently only store TCP packets, we dont really notice the
bug because of retransmits, but this behavior can add unexpected
latencies, particularly on mice flows clamped by elephant flows.

This patch makes sure no packet can stay more than 1 ms in queue, and
only in stress situations.

It also complete packets in the right order to minimize latencies.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Jesse Gross <jesse@nicira.com>
Cc: Tom Herbert <therbert@google.com>
Cc: Yuchung Cheng <ycheng@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

1 files changed, 9 insertions, 6 deletions

diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index a659fd0ba965..0a36fff75bd5 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -1497,19 +1497,22 @@ struct napi_gro_cb {
         /* This indicates where we are processing relative to skb->data. */
         int data_offset;
 
-        /* This is non-zero if the packet may be of the same flow. */
-        int same_flow;
-
         /* This is non-zero if the packet cannot be merged with the new skb. */
         int flush;
 
         /* Number of segments aggregated. */
-        int count;
+        u16     count;
+
+        /* This is non-zero if the packet may be of the same flow. */
+        u8      same_flow;
 
         /* Free the skb? */
-        int free;
+        u8      free;
 #define NAPI_GRO_FREE             1
 #define NAPI_GRO_FREE_STOLEN_HEAD 2
+
+        /* jiffies when first packet was created/queued */
+        unsigned long age;
 };
 
 #define NAPI_GRO_CB(skb) ((struct napi_gro_cb *)(skb)->cb)
@@ -2156,7 +2159,7 @@ extern gro_result_t	dev_gro_receive(struct napi_struct *napi,
 extern gro_result_t     napi_skb_finish(gro_result_t ret, struct sk_buff *skb);
 extern gro_result_t     napi_gro_receive(struct napi_struct *napi,
                                          struct sk_buff *skb);
-extern void             napi_gro_flush(struct napi_struct *napi);
+extern void             napi_gro_flush(struct napi_struct *napi, bool flush_old);
 extern struct sk_buff * napi_get_frags(struct napi_struct *napi);
 extern gro_result_t     napi_frags_finish(struct napi_struct *napi,
                                           struct sk_buff *skb,