tcp: change tcp_skb_pcount() location

Our goal is to access no more than one cache line access per skb in
a write or receive queue when doing the various walks.

After recent TCP_SKB_CB() reorganizations, it is almost done.

Last part is tcp_skb_pcount() which currently uses
skb_shinfo(skb)->gso_segs, which is a terrible choice, because it needs
3 cache lines in current kernel (skb->head, skb->end, and
shinfo->gso_segs are all in 3 different cache lines, far from skb->cb)

This very simple patch reuses space currently taken by tcp_tw_isn
only in input path, as tcp_skb_pcount is only needed for skb stored in
write queue.

This considerably speeds up tcp_ack(), granted we avoid shinfo->tx_flags
to get SKBTX_ACK_TSTAMP, which seems possible.

This also speeds up all sack processing in general.

This speeds up tcp_sendmsg() because it no longer has to access/dirty
shinfo.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

1 files changed, 2 insertions, 2 deletions

diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 553b01f52f71..87289e51be00 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -963,7 +963,7 @@ new_segment:
                 skb->ip_summed = CHECKSUM_PARTIAL;
                 tp->write_seq += copy;
                 TCP_SKB_CB(skb)->end_seq += copy;
-                skb_shinfo(skb)->gso_segs = 0;
+                tcp_skb_pcount_set(skb, 0);
 
                 if (!copied)
                         TCP_SKB_CB(skb)->tcp_flags &= ~TCPHDR_PSH;
@@ -1261,7 +1261,7 @@ new_segment:
 
                         tp->write_seq += copy;
                         TCP_SKB_CB(skb)->end_seq += copy;
-                        skb_shinfo(skb)->gso_segs = 0;
+                        tcp_skb_pcount_set(skb, 0);
 
                         from += copy;
                         copied += copy;