tcp/dccp: fix hashdance race for passive sessions

Multiple cpus can process duplicates of incoming ACK messages
matching a SYN_RECV request socket. This is a rare event under
normal operations, but definitely can happen.

Only one must win the race, otherwise corruption would occur.

To fix this without adding new atomic ops, we use logic in
inet_ehash_nolisten() to detect the request was present in the same
ehash bucket where we try to insert the new child.

If request socket was not found, we have to undo the child creation.

This actually removes a spin_lock()/spin_unlock() pair in
reqsk_queue_unlink() for the fast path.

Fixes: e994b2f0fb92 ("tcp: do not lock listener to process SYN packets")
Fixes: 079096f103fa ("tcp/dccp: install syn_recv requests into ehash table")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

1 files changed, 4 insertions, 7 deletions

diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c
index 1fd5d413a664..3575dd1e5b67 100644
--- a/net/ipv4/tcp_minisocks.c
+++ b/net/ipv4/tcp_minisocks.c
@@ -580,6 +580,7 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
         const struct tcphdr *th = tcp_hdr(skb);
         __be32 flg = tcp_flag_word(th) & (TCP_FLAG_RST|TCP_FLAG_SYN|TCP_FLAG_ACK);
         bool paws_reject = false;
+        bool own_req;
 
         tmp_opt.saw_tstamp = 0;
         if (th->doff > (sizeof(struct tcphdr)>>2)) {
@@ -767,18 +768,14 @@ struct sock *tcp_check_req(struct sock *sk, struct sk_buff *skb,
          * ESTABLISHED STATE. If it will be dropped after
          * socket is created, wait for troubles.
          */
-        child = inet_csk(sk)->icsk_af_ops->syn_recv_sock(sk, skb, req, NULL);
+        child = inet_csk(sk)->icsk_af_ops->syn_recv_sock(sk, skb, req, NULL,
+                                                         req, &own_req);
         if (!child)
                 goto listen_overflow;
 
         sock_rps_save_rxhash(child, skb);
         tcp_synack_rtt_meas(child, req);
-        inet_csk_reqsk_queue_drop(sk, req);
-        inet_csk_reqsk_queue_add(sk, req, child);
-        /* Warning: caller must not call reqsk_put(req);
-         * child stole last reference on it.
-         */
-        return child;
+        return inet_csk_complete_hashdance(sk, child, req, own_req);
 
 listen_overflow:
         if (!sysctl_tcp_abort_on_overflow) {