bpf: sockmap redirect ingress support

Add support for the BPF_F_INGRESS flag in sk_msg redirect helper.
To do this add a scatterlist ring for receiving socks to check
before calling into regular recvmsg call path. Additionally, because
the poll wakeup logic only checked the skb recv queue we need to
add a hook in TCP stack (similar to write side) so that we have
a way to wake up polling socks when a scatterlist is redirected
to that sock.

After this all that is needed is for the redirect helper to
push the scatterlist into the psock receive queue.

Signed-off-by: John Fastabend <john.fastabend@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>

1 files changed, 9 insertions, 1 deletions

diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 0c31be306572..bccc4c270087 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -485,6 +485,14 @@ static void tcp_tx_timestamp(struct sock *sk, u16 tsflags)
         }
 }
 
+static inline bool tcp_stream_is_readable(const struct tcp_sock *tp,
+                                          int target, struct sock *sk)
+{
+        return (tp->rcv_nxt - tp->copied_seq >= target) ||
+                (sk->sk_prot->stream_memory_read ?
+                sk->sk_prot->stream_memory_read(sk) : false);
+}
+
 /*
  *      Wait for a TCP event.
  *
@@ -554,7 +562,7 @@ __poll_t tcp_poll(struct file *file, struct socket *sock, poll_table *wait)
                     tp->urg_data)
                         target++;
 
-                if (tp->rcv_nxt - tp->copied_seq >= target)
+                if (tcp_stream_is_readable(tp, target, sk))
                         mask |= EPOLLIN | EPOLLRDNORM;
 
                 if (!(sk->sk_shutdown & SEND_SHUTDOWN)) {