tcp: refine pacing rate determination

When TCP pacing was added back in linux-3.12, we chose
to apply a fixed ratio of 200 % against current rate,
to allow probing for optimal throughput even during
slow start phase, where cwnd can be doubled every other gRTT.

At Google, we found it was better applying a different ratio
while in Congestion Avoidance phase.
This ratio was set to 120 %.

We've used the normal tcp_in_slow_start() helper for a while,
then tuned the condition to select the conservative ratio
as soon as cwnd >= ssthresh/2 :

- After cwnd reduction, it is safer to ramp up more slowly,
  as we approach optimal cwnd.
- Initial ramp up (ssthresh == INFINITY) still allows doubling
  cwnd every other RTT.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Neal Cardwell <ncardwell@google.com>
Cc: Yuchung Cheng <ycheng@google.com>
Acked-by: Neal Cardwell <ncardwell@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

1 files changed, 17 insertions, 1 deletions

diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 0abca2841de2..dc08e2352665 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -753,13 +753,29 @@ static void tcp_rtt_estimator(struct sock *sk, long mrtt_us)
  * TCP pacing, to smooth the burst on large writes when packets
  * in flight is significantly lower than cwnd (or rwin)
  */
+int sysctl_tcp_pacing_ss_ratio __read_mostly = 200;
+int sysctl_tcp_pacing_ca_ratio __read_mostly = 120;
+
 static void tcp_update_pacing_rate(struct sock *sk)
 {
         const struct tcp_sock *tp = tcp_sk(sk);
         u64 rate;
 
         /* set sk_pacing_rate to 200 % of current rate (mss * cwnd / srtt) */
-        rate = (u64)tp->mss_cache * 2 * (USEC_PER_SEC << 3);
+        rate = (u64)tp->mss_cache * ((USEC_PER_SEC / 100) << 3);
+
+        /* current rate is (cwnd * mss) / srtt
+         * In Slow Start [1], set sk_pacing_rate to 200 % the current rate.
+         * In Congestion Avoidance phase, set it to 120 % the current rate.
+         *
+         * [1] : Normal Slow Start condition is (tp->snd_cwnd < tp->snd_ssthresh)
+         *       If snd_cwnd >= (tp->snd_ssthresh / 2), we are approaching
+         *       end of slow start and should slow down.
+         */
+        if (tp->snd_cwnd < tp->snd_ssthresh / 2)
+                rate *= sysctl_tcp_pacing_ss_ratio;
+        else
+                rate *= sysctl_tcp_pacing_ca_ratio;
 
         rate *= max(tp->snd_cwnd, tp->packets_out);