We will invoke BPF SOCK_OPS prog with BPF_SOCK_OPS_RCVQ_CB to adjust sk->sk_rcvlowat when 1. TCP stack enqueues skb to sk->sk_receive_queue 2. TCP recvmsg() completes Let's provide a kfunc to set sk->sk_rcvlowat. Negative values are clamped to INT_MAX, consistent with SO_RCVLOWAT. The wakeup flag is determined based on bpf_sock_ops_kern.skb: * For the enqueue hook, skb is always non-NULL, and wakeup is set to false because * tcp_data_ready() is always called after the hooks in tcp_queue_rcv() and tcp_ofo_queue(). * when tcp_fastopen_add_skb() is called for TFO SYN, the socket is not yet accept()ed, and when called for TFO SYN+ACK, the socket is woken up by sk->sk_state_change() anyway. * For the recvmsg() hook, skb is always NULL, and wakeup is set to true because tcp_data_ready() is not called in the path. An alternative would be to support bpf_setsockopt() by adding BPF_SOCK_OPS_RCVQ_CB to is_locked_tcp_sock_ops(). However, that approach involves excessive conditionals and an unnecessary memcpy(), costs we do not want to pay for every skb in the TCP fast path. Signed-off-by: Kuniyuki Iwashima --- net/core/filter.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/net/core/filter.c b/net/core/filter.c index fa8a7c7d86eb..3608036632a8 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -12388,6 +12388,22 @@ __bpf_kfunc int bpf_sk_assign_tcp_reqsk(struct __sk_buff *s, struct sock *sk, #endif } +__bpf_kfunc int bpf_sock_ops_tcp_set_rcvlowat(struct bpf_sock_ops_kern *skops, + int rcvlowat) +{ +#ifdef CONFIG_INET + if (skops->op != BPF_SOCK_OPS_RCVQ_CB) + return -EOPNOTSUPP; + + if (rcvlowat < 0) + rcvlowat = INT_MAX; + + return __tcp_set_rcvlowat(skops->sk, rcvlowat, !skops->skb); +#else + return -EOPNOTSUPP; +#endif +} + __bpf_kfunc int bpf_sock_ops_enable_tx_tstamp(struct bpf_sock_ops_kern *skops, u64 flags) { @@ -12539,6 +12555,7 @@ BTF_KFUNCS_END(bpf_kfunc_check_set_tcp_reqsk) BTF_KFUNCS_START(bpf_kfunc_check_set_sock_ops) BTF_ID_FLAGS(func, bpf_sock_ops_enable_tx_tstamp) +BTF_ID_FLAGS(func, bpf_sock_ops_tcp_set_rcvlowat) BTF_KFUNCS_END(bpf_kfunc_check_set_sock_ops) static const struct btf_kfunc_id_set bpf_kfunc_set_skb = { -- 2.54.0.746.g67dd491aae-goog