From: Alexei Starovoitov tcp_bpf_send_verdict() keeps the verdict of the sk_msg prog in psock->eval and runs the prog only when eval is __SK_NONE. For __SK_PASS it resets eval only after tcp_bpf_push() returns 0. But the push can release the socket lock in sk_stream_wait_memory(), and it can fail. Then another tcp_bpf_sendmsg() sends its message and does not run the prog. With bpf_msg_apply_bytes() the verdict covers apply_bytes bytes. After a failed push, apply_bytes still includes the sent bytes. Thus the verdict covers more bytes than the prog requested. Reset eval before the push when apply_bytes is 0. If this push fails, the prog now runs for the rest of the message. The same already occurs when a message does not fit into one sk_msg. When the push fails, subtract the sent bytes from apply_bytes. When apply_bytes is not 0, eval stays __SK_PASS during the push, and the prog does not run for another message, as today. The verdict must still cover the bytes that a failed push does not send. Fixes: 4f738adba30a ("bpf: create tcp_bpf_ulp allowing BPF to monitor socket TX/RX data") Signed-off-by: Alexei Starovoitov --- net/ipv4/tcp_bpf.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c index d90bcc911b36..072ffd8347a6 100644 --- a/net/ipv4/tcp_bpf.c +++ b/net/ipv4/tcp_bpf.c @@ -457,8 +457,14 @@ static int tcp_bpf_send_verdict(struct sock *sk, struct sk_psock *psock, switch (psock->eval) { case __SK_PASS: + /* The push may sleep without the socket lock, and it may fail */ + if (!psock->apply_bytes) + psock->eval = __SK_NONE; + origsize = msg->sg.size; ret = tcp_bpf_push(sk, msg, tosend, flags, true); if (unlikely(ret)) { + /* A part of it may have gone out */ + sk_msg_apply_bytes(psock, origsize - msg->sg.size); *copied -= sk_msg_free(sk, msg); break; } -- 2.55.0