Some tests, like tc_tunnel or tc_edt, sporadically fail in CI with the following logs: (network_helpers.c:309: errno: Operation now in progress) \ Failed to connect to server send_and_test_data:FAIL:connect to server unexpected error: -115 This is due to SO_RCVTIMEO and SO_SNDTIMEO being set on the client socket (see settimeo() in client_socket()), allowing connect() to return an error and to set errno to EINPROGRESS instead of ETIMEDOUT. Increasing the timeout value for those tests is likely not a good solution (and it has already been done by commit 2790db208b44 ("selftests/bpf: Improve tc_tunnel test reliability")): some tests expect some data transfer to fail, and so the timeout value would increase overall test execution duration again (not only the connection, but any socket operation). Another solution is to allocate a timeout budget specific to the connection: we can apply a larger timeout only for connections, and once the connection is established, set back the timeout configured through opts->timeout_ms; this would allow connection to succeed under heavy CI load, while keeping timeout reasonable for the rest of the test traffic. Set a larger SO_SNDTIMEO/SO_RCVTIMEO for the connection step, and reset it back to the timeout configured by the test once the connection has succeeded. Fixes: 99126abec5e5 ("bpf: selftests: A few improvements to network_helpers.c") Signed-off-by: Alexis Lothoré (eBPF Foundation) --- Hello, this is the v4 of the series aiming to reduce the flakyness of tc_tunnel/tc_edt tests in CI. This revision takes a step back, based on Ihor's tests, and drops the poll loop in favor of a bare, larger timeout value applied only for the connection step. The main downside of this new mechanism is a slight increase of the duration for tests expecting a connection failure. In my testing setup (x86-based Qemu on my work laptop), I observed a ~10s increase (on a ~5m18 base for the whole test_progs set). --- Changes in v4: - dropped the polling loop in favor of a larger, connect-specific timeout - drop timeout configuration from tc_edt test - Link to v3: https://patch.msgid.link/20260811-tc_tunnel_flaky-v3-0-876f4e0bc603@bootlin.com Changes in v3: - set errno before logging errors - respect time budget set by - respect opts->timeout_ms when polling: only poll for the remaining time not already consume by connect() - keep polling if poll returns with EINTR - reorder early returns and add intermediate variables to clarify code flow - Link to v2: https://patch.msgid.link/20260803-tc_tunnel_flaky-v2-1-657b287dfa75@bootlin.com Changes in v2: - drop unneeded initialization - add back error message for immediate connection failure, and slightly reword the async connection failure error message - Link to v1: https://patch.msgid.link/20260710-tc_tunnel_flaky-v1-1-42aab5399a49@bootlin.com To: Alexei Starovoitov To: Daniel Borkmann To: Andrii Nakryiko To: Eduard Zingerman To: Kumar Kartikeya Dwivedi To: Martin KaFai Lau To: Song Liu To: Yonghong Song To: Jiri Olsa To: Emil Tsalapatis To: Ihor Solodrai To: Shuah Khan Cc: ebpf@linuxfoundation.org Cc: Bastien Curutchet Cc: Thomas Petazzoni Cc: bpf@vger.kernel.org Cc: linux-kselftest@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- tools/testing/selftests/bpf/network_helpers.c | 34 ++++++++++++++++++++++++--- 1 file changed, 31 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/bpf/network_helpers.c b/tools/testing/selftests/bpf/network_helpers.c index db935a9d9fc1..89336201572f 100644 --- a/tools/testing/selftests/bpf/network_helpers.c +++ b/tools/testing/selftests/bpf/network_helpers.c @@ -49,6 +49,8 @@ errno = __save; \ }) +#define CONNECT_MIN_TIMEOUT_MS 5000 + struct ipv4_packet pkt_v4 = { .eth.h_proto = __bpf_constant_htons(ETH_P_IP), .iph.ihl = 5, @@ -291,6 +293,12 @@ int client_socket(int family, int type, return -1; } +static int connect_timeout_ms(const struct network_helper_opts *opts) +{ + /* Enforce a minimum connect timeout value */ + return MAX(opts->timeout_ms, CONNECT_MIN_TIMEOUT_MS); +} + int connect_to_addr(int type, const struct sockaddr_storage *addr, socklen_t addrlen, const struct network_helper_opts *opts) { @@ -305,13 +313,33 @@ int connect_to_addr(int type, const struct sockaddr_storage *addr, socklen_t add return -1; } + /* Override timeout configuration with a larger value for the + * connection + */ + if (settimeo(fd, connect_timeout_ms(opts))) { + log_err("Failed to set connect timeout"); + goto close; + } + if (connect(fd, (const struct sockaddr *)addr, addrlen)) { - log_err("Failed to connect to server"); - save_errno_close(fd); - return -1; + log_err("Failed to connect"); + goto close; + } + + /* If the timeout configured by the test is different from the + * connect timeout, restore it + */ + if (opts->timeout_ms != CONNECT_MIN_TIMEOUT_MS && + settimeo(fd, opts->timeout_ms)) { + log_err("Failed to set timeout for connected socket"); + goto close; } return fd; + +close: + save_errno_close(fd); + return -1; } int connect_to_addr_str(int family, int type, const char *addr_str, __u16 port, --- base-commit: 673b1f0272d7dedeec78dee2b47040481e4332ba change-id: 20260710-tc_tunnel_flaky-27e9a191bd03 Best regards, -- Alexis Lothoré (eBPF Foundation)