xfrm_replay_seqhi() guesses seq_hi for a given ingress seq per RFC 4303 section A2.2. Packets with seq < bottom are interpreted as being ahead and the function returns the next seq_hi. xfrm_replay_check_esn() redoes the same computation and defers to the ICV check. On SW crypto, such packets then fail the ICV check and are dropped, unless they're actual packets ~2^32 in the future (very unlikely). But on crypto offload, the ICV check already happened in HW and isn't repeated. xfrm_replay_advance_esn() then increments replay_esn->seq_hi. The problem becomes visible when the HW state is renewed (e.g. bond failover or mlx5 mid-sequence ESN state update). The newly offloaded SA carries the wrong seq_hi and completely breaks ingress since all future packets fail ICV. This was verified on b2b ConnectX NICs, with an 'iperf3 -P 2' test over a pair of SAs over a bond device in active-backup mode. After ~10s of traffic at ~10 Gbps, replay_esn->seq_hi was ~2.5M due to the two streams sharing the ESN state. Forcing a bond failover on both sides breaks traffic permanently. Fix this by dropping packets up to 2^31 before the replay window for crypto offload, since the SW seq_hi guess cannot be validated. Packets up to 2^31 ahead are accepted as new, same as old code. A burst loss of more than 2^31 packets now stalls the SA (previously up to 2^32-wsize). SW crypto is unaffected. Fixes: 50bd870a9e5c ("xfrm: Add ESN support for IPSec HW offload") Signed-off-by: Cosmin Ratiu --- net/xfrm/xfrm_replay.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/net/xfrm/xfrm_replay.c b/net/xfrm/xfrm_replay.c index 9394953247ab..37c0c299cb10 100644 --- a/net/xfrm/xfrm_replay.c +++ b/net/xfrm/xfrm_replay.c @@ -451,6 +451,7 @@ static int xfrm_replay_overflow_esn(struct xfrm_state *x, struct sk_buff *skb) static int xfrm_replay_check_esn(struct xfrm_state *x, struct sk_buff *skb, __be32 net_seq) { + struct xfrm_offload *xo = xfrm_offload(skb); unsigned int bitnr, nr; u32 diff; struct xfrm_replay_state_esn *replay_esn = x->replay_esn; @@ -469,7 +470,18 @@ static int xfrm_replay_check_esn(struct xfrm_state *x, diff = top - seq; - if (likely(top >= wsize - 1)) { + if (xo && (xo->flags & CRYPTO_DONE)) { + /* The ICV was already verified by HW with its own seq_hi, + * so the seq_hi guessed by xfrm_replay_seqhi() can't + * be validated in SW. + * + * Treat seq up to 2^31 ahead of top as new, everything else as + * behind: in-window ones go through the bitmap, the rest get + * dropped. + */ + if (likely(diff >= BIT(31))) + return 0; + } else if (likely(top >= wsize - 1)) { /* A. same subspace */ if (likely(seq > top) || seq < bottom) return 0; -- 2.53.0