ip6_route_input() attaches the input route to the skb without taking a reference, relying on the RCU read-side section of the receive path. seg6_lookup_any_nexthop() is called from that same section, either directly from the seg6local input handler or from a netfilter okfn, but still takes and drops a route reference for every packet. Look the route up with RT6_LOOKUP_F_DST_NOREF, as ip6_route_input() does, for the input and table lookups. Throughput at 0.5% packet loss of 94-byte End packets (two-segment SRH, no payload), spread by RSS on an ixgbe 82599ES across two 2.30 GHz Xeon E5-2650 v3 with 10 cores each, offered by TRex and binary-searched over 10 runs of 10 s, in Mpps: queues cores before after 1 1 0.735 0.770 (+4.8%) 10 10 on the NIC's socket 6.468 6.976 (+7.9%) 16 10 + 6 on two sockets 6.931 10.128 (+46.1%) With the default rules, fib6_rule_lookup() tries the local table first, and the miss takes and drops a reference on ip6_null_entry, one route shared by every CPU. At 16 queues that cache line bounces between the sockets, and ip6_hold_safe() and dst_release() together take more than a third of the cycles. Assisted-by: LLM Signed-off-by: Yuya Kusakabe --- net/ipv6/seg6_local.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/net/ipv6/seg6_local.c b/net/ipv6/seg6_local.c index 0c3d1be7f129..4bde3669d0dd 100644 --- a/net/ipv6/seg6_local.c +++ b/net/ipv6/seg6_local.c @@ -342,6 +342,7 @@ seg6_lookup_any_nexthop(struct sk_buff *skb, struct in6_addr *nhaddr, fl6.flowi6_flags = FLOWI_FLAG_KNOWN_NH; if (!tbl_id && !oif) { + flags |= RT6_LOOKUP_F_DST_NOREF; dst = ip6_route_input_lookup(net, skb->dev, &fl6, skb, flags); } else if (tbl_id) { struct fib6_table *table; @@ -350,6 +351,7 @@ seg6_lookup_any_nexthop(struct sk_buff *skb, struct in6_addr *nhaddr, if (!table) goto out; + flags |= RT6_LOOKUP_F_DST_NOREF; rt = ip6_pol_route(net, table, oif, &fl6, skb, flags); dst = &rt->dst; } else { @@ -363,7 +365,7 @@ seg6_lookup_any_nexthop(struct sk_buff *skb, struct in6_addr *nhaddr, dev_flags |= IFF_LOOPBACK; if (dst && (dst_dev(dst)->flags & dev_flags) && !dst->error) { - dst_release(dst); + ip6_rt_put_flags(dst_rt6_info(dst), flags); dst = NULL; } @@ -372,10 +374,14 @@ seg6_lookup_any_nexthop(struct sk_buff *skb, struct in6_addr *nhaddr, rt = net->ipv6.ip6_blk_hole_entry; dst = &rt->dst; dst_hold(dst); + flags &= ~RT6_LOOKUP_F_DST_NOREF; } skb_dst_drop(skb); - skb_dst_set(skb, dst); + if ((flags & RT6_LOOKUP_F_DST_NOREF) && !dst->rt_uncached_list) + skb_dst_set_noref(skb, dst); + else + skb_dst_set(skb, dst); return dst->error; } -- 2.50.1 ip_route_input() looks the input route up with ip_route_input_noref() and then forces a reference on it, which is dropped again when the packet is transmitted. End.DX4 and End.DT4 call it from the RCU read-side section of the receive path, either directly from the seg6local input handler or from a netfilter okfn, and hand the skb to dst_input() before leaving that section. Use ip_route_input_noref(), as ip_rcv_finish() does. Throughput at 0.5% packet loss of 82-byte IPv4-in-IPv6 UDP packets (no SRH), spread by RSS on an ixgbe 82599ES across two 10-core Xeon E5-2650 v3, offered by TRex over 10 runs of 10 s, in Mpps: End.DX4 queues cores before after 1 1 0.816 0.837 (+2.6%) 10 10 on the NIC's socket 6.534 7.621 (+16.6%) 16 10 + 6 on two sockets 3.369 10.208 (+203.0%) End.DT4 queues cores before after 1 1 0.497 0.501 (+0.8%) 10 10 on the NIC's socket 4.410 4.679 (+6.1%) 16 10 + 6 on two sockets 3.259 7.431 (+128.0%) Every CPU takes and drops a reference on the same input route, the one cached on the nexthop. At 16 queues that cache line bounces between the sockets, and skb_dst_force() and dst_release() together take about 60% of the cycles for End.DX4 and over 40% for End.DT4. Assisted-by: LLM Signed-off-by: Yuya Kusakabe --- net/ipv6/seg6_local.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/net/ipv6/seg6_local.c b/net/ipv6/seg6_local.c index 4bde3669d0dd..1dfe20b365b7 100644 --- a/net/ipv6/seg6_local.c +++ b/net/ipv6/seg6_local.c @@ -1079,7 +1079,7 @@ static int input_action_end_dx4_finish(struct net *net, struct sock *sk, skb_dst_drop(skb); - reason = ip_route_input(skb, nhaddr, iph->saddr, 0, skb->dev); + reason = ip_route_input_noref(skb, nhaddr, iph->saddr, 0, skb->dev); if (reason) { kfree_skb_reason(skb, reason); return -EINVAL; @@ -1305,7 +1305,7 @@ static int input_action_end_dt4(struct sk_buff *skb, iph = ip_hdr(skb); - reason = ip_route_input(skb, iph->daddr, iph->saddr, 0, skb->dev); + reason = ip_route_input_noref(skb, iph->daddr, iph->saddr, 0, skb->dev); if (unlikely(reason)) goto drop; -- 2.50.1