From: Jason Xing For xsk, it's not needed to validate and check the skb in validate_xmit_skb_list() in copy mode because xsk_build_skb() doesn't and doesn't need to prepare those requisites to validate. Xsk is just responsible for delivering raw data from userspace to the driver. Skipping numerous checks somehow contributes to the transmission especially in the extremely hot path. Performance-wise, I used './xdpsock -i enp2s0f0np0 -t -S -s 64' to verify the guess and then measured on the machine with ixgbe driver. It stably goes up by 5.48%, which can be seen in the shown below: Before: sock0@enp2s0f0np0:0 txonly xdp-skb pps pkts 1.00 rx 0 0 tx 1,187,410 3,513,536 After: sock0@enp2s0f0np0:0 txonly xdp-skb pps pkts 1.00 rx 0 0 tx 1,252,590 2,459,456 This patch also removes total ~4% consumption which can be observed by perf: |--2.97%--validate_xmit_skb | | | --1.76%--netif_skb_features | | | --0.65%--skb_network_protocol | |--1.06%--validate_xmit_xfrm Signed-off-by: Jason Xing --- include/linux/netdevice.h | 4 ++-- net/core/dev.c | 10 ++++++---- net/xdp/xsk.c | 2 +- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index a80d21a14612..2df44c22406c 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3351,7 +3351,7 @@ u16 dev_pick_tx_zero(struct net_device *dev, struct sk_buff *skb, struct net_device *sb_dev); int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev); -int __dev_direct_xmit(struct sk_buff *skb, u16 queue_id); +int __dev_direct_xmit(struct sk_buff *skb, u16 queue_id, bool validate); static inline int dev_queue_xmit(struct sk_buff *skb) { @@ -3368,7 +3368,7 @@ static inline int dev_direct_xmit(struct sk_buff *skb, u16 queue_id) { int ret; - ret = __dev_direct_xmit(skb, queue_id); + ret = __dev_direct_xmit(skb, queue_id, true); if (!dev_xmit_complete(ret)) kfree_skb(skb); return ret; diff --git a/net/core/dev.c b/net/core/dev.c index e365b099484e..9fa805c26601 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -4741,7 +4741,7 @@ int __dev_queue_xmit(struct sk_buff *skb, struct net_device *sb_dev) } EXPORT_SYMBOL(__dev_queue_xmit); -int __dev_direct_xmit(struct sk_buff *skb, u16 queue_id) +int __dev_direct_xmit(struct sk_buff *skb, u16 queue_id, bool validate) { struct net_device *dev = skb->dev; struct sk_buff *orig_skb = skb; @@ -4753,9 +4753,11 @@ int __dev_direct_xmit(struct sk_buff *skb, u16 queue_id) !netif_carrier_ok(dev))) goto drop; - skb = validate_xmit_skb_list(skb, dev, &again); - if (skb != orig_skb) - goto drop; + if (validate) { + skb = validate_xmit_skb_list(skb, dev, &again); + if (skb != orig_skb) + goto drop; + } skb_set_queue_mapping(skb, queue_id); txq = skb_get_tx_queue(dev, skb); diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index 9c3acecc14b1..55278ad0a558 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -834,7 +834,7 @@ static int __xsk_generic_xmit(struct sock *sk) continue; } - err = __dev_direct_xmit(skb, xs->queue_id); + err = __dev_direct_xmit(skb, xs->queue_id, false); if (err == NETDEV_TX_BUSY) { /* Tell user-space to retry the send */ xskq_cons_cancel_n(xs->tx, xsk_get_num_desc(skb)); -- 2.41.3