Multi-buffer XDP stores information about frags in skb_shared_info that sits at the tailroom of a packet. The storage space is reserved via xdp_data_hard_end(): ((xdp)->data_hard_start + (xdp)->frame_sz - \ SKB_DATA_ALIGN(sizeof(struct skb_shared_info))) and then we refer to it via macro below: static inline struct skb_shared_info * xdp_get_shared_info_from_buff(const struct xdp_buff *xdp) { return (struct skb_shared_info *)xdp_data_hard_end(xdp); } Currently we do not respect this tailroom space in multi-buffer AF_XDP ZC scenario. To address this, introduce xsk_pool_get_tailroom() and use it within xsk_pool_get_rx_frame_size() which is used in ZC drivers to configure length of HW Rx buffer. xsk_pool_get_tailroom() is only reserving necessary space when pool is zc and underlying netdev supports zc multi-buffer. Rely on pool->dev state when configuring tailroom. xsk_pool_get_rx_frame_size() inside ndo_bpf is usually called when bringing up queues and before xsk's dma mappings have been configured, which makes it valid to rely on pool->dev. Fixes: 24ea50127ecf ("xsk: support mbuf on ZC RX") Signed-off-by: Maciej Fijalkowski --- include/net/xdp_sock_drv.h | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/include/net/xdp_sock_drv.h b/include/net/xdp_sock_drv.h index 6b9ebae2dc95..bef4e1b91034 100644 --- a/include/net/xdp_sock_drv.h +++ b/include/net/xdp_sock_drv.h @@ -41,6 +41,19 @@ static inline u32 xsk_pool_get_headroom(struct xsk_buff_pool *pool) return XDP_PACKET_HEADROOM + pool->headroom; } +static inline u32 xsk_pool_get_tailroom(struct xsk_buff_pool *pool) +{ + struct xdp_umem *umem = pool->umem; + + /* Reserve tailroom only for zero-copy pools that opted into + * multi-buffer. The reserved area is used for skb_shared_info, + * matching the XDP core's xdp_data_hard_end() layout. + */ + if (pool->dev && (umem->flags & XDP_UMEM_SG_FLAG)) + return SKB_DATA_ALIGN(sizeof(struct skb_shared_info)); + return 0; +} + static inline u32 xsk_pool_get_chunk_size(struct xsk_buff_pool *pool) { return pool->chunk_size; @@ -48,7 +61,8 @@ static inline u32 xsk_pool_get_chunk_size(struct xsk_buff_pool *pool) static inline u32 xsk_pool_get_rx_frame_size(struct xsk_buff_pool *pool) { - return xsk_pool_get_chunk_size(pool) - xsk_pool_get_headroom(pool); + return xsk_pool_get_chunk_size(pool) - xsk_pool_get_headroom(pool) - + xsk_pool_get_tailroom(pool); } static inline u32 xsk_pool_get_rx_frag_step(struct xsk_buff_pool *pool) -- 2.43.0