When XSK pools are enabled after an XDP program has already been loaded,
XSK pools are registered on pre-existing queues before queues are
re-created with the XSK pool fully registered in DQ RDA mode.
This can lead to a race condition between the RX NAPI and the control
plane thread wherein a pre-existing queue sees the live XSK pool and
attempts to use recycled buffers not backed by XSK buffs for AF_XDP ZC
traffic. This causes the following kernel panic to occur when attempting
to DMA map a NULL XSK buffer:
BUG: kernel NULL pointer dereference, address: 0000000000000050
...
RIP: 0010:gve_rx_post_buffers_dqo+0x99/0x190 [gve]
...
Call Trace:
gve_rx_poll_dqo+0x4d9/0xf10 [gve]
gve_napi_poll_dqo+0x76/0x170 [gve]
__napi_poll+0x28/0x160
net_rx_action+0x2a0/0x350
handle_softirqs+0xd4/0x280
? sort_range+0x20/0x20
run_ksoftirqd+0x2d/0x40
smpboot_thread_fn+0xd5/0x1d0
kthread+0xd7/0x100
? kthread_complete_and_exit+0x20/0x20
ret_from_fork+0x1f/0x30
The XSK pool should only be registered with current queues if XSK
buffers are allocated on-the-fly, as is the case in QPL mode.
Fixes: c1fffc5d66a7 ("gve: implement DQO RX datapath and control path for AF_XDP zero-copy")
Cc: stable@vger.kernel.org
Reviewed-by: Jordan Rhee
Reviewed-by: Tim Hostetler
Signed-off-by: Joshua Washington
---
drivers/net/ethernet/google/gve/gve_main.c | 16 +++++++---------
1 file changed, 7 insertions(+), 9 deletions(-)
diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c
index e4d78ae52daf..453b304016b6 100644
--- a/drivers/net/ethernet/google/gve/gve_main.c
+++ b/drivers/net/ethernet/google/gve/gve_main.c
@@ -1654,20 +1654,18 @@ static int gve_xsk_pool_enable(struct net_device *dev,
if (!priv->xdp_prog || !netif_running(dev))
return 0;
- err = gve_reg_xsk_pool(priv, dev, pool, qid);
- if (err)
- goto err_xsk_pool_dma_mapped;
-
- /* Stop and start RDA queues to repost buffers. */
- if (!gve_is_qpl(priv)) {
+ if (gve_is_qpl(priv)) {
+ err = gve_reg_xsk_pool(priv, dev, pool, qid);
+ if (err)
+ goto err_xsk_pool_dma_mapped;
+ } else {
+ /* Stop and start RDA queues to repost buffers. */
err = gve_configure_rings_xdp(priv, priv->rx_cfg.num_queues);
if (err)
- goto err_xsk_pool_registered;
+ goto err_xsk_pool_dma_mapped;
}
return 0;
-err_xsk_pool_registered:
- gve_unreg_xsk_pool(priv, qid);
err_xsk_pool_dma_mapped:
clear_bit(qid, priv->xsk_pools);
xsk_pool_dma_unmap(pool,
--
2.55.0.691.gc56d675ccc-goog