| Seq | Timestamp | Type | Name | Duration |
|---|---|---|---|---|
| 0/0 | 2026/08/13 05:12 | flow | patch-triage |
0mArgs:null Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"KMSANReasoning": "",
"NeedsKMSAN": false,
"Reasoning": "The patch modifies the Microsoft Azure Network Adapter (MANA) driver, which is a vendor-specific PCI device driver for Azure environments. This hardware is not emulated in standard QEMU or GCE, making the code paths unreachable during standard fuzzing.",
"WorthFuzzing": false
} |
| 1/1 | 2026/08/13 05:12 | action | read-patch-diff |
0mArgs:null Results: {
"PatchDiff": "commit b06486b4a952013e656524e81960b74166cb6fd5\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Thu Aug 13 05:11:58 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/drivers/net/ethernet/microsoft/mana/mana_bpf.c b/drivers/net/ethernet/microsoft/mana/mana_bpf.c\nindex 53308e139cbe9..f47755fa86600 100644\n--- a/drivers/net/ethernet/microsoft/mana/mana_bpf.c\n+++ b/drivers/net/ethernet/microsoft/mana/mana_bpf.c\n@@ -59,6 +59,13 @@ int mana_xdp_xmit(struct net_device *ndev, int n, struct xdp_frame **frames,\n \tif (unlikely(!apc-\u003eport_is_up))\n \t\treturn 0;\n \n+\t/* Pair with the smp_wmb() in mana_publish_qset(), as mana_start_xmit()\n+\t * does. This path is gated only by the flag above, so without the\n+\t * barrier it could pick q_idx from a stale real_num_tx_queues and\n+\t * index a freshly installed, smaller apc-\u003etx_qp[].\n+\t */\n+\tsmp_rmb();\n+\n \tq_idx = smp_processor_id() % ndev-\u003ereal_num_tx_queues;\n \n \tfor (i = 0; i \u003c n; i++) {\n@@ -68,7 +75,7 @@ int mana_xdp_xmit(struct net_device *ndev, int n, struct xdp_frame **frames,\n \t\tcount++;\n \t}\n \n-\ttx_stats = \u0026apc-\u003etx_qp[q_idx]-\u003etxq.stats;\n+\ttx_stats = apc-\u003etx_qp[q_idx]-\u003etxq.stats;\n \n \tu64_stats_update_begin(\u0026tx_stats-\u003esyncp);\n \ttx_stats-\u003exdp_xmit += count;\n@@ -95,7 +102,7 @@ u32 mana_run_xdp(struct net_device *ndev, struct mana_rxq *rxq,\n \n \tact = bpf_prog_run_xdp(prog, xdp);\n \n-\trx_stats = \u0026rxq-\u003estats;\n+\trx_stats = rxq-\u003estats;\n \n \tswitch (act) {\n \tcase XDP_PASS:\n@@ -168,10 +175,17 @@ void mana_chn_setxdp(struct mana_port_context *apc, struct bpf_prog *prog)\n \t\t\tbpf_prog_put(old_prog);\n }\n \n+/* Attaching or detaching XDP changes the RX buffer layout (full pages vs\n+ * fragments), so the RX queues are rebuilt. The swap helpers handle\n+ * refcounting: mana_publish_qset() attaches the program to the new queues,\n+ * mana_free_qset() drops the old set's references.\n+ */\n static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,\n \t\t\tstruct netlink_ext_ack *extack)\n {\n \tstruct mana_port_context *apc = netdev_priv(ndev);\n+\tstruct mana_port_context *scratch;\n+\tstruct mana_qset newq, oldq;\n \tstruct bpf_prog *old_prog;\n \tstruct gdma_context *gc;\n \tint err;\n@@ -191,46 +205,46 @@ static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,\n \t\treturn -EOPNOTSUPP;\n \t}\n \n-\t/* One refcnt of the prog is hold by the caller already, so\n-\t * don't increase refcnt for this one.\n-\t */\n-\tapc-\u003ebpf_prog = prog;\n-\n \tif (apc-\u003eport_is_up) {\n-\t\t/* Re-create rxq's after xdp prog was loaded or unloaded.\n-\t\t * Ex: re create rxq's to switch from full pages to smaller\n-\t\t * size page fragments when xdp prog is unloaded and\n-\t\t * vice-versa.\n-\t\t */\n-\n-\t\t/* Pre-allocate buffers to prevent failure in mana_attach */\n-\t\terr = mana_pre_alloc_rxbufs(apc, ndev-\u003emtu, apc-\u003enum_queues);\n-\t\tif (err) {\n+\t\tscratch = mana_qset_scratch_alloc(apc);\n+\t\tif (!scratch) {\n \t\t\tNL_SET_ERR_MSG_MOD(extack,\n-\t\t\t\t\t \"XDP: Insufficient memory for tx/rx re-config\");\n-\t\t\treturn err;\n+\t\t\t\t\t \"XDP: Insufficient memory for re-config\");\n+\t\t\treturn -ENOMEM;\n \t\t}\n \n-\t\terr = mana_detach(ndev, false);\n+\t\terr = mana_alloc_qset(apc, scratch,\n+\t\t\t\t apc-\u003erx_queue_size, apc-\u003etx_queue_size,\n+\t\t\t\t apc-\u003epriv_flags, apc-\u003econfigured_mtu,\n+\t\t\t\t prog, \u0026newq);\n \t\tif (err) {\n-\t\t\tnetdev_err(ndev,\n-\t\t\t\t \"mana_detach failed at xdp set: %d\\n\", err);\n \t\t\tNL_SET_ERR_MSG_MOD(extack,\n-\t\t\t\t\t \"XDP: Re-config failed at detach\");\n-\t\t\tgoto err_dealloc_rxbuffs;\n+\t\t\t\t\t \"XDP: Re-config failed at alloc\");\n+\t\t\tmana_qset_scratch_free(scratch);\n+\t\t\treturn err;\n \t\t}\n \n-\t\terr = mana_attach(ndev);\n+\t\terr = mana_publish_qset(apc, \u0026newq, \u0026oldq);\n \t\tif (err) {\n-\t\t\tnetdev_err(ndev,\n-\t\t\t\t \"mana_attach failed at xdp set: %d\\n\", err);\n \t\t\tNL_SET_ERR_MSG_MOD(extack,\n-\t\t\t\t\t \"XDP: Re-config failed at attach\");\n-\t\t\tgoto err_dealloc_rxbuffs;\n+\t\t\t\t\t \"XDP: Re-config failed at publish\");\n+\t\t\tmana_free_qset(apc, scratch, \u0026newq);\n+\t\t\t/* After the cleanup above: closing destroys the EQ pool\n+\t\t\t * those queues' CQs were attached to.\n+\t\t\t */\n+\t\t\tmana_publish_close_if_needed(apc);\n+\t\t\tmana_qset_scratch_free(scratch);\n+\t\t\treturn err;\n \t\t}\n \n-\t\tmana_chn_setxdp(apc, prog);\n-\t\tmana_pre_dealloc_rxbufs(apc);\n+\t\tmana_free_qset(apc, scratch, \u0026oldq);\n+\t\tmana_qset_scratch_free(scratch);\n+\t} else {\n+\t\t/* No queues to rebuild; mana_open() will size the RX buffers\n+\t\t * for this program. One refcnt is held by the caller already,\n+\t\t * so don't take another.\n+\t\t */\n+\t\tapc-\u003ebpf_prog = prog;\n \t}\n \n \tif (old_prog)\n@@ -243,11 +257,6 @@ static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,\n \t\tndev-\u003emax_mtu = gc-\u003eadapter_mtu - ETH_HLEN;\n \n \treturn 0;\n-\n-err_dealloc_rxbuffs:\n-\tapc-\u003ebpf_prog = old_prog;\n-\tmana_pre_dealloc_rxbufs(apc);\n-\treturn err;\n }\n \n int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf)\n@@ -265,3 +274,34 @@ int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf)\n \n \treturn ret;\n }\n+\n+/* Read the XDP program a queue set is running, without changing anything. */\n+struct bpf_prog *mana_chn_xdp_peek(struct mana_port_context *apc)\n+{\n+\tASSERT_RTNL();\n+\n+\tif (!apc-\u003erxqs || !apc-\u003erxqs[0])\n+\t\treturn NULL;\n+\n+\treturn rtnl_dereference(apc-\u003erxqs[0]-\u003ebpf_prog);\n+}\n+\n+/* Drop the per-queue references a retiring set holds on @prog.\n+ *\n+ * Kept separate from mana_chn_setxdp() so the pointers can stay in place\n+ * until the queues stop polling: clearing them up front would let packets\n+ * already sitting in a retiring RQ take the pass path and reach the stack\n+ * without the program ever seeing them.\n+ */\n+void mana_chn_xdp_release(struct bpf_prog *prog, unsigned int num_queues)\n+{\n+\tunsigned int i;\n+\n+\tASSERT_RTNL();\n+\n+\tif (!prog)\n+\t\treturn;\n+\n+\tfor (i = 0; i \u003c num_queues; i++)\n+\t\tbpf_prog_put(prog);\n+}\ndiff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c\nindex 3c96e6fc3d81d..2d8fcdedc8b66 100644\n--- a/drivers/net/ethernet/microsoft/mana/mana_en.c\n+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c\n@@ -363,13 +363,25 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev)\n \tif (unlikely(!apc-\u003eport_is_up))\n \t\tgoto tx_drop;\n \n+\t/* Pair with the smp_wmb() in mana_publish_qset(). A control dependency\n+\t * does not order loads, and a stale apc-\u003enum_queues would admit an\n+\t * index past the end of a freshly shrunk apc-\u003etx_qp[].\n+\t */\n+\tsmp_rmb();\n+\n+\t/* XDP_TX from a retiring set carries its own RX queue index, which can\n+\t * be past the end of a smaller replacement apc-\u003etx_qp[].\n+\t */\n+\tif (unlikely(txq_idx \u003e= apc-\u003enum_queues))\n+\t\tgoto tx_drop_count;\n+\n \tif (skb_cow_head(skb, MANA_HEADROOM))\n \t\tgoto tx_drop_count;\n \n \ttxq = \u0026apc-\u003etx_qp[txq_idx]-\u003etxq;\n \tgdma_sq = txq-\u003egdma_sq;\n \tcq = \u0026apc-\u003etx_qp[txq_idx]-\u003etx_cq;\n-\ttx_stats = \u0026txq-\u003estats;\n+\ttx_stats = txq-\u003estats;\n \n \tBUILD_BUG_ON(MAX_TX_WQE_SGL_ENTRIES != MANA_MAX_TX_WQE_SGL_ENTRIES);\n \tif (MAX_SKB_FRAGS + 2 \u003e MAX_TX_WQE_SGL_ENTRIES \u0026\u0026\n@@ -548,7 +560,7 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev)\n \t/* Populated the packet and bytes counters based on post GSO packet\n \t * calculations\n \t */\n-\ttx_stats = \u0026txq-\u003estats;\n+\ttx_stats = txq-\u003estats;\n \tu64_stats_update_begin(\u0026tx_stats-\u003esyncp);\n \ttx_stats-\u003epackets += num_gso_seg;\n \ttx_stats-\u003ebytes += len + ((num_gso_seg - 1) * gso_hs);\n@@ -594,9 +606,9 @@ static void mana_get_stats64(struct net_device *ndev,\n \t\t\t struct rtnl_link_stats64 *st)\n {\n \tstruct mana_port_context *apc = netdev_priv(ndev);\n-\tunsigned int num_queues = apc-\u003enum_queues;\n \tstruct mana_stats_rx *rx_stats;\n \tstruct mana_stats_tx *tx_stats;\n+\tunsigned int num_queues;\n \tunsigned int start;\n \tu64 packets, bytes;\n \tint q;\n@@ -604,6 +616,12 @@ static void mana_get_stats64(struct net_device *ndev,\n \tif (!apc-\u003eport_is_up)\n \t\treturn;\n \n+\t/* Walk every slot, not just the queues currently open: counters\n+\t * accumulated on queues that a later reconfiguration removed must\n+\t * still be reported, or the interface totals would go backwards.\n+\t */\n+\tnum_queues = apc-\u003emax_queues;\n+\n \tnetdev_stats_to_stats64(st, \u0026ndev-\u003estats);\n \n \tif (apc-\u003eac-\u003ehwc_timeout_occurred)\n@@ -612,7 +630,7 @@ static void mana_get_stats64(struct net_device *ndev,\n \tst-\u003erx_missed_errors = apc-\u003eac-\u003ehc_stats.hc_rx_discards_no_wqe;\n \n \tfor (q = 0; q \u003c num_queues; q++) {\n-\t\trx_stats = \u0026apc-\u003erxqs[q]-\u003estats;\n+\t\trx_stats = \u0026apc-\u003erxq_stats[q];\n \n \t\tdo {\n \t\t\tstart = u64_stats_fetch_begin(\u0026rx_stats-\u003esyncp);\n@@ -625,7 +643,7 @@ static void mana_get_stats64(struct net_device *ndev,\n \t}\n \n \tfor (q = 0; q \u003c num_queues; q++) {\n-\t\ttx_stats = \u0026apc-\u003etx_qp[q]-\u003etxq.stats;\n+\t\ttx_stats = \u0026apc-\u003etxq_stats[q];\n \n \t\tdo {\n \t\t\tstart = u64_stats_fetch_begin(\u0026tx_stats-\u003esyncp);\n@@ -872,35 +890,46 @@ int mana_pre_alloc_rxbufs(struct mana_port_context *mpc, int new_mtu, int num_qu\n \treturn -ENOMEM;\n }\n \n+/* ndev-\u003emtu is updated only once the new set is live (mana_publish_qset), so\n+ * a failed allocation leaves the queues and the advertised MTU untouched.\n+ */\n static int mana_change_mtu(struct net_device *ndev, int new_mtu)\n {\n \tstruct mana_port_context *mpc = netdev_priv(ndev);\n-\tunsigned int old_mtu = ndev-\u003emtu;\n+\tstruct mana_port_context *scratch;\n+\tstruct mana_qset newq, oldq;\n \tint err;\n \n-\t/* Pre-allocate buffers to prevent failure in mana_attach later */\n-\terr = mana_pre_alloc_rxbufs(mpc, new_mtu, mpc-\u003enum_queues);\n-\tif (err) {\n-\t\tnetdev_err(ndev, \"Insufficient memory for new MTU\\n\");\n-\t\treturn err;\n+\t/* Port is down: no queues to rebuild, just record the new MTU.\n+\t * mana_open() will size the RX buffers accordingly.\n+\t */\n+\tif (!mpc-\u003eport_is_up) {\n+\t\tmpc-\u003econfigured_mtu = new_mtu;\n+\t\tWRITE_ONCE(ndev-\u003emtu, new_mtu);\n+\t\treturn 0;\n \t}\n \n-\terr = mana_detach(ndev, false);\n-\tif (err) {\n-\t\tnetdev_err(ndev, \"mana_detach failed: %d\\n\", err);\n-\t\tgoto out;\n-\t}\n+\tscratch = mana_qset_scratch_alloc(mpc);\n+\tif (!scratch)\n+\t\treturn -ENOMEM;\n \n-\tWRITE_ONCE(ndev-\u003emtu, new_mtu);\n+\terr = mana_alloc_qset(mpc, scratch, mpc-\u003erx_queue_size,\n+\t\t\t mpc-\u003etx_queue_size, mpc-\u003epriv_flags, new_mtu,\n+\t\t\t mpc-\u003ebpf_prog, \u0026newq);\n+\tif (err)\n+\t\tgoto free_scratch; /* current qset and ndev-\u003emtu untouched */\n \n-\terr = mana_attach(ndev);\n+\terr = mana_publish_qset(mpc, \u0026newq, \u0026oldq);\n \tif (err) {\n-\t\tnetdev_err(ndev, \"mana_attach failed: %d\\n\", err);\n-\t\tWRITE_ONCE(ndev-\u003emtu, old_mtu);\n+\t\tmana_free_qset(mpc, scratch, \u0026newq);\n+\t\tgoto free_scratch;\n \t}\n \n-out:\n-\tmana_pre_dealloc_rxbufs(mpc);\n+\tmana_free_qset(mpc, scratch, \u0026oldq);\n+\n+free_scratch:\n+\tmana_publish_close_if_needed(mpc);\n+\tmana_qset_scratch_free(scratch);\n \treturn err;\n }\n \n@@ -1033,6 +1062,48 @@ static void mana_cleanup_port_context(struct mana_port_context *apc)\n \tapc-\u003erxqs = NULL;\n }\n \n+/* Counters belong to the port, not the queues, so a queue-set replacement\n+ * does not reset them. Sized to max_queues, allocated once.\n+ *\n+ * A swap adds no writer to a TX slot. RX slots do overlap briefly, since a\n+ * retiring rxq keeps its NAPI until mana_free_qset() destroys it. MANA is\n+ * 64-bit only, so u64_stats_sync has no seqcount and at worst a few\n+ * increments are lost; the alternatives are a lock in the receive path or\n+ * per-set slots that make ndo_get_stats64() dip during a swap.\n+ */\n+static int mana_alloc_queue_stats(struct mana_port_context *apc)\n+{\n+\tunsigned int i;\n+\n+\tapc-\u003erxq_stats = kcalloc(apc-\u003emax_queues, sizeof(*apc-\u003erxq_stats),\n+\t\t\t\t GFP_KERNEL);\n+\tif (!apc-\u003erxq_stats)\n+\t\treturn -ENOMEM;\n+\n+\tapc-\u003etxq_stats = kcalloc(apc-\u003emax_queues, sizeof(*apc-\u003etxq_stats),\n+\t\t\t\t GFP_KERNEL);\n+\tif (!apc-\u003etxq_stats) {\n+\t\tkfree(apc-\u003erxq_stats);\n+\t\tapc-\u003erxq_stats = NULL;\n+\t\treturn -ENOMEM;\n+\t}\n+\n+\tfor (i = 0; i \u003c apc-\u003emax_queues; i++) {\n+\t\tu64_stats_init(\u0026apc-\u003erxq_stats[i].syncp);\n+\t\tu64_stats_init(\u0026apc-\u003etxq_stats[i].syncp);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static void mana_free_queue_stats(struct mana_port_context *apc)\n+{\n+\tkfree(apc-\u003erxq_stats);\n+\tapc-\u003erxq_stats = NULL;\n+\tkfree(apc-\u003etxq_stats);\n+\tapc-\u003etxq_stats = NULL;\n+}\n+\n static void mana_cleanup_indir_table(struct mana_port_context *apc)\n {\n \tapc-\u003eindir_table_sz = 0;\n@@ -1042,6 +1113,11 @@ static void mana_cleanup_indir_table(struct mana_port_context *apc)\n \n static int mana_init_port_context(struct mana_port_context *apc)\n {\n+\t/* A port reconfigured while down already has an apc-\u003erxqs, and\n+\t * mana_detach() takes its \"already detached\" early return without\n+\t * releasing it. Free it rather than overwrite the pointer.\n+\t */\n+\tkfree(apc-\u003erxqs);\n \tapc-\u003erxqs = kzalloc_objs(struct mana_rxq *, apc-\u003enum_queues);\n \n \treturn !apc-\u003erxqs ? -ENOMEM : 0;\n@@ -1730,7 +1806,7 @@ void mana_destroy_eq(struct mana_port_context *apc)\n \tdebugfs_remove_recursive(apc-\u003emana_eqs_debugfs);\n \tapc-\u003emana_eqs_debugfs = NULL;\n \n-\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\tfor (i = 0; i \u003c apc-\u003enum_eqs; i++) {\n \t\teq = apc-\u003eeqs[i].eq;\n \t\tif (!eq)\n \t\t\tcontinue;\n@@ -1738,24 +1814,29 @@ void mana_destroy_eq(struct mana_port_context *apc)\n \t\tmsi = eq-\u003eeq.msix_index;\n \t\tmana_gd_destroy_queue(gc, eq);\n \t\tmana_gd_put_gic(gc, !gc-\u003emsi_sharing, msi);\n+\t\tapc-\u003eeqs[i].eq = NULL;\n+\t\t/* Freed with the parent by debugfs_remove_recursive() above. */\n+\t\tapc-\u003eeqs[i].mana_eq_debugfs = NULL;\n \t}\n \n \tkfree(apc-\u003eeqs);\n \tapc-\u003eeqs = NULL;\n+\tapc-\u003enum_eqs = 0;\n }\n EXPORT_SYMBOL_NS(mana_destroy_eq, \"NET_MANA\");\n \n static void mana_create_eq_debugfs(struct mana_port_context *apc, int i)\n {\n-\tstruct mana_eq eq = apc-\u003eeqs[i];\n+\tstruct mana_eq *eq = \u0026apc-\u003eeqs[i];\n \tchar eqnum[32];\n \n \tsprintf(eqnum, \"eq%d\", i);\n-\teq.mana_eq_debugfs = debugfs_create_dir(eqnum, apc-\u003emana_eqs_debugfs);\n-\tdebugfs_create_u32(\"head\", 0400, eq.mana_eq_debugfs, \u0026eq.eq-\u003ehead);\n-\tdebugfs_create_u32(\"tail\", 0400, eq.mana_eq_debugfs, \u0026eq.eq-\u003etail);\n-\tdebugfs_create_u32(\"irq\", 0400, eq.mana_eq_debugfs, \u0026eq.eq-\u003eeq.irq);\n-\tdebugfs_create_file(\"eq_dump\", 0400, eq.mana_eq_debugfs, eq.eq, \u0026mana_dbg_q_fops);\n+\teq-\u003emana_eq_debugfs = debugfs_create_dir(eqnum, apc-\u003emana_eqs_debugfs);\n+\tdebugfs_create_u32(\"head\", 0400, eq-\u003emana_eq_debugfs, \u0026eq-\u003eeq-\u003ehead);\n+\tdebugfs_create_u32(\"tail\", 0400, eq-\u003emana_eq_debugfs, \u0026eq-\u003eeq-\u003etail);\n+\tdebugfs_create_u32(\"irq\", 0400, eq-\u003emana_eq_debugfs, \u0026eq-\u003eeq-\u003eeq.irq);\n+\tdebugfs_create_file(\"eq_dump\", 0400, eq-\u003emana_eq_debugfs, eq-\u003eeq,\n+\t\t\t \u0026mana_dbg_q_fops);\n }\n \n int mana_create_eq(struct mana_port_context *apc)\n@@ -1770,9 +1851,14 @@ int mana_create_eq(struct mana_port_context *apc)\n \n \tif (WARN_ON(apc-\u003eeqs))\n \t\treturn -EEXIST;\n-\tapc-\u003eeqs = kzalloc_objs(struct mana_eq, apc-\u003enum_queues);\n+\t/* Size the array to the largest queue count this port can ever use,\n+\t * so growing it later never has to reallocate (the CQs of a live\n+\t * queue set hold pointers taken from these slots).\n+\t */\n+\tapc-\u003eeqs = kzalloc_objs(struct mana_eq, apc-\u003emax_queues);\n \tif (!apc-\u003eeqs)\n \t\treturn -ENOMEM;\n+\tapc-\u003enum_eqs = 0;\n \n \tspec.type = GDMA_EQ;\n \tspec.monitor_avl_buf = false;\n@@ -1802,6 +1888,7 @@ int mana_create_eq(struct mana_port_context *apc)\n \t\t}\n \t\tapc-\u003eeqs[i].eq-\u003eeq.irq = gic-\u003eirq;\n \t\tmana_create_eq_debugfs(apc, i);\n+\t\tapc-\u003enum_eqs = i + 1;\n \t}\n \n \treturn 0;\n@@ -1811,6 +1898,101 @@ int mana_create_eq(struct mana_port_context *apc)\n }\n EXPORT_SYMBOL_NS(mana_create_eq, \"NET_MANA\");\n \n+/**\n+ * mana_grow_eqs - make sure the port has at least @need EQs\n+ * @apc: port context\n+ * @need: number of EQs the new queue set requires\n+ *\n+ * EQs are bound to MSI-X vectors, so the pool is port-owned and shared across\n+ * a swap: peak usage is max(old, new), not the sum. Grow-only, up to\n+ * apc-\u003emax_queues.\n+ */\n+static int mana_grow_eqs(struct mana_port_context *apc, unsigned int need)\n+{\n+\tstruct gdma_dev *gd = apc-\u003eac-\u003egdma_dev;\n+\tstruct gdma_context *gc = gd-\u003egdma_context;\n+\tstruct gdma_queue_spec spec = {};\n+\tstruct gdma_irq_context *gic;\n+\tunsigned int i;\n+\tint err;\n+\tint msi;\n+\n+\tif (WARN_ON(!apc-\u003eeqs))\n+\t\treturn -EINVAL;\n+\n+\tif (need \u003e apc-\u003emax_queues)\n+\t\treturn -EINVAL;\n+\n+\tif (need \u003c= apc-\u003enum_eqs)\n+\t\treturn 0;\n+\n+\tspec.type = GDMA_EQ;\n+\tspec.monitor_avl_buf = false;\n+\tspec.queue_size = EQ_SIZE;\n+\tspec.eq.callback = NULL;\n+\tspec.eq.context = apc-\u003eeqs;\n+\tspec.eq.log2_throttle_limit = LOG2_EQ_THROTTLE;\n+\n+\tfor (i = apc-\u003enum_eqs; i \u003c need; i++) {\n+\t\tmsi = (i + 1) % gc-\u003enum_msix_usable;\n+\n+\t\tgic = mana_gd_get_gic(gc, !gc-\u003emsi_sharing, \u0026msi);\n+\t\tif (IS_ERR(gic)) {\n+\t\t\terr = PTR_ERR(gic);\n+\t\t\tgoto out;\n+\t\t}\n+\t\tspec.eq.msix_index = msi;\n+\n+\t\terr = mana_gd_create_mana_eq(gd, \u0026spec, \u0026apc-\u003eeqs[i].eq);\n+\t\tif (err) {\n+\t\t\tdev_err(gc-\u003edev, \"Failed to grow EQ %u : %d\\n\", i, err);\n+\t\t\tmana_gd_put_gic(gc, !gc-\u003emsi_sharing, msi);\n+\t\t\tgoto out;\n+\t\t}\n+\t\tapc-\u003eeqs[i].eq-\u003eeq.irq = gic-\u003eirq;\n+\t\tmana_create_eq_debugfs(apc, i);\n+\t\tapc-\u003enum_eqs = i + 1;\n+\t}\n+\n+\treturn 0;\n+out:\n+\t/* Keep whatever was created: the running queue set still needs its\n+\t * own EQs, and the extras are reused by the next attempt.\n+\t */\n+\treturn err;\n+}\n+\n+/* Release EQs above @keep, returning the MSI-X vectors freed. Only safe once\n+ * no set references them, i.e. after mana_free_qset(), or a live CQ would\n+ * point at a destroyed EQ.\n+ */\n+static void mana_shrink_eqs(struct mana_port_context *apc, unsigned int keep)\n+{\n+\tstruct gdma_context *gc = apc-\u003eac-\u003egdma_dev-\u003egdma_context;\n+\tstruct gdma_queue *eq;\n+\tunsigned int msi;\n+\tunsigned int i;\n+\n+\tif (!apc-\u003eeqs || keep \u003e= apc-\u003enum_eqs)\n+\t\treturn;\n+\n+\tfor (i = keep; i \u003c apc-\u003enum_eqs; i++) {\n+\t\teq = apc-\u003eeqs[i].eq;\n+\t\tif (!eq)\n+\t\t\tcontinue;\n+\n+\t\tdebugfs_remove_recursive(apc-\u003eeqs[i].mana_eq_debugfs);\n+\t\tapc-\u003eeqs[i].mana_eq_debugfs = NULL;\n+\n+\t\tmsi = eq-\u003eeq.msix_index;\n+\t\tmana_gd_destroy_queue(gc, eq);\n+\t\tmana_gd_put_gic(gc, !gc-\u003emsi_sharing, msi);\n+\t\tapc-\u003eeqs[i].eq = NULL;\n+\t}\n+\n+\tapc-\u003enum_eqs = keep;\n+}\n+\n static int mana_fence_rq(struct mana_port_context *apc, struct mana_rxq *rxq)\n {\n \tstruct mana_fence_rq_resp resp = {};\n@@ -2015,7 +2197,12 @@ static void mana_poll_tx_cq(struct mana_cq *cq)\n \t/* Ensure checking txq_stopped before apc-\u003eport_is_up. */\n \tsmp_rmb();\n \n-\tif (txq_stopped \u0026\u0026 apc-\u003eport_is_up \u0026\u0026 avail_space \u003e= MAX_TX_WQE_SIZE) {\n+\t/* Ordered by the same barrier: reaching here with txq_stopped set means\n+\t * the replacement queue has already run, which is strictly after this\n+\t * queue was marked retiring.\n+\t */\n+\tif (txq_stopped \u0026\u0026 !READ_ONCE(txq-\u003eretiring) \u0026\u0026 apc-\u003eport_is_up \u0026\u0026\n+\t avail_space \u003e= MAX_TX_WQE_SIZE) {\n \t\tnetif_tx_wake_queue(net_txq);\n \t\tapc-\u003eeth_stats.wake_queue++;\n \t}\n@@ -2078,7 +2265,7 @@ static void mana_rx_skb(void *buf_va, bool from_pool,\n \t\t\tstruct mana_rxcomp_oob *cqe, struct mana_rxq *rxq,\n \t\t\tu32 pkt_len, u32 pkt_hash)\n {\n-\tstruct mana_stats_rx *rx_stats = \u0026rxq-\u003estats;\n+\tstruct mana_stats_rx *rx_stats = rxq-\u003estats;\n \tstruct net_device *ndev = rxq-\u003endev;\n \tu16 rxq_idx = rxq-\u003erxq_idx;\n \tstruct napi_struct *napi;\n@@ -2392,13 +2579,13 @@ static void mana_process_rx_cqe(struct mana_rxq *rxq, struct mana_cq *cq,\n \t * Coalesced CQEs have at least 2 packets, so index is pkt_i - 2.\n \t */\n \tif (pkt_i \u003e 1) {\n-\t\tu64_stats_update_begin(\u0026rxq-\u003estats.syncp);\n-\t\trxq-\u003estats.coalesced_cqe[pkt_i - 2]++;\n-\t\tu64_stats_update_end(\u0026rxq-\u003estats.syncp);\n+\t\tu64_stats_update_begin(\u0026rxq-\u003estats-\u003esyncp);\n+\t\trxq-\u003estats-\u003ecoalesced_cqe[pkt_i - 2]++;\n+\t\tu64_stats_update_end(\u0026rxq-\u003estats-\u003esyncp);\n \t} else if (!pkt_i \u0026\u0026 !pktlen) {\n-\t\tu64_stats_update_begin(\u0026rxq-\u003estats.syncp);\n-\t\trxq-\u003estats.pkt_len0_err++;\n-\t\tu64_stats_update_end(\u0026rxq-\u003estats.syncp);\n+\t\tu64_stats_update_begin(\u0026rxq-\u003estats-\u003esyncp);\n+\t\trxq-\u003estats-\u003epkt_len0_err++;\n+\t\tu64_stats_update_end(\u0026rxq-\u003estats-\u003esyncp);\n \t\tnetdev_err_once(ndev,\n \t\t\t\t\"RX pkt len=0, rq=%u, cq=%u, rxobj=0x%llx\\n\",\n \t\t\t\trxq-\u003egdma_id, cq-\u003egdma_id, rxq-\u003erxobj);\n@@ -2530,8 +2717,8 @@ static void mana_update_rx_dim(struct mana_cq *cq)\n \tif (!smp_load_acquire(\u0026apc-\u003erx_dim_enabled))\n \t\treturn;\n \n-\tdim_update_sample(READ_ONCE(cq-\u003edim_event_ctr), rxq-\u003estats.packets,\n-\t\t\t rxq-\u003estats.bytes, \u0026dim_sample);\n+\tdim_update_sample(READ_ONCE(cq-\u003edim_event_ctr), rxq-\u003estats-\u003epackets,\n+\t\t\t rxq-\u003estats-\u003ebytes, \u0026dim_sample);\n \tnet_dim(\u0026cq-\u003edim, \u0026dim_sample);\n }\n \n@@ -2638,7 +2825,11 @@ static void mana_deinit_txq(struct mana_port_context *apc, struct mana_txq *txq)\n \tmana_gd_destroy_queue(gd-\u003egdma_context, txq-\u003egdma_sq);\n }\n \n-static void mana_destroy_txq(struct mana_port_context *apc)\n+/* The array itself is left in place: the grow path tears down only a range,\n+ * and the queues below @first are still live and still referenced by it.\n+ */\n+static void mana_destroy_txq_from(struct mana_port_context *apc,\n+\t\t\t\t unsigned int first)\n {\n \tstruct napi_struct *napi;\n \tint i;\n@@ -2646,7 +2837,7 @@ static void mana_destroy_txq(struct mana_port_context *apc)\n \tif (!apc-\u003etx_qp)\n \t\treturn;\n \n-\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\tfor (i = first; i \u003c apc-\u003enum_queues; i++) {\n \t\tif (!apc-\u003etx_qp[i])\n \t\t\tcontinue;\n \n@@ -2671,6 +2862,14 @@ static void mana_destroy_txq(struct mana_port_context *apc)\n \n \t\tkvfree(apc-\u003etx_qp[i]);\n \t}\n+}\n+\n+static void mana_destroy_txq(struct mana_port_context *apc)\n+{\n+\tif (!apc-\u003etx_qp)\n+\t\treturn;\n+\n+\tmana_destroy_txq_from(apc, 0);\n \n \tkfree(apc-\u003etx_qp);\n \tapc-\u003etx_qp = NULL;\n@@ -2701,8 +2900,12 @@ static void mana_create_txq_debugfs(struct mana_port_context *apc, int idx)\n \t\t\t tx_qp-\u003etx_cq.gdma_cq, \u0026mana_dbg_q_fops);\n }\n \n+/* @first is non-zero only for the grow path, which supplies an already\n+ * allocated apc-\u003etx_qp[] holding the carried-over queues. On error only the\n+ * queues this call created are torn down.\n+ */\n static int mana_create_txq(struct mana_port_context *apc,\n-\t\t\t struct net_device *net)\n+\t\t\t struct net_device *net, unsigned int first)\n {\n \tstruct mana_context *ac = apc-\u003eac;\n \tstruct gdma_dev *gd = ac-\u003egdma_dev;\n@@ -2717,9 +2920,14 @@ static int mana_create_txq(struct mana_port_context *apc,\n \tint err;\n \tint i;\n \n-\tapc-\u003etx_qp = kzalloc_objs(struct mana_tx_qp *, apc-\u003enum_queues);\n-\tif (!apc-\u003etx_qp)\n-\t\treturn -ENOMEM;\n+\tif (first) {\n+\t\tif (WARN_ON(!apc-\u003etx_qp))\n+\t\t\treturn -EINVAL;\n+\t} else {\n+\t\tapc-\u003etx_qp = kzalloc_objs(struct mana_tx_qp *, apc-\u003enum_queues);\n+\t\tif (!apc-\u003etx_qp)\n+\t\t\treturn -ENOMEM;\n+\t}\n \n \t/* The minimum size of the WQE is 32 bytes, hence\n \t * apc-\u003etx_queue_size represents the maximum number of WQEs\n@@ -2736,7 +2944,7 @@ static int mana_create_txq(struct mana_port_context *apc,\n \n \tgc = gd-\u003egdma_context;\n \n-\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\tfor (i = first; i \u003c apc-\u003enum_queues; i++) {\n \t\tapc-\u003etx_qp[i] = kvzalloc_obj(*apc-\u003etx_qp[i]);\n \t\tif (!apc-\u003etx_qp[i]) {\n \t\t\terr = -ENOMEM;\n@@ -2748,9 +2956,10 @@ static int mana_create_txq(struct mana_port_context *apc,\n \t\t/* Create SQ */\n \t\ttxq = \u0026apc-\u003etx_qp[i]-\u003etxq;\n \n-\t\tu64_stats_init(\u0026txq-\u003estats.syncp);\n+\t\ttxq-\u003estats = \u0026apc-\u003etxq_stats[i];\n \t\ttxq-\u003endev = net;\n \t\ttxq-\u003enet_txq = netdev_get_tx_queue(net, i);\n+\t\ttxq-\u003ereset_gen = READ_ONCE(apc-\u003eac-\u003ereset_gen);\n \t\ttxq-\u003evp_offset = apc-\u003etx_vp_offset;\n \t\ttxq-\u003enapi_initialized = false;\n \t\tskb_queue_head_init(\u0026txq-\u003epending_skbs);\n@@ -2843,7 +3052,10 @@ static int mana_create_txq(struct mana_port_context *apc,\n out:\n \tnetdev_err(net, \"Failed to create %d TX queues, %d\\n\",\n \t\t apc-\u003enum_queues, err);\n-\tmana_destroy_txq(apc);\n+\tif (first)\n+\t\tmana_destroy_txq_from(apc, first);\n+\telse\n+\t\tmana_destroy_txq(apc);\n \treturn err;\n }\n \n@@ -3006,11 +3218,14 @@ static int mana_push_wqe(struct mana_rxq *rxq)\n \n static int mana_create_page_pool(struct mana_rxq *rxq, struct gdma_context *gc)\n {\n-\tstruct mana_port_context *mpc = netdev_priv(rxq-\u003endev);\n \tstruct page_pool_params pprm = {};\n \tint ret;\n \n-\tpprm.pool_size = mpc-\u003erx_queue_size / rxq-\u003efrag_count + 1;\n+\t/* Size the recycle ring from the queue being built, not from the live\n+\t * port context: during a swap the queue may be sized for a ring the\n+\t * running configuration does not use yet.\n+\t */\n+\tpprm.pool_size = rxq-\u003enum_rx_buf / rxq-\u003efrag_count + 1;\n \tpprm.nid = gc-\u003enuma_node;\n \tpprm.napi = \u0026rxq-\u003erx_cq.napi;\n \tpprm.netdev = rxq-\u003endev;\n@@ -3059,11 +3274,14 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc,\n \t\treturn ERR_PTR(-ENOMEM);\n \n \trxq-\u003endev = ndev;\n+\t/* Wire up the port-owned statistics before the queue can be polled. */\n+\trxq-\u003estats = \u0026apc-\u003erxq_stats[rxq_idx];\n \trxq-\u003enum_rx_buf = apc-\u003erx_queue_size;\n \trxq-\u003erxq_idx = rxq_idx;\n \trxq-\u003erxobj = INVALID_MANA_HANDLE;\n \n-\tmana_get_rxbuf_cfg(apc, ndev-\u003emtu, \u0026rxq-\u003edatasize, \u0026rxq-\u003ealloc_size,\n+\tmana_get_rxbuf_cfg(apc, apc-\u003econfigured_mtu, \u0026rxq-\u003edatasize,\n+\t\t\t \u0026rxq-\u003ealloc_size,\n \t\t\t \u0026rxq-\u003eheadroom, \u0026rxq-\u003efrag_count);\n \t/* Create page pool for RX queue */\n \terr = mana_create_page_pool(rxq, gc);\n@@ -3193,14 +3411,18 @@ static void mana_create_rxq_debugfs(struct mana_port_context *apc, int idx)\n \t\t\t \u0026mana_dbg_q_fops);\n }\n \n+/* @first is non-zero only for the grow path; the slots below it already hold\n+ * carried-over queues. Queues created before a failure are left in\n+ * apc-\u003erxqs[] for the caller to tear down.\n+ */\n static int mana_add_rx_queues(struct mana_port_context *apc,\n-\t\t\t struct net_device *ndev)\n+\t\t\t struct net_device *ndev, unsigned int first)\n {\n \tstruct mana_rxq *rxq;\n \tint err = 0;\n \tint i;\n \n-\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\tfor (i = first; i \u003c apc-\u003enum_queues; i++) {\n \t\trxq = mana_create_rxq(apc, i, \u0026apc-\u003eeqs[i], ndev);\n \t\tif (IS_ERR(rxq)) {\n \t\t\terr = PTR_ERR(rxq);\n@@ -3208,7 +3430,6 @@ static int mana_add_rx_queues(struct mana_port_context *apc,\n \t\t\tgoto out;\n \t\t}\n \n-\t\tu64_stats_init(\u0026rxq-\u003estats.syncp);\n \n \t\tapc-\u003erxqs[i] = rxq;\n \n@@ -3220,14 +3441,16 @@ static int mana_add_rx_queues(struct mana_port_context *apc,\n \treturn err;\n }\n \n-static void mana_destroy_rxqs(struct mana_port_context *apc)\n+/* The array is left in place; see mana_destroy_txq_from(). */\n+static void mana_destroy_rxqs_from(struct mana_port_context *apc,\n+\t\t\t\t unsigned int first)\n {\n \tstruct mana_rxq *rxq;\n \tu32 rxq_idx;\n \n \tif (apc-\u003erxqs) {\n \n-\t\tfor (rxq_idx = 0; rxq_idx \u003c apc-\u003enum_queues; rxq_idx++) {\n+\t\tfor (rxq_idx = first; rxq_idx \u003c apc-\u003enum_queues; rxq_idx++) {\n \t\t\trxq = apc-\u003erxqs[rxq_idx];\n \t\t\tif (!rxq)\n \t\t\t\tcontinue;\n@@ -3238,6 +3461,11 @@ static void mana_destroy_rxqs(struct mana_port_context *apc)\n \t}\n }\n \n+static void mana_destroy_rxqs(struct mana_port_context *apc)\n+{\n+\tmana_destroy_rxqs_from(apc, 0);\n+}\n+\n static void mana_destroy_vport(struct mana_port_context *apc)\n {\n \tstruct gdma_dev *gd = apc-\u003eac-\u003egdma_dev;\n@@ -3303,6 +3531,38 @@ static void mana_rss_table_init(struct mana_port_context *apc)\n \t\t\tethtool_rxfh_indir_default(i, apc-\u003enum_queues);\n }\n \n+/* Whether @apc's indirection table can be carried to a set of @num_queues,\n+ * rather than rebuilt from the driver default.\n+ *\n+ * Only a user table (\"ethtool -X\") is kept; a driver one is rebuilt to spread\n+ * over the new count. ethtool_check_max_channel() already refuses a reduction\n+ * that leaves a user table pointing past the last queue, so the bounds check\n+ * below only guards rebuild paths that bypass ethtool.\n+ *\n+ * @lost reports a table that cannot be carried instead of calling\n+ * ethtool_rxfh_indir_lost() here, since the swap may still fail and leave the\n+ * port on queues where the table still applies. False when none was set.\n+ */\n+static bool mana_rss_table_keep(struct mana_port_context *apc,\n+\t\t\t\tunsigned int num_queues, bool *lost)\n+{\n+\tu32 i;\n+\n+\t*lost = false;\n+\n+\tif (!netif_is_rxfh_configured(apc-\u003endev))\n+\t\treturn false;\n+\n+\tfor (i = 0; i \u003c apc-\u003eindir_table_sz; i++) {\n+\t\tif (apc-\u003eindir_table[i] \u003e= num_queues) {\n+\t\t\t*lost = true;\n+\t\t\treturn false;\n+\t\t}\n+\t}\n+\n+\treturn true;\n+}\n+\n int mana_disable_vport_rx(struct mana_port_context *apc)\n {\n \treturn mana_cfg_vport_steering(apc, TRI_STATE_FALSE, false, false,\n@@ -3573,6 +3833,7 @@ int mana_alloc_queues(struct net_device *ndev)\n {\n \tstruct mana_port_context *apc = netdev_priv(ndev);\n \tstruct gdma_dev *gd = apc-\u003eac-\u003egdma_dev;\n+\tbool indir_lost;\n \tint err;\n \n \terr = mana_create_vport(apc, ndev);\n@@ -3589,7 +3850,7 @@ int mana_alloc_queues(struct net_device *ndev)\n \t\tgoto destroy_vport;\n \t}\n \n-\terr = mana_create_txq(apc, ndev);\n+\terr = mana_create_txq(apc, ndev, 0);\n \tif (err) {\n \t\tnetdev_err(ndev, \"Failed to create TXQ on vPort %u: %d\\n\",\n \t\t\t apc-\u003eport_idx, err);\n@@ -3604,7 +3865,7 @@ int mana_alloc_queues(struct net_device *ndev)\n \t\tgoto destroy_txq;\n \t}\n \n-\terr = mana_add_rx_queues(apc, ndev);\n+\terr = mana_add_rx_queues(apc, ndev, 0);\n \tif (err)\n \t\tgoto destroy_rxq;\n \n@@ -3618,7 +3879,18 @@ int mana_alloc_queues(struct net_device *ndev)\n \t\tgoto destroy_rxq;\n \t}\n \n-\tmana_rss_table_init(apc);\n+\t/* Keep a user-configured RSS table across a rebuild; the entries are\n+\t * queue indices, so they stay meaningful as long as the queue count\n+\t * is unchanged. Only a driver-generated table is regenerated here.\n+\t *\n+\t * Nothing to roll back to here, so report the loss as soon as it is\n+\t * decided and keep the table and the core's view of it in step.\n+\t */\n+\tif (!mana_rss_table_keep(apc, apc-\u003enum_queues, \u0026indir_lost)) {\n+\t\tif (indir_lost)\n+\t\t\tethtool_rxfh_indir_lost(ndev);\n+\t\tmana_rss_table_init(apc);\n+\t}\n \n \terr = mana_config_rss(apc, TRI_STATE_TRUE, true, true);\n \tif (err) {\n@@ -3676,15 +3948,114 @@ int mana_attach(struct net_device *ndev)\n \treturn 0;\n }\n \n-static int mana_dealloc_queues(struct net_device *ndev)\n+/* Drain a set about to be destroyed: nothing new can reach it, so wait for the\n+ * hardware to finish what it owns, then release every mapped SKB.\n+ *\n+ * The 120s budget is shared across all queues. On timeout the device is reset,\n+ * since its buffers are about to be freed while it may still DMA into them; if\n+ * that fails too they are leaked.\n+ *\n+ * Returns true only if a reset happened, taking every queue on the function\n+ * down with it.\n+ */\n+static bool mana_drain_txqs(struct mana_port_context *apc)\n {\n-\tstruct mana_port_context *apc = netdev_priv(ndev);\n \tunsigned long timeout = jiffies + 120 * HZ;\n \tstruct gdma_dev *gd = apc-\u003eac-\u003egdma_dev;\n+\tbool quiesced = true;\n+\tbool reset = false;\n \tstruct mana_txq *txq;\n \tstruct sk_buff *skb;\n-\tint i, err;\n \tu32 tsleep;\n+\tint i, err;\n+\n+\tif (!apc-\u003etx_qp)\n+\t\treturn false;\n+\n+\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\t\tif (!apc-\u003etx_qp[i])\n+\t\t\tcontinue;\n+\n+\t\ttxq = \u0026apc-\u003etx_qp[i]-\u003etxq;\n+\n+\t\t/* The function was reset after this queue was created, so the\n+\t\t * device has stopped touching its buffers and the completions\n+\t\t * waited for below can never arrive. Without this the port\n+\t\t * would burn the full timeout under RTNL, then reset the\n+\t\t * function again on the way out.\n+\t\t */\n+\t\tif (READ_ONCE(apc-\u003eac-\u003ereset_gen) != txq-\u003ereset_gen)\n+\t\t\tcontinue;\n+\n+\t\ttsleep = 1000;\n+\t\twhile (atomic_read(\u0026txq-\u003epending_sends) \u003e 0 \u0026\u0026\n+\t\t time_before(jiffies, timeout)) {\n+\t\t\tusleep_range(tsleep, tsleep + 1000);\n+\t\t\ttsleep \u003c\u003c= 1;\n+\t\t}\n+\t\tif (atomic_read(\u0026txq-\u003epending_sends)) {\n+\t\t\t/* The device still owns these buffers, so reset it\n+\t\t\t * before they are freed. pci_try_reset_function()\n+\t\t\t * rather than pcie_flr(): it saves and restores config\n+\t\t\t * space, which a bare FLR wipes behind the PCI core's\n+\t\t\t * back. Trylock because RTNL is held here while the\n+\t\t\t * remove path takes the device lock first.\n+\t\t\t */\n+\t\t\terr = pci_try_reset_function(to_pci_dev(gd-\u003egdma_context-\u003edev));\n+\t\t\tif (err) {\n+\t\t\t\tnetdev_err(apc-\u003endev,\n+\t\t\t\t\t \"function reset failed: %d, %d pkts pending in txq %u\\n\",\n+\t\t\t\t\t err, atomic_read(\u0026txq-\u003epending_sends),\n+\t\t\t\t\t txq-\u003egdma_txq_id);\n+\t\t\t\tquiesced = false;\n+\t\t\t} else {\n+\t\t\t\t/* Every queue on the function is dead now,\n+\t\t\t\t * including the ones this loop has not reached\n+\t\t\t\t * and those of the other ports.\n+\t\t\t\t */\n+\t\t\t\tWRITE_ONCE(apc-\u003eac-\u003ereset_gen,\n+\t\t\t\t\t apc-\u003eac-\u003ereset_gen + 1);\n+\n+\t\t\t\t/* Only a reset that actually happened takes the\n+\t\t\t\t * other ports down with it; reporting a failed\n+\t\t\t\t * one would rebuild them for nothing.\n+\t\t\t\t */\n+\t\t\t\treset = true;\n+\t\t\t}\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\n+\t/* Only a reset that actually happened makes freeing these safe; without\n+\t * one the device still owns them. Leak instead, bounded at one SQ ring\n+\t * of skbs per queue.\n+\t */\n+\tif (!quiesced) {\n+\t\tnetdev_err(apc-\u003endev,\n+\t\t\t \"device not quiesced, leaking pending TX buffers instead of unmapping memory it can still DMA from\\n\");\n+\t\treturn reset;\n+\t}\n+\n+\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\t\tif (!apc-\u003etx_qp[i])\n+\t\t\tcontinue;\n+\n+\t\ttxq = \u0026apc-\u003etx_qp[i]-\u003etxq;\n+\t\twhile ((skb = skb_dequeue(\u0026txq-\u003epending_skbs))) {\n+\t\t\tmana_unmap_skb(skb, apc);\n+\t\t\tdev_kfree_skb_any(skb);\n+\t\t}\n+\t\tatomic_set(\u0026txq-\u003epending_sends, 0);\n+\t}\n+\n+\treturn reset;\n+}\n+\n+static int mana_dealloc_queues(struct net_device *ndev)\n+{\n+\tstruct mana_port_context *apc = netdev_priv(ndev);\n+\tstruct gdma_dev *gd = apc-\u003eac-\u003egdma_dev;\n+\tint err;\n \n \tif (apc-\u003eport_is_up)\n \t\treturn -EINVAL;\n@@ -3702,41 +4073,27 @@ static int mana_dealloc_queues(struct net_device *ndev)\n \t * new packets due to apc-\u003eport_is_up being false.\n \t *\n \t * Drain all the in-flight TX packets.\n-\t * A timeout of 120 seconds for all the queues is used.\n-\t * This will break the while loop when h/w is not responding.\n-\t * This value of 120 has been decided here considering max\n-\t * number of queues.\n+\t *\n+\t * If the drain had to reset the function to get there, every other\n+\t * port on the adapter lost its queues too, so schedule them for a\n+\t * rebuild. This port is being torn down here and needs no such\n+\t * treatment, and a down port stays down: with port_st_save false,\n+\t * detach and attach both skip the queue work.\n \t */\n+\tif (mana_drain_txqs(apc)) {\n+\t\tstruct mana_context *ac = apc-\u003eac;\n+\t\tunsigned int i;\n \n-\tif (apc-\u003etx_qp) {\n-\t\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n-\t\t\ttxq = \u0026apc-\u003etx_qp[i]-\u003etxq;\n-\t\t\ttsleep = 1000;\n-\t\t\twhile (atomic_read(\u0026txq-\u003epending_sends) \u003e 0 \u0026\u0026\n-\t\t\t time_before(jiffies, timeout)) {\n-\t\t\t\tusleep_range(tsleep, tsleep + 1000);\n-\t\t\t\ttsleep \u003c\u003c= 1;\n-\t\t\t}\n-\t\t\tif (atomic_read(\u0026txq-\u003epending_sends)) {\n-\t\t\t\terr =\n-\t\t\t\t pcie_flr(to_pci_dev(gd-\u003egdma_context-\u003edev));\n-\t\t\t\tif (err) {\n-\t\t\t\t\tnetdev_err(ndev, \"flr failed %d with %d pkts pending in txq %u\\n\",\n-\t\t\t\t\t\t err,\n-\t\t\t\t\t atomic_read(\u0026txq-\u003epending_sends),\n-\t\t\t\t\t txq-\u003egdma_txq_id);\n-\t\t\t\t}\n-\t\t\t\tbreak;\n-\t\t\t}\n-\t\t}\n+\t\tfor (i = 0; i \u003c ac-\u003enum_ports; i++) {\n+\t\t\tstruct mana_port_context *sib;\n \n-\t\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n-\t\t\ttxq = \u0026apc-\u003etx_qp[i]-\u003etxq;\n-\t\t\twhile ((skb = skb_dequeue(\u0026txq-\u003epending_skbs))) {\n-\t\t\t\tmana_unmap_skb(skb, apc);\n-\t\t\t\tdev_kfree_skb_any(skb);\n-\t\t\t}\n-\t\t\tatomic_set(\u0026txq-\u003epending_sends, 0);\n+\t\t\tif (!ac-\u003eports[i] || ac-\u003eports[i] == ndev)\n+\t\t\t\tcontinue;\n+\t\t\tsib = netdev_priv(ac-\u003eports[i]);\n+\t\t\tnetdev_err(ac-\u003eports[i],\n+\t\t\t\t \"queues reset by a sibling port, scheduling rebuild\\n\");\n+\t\t\tqueue_work(ac-\u003eper_port_queue_reset_wq,\n+\t\t\t\t \u0026sib-\u003equeue_reset_work);\n \t\t}\n \t}\n \n@@ -3760,6 +4117,868 @@ static int mana_dealloc_queues(struct net_device *ndev)\n \treturn 0;\n }\n \n+/*\n+ * Pre-allocate and swap reconfiguration.\n+ *\n+ * Build a new queue set while the current one serves traffic, publish it, then\n+ * destroy the old one. A failed allocation leaves the running config untouched,\n+ * and the vport is never torn down, so RDMA cannot take it mid-swap. The cost\n+ * is room for both sets at once, so a rebuild at the vport's maximum queue\n+ * count can be refused; EQs are shared from a port-owned pool, not doubled.\n+ *\n+ * Everything builds in a scratch mana_port_context, since mana_start_xmit()\n+ * dereferences apc-\u003etx_qp[] guarded only by port_is_up. Per-queue debugfs is\n+ * suppressed meanwhile, as the names would collide.\n+ */\n+\n+/* Snapshot the queue-set fields of @ctx into @out. */\n+static void mana_qset_snapshot(const struct mana_port_context *ctx,\n+\t\t\t struct mana_qset *out)\n+{\n+\tout-\u003etx_qp\t\t= ctx-\u003etx_qp;\n+\tout-\u003erxqs\t\t= ctx-\u003erxqs;\n+\tout-\u003eindir_table\t= ctx-\u003eindir_table;\n+\tout-\u003eindir_table_sz\t= ctx-\u003eindir_table_sz;\n+\tout-\u003erxobj_table\t= ctx-\u003erxobj_table;\n+\tout-\u003edefault_rxobj\t= ctx-\u003edefault_rxobj;\n+\tout-\u003enum_queues\t\t= ctx-\u003enum_queues;\n+\tout-\u003erx_queue_size\t= ctx-\u003erx_queue_size;\n+\tout-\u003etx_queue_size\t= ctx-\u003etx_queue_size;\n+\tout-\u003epriv_flags\t\t= ctx-\u003epriv_flags;\n+\tout-\u003emtu\t\t= ctx-\u003econfigured_mtu;\n+\tout-\u003ebpf_prog\t\t= ctx-\u003ebpf_prog;\n+\n+\t/* A set taken from a live context has nothing pending; the builders\n+\t * set this after snapshotting if they had to drop the user's table.\n+\t */\n+\tout-\u003erxfh_indir_lost\t= false;\n+}\n+\n+/* The vport (port_handle, vport_use_count) and the port-level debugfs dir are\n+ * not touched: they outlive any individual queue set.\n+ */\n+static void mana_qset_install(struct mana_port_context *ctx,\n+\t\t\t const struct mana_qset *qset)\n+{\n+\tctx-\u003etx_qp\t\t= qset-\u003etx_qp;\n+\tctx-\u003erxqs\t\t= qset-\u003erxqs;\n+\tctx-\u003eindir_table\t= qset-\u003eindir_table;\n+\tctx-\u003eindir_table_sz\t= qset-\u003eindir_table_sz;\n+\tctx-\u003erxobj_table\t= qset-\u003erxobj_table;\n+\tctx-\u003edefault_rxobj\t= qset-\u003edefault_rxobj;\n+\tctx-\u003enum_queues\t\t= qset-\u003enum_queues;\n+\tctx-\u003erx_queue_size\t= qset-\u003erx_queue_size;\n+\tctx-\u003etx_queue_size\t= qset-\u003etx_queue_size;\n+\tctx-\u003epriv_flags\t\t= qset-\u003epriv_flags;\n+\tctx-\u003econfigured_mtu\t= qset-\u003emtu;\n+\tctx-\u003ebpf_prog\t\t= qset-\u003ebpf_prog;\n+}\n+\n+/**\n+ * mana_qset_scratch_alloc - build a scratch port context for queue work\n+ * @apc: the live port context to shadow\n+ *\n+ * Returns a heap copy of @apc that shares its vport identity but owns no\n+ * queues, so the existing allocators and destroyers can run against it\n+ * without touching the live context.\n+ */\n+struct mana_port_context *mana_qset_scratch_alloc(struct mana_port_context *apc)\n+{\n+\tstruct mana_port_context *scratch;\n+\n+\tscratch = kvzalloc(sizeof(*scratch), GFP_KERNEL);\n+\tif (!scratch)\n+\t\treturn NULL;\n+\n+\t*scratch = *apc;\n+\n+\t/* EQs stay shared with the live port: they are a vector-backed\n+\t * resource and must not be duplicated for the new set.\n+\t */\n+\tscratch-\u003etx_qp\t\t= NULL;\n+\tscratch-\u003erxqs\t\t= NULL;\n+\tscratch-\u003eindir_table\t= NULL;\n+\tscratch-\u003erxobj_table\t= NULL;\n+\tscratch-\u003edefault_rxobj\t= INVALID_MANA_HANDLE;\n+\n+\t/* Never consume the live set's pre-allocated RX buffers; the swap path\n+\t * has no post-teardown allocation to de-risk.\n+\t */\n+\tscratch-\u003erxbufs_pre\t= NULL;\n+\tscratch-\u003edas_pre\t= NULL;\n+\tscratch-\u003erxbpre_total\t= 0;\n+\n+\t/* Two sets are alive at once and would collide on the same names under\n+\t * vport%d. An IS_ERR() parent makes every create and remove a no-op.\n+\t */\n+\tscratch-\u003emana_port_debugfs = ERR_PTR(-ENODEV);\n+\n+\treturn scratch;\n+}\n+\n+void mana_qset_scratch_free(struct mana_port_context *scratch)\n+{\n+\tkvfree(scratch);\n+}\n+\n+/* Carve the live set into a kept prefix [0, @new_count) in @out_new and a\n+ * tail to retire in @out_tail. @apc is untouched, on failure too.\n+ *\n+ * Queue i is built from the ring sizes and the buffer layout and keeps EQ i\n+ * at any count, so a reduction carries the survivors over and destroys only\n+ * the tail. Allocates two pointer arrays and a steering table, nothing else:\n+ * the queues that stay keep their page pools, posted buffers and NAPI.\n+ */\n+int mana_split_qset(struct mana_port_context *apc,\n+\t\t struct mana_port_context *scratch, unsigned int new_count,\n+\t\t struct mana_qset *out_new, struct mana_qset *out_tail)\n+{\n+\tunsigned int old_count = apc-\u003enum_queues;\n+\tstruct mana_tx_qp **new_tx, **tail_tx;\n+\tstruct mana_rxq **new_rx, **tail_rx;\n+\tunsigned int tail_count;\n+\tbool indir_lost;\n+\tunsigned int i;\n+\tint err;\n+\n+\tASSERT_RTNL();\n+\n+\tif (WARN_ON(new_count == 0 || new_count \u003e= old_count))\n+\t\treturn -EINVAL;\n+\tif (WARN_ON(!apc-\u003etx_qp || !apc-\u003erxqs))\n+\t\treturn -EINVAL;\n+\n+\ttail_count = old_count - new_count;\n+\n+\t/* Build the smaller set's steering table separately: mana_config_rss()\n+\t * would otherwise index the shorter rxqs[] with entries still referring\n+\t * to retired queues.\n+\t */\n+\tscratch-\u003enum_queues = new_count;\n+\terr = mana_rss_table_alloc(scratch);\n+\tif (err)\n+\t\treturn err;\n+\n+\tif (mana_rss_table_keep(apc, new_count, \u0026indir_lost))\n+\t\tmemcpy(scratch-\u003eindir_table, apc-\u003eindir_table,\n+\t\t apc-\u003eindir_table_sz * sizeof(*apc-\u003eindir_table));\n+\telse\n+\t\tmana_rss_table_init(scratch);\n+\n+\tnew_tx = kzalloc_objs(struct mana_tx_qp *, new_count);\n+\tnew_rx = kzalloc_objs(struct mana_rxq *, new_count);\n+\ttail_tx = kzalloc_objs(struct mana_tx_qp *, tail_count);\n+\ttail_rx = kzalloc_objs(struct mana_rxq *, tail_count);\n+\tif (!new_tx || !new_rx || !tail_tx || !tail_rx) {\n+\t\terr = -ENOMEM;\n+\t\tgoto free_arrays;\n+\t}\n+\n+\tfor (i = 0; i \u003c new_count; i++) {\n+\t\tnew_tx[i] = apc-\u003etx_qp[i];\n+\t\tnew_rx[i] = apc-\u003erxqs[i];\n+\t}\n+\tfor (i = 0; i \u003c tail_count; i++) {\n+\t\ttail_tx[i] = apc-\u003etx_qp[new_count + i];\n+\t\ttail_rx[i] = apc-\u003erxqs[new_count + i];\n+\t}\n+\n+\t/* The kept prefix, with the new steering table. */\n+\tout_new-\u003etx_qp\t\t= new_tx;\n+\tout_new-\u003erxqs\t\t= new_rx;\n+\tout_new-\u003eindir_table\t= scratch-\u003eindir_table;\n+\tout_new-\u003eindir_table_sz\t= scratch-\u003eindir_table_sz;\n+\tout_new-\u003erxobj_table\t= scratch-\u003erxobj_table;\n+\tout_new-\u003edefault_rxobj\t= apc-\u003erxqs[0]-\u003erxobj;\n+\tout_new-\u003enum_queues\t= new_count;\n+\tout_new-\u003erx_queue_size\t= apc-\u003erx_queue_size;\n+\tout_new-\u003etx_queue_size\t= apc-\u003etx_queue_size;\n+\tout_new-\u003epriv_flags\t= apc-\u003epriv_flags;\n+\tout_new-\u003emtu\t\t= apc-\u003econfigured_mtu;\n+\tout_new-\u003ebpf_prog\t= apc-\u003ebpf_prog;\n+\tout_new-\u003erxfh_indir_lost = indir_lost;\n+\n+\t/* Ownership of the table moved to @out_new. */\n+\tscratch-\u003eindir_table\t= NULL;\n+\tscratch-\u003erxobj_table\t= NULL;\n+\n+\t/* The tail. It owns no steering table; bpf_prog is carried so that\n+\t * retiring it drops exactly the tail's per-queue program references\n+\t * and leaves the kept ones alone.\n+\t */\n+\tmemset(out_tail, 0, sizeof(*out_tail));\n+\tout_tail-\u003etx_qp\t\t= tail_tx;\n+\tout_tail-\u003erxqs\t\t= tail_rx;\n+\tout_tail-\u003edefault_rxobj\t= INVALID_MANA_HANDLE;\n+\tout_tail-\u003enum_queues\t= tail_count;\n+\tout_tail-\u003erx_queue_size\t= apc-\u003erx_queue_size;\n+\tout_tail-\u003etx_queue_size\t= apc-\u003etx_queue_size;\n+\tout_tail-\u003epriv_flags\t= apc-\u003epriv_flags;\n+\tout_tail-\u003emtu\t\t= apc-\u003econfigured_mtu;\n+\tout_tail-\u003ebpf_prog\t= apc-\u003ebpf_prog;\n+\n+\treturn 0;\n+\n+free_arrays:\n+\tkfree(new_tx);\n+\tkfree(new_rx);\n+\tkfree(tail_tx);\n+\tkfree(tail_rx);\n+\tmana_cleanup_indir_table(scratch);\n+\treturn err;\n+}\n+\n+/**\n+ * mana_discard_split - drop the containers built by mana_split_qset()\n+ * @newq: set that was never published\n+ * @tailq: matching tail\n+ *\n+ * Frees the pointer arrays and the steering table only: the queues they refer\n+ * to are still owned by the live port context.\n+ */\n+void mana_discard_split(struct mana_qset *newq, struct mana_qset *tailq)\n+{\n+\tkfree(newq-\u003etx_qp);\n+\tkfree(newq-\u003erxqs);\n+\tkfree(newq-\u003eindir_table);\n+\tkfree(newq-\u003erxobj_table);\n+\tkfree(tailq-\u003etx_qp);\n+\tkfree(tailq-\u003erxqs);\n+\tmemset(newq, 0, sizeof(*newq));\n+\tmemset(tailq, 0, sizeof(*tailq));\n+}\n+\n+/* The mirror image of mana_split_qset(): carry the existing queues over into\n+ * @out_new and build only the [old, @new_count) tail. Growing 4 channels to 8\n+ * creates 4 SQ/RQ pairs, not 8, and never holds 12 against the vport maximum.\n+ *\n+ * @out_fresh names just the queues created here, so a failed publish retires\n+ * exactly those. On failure @apc is untouched.\n+ */\n+int mana_grow_qset(struct mana_port_context *apc,\n+\t\t struct mana_port_context *scratch, unsigned int new_count,\n+\t\t struct mana_qset *out_new, struct mana_qset *out_fresh)\n+{\n+\tunsigned int old_count = apc-\u003enum_queues;\n+\tstruct mana_tx_qp **new_tx, **fresh_tx;\n+\tstruct mana_rxq **new_rx, **fresh_rx;\n+\tstruct net_device *ndev = apc-\u003endev;\n+\tunsigned int fresh_count;\n+\tbool indir_lost;\n+\tunsigned int i;\n+\tint err;\n+\n+\tASSERT_RTNL();\n+\n+\tif (WARN_ON(new_count \u003c= old_count))\n+\t\treturn -EINVAL;\n+\tif (WARN_ON(!apc-\u003etx_qp || !apc-\u003erxqs))\n+\t\treturn -EINVAL;\n+\n+\tfresh_count = new_count - old_count;\n+\n+\tnew_tx = kzalloc_objs(struct mana_tx_qp *, new_count);\n+\tnew_rx = kzalloc_objs(struct mana_rxq *, new_count);\n+\tfresh_tx = kzalloc_objs(struct mana_tx_qp *, fresh_count);\n+\tfresh_rx = kzalloc_objs(struct mana_rxq *, fresh_count);\n+\tif (!new_tx || !new_rx || !fresh_tx || !fresh_rx) {\n+\t\terr = -ENOMEM;\n+\t\tgoto free_arrays;\n+\t}\n+\n+\tfor (i = 0; i \u003c old_count; i++) {\n+\t\tnew_tx[i] = apc-\u003etx_qp[i];\n+\t\tnew_rx[i] = apc-\u003erxqs[i];\n+\t}\n+\n+\t/* @scratch now describes the merged set; the builders fill only the\n+\t * [old_count, new_count) slots.\n+\t */\n+\tscratch-\u003enum_queues = new_count;\n+\tscratch-\u003etx_qp = new_tx;\n+\tscratch-\u003erxqs = new_rx;\n+\n+\terr = mana_rss_table_alloc(scratch);\n+\tif (err)\n+\t\tgoto free_arrays;\n+\n+\t/* Same shared, port-owned EQ pool as a full rebuild; this only adds\n+\t * the vectors the extra queues need.\n+\t */\n+\terr = mana_grow_eqs(apc, new_count);\n+\tif (err)\n+\t\tgoto cleanup_rss;\n+\n+\tscratch-\u003eeqs = apc-\u003eeqs;\n+\tscratch-\u003enum_eqs = apc-\u003enum_eqs;\n+\n+\terr = mana_create_txq(scratch, ndev, old_count);\n+\tif (err)\n+\t\tgoto cleanup_rss; /* create_txq already undid its own work */\n+\n+\terr = mana_add_rx_queues(scratch, ndev, old_count);\n+\tif (err)\n+\t\tgoto cleanup_rxq;\n+\n+\tif (mana_rss_table_keep(apc, new_count, \u0026indir_lost))\n+\t\tmemcpy(scratch-\u003eindir_table, apc-\u003eindir_table,\n+\t\t apc-\u003eindir_table_sz * sizeof(*apc-\u003eindir_table));\n+\telse\n+\t\tmana_rss_table_init(scratch);\n+\n+\tmana_qset_snapshot(scratch, out_new);\n+\tout_new-\u003erxfh_indir_lost = indir_lost;\n+\n+\tfor (i = 0; i \u003c fresh_count; i++) {\n+\t\tfresh_tx[i] = new_tx[old_count + i];\n+\t\tfresh_rx[i] = new_rx[old_count + i];\n+\t}\n+\n+\tmemset(out_fresh, 0, sizeof(*out_fresh));\n+\tout_fresh-\u003etx_qp\t= fresh_tx;\n+\tout_fresh-\u003erxqs\t\t= fresh_rx;\n+\tout_fresh-\u003edefault_rxobj = INVALID_MANA_HANDLE;\n+\tout_fresh-\u003enum_queues\t= fresh_count;\n+\tout_fresh-\u003erx_queue_size = apc-\u003erx_queue_size;\n+\tout_fresh-\u003etx_queue_size = apc-\u003etx_queue_size;\n+\tout_fresh-\u003epriv_flags\t= apc-\u003epriv_flags;\n+\tout_fresh-\u003emtu\t\t= apc-\u003econfigured_mtu;\n+\tout_fresh-\u003ebpf_prog\t= apc-\u003ebpf_prog;\n+\n+\t/* mana_publish_qset() cannot do this: mana_chn_setxdp() decides from\n+\t * rxqs[0], a carried-over queue that already holds the program, and\n+\t * returns early. Address only the new queues through @out_fresh so\n+\t * exactly fresh_count references are taken.\n+\t */\n+\tmana_qset_install(scratch, out_fresh);\n+\tmana_chn_setxdp(scratch, mana_xdp_get(apc));\n+\n+\treturn 0;\n+\n+cleanup_rxq:\n+\tmana_destroy_rxqs_from(scratch, old_count);\n+\tmana_destroy_txq_from(scratch, old_count);\n+cleanup_rss:\n+\tmana_cleanup_indir_table(scratch);\n+free_arrays:\n+\t/* Only the containers: every queue they name is still live on @apc. */\n+\tscratch-\u003etx_qp = NULL;\n+\tscratch-\u003erxqs = NULL;\n+\tkfree(new_tx);\n+\tkfree(new_rx);\n+\tkfree(fresh_tx);\n+\tkfree(fresh_rx);\n+\n+\t/* Give back any EQ this attempt added rather than holding its MSI-X\n+\t * vectors: the live set still needs only apc-\u003enum_queues of them, and\n+\t * every CQ this call created has been destroyed above.\n+\t */\n+\tmana_shrink_eqs(apc, apc-\u003enum_queues);\n+\n+\tnetdev_err(ndev, \"mana_grow_qset(num_queues=%u) failed: %d\\n\",\n+\t\t new_count, err);\n+\treturn err;\n+}\n+\n+/**\n+ * mana_discard_grow - drop the merged containers built by mana_grow_qset()\n+ * @newq: set that was never published\n+ *\n+ * Frees the pointer arrays and steering table only: carried-over queues\n+ * belong to the live context, fresh ones are retired through @out_fresh.\n+ */\n+void mana_discard_grow(struct mana_qset *newq)\n+{\n+\tkfree(newq-\u003etx_qp);\n+\tkfree(newq-\u003erxqs);\n+\tkfree(newq-\u003eindir_table);\n+\tkfree(newq-\u003erxobj_table);\n+\tmemset(newq, 0, sizeof(*newq));\n+}\n+\n+/* Rebuild the queues at the current count in @scratch, for callers changing a\n+ * per-queue property; a count change goes through mana_split_qset() or\n+ * mana_grow_qset(), so this never has to add an EQ. The installed set keeps\n+ * serving traffic meanwhile. On error nothing is left allocated.\n+ */\n+int mana_alloc_qset(struct mana_port_context *apc,\n+\t\t struct mana_port_context *scratch,\n+\t\t unsigned int rx_queue_size, unsigned int tx_queue_size,\n+\t\t u32 priv_flags, int mtu, struct bpf_prog *bpf_prog,\n+\t\t struct mana_qset *out)\n+{\n+\tstruct net_device *ndev = scratch-\u003endev;\n+\tbool indir_lost;\n+\tint err;\n+\n+\tASSERT_RTNL();\n+\n+\tscratch-\u003enum_queues\t= apc-\u003enum_queues;\n+\tscratch-\u003erx_queue_size\t= rx_queue_size;\n+\tscratch-\u003etx_queue_size\t= tx_queue_size;\n+\tscratch-\u003epriv_flags\t= priv_flags;\n+\n+\t/* mana_get_rxbuf_cfg() reads both of these when sizing RX buffers,\n+\t * so the new set is built for the requested MTU / XDP program\n+\t * without disturbing the running set.\n+\t */\n+\tscratch-\u003econfigured_mtu\t= mtu;\n+\tscratch-\u003ebpf_prog\t= bpf_prog;\n+\n+\terr = mana_init_port_context(scratch);\n+\tif (err)\n+\t\tgoto out_err;\n+\n+\terr = mana_rss_table_alloc(scratch);\n+\tif (err)\n+\t\tgoto cleanup_rxq_array;\n+\n+\t/* The queue count is unchanged, so the port's shared EQ pool already\n+\t * has an EQ for every queue this set will build. Both sets reference\n+\t * the same pool while they are live, so a swap never needs old + new\n+\t * MSI-X vectors.\n+\t */\n+\tscratch-\u003eeqs = apc-\u003eeqs;\n+\tscratch-\u003enum_eqs = apc-\u003enum_eqs;\n+\n+\terr = mana_create_txq(scratch, ndev, 0);\n+\tif (err)\n+\t\tgoto cleanup_rss;\n+\n+\terr = mana_add_rx_queues(scratch, ndev, 0);\n+\tif (err)\n+\t\tgoto cleanup_rxq;\n+\n+\t/* Carry a user-configured RSS table over to the new set. The entries\n+\t * are queue indices, so mana_config_rss() in mana_publish_qset() maps\n+\t * them onto the new set's RX objects. A driver-generated table is\n+\t * rebuilt instead, so it covers every queue of the new set.\n+\t */\n+\tif (mana_rss_table_keep(apc, scratch-\u003enum_queues, \u0026indir_lost))\n+\t\tmemcpy(scratch-\u003eindir_table, apc-\u003eindir_table,\n+\t\t apc-\u003eindir_table_sz * sizeof(*apc-\u003eindir_table));\n+\telse\n+\t\tmana_rss_table_init(scratch);\n+\n+\tmana_qset_snapshot(scratch, out);\n+\tout-\u003erxfh_indir_lost = indir_lost;\n+\treturn 0;\n+\n+cleanup_rxq:\n+\t/* mana_add_rx_queues() may have created queues before failing; they\n+\t * own RQ/CQ objects, NAPI state and page pools, so tear down whatever\n+\t * made it into scratch-\u003erxqs[] before dropping the array.\n+\t */\n+\tmana_destroy_rxqs(scratch);\n+\tmana_destroy_txq(scratch);\n+cleanup_rss:\n+\tmana_cleanup_indir_table(scratch);\n+cleanup_rxq_array:\n+\tkfree(scratch-\u003erxqs);\n+\tscratch-\u003erxqs = NULL;\n+out_err:\n+\t/* No EQ to give back: this path never adds one, it reuses the pool\n+\t * the live set is already using.\n+\t */\n+\tnetdev_err(ndev, \"mana_alloc_qset(num_queues=%u) failed: %d\\n\",\n+\t\t apc-\u003enum_queues, err);\n+\treturn err;\n+}\n+\n+/* Close a port mana_publish_qset() gave up on; does nothing otherwise. Under\n+ * RTNL.\n+ *\n+ * The caller releases the unpublished set first: closing destroys the shared\n+ * EQ pool its CQs attach to, and only the caller knows whether it owns its\n+ * queues or shares them with the live set. RX is already off.\n+ *\n+ * Merely stopping the port would leave port_is_up false with queues still\n+ * allocated, so mana_detach() skips teardown and the next open trips\n+ * WARN_ON(apc-\u003eeqs).\n+ */\n+void mana_publish_close_if_needed(struct mana_port_context *apc)\n+{\n+\tASSERT_RTNL();\n+\n+\tif (!apc-\u003epublish_dead_end)\n+\t\treturn;\n+\n+\tapc-\u003epublish_dead_end = false;\n+\n+\t/* mana_dealloc_queues() requires the port already marked down, which\n+\t * mana_publish_qset() did before the swap it is unwinding.\n+\t */\n+\tif (mana_dealloc_queues(apc-\u003endev))\n+\t\tnetdev_err(apc-\u003endev,\n+\t\t\t \"failed to close the port after a failed rollback\\n\");\n+}\n+\n+/* Start only the netdev queues that can take work. A carried-over queue may\n+ * still have a full ring, and restarting it would just make mana_start_xmit()\n+ * drop; leave it for mana_poll_tx_cq() to wake. Must run after port_is_up is\n+ * set, or that wakeup is gated off.\n+ */\n+static void mana_start_txqs(struct mana_port_context *apc)\n+{\n+\tstruct net_device *ndev = apc-\u003endev;\n+\tunsigned int i;\n+\n+\tif (!apc-\u003etx_qp)\n+\t\treturn;\n+\n+\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\t\tif (!apc-\u003etx_qp[i])\n+\t\t\tcontinue;\n+\n+\t\tif (mana_can_tx(apc-\u003etx_qp[i]-\u003etxq.gdma_sq))\n+\t\t\tnetif_tx_wake_queue(netdev_get_tx_queue(ndev, i));\n+\t}\n+}\n+\n+/* A retiring queue shares its struct netdev_queue with whatever replaced it\n+ * at the same index, and only ever drains, so it always looks like it has\n+ * room. Without this flag its completions would wake a netdev queue that the\n+ * live queue stopped on a full ring.\n+ *\n+ * A queue both sets own must end up unmarked, so callers mark the leaving set\n+ * first and unmark the incoming one second.\n+ */\n+static void mana_qset_set_retiring(struct mana_qset *qset, bool retiring)\n+{\n+\tunsigned int q;\n+\n+\tif (!qset-\u003etx_qp)\n+\t\treturn;\n+\n+\tfor (q = 0; q \u003c qset-\u003enum_queues; q++) {\n+\t\tif (qset-\u003etx_qp[q])\n+\t\t\tWRITE_ONCE(qset-\u003etx_qp[q]-\u003etxq.retiring, retiring);\n+\t}\n+}\n+\n+/* Give up on a swap. Steering may still point at the set the caller is about\n+ * to free, and restoring it is exactly what failed, so stop delivery before\n+ * those RQs and their buffers go away. This is the narrow steering request -\n+ * no key, table or default-rxobj update - so it can land where the full\n+ * mana_config_rss() restore did not.\n+ *\n+ * Closing the port is left to mana_publish_close_if_needed(), which must run\n+ * after the caller has released that set.\n+ */\n+static void mana_publish_give_up(struct mana_port_context *apc)\n+{\n+\tint err;\n+\n+\tapc-\u003erss_state = TRI_STATE_FALSE;\n+\n+\terr = mana_disable_vport_rx(apc);\n+\tif (err \u0026\u0026 mana_en_need_log(apc, err))\n+\t\tnetdev_err(apc-\u003endev, \"failed to disable vPort RX: %d\\n\", err);\n+\n+\tapc-\u003epublish_dead_end = true;\n+}\n+\n+/* Swap @newq onto @apc, handing the previous set back in @out_old for the\n+ * caller to free. On failure the old set is reinstalled and the caller frees\n+ * only @newq. Must be called under RTNL.\n+ *\n+ * netif_tx_disable() is load-bearing: mana_start_xmit() dereferences\n+ * apc-\u003etx_qp[] guarded only by apc-\u003eport_is_up, and ndo_xdp_xmit() bypasses\n+ * the txq-stopped checks, so port_is_up is cleared over the same window.\n+ */\n+int mana_publish_qset(struct mana_port_context *apc, struct mana_qset *newq,\n+\t\t struct mana_qset *out_old)\n+{\n+\tstruct net_device *ndev = apc-\u003endev;\n+\tbool carrier_ok;\n+\tint err;\n+\n+\tASSERT_RTNL();\n+\n+\tcarrier_ok = netif_carrier_ok(ndev);\n+\tnetif_carrier_off(ndev);\n+\n+\t/* Clear port_is_up before stopping the queues, pairing with the\n+\t * smp_rmb() in mana_poll_tx_cq(): that reader samples\n+\t * netif_tx_queue_stopped() first, so a completion seeing a queue\n+\t * stopped here also sees port_is_up false and will not wake it\n+\t * mid-swap. It also fences mana_xdp_xmit(), which is gated only by\n+\t * port_is_up and would otherwise index a stale apc-\u003etx_qp[].\n+\t */\n+\tWRITE_ONCE(apc-\u003eport_is_up, false);\n+\n+\t/* Ensure port state updated before txq state */\n+\tsmp_wmb();\n+\n+\tnetif_tx_disable(ndev);\n+\n+\tmana_qset_snapshot(apc, out_old);\n+\n+\t/* Mark the outgoing set before the grace period, not after: a\n+\t * completion that saw the flag clear must not still be in flight when\n+\t * the gate reopens, or it could wake a netdev queue that its\n+\t * replacement had stopped on a full ring.\n+\t */\n+\tmana_qset_set_retiring(out_old, true);\n+\n+\t/* Wait out any transmit or ndo_xdp_xmit() that was already past the\n+\t * port_is_up test before the swap touches apc-\u003etx_qp / the counts,\n+\t * and any completion that still saw the flag clear above.\n+\t */\n+\tsynchronize_net();\n+\n+\t/* Anything the incoming set carries over is staying, so clear the flag\n+\t * again - after the marking above, before the gate reopens.\n+\t */\n+\tmana_qset_set_retiring(newq, false);\n+\n+\tmana_qset_install(apc, newq);\n+\tapc-\u003erss_state = apc-\u003enum_queues \u003e 1 ? TRI_STATE_TRUE : TRI_STATE_FALSE;\n+\n+\terr = netif_set_real_num_tx_queues(ndev, apc-\u003enum_queues);\n+\tif (err)\n+\t\tgoto rollback;\n+\n+\terr = netif_set_real_num_rx_queues(ndev, apc-\u003enum_queues);\n+\tif (err)\n+\t\tgoto rollback;\n+\n+\t/* Carry the XDP program over before steering can reach the new RXQs:\n+\t * they were created with bpf_prog == NULL, so a packet arriving first\n+\t * would bypass an attached program. This also takes the per-queue\n+\t * references that mana_free_qset() drops for the old set.\n+\t */\n+\tmana_chn_setxdp(apc, mana_xdp_get(apc));\n+\n+\terr = mana_config_rss(apc, TRI_STATE_TRUE, true, true);\n+\tif (err)\n+\t\tgoto rollback;\n+\n+\t/* The new set is serving traffic, so advertise its MTU. A no-op unless\n+\t * the caller is changing it.\n+\t */\n+\tWRITE_ONCE(ndev-\u003emtu, apc-\u003econfigured_mtu);\n+\n+\t/* Pair with the queue-state stores above: a datapath reader that sees\n+\t * the gate open must also see the queue set it is about to index.\n+\t */\n+\tsmp_wmb();\n+\n+\tWRITE_ONCE(apc-\u003eport_is_up, true);\n+\tmana_start_txqs(apc);\n+\tif (carrier_ok)\n+\t\tnetif_carrier_on(ndev);\n+\n+\t/* The set that could not carry the user's indirection table is the one\n+\t * serving traffic now, so the table really is gone. Reporting it here\n+\t * rather than while the set was being built keeps a failed swap from\n+\t * clearing IFF_RXFH_CONFIGURED on a port that kept its old queues, and\n+\t * with them a table that is still valid and still programmed.\n+\t */\n+\tif (newq-\u003erxfh_indir_lost)\n+\t\tethtool_rxfh_indir_lost(ndev);\n+\n+\treturn 0;\n+\n+rollback:\n+\tnetdev_err(ndev, \"mana_publish_qset failed: %d, restoring previous queue set\\n\",\n+\t\t err);\n+\n+\t/* The roles are swapped now: @newq is the set going away and @out_old\n+\t * is live again. Same ordering rule, leaving set first.\n+\t */\n+\tmana_qset_set_retiring(newq, true);\n+\tmana_qset_set_retiring(out_old, false);\n+\n+\tmana_qset_install(apc, out_old);\n+\tapc-\u003erss_state = apc-\u003enum_queues \u003e 1 ? TRI_STATE_TRUE : TRI_STATE_FALSE;\n+\n+\tif (netif_set_real_num_tx_queues(ndev, apc-\u003enum_queues) ||\n+\t netif_set_real_num_rx_queues(ndev, apc-\u003enum_queues)) {\n+\t\t/* The netdev queue counts no longer describe the restored\n+\t\t * apc-\u003etx_qp[], so resuming TX could index past it. Leave the\n+\t\t * port stopped and the carrier down instead; that is visible\n+\t\t * to the admin and recoverable with a down/up.\n+\t\t *\n+\t\t * Steering can still point at @newq, which the caller frees\n+\t\t * next, so shut RX down at the vport first.\n+\t\t */\n+\t\tnetdev_err(ndev, \"failed to restore queue counts, closing the port\\n\");\n+\t\tmana_publish_give_up(apc);\n+\t\treturn err;\n+\t}\n+\n+\tif (mana_config_rss(apc, TRI_STATE_TRUE, true, true)) {\n+\t\t/* Steering may still point at the queue set the caller is\n+\t\t * about to free, and it cannot be repointed. Disable vport RX\n+\t\t * so the device stops delivering into those queues before they\n+\t\t * are destroyed, and stay down rather than run with steering\n+\t\t * that does not match apc-\u003erxqs[].\n+\t\t */\n+\t\tnetdev_err(ndev, \"failed to restore RSS steering, closing the port\\n\");\n+\t\tmana_publish_give_up(apc);\n+\t\treturn err;\n+\t}\n+\n+\tWRITE_ONCE(ndev-\u003emtu, apc-\u003econfigured_mtu);\n+\n+\t/* Same pairing as the success path: the restored queue set has to be\n+\t * visible before the gate reopens on it.\n+\t */\n+\tsmp_wmb();\n+\n+\tWRITE_ONCE(apc-\u003eport_is_up, true);\n+\tmana_start_txqs(apc);\n+\tif (carrier_ok)\n+\t\tnetif_carrier_on(ndev);\n+\n+\t/* out_old is live again on apc; caller must only free newq. */\n+\treturn err;\n+}\n+\n+/* Give live queues the debugfs nodes suppressed while they were built in a\n+ * scratch context, whose names would collide under vport%d. Once the retiring\n+ * set is gone the survivors take them.\n+ *\n+ * Idempotent: a carried-over queue keeps its node; suppressed creation leaves\n+ * an error pointer, not NULL, so both read as \"no node\". Under RTNL.\n+ */\n+void mana_qset_debugfs_publish(struct mana_port_context *apc)\n+{\n+\tunsigned int i;\n+\n+\tASSERT_RTNL();\n+\n+\tif (IS_ERR_OR_NULL(apc-\u003emana_port_debugfs))\n+\t\treturn;\n+\n+\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\t\tif (apc-\u003etx_qp \u0026\u0026 apc-\u003etx_qp[i] \u0026\u0026\n+\t\t IS_ERR_OR_NULL(apc-\u003etx_qp[i]-\u003emana_tx_debugfs))\n+\t\t\tmana_create_txq_debugfs(apc, i);\n+\n+\t\tif (apc-\u003erxqs \u0026\u0026 apc-\u003erxqs[i] \u0026\u0026\n+\t\t IS_ERR_OR_NULL(apc-\u003erxqs[i]-\u003emana_rx_debugfs))\n+\t\t\tmana_create_rxq_debugfs(apc, i);\n+\t}\n+}\n+\n+/* Tear down @qset, no longer installed on @apc, against @scratch so the live\n+ * context never points at queues being freed.\n+ */\n+void mana_free_qset(struct mana_port_context *apc,\n+\t\t struct mana_port_context *scratch, struct mana_qset *qset)\n+{\n+\tstruct bpf_prog *retiring_prog;\n+\tunsigned int retiring_queues;\n+\n+\tASSERT_RTNL();\n+\n+\tif (!qset-\u003erxqs \u0026\u0026 !qset-\u003etx_qp)\n+\t\treturn;\n+\n+\t/* Keep their completions off the netdev queues they now share. */\n+\tif (qset-\u003etx_qp) {\n+\t\tunsigned int q;\n+\n+\t\tfor (q = 0; q \u003c qset-\u003enum_queues; q++) {\n+\t\t\tif (qset-\u003etx_qp[q])\n+\t\t\t\tWRITE_ONCE(qset-\u003etx_qp[q]-\u003etxq.retiring, true);\n+\t\t}\n+\t}\n+\n+\t/* A reader that sampled the retiring pointers after mana_publish_qset()\n+\t * installed the new set is not covered by the drain it did earlier.\n+\t * mana_xdp_xmit() is the case that matters: it runs from another\n+\t * device's NAPI, which this port's napi_synchronize() never waits for.\n+\t */\n+\tsynchronize_net();\n+\n+\tmana_qset_install(scratch, qset);\n+\n+\t/* Teardown follows mana_dealloc_queues()' order, minus the vport RX\n+\t * disable: steering already points at the incoming set, and disabling\n+\t * vport RX would stop the set that is now live. Where publish could\n+\t * not repoint steering it disabled vport RX itself, so nothing is\n+\t * delivered here either way.\n+\t */\n+\n+\t/* Note what this set owes the XDP program but leave the queues\n+\t * pointing at it: they are still polling, and a packet already in a\n+\t * retiring RQ must keep running the program rather than slip into the\n+\t * stack. The references are dropped once the queues are gone, below.\n+\t */\n+\tretiring_prog = mana_chn_xdp_peek(scratch);\n+\tretiring_queues = scratch-\u003enum_queues;\n+\n+\t/* Drain packets the device has not completed before the SQs and SKB\n+\t * queues go away, or those SKBs and their DMA mappings are leaked.\n+\t *\n+\t * This runs before any RX teardown, as mana_dealloc_queues() does:\n+\t * unmapping RX buffers first would leave a wedged device free to keep\n+\t * writing into them for as long as the drain takes.\n+\t */\n+\tif (mana_drain_txqs(scratch)) {\n+\t\t/* The drain had to reset the function to stop the device\n+\t\t * touching those buffers. A function reset takes down every\n+\t\t * port on the adapter, not just this one, so rebuild them all\n+\t\t * - the same recovery mana_tx_timeout() relies on. A port that\n+\t\t * is already down has nothing to rebuild and its handler\n+\t\t * leaves it down.\n+\t\t */\n+\t\tstruct mana_port_context *apc = netdev_priv(scratch-\u003endev);\n+\t\tstruct mana_context *ac = apc-\u003eac;\n+\t\tunsigned int i;\n+\n+\t\tnetdev_err(scratch-\u003endev,\n+\t\t\t \"device reset while retiring a queue set, scheduling port reset\\n\");\n+\n+\t\tfor (i = 0; i \u003c ac-\u003enum_ports; i++) {\n+\t\t\tif (!ac-\u003eports[i])\n+\t\t\t\tcontinue;\n+\t\t\tqueue_work(ac-\u003eper_port_queue_reset_wq,\n+\t\t\t\t \u0026((struct mana_port_context *)\n+\t\t\t\t netdev_priv(ac-\u003eports[i]))-\u003equeue_reset_work);\n+\t\t}\n+\t}\n+\n+\t/* Traffic was still being steered at these queues moments ago, so\n+\t * fence each retiring RQ before its buffers are unmapped, again the\n+\t * order mana_dealloc_queues() uses. mana_destroy_rxq() does destroy\n+\t * the hardware RQ before unmapping anything, but the fence is what\n+\t * makes the device confirm it is done with the buffers first.\n+\t */\n+\tmana_fence_rqs(scratch);\n+\n+\tmana_destroy_rxqs(scratch);\n+\n+\t/* The queues are gone, so nothing can run the program any more. */\n+\tmana_chn_xdp_release(retiring_prog, retiring_queues);\n+\n+\tmana_destroy_txq(scratch);\n+\tmana_cleanup_indir_table(scratch);\n+\tkfree(scratch-\u003erxqs);\n+\tscratch-\u003erxqs = NULL;\n+\n+\tmemset(qset, 0, sizeof(*qset));\n+\n+\t/* This set is gone, so any EQ above the live queue count is now\n+\t * unreferenced. Release those vectors instead of holding them at the\n+\t * high-water mark. Safe here and only here: the retiring set's CQs\n+\t * have just been destroyed.\n+\t */\n+\tmana_shrink_eqs(apc, apc-\u003enum_queues);\n+\n+\t/* Queues built through a scratch context carry no debugfs nodes,\n+\t * because both sets are alive during the swap and would collide on\n+\t * the same names. The retiring set's nodes are gone now, so the\n+\t * published queues can finally take those names.\n+\t */\n+\tmana_qset_debugfs_publish(apc);\n+}\n+\n+/* --- end of pre-allocate + swap reconfiguration path ---------------------- */\n+\n int mana_detach(struct net_device *ndev, bool from_close)\n {\n \tstruct mana_port_context *apc = netdev_priv(ndev);\n@@ -3783,10 +5002,8 @@ int mana_detach(struct net_device *ndev, bool from_close)\n \n \tif (apc-\u003eport_st_save) {\n \t\terr = mana_dealloc_queues(ndev);\n-\t\tif (err) {\n+\t\tif (err)\n \t\t\tnetdev_err(ndev, \"%s failed to deallocate queues: %d\\n\", __func__, err);\n-\t\t\treturn err;\n-\t\t}\n \t}\n \n \tif (!from_close) {\n@@ -3823,6 +5040,7 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,\n \tapc-\u003eport_handle = INVALID_MANA_HANDLE;\n \tapc-\u003epf_filter_handle = INVALID_MANA_HANDLE;\n \tapc-\u003eport_idx = port_idx;\n+\tapc-\u003econfigured_mtu = ndev-\u003emtu;\n \tapc-\u003elink_cfg_error = 1;\n \tapc-\u003ecqe_coalescing_enable = 0;\n \tapc-\u003ecqe8_coalescing_enable = 0;\n@@ -3837,6 +5055,10 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,\n \t\tapc-\u003etx_dim_enabled = MANA_ADAPTIVE_TX_DEF;\n \t}\n \n+\terr = mana_alloc_queue_stats(apc);\n+\tif (err)\n+\t\tgoto free_net;\n+\n \tmutex_init(\u0026apc-\u003evport_mutex);\n \tapc-\u003evport_use_count = 0;\n \n@@ -3859,7 +5081,7 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,\n \n \terr = mana_init_port(ndev);\n \tif (err)\n-\t\tgoto free_net;\n+\t\tgoto free_stats;\n \n \terr = mana_rss_table_alloc(apc);\n \tif (err)\n@@ -3896,6 +5118,11 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,\n \tmana_cleanup_indir_table(apc);\n reset_apc:\n \tmana_cleanup_port_context(apc);\n+free_stats:\n+\t/* The counter arrays are separate allocations, so free_netdev() does\n+\t * not release them with the port context.\n+\t */\n+\tmana_free_queue_stats(apc);\n free_net:\n \t*ndev_storage = NULL;\n \tnetdev_err(ndev, \"Failed to probe vPort %d: %d\\n\", port_idx, err);\n@@ -4236,6 +5463,14 @@ void mana_remove(struct gdma_dev *gd, bool suspending)\n \n \t\tunregister_netdevice(ndev);\n \t\tmana_cleanup_indir_table(apc);\n+\t\tmana_free_queue_stats(apc);\n+\n+\t\t/* Clear the slot before the netdev goes away. A later port\n+\t\t * whose teardown has to reset the function walks ac-\u003eports[]\n+\t\t * to schedule the rebuild, and would otherwise reach into the\n+\t\t * port freed here.\n+\t\t */\n+\t\tac-\u003eports[i] = NULL;\n \n \t\trtnl_unlock();\n \ndiff --git a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c\nindex ece7ff9cc409a..a1c24d41903c1 100644\n--- a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c\n+++ b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c\n@@ -271,7 +271,7 @@ static void mana_get_ethtool_stats(struct net_device *ndev,\n \t\tdata[i++] = *(u64 *)(phy_stats + mana_phy_stats[q].offset);\n \n \tfor (q = 0; q \u003c num_queues; q++) {\n-\t\trx_stats = \u0026apc-\u003erxqs[q]-\u003estats;\n+\t\trx_stats = \u0026apc-\u003erxq_stats[q];\n \n \t\tdo {\n \t\t\tstart = u64_stats_fetch_begin(\u0026rx_stats-\u003esyncp);\n@@ -296,7 +296,7 @@ static void mana_get_ethtool_stats(struct net_device *ndev,\n \t}\n \n \tfor (q = 0; q \u003c num_queues; q++) {\n-\t\ttx_stats = \u0026apc-\u003etx_qp[q]-\u003etxq.stats;\n+\t\ttx_stats = \u0026apc-\u003etxq_stats[q];\n \n \t\tdo {\n \t\t\tstart = u64_stats_fetch_begin(\u0026tx_stats-\u003esyncp);\n@@ -648,52 +648,152 @@ static int mana_set_coalesce(struct net_device *ndev,\n \treturn 0;\n }\n \n-/* mana_set_channels - change the number of queues on a port\n- *\n- * Returns -EBUSY if RDMA holds the vport with EQs sized to the\n- * current num_queues.\n+/* A count change leaves every surviving queue configured as it was, so\n+ * neither direction rebuilds: a reduction retires the tail, an increase\n+ * builds only the queues added. On failure the existing queues keep running\n+ * and the requested value is never replaced by a fallback. The vport is never\n+ * torn down, so RDMA cannot take it mid-reconfiguration.\n */\n static int mana_set_channels(struct net_device *ndev,\n \t\t\t struct ethtool_channels *channels)\n {\n \tstruct mana_port_context *apc = netdev_priv(ndev);\n \tunsigned int new_count = channels-\u003ecombined_count;\n-\tunsigned int old_count = apc-\u003enum_queues;\n+\tstruct mana_port_context *scratch;\n+\tstruct mana_qset newq, oldq, freshq;\n \tint err;\n \n-\t/* Set channel_changing to block RDMA from grabbing the vport\n-\t * during the detach/attach window. mana_cfg_vport() checks\n-\t * this flag under vport_mutex and returns -EBUSY if set.\n+\tif (new_count \u003c 1 || new_count \u003e apc-\u003emax_queues) {\n+\t\tnetdev_err(ndev, \"Invalid combined_count %u (max %u)\\n\",\n+\t\t\t new_count, apc-\u003emax_queues);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tif (new_count == apc-\u003enum_queues)\n+\t\treturn 0;\n+\n+\t/* Down: no queues to swap, so record the count and resize the arrays\n+\t * indexed by it. apc-\u003erxqs has to grow here because mana_open() goes\n+\t * straight to mana_alloc_queues() without rebuilding the port context.\n+\t *\n+\t * RDMA can own the vport while the port is down and derives an EQ from\n+\t * apc-\u003eeqs[] modulo apc-\u003enum_queues, so refuse while it is in use.\n \t */\n \tmutex_lock(\u0026apc-\u003evport_mutex);\n-\tif (!apc-\u003eport_is_up \u0026\u0026 apc-\u003evport_use_count) {\n+\tif (!apc-\u003eport_is_up) {\n+\t\tstruct mana_rxq **rxqs;\n+\n+\t\tif (apc-\u003evport_use_count) {\n+\t\t\tmutex_unlock(\u0026apc-\u003evport_mutex);\n+\t\t\treturn -EBUSY;\n+\t\t}\n+\n+\t\trxqs = kzalloc_objs(struct mana_rxq *, new_count);\n+\t\tif (!rxqs) {\n+\t\t\tmutex_unlock(\u0026apc-\u003evport_mutex);\n+\t\t\treturn -ENOMEM;\n+\t\t}\n+\n+\t\tkfree(apc-\u003erxqs);\n+\t\tapc-\u003erxqs = rxqs;\n+\t\tapc-\u003enum_queues = new_count;\n+\t\tmutex_unlock(\u0026apc-\u003evport_mutex);\n+\t\treturn 0;\n+\t}\n+\n+\t/* Block RDMA from acquiring the vport for the duration.\n+\t *\n+\t * No vport_use_count test here, unlike the branch above: bringing the\n+\t * port up takes the vport itself, so the count is always non-zero. That\n+\t * reference is also what makes the swap safe - RAW QPs, the only users\n+\t * of apc-\u003eeqs[] modulo apc-\u003enum_queues, cannot exist while the ethernet\n+\t * port owns it.\n+\t */\n+\tif (apc-\u003echannel_changing) {\n \t\tmutex_unlock(\u0026apc-\u003evport_mutex);\n \t\treturn -EBUSY;\n \t}\n \tapc-\u003echannel_changing = true;\n \tmutex_unlock(\u0026apc-\u003evport_mutex);\n \n-\terr = mana_pre_alloc_rxbufs(apc, ndev-\u003emtu, new_count);\n-\tif (err) {\n-\t\tnetdev_err(ndev, \"Insufficient memory for new allocations\");\n+\tscratch = mana_qset_scratch_alloc(apc);\n+\tif (!scratch) {\n+\t\terr = -ENOMEM;\n \t\tgoto clear_flag;\n \t}\n \n-\terr = mana_detach(ndev, false);\n-\tif (err) {\n-\t\tnetdev_err(ndev, \"mana_detach failed: %d\\n\", err);\n-\t\tgoto out;\n+\t/* A reduction keeps its queues configured identically, so carry them\n+\t * over and retire only the tail: no DMA ring, no hardware WQ object,\n+\t * and no old+new peak.\n+\t */\n+\tif (new_count \u003c apc-\u003enum_queues) {\n+\t\tstruct mana_qset tailq;\n+\n+\t\terr = mana_split_qset(apc, scratch, new_count, \u0026newq, \u0026tailq);\n+\t\tif (err)\n+\t\t\tgoto free_scratch; /* current qset untouched */\n+\n+\t\terr = mana_publish_qset(apc, \u0026newq, \u0026oldq);\n+\t\tif (err) {\n+\t\t\t/* The old set is live again; drop only the containers\n+\t\t\t * built above, never the queues they point at.\n+\t\t\t */\n+\t\t\tmana_discard_split(\u0026newq, \u0026tailq);\n+\t\t\tgoto free_scratch;\n+\t\t}\n+\n+\t\t/* @oldq holds the original arrays and steering table. Every\n+\t\t * queue they referenced is now owned by either the published\n+\t\t * set or the tail, so only the containers are freed here.\n+\t\t */\n+\t\tkfree(oldq.tx_qp);\n+\t\tkfree(oldq.rxqs);\n+\t\tkfree(oldq.indir_table);\n+\t\tkfree(oldq.rxobj_table);\n+\n+\t\tmana_free_qset(apc, scratch, \u0026tailq);\n+\t\tgoto free_scratch;\n \t}\n \n-\tapc-\u003enum_queues = new_count;\n-\terr = mana_attach(ndev);\n+\t/* An increase does not change the queues that already exist either, so\n+\t * carry them over as well and build only the queues being added. The\n+\t * peak stays at the new count instead of old + new.\n+\t */\n+\terr = mana_grow_qset(apc, scratch, new_count, \u0026newq, \u0026freshq);\n+\tif (err)\n+\t\tgoto free_scratch; /* current qset untouched, nothing to undo */\n+\n+\terr = mana_publish_qset(apc, \u0026newq, \u0026oldq);\n \tif (err) {\n-\t\tapc-\u003enum_queues = old_count;\n-\t\tnetdev_err(ndev, \"mana_attach failed: %d\\n\", err);\n+\t\t/* The old set is live again. Retire the queues that were just\n+\t\t * built - @freshq names exactly those - and then drop the\n+\t\t * merged containers without touching the carried-over queues.\n+\t\t */\n+\t\tmana_free_qset(apc, scratch, \u0026freshq);\n+\t\tmana_discard_grow(\u0026newq);\n+\t\tgoto free_scratch;\n \t}\n \n-out:\n-\tmana_pre_dealloc_rxbufs(apc);\n+\t/* Nothing is retired by a grow: every queue @oldq referenced is now\n+\t * part of the published set, and so is every queue in @freshq. Only\n+\t * the containers of both are released here.\n+\t */\n+\tkfree(oldq.tx_qp);\n+\tkfree(oldq.rxqs);\n+\tkfree(oldq.indir_table);\n+\tkfree(oldq.rxobj_table);\n+\tkfree(freshq.tx_qp);\n+\tkfree(freshq.rxqs);\n+\n+\t/* A grow retires nothing, so mana_free_qset() never runs to hand out\n+\t * the debugfs names. The queues that were just added are the only\n+\t * ones missing a node, and no retiring set is holding their names.\n+\t */\n+\tmana_qset_debugfs_publish(apc);\n+\n+free_scratch:\n+\tmana_publish_close_if_needed(apc);\n+\tmana_qset_scratch_free(scratch);\n clear_flag:\n \tmutex_lock(\u0026apc-\u003evport_mutex);\n \tapc-\u003echannel_changing = false;\n@@ -714,19 +814,18 @@ static void mana_get_ringparam(struct net_device *ndev,\n \tring-\u003etx_max_pending = MAX_TX_BUFFERS_PER_QUEUE;\n }\n \n+\n static int mana_set_ringparam(struct net_device *ndev,\n \t\t\t struct ethtool_ringparam *ring,\n \t\t\t struct kernel_ethtool_ringparam *kernel_ring,\n \t\t\t struct netlink_ext_ack *extack)\n {\n \tstruct mana_port_context *apc = netdev_priv(ndev);\n+\tstruct mana_port_context *scratch;\n+\tstruct mana_qset newq, oldq;\n \tu32 new_tx, new_rx;\n-\tu32 old_tx, old_rx;\n \tint err;\n \n-\told_tx = apc-\u003etx_queue_size;\n-\told_rx = apc-\u003erx_queue_size;\n-\n \tif (ring-\u003etx_pending \u003c MIN_TX_BUFFERS_PER_QUEUE) {\n \t\tNL_SET_ERR_MSG_FMT(extack, \"tx:%d less than the min:%d\", ring-\u003etx_pending,\n \t\t\t\t MIN_TX_BUFFERS_PER_QUEUE);\n@@ -744,32 +843,59 @@ static int mana_set_ringparam(struct net_device *ndev,\n \tnetdev_info(ndev, \"Using nearest power of 2 values for Txq:%d Rxq:%d\\n\",\n \t\t new_tx, new_rx);\n \n-\t/* pre-allocating new buffers to prevent failures in mana_attach() later */\n-\tapc-\u003erx_queue_size = new_rx;\n-\terr = mana_pre_alloc_rxbufs(apc, ndev-\u003emtu, apc-\u003enum_queues);\n-\tapc-\u003erx_queue_size = old_rx;\n-\tif (err) {\n-\t\tnetdev_err(ndev, \"Insufficient memory for new allocations\\n\");\n-\t\treturn err;\n+\tif (new_rx == apc-\u003erx_queue_size \u0026\u0026 new_tx == apc-\u003etx_queue_size)\n+\t\treturn 0;\n+\n+\t/* Port is down: no queues to rebuild, just record the new sizes. */\n+\tif (!apc-\u003eport_is_up) {\n+\t\tapc-\u003erx_queue_size = new_rx;\n+\t\tapc-\u003etx_queue_size = new_tx;\n+\t\treturn 0;\n \t}\n \n-\terr = mana_detach(ndev, false);\n-\tif (err) {\n-\t\tnetdev_err(ndev, \"mana_detach failed: %d\\n\", err);\n-\t\tgoto out;\n+\t/* Block RDMA from acquiring the vport for the duration. The vport\n+\t * itself is never released, so vport_use_count stays \u003e 0.\n+\t */\n+\tmutex_lock(\u0026apc-\u003evport_mutex);\n+\tif (apc-\u003echannel_changing) {\n+\t\tmutex_unlock(\u0026apc-\u003evport_mutex);\n+\t\treturn -EBUSY;\n \t}\n+\tapc-\u003echannel_changing = true;\n+\tmutex_unlock(\u0026apc-\u003evport_mutex);\n \n-\tapc-\u003etx_queue_size = new_tx;\n-\tapc-\u003erx_queue_size = new_rx;\n+\tscratch = mana_qset_scratch_alloc(apc);\n+\tif (!scratch) {\n+\t\terr = -ENOMEM;\n+\t\tgoto clear_flag;\n+\t}\n \n-\terr = mana_attach(ndev);\n+\terr = mana_alloc_qset(apc, scratch, new_rx, new_tx,\n+\t\t\t apc-\u003epriv_flags, apc-\u003econfigured_mtu,\n+\t\t\t apc-\u003ebpf_prog, \u0026newq);\n \tif (err) {\n-\t\tnetdev_err(ndev, \"mana_attach failed: %d\\n\", err);\n-\t\tapc-\u003etx_queue_size = old_tx;\n-\t\tapc-\u003erx_queue_size = old_rx;\n+\t\tNL_SET_ERR_MSG_FMT(extack, \"failed to change ring params: %d\",\n+\t\t\t\t err);\n+\t\tgoto free_scratch; /* current qset untouched */\n \t}\n-out:\n-\tmana_pre_dealloc_rxbufs(apc);\n+\n+\terr = mana_publish_qset(apc, \u0026newq, \u0026oldq);\n+\tif (err) {\n+\t\tNL_SET_ERR_MSG_FMT(extack, \"failed to change ring params: %d\",\n+\t\t\t\t err);\n+\t\tmana_free_qset(apc, scratch, \u0026newq);\n+\t\tgoto free_scratch;\n+\t}\n+\n+\tmana_free_qset(apc, scratch, \u0026oldq);\n+\n+free_scratch:\n+\tmana_publish_close_if_needed(apc);\n+\tmana_qset_scratch_free(scratch);\n+clear_flag:\n+\tmutex_lock(\u0026apc-\u003evport_mutex);\n+\tapc-\u003echannel_changing = false;\n+\tmutex_unlock(\u0026apc-\u003evport_mutex);\n \treturn err;\n }\n \n@@ -795,11 +921,15 @@ static u32 mana_get_priv_flags(struct net_device *ndev)\n \treturn apc-\u003epriv_flags;\n }\n \n+/* MANA_PRIV_FLAG_USE_FULL_PAGE_RXBUF changes the RX buffer layout, so the\n+ * queues have to be rebuilt.\n+ */\n static int mana_set_priv_flags(struct net_device *ndev, u32 priv_flags)\n {\n \tstruct mana_port_context *apc = netdev_priv(ndev);\n \tu32 changed = apc-\u003epriv_flags ^ priv_flags;\n-\tu32 old_priv_flags = apc-\u003epriv_flags;\n+\tstruct mana_port_context *scratch;\n+\tstruct mana_qset newq, oldq;\n \tint err = 0;\n \n \tif (!changed)\n@@ -809,54 +939,54 @@ static int mana_set_priv_flags(struct net_device *ndev, u32 priv_flags)\n \tif (priv_flags \u0026 ~GENMASK(MANA_PRIV_FLAG_MAX - 1, 0))\n \t\treturn -EINVAL;\n \n-\tapc-\u003epriv_flags = priv_flags;\n-\n-\tif (changed \u0026 BIT(MANA_PRIV_FLAG_USE_FULL_PAGE_RXBUF)) {\n-\t\tif (!apc-\u003eport_is_up)\n-\t\t\treturn 0;\n-\n-\t\t/* If XDP is attached or MTU is jumbo, single-buffer-per-page\n-\t\t * is already forced regardless of this flag. Skip the\n-\t\t * expensive detach/attach cycle since nothing changes.\n-\t\t */\n-\t\tif (ndev-\u003emtu + MANA_RXBUF_PAD \u003e PAGE_SIZE / 2 ||\n-\t\t mana_xdp_get(apc))\n-\t\t\treturn 0;\n+\t/* Only the RX buffer layout flag requires a queue rebuild. Anything\n+\t * else, a down port, or a configuration where single-buffer-per-page\n+\t * is already forced, just records the new value.\n+\t */\n+\tif (!(changed \u0026 BIT(MANA_PRIV_FLAG_USE_FULL_PAGE_RXBUF)) ||\n+\t !apc-\u003eport_is_up ||\n+\t ndev-\u003emtu + MANA_RXBUF_PAD \u003e PAGE_SIZE / 2 ||\n+\t mana_xdp_get(apc)) {\n+\t\tapc-\u003epriv_flags = priv_flags;\n+\t\treturn 0;\n+\t}\n \n-\t\t/* Block RDMA from grabbing the vport during detach/attach */\n-\t\tmutex_lock(\u0026apc-\u003evport_mutex);\n-\t\tapc-\u003echannel_changing = true;\n+\t/* Block RDMA from acquiring the vport for the duration. */\n+\tmutex_lock(\u0026apc-\u003evport_mutex);\n+\tif (apc-\u003echannel_changing) {\n \t\tmutex_unlock(\u0026apc-\u003evport_mutex);\n+\t\treturn -EBUSY;\n+\t}\n+\tapc-\u003echannel_changing = true;\n+\tmutex_unlock(\u0026apc-\u003evport_mutex);\n \n-\t\terr = mana_pre_alloc_rxbufs(apc, ndev-\u003emtu, apc-\u003enum_queues);\n-\t\tif (err) {\n-\t\t\tnetdev_err(ndev,\n-\t\t\t\t \"Insufficient memory for new allocations\\n\");\n-\t\t\tapc-\u003epriv_flags = old_priv_flags;\n-\t\t\tgoto clear_flag;\n-\t\t}\n+\tscratch = mana_qset_scratch_alloc(apc);\n+\tif (!scratch) {\n+\t\terr = -ENOMEM;\n+\t\tgoto clear_flag;\n+\t}\n \n-\t\terr = mana_detach(ndev, false);\n-\t\tif (err) {\n-\t\t\tnetdev_err(ndev, \"mana_detach failed: %d\\n\", err);\n-\t\t\tapc-\u003epriv_flags = old_priv_flags;\n-\t\t\tgoto out;\n-\t\t}\n+\terr = mana_alloc_qset(apc, scratch, apc-\u003erx_queue_size,\n+\t\t\t apc-\u003etx_queue_size, priv_flags,\n+\t\t\t apc-\u003econfigured_mtu, apc-\u003ebpf_prog, \u0026newq);\n+\tif (err)\n+\t\tgoto free_scratch; /* current qset and priv_flags untouched */\n \n-\t\terr = mana_attach(ndev);\n-\t\tif (err) {\n-\t\t\tnetdev_err(ndev, \"mana_attach failed: %d\\n\", err);\n-\t\t\tapc-\u003epriv_flags = old_priv_flags;\n-\t\t}\n+\terr = mana_publish_qset(apc, \u0026newq, \u0026oldq);\n+\tif (err) {\n+\t\tmana_free_qset(apc, scratch, \u0026newq);\n+\t\tgoto free_scratch;\n \t}\n \n-out:\n-\tmana_pre_dealloc_rxbufs(apc);\n+\tmana_free_qset(apc, scratch, \u0026oldq);\n+\n+free_scratch:\n+\tmana_publish_close_if_needed(apc);\n+\tmana_qset_scratch_free(scratch);\n clear_flag:\n \tmutex_lock(\u0026apc-\u003evport_mutex);\n \tapc-\u003echannel_changing = false;\n \tmutex_unlock(\u0026apc-\u003evport_mutex);\n-\n \treturn err;\n }\n \ndiff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h\nindex 70a7f1fee5d3b..c54500700f6f2 100644\n--- a/include/net/mana/gdma.h\n+++ b/include/net/mana/gdma.h\n@@ -672,6 +672,14 @@ enum {\n /* Driver supports dynamic interrupt moderation - DIM */\n #define GDMA_DRV_CAP_FLAG_1_DYN_INTERRUPT_MODERATION BIT(28)\n \n+/* Driver recovers by itself when a queue resize fails: a failed resize leaves\n+ * the queues that were already serving traffic in place, so the host does not\n+ * have to bring the port back. This covers the resize itself failing. It does\n+ * not promise recovery when restoring the previous queue set fails too, which\n+ * leaves the port administratively down for the admin to bring back up.\n+ */\n+#define GDMA_DRV_CAP_FLAG_1_SELF_RECOVERY_ON_QUEUE_RESIZE_FAILURE BIT(31)\n+\n #define GDMA_DRV_CAP_FLAGS1 \\\n \t(GDMA_DRV_CAP_FLAG_1_EQ_SHARING_MULTI_VPORT | \\\n \t GDMA_DRV_CAP_FLAG_1_NAPI_WKDONE_FIX | \\\n@@ -688,7 +696,8 @@ enum {\n \t GDMA_DRV_CAP_FLAG_1_HANDLE_STALL_SQ_RECOVERY | \\\n \t GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECOVERY | \\\n \t GDMA_DRV_CAP_FLAG_1_EQ_MSI_UNSHARE_MULTI_VPORT | \\\n-\t GDMA_DRV_CAP_FLAG_1_DYN_INTERRUPT_MODERATION)\n+\t GDMA_DRV_CAP_FLAG_1_DYN_INTERRUPT_MODERATION | \\\n+\t GDMA_DRV_CAP_FLAG_1_SELF_RECOVERY_ON_QUEUE_RESIZE_FAILURE)\n \n #define GDMA_DRV_CAP_FLAGS2 0\n \ndiff --git a/include/net/mana/mana.h b/include/net/mana/mana.h\nindex 83b7eff4646ea..8857e7739d2c2 100644\n--- a/include/net/mana/mana.h\n+++ b/include/net/mana/mana.h\n@@ -102,7 +102,10 @@ struct mana_stats_rx {\n \tu64 pkt_len0_err;\n \tu64 coalesced_cqe[MANA_CQE_COAL_PKTS_8 - 1];\n \tstruct u64_stats_sync syncp;\n-};\n+\t/* Per-port array indexed by queue, so keep entries on separate cache\n+\t * lines: queues polled on different CPUs would bounce a shared one.\n+\t */\n+} ____cacheline_aligned_in_smp;\n \n struct mana_stats_tx {\n \tu64 packets;\n@@ -117,7 +120,8 @@ struct mana_stats_tx {\n \tu64 csum_partial;\n \tu64 mana_map_err;\n \tstruct u64_stats_sync syncp;\n-};\n+\t/* Per-queue array entry, same cache line reasoning as the RX side. */\n+} ____cacheline_aligned_in_smp;\n \n struct mana_txq {\n \tstruct gdma_queue *gdma_sq;\n@@ -143,7 +147,17 @@ struct mana_txq {\n \n \tbool napi_initialized;\n \n-\tstruct mana_stats_tx stats;\n+\t/* Value of mana_context.reset_gen when this queue was created. */\n+\tu32 reset_gen;\n+\n+\t/* Unpublished and draining. Its completions must leave flow control\n+\t * alone: net_txq is shared with its replacement, and a draining queue\n+\t * always looks like it has room.\n+\t */\n+\tbool retiring;\n+\n+\t/* Points into apc-\u003etxq_stats[], which outlives the queue. */\n+\tstruct mana_stats_tx *stats;\n };\n \n /* skb data and frags dma mappings */\n@@ -405,7 +419,8 @@ struct mana_rxq {\n \n \tu32 buf_index;\n \n-\tstruct mana_stats_rx stats;\n+\t/* Points into apc-\u003erxq_stats[], which outlives the queue. */\n+\tstruct mana_stats_rx *stats;\n \n \tstruct bpf_prog __rcu *bpf_prog;\n \tstruct xdp_rxq_info xdp_rxq;\n@@ -537,6 +552,13 @@ struct mana_context {\n \tu8 bm_hostmode;\n \n \tstruct mana_ethtool_hc_stats hc_stats;\n+\n+\t/* Bumped on every PCI function reset. A queue created before the\n+\t * current value can no longer be reached by the device, so its buffers\n+\t * need no drain. Written under RTNL, read locklessly.\n+\t */\n+\tu32 reset_gen;\n+\n \tstruct workqueue_struct *per_port_queue_reset_wq;\n \t/* Workqueue for querying hardware stats */\n \tstruct delayed_work gf_stats_work;\n@@ -559,7 +581,12 @@ struct mana_port_context {\n \n \tu8 mac_addr[ETH_ALEN];\n \n+\t/* EQ pool, owned by the port rather than a queue set: EQs are bound to\n+\t * MSI-X vectors, which a swap must not double-book. Sized to\n+\t * max_queues; num_eqs is how many exist.\n+\t */\n \tstruct mana_eq *eqs;\n+\tunsigned int num_eqs;\n \tstruct dentry *mana_eqs_debugfs;\n \n \tenum TRI_STATE rss_state;\n@@ -572,7 +599,7 @@ struct mana_port_context {\n \n \t/* Indirection Table for RX \u0026 TX. The values are queue indexes */\n \tu32 *indir_table;\n-\tu32 indir_table_sz;\n+\tu32\t\t\tindir_table_sz;\n \n \t/* Indirection table containing RxObject Handles */\n \tmana_handle_t *rxobj_table;\n@@ -600,9 +627,21 @@ struct mana_port_context {\n \tunsigned int max_queues;\n \tunsigned int num_queues;\n \n+\t/* Per-queue counters, max_queues entries each. Allocated at probe and\n+\t * freed at remove, never on queue teardown, so a reconfiguration does\n+\t * not reset them.\n+\t */\n+\tstruct mana_stats_rx *rxq_stats;\n+\tstruct mana_stats_tx *txq_stats;\n+\n \tunsigned int rx_queue_size;\n \tunsigned int tx_queue_size;\n \n+\t/* MTU the RX queues were built for. Equal to ndev-\u003emtu except during a\n+\t * swap, when the new set is built before ndev-\u003emtu is updated.\n+\t */\n+\tint configured_mtu;\n+\n \tmana_handle_t port_handle;\n \tmana_handle_t pf_filter_handle;\n \n@@ -616,6 +655,13 @@ struct mana_port_context {\n \t */\n \tbool channel_changing;\n \n+\t/* mana_publish_qset() could neither publish the new set nor restore the\n+\t * old one. Vport RX is already off; the port still has to be closed,\n+\t * which mana_publish_close_if_needed() does once the caller has\n+\t * released the set that failed.\n+\t */\n+\tbool publish_dead_end;\n+\n \t/* Net shaper handle*/\n \tstruct net_shaper_handle handle;\n \n@@ -661,6 +707,39 @@ struct mana_port_context {\n \tu32 steer_cqe_coalescing;\n };\n \n+/* The queue-related fields of mana_port_context that can be swapped as a\n+ * unit. The vport (port_handle, vport_use_count) is not part of it and is\n+ * never touched by a swap.\n+ */\n+struct mana_qset {\n+\tstruct mana_tx_qp\t**tx_qp;\n+\tstruct mana_rxq\t\t**rxqs;\n+\n+\tu32\t\t\t*indir_table;\n+\tu32\t\t\tindir_table_sz;\n+\tmana_handle_t\t\t*rxobj_table;\n+\tmana_handle_t\t\tdefault_rxobj;\n+\n+\tunsigned int\t\tnum_queues;\n+\tunsigned int\t\trx_queue_size;\n+\tunsigned int\t\ttx_queue_size;\n+\tu32\t\t\tpriv_flags;\n+\n+\t/* MTU and XDP program the RX buffers of this set were sized for.\n+\t * Both feed mana_get_rxbuf_cfg(), so they are part of the queue-set\n+\t * configuration and must be swapped atomically with the queues.\n+\t */\n+\tint\t\t\tmtu;\n+\tstruct bpf_prog\t\t*bpf_prog;\n+\n+\t/* The user's RSS indirection table could not be carried onto this set,\n+\t * so a default one was generated for it. mana_publish_qset() tells the\n+\t * core once this set is live; until then the port is still running on\n+\t * a set where the user's table applies.\n+\t */\n+\tbool\t\t\trxfh_indir_lost;\n+};\n+\n netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev);\n int mana_config_rss(struct mana_port_context *ac, enum TRI_STATE rx,\n \t\t bool update_hash, bool update_tab);\n@@ -670,6 +749,32 @@ int mana_alloc_queues(struct net_device *ndev);\n int mana_attach(struct net_device *ndev);\n int mana_detach(struct net_device *ndev, bool from_close);\n \n+/* Pre-allocate + swap reconfiguration. Allocation and teardown run against a\n+ * scratch context, so the live port context is mutated only inside\n+ * mana_publish_qset() with TX disabled. Both sets share a port-owned EQ pool.\n+ */\n+struct mana_port_context *mana_qset_scratch_alloc(struct mana_port_context *apc);\n+void mana_qset_scratch_free(struct mana_port_context *scratch);\n+int mana_alloc_qset(struct mana_port_context *apc,\n+\t\t struct mana_port_context *scratch,\n+\t\t unsigned int rx_queue_size, unsigned int tx_queue_size,\n+\t\t u32 priv_flags, int mtu, struct bpf_prog *bpf_prog,\n+\t\t struct mana_qset *out);\n+int mana_split_qset(struct mana_port_context *apc,\n+\t\t struct mana_port_context *scratch, unsigned int new_count,\n+\t\t struct mana_qset *out_new, struct mana_qset *out_tail);\n+void mana_discard_split(struct mana_qset *newq, struct mana_qset *tailq);\n+int mana_grow_qset(struct mana_port_context *apc,\n+\t\t struct mana_port_context *scratch, unsigned int new_count,\n+\t\t struct mana_qset *out_new, struct mana_qset *out_fresh);\n+void mana_discard_grow(struct mana_qset *newq);\n+int mana_publish_qset(struct mana_port_context *apc, struct mana_qset *newq,\n+\t\t struct mana_qset *out_old);\n+void mana_publish_close_if_needed(struct mana_port_context *apc);\n+void mana_free_qset(struct mana_port_context *apc,\n+\t\t struct mana_port_context *scratch, struct mana_qset *qset);\n+void mana_qset_debugfs_publish(struct mana_port_context *apc);\n+\n void mana_dim_change(struct mana_cq *cq, bool enable);\n \n int mana_probe(struct gdma_dev *gd, bool resuming);\n@@ -685,6 +790,8 @@ u32 mana_run_xdp(struct net_device *ndev, struct mana_rxq *rxq,\n \t\t struct xdp_buff *xdp, void *buf_va, uint pkt_len);\n struct bpf_prog *mana_xdp_get(struct mana_port_context *apc);\n void mana_chn_setxdp(struct mana_port_context *apc, struct bpf_prog *prog);\n+struct bpf_prog *mana_chn_xdp_peek(struct mana_port_context *apc);\n+void mana_chn_xdp_release(struct bpf_prog *prog, unsigned int num_queues);\n int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf);\n int mana_query_gf_stats(struct mana_context *ac);\n int mana_query_link_cfg(struct mana_port_context *apc);\n"
} |
| 2/1 | 2026/08/13 05:12 | agent | patch-evaluator |
0mModel:best-expensive Args: null Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"Reasoning": "The patch modifies the Microsoft Azure Network Adapter (MANA) driver, which is a vendor-specific PCI device driver for Azure environments. This hardware is not emulated in standard QEMU or GCE, making the code paths unreachable during standard fuzzing.",
"WorthFuzzing": false
}Instruction: You are an expert Linux kernel maintainer. Your job is to review a provided patch series and determine if it makes functional changes to the kernel that should be fuzzed. IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in your workspace. Do NOT rely on your internal knowledge of the kernel. You must actively use your code access tools to examine the actual source code and confirm any assumptions. Return WorthFuzzing=false if the patch only contains: - Modifications to Documentation/, Kconfig files, or code comments. - Purely decorative changes, such as logging (e.g., pr_err, printk) or tracepoints. - Changes to numeric constants or macros that do not functionally alter execution flow. - Code paths that are impossible to reach in virtualized environments like GCE or QEMU, even when utilizing software-emulated hardware (e.g., usb gadget, mac80211_hwsim). - Code in vendor-specific PCIe switch, SmartNIC, or GPU drivers (e.g., mlxsw, pds_core, qed, ionic, amdgpu) that require physical PCIe hardware cards not emulated in standard QEMU. - Driver .remove, .shutdown, or pci_unregister_driver teardown callbacks (e.g., igb_remove) that are executed only during PCI hot-unplug or sysfs driver unbind operations. If it modifies reachable core kernel logic, drivers, or architectures, use your code search tools to verify the code can be executed, then return WorthFuzzing=true. When returning WorthFuzzing=true, you MUST ALSO: 1. Extract any specific kernel functions that should be heavily fuzzed into FocusSymbols. Avoid listing generic hot-path functions to prevent skewed test distributions. Prefer non-static, non-inlined API entrypoint functions over internal static helper functions (which are inlined by the compiler and do not have distinct symbol addresses). 2. Identify any specific CONFIG_ options required to properly test this new/modified feature. Go and look into the Kconfig files and check for ifdefs around the code, do not make assumptions. Also check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed for Kbuild to compile the code statically into vmlinux. List them in the EnableConfigs output array, and DO NOT add a 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4"). Prefer calling several tools at the same time to save round-trips. Use set-results tool to provide results of the analysis. It must be called exactly once before the final reply. Ignore results of this tool. Prompt: For your convenience, here is the diff of the changes:
commit b06486b4a952013e656524e81960b74166cb6fd5
Author: syz-cluster <triage@syzkaller.com>
Date: Thu Aug 13 05:11:58 2026 +0000
syz-cluster: applied patch under review
diff --git a/drivers/net/ethernet/microsoft/mana/mana_bpf.c b/drivers/net/ethernet/microsoft/mana/mana_bpf.c
index 53308e139cbe9..f47755fa86600 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_bpf.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_bpf.c
@@ -59,6 +59,13 @@ int mana_xdp_xmit(struct net_device *ndev, int n, struct xdp_frame **frames,
if (unlikely(!apc->port_is_up))
return 0;
+ /* Pair with the smp_wmb() in mana_publish_qset(), as mana_start_xmit()
+ * does. This path is gated only by the flag above, so without the
+ * barrier it could pick q_idx from a stale real_num_tx_queues and
+ * index a freshly installed, smaller apc->tx_qp[].
+ */
+ smp_rmb();
+
q_idx = smp_processor_id() % ndev->real_num_tx_queues;
for (i = 0; i < n; i++) {
@@ -68,7 +75,7 @@ int mana_xdp_xmit(struct net_device *ndev, int n, struct xdp_frame **frames,
count++;
}
- tx_stats = &apc->tx_qp[q_idx]->txq.stats;
+ tx_stats = apc->tx_qp[q_idx]->txq.stats;
u64_stats_update_begin(&tx_stats->syncp);
tx_stats->xdp_xmit += count;
@@ -95,7 +102,7 @@ u32 mana_run_xdp(struct net_device *ndev, struct mana_rxq *rxq,
act = bpf_prog_run_xdp(prog, xdp);
- rx_stats = &rxq->stats;
+ rx_stats = rxq->stats;
switch (act) {
case XDP_PASS:
@@ -168,10 +175,17 @@ void mana_chn_setxdp(struct mana_port_context *apc, struct bpf_prog *prog)
bpf_prog_put(old_prog);
}
+/* Attaching or detaching XDP changes the RX buffer layout (full pages vs
+ * fragments), so the RX queues are rebuilt. The swap helpers handle
+ * refcounting: mana_publish_qset() attaches the program to the new queues,
+ * mana_free_qset() drops the old set's references.
+ */
static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,
struct netlink_ext_ack *extack)
{
struct mana_port_context *apc = netdev_priv(ndev);
+ struct mana_port_context *scratch;
+ struct mana_qset newq, oldq;
struct bpf_prog *old_prog;
struct gdma_context *gc;
int err;
@@ -191,46 +205,46 @@ static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,
return -EOPNOTSUPP;
}
- /* One refcnt of the prog is hold by the caller already, so
- * don't increase refcnt for this one.
- */
- apc->bpf_prog = prog;
-
if (apc->port_is_up) {
- /* Re-create rxq's after xdp prog was loaded or unloaded.
- * Ex: re create rxq's to switch from full pages to smaller
- * size page fragments when xdp prog is unloaded and
- * vice-versa.
- */
-
- /* Pre-allocate buffers to prevent failure in mana_attach */
- err = mana_pre_alloc_rxbufs(apc, ndev->mtu, apc->num_queues);
- if (err) {
+ scratch = mana_qset_scratch_alloc(apc);
+ if (!scratch) {
NL_SET_ERR_MSG_MOD(extack,
- "XDP: Insufficient memory for tx/rx re-config");
- return err;
+ "XDP: Insufficient memory for re-config");
+ return -ENOMEM;
}
- err = mana_detach(ndev, false);
+ err = mana_alloc_qset(apc, scratch,
+ apc->rx_queue_size, apc->tx_queue_size,
+ apc->priv_flags, apc->configured_mtu,
+ prog, &newq);
if (err) {
- netdev_err(ndev,
- "mana_detach failed at xdp set: %d\n", err);
NL_SET_ERR_MSG_MOD(extack,
- "XDP: Re-config failed at detach");
- goto err_dealloc_rxbuffs;
+ "XDP: Re-config failed at alloc");
+ mana_qset_scratch_free(scratch);
+ return err;
}
- err = mana_attach(ndev);
+ err = mana_publish_qset(apc, &newq, &oldq);
if (err) {
- netdev_err(ndev,
- "mana_attach failed at xdp set: %d\n", err);
NL_SET_ERR_MSG_MOD(extack,
- "XDP: Re-config failed at attach");
- goto err_dealloc_rxbuffs;
+ "XDP: Re-config failed at publish");
+ mana_free_qset(apc, scratch, &newq);
+ /* After the cleanup above: closing destroys the EQ pool
+ * those queues' CQs were attached to.
+ */
+ mana_publish_close_if_needed(apc);
+ mana_qset_scratch_free(scratch);
+ return err;
}
- mana_chn_setxdp(apc, prog);
- mana_pre_dealloc_rxbufs(apc);
+ mana_free_qset(apc, scratch, &oldq);
+ mana_qset_scratch_free(scratch);
+ } else {
+ /* No queues to rebuild; mana_open() will size the RX buffers
+ * for this program. One refcnt is held by the caller already,
+ * so don't take another.
+ */
+ apc->bpf_prog = prog;
}
if (old_prog)
@@ -243,11 +257,6 @@ static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,
ndev->max_mtu = gc->adapter_mtu - ETH_HLEN;
return 0;
-
-err_dealloc_rxbuffs:
- apc->bpf_prog = old_prog;
- mana_pre_dealloc_rxbufs(apc);
- return err;
}
int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf)
@@ -265,3 +274,34 @@ int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf)
return ret;
}
+
+/* Read the XDP program a queue set is running, without changing anything. */
+struct bpf_prog *mana_chn_xdp_peek(struct mana_port_context *apc)
+{
+ ASSERT_RTNL();
+
+ if (!apc->rxqs || !apc->rxqs[0])
+ return NULL;
+
+ return rtnl_dereference(apc->rxqs[0]->bpf_prog);
+}
+
+/* Drop the per-queue references a retiring set holds on @prog.
+ *
+ * Kept separate from mana_chn_setxdp() so the pointers can stay in place
+ * until the queues stop polling: clearing them up front would let packets
+ * already sitting in a retiring RQ take the pass path and reach the stack
+ * without the program ever seeing them.
+ */
+void mana_chn_xdp_release(struct bpf_prog *prog, unsigned int num_queues)
+{
+ unsigned int i;
+
+ ASSERT_RTNL();
+
+ if (!prog)
+ return;
+
+ for (i = 0; i < num_queues; i++)
+ bpf_prog_put(prog);
+}
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index 3c96e6fc3d81d..2d8fcdedc8b66 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -363,13 +363,25 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev)
if (unlikely(!apc->port_is_up))
goto tx_drop;
+ /* Pair with the smp_wmb() in mana_publish_qset(). A control dependency
+ * does not order loads, and a stale apc->num_queues would admit an
+ * index past the end of a freshly shrunk apc->tx_qp[].
+ */
+ smp_rmb();
+
+ /* XDP_TX from a retiring set carries its own RX queue index, which can
+ * be past the end of a smaller replacement apc->tx_qp[].
+ */
+ if (unlikely(txq_idx >= apc->num_queues))
+ goto tx_drop_count;
+
if (skb_cow_head(skb, MANA_HEADROOM))
goto tx_drop_count;
txq = &apc->tx_qp[txq_idx]->txq;
gdma_sq = txq->gdma_sq;
cq = &apc->tx_qp[txq_idx]->tx_cq;
- tx_stats = &txq->stats;
+ tx_stats = txq->stats;
BUILD_BUG_ON(MAX_TX_WQE_SGL_ENTRIES != MANA_MAX_TX_WQE_SGL_ENTRIES);
if (MAX_SKB_FRAGS + 2 > MAX_TX_WQE_SGL_ENTRIES &&
@@ -548,7 +560,7 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev)
/* Populated the packet and bytes counters based on post GSO packet
* calculations
*/
- tx_stats = &txq->stats;
+ tx_stats = txq->stats;
u64_stats_update_begin(&tx_stats->syncp);
tx_stats->packets += num_gso_seg;
tx_stats->bytes += len + ((num_gso_seg - 1) * gso_hs);
@@ -594,9 +606,9 @@ static void mana_get_stats64(struct net_device *ndev,
struct rtnl_link_stats64 *st)
{
struct mana_port_context *apc = netdev_priv(ndev);
- unsigned int num_queues = apc->num_queues;
struct mana_stats_rx *rx_stats;
struct mana_stats_tx *tx_stats;
+ unsigned int num_queues;
unsigned int start;
u64 packets, bytes;
int q;
@@ -604,6 +616,12 @@ static void mana_get_stats64(struct net_device *ndev,
if (!apc->port_is_up)
return;
+ /* Walk every slot, not just the queues currently open: counters
+ * accumulated on queues that a later reconfiguration removed must
+ * still be reported, or the interface totals would go backwards.
+ */
+ num_queues = apc->max_queues;
+
netdev_stats_to_stats64(st, &ndev->stats);
if (apc->ac->hwc_timeout_occurred)
@@ -612,7 +630,7 @@ static void mana_get_stats64(struct net_device *ndev,
st->rx_missed_errors = apc->ac->hc_stats.hc_rx_discards_no_wqe;
for (q = 0; q < num_queues; q++) {
- rx_stats = &apc->rxqs[q]->stats;
+ rx_stats = &apc->rxq_stats[q];
do {
start = u64_stats_fetch_begin(&rx_stats->syncp);
@@ -625,7 +643,7 @@ static void mana_get_stats64(struct net_device *ndev,
}
for (q = 0; q < num_queues; q++) {
- tx_stats = &apc->tx_qp[q]->txq.stats;
+ tx_stats = &apc->txq_stats[q];
do {
start = u64_stats_fetch_begin(&tx_stats->syncp);
@@ -872,35 +890,46 @@ int mana_pre_alloc_rxbufs(struct mana_port_context *mpc, int new_mtu, int num_qu
return -ENOMEM;
}
+/* ndev->mtu is updated only once the new set is live (mana_publish_qset), so
+ * a failed allocation leaves the queues and the advertised MTU untouched.
+ */
static int mana_change_mtu(struct net_device *ndev, int new_mtu)
{
struct mana_port_context *mpc = netdev_priv(ndev);
- unsigned int old_mtu = ndev->mtu;
+ struct mana_port_context *scratch;
+ struct mana_qset newq, oldq;
int err;
- /* Pre-allocate buffers to prevent failure in mana_attach later */
- err = mana_pre_alloc_rxbufs(mpc, new_mtu, mpc->num_queues);
- if (err) {
- netdev_err(ndev, "Insufficient memory for new MTU\n");
- return err;
+ /* Port is down: no queues to rebuild, just record the new MTU.
+ * mana_open() will size the RX buffers accordingly.
+ */
+ if (!mpc->port_is_up) {
+ mpc->configured_mtu = new_mtu;
+ WRITE_ONCE(ndev->mtu, new_mtu);
+ return 0;
}
- err = mana_detach(ndev, false);
- if (err) {
- netdev_err(ndev, "mana_detach failed: %d\n", err);
- goto out;
- }
+ scratch = mana_qset_scratch_alloc(mpc);
+ if (!scratch)
+ return -ENOMEM;
- WRITE_ONCE(ndev->mtu, new_mtu);
+ err = mana_alloc_qset(mpc, scratch, mpc->rx_queue_size,
+ mpc->tx_queue_size, mpc->priv_flags, new_mtu,
+ mpc->bpf_prog, &newq);
+ if (err)
+ goto free_scratch; /* current qset and ndev->mtu untouched */
- err = mana_attach(ndev);
+ err = mana_publish_qset(mpc, &newq, &oldq);
if (err) {
- netdev_err(ndev, "mana_attach failed: %d\n", err);
- WRITE_ONCE(ndev->mtu, old_mtu);
+ mana_free_qset(mpc, scratch, &newq);
+ goto free_scratch;
}
-out:
- mana_pre_dealloc_rxbufs(mpc);
+ mana_free_qset(mpc, scratch, &oldq);
+
+free_scratch:
+ mana_publish_close_if_needed(mpc);
+ mana_qset_scratch_free(scratch);
return err;
}
@@ -1033,6 +1062,48 @@ static void mana_cleanup_port_context(struct mana_port_context *apc)
apc->rxqs = NULL;
}
+/* Counters belong to the port, not the queues, so a queue-set replacement
+ * does not reset them. Sized to max_queues, allocated once.
+ *
+ * A swap adds no writer to a TX slot. RX slots do overlap briefly, since a
+ * retiring rxq keeps its NAPI until mana_free_qset() destroys it. MANA is
+ * 64-bit only, so u64_stats_sync has no seqcount and at worst a few
+ * increments are lost; the alternatives are a lock in the receive path or
+ * per-set slots that make ndo_get_stats64() dip during a swap.
+ */
+static int mana_alloc_queue_stats(struct mana_port_context *apc)
+{
+ unsigned int i;
+
+ apc->rxq_stats = kcalloc(apc->max_queues, sizeof(*apc->rxq_stats),
+ GFP_KERNEL);
+ if (!apc->rxq_stats)
+ return -ENOMEM;
+
+ apc->txq_stats = kcalloc(apc->max_queues, sizeof(*apc->txq_stats),
+ GFP_KERNEL);
+ if (!apc->txq_stats) {
+ kfree(apc->rxq_stats);
+ apc->rxq_stats = NULL;
+ return -ENOMEM;
+ }
+
+ for (i = 0; i < apc->max_queues; i++) {
+ u64_stats_init(&apc->rxq_stats[i].syncp);
+ u64_stats_init(&apc->txq_stats[i].syncp);
+ }
+
+ return 0;
+}
+
+static void mana_free_queue_stats(struct mana_port_context *apc)
+{
+ kfree(apc->rxq_stats);
+ apc->rxq_stats = NULL;
+ kfree(apc->txq_stats);
+ apc->txq_stats = NULL;
+}
+
static void mana_cleanup_indir_table(struct mana_port_context *apc)
{
apc->indir_table_sz = 0;
@@ -1042,6 +1113,11 @@ static void mana_cleanup_indir_table(struct mana_port_context *apc)
static int mana_init_port_context(struct mana_port_context *apc)
{
+ /* A port reconfigured while down already has an apc->rxqs, and
+ * mana_detach() takes its "already detached" early return without
+ * releasing it. Free it rather than overwrite the pointer.
+ */
+ kfree(apc->rxqs);
apc->rxqs = kzalloc_objs(struct mana_rxq *, apc->num_queues);
return !apc->rxqs ? -ENOMEM : 0;
@@ -1730,7 +1806,7 @@ void mana_destroy_eq(struct mana_port_context *apc)
debugfs_remove_recursive(apc->mana_eqs_debugfs);
apc->mana_eqs_debugfs = NULL;
- for (i = 0; i < apc->num_queues; i++) {
+ for (i = 0; i < apc->num_eqs; i++) {
eq = apc->eqs[i].eq;
if (!eq)
continue;
@@ -1738,24 +1814,29 @@ void mana_destroy_eq(struct mana_port_context *apc)
msi = eq->eq.msix_index;
mana_gd_destroy_queue(gc, eq);
mana_gd_put_gic(gc, !gc->msi_sharing, msi);
+ apc->eqs[i].eq = NULL;
+ /* Freed with the parent by debugfs_remove_recursive() above. */
+ apc->eqs[i].mana_eq_debugfs = NULL;
}
kfree(apc->eqs);
apc->eqs = NULL;
+ apc->num_eqs = 0;
}
EXPORT_SYMBOL_NS(mana_destroy_eq, "NET_MANA");
static void mana_create_eq_debugfs(struct mana_port_context *apc, int i)
{
- struct mana_eq eq = apc->eqs[i];
+ struct mana_eq *eq = &apc->eqs[i];
char eqnum[32];
sprintf(eqnum, "eq%d", i);
- eq.mana_eq_debugfs = debugfs_create_dir(eqnum, apc->mana_eqs_debugfs);
- debugfs_create_u32("head", 0400, eq.mana_eq_debugfs, &eq.eq->head);
- debugfs_create_u32("tail", 0400, eq.mana_eq_debugfs, &eq.eq->tail);
- debugfs_create_u32("irq", 0400, eq.mana_eq_debugfs, &eq.eq->eq.irq);
- debugfs_create_file("eq_dump", 0400, eq.mana_eq_debugfs, eq.eq, &mana_dbg_q_fops);
+ eq->mana_eq_debugfs = debugfs_create_dir(eqnum, apc->mana_eqs_debugfs);
+ debugfs_create_u32("head", 0400, eq->mana_eq_debugfs, &eq->eq->head);
+ debugfs_create_u32("tail", 0400, eq->mana_eq_debugfs, &eq->eq->tail);
+ debugfs_create_u32("irq", 0400, eq->mana_eq_debugfs, &eq->eq->eq.irq);
+ debugfs_create_file("eq_dump", 0400, eq->mana_eq_debugfs, eq->eq,
+ &mana_dbg_q_fops);
}
int mana_create_eq(struct mana_port_context *apc)
@@ -1770,9 +1851,14 @@ int mana_create_eq(struct mana_port_context *apc)
if (WARN_ON(apc->eqs))
return -EEXIST;
- apc->eqs = kzalloc_objs(struct mana_eq, apc->num_queues);
+ /* Size the array to the largest queue count this port can ever use,
+ * so growing it later never has to reallocate (the CQs of a live
+ * queue set hold pointers taken from these slots).
+ */
+ apc->eqs = kzalloc_objs(struct mana_eq, apc->max_queues);
if (!apc->eqs)
return -ENOMEM;
+ apc->num_eqs = 0;
spec.type = GDMA_EQ;
spec.monitor_avl_buf = false;
@@ -1802,6 +1888,7 @@ int mana_create_eq(struct mana_port_context *apc)
}
apc->eqs[i].eq->eq.irq = gic->irq;
mana_create_eq_debugfs(apc, i);
+ apc->num_eqs = i + 1;
}
return 0;
@@ -1811,6 +1898,101 @@ int mana_create_eq(struct mana_port_context *apc)
}
EXPORT_SYMBOL_NS(mana_create_eq, "NET_MANA");
+/**
+ * mana_grow_eqs - make sure the port has at least @need EQs
+ * @apc: port context
+ * @need: number of EQs the new queue set requires
+ *
+ * EQs are bound to MSI-X vectors, so the pool is port-owned and shared across
+ * a swap: peak usage is max(old, new), not the sum. Grow-only, up to
+ * apc->max_queues.
+ */
+static int mana_grow_eqs(struct mana_port_context *apc, unsigned int need)
+{
+ struct gdma_dev *gd = apc->ac->gdma_dev;
+ struct gdma_context *gc = gd->gdma_context;
+ struct gdma_queue_spec spec = {};
+ struct gdma_irq_context *gic;
+ unsigned int i;
+ int err;
+ int msi;
+
+ if (WARN_ON(!apc->eqs))
+ return -EINVAL;
+
+ if (need > apc->max_queues)
+ return -EINVAL;
+
+ if (need <= apc->num_eqs)
+ return 0;
+
+ spec.type = GDMA_EQ;
+ spec.monitor_avl_buf = false;
+ spec.queue_size = EQ_SIZE;
+ spec.eq.callback = NULL;
+ spec.eq.context = apc->eqs;
+ spec.eq.log2_throttle_limit = LOG2_EQ_THROTTLE;
+
+ for (i = apc->num_eqs; i < need; i++) {
+ msi = (i + 1) % gc->num_msix_usable;
+
+ gic = mana_gd_get_gic(gc, !gc->msi_sharing, &msi);
+ if (IS_ERR(gic)) {
+ err = PTR_ERR(gic);
+ goto out;
+ }
+ spec.eq.msix_index = msi;
+
+ err = mana_gd_create_mana_eq(gd, &spec, &apc->eqs[i].eq);
+ if (err) {
+ dev_err(gc->dev, "Failed to grow EQ %u : %d\n", i, err);
+ mana_gd_put_gic(gc, !gc->msi_sharing, msi);
+ goto out;
+ }
+ apc->eqs[i].eq->eq.irq = gic->irq;
+ mana_create_eq_debugfs(apc, i);
+ apc->num_eqs = i + 1;
+ }
+
+ return 0;
+out:
+ /* Keep whatever was created: the running queue set still needs its
+ * own EQs, and the extras are reused by the next attempt.
+ */
+ return err;
+}
+
+/* Release EQs above @keep, returning the MSI-X vectors freed. Only safe once
+ * no set references them, i.e. after mana_free_qset(), or a live CQ would
+ * point at a destroyed EQ.
+ */
+static void mana_shrink_eqs(struct mana_port_context *apc, unsigned int keep)
+{
+ struct gdma_context *gc = apc->ac->gdma_dev->gdma_context;
+ struct gdma_queue *eq;
+ unsigned int msi;
+ unsigned int i;
+
+ if (!apc->eqs || keep >= apc->num_eqs)
+ return;
+
+ for (i = keep; i < apc->num_eqs; i++) {
+ eq = apc->eqs[i].eq;
+ if (!eq)
+ continue;
+
+ debugfs_remove_recursive(apc->eqs[i].mana_eq_debugfs);
+ apc->eqs[i].mana_eq_debugfs = NULL;
+
+ msi = eq->eq.msix_index;
+ mana_gd_destroy_queue(gc, eq);
+ mana_gd_put_gic(gc, !gc->msi_sharing, msi);
+ apc->eqs[i].eq = NULL;
+ }
+
+ apc->num_eqs = keep;
+}
+
static int mana_fence_rq(struct mana_port_context *apc, struct mana_rxq *rxq)
{
struct mana_fence_rq_resp resp = {};
@@ -2015,7 +2197,12 @@ static void mana_poll_tx_cq(struct mana_cq *cq)
/* Ensure checking txq_stopped before apc->port_is_up. */
smp_rmb();
- if (txq_stopped && apc->port_is_up && avail_space >= MAX_TX_WQE_SIZE) {
+ /* Ordered by the same barrier: reaching here with txq_stopped set means
+ * the replacement queue has already run, which is strictly after this
+ * queue was marked retiring.
+ */
+ if (txq_stopped && !READ_ONCE(txq->retiring) && apc->port_is_up &&
+ avail_space >= MAX_TX_WQE_SIZE) {
netif_tx_wake_queue(net_txq);
apc->eth_stats.wake_queue++;
}
@@ -2078,7 +2265,7 @@ static void mana_rx_skb(void *buf_va, bool from_pool,
struct mana_rxcomp_oob *cqe, struct mana_rxq *rxq,
u32 pkt_len, u32 pkt_hash)
{
- struct mana_stats_rx *rx_stats = &rxq->stats;
+ struct mana_stats_rx *rx_stats = rxq->stats;
struct net_device *ndev = rxq->ndev;
u16 rxq_idx = rxq->rxq_idx;
struct napi_struct *napi;
@@ -2392,13 +2579,13 @@ static void mana_process_rx_cqe(struct mana_rxq *rxq, struct mana_cq *cq,
* Coalesced CQEs have at least 2 packets, so index is pkt_i - 2.
*/
if (pkt_i > 1) {
- u64_stats_update_begin(&rxq->stats.syncp);
- rxq->stats.coalesced_cqe[pkt_i - 2]++;
- u64_stats_update_end(&rxq->stats.syncp);
+ u64_stats_update_begin(&rxq->stats->syncp);
+ rxq->stats->coalesced_cqe[pkt_i - 2]++;
+ u64_stats_update_end(&rxq->stats->syncp);
} else if (!pkt_i && !pktlen) {
- u64_stats_update_begin(&rxq->stats.syncp);
- rxq->stats.pkt_len0_err++;
- u64_stats_update_end(&rxq->stats.syncp);
+ u64_stats_update_begin(&rxq->stats->syncp);
+ rxq->stats->pkt_len0_err++;
+ u64_stats_update_end(&rxq->stats->syncp);
netdev_err_once(ndev,
"RX pkt len=0, rq=%u, cq=%u, rxobj=0x%llx\n",
rxq->gdma_id, cq->gdma_id, rxq->rxobj);
@@ -2530,8 +2717,8 @@ static void mana_update_rx_dim(struct mana_cq *cq)
if (!smp_load_acquire(&apc->rx_dim_enabled))
return;
- dim_update_sample(READ_ONCE(cq->dim_event_ctr), rxq->stats.packets,
- rxq->stats.bytes, &dim_sample);
+ dim_update_sample(READ_ONCE(cq->dim_event_ctr), rxq->stats->packets,
+ rxq->stats->bytes, &dim_sample);
net_dim(&cq->dim, &dim_sample);
}
@@ -2638,7 +2825,11 @@ static void mana_deinit_txq(struct mana_port_context *apc, struct mana_txq *txq)
mana_gd_destroy_queue(gd->gdma_context, txq->gdma_sq);
}
-static void mana_destroy_txq(struct mana_port_context *apc)
+/* The array itself is left in place: the grow path tears down only a range,
+ * and the queues below @first are still live and still referenced by it.
+ */
+static void mana_destroy_txq_from(struct mana_port_context *apc,
+ unsigned int first)
{
struct napi_struct *napi;
int i;
@@ -2646,7 +2837,7 @@ static void mana_destroy_txq(struct mana_port_context *apc)
if (!apc->tx_qp)
return;
- for (i = 0; i < apc->num_queues; i++) {
+ for (i = first; i < apc->num_queues; i++) {
if (!apc->tx_qp[i])
continue;
@@ -2671,6 +2862,14 @@ static void mana_destroy_txq(struct mana_port_context *apc)
kvfree(apc->tx_qp[i]);
}
+}
+
+static void mana_destroy_txq(struct mana_port_context *apc)
+{
+ if (!apc->tx_qp)
+ return;
+
+ mana_destroy_txq_from(apc, 0);
kfree(apc->tx_qp);
apc->tx_qp = NULL;
@@ -2701,8 +2900,12 @@ static void mana_create_txq_debugfs(struct mana_port_context *apc, int idx)
tx_qp->tx_cq.gdma_cq, &mana_dbg_q_fops);
}
+/* @first is non-zero only for the grow path, which supplies an already
+ * allocated apc->tx_qp[] holding the carried-over queues. On error only the
+ * queues this call created are torn down.
+ */
static int mana_create_txq(struct mana_port_context *apc,
- struct net_device *net)
+ struct net_device *net, unsigned int first)
{
struct mana_context *ac = apc->ac;
struct gdma_dev *gd = ac->gdma_dev;
@@ -2717,9 +2920,14 @@ static int mana_create_txq(struct mana_port_context *apc,
int err;
int i;
- apc->tx_qp = kzalloc_objs(struct mana_tx_qp *, apc->num_queues);
- if (!apc->tx_qp)
- return -ENOMEM;
+ if (first) {
+ if (WARN_ON(!apc->tx_qp))
+ return -EINVAL;
+ } else {
+ apc->tx_qp = kzalloc_objs(struct mana_tx_qp *, apc->num_queues);
+ if (!apc->tx_qp)
+ return -ENOMEM;
+ }
/* The minimum size of the WQE is 32 bytes, hence
* apc->tx_queue_size represents the maximum number of WQEs
@@ -2736,7 +2944,7 @@ static int mana_create_txq(struct mana_port_context *apc,
gc = gd->gdma_context;
- for (i = 0; i < apc->num_queues; i++) {
+ for (i = first; i < apc->num_queues; i++) {
apc->tx_qp[i] = kvzalloc_obj(*apc->tx_qp[i]);
if (!apc->tx_qp[i]) {
err = -ENOMEM;
@@ -2748,9 +2956,10 @@ static int mana_create_txq(struct mana_port_context *apc,
/* Create SQ */
txq = &apc->tx_qp[i]->txq;
- u64_stats_init(&txq->stats.syncp);
+ txq->stats = &apc->txq_stats[i];
txq->ndev = net;
txq->net_txq = netdev_get_tx_queue(net, i);
+ txq->reset_gen = READ_ONCE(apc->ac->reset_gen);
txq->vp_offset = apc->tx_vp_offset;
txq->napi_initialized = false;
skb_queue_head_init(&txq->pending_skbs);
@@ -2843,7 +3052,10 @@ static int mana_create_txq(struct mana_port_context *apc,
out:
netdev_err(net, "Failed to create %d TX queues, %d\n",
apc->num_queues, err);
- mana_destroy_txq(apc);
+ if (first)
+ mana_destroy_txq_from(apc, first);
+ else
+ mana_destroy_txq(apc);
return err;
}
@@ -3006,11 +3218,14 @@ static int mana_push_wqe(struct mana_rxq *rxq)
static int mana_create_page_pool(struct mana_rxq *rxq, struct gdma_context *gc)
{
- struct mana_port_context *mpc = netdev_priv(rxq->ndev);
struct page_pool_params pprm = {};
int ret;
- pprm.pool_size = mpc->rx_queue_size / rxq->frag_count + 1;
+ /* Size the recycle ring from the queue being built, not from the live
+ * port context: during a swap the queue may be sized for a ring the
+ * running configuration does not use yet.
+ */
+ pprm.pool_size = rxq->num_rx_buf / rxq->frag_count + 1;
pprm.nid = gc->numa_node;
pprm.napi = &rxq->rx_cq.napi;
pprm.netdev = rxq->ndev;
@@ -3059,11 +3274,14 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc,
return ERR_PTR(-ENOMEM);
rxq->ndev = ndev;
+ /* Wire up the port-owned statistics before the queue can be polled. */
+ rxq->stats = &apc->rxq_stats[rxq_idx];
rxq->num_rx_buf = apc->rx_queue_size;
rxq->rxq_idx = rxq_idx;
rxq->rxobj = INVALID_MANA_HANDLE;
- mana_get_rxbuf_cfg(apc, ndev->mtu, &rxq->datasize, &rxq->alloc_size,
+ mana_get_rxbuf_cfg(apc, apc->configured_mtu, &rxq->datasize,
+ &rxq->alloc_size,
&rxq->headroom, &rxq->frag_count);
/* Create page pool for RX queue */
err = mana_create_page_pool(rxq, gc);
@@ -3193,14 +3411,18 @@ static void mana_create_rxq_debugfs(struct mana_port_context *apc, int idx)
&mana_dbg_q_fops);
}
+/* @first is non-zero only for the grow path; the slots below it already hold
+ * carried-over queues. Queues created before a failure are left in
+ * apc->rxqs[] for the caller to tear down.
+ */
static int mana_add_rx_queues(struct mana_port_context *apc,
- struct net_device *ndev)
+ struct net_device *ndev, unsigned int first)
{
struct mana_rxq *rxq;
int err = 0;
int i;
- for (i = 0; i < apc->num_queues; i++) {
+ for (i = first; i < apc->num_queues; i++) {
rxq = mana_create_rxq(apc, i, &apc->eqs[i], ndev);
if (IS_ERR(rxq)) {
err = PTR_ERR(rxq);
@@ -3208,7 +3430,6 @@ static int mana_add_rx_queues(struct mana_port_context *apc,
goto out;
}
- u64_stats_init(&rxq->stats.syncp);
apc->rxqs[i] = rxq;
@@ -3220,14 +3441,16 @@ static int mana_add_rx_queues(struct mana_port_context *apc,
return err;
}
-static void mana_destroy_rxqs(struct mana_port_context *apc)
+/* The array is left in place; see mana_destroy_txq_from(). */
+static void mana_destroy_rxqs_from(struct mana_port_context *apc,
+ unsigned int first)
{
struct mana_rxq *rxq;
u32 rxq_idx;
if (apc->rxqs) {
- for (rxq_idx = 0; rxq_idx < apc->num_queues; rxq_idx++) {
+ for (rxq_idx = first; rxq_idx < apc->num_queues; rxq_idx++) {
rxq = apc->rxqs[rxq_idx];
if (!rxq)
continue;
@@ -3238,6 +3461,11 @@ static void mana_destroy_rxqs(struct mana_port_context *apc)
}
}
+static void mana_destroy_rxqs(struct mana_port_context *apc)
+{
+ mana_destroy_rxqs_from(apc, 0);
+}
+
static void mana_destroy_vport(struct mana_port_context *apc)
{
struct gdma_dev *gd = apc->ac->gdma_dev;
@@ -3303,6 +3531,38 @@ static void mana_rss_table_init(struct mana_port_context *apc)
ethtool_rxfh_indir_default(i, apc->num_queues);
}
+/* Whether @apc's indirection table can be carried to a set of @num_queues,
+ * rather than rebuilt from the driver default.
+ *
+ * Only a user table ("ethtool -X") is kept; a driver one is rebuilt to spread
+ * over the new count. ethtool_check_max_channel() already refuses a reduction
+ * that leaves a user table pointing past the last queue, so the bounds check
+ * below only guards rebuild paths that bypass ethtool.
+ *
+ * @lost reports a table that cannot be carried instead of calling
+ * ethtool_rxfh_indir_lost() here, since the swap may still fail and leave the
+ * port on queues where the table still applies. False when none was set.
+ */
+static bool mana_rss_table_keep(struct mana_port_context *apc,
+ unsigned int num_queues, bool *lost)
+{
+ u32 i;
+
+ *lost = false;
+
+ if (!netif_is_rxfh_configured(apc->ndev))
+ return false;
+
+ for (i = 0; i < apc->indir_table_sz; i++) {
+ if (apc->indir_table[i] >= num_queues) {
+ *lost = true;
+ return false;
+ }
+ }
+
+ return true;
+}
+
int mana_disable_vport_rx(struct mana_port_context *apc)
{
return mana_cfg_vport_steering(apc, TRI_STATE_FALSE, false, false,
@@ -3573,6 +3833,7 @@ int mana_alloc_queues(struct net_device *ndev)
{
struct mana_port_context *apc = netdev_priv(ndev);
struct gdma_dev *gd = apc->ac->gdma_dev;
+ bool indir_lost;
int err;
err = mana_create_vport(apc, ndev);
@@ -3589,7 +3850,7 @@ int mana_alloc_queues(struct net_device *ndev)
goto destroy_vport;
}
- err = mana_create_txq(apc, ndev);
+ err = mana_create_txq(apc, ndev, 0);
if (err) {
netdev_err(ndev, "Failed to create TXQ on vPort %u: %d\n",
apc->port_idx, err);
@@ -3604,7 +3865,7 @@ int mana_alloc_queues(struct net_device *ndev)
goto destroy_txq;
}
- err = mana_add_rx_queues(apc, ndev);
+ err = mana_add_rx_queues(apc, ndev, 0);
if (err)
goto destroy_rxq;
@@ -3618,7 +3879,18 @@ int mana_alloc_queues(struct net_device *ndev)
goto destroy_rxq;
}
- mana_rss_table_init(apc);
+ /* Keep a user-configured RSS table across a rebuild; the entries are
+ * queue indices, so they stay meaningful as long as the queue count
+ * is unchanged. Only a driver-generated table is regenerated here.
+ *
+ * Nothing to roll back to here, so report the loss as soon as it is
+ * decided and keep the table and the core's view of it in step.
+ */
+ if (!mana_rss_table_keep(apc, apc->num_queues, &indir_lost)) {
+ if (indir_lost)
+ ethtool_rxfh_indir_lost(ndev);
+ mana_rss_table_init(apc);
+ }
err = mana_config_rss(apc, TRI_STATE_TRUE, true, true);
if (err) {
@@ -3676,15 +3948,114 @@ int mana_attach(struct net_device *ndev)
return 0;
}
-static int mana_dealloc_queues(struct net_device *ndev)
+/* Drain a set about to be destroyed: nothing new can reach it, so wait for the
+ * hardware to finish what it owns, then release every mapped SKB.
+ *
+ * The 120s budget is shared across all queues. On timeout the device is reset,
+ * since its buffers are about to be freed while it may still DMA into them; if
+ * that fails too they are leaked.
+ *
+ * Returns true only if a reset happened, taking every queue on the function
+ * down with it.
+ */
+static bool mana_drain_txqs(struct mana_port_context *apc)
{
- struct mana_port_context *apc = netdev_priv(ndev);
unsigned long timeout = jiffies + 120 * HZ;
struct gdma_dev *gd = apc->ac->gdma_dev;
+ bool quiesced = true;
+ bool reset = false;
struct mana_txq *txq;
struct sk_buff *skb;
- int i, err;
u32 tsleep;
+ int i, err;
+
+ if (!apc->tx_qp)
+ return false;
+
+ for (i = 0; i < apc->num_queues; i++) {
+ if (!apc->tx_qp[i])
+ continue;
+
+ txq = &apc->tx_qp[i]->txq;
+
+ /* The function was reset after this queue was created, so the
+ * device has stopped touching its buffers and the completions
+ * waited for below can never arrive. Without this the port
+ * would burn the full timeout under RTNL, then reset the
+ * function again on the way out.
+ */
+ if (READ_ONCE(apc->ac->reset_gen) != txq->reset_gen)
+ continue;
+
+ tsleep = 1000;
+ while (atomic_read(&txq->pending_sends) > 0 &&
+ time_before(jiffies, timeout)) {
+ usleep_range(tsleep, tsleep + 1000);
+ tsleep <<= 1;
+ }
+ if (atomic_read(&txq->pending_sends)) {
+ /* The device still owns these buffers, so reset it
+ * before they are freed. pci_try_reset_function()
+ * rather than pcie_flr(): it saves and restores config
+ * space, which a bare FLR wipes behind the PCI core's
+ * back. Trylock because RTNL is held here while the
+ * remove path takes the device lock first.
+ */
+ err = pci_try_reset_function(to_pci_dev(gd->gdma_context->dev));
+ if (err) {
+ netdev_err(apc->ndev,
+ "function reset failed: %d, %d pkts pending in txq %u\n",
+ err, atomic_read(&txq->pending_sends),
+ txq->gdma_txq_id);
+ quiesced = false;
+ } else {
+ /* Every queue on the function is dead now,
+ * including the ones this loop has not reached
+ * and those of the other ports.
+ */
+ WRITE_ONCE(apc->ac->reset_gen,
+ apc->ac->reset_gen + 1);
+
+ /* Only a reset that actually happened takes the
+ * other ports down with it; reporting a failed
+ * one would rebuild them for nothing.
+ */
+ reset = true;
+ }
+ break;
+ }
+ }
+
+ /* Only a reset that actually happened makes freeing these safe; without
+ * one the device still owns them. Leak instead, bounded at one SQ ring
+ * of skbs per queue.
+ */
+ if (!quiesced) {
+ netdev_err(apc->ndev,
+ "device not quiesced, leaking pending TX buffers instead of unmapping memory it can still DMA from\n");
+ return reset;
+ }
+
+ for (i = 0; i < apc->num_queues; i++) {
+ if (!apc->tx_qp[i])
+ continue;
+
+ txq = &apc->tx_qp[i]->txq;
+ while ((skb = skb_dequeue(&txq->pending_skbs))) {
+ mana_unmap_skb(skb, apc);
+ dev_kfree_skb_any(skb);
+ }
+ atomic_set(&txq->pending_sends, 0);
+ }
+
+ return reset;
+}
+
+static int mana_dealloc_queues(struct net_device *ndev)
+{
+ struct mana_port_context *apc = netdev_priv(ndev);
+ struct gdma_dev *gd = apc->ac->gdma_dev;
+ int err;
if (apc->port_is_up)
return -EINVAL;
@@ -3702,41 +4073,27 @@ static int mana_dealloc_queues(struct net_device *ndev)
* new packets due to apc->port_is_up being false.
*
* Drain all the in-flight TX packets.
- * A timeout of 120 seconds for all the queues is used.
- * This will break the while loop when h/w is not responding.
- * This value of 120 has been decided here considering max
- * number of queues.
+ *
+ * If the drain had to reset the function to get there, every other
+ * port on the adapter lost its queues too, so schedule them for a
+ * rebuild. This port is being torn down here and needs no such
+ * treatment, and a down port stays down: with port_st_save false,
+ * detach and attach both skip the queue work.
*/
+ if (mana_drain_txqs(apc)) {
+ struct mana_context *ac = apc->ac;
+ unsigned int i;
- if (apc->tx_qp) {
- for (i = 0; i < apc->num_queues; i++) {
- txq = &apc->tx_qp[i]->txq;
- tsleep = 1000;
- while (atomic_read(&txq->pending_sends) > 0 &&
- time_before(jiffies, timeout)) {
- usleep_range(tsleep, tsleep + 1000);
- tsleep <<= 1;
- }
- if (atomic_read(&txq->pending_sends)) {
- err =
- pcie_flr(to_pci_dev(gd->gdma_context->dev));
- if (err) {
- netdev_err(ndev, "flr failed %d with %d pkts pending in txq %u\n",
- err,
- atomic_read(&txq->pending_sends),
- txq->gdma_txq_id);
- }
- break;
- }
- }
+ for (i = 0; i < ac->num_ports; i++) {
+ struct mana_port_context *sib;
- for (i = 0; i < apc->num_queues; i++) {
- txq = &apc->tx_qp[i]->txq;
- while ((skb = skb_dequeue(&txq->pending_skbs))) {
- mana_unmap_skb(skb, apc);
- dev_kfree_skb_any(skb);
- }
- atomic_set(&txq->pending_sends, 0);
+ if (!ac->ports[i] || ac->ports[i] == ndev)
+ continue;
+ sib = netdev_priv(ac->ports[i]);
+ netdev_err(ac->ports[i],
+ "queues reset by a sibling port, scheduling rebuild\n");
+ queue_work(ac->per_port_queue_reset_wq,
+ &sib->queue_reset_work);
}
}
@@ -3760,6 +4117,868 @@ static int mana_dealloc_queues(struct net_device *ndev)
return 0;
}
+/*
+ * Pre-allocate and swap reconfiguration.
+ *
+ * Build a new queue set while the current one serves traffic, publish it, then
+ * destroy the old one. A failed allocation leaves the running config untouched,
+ * and the vport is never torn down, so RDMA cannot take it mid-swap. The cost
+ * is room for both sets at once, so a rebuild at the vport's maximum queue
+ * count can be refused; EQs are shared from a port-owned pool, not doubled.
+ *
+ * Everything builds in a scratch mana_port_context, since mana_start_xmit()
+ * dereferences apc->tx_qp[] guarded only by port_is_up. Per-queue debugfs is
+ * suppressed meanwhile, as the names would collide.
+ */
+
+/* Snapshot the queue-set fields of @ctx into @out. */
+static void mana_qset_snapshot(const struct mana_port_context *ctx,
+ struct mana_qset *out)
+{
+ out->tx_qp = ctx->tx_qp;
+ out->rxqs = ctx->rxqs;
+ out->indir_table = ctx->indir_table;
+ out->indir_table_sz = ctx->indir_table_sz;
+ out->rxobj_table = ctx->rxobj_table;
+ out->default_rxobj = ctx->default_rxobj;
+ out->num_queues = ctx->num_queues;
+ out->rx_queue_size = ctx->rx_queue_size;
+ out->tx_queue_size = ctx->tx_queue_size;
+ out->priv_flags = ctx->priv_flags;
+ out->mtu = ctx->configured_mtu;
+ out->bpf_prog = ctx->bpf_prog;
+
+ /* A set taken from a live context has nothing pending; the builders
+ * set this after snapshotting if they had to drop the user's table.
+ */
+ out->rxfh_indir_lost = false;
+}
+
+/* The vport (port_handle, vport_use_count) and the port-level debugfs dir are
+ * not touched: they outlive any individual queue set.
+ */
+static void mana_qset_install(struct mana_port_context *ctx,
+ const struct mana_qset *qset)
+{
+ ctx->tx_qp = qset->tx_qp;
+ ctx->rxqs = qset->rxqs;
+ ctx->indir_table = qset->indir_table;
+ ctx->indir_table_sz = qset->indir_table_sz;
+ ctx->rxobj_table = qset->rxobj_table;
+ ctx->default_rxobj = qset->default_rxobj;
+ ctx->num_queues = qset->num_queues;
+ ctx->rx_queue_size = qset->rx_queue_size;
+ ctx->tx_queue_size = qset->tx_queue_size;
+ ctx->priv_flags = qset->priv_flags;
+ ctx->configured_mtu = qset->mtu;
+ ctx->bpf_prog = qset->bpf_prog;
+}
+
+/**
+ * mana_qset_scratch_alloc - build a scratch port context for queue work
+ * @apc: the live port context to shadow
+ *
+ * Returns a heap copy of @apc that shares its vport identity but owns no
+ * queues, so the existing allocators and destroyers can run against it
+ * without touching the live context.
+ */
+struct mana_port_context *mana_qset_scratch_alloc(struct mana_port_context *apc)
+{
+ struct mana_port_context *scratch;
+
+ scratch = kvzalloc(sizeof(*scratch), GFP_KERNEL);
+ if (!scratch)
+ return NULL;
+
+ *scratch = *apc;
+
+ /* EQs stay shared with the live port: they are a vector-backed
+ * resource and must not be duplicated for the new set.
+ */
+ scratch->tx_qp = NULL;
+ scratch->rxqs = NULL;
+ scratch->indir_table = NULL;
+ scratch->rxobj_table = NULL;
+ scratch->default_rxobj = INVALID_MANA_HANDLE;
+
+ /* Never consume the live set's pre-allocated RX buffers; the swap path
+ * has no post-teardown allocation to de-risk.
+ */
+ scratch->rxbufs_pre = NULL;
+ scratch->das_pre = NULL;
+ scratch->rxbpre_total = 0;
+
+ /* Two sets are alive at once and would collide on the same names under
+ * vport%d. An IS_ERR() parent makes every create and remove a no-op.
+ */
+ scratch->mana_port_debugfs = ERR_PTR(-ENODEV);
+
+ return scratch;
+}
+
+void mana_qset_scratch_free(struct mana_port_context *scratch)
+{
+ kvfree(scratch);
+}
+
+/* Carve the live set into a kept prefix [0, @new_count) in @out_new and a
+ * tail to retire in @out_tail. @apc is untouched, on failure too.
+ *
+ * Queue i is built from the ring sizes and the buffer layout and keeps EQ i
+ * at any count, so a reduction carries the survivors over and destroys only
+ * the tail. Allocates two pointer arrays and a steering table, nothing else:
+ * the queues that stay keep their page pools, posted buffers and NAPI.
+ */
+int mana_split_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch, unsigned int new_count,
+ struct mana_qset *out_new, struct mana_qset *out_tail)
+{
+ unsigned int old_count = apc->num_queues;
+ struct mana_tx_qp **new_tx, **tail_tx;
+ struct mana_rxq **new_rx, **tail_rx;
+ unsigned int tail_count;
+ bool indir_lost;
+ unsigned int i;
+ int err;
+
+ ASSERT_RTNL();
+
+ if (WARN_ON(new_count == 0 || new_count >= old_count))
+ return -EINVAL;
+ if (WARN_ON(!apc->tx_qp || !apc->rxqs))
+ return -EINVAL;
+
+ tail_count = old_count - new_count;
+
+ /* Build the smaller set's steering table separately: mana_config_rss()
+ * would otherwise index the shorter rxqs[] with entries still referring
+ * to retired queues.
+ */
+ scratch->num_queues = new_count;
+ err = mana_rss_table_alloc(scratch);
+ if (err)
+ return err;
+
+ if (mana_rss_table_keep(apc, new_count, &indir_lost))
+ memcpy(scratch->indir_table, apc->indir_table,
+ apc->indir_table_sz * sizeof(*apc->indir_table));
+ else
+ mana_rss_table_init(scratch);
+
+ new_tx = kzalloc_objs(struct mana_tx_qp *, new_count);
+ new_rx = kzalloc_objs(struct mana_rxq *, new_count);
+ tail_tx = kzalloc_objs(struct mana_tx_qp *, tail_count);
+ tail_rx = kzalloc_objs(struct mana_rxq *, tail_count);
+ if (!new_tx || !new_rx || !tail_tx || !tail_rx) {
+ err = -ENOMEM;
+ goto free_arrays;
+ }
+
+ for (i = 0; i < new_count; i++) {
+ new_tx[i] = apc->tx_qp[i];
+ new_rx[i] = apc->rxqs[i];
+ }
+ for (i = 0; i < tail_count; i++) {
+ tail_tx[i] = apc->tx_qp[new_count + i];
+ tail_rx[i] = apc->rxqs[new_count + i];
+ }
+
+ /* The kept prefix, with the new steering table. */
+ out_new->tx_qp = new_tx;
+ out_new->rxqs = new_rx;
+ out_new->indir_table = scratch->indir_table;
+ out_new->indir_table_sz = scratch->indir_table_sz;
+ out_new->rxobj_table = scratch->rxobj_table;
+ out_new->default_rxobj = apc->rxqs[0]->rxobj;
+ out_new->num_queues = new_count;
+ out_new->rx_queue_size = apc->rx_queue_size;
+ out_new->tx_queue_size = apc->tx_queue_size;
+ out_new->priv_flags = apc->priv_flags;
+ out_new->mtu = apc->configured_mtu;
+ out_new->bpf_prog = apc->bpf_prog;
+ out_new->rxfh_indir_lost = indir_lost;
+
+ /* Ownership of the table moved to @out_new. */
+ scratch->indir_table = NULL;
+ scratch->rxobj_table = NULL;
+
+ /* The tail. It owns no steering table; bpf_prog is carried so that
+ * retiring it drops exactly the tail's per-queue program references
+ * and leaves the kept ones alone.
+ */
+ memset(out_tail, 0, sizeof(*out_tail));
+ out_tail->tx_qp = tail_tx;
+ out_tail->rxqs = tail_rx;
+ out_tail->default_rxobj = INVALID_MANA_HANDLE;
+ out_tail->num_queues = tail_count;
+ out_tail->rx_queue_size = apc->rx_queue_size;
+ out_tail->tx_queue_size = apc->tx_queue_size;
+ out_tail->priv_flags = apc->priv_flags;
+ out_tail->mtu = apc->configured_mtu;
+ out_tail->bpf_prog = apc->bpf_prog;
+
+ return 0;
+
+free_arrays:
+ kfree(new_tx);
+ kfree(new_rx);
+ kfree(tail_tx);
+ kfree(tail_rx);
+ mana_cleanup_indir_table(scratch);
+ return err;
+}
+
+/**
+ * mana_discard_split - drop the containers built by mana_split_qset()
+ * @newq: set that was never published
+ * @tailq: matching tail
+ *
+ * Frees the pointer arrays and the steering table only: the queues they refer
+ * to are still owned by the live port context.
+ */
+void mana_discard_split(struct mana_qset *newq, struct mana_qset *tailq)
+{
+ kfree(newq->tx_qp);
+ kfree(newq->rxqs);
+ kfree(newq->indir_table);
+ kfree(newq->rxobj_table);
+ kfree(tailq->tx_qp);
+ kfree(tailq->rxqs);
+ memset(newq, 0, sizeof(*newq));
+ memset(tailq, 0, sizeof(*tailq));
+}
+
+/* The mirror image of mana_split_qset(): carry the existing queues over into
+ * @out_new and build only the [old, @new_count) tail. Growing 4 channels to 8
+ * creates 4 SQ/RQ pairs, not 8, and never holds 12 against the vport maximum.
+ *
+ * @out_fresh names just the queues created here, so a failed publish retires
+ * exactly those. On failure @apc is untouched.
+ */
+int mana_grow_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch, unsigned int new_count,
+ struct mana_qset *out_new, struct mana_qset *out_fresh)
+{
+ unsigned int old_count = apc->num_queues;
+ struct mana_tx_qp **new_tx, **fresh_tx;
+ struct mana_rxq **new_rx, **fresh_rx;
+ struct net_device *ndev = apc->ndev;
+ unsigned int fresh_count;
+ bool indir_lost;
+ unsigned int i;
+ int err;
+
+ ASSERT_RTNL();
+
+ if (WARN_ON(new_count <= old_count))
+ return -EINVAL;
+ if (WARN_ON(!apc->tx_qp || !apc->rxqs))
+ return -EINVAL;
+
+ fresh_count = new_count - old_count;
+
+ new_tx = kzalloc_objs(struct mana_tx_qp *, new_count);
+ new_rx = kzalloc_objs(struct mana_rxq *, new_count);
+ fresh_tx = kzalloc_objs(struct mana_tx_qp *, fresh_count);
+ fresh_rx = kzalloc_objs(struct mana_rxq *, fresh_count);
+ if (!new_tx || !new_rx || !fresh_tx || !fresh_rx) {
+ err = -ENOMEM;
+ goto free_arrays;
+ }
+
+ for (i = 0; i < old_count; i++) {
+ new_tx[i] = apc->tx_qp[i];
+ new_rx[i] = apc->rxqs[i];
+ }
+
+ /* @scratch now describes the merged set; the builders fill only the
+ * [old_count, new_count) slots.
+ */
+ scratch->num_queues = new_count;
+ scratch->tx_qp = new_tx;
+ scratch->rxqs = new_rx;
+
+ err = mana_rss_table_alloc(scratch);
+ if (err)
+ goto free_arrays;
+
+ /* Same shared, port-owned EQ pool as a full rebuild; this only adds
+ * the vectors the extra queues need.
+ */
+ err = mana_grow_eqs(apc, new_count);
+ if (err)
+ goto cleanup_rss;
+
+ scratch->eqs = apc->eqs;
+ scratch->num_eqs = apc->num_eqs;
+
+ err = mana_create_txq(scratch, ndev, old_count);
+ if (err)
+ goto cleanup_rss; /* create_txq already undid its own work */
+
+ err = mana_add_rx_queues(scratch, ndev, old_count);
+ if (err)
+ goto cleanup_rxq;
+
+ if (mana_rss_table_keep(apc, new_count, &indir_lost))
+ memcpy(scratch->indir_table, apc->indir_table,
+ apc->indir_table_sz * sizeof(*apc->indir_table));
+ else
+ mana_rss_table_init(scratch);
+
+ mana_qset_snapshot(scratch, out_new);
+ out_new->rxfh_indir_lost = indir_lost;
+
+ for (i = 0; i < fresh_count; i++) {
+ fresh_tx[i] = new_tx[old_count + i];
+ fresh_rx[i] = new_rx[old_count + i];
+ }
+
+ memset(out_fresh, 0, sizeof(*out_fresh));
+ out_fresh->tx_qp = fresh_tx;
+ out_fresh->rxqs = fresh_rx;
+ out_fresh->default_rxobj = INVALID_MANA_HANDLE;
+ out_fresh->num_queues = fresh_count;
+ out_fresh->rx_queue_size = apc->rx_queue_size;
+ out_fresh->tx_queue_size = apc->tx_queue_size;
+ out_fresh->priv_flags = apc->priv_flags;
+ out_fresh->mtu = apc->configured_mtu;
+ out_fresh->bpf_prog = apc->bpf_prog;
+
+ /* mana_publish_qset() cannot do this: mana_chn_setxdp() decides from
+ * rxqs[0], a carried-over queue that already holds the program, and
+ * returns early. Address only the new queues through @out_fresh so
+ * exactly fresh_count references are taken.
+ */
+ mana_qset_install(scratch, out_fresh);
+ mana_chn_setxdp(scratch, mana_xdp_get(apc));
+
+ return 0;
+
+cleanup_rxq:
+ mana_destroy_rxqs_from(scratch, old_count);
+ mana_destroy_txq_from(scratch, old_count);
+cleanup_rss:
+ mana_cleanup_indir_table(scratch);
+free_arrays:
+ /* Only the containers: every queue they name is still live on @apc. */
+ scratch->tx_qp = NULL;
+ scratch->rxqs = NULL;
+ kfree(new_tx);
+ kfree(new_rx);
+ kfree(fresh_tx);
+ kfree(fresh_rx);
+
+ /* Give back any EQ this attempt added rather than holding its MSI-X
+ * vectors: the live set still needs only apc->num_queues of them, and
+ * every CQ this call created has been destroyed above.
+ */
+ mana_shrink_eqs(apc, apc->num_queues);
+
+ netdev_err(ndev, "mana_grow_qset(num_queues=%u) failed: %d\n",
+ new_count, err);
+ return err;
+}
+
+/**
+ * mana_discard_grow - drop the merged containers built by mana_grow_qset()
+ * @newq: set that was never published
+ *
+ * Frees the pointer arrays and steering table only: carried-over queues
+ * belong to the live context, fresh ones are retired through @out_fresh.
+ */
+void mana_discard_grow(struct mana_qset *newq)
+{
+ kfree(newq->tx_qp);
+ kfree(newq->rxqs);
+ kfree(newq->indir_table);
+ kfree(newq->rxobj_table);
+ memset(newq, 0, sizeof(*newq));
+}
+
+/* Rebuild the queues at the current count in @scratch, for callers changing a
+ * per-queue property; a count change goes through mana_split_qset() or
+ * mana_grow_qset(), so this never has to add an EQ. The installed set keeps
+ * serving traffic meanwhile. On error nothing is left allocated.
+ */
+int mana_alloc_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch,
+ unsigned int rx_queue_size, unsigned int tx_queue_size,
+ u32 priv_flags, int mtu, struct bpf_prog *bpf_prog,
+ struct mana_qset *out)
+{
+ struct net_device *ndev = scratch->ndev;
+ bool indir_lost;
+ int err;
+
+ ASSERT_RTNL();
+
+ scratch->num_queues = apc->num_queues;
+ scratch->rx_queue_size = rx_queue_size;
+ scratch->tx_queue_size = tx_queue_size;
+ scratch->priv_flags = priv_flags;
+
+ /* mana_get_rxbuf_cfg() reads both of these when sizing RX buffers,
+ * so the new set is built for the requested MTU / XDP program
+ * without disturbing the running set.
+ */
+ scratch->configured_mtu = mtu;
+ scratch->bpf_prog = bpf_prog;
+
+ err = mana_init_port_context(scratch);
+ if (err)
+ goto out_err;
+
+ err = mana_rss_table_alloc(scratch);
+ if (err)
+ goto cleanup_rxq_array;
+
+ /* The queue count is unchanged, so the port's shared EQ pool already
+ * has an EQ for every queue this set will build. Both sets reference
+ * the same pool while they are live, so a swap never needs old + new
+ * MSI-X vectors.
+ */
+ scratch->eqs = apc->eqs;
+ scratch->num_eqs = apc->num_eqs;
+
+ err = mana_create_txq(scratch, ndev, 0);
+ if (err)
+ goto cleanup_rss;
+
+ err = mana_add_rx_queues(scratch, ndev, 0);
+ if (err)
+ goto cleanup_rxq;
+
+ /* Carry a user-configured RSS table over to the new set. The entries
+ * are queue indices, so mana_config_rss() in mana_publish_qset() maps
+ * them onto the new set's RX objects. A driver-generated table is
+ * rebuilt instead, so it covers every queue of the new set.
+ */
+ if (mana_rss_table_keep(apc, scratch->num_queues, &indir_lost))
+ memcpy(scratch->indir_table, apc->indir_table,
+ apc->indir_table_sz * sizeof(*apc->indir_table));
+ else
+ mana_rss_table_init(scratch);
+
+ mana_qset_snapshot(scratch, out);
+ out->rxfh_indir_lost = indir_lost;
+ return 0;
+
+cleanup_rxq:
+ /* mana_add_rx_queues() may have created queues before failing; they
+ * own RQ/CQ objects, NAPI state and page pools, so tear down whatever
+ * made it into scratch->rxqs[] before dropping the array.
+ */
+ mana_destroy_rxqs(scratch);
+ mana_destroy_txq(scratch);
+cleanup_rss:
+ mana_cleanup_indir_table(scratch);
+cleanup_rxq_array:
+ kfree(scratch->rxqs);
+ scratch->rxqs = NULL;
+out_err:
+ /* No EQ to give back: this path never adds one, it reuses the pool
+ * the live set is already using.
+ */
+ netdev_err(ndev, "mana_alloc_qset(num_queues=%u) failed: %d\n",
+ apc->num_queues, err);
+ return err;
+}
+
+/* Close a port mana_publish_qset() gave up on; does nothing otherwise. Under
+ * RTNL.
+ *
+ * The caller releases the unpublished set first: closing destroys the shared
+ * EQ pool its CQs attach to, and only the caller knows whether it owns its
+ * queues or shares them with the live set. RX is already off.
+ *
+ * Merely stopping the port would leave port_is_up false with queues still
+ * allocated, so mana_detach() skips teardown and the next open trips
+ * WARN_ON(apc->eqs).
+ */
+void mana_publish_close_if_needed(struct mana_port_context *apc)
+{
+ ASSERT_RTNL();
+
+ if (!apc->publish_dead_end)
+ return;
+
+ apc->publish_dead_end = false;
+
+ /* mana_dealloc_queues() requires the port already marked down, which
+ * mana_publish_qset() did before the swap it is unwinding.
+ */
+ if (mana_dealloc_queues(apc->ndev))
+ netdev_err(apc->ndev,
+ "failed to close the port after a failed rollback\n");
+}
+
+/* Start only the netdev queues that can take work. A carried-over queue may
+ * still have a full ring, and restarting it would just make mana_start_xmit()
+ * drop; leave it for mana_poll_tx_cq() to wake. Must run after port_is_up is
+ * set, or that wakeup is gated off.
+ */
+static void mana_start_txqs(struct mana_port_context *apc)
+{
+ struct net_device *ndev = apc->ndev;
+ unsigned int i;
+
+ if (!apc->tx_qp)
+ return;
+
+ for (i = 0; i < apc->num_queues; i++) {
+ if (!apc->tx_qp[i])
+ continue;
+
+ if (mana_can_tx(apc->tx_qp[i]->txq.gdma_sq))
+ netif_tx_wake_queue(netdev_get_tx_queue(ndev, i));
+ }
+}
+
+/* A retiring queue shares its struct netdev_queue with whatever replaced it
+ * at the same index, and only ever drains, so it always looks like it has
+ * room. Without this flag its completions would wake a netdev queue that the
+ * live queue stopped on a full ring.
+ *
+ * A queue both sets own must end up unmarked, so callers mark the leaving set
+ * first and unmark the incoming one second.
+ */
+static void mana_qset_set_retiring(struct mana_qset *qset, bool retiring)
+{
+ unsigned int q;
+
+ if (!qset->tx_qp)
+ return;
+
+ for (q = 0; q < qset->num_queues; q++) {
+ if (qset->tx_qp[q])
+ WRITE_ONCE(qset->tx_qp[q]->txq.retiring, retiring);
+ }
+}
+
+/* Give up on a swap. Steering may still point at the set the caller is about
+ * to free, and restoring it is exactly what failed, so stop delivery before
+ * those RQs and their buffers go away. This is the narrow steering request -
+ * no key, table or default-rxobj update - so it can land where the full
+ * mana_config_rss() restore did not.
+ *
+ * Closing the port is left to mana_publish_close_if_needed(), which must run
+ * after the caller has released that set.
+ */
+static void mana_publish_give_up(struct mana_port_context *apc)
+{
+ int err;
+
+ apc->rss_state = TRI_STATE_FALSE;
+
+ err = mana_disable_vport_rx(apc);
+ if (err && mana_en_need_log(apc, err))
+ netdev_err(apc->ndev, "failed to disable vPort RX: %d\n", err);
+
+ apc->publish_dead_end = true;
+}
+
+/* Swap @newq onto @apc, handing the previous set back in @out_old for the
+ * caller to free. On failure the old set is reinstalled and the caller frees
+ * only @newq. Must be called under RTNL.
+ *
+ * netif_tx_disable() is load-bearing: mana_start_xmit() dereferences
+ * apc->tx_qp[] guarded only by apc->port_is_up, and ndo_xdp_xmit() bypasses
+ * the txq-stopped checks, so port_is_up is cleared over the same window.
+ */
+int mana_publish_qset(struct mana_port_context *apc, struct mana_qset *newq,
+ struct mana_qset *out_old)
+{
+ struct net_device *ndev = apc->ndev;
+ bool carrier_ok;
+ int err;
+
+ ASSERT_RTNL();
+
+ carrier_ok = netif_carrier_ok(ndev);
+ netif_carrier_off(ndev);
+
+ /* Clear port_is_up before stopping the queues, pairing with the
+ * smp_rmb() in mana_poll_tx_cq(): that reader samples
+ * netif_tx_queue_stopped() first, so a completion seeing a queue
+ * stopped here also sees port_is_up false and will not wake it
+ * mid-swap. It also fences mana_xdp_xmit(), which is gated only by
+ * port_is_up and would otherwise index a stale apc->tx_qp[].
+ */
+ WRITE_ONCE(apc->port_is_up, false);
+
+ /* Ensure port state updated before txq state */
+ smp_wmb();
+
+ netif_tx_disable(ndev);
+
+ mana_qset_snapshot(apc, out_old);
+
+ /* Mark the outgoing set before the grace period, not after: a
+ * completion that saw the flag clear must not still be in flight when
+ * the gate reopens, or it could wake a netdev queue that its
+ * replacement had stopped on a full ring.
+ */
+ mana_qset_set_retiring(out_old, true);
+
+ /* Wait out any transmit or ndo_xdp_xmit() that was already past the
+ * port_is_up test before the swap touches apc->tx_qp / the counts,
+ * and any completion that still saw the flag clear above.
+ */
+ synchronize_net();
+
+ /* Anything the incoming set carries over is staying, so clear the flag
+ * again - after the marking above, before the gate reopens.
+ */
+ mana_qset_set_retiring(newq, false);
+
+ mana_qset_install(apc, newq);
+ apc->rss_state = apc->num_queues > 1 ? TRI_STATE_TRUE : TRI_STATE_FALSE;
+
+ err = netif_set_real_num_tx_queues(ndev, apc->num_queues);
+ if (err)
+ goto rollback;
+
+ err = netif_set_real_num_rx_queues(ndev, apc->num_queues);
+ if (err)
+ goto rollback;
+
+ /* Carry the XDP program over before steering can reach the new RXQs:
+ * they were created with bpf_prog == NULL, so a packet arriving first
+ * would bypass an attached program. This also takes the per-queue
+ * references that mana_free_qset() drops for the old set.
+ */
+ mana_chn_setxdp(apc, mana_xdp_get(apc));
+
+ err = mana_config_rss(apc, TRI_STATE_TRUE, true, true);
+ if (err)
+ goto rollback;
+
+ /* The new set is serving traffic, so advertise its MTU. A no-op unless
+ * the caller is changing it.
+ */
+ WRITE_ONCE(ndev->mtu, apc->configured_mtu);
+
+ /* Pair with the queue-state stores above: a datapath reader that sees
+ * the gate open must also see the queue set it is about to index.
+ */
+ smp_wmb();
+
+ WRITE_ONCE(apc->port_is_up, true);
+ mana_start_txqs(apc);
+ if (carrier_ok)
+ netif_carrier_on(ndev);
+
+ /* The set that could not carry the user's indirection table is the one
+ * serving traffic now, so the table really is gone. Reporting it here
+ * rather than while the set was being built keeps a failed swap from
+ * clearing IFF_RXFH_CONFIGURED on a port that kept its old queues, and
+ * with them a table that is still valid and still programmed.
+ */
+ if (newq->rxfh_indir_lost)
+ ethtool_rxfh_indir_lost(ndev);
+
+ return 0;
+
+rollback:
+ netdev_err(ndev, "mana_publish_qset failed: %d, restoring previous queue set\n",
+ err);
+
+ /* The roles are swapped now: @newq is the set going away and @out_old
+ * is live again. Same ordering rule, leaving set first.
+ */
+ mana_qset_set_retiring(newq, true);
+ mana_qset_set_retiring(out_old, false);
+
+ mana_qset_install(apc, out_old);
+ apc->rss_state = apc->num_queues > 1 ? TRI_STATE_TRUE : TRI_STATE_FALSE;
+
+ if (netif_set_real_num_tx_queues(ndev, apc->num_queues) ||
+ netif_set_real_num_rx_queues(ndev, apc->num_queues)) {
+ /* The netdev queue counts no longer describe the restored
+ * apc->tx_qp[], so resuming TX could index past it. Leave the
+ * port stopped and the carrier down instead; that is visible
+ * to the admin and recoverable with a down/up.
+ *
+ * Steering can still point at @newq, which the caller frees
+ * next, so shut RX down at the vport first.
+ */
+ netdev_err(ndev, "failed to restore queue counts, closing the port\n");
+ mana_publish_give_up(apc);
+ return err;
+ }
+
+ if (mana_config_rss(apc, TRI_STATE_TRUE, true, true)) {
+ /* Steering may still point at the queue set the caller is
+ * about to free, and it cannot be repointed. Disable vport RX
+ * so the device stops delivering into those queues before they
+ * are destroyed, and stay down rather than run with steering
+ * that does not match apc->rxqs[].
+ */
+ netdev_err(ndev, "failed to restore RSS steering, closing the port\n");
+ mana_publish_give_up(apc);
+ return err;
+ }
+
+ WRITE_ONCE(ndev->mtu, apc->configured_mtu);
+
+ /* Same pairing as the success path: the restored queue set has to be
+ * visible before the gate reopens on it.
+ */
+ smp_wmb();
+
+ WRITE_ONCE(apc->port_is_up, true);
+ mana_start_txqs(apc);
+ if (carrier_ok)
+ netif_carrier_on(ndev);
+
+ /* out_old is live again on apc; caller must only free newq. */
+ return err;
+}
+
+/* Give live queues the debugfs nodes suppressed while they were built in a
+ * scratch context, whose names would collide under vport%d. Once the retiring
+ * set is gone the survivors take them.
+ *
+ * Idempotent: a carried-over queue keeps its node; suppressed creation leaves
+ * an error pointer, not NULL, so both read as "no node". Under RTNL.
+ */
+void mana_qset_debugfs_publish(struct mana_port_context *apc)
+{
+ unsigned int i;
+
+ ASSERT_RTNL();
+
+ if (IS_ERR_OR_NULL(apc->mana_port_debugfs))
+ return;
+
+ for (i = 0; i < apc->num_queues; i++) {
+ if (apc->tx_qp && apc->tx_qp[i] &&
+ IS_ERR_OR_NULL(apc->tx_qp[i]->mana_tx_debugfs))
+ mana_create_txq_debugfs(apc, i);
+
+ if (apc->rxqs && apc->rxqs[i] &&
+ IS_ERR_OR_NULL(apc->rxqs[i]->mana_rx_debugfs))
+ mana_create_rxq_debugfs(apc, i);
+ }
+}
+
+/* Tear down @qset, no longer installed on @apc, against @scratch so the live
+ * context never points at queues being freed.
+ */
+void mana_free_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch, struct mana_qset *qset)
+{
+ struct bpf_prog *retiring_prog;
+ unsigned int retiring_queues;
+
+ ASSERT_RTNL();
+
+ if (!qset->rxqs && !qset->tx_qp)
+ return;
+
+ /* Keep their completions off the netdev queues they now share. */
+ if (qset->tx_qp) {
+ unsigned int q;
+
+ for (q = 0; q < qset->num_queues; q++) {
+ if (qset->tx_qp[q])
+ WRITE_ONCE(qset->tx_qp[q]->txq.retiring, true);
+ }
+ }
+
+ /* A reader that sampled the retiring pointers after mana_publish_qset()
+ * installed the new set is not covered by the drain it did earlier.
+ * mana_xdp_xmit() is the case that matters: it runs from another
+ * device's NAPI, which this port's napi_synchronize() never waits for.
+ */
+ synchronize_net();
+
+ mana_qset_install(scratch, qset);
+
+ /* Teardown follows mana_dealloc_queues()' order, minus the vport RX
+ * disable: steering already points at the incoming set, and disabling
+ * vport RX would stop the set that is now live. Where publish could
+ * not repoint steering it disabled vport RX itself, so nothing is
+ * delivered here either way.
+ */
+
+ /* Note what this set owes the XDP program but leave the queues
+ * pointing at it: they are still polling, and a packet already in a
+ * retiring RQ must keep running the program rather than slip into the
+ * stack. The references are dropped once the queues are gone, below.
+ */
+ retiring_prog = mana_chn_xdp_peek(scratch);
+ retiring_queues = scratch->num_queues;
+
+ /* Drain packets the device has not completed before the SQs and SKB
+ * queues go away, or those SKBs and their DMA mappings are leaked.
+ *
+ * This runs before any RX teardown, as mana_dealloc_queues() does:
+ * unmapping RX buffers first would leave a wedged device free to keep
+ * writing into them for as long as the drain takes.
+ */
+ if (mana_drain_txqs(scratch)) {
+ /* The drain had to reset the function to stop the device
+ * touching those buffers. A function reset takes down every
+ * port on the adapter, not just this one, so rebuild them all
+ * - the same recovery mana_tx_timeout() relies on. A port that
+ * is already down has nothing to rebuild and its handler
+ * leaves it down.
+ */
+ struct mana_port_context *apc = netdev_priv(scratch->ndev);
+ struct mana_context *ac = apc->ac;
+ unsigned int i;
+
+ netdev_err(scratch->ndev,
+ "device reset while retiring a queue set, scheduling port reset\n");
+
+ for (i = 0; i < ac->num_ports; i++) {
+ if (!ac->ports[i])
+ continue;
+ queue_work(ac->per_port_queue_reset_wq,
+ &((struct mana_port_context *)
+ netdev_priv(ac->ports[i]))->queue_reset_work);
+ }
+ }
+
+ /* Traffic was still being steered at these queues moments ago, so
+ * fence each retiring RQ before its buffers are unmapped, again the
+ * order mana_dealloc_queues() uses. mana_destroy_rxq() does destroy
+ * the hardware RQ before unmapping anything, but the fence is what
+ * makes the device confirm it is done with the buffers first.
+ */
+ mana_fence_rqs(scratch);
+
+ mana_destroy_rxqs(scratch);
+
+ /* The queues are gone, so nothing can run the program any more. */
+ mana_chn_xdp_release(retiring_prog, retiring_queues);
+
+ mana_destroy_txq(scratch);
+ mana_cleanup_indir_table(scratch);
+ kfree(scratch->rxqs);
+ scratch->rxqs = NULL;
+
+ memset(qset, 0, sizeof(*qset));
+
+ /* This set is gone, so any EQ above the live queue count is now
+ * unreferenced. Release those vectors instead of holding them at the
+ * high-water mark. Safe here and only here: the retiring set's CQs
+ * have just been destroyed.
+ */
+ mana_shrink_eqs(apc, apc->num_queues);
+
+ /* Queues built through a scratch context carry no debugfs nodes,
+ * because both sets are alive during the swap and would collide on
+ * the same names. The retiring set's nodes are gone now, so the
+ * published queues can finally take those names.
+ */
+ mana_qset_debugfs_publish(apc);
+}
+
+/* --- end of pre-allocate + swap reconfiguration path ---------------------- */
+
int mana_detach(struct net_device *ndev, bool from_close)
{
struct mana_port_context *apc = netdev_priv(ndev);
@@ -3783,10 +5002,8 @@ int mana_detach(struct net_device *ndev, bool from_close)
if (apc->port_st_save) {
err = mana_dealloc_queues(ndev);
- if (err) {
+ if (err)
netdev_err(ndev, "%s failed to deallocate queues: %d\n", __func__, err);
- return err;
- }
}
if (!from_close) {
@@ -3823,6 +5040,7 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
apc->port_handle = INVALID_MANA_HANDLE;
apc->pf_filter_handle = INVALID_MANA_HANDLE;
apc->port_idx = port_idx;
+ apc->configured_mtu = ndev->mtu;
apc->link_cfg_error = 1;
apc->cqe_coalescing_enable = 0;
apc->cqe8_coalescing_enable = 0;
@@ -3837,6 +5055,10 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
apc->tx_dim_enabled = MANA_ADAPTIVE_TX_DEF;
}
+ err = mana_alloc_queue_stats(apc);
+ if (err)
+ goto free_net;
+
mutex_init(&apc->vport_mutex);
apc->vport_use_count = 0;
@@ -3859,7 +5081,7 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
err = mana_init_port(ndev);
if (err)
- goto free_net;
+ goto free_stats;
err = mana_rss_table_alloc(apc);
if (err)
@@ -3896,6 +5118,11 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
mana_cleanup_indir_table(apc);
reset_apc:
mana_cleanup_port_context(apc);
+free_stats:
+ /* The counter arrays are separate allocations, so free_netdev() does
+ * not release them with the port context.
+ */
+ mana_free_queue_stats(apc);
free_net:
*ndev_storage = NULL;
netdev_err(ndev, "Failed to probe vPort %d: %d\n", port_idx, err);
@@ -4236,6 +5463,14 @@ void mana_remove(struct gdma_dev *gd, bool suspending)
unregister_netdevice(ndev);
mana_cleanup_indir_table(apc);
+ mana_free_queue_stats(apc);
+
+ /* Clear the slot before the netdev goes away. A later port
+ * whose teardown has to reset the function walks ac->ports[]
+ * to schedule the rebuild, and would otherwise reach into the
+ * port freed here.
+ */
+ ac->ports[i] = NULL;
rtnl_unlock();
diff --git a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
index ece7ff9cc409a..a1c24d41903c1 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
@@ -271,7 +271,7 @@ static void mana_get_ethtool_stats(struct net_device *ndev,
data[i++] = *(u64 *)(phy_stats + mana_phy_stats[q].offset);
for (q = 0; q < num_queues; q++) {
- rx_stats = &apc->rxqs[q]->stats;
+ rx_stats = &apc->rxq_stats[q];
do {
start = u64_stats_fetch_begin(&rx_stats->syncp);
@@ -296,7 +296,7 @@ static void mana_get_ethtool_stats(struct net_device *ndev,
}
for (q = 0; q < num_queues; q++) {
- tx_stats = &apc->tx_qp[q]->txq.stats;
+ tx_stats = &apc->txq_stats[q];
do {
start = u64_stats_fetch_begin(&tx_stats->syncp);
@@ -648,52 +648,152 @@ static int mana_set_coalesce(struct net_device *ndev,
return 0;
}
-/* mana_set_channels - change the number of queues on a port
- *
- * Returns -EBUSY if RDMA holds the vport with EQs sized to the
- * current num_queues.
+/* A count change leaves every surviving queue configured as it was, so
+ * neither direction rebuilds: a reduction retires the tail, an increase
+ * builds only the queues added. On failure the existing queues keep running
+ * and the requested value is never replaced by a fallback. The vport is never
+ * torn down, so RDMA cannot take it mid-reconfiguration.
*/
static int mana_set_channels(struct net_device *ndev,
struct ethtool_channels *channels)
{
struct mana_port_context *apc = netdev_priv(ndev);
unsigned int new_count = channels->combined_count;
- unsigned int old_count = apc->num_queues;
+ struct mana_port_context *scratch;
+ struct mana_qset newq, oldq, freshq;
int err;
- /* Set channel_changing to block RDMA from grabbing the vport
- * during the detach/attach window. mana_cfg_vport() checks
- * this flag under vport_mutex and returns -EBUSY if set.
+ if (new_count < 1 || new_count > apc->max_queues) {
+ netdev_err(ndev, "Invalid combined_count %u (max %u)\n",
+ new_count, apc->max_queues);
+ return -EINVAL;
+ }
+
+ if (new_count == apc->num_queues)
+ return 0;
+
+ /* Down: no queues to swap, so record the count and resize the arrays
+ * indexed by it. apc->rxqs has to grow here because mana_open() goes
+ * straight to mana_alloc_queues() without rebuilding the port context.
+ *
+ * RDMA can own the vport while the port is down and derives an EQ from
+ * apc->eqs[] modulo apc->num_queues, so refuse while it is in use.
*/
mutex_lock(&apc->vport_mutex);
- if (!apc->port_is_up && apc->vport_use_count) {
+ if (!apc->port_is_up) {
+ struct mana_rxq **rxqs;
+
+ if (apc->vport_use_count) {
+ mutex_unlock(&apc->vport_mutex);
+ return -EBUSY;
+ }
+
+ rxqs = kzalloc_objs(struct mana_rxq *, new_count);
+ if (!rxqs) {
+ mutex_unlock(&apc->vport_mutex);
+ return -ENOMEM;
+ }
+
+ kfree(apc->rxqs);
+ apc->rxqs = rxqs;
+ apc->num_queues = new_count;
+ mutex_unlock(&apc->vport_mutex);
+ return 0;
+ }
+
+ /* Block RDMA from acquiring the vport for the duration.
+ *
+ * No vport_use_count test here, unlike the branch above: bringing the
+ * port up takes the vport itself, so the count is always non-zero. That
+ * reference is also what makes the swap safe - RAW QPs, the only users
+ * of apc->eqs[] modulo apc->num_queues, cannot exist while the ethernet
+ * port owns it.
+ */
+ if (apc->channel_changing) {
mutex_unlock(&apc->vport_mutex);
return -EBUSY;
}
apc->channel_changing = true;
mutex_unlock(&apc->vport_mutex);
- err = mana_pre_alloc_rxbufs(apc, ndev->mtu, new_count);
- if (err) {
- netdev_err(ndev, "Insufficient memory for new allocations");
+ scratch = mana_qset_scratch_alloc(apc);
+ if (!scratch) {
+ err = -ENOMEM;
goto clear_flag;
}
- err = mana_detach(ndev, false);
- if (err) {
- netdev_err(ndev, "mana_detach failed: %d\n", err);
- goto out;
+ /* A reduction keeps its queues configured identically, so carry them
+ * over and retire only the tail: no DMA ring, no hardware WQ object,
+ * and no old+new peak.
+ */
+ if (new_count < apc->num_queues) {
+ struct mana_qset tailq;
+
+ err = mana_split_qset(apc, scratch, new_count, &newq, &tailq);
+ if (err)
+ goto free_scratch; /* current qset untouched */
+
+ err = mana_publish_qset(apc, &newq, &oldq);
+ if (err) {
+ /* The old set is live again; drop only the containers
+ * built above, never the queues they point at.
+ */
+ mana_discard_split(&newq, &tailq);
+ goto free_scratch;
+ }
+
+ /* @oldq holds the original arrays and steering table. Every
+ * queue they referenced is now owned by either the published
+ * set or the tail, so only the containers are freed here.
+ */
+ kfree(oldq.tx_qp);
+ kfree(oldq.rxqs);
+ kfree(oldq.indir_table);
+ kfree(oldq.rxobj_table);
+
+ mana_free_qset(apc, scratch, &tailq);
+ goto free_scratch;
}
- apc->num_queues = new_count;
- err = mana_attach(ndev);
+ /* An increase does not change the queues that already exist either, so
+ * carry them over as well and build only the queues being added. The
+ * peak stays at the new count instead of old + new.
+ */
+ err = mana_grow_qset(apc, scratch, new_count, &newq, &freshq);
+ if (err)
+ goto free_scratch; /* current qset untouched, nothing to undo */
+
+ err = mana_publish_qset(apc, &newq, &oldq);
if (err) {
- apc->num_queues = old_count;
- netdev_err(ndev, "mana_attach failed: %d\n", err);
+ /* The old set is live again. Retire the queues that were just
+ * built - @freshq names exactly those - and then drop the
+ * merged containers without touching the carried-over queues.
+ */
+ mana_free_qset(apc, scratch, &freshq);
+ mana_discard_grow(&newq);
+ goto free_scratch;
}
-out:
- mana_pre_dealloc_rxbufs(apc);
+ /* Nothing is retired by a grow: every queue @oldq referenced is now
+ * part of the published set, and so is every queue in @freshq. Only
+ * the containers of both are released here.
+ */
+ kfree(oldq.tx_qp);
+ kfree(oldq.rxqs);
+ kfree(oldq.indir_table);
+ kfree(oldq.rxobj_table);
+ kfree(freshq.tx_qp);
+ kfree(freshq.rxqs);
+
+ /* A grow retires nothing, so mana_free_qset() never runs to hand out
+ * the debugfs names. The queues that were just added are the only
+ * ones missing a node, and no retiring set is holding their names.
+ */
+ mana_qset_debugfs_publish(apc);
+
+free_scratch:
+ mana_publish_close_if_needed(apc);
+ mana_qset_scratch_free(scratch);
clear_flag:
mutex_lock(&apc->vport_mutex);
apc->channel_changing = false;
@@ -714,19 +814,18 @@ static void mana_get_ringparam(struct net_device *ndev,
ring->tx_max_pending = MAX_TX_BUFFERS_PER_QUEUE;
}
+
static int mana_set_ringparam(struct net_device *ndev,
struct ethtool_ringparam *ring,
struct kernel_ethtool_ringparam *kernel_ring,
struct netlink_ext_ack *extack)
{
struct mana_port_context *apc = netdev_priv(ndev);
+ struct mana_port_context *scratch;
+ struct mana_qset newq, oldq;
u32 new_tx, new_rx;
- u32 old_tx, old_rx;
int err;
- old_tx = apc->tx_queue_size;
- old_rx = apc->rx_queue_size;
-
if (ring->tx_pending < MIN_TX_BUFFERS_PER_QUEUE) {
NL_SET_ERR_MSG_FMT(extack, "tx:%d less than the min:%d", ring->tx_pending,
MIN_TX_BUFFERS_PER_QUEUE);
@@ -744,32 +843,59 @@ static int mana_set_ringparam(struct net_device *ndev,
netdev_info(ndev, "Using nearest power of 2 values for Txq:%d Rxq:%d\n",
new_tx, new_rx);
- /* pre-allocating new buffers to prevent failures in mana_attach() later */
- apc->rx_queue_size = new_rx;
- err = mana_pre_alloc_rxbufs(apc, ndev->mtu, apc->num_queues);
- apc->rx_queue_size = old_rx;
- if (err) {
- netdev_err(ndev, "Insufficient memory for new allocations\n");
- return err;
+ if (new_rx == apc->rx_queue_size && new_tx == apc->tx_queue_size)
+ return 0;
+
+ /* Port is down: no queues to rebuild, just record the new sizes. */
+ if (!apc->port_is_up) {
+ apc->rx_queue_size = new_rx;
+ apc->tx_queue_size = new_tx;
+ return 0;
}
- err = mana_detach(ndev, false);
- if (err) {
- netdev_err(ndev, "mana_detach failed: %d\n", err);
- goto out;
+ /* Block RDMA from acquiring the vport for the duration. The vport
+ * itself is never released, so vport_use_count stays > 0.
+ */
+ mutex_lock(&apc->vport_mutex);
+ if (apc->channel_changing) {
+ mutex_unlock(&apc->vport_mutex);
+ return -EBUSY;
}
+ apc->channel_changing = true;
+ mutex_unlock(&apc->vport_mutex);
- apc->tx_queue_size = new_tx;
- apc->rx_queue_size = new_rx;
+ scratch = mana_qset_scratch_alloc(apc);
+ if (!scratch) {
+ err = -ENOMEM;
+ goto clear_flag;
+ }
- err = mana_attach(ndev);
+ err = mana_alloc_qset(apc, scratch, new_rx, new_tx,
+ apc->priv_flags, apc->configured_mtu,
+ apc->bpf_prog, &newq);
if (err) {
- netdev_err(ndev, "mana_attach failed: %d\n", err);
- apc->tx_queue_size = old_tx;
- apc->rx_queue_size = old_rx;
+ NL_SET_ERR_MSG_FMT(extack, "failed to change ring params: %d",
+ err);
+ goto free_scratch; /* current qset untouched */
}
-out:
- mana_pre_dealloc_rxbufs(apc);
+
+ err = mana_publish_qset(apc, &newq, &oldq);
+ if (err) {
+ NL_SET_ERR_MSG_FMT(extack, "failed to change ring params: %d",
+ err);
+ mana_free_qset(apc, scratch, &newq);
+ goto free_scratch;
+ }
+
+ mana_free_qset(apc, scratch, &oldq);
+
+free_scratch:
+ mana_publish_close_if_needed(apc);
+ mana_qset_scratch_free(scratch);
+clear_flag:
+ mutex_lock(&apc->vport_mutex);
+ apc->channel_changing = false;
+ mutex_unlock(&apc->vport_mutex);
return err;
}
@@ -795,11 +921,15 @@ static u32 mana_get_priv_flags(struct net_device *ndev)
return apc->priv_flags;
}
+/* MANA_PRIV_FLAG_USE_FULL_PAGE_RXBUF changes the RX buffer layout, so the
+ * queues have to be rebuilt.
+ */
static int mana_set_priv_flags(struct net_device *ndev, u32 priv_flags)
{
struct mana_port_context *apc = netdev_priv(ndev);
u32 changed = apc->priv_flags ^ priv_flags;
- u32 old_priv_flags = apc->priv_flags;
+ struct mana_port_context *scratch;
+ struct mana_qset newq, oldq;
int err = 0;
if (!changed)
@@ -809,54 +939,54 @@ static int mana_set_priv_flags(struct net_device *ndev, u32 priv_flags)
if (priv_flags & ~GENMASK(MANA_PRIV_FLAG_MAX - 1, 0))
return -EINVAL;
- apc->priv_flags = priv_flags;
-
- if (changed & BIT(MANA_PRIV_FLAG_USE_FULL_PAGE_RXBUF)) {
- if (!apc->port_is_up)
- return 0;
-
- /* If XDP is attached or MTU is jumbo, single-buffer-per-page
- * is already forced regardless of this flag. Skip the
- * expensive detach/attach cycle since nothing changes.
- */
- if (ndev->mtu + MANA_RXBUF_PAD > PAGE_SIZE / 2 ||
- mana_xdp_get(apc))
- return 0;
+ /* Only the RX buffer layout flag requires a queue rebuild. Anything
+ * else, a down port, or a configuration where single-buffer-per-page
+ * is already forced, just records the new value.
+ */
+ if (!(changed & BIT(MANA_PRIV_FLAG_USE_FULL_PAGE_RXBUF)) ||
+ !apc->port_is_up ||
+ ndev->mtu + MANA_RXBUF_PAD > PAGE_SIZE / 2 ||
+ mana_xdp_get(apc)) {
+ apc->priv_flags = priv_flags;
+ return 0;
+ }
- /* Block RDMA from grabbing the vport during detach/attach */
- mutex_lock(&apc->vport_mutex);
- apc->channel_changing = true;
+ /* Block RDMA from acquiring the vport for the duration. */
+ mutex_lock(&apc->vport_mutex);
+ if (apc->channel_changing) {
mutex_unlock(&apc->vport_mutex);
+ return -EBUSY;
+ }
+ apc->channel_changing = true;
+ mutex_unlock(&apc->vport_mutex);
- err = mana_pre_alloc_rxbufs(apc, ndev->mtu, apc->num_queues);
- if (err) {
- netdev_err(ndev,
- "Insufficient memory for new allocations\n");
- apc->priv_flags = old_priv_flags;
- goto clear_flag;
- }
+ scratch = mana_qset_scratch_alloc(apc);
+ if (!scratch) {
+ err = -ENOMEM;
+ goto clear_flag;
+ }
- err = mana_detach(ndev, false);
- if (err) {
- netdev_err(ndev, "mana_detach failed: %d\n", err);
- apc->priv_flags = old_priv_flags;
- goto out;
- }
+ err = mana_alloc_qset(apc, scratch, apc->rx_queue_size,
+ apc->tx_queue_size, priv_flags,
+ apc->configured_mtu, apc->bpf_prog, &newq);
+ if (err)
+ goto free_scratch; /* current qset and priv_flags untouched */
- err = mana_attach(ndev);
- if (err) {
- netdev_err(ndev, "mana_attach failed: %d\n", err);
- apc->priv_flags = old_priv_flags;
- }
+ err = mana_publish_qset(apc, &newq, &oldq);
+ if (err) {
+ mana_free_qset(apc, scratch, &newq);
+ goto free_scratch;
}
-out:
- mana_pre_dealloc_rxbufs(apc);
+ mana_free_qset(apc, scratch, &oldq);
+
+free_scratch:
+ mana_publish_close_if_needed(apc);
+ mana_qset_scratch_free(scratch);
clear_flag:
mutex_lock(&apc->vport_mutex);
apc->channel_changing = false;
mutex_unlock(&apc->vport_mutex);
-
return err;
}
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index 70a7f1fee5d3b..c54500700f6f2 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -672,6 +672,14 @@ enum {
/* Driver supports dynamic interrupt moderation - DIM */
#define GDMA_DRV_CAP_FLAG_1_DYN_INTERRUPT_MODERATION BIT(28)
+/* Driver recovers by itself when a queue resize fails: a failed resize leaves
+ * the queues that were already serving traffic in place, so the host does not
+ * have to bring the port back. This covers the resize itself failing. It does
+ * not promise recovery when restoring the previous queue set fails too, which
+ * leaves the port administratively down for the admin to bring back up.
+ */
+#define GDMA_DRV_CAP_FLAG_1_SELF_RECOVERY_ON_QUEUE_RESIZE_FAILURE BIT(31)
+
#define GDMA_DRV_CAP_FLAGS1 \
(GDMA_DRV_CAP_FLAG_1_EQ_SHARING_MULTI_VPORT | \
GDMA_DRV_CAP_FLAG_1_NAPI_WKDONE_FIX | \
@@ -688,7 +696,8 @@ enum {
GDMA_DRV_CAP_FLAG_1_HANDLE_STALL_SQ_RECOVERY | \
GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECOVERY | \
GDMA_DRV_CAP_FLAG_1_EQ_MSI_UNSHARE_MULTI_VPORT | \
- GDMA_DRV_CAP_FLAG_1_DYN_INTERRUPT_MODERATION)
+ GDMA_DRV_CAP_FLAG_1_DYN_INTERRUPT_MODERATION | \
+ GDMA_DRV_CAP_FLAG_1_SELF_RECOVERY_ON_QUEUE_RESIZE_FAILURE)
#define GDMA_DRV_CAP_FLAGS2 0
diff --git a/include/net/mana/mana.h b/include/net/mana/mana.h
index 83b7eff4646ea..8857e7739d2c2 100644
--- a/include/net/mana/mana.h
+++ b/include/net/mana/mana.h
@@ -102,7 +102,10 @@ struct mana_stats_rx {
u64 pkt_len0_err;
u64 coalesced_cqe[MANA_CQE_COAL_PKTS_8 - 1];
struct u64_stats_sync syncp;
-};
+ /* Per-port array indexed by queue, so keep entries on separate cache
+ * lines: queues polled on different CPUs would bounce a shared one.
+ */
+} ____cacheline_aligned_in_smp;
struct mana_stats_tx {
u64 packets;
@@ -117,7 +120,8 @@ struct mana_stats_tx {
u64 csum_partial;
u64 mana_map_err;
struct u64_stats_sync syncp;
-};
+ /* Per-queue array entry, same cache line reasoning as the RX side. */
+} ____cacheline_aligned_in_smp;
struct mana_txq {
struct gdma_queue *gdma_sq;
@@ -143,7 +147,17 @@ struct mana_txq {
bool napi_initialized;
- struct mana_stats_tx stats;
+ /* Value of mana_context.reset_gen when this queue was created. */
+ u32 reset_gen;
+
+ /* Unpublished and draining. Its completions must leave flow control
+ * alone: net_txq is shared with its replacement, and a draining queue
+ * always looks like it has room.
+ */
+ bool retiring;
+
+ /* Points into apc->txq_stats[], which outlives the queue. */
+ struct mana_stats_tx *stats;
};
/* skb data and frags dma mappings */
@@ -405,7 +419,8 @@ struct mana_rxq {
u32 buf_index;
- struct mana_stats_rx stats;
+ /* Points into apc->rxq_stats[], which outlives the queue. */
+ struct mana_stats_rx *stats;
struct bpf_prog __rcu *bpf_prog;
struct xdp_rxq_info xdp_rxq;
@@ -537,6 +552,13 @@ struct mana_context {
u8 bm_hostmode;
struct mana_ethtool_hc_stats hc_stats;
+
+ /* Bumped on every PCI function reset. A queue created before the
+ * current value can no longer be reached by the device, so its buffers
+ * need no drain. Written under RTNL, read locklessly.
+ */
+ u32 reset_gen;
+
struct workqueue_struct *per_port_queue_reset_wq;
/* Workqueue for querying hardware stats */
struct delayed_work gf_stats_work;
@@ -559,7 +581,12 @@ struct mana_port_context {
u8 mac_addr[ETH_ALEN];
+ /* EQ pool, owned by the port rather than a queue set: EQs are bound to
+ * MSI-X vectors, which a swap must not double-book. Sized to
+ * max_queues; num_eqs is how many exist.
+ */
struct mana_eq *eqs;
+ unsigned int num_eqs;
struct dentry *mana_eqs_debugfs;
enum TRI_STATE rss_state;
@@ -572,7 +599,7 @@ struct mana_port_context {
/* Indirection Table for RX & TX. The values are queue indexes */
u32 *indir_table;
- u32 indir_table_sz;
+ u32 indir_table_sz;
/* Indirection table containing RxObject Handles */
mana_handle_t *rxobj_table;
@@ -600,9 +627,21 @@ struct mana_port_context {
unsigned int max_queues;
unsigned int num_queues;
+ /* Per-queue counters, max_queues entries each. Allocated at probe and
+ * freed at remove, never on queue teardown, so a reconfiguration does
+ * not reset them.
+ */
+ struct mana_stats_rx *rxq_stats;
+ struct mana_stats_tx *txq_stats;
+
unsigned int rx_queue_size;
unsigned int tx_queue_size;
+ /* MTU the RX queues were built for. Equal to ndev->mtu except during a
+ * swap, when the new set is built before ndev->mtu is updated.
+ */
+ int configured_mtu;
+
mana_handle_t port_handle;
mana_handle_t pf_filter_handle;
@@ -616,6 +655,13 @@ struct mana_port_context {
*/
bool channel_changing;
+ /* mana_publish_qset() could neither publish the new set nor restore the
+ * old one. Vport RX is already off; the port still has to be closed,
+ * which mana_publish_close_if_needed() does once the caller has
+ * released the set that failed.
+ */
+ bool publish_dead_end;
+
/* Net shaper handle*/
struct net_shaper_handle handle;
@@ -661,6 +707,39 @@ struct mana_port_context {
u32 steer_cqe_coalescing;
};
+/* The queue-related fields of mana_port_context that can be swapped as a
+ * unit. The vport (port_handle, vport_use_count) is not part of it and is
+ * never touched by a swap.
+ */
+struct mana_qset {
+ struct mana_tx_qp **tx_qp;
+ struct mana_rxq **rxqs;
+
+ u32 *indir_table;
+ u32 indir_table_sz;
+ mana_handle_t *rxobj_table;
+ mana_handle_t default_rxobj;
+
+ unsigned int num_queues;
+ unsigned int rx_queue_size;
+ unsigned int tx_queue_size;
+ u32 priv_flags;
+
+ /* MTU and XDP program the RX buffers of this set were sized for.
+ * Both feed mana_get_rxbuf_cfg(), so they are part of the queue-set
+ * configuration and must be swapped atomically with the queues.
+ */
+ int mtu;
+ struct bpf_prog *bpf_prog;
+
+ /* The user's RSS indirection table could not be carried onto this set,
+ * so a default one was generated for it. mana_publish_qset() tells the
+ * core once this set is live; until then the port is still running on
+ * a set where the user's table applies.
+ */
+ bool rxfh_indir_lost;
+};
+
netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev);
int mana_config_rss(struct mana_port_context *ac, enum TRI_STATE rx,
bool update_hash, bool update_tab);
@@ -670,6 +749,32 @@ int mana_alloc_queues(struct net_device *ndev);
int mana_attach(struct net_device *ndev);
int mana_detach(struct net_device *ndev, bool from_close);
+/* Pre-allocate + swap reconfiguration. Allocation and teardown run against a
+ * scratch context, so the live port context is mutated only inside
+ * mana_publish_qset() with TX disabled. Both sets share a port-owned EQ pool.
+ */
+struct mana_port_context *mana_qset_scratch_alloc(struct mana_port_context *apc);
+void mana_qset_scratch_free(struct mana_port_context *scratch);
+int mana_alloc_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch,
+ unsigned int rx_queue_size, unsigned int tx_queue_size,
+ u32 priv_flags, int mtu, struct bpf_prog *bpf_prog,
+ struct mana_qset *out);
+int mana_split_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch, unsigned int new_count,
+ struct mana_qset *out_new, struct mana_qset *out_tail);
+void mana_discard_split(struct mana_qset *newq, struct mana_qset *tailq);
+int mana_grow_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch, unsigned int new_count,
+ struct mana_qset *out_new, struct mana_qset *out_fresh);
+void mana_discard_grow(struct mana_qset *newq);
+int mana_publish_qset(struct mana_port_context *apc, struct mana_qset *newq,
+ struct mana_qset *out_old);
+void mana_publish_close_if_needed(struct mana_port_context *apc);
+void mana_free_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch, struct mana_qset *qset);
+void mana_qset_debugfs_publish(struct mana_port_context *apc);
+
void mana_dim_change(struct mana_cq *cq, bool enable);
int mana_probe(struct gdma_dev *gd, bool resuming);
@@ -685,6 +790,8 @@ u32 mana_run_xdp(struct net_device *ndev, struct mana_rxq *rxq,
struct xdp_buff *xdp, void *buf_va, uint pkt_len);
struct bpf_prog *mana_xdp_get(struct mana_port_context *apc);
void mana_chn_setxdp(struct mana_port_context *apc, struct bpf_prog *prog);
+struct bpf_prog *mana_chn_xdp_peek(struct mana_port_context *apc);
+void mana_chn_xdp_release(struct bpf_prog *prog, unsigned int num_queues);
int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf);
int mana_query_gf_stats(struct mana_context *ac);
int mana_query_link_cfg(struct mana_port_context *apc);
|
| 3/2 | 2026/08/13 05:12 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 36502 output: 25 thoughts: 471 Thoughts: **Analyzing the Reconfiguration Patch for the MANA Driver** Okay, so I've got this patch that fundamentally changes how queue sets are reconfigured in the Microsoft Azure Network Adapter (MANA) driver. It's a pretty substantial change, with functions like `mana_qset_scratch_alloc`, `mana_split_qset`, and all those others involved in allocation, growth, and freeing of the queue sets. This is clearly a big deal for things like MTU changes, channel configurations, ring parameter adjustments, and even XDP. My immediate thought is, can we actually *test* this? Is this even reachable in a standard environment like QEMU or Google Compute Engine (GCE)? MANA is the Microsoft Azure Network Adapter, so it's a PCI device, right? Wait a minute... does standard QEMU even emulate MANA? No, it's specific to Azure's infrastructure. It's a hardware or virtual hardware device. Okay, so QEMU is out. Can we fuzz it? If it's a PCI device, syzkaller *might* be able to get at it through PCI fuzzing, assuming that's enabled. But, let me double-check the instructions… ah, here it is: "Code in vendor-specific PCIe switch, SmartNIC, or GPU drivers (e.g., mlxsw, pds_core, qed, ionic, amdgpu) that require physical PCIe hardware cards not emulated in standard QEMU." Is MANA in that category? It's a virtualized network adapter, yes, but it’s still a PCI device in Azure. But wait, syzkaller *doesn't* emulate MANA directly. It can fuzz PCI devices in *some* cases if they are emulated, or if it utilizes the PCI fuzzing framework. Okay, so syzkaller has some PCI fuzzing, but it’s limited. Alright, what about GCE? Nope, GCE uses its own NICs like `gve` or `virtio-net`. So, GCE is a dead end. QEMU and GCE are out. Okay, let's circle back to the syzkaller approach. Does syzkaller have a syzlang description for MANA? That would be the best way to fuzz it! I need to check if there is one. |
| 4/2 | 2026/08/13 05:12 | tool | grepper |
0mError:no matches Args: {
"Expression": "mana",
"PathPrefix": "sys/linux/"
}Results: {
"error": "no matches"
} |
| 5/2 | 2026/08/13 05:12 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 37515 output: 34 thoughts: 295 Thoughts: **MANA WorthFuzzing Decision** Okay, so there's no syzlang description for MANA, which is the first thing I check. Now, following the instructions, I need to consider whether this code path is worth fuzzing. I remember the critical exclusion criteria: specifically, "Code paths that are impossible to reach in virtualized environments like GCE or QEMU, even when utilizing software-emulated hardware". This immediately triggers a thought process; is MANA something we can get at? MANA, as I recall, is the Microsoft Azure Network Adapter. That screams "vendor-specific PCI device" to me, the kind tied to Hyper-V and Azure environments. It's not standard. And because it isn't something emulated in standard QEMU, it falls squarely into that exclusion: it's a code path I *can't* reach in a normal QEMU setup. Just to be absolutely certain, I'm going to quickly confirm this in the code. I'll search for it, probably in `drivers/net/ethernet/microsoft/mana/`, and yes, it *is* a PCI device. Specifically, a virtualized hardware device provided by the Azure hypervisor. So, it is most certainly *not* emulated in standard QEMU. That settles it. Therefore, `WorthFuzzing=false`. |
| 6/2 | 2026/08/13 05:12 | tool | grepper |
0mArgs:{
"Expression": "mana_probe",
"PathPrefix": "drivers/net/ethernet/microsoft/mana/"
}Results: {
"Output": "drivers/net/ethernet/microsoft/mana/gdma_main.c=178=static int mana_gd_query_max_resources(struct pci_dev *pdev)\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-262-\ndrivers/net/ethernet/microsoft/mana/gdma_main.c:263:\t/* Cap to the same limit used by mana_probe() for port instantiation,\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-264-\t * so MSI-X and queue budgeting matches the actual port count.\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c=2331=static int mana_gd_probe(struct pci_dev *pdev, const struct pci_device_id *ent)\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2386-\ndrivers/net/ethernet/microsoft/mana/gdma_main.c:2387:\terr = mana_probe(\u0026gc-\u003emana, false);\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2388-\tif (err)\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2425-\t * Hardware could be in recovery mode and the HWC returns TIMEDOUT or\ndrivers/net/ethernet/microsoft/mana/gdma_main.c:2426:\t * EPROTO from mana_gd_setup(), mana_probe() or mana_rdma_probe(), or\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2427-\t * we received a hardware reset event over HWC interrupt. In this case,\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c=2491=int mana_gd_resume(struct pci_dev *pdev)\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2499-\ndrivers/net/ethernet/microsoft/mana/gdma_main.c:2500:\terr = mana_probe(\u0026gc-\u003emana, true);\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2501-\tif (err)\n--\ndrivers/net/ethernet/microsoft/mana/mana_en.c=4982=int mana_detach(struct net_device *ndev, bool from_close)\n--\ndrivers/net/ethernet/microsoft/mana/mana_en.c-5016-\ndrivers/net/ethernet/microsoft/mana/mana_en.c:5017:static int mana_probe_port(struct mana_context *ac, int port_idx,\ndrivers/net/ethernet/microsoft/mana/mana_en.c-5018-\t\t\t struct net_device **ndev_storage)\n--\ndrivers/net/ethernet/microsoft/mana/mana_en.c=5268=static void mana_gf_stats_work_handler(struct work_struct *work)\n--\ndrivers/net/ethernet/microsoft/mana/mana_en.c-5290-\ndrivers/net/ethernet/microsoft/mana/mana_en.c:5291:int mana_probe(struct gdma_dev *gd, bool resuming)\ndrivers/net/ethernet/microsoft/mana/mana_en.c-5292-{\n--\ndrivers/net/ethernet/microsoft/mana/mana_en.c-5365-\t\tfor (i = 0; i \u003c ac-\u003enum_ports; i++) {\ndrivers/net/ethernet/microsoft/mana/mana_en.c:5366:\t\t\terr = mana_probe_port(ac, i, \u0026ac-\u003eports[i]);\ndrivers/net/ethernet/microsoft/mana/mana_en.c-5367-\t\t\t/* Log the port for which the probe failed, stop probing\n"
} |
| 7/2 | 2026/08/13 05:12 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 39107 output: 83 thoughts: 103 Thoughts: **My Assessment of the Device Driver** Okay, so I've confirmed it. This `mana_gd_probe` function is definitely a PCI device driver; the fact that it takes a `struct pci_dev *pdev` is the giveaway. Now, the key is that this driver appears to be specific to Microsoft Azure, which is not terribly uncommon, but definitely relevant. The problem is that it isn't emulated in the standard QEMU setup, and I know from experience that GCE also doesn't provide the right environment for this driver. Given this reality, trying to fuzz this driver in a virtualized environment is simply a non-starter. I have to call it. WorthFuzzing is *false*. The driver's vendor-specific nature and lack of emulation in the common virtualization tools mean I'm just spinning my wheels if I attempt to work on it now. It would be a waste of resources. |
| 8/2 | 2026/08/13 05:12 | tool | set-results |
0mArgs:{
"EnableConfigs": [],
"FocusSymbols": [],
"Reasoning": "The patch modifies the Microsoft Azure Network Adapter (MANA) driver, which is a vendor-specific PCI device driver for Azure environments. This hardware is not emulated in standard QEMU or GCE, making the code paths unreachable during standard fuzzing.",
"WorthFuzzing": false
}Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"Reasoning": "The patch modifies the Microsoft Azure Network Adapter (MANA) driver, which is a vendor-specific PCI device driver for Azure environments. This hardware is not emulated in standard QEMU or GCE, making the code paths unreachable during standard fuzzing.",
"WorthFuzzing": false
} |
| 9/1 | 2026/08/13 05:12 | action | If |
0mArgs:{
"WorthFuzzing": false
}Results: null |
| Total Calls | Total Tokens | Avg Tokens | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|
| Total Calls | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|