| Seq | Timestamp | Type | Name | Duration |
|---|---|---|---|---|
| 0/0 | 2026/09/09 22:47 | flow | patch-triage |
0mArgs:null Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"KMSANReasoning": "",
"NeedsKMSAN": false,
"Reasoning": "The patch modifies the Microsoft Azure Network Adapter (MANA) Ethernet driver (drivers/net/ethernet/microsoft/mana/ and include/net/mana/). This driver targets vendor-specific hardware/SmartNICs on Microsoft Azure / Hyper-V environments and is not emulated in standard QEMU or virtualized fuzzing environments. As such, the modified code is structurally unreachable during fuzzing.",
"WorthFuzzing": false
} |
| 1/1 | 2026/09/09 22:47 | action | read-patch-diff |
0mArgs:null Results: {
"PatchDiff": "commit fccf2b5a44a87f73e0b901245f87ab4538bb40df\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Wed Sep 9 22:47:24 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/drivers/net/ethernet/microsoft/mana/mana_bpf.c b/drivers/net/ethernet/microsoft/mana/mana_bpf.c\nindex 70bb8c9dc3413..3acec1bb02a34 100644\n--- a/drivers/net/ethernet/microsoft/mana/mana_bpf.c\n+++ b/drivers/net/ethernet/microsoft/mana/mana_bpf.c\n@@ -59,6 +59,11 @@ int mana_xdp_xmit(struct net_device *ndev, int n, struct xdp_frame **frames,\n \tif (unlikely(!apc-\u003eport_is_up))\n \t\treturn 0;\n \n+\t/* Pair with the smp_wmb() in mana_publish_qset() before reading queue\n+\t * state.\n+\t */\n+\tsmp_rmb();\n+\n \tq_idx = smp_processor_id() % ndev-\u003ereal_num_tx_queues;\n \n \tfor (i = 0; i \u003c n; i++) {\n@@ -68,7 +73,7 @@ int mana_xdp_xmit(struct net_device *ndev, int n, struct xdp_frame **frames,\n \t\tcount++;\n \t}\n \n-\ttx_stats = \u0026apc-\u003etx_qp[q_idx]-\u003etxq.stats;\n+\ttx_stats = apc-\u003etx_qp[q_idx]-\u003etxq.stats;\n \n \tu64_stats_update_begin(\u0026tx_stats-\u003esyncp);\n \ttx_stats-\u003exdp_xmit += count;\n@@ -95,7 +100,7 @@ u32 mana_run_xdp(struct net_device *ndev, struct mana_rxq *rxq,\n \n \tact = bpf_prog_run_xdp(prog, xdp);\n \n-\trx_stats = \u0026rxq-\u003estats;\n+\trx_stats = mana_rxq_stats(rxq);\n \n \tswitch (act) {\n \tcase XDP_PASS:\n@@ -172,6 +177,8 @@ static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,\n \t\t\tstruct netlink_ext_ack *extack)\n {\n \tstruct mana_port_context *apc = netdev_priv(ndev);\n+\tstruct mana_port_context *scratch;\n+\tstruct mana_qset newq, oldq;\n \tstruct bpf_prog *old_prog;\n \tstruct gdma_context *gc;\n \tint err;\n@@ -191,46 +198,44 @@ static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,\n \t\treturn -EOPNOTSUPP;\n \t}\n \n-\t/* One refcnt of the prog is hold by the caller already, so\n-\t * don't increase refcnt for this one.\n-\t */\n-\tapc-\u003ebpf_prog = prog;\n-\n \tif (apc-\u003eport_is_up) {\n-\t\t/* Re-create rxq's after xdp prog was loaded or unloaded.\n-\t\t * Ex: re create rxq's to switch from full pages to smaller\n-\t\t * size page fragments when xdp prog is unloaded and\n-\t\t * vice-versa.\n-\t\t */\n-\n-\t\t/* Pre-allocate buffers to prevent failure in mana_attach */\n-\t\terr = mana_pre_alloc_rxbufs(apc, ndev-\u003emtu, apc-\u003enum_queues);\n-\t\tif (err) {\n+\t\tscratch = mana_qset_scratch_alloc(apc);\n+\t\tif (!scratch) {\n \t\t\tNL_SET_ERR_MSG_MOD(extack,\n-\t\t\t\t\t \"XDP: Insufficient memory for tx/rx re-config\");\n-\t\t\treturn err;\n+\t\t\t\t\t \"XDP: Insufficient memory for re-config\");\n+\t\t\treturn -ENOMEM;\n \t\t}\n \n-\t\terr = mana_detach(ndev, false);\n+\t\terr = mana_alloc_qset(apc, scratch,\n+\t\t\t\t apc-\u003erx_queue_size, apc-\u003etx_queue_size,\n+\t\t\t\t apc-\u003epriv_flags, apc-\u003econfigured_mtu,\n+\t\t\t\t prog, \u0026newq);\n \t\tif (err) {\n-\t\t\tnetdev_err(ndev,\n-\t\t\t\t \"mana_detach failed at xdp set: %d\\n\", err);\n \t\t\tNL_SET_ERR_MSG_MOD(extack,\n-\t\t\t\t\t \"XDP: Re-config failed at detach\");\n-\t\t\tgoto err_dealloc_rxbuffs;\n+\t\t\t\t\t \"XDP: Re-config failed at alloc\");\n+\t\t\tmana_qset_scratch_free(scratch);\n+\t\t\treturn err;\n \t\t}\n \n-\t\terr = mana_attach(ndev);\n+\t\terr = mana_publish_qset(apc, \u0026newq, \u0026oldq);\n \t\tif (err) {\n-\t\t\tnetdev_err(ndev,\n-\t\t\t\t \"mana_attach failed at xdp set: %d\\n\", err);\n \t\t\tNL_SET_ERR_MSG_MOD(extack,\n-\t\t\t\t\t \"XDP: Re-config failed at attach\");\n-\t\t\tgoto err_dealloc_rxbuffs;\n+\t\t\t\t\t \"XDP: Re-config failed at publish\");\n+\t\t\tmana_free_qset(scratch, \u0026newq);\n+\t\t\t/* Free the queues before closing their shared EQ pool.\n+\t\t\t */\n+\t\t\tmana_publish_close_if_needed(apc);\n+\t\t\tmana_qset_scratch_free(scratch);\n+\t\t\treturn err;\n \t\t}\n \n-\t\tmana_chn_setxdp(apc, prog);\n-\t\tmana_pre_dealloc_rxbufs(apc);\n+\t\tmana_free_qset(scratch, \u0026oldq);\n+\t\tmana_qset_scratch_free(scratch);\n+\t} else {\n+\t\t/* Use the caller's program reference; mana_open() installs it\n+\t\t * on queues.\n+\t\t */\n+\t\tapc-\u003ebpf_prog = prog;\n \t}\n \n \tif (old_prog)\n@@ -243,11 +248,6 @@ static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,\n \t\tndev-\u003emax_mtu = gc-\u003eadapter_mtu - ETH_HLEN;\n \n \treturn 0;\n-\n-err_dealloc_rxbuffs:\n-\tapc-\u003ebpf_prog = old_prog;\n-\tmana_pre_dealloc_rxbufs(apc);\n-\treturn err;\n }\n \n int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf)\n@@ -262,3 +262,27 @@ int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf)\n \t\treturn -EOPNOTSUPP;\n \t}\n }\n+\n+struct bpf_prog *mana_chn_xdp_peek(struct mana_port_context *apc)\n+{\n+\tASSERT_RTNL();\n+\n+\tif (!apc-\u003erxqs || !apc-\u003erxqs[0])\n+\t\treturn NULL;\n+\n+\treturn rtnl_dereference(apc-\u003erxqs[0]-\u003ebpf_prog);\n+}\n+\n+/* Keep the per-queue program pointers until RX polling stops. */\n+void mana_chn_xdp_release(struct bpf_prog *prog, unsigned int num_queues)\n+{\n+\tunsigned int i;\n+\n+\tASSERT_RTNL();\n+\n+\tif (!prog)\n+\t\treturn;\n+\n+\tfor (i = 0; i \u003c num_queues; i++)\n+\t\tbpf_prog_put(prog);\n+}\ndiff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c\nindex 7a1ac853e3abc..29f54f91cb009 100644\n--- a/drivers/net/ethernet/microsoft/mana/mana_en.c\n+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c\n@@ -90,6 +90,17 @@ static int mana_open(struct net_device *ndev)\n \tsmp_wmb();\n \n \tnetif_tx_wake_all_queues(ndev);\n+\n+\t/* Undo a forced carrier-off unless a disconnect is pending behind RTNL.\n+\t */\n+\tif (apc-\u003ecarrier_forced_off) {\n+\t\tu32 ev = READ_ONCE(apc-\u003eac-\u003elink_event);\n+\n+\t\tapc-\u003ecarrier_forced_off = false;\n+\t\tif (ev != HWC_DATA_HW_LINK_DISCONNECT)\n+\t\t\tnetif_carrier_on(ndev);\n+\t}\n+\n \tnetdev_dbg(ndev, \"%s successful\\n\", __func__);\n \treturn 0;\n }\n@@ -106,6 +117,7 @@ static int mana_close(struct net_device *ndev)\n \n static void mana_link_state_handle(struct work_struct *w)\n {\n+\tstruct mana_port_context *apc;\n \tstruct mana_context *ac;\n \tstruct net_device *ndev;\n \tu32 link_event;\n@@ -131,6 +143,9 @@ static void mana_link_state_handle(struct work_struct *w)\n \t\tif (!ndev)\n \t\t\tcontinue;\n \n+\t\tapc = netdev_priv(ndev);\n+\t\tapc-\u003ecarrier_forced_off = false;\n+\n \t\tif (link_up) {\n \t\t\tnetif_carrier_on(ndev);\n \n@@ -312,8 +327,8 @@ static void mana_per_port_queue_reset_work_handler(struct work_struct *work)\n \n \trtnl_lock();\n \n-\t/* Block RDMA from grabbing the vport during the detach/attach\n-\t * window, same as mana_set_channels().\n+\t/* Exclude RDMA across detach/attach; RTNL serializes channel_changing\n+\t * writers.\n \t */\n \tmutex_lock(\u0026apc-\u003evport_mutex);\n \tapc-\u003echannel_changing = true;\n@@ -366,13 +381,22 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev)\n \tif (unlikely(!apc-\u003eport_is_up))\n \t\tgoto tx_drop;\n \n+\t/* Pair with mana_publish_qset()'s pre-gate smp_wmb(): observe queue\n+\t * fields after reading port_is_up.\n+\t */\n+\tsmp_rmb();\n+\n+\t/* Retiring RXQs may use indices beyond the live queue count. */\n+\tif (unlikely(txq_idx \u003e= apc-\u003enum_queues))\n+\t\tgoto tx_drop_count;\n+\n \tif (skb_cow_head(skb, MANA_HEADROOM))\n \t\tgoto tx_drop_count;\n \n \ttxq = \u0026apc-\u003etx_qp[txq_idx]-\u003etxq;\n \tgdma_sq = txq-\u003egdma_sq;\n \tcq = \u0026apc-\u003etx_qp[txq_idx]-\u003etx_cq;\n-\ttx_stats = \u0026txq-\u003estats;\n+\ttx_stats = txq-\u003estats;\n \n \tBUILD_BUG_ON(MAX_TX_WQE_SGL_ENTRIES != MANA_MAX_TX_WQE_SGL_ENTRIES);\n \tif (MAX_SKB_FRAGS + 2 \u003e MAX_TX_WQE_SGL_ENTRIES \u0026\u0026\n@@ -551,7 +575,7 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev)\n \t/* Populated the packet and bytes counters based on post GSO packet\n \t * calculations\n \t */\n-\ttx_stats = \u0026txq-\u003estats;\n+\ttx_stats = txq-\u003estats;\n \tu64_stats_update_begin(\u0026tx_stats-\u003esyncp);\n \ttx_stats-\u003epackets += num_gso_seg;\n \ttx_stats-\u003ebytes += len + ((num_gso_seg - 1) * gso_hs);\n@@ -597,15 +621,15 @@ static void mana_get_stats64(struct net_device *ndev,\n \t\t\t struct rtnl_link_stats64 *st)\n {\n \tstruct mana_port_context *apc = netdev_priv(ndev);\n-\tunsigned int num_queues = apc-\u003enum_queues;\n \tstruct mana_stats_rx *rx_stats;\n \tstruct mana_stats_tx *tx_stats;\n+\tunsigned int num_queues;\n \tunsigned int start;\n \tu64 packets, bytes;\n \tint q;\n \n-\tif (!apc-\u003eport_is_up)\n-\t\treturn;\n+\t/* Report even while down; dev_get_stats() zeroes its output. */\n+\tnum_queues = apc-\u003emax_queues;\n \n \tnetdev_stats_to_stats64(st, \u0026ndev-\u003estats);\n \n@@ -615,7 +639,18 @@ static void mana_get_stats64(struct net_device *ndev,\n \tst-\u003erx_missed_errors = apc-\u003eac-\u003ehc_stats.hc_rx_discards_no_wqe;\n \n \tfor (q = 0; q \u003c num_queues; q++) {\n-\t\trx_stats = \u0026apc-\u003erxqs[q]-\u003estats;\n+\t\trx_stats = \u0026apc-\u003erxq_stats[q];\n+\n+\t\tdo {\n+\t\t\tstart = u64_stats_fetch_begin(\u0026rx_stats-\u003esyncp);\n+\t\t\tpackets = rx_stats-\u003epackets;\n+\t\t\tbytes = rx_stats-\u003ebytes;\n+\t\t} while (u64_stats_fetch_retry(\u0026rx_stats-\u003esyncp, start));\n+\n+\t\tst-\u003erx_packets += packets;\n+\t\tst-\u003erx_bytes += bytes;\n+\n+\t\trx_stats = \u0026apc-\u003erxq_stats_ret[q];\n \n \t\tdo {\n \t\t\tstart = u64_stats_fetch_begin(\u0026rx_stats-\u003esyncp);\n@@ -628,7 +663,7 @@ static void mana_get_stats64(struct net_device *ndev,\n \t}\n \n \tfor (q = 0; q \u003c num_queues; q++) {\n-\t\ttx_stats = \u0026apc-\u003etx_qp[q]-\u003etxq.stats;\n+\t\ttx_stats = \u0026apc-\u003etxq_stats[q];\n \n \t\tdo {\n \t\t\tstart = u64_stats_fetch_begin(\u0026tx_stats-\u003esyncp);\n@@ -878,32 +913,37 @@ int mana_pre_alloc_rxbufs(struct mana_port_context *mpc, int new_mtu, int num_qu\n static int mana_change_mtu(struct net_device *ndev, int new_mtu)\n {\n \tstruct mana_port_context *mpc = netdev_priv(ndev);\n-\tunsigned int old_mtu = ndev-\u003emtu;\n+\tstruct mana_port_context *scratch;\n+\tstruct mana_qset newq, oldq;\n \tint err;\n \n-\t/* Pre-allocate buffers to prevent failure in mana_attach later */\n-\terr = mana_pre_alloc_rxbufs(mpc, new_mtu, mpc-\u003enum_queues);\n-\tif (err) {\n-\t\tnetdev_err(ndev, \"Insufficient memory for new MTU\\n\");\n-\t\treturn err;\n+\tif (!mpc-\u003eport_is_up) {\n+\t\tmpc-\u003econfigured_mtu = new_mtu;\n+\t\tWRITE_ONCE(ndev-\u003emtu, new_mtu);\n+\t\treturn 0;\n \t}\n \n-\terr = mana_detach(ndev, false);\n-\tif (err) {\n-\t\tnetdev_err(ndev, \"mana_detach failed: %d\\n\", err);\n-\t\tgoto out;\n-\t}\n+\tscratch = mana_qset_scratch_alloc(mpc);\n+\tif (!scratch)\n+\t\treturn -ENOMEM;\n \n-\tWRITE_ONCE(ndev-\u003emtu, new_mtu);\n+\terr = mana_alloc_qset(mpc, scratch, mpc-\u003erx_queue_size,\n+\t\t\t mpc-\u003etx_queue_size, mpc-\u003epriv_flags, new_mtu,\n+\t\t\t mpc-\u003ebpf_prog, \u0026newq);\n+\tif (err)\n+\t\tgoto free_scratch;\n \n-\terr = mana_attach(ndev);\n+\terr = mana_publish_qset(mpc, \u0026newq, \u0026oldq);\n \tif (err) {\n-\t\tnetdev_err(ndev, \"mana_attach failed: %d\\n\", err);\n-\t\tWRITE_ONCE(ndev-\u003emtu, old_mtu);\n+\t\tmana_free_qset(scratch, \u0026newq);\n+\t\tgoto free_scratch;\n \t}\n \n-out:\n-\tmana_pre_dealloc_rxbufs(mpc);\n+\tmana_free_qset(scratch, \u0026oldq);\n+\n+free_scratch:\n+\tmana_publish_close_if_needed(mpc);\n+\tmana_qset_scratch_free(scratch);\n \treturn err;\n }\n \n@@ -1036,6 +1076,105 @@ static void mana_cleanup_port_context(struct mana_port_context *apc)\n \tapc-\u003erxqs = NULL;\n }\n \n+/* Port lifetime preserves counters across queue replacement. */\n+static int mana_alloc_queue_stats(struct mana_port_context *apc)\n+{\n+\tunsigned int i;\n+\n+\tapc-\u003erxq_stats = kcalloc(apc-\u003emax_queues, sizeof(*apc-\u003erxq_stats),\n+\t\t\t\t GFP_KERNEL);\n+\tif (!apc-\u003erxq_stats)\n+\t\treturn -ENOMEM;\n+\n+\tapc-\u003erxq_stats_ret = kcalloc(apc-\u003emax_queues,\n+\t\t\t\t sizeof(*apc-\u003erxq_stats_ret), GFP_KERNEL);\n+\tif (!apc-\u003erxq_stats_ret)\n+\t\tgoto free_rxq_stats;\n+\n+\tapc-\u003etxq_stats = kcalloc(apc-\u003emax_queues, sizeof(*apc-\u003etxq_stats),\n+\t\t\t\t GFP_KERNEL);\n+\tif (!apc-\u003etxq_stats)\n+\t\tgoto free_rxq_stats_ret;\n+\n+\tfor (i = 0; i \u003c apc-\u003emax_queues; i++) {\n+\t\tu64_stats_init(\u0026apc-\u003erxq_stats[i].syncp);\n+\t\tu64_stats_init(\u0026apc-\u003erxq_stats_ret[i].syncp);\n+\t\tu64_stats_init(\u0026apc-\u003etxq_stats[i].syncp);\n+\t}\n+\n+\treturn 0;\n+\n+free_rxq_stats_ret:\n+\tkfree(apc-\u003erxq_stats_ret);\n+\tapc-\u003erxq_stats_ret = NULL;\n+free_rxq_stats:\n+\tkfree(apc-\u003erxq_stats);\n+\tapc-\u003erxq_stats = NULL;\n+\treturn -ENOMEM;\n+}\n+\n+static void mana_free_queue_stats(struct mana_port_context *apc)\n+{\n+\tkfree(apc-\u003erxq_stats);\n+\tapc-\u003erxq_stats = NULL;\n+\tkfree(apc-\u003erxq_stats_ret);\n+\tapc-\u003erxq_stats_ret = NULL;\n+\tkfree(apc-\u003etxq_stats);\n+\tapc-\u003etxq_stats = NULL;\n+}\n+\n+/* Fold under RTNL after drain_stats writers quiesce. Clear drain_stats to\n+ * prevent double counting on rollback.\n+ */\n+static void mana_fold_rxq_stats(struct mana_port_context *apc,\n+\t\t\t\tstruct mana_rxq *rxq)\n+{\n+\tstruct mana_stats_rx *src = \u0026rxq-\u003edrain_stats;\n+\tstruct mana_stats_rx *dst;\n+\tunsigned int i;\n+\n+\tASSERT_RTNL();\n+\n+\tif (!apc-\u003erxq_stats_ret || rxq-\u003erxq_idx \u003e= apc-\u003emax_queues)\n+\t\treturn;\n+\n+\tdst = \u0026apc-\u003erxq_stats_ret[rxq-\u003erxq_idx];\n+\n+\tu64_stats_update_begin(\u0026dst-\u003esyncp);\n+\tdst-\u003epackets\t\t+= src-\u003epackets;\n+\tdst-\u003ebytes\t\t+= src-\u003ebytes;\n+\tdst-\u003exdp_drop\t\t+= src-\u003exdp_drop;\n+\tdst-\u003exdp_tx\t\t+= src-\u003exdp_tx;\n+\tdst-\u003exdp_redirect\t+= src-\u003exdp_redirect;\n+\tdst-\u003epkt_len0_err\t+= src-\u003epkt_len0_err;\n+\tfor (i = 0; i \u003c ARRAY_SIZE(dst-\u003ecoalesced_cqe); i++)\n+\t\tdst-\u003ecoalesced_cqe[i] += src-\u003ecoalesced_cqe[i];\n+\tu64_stats_update_end(\u0026dst-\u003esyncp);\n+\n+\tsrc-\u003epackets\t\t= 0;\n+\tsrc-\u003ebytes\t\t= 0;\n+\tsrc-\u003exdp_drop\t\t= 0;\n+\tsrc-\u003exdp_tx\t\t= 0;\n+\tsrc-\u003exdp_redirect\t= 0;\n+\tsrc-\u003epkt_len0_err\t= 0;\n+\tfor (i = 0; i \u003c ARRAY_SIZE(src-\u003ecoalesced_cqe); i++)\n+\t\tsrc-\u003ecoalesced_cqe[i] = 0;\n+}\n+\n+static void mana_fold_qset_rx_stats(struct mana_port_context *apc,\n+\t\t\t\t struct mana_qset *qset)\n+{\n+\tunsigned int q;\n+\n+\tif (!qset-\u003erxqs)\n+\t\treturn;\n+\n+\tfor (q = 0; q \u003c qset-\u003enum_queues; q++) {\n+\t\tif (qset-\u003erxqs[q])\n+\t\t\tmana_fold_rxq_stats(apc, qset-\u003erxqs[q]);\n+\t}\n+}\n+\n static void mana_cleanup_indir_table(struct mana_port_context *apc)\n {\n \tapc-\u003eindir_table_sz = 0;\n@@ -1045,6 +1184,7 @@ static void mana_cleanup_indir_table(struct mana_port_context *apc)\n \n static int mana_init_port_context(struct mana_port_context *apc)\n {\n+\tkfree(apc-\u003erxqs);\n \tapc-\u003erxqs = kzalloc_objs(struct mana_rxq *, apc-\u003enum_queues);\n \n \treturn !apc-\u003erxqs ? -ENOMEM : 0;\n@@ -1733,7 +1873,7 @@ void mana_destroy_eq(struct mana_port_context *apc)\n \tdebugfs_remove_recursive(apc-\u003emana_eqs_debugfs);\n \tapc-\u003emana_eqs_debugfs = NULL;\n \n-\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\tfor (i = 0; i \u003c apc-\u003enum_eqs; i++) {\n \t\teq = apc-\u003eeqs[i].eq;\n \t\tif (!eq)\n \t\t\tcontinue;\n@@ -1741,24 +1881,28 @@ void mana_destroy_eq(struct mana_port_context *apc)\n \t\tmsi = eq-\u003eeq.msix_index;\n \t\tmana_gd_destroy_queue(gc, eq);\n \t\tmana_gd_put_gic(gc, !gc-\u003emsi_sharing, msi);\n+\t\tapc-\u003eeqs[i].eq = NULL;\n+\t\tapc-\u003eeqs[i].mana_eq_debugfs = NULL;\n \t}\n \n \tkfree(apc-\u003eeqs);\n \tapc-\u003eeqs = NULL;\n+\tapc-\u003enum_eqs = 0;\n }\n EXPORT_SYMBOL_NS(mana_destroy_eq, \"NET_MANA\");\n \n static void mana_create_eq_debugfs(struct mana_port_context *apc, int i)\n {\n-\tstruct mana_eq eq = apc-\u003eeqs[i];\n+\tstruct mana_eq *eq = \u0026apc-\u003eeqs[i];\n \tchar eqnum[32];\n \n \tsprintf(eqnum, \"eq%d\", i);\n-\teq.mana_eq_debugfs = debugfs_create_dir(eqnum, apc-\u003emana_eqs_debugfs);\n-\tdebugfs_create_u32(\"head\", 0400, eq.mana_eq_debugfs, \u0026eq.eq-\u003ehead);\n-\tdebugfs_create_u32(\"tail\", 0400, eq.mana_eq_debugfs, \u0026eq.eq-\u003etail);\n-\tdebugfs_create_u32(\"irq\", 0400, eq.mana_eq_debugfs, \u0026eq.eq-\u003eeq.irq);\n-\tdebugfs_create_file(\"eq_dump\", 0400, eq.mana_eq_debugfs, eq.eq, \u0026mana_dbg_q_fops);\n+\teq-\u003emana_eq_debugfs = debugfs_create_dir(eqnum, apc-\u003emana_eqs_debugfs);\n+\tdebugfs_create_u32(\"head\", 0400, eq-\u003emana_eq_debugfs, \u0026eq-\u003eeq-\u003ehead);\n+\tdebugfs_create_u32(\"tail\", 0400, eq-\u003emana_eq_debugfs, \u0026eq-\u003eeq-\u003etail);\n+\tdebugfs_create_u32(\"irq\", 0400, eq-\u003emana_eq_debugfs, \u0026eq-\u003eeq-\u003eeq.irq);\n+\tdebugfs_create_file(\"eq_dump\", 0400, eq-\u003emana_eq_debugfs, eq-\u003eeq,\n+\t\t\t \u0026mana_dbg_q_fops);\n }\n \n int mana_create_eq(struct mana_port_context *apc)\n@@ -1773,9 +1917,11 @@ int mana_create_eq(struct mana_port_context *apc)\n \n \tif (WARN_ON(apc-\u003eeqs))\n \t\treturn -EEXIST;\n-\tapc-\u003eeqs = kzalloc_objs(struct mana_eq, apc-\u003enum_queues);\n+\t/* Keep EQ array addresses stable while CQs reference them. */\n+\tapc-\u003eeqs = kzalloc_objs(struct mana_eq, apc-\u003emax_queues);\n \tif (!apc-\u003eeqs)\n \t\treturn -ENOMEM;\n+\tapc-\u003enum_eqs = 0;\n \n \tspec.type = GDMA_EQ;\n \tspec.monitor_avl_buf = false;\n@@ -1805,6 +1951,7 @@ int mana_create_eq(struct mana_port_context *apc)\n \t\t}\n \t\tapc-\u003eeqs[i].eq-\u003eeq.irq = gic-\u003eirq;\n \t\tmana_create_eq_debugfs(apc, i);\n+\t\tapc-\u003enum_eqs = i + 1;\n \t}\n \n \treturn 0;\n@@ -1814,6 +1961,87 @@ int mana_create_eq(struct mana_port_context *apc)\n }\n EXPORT_SYMBOL_NS(mana_create_eq, \"NET_MANA\");\n \n+/* Grow the shared EQ pool without replacing live entries. */\n+static int mana_grow_eqs(struct mana_port_context *apc, unsigned int need)\n+{\n+\tstruct gdma_dev *gd = apc-\u003eac-\u003egdma_dev;\n+\tstruct gdma_context *gc = gd-\u003egdma_context;\n+\tstruct gdma_queue_spec spec = {};\n+\tstruct gdma_irq_context *gic;\n+\tunsigned int i;\n+\tint err;\n+\tint msi;\n+\n+\tif (WARN_ON(!apc-\u003eeqs))\n+\t\treturn -EINVAL;\n+\n+\tif (need \u003e apc-\u003emax_queues)\n+\t\treturn -EINVAL;\n+\n+\tif (need \u003c= apc-\u003enum_eqs)\n+\t\treturn 0;\n+\n+\tspec.type = GDMA_EQ;\n+\tspec.monitor_avl_buf = false;\n+\tspec.queue_size = EQ_SIZE;\n+\tspec.eq.callback = NULL;\n+\tspec.eq.context = apc-\u003eeqs;\n+\tspec.eq.log2_throttle_limit = LOG2_EQ_THROTTLE;\n+\n+\tfor (i = apc-\u003enum_eqs; i \u003c need; i++) {\n+\t\tmsi = (i + 1) % gc-\u003enum_msix_usable;\n+\n+\t\tgic = mana_gd_get_gic(gc, !gc-\u003emsi_sharing, \u0026msi);\n+\t\tif (IS_ERR(gic)) {\n+\t\t\terr = PTR_ERR(gic);\n+\t\t\tgoto out;\n+\t\t}\n+\t\tspec.eq.msix_index = msi;\n+\n+\t\terr = mana_gd_create_mana_eq(gd, \u0026spec, \u0026apc-\u003eeqs[i].eq);\n+\t\tif (err) {\n+\t\t\tdev_err(gc-\u003edev, \"Failed to grow EQ %u : %d\\n\", i, err);\n+\t\t\tmana_gd_put_gic(gc, !gc-\u003emsi_sharing, msi);\n+\t\t\tgoto out;\n+\t\t}\n+\t\tapc-\u003eeqs[i].eq-\u003eeq.irq = gic-\u003eirq;\n+\t\tmana_create_eq_debugfs(apc, i);\n+\t\tapc-\u003enum_eqs = i + 1;\n+\t}\n+\n+\treturn 0;\n+out:\n+\treturn err;\n+}\n+\n+/* All CQs referencing EQs at or above @keep must be destroyed first. */\n+static void mana_shrink_eqs(struct mana_port_context *apc, unsigned int keep)\n+{\n+\tstruct gdma_context *gc = apc-\u003eac-\u003egdma_dev-\u003egdma_context;\n+\tstruct gdma_queue *eq;\n+\tunsigned int msi;\n+\tunsigned int i;\n+\n+\tif (!apc-\u003eeqs || keep \u003e= apc-\u003enum_eqs)\n+\t\treturn;\n+\n+\tfor (i = keep; i \u003c apc-\u003enum_eqs; i++) {\n+\t\teq = apc-\u003eeqs[i].eq;\n+\t\tif (!eq)\n+\t\t\tcontinue;\n+\n+\t\tdebugfs_remove_recursive(apc-\u003eeqs[i].mana_eq_debugfs);\n+\t\tapc-\u003eeqs[i].mana_eq_debugfs = NULL;\n+\n+\t\tmsi = eq-\u003eeq.msix_index;\n+\t\tmana_gd_destroy_queue(gc, eq);\n+\t\tmana_gd_put_gic(gc, !gc-\u003emsi_sharing, msi);\n+\t\tapc-\u003eeqs[i].eq = NULL;\n+\t}\n+\n+\tapc-\u003enum_eqs = keep;\n+}\n+\n static int mana_fence_rq(struct mana_port_context *apc, struct mana_rxq *rxq)\n {\n \tstruct mana_fence_rq_resp resp = {};\n@@ -2018,7 +2246,9 @@ static void mana_poll_tx_cq(struct mana_cq *cq)\n \t/* Ensure checking txq_stopped before apc-\u003eport_is_up. */\n \tsmp_rmb();\n \n-\tif (txq_stopped \u0026\u0026 apc-\u003eport_is_up \u0026\u0026 avail_space \u003e= MAX_TX_WQE_SIZE) {\n+\t/* Order the stopped-state read before the retiring read. */\n+\tif (txq_stopped \u0026\u0026 !READ_ONCE(txq-\u003eretiring) \u0026\u0026 apc-\u003eport_is_up \u0026\u0026\n+\t avail_space \u003e= MAX_TX_WQE_SIZE) {\n \t\tnetif_tx_wake_queue(net_txq);\n \t\tapc-\u003eeth_stats.wake_queue++;\n \t}\n@@ -2081,7 +2311,7 @@ static void mana_rx_skb(void *buf_va, bool from_pool,\n \t\t\tstruct mana_rxcomp_oob *cqe, struct mana_rxq *rxq,\n \t\t\tu32 pkt_len, u32 pkt_hash)\n {\n-\tstruct mana_stats_rx *rx_stats = \u0026rxq-\u003estats;\n+\tstruct mana_stats_rx *rx_stats = mana_rxq_stats(rxq);\n \tstruct net_device *ndev = rxq-\u003endev;\n \tu16 rxq_idx = rxq-\u003erxq_idx;\n \tstruct napi_struct *napi;\n@@ -2314,6 +2544,7 @@ static void mana_process_rx_cqe(struct mana_rxq *rxq, struct mana_cq *cq,\n \tstruct net_device *ndev = rxq-\u003endev;\n \tstruct mana_recv_buf_oob *rxbuf_oob;\n \tstruct mana_port_context *apc;\n+\tstruct mana_stats_rx *rx_stats;\n \tstruct device *dev = gc-\u003edev;\n \tbool coalesced_8 = false;\n \tbool coalesced = false;\n@@ -2395,13 +2626,15 @@ static void mana_process_rx_cqe(struct mana_rxq *rxq, struct mana_cq *cq,\n \t * Coalesced CQEs have at least 2 packets, so index is pkt_i - 2.\n \t */\n \tif (pkt_i \u003e 1) {\n-\t\tu64_stats_update_begin(\u0026rxq-\u003estats.syncp);\n-\t\trxq-\u003estats.coalesced_cqe[pkt_i - 2]++;\n-\t\tu64_stats_update_end(\u0026rxq-\u003estats.syncp);\n+\t\trx_stats = mana_rxq_stats(rxq);\n+\t\tu64_stats_update_begin(\u0026rx_stats-\u003esyncp);\n+\t\trx_stats-\u003ecoalesced_cqe[pkt_i - 2]++;\n+\t\tu64_stats_update_end(\u0026rx_stats-\u003esyncp);\n \t} else if (!pkt_i \u0026\u0026 !pktlen) {\n-\t\tu64_stats_update_begin(\u0026rxq-\u003estats.syncp);\n-\t\trxq-\u003estats.pkt_len0_err++;\n-\t\tu64_stats_update_end(\u0026rxq-\u003estats.syncp);\n+\t\trx_stats = mana_rxq_stats(rxq);\n+\t\tu64_stats_update_begin(\u0026rx_stats-\u003esyncp);\n+\t\trx_stats-\u003epkt_len0_err++;\n+\t\tu64_stats_update_end(\u0026rx_stats-\u003esyncp);\n \t\tnetdev_err_once(ndev,\n \t\t\t\t\"RX pkt len=0, rq=%u, cq=%u, rxobj=0x%llx\\n\",\n \t\t\t\trxq-\u003egdma_id, cq-\u003egdma_id, rxq-\u003erxobj);\n@@ -2533,8 +2766,12 @@ static void mana_update_rx_dim(struct mana_cq *cq)\n \tif (!smp_load_acquire(\u0026apc-\u003erx_dim_enabled))\n \t\treturn;\n \n-\tdim_update_sample(READ_ONCE(cq-\u003edim_event_ctr), rxq-\u003estats.packets,\n-\t\t\t rxq-\u003estats.bytes, \u0026dim_sample);\n+\t/* Skip retiring RXQs; DIM reads shared per-index counters. */\n+\tif (READ_ONCE(rxq-\u003eretiring))\n+\t\treturn;\n+\n+\tdim_update_sample(READ_ONCE(cq-\u003edim_event_ctr), rxq-\u003estats-\u003epackets,\n+\t\t\t rxq-\u003estats-\u003ebytes, \u0026dim_sample);\n \tnet_dim(\u0026cq-\u003edim, \u0026dim_sample);\n }\n \n@@ -2641,7 +2878,9 @@ static void mana_deinit_txq(struct mana_port_context *apc, struct mana_txq *txq)\n \tmana_gd_destroy_queue(gd-\u003egdma_context, txq-\u003egdma_sq);\n }\n \n-static void mana_destroy_txq(struct mana_port_context *apc)\n+/* Keep the array and queues below @first; clear freed slots. */\n+static void mana_destroy_txq_from(struct mana_port_context *apc,\n+\t\t\t\t unsigned int first)\n {\n \tstruct napi_struct *napi;\n \tint i;\n@@ -2649,7 +2888,7 @@ static void mana_destroy_txq(struct mana_port_context *apc)\n \tif (!apc-\u003etx_qp)\n \t\treturn;\n \n-\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\tfor (i = first; i \u003c apc-\u003enum_queues; i++) {\n \t\tif (!apc-\u003etx_qp[i])\n \t\t\tcontinue;\n \n@@ -2673,7 +2912,16 @@ static void mana_destroy_txq(struct mana_port_context *apc)\n \t\tmana_deinit_txq(apc, \u0026apc-\u003etx_qp[i]-\u003etxq);\n \n \t\tkvfree(apc-\u003etx_qp[i]);\n+\t\tapc-\u003etx_qp[i] = NULL;\n \t}\n+}\n+\n+static void mana_destroy_txq(struct mana_port_context *apc)\n+{\n+\tif (!apc-\u003etx_qp)\n+\t\treturn;\n+\n+\tmana_destroy_txq_from(apc, 0);\n \n \tkfree(apc-\u003etx_qp);\n \tapc-\u003etx_qp = NULL;\n@@ -2704,8 +2952,11 @@ static void mana_create_txq_debugfs(struct mana_port_context *apc, int idx)\n \t\t\t tx_qp-\u003etx_cq.gdma_cq, \u0026mana_dbg_q_fops);\n }\n \n+/* With @first nonzero, use the existing array and unwind only new queues on\n+ * failure.\n+ */\n static int mana_create_txq(struct mana_port_context *apc,\n-\t\t\t struct net_device *net)\n+\t\t\t struct net_device *net, unsigned int first)\n {\n \tstruct mana_context *ac = apc-\u003eac;\n \tstruct gdma_dev *gd = ac-\u003egdma_dev;\n@@ -2720,9 +2971,14 @@ static int mana_create_txq(struct mana_port_context *apc,\n \tint err;\n \tint i;\n \n-\tapc-\u003etx_qp = kzalloc_objs(struct mana_tx_qp *, apc-\u003enum_queues);\n-\tif (!apc-\u003etx_qp)\n-\t\treturn -ENOMEM;\n+\tif (first) {\n+\t\tif (WARN_ON(!apc-\u003etx_qp))\n+\t\t\treturn -EINVAL;\n+\t} else {\n+\t\tapc-\u003etx_qp = kzalloc_objs(struct mana_tx_qp *, apc-\u003enum_queues);\n+\t\tif (!apc-\u003etx_qp)\n+\t\t\treturn -ENOMEM;\n+\t}\n \n \t/* The minimum size of the WQE is 32 bytes, hence\n \t * apc-\u003etx_queue_size represents the maximum number of WQEs\n@@ -2739,7 +2995,7 @@ static int mana_create_txq(struct mana_port_context *apc,\n \n \tgc = gd-\u003egdma_context;\n \n-\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\tfor (i = first; i \u003c apc-\u003enum_queues; i++) {\n \t\tapc-\u003etx_qp[i] = kvzalloc_obj(*apc-\u003etx_qp[i]);\n \t\tif (!apc-\u003etx_qp[i]) {\n \t\t\terr = -ENOMEM;\n@@ -2751,9 +3007,10 @@ static int mana_create_txq(struct mana_port_context *apc,\n \t\t/* Create SQ */\n \t\ttxq = \u0026apc-\u003etx_qp[i]-\u003etxq;\n \n-\t\tu64_stats_init(\u0026txq-\u003estats.syncp);\n+\t\ttxq-\u003estats = \u0026apc-\u003etxq_stats[i];\n \t\ttxq-\u003endev = net;\n \t\ttxq-\u003enet_txq = netdev_get_tx_queue(net, i);\n+\t\ttxq-\u003ereset_gen = READ_ONCE(apc-\u003eac-\u003ereset_gen);\n \t\ttxq-\u003evp_offset = apc-\u003etx_vp_offset;\n \t\ttxq-\u003enapi_initialized = false;\n \t\tskb_queue_head_init(\u0026txq-\u003epending_skbs);\n@@ -2846,7 +3103,10 @@ static int mana_create_txq(struct mana_port_context *apc,\n out:\n \tnetdev_err(net, \"Failed to create %d TX queues, %d\\n\",\n \t\t apc-\u003enum_queues, err);\n-\tmana_destroy_txq(apc);\n+\tif (first)\n+\t\tmana_destroy_txq_from(apc, first);\n+\telse\n+\t\tmana_destroy_txq(apc);\n \treturn err;\n }\n \n@@ -2877,6 +3137,9 @@ static void mana_destroy_rxq(struct mana_port_context *apc,\n \t\tnetif_napi_del_locked(napi);\n \t}\n \n+\t/* NAPI is quiesced, so drain_stats has no remaining writer. */\n+\tmana_fold_rxq_stats(apc, rxq);\n+\n \tif (xdp_rxq_info_is_reg(\u0026rxq-\u003exdp_rxq))\n \t\txdp_rxq_info_unreg(\u0026rxq-\u003exdp_rxq);\n \n@@ -3009,11 +3272,11 @@ static int mana_push_wqe(struct mana_rxq *rxq)\n \n static int mana_create_page_pool(struct mana_rxq *rxq, struct gdma_context *gc)\n {\n-\tstruct mana_port_context *mpc = netdev_priv(rxq-\u003endev);\n \tstruct page_pool_params pprm = {};\n \tint ret;\n \n-\tpprm.pool_size = mpc-\u003erx_queue_size / rxq-\u003efrag_count + 1;\n+\t/* Size the pool for this RX queue, not the live configuration. */\n+\tpprm.pool_size = rxq-\u003enum_rx_buf / rxq-\u003efrag_count + 1;\n \tpprm.nid = gc-\u003enuma_node;\n \tpprm.napi = \u0026rxq-\u003erx_cq.napi;\n \tpprm.netdev = rxq-\u003endev;\n@@ -3062,11 +3325,14 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc,\n \t\treturn ERR_PTR(-ENOMEM);\n \n \trxq-\u003endev = ndev;\n+\trxq-\u003estats = \u0026apc-\u003erxq_stats[rxq_idx];\n+\tu64_stats_init(\u0026rxq-\u003edrain_stats.syncp);\n \trxq-\u003enum_rx_buf = apc-\u003erx_queue_size;\n \trxq-\u003erxq_idx = rxq_idx;\n \trxq-\u003erxobj = INVALID_MANA_HANDLE;\n \n-\tmana_get_rxbuf_cfg(apc, ndev-\u003emtu, \u0026rxq-\u003edatasize, \u0026rxq-\u003ealloc_size,\n+\tmana_get_rxbuf_cfg(apc, apc-\u003econfigured_mtu, \u0026rxq-\u003edatasize,\n+\t\t\t \u0026rxq-\u003ealloc_size,\n \t\t\t \u0026rxq-\u003eheadroom, \u0026rxq-\u003efrag_count);\n \t/* Create page pool for RX queue */\n \terr = mana_create_page_pool(rxq, gc);\n@@ -3196,14 +3462,15 @@ static void mana_create_rxq_debugfs(struct mana_port_context *apc, int idx)\n \t\t\t \u0026mana_dbg_q_fops);\n }\n \n+/* The caller must destroy queues added before a failure. */\n static int mana_add_rx_queues(struct mana_port_context *apc,\n-\t\t\t struct net_device *ndev)\n+\t\t\t struct net_device *ndev, unsigned int first)\n {\n \tstruct mana_rxq *rxq;\n \tint err = 0;\n \tint i;\n \n-\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\tfor (i = first; i \u003c apc-\u003enum_queues; i++) {\n \t\trxq = mana_create_rxq(apc, i, \u0026apc-\u003eeqs[i], ndev);\n \t\tif (IS_ERR(rxq)) {\n \t\t\terr = PTR_ERR(rxq);\n@@ -3211,8 +3478,6 @@ static int mana_add_rx_queues(struct mana_port_context *apc,\n \t\t\tgoto out;\n \t\t}\n \n-\t\tu64_stats_init(\u0026rxq-\u003estats.syncp);\n-\n \t\tapc-\u003erxqs[i] = rxq;\n \n \t\tmana_create_rxq_debugfs(apc, i);\n@@ -3223,14 +3488,15 @@ static int mana_add_rx_queues(struct mana_port_context *apc,\n \treturn err;\n }\n \n-static void mana_destroy_rxqs(struct mana_port_context *apc)\n+static void mana_destroy_rxqs_from(struct mana_port_context *apc,\n+\t\t\t\t unsigned int first)\n {\n \tstruct mana_rxq *rxq;\n \tu32 rxq_idx;\n \n \tif (apc-\u003erxqs) {\n \n-\t\tfor (rxq_idx = 0; rxq_idx \u003c apc-\u003enum_queues; rxq_idx++) {\n+\t\tfor (rxq_idx = first; rxq_idx \u003c apc-\u003enum_queues; rxq_idx++) {\n \t\t\trxq = apc-\u003erxqs[rxq_idx];\n \t\t\tif (!rxq)\n \t\t\t\tcontinue;\n@@ -3241,6 +3507,11 @@ static void mana_destroy_rxqs(struct mana_port_context *apc)\n \t}\n }\n \n+static void mana_destroy_rxqs(struct mana_port_context *apc)\n+{\n+\tmana_destroy_rxqs_from(apc, 0);\n+}\n+\n static void mana_destroy_vport(struct mana_port_context *apc)\n {\n \tstruct gdma_dev *gd = apc-\u003eac-\u003egdma_dev;\n@@ -3306,6 +3577,27 @@ static void mana_rss_table_init(struct mana_port_context *apc)\n \t\t\tethtool_rxfh_indir_default(i, apc-\u003enum_queues);\n }\n \n+/* Keep user tables with valid indices; defer loss notification. */\n+static bool mana_rss_table_keep(struct mana_port_context *apc,\n+\t\t\t\tunsigned int num_queues, bool *lost)\n+{\n+\tu32 i;\n+\n+\t*lost = false;\n+\n+\tif (!netif_is_rxfh_configured(apc-\u003endev))\n+\t\treturn false;\n+\n+\tfor (i = 0; i \u003c apc-\u003eindir_table_sz; i++) {\n+\t\tif (apc-\u003eindir_table[i] \u003e= num_queues) {\n+\t\t\t*lost = true;\n+\t\t\treturn false;\n+\t\t}\n+\t}\n+\n+\treturn true;\n+}\n+\n int mana_disable_vport_rx(struct mana_port_context *apc)\n {\n \treturn mana_cfg_vport_steering(apc, TRI_STATE_FALSE, false, false,\n@@ -3576,6 +3868,7 @@ int mana_alloc_queues(struct net_device *ndev)\n {\n \tstruct mana_port_context *apc = netdev_priv(ndev);\n \tstruct gdma_dev *gd = apc-\u003eac-\u003egdma_dev;\n+\tbool indir_lost;\n \tint err;\n \n \terr = mana_create_vport(apc, ndev);\n@@ -3592,7 +3885,7 @@ int mana_alloc_queues(struct net_device *ndev)\n \t\tgoto destroy_vport;\n \t}\n \n-\terr = mana_create_txq(apc, ndev);\n+\terr = mana_create_txq(apc, ndev, 0);\n \tif (err) {\n \t\tnetdev_err(ndev, \"Failed to create TXQ on vPort %u: %d\\n\",\n \t\t\t apc-\u003eport_idx, err);\n@@ -3607,7 +3900,7 @@ int mana_alloc_queues(struct net_device *ndev)\n \t\tgoto destroy_txq;\n \t}\n \n-\terr = mana_add_rx_queues(apc, ndev);\n+\terr = mana_add_rx_queues(apc, ndev, 0);\n \tif (err)\n \t\tgoto destroy_rxq;\n \n@@ -3621,7 +3914,9 @@ int mana_alloc_queues(struct net_device *ndev)\n \t\tgoto destroy_rxq;\n \t}\n \n-\tmana_rss_table_init(apc);\n+\t/* Loss notification needs a netdev instance lock we may lack. */\n+\tif (!mana_rss_table_keep(apc, apc-\u003enum_queues, \u0026indir_lost))\n+\t\tmana_rss_table_init(apc);\n \n \terr = mana_config_rss(apc, TRI_STATE_TRUE, true, true);\n \tif (err) {\n@@ -3679,15 +3974,88 @@ int mana_attach(struct net_device *ndev)\n \treturn 0;\n }\n \n-static int mana_dealloc_queues(struct net_device *ndev)\n+/* Drain with a per-set timeout; return true only for a successful FLR. A false\n+ * return does not guarantee DMA quiescence.\n+ */\n+static bool mana_drain_txqs(struct mana_port_context *apc)\n {\n-\tstruct mana_port_context *apc = netdev_priv(ndev);\n \tunsigned long timeout = jiffies + 120 * HZ;\n-\tstruct gdma_dev *gd = apc-\u003eac-\u003egdma_dev;\n+\tstruct gdma_context *gc = apc-\u003eac-\u003egdma_dev-\u003egdma_context;\n+\tbool quiesced = true;\n+\tbool reset = false;\n \tstruct mana_txq *txq;\n \tstruct sk_buff *skb;\n-\tint i, err;\n \tu32 tsleep;\n+\tint i, err;\n+\n+\tif (!apc-\u003etx_qp)\n+\t\treturn false;\n+\n+\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\t\tif (!apc-\u003etx_qp[i])\n+\t\t\tcontinue;\n+\n+\t\ttxq = \u0026apc-\u003etx_qp[i]-\u003etxq;\n+\n+\t\t/* A previous function reset invalidated these queues. */\n+\t\tif (READ_ONCE(apc-\u003eac-\u003ereset_gen) != txq-\u003ereset_gen)\n+\t\t\tcontinue;\n+\n+\t\ttsleep = 1000;\n+\t\twhile (atomic_read(\u0026txq-\u003epending_sends) \u003e 0 \u0026\u0026\n+\t\t time_before(jiffies, timeout)) {\n+\t\t\tusleep_range(tsleep, tsleep + 1000);\n+\t\t\ttsleep \u003c\u003c= 1;\n+\t\t}\n+\t\tif (atomic_read(\u0026txq-\u003epending_sends)) {\n+\t\t\terr = pcie_flr(to_pci_dev(gc-\u003edev));\n+\t\t\tif (err) {\n+\t\t\t\tnetdev_err(apc-\u003endev,\n+\t\t\t\t\t \"flr failed %d with %d pkts pending in txq %u\\n\",\n+\t\t\t\t\t err,\n+\t\t\t\t\t atomic_read(\u0026txq-\u003epending_sends),\n+\t\t\t\t\t txq-\u003egdma_txq_id);\n+\t\t\t\tquiesced = false;\n+\t\t\t} else {\n+\t\t\t\t/* Invalidate queues on every port after the\n+\t\t\t\t * function reset.\n+\t\t\t\t */\n+\t\t\t\tWRITE_ONCE(apc-\u003eac-\u003ereset_gen,\n+\t\t\t\t\t apc-\u003eac-\u003ereset_gen + 1);\n+\n+\t\t\t\treset = true;\n+\t\t\t}\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\n+\t/* A failed FLR cannot justify unmapping pending TX buffers. */\n+\tif (!quiesced) {\n+\t\tnetdev_err(apc-\u003endev,\n+\t\t\t \"device not quiesced, leaking pending TX buffers instead of unmapping memory it can still DMA from\\n\");\n+\t\treturn reset;\n+\t}\n+\n+\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\t\tif (!apc-\u003etx_qp[i])\n+\t\t\tcontinue;\n+\n+\t\ttxq = \u0026apc-\u003etx_qp[i]-\u003etxq;\n+\t\twhile ((skb = skb_dequeue(\u0026txq-\u003epending_skbs))) {\n+\t\t\tmana_unmap_skb(skb, apc);\n+\t\t\tdev_kfree_skb_any(skb);\n+\t\t}\n+\t\tatomic_set(\u0026txq-\u003epending_sends, 0);\n+\t}\n+\n+\treturn reset;\n+}\n+\n+static int mana_dealloc_queues(struct net_device *ndev)\n+{\n+\tstruct mana_port_context *apc = netdev_priv(ndev);\n+\tstruct gdma_dev *gd = apc-\u003eac-\u003egdma_dev;\n+\tint err;\n \n \tif (apc-\u003eport_is_up)\n \t\treturn -EINVAL;\n@@ -3698,48 +4066,21 @@ static int mana_dealloc_queues(struct net_device *ndev)\n \tif (gd-\u003egdma_context-\u003eis_pf \u0026\u0026 !apc-\u003eac-\u003ebm_hostmode)\n \t\tmana_pf_deregister_filter(apc);\n \n-\t/* No packet can be transmitted now since apc-\u003eport_is_up is false.\n-\t * There is still a tiny chance that mana_poll_tx_cq() can re-enable\n-\t * a txq because it may not timely see apc-\u003eport_is_up being cleared\n-\t * to false, but it doesn't matter since mana_start_xmit() drops any\n-\t * new packets due to apc-\u003eport_is_up being false.\n-\t *\n-\t * Drain all the in-flight TX packets.\n-\t * A timeout of 120 seconds for all the queues is used.\n-\t * This will break the while loop when h/w is not responding.\n-\t * This value of 120 has been decided here considering max\n-\t * number of queues.\n-\t */\n+\t/* After FLR, schedule a best-effort sibling-port rebuild. */\n+\tif (mana_drain_txqs(apc)) {\n+\t\tstruct mana_context *ac = apc-\u003eac;\n+\t\tunsigned int i;\n \n-\tif (apc-\u003etx_qp) {\n-\t\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n-\t\t\ttxq = \u0026apc-\u003etx_qp[i]-\u003etxq;\n-\t\t\ttsleep = 1000;\n-\t\t\twhile (atomic_read(\u0026txq-\u003epending_sends) \u003e 0 \u0026\u0026\n-\t\t\t time_before(jiffies, timeout)) {\n-\t\t\t\tusleep_range(tsleep, tsleep + 1000);\n-\t\t\t\ttsleep \u003c\u003c= 1;\n-\t\t\t}\n-\t\t\tif (atomic_read(\u0026txq-\u003epending_sends)) {\n-\t\t\t\terr =\n-\t\t\t\t pcie_flr(to_pci_dev(gd-\u003egdma_context-\u003edev));\n-\t\t\t\tif (err) {\n-\t\t\t\t\tnetdev_err(ndev, \"flr failed %d with %d pkts pending in txq %u\\n\",\n-\t\t\t\t\t\t err,\n-\t\t\t\t\t atomic_read(\u0026txq-\u003epending_sends),\n-\t\t\t\t\t txq-\u003egdma_txq_id);\n-\t\t\t\t}\n-\t\t\t\tbreak;\n-\t\t\t}\n-\t\t}\n+\t\tfor (i = 0; i \u003c ac-\u003enum_ports; i++) {\n+\t\t\tstruct mana_port_context *sib;\n \n-\t\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n-\t\t\ttxq = \u0026apc-\u003etx_qp[i]-\u003etxq;\n-\t\t\twhile ((skb = skb_dequeue(\u0026txq-\u003epending_skbs))) {\n-\t\t\t\tmana_unmap_skb(skb, apc);\n-\t\t\t\tdev_kfree_skb_any(skb);\n-\t\t\t}\n-\t\t\tatomic_set(\u0026txq-\u003epending_sends, 0);\n+\t\t\tif (!ac-\u003eports[i] || ac-\u003eports[i] == ndev)\n+\t\t\t\tcontinue;\n+\t\t\tsib = netdev_priv(ac-\u003eports[i]);\n+\t\t\tnetdev_err(ac-\u003eports[i],\n+\t\t\t\t \"queues reset by a sibling port, scheduling rebuild\\n\");\n+\t\t\tqueue_work(ac-\u003eper_port_queue_reset_wq,\n+\t\t\t\t \u0026sib-\u003equeue_reset_work);\n \t\t}\n \t}\n \n@@ -3763,6 +4104,685 @@ static int mana_dealloc_queues(struct net_device *ndev)\n \treturn 0;\n }\n \n+static void mana_qset_snapshot(const struct mana_port_context *ctx,\n+\t\t\t struct mana_qset *out)\n+{\n+\tout-\u003etx_qp\t\t= ctx-\u003etx_qp;\n+\tout-\u003erxqs\t\t= ctx-\u003erxqs;\n+\tout-\u003eindir_table\t= ctx-\u003eindir_table;\n+\tout-\u003eindir_table_sz\t= ctx-\u003eindir_table_sz;\n+\tout-\u003erxobj_table\t= ctx-\u003erxobj_table;\n+\tout-\u003edefault_rxobj\t= ctx-\u003edefault_rxobj;\n+\tout-\u003enum_queues\t\t= ctx-\u003enum_queues;\n+\tout-\u003erx_queue_size\t= ctx-\u003erx_queue_size;\n+\tout-\u003etx_queue_size\t= ctx-\u003etx_queue_size;\n+\tout-\u003epriv_flags\t\t= ctx-\u003epriv_flags;\n+\tout-\u003emtu\t\t= ctx-\u003econfigured_mtu;\n+\tout-\u003ebpf_prog\t\t= ctx-\u003ebpf_prog;\n+\n+\tout-\u003erxfh_indir_lost\t= false;\n+}\n+\n+static void mana_qset_install(struct mana_port_context *ctx,\n+\t\t\t const struct mana_qset *qset)\n+{\n+\tctx-\u003etx_qp\t\t= qset-\u003etx_qp;\n+\tctx-\u003erxqs\t\t= qset-\u003erxqs;\n+\tctx-\u003eindir_table\t= qset-\u003eindir_table;\n+\tctx-\u003eindir_table_sz\t= qset-\u003eindir_table_sz;\n+\tctx-\u003erxobj_table\t= qset-\u003erxobj_table;\n+\tctx-\u003edefault_rxobj\t= qset-\u003edefault_rxobj;\n+\tctx-\u003enum_queues\t\t= qset-\u003enum_queues;\n+\tctx-\u003erx_queue_size\t= qset-\u003erx_queue_size;\n+\tctx-\u003etx_queue_size\t= qset-\u003etx_queue_size;\n+\tctx-\u003epriv_flags\t\t= qset-\u003epriv_flags;\n+\tctx-\u003econfigured_mtu\t= qset-\u003emtu;\n+\tctx-\u003ebpf_prog\t\t= qset-\u003ebpf_prog;\n+}\n+\n+/* Scratch starts without SQs/RQs and borrows the port's EQ pool. Never call\n+ * mana_destroy_eq() on it.\n+ */\n+struct mana_port_context *mana_qset_scratch_alloc(struct mana_port_context *apc)\n+{\n+\tstruct mana_port_context *scratch;\n+\n+\tscratch = kvzalloc_obj(*scratch, GFP_KERNEL);\n+\tif (!scratch)\n+\t\treturn NULL;\n+\n+\t*scratch = *apc;\n+\n+\tscratch-\u003etx_qp\t\t= NULL;\n+\tscratch-\u003erxqs\t\t= NULL;\n+\tscratch-\u003eindir_table\t= NULL;\n+\tscratch-\u003erxobj_table\t= NULL;\n+\tscratch-\u003edefault_rxobj\t= INVALID_MANA_HANDLE;\n+\n+\t/* Do not consume the live set's pre-allocated RX buffers. */\n+\tscratch-\u003erxbufs_pre\t= NULL;\n+\tscratch-\u003edas_pre\t= NULL;\n+\tscratch-\u003erxbpre_total\t= 0;\n+\n+\t/* Suppress debugfs names that would collide with the live set. */\n+\tscratch-\u003emana_port_debugfs = ERR_PTR(-ENODEV);\n+\n+\treturn scratch;\n+}\n+\n+void mana_qset_scratch_free(struct mana_port_context *scratch)\n+{\n+\tkvfree(scratch);\n+}\n+\n+/* Split into kept queues and a retiring tail without changing live ownership.\n+ * Queue i retains EQ i.\n+ */\n+int mana_split_qset(struct mana_port_context *apc,\n+\t\t struct mana_port_context *scratch, unsigned int new_count,\n+\t\t struct mana_qset *out_new, struct mana_qset *out_tail)\n+{\n+\tunsigned int old_count = apc-\u003enum_queues;\n+\tstruct mana_tx_qp **new_tx, **tail_tx;\n+\tstruct mana_rxq **new_rx, **tail_rx;\n+\tunsigned int tail_count;\n+\tbool indir_lost;\n+\tunsigned int i;\n+\tint err;\n+\n+\tASSERT_RTNL();\n+\n+\tif (WARN_ON(new_count == 0 || new_count \u003e= old_count))\n+\t\treturn -EINVAL;\n+\tif (WARN_ON(!apc-\u003etx_qp || !apc-\u003erxqs))\n+\t\treturn -EINVAL;\n+\n+\ttail_count = old_count - new_count;\n+\n+\t/* Build steering separately so it cannot index beyond the shortened RX\n+\t * array.\n+\t */\n+\tscratch-\u003enum_queues = new_count;\n+\terr = mana_rss_table_alloc(scratch);\n+\tif (err)\n+\t\treturn err;\n+\n+\tif (mana_rss_table_keep(apc, new_count, \u0026indir_lost))\n+\t\tmemcpy(scratch-\u003eindir_table, apc-\u003eindir_table,\n+\t\t apc-\u003eindir_table_sz * sizeof(*apc-\u003eindir_table));\n+\telse\n+\t\tmana_rss_table_init(scratch);\n+\n+\tnew_tx = kzalloc_objs(struct mana_tx_qp *, new_count);\n+\tnew_rx = kzalloc_objs(struct mana_rxq *, new_count);\n+\ttail_tx = kzalloc_objs(struct mana_tx_qp *, tail_count);\n+\ttail_rx = kzalloc_objs(struct mana_rxq *, tail_count);\n+\tif (!new_tx || !new_rx || !tail_tx || !tail_rx) {\n+\t\terr = -ENOMEM;\n+\t\tgoto free_arrays;\n+\t}\n+\n+\tfor (i = 0; i \u003c new_count; i++) {\n+\t\tnew_tx[i] = apc-\u003etx_qp[i];\n+\t\tnew_rx[i] = apc-\u003erxqs[i];\n+\t}\n+\tfor (i = 0; i \u003c tail_count; i++) {\n+\t\ttail_tx[i] = apc-\u003etx_qp[new_count + i];\n+\t\ttail_rx[i] = apc-\u003erxqs[new_count + i];\n+\t}\n+\n+\tout_new-\u003etx_qp\t\t= new_tx;\n+\tout_new-\u003erxqs\t\t= new_rx;\n+\tout_new-\u003eindir_table\t= scratch-\u003eindir_table;\n+\tout_new-\u003eindir_table_sz\t= scratch-\u003eindir_table_sz;\n+\tout_new-\u003erxobj_table\t= scratch-\u003erxobj_table;\n+\tout_new-\u003edefault_rxobj\t= apc-\u003erxqs[0]-\u003erxobj;\n+\tout_new-\u003enum_queues\t= new_count;\n+\tout_new-\u003erx_queue_size\t= apc-\u003erx_queue_size;\n+\tout_new-\u003etx_queue_size\t= apc-\u003etx_queue_size;\n+\tout_new-\u003epriv_flags\t= apc-\u003epriv_flags;\n+\tout_new-\u003emtu\t\t= apc-\u003econfigured_mtu;\n+\tout_new-\u003ebpf_prog\t= apc-\u003ebpf_prog;\n+\tout_new-\u003erxfh_indir_lost = indir_lost;\n+\n+\tscratch-\u003eindir_table\t= NULL;\n+\tscratch-\u003erxobj_table\t= NULL;\n+\n+\tmemset(out_tail, 0, sizeof(*out_tail));\n+\tout_tail-\u003etx_qp\t\t= tail_tx;\n+\tout_tail-\u003erxqs\t\t= tail_rx;\n+\tout_tail-\u003edefault_rxobj\t= INVALID_MANA_HANDLE;\n+\tout_tail-\u003enum_queues\t= tail_count;\n+\tout_tail-\u003erx_queue_size\t= apc-\u003erx_queue_size;\n+\tout_tail-\u003etx_queue_size\t= apc-\u003etx_queue_size;\n+\tout_tail-\u003epriv_flags\t= apc-\u003epriv_flags;\n+\tout_tail-\u003emtu\t\t= apc-\u003econfigured_mtu;\n+\tout_tail-\u003ebpf_prog\t= apc-\u003ebpf_prog;\n+\n+\treturn 0;\n+\n+free_arrays:\n+\tkfree(new_tx);\n+\tkfree(new_rx);\n+\tkfree(tail_tx);\n+\tkfree(tail_rx);\n+\tmana_cleanup_indir_table(scratch);\n+\treturn err;\n+}\n+\n+/* Free containers only; the live port still owns the queues. */\n+void mana_discard_split(struct mana_qset *newq, struct mana_qset *tailq)\n+{\n+\tkfree(newq-\u003etx_qp);\n+\tkfree(newq-\u003erxqs);\n+\tkfree(newq-\u003eindir_table);\n+\tkfree(newq-\u003erxobj_table);\n+\tkfree(tailq-\u003etx_qp);\n+\tkfree(tailq-\u003erxqs);\n+\tmemset(newq, 0, sizeof(*newq));\n+\tmemset(tailq, 0, sizeof(*tailq));\n+}\n+\n+/* Carry existing queues into @out_new; allocate only the tail. @out_fresh\n+ * isolates new queues for cleanup after a failed publish.\n+ */\n+int mana_grow_qset(struct mana_port_context *apc,\n+\t\t struct mana_port_context *scratch, unsigned int new_count,\n+\t\t struct mana_qset *out_new, struct mana_qset *out_fresh)\n+{\n+\tunsigned int old_count = apc-\u003enum_queues;\n+\tstruct mana_tx_qp **new_tx, **fresh_tx;\n+\tstruct mana_rxq **new_rx, **fresh_rx;\n+\tstruct net_device *ndev = apc-\u003endev;\n+\tunsigned int fresh_count;\n+\tbool indir_lost;\n+\tunsigned int i;\n+\tint err;\n+\n+\tASSERT_RTNL();\n+\n+\tif (WARN_ON(new_count \u003c= old_count))\n+\t\treturn -EINVAL;\n+\tif (WARN_ON(!apc-\u003etx_qp || !apc-\u003erxqs))\n+\t\treturn -EINVAL;\n+\n+\tfresh_count = new_count - old_count;\n+\n+\tnew_tx = kzalloc_objs(struct mana_tx_qp *, new_count);\n+\tnew_rx = kzalloc_objs(struct mana_rxq *, new_count);\n+\tfresh_tx = kzalloc_objs(struct mana_tx_qp *, fresh_count);\n+\tfresh_rx = kzalloc_objs(struct mana_rxq *, fresh_count);\n+\tif (!new_tx || !new_rx || !fresh_tx || !fresh_rx) {\n+\t\terr = -ENOMEM;\n+\t\tgoto free_arrays;\n+\t}\n+\n+\tfor (i = 0; i \u003c old_count; i++) {\n+\t\tnew_tx[i] = apc-\u003etx_qp[i];\n+\t\tnew_rx[i] = apc-\u003erxqs[i];\n+\t}\n+\n+\tscratch-\u003enum_queues = new_count;\n+\tscratch-\u003etx_qp = new_tx;\n+\tscratch-\u003erxqs = new_rx;\n+\n+\terr = mana_rss_table_alloc(scratch);\n+\tif (err)\n+\t\tgoto free_arrays;\n+\n+\terr = mana_grow_eqs(apc, new_count);\n+\tif (err)\n+\t\tgoto cleanup_rss;\n+\n+\tscratch-\u003eeqs = apc-\u003eeqs;\n+\tscratch-\u003enum_eqs = apc-\u003enum_eqs;\n+\n+\terr = mana_create_txq(scratch, ndev, old_count);\n+\tif (err)\n+\t\tgoto cleanup_rss;\n+\n+\terr = mana_add_rx_queues(scratch, ndev, old_count);\n+\tif (err)\n+\t\tgoto cleanup_rxq;\n+\n+\tif (mana_rss_table_keep(apc, new_count, \u0026indir_lost))\n+\t\tmemcpy(scratch-\u003eindir_table, apc-\u003eindir_table,\n+\t\t apc-\u003eindir_table_sz * sizeof(*apc-\u003eindir_table));\n+\telse\n+\t\tmana_rss_table_init(scratch);\n+\n+\tmana_qset_snapshot(scratch, out_new);\n+\tout_new-\u003erxfh_indir_lost = indir_lost;\n+\n+\tfor (i = 0; i \u003c fresh_count; i++) {\n+\t\tfresh_tx[i] = new_tx[old_count + i];\n+\t\tfresh_rx[i] = new_rx[old_count + i];\n+\t}\n+\n+\tmemset(out_fresh, 0, sizeof(*out_fresh));\n+\tout_fresh-\u003etx_qp\t= fresh_tx;\n+\tout_fresh-\u003erxqs\t\t= fresh_rx;\n+\tout_fresh-\u003edefault_rxobj = INVALID_MANA_HANDLE;\n+\tout_fresh-\u003enum_queues\t= fresh_count;\n+\tout_fresh-\u003erx_queue_size = apc-\u003erx_queue_size;\n+\tout_fresh-\u003etx_queue_size = apc-\u003etx_queue_size;\n+\tout_fresh-\u003epriv_flags\t= apc-\u003epriv_flags;\n+\tout_fresh-\u003emtu\t\t= apc-\u003econfigured_mtu;\n+\tout_fresh-\u003ebpf_prog\t= apc-\u003ebpf_prog;\n+\n+\t/* Take XDP refs on fresh RXQs only. On the merged set,\n+\t * mana_chn_setxdp() returns early on the carried rxqs[0].\n+\t */\n+\tmana_qset_install(scratch, out_fresh);\n+\tmana_chn_setxdp(scratch, mana_xdp_get(apc));\n+\n+\treturn 0;\n+\n+cleanup_rxq:\n+\tmana_destroy_rxqs_from(scratch, old_count);\n+\tmana_destroy_txq_from(scratch, old_count);\n+cleanup_rss:\n+\tmana_cleanup_indir_table(scratch);\n+free_arrays:\n+\t/* Free containers only; carried queues remain live. */\n+\tscratch-\u003etx_qp = NULL;\n+\tscratch-\u003erxqs = NULL;\n+\tkfree(new_tx);\n+\tkfree(new_rx);\n+\tkfree(fresh_tx);\n+\tkfree(fresh_rx);\n+\n+\tmana_shrink_eqs(apc, apc-\u003enum_queues);\n+\n+\tnetdev_err(ndev, \"%s(num_queues=%u) failed: %d\\n\", __func__,\n+\t\t new_count, err);\n+\treturn err;\n+}\n+\n+/* Free merged containers only, not carried queues. The caller must retire fresh\n+ * queues separately.\n+ */\n+void mana_discard_grow(struct mana_qset *newq)\n+{\n+\tkfree(newq-\u003etx_qp);\n+\tkfree(newq-\u003erxqs);\n+\tkfree(newq-\u003eindir_table);\n+\tkfree(newq-\u003erxobj_table);\n+\tmemset(newq, 0, sizeof(*newq));\n+}\n+\n+/* Rebuild at the current count; resize uses split/grow. */\n+int mana_alloc_qset(struct mana_port_context *apc,\n+\t\t struct mana_port_context *scratch,\n+\t\t unsigned int rx_queue_size, unsigned int tx_queue_size,\n+\t\t u32 priv_flags, int mtu, struct bpf_prog *bpf_prog,\n+\t\t struct mana_qset *out)\n+{\n+\tstruct net_device *ndev = scratch-\u003endev;\n+\tbool indir_lost;\n+\tint err;\n+\n+\tASSERT_RTNL();\n+\n+\tscratch-\u003enum_queues\t= apc-\u003enum_queues;\n+\tscratch-\u003erx_queue_size\t= rx_queue_size;\n+\tscratch-\u003etx_queue_size\t= tx_queue_size;\n+\tscratch-\u003epriv_flags\t= priv_flags;\n+\n+\tscratch-\u003econfigured_mtu\t= mtu;\n+\tscratch-\u003ebpf_prog\t= bpf_prog;\n+\n+\terr = mana_init_port_context(scratch);\n+\tif (err)\n+\t\tgoto out_err;\n+\n+\terr = mana_rss_table_alloc(scratch);\n+\tif (err)\n+\t\tgoto cleanup_rxq_array;\n+\n+\t/* Reuse the existing EQ pool; the queue count is unchanged. */\n+\tscratch-\u003eeqs = apc-\u003eeqs;\n+\tscratch-\u003enum_eqs = apc-\u003enum_eqs;\n+\n+\terr = mana_create_txq(scratch, ndev, 0);\n+\tif (err)\n+\t\tgoto cleanup_rss;\n+\n+\terr = mana_add_rx_queues(scratch, ndev, 0);\n+\tif (err)\n+\t\tgoto cleanup_rxq;\n+\n+\tif (mana_rss_table_keep(apc, scratch-\u003enum_queues, \u0026indir_lost))\n+\t\tmemcpy(scratch-\u003eindir_table, apc-\u003eindir_table,\n+\t\t apc-\u003eindir_table_sz * sizeof(*apc-\u003eindir_table));\n+\telse\n+\t\tmana_rss_table_init(scratch);\n+\n+\tmana_qset_snapshot(scratch, out);\n+\tout-\u003erxfh_indir_lost = indir_lost;\n+\treturn 0;\n+\n+cleanup_rxq:\n+\tmana_destroy_rxqs(scratch);\n+\tmana_destroy_txq(scratch);\n+cleanup_rss:\n+\tmana_cleanup_indir_table(scratch);\n+cleanup_rxq_array:\n+\tkfree(scratch-\u003erxqs);\n+\tscratch-\u003erxqs = NULL;\n+out_err:\n+\tnetdev_err(ndev, \"%s(num_queues=%u) failed: %d\\n\", __func__,\n+\t\t apc-\u003enum_queues, err);\n+\treturn err;\n+}\n+\n+/* Destroy caller-owned CQs before closing this dead-end port: closing also\n+ * frees the shared EQ pool. Requires RTNL.\n+ */\n+void mana_publish_close_if_needed(struct mana_port_context *apc)\n+{\n+\tASSERT_RTNL();\n+\n+\tif (!apc-\u003epublish_dead_end)\n+\t\treturn;\n+\n+\tapc-\u003epublish_dead_end = false;\n+\n+\tif (mana_dealloc_queues(apc-\u003endev))\n+\t\tnetdev_err(apc-\u003endev,\n+\t\t\t \"failed to close the port after a failed rollback\\n\");\n+}\n+\n+/* Carried-over queues may still have full rings. */\n+static void mana_start_txqs(struct mana_port_context *apc)\n+{\n+\tstruct net_device *ndev = apc-\u003endev;\n+\tunsigned int i;\n+\n+\tif (!apc-\u003etx_qp)\n+\t\treturn;\n+\n+\t/* Order port_is_up=true before ring reads to avoid a missed wakeup.\n+\t * Pair with mana_poll_tx_cq()'s full barrier after its tail update.\n+\t */\n+\tsmp_mb();\n+\n+\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\t\tif (!apc-\u003etx_qp[i])\n+\t\t\tcontinue;\n+\n+\t\tif (mana_can_tx(apc-\u003etx_qp[i]-\u003etxq.gdma_sq))\n+\t\t\tnetif_tx_wake_queue(netdev_get_tx_queue(ndev, i));\n+\t}\n+}\n+\n+/* Retiring completions must not wake replacement queues. Mark the leaving set\n+ * before unmarking the incoming set.\n+ */\n+static void mana_qset_set_retiring(struct mana_qset *qset,\n+\t\t\t\t const struct mana_qset *keep, bool retiring)\n+{\n+\tunsigned int q;\n+\n+\tfor (q = 0; q \u003c qset-\u003enum_queues; q++) {\n+\t\tif (qset-\u003etx_qp \u0026\u0026 qset-\u003etx_qp[q])\n+\t\t\tWRITE_ONCE(qset-\u003etx_qp[q]-\u003etxq.retiring, retiring);\n+\n+\t\tif (!qset-\u003erxqs || !qset-\u003erxqs[q])\n+\t\t\tcontinue;\n+\n+\t\t/* Carried RXQs remain the sole poll writers of shared slots. */\n+\t\tif (retiring \u0026\u0026 keep \u0026\u0026 q \u003c keep-\u003enum_queues \u0026\u0026\n+\t\t keep-\u003erxqs \u0026\u0026 keep-\u003erxqs[q] == qset-\u003erxqs[q])\n+\t\t\tcontinue;\n+\n+\t\t/* Switch to drain_stats; hand off shared slots after a grace\n+\t\t * period.\n+\t\t */\n+\t\tWRITE_ONCE(qset-\u003erxqs[q]-\u003eretiring, retiring);\n+\t}\n+}\n+\n+/* Leave TX stopped and request RX disable; steering may be unrecoverable. */\n+static void mana_publish_give_up(struct mana_port_context *apc)\n+{\n+\tint err;\n+\n+\tapc-\u003erss_state = TRI_STATE_FALSE;\n+\n+\terr = mana_disable_vport_rx(apc);\n+\tif (err \u0026\u0026 mana_en_need_log(apc, err))\n+\t\tnetdev_err(apc-\u003endev, \"failed to disable vPort RX: %d\\n\", err);\n+\n+\tapc-\u003ecarrier_forced_off = netif_carrier_ok(apc-\u003endev);\n+\tnetif_carrier_off(apc-\u003endev);\n+\tapc-\u003epublish_dead_end = true;\n+}\n+\n+/* Keep the RX count high until retiring RQs stop delivering their indices. */\n+static int mana_raise_real_num_rx(struct net_device *ndev, unsigned int count)\n+{\n+\tif (count \u003c= ndev-\u003ereal_num_rx_queues)\n+\t\treturn 0;\n+\n+\treturn netif_set_real_num_rx_queues(ndev, count);\n+}\n+\n+/* Publish under RTNL with TX gated. An error restores old pointers, not\n+ * necessarily service. Free only owned queues.\n+ */\n+int mana_publish_qset(struct mana_port_context *apc, struct mana_qset *newq,\n+\t\t struct mana_qset *out_old)\n+{\n+\tstruct net_device *ndev = apc-\u003endev;\n+\tint err;\n+\n+\tASSERT_RTNL();\n+\n+\t/* Close the XDP gate before stopping TX queues. Pair with\n+\t * mana_poll_tx_cq()'s smp_rmb() to prevent mid-swap wakeups.\n+\t */\n+\tWRITE_ONCE(apc-\u003eport_is_up, false);\n+\n+\t/* Ensure port state updated before txq state */\n+\tsmp_wmb();\n+\n+\tnetif_tx_disable(ndev);\n+\n+\tmana_qset_snapshot(apc, out_old);\n+\n+\t/* Mark before the grace period so old completions cannot wake the\n+\t * replacement's stopped queue.\n+\t */\n+\tmana_qset_set_retiring(out_old, newq, true);\n+\n+\t/* Drain TX/XDP readers past the gate and polls missing retiring. */\n+\tsynchronize_net();\n+\n+\tmana_qset_set_retiring(newq, NULL, false);\n+\n+\tmana_qset_install(apc, newq);\n+\tapc-\u003erss_state = apc-\u003enum_queues \u003e 1 ? TRI_STATE_TRUE : TRI_STATE_FALSE;\n+\n+\terr = netif_set_real_num_tx_queues(ndev, apc-\u003enum_queues);\n+\tif (err)\n+\t\tgoto rollback;\n+\n+\terr = mana_raise_real_num_rx(ndev, apc-\u003enum_queues);\n+\tif (err)\n+\t\tgoto rollback;\n+\n+\t/* Install XDP before steering reaches the incoming RXQs. */\n+\tmana_chn_setxdp(apc, mana_xdp_get(apc));\n+\n+\terr = mana_config_rss(apc, TRI_STATE_TRUE, true, true);\n+\tif (err)\n+\t\tgoto rollback;\n+\n+\tWRITE_ONCE(ndev-\u003emtu, apc-\u003econfigured_mtu);\n+\n+\t/* Publish fields before opening the gate; pair with TX/XDP read\n+\t * barriers. The post-gate full barrier cannot replace this.\n+\t */\n+\tsmp_wmb();\n+\n+\tWRITE_ONCE(apc-\u003eport_is_up, true);\n+\tmana_start_txqs(apc);\n+\n+\t/* Report a lost user table only after successful publication. */\n+\tif (newq-\u003erxfh_indir_lost)\n+\t\tethtool_rxfh_indir_lost(ndev);\n+\n+\treturn 0;\n+\n+rollback:\n+\tnetdev_err(ndev, \"%s failed: %d, restoring previous queue set\\n\",\n+\t\t __func__, err);\n+\n+\tmana_qset_set_retiring(newq, out_old, true);\n+\n+\t/* Quiesce new shared-slot writers before restoring old ones. */\n+\tsynchronize_net();\n+\n+\tmana_qset_set_retiring(out_old, NULL, false);\n+\n+\t/* Quiesce old drain_stats writers before folding. */\n+\tsynchronize_net();\n+\tmana_fold_qset_rx_stats(apc, out_old);\n+\n+\tmana_qset_install(apc, out_old);\n+\tapc-\u003erss_state = apc-\u003enum_queues \u003e 1 ? TRI_STATE_TRUE : TRI_STATE_FALSE;\n+\n+\tif (netif_set_real_num_tx_queues(ndev, apc-\u003enum_queues) ||\n+\t mana_raise_real_num_rx(ndev, apc-\u003enum_queues)) {\n+\t\t/* Inconsistent restored queue counts prohibit TX; leave the\n+\t\t * port stopped.\n+\t\t */\n+\t\tnetdev_err(ndev, \"failed to restore queue counts, closing the port\\n\");\n+\t\tmana_publish_give_up(apc);\n+\t\treturn err;\n+\t}\n+\n+\tif (mana_config_rss(apc, TRI_STATE_TRUE, true, true)) {\n+\t\t/* Do not reopen TX with mismatched steering; RX disable is\n+\t\t * best-effort.\n+\t\t */\n+\t\tnetdev_err(ndev, \"failed to restore RSS steering, closing the port\\n\");\n+\t\tmana_publish_give_up(apc);\n+\t\treturn err;\n+\t}\n+\n+\tWRITE_ONCE(ndev-\u003emtu, apc-\u003econfigured_mtu);\n+\n+\t/* Publish restored fields before reopening the gate, as on success. */\n+\tsmp_wmb();\n+\n+\tWRITE_ONCE(apc-\u003eport_is_up, true);\n+\tmana_start_txqs(apc);\n+\n+\treturn err;\n+}\n+\n+/* Create missing debugfs nodes once retiring names are gone. */\n+void mana_qset_debugfs_publish(struct mana_port_context *apc)\n+{\n+\tunsigned int i;\n+\n+\tASSERT_RTNL();\n+\n+\tif (IS_ERR_OR_NULL(apc-\u003emana_port_debugfs))\n+\t\treturn;\n+\n+\tfor (i = 0; i \u003c apc-\u003enum_queues; i++) {\n+\t\tif (apc-\u003etx_qp \u0026\u0026 apc-\u003etx_qp[i] \u0026\u0026\n+\t\t IS_ERR_OR_NULL(apc-\u003etx_qp[i]-\u003emana_tx_debugfs))\n+\t\t\tmana_create_txq_debugfs(apc, i);\n+\n+\t\tif (apc-\u003erxqs \u0026\u0026 apc-\u003erxqs[i] \u0026\u0026\n+\t\t IS_ERR_OR_NULL(apc-\u003erxqs[i]-\u003emana_rx_debugfs))\n+\t\t\tmana_create_rxq_debugfs(apc, i);\n+\t}\n+}\n+\n+/* Under RTNL, free only queues no longer shared with the installed set. */\n+void mana_free_qset(struct mana_port_context *scratch, struct mana_qset *qset)\n+{\n+\tstruct mana_port_context *apc = netdev_priv(scratch-\u003endev);\n+\tstruct bpf_prog *retiring_prog;\n+\tunsigned int retiring_queues;\n+\n+\tASSERT_RTNL();\n+\n+\tif (!qset-\u003erxqs \u0026\u0026 !qset-\u003etx_qp)\n+\t\treturn;\n+\n+\tif (qset-\u003etx_qp) {\n+\t\tunsigned int q;\n+\n+\t\tfor (q = 0; q \u003c qset-\u003enum_queues; q++) {\n+\t\t\tif (qset-\u003etx_qp[q])\n+\t\t\t\tWRITE_ONCE(qset-\u003etx_qp[q]-\u003etxq.retiring, true);\n+\t\t}\n+\t}\n+\n+\t/* Keep retired queues and arrays through this grace period; local NAPI\n+\t * synchronization does not drain other devices' XDP.\n+\t */\n+\tsynchronize_net();\n+\n+\tmana_qset_install(scratch, qset);\n+\n+\t/* Keep retiring RXQs' XDP programs and references until RX teardown.\n+\t * Read the program from the queues, not queue-set metadata.\n+\t */\n+\tretiring_prog = mana_chn_xdp_peek(scratch);\n+\tretiring_queues = scratch-\u003enum_queues;\n+\n+\t/* Drain TX before unmapping RX buffers. */\n+\tif (mana_drain_txqs(scratch)) {\n+\t\t/* FLR also destroys the HWC; rebuilding ports is best-effort.\n+\t\t * This path does not reinitialize the device.\n+\t\t */\n+\t\tstruct mana_context *ac = apc-\u003eac;\n+\t\tstruct mana_port_context *sib;\n+\t\tunsigned int i;\n+\n+\t\tnetdev_err(scratch-\u003endev,\n+\t\t\t \"device reset while retiring a queue set, scheduling port reset\\n\");\n+\n+\t\tfor (i = 0; i \u003c ac-\u003enum_ports; i++) {\n+\t\t\tif (!ac-\u003eports[i])\n+\t\t\t\tcontinue;\n+\t\t\tsib = netdev_priv(ac-\u003eports[i]);\n+\t\t\tqueue_work(ac-\u003eper_port_queue_reset_wq,\n+\t\t\t\t \u0026sib-\u003equeue_reset_work);\n+\t\t}\n+\t}\n+\n+\t/* Fence RQs before unmapping, but teardown proceeds on errors. */\n+\tmana_fence_rqs(scratch);\n+\n+\tmana_destroy_rxqs(scratch);\n+\n+\tmana_chn_xdp_release(retiring_prog, retiring_queues);\n+\n+\tmana_destroy_txq(scratch);\n+\tmana_cleanup_indir_table(scratch);\n+\tkfree(scratch-\u003erxqs);\n+\tscratch-\u003erxqs = NULL;\n+\n+\tmemset(qset, 0, sizeof(*qset));\n+\n+\t/* Retiring RQs can no longer deliver indices beyond the live queue\n+\t * count.\n+\t */\n+\tnetif_set_real_num_rx_queues(apc-\u003endev, apc-\u003enum_queues);\n+\n+\tmana_shrink_eqs(apc, apc-\u003enum_queues);\n+\n+\tmana_qset_debugfs_publish(apc);\n+}\n+\n int mana_detach(struct net_device *ndev, bool from_close)\n {\n \tstruct mana_port_context *apc = netdev_priv(ndev);\n@@ -3786,10 +4806,8 @@ int mana_detach(struct net_device *ndev, bool from_close)\n \n \tif (apc-\u003eport_st_save) {\n \t\terr = mana_dealloc_queues(ndev);\n-\t\tif (err) {\n+\t\tif (err)\n \t\t\tnetdev_err(ndev, \"%s failed to deallocate queues: %d\\n\", __func__, err);\n-\t\t\treturn err;\n-\t\t}\n \t}\n \n \tif (!from_close) {\n@@ -3826,6 +4844,7 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,\n \tapc-\u003eport_handle = INVALID_MANA_HANDLE;\n \tapc-\u003epf_filter_handle = INVALID_MANA_HANDLE;\n \tapc-\u003eport_idx = port_idx;\n+\tapc-\u003econfigured_mtu = ndev-\u003emtu;\n \tapc-\u003elink_cfg_error = 1;\n \tapc-\u003ecqe_coalescing_enable = 0;\n \tapc-\u003ecqe8_coalescing_enable = 0;\n@@ -3840,6 +4859,10 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,\n \t\tapc-\u003etx_dim_enabled = MANA_ADAPTIVE_TX_DEF;\n \t}\n \n+\terr = mana_alloc_queue_stats(apc);\n+\tif (err)\n+\t\tgoto free_net;\n+\n \tmutex_init(\u0026apc-\u003evport_mutex);\n \tapc-\u003evport_use_count = 0;\n \n@@ -3862,7 +4885,7 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,\n \n \terr = mana_init_port(ndev);\n \tif (err)\n-\t\tgoto free_net;\n+\t\tgoto free_stats;\n \n \terr = mana_rss_table_alloc(apc);\n \tif (err)\n@@ -3899,6 +4922,8 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,\n \tmana_cleanup_indir_table(apc);\n reset_apc:\n \tmana_cleanup_port_context(apc);\n+free_stats:\n+\tmana_free_queue_stats(apc);\n free_net:\n \t*ndev_storage = NULL;\n \tnetdev_err(ndev, \"Failed to probe vPort %d: %d\\n\", port_idx, err);\n@@ -4239,6 +5264,11 @@ void mana_remove(struct gdma_dev *gd, bool suspending)\n \n \t\tunregister_netdevice(ndev);\n \t\tmana_cleanup_indir_table(apc);\n+\t\tmana_free_queue_stats(apc);\n+\n+\t\t/* Remove the port from reset walks before freeing its netdev.\n+\t\t */\n+\t\tac-\u003eports[i] = NULL;\n \n \t\trtnl_unlock();\n \ndiff --git a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c\nindex ece7ff9cc409a..6586a9d2a1343 100644\n--- a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c\n+++ b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c\n@@ -242,6 +242,12 @@ static void mana_get_ethtool_stats(struct net_device *ndev,\n \tu64 xdp_tx;\n \tu64 pkt_len0_err;\n \tu64 coalesced_cqe[MANA_CQE_COAL_PKTS_8 - 1];\n+\tu64 ret_coalesced_cqe[MANA_CQE_COAL_PKTS_8 - 1];\n+\tu64 ret_packets, ret_bytes;\n+\tu64 ret_xdp_redirect;\n+\tu64 ret_pkt_len0_err;\n+\tu64 ret_xdp_drop;\n+\tu64 ret_xdp_tx;\n \tu64 tso_packets;\n \tu64 tso_bytes;\n \tu64 tso_inner_packets;\n@@ -252,14 +258,11 @@ static void mana_get_ethtool_stats(struct net_device *ndev,\n \tu64 mana_map_err;\n \tint q, i = 0, j;\n \n-\tif (!apc-\u003eport_is_up)\n-\t\treturn;\n-\n-\t/* We call this mana function to get the phy stats from GDMA and includes\n-\t * aggregate tx/rx drop counters, Per-TC(Traffic Channel) tx/rx and pause\n-\t * counters.\n+\t/* Counters outlive the queues, but suspend can destroy the HW channel\n+\t * while the netdev remains registered. Gate only the PHY query.\n \t */\n-\tmana_query_phy_stats(apc);\n+\tif (apc-\u003eport_is_up)\n+\t\tmana_query_phy_stats(apc);\n \n \tfor (q = 0; q \u003c ARRAY_SIZE(mana_eth_stats); q++)\n \t\tdata[i++] = *(u64 *)(eth_stats + mana_eth_stats[q].offset);\n@@ -271,7 +274,7 @@ static void mana_get_ethtool_stats(struct net_device *ndev,\n \t\tdata[i++] = *(u64 *)(phy_stats + mana_phy_stats[q].offset);\n \n \tfor (q = 0; q \u003c num_queues; q++) {\n-\t\trx_stats = \u0026apc-\u003erxqs[q]-\u003estats;\n+\t\trx_stats = \u0026apc-\u003erxq_stats[q];\n \n \t\tdo {\n \t\t\tstart = u64_stats_fetch_begin(\u0026rx_stats-\u003esyncp);\n@@ -285,6 +288,33 @@ static void mana_get_ethtool_stats(struct net_device *ndev,\n \t\t\t\tcoalesced_cqe[j] = rx_stats-\u003ecoalesced_cqe[j];\n \t\t} while (u64_stats_fetch_retry(\u0026rx_stats-\u003esyncp, start));\n \n+\t\t/* Snapshot separately so a retry cannot add retired counters\n+\t\t * twice.\n+\t\t */\n+\t\trx_stats = \u0026apc-\u003erxq_stats_ret[q];\n+\n+\t\tdo {\n+\t\t\tstart = u64_stats_fetch_begin(\u0026rx_stats-\u003esyncp);\n+\t\t\tret_packets = rx_stats-\u003epackets;\n+\t\t\tret_bytes = rx_stats-\u003ebytes;\n+\t\t\tret_xdp_drop = rx_stats-\u003exdp_drop;\n+\t\t\tret_xdp_tx = rx_stats-\u003exdp_tx;\n+\t\t\tret_xdp_redirect = rx_stats-\u003exdp_redirect;\n+\t\t\tret_pkt_len0_err = rx_stats-\u003epkt_len0_err;\n+\t\t\tfor (j = 0; j \u003c MANA_CQE_COAL_PKTS_8 - 1; j++)\n+\t\t\t\tret_coalesced_cqe[j] =\n+\t\t\t\t\trx_stats-\u003ecoalesced_cqe[j];\n+\t\t} while (u64_stats_fetch_retry(\u0026rx_stats-\u003esyncp, start));\n+\n+\t\tpackets += ret_packets;\n+\t\tbytes += ret_bytes;\n+\t\txdp_drop += ret_xdp_drop;\n+\t\txdp_tx += ret_xdp_tx;\n+\t\txdp_redirect += ret_xdp_redirect;\n+\t\tpkt_len0_err += ret_pkt_len0_err;\n+\t\tfor (j = 0; j \u003c MANA_CQE_COAL_PKTS_8 - 1; j++)\n+\t\t\tcoalesced_cqe[j] += ret_coalesced_cqe[j];\n+\n \t\tdata[i++] = packets;\n \t\tdata[i++] = bytes;\n \t\tdata[i++] = xdp_drop;\n@@ -296,7 +326,7 @@ static void mana_get_ethtool_stats(struct net_device *ndev,\n \t}\n \n \tfor (q = 0; q \u003c num_queues; q++) {\n-\t\ttx_stats = \u0026apc-\u003etx_qp[q]-\u003etxq.stats;\n+\t\ttx_stats = \u0026apc-\u003etxq_stats[q];\n \n \t\tdo {\n \t\t\tstart = u64_stats_fetch_begin(\u0026tx_stats-\u003esyncp);\n@@ -648,52 +678,126 @@ static int mana_set_coalesce(struct net_device *ndev,\n \treturn 0;\n }\n \n-/* mana_set_channels - change the number of queues on a port\n- *\n- * Returns -EBUSY if RDMA holds the vport with EQs sized to the\n- * current num_queues.\n- */\n static int mana_set_channels(struct net_device *ndev,\n \t\t\t struct ethtool_channels *channels)\n {\n \tstruct mana_port_context *apc = netdev_priv(ndev);\n \tunsigned int new_count = channels-\u003ecombined_count;\n-\tunsigned int old_count = apc-\u003enum_queues;\n+\tstruct mana_port_context *scratch;\n+\tstruct mana_qset newq, oldq, freshq;\n \tint err;\n \n-\t/* Set channel_changing to block RDMA from grabbing the vport\n-\t * during the detach/attach window. mana_cfg_vport() checks\n-\t * this flag under vport_mutex and returns -EBUSY if set.\n+\tif (new_count \u003c 1 || new_count \u003e apc-\u003emax_queues) {\n+\t\tnetdev_err(ndev, \"Invalid combined_count %u (max %u)\\n\",\n+\t\t\t new_count, apc-\u003emax_queues);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tif (new_count == apc-\u003enum_queues)\n+\t\treturn 0;\n+\n+\t/* Resize rxqs while down: mana_open() does not recreate the port\n+\t * context. RDMA must not own the vport while num_queues changes.\n \t */\n \tmutex_lock(\u0026apc-\u003evport_mutex);\n-\tif (!apc-\u003eport_is_up \u0026\u0026 apc-\u003evport_use_count) {\n+\tif (!apc-\u003eport_is_up) {\n+\t\tstruct mana_rxq **rxqs;\n+\n+\t\tif (apc-\u003evport_use_count) {\n+\t\t\tmutex_unlock(\u0026apc-\u003evport_mutex);\n+\t\t\treturn -EBUSY;\n+\t\t}\n+\n+\t\trxqs = kzalloc_objs(struct mana_rxq *, new_count);\n+\t\tif (!rxqs) {\n+\t\t\tmutex_unlock(\u0026apc-\u003evport_mutex);\n+\t\t\treturn -ENOMEM;\n+\t\t}\n+\n+\t\tkfree(apc-\u003erxqs);\n+\t\tapc-\u003erxqs = rxqs;\n+\t\tapc-\u003enum_queues = new_count;\n+\t\tmutex_unlock(\u0026apc-\u003evport_mutex);\n+\t\treturn 0;\n+\t}\n+\n+\t/* The Ethernet port already holds a vport reference; exclude RDMA\n+\t * through failure cleanup.\n+\t */\n+\tif (apc-\u003echannel_changing) {\n \t\tmutex_unlock(\u0026apc-\u003evport_mutex);\n \t\treturn -EBUSY;\n \t}\n \tapc-\u003echannel_changing = true;\n \tmutex_unlock(\u0026apc-\u003evport_mutex);\n \n-\terr = mana_pre_alloc_rxbufs(apc, ndev-\u003emtu, new_count);\n-\tif (err) {\n-\t\tnetdev_err(ndev, \"Insufficient memory for new allocations\");\n+\tscratch = mana_qset_scratch_alloc(apc);\n+\tif (!scratch) {\n+\t\terr = -ENOMEM;\n \t\tgoto clear_flag;\n \t}\n \n-\terr = mana_detach(ndev, false);\n-\tif (err) {\n-\t\tnetdev_err(ndev, \"mana_detach failed: %d\\n\", err);\n-\t\tgoto out;\n+\tif (new_count \u003c apc-\u003enum_queues) {\n+\t\tstruct mana_qset tailq;\n+\n+\t\terr = mana_split_qset(apc, scratch, new_count, \u0026newq, \u0026tailq);\n+\t\tif (err)\n+\t\t\tgoto free_scratch;\n+\n+\t\terr = mana_publish_qset(apc, \u0026newq, \u0026oldq);\n+\t\tif (err) {\n+\t\t\t/* Discard containers only; their queues still belong to\n+\t\t\t * the old set.\n+\t\t\t */\n+\t\t\tmana_discard_split(\u0026newq, \u0026tailq);\n+\t\t\tgoto free_scratch;\n+\t\t}\n+\n+\t\t/* Wait for ndo_select_queue() readers of oldq.indir_table. Free\n+\t\t * only containers; the queues belong to the kept set or tail.\n+\t\t */\n+\t\tsynchronize_net();\n+\n+\t\tkfree(oldq.tx_qp);\n+\t\tkfree(oldq.rxqs);\n+\t\tkfree(oldq.indir_table);\n+\t\tkfree(oldq.rxobj_table);\n+\n+\t\tmana_free_qset(scratch, \u0026tailq);\n+\t\tgoto free_scratch;\n \t}\n \n-\tapc-\u003enum_queues = new_count;\n-\terr = mana_attach(ndev);\n+\terr = mana_grow_qset(apc, scratch, new_count, \u0026newq, \u0026freshq);\n+\tif (err)\n+\t\tgoto free_scratch;\n+\n+\terr = mana_publish_qset(apc, \u0026newq, \u0026oldq);\n \tif (err) {\n-\t\tapc-\u003enum_queues = old_count;\n-\t\tnetdev_err(ndev, \"mana_attach failed: %d\\n\", err);\n+\t\t/* Free only the new queues, then discard the merged containers.\n+\t\t */\n+\t\tmana_free_qset(scratch, \u0026freshq);\n+\t\tmana_discard_grow(\u0026newq);\n+\t\tgoto free_scratch;\n \t}\n \n-out:\n-\tmana_pre_dealloc_rxbufs(apc);\n+\t/* Wait for ndo_select_queue() readers of oldq.indir_table. All queues\n+\t * are now live in newq; free only the old and fresh containers.\n+\t */\n+\tsynchronize_net();\n+\n+\tkfree(oldq.tx_qp);\n+\tkfree(oldq.rxqs);\n+\tkfree(oldq.indir_table);\n+\tkfree(oldq.rxobj_table);\n+\tkfree(freshq.tx_qp);\n+\tkfree(freshq.rxqs);\n+\n+\t/* No retirement runs to publish the new queues' debugfs nodes. */\n+\tmana_qset_debugfs_publish(apc);\n+\n+free_scratch:\n+\tmana_publish_close_if_needed(apc);\n+\tmana_qset_scratch_free(scratch);\n clear_flag:\n \tmutex_lock(\u0026apc-\u003evport_mutex);\n \tapc-\u003echannel_changing = false;\n@@ -720,13 +824,11 @@ static int mana_set_ringparam(struct net_device *ndev,\n \t\t\t struct netlink_ext_ack *extack)\n {\n \tstruct mana_port_context *apc = netdev_priv(ndev);\n+\tstruct mana_port_context *scratch;\n+\tstruct mana_qset newq, oldq;\n \tu32 new_tx, new_rx;\n-\tu32 old_tx, old_rx;\n \tint err;\n \n-\told_tx = apc-\u003etx_queue_size;\n-\told_rx = apc-\u003erx_queue_size;\n-\n \tif (ring-\u003etx_pending \u003c MIN_TX_BUFFERS_PER_QUEUE) {\n \t\tNL_SET_ERR_MSG_FMT(extack, \"tx:%d less than the min:%d\", ring-\u003etx_pending,\n \t\t\t\t MIN_TX_BUFFERS_PER_QUEUE);\n@@ -744,32 +846,56 @@ static int mana_set_ringparam(struct net_device *ndev,\n \tnetdev_info(ndev, \"Using nearest power of 2 values for Txq:%d Rxq:%d\\n\",\n \t\t new_tx, new_rx);\n \n-\t/* pre-allocating new buffers to prevent failures in mana_attach() later */\n-\tapc-\u003erx_queue_size = new_rx;\n-\terr = mana_pre_alloc_rxbufs(apc, ndev-\u003emtu, apc-\u003enum_queues);\n-\tapc-\u003erx_queue_size = old_rx;\n-\tif (err) {\n-\t\tnetdev_err(ndev, \"Insufficient memory for new allocations\\n\");\n-\t\treturn err;\n+\tif (new_rx == apc-\u003erx_queue_size \u0026\u0026 new_tx == apc-\u003etx_queue_size)\n+\t\treturn 0;\n+\n+\tif (!apc-\u003eport_is_up) {\n+\t\tapc-\u003erx_queue_size = new_rx;\n+\t\tapc-\u003etx_queue_size = new_tx;\n+\t\treturn 0;\n \t}\n \n-\terr = mana_detach(ndev, false);\n-\tif (err) {\n-\t\tnetdev_err(ndev, \"mana_detach failed: %d\\n\", err);\n-\t\tgoto out;\n+\t/* Exclude RDMA through failure cleanup, which may release the vport. */\n+\tmutex_lock(\u0026apc-\u003evport_mutex);\n+\tif (apc-\u003echannel_changing) {\n+\t\tmutex_unlock(\u0026apc-\u003evport_mutex);\n+\t\treturn -EBUSY;\n+\t}\n+\tapc-\u003echannel_changing = true;\n+\tmutex_unlock(\u0026apc-\u003evport_mutex);\n+\n+\tscratch = mana_qset_scratch_alloc(apc);\n+\tif (!scratch) {\n+\t\terr = -ENOMEM;\n+\t\tgoto clear_flag;\n \t}\n \n-\tapc-\u003etx_queue_size = new_tx;\n-\tapc-\u003erx_queue_size = new_rx;\n+\terr = mana_alloc_qset(apc, scratch, new_rx, new_tx,\n+\t\t\t apc-\u003epriv_flags, apc-\u003econfigured_mtu,\n+\t\t\t apc-\u003ebpf_prog, \u0026newq);\n+\tif (err) {\n+\t\tNL_SET_ERR_MSG_FMT(extack, \"failed to change ring params: %d\",\n+\t\t\t\t err);\n+\t\tgoto free_scratch;\n+\t}\n \n-\terr = mana_attach(ndev);\n+\terr = mana_publish_qset(apc, \u0026newq, \u0026oldq);\n \tif (err) {\n-\t\tnetdev_err(ndev, \"mana_attach failed: %d\\n\", err);\n-\t\tapc-\u003etx_queue_size = old_tx;\n-\t\tapc-\u003erx_queue_size = old_rx;\n+\t\tNL_SET_ERR_MSG_FMT(extack, \"failed to change ring params: %d\",\n+\t\t\t\t err);\n+\t\tmana_free_qset(scratch, \u0026newq);\n+\t\tgoto free_scratch;\n \t}\n-out:\n-\tmana_pre_dealloc_rxbufs(apc);\n+\n+\tmana_free_qset(scratch, \u0026oldq);\n+\n+free_scratch:\n+\tmana_publish_close_if_needed(apc);\n+\tmana_qset_scratch_free(scratch);\n+clear_flag:\n+\tmutex_lock(\u0026apc-\u003evport_mutex);\n+\tapc-\u003echannel_changing = false;\n+\tmutex_unlock(\u0026apc-\u003evport_mutex);\n \treturn err;\n }\n \n@@ -799,7 +925,8 @@ static int mana_set_priv_flags(struct net_device *ndev, u32 priv_flags)\n {\n \tstruct mana_port_context *apc = netdev_priv(ndev);\n \tu32 changed = apc-\u003epriv_flags ^ priv_flags;\n-\tu32 old_priv_flags = apc-\u003epriv_flags;\n+\tstruct mana_port_context *scratch;\n+\tstruct mana_qset newq, oldq;\n \tint err = 0;\n \n \tif (!changed)\n@@ -809,54 +936,50 @@ static int mana_set_priv_flags(struct net_device *ndev, u32 priv_flags)\n \tif (priv_flags \u0026 ~GENMASK(MANA_PRIV_FLAG_MAX - 1, 0))\n \t\treturn -EINVAL;\n \n-\tapc-\u003epriv_flags = priv_flags;\n-\n-\tif (changed \u0026 BIT(MANA_PRIV_FLAG_USE_FULL_PAGE_RXBUF)) {\n-\t\tif (!apc-\u003eport_is_up)\n-\t\t\treturn 0;\n-\n-\t\t/* If XDP is attached or MTU is jumbo, single-buffer-per-page\n-\t\t * is already forced regardless of this flag. Skip the\n-\t\t * expensive detach/attach cycle since nothing changes.\n-\t\t */\n-\t\tif (ndev-\u003emtu + MANA_RXBUF_PAD \u003e PAGE_SIZE / 2 ||\n-\t\t mana_xdp_get(apc))\n-\t\t\treturn 0;\n+\t/* Skip rebuilding when full-page RX is already required. */\n+\tif (!(changed \u0026 BIT(MANA_PRIV_FLAG_USE_FULL_PAGE_RXBUF)) ||\n+\t !apc-\u003eport_is_up ||\n+\t ndev-\u003emtu + MANA_RXBUF_PAD \u003e PAGE_SIZE / 2 ||\n+\t mana_xdp_get(apc)) {\n+\t\tapc-\u003epriv_flags = priv_flags;\n+\t\treturn 0;\n+\t}\n \n-\t\t/* Block RDMA from grabbing the vport during detach/attach */\n-\t\tmutex_lock(\u0026apc-\u003evport_mutex);\n-\t\tapc-\u003echannel_changing = true;\n+\tmutex_lock(\u0026apc-\u003evport_mutex);\n+\tif (apc-\u003echannel_changing) {\n \t\tmutex_unlock(\u0026apc-\u003evport_mutex);\n+\t\treturn -EBUSY;\n+\t}\n+\tapc-\u003echannel_changing = true;\n+\tmutex_unlock(\u0026apc-\u003evport_mutex);\n \n-\t\terr = mana_pre_alloc_rxbufs(apc, ndev-\u003emtu, apc-\u003enum_queues);\n-\t\tif (err) {\n-\t\t\tnetdev_err(ndev,\n-\t\t\t\t \"Insufficient memory for new allocations\\n\");\n-\t\t\tapc-\u003epriv_flags = old_priv_flags;\n-\t\t\tgoto clear_flag;\n-\t\t}\n+\tscratch = mana_qset_scratch_alloc(apc);\n+\tif (!scratch) {\n+\t\terr = -ENOMEM;\n+\t\tgoto clear_flag;\n+\t}\n \n-\t\terr = mana_detach(ndev, false);\n-\t\tif (err) {\n-\t\t\tnetdev_err(ndev, \"mana_detach failed: %d\\n\", err);\n-\t\t\tapc-\u003epriv_flags = old_priv_flags;\n-\t\t\tgoto out;\n-\t\t}\n+\terr = mana_alloc_qset(apc, scratch, apc-\u003erx_queue_size,\n+\t\t\t apc-\u003etx_queue_size, priv_flags,\n+\t\t\t apc-\u003econfigured_mtu, apc-\u003ebpf_prog, \u0026newq);\n+\tif (err)\n+\t\tgoto free_scratch;\n \n-\t\terr = mana_attach(ndev);\n-\t\tif (err) {\n-\t\t\tnetdev_err(ndev, \"mana_attach failed: %d\\n\", err);\n-\t\t\tapc-\u003epriv_flags = old_priv_flags;\n-\t\t}\n+\terr = mana_publish_qset(apc, \u0026newq, \u0026oldq);\n+\tif (err) {\n+\t\tmana_free_qset(scratch, \u0026newq);\n+\t\tgoto free_scratch;\n \t}\n \n-out:\n-\tmana_pre_dealloc_rxbufs(apc);\n+\tmana_free_qset(scratch, \u0026oldq);\n+\n+free_scratch:\n+\tmana_publish_close_if_needed(apc);\n+\tmana_qset_scratch_free(scratch);\n clear_flag:\n \tmutex_lock(\u0026apc-\u003evport_mutex);\n \tapc-\u003echannel_changing = false;\n \tmutex_unlock(\u0026apc-\u003evport_mutex);\n-\n \treturn err;\n }\n \n@@ -871,7 +994,8 @@ const struct ethtool_ops mana_ethtool_ops = {\n \t.op_needs_rtnl\t\t= ETHTOOL_OP_NEEDS_RTNL_SCHANNELS |\n \t\t\t\t ETHTOOL_OP_NEEDS_RTNL_SRINGPARAM |\n \t\t\t\t ETHTOOL_OP_NEEDS_RTNL_SPFLAGS |\n-\t\t\t\t ETHTOOL_OP_NEEDS_RTNL_GLINK,\n+\t\t\t\t ETHTOOL_OP_NEEDS_RTNL_GLINK |\n+\t\t\t\t ETHTOOL_OP_NEEDS_RTNL_RSS,\n \t.get_ethtool_stats\t= mana_get_ethtool_stats,\n \t.get_sset_count\t\t= mana_get_sset_count,\n \t.get_strings\t\t= mana_get_strings,\ndiff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h\nindex 308950f9b54b0..666565ffb26aa 100644\n--- a/include/net/mana/gdma.h\n+++ b/include/net/mana/gdma.h\n@@ -686,6 +686,11 @@ enum {\n /* Driver supports non-contiguous queue buffers */\n #define GDMA_DRV_CAP_FLAG_1_NON_CONTIGUOUS_BUFFERS BIT(30)\n \n+/* Resize failures are handled in-driver; a failed rollback still needs\n+ * recovery.\n+ */\n+#define GDMA_DRV_CAP_FLAG_1_SELF_RECOVERY_ON_QUEUE_RESIZE_FAILURE BIT(31)\n+\n #define GDMA_DRV_CAP_FLAGS1 \\\n \t(GDMA_DRV_CAP_FLAG_1_EQ_SHARING_MULTI_VPORT | \\\n \t GDMA_DRV_CAP_FLAG_1_NAPI_WKDONE_FIX | \\\n@@ -703,7 +708,8 @@ enum {\n \t GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECOVERY | \\\n \t GDMA_DRV_CAP_FLAG_1_EQ_MSI_UNSHARE_MULTI_VPORT | \\\n \t GDMA_DRV_CAP_FLAG_1_DYN_INTERRUPT_MODERATION | \\\n-\t GDMA_DRV_CAP_FLAG_1_NON_CONTIGUOUS_BUFFERS)\n+\t GDMA_DRV_CAP_FLAG_1_NON_CONTIGUOUS_BUFFERS | \\\n+\t GDMA_DRV_CAP_FLAG_1_SELF_RECOVERY_ON_QUEUE_RESIZE_FAILURE)\n \n #define GDMA_DRV_CAP_FLAGS2 0\n \ndiff --git a/include/net/mana/mana.h b/include/net/mana/mana.h\nindex 83b7eff4646ea..cc936ae97a031 100644\n--- a/include/net/mana/mana.h\n+++ b/include/net/mana/mana.h\n@@ -102,7 +102,7 @@ struct mana_stats_rx {\n \tu64 pkt_len0_err;\n \tu64 coalesced_cqe[MANA_CQE_COAL_PKTS_8 - 1];\n \tstruct u64_stats_sync syncp;\n-};\n+} ____cacheline_aligned_in_smp;\n \n struct mana_stats_tx {\n \tu64 packets;\n@@ -117,7 +117,7 @@ struct mana_stats_tx {\n \tu64 csum_partial;\n \tu64 mana_map_err;\n \tstruct u64_stats_sync syncp;\n-};\n+} ____cacheline_aligned_in_smp;\n \n struct mana_txq {\n \tstruct gdma_queue *gdma_sq;\n@@ -143,7 +143,12 @@ struct mana_txq {\n \n \tbool napi_initialized;\n \n-\tstruct mana_stats_tx stats;\n+\tu32 reset_gen;\n+\n+\t/* Suppress completion wakeups on the replacement's netdev queue. */\n+\tbool retiring;\n+\n+\tstruct mana_stats_tx *stats;\n };\n \n /* skb data and frags dma mappings */\n@@ -405,7 +410,16 @@ struct mana_rxq {\n \n \tu32 buf_index;\n \n-\tstruct mana_stats_rx stats;\n+\t/* Port-owned live slot; use mana_rxq_stats() to select the writer's\n+\t * slot.\n+\t */\n+\tstruct mana_stats_rx *stats;\n+\n+\t/* Set under RTNL before another queue takes over this index. */\n+\tbool retiring;\n+\n+\t/* Folded under RTNL after drain-stat writers quiesce. */\n+\tstruct mana_stats_rx drain_stats;\n \n \tstruct bpf_prog __rcu *bpf_prog;\n \tstruct xdp_rxq_info xdp_rxq;\n@@ -537,6 +551,12 @@ struct mana_context {\n \tu8 bm_hostmode;\n \n \tstruct mana_ethtool_hc_stats hc_stats;\n+\n+\t/* Generation of successful resets issued by mana_drain_txqs(), under\n+\t * RTNL.\n+\t */\n+\tu32 reset_gen;\n+\n \tstruct workqueue_struct *per_port_queue_reset_wq;\n \t/* Workqueue for querying hardware stats */\n \tstruct delayed_work gf_stats_work;\n@@ -559,7 +579,9 @@ struct mana_port_context {\n \n \tu8 mac_addr[ETH_ALEN];\n \n+\t/* Port-owned EQ pool: max_queues slots, num_eqs populated. */\n \tstruct mana_eq *eqs;\n+\tunsigned int num_eqs;\n \tstruct dentry *mana_eqs_debugfs;\n \n \tenum TRI_STATE rss_state;\n@@ -600,9 +622,21 @@ struct mana_port_context {\n \tunsigned int max_queues;\n \tunsigned int num_queues;\n \n+\t/* Port-lifetime arrays with max_queues slots. Live RX queues write\n+\t * rxq_stats[]; teardown and rollback fold drain_stats into\n+\t * rxq_stats_ret[] under RTNL. Readers sum both.\n+\t */\n+\tstruct mana_stats_rx *rxq_stats;\n+\tstruct mana_stats_rx *rxq_stats_ret;\n+\tstruct mana_stats_tx *txq_stats;\n+\n \tunsigned int rx_queue_size;\n \tunsigned int tx_queue_size;\n \n+\t/* MTU used to size RX buffers, independent of ndev-\u003emtu during a swap.\n+\t */\n+\tint configured_mtu;\n+\n \tmana_handle_t port_handle;\n \tmana_handle_t pf_filter_handle;\n \n@@ -610,12 +644,17 @@ struct mana_port_context {\n \tstruct mutex vport_mutex;\n \tint vport_use_count;\n \n-\t/* Set by mana_set_channels() under vport_mutex to block RDMA\n-\t * from grabbing the vport during the detach/attach window.\n-\t * Checked by mana_cfg_vport() when called from the RDMA path.\n-\t */\n+\t/* Exclude RDMA during reconfiguration; protected by vport_mutex. */\n \tbool channel_changing;\n \n+\t/* Caller must close the port after releasing the unpublished set. */\n+\tbool publish_dead_end;\n+\n+\t/* Carrier lowered by failed rollback; cleared on reopen or a link\n+\t * event.\n+\t */\n+\tbool carrier_forced_off;\n+\n \t/* Net shaper handle*/\n \tstruct net_shaper_handle handle;\n \n@@ -661,6 +700,27 @@ struct mana_port_context {\n \tu32 steer_cqe_coalescing;\n };\n \n+struct mana_qset {\n+\tstruct mana_tx_qp\t**tx_qp;\n+\tstruct mana_rxq\t\t**rxqs;\n+\n+\tu32\t\t\t*indir_table;\n+\tu32\t\t\tindir_table_sz;\n+\tmana_handle_t\t\t*rxobj_table;\n+\tmana_handle_t\t\tdefault_rxobj;\n+\n+\tunsigned int\t\tnum_queues;\n+\tunsigned int\t\trx_queue_size;\n+\tunsigned int\t\ttx_queue_size;\n+\tu32\t\t\tpriv_flags;\n+\n+\tint\t\t\tmtu;\n+\tstruct bpf_prog\t\t*bpf_prog;\n+\n+\t/* Notify the core only after this set is published. */\n+\tbool\t\t\trxfh_indir_lost;\n+};\n+\n netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev);\n int mana_config_rss(struct mana_port_context *ac, enum TRI_STATE rx,\n \t\t bool update_hash, bool update_tab);\n@@ -670,6 +730,33 @@ int mana_alloc_queues(struct net_device *ndev);\n int mana_attach(struct net_device *ndev);\n int mana_detach(struct net_device *ndev, bool from_close);\n \n+struct mana_port_context *\n+mana_qset_scratch_alloc(struct mana_port_context *apc);\n+void mana_qset_scratch_free(struct mana_port_context *scratch);\n+static inline struct mana_stats_rx *mana_rxq_stats(struct mana_rxq *rxq)\n+{\n+\treturn READ_ONCE(rxq-\u003eretiring) ? \u0026rxq-\u003edrain_stats : rxq-\u003estats;\n+}\n+\n+int mana_alloc_qset(struct mana_port_context *apc,\n+\t\t struct mana_port_context *scratch,\n+\t\t unsigned int rx_queue_size, unsigned int tx_queue_size,\n+\t\t u32 priv_flags, int mtu, struct bpf_prog *bpf_prog,\n+\t\t struct mana_qset *out);\n+int mana_split_qset(struct mana_port_context *apc,\n+\t\t struct mana_port_context *scratch, unsigned int new_count,\n+\t\t struct mana_qset *out_new, struct mana_qset *out_tail);\n+void mana_discard_split(struct mana_qset *newq, struct mana_qset *tailq);\n+int mana_grow_qset(struct mana_port_context *apc,\n+\t\t struct mana_port_context *scratch, unsigned int new_count,\n+\t\t struct mana_qset *out_new, struct mana_qset *out_fresh);\n+void mana_discard_grow(struct mana_qset *newq);\n+int mana_publish_qset(struct mana_port_context *apc, struct mana_qset *newq,\n+\t\t struct mana_qset *out_old);\n+void mana_publish_close_if_needed(struct mana_port_context *apc);\n+void mana_free_qset(struct mana_port_context *scratch, struct mana_qset *qset);\n+void mana_qset_debugfs_publish(struct mana_port_context *apc);\n+\n void mana_dim_change(struct mana_cq *cq, bool enable);\n \n int mana_probe(struct gdma_dev *gd, bool resuming);\n@@ -685,6 +772,8 @@ u32 mana_run_xdp(struct net_device *ndev, struct mana_rxq *rxq,\n \t\t struct xdp_buff *xdp, void *buf_va, uint pkt_len);\n struct bpf_prog *mana_xdp_get(struct mana_port_context *apc);\n void mana_chn_setxdp(struct mana_port_context *apc, struct bpf_prog *prog);\n+struct bpf_prog *mana_chn_xdp_peek(struct mana_port_context *apc);\n+void mana_chn_xdp_release(struct bpf_prog *prog, unsigned int num_queues);\n int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf);\n int mana_query_gf_stats(struct mana_context *ac);\n int mana_query_link_cfg(struct mana_port_context *apc);\n"
} |
| 2/1 | 2026/09/09 22:47 | agent | patch-evaluator |
0mModel:core Args: null Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"Reasoning": "The patch modifies the Microsoft Azure Network Adapter (MANA) Ethernet driver (drivers/net/ethernet/microsoft/mana/ and include/net/mana/). This driver targets vendor-specific hardware/SmartNICs on Microsoft Azure / Hyper-V environments and is not emulated in standard QEMU or virtualized fuzzing environments. As such, the modified code is structurally unreachable during fuzzing.",
"WorthFuzzing": false
}Instruction: You are an expert Linux kernel maintainer and security engineer.
Your job is to review a provided patch series and evaluate whether it warrants fuzzing with syzkaller.
IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in
your workspace. Do NOT rely on internal assumptions. You must actively use your code access
tools to inspect the actual source code, callers, and surrounding context.
================================================================================
1. CORE TRIAGE PHILOSOPHY
================================================================================
The goal of patch fuzzing is to discover crashes, regressions, exposed latent bugs,
and newly triggered assertions introduced by the patch series.
- REACHABILITY IS THE PRIMARY GATE:
Fuzzing can only discover bugs in code that can actually execute in standard virtualized
environments (GCE or QEMU, utilizing software-emulated devices like USB gadgets, netdev, tun/tap).
If the modified code is structurally unreachable (see Section 2), it MUST NOT be fuzzed,
regardless of whether it adds assertions or complex logic.
- DO NOT BLINDLY TRUST "NO FUNCTIONAL CHANGE" (NFCI) OR "REFACTORING" CLAIMS:
Patch authors routinely label changes as "cleanups", "refactorings", or state
"No functional change intended". Do NOT take these claims at face value.
Code refactorings that rearrange logic, introduce helper functions, or alter state management
in core subsystems frequently introduce subtle semantic shifts or uncover latent kernel bugs.
If reachable executable code is modified or refactored, it MUST be fuzzed.
- NEW OR MODIFIED ASSERTIONS IN REACHABLE CODE MUST BE FUZZED:
When a patch introduces or modifies runtime checks or assertions (e.g., WARN_ON*, VM_WARN_ON*,
BUG_ON*, lockdep_assert*) in reachable code paths, it enforces new or stricter invariants.
Even if the author believes the invariant always holds, fuzzing is essential to verify whether
an unusual sequence of operations can violate it.
================================================================================
2. WHEN TO RETURN WorthFuzzing=false (NEGATIVE CRITERIA)
================================================================================
Return WorthFuzzing=false ONLY IF all modified code falls strictly into one or more of these categories:
- Non-kernel and non-executable changes:
* Modifications to Documentation/, comments, or spelling fixes.
* User-space directories, self-tests, samples, or scripts (e.g., tools/, samples/, scripts/, usr/)
that do not affect the compiled kernel image (vmlinux) or kernel modules.
* Purely decorative logging (e.g., message strings in pr_err, printk, dev_info) or tracepoints
that do not alter control flow or data structures.
* Build system or Kconfig changes that do not alter compiled C logic.
- Structurally unreachable hardware:
* Vendor-specific PCIe switches, SmartNICs, or GPU drivers (e.g., mlxsw, pds_core, qed,
ionic, amdgpu) requiring physical ASIC/PCIe cards not emulated in standard QEMU.
- Unreachable execution paths:
* Driver teardown callbacks (.remove, .shutdown, pci_unregister_driver) executed only during
physical PCI hot-unplug or manual sysfs driver unbinding.
* Code paths exclusive to architectures other than the target architecture.
================================================================================
3. WHEN TO RETURN WorthFuzzing=true (POSITIVE CRITERIA)
================================================================================
Return WorthFuzzing=true whenever the patch touches reachable executable code, including:
- Core Subsystems:
* Any logic modifications in memory management (mm/), synchronization/locking (kernel/locking/),
BPF, scheduler, core networking, VFS, or syscall handling.
- Refactorings and Code Cleanups:
* Any restructuring of reachable data structures, helper abstractions, or algorithm flows.
- Runtime Assertions and Defensive Checks:
* Any introduction or alteration of assertions (WARN_ON*, VM_WARN_ON*, BUG_ON*, etc.) in reachable paths.
- Reachable Drivers and Protocols:
* Drivers accessible via virtual buses (virtio, USB gadget, loopback, netlink, binder, sockets, etc.).
================================================================================
4. EXTRACTING FocusSymbols (PREVENTING DILUTION)
================================================================================
When WorthFuzzing=true, you must extract specific kernel functions into FocusSymbols to guide the fuzzer:
- AVOID UBIQUITOUS LIFECYCLE HOT-PATHS:
Do NOT list generic, ubiquitous functions called by almost every program in the corpus
(including, but not limited to: general memory allocators and deallocators, page fault
and trap handlers, or core synchronization primitives; this is not an exhaustive list).
Listing ubiquitous functions causes the fuzzer to classify thousands of unrelated tests as "focused",
which severely dilutes fuzzing effort away from the actual changes.
- TARGET SPECIFIC FEATURE LOGIC AND ENTRYPOINTS:
List functions that specifically implement the logic being added or altered, or direct API entrypoints
for the subsystem feature under review.
- HANDLING STATIC INLINE FUNCTIONS IN HEADERS (.h):
Compiler-inlined static functions (such as static inlines in mm/*.h or include/linux/*.h) lack
distinct symbol addresses in vmlinux and cannot be targeted directly by symbol coverage filters.
If the changes are primarily in static inline helpers, identify non-static, feature-specific caller
functions in .c files that exercise them (avoiding ubiquitous lifecycle wrappers).
================================================================================
5. IDENTIFYING EnableConfigs
================================================================================
Identify any specific CONFIG_ options required to properly compile and reach the modified code:
- Inspect Kconfig files and #ifdef guards; do not make assumptions.
- Check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed.
- Strip any 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4").
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Target architecture: amd64
For your convenience, here is the diff of the changes:
commit fccf2b5a44a87f73e0b901245f87ab4538bb40df
Author: syz-cluster <triage@syzkaller.com>
Date: Wed Sep 9 22:47:24 2026 +0000
syz-cluster: applied patch under review
diff --git a/drivers/net/ethernet/microsoft/mana/mana_bpf.c b/drivers/net/ethernet/microsoft/mana/mana_bpf.c
index 70bb8c9dc3413..3acec1bb02a34 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_bpf.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_bpf.c
@@ -59,6 +59,11 @@ int mana_xdp_xmit(struct net_device *ndev, int n, struct xdp_frame **frames,
if (unlikely(!apc->port_is_up))
return 0;
+ /* Pair with the smp_wmb() in mana_publish_qset() before reading queue
+ * state.
+ */
+ smp_rmb();
+
q_idx = smp_processor_id() % ndev->real_num_tx_queues;
for (i = 0; i < n; i++) {
@@ -68,7 +73,7 @@ int mana_xdp_xmit(struct net_device *ndev, int n, struct xdp_frame **frames,
count++;
}
- tx_stats = &apc->tx_qp[q_idx]->txq.stats;
+ tx_stats = apc->tx_qp[q_idx]->txq.stats;
u64_stats_update_begin(&tx_stats->syncp);
tx_stats->xdp_xmit += count;
@@ -95,7 +100,7 @@ u32 mana_run_xdp(struct net_device *ndev, struct mana_rxq *rxq,
act = bpf_prog_run_xdp(prog, xdp);
- rx_stats = &rxq->stats;
+ rx_stats = mana_rxq_stats(rxq);
switch (act) {
case XDP_PASS:
@@ -172,6 +177,8 @@ static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,
struct netlink_ext_ack *extack)
{
struct mana_port_context *apc = netdev_priv(ndev);
+ struct mana_port_context *scratch;
+ struct mana_qset newq, oldq;
struct bpf_prog *old_prog;
struct gdma_context *gc;
int err;
@@ -191,46 +198,44 @@ static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,
return -EOPNOTSUPP;
}
- /* One refcnt of the prog is hold by the caller already, so
- * don't increase refcnt for this one.
- */
- apc->bpf_prog = prog;
-
if (apc->port_is_up) {
- /* Re-create rxq's after xdp prog was loaded or unloaded.
- * Ex: re create rxq's to switch from full pages to smaller
- * size page fragments when xdp prog is unloaded and
- * vice-versa.
- */
-
- /* Pre-allocate buffers to prevent failure in mana_attach */
- err = mana_pre_alloc_rxbufs(apc, ndev->mtu, apc->num_queues);
- if (err) {
+ scratch = mana_qset_scratch_alloc(apc);
+ if (!scratch) {
NL_SET_ERR_MSG_MOD(extack,
- "XDP: Insufficient memory for tx/rx re-config");
- return err;
+ "XDP: Insufficient memory for re-config");
+ return -ENOMEM;
}
- err = mana_detach(ndev, false);
+ err = mana_alloc_qset(apc, scratch,
+ apc->rx_queue_size, apc->tx_queue_size,
+ apc->priv_flags, apc->configured_mtu,
+ prog, &newq);
if (err) {
- netdev_err(ndev,
- "mana_detach failed at xdp set: %d\n", err);
NL_SET_ERR_MSG_MOD(extack,
- "XDP: Re-config failed at detach");
- goto err_dealloc_rxbuffs;
+ "XDP: Re-config failed at alloc");
+ mana_qset_scratch_free(scratch);
+ return err;
}
- err = mana_attach(ndev);
+ err = mana_publish_qset(apc, &newq, &oldq);
if (err) {
- netdev_err(ndev,
- "mana_attach failed at xdp set: %d\n", err);
NL_SET_ERR_MSG_MOD(extack,
- "XDP: Re-config failed at attach");
- goto err_dealloc_rxbuffs;
+ "XDP: Re-config failed at publish");
+ mana_free_qset(scratch, &newq);
+ /* Free the queues before closing their shared EQ pool.
+ */
+ mana_publish_close_if_needed(apc);
+ mana_qset_scratch_free(scratch);
+ return err;
}
- mana_chn_setxdp(apc, prog);
- mana_pre_dealloc_rxbufs(apc);
+ mana_free_qset(scratch, &oldq);
+ mana_qset_scratch_free(scratch);
+ } else {
+ /* Use the caller's program reference; mana_open() installs it
+ * on queues.
+ */
+ apc->bpf_prog = prog;
}
if (old_prog)
@@ -243,11 +248,6 @@ static int mana_xdp_set(struct net_device *ndev, struct bpf_prog *prog,
ndev->max_mtu = gc->adapter_mtu - ETH_HLEN;
return 0;
-
-err_dealloc_rxbuffs:
- apc->bpf_prog = old_prog;
- mana_pre_dealloc_rxbufs(apc);
- return err;
}
int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf)
@@ -262,3 +262,27 @@ int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf)
return -EOPNOTSUPP;
}
}
+
+struct bpf_prog *mana_chn_xdp_peek(struct mana_port_context *apc)
+{
+ ASSERT_RTNL();
+
+ if (!apc->rxqs || !apc->rxqs[0])
+ return NULL;
+
+ return rtnl_dereference(apc->rxqs[0]->bpf_prog);
+}
+
+/* Keep the per-queue program pointers until RX polling stops. */
+void mana_chn_xdp_release(struct bpf_prog *prog, unsigned int num_queues)
+{
+ unsigned int i;
+
+ ASSERT_RTNL();
+
+ if (!prog)
+ return;
+
+ for (i = 0; i < num_queues; i++)
+ bpf_prog_put(prog);
+}
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index 7a1ac853e3abc..29f54f91cb009 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -90,6 +90,17 @@ static int mana_open(struct net_device *ndev)
smp_wmb();
netif_tx_wake_all_queues(ndev);
+
+ /* Undo a forced carrier-off unless a disconnect is pending behind RTNL.
+ */
+ if (apc->carrier_forced_off) {
+ u32 ev = READ_ONCE(apc->ac->link_event);
+
+ apc->carrier_forced_off = false;
+ if (ev != HWC_DATA_HW_LINK_DISCONNECT)
+ netif_carrier_on(ndev);
+ }
+
netdev_dbg(ndev, "%s successful\n", __func__);
return 0;
}
@@ -106,6 +117,7 @@ static int mana_close(struct net_device *ndev)
static void mana_link_state_handle(struct work_struct *w)
{
+ struct mana_port_context *apc;
struct mana_context *ac;
struct net_device *ndev;
u32 link_event;
@@ -131,6 +143,9 @@ static void mana_link_state_handle(struct work_struct *w)
if (!ndev)
continue;
+ apc = netdev_priv(ndev);
+ apc->carrier_forced_off = false;
+
if (link_up) {
netif_carrier_on(ndev);
@@ -312,8 +327,8 @@ static void mana_per_port_queue_reset_work_handler(struct work_struct *work)
rtnl_lock();
- /* Block RDMA from grabbing the vport during the detach/attach
- * window, same as mana_set_channels().
+ /* Exclude RDMA across detach/attach; RTNL serializes channel_changing
+ * writers.
*/
mutex_lock(&apc->vport_mutex);
apc->channel_changing = true;
@@ -366,13 +381,22 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev)
if (unlikely(!apc->port_is_up))
goto tx_drop;
+ /* Pair with mana_publish_qset()'s pre-gate smp_wmb(): observe queue
+ * fields after reading port_is_up.
+ */
+ smp_rmb();
+
+ /* Retiring RXQs may use indices beyond the live queue count. */
+ if (unlikely(txq_idx >= apc->num_queues))
+ goto tx_drop_count;
+
if (skb_cow_head(skb, MANA_HEADROOM))
goto tx_drop_count;
txq = &apc->tx_qp[txq_idx]->txq;
gdma_sq = txq->gdma_sq;
cq = &apc->tx_qp[txq_idx]->tx_cq;
- tx_stats = &txq->stats;
+ tx_stats = txq->stats;
BUILD_BUG_ON(MAX_TX_WQE_SGL_ENTRIES != MANA_MAX_TX_WQE_SGL_ENTRIES);
if (MAX_SKB_FRAGS + 2 > MAX_TX_WQE_SGL_ENTRIES &&
@@ -551,7 +575,7 @@ netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev)
/* Populated the packet and bytes counters based on post GSO packet
* calculations
*/
- tx_stats = &txq->stats;
+ tx_stats = txq->stats;
u64_stats_update_begin(&tx_stats->syncp);
tx_stats->packets += num_gso_seg;
tx_stats->bytes += len + ((num_gso_seg - 1) * gso_hs);
@@ -597,15 +621,15 @@ static void mana_get_stats64(struct net_device *ndev,
struct rtnl_link_stats64 *st)
{
struct mana_port_context *apc = netdev_priv(ndev);
- unsigned int num_queues = apc->num_queues;
struct mana_stats_rx *rx_stats;
struct mana_stats_tx *tx_stats;
+ unsigned int num_queues;
unsigned int start;
u64 packets, bytes;
int q;
- if (!apc->port_is_up)
- return;
+ /* Report even while down; dev_get_stats() zeroes its output. */
+ num_queues = apc->max_queues;
netdev_stats_to_stats64(st, &ndev->stats);
@@ -615,7 +639,18 @@ static void mana_get_stats64(struct net_device *ndev,
st->rx_missed_errors = apc->ac->hc_stats.hc_rx_discards_no_wqe;
for (q = 0; q < num_queues; q++) {
- rx_stats = &apc->rxqs[q]->stats;
+ rx_stats = &apc->rxq_stats[q];
+
+ do {
+ start = u64_stats_fetch_begin(&rx_stats->syncp);
+ packets = rx_stats->packets;
+ bytes = rx_stats->bytes;
+ } while (u64_stats_fetch_retry(&rx_stats->syncp, start));
+
+ st->rx_packets += packets;
+ st->rx_bytes += bytes;
+
+ rx_stats = &apc->rxq_stats_ret[q];
do {
start = u64_stats_fetch_begin(&rx_stats->syncp);
@@ -628,7 +663,7 @@ static void mana_get_stats64(struct net_device *ndev,
}
for (q = 0; q < num_queues; q++) {
- tx_stats = &apc->tx_qp[q]->txq.stats;
+ tx_stats = &apc->txq_stats[q];
do {
start = u64_stats_fetch_begin(&tx_stats->syncp);
@@ -878,32 +913,37 @@ int mana_pre_alloc_rxbufs(struct mana_port_context *mpc, int new_mtu, int num_qu
static int mana_change_mtu(struct net_device *ndev, int new_mtu)
{
struct mana_port_context *mpc = netdev_priv(ndev);
- unsigned int old_mtu = ndev->mtu;
+ struct mana_port_context *scratch;
+ struct mana_qset newq, oldq;
int err;
- /* Pre-allocate buffers to prevent failure in mana_attach later */
- err = mana_pre_alloc_rxbufs(mpc, new_mtu, mpc->num_queues);
- if (err) {
- netdev_err(ndev, "Insufficient memory for new MTU\n");
- return err;
+ if (!mpc->port_is_up) {
+ mpc->configured_mtu = new_mtu;
+ WRITE_ONCE(ndev->mtu, new_mtu);
+ return 0;
}
- err = mana_detach(ndev, false);
- if (err) {
- netdev_err(ndev, "mana_detach failed: %d\n", err);
- goto out;
- }
+ scratch = mana_qset_scratch_alloc(mpc);
+ if (!scratch)
+ return -ENOMEM;
- WRITE_ONCE(ndev->mtu, new_mtu);
+ err = mana_alloc_qset(mpc, scratch, mpc->rx_queue_size,
+ mpc->tx_queue_size, mpc->priv_flags, new_mtu,
+ mpc->bpf_prog, &newq);
+ if (err)
+ goto free_scratch;
- err = mana_attach(ndev);
+ err = mana_publish_qset(mpc, &newq, &oldq);
if (err) {
- netdev_err(ndev, "mana_attach failed: %d\n", err);
- WRITE_ONCE(ndev->mtu, old_mtu);
+ mana_free_qset(scratch, &newq);
+ goto free_scratch;
}
-out:
- mana_pre_dealloc_rxbufs(mpc);
+ mana_free_qset(scratch, &oldq);
+
+free_scratch:
+ mana_publish_close_if_needed(mpc);
+ mana_qset_scratch_free(scratch);
return err;
}
@@ -1036,6 +1076,105 @@ static void mana_cleanup_port_context(struct mana_port_context *apc)
apc->rxqs = NULL;
}
+/* Port lifetime preserves counters across queue replacement. */
+static int mana_alloc_queue_stats(struct mana_port_context *apc)
+{
+ unsigned int i;
+
+ apc->rxq_stats = kcalloc(apc->max_queues, sizeof(*apc->rxq_stats),
+ GFP_KERNEL);
+ if (!apc->rxq_stats)
+ return -ENOMEM;
+
+ apc->rxq_stats_ret = kcalloc(apc->max_queues,
+ sizeof(*apc->rxq_stats_ret), GFP_KERNEL);
+ if (!apc->rxq_stats_ret)
+ goto free_rxq_stats;
+
+ apc->txq_stats = kcalloc(apc->max_queues, sizeof(*apc->txq_stats),
+ GFP_KERNEL);
+ if (!apc->txq_stats)
+ goto free_rxq_stats_ret;
+
+ for (i = 0; i < apc->max_queues; i++) {
+ u64_stats_init(&apc->rxq_stats[i].syncp);
+ u64_stats_init(&apc->rxq_stats_ret[i].syncp);
+ u64_stats_init(&apc->txq_stats[i].syncp);
+ }
+
+ return 0;
+
+free_rxq_stats_ret:
+ kfree(apc->rxq_stats_ret);
+ apc->rxq_stats_ret = NULL;
+free_rxq_stats:
+ kfree(apc->rxq_stats);
+ apc->rxq_stats = NULL;
+ return -ENOMEM;
+}
+
+static void mana_free_queue_stats(struct mana_port_context *apc)
+{
+ kfree(apc->rxq_stats);
+ apc->rxq_stats = NULL;
+ kfree(apc->rxq_stats_ret);
+ apc->rxq_stats_ret = NULL;
+ kfree(apc->txq_stats);
+ apc->txq_stats = NULL;
+}
+
+/* Fold under RTNL after drain_stats writers quiesce. Clear drain_stats to
+ * prevent double counting on rollback.
+ */
+static void mana_fold_rxq_stats(struct mana_port_context *apc,
+ struct mana_rxq *rxq)
+{
+ struct mana_stats_rx *src = &rxq->drain_stats;
+ struct mana_stats_rx *dst;
+ unsigned int i;
+
+ ASSERT_RTNL();
+
+ if (!apc->rxq_stats_ret || rxq->rxq_idx >= apc->max_queues)
+ return;
+
+ dst = &apc->rxq_stats_ret[rxq->rxq_idx];
+
+ u64_stats_update_begin(&dst->syncp);
+ dst->packets += src->packets;
+ dst->bytes += src->bytes;
+ dst->xdp_drop += src->xdp_drop;
+ dst->xdp_tx += src->xdp_tx;
+ dst->xdp_redirect += src->xdp_redirect;
+ dst->pkt_len0_err += src->pkt_len0_err;
+ for (i = 0; i < ARRAY_SIZE(dst->coalesced_cqe); i++)
+ dst->coalesced_cqe[i] += src->coalesced_cqe[i];
+ u64_stats_update_end(&dst->syncp);
+
+ src->packets = 0;
+ src->bytes = 0;
+ src->xdp_drop = 0;
+ src->xdp_tx = 0;
+ src->xdp_redirect = 0;
+ src->pkt_len0_err = 0;
+ for (i = 0; i < ARRAY_SIZE(src->coalesced_cqe); i++)
+ src->coalesced_cqe[i] = 0;
+}
+
+static void mana_fold_qset_rx_stats(struct mana_port_context *apc,
+ struct mana_qset *qset)
+{
+ unsigned int q;
+
+ if (!qset->rxqs)
+ return;
+
+ for (q = 0; q < qset->num_queues; q++) {
+ if (qset->rxqs[q])
+ mana_fold_rxq_stats(apc, qset->rxqs[q]);
+ }
+}
+
static void mana_cleanup_indir_table(struct mana_port_context *apc)
{
apc->indir_table_sz = 0;
@@ -1045,6 +1184,7 @@ static void mana_cleanup_indir_table(struct mana_port_context *apc)
static int mana_init_port_context(struct mana_port_context *apc)
{
+ kfree(apc->rxqs);
apc->rxqs = kzalloc_objs(struct mana_rxq *, apc->num_queues);
return !apc->rxqs ? -ENOMEM : 0;
@@ -1733,7 +1873,7 @@ void mana_destroy_eq(struct mana_port_context *apc)
debugfs_remove_recursive(apc->mana_eqs_debugfs);
apc->mana_eqs_debugfs = NULL;
- for (i = 0; i < apc->num_queues; i++) {
+ for (i = 0; i < apc->num_eqs; i++) {
eq = apc->eqs[i].eq;
if (!eq)
continue;
@@ -1741,24 +1881,28 @@ void mana_destroy_eq(struct mana_port_context *apc)
msi = eq->eq.msix_index;
mana_gd_destroy_queue(gc, eq);
mana_gd_put_gic(gc, !gc->msi_sharing, msi);
+ apc->eqs[i].eq = NULL;
+ apc->eqs[i].mana_eq_debugfs = NULL;
}
kfree(apc->eqs);
apc->eqs = NULL;
+ apc->num_eqs = 0;
}
EXPORT_SYMBOL_NS(mana_destroy_eq, "NET_MANA");
static void mana_create_eq_debugfs(struct mana_port_context *apc, int i)
{
- struct mana_eq eq = apc->eqs[i];
+ struct mana_eq *eq = &apc->eqs[i];
char eqnum[32];
sprintf(eqnum, "eq%d", i);
- eq.mana_eq_debugfs = debugfs_create_dir(eqnum, apc->mana_eqs_debugfs);
- debugfs_create_u32("head", 0400, eq.mana_eq_debugfs, &eq.eq->head);
- debugfs_create_u32("tail", 0400, eq.mana_eq_debugfs, &eq.eq->tail);
- debugfs_create_u32("irq", 0400, eq.mana_eq_debugfs, &eq.eq->eq.irq);
- debugfs_create_file("eq_dump", 0400, eq.mana_eq_debugfs, eq.eq, &mana_dbg_q_fops);
+ eq->mana_eq_debugfs = debugfs_create_dir(eqnum, apc->mana_eqs_debugfs);
+ debugfs_create_u32("head", 0400, eq->mana_eq_debugfs, &eq->eq->head);
+ debugfs_create_u32("tail", 0400, eq->mana_eq_debugfs, &eq->eq->tail);
+ debugfs_create_u32("irq", 0400, eq->mana_eq_debugfs, &eq->eq->eq.irq);
+ debugfs_create_file("eq_dump", 0400, eq->mana_eq_debugfs, eq->eq,
+ &mana_dbg_q_fops);
}
int mana_create_eq(struct mana_port_context *apc)
@@ -1773,9 +1917,11 @@ int mana_create_eq(struct mana_port_context *apc)
if (WARN_ON(apc->eqs))
return -EEXIST;
- apc->eqs = kzalloc_objs(struct mana_eq, apc->num_queues);
+ /* Keep EQ array addresses stable while CQs reference them. */
+ apc->eqs = kzalloc_objs(struct mana_eq, apc->max_queues);
if (!apc->eqs)
return -ENOMEM;
+ apc->num_eqs = 0;
spec.type = GDMA_EQ;
spec.monitor_avl_buf = false;
@@ -1805,6 +1951,7 @@ int mana_create_eq(struct mana_port_context *apc)
}
apc->eqs[i].eq->eq.irq = gic->irq;
mana_create_eq_debugfs(apc, i);
+ apc->num_eqs = i + 1;
}
return 0;
@@ -1814,6 +1961,87 @@ int mana_create_eq(struct mana_port_context *apc)
}
EXPORT_SYMBOL_NS(mana_create_eq, "NET_MANA");
+/* Grow the shared EQ pool without replacing live entries. */
+static int mana_grow_eqs(struct mana_port_context *apc, unsigned int need)
+{
+ struct gdma_dev *gd = apc->ac->gdma_dev;
+ struct gdma_context *gc = gd->gdma_context;
+ struct gdma_queue_spec spec = {};
+ struct gdma_irq_context *gic;
+ unsigned int i;
+ int err;
+ int msi;
+
+ if (WARN_ON(!apc->eqs))
+ return -EINVAL;
+
+ if (need > apc->max_queues)
+ return -EINVAL;
+
+ if (need <= apc->num_eqs)
+ return 0;
+
+ spec.type = GDMA_EQ;
+ spec.monitor_avl_buf = false;
+ spec.queue_size = EQ_SIZE;
+ spec.eq.callback = NULL;
+ spec.eq.context = apc->eqs;
+ spec.eq.log2_throttle_limit = LOG2_EQ_THROTTLE;
+
+ for (i = apc->num_eqs; i < need; i++) {
+ msi = (i + 1) % gc->num_msix_usable;
+
+ gic = mana_gd_get_gic(gc, !gc->msi_sharing, &msi);
+ if (IS_ERR(gic)) {
+ err = PTR_ERR(gic);
+ goto out;
+ }
+ spec.eq.msix_index = msi;
+
+ err = mana_gd_create_mana_eq(gd, &spec, &apc->eqs[i].eq);
+ if (err) {
+ dev_err(gc->dev, "Failed to grow EQ %u : %d\n", i, err);
+ mana_gd_put_gic(gc, !gc->msi_sharing, msi);
+ goto out;
+ }
+ apc->eqs[i].eq->eq.irq = gic->irq;
+ mana_create_eq_debugfs(apc, i);
+ apc->num_eqs = i + 1;
+ }
+
+ return 0;
+out:
+ return err;
+}
+
+/* All CQs referencing EQs at or above @keep must be destroyed first. */
+static void mana_shrink_eqs(struct mana_port_context *apc, unsigned int keep)
+{
+ struct gdma_context *gc = apc->ac->gdma_dev->gdma_context;
+ struct gdma_queue *eq;
+ unsigned int msi;
+ unsigned int i;
+
+ if (!apc->eqs || keep >= apc->num_eqs)
+ return;
+
+ for (i = keep; i < apc->num_eqs; i++) {
+ eq = apc->eqs[i].eq;
+ if (!eq)
+ continue;
+
+ debugfs_remove_recursive(apc->eqs[i].mana_eq_debugfs);
+ apc->eqs[i].mana_eq_debugfs = NULL;
+
+ msi = eq->eq.msix_index;
+ mana_gd_destroy_queue(gc, eq);
+ mana_gd_put_gic(gc, !gc->msi_sharing, msi);
+ apc->eqs[i].eq = NULL;
+ }
+
+ apc->num_eqs = keep;
+}
+
static int mana_fence_rq(struct mana_port_context *apc, struct mana_rxq *rxq)
{
struct mana_fence_rq_resp resp = {};
@@ -2018,7 +2246,9 @@ static void mana_poll_tx_cq(struct mana_cq *cq)
/* Ensure checking txq_stopped before apc->port_is_up. */
smp_rmb();
- if (txq_stopped && apc->port_is_up && avail_space >= MAX_TX_WQE_SIZE) {
+ /* Order the stopped-state read before the retiring read. */
+ if (txq_stopped && !READ_ONCE(txq->retiring) && apc->port_is_up &&
+ avail_space >= MAX_TX_WQE_SIZE) {
netif_tx_wake_queue(net_txq);
apc->eth_stats.wake_queue++;
}
@@ -2081,7 +2311,7 @@ static void mana_rx_skb(void *buf_va, bool from_pool,
struct mana_rxcomp_oob *cqe, struct mana_rxq *rxq,
u32 pkt_len, u32 pkt_hash)
{
- struct mana_stats_rx *rx_stats = &rxq->stats;
+ struct mana_stats_rx *rx_stats = mana_rxq_stats(rxq);
struct net_device *ndev = rxq->ndev;
u16 rxq_idx = rxq->rxq_idx;
struct napi_struct *napi;
@@ -2314,6 +2544,7 @@ static void mana_process_rx_cqe(struct mana_rxq *rxq, struct mana_cq *cq,
struct net_device *ndev = rxq->ndev;
struct mana_recv_buf_oob *rxbuf_oob;
struct mana_port_context *apc;
+ struct mana_stats_rx *rx_stats;
struct device *dev = gc->dev;
bool coalesced_8 = false;
bool coalesced = false;
@@ -2395,13 +2626,15 @@ static void mana_process_rx_cqe(struct mana_rxq *rxq, struct mana_cq *cq,
* Coalesced CQEs have at least 2 packets, so index is pkt_i - 2.
*/
if (pkt_i > 1) {
- u64_stats_update_begin(&rxq->stats.syncp);
- rxq->stats.coalesced_cqe[pkt_i - 2]++;
- u64_stats_update_end(&rxq->stats.syncp);
+ rx_stats = mana_rxq_stats(rxq);
+ u64_stats_update_begin(&rx_stats->syncp);
+ rx_stats->coalesced_cqe[pkt_i - 2]++;
+ u64_stats_update_end(&rx_stats->syncp);
} else if (!pkt_i && !pktlen) {
- u64_stats_update_begin(&rxq->stats.syncp);
- rxq->stats.pkt_len0_err++;
- u64_stats_update_end(&rxq->stats.syncp);
+ rx_stats = mana_rxq_stats(rxq);
+ u64_stats_update_begin(&rx_stats->syncp);
+ rx_stats->pkt_len0_err++;
+ u64_stats_update_end(&rx_stats->syncp);
netdev_err_once(ndev,
"RX pkt len=0, rq=%u, cq=%u, rxobj=0x%llx\n",
rxq->gdma_id, cq->gdma_id, rxq->rxobj);
@@ -2533,8 +2766,12 @@ static void mana_update_rx_dim(struct mana_cq *cq)
if (!smp_load_acquire(&apc->rx_dim_enabled))
return;
- dim_update_sample(READ_ONCE(cq->dim_event_ctr), rxq->stats.packets,
- rxq->stats.bytes, &dim_sample);
+ /* Skip retiring RXQs; DIM reads shared per-index counters. */
+ if (READ_ONCE(rxq->retiring))
+ return;
+
+ dim_update_sample(READ_ONCE(cq->dim_event_ctr), rxq->stats->packets,
+ rxq->stats->bytes, &dim_sample);
net_dim(&cq->dim, &dim_sample);
}
@@ -2641,7 +2878,9 @@ static void mana_deinit_txq(struct mana_port_context *apc, struct mana_txq *txq)
mana_gd_destroy_queue(gd->gdma_context, txq->gdma_sq);
}
-static void mana_destroy_txq(struct mana_port_context *apc)
+/* Keep the array and queues below @first; clear freed slots. */
+static void mana_destroy_txq_from(struct mana_port_context *apc,
+ unsigned int first)
{
struct napi_struct *napi;
int i;
@@ -2649,7 +2888,7 @@ static void mana_destroy_txq(struct mana_port_context *apc)
if (!apc->tx_qp)
return;
- for (i = 0; i < apc->num_queues; i++) {
+ for (i = first; i < apc->num_queues; i++) {
if (!apc->tx_qp[i])
continue;
@@ -2673,7 +2912,16 @@ static void mana_destroy_txq(struct mana_port_context *apc)
mana_deinit_txq(apc, &apc->tx_qp[i]->txq);
kvfree(apc->tx_qp[i]);
+ apc->tx_qp[i] = NULL;
}
+}
+
+static void mana_destroy_txq(struct mana_port_context *apc)
+{
+ if (!apc->tx_qp)
+ return;
+
+ mana_destroy_txq_from(apc, 0);
kfree(apc->tx_qp);
apc->tx_qp = NULL;
@@ -2704,8 +2952,11 @@ static void mana_create_txq_debugfs(struct mana_port_context *apc, int idx)
tx_qp->tx_cq.gdma_cq, &mana_dbg_q_fops);
}
+/* With @first nonzero, use the existing array and unwind only new queues on
+ * failure.
+ */
static int mana_create_txq(struct mana_port_context *apc,
- struct net_device *net)
+ struct net_device *net, unsigned int first)
{
struct mana_context *ac = apc->ac;
struct gdma_dev *gd = ac->gdma_dev;
@@ -2720,9 +2971,14 @@ static int mana_create_txq(struct mana_port_context *apc,
int err;
int i;
- apc->tx_qp = kzalloc_objs(struct mana_tx_qp *, apc->num_queues);
- if (!apc->tx_qp)
- return -ENOMEM;
+ if (first) {
+ if (WARN_ON(!apc->tx_qp))
+ return -EINVAL;
+ } else {
+ apc->tx_qp = kzalloc_objs(struct mana_tx_qp *, apc->num_queues);
+ if (!apc->tx_qp)
+ return -ENOMEM;
+ }
/* The minimum size of the WQE is 32 bytes, hence
* apc->tx_queue_size represents the maximum number of WQEs
@@ -2739,7 +2995,7 @@ static int mana_create_txq(struct mana_port_context *apc,
gc = gd->gdma_context;
- for (i = 0; i < apc->num_queues; i++) {
+ for (i = first; i < apc->num_queues; i++) {
apc->tx_qp[i] = kvzalloc_obj(*apc->tx_qp[i]);
if (!apc->tx_qp[i]) {
err = -ENOMEM;
@@ -2751,9 +3007,10 @@ static int mana_create_txq(struct mana_port_context *apc,
/* Create SQ */
txq = &apc->tx_qp[i]->txq;
- u64_stats_init(&txq->stats.syncp);
+ txq->stats = &apc->txq_stats[i];
txq->ndev = net;
txq->net_txq = netdev_get_tx_queue(net, i);
+ txq->reset_gen = READ_ONCE(apc->ac->reset_gen);
txq->vp_offset = apc->tx_vp_offset;
txq->napi_initialized = false;
skb_queue_head_init(&txq->pending_skbs);
@@ -2846,7 +3103,10 @@ static int mana_create_txq(struct mana_port_context *apc,
out:
netdev_err(net, "Failed to create %d TX queues, %d\n",
apc->num_queues, err);
- mana_destroy_txq(apc);
+ if (first)
+ mana_destroy_txq_from(apc, first);
+ else
+ mana_destroy_txq(apc);
return err;
}
@@ -2877,6 +3137,9 @@ static void mana_destroy_rxq(struct mana_port_context *apc,
netif_napi_del_locked(napi);
}
+ /* NAPI is quiesced, so drain_stats has no remaining writer. */
+ mana_fold_rxq_stats(apc, rxq);
+
if (xdp_rxq_info_is_reg(&rxq->xdp_rxq))
xdp_rxq_info_unreg(&rxq->xdp_rxq);
@@ -3009,11 +3272,11 @@ static int mana_push_wqe(struct mana_rxq *rxq)
static int mana_create_page_pool(struct mana_rxq *rxq, struct gdma_context *gc)
{
- struct mana_port_context *mpc = netdev_priv(rxq->ndev);
struct page_pool_params pprm = {};
int ret;
- pprm.pool_size = mpc->rx_queue_size / rxq->frag_count + 1;
+ /* Size the pool for this RX queue, not the live configuration. */
+ pprm.pool_size = rxq->num_rx_buf / rxq->frag_count + 1;
pprm.nid = gc->numa_node;
pprm.napi = &rxq->rx_cq.napi;
pprm.netdev = rxq->ndev;
@@ -3062,11 +3325,14 @@ static struct mana_rxq *mana_create_rxq(struct mana_port_context *apc,
return ERR_PTR(-ENOMEM);
rxq->ndev = ndev;
+ rxq->stats = &apc->rxq_stats[rxq_idx];
+ u64_stats_init(&rxq->drain_stats.syncp);
rxq->num_rx_buf = apc->rx_queue_size;
rxq->rxq_idx = rxq_idx;
rxq->rxobj = INVALID_MANA_HANDLE;
- mana_get_rxbuf_cfg(apc, ndev->mtu, &rxq->datasize, &rxq->alloc_size,
+ mana_get_rxbuf_cfg(apc, apc->configured_mtu, &rxq->datasize,
+ &rxq->alloc_size,
&rxq->headroom, &rxq->frag_count);
/* Create page pool for RX queue */
err = mana_create_page_pool(rxq, gc);
@@ -3196,14 +3462,15 @@ static void mana_create_rxq_debugfs(struct mana_port_context *apc, int idx)
&mana_dbg_q_fops);
}
+/* The caller must destroy queues added before a failure. */
static int mana_add_rx_queues(struct mana_port_context *apc,
- struct net_device *ndev)
+ struct net_device *ndev, unsigned int first)
{
struct mana_rxq *rxq;
int err = 0;
int i;
- for (i = 0; i < apc->num_queues; i++) {
+ for (i = first; i < apc->num_queues; i++) {
rxq = mana_create_rxq(apc, i, &apc->eqs[i], ndev);
if (IS_ERR(rxq)) {
err = PTR_ERR(rxq);
@@ -3211,8 +3478,6 @@ static int mana_add_rx_queues(struct mana_port_context *apc,
goto out;
}
- u64_stats_init(&rxq->stats.syncp);
-
apc->rxqs[i] = rxq;
mana_create_rxq_debugfs(apc, i);
@@ -3223,14 +3488,15 @@ static int mana_add_rx_queues(struct mana_port_context *apc,
return err;
}
-static void mana_destroy_rxqs(struct mana_port_context *apc)
+static void mana_destroy_rxqs_from(struct mana_port_context *apc,
+ unsigned int first)
{
struct mana_rxq *rxq;
u32 rxq_idx;
if (apc->rxqs) {
- for (rxq_idx = 0; rxq_idx < apc->num_queues; rxq_idx++) {
+ for (rxq_idx = first; rxq_idx < apc->num_queues; rxq_idx++) {
rxq = apc->rxqs[rxq_idx];
if (!rxq)
continue;
@@ -3241,6 +3507,11 @@ static void mana_destroy_rxqs(struct mana_port_context *apc)
}
}
+static void mana_destroy_rxqs(struct mana_port_context *apc)
+{
+ mana_destroy_rxqs_from(apc, 0);
+}
+
static void mana_destroy_vport(struct mana_port_context *apc)
{
struct gdma_dev *gd = apc->ac->gdma_dev;
@@ -3306,6 +3577,27 @@ static void mana_rss_table_init(struct mana_port_context *apc)
ethtool_rxfh_indir_default(i, apc->num_queues);
}
+/* Keep user tables with valid indices; defer loss notification. */
+static bool mana_rss_table_keep(struct mana_port_context *apc,
+ unsigned int num_queues, bool *lost)
+{
+ u32 i;
+
+ *lost = false;
+
+ if (!netif_is_rxfh_configured(apc->ndev))
+ return false;
+
+ for (i = 0; i < apc->indir_table_sz; i++) {
+ if (apc->indir_table[i] >= num_queues) {
+ *lost = true;
+ return false;
+ }
+ }
+
+ return true;
+}
+
int mana_disable_vport_rx(struct mana_port_context *apc)
{
return mana_cfg_vport_steering(apc, TRI_STATE_FALSE, false, false,
@@ -3576,6 +3868,7 @@ int mana_alloc_queues(struct net_device *ndev)
{
struct mana_port_context *apc = netdev_priv(ndev);
struct gdma_dev *gd = apc->ac->gdma_dev;
+ bool indir_lost;
int err;
err = mana_create_vport(apc, ndev);
@@ -3592,7 +3885,7 @@ int mana_alloc_queues(struct net_device *ndev)
goto destroy_vport;
}
- err = mana_create_txq(apc, ndev);
+ err = mana_create_txq(apc, ndev, 0);
if (err) {
netdev_err(ndev, "Failed to create TXQ on vPort %u: %d\n",
apc->port_idx, err);
@@ -3607,7 +3900,7 @@ int mana_alloc_queues(struct net_device *ndev)
goto destroy_txq;
}
- err = mana_add_rx_queues(apc, ndev);
+ err = mana_add_rx_queues(apc, ndev, 0);
if (err)
goto destroy_rxq;
@@ -3621,7 +3914,9 @@ int mana_alloc_queues(struct net_device *ndev)
goto destroy_rxq;
}
- mana_rss_table_init(apc);
+ /* Loss notification needs a netdev instance lock we may lack. */
+ if (!mana_rss_table_keep(apc, apc->num_queues, &indir_lost))
+ mana_rss_table_init(apc);
err = mana_config_rss(apc, TRI_STATE_TRUE, true, true);
if (err) {
@@ -3679,15 +3974,88 @@ int mana_attach(struct net_device *ndev)
return 0;
}
-static int mana_dealloc_queues(struct net_device *ndev)
+/* Drain with a per-set timeout; return true only for a successful FLR. A false
+ * return does not guarantee DMA quiescence.
+ */
+static bool mana_drain_txqs(struct mana_port_context *apc)
{
- struct mana_port_context *apc = netdev_priv(ndev);
unsigned long timeout = jiffies + 120 * HZ;
- struct gdma_dev *gd = apc->ac->gdma_dev;
+ struct gdma_context *gc = apc->ac->gdma_dev->gdma_context;
+ bool quiesced = true;
+ bool reset = false;
struct mana_txq *txq;
struct sk_buff *skb;
- int i, err;
u32 tsleep;
+ int i, err;
+
+ if (!apc->tx_qp)
+ return false;
+
+ for (i = 0; i < apc->num_queues; i++) {
+ if (!apc->tx_qp[i])
+ continue;
+
+ txq = &apc->tx_qp[i]->txq;
+
+ /* A previous function reset invalidated these queues. */
+ if (READ_ONCE(apc->ac->reset_gen) != txq->reset_gen)
+ continue;
+
+ tsleep = 1000;
+ while (atomic_read(&txq->pending_sends) > 0 &&
+ time_before(jiffies, timeout)) {
+ usleep_range(tsleep, tsleep + 1000);
+ tsleep <<= 1;
+ }
+ if (atomic_read(&txq->pending_sends)) {
+ err = pcie_flr(to_pci_dev(gc->dev));
+ if (err) {
+ netdev_err(apc->ndev,
+ "flr failed %d with %d pkts pending in txq %u\n",
+ err,
+ atomic_read(&txq->pending_sends),
+ txq->gdma_txq_id);
+ quiesced = false;
+ } else {
+ /* Invalidate queues on every port after the
+ * function reset.
+ */
+ WRITE_ONCE(apc->ac->reset_gen,
+ apc->ac->reset_gen + 1);
+
+ reset = true;
+ }
+ break;
+ }
+ }
+
+ /* A failed FLR cannot justify unmapping pending TX buffers. */
+ if (!quiesced) {
+ netdev_err(apc->ndev,
+ "device not quiesced, leaking pending TX buffers instead of unmapping memory it can still DMA from\n");
+ return reset;
+ }
+
+ for (i = 0; i < apc->num_queues; i++) {
+ if (!apc->tx_qp[i])
+ continue;
+
+ txq = &apc->tx_qp[i]->txq;
+ while ((skb = skb_dequeue(&txq->pending_skbs))) {
+ mana_unmap_skb(skb, apc);
+ dev_kfree_skb_any(skb);
+ }
+ atomic_set(&txq->pending_sends, 0);
+ }
+
+ return reset;
+}
+
+static int mana_dealloc_queues(struct net_device *ndev)
+{
+ struct mana_port_context *apc = netdev_priv(ndev);
+ struct gdma_dev *gd = apc->ac->gdma_dev;
+ int err;
if (apc->port_is_up)
return -EINVAL;
@@ -3698,48 +4066,21 @@ static int mana_dealloc_queues(struct net_device *ndev)
if (gd->gdma_context->is_pf && !apc->ac->bm_hostmode)
mana_pf_deregister_filter(apc);
- /* No packet can be transmitted now since apc->port_is_up is false.
- * There is still a tiny chance that mana_poll_tx_cq() can re-enable
- * a txq because it may not timely see apc->port_is_up being cleared
- * to false, but it doesn't matter since mana_start_xmit() drops any
- * new packets due to apc->port_is_up being false.
- *
- * Drain all the in-flight TX packets.
- * A timeout of 120 seconds for all the queues is used.
- * This will break the while loop when h/w is not responding.
- * This value of 120 has been decided here considering max
- * number of queues.
- */
+ /* After FLR, schedule a best-effort sibling-port rebuild. */
+ if (mana_drain_txqs(apc)) {
+ struct mana_context *ac = apc->ac;
+ unsigned int i;
- if (apc->tx_qp) {
- for (i = 0; i < apc->num_queues; i++) {
- txq = &apc->tx_qp[i]->txq;
- tsleep = 1000;
- while (atomic_read(&txq->pending_sends) > 0 &&
- time_before(jiffies, timeout)) {
- usleep_range(tsleep, tsleep + 1000);
- tsleep <<= 1;
- }
- if (atomic_read(&txq->pending_sends)) {
- err =
- pcie_flr(to_pci_dev(gd->gdma_context->dev));
- if (err) {
- netdev_err(ndev, "flr failed %d with %d pkts pending in txq %u\n",
- err,
- atomic_read(&txq->pending_sends),
- txq->gdma_txq_id);
- }
- break;
- }
- }
+ for (i = 0; i < ac->num_ports; i++) {
+ struct mana_port_context *sib;
- for (i = 0; i < apc->num_queues; i++) {
- txq = &apc->tx_qp[i]->txq;
- while ((skb = skb_dequeue(&txq->pending_skbs))) {
- mana_unmap_skb(skb, apc);
- dev_kfree_skb_any(skb);
- }
- atomic_set(&txq->pending_sends, 0);
+ if (!ac->ports[i] || ac->ports[i] == ndev)
+ continue;
+ sib = netdev_priv(ac->ports[i]);
+ netdev_err(ac->ports[i],
+ "queues reset by a sibling port, scheduling rebuild\n");
+ queue_work(ac->per_port_queue_reset_wq,
+ &sib->queue_reset_work);
}
}
@@ -3763,6 +4104,685 @@ static int mana_dealloc_queues(struct net_device *ndev)
return 0;
}
+static void mana_qset_snapshot(const struct mana_port_context *ctx,
+ struct mana_qset *out)
+{
+ out->tx_qp = ctx->tx_qp;
+ out->rxqs = ctx->rxqs;
+ out->indir_table = ctx->indir_table;
+ out->indir_table_sz = ctx->indir_table_sz;
+ out->rxobj_table = ctx->rxobj_table;
+ out->default_rxobj = ctx->default_rxobj;
+ out->num_queues = ctx->num_queues;
+ out->rx_queue_size = ctx->rx_queue_size;
+ out->tx_queue_size = ctx->tx_queue_size;
+ out->priv_flags = ctx->priv_flags;
+ out->mtu = ctx->configured_mtu;
+ out->bpf_prog = ctx->bpf_prog;
+
+ out->rxfh_indir_lost = false;
+}
+
+static void mana_qset_install(struct mana_port_context *ctx,
+ const struct mana_qset *qset)
+{
+ ctx->tx_qp = qset->tx_qp;
+ ctx->rxqs = qset->rxqs;
+ ctx->indir_table = qset->indir_table;
+ ctx->indir_table_sz = qset->indir_table_sz;
+ ctx->rxobj_table = qset->rxobj_table;
+ ctx->default_rxobj = qset->default_rxobj;
+ ctx->num_queues = qset->num_queues;
+ ctx->rx_queue_size = qset->rx_queue_size;
+ ctx->tx_queue_size = qset->tx_queue_size;
+ ctx->priv_flags = qset->priv_flags;
+ ctx->configured_mtu = qset->mtu;
+ ctx->bpf_prog = qset->bpf_prog;
+}
+
+/* Scratch starts without SQs/RQs and borrows the port's EQ pool. Never call
+ * mana_destroy_eq() on it.
+ */
+struct mana_port_context *mana_qset_scratch_alloc(struct mana_port_context *apc)
+{
+ struct mana_port_context *scratch;
+
+ scratch = kvzalloc_obj(*scratch, GFP_KERNEL);
+ if (!scratch)
+ return NULL;
+
+ *scratch = *apc;
+
+ scratch->tx_qp = NULL;
+ scratch->rxqs = NULL;
+ scratch->indir_table = NULL;
+ scratch->rxobj_table = NULL;
+ scratch->default_rxobj = INVALID_MANA_HANDLE;
+
+ /* Do not consume the live set's pre-allocated RX buffers. */
+ scratch->rxbufs_pre = NULL;
+ scratch->das_pre = NULL;
+ scratch->rxbpre_total = 0;
+
+ /* Suppress debugfs names that would collide with the live set. */
+ scratch->mana_port_debugfs = ERR_PTR(-ENODEV);
+
+ return scratch;
+}
+
+void mana_qset_scratch_free(struct mana_port_context *scratch)
+{
+ kvfree(scratch);
+}
+
+/* Split into kept queues and a retiring tail without changing live ownership.
+ * Queue i retains EQ i.
+ */
+int mana_split_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch, unsigned int new_count,
+ struct mana_qset *out_new, struct mana_qset *out_tail)
+{
+ unsigned int old_count = apc->num_queues;
+ struct mana_tx_qp **new_tx, **tail_tx;
+ struct mana_rxq **new_rx, **tail_rx;
+ unsigned int tail_count;
+ bool indir_lost;
+ unsigned int i;
+ int err;
+
+ ASSERT_RTNL();
+
+ if (WARN_ON(new_count == 0 || new_count >= old_count))
+ return -EINVAL;
+ if (WARN_ON(!apc->tx_qp || !apc->rxqs))
+ return -EINVAL;
+
+ tail_count = old_count - new_count;
+
+ /* Build steering separately so it cannot index beyond the shortened RX
+ * array.
+ */
+ scratch->num_queues = new_count;
+ err = mana_rss_table_alloc(scratch);
+ if (err)
+ return err;
+
+ if (mana_rss_table_keep(apc, new_count, &indir_lost))
+ memcpy(scratch->indir_table, apc->indir_table,
+ apc->indir_table_sz * sizeof(*apc->indir_table));
+ else
+ mana_rss_table_init(scratch);
+
+ new_tx = kzalloc_objs(struct mana_tx_qp *, new_count);
+ new_rx = kzalloc_objs(struct mana_rxq *, new_count);
+ tail_tx = kzalloc_objs(struct mana_tx_qp *, tail_count);
+ tail_rx = kzalloc_objs(struct mana_rxq *, tail_count);
+ if (!new_tx || !new_rx || !tail_tx || !tail_rx) {
+ err = -ENOMEM;
+ goto free_arrays;
+ }
+
+ for (i = 0; i < new_count; i++) {
+ new_tx[i] = apc->tx_qp[i];
+ new_rx[i] = apc->rxqs[i];
+ }
+ for (i = 0; i < tail_count; i++) {
+ tail_tx[i] = apc->tx_qp[new_count + i];
+ tail_rx[i] = apc->rxqs[new_count + i];
+ }
+
+ out_new->tx_qp = new_tx;
+ out_new->rxqs = new_rx;
+ out_new->indir_table = scratch->indir_table;
+ out_new->indir_table_sz = scratch->indir_table_sz;
+ out_new->rxobj_table = scratch->rxobj_table;
+ out_new->default_rxobj = apc->rxqs[0]->rxobj;
+ out_new->num_queues = new_count;
+ out_new->rx_queue_size = apc->rx_queue_size;
+ out_new->tx_queue_size = apc->tx_queue_size;
+ out_new->priv_flags = apc->priv_flags;
+ out_new->mtu = apc->configured_mtu;
+ out_new->bpf_prog = apc->bpf_prog;
+ out_new->rxfh_indir_lost = indir_lost;
+
+ scratch->indir_table = NULL;
+ scratch->rxobj_table = NULL;
+
+ memset(out_tail, 0, sizeof(*out_tail));
+ out_tail->tx_qp = tail_tx;
+ out_tail->rxqs = tail_rx;
+ out_tail->default_rxobj = INVALID_MANA_HANDLE;
+ out_tail->num_queues = tail_count;
+ out_tail->rx_queue_size = apc->rx_queue_size;
+ out_tail->tx_queue_size = apc->tx_queue_size;
+ out_tail->priv_flags = apc->priv_flags;
+ out_tail->mtu = apc->configured_mtu;
+ out_tail->bpf_prog = apc->bpf_prog;
+
+ return 0;
+
+free_arrays:
+ kfree(new_tx);
+ kfree(new_rx);
+ kfree(tail_tx);
+ kfree(tail_rx);
+ mana_cleanup_indir_table(scratch);
+ return err;
+}
+
+/* Free containers only; the live port still owns the queues. */
+void mana_discard_split(struct mana_qset *newq, struct mana_qset *tailq)
+{
+ kfree(newq->tx_qp);
+ kfree(newq->rxqs);
+ kfree(newq->indir_table);
+ kfree(newq->rxobj_table);
+ kfree(tailq->tx_qp);
+ kfree(tailq->rxqs);
+ memset(newq, 0, sizeof(*newq));
+ memset(tailq, 0, sizeof(*tailq));
+}
+
+/* Carry existing queues into @out_new; allocate only the tail. @out_fresh
+ * isolates new queues for cleanup after a failed publish.
+ */
+int mana_grow_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch, unsigned int new_count,
+ struct mana_qset *out_new, struct mana_qset *out_fresh)
+{
+ unsigned int old_count = apc->num_queues;
+ struct mana_tx_qp **new_tx, **fresh_tx;
+ struct mana_rxq **new_rx, **fresh_rx;
+ struct net_device *ndev = apc->ndev;
+ unsigned int fresh_count;
+ bool indir_lost;
+ unsigned int i;
+ int err;
+
+ ASSERT_RTNL();
+
+ if (WARN_ON(new_count <= old_count))
+ return -EINVAL;
+ if (WARN_ON(!apc->tx_qp || !apc->rxqs))
+ return -EINVAL;
+
+ fresh_count = new_count - old_count;
+
+ new_tx = kzalloc_objs(struct mana_tx_qp *, new_count);
+ new_rx = kzalloc_objs(struct mana_rxq *, new_count);
+ fresh_tx = kzalloc_objs(struct mana_tx_qp *, fresh_count);
+ fresh_rx = kzalloc_objs(struct mana_rxq *, fresh_count);
+ if (!new_tx || !new_rx || !fresh_tx || !fresh_rx) {
+ err = -ENOMEM;
+ goto free_arrays;
+ }
+
+ for (i = 0; i < old_count; i++) {
+ new_tx[i] = apc->tx_qp[i];
+ new_rx[i] = apc->rxqs[i];
+ }
+
+ scratch->num_queues = new_count;
+ scratch->tx_qp = new_tx;
+ scratch->rxqs = new_rx;
+
+ err = mana_rss_table_alloc(scratch);
+ if (err)
+ goto free_arrays;
+
+ err = mana_grow_eqs(apc, new_count);
+ if (err)
+ goto cleanup_rss;
+
+ scratch->eqs = apc->eqs;
+ scratch->num_eqs = apc->num_eqs;
+
+ err = mana_create_txq(scratch, ndev, old_count);
+ if (err)
+ goto cleanup_rss;
+
+ err = mana_add_rx_queues(scratch, ndev, old_count);
+ if (err)
+ goto cleanup_rxq;
+
+ if (mana_rss_table_keep(apc, new_count, &indir_lost))
+ memcpy(scratch->indir_table, apc->indir_table,
+ apc->indir_table_sz * sizeof(*apc->indir_table));
+ else
+ mana_rss_table_init(scratch);
+
+ mana_qset_snapshot(scratch, out_new);
+ out_new->rxfh_indir_lost = indir_lost;
+
+ for (i = 0; i < fresh_count; i++) {
+ fresh_tx[i] = new_tx[old_count + i];
+ fresh_rx[i] = new_rx[old_count + i];
+ }
+
+ memset(out_fresh, 0, sizeof(*out_fresh));
+ out_fresh->tx_qp = fresh_tx;
+ out_fresh->rxqs = fresh_rx;
+ out_fresh->default_rxobj = INVALID_MANA_HANDLE;
+ out_fresh->num_queues = fresh_count;
+ out_fresh->rx_queue_size = apc->rx_queue_size;
+ out_fresh->tx_queue_size = apc->tx_queue_size;
+ out_fresh->priv_flags = apc->priv_flags;
+ out_fresh->mtu = apc->configured_mtu;
+ out_fresh->bpf_prog = apc->bpf_prog;
+
+ /* Take XDP refs on fresh RXQs only. On the merged set,
+ * mana_chn_setxdp() returns early on the carried rxqs[0].
+ */
+ mana_qset_install(scratch, out_fresh);
+ mana_chn_setxdp(scratch, mana_xdp_get(apc));
+
+ return 0;
+
+cleanup_rxq:
+ mana_destroy_rxqs_from(scratch, old_count);
+ mana_destroy_txq_from(scratch, old_count);
+cleanup_rss:
+ mana_cleanup_indir_table(scratch);
+free_arrays:
+ /* Free containers only; carried queues remain live. */
+ scratch->tx_qp = NULL;
+ scratch->rxqs = NULL;
+ kfree(new_tx);
+ kfree(new_rx);
+ kfree(fresh_tx);
+ kfree(fresh_rx);
+
+ mana_shrink_eqs(apc, apc->num_queues);
+
+ netdev_err(ndev, "%s(num_queues=%u) failed: %d\n", __func__,
+ new_count, err);
+ return err;
+}
+
+/* Free merged containers only, not carried queues. The caller must retire fresh
+ * queues separately.
+ */
+void mana_discard_grow(struct mana_qset *newq)
+{
+ kfree(newq->tx_qp);
+ kfree(newq->rxqs);
+ kfree(newq->indir_table);
+ kfree(newq->rxobj_table);
+ memset(newq, 0, sizeof(*newq));
+}
+
+/* Rebuild at the current count; resize uses split/grow. */
+int mana_alloc_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch,
+ unsigned int rx_queue_size, unsigned int tx_queue_size,
+ u32 priv_flags, int mtu, struct bpf_prog *bpf_prog,
+ struct mana_qset *out)
+{
+ struct net_device *ndev = scratch->ndev;
+ bool indir_lost;
+ int err;
+
+ ASSERT_RTNL();
+
+ scratch->num_queues = apc->num_queues;
+ scratch->rx_queue_size = rx_queue_size;
+ scratch->tx_queue_size = tx_queue_size;
+ scratch->priv_flags = priv_flags;
+
+ scratch->configured_mtu = mtu;
+ scratch->bpf_prog = bpf_prog;
+
+ err = mana_init_port_context(scratch);
+ if (err)
+ goto out_err;
+
+ err = mana_rss_table_alloc(scratch);
+ if (err)
+ goto cleanup_rxq_array;
+
+ /* Reuse the existing EQ pool; the queue count is unchanged. */
+ scratch->eqs = apc->eqs;
+ scratch->num_eqs = apc->num_eqs;
+
+ err = mana_create_txq(scratch, ndev, 0);
+ if (err)
+ goto cleanup_rss;
+
+ err = mana_add_rx_queues(scratch, ndev, 0);
+ if (err)
+ goto cleanup_rxq;
+
+ if (mana_rss_table_keep(apc, scratch->num_queues, &indir_lost))
+ memcpy(scratch->indir_table, apc->indir_table,
+ apc->indir_table_sz * sizeof(*apc->indir_table));
+ else
+ mana_rss_table_init(scratch);
+
+ mana_qset_snapshot(scratch, out);
+ out->rxfh_indir_lost = indir_lost;
+ return 0;
+
+cleanup_rxq:
+ mana_destroy_rxqs(scratch);
+ mana_destroy_txq(scratch);
+cleanup_rss:
+ mana_cleanup_indir_table(scratch);
+cleanup_rxq_array:
+ kfree(scratch->rxqs);
+ scratch->rxqs = NULL;
+out_err:
+ netdev_err(ndev, "%s(num_queues=%u) failed: %d\n", __func__,
+ apc->num_queues, err);
+ return err;
+}
+
+/* Destroy caller-owned CQs before closing this dead-end port: closing also
+ * frees the shared EQ pool. Requires RTNL.
+ */
+void mana_publish_close_if_needed(struct mana_port_context *apc)
+{
+ ASSERT_RTNL();
+
+ if (!apc->publish_dead_end)
+ return;
+
+ apc->publish_dead_end = false;
+
+ if (mana_dealloc_queues(apc->ndev))
+ netdev_err(apc->ndev,
+ "failed to close the port after a failed rollback\n");
+}
+
+/* Carried-over queues may still have full rings. */
+static void mana_start_txqs(struct mana_port_context *apc)
+{
+ struct net_device *ndev = apc->ndev;
+ unsigned int i;
+
+ if (!apc->tx_qp)
+ return;
+
+ /* Order port_is_up=true before ring reads to avoid a missed wakeup.
+ * Pair with mana_poll_tx_cq()'s full barrier after its tail update.
+ */
+ smp_mb();
+
+ for (i = 0; i < apc->num_queues; i++) {
+ if (!apc->tx_qp[i])
+ continue;
+
+ if (mana_can_tx(apc->tx_qp[i]->txq.gdma_sq))
+ netif_tx_wake_queue(netdev_get_tx_queue(ndev, i));
+ }
+}
+
+/* Retiring completions must not wake replacement queues. Mark the leaving set
+ * before unmarking the incoming set.
+ */
+static void mana_qset_set_retiring(struct mana_qset *qset,
+ const struct mana_qset *keep, bool retiring)
+{
+ unsigned int q;
+
+ for (q = 0; q < qset->num_queues; q++) {
+ if (qset->tx_qp && qset->tx_qp[q])
+ WRITE_ONCE(qset->tx_qp[q]->txq.retiring, retiring);
+
+ if (!qset->rxqs || !qset->rxqs[q])
+ continue;
+
+ /* Carried RXQs remain the sole poll writers of shared slots. */
+ if (retiring && keep && q < keep->num_queues &&
+ keep->rxqs && keep->rxqs[q] == qset->rxqs[q])
+ continue;
+
+ /* Switch to drain_stats; hand off shared slots after a grace
+ * period.
+ */
+ WRITE_ONCE(qset->rxqs[q]->retiring, retiring);
+ }
+}
+
+/* Leave TX stopped and request RX disable; steering may be unrecoverable. */
+static void mana_publish_give_up(struct mana_port_context *apc)
+{
+ int err;
+
+ apc->rss_state = TRI_STATE_FALSE;
+
+ err = mana_disable_vport_rx(apc);
+ if (err && mana_en_need_log(apc, err))
+ netdev_err(apc->ndev, "failed to disable vPort RX: %d\n", err);
+
+ apc->carrier_forced_off = netif_carrier_ok(apc->ndev);
+ netif_carrier_off(apc->ndev);
+ apc->publish_dead_end = true;
+}
+
+/* Keep the RX count high until retiring RQs stop delivering their indices. */
+static int mana_raise_real_num_rx(struct net_device *ndev, unsigned int count)
+{
+ if (count <= ndev->real_num_rx_queues)
+ return 0;
+
+ return netif_set_real_num_rx_queues(ndev, count);
+}
+
+/* Publish under RTNL with TX gated. An error restores old pointers, not
+ * necessarily service. Free only owned queues.
+ */
+int mana_publish_qset(struct mana_port_context *apc, struct mana_qset *newq,
+ struct mana_qset *out_old)
+{
+ struct net_device *ndev = apc->ndev;
+ int err;
+
+ ASSERT_RTNL();
+
+ /* Close the XDP gate before stopping TX queues. Pair with
+ * mana_poll_tx_cq()'s smp_rmb() to prevent mid-swap wakeups.
+ */
+ WRITE_ONCE(apc->port_is_up, false);
+
+ /* Ensure port state updated before txq state */
+ smp_wmb();
+
+ netif_tx_disable(ndev);
+
+ mana_qset_snapshot(apc, out_old);
+
+ /* Mark before the grace period so old completions cannot wake the
+ * replacement's stopped queue.
+ */
+ mana_qset_set_retiring(out_old, newq, true);
+
+ /* Drain TX/XDP readers past the gate and polls missing retiring. */
+ synchronize_net();
+
+ mana_qset_set_retiring(newq, NULL, false);
+
+ mana_qset_install(apc, newq);
+ apc->rss_state = apc->num_queues > 1 ? TRI_STATE_TRUE : TRI_STATE_FALSE;
+
+ err = netif_set_real_num_tx_queues(ndev, apc->num_queues);
+ if (err)
+ goto rollback;
+
+ err = mana_raise_real_num_rx(ndev, apc->num_queues);
+ if (err)
+ goto rollback;
+
+ /* Install XDP before steering reaches the incoming RXQs. */
+ mana_chn_setxdp(apc, mana_xdp_get(apc));
+
+ err = mana_config_rss(apc, TRI_STATE_TRUE, true, true);
+ if (err)
+ goto rollback;
+
+ WRITE_ONCE(ndev->mtu, apc->configured_mtu);
+
+ /* Publish fields before opening the gate; pair with TX/XDP read
+ * barriers. The post-gate full barrier cannot replace this.
+ */
+ smp_wmb();
+
+ WRITE_ONCE(apc->port_is_up, true);
+ mana_start_txqs(apc);
+
+ /* Report a lost user table only after successful publication. */
+ if (newq->rxfh_indir_lost)
+ ethtool_rxfh_indir_lost(ndev);
+
+ return 0;
+
+rollback:
+ netdev_err(ndev, "%s failed: %d, restoring previous queue set\n",
+ __func__, err);
+
+ mana_qset_set_retiring(newq, out_old, true);
+
+ /* Quiesce new shared-slot writers before restoring old ones. */
+ synchronize_net();
+
+ mana_qset_set_retiring(out_old, NULL, false);
+
+ /* Quiesce old drain_stats writers before folding. */
+ synchronize_net();
+ mana_fold_qset_rx_stats(apc, out_old);
+
+ mana_qset_install(apc, out_old);
+ apc->rss_state = apc->num_queues > 1 ? TRI_STATE_TRUE : TRI_STATE_FALSE;
+
+ if (netif_set_real_num_tx_queues(ndev, apc->num_queues) ||
+ mana_raise_real_num_rx(ndev, apc->num_queues)) {
+ /* Inconsistent restored queue counts prohibit TX; leave the
+ * port stopped.
+ */
+ netdev_err(ndev, "failed to restore queue counts, closing the port\n");
+ mana_publish_give_up(apc);
+ return err;
+ }
+
+ if (mana_config_rss(apc, TRI_STATE_TRUE, true, true)) {
+ /* Do not reopen TX with mismatched steering; RX disable is
+ * best-effort.
+ */
+ netdev_err(ndev, "failed to restore RSS steering, closing the port\n");
+ mana_publish_give_up(apc);
+ return err;
+ }
+
+ WRITE_ONCE(ndev->mtu, apc->configured_mtu);
+
+ /* Publish restored fields before reopening the gate, as on success. */
+ smp_wmb();
+
+ WRITE_ONCE(apc->port_is_up, true);
+ mana_start_txqs(apc);
+
+ return err;
+}
+
+/* Create missing debugfs nodes once retiring names are gone. */
+void mana_qset_debugfs_publish(struct mana_port_context *apc)
+{
+ unsigned int i;
+
+ ASSERT_RTNL();
+
+ if (IS_ERR_OR_NULL(apc->mana_port_debugfs))
+ return;
+
+ for (i = 0; i < apc->num_queues; i++) {
+ if (apc->tx_qp && apc->tx_qp[i] &&
+ IS_ERR_OR_NULL(apc->tx_qp[i]->mana_tx_debugfs))
+ mana_create_txq_debugfs(apc, i);
+
+ if (apc->rxqs && apc->rxqs[i] &&
+ IS_ERR_OR_NULL(apc->rxqs[i]->mana_rx_debugfs))
+ mana_create_rxq_debugfs(apc, i);
+ }
+}
+
+/* Under RTNL, free only queues no longer shared with the installed set. */
+void mana_free_qset(struct mana_port_context *scratch, struct mana_qset *qset)
+{
+ struct mana_port_context *apc = netdev_priv(scratch->ndev);
+ struct bpf_prog *retiring_prog;
+ unsigned int retiring_queues;
+
+ ASSERT_RTNL();
+
+ if (!qset->rxqs && !qset->tx_qp)
+ return;
+
+ if (qset->tx_qp) {
+ unsigned int q;
+
+ for (q = 0; q < qset->num_queues; q++) {
+ if (qset->tx_qp[q])
+ WRITE_ONCE(qset->tx_qp[q]->txq.retiring, true);
+ }
+ }
+
+ /* Keep retired queues and arrays through this grace period; local NAPI
+ * synchronization does not drain other devices' XDP.
+ */
+ synchronize_net();
+
+ mana_qset_install(scratch, qset);
+
+ /* Keep retiring RXQs' XDP programs and references until RX teardown.
+ * Read the program from the queues, not queue-set metadata.
+ */
+ retiring_prog = mana_chn_xdp_peek(scratch);
+ retiring_queues = scratch->num_queues;
+
+ /* Drain TX before unmapping RX buffers. */
+ if (mana_drain_txqs(scratch)) {
+ /* FLR also destroys the HWC; rebuilding ports is best-effort.
+ * This path does not reinitialize the device.
+ */
+ struct mana_context *ac = apc->ac;
+ struct mana_port_context *sib;
+ unsigned int i;
+
+ netdev_err(scratch->ndev,
+ "device reset while retiring a queue set, scheduling port reset\n");
+
+ for (i = 0; i < ac->num_ports; i++) {
+ if (!ac->ports[i])
+ continue;
+ sib = netdev_priv(ac->ports[i]);
+ queue_work(ac->per_port_queue_reset_wq,
+ &sib->queue_reset_work);
+ }
+ }
+
+ /* Fence RQs before unmapping, but teardown proceeds on errors. */
+ mana_fence_rqs(scratch);
+
+ mana_destroy_rxqs(scratch);
+
+ mana_chn_xdp_release(retiring_prog, retiring_queues);
+
+ mana_destroy_txq(scratch);
+ mana_cleanup_indir_table(scratch);
+ kfree(scratch->rxqs);
+ scratch->rxqs = NULL;
+
+ memset(qset, 0, sizeof(*qset));
+
+ /* Retiring RQs can no longer deliver indices beyond the live queue
+ * count.
+ */
+ netif_set_real_num_rx_queues(apc->ndev, apc->num_queues);
+
+ mana_shrink_eqs(apc, apc->num_queues);
+
+ mana_qset_debugfs_publish(apc);
+}
+
int mana_detach(struct net_device *ndev, bool from_close)
{
struct mana_port_context *apc = netdev_priv(ndev);
@@ -3786,10 +4806,8 @@ int mana_detach(struct net_device *ndev, bool from_close)
if (apc->port_st_save) {
err = mana_dealloc_queues(ndev);
- if (err) {
+ if (err)
netdev_err(ndev, "%s failed to deallocate queues: %d\n", __func__, err);
- return err;
- }
}
if (!from_close) {
@@ -3826,6 +4844,7 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
apc->port_handle = INVALID_MANA_HANDLE;
apc->pf_filter_handle = INVALID_MANA_HANDLE;
apc->port_idx = port_idx;
+ apc->configured_mtu = ndev->mtu;
apc->link_cfg_error = 1;
apc->cqe_coalescing_enable = 0;
apc->cqe8_coalescing_enable = 0;
@@ -3840,6 +4859,10 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
apc->tx_dim_enabled = MANA_ADAPTIVE_TX_DEF;
}
+ err = mana_alloc_queue_stats(apc);
+ if (err)
+ goto free_net;
+
mutex_init(&apc->vport_mutex);
apc->vport_use_count = 0;
@@ -3862,7 +4885,7 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
err = mana_init_port(ndev);
if (err)
- goto free_net;
+ goto free_stats;
err = mana_rss_table_alloc(apc);
if (err)
@@ -3899,6 +4922,8 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
mana_cleanup_indir_table(apc);
reset_apc:
mana_cleanup_port_context(apc);
+free_stats:
+ mana_free_queue_stats(apc);
free_net:
*ndev_storage = NULL;
netdev_err(ndev, "Failed to probe vPort %d: %d\n", port_idx, err);
@@ -4239,6 +5264,11 @@ void mana_remove(struct gdma_dev *gd, bool suspending)
unregister_netdevice(ndev);
mana_cleanup_indir_table(apc);
+ mana_free_queue_stats(apc);
+
+ /* Remove the port from reset walks before freeing its netdev.
+ */
+ ac->ports[i] = NULL;
rtnl_unlock();
diff --git a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
index ece7ff9cc409a..6586a9d2a1343 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_ethtool.c
@@ -242,6 +242,12 @@ static void mana_get_ethtool_stats(struct net_device *ndev,
u64 xdp_tx;
u64 pkt_len0_err;
u64 coalesced_cqe[MANA_CQE_COAL_PKTS_8 - 1];
+ u64 ret_coalesced_cqe[MANA_CQE_COAL_PKTS_8 - 1];
+ u64 ret_packets, ret_bytes;
+ u64 ret_xdp_redirect;
+ u64 ret_pkt_len0_err;
+ u64 ret_xdp_drop;
+ u64 ret_xdp_tx;
u64 tso_packets;
u64 tso_bytes;
u64 tso_inner_packets;
@@ -252,14 +258,11 @@ static void mana_get_ethtool_stats(struct net_device *ndev,
u64 mana_map_err;
int q, i = 0, j;
- if (!apc->port_is_up)
- return;
-
- /* We call this mana function to get the phy stats from GDMA and includes
- * aggregate tx/rx drop counters, Per-TC(Traffic Channel) tx/rx and pause
- * counters.
+ /* Counters outlive the queues, but suspend can destroy the HW channel
+ * while the netdev remains registered. Gate only the PHY query.
*/
- mana_query_phy_stats(apc);
+ if (apc->port_is_up)
+ mana_query_phy_stats(apc);
for (q = 0; q < ARRAY_SIZE(mana_eth_stats); q++)
data[i++] = *(u64 *)(eth_stats + mana_eth_stats[q].offset);
@@ -271,7 +274,7 @@ static void mana_get_ethtool_stats(struct net_device *ndev,
data[i++] = *(u64 *)(phy_stats + mana_phy_stats[q].offset);
for (q = 0; q < num_queues; q++) {
- rx_stats = &apc->rxqs[q]->stats;
+ rx_stats = &apc->rxq_stats[q];
do {
start = u64_stats_fetch_begin(&rx_stats->syncp);
@@ -285,6 +288,33 @@ static void mana_get_ethtool_stats(struct net_device *ndev,
coalesced_cqe[j] = rx_stats->coalesced_cqe[j];
} while (u64_stats_fetch_retry(&rx_stats->syncp, start));
+ /* Snapshot separately so a retry cannot add retired counters
+ * twice.
+ */
+ rx_stats = &apc->rxq_stats_ret[q];
+
+ do {
+ start = u64_stats_fetch_begin(&rx_stats->syncp);
+ ret_packets = rx_stats->packets;
+ ret_bytes = rx_stats->bytes;
+ ret_xdp_drop = rx_stats->xdp_drop;
+ ret_xdp_tx = rx_stats->xdp_tx;
+ ret_xdp_redirect = rx_stats->xdp_redirect;
+ ret_pkt_len0_err = rx_stats->pkt_len0_err;
+ for (j = 0; j < MANA_CQE_COAL_PKTS_8 - 1; j++)
+ ret_coalesced_cqe[j] =
+ rx_stats->coalesced_cqe[j];
+ } while (u64_stats_fetch_retry(&rx_stats->syncp, start));
+
+ packets += ret_packets;
+ bytes += ret_bytes;
+ xdp_drop += ret_xdp_drop;
+ xdp_tx += ret_xdp_tx;
+ xdp_redirect += ret_xdp_redirect;
+ pkt_len0_err += ret_pkt_len0_err;
+ for (j = 0; j < MANA_CQE_COAL_PKTS_8 - 1; j++)
+ coalesced_cqe[j] += ret_coalesced_cqe[j];
+
data[i++] = packets;
data[i++] = bytes;
data[i++] = xdp_drop;
@@ -296,7 +326,7 @@ static void mana_get_ethtool_stats(struct net_device *ndev,
}
for (q = 0; q < num_queues; q++) {
- tx_stats = &apc->tx_qp[q]->txq.stats;
+ tx_stats = &apc->txq_stats[q];
do {
start = u64_stats_fetch_begin(&tx_stats->syncp);
@@ -648,52 +678,126 @@ static int mana_set_coalesce(struct net_device *ndev,
return 0;
}
-/* mana_set_channels - change the number of queues on a port
- *
- * Returns -EBUSY if RDMA holds the vport with EQs sized to the
- * current num_queues.
- */
static int mana_set_channels(struct net_device *ndev,
struct ethtool_channels *channels)
{
struct mana_port_context *apc = netdev_priv(ndev);
unsigned int new_count = channels->combined_count;
- unsigned int old_count = apc->num_queues;
+ struct mana_port_context *scratch;
+ struct mana_qset newq, oldq, freshq;
int err;
- /* Set channel_changing to block RDMA from grabbing the vport
- * during the detach/attach window. mana_cfg_vport() checks
- * this flag under vport_mutex and returns -EBUSY if set.
+ if (new_count < 1 || new_count > apc->max_queues) {
+ netdev_err(ndev, "Invalid combined_count %u (max %u)\n",
+ new_count, apc->max_queues);
+ return -EINVAL;
+ }
+
+ if (new_count == apc->num_queues)
+ return 0;
+
+ /* Resize rxqs while down: mana_open() does not recreate the port
+ * context. RDMA must not own the vport while num_queues changes.
*/
mutex_lock(&apc->vport_mutex);
- if (!apc->port_is_up && apc->vport_use_count) {
+ if (!apc->port_is_up) {
+ struct mana_rxq **rxqs;
+
+ if (apc->vport_use_count) {
+ mutex_unlock(&apc->vport_mutex);
+ return -EBUSY;
+ }
+
+ rxqs = kzalloc_objs(struct mana_rxq *, new_count);
+ if (!rxqs) {
+ mutex_unlock(&apc->vport_mutex);
+ return -ENOMEM;
+ }
+
+ kfree(apc->rxqs);
+ apc->rxqs = rxqs;
+ apc->num_queues = new_count;
+ mutex_unlock(&apc->vport_mutex);
+ return 0;
+ }
+
+ /* The Ethernet port already holds a vport reference; exclude RDMA
+ * through failure cleanup.
+ */
+ if (apc->channel_changing) {
mutex_unlock(&apc->vport_mutex);
return -EBUSY;
}
apc->channel_changing = true;
mutex_unlock(&apc->vport_mutex);
- err = mana_pre_alloc_rxbufs(apc, ndev->mtu, new_count);
- if (err) {
- netdev_err(ndev, "Insufficient memory for new allocations");
+ scratch = mana_qset_scratch_alloc(apc);
+ if (!scratch) {
+ err = -ENOMEM;
goto clear_flag;
}
- err = mana_detach(ndev, false);
- if (err) {
- netdev_err(ndev, "mana_detach failed: %d\n", err);
- goto out;
+ if (new_count < apc->num_queues) {
+ struct mana_qset tailq;
+
+ err = mana_split_qset(apc, scratch, new_count, &newq, &tailq);
+ if (err)
+ goto free_scratch;
+
+ err = mana_publish_qset(apc, &newq, &oldq);
+ if (err) {
+ /* Discard containers only; their queues still belong to
+ * the old set.
+ */
+ mana_discard_split(&newq, &tailq);
+ goto free_scratch;
+ }
+
+ /* Wait for ndo_select_queue() readers of oldq.indir_table. Free
+ * only containers; the queues belong to the kept set or tail.
+ */
+ synchronize_net();
+
+ kfree(oldq.tx_qp);
+ kfree(oldq.rxqs);
+ kfree(oldq.indir_table);
+ kfree(oldq.rxobj_table);
+
+ mana_free_qset(scratch, &tailq);
+ goto free_scratch;
}
- apc->num_queues = new_count;
- err = mana_attach(ndev);
+ err = mana_grow_qset(apc, scratch, new_count, &newq, &freshq);
+ if (err)
+ goto free_scratch;
+
+ err = mana_publish_qset(apc, &newq, &oldq);
if (err) {
- apc->num_queues = old_count;
- netdev_err(ndev, "mana_attach failed: %d\n", err);
+ /* Free only the new queues, then discard the merged containers.
+ */
+ mana_free_qset(scratch, &freshq);
+ mana_discard_grow(&newq);
+ goto free_scratch;
}
-out:
- mana_pre_dealloc_rxbufs(apc);
+ /* Wait for ndo_select_queue() readers of oldq.indir_table. All queues
+ * are now live in newq; free only the old and fresh containers.
+ */
+ synchronize_net();
+
+ kfree(oldq.tx_qp);
+ kfree(oldq.rxqs);
+ kfree(oldq.indir_table);
+ kfree(oldq.rxobj_table);
+ kfree(freshq.tx_qp);
+ kfree(freshq.rxqs);
+
+ /* No retirement runs to publish the new queues' debugfs nodes. */
+ mana_qset_debugfs_publish(apc);
+
+free_scratch:
+ mana_publish_close_if_needed(apc);
+ mana_qset_scratch_free(scratch);
clear_flag:
mutex_lock(&apc->vport_mutex);
apc->channel_changing = false;
@@ -720,13 +824,11 @@ static int mana_set_ringparam(struct net_device *ndev,
struct netlink_ext_ack *extack)
{
struct mana_port_context *apc = netdev_priv(ndev);
+ struct mana_port_context *scratch;
+ struct mana_qset newq, oldq;
u32 new_tx, new_rx;
- u32 old_tx, old_rx;
int err;
- old_tx = apc->tx_queue_size;
- old_rx = apc->rx_queue_size;
-
if (ring->tx_pending < MIN_TX_BUFFERS_PER_QUEUE) {
NL_SET_ERR_MSG_FMT(extack, "tx:%d less than the min:%d", ring->tx_pending,
MIN_TX_BUFFERS_PER_QUEUE);
@@ -744,32 +846,56 @@ static int mana_set_ringparam(struct net_device *ndev,
netdev_info(ndev, "Using nearest power of 2 values for Txq:%d Rxq:%d\n",
new_tx, new_rx);
- /* pre-allocating new buffers to prevent failures in mana_attach() later */
- apc->rx_queue_size = new_rx;
- err = mana_pre_alloc_rxbufs(apc, ndev->mtu, apc->num_queues);
- apc->rx_queue_size = old_rx;
- if (err) {
- netdev_err(ndev, "Insufficient memory for new allocations\n");
- return err;
+ if (new_rx == apc->rx_queue_size && new_tx == apc->tx_queue_size)
+ return 0;
+
+ if (!apc->port_is_up) {
+ apc->rx_queue_size = new_rx;
+ apc->tx_queue_size = new_tx;
+ return 0;
}
- err = mana_detach(ndev, false);
- if (err) {
- netdev_err(ndev, "mana_detach failed: %d\n", err);
- goto out;
+ /* Exclude RDMA through failure cleanup, which may release the vport. */
+ mutex_lock(&apc->vport_mutex);
+ if (apc->channel_changing) {
+ mutex_unlock(&apc->vport_mutex);
+ return -EBUSY;
+ }
+ apc->channel_changing = true;
+ mutex_unlock(&apc->vport_mutex);
+
+ scratch = mana_qset_scratch_alloc(apc);
+ if (!scratch) {
+ err = -ENOMEM;
+ goto clear_flag;
}
- apc->tx_queue_size = new_tx;
- apc->rx_queue_size = new_rx;
+ err = mana_alloc_qset(apc, scratch, new_rx, new_tx,
+ apc->priv_flags, apc->configured_mtu,
+ apc->bpf_prog, &newq);
+ if (err) {
+ NL_SET_ERR_MSG_FMT(extack, "failed to change ring params: %d",
+ err);
+ goto free_scratch;
+ }
- err = mana_attach(ndev);
+ err = mana_publish_qset(apc, &newq, &oldq);
if (err) {
- netdev_err(ndev, "mana_attach failed: %d\n", err);
- apc->tx_queue_size = old_tx;
- apc->rx_queue_size = old_rx;
+ NL_SET_ERR_MSG_FMT(extack, "failed to change ring params: %d",
+ err);
+ mana_free_qset(scratch, &newq);
+ goto free_scratch;
}
-out:
- mana_pre_dealloc_rxbufs(apc);
+
+ mana_free_qset(scratch, &oldq);
+
+free_scratch:
+ mana_publish_close_if_needed(apc);
+ mana_qset_scratch_free(scratch);
+clear_flag:
+ mutex_lock(&apc->vport_mutex);
+ apc->channel_changing = false;
+ mutex_unlock(&apc->vport_mutex);
return err;
}
@@ -799,7 +925,8 @@ static int mana_set_priv_flags(struct net_device *ndev, u32 priv_flags)
{
struct mana_port_context *apc = netdev_priv(ndev);
u32 changed = apc->priv_flags ^ priv_flags;
- u32 old_priv_flags = apc->priv_flags;
+ struct mana_port_context *scratch;
+ struct mana_qset newq, oldq;
int err = 0;
if (!changed)
@@ -809,54 +936,50 @@ static int mana_set_priv_flags(struct net_device *ndev, u32 priv_flags)
if (priv_flags & ~GENMASK(MANA_PRIV_FLAG_MAX - 1, 0))
return -EINVAL;
- apc->priv_flags = priv_flags;
-
- if (changed & BIT(MANA_PRIV_FLAG_USE_FULL_PAGE_RXBUF)) {
- if (!apc->port_is_up)
- return 0;
-
- /* If XDP is attached or MTU is jumbo, single-buffer-per-page
- * is already forced regardless of this flag. Skip the
- * expensive detach/attach cycle since nothing changes.
- */
- if (ndev->mtu + MANA_RXBUF_PAD > PAGE_SIZE / 2 ||
- mana_xdp_get(apc))
- return 0;
+ /* Skip rebuilding when full-page RX is already required. */
+ if (!(changed & BIT(MANA_PRIV_FLAG_USE_FULL_PAGE_RXBUF)) ||
+ !apc->port_is_up ||
+ ndev->mtu + MANA_RXBUF_PAD > PAGE_SIZE / 2 ||
+ mana_xdp_get(apc)) {
+ apc->priv_flags = priv_flags;
+ return 0;
+ }
- /* Block RDMA from grabbing the vport during detach/attach */
- mutex_lock(&apc->vport_mutex);
- apc->channel_changing = true;
+ mutex_lock(&apc->vport_mutex);
+ if (apc->channel_changing) {
mutex_unlock(&apc->vport_mutex);
+ return -EBUSY;
+ }
+ apc->channel_changing = true;
+ mutex_unlock(&apc->vport_mutex);
- err = mana_pre_alloc_rxbufs(apc, ndev->mtu, apc->num_queues);
- if (err) {
- netdev_err(ndev,
- "Insufficient memory for new allocations\n");
- apc->priv_flags = old_priv_flags;
- goto clear_flag;
- }
+ scratch = mana_qset_scratch_alloc(apc);
+ if (!scratch) {
+ err = -ENOMEM;
+ goto clear_flag;
+ }
- err = mana_detach(ndev, false);
- if (err) {
- netdev_err(ndev, "mana_detach failed: %d\n", err);
- apc->priv_flags = old_priv_flags;
- goto out;
- }
+ err = mana_alloc_qset(apc, scratch, apc->rx_queue_size,
+ apc->tx_queue_size, priv_flags,
+ apc->configured_mtu, apc->bpf_prog, &newq);
+ if (err)
+ goto free_scratch;
- err = mana_attach(ndev);
- if (err) {
- netdev_err(ndev, "mana_attach failed: %d\n", err);
- apc->priv_flags = old_priv_flags;
- }
+ err = mana_publish_qset(apc, &newq, &oldq);
+ if (err) {
+ mana_free_qset(scratch, &newq);
+ goto free_scratch;
}
-out:
- mana_pre_dealloc_rxbufs(apc);
+ mana_free_qset(scratch, &oldq);
+
+free_scratch:
+ mana_publish_close_if_needed(apc);
+ mana_qset_scratch_free(scratch);
clear_flag:
mutex_lock(&apc->vport_mutex);
apc->channel_changing = false;
mutex_unlock(&apc->vport_mutex);
-
return err;
}
@@ -871,7 +994,8 @@ const struct ethtool_ops mana_ethtool_ops = {
.op_needs_rtnl = ETHTOOL_OP_NEEDS_RTNL_SCHANNELS |
ETHTOOL_OP_NEEDS_RTNL_SRINGPARAM |
ETHTOOL_OP_NEEDS_RTNL_SPFLAGS |
- ETHTOOL_OP_NEEDS_RTNL_GLINK,
+ ETHTOOL_OP_NEEDS_RTNL_GLINK |
+ ETHTOOL_OP_NEEDS_RTNL_RSS,
.get_ethtool_stats = mana_get_ethtool_stats,
.get_sset_count = mana_get_sset_count,
.get_strings = mana_get_strings,
diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h
index 308950f9b54b0..666565ffb26aa 100644
--- a/include/net/mana/gdma.h
+++ b/include/net/mana/gdma.h
@@ -686,6 +686,11 @@ enum {
/* Driver supports non-contiguous queue buffers */
#define GDMA_DRV_CAP_FLAG_1_NON_CONTIGUOUS_BUFFERS BIT(30)
+/* Resize failures are handled in-driver; a failed rollback still needs
+ * recovery.
+ */
+#define GDMA_DRV_CAP_FLAG_1_SELF_RECOVERY_ON_QUEUE_RESIZE_FAILURE BIT(31)
+
#define GDMA_DRV_CAP_FLAGS1 \
(GDMA_DRV_CAP_FLAG_1_EQ_SHARING_MULTI_VPORT | \
GDMA_DRV_CAP_FLAG_1_NAPI_WKDONE_FIX | \
@@ -703,7 +708,8 @@ enum {
GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECOVERY | \
GDMA_DRV_CAP_FLAG_1_EQ_MSI_UNSHARE_MULTI_VPORT | \
GDMA_DRV_CAP_FLAG_1_DYN_INTERRUPT_MODERATION | \
- GDMA_DRV_CAP_FLAG_1_NON_CONTIGUOUS_BUFFERS)
+ GDMA_DRV_CAP_FLAG_1_NON_CONTIGUOUS_BUFFERS | \
+ GDMA_DRV_CAP_FLAG_1_SELF_RECOVERY_ON_QUEUE_RESIZE_FAILURE)
#define GDMA_DRV_CAP_FLAGS2 0
diff --git a/include/net/mana/mana.h b/include/net/mana/mana.h
index 83b7eff4646ea..cc936ae97a031 100644
--- a/include/net/mana/mana.h
+++ b/include/net/mana/mana.h
@@ -102,7 +102,7 @@ struct mana_stats_rx {
u64 pkt_len0_err;
u64 coalesced_cqe[MANA_CQE_COAL_PKTS_8 - 1];
struct u64_stats_sync syncp;
-};
+} ____cacheline_aligned_in_smp;
struct mana_stats_tx {
u64 packets;
@@ -117,7 +117,7 @@ struct mana_stats_tx {
u64 csum_partial;
u64 mana_map_err;
struct u64_stats_sync syncp;
-};
+} ____cacheline_aligned_in_smp;
struct mana_txq {
struct gdma_queue *gdma_sq;
@@ -143,7 +143,12 @@ struct mana_txq {
bool napi_initialized;
- struct mana_stats_tx stats;
+ u32 reset_gen;
+
+ /* Suppress completion wakeups on the replacement's netdev queue. */
+ bool retiring;
+
+ struct mana_stats_tx *stats;
};
/* skb data and frags dma mappings */
@@ -405,7 +410,16 @@ struct mana_rxq {
u32 buf_index;
- struct mana_stats_rx stats;
+ /* Port-owned live slot; use mana_rxq_stats() to select the writer's
+ * slot.
+ */
+ struct mana_stats_rx *stats;
+
+ /* Set under RTNL before another queue takes over this index. */
+ bool retiring;
+
+ /* Folded under RTNL after drain-stat writers quiesce. */
+ struct mana_stats_rx drain_stats;
struct bpf_prog __rcu *bpf_prog;
struct xdp_rxq_info xdp_rxq;
@@ -537,6 +551,12 @@ struct mana_context {
u8 bm_hostmode;
struct mana_ethtool_hc_stats hc_stats;
+
+ /* Generation of successful resets issued by mana_drain_txqs(), under
+ * RTNL.
+ */
+ u32 reset_gen;
+
struct workqueue_struct *per_port_queue_reset_wq;
/* Workqueue for querying hardware stats */
struct delayed_work gf_stats_work;
@@ -559,7 +579,9 @@ struct mana_port_context {
u8 mac_addr[ETH_ALEN];
+ /* Port-owned EQ pool: max_queues slots, num_eqs populated. */
struct mana_eq *eqs;
+ unsigned int num_eqs;
struct dentry *mana_eqs_debugfs;
enum TRI_STATE rss_state;
@@ -600,9 +622,21 @@ struct mana_port_context {
unsigned int max_queues;
unsigned int num_queues;
+ /* Port-lifetime arrays with max_queues slots. Live RX queues write
+ * rxq_stats[]; teardown and rollback fold drain_stats into
+ * rxq_stats_ret[] under RTNL. Readers sum both.
+ */
+ struct mana_stats_rx *rxq_stats;
+ struct mana_stats_rx *rxq_stats_ret;
+ struct mana_stats_tx *txq_stats;
+
unsigned int rx_queue_size;
unsigned int tx_queue_size;
+ /* MTU used to size RX buffers, independent of ndev->mtu during a swap.
+ */
+ int configured_mtu;
+
mana_handle_t port_handle;
mana_handle_t pf_filter_handle;
@@ -610,12 +644,17 @@ struct mana_port_context {
struct mutex vport_mutex;
int vport_use_count;
- /* Set by mana_set_channels() under vport_mutex to block RDMA
- * from grabbing the vport during the detach/attach window.
- * Checked by mana_cfg_vport() when called from the RDMA path.
- */
+ /* Exclude RDMA during reconfiguration; protected by vport_mutex. */
bool channel_changing;
+ /* Caller must close the port after releasing the unpublished set. */
+ bool publish_dead_end;
+
+ /* Carrier lowered by failed rollback; cleared on reopen or a link
+ * event.
+ */
+ bool carrier_forced_off;
+
/* Net shaper handle*/
struct net_shaper_handle handle;
@@ -661,6 +700,27 @@ struct mana_port_context {
u32 steer_cqe_coalescing;
};
+struct mana_qset {
+ struct mana_tx_qp **tx_qp;
+ struct mana_rxq **rxqs;
+
+ u32 *indir_table;
+ u32 indir_table_sz;
+ mana_handle_t *rxobj_table;
+ mana_handle_t default_rxobj;
+
+ unsigned int num_queues;
+ unsigned int rx_queue_size;
+ unsigned int tx_queue_size;
+ u32 priv_flags;
+
+ int mtu;
+ struct bpf_prog *bpf_prog;
+
+ /* Notify the core only after this set is published. */
+ bool rxfh_indir_lost;
+};
+
netdev_tx_t mana_start_xmit(struct sk_buff *skb, struct net_device *ndev);
int mana_config_rss(struct mana_port_context *ac, enum TRI_STATE rx,
bool update_hash, bool update_tab);
@@ -670,6 +730,33 @@ int mana_alloc_queues(struct net_device *ndev);
int mana_attach(struct net_device *ndev);
int mana_detach(struct net_device *ndev, bool from_close);
+struct mana_port_context *
+mana_qset_scratch_alloc(struct mana_port_context *apc);
+void mana_qset_scratch_free(struct mana_port_context *scratch);
+static inline struct mana_stats_rx *mana_rxq_stats(struct mana_rxq *rxq)
+{
+ return READ_ONCE(rxq->retiring) ? &rxq->drain_stats : rxq->stats;
+}
+
+int mana_alloc_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch,
+ unsigned int rx_queue_size, unsigned int tx_queue_size,
+ u32 priv_flags, int mtu, struct bpf_prog *bpf_prog,
+ struct mana_qset *out);
+int mana_split_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch, unsigned int new_count,
+ struct mana_qset *out_new, struct mana_qset *out_tail);
+void mana_discard_split(struct mana_qset *newq, struct mana_qset *tailq);
+int mana_grow_qset(struct mana_port_context *apc,
+ struct mana_port_context *scratch, unsigned int new_count,
+ struct mana_qset *out_new, struct mana_qset *out_fresh);
+void mana_discard_grow(struct mana_qset *newq);
+int mana_publish_qset(struct mana_port_context *apc, struct mana_qset *newq,
+ struct mana_qset *out_old);
+void mana_publish_close_if_needed(struct mana_port_context *apc);
+void mana_free_qset(struct mana_port_context *scratch, struct mana_qset *qset);
+void mana_qset_debugfs_publish(struct mana_port_context *apc);
+
void mana_dim_change(struct mana_cq *cq, bool enable);
int mana_probe(struct gdma_dev *gd, bool resuming);
@@ -685,6 +772,8 @@ u32 mana_run_xdp(struct net_device *ndev, struct mana_rxq *rxq,
struct xdp_buff *xdp, void *buf_va, uint pkt_len);
struct bpf_prog *mana_xdp_get(struct mana_port_context *apc);
void mana_chn_setxdp(struct mana_port_context *apc, struct bpf_prog *prog);
+struct bpf_prog *mana_chn_xdp_peek(struct mana_port_context *apc);
+void mana_chn_xdp_release(struct bpf_prog *prog, unsigned int num_queues);
int mana_bpf(struct net_device *ndev, struct netdev_bpf *bpf);
int mana_query_gf_stats(struct mana_context *ac);
int mana_query_link_cfg(struct mana_port_context *apc);
|
| 3/2 | 2026/09/09 22:47 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 34774 output: 35 thoughts: 319 Thoughts: **Analyzing the Patch for Fuzzing Suitability** Okay, let's dive into this patch and figure out if it's a good candidate for fuzzing, keeping the guidelines in mind. I need to understand the scope and implications before I even start considering a fuzzing strategy. First, let's see what's changed. The patch touches: `drivers/net/ethernet/microsoft/mana/mana_bpf.c`, `drivers/net/ethernet/microsoft/mana/mana_en.c`, `drivers/net/ethernet/microsoft/mana/mana_ethtool.c`, and a couple of include files: `include/net/mana/gdma.h`, `include/net/mana/mana.h`. The "mana" part is what jumps out. I don't instantly recognize it. So, what exactly *is* "mana"? Time to go hunting in the Kconfig files or the code itself to get context. Okay, after a quick look, it seems like `drivers/net/ethernet/microsoft/mana/` is the directory for the Microsoft Azure Network Adapter (MANA) Ethernet driver. Now we're getting somewhere. The next critical question is whether this is a hardware-specific driver, meaning it *requires* a specific PCIe card or is emulated in a standard QEMU environment? I need to check the fuzzing guidelines and be very aware of structurally unreachable hardware. Let me quickly refresh that section... Aha! Section 2 states that we should skip things like vendor-specific PCIe switches, SmartNICs, or GPU drivers that need specialized hardware not emulated in standard QEMU. Therefore, the key point is whether Microsoft MANA is a vendor-specific driver that needs physical hardware or can it run in a more standard QEMU environment? Now, I need to check the Kconfig files and kernel documentation to see whether MANA requires specific hardware to operate. I'll search the kernel repo for MANA Kconfig settings. |
| 4/2 | 2026/09/09 22:47 | tool | grepper |
0mArgs:{
"Expression": "config MICROSOFT_MANA",
"PathPrefix": "drivers/net/ethernet/microsoft/"
}Results: {
"Output": "drivers/net/ethernet/microsoft/Kconfig=16=if NET_VENDOR_MICROSOFT\ndrivers/net/ethernet/microsoft/Kconfig-17-\ndrivers/net/ethernet/microsoft/Kconfig:18:config MICROSOFT_MANA\ndrivers/net/ethernet/microsoft/Kconfig-19-\ttristate \"Microsoft Azure Network Adapter (MANA) support\"\n"
} |
| 5/2 | 2026/09/09 22:47 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 35657 output: 37 thoughts: 17 Thoughts: Here's a summary of my thought process, as if I were analyzing that Kconfig file: **Diving into the Microsoft Ethernet Driver Configuration** Okay, time to dig into `drivers/net/ethernet/microsoft/Kconfig`. Right, this Kconfig file will define the configuration options for the Microsoft Ethernet drivers. I'm expecting to see the usual structure: features, hardware support, maybe some debugging options, and dependencies. Given that it's Microsoft, I'm curious to see if they've got some proprietary options in here. Probably specific hardware support, given the focus. I'll need to look for the various `config` symbols, their associated descriptions, and any `depends on` or `select` statements to get a handle on the relationships between these different features. Gotta pay close attention to the default values and help text; that'll tell me a lot about their design philosophy and intended use cases. I'll also be searching for how they handle different network controllers and interfaces. Any mentions of specific MAC addresses or PHYs? Let's see if this code is designed to interface with some proprietary hardware. Let's get reading! |
| 6/2 | 2026/09/09 22:47 | tool | read-file |
0mArgs:{
"File": "drivers/net/ethernet/microsoft/Kconfig",
"FirstLine": 1,
"LineCount": 40
}Results: {
"Contents": " 1:\t#\n 2:\t# Microsoft Azure network device configuration\n 3:\t#\n 4:\t\n 5:\tconfig NET_VENDOR_MICROSOFT\n 6:\t\tbool \"Microsoft Network Devices\"\n 7:\t\tdefault y\n 8:\t\thelp\n 9:\t\t If you have a network (Ethernet) device belonging to this class, say Y.\n 10:\t\n 11:\t\t Note that the answer to this question doesn't directly affect the\n 12:\t\t kernel: saying N will just cause the configurator to skip the\n 13:\t\t question about Microsoft network devices. If you say Y, you will be\n 14:\t\t asked for your specific device in the following question.\n 15:\t\n 16:\tif NET_VENDOR_MICROSOFT\n 17:\t\n 18:\tconfig MICROSOFT_MANA\n 19:\t\ttristate \"Microsoft Azure Network Adapter (MANA) support\"\n 20:\t\tdepends on PCI_MSI\n 21:\t\tdepends on X86_64 || (ARM64 \u0026\u0026 !CPU_BIG_ENDIAN)\n 22:\t\tdepends on PCI_HYPERV\n 23:\t\tselect AUXILIARY_BUS\n 24:\t\tselect DIMLIB\n 25:\t\tselect PAGE_POOL\n 26:\t\tselect NET_SHAPER\n 27:\t\thelp\n 28:\t\t This driver supports Microsoft Azure Network Adapter (MANA).\n 29:\t\t So far, the driver is only supported on X86_64.\n 30:\t\n 31:\t\t To compile this driver as a module, choose M here.\n 32:\t\t The module will be called mana.\n 33:\t\n 34:\tendif #NET_VENDOR_MICROSOFT\n"
} |
| 7/2 | 2026/09/09 22:47 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 36397 output: 33 thoughts: 63 Thoughts: **My Analysis of MICROSOFT_MANA Dependencies** Alright, let's break this down. From the initial information, it seems `MICROSOFT_MANA` is fundamentally tied to the Azure environment, specifically the Azure Network Adapter (MANA) itself. This connection is exclusively within Azure Virtual Machines (VMs). The dependency on `PCI_HYPERV` strongly suggests that the MANA functionality I'm interested in uses a PCI bus, either simulated through Hyper-V virtualization or directly from a physical MANA NIC. My next immediate step will be to dive deeper into the MANA probing mechanisms. I need to understand how the system identifies the bus it's running on, how it interacts with it, and what diagnostic data it pulls when running. I need to understand what the bus interaction is, which is key to seeing if it's a physical or emulated PCI bus. This should clarify how the software determines its operational context and confirm the expected dependency path. This will lead to a very quick understanding of the system's architecture. |
| 8/2 | 2026/09/09 22:47 | tool | grepper |
0mArgs:{
"Expression": "mana_probe",
"PathPrefix": "drivers/net/ethernet/microsoft/mana"
}Results: {
"Output": "drivers/net/ethernet/microsoft/mana/gdma_main.c=179=static int mana_gd_query_max_resources(struct pci_dev *pdev)\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-282-\ndrivers/net/ethernet/microsoft/mana/gdma_main.c:283:\t/* Cap to the same limit used by mana_probe() for port instantiation,\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-284-\t * so MSI-X and queue budgeting matches the actual port count.\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c=2499=static int mana_gd_probe(struct pci_dev *pdev, const struct pci_device_id *ent)\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2554-\ndrivers/net/ethernet/microsoft/mana/gdma_main.c:2555:\terr = mana_probe(\u0026gc-\u003emana, false);\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2556-\tif (err)\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2593-\t * Hardware could be in recovery mode and the HWC returns TIMEDOUT or\ndrivers/net/ethernet/microsoft/mana/gdma_main.c:2594:\t * EPROTO from mana_gd_setup(), mana_probe() or mana_rdma_probe(), or\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2595-\t * we received a hardware reset event over HWC interrupt. In this case,\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c=2659=int mana_gd_resume(struct pci_dev *pdev)\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2667-\ndrivers/net/ethernet/microsoft/mana/gdma_main.c:2668:\terr = mana_probe(\u0026gc-\u003emana, true);\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2669-\tif (err)\n--\ndrivers/net/ethernet/microsoft/mana/mana_en.c=4786=int mana_detach(struct net_device *ndev, bool from_close)\n--\ndrivers/net/ethernet/microsoft/mana/mana_en.c-4820-\ndrivers/net/ethernet/microsoft/mana/mana_en.c:4821:static int mana_probe_port(struct mana_context *ac, int port_idx,\ndrivers/net/ethernet/microsoft/mana/mana_en.c-4822-\t\t\t struct net_device **ndev_storage)\n--\ndrivers/net/ethernet/microsoft/mana/mana_en.c=5069=static void mana_gf_stats_work_handler(struct work_struct *work)\n--\ndrivers/net/ethernet/microsoft/mana/mana_en.c-5091-\ndrivers/net/ethernet/microsoft/mana/mana_en.c:5092:int mana_probe(struct gdma_dev *gd, bool resuming)\ndrivers/net/ethernet/microsoft/mana/mana_en.c-5093-{\n--\ndrivers/net/ethernet/microsoft/mana/mana_en.c-5166-\t\tfor (i = 0; i \u003c ac-\u003enum_ports; i++) {\ndrivers/net/ethernet/microsoft/mana/mana_en.c:5167:\t\t\terr = mana_probe_port(ac, i, \u0026ac-\u003eports[i]);\ndrivers/net/ethernet/microsoft/mana/mana_en.c-5168-\t\t\t/* Log the port for which the probe failed, stop probing\n"
} |
| 9/2 | 2026/09/09 22:47 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 37681 output: 36 thoughts: 20 Thoughts: Here's a summary of my immediate thought process: **MANA Investigation: Device IDs and `mana_gd_probe`** Okay, first things first, let's dive into the guts of this MANA issue. I need to quickly establish the core connection: I'm thinking about the device itself and how it's being recognized. My initial approach is to immediately check the `mana_gd_probe` function. This is usually the entry point for the MANA driver to initialize, so any issues during early device detection would likely surface here. I want to see how the system is trying to identify the device. Then, I'm going to cross-reference that with the PCI device ID table. This is the crucial lookup table where the driver identifies the specific hardware based on its Vendor ID and Device ID. I need to verify that the device's ID is correctly listed and that the driver knows what to do with it. Any misconfiguration or missing entries in this table can lead to all sorts of recognition failures. Essentially, I'm looking for a mismatch or omission. It's a fundamental debugging step. I'll need to confirm that the correct IDs are programmed, or at least that the device IDs align with the driver's expectations. |
| 10/2 | 2026/09/09 22:47 | tool | grepper |
0mArgs:{
"Expression": "pci_device_id",
"PathPrefix": "drivers/net/ethernet/microsoft/mana"
}Results: {
"Output": "drivers/net/ethernet/microsoft/mana/gdma_main.c=2494=static bool mana_is_pf(unsigned short dev_id)\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2498-\ndrivers/net/ethernet/microsoft/mana/gdma_main.c:2499:static int mana_gd_probe(struct pci_dev *pdev, const struct pci_device_id *ent)\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2500-{\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c=2698=static int mana_sriov_configure(struct pci_dev *pdev, int numvfs)\n--\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2718-\ndrivers/net/ethernet/microsoft/mana/gdma_main.c:2719:static const struct pci_device_id mana_id_table[] = {\ndrivers/net/ethernet/microsoft/mana/gdma_main.c-2720-\t{ PCI_DEVICE(PCI_VENDOR_ID_MICROSOFT, MANA_PF_DEVICE_ID) },\n"
} |
| 11/2 | 2026/09/09 22:47 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 38317 output: 46 thoughts: 15 Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience: **Analyzing the `mana_id_table`** Right, let's dive in. First, I need to get a handle on the PCI IDs within the `mana_id_table`. I'm thinking, this table likely contains crucial information for identifying and managing hardware components, specifically those utilizing PCI interfaces. The goal here is probably to establish a correspondence between device IDs and relevant drivers or configurations within the system. Understanding this mapping is fundamental for device detection, resource allocation, and overall system functionality. I'll need to scrutinize the structure and contents to grasp how those IDs are used, and what insights they might provide. Perhaps this table is for a specific system or architecture. I will investigate. |
| 12/2 | 2026/09/09 22:47 | tool | read-file |
0mArgs:{
"File": "drivers/net/ethernet/microsoft/mana/gdma_main.c",
"FirstLine": 2715,
"LineCount": 20
}Results: {
"Contents": "2715:\t\n2716:\t\treturn err ? err : numvfs;\n2717:\t}\n2718:\t\n2719:\tstatic const struct pci_device_id mana_id_table[] = {\n2720:\t\t{ PCI_DEVICE(PCI_VENDOR_ID_MICROSOFT, MANA_PF_DEVICE_ID) },\n2721:\t\t{ PCI_DEVICE(PCI_VENDOR_ID_MICROSOFT, MANA_PF2_DEVICE_ID) },\n2722:\t\t{ PCI_DEVICE(PCI_VENDOR_ID_MICROSOFT, MANA_VF_DEVICE_ID) },\n2723:\t\t{ }\n2724:\t};\n2725:\t\n2726:\tstatic struct pci_driver mana_driver = {\n2727:\t\t.name\t\t= \"mana\",\n2728:\t\t.id_table\t= mana_id_table,\n2729:\t\t.probe\t\t= mana_gd_probe,\n2730:\t\t.remove\t\t= mana_gd_remove,\n2731:\t\t.suspend\t= mana_gd_suspend,\n2732:\t\t.resume\t\t= mana_gd_resume,\n2733:\t\t.shutdown\t= mana_gd_shutdown,\n2734:\t\t.sriov_configure = mana_sriov_configure,\n"
} |
| 13/2 | 2026/09/09 22:47 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 38894 output: 113 thoughts: 532 Thoughts: **WorthFuzzing Decision for Microsoft MANA Driver Changes** Okay, let's break this down. My analysis indicates that this commit modifies code exclusively within the Microsoft Azure Network Adapter (MANA) driver. Specifically, the changes are located within the `drivers/net/ethernet/microsoft/mana/` directory and related `include/net/mana/` headers. The crux of the matter is the MANA driver's hardware dependency. It requires specific Microsoft Azure MANA hardware, including potentially GDMA hardware and Hyper-V PCI devices. The kicker? These devices are *not* emulated in the standard QEMU or GCE virtualized environments that syzkaller utilizes for fuzzing. This means the modified code paths – the packet transmission/reception routines, XDP handling, ethtool integration, and queue management – are, for all intents and purposes, structurally unreachable within our fuzzing setup. There's simply no way to trigger those code paths in a meaningful way. Given this, according to the negative criteria, this is a clear case for setting `WorthFuzzing=false`. Consequently, I'll set `FocusSymbols` to null, or empty, and also `EnableConfigs` to null, or empty as well, as there's no need to target specific symbols or configurations here. Now, let's get those results updated. Calling `set-results`. |
| 14/2 | 2026/09/09 22:47 | tool | set-results |
0mArgs:{
"EnableConfigs": [],
"FocusSymbols": [],
"Reasoning": "The patch modifies the Microsoft Azure Network Adapter (MANA) Ethernet driver (drivers/net/ethernet/microsoft/mana/ and include/net/mana/). This driver targets vendor-specific hardware/SmartNICs on Microsoft Azure / Hyper-V environments and is not emulated in standard QEMU or virtualized fuzzing environments. As such, the modified code is structurally unreachable during fuzzing.",
"WorthFuzzing": false
}Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"Reasoning": "The patch modifies the Microsoft Azure Network Adapter (MANA) Ethernet driver (drivers/net/ethernet/microsoft/mana/ and include/net/mana/). This driver targets vendor-specific hardware/SmartNICs on Microsoft Azure / Hyper-V environments and is not emulated in standard QEMU or virtualized fuzzing environments. As such, the modified code is structurally unreachable during fuzzing.",
"WorthFuzzing": false
} |
| 15/1 | 2026/09/09 22:47 | action | If |
0mArgs:{
"WorthFuzzing": false
}Results: null |
| Total Calls | Total Tokens | Avg Tokens | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|
| Total Calls | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|