| Seq | Timestamp | Type | Name | Duration |
|---|---|---|---|---|
| 0/0 | 2026/06/29 20:22 | flow | patch-triage |
0mArgs:null Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"Reasoning": "The patch exclusively modifies the Broadcom NetXtreme-C/E (bnxt) hardware driver to add TLS offload support. Since there is no software emulation for this specific hardware in virtualized environments like QEMU or GCE, the modified code paths are unreachable during standard fuzzing.",
"WorthFuzzing": false
} |
| 1/1 | 2026/06/29 20:22 | action | read-patch-diff |
0mArgs:null Results: {
"PatchDiff": "commit 4ea71aee837cf1a383bbb583beebf054a50cf55e\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Mon Jun 29 20:22:20 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/drivers/net/ethernet/broadcom/Kconfig b/drivers/net/ethernet/broadcom/Kconfig\nindex f0bac0dd143900..b33b66f038b882 100644\n--- a/drivers/net/ethernet/broadcom/Kconfig\n+++ b/drivers/net/ethernet/broadcom/Kconfig\n@@ -255,6 +255,15 @@ config BNXT_HWMON\n \t Say Y if you want to expose the thermal sensor data on NetXtreme-C/E\n \t devices, via the hwmon sysfs interface.\n \n+config BNXT_TLS\n+\tbool \"Broadcom NetXtreme-C/E TLS offload support\"\n+\tdefault y\n+\tdepends on BNXT \u0026\u0026 TLS_DEVICE\n+\tdepends on TLS=y || BNXT=m\n+\thelp\n+\t Say Y if you want to enable Transport Layer Security (TLS) hardware\n+\t encryption and decryption offload on supported NetXtreme-C/E devices.\n+\n config BNGE\n \ttristate \"Broadcom ThorUltra Ethernet device support\"\n \tdepends on PCI\ndiff --git a/drivers/net/ethernet/broadcom/bnxt/Makefile b/drivers/net/ethernet/broadcom/bnxt/Makefile\nindex debef78c8b6d64..88e68248aad420 100644\n--- a/drivers/net/ethernet/broadcom/bnxt/Makefile\n+++ b/drivers/net/ethernet/broadcom/bnxt/Makefile\n@@ -5,3 +5,4 @@ bnxt_en-y := bnxt.o bnxt_hwrm.o bnxt_sriov.o bnxt_ethtool.o bnxt_dcb.o bnxt_ulp.\n bnxt_en-$(CONFIG_BNXT_FLOWER_OFFLOAD) += bnxt_tc.o\n bnxt_en-$(CONFIG_DEBUG_FS) += bnxt_debugfs.o\n bnxt_en-$(CONFIG_BNXT_HWMON) += bnxt_hwmon.o\n+bnxt_en-$(CONFIG_BNXT_TLS) += bnxt_mpc.o bnxt_crypto.o bnxt_ktls.o\ndiff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c\nindex 7513618793daf3..baca7ee318fbe1 100644\n--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c\n+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c\n@@ -76,6 +76,9 @@\n #include \"bnxt_hwmon.h\"\n #include \"bnxt_gso.h\"\n #include \u003cnet/tso.h\u003e\n+#include \"bnxt_mpc.h\"\n+#include \"bnxt_crypto.h\"\n+#include \"bnxt_ktls.h\"\n \n #define BNXT_TX_TIMEOUT\t\t(5 * HZ)\n #define BNXT_DEF_MSG_ENABLE\t(NETIF_MSG_DRV | NETIF_MSG_HW | \\\n@@ -478,13 +481,15 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)\n \tdma_addr_t mapping;\n \tunsigned int length, pad = 0;\n \tu32 len, free_size, vlan_tag_flags, cfa_action, flags;\n+\tstruct bnxt_ktls_offload_ctx_tx *kctx_tx = NULL;\n+\tu16 prod, start_prod, last_frag, txts_prod;\n \tstruct bnxt_ptp_cfg *ptp = bp-\u003eptp_cfg;\n \tstruct pci_dev *pdev = bp-\u003epdev;\n-\tu16 prod, last_frag, txts_prod;\n \tstruct bnxt_tx_ring_info *txr;\n \tstruct bnxt_sw_tx_bd *tx_buf;\n \t__le32 lflags = 0;\n \tskb_frag_t *frag;\n+\tu32 kid = 0;\n \n \ti = skb_get_queue_mapping(skb);\n \tif (unlikely(i \u003e= bp-\u003etx_nr_rings)) {\n@@ -495,7 +500,6 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)\n \n \ttxq = netdev_get_tx_queue(dev, i);\n \ttxr = \u0026bp-\u003etx_ring[bp-\u003etx_ring_map[i]];\n-\tprod = txr-\u003etx_prod;\n \n #if (MAX_SKB_FRAGS \u003e TX_MAX_FRAGS)\n \tif (skb_shinfo(skb)-\u003enr_frags \u003e TX_MAX_FRAGS) {\n@@ -524,6 +528,14 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)\n \t\t\treturn NETDEV_TX_BUSY;\n \t}\n \n+\tstart_prod = txr-\u003etx_prod;\n+\tskb = bnxt_ktls_xmit(bp, txr, skb, \u0026lflags, \u0026kid, \u0026kctx_tx);\n+\tif (unlikely(!skb)) {\n+\t\tdev_core_stats_tx_dropped_inc(dev);\n+\t\treturn NETDEV_TX_OK;\n+\t}\n+\n+\tprod = txr-\u003etx_prod;\n \tlength = skb-\u003elen;\n \tlen = skb_headlen(skb);\n \tlast_frag = skb_shinfo(skb)-\u003enr_frags;\n@@ -672,7 +684,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)\n \n \tprod = NEXT_TX(prod);\n \ttxbd1 = bnxt_init_ext_bd(bp, txr, prod, lflags, vlan_tag_flags,\n-\t\t\t\t cfa_action);\n+\t\t\t\t cfa_action, kid);\n \n \tif (skb_is_gso(skb)) {\n \t\tbool udp_gso = !!(skb_shinfo(skb)-\u003egso_type \u0026 SKB_GSO_UDP_L4);\n@@ -693,9 +705,10 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)\n \n \t\ttxbd1-\u003etx_bd_hsize_lflags |= cpu_to_le32(TX_BD_FLAGS_LSO |\n \t\t\t\t\tTX_BD_FLAGS_T_IPID |\n-\t\t\t\t\t(hdr_len \u003c\u003c (TX_BD_HSIZE_SHIFT - 1)));\n+\t\t\t\t\t((hdr_len \u003e\u003e 1) \u003c\u003c TX_BD_HSIZE_SHIFT));\n \t\tlength = skb_shinfo(skb)-\u003egso_size;\n-\t\ttxbd1-\u003etx_bd_mss = cpu_to_le32(length);\n+\t\ttxbd1-\u003etx_bd_kid_mss = cpu_to_le32(BNXT_TX_KID_HI(kid) |\n+\t\t\t\t\t\t (length \u0026 TX_BD_MSS));\n \t\tlength += hdr_len;\n \t} else if (skb-\u003eip_summed == CHECKSUM_PARTIAL) {\n \t\ttxbd1-\u003etx_bd_hsize_lflags |=\n@@ -748,6 +761,9 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)\n \tprod = NEXT_TX(prod);\n \tWRITE_ONCE(txr-\u003etx_prod, prod);\n \n+\t/* Commit the kTLS state now that the BD is in the ring. */\n+\tbnxt_ktls_xmit_commit(txr, kctx_tx);\n+\n \tif (!netdev_xmit_more() || netif_xmit_stopped(txq)) {\n \t\tbnxt_txr_db_kick(bp, txr, prod);\n \t} else {\n@@ -802,9 +818,16 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)\n \t\t\t/* set SKB to err so PTP worker will clean up */\n \t\t\tptp-\u003etxts_req[txts_prod].tx_skb = ERR_PTR(-EIO);\n \t}\n+\ttxr-\u003etx_buf_ring[RING_TX(bp, txr-\u003etx_prod)].skb = NULL;\n+\t/* Unwind any kTLS presync BDs */\n+\tif (unlikely(txr-\u003etx_prod != start_prod)) {\n+\t\ttx_buf = \u0026txr-\u003etx_buf_ring[RING_TX(bp, start_prod)];\n+\t\ttx_buf-\u003eis_push = 0;\n+\t\ttx_buf-\u003einline_data_bds = 0;\n+\t\tWRITE_ONCE(txr-\u003etx_prod, start_prod);\n+\t}\n \tif (txr-\u003ekick_pending)\n \t\tbnxt_txr_db_kick(bp, txr, txr-\u003etx_prod);\n-\ttxr-\u003etx_buf_ring[RING_TX(bp, txr-\u003etx_prod)].skb = NULL;\n \tdev_core_stats_tx_dropped_inc(dev);\n \treturn NETDEV_TX_OK;\n }\n@@ -833,7 +856,7 @@ static bool __bnxt_tx_int(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n \t\thead_buf = tx_buf;\n \t\tskb = tx_buf-\u003eskb;\n \n-\t\tif (unlikely(!skb)) {\n+\t\tif (unlikely(!skb \u0026\u0026 !tx_buf-\u003eis_push)) {\n \t\t\tbnxt_sched_reset_txr(bp, txr, cons);\n \t\t\treturn rc;\n \t\t}\n@@ -845,13 +868,22 @@ static bool __bnxt_tx_int(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n \t\t}\n \n \t\tcons = NEXT_TX(cons);\n-\t\ttx_pkts++;\n-\t\ttx_bytes += skb-\u003elen;\n+\t\tif (skb) {\n+\t\t\ttx_pkts++;\n+\t\t\ttx_bytes += skb-\u003elen;\n+\t\t}\n \t\ttx_buf-\u003eskb = NULL;\n \t\ttx_buf-\u003eis_ts_pkt = 0;\n \n \t\tif (tx_buf-\u003eis_push) {\n \t\t\ttx_buf-\u003eis_push = 0;\n+\t\t\tcons += tx_buf-\u003einline_data_bds;\n+\t\t\ttx_buf-\u003einline_data_bds = 0;\n+\t\t\tif (!skb) {\n+\t\t\t\t/* presync BD */\n+\t\t\t\tcons = NEXT_TX(cons);\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tgoto next_tx_int;\n \t\t}\n \n@@ -3113,6 +3145,9 @@ static int __bnxt_poll_work(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,\n \t\t\t\trx_pkts++;\n \t\t\telse if (rc == -EBUSY)\t/* partial completion */\n \t\t\t\tbreak;\n+\t\t} else if (cmp_type == CMP_TYPE_MPC_CMP) {\n+\t\t\tif (bnxt_mpc_cmp(bp, cpr, \u0026raw_cons))\n+\t\t\t\tbreak;\n \t\t} else if (unlikely(cmp_type == CMPL_BASE_TYPE_HWRM_DONE ||\n \t\t\t\t cmp_type == CMPL_BASE_TYPE_HWRM_FWD_REQ ||\n \t\t\t\t cmp_type == CMPL_BASE_TYPE_HWRM_ASYNC_EVENT)) {\n@@ -3684,7 +3719,7 @@ static size_t __bnxt_copy_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem,\n \treturn total_len;\n }\n \n-static void bnxt_free_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem)\n+void bnxt_free_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem)\n {\n \tstruct pci_dev *pdev = bp-\u003epdev;\n \tint i;\n@@ -3717,7 +3752,7 @@ static void bnxt_free_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem)\n \t}\n }\n \n-static int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem)\n+int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem)\n {\n \tstruct pci_dev *pdev = bp-\u003epdev;\n \tu64 valid_bit = 0;\n@@ -4079,6 +4114,8 @@ static void bnxt_free_tx_rings(struct bnxt *bp)\n \n \t\tbnxt_free_tx_inline_buf(txr, pdev);\n \n+\t\tbnxt_ktls_free_tx_ring_stats(txr);\n+\n \t\tring = \u0026txr-\u003etx_ring_struct;\n \n \t\tbnxt_free_ring(bp, \u0026ring-\u003ering_mem);\n@@ -4153,7 +4190,10 @@ static int bnxt_alloc_tx_rings(struct bnxt *bp)\n \t\t}\n \t\tqidx = bp-\u003etc_to_qidx[j];\n \t\tring-\u003equeue_id = bp-\u003eq_info[qidx].queue_id;\n-\t\tspin_lock_init(\u0026txr-\u003exdp_tx_lock);\n+\t\tspin_lock_init(\u0026txr-\u003etx_lock);\n+\t\trc = bnxt_ktls_alloc_tx_ring_stats(bp, txr);\n+\t\tif (rc)\n+\t\t\treturn rc;\n \t\tif (i \u003c bp-\u003etx_nr_rings_xdp)\n \t\t\tcontinue;\n \t\tif (BNXT_RING_TO_TC_OFF(bp, i) == (bp-\u003etx_nr_rings_per_tc - 1))\n@@ -4324,6 +4364,8 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp)\n \t\t\t (!sh \u0026\u0026 i \u003e= bp-\u003erx_nr_rings)) {\n \t\t\tcp_count += tcs;\n \t\t\ttx = 1;\n+\t\t\tif (bnxt_napi_has_mpc(bp, i))\n+\t\t\t\tcp_count++;\n \t\t}\n \n \t\tcpr-\u003ecp_ring_arr = kzalloc_objs(*cpr, cp_count);\n@@ -4345,6 +4387,11 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp)\n \t\t\t} else {\n \t\t\t\tint n, tc = k - rx;\n \n+\t\t\t\t/* MPC rings are at the highest k indices */\n+\t\t\t\tif (tc \u003e= tcs) {\n+\t\t\t\t\tbnxt_set_mpc_cp_ring(bp, i, cpr2);\n+\t\t\t\t\tcontinue;\n+\t\t\t\t}\n \t\t\t\tn = BNXT_TC_TO_RING_BASE(bp, tc) + j;\n \t\t\t\tbp-\u003etx_ring[n].tx_cpr = cpr2;\n \t\t\t\tcpr2-\u003ecp_ring_type = BNXT_NQ_HDL_TYPE_TX;\n@@ -4477,6 +4524,7 @@ static void bnxt_init_ring_struct(struct bnxt *bp)\n \t\t\trmem-\u003evmem = (void **)\u0026txr-\u003etx_buf_ring;\n \t\t}\n \t}\n+\tbnxt_init_mpc_ring_struct(bp);\n }\n \n static void bnxt_init_rxbd_pages(struct bnxt_ring_struct *ring, u32 type)\n@@ -5553,6 +5601,7 @@ static void bnxt_init_l2_fltr_tbl(struct bnxt *bp)\n static void bnxt_free_mem(struct bnxt *bp, bool irq_re_init)\n {\n \tbnxt_free_vnic_attributes(bp);\n+\tbnxt_free_mpc_rings(bp);\n \tbnxt_free_tx_rings(bp);\n \tbnxt_free_rx_rings(bp);\n \tbnxt_free_cp_rings(bp);\n@@ -5566,6 +5615,7 @@ static void bnxt_free_mem(struct bnxt *bp, bool irq_re_init)\n \t\t\tbnxt_free_port_stats(bp);\n \t\tbnxt_free_ring_grps(bp);\n \t\tbnxt_free_vnics(bp);\n+\t\tbnxt_free_mpcs(bp);\n \t\tkfree(bp-\u003etx_ring_map);\n \t\tbp-\u003etx_ring_map = NULL;\n \t\tkfree(bp-\u003etx_ring);\n@@ -5675,6 +5725,10 @@ static int bnxt_alloc_mem(struct bnxt *bp, bool irq_re_init)\n \t\t\t\ttxr-\u003etx_cpr = \u0026bnapi2-\u003ecp_ring;\n \t\t}\n \n+\t\trc = bnxt_alloc_mpcs(bp);\n+\t\tif (rc)\n+\t\t\tgoto alloc_mem_err;\n+\n \t\trc = bnxt_alloc_stats(bp);\n \t\tif (rc)\n \t\t\tgoto alloc_mem_err;\n@@ -5703,6 +5757,10 @@ static int bnxt_alloc_mem(struct bnxt *bp, bool irq_re_init)\n \tif (rc)\n \t\tgoto alloc_mem_err;\n \n+\trc = bnxt_alloc_mpc_rings(bp);\n+\tif (rc)\n+\t\tgoto alloc_mem_err;\n+\n \trc = bnxt_alloc_cp_rings(bp);\n \tif (rc)\n \t\tgoto alloc_mem_err;\n@@ -7257,10 +7315,15 @@ static int hwrm_ring_alloc_send_msg(struct bnxt *bp,\n \t\treq-\u003ecmpl_ring_id = cpu_to_le16(bnxt_cp_ring_for_tx(bp, txr));\n \t\treq-\u003elength = cpu_to_le32(bp-\u003etx_ring_mask + 1);\n \t\treq-\u003estat_ctx_id = cpu_to_le32(grp_info-\u003efw_stats_ctx);\n-\t\treq-\u003equeue_id = cpu_to_le16(ring-\u003equeue_id);\n-\t\tif (bp-\u003eflags \u0026 BNXT_FLAG_TX_COAL_CMPL)\n-\t\t\treq-\u003ecmpl_coal_cnt =\n-\t\t\t\tRING_ALLOC_REQ_CMPL_COAL_CNT_COAL_64;\n+\t\tif (ring-\u003equeue_id == BNXT_MPC_QUEUE_ID) {\n+\t\t\treq-\u003empc_chnls_type = ring-\u003empc_chnl_type;\n+\t\t\treq-\u003eenables |= cpu_to_le32(RING_ALLOC_REQ_ENABLES_MPC_CHNLS_TYPE);\n+\t\t} else {\n+\t\t\treq-\u003equeue_id = cpu_to_le16(ring-\u003equeue_id);\n+\t\t\tif (bp-\u003eflags \u0026 BNXT_FLAG_TX_COAL_CMPL)\n+\t\t\t\treq-\u003ecmpl_coal_cnt =\n+\t\t\t\t\tRING_ALLOC_REQ_CMPL_COAL_CNT_COAL_64;\n+\t\t}\n \t\tif ((bp-\u003efw_cap \u0026 BNXT_FW_CAP_TX_TS_CMP) \u0026\u0026 bp-\u003eptp_cfg)\n \t\t\tflags |= RING_ALLOC_REQ_FLAGS_TX_PKT_TS_CMPL_ENABLE;\n \t\treq-\u003eflags = cpu_to_le16(flags);\n@@ -7455,8 +7518,7 @@ static int bnxt_hwrm_rx_agg_ring_alloc(struct bnxt *bp,\n \treturn 0;\n }\n \n-static int bnxt_hwrm_cp_ring_alloc_p5(struct bnxt *bp,\n-\t\t\t\t struct bnxt_cp_ring_info *cpr)\n+int bnxt_hwrm_cp_ring_alloc_p5(struct bnxt *bp, struct bnxt_cp_ring_info *cpr)\n {\n \tconst u32 type = HWRM_RING_ALLOC_CMPL;\n \tstruct bnxt_napi *bnapi = cpr-\u003ebnapi;\n@@ -7474,8 +7536,8 @@ static int bnxt_hwrm_cp_ring_alloc_p5(struct bnxt *bp,\n \treturn 0;\n }\n \n-static int bnxt_hwrm_tx_ring_alloc(struct bnxt *bp,\n-\t\t\t\t struct bnxt_tx_ring_info *txr, u32 tx_idx)\n+int bnxt_hwrm_tx_ring_alloc(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n+\t\t\t u32 tx_idx)\n {\n \tstruct bnxt_ring_struct *ring = \u0026txr-\u003etx_ring_struct;\n \tconst u32 type = HWRM_RING_ALLOC_TX;\n@@ -7560,6 +7622,9 @@ static int bnxt_hwrm_ring_alloc(struct bnxt *bp)\n \t\t\t\tgoto err_out;\n \t\t}\n \t}\n+\n+\trc = bnxt_hwrm_mpc_ring_alloc(bp);\n+\n err_out:\n \treturn rc;\n }\n@@ -7617,9 +7682,8 @@ static int hwrm_ring_free_send_msg(struct bnxt *bp,\n \treturn 0;\n }\n \n-static void bnxt_hwrm_tx_ring_free(struct bnxt *bp,\n-\t\t\t\t struct bnxt_tx_ring_info *txr,\n-\t\t\t\t bool close_path)\n+void bnxt_hwrm_tx_ring_free(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n+\t\t\t bool close_path)\n {\n \tstruct bnxt_ring_struct *ring = \u0026txr-\u003etx_ring_struct;\n \tu32 cmpl_ring_id;\n@@ -7678,8 +7742,7 @@ static void bnxt_hwrm_rx_agg_ring_free(struct bnxt *bp,\n \tbp-\u003egrp_info[grp_idx].agg_fw_ring_id = INVALID_HW_RING_ID;\n }\n \n-static void bnxt_hwrm_cp_ring_free(struct bnxt *bp,\n-\t\t\t\t struct bnxt_cp_ring_info *cpr)\n+void bnxt_hwrm_cp_ring_free(struct bnxt *bp, struct bnxt_cp_ring_info *cpr)\n {\n \tstruct bnxt_ring_struct *ring;\n \n@@ -7692,7 +7755,7 @@ static void bnxt_hwrm_cp_ring_free(struct bnxt *bp,\n \tring-\u003efw_ring_id = INVALID_HW_RING_ID;\n }\n \n-static void bnxt_clear_one_cp_ring(struct bnxt *bp, struct bnxt_cp_ring_info *cpr)\n+void bnxt_clear_one_cp_ring(struct bnxt *bp, struct bnxt_cp_ring_info *cpr)\n {\n \tstruct bnxt_ring_struct *ring = \u0026cpr-\u003ecp_ring_struct;\n \tint i, size = ring-\u003ering_mem.page_size;\n@@ -7713,6 +7776,8 @@ static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path)\n \tif (!bp-\u003ebnapi)\n \t\treturn;\n \n+\tbnxt_hwrm_mpc_ring_free(bp, close_path);\n+\n \tfor (i = 0; i \u003c bp-\u003etx_nr_rings; i++)\n \t\tbnxt_hwrm_tx_ring_free(bp, \u0026bp-\u003etx_ring[i], close_path);\n \n@@ -7751,6 +7816,11 @@ static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path)\n \t}\n }\n \n+int bnxt_total_tx_rings(struct bnxt *bp)\n+{\n+\treturn bp-\u003etx_nr_rings + bnxt_mpc_tx_rings_in_use(bp);\n+}\n+\n static int __bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max,\n \t\t\t bool shared);\n static int bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max,\n@@ -7759,6 +7829,7 @@ static int bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max,\n static int bnxt_hwrm_get_rings(struct bnxt *bp)\n {\n \tstruct bnxt_hw_resc *hw_resc = \u0026bp-\u003ehw_resc;\n+\tstruct bnxt_hw_crypto_resc *crypto_resc;\n \tstruct hwrm_func_qcfg_output *resp;\n \tstruct hwrm_func_qcfg_input *req;\n \tint rc;\n@@ -7791,25 +7862,38 @@ static int bnxt_hwrm_get_rings(struct bnxt *bp)\n \t\tstats = le16_to_cpu(resp-\u003ealloc_stat_ctx);\n \t\thw_resc-\u003eresv_irqs = cp;\n \t\tif (bp-\u003eflags \u0026 BNXT_FLAG_CHIP_P5_PLUS) {\n+\t\t\tint mpc_tx = bnxt_mpc_tx_rings_in_use(bp);\n+\t\t\tint mpc_cp = bnxt_mpc_cp_rings_in_use(bp);\n \t\t\tint rx = hw_resc-\u003eresv_rx_rings;\n \t\t\tint tx = hw_resc-\u003eresv_tx_rings;\n+\t\t\tint cp_p5;\n \n+\t\t\tif (tx \u003c= mpc_tx || cp \u003c= mpc_cp) {\n+\t\t\t\trc = -ENOMEM;\n+\t\t\t\tgoto get_rings_exit;\n+\t\t\t}\n+\t\t\ttx -= mpc_tx;\n+\t\t\tcp_p5 = cp - mpc_cp;\n \t\t\tif (bp-\u003eflags \u0026 BNXT_FLAG_AGG_RINGS)\n \t\t\t\trx \u003e\u003e= 1;\n-\t\t\tif (cp \u003c (rx + tx)) {\n-\t\t\t\trc = __bnxt_trim_rings(bp, \u0026rx, \u0026tx, cp, false);\n+\t\t\tif (cp_p5 \u003c (rx + tx)) {\n+\t\t\t\trc = __bnxt_trim_rings(bp, \u0026rx, \u0026tx, cp_p5, false);\n \t\t\t\tif (rc)\n \t\t\t\t\tgoto get_rings_exit;\n \t\t\t\tif (bp-\u003eflags \u0026 BNXT_FLAG_AGG_RINGS)\n \t\t\t\t\trx \u003c\u003c= 1;\n \t\t\t\thw_resc-\u003eresv_rx_rings = rx;\n-\t\t\t\thw_resc-\u003eresv_tx_rings = tx;\n+\t\t\t\thw_resc-\u003eresv_tx_rings = tx + mpc_tx;\n \t\t\t}\n \t\t\thw_resc-\u003eresv_irqs = le16_to_cpu(resp-\u003ealloc_msix);\n \t\t\thw_resc-\u003eresv_hw_ring_grps = rx;\n \t\t}\n \t\thw_resc-\u003eresv_cp_rings = cp;\n \t\thw_resc-\u003eresv_stat_ctxs = stats;\n+\n+\t\tcrypto_resc = \u0026hw_resc-\u003ecrypto_resc;\n+\t\tcrypto_resc-\u003eresv_tx_key_ctxs = le32_to_cpu(resp-\u003enum_ktls_tx_key_ctxs);\n+\t\tcrypto_resc-\u003eresv_rx_key_ctxs = le32_to_cpu(resp-\u003enum_ktls_rx_key_ctxs);\n \t}\n get_rings_exit:\n \thwrm_req_drop(bp, req);\n@@ -7880,8 +7964,9 @@ __bnxt_hwrm_reserve_pf_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr)\n \t\t}\n \t\treq-\u003enum_stat_ctxs = cpu_to_le16(hwr-\u003estat);\n \t\treq-\u003enum_vnics = cpu_to_le16(hwr-\u003evnic);\n+\t\tbnxt_hwrm_reserve_pf_key_ctxs(bp, req);\n \t}\n-\treq-\u003eenables = cpu_to_le32(enables);\n+\treq-\u003eenables |= cpu_to_le32(enables);\n \treturn req;\n }\n \n@@ -7995,7 +8080,7 @@ static int bnxt_cp_rings_in_use(struct bnxt *bp)\n \t\treturn bnxt_nq_rings_in_use(bp);\n \n \tcp = bp-\u003etx_nr_rings + bp-\u003erx_nr_rings;\n-\treturn cp;\n+\treturn cp + bnxt_mpc_cp_rings_in_use(bp);\n }\n \n static int bnxt_get_func_stat_ctxs(struct bnxt *bp)\n@@ -8053,7 +8138,7 @@ static void bnxt_get_total_resources(struct bnxt *bp, struct bnxt_hw_rings *hwr)\n \thwr-\u003ecp_p5 = 0;\n \tif (bp-\u003eflags \u0026 BNXT_FLAG_CHIP_P5_PLUS)\n \t\thwr-\u003ecp_p5 = bnxt_cp_rings_in_use(bp);\n-\thwr-\u003etx = bp-\u003etx_nr_rings;\n+\thwr-\u003etx = bnxt_total_tx_rings(bp);\n \thwr-\u003erx = bp-\u003erx_nr_rings;\n \thwr-\u003egrp = hwr-\u003erx;\n \thwr-\u003evnic = bnxt_get_total_vnics(bp, hwr-\u003erx);\n@@ -8159,8 +8244,10 @@ static int __bnxt_reserve_rings(struct bnxt *bp)\n \thwr.rx = bp-\u003erx_nr_rings;\n \tif (bp-\u003eflags \u0026 BNXT_FLAG_SHARED_RINGS)\n \t\tsh = true;\n-\tif (bp-\u003eflags \u0026 BNXT_FLAG_CHIP_P5_PLUS)\n-\t\thwr.cp_p5 = hwr.rx + hwr.tx;\n+\tif (bp-\u003eflags \u0026 BNXT_FLAG_CHIP_P5_PLUS) {\n+\t\thwr.cp_p5 = hwr.rx + hwr.tx + bnxt_mpc_cp_rings_in_use(bp);\n+\t\thwr.tx += bnxt_mpc_tx_rings_in_use(bp);\n+\t}\n \n \thwr.vnic = bnxt_get_total_vnics(bp, hwr.rx);\n \n@@ -8197,6 +8284,9 @@ static int __bnxt_reserve_rings(struct bnxt *bp)\n \tif (bnxt_ulp_registered(edev) \u0026\u0026 hwr.stat \u003e bnxt_get_ulp_stat_ctxs(bp))\n \t\thwr.stat -= bnxt_get_ulp_stat_ctxs(bp);\n \thwr.cp = min_t(int, hwr.cp, hwr.stat);\n+\thwr.tx -= bnxt_mpc_tx_rings_in_use(bp);\n+\tif (hwr.tx \u003c 0)\n+\t\treturn -ENOMEM;\n \trc = bnxt_trim_rings(bp, \u0026rx_rings, \u0026hwr.tx, hwr.cp, sh);\n \tif (bp-\u003eflags \u0026 BNXT_FLAG_AGG_RINGS)\n \t\thwr.rx = rx_rings \u003c\u003c 1;\n@@ -8278,7 +8368,7 @@ static int bnxt_hwrm_check_vf_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr)\n static int bnxt_hwrm_check_pf_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr)\n {\n \tstruct hwrm_func_cfg_input *req;\n-\tu32 flags;\n+\tu32 flags, flags2 = 0;\n \n \treq = __bnxt_hwrm_reserve_pf_rings(bp, hwr);\n \tflags = FUNC_CFG_REQ_FLAGS_TX_ASSETS_TEST;\n@@ -8292,9 +8382,14 @@ static int bnxt_hwrm_check_pf_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr)\n \t\t\t\t FUNC_CFG_REQ_FLAGS_NQ_ASSETS_TEST;\n \t\telse\n \t\t\tflags |= FUNC_CFG_REQ_FLAGS_RING_GRP_ASSETS_TEST;\n+\t\tif (req-\u003eenables \u0026\n+\t\t cpu_to_le32(FUNC_CFG_REQ_ENABLES_KTLS_TX_KEY_CTXS |\n+\t\t\t\tFUNC_CFG_REQ_ENABLES_KTLS_RX_KEY_CTXS))\n+\t\t\tflags2 |= FUNC_CFG_REQ_FLAGS2_KTLS_KEY_CTX_ASSETS_TEST;\n \t}\n \n \treq-\u003eflags = cpu_to_le32(flags);\n+\treq-\u003eflags2 = cpu_to_le32(flags2);\n \treturn hwrm_req_send_silent(bp, req);\n }\n \n@@ -9331,12 +9426,35 @@ static int bnxt_hwrm_func_backing_store_cfg_v2(struct bnxt *bp,\n \n static int bnxt_backing_store_cfg_v2(struct bnxt *bp)\n {\n+\tstruct bnxt_crypto_info *crypto = bp-\u003ecrypto_info;\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n \tstruct bnxt_ctx_mem_info *ctx = bp-\u003ectx;\n \tstruct bnxt_ctx_mem_type *ctxm;\n \tu16 last_type = BNXT_CTX_INV;\n \tint rc = 0;\n \tu16 type;\n \n+\tif (BNXT_SUPPORTS_KTLS(bp)) {\n+\t\tctxm = \u0026ctx-\u003ectx_arr[BNXT_CTX_TCK];\n+\t\trc = bnxt_setup_ctxm_pg_tbls(bp, ctxm,\n+\t\t\t\t\t BNXT_TCK(crypto).max_ctx, 1);\n+\t\tif (rc)\n+\t\t\treturn rc;\n+\t\tctxm = \u0026ctx-\u003ectx_arr[BNXT_CTX_RCK];\n+\t\trc = bnxt_setup_ctxm_pg_tbls(bp, ctxm,\n+\t\t\t\t\t BNXT_RCK(crypto).max_ctx, 1);\n+\t\tif (rc)\n+\t\t\treturn rc;\n+\t\tlast_type = BNXT_CTX_RCK;\n+\t}\n+\tif (mpc \u0026\u0026 mpc-\u003empc_chnls_cap) {\n+\t\tctxm = \u0026ctx-\u003ectx_arr[BNXT_CTX_MTQM];\n+\t\trc = bnxt_setup_ctxm_pg_tbls(bp, ctxm, ctxm-\u003emax_entries, 1);\n+\t\tif (rc)\n+\t\t\treturn rc;\n+\t\tlast_type = BNXT_CTX_MTQM;\n+\t}\n+\n \tfor (type = BNXT_CTX_SRT; type \u003c= BNXT_CTX_QPC; type++) {\n \t\tctxm = \u0026ctx-\u003ectx_arr[type];\n \t\tif (!bnxt_bs_trace_avail(bp, type))\n@@ -9685,6 +9803,7 @@ int bnxt_hwrm_func_resc_qcaps(struct bnxt *bp, bool all)\n \tstruct hwrm_func_resource_qcaps_output *resp;\n \tstruct hwrm_func_resource_qcaps_input *req;\n \tstruct bnxt_hw_resc *hw_resc = \u0026bp-\u003ehw_resc;\n+\tstruct bnxt_hw_crypto_resc *crypto_resc;\n \tint rc;\n \n \trc = hwrm_req_init(bp, req, HWRM_FUNC_RESOURCE_QCAPS);\n@@ -9722,6 +9841,12 @@ int bnxt_hwrm_func_resc_qcaps(struct bnxt *bp, bool all)\n \t hw_resc-\u003emax_vnics * BNXT_LARGE_RSS_TO_VNIC_RATIO)\n \t\tbp-\u003erss_cap |= BNXT_RSS_CAP_LARGE_RSS_CTX;\n \n+\tcrypto_resc = \u0026hw_resc-\u003ecrypto_resc;\n+\tcrypto_resc-\u003emin_tx_key_ctxs = le32_to_cpu(resp-\u003emin_ktls_tx_key_ctxs);\n+\tcrypto_resc-\u003emax_tx_key_ctxs = le32_to_cpu(resp-\u003emax_ktls_tx_key_ctxs);\n+\tcrypto_resc-\u003emin_rx_key_ctxs = le32_to_cpu(resp-\u003emin_ktls_rx_key_ctxs);\n+\tcrypto_resc-\u003emax_rx_key_ctxs = le32_to_cpu(resp-\u003emax_ktls_rx_key_ctxs);\n+\n \tif (bp-\u003eflags \u0026 BNXT_FLAG_CHIP_P5_PLUS) {\n \t\tu16 max_msix = le16_to_cpu(resp-\u003emax_msix);\n \n@@ -9871,6 +9996,10 @@ static int __bnxt_hwrm_func_qcaps(struct bnxt *bp)\n \t\tbp-\u003efw_cap |= BNXT_FW_CAP_BACKING_STORE_V2;\n \tif (flags_ext \u0026 FUNC_QCAPS_RESP_FLAGS_EXT_TX_COAL_CMPL_CAP)\n \t\tbp-\u003eflags |= BNXT_FLAG_TX_COAL_CMPL;\n+\tif (flags_ext \u0026 FUNC_QCAPS_RESP_FLAGS_EXT_KTLS_SUPPORTED)\n+\t\tbnxt_alloc_crypto_info(bp, resp);\n+\telse\n+\t\tbp-\u003efw_cap \u0026= ~BNXT_FW_CAP_KTLS;\n \n \tflags_ext2 = le32_to_cpu(resp-\u003eflags_ext2);\n \tif (flags_ext2 \u0026 FUNC_QCAPS_RESP_FLAGS_EXT2_RX_ALL_PKTS_TIMESTAMPS_SUPPORTED)\n@@ -9943,6 +10072,11 @@ static int __bnxt_hwrm_func_qcaps(struct bnxt *bp)\n \t}\n \tbp-\u003etso_max_segs = le16_to_cpu(resp-\u003emax_tso_segs);\n \n+\tif (resp-\u003empc_chnls_cap)\n+\t\tbnxt_alloc_mpc_info(bp, resp-\u003empc_chnls_cap);\n+\telse\n+\t\tbnxt_free_mpc_info(bp);\n+\n hwrm_func_qcaps_exit:\n \thwrm_req_drop(bp, req);\n \treturn rc;\n@@ -11321,6 +11455,7 @@ static int bnxt_init_nic(struct bnxt *bp, bool irq_re_init)\n \tbnxt_init_cp_rings(bp);\n \tbnxt_init_rx_rings(bp);\n \tbnxt_init_tx_rings(bp);\n+\tbnxt_init_mpc_rings(bp);\n \tbnxt_init_ring_grps(bp, irq_re_init);\n \tbnxt_init_vnics(bp);\n \n@@ -11511,12 +11646,13 @@ unsigned int bnxt_get_max_func_cp_rings(struct bnxt *bp)\n \n static unsigned int bnxt_get_max_func_cp_rings_for_en(struct bnxt *bp)\n {\n+\tunsigned int mpc_cp = (unsigned int)bnxt_mpc_cp_rings_in_use(bp);\n \tunsigned int cp = bp-\u003ehw_resc.max_cp_rings;\n \n \tif (!(bp-\u003eflags \u0026 BNXT_FLAG_CHIP_P5_PLUS))\n \t\tcp -= bnxt_get_ulp_msix_num(bp);\n \n-\treturn cp;\n+\treturn mpc_cp \u003e= cp ? 0 : cp - mpc_cp;\n }\n \n static unsigned int bnxt_get_max_func_irqs(struct bnxt *bp)\n@@ -12801,6 +12937,8 @@ static int bnxt_hwrm_if_change(struct bnxt *bp, bool up)\n \t\t\t\tbnxt_ulp_irq_stop(bp);\n \t\t\tbnxt_free_ctx_mem(bp, false);\n \t\t\tbnxt_dcb_free(bp);\n+\t\t\tif (fw_reset || caps_change)\n+\t\t\t\tbnxt_clear_crypto(bp);\n \t\t\trc = bnxt_fw_init_one(bp);\n \t\t\tif (rc) {\n \t\t\t\tclear_bit(BNXT_STATE_FW_RESET_DET, \u0026bp-\u003estate);\n@@ -13170,6 +13308,7 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)\n \t\treturn rc;\n \n \tbnxt_adj_tx_rings(bp);\n+\tbnxt_trim_mpc_rings(bp);\n \trc = bnxt_alloc_mem(bp, irq_re_init);\n \tif (rc) {\n \t\tnetdev_err(bp-\u003edev, \"bnxt_alloc_mem err: %x\\n\", rc);\n@@ -13222,6 +13361,7 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)\n \t\tstatic_branch_disable(\u0026bnxt_xdp_locking_key);\n \t}\n \tset_bit(BNXT_STATE_OPEN, \u0026bp-\u003estate);\n+\tbnxt_ktls_wake(bp);\n \tbnxt_enable_int(bp);\n \t/* Enable TX queues */\n \tbnxt_tx_enable(bp);\n@@ -13363,7 +13503,8 @@ static int bnxt_open(struct net_device *dev)\n static bool bnxt_drv_busy(struct bnxt *bp)\n {\n \treturn (test_bit(BNXT_STATE_IN_SP_TASK, \u0026bp-\u003estate) ||\n-\t\ttest_bit(BNXT_STATE_READ_STATS, \u0026bp-\u003estate));\n+\t\ttest_bit(BNXT_STATE_READ_STATS, \u0026bp-\u003estate) ||\n+\t\tbnxt_ktls_busy(bp));\n }\n \n static void bnxt_get_ring_stats(struct bnxt *bp,\n@@ -13381,9 +13522,20 @@ static void __bnxt_close_nic(struct bnxt *bp, bool irq_re_init,\n \n \tclear_bit(BNXT_STATE_OPEN, \u0026bp-\u003estate);\n \tsmp_mb__after_atomic();\n+\t/* Wake any kTLS delete waiting on a reconfig so it re-evaluates and\n+\t * either keeps waiting for the reopen or aborts (ifdown / FW reset).\n+\t */\n+\tbnxt_ktls_wake(bp);\n \twhile (bnxt_drv_busy(bp))\n \t\tmsleep(20);\n \n+\t/* Delete all crypto connections and KIDs only on ifdown and FW reset,\n+\t * not ethtool config changes.\n+\t */\n+\tif (!netif_running(bp-\u003edev) ||\n+\t test_bit(BNXT_STATE_IN_FW_RESET, \u0026bp-\u003estate))\n+\t\tbnxt_crypto_del_all(bp);\n+\n \tif (BNXT_SUPPORTS_MULTI_RSS_CTX(bp))\n \t\tbnxt_clear_rss_ctxs(bp);\n \t/* Flush rings and disable interrupts */\n@@ -14320,7 +14472,7 @@ static int bnxt_hwrm_rx_ring_reset(struct bnxt *bp, int ring_nr)\n \treturn hwrm_req_send_silent(bp, req);\n }\n \n-static void bnxt_reset_task(struct bnxt *bp, bool silent)\n+void bnxt_reset_task(struct bnxt *bp, bool silent)\n {\n \tif (!silent)\n \t\tbnxt_dbg_dump_states(bp);\n@@ -14537,6 +14689,7 @@ static void bnxt_fw_reset_close(struct bnxt *bp)\n \tbnxt_hwrm_func_drv_unrgtr(bp);\n \tif (pci_is_enabled(bp-\u003epdev))\n \t\tpci_disable_device(bp-\u003epdev);\n+\tbnxt_clear_crypto(bp);\n \tbnxt_free_ctx_mem(bp, false);\n }\n \n@@ -14918,8 +15071,10 @@ int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs,\n \t\thwr.grp = rx;\n \t\thwr.rss_ctx = bnxt_get_total_rss_ctxs(bp, \u0026hwr);\n \t}\n-\tif (bp-\u003eflags \u0026 BNXT_FLAG_CHIP_P5_PLUS)\n-\t\thwr.cp_p5 = hwr.tx + rx;\n+\tif (bp-\u003eflags \u0026 BNXT_FLAG_CHIP_P5_PLUS) {\n+\t\thwr.cp_p5 = hwr.tx + rx + bnxt_mpc_cp_rings_in_use(bp);\n+\t\thwr.tx += bnxt_mpc_tx_rings_in_use(bp);\n+\t}\n \trc = bnxt_hwrm_check_rings(bp, \u0026hwr);\n \tif (!rc \u0026\u0026 pci_msix_can_alloc_dyn(bp-\u003epdev)) {\n \t\tif (!bnxt_ulp_registered(bp-\u003eedev[BNXT_AUXDEV_RDMA])) {\n@@ -16547,6 +16702,8 @@ static void bnxt_remove_one(struct pci_dev *pdev)\n \tbp-\u003eptp_cfg = NULL;\n \tkfree(bp-\u003efw_health);\n \tbp-\u003efw_health = NULL;\n+\tbnxt_free_crypto_info(bp);\n+\tbnxt_free_mpc_info(bp);\n \tbnxt_cleanup_pci(bp);\n \tbnxt_free_ctx_mem(bp, true);\n \tbnxt_free_crash_dump_mem(bp);\n@@ -16616,7 +16773,8 @@ static void _bnxt_get_max_rings(struct bnxt *bp, int *max_rx, int *max_tx,\n \tstruct bnxt_hw_resc *hw_resc = \u0026bp-\u003ehw_resc;\n \tint max_ring_grps = 0, max_irq;\n \n-\t*max_tx = hw_resc-\u003emax_tx_rings;\n+\t*max_tx = max(0, (int)hw_resc-\u003emax_tx_rings -\n+\t\t\t bnxt_mpc_tx_rings_in_use(bp));\n \t*max_rx = hw_resc-\u003emax_rx_rings;\n \t*max_cp = bnxt_get_max_func_cp_rings_for_en(bp);\n \tmax_irq = min_t(int, bnxt_get_max_func_irqs(bp) -\n@@ -16713,6 +16871,7 @@ static void bnxt_trim_dflt_sh_rings(struct bnxt *bp)\n \tbp-\u003erx_nr_rings = bp-\u003ecp_nr_rings;\n \tbp-\u003etx_nr_rings_per_tc = bp-\u003ecp_nr_rings;\n \tbp-\u003etx_nr_rings = bnxt_tx_nr_rings(bp);\n+\tbnxt_trim_mpc_rings(bp);\n }\n \n static void bnxt_adj_dflt_rings(struct bnxt *bp, bool sh)\n@@ -16764,6 +16923,8 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)\n \t\tbnxt_set_dflt_ulp_stat_ctxs(bp);\n \t}\n \n+\tbnxt_set_dflt_mpc_rings(bp);\n+\n \trc = __bnxt_reserve_rings(bp);\n \tif (rc \u0026\u0026 rc != -ENODEV)\n \t\tnetdev_warn(bp-\u003edev, \"Unable to reserve tx rings\\n\");\n@@ -16778,6 +16939,7 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)\n \t\tif (rc \u0026\u0026 rc != -ENODEV)\n \t\t\tnetdev_warn(bp-\u003edev, \"2nd rings reservation failed.\\n\");\n \t\tbnxt_adj_tx_rings(bp);\n+\t\tbnxt_trim_mpc_rings(bp);\n \t}\n \tif (BNXT_CHIP_TYPE_NITRO_A0(bp)) {\n \t\tbp-\u003erx_nr_rings++;\n@@ -16812,6 +16974,7 @@ static int bnxt_init_dflt_ring_mode(struct bnxt *bp)\n \t\tgoto init_dflt_ring_err;\n \n \tbnxt_adj_tx_rings(bp);\n+\tbnxt_trim_mpc_rings(bp);\n \n \tbnxt_set_dflt_rfs(bp);\n \n@@ -17155,6 +17318,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)\n \t * limited MSIX, so we re-initialize the TX rings per TC.\n \t */\n \tbp-\u003etx_nr_rings_per_tc = bp-\u003etx_nr_rings;\n+\tbnxt_trim_mpc_rings(bp);\n \n \tif (BNXT_PF(bp)) {\n \t\tif (!bnxt_pf_wq) {\n@@ -17176,6 +17340,12 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)\n \trc = bnxt_dl_register(bp);\n \tif (rc)\n \t\tgoto init_err_dl;\n+\trc = bnxt_crypto_init(bp);\n+\tif (rc) {\n+\t\tbnxt_free_crypto_info(bp);\n+\t\tnetdev_warn(bp-\u003edev, \"Failed to initialize crypto offload, err = %d\\n\",\n+\t\t\t rc);\n+\t}\n \n \tINIT_LIST_HEAD(\u0026bp-\u003eusr_fltr_list);\n \n@@ -17218,6 +17388,8 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)\n \tbnxt_ethtool_free(bp);\n \tkfree(bp-\u003efw_health);\n \tbp-\u003efw_health = NULL;\n+\tbnxt_free_crypto_info(bp);\n+\tbnxt_free_mpc_info(bp);\n \tbnxt_cleanup_pci(bp);\n \tbnxt_free_ctx_mem(bp, true);\n \tbnxt_free_crash_dump_mem(bp);\n@@ -17400,6 +17572,7 @@ static pci_ers_result_t bnxt_io_error_detected(struct pci_dev *pdev,\n \n \tif (pci_is_enabled(pdev))\n \t\tpci_disable_device(pdev);\n+\tbnxt_clear_crypto(bp);\n \tbnxt_free_ctx_mem(bp, false);\n \tnetdev_unlock(netdev);\n \ndiff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h\nindex 6335dfc14c985b..19ffbb2cc6b120 100644\n--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h\n+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h\n@@ -47,6 +47,7 @@ struct tx_bd {\n \t__le32 tx_bd_len_flags_type;\n \t#define TX_BD_TYPE\t\t\t\t\t(0x3f \u003c\u003c 0)\n \t #define TX_BD_TYPE_SHORT_TX_BD\t\t\t\t (0x00 \u003c\u003c 0)\n+\t #define TX_BD_TYPE_MPC_TX_BD\t\t\t\t (0x08 \u003c\u003c 0)\n \t #define TX_BD_TYPE_LONG_TX_BD\t\t\t\t (0x10 \u003c\u003c 0)\n \t#define TX_BD_FLAGS_PACKET_END\t\t\t\t(1 \u003c\u003c 6)\n \t#define TX_BD_FLAGS_NO_CMPL\t\t\t\t(1 \u003c\u003c 7)\n@@ -100,10 +101,18 @@ struct tx_bd_ext {\n \t#define TX_BD_FLAGS_LSO\t\t\t\t\t(1 \u003c\u003c 5)\n \t#define TX_BD_FLAGS_IPID_FMT\t\t\t\t(1 \u003c\u003c 6)\n \t#define TX_BD_FLAGS_T_IPID\t\t\t\t(1 \u003c\u003c 7)\n+\t#define TX_BD_FLAGS_CRYPTO_EN\t\t\t\t(1 \u003c\u003c 15)\n \t#define TX_BD_HSIZE\t\t\t\t\t(0xff \u003c\u003c 16)\n \t #define TX_BD_HSIZE_SHIFT\t\t\t\t 16\n-\n-\t__le32 tx_bd_mss;\n+\t#define TX_BD_KID_LO\t\t\t\t\t(0x7f \u003c\u003c 25)\n+\t #define TX_BD_KID_LO_MASK\t\t\t\t 0x7f\n+\t #define TX_BD_KID_LO_SHIFT\t\t\t\t 25\n+\n+\t__le32 tx_bd_kid_mss;\n+\t#define TX_BD_MSS\t\t\t\t\t0x7fff\n+\t#define TX_BD_KID_HI\t\t\t\t\t(0x1ffff \u003c\u003c 15)\n+\t #define TX_BD_KID_HI_MASK\t\t\t\t 0xffff80\n+\t #define TX_BD_KID_HI_SHIFT\t\t\t\t 8\n \t__le32 tx_bd_cfa_action;\n \t#define TX_BD_CFA_ACTION\t\t\t\t(0xffff \u003c\u003c 16)\n \t #define TX_BD_CFA_ACTION_SHIFT\t\t\t\t 16\n@@ -121,6 +130,16 @@ struct tx_bd_ext {\n };\n \n #define BNXT_TX_PTP_IS_SET(lflags) ((lflags) \u0026 cpu_to_le32(TX_BD_FLAGS_STAMP))\n+#define BNXT_TX_KID_LO(kid) (((kid) \u0026 TX_BD_KID_LO_MASK) \u003c\u003c TX_BD_KID_LO_SHIFT)\n+#define BNXT_TX_KID_HI(kid) (((kid) \u0026 TX_BD_KID_HI_MASK) \u003c\u003c TX_BD_KID_HI_SHIFT)\n+\n+struct tx_bd_presync {\n+\t__le32 tx_bd_len_flags_type;\n+\t #define TX_BD_TYPE_PRESYNC_TX_BD\t\t\t (0x09 \u003c\u003c 0)\n+\tu32 tx_bd_opaque;\n+\t__le32 tx_bd_kid;\n+\tu32 tx_bd_unused;\n+};\n \n struct rx_bd {\n \t__le32 rx_bd_len_flags_type;\n@@ -160,6 +179,7 @@ struct tx_cmp {\n \t #define CMP_TYPE_RX_TPA_AGG_CMP\t\t\t 22\n \t #define CMP_TYPE_RX_L2_V3_CMP\t\t\t\t 23\n \t #define CMP_TYPE_RX_L2_TPA_START_V3_CMP\t\t 25\n+\t #define CMP_TYPE_MPC_CMP\t\t\t\t 30\n \t #define CMP_TYPE_STATUS_CMP\t\t\t\t 32\n \t #define CMP_TYPE_REMOTE_DRIVER_REQ\t\t\t 34\n \t #define CMP_TYPE_REMOTE_DRIVER_RESP\t\t\t 36\n@@ -686,6 +706,7 @@ struct nqe_cn {\n #define BNXT_NQ_HDL_TYPE_SHIFT\t24\n #define BNXT_NQ_HDL_TYPE_RX\t0x00\n #define BNXT_NQ_HDL_TYPE_TX\t0x01\n+#define BNXT_NQ_HDL_TYPE_MP\t0x02\n \n #define BNXT_NQ_HDL_IDX(hdl)\t((hdl) \u0026 BNXT_NQ_HDL_IDX_MASK)\n #define BNXT_NQ_HDL_TYPE(hdl)\t(((hdl) \u0026 BNXT_NQ_HDL_TYPE_MASK) \u003e\u003e\t\\\n@@ -896,6 +917,7 @@ struct bnxt_sw_tx_bd {\n \tu8\t\t\tis_push;\n \tu8\t\t\tis_sw_gso;\n \tu8\t\t\taction;\n+\tu8\t\t\tinline_data_bds;\n \tunsigned short\t\tnr_frags;\n \tunion {\n \t\tu16\t\t\trx_prod;\n@@ -951,6 +973,8 @@ struct bnxt_ring_struct {\n \t};\n \tu32\t\t\thandle;\n \tu8\t\t\tqueue_id;\n+#define BNXT_MPC_QUEUE_ID\t0xff\n+\tu8\t\t\tmpc_chnl_type;\n };\n \n struct tx_push_bd {\n@@ -991,12 +1015,16 @@ struct bnxt_tx_ring_info {\n \tu16\t\t\ttx_cons;\n \tu16\t\t\ttx_hw_cons;\n \tu16\t\t\ttxq_index;\n+\t/* index for tx_ring[] or tx_mpc_ring[] in struct bnxt_napi */\n \tu8\t\t\ttx_napi_idx;\n \tu8\t\t\tkick_pending;\n \tstruct bnxt_db_info\ttx_db;\n \n \tstruct tx_bd\t\t*tx_desc_ring[MAX_TX_PAGES];\n-\tstruct bnxt_sw_tx_bd\t*tx_buf_ring;\n+\tunion {\n+\t\tstruct bnxt_sw_tx_bd\t*tx_buf_ring;\n+\t\tstruct bnxt_sw_mpc_tx_bd\t*tx_mpc_buf_ring;\n+\t};\n \n \tdma_addr_t\t\ttx_desc_mapping[MAX_TX_PAGES];\n \n@@ -1014,8 +1042,11 @@ struct bnxt_tx_ring_info {\n \tu32\t\t\tdev_state;\n \n \tstruct bnxt_ring_struct\ttx_ring_struct;\n-\t/* Synchronize simultaneous xdp_xmit on same ring */\n-\tspinlock_t\t\txdp_tx_lock;\n+\t/* Synchronize simultaneous xdp_xmit on same ring or for MPC ring */\n+\tspinlock_t\t\ttx_lock;\n+\n+\t/* Per-TX-ring kTLS counters; allocated only when kTLS is enabled. */\n+\tstruct bnxt_tls_sw_stats *tls_stats;\n };\n \n #define BNXT_LEGACY_COAL_CMPL_PARAMS\t\t\t\t\t\\\n@@ -1157,6 +1188,22 @@ struct bnxt_cmn_sw_stats {\n \tu64\t\t\tmissed_irqs;\n };\n \n+/* Data plane kTLS counters */\n+enum bnxt_ktls_data_counters {\n+\tBNXT_KTLS_TX_PKTS = 0,\n+\tBNXT_KTLS_TX_BYTES,\n+\tBNXT_KTLS_TX_OOO_PKTS,\n+\tBNXT_KTLS_TX_OOO_FALLBACK_NO_SYNC,\n+\tBNXT_KTLS_TX_OOO_FALLBACK_NO_SPACE,\n+\tBNXT_KTLS_TX_OOO_FALLBACK_NO_HDR,\n+\n+\tBNXT_KTLS_MAX_DATA_COUNTERS,\n+};\n+\n+struct bnxt_tls_sw_stats {\n+\tu64\tcounters[BNXT_KTLS_MAX_DATA_COUNTERS];\n+};\n+\n struct bnxt_sw_stats {\n \tstruct bnxt_rx_sw_stats rx;\n \tstruct bnxt_tx_sw_stats tx;\n@@ -1242,6 +1289,7 @@ struct bnxt_napi {\n \tstruct bnxt_cp_ring_info\tcp_ring;\n \tstruct bnxt_rx_ring_info\t*rx_ring;\n \tstruct bnxt_tx_ring_info\t*tx_ring[BNXT_MAX_TXR_PER_NAPI];\n+\tstruct bnxt_tx_ring_info\t**tx_mpc_ring;\n \n \tvoid\t\t\t(*tx_int)(struct bnxt *, struct bnxt_napi *,\n \t\t\t\t\t int budget);\n@@ -1355,6 +1403,15 @@ struct bnxt_hw_rings {\n \tint rss_ctx;\n };\n \n+struct bnxt_hw_crypto_resc {\n+\tu32\tmin_tx_key_ctxs;\n+\tu32\tmax_tx_key_ctxs;\n+\tu32\tresv_tx_key_ctxs;\n+\tu32\tmin_rx_key_ctxs;\n+\tu32\tmax_rx_key_ctxs;\n+\tu32\tresv_rx_key_ctxs;\n+};\n+\n struct bnxt_hw_resc {\n \tu16\tmin_rsscos_ctxs;\n \tu16\tmax_rsscos_ctxs;\n@@ -1389,6 +1446,8 @@ struct bnxt_hw_resc {\n \tu32\tmax_tx_wm_flows;\n \tu32\tmax_rx_em_flows;\n \tu32\tmax_rx_wm_flows;\n+\n+\tstruct bnxt_hw_crypto_resc\tcrypto_resc;\n };\n \n #define BNXT_LARGE_RSS_TO_VNIC_RATIO\t7\n@@ -2452,6 +2511,10 @@ struct bnxt {\n \n \tu8\t\t\ttph_mode;\n \n+\tstruct bnxt_mpc_info\t*mpc_info;\n+\tstruct bnxt_crypto_info\t*crypto_info;\n+\tstruct bnxt_tls_info\t*ktls_info;\n+\n \tunsigned int\t\tcurrent_interval;\n #define BNXT_TIMER_INTERVAL\tHZ\n \n@@ -2538,6 +2601,7 @@ struct bnxt {\n \t#define BNXT_FW_CAP_NPAR_1_2\t\t\tBIT_ULL(42)\n \t#define BNXT_FW_CAP_MIRROR_ON_ROCE\t\tBIT_ULL(43)\n \t#define BNXT_FW_CAP_PTP_PTM\t\t\tBIT_ULL(44)\n+\t#define BNXT_FW_CAP_KTLS\t\t\tBIT_ULL(45)\n \n \tu32\t\t\tfw_dbg_cap;\n \n@@ -2563,6 +2627,8 @@ struct bnxt {\n \t((bp)-\u003efw_cap \u0026 BNXT_FW_CAP_SW_MAX_RESOURCE_LIMITS)\n #define BNXT_MIRROR_ON_ROCE_CAP(bp)\t\\\n \t((bp)-\u003efw_cap \u0026 BNXT_FW_CAP_MIRROR_ON_ROCE)\n+#define BNXT_SUPPORTS_KTLS(bp)\t\\\n+\t((bp)-\u003efw_cap \u0026 BNXT_FW_CAP_KTLS)\n \n \tu32\t\t\thwrm_spec_code;\n \tu16\t\t\thwrm_cmd_seq;\n@@ -2856,14 +2922,14 @@ static inline u32 bnxt_tx_avail(struct bnxt *bp,\n static inline struct tx_bd_ext *\n bnxt_init_ext_bd(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n \t\t u16 prod, __le32 lflags, u32 vlan_tag_flags,\n-\t\t u32 cfa_action)\n+\t\t u32 cfa_action, u32 kid)\n {\n \tstruct tx_bd_ext *txbd1;\n \n \ttxbd1 = (struct tx_bd_ext *)\n \t\t\u0026txr-\u003etx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)];\n \ttxbd1-\u003etx_bd_hsize_lflags = lflags;\n-\ttxbd1-\u003etx_bd_mss = 0;\n+\ttxbd1-\u003etx_bd_kid_mss = cpu_to_le32(BNXT_TX_KID_HI(kid));\n \ttxbd1-\u003etx_bd_cfa_meta = cpu_to_le32(vlan_tag_flags);\n \ttxbd1-\u003etx_bd_cfa_action =\n \t\tcpu_to_le32(cfa_action \u003c\u003c TX_BD_CFA_ACTION_SHIFT);\n@@ -2962,6 +3028,8 @@ int bnxt_alloc_rx_data(struct bnxt *bp, struct bnxt_rx_ring_info *rxr,\n void bnxt_reuse_rx_data(struct bnxt_rx_ring_info *rxr, u16 cons, void *data);\n u32 bnxt_fw_health_readl(struct bnxt *bp, int reg_idx);\n bool bnxt_bs_trace_avail(struct bnxt *bp, u16 type);\n+void bnxt_free_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem);\n+int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem);\n void bnxt_set_tpa_flags(struct bnxt *bp);\n void bnxt_set_ring_params(struct bnxt *);\n void bnxt_set_rx_skb_mode(struct bnxt *bp, bool page_mode);\n@@ -2990,6 +3058,14 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic);\n int bnxt_hwrm_vnic_alloc(struct bnxt *bp, struct bnxt_vnic_info *vnic,\n \t\t\t unsigned int start_rx_ring_idx,\n \t\t\t unsigned int nr_rings);\n+int bnxt_hwrm_cp_ring_alloc_p5(struct bnxt *bp, struct bnxt_cp_ring_info *cpr);\n+int bnxt_hwrm_tx_ring_alloc(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n+\t\t\t u32 tx_idx);\n+void bnxt_hwrm_tx_ring_free(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n+\t\t\t bool close_path);\n+void bnxt_hwrm_cp_ring_free(struct bnxt *bp, struct bnxt_cp_ring_info *cpr);\n+void bnxt_clear_one_cp_ring(struct bnxt *bp, struct bnxt_cp_ring_info *cpr);\n+int bnxt_total_tx_rings(struct bnxt *bp);\n int __bnxt_hwrm_get_tx_rings(struct bnxt *bp, u16 fid, int *tx_rings);\n int bnxt_nq_rings_in_use(struct bnxt *bp);\n int bnxt_hwrm_set_coal(struct bnxt *);\n@@ -3035,6 +3111,7 @@ void bnxt_sync_ring_stats(struct bnxt *bp);\n bool bnxt_rfs_capable(struct bnxt *bp, bool new_rss_ctx);\n int bnxt_dbg_hwrm_rd_reg(struct bnxt *bp, u32 reg_off, u16 num_words,\n \t\t\t u32 *reg_buf);\n+void bnxt_reset_task(struct bnxt *bp, bool silent);\n void bnxt_fw_exception(struct bnxt *bp);\n void bnxt_fw_reset(struct bnxt *bp);\n int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs,\ndiff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c\nnew file mode 100644\nindex 00000000000000..920dda0d208684\n--- /dev/null\n+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c\n@@ -0,0 +1,613 @@\n+// SPDX-License-Identifier: GPL-2.0-only\n+/* Copyright (c) 2026 Broadcom Inc. */\n+\n+#include \u003clinux/stddef.h\u003e\n+#include \u003clinux/types.h\u003e\n+#include \u003clinux/kernel.h\u003e\n+#include \u003clinux/slab.h\u003e\n+#include \u003clinux/pci.h\u003e\n+#include \u003clinux/bnxt/hsi.h\u003e\n+\n+#include \"bnxt.h\"\n+#include \"bnxt_hwrm.h\"\n+#include \"bnxt_mpc.h\"\n+#include \"bnxt_ktls.h\"\n+#include \"bnxt_crypto.h\"\n+\n+static u32 bnxt_get_max_crypto_key_ctx(struct bnxt *bp, int key_type)\n+{\n+\tu32 fw_maj = BNXT_FW_MAJ(bp);\n+\n+\tif (key_type == BNXT_TX_CRYPTO_KEY_TYPE)\n+\t\treturn (fw_maj \u003c 233) ? BNXT_MAX_TX_CRYPTO_KEYS_PRE_233FW :\n+\t\t BNXT_MAX_TX_CRYPTO_KEYS;\n+\n+\treturn (fw_maj \u003c 233) ? BNXT_MAX_RX_CRYPTO_KEYS_PRE_233FW :\n+\t BNXT_MAX_RX_CRYPTO_KEYS;\n+}\n+\n+/**\n+ * bnxt_alloc_crypto_info - Allocate and initialize crypto offload context\n+ * @bp: pointer to bnxt device\n+ * @resp: pointer to firmware capability response\n+ *\n+ * Allocates the main crypto info structure\n+ *\n+ * This function is called during device initialization when firmware\n+ * reports crypto offload capability. If allocation fails, crypto offload\n+ * will not be available but the device will still function.\n+ *\n+ * Context: Process context\n+ */\n+void bnxt_alloc_crypto_info(struct bnxt *bp,\n+\t\t\t struct hwrm_func_qcaps_output *resp)\n+{\n+\tu16 max_keys = le16_to_cpu(resp-\u003emax_key_ctxs_alloc);\n+\tstruct bnxt_crypto_info *crypto = bp-\u003ecrypto_info;\n+\tstruct bnxt_kctx *kctx;\n+\tchar name[64];\n+\tint i;\n+\n+\tif (BNXT_VF(bp))\n+\t\treturn;\n+\tif (!crypto) {\n+\t\tcrypto = kzalloc_obj(*crypto);\n+\t\tif (!crypto) {\n+\t\t\tnetdev_warn(bp-\u003edev,\n+\t\t\t\t \"Unable to allocate crypto info\\n\");\n+\t\t\treturn;\n+\t\t}\n+\t\tsnprintf(name, sizeof(name), \"bnxt_crypto-%s\",\n+\t\t\t dev_name(\u0026bp-\u003epdev-\u003edev));\n+\t\tcrypto-\u003empc_cache =\n+\t\t\tkmem_cache_create(name,\n+\t\t\t\t\t sizeof(struct bnxt_crypto_cmd_ctx),\n+\t\t\t\t\t 0, SLAB_HWCACHE_ALIGN, NULL);\n+\t\tif (!crypto-\u003empc_cache)\n+\t\t\tgoto alloc_err;\n+\n+\t\tfor (i = 0; i \u003c BNXT_MAX_CRYPTO_KEY_TYPE; i++) {\n+\t\t\tkctx = \u0026crypto-\u003ekctx[i];\n+\t\t\tkctx-\u003etype = i;\n+\t\t\tINIT_LIST_HEAD(\u0026kctx-\u003elist);\n+\t\t\tspin_lock_init(\u0026kctx-\u003elock);\n+\t\t\tatomic_set(\u0026kctx-\u003ealloc_pending, 0);\n+\t\t\tinit_waitqueue_head(\u0026kctx-\u003ealloc_pending_wq);\n+\t\t}\n+\t\tbp-\u003ecrypto_info = crypto;\n+\t}\n+\tfor (i = 0; i \u003c BNXT_MAX_CRYPTO_KEY_TYPE; i++) {\n+\t\tkctx = \u0026crypto-\u003ekctx[i];\n+\t\tkctx-\u003emax_ctx = bnxt_get_max_crypto_key_ctx(bp, i);\n+\t}\n+\tcrypto-\u003emax_key_ctxs_alloc = max_keys;\n+\tif (!bp-\u003ektls_info)\n+\t\tbnxt_alloc_ktls_info(bp);\n+\tif (bp-\u003ektls_info)\n+\t\tbp-\u003efw_cap |= BNXT_FW_CAP_KTLS;\n+\treturn;\n+\n+alloc_err:\n+\tkfree(crypto);\n+}\n+\n+int bnxt_crypto_del(struct bnxt *bp, u8 type, u8 kind, u32 kid)\n+{\n+\tstruct bnxt_tx_ring_info *txr;\n+\tstruct ce_delete_cmd cmd = {};\n+\tu32 data;\n+\n+\tif (test_bit(BNXT_STATE_IN_FW_RESET, \u0026bp-\u003estate) \u0026\u0026\n+\t test_bit(BNXT_STATE_FW_FATAL_COND, \u0026bp-\u003estate))\n+\t\treturn 0;\n+\n+\ttxr = bnxt_select_mpc_ring(bp, type);\n+\tif (!txr)\n+\t\treturn -ENODEV;\n+\tif (kind == BNXT_CTX_KIND_CK_TX)\n+\t\tdata = CE_DELETE_CMD_CTX_KIND_CK_TX;\n+\telse if (kind == BNXT_CTX_KIND_CK_RX)\n+\t\tdata = CE_DELETE_CMD_CTX_KIND_CK_RX;\n+\telse\n+\t\treturn -EINVAL;\n+\n+\tdata |= CE_DELETE_CMD_OPCODE_DEL |\n+\t\t(BNXT_KID_HW(kid) \u003c\u003c CE_DELETE_CMD_KID_SFT);\n+\n+\tcmd.ctx_kind_kid_opcode = cpu_to_le32(data);\n+\treturn bnxt_xmit_crypto_cmd(bp, txr, \u0026cmd, sizeof(cmd),\n+\t\t\t\t BNXT_MPC_TMO_MSECS);\n+}\n+\n+static void bnxt_crypto_del_all_kids(struct bnxt *bp, struct bnxt_kid_info *kid)\n+{\n+\tint i, rc;\n+\n+\tfor (i = 0; i \u003c kid-\u003ecount; i++) {\n+\t\tif (!test_bit(i, kid-\u003eids)) {\n+\t\t\trc = bnxt_crypto_del(bp, kid-\u003etype, kid-\u003ekind,\n+\t\t\t\t\t kid-\u003estart_id + i);\n+\t\t\tif (!rc)\n+\t\t\t\tset_bit(i, kid-\u003eids);\n+\t\t}\n+\t}\n+}\n+\n+/**\n+ * bnxt_crypto_del_all - Delete all crypto connections\n+ * @bp: pointer to bnxt device\n+ *\n+ * Delete all crypto connections and free all KIDs for re-use during\n+ * shutdown. Increment the epoch counter to invalidate any outstanding\n+ * key references.\n+ *\n+ * This function assumes serialization (called during shutdown) and does\n+ * not use locking.\n+ *\n+ * Context: Process context during shutdown/reset\n+ */\n+void bnxt_crypto_del_all(struct bnxt *bp)\n+{\n+\tstruct bnxt_crypto_info *crypto = bp-\u003ecrypto_info;\n+\tstruct bnxt_kid_info *kid;\n+\tstruct bnxt_kctx *kctx;\n+\tint i;\n+\n+\tif (!crypto)\n+\t\treturn;\n+\n+\t/* Shutting down, no need to protect the lists. */\n+\tfor (i = 0; i \u003c BNXT_MAX_CRYPTO_KEY_TYPE; i++) {\n+\t\tkctx = \u0026crypto-\u003ekctx[i];\n+\t\tlist_for_each_entry(kid, \u0026kctx-\u003elist, list)\n+\t\t\tbnxt_crypto_del_all_kids(bp, kid);\n+\t\tkctx-\u003eepoch = BNXT_NEXT_EPOCH(kctx-\u003eepoch);\n+\t}\n+}\n+\n+/**\n+ * bnxt_clear_crypto - Clear all crypto key contexts\n+ * @bp: pointer to bnxt device\n+ *\n+ * Clears all key context allocations during shutdown or firmware reset.\n+ * Frees all key info structures and bitmaps, and increments the epoch\n+ * counter to invalidate any outstanding key references.\n+ *\n+ * This function assumes serialization (called during shutdown) and does\n+ * not use locking.\n+ *\n+ * Context: Process context during shutdown/reset\n+ */\n+void bnxt_clear_crypto(struct bnxt *bp)\n+{\n+\tstruct bnxt_crypto_info *crypto = bp-\u003ecrypto_info;\n+\tstruct bnxt_kid_info *kid, *tmp;\n+\tstruct bnxt_kctx *kctx;\n+\tint i;\n+\n+\tif (!crypto)\n+\t\treturn;\n+\n+\t/* Only called when shutting down or FW reset with BNXT_STATE_OPEN\n+\t * cleared, so no concurrent access. No protection needed.\n+\t */\n+\tfor (i = 0; i \u003c BNXT_MAX_CRYPTO_KEY_TYPE; i++) {\n+\t\tkctx = \u0026crypto-\u003ekctx[i];\n+\t\tlist_for_each_entry_safe(kid, tmp, \u0026kctx-\u003elist, list) {\n+\t\t\tlist_del(\u0026kid-\u003elist);\n+\t\t\tkfree(kid);\n+\t\t}\n+\t\tkctx-\u003etotal_alloc = 0;\n+\t\tkctx-\u003eepoch = BNXT_NEXT_EPOCH(kctx-\u003eepoch);\n+\t}\n+}\n+\n+/**\n+ * bnxt_free_crypto_info - Free crypto offload resources\n+ * @bp: pointer to bnxt device\n+ *\n+ * Frees all resources associated with crypto offload. Call this function\n+ * only when it is idle with nothing in-flight.\n+ *\n+ * Context: Process context during device shutdown/removal\n+ */\n+void bnxt_free_crypto_info(struct bnxt *bp)\n+{\n+\tstruct bnxt_crypto_info *crypto = bp-\u003ecrypto_info;\n+\n+\tbnxt_free_ktls_info(bp);\n+\tif (!crypto)\n+\t\treturn;\n+\tbnxt_clear_crypto(bp);\n+\tkmem_cache_destroy(crypto-\u003empc_cache);\n+\tkfree(crypto);\n+\tbp-\u003ecrypto_info = NULL;\n+\tbp-\u003efw_cap \u0026= ~BNXT_FW_CAP_KTLS;\n+}\n+\n+/**\n+ * bnxt_hwrm_reserve_pf_key_ctxs - Reserve key contexts with firmware\n+ * @bp: pointer to bnxt device\n+ * @req: pointer to HWRM function config request\n+ *\n+ * Populates the firmware request with key context reservation parameters\n+ * for crypto offload based on current max settings and capabilities.\n+ *\n+ * Context: Process context during device configuration\n+ */\n+void bnxt_hwrm_reserve_pf_key_ctxs(struct bnxt *bp,\n+\t\t\t\t struct hwrm_func_cfg_input *req)\n+{\n+\tstruct bnxt_crypto_info *crypto = bp-\u003ecrypto_info;\n+\tstruct bnxt_hw_resc *hw_resc = \u0026bp-\u003ehw_resc;\n+\tstruct bnxt_hw_crypto_resc *crypto_resc;\n+\tu32 tx, rx;\n+\n+\tif (!crypto || !BNXT_SUPPORTS_KTLS(bp))\n+\t\treturn;\n+\n+\tcrypto_resc = \u0026hw_resc-\u003ecrypto_resc;\n+\ttx = min(BNXT_TCK(crypto).max_ctx, crypto_resc-\u003emax_tx_key_ctxs);\n+\trx = min(BNXT_RCK(crypto).max_ctx, crypto_resc-\u003emax_rx_key_ctxs);\n+\treq-\u003enum_ktls_tx_key_ctxs = cpu_to_le32(tx);\n+\treq-\u003enum_ktls_rx_key_ctxs = cpu_to_le32(rx);\n+\tif (tx)\n+\t\treq-\u003eenables |= cpu_to_le32(FUNC_CFG_REQ_ENABLES_KTLS_TX_KEY_CTXS);\n+\tif (rx)\n+\t\treq-\u003eenables |= cpu_to_le32(FUNC_CFG_REQ_ENABLES_KTLS_RX_KEY_CTXS);\n+}\n+\n+static int bnxt_key_ctx_store(struct bnxt_kctx *kctx, __le32 *key_buf, u32 num,\n+\t\t\t bool contig, u8 kind, u32 *id)\n+{\n+\tstruct bnxt_kid_info *kid;\n+\tu32 i;\n+\n+\tfor (i = 0; i \u003c num; ) {\n+\t\tkid = kzalloc_obj(*kid);\n+\t\t/* If we cannot store the IDs, they will be lost and only\n+\t\t * reclaimed by the FW during reset/reinit.\n+\t\t */\n+\t\tif (!kid)\n+\t\t\treturn -ENOMEM;\n+\t\tkid-\u003estart_id = le32_to_cpu(key_buf[i]);\n+\t\tkid-\u003etype = kctx-\u003etype;\n+\t\tkid-\u003ekind = kind;\n+\t\tif (contig)\n+\t\t\tkid-\u003ecount = num;\n+\t\telse\n+\t\t\tkid-\u003ecount = 1;\n+\t\tbitmap_set(kid-\u003eids, 0, kid-\u003ecount);\n+\t\tif (id \u0026\u0026 !i) {\n+\t\t\tclear_bit(0, kid-\u003eids);\n+\t\t\t*id = BNXT_SET_KID(kctx, kid-\u003estart_id);\n+\t\t}\n+\t\tspin_lock(\u0026kctx-\u003elock);\n+\t\tlist_add_tail_rcu(\u0026kid-\u003elist, \u0026kctx-\u003elist);\n+\t\tWRITE_ONCE(kctx-\u003etotal_alloc,\n+\t\t\t READ_ONCE(kctx-\u003etotal_alloc) + kid-\u003ecount);\n+\t\tspin_unlock(\u0026kctx-\u003elock);\n+\t\ti += kid-\u003ecount;\n+\t}\n+\treturn 0;\n+}\n+\n+/* Note that the driver does not free the key contexts. They are freed\n+ * by the FW during FLR and HWRM_FUNC_RESET.\n+ */\n+static int bnxt_hwrm_key_ctx_alloc(struct bnxt *bp, struct bnxt_kctx *kctx,\n+\t\t\t\t u8 kind, u32 num, u32 *id)\n+{\n+\tstruct bnxt_crypto_info *crypto = bp-\u003ecrypto_info;\n+\tstruct hwrm_func_key_ctx_alloc_output *resp;\n+\tstruct hwrm_func_key_ctx_alloc_input *req;\n+\tdma_addr_t mapping;\n+\tint pending_count;\n+\t__le32 *key_buf;\n+\tu32 num_alloc;\n+\tbool contig;\n+\tint rc;\n+\n+\tnum = min3(num, crypto-\u003emax_key_ctxs_alloc, (u32)BNXT_KID_BATCH_SIZE);\n+\trc = hwrm_req_init(bp, req, HWRM_FUNC_KEY_CTX_ALLOC);\n+\tif (rc)\n+\t\treturn rc;\n+\n+\tkey_buf = hwrm_req_dma_slice(bp, req, num * 4, \u0026mapping);\n+\tif (!key_buf) {\n+\t\trc = -ENOMEM;\n+\t\tgoto key_alloc_exit;\n+\t}\n+\treq-\u003edma_bufr_size_bytes = cpu_to_le32(num * 4);\n+\treq-\u003ehost_dma_addr = cpu_to_le64(mapping);\n+\tresp = hwrm_req_hold(bp, req);\n+\n+\treq-\u003ekey_ctx_type = kctx-\u003etype;\n+\treq-\u003enum_key_ctxs = cpu_to_le16(num);\n+\n+\tpending_count = atomic_inc_return(\u0026kctx-\u003ealloc_pending);\n+\trc = hwrm_req_send(bp, req);\n+\tatomic_dec(\u0026kctx-\u003ealloc_pending);\n+\tif (rc)\n+\t\tgoto key_alloc_exit_wake;\n+\n+\tnum_alloc = le16_to_cpu(resp-\u003enum_key_ctxs_allocated);\n+\tif (num_alloc \u003e num) {\n+\t\tnetdev_warn(bp-\u003edev,\n+\t\t\t \"FW allocated more type %d keys (%d) than requested (%d)\\n\",\n+\t\t\t kctx-\u003etype, num_alloc, num);\n+\t} else if (!num_alloc) {\n+\t\tnetdev_warn(bp-\u003edev,\n+\t\t\t \"FW allocated 0 type %d keys\\n\", kctx-\u003etype);\n+\t\trc = -ENOENT;\n+\t\tgoto key_alloc_exit_wake;\n+\t} else {\n+\t\tnum = num_alloc;\n+\t}\n+\tcontig = resp-\u003eflags \u0026\n+\t\t FUNC_KEY_CTX_ALLOC_RESP_FLAGS_KEY_CTXS_CONTIGUOUS;\n+\trc = bnxt_key_ctx_store(kctx, key_buf, num, contig, kind, id);\n+\n+key_alloc_exit_wake:\n+\tif (pending_count \u003e= BNXT_KCTX_ALLOC_PENDING_MAX)\n+\t\twake_up_all(\u0026kctx-\u003ealloc_pending_wq);\n+key_alloc_exit:\n+\thwrm_req_drop(bp, req);\n+\treturn rc;\n+}\n+\n+bool bnxt_kid_valid(struct bnxt_kctx *kctx, u32 id)\n+{\n+\tstruct bnxt_kid_info *kid;\n+\tbool valid = false;\n+\tu32 epoch;\n+\n+\tepoch = BNXT_KID_EPOCH(id);\n+\tif (epoch != kctx-\u003eepoch)\n+\t\treturn false;\n+\n+\tid = BNXT_KID_HW(id);\n+\trcu_read_lock();\n+\tlist_for_each_entry_rcu(kid, \u0026kctx-\u003elist, list) {\n+\t\tif (id \u003e= kid-\u003estart_id \u0026\u0026 id \u003c kid-\u003estart_id + kid-\u003ecount) {\n+\t\t\tif (!test_bit(id - kid-\u003estart_id, kid-\u003eids)) {\n+\t\t\t\tvalid = true;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\t}\n+\trcu_read_unlock();\n+\treturn valid;\n+}\n+\n+static int bnxt_alloc_one_kctx(struct bnxt_kctx *kctx, u8 kind, u32 *id)\n+{\n+\tstruct bnxt_kid_info *kid;\n+\tint rc = -ENOMEM;\n+\n+\trcu_read_lock();\n+\tlist_for_each_entry_rcu(kid, \u0026kctx-\u003elist, list) {\n+\t\tu32 idx = 0;\n+\n+\t\tif (kid-\u003ekind != kind)\n+\t\t\tcontinue;\n+\t\tdo {\n+\t\t\tidx = find_next_bit(kid-\u003eids, kid-\u003ecount, idx);\n+\t\t\tif (idx \u003e= kid-\u003ecount)\n+\t\t\t\tbreak;\n+\t\t\tif (test_and_clear_bit(idx, kid-\u003eids)) {\n+\t\t\t\t*id = BNXT_SET_KID(kctx, kid-\u003estart_id + idx);\n+\t\t\t\trc = 0;\n+\t\t\t\tgoto alloc_done;\n+\t\t\t}\n+\t\t} while (1);\n+\t}\n+\n+alloc_done:\n+\trcu_read_unlock();\n+\treturn rc;\n+}\n+\n+/**\n+ * bnxt_free_one_kctx - Free a key context for later re-use\n+ * @kctx: pointer to bnxt_kctx key context structure\n+ * @id: Key context ID\n+ *\n+ * This function is called to free a key context ID when the offload\n+ * using the ID has successfully terminated or aborted. If the offload\n+ * cannot be terminated, the caller should not call this function to free\n+ * the ID. The ID will only be recycled by the FW during reset/reinit.\n+ */\n+void bnxt_free_one_kctx(struct bnxt_kctx *kctx, u32 id)\n+{\n+\tstruct bnxt_kid_info *kid;\n+\n+\tid = BNXT_KID_HW(id);\n+\trcu_read_lock();\n+\tlist_for_each_entry_rcu(kid, \u0026kctx-\u003elist, list) {\n+\t\tif (id \u003e= kid-\u003estart_id \u0026\u0026 id \u003c kid-\u003estart_id + kid-\u003ecount) {\n+\t\t\tset_bit(id - kid-\u003estart_id, kid-\u003eids);\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\trcu_read_unlock();\n+}\n+\n+#define BNXT_KCTX_ALLOC_RETRY_MAX\t3\n+\n+int bnxt_key_ctx_alloc_one(struct bnxt *bp, struct bnxt_kctx *kctx, u8 kind,\n+\t\t\t u32 *id)\n+{\n+\tint rc, retry = 0;\n+\n+\twhile (retry++ \u003c BNXT_KCTX_ALLOC_RETRY_MAX) {\n+\t\trc = bnxt_alloc_one_kctx(kctx, kind, id);\n+\t\tif (!rc)\n+\t\t\treturn 0;\n+\n+\t\t/* When approaching the max, multiple threads may proceed\n+\t\t * and exceed the max. Some may fail the serialized HWRM call\n+\t\t * later when the max is exceeded.\n+\t\t */\n+\t\tif ((READ_ONCE(kctx-\u003etotal_alloc) + BNXT_KID_BATCH_SIZE) \u003e\n+\t\t kctx-\u003emax_ctx)\n+\t\t\treturn -ENOSPC;\n+\n+\t\tif (!BNXT_KCTX_ALLOC_OK(kctx)) {\n+\t\t\twait_event(kctx-\u003ealloc_pending_wq,\n+\t\t\t\t BNXT_KCTX_ALLOC_OK(kctx));\n+\t\t\tcontinue;\n+\t\t}\n+\t\trc = bnxt_hwrm_key_ctx_alloc(bp, kctx, kind,\n+\t\t\t\t\t BNXT_KID_BATCH_SIZE, id);\n+\t\tif (!rc)\n+\t\t\treturn 0;\n+\t}\n+\treturn -EAGAIN;\n+}\n+\n+#define BNXT_XMIT_CRYPTO_RETRY_MAX\t10\n+#define BNXT_XMIT_CRYPTO_MIN_TMO\t100\n+#define BNXT_XMIT_CRYPTO_MAX_TMO\t150\n+\n+int bnxt_xmit_crypto_cmd(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n+\t\t\t void *cmd, unsigned int len, unsigned int tmo)\n+{\n+\tstruct bnxt_crypto_info *crypto = bp-\u003ecrypto_info;\n+\tstruct bnxt_crypto_cmd_ctx *ctx = NULL;\n+\tunsigned long tmo_left, handle = 0;\n+\tint rc, retry = 0;\n+\n+\tif (tmo) {\n+\t\tu32 kid = CE_CMD_KID(cmd);\n+\n+\t\tctx = kmem_cache_alloc(crypto-\u003empc_cache, GFP_KERNEL);\n+\t\tif (!ctx)\n+\t\t\treturn -ENOMEM;\n+\t\tinit_completion(\u0026ctx-\u003ecmp);\n+\t\thandle = (unsigned long)ctx;\n+\t\tctx-\u003ekid = kid;\n+\t\tctx-\u003eclient = txr-\u003etx_ring_struct.mpc_chnl_type;\n+\t\tctx-\u003estatus = 0;\n+\t\t/* One reference for this caller, one for the handle stored in\n+\t\t * the TX buf ring. The latter is dropped by\n+\t\t * bnxt_crypto_mpc_cmp() when the command is completed normally\n+\t\t * or after timeout.\n+\t\t */\n+\t\trefcount_set(\u0026ctx-\u003erefcnt, 2);\n+\t\tretry = BNXT_XMIT_CRYPTO_RETRY_MAX;\n+\t\tmight_sleep();\n+\t}\n+\tdo {\n+\t\tspin_lock_bh(\u0026txr-\u003etx_lock);\n+\t\trc = bnxt_start_xmit_mpc(bp, txr, cmd, len, handle);\n+\t\tspin_unlock_bh(\u0026txr-\u003etx_lock);\n+\t\tif (rc == -EBUSY \u0026\u0026 tmo \u0026\u0026 retry)\n+\t\t\tusleep_range(BNXT_XMIT_CRYPTO_MIN_TMO,\n+\t\t\t\t BNXT_XMIT_CRYPTO_MAX_TMO);\n+\t\telse\n+\t\t\tbreak;\n+\t} while (retry--);\n+\tif (rc || !tmo) {\n+\t\t/* The completion will never arrive, drop one reference */\n+\t\tif (ctx)\n+\t\t\trefcount_dec(\u0026ctx-\u003erefcnt);\n+\t\tgoto xmit_done;\n+\t}\n+\n+\ttmo_left = wait_for_completion_timeout(\u0026ctx-\u003ecmp, msecs_to_jiffies(tmo));\n+\tif (!tmo_left) {\n+\t\tnetdev_warn(bp-\u003edev, \"crypto MP cmd %08x timed out\\n\",\n+\t\t\t *((u32 *)cmd));\n+\t\tbnxt_mpc_timeout(bp, txr);\n+\t\trc = -ETIMEDOUT;\n+\t\tgoto xmit_done;\n+\t}\n+\tif (ctx-\u003estatus == BNXT_CMD_CTX_COMPLETED \u0026\u0026\n+\t CE_CMPL_STATUS(\u0026ctx-\u003ece_cmp) == CE_CMPL_STATUS_OK)\n+\t\trc = 0;\n+\telse\n+\t\trc = -EIO;\n+xmit_done:\n+\tif (rc) {\n+\t\tu8 status = ctx ? ctx-\u003estatus : 0;\n+\n+\t\tnetdev_warn(bp-\u003edev,\n+\t\t\t \"MPC transmit failed, ring idx %d, op 0x%x, kid 0x%x, status 0x%x\\n\",\n+\t\t\t txr-\u003ebnapi-\u003eindex, CE_CMD_OP(cmd), CE_CMD_KID(cmd),\n+\t\t\t status);\n+\t}\n+\tif (ctx \u0026\u0026 refcount_dec_and_test(\u0026ctx-\u003erefcnt))\n+\t\tkmem_cache_free(crypto-\u003empc_cache, ctx);\n+\treturn rc;\n+}\n+\n+int bnxt_crypto_init(struct bnxt *bp)\n+{\n+\tstruct bnxt_crypto_info *crypto = bp-\u003ecrypto_info;\n+\tstruct bnxt_hw_resc *hw_resc = \u0026bp-\u003ehw_resc;\n+\tstruct bnxt_hw_crypto_resc *crypto_resc;\n+\tint rc;\n+\n+\tif (!crypto || !BNXT_SUPPORTS_KTLS(bp))\n+\t\treturn 0;\n+\n+\tcrypto_resc = \u0026hw_resc-\u003ecrypto_resc;\n+\tBNXT_TCK(crypto).max_ctx = crypto_resc-\u003eresv_tx_key_ctxs;\n+\tBNXT_RCK(crypto).max_ctx = crypto_resc-\u003eresv_rx_key_ctxs;\n+\n+\tif (!BNXT_TCK(crypto).max_ctx || !BNXT_RCK(crypto).max_ctx)\n+\t\treturn -ENODEV;\n+\n+\trc = bnxt_hwrm_key_ctx_alloc(bp, \u0026BNXT_TCK(crypto), BNXT_CTX_KIND_CK_TX,\n+\t\t\t\t BNXT_KID_BATCH_SIZE, NULL);\n+\tif (rc)\n+\t\treturn rc;\n+\n+\trc = bnxt_hwrm_key_ctx_alloc(bp, \u0026BNXT_RCK(crypto), BNXT_CTX_KIND_CK_RX,\n+\t\t\t\t BNXT_KID_BATCH_SIZE, NULL);\n+\tif (rc)\n+\t\treturn rc;\n+\n+\tbnxt_ktls_init(bp);\n+\treturn 0;\n+}\n+\n+void bnxt_crypto_mpc_cmp(struct bnxt *bp, u32 client, unsigned long handle,\n+\t\t\t struct bnxt_cmpl_entry cmpl[], u32 entries)\n+{\n+\tstruct bnxt_crypto_cmd_ctx *ctx;\n+\tstruct ce_cmpl *cmp = NULL;\n+\tu32 len, kid;\n+\n+\tif (likely(cmpl))\n+\t\tcmp = cmpl[0].cmpl;\n+\tif (!handle || entries != 1) {\n+\t\tif (entries != 1 \u0026\u0026 cmpl) {\n+\t\t\tnetdev_warn(bp-\u003edev, \"Invalid entries %d with handle %lx cmpl %08x in %s()\\n\",\n+\t\t\t\t entries, handle, *(u32 *)cmp, __func__);\n+\t\t}\n+\t\tif (!handle)\n+\t\t\treturn;\n+\t}\n+\tctx = (void *)handle;\n+\tctx-\u003estatus = BNXT_CMD_CTX_COMPLETED;\n+\tif (unlikely(!cmpl)) {\n+\t\tctx-\u003estatus |= BNXT_CMD_CTX_RESET;\n+\t\tgoto cmp_done;\n+\t}\n+\tkid = CE_CMPL_KID(cmp);\n+\tif (ctx-\u003ekid != kid || ctx-\u003eclient != client || entries != 1) {\n+\t\tnetdev_warn(bp-\u003edev,\n+\t\t\t \"Invalid CE cmpl 0x%08x with entries %d for client %d with status 0x%x, expected kid 0x%x and client %d\\n\",\n+\t\t\t *(u32 *)cmp, entries, client, ctx-\u003estatus, ctx-\u003ekid,\n+\t\t\t ctx-\u003eclient);\n+\t\tctx-\u003estatus |= BNXT_CMD_CTX_ERROR;\n+\t}\n+\tlen = min_t(u32, cmpl[0].len, sizeof(ctx-\u003ece_cmp));\n+\tmemcpy(\u0026ctx-\u003ece_cmp, cmpl[0].cmpl, len);\n+cmp_done:\n+\tcomplete(\u0026ctx-\u003ecmp);\n+\tif (refcount_dec_and_test(\u0026ctx-\u003erefcnt))\n+\t\tkmem_cache_free(bp-\u003ecrypto_info-\u003empc_cache, ctx);\n+}\ndiff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h\nnew file mode 100644\nindex 00000000000000..a4571369fd23ca\n--- /dev/null\n+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h\n@@ -0,0 +1,229 @@\n+/* SPDX-License-Identifier: GPL-2.0-only */\n+/* Copyright (c) 2026 Broadcom Inc. */\n+\n+#ifndef BNXT_CRYPTO_H\n+#define BNXT_CRYPTO_H\n+\n+#include \u003clinux/refcount.h\u003e\n+\n+#define BNXT_MAX_TX_CRYPTO_KEYS\t\t204800\n+#define BNXT_MAX_RX_CRYPTO_KEYS\t\t204800\n+\n+#define BNXT_MAX_TX_CRYPTO_KEYS_PRE_233FW\t65535\n+#define BNXT_MAX_RX_CRYPTO_KEYS_PRE_233FW\t65535\n+\n+enum bnxt_crypto_type {\n+\tBNXT_TX_CRYPTO_KEY_TYPE = FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_TX,\n+\tBNXT_RX_CRYPTO_KEY_TYPE = FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_RX,\n+\tBNXT_MAX_CRYPTO_KEY_TYPE,\n+};\n+\n+#define BNXT_KID_BATCH_SIZE\t128\n+\n+struct bnxt_kid_info {\n+\tstruct list_head\tlist;\n+\tu8\t\t\ttype;\n+\tu8\t\t\tkind;\n+\tu32\t\t\tstart_id;\n+\tu32\t\t\tcount;\n+\tDECLARE_BITMAP(ids, BNXT_KID_BATCH_SIZE);\n+};\n+\n+struct bnxt_kctx {\n+\tstruct list_head\tlist;\n+\t/* to serialize update to the linked list and total_alloc */\n+\tspinlock_t\t\tlock;\n+\tu8\t\t\ttype;\n+\tu16\t\t\tepoch;\n+\tu32\t\t\ttotal_alloc;\n+\tu32\t\t\tmax_ctx;\n+\tatomic_t\t\talloc_pending;\n+#define BNXT_KCTX_ALLOC_PENDING_MAX\t8\n+\twait_queue_head_t\talloc_pending_wq;\n+};\n+\n+#define BNXT_KID_HW_MASK\t0x000fffff\n+#define BNXT_KID_HW(kid)\t((kid) \u0026 BNXT_KID_HW_MASK)\n+#define BNXT_KID_EPOCH_MASK\t0xfff00000\n+#define BNXT_KID_EPOCH_SHIFT\t20\n+#define BNXT_KID_EPOCH(kid)\t(((kid) \u0026 BNXT_KID_EPOCH_MASK) \u003e\u003e\t\\\n+\t\t\t\t BNXT_KID_EPOCH_SHIFT)\n+\n+#define BNXT_NEXT_EPOCH(epoch)\t\\\n+\t(((epoch) + 1) \u0026 (BNXT_KID_EPOCH_MASK \u003e\u003e BNXT_KID_EPOCH_SHIFT))\n+\n+#define BNXT_SET_KID(kctx, kid)\t\t\t\t\t\t\\\n+\t((kid) | ((u32)(kctx)-\u003eepoch \u003c\u003c BNXT_KID_EPOCH_SHIFT))\n+\n+#define BNXT_KCTX_ALLOC_OK(kctx)\t\\\n+\t(atomic_read(\u0026((kctx)-\u003ealloc_pending)) \u003c BNXT_KCTX_ALLOC_PENDING_MAX)\n+\n+struct bnxt_crypto_info {\n+\tu16\t\t\tmax_key_ctxs_alloc;\n+\n+\tstruct bnxt_kctx\tkctx[BNXT_MAX_CRYPTO_KEY_TYPE];\n+\n+\tstruct kmem_cache\t*mpc_cache;\n+};\n+\n+struct ce_delete_cmd {\n+\t__le32 ctx_kind_kid_opcode;\n+\t#define CE_DELETE_CMD_OPCODE_MASK\t\t0xfUL\n+\t#define CE_DELETE_CMD_OPCODE_SFT\t\t0\n+\t#define CE_DELETE_CMD_OPCODE_DEL\t\t 0x2UL\n+\t#define CE_DELETE_CMD_KID_MASK\t\t\t0xfffff0UL\n+\t#define CE_DELETE_CMD_KID_SFT\t\t\t4\n+\t#define CE_DELETE_CMD_CTX_KIND_MASK\t\t0x1f000000UL\n+\t#define CE_DELETE_CMD_CTX_KIND_SFT\t\t24\n+\t#define CE_DELETE_CMD_CTX_KIND_CK_TX\t\t (0x11UL \u003c\u003c 24)\n+\t#define CE_DELETE_CMD_CTX_KIND_CK_RX\t\t (0x12UL \u003c\u003c 24)\n+};\n+\n+#define CE_CMD_OP_MASK\t\t\t0x00000fU\n+#define CE_CMD_KID_MASK\t\t\t0xfffff0U\n+#define CE_CMD_KID_SFT\t\t\t4\n+\n+#define CE_CMD_OP(cmd_p)\t\t\t\t\t\\\n+\t(le32_to_cpu(*(__le32 *)(cmd_p)) \u0026 CE_CMD_OP_MASK)\n+\n+#define CE_CMD_KID(cmd_p)\t\t\t\t\t\\\n+\t((le32_to_cpu(*(__le32 *)(cmd_p)) \u0026 CE_CMD_KID_MASK) \u003e\u003e CE_CMD_KID_SFT)\n+\n+struct ce_cmpl {\n+\t__le16\tclient_subtype_type;\n+\t#define CE_CMPL_TYPE_MASK\t\t\t0x3fUL\n+\t#define CE_CMPL_TYPE_SFT\t\t\t0\n+\t#define CE_CMPL_TYPE_MID_PATH_SHORT\t\t 0x1eUL\n+\t#define CE_CMPL_SUBTYPE_MASK\t\t\t0xf00UL\n+\t#define CE_CMPL_SUBTYPE_SFT\t\t\t8\n+\t#define CE_CMPL_SUBTYPE_SOLICITED\t\t (0x0UL \u003c\u003c 8)\n+\t#define CE_CMPL_SUBTYPE_ERR\t\t\t (0x1UL \u003c\u003c 8)\n+\t#define CE_CMPL_SUBTYPE_RESYNC\t\t\t (0x2UL \u003c\u003c 8)\n+\t#define CE_CMPL_MP_CLIENT_MASK\t\t\t0xf000UL\n+\t#define CE_CMPL_MP_CLIENT_SFT\t\t\t12\n+\t#define CE_CMPL_MP_CLIENT_TCE\t\t\t (0x0UL \u003c\u003c 12)\n+\t#define CE_CMPL_MP_CLIENT_RCE\t\t\t (0x1UL \u003c\u003c 12)\n+\t__le16\tstatus;\n+\t#define CE_CMPL_STATUS_MASK\t\t\t0xfUL\n+\t#define CE_CMPL_STATUS_SFT\t\t\t0\n+\t#define CE_CMPL_STATUS_OK\t\t\t 0x0UL\n+\t#define CE_CMPL_STATUS_CTX_LD_ERR\t\t 0x1UL\n+\t#define CE_CMPL_STATUS_FID_CHK_ERR\t\t 0x2UL\n+\t#define CE_CMPL_STATUS_CTX_VER_ERR\t\t 0x3UL\n+\t#define CE_CMPL_STATUS_DST_ID_ERR\t\t 0x4UL\n+\t#define CE_CMPL_STATUS_MP_CMD_ERR\t\t 0x5UL\n+\tu32\topaque;\n+\t__le32\tv;\n+\t#define CE_CMPL_V 0x1UL\n+\t__le32\tkid;\n+\t#define CE_CMPL_KID_MASK 0xfffffUL\n+\t#define CE_CMPL_KID_SFT 0\n+};\n+\n+#define CE_CMPL_STATUS(ce_cmpl)\t\t\t\t\t\t\\\n+\t(le16_to_cpu((ce_cmpl)-\u003estatus) \u0026 CE_CMPL_STATUS_MASK)\n+\n+#define CE_CMPL_KID(ce_cmpl)\t\t\t\t\t\t\\\n+\t(le32_to_cpu((ce_cmpl)-\u003ekid) \u0026 CE_CMPL_KID_MASK)\n+\n+struct bnxt_crypto_cmd_ctx {\n+\tstruct completion cmp;\n+\tstruct ce_cmpl ce_cmp;\n+\trefcount_t refcnt;\n+\tu32 kid;\n+\tu16 client;\n+\tu8 status;\n+#define BNXT_CMD_CTX_COMPLETED\t0x1\n+#define BNXT_CMD_CTX_ERROR\t0x2\n+#define BNXT_CMD_CTX_RESET\t0x4\n+};\n+\n+#define BNXT_TCK(crypto)\t((crypto)-\u003ekctx[BNXT_TX_CRYPTO_KEY_TYPE])\n+#define BNXT_RCK(crypto)\t((crypto)-\u003ekctx[BNXT_RX_CRYPTO_KEY_TYPE])\n+\n+#define BNXT_CTX_KIND_CK_TX\t0x11\n+#define BNXT_CTX_KIND_CK_RX\t0x12\n+\n+#ifdef CONFIG_BNXT_TLS\n+void bnxt_alloc_crypto_info(struct bnxt *bp,\n+\t\t\t struct hwrm_func_qcaps_output *resp);\n+int bnxt_crypto_del(struct bnxt *bp, u8 type, u8 kind, u32 kid);\n+void bnxt_crypto_del_all(struct bnxt *bp);\n+void bnxt_clear_crypto(struct bnxt *bp);\n+void bnxt_free_crypto_info(struct bnxt *bp);\n+void bnxt_hwrm_reserve_pf_key_ctxs(struct bnxt *bp,\n+\t\t\t\t struct hwrm_func_cfg_input *req);\n+bool bnxt_kid_valid(struct bnxt_kctx *kctx, u32 id);\n+void bnxt_free_one_kctx(struct bnxt_kctx *kctx, u32 id);\n+int bnxt_key_ctx_alloc_one(struct bnxt *bp, struct bnxt_kctx *kctx, u8 kind,\n+\t\t\t u32 *id);\n+int bnxt_xmit_crypto_cmd(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n+\t\t\t void *cmd, unsigned int len, unsigned int tmo);\n+int bnxt_crypto_init(struct bnxt *bp);\n+void bnxt_crypto_mpc_cmp(struct bnxt *bp, u32 client, unsigned long handle,\n+\t\t\t struct bnxt_cmpl_entry cmpl[], u32 entries);\n+#else\n+static inline void bnxt_alloc_crypto_info(struct bnxt *bp,\n+\t\t\t\t\t struct hwrm_func_qcaps_output *resp)\n+{\n+}\n+\n+static inline int bnxt_crypto_del(struct bnxt *bp, u8 type, u8 kind, u32 kid)\n+{\n+\treturn -EOPNOTSUPP;\n+}\n+\n+static inline void bnxt_crypto_del_all(struct bnxt *bp)\n+{\n+}\n+\n+static inline void bnxt_clear_crypto(struct bnxt *bp)\n+{\n+}\n+\n+static inline void bnxt_free_crypto_info(struct bnxt *bp)\n+{\n+}\n+\n+static inline void bnxt_hwrm_reserve_pf_key_ctxs(struct bnxt *bp,\n+\t\t\t\t\t\t struct hwrm_func_cfg_input *req)\n+{\n+}\n+\n+static inline bool bnxt_kid_valid(struct bnxt_kctx *kctx, u32 id)\n+{\n+\treturn false;\n+}\n+\n+static inline void bnxt_free_one_kctx(struct bnxt_kctx *kctx, u32 id)\n+{\n+}\n+\n+static inline int bnxt_key_ctx_alloc_one(struct bnxt *bp,\n+\t\t\t\t\t struct bnxt_kctx *kctx, u8 kind,\n+\t\t\t\t\t u32 *id)\n+{\n+\treturn -EOPNOTSUPP;\n+}\n+\n+static inline int bnxt_xmit_crypto_cmd(struct bnxt *bp,\n+\t\t\t\t struct bnxt_tx_ring_info *txr,\n+\t\t\t\t void *cmd, unsigned int len,\n+\t\t\t\t unsigned int tmo)\n+{\n+\treturn -EOPNOTSUPP;\n+}\n+\n+static inline int bnxt_crypto_init(struct bnxt *bp)\n+{\n+\treturn 0;\n+}\n+\n+static inline void bnxt_crypto_mpc_cmp(struct bnxt *bp, u32 client,\n+\t\t\t\t unsigned long handle,\n+\t\t\t\t struct bnxt_cmpl_entry cmpl[],\n+\t\t\t\t u32 entries)\n+{\n+}\n+#endif\t/* CONFIG_BNXT_TLS */\n+#endif\t/* BNXT_CRYPTO_H */\ndiff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c\nindex 62bc9cae613c38..96083caa834c4d 100644\n--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c\n+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c\n@@ -37,6 +37,8 @@\n #include \"bnxt_nvm_defs.h\"\t/* NVRAM content constant and structure defs */\n #include \"bnxt_fw_hdr.h\"\t/* Firmware hdr constant and structure defs */\n #include \"bnxt_coredump.h\"\n+#include \"bnxt_mpc.h\"\n+#include \"bnxt_ktls.h\"\n \n #define BNXT_NVM_ERR_MSG(dev, extack, msg)\t\t\t\\\n \tdo {\t\t\t\t\t\t\t\\\n@@ -353,6 +355,30 @@ static const char *const bnxt_ring_drv_stats_arr[] = {\n \t\"total_missed_irqs\",\n };\n \n+/* kTLS data plane counter strings indexed by enum bnxt_ktls_data_counters */\n+static const char *const bnxt_ktls_data_stats[] = {\n+\t[BNXT_KTLS_TX_PKTS]\t\t= \"tx_tls_encrypted_packets\",\n+\t[BNXT_KTLS_TX_BYTES]\t\t= \"tx_tls_encrypted_bytes\",\n+\t[BNXT_KTLS_TX_OOO_PKTS]\t\t\t= \"tx_tls_ooo_packets\",\n+\t[BNXT_KTLS_TX_OOO_FALLBACK_NO_SYNC]\t= \"tx_tls_ooo_fallback_no_sync\",\n+\t[BNXT_KTLS_TX_OOO_FALLBACK_NO_SPACE]\t= \"tx_tls_ooo_fallback_no_space\",\n+\t[BNXT_KTLS_TX_OOO_FALLBACK_NO_HDR]\t= \"tx_tls_ooo_fallback_no_hdr\",\n+};\n+\n+/* kTLS control plane counter strings indexed by enum bnxt_ktls_ctrl_counters */\n+static const char *const bnxt_ktls_ctrl_stats[] = {\n+\t[BNXT_KTLS_TX_ADD]\t\t\t= \"tx_tls_ctx\",\n+\t[BNXT_KTLS_TX_DEL]\t\t\t= \"tx_tls_del\",\n+\t[BNXT_KTLS_ERR_NO_MEM]\t\t\t= \"tls_err_no_mem\",\n+\t[BNXT_KTLS_ERR_NO_CAP]\t\t\t= \"tls_err_no_cap\",\n+\t[BNXT_KTLS_ERR_KEY_CTX_ALLOC]\t\t= \"tls_err_key_ctx_alloc\",\n+\t[BNXT_KTLS_ERR_CRYPTO_CMD]\t\t= \"tls_err_crypto_cmd\",\n+\t[BNXT_KTLS_ERR_DEVICE_BUSY]\t\t= \"tls_err_device_busy\",\n+\t[BNXT_KTLS_ERR_INVALID_CIPHER]\t\t= \"tls_err_invalid_cipher\",\n+\t[BNXT_KTLS_ERR_STATE_NOT_OPEN]\t\t= \"tls_err_state_not_open\",\n+\t[BNXT_KTLS_ERR_RETRY_EXCEEDED]\t\t= \"tls_err_retry_exceeded\",\n+};\n+\n #define NUM_RING_RX_SW_STATS\t\tARRAY_SIZE(bnxt_rx_sw_stats_str)\n #define NUM_RING_CMN_SW_STATS\t\tARRAY_SIZE(bnxt_cmn_sw_stats_str)\n #define NUM_RING_RX_HW_STATS\t\tARRAY_SIZE(bnxt_ring_rx_stats_str)\n@@ -535,12 +561,21 @@ static int bnxt_get_num_ring_stats(struct bnxt *bp)\n \t cmn * bp-\u003ecp_nr_rings;\n }\n \n+static int bnxt_get_num_ktls_stats(struct bnxt *bp)\n+{\n+\tif (!bp-\u003ektls_info)\n+\t\treturn 0;\n+\treturn ARRAY_SIZE(bnxt_ktls_ctrl_stats) +\n+\t ARRAY_SIZE(bnxt_ktls_data_stats);\n+}\n+\n static int bnxt_get_num_stats(struct bnxt *bp)\n {\n \tint num_stats = bnxt_get_num_ring_stats(bp);\n \tint len;\n \n \tnum_stats += BNXT_NUM_RING_DRV_STATS;\n+\tnum_stats += bnxt_get_num_ktls_stats(bp);\n \n \tif (bp-\u003eflags \u0026 BNXT_FLAG_PORT_STATS)\n \t\tnum_stats += BNXT_NUM_PORT_STATS;\n@@ -653,6 +688,16 @@ static void bnxt_get_ethtool_stats(struct net_device *dev,\n \tfor (i = 0; i \u003c BNXT_NUM_RING_DRV_STATS; i++, j++, curr++, prev++)\n \t\tbuf[j] = *curr + *prev;\n \n+\tif (bp-\u003ektls_info) {\n+\t\tstruct bnxt_tls_info *ktls = bp-\u003ektls_info;\n+\t\tstruct bnxt_tls_sw_stats tls_stats = {};\n+\n+\t\tbnxt_get_ring_tls_stats(bp, \u0026tls_stats);\n+\t\tfor (i = 0; i \u003c ARRAY_SIZE(bnxt_ktls_data_stats); i++, j++)\n+\t\t\tbuf[j] = tls_stats.counters[i];\n+\t\tfor (i = 0; i \u003c ARRAY_SIZE(bnxt_ktls_ctrl_stats); i++, j++)\n+\t\t\tbuf[j] = atomic64_read(\u0026ktls-\u003ecounters[i]);\n+\t}\n \tif (bp-\u003eflags \u0026 BNXT_FLAG_PORT_STATS) {\n \t\tu64 *port_stats = bp-\u003eport_stats.sw_stats;\n \n@@ -763,6 +808,12 @@ static void bnxt_get_strings(struct net_device *dev, u32 stringset, u8 *buf)\n \t\tfor (i = 0; i \u003c BNXT_NUM_RING_DRV_STATS; i++)\n \t\t\tethtool_puts(\u0026buf, bnxt_ring_drv_stats_arr[i]);\n \n+\t\tif (bp-\u003ektls_info) {\n+\t\t\tfor (i = 0; i \u003c ARRAY_SIZE(bnxt_ktls_data_stats); i++)\n+\t\t\t\tethtool_puts(\u0026buf, bnxt_ktls_data_stats[i]);\n+\t\t\tfor (i = 0; i \u003c ARRAY_SIZE(bnxt_ktls_ctrl_stats); i++)\n+\t\t\t\tethtool_puts(\u0026buf, bnxt_ktls_ctrl_stats[i]);\n+\t\t}\n \t\tif (bp-\u003eflags \u0026 BNXT_FLAG_PORT_STATS)\n \t\t\tfor (i = 0; i \u003c BNXT_NUM_PORT_STATS; i++) {\n \t\t\t\tstr = bnxt_port_stats_arr[i].string;\n@@ -1051,6 +1102,8 @@ static int bnxt_set_channels(struct net_device *dev,\n \n \tbnxt_set_cp_rings(bp, sh);\n \n+\tbnxt_set_dflt_mpc_rings(bp);\n+\n \t/* After changing number of rx channels, update NTUPLE feature. */\n \tnetdev_update_features(dev);\n \tif (netif_running(dev)) {\ndiff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c\nindex f317f60414e813..b4c37a6c9f0fa9 100644\n--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c\n+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c\n@@ -168,7 +168,7 @@ netdev_tx_t bnxt_sw_udp_gso_xmit(struct bnxt *bp,\n \n \t\tprod = NEXT_TX(prod);\n \t\tbnxt_init_ext_bd(bp, txr, prod, csum,\n-\t\t\t\t vlan_tag_flags, cfa_action);\n+\t\t\t\t vlan_tag_flags, cfa_action, 0);\n \n \t\t/* set dma_unmap_len on the LAST BD touching each\n \t\t * region. Since completions are in-order, the last segment\ndiff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c\nnew file mode 100644\nindex 00000000000000..b47075f2b379ee\n--- /dev/null\n+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c\n@@ -0,0 +1,572 @@\n+// SPDX-License-Identifier: GPL-2.0-only\n+/* Copyright (c) 2026 Broadcom Inc. */\n+\n+#include \u003clinux/tcp.h\u003e\n+#include \u003cnet/tls.h\u003e\n+#include \u003clinux/bnxt/hsi.h\u003e\n+\n+#include \"bnxt.h\"\n+#include \"bnxt_mpc.h\"\n+#include \"bnxt_crypto.h\"\n+#include \"bnxt_ktls.h\"\n+\n+/**\n+ * bnxt_alloc_ktls_info - Allocate and initialize kTLS offload context\n+ * @bp: pointer to bnxt device\n+ *\n+ * Allocates the main kTLS crypto info structure\n+ *\n+ * This function is called during device initialization when firmware\n+ * reports kTLS offload capability. If allocation fails, kTLS offload\n+ * will not be available but the device will still function.\n+ *\n+ * Context: Process context\n+ *\n+ * Return: zero on success, negative error code otherwise:\n+ *\tENOMEM: out of memory\n+ */\n+int bnxt_alloc_ktls_info(struct bnxt *bp)\n+{\n+\tstruct bnxt_tls_info *ktls = bp-\u003ektls_info;\n+\n+\tif (BNXT_VF(bp))\n+\t\treturn -EOPNOTSUPP;\n+\tif (ktls)\n+\t\treturn 0;\n+\n+\tktls = kzalloc_obj(*ktls);\n+\tif (!ktls) {\n+\t\tnetdev_warn(bp-\u003edev, \"Unable to allocate kTLS info\\n\");\n+\t\treturn -ENOMEM;\n+\t}\n+\tktls-\u003ecounters = kzalloc_objs(*ktls-\u003ecounters,\n+\t\t\t\t BNXT_KTLS_MAX_CTRL_COUNTERS);\n+\tif (!ktls-\u003ecounters)\n+\t\tgoto ktls_err;\n+\n+\tinit_waitqueue_head(\u0026ktls-\u003eopen_wq);\n+\tbp-\u003ektls_info = ktls;\n+\treturn 0;\n+\n+ktls_err:\n+\tkfree(ktls-\u003ecounters);\n+\tkfree(ktls);\n+\treturn -ENOMEM;\n+}\n+\n+/**\n+ * bnxt_free_ktls_info - Free kTLS crypto offload resources\n+ * @bp: pointer to bnxt device\n+ *\n+ * Frees all resources associated with kTLS crypto offload\n+ *\n+ * Context: Process context during device shutdown/removal\n+ */\n+void bnxt_free_ktls_info(struct bnxt *bp)\n+{\n+\tstruct bnxt_tls_info *ktls = bp-\u003ektls_info;\n+\n+\tif (!ktls)\n+\t\treturn;\n+\tkfree(ktls-\u003ecounters);\n+\tkfree(ktls);\n+\tbp-\u003ektls_info = NULL;\n+}\n+\n+/* Copy in reverse byte order */\n+static void bnxt_copy_tls_mp_data(u8 *dst, u8 *src, int bytes)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i \u003c bytes; i++)\n+\t\tdst[bytes - i - 1] = src[i];\n+}\n+\n+static int bnxt_crypto_add(struct bnxt *bp, enum tls_offload_ctx_dir direction,\n+\t\t\t struct tls_crypto_info *crypto_info, u32 tcp_seq_no,\n+\t\t\t u32 kid)\n+{\n+\tstruct bnxt_tx_ring_info *txr;\n+\tstruct ce_add_cmd cmd = {0};\n+\tu32 data;\n+\tint rc;\n+\n+\tif (direction == TLS_OFFLOAD_CTX_DIR_TX) {\n+\t\ttxr = bnxt_select_mpc_ring(bp, BNXT_MPC_TCE_TYPE);\n+\t\tcmd.ctx_kind = CE_ADD_CMD_CTX_KIND_CK_TX;\n+\t} else {\n+\t\treturn -EOPNOTSUPP;\n+\t}\n+\tif (!txr)\n+\t\treturn -ENODEV;\n+\n+\tdata = CE_ADD_CMD_OPCODE_ADD | (BNXT_KID_HW(kid) \u003c\u003c CE_ADD_CMD_KID_SFT);\n+\tswitch (crypto_info-\u003ecipher_type) {\n+\tcase TLS_CIPHER_AES_GCM_128: {\n+\t\tstruct tls12_crypto_info_aes_gcm_128 *aes;\n+\n+\t\taes = (void *)crypto_info;\n+\t\tdata |= CE_ADD_CMD_ALGORITHM_AES_GCM_128;\n+\t\tif (crypto_info-\u003eversion == TLS_1_3_VERSION)\n+\t\t\tdata |= CE_ADD_CMD_VERSION_TLS1_3;\n+\t\tmemcpy(\u0026cmd.session_key, aes-\u003ekey, sizeof(aes-\u003ekey));\n+\t\tmemcpy(\u0026cmd.salt, aes-\u003esalt, sizeof(aes-\u003esalt));\n+\t\tmemcpy(\u0026cmd.addl_iv, aes-\u003eiv, sizeof(aes-\u003eiv));\n+\t\tbnxt_copy_tls_mp_data(cmd.record_seq_num, aes-\u003erec_seq,\n+\t\t\t\t sizeof(aes-\u003erec_seq));\n+\t\tbreak;\n+\t}\n+\tcase TLS_CIPHER_AES_GCM_256: {\n+\t\tstruct tls12_crypto_info_aes_gcm_256 *aes;\n+\n+\t\taes = (void *)crypto_info;\n+\t\tdata |= CE_ADD_CMD_ALGORITHM_AES_GCM_256;\n+\t\tif (crypto_info-\u003eversion == TLS_1_3_VERSION)\n+\t\t\tdata |= CE_ADD_CMD_VERSION_TLS1_3;\n+\t\tmemcpy(\u0026cmd.session_key, aes-\u003ekey, sizeof(aes-\u003ekey));\n+\t\tmemcpy(\u0026cmd.salt, aes-\u003esalt, sizeof(aes-\u003esalt));\n+\t\tmemcpy(\u0026cmd.addl_iv, aes-\u003eiv, sizeof(aes-\u003eiv));\n+\t\tbnxt_copy_tls_mp_data(cmd.record_seq_num, aes-\u003erec_seq,\n+\t\t\t\t sizeof(aes-\u003erec_seq));\n+\t\tbreak;\n+\t}\n+\tdefault:\n+\t\treturn -EOPNOTSUPP;\n+\t}\n+\tcmd.ver_algo_kid_opcode = cpu_to_le32(data);\n+\tcmd.pkt_tcp_seq_num = cpu_to_le32(tcp_seq_no);\n+\tcmd.tls_header_tcp_seq_num = cmd.pkt_tcp_seq_num;\n+\trc = bnxt_xmit_crypto_cmd(bp, txr, \u0026cmd, sizeof(cmd),\n+\t\t\t\t BNXT_MPC_TMO_MSECS);\n+\tmemzero_explicit(\u0026cmd, sizeof(cmd));\n+\treturn rc;\n+}\n+\n+static bool bnxt_ktls_cipher_supported(struct bnxt *bp,\n+\t\t\t\t struct tls_crypto_info *crypto_info)\n+{\n+\tu16 type = crypto_info-\u003ecipher_type;\n+\tu16 version = crypto_info-\u003eversion;\n+\n+\tif ((type == TLS_CIPHER_AES_GCM_128 ||\n+\t type == TLS_CIPHER_AES_GCM_256) \u0026\u0026\n+\t (version == TLS_1_2_VERSION ||\n+\t version == TLS_1_3_VERSION))\n+\t\treturn true;\n+\treturn false;\n+}\n+\n+static void bnxt_set_ktls_ctx_tx(struct tls_context *tls_ctx,\n+\t\t\t\t struct bnxt_ktls_offload_ctx_tx *kctx_tx)\n+{\n+\tstruct bnxt_ktls_tx_driver_state *tx =\n+\t\t__tls_driver_ctx(tls_ctx, TLS_OFFLOAD_CTX_DIR_TX);\n+\n+\ttx-\u003ectx_tx = kctx_tx;\n+}\n+\n+static struct bnxt_ktls_offload_ctx_tx *\n+bnxt_get_ktls_ctx_tx(struct tls_context *tls_ctx)\n+{\n+\tstruct bnxt_ktls_tx_driver_state *tx =\n+\t\t__tls_driver_ctx(tls_ctx, TLS_OFFLOAD_CTX_DIR_TX);\n+\n+\treturn tx-\u003ectx_tx;\n+}\n+\n+static int bnxt_ktls_dev_add(struct net_device *dev, struct sock *sk,\n+\t\t\t enum tls_offload_ctx_dir direction,\n+\t\t\t struct tls_crypto_info *crypto_info,\n+\t\t\t u32 start_offload_tcp_sn)\n+{\n+\tstruct bnxt_ktls_offload_ctx_tx *kctx_tx;\n+\tstruct bnxt *bp = netdev_priv(dev);\n+\tstruct bnxt_crypto_info *crypto;\n+\tstruct tls_context *tls_ctx;\n+\tstruct bnxt_tls_info *ktls;\n+\tstruct bnxt_kctx *kctx;\n+\tu32 kid;\n+\tint rc;\n+\n+\tBUILD_BUG_ON(sizeof(struct bnxt_ktls_tx_driver_state) \u003e\n+\t\t TLS_DRIVER_STATE_SIZE_TX);\n+\n+\tktls = bp-\u003ektls_info;\n+\tif (direction == TLS_OFFLOAD_CTX_DIR_RX)\n+\t\treturn -EOPNOTSUPP;\n+\n+\tif (!BNXT_SUPPORTS_KTLS(bp)) {\n+\t\tatomic64_inc(\u0026ktls-\u003ecounters[BNXT_KTLS_ERR_NO_CAP]);\n+\t\treturn -EOPNOTSUPP;\n+\t}\n+\tatomic_inc(\u0026ktls-\u003epending);\n+\t/* Make sure bnxt_close_nic() sees pending before we check the\n+\t * BNXT_STATE_OPEN flag.\n+\t */\n+\tsmp_mb__after_atomic();\n+\tif (!test_bit(BNXT_STATE_OPEN, \u0026bp-\u003estate)) {\n+\t\tatomic64_inc(\u0026ktls-\u003ecounters[BNXT_KTLS_ERR_STATE_NOT_OPEN]);\n+\t\trc = -ENODEV;\n+\t\tgoto exit;\n+\t}\n+\n+\tif (!bnxt_ktls_cipher_supported(bp, crypto_info)) {\n+\t\tatomic64_inc(\u0026ktls-\u003ecounters[BNXT_KTLS_ERR_INVALID_CIPHER]);\n+\t\trc = -EOPNOTSUPP;\n+\t\tgoto exit;\n+\t}\n+\n+\tkctx_tx = kzalloc_obj(*kctx_tx);\n+\tif (!kctx_tx) {\n+\t\tatomic64_inc(\u0026ktls-\u003ecounters[BNXT_KTLS_ERR_NO_MEM]);\n+\t\trc = -ENOMEM;\n+\t\tgoto exit;\n+\t}\n+\ttls_ctx = tls_get_ctx(sk);\n+\tcrypto = bp-\u003ecrypto_info;\n+\tkctx = \u0026crypto-\u003ekctx[BNXT_TX_CRYPTO_KEY_TYPE];\n+\trc = bnxt_key_ctx_alloc_one(bp, kctx, BNXT_CTX_KIND_CK_TX, \u0026kid);\n+\tif (rc) {\n+\t\tatomic64_inc(\u0026ktls-\u003ecounters[BNXT_KTLS_ERR_KEY_CTX_ALLOC]);\n+\t\tgoto free_ctx;\n+\t}\n+\trc = bnxt_crypto_add(bp, direction, crypto_info, start_offload_tcp_sn,\n+\t\t\t kid);\n+\tif (rc) {\n+\t\tatomic64_inc(\u0026ktls-\u003ecounters[BNXT_KTLS_ERR_CRYPTO_CMD]);\n+\t\tgoto free_kctx;\n+\t}\n+\tkctx_tx-\u003ekid = kid;\n+\tkctx_tx-\u003etcp_seq_no = start_offload_tcp_sn;\n+\tbnxt_set_ktls_ctx_tx(tls_ctx, kctx_tx);\n+\tatomic64_inc(\u0026ktls-\u003ecounters[BNXT_KTLS_TX_ADD]);\n+\tgoto exit;\n+\n+free_kctx:\n+\tbnxt_free_one_kctx(kctx, kid);\n+free_ctx:\n+\tkfree(kctx_tx);\n+exit:\n+\tatomic_dec(\u0026ktls-\u003epending);\n+\treturn rc;\n+}\n+\n+#define KTLS_RETRY_MAX\t\t100\n+#define KTLS_WAIT_TMO_MS\t100\n+\n+static void bnxt_ktls_dev_del(struct net_device *dev,\n+\t\t\t struct tls_context *tls_ctx,\n+\t\t\t enum tls_offload_ctx_dir direction)\n+{\n+\tstruct bnxt_ktls_offload_ctx_tx *kctx_tx;\n+\tstruct bnxt *bp = netdev_priv(dev);\n+\tstruct bnxt_crypto_info *crypto;\n+\tstruct bnxt_tls_info *ktls;\n+\tstruct bnxt_kctx *kctx;\n+\tint retry_cnt = 0;\n+\tu8 kind;\n+\tu32 kid;\n+\n+\tktls = bp-\u003ektls_info;\n+\tkctx_tx = bnxt_get_ktls_ctx_tx(tls_ctx);\n+retry:\n+\tif (!test_bit(BNXT_STATE_OPEN, \u0026bp-\u003estate)) {\n+\t\t/* During ifdown or FW reset, all connections will be torn\n+\t\t * down by bnxt_crypto_del_all() / FUNC_RESET, so nothing to\n+\t\t * do here. Only a reconfiguration is transient and\n+\t\t * __bnxt_open_nic() will set BNXT_STATE_OPEN again and wake us.\n+\t\t */\n+\t\tif (!netif_running(dev) ||\n+\t\t test_bit(BNXT_STATE_IN_FW_RESET, \u0026bp-\u003estate))\n+\t\t\tgoto free;\n+\t\t/* Bound the wait so a wedged reconfig can't block the kTLS\n+\t\t * destruct work indefinitely.\n+\t\t */\n+\t\tif (retry_cnt++ \u003e KTLS_RETRY_MAX) {\n+\t\t\tatomic64_inc(\u0026ktls-\u003ecounters[BNXT_KTLS_ERR_RETRY_EXCEEDED]);\n+\t\t\tnetdev_warn(dev, \"%s timed out waiting for device, state %lx\\n\",\n+\t\t\t\t __func__, bp-\u003estate);\n+\t\t\tgoto free;\n+\t\t}\n+\t\twait_event_timeout(ktls-\u003eopen_wq,\n+\t\t\t\t test_bit(BNXT_STATE_OPEN, \u0026bp-\u003estate) ||\n+\t\t\t\t !netif_running(dev) ||\n+\t\t\t\t test_bit(BNXT_STATE_IN_FW_RESET, \u0026bp-\u003estate),\n+\t\t\t\t msecs_to_jiffies(KTLS_WAIT_TMO_MS));\n+\t\tgoto retry;\n+\t}\n+\tatomic_inc(\u0026ktls-\u003epending);\n+\t/* Make sure bnxt_close_nic() sees pending before we check the\n+\t * BNXT_STATE_OPEN flag.\n+\t */\n+\tsmp_mb__after_atomic();\n+\tif (!test_bit(BNXT_STATE_OPEN, \u0026bp-\u003estate)) {\n+\t\tatomic_dec(\u0026ktls-\u003epending);\n+\t\tgoto retry;\n+\t}\n+\n+\tcrypto = bp-\u003ecrypto_info;\n+\tkid = kctx_tx-\u003ekid;\n+\tkctx = \u0026crypto-\u003ekctx[BNXT_TX_CRYPTO_KEY_TYPE];\n+\tkind = BNXT_CTX_KIND_CK_TX;\n+\tatomic64_inc(\u0026ktls-\u003ecounters[BNXT_KTLS_TX_DEL]);\n+\tif (bnxt_kid_valid(kctx, kid) \u0026\u0026\n+\t !bnxt_crypto_del(bp, kctx-\u003etype, kind, kid))\n+\t\tbnxt_free_one_kctx(kctx, kid);\n+\n+\tatomic_dec(\u0026ktls-\u003epending);\n+free:\n+\tbnxt_set_ktls_ctx_tx(tls_ctx, NULL);\n+\tkfree(kctx_tx);\n+}\n+\n+static const struct tlsdev_ops bnxt_ktls_ops = {\n+\t.tls_dev_add = bnxt_ktls_dev_add,\n+\t.tls_dev_del = bnxt_ktls_dev_del,\n+};\n+\n+int bnxt_ktls_init(struct bnxt *bp)\n+{\n+\tstruct bnxt_tls_info *ktls = bp-\u003ektls_info;\n+\tstruct net_device *dev = bp-\u003edev;\n+\n+\tif (!ktls)\n+\t\treturn 0;\n+\n+\tdev-\u003etlsdev_ops = \u0026bnxt_ktls_ops;\n+\tdev-\u003ehw_features |= NETIF_F_HW_TLS_TX;\n+\tdev-\u003efeatures |= NETIF_F_HW_TLS_TX;\n+\treturn 0;\n+}\n+\n+static void bnxt_ktls_inc_tx_stats(struct bnxt_tx_ring_info *txr, u32 bytes,\n+\t\t\t\t bool ooo)\n+{\n+\tstruct bnxt_tls_sw_stats *ring_stats = txr-\u003etls_stats;\n+\n+\tif (!ring_stats)\n+\t\treturn;\n+\tring_stats-\u003ecounters[BNXT_KTLS_TX_PKTS]++;\n+\tring_stats-\u003ecounters[BNXT_KTLS_TX_BYTES] += bytes;\n+\tif (ooo)\n+\t\tring_stats-\u003ecounters[BNXT_KTLS_TX_OOO_PKTS]++;\n+}\n+\n+static void bnxt_ktls_pre_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n+\t\t\t u32 kid, struct crypto_prefix_cmd *pre_cmd)\n+{\n+\tstruct bnxt_sw_tx_bd *tx_buf;\n+\tstruct tx_bd_presync *psbd;\n+\tu32 bd_space, space;\n+\tu8 *pcmd;\n+\tu16 prod;\n+\n+\tprod = txr-\u003etx_prod;\n+\ttx_buf = \u0026txr-\u003etx_buf_ring[RING_TX(bp, prod)];\n+\n+\tpsbd = (void *)\u0026txr-\u003etx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)];\n+\tpsbd-\u003etx_bd_len_flags_type = CRYPTO_PRESYNC_BD_CMD;\n+\tpsbd-\u003etx_bd_kid = cpu_to_le32(BNXT_KID_HW(kid));\n+\tpsbd-\u003etx_bd_opaque =\n+\t\tSET_TX_OPAQUE(bp, txr, prod, CRYPTO_PREFIX_CMD_BDS + 1);\n+\n+\tprod = NEXT_TX(prod);\n+\tpcmd = (void *)\u0026txr-\u003etx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)];\n+\tbd_space = TX_DESC_CNT - TX_IDX(prod);\n+\tspace = bd_space * sizeof(struct tx_bd);\n+\tif (space \u003e= CRYPTO_PREFIX_CMD_SIZE) {\n+\t\tmemcpy(pcmd, pre_cmd, CRYPTO_PREFIX_CMD_SIZE);\n+\t\tprod += CRYPTO_PREFIX_CMD_BDS;\n+\t} else {\n+\t\tmemcpy(pcmd, pre_cmd, space);\n+\t\tprod += bd_space;\n+\t\tpcmd = (void *)\u0026txr-\u003etx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)];\n+\t\tmemcpy(pcmd, (u8 *)pre_cmd + space,\n+\t\t CRYPTO_PREFIX_CMD_SIZE - space);\n+\t\tprod += CRYPTO_PREFIX_CMD_BDS - bd_space;\n+\t}\n+\ttxr-\u003etx_prod = prod;\n+\ttx_buf-\u003eis_push = 1;\n+\t/* Minus 1 since the header psbd is a single entry short BD */\n+\ttx_buf-\u003einline_data_bds = CRYPTO_PREFIX_CMD_BDS - 1;\n+}\n+\n+static int bnxt_ktls_tx_ooo(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n+\t\t\t struct sk_buff *skb, u32 payload_len, u32 seq,\n+\t\t\t struct tls_context *tls_ctx)\n+{\n+\tstruct bnxt_tls_sw_stats *ring_stats = txr-\u003etls_stats;\n+\tstruct tls_offload_context_tx *tx_tls_ctx;\n+\tstruct bnxt_ktls_offload_ctx_tx *kctx_tx;\n+\tu32 hdr_tcp_seq, end_seq, total_bds;\n+\tstruct crypto_prefix_cmd pcmd = {};\n+\tstruct tls_record_info *record;\n+\tunsigned long flags;\n+\tbool fwd = false;\n+\t__le64 le_rec_sn;\n+\tu64 rec_sn;\n+\tu8 *hdr;\n+\tint rc;\n+\n+\ttx_tls_ctx = tls_offload_ctx_tx(tls_ctx);\n+\tkctx_tx = bnxt_get_ktls_ctx_tx(tls_ctx);\n+\tend_seq = seq + skb-\u003elen - skb_tcp_all_headers(skb);\n+\tif (unlikely(after(seq, kctx_tx-\u003etcp_seq_no) ||\n+\t\t after(end_seq, kctx_tx-\u003etcp_seq_no))) {\n+\t\tfwd = true;\n+\t\tpcmd.flags = CRYPTO_PREFIX_CMD_FLAGS_UPDATE_IN_ORDER_VAR_LE;\n+\t}\n+\n+\tspin_lock_irqsave(\u0026tx_tls_ctx-\u003elock, flags);\n+\trecord = tls_get_record(tx_tls_ctx, seq, \u0026rec_sn);\n+\tif (!record || !record-\u003enum_frags) {\n+\t\trc = -EPROTO;\n+\t\tring_stats-\u003ecounters[BNXT_KTLS_TX_OOO_FALLBACK_NO_SYNC]++;\n+\t\tgoto unlock_exit;\n+\t}\n+\thdr_tcp_seq = tls_record_start_seq(record);\n+\thdr = skb_frag_address_safe(\u0026record-\u003efrags[0]);\n+\n+\ttotal_bds = CRYPTO_PRESYNC_BDS + skb_shinfo(skb)-\u003enr_frags + 2;\n+\tif (bnxt_tx_avail(bp, txr) \u003c total_bds) {\n+\t\trc = -ENOSPC;\n+\t\tring_stats-\u003ecounters[BNXT_KTLS_TX_OOO_FALLBACK_NO_SPACE]++;\n+\t\tgoto unlock_exit;\n+\t}\n+\n+\tif (before(record-\u003eend_seq - tls_ctx-\u003eprot_info.tag_size,\n+\t\t seq + payload_len)) {\n+\t\t/* retransmission includes tag bytes */\n+\t\trc = -EOPNOTSUPP;\n+\t\tgoto unlock_exit;\n+\t}\n+\tpcmd.header_tcp_seq_num = cpu_to_le32(hdr_tcp_seq);\n+\tpcmd.start_tcp_seq_num = cpu_to_le32(seq);\n+\tpcmd.end_tcp_seq_num = cpu_to_le32(seq + payload_len - 1);\n+\tif (tls_ctx-\u003eprot_info.version == TLS_1_2_VERSION) {\n+\t\tu32 nonce_bytes = tls_ctx-\u003eprot_info.iv_size;\n+\t\tu32 retrans_off = seq - hdr_tcp_seq;\n+\n+\t\tif (!hdr) {\n+\t\t\trc = -ENOBUFS;\n+\t\t\tring_stats-\u003ecounters[BNXT_KTLS_TX_OOO_FALLBACK_NO_HDR]++;\n+\t\t\tgoto unlock_exit;\n+\t\t}\n+\t\tif (retrans_off \u003e 5 \u0026\u0026 retrans_off \u003c 5 + nonce_bytes)\n+\t\t\tnonce_bytes = retrans_off - 5;\n+\t\tmemcpy(pcmd.explicit_nonce, hdr + 5, nonce_bytes);\n+\t}\n+\tle_rec_sn = cpu_to_le64(rec_sn);\n+\tmemcpy(\u0026pcmd.record_seq_num[0], \u0026le_rec_sn, sizeof(le_rec_sn));\n+\n+\trc = 0;\n+\tbnxt_ktls_pre_xmit(bp, txr, kctx_tx-\u003ekid, \u0026pcmd);\n+\n+\tif (fwd) {\n+\t\tkctx_tx-\u003enext_tcp_seq_no = end_seq;\n+\t\tkctx_tx-\u003epending_fwd = 1;\n+\t}\n+\n+unlock_exit:\n+\tspin_unlock_irqrestore(\u0026tx_tls_ctx-\u003elock, flags);\n+\treturn rc;\n+}\n+\n+struct sk_buff *bnxt_ktls_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n+\t\t\t struct sk_buff *skb, __le32 *lflags, u32 *kid,\n+\t\t\t struct bnxt_ktls_offload_ctx_tx **kctx_tx_p)\n+{\n+\tstruct bnxt_tls_info *ktls = bp-\u003ektls_info;\n+\tstruct bnxt_ktls_offload_ctx_tx *kctx_tx;\n+\tstruct tls_context *tls_ctx;\n+\tu32 seq, payload_len;\n+\tint rc;\n+\n+\tif (!IS_ENABLED(CONFIG_TLS_DEVICE) || !ktls ||\n+\t !tls_is_skb_tx_device_offloaded(skb))\n+\t\treturn skb;\n+\n+\tseq = ntohl(tcp_hdr(skb)-\u003eseq);\n+\ttls_ctx = tls_get_ctx(skb-\u003esk);\n+\tkctx_tx = bnxt_get_ktls_ctx_tx(tls_ctx);\n+\tpayload_len = skb-\u003elen - skb_tcp_all_headers(skb);\n+\tif (!payload_len)\n+\t\treturn skb;\n+\tif (kctx_tx-\u003etcp_seq_no == seq) {\n+\t\t/* Stage the advance only. tcp_seq_no and the counters are\n+\t\t * committed by bnxt_ktls_xmit_commit() once the BD reaches the\n+\t\t * ring.\n+\t\t */\n+\t\tkctx_tx-\u003enext_tcp_seq_no = seq + payload_len;\n+\t\tkctx_tx-\u003epending_bytes = payload_len;\n+\t\tkctx_tx-\u003epending_ooo = 0;\n+\t\tkctx_tx-\u003epending_fwd = 1;\n+\t\t*kid = BNXT_KID_HW(kctx_tx-\u003ekid);\n+\t\t*kctx_tx_p = kctx_tx;\n+\t\t*lflags |= cpu_to_le32(TX_BD_FLAGS_CRYPTO_EN |\n+\t\t\t\t BNXT_TX_KID_LO(*kid));\n+\t} else {\n+\t\tkctx_tx-\u003epending_fwd = 0;\n+\t\trc = bnxt_ktls_tx_ooo(bp, txr, skb, payload_len, seq, tls_ctx);\n+\t\tif (rc)\n+\t\t\treturn tls_encrypt_skb(skb);\n+\n+\t\tkctx_tx-\u003epending_bytes = payload_len;\n+\t\tkctx_tx-\u003epending_ooo = 1;\n+\t\t*kid = BNXT_KID_HW(kctx_tx-\u003ekid);\n+\t\t*kctx_tx_p = kctx_tx;\n+\t\t*lflags |= cpu_to_le32(TX_BD_FLAGS_CRYPTO_EN |\n+\t\t\t\t BNXT_TX_KID_LO(*kid));\n+\t\treturn skb;\n+\t}\n+\treturn skb;\n+}\n+\n+void bnxt_ktls_xmit_commit(struct bnxt_tx_ring_info *txr,\n+\t\t\t struct bnxt_ktls_offload_ctx_tx *kctx_tx)\n+{\n+\tif (!kctx_tx)\n+\t\treturn;\n+\tif (kctx_tx-\u003epending_fwd)\n+\t\tkctx_tx-\u003etcp_seq_no = kctx_tx-\u003enext_tcp_seq_no;\n+\tbnxt_ktls_inc_tx_stats(txr, kctx_tx-\u003epending_bytes,\n+\t\t\t kctx_tx-\u003epending_ooo);\n+\tkctx_tx-\u003epending_bytes = 0;\n+\tkctx_tx-\u003epending_fwd = 0;\n+\tkctx_tx-\u003epending_ooo = 0;\n+}\n+\n+int bnxt_ktls_alloc_tx_ring_stats(struct bnxt *bp, struct bnxt_tx_ring_info *txr)\n+{\n+\tstruct bnxt_tls_sw_stats *ring_stats;\n+\n+\tif (!bp-\u003ektls_info)\n+\t\treturn 0;\n+\tring_stats = kzalloc_obj(*ring_stats);\n+\tif (!ring_stats)\n+\t\treturn -ENOMEM;\n+\ttxr-\u003etls_stats = ring_stats;\n+\treturn 0;\n+}\n+\n+void bnxt_ktls_free_tx_ring_stats(struct bnxt_tx_ring_info *txr)\n+{\n+\tkfree(txr-\u003etls_stats);\n+\ttxr-\u003etls_stats = NULL;\n+}\n+\n+void bnxt_get_ring_tls_stats(struct bnxt *bp, struct bnxt_tls_sw_stats *stats)\n+{\n+\tstruct bnxt_tls_sw_stats *ring_stats;\n+\tint i, j;\n+\n+\tif (!bp-\u003ektls_info || !bp-\u003etx_ring)\n+\t\treturn;\n+\tfor (i = 0; i \u003c bp-\u003etx_nr_rings; i++) {\n+\t\tring_stats = bp-\u003etx_ring[i].tls_stats;\n+\t\tif (!ring_stats)\n+\t\t\tcontinue;\n+\t\tfor (j = 0; j \u003c BNXT_KTLS_MAX_DATA_COUNTERS; j++)\n+\t\t\tstats-\u003ecounters[j] += ring_stats-\u003ecounters[j];\n+\t}\n+}\ndiff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h\nnew file mode 100644\nindex 00000000000000..40b94bbf5a3836\n--- /dev/null\n+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h\n@@ -0,0 +1,175 @@\n+/* SPDX-License-Identifier: GPL-2.0-only */\n+/* Copyright (c) 2026 Broadcom Inc. */\n+\n+#ifndef BNXT_KTLS_H\n+#define BNXT_KTLS_H\n+\n+#include \u003clinux/u64_stats_sync.h\u003e\n+#include \u003clinux/wait.h\u003e\n+\n+/* Control plane counters */\n+enum bnxt_ktls_ctrl_counters {\n+\tBNXT_KTLS_TX_ADD = 0,\n+\tBNXT_KTLS_TX_DEL,\n+\n+\t/* Error counters for debugging */\n+\tBNXT_KTLS_ERR_NO_MEM,\t\t\t/* Memory allocation failure */\n+\tBNXT_KTLS_ERR_NO_CAP,\t\t\t/* Capability lost after FW reset */\n+\tBNXT_KTLS_ERR_KEY_CTX_ALLOC,\t\t/* Key context alloc failure */\n+\tBNXT_KTLS_ERR_CRYPTO_CMD,\t\t/* Crypto command failure */\n+\tBNXT_KTLS_ERR_DEVICE_BUSY,\t\t/* Device not ready */\n+\tBNXT_KTLS_ERR_INVALID_CIPHER,\t\t/* Unsupported cipher */\n+\tBNXT_KTLS_ERR_STATE_NOT_OPEN,\t\t/* Device not open */\n+\tBNXT_KTLS_ERR_RETRY_EXCEEDED,\t\t/* Retry limit exceeded */\n+\n+\tBNXT_KTLS_MAX_CTRL_COUNTERS,\n+};\n+\n+struct bnxt_tls_info {\n+\tatomic_t\t\tpending;\n+\n+\t/* Woken from __bnxt_open_nic()/__bnxt_close_nic() when\n+\t * BNXT_STATE_OPEN changes, so a kTLS delete can wait out a ring\n+\t * reconfiguration instead of polling the state bit.\n+\t */\n+\twait_queue_head_t\topen_wq;\n+\n+\t/* Atomic counters for control path */\n+\tatomic64_t\t\t*counters;\n+};\n+\n+struct bnxt_ktls_offload_ctx_tx {\n+\tu32\t\ttcp_seq_no;\t/* tcp seq no in sync with HW */\n+\tu32\t\tnext_tcp_seq_no;/* staged tcp seq no */\n+\tu32\t\tkid;\n+\tu32\t\tpending_bytes;\t/* staged payload bytes */\n+\tu32\t\tpending_fwd:1;\n+\tu32\t\tpending_ooo:1;\n+};\n+\n+struct bnxt_ktls_tx_driver_state {\n+\tstruct bnxt_ktls_offload_ctx_tx *ctx_tx;\n+};\n+\n+struct ce_add_cmd {\n+\t__le32\tver_algo_kid_opcode;\n+\t#define CE_ADD_CMD_OPCODE_MASK\t\t\t0xfUL\n+\t#define CE_ADD_CMD_OPCODE_SFT\t\t\t0\n+\t#define CE_ADD_CMD_OPCODE_ADD\t\t\t 0x1UL\n+\t#define CE_ADD_CMD_KID_MASK\t\t\t0xfffff0UL\n+\t#define CE_ADD_CMD_KID_SFT\t\t\t4\n+\t#define CE_ADD_CMD_ALGORITHM_MASK\t\t0xf000000UL\n+\t#define CE_ADD_CMD_ALGORITHM_SFT\t\t24\n+\t#define CE_ADD_CMD_ALGORITHM_AES_GCM_128\t 0x1000000UL\n+\t#define CE_ADD_CMD_ALGORITHM_AES_GCM_256\t 0x2000000UL\n+\t#define CE_ADD_CMD_VERSION_MASK\t\t\t0xf0000000UL\n+\t#define CE_ADD_CMD_VERSION_SFT\t\t\t28\n+\t#define CE_ADD_CMD_VERSION_TLS1_2\t\t (0x0UL \u003c\u003c 28)\n+\t#define CE_ADD_CMD_VERSION_TLS1_3\t\t (0x1UL \u003c\u003c 28)\n+\tu8\tctx_kind;\n+\t#define CE_ADD_CMD_CTX_KIND_MASK\t\t0x1fUL\n+\t#define CE_ADD_CMD_CTX_KIND_SFT\t\t\t0\n+\t#define CE_ADD_CMD_CTX_KIND_CK_TX\t\t 0x11UL\n+\t#define CE_ADD_CMD_CTX_KIND_CK_RX\t\t 0x12UL\n+\tu8\tunused0[3];\n+\tu8\tsalt[4];\n+\tu8\tunused1[4];\n+\t__le32\tpkt_tcp_seq_num;\n+\t__le32\ttls_header_tcp_seq_num;\n+\tu8\trecord_seq_num[8];\n+\tu8\tsession_key[32];\n+\tu8\taddl_iv[8];\n+};\n+\n+struct crypto_prefix_cmd {\n+\t__le32\tflags;\n+\t#define CRYPTO_PREFIX_CMD_FLAGS_UPDATE_IN_ORDER_VAR\t0x1UL\n+\t#define CRYPTO_PREFIX_CMD_FLAGS_FULL_REPLAY_RETRAN\t0x2UL\n+\t__le32\theader_tcp_seq_num;\n+\t__le32\tstart_tcp_seq_num;\n+\t__le32\tend_tcp_seq_num;\n+\tu8\texplicit_nonce[8];\n+\tu8\trecord_seq_num[8];\n+};\n+\n+#define CRYPTO_PREFIX_CMD_FLAGS_UPDATE_IN_ORDER_VAR_LE\t\\\n+\tcpu_to_le32(CRYPTO_PREFIX_CMD_FLAGS_UPDATE_IN_ORDER_VAR)\n+\n+#define CRYPTO_PREFIX_CMD_SIZE\t((u32)sizeof(struct crypto_prefix_cmd))\n+#define CRYPTO_PREFIX_CMD_BDS\t(CRYPTO_PREFIX_CMD_SIZE / sizeof(struct tx_bd))\n+#define CRYPTO_PRESYNC_BDS\t(CRYPTO_PREFIX_CMD_BDS + 1)\n+\n+#define CRYPTO_PRESYNC_BD_CMD\t\t\t\t\t\t\\\n+\t(cpu_to_le32((CRYPTO_PREFIX_CMD_SIZE \u003c\u003c TX_BD_LEN_SHIFT) |\t\\\n+\t\t TX_BD_CNT(CRYPTO_PRESYNC_BDS) | TX_BD_TYPE_PRESYNC_TX_BD))\n+\n+static inline bool bnxt_ktls_busy(struct bnxt *bp)\n+{\n+\treturn bp-\u003ektls_info \u0026\u0026 atomic_read(\u0026bp-\u003ektls_info-\u003epending) \u003e 0;\n+}\n+\n+/* Wake any kTLS control op waiting for a BNXT_STATE_OPEN transition. */\n+static inline void bnxt_ktls_wake(struct bnxt *bp)\n+{\n+\tif (bp-\u003ektls_info)\n+\t\twake_up_all(\u0026bp-\u003ektls_info-\u003eopen_wq);\n+}\n+\n+#ifdef CONFIG_BNXT_TLS\n+int bnxt_alloc_ktls_info(struct bnxt *bp);\n+void bnxt_free_ktls_info(struct bnxt *bp);\n+int bnxt_ktls_init(struct bnxt *bp);\n+struct sk_buff *bnxt_ktls_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n+\t\t\t struct sk_buff *skb, __le32 *lflags, u32 *kid,\n+\t\t\t struct bnxt_ktls_offload_ctx_tx **kctx_tx_p);\n+void bnxt_ktls_xmit_commit(struct bnxt_tx_ring_info *txr,\n+\t\t\t struct bnxt_ktls_offload_ctx_tx *kctx_tx);\n+int bnxt_ktls_alloc_tx_ring_stats(struct bnxt *bp,\n+\t\t\t\t struct bnxt_tx_ring_info *txr);\n+void bnxt_ktls_free_tx_ring_stats(struct bnxt_tx_ring_info *txr);\n+void bnxt_get_ring_tls_stats(struct bnxt *bp, struct bnxt_tls_sw_stats *stats);\n+#else\n+static inline int bnxt_alloc_ktls_info(struct bnxt *bp)\n+{\n+\treturn -EOPNOTSUPP;\n+}\n+\n+static inline void bnxt_free_ktls_info(struct bnxt *bp)\n+{\n+}\n+\n+static inline int bnxt_ktls_init(struct bnxt *bp)\n+{\n+\treturn -EOPNOTSUPP;\n+}\n+\n+static inline struct sk_buff *\n+bnxt_ktls_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n+\t struct sk_buff *skb, __le32 *lflags, u32 *kid,\n+\t struct bnxt_ktls_offload_ctx_tx **kctx_tx_p)\n+{\n+\treturn skb;\n+}\n+\n+static inline void\n+bnxt_ktls_xmit_commit(struct bnxt_tx_ring_info *txr,\n+\t\t struct bnxt_ktls_offload_ctx_tx *kctx_tx)\n+{\n+}\n+\n+static inline int bnxt_ktls_alloc_tx_ring_stats(struct bnxt *bp,\n+\t\t\t\t\t\tstruct bnxt_tx_ring_info *txr)\n+{\n+\treturn 0;\n+}\n+\n+static inline void bnxt_ktls_free_tx_ring_stats(struct bnxt_tx_ring_info *txr)\n+{\n+}\n+\n+static inline void bnxt_get_ring_tls_stats(struct bnxt *bp,\n+\t\t\t\t\t struct bnxt_tls_sw_stats *stats)\n+{\n+}\n+#endif\t/* CONFIG_BNXT_TLS */\n+#endif\t/* BNXT_KTLS_H */\ndiff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c\nnew file mode 100644\nindex 00000000000000..d66265b43bed4f\n--- /dev/null\n+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c\n@@ -0,0 +1,732 @@\n+// SPDX-License-Identifier: GPL-2.0-only\n+/* Copyright (c) 2026 Broadcom Inc. */\n+\n+#include \u003clinux/stddef.h\u003e\n+#include \u003clinux/types.h\u003e\n+#include \u003clinux/kernel.h\u003e\n+#include \u003clinux/slab.h\u003e\n+#include \u003clinux/bnxt/hsi.h\u003e\n+\n+#include \"bnxt.h\"\n+#include \"bnxt_mpc.h\"\n+#include \"bnxt_crypto.h\"\n+\n+void bnxt_alloc_mpc_info(struct bnxt *bp, u8 mpc_chnls_cap)\n+{\n+\tif (!bp-\u003empc_info)\n+\t\tbp-\u003empc_info = kzalloc_obj(*bp-\u003empc_info);\n+\tif (bp-\u003empc_info)\n+\t\tbp-\u003empc_info-\u003empc_chnls_cap = mpc_chnls_cap;\n+\telse\n+\t\tnetdev_warn(bp-\u003edev, \"Unable to allocate MPC info\\n\");\n+}\n+\n+void bnxt_free_mpc_info(struct bnxt *bp)\n+{\n+\tbnxt_free_mpc_rings(bp);\n+\tbnxt_free_mpcs(bp);\n+\tkfree(bp-\u003empc_info);\n+\tbp-\u003empc_info = NULL;\n+}\n+\n+int bnxt_mpc_tx_rings_in_use(struct bnxt *bp)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tint i, mpc_tx = 0;\n+\n+\tif (!mpc)\n+\t\treturn 0;\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++)\n+\t\tmpc_tx += mpc-\u003empc_ring_count[i];\n+\treturn mpc_tx;\n+}\n+\n+int bnxt_mpc_cp_rings_in_use(struct bnxt *bp)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\n+\tif (!mpc)\n+\t\treturn 0;\n+\treturn mpc-\u003empc_cp_rings;\n+}\n+\n+bool bnxt_napi_has_mpc(struct bnxt *bp, int i)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tstruct bnxt_napi *bnapi = bp-\u003ebnapi[i];\n+\tstruct bnxt_tx_ring_info *txr;\n+\n+\tif (!mpc)\n+\t\treturn false;\n+\n+\ttxr = bnapi-\u003etx_ring[0];\n+\tif (txr \u0026\u0026 !(bnapi-\u003eflags \u0026 BNXT_NAPI_FLAG_XDP))\n+\t\treturn txr-\u003etxq_index \u003c mpc-\u003empc_cp_rings;\n+\treturn false;\n+}\n+\n+void bnxt_set_mpc_cp_ring(struct bnxt *bp, int bnapi_idx,\n+\t\t\t struct bnxt_cp_ring_info *cpr)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tstruct bnxt_napi *bnapi;\n+\tbool found = false;\n+\tint i, j;\n+\n+\tif (!mpc)\n+\t\treturn;\n+\tbnapi = bp-\u003ebnapi[bnapi_idx];\n+\t/* Check both TCE and RCE MPCs for the matching NAPI */\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tint num = mpc-\u003empc_ring_count[i];\n+\n+\t\tfor (j = 0; j \u003c num; j++) {\n+\t\t\tstruct bnxt_tx_ring_info *txr = \u0026mpc-\u003empc_rings[i][j];\n+\n+\t\t\t/* Only 1 ring with index j will use this NAPI */\n+\t\t\tif (txr-\u003ebnapi == bnapi) {\n+\t\t\t\ttxr-\u003etx_cpr = cpr;\n+\t\t\t\ttxr-\u003etx_napi_idx = i;\n+\t\t\t\tbnapi-\u003etx_mpc_ring[i] = txr;\n+\t\t\t\tfound = true;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\t}\n+\tif (!found)\n+\t\tnetdev_warn_once(bp-\u003edev, \"No MPC match for napi index %d\\n\",\n+\t\t\t\t bnapi_idx);\n+\tcpr-\u003ecp_ring_type = BNXT_NQ_HDL_TYPE_MP;\n+}\n+\n+void bnxt_trim_mpc_rings(struct bnxt *bp)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tint max = bp-\u003etx_nr_rings_per_tc;\n+\tu8 max_cp = 0;\n+\tint i;\n+\n+\tif (!mpc)\n+\t\treturn;\n+\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tmpc-\u003empc_ring_count[i] = min_t(int, mpc-\u003empc_ring_count[i],\n+\t\t\t\t\t max);\n+\t\tmax_cp = max(max_cp, mpc-\u003empc_ring_count[i]);\n+\t}\n+\tmpc-\u003empc_cp_rings = max_cp;\n+}\n+\n+void bnxt_set_dflt_mpc_rings(struct bnxt *bp)\n+{\n+\tstruct bnxt_hw_resc *hw_resc = \u0026bp-\u003ehw_resc;\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tint mpc_tce, mpc_rce, avail, mpc_cp, i;\n+\n+\tif (!BNXT_MPC_CRYPTO_CAPABLE(bp))\n+\t\treturn;\n+\n+\tavail = hw_resc-\u003emax_tx_rings - bp-\u003etx_nr_rings;\n+\t/* don't use more than 80% */\n+\tavail = avail * 4 / 5;\n+\n+\tif (avail \u003c (BNXT_MIN_MPC_TCE + BNXT_MIN_MPC_RCE))\n+\t\tgoto disable_mpc;\n+\n+\tmpc_tce = min_t(int, avail / 2, bp-\u003etx_nr_rings_per_tc);\n+\tmpc_rce = mpc_tce;\n+\n+\tmpc_tce = min_t(int, mpc_tce, BNXT_DFLT_MPC_TCE);\n+\tmpc_rce = min_t(int, mpc_rce, BNXT_DFLT_MPC_RCE);\n+\n+\tavail = hw_resc-\u003emax_cp_rings - bp-\u003etx_nr_rings -\n+\t\tbp-\u003erx_nr_rings;\n+\n+\tif (avail \u003c BNXT_MIN_MPC_TCE || avail \u003c BNXT_MIN_MPC_RCE)\n+\t\tgoto disable_mpc;\n+\n+\tmpc_tce = min(mpc_tce, avail);\n+\tmpc_rce = min(mpc_rce, avail);\n+\n+\tmpc_cp = max(mpc_tce, mpc_rce);\n+\n+\tmpc-\u003empc_ring_count[BNXT_MPC_TCE_TYPE] = mpc_tce;\n+\tmpc-\u003empc_ring_count[BNXT_MPC_RCE_TYPE] = mpc_rce;\n+\tmpc-\u003empc_cp_rings = mpc_cp;\n+\treturn;\n+\n+disable_mpc:\n+\tmpc-\u003empc_cp_rings = 0;\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++)\n+\t\tmpc-\u003empc_ring_count[i] = 0;\n+}\n+\n+void bnxt_init_mpc_ring_struct(struct bnxt *bp)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tint i, j;\n+\n+\tif (!BNXT_MPC_CRYPTO_CAPABLE(bp))\n+\t\treturn;\n+\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tint num = mpc-\u003empc_ring_count[i];\n+\n+\t\tif (!mpc-\u003empc_rings[i])\n+\t\t\tcontinue;\n+\t\tfor (j = 0; j \u003c num; j++) {\n+\t\t\tstruct bnxt_ring_mem_info *rmem;\n+\t\t\tstruct bnxt_ring_struct *ring;\n+\t\t\tstruct bnxt_tx_ring_info *txr;\n+\n+\t\t\ttxr = \u0026mpc-\u003empc_rings[i][j];\n+\n+\t\t\ttxr-\u003etx_ring_struct.ring_mem.flags =\n+\t\t\t\tBNXT_RMEM_RING_PTE_FLAG;\n+\t\t\ttxr-\u003ebnapi = bp-\u003etx_ring[bp-\u003etx_ring_map[j]].bnapi;\n+\t\t\ttxr-\u003etxq_index = j;\n+\n+\t\t\tring = \u0026txr-\u003etx_ring_struct;\n+\t\t\trmem = \u0026ring-\u003ering_mem;\n+\t\t\trmem-\u003enr_pages = bp-\u003etx_nr_pages;\n+\t\t\trmem-\u003epage_size = HW_TXBD_RING_SIZE;\n+\t\t\trmem-\u003epg_arr = (void **)txr-\u003etx_desc_ring;\n+\t\t\trmem-\u003edma_arr = txr-\u003etx_desc_mapping;\n+\t\t\trmem-\u003evmem_size = SW_MPC_TXBD_RING_SIZE *\n+\t\t\t\t\t bp-\u003etx_nr_pages;\n+\t\t\trmem-\u003evmem = (void **)\u0026txr-\u003etx_mpc_buf_ring;\n+\t\t}\n+\t}\n+}\n+\n+int bnxt_alloc_mpcs(struct bnxt *bp)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tint i, rc = 0;\n+\n+\tif (!BNXT_MPC_CRYPTO_CAPABLE(bp))\n+\t\treturn 0;\n+\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tint num = mpc-\u003empc_ring_count[i];\n+\t\tstruct bnxt_tx_ring_info *txr;\n+\n+\t\tif (!num)\n+\t\t\tcontinue;\n+\t\ttxr = kzalloc_objs(*txr, num);\n+\t\tif (!txr) {\n+\t\t\trc = -ENOMEM;\n+\t\t\tgoto alloc_mpcs_exit;\n+\t\t}\n+\t\tmpc-\u003empc_rings[i] = txr;\n+\t}\n+\n+\tfor (i = 0; i \u003c bp-\u003ecp_nr_rings; i++) {\n+\t\tstruct bnxt_napi *bnapi = bp-\u003ebnapi[i];\n+\n+\t\tif (!bnxt_napi_has_mpc(bp, i))\n+\t\t\tcontinue;\n+\t\tbnapi-\u003etx_mpc_ring = kzalloc_objs(*bnapi-\u003etx_mpc_ring,\n+\t\t\t\t\t\t BNXT_MPC_TYPE_MAX);\n+\t\tif (!bnapi-\u003etx_mpc_ring) {\n+\t\t\trc = -ENOMEM;\n+\t\t\tgoto alloc_mpcs_exit;\n+\t\t}\n+\t}\n+alloc_mpcs_exit:\n+\tif (rc)\n+\t\tbnxt_free_mpcs(bp);\n+\treturn rc;\n+}\n+\n+void bnxt_free_mpcs(struct bnxt *bp)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tint i;\n+\n+\tif (!mpc)\n+\t\treturn;\n+\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tkfree(mpc-\u003empc_rings[i]);\n+\t\tmpc-\u003empc_rings[i] = NULL;\n+\t}\n+\tif (!bp-\u003ebnapi)\n+\t\treturn;\n+\tfor (i = 0; i \u003c bp-\u003ecp_nr_rings; i++) {\n+\t\tstruct bnxt_napi *bnapi = bp-\u003ebnapi[i];\n+\n+\t\tkfree(bnapi-\u003etx_mpc_ring);\n+\t\tbnapi-\u003etx_mpc_ring = NULL;\n+\t}\n+}\n+\n+int bnxt_alloc_mpc_rings(struct bnxt *bp)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tint i, j, rc = 0;\n+\n+\tif (!mpc)\n+\t\treturn 0;\n+\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tint num = mpc-\u003empc_ring_count[i];\n+\n+\t\tfor (j = 0; j \u003c num; j++) {\n+\t\t\tstruct bnxt_tx_ring_info *txr = \u0026mpc-\u003empc_rings[i][j];\n+\t\t\tstruct bnxt_ring_struct *ring;\n+\n+\t\t\tring = \u0026txr-\u003etx_ring_struct;\n+\t\t\trc = bnxt_alloc_ring(bp, \u0026ring-\u003ering_mem);\n+\t\t\tif (rc)\n+\t\t\t\tgoto alloc_mpc_rings_exit;\n+\t\t\tring-\u003equeue_id = BNXT_MPC_QUEUE_ID;\n+\t\t\tring-\u003empc_chnl_type = i;\n+\t\t\t/* for stats context */\n+\t\t\tring-\u003egrp_idx = txr-\u003ebnapi-\u003eindex;\n+\t\t\tspin_lock_init(\u0026txr-\u003etx_lock);\n+\t\t}\n+\t}\n+alloc_mpc_rings_exit:\n+\tif (rc)\n+\t\tbnxt_free_mpc_rings(bp);\n+\treturn rc;\n+}\n+\n+void bnxt_free_mpc_rings(struct bnxt *bp)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tint i, j;\n+\n+\tif (!mpc)\n+\t\treturn;\n+\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tint num = mpc-\u003empc_ring_count[i];\n+\n+\t\tif (!mpc-\u003empc_rings[i])\n+\t\t\tcontinue;\n+\t\tfor (j = 0; j \u003c num; j++) {\n+\t\t\tstruct bnxt_tx_ring_info *txr = \u0026mpc-\u003empc_rings[i][j];\n+\t\t\tstruct bnxt_ring_struct *ring = \u0026txr-\u003etx_ring_struct;\n+\n+\t\t\tbnxt_free_ring(bp, \u0026ring-\u003ering_mem);\n+\t\t}\n+\t}\n+}\n+\n+void bnxt_init_mpc_rings(struct bnxt *bp)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tint i, j;\n+\n+\tif (!mpc)\n+\t\treturn;\n+\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tint num = mpc-\u003empc_ring_count[i];\n+\n+\t\tfor (j = 0; j \u003c num; j++) {\n+\t\t\tstruct bnxt_tx_ring_info *txr = \u0026mpc-\u003empc_rings[i][j];\n+\t\t\tstruct bnxt_ring_struct *ring = \u0026txr-\u003etx_ring_struct;\n+\n+\t\t\ttxr-\u003etx_prod = 0;\n+\t\t\ttxr-\u003etx_cons = 0;\n+\t\t\ttxr-\u003etx_hw_cons = 0;\n+\t\t\tring-\u003efw_ring_id = INVALID_HW_RING_ID;\n+\t\t}\n+\t}\n+}\n+\n+static int bnxt_hwrm_one_mpc_ring_alloc(struct bnxt *bp,\n+\t\t\t\t\tstruct bnxt_tx_ring_info *txr)\n+{\n+\tstruct bnxt_cp_ring_info *cpr = txr-\u003etx_cpr;\n+\tstruct bnxt_ring_struct *ring;\n+\tint rc;\n+\n+\tring = \u0026cpr-\u003ecp_ring_struct;\n+\tif (ring-\u003efw_ring_id == INVALID_HW_RING_ID) {\n+\t\trc = bnxt_hwrm_cp_ring_alloc_p5(bp, cpr);\n+\t\tif (rc)\n+\t\t\treturn rc;\n+\t}\n+\t/* tx_idx not used on P5_PLUS, so set it to 0 */\n+\treturn bnxt_hwrm_tx_ring_alloc(bp, txr, 0);\n+}\n+\n+int bnxt_hwrm_mpc_ring_alloc(struct bnxt *bp)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tint i, j, rc = 0;\n+\n+\tif (!mpc)\n+\t\treturn 0;\n+\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tint num = mpc-\u003empc_ring_count[i];\n+\n+\t\tfor (j = 0; j \u003c num; j++) {\n+\t\t\tstruct bnxt_tx_ring_info *txr = \u0026mpc-\u003empc_rings[i][j];\n+\n+\t\t\trc = bnxt_hwrm_one_mpc_ring_alloc(bp, txr);\n+\t\t\tif (rc)\n+\t\t\t\tgoto mpc_ring_alloc_exit;\n+\t\t}\n+\t}\n+mpc_ring_alloc_exit:\n+\tif (rc)\n+\t\tbnxt_hwrm_mpc_ring_free(bp, false);\n+\treturn rc;\n+}\n+\n+void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tstruct bnxt_cp_ring_info *cpr;\n+\tint i, j;\n+\n+\tif (!mpc)\n+\t\treturn;\n+\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tfor (j = 0; j \u003c mpc-\u003empc_ring_count[i]; j++)\n+\t\t\tbnxt_hwrm_tx_ring_free(bp, \u0026mpc-\u003empc_rings[i][j],\n+\t\t\t\t\t close_path);\n+\t}\n+\t/* CP rings must be freed at the end to guarantee that the HWRM_DONE\n+\t * responses for HWRM_RING_FREE can still be seen on the CP rings.\n+\t */\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tfor (j = 0; j \u003c mpc-\u003empc_ring_count[i]; j++) {\n+\t\t\tcpr = mpc-\u003empc_rings[i][j].tx_cpr;\n+\t\t\tif (cpr \u0026\u0026 cpr-\u003ecp_ring_struct.fw_ring_id !=\n+\t\t\t INVALID_HW_RING_ID)\n+\t\t\t\tbnxt_hwrm_cp_ring_free(bp, cpr);\n+\t\t}\n+\t}\n+}\n+\n+struct bnxt_tx_ring_info *bnxt_select_mpc_ring(struct bnxt *bp, int ring_type)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tint n;\n+\n+\tif (!mpc || ring_type \u003e= BNXT_MPC_TYPE_MAX ||\n+\t !mpc-\u003empc_ring_count[ring_type])\n+\t\treturn NULL;\n+\n+\tn = raw_smp_processor_id() % mpc-\u003empc_ring_count[ring_type];\n+\treturn \u0026mpc-\u003empc_rings[ring_type][n];\n+}\n+\n+/**\n+ * bnxt_start_xmit_mpc - Transmit message on an MPC ring\n+ * @bp: pointer to bnxt device\n+ * @txr: MPC TX ring structure pointer\n+ * @data: MPC message pointer\n+ * @len: MPC message length\n+ * @handle: Non-zero handle passed back for the completion\n+ *\n+ * This function is called to transmit an MPC message on an MPC TX ring.\n+ * The caller must hold txr-\u003etx_lock. When successful, the HW will return\n+ * a completion and bnxt_crypto_mpc_cmp() will be called with the handle\n+ * passed back.\n+ *\n+ * Return: zero on success, negative error code otherwise:\n+ *\tENODEV: MPC TX ring is shutting down.\n+ *\tEBUSY: MPC TX ring is full\n+ */\n+int bnxt_start_xmit_mpc(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n+\t\t\tvoid *data, unsigned int len, unsigned long handle)\n+{\n+\tu32 bds, total_bds, bd_space, free_size;\n+\tstruct bnxt_sw_mpc_tx_bd *tx_buf;\n+\tstruct tx_bd *txbd;\n+\tu16 prod;\n+\n+\tif (READ_ONCE(txr-\u003edev_state) == BNXT_DEV_STATE_CLOSING)\n+\t\treturn -ENODEV;\n+\n+\tbds = DIV_ROUND_UP(len, sizeof(*txbd));\n+\ttotal_bds = bds + 1;\n+\tfree_size = bnxt_tx_avail(bp, txr);\n+\tif (free_size \u003c total_bds)\n+\t\treturn -EBUSY;\n+\n+\tprod = txr-\u003etx_prod;\n+\ttxbd = \u0026txr-\u003etx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)];\n+\ttx_buf = \u0026txr-\u003etx_mpc_buf_ring[RING_TX(bp, prod)];\n+\ttx_buf-\u003ehandle = handle;\n+\ttx_buf-\u003einline_bds = total_bds;\n+\n+\ttxbd-\u003etx_bd_len_flags_type =\n+\t\tcpu_to_le32((len \u003c\u003c TX_BD_LEN_SHIFT) | TX_BD_TYPE_MPC_TX_BD |\n+\t\t\t TX_BD_CNT(total_bds));\n+\ttxbd-\u003etx_bd_opaque = SET_TX_OPAQUE(bp, txr, prod, total_bds);\n+\n+\tprod = NEXT_TX(prod);\n+\ttxbd = \u0026txr-\u003etx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)];\n+\tbd_space = TX_DESC_CNT - TX_IDX(prod);\n+\tif (bd_space \u003c bds) {\n+\t\tunsigned int len0 = bd_space * sizeof(*txbd);\n+\n+\t\tmemcpy(txbd, data, len0);\n+\t\tprod += bd_space;\n+\t\ttxbd = \u0026txr-\u003etx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)];\n+\t\tbds -= bd_space;\n+\t\tlen -= len0;\n+\t\tdata += len0;\n+\t}\n+\tmemcpy(txbd, data, len);\n+\tprod += bds;\n+\tWRITE_ONCE(txr-\u003etx_prod, prod);\n+\n+\t/* Sync BD data before updating doorbell */\n+\twmb();\n+\tbnxt_db_write(bp, \u0026txr-\u003etx_db, prod);\n+\n+\treturn 0;\n+}\n+\n+/* Returns true if the ring is successfully marked as closing. */\n+static bool bnxt_disable_mpc_ring(struct bnxt_mpc_info *mpc, int mpc_ring)\n+{\n+\tstruct bnxt_tx_ring_info *txr;\n+\tbool disabled = false;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tif (mpc_ring \u003e= mpc-\u003empc_ring_count[i])\n+\t\t\tcontinue;\n+\t\ttxr = \u0026mpc-\u003empc_rings[i][mpc_ring];\n+\t\tspin_lock_bh(\u0026txr-\u003etx_lock);\n+\t\tif (!READ_ONCE(txr-\u003edev_state)) {\n+\t\t\tdisabled = true;\n+\t\t\tWRITE_ONCE(txr-\u003edev_state, BNXT_DEV_STATE_CLOSING);\n+\t\t}\n+\t\tspin_unlock_bh(\u0026txr-\u003etx_lock);\n+\t\tif (!disabled)\n+\t\t\tbreak;\n+\t}\n+\t/* Make sure napi polls see @dev_state change */\n+\tif (disabled)\n+\t\tsynchronize_net();\n+\treturn disabled;\n+}\n+\n+static void bnxt_enable_mpc_ring(struct bnxt_mpc_info *mpc, int mpc_ring)\n+{\n+\tstruct bnxt_tx_ring_info *txr;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tif (mpc_ring \u003e= mpc-\u003empc_ring_count[i])\n+\t\t\tcontinue;\n+\t\ttxr = \u0026mpc-\u003empc_rings[i][mpc_ring];\n+\t\tWRITE_ONCE(txr-\u003edev_state, 0);\n+\t}\n+}\n+\n+static void bnxt_clear_one_mpc_entries(struct bnxt *bp,\n+\t\t\t\t struct bnxt_tx_ring_info *txr)\n+{\n+\tstruct bnxt_sw_mpc_tx_bd *tx_buf;\n+\tunsigned long handle;\n+\tint i, max_idx;\n+\tu32 client;\n+\n+\tmax_idx = bp-\u003etx_nr_pages * TX_DESC_CNT;\n+\n+\tfor (i = 0; i \u003c max_idx; i++) {\n+\t\ttx_buf = \u0026txr-\u003etx_mpc_buf_ring[i];\n+\t\thandle = tx_buf-\u003ehandle;\n+\t\tif (handle) {\n+\t\t\tclient = txr-\u003etx_ring_struct.mpc_chnl_type;\n+\t\t\tbnxt_crypto_mpc_cmp(bp, client, handle, NULL, 0);\n+\t\t\ttx_buf-\u003ehandle = 0;\n+\t\t}\n+\t}\n+}\n+\n+static void bnxt_mpc_ring_stop(struct bnxt *bp, struct bnxt_mpc_info *mpc,\n+\t\t\t int mpc_ring)\n+{\n+\tstruct bnxt_tx_ring_info *txr;\n+\tstruct bnxt_cp_ring_info *cpr;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tif (mpc-\u003empc_ring_count[i] \u003e mpc_ring) {\n+\t\t\ttxr = \u0026mpc-\u003empc_rings[i][mpc_ring];\n+\t\t\tbnxt_hwrm_tx_ring_free(bp, txr, true);\n+\t\t}\n+\t}\n+\t/* CP rings must be freed at the end to guarantee that the HWRM_DONE\n+\t * responses for HWRM_RING_FREE can still be seen on the CP rings.\n+\t */\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tif (mpc-\u003empc_ring_count[i] \u003e mpc_ring) {\n+\t\t\ttxr = \u0026mpc-\u003empc_rings[i][mpc_ring];\n+\t\t\tcpr = txr-\u003etx_cpr;\n+\t\t\tif (cpr) {\n+\t\t\t\tbnxt_hwrm_cp_ring_free(bp, cpr);\n+\t\t\t\tbnxt_clear_one_cp_ring(bp, cpr);\n+\t\t\t}\n+\t\t\tbnxt_clear_one_mpc_entries(bp, txr);\n+\t\t}\n+\t}\n+}\n+\n+static int bnxt_mpc_ring_start(struct bnxt *bp, struct bnxt_mpc_info *mpc,\n+\t\t\t int mpc_ring)\n+{\n+\tstruct bnxt_tx_ring_info *txr;\n+\tint i, rc;\n+\n+\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\tif (mpc-\u003empc_ring_count[i] \u003e mpc_ring) {\n+\t\t\ttxr = \u0026mpc-\u003empc_rings[i][mpc_ring];\n+\t\t\ttxr-\u003etx_prod = 0;\n+\t\t\ttxr-\u003etx_cons = 0;\n+\t\t\ttxr-\u003etx_hw_cons = 0;\n+\t\t\trc = bnxt_hwrm_one_mpc_ring_alloc(bp, txr);\n+\t\t\tif (rc)\n+\t\t\t\treturn rc;\n+\t\t}\n+\t}\n+\treturn 0;\n+}\n+\n+static int bnxt_mpc_ring_reset(struct bnxt *bp, int mpc_ring)\n+{\n+\tstruct bnxt_mpc_info *mpc = bp-\u003empc_info;\n+\tstruct bnxt_tx_ring_info *txr;\n+\tint i, rc;\n+\n+\tif (!mpc)\n+\t\treturn 0;\n+\tif (mpc_ring \u003e= mpc-\u003empc_cp_rings)\n+\t\treturn -EINVAL;\n+\n+\tif (!bnxt_disable_mpc_ring(mpc, mpc_ring))\n+\t\treturn 0;\n+\n+\t/* If device is going down, the MPC rings will be freed anyway so just\n+\t * clear the MPC entries.\n+\t */\n+\tif (!test_bit(BNXT_STATE_OPEN, \u0026bp-\u003estate)) {\n+\t\tfor (i = 0; i \u003c BNXT_MPC_TYPE_MAX; i++) {\n+\t\t\tif (mpc-\u003empc_ring_count[i] \u003e mpc_ring) {\n+\t\t\t\ttxr = \u0026mpc-\u003empc_rings[i][mpc_ring];\n+\t\t\t\tbnxt_clear_one_mpc_entries(bp, txr);\n+\t\t\t}\n+\t\t}\n+\t\tbnxt_enable_mpc_ring(mpc, mpc_ring);\n+\t\treturn 0;\n+\t}\n+\tnetdev_warn(bp-\u003edev, \"Resetting MPC ring %d\\n\", mpc_ring);\n+\tnetdev_lock(bp-\u003edev);\n+\tbnxt_mpc_ring_stop(bp, mpc, mpc_ring);\n+\n+\trc = bnxt_mpc_ring_start(bp, mpc, mpc_ring);\n+\tif (rc) {\n+\t\tnetdev_err(bp-\u003edev, \"Error starting MPC ring %d, rc: %d, resetting device\\n\",\n+\t\t\t mpc_ring, rc);\n+\t\tbnxt_mpc_ring_stop(bp, mpc, mpc_ring);\n+\t\tbnxt_reset_task(bp, true);\n+\t\tnetdev_unlock(bp-\u003edev);\n+\t\t/* Return here as bnxt_reset_task() will clear everything */\n+\t\treturn rc;\n+\t}\n+\tnetdev_unlock(bp-\u003edev);\n+\tbnxt_enable_mpc_ring(mpc, mpc_ring);\n+\treturn 0;\n+}\n+\n+int bnxt_mpc_timeout(struct bnxt *bp, struct bnxt_tx_ring_info *txr)\n+{\n+\tif (txr-\u003etx_ring_struct.queue_id == BNXT_MPC_QUEUE_ID)\n+\t\treturn bnxt_mpc_ring_reset(bp, txr-\u003etxq_index);\n+\treturn -EINVAL;\n+}\n+\n+static bool bnxt_mpc_unsolicit(struct mpc_cmp *mpcmp)\n+{\n+\tu32 client = MPC_CMP_CLIENT_TYPE(mpcmp);\n+\n+\tif (client != MPC_CMP_CLIENT_TCE \u0026\u0026 client != MPC_CMP_CLIENT_RCE)\n+\t\treturn false;\n+\treturn MPC_CMP_UNSOLICIT_SUBTYPE(mpcmp);\n+}\n+\n+int bnxt_mpc_cmp(struct bnxt *bp, struct bnxt_cp_ring_info *cpr, u32 *raw_cons)\n+{\n+\tstruct bnxt_cmpl_entry cmpl_entry_arr[2];\n+\tstruct bnxt_napi *bnapi = cpr-\u003ebnapi;\n+\tu16 cons = RING_CMP(*raw_cons);\n+\tstruct mpc_cmp *mpcmp, *mpcmp1;\n+\tu32 tmp_raw_cons = *raw_cons;\n+\tunsigned long handle = 0;\n+\tu32 client, cmpl_num;\n+\tu8 type;\n+\n+\tmpcmp = (struct mpc_cmp *)\n+\t\t\u0026cpr-\u003ecp_desc_ring[CP_RING(cons)][CP_IDX(cons)];\n+\ttype = MPC_CMP_CMP_TYPE(mpcmp);\n+\tcmpl_entry_arr[0].cmpl = mpcmp;\n+\tcmpl_entry_arr[0].len = sizeof(*mpcmp);\n+\tcmpl_num = 1;\n+\tif (type == MPC_CMP_TYPE_MID_PATH_LONG) {\n+\t\ttmp_raw_cons = NEXT_RAW_CMP(tmp_raw_cons);\n+\t\tcons = RING_CMP(tmp_raw_cons);\n+\t\tmpcmp1 = (struct mpc_cmp *)\n+\t\t\t \u0026cpr-\u003ecp_desc_ring[CP_RING(cons)][CP_IDX(cons)];\n+\n+\t\tif (!MPC_CMP_VALID(bp, mpcmp1, tmp_raw_cons))\n+\t\t\treturn -EBUSY;\n+\t\t/* The valid test of the entry must be done first before\n+\t\t * reading any further.\n+\t\t */\n+\t\tdma_rmb();\n+\t\tif (mpcmp1 == mpcmp + 1) {\n+\t\t\tcmpl_entry_arr[cmpl_num - 1].len += sizeof(*mpcmp1);\n+\t\t} else {\n+\t\t\tcmpl_entry_arr[cmpl_num].cmpl = mpcmp1;\n+\t\t\tcmpl_entry_arr[cmpl_num].len = sizeof(*mpcmp1);\n+\t\t\tcmpl_num++;\n+\t\t}\n+\t}\n+\tclient = MPC_CMP_CLIENT_TYPE(mpcmp) \u003e\u003e MPC_CMP_CLIENT_SFT;\n+\tif (client \u003e= BNXT_MPC_TYPE_MAX)\n+\t\tgoto cmp_done;\n+\n+\tif (!bnxt_mpc_unsolicit(mpcmp)) {\n+\t\tstruct bnxt_sw_mpc_tx_bd *mpc_buf;\n+\t\tstruct bnxt_tx_ring_info *txr;\n+\t\tu16 tx_cons;\n+\t\tu32 opaque;\n+\n+\t\topaque = mpcmp-\u003empc_cmp_opaque;\n+\t\ttxr = bnapi-\u003etx_mpc_ring[client];\n+\t\ttx_cons = txr-\u003etx_cons;\n+\t\tif (TX_OPAQUE_RING(opaque) != txr-\u003etx_napi_idx) {\n+\t\t\tnetdev_warn(bp-\u003edev, \"Wrong opaque %x, expected ring %x, cons idx %x\\n\",\n+\t\t\t\t opaque, txr-\u003etx_napi_idx, txr-\u003etx_cons);\n+\t\t\tgoto cmp_done;\n+\t\t}\n+\t\tmpc_buf = \u0026txr-\u003etx_mpc_buf_ring[RING_TX(bp, tx_cons)];\n+\t\tif (!READ_ONCE(txr-\u003edev_state)) {\n+\t\t\thandle = mpc_buf-\u003ehandle;\n+\t\t\tmpc_buf-\u003ehandle = 0;\n+\t\t}\n+\t\ttx_cons += mpc_buf-\u003einline_bds;\n+\t\tWRITE_ONCE(txr-\u003etx_cons, tx_cons);\n+\t\ttxr-\u003etx_hw_cons = RING_TX(bp, tx_cons);\n+\t}\n+\tbnxt_crypto_mpc_cmp(bp, client, handle, cmpl_entry_arr, cmpl_num);\n+\n+cmp_done:\n+\t*raw_cons = tmp_raw_cons;\n+\treturn 0;\n+}\ndiff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h\nnew file mode 100644\nindex 00000000000000..b9a9fc77166520\n--- /dev/null\n+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h\n@@ -0,0 +1,210 @@\n+/* SPDX-License-Identifier: GPL-2.0-only */\n+/* Copyright (c) 2026 Broadcom Inc. */\n+\n+#ifndef BNXT_MPC_H\n+#define BNXT_MPC_H\n+\n+/* Mid path channel (MPC) definitions. An MPC is special TX/completion\n+ * ring pair to send/receive control plane data to the TCE and RCE\n+ * (Transmit/Receive Crypto Engine) HW blocks.\n+ */\n+\n+enum bnxt_mpc_type {\n+\tBNXT_MPC_TCE_TYPE = RING_ALLOC_REQ_MPC_CHNLS_TYPE_TCE,\n+\tBNXT_MPC_RCE_TYPE = RING_ALLOC_REQ_MPC_CHNLS_TYPE_RCE,\n+\tBNXT_MPC_TYPE_MAX,\n+};\n+\n+#define BNXT_MAX_MPC\t\t8\n+\n+#define BNXT_MIN_MPC_TCE\t1\n+#define BNXT_MIN_MPC_RCE\t1\n+#define BNXT_DFLT_MPC_TCE\tBNXT_MAX_MPC\n+#define BNXT_DFLT_MPC_RCE\tBNXT_MAX_MPC\n+\n+#define BNXT_MPC_TMO_MSECS 1000\n+\n+struct bnxt_mpc_info {\n+\tu8\t\t\tmpc_chnls_cap;\n+\tu8\t\t\tmpc_cp_rings;\n+\tu8\t\t\tmpc_ring_count[BNXT_MPC_TYPE_MAX];\n+\tstruct bnxt_tx_ring_info *mpc_rings[BNXT_MPC_TYPE_MAX];\n+};\n+\n+struct bnxt_sw_mpc_tx_bd {\n+\tu8 inline_bds;\n+\tunsigned long handle;\n+};\n+\n+#define SW_MPC_TXBD_RING_SIZE (sizeof(struct bnxt_sw_mpc_tx_bd) * TX_DESC_CNT)\n+\n+struct bnxt_cmpl_entry {\n+\tvoid *cmpl;\n+\tu32 len;\n+};\n+\n+struct mpc_cmp {\n+\t__le32 mpc_cmp_client_subtype_type;\n+\t#define MPC_CMP_TYPE\t\t\t\t\t(0x3f \u003c\u003c 0)\n+\t #define MPC_CMP_TYPE_MID_PATH_SHORT\t\t\t 0x1e\n+\t #define MPC_CMP_TYPE_MID_PATH_LONG\t\t\t 0x1f\n+\t#define MPC_CMP_SUBTYPE\t\t\t\t\t0xf00\n+\t#define MPC_CMP_SUBTYPE_SFT\t\t\t\t 8\n+\t #define MPC_CMP_SUBTYPE_SOLICITED\t\t\t (0x0 \u003c\u003c 8)\n+\t #define MPC_CMP_SUBTYPE_ERR\t\t\t\t (0x1 \u003c\u003c 8)\n+\t #define MPC_CMP_SUBTYPE_RESYNC\t\t\t\t (0x2 \u003c\u003c 8)\n+\t#define MPC_CMP_CLIENT\t\t\t\t\t(0xf \u003c\u003c 12)\n+\t #define MPC_CMP_CLIENT_SFT\t\t\t\t 12\n+\t #define MPC_CMP_CLIENT_TCE\t\t\t\t (0x0 \u003c\u003c 12)\n+\t #define MPC_CMP_CLIENT_RCE\t\t\t\t (0x1 \u003c\u003c 12)\n+\t #define MPC_CMP_CLIENT_TE_CFA\t\t\t\t (0x2 \u003c\u003c 12)\n+\t #define MPC_CMP_CLIENT_RE_CFA\t\t\t\t (0x3 \u003c\u003c 12)\n+\tu32 mpc_cmp_opaque;\n+\t__le32 mpc_cmp_v;\n+\t#define MPC_CMP_V\t\t\t\t\t(1 \u003c\u003c 0)\n+\t__le32 mpc_cmp_filler;\n+};\n+\n+#define MPC_CMP_CMP_TYPE(mpcmp)\t\t\t\t\t\t\\\n+\t(le32_to_cpu((mpcmp)-\u003empc_cmp_client_subtype_type) \u0026 MPC_CMP_TYPE)\n+\n+#define MPC_CMP_CLIENT_TYPE(mpcmp)\t\t\t\t\t\\\n+\t(le32_to_cpu((mpcmp)-\u003empc_cmp_client_subtype_type) \u0026 MPC_CMP_CLIENT)\n+\n+#define MPC_CMP_UNSOLICIT_SUBTYPE(mpcmp)\t\t\t\t\\\n+\t((le32_to_cpu((mpcmp)-\u003empc_cmp_client_subtype_type) \u0026\t\t\\\n+\t MPC_CMP_SUBTYPE) != MPC_CMP_SUBTYPE_SOLICITED)\n+\n+#define MPC_CMP_VALID(bp, mpcmp, raw_cons)\t\t\t\t\\\n+\t(!!((mpcmp)-\u003empc_cmp_v \u0026 cpu_to_le32(MPC_CMP_V)) ==\t\t\\\n+\t !((raw_cons) \u0026 (bp)-\u003ecp_bit))\n+\n+#define BNXT_MPC_CRYPTO_CAP \\\n+\t(FUNC_QCAPS_RESP_MPC_CHNLS_CAP_TCE | FUNC_QCAPS_RESP_MPC_CHNLS_CAP_RCE)\n+\n+#define BNXT_MPC_CRYPTO_CAPABLE(bp)\t\t\t\t\t\\\n+\t((bp)-\u003empc_info ?\t\t\t\t\t\t\\\n+\t ((bp)-\u003empc_info-\u003empc_chnls_cap \u0026 BNXT_MPC_CRYPTO_CAP) ==\t\\\n+\t BNXT_MPC_CRYPTO_CAP : false)\n+\n+#ifdef CONFIG_BNXT_TLS\n+void bnxt_alloc_mpc_info(struct bnxt *bp, u8 mpc_chnls_cap);\n+void bnxt_free_mpc_info(struct bnxt *bp);\n+int bnxt_mpc_tx_rings_in_use(struct bnxt *bp);\n+int bnxt_mpc_cp_rings_in_use(struct bnxt *bp);\n+bool bnxt_napi_has_mpc(struct bnxt *bp, int i);\n+void bnxt_set_mpc_cp_ring(struct bnxt *bp, int bnapi_idx,\n+\t\t\t struct bnxt_cp_ring_info *cpr);\n+void bnxt_trim_mpc_rings(struct bnxt *bp);\n+void bnxt_set_dflt_mpc_rings(struct bnxt *bp);\n+void bnxt_init_mpc_ring_struct(struct bnxt *bp);\n+int bnxt_alloc_mpcs(struct bnxt *bp);\n+void bnxt_free_mpcs(struct bnxt *bp);\n+int bnxt_alloc_mpc_rings(struct bnxt *bp);\n+void bnxt_free_mpc_rings(struct bnxt *bp);\n+void bnxt_init_mpc_rings(struct bnxt *bp);\n+int bnxt_hwrm_mpc_ring_alloc(struct bnxt *bp);\n+void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path);\n+struct bnxt_tx_ring_info *bnxt_select_mpc_ring(struct bnxt *bp, int ring_type);\n+int bnxt_start_xmit_mpc(struct bnxt *bp, struct bnxt_tx_ring_info *txr,\n+\t\t\tvoid *data, unsigned int len, unsigned long handle);\n+int bnxt_mpc_timeout(struct bnxt *bp, struct bnxt_tx_ring_info *txr);\n+int bnxt_mpc_cmp(struct bnxt *bp, struct bnxt_cp_ring_info *cpr, u32 *raw_cons);\n+#else\n+static inline void bnxt_alloc_mpc_info(struct bnxt *bp, u8 mpc_chnls_cap)\n+{\n+}\n+\n+static inline void bnxt_free_mpc_info(struct bnxt *bp)\n+{\n+}\n+\n+static inline int bnxt_mpc_tx_rings_in_use(struct bnxt *bp)\n+{\n+\treturn 0;\n+}\n+\n+static inline int bnxt_mpc_cp_rings_in_use(struct bnxt *bp)\n+{\n+\treturn 0;\n+}\n+\n+static inline bool bnxt_napi_has_mpc(struct bnxt *bp, int i)\n+{\n+\treturn false;\n+}\n+\n+static inline void bnxt_set_mpc_cp_ring(struct bnxt *bp, int bnapi_idx,\n+\t\t\t\t\tstruct bnxt_cp_ring_info *cpr)\n+{\n+}\n+\n+static inline void bnxt_trim_mpc_rings(struct bnxt *bp)\n+{\n+}\n+\n+static inline void bnxt_set_dflt_mpc_rings(struct bnxt *bp)\n+{\n+}\n+\n+static inline void bnxt_init_mpc_ring_struct(struct bnxt *bp)\n+{\n+}\n+\n+static inline int bnxt_alloc_mpcs(struct bnxt *bp)\n+{\n+\treturn 0;\n+}\n+\n+static inline void bnxt_free_mpcs(struct bnxt *bp)\n+{\n+}\n+\n+static inline int bnxt_alloc_mpc_rings(struct bnxt *bp)\n+{\n+\treturn 0;\n+}\n+\n+static inline void bnxt_free_mpc_rings(struct bnxt *bp)\n+{\n+}\n+\n+static inline void bnxt_init_mpc_rings(struct bnxt *bp)\n+{\n+}\n+\n+static inline int bnxt_hwrm_mpc_ring_alloc(struct bnxt *bp)\n+{\n+\treturn 0;\n+}\n+\n+static inline void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path)\n+{\n+}\n+\n+static inline struct bnxt_tx_ring_info *bnxt_select_mpc_ring(struct bnxt *bp,\n+\t\t\t\t\t\t\t int ring_type)\n+{\n+\treturn NULL;\n+}\n+\n+static inline int bnxt_start_xmit_mpc(struct bnxt *bp,\n+\t\t\t\t struct bnxt_tx_ring_info *txr, void *data,\n+\t\t\t\t unsigned int len, unsigned long handle)\n+{\n+\treturn -EOPNOTSUPP;\n+}\n+\n+static inline int bnxt_mpc_timeout(struct bnxt *bp,\n+\t\t\t\t struct bnxt_tx_ring_info *txr)\n+{\n+\treturn -EOPNOTSUPP;\n+}\n+\n+static inline int bnxt_mpc_cmp(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,\n+\t\t\t u32 *raw_cons)\n+{\n+\treturn 0;\n+}\n+#endif\t/* CONFIG_BNXT_TLS */\n+#endif\t/* BNXT_MPC_H */\ndiff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c\nindex edcc002e4ca358..d57059722f5bb0 100644\n--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c\n+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c\n@@ -640,7 +640,7 @@ static int bnxt_hwrm_func_vf_resc_cfg(struct bnxt *bp, int num_vfs, bool reset)\n \t\tvf_rx_rings = hw_resc-\u003emax_rx_rings - bp-\u003erx_nr_rings * 2;\n \telse\n \t\tvf_rx_rings = hw_resc-\u003emax_rx_rings - bp-\u003erx_nr_rings;\n-\tvf_tx_rings = hw_resc-\u003emax_tx_rings - bp-\u003etx_nr_rings;\n+\tvf_tx_rings = hw_resc-\u003emax_tx_rings - bnxt_total_tx_rings(bp);\n \tvf_vnics = hw_resc-\u003emax_vnics - bp-\u003enr_vnics;\n \tvf_rss = hw_resc-\u003emax_rsscos_ctxs - bp-\u003ersscos_nr_ctxs;\n \n@@ -903,8 +903,8 @@ static int bnxt_sriov_enable(struct bnxt *bp, int *num_vfs)\n \t\t avail_cp \u003c min_rx_rings)\n \t\t\trx_ok = 0;\n \n-\t\tif (hw_resc-\u003emax_tx_rings - bp-\u003etx_nr_rings \u003e= min_tx_rings \u0026\u0026\n-\t\t avail_cp \u003e= min_tx_rings)\n+\t\tif (hw_resc-\u003emax_tx_rings - bnxt_total_tx_rings(bp) \u003e=\n+\t\t min_tx_rings \u0026\u0026 avail_cp \u003e= min_tx_rings)\n \t\t\ttx_ok = 1;\n \n \t\tif (hw_resc-\u003emax_rsscos_ctxs - bp-\u003ersscos_nr_ctxs \u003e=\ndiff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c\nindex 9e5009be8e9898..2a94a77847fe16 100644\n--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c\n+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c\n@@ -351,7 +351,7 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames,\n \t\treturn -EINVAL;\n \n \tif (static_branch_unlikely(\u0026bnxt_xdp_locking_key))\n-\t\tspin_lock(\u0026txr-\u003exdp_tx_lock);\n+\t\tspin_lock(\u0026txr-\u003etx_lock);\n \n \tfor (i = 0; i \u003c num_frames; i++) {\n \t\tstruct xdp_frame *xdp = frames[i];\n@@ -376,7 +376,7 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames,\n \t}\n \n \tif (static_branch_unlikely(\u0026bnxt_xdp_locking_key))\n-\t\tspin_unlock(\u0026txr-\u003exdp_tx_lock);\n+\t\tspin_unlock(\u0026txr-\u003etx_lock);\n \n \treturn nxmit;\n }\ndiff --git a/include/linux/bnxt/hsi.h b/include/linux/bnxt/hsi.h\nindex 74a6bf278d888c..03444b81beb0f3 100644\n--- a/include/linux/bnxt/hsi.h\n+++ b/include/linux/bnxt/hsi.h\n@@ -3837,6 +3837,43 @@ struct hwrm_func_ptp_ext_qcfg_output {\n \tu8\tvalid;\n };\n \n+/* hwrm_func_key_ctx_alloc_input (size:384b/48B) */\n+struct hwrm_func_key_ctx_alloc_input {\n+\t__le16\treq_type;\n+\t__le16\tcmpl_ring;\n+\t__le16\tseq_id;\n+\t__le16\ttarget_id;\n+\t__le64\tresp_addr;\n+\t__le16\tfid;\n+\t__le16\tnum_key_ctxs;\n+\t__le32\tdma_bufr_size_bytes;\n+\tu8\tkey_ctx_type;\n+\t#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_TX 0x0UL\n+\t#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_RX 0x1UL\n+\t#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_QUIC_TX 0x2UL\n+\t#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_QUIC_RX 0x3UL\n+\t#define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_LAST FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_QUIC_RX\n+\tu8\tunused_0[7];\n+\t__le64\thost_dma_addr;\n+\t__le32\tpartition_start_xid;\n+\tu8\tunused_1[4];\n+};\n+\n+/* hwrm_func_key_ctx_alloc_output (size:192b/24B) */\n+struct hwrm_func_key_ctx_alloc_output {\n+\t__le16\terror_code;\n+\t__le16\treq_type;\n+\t__le16\tseq_id;\n+\t__le16\tresp_len;\n+\t__le16\tnum_key_ctxs_allocated;\n+\tu8\tflags;\n+\t#define FUNC_KEY_CTX_ALLOC_RESP_FLAGS_KEY_CTXS_CONTIGUOUS 0x1UL\n+\tu8\tunused_0;\n+\t__le32\tpartition_start_xid;\n+\tu8\tunused_1[7];\n+\tu8\tvalid;\n+};\n+\n /* hwrm_func_backing_store_cfg_v2_input (size:512b/64B) */\n struct hwrm_func_backing_store_cfg_v2_input {\n \t__le16\treq_type;\n"
} |
| 2/1 | 2026/06/29 20:22 | agent | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"Reasoning": "The patch exclusively modifies the Broadcom NetXtreme-C/E (bnxt) hardware driver to add TLS offload support. Since there is no software emulation for this specific hardware in virtualized environments like QEMU or GCE, the modified code paths are unreachable during standard fuzzing.",
"WorthFuzzing": false
}Instruction: You are an expert Linux kernel maintainer. Your job is to review a provided patch series and determine if it makes functional changes to the kernel that should be fuzzed. IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in your workspace. Do NOT rely on your internal knowledge of the kernel. You must actively use your code access tools to examine the actual source code and confirm any assumptions. Return WorthFuzzing=false if the patch only contains: - Modifications to Documentation/, Kconfig files, or code comments. - Purely decorative changes, such as logging (e.g., pr_err, printk) or tracepoints. - Changes to numeric constants or macros that do not functionally alter execution flow. - Code paths that are impossible to reach in virtualized environments like GCE or QEMU, even when utilizing software-emulated hardware (e.g., usb gadget, mac80211_hwsim). If it modifies reachable core kernel logic, drivers, or architectures, use your code search tools to verify the code can be executed, then return WorthFuzzing=true. When returning WorthFuzzing=true, you MUST ALSO: 1. Extract any specific kernel functions that should be heavily fuzzed into FocusSymbols. Avoid listing generic hot-path functions to prevent skewed test distributions. 2. Identify any specific CONFIG_ options required to properly test this new/modified feature. Go and look into the Kconfig files and check for ifdefs around the code, do not make assumptions. Do not list too generic configs (we already have them enabled). Only list those that specifically cover the modified code. List them in the EnableConfigs output array, and DO NOT add a 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4"). Prefer calling several tools at the same time to save round-trips. Use set-results tool to provide results of the analysis. It must be called exactly once before the final reply. Ignore results of this tool. Prompt: For your convenience, here is the diff of the changes:
commit 4ea71aee837cf1a383bbb583beebf054a50cf55e
Author: syz-cluster <triage@syzkaller.com>
Date: Mon Jun 29 20:22:20 2026 +0000
syz-cluster: applied patch under review
diff --git a/drivers/net/ethernet/broadcom/Kconfig b/drivers/net/ethernet/broadcom/Kconfig
index f0bac0dd143900..b33b66f038b882 100644
--- a/drivers/net/ethernet/broadcom/Kconfig
+++ b/drivers/net/ethernet/broadcom/Kconfig
@@ -255,6 +255,15 @@ config BNXT_HWMON
Say Y if you want to expose the thermal sensor data on NetXtreme-C/E
devices, via the hwmon sysfs interface.
+config BNXT_TLS
+ bool "Broadcom NetXtreme-C/E TLS offload support"
+ default y
+ depends on BNXT && TLS_DEVICE
+ depends on TLS=y || BNXT=m
+ help
+ Say Y if you want to enable Transport Layer Security (TLS) hardware
+ encryption and decryption offload on supported NetXtreme-C/E devices.
+
config BNGE
tristate "Broadcom ThorUltra Ethernet device support"
depends on PCI
diff --git a/drivers/net/ethernet/broadcom/bnxt/Makefile b/drivers/net/ethernet/broadcom/bnxt/Makefile
index debef78c8b6d64..88e68248aad420 100644
--- a/drivers/net/ethernet/broadcom/bnxt/Makefile
+++ b/drivers/net/ethernet/broadcom/bnxt/Makefile
@@ -5,3 +5,4 @@ bnxt_en-y := bnxt.o bnxt_hwrm.o bnxt_sriov.o bnxt_ethtool.o bnxt_dcb.o bnxt_ulp.
bnxt_en-$(CONFIG_BNXT_FLOWER_OFFLOAD) += bnxt_tc.o
bnxt_en-$(CONFIG_DEBUG_FS) += bnxt_debugfs.o
bnxt_en-$(CONFIG_BNXT_HWMON) += bnxt_hwmon.o
+bnxt_en-$(CONFIG_BNXT_TLS) += bnxt_mpc.o bnxt_crypto.o bnxt_ktls.o
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 7513618793daf3..baca7ee318fbe1 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -76,6 +76,9 @@
#include "bnxt_hwmon.h"
#include "bnxt_gso.h"
#include <net/tso.h>
+#include "bnxt_mpc.h"
+#include "bnxt_crypto.h"
+#include "bnxt_ktls.h"
#define BNXT_TX_TIMEOUT (5 * HZ)
#define BNXT_DEF_MSG_ENABLE (NETIF_MSG_DRV | NETIF_MSG_HW | \
@@ -478,13 +481,15 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)
dma_addr_t mapping;
unsigned int length, pad = 0;
u32 len, free_size, vlan_tag_flags, cfa_action, flags;
+ struct bnxt_ktls_offload_ctx_tx *kctx_tx = NULL;
+ u16 prod, start_prod, last_frag, txts_prod;
struct bnxt_ptp_cfg *ptp = bp->ptp_cfg;
struct pci_dev *pdev = bp->pdev;
- u16 prod, last_frag, txts_prod;
struct bnxt_tx_ring_info *txr;
struct bnxt_sw_tx_bd *tx_buf;
__le32 lflags = 0;
skb_frag_t *frag;
+ u32 kid = 0;
i = skb_get_queue_mapping(skb);
if (unlikely(i >= bp->tx_nr_rings)) {
@@ -495,7 +500,6 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)
txq = netdev_get_tx_queue(dev, i);
txr = &bp->tx_ring[bp->tx_ring_map[i]];
- prod = txr->tx_prod;
#if (MAX_SKB_FRAGS > TX_MAX_FRAGS)
if (skb_shinfo(skb)->nr_frags > TX_MAX_FRAGS) {
@@ -524,6 +528,14 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)
return NETDEV_TX_BUSY;
}
+ start_prod = txr->tx_prod;
+ skb = bnxt_ktls_xmit(bp, txr, skb, &lflags, &kid, &kctx_tx);
+ if (unlikely(!skb)) {
+ dev_core_stats_tx_dropped_inc(dev);
+ return NETDEV_TX_OK;
+ }
+
+ prod = txr->tx_prod;
length = skb->len;
len = skb_headlen(skb);
last_frag = skb_shinfo(skb)->nr_frags;
@@ -672,7 +684,7 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)
prod = NEXT_TX(prod);
txbd1 = bnxt_init_ext_bd(bp, txr, prod, lflags, vlan_tag_flags,
- cfa_action);
+ cfa_action, kid);
if (skb_is_gso(skb)) {
bool udp_gso = !!(skb_shinfo(skb)->gso_type & SKB_GSO_UDP_L4);
@@ -693,9 +705,10 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)
txbd1->tx_bd_hsize_lflags |= cpu_to_le32(TX_BD_FLAGS_LSO |
TX_BD_FLAGS_T_IPID |
- (hdr_len << (TX_BD_HSIZE_SHIFT - 1)));
+ ((hdr_len >> 1) << TX_BD_HSIZE_SHIFT));
length = skb_shinfo(skb)->gso_size;
- txbd1->tx_bd_mss = cpu_to_le32(length);
+ txbd1->tx_bd_kid_mss = cpu_to_le32(BNXT_TX_KID_HI(kid) |
+ (length & TX_BD_MSS));
length += hdr_len;
} else if (skb->ip_summed == CHECKSUM_PARTIAL) {
txbd1->tx_bd_hsize_lflags |=
@@ -748,6 +761,9 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)
prod = NEXT_TX(prod);
WRITE_ONCE(txr->tx_prod, prod);
+ /* Commit the kTLS state now that the BD is in the ring. */
+ bnxt_ktls_xmit_commit(txr, kctx_tx);
+
if (!netdev_xmit_more() || netif_xmit_stopped(txq)) {
bnxt_txr_db_kick(bp, txr, prod);
} else {
@@ -802,9 +818,16 @@ static netdev_tx_t bnxt_start_xmit(struct sk_buff *skb, struct net_device *dev)
/* set SKB to err so PTP worker will clean up */
ptp->txts_req[txts_prod].tx_skb = ERR_PTR(-EIO);
}
+ txr->tx_buf_ring[RING_TX(bp, txr->tx_prod)].skb = NULL;
+ /* Unwind any kTLS presync BDs */
+ if (unlikely(txr->tx_prod != start_prod)) {
+ tx_buf = &txr->tx_buf_ring[RING_TX(bp, start_prod)];
+ tx_buf->is_push = 0;
+ tx_buf->inline_data_bds = 0;
+ WRITE_ONCE(txr->tx_prod, start_prod);
+ }
if (txr->kick_pending)
bnxt_txr_db_kick(bp, txr, txr->tx_prod);
- txr->tx_buf_ring[RING_TX(bp, txr->tx_prod)].skb = NULL;
dev_core_stats_tx_dropped_inc(dev);
return NETDEV_TX_OK;
}
@@ -833,7 +856,7 @@ static bool __bnxt_tx_int(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
head_buf = tx_buf;
skb = tx_buf->skb;
- if (unlikely(!skb)) {
+ if (unlikely(!skb && !tx_buf->is_push)) {
bnxt_sched_reset_txr(bp, txr, cons);
return rc;
}
@@ -845,13 +868,22 @@ static bool __bnxt_tx_int(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
}
cons = NEXT_TX(cons);
- tx_pkts++;
- tx_bytes += skb->len;
+ if (skb) {
+ tx_pkts++;
+ tx_bytes += skb->len;
+ }
tx_buf->skb = NULL;
tx_buf->is_ts_pkt = 0;
if (tx_buf->is_push) {
tx_buf->is_push = 0;
+ cons += tx_buf->inline_data_bds;
+ tx_buf->inline_data_bds = 0;
+ if (!skb) {
+ /* presync BD */
+ cons = NEXT_TX(cons);
+ continue;
+ }
goto next_tx_int;
}
@@ -3113,6 +3145,9 @@ static int __bnxt_poll_work(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,
rx_pkts++;
else if (rc == -EBUSY) /* partial completion */
break;
+ } else if (cmp_type == CMP_TYPE_MPC_CMP) {
+ if (bnxt_mpc_cmp(bp, cpr, &raw_cons))
+ break;
} else if (unlikely(cmp_type == CMPL_BASE_TYPE_HWRM_DONE ||
cmp_type == CMPL_BASE_TYPE_HWRM_FWD_REQ ||
cmp_type == CMPL_BASE_TYPE_HWRM_ASYNC_EVENT)) {
@@ -3684,7 +3719,7 @@ static size_t __bnxt_copy_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem,
return total_len;
}
-static void bnxt_free_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem)
+void bnxt_free_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem)
{
struct pci_dev *pdev = bp->pdev;
int i;
@@ -3717,7 +3752,7 @@ static void bnxt_free_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem)
}
}
-static int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem)
+int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem)
{
struct pci_dev *pdev = bp->pdev;
u64 valid_bit = 0;
@@ -4079,6 +4114,8 @@ static void bnxt_free_tx_rings(struct bnxt *bp)
bnxt_free_tx_inline_buf(txr, pdev);
+ bnxt_ktls_free_tx_ring_stats(txr);
+
ring = &txr->tx_ring_struct;
bnxt_free_ring(bp, &ring->ring_mem);
@@ -4153,7 +4190,10 @@ static int bnxt_alloc_tx_rings(struct bnxt *bp)
}
qidx = bp->tc_to_qidx[j];
ring->queue_id = bp->q_info[qidx].queue_id;
- spin_lock_init(&txr->xdp_tx_lock);
+ spin_lock_init(&txr->tx_lock);
+ rc = bnxt_ktls_alloc_tx_ring_stats(bp, txr);
+ if (rc)
+ return rc;
if (i < bp->tx_nr_rings_xdp)
continue;
if (BNXT_RING_TO_TC_OFF(bp, i) == (bp->tx_nr_rings_per_tc - 1))
@@ -4324,6 +4364,8 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp)
(!sh && i >= bp->rx_nr_rings)) {
cp_count += tcs;
tx = 1;
+ if (bnxt_napi_has_mpc(bp, i))
+ cp_count++;
}
cpr->cp_ring_arr = kzalloc_objs(*cpr, cp_count);
@@ -4345,6 +4387,11 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp)
} else {
int n, tc = k - rx;
+ /* MPC rings are at the highest k indices */
+ if (tc >= tcs) {
+ bnxt_set_mpc_cp_ring(bp, i, cpr2);
+ continue;
+ }
n = BNXT_TC_TO_RING_BASE(bp, tc) + j;
bp->tx_ring[n].tx_cpr = cpr2;
cpr2->cp_ring_type = BNXT_NQ_HDL_TYPE_TX;
@@ -4477,6 +4524,7 @@ static void bnxt_init_ring_struct(struct bnxt *bp)
rmem->vmem = (void **)&txr->tx_buf_ring;
}
}
+ bnxt_init_mpc_ring_struct(bp);
}
static void bnxt_init_rxbd_pages(struct bnxt_ring_struct *ring, u32 type)
@@ -5553,6 +5601,7 @@ static void bnxt_init_l2_fltr_tbl(struct bnxt *bp)
static void bnxt_free_mem(struct bnxt *bp, bool irq_re_init)
{
bnxt_free_vnic_attributes(bp);
+ bnxt_free_mpc_rings(bp);
bnxt_free_tx_rings(bp);
bnxt_free_rx_rings(bp);
bnxt_free_cp_rings(bp);
@@ -5566,6 +5615,7 @@ static void bnxt_free_mem(struct bnxt *bp, bool irq_re_init)
bnxt_free_port_stats(bp);
bnxt_free_ring_grps(bp);
bnxt_free_vnics(bp);
+ bnxt_free_mpcs(bp);
kfree(bp->tx_ring_map);
bp->tx_ring_map = NULL;
kfree(bp->tx_ring);
@@ -5675,6 +5725,10 @@ static int bnxt_alloc_mem(struct bnxt *bp, bool irq_re_init)
txr->tx_cpr = &bnapi2->cp_ring;
}
+ rc = bnxt_alloc_mpcs(bp);
+ if (rc)
+ goto alloc_mem_err;
+
rc = bnxt_alloc_stats(bp);
if (rc)
goto alloc_mem_err;
@@ -5703,6 +5757,10 @@ static int bnxt_alloc_mem(struct bnxt *bp, bool irq_re_init)
if (rc)
goto alloc_mem_err;
+ rc = bnxt_alloc_mpc_rings(bp);
+ if (rc)
+ goto alloc_mem_err;
+
rc = bnxt_alloc_cp_rings(bp);
if (rc)
goto alloc_mem_err;
@@ -7257,10 +7315,15 @@ static int hwrm_ring_alloc_send_msg(struct bnxt *bp,
req->cmpl_ring_id = cpu_to_le16(bnxt_cp_ring_for_tx(bp, txr));
req->length = cpu_to_le32(bp->tx_ring_mask + 1);
req->stat_ctx_id = cpu_to_le32(grp_info->fw_stats_ctx);
- req->queue_id = cpu_to_le16(ring->queue_id);
- if (bp->flags & BNXT_FLAG_TX_COAL_CMPL)
- req->cmpl_coal_cnt =
- RING_ALLOC_REQ_CMPL_COAL_CNT_COAL_64;
+ if (ring->queue_id == BNXT_MPC_QUEUE_ID) {
+ req->mpc_chnls_type = ring->mpc_chnl_type;
+ req->enables |= cpu_to_le32(RING_ALLOC_REQ_ENABLES_MPC_CHNLS_TYPE);
+ } else {
+ req->queue_id = cpu_to_le16(ring->queue_id);
+ if (bp->flags & BNXT_FLAG_TX_COAL_CMPL)
+ req->cmpl_coal_cnt =
+ RING_ALLOC_REQ_CMPL_COAL_CNT_COAL_64;
+ }
if ((bp->fw_cap & BNXT_FW_CAP_TX_TS_CMP) && bp->ptp_cfg)
flags |= RING_ALLOC_REQ_FLAGS_TX_PKT_TS_CMPL_ENABLE;
req->flags = cpu_to_le16(flags);
@@ -7455,8 +7518,7 @@ static int bnxt_hwrm_rx_agg_ring_alloc(struct bnxt *bp,
return 0;
}
-static int bnxt_hwrm_cp_ring_alloc_p5(struct bnxt *bp,
- struct bnxt_cp_ring_info *cpr)
+int bnxt_hwrm_cp_ring_alloc_p5(struct bnxt *bp, struct bnxt_cp_ring_info *cpr)
{
const u32 type = HWRM_RING_ALLOC_CMPL;
struct bnxt_napi *bnapi = cpr->bnapi;
@@ -7474,8 +7536,8 @@ static int bnxt_hwrm_cp_ring_alloc_p5(struct bnxt *bp,
return 0;
}
-static int bnxt_hwrm_tx_ring_alloc(struct bnxt *bp,
- struct bnxt_tx_ring_info *txr, u32 tx_idx)
+int bnxt_hwrm_tx_ring_alloc(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ u32 tx_idx)
{
struct bnxt_ring_struct *ring = &txr->tx_ring_struct;
const u32 type = HWRM_RING_ALLOC_TX;
@@ -7560,6 +7622,9 @@ static int bnxt_hwrm_ring_alloc(struct bnxt *bp)
goto err_out;
}
}
+
+ rc = bnxt_hwrm_mpc_ring_alloc(bp);
+
err_out:
return rc;
}
@@ -7617,9 +7682,8 @@ static int hwrm_ring_free_send_msg(struct bnxt *bp,
return 0;
}
-static void bnxt_hwrm_tx_ring_free(struct bnxt *bp,
- struct bnxt_tx_ring_info *txr,
- bool close_path)
+void bnxt_hwrm_tx_ring_free(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ bool close_path)
{
struct bnxt_ring_struct *ring = &txr->tx_ring_struct;
u32 cmpl_ring_id;
@@ -7678,8 +7742,7 @@ static void bnxt_hwrm_rx_agg_ring_free(struct bnxt *bp,
bp->grp_info[grp_idx].agg_fw_ring_id = INVALID_HW_RING_ID;
}
-static void bnxt_hwrm_cp_ring_free(struct bnxt *bp,
- struct bnxt_cp_ring_info *cpr)
+void bnxt_hwrm_cp_ring_free(struct bnxt *bp, struct bnxt_cp_ring_info *cpr)
{
struct bnxt_ring_struct *ring;
@@ -7692,7 +7755,7 @@ static void bnxt_hwrm_cp_ring_free(struct bnxt *bp,
ring->fw_ring_id = INVALID_HW_RING_ID;
}
-static void bnxt_clear_one_cp_ring(struct bnxt *bp, struct bnxt_cp_ring_info *cpr)
+void bnxt_clear_one_cp_ring(struct bnxt *bp, struct bnxt_cp_ring_info *cpr)
{
struct bnxt_ring_struct *ring = &cpr->cp_ring_struct;
int i, size = ring->ring_mem.page_size;
@@ -7713,6 +7776,8 @@ static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path)
if (!bp->bnapi)
return;
+ bnxt_hwrm_mpc_ring_free(bp, close_path);
+
for (i = 0; i < bp->tx_nr_rings; i++)
bnxt_hwrm_tx_ring_free(bp, &bp->tx_ring[i], close_path);
@@ -7751,6 +7816,11 @@ static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path)
}
}
+int bnxt_total_tx_rings(struct bnxt *bp)
+{
+ return bp->tx_nr_rings + bnxt_mpc_tx_rings_in_use(bp);
+}
+
static int __bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max,
bool shared);
static int bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max,
@@ -7759,6 +7829,7 @@ static int bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max,
static int bnxt_hwrm_get_rings(struct bnxt *bp)
{
struct bnxt_hw_resc *hw_resc = &bp->hw_resc;
+ struct bnxt_hw_crypto_resc *crypto_resc;
struct hwrm_func_qcfg_output *resp;
struct hwrm_func_qcfg_input *req;
int rc;
@@ -7791,25 +7862,38 @@ static int bnxt_hwrm_get_rings(struct bnxt *bp)
stats = le16_to_cpu(resp->alloc_stat_ctx);
hw_resc->resv_irqs = cp;
if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
+ int mpc_tx = bnxt_mpc_tx_rings_in_use(bp);
+ int mpc_cp = bnxt_mpc_cp_rings_in_use(bp);
int rx = hw_resc->resv_rx_rings;
int tx = hw_resc->resv_tx_rings;
+ int cp_p5;
+ if (tx <= mpc_tx || cp <= mpc_cp) {
+ rc = -ENOMEM;
+ goto get_rings_exit;
+ }
+ tx -= mpc_tx;
+ cp_p5 = cp - mpc_cp;
if (bp->flags & BNXT_FLAG_AGG_RINGS)
rx >>= 1;
- if (cp < (rx + tx)) {
- rc = __bnxt_trim_rings(bp, &rx, &tx, cp, false);
+ if (cp_p5 < (rx + tx)) {
+ rc = __bnxt_trim_rings(bp, &rx, &tx, cp_p5, false);
if (rc)
goto get_rings_exit;
if (bp->flags & BNXT_FLAG_AGG_RINGS)
rx <<= 1;
hw_resc->resv_rx_rings = rx;
- hw_resc->resv_tx_rings = tx;
+ hw_resc->resv_tx_rings = tx + mpc_tx;
}
hw_resc->resv_irqs = le16_to_cpu(resp->alloc_msix);
hw_resc->resv_hw_ring_grps = rx;
}
hw_resc->resv_cp_rings = cp;
hw_resc->resv_stat_ctxs = stats;
+
+ crypto_resc = &hw_resc->crypto_resc;
+ crypto_resc->resv_tx_key_ctxs = le32_to_cpu(resp->num_ktls_tx_key_ctxs);
+ crypto_resc->resv_rx_key_ctxs = le32_to_cpu(resp->num_ktls_rx_key_ctxs);
}
get_rings_exit:
hwrm_req_drop(bp, req);
@@ -7880,8 +7964,9 @@ __bnxt_hwrm_reserve_pf_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr)
}
req->num_stat_ctxs = cpu_to_le16(hwr->stat);
req->num_vnics = cpu_to_le16(hwr->vnic);
+ bnxt_hwrm_reserve_pf_key_ctxs(bp, req);
}
- req->enables = cpu_to_le32(enables);
+ req->enables |= cpu_to_le32(enables);
return req;
}
@@ -7995,7 +8080,7 @@ static int bnxt_cp_rings_in_use(struct bnxt *bp)
return bnxt_nq_rings_in_use(bp);
cp = bp->tx_nr_rings + bp->rx_nr_rings;
- return cp;
+ return cp + bnxt_mpc_cp_rings_in_use(bp);
}
static int bnxt_get_func_stat_ctxs(struct bnxt *bp)
@@ -8053,7 +8138,7 @@ static void bnxt_get_total_resources(struct bnxt *bp, struct bnxt_hw_rings *hwr)
hwr->cp_p5 = 0;
if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS)
hwr->cp_p5 = bnxt_cp_rings_in_use(bp);
- hwr->tx = bp->tx_nr_rings;
+ hwr->tx = bnxt_total_tx_rings(bp);
hwr->rx = bp->rx_nr_rings;
hwr->grp = hwr->rx;
hwr->vnic = bnxt_get_total_vnics(bp, hwr->rx);
@@ -8159,8 +8244,10 @@ static int __bnxt_reserve_rings(struct bnxt *bp)
hwr.rx = bp->rx_nr_rings;
if (bp->flags & BNXT_FLAG_SHARED_RINGS)
sh = true;
- if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS)
- hwr.cp_p5 = hwr.rx + hwr.tx;
+ if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
+ hwr.cp_p5 = hwr.rx + hwr.tx + bnxt_mpc_cp_rings_in_use(bp);
+ hwr.tx += bnxt_mpc_tx_rings_in_use(bp);
+ }
hwr.vnic = bnxt_get_total_vnics(bp, hwr.rx);
@@ -8197,6 +8284,9 @@ static int __bnxt_reserve_rings(struct bnxt *bp)
if (bnxt_ulp_registered(edev) && hwr.stat > bnxt_get_ulp_stat_ctxs(bp))
hwr.stat -= bnxt_get_ulp_stat_ctxs(bp);
hwr.cp = min_t(int, hwr.cp, hwr.stat);
+ hwr.tx -= bnxt_mpc_tx_rings_in_use(bp);
+ if (hwr.tx < 0)
+ return -ENOMEM;
rc = bnxt_trim_rings(bp, &rx_rings, &hwr.tx, hwr.cp, sh);
if (bp->flags & BNXT_FLAG_AGG_RINGS)
hwr.rx = rx_rings << 1;
@@ -8278,7 +8368,7 @@ static int bnxt_hwrm_check_vf_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr)
static int bnxt_hwrm_check_pf_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr)
{
struct hwrm_func_cfg_input *req;
- u32 flags;
+ u32 flags, flags2 = 0;
req = __bnxt_hwrm_reserve_pf_rings(bp, hwr);
flags = FUNC_CFG_REQ_FLAGS_TX_ASSETS_TEST;
@@ -8292,9 +8382,14 @@ static int bnxt_hwrm_check_pf_rings(struct bnxt *bp, struct bnxt_hw_rings *hwr)
FUNC_CFG_REQ_FLAGS_NQ_ASSETS_TEST;
else
flags |= FUNC_CFG_REQ_FLAGS_RING_GRP_ASSETS_TEST;
+ if (req->enables &
+ cpu_to_le32(FUNC_CFG_REQ_ENABLES_KTLS_TX_KEY_CTXS |
+ FUNC_CFG_REQ_ENABLES_KTLS_RX_KEY_CTXS))
+ flags2 |= FUNC_CFG_REQ_FLAGS2_KTLS_KEY_CTX_ASSETS_TEST;
}
req->flags = cpu_to_le32(flags);
+ req->flags2 = cpu_to_le32(flags2);
return hwrm_req_send_silent(bp, req);
}
@@ -9331,12 +9426,35 @@ static int bnxt_hwrm_func_backing_store_cfg_v2(struct bnxt *bp,
static int bnxt_backing_store_cfg_v2(struct bnxt *bp)
{
+ struct bnxt_crypto_info *crypto = bp->crypto_info;
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
struct bnxt_ctx_mem_info *ctx = bp->ctx;
struct bnxt_ctx_mem_type *ctxm;
u16 last_type = BNXT_CTX_INV;
int rc = 0;
u16 type;
+ if (BNXT_SUPPORTS_KTLS(bp)) {
+ ctxm = &ctx->ctx_arr[BNXT_CTX_TCK];
+ rc = bnxt_setup_ctxm_pg_tbls(bp, ctxm,
+ BNXT_TCK(crypto).max_ctx, 1);
+ if (rc)
+ return rc;
+ ctxm = &ctx->ctx_arr[BNXT_CTX_RCK];
+ rc = bnxt_setup_ctxm_pg_tbls(bp, ctxm,
+ BNXT_RCK(crypto).max_ctx, 1);
+ if (rc)
+ return rc;
+ last_type = BNXT_CTX_RCK;
+ }
+ if (mpc && mpc->mpc_chnls_cap) {
+ ctxm = &ctx->ctx_arr[BNXT_CTX_MTQM];
+ rc = bnxt_setup_ctxm_pg_tbls(bp, ctxm, ctxm->max_entries, 1);
+ if (rc)
+ return rc;
+ last_type = BNXT_CTX_MTQM;
+ }
+
for (type = BNXT_CTX_SRT; type <= BNXT_CTX_QPC; type++) {
ctxm = &ctx->ctx_arr[type];
if (!bnxt_bs_trace_avail(bp, type))
@@ -9685,6 +9803,7 @@ int bnxt_hwrm_func_resc_qcaps(struct bnxt *bp, bool all)
struct hwrm_func_resource_qcaps_output *resp;
struct hwrm_func_resource_qcaps_input *req;
struct bnxt_hw_resc *hw_resc = &bp->hw_resc;
+ struct bnxt_hw_crypto_resc *crypto_resc;
int rc;
rc = hwrm_req_init(bp, req, HWRM_FUNC_RESOURCE_QCAPS);
@@ -9722,6 +9841,12 @@ int bnxt_hwrm_func_resc_qcaps(struct bnxt *bp, bool all)
hw_resc->max_vnics * BNXT_LARGE_RSS_TO_VNIC_RATIO)
bp->rss_cap |= BNXT_RSS_CAP_LARGE_RSS_CTX;
+ crypto_resc = &hw_resc->crypto_resc;
+ crypto_resc->min_tx_key_ctxs = le32_to_cpu(resp->min_ktls_tx_key_ctxs);
+ crypto_resc->max_tx_key_ctxs = le32_to_cpu(resp->max_ktls_tx_key_ctxs);
+ crypto_resc->min_rx_key_ctxs = le32_to_cpu(resp->min_ktls_rx_key_ctxs);
+ crypto_resc->max_rx_key_ctxs = le32_to_cpu(resp->max_ktls_rx_key_ctxs);
+
if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
u16 max_msix = le16_to_cpu(resp->max_msix);
@@ -9871,6 +9996,10 @@ static int __bnxt_hwrm_func_qcaps(struct bnxt *bp)
bp->fw_cap |= BNXT_FW_CAP_BACKING_STORE_V2;
if (flags_ext & FUNC_QCAPS_RESP_FLAGS_EXT_TX_COAL_CMPL_CAP)
bp->flags |= BNXT_FLAG_TX_COAL_CMPL;
+ if (flags_ext & FUNC_QCAPS_RESP_FLAGS_EXT_KTLS_SUPPORTED)
+ bnxt_alloc_crypto_info(bp, resp);
+ else
+ bp->fw_cap &= ~BNXT_FW_CAP_KTLS;
flags_ext2 = le32_to_cpu(resp->flags_ext2);
if (flags_ext2 & FUNC_QCAPS_RESP_FLAGS_EXT2_RX_ALL_PKTS_TIMESTAMPS_SUPPORTED)
@@ -9943,6 +10072,11 @@ static int __bnxt_hwrm_func_qcaps(struct bnxt *bp)
}
bp->tso_max_segs = le16_to_cpu(resp->max_tso_segs);
+ if (resp->mpc_chnls_cap)
+ bnxt_alloc_mpc_info(bp, resp->mpc_chnls_cap);
+ else
+ bnxt_free_mpc_info(bp);
+
hwrm_func_qcaps_exit:
hwrm_req_drop(bp, req);
return rc;
@@ -11321,6 +11455,7 @@ static int bnxt_init_nic(struct bnxt *bp, bool irq_re_init)
bnxt_init_cp_rings(bp);
bnxt_init_rx_rings(bp);
bnxt_init_tx_rings(bp);
+ bnxt_init_mpc_rings(bp);
bnxt_init_ring_grps(bp, irq_re_init);
bnxt_init_vnics(bp);
@@ -11511,12 +11646,13 @@ unsigned int bnxt_get_max_func_cp_rings(struct bnxt *bp)
static unsigned int bnxt_get_max_func_cp_rings_for_en(struct bnxt *bp)
{
+ unsigned int mpc_cp = (unsigned int)bnxt_mpc_cp_rings_in_use(bp);
unsigned int cp = bp->hw_resc.max_cp_rings;
if (!(bp->flags & BNXT_FLAG_CHIP_P5_PLUS))
cp -= bnxt_get_ulp_msix_num(bp);
- return cp;
+ return mpc_cp >= cp ? 0 : cp - mpc_cp;
}
static unsigned int bnxt_get_max_func_irqs(struct bnxt *bp)
@@ -12801,6 +12937,8 @@ static int bnxt_hwrm_if_change(struct bnxt *bp, bool up)
bnxt_ulp_irq_stop(bp);
bnxt_free_ctx_mem(bp, false);
bnxt_dcb_free(bp);
+ if (fw_reset || caps_change)
+ bnxt_clear_crypto(bp);
rc = bnxt_fw_init_one(bp);
if (rc) {
clear_bit(BNXT_STATE_FW_RESET_DET, &bp->state);
@@ -13170,6 +13308,7 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)
return rc;
bnxt_adj_tx_rings(bp);
+ bnxt_trim_mpc_rings(bp);
rc = bnxt_alloc_mem(bp, irq_re_init);
if (rc) {
netdev_err(bp->dev, "bnxt_alloc_mem err: %x\n", rc);
@@ -13222,6 +13361,7 @@ static int __bnxt_open_nic(struct bnxt *bp, bool irq_re_init, bool link_re_init)
static_branch_disable(&bnxt_xdp_locking_key);
}
set_bit(BNXT_STATE_OPEN, &bp->state);
+ bnxt_ktls_wake(bp);
bnxt_enable_int(bp);
/* Enable TX queues */
bnxt_tx_enable(bp);
@@ -13363,7 +13503,8 @@ static int bnxt_open(struct net_device *dev)
static bool bnxt_drv_busy(struct bnxt *bp)
{
return (test_bit(BNXT_STATE_IN_SP_TASK, &bp->state) ||
- test_bit(BNXT_STATE_READ_STATS, &bp->state));
+ test_bit(BNXT_STATE_READ_STATS, &bp->state) ||
+ bnxt_ktls_busy(bp));
}
static void bnxt_get_ring_stats(struct bnxt *bp,
@@ -13381,9 +13522,20 @@ static void __bnxt_close_nic(struct bnxt *bp, bool irq_re_init,
clear_bit(BNXT_STATE_OPEN, &bp->state);
smp_mb__after_atomic();
+ /* Wake any kTLS delete waiting on a reconfig so it re-evaluates and
+ * either keeps waiting for the reopen or aborts (ifdown / FW reset).
+ */
+ bnxt_ktls_wake(bp);
while (bnxt_drv_busy(bp))
msleep(20);
+ /* Delete all crypto connections and KIDs only on ifdown and FW reset,
+ * not ethtool config changes.
+ */
+ if (!netif_running(bp->dev) ||
+ test_bit(BNXT_STATE_IN_FW_RESET, &bp->state))
+ bnxt_crypto_del_all(bp);
+
if (BNXT_SUPPORTS_MULTI_RSS_CTX(bp))
bnxt_clear_rss_ctxs(bp);
/* Flush rings and disable interrupts */
@@ -14320,7 +14472,7 @@ static int bnxt_hwrm_rx_ring_reset(struct bnxt *bp, int ring_nr)
return hwrm_req_send_silent(bp, req);
}
-static void bnxt_reset_task(struct bnxt *bp, bool silent)
+void bnxt_reset_task(struct bnxt *bp, bool silent)
{
if (!silent)
bnxt_dbg_dump_states(bp);
@@ -14537,6 +14689,7 @@ static void bnxt_fw_reset_close(struct bnxt *bp)
bnxt_hwrm_func_drv_unrgtr(bp);
if (pci_is_enabled(bp->pdev))
pci_disable_device(bp->pdev);
+ bnxt_clear_crypto(bp);
bnxt_free_ctx_mem(bp, false);
}
@@ -14918,8 +15071,10 @@ int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs,
hwr.grp = rx;
hwr.rss_ctx = bnxt_get_total_rss_ctxs(bp, &hwr);
}
- if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS)
- hwr.cp_p5 = hwr.tx + rx;
+ if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) {
+ hwr.cp_p5 = hwr.tx + rx + bnxt_mpc_cp_rings_in_use(bp);
+ hwr.tx += bnxt_mpc_tx_rings_in_use(bp);
+ }
rc = bnxt_hwrm_check_rings(bp, &hwr);
if (!rc && pci_msix_can_alloc_dyn(bp->pdev)) {
if (!bnxt_ulp_registered(bp->edev[BNXT_AUXDEV_RDMA])) {
@@ -16547,6 +16702,8 @@ static void bnxt_remove_one(struct pci_dev *pdev)
bp->ptp_cfg = NULL;
kfree(bp->fw_health);
bp->fw_health = NULL;
+ bnxt_free_crypto_info(bp);
+ bnxt_free_mpc_info(bp);
bnxt_cleanup_pci(bp);
bnxt_free_ctx_mem(bp, true);
bnxt_free_crash_dump_mem(bp);
@@ -16616,7 +16773,8 @@ static void _bnxt_get_max_rings(struct bnxt *bp, int *max_rx, int *max_tx,
struct bnxt_hw_resc *hw_resc = &bp->hw_resc;
int max_ring_grps = 0, max_irq;
- *max_tx = hw_resc->max_tx_rings;
+ *max_tx = max(0, (int)hw_resc->max_tx_rings -
+ bnxt_mpc_tx_rings_in_use(bp));
*max_rx = hw_resc->max_rx_rings;
*max_cp = bnxt_get_max_func_cp_rings_for_en(bp);
max_irq = min_t(int, bnxt_get_max_func_irqs(bp) -
@@ -16713,6 +16871,7 @@ static void bnxt_trim_dflt_sh_rings(struct bnxt *bp)
bp->rx_nr_rings = bp->cp_nr_rings;
bp->tx_nr_rings_per_tc = bp->cp_nr_rings;
bp->tx_nr_rings = bnxt_tx_nr_rings(bp);
+ bnxt_trim_mpc_rings(bp);
}
static void bnxt_adj_dflt_rings(struct bnxt *bp, bool sh)
@@ -16764,6 +16923,8 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)
bnxt_set_dflt_ulp_stat_ctxs(bp);
}
+ bnxt_set_dflt_mpc_rings(bp);
+
rc = __bnxt_reserve_rings(bp);
if (rc && rc != -ENODEV)
netdev_warn(bp->dev, "Unable to reserve tx rings\n");
@@ -16778,6 +16939,7 @@ static int bnxt_set_dflt_rings(struct bnxt *bp, bool sh)
if (rc && rc != -ENODEV)
netdev_warn(bp->dev, "2nd rings reservation failed.\n");
bnxt_adj_tx_rings(bp);
+ bnxt_trim_mpc_rings(bp);
}
if (BNXT_CHIP_TYPE_NITRO_A0(bp)) {
bp->rx_nr_rings++;
@@ -16812,6 +16974,7 @@ static int bnxt_init_dflt_ring_mode(struct bnxt *bp)
goto init_dflt_ring_err;
bnxt_adj_tx_rings(bp);
+ bnxt_trim_mpc_rings(bp);
bnxt_set_dflt_rfs(bp);
@@ -17155,6 +17318,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
* limited MSIX, so we re-initialize the TX rings per TC.
*/
bp->tx_nr_rings_per_tc = bp->tx_nr_rings;
+ bnxt_trim_mpc_rings(bp);
if (BNXT_PF(bp)) {
if (!bnxt_pf_wq) {
@@ -17176,6 +17340,12 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
rc = bnxt_dl_register(bp);
if (rc)
goto init_err_dl;
+ rc = bnxt_crypto_init(bp);
+ if (rc) {
+ bnxt_free_crypto_info(bp);
+ netdev_warn(bp->dev, "Failed to initialize crypto offload, err = %d\n",
+ rc);
+ }
INIT_LIST_HEAD(&bp->usr_fltr_list);
@@ -17218,6 +17388,8 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
bnxt_ethtool_free(bp);
kfree(bp->fw_health);
bp->fw_health = NULL;
+ bnxt_free_crypto_info(bp);
+ bnxt_free_mpc_info(bp);
bnxt_cleanup_pci(bp);
bnxt_free_ctx_mem(bp, true);
bnxt_free_crash_dump_mem(bp);
@@ -17400,6 +17572,7 @@ static pci_ers_result_t bnxt_io_error_detected(struct pci_dev *pdev,
if (pci_is_enabled(pdev))
pci_disable_device(pdev);
+ bnxt_clear_crypto(bp);
bnxt_free_ctx_mem(bp, false);
netdev_unlock(netdev);
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 6335dfc14c985b..19ffbb2cc6b120 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -47,6 +47,7 @@ struct tx_bd {
__le32 tx_bd_len_flags_type;
#define TX_BD_TYPE (0x3f << 0)
#define TX_BD_TYPE_SHORT_TX_BD (0x00 << 0)
+ #define TX_BD_TYPE_MPC_TX_BD (0x08 << 0)
#define TX_BD_TYPE_LONG_TX_BD (0x10 << 0)
#define TX_BD_FLAGS_PACKET_END (1 << 6)
#define TX_BD_FLAGS_NO_CMPL (1 << 7)
@@ -100,10 +101,18 @@ struct tx_bd_ext {
#define TX_BD_FLAGS_LSO (1 << 5)
#define TX_BD_FLAGS_IPID_FMT (1 << 6)
#define TX_BD_FLAGS_T_IPID (1 << 7)
+ #define TX_BD_FLAGS_CRYPTO_EN (1 << 15)
#define TX_BD_HSIZE (0xff << 16)
#define TX_BD_HSIZE_SHIFT 16
-
- __le32 tx_bd_mss;
+ #define TX_BD_KID_LO (0x7f << 25)
+ #define TX_BD_KID_LO_MASK 0x7f
+ #define TX_BD_KID_LO_SHIFT 25
+
+ __le32 tx_bd_kid_mss;
+ #define TX_BD_MSS 0x7fff
+ #define TX_BD_KID_HI (0x1ffff << 15)
+ #define TX_BD_KID_HI_MASK 0xffff80
+ #define TX_BD_KID_HI_SHIFT 8
__le32 tx_bd_cfa_action;
#define TX_BD_CFA_ACTION (0xffff << 16)
#define TX_BD_CFA_ACTION_SHIFT 16
@@ -121,6 +130,16 @@ struct tx_bd_ext {
};
#define BNXT_TX_PTP_IS_SET(lflags) ((lflags) & cpu_to_le32(TX_BD_FLAGS_STAMP))
+#define BNXT_TX_KID_LO(kid) (((kid) & TX_BD_KID_LO_MASK) << TX_BD_KID_LO_SHIFT)
+#define BNXT_TX_KID_HI(kid) (((kid) & TX_BD_KID_HI_MASK) << TX_BD_KID_HI_SHIFT)
+
+struct tx_bd_presync {
+ __le32 tx_bd_len_flags_type;
+ #define TX_BD_TYPE_PRESYNC_TX_BD (0x09 << 0)
+ u32 tx_bd_opaque;
+ __le32 tx_bd_kid;
+ u32 tx_bd_unused;
+};
struct rx_bd {
__le32 rx_bd_len_flags_type;
@@ -160,6 +179,7 @@ struct tx_cmp {
#define CMP_TYPE_RX_TPA_AGG_CMP 22
#define CMP_TYPE_RX_L2_V3_CMP 23
#define CMP_TYPE_RX_L2_TPA_START_V3_CMP 25
+ #define CMP_TYPE_MPC_CMP 30
#define CMP_TYPE_STATUS_CMP 32
#define CMP_TYPE_REMOTE_DRIVER_REQ 34
#define CMP_TYPE_REMOTE_DRIVER_RESP 36
@@ -686,6 +706,7 @@ struct nqe_cn {
#define BNXT_NQ_HDL_TYPE_SHIFT 24
#define BNXT_NQ_HDL_TYPE_RX 0x00
#define BNXT_NQ_HDL_TYPE_TX 0x01
+#define BNXT_NQ_HDL_TYPE_MP 0x02
#define BNXT_NQ_HDL_IDX(hdl) ((hdl) & BNXT_NQ_HDL_IDX_MASK)
#define BNXT_NQ_HDL_TYPE(hdl) (((hdl) & BNXT_NQ_HDL_TYPE_MASK) >> \
@@ -896,6 +917,7 @@ struct bnxt_sw_tx_bd {
u8 is_push;
u8 is_sw_gso;
u8 action;
+ u8 inline_data_bds;
unsigned short nr_frags;
union {
u16 rx_prod;
@@ -951,6 +973,8 @@ struct bnxt_ring_struct {
};
u32 handle;
u8 queue_id;
+#define BNXT_MPC_QUEUE_ID 0xff
+ u8 mpc_chnl_type;
};
struct tx_push_bd {
@@ -991,12 +1015,16 @@ struct bnxt_tx_ring_info {
u16 tx_cons;
u16 tx_hw_cons;
u16 txq_index;
+ /* index for tx_ring[] or tx_mpc_ring[] in struct bnxt_napi */
u8 tx_napi_idx;
u8 kick_pending;
struct bnxt_db_info tx_db;
struct tx_bd *tx_desc_ring[MAX_TX_PAGES];
- struct bnxt_sw_tx_bd *tx_buf_ring;
+ union {
+ struct bnxt_sw_tx_bd *tx_buf_ring;
+ struct bnxt_sw_mpc_tx_bd *tx_mpc_buf_ring;
+ };
dma_addr_t tx_desc_mapping[MAX_TX_PAGES];
@@ -1014,8 +1042,11 @@ struct bnxt_tx_ring_info {
u32 dev_state;
struct bnxt_ring_struct tx_ring_struct;
- /* Synchronize simultaneous xdp_xmit on same ring */
- spinlock_t xdp_tx_lock;
+ /* Synchronize simultaneous xdp_xmit on same ring or for MPC ring */
+ spinlock_t tx_lock;
+
+ /* Per-TX-ring kTLS counters; allocated only when kTLS is enabled. */
+ struct bnxt_tls_sw_stats *tls_stats;
};
#define BNXT_LEGACY_COAL_CMPL_PARAMS \
@@ -1157,6 +1188,22 @@ struct bnxt_cmn_sw_stats {
u64 missed_irqs;
};
+/* Data plane kTLS counters */
+enum bnxt_ktls_data_counters {
+ BNXT_KTLS_TX_PKTS = 0,
+ BNXT_KTLS_TX_BYTES,
+ BNXT_KTLS_TX_OOO_PKTS,
+ BNXT_KTLS_TX_OOO_FALLBACK_NO_SYNC,
+ BNXT_KTLS_TX_OOO_FALLBACK_NO_SPACE,
+ BNXT_KTLS_TX_OOO_FALLBACK_NO_HDR,
+
+ BNXT_KTLS_MAX_DATA_COUNTERS,
+};
+
+struct bnxt_tls_sw_stats {
+ u64 counters[BNXT_KTLS_MAX_DATA_COUNTERS];
+};
+
struct bnxt_sw_stats {
struct bnxt_rx_sw_stats rx;
struct bnxt_tx_sw_stats tx;
@@ -1242,6 +1289,7 @@ struct bnxt_napi {
struct bnxt_cp_ring_info cp_ring;
struct bnxt_rx_ring_info *rx_ring;
struct bnxt_tx_ring_info *tx_ring[BNXT_MAX_TXR_PER_NAPI];
+ struct bnxt_tx_ring_info **tx_mpc_ring;
void (*tx_int)(struct bnxt *, struct bnxt_napi *,
int budget);
@@ -1355,6 +1403,15 @@ struct bnxt_hw_rings {
int rss_ctx;
};
+struct bnxt_hw_crypto_resc {
+ u32 min_tx_key_ctxs;
+ u32 max_tx_key_ctxs;
+ u32 resv_tx_key_ctxs;
+ u32 min_rx_key_ctxs;
+ u32 max_rx_key_ctxs;
+ u32 resv_rx_key_ctxs;
+};
+
struct bnxt_hw_resc {
u16 min_rsscos_ctxs;
u16 max_rsscos_ctxs;
@@ -1389,6 +1446,8 @@ struct bnxt_hw_resc {
u32 max_tx_wm_flows;
u32 max_rx_em_flows;
u32 max_rx_wm_flows;
+
+ struct bnxt_hw_crypto_resc crypto_resc;
};
#define BNXT_LARGE_RSS_TO_VNIC_RATIO 7
@@ -2452,6 +2511,10 @@ struct bnxt {
u8 tph_mode;
+ struct bnxt_mpc_info *mpc_info;
+ struct bnxt_crypto_info *crypto_info;
+ struct bnxt_tls_info *ktls_info;
+
unsigned int current_interval;
#define BNXT_TIMER_INTERVAL HZ
@@ -2538,6 +2601,7 @@ struct bnxt {
#define BNXT_FW_CAP_NPAR_1_2 BIT_ULL(42)
#define BNXT_FW_CAP_MIRROR_ON_ROCE BIT_ULL(43)
#define BNXT_FW_CAP_PTP_PTM BIT_ULL(44)
+ #define BNXT_FW_CAP_KTLS BIT_ULL(45)
u32 fw_dbg_cap;
@@ -2563,6 +2627,8 @@ struct bnxt {
((bp)->fw_cap & BNXT_FW_CAP_SW_MAX_RESOURCE_LIMITS)
#define BNXT_MIRROR_ON_ROCE_CAP(bp) \
((bp)->fw_cap & BNXT_FW_CAP_MIRROR_ON_ROCE)
+#define BNXT_SUPPORTS_KTLS(bp) \
+ ((bp)->fw_cap & BNXT_FW_CAP_KTLS)
u32 hwrm_spec_code;
u16 hwrm_cmd_seq;
@@ -2856,14 +2922,14 @@ static inline u32 bnxt_tx_avail(struct bnxt *bp,
static inline struct tx_bd_ext *
bnxt_init_ext_bd(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
u16 prod, __le32 lflags, u32 vlan_tag_flags,
- u32 cfa_action)
+ u32 cfa_action, u32 kid)
{
struct tx_bd_ext *txbd1;
txbd1 = (struct tx_bd_ext *)
&txr->tx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)];
txbd1->tx_bd_hsize_lflags = lflags;
- txbd1->tx_bd_mss = 0;
+ txbd1->tx_bd_kid_mss = cpu_to_le32(BNXT_TX_KID_HI(kid));
txbd1->tx_bd_cfa_meta = cpu_to_le32(vlan_tag_flags);
txbd1->tx_bd_cfa_action =
cpu_to_le32(cfa_action << TX_BD_CFA_ACTION_SHIFT);
@@ -2962,6 +3028,8 @@ int bnxt_alloc_rx_data(struct bnxt *bp, struct bnxt_rx_ring_info *rxr,
void bnxt_reuse_rx_data(struct bnxt_rx_ring_info *rxr, u16 cons, void *data);
u32 bnxt_fw_health_readl(struct bnxt *bp, int reg_idx);
bool bnxt_bs_trace_avail(struct bnxt *bp, u16 type);
+void bnxt_free_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem);
+int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem);
void bnxt_set_tpa_flags(struct bnxt *bp);
void bnxt_set_ring_params(struct bnxt *);
void bnxt_set_rx_skb_mode(struct bnxt *bp, bool page_mode);
@@ -2990,6 +3058,14 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic);
int bnxt_hwrm_vnic_alloc(struct bnxt *bp, struct bnxt_vnic_info *vnic,
unsigned int start_rx_ring_idx,
unsigned int nr_rings);
+int bnxt_hwrm_cp_ring_alloc_p5(struct bnxt *bp, struct bnxt_cp_ring_info *cpr);
+int bnxt_hwrm_tx_ring_alloc(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ u32 tx_idx);
+void bnxt_hwrm_tx_ring_free(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ bool close_path);
+void bnxt_hwrm_cp_ring_free(struct bnxt *bp, struct bnxt_cp_ring_info *cpr);
+void bnxt_clear_one_cp_ring(struct bnxt *bp, struct bnxt_cp_ring_info *cpr);
+int bnxt_total_tx_rings(struct bnxt *bp);
int __bnxt_hwrm_get_tx_rings(struct bnxt *bp, u16 fid, int *tx_rings);
int bnxt_nq_rings_in_use(struct bnxt *bp);
int bnxt_hwrm_set_coal(struct bnxt *);
@@ -3035,6 +3111,7 @@ void bnxt_sync_ring_stats(struct bnxt *bp);
bool bnxt_rfs_capable(struct bnxt *bp, bool new_rss_ctx);
int bnxt_dbg_hwrm_rd_reg(struct bnxt *bp, u32 reg_off, u16 num_words,
u32 *reg_buf);
+void bnxt_reset_task(struct bnxt *bp, bool silent);
void bnxt_fw_exception(struct bnxt *bp);
void bnxt_fw_reset(struct bnxt *bp);
int bnxt_check_rings(struct bnxt *bp, int tx, int rx, bool sh, int tcs,
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c
new file mode 100644
index 00000000000000..920dda0d208684
--- /dev/null
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.c
@@ -0,0 +1,613 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (c) 2026 Broadcom Inc. */
+
+#include <linux/stddef.h>
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/pci.h>
+#include <linux/bnxt/hsi.h>
+
+#include "bnxt.h"
+#include "bnxt_hwrm.h"
+#include "bnxt_mpc.h"
+#include "bnxt_ktls.h"
+#include "bnxt_crypto.h"
+
+static u32 bnxt_get_max_crypto_key_ctx(struct bnxt *bp, int key_type)
+{
+ u32 fw_maj = BNXT_FW_MAJ(bp);
+
+ if (key_type == BNXT_TX_CRYPTO_KEY_TYPE)
+ return (fw_maj < 233) ? BNXT_MAX_TX_CRYPTO_KEYS_PRE_233FW :
+ BNXT_MAX_TX_CRYPTO_KEYS;
+
+ return (fw_maj < 233) ? BNXT_MAX_RX_CRYPTO_KEYS_PRE_233FW :
+ BNXT_MAX_RX_CRYPTO_KEYS;
+}
+
+/**
+ * bnxt_alloc_crypto_info - Allocate and initialize crypto offload context
+ * @bp: pointer to bnxt device
+ * @resp: pointer to firmware capability response
+ *
+ * Allocates the main crypto info structure
+ *
+ * This function is called during device initialization when firmware
+ * reports crypto offload capability. If allocation fails, crypto offload
+ * will not be available but the device will still function.
+ *
+ * Context: Process context
+ */
+void bnxt_alloc_crypto_info(struct bnxt *bp,
+ struct hwrm_func_qcaps_output *resp)
+{
+ u16 max_keys = le16_to_cpu(resp->max_key_ctxs_alloc);
+ struct bnxt_crypto_info *crypto = bp->crypto_info;
+ struct bnxt_kctx *kctx;
+ char name[64];
+ int i;
+
+ if (BNXT_VF(bp))
+ return;
+ if (!crypto) {
+ crypto = kzalloc_obj(*crypto);
+ if (!crypto) {
+ netdev_warn(bp->dev,
+ "Unable to allocate crypto info\n");
+ return;
+ }
+ snprintf(name, sizeof(name), "bnxt_crypto-%s",
+ dev_name(&bp->pdev->dev));
+ crypto->mpc_cache =
+ kmem_cache_create(name,
+ sizeof(struct bnxt_crypto_cmd_ctx),
+ 0, SLAB_HWCACHE_ALIGN, NULL);
+ if (!crypto->mpc_cache)
+ goto alloc_err;
+
+ for (i = 0; i < BNXT_MAX_CRYPTO_KEY_TYPE; i++) {
+ kctx = &crypto->kctx[i];
+ kctx->type = i;
+ INIT_LIST_HEAD(&kctx->list);
+ spin_lock_init(&kctx->lock);
+ atomic_set(&kctx->alloc_pending, 0);
+ init_waitqueue_head(&kctx->alloc_pending_wq);
+ }
+ bp->crypto_info = crypto;
+ }
+ for (i = 0; i < BNXT_MAX_CRYPTO_KEY_TYPE; i++) {
+ kctx = &crypto->kctx[i];
+ kctx->max_ctx = bnxt_get_max_crypto_key_ctx(bp, i);
+ }
+ crypto->max_key_ctxs_alloc = max_keys;
+ if (!bp->ktls_info)
+ bnxt_alloc_ktls_info(bp);
+ if (bp->ktls_info)
+ bp->fw_cap |= BNXT_FW_CAP_KTLS;
+ return;
+
+alloc_err:
+ kfree(crypto);
+}
+
+int bnxt_crypto_del(struct bnxt *bp, u8 type, u8 kind, u32 kid)
+{
+ struct bnxt_tx_ring_info *txr;
+ struct ce_delete_cmd cmd = {};
+ u32 data;
+
+ if (test_bit(BNXT_STATE_IN_FW_RESET, &bp->state) &&
+ test_bit(BNXT_STATE_FW_FATAL_COND, &bp->state))
+ return 0;
+
+ txr = bnxt_select_mpc_ring(bp, type);
+ if (!txr)
+ return -ENODEV;
+ if (kind == BNXT_CTX_KIND_CK_TX)
+ data = CE_DELETE_CMD_CTX_KIND_CK_TX;
+ else if (kind == BNXT_CTX_KIND_CK_RX)
+ data = CE_DELETE_CMD_CTX_KIND_CK_RX;
+ else
+ return -EINVAL;
+
+ data |= CE_DELETE_CMD_OPCODE_DEL |
+ (BNXT_KID_HW(kid) << CE_DELETE_CMD_KID_SFT);
+
+ cmd.ctx_kind_kid_opcode = cpu_to_le32(data);
+ return bnxt_xmit_crypto_cmd(bp, txr, &cmd, sizeof(cmd),
+ BNXT_MPC_TMO_MSECS);
+}
+
+static void bnxt_crypto_del_all_kids(struct bnxt *bp, struct bnxt_kid_info *kid)
+{
+ int i, rc;
+
+ for (i = 0; i < kid->count; i++) {
+ if (!test_bit(i, kid->ids)) {
+ rc = bnxt_crypto_del(bp, kid->type, kid->kind,
+ kid->start_id + i);
+ if (!rc)
+ set_bit(i, kid->ids);
+ }
+ }
+}
+
+/**
+ * bnxt_crypto_del_all - Delete all crypto connections
+ * @bp: pointer to bnxt device
+ *
+ * Delete all crypto connections and free all KIDs for re-use during
+ * shutdown. Increment the epoch counter to invalidate any outstanding
+ * key references.
+ *
+ * This function assumes serialization (called during shutdown) and does
+ * not use locking.
+ *
+ * Context: Process context during shutdown/reset
+ */
+void bnxt_crypto_del_all(struct bnxt *bp)
+{
+ struct bnxt_crypto_info *crypto = bp->crypto_info;
+ struct bnxt_kid_info *kid;
+ struct bnxt_kctx *kctx;
+ int i;
+
+ if (!crypto)
+ return;
+
+ /* Shutting down, no need to protect the lists. */
+ for (i = 0; i < BNXT_MAX_CRYPTO_KEY_TYPE; i++) {
+ kctx = &crypto->kctx[i];
+ list_for_each_entry(kid, &kctx->list, list)
+ bnxt_crypto_del_all_kids(bp, kid);
+ kctx->epoch = BNXT_NEXT_EPOCH(kctx->epoch);
+ }
+}
+
+/**
+ * bnxt_clear_crypto - Clear all crypto key contexts
+ * @bp: pointer to bnxt device
+ *
+ * Clears all key context allocations during shutdown or firmware reset.
+ * Frees all key info structures and bitmaps, and increments the epoch
+ * counter to invalidate any outstanding key references.
+ *
+ * This function assumes serialization (called during shutdown) and does
+ * not use locking.
+ *
+ * Context: Process context during shutdown/reset
+ */
+void bnxt_clear_crypto(struct bnxt *bp)
+{
+ struct bnxt_crypto_info *crypto = bp->crypto_info;
+ struct bnxt_kid_info *kid, *tmp;
+ struct bnxt_kctx *kctx;
+ int i;
+
+ if (!crypto)
+ return;
+
+ /* Only called when shutting down or FW reset with BNXT_STATE_OPEN
+ * cleared, so no concurrent access. No protection needed.
+ */
+ for (i = 0; i < BNXT_MAX_CRYPTO_KEY_TYPE; i++) {
+ kctx = &crypto->kctx[i];
+ list_for_each_entry_safe(kid, tmp, &kctx->list, list) {
+ list_del(&kid->list);
+ kfree(kid);
+ }
+ kctx->total_alloc = 0;
+ kctx->epoch = BNXT_NEXT_EPOCH(kctx->epoch);
+ }
+}
+
+/**
+ * bnxt_free_crypto_info - Free crypto offload resources
+ * @bp: pointer to bnxt device
+ *
+ * Frees all resources associated with crypto offload. Call this function
+ * only when it is idle with nothing in-flight.
+ *
+ * Context: Process context during device shutdown/removal
+ */
+void bnxt_free_crypto_info(struct bnxt *bp)
+{
+ struct bnxt_crypto_info *crypto = bp->crypto_info;
+
+ bnxt_free_ktls_info(bp);
+ if (!crypto)
+ return;
+ bnxt_clear_crypto(bp);
+ kmem_cache_destroy(crypto->mpc_cache);
+ kfree(crypto);
+ bp->crypto_info = NULL;
+ bp->fw_cap &= ~BNXT_FW_CAP_KTLS;
+}
+
+/**
+ * bnxt_hwrm_reserve_pf_key_ctxs - Reserve key contexts with firmware
+ * @bp: pointer to bnxt device
+ * @req: pointer to HWRM function config request
+ *
+ * Populates the firmware request with key context reservation parameters
+ * for crypto offload based on current max settings and capabilities.
+ *
+ * Context: Process context during device configuration
+ */
+void bnxt_hwrm_reserve_pf_key_ctxs(struct bnxt *bp,
+ struct hwrm_func_cfg_input *req)
+{
+ struct bnxt_crypto_info *crypto = bp->crypto_info;
+ struct bnxt_hw_resc *hw_resc = &bp->hw_resc;
+ struct bnxt_hw_crypto_resc *crypto_resc;
+ u32 tx, rx;
+
+ if (!crypto || !BNXT_SUPPORTS_KTLS(bp))
+ return;
+
+ crypto_resc = &hw_resc->crypto_resc;
+ tx = min(BNXT_TCK(crypto).max_ctx, crypto_resc->max_tx_key_ctxs);
+ rx = min(BNXT_RCK(crypto).max_ctx, crypto_resc->max_rx_key_ctxs);
+ req->num_ktls_tx_key_ctxs = cpu_to_le32(tx);
+ req->num_ktls_rx_key_ctxs = cpu_to_le32(rx);
+ if (tx)
+ req->enables |= cpu_to_le32(FUNC_CFG_REQ_ENABLES_KTLS_TX_KEY_CTXS);
+ if (rx)
+ req->enables |= cpu_to_le32(FUNC_CFG_REQ_ENABLES_KTLS_RX_KEY_CTXS);
+}
+
+static int bnxt_key_ctx_store(struct bnxt_kctx *kctx, __le32 *key_buf, u32 num,
+ bool contig, u8 kind, u32 *id)
+{
+ struct bnxt_kid_info *kid;
+ u32 i;
+
+ for (i = 0; i < num; ) {
+ kid = kzalloc_obj(*kid);
+ /* If we cannot store the IDs, they will be lost and only
+ * reclaimed by the FW during reset/reinit.
+ */
+ if (!kid)
+ return -ENOMEM;
+ kid->start_id = le32_to_cpu(key_buf[i]);
+ kid->type = kctx->type;
+ kid->kind = kind;
+ if (contig)
+ kid->count = num;
+ else
+ kid->count = 1;
+ bitmap_set(kid->ids, 0, kid->count);
+ if (id && !i) {
+ clear_bit(0, kid->ids);
+ *id = BNXT_SET_KID(kctx, kid->start_id);
+ }
+ spin_lock(&kctx->lock);
+ list_add_tail_rcu(&kid->list, &kctx->list);
+ WRITE_ONCE(kctx->total_alloc,
+ READ_ONCE(kctx->total_alloc) + kid->count);
+ spin_unlock(&kctx->lock);
+ i += kid->count;
+ }
+ return 0;
+}
+
+/* Note that the driver does not free the key contexts. They are freed
+ * by the FW during FLR and HWRM_FUNC_RESET.
+ */
+static int bnxt_hwrm_key_ctx_alloc(struct bnxt *bp, struct bnxt_kctx *kctx,
+ u8 kind, u32 num, u32 *id)
+{
+ struct bnxt_crypto_info *crypto = bp->crypto_info;
+ struct hwrm_func_key_ctx_alloc_output *resp;
+ struct hwrm_func_key_ctx_alloc_input *req;
+ dma_addr_t mapping;
+ int pending_count;
+ __le32 *key_buf;
+ u32 num_alloc;
+ bool contig;
+ int rc;
+
+ num = min3(num, crypto->max_key_ctxs_alloc, (u32)BNXT_KID_BATCH_SIZE);
+ rc = hwrm_req_init(bp, req, HWRM_FUNC_KEY_CTX_ALLOC);
+ if (rc)
+ return rc;
+
+ key_buf = hwrm_req_dma_slice(bp, req, num * 4, &mapping);
+ if (!key_buf) {
+ rc = -ENOMEM;
+ goto key_alloc_exit;
+ }
+ req->dma_bufr_size_bytes = cpu_to_le32(num * 4);
+ req->host_dma_addr = cpu_to_le64(mapping);
+ resp = hwrm_req_hold(bp, req);
+
+ req->key_ctx_type = kctx->type;
+ req->num_key_ctxs = cpu_to_le16(num);
+
+ pending_count = atomic_inc_return(&kctx->alloc_pending);
+ rc = hwrm_req_send(bp, req);
+ atomic_dec(&kctx->alloc_pending);
+ if (rc)
+ goto key_alloc_exit_wake;
+
+ num_alloc = le16_to_cpu(resp->num_key_ctxs_allocated);
+ if (num_alloc > num) {
+ netdev_warn(bp->dev,
+ "FW allocated more type %d keys (%d) than requested (%d)\n",
+ kctx->type, num_alloc, num);
+ } else if (!num_alloc) {
+ netdev_warn(bp->dev,
+ "FW allocated 0 type %d keys\n", kctx->type);
+ rc = -ENOENT;
+ goto key_alloc_exit_wake;
+ } else {
+ num = num_alloc;
+ }
+ contig = resp->flags &
+ FUNC_KEY_CTX_ALLOC_RESP_FLAGS_KEY_CTXS_CONTIGUOUS;
+ rc = bnxt_key_ctx_store(kctx, key_buf, num, contig, kind, id);
+
+key_alloc_exit_wake:
+ if (pending_count >= BNXT_KCTX_ALLOC_PENDING_MAX)
+ wake_up_all(&kctx->alloc_pending_wq);
+key_alloc_exit:
+ hwrm_req_drop(bp, req);
+ return rc;
+}
+
+bool bnxt_kid_valid(struct bnxt_kctx *kctx, u32 id)
+{
+ struct bnxt_kid_info *kid;
+ bool valid = false;
+ u32 epoch;
+
+ epoch = BNXT_KID_EPOCH(id);
+ if (epoch != kctx->epoch)
+ return false;
+
+ id = BNXT_KID_HW(id);
+ rcu_read_lock();
+ list_for_each_entry_rcu(kid, &kctx->list, list) {
+ if (id >= kid->start_id && id < kid->start_id + kid->count) {
+ if (!test_bit(id - kid->start_id, kid->ids)) {
+ valid = true;
+ break;
+ }
+ }
+ }
+ rcu_read_unlock();
+ return valid;
+}
+
+static int bnxt_alloc_one_kctx(struct bnxt_kctx *kctx, u8 kind, u32 *id)
+{
+ struct bnxt_kid_info *kid;
+ int rc = -ENOMEM;
+
+ rcu_read_lock();
+ list_for_each_entry_rcu(kid, &kctx->list, list) {
+ u32 idx = 0;
+
+ if (kid->kind != kind)
+ continue;
+ do {
+ idx = find_next_bit(kid->ids, kid->count, idx);
+ if (idx >= kid->count)
+ break;
+ if (test_and_clear_bit(idx, kid->ids)) {
+ *id = BNXT_SET_KID(kctx, kid->start_id + idx);
+ rc = 0;
+ goto alloc_done;
+ }
+ } while (1);
+ }
+
+alloc_done:
+ rcu_read_unlock();
+ return rc;
+}
+
+/**
+ * bnxt_free_one_kctx - Free a key context for later re-use
+ * @kctx: pointer to bnxt_kctx key context structure
+ * @id: Key context ID
+ *
+ * This function is called to free a key context ID when the offload
+ * using the ID has successfully terminated or aborted. If the offload
+ * cannot be terminated, the caller should not call this function to free
+ * the ID. The ID will only be recycled by the FW during reset/reinit.
+ */
+void bnxt_free_one_kctx(struct bnxt_kctx *kctx, u32 id)
+{
+ struct bnxt_kid_info *kid;
+
+ id = BNXT_KID_HW(id);
+ rcu_read_lock();
+ list_for_each_entry_rcu(kid, &kctx->list, list) {
+ if (id >= kid->start_id && id < kid->start_id + kid->count) {
+ set_bit(id - kid->start_id, kid->ids);
+ break;
+ }
+ }
+ rcu_read_unlock();
+}
+
+#define BNXT_KCTX_ALLOC_RETRY_MAX 3
+
+int bnxt_key_ctx_alloc_one(struct bnxt *bp, struct bnxt_kctx *kctx, u8 kind,
+ u32 *id)
+{
+ int rc, retry = 0;
+
+ while (retry++ < BNXT_KCTX_ALLOC_RETRY_MAX) {
+ rc = bnxt_alloc_one_kctx(kctx, kind, id);
+ if (!rc)
+ return 0;
+
+ /* When approaching the max, multiple threads may proceed
+ * and exceed the max. Some may fail the serialized HWRM call
+ * later when the max is exceeded.
+ */
+ if ((READ_ONCE(kctx->total_alloc) + BNXT_KID_BATCH_SIZE) >
+ kctx->max_ctx)
+ return -ENOSPC;
+
+ if (!BNXT_KCTX_ALLOC_OK(kctx)) {
+ wait_event(kctx->alloc_pending_wq,
+ BNXT_KCTX_ALLOC_OK(kctx));
+ continue;
+ }
+ rc = bnxt_hwrm_key_ctx_alloc(bp, kctx, kind,
+ BNXT_KID_BATCH_SIZE, id);
+ if (!rc)
+ return 0;
+ }
+ return -EAGAIN;
+}
+
+#define BNXT_XMIT_CRYPTO_RETRY_MAX 10
+#define BNXT_XMIT_CRYPTO_MIN_TMO 100
+#define BNXT_XMIT_CRYPTO_MAX_TMO 150
+
+int bnxt_xmit_crypto_cmd(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ void *cmd, unsigned int len, unsigned int tmo)
+{
+ struct bnxt_crypto_info *crypto = bp->crypto_info;
+ struct bnxt_crypto_cmd_ctx *ctx = NULL;
+ unsigned long tmo_left, handle = 0;
+ int rc, retry = 0;
+
+ if (tmo) {
+ u32 kid = CE_CMD_KID(cmd);
+
+ ctx = kmem_cache_alloc(crypto->mpc_cache, GFP_KERNEL);
+ if (!ctx)
+ return -ENOMEM;
+ init_completion(&ctx->cmp);
+ handle = (unsigned long)ctx;
+ ctx->kid = kid;
+ ctx->client = txr->tx_ring_struct.mpc_chnl_type;
+ ctx->status = 0;
+ /* One reference for this caller, one for the handle stored in
+ * the TX buf ring. The latter is dropped by
+ * bnxt_crypto_mpc_cmp() when the command is completed normally
+ * or after timeout.
+ */
+ refcount_set(&ctx->refcnt, 2);
+ retry = BNXT_XMIT_CRYPTO_RETRY_MAX;
+ might_sleep();
+ }
+ do {
+ spin_lock_bh(&txr->tx_lock);
+ rc = bnxt_start_xmit_mpc(bp, txr, cmd, len, handle);
+ spin_unlock_bh(&txr->tx_lock);
+ if (rc == -EBUSY && tmo && retry)
+ usleep_range(BNXT_XMIT_CRYPTO_MIN_TMO,
+ BNXT_XMIT_CRYPTO_MAX_TMO);
+ else
+ break;
+ } while (retry--);
+ if (rc || !tmo) {
+ /* The completion will never arrive, drop one reference */
+ if (ctx)
+ refcount_dec(&ctx->refcnt);
+ goto xmit_done;
+ }
+
+ tmo_left = wait_for_completion_timeout(&ctx->cmp, msecs_to_jiffies(tmo));
+ if (!tmo_left) {
+ netdev_warn(bp->dev, "crypto MP cmd %08x timed out\n",
+ *((u32 *)cmd));
+ bnxt_mpc_timeout(bp, txr);
+ rc = -ETIMEDOUT;
+ goto xmit_done;
+ }
+ if (ctx->status == BNXT_CMD_CTX_COMPLETED &&
+ CE_CMPL_STATUS(&ctx->ce_cmp) == CE_CMPL_STATUS_OK)
+ rc = 0;
+ else
+ rc = -EIO;
+xmit_done:
+ if (rc) {
+ u8 status = ctx ? ctx->status : 0;
+
+ netdev_warn(bp->dev,
+ "MPC transmit failed, ring idx %d, op 0x%x, kid 0x%x, status 0x%x\n",
+ txr->bnapi->index, CE_CMD_OP(cmd), CE_CMD_KID(cmd),
+ status);
+ }
+ if (ctx && refcount_dec_and_test(&ctx->refcnt))
+ kmem_cache_free(crypto->mpc_cache, ctx);
+ return rc;
+}
+
+int bnxt_crypto_init(struct bnxt *bp)
+{
+ struct bnxt_crypto_info *crypto = bp->crypto_info;
+ struct bnxt_hw_resc *hw_resc = &bp->hw_resc;
+ struct bnxt_hw_crypto_resc *crypto_resc;
+ int rc;
+
+ if (!crypto || !BNXT_SUPPORTS_KTLS(bp))
+ return 0;
+
+ crypto_resc = &hw_resc->crypto_resc;
+ BNXT_TCK(crypto).max_ctx = crypto_resc->resv_tx_key_ctxs;
+ BNXT_RCK(crypto).max_ctx = crypto_resc->resv_rx_key_ctxs;
+
+ if (!BNXT_TCK(crypto).max_ctx || !BNXT_RCK(crypto).max_ctx)
+ return -ENODEV;
+
+ rc = bnxt_hwrm_key_ctx_alloc(bp, &BNXT_TCK(crypto), BNXT_CTX_KIND_CK_TX,
+ BNXT_KID_BATCH_SIZE, NULL);
+ if (rc)
+ return rc;
+
+ rc = bnxt_hwrm_key_ctx_alloc(bp, &BNXT_RCK(crypto), BNXT_CTX_KIND_CK_RX,
+ BNXT_KID_BATCH_SIZE, NULL);
+ if (rc)
+ return rc;
+
+ bnxt_ktls_init(bp);
+ return 0;
+}
+
+void bnxt_crypto_mpc_cmp(struct bnxt *bp, u32 client, unsigned long handle,
+ struct bnxt_cmpl_entry cmpl[], u32 entries)
+{
+ struct bnxt_crypto_cmd_ctx *ctx;
+ struct ce_cmpl *cmp = NULL;
+ u32 len, kid;
+
+ if (likely(cmpl))
+ cmp = cmpl[0].cmpl;
+ if (!handle || entries != 1) {
+ if (entries != 1 && cmpl) {
+ netdev_warn(bp->dev, "Invalid entries %d with handle %lx cmpl %08x in %s()\n",
+ entries, handle, *(u32 *)cmp, __func__);
+ }
+ if (!handle)
+ return;
+ }
+ ctx = (void *)handle;
+ ctx->status = BNXT_CMD_CTX_COMPLETED;
+ if (unlikely(!cmpl)) {
+ ctx->status |= BNXT_CMD_CTX_RESET;
+ goto cmp_done;
+ }
+ kid = CE_CMPL_KID(cmp);
+ if (ctx->kid != kid || ctx->client != client || entries != 1) {
+ netdev_warn(bp->dev,
+ "Invalid CE cmpl 0x%08x with entries %d for client %d with status 0x%x, expected kid 0x%x and client %d\n",
+ *(u32 *)cmp, entries, client, ctx->status, ctx->kid,
+ ctx->client);
+ ctx->status |= BNXT_CMD_CTX_ERROR;
+ }
+ len = min_t(u32, cmpl[0].len, sizeof(ctx->ce_cmp));
+ memcpy(&ctx->ce_cmp, cmpl[0].cmpl, len);
+cmp_done:
+ complete(&ctx->cmp);
+ if (refcount_dec_and_test(&ctx->refcnt))
+ kmem_cache_free(bp->crypto_info->mpc_cache, ctx);
+}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h
new file mode 100644
index 00000000000000..a4571369fd23ca
--- /dev/null
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_crypto.h
@@ -0,0 +1,229 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (c) 2026 Broadcom Inc. */
+
+#ifndef BNXT_CRYPTO_H
+#define BNXT_CRYPTO_H
+
+#include <linux/refcount.h>
+
+#define BNXT_MAX_TX_CRYPTO_KEYS 204800
+#define BNXT_MAX_RX_CRYPTO_KEYS 204800
+
+#define BNXT_MAX_TX_CRYPTO_KEYS_PRE_233FW 65535
+#define BNXT_MAX_RX_CRYPTO_KEYS_PRE_233FW 65535
+
+enum bnxt_crypto_type {
+ BNXT_TX_CRYPTO_KEY_TYPE = FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_TX,
+ BNXT_RX_CRYPTO_KEY_TYPE = FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_RX,
+ BNXT_MAX_CRYPTO_KEY_TYPE,
+};
+
+#define BNXT_KID_BATCH_SIZE 128
+
+struct bnxt_kid_info {
+ struct list_head list;
+ u8 type;
+ u8 kind;
+ u32 start_id;
+ u32 count;
+ DECLARE_BITMAP(ids, BNXT_KID_BATCH_SIZE);
+};
+
+struct bnxt_kctx {
+ struct list_head list;
+ /* to serialize update to the linked list and total_alloc */
+ spinlock_t lock;
+ u8 type;
+ u16 epoch;
+ u32 total_alloc;
+ u32 max_ctx;
+ atomic_t alloc_pending;
+#define BNXT_KCTX_ALLOC_PENDING_MAX 8
+ wait_queue_head_t alloc_pending_wq;
+};
+
+#define BNXT_KID_HW_MASK 0x000fffff
+#define BNXT_KID_HW(kid) ((kid) & BNXT_KID_HW_MASK)
+#define BNXT_KID_EPOCH_MASK 0xfff00000
+#define BNXT_KID_EPOCH_SHIFT 20
+#define BNXT_KID_EPOCH(kid) (((kid) & BNXT_KID_EPOCH_MASK) >> \
+ BNXT_KID_EPOCH_SHIFT)
+
+#define BNXT_NEXT_EPOCH(epoch) \
+ (((epoch) + 1) & (BNXT_KID_EPOCH_MASK >> BNXT_KID_EPOCH_SHIFT))
+
+#define BNXT_SET_KID(kctx, kid) \
+ ((kid) | ((u32)(kctx)->epoch << BNXT_KID_EPOCH_SHIFT))
+
+#define BNXT_KCTX_ALLOC_OK(kctx) \
+ (atomic_read(&((kctx)->alloc_pending)) < BNXT_KCTX_ALLOC_PENDING_MAX)
+
+struct bnxt_crypto_info {
+ u16 max_key_ctxs_alloc;
+
+ struct bnxt_kctx kctx[BNXT_MAX_CRYPTO_KEY_TYPE];
+
+ struct kmem_cache *mpc_cache;
+};
+
+struct ce_delete_cmd {
+ __le32 ctx_kind_kid_opcode;
+ #define CE_DELETE_CMD_OPCODE_MASK 0xfUL
+ #define CE_DELETE_CMD_OPCODE_SFT 0
+ #define CE_DELETE_CMD_OPCODE_DEL 0x2UL
+ #define CE_DELETE_CMD_KID_MASK 0xfffff0UL
+ #define CE_DELETE_CMD_KID_SFT 4
+ #define CE_DELETE_CMD_CTX_KIND_MASK 0x1f000000UL
+ #define CE_DELETE_CMD_CTX_KIND_SFT 24
+ #define CE_DELETE_CMD_CTX_KIND_CK_TX (0x11UL << 24)
+ #define CE_DELETE_CMD_CTX_KIND_CK_RX (0x12UL << 24)
+};
+
+#define CE_CMD_OP_MASK 0x00000fU
+#define CE_CMD_KID_MASK 0xfffff0U
+#define CE_CMD_KID_SFT 4
+
+#define CE_CMD_OP(cmd_p) \
+ (le32_to_cpu(*(__le32 *)(cmd_p)) & CE_CMD_OP_MASK)
+
+#define CE_CMD_KID(cmd_p) \
+ ((le32_to_cpu(*(__le32 *)(cmd_p)) & CE_CMD_KID_MASK) >> CE_CMD_KID_SFT)
+
+struct ce_cmpl {
+ __le16 client_subtype_type;
+ #define CE_CMPL_TYPE_MASK 0x3fUL
+ #define CE_CMPL_TYPE_SFT 0
+ #define CE_CMPL_TYPE_MID_PATH_SHORT 0x1eUL
+ #define CE_CMPL_SUBTYPE_MASK 0xf00UL
+ #define CE_CMPL_SUBTYPE_SFT 8
+ #define CE_CMPL_SUBTYPE_SOLICITED (0x0UL << 8)
+ #define CE_CMPL_SUBTYPE_ERR (0x1UL << 8)
+ #define CE_CMPL_SUBTYPE_RESYNC (0x2UL << 8)
+ #define CE_CMPL_MP_CLIENT_MASK 0xf000UL
+ #define CE_CMPL_MP_CLIENT_SFT 12
+ #define CE_CMPL_MP_CLIENT_TCE (0x0UL << 12)
+ #define CE_CMPL_MP_CLIENT_RCE (0x1UL << 12)
+ __le16 status;
+ #define CE_CMPL_STATUS_MASK 0xfUL
+ #define CE_CMPL_STATUS_SFT 0
+ #define CE_CMPL_STATUS_OK 0x0UL
+ #define CE_CMPL_STATUS_CTX_LD_ERR 0x1UL
+ #define CE_CMPL_STATUS_FID_CHK_ERR 0x2UL
+ #define CE_CMPL_STATUS_CTX_VER_ERR 0x3UL
+ #define CE_CMPL_STATUS_DST_ID_ERR 0x4UL
+ #define CE_CMPL_STATUS_MP_CMD_ERR 0x5UL
+ u32 opaque;
+ __le32 v;
+ #define CE_CMPL_V 0x1UL
+ __le32 kid;
+ #define CE_CMPL_KID_MASK 0xfffffUL
+ #define CE_CMPL_KID_SFT 0
+};
+
+#define CE_CMPL_STATUS(ce_cmpl) \
+ (le16_to_cpu((ce_cmpl)->status) & CE_CMPL_STATUS_MASK)
+
+#define CE_CMPL_KID(ce_cmpl) \
+ (le32_to_cpu((ce_cmpl)->kid) & CE_CMPL_KID_MASK)
+
+struct bnxt_crypto_cmd_ctx {
+ struct completion cmp;
+ struct ce_cmpl ce_cmp;
+ refcount_t refcnt;
+ u32 kid;
+ u16 client;
+ u8 status;
+#define BNXT_CMD_CTX_COMPLETED 0x1
+#define BNXT_CMD_CTX_ERROR 0x2
+#define BNXT_CMD_CTX_RESET 0x4
+};
+
+#define BNXT_TCK(crypto) ((crypto)->kctx[BNXT_TX_CRYPTO_KEY_TYPE])
+#define BNXT_RCK(crypto) ((crypto)->kctx[BNXT_RX_CRYPTO_KEY_TYPE])
+
+#define BNXT_CTX_KIND_CK_TX 0x11
+#define BNXT_CTX_KIND_CK_RX 0x12
+
+#ifdef CONFIG_BNXT_TLS
+void bnxt_alloc_crypto_info(struct bnxt *bp,
+ struct hwrm_func_qcaps_output *resp);
+int bnxt_crypto_del(struct bnxt *bp, u8 type, u8 kind, u32 kid);
+void bnxt_crypto_del_all(struct bnxt *bp);
+void bnxt_clear_crypto(struct bnxt *bp);
+void bnxt_free_crypto_info(struct bnxt *bp);
+void bnxt_hwrm_reserve_pf_key_ctxs(struct bnxt *bp,
+ struct hwrm_func_cfg_input *req);
+bool bnxt_kid_valid(struct bnxt_kctx *kctx, u32 id);
+void bnxt_free_one_kctx(struct bnxt_kctx *kctx, u32 id);
+int bnxt_key_ctx_alloc_one(struct bnxt *bp, struct bnxt_kctx *kctx, u8 kind,
+ u32 *id);
+int bnxt_xmit_crypto_cmd(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ void *cmd, unsigned int len, unsigned int tmo);
+int bnxt_crypto_init(struct bnxt *bp);
+void bnxt_crypto_mpc_cmp(struct bnxt *bp, u32 client, unsigned long handle,
+ struct bnxt_cmpl_entry cmpl[], u32 entries);
+#else
+static inline void bnxt_alloc_crypto_info(struct bnxt *bp,
+ struct hwrm_func_qcaps_output *resp)
+{
+}
+
+static inline int bnxt_crypto_del(struct bnxt *bp, u8 type, u8 kind, u32 kid)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline void bnxt_crypto_del_all(struct bnxt *bp)
+{
+}
+
+static inline void bnxt_clear_crypto(struct bnxt *bp)
+{
+}
+
+static inline void bnxt_free_crypto_info(struct bnxt *bp)
+{
+}
+
+static inline void bnxt_hwrm_reserve_pf_key_ctxs(struct bnxt *bp,
+ struct hwrm_func_cfg_input *req)
+{
+}
+
+static inline bool bnxt_kid_valid(struct bnxt_kctx *kctx, u32 id)
+{
+ return false;
+}
+
+static inline void bnxt_free_one_kctx(struct bnxt_kctx *kctx, u32 id)
+{
+}
+
+static inline int bnxt_key_ctx_alloc_one(struct bnxt *bp,
+ struct bnxt_kctx *kctx, u8 kind,
+ u32 *id)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline int bnxt_xmit_crypto_cmd(struct bnxt *bp,
+ struct bnxt_tx_ring_info *txr,
+ void *cmd, unsigned int len,
+ unsigned int tmo)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline int bnxt_crypto_init(struct bnxt *bp)
+{
+ return 0;
+}
+
+static inline void bnxt_crypto_mpc_cmp(struct bnxt *bp, u32 client,
+ unsigned long handle,
+ struct bnxt_cmpl_entry cmpl[],
+ u32 entries)
+{
+}
+#endif /* CONFIG_BNXT_TLS */
+#endif /* BNXT_CRYPTO_H */
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index 62bc9cae613c38..96083caa834c4d 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -37,6 +37,8 @@
#include "bnxt_nvm_defs.h" /* NVRAM content constant and structure defs */
#include "bnxt_fw_hdr.h" /* Firmware hdr constant and structure defs */
#include "bnxt_coredump.h"
+#include "bnxt_mpc.h"
+#include "bnxt_ktls.h"
#define BNXT_NVM_ERR_MSG(dev, extack, msg) \
do { \
@@ -353,6 +355,30 @@ static const char *const bnxt_ring_drv_stats_arr[] = {
"total_missed_irqs",
};
+/* kTLS data plane counter strings indexed by enum bnxt_ktls_data_counters */
+static const char *const bnxt_ktls_data_stats[] = {
+ [BNXT_KTLS_TX_PKTS] = "tx_tls_encrypted_packets",
+ [BNXT_KTLS_TX_BYTES] = "tx_tls_encrypted_bytes",
+ [BNXT_KTLS_TX_OOO_PKTS] = "tx_tls_ooo_packets",
+ [BNXT_KTLS_TX_OOO_FALLBACK_NO_SYNC] = "tx_tls_ooo_fallback_no_sync",
+ [BNXT_KTLS_TX_OOO_FALLBACK_NO_SPACE] = "tx_tls_ooo_fallback_no_space",
+ [BNXT_KTLS_TX_OOO_FALLBACK_NO_HDR] = "tx_tls_ooo_fallback_no_hdr",
+};
+
+/* kTLS control plane counter strings indexed by enum bnxt_ktls_ctrl_counters */
+static const char *const bnxt_ktls_ctrl_stats[] = {
+ [BNXT_KTLS_TX_ADD] = "tx_tls_ctx",
+ [BNXT_KTLS_TX_DEL] = "tx_tls_del",
+ [BNXT_KTLS_ERR_NO_MEM] = "tls_err_no_mem",
+ [BNXT_KTLS_ERR_NO_CAP] = "tls_err_no_cap",
+ [BNXT_KTLS_ERR_KEY_CTX_ALLOC] = "tls_err_key_ctx_alloc",
+ [BNXT_KTLS_ERR_CRYPTO_CMD] = "tls_err_crypto_cmd",
+ [BNXT_KTLS_ERR_DEVICE_BUSY] = "tls_err_device_busy",
+ [BNXT_KTLS_ERR_INVALID_CIPHER] = "tls_err_invalid_cipher",
+ [BNXT_KTLS_ERR_STATE_NOT_OPEN] = "tls_err_state_not_open",
+ [BNXT_KTLS_ERR_RETRY_EXCEEDED] = "tls_err_retry_exceeded",
+};
+
#define NUM_RING_RX_SW_STATS ARRAY_SIZE(bnxt_rx_sw_stats_str)
#define NUM_RING_CMN_SW_STATS ARRAY_SIZE(bnxt_cmn_sw_stats_str)
#define NUM_RING_RX_HW_STATS ARRAY_SIZE(bnxt_ring_rx_stats_str)
@@ -535,12 +561,21 @@ static int bnxt_get_num_ring_stats(struct bnxt *bp)
cmn * bp->cp_nr_rings;
}
+static int bnxt_get_num_ktls_stats(struct bnxt *bp)
+{
+ if (!bp->ktls_info)
+ return 0;
+ return ARRAY_SIZE(bnxt_ktls_ctrl_stats) +
+ ARRAY_SIZE(bnxt_ktls_data_stats);
+}
+
static int bnxt_get_num_stats(struct bnxt *bp)
{
int num_stats = bnxt_get_num_ring_stats(bp);
int len;
num_stats += BNXT_NUM_RING_DRV_STATS;
+ num_stats += bnxt_get_num_ktls_stats(bp);
if (bp->flags & BNXT_FLAG_PORT_STATS)
num_stats += BNXT_NUM_PORT_STATS;
@@ -653,6 +688,16 @@ static void bnxt_get_ethtool_stats(struct net_device *dev,
for (i = 0; i < BNXT_NUM_RING_DRV_STATS; i++, j++, curr++, prev++)
buf[j] = *curr + *prev;
+ if (bp->ktls_info) {
+ struct bnxt_tls_info *ktls = bp->ktls_info;
+ struct bnxt_tls_sw_stats tls_stats = {};
+
+ bnxt_get_ring_tls_stats(bp, &tls_stats);
+ for (i = 0; i < ARRAY_SIZE(bnxt_ktls_data_stats); i++, j++)
+ buf[j] = tls_stats.counters[i];
+ for (i = 0; i < ARRAY_SIZE(bnxt_ktls_ctrl_stats); i++, j++)
+ buf[j] = atomic64_read(&ktls->counters[i]);
+ }
if (bp->flags & BNXT_FLAG_PORT_STATS) {
u64 *port_stats = bp->port_stats.sw_stats;
@@ -763,6 +808,12 @@ static void bnxt_get_strings(struct net_device *dev, u32 stringset, u8 *buf)
for (i = 0; i < BNXT_NUM_RING_DRV_STATS; i++)
ethtool_puts(&buf, bnxt_ring_drv_stats_arr[i]);
+ if (bp->ktls_info) {
+ for (i = 0; i < ARRAY_SIZE(bnxt_ktls_data_stats); i++)
+ ethtool_puts(&buf, bnxt_ktls_data_stats[i]);
+ for (i = 0; i < ARRAY_SIZE(bnxt_ktls_ctrl_stats); i++)
+ ethtool_puts(&buf, bnxt_ktls_ctrl_stats[i]);
+ }
if (bp->flags & BNXT_FLAG_PORT_STATS)
for (i = 0; i < BNXT_NUM_PORT_STATS; i++) {
str = bnxt_port_stats_arr[i].string;
@@ -1051,6 +1102,8 @@ static int bnxt_set_channels(struct net_device *dev,
bnxt_set_cp_rings(bp, sh);
+ bnxt_set_dflt_mpc_rings(bp);
+
/* After changing number of rx channels, update NTUPLE feature. */
netdev_update_features(dev);
if (netif_running(dev)) {
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c
index f317f60414e813..b4c37a6c9f0fa9 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c
@@ -168,7 +168,7 @@ netdev_tx_t bnxt_sw_udp_gso_xmit(struct bnxt *bp,
prod = NEXT_TX(prod);
bnxt_init_ext_bd(bp, txr, prod, csum,
- vlan_tag_flags, cfa_action);
+ vlan_tag_flags, cfa_action, 0);
/* set dma_unmap_len on the LAST BD touching each
* region. Since completions are in-order, the last segment
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c
new file mode 100644
index 00000000000000..b47075f2b379ee
--- /dev/null
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.c
@@ -0,0 +1,572 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (c) 2026 Broadcom Inc. */
+
+#include <linux/tcp.h>
+#include <net/tls.h>
+#include <linux/bnxt/hsi.h>
+
+#include "bnxt.h"
+#include "bnxt_mpc.h"
+#include "bnxt_crypto.h"
+#include "bnxt_ktls.h"
+
+/**
+ * bnxt_alloc_ktls_info - Allocate and initialize kTLS offload context
+ * @bp: pointer to bnxt device
+ *
+ * Allocates the main kTLS crypto info structure
+ *
+ * This function is called during device initialization when firmware
+ * reports kTLS offload capability. If allocation fails, kTLS offload
+ * will not be available but the device will still function.
+ *
+ * Context: Process context
+ *
+ * Return: zero on success, negative error code otherwise:
+ * ENOMEM: out of memory
+ */
+int bnxt_alloc_ktls_info(struct bnxt *bp)
+{
+ struct bnxt_tls_info *ktls = bp->ktls_info;
+
+ if (BNXT_VF(bp))
+ return -EOPNOTSUPP;
+ if (ktls)
+ return 0;
+
+ ktls = kzalloc_obj(*ktls);
+ if (!ktls) {
+ netdev_warn(bp->dev, "Unable to allocate kTLS info\n");
+ return -ENOMEM;
+ }
+ ktls->counters = kzalloc_objs(*ktls->counters,
+ BNXT_KTLS_MAX_CTRL_COUNTERS);
+ if (!ktls->counters)
+ goto ktls_err;
+
+ init_waitqueue_head(&ktls->open_wq);
+ bp->ktls_info = ktls;
+ return 0;
+
+ktls_err:
+ kfree(ktls->counters);
+ kfree(ktls);
+ return -ENOMEM;
+}
+
+/**
+ * bnxt_free_ktls_info - Free kTLS crypto offload resources
+ * @bp: pointer to bnxt device
+ *
+ * Frees all resources associated with kTLS crypto offload
+ *
+ * Context: Process context during device shutdown/removal
+ */
+void bnxt_free_ktls_info(struct bnxt *bp)
+{
+ struct bnxt_tls_info *ktls = bp->ktls_info;
+
+ if (!ktls)
+ return;
+ kfree(ktls->counters);
+ kfree(ktls);
+ bp->ktls_info = NULL;
+}
+
+/* Copy in reverse byte order */
+static void bnxt_copy_tls_mp_data(u8 *dst, u8 *src, int bytes)
+{
+ int i;
+
+ for (i = 0; i < bytes; i++)
+ dst[bytes - i - 1] = src[i];
+}
+
+static int bnxt_crypto_add(struct bnxt *bp, enum tls_offload_ctx_dir direction,
+ struct tls_crypto_info *crypto_info, u32 tcp_seq_no,
+ u32 kid)
+{
+ struct bnxt_tx_ring_info *txr;
+ struct ce_add_cmd cmd = {0};
+ u32 data;
+ int rc;
+
+ if (direction == TLS_OFFLOAD_CTX_DIR_TX) {
+ txr = bnxt_select_mpc_ring(bp, BNXT_MPC_TCE_TYPE);
+ cmd.ctx_kind = CE_ADD_CMD_CTX_KIND_CK_TX;
+ } else {
+ return -EOPNOTSUPP;
+ }
+ if (!txr)
+ return -ENODEV;
+
+ data = CE_ADD_CMD_OPCODE_ADD | (BNXT_KID_HW(kid) << CE_ADD_CMD_KID_SFT);
+ switch (crypto_info->cipher_type) {
+ case TLS_CIPHER_AES_GCM_128: {
+ struct tls12_crypto_info_aes_gcm_128 *aes;
+
+ aes = (void *)crypto_info;
+ data |= CE_ADD_CMD_ALGORITHM_AES_GCM_128;
+ if (crypto_info->version == TLS_1_3_VERSION)
+ data |= CE_ADD_CMD_VERSION_TLS1_3;
+ memcpy(&cmd.session_key, aes->key, sizeof(aes->key));
+ memcpy(&cmd.salt, aes->salt, sizeof(aes->salt));
+ memcpy(&cmd.addl_iv, aes->iv, sizeof(aes->iv));
+ bnxt_copy_tls_mp_data(cmd.record_seq_num, aes->rec_seq,
+ sizeof(aes->rec_seq));
+ break;
+ }
+ case TLS_CIPHER_AES_GCM_256: {
+ struct tls12_crypto_info_aes_gcm_256 *aes;
+
+ aes = (void *)crypto_info;
+ data |= CE_ADD_CMD_ALGORITHM_AES_GCM_256;
+ if (crypto_info->version == TLS_1_3_VERSION)
+ data |= CE_ADD_CMD_VERSION_TLS1_3;
+ memcpy(&cmd.session_key, aes->key, sizeof(aes->key));
+ memcpy(&cmd.salt, aes->salt, sizeof(aes->salt));
+ memcpy(&cmd.addl_iv, aes->iv, sizeof(aes->iv));
+ bnxt_copy_tls_mp_data(cmd.record_seq_num, aes->rec_seq,
+ sizeof(aes->rec_seq));
+ break;
+ }
+ default:
+ return -EOPNOTSUPP;
+ }
+ cmd.ver_algo_kid_opcode = cpu_to_le32(data);
+ cmd.pkt_tcp_seq_num = cpu_to_le32(tcp_seq_no);
+ cmd.tls_header_tcp_seq_num = cmd.pkt_tcp_seq_num;
+ rc = bnxt_xmit_crypto_cmd(bp, txr, &cmd, sizeof(cmd),
+ BNXT_MPC_TMO_MSECS);
+ memzero_explicit(&cmd, sizeof(cmd));
+ return rc;
+}
+
+static bool bnxt_ktls_cipher_supported(struct bnxt *bp,
+ struct tls_crypto_info *crypto_info)
+{
+ u16 type = crypto_info->cipher_type;
+ u16 version = crypto_info->version;
+
+ if ((type == TLS_CIPHER_AES_GCM_128 ||
+ type == TLS_CIPHER_AES_GCM_256) &&
+ (version == TLS_1_2_VERSION ||
+ version == TLS_1_3_VERSION))
+ return true;
+ return false;
+}
+
+static void bnxt_set_ktls_ctx_tx(struct tls_context *tls_ctx,
+ struct bnxt_ktls_offload_ctx_tx *kctx_tx)
+{
+ struct bnxt_ktls_tx_driver_state *tx =
+ __tls_driver_ctx(tls_ctx, TLS_OFFLOAD_CTX_DIR_TX);
+
+ tx->ctx_tx = kctx_tx;
+}
+
+static struct bnxt_ktls_offload_ctx_tx *
+bnxt_get_ktls_ctx_tx(struct tls_context *tls_ctx)
+{
+ struct bnxt_ktls_tx_driver_state *tx =
+ __tls_driver_ctx(tls_ctx, TLS_OFFLOAD_CTX_DIR_TX);
+
+ return tx->ctx_tx;
+}
+
+static int bnxt_ktls_dev_add(struct net_device *dev, struct sock *sk,
+ enum tls_offload_ctx_dir direction,
+ struct tls_crypto_info *crypto_info,
+ u32 start_offload_tcp_sn)
+{
+ struct bnxt_ktls_offload_ctx_tx *kctx_tx;
+ struct bnxt *bp = netdev_priv(dev);
+ struct bnxt_crypto_info *crypto;
+ struct tls_context *tls_ctx;
+ struct bnxt_tls_info *ktls;
+ struct bnxt_kctx *kctx;
+ u32 kid;
+ int rc;
+
+ BUILD_BUG_ON(sizeof(struct bnxt_ktls_tx_driver_state) >
+ TLS_DRIVER_STATE_SIZE_TX);
+
+ ktls = bp->ktls_info;
+ if (direction == TLS_OFFLOAD_CTX_DIR_RX)
+ return -EOPNOTSUPP;
+
+ if (!BNXT_SUPPORTS_KTLS(bp)) {
+ atomic64_inc(&ktls->counters[BNXT_KTLS_ERR_NO_CAP]);
+ return -EOPNOTSUPP;
+ }
+ atomic_inc(&ktls->pending);
+ /* Make sure bnxt_close_nic() sees pending before we check the
+ * BNXT_STATE_OPEN flag.
+ */
+ smp_mb__after_atomic();
+ if (!test_bit(BNXT_STATE_OPEN, &bp->state)) {
+ atomic64_inc(&ktls->counters[BNXT_KTLS_ERR_STATE_NOT_OPEN]);
+ rc = -ENODEV;
+ goto exit;
+ }
+
+ if (!bnxt_ktls_cipher_supported(bp, crypto_info)) {
+ atomic64_inc(&ktls->counters[BNXT_KTLS_ERR_INVALID_CIPHER]);
+ rc = -EOPNOTSUPP;
+ goto exit;
+ }
+
+ kctx_tx = kzalloc_obj(*kctx_tx);
+ if (!kctx_tx) {
+ atomic64_inc(&ktls->counters[BNXT_KTLS_ERR_NO_MEM]);
+ rc = -ENOMEM;
+ goto exit;
+ }
+ tls_ctx = tls_get_ctx(sk);
+ crypto = bp->crypto_info;
+ kctx = &crypto->kctx[BNXT_TX_CRYPTO_KEY_TYPE];
+ rc = bnxt_key_ctx_alloc_one(bp, kctx, BNXT_CTX_KIND_CK_TX, &kid);
+ if (rc) {
+ atomic64_inc(&ktls->counters[BNXT_KTLS_ERR_KEY_CTX_ALLOC]);
+ goto free_ctx;
+ }
+ rc = bnxt_crypto_add(bp, direction, crypto_info, start_offload_tcp_sn,
+ kid);
+ if (rc) {
+ atomic64_inc(&ktls->counters[BNXT_KTLS_ERR_CRYPTO_CMD]);
+ goto free_kctx;
+ }
+ kctx_tx->kid = kid;
+ kctx_tx->tcp_seq_no = start_offload_tcp_sn;
+ bnxt_set_ktls_ctx_tx(tls_ctx, kctx_tx);
+ atomic64_inc(&ktls->counters[BNXT_KTLS_TX_ADD]);
+ goto exit;
+
+free_kctx:
+ bnxt_free_one_kctx(kctx, kid);
+free_ctx:
+ kfree(kctx_tx);
+exit:
+ atomic_dec(&ktls->pending);
+ return rc;
+}
+
+#define KTLS_RETRY_MAX 100
+#define KTLS_WAIT_TMO_MS 100
+
+static void bnxt_ktls_dev_del(struct net_device *dev,
+ struct tls_context *tls_ctx,
+ enum tls_offload_ctx_dir direction)
+{
+ struct bnxt_ktls_offload_ctx_tx *kctx_tx;
+ struct bnxt *bp = netdev_priv(dev);
+ struct bnxt_crypto_info *crypto;
+ struct bnxt_tls_info *ktls;
+ struct bnxt_kctx *kctx;
+ int retry_cnt = 0;
+ u8 kind;
+ u32 kid;
+
+ ktls = bp->ktls_info;
+ kctx_tx = bnxt_get_ktls_ctx_tx(tls_ctx);
+retry:
+ if (!test_bit(BNXT_STATE_OPEN, &bp->state)) {
+ /* During ifdown or FW reset, all connections will be torn
+ * down by bnxt_crypto_del_all() / FUNC_RESET, so nothing to
+ * do here. Only a reconfiguration is transient and
+ * __bnxt_open_nic() will set BNXT_STATE_OPEN again and wake us.
+ */
+ if (!netif_running(dev) ||
+ test_bit(BNXT_STATE_IN_FW_RESET, &bp->state))
+ goto free;
+ /* Bound the wait so a wedged reconfig can't block the kTLS
+ * destruct work indefinitely.
+ */
+ if (retry_cnt++ > KTLS_RETRY_MAX) {
+ atomic64_inc(&ktls->counters[BNXT_KTLS_ERR_RETRY_EXCEEDED]);
+ netdev_warn(dev, "%s timed out waiting for device, state %lx\n",
+ __func__, bp->state);
+ goto free;
+ }
+ wait_event_timeout(ktls->open_wq,
+ test_bit(BNXT_STATE_OPEN, &bp->state) ||
+ !netif_running(dev) ||
+ test_bit(BNXT_STATE_IN_FW_RESET, &bp->state),
+ msecs_to_jiffies(KTLS_WAIT_TMO_MS));
+ goto retry;
+ }
+ atomic_inc(&ktls->pending);
+ /* Make sure bnxt_close_nic() sees pending before we check the
+ * BNXT_STATE_OPEN flag.
+ */
+ smp_mb__after_atomic();
+ if (!test_bit(BNXT_STATE_OPEN, &bp->state)) {
+ atomic_dec(&ktls->pending);
+ goto retry;
+ }
+
+ crypto = bp->crypto_info;
+ kid = kctx_tx->kid;
+ kctx = &crypto->kctx[BNXT_TX_CRYPTO_KEY_TYPE];
+ kind = BNXT_CTX_KIND_CK_TX;
+ atomic64_inc(&ktls->counters[BNXT_KTLS_TX_DEL]);
+ if (bnxt_kid_valid(kctx, kid) &&
+ !bnxt_crypto_del(bp, kctx->type, kind, kid))
+ bnxt_free_one_kctx(kctx, kid);
+
+ atomic_dec(&ktls->pending);
+free:
+ bnxt_set_ktls_ctx_tx(tls_ctx, NULL);
+ kfree(kctx_tx);
+}
+
+static const struct tlsdev_ops bnxt_ktls_ops = {
+ .tls_dev_add = bnxt_ktls_dev_add,
+ .tls_dev_del = bnxt_ktls_dev_del,
+};
+
+int bnxt_ktls_init(struct bnxt *bp)
+{
+ struct bnxt_tls_info *ktls = bp->ktls_info;
+ struct net_device *dev = bp->dev;
+
+ if (!ktls)
+ return 0;
+
+ dev->tlsdev_ops = &bnxt_ktls_ops;
+ dev->hw_features |= NETIF_F_HW_TLS_TX;
+ dev->features |= NETIF_F_HW_TLS_TX;
+ return 0;
+}
+
+static void bnxt_ktls_inc_tx_stats(struct bnxt_tx_ring_info *txr, u32 bytes,
+ bool ooo)
+{
+ struct bnxt_tls_sw_stats *ring_stats = txr->tls_stats;
+
+ if (!ring_stats)
+ return;
+ ring_stats->counters[BNXT_KTLS_TX_PKTS]++;
+ ring_stats->counters[BNXT_KTLS_TX_BYTES] += bytes;
+ if (ooo)
+ ring_stats->counters[BNXT_KTLS_TX_OOO_PKTS]++;
+}
+
+static void bnxt_ktls_pre_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ u32 kid, struct crypto_prefix_cmd *pre_cmd)
+{
+ struct bnxt_sw_tx_bd *tx_buf;
+ struct tx_bd_presync *psbd;
+ u32 bd_space, space;
+ u8 *pcmd;
+ u16 prod;
+
+ prod = txr->tx_prod;
+ tx_buf = &txr->tx_buf_ring[RING_TX(bp, prod)];
+
+ psbd = (void *)&txr->tx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)];
+ psbd->tx_bd_len_flags_type = CRYPTO_PRESYNC_BD_CMD;
+ psbd->tx_bd_kid = cpu_to_le32(BNXT_KID_HW(kid));
+ psbd->tx_bd_opaque =
+ SET_TX_OPAQUE(bp, txr, prod, CRYPTO_PREFIX_CMD_BDS + 1);
+
+ prod = NEXT_TX(prod);
+ pcmd = (void *)&txr->tx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)];
+ bd_space = TX_DESC_CNT - TX_IDX(prod);
+ space = bd_space * sizeof(struct tx_bd);
+ if (space >= CRYPTO_PREFIX_CMD_SIZE) {
+ memcpy(pcmd, pre_cmd, CRYPTO_PREFIX_CMD_SIZE);
+ prod += CRYPTO_PREFIX_CMD_BDS;
+ } else {
+ memcpy(pcmd, pre_cmd, space);
+ prod += bd_space;
+ pcmd = (void *)&txr->tx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)];
+ memcpy(pcmd, (u8 *)pre_cmd + space,
+ CRYPTO_PREFIX_CMD_SIZE - space);
+ prod += CRYPTO_PREFIX_CMD_BDS - bd_space;
+ }
+ txr->tx_prod = prod;
+ tx_buf->is_push = 1;
+ /* Minus 1 since the header psbd is a single entry short BD */
+ tx_buf->inline_data_bds = CRYPTO_PREFIX_CMD_BDS - 1;
+}
+
+static int bnxt_ktls_tx_ooo(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ struct sk_buff *skb, u32 payload_len, u32 seq,
+ struct tls_context *tls_ctx)
+{
+ struct bnxt_tls_sw_stats *ring_stats = txr->tls_stats;
+ struct tls_offload_context_tx *tx_tls_ctx;
+ struct bnxt_ktls_offload_ctx_tx *kctx_tx;
+ u32 hdr_tcp_seq, end_seq, total_bds;
+ struct crypto_prefix_cmd pcmd = {};
+ struct tls_record_info *record;
+ unsigned long flags;
+ bool fwd = false;
+ __le64 le_rec_sn;
+ u64 rec_sn;
+ u8 *hdr;
+ int rc;
+
+ tx_tls_ctx = tls_offload_ctx_tx(tls_ctx);
+ kctx_tx = bnxt_get_ktls_ctx_tx(tls_ctx);
+ end_seq = seq + skb->len - skb_tcp_all_headers(skb);
+ if (unlikely(after(seq, kctx_tx->tcp_seq_no) ||
+ after(end_seq, kctx_tx->tcp_seq_no))) {
+ fwd = true;
+ pcmd.flags = CRYPTO_PREFIX_CMD_FLAGS_UPDATE_IN_ORDER_VAR_LE;
+ }
+
+ spin_lock_irqsave(&tx_tls_ctx->lock, flags);
+ record = tls_get_record(tx_tls_ctx, seq, &rec_sn);
+ if (!record || !record->num_frags) {
+ rc = -EPROTO;
+ ring_stats->counters[BNXT_KTLS_TX_OOO_FALLBACK_NO_SYNC]++;
+ goto unlock_exit;
+ }
+ hdr_tcp_seq = tls_record_start_seq(record);
+ hdr = skb_frag_address_safe(&record->frags[0]);
+
+ total_bds = CRYPTO_PRESYNC_BDS + skb_shinfo(skb)->nr_frags + 2;
+ if (bnxt_tx_avail(bp, txr) < total_bds) {
+ rc = -ENOSPC;
+ ring_stats->counters[BNXT_KTLS_TX_OOO_FALLBACK_NO_SPACE]++;
+ goto unlock_exit;
+ }
+
+ if (before(record->end_seq - tls_ctx->prot_info.tag_size,
+ seq + payload_len)) {
+ /* retransmission includes tag bytes */
+ rc = -EOPNOTSUPP;
+ goto unlock_exit;
+ }
+ pcmd.header_tcp_seq_num = cpu_to_le32(hdr_tcp_seq);
+ pcmd.start_tcp_seq_num = cpu_to_le32(seq);
+ pcmd.end_tcp_seq_num = cpu_to_le32(seq + payload_len - 1);
+ if (tls_ctx->prot_info.version == TLS_1_2_VERSION) {
+ u32 nonce_bytes = tls_ctx->prot_info.iv_size;
+ u32 retrans_off = seq - hdr_tcp_seq;
+
+ if (!hdr) {
+ rc = -ENOBUFS;
+ ring_stats->counters[BNXT_KTLS_TX_OOO_FALLBACK_NO_HDR]++;
+ goto unlock_exit;
+ }
+ if (retrans_off > 5 && retrans_off < 5 + nonce_bytes)
+ nonce_bytes = retrans_off - 5;
+ memcpy(pcmd.explicit_nonce, hdr + 5, nonce_bytes);
+ }
+ le_rec_sn = cpu_to_le64(rec_sn);
+ memcpy(&pcmd.record_seq_num[0], &le_rec_sn, sizeof(le_rec_sn));
+
+ rc = 0;
+ bnxt_ktls_pre_xmit(bp, txr, kctx_tx->kid, &pcmd);
+
+ if (fwd) {
+ kctx_tx->next_tcp_seq_no = end_seq;
+ kctx_tx->pending_fwd = 1;
+ }
+
+unlock_exit:
+ spin_unlock_irqrestore(&tx_tls_ctx->lock, flags);
+ return rc;
+}
+
+struct sk_buff *bnxt_ktls_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ struct sk_buff *skb, __le32 *lflags, u32 *kid,
+ struct bnxt_ktls_offload_ctx_tx **kctx_tx_p)
+{
+ struct bnxt_tls_info *ktls = bp->ktls_info;
+ struct bnxt_ktls_offload_ctx_tx *kctx_tx;
+ struct tls_context *tls_ctx;
+ u32 seq, payload_len;
+ int rc;
+
+ if (!IS_ENABLED(CONFIG_TLS_DEVICE) || !ktls ||
+ !tls_is_skb_tx_device_offloaded(skb))
+ return skb;
+
+ seq = ntohl(tcp_hdr(skb)->seq);
+ tls_ctx = tls_get_ctx(skb->sk);
+ kctx_tx = bnxt_get_ktls_ctx_tx(tls_ctx);
+ payload_len = skb->len - skb_tcp_all_headers(skb);
+ if (!payload_len)
+ return skb;
+ if (kctx_tx->tcp_seq_no == seq) {
+ /* Stage the advance only. tcp_seq_no and the counters are
+ * committed by bnxt_ktls_xmit_commit() once the BD reaches the
+ * ring.
+ */
+ kctx_tx->next_tcp_seq_no = seq + payload_len;
+ kctx_tx->pending_bytes = payload_len;
+ kctx_tx->pending_ooo = 0;
+ kctx_tx->pending_fwd = 1;
+ *kid = BNXT_KID_HW(kctx_tx->kid);
+ *kctx_tx_p = kctx_tx;
+ *lflags |= cpu_to_le32(TX_BD_FLAGS_CRYPTO_EN |
+ BNXT_TX_KID_LO(*kid));
+ } else {
+ kctx_tx->pending_fwd = 0;
+ rc = bnxt_ktls_tx_ooo(bp, txr, skb, payload_len, seq, tls_ctx);
+ if (rc)
+ return tls_encrypt_skb(skb);
+
+ kctx_tx->pending_bytes = payload_len;
+ kctx_tx->pending_ooo = 1;
+ *kid = BNXT_KID_HW(kctx_tx->kid);
+ *kctx_tx_p = kctx_tx;
+ *lflags |= cpu_to_le32(TX_BD_FLAGS_CRYPTO_EN |
+ BNXT_TX_KID_LO(*kid));
+ return skb;
+ }
+ return skb;
+}
+
+void bnxt_ktls_xmit_commit(struct bnxt_tx_ring_info *txr,
+ struct bnxt_ktls_offload_ctx_tx *kctx_tx)
+{
+ if (!kctx_tx)
+ return;
+ if (kctx_tx->pending_fwd)
+ kctx_tx->tcp_seq_no = kctx_tx->next_tcp_seq_no;
+ bnxt_ktls_inc_tx_stats(txr, kctx_tx->pending_bytes,
+ kctx_tx->pending_ooo);
+ kctx_tx->pending_bytes = 0;
+ kctx_tx->pending_fwd = 0;
+ kctx_tx->pending_ooo = 0;
+}
+
+int bnxt_ktls_alloc_tx_ring_stats(struct bnxt *bp, struct bnxt_tx_ring_info *txr)
+{
+ struct bnxt_tls_sw_stats *ring_stats;
+
+ if (!bp->ktls_info)
+ return 0;
+ ring_stats = kzalloc_obj(*ring_stats);
+ if (!ring_stats)
+ return -ENOMEM;
+ txr->tls_stats = ring_stats;
+ return 0;
+}
+
+void bnxt_ktls_free_tx_ring_stats(struct bnxt_tx_ring_info *txr)
+{
+ kfree(txr->tls_stats);
+ txr->tls_stats = NULL;
+}
+
+void bnxt_get_ring_tls_stats(struct bnxt *bp, struct bnxt_tls_sw_stats *stats)
+{
+ struct bnxt_tls_sw_stats *ring_stats;
+ int i, j;
+
+ if (!bp->ktls_info || !bp->tx_ring)
+ return;
+ for (i = 0; i < bp->tx_nr_rings; i++) {
+ ring_stats = bp->tx_ring[i].tls_stats;
+ if (!ring_stats)
+ continue;
+ for (j = 0; j < BNXT_KTLS_MAX_DATA_COUNTERS; j++)
+ stats->counters[j] += ring_stats->counters[j];
+ }
+}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h
new file mode 100644
index 00000000000000..40b94bbf5a3836
--- /dev/null
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ktls.h
@@ -0,0 +1,175 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (c) 2026 Broadcom Inc. */
+
+#ifndef BNXT_KTLS_H
+#define BNXT_KTLS_H
+
+#include <linux/u64_stats_sync.h>
+#include <linux/wait.h>
+
+/* Control plane counters */
+enum bnxt_ktls_ctrl_counters {
+ BNXT_KTLS_TX_ADD = 0,
+ BNXT_KTLS_TX_DEL,
+
+ /* Error counters for debugging */
+ BNXT_KTLS_ERR_NO_MEM, /* Memory allocation failure */
+ BNXT_KTLS_ERR_NO_CAP, /* Capability lost after FW reset */
+ BNXT_KTLS_ERR_KEY_CTX_ALLOC, /* Key context alloc failure */
+ BNXT_KTLS_ERR_CRYPTO_CMD, /* Crypto command failure */
+ BNXT_KTLS_ERR_DEVICE_BUSY, /* Device not ready */
+ BNXT_KTLS_ERR_INVALID_CIPHER, /* Unsupported cipher */
+ BNXT_KTLS_ERR_STATE_NOT_OPEN, /* Device not open */
+ BNXT_KTLS_ERR_RETRY_EXCEEDED, /* Retry limit exceeded */
+
+ BNXT_KTLS_MAX_CTRL_COUNTERS,
+};
+
+struct bnxt_tls_info {
+ atomic_t pending;
+
+ /* Woken from __bnxt_open_nic()/__bnxt_close_nic() when
+ * BNXT_STATE_OPEN changes, so a kTLS delete can wait out a ring
+ * reconfiguration instead of polling the state bit.
+ */
+ wait_queue_head_t open_wq;
+
+ /* Atomic counters for control path */
+ atomic64_t *counters;
+};
+
+struct bnxt_ktls_offload_ctx_tx {
+ u32 tcp_seq_no; /* tcp seq no in sync with HW */
+ u32 next_tcp_seq_no;/* staged tcp seq no */
+ u32 kid;
+ u32 pending_bytes; /* staged payload bytes */
+ u32 pending_fwd:1;
+ u32 pending_ooo:1;
+};
+
+struct bnxt_ktls_tx_driver_state {
+ struct bnxt_ktls_offload_ctx_tx *ctx_tx;
+};
+
+struct ce_add_cmd {
+ __le32 ver_algo_kid_opcode;
+ #define CE_ADD_CMD_OPCODE_MASK 0xfUL
+ #define CE_ADD_CMD_OPCODE_SFT 0
+ #define CE_ADD_CMD_OPCODE_ADD 0x1UL
+ #define CE_ADD_CMD_KID_MASK 0xfffff0UL
+ #define CE_ADD_CMD_KID_SFT 4
+ #define CE_ADD_CMD_ALGORITHM_MASK 0xf000000UL
+ #define CE_ADD_CMD_ALGORITHM_SFT 24
+ #define CE_ADD_CMD_ALGORITHM_AES_GCM_128 0x1000000UL
+ #define CE_ADD_CMD_ALGORITHM_AES_GCM_256 0x2000000UL
+ #define CE_ADD_CMD_VERSION_MASK 0xf0000000UL
+ #define CE_ADD_CMD_VERSION_SFT 28
+ #define CE_ADD_CMD_VERSION_TLS1_2 (0x0UL << 28)
+ #define CE_ADD_CMD_VERSION_TLS1_3 (0x1UL << 28)
+ u8 ctx_kind;
+ #define CE_ADD_CMD_CTX_KIND_MASK 0x1fUL
+ #define CE_ADD_CMD_CTX_KIND_SFT 0
+ #define CE_ADD_CMD_CTX_KIND_CK_TX 0x11UL
+ #define CE_ADD_CMD_CTX_KIND_CK_RX 0x12UL
+ u8 unused0[3];
+ u8 salt[4];
+ u8 unused1[4];
+ __le32 pkt_tcp_seq_num;
+ __le32 tls_header_tcp_seq_num;
+ u8 record_seq_num[8];
+ u8 session_key[32];
+ u8 addl_iv[8];
+};
+
+struct crypto_prefix_cmd {
+ __le32 flags;
+ #define CRYPTO_PREFIX_CMD_FLAGS_UPDATE_IN_ORDER_VAR 0x1UL
+ #define CRYPTO_PREFIX_CMD_FLAGS_FULL_REPLAY_RETRAN 0x2UL
+ __le32 header_tcp_seq_num;
+ __le32 start_tcp_seq_num;
+ __le32 end_tcp_seq_num;
+ u8 explicit_nonce[8];
+ u8 record_seq_num[8];
+};
+
+#define CRYPTO_PREFIX_CMD_FLAGS_UPDATE_IN_ORDER_VAR_LE \
+ cpu_to_le32(CRYPTO_PREFIX_CMD_FLAGS_UPDATE_IN_ORDER_VAR)
+
+#define CRYPTO_PREFIX_CMD_SIZE ((u32)sizeof(struct crypto_prefix_cmd))
+#define CRYPTO_PREFIX_CMD_BDS (CRYPTO_PREFIX_CMD_SIZE / sizeof(struct tx_bd))
+#define CRYPTO_PRESYNC_BDS (CRYPTO_PREFIX_CMD_BDS + 1)
+
+#define CRYPTO_PRESYNC_BD_CMD \
+ (cpu_to_le32((CRYPTO_PREFIX_CMD_SIZE << TX_BD_LEN_SHIFT) | \
+ TX_BD_CNT(CRYPTO_PRESYNC_BDS) | TX_BD_TYPE_PRESYNC_TX_BD))
+
+static inline bool bnxt_ktls_busy(struct bnxt *bp)
+{
+ return bp->ktls_info && atomic_read(&bp->ktls_info->pending) > 0;
+}
+
+/* Wake any kTLS control op waiting for a BNXT_STATE_OPEN transition. */
+static inline void bnxt_ktls_wake(struct bnxt *bp)
+{
+ if (bp->ktls_info)
+ wake_up_all(&bp->ktls_info->open_wq);
+}
+
+#ifdef CONFIG_BNXT_TLS
+int bnxt_alloc_ktls_info(struct bnxt *bp);
+void bnxt_free_ktls_info(struct bnxt *bp);
+int bnxt_ktls_init(struct bnxt *bp);
+struct sk_buff *bnxt_ktls_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ struct sk_buff *skb, __le32 *lflags, u32 *kid,
+ struct bnxt_ktls_offload_ctx_tx **kctx_tx_p);
+void bnxt_ktls_xmit_commit(struct bnxt_tx_ring_info *txr,
+ struct bnxt_ktls_offload_ctx_tx *kctx_tx);
+int bnxt_ktls_alloc_tx_ring_stats(struct bnxt *bp,
+ struct bnxt_tx_ring_info *txr);
+void bnxt_ktls_free_tx_ring_stats(struct bnxt_tx_ring_info *txr);
+void bnxt_get_ring_tls_stats(struct bnxt *bp, struct bnxt_tls_sw_stats *stats);
+#else
+static inline int bnxt_alloc_ktls_info(struct bnxt *bp)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline void bnxt_free_ktls_info(struct bnxt *bp)
+{
+}
+
+static inline int bnxt_ktls_init(struct bnxt *bp)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline struct sk_buff *
+bnxt_ktls_xmit(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ struct sk_buff *skb, __le32 *lflags, u32 *kid,
+ struct bnxt_ktls_offload_ctx_tx **kctx_tx_p)
+{
+ return skb;
+}
+
+static inline void
+bnxt_ktls_xmit_commit(struct bnxt_tx_ring_info *txr,
+ struct bnxt_ktls_offload_ctx_tx *kctx_tx)
+{
+}
+
+static inline int bnxt_ktls_alloc_tx_ring_stats(struct bnxt *bp,
+ struct bnxt_tx_ring_info *txr)
+{
+ return 0;
+}
+
+static inline void bnxt_ktls_free_tx_ring_stats(struct bnxt_tx_ring_info *txr)
+{
+}
+
+static inline void bnxt_get_ring_tls_stats(struct bnxt *bp,
+ struct bnxt_tls_sw_stats *stats)
+{
+}
+#endif /* CONFIG_BNXT_TLS */
+#endif /* BNXT_KTLS_H */
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c
new file mode 100644
index 00000000000000..d66265b43bed4f
--- /dev/null
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.c
@@ -0,0 +1,732 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (c) 2026 Broadcom Inc. */
+
+#include <linux/stddef.h>
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/bnxt/hsi.h>
+
+#include "bnxt.h"
+#include "bnxt_mpc.h"
+#include "bnxt_crypto.h"
+
+void bnxt_alloc_mpc_info(struct bnxt *bp, u8 mpc_chnls_cap)
+{
+ if (!bp->mpc_info)
+ bp->mpc_info = kzalloc_obj(*bp->mpc_info);
+ if (bp->mpc_info)
+ bp->mpc_info->mpc_chnls_cap = mpc_chnls_cap;
+ else
+ netdev_warn(bp->dev, "Unable to allocate MPC info\n");
+}
+
+void bnxt_free_mpc_info(struct bnxt *bp)
+{
+ bnxt_free_mpc_rings(bp);
+ bnxt_free_mpcs(bp);
+ kfree(bp->mpc_info);
+ bp->mpc_info = NULL;
+}
+
+int bnxt_mpc_tx_rings_in_use(struct bnxt *bp)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ int i, mpc_tx = 0;
+
+ if (!mpc)
+ return 0;
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++)
+ mpc_tx += mpc->mpc_ring_count[i];
+ return mpc_tx;
+}
+
+int bnxt_mpc_cp_rings_in_use(struct bnxt *bp)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+
+ if (!mpc)
+ return 0;
+ return mpc->mpc_cp_rings;
+}
+
+bool bnxt_napi_has_mpc(struct bnxt *bp, int i)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ struct bnxt_napi *bnapi = bp->bnapi[i];
+ struct bnxt_tx_ring_info *txr;
+
+ if (!mpc)
+ return false;
+
+ txr = bnapi->tx_ring[0];
+ if (txr && !(bnapi->flags & BNXT_NAPI_FLAG_XDP))
+ return txr->txq_index < mpc->mpc_cp_rings;
+ return false;
+}
+
+void bnxt_set_mpc_cp_ring(struct bnxt *bp, int bnapi_idx,
+ struct bnxt_cp_ring_info *cpr)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ struct bnxt_napi *bnapi;
+ bool found = false;
+ int i, j;
+
+ if (!mpc)
+ return;
+ bnapi = bp->bnapi[bnapi_idx];
+ /* Check both TCE and RCE MPCs for the matching NAPI */
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ int num = mpc->mpc_ring_count[i];
+
+ for (j = 0; j < num; j++) {
+ struct bnxt_tx_ring_info *txr = &mpc->mpc_rings[i][j];
+
+ /* Only 1 ring with index j will use this NAPI */
+ if (txr->bnapi == bnapi) {
+ txr->tx_cpr = cpr;
+ txr->tx_napi_idx = i;
+ bnapi->tx_mpc_ring[i] = txr;
+ found = true;
+ break;
+ }
+ }
+ }
+ if (!found)
+ netdev_warn_once(bp->dev, "No MPC match for napi index %d\n",
+ bnapi_idx);
+ cpr->cp_ring_type = BNXT_NQ_HDL_TYPE_MP;
+}
+
+void bnxt_trim_mpc_rings(struct bnxt *bp)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ int max = bp->tx_nr_rings_per_tc;
+ u8 max_cp = 0;
+ int i;
+
+ if (!mpc)
+ return;
+
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ mpc->mpc_ring_count[i] = min_t(int, mpc->mpc_ring_count[i],
+ max);
+ max_cp = max(max_cp, mpc->mpc_ring_count[i]);
+ }
+ mpc->mpc_cp_rings = max_cp;
+}
+
+void bnxt_set_dflt_mpc_rings(struct bnxt *bp)
+{
+ struct bnxt_hw_resc *hw_resc = &bp->hw_resc;
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ int mpc_tce, mpc_rce, avail, mpc_cp, i;
+
+ if (!BNXT_MPC_CRYPTO_CAPABLE(bp))
+ return;
+
+ avail = hw_resc->max_tx_rings - bp->tx_nr_rings;
+ /* don't use more than 80% */
+ avail = avail * 4 / 5;
+
+ if (avail < (BNXT_MIN_MPC_TCE + BNXT_MIN_MPC_RCE))
+ goto disable_mpc;
+
+ mpc_tce = min_t(int, avail / 2, bp->tx_nr_rings_per_tc);
+ mpc_rce = mpc_tce;
+
+ mpc_tce = min_t(int, mpc_tce, BNXT_DFLT_MPC_TCE);
+ mpc_rce = min_t(int, mpc_rce, BNXT_DFLT_MPC_RCE);
+
+ avail = hw_resc->max_cp_rings - bp->tx_nr_rings -
+ bp->rx_nr_rings;
+
+ if (avail < BNXT_MIN_MPC_TCE || avail < BNXT_MIN_MPC_RCE)
+ goto disable_mpc;
+
+ mpc_tce = min(mpc_tce, avail);
+ mpc_rce = min(mpc_rce, avail);
+
+ mpc_cp = max(mpc_tce, mpc_rce);
+
+ mpc->mpc_ring_count[BNXT_MPC_TCE_TYPE] = mpc_tce;
+ mpc->mpc_ring_count[BNXT_MPC_RCE_TYPE] = mpc_rce;
+ mpc->mpc_cp_rings = mpc_cp;
+ return;
+
+disable_mpc:
+ mpc->mpc_cp_rings = 0;
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++)
+ mpc->mpc_ring_count[i] = 0;
+}
+
+void bnxt_init_mpc_ring_struct(struct bnxt *bp)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ int i, j;
+
+ if (!BNXT_MPC_CRYPTO_CAPABLE(bp))
+ return;
+
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ int num = mpc->mpc_ring_count[i];
+
+ if (!mpc->mpc_rings[i])
+ continue;
+ for (j = 0; j < num; j++) {
+ struct bnxt_ring_mem_info *rmem;
+ struct bnxt_ring_struct *ring;
+ struct bnxt_tx_ring_info *txr;
+
+ txr = &mpc->mpc_rings[i][j];
+
+ txr->tx_ring_struct.ring_mem.flags =
+ BNXT_RMEM_RING_PTE_FLAG;
+ txr->bnapi = bp->tx_ring[bp->tx_ring_map[j]].bnapi;
+ txr->txq_index = j;
+
+ ring = &txr->tx_ring_struct;
+ rmem = &ring->ring_mem;
+ rmem->nr_pages = bp->tx_nr_pages;
+ rmem->page_size = HW_TXBD_RING_SIZE;
+ rmem->pg_arr = (void **)txr->tx_desc_ring;
+ rmem->dma_arr = txr->tx_desc_mapping;
+ rmem->vmem_size = SW_MPC_TXBD_RING_SIZE *
+ bp->tx_nr_pages;
+ rmem->vmem = (void **)&txr->tx_mpc_buf_ring;
+ }
+ }
+}
+
+int bnxt_alloc_mpcs(struct bnxt *bp)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ int i, rc = 0;
+
+ if (!BNXT_MPC_CRYPTO_CAPABLE(bp))
+ return 0;
+
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ int num = mpc->mpc_ring_count[i];
+ struct bnxt_tx_ring_info *txr;
+
+ if (!num)
+ continue;
+ txr = kzalloc_objs(*txr, num);
+ if (!txr) {
+ rc = -ENOMEM;
+ goto alloc_mpcs_exit;
+ }
+ mpc->mpc_rings[i] = txr;
+ }
+
+ for (i = 0; i < bp->cp_nr_rings; i++) {
+ struct bnxt_napi *bnapi = bp->bnapi[i];
+
+ if (!bnxt_napi_has_mpc(bp, i))
+ continue;
+ bnapi->tx_mpc_ring = kzalloc_objs(*bnapi->tx_mpc_ring,
+ BNXT_MPC_TYPE_MAX);
+ if (!bnapi->tx_mpc_ring) {
+ rc = -ENOMEM;
+ goto alloc_mpcs_exit;
+ }
+ }
+alloc_mpcs_exit:
+ if (rc)
+ bnxt_free_mpcs(bp);
+ return rc;
+}
+
+void bnxt_free_mpcs(struct bnxt *bp)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ int i;
+
+ if (!mpc)
+ return;
+
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ kfree(mpc->mpc_rings[i]);
+ mpc->mpc_rings[i] = NULL;
+ }
+ if (!bp->bnapi)
+ return;
+ for (i = 0; i < bp->cp_nr_rings; i++) {
+ struct bnxt_napi *bnapi = bp->bnapi[i];
+
+ kfree(bnapi->tx_mpc_ring);
+ bnapi->tx_mpc_ring = NULL;
+ }
+}
+
+int bnxt_alloc_mpc_rings(struct bnxt *bp)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ int i, j, rc = 0;
+
+ if (!mpc)
+ return 0;
+
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ int num = mpc->mpc_ring_count[i];
+
+ for (j = 0; j < num; j++) {
+ struct bnxt_tx_ring_info *txr = &mpc->mpc_rings[i][j];
+ struct bnxt_ring_struct *ring;
+
+ ring = &txr->tx_ring_struct;
+ rc = bnxt_alloc_ring(bp, &ring->ring_mem);
+ if (rc)
+ goto alloc_mpc_rings_exit;
+ ring->queue_id = BNXT_MPC_QUEUE_ID;
+ ring->mpc_chnl_type = i;
+ /* for stats context */
+ ring->grp_idx = txr->bnapi->index;
+ spin_lock_init(&txr->tx_lock);
+ }
+ }
+alloc_mpc_rings_exit:
+ if (rc)
+ bnxt_free_mpc_rings(bp);
+ return rc;
+}
+
+void bnxt_free_mpc_rings(struct bnxt *bp)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ int i, j;
+
+ if (!mpc)
+ return;
+
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ int num = mpc->mpc_ring_count[i];
+
+ if (!mpc->mpc_rings[i])
+ continue;
+ for (j = 0; j < num; j++) {
+ struct bnxt_tx_ring_info *txr = &mpc->mpc_rings[i][j];
+ struct bnxt_ring_struct *ring = &txr->tx_ring_struct;
+
+ bnxt_free_ring(bp, &ring->ring_mem);
+ }
+ }
+}
+
+void bnxt_init_mpc_rings(struct bnxt *bp)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ int i, j;
+
+ if (!mpc)
+ return;
+
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ int num = mpc->mpc_ring_count[i];
+
+ for (j = 0; j < num; j++) {
+ struct bnxt_tx_ring_info *txr = &mpc->mpc_rings[i][j];
+ struct bnxt_ring_struct *ring = &txr->tx_ring_struct;
+
+ txr->tx_prod = 0;
+ txr->tx_cons = 0;
+ txr->tx_hw_cons = 0;
+ ring->fw_ring_id = INVALID_HW_RING_ID;
+ }
+ }
+}
+
+static int bnxt_hwrm_one_mpc_ring_alloc(struct bnxt *bp,
+ struct bnxt_tx_ring_info *txr)
+{
+ struct bnxt_cp_ring_info *cpr = txr->tx_cpr;
+ struct bnxt_ring_struct *ring;
+ int rc;
+
+ ring = &cpr->cp_ring_struct;
+ if (ring->fw_ring_id == INVALID_HW_RING_ID) {
+ rc = bnxt_hwrm_cp_ring_alloc_p5(bp, cpr);
+ if (rc)
+ return rc;
+ }
+ /* tx_idx not used on P5_PLUS, so set it to 0 */
+ return bnxt_hwrm_tx_ring_alloc(bp, txr, 0);
+}
+
+int bnxt_hwrm_mpc_ring_alloc(struct bnxt *bp)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ int i, j, rc = 0;
+
+ if (!mpc)
+ return 0;
+
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ int num = mpc->mpc_ring_count[i];
+
+ for (j = 0; j < num; j++) {
+ struct bnxt_tx_ring_info *txr = &mpc->mpc_rings[i][j];
+
+ rc = bnxt_hwrm_one_mpc_ring_alloc(bp, txr);
+ if (rc)
+ goto mpc_ring_alloc_exit;
+ }
+ }
+mpc_ring_alloc_exit:
+ if (rc)
+ bnxt_hwrm_mpc_ring_free(bp, false);
+ return rc;
+}
+
+void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ struct bnxt_cp_ring_info *cpr;
+ int i, j;
+
+ if (!mpc)
+ return;
+
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ for (j = 0; j < mpc->mpc_ring_count[i]; j++)
+ bnxt_hwrm_tx_ring_free(bp, &mpc->mpc_rings[i][j],
+ close_path);
+ }
+ /* CP rings must be freed at the end to guarantee that the HWRM_DONE
+ * responses for HWRM_RING_FREE can still be seen on the CP rings.
+ */
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ for (j = 0; j < mpc->mpc_ring_count[i]; j++) {
+ cpr = mpc->mpc_rings[i][j].tx_cpr;
+ if (cpr && cpr->cp_ring_struct.fw_ring_id !=
+ INVALID_HW_RING_ID)
+ bnxt_hwrm_cp_ring_free(bp, cpr);
+ }
+ }
+}
+
+struct bnxt_tx_ring_info *bnxt_select_mpc_ring(struct bnxt *bp, int ring_type)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ int n;
+
+ if (!mpc || ring_type >= BNXT_MPC_TYPE_MAX ||
+ !mpc->mpc_ring_count[ring_type])
+ return NULL;
+
+ n = raw_smp_processor_id() % mpc->mpc_ring_count[ring_type];
+ return &mpc->mpc_rings[ring_type][n];
+}
+
+/**
+ * bnxt_start_xmit_mpc - Transmit message on an MPC ring
+ * @bp: pointer to bnxt device
+ * @txr: MPC TX ring structure pointer
+ * @data: MPC message pointer
+ * @len: MPC message length
+ * @handle: Non-zero handle passed back for the completion
+ *
+ * This function is called to transmit an MPC message on an MPC TX ring.
+ * The caller must hold txr->tx_lock. When successful, the HW will return
+ * a completion and bnxt_crypto_mpc_cmp() will be called with the handle
+ * passed back.
+ *
+ * Return: zero on success, negative error code otherwise:
+ * ENODEV: MPC TX ring is shutting down.
+ * EBUSY: MPC TX ring is full
+ */
+int bnxt_start_xmit_mpc(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ void *data, unsigned int len, unsigned long handle)
+{
+ u32 bds, total_bds, bd_space, free_size;
+ struct bnxt_sw_mpc_tx_bd *tx_buf;
+ struct tx_bd *txbd;
+ u16 prod;
+
+ if (READ_ONCE(txr->dev_state) == BNXT_DEV_STATE_CLOSING)
+ return -ENODEV;
+
+ bds = DIV_ROUND_UP(len, sizeof(*txbd));
+ total_bds = bds + 1;
+ free_size = bnxt_tx_avail(bp, txr);
+ if (free_size < total_bds)
+ return -EBUSY;
+
+ prod = txr->tx_prod;
+ txbd = &txr->tx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)];
+ tx_buf = &txr->tx_mpc_buf_ring[RING_TX(bp, prod)];
+ tx_buf->handle = handle;
+ tx_buf->inline_bds = total_bds;
+
+ txbd->tx_bd_len_flags_type =
+ cpu_to_le32((len << TX_BD_LEN_SHIFT) | TX_BD_TYPE_MPC_TX_BD |
+ TX_BD_CNT(total_bds));
+ txbd->tx_bd_opaque = SET_TX_OPAQUE(bp, txr, prod, total_bds);
+
+ prod = NEXT_TX(prod);
+ txbd = &txr->tx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)];
+ bd_space = TX_DESC_CNT - TX_IDX(prod);
+ if (bd_space < bds) {
+ unsigned int len0 = bd_space * sizeof(*txbd);
+
+ memcpy(txbd, data, len0);
+ prod += bd_space;
+ txbd = &txr->tx_desc_ring[TX_RING(bp, prod)][TX_IDX(prod)];
+ bds -= bd_space;
+ len -= len0;
+ data += len0;
+ }
+ memcpy(txbd, data, len);
+ prod += bds;
+ WRITE_ONCE(txr->tx_prod, prod);
+
+ /* Sync BD data before updating doorbell */
+ wmb();
+ bnxt_db_write(bp, &txr->tx_db, prod);
+
+ return 0;
+}
+
+/* Returns true if the ring is successfully marked as closing. */
+static bool bnxt_disable_mpc_ring(struct bnxt_mpc_info *mpc, int mpc_ring)
+{
+ struct bnxt_tx_ring_info *txr;
+ bool disabled = false;
+ int i;
+
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ if (mpc_ring >= mpc->mpc_ring_count[i])
+ continue;
+ txr = &mpc->mpc_rings[i][mpc_ring];
+ spin_lock_bh(&txr->tx_lock);
+ if (!READ_ONCE(txr->dev_state)) {
+ disabled = true;
+ WRITE_ONCE(txr->dev_state, BNXT_DEV_STATE_CLOSING);
+ }
+ spin_unlock_bh(&txr->tx_lock);
+ if (!disabled)
+ break;
+ }
+ /* Make sure napi polls see @dev_state change */
+ if (disabled)
+ synchronize_net();
+ return disabled;
+}
+
+static void bnxt_enable_mpc_ring(struct bnxt_mpc_info *mpc, int mpc_ring)
+{
+ struct bnxt_tx_ring_info *txr;
+ int i;
+
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ if (mpc_ring >= mpc->mpc_ring_count[i])
+ continue;
+ txr = &mpc->mpc_rings[i][mpc_ring];
+ WRITE_ONCE(txr->dev_state, 0);
+ }
+}
+
+static void bnxt_clear_one_mpc_entries(struct bnxt *bp,
+ struct bnxt_tx_ring_info *txr)
+{
+ struct bnxt_sw_mpc_tx_bd *tx_buf;
+ unsigned long handle;
+ int i, max_idx;
+ u32 client;
+
+ max_idx = bp->tx_nr_pages * TX_DESC_CNT;
+
+ for (i = 0; i < max_idx; i++) {
+ tx_buf = &txr->tx_mpc_buf_ring[i];
+ handle = tx_buf->handle;
+ if (handle) {
+ client = txr->tx_ring_struct.mpc_chnl_type;
+ bnxt_crypto_mpc_cmp(bp, client, handle, NULL, 0);
+ tx_buf->handle = 0;
+ }
+ }
+}
+
+static void bnxt_mpc_ring_stop(struct bnxt *bp, struct bnxt_mpc_info *mpc,
+ int mpc_ring)
+{
+ struct bnxt_tx_ring_info *txr;
+ struct bnxt_cp_ring_info *cpr;
+ int i;
+
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ if (mpc->mpc_ring_count[i] > mpc_ring) {
+ txr = &mpc->mpc_rings[i][mpc_ring];
+ bnxt_hwrm_tx_ring_free(bp, txr, true);
+ }
+ }
+ /* CP rings must be freed at the end to guarantee that the HWRM_DONE
+ * responses for HWRM_RING_FREE can still be seen on the CP rings.
+ */
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ if (mpc->mpc_ring_count[i] > mpc_ring) {
+ txr = &mpc->mpc_rings[i][mpc_ring];
+ cpr = txr->tx_cpr;
+ if (cpr) {
+ bnxt_hwrm_cp_ring_free(bp, cpr);
+ bnxt_clear_one_cp_ring(bp, cpr);
+ }
+ bnxt_clear_one_mpc_entries(bp, txr);
+ }
+ }
+}
+
+static int bnxt_mpc_ring_start(struct bnxt *bp, struct bnxt_mpc_info *mpc,
+ int mpc_ring)
+{
+ struct bnxt_tx_ring_info *txr;
+ int i, rc;
+
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ if (mpc->mpc_ring_count[i] > mpc_ring) {
+ txr = &mpc->mpc_rings[i][mpc_ring];
+ txr->tx_prod = 0;
+ txr->tx_cons = 0;
+ txr->tx_hw_cons = 0;
+ rc = bnxt_hwrm_one_mpc_ring_alloc(bp, txr);
+ if (rc)
+ return rc;
+ }
+ }
+ return 0;
+}
+
+static int bnxt_mpc_ring_reset(struct bnxt *bp, int mpc_ring)
+{
+ struct bnxt_mpc_info *mpc = bp->mpc_info;
+ struct bnxt_tx_ring_info *txr;
+ int i, rc;
+
+ if (!mpc)
+ return 0;
+ if (mpc_ring >= mpc->mpc_cp_rings)
+ return -EINVAL;
+
+ if (!bnxt_disable_mpc_ring(mpc, mpc_ring))
+ return 0;
+
+ /* If device is going down, the MPC rings will be freed anyway so just
+ * clear the MPC entries.
+ */
+ if (!test_bit(BNXT_STATE_OPEN, &bp->state)) {
+ for (i = 0; i < BNXT_MPC_TYPE_MAX; i++) {
+ if (mpc->mpc_ring_count[i] > mpc_ring) {
+ txr = &mpc->mpc_rings[i][mpc_ring];
+ bnxt_clear_one_mpc_entries(bp, txr);
+ }
+ }
+ bnxt_enable_mpc_ring(mpc, mpc_ring);
+ return 0;
+ }
+ netdev_warn(bp->dev, "Resetting MPC ring %d\n", mpc_ring);
+ netdev_lock(bp->dev);
+ bnxt_mpc_ring_stop(bp, mpc, mpc_ring);
+
+ rc = bnxt_mpc_ring_start(bp, mpc, mpc_ring);
+ if (rc) {
+ netdev_err(bp->dev, "Error starting MPC ring %d, rc: %d, resetting device\n",
+ mpc_ring, rc);
+ bnxt_mpc_ring_stop(bp, mpc, mpc_ring);
+ bnxt_reset_task(bp, true);
+ netdev_unlock(bp->dev);
+ /* Return here as bnxt_reset_task() will clear everything */
+ return rc;
+ }
+ netdev_unlock(bp->dev);
+ bnxt_enable_mpc_ring(mpc, mpc_ring);
+ return 0;
+}
+
+int bnxt_mpc_timeout(struct bnxt *bp, struct bnxt_tx_ring_info *txr)
+{
+ if (txr->tx_ring_struct.queue_id == BNXT_MPC_QUEUE_ID)
+ return bnxt_mpc_ring_reset(bp, txr->txq_index);
+ return -EINVAL;
+}
+
+static bool bnxt_mpc_unsolicit(struct mpc_cmp *mpcmp)
+{
+ u32 client = MPC_CMP_CLIENT_TYPE(mpcmp);
+
+ if (client != MPC_CMP_CLIENT_TCE && client != MPC_CMP_CLIENT_RCE)
+ return false;
+ return MPC_CMP_UNSOLICIT_SUBTYPE(mpcmp);
+}
+
+int bnxt_mpc_cmp(struct bnxt *bp, struct bnxt_cp_ring_info *cpr, u32 *raw_cons)
+{
+ struct bnxt_cmpl_entry cmpl_entry_arr[2];
+ struct bnxt_napi *bnapi = cpr->bnapi;
+ u16 cons = RING_CMP(*raw_cons);
+ struct mpc_cmp *mpcmp, *mpcmp1;
+ u32 tmp_raw_cons = *raw_cons;
+ unsigned long handle = 0;
+ u32 client, cmpl_num;
+ u8 type;
+
+ mpcmp = (struct mpc_cmp *)
+ &cpr->cp_desc_ring[CP_RING(cons)][CP_IDX(cons)];
+ type = MPC_CMP_CMP_TYPE(mpcmp);
+ cmpl_entry_arr[0].cmpl = mpcmp;
+ cmpl_entry_arr[0].len = sizeof(*mpcmp);
+ cmpl_num = 1;
+ if (type == MPC_CMP_TYPE_MID_PATH_LONG) {
+ tmp_raw_cons = NEXT_RAW_CMP(tmp_raw_cons);
+ cons = RING_CMP(tmp_raw_cons);
+ mpcmp1 = (struct mpc_cmp *)
+ &cpr->cp_desc_ring[CP_RING(cons)][CP_IDX(cons)];
+
+ if (!MPC_CMP_VALID(bp, mpcmp1, tmp_raw_cons))
+ return -EBUSY;
+ /* The valid test of the entry must be done first before
+ * reading any further.
+ */
+ dma_rmb();
+ if (mpcmp1 == mpcmp + 1) {
+ cmpl_entry_arr[cmpl_num - 1].len += sizeof(*mpcmp1);
+ } else {
+ cmpl_entry_arr[cmpl_num].cmpl = mpcmp1;
+ cmpl_entry_arr[cmpl_num].len = sizeof(*mpcmp1);
+ cmpl_num++;
+ }
+ }
+ client = MPC_CMP_CLIENT_TYPE(mpcmp) >> MPC_CMP_CLIENT_SFT;
+ if (client >= BNXT_MPC_TYPE_MAX)
+ goto cmp_done;
+
+ if (!bnxt_mpc_unsolicit(mpcmp)) {
+ struct bnxt_sw_mpc_tx_bd *mpc_buf;
+ struct bnxt_tx_ring_info *txr;
+ u16 tx_cons;
+ u32 opaque;
+
+ opaque = mpcmp->mpc_cmp_opaque;
+ txr = bnapi->tx_mpc_ring[client];
+ tx_cons = txr->tx_cons;
+ if (TX_OPAQUE_RING(opaque) != txr->tx_napi_idx) {
+ netdev_warn(bp->dev, "Wrong opaque %x, expected ring %x, cons idx %x\n",
+ opaque, txr->tx_napi_idx, txr->tx_cons);
+ goto cmp_done;
+ }
+ mpc_buf = &txr->tx_mpc_buf_ring[RING_TX(bp, tx_cons)];
+ if (!READ_ONCE(txr->dev_state)) {
+ handle = mpc_buf->handle;
+ mpc_buf->handle = 0;
+ }
+ tx_cons += mpc_buf->inline_bds;
+ WRITE_ONCE(txr->tx_cons, tx_cons);
+ txr->tx_hw_cons = RING_TX(bp, tx_cons);
+ }
+ bnxt_crypto_mpc_cmp(bp, client, handle, cmpl_entry_arr, cmpl_num);
+
+cmp_done:
+ *raw_cons = tmp_raw_cons;
+ return 0;
+}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h
new file mode 100644
index 00000000000000..b9a9fc77166520
--- /dev/null
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_mpc.h
@@ -0,0 +1,210 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (c) 2026 Broadcom Inc. */
+
+#ifndef BNXT_MPC_H
+#define BNXT_MPC_H
+
+/* Mid path channel (MPC) definitions. An MPC is special TX/completion
+ * ring pair to send/receive control plane data to the TCE and RCE
+ * (Transmit/Receive Crypto Engine) HW blocks.
+ */
+
+enum bnxt_mpc_type {
+ BNXT_MPC_TCE_TYPE = RING_ALLOC_REQ_MPC_CHNLS_TYPE_TCE,
+ BNXT_MPC_RCE_TYPE = RING_ALLOC_REQ_MPC_CHNLS_TYPE_RCE,
+ BNXT_MPC_TYPE_MAX,
+};
+
+#define BNXT_MAX_MPC 8
+
+#define BNXT_MIN_MPC_TCE 1
+#define BNXT_MIN_MPC_RCE 1
+#define BNXT_DFLT_MPC_TCE BNXT_MAX_MPC
+#define BNXT_DFLT_MPC_RCE BNXT_MAX_MPC
+
+#define BNXT_MPC_TMO_MSECS 1000
+
+struct bnxt_mpc_info {
+ u8 mpc_chnls_cap;
+ u8 mpc_cp_rings;
+ u8 mpc_ring_count[BNXT_MPC_TYPE_MAX];
+ struct bnxt_tx_ring_info *mpc_rings[BNXT_MPC_TYPE_MAX];
+};
+
+struct bnxt_sw_mpc_tx_bd {
+ u8 inline_bds;
+ unsigned long handle;
+};
+
+#define SW_MPC_TXBD_RING_SIZE (sizeof(struct bnxt_sw_mpc_tx_bd) * TX_DESC_CNT)
+
+struct bnxt_cmpl_entry {
+ void *cmpl;
+ u32 len;
+};
+
+struct mpc_cmp {
+ __le32 mpc_cmp_client_subtype_type;
+ #define MPC_CMP_TYPE (0x3f << 0)
+ #define MPC_CMP_TYPE_MID_PATH_SHORT 0x1e
+ #define MPC_CMP_TYPE_MID_PATH_LONG 0x1f
+ #define MPC_CMP_SUBTYPE 0xf00
+ #define MPC_CMP_SUBTYPE_SFT 8
+ #define MPC_CMP_SUBTYPE_SOLICITED (0x0 << 8)
+ #define MPC_CMP_SUBTYPE_ERR (0x1 << 8)
+ #define MPC_CMP_SUBTYPE_RESYNC (0x2 << 8)
+ #define MPC_CMP_CLIENT (0xf << 12)
+ #define MPC_CMP_CLIENT_SFT 12
+ #define MPC_CMP_CLIENT_TCE (0x0 << 12)
+ #define MPC_CMP_CLIENT_RCE (0x1 << 12)
+ #define MPC_CMP_CLIENT_TE_CFA (0x2 << 12)
+ #define MPC_CMP_CLIENT_RE_CFA (0x3 << 12)
+ u32 mpc_cmp_opaque;
+ __le32 mpc_cmp_v;
+ #define MPC_CMP_V (1 << 0)
+ __le32 mpc_cmp_filler;
+};
+
+#define MPC_CMP_CMP_TYPE(mpcmp) \
+ (le32_to_cpu((mpcmp)->mpc_cmp_client_subtype_type) & MPC_CMP_TYPE)
+
+#define MPC_CMP_CLIENT_TYPE(mpcmp) \
+ (le32_to_cpu((mpcmp)->mpc_cmp_client_subtype_type) & MPC_CMP_CLIENT)
+
+#define MPC_CMP_UNSOLICIT_SUBTYPE(mpcmp) \
+ ((le32_to_cpu((mpcmp)->mpc_cmp_client_subtype_type) & \
+ MPC_CMP_SUBTYPE) != MPC_CMP_SUBTYPE_SOLICITED)
+
+#define MPC_CMP_VALID(bp, mpcmp, raw_cons) \
+ (!!((mpcmp)->mpc_cmp_v & cpu_to_le32(MPC_CMP_V)) == \
+ !((raw_cons) & (bp)->cp_bit))
+
+#define BNXT_MPC_CRYPTO_CAP \
+ (FUNC_QCAPS_RESP_MPC_CHNLS_CAP_TCE | FUNC_QCAPS_RESP_MPC_CHNLS_CAP_RCE)
+
+#define BNXT_MPC_CRYPTO_CAPABLE(bp) \
+ ((bp)->mpc_info ? \
+ ((bp)->mpc_info->mpc_chnls_cap & BNXT_MPC_CRYPTO_CAP) == \
+ BNXT_MPC_CRYPTO_CAP : false)
+
+#ifdef CONFIG_BNXT_TLS
+void bnxt_alloc_mpc_info(struct bnxt *bp, u8 mpc_chnls_cap);
+void bnxt_free_mpc_info(struct bnxt *bp);
+int bnxt_mpc_tx_rings_in_use(struct bnxt *bp);
+int bnxt_mpc_cp_rings_in_use(struct bnxt *bp);
+bool bnxt_napi_has_mpc(struct bnxt *bp, int i);
+void bnxt_set_mpc_cp_ring(struct bnxt *bp, int bnapi_idx,
+ struct bnxt_cp_ring_info *cpr);
+void bnxt_trim_mpc_rings(struct bnxt *bp);
+void bnxt_set_dflt_mpc_rings(struct bnxt *bp);
+void bnxt_init_mpc_ring_struct(struct bnxt *bp);
+int bnxt_alloc_mpcs(struct bnxt *bp);
+void bnxt_free_mpcs(struct bnxt *bp);
+int bnxt_alloc_mpc_rings(struct bnxt *bp);
+void bnxt_free_mpc_rings(struct bnxt *bp);
+void bnxt_init_mpc_rings(struct bnxt *bp);
+int bnxt_hwrm_mpc_ring_alloc(struct bnxt *bp);
+void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path);
+struct bnxt_tx_ring_info *bnxt_select_mpc_ring(struct bnxt *bp, int ring_type);
+int bnxt_start_xmit_mpc(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ void *data, unsigned int len, unsigned long handle);
+int bnxt_mpc_timeout(struct bnxt *bp, struct bnxt_tx_ring_info *txr);
+int bnxt_mpc_cmp(struct bnxt *bp, struct bnxt_cp_ring_info *cpr, u32 *raw_cons);
+#else
+static inline void bnxt_alloc_mpc_info(struct bnxt *bp, u8 mpc_chnls_cap)
+{
+}
+
+static inline void bnxt_free_mpc_info(struct bnxt *bp)
+{
+}
+
+static inline int bnxt_mpc_tx_rings_in_use(struct bnxt *bp)
+{
+ return 0;
+}
+
+static inline int bnxt_mpc_cp_rings_in_use(struct bnxt *bp)
+{
+ return 0;
+}
+
+static inline bool bnxt_napi_has_mpc(struct bnxt *bp, int i)
+{
+ return false;
+}
+
+static inline void bnxt_set_mpc_cp_ring(struct bnxt *bp, int bnapi_idx,
+ struct bnxt_cp_ring_info *cpr)
+{
+}
+
+static inline void bnxt_trim_mpc_rings(struct bnxt *bp)
+{
+}
+
+static inline void bnxt_set_dflt_mpc_rings(struct bnxt *bp)
+{
+}
+
+static inline void bnxt_init_mpc_ring_struct(struct bnxt *bp)
+{
+}
+
+static inline int bnxt_alloc_mpcs(struct bnxt *bp)
+{
+ return 0;
+}
+
+static inline void bnxt_free_mpcs(struct bnxt *bp)
+{
+}
+
+static inline int bnxt_alloc_mpc_rings(struct bnxt *bp)
+{
+ return 0;
+}
+
+static inline void bnxt_free_mpc_rings(struct bnxt *bp)
+{
+}
+
+static inline void bnxt_init_mpc_rings(struct bnxt *bp)
+{
+}
+
+static inline int bnxt_hwrm_mpc_ring_alloc(struct bnxt *bp)
+{
+ return 0;
+}
+
+static inline void bnxt_hwrm_mpc_ring_free(struct bnxt *bp, bool close_path)
+{
+}
+
+static inline struct bnxt_tx_ring_info *bnxt_select_mpc_ring(struct bnxt *bp,
+ int ring_type)
+{
+ return NULL;
+}
+
+static inline int bnxt_start_xmit_mpc(struct bnxt *bp,
+ struct bnxt_tx_ring_info *txr, void *data,
+ unsigned int len, unsigned long handle)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline int bnxt_mpc_timeout(struct bnxt *bp,
+ struct bnxt_tx_ring_info *txr)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline int bnxt_mpc_cmp(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,
+ u32 *raw_cons)
+{
+ return 0;
+}
+#endif /* CONFIG_BNXT_TLS */
+#endif /* BNXT_MPC_H */
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c
index edcc002e4ca358..d57059722f5bb0 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c
@@ -640,7 +640,7 @@ static int bnxt_hwrm_func_vf_resc_cfg(struct bnxt *bp, int num_vfs, bool reset)
vf_rx_rings = hw_resc->max_rx_rings - bp->rx_nr_rings * 2;
else
vf_rx_rings = hw_resc->max_rx_rings - bp->rx_nr_rings;
- vf_tx_rings = hw_resc->max_tx_rings - bp->tx_nr_rings;
+ vf_tx_rings = hw_resc->max_tx_rings - bnxt_total_tx_rings(bp);
vf_vnics = hw_resc->max_vnics - bp->nr_vnics;
vf_rss = hw_resc->max_rsscos_ctxs - bp->rsscos_nr_ctxs;
@@ -903,8 +903,8 @@ static int bnxt_sriov_enable(struct bnxt *bp, int *num_vfs)
avail_cp < min_rx_rings)
rx_ok = 0;
- if (hw_resc->max_tx_rings - bp->tx_nr_rings >= min_tx_rings &&
- avail_cp >= min_tx_rings)
+ if (hw_resc->max_tx_rings - bnxt_total_tx_rings(bp) >=
+ min_tx_rings && avail_cp >= min_tx_rings)
tx_ok = 1;
if (hw_resc->max_rsscos_ctxs - bp->rsscos_nr_ctxs >=
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
index 9e5009be8e9898..2a94a77847fe16 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
@@ -351,7 +351,7 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames,
return -EINVAL;
if (static_branch_unlikely(&bnxt_xdp_locking_key))
- spin_lock(&txr->xdp_tx_lock);
+ spin_lock(&txr->tx_lock);
for (i = 0; i < num_frames; i++) {
struct xdp_frame *xdp = frames[i];
@@ -376,7 +376,7 @@ int bnxt_xdp_xmit(struct net_device *dev, int num_frames,
}
if (static_branch_unlikely(&bnxt_xdp_locking_key))
- spin_unlock(&txr->xdp_tx_lock);
+ spin_unlock(&txr->tx_lock);
return nxmit;
}
diff --git a/include/linux/bnxt/hsi.h b/include/linux/bnxt/hsi.h
index 74a6bf278d888c..03444b81beb0f3 100644
--- a/include/linux/bnxt/hsi.h
+++ b/include/linux/bnxt/hsi.h
@@ -3837,6 +3837,43 @@ struct hwrm_func_ptp_ext_qcfg_output {
u8 valid;
};
+/* hwrm_func_key_ctx_alloc_input (size:384b/48B) */
+struct hwrm_func_key_ctx_alloc_input {
+ __le16 req_type;
+ __le16 cmpl_ring;
+ __le16 seq_id;
+ __le16 target_id;
+ __le64 resp_addr;
+ __le16 fid;
+ __le16 num_key_ctxs;
+ __le32 dma_bufr_size_bytes;
+ u8 key_ctx_type;
+ #define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_TX 0x0UL
+ #define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_RX 0x1UL
+ #define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_QUIC_TX 0x2UL
+ #define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_QUIC_RX 0x3UL
+ #define FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_LAST FUNC_KEY_CTX_ALLOC_REQ_KEY_CTX_TYPE_QUIC_RX
+ u8 unused_0[7];
+ __le64 host_dma_addr;
+ __le32 partition_start_xid;
+ u8 unused_1[4];
+};
+
+/* hwrm_func_key_ctx_alloc_output (size:192b/24B) */
+struct hwrm_func_key_ctx_alloc_output {
+ __le16 error_code;
+ __le16 req_type;
+ __le16 seq_id;
+ __le16 resp_len;
+ __le16 num_key_ctxs_allocated;
+ u8 flags;
+ #define FUNC_KEY_CTX_ALLOC_RESP_FLAGS_KEY_CTXS_CONTIGUOUS 0x1UL
+ u8 unused_0;
+ __le32 partition_start_xid;
+ u8 unused_1[7];
+ u8 valid;
+};
+
/* hwrm_func_backing_store_cfg_v2_input (size:512b/64B) */
struct hwrm_func_backing_store_cfg_v2_input {
__le16 req_type;
|
| 3/2 | 2026/06/29 20:22 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 55344 output: 91 thoughts: 1225 Thoughts: **Analysis of TLS Offload Patch for Broadcom NetXtreme-C/E Driver** Okay, so we have a patch that introduces TLS offload support to the `bnxt` driver. This involves adding and modifying a significant chunk of code within the Broadcom NetXtreme-C/E driver directory. Specifically, it seems like `BNXT_TLS` is being introduced via Kconfig and several new files, like `bnxt_mpc.c`, `bnxt_crypto.c`, `bnxt_ktls.c`, are added and some of the existing files like `bnxt.c`, `bnxt_ethtool.c`, `bnxt_gso.c`, `bnxt_sriov.c`, `bnxt_xdp.c` are modified. This all points to a hardware TLS offload implementation. The key question is, can this code actually *be* fuzzed? My initial thought is to consider a virtualized environment. The patch implements `tlsdev_ops` for the `bnxt` driver, which means `bnxt_ktls_init` sets the `dev->tlsdev_ops` and enables `NETIF_F_HW_TLS_TX`. This is explicitly hardware TLS offload. So, can this be triggered by something like Syzkaller? A quick check confirms that `bnxt` is tied to Broadcom NetXtreme-C/E hardware. And here's the problem: Syzkaller, nor QEMU for that matter, doesn't emulate that specific hardware. I need to make sure there's no way to reach this code through some generic PCI emulation. Nope, QEMU doesn't emulate Broadcom NetXtreme-C/E. So, the `bnxt_start_xmit` will never be called, as there is no emulated `bnxt` device. Double-checking, I can use `codesearch-dir-index` or `grepper` to confirm that there's no emulation of the Broadcom NetXtreme-C/E hardware. QEMU does emulate a bunch of other network devices, but not `bnxt`. To solidify this, I'll search the kernel source itself. It's a physical hardware driver. The prompt specifically asks to return `WorthFuzzing=false` if the patch involves code that can't be reached in a virtualized environment. Because the `bnxt` driver is dependent on specific Broadcom hardware with no emulation, it's definitively unreachable. Therefore, since the changes are contained within a hardware driver (`bnxt`), and it requires that specific hardware, it's impossible to reach this code during fuzzing in environments like QEMU or GCE. All the modifications are strictly within the `bnxt` driver, the Makefile, and associated headers. So, the answer is: `WorthFuzzing=false`. I'll confirm that and report this as the conclusion to be made, because `bnxt` depends on physical hardware. |
| 4/2 | 2026/06/29 20:22 | tool | set-results |
0mArgs:{
"EnableConfigs": [],
"FocusSymbols": [],
"Reasoning": "The patch exclusively modifies the Broadcom NetXtreme-C/E (bnxt) hardware driver to add TLS offload support. Since there is no software emulation for this specific hardware in virtualized environments like QEMU or GCE, the modified code paths are unreachable during standard fuzzing.",
"WorthFuzzing": false
}Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"Reasoning": "The patch exclusively modifies the Broadcom NetXtreme-C/E (bnxt) hardware driver to add TLS offload support. Since there is no software emulation for this specific hardware in virtualized environments like QEMU or GCE, the modified code paths are unreachable during standard fuzzing.",
"WorthFuzzing": false
} |
| Total Calls | Total Tokens | Avg Tokens | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|
| Total Calls | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|