| Seq | Timestamp | Type | Name | Duration |
|---|---|---|---|---|
| 0/0 | 2026/10/12 12:01 | flow | patch-triage |
0mArgs:null Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"KMSANReasoning": "",
"NeedsKMSAN": false,
"Reasoning": "The patch series introduces support for Yunsilicon XSC PCIe devices, including the core PCI driver, ethernet driver, and RDMA (InfiniBand) driver. These drivers bind strictly to proprietary physical ASIC/PCIe hardware (Vendor ID 0x1f67) and associated auxiliary devices that are not emulated in standard virtualized QEMU or cloud environments (GCE). Because the modified code is structurally unreachable without specialized physical hardware, it is not worth fuzzing.",
"WorthFuzzing": false
} |
| 1/1 | 2026/10/12 12:01 | action | read-patch-diff |
0mArgs:null Results: {
"PatchDiff": "commit 007a736559afeaa04643f0b8945e43f8484a5614\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Mon Oct 12 12:01:29 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/MAINTAINERS b/MAINTAINERS\nindex c2414447892c2..34902a130ea83 100644\n--- a/MAINTAINERS\n+++ b/MAINTAINERS\n@@ -30069,6 +30069,15 @@ S:\tMaintained\n F:\tDocumentation/input/devices/yealink.rst\n F:\tdrivers/input/misc/yealink.*\n \n+YUNSILICON XSC DRIVERS\n+M:\tHonggang Wei \u003cweihg@yunsilicon.com\u003e\n+M:\tXin Tian \u003ctianx@yunsilicon.com\u003e\n+L:\tnetdev@vger.kernel.org\n+L:\tlinux-rdma@vger.kernel.org\n+S:\tMaintained\n+F:\tdrivers/infiniband/hw/xsc\n+F:\tdrivers/net/ethernet/yunsilicon/xsc\n+\n ZD1211RW WIRELESS DRIVER\n L:\tlinux-wireless@vger.kernel.org\n S:\tOrphan\ndiff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig\nindex 086195758a8ac..b3cc506db57de 100644\n--- a/drivers/infiniband/Kconfig\n+++ b/drivers/infiniband/Kconfig\n@@ -100,6 +100,7 @@ source \"drivers/infiniband/hw/ocrdma/Kconfig\"\n source \"drivers/infiniband/hw/qedr/Kconfig\"\n source \"drivers/infiniband/hw/usnic/Kconfig\"\n source \"drivers/infiniband/hw/vmw_pvrdma/Kconfig\"\n+source \"drivers/infiniband/hw/xsc/Kconfig\"\n source \"drivers/infiniband/sw/rdmavt/Kconfig\"\n endif # !UML\n source \"drivers/infiniband/sw/rxe/Kconfig\"\ndiff --git a/drivers/infiniband/hw/Makefile b/drivers/infiniband/hw/Makefile\nindex c42b22ac3303d..1bcd2d50a76c0 100644\n--- a/drivers/infiniband/hw/Makefile\n+++ b/drivers/infiniband/hw/Makefile\n@@ -16,3 +16,4 @@ obj-$(CONFIG_INFINIBAND_BNXT_RE)\t+= bnxt_re/\n obj-$(CONFIG_INFINIBAND_BNG_RE)\t\t+= bng_re/\n obj-$(CONFIG_INFINIBAND_ERDMA)\t\t+= erdma/\n obj-$(CONFIG_INFINIBAND_IONIC)\t\t+= ionic/\n+obj-$(CONFIG_INFINIBAND_XSC)\t+= xsc/\ndiff --git a/drivers/infiniband/hw/xsc/Kconfig b/drivers/infiniband/hw/xsc/Kconfig\nnew file mode 100644\nindex 0000000000000..5cf46df2d4f12\n--- /dev/null\n+++ b/drivers/infiniband/hw/xsc/Kconfig\n@@ -0,0 +1,17 @@\n+# SPDX-License-Identifier: GPL-2.0\n+# Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+# All rights reserved.\n+\n+config INFINIBAND_XSC\n+\ttristate \"Yunsilicon XSC RDMA driver\"\n+\tdepends on INFINIBAND \u0026\u0026 PCI \u0026\u0026 INET\n+\tdepends on YUNSILICON_XSC_PCI \u0026\u0026 YUNSILICON_XSC_ETH\n+\thelp\n+\t This driver provides RDMA support for Yunsilicon XSC devices.\n+\t It uses the auxiliary device published by the XSC PCI core and,\n+\t for now, the netdev registered by the XSC ethernet driver to\n+\t derive the port state and GID. Because of that netdev\n+\t dependency it also depends on YUNSILICON_XSC_ETH.\n+\n+\t To compile this driver as a module, choose M here. The module\n+\t will be called xsc_ib.\ndiff --git a/drivers/infiniband/hw/xsc/Makefile b/drivers/infiniband/hw/xsc/Makefile\nnew file mode 100644\nindex 0000000000000..48a6d2b0aaace\n--- /dev/null\n+++ b/drivers/infiniband/hw/xsc/Makefile\n@@ -0,0 +1,10 @@\n+# SPDX-License-Identifier: GPL-2.0\n+# Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+# All rights reserved.\n+\n+ccflags-y := -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc\n+\n+obj-$(CONFIG_INFINIBAND_XSC) += xsc_ib.o\n+\n+xsc_ib-y := main.o cq.o qp.o mem.o mr.o ah.o counters.o \\\n+\t\txsc_ib_sysfs.o srq.o\ndiff --git a/drivers/infiniband/hw/xsc/ah.c b/drivers/infiniband/hw/xsc/ah.c\nnew file mode 100644\nindex 0000000000000..4d916dbe7dd25\n--- /dev/null\n+++ b/drivers/infiniband/hw/xsc/ah.c\n@@ -0,0 +1,144 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/kernel.h\u003e\n+#include \u003clinux/module.h\u003e\n+#include \u003clinux/random.h\u003e\n+#include \"xsc_ib.h\"\n+#include \u003crdma/xsc-abi.h\u003e\n+\n+static u32 xsc_calc_roce_udp_flow_label(void)\n+{\n+\t/* Generate a non-zero 20 bit flow label. */\n+\treturn get_random_u32_below(IB_GRH_FLOWLABEL_MASK) + 1;\n+}\n+\n+static u16 xsc_ah_get_udp_sport(const struct xsc_ib_dev *dev,\n+\t\t\t\tstruct rdma_ah_attr *ah_attr)\n+{\n+\tenum ib_gid_type gid_type = ah_attr-\u003egrh.sgid_attr-\u003egid_type;\n+\tu16 sport = 0;\n+\tu32 fl = 0;\n+\n+\tif (gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP \u0026\u0026\n+\t (rdma_ah_get_ah_flags(ah_attr) \u0026 IB_AH_GRH) \u0026\u0026\n+\t (ah_attr-\u003egrh.flow_label \u0026 IB_GRH_FLOWLABEL_MASK)) {\n+\t\tfl = ah_attr-\u003egrh.flow_label;\n+\t} else {\n+\t\t/*generate a 20bit flow_label and output to user layer*/\n+\t\tfl = xsc_calc_roce_udp_flow_label();\n+\t\tah_attr-\u003egrh.flow_label = fl;\n+\t}\n+\n+\tsport = rdma_flow_label_to_udp_sport(fl);\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"fl=0x%x,sport=0x%x\\n\", fl, sport);\n+\treturn sport;\n+}\n+\n+static struct ib_ah *create_ib_ah(struct xsc_ib_dev *dev,\n+\t\t\t\t struct xsc_ib_ah *ah,\n+\t\t\t\t struct rdma_ah_attr *ah_attr)\n+{\n+\tenum ib_gid_type gid_type;\n+\n+\tif (rdma_ah_get_ah_flags(ah_attr) \u0026 IB_AH_GRH) {\n+\t\tconst struct ib_global_route *grh = rdma_ah_read_grh(ah_attr);\n+\n+\t\tmemcpy(ah-\u003eav.rgid, \u0026grh-\u003edgid, 16);\n+\t\tah-\u003eav.grh_gid_fl = cpu_to_be32(grh-\u003eflow_label |\n+\t\t\t\t\t\t(1 \u003c\u003c 30) |\n+\t\t\t\t\t\tgrh-\u003esgid_index \u003c\u003c 20);\n+\t\tah-\u003eav.hop_limit = grh-\u003ehop_limit;\n+\t\tah-\u003eav.tclass = grh-\u003etraffic_class;\n+\t}\n+\n+\tah-\u003eav.stat_rate_sl = (rdma_ah_get_static_rate(ah_attr) \u003c\u003c 4);\n+\n+\tif (ah_attr-\u003etype == RDMA_AH_ATTR_TYPE_ROCE) {\n+\t\tgid_type = ah_attr-\u003egrh.sgid_attr-\u003egid_type;\n+\n+\t\tmemcpy(ah-\u003eav.rmac, ah_attr-\u003eroce.dmac,\n+\t\t sizeof(ah_attr-\u003eroce.dmac));\n+\n+\t\tah-\u003eav.udp_sport = xsc_ah_get_udp_sport(dev, ah_attr);\n+\t\tah-\u003eav.stat_rate_sl |= (rdma_ah_get_sl(ah_attr) \u0026 0x7) \u003c\u003c 1;\n+\t\tif (gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP)\n+#define XSC_ECN_ENABLED BIT(1)\n+\t\t\tah-\u003eav.tclass |= XSC_ECN_ENABLED;\n+\t} else {\n+\t\tah-\u003eav.rlid = cpu_to_be16(rdma_ah_get_dlid(ah_attr));\n+\t\tah-\u003eav.fl_mlid = rdma_ah_get_path_bits(ah_attr) \u0026 0x7f;\n+\t\tah-\u003eav.stat_rate_sl |= (rdma_ah_get_sl(ah_attr) \u0026 0xf);\n+\t}\n+\n+\treturn \u0026ah-\u003eibah;\n+}\n+\n+int xsc_ib_create_ah(struct ib_ah *ibah,\n+\t\t struct rdma_ah_init_attr *init_attr,\n+\t\t struct ib_udata *udata)\n+{\n+\tstruct xsc_ib_ah *ah = to_xah(ibah);\n+\tstruct xsc_ib_dev *dev = to_xdev(ibah-\u003edevice);\n+\tstruct rdma_ah_attr *ah_attr = init_attr-\u003eah_attr;\n+\tenum rdma_ah_attr_type ah_type = ah_attr-\u003etype;\n+\n+\tif (ah_type == RDMA_AH_ATTR_TYPE_ROCE \u0026\u0026\n+\t !(rdma_ah_get_ah_flags(ah_attr) \u0026 IB_AH_GRH))\n+\t\treturn -EINVAL;\n+\n+\tif (ah_type == RDMA_AH_ATTR_TYPE_ROCE \u0026\u0026 udata) {\n+\t\tint err;\n+\t\tstruct xsc_ib_create_ah_resp resp = {};\n+\t\tu32 min_resp_len = offsetof(typeof(resp), dmac) +\n+\t\t\t\t sizeof(resp.dmac);\n+\n+\t\tif (udata-\u003eoutlen \u003c min_resp_len)\n+\t\t\treturn -EINVAL;\n+\n+\t\tresp.response_length = min_resp_len;\n+\t\tmemcpy(resp.dmac, ah_attr-\u003eroce.dmac, ETH_ALEN);\n+\t\terr = ib_copy_to_udata(udata, \u0026resp, resp.response_length);\n+\t\tif (err)\n+\t\t\treturn err;\n+\t}\n+\n+\tcreate_ib_ah(dev, ah, ah_attr); /* never fails */\n+\treturn 0;\n+}\n+\n+int xsc_ib_query_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr)\n+{\n+\tstruct xsc_ib_ah *ah = to_xah(ibah);\n+\tu32 tmp;\n+\n+\tmemset(ah_attr, 0, sizeof(*ah_attr));\n+\tah_attr-\u003etype = ibah-\u003etype;\n+\n+\ttmp = be32_to_cpu(ah-\u003eav.grh_gid_fl);\n+\tif (tmp \u0026 (1 \u003c\u003c 30)) {\n+\t\trdma_ah_set_grh(ah_attr, NULL, tmp \u0026 IB_GRH_FLOWLABEL_MASK,\n+\t\t\t\t(tmp \u003e\u003e 20) \u0026 0xff, ah-\u003eav.hop_limit,\n+\t\t\t\tah-\u003eav.tclass);\n+\t\trdma_ah_set_dgid_raw(ah_attr, ah-\u003eav.rgid);\n+\t}\n+\n+\trdma_ah_set_static_rate(ah_attr, ah-\u003eav.stat_rate_sl \u003e\u003e 4);\n+\tif (ah_attr-\u003etype == RDMA_AH_ATTR_TYPE_ROCE) {\n+\t\trdma_ah_set_sl(ah_attr, (ah-\u003eav.stat_rate_sl \u003e\u003e 1) \u0026 0x7);\n+\t} else {\n+\t\trdma_ah_set_dlid(ah_attr, be16_to_cpu(ah-\u003eav.rlid));\n+\t\trdma_ah_set_path_bits(ah_attr, ah-\u003eav.fl_mlid \u0026 0x7f);\n+\t\trdma_ah_set_sl(ah_attr, ah-\u003eav.stat_rate_sl \u0026 0xf);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+int xsc_ib_destroy_ah(struct ib_ah *ibah, u32 flags)\n+{\n+\treturn 0;\n+}\ndiff --git a/drivers/infiniband/hw/xsc/counters.c b/drivers/infiniband/hw/xsc/counters.c\nnew file mode 100644\nindex 0000000000000..135df95954a75\n--- /dev/null\n+++ b/drivers/infiniband/hw/xsc/counters.c\n@@ -0,0 +1,422 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/sysfs.h\u003e\n+#include \u003clinux/types.h\u003e\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_hsi.h\"\n+#include \"common/driver.h\"\n+#include \"common/xsc_cmd.h\"\n+#include \"counters.h\"\n+#include \"xsc_ib.h\"\n+\n+#define COUNTERS_FILE_NAME \"counters\"\n+#define COUNTERS_NAMES_FILE_NAME \"counters_names\"\n+#define COUNTERS_VALUE_FILE_NAME \"counters_value\"\n+#define COUNTERS_ATTER_GROUP_NAME \"counters\"\n+#define GLOBAL_COUNTERS_GROUP_NAME \"global_counters\"\n+#define GLOBAL_COUNTERS_FILE_NAME \"counters\"\n+\n+static const struct counter_desc hw_rdma_stats_pf_desc[] = {\n+\t/*by mac port*/\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_tx_pkts) },//0\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_tx_bytes) },//1\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_rx_pkts) },//2\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_rx_bytes) },//3\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, np_cnp_sent) },//4\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rp_cnp_handled) },//5\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, np_ecn_marked_roce_packets) },//6\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rp_cnp_ignored) },//7\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, read_rsp_out_of_seq) },//8\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, implied_nak_seq_err) },//9\n+\t/*by function*/\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, out_of_sequence) },//10\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, packet_seq_err) },//11\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, out_of_buffer) },//12\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rnr_nak_retry_err) },//13\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, local_ack_timeout_err) },//14\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rx_read_requests) },//15\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rx_write_requests) },//16\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, duplicate_requests) },//17\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_tx_pkts_func) },//18\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_tx_payload_bytes) },//19\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_rx_pkts_func) },//20\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_rx_payload_bytes) },//21\n+\t/*global*/\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_loopback_pkts) },//22\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_loopback_bytes) },//23\n+\t/*for diamond*/\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, out_of_sequence_sr) },//24\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, packet_seq_err_sr) },//25\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_ndp_rx_pkts) },//26\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_ndp_rx_trimmed_pkts) },//27\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_ndp_trimmed_pkts_sr) },//28\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rx_icrc_encapsulated) },//29\n+\t/*XSC_CMDQ_GET_HW_STATS_RDMA_V1*/\n+};\n+\n+static const struct counter_desc hw_rdma_stats_vf_desc[] = {\n+\t/*by function*/\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, rdma_tx_pkts_func) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, rdma_tx_payload_bytes) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, rdma_rx_pkts_func) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, rdma_rx_payload_bytes) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, out_of_sequence) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, packet_seq_err) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, out_of_buffer) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, rnr_nak_retry_err) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, local_ack_timeout_err) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, rx_read_requests) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, rx_write_requests) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, duplicate_requests) },\n+};\n+\n+static const struct counter_desc hw_global_rdma_stats_desc[] = {\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_global_stats_rdma, rdma_loopback_pkts) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_global_stats_rdma, rdma_loopback_bytes) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_global_stats_rdma, rx_icrc_encapsulated) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_global_stats_rdma, req_cqe_error) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_global_stats_rdma, resp_cqe_error) },\n+\t{ XSC_DECLARE_STAT(struct xsc_hw_global_stats_rdma, cqe_msg_code_error) },\n+};\n+\n+static int get_hw_stats_rdma(struct xsc_core_device *dev, struct xsc_hw_stats_rdma *stats_rdma)\n+{\n+\tint ret;\n+\tstruct xsc_hw_stats_mbox_in in;\n+\tstruct xsc_hw_stats_rdma_mbox_out out;\n+\n+\tmemset(stats_rdma, 0, sizeof(*stats_rdma));\n+\n+\tif (!dev)\n+\t\treturn -1;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(\u0026out, 0, sizeof(out));\n+\tin.mac_port = dev-\u003emac_port;\n+\tin.hdr.ver = cpu_to_be16(XSC_CMDQ_GET_HW_STATS_RDMA_V1);\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_HW_STATS_RDMA);\n+\n+\tret = xsc_cmd_exec(dev, (void *)\u0026in, sizeof(in), (void *)\u0026out, sizeof(out));\n+\tif (ret || out.hdr.status)\n+\t\treturn -1;\n+\n+\tmemcpy(stats_rdma, \u0026out.hw_stats, sizeof(*stats_rdma));\n+\n+\treturn 0;\n+}\n+\n+static ssize_t counters_names_show(struct device *device,\n+\t\t\t\t struct device_attribute *attr, char *buf)\n+{\n+\tstruct xsc_ib_dev *ibdev = container_of(device, struct xsc_ib_dev, ib_dev.dev);\n+\tstruct xsc_core_device *xdev = ibdev-\u003exdev;\n+\tstruct xsc_hw_stats_rdma stats_rdma;\n+\tconst struct counter_desc *desc;\n+\tint desc_size;\n+\tssize_t count = 0;\n+\tu64 mask;\n+\tint i;\n+\n+\tif (get_hw_stats_rdma(xdev, \u0026stats_rdma))\n+\t\treturn 0;\n+\n+\tif (is_support_hw_pf_stats(xdev)) {\n+\t\tdesc = hw_rdma_stats_pf_desc;\n+\t\tdesc_size = ARRAY_SIZE(hw_rdma_stats_pf_desc);\n+\t} else {\n+\t\tdesc = hw_rdma_stats_vf_desc;\n+\t\tdesc_size = ARRAY_SIZE(hw_rdma_stats_vf_desc);\n+\t}\n+\n+\tmask = xsc_get_rdma_stat_mask(xdev, be16_to_cpu(stats_rdma.ver));\n+\tfor (i = 0; i \u003c desc_size; i++) {\n+\t\tif (!((1ULL \u003c\u003c i) \u0026 mask))\n+\t\t\tcontinue;\n+\t\tcount += sysfs_emit_at(buf, count, \"%s\\n\", desc[i].format);\n+\t}\n+\n+\treturn count;\n+}\n+static DEVICE_ATTR_RO(counters_names);\n+\n+static ssize_t counters_show(struct device *device,\n+\t\t\t struct device_attribute *attr, char *buf)\n+{\n+\tstruct xsc_ib_dev *ibdev = container_of(device, struct xsc_ib_dev, ib_dev.dev);\n+\tstruct xsc_core_device *xdev = ibdev-\u003exdev;\n+\tstruct xsc_hw_stats_rdma stats_rdma;\n+\tconst struct counter_desc *desc;\n+\tint desc_size;\n+\tssize_t count = 0;\n+\tu8 *stats;\n+\tu64 mask;\n+\tint i;\n+\n+\tif (get_hw_stats_rdma(xdev, \u0026stats_rdma) ||\n+\t is_support_hw_pf_stats(xdev) != stats_rdma.is_pf)\n+\t\treturn 0;\n+\n+\tif (is_support_hw_pf_stats(xdev)) {\n+\t\tdesc = hw_rdma_stats_pf_desc;\n+\t\tdesc_size = ARRAY_SIZE(hw_rdma_stats_pf_desc);\n+\t\tstats = (u8 *)\u0026stats_rdma.stats.pf_stats;\n+\t} else {\n+\t\tdesc = hw_rdma_stats_vf_desc;\n+\t\tdesc_size = ARRAY_SIZE(hw_rdma_stats_vf_desc);\n+\t\tstats = (u8 *)\u0026stats_rdma.stats.vf_stats;\n+\t}\n+\n+\tmask = xsc_get_rdma_stat_mask(xdev, be16_to_cpu(stats_rdma.ver));\n+\tfor (i = 0; i \u003c desc_size; i++) {\n+\t\tu64 value;\n+\n+\t\tif (!((1ULL \u003c\u003c i) \u0026 mask))\n+\t\t\tcontinue;\n+\t\tvalue = be64_to_cpu(*(u64 *)(stats + desc[i].offset));\n+\t\tcount += sysfs_emit_at(buf, count, \"%-26s %-20llu\\n\",\n+\t\t\t\t desc[i].format, value);\n+\t}\n+\n+\treturn count;\n+}\n+static DEVICE_ATTR_RO(counters);\n+\n+static ssize_t counters_value_read(struct file *file, struct kobject *kob,\n+\t\t\t\t const struct bin_attribute *bin_attr,\n+\t\t\t\t char *buf, loff_t loff, size_t size)\n+{\n+\tstruct xsc_ib_dev *ibdev = container_of(kob, struct xsc_ib_dev, ib_dev.dev.kobj);\n+\tstruct xsc_core_device *xdev = ibdev-\u003exdev;\n+\tstruct xsc_hw_stats_rdma stats_rdma;\n+\tconst struct counter_desc *desc;\n+\tint desc_size;\n+\tint bin_size;\n+\tu64 *tmp_value;\n+\tu8 *stats;\n+\tu64 mask;\n+\tint i, j;\n+\n+\tif (bin_attr-\u003esize \u003e size || bin_attr-\u003esize == 0)\n+\t\treturn 0;\n+\n+\tif (get_hw_stats_rdma(xdev, \u0026stats_rdma) ||\n+\t is_support_hw_pf_stats(xdev) != stats_rdma.is_pf)\n+\t\treturn 0;\n+\n+\tif (is_support_hw_pf_stats(xdev)) {\n+\t\tdesc = hw_rdma_stats_pf_desc;\n+\t\tdesc_size = ARRAY_SIZE(hw_rdma_stats_pf_desc);\n+\t\tstats = (u8 *)\u0026stats_rdma.stats.pf_stats;\n+\t} else {\n+\t\tdesc = hw_rdma_stats_vf_desc;\n+\t\tdesc_size = ARRAY_SIZE(hw_rdma_stats_vf_desc);\n+\t\tstats = (u8 *)\u0026stats_rdma.stats.vf_stats;\n+\t}\n+\n+\tbin_size = desc_size * sizeof(u64);\n+\tif (bin_attr-\u003esize \u003c bin_size)\n+\t\treturn 0;\n+\n+\ttmp_value = kzalloc(bin_attr-\u003esize, GFP_KERNEL);\n+\tif (!tmp_value)\n+\t\treturn 0;\n+\n+\tmask = xsc_get_rdma_stat_mask(xdev, be16_to_cpu(stats_rdma.ver));\n+\tj = 0;\n+\tfor (i = 0; i \u003c desc_size; i++) {\n+\t\tif (!((1ULL \u003c\u003c i) \u0026 mask))\n+\t\t\tcontinue;\n+\t\ttmp_value[j] = be64_to_cpu(*(u64 *)(stats + desc[i].offset));\n+\t\tj++;\n+\t}\n+\n+\tmemcpy(buf, tmp_value, bin_attr-\u003esize);\n+\n+\tkfree(tmp_value);\n+\treturn bin_attr-\u003esize;\n+}\n+static BIN_ATTR_RO(counters_value, sizeof(struct xsc_hw_stats_rdma));\n+\n+static struct attribute *counters_attrs[] = {\n+\t\u0026dev_attr_counters_names.attr,\n+\t\u0026dev_attr_counters.attr,\n+\tNULL,\n+};\n+\n+static const struct bin_attribute *const counters_bin_attrs[] = {\n+\t\u0026bin_attr_counters_value,\n+\tNULL,\n+};\n+\n+static const struct attribute_group counters_group = {\n+\t.name = \"counters\",\n+\t.attrs = counters_attrs,\n+\t.bin_attrs = counters_bin_attrs,\n+};\n+\n+static int counters_sysfs_init(struct ib_device *ib_dev)\n+{\n+\treturn sysfs_create_group(\u0026ib_dev-\u003edev.kobj, \u0026counters_group);\n+}\n+\n+static void counters_sysfs_fini(struct ib_device *ib_dev)\n+{\n+\tsysfs_remove_group(\u0026ib_dev-\u003edev.kobj, \u0026counters_group);\n+}\n+\n+static ssize_t global_cnt_attr_show(struct kobject *kobj,\n+\t\t\t\t struct attribute *attr, char *buf)\n+{\n+\tstruct xsc_global_cnt_attributes *a =\n+\t\tcontainer_of(attr, struct xsc_global_cnt_attributes, attr);\n+\tstruct xsc_global_cnt_interface *g =\n+\t\tcontainer_of(kobj, struct xsc_global_cnt_interface, kobj);\n+\n+\tif (!a-\u003eshow)\n+\t\treturn -EIO;\n+\n+\treturn a-\u003eshow(g, a, buf);\n+}\n+\n+static ssize_t global_cnt_attr_store(struct kobject *kobj,\n+\t\t\t\t struct attribute *attr,\n+\t\t\t\t const char *buf, size_t size)\n+{\n+\tstruct xsc_global_cnt_attributes *a =\n+\t\tcontainer_of(attr, struct xsc_global_cnt_attributes, attr);\n+\tstruct xsc_global_cnt_interface *g =\n+\t\tcontainer_of(kobj, struct xsc_global_cnt_interface, kobj);\n+\n+\tif (!a-\u003estore)\n+\t\treturn -EIO;\n+\n+\treturn a-\u003estore(g, a, buf, size);\n+}\n+\n+static ssize_t global_counters_show(struct xsc_global_cnt_interface *g,\n+\t\t\t\t struct xsc_global_cnt_attributes *a, char *buf)\n+{\n+\tint i;\n+\tint ret;\n+\tu8 *stats;\n+\tu64 value;\n+\tint desc_size;\n+\tssize_t count = 0;\n+\tconst struct counter_desc *desc;\n+\tstruct xsc_hw_global_stats_mbox_in in;\n+\tstruct xsc_hw_global_stats_mbox_out out;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(\u0026out, 0, sizeof(out));\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_HW_GLOBAL_STATS);\n+\tret = xsc_cmd_exec(g-\u003exdev, (void *)\u0026in, sizeof(in),\n+\t\t\t (void *)\u0026out, sizeof(out));\n+\tif (ret || out.hdr.status)\n+\t\treturn 0;\n+\n+\tdesc = \u0026hw_global_rdma_stats_desc[0];\n+\tdesc_size = ARRAY_SIZE(hw_global_rdma_stats_desc);\n+\tstats = (u8 *)\u0026out.hw_stats;\n+\n+\tfor (i = 0 ; i \u003c desc_size; i++) {\n+\t\tvalue = *(u64 *)(stats + desc[i].offset);\n+\t\tvalue = be64_to_cpu(value);\n+\t\tcount += sprintf(\u0026buf[count], \"%-26s %-20llu\\n\",\n+\t\t\t\tdesc[i].format, value);\n+\t}\n+\n+\treturn count;\n+}\n+\n+static ssize_t global_counters_store(struct xsc_global_cnt_interface *g,\n+\t\t\t\t struct xsc_global_cnt_attributes *a,\n+\t\t\t\t const char *buf, size_t count)\n+{\n+\treturn -EOPNOTSUPP;\n+}\n+\n+#define GLOBAL_CNT_ATTR(_name) struct xsc_global_cnt_attributes xsc_global_cnt_attr_##_name = \\\n+\t__ATTR(_name, 0444, global_##_name##_show, global_##_name##_store)\n+\n+GLOBAL_CNT_ATTR(counters);\n+\n+static const struct sysfs_ops global_cnt_sysfs_ops = {\n+\t.show = global_cnt_attr_show,\n+\t.store = global_cnt_attr_store,\n+};\n+\n+static struct attribute *global_cnt_attrs[] = {\n+\t\u0026xsc_global_cnt_attr_counters.attr,\n+\tNULL\n+};\n+\n+ATTRIBUTE_GROUPS(global_cnt);\n+\n+static const struct kobj_type global_cnt_ktype = {\n+\t.sysfs_ops = \u0026global_cnt_sysfs_ops,\n+\t.default_groups = global_cnt_groups,\n+};\n+\n+static struct xsc_global_cnt_interface *g_global_cnt_interface;\n+\n+static int global_cnt_sysfs_init(struct ib_device *ib_dev, struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_global_cnt_interface *tmp;\n+\tint err;\n+\n+\tif (!xdev || !xsc_core_is_pf(xdev) || xdev-\u003epf_id != 0)\n+\t\treturn 0;\n+\n+\ttmp = kzalloc_obj(*tmp, GFP_KERNEL);\n+\tif (!tmp)\n+\t\treturn -ENOMEM;\n+\n+\terr = kobject_init_and_add(\u0026tmp-\u003ekobj, \u0026global_cnt_ktype,\n+\t\t\t\t \u0026ib_dev-\u003edev.kobj, GLOBAL_COUNTERS_GROUP_NAME);\n+\tif (err)\n+\t\tgoto error_return;\n+\n+\tg_global_cnt_interface = tmp;\n+\ttmp-\u003exdev = xdev;\n+\treturn 0;\n+\n+error_return:\n+\tkobject_put(\u0026tmp-\u003ekobj);\n+\tkfree(tmp);\n+\treturn err;\n+}\n+\n+static void global_cnt_sysfs_fini(struct xsc_core_device *xdev)\n+{\n+\tif (!g_global_cnt_interface || !xdev || !xsc_core_is_pf(xdev) || xdev-\u003epf_id != 0)\n+\t\treturn;\n+\n+\tkobject_put(\u0026g_global_cnt_interface-\u003ekobj);\n+\tkfree(g_global_cnt_interface);\n+\tg_global_cnt_interface = NULL;\n+}\n+\n+int xsc_counters_init(struct ib_device *ib_dev, struct xsc_core_device *xdev)\n+{\n+\tint ret;\n+\n+\tret = counters_sysfs_init(ib_dev);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tret = global_cnt_sysfs_init(ib_dev, xdev);\n+\tif (ret)\n+\t\tcounters_sysfs_fini(ib_dev);\n+\n+\treturn ret;\n+}\n+\n+void xsc_counters_fini(struct ib_device *ib_dev, struct xsc_core_device *xdev)\n+{\n+\tcounters_sysfs_fini(ib_dev);\n+\tglobal_cnt_sysfs_fini(xdev);\n+}\ndiff --git a/drivers/infiniband/hw/xsc/counters.h b/drivers/infiniband/hw/xsc/counters.h\nnew file mode 100644\nindex 0000000000000..944dbafb62bc9\n--- /dev/null\n+++ b/drivers/infiniband/hw/xsc/counters.h\n@@ -0,0 +1,31 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/*\n+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __COUNTERS_H__\n+#define __COUNTERS_H__\n+\n+#define STRING_LEN\t\t32\n+#define XSC_DECLARE_STAT(type, fld)\t\"\"#fld, offsetof(type, fld)\n+\n+struct counter_desc {\n+\tchar\t\tformat[STRING_LEN];\n+\tsize_t\t\toffset; /* Byte offset */\n+};\n+\n+struct xsc_global_cnt_interface {\n+\tstruct xsc_core_device *xdev;\n+\tstruct kobject\t\tkobj;\n+};\n+\n+struct xsc_global_cnt_attributes {\n+\tstruct attribute attr;\n+\tssize_t (*show)(struct xsc_global_cnt_interface *g, struct xsc_global_cnt_attributes *a,\n+\t\t\tchar *buf);\n+\tssize_t (*store)(struct xsc_global_cnt_interface *g, struct xsc_global_cnt_attributes *a,\n+\t\t\t const char *buf, size_t count);\n+};\n+\n+#endif\ndiff --git a/drivers/infiniband/hw/xsc/cq.c b/drivers/infiniband/hw/xsc/cq.c\nnew file mode 100644\nindex 0000000000000..04820bf3a7fa4\n--- /dev/null\n+++ b/drivers/infiniband/hw/xsc/cq.c\n@@ -0,0 +1,895 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/kref.h\u003e\n+#include \u003crdma/ib_umem.h\u003e\n+#include \"xsc_ib.h\"\n+#include \u003crdma/xsc-abi.h\u003e\n+#include \"common/xsc_hsi.h\"\n+#include \u003clinux/dma-direct.h\u003e\n+\n+#ifndef IB_UVERBS_CQ_FLAGS_IGNORE_OVERRUN\n+#define IB_UVERBS_CQ_FLAGS_IGNORE_OVERRUN BIT(1)\n+#endif\n+\n+enum {\n+\tCQ_OK\t\t\t\t\t= 0,\n+\tCQ_EMPTY\t\t\t\t= -1,\n+\tCQ_POLL_ERR\t\t\t\t= -2\n+};\n+\n+enum {\n+\tXSC_CQE_APP_TAG_MATCHING = 1,\n+};\n+\n+enum {\n+\tXSC_CQE_APP_OP_TM_CONSUMED = 0x1,\n+\tXSC_CQE_APP_OP_TM_EXPECTED = 0x2,\n+\tXSC_CQE_APP_OP_TM_UNEXPECTED = 0x3,\n+\tXSC_CQE_APP_OP_TM_NO_TAG = 0x4,\n+\tXSC_CQE_APP_OP_TM_APPEND = 0x5,\n+\tXSC_CQE_APP_OP_TM_REMOVE = 0x6,\n+\tXSC_CQE_APP_OP_TM_NOOP = 0x7,\n+\tXSC_CQE_APP_OP_TM_CONSUMED_SW_RDNV = 0x9,\n+\tXSC_CQE_APP_OP_TM_CONSUMED_MSG = 0xA,\n+\tXSC_CQE_APP_OP_TM_CONSUMED_MSG_SW_RDNV = 0xB,\n+\tXSC_CQE_APP_OP_TM_MSG_COMPLETION_CANCELED = 0xC,\n+};\n+\n+static const u32 xsc_cqe_opcode[] = {\n+\t[XSC_OPCODE_RDMA_REQ_SEND]\t\t= IB_WC_SEND,\n+\t[XSC_OPCODE_RDMA_REQ_SEND_IMMDT]\t= IB_WC_SEND,\n+\t[XSC_OPCODE_RDMA_RSP_RECV]\t\t= IB_WC_RECV,\n+\t[XSC_OPCODE_RDMA_RSP_RECV_IMMDT]\t= IB_WC_RECV,\n+\t[XSC_OPCODE_RDMA_REQ_WRITE]\t\t= IB_WC_RDMA_WRITE,\n+\t[XSC_OPCODE_RDMA_REQ_WRITE_IMMDT]\t= IB_WC_RDMA_WRITE,\n+\t[XSC_OPCODE_RDMA_RSP_WRITE_IMMDT]\t= IB_WC_RECV_RDMA_WITH_IMM,\n+\t[XSC_OPCODE_RDMA_REQ_READ]\t\t= IB_WC_RDMA_READ,\n+\t[XSC_OPCODE_RDMA_MAD_REQ_SEND]\t\t= IB_WC_SEND,\n+\t[XSC_OPCODE_RDMA_MAD_RSP_RECV]\t\t= IB_WC_RECV,\n+};\n+\n+static void xsc_ib_cq_comp(struct xsc_core_cq *cq)\n+{\n+\tstruct ib_cq *ibcq = \u0026core_to_xcq(cq)-\u003eibcq;\n+\n+\tibcq-\u003ecomp_handler(ibcq, ibcq-\u003ecq_context);\n+}\n+\n+static void xsc_ib_cq_event(struct xsc_core_cq *xcq, enum xsc_event type)\n+{\n+\tstruct xsc_ib_cq *cq = container_of(xcq, struct xsc_ib_cq, xcq);\n+\tstruct xsc_ib_dev *dev = to_xdev(cq-\u003eibcq.device);\n+\tstruct ib_cq *ibcq = \u0026cq-\u003eibcq;\n+\tstruct ib_event event;\n+\n+\tif (type != XSC_EVENT_TYPE_CQ_ERROR) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"Unexpected event type %d on CQ %06x\\n\",\n+\t\t\t type, xcq-\u003ecqn);\n+\t\treturn;\n+\t}\n+\n+\tif (ibcq-\u003eevent_handler) {\n+\t\tevent.device = \u0026dev-\u003eib_dev;\n+\t\tevent.event = IB_EVENT_CQ_ERR;\n+\t\tevent.element.cq = ibcq;\n+\t\tibcq-\u003eevent_handler(\u0026event, ibcq-\u003ecq_context);\n+\t\tpr_info(\"xsc_ib: xsc async event 0x%x, ib event %d\\n\",\n+\t\t\tXSC_EVENT_TYPE_CQ_ERROR, IB_EVENT_CQ_ERR);\n+\t}\n+}\n+\n+static void *get_cqe_from_buf(struct xsc_ib_cq_buf *buf, int n, int size)\n+{\n+\treturn xsc_buf_offset(\u0026buf-\u003ebuf, n * size);\n+}\n+\n+static void *get_cqe(struct xsc_ib_cq *cq, int n)\n+{\n+\treturn get_cqe_from_buf(\u0026cq-\u003ebuf, n, cq-\u003excq.cqe_sz);\n+}\n+\n+static void *get_sw_cqe(struct xsc_ib_cq *cq, int n)\n+{\n+\tstruct xsc_cqe *cqe;\n+\n+\tcqe = (struct xsc_cqe *)get_cqe(cq, n \u0026 (cq-\u003eibcq.cqe - 1));\n+\n+\treturn ((cqe-\u003eowner \u0026 XSC_CQE_OWNER_MASK) ^\n+\t\t!!(n \u0026 cq-\u003eibcq.cqe)) ? NULL : cqe;\n+}\n+\n+static inline void handle_good_req(struct ib_wc *wc,\n+\t\t\t\t struct xsc_cqe *cqe,\n+\t\t\t\t u8 opcode)\n+{\n+\twc-\u003eopcode = xsc_cqe_opcode[opcode];\n+\tif (opcode == XSC_OPCODE_RDMA_REQ_READ)\n+\t\twc-\u003ebyte_len = RD_LE_32(cqe-\u003emsg_len);\n+\twc-\u003estatus = IB_WC_SUCCESS;\n+}\n+\n+static void handle_responder(struct ib_wc *wc, struct xsc_cqe *cqe,\n+\t\t\t struct xsc_ib_qp *qp, u8 opcode)\n+{\n+\tstruct xsc_ib_wq *wq = \u0026qp-\u003erq;\n+\tu16 idx;\n+\n+\twc-\u003ebyte_len = RD_LE_32(cqe-\u003emsg_len);\n+\twc-\u003eopcode = xsc_cqe_opcode[opcode];\n+\twc-\u003estatus = IB_WC_SUCCESS;\n+\n+\tidx = wq-\u003etail \u0026 (wq-\u003ewqe_cnt - 1);\n+\twc-\u003ewr_id = wq-\u003ewrid[idx];\n+\tatomic_dec(\u0026wq-\u003eflush_wqe_cnt);\n+\t++wq-\u003etail;\n+}\n+\n+static void *get_wqe(struct xsc_ib_qp *qp, int offset)\n+{\n+\treturn xsc_buf_offset(\u0026qp-\u003ebuf, offset);\n+}\n+\n+static void *get_recv_wqe(struct xsc_ib_qp *qp, int n)\n+{\n+\treturn get_wqe(qp, qp-\u003erq.offset + (n \u003c\u003c qp-\u003erq.wqe_shift));\n+}\n+\n+static void *get_seg_wqe(void *first, int n)\n+{\n+\treturn first + (n \u003c\u003c XSC_BASE_WQE_SHIFT);\n+}\n+\n+static void xsc_handle_rdma_mad_resp_recv(struct xsc_ib_cq *cq,\n+\t\t\t\t\t struct xsc_ib_qp **cur_qp,\n+\t\t\t\t\t struct ib_wc *wc,\n+\t\t\t\t\t struct xsc_cqe *cqe,\n+\t\t\t\t\t u8 opcode)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(cq-\u003eibcq.device);\n+\tvoid *recv;\n+\tstruct xsc_wqe_data_seg *data_seg;\n+\tstruct iphdr *ip4h = NULL;\n+\tstruct ipv6hdr *ip6h;\n+\tstruct udphdr *udph;\n+\tstruct ib_unpacked_eth *eth;\n+\tstruct ib_unpacked_vlan *vlan;\n+\tstruct ib_grh *grh;\n+\tstruct ib_mad *mad;\n+\tstruct rxe_bth *bth;\n+\tstruct rxe_deth *deth;\n+\tunsigned int pading_sz = 0;\n+\tstruct xsc_ib_wq *wq;\n+\tint idx;\n+\tu16 eth_type;\n+\tvoid *l3_start;\n+\n+\twq = \u0026(*cur_qp)-\u003erq;\n+\tidx = wq-\u003etail \u0026 (wq-\u003ewqe_cnt - 1);\n+\n+\thandle_responder(wc, cqe, *cur_qp, opcode);\n+\n+\tdata_seg = get_seg_wqe(get_recv_wqe(*cur_qp, idx), 0);\n+\trecv = xsc_ib_recv_mad_sg_virt_addr(\u0026dev-\u003eib_dev, wc, data_seg-\u003eva);\n+\n+\teth = (struct ib_unpacked_eth *)recv;\n+\tgrh = (struct ib_grh *)recv;\n+\tif (eth-\u003etype == htons(ETH_P_8021Q)) {\n+\t\tvlan = (struct ib_unpacked_vlan *)(eth + 1);\n+\t\teth_type = ntohs(vlan-\u003etype);\n+\t\tl3_start = vlan + 1;\n+\n+\t\twc-\u003evlan_id = ntohs(vlan-\u003etag) \u0026 0x0fff;\n+\t\twc-\u003esl = (ntohs(vlan-\u003etag) \u003e\u003e 13) \u0026 0x7;\n+\t\twc-\u003ewc_flags |= IB_WC_WITH_VLAN;\n+\t} else {\n+\t\teth_type = ntohs(eth-\u003etype);\n+\t\tl3_start = eth + 1;\n+\t}\n+\n+\tif (eth_type == ETH_P_IP) {\n+\t\tip4h = (struct iphdr *)l3_start;\n+\t\tudph = (struct udphdr *)(ip4h + 1);\n+\t} else {\n+\t\tip6h = (struct ipv6hdr *)l3_start;\n+\t\tudph = (struct udphdr *)(ip6h + 1);\n+\t}\n+\tbth = (struct rxe_bth *)(udph + 1);\n+\tdeth = (struct rxe_deth *)(bth + 1);\n+\tmad = (struct ib_mad *)(deth + 1);\n+\n+\tif (eth_type == ETH_P_IP) {\n+\t\tpading_sz = sizeof(*grh) - sizeof(*ip4h);\n+\t\tmemmove((u8 *)(grh + 1) - sizeof(*ip4h), ip4h, sizeof(*ip4h));\n+\t\tmemset(grh, 0, pading_sz);\n+\t} else {\n+\t\tmemmove(grh, ip6h, sizeof(*ip6h));\n+\t}\n+\tmemmove(grh + 1, mad, sizeof(*mad));\n+\n+\twc-\u003ewc_flags |= IB_WC_GRH;\n+\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"recv cqe idx:%u, len:%u\\n\", wq-\u003etail, wc-\u003ebyte_len);\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"qp[%d] recv MAD packet, msg_len=%d\\n\",\n+\t\t (*cur_qp)-\u003exqp.qpn, wc-\u003ebyte_len);\n+\twc-\u003estatus = IB_WC_SUCCESS;\n+}\n+\n+static void xsc_generate_lost_cqe(struct xsc_ib_dev *dev, struct xsc_ib_wq *wq,\n+\t\t\t\t int qpn, int idx, struct ib_wc *wc)\n+{\n+\tint lost_cqe_idx = wq-\u003eprev_ce_wqe[idx];\n+\n+\tibdev_err(\u0026dev-\u003eib_dev, \"detected lost cqe, cur wqe idx of qp%d is %d\\n\", qpn, idx);\n+\n+\twhile (wq-\u003eprev_ce_wqe[lost_cqe_idx] != wq-\u003elast_comp_ce_wqe)\n+\t\tlost_cqe_idx = wq-\u003eprev_ce_wqe[lost_cqe_idx];\n+\tibdev_err(\u0026dev-\u003eib_dev, \"generate lost cqe idx of qp%d is %d\\n\", qpn, lost_cqe_idx);\n+\n+\twq-\u003elast_comp_ce_wqe = lost_cqe_idx;\n+\twc-\u003ewr_id = wq-\u003ewrid[lost_cqe_idx];\n+\twq-\u003etail = wq-\u003ewqe_head[lost_cqe_idx] + 1;\n+\tatomic_dec(\u0026wq-\u003eflush_wqe_cnt);\n+\twq-\u003eneed_flush[lost_cqe_idx] = 0;\n+\twc-\u003estatus = IB_WC_GENERAL_ERR;\n+}\n+\n+static int xsc_poll_one(struct xsc_ib_cq *cq,\n+\t\t\tstruct xsc_ib_qp **cur_qp,\n+\t\t\tstruct ib_wc *wc)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(cq-\u003eibcq.device);\n+\tstruct xsc_core_qp *xqp;\n+\tstruct xsc_ib_wq *wq;\n+\tu8 opcode;\n+\tu32 qpn;\n+\tint idx;\n+\tstruct xsc_cqe *cqe;\n+\tu32 *p = NULL;\n+\tstruct xsc_ib_srq *xsrq;\n+\n+\tcqe = get_sw_cqe(cq, cq-\u003excq.cons_index);\n+\tif (!cqe)\n+\t\treturn -EAGAIN;\n+\n+\t++cq-\u003excq.cons_index;\n+\n+\t/* Make sure we read CQ entry contents after we've checked the\n+\t * ownership bit.\n+\t */\n+\trmb();\n+\n+\tp = (u32 *)cqe;\n+\n+\tqpn = cqe-\u003eqp_id;\n+\tqpn = le32_to_cpu(qpn);\n+\tif (!*cur_qp || (qpn != (*cur_qp)-\u003eibqp.qp_num)) {\n+\t\t/* We do not have to take the QP table lock here,\n+\t\t * because CQs will be locked while QPs are removed\n+\t\t * from the table.\n+\t\t */\n+\t\txqp = __xsc_qp_lookup(dev-\u003exdev, qpn);\n+\t\tif (unlikely(!xqp)) {\n+\t\t\tibdev_warn(\u0026dev-\u003eib_dev, \"CQE@CQ %d for unknown QPN %d\\n\",\n+\t\t\t\t cq-\u003excq.cqn, qpn);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\n+\t\t*cur_qp = core_to_xqp(xqp);\n+\t}\n+\n+\tmemset(wc, 0, sizeof(*wc));\n+\twc-\u003eqp = \u0026(*cur_qp)-\u003eibqp;\n+\topcode = xsc_get_cqe_opcode(dev-\u003exdev, cqe);\n+\tswitch (opcode) {\n+\tcase XSC_OPCODE_RDMA_REQ_SEND_IMMDT:\n+\tcase XSC_OPCODE_RDMA_REQ_WRITE_IMMDT:\n+\t\twc-\u003ewc_flags |= IB_WC_WITH_IMM;\n+\t\tfallthrough;\n+\tcase XSC_OPCODE_RDMA_REQ_SEND:\n+\tcase XSC_OPCODE_RDMA_REQ_WRITE:\n+\tcase XSC_OPCODE_RDMA_REQ_READ:\n+\tcase XSC_OPCODE_RDMA_MAD_REQ_SEND:\n+\t\twq = \u0026(*cur_qp)-\u003esq;\n+\t\tidx = xsc_get_cqe_wqe_id(dev-\u003exdev, cqe) \u003e\u003e (wq-\u003ewqe_shift - XSC_BASE_WQE_SHIFT);\n+\t\tidx \u0026= (wq-\u003ewqe_cnt - 1);\n+\t\tif (wq-\u003eprev_ce_wqe[idx] != wq-\u003elast_comp_ce_wqe) {\n+\t\t\txsc_generate_lost_cqe(dev, wq, qpn, idx, wc);\n+\t\t\tcq-\u003einvalid = 1;\n+\t\t\tbreak;\n+\t\t}\n+\t\twq-\u003elast_comp_ce_wqe = idx;\n+\t\thandle_good_req(wc, cqe, opcode);\n+\t\twc-\u003ewr_id = wq-\u003ewrid[idx];\n+\t\twq-\u003etail = wq-\u003ewqe_head[idx] + 1;\n+\t\tif (opcode != XSC_OPCODE_RDMA_MAD_REQ_SEND)\n+\t\t\tatomic_dec(\u0026wq-\u003eflush_wqe_cnt);\n+\t\twq-\u003eneed_flush[idx] = 0;\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"wqeid:%u, wq tail:%u qpn:%u\\n\", idx, wq-\u003etail, qpn);\n+\t\twc-\u003estatus = IB_WC_SUCCESS;\n+\t\tbreak;\n+\tcase XSC_OPCODE_RDMA_RSP_RECV_IMMDT:\n+\tcase XSC_OPCODE_RDMA_RSP_WRITE_IMMDT:\n+\t\twc-\u003ewc_flags |= IB_WC_WITH_IMM;\n+\t\tWR_BE_32(wc-\u003eex.imm_data, RD_LE_32(cqe-\u003eimm_data));\n+\t\tfallthrough;\n+\tcase XSC_OPCODE_RDMA_RSP_RECV:\n+\t\twq = \u0026(*cur_qp)-\u003erq;\n+\t\thandle_responder(wc, cqe, *cur_qp, opcode);\n+\t\tif ((*cur_qp)-\u003eibqp.srq) {\n+\t\t\txsrq = to_xsrq((*cur_qp)-\u003eibqp.srq);\n+\t\t\txsc_free_one_cache_srq_wr(xsrq, *cur_qp, wc-\u003ewr_id);\n+\t\t}\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"recv cqe idx:%u, len:%u, qpn:%u\\n\",\n+\t\t\t wq-\u003etail, wc-\u003ebyte_len, qpn);\n+\t\twc-\u003estatus = IB_WC_SUCCESS;\n+\t\tbreak;\n+\n+\tcase XSC_OPCODE_RDMA_MAD_RSP_RECV:\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"recv MAD, qpn:%u\\n\", qpn);\n+\t\txsc_handle_rdma_mad_resp_recv(cq, cur_qp, wc, cqe, opcode);\n+\t\tbreak;\n+\n+\tcase XSC_OPCODE_RDMA_REQ_ERROR:\n+\t\twq = \u0026(*cur_qp)-\u003esq;\n+\t\tidx = xsc_get_cqe_wqe_id(dev-\u003exdev, cqe) \u003e\u003e (wq-\u003ewqe_shift - XSC_BASE_WQE_SHIFT);\n+\t\tidx \u0026= (wq-\u003ewqe_cnt - 1);\n+\n+\t\tif (!wq-\u003eneed_flush[idx]) {\n+\t\t\tunsigned int i;\n+\t\t\tunsigned int tmp;\n+\t\t\tunsigned int prev = -1;\n+\n+\t\t\tfor (i = wq-\u003etail; i != wq-\u003ehead; i++) {\n+\t\t\t\ttmp = i \u0026 (wq-\u003ewqe_cnt - 1);\n+\t\t\t\tif (tmp == idx)\n+\t\t\t\t\tbreak;\n+\t\t\t\tif (wq-\u003eneed_flush[tmp])\n+\t\t\t\t\tprev = tmp;\n+\t\t\t}\n+\n+\t\t\tif (prev != -1) {\n+\t\t\t\tif (wq-\u003eprev_ce_wqe[prev] != wq-\u003elast_comp_ce_wqe) {\n+\t\t\t\t\txsc_generate_lost_cqe(dev, wq, qpn, prev, wc);\n+\t\t\t\t\tcq-\u003einvalid = 1;\n+\t\t\t\t\tbreak;\n+\t\t\t\t}\n+\t\t\t}\n+\t\t} else {\n+\t\t\tif (wq-\u003eprev_ce_wqe[idx] != wq-\u003elast_comp_ce_wqe) {\n+\t\t\t\txsc_generate_lost_cqe(dev, wq, qpn, idx, wc);\n+\t\t\t\tcq-\u003einvalid = 1;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\n+\t\twc-\u003ewr_id = wq-\u003ewrid[idx];\n+\t\twq-\u003etail = wq-\u003ewqe_head[idx] + 1;\n+\t\tif (wq-\u003eneed_flush[idx])\n+\t\t\tatomic_dec(\u0026wq-\u003eflush_wqe_cnt);\n+\t\twq-\u003eneed_flush[idx] = 0;\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"req error\\n%08x %08x %08x %08x %08x %08x\\n\",\n+\t\t\t p[0], p[1], p[2], p[3], p[5], p[6]);\n+\t\twc-\u003estatus = IB_WC_GENERAL_ERR;\n+\t\txsc_ib_err_state_qp(dev, *cur_qp, (*cur_qp)-\u003estate, XSC_QP_STATE_ERR);\n+\t\tbreak;\n+\tcase XSC_OPCODE_RDMA_RSP_ERROR:\n+\t\twq = \u0026(*cur_qp)-\u003erq;\n+\t\tidx = wq-\u003etail \u0026 (wq-\u003ewqe_cnt - 1);\n+\t\twc-\u003ewr_id = wq-\u003ewrid[idx];\n+\t\twq-\u003etail++;\n+\t\tatomic_dec(\u0026wq-\u003eflush_wqe_cnt);\n+\t\tif ((*cur_qp)-\u003eibqp.srq) {\n+\t\t\txsrq = to_xsrq((*cur_qp)-\u003eibqp.srq);\n+\t\t\txsc_free_one_cache_srq_wr(xsrq, *cur_qp, wc-\u003ewr_id);\n+\t\t}\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"rsp error\\n%08x %08x %08x %08x %08x %08x\\n\",\n+\t\t\t p[0], p[1], p[2], p[3], p[5], p[6]);\n+\t\twc-\u003estatus = IB_WC_GENERAL_ERR;\n+\t\txsc_ib_err_state_qp(dev, *cur_qp, (*cur_qp)-\u003estate, XSC_QP_STATE_ERR);\n+\t\tbreak;\n+\n+\tdefault:\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"completion error\\n%08x %08x %08x %08x %08x %08x\\n\",\n+\t\t\t p[0], p[1], p[2], p[3], p[5], p[6]);\n+\t\twc-\u003estatus = IB_WC_GENERAL_ERR;\n+\t\twc-\u003ewr_id = 0;\n+\t\tbreak;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static inline void gen_flush_err_cqe(struct xsc_err_state_qp_node *err_node,\n+\t\t\t\t struct ib_qp *ibqp, struct xsc_ib_wq *wq, u32 idx,\n+\t\t\t\t struct ib_wc *wc)\n+{\n+\tmemset(wc, 0, sizeof(*wc));\n+\tif (err_node-\u003eis_sq) {\n+\t\tswitch (wq-\u003ewr_opcode[idx]) {\n+\t\tcase IB_WR_SEND:\n+\t\tcase IB_WR_SEND_WITH_IMM:\n+\t\t\twc-\u003eopcode = IB_WC_SEND;\n+\t\t\tbreak;\n+\t\tcase IB_WR_RDMA_WRITE:\n+\t\tcase IB_WR_RDMA_WRITE_WITH_IMM:\n+\t\t\twc-\u003eopcode = IB_WC_RDMA_WRITE;\n+\t\t\tbreak;\n+\t\tcase IB_WR_RDMA_READ:\n+\t\t\twc-\u003eopcode = IB_WC_RDMA_READ;\n+\t\t}\n+\t} else {\n+\t\twc-\u003eopcode = IB_WC_RECV;\n+\t}\n+\n+\twc-\u003eqp = ibqp;\n+\twc-\u003estatus = IB_WC_WR_FLUSH_ERR;\n+\twc-\u003evendor_err = XSC_ERR_CODE_FLUSH;\n+\twc-\u003ewr_id = wq-\u003ewrid[idx];\n+\twq-\u003etail++;\n+\tatomic_dec(\u0026wq-\u003eflush_wqe_cnt);\n+\tif (err_node-\u003eis_sq)\n+\t\twq-\u003eneed_flush[idx] = 0;\n+}\n+\n+static inline int xsc_generate_flush_err_cqe(struct ib_cq *ibcq,\n+\t\t\t\t\t int ne, int *npolled, struct ib_wc *wc)\n+{\n+\tu32 qp_id = 0;\n+\tint flush_wqe_cnt = 0;\n+\tint sw_npolled = 0;\n+\tu32 idx = 0;\n+\tstruct xsc_err_state_qp_node *err_qp_node;\n+\tstruct xsc_core_qp *xqp;\n+\tstruct xsc_ib_cq *cq = to_xcq(ibcq);\n+\tstruct xsc_ib_wq *wq;\n+\tstruct xsc_ib_dev *dev = to_xdev(cq-\u003eibcq.device);\n+\n+\tlist_for_each_entry(err_qp_node, \u0026cq-\u003eerr_state_qp_list, entry) {\n+\t\tif (!err_qp_node)\n+\t\t\tbreak;\n+\n+\t\tsw_npolled = 0;\n+\t\tqp_id = err_qp_node-\u003eqp_id;\n+\t\txqp = __xsc_qp_lookup(dev-\u003exdev, qp_id);\n+\t\tif (unlikely(!xqp)) {\n+\t\t\tibdev_warn(\u0026dev-\u003eib_dev, \"CQE@CQ %d for unknown QPN %d\\n\",\n+\t\t\t\t cq-\u003excq.cqn, qp_id);\n+\t\t\tcontinue;\n+\t\t}\n+\t\twq = err_qp_node-\u003eis_sq ? \u0026(core_to_xqp(xqp)-\u003esq) : \u0026(core_to_xqp(xqp)-\u003erq);\n+\t\tflush_wqe_cnt = atomic_read(\u0026wq-\u003eflush_wqe_cnt);\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"is_sq %d, flush_wq_cnt %d, ne %d, npolled %d, qp_id %d, cqn %d\\n\",\n+\t\t\t err_qp_node-\u003eis_sq, flush_wqe_cnt, ne, *npolled, qp_id, cq-\u003excq.cqn);\n+\n+\t\twhile (wq-\u003etail != wq-\u003ehead) {\n+\t\t\tidx = wq-\u003etail \u0026 (wq-\u003ewqe_cnt - 1);\n+\t\t\tif (err_qp_node-\u003eis_sq \u0026\u0026 !wq-\u003eneed_flush[idx]) {\n+\t\t\t\twq-\u003etail++;\n+\t\t\t\tcontinue;\n+\t\t\t} else {\n+\t\t\t\tgen_flush_err_cqe(err_qp_node, \u0026(core_to_xqp(xqp)-\u003eibqp), wq,\n+\t\t\t\t\t\t idx, wc + *npolled + sw_npolled);\n+\t\t\t\t++sw_npolled;\n+\t\t\t}\n+\n+\t\t\tif (sw_npolled == ne - *npolled)\n+\t\t\t\tbreak;\n+\t\t}\n+\t\t*npolled += sw_npolled;\n+\t\tif (*npolled == ne)\n+\t\t\tbreak;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+int xsc_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)\n+{\n+\tstruct xsc_ib_cq *cq = to_xcq(ibcq);\n+\tstruct xsc_core_cq *xcq = \u0026cq-\u003excq;\n+\tstruct xsc_ib_qp *cur_qp = NULL;\n+\tint npolled = 0;\n+\tint err = 0;\n+\tunsigned long flags;\n+\tu32 next_cid;\n+\n+\tspin_lock_irqsave(\u0026cq-\u003elock, flags);\n+\tnext_cid = xcq-\u003econs_index;\n+\n+\tfor (npolled = 0; npolled \u003c num_entries; npolled++) {\n+\t\tif (cq-\u003einvalid) {\n+\t\t\terr = -EINVAL;\n+\t\t\tbreak;\n+\t\t}\n+\t\terr = xsc_poll_one(cq, \u0026cur_qp, wc + npolled);\n+\t\tif (err)\n+\t\t\tbreak;\n+\t}\n+\n+\tif (err) {\n+\t\tif (npolled \u003c num_entries \u0026\u0026 !(list_empty(\u0026cq-\u003eerr_state_qp_list)))\n+\t\t\txsc_generate_flush_err_cqe(ibcq, num_entries, \u0026npolled, wc);\n+\t}\n+\n+\tif (next_cid != xcq-\u003econs_index)\n+\t\txsc_update_cq_ci(xcq-\u003edev, xcq-\u003ecqn, xcq-\u003econs_index);\n+\n+\tspin_unlock_irqrestore(\u0026cq-\u003elock, flags);\n+\n+\treturn npolled;\n+}\n+\n+static int xsc_cqe_is_empty(struct xsc_ib_cq *cq)\n+{\n+\tstruct xsc_cqe *cqe = get_sw_cqe(cq, cq-\u003excq.cons_index);\n+\n+\tif (cq-\u003einvalid)\n+\t\treturn 1;\n+\n+\tif (!cqe)\n+\t\treturn 1;\n+\n+\treturn 0;\n+}\n+\n+int xsc_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags)\n+{\n+\tstruct xsc_ib_cq *xcq = to_xcq(ibcq);\n+\tstruct xsc_core_cq *cq = \u0026xcq-\u003excq;\n+\tint ret = 0;\n+\tunsigned long irq_flags;\n+\tu8 solicited = 0;\n+\n+\tspin_lock_irqsave(\u0026xcq-\u003elock, irq_flags);\n+\tif (flags \u0026 IB_CQ_NEXT_COMP)\n+\t\tsolicited = 0;\n+\telse if (flags \u0026 IB_CQ_SOLICITED)\n+\t\tsolicited = 1;/* arm next:0 arm solicited:1 */\n+\n+\tif ((flags \u0026 IB_CQ_REPORT_MISSED_EVENTS) \u0026\u0026 (!xsc_cqe_is_empty(xcq))) {\n+\t\tret = 1;\n+\t\tgoto out;\n+\t}\n+\n+\txsc_arm_cq(cq-\u003edev, cq-\u003ecqn, cq-\u003econs_index, solicited);\n+out:\n+\tspin_unlock_irqrestore(\u0026xcq-\u003elock, irq_flags);\n+\treturn ret;\n+}\n+\n+static int alloc_cq_buf(struct xsc_ib_dev *dev, struct xsc_ib_cq_buf *buf,\n+\t\t\tint nent, int cqe_size)\n+{\n+\tint err;\n+\n+\terr = xsc_buf_alloc(dev-\u003exdev, nent * cqe_size,\n+\t\t\t PAGE_SIZE, \u0026buf-\u003ebuf);\n+\tif (err)\n+\t\treturn err;\n+\n+\tbuf-\u003ecqe_size = cqe_size;\n+\n+\treturn 0;\n+}\n+\n+static void free_cq_buf(struct xsc_ib_dev *dev, struct xsc_ib_cq_buf *buf)\n+{\n+\txsc_buf_free(dev-\u003exdev, \u0026buf-\u003ebuf);\n+}\n+\n+static struct ib_umem *cq_umem_get(struct xsc_ib_dev *dev,\n+\t\t\t\t struct ib_udata *udata, struct ib_ucontext *context,\n+\tint entries, struct xsc_ib_create_cq *ucmd)\n+{\n+\tstruct ib_umem *umem;\n+\n+\tumem = ib_umem_get_va(\u0026dev-\u003eib_dev, ucmd-\u003ebuf_addr,\n+\t\t\t entries * ucmd-\u003ecqe_size,\n+\t\t\t IB_ACCESS_LOCAL_WRITE);\n+\treturn umem;\n+}\n+\n+static int create_cq_user(struct xsc_ib_dev *dev, struct ib_udata *udata,\n+\t\t\t struct ib_ucontext *context, struct xsc_ib_cq *cq,\n+\t\t\t int entries, struct xsc_create_cq_ex_mbox_in **cqb,\n+\t\t\t int *cqe_size, int *index, int *inlen)\n+{\n+\tstruct xsc_ib_create_cq ucmd;\n+\tint page_shift;\n+\tint npages;\n+\tint ncont;\n+\tint err;\n+\tint log_cq_sz;\n+\tint hw_npages;\n+\tstruct ib_umem_dmabuf *umem_dmabuf;\n+\n+\tif (ib_copy_from_udata(\u0026ucmd, udata, sizeof(ucmd)))\n+\t\treturn -EFAULT;\n+\n+\t*cqe_size = ucmd.cqe_size;\n+\n+\tif (ucmd.dmabuf_fd != -1) {\n+\t\tumem_dmabuf = ib_umem_dmabuf_get_pinned(\u0026dev-\u003eib_dev, 0, ucmd.dmabuf_sz,\n+\t\t\t\t\t\t\tucmd.dmabuf_fd, IB_ACCESS_LOCAL_WRITE);\n+\n+\t\tif (IS_ERR(umem_dmabuf)) {\n+\t\t\tibdev_warn(\u0026dev-\u003eib_dev, \"umem_dmabuf get failed by fd %d (%ld)\\n\",\n+\t\t\t\t ucmd.dmabuf_fd, PTR_ERR(umem_dmabuf));\n+\t\t\treturn -ENOMEM;\n+\t\t}\n+\n+\t\tcq-\u003ebuf.umem = \u0026umem_dmabuf-\u003eumem;\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"get umem_dmabuf success by fd %d, umem_dmabuf_addr %p\\n\",\n+\t\t\t ucmd.dmabuf_fd, umem_dmabuf);\n+\t} else {\n+\t\tcq-\u003ebuf.umem = cq_umem_get(dev, udata, context, entries, \u0026ucmd);\n+\t}\n+\n+\tif (IS_ERR(cq-\u003ebuf.umem)) {\n+\t\terr = PTR_ERR(cq-\u003ebuf.umem);\n+\t\treturn err;\n+\t}\n+\n+\txsc_ib_cont_pages(cq-\u003ebuf.umem, ucmd.buf_addr, \u0026npages, \u0026page_shift,\n+\t\t\t \u0026ncont);\n+\tif (ncont != npages) {\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"bad page_shift:%d, ncont:%d\\n\", page_shift, ncont);\n+\t\t/* amber doesn't support compound pages */\n+\t\tpage_shift = PAGE_SHIFT;\n+\t\tncont = npages;\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"overwrite to page_shift:%d, ncont:%d\\n\",\n+\t\t\t page_shift, ncont);\n+\t}\n+\tlog_cq_sz = ilog2(entries);\n+\thw_npages = DIV_ROUND_UP((1 \u003c\u003c log_cq_sz) * (*cqe_size), PAGE_SIZE_4K);\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"addr 0x%llx, entries %d, npages %d, hw_npages %d\\n\",\n+\t\t ucmd.buf_addr, entries, npages, hw_npages);\n+\n+\t*inlen = sizeof(**cqb) + sizeof(*(*cqb)-\u003epas) * hw_npages;\n+\t*cqb = xsc_vzalloc(*inlen);\n+\tif (!*cqb) {\n+\t\terr = -ENOMEM;\n+\t\tgoto err_umem;\n+\t}\n+\txsc_ib_populate_pas(dev, cq-\u003ebuf.umem, page_shift, (*cqb)-\u003epas, hw_npages, true);\n+\t(*cqb)-\u003ectx_ex.ctx.pa_num = cpu_to_be16(hw_npages);\n+\t(*cqb)-\u003ectx_ex.pa_num_ex = cpu_to_be32(hw_npages);\n+\n+\treturn 0;\n+\n+err_umem:\n+\tib_umem_release(cq-\u003ebuf.umem);\n+\treturn err;\n+}\n+\n+static void destroy_cq_user(struct xsc_ib_cq *cq, struct ib_udata *udata)\n+{\n+\tib_umem_release(cq-\u003ebuf.umem);\n+}\n+\n+static int create_cq_kernel(struct xsc_ib_dev *dev, struct xsc_ib_cq *cq,\n+\t\t\t int entries, int cqe_size,\n+\t\t\t struct xsc_create_cq_ex_mbox_in **cqb,\n+\t\t\t int *index, int *inlen)\n+{\n+\tint err;\n+\tint i = 0;\n+\tstruct xsc_cqe *cqe = NULL;\n+\tint hw_npages;\n+\n+\tcq-\u003excq.cqe_sz = cqe_size;\n+\n+\terr = alloc_cq_buf(dev, \u0026cq-\u003ebuf, entries, cqe_size);\n+\tif (err)\n+\t\treturn err;\n+\n+\tfor (i = 0; i \u003c entries; i++) {\n+\t\tcqe = (struct xsc_cqe *)get_cqe(cq, i);\n+\t\tcqe-\u003eowner = 1;\n+\t}\n+\n+\thw_npages = DIV_ROUND_UP(entries * cqe_size, PAGE_SIZE_4K);\n+\t*inlen = sizeof(**cqb) + sizeof(*(*cqb)-\u003epas) * hw_npages;\n+\t*cqb = xsc_vzalloc(*inlen);\n+\tif (!*cqb) {\n+\t\terr = -ENOMEM;\n+\t\tgoto err_buf;\n+\t}\n+\txsc_fill_page_array(\u0026cq-\u003ebuf.buf, (*cqb)-\u003epas, hw_npages);\n+\t(*cqb)-\u003ectx_ex.ctx.pa_num = cpu_to_be16(hw_npages);\n+\t(*cqb)-\u003ectx_ex.pa_num_ex = cpu_to_be32(hw_npages);\n+\n+\treturn 0;\n+\n+err_buf:\n+\tfree_cq_buf(dev, \u0026cq-\u003ebuf);\n+\treturn err;\n+}\n+\n+static void destroy_cq_kernel(struct xsc_ib_dev *dev, struct xsc_ib_cq *cq)\n+{\n+\tfree_cq_buf(dev, \u0026cq-\u003ebuf);\n+}\n+\n+int xsc_ib_create_cq(struct ib_cq *ibcq,\n+\t\t const struct ib_cq_init_attr *attr,\n+\t\t struct uverbs_attr_bundle *attrs)\n+{\n+\tstruct ib_device *ibdev = ibcq-\u003edevice;\n+\tstruct ib_udata *udata = attrs ? \u0026attrs-\u003edriver_udata : NULL;\n+\tint entries = attr-\u003ecqe;\n+\tint vector = attr-\u003ecomp_vector;\n+\tstruct xsc_create_cq_ex_mbox_in *cqb = NULL;\n+\tstruct xsc_ib_dev *dev = to_xdev(ibdev);\n+\tstruct xsc_ib_cq *cq;\n+\tint index;\n+\tint inlen = 0;\n+\tint cqe_size;\n+\tint irqn;\n+\tint err;\n+\tunsigned int eqn;\n+\n+\tentries = roundup_pow_of_two(entries);\n+\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"entries:%d, vector:%d, max_cqes:%d\\n\", entries, vector,\n+\t\t dev-\u003exdev-\u003ecaps.max_cqes);\n+\n+\tif (entries \u003e dev-\u003exdev-\u003ecaps.max_cqes)\n+\t\tentries = dev-\u003exdev-\u003ecaps.max_cqes;\n+\tcq = to_xcq(ibcq);\n+\tcq-\u003eibcq.cqe = entries;\n+\tmutex_init(\u0026cq-\u003eresize_mutex);\n+\tspin_lock_init(\u0026cq-\u003elock);\n+\tcq-\u003eresize_buf = NULL;\n+\tcq-\u003eresize_umem = NULL;\n+\tcq-\u003ecreate_flags = attr-\u003eflags;\n+\tcq-\u003einvalid = 0;\n+\n+\tif (udata) {\n+\t\terr = create_cq_user(dev, udata, NULL, cq, entries,\n+\t\t\t\t \u0026cqb, \u0026cqe_size, \u0026index, \u0026inlen);\n+\t\tif (err)\n+\t\t\tgoto err_create;\n+\t} else {\n+\t\tcqe_size = is_support_cqe64(dev-\u003exdev) ?\n+\t\t\tsizeof(struct xsc_cqe64) : sizeof(struct xsc_cqe);\n+\t\terr = create_cq_kernel(dev, cq, entries, cqe_size, \u0026cqb, \u0026index, \u0026inlen);\n+\t\tif (err)\n+\t\t\tgoto err_create;\n+\t}\n+\n+\tcq-\u003ecqe_size = cqe_size;\n+\tcqb-\u003ectx_ex.ctx.log_cq_sz = ilog2(entries);\n+\tcqb-\u003ectx_ex.ctx.glb_func_id = cpu_to_be16(dev-\u003exdev-\u003eglb_func_id);\n+\n+\terr = xsc_vector2eqn(dev-\u003exdev, vector, \u0026eqn, \u0026irqn);\n+\tif (err)\n+\t\tgoto err_cqb;\n+\n+\tcqb-\u003ectx_ex.ctx.eqn = eqn;\n+\tcqb-\u003ectx_ex.ctx.eqn = cpu_to_be16(cqb-\u003ectx_ex.ctx.eqn);\n+\tcqb-\u003ectx_ex.page_shift = PAGE_SHIFT;\n+\tif (cq-\u003ecreate_flags \u0026 IB_UVERBS_CQ_FLAGS_IGNORE_OVERRUN)\n+\t\tcqb-\u003ectx_ex.overrun_ignore = 1;\n+\n+\terr = xsc_core_create_cq(dev-\u003exdev, \u0026cq-\u003excq, cqb, inlen);\n+\tif (err)\n+\t\tgoto err_cqb;\n+\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"succeeded to create cqn %d, vector=%d, cq_sz=%d, eqn=%d\\n\",\n+\t\t cq-\u003excq.cqn, vector, entries, eqn);\n+\tcq-\u003excq.irqn = irqn;\n+\tcq-\u003excq.comp = xsc_ib_cq_comp;\n+\tcq-\u003excq.event = xsc_ib_cq_event;\n+\n+\tif (udata) {\n+\t\tif (ib_copy_to_udata(udata, \u0026cq-\u003excq.cqn, sizeof(__u32))) {\n+\t\t\terr = -EFAULT;\n+\t\t\tgoto err_cmd;\n+\t\t}\n+\t}\n+\n+\tINIT_LIST_HEAD(\u0026cq-\u003esend_qp_list);\n+\tINIT_LIST_HEAD(\u0026cq-\u003erecv_qp_list);\n+\tINIT_LIST_HEAD(\u0026cq-\u003eerr_state_qp_list);\n+\txsc_vfree(cqb);\n+\n+\treturn 0;\n+\n+err_cmd:\n+\txsc_core_destroy_cq(dev-\u003exdev, \u0026cq-\u003excq);\n+\n+err_cqb:\n+\txsc_vfree(cqb);\n+\tif (udata)\n+\t\tdestroy_cq_user(cq, udata);\n+\telse\n+\t\tdestroy_cq_kernel(dev, cq);\n+\n+err_create:\n+\n+\treturn err;\n+}\n+\n+int xsc_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(cq-\u003edevice);\n+\tstruct xsc_ib_cq *xcq = to_xcq(cq);\n+\tstruct xsc_err_state_qp_node *tmp = NULL, *err_qp_node = NULL;\n+\n+\tlist_for_each_entry_safe(err_qp_node, tmp, \u0026xcq-\u003eerr_state_qp_list, entry) {\n+\t\tlist_del(\u0026err_qp_node-\u003eentry);\n+\t\tkfree(err_qp_node);\n+\t}\n+\n+\txsc_core_destroy_cq(dev-\u003exdev, \u0026xcq-\u003excq);\n+\tif (udata)\n+\t\tdestroy_cq_user(xcq, udata);\n+\telse\n+\t\tdestroy_cq_kernel(dev, xcq);\n+\n+\treturn 0;\n+}\n+\n+static int is_equal_rsn(struct xsc_cqe *cqe, u32 rsn)\n+{\n+\tu32 qpn = le32_to_cpu(cqe-\u003eqp_id);\n+\treturn rsn == qpn;\n+}\n+\n+void __xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 rsn)\n+{\n+\tstruct xsc_cqe *cqe, *dest;\n+\tu32 prod_index;\n+\tint nfreed = 0;\n+\tu8 owner_bit;\n+\n+\tif (!cq)\n+\t\treturn;\n+\n+\t/* First we need to find the current producer index, so we\n+\t * know where to start cleaning from. It doesn't matter if HW\n+\t * adds new entries after this loop -- the QP we're worried\n+\t * about is already in RESET, so the new entries won't come\n+\t * from our QP and therefore don't need to be checked.\n+\t */\n+\tfor (prod_index = cq-\u003excq.cons_index; get_sw_cqe(cq, prod_index); prod_index++)\n+\t\tif (prod_index == cq-\u003excq.cons_index + cq-\u003eibcq.cqe)\n+\t\t\tbreak;\n+\n+\t/* Now sweep backwards through the CQ, removing CQ entries\n+\t * that match our QP by copying older entries on top of them.\n+\t */\n+\twhile ((int)(--prod_index) - (int)cq-\u003excq.cons_index \u003e= 0) {\n+\t\tcqe = (struct xsc_cqe *)get_cqe(cq, prod_index \u0026 (cq-\u003eibcq.cqe - 1));\n+\t\tif (is_equal_rsn(cqe, rsn)) {\n+\t\t\t++nfreed;\n+\t\t} else if (nfreed) {\n+\t\t\tdest = (struct xsc_cqe *)get_cqe(cq, (prod_index + nfreed) \u0026\n+\t\t\t\t\t\t\t (cq-\u003eibcq.cqe - 1));\n+\t\t\towner_bit = dest-\u003eowner \u0026 XSC_CQE_OWNER_MASK;\n+\t\t\tmemcpy(dest, cqe, cq-\u003excq.cqe_sz);\n+\t\t\tdest-\u003eowner = owner_bit |\n+\t\t\t\t(dest-\u003eowner \u0026 ~XSC_CQE_OWNER_MASK);\n+\t\t}\n+\t}\n+\n+\tif (nfreed) {\n+\t\tcq-\u003excq.cons_index += nfreed;\n+\t\txsc_update_cq_ci(cq-\u003excq.dev, cq-\u003excq.cqn, cq-\u003excq.cons_index);\n+\t}\n+}\n+\n+void xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 qpn)\n+{\n+\tif (!cq)\n+\t\treturn;\n+\n+\tspin_lock_irq(\u0026cq-\u003elock);\n+\t__xsc_ib_cq_clean(cq, qpn);\n+\tspin_unlock_irq(\u0026cq-\u003elock);\n+}\ndiff --git a/drivers/infiniband/hw/xsc/main.c b/drivers/infiniband/hw/xsc/main.c\nnew file mode 100644\nindex 0000000000000..38c8b4f0690c2\n--- /dev/null\n+++ b/drivers/infiniband/hw/xsc/main.c\n@@ -0,0 +1,887 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/module.h\u003e\n+#include \u003clinux/auxiliary_bus.h\u003e\n+#include \u003clinux/init.h\u003e\n+#include \u003clinux/errno.h\u003e\n+#include \u003clinux/pci.h\u003e\n+#include \u003clinux/namei.h\u003e\n+#include \u003clinux/kobject.h\u003e\n+#include \u003clinux/kernfs.h\u003e\n+#include \u003clinux/dma-mapping.h\u003e\n+#include \u003clinux/slab.h\u003e\n+#include \u003clinux/io-mapping.h\u003e\n+#include \u003clinux/sched.h\u003e\n+#include \u003clinux/netdevice.h\u003e\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_hsi.h\"\n+#include \"common/xsc_cmd.h\"\n+#include \"common/driver.h\"\n+\n+#include \u003crdma/ib_user_verbs.h\u003e\n+#include \u003crdma/ib_smi.h\u003e\n+#include \u003crdma/ib_umem.h\u003e\n+\n+#include \u003crdma/xsc-abi.h\u003e\n+#include \"xsc_ib.h\"\n+\n+#define DRIVER_NAME \"xsc_ib\"\n+\n+#define XSC_IB_GID_TBL_LEN\t256\n+#define XSC_IB_MAX_PKEYS\t0x80\n+\n+MODULE_DESCRIPTION(\"Yunsilicon HCA IB driver\");\n+MODULE_LICENSE(\"GPL\");\n+\n+static int xsc_ib_query_device(struct ib_device *ibdev,\n+\t\t\t struct ib_device_attr *props,\n+\t\t\t struct ib_udata *udata)\n+{\n+\tsize_t udata_outlen = (udata) ? udata-\u003eoutlen : 0;\n+\tstruct xsc_ib_dev *dev = to_xdev(ibdev);\n+\tstruct xsc_ib_query_device_resp resp;\n+\tunion xsc_ib_fw_ver fw_ver;\n+\tint err = -ENOMEM;\n+\tsize_t resp_len;\n+\tint max_rq_sg;\n+\tu64 max_tso;\n+\tu64 flags;\n+\n+\terr = ib_is_udata_in_empty(udata);\n+\tif (err)\n+\t\treturn err;\n+\n+\tmemset(\u0026resp, 0, sizeof(resp));\n+\tmemset(props, 0, sizeof(*props));\n+\n+\tresp_len = sizeof(resp.comp_mask) + sizeof(resp.response_length);\n+\t/*check param*/\n+\tif (udata_outlen \u0026\u0026 udata_outlen \u003c resp_len)\n+\t\treturn -EINVAL;\n+\n+\tresp.response_length = resp_len;\n+\n+\tfw_ver.data = 0;\n+\tfw_ver.s.ver_major = dev-\u003exdev-\u003efw_version_major;\n+\tfw_ver.s.ver_minor = dev-\u003exdev-\u003efw_version_minor;\n+\tfw_ver.s.ver_patch = dev-\u003exdev-\u003efw_version_patch;\n+\tfw_ver.s.ver_tweak = dev-\u003exdev-\u003efw_version_tweak;\n+\tprops-\u003efw_ver = fw_ver.data;\n+\n+\tprops-\u003edevice_cap_flags = IB_DEVICE_CHANGE_PHY_PORT |\n+\t\tIB_DEVICE_PORT_ACTIVE_EVENT\t\t|\n+\t\tIB_DEVICE_SYS_IMAGE_GUID\t\t|\n+\t\tIB_DEVICE_RC_RNR_NAK_GEN;\n+\tprops-\u003ekernel_cap_flags = IBK_BLOCK_MULTICAST_LOOPBACK;\n+\tprops-\u003ekernel_cap_flags |= IBK_LOCAL_DMA_LKEY;\n+\n+\tflags = dev-\u003exdev-\u003ecaps.flags;\n+\tif (flags \u0026 XSC_DEV_CAP_FLAG_BAD_PKEY_CNTR)\n+\t\tprops-\u003edevice_cap_flags |= IB_DEVICE_BAD_PKEY_CNTR;\n+\tif (flags \u0026 XSC_DEV_CAP_FLAG_BAD_QKEY_CNTR)\n+\t\tprops-\u003edevice_cap_flags |= IB_DEVICE_BAD_QKEY_CNTR;\n+\tif (flags \u0026 XSC_DEV_CAP_FLAG_APM)\n+\t\tprops-\u003edevice_cap_flags |= IB_DEVICE_AUTO_PATH_MIG;\n+\tprops-\u003epage_size_cap\t = dev-\u003exdev-\u003ecaps.min_page_sz;\n+\tprops-\u003emax_mr_size\t = xsc_get_max_mr_size(dev-\u003exdev);\n+\tprops-\u003emax_qp\t\t = dev-\u003exdev-\u003ecaps.max_qp;\n+\tprops-\u003emax_qp_wr\t = xsc_get_max_qp_depth(dev-\u003exdev);\n+\tmax_rq_sg = dev-\u003exdev-\u003ecaps.max_rq_desc_sz / sizeof(struct xsc_wqe_data_seg);\n+\n+\tprops-\u003emax_send_sge = dev-\u003exdev-\u003ecaps.send_ds_num - XSC_CTRL_SEG_NUM -\n+\t\tXSC_RADDR_SEG_NUM;\n+\tprops-\u003emax_recv_sge = dev-\u003exdev-\u003ecaps.recv_ds_num;\n+\tprops-\u003emax_sge_rd\t = 1;/*max sge per read wqe*/\n+\tprops-\u003emax_cq\t\t = dev-\u003exdev-\u003ecaps.max_cq;\n+\tprops-\u003emax_cqe\t\t = dev-\u003exdev-\u003ecaps.max_cqes;\n+\tprops-\u003emax_mr\t\t = (1 \u003c\u003c dev-\u003exdev-\u003ecaps.log_max_mkey) - 1;\n+\tprops-\u003emax_pd\t\t = dev-\u003exdev-\u003ecaps.max_pd;\n+\tprops-\u003emax_qp_rd_atom\t = dev-\u003exdev-\u003ecaps.max_ra_res_qp;\n+\tprops-\u003emax_qp_init_rd_atom = dev-\u003exdev-\u003ecaps.max_ra_req_qp;\n+\tprops-\u003emax_res_rd_atom\t = props-\u003emax_qp_rd_atom * props-\u003emax_qp;\n+\tprops-\u003emax_srq\t\t =\n+\t\tdev-\u003exdev-\u003ecaps.log_max_srq ? (1 \u003c\u003c dev-\u003exdev-\u003ecaps.log_max_srq) : 0;\n+\tprops-\u003emax_srq_wr\t = dev-\u003exdev-\u003ecaps.max_srq_wqes - 1;\n+\tprops-\u003emax_srq_sge\t = dev-\u003exdev-\u003ecaps.log_max_srq ? (max_rq_sg - 1) : 0;\n+\tprops-\u003elocal_ca_ack_delay = dev-\u003exdev-\u003ecaps.local_ca_ack_delay;\n+\tprops-\u003eatomic_cap\t = is_support_rdma_atomic(dev-\u003exdev) ?\n+\t\tIB_ATOMIC_HCA : IB_ATOMIC_NONE;\n+\tprops-\u003emasked_atomic_cap = IB_ATOMIC_HCA;\n+\tprops-\u003emax_mcast_grp\t =\n+\t\tdev-\u003exdev-\u003ecaps.log_max_mcg ? (1 \u003c\u003c dev-\u003exdev-\u003ecaps.log_max_mcg) : 0;\n+\tprops-\u003emax_mcast_qp_attach = dev-\u003exdev-\u003ecaps.max_qp_mcg;\n+\tprops-\u003emax_total_mcast_qp_attach = props-\u003emax_mcast_qp_attach *\n+\t\t\t\t\t props-\u003emax_mcast_grp;\n+\n+\tprops-\u003esys_image_guid = dev-\u003exdev-\u003eboard_info-\u003eguid;\n+\tprops-\u003evendor_id = dev-\u003exdev-\u003epdev-\u003evendor;\n+\tprops-\u003evendor_part_id = dev-\u003exdev-\u003epdev-\u003edevice;\n+\tprops-\u003ehw_ver = ((dev-\u003exdev-\u003echip_ver_l \u0026 0xffff) \u003c\u003c 16) |\n+\t\t(dev-\u003exdev-\u003ehotfix_num \u0026 0xffff);\n+\tprops-\u003emax_pkeys = XSC_IB_MAX_PKEYS;\n+\tprops-\u003emax_wq_type_rq = dev-\u003exdev-\u003ecaps.max_qp;\n+\tprops-\u003ehca_core_clock = dev-\u003exdev-\u003ecaps.hca_core_clock * 1000; /* KHz */\n+\tprops-\u003erss_caps.max_rwq_indirection_tables =\n+\t\tdev-\u003exdev-\u003ecaps.max_rwq_indirection_tables;\n+\tprops-\u003erss_caps.max_rwq_indirection_table_size =\n+\t\tdev-\u003exdev-\u003ecaps.max_rwq_indirection_table_size;\n+\tprops-\u003erss_caps.supported_qpts = 1 \u003c\u003c IB_QPT_RAW_PACKET;\n+\n+\t/*response tso_caps extend param*/\n+\tif (offsetofend(typeof(resp), tso_caps) \u003c= udata_outlen) {\n+\t\tmax_tso = dev-\u003exdev-\u003ecaps.log_max_tso ? (1 \u003c\u003c dev-\u003exdev-\u003ecaps.log_max_tso) : 0;\n+\t\tif (max_tso) {\n+\t\t\tresp.tso_caps.max_tso = max_tso;\n+\t\t\tresp.tso_caps.supported_qpts |= 1 \u003c\u003c IB_QPT_RAW_PACKET;\n+\t\t\tresp.response_length += sizeof(resp.tso_caps);\n+\t\t}\n+\t}\n+\n+\t/*response rss_caps extend param*/\n+\tif (offsetofend(typeof(resp), rss_caps) \u003c= udata_outlen) {\n+\t\tresp.rss_caps.rx_hash_function = XSC_RX_HASH_FUNC_TOEPLITZ;\n+\t\tresp.rss_caps.rx_hash_fields_mask =\n+\t\t\tXSC_RX_HASH_SRC_IPV4 |\n+\t\t\tXSC_RX_HASH_DST_IPV4 |\n+\t\t\tXSC_RX_HASH_SRC_IPV6 |\n+\t\t\tXSC_RX_HASH_DST_IPV6 |\n+\t\t\tXSC_RX_HASH_SRC_PORT_TCP |\n+\t\t\tXSC_RX_HASH_DST_PORT_TCP |\n+\t\t\tXSC_RX_HASH_SRC_PORT_UDP |\n+\t\t\tXSC_RX_HASH_DST_PORT_UDP |\n+\t\t\tXSC_RX_HASH_INNER;\n+\t\tresp.response_length += sizeof(resp.rss_caps);\n+\t}\n+\n+\t/*response packet pacing caps*/\n+\tif (offsetofend(typeof(resp), packet_pacing_caps) \u003c= udata_outlen) {\n+\t\tresp.packet_pacing_caps.qp_rate_limit_max =\n+\t\t\tdev-\u003exdev-\u003ecaps.qp_rate_limit_max;\n+\t\tresp.packet_pacing_caps.qp_rate_limit_min =\n+\t\t\tdev-\u003exdev-\u003ecaps.qp_rate_limit_min;\n+\t\tresp.packet_pacing_caps.supported_qpts |= 1 \u003c\u003c IB_QPT_RAW_PACKET;\n+\n+\t\tresp.response_length += sizeof(resp.packet_pacing_caps);\n+\t}\n+\n+\t/*copy response data to user*/\n+\tif (udata_outlen) {\n+\t\terr = ib_copy_to_udata(udata, \u0026resp, resp.response_length);\n+\t\tif (err) {\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"copy response info to udata fail,err=%d\\n\", err);\n+\t\t\treturn err;\n+\t\t}\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static void xsc_calc_link_info(struct xsc_core_device *xdev,\n+\t\t\t struct ib_port_attr *props)\n+{\n+\tswitch (xsc_get_link_speed(xdev)) {\n+\tcase MODULE_SPEED_10G:\n+\t\tprops-\u003eactive_speed = XSC_RDMA_LINK_SPEED_10GB;\n+\t\tprops-\u003eactive_width = 1;\n+\t\tbreak;\n+\tcase MODULE_SPEED_25G:\n+\t\tprops-\u003eactive_speed = XSC_RDMA_LINK_SPEED_25GB;\n+\t\tprops-\u003eactive_width = 1;\n+\t\tbreak;\n+\tcase MODULE_SPEED_40G_R4:\n+\t\tprops-\u003eactive_speed = XSC_RDMA_LINK_SPEED_10GB;\n+\t\tprops-\u003eactive_width = 2;\n+\t\tbreak;\n+\tcase MODULE_SPEED_50G_R:\n+\t\tprops-\u003eactive_speed = XSC_RDMA_LINK_SPEED_50GB;\n+\t\tprops-\u003eactive_width = 1;\n+\t\tbreak;\n+\tcase MODULE_SPEED_50G_R2:\n+\t\tprops-\u003eactive_speed = XSC_RDMA_LINK_SPEED_50GB;\n+\t\tprops-\u003eactive_width = 1;\n+\t\tbreak;\n+\tcase MODULE_SPEED_100G_R2:\n+\t\tprops-\u003eactive_speed = XSC_RDMA_LINK_SPEED_25GB;\n+\t\tprops-\u003eactive_width = 2;\n+\t\tbreak;\n+\tcase MODULE_SPEED_100G_R4:\n+\t\tprops-\u003eactive_speed = XSC_RDMA_LINK_SPEED_25GB;\n+\t\tprops-\u003eactive_width = 2;\n+\t\tbreak;\n+\tcase MODULE_SPEED_200G_R4:\n+\t\tprops-\u003eactive_speed = XSC_RDMA_LINK_SPEED_50GB;\n+\t\tprops-\u003eactive_width = 2;\n+\t\tbreak;\n+\tcase MODULE_SPEED_200G_R8:\n+\t\tprops-\u003eactive_speed = XSC_RDMA_LINK_SPEED_25GB;\n+\t\tprops-\u003eactive_width = 4;\n+\t\tbreak;\n+\tcase MODULE_SPEED_400G_R8:\n+\t\tprops-\u003eactive_speed = XSC_RDMA_LINK_SPEED_50GB;\n+\t\tprops-\u003eactive_width = 4;\n+\t\tbreak;\n+\tcase MODULE_SPEED_400G_R4:\n+\t\tprops-\u003eactive_speed = XSC_RDMA_LINK_SPEED_100GB;\n+\t\tprops-\u003eactive_width = 2;\n+\t\tbreak;\n+\tdefault:\n+\t\tprops-\u003eactive_speed = XSC_RDMA_LINK_SPEED_25GB;\n+\t\tprops-\u003eactive_width = 1;\n+\t\tbreak;\n+\t}\n+}\n+\n+static enum rdma_link_layer xsc_ib_port_link_layer(struct ib_device *ibdev, u32 port)\n+{\n+\treturn IB_LINK_LAYER_ETHERNET;\n+}\n+\n+int xsc_ib_query_port(struct ib_device *ibdev, u32 port,\n+\t\t struct ib_port_attr *props)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(ibdev);\n+\tstruct net_device *ndev = dev-\u003enetdev;\n+\tstruct xsc_core_device *xdev = dev-\u003exdev;\n+\n+\tif (port \u003c 1 || port \u003e xdev-\u003ecaps.num_ports) {\n+\t\tibdev_warn(\u0026dev-\u003eib_dev, \"invalid port number %d\\n\", port);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tmemset(props, 0, sizeof(*props));\n+\n+\tif (netif_running(ndev) \u0026\u0026 netif_carrier_ok(ndev))\n+\t\tprops-\u003estate = IB_PORT_ACTIVE;\n+\telse\n+\t\tprops-\u003estate = IB_PORT_DOWN;\n+\n+\tprops-\u003emax_mtu = is_support_8k_mtu(xdev) ? IB_MTU_8192 : IB_MTU_4096;\n+\tprops-\u003eactive_mtu = min(props-\u003emax_mtu, xsc_net_to_ib_mtu(ndev-\u003emtu));\n+\tprops-\u003egid_tbl_len = XSC_IB_GID_TBL_LEN;\n+\tprops-\u003eport_cap_flags = IB_PORT_CM_SUP | IB_PORT_OTHER_LOCAL_CHANGES_SUP;\n+\tprops-\u003emax_msg_sz = SZ_1G;\n+\tprops-\u003ebad_pkey_cntr = 0;\n+\tprops-\u003eqkey_viol_cntr = 0;\n+\tprops-\u003epkey_tbl_len = 1;\n+\tprops-\u003elid = 0;\n+\tprops-\u003esm_lid = 0;\n+\tprops-\u003elmc = 0;\n+\tprops-\u003emax_vl_num = 0;\n+\tprops-\u003esm_sl = 0;\n+\tprops-\u003esubnet_timeout = 0;\n+\tprops-\u003einit_type_reply = 0;\n+\tif (!is_support_rdma(xdev)) {\n+\t\tprops-\u003eactive_width = 1;\n+\t\tprops-\u003eactive_speed = XSC_RDMA_LINK_SPEED_25GB;\n+\t} else {\n+\t\txsc_calc_link_info(xdev, props);\n+\t}\n+\n+\tprops-\u003ephys_state = netif_carrier_ok(ndev) ? XSC_RDMA_PHY_STATE_LINK_UP :\n+\t\t\t\tXSC_RDMA_PHY_STATE_DISABLED;\n+\treturn 0;\n+}\n+\n+static int xsc_ib_query_pkey(struct ib_device *ibdev, u32 port, u16 index,\n+\t\t\t u16 *pkey)\n+{\n+\t*pkey = 0xffff;\n+\treturn 0;\n+}\n+\n+int xsc_ib_alloc_ucontext(struct ib_ucontext *uctx,\n+\t\t\t struct ib_udata *udata)\n+{\n+\tstruct ib_device *ibdev = uctx-\u003edevice;\n+\tstruct xsc_ib_dev *dev = to_xdev(ibdev);\n+\tstruct xsc_ib_alloc_ucontext_req req;\n+\tstruct xsc_ib_alloc_ucontext_resp resp;\n+\tint err;\n+\n+\tif (!dev-\u003eib_active)\n+\t\treturn -EAGAIN;\n+\n+\terr = ib_copy_from_udata(\u0026req, udata, sizeof(req));\n+\tif (err)\n+\t\treturn err;\n+\n+\tmemset(\u0026resp, 0, sizeof(resp));\n+\tresp.qp_tab_size = dev-\u003exdev-\u003ecaps.max_qp;\n+\tresp.cache_line_size = L1_CACHE_BYTES;\n+\tresp.max_sq_desc_sz = dev-\u003exdev-\u003ecaps.max_sq_desc_sz;\n+\tresp.max_rq_desc_sz = dev-\u003exdev-\u003ecaps.max_rq_desc_sz;\n+\tresp.max_send_wqebb = dev-\u003exdev-\u003ecaps.max_wqes;\n+\tresp.max_recv_wr = dev-\u003exdev-\u003ecaps.max_wqes;\n+\txsc_get_db_addr(dev-\u003exdev, \u0026resp.qpm_tx_db, \u0026resp.qpm_rx_db,\n+\t\t\t\u0026resp.cqm_armdb, \u0026resp.cqm_next_cid_reg, NULL);\n+\tresp.send_ds_num = dev-\u003exdev-\u003ecaps.send_ds_num;\n+\tresp.recv_ds_num = dev-\u003exdev-\u003ecaps.recv_ds_num;\n+\tresp.cmds_supp_uhw |= XSC_USER_CMDS_SUPP_UHW_QUERY_DEVICE;\n+\tresp.device_id = dev-\u003exdev-\u003epdev-\u003edevice;\n+\n+\tif (req.abi_ver \u003e= ALLOC_UCONTEXT_VER_V1) {\n+\t\txsc_get_multidb_info(dev-\u003exdev, \u0026resp.multidb_num, \u0026resp.tx_multidb_base);\n+\t\tresp.rdma_proto_mode = dev-\u003exdev-\u003erdma_proto_mode;\n+\t}\n+\n+\tif (is_support_cqe64(dev-\u003exdev))\n+\t\tresp.hw_feature_flag |= XSC_HW_FEATURE_FLAG_SUPPORT_CQE64;\n+\n+\tresp.num_ports = dev-\u003exdev-\u003ecaps.num_ports;\n+\n+\treturn ib_copy_to_udata(udata, \u0026resp, min_t(int, udata-\u003eoutlen, sizeof(resp)));\n+}\n+\n+void xsc_ib_dealloc_ucontext(struct ib_ucontext *ibcontext)\n+{\n+}\n+\n+static int xsc_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vma)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(ibcontext-\u003edevice);\n+\tstruct xsc_core_device *xdev = dev-\u003exdev;\n+\tunsigned long offset = vma-\u003evm_pgoff \u003c\u003c PAGE_SHIFT;\n+\tresource_size_t reg_base;\n+\tresource_size_t reg_size = vma-\u003evm_end - vma-\u003evm_start;\n+\tu64 tx_db = 0;\n+\tu64 rx_db = 0;\n+\tu64 cq_db = 0;\n+\tu64 cq_reg = 0;\n+\n+\txsc_get_db_addr(xdev, \u0026tx_db, \u0026rx_db, \u0026cq_db, \u0026cq_reg, NULL);\n+\tdev_dbg(\u0026xdev-\u003epdev-\u003edev, \"offset:0x%lx\\n\", offset);\n+\n+\tif (offset == (tx_db \u0026 PAGE_MASK))\n+\t\treg_base = pci_resource_start(xdev-\u003epdev, xdev-\u003ebar_num) + (tx_db \u0026 PAGE_MASK);\n+\telse if (offset == (rx_db \u0026 PAGE_MASK))\n+\t\treg_base = pci_resource_start(xdev-\u003epdev, xdev-\u003ebar_num) + (rx_db \u0026 PAGE_MASK);\n+\telse if (offset == (cq_reg \u0026 PAGE_MASK))\n+\t\treg_base = pci_resource_start(xdev-\u003epdev, xdev-\u003ebar_num) + (cq_reg \u0026 PAGE_MASK);\n+\telse if (offset == (cq_db \u0026 PAGE_MASK))\n+\t\treg_base = pci_resource_start(xdev-\u003epdev, xdev-\u003ebar_num) + (cq_db \u0026 PAGE_MASK);\n+\telse if (offset == (QPM_TX_MDB_BASE_REG_ADDR \u0026 PAGE_MASK))\n+\t\treg_base = pci_resource_start(xdev-\u003epdev, xdev-\u003ebar_num) +\n+\t\t\t(QPM_TX_MDB_BASE_REG_ADDR \u0026 PAGE_MASK);\n+\telse if (offset == (xdev-\u003ecaps.mdb_base \u0026 PAGE_MASK))\n+\t\treg_base = pci_resource_start(xdev-\u003epdev, xdev-\u003ebar_num) +\n+\t\t\t(xdev-\u003ecaps.mdb_base \u0026 PAGE_MASK);\n+\telse\n+\t\treturn -EINVAL;\n+\n+\tdev_dbg(\u0026xdev-\u003epdev-\u003edev, \"regbase:0x%llx\", reg_base);\n+\n+\treg_base = (xsc_core_is_pf(xdev) \u0026\u0026 !is_pf_bar_compressed(xdev)) ?\n+\t\treg_base - 0xA0000000 : reg_base;\n+\n+\tvma-\u003evm_page_prot = pgprot_noncached(vma-\u003evm_page_prot);\n+\treturn remap_pfn_range(vma, vma-\u003evm_start, reg_base \u003e\u003e PAGE_SHIFT,\n+\t\t\t reg_size, vma-\u003evm_page_prot);\n+}\n+\n+int xsc_ib_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata)\n+{\n+\tstruct ib_device *ibdev = ibpd-\u003edevice;\n+\tstruct xsc_ib_alloc_pd_resp resp;\n+\tstruct xsc_ib_pd *pd;\n+\tint err;\n+\n+\tpd = to_xpd(ibpd);\n+\n+\terr = xsc_core_alloc_pd(to_xdev(ibdev)-\u003exdev, \u0026pd-\u003epdn);\n+\tif (err)\n+\t\treturn err;\n+\n+\tif (udata) {\n+\t\tresp.pdn = pd-\u003epdn;\n+\t\tif (ib_copy_to_udata(udata, \u0026resp, sizeof(resp))) {\n+\t\t\txsc_core_dealloc_pd(to_xdev(ibdev)-\u003exdev, pd-\u003epdn);\n+\t\t\treturn -EFAULT;\n+\t\t}\n+\t} else {\n+\t\tpd-\u003epa_lkey = 0;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+int xsc_ib_dealloc_pd(struct ib_pd *pd, struct ib_udata *udata)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(pd-\u003edevice);\n+\tstruct xsc_ib_pd *xpd = to_xpd(pd);\n+\n+\txsc_core_dealloc_pd(dev-\u003exdev, xpd-\u003epdn);\n+\n+\treturn 0;\n+}\n+\n+static int xsc_port_immutable(struct ib_device *ibdev, u32 port_num,\n+\t\t\t struct ib_port_immutable *immutable)\n+{\n+\tstruct ib_port_attr attr;\n+\tint err;\n+\n+\terr = ib_query_port(ibdev, port_num, \u0026attr);\n+\tif (err)\n+\t\treturn err;\n+\n+\timmutable-\u003epkey_tbl_len = attr.pkey_tbl_len;\n+\timmutable-\u003egid_tbl_len = attr.gid_tbl_len;\n+\timmutable-\u003ecore_cap_flags = RDMA_CORE_PORT_IBA_ROCE |\n+\t\t\t\t RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP;\n+\timmutable-\u003emax_mad_size = IB_MGMT_MAD_SIZE * 2;\n+\n+\treturn 0;\n+}\n+\n+static struct net_device *xsc_get_netdev(struct ib_device *ibdev, u32 port_num)\n+{\n+\tstruct xsc_ib_dev *xsc_ib_dev = to_xdev(ibdev);\n+\tstruct net_device *dev = xsc_ib_dev-\u003enetdev;\n+\n+\tif (dev)\n+\t\tdev_hold(dev);\n+\n+\treturn dev;\n+}\n+\n+static void xsc_get_guid(const u8 *dev_addr, u8 *guid)\n+{\n+\tu8 mac[ETH_ALEN];\n+\n+\t/* MAC-48 to EUI-64 mapping */\n+\tmemcpy(mac, dev_addr, ETH_ALEN);\n+\tguid[0] = mac[0] ^ 2;\n+\tguid[1] = mac[1];\n+\tguid[2] = mac[2];\n+\tguid[3] = 0xff;\n+\tguid[4] = 0xfe;\n+\tguid[5] = mac[3];\n+\tguid[6] = mac[4];\n+\tguid[7] = mac[5];\n+}\n+\n+static void init_node_data(struct xsc_ib_dev *dev)\n+{\n+\tstrscpy(dev-\u003eib_dev.node_desc, \"xsc_node_desc\",\n+\t\tsizeof(dev-\u003eib_dev.node_desc));\n+\txsc_get_guid(dev-\u003enetdev-\u003edev_addr, (u8 *)\u0026dev-\u003eib_dev.node_guid);\n+}\n+\n+static void xsc_ib_event(struct xsc_core_device *dev, void *context,\n+\t\t\t enum xsc_dev_event event, unsigned long data)\n+{\n+\tstruct xsc_ib_dev *ibdev = (struct xsc_ib_dev *)context;\n+\tstruct ib_event ibev;\n+\tu8 port = 0;\n+\n+\tswitch (event) {\n+\tcase XSC_DEV_EVENT_SYS_ERROR:\n+\t\tibdev-\u003eib_active = false;\n+\t\tibev.event = IB_EVENT_DEVICE_FATAL;\n+\t\tbreak;\n+\n+\tcase XSC_DEV_EVENT_PORT_UP:\n+\t\tibev.event = IB_EVENT_PORT_ACTIVE;\n+\t\tport = *(u8 *)data;\n+\t\tbreak;\n+\n+\tcase XSC_DEV_EVENT_PORT_DOWN:\n+\t\tibev.event = IB_EVENT_PORT_ERR;\n+\t\tport = *(u8 *)data;\n+\t\tbreak;\n+\n+\tcase XSC_DEV_EVENT_PORT_INITIALIZED:\n+\t\t/* not used by ULPs */\n+\t\treturn;\n+\n+\tcase XSC_DEV_EVENT_LID_CHANGE:\n+\t\tibev.event = IB_EVENT_LID_CHANGE;\n+\t\tport = *(u8 *)data;\n+\t\tbreak;\n+\n+\tcase XSC_DEV_EVENT_PKEY_CHANGE:\n+\t\tibev.event = IB_EVENT_PKEY_CHANGE;\n+\t\tport = *(u8 *)data;\n+\t\tbreak;\n+\n+\tcase XSC_DEV_EVENT_GUID_CHANGE:\n+\t\tibev.event = IB_EVENT_GID_CHANGE;\n+\t\tport = *(u8 *)data;\n+\t\tbreak;\n+\n+\tcase XSC_DEV_EVENT_CLIENT_REREG:\n+\t\tibev.event = IB_EVENT_CLIENT_REREGISTER;\n+\t\tport = *(u8 *)data;\n+\t\tbreak;\n+\t}\n+\n+\tibev.device\t = \u0026ibdev-\u003eib_dev;\n+\tibev.element.port_num = port;\n+\n+\tif (ibdev-\u003eib_active)\n+\t\tib_dispatch_event(\u0026ibev);\n+}\n+\n+static void crc_table_init(struct xsc_ib_dev *dev)\n+{\n+\tu32 c, i, j;\n+\n+\tfor (i = 0; i \u003c 256; i++) {\n+\t\tc = i;\n+\t\tfor (j = 0; j \u003c 8; j++) {\n+\t\t\tif (c \u0026 1)\n+\t\t\t\tc = 0xedb88320L ^ (c \u003e\u003e 1);\n+\t\t\telse\n+\t\t\t\tc = c \u003e\u003e 1;\n+\t\t}\n+\t\tdev-\u003ecrc_32_table[i] = c;\n+\t}\n+}\n+\n+static void xsc_ib_get_dev_fw_str(struct ib_device *ibdev, char *str)\n+{\n+\tstruct xsc_core_device *dev = to_xdev(ibdev)-\u003exdev;\n+\tu8 ver_major = dev-\u003efw_version_major;\n+\tu8 ver_minor = dev-\u003efw_version_minor;\n+\tu16 ver_patch = dev-\u003efw_version_patch;\n+\tu32 ver_tweak = dev-\u003efw_version_tweak;\n+\n+\tif (ver_tweak == 0) {\n+\t\tsnprintf(str, IB_FW_VERSION_NAME_MAX, \"v%u.%u.%u\",\n+\t\t\t ver_major, ver_minor, ver_patch);\n+\t} else {\n+\t\tsnprintf(str, IB_FW_VERSION_NAME_MAX, \"v%u.%u.%u+%u\",\n+\t\t\t ver_major, ver_minor, ver_patch, ver_tweak);\n+\t}\n+}\n+\n+static void xsc_ib_disassociate_ucontext(struct ib_ucontext *ibcontext)\n+{\n+}\n+\n+static void xsc_ib_dev_setting(struct xsc_ib_dev *dev)\n+{\n+\tdev-\u003eib_dev.ops.owner\t\t= THIS_MODULE;\n+\tdev-\u003eib_dev.ops.uverbs_abi_ver\t= XSC_IB_UVERBS_ABI_VERSION;\n+\tdev-\u003eib_dev.ops.driver_id = RDMA_DRIVER_XSC;\n+\tdev-\u003eib_dev.ops.disassociate_ucontext = xsc_ib_disassociate_ucontext;\n+\tdev-\u003eib_dev.ops.query_device\t= xsc_ib_query_device;\n+\tdev-\u003eib_dev.ops.query_port\t= xsc_ib_query_port;\n+\tdev-\u003eib_dev.ops.query_pkey\t= xsc_ib_query_pkey;\n+\n+\tdev-\u003eib_dev.ops.alloc_ucontext\t= xsc_ib_alloc_ucontext;\n+\tdev-\u003eib_dev.ops.dealloc_ucontext = xsc_ib_dealloc_ucontext;\n+\tdev-\u003eib_dev.ops.mmap\t\t= xsc_ib_mmap;\n+\n+\tdev-\u003eib_dev.ops.alloc_pd\t\t= xsc_ib_alloc_pd;\n+\tdev-\u003eib_dev.ops.dealloc_pd\t\t= xsc_ib_dealloc_pd;\n+\tdev-\u003eib_dev.ops.create_ah\t\t= xsc_ib_create_ah;\n+\tdev-\u003eib_dev.ops.query_ah\t\t= xsc_ib_query_ah;\n+\tdev-\u003eib_dev.ops.destroy_ah\t\t= xsc_ib_destroy_ah;\n+\n+\tdev-\u003eib_dev.ops.get_link_layer\t= xsc_ib_port_link_layer;\n+\tdev-\u003eib_dev.ops.get_netdev\t\t= xsc_get_netdev;\n+\n+\tdev-\u003eib_dev.ops.create_qp\t\t= xsc_ib_create_qp;\n+\tdev-\u003eib_dev.ops.modify_qp\t\t= xsc_ib_modify_qp;\n+\tdev-\u003eib_dev.ops.query_qp\t\t= xsc_ib_query_qp;\n+\tdev-\u003eib_dev.ops.destroy_qp\t\t= xsc_ib_destroy_qp;\n+\tdev-\u003eib_dev.ops.post_send\t\t= xsc_ib_post_send;\n+\tdev-\u003eib_dev.ops.post_recv\t\t= xsc_ib_post_recv;\n+\tdev-\u003eib_dev.ops.create_cq\t\t= xsc_ib_create_cq;\n+\tdev-\u003eib_dev.ops.destroy_cq\t\t= xsc_ib_destroy_cq;\n+\tdev-\u003eib_dev.ops.poll_cq\t\t= xsc_ib_poll_cq;\n+\tdev-\u003eib_dev.ops.req_notify_cq\t= xsc_ib_arm_cq;\n+\tdev-\u003eib_dev.ops.get_dma_mr\t\t= xsc_ib_get_dma_mr;\n+\tdev-\u003eib_dev.ops.reg_user_mr\t\t= xsc_ib_reg_user_mr; /* optional */\n+\tdev-\u003eib_dev.ops.reg_user_mr_dmabuf\t= xsc_ib_reg_user_mr_dmabuf;\n+\tdev-\u003eib_dev.ops.dereg_mr\t\t= xsc_ib_dereg_mr;\n+\n+\tdev-\u003eib_dev.ops.get_port_immutable\t\t= xsc_port_immutable;\n+\n+\tdev-\u003eib_dev.ops.drain_sq\t\t= xsc_ib_drain_sq;\n+\tdev-\u003eib_dev.ops.drain_rq\t\t= xsc_ib_drain_rq;\n+\tdev-\u003eib_dev.ops.get_dev_fw_str\t= xsc_ib_get_dev_fw_str;\n+\tdev-\u003eib_dev.ops.create_srq\t\t= xsc_ib_create_srq;\n+\tdev-\u003eib_dev.ops.destroy_srq\t\t= xsc_ib_destroy_srq;\n+\tdev-\u003eib_dev.ops.query_srq\t\t= xsc_ib_query_srq;\n+\tdev-\u003eib_dev.ops.modify_srq\t\t= xsc_ib_modify_srq;\n+\tdev-\u003eib_dev.ops.post_srq_recv\t= xsc_ib_post_srq_recv;\n+\n+\tdev-\u003eib_dev.ops INIT_RDMA_OBJ_SIZE(ib_ucontext, xsc_ib_ucontext, ibucontext);\n+\tdev-\u003eib_dev.ops INIT_RDMA_OBJ_SIZE(ib_ah, xsc_ib_ah, ibah);\n+\tdev-\u003eib_dev.ops INIT_RDMA_OBJ_SIZE(ib_cq, xsc_ib_cq, ibcq);\n+\tdev-\u003eib_dev.ops INIT_RDMA_OBJ_SIZE(ib_pd, xsc_ib_pd, ibpd);\n+\tdev-\u003eib_dev.ops INIT_RDMA_OBJ_SIZE(ib_qp, xsc_ib_qp, ibqp);\n+\tdev-\u003eib_dev.ops INIT_RDMA_OBJ_SIZE(ib_srq, xsc_ib_srq, ibsrq);\n+}\n+\n+static void xsc_get_port_state(struct net_device *ndev, enum xsc_dev_event *ev)\n+{\n+\t*ev = XSC_DEV_EVENT_PORT_DOWN;\n+\tif (netif_running(ndev) \u0026\u0026 netif_carrier_ok(ndev))\n+\t\t*ev = XSC_DEV_EVENT_PORT_UP;\n+}\n+\n+static int xsc_netdev_event(struct notifier_block *this, unsigned long event, void *ptr)\n+{\n+\tstruct xsc_ib_dev *ibdev = container_of(this, struct xsc_ib_dev, nb);\n+\tstruct net_device *ndev = netdev_notifier_info_to_dev(ptr);\n+\tenum xsc_dev_event ev;\n+\tu8 port = 1;\n+\n+\tif (ndev != ibdev-\u003enetdev)\n+\t\tgoto done;\n+\n+\tibdev_dbg(\u0026ibdev-\u003eib_dev, \"netdev notfiy event:%ld\\n\", event);\n+\tswitch (event) {\n+\tcase NETDEV_CHANGE:\n+\tcase NETDEV_UP:\n+\tcase NETDEV_DOWN:\n+\t\txsc_get_port_state(ibdev-\u003enetdev, \u0026ev);\n+\t\txsc_ib_event(ibdev-\u003exdev, ibdev, ev, (unsigned long)\u0026port);\n+\t\tbreak;\n+\tdefault:\n+\t\tbreak;\n+\t}\n+done:\n+\treturn NOTIFY_DONE;\n+}\n+\n+static int xsc_register_netdev_notifier(struct xsc_ib_dev *ibdev)\n+{\n+\tibdev-\u003enb.notifier_call = xsc_netdev_event;\n+\treturn register_netdevice_notifier(\u0026ibdev-\u003enb);\n+}\n+\n+static int xsc_unregister_netdev_notifier(struct xsc_ib_dev *ibdev)\n+{\n+\treturn unregister_netdevice_notifier(\u0026ibdev-\u003enb);\n+}\n+\n+static int init_one(struct xsc_core_device *xdev,\n+\t\t struct xsc_ib_dev **ibdev)\n+{\n+\tstruct xsc_ib_dev *dev;\n+\tint err;\n+\n+\tdev = (struct xsc_ib_dev *)ib_alloc_device(xsc_ib_dev, ib_dev);\n+\tif (!dev)\n+\t\treturn -ENOMEM;\n+\n+\tdev-\u003exdev = xdev;\n+\tdev-\u003enetdev = xdev-\u003enetdev;\n+\tstrscpy(dev-\u003eib_dev.name, \"xscale_%d\", IB_DEVICE_NAME_MAX);\n+\n+\tmemcpy(dev-\u003exdev-\u003epriv.ibdev_name, dev-\u003eib_dev.name, IB_DEVICE_NAME_MAX);\n+\n+\tif (!xdev-\u003ecaps.msix_enable)\n+\t\tdev-\u003enum_comp_vectors = 1;\n+\telse\n+\t\tdev-\u003enum_comp_vectors = xdev-\u003edev_res-\u003eeq_table.num_comp_vectors;\n+\n+\tdev-\u003eib_dev.node_type\t\t= RDMA_NODE_IB_CA;\n+\tdev-\u003eib_dev.local_dma_lkey\t= 0xFF;\n+\tdev-\u003enum_ports\t\t= xdev-\u003ecaps.num_ports;\n+\tdev-\u003eib_dev.phys_port_cnt = dev-\u003enum_ports;\n+\tdev-\u003eib_dev.num_comp_vectors\t= dev-\u003enum_comp_vectors;\n+\tdev-\u003eib_dev.dev.parent = \u0026xdev-\u003epdev-\u003edev;\n+\txsc_ib_dev_setting(dev);\n+\tdev-\u003exdev-\u003epriv.cm_dscp = DSCP_PCP_UNSET;\n+\tdev-\u003exdev-\u003epriv.cm_pcp = DSCP_PCP_UNSET;\n+\tdev-\u003exdev-\u003epriv.force_pcp = DSCP_PCP_UNSET;\n+\tdev-\u003exdev-\u003epriv.force_dscp = DSCP_PCP_UNSET;\n+\tdev-\u003exdev-\u003erdma_proto_mode = RDMA_PROTO_ROCEV2;\n+\tdev-\u003exdev-\u003ecm_udp_dst_port = ROCE_V2_UDP_DPORT;\n+\tdev-\u003exdev-\u003everoce_udp_dst_port = VEROCE_UDP_DPORT;\n+\n+\tdev-\u003eib_dev.uverbs_cmd_mask\t=\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_GET_CONTEXT)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_QUERY_DEVICE)\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_QUERY_PORT)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_ALLOC_PD)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_DEALLOC_PD)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_CREATE_AH)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_DESTROY_AH)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_REG_MR)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_REREG_MR)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_DEREG_MR)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_CREATE_COMP_CHANNEL)\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_CREATE_CQ)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_RESIZE_CQ)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_DESTROY_CQ)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_CREATE_QP)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_MODIFY_QP)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_QUERY_QP)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_DESTROY_QP)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_ATTACH_MCAST)\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_DETACH_MCAST)\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_CREATE_SRQ)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_MODIFY_SRQ)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_QUERY_SRQ)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_DESTROY_SRQ)\t\t|\n+\t\t(1ull \u003c\u003c IB_USER_VERBS_CMD_OPEN_QP);\n+\n+\tinit_node_data(dev);\n+\n+\tcrc_table_init(dev);\n+\n+\tbuild_opcode_map(dev);\n+\n+\txsc_reg_local_dma_mr(xdev);\n+\n+\terr = ib_register_device(\u0026dev-\u003eib_dev, dev-\u003eib_dev.name,\n+\t\t\t\t dev-\u003exdev-\u003edevice);\n+\tif (err)\n+\t\tgoto err_free;\n+\n+\trdma_roce_rescan_device(\u0026dev-\u003eib_dev);\n+\tdev-\u003eib_active = true;\n+\t*ibdev = dev;\n+\n+\txsc_register_netdev_notifier(dev);\n+\n+\txsc_counters_init(\u0026dev-\u003eib_dev, xdev);\n+\n+\txsc_ib_sysfs_init(\u0026dev-\u003eib_dev, xdev);\n+\n+\treturn 0;\n+\n+err_free:\n+\tib_dealloc_device((struct ib_device *)dev);\n+\n+\treturn err;\n+}\n+\n+static void remove_one(struct xsc_core_device *xdev, void *intf_ctx)\n+{\n+\tstruct xsc_ib_dev *dev = (struct xsc_ib_dev *)intf_ctx;\n+\n+\txsc_ib_sysfs_fini(\u0026dev-\u003eib_dev, xdev);\n+\txsc_counters_fini(\u0026dev-\u003eib_dev, xdev);\n+\txsc_unregister_netdev_notifier(dev);\n+\tib_unregister_device(\u0026dev-\u003eib_dev);\n+\tib_dealloc_device(\u0026dev-\u003eib_dev);\n+}\n+\n+static void init_iommu_state(struct xsc_ib_dev *dev)\n+{\n+\tstruct iommu_domain *domain;\n+\n+\tdev-\u003eiommu_state = XSC_IB_IOMMU_MAP_DISABLE;\n+\tdomain = iommu_get_domain_for_dev(dev-\u003eib_dev.dma_device);\n+\tif (domain) {\n+\t\tif (domain-\u003etype \u0026 __IOMMU_DOMAIN_DMA_API)\n+\t\t\tdev-\u003eiommu_state = XSC_IB_IOMMU_MAP_NORMAL;\n+\t} else {\n+\t\t/* Try to allocate DMA memory; if the DMA address differs from\n+\t\t * the physical address, an IOMMU maps it through an unknown\n+\t\t * domain.\n+\t\t */\n+\t\tdma_addr_t dma_addr;\n+\t\tvoid *tmp;\n+\n+\t\ttmp = dma_alloc_coherent(dev-\u003eib_dev.dma_device, PAGE_SIZE,\n+\t\t\t\t\t \u0026dma_addr, GFP_KERNEL);\n+\t\tif (tmp) {\n+\t\t\tif (virt_to_phys(tmp) != dma_addr)\n+\t\t\t\tdev-\u003eiommu_state = XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN;\n+\t\t\tdma_free_coherent(dev-\u003eib_dev.dma_device, PAGE_SIZE,\n+\t\t\t\t\t tmp, dma_addr);\n+\t\t}\n+\t}\n+\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"iommu dma map state=%d\\n\", dev-\u003eiommu_state);\n+}\n+\n+static int xsc_ib_probe(struct auxiliary_device *adev,\n+\t\t\tconst struct auxiliary_device_id *adev_id)\n+{\n+\tstruct xsc_adev *xsc_adev = container_of(adev, struct xsc_adev, adev);\n+\tstruct xsc_core_device *xdev = xsc_adev-\u003exdev;\n+\tstruct xsc_ib_dev *ibdev = NULL;\n+\tint ret;\n+\n+\tret = init_one(xdev, \u0026ibdev);\n+\tif (ret) {\n+\t\tdev_err(\u0026adev-\u003edev, \"failed to add RDMA device, ret = %d\\n\",\n+\t\t\tret);\n+\t\treturn ret;\n+\t}\n+\n+\tinit_iommu_state(ibdev);\n+\tauxiliary_set_drvdata(adev, ibdev);\n+\n+\treturn 0;\n+}\n+\n+static void xsc_ib_remove(struct auxiliary_device *adev)\n+{\n+\tstruct xsc_ib_dev *ibdev = auxiliary_get_drvdata(adev);\n+\n+\tif (!ibdev)\n+\t\treturn;\n+\n+\tremove_one(ibdev-\u003exdev, ibdev);\n+}\n+\n+static const struct auxiliary_device_id xsc_ib_id_table[] = {\n+\t{ .name = XSC_PCI_DRV_NAME \".\" XSC_IB_ADEV_NAME },\n+\t{},\n+};\n+MODULE_DEVICE_TABLE(auxiliary, xsc_ib_id_table);\n+\n+static struct auxiliary_driver xsc_ib_driver = {\n+\t.name = XSC_IB_ADEV_NAME,\n+\t.probe = xsc_ib_probe,\n+\t.remove = xsc_ib_remove,\n+\t.id_table = xsc_ib_id_table,\n+};\n+\n+static int xsc_ib_reboot_event_handler(struct notifier_block *nb, unsigned long action, void *data)\n+{\n+\tpr_info(\"xsc ib driver recv %lu event\\n\", action);\n+\n+\txsc_remove_rdma_driver();\n+\n+\treturn NOTIFY_OK;\n+}\n+\n+struct notifier_block xsc_ib_nb = {\n+\t.notifier_call = xsc_ib_reboot_event_handler,\n+\t.next = NULL,\n+\t.priority = 2,\n+};\n+\n+void xsc_remove_rdma_driver(void)\n+{\n+\tauxiliary_driver_unregister(\u0026xsc_ib_driver);\n+}\n+\n+static int __init xsc_ib_init(void)\n+{\n+\tint ret;\n+\n+\tret = auxiliary_driver_register(\u0026xsc_ib_driver);\n+\tif (ret) {\n+\t\tpr_err(\"failed to register auxiliary driver\\n\");\n+\t\treturn ret;\n+\t}\n+\n+\tregister_reboot_notifier(\u0026xsc_ib_nb);\n+\n+\treturn 0;\n+}\n+\n+static void __exit xsc_ib_cleanup(void)\n+{\n+\tunregister_reboot_notifier(\u0026xsc_ib_nb);\n+\txsc_remove_rdma_driver();\n+}\n+\n+module_init(xsc_ib_init);\n+module_exit(xsc_ib_cleanup);\ndiff --git a/drivers/infiniband/hw/xsc/mem.c b/drivers/infiniband/hw/xsc/mem.c\nnew file mode 100644\nindex 0000000000000..2fbce994339bc\n--- /dev/null\n+++ b/drivers/infiniband/hw/xsc/mem.c\n@@ -0,0 +1,161 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/module.h\u003e\n+#include \u003crdma/ib_umem.h\u003e\n+#include \"xsc_ib.h\"\n+\n+/**\n+ * __xsc_ib_cont_pages - scan a umem for compound pages\n+ * @umem: umem object to scan\n+ * @addr: ib virtual address requested by the user\n+ * @max_page_shift: upper limit for the detected page shift, 0 for none\n+ * @count: number of PAGE_SIZE pages covered by umem\n+ * @shift: page shift for the compound pages found in the region\n+ * @ncont: number of compound pages\n+ */\n+static void __xsc_ib_cont_pages(struct ib_umem *umem, u64 addr,\n+\t\t\t\tunsigned long max_page_shift,\n+\t\t\t\tint *count, int *shift, int *ncont)\n+{\n+\tunsigned long tmp;\n+\tunsigned long m;\n+\tu64 base = ~0, p = 0;\n+\tu64 len, pfn;\n+\tint i = 0;\n+\tstruct scatterlist *sg;\n+\tint entry;\n+\t/* TODO: need peer mem support */\n+\tunsigned long page_shift = PAGE_SHIFT;\n+\n+\taddr = addr \u003e\u003e page_shift;\n+\ttmp = (unsigned long)addr;\n+\tm = find_first_bit(\u0026tmp, BITS_PER_LONG);\n+\tif (max_page_shift)\n+\t\tm = min_t(unsigned long, max_page_shift - page_shift, m);\n+\tfor_each_sgtable_dma_sg(\u0026umem-\u003esgt_append.sgt, sg, entry) {\n+\t\tlen = sg_dma_len(sg) \u003e\u003e page_shift;\n+\t\tpfn = sg_dma_address(sg) \u003e\u003e page_shift;\n+\t\tif (base + p != pfn) {\n+\t\t\t/* If either the offset or the new\n+\t\t\t * base are unaligned update m\n+\t\t\t */\n+\t\t\ttmp = (unsigned long)(pfn | p);\n+\t\t\tif (!IS_ALIGNED(tmp, 1 \u003c\u003c m))\n+\t\t\t\tm = find_first_bit(\u0026tmp, BITS_PER_LONG);\n+\n+\t\t\tbase = pfn;\n+\t\t\tp = 0;\n+\t\t}\n+\n+\t\tp += len;\n+\t\ti += len;\n+\t}\n+\n+\tif (i) {\n+\t\tm = min_t(unsigned long, ilog2(roundup_pow_of_two(i)), m);\n+\t\t*ncont = DIV_ROUND_UP(i, (1 \u003c\u003c m));\n+\t} else {\n+\t\tm = 0;\n+\t\t*ncont = 0;\n+\t}\n+\t*shift = page_shift + m;\n+\t*count = i;\n+}\n+\n+void xsc_ib_cont_pages(struct ib_umem *umem, u64 addr,\n+\t\t int *count, int *shift, int *ncont)\n+{\n+\t/* no limit for page_shift */\n+\t__xsc_ib_cont_pages(umem, addr, 0, count, shift, ncont);\n+}\n+\n+static void __xsc_ib_populate_pas(struct xsc_ib_dev *dev, struct ib_umem *umem,\n+\t\t\t\t int page_shift, size_t offset, size_t num_pages,\n+\t\t\t\t __be64 *pas, int access_flags, bool need_to_devide)\n+{\n+\tunsigned long umem_page_shift = PAGE_SHIFT;\n+\tint shift = page_shift - umem_page_shift;\n+\tint mask = (1 \u003c\u003c shift) - 1;\n+\tint i = 0;\n+\tint k, idx;\n+\tu64 cur = 0;\n+\tu64 base;\n+\tint len;\n+\tstruct scatterlist *sg;\n+\tint entry;\n+\n+\tfor_each_sgtable_dma_sg(\u0026umem-\u003esgt_append.sgt, sg, entry) {\n+\t\tlen = sg_dma_len(sg) \u003e\u003e umem_page_shift;\n+\t\tif (need_to_devide)\n+\t\t\tlen = sg_dma_len(sg) \u003e\u003e PAGE_SHIFT_4K;\n+\t\telse\n+\t\t\tlen = sg_dma_len(sg) \u003e\u003e umem_page_shift;\n+\t\tbase = sg_dma_address(sg);\n+\n+\t\t/* Skip elements below offset */\n+\t\tif (i + len \u003c offset \u003c\u003c shift) {\n+\t\t\ti += len;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/* Skip pages below offset */\n+\t\tif (i \u003c offset \u003c\u003c shift) {\n+\t\t\tk = (offset \u003c\u003c shift) - i;\n+\t\t\ti = offset \u003c\u003c shift;\n+\t\t} else {\n+\t\t\tk = 0;\n+\t\t}\n+\n+\t\tfor (; k \u003c len; k++) {\n+\t\t\tif (!(i \u0026 mask)) {\n+\t\t\t\tif (need_to_devide)\n+\t\t\t\t\tcur = base + (k \u003c\u003c PAGE_SHIFT_4K);\n+\t\t\t\telse\n+\t\t\t\t\tcur = base + (k \u003c\u003c umem_page_shift);\n+\t\t\t\tcur |= access_flags;\n+\t\t\t\tidx = (i \u003e\u003e shift) - offset;\n+\n+\t\t\t\tpas[idx] = cpu_to_be64(cur);\n+\t\t\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"pas[%d] 0x%llx\\n\",\n+\t\t\t\t\t i \u003e\u003e shift, be64_to_cpu(pas[idx]));\n+\t\t\t}\n+\t\t\ti++;\n+\n+\t\t\t/* Stop after num_pages reached */\n+\t\t\tif (i \u003e\u003e shift \u003e= offset + num_pages)\n+\t\t\t\treturn;\n+\t\t}\n+\t}\n+}\n+\n+void xsc_ib_populate_pas(struct xsc_ib_dev *dev, struct ib_umem *umem,\n+\t\t\t int page_shift, __be64 *pas, int npages, bool need_to_devide)\n+{\n+\treturn __xsc_ib_populate_pas(dev, umem, page_shift, 0,\n+\t\t\t\t npages, pas, 0, need_to_devide);\n+}\n+\n+int xsc_ib_get_buf_offset(u64 addr, int page_shift, u32 *offset)\n+{\n+\tu64 page_size;\n+\tu64 page_mask;\n+\tu64 off_size;\n+\tu64 off_mask;\n+\tu64 buf_off;\n+\n+\tpage_size = 1 \u003c\u003c page_shift;\n+\tpage_mask = page_size - 1;\n+\tbuf_off = addr \u0026 page_mask;\n+\toff_size = page_size \u003e\u003e 6;\n+\toff_mask = off_size - 1;\n+\n+\tif (buf_off \u0026 off_mask)\n+\t\treturn -EINVAL;\n+\n+\t*offset = buf_off \u003e\u003e ilog2(off_size);\n+\treturn 0;\n+}\ndiff --git a/drivers/infiniband/hw/xsc/mr.c b/drivers/infiniband/hw/xsc/mr.c\nnew file mode 100644\nindex 0000000000000..efd685e579c44\n--- /dev/null\n+++ b/drivers/infiniband/hw/xsc/mr.c\n@@ -0,0 +1,313 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/kref.h\u003e\n+#include \u003clinux/random.h\u003e\n+#include \u003clinux/debugfs.h\u003e\n+#include \u003clinux/export.h\u003e\n+#include \u003clinux/dma-buf.h\u003e\n+#include \u003clinux/dma-resv.h\u003e\n+#include \u003crdma/ib_umem.h\u003e\n+#include \u003crdma/iter.h\u003e\n+#include \"common/xsc_cmd.h\"\n+#include \u003clinux/dma-direct.h\u003e\n+#include \"xsc_ib.h\"\n+\n+enum {\n+\tDEF_CACHE_SIZE\t= 10,\n+};\n+\n+struct ib_mr *xsc_ib_get_dma_mr(struct ib_pd *pd, int acc)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(pd-\u003edevice);\n+\tstruct xsc_core_device *xdev = dev-\u003exdev;\n+\tstruct xsc_register_mr_mbox_in *in;\n+\tstruct xsc_register_mr_request *req;\n+\tstruct xsc_ib_mr *mr;\n+\tint err;\n+\n+\tmr = kzalloc_obj(*mr, GFP_KERNEL);\n+\tif (!mr)\n+\t\treturn ERR_PTR(-ENOMEM);\n+\n+\tin = kzalloc_obj(*in, GFP_KERNEL);\n+\tif (!in) {\n+\t\terr = -ENOMEM;\n+\t\tgoto err_free;\n+\t}\n+\n+\treq = \u0026in-\u003ereq;\n+\treq-\u003eacc = convert_access(acc);\n+\treq-\u003eva_base = 0;\n+\treq-\u003emap_en = !(XSC_MPT_MAP_EN);\n+\n+\terr = xsc_core_create_mkey(xdev, \u0026mr-\u003exmr);\n+\tif (err)\n+\t\tgoto err_in;\n+\treq-\u003emkey = cpu_to_be32(mr-\u003exmr.key);\n+\terr = xsc_core_register_mr(xdev, \u0026mr-\u003exmr, in, sizeof(*in));\n+\tif (err)\n+\t\tgoto err_reg_mr;\n+\tkfree(in);\n+\tmr-\u003eibmr.lkey = mr-\u003exmr.key;\n+\tmr-\u003eibmr.rkey = mr-\u003exmr.key;\n+\tmr-\u003eumem = NULL;\n+\n+\treturn \u0026mr-\u003eibmr;\n+err_reg_mr:\n+\txsc_core_destroy_mkey(xdev, \u0026mr-\u003exmr);\n+err_in:\n+\tkfree(in);\n+\n+err_free:\n+\tkfree(mr);\n+\n+\treturn ERR_PTR(err);\n+}\n+\n+static struct xsc_ib_mr *reg_create(struct ib_pd *pd, u64 virt_addr,\n+\t\t\t\t u64 length, struct ib_umem *umem,\n+\t\t\t\t int npages, int page_shift,\n+\t\t\t\t int access_flags)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(pd-\u003edevice);\n+\tstruct xsc_register_mr_mbox_in *in;\n+\tstruct xsc_ib_mr *mr;\n+\tstruct ib_block_iter biter;\n+\t__be64 *pas;\n+\tint inlen;\n+\tint err;\n+\n+\tmr = kzalloc_obj(*mr, GFP_KERNEL);\n+\tif (!mr) {\n+\t\terr = -ENOMEM;\n+\t\tgoto err_0;\n+\t}\n+\n+\tinlen = sizeof(*in) + sizeof(*in-\u003ereq.pas) * npages;\n+\tin = xsc_vzalloc(inlen);\n+\tif (!in) {\n+\t\terr = -ENOMEM;\n+\t\tgoto err_1;\n+\t}\n+\terr = xsc_core_create_mkey(dev-\u003exdev, \u0026mr-\u003exmr);\n+\tif (err) {\n+\t\tibdev_warn(\u0026dev-\u003eib_dev, \"create mkey failed\\n\");\n+\t\tgoto err_2;\n+\t}\n+\n+\tpas = in-\u003ereq.pas;\n+\trdma_umem_for_each_dma_block(umem, \u0026biter, 1UL \u003c\u003c page_shift) {\n+\t\t*pas = cpu_to_be64(rdma_block_iter_dma_address(\u0026biter));\n+\t\tpas++;\n+\t}\n+\n+\tin-\u003ereq.acc = convert_access(access_flags);\n+\tin-\u003ereq.pa_num = cpu_to_be32(npages);\n+\tin-\u003ereq.pdn = cpu_to_be32(to_xpd(pd)-\u003epdn);\n+\tin-\u003ereq.va_base = cpu_to_be64(virt_addr);\n+\tin-\u003ereq.map_en = XSC_MPT_MAP_EN;\n+\tin-\u003ereq.len = cpu_to_be64(length);\n+\tin-\u003ereq.page_mode = xsc_get_mr_page_mode(dev-\u003exdev, page_shift);\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"read_flush hwconfig %s\\n\",\n+\t\t dev-\u003exdev-\u003eread_flush ? \"enable\" : \"disable\");\n+\tin-\u003ereq.is_gpu = 0;\n+\tin-\u003ereq.mkey = cpu_to_be32(mr-\u003exmr.key);\n+\terr = xsc_core_register_mr(dev-\u003exdev, \u0026mr-\u003exmr, in, inlen);\n+\tif (err) {\n+\t\tibdev_warn(\u0026dev-\u003eib_dev, \"register mr failed, err = %d\\n\", err);\n+\t\tgoto err_reg_mr;\n+\t}\n+\tmr-\u003eumem = umem;\n+\txsc_vfree(in);\n+\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"mkey = 0x%x\\n\", mr-\u003exmr.key);\n+\n+\treturn mr;\n+err_reg_mr:\n+\txsc_core_destroy_mkey(dev-\u003exdev, \u0026mr-\u003exmr);\n+err_2:\n+\txsc_vfree(in);\n+err_1:\n+\tkfree(mr);\n+err_0:\n+\treturn ERR_PTR(err);\n+}\n+\n+struct ib_mr *xsc_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,\n+\t\t\t\t u64 virt_addr, int access_flags,\n+\t\t\t\t struct ib_dmah *dmah,\n+\t\t\t\t struct ib_udata *udata)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(pd-\u003edevice);\n+\tstruct xsc_ib_mr *mr = NULL;\n+\tstruct ib_umem *umem;\n+\tunsigned long pgsz;\n+\tint page_shift;\n+\tint npages;\n+\tint err;\n+\n+\tif (dmah)\n+\t\treturn ERR_PTR(-EOPNOTSUPP);\n+\n+\tif (length \u003e dev-\u003exdev-\u003ecaps.max_mr_size) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"reg user mr length(%llu) exceeded.\\n\", length);\n+\t\treturn ERR_PTR(-EINVAL);\n+\t}\n+\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"start 0x%llx, virt_addr 0x%llx, length 0x%llx\\n\",\n+\t\t start, virt_addr, length);\n+\n+\tumem = ib_umem_get_va(\u0026dev-\u003eib_dev, start, length, access_flags);\n+\tif (IS_ERR(umem)) {\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"umem get failed\\n\");\n+\t\treturn (void *)umem;\n+\t}\n+\n+\tpgsz = ib_umem_find_best_pgsz(umem, XSC_MR_PAGE_CAP_MASK, virt_addr);\n+\tif (!pgsz) {\n+\t\tibdev_warn(\u0026dev-\u003eib_dev, \"find best page size failed\\n\");\n+\t\terr = -EINVAL;\n+\t\tgoto error;\n+\t}\n+\n+\tpage_shift = order_base_2(pgsz);\n+\tnpages = ib_umem_num_dma_blocks(umem, pgsz);\n+\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"npages %d, page_shift %d\\n\", npages, page_shift);\n+\n+\tmr = reg_create(pd, virt_addr, length, umem, npages, page_shift,\n+\t\t\taccess_flags);\n+\tif (IS_ERR(mr)) {\n+\t\terr = PTR_ERR(mr);\n+\t\tgoto error;\n+\t}\n+\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"mkey 0x%x\\n\", mr-\u003exmr.key);\n+\n+\tmr-\u003eumem = umem;\n+\tmr-\u003enpages = npages;\n+\tmr-\u003eibmr.lkey = mr-\u003exmr.key;\n+\tmr-\u003eibmr.rkey = mr-\u003exmr.key;\n+\tmr-\u003eibmr.length = length;\n+\treturn \u0026mr-\u003eibmr;\n+\n+error:\n+\tib_umem_release(umem);\n+\treturn ERR_PTR(err);\n+}\n+\n+struct ib_mr *xsc_ib_reg_user_mr_dmabuf(struct ib_pd *pd, u64 offset,\n+\t\t\t\t\tu64 length, u64 virt_addr,\n+\t\t\t\t\tint fd, int access_flags,\n+\t\t\t\t\tstruct ib_dmah *dmah,\n+\t\t\t\t\tstruct uverbs_attr_bundle *attrs)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(pd-\u003edevice);\n+\tstruct ib_umem_dmabuf *umem_dmabuf;\n+\tstruct ib_umem *umem;\n+\tstruct xsc_ib_mr *mr;\n+\tunsigned long pgsz;\n+\tint page_shift;\n+\tint npages;\n+\tint err;\n+\n+\tif (!IS_ENABLED(CONFIG_INFINIBAND_USER_MEM))\n+\t\treturn ERR_PTR(-EOPNOTSUPP);\n+\n+\tif (dmah)\n+\t\treturn ERR_PTR(-EOPNOTSUPP);\n+\n+\tumem_dmabuf = ib_umem_dmabuf_get_pinned(\u0026dev-\u003eib_dev,\n+\t\t\t\t\t\toffset, length,\n+\t\t\t\t\t\tfd, access_flags);\n+\tif (IS_ERR(umem_dmabuf)) {\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"umem_dmabuf get failed (%ld)\\n\",\n+\t\t\t PTR_ERR(umem_dmabuf));\n+\t\treturn ERR_CAST(umem_dmabuf);\n+\t}\n+\tumem = \u0026umem_dmabuf-\u003eumem;\n+\n+\tpgsz = ib_umem_find_best_pgsz(umem, XSC_MR_PAGE_CAP_MASK, virt_addr);\n+\tif (!pgsz) {\n+\t\tibdev_warn(\u0026dev-\u003eib_dev, \"find best page size failed\\n\");\n+\t\terr = -EINVAL;\n+\t\tgoto err_release;\n+\t}\n+\n+\tpage_shift = order_base_2(pgsz);\n+\tnpages = ib_umem_num_dma_blocks(umem, pgsz);\n+\n+\tmr = reg_create(pd, virt_addr, length, umem, npages, page_shift,\n+\t\t\taccess_flags);\n+\tif (IS_ERR(mr)) {\n+\t\terr = PTR_ERR(mr);\n+\t\tgoto err_release;\n+\t}\n+\n+\tmr-\u003eumem = umem;\n+\tmr-\u003enpages = npages;\n+\tmr-\u003eibmr.lkey = mr-\u003exmr.key;\n+\tmr-\u003eibmr.rkey = mr-\u003exmr.key;\n+\tmr-\u003eibmr.length = length;\n+\n+\treturn \u0026mr-\u003eibmr;\n+err_release:\n+\tib_umem_release(umem);\n+\treturn ERR_PTR(err);\n+}\n+\n+int xsc_ib_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(ibmr-\u003edevice);\n+\tstruct xsc_ib_mr *mr = to_xmr(ibmr);\n+\tstruct ib_umem *umem = mr-\u003eumem;\n+\tint err;\n+\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"dereg mkey = 0x%x\\n\", mr-\u003exmr.key);\n+\n+\tif (mr-\u003enpages) {\n+\t\terr = xsc_core_dereg_mr(dev-\u003exdev, \u0026mr-\u003exmr);\n+\t\tif (err) {\n+\t\t\tibdev_warn(\u0026dev-\u003eib_dev, \"failed to dereg mr 0x%x (%d)\\n\",\n+\t\t\t\t mr-\u003exmr.key, err);\n+\t\t\treturn err;\n+\t\t}\n+\t}\n+\terr = xsc_core_destroy_mkey(dev-\u003exdev, \u0026mr-\u003exmr);\n+\tif (err) {\n+\t\tibdev_warn(\u0026dev-\u003eib_dev, \"failed to destroy mkey 0x%x (%d)\\n\",\n+\t\t\t mr-\u003exmr.key, err);\n+\t\treturn err;\n+\t}\n+\n+\tif (umem)\n+\t\tib_umem_release(umem);\n+\n+\tkfree(mr);\n+\n+\treturn 0;\n+}\n+\n+void xsc_reg_local_dma_mr(struct xsc_core_device *dev)\n+{\n+\tstruct xsc_register_mr_mbox_in in;\n+\tint err = 0;\n+\n+\tin.req.pdn = 0;\n+\tin.req.pa_num = 0;\n+\tin.req.len = 0;\n+\tin.req.mkey = cpu_to_be32(0xFF);\n+\tin.req.acc = XSC_PERM_LOCAL_WRITE | XSC_PERM_LOCAL_READ;\n+\tin.req.page_mode = xsc_get_mr_page_mode(dev, PAGE_SHIFT_4K);\n+\tin.req.is_gpu = 0;\n+\tin.req.map_en = !(XSC_MPT_MAP_EN);\n+\tin.req.va_base = 0;\n+\n+\terr = xsc_core_register_mr(dev, NULL, \u0026in, sizeof(in));\n+\tif (err)\n+\t\tdev_err(\u0026dev-\u003epdev-\u003edev, \"\\n\");\n+}\ndiff --git a/drivers/infiniband/hw/xsc/qp.c b/drivers/infiniband/hw/xsc/qp.c\nnew file mode 100644\nindex 0000000000000..9c79e09a7fd1f\n--- /dev/null\n+++ b/drivers/infiniband/hw/xsc/qp.c\n@@ -0,0 +1,2251 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/module.h\u003e\n+#include \u003crdma/ib_umem.h\u003e\n+#include \"xsc_ib.h\"\n+#include \u003crdma/xsc-abi.h\u003e\n+#include \"common/xsc_hsi.h\"\n+#include \u003clinux/crc32.h\u003e\n+#include \u003clinux/dma-direct.h\u003e\n+#include \u003crdma/ib_addr.h\u003e\n+\n+#ifndef ETH_P_IBOE\n+#define ETH_P_IBOE\t0x8915\n+#endif\n+\n+#define MAD_QUEUE_DEPTH 128\n+\n+enum {\n+\tXSC_IB_CACHE_LINE_SIZE\t= 64,\n+};\n+\n+#define UDP_SPORT_MASK_EN\t\t0x40000000\n+#define UDP_SPORT_MASK_EN_OFFSET\t30\n+#define UDP_SPORT_MASK\t\t\t0xffff\n+#define UDP_SPORT_OFFSET\t\t0\n+\n+void build_opcode_map(struct xsc_ib_dev *dev)\n+{\n+\tstruct xsc_msg_opcode *xsc_ib_opcode = dev-\u003exdev-\u003eboard_info-\u003emsg_opcode;\n+\n+\tdev-\u003eopcode_map[IB_WR_SEND] = xsc_ib_opcode-\u003esend;\n+\tdev-\u003eopcode_map[IB_WR_SEND_WITH_IMM] = xsc_ib_opcode-\u003esend_with_imm;\n+\tdev-\u003eopcode_map[IB_WR_RDMA_WRITE] = xsc_ib_opcode-\u003erdma_write;\n+\tdev-\u003eopcode_map[IB_WR_RDMA_WRITE_WITH_IMM] = xsc_ib_opcode-\u003erdma_write_with_imm;\n+\tdev-\u003eopcode_map[IB_WR_RDMA_READ] = xsc_ib_opcode-\u003erdma_read;\n+\tdev-\u003eopcode_map[IB_WR_ATOMIC_CMP_AND_SWP] = xsc_ib_opcode-\u003eatomic_cmp_and_swp;\n+\tdev-\u003eopcode_map[IB_WR_ATOMIC_FETCH_AND_ADD] = xsc_ib_opcode-\u003eatomic_fetch_and_add;\n+\tdev-\u003eopcode_map[IB_WR_MASKED_ATOMIC_CMP_AND_SWP] =\n+\t\txsc_ib_opcode-\u003emasked_atomic_cmp_and_swp;\n+\tdev-\u003eopcode_map[IB_WR_MASKED_ATOMIC_FETCH_AND_ADD] =\n+\t\txsc_ib_opcode-\u003emasked_atomic_fetch_and_add;\n+}\n+\n+static int is_qp0(enum ib_qp_type qp_type)\n+{\n+\treturn qp_type == IB_QPT_SMI;\n+}\n+\n+static int is_qp1(enum ib_qp_type qp_type)\n+{\n+\treturn qp_type == IB_QPT_GSI;\n+}\n+\n+static int is_sqp(enum ib_qp_type qp_type)\n+{\n+\treturn is_qp0(qp_type) || is_qp1(qp_type);\n+}\n+\n+static void *get_wqe(struct xsc_ib_qp *qp, int offset)\n+{\n+\treturn xsc_buf_offset(\u0026qp-\u003ebuf, offset);\n+}\n+\n+static void *get_recv_wqe(struct xsc_ib_qp *qp, int n)\n+{\n+\treturn get_wqe(qp, qp-\u003erq.offset + (n \u003c\u003c qp-\u003erq.wqe_shift));\n+}\n+\n+static void *get_seg_wqe(void *first, int n)\n+{\n+\treturn first + (n \u003c\u003c XSC_BASE_WQE_SHIFT);\n+}\n+\n+void *xsc_get_send_wqe(struct xsc_ib_qp *qp, int n)\n+{\n+\treturn get_wqe(qp, qp-\u003esq.offset + (n \u003c\u003c qp-\u003esq.wqe_shift));\n+}\n+\n+static int iboe_tos_to_sl(struct net_device *ndev, int tos)\n+{\n+\tint prio;\n+\tstruct net_device *dev;\n+\n+\tprio = rt_tos2priority(tos);\n+\tdev = is_vlan_dev(ndev) ? vlan_dev_real_dev(ndev) : ndev;\n+\tif (dev-\u003enum_tc)\n+\t\treturn netdev_get_prio_tc_map(dev, prio);\n+\n+#if IS_ENABLED(CONFIG_VLAN_8021Q)\n+\tif (is_vlan_dev(ndev))\n+\t\treturn (vlan_dev_get_egress_qos_mask(ndev, prio) \u0026\n+\t\t\t\tVLAN_PRIO_MASK) \u003e\u003e VLAN_PRIO_SHIFT;\n+#endif\n+\treturn 0;\n+}\n+\n+static inline void set_remote_addr_seg(struct xsc_core_device *xdev,\n+\t\t\t\t struct xsc_wqe_data_seg *remote_seg,\n+\t\t\t\t u32 msg_len, u64 remote_addr, u32 rkey)\n+{\n+\txsc_set_data_seg(xdev, remote_seg, remote_addr, rkey, msg_len);\n+}\n+\n+static void set_local_data_seg(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_wqe_data_seg *data_seg, struct ib_sge *sg)\n+{\n+\txsc_set_data_seg(xdev, data_seg, sg-\u003eaddr, sg-\u003elkey, sg-\u003elength);\n+}\n+\n+static int set_data_inl_seg(struct xsc_ib_qp *qp, const struct ib_send_wr *wr, void *ctrl)\n+{\n+\tstruct xsc_wqe_data_seg *data_seg;\n+\tunsigned int seg_index;\n+\tvoid *addr;\n+\tint len;\n+\tint i;\n+\n+\tfor (i = 0, seg_index = 1; i \u003c wr-\u003enum_sge; ++i, ++seg_index) {\n+\t\tif (likely(wr-\u003esg_list[i].length)) {\n+\t\t\taddr = (void *)wr-\u003esg_list[i].addr;\n+\t\t\tlen = wr-\u003esg_list[i].length;\n+\n+\t\t\tif (unlikely(len \u003e qp-\u003emax_inline_data))\n+\t\t\t\treturn -ENOMEM;\n+\n+\t\t\tdata_seg = get_seg_wqe(ctrl, seg_index);\n+\t\t\tdata_seg-\u003ein_line = 1;\n+\t\t\tdata_seg-\u003elen = len;\n+\t\t\tmemcpy(data_seg-\u003ein_line_data, addr, len);\n+\t\t}\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static __be32 send_ieth(const struct ib_send_wr *wr)\n+{\n+\tswitch (wr-\u003eopcode) {\n+\tcase IB_WR_SEND_WITH_IMM:\n+\tcase IB_WR_RDMA_WRITE_WITH_IMM:\n+\t\treturn wr-\u003eex.imm_data;\n+\tdefault:\n+\t\treturn 0;\n+\t}\n+}\n+\n+static void xsc_ib_qp_event(struct xsc_core_qp *qp, int type)\n+{\n+\tstruct ib_qp *ibqp = \u0026core_to_xqp(qp)-\u003eibqp;\n+\tstruct ib_event event;\n+\n+\tif (ibqp-\u003eevent_handler) {\n+\t\tevent.device = ibqp-\u003edevice;\n+\t\tevent.element.qp = ibqp;\n+\t\tswitch (type) {\n+\t\tcase XSC_EVENT_TYPE_WQ_CATAS_ERROR:\n+\t\t\tevent.event = IB_EVENT_QP_FATAL;\n+\t\t\tbreak;\n+\t\tcase XSC_EVENT_TYPE_WQ_INVAL_REQ_ERROR:\n+\t\t\tevent.event = IB_EVENT_QP_REQ_ERR;\n+\t\t\tbreak;\n+\t\tcase XSC_EVENT_TYPE_WQ_ACCESS_ERROR:\n+\t\t\tevent.event = IB_EVENT_QP_ACCESS_ERR;\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tpr_warn(\"xsc_ib: Unexpected event type %d on QP %06x\\n\", type, qp-\u003eqpn);\n+\t\t\treturn;\n+\t\t}\n+\n+\t\tibqp-\u003eevent_handler(\u0026event, ibqp-\u003eqp_context);\n+\t\tpr_info(\"xsc_ib: xsc async event 0x%x, ib event %d\\n\", type, event.event);\n+\t}\n+}\n+\n+static int set_rq_size(struct xsc_ib_dev *dev, struct ib_qp_init_attr *init_attr,\n+\t\t int has_rq, struct xsc_ib_qp *qp, struct xsc_ib_create_qp *ucmd)\n+{\n+\tbool check_res = false;\n+\tu32 wqe_cnt = init_attr-\u003ecap.max_recv_wr ?\n+\t\troundup_pow_of_two(init_attr-\u003ecap.max_recv_wr) : 0;\n+\n+\tif (init_attr-\u003esrq)\n+\t\twqe_cnt = to_xsrq(init_attr-\u003esrq)-\u003exsrq.max;\n+\n+\t/* Sanity check RQ size before proceeding */\n+\tcheck_res = xsc_check_max_qp_depth(dev-\u003exdev, \u0026wqe_cnt, dev-\u003exdev-\u003ecaps.max_wqes);\n+\tif (check_res) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"max_recv_wr:%d exceed max rq depth\\n\",\n+\t\t\t init_attr-\u003ecap.max_recv_wr);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tif (!has_rq) {\n+\t\tqp-\u003erq.max_gs = 0;\n+\t\tqp-\u003erq.wqe_cnt = 0;\n+\t\tqp-\u003erq.wqe_shift = 0;\n+\t} else {\n+\t\tif (ucmd) {\n+\t\t\tqp-\u003erq.wqe_cnt = ucmd-\u003erq_wqe_count;\n+\t\t\tqp-\u003erq.wqe_shift = ucmd-\u003erq_wqe_shift;\n+\t\t\tqp-\u003erq.max_gs = 1;\n+\t\t\tqp-\u003erq.max_post = qp-\u003erq.wqe_cnt;\n+\t\t} else {\n+\t\t\tqp-\u003erq.wqe_cnt = wqe_cnt;\n+\t\t\tif (qp-\u003exqp.qp_type == IB_QPT_GSI)\n+\t\t\t\tqp-\u003erq.wqe_shift = ilog2(xsc_get_recv_ds_num(dev-\u003exdev))\n+\t\t\t\t\t+ XSC_RECV_WQE_SHIFT;\n+\t\t\telse\n+\t\t\t\tqp-\u003erq.wqe_shift = dev-\u003exdev-\u003ecaps.recv_wqe_shift;\n+\t\t\tqp-\u003erq.max_gs = dev-\u003exdev-\u003ecaps.recv_ds_num;\n+\t\t\tqp-\u003erq.max_post = qp-\u003erq.wqe_cnt;\n+\t\t}\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int calc_sq_size(struct xsc_ib_dev *dev, struct ib_qp_init_attr *attr,\n+\t\t\tstruct xsc_ib_qp *qp)\n+{\n+\tint wqe_size;\n+\tint wq_size;\n+\tbool check_res = false;\n+\n+\tif (!attr-\u003ecap.max_send_wr) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"invalid max_send_wr:%d\\n\", attr-\u003ecap.max_send_wr);\n+\t\treturn -1;\n+\t}\n+\n+\twqe_size = 1 \u003c\u003c dev-\u003exdev-\u003ecaps.send_wqe_shift;\n+\tqp-\u003emax_inline_data = (dev-\u003exdev-\u003ecaps.send_ds_num - 2) * sizeof(struct xsc_wqe_data_seg);\n+\tattr-\u003ecap.max_inline_data = qp-\u003emax_inline_data;\n+\n+\tattr-\u003ecap.max_send_wr = min_t(u32, attr-\u003ecap.max_send_wr, dev-\u003exdev-\u003ecaps.max_wqes);\n+\tqp-\u003esq.wqe_cnt = roundup_pow_of_two(attr-\u003ecap.max_send_wr);\n+\n+\tcheck_res = xsc_check_max_qp_depth(dev-\u003exdev, \u0026qp-\u003esq.wqe_cnt, dev-\u003exdev-\u003ecaps.max_wqes);\n+\tif (check_res) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"max_send_wr:%d exceed max sq depth\\n\",\n+\t\t\t attr-\u003ecap.max_send_wr);\n+\t\treturn -EINVAL;\n+\t}\n+\tqp-\u003esq.ds_cnt = qp-\u003esq.wqe_cnt \u003c\u003c (dev-\u003exdev-\u003ecaps.send_wqe_shift - XSC_BASE_WQE_SHIFT);\n+\twq_size = qp-\u003esq.wqe_cnt * wqe_size;\n+\tqp-\u003esq.wqe_shift = ilog2(wqe_size);\n+\tqp-\u003esq.max_gs = dev-\u003exdev-\u003ecaps.send_ds_num - XSC_CTRL_SEG_NUM - XSC_RADDR_SEG_NUM;\n+\tqp-\u003esq.max_post = qp-\u003esq.wqe_cnt;\n+\n+\treturn wq_size;\n+}\n+\n+static int qp_has_rq(struct ib_qp_init_attr *attr)\n+{\n+\tif (!attr-\u003esrq \u0026\u0026 !attr-\u003ecap.max_recv_wr)\n+\t\treturn 0;\n+\n+\treturn 1;\n+}\n+\n+static enum xsc_qp_state to_xsc_state(enum ib_qp_state state)\n+{\n+\tswitch (state) {\n+\tcase IB_QPS_RESET:\treturn XSC_QP_STATE_RST;\n+\tcase IB_QPS_INIT:\treturn XSC_QP_STATE_INIT;\n+\tcase IB_QPS_RTR:\treturn XSC_QP_STATE_RTR;\n+\tcase IB_QPS_RTS:\treturn XSC_QP_STATE_RTS;\n+\tcase IB_QPS_SQD:\treturn XSC_QP_STATE_SQD;\n+\tcase IB_QPS_SQE:\treturn XSC_QP_STATE_SQER;\n+\tcase IB_QPS_ERR:\treturn XSC_QP_STATE_ERR;\n+\tdefault:\t\treturn -1;\n+\t}\n+}\n+\n+static char *qp_state_to_str(enum ib_qp_state state)\n+{\n+\tswitch (state) {\n+\tcase IB_QPS_RESET:\treturn \"RST\";\n+\tcase IB_QPS_INIT:\treturn \"INIT\";\n+\tcase IB_QPS_RTR:\treturn \"RTR\";\n+\tcase IB_QPS_RTS:\treturn \"RTS\";\n+\tcase IB_QPS_SQD:\treturn \"SQD\";\n+\tcase IB_QPS_SQE:\treturn \"SQE\";\n+\tcase IB_QPS_ERR:\treturn \"ERR\";\n+\tdefault:\t\treturn \"UNKNOWN\";\n+\t}\n+}\n+\n+static struct ib_umem *qp_umem_get(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp,\n+\t\t\t\t struct ib_udata *udata, struct ib_pd *pd,\n+\tstruct xsc_ib_create_qp *ucmd)\n+{\n+\tstruct ib_umem *umem;\n+\n+\tumem = ib_umem_get_va(\u0026dev-\u003eib_dev, ucmd-\u003ebuf_addr, qp-\u003ebuf_size,\n+\t\t\t IB_ACCESS_LOCAL_WRITE);\n+\treturn umem;\n+}\n+\n+static int create_user_qp(struct xsc_ib_dev *dev, struct ib_pd *pd,\n+\t\t\t struct xsc_ib_qp *qp, struct ib_udata *udata,\n+\t\t\t struct xsc_create_qp_mbox_in **in,\n+\t\t\t struct xsc_ib_create_qp_resp *resp, int *inlen)\n+{\n+\tstruct xsc_ib_create_qp ucmd;\n+\tint page_shift;\n+\tint npages;\n+\tu32 offset;\n+\tint ncont;\n+\tint err;\n+\tint hw_npages;\n+\tstruct ib_umem_dmabuf *umem_dmabuf;\n+\n+\terr = ib_copy_from_udata(\u0026ucmd, udata, sizeof(ucmd));\n+\tif (err) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"failed to copy from udata, err=%d\\n\", err);\n+\t\treturn err;\n+\t}\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"buf_addr:0x%lx db_addr:0x%lx sq cnt:%u, rq cnt:%u, rq shift:%u\\n\",\n+\t\t (uintptr_t)ucmd.buf_addr, (uintptr_t)ucmd.db_addr,\n+\t\t ucmd.sq_wqe_count, ucmd.rq_wqe_count, ucmd.rq_wqe_shift);\n+\n+\tqp-\u003esq.ds_cnt = ucmd.sq_wqe_count;\n+\tqp-\u003esq.wqe_cnt = ucmd.sq_wqe_count;\n+\tqp-\u003esq.wqe_shift = XSC_BASE_WQE_SHIFT;\n+\tqp-\u003erq.ds_cnt = ucmd.rq_wqe_count;\n+\tqp-\u003erq.wqe_cnt = ucmd.rq_wqe_count;\n+\tqp-\u003erq.wqe_shift = XSC_BASE_WQE_SHIFT;\n+\n+\tqp-\u003ebuf_size = (qp-\u003esq.wqe_cnt \u003c\u003c qp-\u003esq.wqe_shift) + (qp-\u003erq.wqe_cnt \u003c\u003c qp-\u003erq.wqe_shift);\n+\n+\tif (ucmd.dmabuf_fd != -1) {\n+\t\tumem_dmabuf = ib_umem_dmabuf_get_pinned(\u0026dev-\u003eib_dev, 0, ucmd.dmabuf_sz,\n+\t\t\t\t\t\t\tucmd.dmabuf_fd, IB_ACCESS_LOCAL_WRITE);\n+\n+\t\tif (IS_ERR(umem_dmabuf)) {\n+\t\t\tibdev_warn(\u0026dev-\u003eib_dev, \"qp umem_dmabuf get failed (%ld)\\n\",\n+\t\t\t\t PTR_ERR(umem_dmabuf));\n+\t\t\treturn -ENOMEM;\n+\t\t}\n+\n+\t\tqp-\u003eumem = \u0026umem_dmabuf-\u003eumem;\n+\t} else {\n+\t\tqp-\u003eumem = qp_umem_get(dev, qp, udata, pd, \u0026ucmd);\n+\t}\n+\n+\tif (IS_ERR(qp-\u003eumem)) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"umem_get failed\\n\");\n+\t\terr = PTR_ERR(qp-\u003eumem);\n+\t\tgoto err_uuar;\n+\t}\n+\n+\txsc_ib_cont_pages(qp-\u003eumem, ucmd.buf_addr, \u0026npages, \u0026page_shift,\n+\t\t\t \u0026ncont);\n+\tif (ncont != npages) {\n+\t\t/* TODO: peer memory support failed */\n+\t\tpage_shift = PAGE_SHIFT;\n+\t\tncont = npages;\n+\t}\n+\n+\thw_npages = DIV_ROUND_UP(qp-\u003ebuf_size, PAGE_SIZE_4K);\n+\terr = xsc_ib_get_buf_offset(ucmd.buf_addr, page_shift, \u0026offset);\n+\tif (err) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"bad offset:%d\\n\", offset);\n+\t\tgoto err_umem;\n+\t}\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"npage:%d, page_shift:%d, ncont:%d, offset:%d, hw_npages %d\\n\",\n+\t\t npages, page_shift, ncont, offset, hw_npages);\n+\n+\t*inlen = sizeof(**in) + sizeof(*((*in)-\u003ereq.pas)) * hw_npages;\n+\t*in = xsc_vzalloc(*inlen);\n+\tif (!*in) {\n+\t\terr = -ENOMEM;\n+\t\tgoto err_umem;\n+\t}\n+\txsc_ib_populate_pas(dev, qp-\u003eumem, page_shift, (*in)-\u003ereq.pas, hw_npages, true);\n+\t(*in)-\u003ereq.pa_num = cpu_to_be16(hw_npages);\n+\n+\terr = ib_copy_to_udata(udata, resp, sizeof(*resp));\n+\tif (err) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"failed to copy to udata, err=%d\\n\", err);\n+\t\tgoto err_umem;\n+\t}\n+\tqp-\u003ecreate_type = XSC_QP_USER;\n+\n+\treturn 0;\n+\n+err_umem:\n+\tib_umem_release(qp-\u003eumem);\n+\n+err_uuar:\n+\treturn err;\n+}\n+\n+static void destroy_qp_user(struct ib_pd *__maybe_unused pd, struct xsc_ib_qp *qp)\n+{\n+\tib_umem_release(qp-\u003eumem);\n+}\n+\n+#define MAX_QP1_SQ_HDR_SIZE_V2\t512\n+#define MAX_QP1_SQ_HDR_SIZE\t86\n+\t/* Ethernet header\t= 14 */\n+\t/* ib_grh\t\t= 40 (provided by MAD) */\n+\t/* ib_bth + ib_deth\t= 20 */\n+\t/* MAD\t\t\t= 256 (provided by MAD) */\n+\t/* iCRC\t\t\t= 4 */\n+#define MAX_QP1_RQ_HDR_SIZE_V2\t512\n+\n+static int create_kernel_qp(struct xsc_ib_dev *dev,\n+\t\t\t struct ib_qp_init_attr *init_attr,\n+\t\t\t struct xsc_ib_qp *qp,\n+\t\t\t struct xsc_create_qp_mbox_in **in, int *inlen)\n+{\n+\tint err;\n+\tint sq_size;\n+\tint hw_npages;\n+\n+\tsq_size = calc_sq_size(dev, init_attr, qp);\n+\tif (sq_size \u003c 0) {\n+\t\terr = -EINVAL;\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"err %d\\n\", err);\n+\t\treturn err;\n+\t}\n+\n+\tqp-\u003erq.ds_cnt = qp-\u003erq.wqe_cnt \u003c\u003c (qp-\u003erq.wqe_shift - XSC_BASE_WQE_SHIFT);\n+\tqp-\u003erq.offset = 0;\n+\tqp-\u003esq.offset = qp-\u003erq.wqe_cnt \u003c\u003c qp-\u003erq.wqe_shift;\n+\tqp-\u003ebuf_size = qp-\u003esq.offset + sq_size;\n+\tqp-\u003esend_psn = 0;\n+\n+\terr = xsc_buf_alloc(dev-\u003exdev, qp-\u003ebuf_size, PAGE_SIZE, \u0026qp-\u003ebuf);\n+\tif (err) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"failed to alloc qp buffer,err=%d\\n\", err);\n+\t\treturn err;\n+\t}\n+\n+\tqp-\u003esq.qend = qp-\u003ebuf.direct.buf + qp-\u003esq.offset + sq_size;\n+\thw_npages = DIV_ROUND_UP(qp-\u003ebuf_size, PAGE_SIZE_4K);\n+\t*inlen = sizeof(**in) + sizeof(*(*in)-\u003ereq.pas) * hw_npages;\n+\t*in = xsc_vzalloc(*inlen);\n+\tif (!*in) {\n+\t\terr = -ENOMEM;\n+\t\tgoto err_buf;\n+\t}\n+\n+\txsc_fill_page_array(\u0026qp-\u003ebuf, (*in)-\u003ereq.pas, hw_npages);\n+\t(*in)-\u003ereq.pa_num = cpu_to_be16(hw_npages);\n+\n+\tqp-\u003esq.wrid = kcalloc(qp-\u003esq.wqe_cnt, sizeof(*qp-\u003esq.wrid), GFP_KERNEL);\n+\tqp-\u003esq.wr_data = kcalloc(qp-\u003esq.wqe_cnt, sizeof(*qp-\u003esq.wr_data), GFP_KERNEL);\n+\tqp-\u003erq.wrid = kcalloc(qp-\u003erq.wqe_cnt, sizeof(*qp-\u003erq.wrid), GFP_KERNEL);\n+\tqp-\u003esq.w_list = kcalloc(qp-\u003esq.wqe_cnt, sizeof(*qp-\u003esq.w_list), GFP_KERNEL);\n+\tqp-\u003esq.wqe_head = kcalloc(qp-\u003esq.wqe_cnt, sizeof(*qp-\u003esq.wqe_head), GFP_KERNEL);\n+\tqp-\u003esq.wr_opcode = kcalloc(qp-\u003esq.wqe_cnt, sizeof(*qp-\u003esq.wr_opcode), GFP_KERNEL);\n+\tqp-\u003esq.need_flush = kcalloc(qp-\u003esq.wqe_cnt, sizeof(*qp-\u003esq.need_flush), GFP_KERNEL);\n+\tqp-\u003esq.prev_ce_wqe = kcalloc(qp-\u003esq.wqe_cnt, sizeof(*qp-\u003esq.prev_ce_wqe), GFP_KERNEL);\n+\tqp-\u003esq.last_comp_ce_wqe = -1;\n+\tqp-\u003esq.last_post_ce_wqe = -1;\n+\n+\tif (!qp-\u003esq.wrid || !qp-\u003esq.wr_data || !qp-\u003erq.wrid || !qp-\u003esq.prev_ce_wqe ||\n+\t !qp-\u003esq.w_list || !qp-\u003esq.wqe_head || !qp-\u003esq.wr_opcode || !qp-\u003esq.need_flush) {\n+\t\terr = -ENOMEM;\n+\t\tgoto err_wrid;\n+\t}\n+\tqp-\u003ecreate_type = XSC_QP_KERNEL;\n+\n+\tif (init_attr-\u003eqp_type == IB_QPT_GSI) {\n+\t\tqp-\u003esq.mad_index = 0;\n+\t\tqp-\u003esq.mad_queue_depth = MAD_QUEUE_DEPTH;\n+\t\tqp-\u003esq.hdr_size = MAX_QP1_SQ_HDR_SIZE_V2 * MAD_QUEUE_DEPTH;\n+\t\tqp-\u003esq.hdr_buf = dma_alloc_coherent(dev-\u003eib_dev.dma_device,\n+\t\t\t\t\t\t qp-\u003esq.hdr_size,\n+\t\t\t\t\t\t \u0026qp-\u003esq.hdr_dma,\n+\t\t\t\t\t\t GFP_KERNEL);\n+\t\tif (!qp-\u003esq.hdr_buf) {\n+\t\t\terr = -ENOMEM;\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"Failed to create sq_hdr_buf\");\n+\t\t\tgoto err_wrid;\n+\t\t}\n+\t}\n+\n+\treturn 0;\n+\n+err_wrid:\n+\tkfree(qp-\u003esq.prev_ce_wqe);\n+\tkfree(qp-\u003esq.need_flush);\n+\tkfree(qp-\u003esq.wr_opcode);\n+\tkfree(qp-\u003esq.wqe_head);\n+\tkfree(qp-\u003esq.w_list);\n+\tkfree(qp-\u003esq.wrid);\n+\tkfree(qp-\u003esq.wr_data);\n+\tkfree(qp-\u003erq.wrid);\n+\n+err_buf:\n+\txsc_buf_free(dev-\u003exdev, \u0026qp-\u003ebuf);\n+\treturn err;\n+}\n+\n+static void destroy_qp_kernel(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp)\n+{\n+\tif (qp-\u003esq.hdr_buf)\n+\t\tdma_free_coherent(dev-\u003eib_dev.dma_device, qp-\u003esq.hdr_size,\n+\t\t\t\t qp-\u003esq.hdr_buf, qp-\u003esq.hdr_dma);\n+\tkfree(qp-\u003esq.prev_ce_wqe);\n+\tkfree(qp-\u003esq.need_flush);\n+\tkfree(qp-\u003esq.wr_opcode);\n+\tkfree(qp-\u003esq.wqe_head);\n+\tkfree(qp-\u003esq.w_list);\n+\tkfree(qp-\u003esq.wrid);\n+\tkfree(qp-\u003esq.wr_data);\n+\tkfree(qp-\u003erq.wrid);\n+\txsc_buf_free(dev-\u003exdev, \u0026qp-\u003ebuf);\n+}\n+\n+static u8 ib_to_xsc_qp_type(enum ib_qp_type qp_type, __u32 flags)\n+{\n+\tif (qp_type == IB_QPT_RC) {\n+\t\tif (flags \u0026 XSC_QP_FLAG_WORC)\n+\t\t\treturn XSC_QUEUE_TYPE_WORC;\n+\t\telse\n+\t\t\treturn XSC_QUEUE_TYPE_RDMA_RC;\n+\t} else if ((qp_type == IB_QPT_GSI) || (qp_type == IB_QPT_SMI)) {\n+\t\treturn XSC_QUEUE_TYPE_RDMA_MAD;\n+\t} else if (qp_type == IB_QPT_RAW_PACKET) {\n+\t\tif (flags \u0026 XSC_QP_FLAG_RAWPACKET_TSO)\n+\t\t\treturn XSC_QUEUE_TYPE_RAW_TSO;\n+\t\telse if (flags \u0026 XSC_QP_FLAG_RAWPACKET_TX)\n+\t\t\treturn XSC_QUEUE_TYPE_RAW_TX;\n+\t\telse if (flags \u0026 XSC_QP_FLAG_RAWPACKET_SNIFFER)\n+\t\t\treturn XSC_QUEUE_TYPE_SNIFFER;\n+\t\telse\n+\t\t\treturn XSC_QUEUE_TYPE_RAW;\n+\t} else {\n+\t\treturn XSC_QUEUE_TYPE_INVALID;\n+\t}\n+}\n+\n+static void get_cqs(enum ib_qp_type qp_type,\n+\t\t struct ib_cq *ib_send_cq, struct ib_cq *ib_recv_cq,\n+\t\t struct xsc_ib_cq **send_cq, struct xsc_ib_cq **recv_cq)\n+{\n+\tswitch (qp_type) {\n+\tcase IB_QPT_SMI:\n+\tcase IB_QPT_GSI:\n+\tcase IB_QPT_RC:\n+\tcase IB_QPT_UC:\n+\tcase IB_QPT_UD:\n+\tcase IB_QPT_RAW_PACKET:\n+\t\t*send_cq = ib_send_cq ? to_xcq(ib_send_cq) : NULL;\n+\t\t*recv_cq = ib_recv_cq ? to_xcq(ib_recv_cq) : NULL;\n+\t\tbreak;\n+\tdefault:\n+\t\t*send_cq = NULL;\n+\t\t*recv_cq = NULL;\n+\t\tbreak;\n+\t}\n+}\n+\n+static void xsc_ib_lock_cqs(struct xsc_ib_cq *send_cq, struct xsc_ib_cq *recv_cq)\n+\t__acquires(\u0026send_cq-\u003elock) __acquires(\u0026recv_cq-\u003elock)\n+{\n+\tif (send_cq) {\n+\t\tif (recv_cq) {\n+\t\t\tif (send_cq-\u003excq.cqn \u003c recv_cq-\u003excq.cqn) {\n+\t\t\t\tspin_lock_irq(\u0026send_cq-\u003elock);\n+\t\t\t\tspin_lock_nested(\u0026recv_cq-\u003elock,\n+\t\t\t\t\t\t SINGLE_DEPTH_NESTING);\n+\t\t\t} else if (send_cq-\u003excq.cqn == recv_cq-\u003excq.cqn) {\n+\t\t\t\tspin_lock_irq(\u0026send_cq-\u003elock);\n+\t\t\t\t__acquire(\u0026recv_cq-\u003elock);\n+\t\t\t} else {\n+\t\t\t\tspin_lock_irq(\u0026recv_cq-\u003elock);\n+\t\t\t\tspin_lock_nested(\u0026send_cq-\u003elock,\n+\t\t\t\t\t\t SINGLE_DEPTH_NESTING);\n+\t\t\t}\n+\t\t} else {\n+\t\t\tspin_lock_irq(\u0026send_cq-\u003elock);\n+\t\t}\n+\t} else if (recv_cq) {\n+\t\tspin_lock_irq(\u0026recv_cq-\u003elock);\n+\t}\n+}\n+\n+static void xsc_ib_unlock_cqs(struct xsc_ib_cq *send_cq, struct xsc_ib_cq *recv_cq)\n+\t__releases(\u0026send_cq-\u003elock) __releases(\u0026recv_cq-\u003elock)\n+{\n+\tif (send_cq) {\n+\t\tif (recv_cq) {\n+\t\t\tif (send_cq-\u003excq.cqn \u003c recv_cq-\u003excq.cqn) {\n+\t\t\t\tspin_unlock(\u0026recv_cq-\u003elock);\n+\t\t\t\tspin_unlock_irq(\u0026send_cq-\u003elock);\n+\t\t\t} else if (send_cq-\u003excq.cqn == recv_cq-\u003excq.cqn) {\n+\t\t\t\t__release(\u0026recv_cq-\u003elock);\n+\t\t\t\tspin_unlock_irq(\u0026send_cq-\u003elock);\n+\t\t\t} else {\n+\t\t\t\tspin_unlock(\u0026send_cq-\u003elock);\n+\t\t\t\tspin_unlock_irq(\u0026recv_cq-\u003elock);\n+\t\t\t}\n+\t\t} else {\n+\t\t\tspin_unlock_irq(\u0026send_cq-\u003elock);\n+\t\t}\n+\t} else if (recv_cq) {\n+\t\tspin_unlock_irq(\u0026recv_cq-\u003elock);\n+\t}\n+}\n+\n+static int create_qp_common(struct xsc_ib_dev *dev, struct ib_pd *pd,\n+\t\t\t struct ib_qp_init_attr *init_attr,\n+\t\t\t struct ib_udata *udata, struct xsc_ib_qp *qp)\n+{\n+\tstruct xsc_ib_create_qp_resp resp;\n+\tstruct xsc_create_qp_mbox_in *in = NULL;\n+\tstruct xsc_ib_create_qp ucmd;\n+\tstruct xsc_ib_cq *send_cq;\n+\tstruct xsc_ib_cq *recv_cq;\n+\tstruct xsc_srq_qp_node *qp_node;\n+\tint inlen = sizeof(*in);\n+\tint err;\n+\tchar buf[256];\n+\tchar *ptr = buf;\n+\tint ret = 0;\n+\n+\tmutex_init(\u0026qp-\u003emutex);\n+\tspin_lock_init(\u0026qp-\u003esq.lock);\n+\tspin_lock_init(\u0026qp-\u003erq.lock);\n+\tspin_lock_init(\u0026qp-\u003elock);\n+\n+\tif (init_attr-\u003esq_sig_type == IB_SIGNAL_ALL_WR)\n+\t\tqp-\u003esq_signal_bits = XSC_WQE_CTRL_CQ_UPDATE;\n+\n+\tif (pd-\u003euobject) {\n+\t\tif (ib_copy_from_udata(\u0026ucmd, udata, sizeof(ucmd))) {\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"failed to copy from udata\\n\");\n+\t\t\treturn -EFAULT;\n+\t\t}\n+\t}\n+\n+\tqp-\u003ehas_rq = qp_has_rq(init_attr);\n+\n+\terr = set_rq_size(dev, init_attr, qp-\u003ehas_rq,\n+\t\t\t qp, pd-\u003euobject ? \u0026ucmd : NULL);\n+\tif (err) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"failed to set rq size %d\\n\", err);\n+\t\treturn err;\n+\t}\n+\n+\tif (pd-\u003euobject) {\n+\t\terr = create_user_qp(dev, pd, qp, udata, \u0026in, \u0026resp, \u0026inlen);\n+\t\tif (err)\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"failed to create user qp, err = %d\\n\", err);\n+\t} else {\n+\t\terr = create_kernel_qp(dev, init_attr, qp, \u0026in, \u0026inlen);\n+\t\tif (err)\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"failed to create kernel qp, err = %d\\n\", err);\n+\t\telse\n+\t\t\tqp-\u003epa_lkey = to_xpd(pd)-\u003epa_lkey;\n+\t}\n+\n+\tif (err)\n+\t\treturn err;\n+\n+\tif (is_sqp(init_attr-\u003eqp_type))\n+\t\tqp-\u003eport = init_attr-\u003eport_num;\n+\n+\tif (is_qp1(init_attr-\u003eqp_type))\n+\t\tin-\u003ereq.input_qpn = cpu_to_be16(1);\n+\n+\tin-\u003ereq.pdn = cpu_to_be32(to_xpd(pd)-\u003epdn);\n+\n+\tif (qp-\u003erq.ds_cnt)\n+\t\tin-\u003ereq.log_rq_sz = ilog2(qp-\u003erq.ds_cnt);\n+\n+\tif (qp-\u003esq.ds_cnt)\n+\t\tin-\u003ereq.log_sq_sz = ilog2(qp-\u003esq.ds_cnt);\n+\telse\n+\t\tin-\u003ereq.log_sq_sz = ilog2(0x80);\n+\n+\tif (init_attr-\u003esend_cq) {\n+\t\tqp-\u003esend_cq = init_attr-\u003esend_cq;\n+\t\tin-\u003ereq.cqn_send = to_xcq(init_attr-\u003esend_cq)-\u003excq.cqn;\n+\t\tin-\u003ereq.cqn_send = cpu_to_be16(in-\u003ereq.cqn_send);\n+\t}\n+\n+\tif (init_attr-\u003erecv_cq) {\n+\t\tqp-\u003erecv_cq = init_attr-\u003erecv_cq;\n+\t\tin-\u003ereq.cqn_recv = to_xcq(init_attr-\u003erecv_cq)-\u003excq.cqn;\n+\t\tin-\u003ereq.cqn_recv = cpu_to_be16(in-\u003ereq.cqn_recv);\n+\t}\n+\n+\tin-\u003ereq.qp_type = ib_to_xsc_qp_type(init_attr-\u003eqp_type, pd-\u003euobject ?\n+\t\t\t\t\t ucmd.flags : 0);\n+\n+\tif (in-\u003ereq.qp_type == XSC_QUEUE_TYPE_INVALID) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"invalid qp type:%d\\n\", init_attr-\u003eqp_type);\n+\t\tgoto err_create;\n+\t}\n+\tin-\u003ereq.glb_funcid = cpu_to_be16(dev-\u003exdev-\u003eglb_func_id);\n+\tin-\u003ereq.page_shift = PAGE_SHIFT;\n+\n+\tqp-\u003exqp.qp_type_internal = in-\u003ereq.qp_type;\n+\n+\terr = xsc_core_create_qp(dev-\u003exdev, \u0026qp-\u003exqp, in, inlen);\n+\tif (err) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"create qp failed, err=%d\\n\", err);\n+\t\tgoto err_create;\n+\t}\n+\n+\tqp-\u003edoorbell_qpn = qp-\u003exqp.qpn;\n+\n+\tqp-\u003exqp.event = xsc_ib_qp_event;\n+\tqp-\u003exqp.qp_type = init_attr-\u003eqp_type;\n+\tret += snprintf(ptr + ret, 256 - ret, \"pdn=%d,\", to_xpd(pd)-\u003epdn);\n+\tret += snprintf(ptr + ret, 256 - ret, \"log_rq_sz=%d,\", in-\u003ereq.log_rq_sz);\n+\tret += snprintf(ptr + ret, 256 - ret, \"log_sq_sz=%d,\", in-\u003ereq.log_sq_sz);\n+\tret += snprintf(ptr + ret, 256 - ret, \"scqn=%d,\", to_xcq(init_attr-\u003esend_cq)-\u003excq.cqn);\n+\tret += snprintf(ptr + ret, 256 - ret, \"rcqn=%d\", to_xcq(init_attr-\u003erecv_cq)-\u003excq.cqn);\n+\n+\tget_cqs(init_attr-\u003eqp_type, init_attr-\u003esend_cq, init_attr-\u003erecv_cq, \u0026send_cq, \u0026recv_cq);\n+\txsc_ib_lock_cqs(send_cq, recv_cq);\n+\tif (send_cq)\n+\t\tlist_add_tail(\u0026qp-\u003ecq_send_list, \u0026send_cq-\u003esend_qp_list);\n+\tif (recv_cq)\n+\t\tlist_add_tail(\u0026qp-\u003ecq_recv_list, \u0026recv_cq-\u003erecv_qp_list);\n+\txsc_ib_unlock_cqs(send_cq, recv_cq);\n+\n+\tif (!pd-\u003euobject \u0026\u0026 init_attr-\u003esrq) {\n+\t\tqp_node = kvzalloc_obj(*qp_node, GFP_KERNEL);\n+\t\tif (!qp_node)\n+\t\t\tgoto err_create;\n+\t\tqp_node-\u003eibqp = \u0026qp-\u003eibqp;\n+\t\tqp_node-\u003eqp_id = qp-\u003exqp.qpn;\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"qpn:%u add into srq_qp_list\\n\", qp_node-\u003eqp_id);\n+\t\tlist_add_tail(\u0026qp_node-\u003eentry, \u0026to_xsrq(init_attr-\u003esrq)-\u003exsrq.srq_qp_list);\n+\t}\n+\n+\tqp-\u003exqp.qp_protocol_mode = dev-\u003exdev-\u003erdma_proto_mode;\n+\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"succeeded to create qp:%d, %s\\n\", qp-\u003exqp.qpn, buf);\n+\n+\txsc_vfree(in);\n+\n+\treturn 0;\n+\n+err_create:\n+\tif (qp-\u003ecreate_type == XSC_QP_USER)\n+\t\tdestroy_qp_user(pd, qp);\n+\telse if (qp-\u003ecreate_type == XSC_QP_KERNEL)\n+\t\tdestroy_qp_kernel(dev, qp);\n+\n+\txsc_vfree(in);\n+\treturn err;\n+}\n+\n+static struct xsc_ib_pd *get_pd(struct xsc_ib_qp *qp)\n+{\n+\treturn to_xpd(qp-\u003eibqp.pd);\n+}\n+\n+static void destroy_qp_common(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp)\n+{\n+\tstruct xsc_ib_cq *send_cq = NULL;\n+\tstruct xsc_ib_cq *recv_cq = NULL;\n+\tstruct xsc_modify_qp_mbox_in *in;\n+\tint err;\n+\n+\tin = kzalloc_obj(*in, GFP_KERNEL);\n+\tif (!in)\n+\t\treturn;\n+\n+\tif (qp-\u003exqp.qp_type_internal == XSC_QUEUE_TYPE_RAW ||\n+\t qp-\u003exqp.qp_type_internal == XSC_QUEUE_TYPE_RAW_TSO ||\n+\t qp-\u003exqp.qp_type_internal == XSC_QUEUE_TYPE_RAW_TX ||\n+\t qp-\u003estate != IB_QPS_RESET)\n+\t\tif (xsc_core_qp_modify(dev-\u003exdev, to_xsc_state(qp-\u003estate),\n+\t\t\t\t XSC_QP_STATE_RST, in, sizeof(*in), \u0026qp-\u003exqp))\n+\t\t\tibdev_warn(\u0026dev-\u003eib_dev, \"modify QP %06x to RESET failed\\n\", qp-\u003exqp.qpn);\n+\n+\tget_cqs(qp-\u003exqp.qp_type, qp-\u003eibqp.send_cq, qp-\u003eibqp.recv_cq, \u0026send_cq, \u0026recv_cq);\n+\txsc_ib_lock_cqs(send_cq, recv_cq);\n+\tif (send_cq)\n+\t\tlist_del(\u0026qp-\u003ecq_send_list);\n+\n+\tif (recv_cq)\n+\t\tlist_del(\u0026qp-\u003ecq_recv_list);\n+\n+\tif (qp-\u003ecreate_type == XSC_QP_KERNEL) {\n+\t\t__xsc_ib_cq_clean(recv_cq, qp-\u003exqp.qpn);\n+\t\tif (send_cq != recv_cq)\n+\t\t\t__xsc_ib_cq_clean(send_cq, qp-\u003exqp.qpn);\n+\t}\n+\txsc_ib_unlock_cqs(send_cq, recv_cq);\n+\n+\terr = xsc_core_destroy_qp(dev-\u003exdev, \u0026qp-\u003exqp);\n+\tif (err)\n+\t\tibdev_warn(\u0026dev-\u003eib_dev, \"failed to destroy QP 0x%x\\n\", qp-\u003exqp.qpn);\n+\tkfree(in);\n+\n+\tif (qp-\u003ecreate_type == XSC_QP_KERNEL)\n+\t\tdestroy_qp_kernel(dev, qp);\n+\telse if (qp-\u003ecreate_type == XSC_QP_USER)\n+\t\tdestroy_qp_user(\u0026get_pd(qp)-\u003eibpd, qp);\n+}\n+\n+static const char *ib_qp_type_str(enum ib_qp_type type)\n+{\n+\tswitch (type) {\n+\tcase IB_QPT_SMI:\n+\t\treturn \"IB_QPT_SMI\";\n+\tcase IB_QPT_GSI:\n+\t\treturn \"IB_QPT_GSI\";\n+\tcase IB_QPT_RC:\n+\t\treturn \"IB_QPT_RC\";\n+\tcase IB_QPT_UC:\n+\t\treturn \"IB_QPT_UC\";\n+\tcase IB_QPT_UD:\n+\t\treturn \"IB_QPT_UD\";\n+\tcase IB_QPT_RAW_IPV6:\n+\t\treturn \"IB_QPT_RAW_IPV6\";\n+\tcase IB_QPT_RAW_ETHERTYPE:\n+\t\treturn \"IB_QPT_RAW_ETHERTYPE\";\n+\tcase IB_QPT_RAW_PACKET:\n+\t\treturn \"IB_QPT_RAW_PACKET\";\n+\tcase IB_QPT_MAX:\n+\tdefault:\n+\t\treturn \"Invalid QP type\";\n+\t}\n+}\n+\n+int xsc_ib_create_qp(struct ib_qp *ibqp,\n+\t\t struct ib_qp_init_attr *init_attr,\n+\t\t struct ib_udata *udata)\n+{\n+\tstruct xsc_ib_dev *dev;\n+\tstruct xsc_ib_qp *qp;\n+\tstruct ib_pd *pd = ibqp-\u003epd;\n+\tint err;\n+\n+\tif (pd) {\n+\t\tdev = to_xdev(pd-\u003edevice);\n+\t} else {\n+\t\tpr_warn(\"%s: no PD for transport %s\\n\", __func__,\n+\t\t\tib_qp_type_str(init_attr-\u003eqp_type));\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tif (init_attr-\u003eqp_type != IB_QPT_RAW_PACKET) {\n+\t\tif (!is_support_rdma(dev-\u003exdev) ||\n+\t\t (is_qp1(init_attr-\u003eqp_type) \u0026\u0026 !is_support_rdma_cm(dev-\u003exdev))) {\n+\t\t\treturn -EPROTONOSUPPORT;\n+\t\t}\n+\t}\n+\n+\tqp = to_xqp(ibqp);\n+\n+\tqp-\u003exqp.qp_type = init_attr-\u003eqp_type;\n+\n+\tswitch (init_attr-\u003eqp_type) {\n+\tcase IB_QPT_RC:\n+\tcase IB_QPT_SMI:\n+\tcase IB_QPT_GSI:\n+\tcase IB_QPT_RAW_PACKET:\n+\t\terr = create_qp_common(dev, pd, init_attr, udata, qp);\n+\t\tif (err) {\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"create_qp_common failed\\n\");\n+\t\t\treturn err;\n+\t\t}\n+\n+\t\tif (is_qp0(init_attr-\u003eqp_type)) {\n+\t\t\tqp-\u003eibqp.qp_num = 0;\n+\t\t} else if (is_qp1(init_attr-\u003eqp_type)) {\n+\t\t\tqp-\u003eibqp.qp_num = 1;\n+\t\t\tdev-\u003exdev-\u003egsi_qpn = qp-\u003exqp.qpn;\n+\t\t} else {\n+\t\t\tqp-\u003eibqp.qp_num = qp-\u003exqp.qpn;\n+\t\t}\n+\n+\t\tbreak;\n+\n+\tcase IB_QPT_RAW_IPV6:\n+\tcase IB_QPT_RAW_ETHERTYPE:\n+\tcase IB_QPT_MAX:\n+\tdefault:\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"unsupported qp type %d\\n\",\n+\t\t\t init_attr-\u003eqp_type);\n+\t\t/* Don't support raw QPs */\n+\t\treturn -EINVAL;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+int xsc_ib_destroy_qp(struct ib_qp *qp, struct ib_udata *udata)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(qp-\u003edevice);\n+\tstruct xsc_ib_qp *xqp = to_xqp(qp);\n+\tstruct xsc_err_state_qp_node *tmp = NULL, *err_rq_node = NULL, *err_sq_node = NULL;\n+\tstruct xsc_srq_qp_node *qp_node = NULL, *tmp_qp_node = NULL;\n+\tunsigned long flags;\n+\tstruct xsc_ib_srq *xsrq;\n+\n+\tif (qp-\u003eqp_type == IB_QPT_RC) {\n+\t\tif (xqp-\u003erecv_cq) {\n+\t\t\tspin_lock_irq(\u0026to_xcq(xqp-\u003erecv_cq)-\u003elock);\n+\t\t\tlist_for_each_entry_safe(err_rq_node, tmp,\n+\t\t\t\t\t\t \u0026to_xcq(xqp-\u003erecv_cq)-\u003eerr_state_qp_list, entry) {\n+\t\t\t\tif (err_rq_node-\u003eqp_id == xqp-\u003exqp.qpn \u0026\u0026 !err_rq_node-\u003eis_sq) {\n+\t\t\t\t\tlist_del(\u0026err_rq_node-\u003eentry);\n+\t\t\t\t\tkfree(err_rq_node);\n+\t\t\t\t}\n+\t\t\t}\n+\t\t\tspin_unlock_irq(\u0026to_xcq(xqp-\u003erecv_cq)-\u003elock);\n+\t\t}\n+\n+\t\tif (xqp-\u003esend_cq) {\n+\t\t\tspin_lock_irq(\u0026to_xcq(xqp-\u003esend_cq)-\u003elock);\n+\t\t\tlist_for_each_entry_safe(err_sq_node, tmp,\n+\t\t\t\t\t\t \u0026to_xcq(xqp-\u003esend_cq)-\u003eerr_state_qp_list, entry) {\n+\t\t\t\tif (err_sq_node-\u003eqp_id == xqp-\u003exqp.qpn \u0026\u0026 err_sq_node-\u003eis_sq) {\n+\t\t\t\t\tlist_del(\u0026err_sq_node-\u003eentry);\n+\t\t\t\t\tkfree(err_sq_node);\n+\t\t\t\t}\n+\t\t\t}\n+\t\t\tspin_unlock_irq(\u0026to_xcq(xqp-\u003esend_cq)-\u003elock);\n+\t\t}\n+\t}\n+\n+\tif (xqp-\u003ecreate_type == XSC_QP_KERNEL \u0026\u0026 qp-\u003esrq) {\n+\t\tif (!to_xsrq(qp-\u003esrq) || list_empty(\u0026to_xsrq(qp-\u003esrq)-\u003exsrq.srq_qp_list))\n+\t\t\tibdev_warn(\u0026dev-\u003eib_dev, \"srq_qp_list is empty!\\n\");\n+\n+\t\txsrq = to_xsrq(qp-\u003esrq);\n+\t\tspin_lock_irqsave(\u0026xsrq-\u003elock, flags);\n+\t\txsc_restore_cache_srq_wrs(xsrq, xqp);\n+\t\tlist_for_each_entry_safe(qp_node, tmp_qp_node,\n+\t\t\t\t\t \u0026to_xsrq(qp-\u003esrq)-\u003exsrq.srq_qp_list, entry) {\n+\t\t\tif (qp_node-\u003eqp_id == xqp-\u003exqp.qpn) {\n+\t\t\t\tto_xsrq(qp-\u003esrq)-\u003eqp_num--;\n+\t\t\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"srq_qp_list del node qpn:%u qp_num:%d\\n\",\n+\t\t\t\t\t qp_node-\u003eqp_id, to_xsrq(qp-\u003esrq)-\u003eqp_num);\n+\t\t\t\tlist_del_init(\u0026qp_node-\u003eentry);\n+\t\t\t\tkvfree(qp_node);\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\t\tspin_unlock_irqrestore(\u0026xsrq-\u003elock, flags);\n+\t}\n+\n+\tdestroy_qp_common(dev, xqp);\n+\n+\treturn 0;\n+}\n+\n+static inline u16 xsc_calc_udp_sport(u32 lqpn, u32 rqpn)\n+{\n+\tunsigned char *p;\n+\tu8 ports[2];\n+\tu16 sport;\n+\tu64 tqpn;\n+\n+\ttqpn = ((u64)(lqpn \u0026 0xffffff)) * ((u64)(rqpn \u0026 0xffffff));\n+\tp = (unsigned char *)\u0026tqpn;\n+\tports[0] = p[0] ^ p[2] ^ p[4];\n+\tports[1] = p[1] ^ p[3] ^ p[5];\n+\tsport = *((u16 *)ports) | 0xC000;\n+\n+\treturn sport;\n+}\n+\n+static inline void xsc_path_set_udp_sport(struct xsc_qp_path *path,\n+\t\t\t\t\t const struct rdma_ah_attr *ah,\n+\t\t\t\t\t u32 lqpn, u32 rqpn)\n+{\n+\tif ((ah-\u003egrh.flow_label \u0026 UDP_SPORT_MASK) != 0) {\n+\t\tif ((ah-\u003egrh.flow_label \u0026 UDP_SPORT_MASK_EN) == 0)\n+\t\t\tpath-\u003esport = cpu_to_be16(rdma_flow_label_to_udp_sport(ah-\u003egrh.flow_label));\n+\t\telse\n+\t\t\tpath-\u003esport = cpu_to_be16((ah-\u003egrh.flow_label \u0026 UDP_SPORT_MASK) \u003e\u003e\n+\t\t\t\t\t\t\tUDP_SPORT_OFFSET);\n+\t} else {\n+\t\tpath-\u003esport = cpu_to_be16(xsc_calc_udp_sport(lqpn, rqpn));\n+\t}\n+}\n+\n+static int xsc_set_path(struct xsc_ib_dev *dev, const struct rdma_ah_attr *ah,\n+\t\t\tstruct xsc_qp_path *path, u8 port, int attr_mask,\n+\t\t\tu32 path_flags, const struct ib_qp_attr *attr, struct xsc_ib_qp *qp)\n+{\n+\tstruct ib_global_route *grh = rdma_ah_retrieve_grh((struct rdma_ah_attr *)ah);\n+\tunion ib_gid *dgid = \u0026grh-\u003edgid;\n+\tconst struct ib_gid_attr *sgid_attr = grh-\u003esgid_attr;\n+\tunion ib_gid *sgid = \u0026((struct ib_gid_attr *)sgid_attr)-\u003egid;\n+\tunion {\n+\t\tstruct sockaddr _sockaddr;\n+\t\tstruct sockaddr_in _sockaddr_in;\n+\t\tstruct sockaddr_in6 _sockaddr_in6;\n+\t} sgid_addr, dgid_addr;\n+\tint force_pcp, force_dscp;\n+\tchar buf[256] = {0};\n+\tchar *ptr = buf;\n+\tint ret = 0;\n+\n+\tif (ah-\u003etype == RDMA_AH_ATTR_TYPE_ROCE) {\n+\t\tif (!(rdma_ah_get_ah_flags(ah) \u0026 IB_AH_GRH))\n+\t\t\treturn -EINVAL;\n+\n+\t\tif (qp-\u003eibqp.qp_type == IB_QPT_RC ||\n+\t\t qp-\u003eibqp.qp_type == IB_QPT_UC) {\n+\t\t\tif (((ah-\u003egrh.flow_label \u0026 UDP_SPORT_MASK_EN) != 0) \u0026\u0026\n+\t\t\t qp-\u003exqp.qp_protocol_mode == RDMA_PROTO_VEROCE) {\n+\t\t\t\tibdev_err(\u0026dev-\u003eib_dev,\n+\t\t\t\t\t \"qp type is RDMA_PROTO_VEROCE, using flow_label to set the UDP sport %d is not allowed.\",\n+\t\t\t\t\t (ah-\u003egrh.flow_label \u0026 UDP_SPORT_MASK) \u003e\u003e\n+\t\t\t\t\t UDP_SPORT_OFFSET);\n+\t\t\t\treturn -EINVAL;\n+\t\t\t}\n+\t\t\txsc_path_set_udp_sport(path, ah, qp-\u003eibqp.qp_num, attr-\u003edest_qp_num);\n+\t\t}\n+\n+\t\tif (sgid_attr-\u003egid_type != IB_GID_TYPE_ROCE_UDP_ENCAP) {\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"gid type not ROCEv2\\n\");\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\n+\t\tforce_dscp = dev-\u003exdev-\u003epriv.force_dscp;\n+\t\tif (force_dscp == DSCP_PCP_UNSET)\n+\t\t\tpath-\u003eecn_dscp = (grh-\u003etraffic_class \u003e\u003e 2) \u0026 0x3f;\n+\t\telse\n+\t\t\tpath-\u003eecn_dscp = force_dscp;\n+\t\tpath-\u003ehop_limit = grh-\u003ehop_limit;\n+\n+\t\trdma_gid2ip((struct sockaddr *)\u0026sgid_addr, sgid);\n+\t\trdma_gid2ip((struct sockaddr *)\u0026dgid_addr, dgid);\n+\n+\t\tif (sgid_addr._sockaddr.sa_family == AF_INET \u0026\u0026\n+\t\t dgid_addr._sockaddr.sa_family == AF_INET) {\n+\t\t\tmemcpy(path-\u003esip, \u0026sgid_addr._sockaddr_in.sin_addr.s_addr,\n+\t\t\t sizeof(struct in_addr));\n+\t\t\tmemcpy(path-\u003edip, \u0026dgid_addr._sockaddr_in.sin_addr.s_addr,\n+\t\t\t sizeof(struct in_addr));\n+\t\t\tpath-\u003eaf_type = AF_INET;\n+\t\t\tret += snprintf(ptr + ret, 256 - ret, \"sip=%#x,\",\n+\t\t\t\t\tbe32_to_cpu(path-\u003esip[0]));\n+\t\t\tret += snprintf(ptr + ret, 256 - ret, \"dip=%#x,\",\n+\t\t\t\t\tbe32_to_cpu(path-\u003edip[0]));\n+\t\t} else if (sgid_addr._sockaddr.sa_family == AF_INET6 \u0026\u0026\n+\t\t\t dgid_addr._sockaddr.sa_family == AF_INET6) {\n+\t\t\tmemcpy(path-\u003esip, \u0026sgid_addr._sockaddr_in6.sin6_addr.s6_addr,\n+\t\t\t sizeof(path-\u003esip));\n+\t\t\tmemcpy(path-\u003edip, \u0026dgid_addr._sockaddr_in6.sin6_addr.s6_addr,\n+\t\t\t sizeof(path-\u003edip));\n+\t\t\tpath-\u003eaf_type = AF_INET6;\n+\t\t\tret += snprintf(ptr + ret, 256 - ret, \"sip=%08x%08x%08x%08x,\",\n+\t\t\t\t\tbe32_to_cpu(path-\u003esip[0]), be32_to_cpu(path-\u003esip[1]),\n+\t\t\t\t\tbe32_to_cpu(path-\u003esip[2]), be32_to_cpu(path-\u003esip[3]));\n+\t\t\tret += snprintf(ptr + ret, 256 - ret, \"dip=%08x%08x%08x%08x,\",\n+\t\t\t\t\tbe32_to_cpu(path-\u003edip[0]), be32_to_cpu(path-\u003edip[1]),\n+\t\t\t\t\tbe32_to_cpu(path-\u003edip[2]), be32_to_cpu(path-\u003edip[3]));\n+\t\t} else {\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\n+\t\tether_addr_copy(path-\u003esmac, dev-\u003enetdev-\u003edev_addr);\n+\n+\t\tmemcpy(path-\u003edmac, ah-\u003eroce.dmac, sizeof(ah-\u003eroce.dmac));\n+\t\tret += snprintf(ptr + ret, 256 - ret, \"smac=%02x%02x%02x%02x%02x%02x,\",\n+\t\t\t\tpath-\u003esmac[0], path-\u003esmac[1], path-\u003esmac[2],\n+\t\t\t\tpath-\u003esmac[3], path-\u003esmac[4], path-\u003esmac[5]);\n+\t\tret += snprintf(ptr + ret, 256 - ret, \"dmac=%02x%02x%02x%02x%02x%02x\",\n+\t\t\t\tpath-\u003edmac[0], path-\u003edmac[1], path-\u003edmac[2],\n+\t\t\t\tpath-\u003edmac[3], path-\u003edmac[4], path-\u003edmac[5]);\n+\t\tif (is_zero_ether_addr(path-\u003edmac))\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"path dmac is zero!\\n\");\n+\t\telse\n+\t\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"ib path info:%s\\n\", buf);\n+\n+\t\tif (is_vlan_dev(sgid_attr-\u003endev)) {\n+\t\t\tpath-\u003evlan_valid = 1;\n+\t\t\tpath-\u003evlan_id = cpu_to_be16(vlan_dev_vlan_id(sgid_attr-\u003endev));\n+\n+\t\t\tforce_pcp = dev-\u003exdev-\u003epriv.force_pcp;\n+\t\t\tif (force_pcp == DSCP_PCP_UNSET)\n+\t\t\t\tpath-\u003edci_cfi_prio_sl = (ah-\u003esl \u0026 0x7);\n+\t\t\telse\n+\t\t\t\tpath-\u003edci_cfi_prio_sl = force_pcp;\n+\t\t} else {\n+\t\t\tpath-\u003evlan_valid = 0;\n+\t\t}\n+\t}\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"path dscp %d pcp %d\\n\", path-\u003eecn_dscp, path-\u003edci_cfi_prio_sl);\n+\treturn 0;\n+}\n+\n+static inline void xsc_ib_trigger_cq_comp_handler(struct ib_cq *send_cq, struct ib_cq *recv_cq)\n+{\n+\tif (send_cq)\n+\t\tsend_cq-\u003ecomp_handler(send_cq, send_cq-\u003ecq_context);\n+\n+\tif (recv_cq \u0026\u0026 send_cq != recv_cq)\n+\t\trecv_cq-\u003ecomp_handler(recv_cq, recv_cq-\u003ecq_context);\n+}\n+\n+int xsc_ib_err_state_qp(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp,\n+\t\t\tenum xsc_qp_state cur_state, enum xsc_qp_state state)\n+{\n+\tstruct xsc_err_state_qp_node *tmp = NULL, *err_rq_node = NULL, *err_sq_node = NULL;\n+\tstruct xsc_ib_cq *send_xcq, *recv_xcq;\n+\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"modify qp: qpid %d, cur_qp_state %d, qp_state %d\\n\",\n+\t\t qp-\u003exqp.qpn, cur_state, state);\n+\tif (cur_state == XSC_QP_STATE_ERR \u0026\u0026 state != XSC_QP_STATE_ERR) {\n+\t\tif (!qp-\u003eibqp.srq \u0026\u0026 qp-\u003erecv_cq) {\n+\t\t\trecv_xcq = to_xcq(qp-\u003erecv_cq);\n+\t\t\tlist_for_each_entry_safe(err_rq_node, tmp,\n+\t\t\t\t\t\t \u0026recv_xcq-\u003eerr_state_qp_list, entry) {\n+\t\t\t\tif (err_rq_node-\u003eqp_id == qp-\u003exqp.qpn \u0026\u0026 !err_rq_node-\u003eis_sq) {\n+\t\t\t\t\tlist_del(\u0026err_rq_node-\u003eentry);\n+\t\t\t\t\tkfree(err_rq_node);\n+\t\t\t\t}\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (qp-\u003esend_cq) {\n+\t\t\tsend_xcq = to_xcq(qp-\u003esend_cq);\n+\t\t\tlist_for_each_entry_safe(err_sq_node, tmp,\n+\t\t\t\t\t\t \u0026send_xcq-\u003eerr_state_qp_list, entry) {\n+\t\t\t\tif (err_sq_node-\u003eqp_id == qp-\u003exqp.qpn \u0026\u0026 err_sq_node-\u003eis_sq) {\n+\t\t\t\t\tlist_del(\u0026err_sq_node-\u003eentry);\n+\t\t\t\t\tkfree(err_sq_node);\n+\t\t\t\t}\n+\t\t\t}\n+\t\t}\n+\t\treturn 0;\n+\t}\n+\n+\tif (cur_state != XSC_QP_STATE_ERR \u0026\u0026 state == XSC_QP_STATE_ERR \u0026\u0026\n+\t !qp-\u003ehas_trig_cq_comp_handle) {\n+\t\tif (!qp-\u003eibqp.srq \u0026\u0026 qp-\u003erecv_cq) {\n+\t\t\terr_rq_node = kzalloc_obj(*err_rq_node, GFP_KERNEL);\n+\t\t\tif (!err_rq_node)\n+\t\t\t\treturn -ENOMEM;\n+\t\t\terr_rq_node-\u003eqp_id = qp-\u003exqp.qpn;\n+\t\t\terr_rq_node-\u003eis_sq = false;\n+\t\t\trecv_xcq = to_xcq(qp-\u003erecv_cq);\n+\t\t\tlist_add_tail(\u0026err_rq_node-\u003eentry, \u0026recv_xcq-\u003eerr_state_qp_list);\n+\t\t}\n+\n+\t\tif (qp-\u003esend_cq) {\n+\t\t\terr_sq_node = kzalloc_obj(*err_sq_node, GFP_KERNEL);\n+\t\t\tif (!err_sq_node)\n+\t\t\t\treturn -ENOMEM;\n+\t\t\terr_sq_node-\u003eqp_id = qp-\u003exqp.qpn;\n+\t\t\terr_sq_node-\u003eis_sq = true;\n+\t\t\tsend_xcq = to_xcq(qp-\u003esend_cq);\n+\t\t\tlist_add_tail(\u0026err_sq_node-\u003eentry, \u0026send_xcq-\u003eerr_state_qp_list);\n+\t\t}\n+\n+\t\txsc_ib_trigger_cq_comp_handler(qp-\u003esend_cq, qp-\u003erecv_cq);\n+\t\tqp-\u003ehas_trig_cq_comp_handle = true;\n+\t}\n+\treturn 0;\n+}\n+\n+static inline void xsc_set_qp_access_flag(struct xsc_modify_qp_mbox_in *in,\n+\t\t\t\t\t struct xsc_qp_context *context, int access_flags)\n+{\n+\tin-\u003ehdr.ver = cpu_to_be16(XSC_QP_CONTEXT_V1);\n+\tcontext-\u003eqp_access_flags = 0;\n+\n+\tif (access_flags \u0026 IB_ACCESS_REMOTE_READ)\n+\t\tcontext-\u003eqp_access_flags |= QP_ACCESS_REMOTE_READ;\n+\tif (access_flags \u0026 IB_ACCESS_REMOTE_WRITE)\n+\t\tcontext-\u003eqp_access_flags |= QP_ACCESS_REMOTE_WRITE;\n+\tif (access_flags \u0026 IB_ACCESS_REMOTE_ATOMIC)\n+\t\tcontext-\u003eqp_access_flags |= QP_ACCESS_REMOTE_ATOMIC;\n+\n+\tcontext-\u003eqp_access_flags = cpu_to_be32(context-\u003eqp_access_flags);\n+}\n+\n+static int __xsc_ib_modify_qp(struct ib_qp *ibqp,\n+\t\t\t const struct ib_qp_attr *attr, int attr_mask,\n+\t\t\t enum ib_qp_state cur_state, enum ib_qp_state new_state,\n+\t\t\t const struct xsc_ib_modify_qp *ucmd,\n+\t\t\t struct ib_udata *udata)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(ibqp-\u003edevice);\n+\tstruct xsc_ib_qp *qp = to_xqp(ibqp);\n+\tstruct xsc_ib_cq *send_cq, *recv_cq;\n+\tstruct xsc_qp_context *context;\n+\tstruct xsc_modify_qp_mbox_in *in;\n+\tstruct xsc_qp_path path;\n+\tint sqd_event;\n+\tint err;\n+\tchar buf[256] = {0};\n+\tchar *ptr = buf;\n+\tint ret = 0;\n+\tstruct xsc_core_device *xdev = dev-\u003exdev;\n+\n+\tin = kzalloc_obj(*in, GFP_KERNEL);\n+\tif (!in)\n+\t\treturn -ENOMEM;\n+\n+\tcontext = \u0026qp-\u003ectx;\n+\n+\tif (attr_mask \u0026 IB_QP_PATH_MTU) {\n+\t\tif (attr-\u003epath_mtu != IB_MTU_1024 \u0026\u0026\n+\t\t attr-\u003epath_mtu != IB_MTU_4096 \u0026\u0026\n+\t\t attr-\u003epath_mtu != IB_MTU_8192) {\n+\t\t\tibdev_warn(\u0026dev-\u003eib_dev, \"invalid mtu %d\\n\", attr-\u003epath_mtu);\n+\t\t}\n+\n+\t\tif (attr-\u003epath_mtu \u003c IB_MTU_4096)\n+\t\t\tcontext-\u003emtu_mode = 0;\n+\t\telse if (attr-\u003epath_mtu \u003c IB_MTU_8192)\n+\t\t\tcontext-\u003emtu_mode = 1;\n+\t\telse\n+\t\t\tcontext-\u003emtu_mode = 2;\n+\t\tret = snprintf(ptr, 256, \"path_mtu=%d,\", attr-\u003epath_mtu);\n+\t}\n+\n+\tif (attr_mask \u0026 IB_QP_DEST_QPN) {\n+\t\tcontext-\u003eremote_qpn = cpu_to_be32(attr-\u003edest_qp_num);\n+\t\tret += snprintf(ptr + ret, 256 - ret, \"dest_qp_num=%d,\", attr-\u003edest_qp_num);\n+\t}\n+\n+\tif (attr_mask \u0026 IB_QP_AV) {\n+\t\terr = xsc_set_path(dev, \u0026attr-\u003eah_attr, \u0026path,\n+\t\t\t\t attr_mask \u0026 IB_QP_PORT ? attr-\u003eport_num : qp-\u003eport,\n+\t\t\t\t attr_mask, 0, attr, qp);\n+\t\tif (err)\n+\t\t\tgoto out;\n+\n+\t\tcontext-\u003esrc_udp_port = path.sport;\n+\t\tcontext-\u003edscp = path.ecn_dscp;\n+\t\tcontext-\u003ehop_limit = path.hop_limit;\n+\t\tcontext-\u003eip_type = (path.af_type == AF_INET ? 0 : 1);\n+\t\tcontext-\u003eip_type = cpu_to_be16(context-\u003eip_type);\n+\t\tmemcpy(context-\u003edip, path.dip, sizeof(context-\u003edip));\n+\t\tmemcpy(context-\u003esip, path.sip, sizeof(context-\u003esip));\n+\t\tmemcpy(context-\u003edmac, path.dmac, sizeof(path.dmac));\n+\t\tmemcpy(context-\u003esmac, path.smac, sizeof(path.smac));\n+\n+\t\tcontext-\u003evlan_valid = path.vlan_valid;\n+\t\tcontext-\u003edci_cfi_prio_sl = path.dci_cfi_prio_sl;\n+\t\tcontext-\u003evlan_id = path.vlan_id;\n+\t}\n+\n+\tif (attr_mask \u0026 IB_QP_RNR_RETRY) {\n+\t\tcontext-\u003ernr_retry = attr-\u003ernr_retry;\n+\t\tret += snprintf(ptr + ret, 256 - ret, \"rnr_retry=%d,\", attr-\u003ernr_retry);\n+\t}\n+\n+\tif (attr_mask \u0026 IB_QP_RETRY_CNT) {\n+\t\tcontext-\u003eretry_cnt = attr-\u003eretry_cnt;\n+\t\tret += snprintf(ptr + ret, 256 - ret, \"retry_cnt=%d,\", attr-\u003eretry_cnt);\n+\t}\n+\n+\tif (attr_mask \u0026 IB_QP_MAX_QP_RD_ATOMIC \u0026\u0026 attr-\u003emax_rd_atomic)\n+\t\tcontext-\u003emax_sra = min_t(u32, attr-\u003emax_rd_atomic, dev-\u003exdev-\u003ecaps.max_ra_req_qp);\n+\n+\tif (attr_mask \u0026 IB_QP_MAX_DEST_RD_ATOMIC \u0026\u0026 attr-\u003emax_dest_rd_atomic)\n+\t\tcontext-\u003emax_rra = attr-\u003emax_dest_rd_atomic;\n+\n+\tif (attr_mask \u0026 IB_QP_SQ_PSN) {\n+\t\tcontext-\u003enext_send_psn = cpu_to_be32(attr-\u003esq_psn);\n+\t\tret += snprintf(ptr + ret, 256 - ret, \"sq_psn=%#x,\", attr-\u003esq_psn);\n+\t}\n+\n+\tif (attr_mask \u0026 IB_QP_RQ_PSN) {\n+\t\tcontext-\u003enext_recv_psn = cpu_to_be32(attr-\u003erq_psn);\n+\t\tret += snprintf(ptr + ret, 256 - ret, \"rq_psn=%#x,\", attr-\u003erq_psn);\n+\t}\n+\n+\tif (attr_mask \u0026 IB_QP_ACCESS_FLAGS) {\n+\t\txsc_set_qp_access_flag(in, context, attr-\u003eqp_access_flags);\n+\t\tret += snprintf(ptr + ret, 256 - ret, \"qp_access_flags=%#x,\\n\",\n+\t\t\t\tattr-\u003eqp_access_flags);\n+\t}\n+\n+\tif (udata \u0026\u0026 udata-\u003einlen \u0026\u0026 attr_mask \u0026 IB_QP_STATE \u0026\u0026 new_state == IB_QPS_RTR) {\n+\t\tif (!ucmd-\u003eprofile \u0026\u0026\n+\t\t xdev-\u003erdma_proto_mode == RDMA_PROTO_VEROCE) {\n+\t\t\tcontext-\u003eswith_qp_type = 1;\n+\t\t\tcontext-\u003eqp_access_flags |= QP_ACCESS_MODIFY_QP_TYPE;\n+\t\t\tcontext-\u003eqp_access_flags = cpu_to_be32(context-\u003eqp_access_flags);\n+\t\t\tqp-\u003exqp.qp_protocol_mode =\n+\t\t\t\tucmd-\u003eprofile ? RDMA_PROTO_VEROCE : RDMA_PROTO_ROCEV2;\n+\t\t\tin-\u003ehdr.ver = cpu_to_be16(XSC_QP_CONTEXT_V2);\n+\t\t\tibdev_dbg(\u0026dev-\u003eib_dev,\n+\t\t\t\t \"local is VEROCE,remote is RoCEv2, switch qp_type from VEROCE to RoCEv2!\\n\");\n+\t\t}\n+\t}\n+\n+\tif (attr_mask \u0026 IB_QP_STATE \u0026\u0026 new_state == IB_QPS_RESET)\n+\t\tqp-\u003exqp.qp_protocol_mode = xdev-\u003erdma_proto_mode;\n+\n+\tif (cur_state == IB_QPS_RTS \u0026\u0026 new_state == IB_QPS_SQD\t\u0026\u0026\n+\t attr_mask \u0026 IB_QP_EN_SQD_ASYNC_NOTIFY \u0026\u0026 attr-\u003een_sqd_async_notify)\n+\t\tsqd_event = 1;\n+\telse\n+\t\tsqd_event = 0;\n+\n+\tmemcpy(\u0026in-\u003ectx, context, sizeof(*context));\n+\terr = xsc_core_qp_modify(xdev, to_xsc_state(cur_state),\n+\t\t\t\t to_xsc_state(new_state), in, sqd_event,\n+\t\t\t\t \u0026qp-\u003exqp);\n+\tif (err) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"failed to modify qp[%d] from %s to %s\\n\",\n+\t\t\t qp-\u003exqp.qpn, qp_state_to_str(cur_state), qp_state_to_str(new_state));\n+\t\tgoto out;\n+\t}\n+\n+\tqp-\u003estate = new_state;\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"succeeded to modify qp[%d] from %s to %s with attr_mask=%#x, %s\\n\",\n+\t\t qp-\u003exqp.qpn, qp_state_to_str(cur_state), qp_state_to_str(new_state),\n+\t\t attr_mask, buf);\n+\n+\tif (qp-\u003estate == IB_QPS_INIT) {\n+\t\tqp-\u003epost_recv_invalid = false;\n+\t\tqp-\u003epost_send_invalid = false;\n+\t}\n+\n+\tif (attr_mask \u0026 IB_QP_ACCESS_FLAGS)\n+\t\tqp-\u003eatomic_rd_en = attr-\u003eqp_access_flags;\n+\tif (attr_mask \u0026 IB_QP_MAX_DEST_RD_ATOMIC)\n+\t\tqp-\u003eresp_depth = attr-\u003emax_dest_rd_atomic;\n+\tif (attr_mask \u0026 IB_QP_PORT)\n+\t\tqp-\u003eport = attr-\u003eport_num;\n+\tif (attr_mask \u0026 IB_QP_ALT_PATH)\n+\t\tqp-\u003ealt_port = attr-\u003ealt_port_num;\n+\n+\t/*\n+\t * If we moved a kernel QP to RESET, clean up all old CQ\n+\t * entries and reinitialize the QP.\n+\t */\n+\tif (new_state == IB_QPS_RESET \u0026\u0026 !ibqp-\u003euobject) {\n+\t\tget_cqs(qp-\u003exqp.qp_type, qp-\u003eibqp.send_cq, qp-\u003eibqp.recv_cq, \u0026send_cq, \u0026recv_cq);\n+\t\txsc_ib_cq_clean(recv_cq, qp-\u003exqp.qpn);\n+\t\tif (send_cq != recv_cq)\n+\t\t\txsc_ib_cq_clean(send_cq, qp-\u003exqp.qpn);\n+\n+\t\tqp-\u003erq.head = 0;\n+\t\tqp-\u003erq.tail = 0;\n+\t\tqp-\u003esq.head = 0;\n+\t\tqp-\u003esq.tail = 0;\n+\t\tqp-\u003esq.cur_post = 0;\n+\t\tqp-\u003esq.last_poll = 0;\n+\t\tqp-\u003esq.last_comp_ce_wqe = -1;\n+\t\tqp-\u003esq.last_post_ce_wqe = -1;\n+\t\tqp-\u003ehas_trig_cq_comp_handle = false;\n+\t}\n+\n+out:\n+\tkfree(in);\n+\treturn err;\n+}\n+\n+enum xsc_ib_qp_ex_attr_mask {\n+\tXSC_IB_QP_FLUSH\t\t= 1 \u003c\u003c 21,\n+};\n+\n+int xsc_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,\n+\t\t int attr_mask, struct ib_udata *udata)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(ibqp-\u003edevice);\n+\tstruct xsc_ib_qp *qp = to_xqp(ibqp);\n+\tenum ib_qp_state cur_state, new_state;\n+\tint err = -EINVAL;\n+\tint wr_num;\n+\tstruct xsc_ib_srq *srq;\n+\tunsigned long flags;\n+\tstruct xsc_ib_cq *send_cq;\n+\tstruct xsc_ib_cq *recv_cq;\n+\tstruct xsc_ib_modify_qp ucmd = {};\n+\n+\tif (udata \u0026\u0026 udata-\u003einlen) {\n+\t\tif (udata-\u003einlen \u003c offsetofend(typeof(ucmd), profile))\n+\t\t\treturn -EINVAL;\n+\n+\t\tif (ib_copy_from_udata(\u0026ucmd, udata,\n+\t\t\t\t min(udata-\u003einlen, sizeof(ucmd))))\n+\t\t\treturn -EFAULT;\n+\t}\n+\n+\tmutex_lock(\u0026qp-\u003emutex);\n+\n+\tcur_state = attr_mask \u0026 IB_QP_CUR_STATE ? attr-\u003ecur_qp_state : qp-\u003estate;\n+\tnew_state = attr_mask \u0026 IB_QP_STATE ? attr-\u003eqp_state : cur_state;\n+\n+\tif ((attr_mask \u0026 IB_QP_PORT) \u0026\u0026\n+\t (attr-\u003eport_num == 0 || attr-\u003eport_num \u003e dev-\u003exdev-\u003ecaps.num_ports)) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"error port num\\n\");\n+\t\tgoto out;\n+\t}\n+\n+\tif (attr_mask \u0026 IB_QP_MAX_DEST_RD_ATOMIC \u0026\u0026\n+\t attr-\u003emax_dest_rd_atomic \u003e dev-\u003exdev-\u003ecaps.max_ra_res_qp) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"dest rd atomic:%u exeeded\", attr-\u003emax_dest_rd_atomic);\n+\t\tgoto out;\n+\t}\n+\n+\tif (cur_state == new_state \u0026\u0026 cur_state == IB_QPS_RESET) {\n+\t\terr = 0;\n+\t\tgoto out;\n+\t}\n+\n+\tif (!ib_modify_qp_is_ok(cur_state, new_state, ibqp-\u003eqp_type, attr_mask)) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"ib_modify_qp not ok, type=%d mask=0x%x state from%d to%d\\n\",\n+\t\t\t ibqp-\u003eqp_type, attr_mask, cur_state, new_state);\n+\t\tgoto out;\n+\t}\n+\n+\terr = __xsc_ib_modify_qp(ibqp, attr, attr_mask, cur_state, new_state, \u0026ucmd, udata);\n+\n+\tif (!err \u0026\u0026 (attr_mask \u0026 IB_QP_STATE) \u0026\u0026 !udata) {\n+\t\tget_cqs(qp-\u003exqp.qp_type, qp-\u003esend_cq, qp-\u003erecv_cq, \u0026send_cq, \u0026recv_cq);\n+\t\txsc_ib_lock_cqs(send_cq, recv_cq);\n+\t\terr = xsc_ib_err_state_qp(dev, qp, to_xsc_state(cur_state),\n+\t\t\t\t\t to_xsc_state(new_state));\n+\t\txsc_ib_unlock_cqs(send_cq, recv_cq);\n+\n+\t\tif (!ibqp-\u003esrq)\n+\t\t\tgoto out;\n+\n+\t\tif (cur_state == IB_QPS_RESET \u0026\u0026 new_state == IB_QPS_INIT) {\n+\t\t\tsrq = to_xsrq(ibqp-\u003esrq);\n+\t\t\tspin_lock_irqsave(\u0026srq-\u003elock, flags);\n+\t\t\tsrq-\u003eqp_num++;\n+\t\t\twr_num = XSC_SRQ_MIN_WR_NUM;\n+\t\t\tibdev_dbg(\u0026dev-\u003eib_dev,\n+\t\t\t\t \"SRQ %d: qp %d to INIT, post %d srq wrs, qp_num %d\\n\",\n+\t\t\t\t srq-\u003exsrq.srqn, qp-\u003exqp.qpn, wr_num, srq-\u003eqp_num);\n+\t\t\txsc_post_cache_wrs(srq, qp, wr_num, true);\n+\t\t\tspin_unlock_irqrestore(\u0026srq-\u003elock, flags);\n+\t\t}\n+\t}\n+\n+out:\n+\tmutex_unlock(\u0026qp-\u003emutex);\n+\treturn err;\n+}\n+\n+static int xsc_wq_overflow(struct xsc_ib_wq *wq, int nreq, struct xsc_ib_cq *cq)\n+{\n+\tunsigned int cur;\n+\n+\tcur = wq-\u003ehead - wq-\u003etail;\n+\tif (likely(cur + nreq \u003c wq-\u003emax_post))\n+\t\treturn 0;\n+\n+\tspin_lock(\u0026cq-\u003elock);\n+\tcur = wq-\u003ehead - wq-\u003etail;\n+\tspin_unlock(\u0026cq-\u003elock);\n+\n+\treturn cur + nreq \u003e= wq-\u003emax_post;\n+}\n+\n+static inline void xsc_post_send_db(struct xsc_ib_qp *qp,\n+\t\t\t\t struct xsc_core_device *xdev,\n+\t\t\t\t int nreq)\n+{\n+\tu32 next_pid;\n+\n+\tif (unlikely(!nreq))\n+\t\treturn;\n+\n+\tqp-\u003esq.head += nreq;\n+\n+\tnext_pid = qp-\u003esq.head \u003c\u003c (qp-\u003esq.wqe_shift - XSC_BASE_WQE_SHIFT);\n+\txsc_update_tx_db(xdev, qp-\u003edoorbell_qpn, next_pid);\n+}\n+\n+static inline u32 xsc_crc32(struct xsc_ib_dev *dev, u32 crc, u8 *buf, size_t len)\n+{\n+\tu32 i;\n+\n+\tfor (i = 0; i \u003c len; i++)\n+\t\tcrc = dev-\u003ecrc_32_table[(crc ^ buf[i]) \u0026 0xff] ^ (crc \u003e\u003e 8);\n+\n+\treturn crc;\n+}\n+\n+#define BTH_QPN_MASK\t\t(0x00ffffff)\n+#define BTH_PSN_MASK\t\t(0x00ffffff)\n+\n+/* Compute a partial ICRC for all the IB transport headers. */\n+static u32 xsc_icrc_hdr(struct xsc_ib_dev *dev, void *pkt, u32 size, u32 *icrc)\n+{\n+\tstruct iphdr *ip4h = NULL;\n+\tstruct ipv6hdr *ip6h = NULL;\n+\tstruct udphdr *udph;\n+\tstruct ib_unpacked_eth *eth;\n+\tstruct rxe_bth *bth;\n+\tstruct ib_unpacked_deth *deth;\n+\tstruct ib_unpacked_vlan *vlan;\n+\tint crc;\n+\tint crc_field_len;\n+\t__be16 l3_type;\n+\tu8 *l3_start;\n+\n+\tint hdr_size;\n+\n+\t/* pseudo header buffer size is calculate using ipv6 header size since\n+\t * it is bigger than ipv4\n+\t */\n+\tu8 pshdr[sizeof(struct udphdr) +\n+\t\tsizeof(struct ipv6hdr) +\n+\t\tsizeof(*bth) + sizeof(*deth)];\n+\n+\teth = pkt;\n+\n+\tif (eth-\u003etype == htons(ETH_P_8021Q)) {\n+\t\tvlan = (struct ib_unpacked_vlan *)(eth + 1);\n+\t\tl3_type = vlan-\u003etype;\n+\t\tl3_start = (u8 *)(vlan + 1);\n+\t\tsize -= 4;\n+\t} else {\n+\t\tl3_type = eth-\u003etype;\n+\t\tl3_start = (u8 *)(eth + 1);\n+\t}\n+\n+\thdr_size = sizeof(struct udphdr) +\n+\t\t(l3_type == htons(ETH_P_IP) ?\n+\t\tsizeof(struct iphdr) : sizeof(struct ipv6hdr));\n+\n+\tcrc_field_len = hdr_size + sizeof(*bth) + sizeof(*deth);\n+\n+\tif (crc_field_len != size) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"Unmatched hdr: expect %d actual %d\\n\",\n+\t\t\t crc_field_len, size);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tip4h = (struct iphdr *)(l3_start);\n+\tip6h = (struct ipv6hdr *)(l3_start);\n+\tudph = (struct udphdr *)(ip4h + 1);\n+\tbth = (struct rxe_bth *)(udph + 1);\n+\n+\tmemcpy(pshdr, l3_start, crc_field_len);\n+\n+\t/* This seed is the result of computing a CRC with a seed of\n+\t * 0xfffffff and 8 bytes of 0xff representing a masked LRH.\n+\t */\n+\tcrc = 0xdebb20e3;\n+\n+\tif (l3_type == htons(ETH_P_IP)) { /* IPv4 */\n+\t\tmemcpy(pshdr, ip4h, hdr_size);\n+\t\tip4h = (struct iphdr *)pshdr;\n+\t\tudph = (struct udphdr *)(ip4h + 1);\n+\n+\t\tip4h-\u003ettl = 0xff;\n+\t\tip4h-\u003echeck = CSUM_MANGLED_0;\n+\t\tip4h-\u003etos = 0xff;\n+\t} else {\t\t\t\t/* IPv6 */\n+\t\tmemcpy(pshdr, ip6h, hdr_size);\n+\t\tip6h = (struct ipv6hdr *)pshdr;\n+\t\tudph = (struct udphdr *)(ip6h + 1);\n+\n+\t\tmemset(ip6h-\u003eflow_lbl, 0xff, sizeof(ip6h-\u003eflow_lbl));\n+\t\tip6h-\u003epriority = 0xf;\n+\t\tip6h-\u003ehop_limit = 0xff;\n+\t}\n+\tudph-\u003echeck = CSUM_MANGLED_0;\n+\n+\tbth = (struct rxe_bth *)(udph + 1);\n+\t/* exclude bth.resv8a */\n+\tbth-\u003eqpn |= cpu_to_be32(~BTH_QPN_MASK);\n+\n+\t*icrc = xsc_crc32(dev, crc, pshdr, crc_field_len);\n+\n+\treturn 0;\n+}\n+\n+/* Routine for sending QP1 packets for RoCE V1 an V2 */\n+/* TODO: sq hdr buf should be created dynamically for mult entry */\n+static int build_qp1_send_v2(struct xsc_ib_dev *dev,\n+\t\t\t struct xsc_ib_qp *qp,\n+\t\t\t const struct ib_send_wr *wr,\n+\t\t\t struct ib_sge *sge,\n+\t\t\t int payload_size, u32 *crc)\n+{\n+\tstruct xsc_ib_ah *ah = container_of(ud_wr((struct ib_send_wr *)wr)-\u003eah, struct xsc_ib_ah,\n+\t\t\t\t\t ibah);\n+\tconst struct ib_gid_attr *sgid_attr = ah-\u003eibah.sgid_attr;\n+\tu16 ether_type;\n+\tunion ib_gid dgid;\n+\tbool is_eth = false;\n+\tbool is_vlan = false;\n+\tbool is_grh = false;\n+\tbool is_udp = false;\n+\tu8 ip_version = 0;\n+\tu16 vlan_id = 0xFFFF;\n+\tint rc = 0;\n+\tint cm_pcp = 0;\n+\tvoid *hdr_buf;\n+\n+\tmemset(\u0026qp-\u003eqp1_hdr, 0, sizeof(qp-\u003eqp1_hdr));\n+\n+\tif (!qp-\u003esq.hdr_buf) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"QP1 buffer is empty!\");\n+\t\treturn -ENOMEM;\n+\t}\n+\thdr_buf = (u8 *)qp-\u003esq.hdr_buf + MAX_QP1_SQ_HDR_SIZE_V2 * qp-\u003esq.mad_index;\n+\n+\tif (!sgid_attr || !sgid_attr-\u003endev) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"sgid_addr or ndev is null\\n\");\n+\t\treturn -ENXIO;\n+\t}\n+\n+\tif (is_vlan_dev(sgid_attr-\u003endev))\n+\t\tvlan_id = vlan_dev_vlan_id(sgid_attr-\u003endev);\n+\n+\tis_udp = sgid_attr-\u003egid_type == IB_GID_TYPE_ROCE_UDP_ENCAP;\n+\tmemcpy(\u0026dgid.raw, \u0026ah-\u003eav.rgid, 16);\n+\tif (is_udp) {\n+\t\tif (ipv6_addr_v4mapped((struct in6_addr *)\u0026sgid_attr-\u003egid)) {\n+\t\t\tip_version = 4;\n+\t\t\tether_type = ETH_P_IP;\n+\t\t} else {\n+\t\t\tip_version = 6;\n+\t\t\tether_type = ETH_P_IPV6;\n+\t\t}\n+\t\tis_grh = false;\n+\t} else {\n+\t\tether_type = ETH_P_IBOE;\n+\t\tis_grh = true;\n+\t}\n+\n+\tis_eth = true;\n+\tis_vlan = (vlan_id \u0026\u0026 (vlan_id \u003c 0x1000)) ? true : false;\n+\n+\tib_ud_header_init(payload_size, !is_eth, is_eth, is_vlan, is_grh,\n+\t\t\t ip_version, is_udp, 0, \u0026qp-\u003eqp1_hdr);\n+\n+\t/* ETH */\n+\tether_addr_copy(qp-\u003eqp1_hdr.eth.dmac_h, ah-\u003eav.rmac);\n+\tether_addr_copy(qp-\u003eqp1_hdr.eth.smac_h, dev-\u003enetdev-\u003edev_addr);\n+\n+\t/* For vlan, check the sgid for vlan existence */\n+\tif (!is_vlan) {\n+\t\tqp-\u003eqp1_hdr.eth.type = cpu_to_be16(ether_type);\n+\t} else {\n+\t\tif (dev-\u003exdev-\u003epriv.cm_pcp != DSCP_PCP_UNSET)\n+\t\t\tcm_pcp = dev-\u003exdev-\u003epriv.cm_pcp \u003c\u003c 13;\n+\t\telse\n+\t\t\tcm_pcp = (iboe_tos_to_sl(sgid_attr-\u003endev, ah-\u003eav.tclass) \u003c\u003c 13);\n+\t\tqp-\u003eqp1_hdr.vlan.type = cpu_to_be16(ether_type);\n+\t\tqp-\u003eqp1_hdr.vlan.tag = cpu_to_be16(vlan_id | cm_pcp);\n+\t}\n+\n+#define ECN_CAPABLE_TRANSPORT 0x2\n+\tif (is_grh || ip_version == 6) {\n+\t\tmemcpy(qp-\u003eqp1_hdr.grh.source_gid.raw, sgid_attr-\u003egid.raw,\n+\t\t sizeof(sgid_attr-\u003egid));\n+\t\tmemcpy(qp-\u003eqp1_hdr.grh.destination_gid.raw, ah-\u003eav.rgid,\n+\t\t sizeof(ah-\u003eav.rgid));\n+\t\tqp-\u003eqp1_hdr.grh.hop_limit = ah-\u003eav.hop_limit;\n+\n+\t\tif (dev-\u003exdev-\u003epriv.cm_dscp != DSCP_PCP_UNSET)\n+\t\t\tqp-\u003eqp1_hdr.grh.traffic_class =\n+\t\t\t\t(dev-\u003exdev-\u003epriv.cm_dscp \u003c\u003c 2) | ECN_CAPABLE_TRANSPORT;\n+\t\telse\n+\t\t\tqp-\u003eqp1_hdr.grh.traffic_class = ECN_CAPABLE_TRANSPORT;\n+\t}\n+\n+\tif (ip_version == 4) {\n+\t\tif (dev-\u003exdev-\u003epriv.cm_dscp != DSCP_PCP_UNSET)\n+\t\t\tqp-\u003eqp1_hdr.ip4.tos =\n+\t\t\t\t(dev-\u003exdev-\u003epriv.cm_dscp \u003c\u003c 2) | ECN_CAPABLE_TRANSPORT;\n+\t\telse\n+\t\t\tqp-\u003eqp1_hdr.ip4.tos = ECN_CAPABLE_TRANSPORT;\n+\t\tqp-\u003eqp1_hdr.ip4.id = 0;\n+\t\tqp-\u003eqp1_hdr.ip4.frag_off = htons(IP_DF);\n+\t\tqp-\u003eqp1_hdr.ip4.ttl = ah-\u003eav.hop_limit;\n+\n+\t\tmemcpy(\u0026qp-\u003eqp1_hdr.ip4.saddr, sgid_attr-\u003egid.raw + 12, 4);\n+\t\tmemcpy(\u0026qp-\u003eqp1_hdr.ip4.daddr, ah-\u003eav.rgid + 12, 4);\n+\t\tqp-\u003eqp1_hdr.ip4.check = ib_ud_ip4_csum(\u0026qp-\u003eqp1_hdr);\n+\t}\n+\n+\tif (is_udp) {\n+\t\tqp-\u003eqp1_hdr.udp.dport = htons(dev-\u003exdev-\u003ecm_udp_dst_port);\n+\t\tqp-\u003eqp1_hdr.udp.sport = htons(ah-\u003eav.udp_sport);\n+\t\tqp-\u003eqp1_hdr.udp.csum = 0;\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"CM packet used udp_sport=%d\\n\", ah-\u003eav.udp_sport);\n+\t}\n+\n+\t/* BTH */\n+\tif (wr-\u003eopcode == IB_WR_SEND_WITH_IMM) {\n+\t\tqp-\u003eqp1_hdr.bth.opcode = IB_OPCODE_UD_SEND_ONLY_WITH_IMMEDIATE;\n+\t\tqp-\u003eqp1_hdr.immediate_present = 1;\n+\t} else {\n+\t\tqp-\u003eqp1_hdr.bth.opcode = IB_OPCODE_UD_SEND_ONLY;\n+\t}\n+\tif (wr-\u003esend_flags \u0026 IB_SEND_SOLICITED)\n+\t\tqp-\u003eqp1_hdr.bth.solicited_event = 1;\n+\t/* pad_count */\n+\tqp-\u003eqp1_hdr.bth.pad_count = (4 - payload_size) \u0026 3;\n+\n+\t/* P_key for QP1 is for all members */\n+\tqp-\u003eqp1_hdr.bth.pkey = cpu_to_be16(0xFFFF);\n+\tqp-\u003eqp1_hdr.bth.destination_qpn = IB_QP1;\n+\tqp-\u003eqp1_hdr.bth.ack_req = 0;\n+\tqp-\u003esend_psn++;\n+\tqp-\u003esend_psn \u0026= BTH_PSN_MASK;\n+\tqp-\u003eqp1_hdr.bth.psn = cpu_to_be32(qp-\u003esend_psn);\n+\t/* DETH */\n+\t/* Use the privileged Q_Key for QP1 */\n+\tqp-\u003eqp1_hdr.deth.qkey = cpu_to_be32(IB_QP1_QKEY);\n+\tqp-\u003eqp1_hdr.deth.source_qpn = IB_QP1;\n+\n+\t/* Pack the QP1 to the transmit buffer */\n+\tsge-\u003eaddr = (dma_addr_t)(qp-\u003esq.hdr_dma + MAX_QP1_SQ_HDR_SIZE_V2 * qp-\u003esq.mad_index);\n+\tsge-\u003elkey = 0xFFFFFFFF;\n+\tsge-\u003elength = MAX_QP1_SQ_HDR_SIZE;\n+\n+\tib_ud_header_pack(\u0026qp-\u003eqp1_hdr, hdr_buf);\n+\t/*\n+\t * Max Header buf size for IPV6 RoCE V2 is 86,\n+\t * which is same as the QP1 SQ header buffer.\n+\t * Header buf size for IPV4 RoCE V2 can be 66.\n+\t * ETH(14) + VLAN(4)+ IP(20) + UDP (8) + BTH(20).\n+\t * Subtract 20 bytes from QP1 SQ header buf size\n+\t */\n+\tif (is_udp \u0026\u0026 ip_version == 4)\n+\t\tsge-\u003elength -= 20;\n+\t/*\n+\t * Max Header buf size for RoCE V1 is 78.\n+\t * ETH(14) + VLAN(4) + GRH(40) + BTH(20).\n+\t * Subtract 8 bytes from QP1 SQ header buf size\n+\t */\n+\tif (!is_udp)\n+\t\tsge-\u003elength -= 8;\n+\n+\t/* Subtract 4 bytes for non vlan packets */\n+\tif (!is_vlan)\n+\t\tsge-\u003elength -= 4;\n+\n+\trc = xsc_icrc_hdr(dev, hdr_buf, sge-\u003elength - sizeof(struct ib_unpacked_eth), crc);\n+\tif (rc) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"CRC error: hdr size %ld\\n\",\n+\t\t\t sge-\u003elength - sizeof(struct ib_unpacked_eth));\n+\t}\n+\treturn rc;\n+}\n+\n+static void zero_send_ds(struct xsc_ib_qp *qp, int idx)\n+{\n+\tvoid *seg;\n+\tint i;\n+\tint ds_num;\n+\tu64 *p;\n+\n+\tds_num = XSC_SEND_SEG_NUM \u003c\u003c (qp-\u003esq.wqe_shift - XSC_SEND_WQE_SHIFT);\n+\tseg = (void *)xsc_get_send_wqe(qp, idx);\n+\tfor (i = 1; i \u003c ds_num; i++) {\n+\t\tp = get_seg_wqe(seg, i);\n+\t\tp[0] = 0;\n+\t\tp[1] = 0;\n+\t}\n+}\n+\n+int xsc_ib_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr,\n+\t\t const struct ib_send_wr **bad_wr)\n+{\n+\tstruct xsc_ib_qp *qp = to_xqp(ibqp);\n+\tstruct xsc_ib_dev *dev = to_xdev(ibqp-\u003edevice);\n+\tvoid *seg;\n+\tstruct xsc_send_wqe_ctrl_seg *ctrl;\n+\tstruct xsc_wqe_data_seg *data_seg;\n+\tu32 crc;\n+\tint nreq = 0;\n+\tint err = 0;\n+\tint i;\n+\tunsigned int idx;\n+\tunsigned long irqflag = 0;\n+\tstruct ib_sge sg;\n+\tu8 *cur_p = NULL;\n+\tu8 *mad_send_base = NULL;\n+\tvoid *vaddr;\n+\tstruct xsc_msg_opcode *xsc_ib_opcode = dev-\u003exdev-\u003eboard_info-\u003emsg_opcode;\n+\n+\tspin_lock_irqsave(\u0026qp-\u003esq.lock, irqflag);\n+\n+\tfor (nreq = 0; wr; ++nreq, wr = wr-\u003enext) {\n+\t\tunsigned int seg_index = 1;\n+\t\tunsigned int msg_len = 0;\n+\t\tstruct ib_sge *sgl = \u0026wr-\u003esg_list[0];\n+\t\tint sg_n = wr-\u003enum_sge;\n+\n+\t\tif (unlikely(wr-\u003eopcode == IB_WR_REG_MR ||\n+\t\t\t wr-\u003eopcode == IB_WR_LOCAL_INV ||\n+\t\t\t wr-\u003eopcode == IB_WR_SEND_WITH_INV)) {\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"opcode %d not supported\\n\",\n+\t\t\t\t wr-\u003eopcode);\n+\t\t\terr = -EOPNOTSUPP;\n+\t\t\t*bad_wr = wr;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tif (unlikely(wr-\u003eopcode \u003c 0 || wr-\u003eopcode \u003e= ARRAY_SIZE(dev-\u003eopcode_map))) {\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"bad opcode %d\\n\", wr-\u003eopcode);\n+\t\t\terr = -EINVAL;\n+\t\t\t*bad_wr = wr;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tif (unlikely(xsc_wq_overflow(\u0026qp-\u003esq, nreq,\n+\t\t\t\t\t to_xcq(qp-\u003eibqp.send_cq)))) {\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"send work queue overflow\\n\");\n+\t\t\terr = -ENOMEM;\n+\t\t\t*bad_wr = wr;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tif (unlikely(wr-\u003enum_sge \u003e qp-\u003esq.max_gs)) {\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"max gs exceeded %d (max = %d)\\n\",\n+\t\t\t\t wr-\u003enum_sge, qp-\u003esq.max_gs);\n+\t\t\terr = -ENOMEM;\n+\t\t\t*bad_wr = wr;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tif (unlikely(wr-\u003eopcode == IB_WR_RDMA_READ \u0026\u0026 wr-\u003enum_sge \u003e 1)) {\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"rdma read, max gs exceeded %d (max = 1)\\n\",\n+\t\t\t\t wr-\u003enum_sge);\n+\t\t\terr = -ENOMEM;\n+\t\t\t*bad_wr = wr;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tif (unlikely(qp-\u003epost_send_invalid)) {\n+\t\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"qp %d has been draind, post send invalid...\\n\",\n+\t\t\t\t qp-\u003exqp.qpn);\n+\t\t\terr = -EIO;\n+\t\t\t*bad_wr = wr;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tidx = qp-\u003esq.cur_post \u0026 (qp-\u003esq.wqe_cnt - 1);\n+\t\tzero_send_ds(qp, idx);\n+\t\tseg = xsc_get_send_wqe(qp, idx);\n+\t\tctrl = seg;\n+\t\txsc_set_wqe_id(dev-\u003exdev, ctrl, qp-\u003esq.cur_post \u003c\u003c\n+\t\t\t (qp-\u003esq.wqe_shift - XSC_BASE_WQE_SHIFT));\n+\t\tctrl-\u003eds_data_num = 0;\n+\t\tctrl-\u003ese = wr-\u003esend_flags \u0026 IB_SEND_SOLICITED ? 1 : 0;\n+\t\tctrl-\u003ece = wr-\u003esend_flags \u0026 IB_SEND_SIGNALED ? 1 : 0;\n+\t\tfor (i = 0; i \u003c wr-\u003enum_sge; ++i) {\n+\t\t\tif (likely(wr-\u003esg_list[i].length))\n+\t\t\t\tmsg_len += wr-\u003esg_list[i].length;\n+\t\t}\n+\t\tctrl-\u003emsg_len = msg_len;\n+\t\tctrl-\u003ewith_immdt = 0;\n+\n+\t\tif (unlikely(wr-\u003eopcode == IB_WR_RDMA_READ \u0026\u0026 msg_len == 0)) {\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"rdma read, msg len should not be 0\\n\");\n+\t\t\t/* workaround, return success for posting zero-length read */\n+\t\t\terr = 0;\n+\t\t\tgoto out;\n+\t\t}\n+\t\tswitch (ibqp-\u003eqp_type) {\n+\t\tcase IB_QPT_RC:\n+\t\t\tctrl-\u003eds_data_num = wr-\u003enum_sge;\n+\t\t\tswitch (wr-\u003eopcode) {\n+\t\t\tcase IB_WR_SEND:\n+\t\t\t\tbreak;\n+\t\t\tcase IB_WR_SEND_WITH_IMM:\n+\t\t\t\tctrl-\u003ewith_immdt = 1;\n+\t\t\t\tWR_LE_32(ctrl-\u003eopcode_data, RD_BE_32(send_ieth(wr)));\n+\t\t\t\tbreak;\n+\t\t\tcase IB_WR_RDMA_WRITE_WITH_IMM:\n+\t\t\t\tctrl-\u003ewith_immdt = 1;\n+\t\t\t\tWR_LE_32(ctrl-\u003eopcode_data, RD_BE_32(send_ieth(wr)));\n+\t\t\t\tfallthrough;\n+\t\t\tcase IB_WR_RDMA_READ:\n+\t\t\tcase IB_WR_RDMA_WRITE:\n+\t\t\t\tif (ctrl-\u003emsg_len == 0)\n+\t\t\t\t\tbreak;\n+\t\t\t\tctrl-\u003eds_data_num++;\n+\t\t\t\tdata_seg = get_seg_wqe(ctrl, seg_index);\n+\t\t\t\tset_remote_addr_seg(dev-\u003exdev, data_seg,\n+\t\t\t\t\t\t msg_len,\n+\t\t\t\t\t\t rdma_wr(wr)-\u003eremote_addr,\n+\t\t\t\t\t\t rdma_wr(wr)-\u003erkey);\n+\t\t\t\tseg_index++;\n+\t\t\t\tbreak;\n+\t\t\tdefault:\n+\t\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"unsupported send opcode %u\\n\",\n+\t\t\t\t\t wr-\u003eopcode);\n+\t\t\t\terr = -EPERM;\n+\t\t\t\t*bad_wr = wr;\n+\t\t\t\tgoto out;\n+\t\t\t}\n+\t\t\tctrl-\u003emsg_opcode = dev-\u003eopcode_map[wr-\u003eopcode];\n+\t\t\tbreak;\n+\t\tcase IB_QPT_UD:\n+\t\tcase IB_QPT_GSI:\n+\t\t\tctrl-\u003emsg_opcode = xsc_ib_opcode-\u003emad;\n+\t\t\tctrl-\u003eds_data_num++;\n+\t\t\tdata_seg = get_seg_wqe(ctrl, seg_index);\n+\t\t\tmad_send_base = (u8 *)qp-\u003esq.hdr_buf +\n+\t\t\t\tMAX_QP1_SQ_HDR_SIZE_V2 * qp-\u003esq.mad_index;\n+\n+\t\t\terr = build_qp1_send_v2(dev, qp, wr, \u0026sg, msg_len, \u0026crc);\n+\t\t\tif (err) {\n+\t\t\t\t*bad_wr = wr;\n+\t\t\t\tgoto out;\n+\t\t\t}\n+\n+\t\t\tcur_p = mad_send_base + sg.length;\n+\t\t\tfor (i = 0; i \u003c wr-\u003enum_sge; ++i) {\n+\t\t\t\tif (likely(wr-\u003esg_list[i].length)) {\n+\t\t\t\t\tvaddr = xsc_ib_send_mad_sg_virt_addr(\u0026dev-\u003eib_dev, wr, i);\n+\t\t\t\t\tmemcpy(cur_p, vaddr, wr-\u003esg_list[i].length);\n+\t\t\t\t}\n+\t\t\t\tcur_p += wr-\u003esg_list[i].length;\n+\t\t\t}\n+\t\t\tcrc = xsc_crc32(dev, crc, mad_send_base + sg.length, ctrl-\u003emsg_len);\n+\t\t\tctrl-\u003emsg_len += sg.length;\n+\t\t\tseg_index++;\n+\n+\t\t\t*(u32 *)\u0026mad_send_base[ctrl-\u003emsg_len] = ~crc;\n+\t\t\tctrl-\u003emsg_len += sizeof(crc);\n+\t\t\tsg.length = ctrl-\u003emsg_len;\n+\t\t\tset_local_data_seg(dev-\u003exdev, data_seg, \u0026sg);\n+\t\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"qp[%d] send MAD packet, msg_len:%d\\n\",\n+\t\t\t\t qp-\u003exqp.qpn, ctrl-\u003emsg_len);\n+\t\t\tqp-\u003esq.mad_index = (qp-\u003esq.mad_index + 1) % MAD_QUEUE_DEPTH;\n+\n+\t\t\tsg_n = 0;\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"qp type:%u NOT supported\\n\", ibqp-\u003eqp_type);\n+\t\t\terr = -EPERM;\n+\t\t\t*bad_wr = wr;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tif (wr-\u003esend_flags \u0026 IB_SEND_INLINE \u0026\u0026 wr-\u003enum_sge) {\n+\t\t\terr = set_data_inl_seg(qp, wr, ctrl);\n+\t\t\tif (unlikely(err)) {\n+\t\t\t\t*bad_wr = wr;\n+\t\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"inline layout failed, err %d\\n\", err);\n+\t\t\t\tgoto out;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tfor (i = 0; i \u003c sg_n; ++i, ++seg_index) {\n+\t\t\t\tif (likely(sgl[i].length)) {\n+\t\t\t\t\tdata_seg = get_seg_wqe(ctrl, seg_index);\n+\t\t\t\t\tset_local_data_seg(dev-\u003exdev, data_seg, \u0026sgl[i]);\n+\t\t\t\t}\n+\t\t\t}\n+\t\t}\n+\t\tqp-\u003esq.wrid[idx] = wr-\u003ewr_id;\n+\t\tqp-\u003esq.wqe_head[idx] = qp-\u003esq.head + nreq;\n+\t\tqp-\u003esq.cur_post += 1;\n+\t\tif (ctrl-\u003ece) {\n+\t\t\tatomic_inc(\u0026qp-\u003esq.flush_wqe_cnt);\n+\t\t\tqp-\u003esq.need_flush[idx] = 1;\n+\t\t\tqp-\u003esq.prev_ce_wqe[idx] = qp-\u003esq.last_post_ce_wqe;\n+\t\t\tqp-\u003esq.last_post_ce_wqe = idx;\n+\t\t}\n+\t\tqp-\u003esq.wr_opcode[idx] = wr-\u003eopcode;\n+\t}\n+out:\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"nreq:%d\\n\", nreq);\n+\txsc_post_send_db(qp, dev-\u003exdev, nreq);\n+\tspin_unlock_irqrestore(\u0026qp-\u003esq.lock, irqflag);\n+\n+\treturn err;\n+}\n+\n+int xsc_ib_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr,\n+\t\t const struct ib_recv_wr **bad_wr)\n+{\n+\tstruct xsc_ib_qp *qp = to_xqp(ibqp);\n+\tstruct xsc_ib_dev *dev = to_xdev(ibqp-\u003edevice);\n+\tstruct xsc_core_device *xdev = dev-\u003exdev;\n+\tstruct xsc_wqe_data_seg *recv_head;\n+\tstruct xsc_wqe_data_seg *data_seg;\n+\tunsigned long flags = 0;\n+\tint err = 0;\n+\tu32 next_pid = 0;\n+\tint nreq = 0;\n+\tu32 idx;\n+\tint i;\n+\n+\tspin_lock_irqsave(\u0026qp-\u003erq.lock, flags);\n+\n+\tidx = qp-\u003erq.head \u0026 (qp-\u003erq.wqe_cnt - 1);\n+\n+\tfor (nreq = 0; wr; ++nreq, wr = wr-\u003enext) {\n+\t\tif (unlikely(xsc_wq_overflow(\u0026qp-\u003erq, nreq, to_xcq(qp-\u003eibqp.recv_cq)))) {\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"recv work queue overflow\\n\");\n+\t\t\terr = -ENOMEM;\n+\t\t\t*bad_wr = wr;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tif (unlikely(wr-\u003enum_sge \u003e qp-\u003erq.max_gs)) {\n+\t\t\tibdev_err(\u0026dev-\u003eib_dev, \"max gs exceeded %d (max = %d)\\n\",\n+\t\t\t\t wr-\u003enum_sge, qp-\u003erq.max_gs);\n+\t\t\terr = -EINVAL;\n+\t\t\t*bad_wr = wr;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tif (unlikely(qp-\u003epost_recv_invalid)) {\n+\t\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"qp %d has been draind, post recv invalid...\\n\",\n+\t\t\t\t qp-\u003exqp.qpn);\n+\t\t\terr = -EIO;\n+\t\t\t*bad_wr = wr;\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\trecv_head = get_recv_wqe(qp, idx);\n+\n+\t\tfor (i = 0; i \u003c wr-\u003enum_sge; ++i) {\n+\t\t\tif (unlikely(!wr-\u003esg_list[i].length))\n+\t\t\t\tcontinue;\n+\t\t\tdata_seg = get_seg_wqe(recv_head, i);\n+\t\t\tdata_seg-\u003ein_line = 0;\n+\t\t\tWR_LE_64(data_seg-\u003eva, wr-\u003esg_list[i].addr);\n+\t\t\tWR_LE_32(data_seg-\u003emkey, wr-\u003esg_list[i].lkey);\n+\t\t\tif (is_qp1(qp-\u003exqp.qp_type))\n+\t\t\t\tWR_LE_32(data_seg-\u003eseg_len, xdev-\u003ecaps.rx_pkt_len_max);\n+\t\t\telse\n+\t\t\t\tWR_LE_32(data_seg-\u003eseg_len, wr-\u003esg_list[i].length);\n+\t\t}\n+\n+\t\tqp-\u003erq.wrid[idx] = wr-\u003ewr_id;\n+\n+\t\tidx = (idx + 1) \u0026 (qp-\u003erq.wqe_cnt - 1);\n+\t\tatomic_inc(\u0026qp-\u003erq.flush_wqe_cnt);\n+\t}\n+\n+out:\n+\tif (likely(nreq)) {\n+\t\tqp-\u003erq.head += nreq;\n+\t\tnext_pid = qp-\u003erq.head \u003c\u003c (qp-\u003erq.wqe_shift - XSC_BASE_WQE_SHIFT);\n+\n+\t\txsc_update_rx_db(xdev, qp-\u003edoorbell_qpn, next_pid);\n+\t}\n+\n+\tspin_unlock_irqrestore(\u0026qp-\u003erq.lock, flags);\n+\n+\treturn err;\n+}\n+\n+static inline enum ib_qp_state to_ib_qp_state(enum xsc_qp_state xsc_state)\n+{\n+\tswitch (xsc_state) {\n+\tcase XSC_QP_STATE_RST: return IB_QPS_RESET;\n+\tcase XSC_QP_STATE_INIT: return IB_QPS_INIT;\n+\tcase XSC_QP_STATE_RTR: return IB_QPS_RTR;\n+\tcase XSC_QP_STATE_RTS: return IB_QPS_RTS;\n+\tcase XSC_QP_STATE_SQ_DRAINING:\n+\tcase XSC_QP_STATE_SQD: return IB_QPS_SQD;\n+\tcase XSC_QP_STATE_SQER: return IB_QPS_SQE;\n+\tcase XSC_QP_STATE_ERR: return IB_QPS_ERR;\n+\tdefault:\t\t return -1;\n+\t}\n+}\n+\n+static inline enum ib_mig_state to_ib_mig_state(int xsc_mig_state)\n+{\n+\tswitch (xsc_mig_state) {\n+\tcase XSC_QP_PM_ARMED:\t\treturn IB_MIG_ARMED;\n+\tcase XSC_QP_PM_REARM:\t\treturn IB_MIG_REARM;\n+\tcase XSC_QP_PM_MIGRATED:\treturn IB_MIG_MIGRATED;\n+\tdefault: return -1;\n+\t}\n+}\n+\n+int xsc_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr_mask,\n+\t\t struct ib_qp_init_attr *qp_init_attr)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(ibqp-\u003edevice);\n+\tstruct xsc_ib_qp *qp = to_xqp(ibqp);\n+\tstruct xsc_query_qp_mbox_out *outb;\n+\tstruct xsc_qp_context *context;\n+\tint err = 0;\n+\n+\tmutex_lock(\u0026qp-\u003emutex);\n+\n+\tif (qp_attr_mask \u0026 XSC_IB_QP_FLUSH) {\n+\t\txsc_ib_trigger_cq_comp_handler(qp-\u003esend_cq, qp-\u003erecv_cq);\n+\t\tgoto out;\n+\t}\n+\n+\toutb = kzalloc_obj(*outb, GFP_KERNEL);\n+\tif (!outb) {\n+\t\terr = -ENOMEM;\n+\t\tgoto out;\n+\t}\n+\tcontext = \u0026outb-\u003ectx;\n+\terr = xsc_core_qp_query(dev-\u003exdev, \u0026qp-\u003exqp, outb, sizeof(*outb));\n+\tif (err)\n+\t\tgoto out_free;\n+\n+\tif (qp-\u003exqp.err_occurred) {\n+\t\tqp-\u003estate = IB_QPS_ERR;\n+\t\tqp-\u003exqp.err_occurred = 0;\n+\t}\n+\tqp_attr-\u003eqp_state\t = qp-\u003estate;\n+\tqp_attr-\u003epath_mtu\t = context-\u003emtu_mode ? IB_MTU_4096 : IB_MTU_1024;\n+\tqp_attr-\u003erq_psn\t\t = be32_to_cpu(context-\u003enext_recv_psn) \u0026 0xffffff;\n+\tqp_attr-\u003esq_psn\t\t = be32_to_cpu(context-\u003enext_send_psn) \u0026 0xffffff;\n+\tqp_attr-\u003edest_qp_num\t = be32_to_cpu(context-\u003eremote_qpn) \u0026 0xffffff;\n+\tqp_attr-\u003esq_draining = 0;\n+\tqp_attr-\u003eretry_cnt\t = context-\u003eretry_cnt;\n+\tqp_attr-\u003ernr_retry\t = context-\u003ernr_retry;\n+\tqp_attr-\u003ecur_qp_state\t = qp_attr-\u003eqp_state;\n+\tqp_attr-\u003ecap.max_recv_wr = qp-\u003erq.wqe_cnt;\n+\tqp_attr-\u003ecap.max_recv_sge = qp-\u003erq.max_gs;\n+\tqp_attr-\u003emax_rd_atomic = context-\u003emax_sra;\n+\tqp_attr-\u003emax_dest_rd_atomic = context-\u003emax_rra;\n+\tqp_attr-\u003eport_num = 1;\n+\n+\tif (!ibqp-\u003euobject) {\n+\t\tqp_attr-\u003ecap.max_send_wr = qp-\u003esq.wqe_cnt;\n+\t\tqp_attr-\u003ecap.max_send_sge = qp-\u003esq.max_gs;\n+\t} else {\n+\t\tqp_attr-\u003ecap.max_send_wr = qp-\u003esq.wqe_cnt;\n+\t\tqp_attr-\u003ecap.max_send_sge = qp-\u003esq.max_gs;\n+\t}\n+\n+\t/* We don't support inline sends for kernel QPs (yet), and we\n+\t * don't know what userspace's value should be.\n+\t */\n+\tqp_attr-\u003ecap.max_inline_data = 0;\n+\n+\tqp_init_attr-\u003ecap\t = qp_attr-\u003ecap;\n+\n+\tqp_init_attr-\u003ecreate_flags = 0;\n+\tif (qp-\u003eflags \u0026 XSC_IB_QP_BLOCK_MULTICAST_LOOPBACK)\n+\t\tqp_init_attr-\u003ecreate_flags |= IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK;\n+\n+\tqp_init_attr-\u003esq_sig_type = qp-\u003esq_signal_bits \u0026 XSC_WQE_CTRL_CQ_UPDATE ?\n+\t\tIB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR;\n+\n+out_free:\n+\tkfree(outb);\n+\n+out:\n+\tmutex_unlock(\u0026qp-\u003emutex);\n+\treturn err;\n+}\n+\n+struct xsc_ib_drain_cqe {\n+\tstruct ib_cqe cqe;\n+\tstruct completion done;\n+\tint qpn;\n+};\n+\n+static void xsc_ib_drain_rq_done(struct ib_cq *cq, struct ib_wc *wc)\n+{\n+\tstruct xsc_ib_drain_cqe *cqe = container_of(wc-\u003ewr_cqe,\n+\t\t\t\t\t\t struct xsc_ib_drain_cqe, cqe);\n+\tstruct xsc_ib_dev *dev = to_xdev(cq-\u003edevice);\n+\n+\tcomplete(\u0026cqe-\u003edone);\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"QP %d ib_drain_rq done!\", cqe-\u003eqpn);\n+}\n+\n+static void xsc_ib_drain_sq_done(struct ib_cq *cq, struct ib_wc *wc)\n+{\n+\tstruct xsc_ib_drain_cqe *cqe = container_of(wc-\u003ewr_cqe,\n+\t\t\t\t\t\t struct xsc_ib_drain_cqe, cqe);\n+\tstruct xsc_ib_dev *dev = to_xdev(cq-\u003edevice);\n+\n+\tcomplete(\u0026cqe-\u003edone);\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"QP %d ib_drain_sq done!\", cqe-\u003eqpn);\n+}\n+\n+void xsc_ib_drain_rq(struct ib_qp *qp)\n+{\n+\tstruct ib_cq *cq = qp-\u003erecv_cq;\n+\tstruct xsc_ib_cq *xcq = to_xcq(cq);\n+\tstruct xsc_ib_qp *xqp = to_xqp(qp);\n+\tstruct ib_qp_attr attr = { .qp_state = IB_QPS_ERR };\n+\tstruct xsc_ib_drain_cqe rdrain;\n+\tconst struct ib_recv_wr rwr = {\n+\t\t.next = NULL,\n+\t\t{.wr_cqe = \u0026rdrain.cqe,},\n+\t};\n+\tconst struct ib_recv_wr *bad_rwr;\n+\tint ret;\n+\tstruct xsc_ib_dev *dev = to_xdev(qp-\u003edevice);\n+\n+\tret = ib_modify_qp(qp, \u0026attr, IB_QP_STATE);\n+\tif (ret) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"failed to drain recv queue: %d\\n\", ret);\n+\t\treturn;\n+\t}\n+\n+\trdrain.qpn = xqp-\u003exqp.qpn;\n+\trdrain.cqe.done = xsc_ib_drain_rq_done;\n+\tinit_completion(\u0026rdrain.done);\n+\n+\tret = xsc_ib_post_recv(qp, \u0026rwr, \u0026bad_rwr);\n+\tif (ret) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"failed to drain recv queue: %d\\n\", ret);\n+\t\treturn;\n+\t}\n+\txqp-\u003epost_recv_invalid = true;\n+\n+\txcq-\u003excq.comp(\u0026xcq-\u003excq);\n+\twait_for_completion(\u0026rdrain.done);\n+}\n+\n+void xsc_ib_drain_sq(struct ib_qp *qp)\n+{\n+\tstruct ib_cq *cq = qp-\u003esend_cq;\n+\tstruct xsc_ib_cq *xcq = to_xcq(cq);\n+\tstruct xsc_ib_qp *xqp = to_xqp(qp);\n+\tstruct ib_qp_attr attr = { .qp_state = IB_QPS_ERR };\n+\tstruct xsc_ib_drain_cqe sdrain;\n+\tconst struct ib_send_wr *bad_swr;\n+\tconst struct ib_send_wr swr = {\n+\t\t.next = NULL,\n+\t\t{ .wr_cqe\t= \u0026sdrain.cqe, },\n+\t\t.opcode\t= IB_WR_RDMA_WRITE,\n+\t\t.send_flags = IB_SEND_SIGNALED,\n+\t};\n+\tint ret;\n+\tstruct xsc_ib_dev *dev = to_xdev(qp-\u003edevice);\n+\n+\tret = ib_modify_qp(qp, \u0026attr, IB_QP_STATE);\n+\tif (ret) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"failed to drain send queue: %d\\n\", ret);\n+\t\treturn;\n+\t}\n+\n+\tsdrain.qpn = xqp-\u003exqp.qpn;\n+\tsdrain.cqe.done = xsc_ib_drain_sq_done;\n+\tinit_completion(\u0026sdrain.done);\n+\n+\tret = xsc_ib_post_send(qp, \u0026swr, \u0026bad_swr);\n+\tif (ret) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"failed to drain send queue: %d\\n\", ret);\n+\t\treturn;\n+\t}\n+\txqp-\u003epost_send_invalid = true;\n+\n+\txcq-\u003excq.comp(\u0026xcq-\u003excq);\n+\twait_for_completion(\u0026sdrain.done);\n+}\ndiff --git a/drivers/infiniband/hw/xsc/srq.c b/drivers/infiniband/hw/xsc/srq.c\nnew file mode 100644\nindex 0000000000000..911820fa1e7e7\n--- /dev/null\n+++ b/drivers/infiniband/hw/xsc/srq.c\n@@ -0,0 +1,426 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/slab.h\u003e\n+#include \u003crdma/ib_umem.h\u003e\n+#include \u003crdma/ib_user_verbs.h\u003e\n+#include \"xsc_ib.h\"\n+#include \"common/srq.h\"\n+#include \u003crdma/xsc-abi.h\u003e\n+#include \u003clinux/list_sort.h\u003e\n+\n+static int xsc_init_cache_wr(struct ib_srq_init_attr *attr, struct xsc_ib_srq *srq)\n+{\n+\tint srqn = srq-\u003exsrq.srqn;\n+\tstruct xsc_srq_cache_wr *cache_wr = srq-\u003exsrq.cache_wr;\n+\n+\tif (srqn \u003e= XSC_MAX_SRQ_NUM)\n+\t\treturn -EINVAL;\n+\n+\tmemset(\u0026cache_wr[srqn], 0, sizeof(struct xsc_srq_cache_wr));\n+\tcache_wr[srqn].max_wr_num = attr-\u003eattr.max_wr;\n+\treturn 0;\n+}\n+\n+void xsc_restore_cache_srq_wrs(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp)\n+{\n+\tint srqn = srq-\u003exsrq.srqn;\n+\tstruct xsc_srq_cache_wr *cache_wr = srq-\u003exsrq.cache_wr;\n+\n+\tif (qp-\u003esrq_tail_wr)\n+\t\tqp-\u003esrq_tail_wr-\u003enext = cache_wr[srqn].head_wr;\n+\tif (cache_wr[srqn].head_wr \u0026\u0026 qp-\u003esrq_head_wr)\n+\t\tcache_wr[srqn].head_wr = qp-\u003esrq_head_wr;\n+}\n+\n+void xsc_free_one_cache_srq_wr(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp, u64 wr_id)\n+{\n+\tstruct ib_recv_wr *tmp_wr, *prev_wr = NULL;\n+\tint srqn = srq-\u003exsrq.srqn;\n+\tstruct xsc_srq_cache_wr *cache_wr = srq-\u003exsrq.cache_wr;\n+\tunsigned long flags;\n+\n+\tspin_lock_irqsave(\u0026srq-\u003elock, flags);\n+\ttmp_wr = qp-\u003esrq_head_wr;\n+\tfor (; tmp_wr; tmp_wr = tmp_wr-\u003enext) {\n+\t\tif (tmp_wr-\u003ewr_id == wr_id) {\n+\t\t\tif (prev_wr)\n+\t\t\t\tprev_wr-\u003enext = tmp_wr-\u003enext;\n+\t\t\telse\n+\t\t\t\tqp-\u003esrq_head_wr = tmp_wr-\u003enext;\n+\n+\t\t\tkfree(tmp_wr-\u003esg_list);\n+\t\t\tkfree(tmp_wr);\n+\t\t\tbreak;\n+\t\t}\n+\t\tprev_wr = tmp_wr;\n+\t}\n+\tcache_wr[srqn].wr_num--;\n+\tspin_unlock_irqrestore(\u0026srq-\u003elock, flags);\n+}\n+\n+void xsc_free_all_cache_wrs(struct ib_recv_wr *wr)\n+{\n+\tstruct ib_recv_wr *prev_wr = NULL;\n+\n+\tfor (; wr; wr = wr-\u003enext) {\n+\t\tkfree(prev_wr);\n+\t\tkfree(wr-\u003esg_list);\n+\t\tprev_wr = wr;\n+\t}\n+\tkfree(prev_wr);\n+}\n+\n+static void xsc_destroy_cache_wr(struct xsc_ib_srq *srq)\n+{\n+\tint srqn = srq-\u003exsrq.srqn;\n+\tstruct xsc_srq_cache_wr *cache_wr = srq-\u003exsrq.cache_wr;\n+\n+\txsc_free_all_cache_wrs(cache_wr[srqn].head_wr);\n+\tmemset(\u0026cache_wr[srqn], 0, sizeof(struct xsc_srq_cache_wr));\n+}\n+\n+static void xsc_ib_srq_event(struct xsc_core_srq *srq, enum xsc_event type)\n+{\n+\tstruct ib_event event;\n+\tstruct ib_srq *ibsrq = \u0026core_to_xsrq(srq)-\u003eibsrq;\n+\n+\tif (ibsrq-\u003eevent_handler) {\n+\t\tevent.device = ibsrq-\u003edevice;\n+\t\tevent.element.srq = ibsrq;\n+\t\tswitch (type) {\n+\t\tdefault:\n+\t\t\tpr_warn(\"xsc_ib: Unexpected event type %d on SRQ %06x\\n\",\n+\t\t\t\ttype, srq-\u003esrqn);\n+\t\t\treturn;\n+\t\t}\n+\n+\t\tibsrq-\u003eevent_handler(\u0026event, ibsrq-\u003esrq_context);\n+\t}\n+}\n+\n+static int create_srq_user(struct ib_pd *pd, struct xsc_ib_srq *srq,\n+\t\t\t struct ib_srq_init_attr *init_attr,\n+\t\t\t struct ib_udata *udata, int buf_size)\n+{\n+\tstruct xsc_ib_dev *dev = to_xdev(pd-\u003edevice);\n+\tstruct xsc_ib_create_srq ucmd = {};\n+\tsize_t ucmdlen;\n+\n+\tucmdlen = min(udata-\u003einlen, sizeof(ucmd));\n+\n+\tif (ib_copy_from_udata(\u0026ucmd, udata, ucmdlen)) {\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"failed copy udata\\n\");\n+\t\treturn -EFAULT;\n+\t}\n+\n+\tif (ucmd.reserved0 || ucmd.reserved1)\n+\t\treturn -EINVAL;\n+\n+\tif (udata-\u003einlen \u003e sizeof(ucmd) \u0026\u0026\n+\t !ib_is_udata_cleared(udata, sizeof(ucmd),\n+\t\t\t\t udata-\u003einlen - sizeof(ucmd)))\n+\t\treturn -EINVAL;\n+\n+\tsrq-\u003ecreate_type = XSC_SRQ_USER;\n+\n+\treturn 0;\n+}\n+\n+static int create_srq_kernel(struct xsc_ib_dev *dev, struct xsc_ib_srq *srq,\n+\t\t\t struct ib_srq_init_attr *init_attr, int buf_size)\n+{\n+\tsrq-\u003ecreate_type = XSC_SRQ_KERNEL;\n+\treturn 0;\n+}\n+\n+static void destroy_srq_user(struct xsc_ib_srq *srq, struct ib_udata *udata)\n+{\n+}\n+\n+static void destroy_srq_kernel(struct xsc_ib_dev *dev, struct xsc_ib_srq *srq)\n+{\n+}\n+\n+int xsc_ib_create_srq(struct ib_srq *ib_srq,\n+\t\t struct ib_srq_init_attr *init_attr,\n+\t\t struct ib_udata *udata)\n+{\n+\tstruct xsc_ib_dev *dev;\n+\tstruct xsc_ib_srq *srq;\n+\tint err;\n+\tu32 max_srq_wqes = 1 \u003c\u003c XSC_MAX_SRQ_LOG_SIZE;\n+\tstruct ib_pd *pd = ib_srq-\u003epd;\n+\n+\tif (pd) {\n+\t\tdev = to_xdev(pd-\u003edevice);\n+\t} else {\n+\t\tpr_warn(\"%s: no PD for SRQ\\n\", __func__);\n+\t\terr = -EINVAL;\n+\t\tgoto err_ret;\n+\t}\n+\n+\tif (init_attr-\u003esrq_type != IB_SRQT_BASIC) {\n+\t\terr = -EOPNOTSUPP;\n+\t\tgoto err_ret;\n+\t}\n+\n+\t/* Sanity check SRQ size before proceeding */\n+\tif (init_attr-\u003eattr.max_wr \u003e= max_srq_wqes) {\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"max_wr %d, cap %d\\n\",\n+\t\t\t init_attr-\u003eattr.max_wr,\n+\t\t\t max_srq_wqes);\n+\t\terr = -EINVAL;\n+\t\tgoto err_ret;\n+\t}\n+\n+\tsrq = to_xsrq(ib_srq);\n+\n+\tmutex_init(\u0026srq-\u003emutex);\n+\tspin_lock_init(\u0026srq-\u003elock);\n+\tsrq-\u003exsrq.max = roundup_pow_of_two(init_attr-\u003eattr.max_wr + 1);\n+\tsrq-\u003exsrq.max_gs = init_attr-\u003eattr.max_sge;\n+\n+\tif (udata)\n+\t\terr = create_srq_user(pd, srq, NULL, udata, 0);\n+\telse\n+\t\terr = create_srq_kernel(dev, srq, init_attr, 0);\n+\n+\tif (err) {\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"create srq rsc failed, ret %d\", err);\n+\t\tgoto err_create_rsc;\n+\t}\n+\n+\terr = xsc_core_create_srq(dev-\u003exdev, \u0026srq-\u003exsrq);\n+\tif (err) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"create srq failed, err=%d\\n\", err);\n+\t\tgoto err_create;\n+\t}\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"create SRQ with srqn 0x%x, max_wr %d\\n\",\n+\t\t srq-\u003exsrq.srqn, init_attr-\u003eattr.max_wr);\n+\n+\tsrq-\u003exsrq.event = xsc_ib_srq_event;\n+\tsrq-\u003eibsrq.ext.xrc.srq_num = srq-\u003exsrq.srqn;\n+\n+\tif (udata) {\n+\t\tstruct xsc_ib_create_srq_resp resp = {\n+\t\t\t.srqn = srq-\u003exsrq.srqn,\n+\t\t};\n+\n+\t\tif (ib_copy_to_udata(udata, \u0026resp,\n+\t\t\t\t min(udata-\u003eoutlen, sizeof(resp)))) {\n+\t\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"copy to user failed\\n\");\n+\t\t\terr = -EFAULT;\n+\t\t\tgoto err_rsp;\n+\t\t}\n+\t} else {\n+\t\terr = xsc_init_cache_wr(init_attr, srq);\n+\t\tif (err)\n+\t\t\tgoto err_rsp;\n+\t\tINIT_LIST_HEAD(\u0026srq-\u003exsrq.srq_qp_list);\n+\t}\n+\n+\tinit_attr-\u003eattr.max_wr = srq-\u003exsrq.max - 1;\n+\n+\treturn 0;\n+err_rsp:\n+\txsc_core_destroy_srq(dev-\u003exdev, \u0026srq-\u003exsrq);\n+err_create:\n+\tif (udata)\n+\t\tdestroy_srq_user(srq, udata);\n+\telse\n+\t\tdestroy_srq_kernel(dev, srq);\n+err_create_rsc:\n+err_ret:\n+\treturn err;\n+}\n+\n+int xsc_ib_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr,\n+\t\t enum ib_srq_attr_mask attr_mask, struct ib_udata *udata)\n+{\n+\t/* We don't support resizing SRQs yet */\n+\tif (attr_mask \u0026 IB_SRQ_MAX_WR)\n+\t\treturn -EINVAL;\n+\n+\tif (attr_mask \u0026 IB_SRQ_LIMIT)\n+\t\treturn -EOPNOTSUPP;\n+\n+\treturn 0;\n+}\n+\n+int xsc_ib_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *srq_attr)\n+{\n+\tstruct xsc_ib_srq *srq = to_xsrq(ibsrq);\n+\n+\tsrq_attr-\u003esrq_limit = 0;\n+\tsrq_attr-\u003emax_wr = srq-\u003exsrq.max - 1;\n+\tsrq_attr-\u003emax_sge = srq-\u003exsrq.max_gs;\n+\n+\treturn 0;\n+}\n+\n+int xsc_ib_destroy_srq(struct ib_srq *srq, struct ib_udata *udata)\n+{\n+\tstruct xsc_ib_srq *xsrq = to_xsrq(srq);\n+\tstruct xsc_ib_dev *dev = to_xdev(srq-\u003edevice);\n+\n+\txsc_core_destroy_srq(dev-\u003exdev, \u0026xsrq-\u003exsrq);\n+\tif (xsrq-\u003ecreate_type == XSC_SRQ_USER) {\n+\t\tdestroy_srq_user(xsrq, NULL);\n+\t} else {\n+\t\txsc_destroy_cache_wr(xsrq);\n+\t\tdestroy_srq_kernel(dev, xsrq);\n+\t}\n+\n+\tibdev_dbg(\u0026dev-\u003eib_dev, \"destroy srq %d successfully.\\n\", xsrq-\u003exsrq.srqn);\n+\treturn 0;\n+}\n+\n+static int cmp_flush_wqe_cnt(void *priv,\n+\t\t\t const struct list_head *a,\n+\t\t\t const struct list_head *b)\n+{\n+\tconst struct xsc_srq_qp_node *na =\n+\t\tlist_entry(a, struct xsc_srq_qp_node, entry);\n+\tconst struct xsc_srq_qp_node *nb =\n+\t\tlist_entry(b, struct xsc_srq_qp_node, entry);\n+\n+\tint fa = atomic_read(\u0026(to_xqp(na-\u003eibqp)-\u003erq.flush_wqe_cnt));\n+\tint fb = atomic_read(\u0026(to_xqp(nb-\u003eibqp)-\u003erq.flush_wqe_cnt));\n+\n+\treturn fa - fb;\n+}\n+\n+static int xsc_cache_srq_wr(struct xsc_ib_srq *srq, const struct ib_recv_wr *wr, int *nreq)\n+{\n+\tint i, n;\n+\tint srqn = srq-\u003exsrq.srqn;\n+\tstruct xsc_srq_cache_wr *cache_wr = srq-\u003exsrq.cache_wr;\n+\tstruct ib_recv_wr *head_wr = NULL, *new_wr = NULL, *prev_wr = NULL;\n+\n+\tfor (n = 0; wr; wr = wr-\u003enext, n++) {\n+\t\tnew_wr = kcalloc(1, sizeof(struct ib_recv_wr), GFP_KERNEL);\n+\t\tif (prev_wr)\n+\t\t\tprev_wr-\u003enext = new_wr;\n+\t\telse\n+\t\t\thead_wr = new_wr;\n+\t\tnew_wr-\u003ewr_id = wr-\u003ewr_id;\n+\t\tnew_wr-\u003esg_list = kcalloc(wr-\u003enum_sge, sizeof(struct ib_sge), GFP_KERNEL);\n+\t\tfor (i = 0; i \u003c wr-\u003enum_sge; i++)\n+\t\t\tmemcpy(\u0026new_wr-\u003esg_list[i], \u0026wr-\u003esg_list[i], sizeof(struct ib_sge));\n+\t\tnew_wr-\u003enum_sge = wr-\u003enum_sge;\n+\t\tprev_wr = new_wr;\n+\t};\n+\n+\tif (!cache_wr[srqn].head_wr)\n+\t\tcache_wr[srqn].head_wr = head_wr;\n+\telse\n+\t\tcache_wr[srqn].tail_wr-\u003enext = head_wr;\n+\n+\tcache_wr[srqn].tail_wr = new_wr;\n+\tcache_wr[srqn].wr_num += n;\n+\t*nreq = n;\n+\n+\tif (cache_wr[srqn].wr_num \u003e cache_wr[srqn].max_wr_num)\n+\t\treturn -1;\n+\telse\n+\t\treturn 0;\n+}\n+\n+void xsc_post_cache_wrs(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp, int wr_num, bool init)\n+{\n+\tint srqn = srq-\u003exsrq.srqn;\n+\tstruct xsc_srq_cache_wr *cache_wr = srq-\u003exsrq.cache_wr;\n+\tint err = 0;\n+\tint i;\n+\tconst struct ib_recv_wr *tmp_wr;\n+\tconst struct ib_recv_wr *bad_wr;\n+\tstruct ib_recv_wr *wr_head, *next_wr;\n+\n+\tif (!cache_wr[srqn].head_wr)\n+\t\treturn;\n+\n+\twr_head = cache_wr[srqn].head_wr;\n+\tnext_wr = cache_wr[srqn].head_wr;\n+\n+\tif (init)\n+\t\tqp-\u003esrq_head_wr = wr_head;\n+\telse\n+\t\tqp-\u003esrq_tail_wr-\u003enext = wr_head;\n+\n+\tqp-\u003esrq_tail_wr = wr_head;\n+\tfor (i = 0; next_wr \u0026\u0026 i \u003c (wr_num - 1); i++) {\n+\t\tnext_wr = next_wr-\u003enext;\n+\t\tif (next_wr)\n+\t\t\tqp-\u003esrq_tail_wr = next_wr;\n+\t}\n+\n+\tcache_wr[srqn].head_wr = qp-\u003esrq_tail_wr-\u003enext;\n+\tqp-\u003esrq_tail_wr-\u003enext = NULL;\n+\ttmp_wr = wr_head;\n+\terr = xsc_ib_post_recv(\u0026qp-\u003eibqp, tmp_wr, \u0026bad_wr);\n+\tif (err) {\n+\t\tqp-\u003esrq_tail_wr-\u003enext = cache_wr[srqn].head_wr;\n+\t\tqp-\u003esrq_tail_wr = NULL;\n+\t\tfor (; wr_head \u0026\u0026 (bad_wr-\u003ewr_id != wr_head-\u003ewr_id); wr_head = wr_head-\u003enext)\n+\t\t\tqp-\u003esrq_tail_wr = wr_head;\n+\n+\t\tif (!qp-\u003esrq_tail_wr) {\n+\t\t\tcache_wr[srqn].head_wr = wr_head;\n+\t\t\tqp-\u003esrq_head_wr = NULL;\n+\t\t} else {\n+\t\t\tcache_wr[srq-\u003exsrq.srqn].head_wr = qp-\u003esrq_tail_wr-\u003enext;\n+\t\t\tqp-\u003esrq_tail_wr-\u003enext = NULL;\n+\t\t}\n+\t}\n+}\n+\n+int xsc_ib_post_srq_recv(struct ib_srq *ibsrq, const struct ib_recv_wr *wr,\n+\t\t\t const struct ib_recv_wr **bad_wr)\n+{\n+\tstruct xsc_ib_srq *srq = to_xsrq(ibsrq);\n+\tstruct xsc_ib_dev *dev = to_xdev(ibsrq-\u003edevice);\n+\tstruct xsc_srq_qp_node *qp_node;\n+\tstruct xsc_ib_qp *qp;\n+\tunsigned long flags;\n+\tint err = 0;\n+\tint flush_wqe_cnt = 0;\n+\tint nreq = 0;\n+\tint wr_num = 0;\n+\n+\tspin_lock_irqsave(\u0026srq-\u003elock, flags);\n+\n+\terr = xsc_cache_srq_wr(srq, wr, \u0026nreq);\n+\tif (err) {\n+\t\tibdev_err(\u0026dev-\u003eib_dev, \"No more space to cache SRQ WRs...\");\n+\t\terr = -EINVAL;\n+\t\t*bad_wr = wr;\n+\t\tgoto out;\n+\t}\n+\n+\tif (list_empty(\u0026srq-\u003exsrq.srq_qp_list))\n+\t\tgoto out;\n+\n+\tlist_sort(NULL, \u0026srq-\u003exsrq.srq_qp_list, cmp_flush_wqe_cnt);\n+\n+\tlist_for_each_entry(qp_node, \u0026srq-\u003exsrq.srq_qp_list, entry) {\n+\t\tif (!qp_node || nreq \u003c= 0)\n+\t\t\tbreak;\n+\n+\t\tqp = to_xqp(qp_node-\u003eibqp);\n+\t\tflush_wqe_cnt = atomic_read(\u0026qp-\u003erq.flush_wqe_cnt);\n+\t\twr_num = min_t(int, nreq, (XSC_SRQ_MIN_WR_NUM - flush_wqe_cnt));\n+\t\tnreq -= wr_num;\n+\t\tibdev_dbg(\u0026dev-\u003eib_dev, \"srq_qp_list qp node qp %d, flush cnt %d, nreq %d, wr_num %d\\n\",\n+\t\t\t qp_node-\u003eqp_id, flush_wqe_cnt, nreq, wr_num);\n+\t\txsc_post_cache_wrs(srq, qp, wr_num, false);\n+\t}\n+\n+out:\n+\tspin_unlock_irqrestore(\u0026srq-\u003elock, flags);\n+\n+\treturn err;\n+}\ndiff --git a/drivers/infiniband/hw/xsc/xsc_ib.h b/drivers/infiniband/hw/xsc/xsc_ib.h\nnew file mode 100644\nindex 0000000000000..a2ae382e22ce8\n--- /dev/null\n+++ b/drivers/infiniband/hw/xsc/xsc_ib.h\n@@ -0,0 +1,542 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/*\n+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef XSC_IB_H\n+#define XSC_IB_H\n+\n+#include \u003clinux/kernel.h\u003e\n+#include \u003clinux/sched.h\u003e\n+#include \u003crdma/ib_verbs.h\u003e\n+#include \u003crdma/ib_smi.h\u003e\n+#include \u003crdma/ib_pack.h\u003e\n+#include \u003crdma/ib_mad.h\u003e\n+#include \"common/xsc_core.h\"\n+#include \"common/driver.h\"\n+#include \"common/cq.h\"\n+#include \"common/srq.h\"\n+#include \"common/qp.h\"\n+#include \u003clinux/types.h\u003e\n+#include \u003clinux/iommu.h\u003e\n+#include \u003clinux/dma-direct.h\u003e\n+#include \u003clinux/dma-mapping.h\u003e\n+#include \u003ccrypto/hash.h\u003e\n+#include \u003crdma/uverbs_ioctl.h\u003e\n+\n+#define\tXSC_PAGE_SHIFT_4K\t12\n+#define\tXSC_PAGE_SHIFT_64K\t16\n+#define\tXSC_PAGE_SHIFT_2M\t21\n+#define\tXSC_PAGE_SHIFT_1G\t30\n+#define XSC_PAGE_SZ_4K\t\tBIT(XSC_PAGE_SHIFT_4K)\n+#define XSC_PAGE_SZ_64K\t\tBIT(XSC_PAGE_SHIFT_64K)\n+#define XSC_PAGE_SZ_2M\t\tBIT(XSC_PAGE_SHIFT_2M)\n+#define XSC_PAGE_SZ_1G\t\tBIT(XSC_PAGE_SHIFT_1G)\n+#define XSC_MR_PAGE_CAP_MASK\t(XSC_PAGE_SZ_4K |\t\\\n+\t\t\t\tXSC_PAGE_SZ_64K |\t\\\n+\t\t\t\tXSC_PAGE_SZ_2M |\t\\\n+\t\t\t\tXSC_PAGE_SZ_1G)\n+\n+\n+struct xsc_ib_ucontext {\n+\tstruct ib_ucontext\tibucontext;\n+};\n+\n+struct xsc_ib_pd {\n+\tstruct ib_pd\t\tibpd;\n+\tu32\t\t\tpdn;\n+\tu32\t\t\tpa_lkey;\n+};\n+\n+struct wr_list {\n+\tu16\topcode;\n+\tu16\tnext;\n+};\n+\n+struct xsc_ib_wq {\n+\tu64\t\t *wrid;\n+\tu32\t\t *wr_data;\n+\tstruct wr_list\t *w_list;\n+\tunsigned long\t *wqe_head;\n+\n+\t/* serialize post to the work queue\n+\t */\n+\tspinlock_t\t\tlock;\n+\tint\t\t\twqe_cnt;\n+\tint\t\t\tds_cnt;\n+\tint\t\t\tmax_post;\n+\tint\t\t\tmax_gs;\n+\tint\t\t\toffset;\n+\tint\t\t\twqe_shift;\n+\tunsigned int\t\thead;\n+\tunsigned int\t\ttail;\n+\tu32\t\t\tcur_post;\n+\tu32\t\t\tlast_poll;\n+\tvoid\t\t *qend;\n+\tvoid\t\t*hdr_buf;\n+\tu32\t\t\thdr_size;\n+\tdma_addr_t\thdr_dma;\n+\tint\t\t\tmad_queue_depth;\n+\tint\t\t\tmad_index;\n+\tu32\t\t\t*wr_opcode;\n+\tu32\t\t\t*need_flush;\n+\tatomic_t\tflush_wqe_cnt;\n+\tint\t\t\t*prev_ce_wqe;\n+\tint\t\t\tlast_comp_ce_wqe;\n+\tint\t\t\tlast_post_ce_wqe;\n+};\n+\n+enum {\n+\tXSC_QP_USER,\n+\tXSC_QP_KERNEL,\n+};\n+\n+struct xsc_ib_qp {\n+\tstruct ib_qp\t\tibqp;\n+\tstruct xsc_core_qp\txqp;\n+\tstruct xsc_buf\t\tbuf;\n+\n+\tstruct xsc_ib_wq\trq;\n+\n+\tu32\t\t\tdoorbell_qpn;\n+\tu8\t\t\tsq_signal_bits;\n+\tstruct xsc_ib_wq\tsq;\n+\n+\tstruct ib_umem\t *umem;\n+\tint\t\t\tbuf_size;\n+\n+\t/* serialize qp state modifications\n+\t */\n+\tstruct mutex\t\tmutex;\n+\tu32\t\t\tflags;\n+\tu8\t\t\tport;\n+\tu8\t\t\talt_port;\n+\tu8\t\t\tatomic_rd_en;\n+\tu8\t\t\tresp_depth;\n+\tu8\t\t\tstate;\n+\tint\t\t\tmax_inline_data;\n+\tint\t\t\thas_rq;\n+\n+\tint\t\t\tcreate_type;\n+\tu32\t\t\tpa_lkey;\n+\t/* For QP1 */\n+\tstruct ib_ud_header\tqp1_hdr;\n+\tu32\t\t\tsend_psn;\n+\tstruct xsc_qp_context\tctx;\n+\tstruct ib_cq\t\t*send_cq;\n+\tstruct ib_cq\t\t*recv_cq;\n+\t/* For qp resources */\n+\tspinlock_t\t\tlock;\n+\n+\tstruct list_head\tcq_recv_list;\n+\tstruct list_head\tcq_send_list;\n+\tbool\t\tpost_send_invalid;\n+\tbool\t\tpost_recv_invalid;\n+\tstruct ib_recv_wr *srq_head_wr;\n+\tstruct ib_recv_wr *srq_tail_wr;\n+\tbool has_trig_cq_comp_handle;\n+};\n+\n+struct xsc_ib_cq_buf {\n+\tstruct xsc_buf\t\tbuf;\n+\tstruct ib_umem\t\t*umem;\n+\tint\t\t\tcqe_size;\n+};\n+\n+enum xsc_ib_qp_flags {\n+\tXSC_IB_QP_BLOCK_MULTICAST_LOOPBACK = 1 \u003c\u003c 0,\n+\tXSC_IB_QP_SIGNATURE_HANDLING = 1 \u003c\u003c 1,\n+};\n+\n+struct xsc_ib_cq {\n+\tstruct ib_cq\t\tibcq;\n+\tstruct xsc_core_cq\txcq;\n+\tstruct xsc_ib_cq_buf\tbuf;\n+\n+\t/* serialize access to the CQ\n+\t */\n+\tspinlock_t\t\tlock;\n+\n+\t/* protect resize cq\n+\t */\n+\tstruct mutex\t\tresize_mutex;\n+\tstruct xsc_ib_cq_resize *resize_buf;\n+\tstruct ib_umem\t *resize_umem;\n+\tint\t\t\tcqe_size;\n+\tstruct list_head\tsend_qp_list;\n+\tstruct list_head\trecv_qp_list;\n+\tstruct list_head\terr_state_qp_list;\n+\tu32\t\t\tcreate_flags;\n+\tint\t\t\tinvalid;\n+};\n+\n+enum xsc_srq_wr_per_qp {\n+\tXSC_SRQ_MIN_WR_NUM = 32,\n+\tXSC_SRQ_MAX_WR_NUM = 128,\n+};\n+\n+enum {\n+\tXSC_SRQ_USER,\n+\tXSC_SRQ_KERNEL,\n+};\n+\n+struct xsc_ib_srq {\n+\tstruct ib_srq\t\tibsrq;\n+\tstruct xsc_core_srq\txsrq;\n+\tu64\t\t *wrid;\n+\t/* protect SRQ hanlding\n+\t */\n+\tspinlock_t\t\tlock;\n+\tint\t\t\thead;\n+\tint\t\t\ttail;\n+\tstruct ib_umem\t *umem;\n+\t/* serialize arming a SRQ\n+\t */\n+\tstruct mutex\t\tmutex;\n+\tint\t\t\tqp_num;\n+\tint\t\t\tcreate_type;\n+};\n+\n+void xsc_restore_cache_srq_wrs(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp);\n+void xsc_free_one_cache_srq_wr(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp, u64 wr_id);\n+void xsc_free_all_cache_wrs(struct ib_recv_wr *wr);\n+void xsc_post_cache_wrs(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp, int wr_num, bool init);\n+\n+struct xsc_ib_mr {\n+\tstruct ib_mr\t\tibmr;\n+\tstruct xsc_core_mr\txmr;\n+\tstruct ib_umem\t *umem;\n+\tint\t\t\tnpages;\n+};\n+\n+#define MAX_OPCODE\tIB_WR_MASKED_ATOMIC_FETCH_AND_ADD\n+\n+struct xsc_ib_dev {\n+\tstruct ib_device\t\tib_dev;\n+\tstruct net_device\t*netdev;\n+\tstruct xsc_core_device *xdev;\n+\tint\t\t\t\tnum_ports;\n+\tint\t\t\t\tnum_comp_vectors;\n+\tu8\t\t\t\tib_active;\n+\tu32\t\t\t\tcrc_32_table[256];\n+\tint iommu_state;\n+\tstruct notifier_block nb;\n+\tu32\t\t\t\topcode_map[MAX_OPCODE + 1];\n+};\n+\n+union xsc_ib_fw_ver {\n+\tu64 data;\n+\tstruct {\n+\t\tu8\tver_major;\n+\t\tu8\tver_minor;\n+\t\tu16\tver_patch;\n+\t\tu32\tver_tweak;\n+\t} s;\n+};\n+\n+struct xsc_err_state_qp_node {\n+\tstruct list_head\tentry;\n+\tu32\t\t\t\t\tqp_id;\n+\tbool\t\t\t\tis_sq;\n+};\n+\n+struct xsc_ib_ah {\n+\tstruct ib_ah\t\tibah;\n+\tstruct xsc_av\t\tav;\n+};\n+\n+/*\n+ * Conversion helpers.\n+ *\n+ * The first group goes from a uverbs object to the private structure that\n+ * embeds it; the second goes from the core object to the private structure\n+ * that embeds that one. Both directions exist for cq, qp and srq, so the\n+ * names spell out which object is being converted.\n+ */\n+static inline struct xsc_ib_dev *to_xdev(struct ib_device *ibdev)\n+{\n+\treturn container_of(ibdev, struct xsc_ib_dev, ib_dev);\n+}\n+\n+static inline struct xsc_ib_pd *to_xpd(struct ib_pd *ibpd)\n+{\n+\treturn container_of(ibpd, struct xsc_ib_pd, ibpd);\n+}\n+\n+static inline struct xsc_ib_mr *to_xmr(struct ib_mr *ibmr)\n+{\n+\treturn container_of(ibmr, struct xsc_ib_mr, ibmr);\n+}\n+\n+static inline struct xsc_ib_ah *to_xah(struct ib_ah *ibah)\n+{\n+\treturn container_of(ibah, struct xsc_ib_ah, ibah);\n+}\n+\n+static inline struct xsc_ib_cq *to_xcq(struct ib_cq *ibcq)\n+{\n+\treturn container_of(ibcq, struct xsc_ib_cq, ibcq);\n+}\n+\n+static inline struct xsc_ib_qp *to_xqp(struct ib_qp *ibqp)\n+{\n+\treturn container_of(ibqp, struct xsc_ib_qp, ibqp);\n+}\n+\n+static inline struct xsc_ib_srq *to_xsrq(struct ib_srq *ibsrq)\n+{\n+\treturn container_of(ibsrq, struct xsc_ib_srq, ibsrq);\n+}\n+\n+static inline struct xsc_ib_cq *core_to_xcq(struct xsc_core_cq *xcq)\n+{\n+\treturn container_of(xcq, struct xsc_ib_cq, xcq);\n+}\n+\n+static inline struct xsc_ib_qp *core_to_xqp(struct xsc_core_qp *xqp)\n+{\n+\treturn container_of(xqp, struct xsc_ib_qp, xqp);\n+}\n+\n+static inline struct xsc_ib_srq *core_to_xsrq(struct xsc_core_srq *xsrq)\n+{\n+\treturn container_of(xsrq, struct xsc_ib_srq, xsrq);\n+}\n+\n+void build_opcode_map(struct xsc_ib_dev *dev);\n+\n+int xsc_ib_create_ah(struct ib_ah *ibah,\n+\t\t struct rdma_ah_init_attr *init_attr,\n+\t\t struct ib_udata *udata);\n+int xsc_ib_destroy_ah(struct ib_ah *ibah, u32 flags);\n+int xsc_ib_create_cq(struct ib_cq *ibcq,\n+\t\t const struct ib_cq_init_attr *attr,\n+\t\t struct uverbs_attr_bundle *attrs);\n+int xsc_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata);\n+int xsc_ib_destroy_qp(struct ib_qp *qp, struct ib_udata *udata);\n+int xsc_ib_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);\n+int xsc_ib_alloc_ucontext(struct ib_ucontext *uctx, struct ib_udata *udata);\n+void xsc_ib_dealloc_ucontext(struct ib_ucontext *ibcontext);\n+int xsc_ib_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata);\n+int xsc_ib_dealloc_pd(struct ib_pd *pd, struct ib_udata *udata);\n+\n+int xsc_ib_create_srq(struct ib_srq *ib_srq,\n+\t\t struct ib_srq_init_attr *init_attr,\n+\t\t struct ib_udata *udata);\n+int xsc_ib_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr,\n+\t\t enum ib_srq_attr_mask attr_mask, struct ib_udata *udata);\n+int xsc_ib_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *srq_attr);\n+int xsc_ib_post_srq_recv(struct ib_srq *ibsrq, const struct ib_recv_wr *wr,\n+\t\t\t const struct ib_recv_wr **bad_wr);\n+int xsc_ib_destroy_srq(struct ib_srq *srq, struct ib_udata *udata);\n+\n+int xsc_ib_query_port(struct ib_device *ibdev, u32 port,\n+\t\t struct ib_port_attr *props);\n+\n+int xsc_ib_create_qp(struct ib_qp *ibqp,\n+\t\t struct ib_qp_init_attr *init_attr,\n+\t\t struct ib_udata *udata);\n+\n+void __xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 qpn);\n+void xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 qpn);\n+\n+int xsc_ib_query_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr);\n+int xsc_ib_err_state_qp(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp,\n+\t\t\tenum xsc_qp_state cur_state, enum xsc_qp_state state);\n+int xsc_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,\n+\t\t int attr_mask, struct ib_udata *udata);\n+int xsc_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr_mask,\n+\t\t struct ib_qp_init_attr *qp_init_attr);\n+\n+int xsc_ib_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr,\n+\t\t const struct ib_send_wr **bad_wr);\n+int xsc_ib_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr,\n+\t\t const struct ib_recv_wr **bad_wr);\n+\n+void *xsc_get_send_wqe(struct xsc_ib_qp *qp, int n);\n+int xsc_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc);\n+int xsc_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags);\n+struct ib_mr *xsc_ib_get_dma_mr(struct ib_pd *pd, int acc);\n+struct ib_mr *xsc_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,\n+\t\t\t\t u64 virt_addr, int access_flags,\n+\t\t\t\t struct ib_dmah *dmah,\n+\t\t\t\t struct ib_udata *udata);\n+int xsc_ib_get_buf_offset(u64 addr, int page_shift, u32 *offset);\n+void xsc_ib_cont_pages(struct ib_umem *umem, u64 addr, int *count, int *shift,\n+\t\t int *ncont);\n+void xsc_ib_populate_pas(struct xsc_ib_dev *dev, struct ib_umem *umem,\n+\t\t\t int page_shift, __be64 *pas, int npages, bool need_to_devide);\n+struct ib_mr *xsc_ib_reg_user_mr_dmabuf(struct ib_pd *pd, u64 offset,\n+\t\t\t\t\tu64 length, u64 virt_addr,\n+\t\t\t\t\tint fd, int access_flags,\n+\t\t\t\t\tstruct ib_dmah *dmah,\n+\t\t\t\t\tstruct uverbs_attr_bundle *attrs);\n+\n+void xsc_ib_drain_rq(struct ib_qp *qp);\n+void xsc_ib_drain_sq(struct ib_qp *qp);\n+\n+static inline u8 convert_access(int acc)\n+{\n+\treturn (acc \u0026 IB_ACCESS_REMOTE_ATOMIC ? XSC_PERM_ATOMIC : 0) |\n+\t (acc \u0026 IB_ACCESS_REMOTE_WRITE ? XSC_PERM_REMOTE_WRITE : 0) |\n+\t (acc \u0026 IB_ACCESS_REMOTE_READ ? XSC_PERM_REMOTE_READ : 0) |\n+\t (acc \u0026 IB_ACCESS_LOCAL_WRITE ? XSC_PERM_LOCAL_WRITE : 0) |\n+\t XSC_PERM_LOCAL_READ;\n+}\n+\n+#define IB_MTU_8192\t6\n+\n+static inline enum ib_mtu xsc_net_to_ib_mtu(unsigned int mtu)\n+{\n+\tmtu = mtu - (IB_GRH_BYTES + IB_UDP_BYTES + IB_BTH_BYTES +\n+\t\t IB_EXT_XRC_BYTES + IB_EXT_ATOMICETH_BYTES +\n+\t\t IB_ICRC_BYTES);\n+\tif (mtu \u003e= 8192)\n+\t\treturn IB_MTU_8192;\n+\telse if (mtu \u003e= ib_mtu_enum_to_int(IB_MTU_4096))\n+\t\treturn IB_MTU_4096;\n+\telse if (mtu \u003e= ib_mtu_enum_to_int(IB_MTU_1024))\n+\t\treturn IB_MTU_1024;\n+\telse\n+\t\treturn 0;\n+}\n+\n+#define XSC_IB_IOMMU_MAP_DISABLE 0\n+#define XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN 1\n+#define XSC_IB_IOMMU_MAP_NORMAL 2\n+\n+static inline int xsc_ib_iommu_dma_map(struct ib_device *ibdev)\n+{\n+\treturn to_xdev(ibdev)-\u003eiommu_state;\n+}\n+\n+static inline void *xsc_ib_iova_to_virt(struct ib_device *ibdev, dma_addr_t iova)\n+{\n+\tphys_addr_t phyaddr;\n+\tstruct iommu_domain *domain;\n+\n+\tdomain = iommu_get_domain_for_dev(ibdev-\u003edma_device);\n+\tif (likely(domain)) {\n+\t\tphyaddr = iommu_iova_to_phys(domain, iova);\n+\t\tphyaddr |= iova \u0026 (PAGE_SIZE - 1);\n+\t} else {\n+\t\tphyaddr = dma_to_phys(ibdev-\u003edma_device, iova);\n+\t}\n+\n+\treturn phys_to_virt(phyaddr);\n+}\n+\n+struct ib_mad_list_head {\n+\tstruct list_head list;\n+\tstruct ib_cqe cqe;\n+\tstruct ib_mad_queue *mad_queue;\n+};\n+\n+#define IB_MAD_SEND_REQ_MAX_SG 2\n+struct ib_mad_send_wr_private {\n+\tstruct ib_mad_list_head mad_list;\n+\tstruct list_head agent_list;\n+\tstruct ib_mad_agent_private *mad_agent_priv;\n+\tstruct ib_mad_send_buf send_buf;\n+\tu64 header_mapping;\n+\tu64 payload_mapping;\n+\tstruct ib_ud_wr send_wr;\n+\tstruct ib_sge sg_list[IB_MAD_SEND_REQ_MAX_SG];\n+\t__be64 tid;\n+\tunsigned long timeout;\n+\tint max_retries;\n+\tint retries_left;\n+\tint retry;\n+\tint refcount;\n+\tenum ib_wc_status status;\n+\n+\t/* RMPP control */\n+\tstruct list_head rmpp_list;\n+\tstruct ib_rmpp_segment *last_ack_seg;\n+\tstruct ib_rmpp_segment *cur_seg;\n+\tint last_ack;\n+\tint seg_num;\n+\tint newwin;\n+\tint pad;\n+};\n+\n+struct ib_mad_private_header {\n+\tstruct ib_mad_list_head mad_list;\n+\tstruct ib_mad_recv_wc recv_wc;\n+\tstruct ib_wc wc;\n+\tu64 mapping;\n+} __packed;\n+\n+struct ib_mad_private {\n+\tstruct ib_mad_private_header header;\n+\tsize_t mad_size;\n+\tstruct ib_grh grh;\n+\tDECLARE_FLEX_ARRAY(u8, mad);\n+} __packed;\n+\n+static inline void *xsc_ib_send_mad_sg_virt_addr(struct ib_device *ibdev,\n+\t\t\t\t\t\t const struct ib_send_wr *wr,\n+\t\t\t\t\t\t int sg)\n+{\n+\tstruct ib_mad_send_wr_private *mad_send_wr;\n+\tstruct ib_mad_list_head *mad_list;\n+\tint iommu_state = xsc_ib_iommu_dma_map(ibdev);\n+\n+\t/* direct dma mapping */\n+\tif (!iommu_state)\n+\t\treturn phys_to_virt(dma_to_phys(ibdev-\u003edma_device, wr-\u003esg_list[sg].addr));\n+\n+\tif (iommu_state == XSC_IB_IOMMU_MAP_NORMAL)\n+\t\treturn xsc_ib_iova_to_virt(ibdev, wr-\u003esg_list[sg].addr);\n+\n+\tmad_list = container_of(wr-\u003ewr_cqe, struct ib_mad_list_head, cqe);\n+\tmad_send_wr = container_of(mad_list, struct ib_mad_send_wr_private,\n+\t\t\t\t mad_list);\n+\n+\t/* sg_list[0] */\n+\tif (sg == 0)\n+\t\treturn mad_send_wr-\u003esend_buf.mad;\n+\n+\t/* sg_list[1] */\n+\tif (mad_send_wr-\u003esend_buf.seg_count)\n+\t\treturn ib_get_rmpp_segment(\u0026mad_send_wr-\u003esend_buf,\n+\t\t\t\t\t mad_send_wr-\u003eseg_num);\n+\treturn mad_send_wr-\u003esend_buf.mad + mad_send_wr-\u003esend_buf.hdr_len;\n+}\n+\n+static inline void *xsc_ib_recv_mad_sg_virt_addr(struct ib_device *ibdev,\n+\t\t\t\t\t\t struct ib_wc *wc,\n+\t\t\t\t\t\t u64 sg_addr)\n+{\n+\tstruct ib_mad_private_header *mad_priv_hdr;\n+\tstruct ib_mad_private *recv;\n+\tstruct ib_mad_list_head *mad_list;\n+\tint iommu_state = xsc_ib_iommu_dma_map(ibdev);\n+\n+\t/* direct dma mapping */\n+\tif (!iommu_state)\n+\t\treturn phys_to_virt(dma_to_phys(ibdev-\u003edma_device, sg_addr));\n+\n+\tif (iommu_state == XSC_IB_IOMMU_MAP_NORMAL)\n+\t\treturn xsc_ib_iova_to_virt(ibdev, sg_addr);\n+\n+\tmad_list = container_of(wc-\u003ewr_cqe, struct ib_mad_list_head, cqe);\n+\tmad_priv_hdr = container_of(mad_list, struct ib_mad_private_header, mad_list);\n+\trecv = container_of(mad_priv_hdr, struct ib_mad_private, header);\n+\treturn \u0026recv-\u003egrh;\n+}\n+\n+struct xsc_ib_burst_info {\n+\t__u32 max_burst_sz;\n+\t__u16 typical_pkt_sz;\n+\t__u16 reserved;\n+};\n+\n+struct xsc_ib_modify_qp {\n+\t__u32\t\treserved;\n+\tstruct xsc_ib_burst_info burst_info;\n+\t__u32\t\tprofile;\n+};\n+\n+#endif /* XSC_IB_H */\ndiff --git a/drivers/infiniband/hw/xsc/xsc_ib_sysfs.c b/drivers/infiniband/hw/xsc/xsc_ib_sysfs.c\nnew file mode 100644\nindex 0000000000000..a0b577bdef258\n--- /dev/null\n+++ b/drivers/infiniband/hw/xsc/xsc_ib_sysfs.c\n@@ -0,0 +1,64 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/sysfs.h\u003e\n+#include \u003clinux/types.h\u003e\n+#include \u003clinux/time.h\u003e\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_hsi.h\"\n+#include \"common/driver.h\"\n+#include \"common/xsc_cmd.h\"\n+#include \"xsc_ib.h\"\n+\n+static ssize_t hca_type_show(struct device *device, struct device_attribute *attr, char *buf)\n+{\n+\tstruct ib_device *ib_dev = container_of(device, struct ib_device, dev);\n+\tstruct xsc_core_device *dev = to_xdev(ib_dev)-\u003exdev;\n+\tstruct pci_dev *pdev = dev-\u003epdev;\n+\n+\treturn sprintf(buf, \"%x\\n\", pdev-\u003esubsystem_device);\n+}\n+\n+static DEVICE_ATTR_RO(hca_type);\n+\n+static ssize_t hw_rev_show(struct device *device, struct device_attribute *attr, char *buf)\n+{\n+\tstruct ib_device *ib_dev = container_of(device, struct ib_device, dev);\n+\tstruct xsc_core_device *dev = to_xdev(ib_dev)-\u003exdev;\n+\tu32 hw_ver = 0;\n+\n+\thw_ver = ((dev-\u003echip_ver_l \u0026 0xffff) \u003c\u003c 16) |\n+\t\t(dev-\u003ehotfix_num \u0026 0xffff);\n+\treturn sprintf(buf, \"0x%x\\n\", hw_ver);\n+}\n+\n+static DEVICE_ATTR_RO(hw_rev);\n+\n+static struct device_attribute *xsc_ib_attributes[] = {\n+\t\u0026dev_attr_hca_type,\n+\t\u0026dev_attr_hw_rev,\n+};\n+\n+void xsc_ib_sysfs_init(struct ib_device *ib_dev, struct xsc_core_device *xdev)\n+{\n+\tint err = 0;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c ARRAY_SIZE(xsc_ib_attributes); i++) {\n+\t\terr = device_create_file(\u0026ib_dev-\u003edev, xsc_ib_attributes[i]);\n+\t\tif (err)\n+\t\t\tdev_err(\u0026xdev-\u003epdev-\u003edev, \"Create sysfs file for %s failed.\\n\",\n+\t\t\t\txsc_ib_attributes[i]-\u003eattr.name);\n+\t}\n+}\n+\n+void xsc_ib_sysfs_fini(struct ib_device *ib_dev, struct xsc_core_device *xdev)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i \u003c ARRAY_SIZE(xsc_ib_attributes); i++)\n+\t\tdevice_remove_file(\u0026ib_dev-\u003edev, xsc_ib_attributes[i]);\n+}\ndiff --git a/drivers/net/ethernet/Kconfig b/drivers/net/ethernet/Kconfig\nindex 8581ccba1505e..3f357b7739a60 100644\n--- a/drivers/net/ethernet/Kconfig\n+++ b/drivers/net/ethernet/Kconfig\n@@ -70,6 +70,7 @@ source \"drivers/net/ethernet/i825xx/Kconfig\"\n source \"drivers/net/ethernet/ibm/Kconfig\"\n source \"drivers/net/ethernet/intel/Kconfig\"\n source \"drivers/net/ethernet/xscale/Kconfig\"\n+source \"drivers/net/ethernet/yunsilicon/Kconfig\"\n \n config JME\n \ttristate \"JMicron(R) PCI-Express Gigabit Ethernet support\"\ndiff --git a/drivers/net/ethernet/Makefile b/drivers/net/ethernet/Makefile\nindex 2b1153d35b52b..4ad89b36b7612 100644\n--- a/drivers/net/ethernet/Makefile\n+++ b/drivers/net/ethernet/Makefile\n@@ -50,6 +50,7 @@ obj-$(CONFIG_NET_VENDOR_INTEL) += intel/\n obj-$(CONFIG_NET_VENDOR_I825XX) += i825xx/\n obj-$(CONFIG_NET_VENDOR_MICROSOFT) += microsoft/\n obj-$(CONFIG_NET_VENDOR_XSCALE) += xscale/\n+obj-$(CONFIG_NET_VENDOR_YUNSILICON) += yunsilicon/\n obj-$(CONFIG_JME) += jme.o\n obj-$(CONFIG_KORINA) += korina.o\n obj-$(CONFIG_LANTIQ_ETOP) += lantiq_etop.o\ndiff --git a/drivers/net/ethernet/yunsilicon/Kconfig b/drivers/net/ethernet/yunsilicon/Kconfig\nnew file mode 100644\nindex 0000000000000..46af1fdd01a25\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/Kconfig\n@@ -0,0 +1,26 @@\n+# SPDX-License-Identifier: GPL-2.0\n+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+# All rights reserved.\n+# Yunsilicon driver configuration\n+#\n+\n+config NET_VENDOR_YUNSILICON\n+\tbool \"Yunsilicon devices\"\n+\tdefault y\n+\tdepends on PCI\n+\tdepends on 64BIT || COMPILE_TEST\n+\thelp\n+\t If you have a network (Ethernet) device belonging to this class,\n+\t say Y.\n+\n+\t Note that the answer to this question doesn't directly affect the\n+\t kernel: saying N will just cause the configurator to skip all\n+\t the questions about Yunsilicon cards. If you say Y, you will be asked\n+\t for your specific card in the following questions.\n+\n+if NET_VENDOR_YUNSILICON\n+\n+source \"drivers/net/ethernet/yunsilicon/xsc/net/Kconfig\"\n+source \"drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig\"\n+\n+endif # NET_VENDOR_YUNSILICON\ndiff --git a/drivers/net/ethernet/yunsilicon/Makefile b/drivers/net/ethernet/yunsilicon/Makefile\nnew file mode 100644\nindex 0000000000000..a89a5ce24474e\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/Makefile\n@@ -0,0 +1,8 @@\n+# SPDX-License-Identifier: GPL-2.0\n+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+# All rights reserved.\n+# Makefile for the Yunsilicon device drivers.\n+#\n+\n+obj-$(CONFIG_YUNSILICON_XSC_ETH) += xsc/net/\n+obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc/pci/\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/common/cq.h b/drivers/net/ethernet/yunsilicon/xsc/common/cq.h\nnew file mode 100644\nindex 0000000000000..5217000c54a7c\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/common/cq.h\n@@ -0,0 +1,30 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef XSC_CORE_CQ_H\n+#define XSC_CORE_CQ_H\n+\n+#include \"common/xsc_core.h\"\n+\n+enum {\n+\tXSC_CQE_OWNER_MASK\t= 1,\n+};\n+\n+enum {\n+\tCQE_SIZE_64 = 0,\n+\tCQE_SIZE_128 = 1,\n+};\n+\n+enum {\n+\tXSC_CQ_DB_REQ_NOT_SOL\t\t= 1,\n+\tXSC_CQ_DB_REQ_NOT\t\t= 0,\n+};\n+\n+void xsc_init_cq_table(struct xsc_core_device *xdev);\n+int xsc_core_create_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq,\n+\t\t struct xsc_create_cq_ex_mbox_in *in, int inlen);\n+int xsc_core_destroy_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq);\n+\n+#endif /* XSC_CORE_CQ_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/common/driver.h b/drivers/net/ethernet/yunsilicon/xsc/common/driver.h\nnew file mode 100644\nindex 0000000000000..b3e91c868e49c\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/common/driver.h\n@@ -0,0 +1,95 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef XSC_DRIVER_H\n+#define XSC_DRIVER_H\n+\n+#include \u003clinux/kernel.h\u003e\n+#include \u003clinux/completion.h\u003e\n+#include \u003clinux/pci.h\u003e\n+#include \u003clinux/spinlock_types.h\u003e\n+#include \u003clinux/semaphore.h\u003e\n+#include \u003clinux/vmalloc.h\u003e\n+#include \u003clinux/radix-tree.h\u003e\n+#include \u003crdma/ib_verbs.h\u003e\n+\n+#include \"common/xsc_device.h\"\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_cmd_api.h\"\n+#include \"common/xsc_cmd.h\"\n+#include \"common/xsc_hsi.h\"\n+#include \"common/srq.h\"\n+\n+struct rxe_bth {\n+\tu8\t\t\topcode;\n+\tu8\t\t\tflags;\n+\t__be16\t\t\tpkey;\n+\t__be32\t\t\tqpn;\n+\t__be32\t\t\tapsn;\n+};\n+\n+struct rxe_deth {\n+\t__be32\t\t\tqkey;\n+\t__be32\t\t\tsrc_qp;\n+};\n+\n+static inline void *xsc_vzalloc(unsigned long size)\n+{\n+\tvoid *rtn;\n+\n+\trtn = kzalloc(size, GFP_KERNEL | __GFP_NOWARN);\n+\tif (!rtn)\n+\t\trtn = vzalloc(size);\n+\treturn rtn;\n+}\n+\n+static inline void xsc_vfree(const void *addr)\n+{\n+\tif (addr \u0026\u0026 is_vmalloc_addr(addr))\n+\t\tvfree(addr);\n+\telse\n+\t\tkfree(addr);\n+}\n+\n+int xsc_buf_alloc(struct xsc_core_device *xdev, unsigned long size,\n+\t\t unsigned long max_direct, struct xsc_buf *buf);\n+void xsc_buf_free(struct xsc_core_device *xdev, struct xsc_buf *buf);\n+void xsc_fill_page_array(struct xsc_buf *buf, __be64 *pas, unsigned int npages);\n+\n+int xsc_core_create_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr);\n+int xsc_core_destroy_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr);\n+int xsc_core_register_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr,\n+\t\t\t struct xsc_register_mr_mbox_in *in, int inlen);\n+int xsc_core_dereg_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr);\n+void xsc_reg_local_dma_mr(struct xsc_core_device *xdev);\n+\n+int xsc_core_alloc_pd(struct xsc_core_device *xdev, u32 *pdn);\n+int xsc_core_dealloc_pd(struct xsc_core_device *xdev, u32 pdn);\n+\n+int xsc_core_create_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq);\n+int xsc_core_destroy_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq);\n+\n+int xsc_core_create_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq,\n+\t\t struct xsc_create_cq_ex_mbox_in *in, int inlen);\n+int xsc_core_destroy_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq);\n+\n+int xsc_core_create_qp(struct xsc_core_device *xdev,\n+\t\t struct xsc_core_qp *qp,\n+\t\t struct xsc_create_qp_mbox_in *in,\n+\t\t int inlen);\n+int xsc_core_destroy_qp(struct xsc_core_device *xdev,\n+\t\t\tstruct xsc_core_qp *qp);\n+int xsc_core_qp_modify(struct xsc_core_device *xdev, enum xsc_qp_state cur_state,\n+\t\t enum xsc_qp_state new_state,\n+\t\t struct xsc_modify_qp_mbox_in *in, int sqd_event,\n+\t\t struct xsc_core_qp *qp);\n+int xsc_core_qp_query(struct xsc_core_device *xdev, struct xsc_core_qp *qp,\n+\t\t struct xsc_query_qp_mbox_out *out, int outlen);\n+void xsc_qp_event(struct xsc_core_device *xdev, u32 qpn, int event_type);\n+\n+int xsc_vector2eqn(struct xsc_core_device *xdev, int vector, int *eqn,\n+\t\t unsigned int *irqn);\n+\n+#endif /* XSC_DRIVER_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/common/qp.h b/drivers/net/ethernet/yunsilicon/xsc/common/qp.h\nnew file mode 100644\nindex 0000000000000..4855c97e43c83\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/common/qp.h\n@@ -0,0 +1,104 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef XSC_QP_H\n+#define XSC_QP_H\n+\n+#include \u003clinux/radix-tree.h\u003e\n+\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_cmd.h\"\n+#include \"common/xsc_hsi.h\"\n+\n+enum {\n+\tXSC_QP_PM_MIGRATED\t\t= 0x3,\n+\tXSC_QP_PM_ARMED\t\t\t= 0x0,\n+\tXSC_QP_PM_REARM\t\t\t= 0x1\n+};\n+\n+enum {\n+\tXSC_WQE_CTRL_CQ_UPDATE\t\t= 2 \u003c\u003c 2,\n+\tXSC_WQE_CTRL_SOLICITED\t\t= 1 \u003c\u003c 1,\n+};\n+\n+struct xsc_av {\n+\tunion {\n+\t\tstruct {\n+\t\t\t__be32\tqkey;\n+\t\t\t__be32\treserved;\n+\t\t} qkey;\n+\t\t__be64\tdc_key;\n+\t} key;\n+\t__be32\tdqp_dct;\n+\tu8\tstat_rate_sl;\n+\tu8\tfl_mlid;\n+\tunion {\n+\t\t__be16\trlid;\n+\t\t__be16 udp_sport;\n+\t};\n+\tu8\treserved0[4];\n+\tu8\trmac[6];\n+\tu8\ttclass;\n+\tu8\thop_limit;\n+\t__be32\tgrh_gid_fl;\n+\tu8\trgid[16];\n+};\n+\n+struct xsc_qp_path {\n+\tu8\t\t\tfl;\n+\tu8\t\t\trsvd3;\n+\tu8\t\t\tfree_ar;\n+\tu8\t\t\tpkey_index;\n+\tu8\t\t\trsvd0;\n+\tu8\t\t\tgrh_mlid;\n+\t__be16\t\t\trlid;\n+\tu8\t\t\tackto_lt;\n+\tu8\t\t\tmgid_index;\n+\tu8\t\t\tstatic_rate;\n+\tu8\t\t\thop_limit;\n+\t__be32\t\t\ttclass_flowlabel;\n+\tu8\t\t\trgid[16];\n+\tu8\t\t\trsvd1[4];\n+\tu8\t\t\tsl;\n+\tu8\t\t\tport;\n+\tu8\t\t\trsvd2[6];\n+\tu8\t\t\tdmac[6];\n+\tu8\t\t\tsmac[6];\n+\t__be16\t\taf_type;\n+\t__be32\t\tsip[4];\n+\t__be32\t\tdip[4];\n+\t__be16\t\t\tsport;\n+\tu8\t\t\tecn_dscp;\n+\tu8\t\t\tvlan_valid;\n+\t__be16\t\t\tvlan_id;\n+\tu8\t\t\tdci_cfi_prio_sl; /* not left moved yet. */\n+};\n+\n+static inline struct xsc_core_qp *__xsc_qp_lookup(struct xsc_core_device *xdev, u32 qpn)\n+{\n+\treturn radix_tree_lookup(\u0026xdev-\u003edev_res-\u003eqp_table.tree, qpn);\n+}\n+\n+int xsc_core_create_resource_common(struct xsc_core_device *xdev,\n+\t\t\t\t struct xsc_core_qp *qp);\n+void xsc_core_destroy_resource_common(struct xsc_core_device *xdev,\n+\t\t\t\t struct xsc_core_qp *qp);\n+void xsc_init_qp_table(struct xsc_core_device *xdev);\n+void xsc_qp_event(struct xsc_core_device *xdev, u32 qpn, int event_type);\n+\n+int xsc_core_create_qp(struct xsc_core_device *xdev,\n+\t\t struct xsc_core_qp *qp,\n+\t\t struct xsc_create_qp_mbox_in *in,\n+\t\t int inlen);\n+int xsc_core_destroy_qp(struct xsc_core_device *xdev,\n+\t\t\tstruct xsc_core_qp *qp);\n+int xsc_core_qp_modify(struct xsc_core_device *xdev, enum xsc_qp_state cur_state,\n+\t\t enum xsc_qp_state new_state,\n+\t\t struct xsc_modify_qp_mbox_in *in, int sqd_event,\n+\t\t struct xsc_core_qp *qp);\n+int xsc_core_qp_query(struct xsc_core_device *xdev, struct xsc_core_qp *qp,\n+\t\t struct xsc_query_qp_mbox_out *out, int outlen);\n+\n+#endif /* XSC_QP_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/common/srq.h b/drivers/net/ethernet/yunsilicon/xsc/common/srq.h\nnew file mode 100644\nindex 0000000000000..0a2e660e102a5\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/common/srq.h\n@@ -0,0 +1,54 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef XSC_SRQ_H\n+#define XSC_SRQ_H\n+\n+#include \u003clinux/list.h\u003e\n+#include \u003clinux/spinlock.h\u003e\n+#include \u003clinux/types.h\u003e\n+\n+#include \"common/xsc_core.h\"\n+\n+struct ib_qp;\n+struct ib_recv_wr;\n+\n+#define XSC_MAX_SRQ_NUM 8192\n+#define XSC_MAX_SRQ_LOG_SIZE 13\n+\n+struct xsc_srq_qp_node {\n+\tstruct list_head\tentry;\n+\tu32\t\t\tqp_id;\n+\tstruct ib_qp\t\t*ibqp;\n+};\n+\n+struct xsc_srq_cache_wr {\n+\tint wr_num;\n+\tint max_wr_num;\n+\tstruct ib_recv_wr *head_wr;\n+\tstruct ib_recv_wr *tail_wr;\n+};\n+\n+struct xsc_core_srq {\n+\tu32 srqn;\n+\tint max;\n+\tsize_t max_gs;\n+\tsize_t max_avail_gather;\n+\tint wqe_shift;\n+\tvoid (*event)(struct xsc_core_srq *srq, enum xsc_event e);\n+\n+\t/* protect ctrl list */\n+\tspinlock_t\t\tlock;\n+\tstruct list_head\tctrl_list;\n+\tu16 uid;\n+\n+\tstruct list_head\tsrq_qp_list;\n+\tstruct xsc_srq_cache_wr *cache_wr;\n+};\n+\n+int xsc_core_create_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq);\n+int xsc_core_destroy_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq);\n+\n+#endif /* XSC_SRQ_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd.h\nnew file mode 100644\nindex 0000000000000..0b8f669b5de40\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd.h\n@@ -0,0 +1,1022 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_CMD_H\n+#define __XSC_CMD_H\n+\n+#include \u003clinux/if_ether.h\u003e\n+\n+#define XSC_CMDQ_VERSION\t0x32\n+\n+#define XSC_BOARD_SN_LEN\t32\n+#define DSCP_PCP_UNSET\t\t255\n+#define XSC_MAX_NUM_PCIE_INTF\t\t2\n+#define XSC_MAX_PF_NUM_PER_PCIE\t\t8\n+\n+/* hca_cap mbox version */\n+#define CMD_QUERY_HCA_CAP_V1\t1\n+#define CMD_QUERY_HCA_CAP_V2\t2\n+#define CMD_QUERY_HCA_CAP_V3\t3\n+#define CMD_QUERY_HCA_CAP_V4\t4\n+#define CMD_QUERY_HCA_CAP_V5\t5\n+#define CMD_QUERY_HCA_CAP_V6\t6\n+#define CMD_QUERY_HCA_CAP_V7\t7\n+#define CMD_QUERY_HCA_CAP_V8\t8\n+#define CMD_QUERY_HCA_CAP_V9\t9\n+\n+#define CMD_SET_CQ_CONTEXT_V1\t1\n+\n+enum {\n+\tXSC_CMDQ_GET_HW_STATS_RDMA_V0 = 0,\n+\tXSC_CMDQ_GET_HW_STATS_RDMA_V1 = 1,\n+\n+\tXSC_CMDQ_GET_HW_STATS_RDMA_UNKNOW,\n+};\n+\n+enum {\n+\tXSC_CMD_STAT_OK\t\t\t\t= 0x0,\n+\tXSC_CMD_STAT_INT_ERR\t\t\t= 0x1,\n+\tXSC_CMD_STAT_BAD_OP_ERR\t\t\t= 0x2,\n+\tXSC_CMD_STAT_BAD_PARAM_ERR\t\t= 0x3,\n+\tXSC_CMD_STAT_BAD_SYS_STATE_ERR\t\t= 0x4,\n+\tXSC_CMD_STAT_BAD_RES_ERR\t\t= 0x5,\n+\tXSC_CMD_STAT_RES_BUSY\t\t\t= 0x6,\n+\tXSC_CMD_STAT_LIM_ERR\t\t\t= 0x8,\n+\tXSC_CMD_STAT_BAD_RES_STATE_ERR\t\t= 0x9,\n+\tXSC_CMD_STAT_IX_ERR\t\t\t= 0xa,\n+\tXSC_CMD_STAT_NO_RES_ERR\t\t\t= 0xf,\n+\tXSC_CMD_STAT_BAD_QP_STATE_ERR\t\t= 0x10,\n+\tXSC_CMD_STAT_BAD_PKT_ERR\t\t= 0x30,\n+\tXSC_CMD_STAT_BAD_SIZE_OUTS_CQES_ERR\t= 0x40,\n+\tXSC_CMD_STAT_BAD_INP_LEN_ERR\t\t= 0x50,\n+\tXSC_CMD_STAT_BAD_OUTP_LEN_ERR\t\t= 0x51,\n+};\n+\n+/* status aliases used by the RDMA driver (same protocol values) */\n+#define XSC_CMD_STATUS_OK\t\t\tXSC_CMD_STAT_OK\n+#define XSC_CMD_STATUS_FAIL\t\t\tXSC_CMD_STAT_INT_ERR\n+#define XSC_CMD_STATUS_NOT_SUPPORTED\t\tXSC_CMD_STAT_BAD_OP_ERR\n+#define XSC_CMD_STATUS_BAD_PARAM\t\tXSC_CMD_STAT_BAD_PARAM_ERR\n+#define XSC_CMD_STATUS_INVAL_RES\t\tXSC_CMD_STAT_BAD_RES_ERR\n+#define XSC_CMD_STATUS_BUSY\t\t\tXSC_CMD_STAT_RES_BUSY\n+#define XSC_CMD_STATUS_INVAL_DATA\t\tXSC_CMD_STAT_LIM_ERR\n+#define XSC_CMD_STATUS_NOT_FOUND\t\tXSC_CMD_STAT_IX_ERR\n+#define XSC_CMD_STATUS_NO_RES\t\t\tXSC_CMD_STAT_NO_RES_ERR\n+#define XSC_CMD_STATUS_BAD_INBUF\t\tXSC_CMD_STAT_BAD_INP_LEN_ERR\n+#define XSC_CMD_STATUS_BAD_OUTBUF\t\tXSC_CMD_STAT_BAD_OUTP_LEN_ERR\n+\n+/* extended status codes */\n+#define XSC_CMD_STATUS_INVAL_FUNC\t\t0x41\n+#define XSC_CMD_STATUS_NO_MPT_RES\t\t0x42\n+#define XSC_CMD_STATUS_NO_MTT_RES\t\t0x43\n+#define XSC_CMD_STATUS_NO_EQN_RES\t\t0x44\n+#define XSC_CMD_STATUS_NO_EQ_PA_RES\t\t0x45\n+#define XSC_CMD_STATUS_NO_CQN_RES\t\t0x46\n+#define XSC_CMD_STATUS_NO_CQ_PA_RES\t\t0x47\n+#define XSC_CMD_STATUS_NO_QPN_RES\t\t0x48\n+#define XSC_CMD_STATUS_NO_QP_PA_RES\t\t0x49\n+#define XSC_CMD_STATUS_NO_PDN_RES\t\t0x4a\n+#define XSC_CMD_STATUS_QP_FLUSH_BUSY\t\t0x4b\n+#define XSC_CMD_STATUS_QP_FLUSH_PENDING\t\t0x4c\n+#define XSC_CMD_STATUS_INVAL_OPCODE\t\t0x52\n+\n+enum {\n+\tXSC_CMD_OP_QUERY_HCA_CAP\t\t= 0x100,\n+\tXSC_CMD_OP_QUERY_CMDQ_VERSION\t\t= 0x10a,\n+\tXSC_CMD_OP_FUNCTION_RESET\t\t= 0x10c,\n+\tXSC_CMD_OP_DUMMY\t\t\t= 0x10d,\n+\tXSC_CMD_OP_QUERY_GUID\t\t\t= 0x113,\n+\tXSC_CMD_OP_ACTIVATE_HW_CONFIG\t\t= 0x114,\n+\n+\tXSC_CMD_OP_QUERY_HW_STATS\t\t= 0x1200,\n+\tXSC_CMD_OP_QUERY_HW_STATS_RDMA\t\t= 0x1203,\n+\tXSC_CMD_OP_QUERY_HW_GLOBAL_STATS\t= 0x1210,\n+\n+\tXSC_CMD_OP_CREATE_MKEY\t\t\t= 0x200,\n+\tXSC_CMD_OP_DESTROY_MKEY\t\t\t= 0x202,\n+\tXSC_CMD_OP_REG_MR\t\t\t= 0x204,\n+\tXSC_CMD_OP_DEREG_MR\t\t\t= 0x205,\n+\tXSC_CMD_OP_SET_MPT\t\t\t= 0x206,\n+\tXSC_CMD_OP_SET_MTT\t\t\t= 0x207,\n+\n+\tXSC_CMD_OP_CREATE_EQ\t\t\t= 0x301,\n+\tXSC_CMD_OP_DESTROY_EQ\t\t\t= 0x302,\n+\n+\tXSC_CMD_OP_CREATE_CQ\t\t\t= 0x400,\n+\tXSC_CMD_OP_DESTROY_CQ\t\t\t= 0x401,\n+\tXSC_CMD_OP_SET_CQ_CONTEXT\t\t= 0x406,\n+\tXSC_CMD_OP_SET_CQ_BUF_PA\t\t= 0x407,\n+\tXSC_CMD_OP_CREATE_CQ_EX\t\t\t= 0x408,\n+\n+\tXSC_CMD_OP_CREATE_QP\t\t\t= 0x500,\n+\tXSC_CMD_OP_DESTROY_QP\t\t\t= 0x501,\n+\tXSC_CMD_OP_RST2INIT_QP\t\t\t= 0x502,\n+\tXSC_CMD_OP_INIT2RTR_QP\t\t\t= 0x503,\n+\tXSC_CMD_OP_RTR2RTS_QP\t\t\t= 0x504,\n+\tXSC_CMD_OP_RTS2RTS_QP\t\t\t= 0x505,\n+\tXSC_CMD_OP_SQERR2RTS_QP\t\t\t= 0x506,\n+\tXSC_CMD_OP_2ERR_QP\t\t\t= 0x507,\n+\tXSC_CMD_OP_RTS2SQD_QP\t\t\t= 0x508,\n+\tXSC_CMD_OP_SQD2RTS_QP\t\t\t= 0x509,\n+\tXSC_CMD_OP_2RST_QP\t\t\t= 0x50a,\n+\tXSC_CMD_OP_QUERY_QP\t\t\t= 0x50b,\n+\tXSC_CMD_OP_INIT2INIT_QP\t\t\t= 0x50e,\n+\tXSC_CMD_OP_SQD2SQD_QP\t\t\t= 0x511,\n+\tXSC_CMD_OP_CREATE_MULTI_QP\t\t= 0x515,\n+\n+\tXSC_CMD_OP_ALLOC_PD\t\t\t= 0x800,\n+\tXSC_CMD_OP_DEALLOC_PD\t\t\t= 0x801,\n+\n+\tXSC_CMD_OP_MODIFY_RAW_QP\t\t= 0x81f,\n+\n+\tXSC_CMD_OP_ENABLE_NIC_HCA\t\t= 0x810,\n+\tXSC_CMD_OP_DISABLE_NIC_HCA\t\t= 0x811,\n+\n+\tXSC_CMD_OP_QUERY_VPORT_STATE\t\t= 0x822,\n+\tXSC_CMD_OP_QUERY_LINK_INFO\t\t= 0x832,\n+\tXSC_CMD_OP_MODIFY_LINK_INFO\t\t= 0x834,\n+\tXSC_CMD_OP_QUERY_EVENT_TYPE\t\t= 0x831,\n+\n+\tXSC_CMD_OP_ENABLE_MSIX\t\t\t= 0x850,\n+\n+\tXSC_CMD_OP_SET_MTU\t\t\t= 0x1100,\n+\tXSC_CMD_OP_QUERY_ETH_MAC\t\t= 0X1101,\n+\n+\tXSC_CMD_OP_SET_PORT_ADMIN_STATUS\t= 0x1801,\n+\n+\tXSC_CMD_OP_MAX\n+};\n+\n+enum xsc_dma_direct {\n+\tXSC_DMA_DIR_TO_MAC,\n+\tXSC_DMA_DIR_READ,\n+\tXSC_DMA_DIR_WRITE,\n+\tXSC_DMA_DIR_LOOPBACK,\n+\tXSC_DMA_DIR_MAX\n+};\n+\n+/* hw feature bitmap, 32bit */\n+enum xsc_hw_feature_flag {\n+\tXSC_HW_RDMA_SUPPORT\t\t\t= BIT(0),\n+\tXSC_HW_PFC_PRIO_STATISTIC_SUPPORT\t= BIT(1),\n+\tXSC_HW_PFC_STALL_STATS_SUPPORT\t\t= BIT(3),\n+\tXSC_HW_RDMA_CM_SUPPORT\t\t\t= BIT(5),\n+\tXSC_HW_MTU_8192_SUPPORT\t\t\t= BIT(11),\n+\tXSC_HW_RDMA_ATOMIC_SUPPORT\t\t= BIT(17),\n+\tXSC_HW_MULTIDOORBELL_SUPPORT\t\t= BIT(18),\n+\tXSC_HW_CQE64_SUPPORT\t\t\t= BIT(21),\n+\n+\tXSC_HW_LAST_FEATURE\t\t\t= BIT(31)\n+};\n+\n+struct xsc_inbox_hdr {\n+\t__be16\t\topcode;\n+\tu8\t\trsvd[4];\n+\t__be16\t\tver; /* cmd version */\n+};\n+\n+struct xsc_outbox_hdr {\n+\tu8\t\tstatus;\n+\tu8\t\trsvd[5];\n+\t__be16\t\tver;\n+};\n+\n+/*CQ mbox*/\n+struct xsc_cq_context {\n+\t__be16\t\teqn; /* event queue number */\n+\t__be16\t\tpa_num; /* physical address count in the ctx */\n+\t__be16\t\tglb_func_id;\n+\tu8\t\tlog_cq_sz;\n+\tu8\t\tcq_type;\n+};\n+\n+struct xsc_cq_context_ex {\n+\tstruct xsc_cq_context ctx;\n+\tu8\t\tpage_shift;\n+\tu8\t\toverrun_ignore;\n+\tu8\t\trsvd[2];\n+\tu32\t\tpa_num_ex;\n+};\n+\n+struct xsc_create_cq_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tstruct xsc_cq_context\tctx;\n+\t__be64\t\t\tpas[]; /* physical address list */\n+};\n+\n+struct xsc_create_cq_ex_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tstruct xsc_cq_context_ex\tctx_ex;\n+\t__be64\t\t\tpas[];\n+};\n+\n+struct xsc_create_cq_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\t__be32\t\t\tcqn; /* completion queue number */\n+\tu8\t\t\trsvd[4];\n+};\n+\n+struct xsc_destroy_cq_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be32\t\t\tcqn;\n+\tu8\t\t\trsvd[4];\n+};\n+\n+struct xsc_destroy_cq_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+struct xsc_set_cq_context_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tstruct xsc_cq_context_ex\tctx_ex;\n+};\n+\n+struct xsc_set_cq_context_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\t__be32\t\t\tcqn;\n+\t__be32\t\t\tcq_pa_list_base;\n+};\n+\n+struct xsc_set_cq_buf_pa_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be32\t\t\tpa_list_start;\n+\t__be32\t\t\tpa_num;\n+\t__be64\t\t\tpas[];\n+};\n+\n+struct xsc_set_cq_buf_pa_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+/* MKey / MPT / MTT mbox */\n+struct xsc_create_mkey_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tu8\t\t\trsvd[4];\n+};\n+\n+struct xsc_create_mkey_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\t__be32\t\t\tmkey;\n+};\n+\n+struct xsc_destroy_mkey_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be32\t\t\tmkey;\n+};\n+\n+struct xsc_destroy_mkey_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd;\n+};\n+\n+struct xsc_mpt_item {\n+\t__be32\t\tpdn;\n+\t__be32\t\tpa_num;\n+\t__be32\t\tlen;\n+\t__be32\t\tmkey;\n+\t__be32\t\tlen_h;\n+\tu8\t\tis_gpu;\n+\tu8\t\tacc;\n+\tu8\t\tpage_mode;\n+\tu8\t\tmap_en;\n+\t__be64\t\tva_base;\n+};\n+\n+struct xsc_set_mpt_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tstruct xsc_mpt_item\tmpt_item;\n+};\n+\n+struct xsc_set_mpt_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\t__be32\t\t\tmtt_base;\n+\tu8\t\t\trsvd[4];\n+};\n+\n+struct xsc_mtt_setting {\n+\t__be32\t\tmtt_base;\n+\t__be32\t\tpa_num;\n+\t__be64\t\tpas[];\n+};\n+\n+struct xsc_set_mtt_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tstruct xsc_mtt_setting\tmtt_setting;\n+};\n+\n+struct xsc_set_mtt_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+/*QP mbox*/\n+enum {\n+\tMODULE_SPEED_UNKNOWN,\n+\tMODULE_SPEED_10G,\n+\tMODULE_SPEED_25G,\n+\tMODULE_SPEED_40G_R4,\n+\tMODULE_SPEED_50G_R,\n+\tMODULE_SPEED_50G_R2,\n+\tMODULE_SPEED_100G_R2,\n+\tMODULE_SPEED_100G_R4,\n+\tMODULE_SPEED_200G_R4,\n+\tMODULE_SPEED_200G_R8,\n+\tMODULE_SPEED_400G_R8,\n+\tMODULE_SPEED_400G_R4,\n+};\n+\n+struct xsc_create_qp_request {\n+\t__be16\t\tinput_qpn;\n+\t__be16\t\tpa_num;\n+\tu8\t\tqp_type;\n+\tu8\t\tlog_sq_sz;\n+\tu8\t\tlog_rq_sz;\n+\tu8\t\tdma_direct;\n+\t__be32\t\tpdn; /* protect domain number */\n+\t__be16\t\tcqn_send;\n+\t__be16\t\tcqn_recv;\n+\t__be16\t\tglb_funcid;\n+\t/*rsvd, the old logic_port */\n+\tu8\t\tpage_shift;\n+\tu8\t\trsvd;\n+\t__be64\t\tpas[];\n+};\n+\n+struct xsc_create_qp_mbox_in {\n+\tstruct xsc_inbox_hdr\t\thdr;\n+\tstruct xsc_create_qp_request\treq;\n+};\n+\n+struct xsc_create_qp_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\t__be32\t\t\tqpn; /* queue pair number */\n+\tu8\t\t\trsvd[4];\n+};\n+\n+struct xsc_destroy_qp_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be32\t\t\tqpn;\n+\tu8\t\t\trsvd[4];\n+};\n+\n+struct xsc_destroy_qp_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+/* MR mbox */\n+struct xsc_register_mr_request {\n+\t__be32\t\tpdn;\n+\t__be32\t\tpa_num;\n+\t__be64\t\tlen;\n+\t__be32\t\tmkey;\n+\tu8\t\tis_gpu;\n+\tu8\t\tacc;\n+\tu8\t\tpage_mode;\n+\tu8\t\tmap_en;\n+\t__be64\t\tva_base;\n+\t__be64\t\tpas[];\n+};\n+\n+struct xsc_register_mr_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tstruct xsc_register_mr_request\treq;\n+};\n+\n+struct xsc_unregister_mr_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be32\t\t\tmkey;\n+\tu8\t\t\trsvd[4];\n+};\n+\n+struct xsc_unregister_mr_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+enum qp_access_flag {\n+\tQP_ACCESS_REMOTE_READ = (1 \u003c\u003c 0),\n+\tQP_ACCESS_REMOTE_WRITE = (1 \u003c\u003c 1),\n+\tQP_ACCESS_REMOTE_ATOMIC = (1 \u003c\u003c 2),\n+\tQP_ACCESS_MODIFY_QP_TYPE = (1 \u003c\u003c 3),\n+};\n+\n+#define\tXSC_QP_CONTEXT_V1\t1\n+#define XSC_QP_CONTEXT_V2\t2\n+\n+struct xsc_qp_context {\n+\t__be32\t\tremote_qpn;\n+\t__be32\t\tcqn_send;\n+\t__be32\t\tcqn_recv;\n+\t__be32\t\tnext_send_psn;\n+\t__be32\t\tnext_recv_psn;\n+\t__be32\t\tpdn;\n+\t__be16\t\tsrc_udp_port;\n+\t__be16\t\tpath_id;\n+\tu8\t\tmtu_mode;\n+\tu8\t\tlag_sel;\n+\tu8\t\tlag_sel_en;\n+\tu8\t\tretry_cnt;\n+\tu8\t\trnr_retry;\n+\tu8\t\tdscp;\n+\tu8\t\tstate;\n+\tu8\t\thop_limit;\n+\tu8\t\tdmac[6];\n+\tu8\t\tsmac[6];\n+\t__be32\t\tdip[4];\n+\t__be32\t\tsip[4];\n+\t__be16\t\tip_type;\n+\t__be16\t\tgrp_id;\n+\tu8\t\tvlan_valid;\n+\tu8\t\tdci_cfi_prio_sl;\n+\t__be16\t\tvlan_id;\n+\tu8\t\tqp_out_port;\n+\tu8\t\tpcie_no;\n+\t__be16\t\tlag_id;\n+\t__be16\t\tfunc_id;\n+\tu8\t\tmax_sra;\n+\tu8\t\tmax_rra;\n+\tu8\t\tno_need_wait;\n+\tu8\t\tswith_qp_type;\n+\tu8\t\trsvd0[2];\n+\t__be32\t\tqp_access_flags;\n+};\n+\n+struct xsc_modify_qp_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be32\t\t\tqpn;\n+\tstruct xsc_qp_context\tctx;\n+\tu8\t\t\tno_need_wait;\n+};\n+\n+struct xsc_modify_qp_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+struct xsc_query_qp_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be32\t\t\tqpn;\n+\tu8\t\t\trsvd[4];\n+};\n+\n+struct xsc_query_qp_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tstruct xsc_qp_context\tctx;\n+};\n+\n+struct xsc_create_multiqp_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be16\t\t\tqp_num;\n+\tu8\t\t\tqp_type;\n+\tu8\t\t\trsvd;\n+\t__be32\t\t\treq_len;\n+\tu8\t\t\tdata[];\n+};\n+\n+struct xsc_create_multiqp_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\t__be32\t\t\tqpn_base;\n+};\n+\n+/* PD mbox */\n+struct xsc_alloc_pd_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+struct xsc_alloc_pd_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\t__be32\t\t\tpdn;\n+\tu8\t\t\trsvd[4];\n+};\n+\n+struct xsc_dealloc_pd_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be32\t\t\tpdn;\n+\tu8\t\t\trsvd[4];\n+};\n+\n+struct xsc_dealloc_pd_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+/* MSIX TABLE mbox */\n+struct xsc_msix_table_info_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be16\t\t\tindex;\n+\tu8\t\t\trsvd[6];\n+};\n+\n+struct xsc_msix_table_info_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\t__be32\t\t\taddr_lo;\n+\t__be32\t\t\taddr_hi;\n+\t__be32\t\t\tdata;\n+};\n+\n+/*EQ mbox*/\n+struct xsc_eq_context {\n+\t__be16\t\t\tvecidx;\n+\t__be16\t\t\tpa_num;\n+\tu8\t\t\tlog_eq_sz;\n+\t__be16\t\t\tglb_func_id;\n+\tu8\t\t\tis_async_eq;\n+\tu8\t\t\trsvd;\n+};\n+\n+struct xsc_create_eq_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tstruct xsc_eq_context\tctx;\n+\t__be64\t\t\tpas[];\n+};\n+\n+struct xsc_create_eq_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\t__be32\t\t\teqn;\n+\tu8\t\t\trsvd[4];\n+};\n+\n+struct xsc_destroy_eq_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be32\t\t\teqn;\n+\tu8\t\t\trsvd[4];\n+};\n+\n+struct xsc_destroy_eq_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+struct xsc_cmd_query_cmdq_ver_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+struct xsc_cmd_query_cmdq_ver_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\t__be16\t\t\tcmdq_ver;\n+\tu8\t\t\trsvd[6];\n+};\n+\n+struct xsc_fw_version {\n+\tu8\t\tfw_version_major;\n+\tu8\t\tfw_version_minor;\n+\t__be16\t\tfw_version_patch;\n+\t__be32\t\tfw_version_tweak;\n+\tu8\t\tfw_version_extra_flag;\n+\tu8\t\trsvd[7];\n+};\n+\n+struct xsc_hca_cap {\n+\tu8\t\trsvd1[12];\n+\tu8\t\tsend_seg_num;\n+\tu8\t\tsend_wqe_shift;\n+\tu8\t\trecv_seg_num;\n+\tu8\t\trecv_wqe_shift;\n+\tu8\t\tlog_max_srq_sz;\n+\tu8\t\tlog_max_qp_sz;\n+\tu8\t\tlog_max_mtt;\n+\tu8\t\tlog_max_qp;\n+\tu8\t\tlog_max_strq_sz;\n+\tu8\t\tlog_max_srqs;\n+\tu8\t\trsvd4[2];\n+\tu8\t\tlog_max_tso;\n+\tu8\t\tlog_max_cq_sz;\n+\tu8\t\trsvd6;\n+\tu8\t\tlog_max_cq;\n+\tu8\t\tlog_max_eq_sz;\n+\tu8\t\tlog_max_mkey;\n+\tu8\t\tlog_max_msix;\n+\tu8\t\tlog_max_eq;\n+\tu8\t\tmax_indirection;\n+\tu8\t\tlog_max_mrw_sz;\n+\tu8\t\tlog_max_bsf_list_sz;\n+\tu8\t\tlog_max_klm_list_sz;\n+\tu8\t\trsvd_8_0;\n+\tu8\t\tlog_max_ra_req_dc;\n+\tu8\t\trsvd_8_1;\n+\tu8\t\tlog_max_ra_res_dc;\n+\tu8\t\trsvd9;\n+\tu8\t\tlog_max_ra_req_qp;\n+\tu8\t\tlog_max_qp_depth;\n+\tu8\t\tlog_max_ra_res_qp;\n+\t__be16\t\tmax_vfs;\n+\t__be16\t\traweth_qp_id_end;\n+\t__be16\t\traw_tpe_qp_num;\n+\t__be16\t\tmax_qp_count;\n+\t__be16\t\traweth_qp_id_base;\n+\tu8\t\trsvd13;\n+\tu8\t\tlocal_ca_ack_delay;\n+\tu8\t\tmax_num_eqs;\n+\tu8\t\tnum_ports;\n+\tu8\t\tlog_max_msg;\n+\tu8\t\tmac_port;\n+\t__be16\t\traweth_rss_qp_id_base;\n+\t__be16\t\tstat_rate_support;\n+\tu8\t\trsvd16[2];\n+\t__be64\t\tflags;\n+\tu8\t\trsvd17;\n+\tu8\t\tuar_sz;\n+\tu8\t\trsvd18;\n+\tu8\t\tlog_pg_sz;\n+\t__be16\t\tbf_log_bf_reg_size;\n+\t__be16\t\tmsix_base;\n+\t__be16\t\tmsix_num;\n+\t__be16\t\tmax_desc_sz_sq;\n+\tu8\t\trsvd20[2];\n+\t__be16\t\tmax_desc_sz_rq;\n+\tu8\t\trsvd21[2];\n+\t__be16\t\tmax_desc_sz_sq_dc;\n+\tu8\t\trsvd22[4];\n+\t__be16\t\tmax_qp_mcg;\n+\tu8\t\trsvd23;\n+\tu8\t\tlog_max_mcg;\n+\tu8\t\trsvd24;\n+\tu8\t\tlog_max_pd;\n+\tu8\t\trsvd25;\n+\tu8\t\tlog_max_xrcd;\n+\tu8\t\trsvd26[40];\n+\t__be32\t\tuar_page_sz;\n+\tu8\t\trsvd27[8];\n+\t__be32\t\thw_feature_flag; /* enum xsc_hw_feature_flag */\n+\t__be16\t\tpf0_vf_funcid_base;\n+\t__be16\t\tpf0_vf_funcid_top;\n+\t__be16\t\tpf1_vf_funcid_base;\n+\t__be16\t\tpf1_vf_funcid_top;\n+\t__be16\t\tpcie0_pf_funcid_base;\n+\t__be16\t\tpcie0_pf_funcid_top;\n+\t__be16\t\tpcie1_pf_funcid_base;\n+\t__be16\t\tpcie1_pf_funcid_top;\n+\tu8\t\tlog_msx_atomic_size_qp;\n+\tu8\t\tpcie_host;\n+\tu8\t\trsvd28;\n+\tu8\t\tlog_msx_atomic_size_dc;\n+\tu8\t\tboard_sn[XSC_BOARD_SN_LEN];\n+\tu8\t\tmax_tc;\n+\tu8\t\tmac_bit;\n+\t__be16\t\tfuncid_to_logic_port;\n+\tu8\t\trsvd29[6];\n+\tu8\t\tnif_port_num;\n+\tu8\t\treg_mr_via_cmdq;\n+\t__be32\t\thca_core_clock;\n+\t__be32\t\tmax_rwq_indirection_tables; /* rss_caps */\n+\t__be32\t\tmax_rwq_indirection_table_size; /* rss_caps */\n+\t__be32\t\tchip_ver_h;\n+\t__be32\t\tchip_ver_m;\n+\t__be32\t\tchip_ver_l;\n+\t__be32\t\thotfix_num;\n+\t__be32\t\tfeature_flag;\n+\t__be32\t\trx_pkt_len_max;\n+\t__be32\t\tglb_func_id;\n+\t__be64\t\ttx_db;\n+\t__be64\t\trx_db;\n+\t__be64\t\tcomplete_db;\n+\t__be64\t\tcomplete_reg;\n+\t__be64\t\tevent_db;\n+\t__be32\t\tqp_rate_limit_min;\n+\t__be32\t\tqp_rate_limit_max;\n+\tstruct xsc_fw_version\tfw_ver;\n+\tu8\t\tlag_logic_port_ofst;\n+\t/* V1 */\n+\t__be64\t\tmax_mr_size;\n+\t__be16\t\tmax_cmd_in_len;\n+\t__be16\t\tmax_cmd_out_len;\n+\t/* V2 */\n+\t__be32\t\tmax_qp;\n+\t__be32\t\tmax_cq;\n+\t__be32\t\tmax_pd;\n+\t__be32\t\tmax_mtt;\n+\t/* V3 */\n+\t__be32\t\tmpt_tbl_addr;\n+\t__be32\t\tmpt_tbl_depth;\n+\t__be32\t\tmpt_tbl_width;\n+\t__be32\t\tmtt_inst_base_addr;\n+\t__be32\t\tmtt_inst_stride;\n+\t__be32\t\tmtt_inst_num_log;\n+\t__be32\t\tmtt_inst_depth;\n+\t/* V4 */\n+\t__be16\t\tvf_funcid_base[XSC_MAX_NUM_PCIE_INTF][XSC_MAX_PF_NUM_PER_PCIE];\n+\t__be16\t\tvf_funcid_top[XSC_MAX_NUM_PCIE_INTF][XSC_MAX_PF_NUM_PER_PCIE];\n+\t__be16\t\tpf_funcid_base[XSC_MAX_NUM_PCIE_INTF];\n+\t__be16\t\tpf_funcid_top[XSC_MAX_NUM_PCIE_INTF];\n+\tu8\t\tpcie_no;\n+\tu8\t\tpf_id;\n+\t__be16\t\tvf_id;\n+\tu8\t\tpcie_host_num;\n+\tu8\t\tpf_num_per_pcie;\n+\t/* V6 */\n+\tu16\t\tft_support;\n+\t/* V7 */\n+\t__be16\t\ttotal_dynamic_vf_msix;\n+\t__be16\t\tmin_dynamic_vf_msix;\n+\t__be16\t\tmax_dynamic_vf_msix;\n+\t/* V8 */\n+\tu8\t\tmax_sgl_rd;\n+\tu8\t\trsvd30;\n+\t/* V9 */\n+\t__be32\t\tmdb_num;\n+\tu8\t\trsvd31[4];\n+\t__be64\t\tmdb_base;\n+};\n+\n+struct xsc_cmd_query_hca_cap_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be16\t\t\tcpu_num;\n+\tu8\t\t\trsvd[6];\n+};\n+\n+struct xsc_cmd_query_hca_cap_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd0[8];\n+\tstruct xsc_hca_cap\thca_cap;\n+};\n+\n+struct xsc_query_vport_state_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\tadmin_state:4;\n+\tu8\t\t\tstate:4;\n+};\n+\n+struct xsc_query_vport_state_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be32\t\t\tdata0;\n+#define XSC_QUERY_VPORT_OTHER_VPORT\tBIT(0)\n+#define XSC_QUERY_VPORT_VPORT_NUM_MASK\tGENMASK(16, 1)\n+};\n+\n+enum {\n+\tXSC_CMD_EVENT_RESP_CHANGE_LINK\t\t= BIT(0),\n+\tXSC_CMD_EVENT_RESP_TEMP_WARN\t\t= BIT(1),\n+\tXSC_CMD_EVENT_RESP_OVER_TEMP_PROTECTION\t= BIT(2),\n+};\n+\n+struct xsc_event_resp {\n+\tu8\t\t\tresp_cmd_type;\n+};\n+\n+struct xsc_event_query_type_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tu8\t\t\trsvd[2];\n+};\n+\n+struct xsc_event_query_type_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tstruct xsc_event_resp\tctx;\n+};\n+\n+struct xsc_modify_raw_qp_request {\n+\t__be16\t\tqpn;\n+\t__be16\t\tlag_id;\n+\t__be16\t\tfunc_id;\n+\tu8\t\tdma_direct;\n+\tu8\t\tprio;\n+\tu8\t\tqp_out_port;\n+\tu8\t\trsvd[7];\n+};\n+\n+struct xsc_modify_raw_qp_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tu8\t\t\tpcie_no;\n+\tu8\t\t\trsvd[7];\n+\tstruct xsc_modify_raw_qp_request\treq;\n+};\n+\n+struct xsc_modify_raw_qp_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+struct xsc_set_mtu_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be16\t\t\tmtu;\n+\t__be16\t\t\trx_buf_sz_min;\n+\tu8\t\t\tmac_port;\n+\tu8\t\t\trsvd;\n+};\n+\n+struct xsc_set_mtu_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+};\n+\n+struct xsc_query_eth_mac_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tu8\t\t\tindex;\n+};\n+\n+struct xsc_query_eth_mac_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\tmac[ETH_ALEN];\n+};\n+\n+enum {\n+\tXSC_TBM_CAP_HASH_PPH = 0,\n+\tXSC_TBM_CAP_RSS,\n+\tXSC_TBM_CAP_PP_BYPASS,\n+\tXSC_TBM_CAP_PCT_DROP_CONFIG,\n+};\n+\n+struct xsc_nic_attr {\n+\t__be16\tcaps;\n+\t__be16\tcaps_mask;\n+\tu8\tmac_addr[ETH_ALEN];\n+};\n+\n+struct xsc_rss_attr {\n+\tu8\trss_en;\n+\tu8\thfunc;\n+\t__be16\trqn_base;\n+\t__be16\trqn_num;\n+\t__be32\thash_tmpl;\n+};\n+\n+struct xsc_cmd_enable_nic_hca_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tstruct xsc_nic_attr\tnic;\n+\tstruct xsc_rss_attr\trss;\n+};\n+\n+struct xsc_cmd_enable_nic_hca_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd0[2];\n+};\n+\n+struct xsc_nic_dis_attr {\n+\t__be16\tcaps;\n+};\n+\n+struct xsc_cmd_disable_nic_hca_mbox_in {\n+\tstruct xsc_inbox_hdr\t\thdr;\n+\tstruct xsc_nic_dis_attr\t\tnic;\n+};\n+\n+struct xsc_cmd_disable_nic_hca_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd0[4];\n+};\n+\n+struct xsc_function_reset_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be16\t\t\tglb_func_id;\n+\tu8\t\t\trsvd[6];\n+};\n+\n+struct xsc_function_reset_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+struct xsc_cmd_query_guid_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+struct xsc_cmd_query_guid_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\t__be64\t\t\tguid;\n+};\n+\n+struct xsc_cmd_activate_hw_config_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+struct xsc_cmd_activate_hw_config_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tu8\t\t\trsvd[8];\n+};\n+\n+struct xsc_event_linkinfo {\n+\tu8 linkstatus; /*0:down, 1:up*/\n+\tu8 port;\n+\tu8 duplex;\n+\tu8 autoneg;\n+\tu32 linkspeed;\n+\tu64 supported;\n+\tu64 advertising;\n+\tu64 supported_fec;\t/* reserved, not support currently */\n+\tu64 advertised_fec;\t/* reserved, not support currently */\n+\tu64 supported_speed[2];\n+\tu64 advertising_speed[2];\n+};\n+\n+struct xsc_event_query_linkinfo_mbox_out {\n+\tstruct xsc_outbox_hdr\t\thdr;\n+\tstruct xsc_event_linkinfo\tctx;\n+};\n+\n+struct xsc_event_set_port_admin_status_mbox_in {\n+\tstruct xsc_inbox_hdr\thdr;\n+\t__be16\t\t\tstatus;\n+};\n+\n+struct xsc_event_set_port_admin_status_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\t__be32\t\t\tstatus;\n+};\n+\n+/* HW stats mbox */\n+struct xsc_hw_stats_rdma_pf {\n+\t/*by mac port*/\n+\tu64 rdma_tx_pkts;\n+\tu64 rdma_tx_bytes;\n+\tu64 rdma_rx_pkts;\n+\tu64 rdma_rx_bytes;\n+\tu64 np_cnp_sent;\n+\tu64 rp_cnp_handled;\n+\tu64 np_ecn_marked_roce_packets;\n+\tu64 rp_cnp_ignored;\n+\tu64 read_rsp_out_of_seq;\n+\tu64 implied_nak_seq_err;\n+\t/*by function*/\n+\tu64 out_of_sequence;\n+\tu64 packet_seq_err;\n+\tu64 out_of_buffer;\n+\tu64 rnr_nak_retry_err;\n+\tu64 local_ack_timeout_err;\n+\tu64 rx_read_requests;\n+\tu64 rx_write_requests;\n+\tu64 duplicate_requests;\n+\tu64 rdma_tx_pkts_func;\n+\tu64 rdma_tx_payload_bytes;\n+\tu64 rdma_rx_pkts_func;\n+\tu64 rdma_rx_payload_bytes;\n+\t/*global*/\n+\tu64 rdma_loopback_pkts;\n+\tu64 rdma_loopback_bytes;\n+\t/*for diamond*/\n+\tu64 out_of_sequence_sr;\n+\tu64 packet_seq_err_sr;\n+\tu64 rdma_ndp_rx_pkts;\n+\tu64 rdma_ndp_rx_trimmed_pkts;\n+\tu64 rdma_ndp_trimmed_pkts_sr;\n+\tu64 rx_icrc_encapsulated;\n+};\n+\n+struct xsc_hw_stats_rdma_vf {\n+\t/*by function*/\n+\tu64 rdma_tx_pkts_func;\n+\tu64 rdma_tx_payload_bytes;\n+\tu64 rdma_rx_pkts_func;\n+\tu64 rdma_rx_payload_bytes;\n+\n+\tu64 out_of_sequence;\n+\tu64 packet_seq_err;\n+\tu64 out_of_buffer;\n+\tu64 rnr_nak_retry_err;\n+\tu64 local_ack_timeout_err;\n+\tu64 rx_read_requests;\n+\tu64 rx_write_requests;\n+\tu64 duplicate_requests;\n+};\n+\n+struct xsc_hw_stats_rdma {\n+\tu8 is_pf;\n+\tu8 rsv;\n+\t__be16 ver;\n+\tunion {\n+\t\tstruct xsc_hw_stats_rdma_pf pf_stats;\n+\t\tstruct xsc_hw_stats_rdma_vf vf_stats;\n+\t} stats;\n+};\n+\n+#define CMD_QUERY_HW_STATS_V1 1\n+struct xsc_hw_stats_mbox_in {\n+\tstruct xsc_inbox_hdr hdr;\n+\tu8 mac_port;\n+\tu8 is_lag;\n+\tu8 lag_member_num;\n+\tu8 member_port[];\n+};\n+\n+struct xsc_hw_stats_rdma_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tstruct xsc_hw_stats_rdma\thw_stats;\n+};\n+\n+struct xsc_hw_global_stats_rdma {\n+\t/*by global*/\n+\tu64 rdma_loopback_pkts;\n+\tu64 rdma_loopback_bytes;\n+\tu64 rx_icrc_encapsulated;\n+\tu64 req_cqe_error;\n+\tu64 resp_cqe_error;\n+\tu64 cqe_msg_code_error;\n+};\n+\n+struct xsc_hw_global_stats_mbox_in {\n+\tstruct xsc_inbox_hdr hdr;\n+\tu8 rsv[4];\n+};\n+\n+struct xsc_hw_global_stats_mbox_out {\n+\tstruct xsc_outbox_hdr\thdr;\n+\tstruct xsc_hw_global_stats_rdma\thw_stats;\n+};\n+\n+#endif /* __XSC_CMD_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd_api.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd_api.h\nnew file mode 100644\nindex 0000000000000..3ef6080ae7408\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd_api.h\n@@ -0,0 +1,25 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_DRIVER_H\n+#define __XSC_DRIVER_H\n+\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_cmd.h\"\n+\n+int xsc_cmd_init(struct xsc_core_device *xdev);\n+void xsc_cmd_cleanup(struct xsc_core_device *xdev);\n+void xsc_cmd_use_events(struct xsc_core_device *xdev);\n+void xsc_cmd_use_polling(struct xsc_core_device *xdev);\n+int xsc_cmd_err_handler(struct xsc_core_device *xdev);\n+void xsc_cmd_resp_handler(struct xsc_core_device *xdev);\n+int xsc_cmd_exec(struct xsc_core_device *xdev,\n+\t\t void *in, unsigned int in_size,\n+\t\t void *out, unsigned int out_size);\n+int xsc_cmd_version_check(struct xsc_core_device *xdev);\n+\n+int xsc_cmd_status_to_err(struct xsc_outbox_hdr *hdr);\n+\n+#endif\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmdq.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmdq.h\nnew file mode 100644\nindex 0000000000000..0046111465401\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmdq.h\n@@ -0,0 +1,192 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_CMDQ_H\n+#define __XSC_CMDQ_H\n+\n+#include \u003clinux/if_ether.h\u003e\n+\n+#include \"common/xsc_cmd.h\"\n+\n+#define XSC_CMD_MAX_RETRY_CNT\t3\n+\n+enum {\n+\tXSC_CMD_TIMEOUT_MSEC\t= 10 * 1000,\n+\tXSC_CMD_WQ_MAX_NAME\t= 32,\n+};\n+\n+enum {\n+\tXSC_CMD_DATA, /* print command payload only */\n+\tXSC_CMD_TIME, /* print command execution time */\n+};\n+\n+enum {\n+\tXSC_MAX_COMMANDS\t\t= 32,\n+\tXSC_CMD_DATA_BLOCK_SIZE\t= 512,\n+\tXSC_PCI_CMD_XPORT\t\t= 7,\n+};\n+\n+struct xsc_cmd_prot_block {\n+\tu8\t\tdata[XSC_CMD_DATA_BLOCK_SIZE];\n+\tu8\t\trsvd0[48];\n+\t__be64\t\tnext;\n+\t__be32\t\tblock_num;\n+\t/* init to 0, dma user should change this val to 1 */\n+\tu8\t\towner_status;\n+\tu8\t\ttoken;\n+\tu8\t\tctrl_sig;\n+\tu8\t\tsig;\n+};\n+\n+struct cache_ent {\n+\t/* protect block chain allocations */\n+\tspinlock_t\t\tlock;\n+\tstruct list_head\thead;\n+};\n+\n+struct cmd_msg_cache {\n+\tstruct cache_ent\tlarge;\n+\tstruct cache_ent\tmed;\n+\n+};\n+\n+#define CMD_FIRST_SIZE 8\n+struct xsc_cmd_first {\n+\t__be32\t\tdata[CMD_FIRST_SIZE];\n+};\n+\n+struct xsc_cmd_mailbox {\n+\tvoid\t\t\t*buf;\n+\tdma_addr_t\t\tdma;\n+\tstruct xsc_cmd_mailbox *next;\n+};\n+\n+struct xsc_cmd_msg {\n+\tstruct list_head\tlist;\n+\tstruct cache_ent\t*cache;\n+\tu32\t\t\tlen;\n+\tstruct xsc_cmd_first\tfirst;\n+\tstruct xsc_cmd_mailbox\t*next;\n+};\n+\n+#define RSP_FIRST_SIZE 14\n+struct xsc_rsp_first {\n+\t/* can be larger, xsc_rsp_layout */\n+\t__be32\t\tdata[RSP_FIRST_SIZE];\n+};\n+\n+struct xsc_rsp_msg {\n+\tstruct list_head\t\tlist;\n+\tstruct cache_ent\t\t*cache;\n+\tu32\t\t\t\tlen;\n+\tstruct xsc_rsp_first\t\tfirst;\n+\tstruct xsc_cmd_mailbox\t\t*next;\n+};\n+\n+typedef void (*xsc_cmd_cbk_t)(int status, void *context);\n+\n+/* hw will use this for some records(e.g. vf_id) */\n+struct cmdq_rsv {\n+\tu16\tvf_id;\n+\tu8\trsv[2];\n+};\n+\n+/* related with hw, won't change */\n+\n+struct xsc_cmd_layout {\n+\tstruct cmdq_rsv\trsv0;\n+\t__be32\t\tinlen;\n+\t__be64\t\tin_ptr;\n+\t__be32\t\tin[CMD_FIRST_SIZE];\n+\t__be64\t\tout_ptr;\n+\t__be32\t\toutlen;\n+\tu8\t\ttoken;\n+\tu8\t\tsig;\n+\tu8\t\tidx;\n+\tu8\t\ttype: 7;\n+\t/* rsv for hw, arm will check this bit to make sure mem written */\n+\tu8\t\towner_bit: 1;\n+};\n+\n+struct xsc_rsp_layout {\n+\tstruct cmdq_rsv\trsv0;\n+\t__be32\t\tout[RSP_FIRST_SIZE];\n+\tu8\t\ttoken;\n+\tu8\t\tsig;\n+\tu8\t\tidx;\n+\tu8\t\ttype: 7;\n+\t/* rsv for hw, driver will check this bit to make sure mem written */\n+\tu8\t\towner_bit: 1;\n+};\n+\n+struct xsc_cmd_work_ent {\n+\tstruct xsc_cmd_msg\t*in;\n+\tstruct xsc_rsp_msg\t*out;\n+\tint\t\t\tidx;\n+\tstruct completion\tdone;\n+\tstruct xsc_cmd\t\t*cmd;\n+\tstruct work_struct\twork;\n+\tstruct xsc_cmd_layout\t*lay;\n+\tstruct xsc_rsp_layout\t*rsp_lay;\n+\tint\t\t\tret;\n+\tu8\t\t\tstatus;\n+\tu8\t\t\ttoken;\n+};\n+\n+enum xsc_cmd_status {\n+\tXSC_CMD_STATUS_NORMAL,\n+\tXSC_CMD_STATUS_TIMEDOUT,\n+};\n+\n+enum xsc_cmd_mode {\n+\tXSC_CMD_MODE_POLLING,\n+\tXSC_CMD_MODE_EVENTS,\n+};\n+\n+struct xsc_cmd {\n+\tvoid\t\t\t*cmd_buf;\n+\tvoid\t\t\t*cq_buf;\n+\tdma_addr_t\t\tdma;\n+\tdma_addr_t\t\tcq_dma;\n+\tu16\t\t\tcmd_pid;\n+\tu16\t\t\tcq_cid;\n+\tu8\t\t\towner_bit;\n+\tu8\t\t\tretry_cnt;\n+\tu8\t\t\tcmdif_rev;\n+\tu8\t\t\tlog_sz;\n+\tu8\t\t\tlog_stride;\n+\tint\t\t\tmax_reg_cmds;\n+\tint\t\t\tevents;\n+\tu32 __iomem\t\t*vector;\n+\n+\t/* protect command queue allocations */\n+\tspinlock_t\t\talloc_lock;\n+\t/* protect token allocations */\n+\tspinlock_t\t\ttoken_lock;\n+\t/* protect cmdq req pid doorbell */\n+\tspinlock_t\t\tdoorbell_lock;\n+\tu8\t\t\ttoken;\n+\tunsigned long\t\tcmd_entry_mask;\n+\tchar\t\t\twq_name[XSC_CMD_WQ_MAX_NAME];\n+\tstruct workqueue_struct\t*wq;\n+\tstruct task_struct\t*cq_task;\n+\t/* The semaphore limits the number of working commands\n+\t * to max_reg_cmds. Each exec_cmd call does a down\n+\t * to acquire the semaphore before execution and an up\n+\t * after completion, ensuring no more than max_reg_cmds\n+\t * commands run at the same time.\n+\t */\n+\tstruct semaphore\tsem;\n+\tenum xsc_cmd_mode\tmode;\n+\tstruct xsc_cmd_work_ent\t*ent_arr[XSC_MAX_COMMANDS];\n+\tstruct dma_pool\t\t*pool;\n+\tstruct cmd_msg_cache\tcache;\n+\tint\t\t\tchecksum_disabled;\n+\tunsigned int\t\tirqn;\n+\tu8\t\t\townerbit_learned;\n+\tu8\t\t\tcmd_status;\n+};\n+\n+#endif\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h\nnew file mode 100644\nindex 0000000000000..f491b9d8d8045\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h\n@@ -0,0 +1,707 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_CORE_H\n+#define __XSC_CORE_H\n+\n+#include \u003clinux/pci.h\u003e\n+#include \u003clinux/auxiliary_bus.h\u003e\n+#include \u003clinux/bitfield.h\u003e\n+#include \u003clinux/if_vlan.h\u003e\n+#include \u003clinux/reboot.h\u003e\n+\n+#include \"common/xsc_reg.h\"\n+#include \"common/xsc_cmdq.h\"\n+\n+#define XSC_PCI_VENDOR_ID\t\t0x1f67\n+\n+#define XSC_MC_PF_DEV_ID\t\t0x1011\n+#define XSC_MC_VF_DEV_ID\t\t0x1012\n+\n+#define XSC_MF_HOST_PF_DEV_ID\t\t0x1051\n+#define XSC_MF_HOST_VF_DEV_ID\t\t0x1052\n+#define XSC_MF_SOC_PF_DEV_ID\t\t0x1053\n+\n+#define XSC_MS_PF_DEV_ID\t\t0x1111\n+#define XSC_MS_VF_DEV_ID\t\t0x1112\n+\n+#define XSC_MV_HOST_PF_DEV_ID\t\t0x1151\n+#define XSC_MV_HOST_VF_DEV_ID\t\t0x1152\n+#define XSC_MV_SOC_PF_DEV_ID\t\t0x1153\n+\n+#define PAGE_SHIFT_4K\t\t\t12\n+#define PAGE_SIZE_4K\t\t\t(_AC(1, UL) \u003c\u003c PAGE_SHIFT_4K)\n+\n+#define XSC_ORIGIN_PF_BAR_SIZE\t\t(256 * 1024 * 1024)\n+\n+enum xsc_coredev_type {\n+\tXSC_COREDEV_PF,\n+\tXSC_COREDEV_VF,\n+\tXSC_COREDEV_SF\n+};\n+\n+enum {\n+\tXSC_MAX_EQ_NAME\t= 20\n+};\n+\n+enum {\n+\tXSC_MAX_IRQ_NAME\t= 32\n+};\n+\n+enum {\n+\tXSC_MAX_FW_PORTS\t= 1,\n+};\n+\n+enum {\n+\tXSC_BF_REGS_PER_PAGE\t= 4,\n+\tXSC_MAX_UAR_PAGES\t= 1 \u003c\u003c 8,\n+\tXSC_MAX_UUARS\t\t= XSC_MAX_UAR_PAGES * XSC_BF_REGS_PER_PAGE,\n+};\n+\n+/* alloc */\n+struct xsc_buf_list {\n+\tvoid\t\t\t*buf;\n+\tdma_addr_t\t\tmap;\n+};\n+\n+struct xsc_buf {\n+\tstruct xsc_buf_list\tdirect;\n+\tstruct xsc_buf_list\t*page_list;\n+\tunsigned long\t\tnbufs;\n+\tunsigned long\t\tnpages;\n+\tunsigned long\t\tpage_shift;\n+\tunsigned long\t\tsize;\n+};\n+\n+struct xsc_frag_buf {\n+\tstruct xsc_buf_list\t*frags;\n+\tunsigned long\t\tnpages;\n+\tunsigned long\t\tsize;\n+\tu8\t\t\tpage_shift;\n+};\n+\n+struct xsc_frag_buf_ctrl {\n+\tstruct xsc_buf_list\t*frags;\n+\tu32\t\t\tsz_m1;\n+\tu16\t\t\tfrag_sz_m1;\n+\tu16\t\t\tstrides_offset;\n+\tu8\t\t\tlog_sz;\n+\tu8\t\t\tlog_stride;\n+\tu8\t\t\tlog_frag_strides;\n+};\n+\n+/* qp */\n+struct xsc_send_wqe_ctrl_seg {\n+\tunion {\n+\t\t__le32\t\tdata0;\n+\t\tstruct {\n+\t\t\t__le32\t\tmsg_opcode:8;\n+\t\t\t__le32\t\twith_immdt:1;\n+\t\t\t__le32\t\tcsum_en:2;\n+\t\t\t__le32\t\tds_data_num:5;\n+\t\t\t__le32\t\twqe_id:16;\n+\t\t};\n+\t};\n+#define XSC_WQE_CTRL_SEG_MSG_OPCODE_MASK\tGENMASK(7, 0)\n+#define XSC_WQE_CTRL_SEG_WITH_IMM\t\tBIT(8)\n+#define XSC_WQE_CTRL_SEG_CSUM_EN_MASK\t\tGENMASK(10, 9)\n+#define XSC_WQE_CTRL_SEG_DS_DATA_NUM_MASK\tGENMASK(15, 11)\n+#define XSC_WQE_CTRL_SEG_WQE_ID_MASK\t\tGENMASK(31, 16)\n+\n+\t__le32\t\tmsg_len;\n+\n+\tunion {\n+\t\t__le32\t\tdata2;\n+\t\t__le32\t\topcode_data;\n+\t\tstruct {\n+\t\t\tu8\t\thas_pph:1;\n+\t\t\tu8\t\tso_type:1;\n+\t\t\t__le16\t\tso_data_size:14;\n+\t\t\tu8:8;\n+\t\t\tu8\t\tso_hdr_len:8;\n+\t\t};\n+\t};\n+#define XSC_WQE_CTRL_SEG_HAS_PPH\t\tBIT(0)\n+#define XSC_WQE_CTRL_SEG_SO_TYPE\t\tBIT(1)\n+#define XSC_WQE_CTRL_SEG_SO_DATA_SIZE_MASK\tGENMASK(15, 2)\n+#define XSC_WQE_CTRL_SEG_SO_HDR_LEN_MASK\tGENMASK(31, 24)\n+\n+\tunion {\n+\t\t__le32\t\tdata3;\n+\t\tstruct {\n+\t\t\t__le32\t\tse:1;\n+\t\t\t__le32\t\tce:1;\n+\t\t\t__le32:30;\n+\t\t};\n+\t};\n+#define XSC_WQE_CTRL_SEG_SE\t\t\tBIT(0)\n+#define XSC_WQE_CTRL_SEG_CE\t\t\tBIT(1)\n+};\n+\n+struct xsc_wqe_data_seg {\n+\tunion {\n+\t\t__le32\t\tdata0;\n+\t\t__le32\t\tin_line:1;\n+\t\tstruct {\n+\t\t\t__le32:1;\n+\t\t\t__le32\t\tseg_len:31;\n+\t\t\t__le32\t\tmkey;\n+\t\t\t__le64\t\tva;\n+\t\t};\n+\t\tstruct {\n+\t\t\t__le32:1;\n+\t\t\t__le32\t\tlen:7;\n+\t\t\tu8\t\tin_line_data[15];\n+\t\t};\n+\t};\n+#define XSC_WQE_DATA_SEG_SEG_LEN_MASK\t\tGENMASK(31, 1)\n+};\n+\n+struct xsc_core_qp {\n+\tvoid\t\t\t(*event)(struct xsc_core_qp *qp, int type);\n+\tu32\t\t\tqpn;\n+\tatomic_t\t\trefcount;\n+\tstruct completion\tfree;\n+\tint\t\t\tpid;\n+\tu16\t\t\tqp_type;\n+\tu16\t\t\teth_queue_type;\n+\tu16\t\t\tqp_type_internal;\n+\tu16\t\t\tgrp_id;\n+\tu8\t\t\tmac_id;\n+\tu8\t\t\terr_occurred;\n+\tu32\t\t\tqp_protocol_mode;\n+\tu32\t\t\tqp_access_flags;\n+};\n+\n+struct xsc_qp_table {\n+\tspinlock_t\t\tlock; /* protect radix tree */\n+\tstruct radix_tree_root\ttree;\n+};\n+\n+/* cq */\n+#define XSC_MAX_NAME_LEN\t32\n+#define XSC_MAX_IBDEV_NAME_LEN\t64\n+\n+struct xsc_priv {\n+\tchar\t\t\tname[XSC_MAX_NAME_LEN];\n+\tchar\t\t\tibdev_name[XSC_MAX_IBDEV_NAME_LEN];\n+\tint\t\t\tcm_pcp;\n+\tint\t\t\tcm_dscp;\n+\tint\t\t\tforce_pcp;\n+\tint\t\t\tforce_dscp;\n+};\n+\n+enum xsc_dev_event {\n+\tXSC_DEV_EVENT_SYS_ERROR,\n+\tXSC_DEV_EVENT_PORT_UP,\n+\tXSC_DEV_EVENT_PORT_DOWN,\n+\tXSC_DEV_EVENT_PORT_INITIALIZED,\n+\tXSC_DEV_EVENT_LID_CHANGE,\n+\tXSC_DEV_EVENT_PKEY_CHANGE,\n+\tXSC_DEV_EVENT_GUID_CHANGE,\n+\tXSC_DEV_EVENT_CLIENT_REREG,\n+};\n+\n+enum {\n+\tRDMA_PROTO_ROCEV2,\n+\tRDMA_PROTO_VEROCE,\n+};\n+\n+#define VEROCE_UDP_DPORT\t4794\n+#define MAX_MR_CACHE_ENTRIES\t5000\n+\n+enum xsc_event {\n+\tXSC_EVENT_TYPE_COMP\t\t\t= 0x0,\n+\t/* mad */\n+\tXSC_EVENT_TYPE_COMM_EST\t\t\t= 0x02,\n+\tXSC_EVENT_TYPE_CQ_ERROR\t\t\t= 0x04,\n+\tXSC_EVENT_TYPE_WQ_CATAS_ERROR\t\t= 0x05,\n+\tXSC_EVENT_TYPE_INTERNAL_ERROR\t\t= 0x08,\n+\t/* IBV_EVENT_QP_REQ_ERR */\n+\tXSC_EVENT_TYPE_WQ_INVAL_REQ_ERROR\t= 0x10,\n+\t/* IBV_EVENT_QP_ACCESS_ERR */\n+\tXSC_EVENT_TYPE_WQ_ACCESS_ERROR\t\t= 0x11,\n+};\n+\n+struct xsc_cqe {\n+\tunion {\n+\t\t__le32\t\tdata0;\n+\t\tstruct {\n+\t\t\tu8\t\tplaceholder1;\n+#define XSC_CQE_MSG_OPCD_MASK\t\tGENMASK(7, 0)\n+#define XSC_CQE_ERR_CODE_MASK\t\tGENMASK(6, 0)\n+#define XSC_CQE_IS_ERR\t\t\tBIT(7)\n+#define XSC_CQE_QP_ID_MASK\t\tGENMASK(22, 8)\n+#define XSC_CQE_SE\t\t\tBIT(24)\n+#define XSC_CQE_HAS_PPH\t\t\tBIT(25)\n+#define XSC_CQE_TYPE\t\t\tBIT(26)\n+#define XSC_CQE_WITH_IMM\t\tBIT(27)\n+#define XSC_CQE_CSUM_ERR_MASK\t\tGENMASK(31, 28)\n+\t\t\t__le32\t\tqp_id:15;\n+\t\t\tu8\t\trsv1:1;\n+\t\t\tu8\t\tse:1;\n+\t\t\tu8\t\thas_pph:1;\n+\t\t\tu8\t\ttype:1;\n+\t\t\tu8\t\twith_immdt:1;\n+\t\t\tu8\t\tcsum_err:4;\n+\t\t};\n+\t};\n+\n+\t__le32\t\timm_data;\n+\t__le32\t\tmsg_len;\n+\t__le32\t\tvni;\n+\n+\t__le64\t\tts:48;\n+\n+\t__le16\t\twqe_id;\n+\tu8\t\tplaceholder2;\n+\tu8\t\trsv3;\n+\t__le16\t\trsv[1];\n+\n+\tunion {\n+\t\t__le32\t\tdata1;\n+\t\tstruct {\n+\t\t\t__le16\t\trsv0;\n+\t\t\t__le16\t\trsv2:15;\n+#define XSC_CQE_OWNER\t\t\tBIT(31)\n+\t\t\tu8\t\towner:1;\n+\t\t};\n+\t};\n+};\n+\n+struct xsc_cqe64 {\n+\tstruct xsc_cqe\tcqe;\n+\tu8\t\tpadding[32];\n+};\n+\n+struct xsc_core_cq {\n+\tu32\t\t\tcqn;\n+\tu32\t\t\tcqe_sz;\n+\tstruct xsc_core_device\t*xdev;\n+\tstruct xsc_core_device\t*dev;\n+\tatomic_t\t\trefcount;\n+\tstruct completion\tfree;\n+\tunsigned int\t\tvector;\n+\tunsigned int\t\tirqn;\n+\tvoid\t\t\t(*comp)(struct xsc_core_cq *cq);\n+\tvoid\t\t\t(*event)(struct xsc_core_cq *cq,\n+\t\t\t\t\t enum xsc_event);\n+\tu32\t\t\tcons_index;\n+\tunsigned int\t\tarm_sn;\n+\tint\t\t\tpid;\n+\tu32\t\t\treg_next_cid;\n+\tu32\t\t\treg_done_pid;\n+\tstruct xsc_eq\t\t*eq;\n+};\n+\n+struct xsc_cq_table {\n+\tspinlock_t\t\tlock; /* protect radix tree */\n+\tstruct radix_tree_root\ttree;\n+};\n+\n+/* mr */\n+struct xsc_core_mr {\n+\tu64\t\t\tiova;\n+\tu64\t\t\tsize;\n+\tu32\t\t\tkey;\n+\tu32\t\t\tpd;\n+\tu32\t\t\taccess;\n+};\n+\n+/* eq */\n+struct xsc_eq {\n+\tstruct xsc_core_device\t*dev;\n+\tstruct xsc_cq_table\tcq_table;\n+\tu32\t\t\tcons_index;\n+\tstruct xsc_buf\t\tbuf;\n+\tunsigned int\t\tirqn;\n+\tu32\t\t\teqn;\n+\tu32\t\t\tnent;\n+\tcpumask_var_t\t\tmask;\n+\tchar\t\t\tname[XSC_MAX_EQ_NAME];\n+\tstruct list_head\tlist;\n+\tu16\t\t\tindex;\n+};\n+\n+struct xsc_eq_table {\n+\tvoid __iomem\t\t*update_ci;\n+\tvoid __iomem\t\t*update_arm_ci;\n+\tstruct list_head\tcomp_eqs_list;\n+\tstruct xsc_eq\t\tpages_eq;\n+\tstruct xsc_eq\t\tasync_eq;\n+\tstruct xsc_eq\t\tcmd_eq;\n+\tint\t\t\tnum_comp_vectors;\n+\tint\t\t\teq_vec_comp_base;\n+\t/* protect EQs list */\n+\tspinlock_t\t\tlock;\n+};\n+\n+/* irq */\n+struct xsc_irq_info {\n+\tcpumask_var_t mask;\n+\tchar name[XSC_MAX_IRQ_NAME];\n+};\n+\n+/* adev */\n+#define XSC_PCI_DRV_NAME \"xsc_pci\"\n+#define XSC_ETH_ADEV_NAME \"eth\"\n+#define XSC_IB_ADEV_NAME \"ib\"\n+\n+enum xsc_adev_idx {\n+\tXSC_ADEV_IDX_ETH,\n+\tXSC_ADEV_IDX_IB,\n+\tXSC_ADEV_IDX_MAX,\n+};\n+\n+struct xsc_adev {\n+\tstruct auxiliary_device\tadev;\n+\tstruct xsc_core_device\t*xdev;\n+\n+\tint\t\t\tidx;\n+};\n+\n+struct xsc_board_info {\n+\tu32\t\t\tboard_id;\n+\tchar\t\t\tboard_sn[XSC_BOARD_SN_LEN];\n+\t__be64\t\t\tguid;\n+\tu8\t\t\tguid_valid;\n+\tu8\t\t\thw_config_activated;\n+\tstruct xsc_msg_opcode\t*msg_opcode;\n+};\n+\n+struct xsc_caps {\n+\tu8\t\tlog_max_eq;\n+\tu8\t\tlog_max_cq;\n+\tu8\t\tlog_max_qp;\n+\tu8\t\tlog_max_mkey;\n+\tu8\t\tlog_max_pd;\n+\tu8\t\tlog_max_srq;\n+\tu8\t\tlog_max_msix;\n+\tu32\t\tmax_cq;\n+\tu32\t\tmax_qp;\n+\tu32\t\tmax_pd;\n+\tu32\t\tmax_cqes;\n+\tu32\t\tmax_wqes;\n+\tu32\t\tmax_sq_desc_sz;\n+\tu32\t\tmax_rq_desc_sz;\n+\tu64\t\tflags;\n+\tu16\t\tstat_rate_support;\n+\tu32\t\tlog_max_msg;\n+\tu32\t\tnum_ports;\n+\tu32\t\tmax_ra_res_qp;\n+\tu32\t\tmax_ra_req_qp;\n+\tu32\t\tmax_srq_wqes;\n+\tu32\t\tbf_reg_size;\n+\tu32\t\tbf_regs_per_page;\n+\tu32\t\treserved_lkey;\n+\tu8\t\tlocal_ca_ack_delay;\n+\tu8\t\tlog_max_mcg;\n+\tu16\t\tmax_qp_mcg;\n+\tu32\t\tmin_page_sz;\n+\tu32\t\tsend_ds_num;\n+\tu32\t\tsend_wqe_shift;\n+\tu32\t\trecv_ds_num;\n+\tu32\t\trecv_wqe_shift;\n+\tu32\t\trx_pkt_len_max;\n+\n+\tu32\t\tmsix_enable:1;\n+\tu32\t\tport_type:1;\n+\tu32\t\tembedded_cpu:1;\n+\tu32\t\teswitch_manager:1;\n+\tu32\t\tecpf_vport_exists:1;\n+\tu32\t\tvport_group_manager:1;\n+\tu32\t\tsf:1;\n+\tu32\t\twqe_inline_mode:3;\n+\tu32\t\traweth_qp_id_base:15;\n+\tu32\t\trsvd0:7;\n+\n+\tu16\t\tmax_vfs;\n+\tu8\t\tlog_max_qp_depth;\n+\tu8\t\tlog_max_current_uc_list;\n+\tu8\t\tlog_max_current_mc_list;\n+\tu16\t\tlog_max_vlan_list;\n+\tu8\t\tfdb_multi_path_to_table;\n+\tu8\t\tlog_esw_max_sched_depth;\n+\n+\tu8\t\tmax_num_sf_partitions;\n+\tu8\t\tlog_max_esw_sf;\n+\tu16\t\tsf_base_id;\n+\n+\tu32\t\tmax_tc:8;\n+\tu32\t\tets:1;\n+\tu32\t\tdcbx:1;\n+\tu32\t\tdscp:1;\n+\tu32\t\tsbcam_reg:1;\n+\tu32\t\tqos:1;\n+\tu32\t\tport_buf:1;\n+\tu32\t\trsvd1:2;\n+\tu32\t\traw_tpe_qp_num:16;\n+\tu32\t\tmax_num_eqs:8;\n+\tu32\t\tmac_port:8;\n+\tu32\t\traweth_rss_qp_id_base:16;\n+\tu16\t\tmsix_base;\n+\tu16\t\tmsix_num;\n+\tu8\t\tlog_max_mtt;\n+\tu8\t\tlog_max_tso;\n+\tu32\t\thca_core_clock;\n+\tu32\t\tmax_rwq_indirection_tables; /* rss_caps */\n+\tu32\t\tmax_rwq_indirection_table_size; /* rss_caps */\n+\tu16\t\traweth_qp_id_end;\n+\tu32\t\tqp_rate_limit_min;\n+\tu32\t\tqp_rate_limit_max;\n+\tu32\t\thw_feature_flag;\n+\tu16\t\tpf0_vf_funcid_base;\n+\tu16\t\tpf0_vf_funcid_top;\n+\tu16\t\tpf1_vf_funcid_base;\n+\tu16\t\tpf1_vf_funcid_top;\n+\tu16\t\tpcie0_pf_funcid_base;\n+\tu16\t\tpcie0_pf_funcid_top;\n+\tu16\t\tpcie1_pf_funcid_base;\n+\tu16\t\tpcie1_pf_funcid_top;\n+\tu8\t\tnif_port_num;\n+\tu8\t\tpcie_host;\n+\tu8\t\tmac_bit;\n+\tu16\t\tfuncid_to_logic_port;\n+\tu8\t\tlag_logic_port_ofst;\n+\n+\t/* RDMA / MR resources (populated from hca_cap V1~V9) */\n+\tu64\t\tmax_mr_size;\n+\tu16\t\tmax_cmd_in_len;\n+\tu16\t\tmax_cmd_out_len;\n+\tu32\t\tmax_mtt;\n+\tu32\t\tmpt_tbl_addr;\n+\tu32\t\tmpt_tbl_depth;\n+\tu32\t\tmpt_tbl_width;\n+\tu32\t\tmtt_inst_base_addr;\n+\tu32\t\tmtt_inst_stride;\n+\tu32\t\tmtt_inst_num_log;\n+\tu32\t\tmtt_inst_depth;\n+\tu32\t\tmdb_num;\n+\tu64\t\tmdb_base;\n+};\n+\n+struct xsc_srq_cache_wr;\n+\n+/* xsc_core */\n+struct xsc_dev_resource {\n+\tstruct xsc_qp_table\tqp_table;\n+\tstruct xsc_cq_table\tcq_table;\n+\tstruct xsc_eq_table\teq_table;\n+\tstruct xsc_irq_info\t*irq_info;\n+\t/* protect buffer allocation according to numa node */\n+\tstruct mutex\t\talloc_mutex;\n+\t/* protect mkey key allocation */\n+\tspinlock_t\t\tmkey_lock;\n+\tu8\t\t\tmkey_key;\n+\n+\t/* srq number allocation, srqn is a software index only */\n+\tspinlock_t\t\tsrq_lock;\n+\tunsigned long\t\t*srq_tbl;\n+\tstruct xsc_srq_cache_wr\t*srq_cache_wr;\n+\tu32\t\t\tmax_srq_num;\n+};\n+\n+struct xsc_core_device {\n+\tstruct pci_dev\t\t*pdev;\n+\tstruct device\t\t*device;\n+\tint\t\t\tadev_id;\n+\tstruct xsc_adev\t\t**xsc_adev_list;\n+\tvoid\t\t\t*eth_priv;\n+\tstruct xsc_dev_resource\t*dev_res;\n+\tint\t\t\tnuma_node;\n+\n+\t/* async event plumbing, owned by the PCI layer */\n+\tstruct work_struct\tevent_work;\n+\tstruct workqueue_struct\t*event_wq;\n+\tvoid (*link_event_handler)(void *xdev);\n+\n+\tvoid __iomem\t\t*bar;\n+\tint\t\t\tbar_num;\n+\tenum xsc_coredev_type\tcoredev_type;\n+\tstruct xsc_hw_reg\t*hal_regs;\n+\n+\tu8\t\t\tmac_port;\n+\tu8\t\t\tpcie_no;\n+\tu8\t\t\tpf_id;\n+\tu16\t\t\tglb_func_id;\n+\tu16\t\t\tgsi_qpn;\n+\tatomic_t\t\tnum_qps;\n+\n+\tu16\t\t\tmsix_vec_base;\n+\n+\tvoid\t\t\t*netdev;\n+\n+\tu32\t\t\trdma_proto_mode;\n+\tu16\t\t\tcm_udp_dst_port;\n+\tu16\t\t\tveroce_udp_dst_port;\n+\tu8\t\t\tread_flush;\n+\tu8\t\t\tbql_thresh;\n+\n+\tstruct xsc_priv\t\tpriv;\n+\n+\tstruct xsc_cmd\t\tcmd;\n+\tu16\t\t\tcmdq_ver;\n+\n+\tstruct xsc_caps\t\tcaps;\n+\tstruct xsc_board_info\t*board_info;\n+\n+\tu32\t\t\tchip_ver_h;\n+\tu32\t\t\tchip_ver_m;\n+\tu32\t\t\tchip_ver_l;\n+\tu32\t\t\thotfix_num;\n+\tu32\t\t\tfeature_flag;\n+\n+\tu8\t\t\tfw_version_major;\n+\tu8\t\t\tfw_version_minor;\n+\tu16\t\t\tfw_version_patch;\n+\tu32\t\t\tfw_version_tweak;\n+\tu8\t\t\tfw_version_extra_flag;\n+\tcpumask_var_t\t\txps_cpumask;\n+};\n+\n+int xsc_core_create_resource_common(struct xsc_core_device *xdev,\n+\t\t\t\t struct xsc_core_qp *qp);\n+void xsc_core_destroy_resource_common(struct xsc_core_device *xdev,\n+\t\t\t\t struct xsc_core_qp *qp);\n+int xsc_core_eth_create_qp(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_create_qp_mbox_in *in,\n+\t\t\t int insize, u32 *p_qpn);\n+int xsc_core_eth_modify_qp_status(struct xsc_core_device *xdev,\n+\t\t\t\t u32 qpn, u16 status);\n+int xsc_core_eth_destroy_qp(struct xsc_core_device *xdev, u32 qpn);\n+int xsc_core_eth_create_rss_qp_rqs(struct xsc_core_device *xdev,\n+\t\t\t\t struct xsc_create_multiqp_mbox_in *in,\n+\t\t\t\t int insize,\n+\t\t\t\t u32 *p_qpn_base);\n+int xsc_core_eth_modify_raw_qp(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_modify_raw_qp_mbox_in *in);\n+int xsc_core_eth_create_cq(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_core_cq *xcq,\n+\t\t\t struct xsc_create_cq_mbox_in *in,\n+\t\t\t int insize);\n+int xsc_core_eth_destroy_cq(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_core_cq *xcq);\n+\n+struct xsc_eq *xsc_core_eq_get(struct xsc_core_device *xdev, int i);\n+int xsc_core_vector2eqn(struct xsc_core_device *xdev, int vector, u32 *eqn,\n+\t\t\tunsigned int *irqn);\n+\n+/*\n+ * Link event callback registration. The events are dispatched from the PCI\n+ * layer's own workqueue; unregister detaches the callback and waits for one\n+ * already in flight, so it must precede freeing what the handler touches.\n+ */\n+void xsc_irq_event_register(struct xsc_core_device *xdev,\n+\t\t\t void (*handler)(void *xdev));\n+void xsc_irq_event_unregister(struct xsc_core_device *xdev);\n+void xsc_core_fill_page_frag_array(struct xsc_frag_buf *buf,\n+\t\t\t\t __be64 *pas, unsigned int npages);\n+int xsc_core_frag_buf_alloc_node(struct xsc_core_device *xdev,\n+\t\t\t\t unsigned long size,\n+\t\t\t\t struct xsc_frag_buf *buf,\n+\t\t\t\t int node);\n+void xsc_core_frag_buf_free(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_frag_buf *buf);\n+\n+u8 xsc_core_query_vport_state(struct xsc_core_device *xdev, u16 vport);\n+\n+static inline void *xsc_buf_offset(struct xsc_buf *buf, unsigned long offset)\n+{\n+\tif (likely(buf-\u003enbufs == 1))\n+\t\treturn buf-\u003edirect.buf + offset;\n+\telse\n+\t\treturn buf-\u003epage_list[offset \u003e\u003e PAGE_SHIFT].buf +\n+\t\t\t(offset \u0026 (PAGE_SIZE - 1));\n+}\n+\n+static inline bool xsc_fw_is_available(struct xsc_core_device *xdev)\n+{\n+\treturn xdev-\u003ecmd.cmd_status == XSC_CMD_STATUS_NORMAL;\n+}\n+\n+static inline u8 get_cqe_opcode(struct xsc_cqe *cqe)\n+{\n+\treturn FIELD_GET(XSC_CQE_MSG_OPCD_MASK, le32_to_cpu(cqe-\u003edata0));\n+}\n+\n+static inline bool xsc_core_is_pf(const struct xsc_core_device *dev)\n+{\n+\treturn dev-\u003ecoredev_type == XSC_COREDEV_PF;\n+}\n+\n+static inline bool is_support_hw_pf_stats(struct xsc_core_device *dev)\n+{\n+\treturn xsc_core_is_pf(dev);\n+}\n+\n+static inline bool is_support_cqe64(struct xsc_core_device *dev)\n+{\n+\treturn dev-\u003ecaps.hw_feature_flag \u0026 XSC_HW_CQE64_SUPPORT;\n+}\n+\n+static inline bool is_support_rdma_cm(struct xsc_core_device *dev)\n+{\n+\treturn dev-\u003ecaps.hw_feature_flag \u0026 XSC_HW_RDMA_CM_SUPPORT;\n+}\n+\n+static inline bool is_support_rdma_atomic(struct xsc_core_device *dev)\n+{\n+\treturn dev-\u003ecaps.hw_feature_flag \u0026 XSC_HW_RDMA_ATOMIC_SUPPORT;\n+}\n+\n+static inline bool is_support_rdma(struct xsc_core_device *dev)\n+{\n+\treturn dev-\u003ecaps.hw_feature_flag \u0026 XSC_HW_RDMA_SUPPORT;\n+}\n+\n+static inline bool is_support_8k_mtu(struct xsc_core_device *dev)\n+{\n+\treturn dev-\u003ecaps.hw_feature_flag \u0026 XSC_HW_MTU_8192_SUPPORT;\n+}\n+\n+static inline bool is_pf_bar_compressed(struct xsc_core_device *dev)\n+{\n+\treturn pci_resource_len(dev-\u003epdev, 0) != XSC_ORIGIN_PF_BAR_SIZE;\n+}\n+\n+static inline bool is_support_multi_doorbell(struct xsc_core_device *dev)\n+{\n+\treturn dev-\u003ecaps.hw_feature_flag \u0026 XSC_HW_MULTIDOORBELL_SUPPORT;\n+}\n+\n+void xsc_remove_rdma_driver(void);\n+int xsc_counters_init(struct ib_device *ib_dev, struct xsc_core_device *xdev);\n+void xsc_counters_fini(struct ib_device *ib_dev, struct xsc_core_device *xdev);\n+void xsc_ib_sysfs_init(struct ib_device *ib_dev, struct xsc_core_device *xdev);\n+void xsc_ib_sysfs_fini(struct ib_device *ib_dev, struct xsc_core_device *xdev);\n+\n+void xsc_init_hal(struct xsc_core_device *xdev, u32 device_id);\n+\n+u8 xsc_get_mr_page_mode(struct xsc_core_device *xdev, u8 page_shift);\n+u64 xsc_get_max_mr_size(struct xsc_core_device *xdev);\n+void xsc_set_data_seg(struct xsc_core_device *xdev, void *data_seg, u64 addr,\n+\t\t u32 key, u32 length);\n+u64 xsc_get_rdma_stat_mask(struct xsc_core_device *xdev, u16 ver);\n+u32 xsc_get_max_qp_depth(struct xsc_core_device *xdev);\n+bool xsc_check_max_qp_depth(struct xsc_core_device *xdev, u32 *wqe_cnt,\n+\t\t\t u32 max_qp_depth);\n+void xsc_set_wqe_id(struct xsc_core_device *xdev, void *cseg, u32 wqe_id);\n+int xsc_get_cqe_wqe_id(struct xsc_core_device *xdev, void *cqe);\n+void xsc_get_multidb_info(struct xsc_core_device *xdev, u32 *num, u32 *base);\n+u8 xsc_get_cqe_opcode(struct xsc_core_device *xdev, void *cqe);\n+u32 xsc_get_recv_ds_num(struct xsc_core_device *xdev);\n+int xsc_get_link_speed(struct xsc_core_device *xdev);\n+void xsc_set_mtt_info(struct xsc_core_device *xdev);\n+void xsc_set_pf_db_addr(struct xsc_core_device *xdev,\n+\t\t\tu64 tx_db, u64 rx_db, u64 cq_db, u64 cq_reg, u64 eq_db);\n+void xsc_get_db_addr(struct xsc_core_device *xdev,\n+\t\t u64 *tx_db, u64 *rx_db, u64 *cq_db, u64 *cq_reg, u64 *eq_db);\n+\n+void xsc_update_tx_db(struct xsc_core_device *xdev, u32 sqn, u32 next_pid);\n+void xsc_update_rx_db(struct xsc_core_device *xdev, u32 rqn, u32 next_pid);\n+void xsc_arm_cq(struct xsc_core_device *xdev, u32 cqn, u32 next_cid, u8 solicited);\n+void xsc_update_cq_ci(struct xsc_core_device *xdev, u32 cqn, u32 next_cid);\n+\n+#endif\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_device.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_device.h\nnew file mode 100644\nindex 0000000000000..3025348390dd8\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_device.h\n@@ -0,0 +1,94 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_DEVICE_H\n+#define __XSC_DEVICE_H\n+\n+#include \u003clinux/types.h\u003e\n+\n+/* QP type */\n+enum {\n+\tXSC_QUEUE_TYPE_RDMA_RC\t\t= 0,\n+\tXSC_QUEUE_TYPE_RDMA_MAD\t\t= 1,\n+\tXSC_QUEUE_TYPE_RAW\t\t= 2,\n+\tXSC_QUEUE_TYPE_VIRTIO_NET\t= 3,\n+\tXSC_QUEUE_TYPE_VIRTIO_BLK\t= 4,\n+\tXSC_QUEUE_TYPE_RAW_TPE\t\t= 5,\n+\tXSC_QUEUE_TYPE_RAW_TSO\t\t= 6,\n+\tXSC_QUEUE_TYPE_RAW_TX\t\t= 7,\n+\tXSC_QUEUE_TYPE_SNIFFER\t\t= 8,\n+\tXSC_QUEUE_TYPE_WORC\t\t= 9,\n+\tXSC_QUEUE_TYPE_INVALID\t\t= 0xFF,\n+};\n+\n+enum {\n+\tXSC_PERM_LOCAL_READ\t= 1 \u003c\u003c 0,\n+\tXSC_PERM_LOCAL_WRITE\t= 1 \u003c\u003c 1,\n+\tXSC_PERM_REMOTE_READ\t= 1 \u003c\u003c 2,\n+\tXSC_PERM_REMOTE_WRITE\t= 1 \u003c\u003c 3,\n+\tXSC_PERM_ATOMIC\t\t= 1 \u003c\u003c 4,\n+\tXSC_PERM_UMR_EN\t\t= 1 \u003c\u003c 7,\n+};\n+\n+enum {\n+\tXSC_DEV_CAP_FLAG_RC\t\t= 1LL \u003c\u003c 0,\n+\tXSC_DEV_CAP_FLAG_UC\t\t= 1LL \u003c\u003c 1,\n+\tXSC_DEV_CAP_FLAG_UD\t\t= 1LL \u003c\u003c 2,\n+\tXSC_DEV_CAP_FLAG_XRC\t\t= 1LL \u003c\u003c 3,\n+\tXSC_DEV_CAP_FLAG_SRQ\t\t= 1LL \u003c\u003c 6,\n+\tXSC_DEV_CAP_FLAG_BAD_PKEY_CNTR\t= 1LL \u003c\u003c 8,\n+\tXSC_DEV_CAP_FLAG_BAD_QKEY_CNTR\t= 1LL \u003c\u003c 9,\n+\tXSC_DEV_CAP_FLAG_APM\t\t= 1LL \u003c\u003c 17,\n+\tXSC_DEV_CAP_FLAG_ATOMIC\t\t= 1LL \u003c\u003c 18,\n+\tXSC_DEV_CAP_FLAG_ON_DMND_PG\t= 1LL \u003c\u003c 24,\n+\tXSC_DEV_CAP_FLAG_RESIZE_SRQ\t= 1LL \u003c\u003c 32,\n+\tXSC_DEV_CAP_FLAG_REMOTE_FENCE\t= 1LL \u003c\u003c 38,\n+\tXSC_DEV_CAP_FLAG_TLP_HINTS\t= 1LL \u003c\u003c 39,\n+\tXSC_DEV_CAP_FLAG_SIG_HAND_OVER\t= 1LL \u003c\u003c 40,\n+\tXSC_DEV_CAP_FLAG_DCT\t\t= 1LL \u003c\u003c 41,\n+\tXSC_DEV_CAP_FLAG_CMDIF_CSUM\t= 1LL \u003c\u003c 46,\n+};\n+\n+enum xsc_traffic_types {\n+\tXSC_TT_IPV4,\n+\tXSC_TT_IPV4_TCP,\n+\tXSC_TT_IPV4_UDP,\n+\tXSC_TT_IPV6,\n+\tXSC_TT_IPV6_TCP,\n+\tXSC_TT_IPV6_UDP,\n+\tXSC_TT_IPV4_IPSEC_AH,\n+\tXSC_TT_IPV6_IPSEC_AH,\n+\tXSC_TT_IPV4_IPSEC_ESP,\n+\tXSC_TT_IPV6_IPSEC_ESP,\n+\tXSC_TT_ANY,\n+\tXSC_NUM_TT,\n+};\n+\n+#define XSC_NUM_INDIR_TIRS XSC_NUM_TT\n+\n+enum {\n+\tXSC_L3_PROT_TYPE_IPV4\t= BIT(0),\n+\tXSC_L3_PROT_TYPE_IPV6\t= BIT(1),\n+};\n+\n+enum {\n+\tXSC_L4_PROT_TYPE_TCP\t= BIT(0),\n+\tXSC_L4_PROT_TYPE_UDP\t= BIT(1),\n+};\n+\n+struct xsc_tirc_config {\n+\tu8 l3_prot_type;\n+\tu8 l4_prot_type;\n+\tu32 rx_hash_fields;\n+};\n+\n+enum {\n+\tXSC_HASH_FUNC_XOR\t\t= 0,\n+\tXSC_HASH_FUNC_TOP\t\t= 1,\n+\tXSC_HASH_FUNC_TOP_SYM\t= 2,\n+\tXSC_HASH_FUNC_RSV\t\t= 3,\n+};\n+\n+#endif\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_hsi.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_hsi.h\nnew file mode 100644\nindex 0000000000000..aa2049f89bc9b\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_hsi.h\n@@ -0,0 +1,123 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef XSC_HSI_H\n+#define XSC_HSI_H\n+\n+#include \u003casm/byteorder.h\u003e\n+\n+#include \u003clinux/types.h\u003e\n+#include \u003clinux/bitops.h\u003e\n+\n+#define WR_LE_32(x, val)\t(x = __cpu_to_le32(val))\n+#define WR_LE_64(x, val)\t(x = __cpu_to_le64(val))\n+\n+#define WR_BE_32(x, val)\t(x = __cpu_to_be32(val))\n+\n+#define RD_LE_32(x)\t\t__le32_to_cpu(x)\n+#define RD_BE_32(x)\t\t__be32_to_cpu(x)\n+\n+struct xsc_msg_opcode {\n+\tu32\tsend;\n+\tu32\tsend_with_imm;\n+\tu32\trdma_write;\n+\tu32\trdma_write_with_imm;\n+\tu32\trdma_read;\n+\tu32\tatomic_cmp_and_swp;\n+\tu32\tatomic_fetch_and_add;\n+\tu32\tmasked_atomic_cmp_and_swp;\n+\tu32\tmasked_atomic_fetch_and_add;\n+\tu32\tmad;\n+};\n+\n+/* sw cqe opcode */\n+enum {\n+\tXSC_OPCODE_RDMA_REQ_SEND\t= 0,\n+\tXSC_OPCODE_RDMA_REQ_SEND_IMMDT\t= 1,\n+\tXSC_OPCODE_RDMA_RSP_RECV\t= 2,\n+\tXSC_OPCODE_RDMA_RSP_RECV_IMMDT\t= 3,\n+\tXSC_OPCODE_RDMA_REQ_WRITE\t= 4,\n+\tXSC_OPCODE_RDMA_REQ_WRITE_IMMDT\t= 5,\n+\tXSC_OPCODE_RDMA_RSP_WRITE_IMMDT\t= 6,\n+\tXSC_OPCODE_RDMA_REQ_READ\t= 7,\n+\tXSC_OPCODE_RDMA_REQ_ERROR\t= 8,\n+\tXSC_OPCODE_RDMA_RSP_ERROR\t= 9,\n+\tXSC_OPCODE_RDMA_CQE_ERROR\t= 10,\n+\tXSC_OPCODE_RDMA_MAD_REQ_SEND = 11,\n+\tXSC_OPCODE_RDMA_MAD_RSP_RECV = 12,\n+\tXSC_OPCODE_RDMA_CQE_RAW_SNF = 13,\n+};\n+\n+enum {\n+\tXSC_SEND_SEG_MAX\t\t= 32,\n+\tXSC_BASE_WQE_SHIFT\t\t= 4,\n+\tXSC_SEND_SEG_NUM\t\t= 4,\n+\tXSC_SEND_WQE_SHIFT\t\t= 6,\n+\tXSC_CTRL_SEG_NUM\t\t= 1,\n+\tXSC_RADDR_SEG_NUM\t\t= 1,\n+};\n+\n+enum {\n+\tXSC_RECV_SEG_MAX\t\t= 4,\n+\tXSC_RECV_SEG_NUM\t\t= 1,\n+\tXSC_RECV_WQE_SHIFT\t\t= 4,\n+};\n+\n+enum {\n+\tXSC_INLINE_SIZE_MAX\t\t= 15,\n+};\n+\n+#define XSC_MPT_MAP_EN\t\t0\n+\n+enum {\n+\tXSC_REQ\t\t= 0,\n+\tXSC_RSP\t\t= 1,\n+};\n+\n+enum {\n+\tXSC_WITHOUT_IMMDT\t= 0,\n+\tXSC_WITH_IMMDT\t\t= 1,\n+};\n+\n+/* CQE error codes */\n+enum {\n+\tXSC_ERR_CODE_NAK_RETRY\t\t\t= 0x40,\n+\tXSC_ERR_CODE_NAK_OPCODE\t\t\t= 0x41,\n+\tXSC_ERR_CODE_NAK_MR\t\t\t= 0x42,\n+\tXSC_ERR_CODE_NAK_OPERATION\t\t= 0x43,\n+\tXSC_ERR_CODE_NAK_RNR\t\t\t= 0x44,\n+\tXSC_ERR_CODE_LOCAL_MR\t\t\t= 0x45,\n+\tXSC_ERR_CODE_LOCAL_LEN\t\t\t= 0x46,\n+\tXSC_ERR_CODE_LOCAL_OPCODE\t\t= 0x47,\n+\tXSC_ERR_CODE_CQ_OVER_FLOW\t\t= 0x48,\n+\tXSC_ERR_CODE_STRG_ACC_GEN_CQE\t\t= 0x4c,\n+\tXSC_ERR_CODE_CQE_ACC\t\t\t= 0x4d,\n+\tXSC_ERR_CODE_FLUSH\t\t\t= 0x4e,\n+\tXSC_ERR_CODE_MALF_WQE_HOST\t\t= 0x50,\n+\tXSC_ERR_CODE_MALF_WQE_INFO\t\t= 0x51,\n+\tXSC_ERR_CODE_MR_NON_NAK\t\t\t= 0x52,\n+\tXSC_ERR_CODE_OPCODE_GEN_CQE\t\t= 0x61,\n+\tXSC_ERR_CODE_MANY_READ\t\t\t= 0x62,\n+\tXSC_ERR_CODE_LEN_GEN_CQE\t\t= 0x63,\n+\tXSC_ERR_CODE_MR\t\t\t\t= 0x65,\n+\tXSC_ERR_CODE_MR_GEN_CQE\t\t\t= 0x66,\n+\tXSC_ERR_CODE_OPERATION\t\t\t= 0x67,\n+\tXSC_ERR_CODE_MALF_WQE_INFO_GEN_NAK\t= 0x68,\n+};\n+\n+enum xsc_qp_state {\n+\tXSC_QP_STATE_RST\t\t= 0,\n+\tXSC_QP_STATE_INIT\t\t= 1,\n+\tXSC_QP_STATE_RTR\t\t= 2,\n+\tXSC_QP_STATE_RTS\t\t= 3,\n+\tXSC_QP_STATE_SQER\t\t= 4,\n+\tXSC_QP_STATE_SQD\t\t= 5,\n+\tXSC_QP_STATE_ERR\t\t= 6,\n+\tXSC_QP_STATE_SQ_DRAINING\t= 7,\n+\tXSC_QP_STATE_SUSPENDED\t\t= 9,\n+\tXSC_QP_NUM_STATE\n+};\n+\n+#endif /* XSC_HSI_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_reg.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_reg.h\nnew file mode 100644\nindex 0000000000000..babdb58024f5a\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_reg.h\n@@ -0,0 +1,28 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef XSC_REG_H\n+#define XSC_REG_H\n+#define CMDQM_HOST_REQ_BUF_BASE_H_ADDR_MEM_ADDR 0x0\n+#define CMDQM_HOST_REQ_BUF_BASE_L_ADDR_MEM_ADDR 0x4\n+#define CMDQM_HOST_REQ_PID_MEM_ADDR 0x8\n+#define CMDQM_HOST_REQ_CID_MEM_ADDR 0xc\n+#define CMDQM_HOST_RSP_BUF_BASE_H_ADDR_MEM_ADDR 0x10\n+#define CMDQM_HOST_RSP_BUF_BASE_L_ADDR_MEM_ADDR 0x14\n+#define CMDQM_HOST_RSP_PID_MEM_ADDR 0x18\n+#define CMDQM_HOST_RSP_CID_MEM_ADDR 0x1c\n+#define CMDQM_HOST_VF_ERR_STS_MEM_ADDR 0x20\n+#define CMDQM_VECTOR_ID_MEM_ADDR 0x24\n+\n+#define DB_CQ_FUNC_MEM_ADDR 0x8c4\n+#define DB_EQ_FUNC_MEM_ADDR 0x8c8\n+#define DB_CQ_CID_DIRECT_MEM_ADDR 0x8cc\n+#define TX_DB_FUNC_MEM_ADDR 0x8d0\n+#define RX_DB_FUNC_MEM_ADDR 0x8d4\n+\n+#define MDB_NUM\t64\n+#define QPM_TX_MDB_BASE_REG_ADDR 0xd00\n+\n+#endif\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/Kconfig b/drivers/net/ethernet/yunsilicon/xsc/net/Kconfig\nnew file mode 100644\nindex 0000000000000..7428b26295b66\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/Kconfig\n@@ -0,0 +1,17 @@\n+# SPDX-License-Identifier: GPL-2.0\n+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+# All rights reserved.\n+# Yunsilicon driver configuration\n+#\n+\n+config YUNSILICON_XSC_ETH\n+\ttristate \"Yunsilicon XSC ethernet driver\"\n+\tdepends on YUNSILICON_XSC_PCI\n+\tdepends on NET\n+\tselect PAGE_POOL\n+\thelp\n+\t This driver provides ethernet support for\n+\t Yunsilicon XSC devices.\n+\n+\t To compile this driver as a module, choose M here. The module\n+\t will be called xsc_eth.\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/Makefile b/drivers/net/ethernet/yunsilicon/xsc/net/Makefile\nnew file mode 100644\nindex 0000000000000..43d049e42e40d\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/Makefile\n@@ -0,0 +1,9 @@\n+# SPDX-License-Identifier: GPL-2.0\n+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+# All rights reserved.\n+\n+ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc\n+\n+obj-$(CONFIG_YUNSILICON_XSC_ETH) += xsc_eth.o\n+\n+xsc_eth-y := main.o xsc_eth_wq.o xsc_eth_txrx.o xsc_eth_tx.o xsc_eth_rx.o xsc_eth_stats.o\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/main.c b/drivers/net/ethernet/yunsilicon/xsc/net/main.c\nnew file mode 100644\nindex 0000000000000..d1f72043fd798\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/main.c\n@@ -0,0 +1,1724 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/netdevice.h\u003e\n+#include \u003clinux/etherdevice.h\u003e\n+#include \u003clinux/auxiliary_bus.h\u003e\n+#include \u003clinux/irq.h\u003e\n+\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_cmd_api.h\"\n+#include \"common/xsc_device.h\"\n+#include \"xsc_pp.h\"\n+#include \"xsc_eth_common.h\"\n+#include \"xsc_eth.h\"\n+#include \"xsc_eth_txrx.h\"\n+\n+static const struct xsc_tirc_config tirc_default_config[XSC_NUM_INDIR_TIRS] = {\n+\t[XSC_TT_IPV4] = {\n+\t\t\t\t.l3_prot_type = XSC_L3_PROT_TYPE_IPV4,\n+\t\t\t\t.l4_prot_type = 0,\n+\t\t\t\t.rx_hash_fields = XSC_HASH_IP,\n+\t},\n+\t[XSC_TT_IPV4_TCP] = {\n+\t\t\t\t.l3_prot_type = XSC_L3_PROT_TYPE_IPV4,\n+\t\t\t\t.l4_prot_type = XSC_L4_PROT_TYPE_TCP,\n+\t\t\t\t.rx_hash_fields = XSC_HASH_IP_PORTS,\n+\t},\n+\t[XSC_TT_IPV4_UDP] = {\n+\t\t\t\t.l3_prot_type = XSC_L3_PROT_TYPE_IPV4,\n+\t\t\t\t.l4_prot_type = XSC_L4_PROT_TYPE_UDP,\n+\t\t\t\t.rx_hash_fields = XSC_HASH_IP_PORTS,\n+\t},\n+\t[XSC_TT_IPV6] = {\n+\t\t\t\t.l3_prot_type = XSC_L3_PROT_TYPE_IPV6,\n+\t\t\t\t.l4_prot_type = 0,\n+\t\t\t\t.rx_hash_fields = XSC_HASH_IP6,\n+\t},\n+\t[XSC_TT_IPV6_TCP] = {\n+\t\t\t\t.l3_prot_type = XSC_L3_PROT_TYPE_IPV6,\n+\t\t\t\t.l4_prot_type = XSC_L4_PROT_TYPE_TCP,\n+\t\t\t\t.rx_hash_fields = XSC_HASH_IP6_PORTS,\n+\t},\n+\t[XSC_TT_IPV6_UDP] = {\n+\t\t\t\t.l3_prot_type = XSC_L3_PROT_TYPE_IPV6,\n+\t\t\t\t.l4_prot_type = XSC_L4_PROT_TYPE_UDP,\n+\t\t\t\t.rx_hash_fields = XSC_HASH_IP6_PORTS,\n+\t},\n+};\n+\n+static int xsc_get_max_num_channels(struct xsc_core_device *xdev)\n+{\n+\treturn min_t(int, xdev-\u003edev_res-\u003eeq_table.num_comp_vectors,\n+\t\t XSC_ETH_MAX_NUM_CHANNELS);\n+}\n+\n+static void xsc_build_rss_param(struct xsc_rss_params *rss_param)\n+{\n+\tenum xsc_traffic_types tt;\n+\n+\tfor (tt = 0; tt \u003c XSC_NUM_INDIR_TIRS; tt++) {\n+\t\trss_param-\u003erx_hash_fields[tt] =\n+\t\t\ttirc_default_config[tt].rx_hash_fields;\n+\t}\n+\trss_param-\u003erss_hash_tmpl = XSC_HASH_IP_PORTS | XSC_HASH_IP6_PORTS;\n+}\n+\n+static void xsc_eth_build_nic_params(struct xsc_adapter *adapter, u32 ch_num)\n+{\n+\tstruct xsc_eth_params *params = \u0026adapter-\u003enic_param;\n+\tstruct xsc_core_device *xdev = adapter-\u003exdev;\n+\n+\tparams-\u003emtu = SW_DEFAULT_MTU;\n+\n+\tparams-\u003ecomp_vectors = xdev-\u003edev_res-\u003eeq_table.num_comp_vectors;\n+\tparams-\u003emax_num_ch = ch_num;\n+\tparams-\u003enum_channels = ch_num;\n+\n+\txsc_build_rss_param(\u0026adapter-\u003erss_param);\n+}\n+\n+static int xsc_eth_netdev_init(struct xsc_adapter *adapter)\n+{\n+\tunsigned int node, nch;\n+\n+\tnch = adapter-\u003enic_param.max_num_ch;\n+\tnode = dev_to_node(adapter-\u003edev);\n+\tadapter-\u003etxq2sq = kcalloc_node(nch,\n+\t\t\t\t sizeof(*adapter-\u003etxq2sq),\n+\t\t\t\t GFP_KERNEL, node);\n+\tif (!adapter-\u003etxq2sq)\n+\t\treturn -ENOMEM;\n+\n+\tnetif_carrier_off(adapter-\u003enetdev);\n+\n+\treturn 0;\n+}\n+\n+static void xsc_eth_build_queue_param(struct xsc_adapter *adapter,\n+\t\t\t\t struct xsc_queue_attr *attr, u8 type)\n+{\n+\tstruct xsc_core_device *xdev = adapter-\u003exdev;\n+\n+\tif (adapter-\u003enic_param.sq_size == 0)\n+\t\tadapter-\u003enic_param.sq_size = BIT(xdev-\u003ecaps.log_max_qp_depth);\n+\tif (adapter-\u003enic_param.rq_size == 0)\n+\t\tadapter-\u003enic_param.rq_size = BIT(xdev-\u003ecaps.log_max_qp_depth);\n+\n+\tif (type == XSC_QUEUE_TYPE_RQCQ) {\n+\t\tattr-\u003eele_num = min_t(int, XSC_RQCQ_ELE_NUM,\n+\t\t\t\t xdev-\u003ecaps.max_cqes);\n+\t\tattr-\u003eele_size = XSC_RQCQ_ELE_SZ;\n+\t\tattr-\u003eele_log_size = order_base_2(XSC_RQCQ_ELE_SZ);\n+\t\tattr-\u003eq_log_size = order_base_2(attr-\u003eele_num);\n+\t} else if (type == XSC_QUEUE_TYPE_SQCQ) {\n+\t\tattr-\u003eele_num = min_t(int, XSC_SQCQ_ELE_NUM,\n+\t\t\t\t xdev-\u003ecaps.max_cqes);\n+\t\tattr-\u003eele_size = XSC_SQCQ_ELE_SZ;\n+\t\tattr-\u003eele_log_size = order_base_2(XSC_SQCQ_ELE_SZ);\n+\t\tattr-\u003eq_log_size = order_base_2(attr-\u003eele_num);\n+\t} else if (type == XSC_QUEUE_TYPE_RQ) {\n+\t\tattr-\u003eele_num = adapter-\u003enic_param.rq_size;\n+\t\tattr-\u003eele_size = xdev-\u003ecaps.recv_ds_num * XSC_RECV_WQE_DS;\n+\t\tattr-\u003eele_log_size = order_base_2(attr-\u003eele_size);\n+\t\tattr-\u003eq_log_size = order_base_2(attr-\u003eele_num);\n+\t} else if (type == XSC_QUEUE_TYPE_SQ) {\n+\t\tattr-\u003eele_num = adapter-\u003enic_param.sq_size;\n+\t\tattr-\u003eele_size = xdev-\u003ecaps.send_ds_num * XSC_SEND_WQE_DS;\n+\t\tattr-\u003eele_log_size = order_base_2(attr-\u003eele_size);\n+\t\tattr-\u003eq_log_size = order_base_2(attr-\u003eele_num);\n+\t}\n+}\n+\n+static u32 xsc_rx_get_linear_frag_sz(u32 mtu)\n+{\n+\tu32 byte_count = XSC_SW2HW_FRAG_SIZE(mtu);\n+\n+\treturn XSC_SKB_FRAG_SZ(byte_count);\n+}\n+\n+static bool xsc_rx_is_linear_skb(u32 mtu)\n+{\n+\tu32 linear_frag_sz = xsc_rx_get_linear_frag_sz(mtu);\n+\n+\treturn linear_frag_sz \u003c= PAGE_SIZE;\n+}\n+\n+static u32 xsc_get_rq_frag_info(struct xsc_rq_frags_info *frags_info, u32 mtu)\n+{\n+\tu32 byte_count = XSC_SW2HW_FRAG_SIZE(mtu);\n+\tint frag_stride;\n+\tint i = 0;\n+\n+\tif (xsc_rx_is_linear_skb(mtu)) {\n+\t\tfrag_stride = xsc_rx_get_linear_frag_sz(mtu);\n+\t\tfrag_stride = roundup_pow_of_two(frag_stride);\n+\n+\t\tfrags_info-\u003earr[0].frag_size = byte_count;\n+\t\tfrags_info-\u003earr[0].frag_stride = frag_stride;\n+\t\tfrags_info-\u003enum_frags = 1;\n+\t\tfrags_info-\u003ewqe_bulk = PAGE_SIZE / frag_stride;\n+\t\tfrags_info-\u003ewqe_bulk_min = frags_info-\u003ewqe_bulk;\n+\t\tgoto out;\n+\t}\n+\n+\tif (byte_count \u003c= DEFAULT_FRAG_SIZE) {\n+\t\tfrags_info-\u003earr[0].frag_size = DEFAULT_FRAG_SIZE;\n+\t\tfrags_info-\u003earr[0].frag_stride = DEFAULT_FRAG_SIZE;\n+\t\tfrags_info-\u003enum_frags = 1;\n+\t} else if (byte_count \u003c= PAGE_SIZE_4K) {\n+\t\tfrags_info-\u003earr[0].frag_size = PAGE_SIZE_4K;\n+\t\tfrags_info-\u003earr[0].frag_stride = PAGE_SIZE_4K;\n+\t\tfrags_info-\u003enum_frags = 1;\n+\t} else if (byte_count \u003c= (PAGE_SIZE_4K + DEFAULT_FRAG_SIZE)) {\n+\t\tif (PAGE_SIZE \u003c 2 * PAGE_SIZE_4K) {\n+\t\t\tfrags_info-\u003earr[0].frag_size = PAGE_SIZE_4K;\n+\t\t\tfrags_info-\u003earr[0].frag_stride = PAGE_SIZE_4K;\n+\t\t\tfrags_info-\u003earr[1].frag_size = PAGE_SIZE_4K;\n+\t\t\tfrags_info-\u003earr[1].frag_stride = PAGE_SIZE_4K;\n+\t\t\tfrags_info-\u003enum_frags = 2;\n+\t\t} else {\n+\t\t\tfrags_info-\u003earr[0].frag_size = 2 * PAGE_SIZE_4K;\n+\t\t\tfrags_info-\u003earr[0].frag_stride = 2 * PAGE_SIZE_4K;\n+\t\t\tfrags_info-\u003enum_frags = 1;\n+\t\t}\n+\t} else if (byte_count \u003c= 2 * PAGE_SIZE_4K) {\n+\t\tif (PAGE_SIZE \u003c 2 * PAGE_SIZE_4K) {\n+\t\t\tfrags_info-\u003earr[0].frag_size = PAGE_SIZE_4K;\n+\t\t\tfrags_info-\u003earr[0].frag_stride = PAGE_SIZE_4K;\n+\t\t\tfrags_info-\u003earr[1].frag_size = PAGE_SIZE_4K;\n+\t\t\tfrags_info-\u003earr[1].frag_stride = PAGE_SIZE_4K;\n+\t\t\tfrags_info-\u003enum_frags = 2;\n+\t\t} else {\n+\t\t\tfrags_info-\u003earr[0].frag_size = 2 * PAGE_SIZE_4K;\n+\t\t\tfrags_info-\u003earr[0].frag_stride = 2 * PAGE_SIZE_4K;\n+\t\t\tfrags_info-\u003enum_frags = 1;\n+\t\t}\n+\t} else {\n+\t\tif (PAGE_SIZE \u003c 4 * PAGE_SIZE_4K) {\n+\t\t\tfrags_info-\u003enum_frags =\n+\t\t\t\troundup(byte_count,\n+\t\t\t\t\tPAGE_SIZE_4K) / PAGE_SIZE_4K;\n+\t\t\tfor (i = 0; i \u003c frags_info-\u003enum_frags; i++) {\n+\t\t\t\tfrags_info-\u003earr[i].frag_size = PAGE_SIZE_4K;\n+\t\t\t\tfrags_info-\u003earr[i].frag_stride = PAGE_SIZE_4K;\n+\t\t\t}\n+\t\t} else {\n+\t\t\tfrags_info-\u003earr[0].frag_size = 4 * PAGE_SIZE_4K;\n+\t\t\tfrags_info-\u003earr[0].frag_stride = 4 * PAGE_SIZE_4K;\n+\t\t\tfrags_info-\u003enum_frags = 1;\n+\t\t}\n+\t}\n+\n+\tif (PAGE_SIZE \u003c= PAGE_SIZE_4K) {\n+\t\tfrags_info-\u003ewqe_bulk_min = 4;\n+\t\tfrags_info-\u003ewqe_bulk = max_t(u8, frags_info-\u003ewqe_bulk_min, 8);\n+\t} else if (PAGE_SIZE \u003c= 2 * PAGE_SIZE_4K) {\n+\t\tfrags_info-\u003ewqe_bulk = 2;\n+\t\tfrags_info-\u003ewqe_bulk_min = frags_info-\u003ewqe_bulk;\n+\t} else {\n+\t\tfrags_info-\u003ewqe_bulk =\n+\t\t\tPAGE_SIZE /\n+\t\t\t(frags_info-\u003enum_frags * frags_info-\u003earr[0].frag_size);\n+\t\tfrags_info-\u003ewqe_bulk_min = frags_info-\u003ewqe_bulk;\n+\t}\n+\n+out:\n+\tfrags_info-\u003elog_num_frags = order_base_2(frags_info-\u003enum_frags);\n+\n+\treturn frags_info-\u003enum_frags * frags_info-\u003earr[0].frag_size;\n+}\n+\n+static void xsc_build_rq_frags_info(struct xsc_queue_attr *attr,\n+\t\t\t\t struct xsc_rq_frags_info *frags_info,\n+\t\t\t\t struct xsc_eth_params *params)\n+{\n+\tparams-\u003erq_frags_size = xsc_get_rq_frag_info(frags_info, params-\u003emtu);\n+\tfrags_info-\u003efrags_max_num = attr-\u003eele_size / XSC_RECV_WQE_DS;\n+}\n+\n+static void xsc_eth_build_channel_param(struct xsc_adapter *adapter,\n+\t\t\t\t\tstruct xsc_channel_param *chl_param)\n+{\n+\txsc_eth_build_queue_param(adapter, \u0026chl_param-\u003erqcq_param.cq_attr,\n+\t\t\t\t XSC_QUEUE_TYPE_RQCQ);\n+\tchl_param-\u003erqcq_param.wq.buf_numa_node = dev_to_node(adapter-\u003edev);\n+\n+\txsc_eth_build_queue_param(adapter, \u0026chl_param-\u003esqcq_param.cq_attr,\n+\t\t\t\t XSC_QUEUE_TYPE_SQCQ);\n+\tchl_param-\u003esqcq_param.wq.buf_numa_node = dev_to_node(adapter-\u003edev);\n+\n+\txsc_eth_build_queue_param(adapter, \u0026chl_param-\u003esq_param.sq_attr,\n+\t\t\t\t XSC_QUEUE_TYPE_SQ);\n+\tchl_param-\u003esq_param.wq.buf_numa_node = dev_to_node(adapter-\u003edev);\n+\n+\txsc_eth_build_queue_param(adapter, \u0026chl_param-\u003erq_param.rq_attr,\n+\t\t\t\t XSC_QUEUE_TYPE_RQ);\n+\tchl_param-\u003erq_param.wq.buf_numa_node = dev_to_node(adapter-\u003edev);\n+\n+\txsc_build_rq_frags_info(\u0026chl_param-\u003erq_param.rq_attr,\n+\t\t\t\t\u0026chl_param-\u003erq_param.frags_info,\n+\t\t\t\t\u0026adapter-\u003enic_param);\n+}\n+\n+static void xsc_eth_cq_error_event(struct xsc_core_cq *xcq,\n+\t\t\t\t enum xsc_event event)\n+{\n+\tstruct xsc_cq *xsc_cq = container_of(xcq, struct xsc_cq, xcq);\n+\tstruct xsc_core_device *xdev = xsc_cq-\u003exdev;\n+\n+\tif (event != XSC_EVENT_TYPE_CQ_ERROR) {\n+\t\tnetdev_err(xsc_dev_to_netdev(xdev),\n+\t\t\t \"Unexpected event type %d on CQ %06x\\n\",\n+\t\t\t event, xcq-\u003ecqn);\n+\t\treturn;\n+\t}\n+\n+\tnetdev_err(xsc_dev_to_netdev(xdev),\n+\t\t \"Eth catch CQ ERROR: %x, cqn: %d\\n\", event, xcq-\u003ecqn);\n+}\n+\n+static void xsc_eth_completion_event(struct xsc_core_cq *xcq)\n+{\n+\tstruct xsc_cq *cq = container_of(xcq, struct xsc_cq, xcq);\n+\tstruct xsc_core_device *xdev = cq-\u003exdev;\n+\tstruct xsc_rq *rq = NULL;\n+\n+\tif (unlikely(!cq-\u003echannel)) {\n+\t\tnetdev_err(xsc_dev_to_netdev(xdev), \"cq%d-\u003echannel is null\\n\",\n+\t\t\t xcq-\u003ecqn);\n+\t\treturn;\n+\t}\n+\n+\trq = \u0026cq-\u003echannel-\u003eqp.rq;\n+\n+\tif (!test_bit(XSC_ETH_RQ_STATE_ENABLED, \u0026rq-\u003estate))\n+\t\tnetdev_err(xsc_dev_to_netdev(xdev),\n+\t\t\t \"ch%d_cq%d, napi_flag=0x%lx\\n\",\n+\t\t\t cq-\u003echannel-\u003echl_idx, xcq-\u003ecqn,\n+\t\t\t cq-\u003enapi-\u003estate);\n+\n+\tnapi_schedule(cq-\u003enapi);\n+\tcq-\u003eevent_ctr++;\n+}\n+\n+static int xsc_eth_alloc_cq(struct xsc_channel *c, struct xsc_cq *pcq,\n+\t\t\t struct xsc_cq_param *pcq_param)\n+{\n+\tu8 ele_log_size = pcq_param-\u003ecq_attr.ele_log_size;\n+\tstruct xsc_core_device *xdev = c-\u003eadapter-\u003exdev;\n+\tu8 q_log_size = pcq_param-\u003ecq_attr.q_log_size;\n+\tstruct xsc_core_cq *core_cq = \u0026pcq-\u003excq;\n+\tint ret;\n+\tu32 i;\n+\n+\tpcq_param-\u003ewq.db_numa_node = cpu_to_node(c-\u003ecpu);\n+\tpcq_param-\u003ewq.buf_numa_node = cpu_to_node(c-\u003ecpu);\n+\n+\tret = xsc_eth_cqwq_create(xdev, \u0026pcq_param-\u003ewq,\n+\t\t\t\t q_log_size, ele_log_size, \u0026pcq-\u003ewq,\n+\t\t\t\t \u0026pcq-\u003ewq_ctrl);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tcore_cq-\u003ecqe_sz = pcq_param-\u003ecq_attr.ele_num;\n+\tcore_cq-\u003ecomp = xsc_eth_completion_event;\n+\tcore_cq-\u003eevent = xsc_eth_cq_error_event;\n+\tcore_cq-\u003evector = c-\u003echl_idx;\n+\n+\tfor (i = 0; i \u003c xsc_cqwq_get_size(\u0026pcq-\u003ewq); i++) {\n+\t\tstruct xsc_cqe *cqe = xsc_cqwq_get_wqe(\u0026pcq-\u003ewq, i);\n+\n+\t\tcqe-\u003edata1 |= cpu_to_le32(XSC_CQE_OWNER);\n+\t}\n+\tpcq-\u003exdev = xdev;\n+\n+\treturn ret;\n+}\n+\n+static int xsc_eth_set_cq(struct xsc_channel *c,\n+\t\t\t struct xsc_cq *pcq,\n+\t\t\t struct xsc_cq_param *pcq_param)\n+{\n+\tstruct xsc_core_device *xdev = c-\u003eadapter-\u003exdev;\n+\tstruct xsc_create_cq_mbox_in *in;\n+\tunsigned int hw_npages;\n+\tunsigned int irqn;\n+\tint inlen;\n+\tu32 eqn;\n+\tint ret;\n+\n+\thw_npages = DIV_ROUND_UP(pcq-\u003ewq_ctrl.buf.size, PAGE_SIZE_4K);\n+\t/*mbox size + pas size*/\n+\tinlen = sizeof(struct xsc_create_cq_mbox_in) +\n+\t\tsizeof(__be64) * hw_npages;\n+\n+\tin = kvzalloc(inlen, GFP_KERNEL);\n+\tif (!in)\n+\t\treturn -ENOMEM;\n+\n+\t/*construct param of in struct*/\n+\tret = xsc_core_vector2eqn(xdev, c-\u003echl_idx, \u0026eqn, \u0026irqn);\n+\tif (ret)\n+\t\tgoto err_free;\n+\n+\tin-\u003ectx.eqn = cpu_to_be16(eqn);\n+\tin-\u003ectx.log_cq_sz = pcq_param-\u003ecq_attr.q_log_size;\n+\tin-\u003ectx.pa_num = cpu_to_be16(hw_npages);\n+\tin-\u003ectx.glb_func_id = cpu_to_be16(xdev-\u003eglb_func_id);\n+\n+\txsc_core_fill_page_frag_array(\u0026pcq-\u003ewq_ctrl.buf,\n+\t\t\t\t \u0026in-\u003epas[0],\n+\t\t\t\t hw_npages);\n+\n+\tret = xsc_core_eth_create_cq(c-\u003eadapter-\u003exdev, \u0026pcq-\u003excq, in, inlen);\n+\tif (ret)\n+\t\tgoto err_free;\n+\tpcq-\u003excq.irqn = irqn;\n+\tpcq-\u003excq.eq = xsc_core_eq_get(xdev, pcq-\u003excq.vector);\n+\treturn 0;\n+\n+err_free:\n+\tkvfree(in);\n+\treturn ret;\n+}\n+\n+static void xsc_eth_free_cq(struct xsc_cq *cq)\n+{\n+\txsc_eth_wq_destroy(\u0026cq-\u003ewq_ctrl);\n+}\n+\n+static int xsc_eth_open_cq(struct xsc_channel *c,\n+\t\t\t struct xsc_cq *pcq,\n+\t\t\t struct xsc_cq_param *pcq_param)\n+{\n+\tint ret;\n+\n+\tret = xsc_eth_alloc_cq(c, pcq, pcq_param);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tret = xsc_eth_set_cq(c, pcq, pcq_param);\n+\tif (ret)\n+\t\tgoto err_free_cq;\n+\n+\txsc_cq_notify_hw_rearm(pcq);\n+\n+\tpcq-\u003enapi = \u0026c-\u003enapi;\n+\tpcq-\u003echannel = c;\n+\n+\treturn 0;\n+\n+err_free_cq:\n+\txsc_eth_free_cq(pcq);\n+\treturn ret;\n+}\n+\n+static int xsc_eth_close_cq(struct xsc_channel *c, struct xsc_cq *pcq)\n+{\n+\tstruct xsc_core_device *xdev = c-\u003eadapter-\u003exdev;\n+\tint ret;\n+\n+\tret = xsc_core_eth_destroy_cq(xdev, \u0026pcq-\u003excq);\n+\tif (ret) {\n+\t\tnetdev_err(xsc_dev_to_netdev(xdev), \"failed to close ch%d cq%d, ret=%d\\n\",\n+\t\t\t c-\u003echl_idx, pcq-\u003excq.cqn, ret);\n+\t\treturn ret;\n+\t}\n+\n+\txsc_eth_free_cq(pcq);\n+\n+\treturn 0;\n+}\n+\n+static void xsc_free_qp_sq_db(struct xsc_sq *sq)\n+{\n+\tkvfree(sq-\u003edb.wqe_info);\n+\tkvfree(sq-\u003edb.dma_fifo);\n+}\n+\n+static void xsc_free_qp_sq(struct xsc_sq *sq)\n+{\n+\txsc_free_qp_sq_db(sq);\n+\txsc_eth_wq_destroy(\u0026sq-\u003ewq_ctrl);\n+}\n+\n+static int xsc_eth_alloc_qp_sq_db(struct xsc_sq *sq, int numa)\n+{\n+\tstruct xsc_core_device *xdev = sq-\u003ecq.xdev;\n+\tint wq_sz;\n+\tint df_sz;\n+\n+\twq_sz = xsc_wq_cyc_get_size(\u0026sq-\u003ewq);\n+\tdf_sz = wq_sz * xdev-\u003ecaps.send_ds_num;\n+\tsq-\u003edb.dma_fifo = kvzalloc_node(array_size(df_sz,\n+\t\t\t\t\t\t sizeof(*sq-\u003edb.dma_fifo)),\n+\t\t\t\t\tGFP_KERNEL,\n+\t\t\t\t\tnuma);\n+\tsq-\u003edb.wqe_info = kvzalloc_node(array_size(wq_sz,\n+\t\t\t\t\t\t sizeof(*sq-\u003edb.wqe_info)),\n+\t\t\t\t\tGFP_KERNEL,\n+\t\t\t\t\tnuma);\n+\n+\tif (!sq-\u003edb.dma_fifo || !sq-\u003edb.wqe_info) {\n+\t\txsc_free_qp_sq_db(sq);\n+\t\treturn -ENOMEM;\n+\t}\n+\n+\tsq-\u003edma_fifo_mask = df_sz - 1;\n+\n+\treturn 0;\n+}\n+\n+static void xsc_eth_qp_event(struct xsc_core_qp *qp, int type)\n+{\n+\tstruct xsc_core_device *xdev;\n+\tstruct xsc_rq *rq;\n+\tstruct xsc_sq *sq;\n+\n+\tif (qp-\u003eeth_queue_type == XSC_RES_RQ) {\n+\t\trq = container_of(qp, struct xsc_rq, cqp);\n+\t\txdev = rq-\u003ecq.xdev;\n+\t} else if (qp-\u003eeth_queue_type == XSC_RES_SQ) {\n+\t\tsq = container_of(qp, struct xsc_sq, cqp);\n+\t\txdev = sq-\u003ecq.xdev;\n+\t} else {\n+\t\tpr_err(\"%s:Unknown eth qp type %d\\n\", __func__, type);\n+\t\treturn;\n+\t}\n+\n+\tswitch (type) {\n+\tcase XSC_EVENT_TYPE_WQ_CATAS_ERROR:\n+\tcase XSC_EVENT_TYPE_WQ_INVAL_REQ_ERROR:\n+\tcase XSC_EVENT_TYPE_WQ_ACCESS_ERROR:\n+\t\tnetdev_err(xsc_dev_to_netdev(xdev),\n+\t\t\t \"%s:Async event %x on QP %d\\n\",\n+\t\t\t __func__, type, qp-\u003eqpn);\n+\t\tbreak;\n+\tdefault:\n+\t\tnetdev_err(xsc_dev_to_netdev(xdev),\n+\t\t\t \"%s: Unexpected event type %d on QP %d\\n\",\n+\t\t\t __func__, type, qp-\u003eqpn);\n+\t\treturn;\n+\t}\n+}\n+\n+static int xsc_eth_open_qp_sq(struct xsc_channel *c,\n+\t\t\t struct xsc_sq *psq,\n+\t\t\t struct xsc_sq_param *psq_param)\n+{\n+\tu8 ele_log_size = psq_param-\u003esq_attr.ele_log_size;\n+\tu8 q_log_size = psq_param-\u003esq_attr.q_log_size;\n+\tstruct xsc_modify_raw_qp_mbox_in *modify_in;\n+\tstruct xsc_channel_stats *channel_stats;\n+\tstruct xsc_create_qp_mbox_in *in;\n+\tstruct xsc_core_device *xdev;\n+\tstruct xsc_adapter *adapter;\n+\tstruct xsc_stats *stats;\n+\tunsigned int hw_npages;\n+\tint inlen;\n+\tint ret;\n+\n+\tadapter = c-\u003eadapter;\n+\txdev = adapter-\u003exdev;\n+\tstats = adapter-\u003estats;\n+\tchannel_stats = \u0026stats-\u003echannel_stats[c-\u003echl_idx];\n+\tpsq-\u003estats = \u0026channel_stats-\u003esq;\n+\tpsq_param-\u003ewq.db_numa_node = cpu_to_node(c-\u003ecpu);\n+\n+\tret = xsc_eth_wq_cyc_create(xdev, \u0026psq_param-\u003ewq,\n+\t\t\t\t q_log_size, ele_log_size, \u0026psq-\u003ewq,\n+\t\t\t\t \u0026psq-\u003ewq_ctrl);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\thw_npages = DIV_ROUND_UP(psq-\u003ewq_ctrl.buf.size, PAGE_SIZE_4K);\n+\tinlen = sizeof(struct xsc_create_qp_mbox_in) +\n+\t\tsizeof(__be64) * hw_npages;\n+\n+\tin = kvzalloc(inlen, GFP_KERNEL);\n+\tif (!in) {\n+\t\tret = -ENOMEM;\n+\t\tgoto err_sq_wq_destroy;\n+\t}\n+\tin-\u003ereq.input_qpn = cpu_to_be16(XSC_QPN_SQN_STUB); /*no use for eth*/\n+\tin-\u003ereq.qp_type = XSC_QUEUE_TYPE_RAW_TSO; /*default sq is tso qp*/\n+\tin-\u003ereq.log_sq_sz = ilog2(xdev-\u003ecaps.send_ds_num) + q_log_size;\n+\tin-\u003ereq.pa_num = cpu_to_be16(hw_npages);\n+\tin-\u003ereq.cqn_send = cpu_to_be16(psq-\u003ecq.xcq.cqn);\n+\tin-\u003ereq.cqn_recv = in-\u003ereq.cqn_send;\n+\tin-\u003ereq.glb_funcid = cpu_to_be16(xdev-\u003eglb_func_id);\n+\n+\txsc_core_fill_page_frag_array(\u0026psq-\u003ewq_ctrl.buf,\n+\t\t\t\t \u0026in-\u003ereq.pas[0], hw_npages);\n+\n+\tret = xsc_core_eth_create_qp(xdev, in, inlen, \u0026psq-\u003esqn);\n+\tif (ret)\n+\t\tgoto err_sq_in_destroy;\n+\n+\tpsq-\u003ecqp.qpn = psq-\u003esqn;\n+\tpsq-\u003ecqp.event = xsc_eth_qp_event;\n+\tpsq-\u003ecqp.eth_queue_type = XSC_RES_SQ;\n+\n+\tret = xsc_core_create_resource_common(xdev, \u0026psq-\u003ecqp);\n+\tif (ret) {\n+\t\tnetdev_err(adapter-\u003enetdev, \"%s:error qp:%d errno:%d\\n\",\n+\t\t\t __func__, psq-\u003esqn, ret);\n+\t\tgoto err_sq_destroy;\n+\t}\n+\n+\tpsq-\u003echannel = c;\n+\tpsq-\u003ech_ix = c-\u003echl_idx;\n+\tpsq-\u003etxq_ix = psq-\u003ech_ix;\n+\n+\t/*need to querify from hardware*/\n+\tpsq-\u003ehw_mtu = XSC_ETH_HW_MTU_SEND;\n+\tpsq-\u003estop_room = 1;\n+\n+\tret = xsc_eth_alloc_qp_sq_db(psq, psq_param-\u003ewq.db_numa_node);\n+\tif (ret)\n+\t\tgoto err_sq_common_destroy;\n+\n+\tinlen = sizeof(struct xsc_modify_raw_qp_mbox_in);\n+\tmodify_in = kvzalloc(inlen, GFP_KERNEL);\n+\tif (!modify_in) {\n+\t\tret = -ENOMEM;\n+\t\tgoto err_sq_common_destroy;\n+\t}\n+\n+\tmodify_in-\u003ereq.qp_out_port = xdev-\u003epf_id;\n+\tmodify_in-\u003epcie_no = xdev-\u003epcie_no;\n+\tmodify_in-\u003ereq.qpn = cpu_to_be16((u16)(psq-\u003esqn));\n+\tmodify_in-\u003ereq.func_id = cpu_to_be16(xdev-\u003eglb_func_id);\n+\tmodify_in-\u003ereq.dma_direct = XSC_DMA_DIR_TO_MAC;\n+\tmodify_in-\u003ereq.prio = 0; /* single traffic class */\n+\tret = xsc_core_eth_modify_raw_qp(xdev, modify_in);\n+\tif (ret)\n+\t\tgoto err_sq_modify_in_destroy;\n+\n+\tkvfree(modify_in);\n+\tkvfree(in);\n+\n+\treturn 0;\n+\n+err_sq_modify_in_destroy:\n+\tkvfree(modify_in);\n+\n+err_sq_common_destroy:\n+\txsc_core_destroy_resource_common(xdev, \u0026psq-\u003ecqp);\n+\n+err_sq_destroy:\n+\txsc_core_eth_destroy_qp(xdev, psq-\u003ecqp.qpn);\n+\n+err_sq_in_destroy:\n+\tkvfree(in);\n+\n+err_sq_wq_destroy:\n+\txsc_eth_wq_destroy(\u0026psq-\u003ewq_ctrl);\n+\treturn ret;\n+}\n+\n+static int xsc_eth_close_qp_sq(struct xsc_channel *c, struct xsc_sq *psq)\n+{\n+\tstruct xsc_core_device *xdev = c-\u003eadapter-\u003exdev;\n+\tint ret;\n+\n+\txsc_core_destroy_resource_common(xdev, \u0026psq-\u003ecqp);\n+\n+\tret = xsc_core_eth_destroy_qp(xdev, psq-\u003ecqp.qpn);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\txsc_free_qp_sq(psq);\n+\n+\treturn 0;\n+}\n+\n+static int xsc_eth_open_channel(struct xsc_adapter *adapter,\n+\t\t\t\tint idx,\n+\t\t\t\tstruct xsc_channel *c,\n+\t\t\t\tstruct xsc_channel_param *chl_param)\n+{\n+\tstruct xsc_core_device *xdev = adapter-\u003exdev;\n+\tstruct net_device *netdev = adapter-\u003enetdev;\n+\tconst struct cpumask *aff;\n+\tunsigned int irqn;\n+\tu32 eqn;\n+\tint ret;\n+\n+\tc-\u003eadapter = adapter;\n+\tc-\u003enetdev = adapter-\u003enetdev;\n+\tc-\u003echl_idx = idx;\n+\n+\tif (xdev-\u003ecaps.msix_enable) {\n+\t\tret = xsc_core_vector2eqn(xdev, c-\u003echl_idx, \u0026eqn, \u0026irqn);\n+\t\tif (ret)\n+\t\t\tgoto err_out;\n+\t\taff = irq_get_affinity_mask(irqn);\n+\t\tc-\u003eaff_mask = aff;\n+\t\tc-\u003ecpu = cpumask_first(aff);\n+\t}\n+\n+\tret = xsc_eth_open_cq(c, \u0026c-\u003eqp.rq.cq, \u0026chl_param-\u003erqcq_param);\n+\tif (ret)\n+\t\tgoto err_out;\n+\n+\tret = xsc_eth_open_cq(c, \u0026c-\u003eqp.sq.cq, \u0026chl_param-\u003esqcq_param);\n+\tif (ret)\n+\t\tgoto err_close_rq_cq;\n+\n+\tret = xsc_eth_open_qp_sq(c, \u0026c-\u003eqp.sq, \u0026chl_param-\u003esq_param);\n+\tif (ret)\n+\t\tgoto err_close_sq_cq;\n+\n+\tnetif_napi_add(netdev, \u0026c-\u003enapi, xsc_eth_napi_poll);\n+\n+\tnetdev_dbg(adapter-\u003enetdev, \"open channel%d ok\\n\", idx);\n+\treturn 0;\n+\n+err_close_sq_cq:\n+\txsc_eth_close_cq(c, \u0026c-\u003eqp.sq.cq);\n+err_close_rq_cq:\n+\txsc_eth_close_cq(c, \u0026c-\u003eqp.rq.cq);\n+err_out:\n+\tnetdev_err(adapter-\u003enetdev, \"failed to open channel: ch%d, err=%d\\n\",\n+\t\t idx, ret);\n+\treturn ret;\n+}\n+\n+static int xsc_eth_modify_qps_channel(struct xsc_adapter *adapter,\n+\t\t\t\t struct xsc_channel *c)\n+{\n+\tint ret;\n+\n+\tc-\u003eqp.rq.post_wqes(\u0026c-\u003eqp.rq);\n+\tret = xsc_core_eth_modify_qp_status(adapter-\u003exdev, c-\u003eqp.rq.rqn,\n+\t\t\t\t\t XSC_CMD_OP_RTR2RTS_QP);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\treturn xsc_core_eth_modify_qp_status(adapter-\u003exdev, c-\u003eqp.sq.sqn,\n+\t\t\t\t\t XSC_CMD_OP_RTR2RTS_QP);\n+}\n+\n+static int xsc_eth_modify_qps(struct xsc_adapter *adapter,\n+\t\t\t struct xsc_eth_channels *chls)\n+{\n+\tint ret;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c chls-\u003enum_chl; i++) {\n+\t\tstruct xsc_channel *c = \u0026chls-\u003ec[i];\n+\n+\t\tret = xsc_eth_modify_qps_channel(adapter, c);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static void xsc_eth_init_frags_partition(struct xsc_rq *rq)\n+{\n+\tstruct xsc_wqe_frag_info next_frag = {};\n+\tstruct xsc_wqe_frag_info *prev;\n+\tint i;\n+\n+\tnext_frag.di = \u0026rq-\u003ewqe.di[0];\n+\tnext_frag.offset = 0;\n+\tprev = NULL;\n+\n+\tfor (i = 0; i \u003c xsc_wq_cyc_get_size(\u0026rq-\u003ewqe.wq); i++) {\n+\t\tstruct xsc_rq_frag_info *frag_info = \u0026rq-\u003ewqe.info.arr[0];\n+\t\tstruct xsc_wqe_frag_info *frag =\n+\t\t\t\u0026rq-\u003ewqe.frags[i \u003c\u003c rq-\u003ewqe.info.log_num_frags];\n+\t\tint f;\n+\n+\t\tfor (f = 0; f \u003c rq-\u003ewqe.info.num_frags; f++, frag++) {\n+\t\t\tif (next_frag.offset + frag_info[f].frag_stride \u003e\n+\t\t\t\tXSC_RX_FRAG_SZ) {\n+\t\t\t\tnext_frag.di++;\n+\t\t\t\tnext_frag.offset = 0;\n+\t\t\t\tif (prev)\n+\t\t\t\t\tprev-\u003elast_in_page = 1;\n+\t\t\t}\n+\t\t\t*frag = next_frag;\n+\n+\t\t\t/* prepare next */\n+\t\t\tnext_frag.offset += frag_info[f].frag_stride;\n+\t\t\tprev = frag;\n+\t\t}\n+\t}\n+\n+\tif (prev)\n+\t\tprev-\u003elast_in_page = 1;\n+}\n+\n+static int xsc_eth_init_di_list(struct xsc_rq *rq, u32 wq_sz, int cpu)\n+{\n+\tint len = wq_sz \u003c\u003c rq-\u003ewqe.info.log_num_frags;\n+\n+\trq-\u003ewqe.di = kvzalloc_node(array_size(len, sizeof(*rq-\u003ewqe.di)),\n+\t\t\t\t GFP_KERNEL, cpu_to_node(cpu));\n+\tif (!rq-\u003ewqe.di)\n+\t\treturn -ENOMEM;\n+\n+\txsc_eth_init_frags_partition(rq);\n+\n+\treturn 0;\n+}\n+\n+static void xsc_eth_free_di_list(struct xsc_rq *rq)\n+{\n+\tkvfree(rq-\u003ewqe.di);\n+}\n+\n+static int xsc_eth_alloc_rq(struct xsc_channel *c,\n+\t\t\t struct xsc_rq *prq,\n+\t\t\t struct xsc_rq_param *prq_param)\n+{\n+\tu8 ele_log_size = prq_param-\u003erq_attr.ele_log_size;\n+\tu8 q_log_size = prq_param-\u003erq_attr.q_log_size;\n+\tstruct page_pool_params pagepool_params = {0};\n+\tstruct xsc_adapter *adapter = c-\u003eadapter;\n+\tstruct xsc_channel_stats *channel_stats;\n+\tu32 pool_size = 1 \u003c\u003c q_log_size;\n+\tstruct xsc_stats *stats;\n+\tint ret = 0;\n+\tu32 wq_sz;\n+\tint i, f;\n+\n+\tstats = c-\u003eadapter-\u003estats;\n+\tchannel_stats = \u0026stats-\u003echannel_stats[c-\u003echl_idx];\n+\tprq-\u003estats = \u0026channel_stats-\u003erq;\n+\tprq_param-\u003ewq.db_numa_node = cpu_to_node(c-\u003ecpu);\n+\n+\tret = xsc_eth_wq_cyc_create(c-\u003eadapter-\u003exdev, \u0026prq_param-\u003ewq,\n+\t\t\t\t q_log_size, ele_log_size, \u0026prq-\u003ewqe.wq,\n+\t\t\t\t \u0026prq-\u003ewq_ctrl);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\twq_sz = xsc_wq_cyc_get_size(\u0026prq-\u003ewqe.wq);\n+\n+\tprq-\u003ewqe.info = prq_param-\u003efrags_info;\n+\tprq-\u003ewqe.frags =\n+\t\tkvzalloc_node(array_size((wq_sz \u003c\u003c prq-\u003ewqe.info.log_num_frags),\n+\t\t\t\t\t sizeof(*prq-\u003ewqe.frags)),\n+\t\t\t GFP_KERNEL,\n+\t\t\t cpu_to_node(c-\u003ecpu));\n+\tif (!prq-\u003ewqe.frags) {\n+\t\tret = -ENOMEM;\n+\t\tgoto err_destroy_wq;\n+\t}\n+\n+\tret = xsc_eth_init_di_list(prq, wq_sz, c-\u003ecpu);\n+\tif (ret)\n+\t\tgoto err_free_frags;\n+\n+\tprq-\u003ebuff.map_dir = DMA_FROM_DEVICE;\n+\n+\t/* Create a page_pool and register it with rxq */\n+\tpool_size = wq_sz \u003c\u003c prq-\u003ewqe.info.log_num_frags;\n+\tpagepool_params.order = XSC_RX_FRAG_SZ_ORDER;\n+\tpagepool_params.flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV;\n+\tpagepool_params.pool_size = pool_size;\n+\tpagepool_params.nid = cpu_to_node(c-\u003ecpu);\n+\tpagepool_params.dev = c-\u003eadapter-\u003edev;\n+\tpagepool_params.dma_dir\t= prq-\u003ebuff.map_dir;\n+\tpagepool_params.max_len = PAGE_SIZE;\n+\n+\tprq-\u003epage_pool = page_pool_create(\u0026pagepool_params);\n+\tif (IS_ERR(prq-\u003epage_pool)) {\n+\t\tret = PTR_ERR(prq-\u003epage_pool);\n+\t\tprq-\u003epage_pool = NULL;\n+\t\tgoto err_free_di_list;\n+\t}\n+\n+\tif (c-\u003echl_idx == 0)\n+\t\tnetdev_dbg(adapter-\u003enetdev,\n+\t\t\t \"page pool: size=%d, cpu=%d, pool_numa=%d, mtu=%d, wqe_numa=%d\\n\",\n+\t\t\t pool_size, c-\u003ecpu, pagepool_params.nid,\n+\t\t\t adapter-\u003enic_param.mtu,\n+\t\t\t prq_param-\u003ewq.buf_numa_node);\n+\n+\tfor (i = 0; i \u003c wq_sz; i++) {\n+\t\tstruct xsc_eth_rx_wqe_cyc *wqe =\n+\t\t\txsc_wq_cyc_get_wqe(\u0026prq-\u003ewqe.wq, i);\n+\n+\t\tfor (f = 0; f \u003c prq-\u003ewqe.info.num_frags; f++) {\n+\t\t\tu32 frag_size = prq-\u003ewqe.info.arr[f].frag_size;\n+\t\t\tu32 val = 0;\n+\n+\t\t\tval |= FIELD_PREP(XSC_WQE_DATA_SEG_SEG_LEN_MASK,\n+\t\t\t\t\t frag_size);\n+\t\t\twqe-\u003edata[f].data0 = cpu_to_le32(val);\n+\t\t\twqe-\u003edata[f].mkey = cpu_to_le32(XSC_INVALID_LKEY);\n+\t\t}\n+\n+\t\tfor (; f \u003c prq-\u003ewqe.info.frags_max_num; f++) {\n+\t\t\twqe-\u003edata[f].data0 = 0;\n+\t\t\twqe-\u003edata[f].mkey = cpu_to_le32(XSC_INVALID_LKEY);\n+\t\t\twqe-\u003edata[f].va = 0;\n+\t\t}\n+\t}\n+\n+\tprq-\u003epost_wqes = xsc_eth_post_rx_wqes;\n+\tprq-\u003ehandle_rx_cqe = xsc_eth_handle_rx_cqe;\n+\tprq-\u003edealloc_wqe = xsc_eth_dealloc_rx_wqe;\n+\tprq-\u003ewqe.skb_from_cqe = xsc_rx_is_linear_skb(adapter-\u003enic_param.mtu) ?\n+\t\t\t\t\txsc_skb_from_cqe_linear :\n+\t\t\t\t\txsc_skb_from_cqe_nonlinear;\n+\tprq-\u003eix = c-\u003echl_idx;\n+\tprq-\u003efrags_sz = adapter-\u003enic_param.rq_frags_size;\n+\n+\treturn 0;\n+\n+err_free_di_list:\n+\txsc_eth_free_di_list(prq);\n+err_free_frags:\n+\tkvfree(prq-\u003ewqe.frags);\n+err_destroy_wq:\n+\txsc_eth_wq_destroy(\u0026prq-\u003ewq_ctrl);\n+\treturn ret;\n+}\n+\n+static void xsc_free_qp_rq(struct xsc_rq *rq)\n+{\n+\tkvfree(rq-\u003ewqe.frags);\n+\tkvfree(rq-\u003ewqe.di);\n+\n+\tif (rq-\u003epage_pool)\n+\t\tpage_pool_destroy(rq-\u003epage_pool);\n+\n+\txsc_eth_wq_destroy(\u0026rq-\u003ewq_ctrl);\n+}\n+\n+static int xsc_eth_open_rss_qp_rqs(struct xsc_adapter *adapter,\n+\t\t\t\t struct xsc_rq_param *prq_param,\n+\t\t\t\t struct xsc_eth_channels *chls,\n+\t\t\t\t unsigned int num_chl)\n+{\n+\tu8 q_log_size = prq_param-\u003erq_attr.q_log_size;\n+\tstruct xsc_create_multiqp_mbox_in *in;\n+\tstruct xsc_create_qp_request *req;\n+\tunsigned int hw_npages;\n+\tstruct xsc_channel *c;\n+\tint ret = 0, err = 0;\n+\tstruct xsc_rq *prq;\n+\tint paslen = 0;\n+\tint entry_len;\n+\tu32 rqn_base;\n+\tint i, n;\n+\tint inlen;\n+\n+\tfor (i = 0; i \u003c num_chl; i++) {\n+\t\tc = \u0026chls-\u003ec[i];\n+\t\tprq = \u0026c-\u003eqp.rq;\n+\n+\t\tret = xsc_eth_alloc_rq(c, prq, prq_param);\n+\t\tif (ret)\n+\t\t\tgoto err_alloc_rqs;\n+\n+\t\thw_npages = DIV_ROUND_UP(prq-\u003ewq_ctrl.buf.size, PAGE_SIZE_4K);\n+\t\t/*support different npages number smoothly*/\n+\t\tentry_len = sizeof(struct xsc_create_qp_request) +\n+\t\t\tsizeof(__be64) * hw_npages;\n+\n+\t\tpaslen += entry_len;\n+\t}\n+\n+\tinlen = sizeof(struct xsc_create_multiqp_mbox_in) + paslen;\n+\tin = kvzalloc(inlen, GFP_KERNEL);\n+\tif (!in) {\n+\t\tret = -ENOMEM;\n+\t\tgoto err_create_rss_rqs;\n+\t}\n+\n+\tin-\u003eqp_num = cpu_to_be16(num_chl);\n+\tin-\u003eqp_type = XSC_QUEUE_TYPE_RAW;\n+\tin-\u003ereq_len = cpu_to_be32(inlen);\n+\n+\treq = (struct xsc_create_qp_request *)\u0026in-\u003edata[0];\n+\tn = 0;\n+\tfor (i = 0; i \u003c num_chl; i++) {\n+\t\tc = \u0026chls-\u003ec[i];\n+\t\tprq = \u0026c-\u003eqp.rq;\n+\n+\t\thw_npages = DIV_ROUND_UP(prq-\u003ewq_ctrl.buf.size, PAGE_SIZE_4K);\n+\t\t/* no use for eth */\n+\t\treq-\u003einput_qpn = cpu_to_be16(0);\n+\t\treq-\u003eqp_type = XSC_QUEUE_TYPE_RAW;\n+\t\treq-\u003elog_rq_sz = ilog2(adapter-\u003exdev-\u003ecaps.recv_ds_num)\n+\t\t\t\t\t+ q_log_size;\n+\t\treq-\u003epa_num = cpu_to_be16(hw_npages);\n+\t\treq-\u003ecqn_recv = cpu_to_be16(prq-\u003ecq.xcq.cqn);\n+\t\treq-\u003ecqn_send = req-\u003ecqn_recv;\n+\t\treq-\u003eglb_funcid = cpu_to_be16(adapter-\u003exdev-\u003eglb_func_id);\n+\n+\t\txsc_core_fill_page_frag_array(\u0026prq-\u003ewq_ctrl.buf,\n+\t\t\t\t\t \u0026req-\u003epas[0], hw_npages);\n+\t\tn++;\n+\t\tentry_len = sizeof(struct xsc_create_qp_request) +\n+\t\t\tsizeof(__be64) * hw_npages;\n+\t\treq = (struct xsc_create_qp_request *)\n+\t\t\t((u8 *)req + entry_len);\n+\t}\n+\n+\tret = xsc_core_eth_create_rss_qp_rqs(adapter-\u003exdev, in, inlen,\n+\t\t\t\t\t \u0026rqn_base);\n+\tkvfree(in);\n+\tif (ret)\n+\t\tgoto err_create_rss_rqs;\n+\n+\tn = 0;\n+\tfor (i = 0; i \u003c num_chl; i++) {\n+\t\tc = \u0026chls-\u003ec[i];\n+\t\tprq = \u0026c-\u003eqp.rq;\n+\n+\t\tprq-\u003erqn = rqn_base + n;\n+\t\tprq-\u003ecqp.qpn = prq-\u003erqn;\n+\t\tprq-\u003ecqp.event = xsc_eth_qp_event;\n+\t\tprq-\u003ecqp.eth_queue_type = XSC_RES_RQ;\n+\t\tret = xsc_core_create_resource_common(adapter-\u003exdev,\n+\t\t\t\t\t\t \u0026prq-\u003ecqp);\n+\t\tif (ret) {\n+\t\t\terr = ret;\n+\t\t\tnetdev_err(adapter-\u003enetdev,\n+\t\t\t\t \"create resource common error qp:%d errno:%d\\n\",\n+\t\t\t\t prq-\u003erqn, ret);\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tn++;\n+\t}\n+\tif (err)\n+\t\treturn err;\n+\n+\tadapter-\u003echannels.rqn_base = rqn_base;\n+\treturn 0;\n+\n+err_create_rss_rqs:\n+\ti = num_chl;\n+err_alloc_rqs:\n+\tfor (--i; i \u003e= 0; i--) {\n+\t\tc = \u0026chls-\u003ec[i];\n+\t\txsc_free_qp_rq(\u0026c-\u003eqp.rq);\n+\t}\n+\treturn ret;\n+}\n+\n+static void xsc_eth_free_rx_wqe(struct xsc_rq *rq)\n+{\n+\tstruct xsc_wq_cyc *wq = \u0026rq-\u003ewqe.wq;\n+\tu16 wqe_ix;\n+\n+\twhile (!xsc_wq_cyc_is_empty(wq)) {\n+\t\twqe_ix = xsc_wq_cyc_get_tail(wq);\n+\t\trq-\u003edealloc_wqe(rq, wqe_ix);\n+\t\txsc_wq_cyc_pop(wq);\n+\t}\n+}\n+\n+static int xsc_eth_close_qp_rq(struct xsc_channel *c, struct xsc_rq *prq)\n+{\n+\tstruct xsc_core_device *xdev = c-\u003eadapter-\u003exdev;\n+\tint ret;\n+\n+\txsc_core_destroy_resource_common(xdev, \u0026prq-\u003ecqp);\n+\n+\tret = xsc_core_eth_destroy_qp(xdev, prq-\u003ecqp.qpn);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\txsc_eth_free_rx_wqe(prq);\n+\txsc_free_qp_rq(prq);\n+\n+\treturn 0;\n+}\n+\n+static void xsc_eth_close_channel(struct xsc_channel *c, bool free_rq)\n+{\n+\tif (free_rq)\n+\t\txsc_eth_close_qp_rq(c, \u0026c-\u003eqp.rq);\n+\txsc_eth_close_cq(c, \u0026c-\u003eqp.rq.cq);\n+\tmemset(\u0026c-\u003eqp.rq, 0, sizeof(struct xsc_rq));\n+\n+\txsc_eth_close_qp_sq(c, \u0026c-\u003eqp.sq);\n+\txsc_eth_close_cq(c, \u0026c-\u003eqp.sq.cq);\n+\n+\tnetif_napi_del(\u0026c-\u003enapi);\n+}\n+\n+static int xsc_eth_open_channels(struct xsc_adapter *adapter)\n+{\n+\tstruct xsc_eth_channels *chls = \u0026adapter-\u003echannels;\n+\tstruct xsc_core_device *xdev = adapter-\u003exdev;\n+\tstruct xsc_channel_param *chl_param;\n+\tbool free_rq = false;\n+\tint ret = 0;\n+\tint i;\n+\n+\tchls-\u003enum_chl = adapter-\u003enic_param.num_channels;\n+\tchls-\u003ec = kcalloc_node(chls-\u003enum_chl, sizeof(struct xsc_channel),\n+\t\t\t GFP_KERNEL, xdev-\u003enuma_node);\n+\tif (!chls-\u003ec) {\n+\t\tret = -ENOMEM;\n+\t\tgoto err_out;\n+\t}\n+\n+\tchl_param = kvzalloc_obj(*chl_param, GFP_KERNEL);\n+\tif (!chl_param) {\n+\t\tret = -ENOMEM;\n+\t\tgoto err_free_ch;\n+\t}\n+\n+\txsc_eth_build_channel_param(adapter, chl_param);\n+\n+\tfor (i = 0; i \u003c chls-\u003enum_chl; i++) {\n+\t\tret = xsc_eth_open_channel(adapter, i, \u0026chls-\u003ec[i], chl_param);\n+\t\tif (ret)\n+\t\t\tgoto err_close_channel;\n+\t}\n+\n+\tret = xsc_eth_open_rss_qp_rqs(adapter,\n+\t\t\t\t \u0026chl_param-\u003erq_param,\n+\t\t\t\t chls,\n+\t\t\t\t chls-\u003enum_chl);\n+\tif (ret)\n+\t\tgoto err_close_channel;\n+\tfree_rq = true;\n+\n+\tfor (i = 0; i \u003c chls-\u003enum_chl; i++)\n+\t\tnapi_enable(\u0026chls-\u003ec[i].napi);\n+\n+\t/* flush cache to memory before interrupt and napi_poll running */\n+\tsmp_wmb();\n+\n+\tret = xsc_eth_modify_qps(adapter, chls);\n+\tif (ret)\n+\t\tgoto err_close_channel;\n+\n+\tkvfree(chl_param);\n+\treturn 0;\n+\n+err_close_channel:\n+\tfor (--i; i \u003e= 0; i--)\n+\t\txsc_eth_close_channel(\u0026chls-\u003ec[i], free_rq);\n+\n+\tkvfree(chl_param);\n+err_free_ch:\n+\tkfree(chls-\u003ec);\n+err_out:\n+\tchls-\u003enum_chl = 0;\n+\tnetdev_err(adapter-\u003enetdev, \"failed to open %d channels, err=%d\\n\",\n+\t\t chls-\u003enum_chl, ret);\n+\treturn ret;\n+}\n+\n+static void xsc_eth_close_channels(struct xsc_adapter *adapter)\n+{\n+\tstruct xsc_channel *c;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c adapter-\u003echannels.num_chl; i++) {\n+\t\tc = \u0026adapter-\u003echannels.c[i];\n+\n+\t\txsc_eth_close_channel(c, true);\n+\t}\n+\n+\tkfree(adapter-\u003echannels.c);\n+\tadapter-\u003echannels.num_chl = 0;\n+}\n+\n+static void xsc_eth_build_tx2sq_maps(struct xsc_adapter *adapter)\n+{\n+\tstruct xsc_channel *c;\n+\tstruct xsc_sq *psq;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c adapter-\u003echannels.num_chl; i++) {\n+\t\tc = \u0026adapter-\u003echannels.c[i];\n+\t\tpsq = \u0026c-\u003eqp.sq;\n+\t\tadapter-\u003etxq2sq[psq-\u003etxq_ix] = psq;\n+\t}\n+}\n+\n+static void xsc_eth_activate_txqsq(struct xsc_channel *c)\n+{\n+\tstruct xsc_sq *psq = \u0026c-\u003eqp.sq;\n+\n+\tpsq-\u003etxq = netdev_get_tx_queue(psq-\u003echannel-\u003enetdev, psq-\u003etxq_ix);\n+\tset_bit(XSC_ETH_SQ_STATE_ENABLED, \u0026psq-\u003estate);\n+\tnetdev_tx_reset_queue(psq-\u003etxq);\n+\tnetif_tx_start_queue(psq-\u003etxq);\n+}\n+\n+static void xsc_eth_deactivate_txqsq(struct xsc_channel *c)\n+{\n+\tclear_bit(XSC_ETH_SQ_STATE_ENABLED, \u0026c-\u003eqp.sq.state);\n+}\n+\n+static void xsc_activate_rq(struct xsc_channel *c)\n+{\n+\tset_bit(XSC_ETH_RQ_STATE_ENABLED, \u0026c-\u003eqp.rq.state);\n+}\n+\n+static void xsc_deactivate_rq(struct xsc_channel *c)\n+{\n+\tclear_bit(XSC_ETH_RQ_STATE_ENABLED, \u0026c-\u003eqp.rq.state);\n+}\n+\n+static void xsc_eth_activate_channel(struct xsc_channel *c)\n+{\n+\txsc_eth_activate_txqsq(c);\n+\txsc_activate_rq(c);\n+}\n+\n+static void xsc_eth_deactivate_channel(struct xsc_channel *c)\n+{\n+\txsc_deactivate_rq(c);\n+\txsc_eth_deactivate_txqsq(c);\n+}\n+\n+static void xsc_eth_activate_channels(struct xsc_eth_channels *chs)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i \u003c chs-\u003enum_chl; i++)\n+\t\txsc_eth_activate_channel(\u0026chs-\u003ec[i]);\n+}\n+\n+static void xsc_eth_deactivate_channels(struct xsc_eth_channels *chs)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i \u003c chs-\u003enum_chl; i++)\n+\t\txsc_eth_deactivate_channel(\u0026chs-\u003ec[i]);\n+\n+\t/* Sync with all NAPIs to wait until they stop using queues. */\n+\tsynchronize_net();\n+\n+\tfor (i = 0; i \u003c chs-\u003enum_chl; i++)\n+\t\t/* last doorbell out */\n+\t\tnapi_disable(\u0026chs-\u003ec[i].napi);\n+}\n+\n+static void xsc_eth_activate_priv_channels(struct xsc_adapter *adapter)\n+{\n+\tstruct net_device *netdev = adapter-\u003enetdev;\n+\n+\tnetif_set_real_num_tx_queues(netdev, adapter-\u003echannels.num_chl);\n+\tnetif_set_real_num_rx_queues(netdev, adapter-\u003echannels.num_chl);\n+\n+\txsc_eth_build_tx2sq_maps(adapter);\n+\txsc_eth_activate_channels(\u0026adapter-\u003echannels);\n+\tnetif_tx_start_all_queues(adapter-\u003enetdev);\n+}\n+\n+static void xsc_eth_deactivate_priv_channels(struct xsc_adapter *adapter)\n+{\n+\tnetif_tx_disable(adapter-\u003enetdev);\n+\txsc_eth_deactivate_channels(\u0026adapter-\u003echannels);\n+}\n+\n+static int xsc_eth_sw_init(struct xsc_adapter *adapter)\n+{\n+\tint ret;\n+\n+\tret = xsc_eth_open_channels(adapter);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\txsc_eth_activate_priv_channels(adapter);\n+\n+\treturn 0;\n+}\n+\n+static void xsc_eth_sw_deinit(struct xsc_adapter *adapter)\n+{\n+\txsc_eth_deactivate_priv_channels(adapter);\n+\n+\treturn xsc_eth_close_channels(adapter);\n+}\n+\n+static bool xsc_eth_get_link_status(struct xsc_adapter *adapter)\n+{\n+\tstruct xsc_core_device *xdev = adapter-\u003exdev;\n+\tu16 vport = 0;\n+\tbool link_up;\n+\n+\tlink_up = xsc_core_query_vport_state(xdev, vport);\n+\n+\treturn link_up ? true : false;\n+}\n+\n+static int xsc_eth_change_link_status(struct xsc_adapter *adapter)\n+{\n+\tbool link_up;\n+\n+\tlink_up = xsc_eth_get_link_status(adapter);\n+\n+\tif (link_up \u0026\u0026 !netif_carrier_ok(adapter-\u003enetdev)) {\n+\t\tnetdev_info(adapter-\u003enetdev, \"Link up\\n\");\n+\t\tnetif_carrier_on(adapter-\u003enetdev);\n+\t} else if (!link_up \u0026\u0026 netif_carrier_ok(adapter-\u003enetdev)) {\n+\t\tnetdev_info(adapter-\u003enetdev, \"Link down\\n\");\n+\t\tnetif_carrier_off(adapter-\u003enetdev);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Registered with the PCI layer, which runs it on its own workqueue for\n+ * every XSC_CMD_EVENT_RESP_CHANGE_LINK event.\n+ */\n+static void xsc_eth_link_event_handler(void *arg)\n+{\n+\tstruct xsc_core_device *xdev = arg;\n+\tstruct xsc_adapter *adapter = xdev-\u003eeth_priv;\n+\tint err;\n+\n+\tif (!adapter || !adapter-\u003eopened)\n+\t\treturn;\n+\n+\terr = xsc_eth_change_link_status(adapter);\n+\tif (err)\n+\t\tnetdev_err(adapter-\u003enetdev, \"failed to change linkstatus, err=%d\\n\",\n+\t\t\t err);\n+}\n+\n+static bool xsc_get_pct_drop_config(struct xsc_core_device *xdev)\n+{\n+\treturn (xdev-\u003epdev-\u003edevice == XSC_MC_PF_DEV_ID) ||\n+\t\t(xdev-\u003epdev-\u003edevice == XSC_MF_SOC_PF_DEV_ID) ||\n+\t\t(xdev-\u003epdev-\u003edevice == XSC_MS_PF_DEV_ID) ||\n+\t\t(xdev-\u003epdev-\u003edevice == XSC_MV_SOC_PF_DEV_ID);\n+}\n+\n+static int xsc_eth_enable_nic_hca(struct xsc_adapter *adapter)\n+{\n+\tstruct xsc_cmd_enable_nic_hca_mbox_out out = {};\n+\tstruct xsc_cmd_enable_nic_hca_mbox_in in = {};\n+\tstruct xsc_core_device *xdev = adapter-\u003exdev;\n+\tstruct net_device *netdev = adapter-\u003enetdev;\n+\tu16 caps_mask = 0;\n+\tu16 caps = 0;\n+\tint err;\n+\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_ENABLE_NIC_HCA);\n+\n+\tin.rss.rss_en = 1;\n+\tin.rss.rqn_base = cpu_to_be16(adapter-\u003echannels.rqn_base -\n+\t\t\t\txdev-\u003ecaps.raweth_rss_qp_id_base);\n+\tin.rss.rqn_num = cpu_to_be16(adapter-\u003echannels.num_chl);\n+\t/* hfunc is a constant: the driver never negotiates anything else */\n+\tin.rss.hfunc = XSC_HASH_FUNC_TOP;\n+\tin.rss.hash_tmpl = cpu_to_be32(adapter-\u003erss_param.rss_hash_tmpl);\n+\tcaps_mask |= BIT(XSC_TBM_CAP_RSS);\n+\n+\tif (netdev-\u003efeatures \u0026 NETIF_F_RXCSUM)\n+\t\tcaps |= BIT(XSC_TBM_CAP_HASH_PPH);\n+\tcaps_mask |= BIT(XSC_TBM_CAP_HASH_PPH);\n+\n+\tif (xsc_get_pct_drop_config(xdev) \u0026\u0026 !(netdev-\u003eflags \u0026 IFF_SLAVE))\n+\t\tcaps |= BIT(XSC_TBM_CAP_PCT_DROP_CONFIG);\n+\tcaps_mask |= BIT(XSC_TBM_CAP_PCT_DROP_CONFIG);\n+\n+\tmemcpy(in.nic.mac_addr, netdev-\u003edev_addr, ETH_ALEN);\n+\n+\tin.nic.caps = cpu_to_be16(caps);\n+\tin.nic.caps_mask = cpu_to_be16(caps_mask);\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err) {\n+\t\tnetdev_err(netdev, \"failed!! err=%d, status=%d\\n\",\n+\t\t\t err, out.hdr.status);\n+\t\treturn -ENOEXEC;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int xsc_eth_disable_nic_hca(struct xsc_adapter *adapter)\n+{\n+\tstruct xsc_cmd_disable_nic_hca_mbox_out out = {};\n+\tstruct xsc_cmd_disable_nic_hca_mbox_in in = {};\n+\tstruct xsc_core_device *xdev = adapter-\u003exdev;\n+\tstruct net_device *netdev = adapter-\u003enetdev;\n+\tu16 caps = 0;\n+\tint err;\n+\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DISABLE_NIC_HCA);\n+\n+\tif (xsc_get_pct_drop_config(xdev) \u0026\u0026\n+\t !(netdev-\u003epriv_flags \u0026 IFF_BONDING))\n+\t\tcaps |= BIT(XSC_TBM_CAP_PCT_DROP_CONFIG);\n+\n+\tin.nic.caps = cpu_to_be16(caps);\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err) {\n+\t\tnetdev_err(netdev, \"failed!! err=%d, status=%d\\n\",\n+\t\t\t err, out.hdr.status);\n+\t\treturn -ENOEXEC;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static void xsc_set_default_xps_cpumasks(struct xsc_adapter *adapter)\n+{\n+\tstruct xsc_core_device *xdev = adapter-\u003exdev;\n+\tint num_comp_vectors, irq;\n+\n+\tnum_comp_vectors = adapter-\u003enic_param.comp_vectors;\n+\tcpumask_clear(xdev-\u003exps_cpumask);\n+\n+\tfor (irq = 0; irq \u003c num_comp_vectors; irq++) {\n+\t\tcpumask_set_cpu(cpumask_local_spread(irq, xdev-\u003enuma_node),\n+\t\t\t\txdev-\u003exps_cpumask);\n+\t\tnetif_set_xps_queue(adapter-\u003enetdev, xdev-\u003exps_cpumask, irq);\n+\t}\n+}\n+\n+static int xsc_set_port_admin_status(struct xsc_adapter *adapter,\n+\t\t\t\t enum xsc_port_status status)\n+{\n+\tstruct xsc_event_set_port_admin_status_mbox_out out;\n+\tstruct xsc_event_set_port_admin_status_mbox_in in = {};\n+\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_PORT_ADMIN_STATUS);\n+\tin.status = cpu_to_be16(status);\n+\n+\treturn xsc_cmd_exec(adapter-\u003exdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+}\n+\n+static int xsc_eth_open(struct net_device *netdev)\n+{\n+\tstruct xsc_adapter *adapter = netdev_priv(netdev);\n+\tstruct xsc_core_device *xdev = adapter-\u003exdev;\n+\tint ret;\n+\n+\tif (adapter-\u003eopened) {\n+\t\tnetdev_err(adapter-\u003enetdev, \"ndo_open on an already opened device\\n\");\n+\t\treturn 0;\n+\t}\n+\tret = xsc_eth_sw_init(adapter);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tret = xsc_eth_enable_nic_hca(adapter);\n+\tif (ret)\n+\t\tgoto err_sw_deinit;\n+\n+\txsc_irq_event_register(xdev, xsc_eth_link_event_handler);\n+\n+\tif (xsc_eth_get_link_status(adapter))\t{\n+\t\tnetdev_info(netdev, \"Link up\\n\");\n+\t\tnetif_carrier_on(adapter-\u003enetdev);\n+\t} else {\n+\t\tnetdev_info(netdev, \"Link down\\n\");\n+\t}\n+\n+\tadapter-\u003eopened = true;\n+\n+\txsc_set_default_xps_cpumasks(adapter);\n+\n+\txsc_set_port_admin_status(adapter, XSC_PORT_UP);\n+\n+\treturn 0;\n+\n+err_sw_deinit:\n+\txsc_eth_sw_deinit(adapter);\n+\treturn ret;\n+}\n+\n+static int xsc_eth_close(struct net_device *netdev)\n+{\n+\tstruct xsc_adapter *adapter = netdev_priv(netdev);\n+\tint ret;\n+\n+\tif (!adapter-\u003eopened)\n+\t\treturn 0;\n+\tadapter-\u003eopened = false;\n+\n+\tnetif_carrier_off(adapter-\u003enetdev);\n+\n+\txsc_eth_sw_deinit(adapter);\n+\n+\tret = xsc_eth_disable_nic_hca(adapter);\n+\tif (ret)\n+\t\tnetdev_err(adapter-\u003enetdev, \"failed to disable nic hca, err=%d\\n\",\n+\t\t\t ret);\n+\n+\txsc_set_port_admin_status(adapter, XSC_PORT_DOWN);\n+\n+\treturn ret;\n+}\n+\n+static void xsc_eth_get_stats(struct net_device *netdev,\n+\t\t\t struct rtnl_link_stats64 *stats)\n+{\n+\tstruct xsc_adapter *adapter = netdev_priv(netdev);\n+\n+\txsc_eth_fold_sw_stats64(adapter, stats);\n+}\n+\n+static int xsc_eth_set_hw_mtu(struct xsc_core_device *xdev,\n+\t\t\t u16 mtu, u16 rx_buf_sz)\n+{\n+\tstruct xsc_set_mtu_mbox_out out;\n+\tstruct xsc_set_mtu_mbox_in in;\n+\tint err;\n+\n+\tmemset(\u0026in, 0, sizeof(struct xsc_set_mtu_mbox_in));\n+\tmemset(\u0026out, 0, sizeof(struct xsc_set_mtu_mbox_out));\n+\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_MTU);\n+\tin.mtu = cpu_to_be16(mtu);\n+\tin.rx_buf_sz_min = cpu_to_be16(rx_buf_sz);\n+\tin.mac_port = xdev-\u003emac_port;\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(struct xsc_set_mtu_mbox_in), \u0026out,\n+\t\t\t sizeof(struct xsc_set_mtu_mbox_out));\n+\tif (err) {\n+\t\tnetdev_err(xsc_dev_to_netdev(xdev),\n+\t\t\t \"failed to set hw_mtu=%u rx_buf_sz=%u, err=%d, status=%d\\n\",\n+\t\t\t mtu, rx_buf_sz, err, out.hdr.status);\n+\t\treturn err;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static const struct net_device_ops xsc_netdev_ops = {\n+\t.ndo_open\t\t= xsc_eth_open,\n+\t.ndo_stop\t\t= xsc_eth_close,\n+\t.ndo_start_xmit\t\t= xsc_eth_xmit_start,\n+\t.ndo_get_stats64\t= xsc_eth_get_stats,\n+};\n+\n+static void xsc_eth_build_nic_netdev(struct xsc_adapter *adapter)\n+{\n+\tstruct net_device *netdev = adapter-\u003enetdev;\n+\n+\t/* Set up network device as normal. */\n+\tnetdev-\u003enetdev_ops = \u0026xsc_netdev_ops;\n+\n+\tnetdev-\u003emin_mtu = SW_DEFAULT_MTU;\n+\tnetdev-\u003emax_mtu = SW_DEFAULT_MTU;\n+\t/*mtu - macheaderlen - ipheaderlen should be aligned in 8B*/\n+\tnetdev-\u003emtu = SW_DEFAULT_MTU;\n+\n+\tnetdev-\u003evlan_features |= NETIF_F_SG |\n+\t\t\t\t NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |\n+\t\t\t\t NETIF_F_TSO | NETIF_F_TSO6 |\n+\t\t\t\t NETIF_F_RXCSUM | NETIF_F_RXHASH;\n+\n+\tnetdev-\u003ehw_features = netdev-\u003evlan_features;\n+\n+\tnetdev-\u003efeatures = netdev-\u003ehw_features | NETIF_F_HIGHDMA;\n+}\n+\n+static int xsc_eth_nic_init(struct xsc_adapter *adapter, u32 ch_num)\n+{\n+\tint err;\n+\n+\txsc_eth_build_nic_params(adapter, ch_num);\n+\n+\terr = xsc_eth_netdev_init(adapter);\n+\tif (err)\n+\t\treturn err;\n+\n+\txsc_eth_build_nic_netdev(adapter);\n+\n+\treturn 0;\n+}\n+\n+static void xsc_eth_nic_cleanup(struct xsc_adapter *adapter)\n+{\n+\tkfree(adapter-\u003etxq2sq);\n+}\n+\n+static int xsc_eth_get_mac(struct xsc_core_device *xdev, char *mac)\n+{\n+\tstruct xsc_query_eth_mac_mbox_out *out;\n+\tstruct xsc_query_eth_mac_mbox_in in;\n+\tint err;\n+\n+\tout = kzalloc_obj(*out, GFP_KERNEL);\n+\tif (!out)\n+\t\treturn -ENOMEM;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_ETH_MAC);\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), out, sizeof(*out));\n+\tif (err || out-\u003ehdr.status) {\n+\t\tnetdev_warn(xsc_dev_to_netdev(xdev),\n+\t\t\t \"get mac failed! err=%d, out.status=%u\\n\",\n+\t\t\t err, out-\u003ehdr.status);\n+\t\terr = -ENOEXEC;\n+\t\tgoto exit;\n+\t}\n+\n+\tmemcpy(mac, out-\u003emac, ETH_ALEN);\n+\tnetdev_dbg(xsc_dev_to_netdev(xdev),\n+\t\t \"get mac %02x:%02x:%02x:%02x:%02x:%02x\\n\",\n+\t\t mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);\n+\n+exit:\n+\tkfree(out);\n+\treturn err;\n+}\n+\n+static void xsc_eth_l2_addr_init(struct xsc_adapter *adapter)\n+{\n+\tstruct net_device *netdev = adapter-\u003enetdev;\n+\tchar mac[6] = {0};\n+\tint ret = 0;\n+\n+\tret = xsc_eth_get_mac(adapter-\u003exdev, mac);\n+\tif (ret) {\n+\t\tnetdev_warn(netdev, \"get mac failed %d, generate random mac...\",\n+\t\t\t ret);\n+\t\teth_random_addr(mac);\n+\t}\n+\teth_hw_addr_set(netdev, mac);\n+\n+\tif (!is_valid_ether_addr(netdev-\u003eperm_addr))\n+\t\tmemcpy(netdev-\u003eperm_addr, netdev-\u003edev_addr, netdev-\u003eaddr_len);\n+}\n+\n+/*\n+ * Program the MAC address and the MTU into the hardware. Called once from\n+ * probe(), before the netdev is registered: the netdev layer marks the\n+ * device present itself in register_netdevice(), so there is nothing to\n+ * attach here and no need to take the rtnl lock.\n+ */\n+static int xsc_eth_nic_enable(struct xsc_adapter *adapter)\n+{\n+\tstruct xsc_core_device *xdev = adapter-\u003exdev;\n+\n+\txsc_eth_l2_addr_init(adapter);\n+\n+\treturn xsc_eth_set_hw_mtu(xdev, XSC_SW2HW_MTU(adapter-\u003enic_param.mtu),\n+\t\t\t\t XSC_SW2HW_RX_PKT_LEN(adapter-\u003enic_param.mtu));\n+}\n+\n+static int xsc_eth_probe(struct auxiliary_device *adev,\n+\t\t\t const struct auxiliary_device_id *adev_id)\n+{\n+\tstruct xsc_adev *xsc_adev = container_of(adev, struct xsc_adev, adev);\n+\tstruct xsc_core_device *xdev = xsc_adev-\u003exdev;\n+\tstruct xsc_adapter *adapter;\n+\tstruct net_device *netdev;\n+\tint num_chl;\n+\tint err;\n+\n+\tnum_chl = xsc_get_max_num_channels(xdev);\n+\n+\tnetdev = alloc_etherdev_mqs(sizeof(struct xsc_adapter),\n+\t\t\t\t num_chl, num_chl);\n+\tif (!netdev) {\n+\t\tpr_err(\"alloc_etherdev_mqs failed, txq=%d, rxq=%d\\n\",\n+\t\t num_chl, num_chl);\n+\t\treturn -ENOMEM;\n+\t}\n+\n+\tnetdev-\u003edev.parent = \u0026xdev-\u003epdev-\u003edev;\n+\tadapter = netdev_priv(netdev);\n+\tadapter-\u003enetdev = netdev;\n+\tadapter-\u003epdev = xdev-\u003epdev;\n+\tadapter-\u003edev = \u0026adapter-\u003epdev-\u003edev;\n+\tadapter-\u003exdev = xdev;\n+\txdev-\u003eeth_priv = adapter;\n+\n+\terr = xsc_eth_nic_init(adapter, num_chl);\n+\tif (err) {\n+\t\tnetdev_err(netdev, \"xsc_eth_nic_init failed, err=%d\\n\", err);\n+\t\tgoto err_free_netdev;\n+\t}\n+\n+\terr = xsc_eth_nic_enable(adapter);\n+\tif (err) {\n+\t\tnetdev_err(netdev, \"xsc_eth_nic_enable failed, err=%d\\n\", err);\n+\t\tgoto err_nic_cleanup;\n+\t}\n+\n+\tadapter-\u003estats = kvzalloc_obj(*adapter-\u003estats, GFP_KERNEL);\n+\tif (!adapter-\u003estats) {\n+\t\terr = -ENOMEM;\n+\t\tgoto err_nic_cleanup;\n+\t}\n+\n+\terr = register_netdev(netdev);\n+\tif (err) {\n+\t\tnetdev_err(netdev, \"register_netdev failed, err=%d\\n\", err);\n+\t\tgoto err_free_stats;\n+\t}\n+\n+\txdev-\u003enetdev = netdev;\n+\n+\treturn 0;\n+\n+err_free_stats:\n+\tkvfree(adapter-\u003estats);\n+err_nic_cleanup:\n+\txsc_eth_nic_cleanup(adapter);\n+err_free_netdev:\n+\txdev-\u003eeth_priv = NULL;\n+\tfree_netdev(netdev);\n+\n+\treturn err;\n+}\n+\n+static void xsc_eth_remove(struct auxiliary_device *adev)\n+{\n+\tstruct xsc_adev *xsc_adev = container_of(adev, struct xsc_adev, adev);\n+\tstruct xsc_core_device *xdev = xsc_adev-\u003exdev;\n+\tstruct xsc_adapter *adapter;\n+\n+\tif (!xdev)\n+\t\treturn;\n+\n+\tadapter = xdev-\u003eeth_priv;\n+\tif (!adapter)\n+\t\treturn;\n+\n+\t/* Stop link events reaching this adapter and wait out one already in\n+\t * flight, which would otherwise touch the memory freed below.\n+\t */\n+\txsc_irq_event_unregister(xdev);\n+\n+\t/* Runs ndo_stop(), so the queues and the HCA are shut down first. */\n+\tunregister_netdev(adapter-\u003enetdev);\n+\n+\t/* Must precede free_netdev(): the adapter lives in the netdev priv. */\n+\txsc_eth_nic_cleanup(adapter);\n+\tkvfree(adapter-\u003estats);\n+\tfree_netdev(adapter-\u003enetdev);\n+\n+\txdev-\u003enetdev = NULL;\n+\txdev-\u003eeth_priv = NULL;\n+}\n+\n+static const struct auxiliary_device_id xsc_eth_id_table[] = {\n+\t{ .name = XSC_PCI_DRV_NAME \".\" XSC_ETH_ADEV_NAME },\n+\t{},\n+};\n+MODULE_DEVICE_TABLE(auxiliary, xsc_eth_id_table);\n+\n+static struct auxiliary_driver xsc_eth_driver = {\n+\t.name = \"eth\",\n+\t.probe = xsc_eth_probe,\n+\t.remove = xsc_eth_remove,\n+\t.id_table = xsc_eth_id_table,\n+};\n+module_auxiliary_driver(xsc_eth_driver);\n+\n+MODULE_LICENSE(\"GPL\");\n+MODULE_DESCRIPTION(\"Yunsilicon XSC ethernet driver\");\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth.h\nnew file mode 100644\nindex 0000000000000..4d9f92e094ae0\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth.h\n@@ -0,0 +1,45 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_ETH_H\n+#define __XSC_ETH_H\n+\n+#include \u003clinux/udp.h\u003e\n+\n+#include \"common/xsc_device.h\"\n+#include \"xsc_eth_common.h\"\n+#include \"xsc_eth_stats.h\"\n+\n+#define XSC_INVALID_LKEY\t0x100\n+\n+struct xsc_rss_params {\n+\tu32\trx_hash_fields[XSC_NUM_INDIR_TIRS];\n+\tu32\trss_hash_tmpl;\n+};\n+\n+struct xsc_adapter {\n+\tstruct net_device\t*netdev;\n+\tstruct pci_dev\t\t*pdev;\n+\tstruct device\t\t*dev;\n+\tstruct xsc_core_device\t*xdev;\n+\n+\tstruct xsc_eth_params\tnic_param;\n+\tstruct xsc_rss_params\trss_param;\n+\n+\tstruct xsc_eth_channels\tchannels;\n+\tstruct xsc_sq\t\t**txq2sq;\n+\n+\t/* true between ndo_open and ndo_stop */\n+\tbool\t\t\topened;\n+\n+\tstruct xsc_stats\t*stats;\n+};\n+\n+static inline struct net_device *xsc_dev_to_netdev(struct xsc_core_device *xdev)\n+{\n+\treturn ((struct xsc_adapter *)xdev-\u003eeth_priv)-\u003enetdev;\n+}\n+\n+#endif /* __XSC_ETH_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_common.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_common.h\nnew file mode 100644\nindex 0000000000000..be476f31f31e7\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_common.h\n@@ -0,0 +1,124 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_ETH_COMMON_H\n+#define __XSC_ETH_COMMON_H\n+\n+#include \"xsc_queue.h\"\n+#include \"xsc_pph.h\"\n+\n+#define SW_DEFAULT_MTU\t\tETH_DATA_LEN\n+\n+#define XSC_ETH_HW_MTU_SEND\t9800\n+#define XSC_ETH_HARD_MTU\t(ETH_HLEN + VLAN_HLEN * 2 + ETH_FCS_LEN)\n+#define XSC_SW2HW_MTU(mtu)\t((mtu) + XSC_ETH_HARD_MTU)\n+#define XSC_SW2HW_HLEN\t\t(ETH_HLEN + VLAN_HLEN * 2 + ETH_FCS_LEN + \\\n+\t\t\t\t XSC_PPH_HEAD_LEN)\n+#define XSC_SW2HW_FRAG_SIZE(mtu)\t((mtu) + XSC_SW2HW_HLEN)\n+#define XSC_ETH_RX_MAX_HEAD_ROOM\t256\n+#define XSC_SW2HW_RX_PKT_LEN(mtu)\t\\\n+\t((mtu) + ETH_HLEN + XSC_ETH_RX_MAX_HEAD_ROOM)\n+\n+#define XSC_RX_MAX_HEAD\t\t\t(256)\n+\n+#define XSC_QPN_SQN_STUB\t\t1025\n+\n+#define XSC_ETH_MAX_NUM_CHANNELS\t256\n+\n+#define XSC_SKB_FRAG_SZ(len)\t(SKB_DATA_ALIGN(len) +\t\\\n+\t\t\t\tSKB_DATA_ALIGN(sizeof(struct skb_shared_info)))\n+\n+#define XSC_RQCQ_ELE_SZ\t\t32\n+#define XSC_SQCQ_ELE_SZ\t\t32\n+\n+#define XSC_LOG_RQCQ_SZ\t\t0xb\n+#define XSC_LOG_SQCQ_SZ\t\t0xa\n+\n+#define XSC_RQCQ_ELE_NUM\tBIT(XSC_LOG_RQCQ_SZ)\n+#define XSC_SQCQ_ELE_NUM\tBIT(XSC_LOG_SQCQ_SZ)\n+\n+enum xsc_port_status {\n+\tXSC_PORT_DOWN = 0,\n+\tXSC_PORT_UP = 1,\n+};\n+\n+enum xsc_queue_type {\n+\tXSC_QUEUE_TYPE_RQCQ,\n+\tXSC_QUEUE_TYPE_SQCQ,\n+\tXSC_QUEUE_TYPE_RQ,\n+\tXSC_QUEUE_TYPE_SQ,\n+};\n+\n+struct xsc_queue_attr {\n+\tu32 ele_num;\n+\tu32 ele_size;\n+\tu8 ele_log_size;\n+\tu8 q_log_size;\n+};\n+\n+struct xsc_eth_rx_wqe_cyc {\n+\tDECLARE_FLEX_ARRAY(struct xsc_wqe_data_seg, data);\n+};\n+\n+struct xsc_cq_param {\n+\tstruct xsc_wq_param wq;\n+\tstruct xsc_queue_attr cq_attr;\n+};\n+\n+struct xsc_rq_param {\n+\tstruct xsc_wq_param wq;\n+\tstruct xsc_queue_attr rq_attr;\n+\tstruct xsc_rq_frags_info frags_info;\n+};\n+\n+struct xsc_sq_param {\n+\tstruct xsc_wq_param wq;\n+\tstruct xsc_queue_attr sq_attr;\n+};\n+\n+struct xsc_channel_param {\n+\tstruct xsc_cq_param rqcq_param;\n+\tstruct xsc_cq_param sqcq_param;\n+\tstruct xsc_rq_param rq_param;\n+\tstruct xsc_sq_param sq_param;\n+};\n+\n+struct xsc_eth_qp {\n+\tstruct xsc_rq rq;\n+\tstruct xsc_sq sq;\n+};\n+\n+struct xsc_eth_params {\n+\tu16\tnum_channels;\n+\tu16\tmax_num_ch;\n+\tu32\tmtu;\n+\tu32\tcomp_vectors;\n+\tu32\tsq_size;\n+\tu32\trq_size;\n+\tu32\trq_frags_size;\n+};\n+\n+struct xsc_channel {\n+\t/* data path */\n+\tstruct xsc_eth_qp\tqp;\n+\tstruct napi_struct\tnapi;\n+\tint\t\t\tchl_idx;\n+\n+\t/*relationship*/\n+\tstruct xsc_adapter\t*adapter;\n+\tstruct net_device\t*netdev;\n+\tint\t\t\tcpu;\n+\n+\t/* data path - accessed per napi poll */\n+\tconst struct cpumask\t*aff_mask;\n+} ____cacheline_aligned_in_smp;\n+\n+struct xsc_eth_channels {\n+\tstruct xsc_channel\t*c;\n+\tunsigned int\t\tnum_chl;\n+\tu32\t\t\trqn_base;\n+};\n+\n+#endif\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_rx.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_rx.c\nnew file mode 100644\nindex 0000000000000..3913a9671cc19\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_rx.c\n@@ -0,0 +1,551 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd. All\n+ * rights reserved.\n+ * Copyright (c) 2015-2016, Mellanox Technologies. All rights reserved.\n+ */\n+\n+#include \u003clinux/net_tstamp.h\u003e\n+#include \u003clinux/device.h\u003e\n+\n+#include \"xsc_pp.h\"\n+#include \"xsc_eth.h\"\n+#include \"xsc_eth_txrx.h\"\n+#include \"xsc_eth_common.h\"\n+#include \"xsc_pph.h\"\n+\n+static void xsc_rq_notify_hw(struct xsc_rq *rq)\n+{\n+\tstruct xsc_core_device *xdev = rq-\u003ecq.xdev;\n+\tstruct xsc_wq_cyc *wq = \u0026rq-\u003ewqe.wq;\n+\tu64 rqwqe_id;\n+\n+\trqwqe_id = wq-\u003ewqe_ctr \u003c\u003c (ilog2(xdev-\u003ecaps.recv_ds_num));\n+\t/* reverse wqe index to ds index */\n+\txsc_update_rx_db(xdev, rq-\u003erqn, rqwqe_id);\n+}\n+\n+static void xsc_skb_set_hash(struct xsc_adapter *adapter,\n+\t\t\t struct xsc_cqe *cqe,\n+\t\t\t struct sk_buff *skb)\n+{\n+\tstruct xsc_rss_params *rss = \u0026adapter-\u003erss_param;\n+\tbool l3_hash = false;\n+\tbool l4_hash = false;\n+\tu32 hash_field;\n+\tint ht = 0;\n+\n+\tif (adapter-\u003enetdev-\u003efeatures \u0026 NETIF_F_RXHASH) {\n+\t\tif (skb-\u003eprotocol == htons(ETH_P_IP)) {\n+\t\t\thash_field = rss-\u003erx_hash_fields[XSC_TT_IPV4_TCP];\n+\t\t\tif (hash_field \u0026 XSC_HASH_FIELD_SEL_SRC_IP ||\n+\t\t\t hash_field \u0026 XSC_HASH_FIELD_SEL_DST_IP)\n+\t\t\t\tl3_hash = true;\n+\n+\t\t\tif (hash_field \u0026 XSC_HASH_FIELD_SEL_SPORT ||\n+\t\t\t hash_field \u0026 XSC_HASH_FIELD_SEL_DPORT)\n+\t\t\t\tl4_hash = true;\n+\t\t} else if (skb-\u003eprotocol == htons(ETH_P_IPV6)) {\n+\t\t\thash_field = rss-\u003erx_hash_fields[XSC_TT_IPV6_TCP];\n+\t\t\tif (hash_field \u0026 XSC_HASH_FIELD_SEL_SRC_IPV6 ||\n+\t\t\t hash_field \u0026 XSC_HASH_FIELD_SEL_DST_IPV6)\n+\t\t\t\tl3_hash = true;\n+\n+\t\t\tif (hash_field \u0026 XSC_HASH_FIELD_SEL_SPORT_V6 ||\n+\t\t\t hash_field \u0026 XSC_HASH_FIELD_SEL_DPORT_V6)\n+\t\t\t\tl4_hash = true;\n+\t\t}\n+\n+\t\tif (l3_hash \u0026\u0026 l4_hash)\n+\t\t\tht = PKT_HASH_TYPE_L4;\n+\t\telse if (l3_hash)\n+\t\t\tht = PKT_HASH_TYPE_L3;\n+\t\tif (ht)\n+\t\t\tskb_set_hash(skb, le32_to_cpu(cqe-\u003evni), ht);\n+\t}\n+}\n+\n+static void xsc_handle_csum(struct xsc_cqe *cqe, struct xsc_rq *rq,\n+\t\t\t struct sk_buff *skb, struct xsc_wqe_frag_info *wi)\n+{\n+\tstruct xsc_dma_info *dma_info;\n+\tstruct net_device *netdev;\n+\tvoid *hw_pph;\n+\tstruct xsc_channel *c;\n+\tint offset_from;\n+\n+\tc = rq-\u003ecq.channel;\n+\tnetdev = c-\u003eadapter-\u003enetdev;\n+\tdma_info = wi-\u003edi;\n+\toffset_from = wi-\u003eoffset;\n+\thw_pph = page_address(dma_info-\u003epage) + offset_from;\n+\n+\tif (unlikely((netdev-\u003efeatures \u0026 NETIF_F_RXCSUM) == 0))\n+\t\tgoto csum_none;\n+\n+\tif (unlikely(XSC_GET_EPP2SOC_PPH_ERROR_BITMAP(hw_pph) \u0026 PACKET_UNKNOWN))\n+\t\tgoto csum_none;\n+\n+\tif (XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(hw_pph) \u0026\u0026\n+\t (!(FIELD_GET(XSC_CQE_CSUM_ERR_MASK, le32_to_cpu(cqe-\u003edata0)) \u0026\n+\t OUTER_AND_INNER))) {\n+\t\tskb-\u003eip_summed = CHECKSUM_UNNECESSARY;\n+\t\tskb-\u003ecsum_level = 1;\n+\t\tskb-\u003eencapsulation = 1;\n+\t} else if (XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(hw_pph) \u0026\u0026\n+\t\t (!(FIELD_GET(XSC_CQE_CSUM_ERR_MASK,\n+\t\t\t\tle32_to_cpu(cqe-\u003edata0)) \u0026\n+\t\t OUTER_BIT) \u0026\u0026\n+\t\t (FIELD_GET(XSC_CQE_CSUM_ERR_MASK,\n+\t\t\t le32_to_cpu(cqe-\u003edata0)) \u0026\n+\t\t INNER_BIT))) {\n+\t\tskb-\u003eip_summed = CHECKSUM_UNNECESSARY;\n+\t\tskb-\u003ecsum_level = 0;\n+\t\tskb-\u003eencapsulation = 1;\n+\t} else if (!XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(hw_pph) \u0026\u0026\n+\t\t (!(FIELD_GET(XSC_CQE_CSUM_ERR_MASK,\n+\t\t\t\tle32_to_cpu(cqe-\u003edata0)) \u0026\n+\t\t OUTER_BIT))) {\n+\t\tskb-\u003eip_summed = CHECKSUM_UNNECESSARY;\n+\t}\n+\n+\tgoto out;\n+\n+csum_none:\n+\tskb-\u003ecsum = 0;\n+\tskb-\u003eip_summed = CHECKSUM_NONE;\n+out:\n+\treturn;\n+}\n+\n+static void xsc_build_rx_skb(struct xsc_cqe *cqe,\n+\t\t\t u32 cqe_bcnt,\n+\t\t\t struct xsc_rq *rq,\n+\t\t\t struct sk_buff *skb,\n+\t\t\t struct xsc_wqe_frag_info *wi)\n+{\n+\tstruct xsc_adapter *adapter;\n+\tstruct net_device *netdev;\n+\tstruct xsc_channel *c;\n+\n+\tc = rq-\u003ecq.channel;\n+\tadapter = c-\u003eadapter;\n+\tnetdev = c-\u003enetdev;\n+\n+\tskb-\u003emac_len = ETH_HLEN;\n+\n+\tskb_record_rx_queue(skb, rq-\u003eix);\n+\txsc_handle_csum(cqe, rq, skb, wi);\n+\n+\tskb-\u003eprotocol = eth_type_trans(skb, netdev);\n+\txsc_skb_set_hash(adapter, cqe, skb);\n+}\n+\n+static void xsc_complete_rx_cqe(struct xsc_rq *rq,\n+\t\t\t\tstruct xsc_cqe *cqe,\n+\t\t\t\tu32 cqe_bcnt,\n+\t\t\t\tstruct sk_buff *skb,\n+\t\t\t\tstruct xsc_wqe_frag_info *wi)\n+{\n+\tstruct xsc_rq_stats *stats = rq-\u003estats;\n+\n+\tstats-\u003epackets++;\n+\tstats-\u003ebytes += cqe_bcnt;\n+\txsc_build_rx_skb(cqe, cqe_bcnt, rq, skb, wi);\n+}\n+\n+static void xsc_add_skb_frag(struct xsc_rq *rq,\n+\t\t\t struct sk_buff *skb,\n+\t\t\t struct xsc_dma_info *di,\n+\t\t\t u32 frag_offset, u32 len,\n+\t\t\t unsigned int truesize)\n+{\n+\tstruct xsc_channel *c = rq-\u003ecq.channel;\n+\tstruct device *dev = c-\u003eadapter-\u003edev;\n+\n+\tdma_sync_single_for_cpu(dev, di-\u003eaddr + frag_offset,\n+\t\t\t\tlen, DMA_FROM_DEVICE);\n+\tpage_ref_inc(di-\u003epage);\n+\tskb_add_rx_frag(skb, skb_shinfo(skb)-\u003enr_frags,\n+\t\t\tdi-\u003epage, frag_offset, len, truesize);\n+}\n+\n+static void xsc_copy_skb_header(struct device *dev,\n+\t\t\t\tstruct sk_buff *skb,\n+\t\t\t\tstruct xsc_dma_info *dma_info,\n+\t\t\t\tint offset_from, u32 headlen)\n+{\n+\tvoid *from = page_address(dma_info-\u003epage) + offset_from;\n+\t/* Aligning len to sizeof(long) optimizes memcpy performance */\n+\tunsigned int len = ALIGN(headlen, sizeof(long));\n+\n+\tdma_sync_single_for_cpu(dev, dma_info-\u003eaddr + offset_from, len,\n+\t\t\t\tDMA_FROM_DEVICE);\n+\tskb_copy_to_linear_data(skb, from, len);\n+}\n+\n+static struct sk_buff *xsc_build_linear_skb(struct xsc_rq *rq, void *va,\n+\t\t\t\t\t u32 frag_size, u16 headroom,\n+\t\t\t\t\t u32 cqe_bcnt)\n+{\n+\tstruct sk_buff *skb = build_skb(va, frag_size);\n+\n+\tif (unlikely(!skb))\n+\t\treturn NULL;\n+\n+\tskb_reserve(skb, headroom);\n+\tskb_put(skb, cqe_bcnt);\n+\n+\treturn skb;\n+}\n+\n+struct sk_buff *xsc_skb_from_cqe_linear(struct xsc_rq *rq,\n+\t\t\t\t\tstruct xsc_wqe_frag_info *wi,\n+\t\t\t\t\tu32 cqe_bcnt, u8 has_pph)\n+{\n+\tint pph_len = has_pph ? XSC_PPH_HEAD_LEN : 0;\n+\tu16 rx_headroom = rq-\u003ebuff.headroom;\n+\tstruct xsc_dma_info *di = wi-\u003edi;\n+\tstruct sk_buff *skb;\n+\tvoid *va, *data;\n+\tu32 frag_size;\n+\n+\tva = page_address(di-\u003epage) + wi-\u003eoffset;\n+\tdata = va + rx_headroom + pph_len;\n+\tfrag_size = XSC_SKB_FRAG_SZ(rx_headroom + cqe_bcnt);\n+\n+\tdma_sync_single_range_for_cpu(rq-\u003ecq.xdev-\u003edevice, di-\u003eaddr, wi-\u003eoffset,\n+\t\t\t\t frag_size, DMA_FROM_DEVICE);\n+\tnet_prefetchw(va); /* xdp_frame data area */\n+\tnet_prefetch(data);\n+\n+\tskb = xsc_build_linear_skb(rq, va, frag_size, (rx_headroom + pph_len),\n+\t\t\t\t (cqe_bcnt - pph_len));\n+\tif (unlikely(!skb))\n+\t\treturn NULL;\n+\n+\t/* queue up for recycling/reuse */\n+\tpage_ref_inc(di-\u003epage);\n+\n+\treturn skb;\n+}\n+\n+struct sk_buff *xsc_skb_from_cqe_nonlinear(struct xsc_rq *rq,\n+\t\t\t\t\t struct xsc_wqe_frag_info *wi,\n+\t\t\t\t\t u32 cqe_bcnt, u8 has_pph)\n+{\n+\tstruct xsc_rq_frag_info *frag_info = \u0026rq-\u003ewqe.info.arr[0];\n+\tu16 headlen = min_t(u32, XSC_RX_MAX_HEAD, cqe_bcnt);\n+\tstruct xsc_wqe_frag_info *head_wi = wi;\n+\tstruct xsc_wqe_frag_info *rx_wi = wi;\n+\tu16 head_offset = head_wi-\u003eoffset;\n+\tu16 byte_cnt = cqe_bcnt - headlen;\n+\tu16 frag_consumed_bytes = 0;\n+\tu16 frag_headlen = headlen;\n+\tstruct net_device *netdev;\n+\tstruct xsc_channel *c;\n+\tstruct sk_buff *skb;\n+\tstruct device *dev;\n+\tu8 fragcnt = 0;\n+\tint i = 0;\n+\n+\tc = rq-\u003ecq.channel;\n+\tdev = c-\u003eadapter-\u003edev;\n+\tnetdev = c-\u003eadapter-\u003enetdev;\n+\n+\tskb = napi_alloc_skb(rq-\u003ecq.napi, ALIGN(XSC_RX_MAX_HEAD, sizeof(long)));\n+\tif (unlikely(!skb))\n+\t\treturn NULL;\n+\n+\tnet_prefetchw(skb-\u003edata);\n+\n+\tif (likely(has_pph)) {\n+\t\theadlen = min_t(u32, XSC_RX_MAX_HEAD,\n+\t\t\t\t(cqe_bcnt - XSC_PPH_HEAD_LEN));\n+\t\tfrag_headlen = headlen + XSC_PPH_HEAD_LEN;\n+\t\tbyte_cnt = cqe_bcnt - headlen - XSC_PPH_HEAD_LEN;\n+\t\thead_offset += XSC_PPH_HEAD_LEN;\n+\t}\n+\n+\tfor (i = 0; i \u003c rq-\u003ewqe.info.num_frags; i++, rx_wi++)\n+\t\trx_wi-\u003eis_available = 0;\n+\n+\twhile (byte_cnt) {\n+\t\t/*figure out whether the first fragment can be a page ?*/\n+\t\tfrag_consumed_bytes =\n+\t\t\tmin_t(u16, frag_info-\u003efrag_size - frag_headlen,\n+\t\t\t byte_cnt);\n+\n+\t\txsc_add_skb_frag(rq, skb, wi-\u003edi, wi-\u003eoffset + frag_headlen,\n+\t\t\t\t frag_consumed_bytes, frag_info-\u003efrag_stride);\n+\t\tbyte_cnt -= frag_consumed_bytes;\n+\n+\t\t/*to protect extend wqe read, drop exceed bytes*/\n+\t\tfrag_headlen = 0;\n+\t\tfragcnt++;\n+\t\tif (fragcnt == rq-\u003ewqe.info.num_frags) {\n+\t\t\tif (byte_cnt) {\n+\t\t\t\tnetdev_warn(netdev,\n+\t\t\t\t\t \"large packet reach the maximum rev-wqe num.\\n\");\n+\t\t\t\tnetdev_warn(netdev,\n+\t\t\t\t\t \"%u bytes dropped: frag_num=%d, headlen=%d, cqe_cnt=%d, frag0_bytes=%d, frag_size=%d\\n\",\n+\t\t\t\t\t byte_cnt, fragcnt,\n+\t\t\t\t\t headlen, cqe_bcnt,\n+\t\t\t\t\t frag_consumed_bytes,\n+\t\t\t\t\t frag_info-\u003efrag_size);\n+\t\t\t}\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tfrag_info++;\n+\t\twi++;\n+\t}\n+\n+\t/* copy header */\n+\txsc_copy_skb_header(dev, skb, head_wi-\u003edi, head_offset, headlen);\n+\n+\t/* skb linear part was allocated with headlen and aligned to long */\n+\tskb-\u003etail += headlen;\n+\tskb-\u003elen += headlen;\n+\n+\treturn skb;\n+}\n+\n+static void xsc_put_rx_frag(struct xsc_rq *rq,\n+\t\t\t struct xsc_wqe_frag_info *frag, bool recycle)\n+{\n+\tif (frag-\u003elast_in_page)\n+\t\tpage_pool_recycle_direct(rq-\u003epage_pool, frag-\u003edi-\u003epage);\n+}\n+\n+static struct xsc_wqe_frag_info *get_frag(struct xsc_rq *rq, u16 ix)\n+{\n+\treturn \u0026rq-\u003ewqe.frags[ix \u003c\u003c rq-\u003ewqe.info.log_num_frags];\n+}\n+\n+static void xsc_free_rx_wqe(struct xsc_rq *rq,\n+\t\t\t struct xsc_wqe_frag_info *wi, bool recycle)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i \u003c rq-\u003ewqe.info.num_frags; i++, wi++) {\n+\t\tif (wi-\u003eis_available \u0026\u0026 recycle)\n+\t\t\tcontinue;\n+\t\txsc_put_rx_frag(rq, wi, recycle);\n+\t}\n+}\n+\n+static void xsc_dump_error_rqcqe(struct xsc_rq *rq,\n+\t\t\t\t struct xsc_cqe *cqe)\n+{\n+\tstruct net_device *netdev;\n+\tstruct xsc_channel *c;\n+\tu32 ci;\n+\n+\tc = rq-\u003ecq.channel;\n+\tnetdev = c-\u003eadapter-\u003enetdev;\n+\tci = xsc_cqwq_get_ci(\u0026rq-\u003ecq.wq);\n+\n+\tnet_err_ratelimited(\"Error cqe on dev=%s, cqn=%d, ci=%d, rqn=%d, qpn=%ld, error_code=0x%x\\n\",\n+\t\t\t netdev-\u003ename, rq-\u003ecq.xcq.cqn, ci,\n+\t\t\t rq-\u003erqn,\n+\t\t\t FIELD_GET(XSC_CQE_QP_ID_MASK,\n+\t\t\t\t le32_to_cpu(cqe-\u003edata0)),\n+\t\t\t get_cqe_opcode(cqe));\n+}\n+\n+void xsc_eth_handle_rx_cqe(struct xsc_cqwq *cqwq,\n+\t\t\t struct xsc_rq *rq, struct xsc_cqe *cqe)\n+{\n+\tstruct xsc_wq_cyc *wq = \u0026rq-\u003ewqe.wq;\n+\tu8 cqe_opcode = get_cqe_opcode(cqe);\n+\tstruct xsc_wqe_frag_info *wi;\n+\tstruct sk_buff *skb;\n+\tu32 cqe_bcnt;\n+\tu16 ci;\n+\n+\tci = xsc_wq_cyc_ctr2ix(wq, cqwq-\u003ecc);\n+\twi = get_frag(rq, ci);\n+\tif (unlikely(cqe_opcode \u0026 BIT(7))) {\n+\t\txsc_dump_error_rqcqe(rq, cqe);\n+\t\tgoto free_wqe;\n+\t}\n+\n+\tcqe_bcnt = le32_to_cpu(cqe-\u003emsg_len);\n+\tif ((le32_to_cpu(cqe-\u003edata0) \u0026 XSC_CQE_HAS_PPH) \u0026\u0026\n+\t cqe_bcnt \u003c= XSC_PPH_HEAD_LEN)\n+\t\tgoto free_wqe;\n+\n+\tif (unlikely(cqe_bcnt \u003e rq-\u003efrags_sz))\n+\t\tgoto free_wqe;\n+\n+\tcqe_bcnt = min_t(u32, cqe_bcnt, rq-\u003efrags_sz);\n+\tskb = rq-\u003ewqe.skb_from_cqe(rq, wi, cqe_bcnt,\n+\t\t\t\t !!(le32_to_cpu(cqe-\u003edata0) \u0026\n+\t\t\t\t XSC_CQE_HAS_PPH));\n+\tif (!skb)\n+\t\tgoto free_wqe;\n+\n+\txsc_complete_rx_cqe(rq, cqe,\n+\t\t\t (le32_to_cpu(cqe-\u003edata0) \u0026 XSC_CQE_HAS_PPH) ?\n+\t\t\t cqe_bcnt - XSC_PPH_HEAD_LEN : cqe_bcnt,\n+\t\t\t skb, wi);\n+\n+\tnapi_gro_receive(rq-\u003ecq.napi, skb);\n+\n+free_wqe:\n+\txsc_free_rx_wqe(rq, wi, true);\n+\txsc_wq_cyc_pop(wq);\n+}\n+\n+int xsc_poll_rx_cq(struct xsc_cq *cq, int budget)\n+{\n+\tstruct xsc_rq *rq = container_of(cq, struct xsc_rq, cq);\n+\tstruct xsc_cqwq *cqwq = \u0026cq-\u003ewq;\n+\tstruct xsc_cqe *cqe;\n+\tint work_done = 0;\n+\n+\tif (!test_bit(XSC_ETH_RQ_STATE_ENABLED, \u0026rq-\u003estate))\n+\t\treturn 0;\n+\n+\twhile ((work_done \u003c budget) \u0026\u0026 (cqe = xsc_cqwq_get_cqe(cqwq))) {\n+\t\trq-\u003ehandle_rx_cqe(cqwq, rq, cqe);\n+\t\t++work_done;\n+\n+\t\txsc_cqwq_pop(cqwq);\n+\t}\n+\n+\tif (!work_done)\n+\t\tgoto out;\n+\n+\txsc_cq_notify_hw(cq);\n+\t/* ensure cq space is freed before enabling more cqes */\n+\twmb();\n+\n+out:\n+\treturn work_done;\n+}\n+\n+static int xsc_page_alloc_pool(struct xsc_rq *rq,\n+\t\t\t struct xsc_dma_info *dma_info)\n+{\n+\tdma_info-\u003epage = page_pool_dev_alloc_pages(rq-\u003epage_pool);\n+\tif (unlikely(!dma_info-\u003epage))\n+\t\treturn -ENOMEM;\n+\tdma_info-\u003eaddr = page_pool_get_dma_addr(dma_info-\u003epage);\n+\n+\treturn 0;\n+}\n+\n+static int xsc_get_rx_frag(struct xsc_rq *rq,\n+\t\t\t struct xsc_wqe_frag_info *frag)\n+{\n+\tint err = 0;\n+\n+\tif (!frag-\u003eoffset \u0026\u0026 !frag-\u003eis_available)\n+\t\t/* On first frag (offset == 0), replenish page (dma_info\n+\t\t * actually). Other frags that point to the same dma_info\n+\t\t * (with a different offset) should just use the new one\n+\t\t * without replenishing again by themselves.\n+\t\t */\n+\t\terr = xsc_page_alloc_pool(rq, frag-\u003edi);\n+\n+\treturn err;\n+}\n+\n+static int xsc_alloc_rx_wqe(struct xsc_rq *rq,\n+\t\t\t struct xsc_eth_rx_wqe_cyc *wqe,\n+\t\t\t u16 ix)\n+{\n+\tstruct xsc_wqe_frag_info *frag = get_frag(rq, ix);\n+\tu64 addr;\n+\tint err;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c rq-\u003ewqe.info.num_frags; i++, frag++) {\n+\t\terr = xsc_get_rx_frag(rq, frag);\n+\t\tif (unlikely(err))\n+\t\t\tgoto err_free_frags;\n+\n+\t\taddr = frag-\u003edi-\u003eaddr + frag-\u003eoffset + rq-\u003ebuff.headroom;\n+\t\twqe-\u003edata[i].va = cpu_to_le64(addr);\n+\t}\n+\n+\treturn 0;\n+\n+err_free_frags:\n+\twhile (--i \u003e= 0)\n+\t\txsc_put_rx_frag(rq, --frag, true);\n+\n+\treturn err;\n+}\n+\n+void xsc_eth_dealloc_rx_wqe(struct xsc_rq *rq, u16 ix)\n+{\n+\tstruct xsc_wqe_frag_info *wi = get_frag(rq, ix);\n+\n+\txsc_free_rx_wqe(rq, wi, false);\n+}\n+\n+static int xsc_alloc_rx_wqes(struct xsc_rq *rq, u16 ix, u8 wqe_bulk)\n+{\n+\tstruct xsc_wq_cyc *wq = \u0026rq-\u003ewqe.wq;\n+\tstruct xsc_eth_rx_wqe_cyc *wqe;\n+\tint err;\n+\tint idx;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c wqe_bulk; i++) {\n+\t\tidx = xsc_wq_cyc_ctr2ix(wq, (ix + i));\n+\t\twqe = xsc_wq_cyc_get_wqe(wq, idx);\n+\n+\t\terr = xsc_alloc_rx_wqe(rq, wqe, idx);\n+\t\tif (unlikely(err))\n+\t\t\tgoto err_free_wqes;\n+\t}\n+\n+\treturn 0;\n+\n+err_free_wqes:\n+\twhile (--i \u003e= 0)\n+\t\txsc_eth_dealloc_rx_wqe(rq, ix + i);\n+\n+\treturn err;\n+}\n+\n+bool xsc_eth_post_rx_wqes(struct xsc_rq *rq)\n+{\n+\tstruct xsc_wq_cyc *wq = \u0026rq-\u003ewqe.wq;\n+\tu8 wqe_bulk, wqe_bulk_min;\n+\tint err = 0;\n+\tint alloc;\n+\tu16 head;\n+\n+\twqe_bulk = rq-\u003ewqe.info.wqe_bulk;\n+\twqe_bulk_min = rq-\u003ewqe.info.wqe_bulk_min;\n+\tif (xsc_wq_cyc_missing(wq) \u003c wqe_bulk)\n+\t\treturn false;\n+\n+\tdo {\n+\t\thead = xsc_wq_cyc_get_head(wq);\n+\n+\t\talloc = min_t(int, wqe_bulk, xsc_wq_cyc_missing(wq));\n+\t\tif (alloc \u003c wqe_bulk \u0026\u0026 alloc \u003e= wqe_bulk_min)\n+\t\t\talloc = alloc \u0026 0xfffffffe;\n+\n+\t\tif (alloc \u003e 0) {\n+\t\t\terr = xsc_alloc_rx_wqes(rq, head, alloc);\n+\t\t\tif (unlikely(err))\n+\t\t\t\tbreak;\n+\n+\t\t\txsc_wq_cyc_push_n(wq, alloc);\n+\t\t}\n+\t} while (xsc_wq_cyc_missing(wq) \u003e= wqe_bulk_min);\n+\n+\tdma_wmb();\n+\n+\t/* ensure wqes are visible to device before updating doorbell record */\n+\txsc_rq_notify_hw(rq);\n+\n+\treturn !!err;\n+}\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.c\nnew file mode 100644\nindex 0000000000000..b466fc1f026d4\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.c\n@@ -0,0 +1,39 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \"xsc_eth_stats.h\"\n+#include \"xsc_eth.h\"\n+\n+static int xsc_get_netdev_max_channels(struct xsc_adapter *adapter)\n+{\n+\tstruct net_device *netdev = adapter-\u003enetdev;\n+\n+\treturn min_t(unsigned int, netdev-\u003enum_rx_queues,\n+\t\t netdev-\u003enum_tx_queues);\n+}\n+\n+void xsc_eth_fold_sw_stats64(struct xsc_adapter *adapter,\n+\t\t\t struct rtnl_link_stats64 *s)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i \u003c xsc_get_netdev_max_channels(adapter); i++) {\n+\t\tstruct xsc_channel_stats *channel_stats;\n+\t\tstruct xsc_rq_stats *rq_stats;\n+\t\tstruct xsc_sq_stats *sq_stats;\n+\n+\t\tchannel_stats = \u0026adapter-\u003estats-\u003echannel_stats[i];\n+\t\trq_stats = \u0026channel_stats-\u003erq;\n+\t\tsq_stats = \u0026channel_stats-\u003esq;\n+\n+\t\ts-\u003erx_packets += rq_stats-\u003epackets;\n+\t\ts-\u003erx_bytes += rq_stats-\u003ebytes;\n+\n+\t\ts-\u003etx_packets += sq_stats-\u003epackets;\n+\t\ts-\u003etx_bytes += sq_stats-\u003ebytes;\n+\t\ts-\u003etx_dropped += sq_stats-\u003edropped;\n+\t}\n+}\n+\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.h\nnew file mode 100644\nindex 0000000000000..a391afe97b418\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.h\n@@ -0,0 +1,34 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_EN_STATS_H\n+#define __XSC_EN_STATS_H\n+\n+#include \"xsc_eth_common.h\"\n+\n+struct xsc_rq_stats {\n+\tu64 packets;\n+\tu64 bytes;\n+};\n+\n+struct xsc_sq_stats {\n+\tu64 packets;\n+\tu64 bytes;\n+\tu64 dropped;\n+};\n+\n+struct xsc_channel_stats {\n+\tstruct xsc_sq_stats sq;\n+\tstruct xsc_rq_stats rq;\n+} ____cacheline_aligned_in_smp;\n+\n+struct xsc_stats {\n+\tstruct xsc_channel_stats channel_stats[XSC_ETH_MAX_NUM_CHANNELS];\n+};\n+\n+void xsc_eth_fold_sw_stats64(struct xsc_adapter *adapter,\n+\t\t\t struct rtnl_link_stats64 *s);\n+\n+#endif /* XSC_EN_STATS_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_tx.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_tx.c\nnew file mode 100644\nindex 0000000000000..7495b12b7b442\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_tx.c\n@@ -0,0 +1,309 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/tcp.h\u003e\n+\n+#include \"xsc_eth.h\"\n+#include \"xsc_eth_txrx.h\"\n+\n+#define XSC_OPCODE_RAW 7\n+\n+static void xsc_dma_push(struct xsc_sq *sq, dma_addr_t addr, u32 size,\n+\t\t\t enum xsc_dma_map_type map_type)\n+{\n+\tstruct xsc_sq_dma *dma = xsc_dma_get(sq, sq-\u003edma_fifo_pc++);\n+\n+\tdma-\u003eaddr = addr;\n+\tdma-\u003esize = size;\n+\tdma-\u003etype = map_type;\n+}\n+\n+static void xsc_dma_unmap_wqe(struct xsc_sq *sq, u8 num_dma)\n+{\n+\tstruct xsc_adapter *adapter = sq-\u003echannel-\u003eadapter;\n+\tstruct device *dev = adapter-\u003edev;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c num_dma; i++) {\n+\t\tstruct xsc_sq_dma *last_pushed_dma;\n+\n+\t\tlast_pushed_dma = xsc_dma_get(sq, --sq-\u003edma_fifo_pc);\n+\t\txsc_tx_dma_unmap(dev, last_pushed_dma);\n+\t}\n+}\n+\n+static void *xsc_sq_fetch_wqe(struct xsc_sq *sq, size_t size, u16 *pi)\n+{\n+\tstruct xsc_wq_cyc *wq = \u0026sq-\u003ewq;\n+\tvoid *wqe;\n+\n+\t/*caution, sp-\u003epc is default to be zero*/\n+\t*pi = xsc_wq_cyc_ctr2ix(wq, sq-\u003epc);\n+\twqe = xsc_wq_cyc_get_wqe(wq, *pi);\n+\tmemset(wqe, 0, size);\n+\n+\treturn wqe;\n+}\n+\n+static u16 xsc_tx_get_gso_ihs(struct xsc_sq *sq, struct sk_buff *skb)\n+{\n+\tu16 ihs;\n+\n+\tif (skb-\u003eencapsulation) {\n+\t\tihs = skb_inner_transport_offset(skb) + inner_tcp_hdrlen(skb);\n+\t} else {\n+\t\tif (skb_shinfo(skb)-\u003egso_type \u0026 SKB_GSO_UDP_L4)\n+\t\t\tihs = skb_transport_offset(skb) + sizeof(struct udphdr);\n+\t\telse\n+\t\t\tihs = skb_transport_offset(skb) + tcp_hdrlen(skb);\n+\t}\n+\n+\treturn ihs;\n+}\n+\n+static void xsc_txwqe_build_cseg_csum(struct xsc_sq *sq,\n+\t\t\t\t struct sk_buff *skb,\n+\t\t\t\t struct xsc_send_wqe_ctrl_seg *cseg)\n+{\n+\tu32 val = le32_to_cpu(cseg-\u003edata0);\n+\n+\tif (likely(skb-\u003eip_summed == CHECKSUM_PARTIAL)) {\n+\t\tif (skb-\u003eencapsulation)\n+\t\t\tval |= FIELD_PREP(XSC_WQE_CTRL_SEG_CSUM_EN_MASK,\n+\t\t\t\t\t XSC_ETH_WQE_INNER_AND_OUTER_CSUM);\n+\t\telse\n+\t\t\tval |= FIELD_PREP(XSC_WQE_CTRL_SEG_CSUM_EN_MASK,\n+\t\t\t\t\t XSC_ETH_WQE_OUTER_CSUM);\n+\t} else {\n+\t\tval |= FIELD_PREP(XSC_WQE_CTRL_SEG_CSUM_EN_MASK,\n+\t\t\t\t XSC_ETH_WQE_NONE_CSUM);\n+\t}\n+\tcseg-\u003edata0 = cpu_to_le32(val);\n+}\n+\n+static void xsc_txwqe_build_csegs(struct xsc_sq *sq, struct sk_buff *skb,\n+\t\t\t\t u16 mss, u16 ihs, u16 headlen,\n+\t\t\t\t u8 opcode, u16 ds_cnt, u32 msglen,\n+\t\t\t\t struct xsc_send_wqe_ctrl_seg *cseg)\n+{\n+\tstruct xsc_core_device *xdev = sq-\u003ecq.xdev;\n+\tint send_wqe_ds_num_log;\n+\tu32 val = 0;\n+\n+\tsend_wqe_ds_num_log = ilog2(xdev-\u003ecaps.send_ds_num);\n+\txsc_txwqe_build_cseg_csum(sq, skb, cseg);\n+\n+\tif (mss != 0) {\n+\t\tval |= XSC_WQE_CTRL_SEG_HAS_PPH |\n+\t\t XSC_WQE_CTRL_SEG_SO_TYPE |\n+\t\t FIELD_PREP(XSC_WQE_CTRL_SEG_SO_HDR_LEN_MASK, ihs) |\n+\t\t FIELD_PREP(XSC_WQE_CTRL_SEG_SO_DATA_SIZE_MASK, mss);\n+\t\tcseg-\u003edata2 = cpu_to_le32(val);\n+\t}\n+\n+\tval = le32_to_cpu(cseg-\u003edata0);\n+\tval |= FIELD_PREP(XSC_WQE_CTRL_SEG_MSG_OPCODE_MASK, opcode) |\n+\t FIELD_PREP(XSC_WQE_CTRL_SEG_WQE_ID_MASK,\n+\t\t\t sq-\u003epc \u003c\u003c send_wqe_ds_num_log) |\n+\t FIELD_PREP(XSC_WQE_CTRL_SEG_DS_DATA_NUM_MASK,\n+\t\t\t ds_cnt - XSC_SEND_WQEBB_CTRL_NUM_DS);\n+\tcseg-\u003edata0 = cpu_to_le32(val);\n+\tcseg-\u003emsg_len = cpu_to_le32(msglen);\n+\tcseg-\u003edata3 = cpu_to_le32(XSC_WQE_CTRL_SEG_CE);\n+}\n+\n+static int xsc_txwqe_build_dsegs(struct xsc_sq *sq, struct sk_buff *skb,\n+\t\t\t\t u16 ihs, u16 headlen,\n+\t\t\t\t struct xsc_wqe_data_seg *dseg)\n+{\n+\tstruct xsc_adapter *adapter = sq-\u003echannel-\u003eadapter;\n+\tstruct device *dev = adapter-\u003edev;\n+\tdma_addr_t dma_addr = 0;\n+\tu8 num_dma = 0;\n+\tint i;\n+\n+\tif (headlen) {\n+\t\tdma_addr = dma_map_single(dev,\n+\t\t\t\t\t skb-\u003edata,\n+\t\t\t\t\t headlen,\n+\t\t\t\t\t DMA_TO_DEVICE);\n+\t\tif (unlikely(dma_mapping_error(dev, dma_addr)))\n+\t\t\tgoto err_dma_unmap_wqe;\n+\n+\t\tdseg-\u003eva = cpu_to_le64(dma_addr);\n+\t\tdseg-\u003emkey = cpu_to_le32(be32_to_cpu(sq-\u003emkey_be));\n+\t\tdseg-\u003edata0 |=\n+\t\t\tcpu_to_le32(FIELD_PREP(XSC_WQE_DATA_SEG_SEG_LEN_MASK,\n+\t\t\t\t\t headlen));\n+\n+\t\txsc_dma_push(sq, dma_addr, headlen, XSC_DMA_MAP_SINGLE);\n+\t\tnum_dma++;\n+\t\tdseg++;\n+\t}\n+\n+\tfor (i = 0; i \u003c skb_shinfo(skb)-\u003enr_frags; i++) {\n+\t\tskb_frag_t *frag = \u0026skb_shinfo(skb)-\u003efrags[i];\n+\t\tint fsz = skb_frag_size(frag);\n+\n+\t\tdma_addr = skb_frag_dma_map(dev, frag, 0, fsz, DMA_TO_DEVICE);\n+\t\tif (unlikely(dma_mapping_error(dev, dma_addr)))\n+\t\t\tgoto err_dma_unmap_wqe;\n+\n+\t\tdseg-\u003eva = cpu_to_le64(dma_addr);\n+\t\tdseg-\u003emkey = cpu_to_le32(be32_to_cpu(sq-\u003emkey_be));\n+\t\tdseg-\u003edata0 |=\n+\t\t\tcpu_to_le32(FIELD_PREP(XSC_WQE_DATA_SEG_SEG_LEN_MASK,\n+\t\t\t\t\t fsz));\n+\n+\t\txsc_dma_push(sq, dma_addr, fsz, XSC_DMA_MAP_PAGE);\n+\t\tnum_dma++;\n+\t\tdseg++;\n+\t}\n+\n+\treturn num_dma;\n+\n+err_dma_unmap_wqe:\n+\txsc_dma_unmap_wqe(sq, num_dma);\n+\treturn -ENOMEM;\n+}\n+\n+static void xsc_sq_notify_hw(struct xsc_wq_cyc *wq, u16 pc,\n+\t\t\t struct xsc_sq *sq)\n+{\n+\tstruct xsc_adapter *adapter = sq-\u003echannel-\u003eadapter;\n+\tstruct xsc_core_device *xdev = adapter-\u003exdev;\n+\tint send_ds_num_log = ilog2(xdev-\u003ecaps.send_ds_num);\n+\n+\t/* reverse wqe index to ds index */\n+\txsc_update_tx_db(xdev, sq-\u003esqn, pc \u003c\u003c send_ds_num_log);\n+}\n+\n+static void xsc_txwqe_complete(struct xsc_sq *sq, struct sk_buff *skb,\n+\t\t\t u8 opcode, u16 ds_cnt,\n+\t\t\t u8 num_wqebbs, u32 num_bytes, u8 num_dma,\n+\t\t\t struct xsc_tx_wqe_info *wi)\n+{\n+\tstruct xsc_wq_cyc *wq = \u0026sq-\u003ewq;\n+\n+\twi-\u003enum_bytes = num_bytes;\n+\twi-\u003enum_dma = num_dma;\n+\twi-\u003enum_wqebbs = num_wqebbs;\n+\twi-\u003eskb = skb;\n+\n+\tnetdev_tx_sent_queue(sq-\u003etxq, num_bytes);\n+\n+\tsq-\u003epc += wi-\u003enum_wqebbs;\n+\n+\tif (unlikely(!xsc_wqc_has_room_for(wq, sq-\u003ecc, sq-\u003epc, sq-\u003estop_room)))\n+\t\tnetif_tx_stop_queue(sq-\u003etxq);\n+\n+\tif (!netdev_xmit_more() || netif_xmit_stopped(sq-\u003etxq))\n+\t\txsc_sq_notify_hw(wq, sq-\u003epc, sq);\n+}\n+\n+static uint32_t xsc_eth_xmit_frame(struct sk_buff *skb,\n+\t\t\t\t struct xsc_sq *sq,\n+\t\t\t\t struct xsc_tx_wqe *wqe,\n+\t\t\t\t u16 pi)\n+{\n+\tstruct xsc_core_device *xdev = sq-\u003ecq.xdev;\n+\tstruct xsc_sq_stats *stats = sq-\u003estats;\n+\tstruct xsc_send_wqe_ctrl_seg *cseg;\n+\tstruct xsc_wqe_data_seg *dseg;\n+\tstruct xsc_tx_wqe_info *wi;\n+\tu16 mss, ihs, headlen;\n+\tu32 num_bytes;\n+\tu8 num_wqebbs;\n+\tint num_dma;\n+\tu16 ds_cnt;\n+\tu8 opcode;\n+\n+retry_send:\n+\t/* Calc ihs and ds cnt, no writes to wqe yet; ctrl-ds is reduced by\n+\t * ds_data_num.\n+\t */\n+\tds_cnt = XSC_SEND_WQEBB_CTRL_NUM_DS;\n+\n+\tif (skb_is_gso(skb)) {\n+\t\topcode\t\t= XSC_OPCODE_RAW;\n+\t\tmss\t\t= skb_shinfo(skb)-\u003egso_size;\n+\t\tihs\t\t= xsc_tx_get_gso_ihs(sq, skb);\n+\t\tnum_bytes\t= skb-\u003elen +\n+\t\t\t\t (skb_shinfo(skb)-\u003egso_segs - 1) * ihs;\n+\t\tstats-\u003epackets\t+= skb_shinfo(skb)-\u003egso_segs;\n+\t} else {\n+\t\topcode\t\t= XSC_OPCODE_RAW;\n+\t\tmss\t\t= 0;\n+\t\tihs\t\t= 0;\n+\t\tnum_bytes\t= skb-\u003elen;\n+\t\tstats-\u003epackets++;\n+\t}\n+\n+\t/*linear data in skb*/\n+\theadlen = skb-\u003elen - skb-\u003edata_len;\n+\tds_cnt += !!headlen;\n+\tds_cnt += skb_shinfo(skb)-\u003enr_frags;\n+\n+\t/* Check packet size. */\n+\tif (unlikely(mss == 0 \u0026\u0026 skb-\u003elen \u003e sq-\u003ehw_mtu))\n+\t\tgoto err_drop;\n+\n+\tnum_wqebbs = DIV_ROUND_UP(ds_cnt, xdev-\u003ecaps.send_ds_num);\n+\t/*if ds_cnt exceed one wqe, drop it*/\n+\tif (num_wqebbs != 1) {\n+\t\tif (skb_linearize(skb))\n+\t\t\tgoto err_drop;\n+\t\tgoto retry_send;\n+\t}\n+\n+\t/* fill wqe */\n+\twi = (struct xsc_tx_wqe_info *)\u0026sq-\u003edb.wqe_info[pi];\n+\tcseg = \u0026wqe-\u003ectrl;\n+\tdseg = \u0026wqe-\u003edata[0];\n+\n+\tif (unlikely(skb-\u003elen == 0))\n+\t\tgoto err_drop;\n+\n+\txsc_txwqe_build_csegs(sq, skb, mss, ihs, headlen,\n+\t\t\t opcode, ds_cnt, skb-\u003elen, cseg);\n+\n+\t/*inline header is also use dma to transport*/\n+\tnum_dma = xsc_txwqe_build_dsegs(sq, skb, ihs, headlen, dseg);\n+\tif (unlikely(num_dma \u003c 0))\n+\t\tgoto err_drop;\n+\n+\txsc_txwqe_complete(sq, skb, opcode, ds_cnt, num_wqebbs, num_bytes,\n+\t\t\t num_dma, wi);\n+\tstats-\u003ebytes += num_bytes;\n+\n+\treturn NETDEV_TX_OK;\n+\n+err_drop:\n+\tstats-\u003edropped++;\n+\tdev_kfree_skb_any(skb);\n+\n+\treturn NETDEV_TX_OK;\n+}\n+\n+netdev_tx_t xsc_eth_xmit_start(struct sk_buff *skb, struct net_device *netdev)\n+{\n+\tstruct xsc_adapter *adapter = netdev_priv(netdev);\n+\tstruct xsc_tx_wqe *wqe;\n+\tstruct xsc_sq *sq;\n+\tu32 ds_num;\n+\tu16 pi;\n+\n+\tif (!adapter-\u003eopened)\n+\t\treturn NETDEV_TX_BUSY;\n+\n+\tsq = adapter-\u003etxq2sq[skb_get_queue_mapping(skb)];\n+\tif (unlikely(!sq))\n+\t\treturn NETDEV_TX_BUSY;\n+\n+\tds_num = adapter-\u003exdev-\u003ecaps.send_ds_num;\n+\twqe = xsc_sq_fetch_wqe(sq, ds_num * XSC_SEND_WQE_DS, \u0026pi);\n+\n+\treturn xsc_eth_xmit_frame(skb, sq, wqe, pi);\n+}\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c\nnew file mode 100644\nindex 0000000000000..3effa79efce06\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c\n@@ -0,0 +1,159 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \"xsc_eth_common.h\"\n+#include \"xsc_eth_txrx.h\"\n+\n+void xsc_cq_notify_hw_rearm(struct xsc_cq *cq)\n+{\n+\txsc_arm_cq(cq-\u003exdev, cq-\u003excq.cqn, cq-\u003ewq.cc, 0);\n+}\n+\n+void xsc_cq_notify_hw(struct xsc_cq *cq)\n+{\n+\txsc_update_cq_ci(cq-\u003exdev, cq-\u003excq.cqn, cq-\u003ewq.cc);\n+}\n+\n+static bool xsc_channel_no_affinity_change(struct xsc_channel *c)\n+{\n+\tint current_cpu = smp_processor_id();\n+\n+\treturn cpumask_test_cpu(current_cpu, c-\u003eaff_mask);\n+}\n+\n+static void xsc_dump_error_sqcqe(struct xsc_sq *sq,\n+\t\t\t\t struct xsc_cqe *cqe)\n+{\n+\tstruct net_device *netdev = sq-\u003echannel-\u003enetdev;\n+\tu32 ci = xsc_cqwq_get_ci(\u0026sq-\u003ecq.wq);\n+\n+\tnet_err_ratelimited(\"Err cqe on dev %s cqn=0x%x ci=0x%x sqn=0x%x err_code=0x%x qpid=0x%lx\\n\",\n+\t\t\t netdev-\u003ename, sq-\u003ecq.xcq.cqn, ci,\n+\t\t\t sq-\u003esqn, get_cqe_opcode(cqe),\n+\t\t\t FIELD_GET(XSC_CQE_QP_ID_MASK,\n+\t\t\t\t le32_to_cpu(cqe-\u003edata0)));\n+}\n+\n+static bool xsc_poll_tx_cq(struct xsc_cq *cq, int napi_budget)\n+{\n+\tstruct xsc_adapter *adapter;\n+\tstruct xsc_cqe *cqe;\n+\tstruct device *dev;\n+\tstruct xsc_sq *sq;\n+\tu32 dma_fifo_cc;\n+\tu32 nbytes = 0;\n+\tu16 npkts = 0;\n+\tint i = 0;\n+\tu16 sqcc;\n+\n+\tsq = container_of(cq, struct xsc_sq, cq);\n+\tif (!test_bit(XSC_ETH_SQ_STATE_ENABLED, \u0026sq-\u003estate))\n+\t\treturn false;\n+\n+\tadapter = sq-\u003echannel-\u003eadapter;\n+\tdev = adapter-\u003edev;\n+\n+\tcqe = xsc_cqwq_get_cqe(\u0026cq-\u003ewq);\n+\tif (!cqe)\n+\t\tgoto out;\n+\n+\tif (unlikely(get_cqe_opcode(cqe) \u0026 BIT(7))) {\n+\t\txsc_dump_error_sqcqe(sq, cqe);\n+\t\treturn false;\n+\t}\n+\n+\tsqcc = sq-\u003ecc;\n+\n+\t/* avoid dirtying sq cache line every cqe */\n+\tdma_fifo_cc = sq-\u003edma_fifo_cc;\n+\ti = 0;\n+\tdo {\n+\t\tstruct xsc_tx_wqe_info *wi;\n+\t\tstruct sk_buff *skb;\n+\t\tint j;\n+\t\tu16 ci;\n+\n+\t\txsc_cqwq_pop(\u0026cq-\u003ewq);\n+\n+\t\tci = xsc_wq_cyc_ctr2ix(\u0026sq-\u003ewq, sqcc);\n+\t\twi = \u0026sq-\u003edb.wqe_info[ci];\n+\t\tskb = wi-\u003eskb;\n+\n+\t\t/*cqe may be overstanding in real test, not by nop in other*/\n+\t\tif (unlikely(!skb))\n+\t\t\tcontinue;\n+\n+\t\tfor (j = 0; j \u003c wi-\u003enum_dma; j++) {\n+\t\t\tstruct xsc_sq_dma *dma = xsc_dma_get(sq, dma_fifo_cc++);\n+\n+\t\t\txsc_tx_dma_unmap(dev, dma);\n+\t\t}\n+\n+\t\tnpkts++;\n+\t\tnbytes += wi-\u003enum_bytes;\n+\t\tsqcc += wi-\u003enum_wqebbs;\n+\t\tnapi_consume_skb(skb, 0);\n+\n+\t} while ((++i \u003c= napi_budget) \u0026\u0026 (cqe = xsc_cqwq_get_cqe(\u0026cq-\u003ewq)));\n+\n+\txsc_cq_notify_hw(cq);\n+\n+\t/* ensure cq space is freed before enabling more cqes */\n+\twmb();\n+\n+\tsq-\u003edma_fifo_cc = dma_fifo_cc;\n+\tsq-\u003ecc = sqcc;\n+\n+\tnetdev_tx_completed_queue(sq-\u003etxq, npkts, nbytes);\n+\n+\tif (netif_tx_queue_stopped(sq-\u003etxq) \u0026\u0026\n+\t xsc_wqc_has_room_for(\u0026sq-\u003ewq, sq-\u003ecc, sq-\u003epc, sq-\u003estop_room)) {\n+\t\tnetif_tx_wake_queue(sq-\u003etxq);\n+\t}\n+\n+out:\n+\treturn (i == napi_budget);\n+}\n+\n+int xsc_eth_napi_poll(struct napi_struct *napi, int budget)\n+{\n+\tstruct xsc_channel *c = container_of(napi, struct xsc_channel, napi);\n+\tstruct xsc_eth_params *params = \u0026c-\u003eadapter-\u003enic_param;\n+\tstruct xsc_rq *rq = \u0026c-\u003eqp.rq;\n+\tbool busy = false;\n+\tint work_done = 0;\n+\tint tx_budget;\n+\n+\trcu_read_lock();\n+\n+\ttx_budget = params-\u003esq_size \u003e\u003e 2;\n+\tbusy |= xsc_poll_tx_cq(\u0026c-\u003eqp.sq.cq, tx_budget);\n+\n+\t/* budget=0 means: don't poll rx rings */\n+\tif (likely(budget)) {\n+\t\twork_done = xsc_poll_rx_cq(\u0026rq-\u003ecq, budget);\n+\t\tbusy |= work_done == budget;\n+\t}\n+\n+\tbusy |= rq-\u003epost_wqes(rq);\n+\n+\tif (busy) {\n+\t\tif (likely(xsc_channel_no_affinity_change(c))) {\n+\t\t\trcu_read_unlock();\n+\t\t\treturn budget;\n+\t\t}\n+\t\tif (budget \u0026\u0026 work_done == budget)\n+\t\t\twork_done--;\n+\t}\n+\n+\tif (unlikely(!napi_complete_done(napi, work_done)))\n+\t\tgoto err_out;\n+\n+\txsc_cq_notify_hw_rearm(\u0026c-\u003eqp.sq.cq);\n+\txsc_cq_notify_hw_rearm(\u0026rq-\u003ecq);\n+err_out:\n+\trcu_read_unlock();\n+\treturn work_done;\n+}\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.h\nnew file mode 100644\nindex 0000000000000..8d14f943ea778\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.h\n@@ -0,0 +1,91 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_RXTX_H\n+#define __XSC_RXTX_H\n+\n+#include \u003clinux/netdevice.h\u003e\n+#include \u003clinux/skbuff.h\u003e\n+\n+#include \"xsc_eth.h\"\n+\n+enum {\n+\tXSC_ETH_WQE_NONE_CSUM,\n+\tXSC_ETH_WQE_INNER_CSUM,\n+\tXSC_ETH_WQE_OUTER_CSUM,\n+\tXSC_ETH_WQE_INNER_AND_OUTER_CSUM,\n+};\n+\n+void xsc_cq_notify_hw_rearm(struct xsc_cq *cq);\n+void xsc_cq_notify_hw(struct xsc_cq *cq);\n+int xsc_eth_napi_poll(struct napi_struct *napi, int budget);\n+bool xsc_eth_post_rx_wqes(struct xsc_rq *rq);\n+void xsc_eth_handle_rx_cqe(struct xsc_cqwq *cqwq,\n+\t\t\t struct xsc_rq *rq, struct xsc_cqe *cqe);\n+void xsc_eth_dealloc_rx_wqe(struct xsc_rq *rq, u16 ix);\n+struct sk_buff *xsc_skb_from_cqe_linear(struct xsc_rq *rq,\n+\t\t\t\t\tstruct xsc_wqe_frag_info *wi,\n+\t\t\t\t\tu32 cqe_bcnt, u8 has_pph);\n+struct sk_buff *xsc_skb_from_cqe_nonlinear(struct xsc_rq *rq,\n+\t\t\t\t\t struct xsc_wqe_frag_info *wi,\n+\t\t\t\t\t u32 cqe_bcnt, u8 has_pph);\n+int xsc_poll_rx_cq(struct xsc_cq *cq, int budget);\n+\n+netdev_tx_t xsc_eth_xmit_start(struct sk_buff *skb, struct net_device *netdev);\n+\n+static inline void xsc_tx_dma_unmap(struct device *dev, struct xsc_sq_dma *dma)\n+{\n+\tswitch (dma-\u003etype) {\n+\tcase XSC_DMA_MAP_SINGLE:\n+\t\tdma_unmap_single(dev, dma-\u003eaddr, dma-\u003esize, DMA_TO_DEVICE);\n+\t\tbreak;\n+\tcase XSC_DMA_MAP_PAGE:\n+\t\tdma_unmap_page(dev, dma-\u003eaddr, dma-\u003esize, DMA_TO_DEVICE);\n+\t\tbreak;\n+\tdefault:\n+\t\tbreak;\n+\t}\n+}\n+\n+static inline struct xsc_sq_dma *xsc_dma_get(struct xsc_sq *sq, u32 i)\n+{\n+\treturn \u0026sq-\u003edb.dma_fifo[i \u0026 sq-\u003edma_fifo_mask];\n+}\n+\n+static inline bool xsc_wqc_has_room_for(struct xsc_wq_cyc *wq,\n+\t\t\t\t\tu16 cc, u16 pc, u16 n)\n+{\n+\treturn (xsc_wq_cyc_ctr2ix(wq, cc - pc) \u003e= n) || (cc == pc);\n+}\n+\n+static inline struct xsc_cqe *xsc_cqwq_get_cqe_buff(struct xsc_cqwq *wq, u32 ix)\n+{\n+\tstruct xsc_cqe *cqe = xsc_frag_buf_get_wqe(\u0026wq-\u003efbc, ix);\n+\n+\treturn cqe;\n+}\n+\n+static inline struct xsc_cqe *xsc_cqwq_get_cqe(struct xsc_cqwq *wq)\n+{\n+\tu32 ci = xsc_cqwq_get_ci(wq);\n+\tu8 cqe_ownership_bit;\n+\tstruct xsc_cqe *cqe;\n+\tu8 sw_ownership_val;\n+\n+\tcqe = xsc_cqwq_get_cqe_buff(wq, ci);\n+\n+\tcqe_ownership_bit = !!(le32_to_cpu(cqe-\u003edata1) \u0026 XSC_CQE_OWNER);\n+\tsw_ownership_val = xsc_cqwq_get_wrap_cnt(wq) \u0026 1;\n+\n+\tif (cqe_ownership_bit != sw_ownership_val)\n+\t\treturn NULL;\n+\n+\t/* ensure cqe content is read after cqe ownership bit */\n+\tdma_rmb();\n+\n+\treturn cqe;\n+}\n+\n+#endif /* XSC_RXTX_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.c\nnew file mode 100644\nindex 0000000000000..b69a4e373923a\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.c\n@@ -0,0 +1,80 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd. All\n+ * rights reserved.\n+ * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved.\n+ */\n+\n+#include \"xsc_eth_wq.h\"\n+#include \"xsc_eth.h\"\n+\n+u32 xsc_wq_cyc_get_size(struct xsc_wq_cyc *wq)\n+{\n+\treturn (u32)wq-\u003efbc.sz_m1 + 1;\n+}\n+\n+static u32 wq_get_byte_sz(u8 log_sz, u8 log_stride)\n+{\n+\treturn ((u32)1 \u003c\u003c log_sz) \u003c\u003c log_stride;\n+}\n+\n+int xsc_eth_cqwq_create(struct xsc_core_device *xdev,\n+\t\t\tstruct xsc_wq_param *param,\n+\t\t\tu8 q_log_size,\n+\t\t\tu8 ele_log_size,\n+\t\t\tstruct xsc_cqwq *wq,\n+\t\t\tstruct xsc_wq_ctrl *wq_ctrl)\n+{\n+\tu8 log_wq_stride = ele_log_size;\n+\tu8 log_wq_sz = q_log_size;\n+\tint err;\n+\n+\terr = xsc_core_frag_buf_alloc_node(xdev,\n+\t\t\t\t\t wq_get_byte_sz(log_wq_sz,\n+\t\t\t\t\t\t\t log_wq_stride),\n+\t\t\t\t\t \u0026wq_ctrl-\u003ebuf,\n+\t\t\t\t\t param-\u003ebuf_numa_node);\n+\tif (err) {\n+\t\tnetdev_err(xsc_dev_to_netdev(xdev),\n+\t\t\t \"xsc_core_frag_buf_alloc_node failed, %d\\n\", err);\n+\t\treturn err;\n+\t}\n+\n+\txsc_init_fbc(wq_ctrl-\u003ebuf.frags, log_wq_stride, log_wq_sz, \u0026wq-\u003efbc);\n+\twq_ctrl-\u003exdev = xdev;\n+\treturn 0;\n+}\n+\n+int xsc_eth_wq_cyc_create(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_wq_param *param,\n+\t\t\t u8 q_log_size,\n+\t\t\t u8 ele_log_size,\n+\t\t\t struct xsc_wq_cyc *wq,\n+\t\t\t struct xsc_wq_ctrl *wq_ctrl)\n+{\n+\tstruct xsc_frag_buf_ctrl *fbc = \u0026wq-\u003efbc;\n+\tu8 log_wq_stride = ele_log_size;\n+\tu8 log_wq_sz = q_log_size;\n+\tint err;\n+\n+\terr = xsc_core_frag_buf_alloc_node(xdev,\n+\t\t\t\t\t wq_get_byte_sz(log_wq_sz,\n+\t\t\t\t\t\t\t log_wq_stride),\n+\t\t\t\t\t \u0026wq_ctrl-\u003ebuf,\n+\t\t\t\t\t param-\u003ebuf_numa_node);\n+\tif (err) {\n+\t\tnetdev_err(xsc_dev_to_netdev(xdev),\n+\t\t\t \"xsc_core_frag_buf_alloc_node failed, %d\\n\", err);\n+\t\treturn err;\n+\t}\n+\n+\txsc_init_fbc(wq_ctrl-\u003ebuf.frags, log_wq_stride, log_wq_sz, fbc);\n+\twq-\u003esz = xsc_wq_cyc_get_size(wq);\n+\twq_ctrl-\u003exdev = xdev;\n+\treturn 0;\n+}\n+\n+void xsc_eth_wq_destroy(struct xsc_wq_ctrl *wq_ctrl)\n+{\n+\txsc_core_frag_buf_free(wq_ctrl-\u003exdev, \u0026wq_ctrl-\u003ebuf);\n+}\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.h\nnew file mode 100644\nindex 0000000000000..01cee9fd1d9de\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.h\n@@ -0,0 +1,185 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/*\n+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd. All\n+ * rights reserved.\n+ * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved.\n+ */\n+\n+#ifndef __XSC_WQ_H\n+#define __XSC_WQ_H\n+\n+#include \"common/xsc_core.h\"\n+\n+struct xsc_wq_param {\n+\tint\t\tbuf_numa_node;\n+\tint\t\tdb_numa_node;\n+};\n+\n+struct xsc_wq_ctrl {\n+\tstruct xsc_core_device\t*xdev;\n+\tstruct xsc_frag_buf\tbuf;\n+};\n+\n+struct xsc_wq_cyc {\n+\tstruct xsc_frag_buf_ctrl fbc;\n+\tu16\t\t\tsz;\n+\tu16\t\t\twqe_ctr;\n+\tu16\t\t\tcur_sz;\n+};\n+\n+struct xsc_cqwq {\n+\tstruct xsc_frag_buf_ctrl fbc;\n+\t__be32\t\t\t *db;\n+\tu32\t\t\t cc; /* consumer counter */\n+};\n+\n+enum xsc_res_type {\n+\tXSC_RES_RQ,\n+\tXSC_RES_SQ,\n+};\n+\n+u32 xsc_wq_cyc_get_size(struct xsc_wq_cyc *wq);\n+\n+/*api for eth driver*/\n+int xsc_eth_cqwq_create(struct xsc_core_device *xdev,\n+\t\t\tstruct xsc_wq_param *param,\n+\t\t\tu8 q_log_size,\n+\t\t\tu8 ele_log_size,\n+\t\t\tstruct xsc_cqwq *wq,\n+\t\t\tstruct xsc_wq_ctrl *wq_ctrl);\n+\n+int xsc_eth_wq_cyc_create(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_wq_param *param,\n+\t\t\t u8 q_log_size,\n+\t\t\t u8 ele_log_size,\n+\t\t\t struct xsc_wq_cyc *wq,\n+\t\t\t struct xsc_wq_ctrl *wq_ctrl);\n+void xsc_eth_wq_destroy(struct xsc_wq_ctrl *wq_ctrl);\n+\n+static inline void xsc_init_fbc_offset(struct xsc_buf_list *frags,\n+\t\t\t\t u8 log_stride, u8 log_sz,\n+\t\t\t\t u16 strides_offset,\n+\t\t\t\t struct xsc_frag_buf_ctrl *fbc)\n+{\n+\tfbc-\u003efrags = frags;\n+\tfbc-\u003elog_stride = log_stride;\n+\tfbc-\u003elog_sz = log_sz;\n+\tfbc-\u003esz_m1\t= (1 \u003c\u003c fbc-\u003elog_sz) - 1;\n+\tfbc-\u003elog_frag_strides = PAGE_SHIFT - fbc-\u003elog_stride;\n+\tfbc-\u003efrag_sz_m1\t= (1 \u003c\u003c fbc-\u003elog_frag_strides) - 1;\n+\tfbc-\u003estrides_offset = strides_offset;\n+}\n+\n+static inline void xsc_init_fbc(struct xsc_buf_list *frags,\n+\t\t\t\tu8 log_stride, u8 log_sz,\n+\t\t\t\tstruct xsc_frag_buf_ctrl *fbc)\n+{\n+\txsc_init_fbc_offset(frags, log_stride, log_sz, 0, fbc);\n+}\n+\n+static inline void *xsc_frag_buf_get_wqe(struct xsc_frag_buf_ctrl *fbc,\n+\t\t\t\t\t u32 ix)\n+{\n+\tunsigned int frag;\n+\n+\tix += fbc-\u003estrides_offset;\n+\tfrag = ix \u003e\u003e fbc-\u003elog_frag_strides;\n+\n+\treturn fbc-\u003efrags[frag].buf +\n+\t\t((fbc-\u003efrag_sz_m1 \u0026 ix) \u003c\u003c fbc-\u003elog_stride);\n+}\n+\n+static inline u32\n+xsc_frag_buf_get_idx_last_contig_stride(struct xsc_frag_buf_ctrl *fbc, u32 ix)\n+{\n+\tu32 last_frag_stride_idx = (ix + fbc-\u003estrides_offset) | fbc-\u003efrag_sz_m1;\n+\n+\treturn min_t(u32, last_frag_stride_idx - fbc-\u003estrides_offset,\n+\t\t fbc-\u003esz_m1);\n+}\n+\n+static inline int xsc_wq_cyc_missing(struct xsc_wq_cyc *wq)\n+{\n+\treturn wq-\u003esz - wq-\u003ecur_sz;\n+}\n+\n+static inline int xsc_wq_cyc_is_empty(struct xsc_wq_cyc *wq)\n+{\n+\treturn !wq-\u003ecur_sz;\n+}\n+\n+static inline void xsc_wq_cyc_push(struct xsc_wq_cyc *wq)\n+{\n+\twq-\u003ewqe_ctr++;\n+\twq-\u003ecur_sz++;\n+}\n+\n+static inline void xsc_wq_cyc_push_n(struct xsc_wq_cyc *wq, u8 n)\n+{\n+\twq-\u003ewqe_ctr += n;\n+\twq-\u003ecur_sz += n;\n+}\n+\n+static inline void xsc_wq_cyc_pop(struct xsc_wq_cyc *wq)\n+{\n+\twq-\u003ecur_sz--;\n+}\n+\n+static inline u16 xsc_wq_cyc_ctr2ix(struct xsc_wq_cyc *wq, u16 ctr)\n+{\n+\treturn ctr \u0026 wq-\u003efbc.sz_m1;\n+}\n+\n+static inline u16 xsc_wq_cyc_get_head(struct xsc_wq_cyc *wq)\n+{\n+\treturn xsc_wq_cyc_ctr2ix(wq, wq-\u003ewqe_ctr);\n+}\n+\n+static inline u16 xsc_wq_cyc_get_tail(struct xsc_wq_cyc *wq)\n+{\n+\treturn xsc_wq_cyc_ctr2ix(wq, wq-\u003ewqe_ctr - wq-\u003ecur_sz);\n+}\n+\n+static inline void *xsc_wq_cyc_get_wqe(struct xsc_wq_cyc *wq, u16 ix)\n+{\n+\treturn xsc_frag_buf_get_wqe(\u0026wq-\u003efbc, ix);\n+}\n+\n+static inline u32 xsc_cqwq_ctr2ix(struct xsc_cqwq *wq, u32 ctr)\n+{\n+\treturn ctr \u0026 wq-\u003efbc.sz_m1;\n+}\n+\n+static inline u32 xsc_cqwq_get_ci(struct xsc_cqwq *wq)\n+{\n+\treturn xsc_cqwq_ctr2ix(wq, wq-\u003ecc);\n+}\n+\n+static inline u32 xsc_cqwq_get_ctr_wrap_cnt(struct xsc_cqwq *wq, u32 ctr)\n+{\n+\treturn ctr \u003e\u003e wq-\u003efbc.log_sz;\n+}\n+\n+static inline u32 xsc_cqwq_get_wrap_cnt(struct xsc_cqwq *wq)\n+{\n+\treturn xsc_cqwq_get_ctr_wrap_cnt(wq, wq-\u003ecc);\n+}\n+\n+static inline void xsc_cqwq_pop(struct xsc_cqwq *wq)\n+{\n+\twq-\u003ecc++;\n+}\n+\n+static inline u32 xsc_cqwq_get_size(struct xsc_cqwq *wq)\n+{\n+\treturn wq-\u003efbc.sz_m1 + 1;\n+}\n+\n+static inline struct xsc_cqe *xsc_cqwq_get_wqe(struct xsc_cqwq *wq, u32 ix)\n+{\n+\tstruct xsc_cqe *cqe = xsc_frag_buf_get_wqe(\u0026wq-\u003efbc, ix);\n+\n+\treturn cqe;\n+}\n+\n+#endif /* __XSC_WQ_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pp.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pp.h\nnew file mode 100644\nindex 0000000000000..cc9eec866d523\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pp.h\n@@ -0,0 +1,38 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_PP_H\n+#define __XSC_PP_H\n+\n+enum {\n+\tXSC_HASH_FIELD_SEL_SRC_IP\t= BIT(0),\n+\tXSC_HASH_FIELD_SEL_PROTO\t= BIT(1),\n+\tXSC_HASH_FIELD_SEL_DST_IP\t= BIT(2),\n+\tXSC_HASH_FIELD_SEL_SPORT\t= BIT(3),\n+\tXSC_HASH_FIELD_SEL_DPORT\t= BIT(4),\n+\tXSC_HASH_FIELD_SEL_SRC_IPV6\t= BIT(5),\n+\tXSC_HASH_FIELD_SEL_DST_IPV6\t= BIT(6),\n+\tXSC_HASH_FIELD_SEL_SPORT_V6\t= BIT(7),\n+\tXSC_HASH_FIELD_SEL_DPORT_V6\t= BIT(8),\n+};\n+\n+#define XSC_HASH_IP\t\t(XSC_HASH_FIELD_SEL_SRC_IP\t|\\\n+\t\t\t\tXSC_HASH_FIELD_SEL_DST_IP\t|\\\n+\t\t\t\tXSC_HASH_FIELD_SEL_PROTO)\n+#define XSC_HASH_IP_PORTS\t(XSC_HASH_FIELD_SEL_SRC_IP\t|\\\n+\t\t\t\tXSC_HASH_FIELD_SEL_DST_IP\t|\\\n+\t\t\t\tXSC_HASH_FIELD_SEL_SPORT\t|\\\n+\t\t\t\tXSC_HASH_FIELD_SEL_DPORT\t|\\\n+\t\t\t\tXSC_HASH_FIELD_SEL_PROTO)\n+#define XSC_HASH_IP6\t\t(XSC_HASH_FIELD_SEL_SRC_IPV6\t|\\\n+\t\t\t\tXSC_HASH_FIELD_SEL_DST_IPV6\t|\\\n+\t\t\t\tXSC_HASH_FIELD_SEL_PROTO)\n+#define XSC_HASH_IP6_PORTS\t(XSC_HASH_FIELD_SEL_SRC_IPV6\t|\\\n+\t\t\t\tXSC_HASH_FIELD_SEL_DST_IPV6\t|\\\n+\t\t\t\tXSC_HASH_FIELD_SEL_SPORT_V6\t|\\\n+\t\t\t\tXSC_HASH_FIELD_SEL_DPORT_V6\t|\\\n+\t\t\t\tXSC_HASH_FIELD_SEL_PROTO)\n+\n+#endif /* __XSC_PP_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pph.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pph.h\nnew file mode 100644\nindex 0000000000000..cc544df2cfce4\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pph.h\n@@ -0,0 +1,42 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_PPH_H\n+#define __XSC_PPH_H\n+\n+#define XSC_PPH_HEAD_LEN\t64\n+\n+#define OUTER_L3_BIT\tBIT(3)\n+#define OUTER_L4_BIT\tBIT(2)\n+#define INNER_L3_BIT\tBIT(1)\n+#define INNER_L4_BIT\tBIT(0)\n+#define OUTER_BIT\t\t(OUTER_L3_BIT | OUTER_L4_BIT)\n+#define INNER_BIT\t\t(INNER_L3_BIT | INNER_L4_BIT)\n+#define OUTER_AND_INNER\t(OUTER_BIT | INNER_BIT)\n+\n+#define PACKET_UNKNOWN\tBIT(4)\n+\n+#define EPP2SOC_PPH_EXT_TUNNEL_TYPE_OFFSET (6UL)\n+#define EPP2SOC_PPH_EXT_TUNNEL_TYPE_BIT_MASK (0XF00)\n+#define EPP2SOC_PPH_EXT_TUNNEL_TYPE_BIT_OFFSET (8)\n+\n+#define EPP2SOC_PPH_EXT_ERROR_BITMAP_OFFSET (20UL)\n+#define EPP2SOC_PPH_EXT_ERROR_BITMAP_BIT_MASK (0XFF)\n+#define EPP2SOC_PPH_EXT_ERROR_BITMAP_BIT_OFFSET (0)\n+\n+#define XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(PPH_BASE_ADDR)\t\\\n+\t((*(u16 *)((u8 *)(PPH_BASE_ADDR) +\t\\\n+\tEPP2SOC_PPH_EXT_TUNNEL_TYPE_OFFSET) \u0026\t\\\n+\tEPP2SOC_PPH_EXT_TUNNEL_TYPE_BIT_MASK) \u003e\u003e\t\\\n+\tEPP2SOC_PPH_EXT_TUNNEL_TYPE_BIT_OFFSET)\n+\n+#define XSC_GET_EPP2SOC_PPH_ERROR_BITMAP(PPH_BASE_ADDR)\t\\\n+\t((*(u8 *)((u8 *)(PPH_BASE_ADDR) +\t\\\n+\tEPP2SOC_PPH_EXT_ERROR_BITMAP_OFFSET) \u0026\t\\\n+\tEPP2SOC_PPH_EXT_ERROR_BITMAP_BIT_MASK) \u003e\u003e\t\\\n+\tEPP2SOC_PPH_EXT_ERROR_BITMAP_BIT_OFFSET)\n+\n+#endif /* __XSC_PPH_H */\n+\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_queue.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_queue.h\nnew file mode 100644\nindex 0000000000000..49c3229e823e6\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_queue.h\n@@ -0,0 +1,185 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/*\n+ * Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ * Copyright (c) 2015-2016, Mellanox Technologies. All rights reserved.\n+ */\n+\n+#ifndef __XSC_QUEUE_H\n+#define __XSC_QUEUE_H\n+\n+#include \u003cnet/page_pool/types.h\u003e\n+#include \u003cnet/page_pool/helpers.h\u003e\n+\n+#include \"common/xsc_core.h\"\n+#include \"xsc_eth_wq.h\"\n+\n+enum {\n+\tXSC_SEND_WQE_DS\t= 16,\n+\tXSC_SEND_WQE_BB\t= 64,\n+};\n+\n+enum {\n+\tXSC_RECV_WQE_DS\t= 16,\n+\tXSC_RECV_WQE_BB\t= 16,\n+};\n+\n+enum {\n+\tXSC_ETH_RQ_STATE_ENABLED,\n+};\n+\n+#define XSC_SEND_WQEBB_CTRL_NUM_DS\t1\n+\n+/* each ds holds one fragment in skb */\n+#define XSC_MAX_RX_FRAGS 4\n+#define XSC_RX_FRAG_SZ_ORDER 0\n+#define XSC_RX_FRAG_SZ (PAGE_SIZE \u003c\u003c XSC_RX_FRAG_SZ_ORDER)\n+#define DEFAULT_FRAG_SIZE (2048)\n+\n+enum {\n+\tXSC_ETH_SQ_STATE_ENABLED,\n+};\n+\n+struct xsc_cq {\n+\t/* data path - accessed per cqe */\n+\tstruct xsc_cqwq\twq;\n+\n+\t/* data path - accessed per napi poll */\n+\tu16\t\t\tevent_ctr;\n+\tstruct napi_struct\t*napi;\n+\tstruct xsc_core_cq\txcq;\n+\tstruct xsc_channel\t*channel;\n+\n+\t/* control */\n+\tstruct xsc_core_device\t*xdev;\n+\tstruct xsc_wq_ctrl\twq_ctrl;\n+} ____cacheline_aligned_in_smp;\n+\n+struct xsc_wqe_frag_info {\n+\tstruct xsc_dma_info *di;\n+\tu32 offset;\n+\tu8 last_in_page;\n+\tu8 is_available;\n+};\n+\n+struct xsc_dma_info {\n+\tstruct page\t*page;\n+\tdma_addr_t\taddr;\n+};\n+\n+struct xsc_rq_frag_info {\n+\tint frag_size;\n+\tint frag_stride;\n+};\n+\n+struct xsc_rq_frags_info {\n+\tstruct xsc_rq_frag_info arr[XSC_MAX_RX_FRAGS];\n+\tu8 num_frags;\n+\tu8 log_num_frags;\n+\tu8 wqe_bulk;\n+\tu8 wqe_bulk_min;\n+\tu8 frags_max_num;\n+};\n+\n+struct xsc_rq;\n+typedef void (*xsc_fp_handle_rx_cqe)(struct xsc_cqwq *cqwq, struct xsc_rq *rq,\n+\t\t\t\t struct xsc_cqe *cqe);\n+typedef bool (*xsc_fp_post_rx_wqes)(struct xsc_rq *rq);\n+typedef void (*xsc_fp_dealloc_wqe)(struct xsc_rq *rq, u16 ix);\n+typedef struct sk_buff * (*xsc_fp_skb_from_cqe)(struct xsc_rq *rq,\n+\t\t\t\t\t\tstruct xsc_wqe_frag_info *wi,\n+\t\t\t\t\t\tu32 cqe_bcnt,\n+\t\t\t\t\t\tu8 has_pph);\n+\n+struct xsc_rq {\n+\tstruct xsc_core_qp\t\tcqp;\n+\tstruct {\n+\t\tstruct xsc_wq_cyc\twq;\n+\t\tstruct xsc_wqe_frag_info\t*frags;\n+\t\tstruct xsc_dma_info\t*di;\n+\t\tstruct xsc_rq_frags_info\tinfo;\n+\t\txsc_fp_skb_from_cqe\tskb_from_cqe;\n+\t} wqe;\n+\n+\tstruct {\n+\t\tu16\theadroom;\n+\t\tu8\tmap_dir;\t/* dma map direction */\n+\t} buff;\n+\n+\tstruct page_pool\t*page_pool;\n+\tstruct xsc_wq_ctrl\twq_ctrl;\n+\tstruct xsc_cq\t\tcq;\n+\tu32\trqn;\n+\tint\tix;\n+\n+\tunsigned long\tstate;\n+\tstruct xsc_rq_stats *stats;\n+\n+\tu32 hw_mtu;\n+\tu32 frags_sz;\n+\n+\txsc_fp_handle_rx_cqe\thandle_rx_cqe;\n+\txsc_fp_post_rx_wqes\tpost_wqes;\n+\txsc_fp_dealloc_wqe\tdealloc_wqe;\n+} ____cacheline_aligned_in_smp;\n+\n+enum xsc_dma_map_type {\n+\tXSC_DMA_MAP_SINGLE,\n+\tXSC_DMA_MAP_PAGE\n+};\n+\n+struct xsc_sq_dma {\n+\tdma_addr_t\taddr;\n+\tu32\t\tsize;\n+\tenum xsc_dma_map_type\ttype;\n+};\n+\n+struct xsc_tx_wqe_info {\n+\tstruct sk_buff *skb;\n+\tu32 num_bytes;\n+\tu8 num_wqebbs;\n+\tu8 num_dma;\n+};\n+\n+struct xsc_tx_wqe {\n+\tstruct xsc_send_wqe_ctrl_seg ctrl;\n+\tstruct xsc_wqe_data_seg data[];\n+};\n+\n+struct xsc_sq {\n+\tstruct xsc_core_qp\t\tcqp;\n+\t/* dirtied @completion */\n+\tu16 cc;\n+\tu32 dma_fifo_cc;\n+\n+\t/* dirtied @xmit */\n+\tu16 pc ____cacheline_aligned_in_smp;\n+\tu32 dma_fifo_pc;\n+\n+\tstruct xsc_cq cq;\n+\n+\t/* read only */\n+\tstruct xsc_wq_cyc wq;\n+\tu32 dma_fifo_mask;\n+\tstruct xsc_sq_stats *stats;\n+\tstruct {\n+\t\tstruct xsc_sq_dma *dma_fifo;\n+\t\tstruct xsc_tx_wqe_info *wqe_info;\n+\t} db;\n+\tvoid __iomem *uar_map;\n+\tstruct netdev_queue *txq;\n+\tu32 sqn;\n+\tu16 stop_room;\n+\n+\t__be32 mkey_be;\n+\tunsigned long state;\n+\tunsigned int hw_mtu;\n+\n+\t/* control path */\n+\tstruct xsc_wq_ctrl wq_ctrl;\n+\tstruct xsc_channel *channel;\n+\tint ch_ix;\n+\tint txq_ix;\n+} ____cacheline_aligned_in_smp;\n+\n+#endif /* __XSC_QUEUE_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig b/drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig\nnew file mode 100644\nindex 0000000000000..1cc3f7b49d5a2\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig\n@@ -0,0 +1,15 @@\n+# SPDX-License-Identifier: GPL-2.0\n+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+# All rights reserved.\n+# Yunsilicon PCI configuration\n+#\n+\n+config YUNSILICON_XSC_PCI\n+\ttristate \"Yunsilicon XSC PCI driver\"\n+\tdepends on PCI\n+\thelp\n+\t This driver is common for Yunsilicon XSC\n+\t ethernet and RDMA drivers.\n+\n+\t To compile this driver as a module, choose M here. The module\n+\t will be called xsc_pci.\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile\nnew file mode 100644\nindex 0000000000000..fe0f9d175ea22\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile\n@@ -0,0 +1,10 @@\n+# SPDX-License-Identifier: GPL-2.0\n+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+# All rights reserved.\n+\n+ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc\n+\n+obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc_pci.o\n+\n+xsc_pci-y := main.o cmdq.o hw.o qp.o cq.o pd.o mr.o srq.o link.o alloc.o eq.o pci_irq.o adev.o vport.o \\\n+\t\thal/andes_impl.o\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/adev.c b/drivers/net/ethernet/yunsilicon/xsc/pci/adev.c\nnew file mode 100644\nindex 0000000000000..c8e8f1a9ff5e3\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/adev.c\n@@ -0,0 +1,142 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/auxiliary_bus.h\u003e\n+#include \u003clinux/idr.h\u003e\n+\n+#include \"adev.h\"\n+\n+static DEFINE_IDA(xsc_adev_ida);\n+\n+static const char * const xsc_adev_name[] = {\n+\t[XSC_ADEV_IDX_ETH] = XSC_ETH_ADEV_NAME,\n+\t[XSC_ADEV_IDX_IB] = XSC_IB_ADEV_NAME,\n+};\n+\n+static void xsc_release_adev(struct device *dev)\n+{\n+\tstruct xsc_adev *xsc_adev =\n+\t\tcontainer_of(dev, struct xsc_adev, adev.dev);\n+\n+\tkfree(xsc_adev);\n+}\n+\n+int xsc_adev_add(struct xsc_core_device *xdev, int idx)\n+{\n+\tstruct auxiliary_device\t*adev;\n+\tstruct xsc_adev *xsc_adev;\n+\tint ret;\n+\n+\tif (!xdev-\u003exsc_adev_list)\n+\t\treturn -ENODEV;\n+\n+\tif (xdev-\u003exsc_adev_list[idx])\n+\t\treturn 0;\n+\n+\txsc_adev = kzalloc_obj(*xsc_adev, GFP_KERNEL);\n+\tif (!xsc_adev)\n+\t\treturn -ENOMEM;\n+\n+\tadev = \u0026xsc_adev-\u003eadev;\n+\tadev-\u003ename = xsc_adev_name[idx];\n+\tadev-\u003eid = xdev-\u003eadev_id;\n+\tadev-\u003edev.parent = \u0026xdev-\u003epdev-\u003edev;\n+\tadev-\u003edev.release = xsc_release_adev;\n+\txsc_adev-\u003exdev = xdev;\n+\txsc_adev-\u003eidx = idx;\n+\n+\tret = auxiliary_device_init(adev);\n+\tif (ret) {\n+\t\tkfree(xsc_adev);\n+\t\treturn ret;\n+\t}\n+\n+\tret = auxiliary_device_add(adev);\n+\tif (ret) {\n+\t\tauxiliary_device_uninit(adev);\n+\t\treturn ret;\n+\t}\n+\n+\txdev-\u003exsc_adev_list[idx] = xsc_adev;\n+\n+\treturn 0;\n+}\n+\n+void xsc_adev_del(struct xsc_core_device *xdev, int idx)\n+{\n+\tstruct xsc_adev *xsc_adev;\n+\tstruct auxiliary_device *adev;\n+\n+\tif (!xdev-\u003exsc_adev_list)\n+\t\treturn;\n+\n+\txsc_adev = xdev-\u003exsc_adev_list[idx];\n+\tif (!xsc_adev)\n+\t\treturn;\n+\n+\t/* Clear the slot before teardown: the driver's remove callback runs\n+\t * synchronously inside auxiliary_device_delete() and may re-enter\n+\t * xsc_adev_add() for this index.\n+\t */\n+\txdev-\u003exsc_adev_list[idx] = NULL;\n+\n+\tadev = \u0026xsc_adev-\u003eadev;\n+\tauxiliary_device_delete(adev);\n+\tauxiliary_device_uninit(adev);\n+}\n+\n+int xsc_adev_init(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_adev **xsc_adev_list;\n+\tint adev_id;\n+\tint ret;\n+\n+\txsc_adev_list = kcalloc(ARRAY_SIZE(xsc_adev_name),\n+\t\t\t\tsizeof(struct xsc_adev *),\n+\t\t\t\tGFP_KERNEL);\n+\tif (!xsc_adev_list)\n+\t\treturn -ENOMEM;\n+\txdev-\u003exsc_adev_list = xsc_adev_list;\n+\n+\tadev_id = ida_alloc(\u0026xsc_adev_ida, GFP_KERNEL);\n+\tif (adev_id \u003c 0) {\n+\t\tret = adev_id;\n+\t\tgoto err_free_adev_list;\n+\t}\n+\txdev-\u003eadev_id = adev_id;\n+\n+\tret = xsc_adev_add(xdev, XSC_ADEV_IDX_ETH);\n+\tif (ret)\n+\t\tgoto err_dalloc_adev_id;\n+\n+\tret = xsc_adev_add(xdev, XSC_ADEV_IDX_IB);\n+\tif (ret)\n+\t\tgoto err_unreg_eth_adev;\n+\n+\treturn 0;\n+\n+err_unreg_eth_adev:\n+\txsc_adev_del(xdev, XSC_ADEV_IDX_ETH);\n+err_dalloc_adev_id:\n+\tida_free(\u0026xsc_adev_ida, xdev-\u003eadev_id);\n+err_free_adev_list:\n+\tkfree(xsc_adev_list);\n+\txdev-\u003exsc_adev_list = NULL;\n+\n+\treturn ret;\n+}\n+\n+void xsc_adev_uninit(struct xsc_core_device *xdev)\n+{\n+\tif (!xdev-\u003exsc_adev_list)\n+\t\treturn;\n+\n+\txsc_adev_del(xdev, XSC_ADEV_IDX_IB);\n+\txsc_adev_del(xdev, XSC_ADEV_IDX_ETH);\n+\tida_free(\u0026xsc_adev_ida, xdev-\u003eadev_id);\n+\tkfree(xdev-\u003exsc_adev_list);\n+\txdev-\u003exsc_adev_list = NULL;\n+}\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/adev.h b/drivers/net/ethernet/yunsilicon/xsc/pci/adev.h\nnew file mode 100644\nindex 0000000000000..fb5789ec42f91\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/adev.h\n@@ -0,0 +1,17 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_ADEV_H\n+#define __XSC_ADEV_H\n+\n+#include \"common/xsc_core.h\"\n+\n+int xsc_adev_init(struct xsc_core_device *xdev);\n+void xsc_adev_uninit(struct xsc_core_device *xdev);\n+\n+int xsc_adev_add(struct xsc_core_device *xdev, int idx);\n+void xsc_adev_del(struct xsc_core_device *xdev, int idx);\n+\n+#endif\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.c b/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.c\nnew file mode 100644\nindex 0000000000000..70176797a46fc\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.c\n@@ -0,0 +1,237 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/errno.h\u003e\n+#include \u003clinux/slab.h\u003e\n+#include \u003clinux/mm.h\u003e\n+#include \u003clinux/export.h\u003e\n+#include \u003clinux/bitmap.h\u003e\n+#include \u003clinux/dma-mapping.h\u003e\n+#include \u003clinux/vmalloc.h\u003e\n+\n+#include \"alloc.h\"\n+\n+/* Handling for queue buffers -- we allocate a bunch of memory and\n+ * register it in a memory region at HCA virtual address 0. If the\n+ * requested size is \u003e max_direct, we split the allocation into\n+ * multiple pages, so we don't require too much contiguous memory.\n+ */\n+int xsc_buf_alloc(struct xsc_core_device *xdev,\n+\t\t unsigned long size,\n+\t\t unsigned long max_direct,\n+\t\t struct xsc_buf *buf)\n+{\n+\tdma_addr_t t;\n+\n+\tbuf-\u003esize = size;\n+\tif (size \u003c= max_direct) {\n+\t\tbuf-\u003enbufs = 1;\n+\t\tbuf-\u003enpages = 1;\n+\t\tbuf-\u003epage_shift = get_order(size) + PAGE_SHIFT;\n+\t\tbuf-\u003edirect.buf = dma_alloc_coherent(\u0026xdev-\u003epdev-\u003edev,\n+\t\t\t\t\t\t size,\n+\t\t\t\t\t\t \u0026t,\n+\t\t\t\t\t\t GFP_KERNEL);\n+\t\tif (!buf-\u003edirect.buf)\n+\t\t\treturn -ENOMEM;\n+\n+\t\tbuf-\u003edirect.map = t;\n+\n+\t\twhile (t \u0026 GENMASK(buf-\u003epage_shift - 1, 0)) {\n+\t\t\t--buf-\u003epage_shift;\n+\t\t\tbuf-\u003enpages *= 2;\n+\t\t}\n+\t} else {\n+\t\tstruct page **pages;\n+\t\tint i;\n+\n+\t\tbuf-\u003edirect.buf = NULL;\n+\t\tbuf-\u003enbufs = DIV_ROUND_UP(size, PAGE_SIZE);\n+\t\tbuf-\u003enpages = buf-\u003enbufs;\n+\t\tbuf-\u003epage_shift = PAGE_SHIFT;\n+\t\tbuf-\u003epage_list = kcalloc(buf-\u003enbufs, sizeof(*buf-\u003epage_list),\n+\t\t\t\t\t GFP_KERNEL);\n+\t\tif (!buf-\u003epage_list)\n+\t\t\treturn -ENOMEM;\n+\n+\t\tfor (i = 0; i \u003c buf-\u003enbufs; i++) {\n+\t\t\tbuf-\u003epage_list[i].buf =\n+\t\t\t\tdma_alloc_coherent(\u0026xdev-\u003epdev-\u003edev, PAGE_SIZE,\n+\t\t\t\t\t\t \u0026t, GFP_KERNEL);\n+\t\t\tif (!buf-\u003epage_list[i].buf)\n+\t\t\t\tgoto err_free;\n+\n+\t\t\tbuf-\u003epage_list[i].map = t;\n+\t\t}\n+\n+\t\tpages = kmalloc_array(buf-\u003enbufs, sizeof(*pages), GFP_KERNEL);\n+\t\tif (!pages)\n+\t\t\tgoto err_free;\n+\t\tfor (i = 0; i \u003c buf-\u003enbufs; i++) {\n+\t\t\tvoid *addr = buf-\u003epage_list[i].buf;\n+\n+\t\t\tif (is_vmalloc_addr(addr))\n+\t\t\t\tpages[i] = vmalloc_to_page(addr);\n+\t\t\telse\n+\t\t\t\tpages[i] = virt_to_page(addr);\n+\t\t}\n+\t\tbuf-\u003edirect.buf = vmap(pages, buf-\u003enbufs, VM_MAP, PAGE_KERNEL);\n+\t\tkfree(pages);\n+\t\tif (!buf-\u003edirect.buf)\n+\t\t\tgoto err_free;\n+\t}\n+\n+\treturn 0;\n+\n+err_free:\n+\txsc_buf_free(xdev, buf);\n+\n+\treturn -ENOMEM;\n+}\n+EXPORT_SYMBOL_GPL(xsc_buf_alloc);\n+\n+void xsc_buf_free(struct xsc_core_device *xdev, struct xsc_buf *buf)\n+{\n+\tint i;\n+\n+\tif (buf-\u003enbufs == 1) {\n+\t\tdma_free_coherent(\u0026xdev-\u003epdev-\u003edev, buf-\u003esize, buf-\u003edirect.buf,\n+\t\t\t\t buf-\u003edirect.map);\n+\t} else {\n+\t\tif (BITS_PER_LONG == 64 \u0026\u0026 buf-\u003edirect.buf)\n+\t\t\tvunmap(buf-\u003edirect.buf);\n+\n+\t\tfor (i = 0; i \u003c buf-\u003enbufs; i++)\n+\t\t\tif (buf-\u003epage_list[i].buf)\n+\t\t\t\tdma_free_coherent(\u0026xdev-\u003epdev-\u003edev, PAGE_SIZE,\n+\t\t\t\t\t\t buf-\u003epage_list[i].buf,\n+\t\t\t\t\t\t buf-\u003epage_list[i].map);\n+\t\tkfree(buf-\u003epage_list);\n+\t}\n+}\n+EXPORT_SYMBOL_GPL(xsc_buf_free);\n+\n+void xsc_fill_page_array(struct xsc_buf *buf, __be64 *pas, unsigned int npages)\n+{\n+\tunsigned int shift = PAGE_SHIFT - PAGE_SHIFT_4K;\n+\tunsigned int mask = (1 \u003c\u003c shift) - 1;\n+\tu64 addr;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c npages; i++) {\n+\t\tif (buf-\u003enbufs == 1)\n+\t\t\taddr = buf-\u003edirect.map + (i \u003c\u003c PAGE_SHIFT_4K);\n+\t\telse\n+\t\t\taddr = buf-\u003epage_list[i \u003e\u003e shift].map +\n+\t\t\t ((i \u0026 mask) \u003c\u003c PAGE_SHIFT_4K);\n+\n+\t\tpas[i] = cpu_to_be64(addr);\n+\t}\n+}\n+EXPORT_SYMBOL_GPL(xsc_fill_page_array);\n+\n+void xsc_core_fill_page_frag_array(struct xsc_frag_buf *buf,\n+\t\t\t\t __be64 *pas,\n+\t\t\t\t unsigned int npages)\n+{\n+\tint i;\n+\tdma_addr_t addr;\n+\tunsigned int shift = PAGE_SHIFT - PAGE_SHIFT_4K;\n+\tunsigned int mask = (1 \u003c\u003c shift) - 1;\n+\n+\tfor (i = 0; i \u003c npages; i++) {\n+\t\taddr = buf-\u003efrags[i \u003e\u003e shift].map +\n+\t\t\t((i \u0026 mask) \u003c\u003c PAGE_SHIFT_4K);\n+\t\tpas[i] = cpu_to_be64(addr);\n+\t}\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_fill_page_frag_array);\n+\n+static void *xsc_dma_zalloc_coherent_node(struct xsc_core_device *xdev,\n+\t\t\t\t\t size_t size, dma_addr_t *dma_handle,\n+\t\t\t\t\t int node)\n+{\n+\tstruct xsc_dev_resource *dev_res = xdev-\u003edev_res;\n+\tstruct device *device = \u0026xdev-\u003epdev-\u003edev;\n+\tint original_node;\n+\tvoid *cpu_handle;\n+\n+\t/* WA for kernels that don't use numa_mem_id in alloc_pages_node */\n+\tif (node == NUMA_NO_NODE)\n+\t\tnode = numa_mem_id();\n+\n+\tmutex_lock(\u0026dev_res-\u003ealloc_mutex);\n+\toriginal_node = dev_to_node(device);\n+\tset_dev_node(device, node);\n+\tcpu_handle = dma_alloc_coherent(device, size, dma_handle,\n+\t\t\t\t\tGFP_KERNEL);\n+\tset_dev_node(device, original_node);\n+\tmutex_unlock(\u0026dev_res-\u003ealloc_mutex);\n+\treturn cpu_handle;\n+}\n+\n+int xsc_core_frag_buf_alloc_node(struct xsc_core_device *xdev,\n+\t\t\t\t unsigned long size,\n+\t\t\t\t struct xsc_frag_buf *buf,\n+\t\t\t\t int node)\n+{\n+\tint i;\n+\n+\tbuf-\u003esize = size;\n+\tbuf-\u003enpages = DIV_ROUND_UP(size, PAGE_SIZE);\n+\tbuf-\u003epage_shift = PAGE_SHIFT;\n+\tbuf-\u003efrags = kcalloc(buf-\u003enpages, sizeof(struct xsc_buf_list),\n+\t\t\t GFP_KERNEL);\n+\tif (!buf-\u003efrags)\n+\t\tgoto err_out;\n+\n+\tfor (i = 0; i \u003c buf-\u003enpages; i++) {\n+\t\tunsigned long frag_sz = min_t(unsigned long, size, PAGE_SIZE);\n+\t\tstruct xsc_buf_list *frag = \u0026buf-\u003efrags[i];\n+\n+\t\tfrag-\u003ebuf = xsc_dma_zalloc_coherent_node(xdev, frag_sz,\n+\t\t\t\t\t\t\t \u0026frag-\u003emap, node);\n+\t\tif (!frag-\u003ebuf)\n+\t\t\tgoto err_free_buf;\n+\t\tif (frag-\u003emap \u0026 ((1 \u003c\u003c buf-\u003epage_shift) - 1)) {\n+\t\t\tdma_free_coherent(\u0026xdev-\u003epdev-\u003edev, frag_sz,\n+\t\t\t\t\t buf-\u003efrags[i].buf, buf-\u003efrags[i].map);\n+\t\t\tpci_err(xdev-\u003epdev, \"unexpected map alignment: %pad, page_shift=%d\\n\",\n+\t\t\t\t\u0026frag-\u003emap, buf-\u003epage_shift);\n+\t\t\tgoto err_free_buf;\n+\t\t}\n+\t\tsize -= frag_sz;\n+\t}\n+\n+\treturn 0;\n+\n+err_free_buf:\n+\twhile (i--)\n+\t\tdma_free_coherent(\u0026xdev-\u003epdev-\u003edev,\n+\t\t\t\t PAGE_SIZE,\n+\t\t\t\t buf-\u003efrags[i].buf,\n+\t\t\t\t buf-\u003efrags[i].map);\n+\tkfree(buf-\u003efrags);\n+err_out:\n+\treturn -ENOMEM;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_frag_buf_alloc_node);\n+\n+void xsc_core_frag_buf_free(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_frag_buf *buf)\n+{\n+\tunsigned long size = buf-\u003esize;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c buf-\u003enpages; i++) {\n+\t\tunsigned long frag_sz = min_t(unsigned long, size, PAGE_SIZE);\n+\n+\t\tdma_free_coherent(\u0026xdev-\u003epdev-\u003edev, frag_sz, buf-\u003efrags[i].buf,\n+\t\t\t\t buf-\u003efrags[i].map);\n+\t\tsize -= frag_sz;\n+\t}\n+\tkfree(buf-\u003efrags);\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_frag_buf_free);\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.h b/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.h\nnew file mode 100644\nindex 0000000000000..aef39e63235bc\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.h\n@@ -0,0 +1,17 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_ALLOC_H\n+#define __XSC_ALLOC_H\n+\n+#include \"common/xsc_core.h\"\n+\n+int xsc_buf_alloc(struct xsc_core_device *xdev,\n+\t\t unsigned long size,\n+\t\t unsigned long max_direct,\n+\t\t struct xsc_buf *buf);\n+void xsc_buf_free(struct xsc_core_device *xdev, struct xsc_buf *buf);\n+void xsc_fill_page_array(struct xsc_buf *buf, __be64 *pas, unsigned int npages);\n+#endif\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c\nnew file mode 100644\nindex 0000000000000..48832b26dca9e\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c\n@@ -0,0 +1,1247 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ * Copyright (c) 2013-2016, Mellanox Technologies. All rights reserved.\n+ */\n+\n+#include \u003clinux/module.h\u003e\n+#include \u003clinux/errno.h\u003e\n+#include \u003clinux/pci.h\u003e\n+#include \u003clinux/dma-mapping.h\u003e\n+#include \u003clinux/slab.h\u003e\n+#include \u003clinux/delay.h\u003e\n+#include \u003clinux/random.h\u003e\n+#include \u003clinux/kthread.h\u003e\n+#include \u003clinux/io-mapping.h\u003e\n+\n+#include \"common/xsc_cmd_api.h\"\n+#include \"common/xsc_cmd.h\"\n+#include \"common/xsc_core.h\"\n+#include \"hal/xsc_hal.h\"\n+\n+enum {\n+\tCMD_IF_REV = 3,\n+};\n+\n+enum {\n+\tNUM_LONG_LISTS\t = 2,\n+\tNUM_MED_LISTS\t = 64,\n+\tLONG_LIST_SIZE\t = (2ULL * 1024 * 1024 * 1024 / PAGE_SIZE) * 8 + 16 +\n+\t\t\t\tXSC_CMD_DATA_BLOCK_SIZE,\n+\tMED_LIST_SIZE\t = 16 + XSC_CMD_DATA_BLOCK_SIZE,\n+};\n+\n+enum {\n+\tXSC_CMD_DELIVERY_STAT_OK\t\t\t= 0x0,\n+\tXSC_CMD_DELIVERY_STAT_SIGNAT_ERR\t\t= 0x1,\n+\tXSC_CMD_DELIVERY_STAT_TOK_ERR\t\t\t= 0x2,\n+\tXSC_CMD_DELIVERY_STAT_BAD_BLK_NUM_ERR\t\t= 0x3,\n+\tXSC_CMD_DELIVERY_STAT_OUT_PTR_ALIGN_ERR\t\t= 0x4,\n+\tXSC_CMD_DELIVERY_STAT_IN_PTR_ALIGN_ERR\t\t= 0x5,\n+\tXSC_CMD_DELIVERY_STAT_FW_ERR\t\t\t= 0x6,\n+\tXSC_CMD_DELIVERY_STAT_IN_LENGTH_ERR\t\t= 0x7,\n+\tXSC_CMD_DELIVERY_STAT_OUT_LENGTH_ERR\t\t= 0x8,\n+\tXSC_CMD_DELIVERY_STAT_RES_FLD_NOT_CLR_ERR\t= 0x9,\n+\tXSC_CMD_DELIVERY_STAT_CMD_DESCR_ERR\t\t= 0x10,\n+};\n+\n+static struct xsc_cmd_work_ent *xsc_alloc_cmd(struct xsc_cmd *cmd,\n+\t\t\t\t\t struct xsc_cmd_msg *in,\n+\t\t\t\t\t struct xsc_rsp_msg *out)\n+{\n+\tstruct xsc_cmd_work_ent *ent;\n+\n+\tent = kzalloc_obj(*ent, GFP_KERNEL);\n+\tif (!ent)\n+\t\treturn ERR_PTR(-ENOMEM);\n+\n+\tent-\u003ein\t\t= in;\n+\tent-\u003eout\t= out;\n+\tent-\u003ecmd\t= cmd;\n+\n+\treturn ent;\n+}\n+\n+static void xsc_free_cmd(struct xsc_cmd_work_ent *ent)\n+{\n+\tkfree(ent);\n+}\n+\n+static u8 xsc_alloc_token(struct xsc_cmd *cmd)\n+{\n+\tu8 token;\n+\n+\tspin_lock(\u0026cmd-\u003etoken_lock);\n+\ttoken = cmd-\u003etoken++ % 255 + 1;\n+\tspin_unlock(\u0026cmd-\u003etoken_lock);\n+\n+\treturn token;\n+}\n+\n+static int xsc_alloc_ent(struct xsc_cmd *cmd)\n+{\n+\tunsigned long flags;\n+\tint ret;\n+\n+\tspin_lock_irqsave(\u0026cmd-\u003ealloc_lock, flags);\n+\tret = find_first_bit(\u0026cmd-\u003ecmd_entry_mask, cmd-\u003emax_reg_cmds);\n+\tif (ret \u003c cmd-\u003emax_reg_cmds)\n+\t\tclear_bit(ret, \u0026cmd-\u003ecmd_entry_mask);\n+\tspin_unlock_irqrestore(\u0026cmd-\u003ealloc_lock, flags);\n+\n+\treturn ret \u003c cmd-\u003emax_reg_cmds ? ret : -ENOSPC;\n+}\n+\n+static void xsc_free_ent(struct xsc_cmd *cmd, int idx)\n+{\n+\tunsigned long flags;\n+\n+\tspin_lock_irqsave(\u0026cmd-\u003ealloc_lock, flags);\n+\tset_bit(idx, \u0026cmd-\u003ecmd_entry_mask);\n+\tspin_unlock_irqrestore(\u0026cmd-\u003ealloc_lock, flags);\n+}\n+\n+static struct xsc_cmd_layout *xsc_get_inst(struct xsc_cmd *cmd, int idx)\n+{\n+\treturn cmd-\u003ecmd_buf + (idx \u003c\u003c cmd-\u003elog_stride);\n+}\n+\n+static struct xsc_rsp_layout *xsc_get_cq_inst(struct xsc_cmd *cmd, int idx)\n+{\n+\treturn cmd-\u003ecq_buf + (idx \u003c\u003c cmd-\u003elog_stride);\n+}\n+\n+static u8 xsc_xor8_buf(void *buf, int len)\n+{\n+\tu8 *ptr = buf;\n+\tu8 sum = 0;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c len; i++)\n+\t\tsum ^= ptr[i];\n+\n+\treturn sum;\n+}\n+\n+static int xsc_verify_block_sig(struct xsc_cmd_prot_block *block)\n+{\n+\t/* rsvd0 was set to 0xFF by fw */\n+\tif (xsc_xor8_buf(block-\u003ersvd0,\n+\t\t\t sizeof(*block) - sizeof(block-\u003edata) - 1) != 0xff)\n+\t\treturn -EINVAL;\n+\n+\tif (xsc_xor8_buf(block, sizeof(*block)) != 0xff)\n+\t\treturn -EINVAL;\n+\n+\treturn 0;\n+}\n+\n+static void xsc_calc_block_sig(struct xsc_cmd_prot_block *block, u8 token)\n+{\n+\tblock-\u003etoken = token;\n+\tblock-\u003ectrl_sig = ~xsc_xor8_buf(block-\u003ersvd0,\n+\t\t\t\t sizeof(*block) - sizeof(block-\u003edata) - 2);\n+\tblock-\u003esig = ~xsc_xor8_buf(block, sizeof(*block) - 1);\n+}\n+\n+static void xsc_calc_chain_sig(struct xsc_cmd_mailbox *head, u8 token)\n+{\n+\tstruct xsc_cmd_mailbox *next = head;\n+\n+\twhile (next) {\n+\t\txsc_calc_block_sig(next-\u003ebuf, token);\n+\t\tnext = next-\u003enext;\n+\t}\n+}\n+\n+static void xsc_set_signature(struct xsc_cmd_work_ent *ent)\n+{\n+\tent-\u003elay-\u003esig = ~xsc_xor8_buf(ent-\u003elay, sizeof(*ent-\u003elay));\n+\txsc_calc_chain_sig(ent-\u003ein-\u003enext, ent-\u003etoken);\n+\txsc_calc_chain_sig(ent-\u003eout-\u003enext, ent-\u003etoken);\n+}\n+\n+static int xsc_verify_signature(struct xsc_cmd_work_ent *ent)\n+{\n+\tstruct xsc_cmd_mailbox *next = ent-\u003eout-\u003enext;\n+\tint err;\n+\tu8 sig;\n+\n+\tsig = xsc_xor8_buf(ent-\u003ersp_lay, sizeof(*ent-\u003ersp_lay));\n+\tif (sig != 0xff)\n+\t\treturn -EINVAL;\n+\n+\twhile (next) {\n+\t\terr = xsc_verify_block_sig(next-\u003ebuf);\n+\t\tif (err)\n+\t\t\treturn err;\n+\n+\t\tnext = next-\u003enext;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static const char *xsc_command_str(int command)\n+{\n+\tswitch (command) {\n+\tcase XSC_CMD_OP_QUERY_HCA_CAP:\n+\t\treturn \"QUERY_HCA_CAP\";\n+\n+\tcase XSC_CMD_OP_QUERY_CMDQ_VERSION:\n+\t\treturn \"QUERY_CMDQ_VERSION\";\n+\n+\tcase XSC_CMD_OP_FUNCTION_RESET:\n+\t\treturn \"FUNCTION_RESET\";\n+\n+\tcase XSC_CMD_OP_DUMMY:\n+\t\treturn \"DUMMY_CMD\";\n+\n+\tcase XSC_CMD_OP_CREATE_EQ:\n+\t\treturn \"CREATE_EQ\";\n+\n+\tcase XSC_CMD_OP_DESTROY_EQ:\n+\t\treturn \"DESTROY_EQ\";\n+\n+\tcase XSC_CMD_OP_CREATE_CQ:\n+\t\treturn \"CREATE_CQ\";\n+\n+\tcase XSC_CMD_OP_DESTROY_CQ:\n+\t\treturn \"DESTROY_CQ\";\n+\n+\tcase XSC_CMD_OP_CREATE_QP:\n+\t\treturn \"CREATE_QP\";\n+\n+\tcase XSC_CMD_OP_DESTROY_QP:\n+\t\treturn \"DESTROY_QP\";\n+\n+\tcase XSC_CMD_OP_RST2INIT_QP:\n+\t\treturn \"RST2INIT_QP\";\n+\n+\tcase XSC_CMD_OP_INIT2RTR_QP:\n+\t\treturn \"INIT2RTR_QP\";\n+\n+\tcase XSC_CMD_OP_RTR2RTS_QP:\n+\t\treturn \"RTR2RTS_QP\";\n+\n+\tcase XSC_CMD_OP_RTS2RTS_QP:\n+\t\treturn \"RTS2RTS_QP\";\n+\n+\tcase XSC_CMD_OP_SQERR2RTS_QP:\n+\t\treturn \"SQERR2RTS_QP\";\n+\n+\tcase XSC_CMD_OP_2ERR_QP:\n+\t\treturn \"2ERR_QP\";\n+\n+\tcase XSC_CMD_OP_RTS2SQD_QP:\n+\t\treturn \"RTS2SQD_QP\";\n+\n+\tcase XSC_CMD_OP_SQD2RTS_QP:\n+\t\treturn \"SQD2RTS_QP\";\n+\n+\tcase XSC_CMD_OP_2RST_QP:\n+\t\treturn \"2RST_QP\";\n+\n+\tcase XSC_CMD_OP_INIT2INIT_QP:\n+\t\treturn \"INIT2INIT_QP\";\n+\n+\tcase XSC_CMD_OP_MODIFY_RAW_QP:\n+\t\treturn \"MODIFY_RAW_QP\";\n+\n+\tcase XSC_CMD_OP_ENABLE_NIC_HCA:\n+\t\treturn \"ENABLE_NIC_HCA\";\n+\n+\tcase XSC_CMD_OP_DISABLE_NIC_HCA:\n+\t\treturn \"DISABLE_NIC_HCA\";\n+\n+\tcase XSC_CMD_OP_QUERY_VPORT_STATE:\n+\t\treturn \"QUERY_VPORT_STATE\";\n+\n+\tcase XSC_CMD_OP_QUERY_EVENT_TYPE:\n+\t\treturn \"QUERY_EVENT_TYPE\";\n+\n+\tcase XSC_CMD_OP_ENABLE_MSIX:\n+\t\treturn \"ENABLE_MSIX\";\n+\n+\tcase XSC_CMD_OP_SET_MTU:\n+\t\treturn \"SET_MTU\";\n+\n+\tcase XSC_CMD_OP_QUERY_ETH_MAC:\n+\t\treturn \"QUERY_ETH_MAC\";\n+\n+\tdefault: return \"unknown command opcode\";\n+\t}\n+}\n+\n+static void cmd_work_handler(struct work_struct *work)\n+{\n+\tstruct xsc_cmd_work_ent *ent;\n+\tstruct xsc_core_device *xdev;\n+\tstruct xsc_cmd_layout *lay;\n+\tstruct semaphore *sem;\n+\tstruct xsc_cmd *cmd;\n+\tunsigned long flags;\n+\n+\tent = container_of(work, struct xsc_cmd_work_ent, work);\n+\tcmd = ent-\u003ecmd;\n+\txdev = container_of(cmd, struct xsc_core_device, cmd);\n+\tsem = \u0026cmd-\u003esem;\n+\tdown(sem);\n+\tent-\u003eidx = xsc_alloc_ent(cmd);\n+\tif (ent-\u003eidx \u003c 0) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to allocate command entry\\n\");\n+\t\tup(sem);\n+\t\treturn;\n+\t}\n+\n+\tent-\u003etoken = xsc_alloc_token(cmd);\n+\tcmd-\u003eent_arr[ent-\u003eidx] = ent;\n+\n+\tspin_lock_irqsave(\u0026cmd-\u003edoorbell_lock, flags);\n+\tlay = xsc_get_inst(cmd, cmd-\u003ecmd_pid);\n+\tent-\u003elay = lay;\n+\tmemset(lay, 0, sizeof(*lay));\n+\tmemcpy(lay-\u003ein, ent-\u003ein-\u003efirst.data, sizeof(lay-\u003ein));\n+\tif (ent-\u003ein-\u003enext)\n+\t\tlay-\u003ein_ptr = cpu_to_be64(ent-\u003ein-\u003enext-\u003edma);\n+\tlay-\u003einlen = cpu_to_be32(ent-\u003ein-\u003elen);\n+\tif (ent-\u003eout-\u003enext)\n+\t\tlay-\u003eout_ptr = cpu_to_be64(ent-\u003eout-\u003enext-\u003edma);\n+\tlay-\u003eoutlen = cpu_to_be32(ent-\u003eout-\u003elen);\n+\tlay-\u003etype = XSC_PCI_CMD_XPORT;\n+\tlay-\u003etoken = ent-\u003etoken;\n+\tlay-\u003eidx = ent-\u003eidx;\n+\tif (!cmd-\u003echecksum_disabled)\n+\t\txsc_set_signature(ent);\n+\telse\n+\t\tlay-\u003esig = 0xff;\n+\n+\t/* ring doorbell after the descriptor is valid */\n+\twmb();\n+\n+\tcmd-\u003ecmd_pid = (cmd-\u003ecmd_pid + 1) % (1 \u003c\u003c cmd-\u003elog_sz);\n+\txsc_update_cmdq_req_pid(xdev, cmd-\u003ecmd_pid);\n+\tspin_unlock_irqrestore(\u0026cmd-\u003edoorbell_lock, flags);\n+}\n+\n+static const char *xsc_deliv_status_to_str(u8 status)\n+{\n+\tswitch (status) {\n+\tcase XSC_CMD_DELIVERY_STAT_OK:\n+\t\treturn \"no errors\";\n+\tcase XSC_CMD_DELIVERY_STAT_SIGNAT_ERR:\n+\t\treturn \"signature error\";\n+\tcase XSC_CMD_DELIVERY_STAT_TOK_ERR:\n+\t\treturn \"token error\";\n+\tcase XSC_CMD_DELIVERY_STAT_BAD_BLK_NUM_ERR:\n+\t\treturn \"bad block number\";\n+\tcase XSC_CMD_DELIVERY_STAT_OUT_PTR_ALIGN_ERR:\n+\t\treturn \"output pointer not aligned to block size\";\n+\tcase XSC_CMD_DELIVERY_STAT_IN_PTR_ALIGN_ERR:\n+\t\treturn \"input pointer not aligned to block size\";\n+\tcase XSC_CMD_DELIVERY_STAT_FW_ERR:\n+\t\treturn \"firmware internal error\";\n+\tcase XSC_CMD_DELIVERY_STAT_IN_LENGTH_ERR:\n+\t\treturn \"command input length error\";\n+\tcase XSC_CMD_DELIVERY_STAT_OUT_LENGTH_ERR:\n+\t\treturn \"command output length error\";\n+\tcase XSC_CMD_DELIVERY_STAT_RES_FLD_NOT_CLR_ERR:\n+\t\treturn \"reserved fields not cleared\";\n+\tcase XSC_CMD_DELIVERY_STAT_CMD_DESCR_ERR:\n+\t\treturn \"bad command descriptor type\";\n+\tdefault:\n+\t\treturn \"unknown status code\";\n+\t}\n+}\n+\n+static u16 xsc_msg_to_opcode(struct xsc_cmd_msg *in)\n+{\n+\tstruct xsc_inbox_hdr *hdr = (struct xsc_inbox_hdr *)(in-\u003efirst.data);\n+\n+\treturn be16_to_cpu(hdr-\u003eopcode);\n+}\n+\n+static int xsc_wait_func(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_cmd_work_ent *ent)\n+{\n+\tunsigned long timeout = msecs_to_jiffies(XSC_CMD_TIMEOUT_MSEC);\n+\tstruct xsc_cmd *cmd = \u0026xdev-\u003ecmd;\n+\tint err;\n+\n+\tif (!wait_for_completion_timeout(\u0026ent-\u003edone, timeout))\n+\t\terr = -ETIMEDOUT;\n+\telse\n+\t\terr = ent-\u003eret;\n+\n+\tif (err == -ETIMEDOUT) {\n+\t\tcmd-\u003ecmd_status = XSC_CMD_STATUS_TIMEDOUT;\n+\t\tpci_err(xdev-\u003epdev, \"wait for %s(0x%x) response timeout!\\n\",\n+\t\t\txsc_command_str(xsc_msg_to_opcode(ent-\u003ein)),\n+\t\t\txsc_msg_to_opcode(ent-\u003ein));\n+\t} else if (err) {\n+\t\tpci_err(xdev-\u003epdev, \"err %d, delivery status %s(%d)\\n\", err,\n+\t\t\txsc_deliv_status_to_str(ent-\u003estatus), ent-\u003estatus);\n+\t}\n+\n+\treturn err;\n+}\n+\n+/* Notes:\n+ * 1. Callback functions may not sleep\n+ * 2. page queue commands do not support asynchrous completion\n+ */\n+static int xsc_cmd_invoke(struct xsc_core_device *xdev, struct xsc_cmd_msg *in,\n+\t\t\t struct xsc_rsp_msg *out, u8 *status)\n+{\n+\tstruct xsc_cmd *cmd = \u0026xdev-\u003ecmd;\n+\tstruct xsc_cmd_work_ent *ent;\n+\tstruct semaphore *sem;\n+\tint err = 0;\n+\n+\tent = xsc_alloc_cmd(cmd, in, out);\n+\tif (IS_ERR(ent))\n+\t\treturn PTR_ERR(ent);\n+\n+\tinit_completion(\u0026ent-\u003edone);\n+\tINIT_WORK(\u0026ent-\u003ework, cmd_work_handler);\n+\tif (!queue_work(cmd-\u003ewq, \u0026ent-\u003ework)) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to queue work\\n\");\n+\t\terr = -ENOMEM;\n+\t\tgoto err_free;\n+\t}\n+\n+\terr = xsc_wait_func(xdev, ent);\n+\tif (err == -ETIMEDOUT)\n+\t\tgoto err_sem_up;\n+\t*status = ent-\u003estatus;\n+\txsc_free_cmd(ent);\n+\n+\treturn err;\n+\n+err_sem_up:\n+\tsem = \u0026cmd-\u003esem;\n+\tup(sem);\n+err_free:\n+\txsc_free_cmd(ent);\n+\treturn err;\n+}\n+\n+static int xsc_copy_to_cmd_msg(struct xsc_cmd_msg *to, void *from,\n+\t\t\t unsigned int size)\n+{\n+\tstruct xsc_cmd_prot_block *block;\n+\tstruct xsc_cmd_mailbox *next;\n+\tunsigned int copy;\n+\n+\tif (!to || !from)\n+\t\treturn -ENOMEM;\n+\n+\tcopy = min(size, sizeof(to-\u003efirst.data));\n+\tmemcpy(to-\u003efirst.data, from, copy);\n+\tsize -= copy;\n+\tfrom += copy;\n+\n+\tnext = to-\u003enext;\n+\twhile (size) {\n+\t\tif (!next) {\n+\t\t\tWARN_ONCE(1, \"Mail box not enough\\n\");\n+\t\t\treturn -ENOMEM;\n+\t\t}\n+\n+\t\tcopy = min(size, XSC_CMD_DATA_BLOCK_SIZE);\n+\t\tblock = next-\u003ebuf;\n+\t\tmemcpy(block-\u003edata, from, copy);\n+\t\tblock-\u003eowner_status = 0;\n+\t\tfrom += copy;\n+\t\tsize -= copy;\n+\t\tnext = next-\u003enext;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int xsc_copy_from_rsp_msg(void *to, struct xsc_rsp_msg *from,\n+\t\t\t\t unsigned int size)\n+{\n+\tstruct xsc_cmd_prot_block *block;\n+\tstruct xsc_cmd_mailbox *next;\n+\tunsigned int copy;\n+\n+\tif (!to || !from)\n+\t\treturn -ENOMEM;\n+\n+\tcopy = min(size, sizeof(from-\u003efirst.data));\n+\tmemcpy(to, from-\u003efirst.data, copy);\n+\tsize -= copy;\n+\tto += copy;\n+\n+\tnext = from-\u003enext;\n+\twhile (size) {\n+\t\tif (!next) {\n+\t\t\tWARN_ONCE(1, \"Mail too short\\n\");\n+\t\t\treturn -ENOMEM;\n+\t\t}\n+\n+\t\tcopy = min(size, XSC_CMD_DATA_BLOCK_SIZE);\n+\t\tblock = next-\u003ebuf;\n+\t\tif (!block-\u003eowner_status)\n+\t\t\tpr_err(\"block ownership check failed\\n\");\n+\n+\t\tmemcpy(to, block-\u003edata, copy);\n+\t\tto += copy;\n+\t\tsize -= copy;\n+\t\tnext = next-\u003enext;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static struct xsc_cmd_mailbox *xsc_alloc_cmd_box(struct xsc_core_device *xdev,\n+\t\t\t\t\t\t gfp_t flags)\n+{\n+\tstruct xsc_cmd_mailbox *mailbox;\n+\n+\tmailbox = kmalloc_obj(*mailbox, flags);\n+\tif (!mailbox)\n+\t\treturn ERR_PTR(-ENOMEM);\n+\n+\tmailbox-\u003ebuf = dma_pool_zalloc(xdev-\u003ecmd.pool, flags,\n+\t\t\t\t \u0026mailbox-\u003edma);\n+\tif (!mailbox-\u003ebuf) {\n+\t\tkfree(mailbox);\n+\t\treturn ERR_PTR(-ENOMEM);\n+\t}\n+\tmailbox-\u003enext = NULL;\n+\n+\treturn mailbox;\n+}\n+\n+static void xsc_free_cmd_box(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_cmd_mailbox *mailbox)\n+{\n+\tdma_pool_free(xdev-\u003ecmd.pool, mailbox-\u003ebuf, mailbox-\u003edma);\n+\n+\tkfree(mailbox);\n+}\n+\n+static struct xsc_cmd_msg *xsc_alloc_cmd_msg(struct xsc_core_device *xdev,\n+\t\t\t\t\t gfp_t flags, unsigned int size)\n+{\n+\tstruct xsc_cmd_mailbox *tmp, *head = NULL;\n+\tstruct xsc_cmd_prot_block *block;\n+\tstruct xsc_cmd_msg *msg;\n+\tint blen;\n+\tint err;\n+\tint n;\n+\tint i;\n+\n+\tmsg = kzalloc_obj(*msg, GFP_KERNEL);\n+\tif (!msg)\n+\t\treturn ERR_PTR(-ENOMEM);\n+\n+\tblen = size - min(sizeof(msg-\u003efirst.data), size);\n+\tn = (blen + XSC_CMD_DATA_BLOCK_SIZE - 1) / XSC_CMD_DATA_BLOCK_SIZE;\n+\n+\tfor (i = 0; i \u003c n; i++) {\n+\t\ttmp = xsc_alloc_cmd_box(xdev, flags);\n+\t\tif (IS_ERR(tmp)) {\n+\t\t\tpci_err(xdev-\u003epdev, \"failed allocating block\\n\");\n+\t\t\terr = PTR_ERR(tmp);\n+\t\t\tgoto err_free;\n+\t\t}\n+\n+\t\tblock = tmp-\u003ebuf;\n+\t\ttmp-\u003enext = head;\n+\t\tblock-\u003enext = cpu_to_be64(tmp-\u003enext ? tmp-\u003enext-\u003edma : 0);\n+\t\tblock-\u003eblock_num = cpu_to_be32(n - i - 1);\n+\t\thead = tmp;\n+\t}\n+\tmsg-\u003enext = head;\n+\tmsg-\u003elen = size;\n+\treturn msg;\n+\n+err_free:\n+\twhile (head) {\n+\t\ttmp = head-\u003enext;\n+\t\txsc_free_cmd_box(xdev, head);\n+\t\thead = tmp;\n+\t}\n+\tkfree(msg);\n+\n+\treturn ERR_PTR(err);\n+}\n+\n+static void xsc_free_cmd_msg(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_cmd_msg *msg)\n+{\n+\tstruct xsc_cmd_mailbox *head = msg-\u003enext;\n+\tstruct xsc_cmd_mailbox *next;\n+\n+\twhile (head) {\n+\t\tnext = head-\u003enext;\n+\t\txsc_free_cmd_box(xdev, head);\n+\t\thead = next;\n+\t}\n+\tkfree(msg);\n+}\n+\n+static struct xsc_rsp_msg *xsc_alloc_rsp_msg(struct xsc_core_device *xdev,\n+\t\t\t\t\t gfp_t flags, unsigned int size)\n+{\n+\tstruct xsc_cmd_mailbox *tmp, *head = NULL;\n+\tstruct xsc_cmd_prot_block *block;\n+\tstruct xsc_rsp_msg *msg;\n+\tint blen;\n+\tint err;\n+\tint n;\n+\tint i;\n+\n+\tmsg = kzalloc_obj(*msg, GFP_KERNEL);\n+\tif (!msg)\n+\t\treturn ERR_PTR(-ENOMEM);\n+\n+\tblen = size - min_t(int, sizeof(msg-\u003efirst.data), size);\n+\tn = (blen + XSC_CMD_DATA_BLOCK_SIZE - 1) / XSC_CMD_DATA_BLOCK_SIZE;\n+\n+\tfor (i = 0; i \u003c n; i++) {\n+\t\ttmp = xsc_alloc_cmd_box(xdev, flags);\n+\t\tif (IS_ERR(tmp)) {\n+\t\t\tpci_err(xdev-\u003epdev, \"failed allocating block\\n\");\n+\t\t\terr = PTR_ERR(tmp);\n+\t\t\tgoto err_free;\n+\t\t}\n+\n+\t\tblock = tmp-\u003ebuf;\n+\t\ttmp-\u003enext = head;\n+\t\tblock-\u003enext = cpu_to_be64(tmp-\u003enext ? tmp-\u003enext-\u003edma : 0);\n+\t\tblock-\u003eblock_num = cpu_to_be32(n - i - 1);\n+\t\thead = tmp;\n+\t}\n+\tmsg-\u003enext = head;\n+\tmsg-\u003elen = size;\n+\treturn msg;\n+\n+err_free:\n+\twhile (head) {\n+\t\ttmp = head-\u003enext;\n+\t\txsc_free_cmd_box(xdev, head);\n+\t\thead = tmp;\n+\t}\n+\tkfree(msg);\n+\n+\treturn ERR_PTR(err);\n+}\n+\n+static void xsc_free_rsp_msg(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_rsp_msg *msg)\n+{\n+\tstruct xsc_cmd_mailbox *head = msg-\u003enext;\n+\tstruct xsc_cmd_mailbox *next;\n+\n+\twhile (head) {\n+\t\tnext = head-\u003enext;\n+\t\txsc_free_cmd_box(xdev, head);\n+\t\thead = next;\n+\t}\n+\tkfree(msg);\n+}\n+\n+static void xsc_set_wqname(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_cmd *cmd = \u0026xdev-\u003ecmd;\n+\n+\tsnprintf(cmd-\u003ewq_name, sizeof(cmd-\u003ewq_name), \"xsc_cmd_%s\",\n+\t\t dev_name(\u0026xdev-\u003epdev-\u003edev));\n+}\n+\n+void xsc_cmd_use_events(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_cmd *cmd = \u0026xdev-\u003ecmd;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c cmd-\u003emax_reg_cmds; i++)\n+\t\tdown(\u0026cmd-\u003esem);\n+\n+\tflush_workqueue(cmd-\u003ewq);\n+\n+\tcmd-\u003emode = XSC_CMD_MODE_EVENTS;\n+\n+\twhile (cmd-\u003ecmd_pid != cmd-\u003ecq_cid)\n+\t\tmsleep(20);\n+\tkthread_stop(cmd-\u003ecq_task);\n+\tcmd-\u003ecq_task = NULL;\n+\n+\tfor (i = 0; i \u003c cmd-\u003emax_reg_cmds; i++)\n+\t\tup(\u0026cmd-\u003esem);\n+}\n+\n+static int xsc_cmd_cq_polling(void *data);\n+void xsc_cmd_use_polling(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_cmd *cmd = \u0026xdev-\u003ecmd;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c cmd-\u003emax_reg_cmds; i++)\n+\t\tdown(\u0026cmd-\u003esem);\n+\n+\tflush_workqueue(cmd-\u003ewq);\n+\tcmd-\u003emode = XSC_CMD_MODE_POLLING;\n+\tcmd-\u003ecq_task = kthread_create(xsc_cmd_cq_polling,\n+\t\t\t\t (void *)xdev,\n+\t\t\t\t \"xsc_cmd_cq_polling\");\n+\tif (cmd-\u003ecq_task)\n+\t\twake_up_process(cmd-\u003ecq_task);\n+\n+\tfor (i = 0; i \u003c cmd-\u003emax_reg_cmds; i++)\n+\t\tup(\u0026cmd-\u003esem);\n+}\n+\n+int xsc_cmd_status_to_err(struct xsc_outbox_hdr *hdr)\n+{\n+\tswitch (hdr-\u003estatus) {\n+\tcase XSC_CMD_STATUS_OK:\n+\t\treturn 0;\n+\tcase XSC_CMD_STATUS_FAIL:\n+\t\treturn -EIO;\n+\tcase XSC_CMD_STATUS_NOT_SUPPORTED:\n+\t\treturn -EOPNOTSUPP;\n+\tcase XSC_CMD_STATUS_BAD_PARAM:\n+\t\treturn -EINVAL;\n+\tcase XSC_CMD_STATUS_INVAL_RES:\n+\tcase XSC_CMD_STATUS_INVAL_DATA:\n+\tcase XSC_CMD_STATUS_NO_RES:\n+\tcase XSC_CMD_STATUS_NO_MPT_RES:\n+\tcase XSC_CMD_STATUS_NO_MTT_RES:\n+\tcase XSC_CMD_STATUS_NO_EQN_RES:\n+\tcase XSC_CMD_STATUS_NO_EQ_PA_RES:\n+\tcase XSC_CMD_STATUS_NO_CQN_RES:\n+\tcase XSC_CMD_STATUS_NO_CQ_PA_RES:\n+\tcase XSC_CMD_STATUS_NO_QPN_RES:\n+\tcase XSC_CMD_STATUS_NO_QP_PA_RES:\n+\tcase XSC_CMD_STATUS_NO_PDN_RES:\n+\tcase XSC_CMD_STATUS_BAD_INBUF:\n+\tcase XSC_CMD_STATUS_BAD_OUTBUF:\n+\t\treturn -EIO;\n+\tcase XSC_CMD_STATUS_BUSY:\n+\tcase XSC_CMD_STATUS_QP_FLUSH_BUSY:\n+\t\treturn -EBUSY;\n+\tcase XSC_CMD_STATUS_QP_FLUSH_PENDING:\n+\t\treturn -EAGAIN;\n+\tcase XSC_CMD_STATUS_NOT_FOUND:\n+\tcase XSC_CMD_STATUS_INVAL_FUNC:\n+\t\treturn -ENODEV;\n+\tdefault:\n+\t\treturn -EIO;\n+\t}\n+}\n+\n+static int xsc_status_to_err(u8 status)\n+{\n+\tstruct xsc_outbox_hdr hdr = { .status = status };\n+\n+\treturn xsc_cmd_status_to_err(\u0026hdr);\n+}\n+\n+static struct xsc_cmd_msg *xsc_alloc_msg(struct xsc_core_device *xdev,\n+\t\t\t\t\t unsigned int in_size)\n+{\n+\tstruct xsc_cmd_msg *msg = ERR_PTR(-ENOMEM);\n+\tstruct xsc_cmd *cmd = \u0026xdev-\u003ecmd;\n+\tstruct cache_ent *ent = NULL;\n+\n+\tif (in_size \u003e MED_LIST_SIZE \u0026\u0026 in_size \u003c= LONG_LIST_SIZE)\n+\t\tent = \u0026cmd-\u003ecache.large;\n+\telse if (in_size \u003e 16 \u0026\u0026 in_size \u003c= MED_LIST_SIZE)\n+\t\tent = \u0026cmd-\u003ecache.med;\n+\n+\tif (ent) {\n+\t\tspin_lock(\u0026ent-\u003elock);\n+\t\tif (!list_empty(\u0026ent-\u003ehead)) {\n+\t\t\tmsg = list_entry(ent-\u003ehead.next, typeof(*msg), list);\n+\t\t\t/* For cached lists, we must explicitly state what is\n+\t\t\t * the real size\n+\t\t\t */\n+\t\t\tmsg-\u003elen = in_size;\n+\t\t\tlist_del(\u0026msg-\u003elist);\n+\t\t}\n+\t\tspin_unlock(\u0026ent-\u003elock);\n+\t}\n+\n+\tif (IS_ERR(msg))\n+\t\tmsg = xsc_alloc_cmd_msg(xdev, GFP_KERNEL, in_size);\n+\n+\treturn msg;\n+}\n+\n+static void xsc_free_msg(struct xsc_core_device *xdev, struct xsc_cmd_msg *msg)\n+{\n+\tif (msg-\u003ecache) {\n+\t\tspin_lock(\u0026msg-\u003ecache-\u003elock);\n+\t\tlist_add_tail(\u0026msg-\u003elist, \u0026msg-\u003ecache-\u003ehead);\n+\t\tspin_unlock(\u0026msg-\u003ecache-\u003elock);\n+\t} else {\n+\t\txsc_free_cmd_msg(xdev, msg);\n+\t}\n+}\n+\n+static int _xsc_cmd_exec(struct xsc_core_device *xdev,\n+\t\t\t void *in, unsigned int in_size,\n+\t\t\t void *out, unsigned int out_size)\n+{\n+\tstruct xsc_cmd *cmd = \u0026xdev-\u003ecmd;\n+\tstruct xsc_rsp_msg *outb;\n+\tstruct xsc_cmd_msg *inb;\n+\tu8 status = 0;\n+\tint err;\n+\n+\tif (cmd-\u003ecmd_status == XSC_CMD_STATUS_TIMEDOUT)\n+\t\treturn -ETIMEDOUT;\n+\n+\tinb = xsc_alloc_msg(xdev, in_size);\n+\tif (IS_ERR(inb)) {\n+\t\terr = PTR_ERR(inb);\n+\t\treturn err;\n+\t}\n+\n+\terr = xsc_copy_to_cmd_msg(inb, in, in_size);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"copy to cmd_msg err %d\\n\", err);\n+\t\tgoto err_free_msg;\n+\t}\n+\n+\toutb = xsc_alloc_rsp_msg(xdev, GFP_KERNEL, out_size);\n+\tif (IS_ERR(outb)) {\n+\t\terr = PTR_ERR(outb);\n+\t\tgoto err_free_msg;\n+\t}\n+\n+\terr = xsc_cmd_invoke(xdev, inb, outb, \u0026status);\n+\tif (err)\n+\t\tgoto err_free_rsp;\n+\n+\tif (status) {\n+\t\tpci_err(xdev-\u003epdev, \"opcode:%#x, err %d, status %d\\n\",\n+\t\t\txsc_msg_to_opcode(inb), err, status);\n+\t\terr = xsc_status_to_err(status);\n+\t\tgoto err_free_rsp;\n+\t}\n+\n+\terr = xsc_copy_from_rsp_msg(out, outb, out_size);\n+\n+err_free_rsp:\n+\txsc_free_rsp_msg(xdev, outb);\n+\n+err_free_msg:\n+\txsc_free_msg(xdev, inb);\n+\treturn err;\n+}\n+\n+int xsc_cmd_exec(struct xsc_core_device *xdev,\n+\t\t void *in, unsigned int in_size,\n+\t\t void *out, unsigned int out_size)\n+{\n+\tstruct xsc_outbox_hdr *out_hdr = (struct xsc_outbox_hdr *)out;\n+\tstruct xsc_inbox_hdr *in_hdr = (struct xsc_inbox_hdr *)in;\n+\tint err;\n+\n+\tin_hdr-\u003ever = 0;\n+\tif (in_hdr-\u003ever != 0) {\n+\t\tpci_err(xdev-\u003epdev, \"recv an unexpected cmd ver = %d, opcode = %d\\n\",\n+\t\t\tbe16_to_cpu(in_hdr-\u003ever), be16_to_cpu(in_hdr-\u003eopcode));\n+\t\tWARN_ON(in_hdr-\u003ever != 0);\n+\t}\n+\n+\terr = _xsc_cmd_exec(xdev, in, in_size, out, out_size);\n+\tif (err)\n+\t\treturn err;\n+\tif (out_hdr-\u003estatus) {\n+\t\terr = -EIO;\n+\t\treturn err;\n+\t}\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_cmd_exec);\n+\n+static void xsc_destroy_msg_cache(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_cmd *cmd = \u0026xdev-\u003ecmd;\n+\tstruct xsc_cmd_msg *msg;\n+\tstruct xsc_cmd_msg *n;\n+\n+\tlist_for_each_entry_safe(msg, n, \u0026cmd-\u003ecache.large.head, list) {\n+\t\tlist_del(\u0026msg-\u003elist);\n+\t\txsc_free_cmd_msg(xdev, msg);\n+\t}\n+\n+\tlist_for_each_entry_safe(msg, n, \u0026cmd-\u003ecache.med.head, list) {\n+\t\tlist_del(\u0026msg-\u003elist);\n+\t\txsc_free_cmd_msg(xdev, msg);\n+\t}\n+}\n+\n+static int xsc_create_msg_cache(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_cmd *cmd = \u0026xdev-\u003ecmd;\n+\tstruct xsc_cmd_msg *msg;\n+\tint err;\n+\tint i;\n+\n+\tspin_lock_init(\u0026cmd-\u003ecache.large.lock);\n+\tINIT_LIST_HEAD(\u0026cmd-\u003ecache.large.head);\n+\tspin_lock_init(\u0026cmd-\u003ecache.med.lock);\n+\tINIT_LIST_HEAD(\u0026cmd-\u003ecache.med.head);\n+\n+\tfor (i = 0; i \u003c NUM_LONG_LISTS; i++) {\n+\t\tmsg = xsc_alloc_cmd_msg(xdev, GFP_KERNEL, LONG_LIST_SIZE);\n+\t\tif (IS_ERR(msg)) {\n+\t\t\terr = PTR_ERR(msg);\n+\t\t\tgoto err_destroy;\n+\t\t}\n+\t\tmsg-\u003ecache = \u0026cmd-\u003ecache.large;\n+\t\tlist_add_tail(\u0026msg-\u003elist, \u0026cmd-\u003ecache.large.head);\n+\t}\n+\n+\tfor (i = 0; i \u003c NUM_MED_LISTS; i++) {\n+\t\tmsg = xsc_alloc_cmd_msg(xdev, GFP_KERNEL, MED_LIST_SIZE);\n+\t\tif (IS_ERR(msg)) {\n+\t\t\terr = PTR_ERR(msg);\n+\t\t\tgoto err_destroy;\n+\t\t}\n+\t\tmsg-\u003ecache = \u0026cmd-\u003ecache.med;\n+\t\tlist_add_tail(\u0026msg-\u003elist, \u0026cmd-\u003ecache.med.head);\n+\t}\n+\n+\treturn 0;\n+\n+err_destroy:\n+\txsc_destroy_msg_cache(xdev);\n+\treturn err;\n+}\n+\n+static void xsc_cmd_comp_handler(struct xsc_core_device *xdev, u8 idx,\n+\t\t\t\t struct xsc_rsp_layout *rsp)\n+{\n+\tstruct xsc_cmd *cmd = \u0026xdev-\u003ecmd;\n+\tstruct xsc_cmd_work_ent *ent;\n+\n+\tif (idx \u003e cmd-\u003emax_reg_cmds || (cmd-\u003ecmd_entry_mask \u0026 (1 \u003c\u003c idx))) {\n+\t\tpci_err(xdev-\u003epdev, \"idx[%d] exceed max cmds, or has no relative request.\\n\",\n+\t\t\tidx);\n+\t\treturn;\n+\t}\n+\tent = cmd-\u003eent_arr[idx];\n+\tent-\u003ersp_lay = rsp;\n+\n+\tmemcpy(ent-\u003eout-\u003efirst.data,\n+\t ent-\u003ersp_lay-\u003eout,\n+\t sizeof(ent-\u003ersp_lay-\u003eout));\n+\tif (!cmd-\u003echecksum_disabled)\n+\t\tent-\u003eret = xsc_verify_signature(ent);\n+\telse\n+\t\tent-\u003eret = 0;\n+\tent-\u003estatus = 0;\n+\n+\txsc_free_ent(cmd, ent-\u003eidx);\n+\tcomplete(\u0026ent-\u003edone);\n+\tup(\u0026cmd-\u003esem);\n+}\n+\n+static int xsc_cmd_cq_polling(void *data)\n+{\n+\tstruct xsc_core_device *xdev = data;\n+\tstruct xsc_cmd *cmd = \u0026xdev-\u003ecmd;\n+\tstruct xsc_rsp_layout *rsp;\n+\tu32 cq_pid;\n+\n+\twhile (!kthread_should_stop()) {\n+\t\tif (need_resched())\n+\t\t\tschedule();\n+\t\tcq_pid = xsc_get_cmdq_rsp_pid(xdev);\n+\t\tif (cmd-\u003ecq_cid == cq_pid) {\n+\t\t\tmdelay(3);\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\trsp = xsc_get_cq_inst(cmd, cmd-\u003ecq_cid);\n+\t\tif (!cmd-\u003eownerbit_learned) {\n+\t\t\tcmd-\u003eownerbit_learned = 1;\n+\t\t\tcmd-\u003eowner_bit = rsp-\u003eowner_bit;\n+\t\t}\n+\t\tif (cmd-\u003eowner_bit != rsp-\u003eowner_bit) {\n+\t\t\tpci_err(xdev-\u003epdev, \"hw update cq doorbell but buf not ready %u %u\\n\",\n+\t\t\t\tcmd-\u003ecq_cid, cq_pid);\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\txsc_cmd_comp_handler(xdev, rsp-\u003eidx, rsp);\n+\n+\t\tcmd-\u003ecq_cid = (cmd-\u003ecq_cid + 1) % (1 \u003c\u003c cmd-\u003elog_sz);\n+\n+\t\txsc_update_cmdq_rsp_cid(xdev, cmd-\u003ecq_cid);\n+\t\tif (cmd-\u003ecq_cid == 0)\n+\t\t\tcmd-\u003eowner_bit = !cmd-\u003eowner_bit;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+int xsc_cmd_err_handler(struct xsc_core_device *xdev)\n+{\n+\treturn xsc_handle_cmdq_interrupt(xdev);\n+}\n+\n+void xsc_cmd_resp_handler(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_cmd *cmd = \u0026xdev-\u003ecmd;\n+\tstruct xsc_rsp_layout *rsp;\n+\tconst int budget = 32;\n+\tint count = 0;\n+\tu32 cq_pid;\n+\n+\twhile (count \u003c budget) {\n+\t\tcq_pid = xsc_get_cmdq_rsp_pid(xdev);\n+\t\tif (cq_pid == cmd-\u003ecq_cid)\n+\t\t\treturn;\n+\n+\t\trsp = xsc_get_cq_inst(cmd, cmd-\u003ecq_cid);\n+\t\tif (!cmd-\u003eownerbit_learned) {\n+\t\t\tcmd-\u003eownerbit_learned = 1;\n+\t\t\tcmd-\u003eowner_bit = rsp-\u003eowner_bit;\n+\t\t}\n+\t\tif (cmd-\u003eowner_bit != rsp-\u003eowner_bit) {\n+\t\t\tpci_err(xdev-\u003epdev, \"hw update cq doorbell but buf not ready %u %u\\n\",\n+\t\t\t\tcmd-\u003ecq_cid, cq_pid);\n+\t\t\treturn;\n+\t\t}\n+\n+\t\txsc_cmd_comp_handler(xdev, rsp-\u003eidx, rsp);\n+\n+\t\tcmd-\u003ecq_cid = (cmd-\u003ecq_cid + 1) % (1 \u003c\u003c cmd-\u003elog_sz);\n+\t\txsc_update_cmdq_rsp_cid(xdev, cmd-\u003ecq_cid);\n+\t\tif (cmd-\u003ecq_cid == 0)\n+\t\t\tcmd-\u003eowner_bit = !cmd-\u003eowner_bit;\n+\n+\t\tcount++;\n+\t}\n+}\n+\n+static void xsc_cmd_handle_rsp_before_reload\n+(struct xsc_cmd *cmd, struct xsc_core_device *xdev)\n+{\n+\tu32 rsp_pid, rsp_cid;\n+\n+\trsp_pid = xsc_get_cmdq_rsp_pid(xdev);\n+\trsp_cid = xsc_get_cmdq_rsp_cid(xdev);\n+\tif (rsp_pid == rsp_cid)\n+\t\treturn;\n+\n+\tcmd-\u003ecq_cid = rsp_pid;\n+\n+\txsc_update_cmdq_rsp_cid(xdev, cmd-\u003ecq_cid);\n+}\n+\n+int xsc_cmd_init(struct xsc_core_device *xdev)\n+{\n+\tunsigned int size = sizeof(struct xsc_cmd_prot_block);\n+\tint align = roundup_pow_of_two(size);\n+\tstruct xsc_cmd *cmd = \u0026xdev-\u003ecmd;\n+\tu32 cmd_h, cmd_l;\n+\tint err;\n+\n+\tcmd-\u003epool = dma_pool_create(\"xsc_cmd\",\n+\t\t\t\t \u0026xdev-\u003epdev-\u003edev,\n+\t\t\t\t size, align, 0);\n+\tif (!cmd-\u003epool)\n+\t\treturn -ENOMEM;\n+\n+\tcmd-\u003ecmd_buf = (void *)__get_free_pages(GFP_ATOMIC, 0);\n+\tif (!cmd-\u003ecmd_buf) {\n+\t\terr = -ENOMEM;\n+\t\tgoto err_free_pool;\n+\t}\n+\tcmd-\u003ecq_buf = (void *)__get_free_pages(GFP_ATOMIC, 0);\n+\tif (!cmd-\u003ecq_buf) {\n+\t\terr = -ENOMEM;\n+\t\tgoto err_free_cmd;\n+\t}\n+\n+\tmemset(cmd-\u003ecmd_buf, 0, PAGE_SIZE);\n+\tmemset(cmd-\u003ecq_buf, 0, PAGE_SIZE);\n+\n+\tcmd-\u003edma = dma_map_single(\u0026xdev-\u003epdev-\u003edev, cmd-\u003ecmd_buf, PAGE_SIZE,\n+\t\t\t\t DMA_BIDIRECTIONAL);\n+\tif (dma_mapping_error(\u0026xdev-\u003epdev-\u003edev, cmd-\u003edma)) {\n+\t\terr = -ENOMEM;\n+\t\tgoto err_free_cq;\n+\t}\n+\n+\tcmd-\u003ecq_dma = dma_map_single(\u0026xdev-\u003epdev-\u003edev, cmd-\u003ecq_buf, PAGE_SIZE,\n+\t\t\t\t DMA_BIDIRECTIONAL);\n+\tif (dma_mapping_error(\u0026xdev-\u003epdev-\u003edev, cmd-\u003ecq_dma)) {\n+\t\terr = -ENOMEM;\n+\t\tgoto err_unmap_cmd;\n+\t}\n+\n+\tcmd-\u003ecmd_pid = xsc_get_cmdq_req_pid(xdev);\n+\tcmd-\u003ecq_cid = xsc_get_cmdq_rsp_cid(xdev);\n+\tcmd-\u003eownerbit_learned = 0;\n+\n+\txsc_cmd_handle_rsp_before_reload(cmd, xdev);\n+\n+#define ELEMENT_SIZE_LOG\t6 /* 64B */\n+#define Q_DEPTH_LOG\t\t5 /* 32 */\n+\n+\tcmd-\u003elog_sz = Q_DEPTH_LOG;\n+\tcmd-\u003elog_stride = ELEMENT_SIZE_LOG;\n+\n+\tif (1 \u003c\u003c cmd-\u003elog_sz \u003e XSC_MAX_COMMANDS) {\n+\t\tdev_err(\u0026xdev-\u003epdev-\u003edev, \"firmware reports too many outstanding commands %d\\n\",\n+\t\t\t1 \u003c\u003c cmd-\u003elog_sz);\n+\t\terr = -EINVAL;\n+\t\tgoto err_unmap_cq;\n+\t}\n+\n+\tif (cmd-\u003elog_sz + cmd-\u003elog_stride \u003e PAGE_SHIFT) {\n+\t\tdev_err(\u0026xdev-\u003epdev-\u003edev, \"command queue size overflow\\n\");\n+\t\terr = -EINVAL;\n+\t\tgoto err_unmap_cq;\n+\t}\n+\n+\tcmd-\u003echecksum_disabled = 1;\n+\tcmd-\u003emax_reg_cmds = (1 \u003c\u003c cmd-\u003elog_sz) - 1;\n+\tcmd-\u003ecmd_entry_mask = (1 \u003c\u003c cmd-\u003emax_reg_cmds) - 1;\n+\n+\tspin_lock_init(\u0026cmd-\u003ealloc_lock);\n+\tspin_lock_init(\u0026cmd-\u003etoken_lock);\n+\tspin_lock_init(\u0026cmd-\u003edoorbell_lock);\n+\n+\tsema_init(\u0026cmd-\u003esem, cmd-\u003emax_reg_cmds);\n+\n+\tcmd_h = upper_32_bits(cmd-\u003edma);\n+\tcmd_l = lower_32_bits(cmd-\u003edma);\n+\tif (cmd_l \u0026 0xfff) {\n+\t\tdev_err(\u0026xdev-\u003epdev-\u003edev, \"invalid command queue address\\n\");\n+\t\terr = -ENOMEM;\n+\t\tgoto err_unmap_cq;\n+\t}\n+\n+\txsc_set_cmdq_req_buf_addr(xdev, cmd_h, cmd_l);\n+\n+\tcmd_h = upper_32_bits(cmd-\u003ecq_dma);\n+\tcmd_l = lower_32_bits(cmd-\u003ecq_dma);\n+\tif (cmd_l \u0026 0xfff) {\n+\t\tdev_err(\u0026xdev-\u003epdev-\u003edev, \"invalid command queue address\\n\");\n+\t\terr = -ENOMEM;\n+\t\tgoto err_unmap_cq;\n+\t}\n+\txsc_set_cmdq_rsp_buf_addr(xdev, cmd_h, cmd_l);\n+\n+\t/* Make sure firmware sees the complete address before we proceed */\n+\twmb();\n+\n+\tcmd-\u003emode = XSC_CMD_MODE_POLLING;\n+\tcmd-\u003ecmd_status = XSC_CMD_STATUS_NORMAL;\n+\n+\terr = xsc_create_msg_cache(xdev);\n+\tif (err) {\n+\t\tdev_err(\u0026xdev-\u003epdev-\u003edev, \"failed to create command cache\\n\");\n+\t\tgoto err_unmap_cq;\n+\t}\n+\n+\txsc_set_wqname(xdev);\n+\tcmd-\u003ewq = create_singlethread_workqueue(cmd-\u003ewq_name);\n+\tif (!cmd-\u003ewq) {\n+\t\tdev_err(\u0026xdev-\u003epdev-\u003edev, \"failed to create command workqueue\\n\");\n+\t\terr = -ENOMEM;\n+\t\tgoto err_destroy_cache;\n+\t}\n+\n+\tcmd-\u003ecq_task = kthread_create(xsc_cmd_cq_polling,\n+\t\t\t\t (void *)xdev,\n+\t\t\t\t \"xsc_cmd_cq_polling\");\n+\tif (!cmd-\u003ecq_task) {\n+\t\tdev_err(\u0026xdev-\u003epdev-\u003edev, \"failed to create cq task\\n\");\n+\t\terr = -ENOMEM;\n+\t\tgoto err_destroy_wq;\n+\t}\n+\twake_up_process(cmd-\u003ecq_task);\n+\n+\t/* clear abnormal state to avoid the impact of previous error */\n+\txsc_check_cmdq_status(xdev);\n+\n+\treturn 0;\n+\n+err_destroy_wq:\n+\tdestroy_workqueue(cmd-\u003ewq);\n+\n+err_destroy_cache:\n+\txsc_destroy_msg_cache(xdev);\n+\n+err_unmap_cq:\n+\tdma_unmap_single(\u0026xdev-\u003epdev-\u003edev, cmd-\u003ecq_dma, PAGE_SIZE,\n+\t\t\t DMA_BIDIRECTIONAL);\n+\n+err_unmap_cmd:\n+\tdma_unmap_single(\u0026xdev-\u003epdev-\u003edev, cmd-\u003edma, PAGE_SIZE,\n+\t\t\t DMA_BIDIRECTIONAL);\n+err_free_cq:\n+\tfree_pages((unsigned long)cmd-\u003ecq_buf, 0);\n+\n+err_free_cmd:\n+\tfree_pages((unsigned long)cmd-\u003ecmd_buf, 0);\n+\n+err_free_pool:\n+\tdma_pool_destroy(cmd-\u003epool);\n+\n+\treturn err;\n+}\n+\n+void xsc_cmd_cleanup(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_cmd *cmd = \u0026xdev-\u003ecmd;\n+\n+\tdestroy_workqueue(cmd-\u003ewq);\n+\tif (cmd-\u003ecq_task)\n+\t\tkthread_stop(cmd-\u003ecq_task);\n+\txsc_destroy_msg_cache(xdev);\n+\tdma_unmap_single(\u0026xdev-\u003epdev-\u003edev, cmd-\u003edma, PAGE_SIZE,\n+\t\t\t DMA_BIDIRECTIONAL);\n+\tfree_pages((unsigned long)cmd-\u003ecq_buf, 0);\n+\tdma_unmap_single(\u0026xdev-\u003epdev-\u003edev, cmd-\u003ecq_dma, PAGE_SIZE,\n+\t\t\t DMA_BIDIRECTIONAL);\n+\tfree_pages((unsigned long)cmd-\u003ecmd_buf, 0);\n+\tdma_pool_destroy(cmd-\u003epool);\n+}\n+\n+int xsc_cmd_version_check(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_cmd_query_cmdq_ver_mbox_out *out;\n+\tstruct xsc_cmd_query_cmdq_ver_mbox_in in;\n+\n+\tint err;\n+\n+\tout = kzalloc_obj(*out, GFP_KERNEL);\n+\tif (!out)\n+\t\treturn -ENOMEM;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_CMDQ_VERSION);\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), out, sizeof(*out));\n+\tif (err)\n+\t\tgoto err_free;\n+\n+\tif (be16_to_cpu(out-\u003ecmdq_ver) != XSC_CMDQ_VERSION) {\n+\t\tpci_err(xdev-\u003epdev, \"cmdq version check failed, expecting %d, actual %d\\n\",\n+\t\t\tXSC_CMDQ_VERSION, be16_to_cpu(out-\u003ecmdq_ver));\n+\t\terr = -EINVAL;\n+\t\tgoto err_free;\n+\t}\n+\txdev-\u003ecmdq_ver = XSC_CMDQ_VERSION;\n+\n+err_free:\n+\tkfree(out);\n+\treturn err;\n+}\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c\nnew file mode 100644\nindex 0000000000000..39715682fecc3\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c\n@@ -0,0 +1,316 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_cmd_api.h\"\n+#include \"cq.h\"\n+\n+void xsc_init_cq_table(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_cq_table *table = \u0026xdev-\u003edev_res-\u003ecq_table;\n+\n+\tspin_lock_init(\u0026table-\u003elock);\n+\tINIT_RADIX_TREE(\u0026table-\u003etree, GFP_ATOMIC);\n+}\n+\n+static int xsc_create_cq(struct xsc_core_device *xdev, u32 *p_cqn,\n+\t\t\t struct xsc_create_cq_mbox_in *in, int insize)\n+{\n+\tstruct xsc_create_cq_mbox_out out;\n+\tint ret;\n+\n+\tmemset(\u0026out, 0, sizeof(out));\n+\tin-\u003ehdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_CQ);\n+\tret = xsc_cmd_exec(xdev, in, insize, \u0026out, sizeof(out));\n+\tif (ret) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to create cq, err=%d out.status=%u\\n\",\n+\t\t\tret, out.hdr.status);\n+\t\treturn ret;\n+\t}\n+\n+\t*p_cqn = be32_to_cpu(out.cqn) \u0026 0xffffff;\n+\treturn 0;\n+}\n+\n+static int xsc_destroy_cq(struct xsc_core_device *xdev, u32 cqn)\n+{\n+\tstruct xsc_destroy_cq_mbox_out out;\n+\tstruct xsc_destroy_cq_mbox_in in;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(\u0026out, 0, sizeof(out));\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_CQ);\n+\tin.cqn = cpu_to_be32(cqn);\n+\n+\treturn xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+}\n+\n+int xsc_core_eth_create_cq(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_core_cq *xcq,\n+\t\t\t struct xsc_create_cq_mbox_in *in,\n+\t\t\t int insize)\n+{\n+\tstruct xsc_cq_table *table = \u0026xdev-\u003edev_res-\u003ecq_table;\n+\tu32 cqn;\n+\tint ret;\n+\tint err;\n+\n+\tret = xsc_create_cq(xdev, \u0026cqn, in, insize);\n+\tif (ret) {\n+\t\tpci_err(xdev-\u003epdev, \"xsc_create_cq failed\\n\");\n+\t\treturn -ENOEXEC;\n+\t}\n+\txcq-\u003ecqn = cqn;\n+\txcq-\u003econs_index = 0;\n+\txcq-\u003earm_sn = 0;\n+\tatomic_set(\u0026xcq-\u003erefcount, 1);\n+\tinit_completion(\u0026xcq-\u003efree);\n+\n+\tspin_lock_irq(\u0026table-\u003elock);\n+\tret = radix_tree_insert(\u0026table-\u003etree, xcq-\u003ecqn, xcq);\n+\tspin_unlock_irq(\u0026table-\u003elock);\n+\tif (ret)\n+\t\tgoto err_insert_cq;\n+\treturn 0;\n+err_insert_cq:\n+\terr = xsc_destroy_cq(xdev, cqn);\n+\tif (err)\n+\t\tpci_err(xdev-\u003epdev, \"failed to destroy cqn=%d, err=%d\\n\",\n+\t\t\txcq-\u003ecqn, err);\n+\treturn ret;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_eth_create_cq);\n+\n+int xsc_core_eth_destroy_cq(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_core_cq *xcq)\n+{\n+\tstruct xsc_cq_table *table = \u0026xdev-\u003edev_res-\u003ecq_table;\n+\tstruct xsc_core_cq *tmp;\n+\tint err;\n+\n+\tspin_lock_irq(\u0026table-\u003elock);\n+\ttmp = radix_tree_delete(\u0026table-\u003etree, xcq-\u003ecqn);\n+\tspin_unlock_irq(\u0026table-\u003elock);\n+\tif (!tmp) {\n+\t\terr = -ENOENT;\n+\t\tgoto err_delete_cq;\n+\t}\n+\n+\tif (tmp != xcq) {\n+\t\terr = -EINVAL;\n+\t\tgoto err_delete_cq;\n+\t}\n+\n+\terr = xsc_destroy_cq(xdev, xcq-\u003ecqn);\n+\tif (err)\n+\t\tgoto err_destroy_cq;\n+\n+\tif (atomic_dec_and_test(\u0026xcq-\u003erefcount))\n+\t\tcomplete(\u0026xcq-\u003efree);\n+\twait_for_completion(\u0026xcq-\u003efree);\n+\treturn 0;\n+\n+err_destroy_cq:\n+\tpci_err(xdev-\u003epdev, \"failed to destroy cqn=%d, err=%d\\n\",\n+\t\txcq-\u003ecqn, err);\n+\treturn err;\n+err_delete_cq:\n+\tpci_err(xdev-\u003epdev, \"cqn=%d not found in tree, err=%d\\n\",\n+\t\txcq-\u003ecqn, err);\n+\treturn err;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_eth_destroy_cq);\n+\n+static int xsc_set_cq_context(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_cq_context_ex *ctx_ex,\n+\t\t\t u32 *pa_list_base, u32 *cqn)\n+{\n+\tstruct xsc_set_cq_context_mbox_in in;\n+\tstruct xsc_set_cq_context_mbox_out out;\n+\tint err;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemcpy(\u0026in.ctx_ex, ctx_ex, sizeof(*ctx_ex));\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_CQ_CONTEXT);\n+\tin.hdr.ver = cpu_to_be16(CMD_SET_CQ_CONTEXT_V1);\n+\n+\tmemset(\u0026out, 0, sizeof(out));\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err || out.hdr.status) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to set cq context\\n\");\n+\t\treturn -EIO;\n+\t}\n+\n+\t*pa_list_base = be32_to_cpu(out.cq_pa_list_base);\n+\t*cqn = be32_to_cpu(out.cqn);\n+\treturn 0;\n+}\n+\n+static int xsc_set_cq_buf_pa(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_create_cq_ex_mbox_in *req,\n+\t\t\t u32 pa_list_base, u32 cqn)\n+{\n+\tstruct xsc_set_cq_buf_pa_mbox_in *in;\n+\tstruct xsc_set_cq_buf_pa_mbox_out out;\n+\tu32 pa_num_total = be32_to_cpu(req-\u003ectx_ex.pa_num_ex);\n+\tu32 pa_num_max = (xdev-\u003ecaps.max_cmd_in_len - sizeof(*in)) / sizeof(__be64);\n+\tu32 pa_num_left = pa_num_total;\n+\tu32 pa_num, copy_len;\n+\tint in_len, err;\n+\n+\twhile (pa_num_left) {\n+\t\tpa_num = min(pa_num_max, pa_num_left);\n+\t\tcopy_len = pa_num * sizeof(__be64);\n+\t\tin_len = sizeof(*in) + copy_len;\n+\n+\t\tin = kvzalloc(in_len, GFP_KERNEL);\n+\t\tif (!in)\n+\t\t\treturn -ENOMEM;\n+\n+\t\tin-\u003ehdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_CQ_BUF_PA);\n+\t\tin-\u003epa_list_start = cpu_to_be32(pa_list_base);\n+\t\tin-\u003epa_num = cpu_to_be32(pa_num);\n+\t\tmemcpy(in-\u003epas, \u0026req-\u003epas[pa_num_total - pa_num_left], copy_len);\n+\t\tpa_num_left -= pa_num;\n+\t\tpa_list_base += pa_num;\n+\t\tmemset(\u0026out, 0, sizeof(out));\n+\n+\t\terr = xsc_cmd_exec(xdev, in, in_len, \u0026out, sizeof(out));\n+\t\tkvfree(in);\n+\n+\t\tif (err || out.hdr.status) {\n+\t\t\tpci_err(xdev-\u003epdev, \"failed to set cq buf pa, cqn=%d\\n\", cqn);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int xsc_create_cq_compat(struct xsc_core_device *xdev,\n+\t\t\t\tstruct xsc_create_cq_ex_mbox_in *in,\n+\t\t\t\tstruct xsc_create_cq_mbox_out *out)\n+{\n+\tstruct xsc_create_cq_mbox_in *_in;\n+\tint _inlen = sizeof(*_in) +\n+\t\tbe16_to_cpu(in-\u003ectx_ex.ctx.pa_num) * sizeof(__be64);\n+\tint err;\n+\n+\t_in = kvzalloc(_inlen, GFP_KERNEL);\n+\tif (!_in)\n+\t\treturn -ENOMEM;\n+\n+\t_in-\u003ehdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_CQ);\n+\tmemcpy(\u0026_in-\u003ectx, \u0026in-\u003ectx_ex.ctx, sizeof(_in-\u003ectx));\n+\tmemcpy(\u0026_in-\u003epas, \u0026in-\u003epas, _inlen - sizeof(*_in));\n+\tmemset(out, 0, sizeof(*out));\n+\n+\terr = xsc_cmd_exec(xdev, _in, _inlen, out, sizeof(*out));\n+\tkvfree(_in);\n+\n+\tif (err)\n+\t\treturn err;\n+\tif (out-\u003ehdr.status)\n+\t\treturn xsc_cmd_status_to_err(\u0026out-\u003ehdr);\n+\n+\treturn 0;\n+}\n+\n+int xsc_core_create_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq,\n+\t\t struct xsc_create_cq_ex_mbox_in *in, int inlen)\n+{\n+\tstruct xsc_cq_table *table = \u0026xdev-\u003edev_res-\u003ecq_table;\n+\tstruct xsc_create_cq_mbox_out out;\n+\tu32 pa_list_base = 0;\n+\tu32 cqn = 0;\n+\tint err;\n+\n+\tif (inlen \u003c xdev-\u003ecaps.max_cmd_in_len) {\n+\t\tin-\u003ehdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_CQ_EX);\n+\t\tmemset(\u0026out, 0, sizeof(out));\n+\t\terr = xsc_cmd_exec(xdev, in, inlen, \u0026out, sizeof(out));\n+\t\tif (err)\n+\t\t\treturn err;\n+\n+\t\tif (out.hdr.status \u0026\u0026 out.hdr.status != XSC_CMD_STATUS_NOT_SUPPORTED)\n+\t\t\treturn xsc_cmd_status_to_err(\u0026out.hdr);\n+\n+\t\tif (out.hdr.status == XSC_CMD_STATUS_NOT_SUPPORTED) {\n+\t\t\terr = xsc_create_cq_compat(xdev, in, \u0026out);\n+\t\t\tif (err)\n+\t\t\t\treturn err;\n+\t\t}\n+\n+\t\tcqn = be32_to_cpu(out.cqn);\n+\t} else {\n+\t\terr = xsc_set_cq_context(xdev, \u0026in-\u003ectx_ex, \u0026pa_list_base, \u0026cqn);\n+\t\tif (err)\n+\t\t\treturn err;\n+\n+\t\terr = xsc_set_cq_buf_pa(xdev, in, pa_list_base, cqn);\n+\t\tif (err)\n+\t\t\tgoto err_cmd;\n+\t}\n+\n+\tcq-\u003ecqn = cqn;\n+\tcq-\u003econs_index = 0;\n+\tcq-\u003earm_sn = 0;\n+\tcq-\u003exdev = xdev;\n+\tcq-\u003edev = xdev;\n+\tatomic_set(\u0026cq-\u003erefcount, 1);\n+\tinit_completion(\u0026cq-\u003efree);\n+\n+\txsc_arm_cq(xdev, cq-\u003ecqn, 0, 0);\n+\n+\tspin_lock_irq(\u0026table-\u003elock);\n+\terr = radix_tree_insert(\u0026table-\u003etree, cq-\u003ecqn, cq);\n+\tspin_unlock_irq(\u0026table-\u003elock);\n+\tif (err)\n+\t\tgoto err_cmd;\n+\n+\tcq-\u003epid = current-\u003epid;\n+\treturn 0;\n+\n+err_cmd:\n+\txsc_destroy_cq(xdev, cqn);\n+\treturn err;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_create_cq);\n+\n+int xsc_core_destroy_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq)\n+{\n+\tstruct xsc_cq_table *table = \u0026xdev-\u003edev_res-\u003ecq_table;\n+\tstruct xsc_destroy_cq_mbox_in in;\n+\tstruct xsc_destroy_cq_mbox_out out;\n+\tstruct xsc_core_cq *tmp;\n+\tint err;\n+\n+\tspin_lock_irq(\u0026table-\u003elock);\n+\ttmp = radix_tree_delete(\u0026table-\u003etree, cq-\u003ecqn);\n+\tspin_unlock_irq(\u0026table-\u003elock);\n+\tif (!tmp || tmp != cq) {\n+\t\tpci_warn(xdev-\u003epdev, \"cq 0x%x not found in tree\\n\", cq-\u003ecqn);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(\u0026out, 0, sizeof(out));\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_CQ);\n+\tin.cqn = cpu_to_be32(cq-\u003ecqn);\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err)\n+\t\treturn err;\n+\n+\tif (out.hdr.status)\n+\t\treturn xsc_cmd_status_to_err(\u0026out.hdr);\n+\n+\tif (atomic_dec_and_test(\u0026cq-\u003erefcount))\n+\t\tcomplete(\u0026cq-\u003efree);\n+\twait_for_completion(\u0026cq-\u003efree);\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_destroy_cq);\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/cq.h b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.h\nnew file mode 100644\nindex 0000000000000..afe6cbb0daa2b\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.h\n@@ -0,0 +1,17 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_CQ_H\n+#define __XSC_CQ_H\n+\n+#include \"common/xsc_core.h\"\n+\n+void xsc_init_cq_table(struct xsc_core_device *xdev);\n+\n+int xsc_core_create_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq,\n+\t\t struct xsc_create_cq_ex_mbox_in *in, int inlen);\n+int xsc_core_destroy_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq);\n+\n+#endif /* __XSC_CQ_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/eq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/eq.c\nnew file mode 100644\nindex 0000000000000..ff4bcb42b33ff\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/eq.c\n@@ -0,0 +1,320 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+#include \u003clinux/interrupt.h\u003e\n+#include \u003clinux/module.h\u003e\n+#include \u003clinux/mm.h\u003e\n+\n+#include \"common/xsc_cmd_api.h\"\n+#include \"common/xsc_core.h\"\n+#include \"qp.h\"\n+#include \"alloc.h\"\n+#include \"eq.h\"\n+#include \"hal/xsc_hal.h\"\n+\n+enum {\n+\tXSC_EQE_SIZE\t\t= sizeof(struct xsc_eqe),\n+};\n+\n+enum {\n+\tXSC_NUM_SPARE_EQE\t= 0x80,\n+\tXSC_NUM_ASYNC_EQE\t= 0x100,\n+};\n+\n+static int xsc_cmd_destroy_eq(struct xsc_core_device *xdev, u32 eqn)\n+{\n+\tstruct xsc_destroy_eq_mbox_out out = {0};\n+\tstruct xsc_destroy_eq_mbox_in in = {0};\n+\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_EQ);\n+\tin.eqn = cpu_to_be32(eqn);\n+\n+\treturn xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+}\n+\n+static struct xsc_eqe *get_eqe(struct xsc_eq *eq, u32 entry)\n+{\n+\treturn xsc_buf_offset(\u0026eq-\u003ebuf, entry * XSC_EQE_SIZE);\n+}\n+\n+static struct xsc_eqe *next_eqe_sw(struct xsc_eq *eq)\n+{\n+\tstruct xsc_eqe *eqe = get_eqe(eq, eq-\u003econs_index \u0026 (eq-\u003enent - 1));\n+\n+\treturn ((eqe-\u003eowner_data \u0026 XSC_EQE_OWNER) ^\n+\t\t!!(eq-\u003econs_index \u0026 eq-\u003enent)) ? NULL : eqe;\n+}\n+\n+static void eq_update_ci(struct xsc_eq *eq, int arm)\n+{\n+\txsc_update_eq_ci(eq-\u003edev, eq-\u003eeqn, eq-\u003econs_index, arm);\n+}\n+\n+static void xsc_cq_completion(struct xsc_core_device *xdev, u32 cqn)\n+{\n+\tstruct xsc_cq_table *table = \u0026xdev-\u003edev_res-\u003ecq_table;\n+\tstruct xsc_core_cq *cq;\n+\n+\trcu_read_lock();\n+\tcq = radix_tree_lookup(\u0026table-\u003etree, cqn);\n+\tif (likely(cq))\n+\t\tatomic_inc(\u0026cq-\u003erefcount);\n+\trcu_read_unlock();\n+\n+\tif (!cq) {\n+\t\tpci_err(xdev-\u003epdev, \"Completion event for bogus CQ, cqn=%d\\n\",\n+\t\t\tcqn);\n+\t\treturn;\n+\t}\n+\n+\t++cq-\u003earm_sn;\n+\n+\tif (!cq-\u003ecomp)\n+\t\tpci_err(xdev-\u003epdev, \"cq-\u003ecomp is NULL\\n\");\n+\telse\n+\t\tcq-\u003ecomp(cq);\n+\n+\tif (atomic_dec_and_test(\u0026cq-\u003erefcount))\n+\t\tcomplete(\u0026cq-\u003efree);\n+}\n+\n+static void xsc_eq_cq_event(struct xsc_core_device *xdev,\n+\t\t\t u32 cqn, int event_type)\n+{\n+\tstruct xsc_cq_table *table = \u0026xdev-\u003edev_res-\u003ecq_table;\n+\tstruct xsc_core_cq *cq;\n+\n+\tspin_lock(\u0026table-\u003elock);\n+\tcq = radix_tree_lookup(\u0026table-\u003etree, cqn);\n+\tif (likely(cq))\n+\t\tatomic_inc(\u0026cq-\u003erefcount);\n+\tspin_unlock(\u0026table-\u003elock);\n+\n+\tif (unlikely(!cq)) {\n+\t\tpci_err(xdev-\u003epdev, \"Async event for bogus CQ, cqn=%d\\n\",\n+\t\t\tcqn);\n+\t\treturn;\n+\t}\n+\n+\tcq-\u003eevent(cq, event_type);\n+\n+\tif (atomic_dec_and_test(\u0026cq-\u003erefcount))\n+\t\tcomplete(\u0026cq-\u003efree);\n+}\n+\n+static int xsc_eq_int(struct xsc_core_device *xdev, struct xsc_eq *eq)\n+{\n+\tu32 cqn, qpn, queue_id;\n+\tstruct xsc_eqe *eqe;\n+\tint eqes_found = 0;\n+\tint set_ci = 0;\n+\n+\twhile ((eqe = next_eqe_sw(eq))) {\n+\t\t/* Make sure we read EQ entry contents after we've\n+\t\t * checked the ownership bit.\n+\t\t */\n+\t\trmb();\n+\t\tswitch (eqe-\u003etype) {\n+\t\tcase XSC_EVENT_TYPE_COMP:\n+\t\tcase XSC_EVENT_TYPE_INTERNAL_ERROR:\n+\t\t\t/* eqe is changing */\n+\t\t\tqueue_id = FIELD_GET(XSC_EQE_QUEUE_ID_MASK,\n+\t\t\t\t\t le16_to_cpu(eqe-\u003equeue_id_data));\n+\t\t\tcqn = queue_id;\n+\t\t\txsc_cq_completion(xdev, cqn);\n+\t\t\tbreak;\n+\n+\t\tcase XSC_EVENT_TYPE_CQ_ERROR:\n+\t\t\tqueue_id = FIELD_GET(XSC_EQE_QUEUE_ID_MASK,\n+\t\t\t\t\t le16_to_cpu(eqe-\u003equeue_id_data));\n+\t\t\tcqn = queue_id;\n+\t\t\txsc_eq_cq_event(xdev, cqn, eqe-\u003etype);\n+\t\t\tbreak;\n+\t\tcase XSC_EVENT_TYPE_WQ_CATAS_ERROR:\n+\t\tcase XSC_EVENT_TYPE_WQ_INVAL_REQ_ERROR:\n+\t\tcase XSC_EVENT_TYPE_WQ_ACCESS_ERROR:\n+\t\t\tqueue_id = FIELD_GET(XSC_EQE_QUEUE_ID_MASK,\n+\t\t\t\t\t le16_to_cpu(eqe-\u003equeue_id_data));\n+\t\t\tqpn = queue_id;\n+\t\t\txsc_qp_event(xdev, qpn, eqe-\u003etype);\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\t++eq-\u003econs_index;\n+\t\teqes_found = 1;\n+\t\t++set_ci;\n+\n+\t\t/* The HCA will think the queue has overflowed if we\n+\t\t * don't tell it we've been processing events. We\n+\t\t * create our EQs with XSC_NUM_SPARE_EQE extra\n+\t\t * entries, so we must update our consumer index at\n+\t\t * least that often.\n+\t\t */\n+\t\tif (unlikely(set_ci \u003e= XSC_NUM_SPARE_EQE)) {\n+\t\t\teq_update_ci(eq, 0);\n+\t\t\tset_ci = 0;\n+\t\t}\n+\t}\n+\n+\teq_update_ci(eq, 1);\n+\n+\treturn eqes_found;\n+}\n+\n+static irqreturn_t xsc_msix_handler(int irq, void *eq_ptr)\n+{\n+\tstruct xsc_core_device *xdev;\n+\tstruct xsc_eq *eq = eq_ptr;\n+\n+\txdev = eq-\u003edev;\n+\txsc_eq_int(xdev, eq);\n+\n+\t/* MSI-X vectors always belong to us */\n+\treturn IRQ_HANDLED;\n+}\n+\n+static void init_eq_buf(struct xsc_eq *eq)\n+{\n+\tstruct xsc_eqe *eqe;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c eq-\u003enent; i++) {\n+\t\teqe = get_eqe(eq, i);\n+\t\teqe-\u003eowner_data |= XSC_EQE_OWNER;\n+\t}\n+}\n+\n+int xsc_create_map_eq(struct xsc_core_device *xdev,\n+\t\t struct xsc_eq *eq, u16 vecidx,\n+\t\t u32 nent, const char *name)\n+{\n+\tu16 msix_vec_offset = xdev-\u003emsix_vec_base + vecidx;\n+\tstruct xsc_dev_resource *dev_res = xdev-\u003edev_res;\n+\tstruct xsc_create_eq_mbox_out out;\n+\tstruct xsc_create_eq_mbox_in *in;\n+\tunsigned int hw_npages;\n+\tu32 inlen;\n+\tint err;\n+\n+\teq-\u003enent = roundup_pow_of_two(roundup(nent, XSC_NUM_SPARE_EQE));\n+\terr = xsc_buf_alloc(xdev, eq-\u003enent * XSC_EQE_SIZE, PAGE_SIZE, \u0026eq-\u003ebuf);\n+\tif (err)\n+\t\treturn err;\n+\n+\tinit_eq_buf(eq);\n+\n+\thw_npages = DIV_ROUND_UP(eq-\u003enent * XSC_EQE_SIZE, PAGE_SIZE_4K);\n+\tinlen = sizeof(*in) + sizeof(in-\u003epas[0]) * hw_npages;\n+\tin = kvzalloc(inlen, GFP_KERNEL);\n+\tif (!in) {\n+\t\terr = -ENOMEM;\n+\t\tgoto err_free_eq_buf;\n+\t}\n+\tmemset(\u0026out, 0, sizeof(out));\n+\n+\txsc_fill_page_array(\u0026eq-\u003ebuf, in-\u003epas, hw_npages);\n+\n+\tin-\u003ehdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_EQ);\n+\tin-\u003ectx.log_eq_sz = ilog2(eq-\u003enent);\n+\tin-\u003ectx.vecidx = cpu_to_be16(msix_vec_offset);\n+\tin-\u003ectx.pa_num = cpu_to_be16(hw_npages);\n+\tin-\u003ectx.glb_func_id = cpu_to_be16(xdev-\u003eglb_func_id);\n+\tin-\u003ectx.is_async_eq = (vecidx == XSC_EQ_VEC_ASYNC ? 1 : 0);\n+\n+\terr = xsc_cmd_exec(xdev, in, inlen, \u0026out, sizeof(out));\n+\tif (err)\n+\t\tgoto err_free_in;\n+\n+\tsnprintf(dev_res-\u003eirq_info[vecidx].name, XSC_MAX_IRQ_NAME, \"%s@pci:%s\",\n+\t\t name, pci_name(xdev-\u003epdev));\n+\n+\teq-\u003eeqn = be32_to_cpu(out.eqn);\n+\teq-\u003eirqn = pci_irq_vector(xdev-\u003epdev, vecidx);\n+\teq-\u003edev = xdev;\n+\teq-\u003eindex = vecidx;\n+\n+\terr = request_irq(eq-\u003eirqn, xsc_msix_handler, 0,\n+\t\t\t dev_res-\u003eirq_info[vecidx].name, eq);\n+\tif (err)\n+\t\tgoto err_destroy_eq;\n+\n+\t/* EQs are created in ARMED state\n+\t */\n+\teq_update_ci(eq, 1);\n+\tkvfree(in);\n+\treturn 0;\n+\n+err_destroy_eq:\n+\txsc_cmd_destroy_eq(xdev, eq-\u003eeqn);\n+\n+err_free_in:\n+\tkvfree(in);\n+\n+err_free_eq_buf:\n+\txsc_buf_free(xdev, \u0026eq-\u003ebuf);\n+\treturn err;\n+}\n+\n+int xsc_destroy_unmap_eq(struct xsc_core_device *xdev, struct xsc_eq *eq)\n+{\n+\tint err;\n+\n+\tif (!xsc_fw_is_available(xdev))\n+\t\treturn 0;\n+\n+\tfree_irq(eq-\u003eirqn, eq);\n+\terr = xsc_cmd_destroy_eq(xdev, eq-\u003eeqn);\n+\tif (err)\n+\t\tpci_err(xdev-\u003epdev, \"failed to destroy a previously created eq: eqn %d\\n\",\n+\t\t\teq-\u003eeqn);\n+\txsc_buf_free(xdev, \u0026eq-\u003ebuf);\n+\n+\treturn err;\n+}\n+\n+void xsc_eq_init(struct xsc_core_device *xdev)\n+{\n+\tspin_lock_init(\u0026xdev-\u003edev_res-\u003eeq_table.lock);\n+}\n+\n+int xsc_start_eqs(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_eq_table *table = \u0026xdev-\u003edev_res-\u003eeq_table;\n+\tint err;\n+\n+\terr = xsc_create_map_eq(xdev, \u0026table-\u003easync_eq, XSC_EQ_VEC_ASYNC,\n+\t\t\t\tXSC_NUM_ASYNC_EQE, \"xsc_async_eq\");\n+\tif (err)\n+\t\tpci_err(xdev-\u003epdev, \"failed to create async EQ %d\\n\", err);\n+\n+\treturn err;\n+}\n+\n+void xsc_stop_eqs(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_eq_table *table = \u0026xdev-\u003edev_res-\u003eeq_table;\n+\n+\txsc_destroy_unmap_eq(xdev, \u0026table-\u003easync_eq);\n+}\n+\n+struct xsc_eq *xsc_core_eq_get(struct xsc_core_device *xdev, int i)\n+{\n+\tstruct xsc_eq_table *table = \u0026xdev-\u003edev_res-\u003eeq_table;\n+\tstruct xsc_eq *eq_ret = NULL;\n+\tstruct xsc_eq *eq, *n;\n+\n+\tspin_lock(\u0026table-\u003elock);\n+\tlist_for_each_entry_safe(eq, n, \u0026table-\u003ecomp_eqs_list, list) {\n+\t\tif (eq-\u003eindex == i) {\n+\t\t\teq_ret = eq;\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\tspin_unlock(\u0026table-\u003elock);\n+\n+\treturn eq_ret;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_eq_get);\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/eq.h b/drivers/net/ethernet/yunsilicon/xsc/pci/eq.h\nnew file mode 100644\nindex 0000000000000..3e7869e12a31c\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/eq.h\n@@ -0,0 +1,39 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __EQ_H\n+#define __EQ_H\n+\n+#include \"common/xsc_core.h\"\n+\n+enum {\n+\tXSC_EQ_VEC_ASYNC\t\t= 0,\n+\tXSC_VEC_CMD\t\t\t= 1,\n+\tXSC_VEC_CMD_EVENT\t\t= 2,\n+\tXSC_DMA_READ_DONE_VEC\t\t= 3,\n+\tXSC_EQ_VEC_COMP_BASE,\n+};\n+\n+struct xsc_eqe {\n+\tu8 type;\n+\tu8 sub_type;\n+\t__le16 queue_id_data;\n+#define XSC_EQE_QUEUE_ID_MASK\tGENMASK(14, 0)\n+\n+\tu8 err_code;\n+\tu8 rsvd[2];\n+\tu8 owner_data;\n+#define XSC_EQE_OWNER\t\tBIT(7)\n+};\n+\n+int xsc_create_map_eq(struct xsc_core_device *xdev,\n+\t\t struct xsc_eq *eq, u16 vecidx,\n+\t\t u32 nent, const char *name);\n+int xsc_destroy_unmap_eq(struct xsc_core_device *xdev, struct xsc_eq *eq);\n+void xsc_eq_init(struct xsc_core_device *xdev);\n+int xsc_start_eqs(struct xsc_core_device *xdev);\n+void xsc_stop_eqs(struct xsc_core_device *xdev);\n+\n+#endif\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_impl.c b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_impl.c\nnew file mode 100644\nindex 0000000000000..46fc045f92c02\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_impl.c\n@@ -0,0 +1,708 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2024 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/kernel.h\u003e\n+#include \u003clinux/pci.h\u003e\n+#include \"andes_reg.h\"\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_cmd.h\"\n+#include \"common/xsc_hsi.h\"\n+#include \"common/xsc_reg.h\"\n+#include \"xsc_hal.h\"\n+\n+/* message opcode */\n+enum {\n+\tXSC_MSG_OPCODE_SEND\t\t= 0,\n+\tXSC_MSG_OPCODE_RDMA_WRITE\t= 1,\n+\tXSC_MSG_OPCODE_RDMA_READ\t= 2,\n+\tXSC_MSG_OPCODE_MAD\t\t= 3,\n+\tXSC_MSG_OPCODE_RDMA_ACK\t\t= 4,\n+\tXSC_MSG_OPCODE_RDMA_ACK_READ\t= 5,\n+\tXSC_MSG_OPCODE_RDMA_CNP\t\t= 6,\n+\tXSC_MSG_OPCODE_RAW\t\t= 7,\n+\tXSC_MSG_OPCODE_VIRTIO_NET\t= 8,\n+\tXSC_MSG_OPCODE_VIRTIO_BLK\t= 9,\n+\tXSC_MSG_OPCODE_RAW_TPE\t\t= 10,\n+\tXSC_MSG_OPCODE_INIT_QP_REQ\t= 11,\n+\tXSC_MSG_OPCODE_INIT_QP_RSP\t= 12,\n+\tXSC_MSG_OPCODE_INIT_PATH_REQ\t= 13,\n+\tXSC_MSG_OPCODE_INIT_PATH_RSP\t= 14,\n+\n+\tXSC_MSG_OPCODE_UNSUPPORT\t= -1,\n+};\n+\n+#define REG_ADDR(bp, offset)\t((bp) + (offset))\n+\n+static void xsc_read32(void __iomem *bar, u32 off, void *data, int len)\n+{\n+\tu32 val = readl(REG_ADDR(bar, off));\n+\n+\tmemcpy(data, \u0026val, len);\n+}\n+\n+static void xsc_write32(void __iomem *bar, u32 off, void *data)\n+{\n+\twritel(*(u32 *)data, REG_ADDR(bar, off));\n+}\n+\n+static void andes_ring_tx_doorbell(struct xsc_core_device *xdev,\n+\t\t\t\t struct xsc_hw_reg *regs,\n+\t\t\t\t void __iomem *bar, u32 sqn, u32 next_pid,\n+\t\t\t\t bool mdb)\n+{\n+\tunion xsc_send_doorbell {\n+\t\tstruct{\n+\t\t\tu32 next_pid : 16;\n+\t\t\tu32 qp_id : 15;\n+\t\t};\n+\t\tu32 raw;\n+\t} db;\n+\tu32 db_reg = mdb ? QPM_TX_MDB_BASE_REG_ADDR + (sqn \u0026 (MDB_NUM - 1)) * sizeof(u32) :\n+\t\tregs-\u003etx_db;\n+\n+\tdb.next_pid = next_pid;\n+\tdb.qp_id = sqn;\n+\n+\t/* Make sure that descriptors are written before\n+\t * updating doorbell record and ringing the doorbell\n+\t */\n+\twmb();\n+\n+\twritel(db.raw, REG_ADDR(bar, db_reg));\n+}\n+\n+static void andes_ring_rx_doorbell(struct xsc_hw_reg *regs,\n+\t\t\t\t void __iomem *bar, u32 rqn, u32 next_pid)\n+{\n+\tunion xsc_recv_doorbell {\n+\t\tstruct{\n+\t\t\tu32 next_pid : 13;\n+\t\t\tu32 qp_id : 15;\n+\t\t};\n+\t\tu32 raw;\n+\t} db;\n+\n+\tdb.next_pid = next_pid;\n+\tdb.qp_id = rqn;\n+\n+\t/* Make sure that descriptors are written before\n+\t * updating doorbell record and ringing the doorbell\n+\t */\n+\twmb();\n+\n+\twritel(db.raw, REG_ADDR(bar, regs-\u003erx_db));\n+}\n+\n+static void andes_update_cq_db(struct xsc_hw_reg *regs,\n+\t\t\t void __iomem *bar, u32 cqn, u32 next_cid,\n+\t\t\t u8 solicited)\n+{\n+\tunion andes_cq_doorbell {\n+\t\tstruct{\n+\t\t\tu32 cq_next_cid:16;\n+\t\t\tu32 cq_id:15;\n+\t\t\tu32 arm:1;\n+\t\t};\n+\t\tu32 val;\n+\t} db;\n+\n+\tdb.cq_next_cid = next_cid;\n+\tdb.cq_id = cqn;\n+\tdb.arm = solicited;\n+\n+\t/* Make sure that the doorbell record in host memory is\n+\t * written before ringing the doorbell via PCI MMIO.\n+\t */\n+\twmb();\n+\twritel(db.val, REG_ADDR(bar, regs-\u003ecomplete_db));\n+}\n+\n+static void andes_set_cq_ci(struct xsc_hw_reg *regs,\n+\t\t\t void __iomem *bar, u32 cqn, u32 next_cid)\n+{\n+\tunion andes_cq_doorbell {\n+\t\tstruct{\n+\t\t\tu32 cq_next_cid:16;\n+\t\t\tu32 cq_id:15;\n+\t\t\tu32 arm:1;\n+\t\t};\n+\t\tu32 val;\n+\t} db;\n+\n+\tdb.val = 0;\n+\tdb.cq_next_cid = next_cid;\n+\tdb.cq_id = cqn;\n+\t/* make sure val write to memory done */\n+\twmb();\n+\twritel(db.val, REG_ADDR(bar, regs-\u003ecomplete_reg));\n+}\n+\n+static void andes_set_eq_ci(struct xsc_hw_reg *regs,\n+\t\t\t void __iomem *bar, u32 eqn, u32 next_cid,\n+\t\t\t u8 arm)\n+{\n+\tunion andes_eq_doorbell {\n+\t\tstruct{\n+\t\t\tu32 eq_next_cid : 11;\n+\t\t\tu32 eq_id : 11;\n+\t\t\tu32 arm : 1;\n+\t\t};\n+\t\tu32 val;\n+\t} db;\n+\n+\tdb.eq_next_cid = next_cid;\n+\tdb.eq_id = eqn;\n+\tdb.arm = !!arm;\n+\n+\t/* make sure val write to memory done */\n+\twmb();\n+\twritel(db.val, REG_ADDR(bar, regs-\u003eevent_db));\n+}\n+\n+static u8 andes_get_mr_page_mode(u8 page_shift)\n+{\n+enum {\n+\tXSC_PAGE_SHIFT_4K\t= 12,\n+\tXSC_PAGE_SHIFT_64K\t= 16,\n+\tXSC_PAGE_SHIFT_2M\t= 21,\n+\tXSC_PAGE_SHIFT_1G\t= 30,\n+};\n+\n+enum {\n+\tXSC_PAGE_MODE_4K\t= 0,\n+\tXSC_PAGE_MODE_64K\t= 1,\n+\tXSC_PAGE_MODE_2M\t= 2,\n+\tXSC_PAGE_MODE_1G\t= 3,\n+};\n+\n+\treturn (page_shift == XSC_PAGE_SHIFT_4K ? XSC_PAGE_MODE_4K :\n+\t\t(page_shift == XSC_PAGE_SHIFT_64K ? XSC_PAGE_MODE_64K :\n+\t\t(page_shift == XSC_PAGE_SHIFT_2M ? XSC_PAGE_MODE_2M : XSC_PAGE_MODE_1G)));\n+}\n+\n+static inline u32 andes_mkey_to_idx(u32 mkey)\n+{\n+\treturn mkey \u003e\u003e 17;\n+}\n+\n+static inline u32 andes_idx_to_mkey(u32 mkey_idx)\n+{\n+\treturn mkey_idx \u003c\u003c 17;\n+}\n+\n+static const u32 xsc_msg_opcode[][2][2] = {\n+\t[XSC_MSG_OPCODE_SEND][XSC_REQ][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_REQ_SEND,\n+\t[XSC_MSG_OPCODE_SEND][XSC_REQ][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_REQ_SEND_IMMDT,\n+\t[XSC_MSG_OPCODE_SEND][XSC_RSP][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_RSP_RECV,\n+\t[XSC_MSG_OPCODE_SEND][XSC_RSP][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_RSP_RECV_IMMDT,\n+\t[XSC_MSG_OPCODE_RDMA_WRITE][XSC_REQ][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_REQ_WRITE,\n+\t[XSC_MSG_OPCODE_RDMA_WRITE][XSC_REQ][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_REQ_WRITE_IMMDT,\n+\t[XSC_MSG_OPCODE_RDMA_WRITE][XSC_RSP][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_CQE_ERROR,\n+\t[XSC_MSG_OPCODE_RDMA_WRITE][XSC_RSP][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_RSP_WRITE_IMMDT,\n+\t[XSC_MSG_OPCODE_RDMA_READ][XSC_REQ][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_REQ_READ,\n+\t[XSC_MSG_OPCODE_RDMA_READ][XSC_REQ][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_CQE_ERROR,\n+\t[XSC_MSG_OPCODE_RDMA_READ][XSC_RSP][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_CQE_ERROR,\n+\t[XSC_MSG_OPCODE_RDMA_READ][XSC_RSP][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_CQE_ERROR,\n+\t[XSC_MSG_OPCODE_MAD][XSC_REQ][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_MAD_REQ_SEND,\n+\t[XSC_MSG_OPCODE_MAD][XSC_RSP][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_MAD_RSP_RECV,\n+};\n+\n+struct andes_cqe {\n+\tunion {\n+\t\tu8\t\tmsg_opcode;\n+\t\tstruct {\n+\t\t\tu8\t\terror_code:7;\n+\t\t\tu8\t\tis_error:1;\n+\t\t};\n+\t};\n+\t__le32\t\tqp_id:15;\n+\tu8\t\trsv1:1;\n+\tu8\t\tse:1;\n+\tu8\t\thas_pph:1;\n+\tu8\t\ttype:1;\n+\tu8\t\twith_immdt:1;\n+\tu8\t\tcsum_err:4;\n+\t__le32\t\timm_data;\n+\t__le32\t\tmsg_len;\n+\t__le32\t\tvni;\n+\t__le64\t\tts:48;\n+\t__le16\t\twqe_id;\n+\t__le16\t\trsv[3];\n+\t__le16\t\trsv2:15;\n+\tu8\t\towner:1;\n+};\n+\n+static int andes_get_cqe_wqe_id(void *cqe)\n+{\n+\treturn ((struct andes_cqe *)cqe)-\u003ewqe_id;\n+}\n+\n+static u8 andes_get_cqe_opcode(void *cqe)\n+{\n+\tstruct andes_cqe *_cqe = cqe;\n+\tu8 msg_opcode = _cqe-\u003emsg_opcode;\n+\n+\tif (_cqe-\u003eis_error)\n+\t\treturn _cqe-\u003etype ? XSC_OPCODE_RDMA_RSP_ERROR : XSC_OPCODE_RDMA_REQ_ERROR;\n+\tif (msg_opcode \u003e XSC_MSG_OPCODE_MAD)\n+\t\treturn XSC_OPCODE_RDMA_CQE_ERROR;\n+\treturn xsc_msg_opcode[msg_opcode][_cqe-\u003etype][_cqe-\u003ewith_immdt];\n+}\n+\n+struct andes_data_seg {\n+\tu32\tin_line:1;\n+\tu32\tlength:31;\n+\tu32\tkey;\n+\tu64\taddr;\n+};\n+\n+static void andes_set_data_seg(void *data_seg, u32 length, u32 key, u64 addr)\n+{\n+\tstruct andes_data_seg *seg = data_seg;\n+\n+\tseg-\u003elength = length;\n+\tseg-\u003ekey = key;\n+\tseg-\u003eaddr = addr;\n+}\n+\n+struct andes_send_wqe_ctrl_seg {\n+\t__le32\t\tmsg_opcode:8;\n+\t__le32\t\twith_immdt:1;\n+\t__le32\t\tcsum_en:2;\n+\t__le32\t\tds_data_num:5;\n+\t__le32\t\twqe_id:16;\n+\t__le32\t\tmsg_len;\n+\tunion {\n+\t\t__le32\t\topcode_data;\n+\t\tstruct {\n+\t\t\tu8\t\thas_pph:1;\n+\t\t\tu8\t\tso_type:1;\n+\t\t\t__le16\t\tso_data_size:14;\n+\t\t\tu8:8;\n+\t\t\tu8\t\tso_hdr_len:8;\n+\t\t};\n+\t\tstruct {\n+\t\t\t__le16\t\tdesc_id;\n+\t\t\t__le16\t\tis_last_wqe:1;\n+\t\t\t__le16\t\tdst_qp_id:15;\n+\t\t};\n+\t};\n+\t__le32\t\tse:1;\n+\t__le32\t\tce:1;\n+\t__le32:30;\n+};\n+\n+static void andes_set_wqe_id(void *cseg, u32 wqe_id)\n+{\n+\tu16 _wqe_id = wqe_id;\n+\n+\t((struct andes_send_wqe_ctrl_seg *)cseg)-\u003ewqe_id = cpu_to_le16(_wqe_id);\n+}\n+\n+static struct xsc_msg_opcode xsc_ib_opcode = {\n+\t.send = XSC_MSG_OPCODE_SEND,\n+\t.send_with_imm = XSC_MSG_OPCODE_SEND,\n+\t.rdma_write = XSC_MSG_OPCODE_RDMA_WRITE,\n+\t.rdma_write_with_imm = XSC_MSG_OPCODE_RDMA_WRITE,\n+\t.rdma_read = XSC_MSG_OPCODE_RDMA_READ,\n+\t.atomic_cmp_and_swp = XSC_MSG_OPCODE_UNSUPPORT,\n+\t.atomic_fetch_and_add = XSC_MSG_OPCODE_UNSUPPORT,\n+\t.masked_atomic_cmp_and_swp = XSC_MSG_OPCODE_UNSUPPORT,\n+\t.masked_atomic_fetch_and_add = XSC_MSG_OPCODE_UNSUPPORT,\n+\t.mad = XSC_MSG_OPCODE_MAD,\n+};\n+\n+static struct xsc_msg_opcode *andes_get_msg_opcode(void)\n+{\n+\treturn \u0026xsc_ib_opcode;\n+}\n+\n+static struct xsc_hw_reg andes_pf_regs = {\n+\t.req_pid_addr = HIF_CMDQM_HOST_REQ_PID_MEM_ADDR - 0xa0000000,\n+\t.req_cid_addr = HIF_CMDQM_HOST_REQ_CID_MEM_ADDR - 0xa0000000,\n+\t.rsp_pid_addr = HIF_CMDQM_HOST_RSP_PID_MEM_ADDR - 0xa0000000,\n+\t.rsp_cid_addr = HIF_CMDQM_HOST_RSP_CID_MEM_ADDR - 0xa0000000,\n+\t.req_buf_h_addr = HIF_CMDQM_HOST_REQ_BUF_BASE_H_ADDR_MEM_ADDR - 0xa0000000,\n+\t.req_buf_l_addr = HIF_CMDQM_HOST_REQ_BUF_BASE_L_ADDR_MEM_ADDR - 0xa0000000,\n+\t.rsp_buf_h_addr = HIF_CMDQM_HOST_RSP_BUF_BASE_H_ADDR_MEM_ADDR - 0xa0000000,\n+\t.rsp_buf_l_addr = HIF_CMDQM_HOST_RSP_BUF_BASE_L_ADDR_MEM_ADDR - 0xa0000000,\n+\t.msix_vec_addr = HIF_CMDQM_VECTOR_ID_MEM_ADDR - 0xa0000000,\n+\t.interrupt_stat_addr = HIF_CMDQM_HOST_VF_ERR_STS_MEM_ADDR - 0xa0000000,\n+\t.mpt_tbl_addr = MMC_MPT_TBL_MEM_ADDR - 0xa0000000,\n+\t.mpt_tbl_depth = MMC_MPT_TBL_MEM_DEPTH,\n+\t.mpt_tbl_width = MMC_MPT_TBL_MEM_WIDTH,\n+\t.mtt_inst_base_addr = MMC_MTT_TBL_MEM_ADDR - 0xa0000000,\n+\t.mtt_inst_stride = 0,\n+\t.mtt_inst_num_log = 0,\n+\t.mtt_inst_depth = MMC_MTT_TBL_MEM_DEPTH,\n+};\n+\n+/* BAR-relative window shared by bar-compressed PF and VF */\n+static struct xsc_hw_reg andes_compressed_regs = {\n+\t.tx_db = TX_DB_FUNC_MEM_ADDR,\n+\t.rx_db = RX_DB_FUNC_MEM_ADDR,\n+\t.complete_db = DB_CQ_FUNC_MEM_ADDR,\n+\t.complete_reg = DB_CQ_CID_DIRECT_MEM_ADDR,\n+\t.event_db = DB_EQ_FUNC_MEM_ADDR,\n+\t.req_pid_addr = CMDQM_HOST_REQ_PID_MEM_ADDR,\n+\t.req_cid_addr = CMDQM_HOST_REQ_CID_MEM_ADDR,\n+\t.rsp_pid_addr = CMDQM_HOST_RSP_PID_MEM_ADDR,\n+\t.rsp_cid_addr = CMDQM_HOST_RSP_CID_MEM_ADDR,\n+\t.req_buf_h_addr = CMDQM_HOST_REQ_BUF_BASE_H_ADDR_MEM_ADDR,\n+\t.req_buf_l_addr = CMDQM_HOST_REQ_BUF_BASE_L_ADDR_MEM_ADDR,\n+\t.rsp_buf_h_addr = CMDQM_HOST_RSP_BUF_BASE_H_ADDR_MEM_ADDR,\n+\t.rsp_buf_l_addr = CMDQM_HOST_RSP_BUF_BASE_L_ADDR_MEM_ADDR,\n+\t.msix_vec_addr = CMDQM_VECTOR_ID_MEM_ADDR,\n+\t.interrupt_stat_addr = CMDQM_HOST_VF_ERR_STS_MEM_ADDR,\n+\t.mpt_tbl_addr = MMC_MPT_TBL_MEM_ADDR - 0xa0000000,\n+\t.mpt_tbl_depth = MMC_MPT_TBL_MEM_DEPTH,\n+\t.mpt_tbl_width = MMC_MPT_TBL_MEM_WIDTH,\n+\t.mtt_inst_base_addr = MMC_MTT_TBL_MEM_ADDR - 0xa0000000,\n+\t.mtt_inst_stride = 0,\n+\t.mtt_inst_num_log = 0,\n+\t.mtt_inst_depth = MMC_MTT_TBL_MEM_DEPTH,\n+};\n+\n+void xsc_init_hal(struct xsc_core_device *xdev, u32 device_id)\n+{\n+\tswitch (device_id) {\n+\tcase XSC_MC_VF_DEV_ID:\n+\tcase XSC_MF_HOST_VF_DEV_ID:\n+\tcase XSC_MS_VF_DEV_ID:\n+\tcase XSC_MV_HOST_VF_DEV_ID:\n+\t\t/* VFs expose the same BAR-relative window as a bar-compressed PF */\n+\t\txdev-\u003ehal_regs = \u0026andes_compressed_regs;\n+\t\tbreak;\n+\tdefault:\n+\t\tif (is_pf_bar_compressed(xdev))\n+\t\t\txdev-\u003ehal_regs = \u0026andes_compressed_regs;\n+\t\telse\n+\t\t\txdev-\u003ehal_regs = \u0026andes_pf_regs;\n+\t\tbreak;\n+\t}\n+}\n+\n+void xsc_set_mtt_info(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_hw_reg *regs = xdev-\u003ehal_regs;\n+\n+\tregs-\u003empt_tbl_addr = xdev-\u003ecaps.mpt_tbl_addr - 0xa0000000;\n+\tregs-\u003empt_tbl_depth = xdev-\u003ecaps.mpt_tbl_depth;\n+\tregs-\u003empt_tbl_width = xdev-\u003ecaps.mpt_tbl_width;\n+\tregs-\u003emtt_inst_base_addr = xdev-\u003ecaps.mtt_inst_base_addr - 0xa0000000;\n+\tregs-\u003emtt_inst_stride = xdev-\u003ecaps.mtt_inst_stride;\n+\tregs-\u003emtt_inst_num_log = xdev-\u003ecaps.mtt_inst_num_log;\n+\tregs-\u003emtt_inst_depth = xdev-\u003ecaps.mtt_inst_depth;\n+}\n+\n+void xsc_set_pf_db_addr(struct xsc_core_device *xdev,\n+\t\t\tu64 tx_db, u64 rx_db, u64 cq_db, u64 cq_reg, u64 eq_db)\n+{\n+\tstruct xsc_hw_reg *regs = xdev-\u003ehal_regs;\n+\n+\tif (is_pf_bar_compressed(xdev))\n+\t\treturn;\n+\n+\tif (xsc_core_is_pf(xdev)) {\n+\t\tregs-\u003etx_db = tx_db - 0xa0000000;\n+\t\tregs-\u003erx_db = rx_db - 0xa0000000;\n+\t\tregs-\u003ecomplete_db = cq_db - 0xa0000000;\n+\t\tregs-\u003ecomplete_reg = cq_reg - 0xa0000000;\n+\t\tregs-\u003eevent_db = eq_db - 0xa0000000;\n+\t}\n+}\n+\n+void xsc_get_db_addr(struct xsc_core_device *xdev,\n+\t\t u64 *tx_db, u64 *rx_db, u64 *cq_db, u64 *cq_reg, u64 *eq_db)\n+{\n+\tstruct xsc_hw_reg *regs = xdev-\u003ehal_regs;\n+\n+\tif (!xsc_core_is_pf(xdev) || is_pf_bar_compressed(xdev)) {\n+\t\tif (tx_db)\n+\t\t\t*tx_db = regs-\u003etx_db;\n+\t\tif (rx_db)\n+\t\t\t*rx_db = regs-\u003erx_db;\n+\t\tif (cq_db)\n+\t\t\t*cq_db = regs-\u003ecomplete_db;\n+\t\tif (cq_reg)\n+\t\t\t*cq_reg = regs-\u003ecomplete_reg;\n+\t\tif (eq_db)\n+\t\t\t*eq_db = regs-\u003eevent_db;\n+\t} else {\n+\t\tif (tx_db)\n+\t\t\t*tx_db = regs-\u003etx_db + 0xa0000000;\n+\t\tif (rx_db)\n+\t\t\t*rx_db = regs-\u003erx_db + 0xa0000000;\n+\t\tif (cq_db)\n+\t\t\t*cq_db = regs-\u003ecomplete_db + 0xa0000000;\n+\t\tif (cq_reg)\n+\t\t\t*cq_reg = regs-\u003ecomplete_reg + 0xa0000000;\n+\t\tif (eq_db)\n+\t\t\t*eq_db = regs-\u003eevent_db + 0xa0000000;\n+\t}\n+}\n+EXPORT_SYMBOL_GPL(xsc_get_db_addr);\n+\n+void xsc_update_tx_db(struct xsc_core_device *xdev, u32 sqn, u32 next_pid)\n+{\n+\tbool mdb = is_support_multi_doorbell(xdev);\n+\n+\tandes_ring_tx_doorbell(xdev, xdev-\u003ehal_regs, xdev-\u003ebar, sqn,\n+\t\t\t next_pid, mdb);\n+}\n+EXPORT_SYMBOL_GPL(xsc_update_tx_db);\n+\n+void xsc_update_rx_db(struct xsc_core_device *xdev, u32 rqn, u32 next_pid)\n+{\n+\tandes_ring_rx_doorbell(xdev-\u003ehal_regs, xdev-\u003ebar, rqn, next_pid);\n+}\n+EXPORT_SYMBOL_GPL(xsc_update_rx_db);\n+\n+void xsc_arm_cq(struct xsc_core_device *xdev, u32 cqn, u32 next_cid, u8 solicited)\n+{\n+\tandes_update_cq_db(xdev-\u003ehal_regs, xdev-\u003ebar, cqn, next_cid, solicited);\n+}\n+EXPORT_SYMBOL_GPL(xsc_arm_cq);\n+\n+void xsc_update_cq_ci(struct xsc_core_device *xdev, u32 cqn, u32 next_cid)\n+{\n+\tandes_set_cq_ci(xdev-\u003ehal_regs, xdev-\u003ebar, cqn, next_cid);\n+}\n+EXPORT_SYMBOL_GPL(xsc_update_cq_ci);\n+\n+void xsc_update_eq_ci(struct xsc_core_device *xdev, u32 eqn, u32 next_cid, u8 arm)\n+{\n+\tandes_set_eq_ci(xdev-\u003ehal_regs, xdev-\u003ebar, eqn, next_cid, arm);\n+}\n+\n+void xsc_update_cmdq_req_pid(struct xsc_core_device *xdev, u32 req_pid)\n+{\n+\txsc_write32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003ereq_pid_addr, \u0026req_pid);\n+}\n+\n+void xsc_update_cmdq_rsp_cid(struct xsc_core_device *xdev, u32 rsp_cid)\n+{\n+\txsc_write32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003ersp_cid_addr, \u0026rsp_cid);\n+}\n+\n+u32 xsc_get_cmdq_req_pid(struct xsc_core_device *xdev)\n+{\n+\tu32 val;\n+\n+\txsc_read32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003ereq_pid_addr, \u0026val, sizeof(val));\n+\treturn val;\n+}\n+\n+u32 xsc_get_cmdq_req_cid(struct xsc_core_device *xdev)\n+{\n+\tu32 val;\n+\n+\txsc_read32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003ereq_cid_addr, \u0026val, sizeof(val));\n+\treturn val;\n+}\n+\n+u32 xsc_get_cmdq_rsp_pid(struct xsc_core_device *xdev)\n+{\n+\tu32 val;\n+\n+\txsc_read32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003ersp_pid_addr, \u0026val, sizeof(val));\n+\treturn val;\n+}\n+\n+u32 xsc_get_cmdq_rsp_cid(struct xsc_core_device *xdev)\n+{\n+\tu32 val;\n+\n+\txsc_read32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003ersp_cid_addr, \u0026val, sizeof(val));\n+\treturn val;\n+}\n+\n+void xsc_set_cmdq_req_buf_addr(struct xsc_core_device *xdev, u32 haddr, u32 laddr)\n+{\n+\txsc_write32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003ereq_buf_h_addr, \u0026haddr);\n+\txsc_write32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003ereq_buf_l_addr, \u0026laddr);\n+}\n+\n+void xsc_set_cmdq_rsp_buf_addr(struct xsc_core_device *xdev, u32 haddr, u32 laddr)\n+{\n+\txsc_write32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003ersp_buf_h_addr, \u0026haddr);\n+\txsc_write32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003ersp_buf_l_addr, \u0026laddr);\n+}\n+\n+void xsc_set_cmdq_msix_vector(struct xsc_core_device *xdev, u32 vector)\n+{\n+\txsc_write32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003emsix_vec_addr, \u0026vector);\n+}\n+\n+void xsc_check_cmdq_status(struct xsc_core_device *xdev)\n+{\n+\tu32 status;\n+\n+\txsc_read32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003einterrupt_stat_addr, \u0026status,\n+\t\t sizeof(status));\n+\tif (status) {\n+\t\tstatus = 0xf;\n+\t\txsc_write32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003einterrupt_stat_addr, \u0026status);\n+\t}\n+}\n+\n+int xsc_handle_cmdq_interrupt(struct xsc_core_device *xdev)\n+{\n+\tunion interrupt_stat {\n+\t\tstruct {\n+\t\t\tu32\thw_read_req_err:1;\n+\t\t\tu32\thw_write_req_err:1;\n+\t\t\tu32\treq_pid_err:1;\n+\t\t\tu32\trsp_cid_err:1;\n+\t\t};\n+\t\tu32\traw;\n+\t} stat;\n+\n+\tint err = 0;\n+\tint retry = 0;\n+\n+\txsc_read32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003einterrupt_stat_addr, \u0026stat.raw,\n+\t\t sizeof(stat.raw));\n+\twhile (stat.raw != 0) {\n+\t\terr++;\n+\t\tif (stat.hw_read_req_err) {\n+\t\t\tretry = 1;\n+\t\t\tstat.hw_read_req_err = 0;\n+\t\t\tpci_err(xdev-\u003epdev, \"hw report read req from host failed!\\n\");\n+\t\t} else if (stat.hw_write_req_err) {\n+\t\t\tretry = 1;\n+\t\t\tstat.hw_write_req_err = 0;\n+\t\t\tpci_err(xdev-\u003epdev, \"hw report write req to fw failed!\\n\");\n+\t\t} else if (stat.req_pid_err) {\n+\t\t\tstat.req_pid_err = 0;\n+\t\t\tpci_err(xdev-\u003epdev, \"hw report unexpected req pid!\\n\");\n+\t\t} else if (stat.rsp_cid_err) {\n+\t\t\tstat.rsp_cid_err = 0;\n+\t\t\tpci_err(xdev-\u003epdev, \"hw report unexpected rsp cid!\\n\");\n+\t\t} else {\n+\t\t\tstat.raw = 0;\n+\t\t\tpci_err(xdev-\u003epdev, \"ignore unknown interrupt!\\n\");\n+\t\t}\n+\t}\n+\n+\tif (retry) {\n+\t\tif (xdev-\u003ecmd.retry_cnt \u003e= XSC_CMD_MAX_RETRY_CNT) {\n+\t\t\tpci_warn(xdev-\u003epdev, \"err handler have retried for %d times, stop retry\\n\",\n+\t\t\t\t xdev-\u003ecmd.retry_cnt);\n+\t\t\tgoto clear_err_int;\n+\t\t}\n+\t\txsc_write32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003ereq_pid_addr,\n+\t\t\t \u0026xdev-\u003ecmd.cmd_pid);\n+\t\txdev-\u003ecmd.retry_cnt++;\n+\t}\n+\n+clear_err_int:\n+\tif (err) {\n+\t\tstat.raw = 0xf;\n+\t\txsc_write32(xdev-\u003ebar, xdev-\u003ehal_regs-\u003einterrupt_stat_addr, \u0026stat.raw);\n+\t}\n+\n+\treturn err;\n+}\n+\n+u8 xsc_get_mr_page_mode(struct xsc_core_device *xdev, u8 page_shift)\n+{\n+\treturn andes_get_mr_page_mode(page_shift);\n+}\n+EXPORT_SYMBOL_GPL(xsc_get_mr_page_mode);\n+\n+u32 xsc_mkey_to_idx(struct xsc_core_device *xdev, u32 mkey)\n+{\n+\treturn andes_mkey_to_idx(mkey);\n+}\n+\n+u32 xsc_idx_to_mkey(struct xsc_core_device *xdev, u32 mkey_idx)\n+{\n+\treturn andes_idx_to_mkey(mkey_idx);\n+}\n+\n+u8 xsc_get_cqe_opcode(struct xsc_core_device *xdev, void *cqe)\n+{\n+\treturn andes_get_cqe_opcode(cqe);\n+}\n+EXPORT_SYMBOL_GPL(xsc_get_cqe_opcode);\n+\n+u64 xsc_get_rdma_stat_mask(struct xsc_core_device *xdev, u16 ver)\n+{\n+\tu64 mask;\n+\n+\tif (xsc_core_is_pf(xdev))\n+\t\tmask = 0xFFFFFF;\n+\telse\n+\t\tmask = 0xfff;\n+\n+\treturn mask;\n+}\n+EXPORT_SYMBOL_GPL(xsc_get_rdma_stat_mask);\n+\n+void xsc_set_data_seg(struct xsc_core_device *xdev, void *data_seg, u64 addr, u32 key, u32 length)\n+{\n+\tandes_set_data_seg(data_seg, length, key, addr);\n+}\n+EXPORT_SYMBOL_GPL(xsc_set_data_seg);\n+\n+u32 xsc_get_max_qp_depth(struct xsc_core_device *xdev)\n+{\n+\treturn 32 * 1024;\n+}\n+EXPORT_SYMBOL_GPL(xsc_get_max_qp_depth);\n+\n+bool xsc_check_max_qp_depth(struct xsc_core_device *xdev, u32 *wqe_cnt, u32 max_qp_depth)\n+{\n+\tif (*wqe_cnt \u003e max_qp_depth)\n+\t\t*wqe_cnt = max_qp_depth;\n+\n+\treturn false;\n+}\n+EXPORT_SYMBOL_GPL(xsc_check_max_qp_depth);\n+\n+u32 xsc_get_recv_ds_num(struct xsc_core_device *xdev)\n+{\n+\treturn xdev-\u003ecaps.recv_ds_num;\n+}\n+EXPORT_SYMBOL_GPL(xsc_get_recv_ds_num);\n+\n+void *xsc_get_msg_opcode(struct xsc_core_device *xdev)\n+{\n+\treturn andes_get_msg_opcode();\n+}\n+\n+u64 xsc_get_max_mr_size(struct xsc_core_device *xdev)\n+{\n+\tu64 max_mpt_mr_size = xdev-\u003ecaps.max_mr_size;\n+\tu64 max_mtt_mr_size = xdev-\u003ecaps.max_mtt * PAGE_SIZE;\n+\n+\treturn min_t(u64, max_mpt_mr_size, max_mtt_mr_size);\n+}\n+EXPORT_SYMBOL_GPL(xsc_get_max_mr_size);\n+\n+void xsc_get_multidb_info(struct xsc_core_device *xdev, u32 *num, u32 *base)\n+{\n+\tif (is_support_multi_doorbell(xdev)) {\n+\t\t*num = MDB_NUM;\n+\t\t*base = QPM_TX_MDB_BASE_REG_ADDR;\n+\t} else {\n+\t\t*num = 0;\n+\t\t*base = 0;\n+\t}\n+}\n+EXPORT_SYMBOL_GPL(xsc_get_multidb_info);\n+\n+void xsc_set_wqe_id(struct xsc_core_device *dev, void *cseg, u32 wqe_id)\n+{\n+\tandes_set_wqe_id(cseg, wqe_id);\n+}\n+EXPORT_SYMBOL_GPL(xsc_set_wqe_id);\n+\n+int xsc_get_cqe_wqe_id(struct xsc_core_device *dev, void *cqe)\n+{\n+\treturn andes_get_cqe_wqe_id(cqe);\n+}\n+EXPORT_SYMBOL_GPL(xsc_get_cqe_wqe_id);\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_reg.h b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_reg.h\nnew file mode 100644\nindex 0000000000000..cd25b64a58184\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_reg.h\n@@ -0,0 +1,43 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/*\n+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+/* generated time:\n+ * Thu Feb 29 15:33:50 CST 2024\n+ */\n+\n+#ifndef ANDES_REG_H\n+#define ANDES_REG_H\n+\n+//hif_irq_csr_defines.h\n+\n+//hif_cpm_csr_defines.h\n+\n+//mmc_csr_defines.h\n+#define MMC_MPT_TBL_MEM_DEPTH 32768\n+#define MMC_MTT_TBL_MEM_DEPTH 262144\n+#define MMC_MPT_TBL_MEM_WIDTH 256\n+#define MMC_MPT_TBL_MEM_ADDR 0xa4100000\n+#define MMC_MTT_TBL_MEM_ADDR 0xa4200000\n+\n+//clsf_dma_csr_defines.h\n+\n+//hif_tbl_csr_defines.h\n+\n+//hif_cmdqm_csr_defines.h\n+#define HIF_CMDQM_HOST_REQ_PID_MEM_ADDR 0xa1026000\n+#define HIF_CMDQM_HOST_REQ_CID_MEM_ADDR 0xa1028000\n+#define HIF_CMDQM_HOST_RSP_PID_MEM_ADDR 0xa102e000\n+#define HIF_CMDQM_HOST_RSP_CID_MEM_ADDR 0xa1030000\n+#define HIF_CMDQM_HOST_REQ_BUF_BASE_H_ADDR_MEM_ADDR 0xa1022000\n+#define HIF_CMDQM_HOST_REQ_BUF_BASE_L_ADDR_MEM_ADDR 0xa1024000\n+#define HIF_CMDQM_HOST_RSP_BUF_BASE_H_ADDR_MEM_ADDR 0xa102a000\n+#define HIF_CMDQM_HOST_RSP_BUF_BASE_L_ADDR_MEM_ADDR 0xa102c000\n+#define HIF_CMDQM_VECTOR_ID_MEM_ADDR 0xa1034000\n+#define HIF_CMDQM_HOST_VF_ERR_STS_MEM_ADDR 0xa1032000\n+\n+//PSV use\n+//hif_irq_csr_defines.h\n+\n+#endif /* XSC_HW_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h\nnew file mode 100644\nindex 0000000000000..5a3a7e27f34b1\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h\n@@ -0,0 +1,85 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2024 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef XSC_HAL_H\n+#define XSC_HAL_H\n+\n+struct xsc_core_device;\n+\n+struct xsc_hw_reg {\n+\tu32 tx_db;\n+\tu32 rx_db;\n+\tu32 complete_db;\n+\tu32 complete_reg;\n+\tu32 event_db;\n+\tu32 req_pid_addr;\n+\tu32 req_cid_addr;\n+\tu32 rsp_pid_addr;\n+\tu32 rsp_cid_addr;\n+\tu32 req_buf_h_addr;\n+\tu32 req_buf_l_addr;\n+\tu32 rsp_buf_h_addr;\n+\tu32 rsp_buf_l_addr;\n+\tu32 msix_vec_addr;\n+\tu32 interrupt_stat_addr;\n+\tu32 mpt_tbl_addr;\n+\tu32 mpt_tbl_depth;\n+\tu32 mpt_tbl_width;\n+\tu32 mtt_inst_base_addr;\n+\tu32 mtt_inst_stride;\n+\tu32 mtt_inst_num_log;\n+\tu32 mtt_inst_depth;\n+};\n+\n+/* HAL entry points exported to the PCI/RDMA/ETH layers */\n+void xsc_init_hal(struct xsc_core_device *xdev, u32 device_id);\n+void xsc_set_mtt_info(struct xsc_core_device *xdev);\n+void xsc_set_pf_db_addr(struct xsc_core_device *xdev,\n+\t\t\tu64 tx_db, u64 rx_db, u64 cq_db, u64 cq_reg, u64 eq_db);\n+void xsc_get_db_addr(struct xsc_core_device *xdev,\n+\t\t u64 *tx_db, u64 *rx_db, u64 *cq_db, u64 *cq_reg, u64 *eq_db);\n+\n+void xsc_update_tx_db(struct xsc_core_device *xdev, u32 sqn, u32 next_pid);\n+void xsc_update_rx_db(struct xsc_core_device *xdev, u32 rqn, u32 next_pid);\n+void xsc_arm_cq(struct xsc_core_device *xdev, u32 cqn, u32 next_cid, u8 solicited);\n+void xsc_update_cq_ci(struct xsc_core_device *xdev, u32 cqn, u32 next_cid);\n+void xsc_update_eq_ci(struct xsc_core_device *xdev, u32 eqn, u32 next_cid, u8 arm);\n+\n+void xsc_update_cmdq_req_pid(struct xsc_core_device *xdev, u32 req_pid);\n+void xsc_update_cmdq_rsp_cid(struct xsc_core_device *xdev, u32 rsp_cid);\n+u32 xsc_get_cmdq_req_pid(struct xsc_core_device *xdev);\n+u32 xsc_get_cmdq_req_cid(struct xsc_core_device *xdev);\n+u32 xsc_get_cmdq_rsp_pid(struct xsc_core_device *xdev);\n+u32 xsc_get_cmdq_rsp_cid(struct xsc_core_device *xdev);\n+void xsc_set_cmdq_req_buf_addr(struct xsc_core_device *xdev, u32 haddr, u32 laddr);\n+void xsc_set_cmdq_rsp_buf_addr(struct xsc_core_device *xdev, u32 haddr, u32 laddr);\n+void xsc_set_cmdq_msix_vector(struct xsc_core_device *xdev, u32 vector);\n+void xsc_check_cmdq_status(struct xsc_core_device *xdev);\n+int xsc_handle_cmdq_interrupt(struct xsc_core_device *xdev);\n+\n+u8 xsc_get_mr_page_mode(struct xsc_core_device *xdev, u8 page_shift);\n+u32 xsc_mkey_to_idx(struct xsc_core_device *xdev, u32 mkey);\n+u32 xsc_idx_to_mkey(struct xsc_core_device *xdev, u32 mkey_idx);\n+\n+u8 xsc_get_cqe_opcode(struct xsc_core_device *xdev, void *cqe);\n+u64 xsc_get_rdma_stat_mask(struct xsc_core_device *xdev, u16 ver);\n+void xsc_set_data_seg(struct xsc_core_device *xdev, void *data_seg, u64 addr, u32 key, u32 length);\n+u32 xsc_get_max_qp_depth(struct xsc_core_device *xdev);\n+bool xsc_check_max_qp_depth(struct xsc_core_device *xdev, u32 *wqe_cnt, u32 max_qp_depth);\n+u32 xsc_get_recv_ds_num(struct xsc_core_device *xdev);\n+void *xsc_get_msg_opcode(struct xsc_core_device *xdev);\n+u64 xsc_get_max_mr_size(struct xsc_core_device *xdev);\n+void xsc_get_multidb_info(struct xsc_core_device *xdev, u32 *num, u32 *base);\n+void xsc_set_wqe_id(struct xsc_core_device *dev, void *cseg, u32 wqe_id);\n+int xsc_get_cqe_wqe_id(struct xsc_core_device *dev, void *cqe);\n+\n+enum {\n+\tCQ_STAT_FIRED,\n+\tCQ_STAT_KEEP,\n+\tCQ_STAT_ARM_NEXT,\n+\tCQ_STAT_ARM_SOLICITED,\n+};\n+\n+#endif\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hw.c b/drivers/net/ethernet/yunsilicon/xsc/pci/hw.c\nnew file mode 100644\nindex 0000000000000..321e236e8e667\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hw.c\n@@ -0,0 +1,310 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/module.h\u003e\n+#include \u003clinux/vmalloc.h\u003e\n+\n+#include \"common/xsc_cmd_api.h\"\n+#include \"hw.h\"\n+#include \"hal/xsc_hal.h\"\n+\n+#define MAX_BOARD_NUM\t32\n+\n+static struct xsc_board_info *board_info[MAX_BOARD_NUM];\n+\n+static struct xsc_board_info *xsc_get_board_info(char *board_sn)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i \u003c MAX_BOARD_NUM; i++) {\n+\t\tif (!board_info[i])\n+\t\t\tcontinue;\n+\t\tif (!strncmp(board_info[i]-\u003eboard_sn, board_sn,\n+\t\t\t XSC_BOARD_SN_LEN))\n+\t\t\treturn board_info[i];\n+\t}\n+\n+\treturn NULL;\n+}\n+\n+static struct xsc_board_info *xsc_alloc_board_info(void)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i \u003c MAX_BOARD_NUM; i++) {\n+\t\tif (!board_info[i])\n+\t\t\tbreak;\n+\t}\n+\tif (i == MAX_BOARD_NUM)\n+\t\treturn NULL;\n+\tboard_info[i] = vzalloc(sizeof(*board_info[i]));\n+\tif (!board_info[i])\n+\t\treturn NULL;\n+\tboard_info[i]-\u003eboard_id = i;\n+\treturn board_info[i];\n+}\n+\n+void xsc_free_board_info(void)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i \u003c MAX_BOARD_NUM; i++)\n+\t\tvfree(board_info[i]);\n+}\n+\n+int xsc_cmd_query_hca_cap(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_caps *caps)\n+{\n+\tstruct xsc_cmd_query_hca_cap_mbox_out *out;\n+\tstruct xsc_cmd_query_hca_cap_mbox_in in;\n+\tstruct xsc_board_info *board_info;\n+\tint err;\n+\tu16 t16;\n+\n+\tout = kzalloc_obj(*out, GFP_KERNEL);\n+\tif (!out)\n+\t\treturn -ENOMEM;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_HCA_CAP);\n+\tin.cpu_num = cpu_to_be16(num_online_cpus());\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), out, sizeof(*out));\n+\tif (err)\n+\t\tgoto err_free;\n+\n+\txdev-\u003eglb_func_id = be32_to_cpu(out-\u003ehca_cap.glb_func_id);\n+\tcaps-\u003epcie0_pf_funcid_base =\n+\t\tbe16_to_cpu(out-\u003ehca_cap.pcie0_pf_funcid_base);\n+\tcaps-\u003epcie0_pf_funcid_top =\n+\t\tbe16_to_cpu(out-\u003ehca_cap.pcie0_pf_funcid_top);\n+\tcaps-\u003epcie1_pf_funcid_base =\n+\t\tbe16_to_cpu(out-\u003ehca_cap.pcie1_pf_funcid_base);\n+\tcaps-\u003epcie1_pf_funcid_top =\n+\t\tbe16_to_cpu(out-\u003ehca_cap.pcie1_pf_funcid_top);\n+\tcaps-\u003efuncid_to_logic_port =\n+\t\tbe16_to_cpu(out-\u003ehca_cap.funcid_to_logic_port);\n+\n+\tcaps-\u003epcie_host = out-\u003ehca_cap.pcie_host;\n+\tcaps-\u003enif_port_num = out-\u003ehca_cap.nif_port_num;\n+\tcaps-\u003ehw_feature_flag = be32_to_cpu(out-\u003ehca_cap.hw_feature_flag);\n+\n+\tcaps-\u003eraweth_qp_id_base = be16_to_cpu(out-\u003ehca_cap.raweth_qp_id_base);\n+\tcaps-\u003eraweth_qp_id_end = be16_to_cpu(out-\u003ehca_cap.raweth_qp_id_end);\n+\tcaps-\u003eraweth_rss_qp_id_base =\n+\t\tbe16_to_cpu(out-\u003ehca_cap.raweth_rss_qp_id_base);\n+\tcaps-\u003eraw_tpe_qp_num = be16_to_cpu(out-\u003ehca_cap.raw_tpe_qp_num);\n+\tcaps-\u003emax_cqes = 1 \u003c\u003c out-\u003ehca_cap.log_max_cq_sz;\n+\tcaps-\u003emax_wqes = 1 \u003c\u003c out-\u003ehca_cap.log_max_qp_sz;\n+\tcaps-\u003emax_sq_desc_sz = be16_to_cpu(out-\u003ehca_cap.max_desc_sz_sq);\n+\tcaps-\u003emax_rq_desc_sz = be16_to_cpu(out-\u003ehca_cap.max_desc_sz_rq);\n+\tcaps-\u003eflags = be64_to_cpu(out-\u003ehca_cap.flags);\n+\tcaps-\u003estat_rate_support = be16_to_cpu(out-\u003ehca_cap.stat_rate_support);\n+\tcaps-\u003elog_max_msg = out-\u003ehca_cap.log_max_msg \u0026 0x1f;\n+\tcaps-\u003enum_ports = out-\u003ehca_cap.num_ports \u0026 0xf;\n+\tcaps-\u003elog_max_cq = out-\u003ehca_cap.log_max_cq \u0026 0x1f;\n+\tcaps-\u003elog_max_eq = out-\u003ehca_cap.log_max_eq \u0026 0xf;\n+\tcaps-\u003elog_max_msix = out-\u003ehca_cap.log_max_msix \u0026 0xf;\n+\tcaps-\u003emac_port = out-\u003ehca_cap.mac_port \u0026 0xff;\n+\txdev-\u003emac_port = caps-\u003emac_port;\n+\tif (caps-\u003enum_ports \u003e XSC_MAX_FW_PORTS) {\n+\t\tpci_err(xdev-\u003epdev, \"device has %d ports while the driver supports max %d ports\\n\",\n+\t\t\tcaps-\u003enum_ports, XSC_MAX_FW_PORTS);\n+\t\terr = -EINVAL;\n+\t\tgoto err_free;\n+\t}\n+\tcaps-\u003esend_ds_num = out-\u003ehca_cap.send_seg_num;\n+\tcaps-\u003esend_wqe_shift = out-\u003ehca_cap.send_wqe_shift;\n+\tcaps-\u003erecv_ds_num = out-\u003ehca_cap.recv_seg_num;\n+\tcaps-\u003erecv_wqe_shift = out-\u003ehca_cap.recv_wqe_shift;\n+\n+\tcaps-\u003eembedded_cpu = 0;\n+\tcaps-\u003eecpf_vport_exists = 0;\n+\tcaps-\u003elog_max_current_uc_list = 0;\n+\tcaps-\u003elog_max_current_mc_list = 0;\n+\tcaps-\u003elog_max_vlan_list = 8;\n+\tcaps-\u003elog_max_qp = out-\u003ehca_cap.log_max_qp \u0026 0x1f;\n+\tcaps-\u003elog_max_mkey = out-\u003ehca_cap.log_max_mkey \u0026 0x3f;\n+\tcaps-\u003elog_max_pd = out-\u003ehca_cap.log_max_pd \u0026 0x1f;\n+\tcaps-\u003elog_max_srq = out-\u003ehca_cap.log_max_srqs \u0026 0x1f;\n+\tcaps-\u003elocal_ca_ack_delay = out-\u003ehca_cap.local_ca_ack_delay \u0026 0x1f;\n+\tcaps-\u003elog_max_mcg = out-\u003ehca_cap.log_max_mcg;\n+\tcaps-\u003elog_max_mtt = out-\u003ehca_cap.log_max_mtt;\n+\tcaps-\u003elog_max_tso = out-\u003ehca_cap.log_max_tso;\n+\tcaps-\u003ehca_core_clock = be32_to_cpu(out-\u003ehca_cap.hca_core_clock);\n+\tcaps-\u003emax_rwq_indirection_tables =\n+\t\tbe32_to_cpu(out-\u003ehca_cap.max_rwq_indirection_tables);\n+\tcaps-\u003emax_rwq_indirection_table_size =\n+\t\tbe32_to_cpu(out-\u003ehca_cap.max_rwq_indirection_table_size);\n+\tcaps-\u003emax_qp_mcg = be16_to_cpu(out-\u003ehca_cap.max_qp_mcg);\n+\tcaps-\u003emax_ra_res_qp = 1 \u003c\u003c (out-\u003ehca_cap.log_max_ra_res_qp \u0026 0x3f);\n+\tcaps-\u003emax_ra_req_qp = 1 \u003c\u003c (out-\u003ehca_cap.log_max_ra_req_qp \u0026 0x3f);\n+\tcaps-\u003emax_srq_wqes = 1 \u003c\u003c out-\u003ehca_cap.log_max_srq_sz;\n+\tcaps-\u003erx_pkt_len_max = be32_to_cpu(out-\u003ehca_cap.rx_pkt_len_max);\n+\tcaps-\u003emax_vfs = be16_to_cpu(out-\u003ehca_cap.max_vfs);\n+\tcaps-\u003eqp_rate_limit_min = be32_to_cpu(out-\u003ehca_cap.qp_rate_limit_min);\n+\tcaps-\u003eqp_rate_limit_max = be32_to_cpu(out-\u003ehca_cap.qp_rate_limit_max);\n+\n+\tcaps-\u003emsix_enable = 1;\n+\tcaps-\u003emsix_base = be16_to_cpu(out-\u003ehca_cap.msix_base);\n+\tcaps-\u003emsix_num = be16_to_cpu(out-\u003ehca_cap.msix_num);\n+\n+\tt16 = be16_to_cpu(out-\u003ehca_cap.bf_log_bf_reg_size);\n+\tif (t16 \u0026 0x8000) {\n+\t\tcaps-\u003ebf_reg_size = 1 \u003c\u003c (t16 \u0026 0x1f);\n+\t\tcaps-\u003ebf_regs_per_page = XSC_BF_REGS_PER_PAGE;\n+\t} else {\n+\t\tcaps-\u003ebf_reg_size = 0;\n+\t\tcaps-\u003ebf_regs_per_page = 0;\n+\t}\n+\tcaps-\u003emin_page_sz = ~(u32)((1 \u003c\u003c PAGE_SHIFT) - 1);\n+\n+\tcaps-\u003edcbx = 1;\n+\tcaps-\u003eqos = 1;\n+\tcaps-\u003eets = 1;\n+\tcaps-\u003edscp = 1;\n+\tcaps-\u003emax_tc = out-\u003ehca_cap.max_tc;\n+\tcaps-\u003elog_max_qp_depth = out-\u003ehca_cap.log_max_qp_depth \u0026 0xff;\n+\tcaps-\u003emac_bit = out-\u003ehca_cap.mac_bit;\n+\tcaps-\u003elag_logic_port_ofst = out-\u003ehca_cap.lag_logic_port_ofst;\n+\n+\tcaps-\u003emax_qp = be32_to_cpu(out-\u003ehca_cap.max_qp);\n+\tcaps-\u003emax_cq = be32_to_cpu(out-\u003ehca_cap.max_cq);\n+\tcaps-\u003emax_pd = be32_to_cpu(out-\u003ehca_cap.max_pd);\n+\n+\tif (be16_to_cpu(out-\u003ehdr.ver) \u003e= CMD_QUERY_HCA_CAP_V2)\n+\t\tcaps-\u003emax_mtt = be32_to_cpu(out-\u003ehca_cap.max_mtt);\n+\telse\n+\t\tcaps-\u003emax_mtt = 1 \u003c\u003c (out-\u003ehca_cap.log_max_mtt \u0026 0x3f);\n+\n+\tif (be16_to_cpu(out-\u003ehdr.ver) \u003e= CMD_QUERY_HCA_CAP_V1) {\n+\t\tcaps-\u003emax_mr_size = be64_to_cpu(out-\u003ehca_cap.max_mr_size);\n+\t\tcaps-\u003emax_cmd_in_len = be16_to_cpu(out-\u003ehca_cap.max_cmd_in_len);\n+\t\tcaps-\u003emax_cmd_out_len = be16_to_cpu(out-\u003ehca_cap.max_cmd_out_len);\n+\t} else {\n+\t\tcaps-\u003emax_mr_size = (1ull \u003c\u003c 32) - 1;\n+\t\tcaps-\u003emax_cmd_in_len = 10240;\n+\t\tcaps-\u003emax_cmd_out_len = 2048;\n+\t}\n+\n+\tif (be16_to_cpu(out-\u003ehdr.ver) \u003e= CMD_QUERY_HCA_CAP_V3) {\n+\t\tcaps-\u003empt_tbl_addr = be32_to_cpu(out-\u003ehca_cap.mpt_tbl_addr);\n+\t\tcaps-\u003empt_tbl_depth = be32_to_cpu(out-\u003ehca_cap.mpt_tbl_depth);\n+\t\tcaps-\u003empt_tbl_width = be32_to_cpu(out-\u003ehca_cap.mpt_tbl_width);\n+\t\tcaps-\u003emtt_inst_base_addr = be32_to_cpu(out-\u003ehca_cap.mtt_inst_base_addr);\n+\t\tcaps-\u003emtt_inst_stride = be32_to_cpu(out-\u003ehca_cap.mtt_inst_stride);\n+\t\tcaps-\u003emtt_inst_num_log = be32_to_cpu(out-\u003ehca_cap.mtt_inst_num_log);\n+\t\tcaps-\u003emtt_inst_depth = be32_to_cpu(out-\u003ehca_cap.mtt_inst_depth);\n+\t\txsc_set_mtt_info(xdev);\n+\t}\n+\n+\txdev-\u003epcie_no = out-\u003ehca_cap.pcie_no;\n+\txdev-\u003epf_id = out-\u003ehca_cap.pf_id;\n+\n+\tif (be16_to_cpu(out-\u003ehdr.ver) \u003e= CMD_QUERY_HCA_CAP_V9) {\n+\t\tcaps-\u003emdb_base = be64_to_cpu(out-\u003ehca_cap.mdb_base);\n+\t\tcaps-\u003emdb_num = be32_to_cpu(out-\u003ehca_cap.mdb_num);\n+\t}\n+\n+\txdev-\u003echip_ver_h = be32_to_cpu(out-\u003ehca_cap.chip_ver_h);\n+\txdev-\u003echip_ver_m = be32_to_cpu(out-\u003ehca_cap.chip_ver_m);\n+\txdev-\u003echip_ver_l = be32_to_cpu(out-\u003ehca_cap.chip_ver_l);\n+\txdev-\u003ehotfix_num = be32_to_cpu(out-\u003ehca_cap.hotfix_num);\n+\txdev-\u003efeature_flag = be32_to_cpu(out-\u003ehca_cap.feature_flag);\n+\n+\tboard_info = xsc_get_board_info(out-\u003ehca_cap.board_sn);\n+\tif (!board_info) {\n+\t\tboard_info = xsc_alloc_board_info();\n+\t\tif (!board_info) {\n+\t\t\terr = -ENOMEM;\n+\t\t\tgoto err_free;\n+\t\t}\n+\n+\t\tmemcpy(board_info-\u003eboard_sn,\n+\t\t out-\u003ehca_cap.board_sn,\n+\t\t sizeof(out-\u003ehca_cap.board_sn));\n+\t}\n+\n+\txdev-\u003eboard_info = board_info;\n+\n+\txsc_set_pf_db_addr(xdev, be64_to_cpu(out-\u003ehca_cap.tx_db),\n+\t\t\t be64_to_cpu(out-\u003ehca_cap.rx_db),\n+\t\t\t be64_to_cpu(out-\u003ehca_cap.complete_db),\n+\t\t\t be64_to_cpu(out-\u003ehca_cap.complete_reg),\n+\t\t\t be64_to_cpu(out-\u003ehca_cap.event_db));\n+\n+\txdev-\u003efw_version_major = out-\u003ehca_cap.fw_ver.fw_version_major;\n+\txdev-\u003efw_version_minor = out-\u003ehca_cap.fw_ver.fw_version_minor;\n+\txdev-\u003efw_version_patch =\n+\t\tbe16_to_cpu(out-\u003ehca_cap.fw_ver.fw_version_patch);\n+\txdev-\u003efw_version_tweak =\n+\t\tbe32_to_cpu(out-\u003ehca_cap.fw_ver.fw_version_tweak);\n+\txdev-\u003efw_version_extra_flag = out-\u003ehca_cap.fw_ver.fw_version_extra_flag;\n+\n+\tkfree(out);\n+\treturn 0;\n+err_free:\n+\tkfree(out);\n+\treturn err;\n+}\n+\n+static int xsc_cmd_query_guid(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_cmd_query_guid_mbox_out out;\n+\tstruct xsc_cmd_query_guid_mbox_in in = {};\n+\tint err;\n+\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_GUID);\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err)\n+\t\treturn err;\n+\n+\txdev-\u003eboard_info-\u003eguid = out.guid;\n+\txdev-\u003eboard_info-\u003eguid_valid = 1;\n+\treturn 0;\n+}\n+\n+int xsc_query_guid(struct xsc_core_device *xdev)\n+{\n+\tif (xdev-\u003eboard_info-\u003eguid_valid)\n+\t\treturn 0;\n+\n+\treturn xsc_cmd_query_guid(xdev);\n+}\n+\n+static int xsc_cmd_activate_hw_config(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_cmd_activate_hw_config_mbox_out out;\n+\tstruct xsc_cmd_activate_hw_config_mbox_in in = {};\n+\tint err = 0;\n+\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_ACTIVATE_HW_CONFIG);\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err)\n+\t\treturn err;\n+\n+\txdev-\u003eboard_info-\u003ehw_config_activated = 1;\n+\treturn 0;\n+}\n+\n+int xsc_activate_hw_config(struct xsc_core_device *xdev)\n+{\n+\tif (xdev-\u003eboard_info-\u003ehw_config_activated)\n+\t\treturn 0;\n+\n+\treturn xsc_cmd_activate_hw_config(xdev);\n+}\n+\n+int xsc_reset_function_resource(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_function_reset_mbox_out out;\n+\tstruct xsc_function_reset_mbox_in in;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_FUNCTION_RESET);\n+\tin.glb_func_id = cpu_to_be16(xdev-\u003eglb_func_id);\n+\n+\treturn xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+}\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hw.h b/drivers/net/ethernet/yunsilicon/xsc/pci/hw.h\nnew file mode 100644\nindex 0000000000000..5843a72c84bb2\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hw.h\n@@ -0,0 +1,18 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_HW_H\n+#define __XSC_HW_H\n+\n+#include \"common/xsc_core.h\"\n+\n+void xsc_free_board_info(void);\n+int xsc_cmd_query_hca_cap(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_caps *caps);\n+int xsc_query_guid(struct xsc_core_device *xdev);\n+int xsc_activate_hw_config(struct xsc_core_device *xdev);\n+int xsc_reset_function_resource(struct xsc_core_device *xdev);\n+\n+#endif\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/link.c b/drivers/net/ethernet/yunsilicon/xsc/pci/link.c\nnew file mode 100644\nindex 0000000000000..b991eddaee270\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/link.c\n@@ -0,0 +1,35 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/kernel.h\u003e\n+#include \u003clinux/module.h\u003e\n+\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_cmd_api.h\"\n+\n+int xsc_get_link_speed(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_event_query_linkinfo_mbox_out out;\n+\tstruct xsc_event_linkinfo linkinfo;\n+\tstruct xsc_inbox_hdr in;\n+\tint err;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(\u0026out, 0, sizeof(out));\n+\tin.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_LINK_INFO);\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err || out.hdr.status) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to get link info, err=%d, status=%d\\n\",\n+\t\t\terr, out.hdr.status);\n+\t\treturn MODULE_SPEED_25G;\n+\t}\n+\n+\tlinkinfo = out.ctx;\n+\tlinkinfo.linkspeed = be32_to_cpu(linkinfo.linkspeed);\n+\n+\treturn linkinfo.linkspeed;\n+}\n+EXPORT_SYMBOL_GPL(xsc_get_link_speed);\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/main.c b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c\nnew file mode 100644\nindex 0000000000000..c2a9f4863da78\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c\n@@ -0,0 +1,348 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_cmd_api.h\"\n+#include \"hw.h\"\n+#include \"qp.h\"\n+#include \"cq.h\"\n+#include \"srq.h\"\n+#include \"eq.h\"\n+#include \"pci_irq.h\"\n+#include \"adev.h\"\n+#include \"hal/xsc_hal.h\"\n+\n+static const struct pci_device_id xsc_pci_id_table[] = {\n+\t{ PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MC_PF_DEV_ID) },\n+\t{ PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MF_HOST_PF_DEV_ID) },\n+\t{ PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MF_SOC_PF_DEV_ID) },\n+\t{ PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MS_PF_DEV_ID) },\n+\t{ PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MV_HOST_PF_DEV_ID) },\n+\t{ PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MV_SOC_PF_DEV_ID) },\n+\t{ 0 }\n+};\n+\n+static int xsc_set_dma_caps(struct pci_dev *pdev)\n+{\n+\tint err;\n+\n+\terr = dma_set_coherent_mask(\u0026pdev-\u003edev, DMA_BIT_MASK(64));\n+\tif (!err)\n+\t\tdma_set_max_seg_size(\u0026pdev-\u003edev, SZ_2G);\n+\n+\treturn err;\n+}\n+\n+static int xsc_pci_init(struct xsc_core_device *xdev,\n+\t\t\tconst struct pci_device_id *id)\n+{\n+\tstruct pci_dev *pdev = xdev-\u003epdev;\n+\tvoid __iomem *bar_base;\n+\tint bar_num = 0;\n+\tint err;\n+\n+\txdev-\u003enuma_node = dev_to_node(\u0026pdev-\u003edev);\n+\n+\terr = pci_enable_device(pdev);\n+\tif (err) {\n+\t\tpci_err(pdev, \"failed to enable PCI device: err=%d\\n\", err);\n+\t\tgoto err_out;\n+\t}\n+\n+\terr = pci_request_region(pdev, bar_num, KBUILD_MODNAME);\n+\tif (err) {\n+\t\tpci_err(pdev, \"failed to request %s pci_region=%d: err=%d\\n\",\n+\t\t\tKBUILD_MODNAME, bar_num, err);\n+\t\tgoto err_disable;\n+\t}\n+\n+\tpci_set_master(pdev);\n+\n+\terr = xsc_set_dma_caps(pdev);\n+\tif (err) {\n+\t\tpci_err(pdev, \"failed to set DMA capabilities mask: err=%d\\n\",\n+\t\t\terr);\n+\t\tgoto err_clr_master;\n+\t}\n+\n+\tbar_base = pci_ioremap_bar(pdev, bar_num);\n+\tif (!bar_base) {\n+\t\tpci_err(pdev, \"failed to ioremap %s bar%d\\n\", KBUILD_MODNAME,\n+\t\t\tbar_num);\n+\t\terr = -ENOMEM;\n+\t\tgoto err_clr_master;\n+\t}\n+\n+\terr = pci_save_state(pdev);\n+\tif (err) {\n+\t\tpci_err(pdev, \"pci_save_state failed: err=%d\\n\", err);\n+\t\tgoto err_io_unmap;\n+\t}\n+\n+\txdev-\u003ebar_num = bar_num;\n+\txdev-\u003ebar = bar_base;\n+\n+\txsc_init_hal(xdev, pdev-\u003edevice);\n+\n+\treturn 0;\n+\n+err_io_unmap:\n+\tpci_iounmap(pdev, bar_base);\n+err_clr_master:\n+\tpci_clear_master(pdev);\n+\tpci_release_region(pdev, bar_num);\n+err_disable:\n+\tpci_disable_device(pdev);\n+err_out:\n+\treturn err;\n+}\n+\n+static void xsc_pci_fini(struct xsc_core_device *xdev)\n+{\n+\tstruct pci_dev *pdev = xdev-\u003epdev;\n+\n+\tpci_iounmap(pdev, xdev-\u003ebar);\n+\tpci_clear_master(pdev);\n+\tpci_release_region(pdev, xdev-\u003ebar_num);\n+\tpci_disable_device(pdev);\n+}\n+\n+static int xsc_dev_res_init(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_dev_resource *dev_res;\n+\n+\tdev_res = kvzalloc_obj(*dev_res, GFP_KERNEL);\n+\tif (!dev_res)\n+\t\treturn -ENOMEM;\n+\n+\txdev-\u003edev_res = dev_res;\n+\tmutex_init(\u0026dev_res-\u003ealloc_mutex);\n+\tspin_lock_init(\u0026dev_res-\u003emkey_lock);\n+\n+\tif (xsc_create_srq_res(xdev))\n+\t\tgoto err_free;\n+\n+\treturn 0;\n+\n+err_free:\n+\tkfree(dev_res);\n+\txdev-\u003edev_res = NULL;\n+\treturn -ENOMEM;\n+}\n+\n+static void xsc_dev_res_cleanup(struct xsc_core_device *xdev)\n+{\n+\txsc_destroy_srq_res(xdev);\n+\tkfree(xdev-\u003edev_res);\n+}\n+\n+static int xsc_core_dev_init(struct xsc_core_device *xdev)\n+{\n+\tint err;\n+\n+\terr = xsc_dev_res_init(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"xsc dev res init failed %d\\n\", err);\n+\t\treturn err;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static void xsc_core_dev_cleanup(struct xsc_core_device *xdev)\n+{\n+\txsc_dev_res_cleanup(xdev);\n+}\n+\n+static int xsc_hw_setup(struct xsc_core_device *xdev)\n+{\n+\tint err;\n+\n+\terr = xsc_cmd_init(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"Failed initializing command interface, aborting\\n\");\n+\t\tgoto err_out;\n+\t}\n+\n+\terr = xsc_cmd_version_check(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"Failed to check cmdq version\\n\");\n+\t\tgoto err_cmd_cleanup;\n+\t}\n+\n+\terr = xsc_cmd_query_hca_cap(xdev, \u0026xdev-\u003ecaps);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"Failed to query hca, err=%d\\n\", err);\n+\t\tgoto err_cmd_cleanup;\n+\t}\n+\n+\terr = xsc_query_guid(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to query guid, err=%d\\n\", err);\n+\t\tgoto err_cmd_cleanup;\n+\t}\n+\n+\terr = xsc_activate_hw_config(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to activate hw config, err=%d\\n\",\n+\t\t\terr);\n+\t\tgoto err_cmd_cleanup;\n+\t}\n+\n+\terr = xsc_reset_function_resource(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"Failed to reset function resource\\n\");\n+\t\tgoto err_cmd_cleanup;\n+\t}\n+\n+\t/* Required by the rdma client, which is probed right after this. */\n+\txdev-\u003eboard_info-\u003emsg_opcode = xsc_get_msg_opcode(xdev);\n+\tif (!xdev-\u003eboard_info-\u003emsg_opcode) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to get msg opcode table\\n\");\n+\t\terr = -ENODEV;\n+\t\tgoto err_cmd_cleanup;\n+\t}\n+\n+\txsc_init_cq_table(xdev);\n+\txsc_init_qp_table(xdev);\n+\txsc_eq_init(xdev);\n+\n+\treturn 0;\n+err_cmd_cleanup:\n+\txsc_cmd_cleanup(xdev);\n+err_out:\n+\treturn err;\n+}\n+\n+static void xsc_hw_cleanup(struct xsc_core_device *xdev)\n+{\n+\txsc_cmd_cleanup(xdev);\n+}\n+\n+static int xsc_load(struct xsc_core_device *xdev)\n+{\n+\tint err = 0;\n+\n+\terr = xsc_hw_setup(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"xsc_hw_setup failed %d\\n\", err);\n+\t\tgoto err_out;\n+\t}\n+\n+\terr = xsc_irq_eq_create(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"xsc_irq_eq_create failed %d\\n\", err);\n+\t\tgoto err_hw_cleanup;\n+\t}\n+\n+\terr = xsc_adev_init(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"xsc_adev_init failed %d\\n\", err);\n+\t\tgoto err_irq_eq_destroy;\n+\t}\n+\n+\treturn 0;\n+err_irq_eq_destroy:\n+\txsc_irq_eq_destroy(xdev);\n+err_hw_cleanup:\n+\txsc_hw_cleanup(xdev);\n+err_out:\n+\treturn err;\n+}\n+\n+static void xsc_unload(struct xsc_core_device *xdev)\n+{\n+\txsc_adev_uninit(xdev);\n+\txsc_irq_eq_destroy(xdev);\n+\txsc_hw_cleanup(xdev);\n+}\n+\n+static int xsc_pci_probe(struct pci_dev *pci_dev,\n+\t\t\t const struct pci_device_id *id)\n+{\n+\tstruct xsc_core_device *xdev;\n+\tint err;\n+\n+\txdev = kzalloc_obj(*xdev, GFP_KERNEL);\n+\tif (!xdev)\n+\t\treturn -ENOMEM;\n+\n+\txdev-\u003epdev = pci_dev;\n+\txdev-\u003edevice = \u0026pci_dev-\u003edev;\n+\txdev-\u003ecoredev_type = pci_dev-\u003eis_virtfn ? XSC_COREDEV_VF : XSC_COREDEV_PF;\n+\n+\tpci_set_drvdata(pci_dev, xdev);\n+\terr = xsc_pci_init(xdev, id);\n+\tif (err) {\n+\t\tpci_err(pci_dev, \"xsc_pci_init failed %d\\n\", err);\n+\t\tgoto err_unset_pci_drvdata;\n+\t}\n+\n+\terr = xsc_core_dev_init(xdev);\n+\tif (err) {\n+\t\tpci_err(pci_dev, \"xsc_core_dev_init failed %d\\n\", err);\n+\t\tgoto err_pci_fini;\n+\t}\n+\n+\terr = xsc_load(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"xsc_load failed %d\\n\", err);\n+\t\tgoto err_core_dev_cleanup;\n+\t}\n+\n+\treturn 0;\n+err_core_dev_cleanup:\n+\txsc_core_dev_cleanup(xdev);\n+err_pci_fini:\n+\txsc_pci_fini(xdev);\n+err_unset_pci_drvdata:\n+\tpci_set_drvdata(pci_dev, NULL);\n+\tkfree(xdev);\n+\n+\treturn err;\n+}\n+\n+static void xsc_pci_remove(struct pci_dev *pci_dev)\n+{\n+\tstruct xsc_core_device *xdev = pci_get_drvdata(pci_dev);\n+\n+\txsc_unload(xdev);\n+\txsc_core_dev_cleanup(xdev);\n+\txsc_pci_fini(xdev);\n+\tpci_set_drvdata(pci_dev, NULL);\n+\tkfree(xdev);\n+}\n+\n+static struct pci_driver xsc_pci_driver = {\n+\t.name\t\t= \"xsc-pci\",\n+\t.id_table\t= xsc_pci_id_table,\n+\t.probe\t\t= xsc_pci_probe,\n+\t.remove\t\t= xsc_pci_remove,\n+};\n+\n+static int __init xsc_init(void)\n+{\n+\tint err;\n+\n+\terr = pci_register_driver(\u0026xsc_pci_driver);\n+\tif (err) {\n+\t\tpr_err(\"failed to register pci driver\\n\");\n+\t\treturn err;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static void __exit xsc_fini(void)\n+{\n+\tpci_unregister_driver(\u0026xsc_pci_driver);\n+\txsc_free_board_info();\n+}\n+\n+module_init(xsc_init);\n+module_exit(xsc_fini);\n+\n+MODULE_LICENSE(\"GPL\");\n+MODULE_DESCRIPTION(\"Yunsilicon XSC PCI driver\");\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/mr.c b/drivers/net/ethernet/yunsilicon/xsc/pci/mr.c\nnew file mode 100644\nindex 0000000000000..5c68f0fd8db47\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/mr.c\n@@ -0,0 +1,212 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/kernel.h\u003e\n+#include \u003clinux/module.h\u003e\n+#include \u003clinux/slab.h\u003e\n+\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_cmd_api.h\"\n+#include \"hal/xsc_hal.h\"\n+#include \"mr.h\"\n+\n+int xsc_core_create_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr)\n+{\n+\tstruct xsc_create_mkey_mbox_in in;\n+\tstruct xsc_create_mkey_mbox_out out;\n+\tu8 key;\n+\tint err;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(\u0026out, 0, sizeof(out));\n+\n+\tspin_lock(\u0026xdev-\u003edev_res-\u003emkey_lock);\n+\tkey = 0x80 + xdev-\u003edev_res-\u003emkey_key++;\n+\tspin_unlock(\u0026xdev-\u003edev_res-\u003emkey_lock);\n+\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_MKEY);\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to create mkey, err = %d\\n\", err);\n+\t\treturn err;\n+\t}\n+\n+\tif (out.hdr.status) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to create mkey, status = %d\\n\", out.hdr.status);\n+\t\treturn xsc_cmd_status_to_err(\u0026out.hdr);\n+\t}\n+\n+\tmr-\u003ekey = xsc_idx_to_mkey(xdev, be32_to_cpu(out.mkey) \u0026 0xffffff) | key;\n+\tpci_dbg(xdev-\u003epdev, \"out 0x%x, key 0x%x, mkey 0x%x\\n\",\n+\t\tbe32_to_cpu(out.mkey), key, mr-\u003ekey);\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_create_mkey);\n+\n+int xsc_core_destroy_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr)\n+{\n+\tstruct xsc_destroy_mkey_mbox_in in;\n+\tstruct xsc_destroy_mkey_mbox_out out;\n+\tint err;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(\u0026out, 0, sizeof(out));\n+\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_MKEY);\n+\tin.mkey = cpu_to_be32(mr-\u003ekey);\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err)\n+\t\treturn err;\n+\n+\tif (out.hdr.status)\n+\t\treturn xsc_cmd_status_to_err(\u0026out.hdr);\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_destroy_mkey);\n+\n+static int xsc_set_mpt_via_cmdq(struct xsc_core_device *xdev,\n+\t\t\t\tstruct xsc_register_mr_mbox_in *in_cmd,\n+\t\t\t\tu32 *mtt_base)\n+{\n+\tstruct xsc_register_mr_request *req = \u0026in_cmd-\u003ereq;\n+\tstruct xsc_set_mpt_mbox_in *in;\n+\tstruct xsc_set_mpt_mbox_out out;\n+\tu64 mem_size;\n+\tint err;\n+\n+\tin = kzalloc_obj(*in, GFP_KERNEL);\n+\tif (!in)\n+\t\treturn -ENOMEM;\n+\n+\tmem_size = be64_to_cpu(req-\u003elen);\n+\tin-\u003empt_item.pdn = req-\u003epdn;\n+\tin-\u003empt_item.pa_num = req-\u003epa_num;\n+\tin-\u003empt_item.len = cpu_to_be32((u32)mem_size);\n+\tin-\u003empt_item.len_h = cpu_to_be32((u32)(mem_size \u003e\u003e 32));\n+\tin-\u003empt_item.mkey = req-\u003emkey;\n+\tin-\u003empt_item.is_gpu = req-\u003eis_gpu;\n+\tin-\u003empt_item.acc = req-\u003eacc;\n+\tin-\u003empt_item.page_mode = req-\u003epage_mode;\n+\tin-\u003empt_item.map_en = req-\u003emap_en;\n+\tin-\u003empt_item.va_base = req-\u003eva_base;\n+\tin-\u003ehdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_MPT);\n+\n+\tmemset(\u0026out, 0, sizeof(out));\n+\terr = xsc_cmd_exec(xdev, in, sizeof(*in), \u0026out, sizeof(out));\n+\tkfree(in);\n+\tif (err || out.hdr.status) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to set mpt, err = %d, status = %d\\n\",\n+\t\t\terr, out.hdr.status);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\t*mtt_base = be32_to_cpu(out.mtt_base);\n+\treturn 0;\n+}\n+\n+#define PA_NUM_PER_CMD 1024\n+\n+static int xsc_set_mtt_via_cmdq(struct xsc_core_device *xdev,\n+\t\t\t\tstruct xsc_register_mr_mbox_in *in_cmd,\n+\t\t\t\tu32 mtt_base)\n+{\n+\tstruct xsc_register_mr_request *req = \u0026in_cmd-\u003ereq;\n+\tstruct xsc_set_mtt_mbox_in *seg_in;\n+\tstruct xsc_set_mtt_mbox_out seg_out;\n+\tint tot_pg_num = be32_to_cpu(req-\u003epa_num);\n+\tint tot_seg_num, seg_pa_num, seg_idx;\n+\tint pa_idx_base = 0;\n+\tint in_len, err, i;\n+\n+\ttot_seg_num = (tot_pg_num \u0026 (PA_NUM_PER_CMD - 1)) ?\n+\t\t ((tot_pg_num \u003e\u003e 10) + 1) : (tot_pg_num \u003e\u003e 10);\n+\n+\tfor (seg_idx = 0; seg_idx \u003c tot_seg_num; seg_idx++) {\n+\t\tseg_pa_num = (seg_idx != tot_seg_num - 1) ? PA_NUM_PER_CMD :\n+\t\t\t (tot_pg_num - ((tot_seg_num - 1) \u003c\u003c 10));\n+\t\tin_len = (seg_pa_num \u003c\u003c 3) + sizeof(*seg_in);\n+\t\tseg_in = kzalloc(in_len, GFP_KERNEL);\n+\t\tif (!seg_in)\n+\t\t\treturn -ENOMEM;\n+\n+\t\tseg_in-\u003emtt_setting.mtt_base = cpu_to_be32(mtt_base);\n+\t\tseg_in-\u003emtt_setting.pa_num = cpu_to_be32(seg_pa_num);\n+\t\tfor (i = 0; i \u003c seg_pa_num; i++)\n+\t\t\tseg_in-\u003emtt_setting.pas[i] = req-\u003epas[pa_idx_base + i];\n+\t\tseg_in-\u003ehdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_MTT);\n+\n+\t\tmemset(\u0026seg_out, 0, sizeof(seg_out));\n+\t\tpci_dbg(xdev-\u003epdev,\n+\t\t\t\"set mtt seg %d, pa_num %d, pa_idx_base %d, tot_seg %d\\n\",\n+\t\t\tseg_idx, seg_pa_num, pa_idx_base, tot_seg_num);\n+\t\terr = xsc_cmd_exec(xdev, seg_in, in_len, \u0026seg_out, sizeof(seg_out));\n+\t\tkfree(seg_in);\n+\t\tif (err || seg_out.hdr.status) {\n+\t\t\tpci_err(xdev-\u003epdev, \"failed to set mtt seg %d, err = %d, status = %d\\n\",\n+\t\t\t\tseg_idx, err, seg_out.hdr.status);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\n+\t\tpa_idx_base += seg_pa_num;\n+\t\tmtt_base += seg_pa_num;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int xsc_dereg_mr_via_cmdq(struct xsc_core_device *xdev, u32 mkey)\n+{\n+\tstruct xsc_unregister_mr_mbox_in in;\n+\tstruct xsc_unregister_mr_mbox_out out;\n+\tint err;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(\u0026out, 0, sizeof(out));\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DEREG_MR);\n+\tin.mkey = cpu_to_be32(xsc_mkey_to_idx(xdev, mkey));\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err || out.hdr.status) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to dereg mr, err = %d, status = %d\\n\",\n+\t\t\terr, out.hdr.status);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+int xsc_core_register_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr,\n+\t\t\t struct xsc_register_mr_mbox_in *in, int inlen)\n+{\n+\tu32 mtt_base;\n+\tint err;\n+\n+\tin-\u003ehdr.opcode = cpu_to_be16(XSC_CMD_OP_REG_MR);\n+\n+\terr = xsc_set_mpt_via_cmdq(xdev, in, \u0026mtt_base);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to set mpt via cmdq\\n\");\n+\t\treturn err;\n+\t}\n+\n+\terr = xsc_set_mtt_via_cmdq(xdev, in, mtt_base);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to set mtt via cmdq\\n\");\n+\t\txsc_dereg_mr_via_cmdq(xdev, be32_to_cpu(in-\u003ereq.mkey));\n+\t\treturn err;\n+\t}\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_register_mr);\n+\n+int xsc_core_dereg_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr)\n+{\n+\treturn xsc_dereg_mr_via_cmdq(xdev, mr-\u003ekey);\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_dereg_mr);\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/mr.h b/drivers/net/ethernet/yunsilicon/xsc/pci/mr.h\nnew file mode 100644\nindex 0000000000000..edafecda63e12\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/mr.h\n@@ -0,0 +1,18 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_MR_H\n+#define __XSC_MR_H\n+\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_cmd.h\"\n+\n+int xsc_core_create_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr);\n+int xsc_core_destroy_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr);\n+int xsc_core_register_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr,\n+\t\t\t struct xsc_register_mr_mbox_in *in, int inlen);\n+int xsc_core_dereg_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr);\n+\n+#endif /* __XSC_MR_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c\nnew file mode 100644\nindex 0000000000000..c0f0b9d1f34f0\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c\n@@ -0,0 +1,498 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/irqdomain.h\u003e\n+#include \u003clinux/msi.h\u003e\n+#include \u003clinux/interrupt.h\u003e\n+#include \u003clinux/notifier.h\u003e\n+#include \u003clinux/module.h\u003e\n+\n+#include \"common/xsc_cmd_api.h\"\n+#include \"common/xsc_core.h\"\n+#include \"eq.h\"\n+#include \"pci_irq.h\"\n+#include \"hal/xsc_hal.h\"\n+\n+enum {\n+\tXSC_COMP_EQ_SIZE = 1024,\n+};\n+\n+static void xsc_free_irq(struct xsc_core_device *xdev, unsigned int vector)\n+{\n+\tunsigned int irqn = 0;\n+\n+\tirqn = pci_irq_vector(xdev-\u003epdev, vector);\n+\tdisable_irq(irqn);\n+\n+\tif (xsc_fw_is_available(xdev))\n+\t\tfree_irq(irqn, xdev);\n+}\n+\n+static int set_comp_irq_affinity_hint(struct xsc_core_device *xdev, int i)\n+{\n+\tstruct xsc_eq_table *table = \u0026xdev-\u003edev_res-\u003eeq_table;\n+\tstruct xsc_eq *eq = xsc_core_eq_get(xdev, i);\n+\tint vecidx = table-\u003eeq_vec_comp_base + i;\n+\tunsigned int irqn;\n+\tint ret;\n+\n+\tirqn = pci_irq_vector(xdev-\u003epdev, vecidx);\n+\tif (!zalloc_cpumask_var(\u0026eq-\u003emask, GFP_KERNEL)) {\n+\t\tpci_err(xdev-\u003epdev, \"zalloc_cpumask_var rx cpumask failed\");\n+\t\treturn -ENOMEM;\n+\t}\n+\n+\tif (!zalloc_cpumask_var(\u0026xdev-\u003exps_cpumask, GFP_KERNEL)) {\n+\t\tpci_err(xdev-\u003epdev, \"zalloc_cpumask_var tx cpumask failed\");\n+\t\treturn -ENOMEM;\n+\t}\n+\n+\tcpumask_set_cpu(cpumask_local_spread(i, xdev-\u003enuma_node),\n+\t\t\txdev-\u003exps_cpumask);\n+\tret = irq_set_affinity_hint(irqn, eq-\u003emask);\n+\n+\treturn ret;\n+}\n+\n+static void clear_comp_irq_affinity_hint(struct xsc_core_device *xdev, int i)\n+{\n+\tstruct xsc_eq_table *table = \u0026xdev-\u003edev_res-\u003eeq_table;\n+\tstruct xsc_eq *eq = xsc_core_eq_get(xdev, i);\n+\tint vecidx = table-\u003eeq_vec_comp_base + i;\n+\tint irqn;\n+\n+\tirqn = pci_irq_vector(xdev-\u003epdev, vecidx);\n+\tirq_set_affinity_hint(irqn, NULL);\n+\tfree_cpumask_var(eq-\u003emask);\n+}\n+\n+static int set_comp_irq_affinity_hints(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_eq_table *table = \u0026xdev-\u003edev_res-\u003eeq_table;\n+\tint nvec = table-\u003enum_comp_vectors;\n+\tint err;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c nvec; i++) {\n+\t\terr = set_comp_irq_affinity_hint(xdev, i);\n+\t\tif (err)\n+\t\t\tgoto err_out;\n+\t}\n+\n+\treturn 0;\n+\n+err_out:\n+\tfor (i--; i \u003e= 0; i--)\n+\t\tclear_comp_irq_affinity_hint(xdev, i);\n+\tfree_cpumask_var(xdev-\u003exps_cpumask);\n+\n+\treturn err;\n+}\n+\n+static void clear_comp_irq_affinity_hints(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_eq_table *table = \u0026xdev-\u003edev_res-\u003eeq_table;\n+\tint nvec = table-\u003enum_comp_vectors;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c nvec; i++)\n+\t\tclear_comp_irq_affinity_hint(xdev, i);\n+\tfree_cpumask_var(xdev-\u003exps_cpumask);\n+}\n+\n+static int xsc_alloc_irq_vectors(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_dev_resource *dev_res = xdev-\u003edev_res;\n+\tstruct xsc_eq_table *table = \u0026dev_res-\u003eeq_table;\n+\tint nvec = xdev-\u003ecaps.msix_num;\n+\tint nvec_base;\n+\tint err;\n+\n+\tnvec_base = XSC_EQ_VEC_COMP_BASE;\n+\tif (nvec \u003c= nvec_base) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to alloc irq vector(%d)\\n\", nvec);\n+\t\treturn -ENOMEM;\n+\t}\n+\n+\tdev_res-\u003eirq_info = kcalloc(nvec, sizeof(*dev_res-\u003eirq_info),\n+\t\t\t\t GFP_KERNEL);\n+\tif (!dev_res-\u003eirq_info)\n+\t\treturn -ENOMEM;\n+\n+\tnvec = pci_alloc_irq_vectors(xdev-\u003epdev, nvec_base + 1, nvec,\n+\t\t\t\t PCI_IRQ_MSIX);\n+\tif (nvec \u003c 0) {\n+\t\terr = nvec;\n+\t\tgoto err_free_irq_info;\n+\t}\n+\n+\ttable-\u003eeq_vec_comp_base = nvec_base;\n+\ttable-\u003enum_comp_vectors = nvec - nvec_base;\n+\txdev-\u003emsix_vec_base = xdev-\u003ecaps.msix_base;\n+\n+\treturn 0;\n+\n+err_free_irq_info:\n+\tkfree(dev_res-\u003eirq_info);\n+\treturn err;\n+}\n+\n+static void xsc_free_irq_vectors(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_dev_resource *dev_res = xdev-\u003edev_res;\n+\n+\tif (!xsc_fw_is_available(xdev))\n+\t\treturn;\n+\n+\tpci_free_irq_vectors(xdev-\u003epdev);\n+\tkfree(dev_res-\u003eirq_info);\n+}\n+\n+int xsc_core_vector2eqn(struct xsc_core_device *xdev, int vector, u32 *eqn,\n+\t\t\tunsigned int *irqn)\n+{\n+\tstruct xsc_eq_table *table = \u0026xdev-\u003edev_res-\u003eeq_table;\n+\tstruct xsc_eq *eq, *n;\n+\tint err = -ENOENT;\n+\n+\tif (!xdev-\u003ecaps.msix_enable)\n+\t\treturn 0;\n+\n+\tspin_lock(\u0026table-\u003elock);\n+\tlist_for_each_entry_safe(eq, n, \u0026table-\u003ecomp_eqs_list, list) {\n+\t\tif (eq-\u003eindex == vector) {\n+\t\t\t*eqn = eq-\u003eeqn;\n+\t\t\t*irqn = eq-\u003eirqn;\n+\t\t\terr = 0;\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\tspin_unlock(\u0026table-\u003elock);\n+\n+\treturn err;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_vector2eqn);\n+\n+int xsc_vector2eqn(struct xsc_core_device *xdev, int vector, int *eqn,\n+\t\t unsigned int *irqn)\n+{\n+\tu32 cqn = 0;\n+\tint err;\n+\n+\terr = xsc_core_vector2eqn(xdev, vector, \u0026cqn, irqn);\n+\tif (!err)\n+\t\t*eqn = (int)cqn;\n+\n+\treturn err;\n+}\n+EXPORT_SYMBOL_GPL(xsc_vector2eqn);\n+\n+static void free_comp_eqs(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_eq_table *table = \u0026xdev-\u003edev_res-\u003eeq_table;\n+\tstruct xsc_eq *eq, *n;\n+\n+\tspin_lock(\u0026table-\u003elock);\n+\tlist_for_each_entry_safe(eq, n, \u0026table-\u003ecomp_eqs_list, list) {\n+\t\tlist_del(\u0026eq-\u003elist);\n+\t\tspin_unlock(\u0026table-\u003elock);\n+\t\tif (xsc_destroy_unmap_eq(xdev, eq))\n+\t\t\tpci_err(xdev-\u003epdev, \"failed to destroy EQ 0x%x\\n\",\n+\t\t\t\teq-\u003eeqn);\n+\t\tkfree(eq);\n+\t\tspin_lock(\u0026table-\u003elock);\n+\t}\n+\tspin_unlock(\u0026table-\u003elock);\n+}\n+\n+static int alloc_comp_eqs(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_eq_table *table = \u0026xdev-\u003edev_res-\u003eeq_table;\n+\tchar name[XSC_MAX_IRQ_NAME];\n+\tstruct xsc_eq *eq;\n+\tint ncomp_vec;\n+\tu32 nent;\n+\tint err;\n+\tint i;\n+\n+\tINIT_LIST_HEAD(\u0026table-\u003ecomp_eqs_list);\n+\tncomp_vec = table-\u003enum_comp_vectors;\n+\tnent = XSC_COMP_EQ_SIZE;\n+\n+\tfor (i = 0; i \u003c ncomp_vec; i++) {\n+\t\teq = kzalloc_obj(*eq, GFP_KERNEL);\n+\t\tif (!eq) {\n+\t\t\terr = -ENOMEM;\n+\t\t\tgoto err_free;\n+\t\t}\n+\n+\t\tsnprintf(name, XSC_MAX_IRQ_NAME, \"xsc_comp%d\", i);\n+\t\terr = xsc_create_map_eq(xdev, eq,\n+\t\t\t\t\ti + table-\u003eeq_vec_comp_base,\n+\t\t\t\t\tnent, name);\n+\t\tif (err) {\n+\t\t\tkfree(eq);\n+\t\t\tgoto err_free;\n+\t\t}\n+\n+\t\teq-\u003eindex = i;\n+\t\tspin_lock(\u0026table-\u003elock);\n+\t\tlist_add_tail(\u0026eq-\u003elist, \u0026table-\u003ecomp_eqs_list);\n+\t\tspin_unlock(\u0026table-\u003elock);\n+\t}\n+\n+\treturn 0;\n+\n+err_free:\n+\tfree_comp_eqs(xdev);\n+\treturn err;\n+}\n+\n+static irqreturn_t xsc_cmd_handler(int irq, void *arg)\n+{\n+\tstruct xsc_core_device *xdev = (struct xsc_core_device *)arg;\n+\tint err;\n+\n+\tdisable_irq_nosync(xdev-\u003ecmd.irqn);\n+\terr = xsc_cmd_err_handler(xdev);\n+\tif (!err)\n+\t\txsc_cmd_resp_handler(xdev);\n+\tenable_irq(xdev-\u003ecmd.irqn);\n+\n+\treturn IRQ_HANDLED;\n+}\n+\n+static int xsc_request_irq_for_cmdq(struct xsc_core_device *xdev, u8 vecidx)\n+{\n+\tstruct xsc_dev_resource *dev_res = xdev-\u003edev_res;\n+\n+\txsc_set_cmdq_msix_vector(xdev, xdev-\u003emsix_vec_base + vecidx);\n+\n+\tsnprintf(dev_res-\u003eirq_info[vecidx].name, XSC_MAX_IRQ_NAME, \"%s@pci:%s\",\n+\t\t \"xsc_cmd\", pci_name(xdev-\u003epdev));\n+\txdev-\u003ecmd.irqn = pci_irq_vector(xdev-\u003epdev, vecidx);\n+\treturn request_irq(xdev-\u003ecmd.irqn, xsc_cmd_handler, 0,\n+\t\tdev_res-\u003eirq_info[vecidx].name, xdev);\n+}\n+\n+static void xsc_free_irq_for_cmdq(struct xsc_core_device *xdev)\n+{\n+\txsc_free_irq(xdev, XSC_VEC_CMD);\n+}\n+\n+/*\n+ * Handle an asynchronous event. This runs on the PCI layer's own workqueue,\n+ * so it outlives any auxiliary driver: the driver only ever registers a\n+ * callback here, and the memory this touches belongs to xsc_core_device\n+ * rather than to the eth or RDMA private area.\n+ */\n+static void xsc_event_work(struct work_struct *work)\n+{\n+\tstruct xsc_core_device *xdev = container_of(work, struct xsc_core_device,\n+\t\t\t\t\t\t event_work);\n+\tstruct xsc_event_query_type_mbox_out out;\n+\tstruct xsc_event_query_type_mbox_in in = {};\n+\tint err;\n+\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_EVENT_TYPE);\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err || out.hdr.status) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to query event type, err=%d, status=%d\\n\",\n+\t\t\terr, out.hdr.status);\n+\t\treturn;\n+\t}\n+\n+\tswitch (out.ctx.resp_cmd_type) {\n+\tcase XSC_CMD_EVENT_RESP_CHANGE_LINK:\n+\t\tif (xdev-\u003elink_event_handler)\n+\t\t\txdev-\u003elink_event_handler(xdev);\n+\t\tbreak;\n+\tcase XSC_CMD_EVENT_RESP_TEMP_WARN:\n+\t\tpci_warn(xdev-\u003epdev, \"[Minor]nic chip temperature high warning\\n\");\n+\t\tbreak;\n+\tcase XSC_CMD_EVENT_RESP_OVER_TEMP_PROTECTION:\n+\t\tpci_warn(xdev-\u003epdev, \"[Critical]nic chip was over-temperature\\n\");\n+\t\tbreak;\n+\tdefault:\n+\t\tpci_dbg(xdev-\u003epdev, \"unknown event cmdtype=0x%x\\n\",\n+\t\t\tout.ctx.resp_cmd_type);\n+\t\tbreak;\n+\t}\n+}\n+\n+static irqreturn_t xsc_event_handler(int irq, void *arg)\n+{\n+\tstruct xsc_core_device *xdev = (struct xsc_core_device *)arg;\n+\n+\tqueue_work(xdev-\u003eevent_wq, \u0026xdev-\u003eevent_work);\n+\n+\treturn IRQ_HANDLED;\n+}\n+\n+/*\n+ * Register the link event callback.\n+ *\n+ * The events are dispatched from the PCI layer's own workqueue, whose\n+ * contents are not the client's business (it also carries the temperature\n+ * warnings, which are logged here), so the client must not touch\n+ * xdev-\u003eevent_wq itself.\n+ */\n+void xsc_irq_event_register(struct xsc_core_device *xdev,\n+\t\t\t void (*handler)(void *xdev))\n+{\n+\txdev-\u003elink_event_handler = handler;\n+}\n+EXPORT_SYMBOL_GPL(xsc_irq_event_register);\n+\n+/*\n+ * Detach the callback and wait for an event that is already in flight to\n+ * finish with it, in the same way free_irq() both unregisters and waits.\n+ * Must be called before the caller frees whatever its handler dereferences.\n+ * Safe to call when no handler is registered.\n+ */\n+void xsc_irq_event_unregister(struct xsc_core_device *xdev)\n+{\n+\txdev-\u003elink_event_handler = NULL;\n+\n+\t/* Only exists once MSI-X, which requests the event IRQ, is up. */\n+\tif (xdev-\u003eevent_wq)\n+\t\tflush_workqueue(xdev-\u003eevent_wq);\n+}\n+EXPORT_SYMBOL_GPL(xsc_irq_event_unregister);\n+\n+static int xsc_request_irq_for_event(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_dev_resource *dev_res = xdev-\u003edev_res;\n+\n+\tsnprintf(dev_res-\u003eirq_info[XSC_VEC_CMD_EVENT].name,\n+\t\t XSC_MAX_IRQ_NAME, \"%s@pci:%s\",\n+\t\t \"xsc_eth_event\", pci_name(xdev-\u003epdev));\n+\treturn request_irq(pci_irq_vector(xdev-\u003epdev, XSC_VEC_CMD_EVENT),\n+\t\t\t xsc_event_handler, 0,\n+\t\t\t dev_res-\u003eirq_info[XSC_VEC_CMD_EVENT].name, xdev);\n+}\n+\n+static void xsc_free_irq_for_event(struct xsc_core_device *xdev)\n+{\n+\txsc_free_irq(xdev, XSC_VEC_CMD_EVENT);\n+}\n+\n+static int xsc_cmd_enable_msix(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_msix_table_info_mbox_out out;\n+\tstruct xsc_msix_table_info_mbox_in in;\n+\tint err;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(\u0026out, 0, sizeof(out));\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_ENABLE_MSIX);\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"xsc_cmd_exec enable msix failed %d\\n\",\n+\t\t\terr);\n+\t\treturn err;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+int xsc_irq_eq_create(struct xsc_core_device *xdev)\n+{\n+\tint err;\n+\n+\tif (xdev-\u003ecaps.msix_enable == 0)\n+\t\treturn 0;\n+\n+\terr = xsc_alloc_irq_vectors(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"enable msix failed, err=%d\\n\", err);\n+\t\tgoto err_out;\n+\t}\n+\n+\terr = xsc_start_eqs(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to start EQs, err=%d\\n\", err);\n+\t\tgoto err_free_irq_vectors;\n+\t}\n+\n+\terr = alloc_comp_eqs(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to alloc comp EQs, err=%d\\n\", err);\n+\t\tgoto err_stop_eqs;\n+\t}\n+\n+\terr = xsc_request_irq_for_cmdq(xdev, XSC_VEC_CMD);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to request irq for cmdq, err=%d\\n\",\n+\t\t\terr);\n+\t\tgoto err_free_comp_eqs;\n+\t}\n+\n+\txdev-\u003eevent_wq = create_singlethread_workqueue(\"xsc_event\");\n+\tif (!xdev-\u003eevent_wq) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to create event workqueue\\n\");\n+\t\terr = -ENOMEM;\n+\t\tgoto err_free_irq_cmdq;\n+\t}\n+\tINIT_WORK(\u0026xdev-\u003eevent_work, xsc_event_work);\n+\n+\terr = xsc_request_irq_for_event(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to request irq for event, err=%d\\n\",\n+\t\t\terr);\n+\t\tgoto err_destroy_event_wq;\n+\t}\n+\n+\terr = set_comp_irq_affinity_hints(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to alloc affinity hint cpumask, err=%d\\n\",\n+\t\t\terr);\n+\t\tgoto err_free_irq_evnt;\n+\t}\n+\n+\txsc_cmd_use_events(xdev);\n+\terr = xsc_cmd_enable_msix(xdev);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"xsc_cmd_enable_msix failed %d.\\n\", err);\n+\t\txsc_cmd_use_polling(xdev);\n+\t\tgoto err_free_irq_evnt;\n+\t}\n+\treturn 0;\n+\n+err_free_irq_evnt:\n+\txsc_free_irq_for_event(xdev);\n+err_destroy_event_wq:\n+\tdestroy_workqueue(xdev-\u003eevent_wq);\n+err_free_irq_cmdq:\n+\txsc_free_irq_for_cmdq(xdev);\n+err_free_comp_eqs:\n+\tfree_comp_eqs(xdev);\n+err_stop_eqs:\n+\txsc_stop_eqs(xdev);\n+err_free_irq_vectors:\n+\txsc_free_irq_vectors(xdev);\n+err_out:\n+\treturn err;\n+}\n+\n+void xsc_irq_eq_destroy(struct xsc_core_device *xdev)\n+{\n+\tif (xdev-\u003ecaps.msix_enable == 0)\n+\t\treturn;\n+\n+\txsc_stop_eqs(xdev);\n+\tclear_comp_irq_affinity_hints(xdev);\n+\tfree_comp_eqs(xdev);\n+\n+\t/* xsc_free_irq() disables and synchronises the event IRQ, so no new\n+\t * work can be queued once this returns.\n+\t */\n+\txsc_free_irq_for_event(xdev);\n+\tdestroy_workqueue(xdev-\u003eevent_wq);\n+\txsc_free_irq_for_cmdq(xdev);\n+\txsc_free_irq_vectors(xdev);\n+}\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.h b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.h\nnew file mode 100644\nindex 0000000000000..750251d1b90e6\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.h\n@@ -0,0 +1,18 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_PCI_IRQ_H\n+#define __XSC_PCI_IRQ_H\n+\n+#include \"common/xsc_core.h\"\n+\n+int xsc_irq_eq_create(struct xsc_core_device *xdev);\n+void xsc_irq_eq_destroy(struct xsc_core_device *xdev);\n+int xsc_core_vector2eqn(struct xsc_core_device *xdev, int vector, u32 *eqn,\n+\t\t\tunsigned int *irqn);\n+int xsc_vector2eqn(struct xsc_core_device *xdev, int vector, int *eqn,\n+\t\t unsigned int *irqn);\n+\n+#endif\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/pd.c b/drivers/net/ethernet/yunsilicon/xsc/pci/pd.c\nnew file mode 100644\nindex 0000000000000..7969dfefc2e13\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/pd.c\n@@ -0,0 +1,55 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/kernel.h\u003e\n+#include \u003clinux/module.h\u003e\n+\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_cmd_api.h\"\n+#include \"pd.h\"\n+\n+int xsc_core_alloc_pd(struct xsc_core_device *xdev, u32 *pdn)\n+{\n+\tstruct xsc_alloc_pd_mbox_in in;\n+\tstruct xsc_alloc_pd_mbox_out out;\n+\tint err;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(\u0026out, 0, sizeof(out));\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_ALLOC_PD);\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err)\n+\t\treturn err;\n+\n+\tif (out.hdr.status)\n+\t\treturn xsc_cmd_status_to_err(\u0026out.hdr);\n+\n+\t*pdn = be32_to_cpu(out.pdn) \u0026 0xffffff;\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_alloc_pd);\n+\n+int xsc_core_dealloc_pd(struct xsc_core_device *xdev, u32 pdn)\n+{\n+\tstruct xsc_dealloc_pd_mbox_in in;\n+\tstruct xsc_dealloc_pd_mbox_out out;\n+\tint err;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(\u0026out, 0, sizeof(out));\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DEALLOC_PD);\n+\tin.pdn = cpu_to_be32(pdn);\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err)\n+\t\treturn err;\n+\n+\tif (out.hdr.status)\n+\t\treturn xsc_cmd_status_to_err(\u0026out.hdr);\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_dealloc_pd);\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/pd.h b/drivers/net/ethernet/yunsilicon/xsc/pci/pd.h\nnew file mode 100644\nindex 0000000000000..d9a99baea3f5b\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/pd.h\n@@ -0,0 +1,14 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_PD_H\n+#define __XSC_PD_H\n+\n+#include \"common/xsc_core.h\"\n+\n+int xsc_core_alloc_pd(struct xsc_core_device *xdev, u32 *pdn);\n+int xsc_core_dealloc_pd(struct xsc_core_device *xdev, u32 pdn);\n+\n+#endif /* __XSC_PD_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/qp.c b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.c\nnew file mode 100644\nindex 0000000000000..8ea57a09f6ceb\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.c\n@@ -0,0 +1,356 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/gfp.h\u003e\n+#include \u003clinux/time.h\u003e\n+#include \u003clinux/export.h\u003e\n+#include \u003clinux/kthread.h\u003e\n+\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_cmd_api.h\"\n+#include \"qp.h\"\n+\n+int xsc_core_create_resource_common(struct xsc_core_device *xdev,\n+\t\t\t\t struct xsc_core_qp *qp)\n+{\n+\tstruct xsc_qp_table *table = \u0026xdev-\u003edev_res-\u003eqp_table;\n+\tint err;\n+\n+\tspin_lock_irq(\u0026table-\u003elock);\n+\terr = radix_tree_insert(\u0026table-\u003etree, qp-\u003eqpn, qp);\n+\tspin_unlock_irq(\u0026table-\u003elock);\n+\tif (err)\n+\t\treturn err;\n+\n+\tatomic_set(\u0026qp-\u003erefcount, 1);\n+\tinit_completion(\u0026qp-\u003efree);\n+\tqp-\u003epid = current-\u003epid;\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_create_resource_common);\n+\n+void xsc_core_destroy_resource_common(struct xsc_core_device *xdev,\n+\t\t\t\t struct xsc_core_qp *qp)\n+{\n+\tstruct xsc_qp_table *table = \u0026xdev-\u003edev_res-\u003eqp_table;\n+\tunsigned long flags;\n+\n+\tspin_lock_irqsave(\u0026table-\u003elock, flags);\n+\tradix_tree_delete(\u0026table-\u003etree, qp-\u003eqpn);\n+\tspin_unlock_irqrestore(\u0026table-\u003elock, flags);\n+\n+\tif (atomic_dec_and_test(\u0026qp-\u003erefcount))\n+\t\tcomplete(\u0026qp-\u003efree);\n+\twait_for_completion(\u0026qp-\u003efree);\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_destroy_resource_common);\n+\n+void xsc_qp_event(struct xsc_core_device *xdev, u32 qpn, int event_type)\n+{\n+\tstruct xsc_qp_table *table = \u0026xdev-\u003edev_res-\u003eqp_table;\n+\tstruct xsc_core_qp *qp;\n+\n+\tspin_lock(\u0026table-\u003elock);\n+\n+\tqp = radix_tree_lookup(\u0026table-\u003etree, qpn);\n+\tif (qp)\n+\t\tatomic_inc(\u0026qp-\u003erefcount);\n+\n+\tspin_unlock(\u0026table-\u003elock);\n+\n+\tif (!qp) {\n+\t\tpci_err(xdev-\u003epdev, \"Async event for bogus QP 0x%x\\n\", qpn);\n+\t\treturn;\n+\t}\n+\n+\tqp-\u003eevent(qp, event_type);\n+\n+\tif (atomic_dec_and_test(\u0026qp-\u003erefcount))\n+\t\tcomplete(\u0026qp-\u003efree);\n+}\n+\n+void xsc_init_qp_table(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_qp_table *table = \u0026xdev-\u003edev_res-\u003eqp_table;\n+\n+\tspin_lock_init(\u0026table-\u003elock);\n+\tINIT_RADIX_TREE(\u0026table-\u003etree, GFP_ATOMIC);\n+}\n+\n+int xsc_core_eth_create_qp(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_create_qp_mbox_in *in,\n+\t\t\t int insize, u32 *p_qpn)\n+{\n+\tstruct xsc_create_qp_mbox_out out;\n+\tint ret;\n+\n+\tin-\u003ehdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_QP);\n+\tret = xsc_cmd_exec(xdev, in, insize, \u0026out, sizeof(out));\n+\tif (ret) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to create sq, err=%d out.status=%u\\n\",\n+\t\t\tret, out.hdr.status);\n+\t\treturn ret;\n+\t}\n+\n+\t*p_qpn = be32_to_cpu(out.qpn) \u0026 0xffffff;\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_eth_create_qp);\n+\n+int xsc_core_eth_modify_qp_status(struct xsc_core_device *xdev,\n+\t\t\t\t u32 qpn, u16 status)\n+{\n+\tstruct xsc_modify_qp_mbox_out out;\n+\tstruct xsc_modify_qp_mbox_in in = {};\n+\tint ret = 0;\n+\n+\tin.hdr.opcode = cpu_to_be16(status);\n+\tin.qpn = cpu_to_be32(qpn);\n+\tin.no_need_wait = 1;\n+\n+\tret = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (ret) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to modify qp %u status=%u, err=%d out.status %u\\n\",\n+\t\t\tqpn, status, ret, out.hdr.status);\n+\t\treturn ret;\n+\t}\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_eth_modify_qp_status);\n+\n+int xsc_core_eth_destroy_qp(struct xsc_core_device *xdev, u32 qpn)\n+{\n+\tstruct xsc_destroy_qp_mbox_out out;\n+\tstruct xsc_destroy_qp_mbox_in in;\n+\tint err;\n+\n+\terr = xsc_core_eth_modify_qp_status(xdev, qpn, XSC_CMD_OP_2RST_QP);\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to set sq%d status=rst, err=%d\\n\",\n+\t\t\tqpn, err);\n+\t\treturn err;\n+\t}\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(\u0026out, 0, sizeof(out));\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_QP);\n+\tin.qpn = cpu_to_be32(qpn);\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to destroy sq%d, err=%d out.status=%u\\n\",\n+\t\t\tqpn, err, out.hdr.status);\n+\t\treturn err;\n+\t}\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_eth_destroy_qp);\n+\n+int xsc_core_eth_modify_raw_qp(struct xsc_core_device *xdev,\n+\t\t\t struct xsc_modify_raw_qp_mbox_in *in)\n+{\n+\tstruct xsc_modify_raw_qp_mbox_out out;\n+\tint err;\n+\n+\tin-\u003ehdr.opcode = cpu_to_be16(XSC_CMD_OP_MODIFY_RAW_QP);\n+\n+\terr = xsc_cmd_exec(xdev, in, sizeof(struct xsc_modify_raw_qp_mbox_in),\n+\t\t\t \u0026out, sizeof(struct xsc_modify_raw_qp_mbox_out));\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to modify sq, err=%d out.status=%u\\n\",\n+\t\t\terr, out.hdr.status);\n+\t\treturn err;\n+\t}\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_eth_modify_raw_qp);\n+\n+int xsc_core_eth_create_rss_qp_rqs(struct xsc_core_device *xdev,\n+\t\t\t\t struct xsc_create_multiqp_mbox_in *in,\n+\t\t\t\t int insize,\n+\t\t\t\t u32 *p_qpn_base)\n+{\n+\tstruct xsc_create_multiqp_mbox_out out;\n+\tint err;\n+\n+\tin-\u003ehdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_MULTI_QP);\n+\terr = xsc_cmd_exec(xdev, in, insize, \u0026out, sizeof(out));\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev,\n+\t\t\t\"failed to create rss rq, qp_num=%d, type=%d, err=%d out.status=%u\\n\",\n+\t\t\tin-\u003eqp_num, in-\u003eqp_type, err, out.hdr.status);\n+\t\treturn err;\n+\t}\n+\n+\t*p_qpn_base = be32_to_cpu(out.qpn_base) \u0026 0xffffff;\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_eth_create_rss_qp_rqs);\n+\n+int xsc_core_create_qp(struct xsc_core_device *xdev,\n+\t\t struct xsc_core_qp *qp,\n+\t\t struct xsc_create_qp_mbox_in *in,\n+\t\t int inlen)\n+{\n+\tstruct xsc_create_qp_mbox_out out;\n+\tstruct xsc_destroy_qp_mbox_in din;\n+\tstruct xsc_destroy_qp_mbox_out dout;\n+\tint err;\n+\n+\tmemset(\u0026out, 0, sizeof(out));\n+\tin-\u003ehdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_QP);\n+\n+\terr = xsc_cmd_exec(xdev, in, inlen, \u0026out, sizeof(out));\n+\tif (err)\n+\t\treturn err;\n+\n+\tif (out.hdr.status)\n+\t\treturn xsc_cmd_status_to_err(\u0026out.hdr);\n+\n+\tqp-\u003eqpn = be32_to_cpu(out.qpn) \u0026 0xffffff;\n+\n+\terr = xsc_core_create_resource_common(xdev, qp);\n+\tif (err)\n+\t\tgoto err_cmd;\n+\n+\tatomic_inc(\u0026xdev-\u003enum_qps);\n+\treturn 0;\n+\n+err_cmd:\n+\tmemset(\u0026din, 0, sizeof(din));\n+\tmemset(\u0026dout, 0, sizeof(dout));\n+\tdin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_QP);\n+\tdin.qpn = cpu_to_be32(qp-\u003eqpn);\n+\txsc_cmd_exec(xdev, \u0026din, sizeof(din), \u0026dout, sizeof(dout));\n+\n+\treturn err;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_create_qp);\n+\n+int xsc_core_destroy_qp(struct xsc_core_device *xdev,\n+\t\t\tstruct xsc_core_qp *qp)\n+{\n+\tstruct xsc_destroy_qp_mbox_in in;\n+\tstruct xsc_destroy_qp_mbox_out out;\n+\tint err;\n+\n+\txsc_core_destroy_resource_common(xdev, qp);\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(\u0026out, 0, sizeof(out));\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_QP);\n+\tin.qpn = cpu_to_be32(qp-\u003eqpn);\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err)\n+\t\treturn err;\n+\n+\tif (out.hdr.status)\n+\t\treturn xsc_cmd_status_to_err(\u0026out.hdr);\n+\n+\tatomic_dec(\u0026xdev-\u003enum_qps);\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_destroy_qp);\n+\n+int xsc_core_qp_modify(struct xsc_core_device *xdev, enum xsc_qp_state cur_state,\n+\t\t enum xsc_qp_state new_state,\n+\t\t struct xsc_modify_qp_mbox_in *in, int sqd_event,\n+\t\t struct xsc_core_qp *qp)\n+{\n+\tstatic const u16 optab[XSC_QP_NUM_STATE][XSC_QP_NUM_STATE] = {\n+\t\t[XSC_QP_STATE_RST] = {\n+\t\t\t[XSC_QP_STATE_RST]\t= XSC_CMD_OP_2RST_QP,\n+\t\t\t[XSC_QP_STATE_ERR]\t= XSC_CMD_OP_2ERR_QP,\n+\t\t\t[XSC_QP_STATE_INIT]\t= XSC_CMD_OP_RST2INIT_QP,\n+\t\t},\n+\t\t[XSC_QP_STATE_INIT] = {\n+\t\t\t[XSC_QP_STATE_RST]\t= XSC_CMD_OP_2RST_QP,\n+\t\t\t[XSC_QP_STATE_ERR]\t= XSC_CMD_OP_2ERR_QP,\n+\t\t\t[XSC_QP_STATE_INIT]\t= XSC_CMD_OP_INIT2INIT_QP,\n+\t\t\t[XSC_QP_STATE_RTR]\t= XSC_CMD_OP_INIT2RTR_QP,\n+\t\t},\n+\t\t[XSC_QP_STATE_RTR] = {\n+\t\t\t[XSC_QP_STATE_RST]\t= XSC_CMD_OP_2RST_QP,\n+\t\t\t[XSC_QP_STATE_ERR]\t= XSC_CMD_OP_2ERR_QP,\n+\t\t\t[XSC_QP_STATE_RTS]\t= XSC_CMD_OP_RTR2RTS_QP,\n+\t\t},\n+\t\t[XSC_QP_STATE_RTS] = {\n+\t\t\t[XSC_QP_STATE_RST]\t= XSC_CMD_OP_2RST_QP,\n+\t\t\t[XSC_QP_STATE_ERR]\t= XSC_CMD_OP_2ERR_QP,\n+\t\t\t[XSC_QP_STATE_RTS]\t= XSC_CMD_OP_RTS2RTS_QP,\n+\t\t\t[XSC_QP_STATE_SQD]\t= XSC_CMD_OP_RTS2SQD_QP,\n+\t\t},\n+\t\t[XSC_QP_STATE_SQD] = {\n+\t\t\t[XSC_QP_STATE_RST]\t= XSC_CMD_OP_2RST_QP,\n+\t\t\t[XSC_QP_STATE_ERR]\t= XSC_CMD_OP_2ERR_QP,\n+\t\t\t[XSC_QP_STATE_RTS]\t= XSC_CMD_OP_SQD2RTS_QP,\n+\t\t\t[XSC_QP_STATE_SQD]\t= XSC_CMD_OP_SQD2SQD_QP,\n+\t\t},\n+\t\t[XSC_QP_STATE_SQER] = {\n+\t\t\t[XSC_QP_STATE_RST]\t= XSC_CMD_OP_2RST_QP,\n+\t\t\t[XSC_QP_STATE_ERR]\t= XSC_CMD_OP_2ERR_QP,\n+\t\t\t[XSC_QP_STATE_RTS]\t= XSC_CMD_OP_SQERR2RTS_QP,\n+\t\t},\n+\t\t[XSC_QP_STATE_ERR] = {\n+\t\t\t[XSC_QP_STATE_RST]\t= XSC_CMD_OP_2RST_QP,\n+\t\t\t[XSC_QP_STATE_ERR]\t= XSC_CMD_OP_2ERR_QP,\n+\t\t}\n+\t};\n+\n+\tstruct xsc_modify_qp_mbox_out out;\n+\tu16 op;\n+\n+\tif (cur_state \u003e= XSC_QP_NUM_STATE || new_state \u003e= XSC_QP_NUM_STATE ||\n+\t !optab[cur_state][new_state])\n+\t\treturn -EINVAL;\n+\n+\tmemset(\u0026out, 0, sizeof(out));\n+\top = optab[cur_state][new_state];\n+\n+\tif (new_state == XSC_QP_STATE_RTR) {\n+\t\tin-\u003ectx.pcie_no = xdev-\u003epcie_no;\n+\t\tin-\u003ectx.func_id = cpu_to_be16(xdev-\u003eglb_func_id);\n+\t}\n+\n+\tin-\u003ehdr.opcode = cpu_to_be16(op);\n+\tin-\u003eqpn = cpu_to_be32(qp-\u003eqpn);\n+\tin-\u003eno_need_wait = 1;\n+\n+\tif (xsc_cmd_exec(xdev, in, sizeof(*in), \u0026out, sizeof(out)))\n+\t\treturn -ENOEXEC;\n+\n+\tif (out.hdr.status)\n+\t\treturn xsc_cmd_status_to_err(\u0026out.hdr);\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_qp_modify);\n+\n+int xsc_core_qp_query(struct xsc_core_device *xdev, struct xsc_core_qp *qp,\n+\t\t struct xsc_query_qp_mbox_out *out, int outlen)\n+{\n+\tstruct xsc_query_qp_mbox_in in;\n+\tint err;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(out, 0, outlen);\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_QP);\n+\tin.qpn = cpu_to_be32(qp-\u003eqpn);\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), out, outlen);\n+\tif (err)\n+\t\treturn err;\n+\n+\tif (out-\u003ehdr.status)\n+\t\treturn xsc_cmd_status_to_err(\u0026out-\u003ehdr);\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_qp_query);\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/qp.h b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.h\nnew file mode 100644\nindex 0000000000000..08af2008a22ff\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.h\n@@ -0,0 +1,28 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_QP_H\n+#define __XSC_QP_H\n+\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_hsi.h\"\n+\n+void xsc_init_qp_table(struct xsc_core_device *xdev);\n+void xsc_qp_event(struct xsc_core_device *xdev, u32 qpn, int event_type);\n+\n+int xsc_core_create_qp(struct xsc_core_device *xdev,\n+\t\t struct xsc_core_qp *qp,\n+\t\t struct xsc_create_qp_mbox_in *in,\n+\t\t int inlen);\n+int xsc_core_destroy_qp(struct xsc_core_device *xdev,\n+\t\t\tstruct xsc_core_qp *qp);\n+int xsc_core_qp_modify(struct xsc_core_device *xdev, enum xsc_qp_state cur_state,\n+\t\t enum xsc_qp_state new_state,\n+\t\t struct xsc_modify_qp_mbox_in *in, int sqd_event,\n+\t\t struct xsc_core_qp *qp);\n+int xsc_core_qp_query(struct xsc_core_device *xdev, struct xsc_core_qp *qp,\n+\t\t struct xsc_query_qp_mbox_out *out, int outlen);\n+\n+#endif /* __XSC_QP_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/srq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/srq.c\nnew file mode 100644\nindex 0000000000000..c2f9c1a88259f\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/srq.c\n@@ -0,0 +1,92 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \u003clinux/kernel.h\u003e\n+#include \u003clinux/module.h\u003e\n+#include \u003clinux/slab.h\u003e\n+#include \u003clinux/vmalloc.h\u003e\n+\n+#include \"common/xsc_core.h\"\n+#include \"common/srq.h\"\n+#include \"srq.h\"\n+\n+int xsc_create_srq_res(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_dev_resource *dev_res = xdev-\u003edev_res;\n+\tu32 max_srq_num = XSC_MAX_SRQ_NUM;\n+\n+\tdev_res-\u003esrq_tbl = bitmap_zalloc(max_srq_num, GFP_KERNEL);\n+\tif (!dev_res-\u003esrq_tbl)\n+\t\treturn -ENOMEM;\n+\n+\tdev_res-\u003esrq_cache_wr = vzalloc(max_srq_num *\n+\t\t\t\t\tsizeof(*dev_res-\u003esrq_cache_wr));\n+\tif (!dev_res-\u003esrq_cache_wr) {\n+\t\tbitmap_free(dev_res-\u003esrq_tbl);\n+\t\tdev_res-\u003esrq_tbl = NULL;\n+\t\treturn -ENOMEM;\n+\t}\n+\n+\tspin_lock_init(\u0026dev_res-\u003esrq_lock);\n+\tdev_res-\u003emax_srq_num = max_srq_num;\n+\n+\treturn 0;\n+}\n+\n+void xsc_destroy_srq_res(struct xsc_core_device *xdev)\n+{\n+\tstruct xsc_dev_resource *dev_res = xdev-\u003edev_res;\n+\n+\tvfree(dev_res-\u003esrq_cache_wr);\n+\tdev_res-\u003esrq_cache_wr = NULL;\n+\tbitmap_free(dev_res-\u003esrq_tbl);\n+\tdev_res-\u003esrq_tbl = NULL;\n+\tdev_res-\u003emax_srq_num = 0;\n+}\n+\n+int xsc_core_create_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq)\n+{\n+\tstruct xsc_dev_resource *dev_res = xdev-\u003edev_res;\n+\tu32 srqn;\n+\n+\tspin_lock(\u0026dev_res-\u003esrq_lock);\n+\tsrqn = find_first_zero_bit(dev_res-\u003esrq_tbl, dev_res-\u003emax_srq_num);\n+\tif (srqn == dev_res-\u003emax_srq_num) {\n+\t\tspin_unlock(\u0026dev_res-\u003esrq_lock);\n+\t\tpci_err(xdev-\u003epdev, \"no free srq number\\n\");\n+\t\treturn -ENOMEM;\n+\t}\n+\tset_bit(srqn, dev_res-\u003esrq_tbl);\n+\tspin_unlock(\u0026dev_res-\u003esrq_lock);\n+\n+\tsrq-\u003esrqn = srqn;\n+\t/* array base: callers index it by srqn */\n+\tsrq-\u003ecache_wr = dev_res-\u003esrq_cache_wr;\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_create_srq);\n+\n+int xsc_core_destroy_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq)\n+{\n+\tstruct xsc_dev_resource *dev_res = xdev-\u003edev_res;\n+\n+\tif (srq-\u003esrqn \u003e= dev_res-\u003emax_srq_num) {\n+\t\tpci_err(xdev-\u003epdev, \"invalid srqn %u\\n\", srq-\u003esrqn);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tspin_lock(\u0026dev_res-\u003esrq_lock);\n+\tclear_bit(srq-\u003esrqn, dev_res-\u003esrq_tbl);\n+\tspin_unlock(\u0026dev_res-\u003esrq_lock);\n+\n+\t/*\n+\t * The caller still owns the cached recv WRs in cache_wr[srqn] and\n+\t * releases them right after this call, so the slot must be left\n+\t * untouched here.\n+\t */\n+\treturn 0;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_destroy_srq);\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/srq.h b/drivers/net/ethernet/yunsilicon/xsc/pci/srq.h\nnew file mode 100644\nindex 0000000000000..a70faf9e70ef8\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/srq.h\n@@ -0,0 +1,16 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_SRQ_H\n+#define __XSC_SRQ_H\n+\n+#include \"common/xsc_core.h\"\n+#include \"common/srq.h\"\n+\n+/* return 0 on success, negative errno otherwise */\n+int xsc_create_srq_res(struct xsc_core_device *xdev);\n+void xsc_destroy_srq_res(struct xsc_core_device *xdev);\n+\n+#endif /* __XSC_SRQ_H */\ndiff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/vport.c b/drivers/net/ethernet/yunsilicon/xsc/pci/vport.c\nnew file mode 100644\nindex 0000000000000..5118ff36b8f38\n--- /dev/null\n+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/vport.c\n@@ -0,0 +1,34 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/* Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#include \"common/xsc_core.h\"\n+#include \"common/xsc_cmd_api.h\"\n+\n+u8 xsc_core_query_vport_state(struct xsc_core_device *xdev, u16 vport)\n+{\n+\tstruct xsc_query_vport_state_out out;\n+\tstruct xsc_query_vport_state_in in;\n+\tu32 val = 0;\n+\tint err;\n+\n+\tmemset(\u0026in, 0, sizeof(in));\n+\tmemset(\u0026out, 0, sizeof(out));\n+\n+\tin.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_VPORT_STATE);\n+\tval |= FIELD_PREP(XSC_QUERY_VPORT_VPORT_NUM_MASK, vport);\n+\tif (vport)\n+\t\tval |= XSC_QUERY_VPORT_OTHER_VPORT;\n+\tin.data0 = cpu_to_be32(val);\n+\n+\terr = xsc_cmd_exec(xdev, \u0026in, sizeof(in), \u0026out, sizeof(out));\n+\tif (err) {\n+\t\tpci_err(xdev-\u003epdev, \"failed to query vport state, err=%d, status=%d\\n\",\n+\t\t\terr, out.hdr.status);\n+\t\treturn err;\n+\t}\n+\n+\treturn out.state;\n+}\n+EXPORT_SYMBOL_GPL(xsc_core_query_vport_state);\ndiff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h\nindex 21f86cc7bb1f1..53fc990578200 100644\n--- a/include/uapi/rdma/ib_user_ioctl_verbs.h\n+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h\n@@ -257,6 +257,7 @@ enum rdma_driver_id {\n \tRDMA_DRIVER_ERDMA,\n \tRDMA_DRIVER_MANA,\n \tRDMA_DRIVER_IONIC,\n+\tRDMA_DRIVER_XSC,\n };\n \n enum ib_uverbs_gid_type {\ndiff --git a/include/uapi/rdma/xsc-abi.h b/include/uapi/rdma/xsc-abi.h\nnew file mode 100644\nindex 0000000000000..c4d0db30e43c4\n--- /dev/null\n+++ b/include/uapi/rdma/xsc-abi.h\n@@ -0,0 +1,295 @@\n+/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */\n+/*\n+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.\n+ * All rights reserved.\n+ */\n+\n+#ifndef __XSC_ABI_H__\n+#define __XSC_ABI_H__\n+\n+#include \u003clinux/types.h\u003e\n+#include \u003clinux/if_ether.h\u003e\t/* For ETH_ALEN. */\n+\n+/* Increment this value if any changes that break userspace ABI\n+ * compatibility are made.\n+ */\n+#define XSC_IB_UVERBS_ABI_VERSION\t1\n+\n+/* Make sure that all structs defined in this file remain laid out so\n+ * that they pack the same way on 32-bit and 64-bit architectures (to\n+ * avoid incompatibility between 32-bit userspace and 64-bit kernels).\n+ * In particular do not use pointer types -- pass pointers in __u64\n+ * instead.\n+ */\n+\n+enum {\n+\tXSC_QP_FLAG_SIGNATURE\t\t= 1 \u003c\u003c 0,\n+\tXSC_QP_FLAG_SCATTER_CQE\t= 1 \u003c\u003c 1,\n+\tXSC_QP_FLAG_TUNNEL_OFFLOADS\t= 1 \u003c\u003c 2,\n+\tXSC_QP_FLAG_BFREG_INDEX\t= 1 \u003c\u003c 3,\n+\tXSC_QP_FLAG_TYPE_DCT\t\t= 1 \u003c\u003c 4,\n+\tXSC_QP_FLAG_TYPE_DCI\t\t= 1 \u003c\u003c 5,\n+\tXSC_QP_FLAG_TIR_ALLOW_SELF_LB_UC = 1 \u003c\u003c 6,\n+\tXSC_QP_FLAG_TIR_ALLOW_SELF_LB_MC = 1 \u003c\u003c 7,\n+\tXSC_QP_FLAG_ALLOW_SCATTER_CQE\t= 1 \u003c\u003c 8,\n+\tXSC_QP_FLAG_RAWPACKET_TSO = 1 \u003c\u003c 9,\n+\tXSC_QP_FLAG_RAWPACKET_TX\t= 1 \u003c\u003c 10,\n+\tXSC_QP_FLAG_RAWPACKET_SNIFFER\t= 1 \u003c\u003c 11,\n+\tXSC_QP_FLAG_WORC\t\t= 1 \u003c\u003c 12,\n+};\n+\n+#define ALLOC_UCONTEXT_VER_V1 1\n+#define ALLOC_UCONTEXT_VER_V2 2\n+\n+#define CURRENT_UCONTEXT_ABI_VER\tALLOC_UCONTEXT_VER_V2\n+\n+struct xsc_ib_alloc_ucontext_req {\n+\t__u32\tabi_ver;\n+\t__u32\trsvd1;\n+};\n+\n+enum xsc_user_cmds_supp_uhw {\n+\tXSC_USER_CMDS_SUPP_UHW_QUERY_DEVICE = 1 \u003c\u003c 0,\n+\tXSC_USER_CMDS_SUPP_UHW_CREATE_AH = 1 \u003c\u003c 1,\n+};\n+\n+enum {\n+\tXSC_HW_FEATURE_FLAG_SUPPORT_CQE64 = 1 \u003c\u003c 0,\n+};\n+\n+struct xsc_ib_alloc_ucontext_resp {\n+\t__u32\tqp_tab_size;\n+\t__u32\tcache_line_size;\n+\t__u16\tmax_sq_desc_sz;\n+\t__u16\tmax_rq_desc_sz;\n+\t__u32\tmax_send_wqebb;\n+\t__u32\tmax_recv_wr;\n+\t__u16\tnum_ports;\n+\t__u16\tdevice_id;\n+\t__u64\tqpm_tx_db;\n+\t__u64\tqpm_rx_db;\n+\t__u64\tcqm_next_cid_reg;\n+\t__u64\tcqm_armdb;\n+\t__u32\tsend_ds_num;\n+\t__u32\trecv_ds_num;\n+\t__u32 cmds_supp_uhw;\n+\t__u32\tpad;\n+\t__u32\tmultidb_num;\n+\t__u32\ttx_multidb_base;\n+\t__u32\trdma_proto_mode;\n+\t__u32\thw_feature_flag;\n+\t__u32\treserved[32];\n+};\n+\n+struct xsc_ib_create_qp {\n+\t__u64\tbuf_addr;\n+\t__u64\tdb_addr;\n+\t__u32\tsq_wqe_count;\n+\t__u32\trq_wqe_count;\n+\t__u32\trq_wqe_shift;\n+\t__u32\tflags;\n+\n+\tint\tdmabuf_fd;\n+\tsize_t\tdmabuf_sz;\n+};\n+\n+struct xsc_ib_create_qp_resp {\n+\t__u32\tuuar_index;\n+\t__u32 reserved;\n+};\n+\n+struct xsc_ib_create_srq {\n+\t__u64\tbuf_addr;\n+\t__u64\tdb_addr;\n+\t__u32\tflags;\n+\t__u32\treserved0;\n+\t__u32\tuidx;\n+\t__u32\treserved1;\n+};\n+\n+struct xsc_ib_create_srq_resp {\n+\t__u32\tsrqn;\n+\t__u32\treserved;\n+};\n+\n+struct xsc_ib_create_cq {\n+\t__u64 buf_addr;\n+\t__u64 db_addr;\n+\t__u32\tcqe_size;\n+\n+\tint\tdmabuf_fd;\n+\tsize_t\tdmabuf_sz;\n+};\n+\n+struct xsc_ib_create_cq_resp {\n+\t__u32\tcqn;\n+\t__u32\treserved;\n+};\n+\n+struct xsc_ib_create_ah_resp {\n+\t__u32\tresponse_length;\n+\t__u8\tdmac[ETH_ALEN];\n+\t__u8\treserved[6];\n+};\n+\n+struct xsc_ib_alloc_pd_resp {\n+\t__u32\tpdn;\n+};\n+\n+struct xsc_ib_tso_caps {\n+\t__u32 max_tso; /* Maximum tso payload size in bytes */\n+\n+\t/* Corresponding bit will be set if qp type from\n+\t * 'enum ib_qp_type' is supported, e.g.\n+\t * supported_qpts |= 1 \u003c\u003c IB_QPT_UD\n+\t */\n+\t__u32 supported_qpts;\n+};\n+\n+/* RX Hash function flags */\n+enum xsc_rx_hash_function_flags {\n+\tXSC_RX_HASH_FUNC_TOEPLITZ\t= 1 \u003c\u003c 0,\n+};\n+\n+enum xsc_rdma_link_speed {\n+\tXSC_RDMA_LINK_SPEED_2_5GB\t= 1 \u003c\u003c 0,\n+\tXSC_RDMA_LINK_SPEED_5GB\t\t= 1 \u003c\u003c 1,\n+\tXSC_RDMA_LINK_SPEED_10GB\t= 1 \u003c\u003c 3,\n+\tXSC_RDMA_LINK_SPEED_14GB\t= 1 \u003c\u003c 4,\n+\tXSC_RDMA_LINK_SPEED_25GB\t= 1 \u003c\u003c 5,\n+\tXSC_RDMA_LINK_SPEED_50GB\t= 1 \u003c\u003c 6,\n+\tXSC_RDMA_LINK_SPEED_100GB\t= 1 \u003c\u003c 7,\n+};\n+\n+enum xsc_rdma_phys_state {\n+\tXSC_RDMA_PHY_STATE_SLEEP\t= 1,\n+\tXSC_RDMA_PHY_STATE_POLLING,\n+\tXSC_RDMA_PHY_STATE_DISABLED,\n+\tXSC_RDMA_PHY_STATE_PORT_CONFIGURATION_TRAINNING,\n+\tXSC_RDMA_PHY_STATE_LINK_UP,\n+\tXSC_RDMA_PHY_STATE_LINK_ERROR_RECOVERY,\n+\tXSC_RDMA_PHY_STATE_PHY_TEST,\n+};\n+\n+/*\n+ * RX Hash flags, these flags allows to set which incoming packet's field should\n+ * participates in RX Hash. Each flag represent certain packet's field,\n+ * when the flag is set the field that is represented by the flag will\n+ * participate in RX Hash calculation.\n+ * Note: *IPV4 and *IPV6 flags can't be enabled together on the same QP\n+ * and *TCP and *UDP flags can't be enabled together on the same QP.\n+ */\n+enum xsc_rx_hash_fields {\n+\tXSC_RX_HASH_SRC_IPV4\t= 1 \u003c\u003c 0,\n+\tXSC_RX_HASH_DST_IPV4\t= 1 \u003c\u003c 1,\n+\tXSC_RX_HASH_SRC_IPV6\t= 1 \u003c\u003c 2,\n+\tXSC_RX_HASH_DST_IPV6\t= 1 \u003c\u003c 3,\n+\tXSC_RX_HASH_SRC_PORT_TCP\t= 1 \u003c\u003c 4,\n+\tXSC_RX_HASH_DST_PORT_TCP\t= 1 \u003c\u003c 5,\n+\tXSC_RX_HASH_SRC_PORT_UDP\t= 1 \u003c\u003c 6,\n+\tXSC_RX_HASH_DST_PORT_UDP\t= 1 \u003c\u003c 7,\n+\tXSC_RX_HASH_IPSEC_SPI\t\t= 1 \u003c\u003c 8,\n+\t/* Save bits for future fields */\n+\tXSC_RX_HASH_INNER\t\t= (1UL \u003c\u003c 31),\n+};\n+\n+struct xsc_ib_rss_caps {\n+\t__aligned_u64 rx_hash_fields_mask; /* enum xsc_rx_hash_fields */\n+\t__u8 rx_hash_function; /* enum xsc_rx_hash_function_flags */\n+\t__u8 reserved[7];\n+};\n+\n+enum xsc_ib_cqe_comp_res_format {\n+\tXSC_IB_CQE_RES_FORMAT_HASH\t= 1 \u003c\u003c 0,\n+\tXSC_IB_CQE_RES_FORMAT_CSUM\t= 1 \u003c\u003c 1,\n+\tXSC_IB_CQE_RES_FORMAT_CSUM_STRIDX = 1 \u003c\u003c 2,\n+};\n+\n+struct xsc_ib_cqe_comp_caps {\n+\t__u32 max_num;\n+\t__u32 supported_format; /* enum xsc_ib_cqe_comp_res_format */\n+};\n+\n+enum xsc_ib_packet_pacing_cap_flags {\n+\tXSC_IB_PP_SUPPORT_BURST\t= 1 \u003c\u003c 0,\n+};\n+\n+struct xsc_packet_pacing_caps {\n+\t__u32 qp_rate_limit_min;\n+\t__u32 qp_rate_limit_max; /* In kpbs */\n+\n+\t/* Corresponding bit will be set if qp type from\n+\t * 'enum ib_qp_type' is supported, e.g.\n+\t * supported_qpts |= 1 \u003c\u003c IB_QPT_RAW_PACKET\n+\t */\n+\t__u32 supported_qpts;\n+\t__u8 cap_flags; /* enum xsc_ib_packet_pacing_cap_flags */\n+\t__u8 reserved[3];\n+};\n+\n+enum xsc_ib_mpw_caps {\n+\tMPW_RESERVED\t\t= 1 \u003c\u003c 0,\n+\tXSC_IB_ALLOW_MPW\t= 1 \u003c\u003c 1,\n+\tXSC_IB_SUPPORT_EMPW\t= 1 \u003c\u003c 2,\n+};\n+\n+enum xsc_ib_sw_parsing_offloads {\n+\tXSC_IB_SW_PARSING = 1 \u003c\u003c 0,\n+\tXSC_IB_SW_PARSING_CSUM = 1 \u003c\u003c 1,\n+\tXSC_IB_SW_PARSING_LSO = 1 \u003c\u003c 2,\n+};\n+\n+struct xsc_ib_sw_parsing_caps {\n+\t__u32 sw_parsing_offloads; /* enum xsc_ib_sw_parsing_offloads */\n+\n+\t/* Corresponding bit will be set if qp type from\n+\t * 'enum ib_qp_type' is supported, e.g.\n+\t * supported_qpts |= 1 \u003c\u003c IB_QPT_RAW_PACKET\n+\t */\n+\t__u32 supported_qpts;\n+};\n+\n+struct xsc_ib_striding_rq_caps {\n+\t__u32 min_single_stride_log_num_of_bytes;\n+\t__u32 max_single_stride_log_num_of_bytes;\n+\t__u32 min_single_wqe_log_num_of_strides;\n+\t__u32 max_single_wqe_log_num_of_strides;\n+\n+\t/* Corresponding bit will be set if qp type from\n+\t * 'enum ib_qp_type' is supported, e.g.\n+\t * supported_qpts |= 1 \u003c\u003c IB_QPT_RAW_PACKET\n+\t */\n+\t__u32 supported_qpts;\n+\t__u32 reserved;\n+};\n+\n+enum xsc_ib_query_dev_resp_flags {\n+\t/* Support 128B CQE compression */\n+\tXSC_IB_QUERY_DEV_RESP_FLAGS_CQE_128B_COMP = 1 \u003c\u003c 0,\n+\tXSC_IB_QUERY_DEV_RESP_FLAGS_CQE_128B_PAD = 1 \u003c\u003c 1,\n+};\n+\n+enum xsc_ib_tunnel_offloads {\n+\tXSC_IB_TUNNELED_OFFLOADS_VXLAN = 1 \u003c\u003c 0,\n+\tXSC_IB_TUNNELED_OFFLOADS_GRE = 1 \u003c\u003c 1,\n+\tXSC_IB_TUNNELED_OFFLOADS_GENEVE = 1 \u003c\u003c 2,\n+\tXSC_IB_TUNNELED_OFFLOADS_MPLS_GRE = 1 \u003c\u003c 3,\n+\tXSC_IB_TUNNELED_OFFLOADS_MPLS_UDP = 1 \u003c\u003c 4,\n+};\n+\n+struct xsc_ib_query_device_resp {\n+\t__u32\tcomp_mask;\n+\t__u32\tresponse_length;\n+\tstruct\txsc_ib_tso_caps tso_caps;\n+\tstruct\txsc_ib_rss_caps rss_caps;\n+\tstruct\txsc_ib_cqe_comp_caps cqe_comp_caps;\n+\tstruct\txsc_packet_pacing_caps packet_pacing_caps;\n+\t__u32\txsc_ib_support_multi_pkt_send_wqes;\n+\t__u32\tflags; /* Use enum xsc_ib_query_dev_resp_flags */\n+\tstruct xsc_ib_sw_parsing_caps sw_parsing_caps;\n+\tstruct xsc_ib_striding_rq_caps striding_rq_caps;\n+\t__u32\ttunnel_offloads_caps; /* enum xsc_ib_tunnel_offloads */\n+\t__u32\treserved;\n+};\n+\n+#endif /* __XSC_ABI_H__ */\n"
} |
| 2/1 | 2026/10/12 12:01 | agent | patch-evaluator |
0mModel:core Args: null Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"Reasoning": "The patch series introduces support for Yunsilicon XSC PCIe devices, including the core PCI driver, ethernet driver, and RDMA (InfiniBand) driver. These drivers bind strictly to proprietary physical ASIC/PCIe hardware (Vendor ID 0x1f67) and associated auxiliary devices that are not emulated in standard virtualized QEMU or cloud environments (GCE). Because the modified code is structurally unreachable without specialized physical hardware, it is not worth fuzzing.",
"WorthFuzzing": false
}Instruction: You are an expert Linux kernel maintainer and security engineer.
Your job is to review a provided patch series and evaluate whether it warrants fuzzing with syzkaller.
IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in
your workspace. Do NOT rely on internal assumptions. You must actively use your code access
tools to inspect the actual source code, callers, and surrounding context.
================================================================================
1. CORE TRIAGE PHILOSOPHY
================================================================================
The goal of patch fuzzing is to discover crashes, regressions, exposed latent bugs,
and newly triggered assertions introduced by the patch series.
- REACHABILITY IS THE PRIMARY GATE:
Fuzzing can only discover bugs in code that can actually execute in standard virtualized
environments (GCE or QEMU, utilizing software-emulated devices like USB gadgets, netdev, tun/tap).
If the modified code is structurally unreachable (see Section 2), it MUST NOT be fuzzed,
regardless of whether it adds assertions or complex logic.
- DO NOT BLINDLY TRUST "NO FUNCTIONAL CHANGE" (NFCI) OR "REFACTORING" CLAIMS:
Patch authors routinely label changes as "cleanups", "refactorings", or state
"No functional change intended". Do NOT take these claims at face value.
Code refactorings that rearrange logic, introduce helper functions, or alter state management
in core subsystems frequently introduce subtle semantic shifts or uncover latent kernel bugs.
If reachable executable code is modified or refactored, it MUST be fuzzed.
- NEW OR MODIFIED ASSERTIONS IN REACHABLE CODE MUST BE FUZZED:
When a patch introduces or modifies runtime checks or assertions (e.g., WARN_ON*, VM_WARN_ON*,
BUG_ON*, lockdep_assert*) in reachable code paths, it enforces new or stricter invariants.
Even if the author believes the invariant always holds, fuzzing is essential to verify whether
an unusual sequence of operations can violate it.
================================================================================
2. WHEN TO RETURN WorthFuzzing=false (NEGATIVE CRITERIA)
================================================================================
Return WorthFuzzing=false ONLY IF all modified code falls strictly into one or more of these categories:
- Non-kernel and non-executable changes:
* Modifications to Documentation/, comments, or spelling fixes.
* User-space directories, self-tests, samples, or scripts (e.g., tools/, samples/, scripts/, usr/)
that do not affect the compiled kernel image (vmlinux) or kernel modules.
* Purely decorative logging (e.g., message strings in pr_err, printk, dev_info) or tracepoints
that do not alter control flow or data structures.
* Build system or Kconfig changes that do not alter compiled C logic.
- Structurally unreachable hardware:
* Vendor-specific PCIe switches, SmartNICs, or GPU drivers (e.g., mlxsw, pds_core, qed,
ionic, amdgpu) requiring physical ASIC/PCIe cards not emulated in standard QEMU.
- Unreachable execution paths:
* Driver teardown callbacks (.remove, .shutdown, pci_unregister_driver) executed only during
physical PCI hot-unplug or manual sysfs driver unbinding.
* Code paths exclusive to architectures other than the target architecture.
================================================================================
3. WHEN TO RETURN WorthFuzzing=true (POSITIVE CRITERIA)
================================================================================
Return WorthFuzzing=true whenever the patch touches reachable executable code, including:
- Core Subsystems:
* Any logic modifications in memory management (mm/), synchronization/locking (kernel/locking/),
BPF, scheduler, core networking, VFS, or syscall handling.
- Refactorings and Code Cleanups:
* Any restructuring of reachable data structures, helper abstractions, or algorithm flows.
- Runtime Assertions and Defensive Checks:
* Any introduction or alteration of assertions (WARN_ON*, VM_WARN_ON*, BUG_ON*, etc.) in reachable paths.
- Reachable Drivers and Protocols:
* Drivers accessible via virtual buses (virtio, USB gadget, loopback, netlink, binder, sockets, etc.).
================================================================================
4. EXTRACTING FocusSymbols (PREVENTING DILUTION)
================================================================================
When WorthFuzzing=true, you must extract specific kernel functions into FocusSymbols to guide the fuzzer:
- AVOID UBIQUITOUS LIFECYCLE HOT-PATHS:
Do NOT list generic, ubiquitous functions called by almost every program in the corpus
(including, but not limited to: general memory allocators and deallocators, page fault
and trap handlers, or core synchronization primitives; this is not an exhaustive list).
Listing ubiquitous functions causes the fuzzer to classify thousands of unrelated tests as "focused",
which severely dilutes fuzzing effort away from the actual changes.
- TARGET SPECIFIC FEATURE LOGIC AND ENTRYPOINTS:
List functions that specifically implement the logic being added or altered, or direct API entrypoints
for the subsystem feature under review.
- HANDLING STATIC INLINE FUNCTIONS IN HEADERS (.h):
Compiler-inlined static functions (such as static inlines in mm/*.h or include/linux/*.h) lack
distinct symbol addresses in vmlinux and cannot be targeted directly by symbol coverage filters.
If the changes are primarily in static inline helpers, identify non-static, feature-specific caller
functions in .c files that exercise them (avoiding ubiquitous lifecycle wrappers).
================================================================================
5. IDENTIFYING EnableConfigs
================================================================================
Identify any specific CONFIG_ options required to properly compile and reach the modified code:
- Inspect Kconfig files and #ifdef guards; do not make assumptions.
- Check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed.
- Strip any 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4").
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Target architecture: amd64
For your convenience, here is the diff of the changes:
commit 007a736559afeaa04643f0b8945e43f8484a5614
Author: syz-cluster <triage@syzkaller.com>
Date: Mon Oct 12 12:01:29 2026 +0000
syz-cluster: applied patch under review
diff --git a/MAINTAINERS b/MAINTAINERS
index c2414447892c2..34902a130ea83 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -30069,6 +30069,15 @@ S: Maintained
F: Documentation/input/devices/yealink.rst
F: drivers/input/misc/yealink.*
+YUNSILICON XSC DRIVERS
+M: Honggang Wei <weihg@yunsilicon.com>
+M: Xin Tian <tianx@yunsilicon.com>
+L: netdev@vger.kernel.org
+L: linux-rdma@vger.kernel.org
+S: Maintained
+F: drivers/infiniband/hw/xsc
+F: drivers/net/ethernet/yunsilicon/xsc
+
ZD1211RW WIRELESS DRIVER
L: linux-wireless@vger.kernel.org
S: Orphan
diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig
index 086195758a8ac..b3cc506db57de 100644
--- a/drivers/infiniband/Kconfig
+++ b/drivers/infiniband/Kconfig
@@ -100,6 +100,7 @@ source "drivers/infiniband/hw/ocrdma/Kconfig"
source "drivers/infiniband/hw/qedr/Kconfig"
source "drivers/infiniband/hw/usnic/Kconfig"
source "drivers/infiniband/hw/vmw_pvrdma/Kconfig"
+source "drivers/infiniband/hw/xsc/Kconfig"
source "drivers/infiniband/sw/rdmavt/Kconfig"
endif # !UML
source "drivers/infiniband/sw/rxe/Kconfig"
diff --git a/drivers/infiniband/hw/Makefile b/drivers/infiniband/hw/Makefile
index c42b22ac3303d..1bcd2d50a76c0 100644
--- a/drivers/infiniband/hw/Makefile
+++ b/drivers/infiniband/hw/Makefile
@@ -16,3 +16,4 @@ obj-$(CONFIG_INFINIBAND_BNXT_RE) += bnxt_re/
obj-$(CONFIG_INFINIBAND_BNG_RE) += bng_re/
obj-$(CONFIG_INFINIBAND_ERDMA) += erdma/
obj-$(CONFIG_INFINIBAND_IONIC) += ionic/
+obj-$(CONFIG_INFINIBAND_XSC) += xsc/
diff --git a/drivers/infiniband/hw/xsc/Kconfig b/drivers/infiniband/hw/xsc/Kconfig
new file mode 100644
index 0000000000000..5cf46df2d4f12
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/Kconfig
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+
+config INFINIBAND_XSC
+ tristate "Yunsilicon XSC RDMA driver"
+ depends on INFINIBAND && PCI && INET
+ depends on YUNSILICON_XSC_PCI && YUNSILICON_XSC_ETH
+ help
+ This driver provides RDMA support for Yunsilicon XSC devices.
+ It uses the auxiliary device published by the XSC PCI core and,
+ for now, the netdev registered by the XSC ethernet driver to
+ derive the port state and GID. Because of that netdev
+ dependency it also depends on YUNSILICON_XSC_ETH.
+
+ To compile this driver as a module, choose M here. The module
+ will be called xsc_ib.
diff --git a/drivers/infiniband/hw/xsc/Makefile b/drivers/infiniband/hw/xsc/Makefile
new file mode 100644
index 0000000000000..48a6d2b0aaace
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/Makefile
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+
+ccflags-y := -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
+
+obj-$(CONFIG_INFINIBAND_XSC) += xsc_ib.o
+
+xsc_ib-y := main.o cq.o qp.o mem.o mr.o ah.o counters.o \
+ xsc_ib_sysfs.o srq.o
diff --git a/drivers/infiniband/hw/xsc/ah.c b/drivers/infiniband/hw/xsc/ah.c
new file mode 100644
index 0000000000000..4d916dbe7dd25
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/ah.c
@@ -0,0 +1,144 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/random.h>
+#include "xsc_ib.h"
+#include <rdma/xsc-abi.h>
+
+static u32 xsc_calc_roce_udp_flow_label(void)
+{
+ /* Generate a non-zero 20 bit flow label. */
+ return get_random_u32_below(IB_GRH_FLOWLABEL_MASK) + 1;
+}
+
+static u16 xsc_ah_get_udp_sport(const struct xsc_ib_dev *dev,
+ struct rdma_ah_attr *ah_attr)
+{
+ enum ib_gid_type gid_type = ah_attr->grh.sgid_attr->gid_type;
+ u16 sport = 0;
+ u32 fl = 0;
+
+ if (gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP &&
+ (rdma_ah_get_ah_flags(ah_attr) & IB_AH_GRH) &&
+ (ah_attr->grh.flow_label & IB_GRH_FLOWLABEL_MASK)) {
+ fl = ah_attr->grh.flow_label;
+ } else {
+ /*generate a 20bit flow_label and output to user layer*/
+ fl = xsc_calc_roce_udp_flow_label();
+ ah_attr->grh.flow_label = fl;
+ }
+
+ sport = rdma_flow_label_to_udp_sport(fl);
+ ibdev_dbg(&dev->ib_dev, "fl=0x%x,sport=0x%x\n", fl, sport);
+ return sport;
+}
+
+static struct ib_ah *create_ib_ah(struct xsc_ib_dev *dev,
+ struct xsc_ib_ah *ah,
+ struct rdma_ah_attr *ah_attr)
+{
+ enum ib_gid_type gid_type;
+
+ if (rdma_ah_get_ah_flags(ah_attr) & IB_AH_GRH) {
+ const struct ib_global_route *grh = rdma_ah_read_grh(ah_attr);
+
+ memcpy(ah->av.rgid, &grh->dgid, 16);
+ ah->av.grh_gid_fl = cpu_to_be32(grh->flow_label |
+ (1 << 30) |
+ grh->sgid_index << 20);
+ ah->av.hop_limit = grh->hop_limit;
+ ah->av.tclass = grh->traffic_class;
+ }
+
+ ah->av.stat_rate_sl = (rdma_ah_get_static_rate(ah_attr) << 4);
+
+ if (ah_attr->type == RDMA_AH_ATTR_TYPE_ROCE) {
+ gid_type = ah_attr->grh.sgid_attr->gid_type;
+
+ memcpy(ah->av.rmac, ah_attr->roce.dmac,
+ sizeof(ah_attr->roce.dmac));
+
+ ah->av.udp_sport = xsc_ah_get_udp_sport(dev, ah_attr);
+ ah->av.stat_rate_sl |= (rdma_ah_get_sl(ah_attr) & 0x7) << 1;
+ if (gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP)
+#define XSC_ECN_ENABLED BIT(1)
+ ah->av.tclass |= XSC_ECN_ENABLED;
+ } else {
+ ah->av.rlid = cpu_to_be16(rdma_ah_get_dlid(ah_attr));
+ ah->av.fl_mlid = rdma_ah_get_path_bits(ah_attr) & 0x7f;
+ ah->av.stat_rate_sl |= (rdma_ah_get_sl(ah_attr) & 0xf);
+ }
+
+ return &ah->ibah;
+}
+
+int xsc_ib_create_ah(struct ib_ah *ibah,
+ struct rdma_ah_init_attr *init_attr,
+ struct ib_udata *udata)
+{
+ struct xsc_ib_ah *ah = to_xah(ibah);
+ struct xsc_ib_dev *dev = to_xdev(ibah->device);
+ struct rdma_ah_attr *ah_attr = init_attr->ah_attr;
+ enum rdma_ah_attr_type ah_type = ah_attr->type;
+
+ if (ah_type == RDMA_AH_ATTR_TYPE_ROCE &&
+ !(rdma_ah_get_ah_flags(ah_attr) & IB_AH_GRH))
+ return -EINVAL;
+
+ if (ah_type == RDMA_AH_ATTR_TYPE_ROCE && udata) {
+ int err;
+ struct xsc_ib_create_ah_resp resp = {};
+ u32 min_resp_len = offsetof(typeof(resp), dmac) +
+ sizeof(resp.dmac);
+
+ if (udata->outlen < min_resp_len)
+ return -EINVAL;
+
+ resp.response_length = min_resp_len;
+ memcpy(resp.dmac, ah_attr->roce.dmac, ETH_ALEN);
+ err = ib_copy_to_udata(udata, &resp, resp.response_length);
+ if (err)
+ return err;
+ }
+
+ create_ib_ah(dev, ah, ah_attr); /* never fails */
+ return 0;
+}
+
+int xsc_ib_query_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr)
+{
+ struct xsc_ib_ah *ah = to_xah(ibah);
+ u32 tmp;
+
+ memset(ah_attr, 0, sizeof(*ah_attr));
+ ah_attr->type = ibah->type;
+
+ tmp = be32_to_cpu(ah->av.grh_gid_fl);
+ if (tmp & (1 << 30)) {
+ rdma_ah_set_grh(ah_attr, NULL, tmp & IB_GRH_FLOWLABEL_MASK,
+ (tmp >> 20) & 0xff, ah->av.hop_limit,
+ ah->av.tclass);
+ rdma_ah_set_dgid_raw(ah_attr, ah->av.rgid);
+ }
+
+ rdma_ah_set_static_rate(ah_attr, ah->av.stat_rate_sl >> 4);
+ if (ah_attr->type == RDMA_AH_ATTR_TYPE_ROCE) {
+ rdma_ah_set_sl(ah_attr, (ah->av.stat_rate_sl >> 1) & 0x7);
+ } else {
+ rdma_ah_set_dlid(ah_attr, be16_to_cpu(ah->av.rlid));
+ rdma_ah_set_path_bits(ah_attr, ah->av.fl_mlid & 0x7f);
+ rdma_ah_set_sl(ah_attr, ah->av.stat_rate_sl & 0xf);
+ }
+
+ return 0;
+}
+
+int xsc_ib_destroy_ah(struct ib_ah *ibah, u32 flags)
+{
+ return 0;
+}
diff --git a/drivers/infiniband/hw/xsc/counters.c b/drivers/infiniband/hw/xsc/counters.c
new file mode 100644
index 0000000000000..135df95954a75
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/counters.c
@@ -0,0 +1,422 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/sysfs.h>
+#include <linux/types.h>
+#include "common/xsc_core.h"
+#include "common/xsc_hsi.h"
+#include "common/driver.h"
+#include "common/xsc_cmd.h"
+#include "counters.h"
+#include "xsc_ib.h"
+
+#define COUNTERS_FILE_NAME "counters"
+#define COUNTERS_NAMES_FILE_NAME "counters_names"
+#define COUNTERS_VALUE_FILE_NAME "counters_value"
+#define COUNTERS_ATTER_GROUP_NAME "counters"
+#define GLOBAL_COUNTERS_GROUP_NAME "global_counters"
+#define GLOBAL_COUNTERS_FILE_NAME "counters"
+
+static const struct counter_desc hw_rdma_stats_pf_desc[] = {
+ /*by mac port*/
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_tx_pkts) },//0
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_tx_bytes) },//1
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_rx_pkts) },//2
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_rx_bytes) },//3
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, np_cnp_sent) },//4
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rp_cnp_handled) },//5
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, np_ecn_marked_roce_packets) },//6
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rp_cnp_ignored) },//7
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, read_rsp_out_of_seq) },//8
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, implied_nak_seq_err) },//9
+ /*by function*/
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, out_of_sequence) },//10
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, packet_seq_err) },//11
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, out_of_buffer) },//12
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rnr_nak_retry_err) },//13
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, local_ack_timeout_err) },//14
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rx_read_requests) },//15
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rx_write_requests) },//16
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, duplicate_requests) },//17
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_tx_pkts_func) },//18
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_tx_payload_bytes) },//19
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_rx_pkts_func) },//20
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_rx_payload_bytes) },//21
+ /*global*/
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_loopback_pkts) },//22
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_loopback_bytes) },//23
+ /*for diamond*/
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, out_of_sequence_sr) },//24
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, packet_seq_err_sr) },//25
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_ndp_rx_pkts) },//26
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_ndp_rx_trimmed_pkts) },//27
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rdma_ndp_trimmed_pkts_sr) },//28
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_pf, rx_icrc_encapsulated) },//29
+ /*XSC_CMDQ_GET_HW_STATS_RDMA_V1*/
+};
+
+static const struct counter_desc hw_rdma_stats_vf_desc[] = {
+ /*by function*/
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, rdma_tx_pkts_func) },
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, rdma_tx_payload_bytes) },
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, rdma_rx_pkts_func) },
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, rdma_rx_payload_bytes) },
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, out_of_sequence) },
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, packet_seq_err) },
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, out_of_buffer) },
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, rnr_nak_retry_err) },
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, local_ack_timeout_err) },
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, rx_read_requests) },
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, rx_write_requests) },
+ { XSC_DECLARE_STAT(struct xsc_hw_stats_rdma_vf, duplicate_requests) },
+};
+
+static const struct counter_desc hw_global_rdma_stats_desc[] = {
+ { XSC_DECLARE_STAT(struct xsc_hw_global_stats_rdma, rdma_loopback_pkts) },
+ { XSC_DECLARE_STAT(struct xsc_hw_global_stats_rdma, rdma_loopback_bytes) },
+ { XSC_DECLARE_STAT(struct xsc_hw_global_stats_rdma, rx_icrc_encapsulated) },
+ { XSC_DECLARE_STAT(struct xsc_hw_global_stats_rdma, req_cqe_error) },
+ { XSC_DECLARE_STAT(struct xsc_hw_global_stats_rdma, resp_cqe_error) },
+ { XSC_DECLARE_STAT(struct xsc_hw_global_stats_rdma, cqe_msg_code_error) },
+};
+
+static int get_hw_stats_rdma(struct xsc_core_device *dev, struct xsc_hw_stats_rdma *stats_rdma)
+{
+ int ret;
+ struct xsc_hw_stats_mbox_in in;
+ struct xsc_hw_stats_rdma_mbox_out out;
+
+ memset(stats_rdma, 0, sizeof(*stats_rdma));
+
+ if (!dev)
+ return -1;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.mac_port = dev->mac_port;
+ in.hdr.ver = cpu_to_be16(XSC_CMDQ_GET_HW_STATS_RDMA_V1);
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_HW_STATS_RDMA);
+
+ ret = xsc_cmd_exec(dev, (void *)&in, sizeof(in), (void *)&out, sizeof(out));
+ if (ret || out.hdr.status)
+ return -1;
+
+ memcpy(stats_rdma, &out.hw_stats, sizeof(*stats_rdma));
+
+ return 0;
+}
+
+static ssize_t counters_names_show(struct device *device,
+ struct device_attribute *attr, char *buf)
+{
+ struct xsc_ib_dev *ibdev = container_of(device, struct xsc_ib_dev, ib_dev.dev);
+ struct xsc_core_device *xdev = ibdev->xdev;
+ struct xsc_hw_stats_rdma stats_rdma;
+ const struct counter_desc *desc;
+ int desc_size;
+ ssize_t count = 0;
+ u64 mask;
+ int i;
+
+ if (get_hw_stats_rdma(xdev, &stats_rdma))
+ return 0;
+
+ if (is_support_hw_pf_stats(xdev)) {
+ desc = hw_rdma_stats_pf_desc;
+ desc_size = ARRAY_SIZE(hw_rdma_stats_pf_desc);
+ } else {
+ desc = hw_rdma_stats_vf_desc;
+ desc_size = ARRAY_SIZE(hw_rdma_stats_vf_desc);
+ }
+
+ mask = xsc_get_rdma_stat_mask(xdev, be16_to_cpu(stats_rdma.ver));
+ for (i = 0; i < desc_size; i++) {
+ if (!((1ULL << i) & mask))
+ continue;
+ count += sysfs_emit_at(buf, count, "%s\n", desc[i].format);
+ }
+
+ return count;
+}
+static DEVICE_ATTR_RO(counters_names);
+
+static ssize_t counters_show(struct device *device,
+ struct device_attribute *attr, char *buf)
+{
+ struct xsc_ib_dev *ibdev = container_of(device, struct xsc_ib_dev, ib_dev.dev);
+ struct xsc_core_device *xdev = ibdev->xdev;
+ struct xsc_hw_stats_rdma stats_rdma;
+ const struct counter_desc *desc;
+ int desc_size;
+ ssize_t count = 0;
+ u8 *stats;
+ u64 mask;
+ int i;
+
+ if (get_hw_stats_rdma(xdev, &stats_rdma) ||
+ is_support_hw_pf_stats(xdev) != stats_rdma.is_pf)
+ return 0;
+
+ if (is_support_hw_pf_stats(xdev)) {
+ desc = hw_rdma_stats_pf_desc;
+ desc_size = ARRAY_SIZE(hw_rdma_stats_pf_desc);
+ stats = (u8 *)&stats_rdma.stats.pf_stats;
+ } else {
+ desc = hw_rdma_stats_vf_desc;
+ desc_size = ARRAY_SIZE(hw_rdma_stats_vf_desc);
+ stats = (u8 *)&stats_rdma.stats.vf_stats;
+ }
+
+ mask = xsc_get_rdma_stat_mask(xdev, be16_to_cpu(stats_rdma.ver));
+ for (i = 0; i < desc_size; i++) {
+ u64 value;
+
+ if (!((1ULL << i) & mask))
+ continue;
+ value = be64_to_cpu(*(u64 *)(stats + desc[i].offset));
+ count += sysfs_emit_at(buf, count, "%-26s %-20llu\n",
+ desc[i].format, value);
+ }
+
+ return count;
+}
+static DEVICE_ATTR_RO(counters);
+
+static ssize_t counters_value_read(struct file *file, struct kobject *kob,
+ const struct bin_attribute *bin_attr,
+ char *buf, loff_t loff, size_t size)
+{
+ struct xsc_ib_dev *ibdev = container_of(kob, struct xsc_ib_dev, ib_dev.dev.kobj);
+ struct xsc_core_device *xdev = ibdev->xdev;
+ struct xsc_hw_stats_rdma stats_rdma;
+ const struct counter_desc *desc;
+ int desc_size;
+ int bin_size;
+ u64 *tmp_value;
+ u8 *stats;
+ u64 mask;
+ int i, j;
+
+ if (bin_attr->size > size || bin_attr->size == 0)
+ return 0;
+
+ if (get_hw_stats_rdma(xdev, &stats_rdma) ||
+ is_support_hw_pf_stats(xdev) != stats_rdma.is_pf)
+ return 0;
+
+ if (is_support_hw_pf_stats(xdev)) {
+ desc = hw_rdma_stats_pf_desc;
+ desc_size = ARRAY_SIZE(hw_rdma_stats_pf_desc);
+ stats = (u8 *)&stats_rdma.stats.pf_stats;
+ } else {
+ desc = hw_rdma_stats_vf_desc;
+ desc_size = ARRAY_SIZE(hw_rdma_stats_vf_desc);
+ stats = (u8 *)&stats_rdma.stats.vf_stats;
+ }
+
+ bin_size = desc_size * sizeof(u64);
+ if (bin_attr->size < bin_size)
+ return 0;
+
+ tmp_value = kzalloc(bin_attr->size, GFP_KERNEL);
+ if (!tmp_value)
+ return 0;
+
+ mask = xsc_get_rdma_stat_mask(xdev, be16_to_cpu(stats_rdma.ver));
+ j = 0;
+ for (i = 0; i < desc_size; i++) {
+ if (!((1ULL << i) & mask))
+ continue;
+ tmp_value[j] = be64_to_cpu(*(u64 *)(stats + desc[i].offset));
+ j++;
+ }
+
+ memcpy(buf, tmp_value, bin_attr->size);
+
+ kfree(tmp_value);
+ return bin_attr->size;
+}
+static BIN_ATTR_RO(counters_value, sizeof(struct xsc_hw_stats_rdma));
+
+static struct attribute *counters_attrs[] = {
+ &dev_attr_counters_names.attr,
+ &dev_attr_counters.attr,
+ NULL,
+};
+
+static const struct bin_attribute *const counters_bin_attrs[] = {
+ &bin_attr_counters_value,
+ NULL,
+};
+
+static const struct attribute_group counters_group = {
+ .name = "counters",
+ .attrs = counters_attrs,
+ .bin_attrs = counters_bin_attrs,
+};
+
+static int counters_sysfs_init(struct ib_device *ib_dev)
+{
+ return sysfs_create_group(&ib_dev->dev.kobj, &counters_group);
+}
+
+static void counters_sysfs_fini(struct ib_device *ib_dev)
+{
+ sysfs_remove_group(&ib_dev->dev.kobj, &counters_group);
+}
+
+static ssize_t global_cnt_attr_show(struct kobject *kobj,
+ struct attribute *attr, char *buf)
+{
+ struct xsc_global_cnt_attributes *a =
+ container_of(attr, struct xsc_global_cnt_attributes, attr);
+ struct xsc_global_cnt_interface *g =
+ container_of(kobj, struct xsc_global_cnt_interface, kobj);
+
+ if (!a->show)
+ return -EIO;
+
+ return a->show(g, a, buf);
+}
+
+static ssize_t global_cnt_attr_store(struct kobject *kobj,
+ struct attribute *attr,
+ const char *buf, size_t size)
+{
+ struct xsc_global_cnt_attributes *a =
+ container_of(attr, struct xsc_global_cnt_attributes, attr);
+ struct xsc_global_cnt_interface *g =
+ container_of(kobj, struct xsc_global_cnt_interface, kobj);
+
+ if (!a->store)
+ return -EIO;
+
+ return a->store(g, a, buf, size);
+}
+
+static ssize_t global_counters_show(struct xsc_global_cnt_interface *g,
+ struct xsc_global_cnt_attributes *a, char *buf)
+{
+ int i;
+ int ret;
+ u8 *stats;
+ u64 value;
+ int desc_size;
+ ssize_t count = 0;
+ const struct counter_desc *desc;
+ struct xsc_hw_global_stats_mbox_in in;
+ struct xsc_hw_global_stats_mbox_out out;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_HW_GLOBAL_STATS);
+ ret = xsc_cmd_exec(g->xdev, (void *)&in, sizeof(in),
+ (void *)&out, sizeof(out));
+ if (ret || out.hdr.status)
+ return 0;
+
+ desc = &hw_global_rdma_stats_desc[0];
+ desc_size = ARRAY_SIZE(hw_global_rdma_stats_desc);
+ stats = (u8 *)&out.hw_stats;
+
+ for (i = 0 ; i < desc_size; i++) {
+ value = *(u64 *)(stats + desc[i].offset);
+ value = be64_to_cpu(value);
+ count += sprintf(&buf[count], "%-26s %-20llu\n",
+ desc[i].format, value);
+ }
+
+ return count;
+}
+
+static ssize_t global_counters_store(struct xsc_global_cnt_interface *g,
+ struct xsc_global_cnt_attributes *a,
+ const char *buf, size_t count)
+{
+ return -EOPNOTSUPP;
+}
+
+#define GLOBAL_CNT_ATTR(_name) struct xsc_global_cnt_attributes xsc_global_cnt_attr_##_name = \
+ __ATTR(_name, 0444, global_##_name##_show, global_##_name##_store)
+
+GLOBAL_CNT_ATTR(counters);
+
+static const struct sysfs_ops global_cnt_sysfs_ops = {
+ .show = global_cnt_attr_show,
+ .store = global_cnt_attr_store,
+};
+
+static struct attribute *global_cnt_attrs[] = {
+ &xsc_global_cnt_attr_counters.attr,
+ NULL
+};
+
+ATTRIBUTE_GROUPS(global_cnt);
+
+static const struct kobj_type global_cnt_ktype = {
+ .sysfs_ops = &global_cnt_sysfs_ops,
+ .default_groups = global_cnt_groups,
+};
+
+static struct xsc_global_cnt_interface *g_global_cnt_interface;
+
+static int global_cnt_sysfs_init(struct ib_device *ib_dev, struct xsc_core_device *xdev)
+{
+ struct xsc_global_cnt_interface *tmp;
+ int err;
+
+ if (!xdev || !xsc_core_is_pf(xdev) || xdev->pf_id != 0)
+ return 0;
+
+ tmp = kzalloc_obj(*tmp, GFP_KERNEL);
+ if (!tmp)
+ return -ENOMEM;
+
+ err = kobject_init_and_add(&tmp->kobj, &global_cnt_ktype,
+ &ib_dev->dev.kobj, GLOBAL_COUNTERS_GROUP_NAME);
+ if (err)
+ goto error_return;
+
+ g_global_cnt_interface = tmp;
+ tmp->xdev = xdev;
+ return 0;
+
+error_return:
+ kobject_put(&tmp->kobj);
+ kfree(tmp);
+ return err;
+}
+
+static void global_cnt_sysfs_fini(struct xsc_core_device *xdev)
+{
+ if (!g_global_cnt_interface || !xdev || !xsc_core_is_pf(xdev) || xdev->pf_id != 0)
+ return;
+
+ kobject_put(&g_global_cnt_interface->kobj);
+ kfree(g_global_cnt_interface);
+ g_global_cnt_interface = NULL;
+}
+
+int xsc_counters_init(struct ib_device *ib_dev, struct xsc_core_device *xdev)
+{
+ int ret;
+
+ ret = counters_sysfs_init(ib_dev);
+ if (ret)
+ return ret;
+
+ ret = global_cnt_sysfs_init(ib_dev, xdev);
+ if (ret)
+ counters_sysfs_fini(ib_dev);
+
+ return ret;
+}
+
+void xsc_counters_fini(struct ib_device *ib_dev, struct xsc_core_device *xdev)
+{
+ counters_sysfs_fini(ib_dev);
+ global_cnt_sysfs_fini(xdev);
+}
diff --git a/drivers/infiniband/hw/xsc/counters.h b/drivers/infiniband/hw/xsc/counters.h
new file mode 100644
index 0000000000000..944dbafb62bc9
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/counters.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __COUNTERS_H__
+#define __COUNTERS_H__
+
+#define STRING_LEN 32
+#define XSC_DECLARE_STAT(type, fld) ""#fld, offsetof(type, fld)
+
+struct counter_desc {
+ char format[STRING_LEN];
+ size_t offset; /* Byte offset */
+};
+
+struct xsc_global_cnt_interface {
+ struct xsc_core_device *xdev;
+ struct kobject kobj;
+};
+
+struct xsc_global_cnt_attributes {
+ struct attribute attr;
+ ssize_t (*show)(struct xsc_global_cnt_interface *g, struct xsc_global_cnt_attributes *a,
+ char *buf);
+ ssize_t (*store)(struct xsc_global_cnt_interface *g, struct xsc_global_cnt_attributes *a,
+ const char *buf, size_t count);
+};
+
+#endif
diff --git a/drivers/infiniband/hw/xsc/cq.c b/drivers/infiniband/hw/xsc/cq.c
new file mode 100644
index 0000000000000..04820bf3a7fa4
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/cq.c
@@ -0,0 +1,895 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kref.h>
+#include <rdma/ib_umem.h>
+#include "xsc_ib.h"
+#include <rdma/xsc-abi.h>
+#include "common/xsc_hsi.h"
+#include <linux/dma-direct.h>
+
+#ifndef IB_UVERBS_CQ_FLAGS_IGNORE_OVERRUN
+#define IB_UVERBS_CQ_FLAGS_IGNORE_OVERRUN BIT(1)
+#endif
+
+enum {
+ CQ_OK = 0,
+ CQ_EMPTY = -1,
+ CQ_POLL_ERR = -2
+};
+
+enum {
+ XSC_CQE_APP_TAG_MATCHING = 1,
+};
+
+enum {
+ XSC_CQE_APP_OP_TM_CONSUMED = 0x1,
+ XSC_CQE_APP_OP_TM_EXPECTED = 0x2,
+ XSC_CQE_APP_OP_TM_UNEXPECTED = 0x3,
+ XSC_CQE_APP_OP_TM_NO_TAG = 0x4,
+ XSC_CQE_APP_OP_TM_APPEND = 0x5,
+ XSC_CQE_APP_OP_TM_REMOVE = 0x6,
+ XSC_CQE_APP_OP_TM_NOOP = 0x7,
+ XSC_CQE_APP_OP_TM_CONSUMED_SW_RDNV = 0x9,
+ XSC_CQE_APP_OP_TM_CONSUMED_MSG = 0xA,
+ XSC_CQE_APP_OP_TM_CONSUMED_MSG_SW_RDNV = 0xB,
+ XSC_CQE_APP_OP_TM_MSG_COMPLETION_CANCELED = 0xC,
+};
+
+static const u32 xsc_cqe_opcode[] = {
+ [XSC_OPCODE_RDMA_REQ_SEND] = IB_WC_SEND,
+ [XSC_OPCODE_RDMA_REQ_SEND_IMMDT] = IB_WC_SEND,
+ [XSC_OPCODE_RDMA_RSP_RECV] = IB_WC_RECV,
+ [XSC_OPCODE_RDMA_RSP_RECV_IMMDT] = IB_WC_RECV,
+ [XSC_OPCODE_RDMA_REQ_WRITE] = IB_WC_RDMA_WRITE,
+ [XSC_OPCODE_RDMA_REQ_WRITE_IMMDT] = IB_WC_RDMA_WRITE,
+ [XSC_OPCODE_RDMA_RSP_WRITE_IMMDT] = IB_WC_RECV_RDMA_WITH_IMM,
+ [XSC_OPCODE_RDMA_REQ_READ] = IB_WC_RDMA_READ,
+ [XSC_OPCODE_RDMA_MAD_REQ_SEND] = IB_WC_SEND,
+ [XSC_OPCODE_RDMA_MAD_RSP_RECV] = IB_WC_RECV,
+};
+
+static void xsc_ib_cq_comp(struct xsc_core_cq *cq)
+{
+ struct ib_cq *ibcq = &core_to_xcq(cq)->ibcq;
+
+ ibcq->comp_handler(ibcq, ibcq->cq_context);
+}
+
+static void xsc_ib_cq_event(struct xsc_core_cq *xcq, enum xsc_event type)
+{
+ struct xsc_ib_cq *cq = container_of(xcq, struct xsc_ib_cq, xcq);
+ struct xsc_ib_dev *dev = to_xdev(cq->ibcq.device);
+ struct ib_cq *ibcq = &cq->ibcq;
+ struct ib_event event;
+
+ if (type != XSC_EVENT_TYPE_CQ_ERROR) {
+ ibdev_err(&dev->ib_dev, "Unexpected event type %d on CQ %06x\n",
+ type, xcq->cqn);
+ return;
+ }
+
+ if (ibcq->event_handler) {
+ event.device = &dev->ib_dev;
+ event.event = IB_EVENT_CQ_ERR;
+ event.element.cq = ibcq;
+ ibcq->event_handler(&event, ibcq->cq_context);
+ pr_info("xsc_ib: xsc async event 0x%x, ib event %d\n",
+ XSC_EVENT_TYPE_CQ_ERROR, IB_EVENT_CQ_ERR);
+ }
+}
+
+static void *get_cqe_from_buf(struct xsc_ib_cq_buf *buf, int n, int size)
+{
+ return xsc_buf_offset(&buf->buf, n * size);
+}
+
+static void *get_cqe(struct xsc_ib_cq *cq, int n)
+{
+ return get_cqe_from_buf(&cq->buf, n, cq->xcq.cqe_sz);
+}
+
+static void *get_sw_cqe(struct xsc_ib_cq *cq, int n)
+{
+ struct xsc_cqe *cqe;
+
+ cqe = (struct xsc_cqe *)get_cqe(cq, n & (cq->ibcq.cqe - 1));
+
+ return ((cqe->owner & XSC_CQE_OWNER_MASK) ^
+ !!(n & cq->ibcq.cqe)) ? NULL : cqe;
+}
+
+static inline void handle_good_req(struct ib_wc *wc,
+ struct xsc_cqe *cqe,
+ u8 opcode)
+{
+ wc->opcode = xsc_cqe_opcode[opcode];
+ if (opcode == XSC_OPCODE_RDMA_REQ_READ)
+ wc->byte_len = RD_LE_32(cqe->msg_len);
+ wc->status = IB_WC_SUCCESS;
+}
+
+static void handle_responder(struct ib_wc *wc, struct xsc_cqe *cqe,
+ struct xsc_ib_qp *qp, u8 opcode)
+{
+ struct xsc_ib_wq *wq = &qp->rq;
+ u16 idx;
+
+ wc->byte_len = RD_LE_32(cqe->msg_len);
+ wc->opcode = xsc_cqe_opcode[opcode];
+ wc->status = IB_WC_SUCCESS;
+
+ idx = wq->tail & (wq->wqe_cnt - 1);
+ wc->wr_id = wq->wrid[idx];
+ atomic_dec(&wq->flush_wqe_cnt);
+ ++wq->tail;
+}
+
+static void *get_wqe(struct xsc_ib_qp *qp, int offset)
+{
+ return xsc_buf_offset(&qp->buf, offset);
+}
+
+static void *get_recv_wqe(struct xsc_ib_qp *qp, int n)
+{
+ return get_wqe(qp, qp->rq.offset + (n << qp->rq.wqe_shift));
+}
+
+static void *get_seg_wqe(void *first, int n)
+{
+ return first + (n << XSC_BASE_WQE_SHIFT);
+}
+
+static void xsc_handle_rdma_mad_resp_recv(struct xsc_ib_cq *cq,
+ struct xsc_ib_qp **cur_qp,
+ struct ib_wc *wc,
+ struct xsc_cqe *cqe,
+ u8 opcode)
+{
+ struct xsc_ib_dev *dev = to_xdev(cq->ibcq.device);
+ void *recv;
+ struct xsc_wqe_data_seg *data_seg;
+ struct iphdr *ip4h = NULL;
+ struct ipv6hdr *ip6h;
+ struct udphdr *udph;
+ struct ib_unpacked_eth *eth;
+ struct ib_unpacked_vlan *vlan;
+ struct ib_grh *grh;
+ struct ib_mad *mad;
+ struct rxe_bth *bth;
+ struct rxe_deth *deth;
+ unsigned int pading_sz = 0;
+ struct xsc_ib_wq *wq;
+ int idx;
+ u16 eth_type;
+ void *l3_start;
+
+ wq = &(*cur_qp)->rq;
+ idx = wq->tail & (wq->wqe_cnt - 1);
+
+ handle_responder(wc, cqe, *cur_qp, opcode);
+
+ data_seg = get_seg_wqe(get_recv_wqe(*cur_qp, idx), 0);
+ recv = xsc_ib_recv_mad_sg_virt_addr(&dev->ib_dev, wc, data_seg->va);
+
+ eth = (struct ib_unpacked_eth *)recv;
+ grh = (struct ib_grh *)recv;
+ if (eth->type == htons(ETH_P_8021Q)) {
+ vlan = (struct ib_unpacked_vlan *)(eth + 1);
+ eth_type = ntohs(vlan->type);
+ l3_start = vlan + 1;
+
+ wc->vlan_id = ntohs(vlan->tag) & 0x0fff;
+ wc->sl = (ntohs(vlan->tag) >> 13) & 0x7;
+ wc->wc_flags |= IB_WC_WITH_VLAN;
+ } else {
+ eth_type = ntohs(eth->type);
+ l3_start = eth + 1;
+ }
+
+ if (eth_type == ETH_P_IP) {
+ ip4h = (struct iphdr *)l3_start;
+ udph = (struct udphdr *)(ip4h + 1);
+ } else {
+ ip6h = (struct ipv6hdr *)l3_start;
+ udph = (struct udphdr *)(ip6h + 1);
+ }
+ bth = (struct rxe_bth *)(udph + 1);
+ deth = (struct rxe_deth *)(bth + 1);
+ mad = (struct ib_mad *)(deth + 1);
+
+ if (eth_type == ETH_P_IP) {
+ pading_sz = sizeof(*grh) - sizeof(*ip4h);
+ memmove((u8 *)(grh + 1) - sizeof(*ip4h), ip4h, sizeof(*ip4h));
+ memset(grh, 0, pading_sz);
+ } else {
+ memmove(grh, ip6h, sizeof(*ip6h));
+ }
+ memmove(grh + 1, mad, sizeof(*mad));
+
+ wc->wc_flags |= IB_WC_GRH;
+
+ ibdev_dbg(&dev->ib_dev, "recv cqe idx:%u, len:%u\n", wq->tail, wc->byte_len);
+ ibdev_dbg(&dev->ib_dev, "qp[%d] recv MAD packet, msg_len=%d\n",
+ (*cur_qp)->xqp.qpn, wc->byte_len);
+ wc->status = IB_WC_SUCCESS;
+}
+
+static void xsc_generate_lost_cqe(struct xsc_ib_dev *dev, struct xsc_ib_wq *wq,
+ int qpn, int idx, struct ib_wc *wc)
+{
+ int lost_cqe_idx = wq->prev_ce_wqe[idx];
+
+ ibdev_err(&dev->ib_dev, "detected lost cqe, cur wqe idx of qp%d is %d\n", qpn, idx);
+
+ while (wq->prev_ce_wqe[lost_cqe_idx] != wq->last_comp_ce_wqe)
+ lost_cqe_idx = wq->prev_ce_wqe[lost_cqe_idx];
+ ibdev_err(&dev->ib_dev, "generate lost cqe idx of qp%d is %d\n", qpn, lost_cqe_idx);
+
+ wq->last_comp_ce_wqe = lost_cqe_idx;
+ wc->wr_id = wq->wrid[lost_cqe_idx];
+ wq->tail = wq->wqe_head[lost_cqe_idx] + 1;
+ atomic_dec(&wq->flush_wqe_cnt);
+ wq->need_flush[lost_cqe_idx] = 0;
+ wc->status = IB_WC_GENERAL_ERR;
+}
+
+static int xsc_poll_one(struct xsc_ib_cq *cq,
+ struct xsc_ib_qp **cur_qp,
+ struct ib_wc *wc)
+{
+ struct xsc_ib_dev *dev = to_xdev(cq->ibcq.device);
+ struct xsc_core_qp *xqp;
+ struct xsc_ib_wq *wq;
+ u8 opcode;
+ u32 qpn;
+ int idx;
+ struct xsc_cqe *cqe;
+ u32 *p = NULL;
+ struct xsc_ib_srq *xsrq;
+
+ cqe = get_sw_cqe(cq, cq->xcq.cons_index);
+ if (!cqe)
+ return -EAGAIN;
+
+ ++cq->xcq.cons_index;
+
+ /* Make sure we read CQ entry contents after we've checked the
+ * ownership bit.
+ */
+ rmb();
+
+ p = (u32 *)cqe;
+
+ qpn = cqe->qp_id;
+ qpn = le32_to_cpu(qpn);
+ if (!*cur_qp || (qpn != (*cur_qp)->ibqp.qp_num)) {
+ /* We do not have to take the QP table lock here,
+ * because CQs will be locked while QPs are removed
+ * from the table.
+ */
+ xqp = __xsc_qp_lookup(dev->xdev, qpn);
+ if (unlikely(!xqp)) {
+ ibdev_warn(&dev->ib_dev, "CQE@CQ %d for unknown QPN %d\n",
+ cq->xcq.cqn, qpn);
+ return -EINVAL;
+ }
+
+ *cur_qp = core_to_xqp(xqp);
+ }
+
+ memset(wc, 0, sizeof(*wc));
+ wc->qp = &(*cur_qp)->ibqp;
+ opcode = xsc_get_cqe_opcode(dev->xdev, cqe);
+ switch (opcode) {
+ case XSC_OPCODE_RDMA_REQ_SEND_IMMDT:
+ case XSC_OPCODE_RDMA_REQ_WRITE_IMMDT:
+ wc->wc_flags |= IB_WC_WITH_IMM;
+ fallthrough;
+ case XSC_OPCODE_RDMA_REQ_SEND:
+ case XSC_OPCODE_RDMA_REQ_WRITE:
+ case XSC_OPCODE_RDMA_REQ_READ:
+ case XSC_OPCODE_RDMA_MAD_REQ_SEND:
+ wq = &(*cur_qp)->sq;
+ idx = xsc_get_cqe_wqe_id(dev->xdev, cqe) >> (wq->wqe_shift - XSC_BASE_WQE_SHIFT);
+ idx &= (wq->wqe_cnt - 1);
+ if (wq->prev_ce_wqe[idx] != wq->last_comp_ce_wqe) {
+ xsc_generate_lost_cqe(dev, wq, qpn, idx, wc);
+ cq->invalid = 1;
+ break;
+ }
+ wq->last_comp_ce_wqe = idx;
+ handle_good_req(wc, cqe, opcode);
+ wc->wr_id = wq->wrid[idx];
+ wq->tail = wq->wqe_head[idx] + 1;
+ if (opcode != XSC_OPCODE_RDMA_MAD_REQ_SEND)
+ atomic_dec(&wq->flush_wqe_cnt);
+ wq->need_flush[idx] = 0;
+ ibdev_dbg(&dev->ib_dev, "wqeid:%u, wq tail:%u qpn:%u\n", idx, wq->tail, qpn);
+ wc->status = IB_WC_SUCCESS;
+ break;
+ case XSC_OPCODE_RDMA_RSP_RECV_IMMDT:
+ case XSC_OPCODE_RDMA_RSP_WRITE_IMMDT:
+ wc->wc_flags |= IB_WC_WITH_IMM;
+ WR_BE_32(wc->ex.imm_data, RD_LE_32(cqe->imm_data));
+ fallthrough;
+ case XSC_OPCODE_RDMA_RSP_RECV:
+ wq = &(*cur_qp)->rq;
+ handle_responder(wc, cqe, *cur_qp, opcode);
+ if ((*cur_qp)->ibqp.srq) {
+ xsrq = to_xsrq((*cur_qp)->ibqp.srq);
+ xsc_free_one_cache_srq_wr(xsrq, *cur_qp, wc->wr_id);
+ }
+ ibdev_dbg(&dev->ib_dev, "recv cqe idx:%u, len:%u, qpn:%u\n",
+ wq->tail, wc->byte_len, qpn);
+ wc->status = IB_WC_SUCCESS;
+ break;
+
+ case XSC_OPCODE_RDMA_MAD_RSP_RECV:
+ ibdev_dbg(&dev->ib_dev, "recv MAD, qpn:%u\n", qpn);
+ xsc_handle_rdma_mad_resp_recv(cq, cur_qp, wc, cqe, opcode);
+ break;
+
+ case XSC_OPCODE_RDMA_REQ_ERROR:
+ wq = &(*cur_qp)->sq;
+ idx = xsc_get_cqe_wqe_id(dev->xdev, cqe) >> (wq->wqe_shift - XSC_BASE_WQE_SHIFT);
+ idx &= (wq->wqe_cnt - 1);
+
+ if (!wq->need_flush[idx]) {
+ unsigned int i;
+ unsigned int tmp;
+ unsigned int prev = -1;
+
+ for (i = wq->tail; i != wq->head; i++) {
+ tmp = i & (wq->wqe_cnt - 1);
+ if (tmp == idx)
+ break;
+ if (wq->need_flush[tmp])
+ prev = tmp;
+ }
+
+ if (prev != -1) {
+ if (wq->prev_ce_wqe[prev] != wq->last_comp_ce_wqe) {
+ xsc_generate_lost_cqe(dev, wq, qpn, prev, wc);
+ cq->invalid = 1;
+ break;
+ }
+ }
+ } else {
+ if (wq->prev_ce_wqe[idx] != wq->last_comp_ce_wqe) {
+ xsc_generate_lost_cqe(dev, wq, qpn, idx, wc);
+ cq->invalid = 1;
+ break;
+ }
+ }
+
+ wc->wr_id = wq->wrid[idx];
+ wq->tail = wq->wqe_head[idx] + 1;
+ if (wq->need_flush[idx])
+ atomic_dec(&wq->flush_wqe_cnt);
+ wq->need_flush[idx] = 0;
+ ibdev_err(&dev->ib_dev, "req error\n%08x %08x %08x %08x %08x %08x\n",
+ p[0], p[1], p[2], p[3], p[5], p[6]);
+ wc->status = IB_WC_GENERAL_ERR;
+ xsc_ib_err_state_qp(dev, *cur_qp, (*cur_qp)->state, XSC_QP_STATE_ERR);
+ break;
+ case XSC_OPCODE_RDMA_RSP_ERROR:
+ wq = &(*cur_qp)->rq;
+ idx = wq->tail & (wq->wqe_cnt - 1);
+ wc->wr_id = wq->wrid[idx];
+ wq->tail++;
+ atomic_dec(&wq->flush_wqe_cnt);
+ if ((*cur_qp)->ibqp.srq) {
+ xsrq = to_xsrq((*cur_qp)->ibqp.srq);
+ xsc_free_one_cache_srq_wr(xsrq, *cur_qp, wc->wr_id);
+ }
+ ibdev_err(&dev->ib_dev, "rsp error\n%08x %08x %08x %08x %08x %08x\n",
+ p[0], p[1], p[2], p[3], p[5], p[6]);
+ wc->status = IB_WC_GENERAL_ERR;
+ xsc_ib_err_state_qp(dev, *cur_qp, (*cur_qp)->state, XSC_QP_STATE_ERR);
+ break;
+
+ default:
+ ibdev_err(&dev->ib_dev, "completion error\n%08x %08x %08x %08x %08x %08x\n",
+ p[0], p[1], p[2], p[3], p[5], p[6]);
+ wc->status = IB_WC_GENERAL_ERR;
+ wc->wr_id = 0;
+ break;
+ }
+
+ return 0;
+}
+
+static inline void gen_flush_err_cqe(struct xsc_err_state_qp_node *err_node,
+ struct ib_qp *ibqp, struct xsc_ib_wq *wq, u32 idx,
+ struct ib_wc *wc)
+{
+ memset(wc, 0, sizeof(*wc));
+ if (err_node->is_sq) {
+ switch (wq->wr_opcode[idx]) {
+ case IB_WR_SEND:
+ case IB_WR_SEND_WITH_IMM:
+ wc->opcode = IB_WC_SEND;
+ break;
+ case IB_WR_RDMA_WRITE:
+ case IB_WR_RDMA_WRITE_WITH_IMM:
+ wc->opcode = IB_WC_RDMA_WRITE;
+ break;
+ case IB_WR_RDMA_READ:
+ wc->opcode = IB_WC_RDMA_READ;
+ }
+ } else {
+ wc->opcode = IB_WC_RECV;
+ }
+
+ wc->qp = ibqp;
+ wc->status = IB_WC_WR_FLUSH_ERR;
+ wc->vendor_err = XSC_ERR_CODE_FLUSH;
+ wc->wr_id = wq->wrid[idx];
+ wq->tail++;
+ atomic_dec(&wq->flush_wqe_cnt);
+ if (err_node->is_sq)
+ wq->need_flush[idx] = 0;
+}
+
+static inline int xsc_generate_flush_err_cqe(struct ib_cq *ibcq,
+ int ne, int *npolled, struct ib_wc *wc)
+{
+ u32 qp_id = 0;
+ int flush_wqe_cnt = 0;
+ int sw_npolled = 0;
+ u32 idx = 0;
+ struct xsc_err_state_qp_node *err_qp_node;
+ struct xsc_core_qp *xqp;
+ struct xsc_ib_cq *cq = to_xcq(ibcq);
+ struct xsc_ib_wq *wq;
+ struct xsc_ib_dev *dev = to_xdev(cq->ibcq.device);
+
+ list_for_each_entry(err_qp_node, &cq->err_state_qp_list, entry) {
+ if (!err_qp_node)
+ break;
+
+ sw_npolled = 0;
+ qp_id = err_qp_node->qp_id;
+ xqp = __xsc_qp_lookup(dev->xdev, qp_id);
+ if (unlikely(!xqp)) {
+ ibdev_warn(&dev->ib_dev, "CQE@CQ %d for unknown QPN %d\n",
+ cq->xcq.cqn, qp_id);
+ continue;
+ }
+ wq = err_qp_node->is_sq ? &(core_to_xqp(xqp)->sq) : &(core_to_xqp(xqp)->rq);
+ flush_wqe_cnt = atomic_read(&wq->flush_wqe_cnt);
+ ibdev_dbg(&dev->ib_dev, "is_sq %d, flush_wq_cnt %d, ne %d, npolled %d, qp_id %d, cqn %d\n",
+ err_qp_node->is_sq, flush_wqe_cnt, ne, *npolled, qp_id, cq->xcq.cqn);
+
+ while (wq->tail != wq->head) {
+ idx = wq->tail & (wq->wqe_cnt - 1);
+ if (err_qp_node->is_sq && !wq->need_flush[idx]) {
+ wq->tail++;
+ continue;
+ } else {
+ gen_flush_err_cqe(err_qp_node, &(core_to_xqp(xqp)->ibqp), wq,
+ idx, wc + *npolled + sw_npolled);
+ ++sw_npolled;
+ }
+
+ if (sw_npolled == ne - *npolled)
+ break;
+ }
+ *npolled += sw_npolled;
+ if (*npolled == ne)
+ break;
+ }
+
+ return 0;
+}
+
+int xsc_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
+{
+ struct xsc_ib_cq *cq = to_xcq(ibcq);
+ struct xsc_core_cq *xcq = &cq->xcq;
+ struct xsc_ib_qp *cur_qp = NULL;
+ int npolled = 0;
+ int err = 0;
+ unsigned long flags;
+ u32 next_cid;
+
+ spin_lock_irqsave(&cq->lock, flags);
+ next_cid = xcq->cons_index;
+
+ for (npolled = 0; npolled < num_entries; npolled++) {
+ if (cq->invalid) {
+ err = -EINVAL;
+ break;
+ }
+ err = xsc_poll_one(cq, &cur_qp, wc + npolled);
+ if (err)
+ break;
+ }
+
+ if (err) {
+ if (npolled < num_entries && !(list_empty(&cq->err_state_qp_list)))
+ xsc_generate_flush_err_cqe(ibcq, num_entries, &npolled, wc);
+ }
+
+ if (next_cid != xcq->cons_index)
+ xsc_update_cq_ci(xcq->dev, xcq->cqn, xcq->cons_index);
+
+ spin_unlock_irqrestore(&cq->lock, flags);
+
+ return npolled;
+}
+
+static int xsc_cqe_is_empty(struct xsc_ib_cq *cq)
+{
+ struct xsc_cqe *cqe = get_sw_cqe(cq, cq->xcq.cons_index);
+
+ if (cq->invalid)
+ return 1;
+
+ if (!cqe)
+ return 1;
+
+ return 0;
+}
+
+int xsc_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags)
+{
+ struct xsc_ib_cq *xcq = to_xcq(ibcq);
+ struct xsc_core_cq *cq = &xcq->xcq;
+ int ret = 0;
+ unsigned long irq_flags;
+ u8 solicited = 0;
+
+ spin_lock_irqsave(&xcq->lock, irq_flags);
+ if (flags & IB_CQ_NEXT_COMP)
+ solicited = 0;
+ else if (flags & IB_CQ_SOLICITED)
+ solicited = 1;/* arm next:0 arm solicited:1 */
+
+ if ((flags & IB_CQ_REPORT_MISSED_EVENTS) && (!xsc_cqe_is_empty(xcq))) {
+ ret = 1;
+ goto out;
+ }
+
+ xsc_arm_cq(cq->dev, cq->cqn, cq->cons_index, solicited);
+out:
+ spin_unlock_irqrestore(&xcq->lock, irq_flags);
+ return ret;
+}
+
+static int alloc_cq_buf(struct xsc_ib_dev *dev, struct xsc_ib_cq_buf *buf,
+ int nent, int cqe_size)
+{
+ int err;
+
+ err = xsc_buf_alloc(dev->xdev, nent * cqe_size,
+ PAGE_SIZE, &buf->buf);
+ if (err)
+ return err;
+
+ buf->cqe_size = cqe_size;
+
+ return 0;
+}
+
+static void free_cq_buf(struct xsc_ib_dev *dev, struct xsc_ib_cq_buf *buf)
+{
+ xsc_buf_free(dev->xdev, &buf->buf);
+}
+
+static struct ib_umem *cq_umem_get(struct xsc_ib_dev *dev,
+ struct ib_udata *udata, struct ib_ucontext *context,
+ int entries, struct xsc_ib_create_cq *ucmd)
+{
+ struct ib_umem *umem;
+
+ umem = ib_umem_get_va(&dev->ib_dev, ucmd->buf_addr,
+ entries * ucmd->cqe_size,
+ IB_ACCESS_LOCAL_WRITE);
+ return umem;
+}
+
+static int create_cq_user(struct xsc_ib_dev *dev, struct ib_udata *udata,
+ struct ib_ucontext *context, struct xsc_ib_cq *cq,
+ int entries, struct xsc_create_cq_ex_mbox_in **cqb,
+ int *cqe_size, int *index, int *inlen)
+{
+ struct xsc_ib_create_cq ucmd;
+ int page_shift;
+ int npages;
+ int ncont;
+ int err;
+ int log_cq_sz;
+ int hw_npages;
+ struct ib_umem_dmabuf *umem_dmabuf;
+
+ if (ib_copy_from_udata(&ucmd, udata, sizeof(ucmd)))
+ return -EFAULT;
+
+ *cqe_size = ucmd.cqe_size;
+
+ if (ucmd.dmabuf_fd != -1) {
+ umem_dmabuf = ib_umem_dmabuf_get_pinned(&dev->ib_dev, 0, ucmd.dmabuf_sz,
+ ucmd.dmabuf_fd, IB_ACCESS_LOCAL_WRITE);
+
+ if (IS_ERR(umem_dmabuf)) {
+ ibdev_warn(&dev->ib_dev, "umem_dmabuf get failed by fd %d (%ld)\n",
+ ucmd.dmabuf_fd, PTR_ERR(umem_dmabuf));
+ return -ENOMEM;
+ }
+
+ cq->buf.umem = &umem_dmabuf->umem;
+ ibdev_dbg(&dev->ib_dev, "get umem_dmabuf success by fd %d, umem_dmabuf_addr %p\n",
+ ucmd.dmabuf_fd, umem_dmabuf);
+ } else {
+ cq->buf.umem = cq_umem_get(dev, udata, context, entries, &ucmd);
+ }
+
+ if (IS_ERR(cq->buf.umem)) {
+ err = PTR_ERR(cq->buf.umem);
+ return err;
+ }
+
+ xsc_ib_cont_pages(cq->buf.umem, ucmd.buf_addr, &npages, &page_shift,
+ &ncont);
+ if (ncont != npages) {
+ ibdev_dbg(&dev->ib_dev, "bad page_shift:%d, ncont:%d\n", page_shift, ncont);
+ /* amber doesn't support compound pages */
+ page_shift = PAGE_SHIFT;
+ ncont = npages;
+ ibdev_dbg(&dev->ib_dev, "overwrite to page_shift:%d, ncont:%d\n",
+ page_shift, ncont);
+ }
+ log_cq_sz = ilog2(entries);
+ hw_npages = DIV_ROUND_UP((1 << log_cq_sz) * (*cqe_size), PAGE_SIZE_4K);
+ ibdev_dbg(&dev->ib_dev, "addr 0x%llx, entries %d, npages %d, hw_npages %d\n",
+ ucmd.buf_addr, entries, npages, hw_npages);
+
+ *inlen = sizeof(**cqb) + sizeof(*(*cqb)->pas) * hw_npages;
+ *cqb = xsc_vzalloc(*inlen);
+ if (!*cqb) {
+ err = -ENOMEM;
+ goto err_umem;
+ }
+ xsc_ib_populate_pas(dev, cq->buf.umem, page_shift, (*cqb)->pas, hw_npages, true);
+ (*cqb)->ctx_ex.ctx.pa_num = cpu_to_be16(hw_npages);
+ (*cqb)->ctx_ex.pa_num_ex = cpu_to_be32(hw_npages);
+
+ return 0;
+
+err_umem:
+ ib_umem_release(cq->buf.umem);
+ return err;
+}
+
+static void destroy_cq_user(struct xsc_ib_cq *cq, struct ib_udata *udata)
+{
+ ib_umem_release(cq->buf.umem);
+}
+
+static int create_cq_kernel(struct xsc_ib_dev *dev, struct xsc_ib_cq *cq,
+ int entries, int cqe_size,
+ struct xsc_create_cq_ex_mbox_in **cqb,
+ int *index, int *inlen)
+{
+ int err;
+ int i = 0;
+ struct xsc_cqe *cqe = NULL;
+ int hw_npages;
+
+ cq->xcq.cqe_sz = cqe_size;
+
+ err = alloc_cq_buf(dev, &cq->buf, entries, cqe_size);
+ if (err)
+ return err;
+
+ for (i = 0; i < entries; i++) {
+ cqe = (struct xsc_cqe *)get_cqe(cq, i);
+ cqe->owner = 1;
+ }
+
+ hw_npages = DIV_ROUND_UP(entries * cqe_size, PAGE_SIZE_4K);
+ *inlen = sizeof(**cqb) + sizeof(*(*cqb)->pas) * hw_npages;
+ *cqb = xsc_vzalloc(*inlen);
+ if (!*cqb) {
+ err = -ENOMEM;
+ goto err_buf;
+ }
+ xsc_fill_page_array(&cq->buf.buf, (*cqb)->pas, hw_npages);
+ (*cqb)->ctx_ex.ctx.pa_num = cpu_to_be16(hw_npages);
+ (*cqb)->ctx_ex.pa_num_ex = cpu_to_be32(hw_npages);
+
+ return 0;
+
+err_buf:
+ free_cq_buf(dev, &cq->buf);
+ return err;
+}
+
+static void destroy_cq_kernel(struct xsc_ib_dev *dev, struct xsc_ib_cq *cq)
+{
+ free_cq_buf(dev, &cq->buf);
+}
+
+int xsc_ib_create_cq(struct ib_cq *ibcq,
+ const struct ib_cq_init_attr *attr,
+ struct uverbs_attr_bundle *attrs)
+{
+ struct ib_device *ibdev = ibcq->device;
+ struct ib_udata *udata = attrs ? &attrs->driver_udata : NULL;
+ int entries = attr->cqe;
+ int vector = attr->comp_vector;
+ struct xsc_create_cq_ex_mbox_in *cqb = NULL;
+ struct xsc_ib_dev *dev = to_xdev(ibdev);
+ struct xsc_ib_cq *cq;
+ int index;
+ int inlen = 0;
+ int cqe_size;
+ int irqn;
+ int err;
+ unsigned int eqn;
+
+ entries = roundup_pow_of_two(entries);
+
+ ibdev_dbg(&dev->ib_dev, "entries:%d, vector:%d, max_cqes:%d\n", entries, vector,
+ dev->xdev->caps.max_cqes);
+
+ if (entries > dev->xdev->caps.max_cqes)
+ entries = dev->xdev->caps.max_cqes;
+ cq = to_xcq(ibcq);
+ cq->ibcq.cqe = entries;
+ mutex_init(&cq->resize_mutex);
+ spin_lock_init(&cq->lock);
+ cq->resize_buf = NULL;
+ cq->resize_umem = NULL;
+ cq->create_flags = attr->flags;
+ cq->invalid = 0;
+
+ if (udata) {
+ err = create_cq_user(dev, udata, NULL, cq, entries,
+ &cqb, &cqe_size, &index, &inlen);
+ if (err)
+ goto err_create;
+ } else {
+ cqe_size = is_support_cqe64(dev->xdev) ?
+ sizeof(struct xsc_cqe64) : sizeof(struct xsc_cqe);
+ err = create_cq_kernel(dev, cq, entries, cqe_size, &cqb, &index, &inlen);
+ if (err)
+ goto err_create;
+ }
+
+ cq->cqe_size = cqe_size;
+ cqb->ctx_ex.ctx.log_cq_sz = ilog2(entries);
+ cqb->ctx_ex.ctx.glb_func_id = cpu_to_be16(dev->xdev->glb_func_id);
+
+ err = xsc_vector2eqn(dev->xdev, vector, &eqn, &irqn);
+ if (err)
+ goto err_cqb;
+
+ cqb->ctx_ex.ctx.eqn = eqn;
+ cqb->ctx_ex.ctx.eqn = cpu_to_be16(cqb->ctx_ex.ctx.eqn);
+ cqb->ctx_ex.page_shift = PAGE_SHIFT;
+ if (cq->create_flags & IB_UVERBS_CQ_FLAGS_IGNORE_OVERRUN)
+ cqb->ctx_ex.overrun_ignore = 1;
+
+ err = xsc_core_create_cq(dev->xdev, &cq->xcq, cqb, inlen);
+ if (err)
+ goto err_cqb;
+
+ ibdev_dbg(&dev->ib_dev, "succeeded to create cqn %d, vector=%d, cq_sz=%d, eqn=%d\n",
+ cq->xcq.cqn, vector, entries, eqn);
+ cq->xcq.irqn = irqn;
+ cq->xcq.comp = xsc_ib_cq_comp;
+ cq->xcq.event = xsc_ib_cq_event;
+
+ if (udata) {
+ if (ib_copy_to_udata(udata, &cq->xcq.cqn, sizeof(__u32))) {
+ err = -EFAULT;
+ goto err_cmd;
+ }
+ }
+
+ INIT_LIST_HEAD(&cq->send_qp_list);
+ INIT_LIST_HEAD(&cq->recv_qp_list);
+ INIT_LIST_HEAD(&cq->err_state_qp_list);
+ xsc_vfree(cqb);
+
+ return 0;
+
+err_cmd:
+ xsc_core_destroy_cq(dev->xdev, &cq->xcq);
+
+err_cqb:
+ xsc_vfree(cqb);
+ if (udata)
+ destroy_cq_user(cq, udata);
+ else
+ destroy_cq_kernel(dev, cq);
+
+err_create:
+
+ return err;
+}
+
+int xsc_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev = to_xdev(cq->device);
+ struct xsc_ib_cq *xcq = to_xcq(cq);
+ struct xsc_err_state_qp_node *tmp = NULL, *err_qp_node = NULL;
+
+ list_for_each_entry_safe(err_qp_node, tmp, &xcq->err_state_qp_list, entry) {
+ list_del(&err_qp_node->entry);
+ kfree(err_qp_node);
+ }
+
+ xsc_core_destroy_cq(dev->xdev, &xcq->xcq);
+ if (udata)
+ destroy_cq_user(xcq, udata);
+ else
+ destroy_cq_kernel(dev, xcq);
+
+ return 0;
+}
+
+static int is_equal_rsn(struct xsc_cqe *cqe, u32 rsn)
+{
+ u32 qpn = le32_to_cpu(cqe->qp_id);
+ return rsn == qpn;
+}
+
+void __xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 rsn)
+{
+ struct xsc_cqe *cqe, *dest;
+ u32 prod_index;
+ int nfreed = 0;
+ u8 owner_bit;
+
+ if (!cq)
+ return;
+
+ /* First we need to find the current producer index, so we
+ * know where to start cleaning from. It doesn't matter if HW
+ * adds new entries after this loop -- the QP we're worried
+ * about is already in RESET, so the new entries won't come
+ * from our QP and therefore don't need to be checked.
+ */
+ for (prod_index = cq->xcq.cons_index; get_sw_cqe(cq, prod_index); prod_index++)
+ if (prod_index == cq->xcq.cons_index + cq->ibcq.cqe)
+ break;
+
+ /* Now sweep backwards through the CQ, removing CQ entries
+ * that match our QP by copying older entries on top of them.
+ */
+ while ((int)(--prod_index) - (int)cq->xcq.cons_index >= 0) {
+ cqe = (struct xsc_cqe *)get_cqe(cq, prod_index & (cq->ibcq.cqe - 1));
+ if (is_equal_rsn(cqe, rsn)) {
+ ++nfreed;
+ } else if (nfreed) {
+ dest = (struct xsc_cqe *)get_cqe(cq, (prod_index + nfreed) &
+ (cq->ibcq.cqe - 1));
+ owner_bit = dest->owner & XSC_CQE_OWNER_MASK;
+ memcpy(dest, cqe, cq->xcq.cqe_sz);
+ dest->owner = owner_bit |
+ (dest->owner & ~XSC_CQE_OWNER_MASK);
+ }
+ }
+
+ if (nfreed) {
+ cq->xcq.cons_index += nfreed;
+ xsc_update_cq_ci(cq->xcq.dev, cq->xcq.cqn, cq->xcq.cons_index);
+ }
+}
+
+void xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 qpn)
+{
+ if (!cq)
+ return;
+
+ spin_lock_irq(&cq->lock);
+ __xsc_ib_cq_clean(cq, qpn);
+ spin_unlock_irq(&cq->lock);
+}
diff --git a/drivers/infiniband/hw/xsc/main.c b/drivers/infiniband/hw/xsc/main.c
new file mode 100644
index 0000000000000..38c8b4f0690c2
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/main.c
@@ -0,0 +1,887 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/module.h>
+#include <linux/auxiliary_bus.h>
+#include <linux/init.h>
+#include <linux/errno.h>
+#include <linux/pci.h>
+#include <linux/namei.h>
+#include <linux/kobject.h>
+#include <linux/kernfs.h>
+#include <linux/dma-mapping.h>
+#include <linux/slab.h>
+#include <linux/io-mapping.h>
+#include <linux/sched.h>
+#include <linux/netdevice.h>
+#include "common/xsc_core.h"
+#include "common/xsc_hsi.h"
+#include "common/xsc_cmd.h"
+#include "common/driver.h"
+
+#include <rdma/ib_user_verbs.h>
+#include <rdma/ib_smi.h>
+#include <rdma/ib_umem.h>
+
+#include <rdma/xsc-abi.h>
+#include "xsc_ib.h"
+
+#define DRIVER_NAME "xsc_ib"
+
+#define XSC_IB_GID_TBL_LEN 256
+#define XSC_IB_MAX_PKEYS 0x80
+
+MODULE_DESCRIPTION("Yunsilicon HCA IB driver");
+MODULE_LICENSE("GPL");
+
+static int xsc_ib_query_device(struct ib_device *ibdev,
+ struct ib_device_attr *props,
+ struct ib_udata *udata)
+{
+ size_t udata_outlen = (udata) ? udata->outlen : 0;
+ struct xsc_ib_dev *dev = to_xdev(ibdev);
+ struct xsc_ib_query_device_resp resp;
+ union xsc_ib_fw_ver fw_ver;
+ int err = -ENOMEM;
+ size_t resp_len;
+ int max_rq_sg;
+ u64 max_tso;
+ u64 flags;
+
+ err = ib_is_udata_in_empty(udata);
+ if (err)
+ return err;
+
+ memset(&resp, 0, sizeof(resp));
+ memset(props, 0, sizeof(*props));
+
+ resp_len = sizeof(resp.comp_mask) + sizeof(resp.response_length);
+ /*check param*/
+ if (udata_outlen && udata_outlen < resp_len)
+ return -EINVAL;
+
+ resp.response_length = resp_len;
+
+ fw_ver.data = 0;
+ fw_ver.s.ver_major = dev->xdev->fw_version_major;
+ fw_ver.s.ver_minor = dev->xdev->fw_version_minor;
+ fw_ver.s.ver_patch = dev->xdev->fw_version_patch;
+ fw_ver.s.ver_tweak = dev->xdev->fw_version_tweak;
+ props->fw_ver = fw_ver.data;
+
+ props->device_cap_flags = IB_DEVICE_CHANGE_PHY_PORT |
+ IB_DEVICE_PORT_ACTIVE_EVENT |
+ IB_DEVICE_SYS_IMAGE_GUID |
+ IB_DEVICE_RC_RNR_NAK_GEN;
+ props->kernel_cap_flags = IBK_BLOCK_MULTICAST_LOOPBACK;
+ props->kernel_cap_flags |= IBK_LOCAL_DMA_LKEY;
+
+ flags = dev->xdev->caps.flags;
+ if (flags & XSC_DEV_CAP_FLAG_BAD_PKEY_CNTR)
+ props->device_cap_flags |= IB_DEVICE_BAD_PKEY_CNTR;
+ if (flags & XSC_DEV_CAP_FLAG_BAD_QKEY_CNTR)
+ props->device_cap_flags |= IB_DEVICE_BAD_QKEY_CNTR;
+ if (flags & XSC_DEV_CAP_FLAG_APM)
+ props->device_cap_flags |= IB_DEVICE_AUTO_PATH_MIG;
+ props->page_size_cap = dev->xdev->caps.min_page_sz;
+ props->max_mr_size = xsc_get_max_mr_size(dev->xdev);
+ props->max_qp = dev->xdev->caps.max_qp;
+ props->max_qp_wr = xsc_get_max_qp_depth(dev->xdev);
+ max_rq_sg = dev->xdev->caps.max_rq_desc_sz / sizeof(struct xsc_wqe_data_seg);
+
+ props->max_send_sge = dev->xdev->caps.send_ds_num - XSC_CTRL_SEG_NUM -
+ XSC_RADDR_SEG_NUM;
+ props->max_recv_sge = dev->xdev->caps.recv_ds_num;
+ props->max_sge_rd = 1;/*max sge per read wqe*/
+ props->max_cq = dev->xdev->caps.max_cq;
+ props->max_cqe = dev->xdev->caps.max_cqes;
+ props->max_mr = (1 << dev->xdev->caps.log_max_mkey) - 1;
+ props->max_pd = dev->xdev->caps.max_pd;
+ props->max_qp_rd_atom = dev->xdev->caps.max_ra_res_qp;
+ props->max_qp_init_rd_atom = dev->xdev->caps.max_ra_req_qp;
+ props->max_res_rd_atom = props->max_qp_rd_atom * props->max_qp;
+ props->max_srq =
+ dev->xdev->caps.log_max_srq ? (1 << dev->xdev->caps.log_max_srq) : 0;
+ props->max_srq_wr = dev->xdev->caps.max_srq_wqes - 1;
+ props->max_srq_sge = dev->xdev->caps.log_max_srq ? (max_rq_sg - 1) : 0;
+ props->local_ca_ack_delay = dev->xdev->caps.local_ca_ack_delay;
+ props->atomic_cap = is_support_rdma_atomic(dev->xdev) ?
+ IB_ATOMIC_HCA : IB_ATOMIC_NONE;
+ props->masked_atomic_cap = IB_ATOMIC_HCA;
+ props->max_mcast_grp =
+ dev->xdev->caps.log_max_mcg ? (1 << dev->xdev->caps.log_max_mcg) : 0;
+ props->max_mcast_qp_attach = dev->xdev->caps.max_qp_mcg;
+ props->max_total_mcast_qp_attach = props->max_mcast_qp_attach *
+ props->max_mcast_grp;
+
+ props->sys_image_guid = dev->xdev->board_info->guid;
+ props->vendor_id = dev->xdev->pdev->vendor;
+ props->vendor_part_id = dev->xdev->pdev->device;
+ props->hw_ver = ((dev->xdev->chip_ver_l & 0xffff) << 16) |
+ (dev->xdev->hotfix_num & 0xffff);
+ props->max_pkeys = XSC_IB_MAX_PKEYS;
+ props->max_wq_type_rq = dev->xdev->caps.max_qp;
+ props->hca_core_clock = dev->xdev->caps.hca_core_clock * 1000; /* KHz */
+ props->rss_caps.max_rwq_indirection_tables =
+ dev->xdev->caps.max_rwq_indirection_tables;
+ props->rss_caps.max_rwq_indirection_table_size =
+ dev->xdev->caps.max_rwq_indirection_table_size;
+ props->rss_caps.supported_qpts = 1 << IB_QPT_RAW_PACKET;
+
+ /*response tso_caps extend param*/
+ if (offsetofend(typeof(resp), tso_caps) <= udata_outlen) {
+ max_tso = dev->xdev->caps.log_max_tso ? (1 << dev->xdev->caps.log_max_tso) : 0;
+ if (max_tso) {
+ resp.tso_caps.max_tso = max_tso;
+ resp.tso_caps.supported_qpts |= 1 << IB_QPT_RAW_PACKET;
+ resp.response_length += sizeof(resp.tso_caps);
+ }
+ }
+
+ /*response rss_caps extend param*/
+ if (offsetofend(typeof(resp), rss_caps) <= udata_outlen) {
+ resp.rss_caps.rx_hash_function = XSC_RX_HASH_FUNC_TOEPLITZ;
+ resp.rss_caps.rx_hash_fields_mask =
+ XSC_RX_HASH_SRC_IPV4 |
+ XSC_RX_HASH_DST_IPV4 |
+ XSC_RX_HASH_SRC_IPV6 |
+ XSC_RX_HASH_DST_IPV6 |
+ XSC_RX_HASH_SRC_PORT_TCP |
+ XSC_RX_HASH_DST_PORT_TCP |
+ XSC_RX_HASH_SRC_PORT_UDP |
+ XSC_RX_HASH_DST_PORT_UDP |
+ XSC_RX_HASH_INNER;
+ resp.response_length += sizeof(resp.rss_caps);
+ }
+
+ /*response packet pacing caps*/
+ if (offsetofend(typeof(resp), packet_pacing_caps) <= udata_outlen) {
+ resp.packet_pacing_caps.qp_rate_limit_max =
+ dev->xdev->caps.qp_rate_limit_max;
+ resp.packet_pacing_caps.qp_rate_limit_min =
+ dev->xdev->caps.qp_rate_limit_min;
+ resp.packet_pacing_caps.supported_qpts |= 1 << IB_QPT_RAW_PACKET;
+
+ resp.response_length += sizeof(resp.packet_pacing_caps);
+ }
+
+ /*copy response data to user*/
+ if (udata_outlen) {
+ err = ib_copy_to_udata(udata, &resp, resp.response_length);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "copy response info to udata fail,err=%d\n", err);
+ return err;
+ }
+ }
+
+ return 0;
+}
+
+static void xsc_calc_link_info(struct xsc_core_device *xdev,
+ struct ib_port_attr *props)
+{
+ switch (xsc_get_link_speed(xdev)) {
+ case MODULE_SPEED_10G:
+ props->active_speed = XSC_RDMA_LINK_SPEED_10GB;
+ props->active_width = 1;
+ break;
+ case MODULE_SPEED_25G:
+ props->active_speed = XSC_RDMA_LINK_SPEED_25GB;
+ props->active_width = 1;
+ break;
+ case MODULE_SPEED_40G_R4:
+ props->active_speed = XSC_RDMA_LINK_SPEED_10GB;
+ props->active_width = 2;
+ break;
+ case MODULE_SPEED_50G_R:
+ props->active_speed = XSC_RDMA_LINK_SPEED_50GB;
+ props->active_width = 1;
+ break;
+ case MODULE_SPEED_50G_R2:
+ props->active_speed = XSC_RDMA_LINK_SPEED_50GB;
+ props->active_width = 1;
+ break;
+ case MODULE_SPEED_100G_R2:
+ props->active_speed = XSC_RDMA_LINK_SPEED_25GB;
+ props->active_width = 2;
+ break;
+ case MODULE_SPEED_100G_R4:
+ props->active_speed = XSC_RDMA_LINK_SPEED_25GB;
+ props->active_width = 2;
+ break;
+ case MODULE_SPEED_200G_R4:
+ props->active_speed = XSC_RDMA_LINK_SPEED_50GB;
+ props->active_width = 2;
+ break;
+ case MODULE_SPEED_200G_R8:
+ props->active_speed = XSC_RDMA_LINK_SPEED_25GB;
+ props->active_width = 4;
+ break;
+ case MODULE_SPEED_400G_R8:
+ props->active_speed = XSC_RDMA_LINK_SPEED_50GB;
+ props->active_width = 4;
+ break;
+ case MODULE_SPEED_400G_R4:
+ props->active_speed = XSC_RDMA_LINK_SPEED_100GB;
+ props->active_width = 2;
+ break;
+ default:
+ props->active_speed = XSC_RDMA_LINK_SPEED_25GB;
+ props->active_width = 1;
+ break;
+ }
+}
+
+static enum rdma_link_layer xsc_ib_port_link_layer(struct ib_device *ibdev, u32 port)
+{
+ return IB_LINK_LAYER_ETHERNET;
+}
+
+int xsc_ib_query_port(struct ib_device *ibdev, u32 port,
+ struct ib_port_attr *props)
+{
+ struct xsc_ib_dev *dev = to_xdev(ibdev);
+ struct net_device *ndev = dev->netdev;
+ struct xsc_core_device *xdev = dev->xdev;
+
+ if (port < 1 || port > xdev->caps.num_ports) {
+ ibdev_warn(&dev->ib_dev, "invalid port number %d\n", port);
+ return -EINVAL;
+ }
+
+ memset(props, 0, sizeof(*props));
+
+ if (netif_running(ndev) && netif_carrier_ok(ndev))
+ props->state = IB_PORT_ACTIVE;
+ else
+ props->state = IB_PORT_DOWN;
+
+ props->max_mtu = is_support_8k_mtu(xdev) ? IB_MTU_8192 : IB_MTU_4096;
+ props->active_mtu = min(props->max_mtu, xsc_net_to_ib_mtu(ndev->mtu));
+ props->gid_tbl_len = XSC_IB_GID_TBL_LEN;
+ props->port_cap_flags = IB_PORT_CM_SUP | IB_PORT_OTHER_LOCAL_CHANGES_SUP;
+ props->max_msg_sz = SZ_1G;
+ props->bad_pkey_cntr = 0;
+ props->qkey_viol_cntr = 0;
+ props->pkey_tbl_len = 1;
+ props->lid = 0;
+ props->sm_lid = 0;
+ props->lmc = 0;
+ props->max_vl_num = 0;
+ props->sm_sl = 0;
+ props->subnet_timeout = 0;
+ props->init_type_reply = 0;
+ if (!is_support_rdma(xdev)) {
+ props->active_width = 1;
+ props->active_speed = XSC_RDMA_LINK_SPEED_25GB;
+ } else {
+ xsc_calc_link_info(xdev, props);
+ }
+
+ props->phys_state = netif_carrier_ok(ndev) ? XSC_RDMA_PHY_STATE_LINK_UP :
+ XSC_RDMA_PHY_STATE_DISABLED;
+ return 0;
+}
+
+static int xsc_ib_query_pkey(struct ib_device *ibdev, u32 port, u16 index,
+ u16 *pkey)
+{
+ *pkey = 0xffff;
+ return 0;
+}
+
+int xsc_ib_alloc_ucontext(struct ib_ucontext *uctx,
+ struct ib_udata *udata)
+{
+ struct ib_device *ibdev = uctx->device;
+ struct xsc_ib_dev *dev = to_xdev(ibdev);
+ struct xsc_ib_alloc_ucontext_req req;
+ struct xsc_ib_alloc_ucontext_resp resp;
+ int err;
+
+ if (!dev->ib_active)
+ return -EAGAIN;
+
+ err = ib_copy_from_udata(&req, udata, sizeof(req));
+ if (err)
+ return err;
+
+ memset(&resp, 0, sizeof(resp));
+ resp.qp_tab_size = dev->xdev->caps.max_qp;
+ resp.cache_line_size = L1_CACHE_BYTES;
+ resp.max_sq_desc_sz = dev->xdev->caps.max_sq_desc_sz;
+ resp.max_rq_desc_sz = dev->xdev->caps.max_rq_desc_sz;
+ resp.max_send_wqebb = dev->xdev->caps.max_wqes;
+ resp.max_recv_wr = dev->xdev->caps.max_wqes;
+ xsc_get_db_addr(dev->xdev, &resp.qpm_tx_db, &resp.qpm_rx_db,
+ &resp.cqm_armdb, &resp.cqm_next_cid_reg, NULL);
+ resp.send_ds_num = dev->xdev->caps.send_ds_num;
+ resp.recv_ds_num = dev->xdev->caps.recv_ds_num;
+ resp.cmds_supp_uhw |= XSC_USER_CMDS_SUPP_UHW_QUERY_DEVICE;
+ resp.device_id = dev->xdev->pdev->device;
+
+ if (req.abi_ver >= ALLOC_UCONTEXT_VER_V1) {
+ xsc_get_multidb_info(dev->xdev, &resp.multidb_num, &resp.tx_multidb_base);
+ resp.rdma_proto_mode = dev->xdev->rdma_proto_mode;
+ }
+
+ if (is_support_cqe64(dev->xdev))
+ resp.hw_feature_flag |= XSC_HW_FEATURE_FLAG_SUPPORT_CQE64;
+
+ resp.num_ports = dev->xdev->caps.num_ports;
+
+ return ib_copy_to_udata(udata, &resp, min_t(int, udata->outlen, sizeof(resp)));
+}
+
+void xsc_ib_dealloc_ucontext(struct ib_ucontext *ibcontext)
+{
+}
+
+static int xsc_ib_mmap(struct ib_ucontext *ibcontext, struct vm_area_struct *vma)
+{
+ struct xsc_ib_dev *dev = to_xdev(ibcontext->device);
+ struct xsc_core_device *xdev = dev->xdev;
+ unsigned long offset = vma->vm_pgoff << PAGE_SHIFT;
+ resource_size_t reg_base;
+ resource_size_t reg_size = vma->vm_end - vma->vm_start;
+ u64 tx_db = 0;
+ u64 rx_db = 0;
+ u64 cq_db = 0;
+ u64 cq_reg = 0;
+
+ xsc_get_db_addr(xdev, &tx_db, &rx_db, &cq_db, &cq_reg, NULL);
+ dev_dbg(&xdev->pdev->dev, "offset:0x%lx\n", offset);
+
+ if (offset == (tx_db & PAGE_MASK))
+ reg_base = pci_resource_start(xdev->pdev, xdev->bar_num) + (tx_db & PAGE_MASK);
+ else if (offset == (rx_db & PAGE_MASK))
+ reg_base = pci_resource_start(xdev->pdev, xdev->bar_num) + (rx_db & PAGE_MASK);
+ else if (offset == (cq_reg & PAGE_MASK))
+ reg_base = pci_resource_start(xdev->pdev, xdev->bar_num) + (cq_reg & PAGE_MASK);
+ else if (offset == (cq_db & PAGE_MASK))
+ reg_base = pci_resource_start(xdev->pdev, xdev->bar_num) + (cq_db & PAGE_MASK);
+ else if (offset == (QPM_TX_MDB_BASE_REG_ADDR & PAGE_MASK))
+ reg_base = pci_resource_start(xdev->pdev, xdev->bar_num) +
+ (QPM_TX_MDB_BASE_REG_ADDR & PAGE_MASK);
+ else if (offset == (xdev->caps.mdb_base & PAGE_MASK))
+ reg_base = pci_resource_start(xdev->pdev, xdev->bar_num) +
+ (xdev->caps.mdb_base & PAGE_MASK);
+ else
+ return -EINVAL;
+
+ dev_dbg(&xdev->pdev->dev, "regbase:0x%llx", reg_base);
+
+ reg_base = (xsc_core_is_pf(xdev) && !is_pf_bar_compressed(xdev)) ?
+ reg_base - 0xA0000000 : reg_base;
+
+ vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
+ return remap_pfn_range(vma, vma->vm_start, reg_base >> PAGE_SHIFT,
+ reg_size, vma->vm_page_prot);
+}
+
+int xsc_ib_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata)
+{
+ struct ib_device *ibdev = ibpd->device;
+ struct xsc_ib_alloc_pd_resp resp;
+ struct xsc_ib_pd *pd;
+ int err;
+
+ pd = to_xpd(ibpd);
+
+ err = xsc_core_alloc_pd(to_xdev(ibdev)->xdev, &pd->pdn);
+ if (err)
+ return err;
+
+ if (udata) {
+ resp.pdn = pd->pdn;
+ if (ib_copy_to_udata(udata, &resp, sizeof(resp))) {
+ xsc_core_dealloc_pd(to_xdev(ibdev)->xdev, pd->pdn);
+ return -EFAULT;
+ }
+ } else {
+ pd->pa_lkey = 0;
+ }
+
+ return 0;
+}
+
+int xsc_ib_dealloc_pd(struct ib_pd *pd, struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev = to_xdev(pd->device);
+ struct xsc_ib_pd *xpd = to_xpd(pd);
+
+ xsc_core_dealloc_pd(dev->xdev, xpd->pdn);
+
+ return 0;
+}
+
+static int xsc_port_immutable(struct ib_device *ibdev, u32 port_num,
+ struct ib_port_immutable *immutable)
+{
+ struct ib_port_attr attr;
+ int err;
+
+ err = ib_query_port(ibdev, port_num, &attr);
+ if (err)
+ return err;
+
+ immutable->pkey_tbl_len = attr.pkey_tbl_len;
+ immutable->gid_tbl_len = attr.gid_tbl_len;
+ immutable->core_cap_flags = RDMA_CORE_PORT_IBA_ROCE |
+ RDMA_CORE_PORT_IBA_ROCE_UDP_ENCAP;
+ immutable->max_mad_size = IB_MGMT_MAD_SIZE * 2;
+
+ return 0;
+}
+
+static struct net_device *xsc_get_netdev(struct ib_device *ibdev, u32 port_num)
+{
+ struct xsc_ib_dev *xsc_ib_dev = to_xdev(ibdev);
+ struct net_device *dev = xsc_ib_dev->netdev;
+
+ if (dev)
+ dev_hold(dev);
+
+ return dev;
+}
+
+static void xsc_get_guid(const u8 *dev_addr, u8 *guid)
+{
+ u8 mac[ETH_ALEN];
+
+ /* MAC-48 to EUI-64 mapping */
+ memcpy(mac, dev_addr, ETH_ALEN);
+ guid[0] = mac[0] ^ 2;
+ guid[1] = mac[1];
+ guid[2] = mac[2];
+ guid[3] = 0xff;
+ guid[4] = 0xfe;
+ guid[5] = mac[3];
+ guid[6] = mac[4];
+ guid[7] = mac[5];
+}
+
+static void init_node_data(struct xsc_ib_dev *dev)
+{
+ strscpy(dev->ib_dev.node_desc, "xsc_node_desc",
+ sizeof(dev->ib_dev.node_desc));
+ xsc_get_guid(dev->netdev->dev_addr, (u8 *)&dev->ib_dev.node_guid);
+}
+
+static void xsc_ib_event(struct xsc_core_device *dev, void *context,
+ enum xsc_dev_event event, unsigned long data)
+{
+ struct xsc_ib_dev *ibdev = (struct xsc_ib_dev *)context;
+ struct ib_event ibev;
+ u8 port = 0;
+
+ switch (event) {
+ case XSC_DEV_EVENT_SYS_ERROR:
+ ibdev->ib_active = false;
+ ibev.event = IB_EVENT_DEVICE_FATAL;
+ break;
+
+ case XSC_DEV_EVENT_PORT_UP:
+ ibev.event = IB_EVENT_PORT_ACTIVE;
+ port = *(u8 *)data;
+ break;
+
+ case XSC_DEV_EVENT_PORT_DOWN:
+ ibev.event = IB_EVENT_PORT_ERR;
+ port = *(u8 *)data;
+ break;
+
+ case XSC_DEV_EVENT_PORT_INITIALIZED:
+ /* not used by ULPs */
+ return;
+
+ case XSC_DEV_EVENT_LID_CHANGE:
+ ibev.event = IB_EVENT_LID_CHANGE;
+ port = *(u8 *)data;
+ break;
+
+ case XSC_DEV_EVENT_PKEY_CHANGE:
+ ibev.event = IB_EVENT_PKEY_CHANGE;
+ port = *(u8 *)data;
+ break;
+
+ case XSC_DEV_EVENT_GUID_CHANGE:
+ ibev.event = IB_EVENT_GID_CHANGE;
+ port = *(u8 *)data;
+ break;
+
+ case XSC_DEV_EVENT_CLIENT_REREG:
+ ibev.event = IB_EVENT_CLIENT_REREGISTER;
+ port = *(u8 *)data;
+ break;
+ }
+
+ ibev.device = &ibdev->ib_dev;
+ ibev.element.port_num = port;
+
+ if (ibdev->ib_active)
+ ib_dispatch_event(&ibev);
+}
+
+static void crc_table_init(struct xsc_ib_dev *dev)
+{
+ u32 c, i, j;
+
+ for (i = 0; i < 256; i++) {
+ c = i;
+ for (j = 0; j < 8; j++) {
+ if (c & 1)
+ c = 0xedb88320L ^ (c >> 1);
+ else
+ c = c >> 1;
+ }
+ dev->crc_32_table[i] = c;
+ }
+}
+
+static void xsc_ib_get_dev_fw_str(struct ib_device *ibdev, char *str)
+{
+ struct xsc_core_device *dev = to_xdev(ibdev)->xdev;
+ u8 ver_major = dev->fw_version_major;
+ u8 ver_minor = dev->fw_version_minor;
+ u16 ver_patch = dev->fw_version_patch;
+ u32 ver_tweak = dev->fw_version_tweak;
+
+ if (ver_tweak == 0) {
+ snprintf(str, IB_FW_VERSION_NAME_MAX, "v%u.%u.%u",
+ ver_major, ver_minor, ver_patch);
+ } else {
+ snprintf(str, IB_FW_VERSION_NAME_MAX, "v%u.%u.%u+%u",
+ ver_major, ver_minor, ver_patch, ver_tweak);
+ }
+}
+
+static void xsc_ib_disassociate_ucontext(struct ib_ucontext *ibcontext)
+{
+}
+
+static void xsc_ib_dev_setting(struct xsc_ib_dev *dev)
+{
+ dev->ib_dev.ops.owner = THIS_MODULE;
+ dev->ib_dev.ops.uverbs_abi_ver = XSC_IB_UVERBS_ABI_VERSION;
+ dev->ib_dev.ops.driver_id = RDMA_DRIVER_XSC;
+ dev->ib_dev.ops.disassociate_ucontext = xsc_ib_disassociate_ucontext;
+ dev->ib_dev.ops.query_device = xsc_ib_query_device;
+ dev->ib_dev.ops.query_port = xsc_ib_query_port;
+ dev->ib_dev.ops.query_pkey = xsc_ib_query_pkey;
+
+ dev->ib_dev.ops.alloc_ucontext = xsc_ib_alloc_ucontext;
+ dev->ib_dev.ops.dealloc_ucontext = xsc_ib_dealloc_ucontext;
+ dev->ib_dev.ops.mmap = xsc_ib_mmap;
+
+ dev->ib_dev.ops.alloc_pd = xsc_ib_alloc_pd;
+ dev->ib_dev.ops.dealloc_pd = xsc_ib_dealloc_pd;
+ dev->ib_dev.ops.create_ah = xsc_ib_create_ah;
+ dev->ib_dev.ops.query_ah = xsc_ib_query_ah;
+ dev->ib_dev.ops.destroy_ah = xsc_ib_destroy_ah;
+
+ dev->ib_dev.ops.get_link_layer = xsc_ib_port_link_layer;
+ dev->ib_dev.ops.get_netdev = xsc_get_netdev;
+
+ dev->ib_dev.ops.create_qp = xsc_ib_create_qp;
+ dev->ib_dev.ops.modify_qp = xsc_ib_modify_qp;
+ dev->ib_dev.ops.query_qp = xsc_ib_query_qp;
+ dev->ib_dev.ops.destroy_qp = xsc_ib_destroy_qp;
+ dev->ib_dev.ops.post_send = xsc_ib_post_send;
+ dev->ib_dev.ops.post_recv = xsc_ib_post_recv;
+ dev->ib_dev.ops.create_cq = xsc_ib_create_cq;
+ dev->ib_dev.ops.destroy_cq = xsc_ib_destroy_cq;
+ dev->ib_dev.ops.poll_cq = xsc_ib_poll_cq;
+ dev->ib_dev.ops.req_notify_cq = xsc_ib_arm_cq;
+ dev->ib_dev.ops.get_dma_mr = xsc_ib_get_dma_mr;
+ dev->ib_dev.ops.reg_user_mr = xsc_ib_reg_user_mr; /* optional */
+ dev->ib_dev.ops.reg_user_mr_dmabuf = xsc_ib_reg_user_mr_dmabuf;
+ dev->ib_dev.ops.dereg_mr = xsc_ib_dereg_mr;
+
+ dev->ib_dev.ops.get_port_immutable = xsc_port_immutable;
+
+ dev->ib_dev.ops.drain_sq = xsc_ib_drain_sq;
+ dev->ib_dev.ops.drain_rq = xsc_ib_drain_rq;
+ dev->ib_dev.ops.get_dev_fw_str = xsc_ib_get_dev_fw_str;
+ dev->ib_dev.ops.create_srq = xsc_ib_create_srq;
+ dev->ib_dev.ops.destroy_srq = xsc_ib_destroy_srq;
+ dev->ib_dev.ops.query_srq = xsc_ib_query_srq;
+ dev->ib_dev.ops.modify_srq = xsc_ib_modify_srq;
+ dev->ib_dev.ops.post_srq_recv = xsc_ib_post_srq_recv;
+
+ dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_ucontext, xsc_ib_ucontext, ibucontext);
+ dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_ah, xsc_ib_ah, ibah);
+ dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_cq, xsc_ib_cq, ibcq);
+ dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_pd, xsc_ib_pd, ibpd);
+ dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_qp, xsc_ib_qp, ibqp);
+ dev->ib_dev.ops INIT_RDMA_OBJ_SIZE(ib_srq, xsc_ib_srq, ibsrq);
+}
+
+static void xsc_get_port_state(struct net_device *ndev, enum xsc_dev_event *ev)
+{
+ *ev = XSC_DEV_EVENT_PORT_DOWN;
+ if (netif_running(ndev) && netif_carrier_ok(ndev))
+ *ev = XSC_DEV_EVENT_PORT_UP;
+}
+
+static int xsc_netdev_event(struct notifier_block *this, unsigned long event, void *ptr)
+{
+ struct xsc_ib_dev *ibdev = container_of(this, struct xsc_ib_dev, nb);
+ struct net_device *ndev = netdev_notifier_info_to_dev(ptr);
+ enum xsc_dev_event ev;
+ u8 port = 1;
+
+ if (ndev != ibdev->netdev)
+ goto done;
+
+ ibdev_dbg(&ibdev->ib_dev, "netdev notfiy event:%ld\n", event);
+ switch (event) {
+ case NETDEV_CHANGE:
+ case NETDEV_UP:
+ case NETDEV_DOWN:
+ xsc_get_port_state(ibdev->netdev, &ev);
+ xsc_ib_event(ibdev->xdev, ibdev, ev, (unsigned long)&port);
+ break;
+ default:
+ break;
+ }
+done:
+ return NOTIFY_DONE;
+}
+
+static int xsc_register_netdev_notifier(struct xsc_ib_dev *ibdev)
+{
+ ibdev->nb.notifier_call = xsc_netdev_event;
+ return register_netdevice_notifier(&ibdev->nb);
+}
+
+static int xsc_unregister_netdev_notifier(struct xsc_ib_dev *ibdev)
+{
+ return unregister_netdevice_notifier(&ibdev->nb);
+}
+
+static int init_one(struct xsc_core_device *xdev,
+ struct xsc_ib_dev **ibdev)
+{
+ struct xsc_ib_dev *dev;
+ int err;
+
+ dev = (struct xsc_ib_dev *)ib_alloc_device(xsc_ib_dev, ib_dev);
+ if (!dev)
+ return -ENOMEM;
+
+ dev->xdev = xdev;
+ dev->netdev = xdev->netdev;
+ strscpy(dev->ib_dev.name, "xscale_%d", IB_DEVICE_NAME_MAX);
+
+ memcpy(dev->xdev->priv.ibdev_name, dev->ib_dev.name, IB_DEVICE_NAME_MAX);
+
+ if (!xdev->caps.msix_enable)
+ dev->num_comp_vectors = 1;
+ else
+ dev->num_comp_vectors = xdev->dev_res->eq_table.num_comp_vectors;
+
+ dev->ib_dev.node_type = RDMA_NODE_IB_CA;
+ dev->ib_dev.local_dma_lkey = 0xFF;
+ dev->num_ports = xdev->caps.num_ports;
+ dev->ib_dev.phys_port_cnt = dev->num_ports;
+ dev->ib_dev.num_comp_vectors = dev->num_comp_vectors;
+ dev->ib_dev.dev.parent = &xdev->pdev->dev;
+ xsc_ib_dev_setting(dev);
+ dev->xdev->priv.cm_dscp = DSCP_PCP_UNSET;
+ dev->xdev->priv.cm_pcp = DSCP_PCP_UNSET;
+ dev->xdev->priv.force_pcp = DSCP_PCP_UNSET;
+ dev->xdev->priv.force_dscp = DSCP_PCP_UNSET;
+ dev->xdev->rdma_proto_mode = RDMA_PROTO_ROCEV2;
+ dev->xdev->cm_udp_dst_port = ROCE_V2_UDP_DPORT;
+ dev->xdev->veroce_udp_dst_port = VEROCE_UDP_DPORT;
+
+ dev->ib_dev.uverbs_cmd_mask =
+ (1ull << IB_USER_VERBS_CMD_GET_CONTEXT) |
+ (1ull << IB_USER_VERBS_CMD_QUERY_DEVICE) |
+ (1ull << IB_USER_VERBS_CMD_QUERY_PORT) |
+ (1ull << IB_USER_VERBS_CMD_ALLOC_PD) |
+ (1ull << IB_USER_VERBS_CMD_DEALLOC_PD) |
+ (1ull << IB_USER_VERBS_CMD_CREATE_AH) |
+ (1ull << IB_USER_VERBS_CMD_DESTROY_AH) |
+ (1ull << IB_USER_VERBS_CMD_REG_MR) |
+ (1ull << IB_USER_VERBS_CMD_REREG_MR) |
+ (1ull << IB_USER_VERBS_CMD_DEREG_MR) |
+ (1ull << IB_USER_VERBS_CMD_CREATE_COMP_CHANNEL) |
+ (1ull << IB_USER_VERBS_CMD_CREATE_CQ) |
+ (1ull << IB_USER_VERBS_CMD_RESIZE_CQ) |
+ (1ull << IB_USER_VERBS_CMD_DESTROY_CQ) |
+ (1ull << IB_USER_VERBS_CMD_CREATE_QP) |
+ (1ull << IB_USER_VERBS_CMD_MODIFY_QP) |
+ (1ull << IB_USER_VERBS_CMD_QUERY_QP) |
+ (1ull << IB_USER_VERBS_CMD_DESTROY_QP) |
+ (1ull << IB_USER_VERBS_CMD_ATTACH_MCAST) |
+ (1ull << IB_USER_VERBS_CMD_DETACH_MCAST) |
+ (1ull << IB_USER_VERBS_CMD_CREATE_SRQ) |
+ (1ull << IB_USER_VERBS_CMD_MODIFY_SRQ) |
+ (1ull << IB_USER_VERBS_CMD_QUERY_SRQ) |
+ (1ull << IB_USER_VERBS_CMD_DESTROY_SRQ) |
+ (1ull << IB_USER_VERBS_CMD_OPEN_QP);
+
+ init_node_data(dev);
+
+ crc_table_init(dev);
+
+ build_opcode_map(dev);
+
+ xsc_reg_local_dma_mr(xdev);
+
+ err = ib_register_device(&dev->ib_dev, dev->ib_dev.name,
+ dev->xdev->device);
+ if (err)
+ goto err_free;
+
+ rdma_roce_rescan_device(&dev->ib_dev);
+ dev->ib_active = true;
+ *ibdev = dev;
+
+ xsc_register_netdev_notifier(dev);
+
+ xsc_counters_init(&dev->ib_dev, xdev);
+
+ xsc_ib_sysfs_init(&dev->ib_dev, xdev);
+
+ return 0;
+
+err_free:
+ ib_dealloc_device((struct ib_device *)dev);
+
+ return err;
+}
+
+static void remove_one(struct xsc_core_device *xdev, void *intf_ctx)
+{
+ struct xsc_ib_dev *dev = (struct xsc_ib_dev *)intf_ctx;
+
+ xsc_ib_sysfs_fini(&dev->ib_dev, xdev);
+ xsc_counters_fini(&dev->ib_dev, xdev);
+ xsc_unregister_netdev_notifier(dev);
+ ib_unregister_device(&dev->ib_dev);
+ ib_dealloc_device(&dev->ib_dev);
+}
+
+static void init_iommu_state(struct xsc_ib_dev *dev)
+{
+ struct iommu_domain *domain;
+
+ dev->iommu_state = XSC_IB_IOMMU_MAP_DISABLE;
+ domain = iommu_get_domain_for_dev(dev->ib_dev.dma_device);
+ if (domain) {
+ if (domain->type & __IOMMU_DOMAIN_DMA_API)
+ dev->iommu_state = XSC_IB_IOMMU_MAP_NORMAL;
+ } else {
+ /* Try to allocate DMA memory; if the DMA address differs from
+ * the physical address, an IOMMU maps it through an unknown
+ * domain.
+ */
+ dma_addr_t dma_addr;
+ void *tmp;
+
+ tmp = dma_alloc_coherent(dev->ib_dev.dma_device, PAGE_SIZE,
+ &dma_addr, GFP_KERNEL);
+ if (tmp) {
+ if (virt_to_phys(tmp) != dma_addr)
+ dev->iommu_state = XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN;
+ dma_free_coherent(dev->ib_dev.dma_device, PAGE_SIZE,
+ tmp, dma_addr);
+ }
+ }
+
+ ibdev_dbg(&dev->ib_dev, "iommu dma map state=%d\n", dev->iommu_state);
+}
+
+static int xsc_ib_probe(struct auxiliary_device *adev,
+ const struct auxiliary_device_id *adev_id)
+{
+ struct xsc_adev *xsc_adev = container_of(adev, struct xsc_adev, adev);
+ struct xsc_core_device *xdev = xsc_adev->xdev;
+ struct xsc_ib_dev *ibdev = NULL;
+ int ret;
+
+ ret = init_one(xdev, &ibdev);
+ if (ret) {
+ dev_err(&adev->dev, "failed to add RDMA device, ret = %d\n",
+ ret);
+ return ret;
+ }
+
+ init_iommu_state(ibdev);
+ auxiliary_set_drvdata(adev, ibdev);
+
+ return 0;
+}
+
+static void xsc_ib_remove(struct auxiliary_device *adev)
+{
+ struct xsc_ib_dev *ibdev = auxiliary_get_drvdata(adev);
+
+ if (!ibdev)
+ return;
+
+ remove_one(ibdev->xdev, ibdev);
+}
+
+static const struct auxiliary_device_id xsc_ib_id_table[] = {
+ { .name = XSC_PCI_DRV_NAME "." XSC_IB_ADEV_NAME },
+ {},
+};
+MODULE_DEVICE_TABLE(auxiliary, xsc_ib_id_table);
+
+static struct auxiliary_driver xsc_ib_driver = {
+ .name = XSC_IB_ADEV_NAME,
+ .probe = xsc_ib_probe,
+ .remove = xsc_ib_remove,
+ .id_table = xsc_ib_id_table,
+};
+
+static int xsc_ib_reboot_event_handler(struct notifier_block *nb, unsigned long action, void *data)
+{
+ pr_info("xsc ib driver recv %lu event\n", action);
+
+ xsc_remove_rdma_driver();
+
+ return NOTIFY_OK;
+}
+
+struct notifier_block xsc_ib_nb = {
+ .notifier_call = xsc_ib_reboot_event_handler,
+ .next = NULL,
+ .priority = 2,
+};
+
+void xsc_remove_rdma_driver(void)
+{
+ auxiliary_driver_unregister(&xsc_ib_driver);
+}
+
+static int __init xsc_ib_init(void)
+{
+ int ret;
+
+ ret = auxiliary_driver_register(&xsc_ib_driver);
+ if (ret) {
+ pr_err("failed to register auxiliary driver\n");
+ return ret;
+ }
+
+ register_reboot_notifier(&xsc_ib_nb);
+
+ return 0;
+}
+
+static void __exit xsc_ib_cleanup(void)
+{
+ unregister_reboot_notifier(&xsc_ib_nb);
+ xsc_remove_rdma_driver();
+}
+
+module_init(xsc_ib_init);
+module_exit(xsc_ib_cleanup);
diff --git a/drivers/infiniband/hw/xsc/mem.c b/drivers/infiniband/hw/xsc/mem.c
new file mode 100644
index 0000000000000..2fbce994339bc
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/mem.c
@@ -0,0 +1,161 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/module.h>
+#include <rdma/ib_umem.h>
+#include "xsc_ib.h"
+
+/**
+ * __xsc_ib_cont_pages - scan a umem for compound pages
+ * @umem: umem object to scan
+ * @addr: ib virtual address requested by the user
+ * @max_page_shift: upper limit for the detected page shift, 0 for none
+ * @count: number of PAGE_SIZE pages covered by umem
+ * @shift: page shift for the compound pages found in the region
+ * @ncont: number of compound pages
+ */
+static void __xsc_ib_cont_pages(struct ib_umem *umem, u64 addr,
+ unsigned long max_page_shift,
+ int *count, int *shift, int *ncont)
+{
+ unsigned long tmp;
+ unsigned long m;
+ u64 base = ~0, p = 0;
+ u64 len, pfn;
+ int i = 0;
+ struct scatterlist *sg;
+ int entry;
+ /* TODO: need peer mem support */
+ unsigned long page_shift = PAGE_SHIFT;
+
+ addr = addr >> page_shift;
+ tmp = (unsigned long)addr;
+ m = find_first_bit(&tmp, BITS_PER_LONG);
+ if (max_page_shift)
+ m = min_t(unsigned long, max_page_shift - page_shift, m);
+ for_each_sgtable_dma_sg(&umem->sgt_append.sgt, sg, entry) {
+ len = sg_dma_len(sg) >> page_shift;
+ pfn = sg_dma_address(sg) >> page_shift;
+ if (base + p != pfn) {
+ /* If either the offset or the new
+ * base are unaligned update m
+ */
+ tmp = (unsigned long)(pfn | p);
+ if (!IS_ALIGNED(tmp, 1 << m))
+ m = find_first_bit(&tmp, BITS_PER_LONG);
+
+ base = pfn;
+ p = 0;
+ }
+
+ p += len;
+ i += len;
+ }
+
+ if (i) {
+ m = min_t(unsigned long, ilog2(roundup_pow_of_two(i)), m);
+ *ncont = DIV_ROUND_UP(i, (1 << m));
+ } else {
+ m = 0;
+ *ncont = 0;
+ }
+ *shift = page_shift + m;
+ *count = i;
+}
+
+void xsc_ib_cont_pages(struct ib_umem *umem, u64 addr,
+ int *count, int *shift, int *ncont)
+{
+ /* no limit for page_shift */
+ __xsc_ib_cont_pages(umem, addr, 0, count, shift, ncont);
+}
+
+static void __xsc_ib_populate_pas(struct xsc_ib_dev *dev, struct ib_umem *umem,
+ int page_shift, size_t offset, size_t num_pages,
+ __be64 *pas, int access_flags, bool need_to_devide)
+{
+ unsigned long umem_page_shift = PAGE_SHIFT;
+ int shift = page_shift - umem_page_shift;
+ int mask = (1 << shift) - 1;
+ int i = 0;
+ int k, idx;
+ u64 cur = 0;
+ u64 base;
+ int len;
+ struct scatterlist *sg;
+ int entry;
+
+ for_each_sgtable_dma_sg(&umem->sgt_append.sgt, sg, entry) {
+ len = sg_dma_len(sg) >> umem_page_shift;
+ if (need_to_devide)
+ len = sg_dma_len(sg) >> PAGE_SHIFT_4K;
+ else
+ len = sg_dma_len(sg) >> umem_page_shift;
+ base = sg_dma_address(sg);
+
+ /* Skip elements below offset */
+ if (i + len < offset << shift) {
+ i += len;
+ continue;
+ }
+
+ /* Skip pages below offset */
+ if (i < offset << shift) {
+ k = (offset << shift) - i;
+ i = offset << shift;
+ } else {
+ k = 0;
+ }
+
+ for (; k < len; k++) {
+ if (!(i & mask)) {
+ if (need_to_devide)
+ cur = base + (k << PAGE_SHIFT_4K);
+ else
+ cur = base + (k << umem_page_shift);
+ cur |= access_flags;
+ idx = (i >> shift) - offset;
+
+ pas[idx] = cpu_to_be64(cur);
+ ibdev_dbg(&dev->ib_dev, "pas[%d] 0x%llx\n",
+ i >> shift, be64_to_cpu(pas[idx]));
+ }
+ i++;
+
+ /* Stop after num_pages reached */
+ if (i >> shift >= offset + num_pages)
+ return;
+ }
+ }
+}
+
+void xsc_ib_populate_pas(struct xsc_ib_dev *dev, struct ib_umem *umem,
+ int page_shift, __be64 *pas, int npages, bool need_to_devide)
+{
+ return __xsc_ib_populate_pas(dev, umem, page_shift, 0,
+ npages, pas, 0, need_to_devide);
+}
+
+int xsc_ib_get_buf_offset(u64 addr, int page_shift, u32 *offset)
+{
+ u64 page_size;
+ u64 page_mask;
+ u64 off_size;
+ u64 off_mask;
+ u64 buf_off;
+
+ page_size = 1 << page_shift;
+ page_mask = page_size - 1;
+ buf_off = addr & page_mask;
+ off_size = page_size >> 6;
+ off_mask = off_size - 1;
+
+ if (buf_off & off_mask)
+ return -EINVAL;
+
+ *offset = buf_off >> ilog2(off_size);
+ return 0;
+}
diff --git a/drivers/infiniband/hw/xsc/mr.c b/drivers/infiniband/hw/xsc/mr.c
new file mode 100644
index 0000000000000..efd685e579c44
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/mr.c
@@ -0,0 +1,313 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kref.h>
+#include <linux/random.h>
+#include <linux/debugfs.h>
+#include <linux/export.h>
+#include <linux/dma-buf.h>
+#include <linux/dma-resv.h>
+#include <rdma/ib_umem.h>
+#include <rdma/iter.h>
+#include "common/xsc_cmd.h"
+#include <linux/dma-direct.h>
+#include "xsc_ib.h"
+
+enum {
+ DEF_CACHE_SIZE = 10,
+};
+
+struct ib_mr *xsc_ib_get_dma_mr(struct ib_pd *pd, int acc)
+{
+ struct xsc_ib_dev *dev = to_xdev(pd->device);
+ struct xsc_core_device *xdev = dev->xdev;
+ struct xsc_register_mr_mbox_in *in;
+ struct xsc_register_mr_request *req;
+ struct xsc_ib_mr *mr;
+ int err;
+
+ mr = kzalloc_obj(*mr, GFP_KERNEL);
+ if (!mr)
+ return ERR_PTR(-ENOMEM);
+
+ in = kzalloc_obj(*in, GFP_KERNEL);
+ if (!in) {
+ err = -ENOMEM;
+ goto err_free;
+ }
+
+ req = &in->req;
+ req->acc = convert_access(acc);
+ req->va_base = 0;
+ req->map_en = !(XSC_MPT_MAP_EN);
+
+ err = xsc_core_create_mkey(xdev, &mr->xmr);
+ if (err)
+ goto err_in;
+ req->mkey = cpu_to_be32(mr->xmr.key);
+ err = xsc_core_register_mr(xdev, &mr->xmr, in, sizeof(*in));
+ if (err)
+ goto err_reg_mr;
+ kfree(in);
+ mr->ibmr.lkey = mr->xmr.key;
+ mr->ibmr.rkey = mr->xmr.key;
+ mr->umem = NULL;
+
+ return &mr->ibmr;
+err_reg_mr:
+ xsc_core_destroy_mkey(xdev, &mr->xmr);
+err_in:
+ kfree(in);
+
+err_free:
+ kfree(mr);
+
+ return ERR_PTR(err);
+}
+
+static struct xsc_ib_mr *reg_create(struct ib_pd *pd, u64 virt_addr,
+ u64 length, struct ib_umem *umem,
+ int npages, int page_shift,
+ int access_flags)
+{
+ struct xsc_ib_dev *dev = to_xdev(pd->device);
+ struct xsc_register_mr_mbox_in *in;
+ struct xsc_ib_mr *mr;
+ struct ib_block_iter biter;
+ __be64 *pas;
+ int inlen;
+ int err;
+
+ mr = kzalloc_obj(*mr, GFP_KERNEL);
+ if (!mr) {
+ err = -ENOMEM;
+ goto err_0;
+ }
+
+ inlen = sizeof(*in) + sizeof(*in->req.pas) * npages;
+ in = xsc_vzalloc(inlen);
+ if (!in) {
+ err = -ENOMEM;
+ goto err_1;
+ }
+ err = xsc_core_create_mkey(dev->xdev, &mr->xmr);
+ if (err) {
+ ibdev_warn(&dev->ib_dev, "create mkey failed\n");
+ goto err_2;
+ }
+
+ pas = in->req.pas;
+ rdma_umem_for_each_dma_block(umem, &biter, 1UL << page_shift) {
+ *pas = cpu_to_be64(rdma_block_iter_dma_address(&biter));
+ pas++;
+ }
+
+ in->req.acc = convert_access(access_flags);
+ in->req.pa_num = cpu_to_be32(npages);
+ in->req.pdn = cpu_to_be32(to_xpd(pd)->pdn);
+ in->req.va_base = cpu_to_be64(virt_addr);
+ in->req.map_en = XSC_MPT_MAP_EN;
+ in->req.len = cpu_to_be64(length);
+ in->req.page_mode = xsc_get_mr_page_mode(dev->xdev, page_shift);
+ ibdev_dbg(&dev->ib_dev, "read_flush hwconfig %s\n",
+ dev->xdev->read_flush ? "enable" : "disable");
+ in->req.is_gpu = 0;
+ in->req.mkey = cpu_to_be32(mr->xmr.key);
+ err = xsc_core_register_mr(dev->xdev, &mr->xmr, in, inlen);
+ if (err) {
+ ibdev_warn(&dev->ib_dev, "register mr failed, err = %d\n", err);
+ goto err_reg_mr;
+ }
+ mr->umem = umem;
+ xsc_vfree(in);
+
+ ibdev_dbg(&dev->ib_dev, "mkey = 0x%x\n", mr->xmr.key);
+
+ return mr;
+err_reg_mr:
+ xsc_core_destroy_mkey(dev->xdev, &mr->xmr);
+err_2:
+ xsc_vfree(in);
+err_1:
+ kfree(mr);
+err_0:
+ return ERR_PTR(err);
+}
+
+struct ib_mr *xsc_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
+ u64 virt_addr, int access_flags,
+ struct ib_dmah *dmah,
+ struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev = to_xdev(pd->device);
+ struct xsc_ib_mr *mr = NULL;
+ struct ib_umem *umem;
+ unsigned long pgsz;
+ int page_shift;
+ int npages;
+ int err;
+
+ if (dmah)
+ return ERR_PTR(-EOPNOTSUPP);
+
+ if (length > dev->xdev->caps.max_mr_size) {
+ ibdev_err(&dev->ib_dev, "reg user mr length(%llu) exceeded.\n", length);
+ return ERR_PTR(-EINVAL);
+ }
+
+ ibdev_dbg(&dev->ib_dev, "start 0x%llx, virt_addr 0x%llx, length 0x%llx\n",
+ start, virt_addr, length);
+
+ umem = ib_umem_get_va(&dev->ib_dev, start, length, access_flags);
+ if (IS_ERR(umem)) {
+ ibdev_dbg(&dev->ib_dev, "umem get failed\n");
+ return (void *)umem;
+ }
+
+ pgsz = ib_umem_find_best_pgsz(umem, XSC_MR_PAGE_CAP_MASK, virt_addr);
+ if (!pgsz) {
+ ibdev_warn(&dev->ib_dev, "find best page size failed\n");
+ err = -EINVAL;
+ goto error;
+ }
+
+ page_shift = order_base_2(pgsz);
+ npages = ib_umem_num_dma_blocks(umem, pgsz);
+
+ ibdev_dbg(&dev->ib_dev, "npages %d, page_shift %d\n", npages, page_shift);
+
+ mr = reg_create(pd, virt_addr, length, umem, npages, page_shift,
+ access_flags);
+ if (IS_ERR(mr)) {
+ err = PTR_ERR(mr);
+ goto error;
+ }
+
+ ibdev_dbg(&dev->ib_dev, "mkey 0x%x\n", mr->xmr.key);
+
+ mr->umem = umem;
+ mr->npages = npages;
+ mr->ibmr.lkey = mr->xmr.key;
+ mr->ibmr.rkey = mr->xmr.key;
+ mr->ibmr.length = length;
+ return &mr->ibmr;
+
+error:
+ ib_umem_release(umem);
+ return ERR_PTR(err);
+}
+
+struct ib_mr *xsc_ib_reg_user_mr_dmabuf(struct ib_pd *pd, u64 offset,
+ u64 length, u64 virt_addr,
+ int fd, int access_flags,
+ struct ib_dmah *dmah,
+ struct uverbs_attr_bundle *attrs)
+{
+ struct xsc_ib_dev *dev = to_xdev(pd->device);
+ struct ib_umem_dmabuf *umem_dmabuf;
+ struct ib_umem *umem;
+ struct xsc_ib_mr *mr;
+ unsigned long pgsz;
+ int page_shift;
+ int npages;
+ int err;
+
+ if (!IS_ENABLED(CONFIG_INFINIBAND_USER_MEM))
+ return ERR_PTR(-EOPNOTSUPP);
+
+ if (dmah)
+ return ERR_PTR(-EOPNOTSUPP);
+
+ umem_dmabuf = ib_umem_dmabuf_get_pinned(&dev->ib_dev,
+ offset, length,
+ fd, access_flags);
+ if (IS_ERR(umem_dmabuf)) {
+ ibdev_dbg(&dev->ib_dev, "umem_dmabuf get failed (%ld)\n",
+ PTR_ERR(umem_dmabuf));
+ return ERR_CAST(umem_dmabuf);
+ }
+ umem = &umem_dmabuf->umem;
+
+ pgsz = ib_umem_find_best_pgsz(umem, XSC_MR_PAGE_CAP_MASK, virt_addr);
+ if (!pgsz) {
+ ibdev_warn(&dev->ib_dev, "find best page size failed\n");
+ err = -EINVAL;
+ goto err_release;
+ }
+
+ page_shift = order_base_2(pgsz);
+ npages = ib_umem_num_dma_blocks(umem, pgsz);
+
+ mr = reg_create(pd, virt_addr, length, umem, npages, page_shift,
+ access_flags);
+ if (IS_ERR(mr)) {
+ err = PTR_ERR(mr);
+ goto err_release;
+ }
+
+ mr->umem = umem;
+ mr->npages = npages;
+ mr->ibmr.lkey = mr->xmr.key;
+ mr->ibmr.rkey = mr->xmr.key;
+ mr->ibmr.length = length;
+
+ return &mr->ibmr;
+err_release:
+ ib_umem_release(umem);
+ return ERR_PTR(err);
+}
+
+int xsc_ib_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev = to_xdev(ibmr->device);
+ struct xsc_ib_mr *mr = to_xmr(ibmr);
+ struct ib_umem *umem = mr->umem;
+ int err;
+
+ ibdev_dbg(&dev->ib_dev, "dereg mkey = 0x%x\n", mr->xmr.key);
+
+ if (mr->npages) {
+ err = xsc_core_dereg_mr(dev->xdev, &mr->xmr);
+ if (err) {
+ ibdev_warn(&dev->ib_dev, "failed to dereg mr 0x%x (%d)\n",
+ mr->xmr.key, err);
+ return err;
+ }
+ }
+ err = xsc_core_destroy_mkey(dev->xdev, &mr->xmr);
+ if (err) {
+ ibdev_warn(&dev->ib_dev, "failed to destroy mkey 0x%x (%d)\n",
+ mr->xmr.key, err);
+ return err;
+ }
+
+ if (umem)
+ ib_umem_release(umem);
+
+ kfree(mr);
+
+ return 0;
+}
+
+void xsc_reg_local_dma_mr(struct xsc_core_device *dev)
+{
+ struct xsc_register_mr_mbox_in in;
+ int err = 0;
+
+ in.req.pdn = 0;
+ in.req.pa_num = 0;
+ in.req.len = 0;
+ in.req.mkey = cpu_to_be32(0xFF);
+ in.req.acc = XSC_PERM_LOCAL_WRITE | XSC_PERM_LOCAL_READ;
+ in.req.page_mode = xsc_get_mr_page_mode(dev, PAGE_SHIFT_4K);
+ in.req.is_gpu = 0;
+ in.req.map_en = !(XSC_MPT_MAP_EN);
+ in.req.va_base = 0;
+
+ err = xsc_core_register_mr(dev, NULL, &in, sizeof(in));
+ if (err)
+ dev_err(&dev->pdev->dev, "\n");
+}
diff --git a/drivers/infiniband/hw/xsc/qp.c b/drivers/infiniband/hw/xsc/qp.c
new file mode 100644
index 0000000000000..9c79e09a7fd1f
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/qp.c
@@ -0,0 +1,2251 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/module.h>
+#include <rdma/ib_umem.h>
+#include "xsc_ib.h"
+#include <rdma/xsc-abi.h>
+#include "common/xsc_hsi.h"
+#include <linux/crc32.h>
+#include <linux/dma-direct.h>
+#include <rdma/ib_addr.h>
+
+#ifndef ETH_P_IBOE
+#define ETH_P_IBOE 0x8915
+#endif
+
+#define MAD_QUEUE_DEPTH 128
+
+enum {
+ XSC_IB_CACHE_LINE_SIZE = 64,
+};
+
+#define UDP_SPORT_MASK_EN 0x40000000
+#define UDP_SPORT_MASK_EN_OFFSET 30
+#define UDP_SPORT_MASK 0xffff
+#define UDP_SPORT_OFFSET 0
+
+void build_opcode_map(struct xsc_ib_dev *dev)
+{
+ struct xsc_msg_opcode *xsc_ib_opcode = dev->xdev->board_info->msg_opcode;
+
+ dev->opcode_map[IB_WR_SEND] = xsc_ib_opcode->send;
+ dev->opcode_map[IB_WR_SEND_WITH_IMM] = xsc_ib_opcode->send_with_imm;
+ dev->opcode_map[IB_WR_RDMA_WRITE] = xsc_ib_opcode->rdma_write;
+ dev->opcode_map[IB_WR_RDMA_WRITE_WITH_IMM] = xsc_ib_opcode->rdma_write_with_imm;
+ dev->opcode_map[IB_WR_RDMA_READ] = xsc_ib_opcode->rdma_read;
+ dev->opcode_map[IB_WR_ATOMIC_CMP_AND_SWP] = xsc_ib_opcode->atomic_cmp_and_swp;
+ dev->opcode_map[IB_WR_ATOMIC_FETCH_AND_ADD] = xsc_ib_opcode->atomic_fetch_and_add;
+ dev->opcode_map[IB_WR_MASKED_ATOMIC_CMP_AND_SWP] =
+ xsc_ib_opcode->masked_atomic_cmp_and_swp;
+ dev->opcode_map[IB_WR_MASKED_ATOMIC_FETCH_AND_ADD] =
+ xsc_ib_opcode->masked_atomic_fetch_and_add;
+}
+
+static int is_qp0(enum ib_qp_type qp_type)
+{
+ return qp_type == IB_QPT_SMI;
+}
+
+static int is_qp1(enum ib_qp_type qp_type)
+{
+ return qp_type == IB_QPT_GSI;
+}
+
+static int is_sqp(enum ib_qp_type qp_type)
+{
+ return is_qp0(qp_type) || is_qp1(qp_type);
+}
+
+static void *get_wqe(struct xsc_ib_qp *qp, int offset)
+{
+ return xsc_buf_offset(&qp->buf, offset);
+}
+
+static void *get_recv_wqe(struct xsc_ib_qp *qp, int n)
+{
+ return get_wqe(qp, qp->rq.offset + (n << qp->rq.wqe_shift));
+}
+
+static void *get_seg_wqe(void *first, int n)
+{
+ return first + (n << XSC_BASE_WQE_SHIFT);
+}
+
+void *xsc_get_send_wqe(struct xsc_ib_qp *qp, int n)
+{
+ return get_wqe(qp, qp->sq.offset + (n << qp->sq.wqe_shift));
+}
+
+static int iboe_tos_to_sl(struct net_device *ndev, int tos)
+{
+ int prio;
+ struct net_device *dev;
+
+ prio = rt_tos2priority(tos);
+ dev = is_vlan_dev(ndev) ? vlan_dev_real_dev(ndev) : ndev;
+ if (dev->num_tc)
+ return netdev_get_prio_tc_map(dev, prio);
+
+#if IS_ENABLED(CONFIG_VLAN_8021Q)
+ if (is_vlan_dev(ndev))
+ return (vlan_dev_get_egress_qos_mask(ndev, prio) &
+ VLAN_PRIO_MASK) >> VLAN_PRIO_SHIFT;
+#endif
+ return 0;
+}
+
+static inline void set_remote_addr_seg(struct xsc_core_device *xdev,
+ struct xsc_wqe_data_seg *remote_seg,
+ u32 msg_len, u64 remote_addr, u32 rkey)
+{
+ xsc_set_data_seg(xdev, remote_seg, remote_addr, rkey, msg_len);
+}
+
+static void set_local_data_seg(struct xsc_core_device *xdev,
+ struct xsc_wqe_data_seg *data_seg, struct ib_sge *sg)
+{
+ xsc_set_data_seg(xdev, data_seg, sg->addr, sg->lkey, sg->length);
+}
+
+static int set_data_inl_seg(struct xsc_ib_qp *qp, const struct ib_send_wr *wr, void *ctrl)
+{
+ struct xsc_wqe_data_seg *data_seg;
+ unsigned int seg_index;
+ void *addr;
+ int len;
+ int i;
+
+ for (i = 0, seg_index = 1; i < wr->num_sge; ++i, ++seg_index) {
+ if (likely(wr->sg_list[i].length)) {
+ addr = (void *)wr->sg_list[i].addr;
+ len = wr->sg_list[i].length;
+
+ if (unlikely(len > qp->max_inline_data))
+ return -ENOMEM;
+
+ data_seg = get_seg_wqe(ctrl, seg_index);
+ data_seg->in_line = 1;
+ data_seg->len = len;
+ memcpy(data_seg->in_line_data, addr, len);
+ }
+ }
+
+ return 0;
+}
+
+static __be32 send_ieth(const struct ib_send_wr *wr)
+{
+ switch (wr->opcode) {
+ case IB_WR_SEND_WITH_IMM:
+ case IB_WR_RDMA_WRITE_WITH_IMM:
+ return wr->ex.imm_data;
+ default:
+ return 0;
+ }
+}
+
+static void xsc_ib_qp_event(struct xsc_core_qp *qp, int type)
+{
+ struct ib_qp *ibqp = &core_to_xqp(qp)->ibqp;
+ struct ib_event event;
+
+ if (ibqp->event_handler) {
+ event.device = ibqp->device;
+ event.element.qp = ibqp;
+ switch (type) {
+ case XSC_EVENT_TYPE_WQ_CATAS_ERROR:
+ event.event = IB_EVENT_QP_FATAL;
+ break;
+ case XSC_EVENT_TYPE_WQ_INVAL_REQ_ERROR:
+ event.event = IB_EVENT_QP_REQ_ERR;
+ break;
+ case XSC_EVENT_TYPE_WQ_ACCESS_ERROR:
+ event.event = IB_EVENT_QP_ACCESS_ERR;
+ break;
+ default:
+ pr_warn("xsc_ib: Unexpected event type %d on QP %06x\n", type, qp->qpn);
+ return;
+ }
+
+ ibqp->event_handler(&event, ibqp->qp_context);
+ pr_info("xsc_ib: xsc async event 0x%x, ib event %d\n", type, event.event);
+ }
+}
+
+static int set_rq_size(struct xsc_ib_dev *dev, struct ib_qp_init_attr *init_attr,
+ int has_rq, struct xsc_ib_qp *qp, struct xsc_ib_create_qp *ucmd)
+{
+ bool check_res = false;
+ u32 wqe_cnt = init_attr->cap.max_recv_wr ?
+ roundup_pow_of_two(init_attr->cap.max_recv_wr) : 0;
+
+ if (init_attr->srq)
+ wqe_cnt = to_xsrq(init_attr->srq)->xsrq.max;
+
+ /* Sanity check RQ size before proceeding */
+ check_res = xsc_check_max_qp_depth(dev->xdev, &wqe_cnt, dev->xdev->caps.max_wqes);
+ if (check_res) {
+ ibdev_err(&dev->ib_dev, "max_recv_wr:%d exceed max rq depth\n",
+ init_attr->cap.max_recv_wr);
+ return -EINVAL;
+ }
+
+ if (!has_rq) {
+ qp->rq.max_gs = 0;
+ qp->rq.wqe_cnt = 0;
+ qp->rq.wqe_shift = 0;
+ } else {
+ if (ucmd) {
+ qp->rq.wqe_cnt = ucmd->rq_wqe_count;
+ qp->rq.wqe_shift = ucmd->rq_wqe_shift;
+ qp->rq.max_gs = 1;
+ qp->rq.max_post = qp->rq.wqe_cnt;
+ } else {
+ qp->rq.wqe_cnt = wqe_cnt;
+ if (qp->xqp.qp_type == IB_QPT_GSI)
+ qp->rq.wqe_shift = ilog2(xsc_get_recv_ds_num(dev->xdev))
+ + XSC_RECV_WQE_SHIFT;
+ else
+ qp->rq.wqe_shift = dev->xdev->caps.recv_wqe_shift;
+ qp->rq.max_gs = dev->xdev->caps.recv_ds_num;
+ qp->rq.max_post = qp->rq.wqe_cnt;
+ }
+ }
+
+ return 0;
+}
+
+static int calc_sq_size(struct xsc_ib_dev *dev, struct ib_qp_init_attr *attr,
+ struct xsc_ib_qp *qp)
+{
+ int wqe_size;
+ int wq_size;
+ bool check_res = false;
+
+ if (!attr->cap.max_send_wr) {
+ ibdev_err(&dev->ib_dev, "invalid max_send_wr:%d\n", attr->cap.max_send_wr);
+ return -1;
+ }
+
+ wqe_size = 1 << dev->xdev->caps.send_wqe_shift;
+ qp->max_inline_data = (dev->xdev->caps.send_ds_num - 2) * sizeof(struct xsc_wqe_data_seg);
+ attr->cap.max_inline_data = qp->max_inline_data;
+
+ attr->cap.max_send_wr = min_t(u32, attr->cap.max_send_wr, dev->xdev->caps.max_wqes);
+ qp->sq.wqe_cnt = roundup_pow_of_two(attr->cap.max_send_wr);
+
+ check_res = xsc_check_max_qp_depth(dev->xdev, &qp->sq.wqe_cnt, dev->xdev->caps.max_wqes);
+ if (check_res) {
+ ibdev_err(&dev->ib_dev, "max_send_wr:%d exceed max sq depth\n",
+ attr->cap.max_send_wr);
+ return -EINVAL;
+ }
+ qp->sq.ds_cnt = qp->sq.wqe_cnt << (dev->xdev->caps.send_wqe_shift - XSC_BASE_WQE_SHIFT);
+ wq_size = qp->sq.wqe_cnt * wqe_size;
+ qp->sq.wqe_shift = ilog2(wqe_size);
+ qp->sq.max_gs = dev->xdev->caps.send_ds_num - XSC_CTRL_SEG_NUM - XSC_RADDR_SEG_NUM;
+ qp->sq.max_post = qp->sq.wqe_cnt;
+
+ return wq_size;
+}
+
+static int qp_has_rq(struct ib_qp_init_attr *attr)
+{
+ if (!attr->srq && !attr->cap.max_recv_wr)
+ return 0;
+
+ return 1;
+}
+
+static enum xsc_qp_state to_xsc_state(enum ib_qp_state state)
+{
+ switch (state) {
+ case IB_QPS_RESET: return XSC_QP_STATE_RST;
+ case IB_QPS_INIT: return XSC_QP_STATE_INIT;
+ case IB_QPS_RTR: return XSC_QP_STATE_RTR;
+ case IB_QPS_RTS: return XSC_QP_STATE_RTS;
+ case IB_QPS_SQD: return XSC_QP_STATE_SQD;
+ case IB_QPS_SQE: return XSC_QP_STATE_SQER;
+ case IB_QPS_ERR: return XSC_QP_STATE_ERR;
+ default: return -1;
+ }
+}
+
+static char *qp_state_to_str(enum ib_qp_state state)
+{
+ switch (state) {
+ case IB_QPS_RESET: return "RST";
+ case IB_QPS_INIT: return "INIT";
+ case IB_QPS_RTR: return "RTR";
+ case IB_QPS_RTS: return "RTS";
+ case IB_QPS_SQD: return "SQD";
+ case IB_QPS_SQE: return "SQE";
+ case IB_QPS_ERR: return "ERR";
+ default: return "UNKNOWN";
+ }
+}
+
+static struct ib_umem *qp_umem_get(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp,
+ struct ib_udata *udata, struct ib_pd *pd,
+ struct xsc_ib_create_qp *ucmd)
+{
+ struct ib_umem *umem;
+
+ umem = ib_umem_get_va(&dev->ib_dev, ucmd->buf_addr, qp->buf_size,
+ IB_ACCESS_LOCAL_WRITE);
+ return umem;
+}
+
+static int create_user_qp(struct xsc_ib_dev *dev, struct ib_pd *pd,
+ struct xsc_ib_qp *qp, struct ib_udata *udata,
+ struct xsc_create_qp_mbox_in **in,
+ struct xsc_ib_create_qp_resp *resp, int *inlen)
+{
+ struct xsc_ib_create_qp ucmd;
+ int page_shift;
+ int npages;
+ u32 offset;
+ int ncont;
+ int err;
+ int hw_npages;
+ struct ib_umem_dmabuf *umem_dmabuf;
+
+ err = ib_copy_from_udata(&ucmd, udata, sizeof(ucmd));
+ if (err) {
+ ibdev_err(&dev->ib_dev, "failed to copy from udata, err=%d\n", err);
+ return err;
+ }
+ ibdev_dbg(&dev->ib_dev, "buf_addr:0x%lx db_addr:0x%lx sq cnt:%u, rq cnt:%u, rq shift:%u\n",
+ (uintptr_t)ucmd.buf_addr, (uintptr_t)ucmd.db_addr,
+ ucmd.sq_wqe_count, ucmd.rq_wqe_count, ucmd.rq_wqe_shift);
+
+ qp->sq.ds_cnt = ucmd.sq_wqe_count;
+ qp->sq.wqe_cnt = ucmd.sq_wqe_count;
+ qp->sq.wqe_shift = XSC_BASE_WQE_SHIFT;
+ qp->rq.ds_cnt = ucmd.rq_wqe_count;
+ qp->rq.wqe_cnt = ucmd.rq_wqe_count;
+ qp->rq.wqe_shift = XSC_BASE_WQE_SHIFT;
+
+ qp->buf_size = (qp->sq.wqe_cnt << qp->sq.wqe_shift) + (qp->rq.wqe_cnt << qp->rq.wqe_shift);
+
+ if (ucmd.dmabuf_fd != -1) {
+ umem_dmabuf = ib_umem_dmabuf_get_pinned(&dev->ib_dev, 0, ucmd.dmabuf_sz,
+ ucmd.dmabuf_fd, IB_ACCESS_LOCAL_WRITE);
+
+ if (IS_ERR(umem_dmabuf)) {
+ ibdev_warn(&dev->ib_dev, "qp umem_dmabuf get failed (%ld)\n",
+ PTR_ERR(umem_dmabuf));
+ return -ENOMEM;
+ }
+
+ qp->umem = &umem_dmabuf->umem;
+ } else {
+ qp->umem = qp_umem_get(dev, qp, udata, pd, &ucmd);
+ }
+
+ if (IS_ERR(qp->umem)) {
+ ibdev_err(&dev->ib_dev, "umem_get failed\n");
+ err = PTR_ERR(qp->umem);
+ goto err_uuar;
+ }
+
+ xsc_ib_cont_pages(qp->umem, ucmd.buf_addr, &npages, &page_shift,
+ &ncont);
+ if (ncont != npages) {
+ /* TODO: peer memory support failed */
+ page_shift = PAGE_SHIFT;
+ ncont = npages;
+ }
+
+ hw_npages = DIV_ROUND_UP(qp->buf_size, PAGE_SIZE_4K);
+ err = xsc_ib_get_buf_offset(ucmd.buf_addr, page_shift, &offset);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "bad offset:%d\n", offset);
+ goto err_umem;
+ }
+ ibdev_dbg(&dev->ib_dev, "npage:%d, page_shift:%d, ncont:%d, offset:%d, hw_npages %d\n",
+ npages, page_shift, ncont, offset, hw_npages);
+
+ *inlen = sizeof(**in) + sizeof(*((*in)->req.pas)) * hw_npages;
+ *in = xsc_vzalloc(*inlen);
+ if (!*in) {
+ err = -ENOMEM;
+ goto err_umem;
+ }
+ xsc_ib_populate_pas(dev, qp->umem, page_shift, (*in)->req.pas, hw_npages, true);
+ (*in)->req.pa_num = cpu_to_be16(hw_npages);
+
+ err = ib_copy_to_udata(udata, resp, sizeof(*resp));
+ if (err) {
+ ibdev_err(&dev->ib_dev, "failed to copy to udata, err=%d\n", err);
+ goto err_umem;
+ }
+ qp->create_type = XSC_QP_USER;
+
+ return 0;
+
+err_umem:
+ ib_umem_release(qp->umem);
+
+err_uuar:
+ return err;
+}
+
+static void destroy_qp_user(struct ib_pd *__maybe_unused pd, struct xsc_ib_qp *qp)
+{
+ ib_umem_release(qp->umem);
+}
+
+#define MAX_QP1_SQ_HDR_SIZE_V2 512
+#define MAX_QP1_SQ_HDR_SIZE 86
+ /* Ethernet header = 14 */
+ /* ib_grh = 40 (provided by MAD) */
+ /* ib_bth + ib_deth = 20 */
+ /* MAD = 256 (provided by MAD) */
+ /* iCRC = 4 */
+#define MAX_QP1_RQ_HDR_SIZE_V2 512
+
+static int create_kernel_qp(struct xsc_ib_dev *dev,
+ struct ib_qp_init_attr *init_attr,
+ struct xsc_ib_qp *qp,
+ struct xsc_create_qp_mbox_in **in, int *inlen)
+{
+ int err;
+ int sq_size;
+ int hw_npages;
+
+ sq_size = calc_sq_size(dev, init_attr, qp);
+ if (sq_size < 0) {
+ err = -EINVAL;
+ ibdev_err(&dev->ib_dev, "err %d\n", err);
+ return err;
+ }
+
+ qp->rq.ds_cnt = qp->rq.wqe_cnt << (qp->rq.wqe_shift - XSC_BASE_WQE_SHIFT);
+ qp->rq.offset = 0;
+ qp->sq.offset = qp->rq.wqe_cnt << qp->rq.wqe_shift;
+ qp->buf_size = qp->sq.offset + sq_size;
+ qp->send_psn = 0;
+
+ err = xsc_buf_alloc(dev->xdev, qp->buf_size, PAGE_SIZE, &qp->buf);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "failed to alloc qp buffer,err=%d\n", err);
+ return err;
+ }
+
+ qp->sq.qend = qp->buf.direct.buf + qp->sq.offset + sq_size;
+ hw_npages = DIV_ROUND_UP(qp->buf_size, PAGE_SIZE_4K);
+ *inlen = sizeof(**in) + sizeof(*(*in)->req.pas) * hw_npages;
+ *in = xsc_vzalloc(*inlen);
+ if (!*in) {
+ err = -ENOMEM;
+ goto err_buf;
+ }
+
+ xsc_fill_page_array(&qp->buf, (*in)->req.pas, hw_npages);
+ (*in)->req.pa_num = cpu_to_be16(hw_npages);
+
+ qp->sq.wrid = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.wrid), GFP_KERNEL);
+ qp->sq.wr_data = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.wr_data), GFP_KERNEL);
+ qp->rq.wrid = kcalloc(qp->rq.wqe_cnt, sizeof(*qp->rq.wrid), GFP_KERNEL);
+ qp->sq.w_list = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.w_list), GFP_KERNEL);
+ qp->sq.wqe_head = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.wqe_head), GFP_KERNEL);
+ qp->sq.wr_opcode = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.wr_opcode), GFP_KERNEL);
+ qp->sq.need_flush = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.need_flush), GFP_KERNEL);
+ qp->sq.prev_ce_wqe = kcalloc(qp->sq.wqe_cnt, sizeof(*qp->sq.prev_ce_wqe), GFP_KERNEL);
+ qp->sq.last_comp_ce_wqe = -1;
+ qp->sq.last_post_ce_wqe = -1;
+
+ if (!qp->sq.wrid || !qp->sq.wr_data || !qp->rq.wrid || !qp->sq.prev_ce_wqe ||
+ !qp->sq.w_list || !qp->sq.wqe_head || !qp->sq.wr_opcode || !qp->sq.need_flush) {
+ err = -ENOMEM;
+ goto err_wrid;
+ }
+ qp->create_type = XSC_QP_KERNEL;
+
+ if (init_attr->qp_type == IB_QPT_GSI) {
+ qp->sq.mad_index = 0;
+ qp->sq.mad_queue_depth = MAD_QUEUE_DEPTH;
+ qp->sq.hdr_size = MAX_QP1_SQ_HDR_SIZE_V2 * MAD_QUEUE_DEPTH;
+ qp->sq.hdr_buf = dma_alloc_coherent(dev->ib_dev.dma_device,
+ qp->sq.hdr_size,
+ &qp->sq.hdr_dma,
+ GFP_KERNEL);
+ if (!qp->sq.hdr_buf) {
+ err = -ENOMEM;
+ ibdev_err(&dev->ib_dev, "Failed to create sq_hdr_buf");
+ goto err_wrid;
+ }
+ }
+
+ return 0;
+
+err_wrid:
+ kfree(qp->sq.prev_ce_wqe);
+ kfree(qp->sq.need_flush);
+ kfree(qp->sq.wr_opcode);
+ kfree(qp->sq.wqe_head);
+ kfree(qp->sq.w_list);
+ kfree(qp->sq.wrid);
+ kfree(qp->sq.wr_data);
+ kfree(qp->rq.wrid);
+
+err_buf:
+ xsc_buf_free(dev->xdev, &qp->buf);
+ return err;
+}
+
+static void destroy_qp_kernel(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp)
+{
+ if (qp->sq.hdr_buf)
+ dma_free_coherent(dev->ib_dev.dma_device, qp->sq.hdr_size,
+ qp->sq.hdr_buf, qp->sq.hdr_dma);
+ kfree(qp->sq.prev_ce_wqe);
+ kfree(qp->sq.need_flush);
+ kfree(qp->sq.wr_opcode);
+ kfree(qp->sq.wqe_head);
+ kfree(qp->sq.w_list);
+ kfree(qp->sq.wrid);
+ kfree(qp->sq.wr_data);
+ kfree(qp->rq.wrid);
+ xsc_buf_free(dev->xdev, &qp->buf);
+}
+
+static u8 ib_to_xsc_qp_type(enum ib_qp_type qp_type, __u32 flags)
+{
+ if (qp_type == IB_QPT_RC) {
+ if (flags & XSC_QP_FLAG_WORC)
+ return XSC_QUEUE_TYPE_WORC;
+ else
+ return XSC_QUEUE_TYPE_RDMA_RC;
+ } else if ((qp_type == IB_QPT_GSI) || (qp_type == IB_QPT_SMI)) {
+ return XSC_QUEUE_TYPE_RDMA_MAD;
+ } else if (qp_type == IB_QPT_RAW_PACKET) {
+ if (flags & XSC_QP_FLAG_RAWPACKET_TSO)
+ return XSC_QUEUE_TYPE_RAW_TSO;
+ else if (flags & XSC_QP_FLAG_RAWPACKET_TX)
+ return XSC_QUEUE_TYPE_RAW_TX;
+ else if (flags & XSC_QP_FLAG_RAWPACKET_SNIFFER)
+ return XSC_QUEUE_TYPE_SNIFFER;
+ else
+ return XSC_QUEUE_TYPE_RAW;
+ } else {
+ return XSC_QUEUE_TYPE_INVALID;
+ }
+}
+
+static void get_cqs(enum ib_qp_type qp_type,
+ struct ib_cq *ib_send_cq, struct ib_cq *ib_recv_cq,
+ struct xsc_ib_cq **send_cq, struct xsc_ib_cq **recv_cq)
+{
+ switch (qp_type) {
+ case IB_QPT_SMI:
+ case IB_QPT_GSI:
+ case IB_QPT_RC:
+ case IB_QPT_UC:
+ case IB_QPT_UD:
+ case IB_QPT_RAW_PACKET:
+ *send_cq = ib_send_cq ? to_xcq(ib_send_cq) : NULL;
+ *recv_cq = ib_recv_cq ? to_xcq(ib_recv_cq) : NULL;
+ break;
+ default:
+ *send_cq = NULL;
+ *recv_cq = NULL;
+ break;
+ }
+}
+
+static void xsc_ib_lock_cqs(struct xsc_ib_cq *send_cq, struct xsc_ib_cq *recv_cq)
+ __acquires(&send_cq->lock) __acquires(&recv_cq->lock)
+{
+ if (send_cq) {
+ if (recv_cq) {
+ if (send_cq->xcq.cqn < recv_cq->xcq.cqn) {
+ spin_lock_irq(&send_cq->lock);
+ spin_lock_nested(&recv_cq->lock,
+ SINGLE_DEPTH_NESTING);
+ } else if (send_cq->xcq.cqn == recv_cq->xcq.cqn) {
+ spin_lock_irq(&send_cq->lock);
+ __acquire(&recv_cq->lock);
+ } else {
+ spin_lock_irq(&recv_cq->lock);
+ spin_lock_nested(&send_cq->lock,
+ SINGLE_DEPTH_NESTING);
+ }
+ } else {
+ spin_lock_irq(&send_cq->lock);
+ }
+ } else if (recv_cq) {
+ spin_lock_irq(&recv_cq->lock);
+ }
+}
+
+static void xsc_ib_unlock_cqs(struct xsc_ib_cq *send_cq, struct xsc_ib_cq *recv_cq)
+ __releases(&send_cq->lock) __releases(&recv_cq->lock)
+{
+ if (send_cq) {
+ if (recv_cq) {
+ if (send_cq->xcq.cqn < recv_cq->xcq.cqn) {
+ spin_unlock(&recv_cq->lock);
+ spin_unlock_irq(&send_cq->lock);
+ } else if (send_cq->xcq.cqn == recv_cq->xcq.cqn) {
+ __release(&recv_cq->lock);
+ spin_unlock_irq(&send_cq->lock);
+ } else {
+ spin_unlock(&send_cq->lock);
+ spin_unlock_irq(&recv_cq->lock);
+ }
+ } else {
+ spin_unlock_irq(&send_cq->lock);
+ }
+ } else if (recv_cq) {
+ spin_unlock_irq(&recv_cq->lock);
+ }
+}
+
+static int create_qp_common(struct xsc_ib_dev *dev, struct ib_pd *pd,
+ struct ib_qp_init_attr *init_attr,
+ struct ib_udata *udata, struct xsc_ib_qp *qp)
+{
+ struct xsc_ib_create_qp_resp resp;
+ struct xsc_create_qp_mbox_in *in = NULL;
+ struct xsc_ib_create_qp ucmd;
+ struct xsc_ib_cq *send_cq;
+ struct xsc_ib_cq *recv_cq;
+ struct xsc_srq_qp_node *qp_node;
+ int inlen = sizeof(*in);
+ int err;
+ char buf[256];
+ char *ptr = buf;
+ int ret = 0;
+
+ mutex_init(&qp->mutex);
+ spin_lock_init(&qp->sq.lock);
+ spin_lock_init(&qp->rq.lock);
+ spin_lock_init(&qp->lock);
+
+ if (init_attr->sq_sig_type == IB_SIGNAL_ALL_WR)
+ qp->sq_signal_bits = XSC_WQE_CTRL_CQ_UPDATE;
+
+ if (pd->uobject) {
+ if (ib_copy_from_udata(&ucmd, udata, sizeof(ucmd))) {
+ ibdev_err(&dev->ib_dev, "failed to copy from udata\n");
+ return -EFAULT;
+ }
+ }
+
+ qp->has_rq = qp_has_rq(init_attr);
+
+ err = set_rq_size(dev, init_attr, qp->has_rq,
+ qp, pd->uobject ? &ucmd : NULL);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "failed to set rq size %d\n", err);
+ return err;
+ }
+
+ if (pd->uobject) {
+ err = create_user_qp(dev, pd, qp, udata, &in, &resp, &inlen);
+ if (err)
+ ibdev_err(&dev->ib_dev, "failed to create user qp, err = %d\n", err);
+ } else {
+ err = create_kernel_qp(dev, init_attr, qp, &in, &inlen);
+ if (err)
+ ibdev_err(&dev->ib_dev, "failed to create kernel qp, err = %d\n", err);
+ else
+ qp->pa_lkey = to_xpd(pd)->pa_lkey;
+ }
+
+ if (err)
+ return err;
+
+ if (is_sqp(init_attr->qp_type))
+ qp->port = init_attr->port_num;
+
+ if (is_qp1(init_attr->qp_type))
+ in->req.input_qpn = cpu_to_be16(1);
+
+ in->req.pdn = cpu_to_be32(to_xpd(pd)->pdn);
+
+ if (qp->rq.ds_cnt)
+ in->req.log_rq_sz = ilog2(qp->rq.ds_cnt);
+
+ if (qp->sq.ds_cnt)
+ in->req.log_sq_sz = ilog2(qp->sq.ds_cnt);
+ else
+ in->req.log_sq_sz = ilog2(0x80);
+
+ if (init_attr->send_cq) {
+ qp->send_cq = init_attr->send_cq;
+ in->req.cqn_send = to_xcq(init_attr->send_cq)->xcq.cqn;
+ in->req.cqn_send = cpu_to_be16(in->req.cqn_send);
+ }
+
+ if (init_attr->recv_cq) {
+ qp->recv_cq = init_attr->recv_cq;
+ in->req.cqn_recv = to_xcq(init_attr->recv_cq)->xcq.cqn;
+ in->req.cqn_recv = cpu_to_be16(in->req.cqn_recv);
+ }
+
+ in->req.qp_type = ib_to_xsc_qp_type(init_attr->qp_type, pd->uobject ?
+ ucmd.flags : 0);
+
+ if (in->req.qp_type == XSC_QUEUE_TYPE_INVALID) {
+ ibdev_err(&dev->ib_dev, "invalid qp type:%d\n", init_attr->qp_type);
+ goto err_create;
+ }
+ in->req.glb_funcid = cpu_to_be16(dev->xdev->glb_func_id);
+ in->req.page_shift = PAGE_SHIFT;
+
+ qp->xqp.qp_type_internal = in->req.qp_type;
+
+ err = xsc_core_create_qp(dev->xdev, &qp->xqp, in, inlen);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "create qp failed, err=%d\n", err);
+ goto err_create;
+ }
+
+ qp->doorbell_qpn = qp->xqp.qpn;
+
+ qp->xqp.event = xsc_ib_qp_event;
+ qp->xqp.qp_type = init_attr->qp_type;
+ ret += snprintf(ptr + ret, 256 - ret, "pdn=%d,", to_xpd(pd)->pdn);
+ ret += snprintf(ptr + ret, 256 - ret, "log_rq_sz=%d,", in->req.log_rq_sz);
+ ret += snprintf(ptr + ret, 256 - ret, "log_sq_sz=%d,", in->req.log_sq_sz);
+ ret += snprintf(ptr + ret, 256 - ret, "scqn=%d,", to_xcq(init_attr->send_cq)->xcq.cqn);
+ ret += snprintf(ptr + ret, 256 - ret, "rcqn=%d", to_xcq(init_attr->recv_cq)->xcq.cqn);
+
+ get_cqs(init_attr->qp_type, init_attr->send_cq, init_attr->recv_cq, &send_cq, &recv_cq);
+ xsc_ib_lock_cqs(send_cq, recv_cq);
+ if (send_cq)
+ list_add_tail(&qp->cq_send_list, &send_cq->send_qp_list);
+ if (recv_cq)
+ list_add_tail(&qp->cq_recv_list, &recv_cq->recv_qp_list);
+ xsc_ib_unlock_cqs(send_cq, recv_cq);
+
+ if (!pd->uobject && init_attr->srq) {
+ qp_node = kvzalloc_obj(*qp_node, GFP_KERNEL);
+ if (!qp_node)
+ goto err_create;
+ qp_node->ibqp = &qp->ibqp;
+ qp_node->qp_id = qp->xqp.qpn;
+ ibdev_dbg(&dev->ib_dev, "qpn:%u add into srq_qp_list\n", qp_node->qp_id);
+ list_add_tail(&qp_node->entry, &to_xsrq(init_attr->srq)->xsrq.srq_qp_list);
+ }
+
+ qp->xqp.qp_protocol_mode = dev->xdev->rdma_proto_mode;
+
+ ibdev_dbg(&dev->ib_dev, "succeeded to create qp:%d, %s\n", qp->xqp.qpn, buf);
+
+ xsc_vfree(in);
+
+ return 0;
+
+err_create:
+ if (qp->create_type == XSC_QP_USER)
+ destroy_qp_user(pd, qp);
+ else if (qp->create_type == XSC_QP_KERNEL)
+ destroy_qp_kernel(dev, qp);
+
+ xsc_vfree(in);
+ return err;
+}
+
+static struct xsc_ib_pd *get_pd(struct xsc_ib_qp *qp)
+{
+ return to_xpd(qp->ibqp.pd);
+}
+
+static void destroy_qp_common(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp)
+{
+ struct xsc_ib_cq *send_cq = NULL;
+ struct xsc_ib_cq *recv_cq = NULL;
+ struct xsc_modify_qp_mbox_in *in;
+ int err;
+
+ in = kzalloc_obj(*in, GFP_KERNEL);
+ if (!in)
+ return;
+
+ if (qp->xqp.qp_type_internal == XSC_QUEUE_TYPE_RAW ||
+ qp->xqp.qp_type_internal == XSC_QUEUE_TYPE_RAW_TSO ||
+ qp->xqp.qp_type_internal == XSC_QUEUE_TYPE_RAW_TX ||
+ qp->state != IB_QPS_RESET)
+ if (xsc_core_qp_modify(dev->xdev, to_xsc_state(qp->state),
+ XSC_QP_STATE_RST, in, sizeof(*in), &qp->xqp))
+ ibdev_warn(&dev->ib_dev, "modify QP %06x to RESET failed\n", qp->xqp.qpn);
+
+ get_cqs(qp->xqp.qp_type, qp->ibqp.send_cq, qp->ibqp.recv_cq, &send_cq, &recv_cq);
+ xsc_ib_lock_cqs(send_cq, recv_cq);
+ if (send_cq)
+ list_del(&qp->cq_send_list);
+
+ if (recv_cq)
+ list_del(&qp->cq_recv_list);
+
+ if (qp->create_type == XSC_QP_KERNEL) {
+ __xsc_ib_cq_clean(recv_cq, qp->xqp.qpn);
+ if (send_cq != recv_cq)
+ __xsc_ib_cq_clean(send_cq, qp->xqp.qpn);
+ }
+ xsc_ib_unlock_cqs(send_cq, recv_cq);
+
+ err = xsc_core_destroy_qp(dev->xdev, &qp->xqp);
+ if (err)
+ ibdev_warn(&dev->ib_dev, "failed to destroy QP 0x%x\n", qp->xqp.qpn);
+ kfree(in);
+
+ if (qp->create_type == XSC_QP_KERNEL)
+ destroy_qp_kernel(dev, qp);
+ else if (qp->create_type == XSC_QP_USER)
+ destroy_qp_user(&get_pd(qp)->ibpd, qp);
+}
+
+static const char *ib_qp_type_str(enum ib_qp_type type)
+{
+ switch (type) {
+ case IB_QPT_SMI:
+ return "IB_QPT_SMI";
+ case IB_QPT_GSI:
+ return "IB_QPT_GSI";
+ case IB_QPT_RC:
+ return "IB_QPT_RC";
+ case IB_QPT_UC:
+ return "IB_QPT_UC";
+ case IB_QPT_UD:
+ return "IB_QPT_UD";
+ case IB_QPT_RAW_IPV6:
+ return "IB_QPT_RAW_IPV6";
+ case IB_QPT_RAW_ETHERTYPE:
+ return "IB_QPT_RAW_ETHERTYPE";
+ case IB_QPT_RAW_PACKET:
+ return "IB_QPT_RAW_PACKET";
+ case IB_QPT_MAX:
+ default:
+ return "Invalid QP type";
+ }
+}
+
+int xsc_ib_create_qp(struct ib_qp *ibqp,
+ struct ib_qp_init_attr *init_attr,
+ struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev;
+ struct xsc_ib_qp *qp;
+ struct ib_pd *pd = ibqp->pd;
+ int err;
+
+ if (pd) {
+ dev = to_xdev(pd->device);
+ } else {
+ pr_warn("%s: no PD for transport %s\n", __func__,
+ ib_qp_type_str(init_attr->qp_type));
+ return -EINVAL;
+ }
+
+ if (init_attr->qp_type != IB_QPT_RAW_PACKET) {
+ if (!is_support_rdma(dev->xdev) ||
+ (is_qp1(init_attr->qp_type) && !is_support_rdma_cm(dev->xdev))) {
+ return -EPROTONOSUPPORT;
+ }
+ }
+
+ qp = to_xqp(ibqp);
+
+ qp->xqp.qp_type = init_attr->qp_type;
+
+ switch (init_attr->qp_type) {
+ case IB_QPT_RC:
+ case IB_QPT_SMI:
+ case IB_QPT_GSI:
+ case IB_QPT_RAW_PACKET:
+ err = create_qp_common(dev, pd, init_attr, udata, qp);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "create_qp_common failed\n");
+ return err;
+ }
+
+ if (is_qp0(init_attr->qp_type)) {
+ qp->ibqp.qp_num = 0;
+ } else if (is_qp1(init_attr->qp_type)) {
+ qp->ibqp.qp_num = 1;
+ dev->xdev->gsi_qpn = qp->xqp.qpn;
+ } else {
+ qp->ibqp.qp_num = qp->xqp.qpn;
+ }
+
+ break;
+
+ case IB_QPT_RAW_IPV6:
+ case IB_QPT_RAW_ETHERTYPE:
+ case IB_QPT_MAX:
+ default:
+ ibdev_err(&dev->ib_dev, "unsupported qp type %d\n",
+ init_attr->qp_type);
+ /* Don't support raw QPs */
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+int xsc_ib_destroy_qp(struct ib_qp *qp, struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev = to_xdev(qp->device);
+ struct xsc_ib_qp *xqp = to_xqp(qp);
+ struct xsc_err_state_qp_node *tmp = NULL, *err_rq_node = NULL, *err_sq_node = NULL;
+ struct xsc_srq_qp_node *qp_node = NULL, *tmp_qp_node = NULL;
+ unsigned long flags;
+ struct xsc_ib_srq *xsrq;
+
+ if (qp->qp_type == IB_QPT_RC) {
+ if (xqp->recv_cq) {
+ spin_lock_irq(&to_xcq(xqp->recv_cq)->lock);
+ list_for_each_entry_safe(err_rq_node, tmp,
+ &to_xcq(xqp->recv_cq)->err_state_qp_list, entry) {
+ if (err_rq_node->qp_id == xqp->xqp.qpn && !err_rq_node->is_sq) {
+ list_del(&err_rq_node->entry);
+ kfree(err_rq_node);
+ }
+ }
+ spin_unlock_irq(&to_xcq(xqp->recv_cq)->lock);
+ }
+
+ if (xqp->send_cq) {
+ spin_lock_irq(&to_xcq(xqp->send_cq)->lock);
+ list_for_each_entry_safe(err_sq_node, tmp,
+ &to_xcq(xqp->send_cq)->err_state_qp_list, entry) {
+ if (err_sq_node->qp_id == xqp->xqp.qpn && err_sq_node->is_sq) {
+ list_del(&err_sq_node->entry);
+ kfree(err_sq_node);
+ }
+ }
+ spin_unlock_irq(&to_xcq(xqp->send_cq)->lock);
+ }
+ }
+
+ if (xqp->create_type == XSC_QP_KERNEL && qp->srq) {
+ if (!to_xsrq(qp->srq) || list_empty(&to_xsrq(qp->srq)->xsrq.srq_qp_list))
+ ibdev_warn(&dev->ib_dev, "srq_qp_list is empty!\n");
+
+ xsrq = to_xsrq(qp->srq);
+ spin_lock_irqsave(&xsrq->lock, flags);
+ xsc_restore_cache_srq_wrs(xsrq, xqp);
+ list_for_each_entry_safe(qp_node, tmp_qp_node,
+ &to_xsrq(qp->srq)->xsrq.srq_qp_list, entry) {
+ if (qp_node->qp_id == xqp->xqp.qpn) {
+ to_xsrq(qp->srq)->qp_num--;
+ ibdev_dbg(&dev->ib_dev, "srq_qp_list del node qpn:%u qp_num:%d\n",
+ qp_node->qp_id, to_xsrq(qp->srq)->qp_num);
+ list_del_init(&qp_node->entry);
+ kvfree(qp_node);
+ break;
+ }
+ }
+ spin_unlock_irqrestore(&xsrq->lock, flags);
+ }
+
+ destroy_qp_common(dev, xqp);
+
+ return 0;
+}
+
+static inline u16 xsc_calc_udp_sport(u32 lqpn, u32 rqpn)
+{
+ unsigned char *p;
+ u8 ports[2];
+ u16 sport;
+ u64 tqpn;
+
+ tqpn = ((u64)(lqpn & 0xffffff)) * ((u64)(rqpn & 0xffffff));
+ p = (unsigned char *)&tqpn;
+ ports[0] = p[0] ^ p[2] ^ p[4];
+ ports[1] = p[1] ^ p[3] ^ p[5];
+ sport = *((u16 *)ports) | 0xC000;
+
+ return sport;
+}
+
+static inline void xsc_path_set_udp_sport(struct xsc_qp_path *path,
+ const struct rdma_ah_attr *ah,
+ u32 lqpn, u32 rqpn)
+{
+ if ((ah->grh.flow_label & UDP_SPORT_MASK) != 0) {
+ if ((ah->grh.flow_label & UDP_SPORT_MASK_EN) == 0)
+ path->sport = cpu_to_be16(rdma_flow_label_to_udp_sport(ah->grh.flow_label));
+ else
+ path->sport = cpu_to_be16((ah->grh.flow_label & UDP_SPORT_MASK) >>
+ UDP_SPORT_OFFSET);
+ } else {
+ path->sport = cpu_to_be16(xsc_calc_udp_sport(lqpn, rqpn));
+ }
+}
+
+static int xsc_set_path(struct xsc_ib_dev *dev, const struct rdma_ah_attr *ah,
+ struct xsc_qp_path *path, u8 port, int attr_mask,
+ u32 path_flags, const struct ib_qp_attr *attr, struct xsc_ib_qp *qp)
+{
+ struct ib_global_route *grh = rdma_ah_retrieve_grh((struct rdma_ah_attr *)ah);
+ union ib_gid *dgid = &grh->dgid;
+ const struct ib_gid_attr *sgid_attr = grh->sgid_attr;
+ union ib_gid *sgid = &((struct ib_gid_attr *)sgid_attr)->gid;
+ union {
+ struct sockaddr _sockaddr;
+ struct sockaddr_in _sockaddr_in;
+ struct sockaddr_in6 _sockaddr_in6;
+ } sgid_addr, dgid_addr;
+ int force_pcp, force_dscp;
+ char buf[256] = {0};
+ char *ptr = buf;
+ int ret = 0;
+
+ if (ah->type == RDMA_AH_ATTR_TYPE_ROCE) {
+ if (!(rdma_ah_get_ah_flags(ah) & IB_AH_GRH))
+ return -EINVAL;
+
+ if (qp->ibqp.qp_type == IB_QPT_RC ||
+ qp->ibqp.qp_type == IB_QPT_UC) {
+ if (((ah->grh.flow_label & UDP_SPORT_MASK_EN) != 0) &&
+ qp->xqp.qp_protocol_mode == RDMA_PROTO_VEROCE) {
+ ibdev_err(&dev->ib_dev,
+ "qp type is RDMA_PROTO_VEROCE, using flow_label to set the UDP sport %d is not allowed.",
+ (ah->grh.flow_label & UDP_SPORT_MASK) >>
+ UDP_SPORT_OFFSET);
+ return -EINVAL;
+ }
+ xsc_path_set_udp_sport(path, ah, qp->ibqp.qp_num, attr->dest_qp_num);
+ }
+
+ if (sgid_attr->gid_type != IB_GID_TYPE_ROCE_UDP_ENCAP) {
+ ibdev_err(&dev->ib_dev, "gid type not ROCEv2\n");
+ return -EINVAL;
+ }
+
+ force_dscp = dev->xdev->priv.force_dscp;
+ if (force_dscp == DSCP_PCP_UNSET)
+ path->ecn_dscp = (grh->traffic_class >> 2) & 0x3f;
+ else
+ path->ecn_dscp = force_dscp;
+ path->hop_limit = grh->hop_limit;
+
+ rdma_gid2ip((struct sockaddr *)&sgid_addr, sgid);
+ rdma_gid2ip((struct sockaddr *)&dgid_addr, dgid);
+
+ if (sgid_addr._sockaddr.sa_family == AF_INET &&
+ dgid_addr._sockaddr.sa_family == AF_INET) {
+ memcpy(path->sip, &sgid_addr._sockaddr_in.sin_addr.s_addr,
+ sizeof(struct in_addr));
+ memcpy(path->dip, &dgid_addr._sockaddr_in.sin_addr.s_addr,
+ sizeof(struct in_addr));
+ path->af_type = AF_INET;
+ ret += snprintf(ptr + ret, 256 - ret, "sip=%#x,",
+ be32_to_cpu(path->sip[0]));
+ ret += snprintf(ptr + ret, 256 - ret, "dip=%#x,",
+ be32_to_cpu(path->dip[0]));
+ } else if (sgid_addr._sockaddr.sa_family == AF_INET6 &&
+ dgid_addr._sockaddr.sa_family == AF_INET6) {
+ memcpy(path->sip, &sgid_addr._sockaddr_in6.sin6_addr.s6_addr,
+ sizeof(path->sip));
+ memcpy(path->dip, &dgid_addr._sockaddr_in6.sin6_addr.s6_addr,
+ sizeof(path->dip));
+ path->af_type = AF_INET6;
+ ret += snprintf(ptr + ret, 256 - ret, "sip=%08x%08x%08x%08x,",
+ be32_to_cpu(path->sip[0]), be32_to_cpu(path->sip[1]),
+ be32_to_cpu(path->sip[2]), be32_to_cpu(path->sip[3]));
+ ret += snprintf(ptr + ret, 256 - ret, "dip=%08x%08x%08x%08x,",
+ be32_to_cpu(path->dip[0]), be32_to_cpu(path->dip[1]),
+ be32_to_cpu(path->dip[2]), be32_to_cpu(path->dip[3]));
+ } else {
+ return -EINVAL;
+ }
+
+ ether_addr_copy(path->smac, dev->netdev->dev_addr);
+
+ memcpy(path->dmac, ah->roce.dmac, sizeof(ah->roce.dmac));
+ ret += snprintf(ptr + ret, 256 - ret, "smac=%02x%02x%02x%02x%02x%02x,",
+ path->smac[0], path->smac[1], path->smac[2],
+ path->smac[3], path->smac[4], path->smac[5]);
+ ret += snprintf(ptr + ret, 256 - ret, "dmac=%02x%02x%02x%02x%02x%02x",
+ path->dmac[0], path->dmac[1], path->dmac[2],
+ path->dmac[3], path->dmac[4], path->dmac[5]);
+ if (is_zero_ether_addr(path->dmac))
+ ibdev_err(&dev->ib_dev, "path dmac is zero!\n");
+ else
+ ibdev_dbg(&dev->ib_dev, "ib path info:%s\n", buf);
+
+ if (is_vlan_dev(sgid_attr->ndev)) {
+ path->vlan_valid = 1;
+ path->vlan_id = cpu_to_be16(vlan_dev_vlan_id(sgid_attr->ndev));
+
+ force_pcp = dev->xdev->priv.force_pcp;
+ if (force_pcp == DSCP_PCP_UNSET)
+ path->dci_cfi_prio_sl = (ah->sl & 0x7);
+ else
+ path->dci_cfi_prio_sl = force_pcp;
+ } else {
+ path->vlan_valid = 0;
+ }
+ }
+ ibdev_dbg(&dev->ib_dev, "path dscp %d pcp %d\n", path->ecn_dscp, path->dci_cfi_prio_sl);
+ return 0;
+}
+
+static inline void xsc_ib_trigger_cq_comp_handler(struct ib_cq *send_cq, struct ib_cq *recv_cq)
+{
+ if (send_cq)
+ send_cq->comp_handler(send_cq, send_cq->cq_context);
+
+ if (recv_cq && send_cq != recv_cq)
+ recv_cq->comp_handler(recv_cq, recv_cq->cq_context);
+}
+
+int xsc_ib_err_state_qp(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp,
+ enum xsc_qp_state cur_state, enum xsc_qp_state state)
+{
+ struct xsc_err_state_qp_node *tmp = NULL, *err_rq_node = NULL, *err_sq_node = NULL;
+ struct xsc_ib_cq *send_xcq, *recv_xcq;
+
+ ibdev_dbg(&dev->ib_dev, "modify qp: qpid %d, cur_qp_state %d, qp_state %d\n",
+ qp->xqp.qpn, cur_state, state);
+ if (cur_state == XSC_QP_STATE_ERR && state != XSC_QP_STATE_ERR) {
+ if (!qp->ibqp.srq && qp->recv_cq) {
+ recv_xcq = to_xcq(qp->recv_cq);
+ list_for_each_entry_safe(err_rq_node, tmp,
+ &recv_xcq->err_state_qp_list, entry) {
+ if (err_rq_node->qp_id == qp->xqp.qpn && !err_rq_node->is_sq) {
+ list_del(&err_rq_node->entry);
+ kfree(err_rq_node);
+ }
+ }
+ }
+
+ if (qp->send_cq) {
+ send_xcq = to_xcq(qp->send_cq);
+ list_for_each_entry_safe(err_sq_node, tmp,
+ &send_xcq->err_state_qp_list, entry) {
+ if (err_sq_node->qp_id == qp->xqp.qpn && err_sq_node->is_sq) {
+ list_del(&err_sq_node->entry);
+ kfree(err_sq_node);
+ }
+ }
+ }
+ return 0;
+ }
+
+ if (cur_state != XSC_QP_STATE_ERR && state == XSC_QP_STATE_ERR &&
+ !qp->has_trig_cq_comp_handle) {
+ if (!qp->ibqp.srq && qp->recv_cq) {
+ err_rq_node = kzalloc_obj(*err_rq_node, GFP_KERNEL);
+ if (!err_rq_node)
+ return -ENOMEM;
+ err_rq_node->qp_id = qp->xqp.qpn;
+ err_rq_node->is_sq = false;
+ recv_xcq = to_xcq(qp->recv_cq);
+ list_add_tail(&err_rq_node->entry, &recv_xcq->err_state_qp_list);
+ }
+
+ if (qp->send_cq) {
+ err_sq_node = kzalloc_obj(*err_sq_node, GFP_KERNEL);
+ if (!err_sq_node)
+ return -ENOMEM;
+ err_sq_node->qp_id = qp->xqp.qpn;
+ err_sq_node->is_sq = true;
+ send_xcq = to_xcq(qp->send_cq);
+ list_add_tail(&err_sq_node->entry, &send_xcq->err_state_qp_list);
+ }
+
+ xsc_ib_trigger_cq_comp_handler(qp->send_cq, qp->recv_cq);
+ qp->has_trig_cq_comp_handle = true;
+ }
+ return 0;
+}
+
+static inline void xsc_set_qp_access_flag(struct xsc_modify_qp_mbox_in *in,
+ struct xsc_qp_context *context, int access_flags)
+{
+ in->hdr.ver = cpu_to_be16(XSC_QP_CONTEXT_V1);
+ context->qp_access_flags = 0;
+
+ if (access_flags & IB_ACCESS_REMOTE_READ)
+ context->qp_access_flags |= QP_ACCESS_REMOTE_READ;
+ if (access_flags & IB_ACCESS_REMOTE_WRITE)
+ context->qp_access_flags |= QP_ACCESS_REMOTE_WRITE;
+ if (access_flags & IB_ACCESS_REMOTE_ATOMIC)
+ context->qp_access_flags |= QP_ACCESS_REMOTE_ATOMIC;
+
+ context->qp_access_flags = cpu_to_be32(context->qp_access_flags);
+}
+
+static int __xsc_ib_modify_qp(struct ib_qp *ibqp,
+ const struct ib_qp_attr *attr, int attr_mask,
+ enum ib_qp_state cur_state, enum ib_qp_state new_state,
+ const struct xsc_ib_modify_qp *ucmd,
+ struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev = to_xdev(ibqp->device);
+ struct xsc_ib_qp *qp = to_xqp(ibqp);
+ struct xsc_ib_cq *send_cq, *recv_cq;
+ struct xsc_qp_context *context;
+ struct xsc_modify_qp_mbox_in *in;
+ struct xsc_qp_path path;
+ int sqd_event;
+ int err;
+ char buf[256] = {0};
+ char *ptr = buf;
+ int ret = 0;
+ struct xsc_core_device *xdev = dev->xdev;
+
+ in = kzalloc_obj(*in, GFP_KERNEL);
+ if (!in)
+ return -ENOMEM;
+
+ context = &qp->ctx;
+
+ if (attr_mask & IB_QP_PATH_MTU) {
+ if (attr->path_mtu != IB_MTU_1024 &&
+ attr->path_mtu != IB_MTU_4096 &&
+ attr->path_mtu != IB_MTU_8192) {
+ ibdev_warn(&dev->ib_dev, "invalid mtu %d\n", attr->path_mtu);
+ }
+
+ if (attr->path_mtu < IB_MTU_4096)
+ context->mtu_mode = 0;
+ else if (attr->path_mtu < IB_MTU_8192)
+ context->mtu_mode = 1;
+ else
+ context->mtu_mode = 2;
+ ret = snprintf(ptr, 256, "path_mtu=%d,", attr->path_mtu);
+ }
+
+ if (attr_mask & IB_QP_DEST_QPN) {
+ context->remote_qpn = cpu_to_be32(attr->dest_qp_num);
+ ret += snprintf(ptr + ret, 256 - ret, "dest_qp_num=%d,", attr->dest_qp_num);
+ }
+
+ if (attr_mask & IB_QP_AV) {
+ err = xsc_set_path(dev, &attr->ah_attr, &path,
+ attr_mask & IB_QP_PORT ? attr->port_num : qp->port,
+ attr_mask, 0, attr, qp);
+ if (err)
+ goto out;
+
+ context->src_udp_port = path.sport;
+ context->dscp = path.ecn_dscp;
+ context->hop_limit = path.hop_limit;
+ context->ip_type = (path.af_type == AF_INET ? 0 : 1);
+ context->ip_type = cpu_to_be16(context->ip_type);
+ memcpy(context->dip, path.dip, sizeof(context->dip));
+ memcpy(context->sip, path.sip, sizeof(context->sip));
+ memcpy(context->dmac, path.dmac, sizeof(path.dmac));
+ memcpy(context->smac, path.smac, sizeof(path.smac));
+
+ context->vlan_valid = path.vlan_valid;
+ context->dci_cfi_prio_sl = path.dci_cfi_prio_sl;
+ context->vlan_id = path.vlan_id;
+ }
+
+ if (attr_mask & IB_QP_RNR_RETRY) {
+ context->rnr_retry = attr->rnr_retry;
+ ret += snprintf(ptr + ret, 256 - ret, "rnr_retry=%d,", attr->rnr_retry);
+ }
+
+ if (attr_mask & IB_QP_RETRY_CNT) {
+ context->retry_cnt = attr->retry_cnt;
+ ret += snprintf(ptr + ret, 256 - ret, "retry_cnt=%d,", attr->retry_cnt);
+ }
+
+ if (attr_mask & IB_QP_MAX_QP_RD_ATOMIC && attr->max_rd_atomic)
+ context->max_sra = min_t(u32, attr->max_rd_atomic, dev->xdev->caps.max_ra_req_qp);
+
+ if (attr_mask & IB_QP_MAX_DEST_RD_ATOMIC && attr->max_dest_rd_atomic)
+ context->max_rra = attr->max_dest_rd_atomic;
+
+ if (attr_mask & IB_QP_SQ_PSN) {
+ context->next_send_psn = cpu_to_be32(attr->sq_psn);
+ ret += snprintf(ptr + ret, 256 - ret, "sq_psn=%#x,", attr->sq_psn);
+ }
+
+ if (attr_mask & IB_QP_RQ_PSN) {
+ context->next_recv_psn = cpu_to_be32(attr->rq_psn);
+ ret += snprintf(ptr + ret, 256 - ret, "rq_psn=%#x,", attr->rq_psn);
+ }
+
+ if (attr_mask & IB_QP_ACCESS_FLAGS) {
+ xsc_set_qp_access_flag(in, context, attr->qp_access_flags);
+ ret += snprintf(ptr + ret, 256 - ret, "qp_access_flags=%#x,\n",
+ attr->qp_access_flags);
+ }
+
+ if (udata && udata->inlen && attr_mask & IB_QP_STATE && new_state == IB_QPS_RTR) {
+ if (!ucmd->profile &&
+ xdev->rdma_proto_mode == RDMA_PROTO_VEROCE) {
+ context->swith_qp_type = 1;
+ context->qp_access_flags |= QP_ACCESS_MODIFY_QP_TYPE;
+ context->qp_access_flags = cpu_to_be32(context->qp_access_flags);
+ qp->xqp.qp_protocol_mode =
+ ucmd->profile ? RDMA_PROTO_VEROCE : RDMA_PROTO_ROCEV2;
+ in->hdr.ver = cpu_to_be16(XSC_QP_CONTEXT_V2);
+ ibdev_dbg(&dev->ib_dev,
+ "local is VEROCE,remote is RoCEv2, switch qp_type from VEROCE to RoCEv2!\n");
+ }
+ }
+
+ if (attr_mask & IB_QP_STATE && new_state == IB_QPS_RESET)
+ qp->xqp.qp_protocol_mode = xdev->rdma_proto_mode;
+
+ if (cur_state == IB_QPS_RTS && new_state == IB_QPS_SQD &&
+ attr_mask & IB_QP_EN_SQD_ASYNC_NOTIFY && attr->en_sqd_async_notify)
+ sqd_event = 1;
+ else
+ sqd_event = 0;
+
+ memcpy(&in->ctx, context, sizeof(*context));
+ err = xsc_core_qp_modify(xdev, to_xsc_state(cur_state),
+ to_xsc_state(new_state), in, sqd_event,
+ &qp->xqp);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "failed to modify qp[%d] from %s to %s\n",
+ qp->xqp.qpn, qp_state_to_str(cur_state), qp_state_to_str(new_state));
+ goto out;
+ }
+
+ qp->state = new_state;
+ ibdev_dbg(&dev->ib_dev, "succeeded to modify qp[%d] from %s to %s with attr_mask=%#x, %s\n",
+ qp->xqp.qpn, qp_state_to_str(cur_state), qp_state_to_str(new_state),
+ attr_mask, buf);
+
+ if (qp->state == IB_QPS_INIT) {
+ qp->post_recv_invalid = false;
+ qp->post_send_invalid = false;
+ }
+
+ if (attr_mask & IB_QP_ACCESS_FLAGS)
+ qp->atomic_rd_en = attr->qp_access_flags;
+ if (attr_mask & IB_QP_MAX_DEST_RD_ATOMIC)
+ qp->resp_depth = attr->max_dest_rd_atomic;
+ if (attr_mask & IB_QP_PORT)
+ qp->port = attr->port_num;
+ if (attr_mask & IB_QP_ALT_PATH)
+ qp->alt_port = attr->alt_port_num;
+
+ /*
+ * If we moved a kernel QP to RESET, clean up all old CQ
+ * entries and reinitialize the QP.
+ */
+ if (new_state == IB_QPS_RESET && !ibqp->uobject) {
+ get_cqs(qp->xqp.qp_type, qp->ibqp.send_cq, qp->ibqp.recv_cq, &send_cq, &recv_cq);
+ xsc_ib_cq_clean(recv_cq, qp->xqp.qpn);
+ if (send_cq != recv_cq)
+ xsc_ib_cq_clean(send_cq, qp->xqp.qpn);
+
+ qp->rq.head = 0;
+ qp->rq.tail = 0;
+ qp->sq.head = 0;
+ qp->sq.tail = 0;
+ qp->sq.cur_post = 0;
+ qp->sq.last_poll = 0;
+ qp->sq.last_comp_ce_wqe = -1;
+ qp->sq.last_post_ce_wqe = -1;
+ qp->has_trig_cq_comp_handle = false;
+ }
+
+out:
+ kfree(in);
+ return err;
+}
+
+enum xsc_ib_qp_ex_attr_mask {
+ XSC_IB_QP_FLUSH = 1 << 21,
+};
+
+int xsc_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
+ int attr_mask, struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev = to_xdev(ibqp->device);
+ struct xsc_ib_qp *qp = to_xqp(ibqp);
+ enum ib_qp_state cur_state, new_state;
+ int err = -EINVAL;
+ int wr_num;
+ struct xsc_ib_srq *srq;
+ unsigned long flags;
+ struct xsc_ib_cq *send_cq;
+ struct xsc_ib_cq *recv_cq;
+ struct xsc_ib_modify_qp ucmd = {};
+
+ if (udata && udata->inlen) {
+ if (udata->inlen < offsetofend(typeof(ucmd), profile))
+ return -EINVAL;
+
+ if (ib_copy_from_udata(&ucmd, udata,
+ min(udata->inlen, sizeof(ucmd))))
+ return -EFAULT;
+ }
+
+ mutex_lock(&qp->mutex);
+
+ cur_state = attr_mask & IB_QP_CUR_STATE ? attr->cur_qp_state : qp->state;
+ new_state = attr_mask & IB_QP_STATE ? attr->qp_state : cur_state;
+
+ if ((attr_mask & IB_QP_PORT) &&
+ (attr->port_num == 0 || attr->port_num > dev->xdev->caps.num_ports)) {
+ ibdev_err(&dev->ib_dev, "error port num\n");
+ goto out;
+ }
+
+ if (attr_mask & IB_QP_MAX_DEST_RD_ATOMIC &&
+ attr->max_dest_rd_atomic > dev->xdev->caps.max_ra_res_qp) {
+ ibdev_err(&dev->ib_dev, "dest rd atomic:%u exeeded", attr->max_dest_rd_atomic);
+ goto out;
+ }
+
+ if (cur_state == new_state && cur_state == IB_QPS_RESET) {
+ err = 0;
+ goto out;
+ }
+
+ if (!ib_modify_qp_is_ok(cur_state, new_state, ibqp->qp_type, attr_mask)) {
+ ibdev_err(&dev->ib_dev, "ib_modify_qp not ok, type=%d mask=0x%x state from%d to%d\n",
+ ibqp->qp_type, attr_mask, cur_state, new_state);
+ goto out;
+ }
+
+ err = __xsc_ib_modify_qp(ibqp, attr, attr_mask, cur_state, new_state, &ucmd, udata);
+
+ if (!err && (attr_mask & IB_QP_STATE) && !udata) {
+ get_cqs(qp->xqp.qp_type, qp->send_cq, qp->recv_cq, &send_cq, &recv_cq);
+ xsc_ib_lock_cqs(send_cq, recv_cq);
+ err = xsc_ib_err_state_qp(dev, qp, to_xsc_state(cur_state),
+ to_xsc_state(new_state));
+ xsc_ib_unlock_cqs(send_cq, recv_cq);
+
+ if (!ibqp->srq)
+ goto out;
+
+ if (cur_state == IB_QPS_RESET && new_state == IB_QPS_INIT) {
+ srq = to_xsrq(ibqp->srq);
+ spin_lock_irqsave(&srq->lock, flags);
+ srq->qp_num++;
+ wr_num = XSC_SRQ_MIN_WR_NUM;
+ ibdev_dbg(&dev->ib_dev,
+ "SRQ %d: qp %d to INIT, post %d srq wrs, qp_num %d\n",
+ srq->xsrq.srqn, qp->xqp.qpn, wr_num, srq->qp_num);
+ xsc_post_cache_wrs(srq, qp, wr_num, true);
+ spin_unlock_irqrestore(&srq->lock, flags);
+ }
+ }
+
+out:
+ mutex_unlock(&qp->mutex);
+ return err;
+}
+
+static int xsc_wq_overflow(struct xsc_ib_wq *wq, int nreq, struct xsc_ib_cq *cq)
+{
+ unsigned int cur;
+
+ cur = wq->head - wq->tail;
+ if (likely(cur + nreq < wq->max_post))
+ return 0;
+
+ spin_lock(&cq->lock);
+ cur = wq->head - wq->tail;
+ spin_unlock(&cq->lock);
+
+ return cur + nreq >= wq->max_post;
+}
+
+static inline void xsc_post_send_db(struct xsc_ib_qp *qp,
+ struct xsc_core_device *xdev,
+ int nreq)
+{
+ u32 next_pid;
+
+ if (unlikely(!nreq))
+ return;
+
+ qp->sq.head += nreq;
+
+ next_pid = qp->sq.head << (qp->sq.wqe_shift - XSC_BASE_WQE_SHIFT);
+ xsc_update_tx_db(xdev, qp->doorbell_qpn, next_pid);
+}
+
+static inline u32 xsc_crc32(struct xsc_ib_dev *dev, u32 crc, u8 *buf, size_t len)
+{
+ u32 i;
+
+ for (i = 0; i < len; i++)
+ crc = dev->crc_32_table[(crc ^ buf[i]) & 0xff] ^ (crc >> 8);
+
+ return crc;
+}
+
+#define BTH_QPN_MASK (0x00ffffff)
+#define BTH_PSN_MASK (0x00ffffff)
+
+/* Compute a partial ICRC for all the IB transport headers. */
+static u32 xsc_icrc_hdr(struct xsc_ib_dev *dev, void *pkt, u32 size, u32 *icrc)
+{
+ struct iphdr *ip4h = NULL;
+ struct ipv6hdr *ip6h = NULL;
+ struct udphdr *udph;
+ struct ib_unpacked_eth *eth;
+ struct rxe_bth *bth;
+ struct ib_unpacked_deth *deth;
+ struct ib_unpacked_vlan *vlan;
+ int crc;
+ int crc_field_len;
+ __be16 l3_type;
+ u8 *l3_start;
+
+ int hdr_size;
+
+ /* pseudo header buffer size is calculate using ipv6 header size since
+ * it is bigger than ipv4
+ */
+ u8 pshdr[sizeof(struct udphdr) +
+ sizeof(struct ipv6hdr) +
+ sizeof(*bth) + sizeof(*deth)];
+
+ eth = pkt;
+
+ if (eth->type == htons(ETH_P_8021Q)) {
+ vlan = (struct ib_unpacked_vlan *)(eth + 1);
+ l3_type = vlan->type;
+ l3_start = (u8 *)(vlan + 1);
+ size -= 4;
+ } else {
+ l3_type = eth->type;
+ l3_start = (u8 *)(eth + 1);
+ }
+
+ hdr_size = sizeof(struct udphdr) +
+ (l3_type == htons(ETH_P_IP) ?
+ sizeof(struct iphdr) : sizeof(struct ipv6hdr));
+
+ crc_field_len = hdr_size + sizeof(*bth) + sizeof(*deth);
+
+ if (crc_field_len != size) {
+ ibdev_err(&dev->ib_dev, "Unmatched hdr: expect %d actual %d\n",
+ crc_field_len, size);
+ return -EINVAL;
+ }
+
+ ip4h = (struct iphdr *)(l3_start);
+ ip6h = (struct ipv6hdr *)(l3_start);
+ udph = (struct udphdr *)(ip4h + 1);
+ bth = (struct rxe_bth *)(udph + 1);
+
+ memcpy(pshdr, l3_start, crc_field_len);
+
+ /* This seed is the result of computing a CRC with a seed of
+ * 0xfffffff and 8 bytes of 0xff representing a masked LRH.
+ */
+ crc = 0xdebb20e3;
+
+ if (l3_type == htons(ETH_P_IP)) { /* IPv4 */
+ memcpy(pshdr, ip4h, hdr_size);
+ ip4h = (struct iphdr *)pshdr;
+ udph = (struct udphdr *)(ip4h + 1);
+
+ ip4h->ttl = 0xff;
+ ip4h->check = CSUM_MANGLED_0;
+ ip4h->tos = 0xff;
+ } else { /* IPv6 */
+ memcpy(pshdr, ip6h, hdr_size);
+ ip6h = (struct ipv6hdr *)pshdr;
+ udph = (struct udphdr *)(ip6h + 1);
+
+ memset(ip6h->flow_lbl, 0xff, sizeof(ip6h->flow_lbl));
+ ip6h->priority = 0xf;
+ ip6h->hop_limit = 0xff;
+ }
+ udph->check = CSUM_MANGLED_0;
+
+ bth = (struct rxe_bth *)(udph + 1);
+ /* exclude bth.resv8a */
+ bth->qpn |= cpu_to_be32(~BTH_QPN_MASK);
+
+ *icrc = xsc_crc32(dev, crc, pshdr, crc_field_len);
+
+ return 0;
+}
+
+/* Routine for sending QP1 packets for RoCE V1 an V2 */
+/* TODO: sq hdr buf should be created dynamically for mult entry */
+static int build_qp1_send_v2(struct xsc_ib_dev *dev,
+ struct xsc_ib_qp *qp,
+ const struct ib_send_wr *wr,
+ struct ib_sge *sge,
+ int payload_size, u32 *crc)
+{
+ struct xsc_ib_ah *ah = container_of(ud_wr((struct ib_send_wr *)wr)->ah, struct xsc_ib_ah,
+ ibah);
+ const struct ib_gid_attr *sgid_attr = ah->ibah.sgid_attr;
+ u16 ether_type;
+ union ib_gid dgid;
+ bool is_eth = false;
+ bool is_vlan = false;
+ bool is_grh = false;
+ bool is_udp = false;
+ u8 ip_version = 0;
+ u16 vlan_id = 0xFFFF;
+ int rc = 0;
+ int cm_pcp = 0;
+ void *hdr_buf;
+
+ memset(&qp->qp1_hdr, 0, sizeof(qp->qp1_hdr));
+
+ if (!qp->sq.hdr_buf) {
+ ibdev_err(&dev->ib_dev, "QP1 buffer is empty!");
+ return -ENOMEM;
+ }
+ hdr_buf = (u8 *)qp->sq.hdr_buf + MAX_QP1_SQ_HDR_SIZE_V2 * qp->sq.mad_index;
+
+ if (!sgid_attr || !sgid_attr->ndev) {
+ ibdev_err(&dev->ib_dev, "sgid_addr or ndev is null\n");
+ return -ENXIO;
+ }
+
+ if (is_vlan_dev(sgid_attr->ndev))
+ vlan_id = vlan_dev_vlan_id(sgid_attr->ndev);
+
+ is_udp = sgid_attr->gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP;
+ memcpy(&dgid.raw, &ah->av.rgid, 16);
+ if (is_udp) {
+ if (ipv6_addr_v4mapped((struct in6_addr *)&sgid_attr->gid)) {
+ ip_version = 4;
+ ether_type = ETH_P_IP;
+ } else {
+ ip_version = 6;
+ ether_type = ETH_P_IPV6;
+ }
+ is_grh = false;
+ } else {
+ ether_type = ETH_P_IBOE;
+ is_grh = true;
+ }
+
+ is_eth = true;
+ is_vlan = (vlan_id && (vlan_id < 0x1000)) ? true : false;
+
+ ib_ud_header_init(payload_size, !is_eth, is_eth, is_vlan, is_grh,
+ ip_version, is_udp, 0, &qp->qp1_hdr);
+
+ /* ETH */
+ ether_addr_copy(qp->qp1_hdr.eth.dmac_h, ah->av.rmac);
+ ether_addr_copy(qp->qp1_hdr.eth.smac_h, dev->netdev->dev_addr);
+
+ /* For vlan, check the sgid for vlan existence */
+ if (!is_vlan) {
+ qp->qp1_hdr.eth.type = cpu_to_be16(ether_type);
+ } else {
+ if (dev->xdev->priv.cm_pcp != DSCP_PCP_UNSET)
+ cm_pcp = dev->xdev->priv.cm_pcp << 13;
+ else
+ cm_pcp = (iboe_tos_to_sl(sgid_attr->ndev, ah->av.tclass) << 13);
+ qp->qp1_hdr.vlan.type = cpu_to_be16(ether_type);
+ qp->qp1_hdr.vlan.tag = cpu_to_be16(vlan_id | cm_pcp);
+ }
+
+#define ECN_CAPABLE_TRANSPORT 0x2
+ if (is_grh || ip_version == 6) {
+ memcpy(qp->qp1_hdr.grh.source_gid.raw, sgid_attr->gid.raw,
+ sizeof(sgid_attr->gid));
+ memcpy(qp->qp1_hdr.grh.destination_gid.raw, ah->av.rgid,
+ sizeof(ah->av.rgid));
+ qp->qp1_hdr.grh.hop_limit = ah->av.hop_limit;
+
+ if (dev->xdev->priv.cm_dscp != DSCP_PCP_UNSET)
+ qp->qp1_hdr.grh.traffic_class =
+ (dev->xdev->priv.cm_dscp << 2) | ECN_CAPABLE_TRANSPORT;
+ else
+ qp->qp1_hdr.grh.traffic_class = ECN_CAPABLE_TRANSPORT;
+ }
+
+ if (ip_version == 4) {
+ if (dev->xdev->priv.cm_dscp != DSCP_PCP_UNSET)
+ qp->qp1_hdr.ip4.tos =
+ (dev->xdev->priv.cm_dscp << 2) | ECN_CAPABLE_TRANSPORT;
+ else
+ qp->qp1_hdr.ip4.tos = ECN_CAPABLE_TRANSPORT;
+ qp->qp1_hdr.ip4.id = 0;
+ qp->qp1_hdr.ip4.frag_off = htons(IP_DF);
+ qp->qp1_hdr.ip4.ttl = ah->av.hop_limit;
+
+ memcpy(&qp->qp1_hdr.ip4.saddr, sgid_attr->gid.raw + 12, 4);
+ memcpy(&qp->qp1_hdr.ip4.daddr, ah->av.rgid + 12, 4);
+ qp->qp1_hdr.ip4.check = ib_ud_ip4_csum(&qp->qp1_hdr);
+ }
+
+ if (is_udp) {
+ qp->qp1_hdr.udp.dport = htons(dev->xdev->cm_udp_dst_port);
+ qp->qp1_hdr.udp.sport = htons(ah->av.udp_sport);
+ qp->qp1_hdr.udp.csum = 0;
+ ibdev_dbg(&dev->ib_dev, "CM packet used udp_sport=%d\n", ah->av.udp_sport);
+ }
+
+ /* BTH */
+ if (wr->opcode == IB_WR_SEND_WITH_IMM) {
+ qp->qp1_hdr.bth.opcode = IB_OPCODE_UD_SEND_ONLY_WITH_IMMEDIATE;
+ qp->qp1_hdr.immediate_present = 1;
+ } else {
+ qp->qp1_hdr.bth.opcode = IB_OPCODE_UD_SEND_ONLY;
+ }
+ if (wr->send_flags & IB_SEND_SOLICITED)
+ qp->qp1_hdr.bth.solicited_event = 1;
+ /* pad_count */
+ qp->qp1_hdr.bth.pad_count = (4 - payload_size) & 3;
+
+ /* P_key for QP1 is for all members */
+ qp->qp1_hdr.bth.pkey = cpu_to_be16(0xFFFF);
+ qp->qp1_hdr.bth.destination_qpn = IB_QP1;
+ qp->qp1_hdr.bth.ack_req = 0;
+ qp->send_psn++;
+ qp->send_psn &= BTH_PSN_MASK;
+ qp->qp1_hdr.bth.psn = cpu_to_be32(qp->send_psn);
+ /* DETH */
+ /* Use the privileged Q_Key for QP1 */
+ qp->qp1_hdr.deth.qkey = cpu_to_be32(IB_QP1_QKEY);
+ qp->qp1_hdr.deth.source_qpn = IB_QP1;
+
+ /* Pack the QP1 to the transmit buffer */
+ sge->addr = (dma_addr_t)(qp->sq.hdr_dma + MAX_QP1_SQ_HDR_SIZE_V2 * qp->sq.mad_index);
+ sge->lkey = 0xFFFFFFFF;
+ sge->length = MAX_QP1_SQ_HDR_SIZE;
+
+ ib_ud_header_pack(&qp->qp1_hdr, hdr_buf);
+ /*
+ * Max Header buf size for IPV6 RoCE V2 is 86,
+ * which is same as the QP1 SQ header buffer.
+ * Header buf size for IPV4 RoCE V2 can be 66.
+ * ETH(14) + VLAN(4)+ IP(20) + UDP (8) + BTH(20).
+ * Subtract 20 bytes from QP1 SQ header buf size
+ */
+ if (is_udp && ip_version == 4)
+ sge->length -= 20;
+ /*
+ * Max Header buf size for RoCE V1 is 78.
+ * ETH(14) + VLAN(4) + GRH(40) + BTH(20).
+ * Subtract 8 bytes from QP1 SQ header buf size
+ */
+ if (!is_udp)
+ sge->length -= 8;
+
+ /* Subtract 4 bytes for non vlan packets */
+ if (!is_vlan)
+ sge->length -= 4;
+
+ rc = xsc_icrc_hdr(dev, hdr_buf, sge->length - sizeof(struct ib_unpacked_eth), crc);
+ if (rc) {
+ ibdev_err(&dev->ib_dev, "CRC error: hdr size %ld\n",
+ sge->length - sizeof(struct ib_unpacked_eth));
+ }
+ return rc;
+}
+
+static void zero_send_ds(struct xsc_ib_qp *qp, int idx)
+{
+ void *seg;
+ int i;
+ int ds_num;
+ u64 *p;
+
+ ds_num = XSC_SEND_SEG_NUM << (qp->sq.wqe_shift - XSC_SEND_WQE_SHIFT);
+ seg = (void *)xsc_get_send_wqe(qp, idx);
+ for (i = 1; i < ds_num; i++) {
+ p = get_seg_wqe(seg, i);
+ p[0] = 0;
+ p[1] = 0;
+ }
+}
+
+int xsc_ib_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr,
+ const struct ib_send_wr **bad_wr)
+{
+ struct xsc_ib_qp *qp = to_xqp(ibqp);
+ struct xsc_ib_dev *dev = to_xdev(ibqp->device);
+ void *seg;
+ struct xsc_send_wqe_ctrl_seg *ctrl;
+ struct xsc_wqe_data_seg *data_seg;
+ u32 crc;
+ int nreq = 0;
+ int err = 0;
+ int i;
+ unsigned int idx;
+ unsigned long irqflag = 0;
+ struct ib_sge sg;
+ u8 *cur_p = NULL;
+ u8 *mad_send_base = NULL;
+ void *vaddr;
+ struct xsc_msg_opcode *xsc_ib_opcode = dev->xdev->board_info->msg_opcode;
+
+ spin_lock_irqsave(&qp->sq.lock, irqflag);
+
+ for (nreq = 0; wr; ++nreq, wr = wr->next) {
+ unsigned int seg_index = 1;
+ unsigned int msg_len = 0;
+ struct ib_sge *sgl = &wr->sg_list[0];
+ int sg_n = wr->num_sge;
+
+ if (unlikely(wr->opcode == IB_WR_REG_MR ||
+ wr->opcode == IB_WR_LOCAL_INV ||
+ wr->opcode == IB_WR_SEND_WITH_INV)) {
+ ibdev_err(&dev->ib_dev, "opcode %d not supported\n",
+ wr->opcode);
+ err = -EOPNOTSUPP;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(wr->opcode < 0 || wr->opcode >= ARRAY_SIZE(dev->opcode_map))) {
+ ibdev_err(&dev->ib_dev, "bad opcode %d\n", wr->opcode);
+ err = -EINVAL;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(xsc_wq_overflow(&qp->sq, nreq,
+ to_xcq(qp->ibqp.send_cq)))) {
+ ibdev_err(&dev->ib_dev, "send work queue overflow\n");
+ err = -ENOMEM;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(wr->num_sge > qp->sq.max_gs)) {
+ ibdev_err(&dev->ib_dev, "max gs exceeded %d (max = %d)\n",
+ wr->num_sge, qp->sq.max_gs);
+ err = -ENOMEM;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(wr->opcode == IB_WR_RDMA_READ && wr->num_sge > 1)) {
+ ibdev_err(&dev->ib_dev, "rdma read, max gs exceeded %d (max = 1)\n",
+ wr->num_sge);
+ err = -ENOMEM;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(qp->post_send_invalid)) {
+ ibdev_dbg(&dev->ib_dev, "qp %d has been draind, post send invalid...\n",
+ qp->xqp.qpn);
+ err = -EIO;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ idx = qp->sq.cur_post & (qp->sq.wqe_cnt - 1);
+ zero_send_ds(qp, idx);
+ seg = xsc_get_send_wqe(qp, idx);
+ ctrl = seg;
+ xsc_set_wqe_id(dev->xdev, ctrl, qp->sq.cur_post <<
+ (qp->sq.wqe_shift - XSC_BASE_WQE_SHIFT));
+ ctrl->ds_data_num = 0;
+ ctrl->se = wr->send_flags & IB_SEND_SOLICITED ? 1 : 0;
+ ctrl->ce = wr->send_flags & IB_SEND_SIGNALED ? 1 : 0;
+ for (i = 0; i < wr->num_sge; ++i) {
+ if (likely(wr->sg_list[i].length))
+ msg_len += wr->sg_list[i].length;
+ }
+ ctrl->msg_len = msg_len;
+ ctrl->with_immdt = 0;
+
+ if (unlikely(wr->opcode == IB_WR_RDMA_READ && msg_len == 0)) {
+ ibdev_err(&dev->ib_dev, "rdma read, msg len should not be 0\n");
+ /* workaround, return success for posting zero-length read */
+ err = 0;
+ goto out;
+ }
+ switch (ibqp->qp_type) {
+ case IB_QPT_RC:
+ ctrl->ds_data_num = wr->num_sge;
+ switch (wr->opcode) {
+ case IB_WR_SEND:
+ break;
+ case IB_WR_SEND_WITH_IMM:
+ ctrl->with_immdt = 1;
+ WR_LE_32(ctrl->opcode_data, RD_BE_32(send_ieth(wr)));
+ break;
+ case IB_WR_RDMA_WRITE_WITH_IMM:
+ ctrl->with_immdt = 1;
+ WR_LE_32(ctrl->opcode_data, RD_BE_32(send_ieth(wr)));
+ fallthrough;
+ case IB_WR_RDMA_READ:
+ case IB_WR_RDMA_WRITE:
+ if (ctrl->msg_len == 0)
+ break;
+ ctrl->ds_data_num++;
+ data_seg = get_seg_wqe(ctrl, seg_index);
+ set_remote_addr_seg(dev->xdev, data_seg,
+ msg_len,
+ rdma_wr(wr)->remote_addr,
+ rdma_wr(wr)->rkey);
+ seg_index++;
+ break;
+ default:
+ ibdev_err(&dev->ib_dev, "unsupported send opcode %u\n",
+ wr->opcode);
+ err = -EPERM;
+ *bad_wr = wr;
+ goto out;
+ }
+ ctrl->msg_opcode = dev->opcode_map[wr->opcode];
+ break;
+ case IB_QPT_UD:
+ case IB_QPT_GSI:
+ ctrl->msg_opcode = xsc_ib_opcode->mad;
+ ctrl->ds_data_num++;
+ data_seg = get_seg_wqe(ctrl, seg_index);
+ mad_send_base = (u8 *)qp->sq.hdr_buf +
+ MAX_QP1_SQ_HDR_SIZE_V2 * qp->sq.mad_index;
+
+ err = build_qp1_send_v2(dev, qp, wr, &sg, msg_len, &crc);
+ if (err) {
+ *bad_wr = wr;
+ goto out;
+ }
+
+ cur_p = mad_send_base + sg.length;
+ for (i = 0; i < wr->num_sge; ++i) {
+ if (likely(wr->sg_list[i].length)) {
+ vaddr = xsc_ib_send_mad_sg_virt_addr(&dev->ib_dev, wr, i);
+ memcpy(cur_p, vaddr, wr->sg_list[i].length);
+ }
+ cur_p += wr->sg_list[i].length;
+ }
+ crc = xsc_crc32(dev, crc, mad_send_base + sg.length, ctrl->msg_len);
+ ctrl->msg_len += sg.length;
+ seg_index++;
+
+ *(u32 *)&mad_send_base[ctrl->msg_len] = ~crc;
+ ctrl->msg_len += sizeof(crc);
+ sg.length = ctrl->msg_len;
+ set_local_data_seg(dev->xdev, data_seg, &sg);
+ ibdev_dbg(&dev->ib_dev, "qp[%d] send MAD packet, msg_len:%d\n",
+ qp->xqp.qpn, ctrl->msg_len);
+ qp->sq.mad_index = (qp->sq.mad_index + 1) % MAD_QUEUE_DEPTH;
+
+ sg_n = 0;
+ break;
+ default:
+ ibdev_err(&dev->ib_dev, "qp type:%u NOT supported\n", ibqp->qp_type);
+ err = -EPERM;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
+ err = set_data_inl_seg(qp, wr, ctrl);
+ if (unlikely(err)) {
+ *bad_wr = wr;
+ ibdev_err(&dev->ib_dev, "inline layout failed, err %d\n", err);
+ goto out;
+ }
+ } else {
+ for (i = 0; i < sg_n; ++i, ++seg_index) {
+ if (likely(sgl[i].length)) {
+ data_seg = get_seg_wqe(ctrl, seg_index);
+ set_local_data_seg(dev->xdev, data_seg, &sgl[i]);
+ }
+ }
+ }
+ qp->sq.wrid[idx] = wr->wr_id;
+ qp->sq.wqe_head[idx] = qp->sq.head + nreq;
+ qp->sq.cur_post += 1;
+ if (ctrl->ce) {
+ atomic_inc(&qp->sq.flush_wqe_cnt);
+ qp->sq.need_flush[idx] = 1;
+ qp->sq.prev_ce_wqe[idx] = qp->sq.last_post_ce_wqe;
+ qp->sq.last_post_ce_wqe = idx;
+ }
+ qp->sq.wr_opcode[idx] = wr->opcode;
+ }
+out:
+ ibdev_dbg(&dev->ib_dev, "nreq:%d\n", nreq);
+ xsc_post_send_db(qp, dev->xdev, nreq);
+ spin_unlock_irqrestore(&qp->sq.lock, irqflag);
+
+ return err;
+}
+
+int xsc_ib_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr,
+ const struct ib_recv_wr **bad_wr)
+{
+ struct xsc_ib_qp *qp = to_xqp(ibqp);
+ struct xsc_ib_dev *dev = to_xdev(ibqp->device);
+ struct xsc_core_device *xdev = dev->xdev;
+ struct xsc_wqe_data_seg *recv_head;
+ struct xsc_wqe_data_seg *data_seg;
+ unsigned long flags = 0;
+ int err = 0;
+ u32 next_pid = 0;
+ int nreq = 0;
+ u32 idx;
+ int i;
+
+ spin_lock_irqsave(&qp->rq.lock, flags);
+
+ idx = qp->rq.head & (qp->rq.wqe_cnt - 1);
+
+ for (nreq = 0; wr; ++nreq, wr = wr->next) {
+ if (unlikely(xsc_wq_overflow(&qp->rq, nreq, to_xcq(qp->ibqp.recv_cq)))) {
+ ibdev_err(&dev->ib_dev, "recv work queue overflow\n");
+ err = -ENOMEM;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(wr->num_sge > qp->rq.max_gs)) {
+ ibdev_err(&dev->ib_dev, "max gs exceeded %d (max = %d)\n",
+ wr->num_sge, qp->rq.max_gs);
+ err = -EINVAL;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(qp->post_recv_invalid)) {
+ ibdev_dbg(&dev->ib_dev, "qp %d has been draind, post recv invalid...\n",
+ qp->xqp.qpn);
+ err = -EIO;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ recv_head = get_recv_wqe(qp, idx);
+
+ for (i = 0; i < wr->num_sge; ++i) {
+ if (unlikely(!wr->sg_list[i].length))
+ continue;
+ data_seg = get_seg_wqe(recv_head, i);
+ data_seg->in_line = 0;
+ WR_LE_64(data_seg->va, wr->sg_list[i].addr);
+ WR_LE_32(data_seg->mkey, wr->sg_list[i].lkey);
+ if (is_qp1(qp->xqp.qp_type))
+ WR_LE_32(data_seg->seg_len, xdev->caps.rx_pkt_len_max);
+ else
+ WR_LE_32(data_seg->seg_len, wr->sg_list[i].length);
+ }
+
+ qp->rq.wrid[idx] = wr->wr_id;
+
+ idx = (idx + 1) & (qp->rq.wqe_cnt - 1);
+ atomic_inc(&qp->rq.flush_wqe_cnt);
+ }
+
+out:
+ if (likely(nreq)) {
+ qp->rq.head += nreq;
+ next_pid = qp->rq.head << (qp->rq.wqe_shift - XSC_BASE_WQE_SHIFT);
+
+ xsc_update_rx_db(xdev, qp->doorbell_qpn, next_pid);
+ }
+
+ spin_unlock_irqrestore(&qp->rq.lock, flags);
+
+ return err;
+}
+
+static inline enum ib_qp_state to_ib_qp_state(enum xsc_qp_state xsc_state)
+{
+ switch (xsc_state) {
+ case XSC_QP_STATE_RST: return IB_QPS_RESET;
+ case XSC_QP_STATE_INIT: return IB_QPS_INIT;
+ case XSC_QP_STATE_RTR: return IB_QPS_RTR;
+ case XSC_QP_STATE_RTS: return IB_QPS_RTS;
+ case XSC_QP_STATE_SQ_DRAINING:
+ case XSC_QP_STATE_SQD: return IB_QPS_SQD;
+ case XSC_QP_STATE_SQER: return IB_QPS_SQE;
+ case XSC_QP_STATE_ERR: return IB_QPS_ERR;
+ default: return -1;
+ }
+}
+
+static inline enum ib_mig_state to_ib_mig_state(int xsc_mig_state)
+{
+ switch (xsc_mig_state) {
+ case XSC_QP_PM_ARMED: return IB_MIG_ARMED;
+ case XSC_QP_PM_REARM: return IB_MIG_REARM;
+ case XSC_QP_PM_MIGRATED: return IB_MIG_MIGRATED;
+ default: return -1;
+ }
+}
+
+int xsc_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr_mask,
+ struct ib_qp_init_attr *qp_init_attr)
+{
+ struct xsc_ib_dev *dev = to_xdev(ibqp->device);
+ struct xsc_ib_qp *qp = to_xqp(ibqp);
+ struct xsc_query_qp_mbox_out *outb;
+ struct xsc_qp_context *context;
+ int err = 0;
+
+ mutex_lock(&qp->mutex);
+
+ if (qp_attr_mask & XSC_IB_QP_FLUSH) {
+ xsc_ib_trigger_cq_comp_handler(qp->send_cq, qp->recv_cq);
+ goto out;
+ }
+
+ outb = kzalloc_obj(*outb, GFP_KERNEL);
+ if (!outb) {
+ err = -ENOMEM;
+ goto out;
+ }
+ context = &outb->ctx;
+ err = xsc_core_qp_query(dev->xdev, &qp->xqp, outb, sizeof(*outb));
+ if (err)
+ goto out_free;
+
+ if (qp->xqp.err_occurred) {
+ qp->state = IB_QPS_ERR;
+ qp->xqp.err_occurred = 0;
+ }
+ qp_attr->qp_state = qp->state;
+ qp_attr->path_mtu = context->mtu_mode ? IB_MTU_4096 : IB_MTU_1024;
+ qp_attr->rq_psn = be32_to_cpu(context->next_recv_psn) & 0xffffff;
+ qp_attr->sq_psn = be32_to_cpu(context->next_send_psn) & 0xffffff;
+ qp_attr->dest_qp_num = be32_to_cpu(context->remote_qpn) & 0xffffff;
+ qp_attr->sq_draining = 0;
+ qp_attr->retry_cnt = context->retry_cnt;
+ qp_attr->rnr_retry = context->rnr_retry;
+ qp_attr->cur_qp_state = qp_attr->qp_state;
+ qp_attr->cap.max_recv_wr = qp->rq.wqe_cnt;
+ qp_attr->cap.max_recv_sge = qp->rq.max_gs;
+ qp_attr->max_rd_atomic = context->max_sra;
+ qp_attr->max_dest_rd_atomic = context->max_rra;
+ qp_attr->port_num = 1;
+
+ if (!ibqp->uobject) {
+ qp_attr->cap.max_send_wr = qp->sq.wqe_cnt;
+ qp_attr->cap.max_send_sge = qp->sq.max_gs;
+ } else {
+ qp_attr->cap.max_send_wr = qp->sq.wqe_cnt;
+ qp_attr->cap.max_send_sge = qp->sq.max_gs;
+ }
+
+ /* We don't support inline sends for kernel QPs (yet), and we
+ * don't know what userspace's value should be.
+ */
+ qp_attr->cap.max_inline_data = 0;
+
+ qp_init_attr->cap = qp_attr->cap;
+
+ qp_init_attr->create_flags = 0;
+ if (qp->flags & XSC_IB_QP_BLOCK_MULTICAST_LOOPBACK)
+ qp_init_attr->create_flags |= IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK;
+
+ qp_init_attr->sq_sig_type = qp->sq_signal_bits & XSC_WQE_CTRL_CQ_UPDATE ?
+ IB_SIGNAL_ALL_WR : IB_SIGNAL_REQ_WR;
+
+out_free:
+ kfree(outb);
+
+out:
+ mutex_unlock(&qp->mutex);
+ return err;
+}
+
+struct xsc_ib_drain_cqe {
+ struct ib_cqe cqe;
+ struct completion done;
+ int qpn;
+};
+
+static void xsc_ib_drain_rq_done(struct ib_cq *cq, struct ib_wc *wc)
+{
+ struct xsc_ib_drain_cqe *cqe = container_of(wc->wr_cqe,
+ struct xsc_ib_drain_cqe, cqe);
+ struct xsc_ib_dev *dev = to_xdev(cq->device);
+
+ complete(&cqe->done);
+ ibdev_dbg(&dev->ib_dev, "QP %d ib_drain_rq done!", cqe->qpn);
+}
+
+static void xsc_ib_drain_sq_done(struct ib_cq *cq, struct ib_wc *wc)
+{
+ struct xsc_ib_drain_cqe *cqe = container_of(wc->wr_cqe,
+ struct xsc_ib_drain_cqe, cqe);
+ struct xsc_ib_dev *dev = to_xdev(cq->device);
+
+ complete(&cqe->done);
+ ibdev_dbg(&dev->ib_dev, "QP %d ib_drain_sq done!", cqe->qpn);
+}
+
+void xsc_ib_drain_rq(struct ib_qp *qp)
+{
+ struct ib_cq *cq = qp->recv_cq;
+ struct xsc_ib_cq *xcq = to_xcq(cq);
+ struct xsc_ib_qp *xqp = to_xqp(qp);
+ struct ib_qp_attr attr = { .qp_state = IB_QPS_ERR };
+ struct xsc_ib_drain_cqe rdrain;
+ const struct ib_recv_wr rwr = {
+ .next = NULL,
+ {.wr_cqe = &rdrain.cqe,},
+ };
+ const struct ib_recv_wr *bad_rwr;
+ int ret;
+ struct xsc_ib_dev *dev = to_xdev(qp->device);
+
+ ret = ib_modify_qp(qp, &attr, IB_QP_STATE);
+ if (ret) {
+ ibdev_err(&dev->ib_dev, "failed to drain recv queue: %d\n", ret);
+ return;
+ }
+
+ rdrain.qpn = xqp->xqp.qpn;
+ rdrain.cqe.done = xsc_ib_drain_rq_done;
+ init_completion(&rdrain.done);
+
+ ret = xsc_ib_post_recv(qp, &rwr, &bad_rwr);
+ if (ret) {
+ ibdev_err(&dev->ib_dev, "failed to drain recv queue: %d\n", ret);
+ return;
+ }
+ xqp->post_recv_invalid = true;
+
+ xcq->xcq.comp(&xcq->xcq);
+ wait_for_completion(&rdrain.done);
+}
+
+void xsc_ib_drain_sq(struct ib_qp *qp)
+{
+ struct ib_cq *cq = qp->send_cq;
+ struct xsc_ib_cq *xcq = to_xcq(cq);
+ struct xsc_ib_qp *xqp = to_xqp(qp);
+ struct ib_qp_attr attr = { .qp_state = IB_QPS_ERR };
+ struct xsc_ib_drain_cqe sdrain;
+ const struct ib_send_wr *bad_swr;
+ const struct ib_send_wr swr = {
+ .next = NULL,
+ { .wr_cqe = &sdrain.cqe, },
+ .opcode = IB_WR_RDMA_WRITE,
+ .send_flags = IB_SEND_SIGNALED,
+ };
+ int ret;
+ struct xsc_ib_dev *dev = to_xdev(qp->device);
+
+ ret = ib_modify_qp(qp, &attr, IB_QP_STATE);
+ if (ret) {
+ ibdev_err(&dev->ib_dev, "failed to drain send queue: %d\n", ret);
+ return;
+ }
+
+ sdrain.qpn = xqp->xqp.qpn;
+ sdrain.cqe.done = xsc_ib_drain_sq_done;
+ init_completion(&sdrain.done);
+
+ ret = xsc_ib_post_send(qp, &swr, &bad_swr);
+ if (ret) {
+ ibdev_err(&dev->ib_dev, "failed to drain send queue: %d\n", ret);
+ return;
+ }
+ xqp->post_send_invalid = true;
+
+ xcq->xcq.comp(&xcq->xcq);
+ wait_for_completion(&sdrain.done);
+}
diff --git a/drivers/infiniband/hw/xsc/srq.c b/drivers/infiniband/hw/xsc/srq.c
new file mode 100644
index 0000000000000..911820fa1e7e7
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/srq.c
@@ -0,0 +1,426 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/slab.h>
+#include <rdma/ib_umem.h>
+#include <rdma/ib_user_verbs.h>
+#include "xsc_ib.h"
+#include "common/srq.h"
+#include <rdma/xsc-abi.h>
+#include <linux/list_sort.h>
+
+static int xsc_init_cache_wr(struct ib_srq_init_attr *attr, struct xsc_ib_srq *srq)
+{
+ int srqn = srq->xsrq.srqn;
+ struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
+
+ if (srqn >= XSC_MAX_SRQ_NUM)
+ return -EINVAL;
+
+ memset(&cache_wr[srqn], 0, sizeof(struct xsc_srq_cache_wr));
+ cache_wr[srqn].max_wr_num = attr->attr.max_wr;
+ return 0;
+}
+
+void xsc_restore_cache_srq_wrs(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp)
+{
+ int srqn = srq->xsrq.srqn;
+ struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
+
+ if (qp->srq_tail_wr)
+ qp->srq_tail_wr->next = cache_wr[srqn].head_wr;
+ if (cache_wr[srqn].head_wr && qp->srq_head_wr)
+ cache_wr[srqn].head_wr = qp->srq_head_wr;
+}
+
+void xsc_free_one_cache_srq_wr(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp, u64 wr_id)
+{
+ struct ib_recv_wr *tmp_wr, *prev_wr = NULL;
+ int srqn = srq->xsrq.srqn;
+ struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
+ unsigned long flags;
+
+ spin_lock_irqsave(&srq->lock, flags);
+ tmp_wr = qp->srq_head_wr;
+ for (; tmp_wr; tmp_wr = tmp_wr->next) {
+ if (tmp_wr->wr_id == wr_id) {
+ if (prev_wr)
+ prev_wr->next = tmp_wr->next;
+ else
+ qp->srq_head_wr = tmp_wr->next;
+
+ kfree(tmp_wr->sg_list);
+ kfree(tmp_wr);
+ break;
+ }
+ prev_wr = tmp_wr;
+ }
+ cache_wr[srqn].wr_num--;
+ spin_unlock_irqrestore(&srq->lock, flags);
+}
+
+void xsc_free_all_cache_wrs(struct ib_recv_wr *wr)
+{
+ struct ib_recv_wr *prev_wr = NULL;
+
+ for (; wr; wr = wr->next) {
+ kfree(prev_wr);
+ kfree(wr->sg_list);
+ prev_wr = wr;
+ }
+ kfree(prev_wr);
+}
+
+static void xsc_destroy_cache_wr(struct xsc_ib_srq *srq)
+{
+ int srqn = srq->xsrq.srqn;
+ struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
+
+ xsc_free_all_cache_wrs(cache_wr[srqn].head_wr);
+ memset(&cache_wr[srqn], 0, sizeof(struct xsc_srq_cache_wr));
+}
+
+static void xsc_ib_srq_event(struct xsc_core_srq *srq, enum xsc_event type)
+{
+ struct ib_event event;
+ struct ib_srq *ibsrq = &core_to_xsrq(srq)->ibsrq;
+
+ if (ibsrq->event_handler) {
+ event.device = ibsrq->device;
+ event.element.srq = ibsrq;
+ switch (type) {
+ default:
+ pr_warn("xsc_ib: Unexpected event type %d on SRQ %06x\n",
+ type, srq->srqn);
+ return;
+ }
+
+ ibsrq->event_handler(&event, ibsrq->srq_context);
+ }
+}
+
+static int create_srq_user(struct ib_pd *pd, struct xsc_ib_srq *srq,
+ struct ib_srq_init_attr *init_attr,
+ struct ib_udata *udata, int buf_size)
+{
+ struct xsc_ib_dev *dev = to_xdev(pd->device);
+ struct xsc_ib_create_srq ucmd = {};
+ size_t ucmdlen;
+
+ ucmdlen = min(udata->inlen, sizeof(ucmd));
+
+ if (ib_copy_from_udata(&ucmd, udata, ucmdlen)) {
+ ibdev_dbg(&dev->ib_dev, "failed copy udata\n");
+ return -EFAULT;
+ }
+
+ if (ucmd.reserved0 || ucmd.reserved1)
+ return -EINVAL;
+
+ if (udata->inlen > sizeof(ucmd) &&
+ !ib_is_udata_cleared(udata, sizeof(ucmd),
+ udata->inlen - sizeof(ucmd)))
+ return -EINVAL;
+
+ srq->create_type = XSC_SRQ_USER;
+
+ return 0;
+}
+
+static int create_srq_kernel(struct xsc_ib_dev *dev, struct xsc_ib_srq *srq,
+ struct ib_srq_init_attr *init_attr, int buf_size)
+{
+ srq->create_type = XSC_SRQ_KERNEL;
+ return 0;
+}
+
+static void destroy_srq_user(struct xsc_ib_srq *srq, struct ib_udata *udata)
+{
+}
+
+static void destroy_srq_kernel(struct xsc_ib_dev *dev, struct xsc_ib_srq *srq)
+{
+}
+
+int xsc_ib_create_srq(struct ib_srq *ib_srq,
+ struct ib_srq_init_attr *init_attr,
+ struct ib_udata *udata)
+{
+ struct xsc_ib_dev *dev;
+ struct xsc_ib_srq *srq;
+ int err;
+ u32 max_srq_wqes = 1 << XSC_MAX_SRQ_LOG_SIZE;
+ struct ib_pd *pd = ib_srq->pd;
+
+ if (pd) {
+ dev = to_xdev(pd->device);
+ } else {
+ pr_warn("%s: no PD for SRQ\n", __func__);
+ err = -EINVAL;
+ goto err_ret;
+ }
+
+ if (init_attr->srq_type != IB_SRQT_BASIC) {
+ err = -EOPNOTSUPP;
+ goto err_ret;
+ }
+
+ /* Sanity check SRQ size before proceeding */
+ if (init_attr->attr.max_wr >= max_srq_wqes) {
+ ibdev_dbg(&dev->ib_dev, "max_wr %d, cap %d\n",
+ init_attr->attr.max_wr,
+ max_srq_wqes);
+ err = -EINVAL;
+ goto err_ret;
+ }
+
+ srq = to_xsrq(ib_srq);
+
+ mutex_init(&srq->mutex);
+ spin_lock_init(&srq->lock);
+ srq->xsrq.max = roundup_pow_of_two(init_attr->attr.max_wr + 1);
+ srq->xsrq.max_gs = init_attr->attr.max_sge;
+
+ if (udata)
+ err = create_srq_user(pd, srq, NULL, udata, 0);
+ else
+ err = create_srq_kernel(dev, srq, init_attr, 0);
+
+ if (err) {
+ ibdev_dbg(&dev->ib_dev, "create srq rsc failed, ret %d", err);
+ goto err_create_rsc;
+ }
+
+ err = xsc_core_create_srq(dev->xdev, &srq->xsrq);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "create srq failed, err=%d\n", err);
+ goto err_create;
+ }
+ ibdev_dbg(&dev->ib_dev, "create SRQ with srqn 0x%x, max_wr %d\n",
+ srq->xsrq.srqn, init_attr->attr.max_wr);
+
+ srq->xsrq.event = xsc_ib_srq_event;
+ srq->ibsrq.ext.xrc.srq_num = srq->xsrq.srqn;
+
+ if (udata) {
+ struct xsc_ib_create_srq_resp resp = {
+ .srqn = srq->xsrq.srqn,
+ };
+
+ if (ib_copy_to_udata(udata, &resp,
+ min(udata->outlen, sizeof(resp)))) {
+ ibdev_dbg(&dev->ib_dev, "copy to user failed\n");
+ err = -EFAULT;
+ goto err_rsp;
+ }
+ } else {
+ err = xsc_init_cache_wr(init_attr, srq);
+ if (err)
+ goto err_rsp;
+ INIT_LIST_HEAD(&srq->xsrq.srq_qp_list);
+ }
+
+ init_attr->attr.max_wr = srq->xsrq.max - 1;
+
+ return 0;
+err_rsp:
+ xsc_core_destroy_srq(dev->xdev, &srq->xsrq);
+err_create:
+ if (udata)
+ destroy_srq_user(srq, udata);
+ else
+ destroy_srq_kernel(dev, srq);
+err_create_rsc:
+err_ret:
+ return err;
+}
+
+int xsc_ib_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr,
+ enum ib_srq_attr_mask attr_mask, struct ib_udata *udata)
+{
+ /* We don't support resizing SRQs yet */
+ if (attr_mask & IB_SRQ_MAX_WR)
+ return -EINVAL;
+
+ if (attr_mask & IB_SRQ_LIMIT)
+ return -EOPNOTSUPP;
+
+ return 0;
+}
+
+int xsc_ib_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *srq_attr)
+{
+ struct xsc_ib_srq *srq = to_xsrq(ibsrq);
+
+ srq_attr->srq_limit = 0;
+ srq_attr->max_wr = srq->xsrq.max - 1;
+ srq_attr->max_sge = srq->xsrq.max_gs;
+
+ return 0;
+}
+
+int xsc_ib_destroy_srq(struct ib_srq *srq, struct ib_udata *udata)
+{
+ struct xsc_ib_srq *xsrq = to_xsrq(srq);
+ struct xsc_ib_dev *dev = to_xdev(srq->device);
+
+ xsc_core_destroy_srq(dev->xdev, &xsrq->xsrq);
+ if (xsrq->create_type == XSC_SRQ_USER) {
+ destroy_srq_user(xsrq, NULL);
+ } else {
+ xsc_destroy_cache_wr(xsrq);
+ destroy_srq_kernel(dev, xsrq);
+ }
+
+ ibdev_dbg(&dev->ib_dev, "destroy srq %d successfully.\n", xsrq->xsrq.srqn);
+ return 0;
+}
+
+static int cmp_flush_wqe_cnt(void *priv,
+ const struct list_head *a,
+ const struct list_head *b)
+{
+ const struct xsc_srq_qp_node *na =
+ list_entry(a, struct xsc_srq_qp_node, entry);
+ const struct xsc_srq_qp_node *nb =
+ list_entry(b, struct xsc_srq_qp_node, entry);
+
+ int fa = atomic_read(&(to_xqp(na->ibqp)->rq.flush_wqe_cnt));
+ int fb = atomic_read(&(to_xqp(nb->ibqp)->rq.flush_wqe_cnt));
+
+ return fa - fb;
+}
+
+static int xsc_cache_srq_wr(struct xsc_ib_srq *srq, const struct ib_recv_wr *wr, int *nreq)
+{
+ int i, n;
+ int srqn = srq->xsrq.srqn;
+ struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
+ struct ib_recv_wr *head_wr = NULL, *new_wr = NULL, *prev_wr = NULL;
+
+ for (n = 0; wr; wr = wr->next, n++) {
+ new_wr = kcalloc(1, sizeof(struct ib_recv_wr), GFP_KERNEL);
+ if (prev_wr)
+ prev_wr->next = new_wr;
+ else
+ head_wr = new_wr;
+ new_wr->wr_id = wr->wr_id;
+ new_wr->sg_list = kcalloc(wr->num_sge, sizeof(struct ib_sge), GFP_KERNEL);
+ for (i = 0; i < wr->num_sge; i++)
+ memcpy(&new_wr->sg_list[i], &wr->sg_list[i], sizeof(struct ib_sge));
+ new_wr->num_sge = wr->num_sge;
+ prev_wr = new_wr;
+ };
+
+ if (!cache_wr[srqn].head_wr)
+ cache_wr[srqn].head_wr = head_wr;
+ else
+ cache_wr[srqn].tail_wr->next = head_wr;
+
+ cache_wr[srqn].tail_wr = new_wr;
+ cache_wr[srqn].wr_num += n;
+ *nreq = n;
+
+ if (cache_wr[srqn].wr_num > cache_wr[srqn].max_wr_num)
+ return -1;
+ else
+ return 0;
+}
+
+void xsc_post_cache_wrs(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp, int wr_num, bool init)
+{
+ int srqn = srq->xsrq.srqn;
+ struct xsc_srq_cache_wr *cache_wr = srq->xsrq.cache_wr;
+ int err = 0;
+ int i;
+ const struct ib_recv_wr *tmp_wr;
+ const struct ib_recv_wr *bad_wr;
+ struct ib_recv_wr *wr_head, *next_wr;
+
+ if (!cache_wr[srqn].head_wr)
+ return;
+
+ wr_head = cache_wr[srqn].head_wr;
+ next_wr = cache_wr[srqn].head_wr;
+
+ if (init)
+ qp->srq_head_wr = wr_head;
+ else
+ qp->srq_tail_wr->next = wr_head;
+
+ qp->srq_tail_wr = wr_head;
+ for (i = 0; next_wr && i < (wr_num - 1); i++) {
+ next_wr = next_wr->next;
+ if (next_wr)
+ qp->srq_tail_wr = next_wr;
+ }
+
+ cache_wr[srqn].head_wr = qp->srq_tail_wr->next;
+ qp->srq_tail_wr->next = NULL;
+ tmp_wr = wr_head;
+ err = xsc_ib_post_recv(&qp->ibqp, tmp_wr, &bad_wr);
+ if (err) {
+ qp->srq_tail_wr->next = cache_wr[srqn].head_wr;
+ qp->srq_tail_wr = NULL;
+ for (; wr_head && (bad_wr->wr_id != wr_head->wr_id); wr_head = wr_head->next)
+ qp->srq_tail_wr = wr_head;
+
+ if (!qp->srq_tail_wr) {
+ cache_wr[srqn].head_wr = wr_head;
+ qp->srq_head_wr = NULL;
+ } else {
+ cache_wr[srq->xsrq.srqn].head_wr = qp->srq_tail_wr->next;
+ qp->srq_tail_wr->next = NULL;
+ }
+ }
+}
+
+int xsc_ib_post_srq_recv(struct ib_srq *ibsrq, const struct ib_recv_wr *wr,
+ const struct ib_recv_wr **bad_wr)
+{
+ struct xsc_ib_srq *srq = to_xsrq(ibsrq);
+ struct xsc_ib_dev *dev = to_xdev(ibsrq->device);
+ struct xsc_srq_qp_node *qp_node;
+ struct xsc_ib_qp *qp;
+ unsigned long flags;
+ int err = 0;
+ int flush_wqe_cnt = 0;
+ int nreq = 0;
+ int wr_num = 0;
+
+ spin_lock_irqsave(&srq->lock, flags);
+
+ err = xsc_cache_srq_wr(srq, wr, &nreq);
+ if (err) {
+ ibdev_err(&dev->ib_dev, "No more space to cache SRQ WRs...");
+ err = -EINVAL;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (list_empty(&srq->xsrq.srq_qp_list))
+ goto out;
+
+ list_sort(NULL, &srq->xsrq.srq_qp_list, cmp_flush_wqe_cnt);
+
+ list_for_each_entry(qp_node, &srq->xsrq.srq_qp_list, entry) {
+ if (!qp_node || nreq <= 0)
+ break;
+
+ qp = to_xqp(qp_node->ibqp);
+ flush_wqe_cnt = atomic_read(&qp->rq.flush_wqe_cnt);
+ wr_num = min_t(int, nreq, (XSC_SRQ_MIN_WR_NUM - flush_wqe_cnt));
+ nreq -= wr_num;
+ ibdev_dbg(&dev->ib_dev, "srq_qp_list qp node qp %d, flush cnt %d, nreq %d, wr_num %d\n",
+ qp_node->qp_id, flush_wqe_cnt, nreq, wr_num);
+ xsc_post_cache_wrs(srq, qp, wr_num, false);
+ }
+
+out:
+ spin_unlock_irqrestore(&srq->lock, flags);
+
+ return err;
+}
diff --git a/drivers/infiniband/hw/xsc/xsc_ib.h b/drivers/infiniband/hw/xsc/xsc_ib.h
new file mode 100644
index 0000000000000..a2ae382e22ce8
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/xsc_ib.h
@@ -0,0 +1,542 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_IB_H
+#define XSC_IB_H
+
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <rdma/ib_verbs.h>
+#include <rdma/ib_smi.h>
+#include <rdma/ib_pack.h>
+#include <rdma/ib_mad.h>
+#include "common/xsc_core.h"
+#include "common/driver.h"
+#include "common/cq.h"
+#include "common/srq.h"
+#include "common/qp.h"
+#include <linux/types.h>
+#include <linux/iommu.h>
+#include <linux/dma-direct.h>
+#include <linux/dma-mapping.h>
+#include <crypto/hash.h>
+#include <rdma/uverbs_ioctl.h>
+
+#define XSC_PAGE_SHIFT_4K 12
+#define XSC_PAGE_SHIFT_64K 16
+#define XSC_PAGE_SHIFT_2M 21
+#define XSC_PAGE_SHIFT_1G 30
+#define XSC_PAGE_SZ_4K BIT(XSC_PAGE_SHIFT_4K)
+#define XSC_PAGE_SZ_64K BIT(XSC_PAGE_SHIFT_64K)
+#define XSC_PAGE_SZ_2M BIT(XSC_PAGE_SHIFT_2M)
+#define XSC_PAGE_SZ_1G BIT(XSC_PAGE_SHIFT_1G)
+#define XSC_MR_PAGE_CAP_MASK (XSC_PAGE_SZ_4K | \
+ XSC_PAGE_SZ_64K | \
+ XSC_PAGE_SZ_2M | \
+ XSC_PAGE_SZ_1G)
+
+
+struct xsc_ib_ucontext {
+ struct ib_ucontext ibucontext;
+};
+
+struct xsc_ib_pd {
+ struct ib_pd ibpd;
+ u32 pdn;
+ u32 pa_lkey;
+};
+
+struct wr_list {
+ u16 opcode;
+ u16 next;
+};
+
+struct xsc_ib_wq {
+ u64 *wrid;
+ u32 *wr_data;
+ struct wr_list *w_list;
+ unsigned long *wqe_head;
+
+ /* serialize post to the work queue
+ */
+ spinlock_t lock;
+ int wqe_cnt;
+ int ds_cnt;
+ int max_post;
+ int max_gs;
+ int offset;
+ int wqe_shift;
+ unsigned int head;
+ unsigned int tail;
+ u32 cur_post;
+ u32 last_poll;
+ void *qend;
+ void *hdr_buf;
+ u32 hdr_size;
+ dma_addr_t hdr_dma;
+ int mad_queue_depth;
+ int mad_index;
+ u32 *wr_opcode;
+ u32 *need_flush;
+ atomic_t flush_wqe_cnt;
+ int *prev_ce_wqe;
+ int last_comp_ce_wqe;
+ int last_post_ce_wqe;
+};
+
+enum {
+ XSC_QP_USER,
+ XSC_QP_KERNEL,
+};
+
+struct xsc_ib_qp {
+ struct ib_qp ibqp;
+ struct xsc_core_qp xqp;
+ struct xsc_buf buf;
+
+ struct xsc_ib_wq rq;
+
+ u32 doorbell_qpn;
+ u8 sq_signal_bits;
+ struct xsc_ib_wq sq;
+
+ struct ib_umem *umem;
+ int buf_size;
+
+ /* serialize qp state modifications
+ */
+ struct mutex mutex;
+ u32 flags;
+ u8 port;
+ u8 alt_port;
+ u8 atomic_rd_en;
+ u8 resp_depth;
+ u8 state;
+ int max_inline_data;
+ int has_rq;
+
+ int create_type;
+ u32 pa_lkey;
+ /* For QP1 */
+ struct ib_ud_header qp1_hdr;
+ u32 send_psn;
+ struct xsc_qp_context ctx;
+ struct ib_cq *send_cq;
+ struct ib_cq *recv_cq;
+ /* For qp resources */
+ spinlock_t lock;
+
+ struct list_head cq_recv_list;
+ struct list_head cq_send_list;
+ bool post_send_invalid;
+ bool post_recv_invalid;
+ struct ib_recv_wr *srq_head_wr;
+ struct ib_recv_wr *srq_tail_wr;
+ bool has_trig_cq_comp_handle;
+};
+
+struct xsc_ib_cq_buf {
+ struct xsc_buf buf;
+ struct ib_umem *umem;
+ int cqe_size;
+};
+
+enum xsc_ib_qp_flags {
+ XSC_IB_QP_BLOCK_MULTICAST_LOOPBACK = 1 << 0,
+ XSC_IB_QP_SIGNATURE_HANDLING = 1 << 1,
+};
+
+struct xsc_ib_cq {
+ struct ib_cq ibcq;
+ struct xsc_core_cq xcq;
+ struct xsc_ib_cq_buf buf;
+
+ /* serialize access to the CQ
+ */
+ spinlock_t lock;
+
+ /* protect resize cq
+ */
+ struct mutex resize_mutex;
+ struct xsc_ib_cq_resize *resize_buf;
+ struct ib_umem *resize_umem;
+ int cqe_size;
+ struct list_head send_qp_list;
+ struct list_head recv_qp_list;
+ struct list_head err_state_qp_list;
+ u32 create_flags;
+ int invalid;
+};
+
+enum xsc_srq_wr_per_qp {
+ XSC_SRQ_MIN_WR_NUM = 32,
+ XSC_SRQ_MAX_WR_NUM = 128,
+};
+
+enum {
+ XSC_SRQ_USER,
+ XSC_SRQ_KERNEL,
+};
+
+struct xsc_ib_srq {
+ struct ib_srq ibsrq;
+ struct xsc_core_srq xsrq;
+ u64 *wrid;
+ /* protect SRQ hanlding
+ */
+ spinlock_t lock;
+ int head;
+ int tail;
+ struct ib_umem *umem;
+ /* serialize arming a SRQ
+ */
+ struct mutex mutex;
+ int qp_num;
+ int create_type;
+};
+
+void xsc_restore_cache_srq_wrs(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp);
+void xsc_free_one_cache_srq_wr(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp, u64 wr_id);
+void xsc_free_all_cache_wrs(struct ib_recv_wr *wr);
+void xsc_post_cache_wrs(struct xsc_ib_srq *srq, struct xsc_ib_qp *qp, int wr_num, bool init);
+
+struct xsc_ib_mr {
+ struct ib_mr ibmr;
+ struct xsc_core_mr xmr;
+ struct ib_umem *umem;
+ int npages;
+};
+
+#define MAX_OPCODE IB_WR_MASKED_ATOMIC_FETCH_AND_ADD
+
+struct xsc_ib_dev {
+ struct ib_device ib_dev;
+ struct net_device *netdev;
+ struct xsc_core_device *xdev;
+ int num_ports;
+ int num_comp_vectors;
+ u8 ib_active;
+ u32 crc_32_table[256];
+ int iommu_state;
+ struct notifier_block nb;
+ u32 opcode_map[MAX_OPCODE + 1];
+};
+
+union xsc_ib_fw_ver {
+ u64 data;
+ struct {
+ u8 ver_major;
+ u8 ver_minor;
+ u16 ver_patch;
+ u32 ver_tweak;
+ } s;
+};
+
+struct xsc_err_state_qp_node {
+ struct list_head entry;
+ u32 qp_id;
+ bool is_sq;
+};
+
+struct xsc_ib_ah {
+ struct ib_ah ibah;
+ struct xsc_av av;
+};
+
+/*
+ * Conversion helpers.
+ *
+ * The first group goes from a uverbs object to the private structure that
+ * embeds it; the second goes from the core object to the private structure
+ * that embeds that one. Both directions exist for cq, qp and srq, so the
+ * names spell out which object is being converted.
+ */
+static inline struct xsc_ib_dev *to_xdev(struct ib_device *ibdev)
+{
+ return container_of(ibdev, struct xsc_ib_dev, ib_dev);
+}
+
+static inline struct xsc_ib_pd *to_xpd(struct ib_pd *ibpd)
+{
+ return container_of(ibpd, struct xsc_ib_pd, ibpd);
+}
+
+static inline struct xsc_ib_mr *to_xmr(struct ib_mr *ibmr)
+{
+ return container_of(ibmr, struct xsc_ib_mr, ibmr);
+}
+
+static inline struct xsc_ib_ah *to_xah(struct ib_ah *ibah)
+{
+ return container_of(ibah, struct xsc_ib_ah, ibah);
+}
+
+static inline struct xsc_ib_cq *to_xcq(struct ib_cq *ibcq)
+{
+ return container_of(ibcq, struct xsc_ib_cq, ibcq);
+}
+
+static inline struct xsc_ib_qp *to_xqp(struct ib_qp *ibqp)
+{
+ return container_of(ibqp, struct xsc_ib_qp, ibqp);
+}
+
+static inline struct xsc_ib_srq *to_xsrq(struct ib_srq *ibsrq)
+{
+ return container_of(ibsrq, struct xsc_ib_srq, ibsrq);
+}
+
+static inline struct xsc_ib_cq *core_to_xcq(struct xsc_core_cq *xcq)
+{
+ return container_of(xcq, struct xsc_ib_cq, xcq);
+}
+
+static inline struct xsc_ib_qp *core_to_xqp(struct xsc_core_qp *xqp)
+{
+ return container_of(xqp, struct xsc_ib_qp, xqp);
+}
+
+static inline struct xsc_ib_srq *core_to_xsrq(struct xsc_core_srq *xsrq)
+{
+ return container_of(xsrq, struct xsc_ib_srq, xsrq);
+}
+
+void build_opcode_map(struct xsc_ib_dev *dev);
+
+int xsc_ib_create_ah(struct ib_ah *ibah,
+ struct rdma_ah_init_attr *init_attr,
+ struct ib_udata *udata);
+int xsc_ib_destroy_ah(struct ib_ah *ibah, u32 flags);
+int xsc_ib_create_cq(struct ib_cq *ibcq,
+ const struct ib_cq_init_attr *attr,
+ struct uverbs_attr_bundle *attrs);
+int xsc_ib_destroy_cq(struct ib_cq *cq, struct ib_udata *udata);
+int xsc_ib_destroy_qp(struct ib_qp *qp, struct ib_udata *udata);
+int xsc_ib_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
+int xsc_ib_alloc_ucontext(struct ib_ucontext *uctx, struct ib_udata *udata);
+void xsc_ib_dealloc_ucontext(struct ib_ucontext *ibcontext);
+int xsc_ib_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata);
+int xsc_ib_dealloc_pd(struct ib_pd *pd, struct ib_udata *udata);
+
+int xsc_ib_create_srq(struct ib_srq *ib_srq,
+ struct ib_srq_init_attr *init_attr,
+ struct ib_udata *udata);
+int xsc_ib_modify_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr,
+ enum ib_srq_attr_mask attr_mask, struct ib_udata *udata);
+int xsc_ib_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *srq_attr);
+int xsc_ib_post_srq_recv(struct ib_srq *ibsrq, const struct ib_recv_wr *wr,
+ const struct ib_recv_wr **bad_wr);
+int xsc_ib_destroy_srq(struct ib_srq *srq, struct ib_udata *udata);
+
+int xsc_ib_query_port(struct ib_device *ibdev, u32 port,
+ struct ib_port_attr *props);
+
+int xsc_ib_create_qp(struct ib_qp *ibqp,
+ struct ib_qp_init_attr *init_attr,
+ struct ib_udata *udata);
+
+void __xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 qpn);
+void xsc_ib_cq_clean(struct xsc_ib_cq *cq, u32 qpn);
+
+int xsc_ib_query_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr);
+int xsc_ib_err_state_qp(struct xsc_ib_dev *dev, struct xsc_ib_qp *qp,
+ enum xsc_qp_state cur_state, enum xsc_qp_state state);
+int xsc_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
+ int attr_mask, struct ib_udata *udata);
+int xsc_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr, int qp_attr_mask,
+ struct ib_qp_init_attr *qp_init_attr);
+
+int xsc_ib_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr,
+ const struct ib_send_wr **bad_wr);
+int xsc_ib_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr,
+ const struct ib_recv_wr **bad_wr);
+
+void *xsc_get_send_wqe(struct xsc_ib_qp *qp, int n);
+int xsc_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc);
+int xsc_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags);
+struct ib_mr *xsc_ib_get_dma_mr(struct ib_pd *pd, int acc);
+struct ib_mr *xsc_ib_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
+ u64 virt_addr, int access_flags,
+ struct ib_dmah *dmah,
+ struct ib_udata *udata);
+int xsc_ib_get_buf_offset(u64 addr, int page_shift, u32 *offset);
+void xsc_ib_cont_pages(struct ib_umem *umem, u64 addr, int *count, int *shift,
+ int *ncont);
+void xsc_ib_populate_pas(struct xsc_ib_dev *dev, struct ib_umem *umem,
+ int page_shift, __be64 *pas, int npages, bool need_to_devide);
+struct ib_mr *xsc_ib_reg_user_mr_dmabuf(struct ib_pd *pd, u64 offset,
+ u64 length, u64 virt_addr,
+ int fd, int access_flags,
+ struct ib_dmah *dmah,
+ struct uverbs_attr_bundle *attrs);
+
+void xsc_ib_drain_rq(struct ib_qp *qp);
+void xsc_ib_drain_sq(struct ib_qp *qp);
+
+static inline u8 convert_access(int acc)
+{
+ return (acc & IB_ACCESS_REMOTE_ATOMIC ? XSC_PERM_ATOMIC : 0) |
+ (acc & IB_ACCESS_REMOTE_WRITE ? XSC_PERM_REMOTE_WRITE : 0) |
+ (acc & IB_ACCESS_REMOTE_READ ? XSC_PERM_REMOTE_READ : 0) |
+ (acc & IB_ACCESS_LOCAL_WRITE ? XSC_PERM_LOCAL_WRITE : 0) |
+ XSC_PERM_LOCAL_READ;
+}
+
+#define IB_MTU_8192 6
+
+static inline enum ib_mtu xsc_net_to_ib_mtu(unsigned int mtu)
+{
+ mtu = mtu - (IB_GRH_BYTES + IB_UDP_BYTES + IB_BTH_BYTES +
+ IB_EXT_XRC_BYTES + IB_EXT_ATOMICETH_BYTES +
+ IB_ICRC_BYTES);
+ if (mtu >= 8192)
+ return IB_MTU_8192;
+ else if (mtu >= ib_mtu_enum_to_int(IB_MTU_4096))
+ return IB_MTU_4096;
+ else if (mtu >= ib_mtu_enum_to_int(IB_MTU_1024))
+ return IB_MTU_1024;
+ else
+ return 0;
+}
+
+#define XSC_IB_IOMMU_MAP_DISABLE 0
+#define XSC_IB_IOMMU_MAP_UNKNOWN_DOMAIN 1
+#define XSC_IB_IOMMU_MAP_NORMAL 2
+
+static inline int xsc_ib_iommu_dma_map(struct ib_device *ibdev)
+{
+ return to_xdev(ibdev)->iommu_state;
+}
+
+static inline void *xsc_ib_iova_to_virt(struct ib_device *ibdev, dma_addr_t iova)
+{
+ phys_addr_t phyaddr;
+ struct iommu_domain *domain;
+
+ domain = iommu_get_domain_for_dev(ibdev->dma_device);
+ if (likely(domain)) {
+ phyaddr = iommu_iova_to_phys(domain, iova);
+ phyaddr |= iova & (PAGE_SIZE - 1);
+ } else {
+ phyaddr = dma_to_phys(ibdev->dma_device, iova);
+ }
+
+ return phys_to_virt(phyaddr);
+}
+
+struct ib_mad_list_head {
+ struct list_head list;
+ struct ib_cqe cqe;
+ struct ib_mad_queue *mad_queue;
+};
+
+#define IB_MAD_SEND_REQ_MAX_SG 2
+struct ib_mad_send_wr_private {
+ struct ib_mad_list_head mad_list;
+ struct list_head agent_list;
+ struct ib_mad_agent_private *mad_agent_priv;
+ struct ib_mad_send_buf send_buf;
+ u64 header_mapping;
+ u64 payload_mapping;
+ struct ib_ud_wr send_wr;
+ struct ib_sge sg_list[IB_MAD_SEND_REQ_MAX_SG];
+ __be64 tid;
+ unsigned long timeout;
+ int max_retries;
+ int retries_left;
+ int retry;
+ int refcount;
+ enum ib_wc_status status;
+
+ /* RMPP control */
+ struct list_head rmpp_list;
+ struct ib_rmpp_segment *last_ack_seg;
+ struct ib_rmpp_segment *cur_seg;
+ int last_ack;
+ int seg_num;
+ int newwin;
+ int pad;
+};
+
+struct ib_mad_private_header {
+ struct ib_mad_list_head mad_list;
+ struct ib_mad_recv_wc recv_wc;
+ struct ib_wc wc;
+ u64 mapping;
+} __packed;
+
+struct ib_mad_private {
+ struct ib_mad_private_header header;
+ size_t mad_size;
+ struct ib_grh grh;
+ DECLARE_FLEX_ARRAY(u8, mad);
+} __packed;
+
+static inline void *xsc_ib_send_mad_sg_virt_addr(struct ib_device *ibdev,
+ const struct ib_send_wr *wr,
+ int sg)
+{
+ struct ib_mad_send_wr_private *mad_send_wr;
+ struct ib_mad_list_head *mad_list;
+ int iommu_state = xsc_ib_iommu_dma_map(ibdev);
+
+ /* direct dma mapping */
+ if (!iommu_state)
+ return phys_to_virt(dma_to_phys(ibdev->dma_device, wr->sg_list[sg].addr));
+
+ if (iommu_state == XSC_IB_IOMMU_MAP_NORMAL)
+ return xsc_ib_iova_to_virt(ibdev, wr->sg_list[sg].addr);
+
+ mad_list = container_of(wr->wr_cqe, struct ib_mad_list_head, cqe);
+ mad_send_wr = container_of(mad_list, struct ib_mad_send_wr_private,
+ mad_list);
+
+ /* sg_list[0] */
+ if (sg == 0)
+ return mad_send_wr->send_buf.mad;
+
+ /* sg_list[1] */
+ if (mad_send_wr->send_buf.seg_count)
+ return ib_get_rmpp_segment(&mad_send_wr->send_buf,
+ mad_send_wr->seg_num);
+ return mad_send_wr->send_buf.mad + mad_send_wr->send_buf.hdr_len;
+}
+
+static inline void *xsc_ib_recv_mad_sg_virt_addr(struct ib_device *ibdev,
+ struct ib_wc *wc,
+ u64 sg_addr)
+{
+ struct ib_mad_private_header *mad_priv_hdr;
+ struct ib_mad_private *recv;
+ struct ib_mad_list_head *mad_list;
+ int iommu_state = xsc_ib_iommu_dma_map(ibdev);
+
+ /* direct dma mapping */
+ if (!iommu_state)
+ return phys_to_virt(dma_to_phys(ibdev->dma_device, sg_addr));
+
+ if (iommu_state == XSC_IB_IOMMU_MAP_NORMAL)
+ return xsc_ib_iova_to_virt(ibdev, sg_addr);
+
+ mad_list = container_of(wc->wr_cqe, struct ib_mad_list_head, cqe);
+ mad_priv_hdr = container_of(mad_list, struct ib_mad_private_header, mad_list);
+ recv = container_of(mad_priv_hdr, struct ib_mad_private, header);
+ return &recv->grh;
+}
+
+struct xsc_ib_burst_info {
+ __u32 max_burst_sz;
+ __u16 typical_pkt_sz;
+ __u16 reserved;
+};
+
+struct xsc_ib_modify_qp {
+ __u32 reserved;
+ struct xsc_ib_burst_info burst_info;
+ __u32 profile;
+};
+
+#endif /* XSC_IB_H */
diff --git a/drivers/infiniband/hw/xsc/xsc_ib_sysfs.c b/drivers/infiniband/hw/xsc/xsc_ib_sysfs.c
new file mode 100644
index 0000000000000..a0b577bdef258
--- /dev/null
+++ b/drivers/infiniband/hw/xsc/xsc_ib_sysfs.c
@@ -0,0 +1,64 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/sysfs.h>
+#include <linux/types.h>
+#include <linux/time.h>
+#include "common/xsc_core.h"
+#include "common/xsc_hsi.h"
+#include "common/driver.h"
+#include "common/xsc_cmd.h"
+#include "xsc_ib.h"
+
+static ssize_t hca_type_show(struct device *device, struct device_attribute *attr, char *buf)
+{
+ struct ib_device *ib_dev = container_of(device, struct ib_device, dev);
+ struct xsc_core_device *dev = to_xdev(ib_dev)->xdev;
+ struct pci_dev *pdev = dev->pdev;
+
+ return sprintf(buf, "%x\n", pdev->subsystem_device);
+}
+
+static DEVICE_ATTR_RO(hca_type);
+
+static ssize_t hw_rev_show(struct device *device, struct device_attribute *attr, char *buf)
+{
+ struct ib_device *ib_dev = container_of(device, struct ib_device, dev);
+ struct xsc_core_device *dev = to_xdev(ib_dev)->xdev;
+ u32 hw_ver = 0;
+
+ hw_ver = ((dev->chip_ver_l & 0xffff) << 16) |
+ (dev->hotfix_num & 0xffff);
+ return sprintf(buf, "0x%x\n", hw_ver);
+}
+
+static DEVICE_ATTR_RO(hw_rev);
+
+static struct device_attribute *xsc_ib_attributes[] = {
+ &dev_attr_hca_type,
+ &dev_attr_hw_rev,
+};
+
+void xsc_ib_sysfs_init(struct ib_device *ib_dev, struct xsc_core_device *xdev)
+{
+ int err = 0;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(xsc_ib_attributes); i++) {
+ err = device_create_file(&ib_dev->dev, xsc_ib_attributes[i]);
+ if (err)
+ dev_err(&xdev->pdev->dev, "Create sysfs file for %s failed.\n",
+ xsc_ib_attributes[i]->attr.name);
+ }
+}
+
+void xsc_ib_sysfs_fini(struct ib_device *ib_dev, struct xsc_core_device *xdev)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(xsc_ib_attributes); i++)
+ device_remove_file(&ib_dev->dev, xsc_ib_attributes[i]);
+}
diff --git a/drivers/net/ethernet/Kconfig b/drivers/net/ethernet/Kconfig
index 8581ccba1505e..3f357b7739a60 100644
--- a/drivers/net/ethernet/Kconfig
+++ b/drivers/net/ethernet/Kconfig
@@ -70,6 +70,7 @@ source "drivers/net/ethernet/i825xx/Kconfig"
source "drivers/net/ethernet/ibm/Kconfig"
source "drivers/net/ethernet/intel/Kconfig"
source "drivers/net/ethernet/xscale/Kconfig"
+source "drivers/net/ethernet/yunsilicon/Kconfig"
config JME
tristate "JMicron(R) PCI-Express Gigabit Ethernet support"
diff --git a/drivers/net/ethernet/Makefile b/drivers/net/ethernet/Makefile
index 2b1153d35b52b..4ad89b36b7612 100644
--- a/drivers/net/ethernet/Makefile
+++ b/drivers/net/ethernet/Makefile
@@ -50,6 +50,7 @@ obj-$(CONFIG_NET_VENDOR_INTEL) += intel/
obj-$(CONFIG_NET_VENDOR_I825XX) += i825xx/
obj-$(CONFIG_NET_VENDOR_MICROSOFT) += microsoft/
obj-$(CONFIG_NET_VENDOR_XSCALE) += xscale/
+obj-$(CONFIG_NET_VENDOR_YUNSILICON) += yunsilicon/
obj-$(CONFIG_JME) += jme.o
obj-$(CONFIG_KORINA) += korina.o
obj-$(CONFIG_LANTIQ_ETOP) += lantiq_etop.o
diff --git a/drivers/net/ethernet/yunsilicon/Kconfig b/drivers/net/ethernet/yunsilicon/Kconfig
new file mode 100644
index 0000000000000..46af1fdd01a25
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/Kconfig
@@ -0,0 +1,26 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+# Yunsilicon driver configuration
+#
+
+config NET_VENDOR_YUNSILICON
+ bool "Yunsilicon devices"
+ default y
+ depends on PCI
+ depends on 64BIT || COMPILE_TEST
+ help
+ If you have a network (Ethernet) device belonging to this class,
+ say Y.
+
+ Note that the answer to this question doesn't directly affect the
+ kernel: saying N will just cause the configurator to skip all
+ the questions about Yunsilicon cards. If you say Y, you will be asked
+ for your specific card in the following questions.
+
+if NET_VENDOR_YUNSILICON
+
+source "drivers/net/ethernet/yunsilicon/xsc/net/Kconfig"
+source "drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig"
+
+endif # NET_VENDOR_YUNSILICON
diff --git a/drivers/net/ethernet/yunsilicon/Makefile b/drivers/net/ethernet/yunsilicon/Makefile
new file mode 100644
index 0000000000000..a89a5ce24474e
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+# Makefile for the Yunsilicon device drivers.
+#
+
+obj-$(CONFIG_YUNSILICON_XSC_ETH) += xsc/net/
+obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc/pci/
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/cq.h b/drivers/net/ethernet/yunsilicon/xsc/common/cq.h
new file mode 100644
index 0000000000000..5217000c54a7c
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/cq.h
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_CORE_CQ_H
+#define XSC_CORE_CQ_H
+
+#include "common/xsc_core.h"
+
+enum {
+ XSC_CQE_OWNER_MASK = 1,
+};
+
+enum {
+ CQE_SIZE_64 = 0,
+ CQE_SIZE_128 = 1,
+};
+
+enum {
+ XSC_CQ_DB_REQ_NOT_SOL = 1,
+ XSC_CQ_DB_REQ_NOT = 0,
+};
+
+void xsc_init_cq_table(struct xsc_core_device *xdev);
+int xsc_core_create_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq,
+ struct xsc_create_cq_ex_mbox_in *in, int inlen);
+int xsc_core_destroy_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq);
+
+#endif /* XSC_CORE_CQ_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/driver.h b/drivers/net/ethernet/yunsilicon/xsc/common/driver.h
new file mode 100644
index 0000000000000..b3e91c868e49c
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/driver.h
@@ -0,0 +1,95 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_DRIVER_H
+#define XSC_DRIVER_H
+
+#include <linux/kernel.h>
+#include <linux/completion.h>
+#include <linux/pci.h>
+#include <linux/spinlock_types.h>
+#include <linux/semaphore.h>
+#include <linux/vmalloc.h>
+#include <linux/radix-tree.h>
+#include <rdma/ib_verbs.h>
+
+#include "common/xsc_device.h"
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+#include "common/xsc_cmd.h"
+#include "common/xsc_hsi.h"
+#include "common/srq.h"
+
+struct rxe_bth {
+ u8 opcode;
+ u8 flags;
+ __be16 pkey;
+ __be32 qpn;
+ __be32 apsn;
+};
+
+struct rxe_deth {
+ __be32 qkey;
+ __be32 src_qp;
+};
+
+static inline void *xsc_vzalloc(unsigned long size)
+{
+ void *rtn;
+
+ rtn = kzalloc(size, GFP_KERNEL | __GFP_NOWARN);
+ if (!rtn)
+ rtn = vzalloc(size);
+ return rtn;
+}
+
+static inline void xsc_vfree(const void *addr)
+{
+ if (addr && is_vmalloc_addr(addr))
+ vfree(addr);
+ else
+ kfree(addr);
+}
+
+int xsc_buf_alloc(struct xsc_core_device *xdev, unsigned long size,
+ unsigned long max_direct, struct xsc_buf *buf);
+void xsc_buf_free(struct xsc_core_device *xdev, struct xsc_buf *buf);
+void xsc_fill_page_array(struct xsc_buf *buf, __be64 *pas, unsigned int npages);
+
+int xsc_core_create_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr);
+int xsc_core_destroy_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr);
+int xsc_core_register_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr,
+ struct xsc_register_mr_mbox_in *in, int inlen);
+int xsc_core_dereg_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr);
+void xsc_reg_local_dma_mr(struct xsc_core_device *xdev);
+
+int xsc_core_alloc_pd(struct xsc_core_device *xdev, u32 *pdn);
+int xsc_core_dealloc_pd(struct xsc_core_device *xdev, u32 pdn);
+
+int xsc_core_create_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq);
+int xsc_core_destroy_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq);
+
+int xsc_core_create_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq,
+ struct xsc_create_cq_ex_mbox_in *in, int inlen);
+int xsc_core_destroy_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq);
+
+int xsc_core_create_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp,
+ struct xsc_create_qp_mbox_in *in,
+ int inlen);
+int xsc_core_destroy_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp);
+int xsc_core_qp_modify(struct xsc_core_device *xdev, enum xsc_qp_state cur_state,
+ enum xsc_qp_state new_state,
+ struct xsc_modify_qp_mbox_in *in, int sqd_event,
+ struct xsc_core_qp *qp);
+int xsc_core_qp_query(struct xsc_core_device *xdev, struct xsc_core_qp *qp,
+ struct xsc_query_qp_mbox_out *out, int outlen);
+void xsc_qp_event(struct xsc_core_device *xdev, u32 qpn, int event_type);
+
+int xsc_vector2eqn(struct xsc_core_device *xdev, int vector, int *eqn,
+ unsigned int *irqn);
+
+#endif /* XSC_DRIVER_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/qp.h b/drivers/net/ethernet/yunsilicon/xsc/common/qp.h
new file mode 100644
index 0000000000000..4855c97e43c83
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/qp.h
@@ -0,0 +1,104 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_QP_H
+#define XSC_QP_H
+
+#include <linux/radix-tree.h>
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd.h"
+#include "common/xsc_hsi.h"
+
+enum {
+ XSC_QP_PM_MIGRATED = 0x3,
+ XSC_QP_PM_ARMED = 0x0,
+ XSC_QP_PM_REARM = 0x1
+};
+
+enum {
+ XSC_WQE_CTRL_CQ_UPDATE = 2 << 2,
+ XSC_WQE_CTRL_SOLICITED = 1 << 1,
+};
+
+struct xsc_av {
+ union {
+ struct {
+ __be32 qkey;
+ __be32 reserved;
+ } qkey;
+ __be64 dc_key;
+ } key;
+ __be32 dqp_dct;
+ u8 stat_rate_sl;
+ u8 fl_mlid;
+ union {
+ __be16 rlid;
+ __be16 udp_sport;
+ };
+ u8 reserved0[4];
+ u8 rmac[6];
+ u8 tclass;
+ u8 hop_limit;
+ __be32 grh_gid_fl;
+ u8 rgid[16];
+};
+
+struct xsc_qp_path {
+ u8 fl;
+ u8 rsvd3;
+ u8 free_ar;
+ u8 pkey_index;
+ u8 rsvd0;
+ u8 grh_mlid;
+ __be16 rlid;
+ u8 ackto_lt;
+ u8 mgid_index;
+ u8 static_rate;
+ u8 hop_limit;
+ __be32 tclass_flowlabel;
+ u8 rgid[16];
+ u8 rsvd1[4];
+ u8 sl;
+ u8 port;
+ u8 rsvd2[6];
+ u8 dmac[6];
+ u8 smac[6];
+ __be16 af_type;
+ __be32 sip[4];
+ __be32 dip[4];
+ __be16 sport;
+ u8 ecn_dscp;
+ u8 vlan_valid;
+ __be16 vlan_id;
+ u8 dci_cfi_prio_sl; /* not left moved yet. */
+};
+
+static inline struct xsc_core_qp *__xsc_qp_lookup(struct xsc_core_device *xdev, u32 qpn)
+{
+ return radix_tree_lookup(&xdev->dev_res->qp_table.tree, qpn);
+}
+
+int xsc_core_create_resource_common(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp);
+void xsc_core_destroy_resource_common(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp);
+void xsc_init_qp_table(struct xsc_core_device *xdev);
+void xsc_qp_event(struct xsc_core_device *xdev, u32 qpn, int event_type);
+
+int xsc_core_create_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp,
+ struct xsc_create_qp_mbox_in *in,
+ int inlen);
+int xsc_core_destroy_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp);
+int xsc_core_qp_modify(struct xsc_core_device *xdev, enum xsc_qp_state cur_state,
+ enum xsc_qp_state new_state,
+ struct xsc_modify_qp_mbox_in *in, int sqd_event,
+ struct xsc_core_qp *qp);
+int xsc_core_qp_query(struct xsc_core_device *xdev, struct xsc_core_qp *qp,
+ struct xsc_query_qp_mbox_out *out, int outlen);
+
+#endif /* XSC_QP_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/srq.h b/drivers/net/ethernet/yunsilicon/xsc/common/srq.h
new file mode 100644
index 0000000000000..0a2e660e102a5
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/srq.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_SRQ_H
+#define XSC_SRQ_H
+
+#include <linux/list.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+
+#include "common/xsc_core.h"
+
+struct ib_qp;
+struct ib_recv_wr;
+
+#define XSC_MAX_SRQ_NUM 8192
+#define XSC_MAX_SRQ_LOG_SIZE 13
+
+struct xsc_srq_qp_node {
+ struct list_head entry;
+ u32 qp_id;
+ struct ib_qp *ibqp;
+};
+
+struct xsc_srq_cache_wr {
+ int wr_num;
+ int max_wr_num;
+ struct ib_recv_wr *head_wr;
+ struct ib_recv_wr *tail_wr;
+};
+
+struct xsc_core_srq {
+ u32 srqn;
+ int max;
+ size_t max_gs;
+ size_t max_avail_gather;
+ int wqe_shift;
+ void (*event)(struct xsc_core_srq *srq, enum xsc_event e);
+
+ /* protect ctrl list */
+ spinlock_t lock;
+ struct list_head ctrl_list;
+ u16 uid;
+
+ struct list_head srq_qp_list;
+ struct xsc_srq_cache_wr *cache_wr;
+};
+
+int xsc_core_create_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq);
+int xsc_core_destroy_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq);
+
+#endif /* XSC_SRQ_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd.h
new file mode 100644
index 0000000000000..0b8f669b5de40
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd.h
@@ -0,0 +1,1022 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_CMD_H
+#define __XSC_CMD_H
+
+#include <linux/if_ether.h>
+
+#define XSC_CMDQ_VERSION 0x32
+
+#define XSC_BOARD_SN_LEN 32
+#define DSCP_PCP_UNSET 255
+#define XSC_MAX_NUM_PCIE_INTF 2
+#define XSC_MAX_PF_NUM_PER_PCIE 8
+
+/* hca_cap mbox version */
+#define CMD_QUERY_HCA_CAP_V1 1
+#define CMD_QUERY_HCA_CAP_V2 2
+#define CMD_QUERY_HCA_CAP_V3 3
+#define CMD_QUERY_HCA_CAP_V4 4
+#define CMD_QUERY_HCA_CAP_V5 5
+#define CMD_QUERY_HCA_CAP_V6 6
+#define CMD_QUERY_HCA_CAP_V7 7
+#define CMD_QUERY_HCA_CAP_V8 8
+#define CMD_QUERY_HCA_CAP_V9 9
+
+#define CMD_SET_CQ_CONTEXT_V1 1
+
+enum {
+ XSC_CMDQ_GET_HW_STATS_RDMA_V0 = 0,
+ XSC_CMDQ_GET_HW_STATS_RDMA_V1 = 1,
+
+ XSC_CMDQ_GET_HW_STATS_RDMA_UNKNOW,
+};
+
+enum {
+ XSC_CMD_STAT_OK = 0x0,
+ XSC_CMD_STAT_INT_ERR = 0x1,
+ XSC_CMD_STAT_BAD_OP_ERR = 0x2,
+ XSC_CMD_STAT_BAD_PARAM_ERR = 0x3,
+ XSC_CMD_STAT_BAD_SYS_STATE_ERR = 0x4,
+ XSC_CMD_STAT_BAD_RES_ERR = 0x5,
+ XSC_CMD_STAT_RES_BUSY = 0x6,
+ XSC_CMD_STAT_LIM_ERR = 0x8,
+ XSC_CMD_STAT_BAD_RES_STATE_ERR = 0x9,
+ XSC_CMD_STAT_IX_ERR = 0xa,
+ XSC_CMD_STAT_NO_RES_ERR = 0xf,
+ XSC_CMD_STAT_BAD_QP_STATE_ERR = 0x10,
+ XSC_CMD_STAT_BAD_PKT_ERR = 0x30,
+ XSC_CMD_STAT_BAD_SIZE_OUTS_CQES_ERR = 0x40,
+ XSC_CMD_STAT_BAD_INP_LEN_ERR = 0x50,
+ XSC_CMD_STAT_BAD_OUTP_LEN_ERR = 0x51,
+};
+
+/* status aliases used by the RDMA driver (same protocol values) */
+#define XSC_CMD_STATUS_OK XSC_CMD_STAT_OK
+#define XSC_CMD_STATUS_FAIL XSC_CMD_STAT_INT_ERR
+#define XSC_CMD_STATUS_NOT_SUPPORTED XSC_CMD_STAT_BAD_OP_ERR
+#define XSC_CMD_STATUS_BAD_PARAM XSC_CMD_STAT_BAD_PARAM_ERR
+#define XSC_CMD_STATUS_INVAL_RES XSC_CMD_STAT_BAD_RES_ERR
+#define XSC_CMD_STATUS_BUSY XSC_CMD_STAT_RES_BUSY
+#define XSC_CMD_STATUS_INVAL_DATA XSC_CMD_STAT_LIM_ERR
+#define XSC_CMD_STATUS_NOT_FOUND XSC_CMD_STAT_IX_ERR
+#define XSC_CMD_STATUS_NO_RES XSC_CMD_STAT_NO_RES_ERR
+#define XSC_CMD_STATUS_BAD_INBUF XSC_CMD_STAT_BAD_INP_LEN_ERR
+#define XSC_CMD_STATUS_BAD_OUTBUF XSC_CMD_STAT_BAD_OUTP_LEN_ERR
+
+/* extended status codes */
+#define XSC_CMD_STATUS_INVAL_FUNC 0x41
+#define XSC_CMD_STATUS_NO_MPT_RES 0x42
+#define XSC_CMD_STATUS_NO_MTT_RES 0x43
+#define XSC_CMD_STATUS_NO_EQN_RES 0x44
+#define XSC_CMD_STATUS_NO_EQ_PA_RES 0x45
+#define XSC_CMD_STATUS_NO_CQN_RES 0x46
+#define XSC_CMD_STATUS_NO_CQ_PA_RES 0x47
+#define XSC_CMD_STATUS_NO_QPN_RES 0x48
+#define XSC_CMD_STATUS_NO_QP_PA_RES 0x49
+#define XSC_CMD_STATUS_NO_PDN_RES 0x4a
+#define XSC_CMD_STATUS_QP_FLUSH_BUSY 0x4b
+#define XSC_CMD_STATUS_QP_FLUSH_PENDING 0x4c
+#define XSC_CMD_STATUS_INVAL_OPCODE 0x52
+
+enum {
+ XSC_CMD_OP_QUERY_HCA_CAP = 0x100,
+ XSC_CMD_OP_QUERY_CMDQ_VERSION = 0x10a,
+ XSC_CMD_OP_FUNCTION_RESET = 0x10c,
+ XSC_CMD_OP_DUMMY = 0x10d,
+ XSC_CMD_OP_QUERY_GUID = 0x113,
+ XSC_CMD_OP_ACTIVATE_HW_CONFIG = 0x114,
+
+ XSC_CMD_OP_QUERY_HW_STATS = 0x1200,
+ XSC_CMD_OP_QUERY_HW_STATS_RDMA = 0x1203,
+ XSC_CMD_OP_QUERY_HW_GLOBAL_STATS = 0x1210,
+
+ XSC_CMD_OP_CREATE_MKEY = 0x200,
+ XSC_CMD_OP_DESTROY_MKEY = 0x202,
+ XSC_CMD_OP_REG_MR = 0x204,
+ XSC_CMD_OP_DEREG_MR = 0x205,
+ XSC_CMD_OP_SET_MPT = 0x206,
+ XSC_CMD_OP_SET_MTT = 0x207,
+
+ XSC_CMD_OP_CREATE_EQ = 0x301,
+ XSC_CMD_OP_DESTROY_EQ = 0x302,
+
+ XSC_CMD_OP_CREATE_CQ = 0x400,
+ XSC_CMD_OP_DESTROY_CQ = 0x401,
+ XSC_CMD_OP_SET_CQ_CONTEXT = 0x406,
+ XSC_CMD_OP_SET_CQ_BUF_PA = 0x407,
+ XSC_CMD_OP_CREATE_CQ_EX = 0x408,
+
+ XSC_CMD_OP_CREATE_QP = 0x500,
+ XSC_CMD_OP_DESTROY_QP = 0x501,
+ XSC_CMD_OP_RST2INIT_QP = 0x502,
+ XSC_CMD_OP_INIT2RTR_QP = 0x503,
+ XSC_CMD_OP_RTR2RTS_QP = 0x504,
+ XSC_CMD_OP_RTS2RTS_QP = 0x505,
+ XSC_CMD_OP_SQERR2RTS_QP = 0x506,
+ XSC_CMD_OP_2ERR_QP = 0x507,
+ XSC_CMD_OP_RTS2SQD_QP = 0x508,
+ XSC_CMD_OP_SQD2RTS_QP = 0x509,
+ XSC_CMD_OP_2RST_QP = 0x50a,
+ XSC_CMD_OP_QUERY_QP = 0x50b,
+ XSC_CMD_OP_INIT2INIT_QP = 0x50e,
+ XSC_CMD_OP_SQD2SQD_QP = 0x511,
+ XSC_CMD_OP_CREATE_MULTI_QP = 0x515,
+
+ XSC_CMD_OP_ALLOC_PD = 0x800,
+ XSC_CMD_OP_DEALLOC_PD = 0x801,
+
+ XSC_CMD_OP_MODIFY_RAW_QP = 0x81f,
+
+ XSC_CMD_OP_ENABLE_NIC_HCA = 0x810,
+ XSC_CMD_OP_DISABLE_NIC_HCA = 0x811,
+
+ XSC_CMD_OP_QUERY_VPORT_STATE = 0x822,
+ XSC_CMD_OP_QUERY_LINK_INFO = 0x832,
+ XSC_CMD_OP_MODIFY_LINK_INFO = 0x834,
+ XSC_CMD_OP_QUERY_EVENT_TYPE = 0x831,
+
+ XSC_CMD_OP_ENABLE_MSIX = 0x850,
+
+ XSC_CMD_OP_SET_MTU = 0x1100,
+ XSC_CMD_OP_QUERY_ETH_MAC = 0X1101,
+
+ XSC_CMD_OP_SET_PORT_ADMIN_STATUS = 0x1801,
+
+ XSC_CMD_OP_MAX
+};
+
+enum xsc_dma_direct {
+ XSC_DMA_DIR_TO_MAC,
+ XSC_DMA_DIR_READ,
+ XSC_DMA_DIR_WRITE,
+ XSC_DMA_DIR_LOOPBACK,
+ XSC_DMA_DIR_MAX
+};
+
+/* hw feature bitmap, 32bit */
+enum xsc_hw_feature_flag {
+ XSC_HW_RDMA_SUPPORT = BIT(0),
+ XSC_HW_PFC_PRIO_STATISTIC_SUPPORT = BIT(1),
+ XSC_HW_PFC_STALL_STATS_SUPPORT = BIT(3),
+ XSC_HW_RDMA_CM_SUPPORT = BIT(5),
+ XSC_HW_MTU_8192_SUPPORT = BIT(11),
+ XSC_HW_RDMA_ATOMIC_SUPPORT = BIT(17),
+ XSC_HW_MULTIDOORBELL_SUPPORT = BIT(18),
+ XSC_HW_CQE64_SUPPORT = BIT(21),
+
+ XSC_HW_LAST_FEATURE = BIT(31)
+};
+
+struct xsc_inbox_hdr {
+ __be16 opcode;
+ u8 rsvd[4];
+ __be16 ver; /* cmd version */
+};
+
+struct xsc_outbox_hdr {
+ u8 status;
+ u8 rsvd[5];
+ __be16 ver;
+};
+
+/*CQ mbox*/
+struct xsc_cq_context {
+ __be16 eqn; /* event queue number */
+ __be16 pa_num; /* physical address count in the ctx */
+ __be16 glb_func_id;
+ u8 log_cq_sz;
+ u8 cq_type;
+};
+
+struct xsc_cq_context_ex {
+ struct xsc_cq_context ctx;
+ u8 page_shift;
+ u8 overrun_ignore;
+ u8 rsvd[2];
+ u32 pa_num_ex;
+};
+
+struct xsc_create_cq_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_cq_context ctx;
+ __be64 pas[]; /* physical address list */
+};
+
+struct xsc_create_cq_ex_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_cq_context_ex ctx_ex;
+ __be64 pas[];
+};
+
+struct xsc_create_cq_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 cqn; /* completion queue number */
+ u8 rsvd[4];
+};
+
+struct xsc_destroy_cq_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 cqn;
+ u8 rsvd[4];
+};
+
+struct xsc_destroy_cq_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_set_cq_context_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_cq_context_ex ctx_ex;
+};
+
+struct xsc_set_cq_context_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 cqn;
+ __be32 cq_pa_list_base;
+};
+
+struct xsc_set_cq_buf_pa_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 pa_list_start;
+ __be32 pa_num;
+ __be64 pas[];
+};
+
+struct xsc_set_cq_buf_pa_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+/* MKey / MPT / MTT mbox */
+struct xsc_create_mkey_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 rsvd[4];
+};
+
+struct xsc_create_mkey_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 mkey;
+};
+
+struct xsc_destroy_mkey_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 mkey;
+};
+
+struct xsc_destroy_mkey_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd;
+};
+
+struct xsc_mpt_item {
+ __be32 pdn;
+ __be32 pa_num;
+ __be32 len;
+ __be32 mkey;
+ __be32 len_h;
+ u8 is_gpu;
+ u8 acc;
+ u8 page_mode;
+ u8 map_en;
+ __be64 va_base;
+};
+
+struct xsc_set_mpt_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_mpt_item mpt_item;
+};
+
+struct xsc_set_mpt_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 mtt_base;
+ u8 rsvd[4];
+};
+
+struct xsc_mtt_setting {
+ __be32 mtt_base;
+ __be32 pa_num;
+ __be64 pas[];
+};
+
+struct xsc_set_mtt_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_mtt_setting mtt_setting;
+};
+
+struct xsc_set_mtt_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+/*QP mbox*/
+enum {
+ MODULE_SPEED_UNKNOWN,
+ MODULE_SPEED_10G,
+ MODULE_SPEED_25G,
+ MODULE_SPEED_40G_R4,
+ MODULE_SPEED_50G_R,
+ MODULE_SPEED_50G_R2,
+ MODULE_SPEED_100G_R2,
+ MODULE_SPEED_100G_R4,
+ MODULE_SPEED_200G_R4,
+ MODULE_SPEED_200G_R8,
+ MODULE_SPEED_400G_R8,
+ MODULE_SPEED_400G_R4,
+};
+
+struct xsc_create_qp_request {
+ __be16 input_qpn;
+ __be16 pa_num;
+ u8 qp_type;
+ u8 log_sq_sz;
+ u8 log_rq_sz;
+ u8 dma_direct;
+ __be32 pdn; /* protect domain number */
+ __be16 cqn_send;
+ __be16 cqn_recv;
+ __be16 glb_funcid;
+ /*rsvd, the old logic_port */
+ u8 page_shift;
+ u8 rsvd;
+ __be64 pas[];
+};
+
+struct xsc_create_qp_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_create_qp_request req;
+};
+
+struct xsc_create_qp_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 qpn; /* queue pair number */
+ u8 rsvd[4];
+};
+
+struct xsc_destroy_qp_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 qpn;
+ u8 rsvd[4];
+};
+
+struct xsc_destroy_qp_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+/* MR mbox */
+struct xsc_register_mr_request {
+ __be32 pdn;
+ __be32 pa_num;
+ __be64 len;
+ __be32 mkey;
+ u8 is_gpu;
+ u8 acc;
+ u8 page_mode;
+ u8 map_en;
+ __be64 va_base;
+ __be64 pas[];
+};
+
+struct xsc_register_mr_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_register_mr_request req;
+};
+
+struct xsc_unregister_mr_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 mkey;
+ u8 rsvd[4];
+};
+
+struct xsc_unregister_mr_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+enum qp_access_flag {
+ QP_ACCESS_REMOTE_READ = (1 << 0),
+ QP_ACCESS_REMOTE_WRITE = (1 << 1),
+ QP_ACCESS_REMOTE_ATOMIC = (1 << 2),
+ QP_ACCESS_MODIFY_QP_TYPE = (1 << 3),
+};
+
+#define XSC_QP_CONTEXT_V1 1
+#define XSC_QP_CONTEXT_V2 2
+
+struct xsc_qp_context {
+ __be32 remote_qpn;
+ __be32 cqn_send;
+ __be32 cqn_recv;
+ __be32 next_send_psn;
+ __be32 next_recv_psn;
+ __be32 pdn;
+ __be16 src_udp_port;
+ __be16 path_id;
+ u8 mtu_mode;
+ u8 lag_sel;
+ u8 lag_sel_en;
+ u8 retry_cnt;
+ u8 rnr_retry;
+ u8 dscp;
+ u8 state;
+ u8 hop_limit;
+ u8 dmac[6];
+ u8 smac[6];
+ __be32 dip[4];
+ __be32 sip[4];
+ __be16 ip_type;
+ __be16 grp_id;
+ u8 vlan_valid;
+ u8 dci_cfi_prio_sl;
+ __be16 vlan_id;
+ u8 qp_out_port;
+ u8 pcie_no;
+ __be16 lag_id;
+ __be16 func_id;
+ u8 max_sra;
+ u8 max_rra;
+ u8 no_need_wait;
+ u8 swith_qp_type;
+ u8 rsvd0[2];
+ __be32 qp_access_flags;
+};
+
+struct xsc_modify_qp_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 qpn;
+ struct xsc_qp_context ctx;
+ u8 no_need_wait;
+};
+
+struct xsc_modify_qp_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_query_qp_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 qpn;
+ u8 rsvd[4];
+};
+
+struct xsc_query_qp_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ struct xsc_qp_context ctx;
+};
+
+struct xsc_create_multiqp_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be16 qp_num;
+ u8 qp_type;
+ u8 rsvd;
+ __be32 req_len;
+ u8 data[];
+};
+
+struct xsc_create_multiqp_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 qpn_base;
+};
+
+/* PD mbox */
+struct xsc_alloc_pd_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_alloc_pd_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 pdn;
+ u8 rsvd[4];
+};
+
+struct xsc_dealloc_pd_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 pdn;
+ u8 rsvd[4];
+};
+
+struct xsc_dealloc_pd_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+/* MSIX TABLE mbox */
+struct xsc_msix_table_info_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be16 index;
+ u8 rsvd[6];
+};
+
+struct xsc_msix_table_info_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 addr_lo;
+ __be32 addr_hi;
+ __be32 data;
+};
+
+/*EQ mbox*/
+struct xsc_eq_context {
+ __be16 vecidx;
+ __be16 pa_num;
+ u8 log_eq_sz;
+ __be16 glb_func_id;
+ u8 is_async_eq;
+ u8 rsvd;
+};
+
+struct xsc_create_eq_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_eq_context ctx;
+ __be64 pas[];
+};
+
+struct xsc_create_eq_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 eqn;
+ u8 rsvd[4];
+};
+
+struct xsc_destroy_eq_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 eqn;
+ u8 rsvd[4];
+};
+
+struct xsc_destroy_eq_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_cmd_query_cmdq_ver_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_cmd_query_cmdq_ver_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be16 cmdq_ver;
+ u8 rsvd[6];
+};
+
+struct xsc_fw_version {
+ u8 fw_version_major;
+ u8 fw_version_minor;
+ __be16 fw_version_patch;
+ __be32 fw_version_tweak;
+ u8 fw_version_extra_flag;
+ u8 rsvd[7];
+};
+
+struct xsc_hca_cap {
+ u8 rsvd1[12];
+ u8 send_seg_num;
+ u8 send_wqe_shift;
+ u8 recv_seg_num;
+ u8 recv_wqe_shift;
+ u8 log_max_srq_sz;
+ u8 log_max_qp_sz;
+ u8 log_max_mtt;
+ u8 log_max_qp;
+ u8 log_max_strq_sz;
+ u8 log_max_srqs;
+ u8 rsvd4[2];
+ u8 log_max_tso;
+ u8 log_max_cq_sz;
+ u8 rsvd6;
+ u8 log_max_cq;
+ u8 log_max_eq_sz;
+ u8 log_max_mkey;
+ u8 log_max_msix;
+ u8 log_max_eq;
+ u8 max_indirection;
+ u8 log_max_mrw_sz;
+ u8 log_max_bsf_list_sz;
+ u8 log_max_klm_list_sz;
+ u8 rsvd_8_0;
+ u8 log_max_ra_req_dc;
+ u8 rsvd_8_1;
+ u8 log_max_ra_res_dc;
+ u8 rsvd9;
+ u8 log_max_ra_req_qp;
+ u8 log_max_qp_depth;
+ u8 log_max_ra_res_qp;
+ __be16 max_vfs;
+ __be16 raweth_qp_id_end;
+ __be16 raw_tpe_qp_num;
+ __be16 max_qp_count;
+ __be16 raweth_qp_id_base;
+ u8 rsvd13;
+ u8 local_ca_ack_delay;
+ u8 max_num_eqs;
+ u8 num_ports;
+ u8 log_max_msg;
+ u8 mac_port;
+ __be16 raweth_rss_qp_id_base;
+ __be16 stat_rate_support;
+ u8 rsvd16[2];
+ __be64 flags;
+ u8 rsvd17;
+ u8 uar_sz;
+ u8 rsvd18;
+ u8 log_pg_sz;
+ __be16 bf_log_bf_reg_size;
+ __be16 msix_base;
+ __be16 msix_num;
+ __be16 max_desc_sz_sq;
+ u8 rsvd20[2];
+ __be16 max_desc_sz_rq;
+ u8 rsvd21[2];
+ __be16 max_desc_sz_sq_dc;
+ u8 rsvd22[4];
+ __be16 max_qp_mcg;
+ u8 rsvd23;
+ u8 log_max_mcg;
+ u8 rsvd24;
+ u8 log_max_pd;
+ u8 rsvd25;
+ u8 log_max_xrcd;
+ u8 rsvd26[40];
+ __be32 uar_page_sz;
+ u8 rsvd27[8];
+ __be32 hw_feature_flag; /* enum xsc_hw_feature_flag */
+ __be16 pf0_vf_funcid_base;
+ __be16 pf0_vf_funcid_top;
+ __be16 pf1_vf_funcid_base;
+ __be16 pf1_vf_funcid_top;
+ __be16 pcie0_pf_funcid_base;
+ __be16 pcie0_pf_funcid_top;
+ __be16 pcie1_pf_funcid_base;
+ __be16 pcie1_pf_funcid_top;
+ u8 log_msx_atomic_size_qp;
+ u8 pcie_host;
+ u8 rsvd28;
+ u8 log_msx_atomic_size_dc;
+ u8 board_sn[XSC_BOARD_SN_LEN];
+ u8 max_tc;
+ u8 mac_bit;
+ __be16 funcid_to_logic_port;
+ u8 rsvd29[6];
+ u8 nif_port_num;
+ u8 reg_mr_via_cmdq;
+ __be32 hca_core_clock;
+ __be32 max_rwq_indirection_tables; /* rss_caps */
+ __be32 max_rwq_indirection_table_size; /* rss_caps */
+ __be32 chip_ver_h;
+ __be32 chip_ver_m;
+ __be32 chip_ver_l;
+ __be32 hotfix_num;
+ __be32 feature_flag;
+ __be32 rx_pkt_len_max;
+ __be32 glb_func_id;
+ __be64 tx_db;
+ __be64 rx_db;
+ __be64 complete_db;
+ __be64 complete_reg;
+ __be64 event_db;
+ __be32 qp_rate_limit_min;
+ __be32 qp_rate_limit_max;
+ struct xsc_fw_version fw_ver;
+ u8 lag_logic_port_ofst;
+ /* V1 */
+ __be64 max_mr_size;
+ __be16 max_cmd_in_len;
+ __be16 max_cmd_out_len;
+ /* V2 */
+ __be32 max_qp;
+ __be32 max_cq;
+ __be32 max_pd;
+ __be32 max_mtt;
+ /* V3 */
+ __be32 mpt_tbl_addr;
+ __be32 mpt_tbl_depth;
+ __be32 mpt_tbl_width;
+ __be32 mtt_inst_base_addr;
+ __be32 mtt_inst_stride;
+ __be32 mtt_inst_num_log;
+ __be32 mtt_inst_depth;
+ /* V4 */
+ __be16 vf_funcid_base[XSC_MAX_NUM_PCIE_INTF][XSC_MAX_PF_NUM_PER_PCIE];
+ __be16 vf_funcid_top[XSC_MAX_NUM_PCIE_INTF][XSC_MAX_PF_NUM_PER_PCIE];
+ __be16 pf_funcid_base[XSC_MAX_NUM_PCIE_INTF];
+ __be16 pf_funcid_top[XSC_MAX_NUM_PCIE_INTF];
+ u8 pcie_no;
+ u8 pf_id;
+ __be16 vf_id;
+ u8 pcie_host_num;
+ u8 pf_num_per_pcie;
+ /* V6 */
+ u16 ft_support;
+ /* V7 */
+ __be16 total_dynamic_vf_msix;
+ __be16 min_dynamic_vf_msix;
+ __be16 max_dynamic_vf_msix;
+ /* V8 */
+ u8 max_sgl_rd;
+ u8 rsvd30;
+ /* V9 */
+ __be32 mdb_num;
+ u8 rsvd31[4];
+ __be64 mdb_base;
+};
+
+struct xsc_cmd_query_hca_cap_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be16 cpu_num;
+ u8 rsvd[6];
+};
+
+struct xsc_cmd_query_hca_cap_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd0[8];
+ struct xsc_hca_cap hca_cap;
+};
+
+struct xsc_query_vport_state_out {
+ struct xsc_outbox_hdr hdr;
+ u8 admin_state:4;
+ u8 state:4;
+};
+
+struct xsc_query_vport_state_in {
+ struct xsc_inbox_hdr hdr;
+ __be32 data0;
+#define XSC_QUERY_VPORT_OTHER_VPORT BIT(0)
+#define XSC_QUERY_VPORT_VPORT_NUM_MASK GENMASK(16, 1)
+};
+
+enum {
+ XSC_CMD_EVENT_RESP_CHANGE_LINK = BIT(0),
+ XSC_CMD_EVENT_RESP_TEMP_WARN = BIT(1),
+ XSC_CMD_EVENT_RESP_OVER_TEMP_PROTECTION = BIT(2),
+};
+
+struct xsc_event_resp {
+ u8 resp_cmd_type;
+};
+
+struct xsc_event_query_type_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 rsvd[2];
+};
+
+struct xsc_event_query_type_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ struct xsc_event_resp ctx;
+};
+
+struct xsc_modify_raw_qp_request {
+ __be16 qpn;
+ __be16 lag_id;
+ __be16 func_id;
+ u8 dma_direct;
+ u8 prio;
+ u8 qp_out_port;
+ u8 rsvd[7];
+};
+
+struct xsc_modify_raw_qp_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 pcie_no;
+ u8 rsvd[7];
+ struct xsc_modify_raw_qp_request req;
+};
+
+struct xsc_modify_raw_qp_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_set_mtu_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be16 mtu;
+ __be16 rx_buf_sz_min;
+ u8 mac_port;
+ u8 rsvd;
+};
+
+struct xsc_set_mtu_mbox_out {
+ struct xsc_outbox_hdr hdr;
+};
+
+struct xsc_query_eth_mac_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 index;
+};
+
+struct xsc_query_eth_mac_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 mac[ETH_ALEN];
+};
+
+enum {
+ XSC_TBM_CAP_HASH_PPH = 0,
+ XSC_TBM_CAP_RSS,
+ XSC_TBM_CAP_PP_BYPASS,
+ XSC_TBM_CAP_PCT_DROP_CONFIG,
+};
+
+struct xsc_nic_attr {
+ __be16 caps;
+ __be16 caps_mask;
+ u8 mac_addr[ETH_ALEN];
+};
+
+struct xsc_rss_attr {
+ u8 rss_en;
+ u8 hfunc;
+ __be16 rqn_base;
+ __be16 rqn_num;
+ __be32 hash_tmpl;
+};
+
+struct xsc_cmd_enable_nic_hca_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_nic_attr nic;
+ struct xsc_rss_attr rss;
+};
+
+struct xsc_cmd_enable_nic_hca_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd0[2];
+};
+
+struct xsc_nic_dis_attr {
+ __be16 caps;
+};
+
+struct xsc_cmd_disable_nic_hca_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ struct xsc_nic_dis_attr nic;
+};
+
+struct xsc_cmd_disable_nic_hca_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd0[4];
+};
+
+struct xsc_function_reset_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be16 glb_func_id;
+ u8 rsvd[6];
+};
+
+struct xsc_function_reset_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_cmd_query_guid_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_cmd_query_guid_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be64 guid;
+};
+
+struct xsc_cmd_activate_hw_config_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_cmd_activate_hw_config_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ u8 rsvd[8];
+};
+
+struct xsc_event_linkinfo {
+ u8 linkstatus; /*0:down, 1:up*/
+ u8 port;
+ u8 duplex;
+ u8 autoneg;
+ u32 linkspeed;
+ u64 supported;
+ u64 advertising;
+ u64 supported_fec; /* reserved, not support currently */
+ u64 advertised_fec; /* reserved, not support currently */
+ u64 supported_speed[2];
+ u64 advertising_speed[2];
+};
+
+struct xsc_event_query_linkinfo_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ struct xsc_event_linkinfo ctx;
+};
+
+struct xsc_event_set_port_admin_status_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ __be16 status;
+};
+
+struct xsc_event_set_port_admin_status_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ __be32 status;
+};
+
+/* HW stats mbox */
+struct xsc_hw_stats_rdma_pf {
+ /*by mac port*/
+ u64 rdma_tx_pkts;
+ u64 rdma_tx_bytes;
+ u64 rdma_rx_pkts;
+ u64 rdma_rx_bytes;
+ u64 np_cnp_sent;
+ u64 rp_cnp_handled;
+ u64 np_ecn_marked_roce_packets;
+ u64 rp_cnp_ignored;
+ u64 read_rsp_out_of_seq;
+ u64 implied_nak_seq_err;
+ /*by function*/
+ u64 out_of_sequence;
+ u64 packet_seq_err;
+ u64 out_of_buffer;
+ u64 rnr_nak_retry_err;
+ u64 local_ack_timeout_err;
+ u64 rx_read_requests;
+ u64 rx_write_requests;
+ u64 duplicate_requests;
+ u64 rdma_tx_pkts_func;
+ u64 rdma_tx_payload_bytes;
+ u64 rdma_rx_pkts_func;
+ u64 rdma_rx_payload_bytes;
+ /*global*/
+ u64 rdma_loopback_pkts;
+ u64 rdma_loopback_bytes;
+ /*for diamond*/
+ u64 out_of_sequence_sr;
+ u64 packet_seq_err_sr;
+ u64 rdma_ndp_rx_pkts;
+ u64 rdma_ndp_rx_trimmed_pkts;
+ u64 rdma_ndp_trimmed_pkts_sr;
+ u64 rx_icrc_encapsulated;
+};
+
+struct xsc_hw_stats_rdma_vf {
+ /*by function*/
+ u64 rdma_tx_pkts_func;
+ u64 rdma_tx_payload_bytes;
+ u64 rdma_rx_pkts_func;
+ u64 rdma_rx_payload_bytes;
+
+ u64 out_of_sequence;
+ u64 packet_seq_err;
+ u64 out_of_buffer;
+ u64 rnr_nak_retry_err;
+ u64 local_ack_timeout_err;
+ u64 rx_read_requests;
+ u64 rx_write_requests;
+ u64 duplicate_requests;
+};
+
+struct xsc_hw_stats_rdma {
+ u8 is_pf;
+ u8 rsv;
+ __be16 ver;
+ union {
+ struct xsc_hw_stats_rdma_pf pf_stats;
+ struct xsc_hw_stats_rdma_vf vf_stats;
+ } stats;
+};
+
+#define CMD_QUERY_HW_STATS_V1 1
+struct xsc_hw_stats_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 mac_port;
+ u8 is_lag;
+ u8 lag_member_num;
+ u8 member_port[];
+};
+
+struct xsc_hw_stats_rdma_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ struct xsc_hw_stats_rdma hw_stats;
+};
+
+struct xsc_hw_global_stats_rdma {
+ /*by global*/
+ u64 rdma_loopback_pkts;
+ u64 rdma_loopback_bytes;
+ u64 rx_icrc_encapsulated;
+ u64 req_cqe_error;
+ u64 resp_cqe_error;
+ u64 cqe_msg_code_error;
+};
+
+struct xsc_hw_global_stats_mbox_in {
+ struct xsc_inbox_hdr hdr;
+ u8 rsv[4];
+};
+
+struct xsc_hw_global_stats_mbox_out {
+ struct xsc_outbox_hdr hdr;
+ struct xsc_hw_global_stats_rdma hw_stats;
+};
+
+#endif /* __XSC_CMD_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd_api.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd_api.h
new file mode 100644
index 0000000000000..3ef6080ae7408
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmd_api.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_DRIVER_H
+#define __XSC_DRIVER_H
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd.h"
+
+int xsc_cmd_init(struct xsc_core_device *xdev);
+void xsc_cmd_cleanup(struct xsc_core_device *xdev);
+void xsc_cmd_use_events(struct xsc_core_device *xdev);
+void xsc_cmd_use_polling(struct xsc_core_device *xdev);
+int xsc_cmd_err_handler(struct xsc_core_device *xdev);
+void xsc_cmd_resp_handler(struct xsc_core_device *xdev);
+int xsc_cmd_exec(struct xsc_core_device *xdev,
+ void *in, unsigned int in_size,
+ void *out, unsigned int out_size);
+int xsc_cmd_version_check(struct xsc_core_device *xdev);
+
+int xsc_cmd_status_to_err(struct xsc_outbox_hdr *hdr);
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmdq.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmdq.h
new file mode 100644
index 0000000000000..0046111465401
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_cmdq.h
@@ -0,0 +1,192 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_CMDQ_H
+#define __XSC_CMDQ_H
+
+#include <linux/if_ether.h>
+
+#include "common/xsc_cmd.h"
+
+#define XSC_CMD_MAX_RETRY_CNT 3
+
+enum {
+ XSC_CMD_TIMEOUT_MSEC = 10 * 1000,
+ XSC_CMD_WQ_MAX_NAME = 32,
+};
+
+enum {
+ XSC_CMD_DATA, /* print command payload only */
+ XSC_CMD_TIME, /* print command execution time */
+};
+
+enum {
+ XSC_MAX_COMMANDS = 32,
+ XSC_CMD_DATA_BLOCK_SIZE = 512,
+ XSC_PCI_CMD_XPORT = 7,
+};
+
+struct xsc_cmd_prot_block {
+ u8 data[XSC_CMD_DATA_BLOCK_SIZE];
+ u8 rsvd0[48];
+ __be64 next;
+ __be32 block_num;
+ /* init to 0, dma user should change this val to 1 */
+ u8 owner_status;
+ u8 token;
+ u8 ctrl_sig;
+ u8 sig;
+};
+
+struct cache_ent {
+ /* protect block chain allocations */
+ spinlock_t lock;
+ struct list_head head;
+};
+
+struct cmd_msg_cache {
+ struct cache_ent large;
+ struct cache_ent med;
+
+};
+
+#define CMD_FIRST_SIZE 8
+struct xsc_cmd_first {
+ __be32 data[CMD_FIRST_SIZE];
+};
+
+struct xsc_cmd_mailbox {
+ void *buf;
+ dma_addr_t dma;
+ struct xsc_cmd_mailbox *next;
+};
+
+struct xsc_cmd_msg {
+ struct list_head list;
+ struct cache_ent *cache;
+ u32 len;
+ struct xsc_cmd_first first;
+ struct xsc_cmd_mailbox *next;
+};
+
+#define RSP_FIRST_SIZE 14
+struct xsc_rsp_first {
+ /* can be larger, xsc_rsp_layout */
+ __be32 data[RSP_FIRST_SIZE];
+};
+
+struct xsc_rsp_msg {
+ struct list_head list;
+ struct cache_ent *cache;
+ u32 len;
+ struct xsc_rsp_first first;
+ struct xsc_cmd_mailbox *next;
+};
+
+typedef void (*xsc_cmd_cbk_t)(int status, void *context);
+
+/* hw will use this for some records(e.g. vf_id) */
+struct cmdq_rsv {
+ u16 vf_id;
+ u8 rsv[2];
+};
+
+/* related with hw, won't change */
+
+struct xsc_cmd_layout {
+ struct cmdq_rsv rsv0;
+ __be32 inlen;
+ __be64 in_ptr;
+ __be32 in[CMD_FIRST_SIZE];
+ __be64 out_ptr;
+ __be32 outlen;
+ u8 token;
+ u8 sig;
+ u8 idx;
+ u8 type: 7;
+ /* rsv for hw, arm will check this bit to make sure mem written */
+ u8 owner_bit: 1;
+};
+
+struct xsc_rsp_layout {
+ struct cmdq_rsv rsv0;
+ __be32 out[RSP_FIRST_SIZE];
+ u8 token;
+ u8 sig;
+ u8 idx;
+ u8 type: 7;
+ /* rsv for hw, driver will check this bit to make sure mem written */
+ u8 owner_bit: 1;
+};
+
+struct xsc_cmd_work_ent {
+ struct xsc_cmd_msg *in;
+ struct xsc_rsp_msg *out;
+ int idx;
+ struct completion done;
+ struct xsc_cmd *cmd;
+ struct work_struct work;
+ struct xsc_cmd_layout *lay;
+ struct xsc_rsp_layout *rsp_lay;
+ int ret;
+ u8 status;
+ u8 token;
+};
+
+enum xsc_cmd_status {
+ XSC_CMD_STATUS_NORMAL,
+ XSC_CMD_STATUS_TIMEDOUT,
+};
+
+enum xsc_cmd_mode {
+ XSC_CMD_MODE_POLLING,
+ XSC_CMD_MODE_EVENTS,
+};
+
+struct xsc_cmd {
+ void *cmd_buf;
+ void *cq_buf;
+ dma_addr_t dma;
+ dma_addr_t cq_dma;
+ u16 cmd_pid;
+ u16 cq_cid;
+ u8 owner_bit;
+ u8 retry_cnt;
+ u8 cmdif_rev;
+ u8 log_sz;
+ u8 log_stride;
+ int max_reg_cmds;
+ int events;
+ u32 __iomem *vector;
+
+ /* protect command queue allocations */
+ spinlock_t alloc_lock;
+ /* protect token allocations */
+ spinlock_t token_lock;
+ /* protect cmdq req pid doorbell */
+ spinlock_t doorbell_lock;
+ u8 token;
+ unsigned long cmd_entry_mask;
+ char wq_name[XSC_CMD_WQ_MAX_NAME];
+ struct workqueue_struct *wq;
+ struct task_struct *cq_task;
+ /* The semaphore limits the number of working commands
+ * to max_reg_cmds. Each exec_cmd call does a down
+ * to acquire the semaphore before execution and an up
+ * after completion, ensuring no more than max_reg_cmds
+ * commands run at the same time.
+ */
+ struct semaphore sem;
+ enum xsc_cmd_mode mode;
+ struct xsc_cmd_work_ent *ent_arr[XSC_MAX_COMMANDS];
+ struct dma_pool *pool;
+ struct cmd_msg_cache cache;
+ int checksum_disabled;
+ unsigned int irqn;
+ u8 ownerbit_learned;
+ u8 cmd_status;
+};
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h
new file mode 100644
index 0000000000000..f491b9d8d8045
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_core.h
@@ -0,0 +1,707 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_CORE_H
+#define __XSC_CORE_H
+
+#include <linux/pci.h>
+#include <linux/auxiliary_bus.h>
+#include <linux/bitfield.h>
+#include <linux/if_vlan.h>
+#include <linux/reboot.h>
+
+#include "common/xsc_reg.h"
+#include "common/xsc_cmdq.h"
+
+#define XSC_PCI_VENDOR_ID 0x1f67
+
+#define XSC_MC_PF_DEV_ID 0x1011
+#define XSC_MC_VF_DEV_ID 0x1012
+
+#define XSC_MF_HOST_PF_DEV_ID 0x1051
+#define XSC_MF_HOST_VF_DEV_ID 0x1052
+#define XSC_MF_SOC_PF_DEV_ID 0x1053
+
+#define XSC_MS_PF_DEV_ID 0x1111
+#define XSC_MS_VF_DEV_ID 0x1112
+
+#define XSC_MV_HOST_PF_DEV_ID 0x1151
+#define XSC_MV_HOST_VF_DEV_ID 0x1152
+#define XSC_MV_SOC_PF_DEV_ID 0x1153
+
+#define PAGE_SHIFT_4K 12
+#define PAGE_SIZE_4K (_AC(1, UL) << PAGE_SHIFT_4K)
+
+#define XSC_ORIGIN_PF_BAR_SIZE (256 * 1024 * 1024)
+
+enum xsc_coredev_type {
+ XSC_COREDEV_PF,
+ XSC_COREDEV_VF,
+ XSC_COREDEV_SF
+};
+
+enum {
+ XSC_MAX_EQ_NAME = 20
+};
+
+enum {
+ XSC_MAX_IRQ_NAME = 32
+};
+
+enum {
+ XSC_MAX_FW_PORTS = 1,
+};
+
+enum {
+ XSC_BF_REGS_PER_PAGE = 4,
+ XSC_MAX_UAR_PAGES = 1 << 8,
+ XSC_MAX_UUARS = XSC_MAX_UAR_PAGES * XSC_BF_REGS_PER_PAGE,
+};
+
+/* alloc */
+struct xsc_buf_list {
+ void *buf;
+ dma_addr_t map;
+};
+
+struct xsc_buf {
+ struct xsc_buf_list direct;
+ struct xsc_buf_list *page_list;
+ unsigned long nbufs;
+ unsigned long npages;
+ unsigned long page_shift;
+ unsigned long size;
+};
+
+struct xsc_frag_buf {
+ struct xsc_buf_list *frags;
+ unsigned long npages;
+ unsigned long size;
+ u8 page_shift;
+};
+
+struct xsc_frag_buf_ctrl {
+ struct xsc_buf_list *frags;
+ u32 sz_m1;
+ u16 frag_sz_m1;
+ u16 strides_offset;
+ u8 log_sz;
+ u8 log_stride;
+ u8 log_frag_strides;
+};
+
+/* qp */
+struct xsc_send_wqe_ctrl_seg {
+ union {
+ __le32 data0;
+ struct {
+ __le32 msg_opcode:8;
+ __le32 with_immdt:1;
+ __le32 csum_en:2;
+ __le32 ds_data_num:5;
+ __le32 wqe_id:16;
+ };
+ };
+#define XSC_WQE_CTRL_SEG_MSG_OPCODE_MASK GENMASK(7, 0)
+#define XSC_WQE_CTRL_SEG_WITH_IMM BIT(8)
+#define XSC_WQE_CTRL_SEG_CSUM_EN_MASK GENMASK(10, 9)
+#define XSC_WQE_CTRL_SEG_DS_DATA_NUM_MASK GENMASK(15, 11)
+#define XSC_WQE_CTRL_SEG_WQE_ID_MASK GENMASK(31, 16)
+
+ __le32 msg_len;
+
+ union {
+ __le32 data2;
+ __le32 opcode_data;
+ struct {
+ u8 has_pph:1;
+ u8 so_type:1;
+ __le16 so_data_size:14;
+ u8:8;
+ u8 so_hdr_len:8;
+ };
+ };
+#define XSC_WQE_CTRL_SEG_HAS_PPH BIT(0)
+#define XSC_WQE_CTRL_SEG_SO_TYPE BIT(1)
+#define XSC_WQE_CTRL_SEG_SO_DATA_SIZE_MASK GENMASK(15, 2)
+#define XSC_WQE_CTRL_SEG_SO_HDR_LEN_MASK GENMASK(31, 24)
+
+ union {
+ __le32 data3;
+ struct {
+ __le32 se:1;
+ __le32 ce:1;
+ __le32:30;
+ };
+ };
+#define XSC_WQE_CTRL_SEG_SE BIT(0)
+#define XSC_WQE_CTRL_SEG_CE BIT(1)
+};
+
+struct xsc_wqe_data_seg {
+ union {
+ __le32 data0;
+ __le32 in_line:1;
+ struct {
+ __le32:1;
+ __le32 seg_len:31;
+ __le32 mkey;
+ __le64 va;
+ };
+ struct {
+ __le32:1;
+ __le32 len:7;
+ u8 in_line_data[15];
+ };
+ };
+#define XSC_WQE_DATA_SEG_SEG_LEN_MASK GENMASK(31, 1)
+};
+
+struct xsc_core_qp {
+ void (*event)(struct xsc_core_qp *qp, int type);
+ u32 qpn;
+ atomic_t refcount;
+ struct completion free;
+ int pid;
+ u16 qp_type;
+ u16 eth_queue_type;
+ u16 qp_type_internal;
+ u16 grp_id;
+ u8 mac_id;
+ u8 err_occurred;
+ u32 qp_protocol_mode;
+ u32 qp_access_flags;
+};
+
+struct xsc_qp_table {
+ spinlock_t lock; /* protect radix tree */
+ struct radix_tree_root tree;
+};
+
+/* cq */
+#define XSC_MAX_NAME_LEN 32
+#define XSC_MAX_IBDEV_NAME_LEN 64
+
+struct xsc_priv {
+ char name[XSC_MAX_NAME_LEN];
+ char ibdev_name[XSC_MAX_IBDEV_NAME_LEN];
+ int cm_pcp;
+ int cm_dscp;
+ int force_pcp;
+ int force_dscp;
+};
+
+enum xsc_dev_event {
+ XSC_DEV_EVENT_SYS_ERROR,
+ XSC_DEV_EVENT_PORT_UP,
+ XSC_DEV_EVENT_PORT_DOWN,
+ XSC_DEV_EVENT_PORT_INITIALIZED,
+ XSC_DEV_EVENT_LID_CHANGE,
+ XSC_DEV_EVENT_PKEY_CHANGE,
+ XSC_DEV_EVENT_GUID_CHANGE,
+ XSC_DEV_EVENT_CLIENT_REREG,
+};
+
+enum {
+ RDMA_PROTO_ROCEV2,
+ RDMA_PROTO_VEROCE,
+};
+
+#define VEROCE_UDP_DPORT 4794
+#define MAX_MR_CACHE_ENTRIES 5000
+
+enum xsc_event {
+ XSC_EVENT_TYPE_COMP = 0x0,
+ /* mad */
+ XSC_EVENT_TYPE_COMM_EST = 0x02,
+ XSC_EVENT_TYPE_CQ_ERROR = 0x04,
+ XSC_EVENT_TYPE_WQ_CATAS_ERROR = 0x05,
+ XSC_EVENT_TYPE_INTERNAL_ERROR = 0x08,
+ /* IBV_EVENT_QP_REQ_ERR */
+ XSC_EVENT_TYPE_WQ_INVAL_REQ_ERROR = 0x10,
+ /* IBV_EVENT_QP_ACCESS_ERR */
+ XSC_EVENT_TYPE_WQ_ACCESS_ERROR = 0x11,
+};
+
+struct xsc_cqe {
+ union {
+ __le32 data0;
+ struct {
+ u8 placeholder1;
+#define XSC_CQE_MSG_OPCD_MASK GENMASK(7, 0)
+#define XSC_CQE_ERR_CODE_MASK GENMASK(6, 0)
+#define XSC_CQE_IS_ERR BIT(7)
+#define XSC_CQE_QP_ID_MASK GENMASK(22, 8)
+#define XSC_CQE_SE BIT(24)
+#define XSC_CQE_HAS_PPH BIT(25)
+#define XSC_CQE_TYPE BIT(26)
+#define XSC_CQE_WITH_IMM BIT(27)
+#define XSC_CQE_CSUM_ERR_MASK GENMASK(31, 28)
+ __le32 qp_id:15;
+ u8 rsv1:1;
+ u8 se:1;
+ u8 has_pph:1;
+ u8 type:1;
+ u8 with_immdt:1;
+ u8 csum_err:4;
+ };
+ };
+
+ __le32 imm_data;
+ __le32 msg_len;
+ __le32 vni;
+
+ __le64 ts:48;
+
+ __le16 wqe_id;
+ u8 placeholder2;
+ u8 rsv3;
+ __le16 rsv[1];
+
+ union {
+ __le32 data1;
+ struct {
+ __le16 rsv0;
+ __le16 rsv2:15;
+#define XSC_CQE_OWNER BIT(31)
+ u8 owner:1;
+ };
+ };
+};
+
+struct xsc_cqe64 {
+ struct xsc_cqe cqe;
+ u8 padding[32];
+};
+
+struct xsc_core_cq {
+ u32 cqn;
+ u32 cqe_sz;
+ struct xsc_core_device *xdev;
+ struct xsc_core_device *dev;
+ atomic_t refcount;
+ struct completion free;
+ unsigned int vector;
+ unsigned int irqn;
+ void (*comp)(struct xsc_core_cq *cq);
+ void (*event)(struct xsc_core_cq *cq,
+ enum xsc_event);
+ u32 cons_index;
+ unsigned int arm_sn;
+ int pid;
+ u32 reg_next_cid;
+ u32 reg_done_pid;
+ struct xsc_eq *eq;
+};
+
+struct xsc_cq_table {
+ spinlock_t lock; /* protect radix tree */
+ struct radix_tree_root tree;
+};
+
+/* mr */
+struct xsc_core_mr {
+ u64 iova;
+ u64 size;
+ u32 key;
+ u32 pd;
+ u32 access;
+};
+
+/* eq */
+struct xsc_eq {
+ struct xsc_core_device *dev;
+ struct xsc_cq_table cq_table;
+ u32 cons_index;
+ struct xsc_buf buf;
+ unsigned int irqn;
+ u32 eqn;
+ u32 nent;
+ cpumask_var_t mask;
+ char name[XSC_MAX_EQ_NAME];
+ struct list_head list;
+ u16 index;
+};
+
+struct xsc_eq_table {
+ void __iomem *update_ci;
+ void __iomem *update_arm_ci;
+ struct list_head comp_eqs_list;
+ struct xsc_eq pages_eq;
+ struct xsc_eq async_eq;
+ struct xsc_eq cmd_eq;
+ int num_comp_vectors;
+ int eq_vec_comp_base;
+ /* protect EQs list */
+ spinlock_t lock;
+};
+
+/* irq */
+struct xsc_irq_info {
+ cpumask_var_t mask;
+ char name[XSC_MAX_IRQ_NAME];
+};
+
+/* adev */
+#define XSC_PCI_DRV_NAME "xsc_pci"
+#define XSC_ETH_ADEV_NAME "eth"
+#define XSC_IB_ADEV_NAME "ib"
+
+enum xsc_adev_idx {
+ XSC_ADEV_IDX_ETH,
+ XSC_ADEV_IDX_IB,
+ XSC_ADEV_IDX_MAX,
+};
+
+struct xsc_adev {
+ struct auxiliary_device adev;
+ struct xsc_core_device *xdev;
+
+ int idx;
+};
+
+struct xsc_board_info {
+ u32 board_id;
+ char board_sn[XSC_BOARD_SN_LEN];
+ __be64 guid;
+ u8 guid_valid;
+ u8 hw_config_activated;
+ struct xsc_msg_opcode *msg_opcode;
+};
+
+struct xsc_caps {
+ u8 log_max_eq;
+ u8 log_max_cq;
+ u8 log_max_qp;
+ u8 log_max_mkey;
+ u8 log_max_pd;
+ u8 log_max_srq;
+ u8 log_max_msix;
+ u32 max_cq;
+ u32 max_qp;
+ u32 max_pd;
+ u32 max_cqes;
+ u32 max_wqes;
+ u32 max_sq_desc_sz;
+ u32 max_rq_desc_sz;
+ u64 flags;
+ u16 stat_rate_support;
+ u32 log_max_msg;
+ u32 num_ports;
+ u32 max_ra_res_qp;
+ u32 max_ra_req_qp;
+ u32 max_srq_wqes;
+ u32 bf_reg_size;
+ u32 bf_regs_per_page;
+ u32 reserved_lkey;
+ u8 local_ca_ack_delay;
+ u8 log_max_mcg;
+ u16 max_qp_mcg;
+ u32 min_page_sz;
+ u32 send_ds_num;
+ u32 send_wqe_shift;
+ u32 recv_ds_num;
+ u32 recv_wqe_shift;
+ u32 rx_pkt_len_max;
+
+ u32 msix_enable:1;
+ u32 port_type:1;
+ u32 embedded_cpu:1;
+ u32 eswitch_manager:1;
+ u32 ecpf_vport_exists:1;
+ u32 vport_group_manager:1;
+ u32 sf:1;
+ u32 wqe_inline_mode:3;
+ u32 raweth_qp_id_base:15;
+ u32 rsvd0:7;
+
+ u16 max_vfs;
+ u8 log_max_qp_depth;
+ u8 log_max_current_uc_list;
+ u8 log_max_current_mc_list;
+ u16 log_max_vlan_list;
+ u8 fdb_multi_path_to_table;
+ u8 log_esw_max_sched_depth;
+
+ u8 max_num_sf_partitions;
+ u8 log_max_esw_sf;
+ u16 sf_base_id;
+
+ u32 max_tc:8;
+ u32 ets:1;
+ u32 dcbx:1;
+ u32 dscp:1;
+ u32 sbcam_reg:1;
+ u32 qos:1;
+ u32 port_buf:1;
+ u32 rsvd1:2;
+ u32 raw_tpe_qp_num:16;
+ u32 max_num_eqs:8;
+ u32 mac_port:8;
+ u32 raweth_rss_qp_id_base:16;
+ u16 msix_base;
+ u16 msix_num;
+ u8 log_max_mtt;
+ u8 log_max_tso;
+ u32 hca_core_clock;
+ u32 max_rwq_indirection_tables; /* rss_caps */
+ u32 max_rwq_indirection_table_size; /* rss_caps */
+ u16 raweth_qp_id_end;
+ u32 qp_rate_limit_min;
+ u32 qp_rate_limit_max;
+ u32 hw_feature_flag;
+ u16 pf0_vf_funcid_base;
+ u16 pf0_vf_funcid_top;
+ u16 pf1_vf_funcid_base;
+ u16 pf1_vf_funcid_top;
+ u16 pcie0_pf_funcid_base;
+ u16 pcie0_pf_funcid_top;
+ u16 pcie1_pf_funcid_base;
+ u16 pcie1_pf_funcid_top;
+ u8 nif_port_num;
+ u8 pcie_host;
+ u8 mac_bit;
+ u16 funcid_to_logic_port;
+ u8 lag_logic_port_ofst;
+
+ /* RDMA / MR resources (populated from hca_cap V1~V9) */
+ u64 max_mr_size;
+ u16 max_cmd_in_len;
+ u16 max_cmd_out_len;
+ u32 max_mtt;
+ u32 mpt_tbl_addr;
+ u32 mpt_tbl_depth;
+ u32 mpt_tbl_width;
+ u32 mtt_inst_base_addr;
+ u32 mtt_inst_stride;
+ u32 mtt_inst_num_log;
+ u32 mtt_inst_depth;
+ u32 mdb_num;
+ u64 mdb_base;
+};
+
+struct xsc_srq_cache_wr;
+
+/* xsc_core */
+struct xsc_dev_resource {
+ struct xsc_qp_table qp_table;
+ struct xsc_cq_table cq_table;
+ struct xsc_eq_table eq_table;
+ struct xsc_irq_info *irq_info;
+ /* protect buffer allocation according to numa node */
+ struct mutex alloc_mutex;
+ /* protect mkey key allocation */
+ spinlock_t mkey_lock;
+ u8 mkey_key;
+
+ /* srq number allocation, srqn is a software index only */
+ spinlock_t srq_lock;
+ unsigned long *srq_tbl;
+ struct xsc_srq_cache_wr *srq_cache_wr;
+ u32 max_srq_num;
+};
+
+struct xsc_core_device {
+ struct pci_dev *pdev;
+ struct device *device;
+ int adev_id;
+ struct xsc_adev **xsc_adev_list;
+ void *eth_priv;
+ struct xsc_dev_resource *dev_res;
+ int numa_node;
+
+ /* async event plumbing, owned by the PCI layer */
+ struct work_struct event_work;
+ struct workqueue_struct *event_wq;
+ void (*link_event_handler)(void *xdev);
+
+ void __iomem *bar;
+ int bar_num;
+ enum xsc_coredev_type coredev_type;
+ struct xsc_hw_reg *hal_regs;
+
+ u8 mac_port;
+ u8 pcie_no;
+ u8 pf_id;
+ u16 glb_func_id;
+ u16 gsi_qpn;
+ atomic_t num_qps;
+
+ u16 msix_vec_base;
+
+ void *netdev;
+
+ u32 rdma_proto_mode;
+ u16 cm_udp_dst_port;
+ u16 veroce_udp_dst_port;
+ u8 read_flush;
+ u8 bql_thresh;
+
+ struct xsc_priv priv;
+
+ struct xsc_cmd cmd;
+ u16 cmdq_ver;
+
+ struct xsc_caps caps;
+ struct xsc_board_info *board_info;
+
+ u32 chip_ver_h;
+ u32 chip_ver_m;
+ u32 chip_ver_l;
+ u32 hotfix_num;
+ u32 feature_flag;
+
+ u8 fw_version_major;
+ u8 fw_version_minor;
+ u16 fw_version_patch;
+ u32 fw_version_tweak;
+ u8 fw_version_extra_flag;
+ cpumask_var_t xps_cpumask;
+};
+
+int xsc_core_create_resource_common(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp);
+void xsc_core_destroy_resource_common(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp);
+int xsc_core_eth_create_qp(struct xsc_core_device *xdev,
+ struct xsc_create_qp_mbox_in *in,
+ int insize, u32 *p_qpn);
+int xsc_core_eth_modify_qp_status(struct xsc_core_device *xdev,
+ u32 qpn, u16 status);
+int xsc_core_eth_destroy_qp(struct xsc_core_device *xdev, u32 qpn);
+int xsc_core_eth_create_rss_qp_rqs(struct xsc_core_device *xdev,
+ struct xsc_create_multiqp_mbox_in *in,
+ int insize,
+ u32 *p_qpn_base);
+int xsc_core_eth_modify_raw_qp(struct xsc_core_device *xdev,
+ struct xsc_modify_raw_qp_mbox_in *in);
+int xsc_core_eth_create_cq(struct xsc_core_device *xdev,
+ struct xsc_core_cq *xcq,
+ struct xsc_create_cq_mbox_in *in,
+ int insize);
+int xsc_core_eth_destroy_cq(struct xsc_core_device *xdev,
+ struct xsc_core_cq *xcq);
+
+struct xsc_eq *xsc_core_eq_get(struct xsc_core_device *xdev, int i);
+int xsc_core_vector2eqn(struct xsc_core_device *xdev, int vector, u32 *eqn,
+ unsigned int *irqn);
+
+/*
+ * Link event callback registration. The events are dispatched from the PCI
+ * layer's own workqueue; unregister detaches the callback and waits for one
+ * already in flight, so it must precede freeing what the handler touches.
+ */
+void xsc_irq_event_register(struct xsc_core_device *xdev,
+ void (*handler)(void *xdev));
+void xsc_irq_event_unregister(struct xsc_core_device *xdev);
+void xsc_core_fill_page_frag_array(struct xsc_frag_buf *buf,
+ __be64 *pas, unsigned int npages);
+int xsc_core_frag_buf_alloc_node(struct xsc_core_device *xdev,
+ unsigned long size,
+ struct xsc_frag_buf *buf,
+ int node);
+void xsc_core_frag_buf_free(struct xsc_core_device *xdev,
+ struct xsc_frag_buf *buf);
+
+u8 xsc_core_query_vport_state(struct xsc_core_device *xdev, u16 vport);
+
+static inline void *xsc_buf_offset(struct xsc_buf *buf, unsigned long offset)
+{
+ if (likely(buf->nbufs == 1))
+ return buf->direct.buf + offset;
+ else
+ return buf->page_list[offset >> PAGE_SHIFT].buf +
+ (offset & (PAGE_SIZE - 1));
+}
+
+static inline bool xsc_fw_is_available(struct xsc_core_device *xdev)
+{
+ return xdev->cmd.cmd_status == XSC_CMD_STATUS_NORMAL;
+}
+
+static inline u8 get_cqe_opcode(struct xsc_cqe *cqe)
+{
+ return FIELD_GET(XSC_CQE_MSG_OPCD_MASK, le32_to_cpu(cqe->data0));
+}
+
+static inline bool xsc_core_is_pf(const struct xsc_core_device *dev)
+{
+ return dev->coredev_type == XSC_COREDEV_PF;
+}
+
+static inline bool is_support_hw_pf_stats(struct xsc_core_device *dev)
+{
+ return xsc_core_is_pf(dev);
+}
+
+static inline bool is_support_cqe64(struct xsc_core_device *dev)
+{
+ return dev->caps.hw_feature_flag & XSC_HW_CQE64_SUPPORT;
+}
+
+static inline bool is_support_rdma_cm(struct xsc_core_device *dev)
+{
+ return dev->caps.hw_feature_flag & XSC_HW_RDMA_CM_SUPPORT;
+}
+
+static inline bool is_support_rdma_atomic(struct xsc_core_device *dev)
+{
+ return dev->caps.hw_feature_flag & XSC_HW_RDMA_ATOMIC_SUPPORT;
+}
+
+static inline bool is_support_rdma(struct xsc_core_device *dev)
+{
+ return dev->caps.hw_feature_flag & XSC_HW_RDMA_SUPPORT;
+}
+
+static inline bool is_support_8k_mtu(struct xsc_core_device *dev)
+{
+ return dev->caps.hw_feature_flag & XSC_HW_MTU_8192_SUPPORT;
+}
+
+static inline bool is_pf_bar_compressed(struct xsc_core_device *dev)
+{
+ return pci_resource_len(dev->pdev, 0) != XSC_ORIGIN_PF_BAR_SIZE;
+}
+
+static inline bool is_support_multi_doorbell(struct xsc_core_device *dev)
+{
+ return dev->caps.hw_feature_flag & XSC_HW_MULTIDOORBELL_SUPPORT;
+}
+
+void xsc_remove_rdma_driver(void);
+int xsc_counters_init(struct ib_device *ib_dev, struct xsc_core_device *xdev);
+void xsc_counters_fini(struct ib_device *ib_dev, struct xsc_core_device *xdev);
+void xsc_ib_sysfs_init(struct ib_device *ib_dev, struct xsc_core_device *xdev);
+void xsc_ib_sysfs_fini(struct ib_device *ib_dev, struct xsc_core_device *xdev);
+
+void xsc_init_hal(struct xsc_core_device *xdev, u32 device_id);
+
+u8 xsc_get_mr_page_mode(struct xsc_core_device *xdev, u8 page_shift);
+u64 xsc_get_max_mr_size(struct xsc_core_device *xdev);
+void xsc_set_data_seg(struct xsc_core_device *xdev, void *data_seg, u64 addr,
+ u32 key, u32 length);
+u64 xsc_get_rdma_stat_mask(struct xsc_core_device *xdev, u16 ver);
+u32 xsc_get_max_qp_depth(struct xsc_core_device *xdev);
+bool xsc_check_max_qp_depth(struct xsc_core_device *xdev, u32 *wqe_cnt,
+ u32 max_qp_depth);
+void xsc_set_wqe_id(struct xsc_core_device *xdev, void *cseg, u32 wqe_id);
+int xsc_get_cqe_wqe_id(struct xsc_core_device *xdev, void *cqe);
+void xsc_get_multidb_info(struct xsc_core_device *xdev, u32 *num, u32 *base);
+u8 xsc_get_cqe_opcode(struct xsc_core_device *xdev, void *cqe);
+u32 xsc_get_recv_ds_num(struct xsc_core_device *xdev);
+int xsc_get_link_speed(struct xsc_core_device *xdev);
+void xsc_set_mtt_info(struct xsc_core_device *xdev);
+void xsc_set_pf_db_addr(struct xsc_core_device *xdev,
+ u64 tx_db, u64 rx_db, u64 cq_db, u64 cq_reg, u64 eq_db);
+void xsc_get_db_addr(struct xsc_core_device *xdev,
+ u64 *tx_db, u64 *rx_db, u64 *cq_db, u64 *cq_reg, u64 *eq_db);
+
+void xsc_update_tx_db(struct xsc_core_device *xdev, u32 sqn, u32 next_pid);
+void xsc_update_rx_db(struct xsc_core_device *xdev, u32 rqn, u32 next_pid);
+void xsc_arm_cq(struct xsc_core_device *xdev, u32 cqn, u32 next_cid, u8 solicited);
+void xsc_update_cq_ci(struct xsc_core_device *xdev, u32 cqn, u32 next_cid);
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_device.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_device.h
new file mode 100644
index 0000000000000..3025348390dd8
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_device.h
@@ -0,0 +1,94 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_DEVICE_H
+#define __XSC_DEVICE_H
+
+#include <linux/types.h>
+
+/* QP type */
+enum {
+ XSC_QUEUE_TYPE_RDMA_RC = 0,
+ XSC_QUEUE_TYPE_RDMA_MAD = 1,
+ XSC_QUEUE_TYPE_RAW = 2,
+ XSC_QUEUE_TYPE_VIRTIO_NET = 3,
+ XSC_QUEUE_TYPE_VIRTIO_BLK = 4,
+ XSC_QUEUE_TYPE_RAW_TPE = 5,
+ XSC_QUEUE_TYPE_RAW_TSO = 6,
+ XSC_QUEUE_TYPE_RAW_TX = 7,
+ XSC_QUEUE_TYPE_SNIFFER = 8,
+ XSC_QUEUE_TYPE_WORC = 9,
+ XSC_QUEUE_TYPE_INVALID = 0xFF,
+};
+
+enum {
+ XSC_PERM_LOCAL_READ = 1 << 0,
+ XSC_PERM_LOCAL_WRITE = 1 << 1,
+ XSC_PERM_REMOTE_READ = 1 << 2,
+ XSC_PERM_REMOTE_WRITE = 1 << 3,
+ XSC_PERM_ATOMIC = 1 << 4,
+ XSC_PERM_UMR_EN = 1 << 7,
+};
+
+enum {
+ XSC_DEV_CAP_FLAG_RC = 1LL << 0,
+ XSC_DEV_CAP_FLAG_UC = 1LL << 1,
+ XSC_DEV_CAP_FLAG_UD = 1LL << 2,
+ XSC_DEV_CAP_FLAG_XRC = 1LL << 3,
+ XSC_DEV_CAP_FLAG_SRQ = 1LL << 6,
+ XSC_DEV_CAP_FLAG_BAD_PKEY_CNTR = 1LL << 8,
+ XSC_DEV_CAP_FLAG_BAD_QKEY_CNTR = 1LL << 9,
+ XSC_DEV_CAP_FLAG_APM = 1LL << 17,
+ XSC_DEV_CAP_FLAG_ATOMIC = 1LL << 18,
+ XSC_DEV_CAP_FLAG_ON_DMND_PG = 1LL << 24,
+ XSC_DEV_CAP_FLAG_RESIZE_SRQ = 1LL << 32,
+ XSC_DEV_CAP_FLAG_REMOTE_FENCE = 1LL << 38,
+ XSC_DEV_CAP_FLAG_TLP_HINTS = 1LL << 39,
+ XSC_DEV_CAP_FLAG_SIG_HAND_OVER = 1LL << 40,
+ XSC_DEV_CAP_FLAG_DCT = 1LL << 41,
+ XSC_DEV_CAP_FLAG_CMDIF_CSUM = 1LL << 46,
+};
+
+enum xsc_traffic_types {
+ XSC_TT_IPV4,
+ XSC_TT_IPV4_TCP,
+ XSC_TT_IPV4_UDP,
+ XSC_TT_IPV6,
+ XSC_TT_IPV6_TCP,
+ XSC_TT_IPV6_UDP,
+ XSC_TT_IPV4_IPSEC_AH,
+ XSC_TT_IPV6_IPSEC_AH,
+ XSC_TT_IPV4_IPSEC_ESP,
+ XSC_TT_IPV6_IPSEC_ESP,
+ XSC_TT_ANY,
+ XSC_NUM_TT,
+};
+
+#define XSC_NUM_INDIR_TIRS XSC_NUM_TT
+
+enum {
+ XSC_L3_PROT_TYPE_IPV4 = BIT(0),
+ XSC_L3_PROT_TYPE_IPV6 = BIT(1),
+};
+
+enum {
+ XSC_L4_PROT_TYPE_TCP = BIT(0),
+ XSC_L4_PROT_TYPE_UDP = BIT(1),
+};
+
+struct xsc_tirc_config {
+ u8 l3_prot_type;
+ u8 l4_prot_type;
+ u32 rx_hash_fields;
+};
+
+enum {
+ XSC_HASH_FUNC_XOR = 0,
+ XSC_HASH_FUNC_TOP = 1,
+ XSC_HASH_FUNC_TOP_SYM = 2,
+ XSC_HASH_FUNC_RSV = 3,
+};
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_hsi.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_hsi.h
new file mode 100644
index 0000000000000..aa2049f89bc9b
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_hsi.h
@@ -0,0 +1,123 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_HSI_H
+#define XSC_HSI_H
+
+#include <asm/byteorder.h>
+
+#include <linux/types.h>
+#include <linux/bitops.h>
+
+#define WR_LE_32(x, val) (x = __cpu_to_le32(val))
+#define WR_LE_64(x, val) (x = __cpu_to_le64(val))
+
+#define WR_BE_32(x, val) (x = __cpu_to_be32(val))
+
+#define RD_LE_32(x) __le32_to_cpu(x)
+#define RD_BE_32(x) __be32_to_cpu(x)
+
+struct xsc_msg_opcode {
+ u32 send;
+ u32 send_with_imm;
+ u32 rdma_write;
+ u32 rdma_write_with_imm;
+ u32 rdma_read;
+ u32 atomic_cmp_and_swp;
+ u32 atomic_fetch_and_add;
+ u32 masked_atomic_cmp_and_swp;
+ u32 masked_atomic_fetch_and_add;
+ u32 mad;
+};
+
+/* sw cqe opcode */
+enum {
+ XSC_OPCODE_RDMA_REQ_SEND = 0,
+ XSC_OPCODE_RDMA_REQ_SEND_IMMDT = 1,
+ XSC_OPCODE_RDMA_RSP_RECV = 2,
+ XSC_OPCODE_RDMA_RSP_RECV_IMMDT = 3,
+ XSC_OPCODE_RDMA_REQ_WRITE = 4,
+ XSC_OPCODE_RDMA_REQ_WRITE_IMMDT = 5,
+ XSC_OPCODE_RDMA_RSP_WRITE_IMMDT = 6,
+ XSC_OPCODE_RDMA_REQ_READ = 7,
+ XSC_OPCODE_RDMA_REQ_ERROR = 8,
+ XSC_OPCODE_RDMA_RSP_ERROR = 9,
+ XSC_OPCODE_RDMA_CQE_ERROR = 10,
+ XSC_OPCODE_RDMA_MAD_REQ_SEND = 11,
+ XSC_OPCODE_RDMA_MAD_RSP_RECV = 12,
+ XSC_OPCODE_RDMA_CQE_RAW_SNF = 13,
+};
+
+enum {
+ XSC_SEND_SEG_MAX = 32,
+ XSC_BASE_WQE_SHIFT = 4,
+ XSC_SEND_SEG_NUM = 4,
+ XSC_SEND_WQE_SHIFT = 6,
+ XSC_CTRL_SEG_NUM = 1,
+ XSC_RADDR_SEG_NUM = 1,
+};
+
+enum {
+ XSC_RECV_SEG_MAX = 4,
+ XSC_RECV_SEG_NUM = 1,
+ XSC_RECV_WQE_SHIFT = 4,
+};
+
+enum {
+ XSC_INLINE_SIZE_MAX = 15,
+};
+
+#define XSC_MPT_MAP_EN 0
+
+enum {
+ XSC_REQ = 0,
+ XSC_RSP = 1,
+};
+
+enum {
+ XSC_WITHOUT_IMMDT = 0,
+ XSC_WITH_IMMDT = 1,
+};
+
+/* CQE error codes */
+enum {
+ XSC_ERR_CODE_NAK_RETRY = 0x40,
+ XSC_ERR_CODE_NAK_OPCODE = 0x41,
+ XSC_ERR_CODE_NAK_MR = 0x42,
+ XSC_ERR_CODE_NAK_OPERATION = 0x43,
+ XSC_ERR_CODE_NAK_RNR = 0x44,
+ XSC_ERR_CODE_LOCAL_MR = 0x45,
+ XSC_ERR_CODE_LOCAL_LEN = 0x46,
+ XSC_ERR_CODE_LOCAL_OPCODE = 0x47,
+ XSC_ERR_CODE_CQ_OVER_FLOW = 0x48,
+ XSC_ERR_CODE_STRG_ACC_GEN_CQE = 0x4c,
+ XSC_ERR_CODE_CQE_ACC = 0x4d,
+ XSC_ERR_CODE_FLUSH = 0x4e,
+ XSC_ERR_CODE_MALF_WQE_HOST = 0x50,
+ XSC_ERR_CODE_MALF_WQE_INFO = 0x51,
+ XSC_ERR_CODE_MR_NON_NAK = 0x52,
+ XSC_ERR_CODE_OPCODE_GEN_CQE = 0x61,
+ XSC_ERR_CODE_MANY_READ = 0x62,
+ XSC_ERR_CODE_LEN_GEN_CQE = 0x63,
+ XSC_ERR_CODE_MR = 0x65,
+ XSC_ERR_CODE_MR_GEN_CQE = 0x66,
+ XSC_ERR_CODE_OPERATION = 0x67,
+ XSC_ERR_CODE_MALF_WQE_INFO_GEN_NAK = 0x68,
+};
+
+enum xsc_qp_state {
+ XSC_QP_STATE_RST = 0,
+ XSC_QP_STATE_INIT = 1,
+ XSC_QP_STATE_RTR = 2,
+ XSC_QP_STATE_RTS = 3,
+ XSC_QP_STATE_SQER = 4,
+ XSC_QP_STATE_SQD = 5,
+ XSC_QP_STATE_ERR = 6,
+ XSC_QP_STATE_SQ_DRAINING = 7,
+ XSC_QP_STATE_SUSPENDED = 9,
+ XSC_QP_NUM_STATE
+};
+
+#endif /* XSC_HSI_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/common/xsc_reg.h b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_reg.h
new file mode 100644
index 0000000000000..babdb58024f5a
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/common/xsc_reg.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_REG_H
+#define XSC_REG_H
+#define CMDQM_HOST_REQ_BUF_BASE_H_ADDR_MEM_ADDR 0x0
+#define CMDQM_HOST_REQ_BUF_BASE_L_ADDR_MEM_ADDR 0x4
+#define CMDQM_HOST_REQ_PID_MEM_ADDR 0x8
+#define CMDQM_HOST_REQ_CID_MEM_ADDR 0xc
+#define CMDQM_HOST_RSP_BUF_BASE_H_ADDR_MEM_ADDR 0x10
+#define CMDQM_HOST_RSP_BUF_BASE_L_ADDR_MEM_ADDR 0x14
+#define CMDQM_HOST_RSP_PID_MEM_ADDR 0x18
+#define CMDQM_HOST_RSP_CID_MEM_ADDR 0x1c
+#define CMDQM_HOST_VF_ERR_STS_MEM_ADDR 0x20
+#define CMDQM_VECTOR_ID_MEM_ADDR 0x24
+
+#define DB_CQ_FUNC_MEM_ADDR 0x8c4
+#define DB_EQ_FUNC_MEM_ADDR 0x8c8
+#define DB_CQ_CID_DIRECT_MEM_ADDR 0x8cc
+#define TX_DB_FUNC_MEM_ADDR 0x8d0
+#define RX_DB_FUNC_MEM_ADDR 0x8d4
+
+#define MDB_NUM 64
+#define QPM_TX_MDB_BASE_REG_ADDR 0xd00
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/Kconfig b/drivers/net/ethernet/yunsilicon/xsc/net/Kconfig
new file mode 100644
index 0000000000000..7428b26295b66
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/Kconfig
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+# Yunsilicon driver configuration
+#
+
+config YUNSILICON_XSC_ETH
+ tristate "Yunsilicon XSC ethernet driver"
+ depends on YUNSILICON_XSC_PCI
+ depends on NET
+ select PAGE_POOL
+ help
+ This driver provides ethernet support for
+ Yunsilicon XSC devices.
+
+ To compile this driver as a module, choose M here. The module
+ will be called xsc_eth.
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/Makefile b/drivers/net/ethernet/yunsilicon/xsc/net/Makefile
new file mode 100644
index 0000000000000..43d049e42e40d
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/Makefile
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+
+ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
+
+obj-$(CONFIG_YUNSILICON_XSC_ETH) += xsc_eth.o
+
+xsc_eth-y := main.o xsc_eth_wq.o xsc_eth_txrx.o xsc_eth_tx.o xsc_eth_rx.o xsc_eth_stats.o
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/main.c b/drivers/net/ethernet/yunsilicon/xsc/net/main.c
new file mode 100644
index 0000000000000..d1f72043fd798
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/main.c
@@ -0,0 +1,1724 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/auxiliary_bus.h>
+#include <linux/irq.h>
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+#include "common/xsc_device.h"
+#include "xsc_pp.h"
+#include "xsc_eth_common.h"
+#include "xsc_eth.h"
+#include "xsc_eth_txrx.h"
+
+static const struct xsc_tirc_config tirc_default_config[XSC_NUM_INDIR_TIRS] = {
+ [XSC_TT_IPV4] = {
+ .l3_prot_type = XSC_L3_PROT_TYPE_IPV4,
+ .l4_prot_type = 0,
+ .rx_hash_fields = XSC_HASH_IP,
+ },
+ [XSC_TT_IPV4_TCP] = {
+ .l3_prot_type = XSC_L3_PROT_TYPE_IPV4,
+ .l4_prot_type = XSC_L4_PROT_TYPE_TCP,
+ .rx_hash_fields = XSC_HASH_IP_PORTS,
+ },
+ [XSC_TT_IPV4_UDP] = {
+ .l3_prot_type = XSC_L3_PROT_TYPE_IPV4,
+ .l4_prot_type = XSC_L4_PROT_TYPE_UDP,
+ .rx_hash_fields = XSC_HASH_IP_PORTS,
+ },
+ [XSC_TT_IPV6] = {
+ .l3_prot_type = XSC_L3_PROT_TYPE_IPV6,
+ .l4_prot_type = 0,
+ .rx_hash_fields = XSC_HASH_IP6,
+ },
+ [XSC_TT_IPV6_TCP] = {
+ .l3_prot_type = XSC_L3_PROT_TYPE_IPV6,
+ .l4_prot_type = XSC_L4_PROT_TYPE_TCP,
+ .rx_hash_fields = XSC_HASH_IP6_PORTS,
+ },
+ [XSC_TT_IPV6_UDP] = {
+ .l3_prot_type = XSC_L3_PROT_TYPE_IPV6,
+ .l4_prot_type = XSC_L4_PROT_TYPE_UDP,
+ .rx_hash_fields = XSC_HASH_IP6_PORTS,
+ },
+};
+
+static int xsc_get_max_num_channels(struct xsc_core_device *xdev)
+{
+ return min_t(int, xdev->dev_res->eq_table.num_comp_vectors,
+ XSC_ETH_MAX_NUM_CHANNELS);
+}
+
+static void xsc_build_rss_param(struct xsc_rss_params *rss_param)
+{
+ enum xsc_traffic_types tt;
+
+ for (tt = 0; tt < XSC_NUM_INDIR_TIRS; tt++) {
+ rss_param->rx_hash_fields[tt] =
+ tirc_default_config[tt].rx_hash_fields;
+ }
+ rss_param->rss_hash_tmpl = XSC_HASH_IP_PORTS | XSC_HASH_IP6_PORTS;
+}
+
+static void xsc_eth_build_nic_params(struct xsc_adapter *adapter, u32 ch_num)
+{
+ struct xsc_eth_params *params = &adapter->nic_param;
+ struct xsc_core_device *xdev = adapter->xdev;
+
+ params->mtu = SW_DEFAULT_MTU;
+
+ params->comp_vectors = xdev->dev_res->eq_table.num_comp_vectors;
+ params->max_num_ch = ch_num;
+ params->num_channels = ch_num;
+
+ xsc_build_rss_param(&adapter->rss_param);
+}
+
+static int xsc_eth_netdev_init(struct xsc_adapter *adapter)
+{
+ unsigned int node, nch;
+
+ nch = adapter->nic_param.max_num_ch;
+ node = dev_to_node(adapter->dev);
+ adapter->txq2sq = kcalloc_node(nch,
+ sizeof(*adapter->txq2sq),
+ GFP_KERNEL, node);
+ if (!adapter->txq2sq)
+ return -ENOMEM;
+
+ netif_carrier_off(adapter->netdev);
+
+ return 0;
+}
+
+static void xsc_eth_build_queue_param(struct xsc_adapter *adapter,
+ struct xsc_queue_attr *attr, u8 type)
+{
+ struct xsc_core_device *xdev = adapter->xdev;
+
+ if (adapter->nic_param.sq_size == 0)
+ adapter->nic_param.sq_size = BIT(xdev->caps.log_max_qp_depth);
+ if (adapter->nic_param.rq_size == 0)
+ adapter->nic_param.rq_size = BIT(xdev->caps.log_max_qp_depth);
+
+ if (type == XSC_QUEUE_TYPE_RQCQ) {
+ attr->ele_num = min_t(int, XSC_RQCQ_ELE_NUM,
+ xdev->caps.max_cqes);
+ attr->ele_size = XSC_RQCQ_ELE_SZ;
+ attr->ele_log_size = order_base_2(XSC_RQCQ_ELE_SZ);
+ attr->q_log_size = order_base_2(attr->ele_num);
+ } else if (type == XSC_QUEUE_TYPE_SQCQ) {
+ attr->ele_num = min_t(int, XSC_SQCQ_ELE_NUM,
+ xdev->caps.max_cqes);
+ attr->ele_size = XSC_SQCQ_ELE_SZ;
+ attr->ele_log_size = order_base_2(XSC_SQCQ_ELE_SZ);
+ attr->q_log_size = order_base_2(attr->ele_num);
+ } else if (type == XSC_QUEUE_TYPE_RQ) {
+ attr->ele_num = adapter->nic_param.rq_size;
+ attr->ele_size = xdev->caps.recv_ds_num * XSC_RECV_WQE_DS;
+ attr->ele_log_size = order_base_2(attr->ele_size);
+ attr->q_log_size = order_base_2(attr->ele_num);
+ } else if (type == XSC_QUEUE_TYPE_SQ) {
+ attr->ele_num = adapter->nic_param.sq_size;
+ attr->ele_size = xdev->caps.send_ds_num * XSC_SEND_WQE_DS;
+ attr->ele_log_size = order_base_2(attr->ele_size);
+ attr->q_log_size = order_base_2(attr->ele_num);
+ }
+}
+
+static u32 xsc_rx_get_linear_frag_sz(u32 mtu)
+{
+ u32 byte_count = XSC_SW2HW_FRAG_SIZE(mtu);
+
+ return XSC_SKB_FRAG_SZ(byte_count);
+}
+
+static bool xsc_rx_is_linear_skb(u32 mtu)
+{
+ u32 linear_frag_sz = xsc_rx_get_linear_frag_sz(mtu);
+
+ return linear_frag_sz <= PAGE_SIZE;
+}
+
+static u32 xsc_get_rq_frag_info(struct xsc_rq_frags_info *frags_info, u32 mtu)
+{
+ u32 byte_count = XSC_SW2HW_FRAG_SIZE(mtu);
+ int frag_stride;
+ int i = 0;
+
+ if (xsc_rx_is_linear_skb(mtu)) {
+ frag_stride = xsc_rx_get_linear_frag_sz(mtu);
+ frag_stride = roundup_pow_of_two(frag_stride);
+
+ frags_info->arr[0].frag_size = byte_count;
+ frags_info->arr[0].frag_stride = frag_stride;
+ frags_info->num_frags = 1;
+ frags_info->wqe_bulk = PAGE_SIZE / frag_stride;
+ frags_info->wqe_bulk_min = frags_info->wqe_bulk;
+ goto out;
+ }
+
+ if (byte_count <= DEFAULT_FRAG_SIZE) {
+ frags_info->arr[0].frag_size = DEFAULT_FRAG_SIZE;
+ frags_info->arr[0].frag_stride = DEFAULT_FRAG_SIZE;
+ frags_info->num_frags = 1;
+ } else if (byte_count <= PAGE_SIZE_4K) {
+ frags_info->arr[0].frag_size = PAGE_SIZE_4K;
+ frags_info->arr[0].frag_stride = PAGE_SIZE_4K;
+ frags_info->num_frags = 1;
+ } else if (byte_count <= (PAGE_SIZE_4K + DEFAULT_FRAG_SIZE)) {
+ if (PAGE_SIZE < 2 * PAGE_SIZE_4K) {
+ frags_info->arr[0].frag_size = PAGE_SIZE_4K;
+ frags_info->arr[0].frag_stride = PAGE_SIZE_4K;
+ frags_info->arr[1].frag_size = PAGE_SIZE_4K;
+ frags_info->arr[1].frag_stride = PAGE_SIZE_4K;
+ frags_info->num_frags = 2;
+ } else {
+ frags_info->arr[0].frag_size = 2 * PAGE_SIZE_4K;
+ frags_info->arr[0].frag_stride = 2 * PAGE_SIZE_4K;
+ frags_info->num_frags = 1;
+ }
+ } else if (byte_count <= 2 * PAGE_SIZE_4K) {
+ if (PAGE_SIZE < 2 * PAGE_SIZE_4K) {
+ frags_info->arr[0].frag_size = PAGE_SIZE_4K;
+ frags_info->arr[0].frag_stride = PAGE_SIZE_4K;
+ frags_info->arr[1].frag_size = PAGE_SIZE_4K;
+ frags_info->arr[1].frag_stride = PAGE_SIZE_4K;
+ frags_info->num_frags = 2;
+ } else {
+ frags_info->arr[0].frag_size = 2 * PAGE_SIZE_4K;
+ frags_info->arr[0].frag_stride = 2 * PAGE_SIZE_4K;
+ frags_info->num_frags = 1;
+ }
+ } else {
+ if (PAGE_SIZE < 4 * PAGE_SIZE_4K) {
+ frags_info->num_frags =
+ roundup(byte_count,
+ PAGE_SIZE_4K) / PAGE_SIZE_4K;
+ for (i = 0; i < frags_info->num_frags; i++) {
+ frags_info->arr[i].frag_size = PAGE_SIZE_4K;
+ frags_info->arr[i].frag_stride = PAGE_SIZE_4K;
+ }
+ } else {
+ frags_info->arr[0].frag_size = 4 * PAGE_SIZE_4K;
+ frags_info->arr[0].frag_stride = 4 * PAGE_SIZE_4K;
+ frags_info->num_frags = 1;
+ }
+ }
+
+ if (PAGE_SIZE <= PAGE_SIZE_4K) {
+ frags_info->wqe_bulk_min = 4;
+ frags_info->wqe_bulk = max_t(u8, frags_info->wqe_bulk_min, 8);
+ } else if (PAGE_SIZE <= 2 * PAGE_SIZE_4K) {
+ frags_info->wqe_bulk = 2;
+ frags_info->wqe_bulk_min = frags_info->wqe_bulk;
+ } else {
+ frags_info->wqe_bulk =
+ PAGE_SIZE /
+ (frags_info->num_frags * frags_info->arr[0].frag_size);
+ frags_info->wqe_bulk_min = frags_info->wqe_bulk;
+ }
+
+out:
+ frags_info->log_num_frags = order_base_2(frags_info->num_frags);
+
+ return frags_info->num_frags * frags_info->arr[0].frag_size;
+}
+
+static void xsc_build_rq_frags_info(struct xsc_queue_attr *attr,
+ struct xsc_rq_frags_info *frags_info,
+ struct xsc_eth_params *params)
+{
+ params->rq_frags_size = xsc_get_rq_frag_info(frags_info, params->mtu);
+ frags_info->frags_max_num = attr->ele_size / XSC_RECV_WQE_DS;
+}
+
+static void xsc_eth_build_channel_param(struct xsc_adapter *adapter,
+ struct xsc_channel_param *chl_param)
+{
+ xsc_eth_build_queue_param(adapter, &chl_param->rqcq_param.cq_attr,
+ XSC_QUEUE_TYPE_RQCQ);
+ chl_param->rqcq_param.wq.buf_numa_node = dev_to_node(adapter->dev);
+
+ xsc_eth_build_queue_param(adapter, &chl_param->sqcq_param.cq_attr,
+ XSC_QUEUE_TYPE_SQCQ);
+ chl_param->sqcq_param.wq.buf_numa_node = dev_to_node(adapter->dev);
+
+ xsc_eth_build_queue_param(adapter, &chl_param->sq_param.sq_attr,
+ XSC_QUEUE_TYPE_SQ);
+ chl_param->sq_param.wq.buf_numa_node = dev_to_node(adapter->dev);
+
+ xsc_eth_build_queue_param(adapter, &chl_param->rq_param.rq_attr,
+ XSC_QUEUE_TYPE_RQ);
+ chl_param->rq_param.wq.buf_numa_node = dev_to_node(adapter->dev);
+
+ xsc_build_rq_frags_info(&chl_param->rq_param.rq_attr,
+ &chl_param->rq_param.frags_info,
+ &adapter->nic_param);
+}
+
+static void xsc_eth_cq_error_event(struct xsc_core_cq *xcq,
+ enum xsc_event event)
+{
+ struct xsc_cq *xsc_cq = container_of(xcq, struct xsc_cq, xcq);
+ struct xsc_core_device *xdev = xsc_cq->xdev;
+
+ if (event != XSC_EVENT_TYPE_CQ_ERROR) {
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "Unexpected event type %d on CQ %06x\n",
+ event, xcq->cqn);
+ return;
+ }
+
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "Eth catch CQ ERROR: %x, cqn: %d\n", event, xcq->cqn);
+}
+
+static void xsc_eth_completion_event(struct xsc_core_cq *xcq)
+{
+ struct xsc_cq *cq = container_of(xcq, struct xsc_cq, xcq);
+ struct xsc_core_device *xdev = cq->xdev;
+ struct xsc_rq *rq = NULL;
+
+ if (unlikely(!cq->channel)) {
+ netdev_err(xsc_dev_to_netdev(xdev), "cq%d->channel is null\n",
+ xcq->cqn);
+ return;
+ }
+
+ rq = &cq->channel->qp.rq;
+
+ if (!test_bit(XSC_ETH_RQ_STATE_ENABLED, &rq->state))
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "ch%d_cq%d, napi_flag=0x%lx\n",
+ cq->channel->chl_idx, xcq->cqn,
+ cq->napi->state);
+
+ napi_schedule(cq->napi);
+ cq->event_ctr++;
+}
+
+static int xsc_eth_alloc_cq(struct xsc_channel *c, struct xsc_cq *pcq,
+ struct xsc_cq_param *pcq_param)
+{
+ u8 ele_log_size = pcq_param->cq_attr.ele_log_size;
+ struct xsc_core_device *xdev = c->adapter->xdev;
+ u8 q_log_size = pcq_param->cq_attr.q_log_size;
+ struct xsc_core_cq *core_cq = &pcq->xcq;
+ int ret;
+ u32 i;
+
+ pcq_param->wq.db_numa_node = cpu_to_node(c->cpu);
+ pcq_param->wq.buf_numa_node = cpu_to_node(c->cpu);
+
+ ret = xsc_eth_cqwq_create(xdev, &pcq_param->wq,
+ q_log_size, ele_log_size, &pcq->wq,
+ &pcq->wq_ctrl);
+ if (ret)
+ return ret;
+
+ core_cq->cqe_sz = pcq_param->cq_attr.ele_num;
+ core_cq->comp = xsc_eth_completion_event;
+ core_cq->event = xsc_eth_cq_error_event;
+ core_cq->vector = c->chl_idx;
+
+ for (i = 0; i < xsc_cqwq_get_size(&pcq->wq); i++) {
+ struct xsc_cqe *cqe = xsc_cqwq_get_wqe(&pcq->wq, i);
+
+ cqe->data1 |= cpu_to_le32(XSC_CQE_OWNER);
+ }
+ pcq->xdev = xdev;
+
+ return ret;
+}
+
+static int xsc_eth_set_cq(struct xsc_channel *c,
+ struct xsc_cq *pcq,
+ struct xsc_cq_param *pcq_param)
+{
+ struct xsc_core_device *xdev = c->adapter->xdev;
+ struct xsc_create_cq_mbox_in *in;
+ unsigned int hw_npages;
+ unsigned int irqn;
+ int inlen;
+ u32 eqn;
+ int ret;
+
+ hw_npages = DIV_ROUND_UP(pcq->wq_ctrl.buf.size, PAGE_SIZE_4K);
+ /*mbox size + pas size*/
+ inlen = sizeof(struct xsc_create_cq_mbox_in) +
+ sizeof(__be64) * hw_npages;
+
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in)
+ return -ENOMEM;
+
+ /*construct param of in struct*/
+ ret = xsc_core_vector2eqn(xdev, c->chl_idx, &eqn, &irqn);
+ if (ret)
+ goto err_free;
+
+ in->ctx.eqn = cpu_to_be16(eqn);
+ in->ctx.log_cq_sz = pcq_param->cq_attr.q_log_size;
+ in->ctx.pa_num = cpu_to_be16(hw_npages);
+ in->ctx.glb_func_id = cpu_to_be16(xdev->glb_func_id);
+
+ xsc_core_fill_page_frag_array(&pcq->wq_ctrl.buf,
+ &in->pas[0],
+ hw_npages);
+
+ ret = xsc_core_eth_create_cq(c->adapter->xdev, &pcq->xcq, in, inlen);
+ if (ret)
+ goto err_free;
+ pcq->xcq.irqn = irqn;
+ pcq->xcq.eq = xsc_core_eq_get(xdev, pcq->xcq.vector);
+ return 0;
+
+err_free:
+ kvfree(in);
+ return ret;
+}
+
+static void xsc_eth_free_cq(struct xsc_cq *cq)
+{
+ xsc_eth_wq_destroy(&cq->wq_ctrl);
+}
+
+static int xsc_eth_open_cq(struct xsc_channel *c,
+ struct xsc_cq *pcq,
+ struct xsc_cq_param *pcq_param)
+{
+ int ret;
+
+ ret = xsc_eth_alloc_cq(c, pcq, pcq_param);
+ if (ret)
+ return ret;
+
+ ret = xsc_eth_set_cq(c, pcq, pcq_param);
+ if (ret)
+ goto err_free_cq;
+
+ xsc_cq_notify_hw_rearm(pcq);
+
+ pcq->napi = &c->napi;
+ pcq->channel = c;
+
+ return 0;
+
+err_free_cq:
+ xsc_eth_free_cq(pcq);
+ return ret;
+}
+
+static int xsc_eth_close_cq(struct xsc_channel *c, struct xsc_cq *pcq)
+{
+ struct xsc_core_device *xdev = c->adapter->xdev;
+ int ret;
+
+ ret = xsc_core_eth_destroy_cq(xdev, &pcq->xcq);
+ if (ret) {
+ netdev_err(xsc_dev_to_netdev(xdev), "failed to close ch%d cq%d, ret=%d\n",
+ c->chl_idx, pcq->xcq.cqn, ret);
+ return ret;
+ }
+
+ xsc_eth_free_cq(pcq);
+
+ return 0;
+}
+
+static void xsc_free_qp_sq_db(struct xsc_sq *sq)
+{
+ kvfree(sq->db.wqe_info);
+ kvfree(sq->db.dma_fifo);
+}
+
+static void xsc_free_qp_sq(struct xsc_sq *sq)
+{
+ xsc_free_qp_sq_db(sq);
+ xsc_eth_wq_destroy(&sq->wq_ctrl);
+}
+
+static int xsc_eth_alloc_qp_sq_db(struct xsc_sq *sq, int numa)
+{
+ struct xsc_core_device *xdev = sq->cq.xdev;
+ int wq_sz;
+ int df_sz;
+
+ wq_sz = xsc_wq_cyc_get_size(&sq->wq);
+ df_sz = wq_sz * xdev->caps.send_ds_num;
+ sq->db.dma_fifo = kvzalloc_node(array_size(df_sz,
+ sizeof(*sq->db.dma_fifo)),
+ GFP_KERNEL,
+ numa);
+ sq->db.wqe_info = kvzalloc_node(array_size(wq_sz,
+ sizeof(*sq->db.wqe_info)),
+ GFP_KERNEL,
+ numa);
+
+ if (!sq->db.dma_fifo || !sq->db.wqe_info) {
+ xsc_free_qp_sq_db(sq);
+ return -ENOMEM;
+ }
+
+ sq->dma_fifo_mask = df_sz - 1;
+
+ return 0;
+}
+
+static void xsc_eth_qp_event(struct xsc_core_qp *qp, int type)
+{
+ struct xsc_core_device *xdev;
+ struct xsc_rq *rq;
+ struct xsc_sq *sq;
+
+ if (qp->eth_queue_type == XSC_RES_RQ) {
+ rq = container_of(qp, struct xsc_rq, cqp);
+ xdev = rq->cq.xdev;
+ } else if (qp->eth_queue_type == XSC_RES_SQ) {
+ sq = container_of(qp, struct xsc_sq, cqp);
+ xdev = sq->cq.xdev;
+ } else {
+ pr_err("%s:Unknown eth qp type %d\n", __func__, type);
+ return;
+ }
+
+ switch (type) {
+ case XSC_EVENT_TYPE_WQ_CATAS_ERROR:
+ case XSC_EVENT_TYPE_WQ_INVAL_REQ_ERROR:
+ case XSC_EVENT_TYPE_WQ_ACCESS_ERROR:
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "%s:Async event %x on QP %d\n",
+ __func__, type, qp->qpn);
+ break;
+ default:
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "%s: Unexpected event type %d on QP %d\n",
+ __func__, type, qp->qpn);
+ return;
+ }
+}
+
+static int xsc_eth_open_qp_sq(struct xsc_channel *c,
+ struct xsc_sq *psq,
+ struct xsc_sq_param *psq_param)
+{
+ u8 ele_log_size = psq_param->sq_attr.ele_log_size;
+ u8 q_log_size = psq_param->sq_attr.q_log_size;
+ struct xsc_modify_raw_qp_mbox_in *modify_in;
+ struct xsc_channel_stats *channel_stats;
+ struct xsc_create_qp_mbox_in *in;
+ struct xsc_core_device *xdev;
+ struct xsc_adapter *adapter;
+ struct xsc_stats *stats;
+ unsigned int hw_npages;
+ int inlen;
+ int ret;
+
+ adapter = c->adapter;
+ xdev = adapter->xdev;
+ stats = adapter->stats;
+ channel_stats = &stats->channel_stats[c->chl_idx];
+ psq->stats = &channel_stats->sq;
+ psq_param->wq.db_numa_node = cpu_to_node(c->cpu);
+
+ ret = xsc_eth_wq_cyc_create(xdev, &psq_param->wq,
+ q_log_size, ele_log_size, &psq->wq,
+ &psq->wq_ctrl);
+ if (ret)
+ return ret;
+
+ hw_npages = DIV_ROUND_UP(psq->wq_ctrl.buf.size, PAGE_SIZE_4K);
+ inlen = sizeof(struct xsc_create_qp_mbox_in) +
+ sizeof(__be64) * hw_npages;
+
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in) {
+ ret = -ENOMEM;
+ goto err_sq_wq_destroy;
+ }
+ in->req.input_qpn = cpu_to_be16(XSC_QPN_SQN_STUB); /*no use for eth*/
+ in->req.qp_type = XSC_QUEUE_TYPE_RAW_TSO; /*default sq is tso qp*/
+ in->req.log_sq_sz = ilog2(xdev->caps.send_ds_num) + q_log_size;
+ in->req.pa_num = cpu_to_be16(hw_npages);
+ in->req.cqn_send = cpu_to_be16(psq->cq.xcq.cqn);
+ in->req.cqn_recv = in->req.cqn_send;
+ in->req.glb_funcid = cpu_to_be16(xdev->glb_func_id);
+
+ xsc_core_fill_page_frag_array(&psq->wq_ctrl.buf,
+ &in->req.pas[0], hw_npages);
+
+ ret = xsc_core_eth_create_qp(xdev, in, inlen, &psq->sqn);
+ if (ret)
+ goto err_sq_in_destroy;
+
+ psq->cqp.qpn = psq->sqn;
+ psq->cqp.event = xsc_eth_qp_event;
+ psq->cqp.eth_queue_type = XSC_RES_SQ;
+
+ ret = xsc_core_create_resource_common(xdev, &psq->cqp);
+ if (ret) {
+ netdev_err(adapter->netdev, "%s:error qp:%d errno:%d\n",
+ __func__, psq->sqn, ret);
+ goto err_sq_destroy;
+ }
+
+ psq->channel = c;
+ psq->ch_ix = c->chl_idx;
+ psq->txq_ix = psq->ch_ix;
+
+ /*need to querify from hardware*/
+ psq->hw_mtu = XSC_ETH_HW_MTU_SEND;
+ psq->stop_room = 1;
+
+ ret = xsc_eth_alloc_qp_sq_db(psq, psq_param->wq.db_numa_node);
+ if (ret)
+ goto err_sq_common_destroy;
+
+ inlen = sizeof(struct xsc_modify_raw_qp_mbox_in);
+ modify_in = kvzalloc(inlen, GFP_KERNEL);
+ if (!modify_in) {
+ ret = -ENOMEM;
+ goto err_sq_common_destroy;
+ }
+
+ modify_in->req.qp_out_port = xdev->pf_id;
+ modify_in->pcie_no = xdev->pcie_no;
+ modify_in->req.qpn = cpu_to_be16((u16)(psq->sqn));
+ modify_in->req.func_id = cpu_to_be16(xdev->glb_func_id);
+ modify_in->req.dma_direct = XSC_DMA_DIR_TO_MAC;
+ modify_in->req.prio = 0; /* single traffic class */
+ ret = xsc_core_eth_modify_raw_qp(xdev, modify_in);
+ if (ret)
+ goto err_sq_modify_in_destroy;
+
+ kvfree(modify_in);
+ kvfree(in);
+
+ return 0;
+
+err_sq_modify_in_destroy:
+ kvfree(modify_in);
+
+err_sq_common_destroy:
+ xsc_core_destroy_resource_common(xdev, &psq->cqp);
+
+err_sq_destroy:
+ xsc_core_eth_destroy_qp(xdev, psq->cqp.qpn);
+
+err_sq_in_destroy:
+ kvfree(in);
+
+err_sq_wq_destroy:
+ xsc_eth_wq_destroy(&psq->wq_ctrl);
+ return ret;
+}
+
+static int xsc_eth_close_qp_sq(struct xsc_channel *c, struct xsc_sq *psq)
+{
+ struct xsc_core_device *xdev = c->adapter->xdev;
+ int ret;
+
+ xsc_core_destroy_resource_common(xdev, &psq->cqp);
+
+ ret = xsc_core_eth_destroy_qp(xdev, psq->cqp.qpn);
+ if (ret)
+ return ret;
+
+ xsc_free_qp_sq(psq);
+
+ return 0;
+}
+
+static int xsc_eth_open_channel(struct xsc_adapter *adapter,
+ int idx,
+ struct xsc_channel *c,
+ struct xsc_channel_param *chl_param)
+{
+ struct xsc_core_device *xdev = adapter->xdev;
+ struct net_device *netdev = adapter->netdev;
+ const struct cpumask *aff;
+ unsigned int irqn;
+ u32 eqn;
+ int ret;
+
+ c->adapter = adapter;
+ c->netdev = adapter->netdev;
+ c->chl_idx = idx;
+
+ if (xdev->caps.msix_enable) {
+ ret = xsc_core_vector2eqn(xdev, c->chl_idx, &eqn, &irqn);
+ if (ret)
+ goto err_out;
+ aff = irq_get_affinity_mask(irqn);
+ c->aff_mask = aff;
+ c->cpu = cpumask_first(aff);
+ }
+
+ ret = xsc_eth_open_cq(c, &c->qp.rq.cq, &chl_param->rqcq_param);
+ if (ret)
+ goto err_out;
+
+ ret = xsc_eth_open_cq(c, &c->qp.sq.cq, &chl_param->sqcq_param);
+ if (ret)
+ goto err_close_rq_cq;
+
+ ret = xsc_eth_open_qp_sq(c, &c->qp.sq, &chl_param->sq_param);
+ if (ret)
+ goto err_close_sq_cq;
+
+ netif_napi_add(netdev, &c->napi, xsc_eth_napi_poll);
+
+ netdev_dbg(adapter->netdev, "open channel%d ok\n", idx);
+ return 0;
+
+err_close_sq_cq:
+ xsc_eth_close_cq(c, &c->qp.sq.cq);
+err_close_rq_cq:
+ xsc_eth_close_cq(c, &c->qp.rq.cq);
+err_out:
+ netdev_err(adapter->netdev, "failed to open channel: ch%d, err=%d\n",
+ idx, ret);
+ return ret;
+}
+
+static int xsc_eth_modify_qps_channel(struct xsc_adapter *adapter,
+ struct xsc_channel *c)
+{
+ int ret;
+
+ c->qp.rq.post_wqes(&c->qp.rq);
+ ret = xsc_core_eth_modify_qp_status(adapter->xdev, c->qp.rq.rqn,
+ XSC_CMD_OP_RTR2RTS_QP);
+ if (ret)
+ return ret;
+
+ return xsc_core_eth_modify_qp_status(adapter->xdev, c->qp.sq.sqn,
+ XSC_CMD_OP_RTR2RTS_QP);
+}
+
+static int xsc_eth_modify_qps(struct xsc_adapter *adapter,
+ struct xsc_eth_channels *chls)
+{
+ int ret;
+ int i;
+
+ for (i = 0; i < chls->num_chl; i++) {
+ struct xsc_channel *c = &chls->c[i];
+
+ ret = xsc_eth_modify_qps_channel(adapter, c);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static void xsc_eth_init_frags_partition(struct xsc_rq *rq)
+{
+ struct xsc_wqe_frag_info next_frag = {};
+ struct xsc_wqe_frag_info *prev;
+ int i;
+
+ next_frag.di = &rq->wqe.di[0];
+ next_frag.offset = 0;
+ prev = NULL;
+
+ for (i = 0; i < xsc_wq_cyc_get_size(&rq->wqe.wq); i++) {
+ struct xsc_rq_frag_info *frag_info = &rq->wqe.info.arr[0];
+ struct xsc_wqe_frag_info *frag =
+ &rq->wqe.frags[i << rq->wqe.info.log_num_frags];
+ int f;
+
+ for (f = 0; f < rq->wqe.info.num_frags; f++, frag++) {
+ if (next_frag.offset + frag_info[f].frag_stride >
+ XSC_RX_FRAG_SZ) {
+ next_frag.di++;
+ next_frag.offset = 0;
+ if (prev)
+ prev->last_in_page = 1;
+ }
+ *frag = next_frag;
+
+ /* prepare next */
+ next_frag.offset += frag_info[f].frag_stride;
+ prev = frag;
+ }
+ }
+
+ if (prev)
+ prev->last_in_page = 1;
+}
+
+static int xsc_eth_init_di_list(struct xsc_rq *rq, u32 wq_sz, int cpu)
+{
+ int len = wq_sz << rq->wqe.info.log_num_frags;
+
+ rq->wqe.di = kvzalloc_node(array_size(len, sizeof(*rq->wqe.di)),
+ GFP_KERNEL, cpu_to_node(cpu));
+ if (!rq->wqe.di)
+ return -ENOMEM;
+
+ xsc_eth_init_frags_partition(rq);
+
+ return 0;
+}
+
+static void xsc_eth_free_di_list(struct xsc_rq *rq)
+{
+ kvfree(rq->wqe.di);
+}
+
+static int xsc_eth_alloc_rq(struct xsc_channel *c,
+ struct xsc_rq *prq,
+ struct xsc_rq_param *prq_param)
+{
+ u8 ele_log_size = prq_param->rq_attr.ele_log_size;
+ u8 q_log_size = prq_param->rq_attr.q_log_size;
+ struct page_pool_params pagepool_params = {0};
+ struct xsc_adapter *adapter = c->adapter;
+ struct xsc_channel_stats *channel_stats;
+ u32 pool_size = 1 << q_log_size;
+ struct xsc_stats *stats;
+ int ret = 0;
+ u32 wq_sz;
+ int i, f;
+
+ stats = c->adapter->stats;
+ channel_stats = &stats->channel_stats[c->chl_idx];
+ prq->stats = &channel_stats->rq;
+ prq_param->wq.db_numa_node = cpu_to_node(c->cpu);
+
+ ret = xsc_eth_wq_cyc_create(c->adapter->xdev, &prq_param->wq,
+ q_log_size, ele_log_size, &prq->wqe.wq,
+ &prq->wq_ctrl);
+ if (ret)
+ return ret;
+
+ wq_sz = xsc_wq_cyc_get_size(&prq->wqe.wq);
+
+ prq->wqe.info = prq_param->frags_info;
+ prq->wqe.frags =
+ kvzalloc_node(array_size((wq_sz << prq->wqe.info.log_num_frags),
+ sizeof(*prq->wqe.frags)),
+ GFP_KERNEL,
+ cpu_to_node(c->cpu));
+ if (!prq->wqe.frags) {
+ ret = -ENOMEM;
+ goto err_destroy_wq;
+ }
+
+ ret = xsc_eth_init_di_list(prq, wq_sz, c->cpu);
+ if (ret)
+ goto err_free_frags;
+
+ prq->buff.map_dir = DMA_FROM_DEVICE;
+
+ /* Create a page_pool and register it with rxq */
+ pool_size = wq_sz << prq->wqe.info.log_num_frags;
+ pagepool_params.order = XSC_RX_FRAG_SZ_ORDER;
+ pagepool_params.flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV;
+ pagepool_params.pool_size = pool_size;
+ pagepool_params.nid = cpu_to_node(c->cpu);
+ pagepool_params.dev = c->adapter->dev;
+ pagepool_params.dma_dir = prq->buff.map_dir;
+ pagepool_params.max_len = PAGE_SIZE;
+
+ prq->page_pool = page_pool_create(&pagepool_params);
+ if (IS_ERR(prq->page_pool)) {
+ ret = PTR_ERR(prq->page_pool);
+ prq->page_pool = NULL;
+ goto err_free_di_list;
+ }
+
+ if (c->chl_idx == 0)
+ netdev_dbg(adapter->netdev,
+ "page pool: size=%d, cpu=%d, pool_numa=%d, mtu=%d, wqe_numa=%d\n",
+ pool_size, c->cpu, pagepool_params.nid,
+ adapter->nic_param.mtu,
+ prq_param->wq.buf_numa_node);
+
+ for (i = 0; i < wq_sz; i++) {
+ struct xsc_eth_rx_wqe_cyc *wqe =
+ xsc_wq_cyc_get_wqe(&prq->wqe.wq, i);
+
+ for (f = 0; f < prq->wqe.info.num_frags; f++) {
+ u32 frag_size = prq->wqe.info.arr[f].frag_size;
+ u32 val = 0;
+
+ val |= FIELD_PREP(XSC_WQE_DATA_SEG_SEG_LEN_MASK,
+ frag_size);
+ wqe->data[f].data0 = cpu_to_le32(val);
+ wqe->data[f].mkey = cpu_to_le32(XSC_INVALID_LKEY);
+ }
+
+ for (; f < prq->wqe.info.frags_max_num; f++) {
+ wqe->data[f].data0 = 0;
+ wqe->data[f].mkey = cpu_to_le32(XSC_INVALID_LKEY);
+ wqe->data[f].va = 0;
+ }
+ }
+
+ prq->post_wqes = xsc_eth_post_rx_wqes;
+ prq->handle_rx_cqe = xsc_eth_handle_rx_cqe;
+ prq->dealloc_wqe = xsc_eth_dealloc_rx_wqe;
+ prq->wqe.skb_from_cqe = xsc_rx_is_linear_skb(adapter->nic_param.mtu) ?
+ xsc_skb_from_cqe_linear :
+ xsc_skb_from_cqe_nonlinear;
+ prq->ix = c->chl_idx;
+ prq->frags_sz = adapter->nic_param.rq_frags_size;
+
+ return 0;
+
+err_free_di_list:
+ xsc_eth_free_di_list(prq);
+err_free_frags:
+ kvfree(prq->wqe.frags);
+err_destroy_wq:
+ xsc_eth_wq_destroy(&prq->wq_ctrl);
+ return ret;
+}
+
+static void xsc_free_qp_rq(struct xsc_rq *rq)
+{
+ kvfree(rq->wqe.frags);
+ kvfree(rq->wqe.di);
+
+ if (rq->page_pool)
+ page_pool_destroy(rq->page_pool);
+
+ xsc_eth_wq_destroy(&rq->wq_ctrl);
+}
+
+static int xsc_eth_open_rss_qp_rqs(struct xsc_adapter *adapter,
+ struct xsc_rq_param *prq_param,
+ struct xsc_eth_channels *chls,
+ unsigned int num_chl)
+{
+ u8 q_log_size = prq_param->rq_attr.q_log_size;
+ struct xsc_create_multiqp_mbox_in *in;
+ struct xsc_create_qp_request *req;
+ unsigned int hw_npages;
+ struct xsc_channel *c;
+ int ret = 0, err = 0;
+ struct xsc_rq *prq;
+ int paslen = 0;
+ int entry_len;
+ u32 rqn_base;
+ int i, n;
+ int inlen;
+
+ for (i = 0; i < num_chl; i++) {
+ c = &chls->c[i];
+ prq = &c->qp.rq;
+
+ ret = xsc_eth_alloc_rq(c, prq, prq_param);
+ if (ret)
+ goto err_alloc_rqs;
+
+ hw_npages = DIV_ROUND_UP(prq->wq_ctrl.buf.size, PAGE_SIZE_4K);
+ /*support different npages number smoothly*/
+ entry_len = sizeof(struct xsc_create_qp_request) +
+ sizeof(__be64) * hw_npages;
+
+ paslen += entry_len;
+ }
+
+ inlen = sizeof(struct xsc_create_multiqp_mbox_in) + paslen;
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in) {
+ ret = -ENOMEM;
+ goto err_create_rss_rqs;
+ }
+
+ in->qp_num = cpu_to_be16(num_chl);
+ in->qp_type = XSC_QUEUE_TYPE_RAW;
+ in->req_len = cpu_to_be32(inlen);
+
+ req = (struct xsc_create_qp_request *)&in->data[0];
+ n = 0;
+ for (i = 0; i < num_chl; i++) {
+ c = &chls->c[i];
+ prq = &c->qp.rq;
+
+ hw_npages = DIV_ROUND_UP(prq->wq_ctrl.buf.size, PAGE_SIZE_4K);
+ /* no use for eth */
+ req->input_qpn = cpu_to_be16(0);
+ req->qp_type = XSC_QUEUE_TYPE_RAW;
+ req->log_rq_sz = ilog2(adapter->xdev->caps.recv_ds_num)
+ + q_log_size;
+ req->pa_num = cpu_to_be16(hw_npages);
+ req->cqn_recv = cpu_to_be16(prq->cq.xcq.cqn);
+ req->cqn_send = req->cqn_recv;
+ req->glb_funcid = cpu_to_be16(adapter->xdev->glb_func_id);
+
+ xsc_core_fill_page_frag_array(&prq->wq_ctrl.buf,
+ &req->pas[0], hw_npages);
+ n++;
+ entry_len = sizeof(struct xsc_create_qp_request) +
+ sizeof(__be64) * hw_npages;
+ req = (struct xsc_create_qp_request *)
+ ((u8 *)req + entry_len);
+ }
+
+ ret = xsc_core_eth_create_rss_qp_rqs(adapter->xdev, in, inlen,
+ &rqn_base);
+ kvfree(in);
+ if (ret)
+ goto err_create_rss_rqs;
+
+ n = 0;
+ for (i = 0; i < num_chl; i++) {
+ c = &chls->c[i];
+ prq = &c->qp.rq;
+
+ prq->rqn = rqn_base + n;
+ prq->cqp.qpn = prq->rqn;
+ prq->cqp.event = xsc_eth_qp_event;
+ prq->cqp.eth_queue_type = XSC_RES_RQ;
+ ret = xsc_core_create_resource_common(adapter->xdev,
+ &prq->cqp);
+ if (ret) {
+ err = ret;
+ netdev_err(adapter->netdev,
+ "create resource common error qp:%d errno:%d\n",
+ prq->rqn, ret);
+ continue;
+ }
+
+ n++;
+ }
+ if (err)
+ return err;
+
+ adapter->channels.rqn_base = rqn_base;
+ return 0;
+
+err_create_rss_rqs:
+ i = num_chl;
+err_alloc_rqs:
+ for (--i; i >= 0; i--) {
+ c = &chls->c[i];
+ xsc_free_qp_rq(&c->qp.rq);
+ }
+ return ret;
+}
+
+static void xsc_eth_free_rx_wqe(struct xsc_rq *rq)
+{
+ struct xsc_wq_cyc *wq = &rq->wqe.wq;
+ u16 wqe_ix;
+
+ while (!xsc_wq_cyc_is_empty(wq)) {
+ wqe_ix = xsc_wq_cyc_get_tail(wq);
+ rq->dealloc_wqe(rq, wqe_ix);
+ xsc_wq_cyc_pop(wq);
+ }
+}
+
+static int xsc_eth_close_qp_rq(struct xsc_channel *c, struct xsc_rq *prq)
+{
+ struct xsc_core_device *xdev = c->adapter->xdev;
+ int ret;
+
+ xsc_core_destroy_resource_common(xdev, &prq->cqp);
+
+ ret = xsc_core_eth_destroy_qp(xdev, prq->cqp.qpn);
+ if (ret)
+ return ret;
+
+ xsc_eth_free_rx_wqe(prq);
+ xsc_free_qp_rq(prq);
+
+ return 0;
+}
+
+static void xsc_eth_close_channel(struct xsc_channel *c, bool free_rq)
+{
+ if (free_rq)
+ xsc_eth_close_qp_rq(c, &c->qp.rq);
+ xsc_eth_close_cq(c, &c->qp.rq.cq);
+ memset(&c->qp.rq, 0, sizeof(struct xsc_rq));
+
+ xsc_eth_close_qp_sq(c, &c->qp.sq);
+ xsc_eth_close_cq(c, &c->qp.sq.cq);
+
+ netif_napi_del(&c->napi);
+}
+
+static int xsc_eth_open_channels(struct xsc_adapter *adapter)
+{
+ struct xsc_eth_channels *chls = &adapter->channels;
+ struct xsc_core_device *xdev = adapter->xdev;
+ struct xsc_channel_param *chl_param;
+ bool free_rq = false;
+ int ret = 0;
+ int i;
+
+ chls->num_chl = adapter->nic_param.num_channels;
+ chls->c = kcalloc_node(chls->num_chl, sizeof(struct xsc_channel),
+ GFP_KERNEL, xdev->numa_node);
+ if (!chls->c) {
+ ret = -ENOMEM;
+ goto err_out;
+ }
+
+ chl_param = kvzalloc_obj(*chl_param, GFP_KERNEL);
+ if (!chl_param) {
+ ret = -ENOMEM;
+ goto err_free_ch;
+ }
+
+ xsc_eth_build_channel_param(adapter, chl_param);
+
+ for (i = 0; i < chls->num_chl; i++) {
+ ret = xsc_eth_open_channel(adapter, i, &chls->c[i], chl_param);
+ if (ret)
+ goto err_close_channel;
+ }
+
+ ret = xsc_eth_open_rss_qp_rqs(adapter,
+ &chl_param->rq_param,
+ chls,
+ chls->num_chl);
+ if (ret)
+ goto err_close_channel;
+ free_rq = true;
+
+ for (i = 0; i < chls->num_chl; i++)
+ napi_enable(&chls->c[i].napi);
+
+ /* flush cache to memory before interrupt and napi_poll running */
+ smp_wmb();
+
+ ret = xsc_eth_modify_qps(adapter, chls);
+ if (ret)
+ goto err_close_channel;
+
+ kvfree(chl_param);
+ return 0;
+
+err_close_channel:
+ for (--i; i >= 0; i--)
+ xsc_eth_close_channel(&chls->c[i], free_rq);
+
+ kvfree(chl_param);
+err_free_ch:
+ kfree(chls->c);
+err_out:
+ chls->num_chl = 0;
+ netdev_err(adapter->netdev, "failed to open %d channels, err=%d\n",
+ chls->num_chl, ret);
+ return ret;
+}
+
+static void xsc_eth_close_channels(struct xsc_adapter *adapter)
+{
+ struct xsc_channel *c;
+ int i;
+
+ for (i = 0; i < adapter->channels.num_chl; i++) {
+ c = &adapter->channels.c[i];
+
+ xsc_eth_close_channel(c, true);
+ }
+
+ kfree(adapter->channels.c);
+ adapter->channels.num_chl = 0;
+}
+
+static void xsc_eth_build_tx2sq_maps(struct xsc_adapter *adapter)
+{
+ struct xsc_channel *c;
+ struct xsc_sq *psq;
+ int i;
+
+ for (i = 0; i < adapter->channels.num_chl; i++) {
+ c = &adapter->channels.c[i];
+ psq = &c->qp.sq;
+ adapter->txq2sq[psq->txq_ix] = psq;
+ }
+}
+
+static void xsc_eth_activate_txqsq(struct xsc_channel *c)
+{
+ struct xsc_sq *psq = &c->qp.sq;
+
+ psq->txq = netdev_get_tx_queue(psq->channel->netdev, psq->txq_ix);
+ set_bit(XSC_ETH_SQ_STATE_ENABLED, &psq->state);
+ netdev_tx_reset_queue(psq->txq);
+ netif_tx_start_queue(psq->txq);
+}
+
+static void xsc_eth_deactivate_txqsq(struct xsc_channel *c)
+{
+ clear_bit(XSC_ETH_SQ_STATE_ENABLED, &c->qp.sq.state);
+}
+
+static void xsc_activate_rq(struct xsc_channel *c)
+{
+ set_bit(XSC_ETH_RQ_STATE_ENABLED, &c->qp.rq.state);
+}
+
+static void xsc_deactivate_rq(struct xsc_channel *c)
+{
+ clear_bit(XSC_ETH_RQ_STATE_ENABLED, &c->qp.rq.state);
+}
+
+static void xsc_eth_activate_channel(struct xsc_channel *c)
+{
+ xsc_eth_activate_txqsq(c);
+ xsc_activate_rq(c);
+}
+
+static void xsc_eth_deactivate_channel(struct xsc_channel *c)
+{
+ xsc_deactivate_rq(c);
+ xsc_eth_deactivate_txqsq(c);
+}
+
+static void xsc_eth_activate_channels(struct xsc_eth_channels *chs)
+{
+ int i;
+
+ for (i = 0; i < chs->num_chl; i++)
+ xsc_eth_activate_channel(&chs->c[i]);
+}
+
+static void xsc_eth_deactivate_channels(struct xsc_eth_channels *chs)
+{
+ int i;
+
+ for (i = 0; i < chs->num_chl; i++)
+ xsc_eth_deactivate_channel(&chs->c[i]);
+
+ /* Sync with all NAPIs to wait until they stop using queues. */
+ synchronize_net();
+
+ for (i = 0; i < chs->num_chl; i++)
+ /* last doorbell out */
+ napi_disable(&chs->c[i].napi);
+}
+
+static void xsc_eth_activate_priv_channels(struct xsc_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+
+ netif_set_real_num_tx_queues(netdev, adapter->channels.num_chl);
+ netif_set_real_num_rx_queues(netdev, adapter->channels.num_chl);
+
+ xsc_eth_build_tx2sq_maps(adapter);
+ xsc_eth_activate_channels(&adapter->channels);
+ netif_tx_start_all_queues(adapter->netdev);
+}
+
+static void xsc_eth_deactivate_priv_channels(struct xsc_adapter *adapter)
+{
+ netif_tx_disable(adapter->netdev);
+ xsc_eth_deactivate_channels(&adapter->channels);
+}
+
+static int xsc_eth_sw_init(struct xsc_adapter *adapter)
+{
+ int ret;
+
+ ret = xsc_eth_open_channels(adapter);
+ if (ret)
+ return ret;
+
+ xsc_eth_activate_priv_channels(adapter);
+
+ return 0;
+}
+
+static void xsc_eth_sw_deinit(struct xsc_adapter *adapter)
+{
+ xsc_eth_deactivate_priv_channels(adapter);
+
+ return xsc_eth_close_channels(adapter);
+}
+
+static bool xsc_eth_get_link_status(struct xsc_adapter *adapter)
+{
+ struct xsc_core_device *xdev = adapter->xdev;
+ u16 vport = 0;
+ bool link_up;
+
+ link_up = xsc_core_query_vport_state(xdev, vport);
+
+ return link_up ? true : false;
+}
+
+static int xsc_eth_change_link_status(struct xsc_adapter *adapter)
+{
+ bool link_up;
+
+ link_up = xsc_eth_get_link_status(adapter);
+
+ if (link_up && !netif_carrier_ok(adapter->netdev)) {
+ netdev_info(adapter->netdev, "Link up\n");
+ netif_carrier_on(adapter->netdev);
+ } else if (!link_up && netif_carrier_ok(adapter->netdev)) {
+ netdev_info(adapter->netdev, "Link down\n");
+ netif_carrier_off(adapter->netdev);
+ }
+
+ return 0;
+}
+
+/*
+ * Registered with the PCI layer, which runs it on its own workqueue for
+ * every XSC_CMD_EVENT_RESP_CHANGE_LINK event.
+ */
+static void xsc_eth_link_event_handler(void *arg)
+{
+ struct xsc_core_device *xdev = arg;
+ struct xsc_adapter *adapter = xdev->eth_priv;
+ int err;
+
+ if (!adapter || !adapter->opened)
+ return;
+
+ err = xsc_eth_change_link_status(adapter);
+ if (err)
+ netdev_err(adapter->netdev, "failed to change linkstatus, err=%d\n",
+ err);
+}
+
+static bool xsc_get_pct_drop_config(struct xsc_core_device *xdev)
+{
+ return (xdev->pdev->device == XSC_MC_PF_DEV_ID) ||
+ (xdev->pdev->device == XSC_MF_SOC_PF_DEV_ID) ||
+ (xdev->pdev->device == XSC_MS_PF_DEV_ID) ||
+ (xdev->pdev->device == XSC_MV_SOC_PF_DEV_ID);
+}
+
+static int xsc_eth_enable_nic_hca(struct xsc_adapter *adapter)
+{
+ struct xsc_cmd_enable_nic_hca_mbox_out out = {};
+ struct xsc_cmd_enable_nic_hca_mbox_in in = {};
+ struct xsc_core_device *xdev = adapter->xdev;
+ struct net_device *netdev = adapter->netdev;
+ u16 caps_mask = 0;
+ u16 caps = 0;
+ int err;
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_ENABLE_NIC_HCA);
+
+ in.rss.rss_en = 1;
+ in.rss.rqn_base = cpu_to_be16(adapter->channels.rqn_base -
+ xdev->caps.raweth_rss_qp_id_base);
+ in.rss.rqn_num = cpu_to_be16(adapter->channels.num_chl);
+ /* hfunc is a constant: the driver never negotiates anything else */
+ in.rss.hfunc = XSC_HASH_FUNC_TOP;
+ in.rss.hash_tmpl = cpu_to_be32(adapter->rss_param.rss_hash_tmpl);
+ caps_mask |= BIT(XSC_TBM_CAP_RSS);
+
+ if (netdev->features & NETIF_F_RXCSUM)
+ caps |= BIT(XSC_TBM_CAP_HASH_PPH);
+ caps_mask |= BIT(XSC_TBM_CAP_HASH_PPH);
+
+ if (xsc_get_pct_drop_config(xdev) && !(netdev->flags & IFF_SLAVE))
+ caps |= BIT(XSC_TBM_CAP_PCT_DROP_CONFIG);
+ caps_mask |= BIT(XSC_TBM_CAP_PCT_DROP_CONFIG);
+
+ memcpy(in.nic.mac_addr, netdev->dev_addr, ETH_ALEN);
+
+ in.nic.caps = cpu_to_be16(caps);
+ in.nic.caps_mask = cpu_to_be16(caps_mask);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err) {
+ netdev_err(netdev, "failed!! err=%d, status=%d\n",
+ err, out.hdr.status);
+ return -ENOEXEC;
+ }
+
+ return 0;
+}
+
+static int xsc_eth_disable_nic_hca(struct xsc_adapter *adapter)
+{
+ struct xsc_cmd_disable_nic_hca_mbox_out out = {};
+ struct xsc_cmd_disable_nic_hca_mbox_in in = {};
+ struct xsc_core_device *xdev = adapter->xdev;
+ struct net_device *netdev = adapter->netdev;
+ u16 caps = 0;
+ int err;
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DISABLE_NIC_HCA);
+
+ if (xsc_get_pct_drop_config(xdev) &&
+ !(netdev->priv_flags & IFF_BONDING))
+ caps |= BIT(XSC_TBM_CAP_PCT_DROP_CONFIG);
+
+ in.nic.caps = cpu_to_be16(caps);
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err) {
+ netdev_err(netdev, "failed!! err=%d, status=%d\n",
+ err, out.hdr.status);
+ return -ENOEXEC;
+ }
+
+ return 0;
+}
+
+static void xsc_set_default_xps_cpumasks(struct xsc_adapter *adapter)
+{
+ struct xsc_core_device *xdev = adapter->xdev;
+ int num_comp_vectors, irq;
+
+ num_comp_vectors = adapter->nic_param.comp_vectors;
+ cpumask_clear(xdev->xps_cpumask);
+
+ for (irq = 0; irq < num_comp_vectors; irq++) {
+ cpumask_set_cpu(cpumask_local_spread(irq, xdev->numa_node),
+ xdev->xps_cpumask);
+ netif_set_xps_queue(adapter->netdev, xdev->xps_cpumask, irq);
+ }
+}
+
+static int xsc_set_port_admin_status(struct xsc_adapter *adapter,
+ enum xsc_port_status status)
+{
+ struct xsc_event_set_port_admin_status_mbox_out out;
+ struct xsc_event_set_port_admin_status_mbox_in in = {};
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_PORT_ADMIN_STATUS);
+ in.status = cpu_to_be16(status);
+
+ return xsc_cmd_exec(adapter->xdev, &in, sizeof(in), &out, sizeof(out));
+}
+
+static int xsc_eth_open(struct net_device *netdev)
+{
+ struct xsc_adapter *adapter = netdev_priv(netdev);
+ struct xsc_core_device *xdev = adapter->xdev;
+ int ret;
+
+ if (adapter->opened) {
+ netdev_err(adapter->netdev, "ndo_open on an already opened device\n");
+ return 0;
+ }
+ ret = xsc_eth_sw_init(adapter);
+ if (ret)
+ return ret;
+
+ ret = xsc_eth_enable_nic_hca(adapter);
+ if (ret)
+ goto err_sw_deinit;
+
+ xsc_irq_event_register(xdev, xsc_eth_link_event_handler);
+
+ if (xsc_eth_get_link_status(adapter)) {
+ netdev_info(netdev, "Link up\n");
+ netif_carrier_on(adapter->netdev);
+ } else {
+ netdev_info(netdev, "Link down\n");
+ }
+
+ adapter->opened = true;
+
+ xsc_set_default_xps_cpumasks(adapter);
+
+ xsc_set_port_admin_status(adapter, XSC_PORT_UP);
+
+ return 0;
+
+err_sw_deinit:
+ xsc_eth_sw_deinit(adapter);
+ return ret;
+}
+
+static int xsc_eth_close(struct net_device *netdev)
+{
+ struct xsc_adapter *adapter = netdev_priv(netdev);
+ int ret;
+
+ if (!adapter->opened)
+ return 0;
+ adapter->opened = false;
+
+ netif_carrier_off(adapter->netdev);
+
+ xsc_eth_sw_deinit(adapter);
+
+ ret = xsc_eth_disable_nic_hca(adapter);
+ if (ret)
+ netdev_err(adapter->netdev, "failed to disable nic hca, err=%d\n",
+ ret);
+
+ xsc_set_port_admin_status(adapter, XSC_PORT_DOWN);
+
+ return ret;
+}
+
+static void xsc_eth_get_stats(struct net_device *netdev,
+ struct rtnl_link_stats64 *stats)
+{
+ struct xsc_adapter *adapter = netdev_priv(netdev);
+
+ xsc_eth_fold_sw_stats64(adapter, stats);
+}
+
+static int xsc_eth_set_hw_mtu(struct xsc_core_device *xdev,
+ u16 mtu, u16 rx_buf_sz)
+{
+ struct xsc_set_mtu_mbox_out out;
+ struct xsc_set_mtu_mbox_in in;
+ int err;
+
+ memset(&in, 0, sizeof(struct xsc_set_mtu_mbox_in));
+ memset(&out, 0, sizeof(struct xsc_set_mtu_mbox_out));
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_MTU);
+ in.mtu = cpu_to_be16(mtu);
+ in.rx_buf_sz_min = cpu_to_be16(rx_buf_sz);
+ in.mac_port = xdev->mac_port;
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(struct xsc_set_mtu_mbox_in), &out,
+ sizeof(struct xsc_set_mtu_mbox_out));
+ if (err) {
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "failed to set hw_mtu=%u rx_buf_sz=%u, err=%d, status=%d\n",
+ mtu, rx_buf_sz, err, out.hdr.status);
+ return err;
+ }
+
+ return 0;
+}
+
+static const struct net_device_ops xsc_netdev_ops = {
+ .ndo_open = xsc_eth_open,
+ .ndo_stop = xsc_eth_close,
+ .ndo_start_xmit = xsc_eth_xmit_start,
+ .ndo_get_stats64 = xsc_eth_get_stats,
+};
+
+static void xsc_eth_build_nic_netdev(struct xsc_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+
+ /* Set up network device as normal. */
+ netdev->netdev_ops = &xsc_netdev_ops;
+
+ netdev->min_mtu = SW_DEFAULT_MTU;
+ netdev->max_mtu = SW_DEFAULT_MTU;
+ /*mtu - macheaderlen - ipheaderlen should be aligned in 8B*/
+ netdev->mtu = SW_DEFAULT_MTU;
+
+ netdev->vlan_features |= NETIF_F_SG |
+ NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM |
+ NETIF_F_TSO | NETIF_F_TSO6 |
+ NETIF_F_RXCSUM | NETIF_F_RXHASH;
+
+ netdev->hw_features = netdev->vlan_features;
+
+ netdev->features = netdev->hw_features | NETIF_F_HIGHDMA;
+}
+
+static int xsc_eth_nic_init(struct xsc_adapter *adapter, u32 ch_num)
+{
+ int err;
+
+ xsc_eth_build_nic_params(adapter, ch_num);
+
+ err = xsc_eth_netdev_init(adapter);
+ if (err)
+ return err;
+
+ xsc_eth_build_nic_netdev(adapter);
+
+ return 0;
+}
+
+static void xsc_eth_nic_cleanup(struct xsc_adapter *adapter)
+{
+ kfree(adapter->txq2sq);
+}
+
+static int xsc_eth_get_mac(struct xsc_core_device *xdev, char *mac)
+{
+ struct xsc_query_eth_mac_mbox_out *out;
+ struct xsc_query_eth_mac_mbox_in in;
+ int err;
+
+ out = kzalloc_obj(*out, GFP_KERNEL);
+ if (!out)
+ return -ENOMEM;
+
+ memset(&in, 0, sizeof(in));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_ETH_MAC);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), out, sizeof(*out));
+ if (err || out->hdr.status) {
+ netdev_warn(xsc_dev_to_netdev(xdev),
+ "get mac failed! err=%d, out.status=%u\n",
+ err, out->hdr.status);
+ err = -ENOEXEC;
+ goto exit;
+ }
+
+ memcpy(mac, out->mac, ETH_ALEN);
+ netdev_dbg(xsc_dev_to_netdev(xdev),
+ "get mac %02x:%02x:%02x:%02x:%02x:%02x\n",
+ mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
+
+exit:
+ kfree(out);
+ return err;
+}
+
+static void xsc_eth_l2_addr_init(struct xsc_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+ char mac[6] = {0};
+ int ret = 0;
+
+ ret = xsc_eth_get_mac(adapter->xdev, mac);
+ if (ret) {
+ netdev_warn(netdev, "get mac failed %d, generate random mac...",
+ ret);
+ eth_random_addr(mac);
+ }
+ eth_hw_addr_set(netdev, mac);
+
+ if (!is_valid_ether_addr(netdev->perm_addr))
+ memcpy(netdev->perm_addr, netdev->dev_addr, netdev->addr_len);
+}
+
+/*
+ * Program the MAC address and the MTU into the hardware. Called once from
+ * probe(), before the netdev is registered: the netdev layer marks the
+ * device present itself in register_netdevice(), so there is nothing to
+ * attach here and no need to take the rtnl lock.
+ */
+static int xsc_eth_nic_enable(struct xsc_adapter *adapter)
+{
+ struct xsc_core_device *xdev = adapter->xdev;
+
+ xsc_eth_l2_addr_init(adapter);
+
+ return xsc_eth_set_hw_mtu(xdev, XSC_SW2HW_MTU(adapter->nic_param.mtu),
+ XSC_SW2HW_RX_PKT_LEN(adapter->nic_param.mtu));
+}
+
+static int xsc_eth_probe(struct auxiliary_device *adev,
+ const struct auxiliary_device_id *adev_id)
+{
+ struct xsc_adev *xsc_adev = container_of(adev, struct xsc_adev, adev);
+ struct xsc_core_device *xdev = xsc_adev->xdev;
+ struct xsc_adapter *adapter;
+ struct net_device *netdev;
+ int num_chl;
+ int err;
+
+ num_chl = xsc_get_max_num_channels(xdev);
+
+ netdev = alloc_etherdev_mqs(sizeof(struct xsc_adapter),
+ num_chl, num_chl);
+ if (!netdev) {
+ pr_err("alloc_etherdev_mqs failed, txq=%d, rxq=%d\n",
+ num_chl, num_chl);
+ return -ENOMEM;
+ }
+
+ netdev->dev.parent = &xdev->pdev->dev;
+ adapter = netdev_priv(netdev);
+ adapter->netdev = netdev;
+ adapter->pdev = xdev->pdev;
+ adapter->dev = &adapter->pdev->dev;
+ adapter->xdev = xdev;
+ xdev->eth_priv = adapter;
+
+ err = xsc_eth_nic_init(adapter, num_chl);
+ if (err) {
+ netdev_err(netdev, "xsc_eth_nic_init failed, err=%d\n", err);
+ goto err_free_netdev;
+ }
+
+ err = xsc_eth_nic_enable(adapter);
+ if (err) {
+ netdev_err(netdev, "xsc_eth_nic_enable failed, err=%d\n", err);
+ goto err_nic_cleanup;
+ }
+
+ adapter->stats = kvzalloc_obj(*adapter->stats, GFP_KERNEL);
+ if (!adapter->stats) {
+ err = -ENOMEM;
+ goto err_nic_cleanup;
+ }
+
+ err = register_netdev(netdev);
+ if (err) {
+ netdev_err(netdev, "register_netdev failed, err=%d\n", err);
+ goto err_free_stats;
+ }
+
+ xdev->netdev = netdev;
+
+ return 0;
+
+err_free_stats:
+ kvfree(adapter->stats);
+err_nic_cleanup:
+ xsc_eth_nic_cleanup(adapter);
+err_free_netdev:
+ xdev->eth_priv = NULL;
+ free_netdev(netdev);
+
+ return err;
+}
+
+static void xsc_eth_remove(struct auxiliary_device *adev)
+{
+ struct xsc_adev *xsc_adev = container_of(adev, struct xsc_adev, adev);
+ struct xsc_core_device *xdev = xsc_adev->xdev;
+ struct xsc_adapter *adapter;
+
+ if (!xdev)
+ return;
+
+ adapter = xdev->eth_priv;
+ if (!adapter)
+ return;
+
+ /* Stop link events reaching this adapter and wait out one already in
+ * flight, which would otherwise touch the memory freed below.
+ */
+ xsc_irq_event_unregister(xdev);
+
+ /* Runs ndo_stop(), so the queues and the HCA are shut down first. */
+ unregister_netdev(adapter->netdev);
+
+ /* Must precede free_netdev(): the adapter lives in the netdev priv. */
+ xsc_eth_nic_cleanup(adapter);
+ kvfree(adapter->stats);
+ free_netdev(adapter->netdev);
+
+ xdev->netdev = NULL;
+ xdev->eth_priv = NULL;
+}
+
+static const struct auxiliary_device_id xsc_eth_id_table[] = {
+ { .name = XSC_PCI_DRV_NAME "." XSC_ETH_ADEV_NAME },
+ {},
+};
+MODULE_DEVICE_TABLE(auxiliary, xsc_eth_id_table);
+
+static struct auxiliary_driver xsc_eth_driver = {
+ .name = "eth",
+ .probe = xsc_eth_probe,
+ .remove = xsc_eth_remove,
+ .id_table = xsc_eth_id_table,
+};
+module_auxiliary_driver(xsc_eth_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Yunsilicon XSC ethernet driver");
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth.h
new file mode 100644
index 0000000000000..4d9f92e094ae0
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth.h
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_ETH_H
+#define __XSC_ETH_H
+
+#include <linux/udp.h>
+
+#include "common/xsc_device.h"
+#include "xsc_eth_common.h"
+#include "xsc_eth_stats.h"
+
+#define XSC_INVALID_LKEY 0x100
+
+struct xsc_rss_params {
+ u32 rx_hash_fields[XSC_NUM_INDIR_TIRS];
+ u32 rss_hash_tmpl;
+};
+
+struct xsc_adapter {
+ struct net_device *netdev;
+ struct pci_dev *pdev;
+ struct device *dev;
+ struct xsc_core_device *xdev;
+
+ struct xsc_eth_params nic_param;
+ struct xsc_rss_params rss_param;
+
+ struct xsc_eth_channels channels;
+ struct xsc_sq **txq2sq;
+
+ /* true between ndo_open and ndo_stop */
+ bool opened;
+
+ struct xsc_stats *stats;
+};
+
+static inline struct net_device *xsc_dev_to_netdev(struct xsc_core_device *xdev)
+{
+ return ((struct xsc_adapter *)xdev->eth_priv)->netdev;
+}
+
+#endif /* __XSC_ETH_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_common.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_common.h
new file mode 100644
index 0000000000000..be476f31f31e7
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_common.h
@@ -0,0 +1,124 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_ETH_COMMON_H
+#define __XSC_ETH_COMMON_H
+
+#include "xsc_queue.h"
+#include "xsc_pph.h"
+
+#define SW_DEFAULT_MTU ETH_DATA_LEN
+
+#define XSC_ETH_HW_MTU_SEND 9800
+#define XSC_ETH_HARD_MTU (ETH_HLEN + VLAN_HLEN * 2 + ETH_FCS_LEN)
+#define XSC_SW2HW_MTU(mtu) ((mtu) + XSC_ETH_HARD_MTU)
+#define XSC_SW2HW_HLEN (ETH_HLEN + VLAN_HLEN * 2 + ETH_FCS_LEN + \
+ XSC_PPH_HEAD_LEN)
+#define XSC_SW2HW_FRAG_SIZE(mtu) ((mtu) + XSC_SW2HW_HLEN)
+#define XSC_ETH_RX_MAX_HEAD_ROOM 256
+#define XSC_SW2HW_RX_PKT_LEN(mtu) \
+ ((mtu) + ETH_HLEN + XSC_ETH_RX_MAX_HEAD_ROOM)
+
+#define XSC_RX_MAX_HEAD (256)
+
+#define XSC_QPN_SQN_STUB 1025
+
+#define XSC_ETH_MAX_NUM_CHANNELS 256
+
+#define XSC_SKB_FRAG_SZ(len) (SKB_DATA_ALIGN(len) + \
+ SKB_DATA_ALIGN(sizeof(struct skb_shared_info)))
+
+#define XSC_RQCQ_ELE_SZ 32
+#define XSC_SQCQ_ELE_SZ 32
+
+#define XSC_LOG_RQCQ_SZ 0xb
+#define XSC_LOG_SQCQ_SZ 0xa
+
+#define XSC_RQCQ_ELE_NUM BIT(XSC_LOG_RQCQ_SZ)
+#define XSC_SQCQ_ELE_NUM BIT(XSC_LOG_SQCQ_SZ)
+
+enum xsc_port_status {
+ XSC_PORT_DOWN = 0,
+ XSC_PORT_UP = 1,
+};
+
+enum xsc_queue_type {
+ XSC_QUEUE_TYPE_RQCQ,
+ XSC_QUEUE_TYPE_SQCQ,
+ XSC_QUEUE_TYPE_RQ,
+ XSC_QUEUE_TYPE_SQ,
+};
+
+struct xsc_queue_attr {
+ u32 ele_num;
+ u32 ele_size;
+ u8 ele_log_size;
+ u8 q_log_size;
+};
+
+struct xsc_eth_rx_wqe_cyc {
+ DECLARE_FLEX_ARRAY(struct xsc_wqe_data_seg, data);
+};
+
+struct xsc_cq_param {
+ struct xsc_wq_param wq;
+ struct xsc_queue_attr cq_attr;
+};
+
+struct xsc_rq_param {
+ struct xsc_wq_param wq;
+ struct xsc_queue_attr rq_attr;
+ struct xsc_rq_frags_info frags_info;
+};
+
+struct xsc_sq_param {
+ struct xsc_wq_param wq;
+ struct xsc_queue_attr sq_attr;
+};
+
+struct xsc_channel_param {
+ struct xsc_cq_param rqcq_param;
+ struct xsc_cq_param sqcq_param;
+ struct xsc_rq_param rq_param;
+ struct xsc_sq_param sq_param;
+};
+
+struct xsc_eth_qp {
+ struct xsc_rq rq;
+ struct xsc_sq sq;
+};
+
+struct xsc_eth_params {
+ u16 num_channels;
+ u16 max_num_ch;
+ u32 mtu;
+ u32 comp_vectors;
+ u32 sq_size;
+ u32 rq_size;
+ u32 rq_frags_size;
+};
+
+struct xsc_channel {
+ /* data path */
+ struct xsc_eth_qp qp;
+ struct napi_struct napi;
+ int chl_idx;
+
+ /*relationship*/
+ struct xsc_adapter *adapter;
+ struct net_device *netdev;
+ int cpu;
+
+ /* data path - accessed per napi poll */
+ const struct cpumask *aff_mask;
+} ____cacheline_aligned_in_smp;
+
+struct xsc_eth_channels {
+ struct xsc_channel *c;
+ unsigned int num_chl;
+ u32 rqn_base;
+};
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_rx.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_rx.c
new file mode 100644
index 0000000000000..3913a9671cc19
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_rx.c
@@ -0,0 +1,551 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd. All
+ * rights reserved.
+ * Copyright (c) 2015-2016, Mellanox Technologies. All rights reserved.
+ */
+
+#include <linux/net_tstamp.h>
+#include <linux/device.h>
+
+#include "xsc_pp.h"
+#include "xsc_eth.h"
+#include "xsc_eth_txrx.h"
+#include "xsc_eth_common.h"
+#include "xsc_pph.h"
+
+static void xsc_rq_notify_hw(struct xsc_rq *rq)
+{
+ struct xsc_core_device *xdev = rq->cq.xdev;
+ struct xsc_wq_cyc *wq = &rq->wqe.wq;
+ u64 rqwqe_id;
+
+ rqwqe_id = wq->wqe_ctr << (ilog2(xdev->caps.recv_ds_num));
+ /* reverse wqe index to ds index */
+ xsc_update_rx_db(xdev, rq->rqn, rqwqe_id);
+}
+
+static void xsc_skb_set_hash(struct xsc_adapter *adapter,
+ struct xsc_cqe *cqe,
+ struct sk_buff *skb)
+{
+ struct xsc_rss_params *rss = &adapter->rss_param;
+ bool l3_hash = false;
+ bool l4_hash = false;
+ u32 hash_field;
+ int ht = 0;
+
+ if (adapter->netdev->features & NETIF_F_RXHASH) {
+ if (skb->protocol == htons(ETH_P_IP)) {
+ hash_field = rss->rx_hash_fields[XSC_TT_IPV4_TCP];
+ if (hash_field & XSC_HASH_FIELD_SEL_SRC_IP ||
+ hash_field & XSC_HASH_FIELD_SEL_DST_IP)
+ l3_hash = true;
+
+ if (hash_field & XSC_HASH_FIELD_SEL_SPORT ||
+ hash_field & XSC_HASH_FIELD_SEL_DPORT)
+ l4_hash = true;
+ } else if (skb->protocol == htons(ETH_P_IPV6)) {
+ hash_field = rss->rx_hash_fields[XSC_TT_IPV6_TCP];
+ if (hash_field & XSC_HASH_FIELD_SEL_SRC_IPV6 ||
+ hash_field & XSC_HASH_FIELD_SEL_DST_IPV6)
+ l3_hash = true;
+
+ if (hash_field & XSC_HASH_FIELD_SEL_SPORT_V6 ||
+ hash_field & XSC_HASH_FIELD_SEL_DPORT_V6)
+ l4_hash = true;
+ }
+
+ if (l3_hash && l4_hash)
+ ht = PKT_HASH_TYPE_L4;
+ else if (l3_hash)
+ ht = PKT_HASH_TYPE_L3;
+ if (ht)
+ skb_set_hash(skb, le32_to_cpu(cqe->vni), ht);
+ }
+}
+
+static void xsc_handle_csum(struct xsc_cqe *cqe, struct xsc_rq *rq,
+ struct sk_buff *skb, struct xsc_wqe_frag_info *wi)
+{
+ struct xsc_dma_info *dma_info;
+ struct net_device *netdev;
+ void *hw_pph;
+ struct xsc_channel *c;
+ int offset_from;
+
+ c = rq->cq.channel;
+ netdev = c->adapter->netdev;
+ dma_info = wi->di;
+ offset_from = wi->offset;
+ hw_pph = page_address(dma_info->page) + offset_from;
+
+ if (unlikely((netdev->features & NETIF_F_RXCSUM) == 0))
+ goto csum_none;
+
+ if (unlikely(XSC_GET_EPP2SOC_PPH_ERROR_BITMAP(hw_pph) & PACKET_UNKNOWN))
+ goto csum_none;
+
+ if (XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(hw_pph) &&
+ (!(FIELD_GET(XSC_CQE_CSUM_ERR_MASK, le32_to_cpu(cqe->data0)) &
+ OUTER_AND_INNER))) {
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+ skb->csum_level = 1;
+ skb->encapsulation = 1;
+ } else if (XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(hw_pph) &&
+ (!(FIELD_GET(XSC_CQE_CSUM_ERR_MASK,
+ le32_to_cpu(cqe->data0)) &
+ OUTER_BIT) &&
+ (FIELD_GET(XSC_CQE_CSUM_ERR_MASK,
+ le32_to_cpu(cqe->data0)) &
+ INNER_BIT))) {
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+ skb->csum_level = 0;
+ skb->encapsulation = 1;
+ } else if (!XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(hw_pph) &&
+ (!(FIELD_GET(XSC_CQE_CSUM_ERR_MASK,
+ le32_to_cpu(cqe->data0)) &
+ OUTER_BIT))) {
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+ }
+
+ goto out;
+
+csum_none:
+ skb->csum = 0;
+ skb->ip_summed = CHECKSUM_NONE;
+out:
+ return;
+}
+
+static void xsc_build_rx_skb(struct xsc_cqe *cqe,
+ u32 cqe_bcnt,
+ struct xsc_rq *rq,
+ struct sk_buff *skb,
+ struct xsc_wqe_frag_info *wi)
+{
+ struct xsc_adapter *adapter;
+ struct net_device *netdev;
+ struct xsc_channel *c;
+
+ c = rq->cq.channel;
+ adapter = c->adapter;
+ netdev = c->netdev;
+
+ skb->mac_len = ETH_HLEN;
+
+ skb_record_rx_queue(skb, rq->ix);
+ xsc_handle_csum(cqe, rq, skb, wi);
+
+ skb->protocol = eth_type_trans(skb, netdev);
+ xsc_skb_set_hash(adapter, cqe, skb);
+}
+
+static void xsc_complete_rx_cqe(struct xsc_rq *rq,
+ struct xsc_cqe *cqe,
+ u32 cqe_bcnt,
+ struct sk_buff *skb,
+ struct xsc_wqe_frag_info *wi)
+{
+ struct xsc_rq_stats *stats = rq->stats;
+
+ stats->packets++;
+ stats->bytes += cqe_bcnt;
+ xsc_build_rx_skb(cqe, cqe_bcnt, rq, skb, wi);
+}
+
+static void xsc_add_skb_frag(struct xsc_rq *rq,
+ struct sk_buff *skb,
+ struct xsc_dma_info *di,
+ u32 frag_offset, u32 len,
+ unsigned int truesize)
+{
+ struct xsc_channel *c = rq->cq.channel;
+ struct device *dev = c->adapter->dev;
+
+ dma_sync_single_for_cpu(dev, di->addr + frag_offset,
+ len, DMA_FROM_DEVICE);
+ page_ref_inc(di->page);
+ skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
+ di->page, frag_offset, len, truesize);
+}
+
+static void xsc_copy_skb_header(struct device *dev,
+ struct sk_buff *skb,
+ struct xsc_dma_info *dma_info,
+ int offset_from, u32 headlen)
+{
+ void *from = page_address(dma_info->page) + offset_from;
+ /* Aligning len to sizeof(long) optimizes memcpy performance */
+ unsigned int len = ALIGN(headlen, sizeof(long));
+
+ dma_sync_single_for_cpu(dev, dma_info->addr + offset_from, len,
+ DMA_FROM_DEVICE);
+ skb_copy_to_linear_data(skb, from, len);
+}
+
+static struct sk_buff *xsc_build_linear_skb(struct xsc_rq *rq, void *va,
+ u32 frag_size, u16 headroom,
+ u32 cqe_bcnt)
+{
+ struct sk_buff *skb = build_skb(va, frag_size);
+
+ if (unlikely(!skb))
+ return NULL;
+
+ skb_reserve(skb, headroom);
+ skb_put(skb, cqe_bcnt);
+
+ return skb;
+}
+
+struct sk_buff *xsc_skb_from_cqe_linear(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *wi,
+ u32 cqe_bcnt, u8 has_pph)
+{
+ int pph_len = has_pph ? XSC_PPH_HEAD_LEN : 0;
+ u16 rx_headroom = rq->buff.headroom;
+ struct xsc_dma_info *di = wi->di;
+ struct sk_buff *skb;
+ void *va, *data;
+ u32 frag_size;
+
+ va = page_address(di->page) + wi->offset;
+ data = va + rx_headroom + pph_len;
+ frag_size = XSC_SKB_FRAG_SZ(rx_headroom + cqe_bcnt);
+
+ dma_sync_single_range_for_cpu(rq->cq.xdev->device, di->addr, wi->offset,
+ frag_size, DMA_FROM_DEVICE);
+ net_prefetchw(va); /* xdp_frame data area */
+ net_prefetch(data);
+
+ skb = xsc_build_linear_skb(rq, va, frag_size, (rx_headroom + pph_len),
+ (cqe_bcnt - pph_len));
+ if (unlikely(!skb))
+ return NULL;
+
+ /* queue up for recycling/reuse */
+ page_ref_inc(di->page);
+
+ return skb;
+}
+
+struct sk_buff *xsc_skb_from_cqe_nonlinear(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *wi,
+ u32 cqe_bcnt, u8 has_pph)
+{
+ struct xsc_rq_frag_info *frag_info = &rq->wqe.info.arr[0];
+ u16 headlen = min_t(u32, XSC_RX_MAX_HEAD, cqe_bcnt);
+ struct xsc_wqe_frag_info *head_wi = wi;
+ struct xsc_wqe_frag_info *rx_wi = wi;
+ u16 head_offset = head_wi->offset;
+ u16 byte_cnt = cqe_bcnt - headlen;
+ u16 frag_consumed_bytes = 0;
+ u16 frag_headlen = headlen;
+ struct net_device *netdev;
+ struct xsc_channel *c;
+ struct sk_buff *skb;
+ struct device *dev;
+ u8 fragcnt = 0;
+ int i = 0;
+
+ c = rq->cq.channel;
+ dev = c->adapter->dev;
+ netdev = c->adapter->netdev;
+
+ skb = napi_alloc_skb(rq->cq.napi, ALIGN(XSC_RX_MAX_HEAD, sizeof(long)));
+ if (unlikely(!skb))
+ return NULL;
+
+ net_prefetchw(skb->data);
+
+ if (likely(has_pph)) {
+ headlen = min_t(u32, XSC_RX_MAX_HEAD,
+ (cqe_bcnt - XSC_PPH_HEAD_LEN));
+ frag_headlen = headlen + XSC_PPH_HEAD_LEN;
+ byte_cnt = cqe_bcnt - headlen - XSC_PPH_HEAD_LEN;
+ head_offset += XSC_PPH_HEAD_LEN;
+ }
+
+ for (i = 0; i < rq->wqe.info.num_frags; i++, rx_wi++)
+ rx_wi->is_available = 0;
+
+ while (byte_cnt) {
+ /*figure out whether the first fragment can be a page ?*/
+ frag_consumed_bytes =
+ min_t(u16, frag_info->frag_size - frag_headlen,
+ byte_cnt);
+
+ xsc_add_skb_frag(rq, skb, wi->di, wi->offset + frag_headlen,
+ frag_consumed_bytes, frag_info->frag_stride);
+ byte_cnt -= frag_consumed_bytes;
+
+ /*to protect extend wqe read, drop exceed bytes*/
+ frag_headlen = 0;
+ fragcnt++;
+ if (fragcnt == rq->wqe.info.num_frags) {
+ if (byte_cnt) {
+ netdev_warn(netdev,
+ "large packet reach the maximum rev-wqe num.\n");
+ netdev_warn(netdev,
+ "%u bytes dropped: frag_num=%d, headlen=%d, cqe_cnt=%d, frag0_bytes=%d, frag_size=%d\n",
+ byte_cnt, fragcnt,
+ headlen, cqe_bcnt,
+ frag_consumed_bytes,
+ frag_info->frag_size);
+ }
+ break;
+ }
+
+ frag_info++;
+ wi++;
+ }
+
+ /* copy header */
+ xsc_copy_skb_header(dev, skb, head_wi->di, head_offset, headlen);
+
+ /* skb linear part was allocated with headlen and aligned to long */
+ skb->tail += headlen;
+ skb->len += headlen;
+
+ return skb;
+}
+
+static void xsc_put_rx_frag(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *frag, bool recycle)
+{
+ if (frag->last_in_page)
+ page_pool_recycle_direct(rq->page_pool, frag->di->page);
+}
+
+static struct xsc_wqe_frag_info *get_frag(struct xsc_rq *rq, u16 ix)
+{
+ return &rq->wqe.frags[ix << rq->wqe.info.log_num_frags];
+}
+
+static void xsc_free_rx_wqe(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *wi, bool recycle)
+{
+ int i;
+
+ for (i = 0; i < rq->wqe.info.num_frags; i++, wi++) {
+ if (wi->is_available && recycle)
+ continue;
+ xsc_put_rx_frag(rq, wi, recycle);
+ }
+}
+
+static void xsc_dump_error_rqcqe(struct xsc_rq *rq,
+ struct xsc_cqe *cqe)
+{
+ struct net_device *netdev;
+ struct xsc_channel *c;
+ u32 ci;
+
+ c = rq->cq.channel;
+ netdev = c->adapter->netdev;
+ ci = xsc_cqwq_get_ci(&rq->cq.wq);
+
+ net_err_ratelimited("Error cqe on dev=%s, cqn=%d, ci=%d, rqn=%d, qpn=%ld, error_code=0x%x\n",
+ netdev->name, rq->cq.xcq.cqn, ci,
+ rq->rqn,
+ FIELD_GET(XSC_CQE_QP_ID_MASK,
+ le32_to_cpu(cqe->data0)),
+ get_cqe_opcode(cqe));
+}
+
+void xsc_eth_handle_rx_cqe(struct xsc_cqwq *cqwq,
+ struct xsc_rq *rq, struct xsc_cqe *cqe)
+{
+ struct xsc_wq_cyc *wq = &rq->wqe.wq;
+ u8 cqe_opcode = get_cqe_opcode(cqe);
+ struct xsc_wqe_frag_info *wi;
+ struct sk_buff *skb;
+ u32 cqe_bcnt;
+ u16 ci;
+
+ ci = xsc_wq_cyc_ctr2ix(wq, cqwq->cc);
+ wi = get_frag(rq, ci);
+ if (unlikely(cqe_opcode & BIT(7))) {
+ xsc_dump_error_rqcqe(rq, cqe);
+ goto free_wqe;
+ }
+
+ cqe_bcnt = le32_to_cpu(cqe->msg_len);
+ if ((le32_to_cpu(cqe->data0) & XSC_CQE_HAS_PPH) &&
+ cqe_bcnt <= XSC_PPH_HEAD_LEN)
+ goto free_wqe;
+
+ if (unlikely(cqe_bcnt > rq->frags_sz))
+ goto free_wqe;
+
+ cqe_bcnt = min_t(u32, cqe_bcnt, rq->frags_sz);
+ skb = rq->wqe.skb_from_cqe(rq, wi, cqe_bcnt,
+ !!(le32_to_cpu(cqe->data0) &
+ XSC_CQE_HAS_PPH));
+ if (!skb)
+ goto free_wqe;
+
+ xsc_complete_rx_cqe(rq, cqe,
+ (le32_to_cpu(cqe->data0) & XSC_CQE_HAS_PPH) ?
+ cqe_bcnt - XSC_PPH_HEAD_LEN : cqe_bcnt,
+ skb, wi);
+
+ napi_gro_receive(rq->cq.napi, skb);
+
+free_wqe:
+ xsc_free_rx_wqe(rq, wi, true);
+ xsc_wq_cyc_pop(wq);
+}
+
+int xsc_poll_rx_cq(struct xsc_cq *cq, int budget)
+{
+ struct xsc_rq *rq = container_of(cq, struct xsc_rq, cq);
+ struct xsc_cqwq *cqwq = &cq->wq;
+ struct xsc_cqe *cqe;
+ int work_done = 0;
+
+ if (!test_bit(XSC_ETH_RQ_STATE_ENABLED, &rq->state))
+ return 0;
+
+ while ((work_done < budget) && (cqe = xsc_cqwq_get_cqe(cqwq))) {
+ rq->handle_rx_cqe(cqwq, rq, cqe);
+ ++work_done;
+
+ xsc_cqwq_pop(cqwq);
+ }
+
+ if (!work_done)
+ goto out;
+
+ xsc_cq_notify_hw(cq);
+ /* ensure cq space is freed before enabling more cqes */
+ wmb();
+
+out:
+ return work_done;
+}
+
+static int xsc_page_alloc_pool(struct xsc_rq *rq,
+ struct xsc_dma_info *dma_info)
+{
+ dma_info->page = page_pool_dev_alloc_pages(rq->page_pool);
+ if (unlikely(!dma_info->page))
+ return -ENOMEM;
+ dma_info->addr = page_pool_get_dma_addr(dma_info->page);
+
+ return 0;
+}
+
+static int xsc_get_rx_frag(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *frag)
+{
+ int err = 0;
+
+ if (!frag->offset && !frag->is_available)
+ /* On first frag (offset == 0), replenish page (dma_info
+ * actually). Other frags that point to the same dma_info
+ * (with a different offset) should just use the new one
+ * without replenishing again by themselves.
+ */
+ err = xsc_page_alloc_pool(rq, frag->di);
+
+ return err;
+}
+
+static int xsc_alloc_rx_wqe(struct xsc_rq *rq,
+ struct xsc_eth_rx_wqe_cyc *wqe,
+ u16 ix)
+{
+ struct xsc_wqe_frag_info *frag = get_frag(rq, ix);
+ u64 addr;
+ int err;
+ int i;
+
+ for (i = 0; i < rq->wqe.info.num_frags; i++, frag++) {
+ err = xsc_get_rx_frag(rq, frag);
+ if (unlikely(err))
+ goto err_free_frags;
+
+ addr = frag->di->addr + frag->offset + rq->buff.headroom;
+ wqe->data[i].va = cpu_to_le64(addr);
+ }
+
+ return 0;
+
+err_free_frags:
+ while (--i >= 0)
+ xsc_put_rx_frag(rq, --frag, true);
+
+ return err;
+}
+
+void xsc_eth_dealloc_rx_wqe(struct xsc_rq *rq, u16 ix)
+{
+ struct xsc_wqe_frag_info *wi = get_frag(rq, ix);
+
+ xsc_free_rx_wqe(rq, wi, false);
+}
+
+static int xsc_alloc_rx_wqes(struct xsc_rq *rq, u16 ix, u8 wqe_bulk)
+{
+ struct xsc_wq_cyc *wq = &rq->wqe.wq;
+ struct xsc_eth_rx_wqe_cyc *wqe;
+ int err;
+ int idx;
+ int i;
+
+ for (i = 0; i < wqe_bulk; i++) {
+ idx = xsc_wq_cyc_ctr2ix(wq, (ix + i));
+ wqe = xsc_wq_cyc_get_wqe(wq, idx);
+
+ err = xsc_alloc_rx_wqe(rq, wqe, idx);
+ if (unlikely(err))
+ goto err_free_wqes;
+ }
+
+ return 0;
+
+err_free_wqes:
+ while (--i >= 0)
+ xsc_eth_dealloc_rx_wqe(rq, ix + i);
+
+ return err;
+}
+
+bool xsc_eth_post_rx_wqes(struct xsc_rq *rq)
+{
+ struct xsc_wq_cyc *wq = &rq->wqe.wq;
+ u8 wqe_bulk, wqe_bulk_min;
+ int err = 0;
+ int alloc;
+ u16 head;
+
+ wqe_bulk = rq->wqe.info.wqe_bulk;
+ wqe_bulk_min = rq->wqe.info.wqe_bulk_min;
+ if (xsc_wq_cyc_missing(wq) < wqe_bulk)
+ return false;
+
+ do {
+ head = xsc_wq_cyc_get_head(wq);
+
+ alloc = min_t(int, wqe_bulk, xsc_wq_cyc_missing(wq));
+ if (alloc < wqe_bulk && alloc >= wqe_bulk_min)
+ alloc = alloc & 0xfffffffe;
+
+ if (alloc > 0) {
+ err = xsc_alloc_rx_wqes(rq, head, alloc);
+ if (unlikely(err))
+ break;
+
+ xsc_wq_cyc_push_n(wq, alloc);
+ }
+ } while (xsc_wq_cyc_missing(wq) >= wqe_bulk_min);
+
+ dma_wmb();
+
+ /* ensure wqes are visible to device before updating doorbell record */
+ xsc_rq_notify_hw(rq);
+
+ return !!err;
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.c
new file mode 100644
index 0000000000000..b466fc1f026d4
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.c
@@ -0,0 +1,39 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include "xsc_eth_stats.h"
+#include "xsc_eth.h"
+
+static int xsc_get_netdev_max_channels(struct xsc_adapter *adapter)
+{
+ struct net_device *netdev = adapter->netdev;
+
+ return min_t(unsigned int, netdev->num_rx_queues,
+ netdev->num_tx_queues);
+}
+
+void xsc_eth_fold_sw_stats64(struct xsc_adapter *adapter,
+ struct rtnl_link_stats64 *s)
+{
+ int i;
+
+ for (i = 0; i < xsc_get_netdev_max_channels(adapter); i++) {
+ struct xsc_channel_stats *channel_stats;
+ struct xsc_rq_stats *rq_stats;
+ struct xsc_sq_stats *sq_stats;
+
+ channel_stats = &adapter->stats->channel_stats[i];
+ rq_stats = &channel_stats->rq;
+ sq_stats = &channel_stats->sq;
+
+ s->rx_packets += rq_stats->packets;
+ s->rx_bytes += rq_stats->bytes;
+
+ s->tx_packets += sq_stats->packets;
+ s->tx_bytes += sq_stats->bytes;
+ s->tx_dropped += sq_stats->dropped;
+ }
+}
+
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.h
new file mode 100644
index 0000000000000..a391afe97b418
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_stats.h
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_EN_STATS_H
+#define __XSC_EN_STATS_H
+
+#include "xsc_eth_common.h"
+
+struct xsc_rq_stats {
+ u64 packets;
+ u64 bytes;
+};
+
+struct xsc_sq_stats {
+ u64 packets;
+ u64 bytes;
+ u64 dropped;
+};
+
+struct xsc_channel_stats {
+ struct xsc_sq_stats sq;
+ struct xsc_rq_stats rq;
+} ____cacheline_aligned_in_smp;
+
+struct xsc_stats {
+ struct xsc_channel_stats channel_stats[XSC_ETH_MAX_NUM_CHANNELS];
+};
+
+void xsc_eth_fold_sw_stats64(struct xsc_adapter *adapter,
+ struct rtnl_link_stats64 *s);
+
+#endif /* XSC_EN_STATS_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_tx.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_tx.c
new file mode 100644
index 0000000000000..7495b12b7b442
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_tx.c
@@ -0,0 +1,309 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/tcp.h>
+
+#include "xsc_eth.h"
+#include "xsc_eth_txrx.h"
+
+#define XSC_OPCODE_RAW 7
+
+static void xsc_dma_push(struct xsc_sq *sq, dma_addr_t addr, u32 size,
+ enum xsc_dma_map_type map_type)
+{
+ struct xsc_sq_dma *dma = xsc_dma_get(sq, sq->dma_fifo_pc++);
+
+ dma->addr = addr;
+ dma->size = size;
+ dma->type = map_type;
+}
+
+static void xsc_dma_unmap_wqe(struct xsc_sq *sq, u8 num_dma)
+{
+ struct xsc_adapter *adapter = sq->channel->adapter;
+ struct device *dev = adapter->dev;
+ int i;
+
+ for (i = 0; i < num_dma; i++) {
+ struct xsc_sq_dma *last_pushed_dma;
+
+ last_pushed_dma = xsc_dma_get(sq, --sq->dma_fifo_pc);
+ xsc_tx_dma_unmap(dev, last_pushed_dma);
+ }
+}
+
+static void *xsc_sq_fetch_wqe(struct xsc_sq *sq, size_t size, u16 *pi)
+{
+ struct xsc_wq_cyc *wq = &sq->wq;
+ void *wqe;
+
+ /*caution, sp->pc is default to be zero*/
+ *pi = xsc_wq_cyc_ctr2ix(wq, sq->pc);
+ wqe = xsc_wq_cyc_get_wqe(wq, *pi);
+ memset(wqe, 0, size);
+
+ return wqe;
+}
+
+static u16 xsc_tx_get_gso_ihs(struct xsc_sq *sq, struct sk_buff *skb)
+{
+ u16 ihs;
+
+ if (skb->encapsulation) {
+ ihs = skb_inner_transport_offset(skb) + inner_tcp_hdrlen(skb);
+ } else {
+ if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_L4)
+ ihs = skb_transport_offset(skb) + sizeof(struct udphdr);
+ else
+ ihs = skb_transport_offset(skb) + tcp_hdrlen(skb);
+ }
+
+ return ihs;
+}
+
+static void xsc_txwqe_build_cseg_csum(struct xsc_sq *sq,
+ struct sk_buff *skb,
+ struct xsc_send_wqe_ctrl_seg *cseg)
+{
+ u32 val = le32_to_cpu(cseg->data0);
+
+ if (likely(skb->ip_summed == CHECKSUM_PARTIAL)) {
+ if (skb->encapsulation)
+ val |= FIELD_PREP(XSC_WQE_CTRL_SEG_CSUM_EN_MASK,
+ XSC_ETH_WQE_INNER_AND_OUTER_CSUM);
+ else
+ val |= FIELD_PREP(XSC_WQE_CTRL_SEG_CSUM_EN_MASK,
+ XSC_ETH_WQE_OUTER_CSUM);
+ } else {
+ val |= FIELD_PREP(XSC_WQE_CTRL_SEG_CSUM_EN_MASK,
+ XSC_ETH_WQE_NONE_CSUM);
+ }
+ cseg->data0 = cpu_to_le32(val);
+}
+
+static void xsc_txwqe_build_csegs(struct xsc_sq *sq, struct sk_buff *skb,
+ u16 mss, u16 ihs, u16 headlen,
+ u8 opcode, u16 ds_cnt, u32 msglen,
+ struct xsc_send_wqe_ctrl_seg *cseg)
+{
+ struct xsc_core_device *xdev = sq->cq.xdev;
+ int send_wqe_ds_num_log;
+ u32 val = 0;
+
+ send_wqe_ds_num_log = ilog2(xdev->caps.send_ds_num);
+ xsc_txwqe_build_cseg_csum(sq, skb, cseg);
+
+ if (mss != 0) {
+ val |= XSC_WQE_CTRL_SEG_HAS_PPH |
+ XSC_WQE_CTRL_SEG_SO_TYPE |
+ FIELD_PREP(XSC_WQE_CTRL_SEG_SO_HDR_LEN_MASK, ihs) |
+ FIELD_PREP(XSC_WQE_CTRL_SEG_SO_DATA_SIZE_MASK, mss);
+ cseg->data2 = cpu_to_le32(val);
+ }
+
+ val = le32_to_cpu(cseg->data0);
+ val |= FIELD_PREP(XSC_WQE_CTRL_SEG_MSG_OPCODE_MASK, opcode) |
+ FIELD_PREP(XSC_WQE_CTRL_SEG_WQE_ID_MASK,
+ sq->pc << send_wqe_ds_num_log) |
+ FIELD_PREP(XSC_WQE_CTRL_SEG_DS_DATA_NUM_MASK,
+ ds_cnt - XSC_SEND_WQEBB_CTRL_NUM_DS);
+ cseg->data0 = cpu_to_le32(val);
+ cseg->msg_len = cpu_to_le32(msglen);
+ cseg->data3 = cpu_to_le32(XSC_WQE_CTRL_SEG_CE);
+}
+
+static int xsc_txwqe_build_dsegs(struct xsc_sq *sq, struct sk_buff *skb,
+ u16 ihs, u16 headlen,
+ struct xsc_wqe_data_seg *dseg)
+{
+ struct xsc_adapter *adapter = sq->channel->adapter;
+ struct device *dev = adapter->dev;
+ dma_addr_t dma_addr = 0;
+ u8 num_dma = 0;
+ int i;
+
+ if (headlen) {
+ dma_addr = dma_map_single(dev,
+ skb->data,
+ headlen,
+ DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(dev, dma_addr)))
+ goto err_dma_unmap_wqe;
+
+ dseg->va = cpu_to_le64(dma_addr);
+ dseg->mkey = cpu_to_le32(be32_to_cpu(sq->mkey_be));
+ dseg->data0 |=
+ cpu_to_le32(FIELD_PREP(XSC_WQE_DATA_SEG_SEG_LEN_MASK,
+ headlen));
+
+ xsc_dma_push(sq, dma_addr, headlen, XSC_DMA_MAP_SINGLE);
+ num_dma++;
+ dseg++;
+ }
+
+ for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) {
+ skb_frag_t *frag = &skb_shinfo(skb)->frags[i];
+ int fsz = skb_frag_size(frag);
+
+ dma_addr = skb_frag_dma_map(dev, frag, 0, fsz, DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(dev, dma_addr)))
+ goto err_dma_unmap_wqe;
+
+ dseg->va = cpu_to_le64(dma_addr);
+ dseg->mkey = cpu_to_le32(be32_to_cpu(sq->mkey_be));
+ dseg->data0 |=
+ cpu_to_le32(FIELD_PREP(XSC_WQE_DATA_SEG_SEG_LEN_MASK,
+ fsz));
+
+ xsc_dma_push(sq, dma_addr, fsz, XSC_DMA_MAP_PAGE);
+ num_dma++;
+ dseg++;
+ }
+
+ return num_dma;
+
+err_dma_unmap_wqe:
+ xsc_dma_unmap_wqe(sq, num_dma);
+ return -ENOMEM;
+}
+
+static void xsc_sq_notify_hw(struct xsc_wq_cyc *wq, u16 pc,
+ struct xsc_sq *sq)
+{
+ struct xsc_adapter *adapter = sq->channel->adapter;
+ struct xsc_core_device *xdev = adapter->xdev;
+ int send_ds_num_log = ilog2(xdev->caps.send_ds_num);
+
+ /* reverse wqe index to ds index */
+ xsc_update_tx_db(xdev, sq->sqn, pc << send_ds_num_log);
+}
+
+static void xsc_txwqe_complete(struct xsc_sq *sq, struct sk_buff *skb,
+ u8 opcode, u16 ds_cnt,
+ u8 num_wqebbs, u32 num_bytes, u8 num_dma,
+ struct xsc_tx_wqe_info *wi)
+{
+ struct xsc_wq_cyc *wq = &sq->wq;
+
+ wi->num_bytes = num_bytes;
+ wi->num_dma = num_dma;
+ wi->num_wqebbs = num_wqebbs;
+ wi->skb = skb;
+
+ netdev_tx_sent_queue(sq->txq, num_bytes);
+
+ sq->pc += wi->num_wqebbs;
+
+ if (unlikely(!xsc_wqc_has_room_for(wq, sq->cc, sq->pc, sq->stop_room)))
+ netif_tx_stop_queue(sq->txq);
+
+ if (!netdev_xmit_more() || netif_xmit_stopped(sq->txq))
+ xsc_sq_notify_hw(wq, sq->pc, sq);
+}
+
+static uint32_t xsc_eth_xmit_frame(struct sk_buff *skb,
+ struct xsc_sq *sq,
+ struct xsc_tx_wqe *wqe,
+ u16 pi)
+{
+ struct xsc_core_device *xdev = sq->cq.xdev;
+ struct xsc_sq_stats *stats = sq->stats;
+ struct xsc_send_wqe_ctrl_seg *cseg;
+ struct xsc_wqe_data_seg *dseg;
+ struct xsc_tx_wqe_info *wi;
+ u16 mss, ihs, headlen;
+ u32 num_bytes;
+ u8 num_wqebbs;
+ int num_dma;
+ u16 ds_cnt;
+ u8 opcode;
+
+retry_send:
+ /* Calc ihs and ds cnt, no writes to wqe yet; ctrl-ds is reduced by
+ * ds_data_num.
+ */
+ ds_cnt = XSC_SEND_WQEBB_CTRL_NUM_DS;
+
+ if (skb_is_gso(skb)) {
+ opcode = XSC_OPCODE_RAW;
+ mss = skb_shinfo(skb)->gso_size;
+ ihs = xsc_tx_get_gso_ihs(sq, skb);
+ num_bytes = skb->len +
+ (skb_shinfo(skb)->gso_segs - 1) * ihs;
+ stats->packets += skb_shinfo(skb)->gso_segs;
+ } else {
+ opcode = XSC_OPCODE_RAW;
+ mss = 0;
+ ihs = 0;
+ num_bytes = skb->len;
+ stats->packets++;
+ }
+
+ /*linear data in skb*/
+ headlen = skb->len - skb->data_len;
+ ds_cnt += !!headlen;
+ ds_cnt += skb_shinfo(skb)->nr_frags;
+
+ /* Check packet size. */
+ if (unlikely(mss == 0 && skb->len > sq->hw_mtu))
+ goto err_drop;
+
+ num_wqebbs = DIV_ROUND_UP(ds_cnt, xdev->caps.send_ds_num);
+ /*if ds_cnt exceed one wqe, drop it*/
+ if (num_wqebbs != 1) {
+ if (skb_linearize(skb))
+ goto err_drop;
+ goto retry_send;
+ }
+
+ /* fill wqe */
+ wi = (struct xsc_tx_wqe_info *)&sq->db.wqe_info[pi];
+ cseg = &wqe->ctrl;
+ dseg = &wqe->data[0];
+
+ if (unlikely(skb->len == 0))
+ goto err_drop;
+
+ xsc_txwqe_build_csegs(sq, skb, mss, ihs, headlen,
+ opcode, ds_cnt, skb->len, cseg);
+
+ /*inline header is also use dma to transport*/
+ num_dma = xsc_txwqe_build_dsegs(sq, skb, ihs, headlen, dseg);
+ if (unlikely(num_dma < 0))
+ goto err_drop;
+
+ xsc_txwqe_complete(sq, skb, opcode, ds_cnt, num_wqebbs, num_bytes,
+ num_dma, wi);
+ stats->bytes += num_bytes;
+
+ return NETDEV_TX_OK;
+
+err_drop:
+ stats->dropped++;
+ dev_kfree_skb_any(skb);
+
+ return NETDEV_TX_OK;
+}
+
+netdev_tx_t xsc_eth_xmit_start(struct sk_buff *skb, struct net_device *netdev)
+{
+ struct xsc_adapter *adapter = netdev_priv(netdev);
+ struct xsc_tx_wqe *wqe;
+ struct xsc_sq *sq;
+ u32 ds_num;
+ u16 pi;
+
+ if (!adapter->opened)
+ return NETDEV_TX_BUSY;
+
+ sq = adapter->txq2sq[skb_get_queue_mapping(skb)];
+ if (unlikely(!sq))
+ return NETDEV_TX_BUSY;
+
+ ds_num = adapter->xdev->caps.send_ds_num;
+ wqe = xsc_sq_fetch_wqe(sq, ds_num * XSC_SEND_WQE_DS, &pi);
+
+ return xsc_eth_xmit_frame(skb, sq, wqe, pi);
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c
new file mode 100644
index 0000000000000..3effa79efce06
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.c
@@ -0,0 +1,159 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include "xsc_eth_common.h"
+#include "xsc_eth_txrx.h"
+
+void xsc_cq_notify_hw_rearm(struct xsc_cq *cq)
+{
+ xsc_arm_cq(cq->xdev, cq->xcq.cqn, cq->wq.cc, 0);
+}
+
+void xsc_cq_notify_hw(struct xsc_cq *cq)
+{
+ xsc_update_cq_ci(cq->xdev, cq->xcq.cqn, cq->wq.cc);
+}
+
+static bool xsc_channel_no_affinity_change(struct xsc_channel *c)
+{
+ int current_cpu = smp_processor_id();
+
+ return cpumask_test_cpu(current_cpu, c->aff_mask);
+}
+
+static void xsc_dump_error_sqcqe(struct xsc_sq *sq,
+ struct xsc_cqe *cqe)
+{
+ struct net_device *netdev = sq->channel->netdev;
+ u32 ci = xsc_cqwq_get_ci(&sq->cq.wq);
+
+ net_err_ratelimited("Err cqe on dev %s cqn=0x%x ci=0x%x sqn=0x%x err_code=0x%x qpid=0x%lx\n",
+ netdev->name, sq->cq.xcq.cqn, ci,
+ sq->sqn, get_cqe_opcode(cqe),
+ FIELD_GET(XSC_CQE_QP_ID_MASK,
+ le32_to_cpu(cqe->data0)));
+}
+
+static bool xsc_poll_tx_cq(struct xsc_cq *cq, int napi_budget)
+{
+ struct xsc_adapter *adapter;
+ struct xsc_cqe *cqe;
+ struct device *dev;
+ struct xsc_sq *sq;
+ u32 dma_fifo_cc;
+ u32 nbytes = 0;
+ u16 npkts = 0;
+ int i = 0;
+ u16 sqcc;
+
+ sq = container_of(cq, struct xsc_sq, cq);
+ if (!test_bit(XSC_ETH_SQ_STATE_ENABLED, &sq->state))
+ return false;
+
+ adapter = sq->channel->adapter;
+ dev = adapter->dev;
+
+ cqe = xsc_cqwq_get_cqe(&cq->wq);
+ if (!cqe)
+ goto out;
+
+ if (unlikely(get_cqe_opcode(cqe) & BIT(7))) {
+ xsc_dump_error_sqcqe(sq, cqe);
+ return false;
+ }
+
+ sqcc = sq->cc;
+
+ /* avoid dirtying sq cache line every cqe */
+ dma_fifo_cc = sq->dma_fifo_cc;
+ i = 0;
+ do {
+ struct xsc_tx_wqe_info *wi;
+ struct sk_buff *skb;
+ int j;
+ u16 ci;
+
+ xsc_cqwq_pop(&cq->wq);
+
+ ci = xsc_wq_cyc_ctr2ix(&sq->wq, sqcc);
+ wi = &sq->db.wqe_info[ci];
+ skb = wi->skb;
+
+ /*cqe may be overstanding in real test, not by nop in other*/
+ if (unlikely(!skb))
+ continue;
+
+ for (j = 0; j < wi->num_dma; j++) {
+ struct xsc_sq_dma *dma = xsc_dma_get(sq, dma_fifo_cc++);
+
+ xsc_tx_dma_unmap(dev, dma);
+ }
+
+ npkts++;
+ nbytes += wi->num_bytes;
+ sqcc += wi->num_wqebbs;
+ napi_consume_skb(skb, 0);
+
+ } while ((++i <= napi_budget) && (cqe = xsc_cqwq_get_cqe(&cq->wq)));
+
+ xsc_cq_notify_hw(cq);
+
+ /* ensure cq space is freed before enabling more cqes */
+ wmb();
+
+ sq->dma_fifo_cc = dma_fifo_cc;
+ sq->cc = sqcc;
+
+ netdev_tx_completed_queue(sq->txq, npkts, nbytes);
+
+ if (netif_tx_queue_stopped(sq->txq) &&
+ xsc_wqc_has_room_for(&sq->wq, sq->cc, sq->pc, sq->stop_room)) {
+ netif_tx_wake_queue(sq->txq);
+ }
+
+out:
+ return (i == napi_budget);
+}
+
+int xsc_eth_napi_poll(struct napi_struct *napi, int budget)
+{
+ struct xsc_channel *c = container_of(napi, struct xsc_channel, napi);
+ struct xsc_eth_params *params = &c->adapter->nic_param;
+ struct xsc_rq *rq = &c->qp.rq;
+ bool busy = false;
+ int work_done = 0;
+ int tx_budget;
+
+ rcu_read_lock();
+
+ tx_budget = params->sq_size >> 2;
+ busy |= xsc_poll_tx_cq(&c->qp.sq.cq, tx_budget);
+
+ /* budget=0 means: don't poll rx rings */
+ if (likely(budget)) {
+ work_done = xsc_poll_rx_cq(&rq->cq, budget);
+ busy |= work_done == budget;
+ }
+
+ busy |= rq->post_wqes(rq);
+
+ if (busy) {
+ if (likely(xsc_channel_no_affinity_change(c))) {
+ rcu_read_unlock();
+ return budget;
+ }
+ if (budget && work_done == budget)
+ work_done--;
+ }
+
+ if (unlikely(!napi_complete_done(napi, work_done)))
+ goto err_out;
+
+ xsc_cq_notify_hw_rearm(&c->qp.sq.cq);
+ xsc_cq_notify_hw_rearm(&rq->cq);
+err_out:
+ rcu_read_unlock();
+ return work_done;
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.h
new file mode 100644
index 0000000000000..8d14f943ea778
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_txrx.h
@@ -0,0 +1,91 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_RXTX_H
+#define __XSC_RXTX_H
+
+#include <linux/netdevice.h>
+#include <linux/skbuff.h>
+
+#include "xsc_eth.h"
+
+enum {
+ XSC_ETH_WQE_NONE_CSUM,
+ XSC_ETH_WQE_INNER_CSUM,
+ XSC_ETH_WQE_OUTER_CSUM,
+ XSC_ETH_WQE_INNER_AND_OUTER_CSUM,
+};
+
+void xsc_cq_notify_hw_rearm(struct xsc_cq *cq);
+void xsc_cq_notify_hw(struct xsc_cq *cq);
+int xsc_eth_napi_poll(struct napi_struct *napi, int budget);
+bool xsc_eth_post_rx_wqes(struct xsc_rq *rq);
+void xsc_eth_handle_rx_cqe(struct xsc_cqwq *cqwq,
+ struct xsc_rq *rq, struct xsc_cqe *cqe);
+void xsc_eth_dealloc_rx_wqe(struct xsc_rq *rq, u16 ix);
+struct sk_buff *xsc_skb_from_cqe_linear(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *wi,
+ u32 cqe_bcnt, u8 has_pph);
+struct sk_buff *xsc_skb_from_cqe_nonlinear(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *wi,
+ u32 cqe_bcnt, u8 has_pph);
+int xsc_poll_rx_cq(struct xsc_cq *cq, int budget);
+
+netdev_tx_t xsc_eth_xmit_start(struct sk_buff *skb, struct net_device *netdev);
+
+static inline void xsc_tx_dma_unmap(struct device *dev, struct xsc_sq_dma *dma)
+{
+ switch (dma->type) {
+ case XSC_DMA_MAP_SINGLE:
+ dma_unmap_single(dev, dma->addr, dma->size, DMA_TO_DEVICE);
+ break;
+ case XSC_DMA_MAP_PAGE:
+ dma_unmap_page(dev, dma->addr, dma->size, DMA_TO_DEVICE);
+ break;
+ default:
+ break;
+ }
+}
+
+static inline struct xsc_sq_dma *xsc_dma_get(struct xsc_sq *sq, u32 i)
+{
+ return &sq->db.dma_fifo[i & sq->dma_fifo_mask];
+}
+
+static inline bool xsc_wqc_has_room_for(struct xsc_wq_cyc *wq,
+ u16 cc, u16 pc, u16 n)
+{
+ return (xsc_wq_cyc_ctr2ix(wq, cc - pc) >= n) || (cc == pc);
+}
+
+static inline struct xsc_cqe *xsc_cqwq_get_cqe_buff(struct xsc_cqwq *wq, u32 ix)
+{
+ struct xsc_cqe *cqe = xsc_frag_buf_get_wqe(&wq->fbc, ix);
+
+ return cqe;
+}
+
+static inline struct xsc_cqe *xsc_cqwq_get_cqe(struct xsc_cqwq *wq)
+{
+ u32 ci = xsc_cqwq_get_ci(wq);
+ u8 cqe_ownership_bit;
+ struct xsc_cqe *cqe;
+ u8 sw_ownership_val;
+
+ cqe = xsc_cqwq_get_cqe_buff(wq, ci);
+
+ cqe_ownership_bit = !!(le32_to_cpu(cqe->data1) & XSC_CQE_OWNER);
+ sw_ownership_val = xsc_cqwq_get_wrap_cnt(wq) & 1;
+
+ if (cqe_ownership_bit != sw_ownership_val)
+ return NULL;
+
+ /* ensure cqe content is read after cqe ownership bit */
+ dma_rmb();
+
+ return cqe;
+}
+
+#endif /* XSC_RXTX_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.c b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.c
new file mode 100644
index 0000000000000..b69a4e373923a
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.c
@@ -0,0 +1,80 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd. All
+ * rights reserved.
+ * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved.
+ */
+
+#include "xsc_eth_wq.h"
+#include "xsc_eth.h"
+
+u32 xsc_wq_cyc_get_size(struct xsc_wq_cyc *wq)
+{
+ return (u32)wq->fbc.sz_m1 + 1;
+}
+
+static u32 wq_get_byte_sz(u8 log_sz, u8 log_stride)
+{
+ return ((u32)1 << log_sz) << log_stride;
+}
+
+int xsc_eth_cqwq_create(struct xsc_core_device *xdev,
+ struct xsc_wq_param *param,
+ u8 q_log_size,
+ u8 ele_log_size,
+ struct xsc_cqwq *wq,
+ struct xsc_wq_ctrl *wq_ctrl)
+{
+ u8 log_wq_stride = ele_log_size;
+ u8 log_wq_sz = q_log_size;
+ int err;
+
+ err = xsc_core_frag_buf_alloc_node(xdev,
+ wq_get_byte_sz(log_wq_sz,
+ log_wq_stride),
+ &wq_ctrl->buf,
+ param->buf_numa_node);
+ if (err) {
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "xsc_core_frag_buf_alloc_node failed, %d\n", err);
+ return err;
+ }
+
+ xsc_init_fbc(wq_ctrl->buf.frags, log_wq_stride, log_wq_sz, &wq->fbc);
+ wq_ctrl->xdev = xdev;
+ return 0;
+}
+
+int xsc_eth_wq_cyc_create(struct xsc_core_device *xdev,
+ struct xsc_wq_param *param,
+ u8 q_log_size,
+ u8 ele_log_size,
+ struct xsc_wq_cyc *wq,
+ struct xsc_wq_ctrl *wq_ctrl)
+{
+ struct xsc_frag_buf_ctrl *fbc = &wq->fbc;
+ u8 log_wq_stride = ele_log_size;
+ u8 log_wq_sz = q_log_size;
+ int err;
+
+ err = xsc_core_frag_buf_alloc_node(xdev,
+ wq_get_byte_sz(log_wq_sz,
+ log_wq_stride),
+ &wq_ctrl->buf,
+ param->buf_numa_node);
+ if (err) {
+ netdev_err(xsc_dev_to_netdev(xdev),
+ "xsc_core_frag_buf_alloc_node failed, %d\n", err);
+ return err;
+ }
+
+ xsc_init_fbc(wq_ctrl->buf.frags, log_wq_stride, log_wq_sz, fbc);
+ wq->sz = xsc_wq_cyc_get_size(wq);
+ wq_ctrl->xdev = xdev;
+ return 0;
+}
+
+void xsc_eth_wq_destroy(struct xsc_wq_ctrl *wq_ctrl)
+{
+ xsc_core_frag_buf_free(wq_ctrl->xdev, &wq_ctrl->buf);
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.h
new file mode 100644
index 0000000000000..01cee9fd1d9de
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_eth_wq.h
@@ -0,0 +1,185 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd. All
+ * rights reserved.
+ * Copyright (c) 2013-2015, Mellanox Technologies, Ltd. All rights reserved.
+ */
+
+#ifndef __XSC_WQ_H
+#define __XSC_WQ_H
+
+#include "common/xsc_core.h"
+
+struct xsc_wq_param {
+ int buf_numa_node;
+ int db_numa_node;
+};
+
+struct xsc_wq_ctrl {
+ struct xsc_core_device *xdev;
+ struct xsc_frag_buf buf;
+};
+
+struct xsc_wq_cyc {
+ struct xsc_frag_buf_ctrl fbc;
+ u16 sz;
+ u16 wqe_ctr;
+ u16 cur_sz;
+};
+
+struct xsc_cqwq {
+ struct xsc_frag_buf_ctrl fbc;
+ __be32 *db;
+ u32 cc; /* consumer counter */
+};
+
+enum xsc_res_type {
+ XSC_RES_RQ,
+ XSC_RES_SQ,
+};
+
+u32 xsc_wq_cyc_get_size(struct xsc_wq_cyc *wq);
+
+/*api for eth driver*/
+int xsc_eth_cqwq_create(struct xsc_core_device *xdev,
+ struct xsc_wq_param *param,
+ u8 q_log_size,
+ u8 ele_log_size,
+ struct xsc_cqwq *wq,
+ struct xsc_wq_ctrl *wq_ctrl);
+
+int xsc_eth_wq_cyc_create(struct xsc_core_device *xdev,
+ struct xsc_wq_param *param,
+ u8 q_log_size,
+ u8 ele_log_size,
+ struct xsc_wq_cyc *wq,
+ struct xsc_wq_ctrl *wq_ctrl);
+void xsc_eth_wq_destroy(struct xsc_wq_ctrl *wq_ctrl);
+
+static inline void xsc_init_fbc_offset(struct xsc_buf_list *frags,
+ u8 log_stride, u8 log_sz,
+ u16 strides_offset,
+ struct xsc_frag_buf_ctrl *fbc)
+{
+ fbc->frags = frags;
+ fbc->log_stride = log_stride;
+ fbc->log_sz = log_sz;
+ fbc->sz_m1 = (1 << fbc->log_sz) - 1;
+ fbc->log_frag_strides = PAGE_SHIFT - fbc->log_stride;
+ fbc->frag_sz_m1 = (1 << fbc->log_frag_strides) - 1;
+ fbc->strides_offset = strides_offset;
+}
+
+static inline void xsc_init_fbc(struct xsc_buf_list *frags,
+ u8 log_stride, u8 log_sz,
+ struct xsc_frag_buf_ctrl *fbc)
+{
+ xsc_init_fbc_offset(frags, log_stride, log_sz, 0, fbc);
+}
+
+static inline void *xsc_frag_buf_get_wqe(struct xsc_frag_buf_ctrl *fbc,
+ u32 ix)
+{
+ unsigned int frag;
+
+ ix += fbc->strides_offset;
+ frag = ix >> fbc->log_frag_strides;
+
+ return fbc->frags[frag].buf +
+ ((fbc->frag_sz_m1 & ix) << fbc->log_stride);
+}
+
+static inline u32
+xsc_frag_buf_get_idx_last_contig_stride(struct xsc_frag_buf_ctrl *fbc, u32 ix)
+{
+ u32 last_frag_stride_idx = (ix + fbc->strides_offset) | fbc->frag_sz_m1;
+
+ return min_t(u32, last_frag_stride_idx - fbc->strides_offset,
+ fbc->sz_m1);
+}
+
+static inline int xsc_wq_cyc_missing(struct xsc_wq_cyc *wq)
+{
+ return wq->sz - wq->cur_sz;
+}
+
+static inline int xsc_wq_cyc_is_empty(struct xsc_wq_cyc *wq)
+{
+ return !wq->cur_sz;
+}
+
+static inline void xsc_wq_cyc_push(struct xsc_wq_cyc *wq)
+{
+ wq->wqe_ctr++;
+ wq->cur_sz++;
+}
+
+static inline void xsc_wq_cyc_push_n(struct xsc_wq_cyc *wq, u8 n)
+{
+ wq->wqe_ctr += n;
+ wq->cur_sz += n;
+}
+
+static inline void xsc_wq_cyc_pop(struct xsc_wq_cyc *wq)
+{
+ wq->cur_sz--;
+}
+
+static inline u16 xsc_wq_cyc_ctr2ix(struct xsc_wq_cyc *wq, u16 ctr)
+{
+ return ctr & wq->fbc.sz_m1;
+}
+
+static inline u16 xsc_wq_cyc_get_head(struct xsc_wq_cyc *wq)
+{
+ return xsc_wq_cyc_ctr2ix(wq, wq->wqe_ctr);
+}
+
+static inline u16 xsc_wq_cyc_get_tail(struct xsc_wq_cyc *wq)
+{
+ return xsc_wq_cyc_ctr2ix(wq, wq->wqe_ctr - wq->cur_sz);
+}
+
+static inline void *xsc_wq_cyc_get_wqe(struct xsc_wq_cyc *wq, u16 ix)
+{
+ return xsc_frag_buf_get_wqe(&wq->fbc, ix);
+}
+
+static inline u32 xsc_cqwq_ctr2ix(struct xsc_cqwq *wq, u32 ctr)
+{
+ return ctr & wq->fbc.sz_m1;
+}
+
+static inline u32 xsc_cqwq_get_ci(struct xsc_cqwq *wq)
+{
+ return xsc_cqwq_ctr2ix(wq, wq->cc);
+}
+
+static inline u32 xsc_cqwq_get_ctr_wrap_cnt(struct xsc_cqwq *wq, u32 ctr)
+{
+ return ctr >> wq->fbc.log_sz;
+}
+
+static inline u32 xsc_cqwq_get_wrap_cnt(struct xsc_cqwq *wq)
+{
+ return xsc_cqwq_get_ctr_wrap_cnt(wq, wq->cc);
+}
+
+static inline void xsc_cqwq_pop(struct xsc_cqwq *wq)
+{
+ wq->cc++;
+}
+
+static inline u32 xsc_cqwq_get_size(struct xsc_cqwq *wq)
+{
+ return wq->fbc.sz_m1 + 1;
+}
+
+static inline struct xsc_cqe *xsc_cqwq_get_wqe(struct xsc_cqwq *wq, u32 ix)
+{
+ struct xsc_cqe *cqe = xsc_frag_buf_get_wqe(&wq->fbc, ix);
+
+ return cqe;
+}
+
+#endif /* __XSC_WQ_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pp.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pp.h
new file mode 100644
index 0000000000000..cc9eec866d523
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pp.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_PP_H
+#define __XSC_PP_H
+
+enum {
+ XSC_HASH_FIELD_SEL_SRC_IP = BIT(0),
+ XSC_HASH_FIELD_SEL_PROTO = BIT(1),
+ XSC_HASH_FIELD_SEL_DST_IP = BIT(2),
+ XSC_HASH_FIELD_SEL_SPORT = BIT(3),
+ XSC_HASH_FIELD_SEL_DPORT = BIT(4),
+ XSC_HASH_FIELD_SEL_SRC_IPV6 = BIT(5),
+ XSC_HASH_FIELD_SEL_DST_IPV6 = BIT(6),
+ XSC_HASH_FIELD_SEL_SPORT_V6 = BIT(7),
+ XSC_HASH_FIELD_SEL_DPORT_V6 = BIT(8),
+};
+
+#define XSC_HASH_IP (XSC_HASH_FIELD_SEL_SRC_IP |\
+ XSC_HASH_FIELD_SEL_DST_IP |\
+ XSC_HASH_FIELD_SEL_PROTO)
+#define XSC_HASH_IP_PORTS (XSC_HASH_FIELD_SEL_SRC_IP |\
+ XSC_HASH_FIELD_SEL_DST_IP |\
+ XSC_HASH_FIELD_SEL_SPORT |\
+ XSC_HASH_FIELD_SEL_DPORT |\
+ XSC_HASH_FIELD_SEL_PROTO)
+#define XSC_HASH_IP6 (XSC_HASH_FIELD_SEL_SRC_IPV6 |\
+ XSC_HASH_FIELD_SEL_DST_IPV6 |\
+ XSC_HASH_FIELD_SEL_PROTO)
+#define XSC_HASH_IP6_PORTS (XSC_HASH_FIELD_SEL_SRC_IPV6 |\
+ XSC_HASH_FIELD_SEL_DST_IPV6 |\
+ XSC_HASH_FIELD_SEL_SPORT_V6 |\
+ XSC_HASH_FIELD_SEL_DPORT_V6 |\
+ XSC_HASH_FIELD_SEL_PROTO)
+
+#endif /* __XSC_PP_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pph.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pph.h
new file mode 100644
index 0000000000000..cc544df2cfce4
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_pph.h
@@ -0,0 +1,42 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_PPH_H
+#define __XSC_PPH_H
+
+#define XSC_PPH_HEAD_LEN 64
+
+#define OUTER_L3_BIT BIT(3)
+#define OUTER_L4_BIT BIT(2)
+#define INNER_L3_BIT BIT(1)
+#define INNER_L4_BIT BIT(0)
+#define OUTER_BIT (OUTER_L3_BIT | OUTER_L4_BIT)
+#define INNER_BIT (INNER_L3_BIT | INNER_L4_BIT)
+#define OUTER_AND_INNER (OUTER_BIT | INNER_BIT)
+
+#define PACKET_UNKNOWN BIT(4)
+
+#define EPP2SOC_PPH_EXT_TUNNEL_TYPE_OFFSET (6UL)
+#define EPP2SOC_PPH_EXT_TUNNEL_TYPE_BIT_MASK (0XF00)
+#define EPP2SOC_PPH_EXT_TUNNEL_TYPE_BIT_OFFSET (8)
+
+#define EPP2SOC_PPH_EXT_ERROR_BITMAP_OFFSET (20UL)
+#define EPP2SOC_PPH_EXT_ERROR_BITMAP_BIT_MASK (0XFF)
+#define EPP2SOC_PPH_EXT_ERROR_BITMAP_BIT_OFFSET (0)
+
+#define XSC_GET_EPP2SOC_PPH_EXT_TUNNEL_TYPE(PPH_BASE_ADDR) \
+ ((*(u16 *)((u8 *)(PPH_BASE_ADDR) + \
+ EPP2SOC_PPH_EXT_TUNNEL_TYPE_OFFSET) & \
+ EPP2SOC_PPH_EXT_TUNNEL_TYPE_BIT_MASK) >> \
+ EPP2SOC_PPH_EXT_TUNNEL_TYPE_BIT_OFFSET)
+
+#define XSC_GET_EPP2SOC_PPH_ERROR_BITMAP(PPH_BASE_ADDR) \
+ ((*(u8 *)((u8 *)(PPH_BASE_ADDR) + \
+ EPP2SOC_PPH_EXT_ERROR_BITMAP_OFFSET) & \
+ EPP2SOC_PPH_EXT_ERROR_BITMAP_BIT_MASK) >> \
+ EPP2SOC_PPH_EXT_ERROR_BITMAP_BIT_OFFSET)
+
+#endif /* __XSC_PPH_H */
+
diff --git a/drivers/net/ethernet/yunsilicon/xsc/net/xsc_queue.h b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_queue.h
new file mode 100644
index 0000000000000..49c3229e823e6
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/net/xsc_queue.h
@@ -0,0 +1,185 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ * Copyright (c) 2015-2016, Mellanox Technologies. All rights reserved.
+ */
+
+#ifndef __XSC_QUEUE_H
+#define __XSC_QUEUE_H
+
+#include <net/page_pool/types.h>
+#include <net/page_pool/helpers.h>
+
+#include "common/xsc_core.h"
+#include "xsc_eth_wq.h"
+
+enum {
+ XSC_SEND_WQE_DS = 16,
+ XSC_SEND_WQE_BB = 64,
+};
+
+enum {
+ XSC_RECV_WQE_DS = 16,
+ XSC_RECV_WQE_BB = 16,
+};
+
+enum {
+ XSC_ETH_RQ_STATE_ENABLED,
+};
+
+#define XSC_SEND_WQEBB_CTRL_NUM_DS 1
+
+/* each ds holds one fragment in skb */
+#define XSC_MAX_RX_FRAGS 4
+#define XSC_RX_FRAG_SZ_ORDER 0
+#define XSC_RX_FRAG_SZ (PAGE_SIZE << XSC_RX_FRAG_SZ_ORDER)
+#define DEFAULT_FRAG_SIZE (2048)
+
+enum {
+ XSC_ETH_SQ_STATE_ENABLED,
+};
+
+struct xsc_cq {
+ /* data path - accessed per cqe */
+ struct xsc_cqwq wq;
+
+ /* data path - accessed per napi poll */
+ u16 event_ctr;
+ struct napi_struct *napi;
+ struct xsc_core_cq xcq;
+ struct xsc_channel *channel;
+
+ /* control */
+ struct xsc_core_device *xdev;
+ struct xsc_wq_ctrl wq_ctrl;
+} ____cacheline_aligned_in_smp;
+
+struct xsc_wqe_frag_info {
+ struct xsc_dma_info *di;
+ u32 offset;
+ u8 last_in_page;
+ u8 is_available;
+};
+
+struct xsc_dma_info {
+ struct page *page;
+ dma_addr_t addr;
+};
+
+struct xsc_rq_frag_info {
+ int frag_size;
+ int frag_stride;
+};
+
+struct xsc_rq_frags_info {
+ struct xsc_rq_frag_info arr[XSC_MAX_RX_FRAGS];
+ u8 num_frags;
+ u8 log_num_frags;
+ u8 wqe_bulk;
+ u8 wqe_bulk_min;
+ u8 frags_max_num;
+};
+
+struct xsc_rq;
+typedef void (*xsc_fp_handle_rx_cqe)(struct xsc_cqwq *cqwq, struct xsc_rq *rq,
+ struct xsc_cqe *cqe);
+typedef bool (*xsc_fp_post_rx_wqes)(struct xsc_rq *rq);
+typedef void (*xsc_fp_dealloc_wqe)(struct xsc_rq *rq, u16 ix);
+typedef struct sk_buff * (*xsc_fp_skb_from_cqe)(struct xsc_rq *rq,
+ struct xsc_wqe_frag_info *wi,
+ u32 cqe_bcnt,
+ u8 has_pph);
+
+struct xsc_rq {
+ struct xsc_core_qp cqp;
+ struct {
+ struct xsc_wq_cyc wq;
+ struct xsc_wqe_frag_info *frags;
+ struct xsc_dma_info *di;
+ struct xsc_rq_frags_info info;
+ xsc_fp_skb_from_cqe skb_from_cqe;
+ } wqe;
+
+ struct {
+ u16 headroom;
+ u8 map_dir; /* dma map direction */
+ } buff;
+
+ struct page_pool *page_pool;
+ struct xsc_wq_ctrl wq_ctrl;
+ struct xsc_cq cq;
+ u32 rqn;
+ int ix;
+
+ unsigned long state;
+ struct xsc_rq_stats *stats;
+
+ u32 hw_mtu;
+ u32 frags_sz;
+
+ xsc_fp_handle_rx_cqe handle_rx_cqe;
+ xsc_fp_post_rx_wqes post_wqes;
+ xsc_fp_dealloc_wqe dealloc_wqe;
+} ____cacheline_aligned_in_smp;
+
+enum xsc_dma_map_type {
+ XSC_DMA_MAP_SINGLE,
+ XSC_DMA_MAP_PAGE
+};
+
+struct xsc_sq_dma {
+ dma_addr_t addr;
+ u32 size;
+ enum xsc_dma_map_type type;
+};
+
+struct xsc_tx_wqe_info {
+ struct sk_buff *skb;
+ u32 num_bytes;
+ u8 num_wqebbs;
+ u8 num_dma;
+};
+
+struct xsc_tx_wqe {
+ struct xsc_send_wqe_ctrl_seg ctrl;
+ struct xsc_wqe_data_seg data[];
+};
+
+struct xsc_sq {
+ struct xsc_core_qp cqp;
+ /* dirtied @completion */
+ u16 cc;
+ u32 dma_fifo_cc;
+
+ /* dirtied @xmit */
+ u16 pc ____cacheline_aligned_in_smp;
+ u32 dma_fifo_pc;
+
+ struct xsc_cq cq;
+
+ /* read only */
+ struct xsc_wq_cyc wq;
+ u32 dma_fifo_mask;
+ struct xsc_sq_stats *stats;
+ struct {
+ struct xsc_sq_dma *dma_fifo;
+ struct xsc_tx_wqe_info *wqe_info;
+ } db;
+ void __iomem *uar_map;
+ struct netdev_queue *txq;
+ u32 sqn;
+ u16 stop_room;
+
+ __be32 mkey_be;
+ unsigned long state;
+ unsigned int hw_mtu;
+
+ /* control path */
+ struct xsc_wq_ctrl wq_ctrl;
+ struct xsc_channel *channel;
+ int ch_ix;
+ int txq_ix;
+} ____cacheline_aligned_in_smp;
+
+#endif /* __XSC_QUEUE_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig b/drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig
new file mode 100644
index 0000000000000..1cc3f7b49d5a2
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/Kconfig
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+# Yunsilicon PCI configuration
+#
+
+config YUNSILICON_XSC_PCI
+ tristate "Yunsilicon XSC PCI driver"
+ depends on PCI
+ help
+ This driver is common for Yunsilicon XSC
+ ethernet and RDMA drivers.
+
+ To compile this driver as a module, choose M here. The module
+ will be called xsc_pci.
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
new file mode 100644
index 0000000000000..fe0f9d175ea22
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/Makefile
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+# All rights reserved.
+
+ccflags-y += -I$(srctree)/drivers/net/ethernet/yunsilicon/xsc
+
+obj-$(CONFIG_YUNSILICON_XSC_PCI) += xsc_pci.o
+
+xsc_pci-y := main.o cmdq.o hw.o qp.o cq.o pd.o mr.o srq.o link.o alloc.o eq.o pci_irq.o adev.o vport.o \
+ hal/andes_impl.o
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/adev.c b/drivers/net/ethernet/yunsilicon/xsc/pci/adev.c
new file mode 100644
index 0000000000000..c8e8f1a9ff5e3
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/adev.c
@@ -0,0 +1,142 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/auxiliary_bus.h>
+#include <linux/idr.h>
+
+#include "adev.h"
+
+static DEFINE_IDA(xsc_adev_ida);
+
+static const char * const xsc_adev_name[] = {
+ [XSC_ADEV_IDX_ETH] = XSC_ETH_ADEV_NAME,
+ [XSC_ADEV_IDX_IB] = XSC_IB_ADEV_NAME,
+};
+
+static void xsc_release_adev(struct device *dev)
+{
+ struct xsc_adev *xsc_adev =
+ container_of(dev, struct xsc_adev, adev.dev);
+
+ kfree(xsc_adev);
+}
+
+int xsc_adev_add(struct xsc_core_device *xdev, int idx)
+{
+ struct auxiliary_device *adev;
+ struct xsc_adev *xsc_adev;
+ int ret;
+
+ if (!xdev->xsc_adev_list)
+ return -ENODEV;
+
+ if (xdev->xsc_adev_list[idx])
+ return 0;
+
+ xsc_adev = kzalloc_obj(*xsc_adev, GFP_KERNEL);
+ if (!xsc_adev)
+ return -ENOMEM;
+
+ adev = &xsc_adev->adev;
+ adev->name = xsc_adev_name[idx];
+ adev->id = xdev->adev_id;
+ adev->dev.parent = &xdev->pdev->dev;
+ adev->dev.release = xsc_release_adev;
+ xsc_adev->xdev = xdev;
+ xsc_adev->idx = idx;
+
+ ret = auxiliary_device_init(adev);
+ if (ret) {
+ kfree(xsc_adev);
+ return ret;
+ }
+
+ ret = auxiliary_device_add(adev);
+ if (ret) {
+ auxiliary_device_uninit(adev);
+ return ret;
+ }
+
+ xdev->xsc_adev_list[idx] = xsc_adev;
+
+ return 0;
+}
+
+void xsc_adev_del(struct xsc_core_device *xdev, int idx)
+{
+ struct xsc_adev *xsc_adev;
+ struct auxiliary_device *adev;
+
+ if (!xdev->xsc_adev_list)
+ return;
+
+ xsc_adev = xdev->xsc_adev_list[idx];
+ if (!xsc_adev)
+ return;
+
+ /* Clear the slot before teardown: the driver's remove callback runs
+ * synchronously inside auxiliary_device_delete() and may re-enter
+ * xsc_adev_add() for this index.
+ */
+ xdev->xsc_adev_list[idx] = NULL;
+
+ adev = &xsc_adev->adev;
+ auxiliary_device_delete(adev);
+ auxiliary_device_uninit(adev);
+}
+
+int xsc_adev_init(struct xsc_core_device *xdev)
+{
+ struct xsc_adev **xsc_adev_list;
+ int adev_id;
+ int ret;
+
+ xsc_adev_list = kcalloc(ARRAY_SIZE(xsc_adev_name),
+ sizeof(struct xsc_adev *),
+ GFP_KERNEL);
+ if (!xsc_adev_list)
+ return -ENOMEM;
+ xdev->xsc_adev_list = xsc_adev_list;
+
+ adev_id = ida_alloc(&xsc_adev_ida, GFP_KERNEL);
+ if (adev_id < 0) {
+ ret = adev_id;
+ goto err_free_adev_list;
+ }
+ xdev->adev_id = adev_id;
+
+ ret = xsc_adev_add(xdev, XSC_ADEV_IDX_ETH);
+ if (ret)
+ goto err_dalloc_adev_id;
+
+ ret = xsc_adev_add(xdev, XSC_ADEV_IDX_IB);
+ if (ret)
+ goto err_unreg_eth_adev;
+
+ return 0;
+
+err_unreg_eth_adev:
+ xsc_adev_del(xdev, XSC_ADEV_IDX_ETH);
+err_dalloc_adev_id:
+ ida_free(&xsc_adev_ida, xdev->adev_id);
+err_free_adev_list:
+ kfree(xsc_adev_list);
+ xdev->xsc_adev_list = NULL;
+
+ return ret;
+}
+
+void xsc_adev_uninit(struct xsc_core_device *xdev)
+{
+ if (!xdev->xsc_adev_list)
+ return;
+
+ xsc_adev_del(xdev, XSC_ADEV_IDX_IB);
+ xsc_adev_del(xdev, XSC_ADEV_IDX_ETH);
+ ida_free(&xsc_adev_ida, xdev->adev_id);
+ kfree(xdev->xsc_adev_list);
+ xdev->xsc_adev_list = NULL;
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/adev.h b/drivers/net/ethernet/yunsilicon/xsc/pci/adev.h
new file mode 100644
index 0000000000000..fb5789ec42f91
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/adev.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_ADEV_H
+#define __XSC_ADEV_H
+
+#include "common/xsc_core.h"
+
+int xsc_adev_init(struct xsc_core_device *xdev);
+void xsc_adev_uninit(struct xsc_core_device *xdev);
+
+int xsc_adev_add(struct xsc_core_device *xdev, int idx);
+void xsc_adev_del(struct xsc_core_device *xdev, int idx);
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.c b/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.c
new file mode 100644
index 0000000000000..70176797a46fc
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.c
@@ -0,0 +1,237 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/errno.h>
+#include <linux/slab.h>
+#include <linux/mm.h>
+#include <linux/export.h>
+#include <linux/bitmap.h>
+#include <linux/dma-mapping.h>
+#include <linux/vmalloc.h>
+
+#include "alloc.h"
+
+/* Handling for queue buffers -- we allocate a bunch of memory and
+ * register it in a memory region at HCA virtual address 0. If the
+ * requested size is > max_direct, we split the allocation into
+ * multiple pages, so we don't require too much contiguous memory.
+ */
+int xsc_buf_alloc(struct xsc_core_device *xdev,
+ unsigned long size,
+ unsigned long max_direct,
+ struct xsc_buf *buf)
+{
+ dma_addr_t t;
+
+ buf->size = size;
+ if (size <= max_direct) {
+ buf->nbufs = 1;
+ buf->npages = 1;
+ buf->page_shift = get_order(size) + PAGE_SHIFT;
+ buf->direct.buf = dma_alloc_coherent(&xdev->pdev->dev,
+ size,
+ &t,
+ GFP_KERNEL);
+ if (!buf->direct.buf)
+ return -ENOMEM;
+
+ buf->direct.map = t;
+
+ while (t & GENMASK(buf->page_shift - 1, 0)) {
+ --buf->page_shift;
+ buf->npages *= 2;
+ }
+ } else {
+ struct page **pages;
+ int i;
+
+ buf->direct.buf = NULL;
+ buf->nbufs = DIV_ROUND_UP(size, PAGE_SIZE);
+ buf->npages = buf->nbufs;
+ buf->page_shift = PAGE_SHIFT;
+ buf->page_list = kcalloc(buf->nbufs, sizeof(*buf->page_list),
+ GFP_KERNEL);
+ if (!buf->page_list)
+ return -ENOMEM;
+
+ for (i = 0; i < buf->nbufs; i++) {
+ buf->page_list[i].buf =
+ dma_alloc_coherent(&xdev->pdev->dev, PAGE_SIZE,
+ &t, GFP_KERNEL);
+ if (!buf->page_list[i].buf)
+ goto err_free;
+
+ buf->page_list[i].map = t;
+ }
+
+ pages = kmalloc_array(buf->nbufs, sizeof(*pages), GFP_KERNEL);
+ if (!pages)
+ goto err_free;
+ for (i = 0; i < buf->nbufs; i++) {
+ void *addr = buf->page_list[i].buf;
+
+ if (is_vmalloc_addr(addr))
+ pages[i] = vmalloc_to_page(addr);
+ else
+ pages[i] = virt_to_page(addr);
+ }
+ buf->direct.buf = vmap(pages, buf->nbufs, VM_MAP, PAGE_KERNEL);
+ kfree(pages);
+ if (!buf->direct.buf)
+ goto err_free;
+ }
+
+ return 0;
+
+err_free:
+ xsc_buf_free(xdev, buf);
+
+ return -ENOMEM;
+}
+EXPORT_SYMBOL_GPL(xsc_buf_alloc);
+
+void xsc_buf_free(struct xsc_core_device *xdev, struct xsc_buf *buf)
+{
+ int i;
+
+ if (buf->nbufs == 1) {
+ dma_free_coherent(&xdev->pdev->dev, buf->size, buf->direct.buf,
+ buf->direct.map);
+ } else {
+ if (BITS_PER_LONG == 64 && buf->direct.buf)
+ vunmap(buf->direct.buf);
+
+ for (i = 0; i < buf->nbufs; i++)
+ if (buf->page_list[i].buf)
+ dma_free_coherent(&xdev->pdev->dev, PAGE_SIZE,
+ buf->page_list[i].buf,
+ buf->page_list[i].map);
+ kfree(buf->page_list);
+ }
+}
+EXPORT_SYMBOL_GPL(xsc_buf_free);
+
+void xsc_fill_page_array(struct xsc_buf *buf, __be64 *pas, unsigned int npages)
+{
+ unsigned int shift = PAGE_SHIFT - PAGE_SHIFT_4K;
+ unsigned int mask = (1 << shift) - 1;
+ u64 addr;
+ int i;
+
+ for (i = 0; i < npages; i++) {
+ if (buf->nbufs == 1)
+ addr = buf->direct.map + (i << PAGE_SHIFT_4K);
+ else
+ addr = buf->page_list[i >> shift].map +
+ ((i & mask) << PAGE_SHIFT_4K);
+
+ pas[i] = cpu_to_be64(addr);
+ }
+}
+EXPORT_SYMBOL_GPL(xsc_fill_page_array);
+
+void xsc_core_fill_page_frag_array(struct xsc_frag_buf *buf,
+ __be64 *pas,
+ unsigned int npages)
+{
+ int i;
+ dma_addr_t addr;
+ unsigned int shift = PAGE_SHIFT - PAGE_SHIFT_4K;
+ unsigned int mask = (1 << shift) - 1;
+
+ for (i = 0; i < npages; i++) {
+ addr = buf->frags[i >> shift].map +
+ ((i & mask) << PAGE_SHIFT_4K);
+ pas[i] = cpu_to_be64(addr);
+ }
+}
+EXPORT_SYMBOL_GPL(xsc_core_fill_page_frag_array);
+
+static void *xsc_dma_zalloc_coherent_node(struct xsc_core_device *xdev,
+ size_t size, dma_addr_t *dma_handle,
+ int node)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+ struct device *device = &xdev->pdev->dev;
+ int original_node;
+ void *cpu_handle;
+
+ /* WA for kernels that don't use numa_mem_id in alloc_pages_node */
+ if (node == NUMA_NO_NODE)
+ node = numa_mem_id();
+
+ mutex_lock(&dev_res->alloc_mutex);
+ original_node = dev_to_node(device);
+ set_dev_node(device, node);
+ cpu_handle = dma_alloc_coherent(device, size, dma_handle,
+ GFP_KERNEL);
+ set_dev_node(device, original_node);
+ mutex_unlock(&dev_res->alloc_mutex);
+ return cpu_handle;
+}
+
+int xsc_core_frag_buf_alloc_node(struct xsc_core_device *xdev,
+ unsigned long size,
+ struct xsc_frag_buf *buf,
+ int node)
+{
+ int i;
+
+ buf->size = size;
+ buf->npages = DIV_ROUND_UP(size, PAGE_SIZE);
+ buf->page_shift = PAGE_SHIFT;
+ buf->frags = kcalloc(buf->npages, sizeof(struct xsc_buf_list),
+ GFP_KERNEL);
+ if (!buf->frags)
+ goto err_out;
+
+ for (i = 0; i < buf->npages; i++) {
+ unsigned long frag_sz = min_t(unsigned long, size, PAGE_SIZE);
+ struct xsc_buf_list *frag = &buf->frags[i];
+
+ frag->buf = xsc_dma_zalloc_coherent_node(xdev, frag_sz,
+ &frag->map, node);
+ if (!frag->buf)
+ goto err_free_buf;
+ if (frag->map & ((1 << buf->page_shift) - 1)) {
+ dma_free_coherent(&xdev->pdev->dev, frag_sz,
+ buf->frags[i].buf, buf->frags[i].map);
+ pci_err(xdev->pdev, "unexpected map alignment: %pad, page_shift=%d\n",
+ &frag->map, buf->page_shift);
+ goto err_free_buf;
+ }
+ size -= frag_sz;
+ }
+
+ return 0;
+
+err_free_buf:
+ while (i--)
+ dma_free_coherent(&xdev->pdev->dev,
+ PAGE_SIZE,
+ buf->frags[i].buf,
+ buf->frags[i].map);
+ kfree(buf->frags);
+err_out:
+ return -ENOMEM;
+}
+EXPORT_SYMBOL_GPL(xsc_core_frag_buf_alloc_node);
+
+void xsc_core_frag_buf_free(struct xsc_core_device *xdev,
+ struct xsc_frag_buf *buf)
+{
+ unsigned long size = buf->size;
+ int i;
+
+ for (i = 0; i < buf->npages; i++) {
+ unsigned long frag_sz = min_t(unsigned long, size, PAGE_SIZE);
+
+ dma_free_coherent(&xdev->pdev->dev, frag_sz, buf->frags[i].buf,
+ buf->frags[i].map);
+ size -= frag_sz;
+ }
+ kfree(buf->frags);
+}
+EXPORT_SYMBOL_GPL(xsc_core_frag_buf_free);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.h b/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.h
new file mode 100644
index 0000000000000..aef39e63235bc
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/alloc.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_ALLOC_H
+#define __XSC_ALLOC_H
+
+#include "common/xsc_core.h"
+
+int xsc_buf_alloc(struct xsc_core_device *xdev,
+ unsigned long size,
+ unsigned long max_direct,
+ struct xsc_buf *buf);
+void xsc_buf_free(struct xsc_core_device *xdev, struct xsc_buf *buf);
+void xsc_fill_page_array(struct xsc_buf *buf, __be64 *pas, unsigned int npages);
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c
new file mode 100644
index 0000000000000..48832b26dca9e
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/cmdq.c
@@ -0,0 +1,1247 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ * Copyright (c) 2013-2016, Mellanox Technologies. All rights reserved.
+ */
+
+#include <linux/module.h>
+#include <linux/errno.h>
+#include <linux/pci.h>
+#include <linux/dma-mapping.h>
+#include <linux/slab.h>
+#include <linux/delay.h>
+#include <linux/random.h>
+#include <linux/kthread.h>
+#include <linux/io-mapping.h>
+
+#include "common/xsc_cmd_api.h"
+#include "common/xsc_cmd.h"
+#include "common/xsc_core.h"
+#include "hal/xsc_hal.h"
+
+enum {
+ CMD_IF_REV = 3,
+};
+
+enum {
+ NUM_LONG_LISTS = 2,
+ NUM_MED_LISTS = 64,
+ LONG_LIST_SIZE = (2ULL * 1024 * 1024 * 1024 / PAGE_SIZE) * 8 + 16 +
+ XSC_CMD_DATA_BLOCK_SIZE,
+ MED_LIST_SIZE = 16 + XSC_CMD_DATA_BLOCK_SIZE,
+};
+
+enum {
+ XSC_CMD_DELIVERY_STAT_OK = 0x0,
+ XSC_CMD_DELIVERY_STAT_SIGNAT_ERR = 0x1,
+ XSC_CMD_DELIVERY_STAT_TOK_ERR = 0x2,
+ XSC_CMD_DELIVERY_STAT_BAD_BLK_NUM_ERR = 0x3,
+ XSC_CMD_DELIVERY_STAT_OUT_PTR_ALIGN_ERR = 0x4,
+ XSC_CMD_DELIVERY_STAT_IN_PTR_ALIGN_ERR = 0x5,
+ XSC_CMD_DELIVERY_STAT_FW_ERR = 0x6,
+ XSC_CMD_DELIVERY_STAT_IN_LENGTH_ERR = 0x7,
+ XSC_CMD_DELIVERY_STAT_OUT_LENGTH_ERR = 0x8,
+ XSC_CMD_DELIVERY_STAT_RES_FLD_NOT_CLR_ERR = 0x9,
+ XSC_CMD_DELIVERY_STAT_CMD_DESCR_ERR = 0x10,
+};
+
+static struct xsc_cmd_work_ent *xsc_alloc_cmd(struct xsc_cmd *cmd,
+ struct xsc_cmd_msg *in,
+ struct xsc_rsp_msg *out)
+{
+ struct xsc_cmd_work_ent *ent;
+
+ ent = kzalloc_obj(*ent, GFP_KERNEL);
+ if (!ent)
+ return ERR_PTR(-ENOMEM);
+
+ ent->in = in;
+ ent->out = out;
+ ent->cmd = cmd;
+
+ return ent;
+}
+
+static void xsc_free_cmd(struct xsc_cmd_work_ent *ent)
+{
+ kfree(ent);
+}
+
+static u8 xsc_alloc_token(struct xsc_cmd *cmd)
+{
+ u8 token;
+
+ spin_lock(&cmd->token_lock);
+ token = cmd->token++ % 255 + 1;
+ spin_unlock(&cmd->token_lock);
+
+ return token;
+}
+
+static int xsc_alloc_ent(struct xsc_cmd *cmd)
+{
+ unsigned long flags;
+ int ret;
+
+ spin_lock_irqsave(&cmd->alloc_lock, flags);
+ ret = find_first_bit(&cmd->cmd_entry_mask, cmd->max_reg_cmds);
+ if (ret < cmd->max_reg_cmds)
+ clear_bit(ret, &cmd->cmd_entry_mask);
+ spin_unlock_irqrestore(&cmd->alloc_lock, flags);
+
+ return ret < cmd->max_reg_cmds ? ret : -ENOSPC;
+}
+
+static void xsc_free_ent(struct xsc_cmd *cmd, int idx)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&cmd->alloc_lock, flags);
+ set_bit(idx, &cmd->cmd_entry_mask);
+ spin_unlock_irqrestore(&cmd->alloc_lock, flags);
+}
+
+static struct xsc_cmd_layout *xsc_get_inst(struct xsc_cmd *cmd, int idx)
+{
+ return cmd->cmd_buf + (idx << cmd->log_stride);
+}
+
+static struct xsc_rsp_layout *xsc_get_cq_inst(struct xsc_cmd *cmd, int idx)
+{
+ return cmd->cq_buf + (idx << cmd->log_stride);
+}
+
+static u8 xsc_xor8_buf(void *buf, int len)
+{
+ u8 *ptr = buf;
+ u8 sum = 0;
+ int i;
+
+ for (i = 0; i < len; i++)
+ sum ^= ptr[i];
+
+ return sum;
+}
+
+static int xsc_verify_block_sig(struct xsc_cmd_prot_block *block)
+{
+ /* rsvd0 was set to 0xFF by fw */
+ if (xsc_xor8_buf(block->rsvd0,
+ sizeof(*block) - sizeof(block->data) - 1) != 0xff)
+ return -EINVAL;
+
+ if (xsc_xor8_buf(block, sizeof(*block)) != 0xff)
+ return -EINVAL;
+
+ return 0;
+}
+
+static void xsc_calc_block_sig(struct xsc_cmd_prot_block *block, u8 token)
+{
+ block->token = token;
+ block->ctrl_sig = ~xsc_xor8_buf(block->rsvd0,
+ sizeof(*block) - sizeof(block->data) - 2);
+ block->sig = ~xsc_xor8_buf(block, sizeof(*block) - 1);
+}
+
+static void xsc_calc_chain_sig(struct xsc_cmd_mailbox *head, u8 token)
+{
+ struct xsc_cmd_mailbox *next = head;
+
+ while (next) {
+ xsc_calc_block_sig(next->buf, token);
+ next = next->next;
+ }
+}
+
+static void xsc_set_signature(struct xsc_cmd_work_ent *ent)
+{
+ ent->lay->sig = ~xsc_xor8_buf(ent->lay, sizeof(*ent->lay));
+ xsc_calc_chain_sig(ent->in->next, ent->token);
+ xsc_calc_chain_sig(ent->out->next, ent->token);
+}
+
+static int xsc_verify_signature(struct xsc_cmd_work_ent *ent)
+{
+ struct xsc_cmd_mailbox *next = ent->out->next;
+ int err;
+ u8 sig;
+
+ sig = xsc_xor8_buf(ent->rsp_lay, sizeof(*ent->rsp_lay));
+ if (sig != 0xff)
+ return -EINVAL;
+
+ while (next) {
+ err = xsc_verify_block_sig(next->buf);
+ if (err)
+ return err;
+
+ next = next->next;
+ }
+
+ return 0;
+}
+
+static const char *xsc_command_str(int command)
+{
+ switch (command) {
+ case XSC_CMD_OP_QUERY_HCA_CAP:
+ return "QUERY_HCA_CAP";
+
+ case XSC_CMD_OP_QUERY_CMDQ_VERSION:
+ return "QUERY_CMDQ_VERSION";
+
+ case XSC_CMD_OP_FUNCTION_RESET:
+ return "FUNCTION_RESET";
+
+ case XSC_CMD_OP_DUMMY:
+ return "DUMMY_CMD";
+
+ case XSC_CMD_OP_CREATE_EQ:
+ return "CREATE_EQ";
+
+ case XSC_CMD_OP_DESTROY_EQ:
+ return "DESTROY_EQ";
+
+ case XSC_CMD_OP_CREATE_CQ:
+ return "CREATE_CQ";
+
+ case XSC_CMD_OP_DESTROY_CQ:
+ return "DESTROY_CQ";
+
+ case XSC_CMD_OP_CREATE_QP:
+ return "CREATE_QP";
+
+ case XSC_CMD_OP_DESTROY_QP:
+ return "DESTROY_QP";
+
+ case XSC_CMD_OP_RST2INIT_QP:
+ return "RST2INIT_QP";
+
+ case XSC_CMD_OP_INIT2RTR_QP:
+ return "INIT2RTR_QP";
+
+ case XSC_CMD_OP_RTR2RTS_QP:
+ return "RTR2RTS_QP";
+
+ case XSC_CMD_OP_RTS2RTS_QP:
+ return "RTS2RTS_QP";
+
+ case XSC_CMD_OP_SQERR2RTS_QP:
+ return "SQERR2RTS_QP";
+
+ case XSC_CMD_OP_2ERR_QP:
+ return "2ERR_QP";
+
+ case XSC_CMD_OP_RTS2SQD_QP:
+ return "RTS2SQD_QP";
+
+ case XSC_CMD_OP_SQD2RTS_QP:
+ return "SQD2RTS_QP";
+
+ case XSC_CMD_OP_2RST_QP:
+ return "2RST_QP";
+
+ case XSC_CMD_OP_INIT2INIT_QP:
+ return "INIT2INIT_QP";
+
+ case XSC_CMD_OP_MODIFY_RAW_QP:
+ return "MODIFY_RAW_QP";
+
+ case XSC_CMD_OP_ENABLE_NIC_HCA:
+ return "ENABLE_NIC_HCA";
+
+ case XSC_CMD_OP_DISABLE_NIC_HCA:
+ return "DISABLE_NIC_HCA";
+
+ case XSC_CMD_OP_QUERY_VPORT_STATE:
+ return "QUERY_VPORT_STATE";
+
+ case XSC_CMD_OP_QUERY_EVENT_TYPE:
+ return "QUERY_EVENT_TYPE";
+
+ case XSC_CMD_OP_ENABLE_MSIX:
+ return "ENABLE_MSIX";
+
+ case XSC_CMD_OP_SET_MTU:
+ return "SET_MTU";
+
+ case XSC_CMD_OP_QUERY_ETH_MAC:
+ return "QUERY_ETH_MAC";
+
+ default: return "unknown command opcode";
+ }
+}
+
+static void cmd_work_handler(struct work_struct *work)
+{
+ struct xsc_cmd_work_ent *ent;
+ struct xsc_core_device *xdev;
+ struct xsc_cmd_layout *lay;
+ struct semaphore *sem;
+ struct xsc_cmd *cmd;
+ unsigned long flags;
+
+ ent = container_of(work, struct xsc_cmd_work_ent, work);
+ cmd = ent->cmd;
+ xdev = container_of(cmd, struct xsc_core_device, cmd);
+ sem = &cmd->sem;
+ down(sem);
+ ent->idx = xsc_alloc_ent(cmd);
+ if (ent->idx < 0) {
+ pci_err(xdev->pdev, "failed to allocate command entry\n");
+ up(sem);
+ return;
+ }
+
+ ent->token = xsc_alloc_token(cmd);
+ cmd->ent_arr[ent->idx] = ent;
+
+ spin_lock_irqsave(&cmd->doorbell_lock, flags);
+ lay = xsc_get_inst(cmd, cmd->cmd_pid);
+ ent->lay = lay;
+ memset(lay, 0, sizeof(*lay));
+ memcpy(lay->in, ent->in->first.data, sizeof(lay->in));
+ if (ent->in->next)
+ lay->in_ptr = cpu_to_be64(ent->in->next->dma);
+ lay->inlen = cpu_to_be32(ent->in->len);
+ if (ent->out->next)
+ lay->out_ptr = cpu_to_be64(ent->out->next->dma);
+ lay->outlen = cpu_to_be32(ent->out->len);
+ lay->type = XSC_PCI_CMD_XPORT;
+ lay->token = ent->token;
+ lay->idx = ent->idx;
+ if (!cmd->checksum_disabled)
+ xsc_set_signature(ent);
+ else
+ lay->sig = 0xff;
+
+ /* ring doorbell after the descriptor is valid */
+ wmb();
+
+ cmd->cmd_pid = (cmd->cmd_pid + 1) % (1 << cmd->log_sz);
+ xsc_update_cmdq_req_pid(xdev, cmd->cmd_pid);
+ spin_unlock_irqrestore(&cmd->doorbell_lock, flags);
+}
+
+static const char *xsc_deliv_status_to_str(u8 status)
+{
+ switch (status) {
+ case XSC_CMD_DELIVERY_STAT_OK:
+ return "no errors";
+ case XSC_CMD_DELIVERY_STAT_SIGNAT_ERR:
+ return "signature error";
+ case XSC_CMD_DELIVERY_STAT_TOK_ERR:
+ return "token error";
+ case XSC_CMD_DELIVERY_STAT_BAD_BLK_NUM_ERR:
+ return "bad block number";
+ case XSC_CMD_DELIVERY_STAT_OUT_PTR_ALIGN_ERR:
+ return "output pointer not aligned to block size";
+ case XSC_CMD_DELIVERY_STAT_IN_PTR_ALIGN_ERR:
+ return "input pointer not aligned to block size";
+ case XSC_CMD_DELIVERY_STAT_FW_ERR:
+ return "firmware internal error";
+ case XSC_CMD_DELIVERY_STAT_IN_LENGTH_ERR:
+ return "command input length error";
+ case XSC_CMD_DELIVERY_STAT_OUT_LENGTH_ERR:
+ return "command output length error";
+ case XSC_CMD_DELIVERY_STAT_RES_FLD_NOT_CLR_ERR:
+ return "reserved fields not cleared";
+ case XSC_CMD_DELIVERY_STAT_CMD_DESCR_ERR:
+ return "bad command descriptor type";
+ default:
+ return "unknown status code";
+ }
+}
+
+static u16 xsc_msg_to_opcode(struct xsc_cmd_msg *in)
+{
+ struct xsc_inbox_hdr *hdr = (struct xsc_inbox_hdr *)(in->first.data);
+
+ return be16_to_cpu(hdr->opcode);
+}
+
+static int xsc_wait_func(struct xsc_core_device *xdev,
+ struct xsc_cmd_work_ent *ent)
+{
+ unsigned long timeout = msecs_to_jiffies(XSC_CMD_TIMEOUT_MSEC);
+ struct xsc_cmd *cmd = &xdev->cmd;
+ int err;
+
+ if (!wait_for_completion_timeout(&ent->done, timeout))
+ err = -ETIMEDOUT;
+ else
+ err = ent->ret;
+
+ if (err == -ETIMEDOUT) {
+ cmd->cmd_status = XSC_CMD_STATUS_TIMEDOUT;
+ pci_err(xdev->pdev, "wait for %s(0x%x) response timeout!\n",
+ xsc_command_str(xsc_msg_to_opcode(ent->in)),
+ xsc_msg_to_opcode(ent->in));
+ } else if (err) {
+ pci_err(xdev->pdev, "err %d, delivery status %s(%d)\n", err,
+ xsc_deliv_status_to_str(ent->status), ent->status);
+ }
+
+ return err;
+}
+
+/* Notes:
+ * 1. Callback functions may not sleep
+ * 2. page queue commands do not support asynchrous completion
+ */
+static int xsc_cmd_invoke(struct xsc_core_device *xdev, struct xsc_cmd_msg *in,
+ struct xsc_rsp_msg *out, u8 *status)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct xsc_cmd_work_ent *ent;
+ struct semaphore *sem;
+ int err = 0;
+
+ ent = xsc_alloc_cmd(cmd, in, out);
+ if (IS_ERR(ent))
+ return PTR_ERR(ent);
+
+ init_completion(&ent->done);
+ INIT_WORK(&ent->work, cmd_work_handler);
+ if (!queue_work(cmd->wq, &ent->work)) {
+ pci_err(xdev->pdev, "failed to queue work\n");
+ err = -ENOMEM;
+ goto err_free;
+ }
+
+ err = xsc_wait_func(xdev, ent);
+ if (err == -ETIMEDOUT)
+ goto err_sem_up;
+ *status = ent->status;
+ xsc_free_cmd(ent);
+
+ return err;
+
+err_sem_up:
+ sem = &cmd->sem;
+ up(sem);
+err_free:
+ xsc_free_cmd(ent);
+ return err;
+}
+
+static int xsc_copy_to_cmd_msg(struct xsc_cmd_msg *to, void *from,
+ unsigned int size)
+{
+ struct xsc_cmd_prot_block *block;
+ struct xsc_cmd_mailbox *next;
+ unsigned int copy;
+
+ if (!to || !from)
+ return -ENOMEM;
+
+ copy = min(size, sizeof(to->first.data));
+ memcpy(to->first.data, from, copy);
+ size -= copy;
+ from += copy;
+
+ next = to->next;
+ while (size) {
+ if (!next) {
+ WARN_ONCE(1, "Mail box not enough\n");
+ return -ENOMEM;
+ }
+
+ copy = min(size, XSC_CMD_DATA_BLOCK_SIZE);
+ block = next->buf;
+ memcpy(block->data, from, copy);
+ block->owner_status = 0;
+ from += copy;
+ size -= copy;
+ next = next->next;
+ }
+
+ return 0;
+}
+
+static int xsc_copy_from_rsp_msg(void *to, struct xsc_rsp_msg *from,
+ unsigned int size)
+{
+ struct xsc_cmd_prot_block *block;
+ struct xsc_cmd_mailbox *next;
+ unsigned int copy;
+
+ if (!to || !from)
+ return -ENOMEM;
+
+ copy = min(size, sizeof(from->first.data));
+ memcpy(to, from->first.data, copy);
+ size -= copy;
+ to += copy;
+
+ next = from->next;
+ while (size) {
+ if (!next) {
+ WARN_ONCE(1, "Mail too short\n");
+ return -ENOMEM;
+ }
+
+ copy = min(size, XSC_CMD_DATA_BLOCK_SIZE);
+ block = next->buf;
+ if (!block->owner_status)
+ pr_err("block ownership check failed\n");
+
+ memcpy(to, block->data, copy);
+ to += copy;
+ size -= copy;
+ next = next->next;
+ }
+
+ return 0;
+}
+
+static struct xsc_cmd_mailbox *xsc_alloc_cmd_box(struct xsc_core_device *xdev,
+ gfp_t flags)
+{
+ struct xsc_cmd_mailbox *mailbox;
+
+ mailbox = kmalloc_obj(*mailbox, flags);
+ if (!mailbox)
+ return ERR_PTR(-ENOMEM);
+
+ mailbox->buf = dma_pool_zalloc(xdev->cmd.pool, flags,
+ &mailbox->dma);
+ if (!mailbox->buf) {
+ kfree(mailbox);
+ return ERR_PTR(-ENOMEM);
+ }
+ mailbox->next = NULL;
+
+ return mailbox;
+}
+
+static void xsc_free_cmd_box(struct xsc_core_device *xdev,
+ struct xsc_cmd_mailbox *mailbox)
+{
+ dma_pool_free(xdev->cmd.pool, mailbox->buf, mailbox->dma);
+
+ kfree(mailbox);
+}
+
+static struct xsc_cmd_msg *xsc_alloc_cmd_msg(struct xsc_core_device *xdev,
+ gfp_t flags, unsigned int size)
+{
+ struct xsc_cmd_mailbox *tmp, *head = NULL;
+ struct xsc_cmd_prot_block *block;
+ struct xsc_cmd_msg *msg;
+ int blen;
+ int err;
+ int n;
+ int i;
+
+ msg = kzalloc_obj(*msg, GFP_KERNEL);
+ if (!msg)
+ return ERR_PTR(-ENOMEM);
+
+ blen = size - min(sizeof(msg->first.data), size);
+ n = (blen + XSC_CMD_DATA_BLOCK_SIZE - 1) / XSC_CMD_DATA_BLOCK_SIZE;
+
+ for (i = 0; i < n; i++) {
+ tmp = xsc_alloc_cmd_box(xdev, flags);
+ if (IS_ERR(tmp)) {
+ pci_err(xdev->pdev, "failed allocating block\n");
+ err = PTR_ERR(tmp);
+ goto err_free;
+ }
+
+ block = tmp->buf;
+ tmp->next = head;
+ block->next = cpu_to_be64(tmp->next ? tmp->next->dma : 0);
+ block->block_num = cpu_to_be32(n - i - 1);
+ head = tmp;
+ }
+ msg->next = head;
+ msg->len = size;
+ return msg;
+
+err_free:
+ while (head) {
+ tmp = head->next;
+ xsc_free_cmd_box(xdev, head);
+ head = tmp;
+ }
+ kfree(msg);
+
+ return ERR_PTR(err);
+}
+
+static void xsc_free_cmd_msg(struct xsc_core_device *xdev,
+ struct xsc_cmd_msg *msg)
+{
+ struct xsc_cmd_mailbox *head = msg->next;
+ struct xsc_cmd_mailbox *next;
+
+ while (head) {
+ next = head->next;
+ xsc_free_cmd_box(xdev, head);
+ head = next;
+ }
+ kfree(msg);
+}
+
+static struct xsc_rsp_msg *xsc_alloc_rsp_msg(struct xsc_core_device *xdev,
+ gfp_t flags, unsigned int size)
+{
+ struct xsc_cmd_mailbox *tmp, *head = NULL;
+ struct xsc_cmd_prot_block *block;
+ struct xsc_rsp_msg *msg;
+ int blen;
+ int err;
+ int n;
+ int i;
+
+ msg = kzalloc_obj(*msg, GFP_KERNEL);
+ if (!msg)
+ return ERR_PTR(-ENOMEM);
+
+ blen = size - min_t(int, sizeof(msg->first.data), size);
+ n = (blen + XSC_CMD_DATA_BLOCK_SIZE - 1) / XSC_CMD_DATA_BLOCK_SIZE;
+
+ for (i = 0; i < n; i++) {
+ tmp = xsc_alloc_cmd_box(xdev, flags);
+ if (IS_ERR(tmp)) {
+ pci_err(xdev->pdev, "failed allocating block\n");
+ err = PTR_ERR(tmp);
+ goto err_free;
+ }
+
+ block = tmp->buf;
+ tmp->next = head;
+ block->next = cpu_to_be64(tmp->next ? tmp->next->dma : 0);
+ block->block_num = cpu_to_be32(n - i - 1);
+ head = tmp;
+ }
+ msg->next = head;
+ msg->len = size;
+ return msg;
+
+err_free:
+ while (head) {
+ tmp = head->next;
+ xsc_free_cmd_box(xdev, head);
+ head = tmp;
+ }
+ kfree(msg);
+
+ return ERR_PTR(err);
+}
+
+static void xsc_free_rsp_msg(struct xsc_core_device *xdev,
+ struct xsc_rsp_msg *msg)
+{
+ struct xsc_cmd_mailbox *head = msg->next;
+ struct xsc_cmd_mailbox *next;
+
+ while (head) {
+ next = head->next;
+ xsc_free_cmd_box(xdev, head);
+ head = next;
+ }
+ kfree(msg);
+}
+
+static void xsc_set_wqname(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+
+ snprintf(cmd->wq_name, sizeof(cmd->wq_name), "xsc_cmd_%s",
+ dev_name(&xdev->pdev->dev));
+}
+
+void xsc_cmd_use_events(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ int i;
+
+ for (i = 0; i < cmd->max_reg_cmds; i++)
+ down(&cmd->sem);
+
+ flush_workqueue(cmd->wq);
+
+ cmd->mode = XSC_CMD_MODE_EVENTS;
+
+ while (cmd->cmd_pid != cmd->cq_cid)
+ msleep(20);
+ kthread_stop(cmd->cq_task);
+ cmd->cq_task = NULL;
+
+ for (i = 0; i < cmd->max_reg_cmds; i++)
+ up(&cmd->sem);
+}
+
+static int xsc_cmd_cq_polling(void *data);
+void xsc_cmd_use_polling(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ int i;
+
+ for (i = 0; i < cmd->max_reg_cmds; i++)
+ down(&cmd->sem);
+
+ flush_workqueue(cmd->wq);
+ cmd->mode = XSC_CMD_MODE_POLLING;
+ cmd->cq_task = kthread_create(xsc_cmd_cq_polling,
+ (void *)xdev,
+ "xsc_cmd_cq_polling");
+ if (cmd->cq_task)
+ wake_up_process(cmd->cq_task);
+
+ for (i = 0; i < cmd->max_reg_cmds; i++)
+ up(&cmd->sem);
+}
+
+int xsc_cmd_status_to_err(struct xsc_outbox_hdr *hdr)
+{
+ switch (hdr->status) {
+ case XSC_CMD_STATUS_OK:
+ return 0;
+ case XSC_CMD_STATUS_FAIL:
+ return -EIO;
+ case XSC_CMD_STATUS_NOT_SUPPORTED:
+ return -EOPNOTSUPP;
+ case XSC_CMD_STATUS_BAD_PARAM:
+ return -EINVAL;
+ case XSC_CMD_STATUS_INVAL_RES:
+ case XSC_CMD_STATUS_INVAL_DATA:
+ case XSC_CMD_STATUS_NO_RES:
+ case XSC_CMD_STATUS_NO_MPT_RES:
+ case XSC_CMD_STATUS_NO_MTT_RES:
+ case XSC_CMD_STATUS_NO_EQN_RES:
+ case XSC_CMD_STATUS_NO_EQ_PA_RES:
+ case XSC_CMD_STATUS_NO_CQN_RES:
+ case XSC_CMD_STATUS_NO_CQ_PA_RES:
+ case XSC_CMD_STATUS_NO_QPN_RES:
+ case XSC_CMD_STATUS_NO_QP_PA_RES:
+ case XSC_CMD_STATUS_NO_PDN_RES:
+ case XSC_CMD_STATUS_BAD_INBUF:
+ case XSC_CMD_STATUS_BAD_OUTBUF:
+ return -EIO;
+ case XSC_CMD_STATUS_BUSY:
+ case XSC_CMD_STATUS_QP_FLUSH_BUSY:
+ return -EBUSY;
+ case XSC_CMD_STATUS_QP_FLUSH_PENDING:
+ return -EAGAIN;
+ case XSC_CMD_STATUS_NOT_FOUND:
+ case XSC_CMD_STATUS_INVAL_FUNC:
+ return -ENODEV;
+ default:
+ return -EIO;
+ }
+}
+
+static int xsc_status_to_err(u8 status)
+{
+ struct xsc_outbox_hdr hdr = { .status = status };
+
+ return xsc_cmd_status_to_err(&hdr);
+}
+
+static struct xsc_cmd_msg *xsc_alloc_msg(struct xsc_core_device *xdev,
+ unsigned int in_size)
+{
+ struct xsc_cmd_msg *msg = ERR_PTR(-ENOMEM);
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct cache_ent *ent = NULL;
+
+ if (in_size > MED_LIST_SIZE && in_size <= LONG_LIST_SIZE)
+ ent = &cmd->cache.large;
+ else if (in_size > 16 && in_size <= MED_LIST_SIZE)
+ ent = &cmd->cache.med;
+
+ if (ent) {
+ spin_lock(&ent->lock);
+ if (!list_empty(&ent->head)) {
+ msg = list_entry(ent->head.next, typeof(*msg), list);
+ /* For cached lists, we must explicitly state what is
+ * the real size
+ */
+ msg->len = in_size;
+ list_del(&msg->list);
+ }
+ spin_unlock(&ent->lock);
+ }
+
+ if (IS_ERR(msg))
+ msg = xsc_alloc_cmd_msg(xdev, GFP_KERNEL, in_size);
+
+ return msg;
+}
+
+static void xsc_free_msg(struct xsc_core_device *xdev, struct xsc_cmd_msg *msg)
+{
+ if (msg->cache) {
+ spin_lock(&msg->cache->lock);
+ list_add_tail(&msg->list, &msg->cache->head);
+ spin_unlock(&msg->cache->lock);
+ } else {
+ xsc_free_cmd_msg(xdev, msg);
+ }
+}
+
+static int _xsc_cmd_exec(struct xsc_core_device *xdev,
+ void *in, unsigned int in_size,
+ void *out, unsigned int out_size)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct xsc_rsp_msg *outb;
+ struct xsc_cmd_msg *inb;
+ u8 status = 0;
+ int err;
+
+ if (cmd->cmd_status == XSC_CMD_STATUS_TIMEDOUT)
+ return -ETIMEDOUT;
+
+ inb = xsc_alloc_msg(xdev, in_size);
+ if (IS_ERR(inb)) {
+ err = PTR_ERR(inb);
+ return err;
+ }
+
+ err = xsc_copy_to_cmd_msg(inb, in, in_size);
+ if (err) {
+ pci_err(xdev->pdev, "copy to cmd_msg err %d\n", err);
+ goto err_free_msg;
+ }
+
+ outb = xsc_alloc_rsp_msg(xdev, GFP_KERNEL, out_size);
+ if (IS_ERR(outb)) {
+ err = PTR_ERR(outb);
+ goto err_free_msg;
+ }
+
+ err = xsc_cmd_invoke(xdev, inb, outb, &status);
+ if (err)
+ goto err_free_rsp;
+
+ if (status) {
+ pci_err(xdev->pdev, "opcode:%#x, err %d, status %d\n",
+ xsc_msg_to_opcode(inb), err, status);
+ err = xsc_status_to_err(status);
+ goto err_free_rsp;
+ }
+
+ err = xsc_copy_from_rsp_msg(out, outb, out_size);
+
+err_free_rsp:
+ xsc_free_rsp_msg(xdev, outb);
+
+err_free_msg:
+ xsc_free_msg(xdev, inb);
+ return err;
+}
+
+int xsc_cmd_exec(struct xsc_core_device *xdev,
+ void *in, unsigned int in_size,
+ void *out, unsigned int out_size)
+{
+ struct xsc_outbox_hdr *out_hdr = (struct xsc_outbox_hdr *)out;
+ struct xsc_inbox_hdr *in_hdr = (struct xsc_inbox_hdr *)in;
+ int err;
+
+ in_hdr->ver = 0;
+ if (in_hdr->ver != 0) {
+ pci_err(xdev->pdev, "recv an unexpected cmd ver = %d, opcode = %d\n",
+ be16_to_cpu(in_hdr->ver), be16_to_cpu(in_hdr->opcode));
+ WARN_ON(in_hdr->ver != 0);
+ }
+
+ err = _xsc_cmd_exec(xdev, in, in_size, out, out_size);
+ if (err)
+ return err;
+ if (out_hdr->status) {
+ err = -EIO;
+ return err;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_cmd_exec);
+
+static void xsc_destroy_msg_cache(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct xsc_cmd_msg *msg;
+ struct xsc_cmd_msg *n;
+
+ list_for_each_entry_safe(msg, n, &cmd->cache.large.head, list) {
+ list_del(&msg->list);
+ xsc_free_cmd_msg(xdev, msg);
+ }
+
+ list_for_each_entry_safe(msg, n, &cmd->cache.med.head, list) {
+ list_del(&msg->list);
+ xsc_free_cmd_msg(xdev, msg);
+ }
+}
+
+static int xsc_create_msg_cache(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct xsc_cmd_msg *msg;
+ int err;
+ int i;
+
+ spin_lock_init(&cmd->cache.large.lock);
+ INIT_LIST_HEAD(&cmd->cache.large.head);
+ spin_lock_init(&cmd->cache.med.lock);
+ INIT_LIST_HEAD(&cmd->cache.med.head);
+
+ for (i = 0; i < NUM_LONG_LISTS; i++) {
+ msg = xsc_alloc_cmd_msg(xdev, GFP_KERNEL, LONG_LIST_SIZE);
+ if (IS_ERR(msg)) {
+ err = PTR_ERR(msg);
+ goto err_destroy;
+ }
+ msg->cache = &cmd->cache.large;
+ list_add_tail(&msg->list, &cmd->cache.large.head);
+ }
+
+ for (i = 0; i < NUM_MED_LISTS; i++) {
+ msg = xsc_alloc_cmd_msg(xdev, GFP_KERNEL, MED_LIST_SIZE);
+ if (IS_ERR(msg)) {
+ err = PTR_ERR(msg);
+ goto err_destroy;
+ }
+ msg->cache = &cmd->cache.med;
+ list_add_tail(&msg->list, &cmd->cache.med.head);
+ }
+
+ return 0;
+
+err_destroy:
+ xsc_destroy_msg_cache(xdev);
+ return err;
+}
+
+static void xsc_cmd_comp_handler(struct xsc_core_device *xdev, u8 idx,
+ struct xsc_rsp_layout *rsp)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct xsc_cmd_work_ent *ent;
+
+ if (idx > cmd->max_reg_cmds || (cmd->cmd_entry_mask & (1 << idx))) {
+ pci_err(xdev->pdev, "idx[%d] exceed max cmds, or has no relative request.\n",
+ idx);
+ return;
+ }
+ ent = cmd->ent_arr[idx];
+ ent->rsp_lay = rsp;
+
+ memcpy(ent->out->first.data,
+ ent->rsp_lay->out,
+ sizeof(ent->rsp_lay->out));
+ if (!cmd->checksum_disabled)
+ ent->ret = xsc_verify_signature(ent);
+ else
+ ent->ret = 0;
+ ent->status = 0;
+
+ xsc_free_ent(cmd, ent->idx);
+ complete(&ent->done);
+ up(&cmd->sem);
+}
+
+static int xsc_cmd_cq_polling(void *data)
+{
+ struct xsc_core_device *xdev = data;
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct xsc_rsp_layout *rsp;
+ u32 cq_pid;
+
+ while (!kthread_should_stop()) {
+ if (need_resched())
+ schedule();
+ cq_pid = xsc_get_cmdq_rsp_pid(xdev);
+ if (cmd->cq_cid == cq_pid) {
+ mdelay(3);
+ continue;
+ }
+
+ rsp = xsc_get_cq_inst(cmd, cmd->cq_cid);
+ if (!cmd->ownerbit_learned) {
+ cmd->ownerbit_learned = 1;
+ cmd->owner_bit = rsp->owner_bit;
+ }
+ if (cmd->owner_bit != rsp->owner_bit) {
+ pci_err(xdev->pdev, "hw update cq doorbell but buf not ready %u %u\n",
+ cmd->cq_cid, cq_pid);
+ continue;
+ }
+
+ xsc_cmd_comp_handler(xdev, rsp->idx, rsp);
+
+ cmd->cq_cid = (cmd->cq_cid + 1) % (1 << cmd->log_sz);
+
+ xsc_update_cmdq_rsp_cid(xdev, cmd->cq_cid);
+ if (cmd->cq_cid == 0)
+ cmd->owner_bit = !cmd->owner_bit;
+ }
+
+ return 0;
+}
+
+int xsc_cmd_err_handler(struct xsc_core_device *xdev)
+{
+ return xsc_handle_cmdq_interrupt(xdev);
+}
+
+void xsc_cmd_resp_handler(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+ struct xsc_rsp_layout *rsp;
+ const int budget = 32;
+ int count = 0;
+ u32 cq_pid;
+
+ while (count < budget) {
+ cq_pid = xsc_get_cmdq_rsp_pid(xdev);
+ if (cq_pid == cmd->cq_cid)
+ return;
+
+ rsp = xsc_get_cq_inst(cmd, cmd->cq_cid);
+ if (!cmd->ownerbit_learned) {
+ cmd->ownerbit_learned = 1;
+ cmd->owner_bit = rsp->owner_bit;
+ }
+ if (cmd->owner_bit != rsp->owner_bit) {
+ pci_err(xdev->pdev, "hw update cq doorbell but buf not ready %u %u\n",
+ cmd->cq_cid, cq_pid);
+ return;
+ }
+
+ xsc_cmd_comp_handler(xdev, rsp->idx, rsp);
+
+ cmd->cq_cid = (cmd->cq_cid + 1) % (1 << cmd->log_sz);
+ xsc_update_cmdq_rsp_cid(xdev, cmd->cq_cid);
+ if (cmd->cq_cid == 0)
+ cmd->owner_bit = !cmd->owner_bit;
+
+ count++;
+ }
+}
+
+static void xsc_cmd_handle_rsp_before_reload
+(struct xsc_cmd *cmd, struct xsc_core_device *xdev)
+{
+ u32 rsp_pid, rsp_cid;
+
+ rsp_pid = xsc_get_cmdq_rsp_pid(xdev);
+ rsp_cid = xsc_get_cmdq_rsp_cid(xdev);
+ if (rsp_pid == rsp_cid)
+ return;
+
+ cmd->cq_cid = rsp_pid;
+
+ xsc_update_cmdq_rsp_cid(xdev, cmd->cq_cid);
+}
+
+int xsc_cmd_init(struct xsc_core_device *xdev)
+{
+ unsigned int size = sizeof(struct xsc_cmd_prot_block);
+ int align = roundup_pow_of_two(size);
+ struct xsc_cmd *cmd = &xdev->cmd;
+ u32 cmd_h, cmd_l;
+ int err;
+
+ cmd->pool = dma_pool_create("xsc_cmd",
+ &xdev->pdev->dev,
+ size, align, 0);
+ if (!cmd->pool)
+ return -ENOMEM;
+
+ cmd->cmd_buf = (void *)__get_free_pages(GFP_ATOMIC, 0);
+ if (!cmd->cmd_buf) {
+ err = -ENOMEM;
+ goto err_free_pool;
+ }
+ cmd->cq_buf = (void *)__get_free_pages(GFP_ATOMIC, 0);
+ if (!cmd->cq_buf) {
+ err = -ENOMEM;
+ goto err_free_cmd;
+ }
+
+ memset(cmd->cmd_buf, 0, PAGE_SIZE);
+ memset(cmd->cq_buf, 0, PAGE_SIZE);
+
+ cmd->dma = dma_map_single(&xdev->pdev->dev, cmd->cmd_buf, PAGE_SIZE,
+ DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(&xdev->pdev->dev, cmd->dma)) {
+ err = -ENOMEM;
+ goto err_free_cq;
+ }
+
+ cmd->cq_dma = dma_map_single(&xdev->pdev->dev, cmd->cq_buf, PAGE_SIZE,
+ DMA_BIDIRECTIONAL);
+ if (dma_mapping_error(&xdev->pdev->dev, cmd->cq_dma)) {
+ err = -ENOMEM;
+ goto err_unmap_cmd;
+ }
+
+ cmd->cmd_pid = xsc_get_cmdq_req_pid(xdev);
+ cmd->cq_cid = xsc_get_cmdq_rsp_cid(xdev);
+ cmd->ownerbit_learned = 0;
+
+ xsc_cmd_handle_rsp_before_reload(cmd, xdev);
+
+#define ELEMENT_SIZE_LOG 6 /* 64B */
+#define Q_DEPTH_LOG 5 /* 32 */
+
+ cmd->log_sz = Q_DEPTH_LOG;
+ cmd->log_stride = ELEMENT_SIZE_LOG;
+
+ if (1 << cmd->log_sz > XSC_MAX_COMMANDS) {
+ dev_err(&xdev->pdev->dev, "firmware reports too many outstanding commands %d\n",
+ 1 << cmd->log_sz);
+ err = -EINVAL;
+ goto err_unmap_cq;
+ }
+
+ if (cmd->log_sz + cmd->log_stride > PAGE_SHIFT) {
+ dev_err(&xdev->pdev->dev, "command queue size overflow\n");
+ err = -EINVAL;
+ goto err_unmap_cq;
+ }
+
+ cmd->checksum_disabled = 1;
+ cmd->max_reg_cmds = (1 << cmd->log_sz) - 1;
+ cmd->cmd_entry_mask = (1 << cmd->max_reg_cmds) - 1;
+
+ spin_lock_init(&cmd->alloc_lock);
+ spin_lock_init(&cmd->token_lock);
+ spin_lock_init(&cmd->doorbell_lock);
+
+ sema_init(&cmd->sem, cmd->max_reg_cmds);
+
+ cmd_h = upper_32_bits(cmd->dma);
+ cmd_l = lower_32_bits(cmd->dma);
+ if (cmd_l & 0xfff) {
+ dev_err(&xdev->pdev->dev, "invalid command queue address\n");
+ err = -ENOMEM;
+ goto err_unmap_cq;
+ }
+
+ xsc_set_cmdq_req_buf_addr(xdev, cmd_h, cmd_l);
+
+ cmd_h = upper_32_bits(cmd->cq_dma);
+ cmd_l = lower_32_bits(cmd->cq_dma);
+ if (cmd_l & 0xfff) {
+ dev_err(&xdev->pdev->dev, "invalid command queue address\n");
+ err = -ENOMEM;
+ goto err_unmap_cq;
+ }
+ xsc_set_cmdq_rsp_buf_addr(xdev, cmd_h, cmd_l);
+
+ /* Make sure firmware sees the complete address before we proceed */
+ wmb();
+
+ cmd->mode = XSC_CMD_MODE_POLLING;
+ cmd->cmd_status = XSC_CMD_STATUS_NORMAL;
+
+ err = xsc_create_msg_cache(xdev);
+ if (err) {
+ dev_err(&xdev->pdev->dev, "failed to create command cache\n");
+ goto err_unmap_cq;
+ }
+
+ xsc_set_wqname(xdev);
+ cmd->wq = create_singlethread_workqueue(cmd->wq_name);
+ if (!cmd->wq) {
+ dev_err(&xdev->pdev->dev, "failed to create command workqueue\n");
+ err = -ENOMEM;
+ goto err_destroy_cache;
+ }
+
+ cmd->cq_task = kthread_create(xsc_cmd_cq_polling,
+ (void *)xdev,
+ "xsc_cmd_cq_polling");
+ if (!cmd->cq_task) {
+ dev_err(&xdev->pdev->dev, "failed to create cq task\n");
+ err = -ENOMEM;
+ goto err_destroy_wq;
+ }
+ wake_up_process(cmd->cq_task);
+
+ /* clear abnormal state to avoid the impact of previous error */
+ xsc_check_cmdq_status(xdev);
+
+ return 0;
+
+err_destroy_wq:
+ destroy_workqueue(cmd->wq);
+
+err_destroy_cache:
+ xsc_destroy_msg_cache(xdev);
+
+err_unmap_cq:
+ dma_unmap_single(&xdev->pdev->dev, cmd->cq_dma, PAGE_SIZE,
+ DMA_BIDIRECTIONAL);
+
+err_unmap_cmd:
+ dma_unmap_single(&xdev->pdev->dev, cmd->dma, PAGE_SIZE,
+ DMA_BIDIRECTIONAL);
+err_free_cq:
+ free_pages((unsigned long)cmd->cq_buf, 0);
+
+err_free_cmd:
+ free_pages((unsigned long)cmd->cmd_buf, 0);
+
+err_free_pool:
+ dma_pool_destroy(cmd->pool);
+
+ return err;
+}
+
+void xsc_cmd_cleanup(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd *cmd = &xdev->cmd;
+
+ destroy_workqueue(cmd->wq);
+ if (cmd->cq_task)
+ kthread_stop(cmd->cq_task);
+ xsc_destroy_msg_cache(xdev);
+ dma_unmap_single(&xdev->pdev->dev, cmd->dma, PAGE_SIZE,
+ DMA_BIDIRECTIONAL);
+ free_pages((unsigned long)cmd->cq_buf, 0);
+ dma_unmap_single(&xdev->pdev->dev, cmd->cq_dma, PAGE_SIZE,
+ DMA_BIDIRECTIONAL);
+ free_pages((unsigned long)cmd->cmd_buf, 0);
+ dma_pool_destroy(cmd->pool);
+}
+
+int xsc_cmd_version_check(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd_query_cmdq_ver_mbox_out *out;
+ struct xsc_cmd_query_cmdq_ver_mbox_in in;
+
+ int err;
+
+ out = kzalloc_obj(*out, GFP_KERNEL);
+ if (!out)
+ return -ENOMEM;
+
+ memset(&in, 0, sizeof(in));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_CMDQ_VERSION);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), out, sizeof(*out));
+ if (err)
+ goto err_free;
+
+ if (be16_to_cpu(out->cmdq_ver) != XSC_CMDQ_VERSION) {
+ pci_err(xdev->pdev, "cmdq version check failed, expecting %d, actual %d\n",
+ XSC_CMDQ_VERSION, be16_to_cpu(out->cmdq_ver));
+ err = -EINVAL;
+ goto err_free;
+ }
+ xdev->cmdq_ver = XSC_CMDQ_VERSION;
+
+err_free:
+ kfree(out);
+ return err;
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c
new file mode 100644
index 0000000000000..39715682fecc3
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.c
@@ -0,0 +1,316 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+#include "cq.h"
+
+void xsc_init_cq_table(struct xsc_core_device *xdev)
+{
+ struct xsc_cq_table *table = &xdev->dev_res->cq_table;
+
+ spin_lock_init(&table->lock);
+ INIT_RADIX_TREE(&table->tree, GFP_ATOMIC);
+}
+
+static int xsc_create_cq(struct xsc_core_device *xdev, u32 *p_cqn,
+ struct xsc_create_cq_mbox_in *in, int insize)
+{
+ struct xsc_create_cq_mbox_out out;
+ int ret;
+
+ memset(&out, 0, sizeof(out));
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_CQ);
+ ret = xsc_cmd_exec(xdev, in, insize, &out, sizeof(out));
+ if (ret) {
+ pci_err(xdev->pdev, "failed to create cq, err=%d out.status=%u\n",
+ ret, out.hdr.status);
+ return ret;
+ }
+
+ *p_cqn = be32_to_cpu(out.cqn) & 0xffffff;
+ return 0;
+}
+
+static int xsc_destroy_cq(struct xsc_core_device *xdev, u32 cqn)
+{
+ struct xsc_destroy_cq_mbox_out out;
+ struct xsc_destroy_cq_mbox_in in;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_CQ);
+ in.cqn = cpu_to_be32(cqn);
+
+ return xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+}
+
+int xsc_core_eth_create_cq(struct xsc_core_device *xdev,
+ struct xsc_core_cq *xcq,
+ struct xsc_create_cq_mbox_in *in,
+ int insize)
+{
+ struct xsc_cq_table *table = &xdev->dev_res->cq_table;
+ u32 cqn;
+ int ret;
+ int err;
+
+ ret = xsc_create_cq(xdev, &cqn, in, insize);
+ if (ret) {
+ pci_err(xdev->pdev, "xsc_create_cq failed\n");
+ return -ENOEXEC;
+ }
+ xcq->cqn = cqn;
+ xcq->cons_index = 0;
+ xcq->arm_sn = 0;
+ atomic_set(&xcq->refcount, 1);
+ init_completion(&xcq->free);
+
+ spin_lock_irq(&table->lock);
+ ret = radix_tree_insert(&table->tree, xcq->cqn, xcq);
+ spin_unlock_irq(&table->lock);
+ if (ret)
+ goto err_insert_cq;
+ return 0;
+err_insert_cq:
+ err = xsc_destroy_cq(xdev, cqn);
+ if (err)
+ pci_err(xdev->pdev, "failed to destroy cqn=%d, err=%d\n",
+ xcq->cqn, err);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eth_create_cq);
+
+int xsc_core_eth_destroy_cq(struct xsc_core_device *xdev,
+ struct xsc_core_cq *xcq)
+{
+ struct xsc_cq_table *table = &xdev->dev_res->cq_table;
+ struct xsc_core_cq *tmp;
+ int err;
+
+ spin_lock_irq(&table->lock);
+ tmp = radix_tree_delete(&table->tree, xcq->cqn);
+ spin_unlock_irq(&table->lock);
+ if (!tmp) {
+ err = -ENOENT;
+ goto err_delete_cq;
+ }
+
+ if (tmp != xcq) {
+ err = -EINVAL;
+ goto err_delete_cq;
+ }
+
+ err = xsc_destroy_cq(xdev, xcq->cqn);
+ if (err)
+ goto err_destroy_cq;
+
+ if (atomic_dec_and_test(&xcq->refcount))
+ complete(&xcq->free);
+ wait_for_completion(&xcq->free);
+ return 0;
+
+err_destroy_cq:
+ pci_err(xdev->pdev, "failed to destroy cqn=%d, err=%d\n",
+ xcq->cqn, err);
+ return err;
+err_delete_cq:
+ pci_err(xdev->pdev, "cqn=%d not found in tree, err=%d\n",
+ xcq->cqn, err);
+ return err;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eth_destroy_cq);
+
+static int xsc_set_cq_context(struct xsc_core_device *xdev,
+ struct xsc_cq_context_ex *ctx_ex,
+ u32 *pa_list_base, u32 *cqn)
+{
+ struct xsc_set_cq_context_mbox_in in;
+ struct xsc_set_cq_context_mbox_out out;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memcpy(&in.ctx_ex, ctx_ex, sizeof(*ctx_ex));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_CQ_CONTEXT);
+ in.hdr.ver = cpu_to_be16(CMD_SET_CQ_CONTEXT_V1);
+
+ memset(&out, 0, sizeof(out));
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err || out.hdr.status) {
+ pci_err(xdev->pdev, "failed to set cq context\n");
+ return -EIO;
+ }
+
+ *pa_list_base = be32_to_cpu(out.cq_pa_list_base);
+ *cqn = be32_to_cpu(out.cqn);
+ return 0;
+}
+
+static int xsc_set_cq_buf_pa(struct xsc_core_device *xdev,
+ struct xsc_create_cq_ex_mbox_in *req,
+ u32 pa_list_base, u32 cqn)
+{
+ struct xsc_set_cq_buf_pa_mbox_in *in;
+ struct xsc_set_cq_buf_pa_mbox_out out;
+ u32 pa_num_total = be32_to_cpu(req->ctx_ex.pa_num_ex);
+ u32 pa_num_max = (xdev->caps.max_cmd_in_len - sizeof(*in)) / sizeof(__be64);
+ u32 pa_num_left = pa_num_total;
+ u32 pa_num, copy_len;
+ int in_len, err;
+
+ while (pa_num_left) {
+ pa_num = min(pa_num_max, pa_num_left);
+ copy_len = pa_num * sizeof(__be64);
+ in_len = sizeof(*in) + copy_len;
+
+ in = kvzalloc(in_len, GFP_KERNEL);
+ if (!in)
+ return -ENOMEM;
+
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_CQ_BUF_PA);
+ in->pa_list_start = cpu_to_be32(pa_list_base);
+ in->pa_num = cpu_to_be32(pa_num);
+ memcpy(in->pas, &req->pas[pa_num_total - pa_num_left], copy_len);
+ pa_num_left -= pa_num;
+ pa_list_base += pa_num;
+ memset(&out, 0, sizeof(out));
+
+ err = xsc_cmd_exec(xdev, in, in_len, &out, sizeof(out));
+ kvfree(in);
+
+ if (err || out.hdr.status) {
+ pci_err(xdev->pdev, "failed to set cq buf pa, cqn=%d\n", cqn);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static int xsc_create_cq_compat(struct xsc_core_device *xdev,
+ struct xsc_create_cq_ex_mbox_in *in,
+ struct xsc_create_cq_mbox_out *out)
+{
+ struct xsc_create_cq_mbox_in *_in;
+ int _inlen = sizeof(*_in) +
+ be16_to_cpu(in->ctx_ex.ctx.pa_num) * sizeof(__be64);
+ int err;
+
+ _in = kvzalloc(_inlen, GFP_KERNEL);
+ if (!_in)
+ return -ENOMEM;
+
+ _in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_CQ);
+ memcpy(&_in->ctx, &in->ctx_ex.ctx, sizeof(_in->ctx));
+ memcpy(&_in->pas, &in->pas, _inlen - sizeof(*_in));
+ memset(out, 0, sizeof(*out));
+
+ err = xsc_cmd_exec(xdev, _in, _inlen, out, sizeof(*out));
+ kvfree(_in);
+
+ if (err)
+ return err;
+ if (out->hdr.status)
+ return xsc_cmd_status_to_err(&out->hdr);
+
+ return 0;
+}
+
+int xsc_core_create_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq,
+ struct xsc_create_cq_ex_mbox_in *in, int inlen)
+{
+ struct xsc_cq_table *table = &xdev->dev_res->cq_table;
+ struct xsc_create_cq_mbox_out out;
+ u32 pa_list_base = 0;
+ u32 cqn = 0;
+ int err;
+
+ if (inlen < xdev->caps.max_cmd_in_len) {
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_CQ_EX);
+ memset(&out, 0, sizeof(out));
+ err = xsc_cmd_exec(xdev, in, inlen, &out, sizeof(out));
+ if (err)
+ return err;
+
+ if (out.hdr.status && out.hdr.status != XSC_CMD_STATUS_NOT_SUPPORTED)
+ return xsc_cmd_status_to_err(&out.hdr);
+
+ if (out.hdr.status == XSC_CMD_STATUS_NOT_SUPPORTED) {
+ err = xsc_create_cq_compat(xdev, in, &out);
+ if (err)
+ return err;
+ }
+
+ cqn = be32_to_cpu(out.cqn);
+ } else {
+ err = xsc_set_cq_context(xdev, &in->ctx_ex, &pa_list_base, &cqn);
+ if (err)
+ return err;
+
+ err = xsc_set_cq_buf_pa(xdev, in, pa_list_base, cqn);
+ if (err)
+ goto err_cmd;
+ }
+
+ cq->cqn = cqn;
+ cq->cons_index = 0;
+ cq->arm_sn = 0;
+ cq->xdev = xdev;
+ cq->dev = xdev;
+ atomic_set(&cq->refcount, 1);
+ init_completion(&cq->free);
+
+ xsc_arm_cq(xdev, cq->cqn, 0, 0);
+
+ spin_lock_irq(&table->lock);
+ err = radix_tree_insert(&table->tree, cq->cqn, cq);
+ spin_unlock_irq(&table->lock);
+ if (err)
+ goto err_cmd;
+
+ cq->pid = current->pid;
+ return 0;
+
+err_cmd:
+ xsc_destroy_cq(xdev, cqn);
+ return err;
+}
+EXPORT_SYMBOL_GPL(xsc_core_create_cq);
+
+int xsc_core_destroy_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq)
+{
+ struct xsc_cq_table *table = &xdev->dev_res->cq_table;
+ struct xsc_destroy_cq_mbox_in in;
+ struct xsc_destroy_cq_mbox_out out;
+ struct xsc_core_cq *tmp;
+ int err;
+
+ spin_lock_irq(&table->lock);
+ tmp = radix_tree_delete(&table->tree, cq->cqn);
+ spin_unlock_irq(&table->lock);
+ if (!tmp || tmp != cq) {
+ pci_warn(xdev->pdev, "cq 0x%x not found in tree\n", cq->cqn);
+ return -EINVAL;
+ }
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_CQ);
+ in.cqn = cpu_to_be32(cq->cqn);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err)
+ return err;
+
+ if (out.hdr.status)
+ return xsc_cmd_status_to_err(&out.hdr);
+
+ if (atomic_dec_and_test(&cq->refcount))
+ complete(&cq->free);
+ wait_for_completion(&cq->free);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_destroy_cq);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/cq.h b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.h
new file mode 100644
index 0000000000000..afe6cbb0daa2b
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/cq.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_CQ_H
+#define __XSC_CQ_H
+
+#include "common/xsc_core.h"
+
+void xsc_init_cq_table(struct xsc_core_device *xdev);
+
+int xsc_core_create_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq,
+ struct xsc_create_cq_ex_mbox_in *in, int inlen);
+int xsc_core_destroy_cq(struct xsc_core_device *xdev, struct xsc_core_cq *cq);
+
+#endif /* __XSC_CQ_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/eq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/eq.c
new file mode 100644
index 0000000000000..ff4bcb42b33ff
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/eq.c
@@ -0,0 +1,320 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/mm.h>
+
+#include "common/xsc_cmd_api.h"
+#include "common/xsc_core.h"
+#include "qp.h"
+#include "alloc.h"
+#include "eq.h"
+#include "hal/xsc_hal.h"
+
+enum {
+ XSC_EQE_SIZE = sizeof(struct xsc_eqe),
+};
+
+enum {
+ XSC_NUM_SPARE_EQE = 0x80,
+ XSC_NUM_ASYNC_EQE = 0x100,
+};
+
+static int xsc_cmd_destroy_eq(struct xsc_core_device *xdev, u32 eqn)
+{
+ struct xsc_destroy_eq_mbox_out out = {0};
+ struct xsc_destroy_eq_mbox_in in = {0};
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_EQ);
+ in.eqn = cpu_to_be32(eqn);
+
+ return xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+}
+
+static struct xsc_eqe *get_eqe(struct xsc_eq *eq, u32 entry)
+{
+ return xsc_buf_offset(&eq->buf, entry * XSC_EQE_SIZE);
+}
+
+static struct xsc_eqe *next_eqe_sw(struct xsc_eq *eq)
+{
+ struct xsc_eqe *eqe = get_eqe(eq, eq->cons_index & (eq->nent - 1));
+
+ return ((eqe->owner_data & XSC_EQE_OWNER) ^
+ !!(eq->cons_index & eq->nent)) ? NULL : eqe;
+}
+
+static void eq_update_ci(struct xsc_eq *eq, int arm)
+{
+ xsc_update_eq_ci(eq->dev, eq->eqn, eq->cons_index, arm);
+}
+
+static void xsc_cq_completion(struct xsc_core_device *xdev, u32 cqn)
+{
+ struct xsc_cq_table *table = &xdev->dev_res->cq_table;
+ struct xsc_core_cq *cq;
+
+ rcu_read_lock();
+ cq = radix_tree_lookup(&table->tree, cqn);
+ if (likely(cq))
+ atomic_inc(&cq->refcount);
+ rcu_read_unlock();
+
+ if (!cq) {
+ pci_err(xdev->pdev, "Completion event for bogus CQ, cqn=%d\n",
+ cqn);
+ return;
+ }
+
+ ++cq->arm_sn;
+
+ if (!cq->comp)
+ pci_err(xdev->pdev, "cq->comp is NULL\n");
+ else
+ cq->comp(cq);
+
+ if (atomic_dec_and_test(&cq->refcount))
+ complete(&cq->free);
+}
+
+static void xsc_eq_cq_event(struct xsc_core_device *xdev,
+ u32 cqn, int event_type)
+{
+ struct xsc_cq_table *table = &xdev->dev_res->cq_table;
+ struct xsc_core_cq *cq;
+
+ spin_lock(&table->lock);
+ cq = radix_tree_lookup(&table->tree, cqn);
+ if (likely(cq))
+ atomic_inc(&cq->refcount);
+ spin_unlock(&table->lock);
+
+ if (unlikely(!cq)) {
+ pci_err(xdev->pdev, "Async event for bogus CQ, cqn=%d\n",
+ cqn);
+ return;
+ }
+
+ cq->event(cq, event_type);
+
+ if (atomic_dec_and_test(&cq->refcount))
+ complete(&cq->free);
+}
+
+static int xsc_eq_int(struct xsc_core_device *xdev, struct xsc_eq *eq)
+{
+ u32 cqn, qpn, queue_id;
+ struct xsc_eqe *eqe;
+ int eqes_found = 0;
+ int set_ci = 0;
+
+ while ((eqe = next_eqe_sw(eq))) {
+ /* Make sure we read EQ entry contents after we've
+ * checked the ownership bit.
+ */
+ rmb();
+ switch (eqe->type) {
+ case XSC_EVENT_TYPE_COMP:
+ case XSC_EVENT_TYPE_INTERNAL_ERROR:
+ /* eqe is changing */
+ queue_id = FIELD_GET(XSC_EQE_QUEUE_ID_MASK,
+ le16_to_cpu(eqe->queue_id_data));
+ cqn = queue_id;
+ xsc_cq_completion(xdev, cqn);
+ break;
+
+ case XSC_EVENT_TYPE_CQ_ERROR:
+ queue_id = FIELD_GET(XSC_EQE_QUEUE_ID_MASK,
+ le16_to_cpu(eqe->queue_id_data));
+ cqn = queue_id;
+ xsc_eq_cq_event(xdev, cqn, eqe->type);
+ break;
+ case XSC_EVENT_TYPE_WQ_CATAS_ERROR:
+ case XSC_EVENT_TYPE_WQ_INVAL_REQ_ERROR:
+ case XSC_EVENT_TYPE_WQ_ACCESS_ERROR:
+ queue_id = FIELD_GET(XSC_EQE_QUEUE_ID_MASK,
+ le16_to_cpu(eqe->queue_id_data));
+ qpn = queue_id;
+ xsc_qp_event(xdev, qpn, eqe->type);
+ break;
+ default:
+ break;
+ }
+
+ ++eq->cons_index;
+ eqes_found = 1;
+ ++set_ci;
+
+ /* The HCA will think the queue has overflowed if we
+ * don't tell it we've been processing events. We
+ * create our EQs with XSC_NUM_SPARE_EQE extra
+ * entries, so we must update our consumer index at
+ * least that often.
+ */
+ if (unlikely(set_ci >= XSC_NUM_SPARE_EQE)) {
+ eq_update_ci(eq, 0);
+ set_ci = 0;
+ }
+ }
+
+ eq_update_ci(eq, 1);
+
+ return eqes_found;
+}
+
+static irqreturn_t xsc_msix_handler(int irq, void *eq_ptr)
+{
+ struct xsc_core_device *xdev;
+ struct xsc_eq *eq = eq_ptr;
+
+ xdev = eq->dev;
+ xsc_eq_int(xdev, eq);
+
+ /* MSI-X vectors always belong to us */
+ return IRQ_HANDLED;
+}
+
+static void init_eq_buf(struct xsc_eq *eq)
+{
+ struct xsc_eqe *eqe;
+ int i;
+
+ for (i = 0; i < eq->nent; i++) {
+ eqe = get_eqe(eq, i);
+ eqe->owner_data |= XSC_EQE_OWNER;
+ }
+}
+
+int xsc_create_map_eq(struct xsc_core_device *xdev,
+ struct xsc_eq *eq, u16 vecidx,
+ u32 nent, const char *name)
+{
+ u16 msix_vec_offset = xdev->msix_vec_base + vecidx;
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+ struct xsc_create_eq_mbox_out out;
+ struct xsc_create_eq_mbox_in *in;
+ unsigned int hw_npages;
+ u32 inlen;
+ int err;
+
+ eq->nent = roundup_pow_of_two(roundup(nent, XSC_NUM_SPARE_EQE));
+ err = xsc_buf_alloc(xdev, eq->nent * XSC_EQE_SIZE, PAGE_SIZE, &eq->buf);
+ if (err)
+ return err;
+
+ init_eq_buf(eq);
+
+ hw_npages = DIV_ROUND_UP(eq->nent * XSC_EQE_SIZE, PAGE_SIZE_4K);
+ inlen = sizeof(*in) + sizeof(in->pas[0]) * hw_npages;
+ in = kvzalloc(inlen, GFP_KERNEL);
+ if (!in) {
+ err = -ENOMEM;
+ goto err_free_eq_buf;
+ }
+ memset(&out, 0, sizeof(out));
+
+ xsc_fill_page_array(&eq->buf, in->pas, hw_npages);
+
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_EQ);
+ in->ctx.log_eq_sz = ilog2(eq->nent);
+ in->ctx.vecidx = cpu_to_be16(msix_vec_offset);
+ in->ctx.pa_num = cpu_to_be16(hw_npages);
+ in->ctx.glb_func_id = cpu_to_be16(xdev->glb_func_id);
+ in->ctx.is_async_eq = (vecidx == XSC_EQ_VEC_ASYNC ? 1 : 0);
+
+ err = xsc_cmd_exec(xdev, in, inlen, &out, sizeof(out));
+ if (err)
+ goto err_free_in;
+
+ snprintf(dev_res->irq_info[vecidx].name, XSC_MAX_IRQ_NAME, "%s@pci:%s",
+ name, pci_name(xdev->pdev));
+
+ eq->eqn = be32_to_cpu(out.eqn);
+ eq->irqn = pci_irq_vector(xdev->pdev, vecidx);
+ eq->dev = xdev;
+ eq->index = vecidx;
+
+ err = request_irq(eq->irqn, xsc_msix_handler, 0,
+ dev_res->irq_info[vecidx].name, eq);
+ if (err)
+ goto err_destroy_eq;
+
+ /* EQs are created in ARMED state
+ */
+ eq_update_ci(eq, 1);
+ kvfree(in);
+ return 0;
+
+err_destroy_eq:
+ xsc_cmd_destroy_eq(xdev, eq->eqn);
+
+err_free_in:
+ kvfree(in);
+
+err_free_eq_buf:
+ xsc_buf_free(xdev, &eq->buf);
+ return err;
+}
+
+int xsc_destroy_unmap_eq(struct xsc_core_device *xdev, struct xsc_eq *eq)
+{
+ int err;
+
+ if (!xsc_fw_is_available(xdev))
+ return 0;
+
+ free_irq(eq->irqn, eq);
+ err = xsc_cmd_destroy_eq(xdev, eq->eqn);
+ if (err)
+ pci_err(xdev->pdev, "failed to destroy a previously created eq: eqn %d\n",
+ eq->eqn);
+ xsc_buf_free(xdev, &eq->buf);
+
+ return err;
+}
+
+void xsc_eq_init(struct xsc_core_device *xdev)
+{
+ spin_lock_init(&xdev->dev_res->eq_table.lock);
+}
+
+int xsc_start_eqs(struct xsc_core_device *xdev)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ int err;
+
+ err = xsc_create_map_eq(xdev, &table->async_eq, XSC_EQ_VEC_ASYNC,
+ XSC_NUM_ASYNC_EQE, "xsc_async_eq");
+ if (err)
+ pci_err(xdev->pdev, "failed to create async EQ %d\n", err);
+
+ return err;
+}
+
+void xsc_stop_eqs(struct xsc_core_device *xdev)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+
+ xsc_destroy_unmap_eq(xdev, &table->async_eq);
+}
+
+struct xsc_eq *xsc_core_eq_get(struct xsc_core_device *xdev, int i)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ struct xsc_eq *eq_ret = NULL;
+ struct xsc_eq *eq, *n;
+
+ spin_lock(&table->lock);
+ list_for_each_entry_safe(eq, n, &table->comp_eqs_list, list) {
+ if (eq->index == i) {
+ eq_ret = eq;
+ break;
+ }
+ }
+ spin_unlock(&table->lock);
+
+ return eq_ret;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eq_get);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/eq.h b/drivers/net/ethernet/yunsilicon/xsc/pci/eq.h
new file mode 100644
index 0000000000000..3e7869e12a31c
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/eq.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __EQ_H
+#define __EQ_H
+
+#include "common/xsc_core.h"
+
+enum {
+ XSC_EQ_VEC_ASYNC = 0,
+ XSC_VEC_CMD = 1,
+ XSC_VEC_CMD_EVENT = 2,
+ XSC_DMA_READ_DONE_VEC = 3,
+ XSC_EQ_VEC_COMP_BASE,
+};
+
+struct xsc_eqe {
+ u8 type;
+ u8 sub_type;
+ __le16 queue_id_data;
+#define XSC_EQE_QUEUE_ID_MASK GENMASK(14, 0)
+
+ u8 err_code;
+ u8 rsvd[2];
+ u8 owner_data;
+#define XSC_EQE_OWNER BIT(7)
+};
+
+int xsc_create_map_eq(struct xsc_core_device *xdev,
+ struct xsc_eq *eq, u16 vecidx,
+ u32 nent, const char *name);
+int xsc_destroy_unmap_eq(struct xsc_core_device *xdev, struct xsc_eq *eq);
+void xsc_eq_init(struct xsc_core_device *xdev);
+int xsc_start_eqs(struct xsc_core_device *xdev);
+void xsc_stop_eqs(struct xsc_core_device *xdev);
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_impl.c b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_impl.c
new file mode 100644
index 0000000000000..46fc045f92c02
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_impl.c
@@ -0,0 +1,708 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2024 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include "andes_reg.h"
+#include "common/xsc_core.h"
+#include "common/xsc_cmd.h"
+#include "common/xsc_hsi.h"
+#include "common/xsc_reg.h"
+#include "xsc_hal.h"
+
+/* message opcode */
+enum {
+ XSC_MSG_OPCODE_SEND = 0,
+ XSC_MSG_OPCODE_RDMA_WRITE = 1,
+ XSC_MSG_OPCODE_RDMA_READ = 2,
+ XSC_MSG_OPCODE_MAD = 3,
+ XSC_MSG_OPCODE_RDMA_ACK = 4,
+ XSC_MSG_OPCODE_RDMA_ACK_READ = 5,
+ XSC_MSG_OPCODE_RDMA_CNP = 6,
+ XSC_MSG_OPCODE_RAW = 7,
+ XSC_MSG_OPCODE_VIRTIO_NET = 8,
+ XSC_MSG_OPCODE_VIRTIO_BLK = 9,
+ XSC_MSG_OPCODE_RAW_TPE = 10,
+ XSC_MSG_OPCODE_INIT_QP_REQ = 11,
+ XSC_MSG_OPCODE_INIT_QP_RSP = 12,
+ XSC_MSG_OPCODE_INIT_PATH_REQ = 13,
+ XSC_MSG_OPCODE_INIT_PATH_RSP = 14,
+
+ XSC_MSG_OPCODE_UNSUPPORT = -1,
+};
+
+#define REG_ADDR(bp, offset) ((bp) + (offset))
+
+static void xsc_read32(void __iomem *bar, u32 off, void *data, int len)
+{
+ u32 val = readl(REG_ADDR(bar, off));
+
+ memcpy(data, &val, len);
+}
+
+static void xsc_write32(void __iomem *bar, u32 off, void *data)
+{
+ writel(*(u32 *)data, REG_ADDR(bar, off));
+}
+
+static void andes_ring_tx_doorbell(struct xsc_core_device *xdev,
+ struct xsc_hw_reg *regs,
+ void __iomem *bar, u32 sqn, u32 next_pid,
+ bool mdb)
+{
+ union xsc_send_doorbell {
+ struct{
+ u32 next_pid : 16;
+ u32 qp_id : 15;
+ };
+ u32 raw;
+ } db;
+ u32 db_reg = mdb ? QPM_TX_MDB_BASE_REG_ADDR + (sqn & (MDB_NUM - 1)) * sizeof(u32) :
+ regs->tx_db;
+
+ db.next_pid = next_pid;
+ db.qp_id = sqn;
+
+ /* Make sure that descriptors are written before
+ * updating doorbell record and ringing the doorbell
+ */
+ wmb();
+
+ writel(db.raw, REG_ADDR(bar, db_reg));
+}
+
+static void andes_ring_rx_doorbell(struct xsc_hw_reg *regs,
+ void __iomem *bar, u32 rqn, u32 next_pid)
+{
+ union xsc_recv_doorbell {
+ struct{
+ u32 next_pid : 13;
+ u32 qp_id : 15;
+ };
+ u32 raw;
+ } db;
+
+ db.next_pid = next_pid;
+ db.qp_id = rqn;
+
+ /* Make sure that descriptors are written before
+ * updating doorbell record and ringing the doorbell
+ */
+ wmb();
+
+ writel(db.raw, REG_ADDR(bar, regs->rx_db));
+}
+
+static void andes_update_cq_db(struct xsc_hw_reg *regs,
+ void __iomem *bar, u32 cqn, u32 next_cid,
+ u8 solicited)
+{
+ union andes_cq_doorbell {
+ struct{
+ u32 cq_next_cid:16;
+ u32 cq_id:15;
+ u32 arm:1;
+ };
+ u32 val;
+ } db;
+
+ db.cq_next_cid = next_cid;
+ db.cq_id = cqn;
+ db.arm = solicited;
+
+ /* Make sure that the doorbell record in host memory is
+ * written before ringing the doorbell via PCI MMIO.
+ */
+ wmb();
+ writel(db.val, REG_ADDR(bar, regs->complete_db));
+}
+
+static void andes_set_cq_ci(struct xsc_hw_reg *regs,
+ void __iomem *bar, u32 cqn, u32 next_cid)
+{
+ union andes_cq_doorbell {
+ struct{
+ u32 cq_next_cid:16;
+ u32 cq_id:15;
+ u32 arm:1;
+ };
+ u32 val;
+ } db;
+
+ db.val = 0;
+ db.cq_next_cid = next_cid;
+ db.cq_id = cqn;
+ /* make sure val write to memory done */
+ wmb();
+ writel(db.val, REG_ADDR(bar, regs->complete_reg));
+}
+
+static void andes_set_eq_ci(struct xsc_hw_reg *regs,
+ void __iomem *bar, u32 eqn, u32 next_cid,
+ u8 arm)
+{
+ union andes_eq_doorbell {
+ struct{
+ u32 eq_next_cid : 11;
+ u32 eq_id : 11;
+ u32 arm : 1;
+ };
+ u32 val;
+ } db;
+
+ db.eq_next_cid = next_cid;
+ db.eq_id = eqn;
+ db.arm = !!arm;
+
+ /* make sure val write to memory done */
+ wmb();
+ writel(db.val, REG_ADDR(bar, regs->event_db));
+}
+
+static u8 andes_get_mr_page_mode(u8 page_shift)
+{
+enum {
+ XSC_PAGE_SHIFT_4K = 12,
+ XSC_PAGE_SHIFT_64K = 16,
+ XSC_PAGE_SHIFT_2M = 21,
+ XSC_PAGE_SHIFT_1G = 30,
+};
+
+enum {
+ XSC_PAGE_MODE_4K = 0,
+ XSC_PAGE_MODE_64K = 1,
+ XSC_PAGE_MODE_2M = 2,
+ XSC_PAGE_MODE_1G = 3,
+};
+
+ return (page_shift == XSC_PAGE_SHIFT_4K ? XSC_PAGE_MODE_4K :
+ (page_shift == XSC_PAGE_SHIFT_64K ? XSC_PAGE_MODE_64K :
+ (page_shift == XSC_PAGE_SHIFT_2M ? XSC_PAGE_MODE_2M : XSC_PAGE_MODE_1G)));
+}
+
+static inline u32 andes_mkey_to_idx(u32 mkey)
+{
+ return mkey >> 17;
+}
+
+static inline u32 andes_idx_to_mkey(u32 mkey_idx)
+{
+ return mkey_idx << 17;
+}
+
+static const u32 xsc_msg_opcode[][2][2] = {
+ [XSC_MSG_OPCODE_SEND][XSC_REQ][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_REQ_SEND,
+ [XSC_MSG_OPCODE_SEND][XSC_REQ][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_REQ_SEND_IMMDT,
+ [XSC_MSG_OPCODE_SEND][XSC_RSP][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_RSP_RECV,
+ [XSC_MSG_OPCODE_SEND][XSC_RSP][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_RSP_RECV_IMMDT,
+ [XSC_MSG_OPCODE_RDMA_WRITE][XSC_REQ][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_REQ_WRITE,
+ [XSC_MSG_OPCODE_RDMA_WRITE][XSC_REQ][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_REQ_WRITE_IMMDT,
+ [XSC_MSG_OPCODE_RDMA_WRITE][XSC_RSP][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_CQE_ERROR,
+ [XSC_MSG_OPCODE_RDMA_WRITE][XSC_RSP][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_RSP_WRITE_IMMDT,
+ [XSC_MSG_OPCODE_RDMA_READ][XSC_REQ][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_REQ_READ,
+ [XSC_MSG_OPCODE_RDMA_READ][XSC_REQ][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_CQE_ERROR,
+ [XSC_MSG_OPCODE_RDMA_READ][XSC_RSP][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_CQE_ERROR,
+ [XSC_MSG_OPCODE_RDMA_READ][XSC_RSP][XSC_WITH_IMMDT] = XSC_OPCODE_RDMA_CQE_ERROR,
+ [XSC_MSG_OPCODE_MAD][XSC_REQ][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_MAD_REQ_SEND,
+ [XSC_MSG_OPCODE_MAD][XSC_RSP][XSC_WITHOUT_IMMDT] = XSC_OPCODE_RDMA_MAD_RSP_RECV,
+};
+
+struct andes_cqe {
+ union {
+ u8 msg_opcode;
+ struct {
+ u8 error_code:7;
+ u8 is_error:1;
+ };
+ };
+ __le32 qp_id:15;
+ u8 rsv1:1;
+ u8 se:1;
+ u8 has_pph:1;
+ u8 type:1;
+ u8 with_immdt:1;
+ u8 csum_err:4;
+ __le32 imm_data;
+ __le32 msg_len;
+ __le32 vni;
+ __le64 ts:48;
+ __le16 wqe_id;
+ __le16 rsv[3];
+ __le16 rsv2:15;
+ u8 owner:1;
+};
+
+static int andes_get_cqe_wqe_id(void *cqe)
+{
+ return ((struct andes_cqe *)cqe)->wqe_id;
+}
+
+static u8 andes_get_cqe_opcode(void *cqe)
+{
+ struct andes_cqe *_cqe = cqe;
+ u8 msg_opcode = _cqe->msg_opcode;
+
+ if (_cqe->is_error)
+ return _cqe->type ? XSC_OPCODE_RDMA_RSP_ERROR : XSC_OPCODE_RDMA_REQ_ERROR;
+ if (msg_opcode > XSC_MSG_OPCODE_MAD)
+ return XSC_OPCODE_RDMA_CQE_ERROR;
+ return xsc_msg_opcode[msg_opcode][_cqe->type][_cqe->with_immdt];
+}
+
+struct andes_data_seg {
+ u32 in_line:1;
+ u32 length:31;
+ u32 key;
+ u64 addr;
+};
+
+static void andes_set_data_seg(void *data_seg, u32 length, u32 key, u64 addr)
+{
+ struct andes_data_seg *seg = data_seg;
+
+ seg->length = length;
+ seg->key = key;
+ seg->addr = addr;
+}
+
+struct andes_send_wqe_ctrl_seg {
+ __le32 msg_opcode:8;
+ __le32 with_immdt:1;
+ __le32 csum_en:2;
+ __le32 ds_data_num:5;
+ __le32 wqe_id:16;
+ __le32 msg_len;
+ union {
+ __le32 opcode_data;
+ struct {
+ u8 has_pph:1;
+ u8 so_type:1;
+ __le16 so_data_size:14;
+ u8:8;
+ u8 so_hdr_len:8;
+ };
+ struct {
+ __le16 desc_id;
+ __le16 is_last_wqe:1;
+ __le16 dst_qp_id:15;
+ };
+ };
+ __le32 se:1;
+ __le32 ce:1;
+ __le32:30;
+};
+
+static void andes_set_wqe_id(void *cseg, u32 wqe_id)
+{
+ u16 _wqe_id = wqe_id;
+
+ ((struct andes_send_wqe_ctrl_seg *)cseg)->wqe_id = cpu_to_le16(_wqe_id);
+}
+
+static struct xsc_msg_opcode xsc_ib_opcode = {
+ .send = XSC_MSG_OPCODE_SEND,
+ .send_with_imm = XSC_MSG_OPCODE_SEND,
+ .rdma_write = XSC_MSG_OPCODE_RDMA_WRITE,
+ .rdma_write_with_imm = XSC_MSG_OPCODE_RDMA_WRITE,
+ .rdma_read = XSC_MSG_OPCODE_RDMA_READ,
+ .atomic_cmp_and_swp = XSC_MSG_OPCODE_UNSUPPORT,
+ .atomic_fetch_and_add = XSC_MSG_OPCODE_UNSUPPORT,
+ .masked_atomic_cmp_and_swp = XSC_MSG_OPCODE_UNSUPPORT,
+ .masked_atomic_fetch_and_add = XSC_MSG_OPCODE_UNSUPPORT,
+ .mad = XSC_MSG_OPCODE_MAD,
+};
+
+static struct xsc_msg_opcode *andes_get_msg_opcode(void)
+{
+ return &xsc_ib_opcode;
+}
+
+static struct xsc_hw_reg andes_pf_regs = {
+ .req_pid_addr = HIF_CMDQM_HOST_REQ_PID_MEM_ADDR - 0xa0000000,
+ .req_cid_addr = HIF_CMDQM_HOST_REQ_CID_MEM_ADDR - 0xa0000000,
+ .rsp_pid_addr = HIF_CMDQM_HOST_RSP_PID_MEM_ADDR - 0xa0000000,
+ .rsp_cid_addr = HIF_CMDQM_HOST_RSP_CID_MEM_ADDR - 0xa0000000,
+ .req_buf_h_addr = HIF_CMDQM_HOST_REQ_BUF_BASE_H_ADDR_MEM_ADDR - 0xa0000000,
+ .req_buf_l_addr = HIF_CMDQM_HOST_REQ_BUF_BASE_L_ADDR_MEM_ADDR - 0xa0000000,
+ .rsp_buf_h_addr = HIF_CMDQM_HOST_RSP_BUF_BASE_H_ADDR_MEM_ADDR - 0xa0000000,
+ .rsp_buf_l_addr = HIF_CMDQM_HOST_RSP_BUF_BASE_L_ADDR_MEM_ADDR - 0xa0000000,
+ .msix_vec_addr = HIF_CMDQM_VECTOR_ID_MEM_ADDR - 0xa0000000,
+ .interrupt_stat_addr = HIF_CMDQM_HOST_VF_ERR_STS_MEM_ADDR - 0xa0000000,
+ .mpt_tbl_addr = MMC_MPT_TBL_MEM_ADDR - 0xa0000000,
+ .mpt_tbl_depth = MMC_MPT_TBL_MEM_DEPTH,
+ .mpt_tbl_width = MMC_MPT_TBL_MEM_WIDTH,
+ .mtt_inst_base_addr = MMC_MTT_TBL_MEM_ADDR - 0xa0000000,
+ .mtt_inst_stride = 0,
+ .mtt_inst_num_log = 0,
+ .mtt_inst_depth = MMC_MTT_TBL_MEM_DEPTH,
+};
+
+/* BAR-relative window shared by bar-compressed PF and VF */
+static struct xsc_hw_reg andes_compressed_regs = {
+ .tx_db = TX_DB_FUNC_MEM_ADDR,
+ .rx_db = RX_DB_FUNC_MEM_ADDR,
+ .complete_db = DB_CQ_FUNC_MEM_ADDR,
+ .complete_reg = DB_CQ_CID_DIRECT_MEM_ADDR,
+ .event_db = DB_EQ_FUNC_MEM_ADDR,
+ .req_pid_addr = CMDQM_HOST_REQ_PID_MEM_ADDR,
+ .req_cid_addr = CMDQM_HOST_REQ_CID_MEM_ADDR,
+ .rsp_pid_addr = CMDQM_HOST_RSP_PID_MEM_ADDR,
+ .rsp_cid_addr = CMDQM_HOST_RSP_CID_MEM_ADDR,
+ .req_buf_h_addr = CMDQM_HOST_REQ_BUF_BASE_H_ADDR_MEM_ADDR,
+ .req_buf_l_addr = CMDQM_HOST_REQ_BUF_BASE_L_ADDR_MEM_ADDR,
+ .rsp_buf_h_addr = CMDQM_HOST_RSP_BUF_BASE_H_ADDR_MEM_ADDR,
+ .rsp_buf_l_addr = CMDQM_HOST_RSP_BUF_BASE_L_ADDR_MEM_ADDR,
+ .msix_vec_addr = CMDQM_VECTOR_ID_MEM_ADDR,
+ .interrupt_stat_addr = CMDQM_HOST_VF_ERR_STS_MEM_ADDR,
+ .mpt_tbl_addr = MMC_MPT_TBL_MEM_ADDR - 0xa0000000,
+ .mpt_tbl_depth = MMC_MPT_TBL_MEM_DEPTH,
+ .mpt_tbl_width = MMC_MPT_TBL_MEM_WIDTH,
+ .mtt_inst_base_addr = MMC_MTT_TBL_MEM_ADDR - 0xa0000000,
+ .mtt_inst_stride = 0,
+ .mtt_inst_num_log = 0,
+ .mtt_inst_depth = MMC_MTT_TBL_MEM_DEPTH,
+};
+
+void xsc_init_hal(struct xsc_core_device *xdev, u32 device_id)
+{
+ switch (device_id) {
+ case XSC_MC_VF_DEV_ID:
+ case XSC_MF_HOST_VF_DEV_ID:
+ case XSC_MS_VF_DEV_ID:
+ case XSC_MV_HOST_VF_DEV_ID:
+ /* VFs expose the same BAR-relative window as a bar-compressed PF */
+ xdev->hal_regs = &andes_compressed_regs;
+ break;
+ default:
+ if (is_pf_bar_compressed(xdev))
+ xdev->hal_regs = &andes_compressed_regs;
+ else
+ xdev->hal_regs = &andes_pf_regs;
+ break;
+ }
+}
+
+void xsc_set_mtt_info(struct xsc_core_device *xdev)
+{
+ struct xsc_hw_reg *regs = xdev->hal_regs;
+
+ regs->mpt_tbl_addr = xdev->caps.mpt_tbl_addr - 0xa0000000;
+ regs->mpt_tbl_depth = xdev->caps.mpt_tbl_depth;
+ regs->mpt_tbl_width = xdev->caps.mpt_tbl_width;
+ regs->mtt_inst_base_addr = xdev->caps.mtt_inst_base_addr - 0xa0000000;
+ regs->mtt_inst_stride = xdev->caps.mtt_inst_stride;
+ regs->mtt_inst_num_log = xdev->caps.mtt_inst_num_log;
+ regs->mtt_inst_depth = xdev->caps.mtt_inst_depth;
+}
+
+void xsc_set_pf_db_addr(struct xsc_core_device *xdev,
+ u64 tx_db, u64 rx_db, u64 cq_db, u64 cq_reg, u64 eq_db)
+{
+ struct xsc_hw_reg *regs = xdev->hal_regs;
+
+ if (is_pf_bar_compressed(xdev))
+ return;
+
+ if (xsc_core_is_pf(xdev)) {
+ regs->tx_db = tx_db - 0xa0000000;
+ regs->rx_db = rx_db - 0xa0000000;
+ regs->complete_db = cq_db - 0xa0000000;
+ regs->complete_reg = cq_reg - 0xa0000000;
+ regs->event_db = eq_db - 0xa0000000;
+ }
+}
+
+void xsc_get_db_addr(struct xsc_core_device *xdev,
+ u64 *tx_db, u64 *rx_db, u64 *cq_db, u64 *cq_reg, u64 *eq_db)
+{
+ struct xsc_hw_reg *regs = xdev->hal_regs;
+
+ if (!xsc_core_is_pf(xdev) || is_pf_bar_compressed(xdev)) {
+ if (tx_db)
+ *tx_db = regs->tx_db;
+ if (rx_db)
+ *rx_db = regs->rx_db;
+ if (cq_db)
+ *cq_db = regs->complete_db;
+ if (cq_reg)
+ *cq_reg = regs->complete_reg;
+ if (eq_db)
+ *eq_db = regs->event_db;
+ } else {
+ if (tx_db)
+ *tx_db = regs->tx_db + 0xa0000000;
+ if (rx_db)
+ *rx_db = regs->rx_db + 0xa0000000;
+ if (cq_db)
+ *cq_db = regs->complete_db + 0xa0000000;
+ if (cq_reg)
+ *cq_reg = regs->complete_reg + 0xa0000000;
+ if (eq_db)
+ *eq_db = regs->event_db + 0xa0000000;
+ }
+}
+EXPORT_SYMBOL_GPL(xsc_get_db_addr);
+
+void xsc_update_tx_db(struct xsc_core_device *xdev, u32 sqn, u32 next_pid)
+{
+ bool mdb = is_support_multi_doorbell(xdev);
+
+ andes_ring_tx_doorbell(xdev, xdev->hal_regs, xdev->bar, sqn,
+ next_pid, mdb);
+}
+EXPORT_SYMBOL_GPL(xsc_update_tx_db);
+
+void xsc_update_rx_db(struct xsc_core_device *xdev, u32 rqn, u32 next_pid)
+{
+ andes_ring_rx_doorbell(xdev->hal_regs, xdev->bar, rqn, next_pid);
+}
+EXPORT_SYMBOL_GPL(xsc_update_rx_db);
+
+void xsc_arm_cq(struct xsc_core_device *xdev, u32 cqn, u32 next_cid, u8 solicited)
+{
+ andes_update_cq_db(xdev->hal_regs, xdev->bar, cqn, next_cid, solicited);
+}
+EXPORT_SYMBOL_GPL(xsc_arm_cq);
+
+void xsc_update_cq_ci(struct xsc_core_device *xdev, u32 cqn, u32 next_cid)
+{
+ andes_set_cq_ci(xdev->hal_regs, xdev->bar, cqn, next_cid);
+}
+EXPORT_SYMBOL_GPL(xsc_update_cq_ci);
+
+void xsc_update_eq_ci(struct xsc_core_device *xdev, u32 eqn, u32 next_cid, u8 arm)
+{
+ andes_set_eq_ci(xdev->hal_regs, xdev->bar, eqn, next_cid, arm);
+}
+
+void xsc_update_cmdq_req_pid(struct xsc_core_device *xdev, u32 req_pid)
+{
+ xsc_write32(xdev->bar, xdev->hal_regs->req_pid_addr, &req_pid);
+}
+
+void xsc_update_cmdq_rsp_cid(struct xsc_core_device *xdev, u32 rsp_cid)
+{
+ xsc_write32(xdev->bar, xdev->hal_regs->rsp_cid_addr, &rsp_cid);
+}
+
+u32 xsc_get_cmdq_req_pid(struct xsc_core_device *xdev)
+{
+ u32 val;
+
+ xsc_read32(xdev->bar, xdev->hal_regs->req_pid_addr, &val, sizeof(val));
+ return val;
+}
+
+u32 xsc_get_cmdq_req_cid(struct xsc_core_device *xdev)
+{
+ u32 val;
+
+ xsc_read32(xdev->bar, xdev->hal_regs->req_cid_addr, &val, sizeof(val));
+ return val;
+}
+
+u32 xsc_get_cmdq_rsp_pid(struct xsc_core_device *xdev)
+{
+ u32 val;
+
+ xsc_read32(xdev->bar, xdev->hal_regs->rsp_pid_addr, &val, sizeof(val));
+ return val;
+}
+
+u32 xsc_get_cmdq_rsp_cid(struct xsc_core_device *xdev)
+{
+ u32 val;
+
+ xsc_read32(xdev->bar, xdev->hal_regs->rsp_cid_addr, &val, sizeof(val));
+ return val;
+}
+
+void xsc_set_cmdq_req_buf_addr(struct xsc_core_device *xdev, u32 haddr, u32 laddr)
+{
+ xsc_write32(xdev->bar, xdev->hal_regs->req_buf_h_addr, &haddr);
+ xsc_write32(xdev->bar, xdev->hal_regs->req_buf_l_addr, &laddr);
+}
+
+void xsc_set_cmdq_rsp_buf_addr(struct xsc_core_device *xdev, u32 haddr, u32 laddr)
+{
+ xsc_write32(xdev->bar, xdev->hal_regs->rsp_buf_h_addr, &haddr);
+ xsc_write32(xdev->bar, xdev->hal_regs->rsp_buf_l_addr, &laddr);
+}
+
+void xsc_set_cmdq_msix_vector(struct xsc_core_device *xdev, u32 vector)
+{
+ xsc_write32(xdev->bar, xdev->hal_regs->msix_vec_addr, &vector);
+}
+
+void xsc_check_cmdq_status(struct xsc_core_device *xdev)
+{
+ u32 status;
+
+ xsc_read32(xdev->bar, xdev->hal_regs->interrupt_stat_addr, &status,
+ sizeof(status));
+ if (status) {
+ status = 0xf;
+ xsc_write32(xdev->bar, xdev->hal_regs->interrupt_stat_addr, &status);
+ }
+}
+
+int xsc_handle_cmdq_interrupt(struct xsc_core_device *xdev)
+{
+ union interrupt_stat {
+ struct {
+ u32 hw_read_req_err:1;
+ u32 hw_write_req_err:1;
+ u32 req_pid_err:1;
+ u32 rsp_cid_err:1;
+ };
+ u32 raw;
+ } stat;
+
+ int err = 0;
+ int retry = 0;
+
+ xsc_read32(xdev->bar, xdev->hal_regs->interrupt_stat_addr, &stat.raw,
+ sizeof(stat.raw));
+ while (stat.raw != 0) {
+ err++;
+ if (stat.hw_read_req_err) {
+ retry = 1;
+ stat.hw_read_req_err = 0;
+ pci_err(xdev->pdev, "hw report read req from host failed!\n");
+ } else if (stat.hw_write_req_err) {
+ retry = 1;
+ stat.hw_write_req_err = 0;
+ pci_err(xdev->pdev, "hw report write req to fw failed!\n");
+ } else if (stat.req_pid_err) {
+ stat.req_pid_err = 0;
+ pci_err(xdev->pdev, "hw report unexpected req pid!\n");
+ } else if (stat.rsp_cid_err) {
+ stat.rsp_cid_err = 0;
+ pci_err(xdev->pdev, "hw report unexpected rsp cid!\n");
+ } else {
+ stat.raw = 0;
+ pci_err(xdev->pdev, "ignore unknown interrupt!\n");
+ }
+ }
+
+ if (retry) {
+ if (xdev->cmd.retry_cnt >= XSC_CMD_MAX_RETRY_CNT) {
+ pci_warn(xdev->pdev, "err handler have retried for %d times, stop retry\n",
+ xdev->cmd.retry_cnt);
+ goto clear_err_int;
+ }
+ xsc_write32(xdev->bar, xdev->hal_regs->req_pid_addr,
+ &xdev->cmd.cmd_pid);
+ xdev->cmd.retry_cnt++;
+ }
+
+clear_err_int:
+ if (err) {
+ stat.raw = 0xf;
+ xsc_write32(xdev->bar, xdev->hal_regs->interrupt_stat_addr, &stat.raw);
+ }
+
+ return err;
+}
+
+u8 xsc_get_mr_page_mode(struct xsc_core_device *xdev, u8 page_shift)
+{
+ return andes_get_mr_page_mode(page_shift);
+}
+EXPORT_SYMBOL_GPL(xsc_get_mr_page_mode);
+
+u32 xsc_mkey_to_idx(struct xsc_core_device *xdev, u32 mkey)
+{
+ return andes_mkey_to_idx(mkey);
+}
+
+u32 xsc_idx_to_mkey(struct xsc_core_device *xdev, u32 mkey_idx)
+{
+ return andes_idx_to_mkey(mkey_idx);
+}
+
+u8 xsc_get_cqe_opcode(struct xsc_core_device *xdev, void *cqe)
+{
+ return andes_get_cqe_opcode(cqe);
+}
+EXPORT_SYMBOL_GPL(xsc_get_cqe_opcode);
+
+u64 xsc_get_rdma_stat_mask(struct xsc_core_device *xdev, u16 ver)
+{
+ u64 mask;
+
+ if (xsc_core_is_pf(xdev))
+ mask = 0xFFFFFF;
+ else
+ mask = 0xfff;
+
+ return mask;
+}
+EXPORT_SYMBOL_GPL(xsc_get_rdma_stat_mask);
+
+void xsc_set_data_seg(struct xsc_core_device *xdev, void *data_seg, u64 addr, u32 key, u32 length)
+{
+ andes_set_data_seg(data_seg, length, key, addr);
+}
+EXPORT_SYMBOL_GPL(xsc_set_data_seg);
+
+u32 xsc_get_max_qp_depth(struct xsc_core_device *xdev)
+{
+ return 32 * 1024;
+}
+EXPORT_SYMBOL_GPL(xsc_get_max_qp_depth);
+
+bool xsc_check_max_qp_depth(struct xsc_core_device *xdev, u32 *wqe_cnt, u32 max_qp_depth)
+{
+ if (*wqe_cnt > max_qp_depth)
+ *wqe_cnt = max_qp_depth;
+
+ return false;
+}
+EXPORT_SYMBOL_GPL(xsc_check_max_qp_depth);
+
+u32 xsc_get_recv_ds_num(struct xsc_core_device *xdev)
+{
+ return xdev->caps.recv_ds_num;
+}
+EXPORT_SYMBOL_GPL(xsc_get_recv_ds_num);
+
+void *xsc_get_msg_opcode(struct xsc_core_device *xdev)
+{
+ return andes_get_msg_opcode();
+}
+
+u64 xsc_get_max_mr_size(struct xsc_core_device *xdev)
+{
+ u64 max_mpt_mr_size = xdev->caps.max_mr_size;
+ u64 max_mtt_mr_size = xdev->caps.max_mtt * PAGE_SIZE;
+
+ return min_t(u64, max_mpt_mr_size, max_mtt_mr_size);
+}
+EXPORT_SYMBOL_GPL(xsc_get_max_mr_size);
+
+void xsc_get_multidb_info(struct xsc_core_device *xdev, u32 *num, u32 *base)
+{
+ if (is_support_multi_doorbell(xdev)) {
+ *num = MDB_NUM;
+ *base = QPM_TX_MDB_BASE_REG_ADDR;
+ } else {
+ *num = 0;
+ *base = 0;
+ }
+}
+EXPORT_SYMBOL_GPL(xsc_get_multidb_info);
+
+void xsc_set_wqe_id(struct xsc_core_device *dev, void *cseg, u32 wqe_id)
+{
+ andes_set_wqe_id(cseg, wqe_id);
+}
+EXPORT_SYMBOL_GPL(xsc_set_wqe_id);
+
+int xsc_get_cqe_wqe_id(struct xsc_core_device *dev, void *cqe)
+{
+ return andes_get_cqe_wqe_id(cqe);
+}
+EXPORT_SYMBOL_GPL(xsc_get_cqe_wqe_id);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_reg.h b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_reg.h
new file mode 100644
index 0000000000000..cd25b64a58184
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/andes_reg.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+/* generated time:
+ * Thu Feb 29 15:33:50 CST 2024
+ */
+
+#ifndef ANDES_REG_H
+#define ANDES_REG_H
+
+//hif_irq_csr_defines.h
+
+//hif_cpm_csr_defines.h
+
+//mmc_csr_defines.h
+#define MMC_MPT_TBL_MEM_DEPTH 32768
+#define MMC_MTT_TBL_MEM_DEPTH 262144
+#define MMC_MPT_TBL_MEM_WIDTH 256
+#define MMC_MPT_TBL_MEM_ADDR 0xa4100000
+#define MMC_MTT_TBL_MEM_ADDR 0xa4200000
+
+//clsf_dma_csr_defines.h
+
+//hif_tbl_csr_defines.h
+
+//hif_cmdqm_csr_defines.h
+#define HIF_CMDQM_HOST_REQ_PID_MEM_ADDR 0xa1026000
+#define HIF_CMDQM_HOST_REQ_CID_MEM_ADDR 0xa1028000
+#define HIF_CMDQM_HOST_RSP_PID_MEM_ADDR 0xa102e000
+#define HIF_CMDQM_HOST_RSP_CID_MEM_ADDR 0xa1030000
+#define HIF_CMDQM_HOST_REQ_BUF_BASE_H_ADDR_MEM_ADDR 0xa1022000
+#define HIF_CMDQM_HOST_REQ_BUF_BASE_L_ADDR_MEM_ADDR 0xa1024000
+#define HIF_CMDQM_HOST_RSP_BUF_BASE_H_ADDR_MEM_ADDR 0xa102a000
+#define HIF_CMDQM_HOST_RSP_BUF_BASE_L_ADDR_MEM_ADDR 0xa102c000
+#define HIF_CMDQM_VECTOR_ID_MEM_ADDR 0xa1034000
+#define HIF_CMDQM_HOST_VF_ERR_STS_MEM_ADDR 0xa1032000
+
+//PSV use
+//hif_irq_csr_defines.h
+
+#endif /* XSC_HW_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h
new file mode 100644
index 0000000000000..5a3a7e27f34b1
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hal/xsc_hal.h
@@ -0,0 +1,85 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2024 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef XSC_HAL_H
+#define XSC_HAL_H
+
+struct xsc_core_device;
+
+struct xsc_hw_reg {
+ u32 tx_db;
+ u32 rx_db;
+ u32 complete_db;
+ u32 complete_reg;
+ u32 event_db;
+ u32 req_pid_addr;
+ u32 req_cid_addr;
+ u32 rsp_pid_addr;
+ u32 rsp_cid_addr;
+ u32 req_buf_h_addr;
+ u32 req_buf_l_addr;
+ u32 rsp_buf_h_addr;
+ u32 rsp_buf_l_addr;
+ u32 msix_vec_addr;
+ u32 interrupt_stat_addr;
+ u32 mpt_tbl_addr;
+ u32 mpt_tbl_depth;
+ u32 mpt_tbl_width;
+ u32 mtt_inst_base_addr;
+ u32 mtt_inst_stride;
+ u32 mtt_inst_num_log;
+ u32 mtt_inst_depth;
+};
+
+/* HAL entry points exported to the PCI/RDMA/ETH layers */
+void xsc_init_hal(struct xsc_core_device *xdev, u32 device_id);
+void xsc_set_mtt_info(struct xsc_core_device *xdev);
+void xsc_set_pf_db_addr(struct xsc_core_device *xdev,
+ u64 tx_db, u64 rx_db, u64 cq_db, u64 cq_reg, u64 eq_db);
+void xsc_get_db_addr(struct xsc_core_device *xdev,
+ u64 *tx_db, u64 *rx_db, u64 *cq_db, u64 *cq_reg, u64 *eq_db);
+
+void xsc_update_tx_db(struct xsc_core_device *xdev, u32 sqn, u32 next_pid);
+void xsc_update_rx_db(struct xsc_core_device *xdev, u32 rqn, u32 next_pid);
+void xsc_arm_cq(struct xsc_core_device *xdev, u32 cqn, u32 next_cid, u8 solicited);
+void xsc_update_cq_ci(struct xsc_core_device *xdev, u32 cqn, u32 next_cid);
+void xsc_update_eq_ci(struct xsc_core_device *xdev, u32 eqn, u32 next_cid, u8 arm);
+
+void xsc_update_cmdq_req_pid(struct xsc_core_device *xdev, u32 req_pid);
+void xsc_update_cmdq_rsp_cid(struct xsc_core_device *xdev, u32 rsp_cid);
+u32 xsc_get_cmdq_req_pid(struct xsc_core_device *xdev);
+u32 xsc_get_cmdq_req_cid(struct xsc_core_device *xdev);
+u32 xsc_get_cmdq_rsp_pid(struct xsc_core_device *xdev);
+u32 xsc_get_cmdq_rsp_cid(struct xsc_core_device *xdev);
+void xsc_set_cmdq_req_buf_addr(struct xsc_core_device *xdev, u32 haddr, u32 laddr);
+void xsc_set_cmdq_rsp_buf_addr(struct xsc_core_device *xdev, u32 haddr, u32 laddr);
+void xsc_set_cmdq_msix_vector(struct xsc_core_device *xdev, u32 vector);
+void xsc_check_cmdq_status(struct xsc_core_device *xdev);
+int xsc_handle_cmdq_interrupt(struct xsc_core_device *xdev);
+
+u8 xsc_get_mr_page_mode(struct xsc_core_device *xdev, u8 page_shift);
+u32 xsc_mkey_to_idx(struct xsc_core_device *xdev, u32 mkey);
+u32 xsc_idx_to_mkey(struct xsc_core_device *xdev, u32 mkey_idx);
+
+u8 xsc_get_cqe_opcode(struct xsc_core_device *xdev, void *cqe);
+u64 xsc_get_rdma_stat_mask(struct xsc_core_device *xdev, u16 ver);
+void xsc_set_data_seg(struct xsc_core_device *xdev, void *data_seg, u64 addr, u32 key, u32 length);
+u32 xsc_get_max_qp_depth(struct xsc_core_device *xdev);
+bool xsc_check_max_qp_depth(struct xsc_core_device *xdev, u32 *wqe_cnt, u32 max_qp_depth);
+u32 xsc_get_recv_ds_num(struct xsc_core_device *xdev);
+void *xsc_get_msg_opcode(struct xsc_core_device *xdev);
+u64 xsc_get_max_mr_size(struct xsc_core_device *xdev);
+void xsc_get_multidb_info(struct xsc_core_device *xdev, u32 *num, u32 *base);
+void xsc_set_wqe_id(struct xsc_core_device *dev, void *cseg, u32 wqe_id);
+int xsc_get_cqe_wqe_id(struct xsc_core_device *dev, void *cqe);
+
+enum {
+ CQ_STAT_FIRED,
+ CQ_STAT_KEEP,
+ CQ_STAT_ARM_NEXT,
+ CQ_STAT_ARM_SOLICITED,
+};
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hw.c b/drivers/net/ethernet/yunsilicon/xsc/pci/hw.c
new file mode 100644
index 0000000000000..321e236e8e667
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hw.c
@@ -0,0 +1,310 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/module.h>
+#include <linux/vmalloc.h>
+
+#include "common/xsc_cmd_api.h"
+#include "hw.h"
+#include "hal/xsc_hal.h"
+
+#define MAX_BOARD_NUM 32
+
+static struct xsc_board_info *board_info[MAX_BOARD_NUM];
+
+static struct xsc_board_info *xsc_get_board_info(char *board_sn)
+{
+ int i;
+
+ for (i = 0; i < MAX_BOARD_NUM; i++) {
+ if (!board_info[i])
+ continue;
+ if (!strncmp(board_info[i]->board_sn, board_sn,
+ XSC_BOARD_SN_LEN))
+ return board_info[i];
+ }
+
+ return NULL;
+}
+
+static struct xsc_board_info *xsc_alloc_board_info(void)
+{
+ int i;
+
+ for (i = 0; i < MAX_BOARD_NUM; i++) {
+ if (!board_info[i])
+ break;
+ }
+ if (i == MAX_BOARD_NUM)
+ return NULL;
+ board_info[i] = vzalloc(sizeof(*board_info[i]));
+ if (!board_info[i])
+ return NULL;
+ board_info[i]->board_id = i;
+ return board_info[i];
+}
+
+void xsc_free_board_info(void)
+{
+ int i;
+
+ for (i = 0; i < MAX_BOARD_NUM; i++)
+ vfree(board_info[i]);
+}
+
+int xsc_cmd_query_hca_cap(struct xsc_core_device *xdev,
+ struct xsc_caps *caps)
+{
+ struct xsc_cmd_query_hca_cap_mbox_out *out;
+ struct xsc_cmd_query_hca_cap_mbox_in in;
+ struct xsc_board_info *board_info;
+ int err;
+ u16 t16;
+
+ out = kzalloc_obj(*out, GFP_KERNEL);
+ if (!out)
+ return -ENOMEM;
+
+ memset(&in, 0, sizeof(in));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_HCA_CAP);
+ in.cpu_num = cpu_to_be16(num_online_cpus());
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), out, sizeof(*out));
+ if (err)
+ goto err_free;
+
+ xdev->glb_func_id = be32_to_cpu(out->hca_cap.glb_func_id);
+ caps->pcie0_pf_funcid_base =
+ be16_to_cpu(out->hca_cap.pcie0_pf_funcid_base);
+ caps->pcie0_pf_funcid_top =
+ be16_to_cpu(out->hca_cap.pcie0_pf_funcid_top);
+ caps->pcie1_pf_funcid_base =
+ be16_to_cpu(out->hca_cap.pcie1_pf_funcid_base);
+ caps->pcie1_pf_funcid_top =
+ be16_to_cpu(out->hca_cap.pcie1_pf_funcid_top);
+ caps->funcid_to_logic_port =
+ be16_to_cpu(out->hca_cap.funcid_to_logic_port);
+
+ caps->pcie_host = out->hca_cap.pcie_host;
+ caps->nif_port_num = out->hca_cap.nif_port_num;
+ caps->hw_feature_flag = be32_to_cpu(out->hca_cap.hw_feature_flag);
+
+ caps->raweth_qp_id_base = be16_to_cpu(out->hca_cap.raweth_qp_id_base);
+ caps->raweth_qp_id_end = be16_to_cpu(out->hca_cap.raweth_qp_id_end);
+ caps->raweth_rss_qp_id_base =
+ be16_to_cpu(out->hca_cap.raweth_rss_qp_id_base);
+ caps->raw_tpe_qp_num = be16_to_cpu(out->hca_cap.raw_tpe_qp_num);
+ caps->max_cqes = 1 << out->hca_cap.log_max_cq_sz;
+ caps->max_wqes = 1 << out->hca_cap.log_max_qp_sz;
+ caps->max_sq_desc_sz = be16_to_cpu(out->hca_cap.max_desc_sz_sq);
+ caps->max_rq_desc_sz = be16_to_cpu(out->hca_cap.max_desc_sz_rq);
+ caps->flags = be64_to_cpu(out->hca_cap.flags);
+ caps->stat_rate_support = be16_to_cpu(out->hca_cap.stat_rate_support);
+ caps->log_max_msg = out->hca_cap.log_max_msg & 0x1f;
+ caps->num_ports = out->hca_cap.num_ports & 0xf;
+ caps->log_max_cq = out->hca_cap.log_max_cq & 0x1f;
+ caps->log_max_eq = out->hca_cap.log_max_eq & 0xf;
+ caps->log_max_msix = out->hca_cap.log_max_msix & 0xf;
+ caps->mac_port = out->hca_cap.mac_port & 0xff;
+ xdev->mac_port = caps->mac_port;
+ if (caps->num_ports > XSC_MAX_FW_PORTS) {
+ pci_err(xdev->pdev, "device has %d ports while the driver supports max %d ports\n",
+ caps->num_ports, XSC_MAX_FW_PORTS);
+ err = -EINVAL;
+ goto err_free;
+ }
+ caps->send_ds_num = out->hca_cap.send_seg_num;
+ caps->send_wqe_shift = out->hca_cap.send_wqe_shift;
+ caps->recv_ds_num = out->hca_cap.recv_seg_num;
+ caps->recv_wqe_shift = out->hca_cap.recv_wqe_shift;
+
+ caps->embedded_cpu = 0;
+ caps->ecpf_vport_exists = 0;
+ caps->log_max_current_uc_list = 0;
+ caps->log_max_current_mc_list = 0;
+ caps->log_max_vlan_list = 8;
+ caps->log_max_qp = out->hca_cap.log_max_qp & 0x1f;
+ caps->log_max_mkey = out->hca_cap.log_max_mkey & 0x3f;
+ caps->log_max_pd = out->hca_cap.log_max_pd & 0x1f;
+ caps->log_max_srq = out->hca_cap.log_max_srqs & 0x1f;
+ caps->local_ca_ack_delay = out->hca_cap.local_ca_ack_delay & 0x1f;
+ caps->log_max_mcg = out->hca_cap.log_max_mcg;
+ caps->log_max_mtt = out->hca_cap.log_max_mtt;
+ caps->log_max_tso = out->hca_cap.log_max_tso;
+ caps->hca_core_clock = be32_to_cpu(out->hca_cap.hca_core_clock);
+ caps->max_rwq_indirection_tables =
+ be32_to_cpu(out->hca_cap.max_rwq_indirection_tables);
+ caps->max_rwq_indirection_table_size =
+ be32_to_cpu(out->hca_cap.max_rwq_indirection_table_size);
+ caps->max_qp_mcg = be16_to_cpu(out->hca_cap.max_qp_mcg);
+ caps->max_ra_res_qp = 1 << (out->hca_cap.log_max_ra_res_qp & 0x3f);
+ caps->max_ra_req_qp = 1 << (out->hca_cap.log_max_ra_req_qp & 0x3f);
+ caps->max_srq_wqes = 1 << out->hca_cap.log_max_srq_sz;
+ caps->rx_pkt_len_max = be32_to_cpu(out->hca_cap.rx_pkt_len_max);
+ caps->max_vfs = be16_to_cpu(out->hca_cap.max_vfs);
+ caps->qp_rate_limit_min = be32_to_cpu(out->hca_cap.qp_rate_limit_min);
+ caps->qp_rate_limit_max = be32_to_cpu(out->hca_cap.qp_rate_limit_max);
+
+ caps->msix_enable = 1;
+ caps->msix_base = be16_to_cpu(out->hca_cap.msix_base);
+ caps->msix_num = be16_to_cpu(out->hca_cap.msix_num);
+
+ t16 = be16_to_cpu(out->hca_cap.bf_log_bf_reg_size);
+ if (t16 & 0x8000) {
+ caps->bf_reg_size = 1 << (t16 & 0x1f);
+ caps->bf_regs_per_page = XSC_BF_REGS_PER_PAGE;
+ } else {
+ caps->bf_reg_size = 0;
+ caps->bf_regs_per_page = 0;
+ }
+ caps->min_page_sz = ~(u32)((1 << PAGE_SHIFT) - 1);
+
+ caps->dcbx = 1;
+ caps->qos = 1;
+ caps->ets = 1;
+ caps->dscp = 1;
+ caps->max_tc = out->hca_cap.max_tc;
+ caps->log_max_qp_depth = out->hca_cap.log_max_qp_depth & 0xff;
+ caps->mac_bit = out->hca_cap.mac_bit;
+ caps->lag_logic_port_ofst = out->hca_cap.lag_logic_port_ofst;
+
+ caps->max_qp = be32_to_cpu(out->hca_cap.max_qp);
+ caps->max_cq = be32_to_cpu(out->hca_cap.max_cq);
+ caps->max_pd = be32_to_cpu(out->hca_cap.max_pd);
+
+ if (be16_to_cpu(out->hdr.ver) >= CMD_QUERY_HCA_CAP_V2)
+ caps->max_mtt = be32_to_cpu(out->hca_cap.max_mtt);
+ else
+ caps->max_mtt = 1 << (out->hca_cap.log_max_mtt & 0x3f);
+
+ if (be16_to_cpu(out->hdr.ver) >= CMD_QUERY_HCA_CAP_V1) {
+ caps->max_mr_size = be64_to_cpu(out->hca_cap.max_mr_size);
+ caps->max_cmd_in_len = be16_to_cpu(out->hca_cap.max_cmd_in_len);
+ caps->max_cmd_out_len = be16_to_cpu(out->hca_cap.max_cmd_out_len);
+ } else {
+ caps->max_mr_size = (1ull << 32) - 1;
+ caps->max_cmd_in_len = 10240;
+ caps->max_cmd_out_len = 2048;
+ }
+
+ if (be16_to_cpu(out->hdr.ver) >= CMD_QUERY_HCA_CAP_V3) {
+ caps->mpt_tbl_addr = be32_to_cpu(out->hca_cap.mpt_tbl_addr);
+ caps->mpt_tbl_depth = be32_to_cpu(out->hca_cap.mpt_tbl_depth);
+ caps->mpt_tbl_width = be32_to_cpu(out->hca_cap.mpt_tbl_width);
+ caps->mtt_inst_base_addr = be32_to_cpu(out->hca_cap.mtt_inst_base_addr);
+ caps->mtt_inst_stride = be32_to_cpu(out->hca_cap.mtt_inst_stride);
+ caps->mtt_inst_num_log = be32_to_cpu(out->hca_cap.mtt_inst_num_log);
+ caps->mtt_inst_depth = be32_to_cpu(out->hca_cap.mtt_inst_depth);
+ xsc_set_mtt_info(xdev);
+ }
+
+ xdev->pcie_no = out->hca_cap.pcie_no;
+ xdev->pf_id = out->hca_cap.pf_id;
+
+ if (be16_to_cpu(out->hdr.ver) >= CMD_QUERY_HCA_CAP_V9) {
+ caps->mdb_base = be64_to_cpu(out->hca_cap.mdb_base);
+ caps->mdb_num = be32_to_cpu(out->hca_cap.mdb_num);
+ }
+
+ xdev->chip_ver_h = be32_to_cpu(out->hca_cap.chip_ver_h);
+ xdev->chip_ver_m = be32_to_cpu(out->hca_cap.chip_ver_m);
+ xdev->chip_ver_l = be32_to_cpu(out->hca_cap.chip_ver_l);
+ xdev->hotfix_num = be32_to_cpu(out->hca_cap.hotfix_num);
+ xdev->feature_flag = be32_to_cpu(out->hca_cap.feature_flag);
+
+ board_info = xsc_get_board_info(out->hca_cap.board_sn);
+ if (!board_info) {
+ board_info = xsc_alloc_board_info();
+ if (!board_info) {
+ err = -ENOMEM;
+ goto err_free;
+ }
+
+ memcpy(board_info->board_sn,
+ out->hca_cap.board_sn,
+ sizeof(out->hca_cap.board_sn));
+ }
+
+ xdev->board_info = board_info;
+
+ xsc_set_pf_db_addr(xdev, be64_to_cpu(out->hca_cap.tx_db),
+ be64_to_cpu(out->hca_cap.rx_db),
+ be64_to_cpu(out->hca_cap.complete_db),
+ be64_to_cpu(out->hca_cap.complete_reg),
+ be64_to_cpu(out->hca_cap.event_db));
+
+ xdev->fw_version_major = out->hca_cap.fw_ver.fw_version_major;
+ xdev->fw_version_minor = out->hca_cap.fw_ver.fw_version_minor;
+ xdev->fw_version_patch =
+ be16_to_cpu(out->hca_cap.fw_ver.fw_version_patch);
+ xdev->fw_version_tweak =
+ be32_to_cpu(out->hca_cap.fw_ver.fw_version_tweak);
+ xdev->fw_version_extra_flag = out->hca_cap.fw_ver.fw_version_extra_flag;
+
+ kfree(out);
+ return 0;
+err_free:
+ kfree(out);
+ return err;
+}
+
+static int xsc_cmd_query_guid(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd_query_guid_mbox_out out;
+ struct xsc_cmd_query_guid_mbox_in in = {};
+ int err;
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_GUID);
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err)
+ return err;
+
+ xdev->board_info->guid = out.guid;
+ xdev->board_info->guid_valid = 1;
+ return 0;
+}
+
+int xsc_query_guid(struct xsc_core_device *xdev)
+{
+ if (xdev->board_info->guid_valid)
+ return 0;
+
+ return xsc_cmd_query_guid(xdev);
+}
+
+static int xsc_cmd_activate_hw_config(struct xsc_core_device *xdev)
+{
+ struct xsc_cmd_activate_hw_config_mbox_out out;
+ struct xsc_cmd_activate_hw_config_mbox_in in = {};
+ int err = 0;
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_ACTIVATE_HW_CONFIG);
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err)
+ return err;
+
+ xdev->board_info->hw_config_activated = 1;
+ return 0;
+}
+
+int xsc_activate_hw_config(struct xsc_core_device *xdev)
+{
+ if (xdev->board_info->hw_config_activated)
+ return 0;
+
+ return xsc_cmd_activate_hw_config(xdev);
+}
+
+int xsc_reset_function_resource(struct xsc_core_device *xdev)
+{
+ struct xsc_function_reset_mbox_out out;
+ struct xsc_function_reset_mbox_in in;
+
+ memset(&in, 0, sizeof(in));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_FUNCTION_RESET);
+ in.glb_func_id = cpu_to_be16(xdev->glb_func_id);
+
+ return xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/hw.h b/drivers/net/ethernet/yunsilicon/xsc/pci/hw.h
new file mode 100644
index 0000000000000..5843a72c84bb2
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/hw.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_HW_H
+#define __XSC_HW_H
+
+#include "common/xsc_core.h"
+
+void xsc_free_board_info(void);
+int xsc_cmd_query_hca_cap(struct xsc_core_device *xdev,
+ struct xsc_caps *caps);
+int xsc_query_guid(struct xsc_core_device *xdev);
+int xsc_activate_hw_config(struct xsc_core_device *xdev);
+int xsc_reset_function_resource(struct xsc_core_device *xdev);
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/link.c b/drivers/net/ethernet/yunsilicon/xsc/pci/link.c
new file mode 100644
index 0000000000000..b991eddaee270
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/link.c
@@ -0,0 +1,35 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+
+int xsc_get_link_speed(struct xsc_core_device *xdev)
+{
+ struct xsc_event_query_linkinfo_mbox_out out;
+ struct xsc_event_linkinfo linkinfo;
+ struct xsc_inbox_hdr in;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_LINK_INFO);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err || out.hdr.status) {
+ pci_err(xdev->pdev, "failed to get link info, err=%d, status=%d\n",
+ err, out.hdr.status);
+ return MODULE_SPEED_25G;
+ }
+
+ linkinfo = out.ctx;
+ linkinfo.linkspeed = be32_to_cpu(linkinfo.linkspeed);
+
+ return linkinfo.linkspeed;
+}
+EXPORT_SYMBOL_GPL(xsc_get_link_speed);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/main.c b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
new file mode 100644
index 0000000000000..c2a9f4863da78
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/main.c
@@ -0,0 +1,348 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+#include "hw.h"
+#include "qp.h"
+#include "cq.h"
+#include "srq.h"
+#include "eq.h"
+#include "pci_irq.h"
+#include "adev.h"
+#include "hal/xsc_hal.h"
+
+static const struct pci_device_id xsc_pci_id_table[] = {
+ { PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MC_PF_DEV_ID) },
+ { PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MF_HOST_PF_DEV_ID) },
+ { PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MF_SOC_PF_DEV_ID) },
+ { PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MS_PF_DEV_ID) },
+ { PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MV_HOST_PF_DEV_ID) },
+ { PCI_DEVICE(XSC_PCI_VENDOR_ID, XSC_MV_SOC_PF_DEV_ID) },
+ { 0 }
+};
+
+static int xsc_set_dma_caps(struct pci_dev *pdev)
+{
+ int err;
+
+ err = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64));
+ if (!err)
+ dma_set_max_seg_size(&pdev->dev, SZ_2G);
+
+ return err;
+}
+
+static int xsc_pci_init(struct xsc_core_device *xdev,
+ const struct pci_device_id *id)
+{
+ struct pci_dev *pdev = xdev->pdev;
+ void __iomem *bar_base;
+ int bar_num = 0;
+ int err;
+
+ xdev->numa_node = dev_to_node(&pdev->dev);
+
+ err = pci_enable_device(pdev);
+ if (err) {
+ pci_err(pdev, "failed to enable PCI device: err=%d\n", err);
+ goto err_out;
+ }
+
+ err = pci_request_region(pdev, bar_num, KBUILD_MODNAME);
+ if (err) {
+ pci_err(pdev, "failed to request %s pci_region=%d: err=%d\n",
+ KBUILD_MODNAME, bar_num, err);
+ goto err_disable;
+ }
+
+ pci_set_master(pdev);
+
+ err = xsc_set_dma_caps(pdev);
+ if (err) {
+ pci_err(pdev, "failed to set DMA capabilities mask: err=%d\n",
+ err);
+ goto err_clr_master;
+ }
+
+ bar_base = pci_ioremap_bar(pdev, bar_num);
+ if (!bar_base) {
+ pci_err(pdev, "failed to ioremap %s bar%d\n", KBUILD_MODNAME,
+ bar_num);
+ err = -ENOMEM;
+ goto err_clr_master;
+ }
+
+ err = pci_save_state(pdev);
+ if (err) {
+ pci_err(pdev, "pci_save_state failed: err=%d\n", err);
+ goto err_io_unmap;
+ }
+
+ xdev->bar_num = bar_num;
+ xdev->bar = bar_base;
+
+ xsc_init_hal(xdev, pdev->device);
+
+ return 0;
+
+err_io_unmap:
+ pci_iounmap(pdev, bar_base);
+err_clr_master:
+ pci_clear_master(pdev);
+ pci_release_region(pdev, bar_num);
+err_disable:
+ pci_disable_device(pdev);
+err_out:
+ return err;
+}
+
+static void xsc_pci_fini(struct xsc_core_device *xdev)
+{
+ struct pci_dev *pdev = xdev->pdev;
+
+ pci_iounmap(pdev, xdev->bar);
+ pci_clear_master(pdev);
+ pci_release_region(pdev, xdev->bar_num);
+ pci_disable_device(pdev);
+}
+
+static int xsc_dev_res_init(struct xsc_core_device *xdev)
+{
+ struct xsc_dev_resource *dev_res;
+
+ dev_res = kvzalloc_obj(*dev_res, GFP_KERNEL);
+ if (!dev_res)
+ return -ENOMEM;
+
+ xdev->dev_res = dev_res;
+ mutex_init(&dev_res->alloc_mutex);
+ spin_lock_init(&dev_res->mkey_lock);
+
+ if (xsc_create_srq_res(xdev))
+ goto err_free;
+
+ return 0;
+
+err_free:
+ kfree(dev_res);
+ xdev->dev_res = NULL;
+ return -ENOMEM;
+}
+
+static void xsc_dev_res_cleanup(struct xsc_core_device *xdev)
+{
+ xsc_destroy_srq_res(xdev);
+ kfree(xdev->dev_res);
+}
+
+static int xsc_core_dev_init(struct xsc_core_device *xdev)
+{
+ int err;
+
+ err = xsc_dev_res_init(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "xsc dev res init failed %d\n", err);
+ return err;
+ }
+
+ return 0;
+}
+
+static void xsc_core_dev_cleanup(struct xsc_core_device *xdev)
+{
+ xsc_dev_res_cleanup(xdev);
+}
+
+static int xsc_hw_setup(struct xsc_core_device *xdev)
+{
+ int err;
+
+ err = xsc_cmd_init(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "Failed initializing command interface, aborting\n");
+ goto err_out;
+ }
+
+ err = xsc_cmd_version_check(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "Failed to check cmdq version\n");
+ goto err_cmd_cleanup;
+ }
+
+ err = xsc_cmd_query_hca_cap(xdev, &xdev->caps);
+ if (err) {
+ pci_err(xdev->pdev, "Failed to query hca, err=%d\n", err);
+ goto err_cmd_cleanup;
+ }
+
+ err = xsc_query_guid(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "failed to query guid, err=%d\n", err);
+ goto err_cmd_cleanup;
+ }
+
+ err = xsc_activate_hw_config(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "failed to activate hw config, err=%d\n",
+ err);
+ goto err_cmd_cleanup;
+ }
+
+ err = xsc_reset_function_resource(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "Failed to reset function resource\n");
+ goto err_cmd_cleanup;
+ }
+
+ /* Required by the rdma client, which is probed right after this. */
+ xdev->board_info->msg_opcode = xsc_get_msg_opcode(xdev);
+ if (!xdev->board_info->msg_opcode) {
+ pci_err(xdev->pdev, "failed to get msg opcode table\n");
+ err = -ENODEV;
+ goto err_cmd_cleanup;
+ }
+
+ xsc_init_cq_table(xdev);
+ xsc_init_qp_table(xdev);
+ xsc_eq_init(xdev);
+
+ return 0;
+err_cmd_cleanup:
+ xsc_cmd_cleanup(xdev);
+err_out:
+ return err;
+}
+
+static void xsc_hw_cleanup(struct xsc_core_device *xdev)
+{
+ xsc_cmd_cleanup(xdev);
+}
+
+static int xsc_load(struct xsc_core_device *xdev)
+{
+ int err = 0;
+
+ err = xsc_hw_setup(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "xsc_hw_setup failed %d\n", err);
+ goto err_out;
+ }
+
+ err = xsc_irq_eq_create(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "xsc_irq_eq_create failed %d\n", err);
+ goto err_hw_cleanup;
+ }
+
+ err = xsc_adev_init(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "xsc_adev_init failed %d\n", err);
+ goto err_irq_eq_destroy;
+ }
+
+ return 0;
+err_irq_eq_destroy:
+ xsc_irq_eq_destroy(xdev);
+err_hw_cleanup:
+ xsc_hw_cleanup(xdev);
+err_out:
+ return err;
+}
+
+static void xsc_unload(struct xsc_core_device *xdev)
+{
+ xsc_adev_uninit(xdev);
+ xsc_irq_eq_destroy(xdev);
+ xsc_hw_cleanup(xdev);
+}
+
+static int xsc_pci_probe(struct pci_dev *pci_dev,
+ const struct pci_device_id *id)
+{
+ struct xsc_core_device *xdev;
+ int err;
+
+ xdev = kzalloc_obj(*xdev, GFP_KERNEL);
+ if (!xdev)
+ return -ENOMEM;
+
+ xdev->pdev = pci_dev;
+ xdev->device = &pci_dev->dev;
+ xdev->coredev_type = pci_dev->is_virtfn ? XSC_COREDEV_VF : XSC_COREDEV_PF;
+
+ pci_set_drvdata(pci_dev, xdev);
+ err = xsc_pci_init(xdev, id);
+ if (err) {
+ pci_err(pci_dev, "xsc_pci_init failed %d\n", err);
+ goto err_unset_pci_drvdata;
+ }
+
+ err = xsc_core_dev_init(xdev);
+ if (err) {
+ pci_err(pci_dev, "xsc_core_dev_init failed %d\n", err);
+ goto err_pci_fini;
+ }
+
+ err = xsc_load(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "xsc_load failed %d\n", err);
+ goto err_core_dev_cleanup;
+ }
+
+ return 0;
+err_core_dev_cleanup:
+ xsc_core_dev_cleanup(xdev);
+err_pci_fini:
+ xsc_pci_fini(xdev);
+err_unset_pci_drvdata:
+ pci_set_drvdata(pci_dev, NULL);
+ kfree(xdev);
+
+ return err;
+}
+
+static void xsc_pci_remove(struct pci_dev *pci_dev)
+{
+ struct xsc_core_device *xdev = pci_get_drvdata(pci_dev);
+
+ xsc_unload(xdev);
+ xsc_core_dev_cleanup(xdev);
+ xsc_pci_fini(xdev);
+ pci_set_drvdata(pci_dev, NULL);
+ kfree(xdev);
+}
+
+static struct pci_driver xsc_pci_driver = {
+ .name = "xsc-pci",
+ .id_table = xsc_pci_id_table,
+ .probe = xsc_pci_probe,
+ .remove = xsc_pci_remove,
+};
+
+static int __init xsc_init(void)
+{
+ int err;
+
+ err = pci_register_driver(&xsc_pci_driver);
+ if (err) {
+ pr_err("failed to register pci driver\n");
+ return err;
+ }
+
+ return 0;
+}
+
+static void __exit xsc_fini(void)
+{
+ pci_unregister_driver(&xsc_pci_driver);
+ xsc_free_board_info();
+}
+
+module_init(xsc_init);
+module_exit(xsc_fini);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Yunsilicon XSC PCI driver");
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/mr.c b/drivers/net/ethernet/yunsilicon/xsc/pci/mr.c
new file mode 100644
index 0000000000000..5c68f0fd8db47
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/mr.c
@@ -0,0 +1,212 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+#include "hal/xsc_hal.h"
+#include "mr.h"
+
+int xsc_core_create_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr)
+{
+ struct xsc_create_mkey_mbox_in in;
+ struct xsc_create_mkey_mbox_out out;
+ u8 key;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+
+ spin_lock(&xdev->dev_res->mkey_lock);
+ key = 0x80 + xdev->dev_res->mkey_key++;
+ spin_unlock(&xdev->dev_res->mkey_lock);
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_MKEY);
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err) {
+ pci_err(xdev->pdev, "failed to create mkey, err = %d\n", err);
+ return err;
+ }
+
+ if (out.hdr.status) {
+ pci_err(xdev->pdev, "failed to create mkey, status = %d\n", out.hdr.status);
+ return xsc_cmd_status_to_err(&out.hdr);
+ }
+
+ mr->key = xsc_idx_to_mkey(xdev, be32_to_cpu(out.mkey) & 0xffffff) | key;
+ pci_dbg(xdev->pdev, "out 0x%x, key 0x%x, mkey 0x%x\n",
+ be32_to_cpu(out.mkey), key, mr->key);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_create_mkey);
+
+int xsc_core_destroy_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr)
+{
+ struct xsc_destroy_mkey_mbox_in in;
+ struct xsc_destroy_mkey_mbox_out out;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_MKEY);
+ in.mkey = cpu_to_be32(mr->key);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err)
+ return err;
+
+ if (out.hdr.status)
+ return xsc_cmd_status_to_err(&out.hdr);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_destroy_mkey);
+
+static int xsc_set_mpt_via_cmdq(struct xsc_core_device *xdev,
+ struct xsc_register_mr_mbox_in *in_cmd,
+ u32 *mtt_base)
+{
+ struct xsc_register_mr_request *req = &in_cmd->req;
+ struct xsc_set_mpt_mbox_in *in;
+ struct xsc_set_mpt_mbox_out out;
+ u64 mem_size;
+ int err;
+
+ in = kzalloc_obj(*in, GFP_KERNEL);
+ if (!in)
+ return -ENOMEM;
+
+ mem_size = be64_to_cpu(req->len);
+ in->mpt_item.pdn = req->pdn;
+ in->mpt_item.pa_num = req->pa_num;
+ in->mpt_item.len = cpu_to_be32((u32)mem_size);
+ in->mpt_item.len_h = cpu_to_be32((u32)(mem_size >> 32));
+ in->mpt_item.mkey = req->mkey;
+ in->mpt_item.is_gpu = req->is_gpu;
+ in->mpt_item.acc = req->acc;
+ in->mpt_item.page_mode = req->page_mode;
+ in->mpt_item.map_en = req->map_en;
+ in->mpt_item.va_base = req->va_base;
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_MPT);
+
+ memset(&out, 0, sizeof(out));
+ err = xsc_cmd_exec(xdev, in, sizeof(*in), &out, sizeof(out));
+ kfree(in);
+ if (err || out.hdr.status) {
+ pci_err(xdev->pdev, "failed to set mpt, err = %d, status = %d\n",
+ err, out.hdr.status);
+ return -EINVAL;
+ }
+
+ *mtt_base = be32_to_cpu(out.mtt_base);
+ return 0;
+}
+
+#define PA_NUM_PER_CMD 1024
+
+static int xsc_set_mtt_via_cmdq(struct xsc_core_device *xdev,
+ struct xsc_register_mr_mbox_in *in_cmd,
+ u32 mtt_base)
+{
+ struct xsc_register_mr_request *req = &in_cmd->req;
+ struct xsc_set_mtt_mbox_in *seg_in;
+ struct xsc_set_mtt_mbox_out seg_out;
+ int tot_pg_num = be32_to_cpu(req->pa_num);
+ int tot_seg_num, seg_pa_num, seg_idx;
+ int pa_idx_base = 0;
+ int in_len, err, i;
+
+ tot_seg_num = (tot_pg_num & (PA_NUM_PER_CMD - 1)) ?
+ ((tot_pg_num >> 10) + 1) : (tot_pg_num >> 10);
+
+ for (seg_idx = 0; seg_idx < tot_seg_num; seg_idx++) {
+ seg_pa_num = (seg_idx != tot_seg_num - 1) ? PA_NUM_PER_CMD :
+ (tot_pg_num - ((tot_seg_num - 1) << 10));
+ in_len = (seg_pa_num << 3) + sizeof(*seg_in);
+ seg_in = kzalloc(in_len, GFP_KERNEL);
+ if (!seg_in)
+ return -ENOMEM;
+
+ seg_in->mtt_setting.mtt_base = cpu_to_be32(mtt_base);
+ seg_in->mtt_setting.pa_num = cpu_to_be32(seg_pa_num);
+ for (i = 0; i < seg_pa_num; i++)
+ seg_in->mtt_setting.pas[i] = req->pas[pa_idx_base + i];
+ seg_in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_SET_MTT);
+
+ memset(&seg_out, 0, sizeof(seg_out));
+ pci_dbg(xdev->pdev,
+ "set mtt seg %d, pa_num %d, pa_idx_base %d, tot_seg %d\n",
+ seg_idx, seg_pa_num, pa_idx_base, tot_seg_num);
+ err = xsc_cmd_exec(xdev, seg_in, in_len, &seg_out, sizeof(seg_out));
+ kfree(seg_in);
+ if (err || seg_out.hdr.status) {
+ pci_err(xdev->pdev, "failed to set mtt seg %d, err = %d, status = %d\n",
+ seg_idx, err, seg_out.hdr.status);
+ return -EINVAL;
+ }
+
+ pa_idx_base += seg_pa_num;
+ mtt_base += seg_pa_num;
+ }
+
+ return 0;
+}
+
+static int xsc_dereg_mr_via_cmdq(struct xsc_core_device *xdev, u32 mkey)
+{
+ struct xsc_unregister_mr_mbox_in in;
+ struct xsc_unregister_mr_mbox_out out;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DEREG_MR);
+ in.mkey = cpu_to_be32(xsc_mkey_to_idx(xdev, mkey));
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err || out.hdr.status) {
+ pci_err(xdev->pdev, "failed to dereg mr, err = %d, status = %d\n",
+ err, out.hdr.status);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+int xsc_core_register_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr,
+ struct xsc_register_mr_mbox_in *in, int inlen)
+{
+ u32 mtt_base;
+ int err;
+
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_REG_MR);
+
+ err = xsc_set_mpt_via_cmdq(xdev, in, &mtt_base);
+ if (err) {
+ pci_err(xdev->pdev, "failed to set mpt via cmdq\n");
+ return err;
+ }
+
+ err = xsc_set_mtt_via_cmdq(xdev, in, mtt_base);
+ if (err) {
+ pci_err(xdev->pdev, "failed to set mtt via cmdq\n");
+ xsc_dereg_mr_via_cmdq(xdev, be32_to_cpu(in->req.mkey));
+ return err;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_register_mr);
+
+int xsc_core_dereg_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr)
+{
+ return xsc_dereg_mr_via_cmdq(xdev, mr->key);
+}
+EXPORT_SYMBOL_GPL(xsc_core_dereg_mr);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/mr.h b/drivers/net/ethernet/yunsilicon/xsc/pci/mr.h
new file mode 100644
index 0000000000000..edafecda63e12
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/mr.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_MR_H
+#define __XSC_MR_H
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd.h"
+
+int xsc_core_create_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr);
+int xsc_core_destroy_mkey(struct xsc_core_device *xdev, struct xsc_core_mr *mr);
+int xsc_core_register_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr,
+ struct xsc_register_mr_mbox_in *in, int inlen);
+int xsc_core_dereg_mr(struct xsc_core_device *xdev, struct xsc_core_mr *mr);
+
+#endif /* __XSC_MR_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c
new file mode 100644
index 0000000000000..c0f0b9d1f34f0
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.c
@@ -0,0 +1,498 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/irqdomain.h>
+#include <linux/msi.h>
+#include <linux/interrupt.h>
+#include <linux/notifier.h>
+#include <linux/module.h>
+
+#include "common/xsc_cmd_api.h"
+#include "common/xsc_core.h"
+#include "eq.h"
+#include "pci_irq.h"
+#include "hal/xsc_hal.h"
+
+enum {
+ XSC_COMP_EQ_SIZE = 1024,
+};
+
+static void xsc_free_irq(struct xsc_core_device *xdev, unsigned int vector)
+{
+ unsigned int irqn = 0;
+
+ irqn = pci_irq_vector(xdev->pdev, vector);
+ disable_irq(irqn);
+
+ if (xsc_fw_is_available(xdev))
+ free_irq(irqn, xdev);
+}
+
+static int set_comp_irq_affinity_hint(struct xsc_core_device *xdev, int i)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ struct xsc_eq *eq = xsc_core_eq_get(xdev, i);
+ int vecidx = table->eq_vec_comp_base + i;
+ unsigned int irqn;
+ int ret;
+
+ irqn = pci_irq_vector(xdev->pdev, vecidx);
+ if (!zalloc_cpumask_var(&eq->mask, GFP_KERNEL)) {
+ pci_err(xdev->pdev, "zalloc_cpumask_var rx cpumask failed");
+ return -ENOMEM;
+ }
+
+ if (!zalloc_cpumask_var(&xdev->xps_cpumask, GFP_KERNEL)) {
+ pci_err(xdev->pdev, "zalloc_cpumask_var tx cpumask failed");
+ return -ENOMEM;
+ }
+
+ cpumask_set_cpu(cpumask_local_spread(i, xdev->numa_node),
+ xdev->xps_cpumask);
+ ret = irq_set_affinity_hint(irqn, eq->mask);
+
+ return ret;
+}
+
+static void clear_comp_irq_affinity_hint(struct xsc_core_device *xdev, int i)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ struct xsc_eq *eq = xsc_core_eq_get(xdev, i);
+ int vecidx = table->eq_vec_comp_base + i;
+ int irqn;
+
+ irqn = pci_irq_vector(xdev->pdev, vecidx);
+ irq_set_affinity_hint(irqn, NULL);
+ free_cpumask_var(eq->mask);
+}
+
+static int set_comp_irq_affinity_hints(struct xsc_core_device *xdev)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ int nvec = table->num_comp_vectors;
+ int err;
+ int i;
+
+ for (i = 0; i < nvec; i++) {
+ err = set_comp_irq_affinity_hint(xdev, i);
+ if (err)
+ goto err_out;
+ }
+
+ return 0;
+
+err_out:
+ for (i--; i >= 0; i--)
+ clear_comp_irq_affinity_hint(xdev, i);
+ free_cpumask_var(xdev->xps_cpumask);
+
+ return err;
+}
+
+static void clear_comp_irq_affinity_hints(struct xsc_core_device *xdev)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ int nvec = table->num_comp_vectors;
+ int i;
+
+ for (i = 0; i < nvec; i++)
+ clear_comp_irq_affinity_hint(xdev, i);
+ free_cpumask_var(xdev->xps_cpumask);
+}
+
+static int xsc_alloc_irq_vectors(struct xsc_core_device *xdev)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+ struct xsc_eq_table *table = &dev_res->eq_table;
+ int nvec = xdev->caps.msix_num;
+ int nvec_base;
+ int err;
+
+ nvec_base = XSC_EQ_VEC_COMP_BASE;
+ if (nvec <= nvec_base) {
+ pci_err(xdev->pdev, "failed to alloc irq vector(%d)\n", nvec);
+ return -ENOMEM;
+ }
+
+ dev_res->irq_info = kcalloc(nvec, sizeof(*dev_res->irq_info),
+ GFP_KERNEL);
+ if (!dev_res->irq_info)
+ return -ENOMEM;
+
+ nvec = pci_alloc_irq_vectors(xdev->pdev, nvec_base + 1, nvec,
+ PCI_IRQ_MSIX);
+ if (nvec < 0) {
+ err = nvec;
+ goto err_free_irq_info;
+ }
+
+ table->eq_vec_comp_base = nvec_base;
+ table->num_comp_vectors = nvec - nvec_base;
+ xdev->msix_vec_base = xdev->caps.msix_base;
+
+ return 0;
+
+err_free_irq_info:
+ kfree(dev_res->irq_info);
+ return err;
+}
+
+static void xsc_free_irq_vectors(struct xsc_core_device *xdev)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+
+ if (!xsc_fw_is_available(xdev))
+ return;
+
+ pci_free_irq_vectors(xdev->pdev);
+ kfree(dev_res->irq_info);
+}
+
+int xsc_core_vector2eqn(struct xsc_core_device *xdev, int vector, u32 *eqn,
+ unsigned int *irqn)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ struct xsc_eq *eq, *n;
+ int err = -ENOENT;
+
+ if (!xdev->caps.msix_enable)
+ return 0;
+
+ spin_lock(&table->lock);
+ list_for_each_entry_safe(eq, n, &table->comp_eqs_list, list) {
+ if (eq->index == vector) {
+ *eqn = eq->eqn;
+ *irqn = eq->irqn;
+ err = 0;
+ break;
+ }
+ }
+ spin_unlock(&table->lock);
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(xsc_core_vector2eqn);
+
+int xsc_vector2eqn(struct xsc_core_device *xdev, int vector, int *eqn,
+ unsigned int *irqn)
+{
+ u32 cqn = 0;
+ int err;
+
+ err = xsc_core_vector2eqn(xdev, vector, &cqn, irqn);
+ if (!err)
+ *eqn = (int)cqn;
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(xsc_vector2eqn);
+
+static void free_comp_eqs(struct xsc_core_device *xdev)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ struct xsc_eq *eq, *n;
+
+ spin_lock(&table->lock);
+ list_for_each_entry_safe(eq, n, &table->comp_eqs_list, list) {
+ list_del(&eq->list);
+ spin_unlock(&table->lock);
+ if (xsc_destroy_unmap_eq(xdev, eq))
+ pci_err(xdev->pdev, "failed to destroy EQ 0x%x\n",
+ eq->eqn);
+ kfree(eq);
+ spin_lock(&table->lock);
+ }
+ spin_unlock(&table->lock);
+}
+
+static int alloc_comp_eqs(struct xsc_core_device *xdev)
+{
+ struct xsc_eq_table *table = &xdev->dev_res->eq_table;
+ char name[XSC_MAX_IRQ_NAME];
+ struct xsc_eq *eq;
+ int ncomp_vec;
+ u32 nent;
+ int err;
+ int i;
+
+ INIT_LIST_HEAD(&table->comp_eqs_list);
+ ncomp_vec = table->num_comp_vectors;
+ nent = XSC_COMP_EQ_SIZE;
+
+ for (i = 0; i < ncomp_vec; i++) {
+ eq = kzalloc_obj(*eq, GFP_KERNEL);
+ if (!eq) {
+ err = -ENOMEM;
+ goto err_free;
+ }
+
+ snprintf(name, XSC_MAX_IRQ_NAME, "xsc_comp%d", i);
+ err = xsc_create_map_eq(xdev, eq,
+ i + table->eq_vec_comp_base,
+ nent, name);
+ if (err) {
+ kfree(eq);
+ goto err_free;
+ }
+
+ eq->index = i;
+ spin_lock(&table->lock);
+ list_add_tail(&eq->list, &table->comp_eqs_list);
+ spin_unlock(&table->lock);
+ }
+
+ return 0;
+
+err_free:
+ free_comp_eqs(xdev);
+ return err;
+}
+
+static irqreturn_t xsc_cmd_handler(int irq, void *arg)
+{
+ struct xsc_core_device *xdev = (struct xsc_core_device *)arg;
+ int err;
+
+ disable_irq_nosync(xdev->cmd.irqn);
+ err = xsc_cmd_err_handler(xdev);
+ if (!err)
+ xsc_cmd_resp_handler(xdev);
+ enable_irq(xdev->cmd.irqn);
+
+ return IRQ_HANDLED;
+}
+
+static int xsc_request_irq_for_cmdq(struct xsc_core_device *xdev, u8 vecidx)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+
+ xsc_set_cmdq_msix_vector(xdev, xdev->msix_vec_base + vecidx);
+
+ snprintf(dev_res->irq_info[vecidx].name, XSC_MAX_IRQ_NAME, "%s@pci:%s",
+ "xsc_cmd", pci_name(xdev->pdev));
+ xdev->cmd.irqn = pci_irq_vector(xdev->pdev, vecidx);
+ return request_irq(xdev->cmd.irqn, xsc_cmd_handler, 0,
+ dev_res->irq_info[vecidx].name, xdev);
+}
+
+static void xsc_free_irq_for_cmdq(struct xsc_core_device *xdev)
+{
+ xsc_free_irq(xdev, XSC_VEC_CMD);
+}
+
+/*
+ * Handle an asynchronous event. This runs on the PCI layer's own workqueue,
+ * so it outlives any auxiliary driver: the driver only ever registers a
+ * callback here, and the memory this touches belongs to xsc_core_device
+ * rather than to the eth or RDMA private area.
+ */
+static void xsc_event_work(struct work_struct *work)
+{
+ struct xsc_core_device *xdev = container_of(work, struct xsc_core_device,
+ event_work);
+ struct xsc_event_query_type_mbox_out out;
+ struct xsc_event_query_type_mbox_in in = {};
+ int err;
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_EVENT_TYPE);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err || out.hdr.status) {
+ pci_err(xdev->pdev, "failed to query event type, err=%d, status=%d\n",
+ err, out.hdr.status);
+ return;
+ }
+
+ switch (out.ctx.resp_cmd_type) {
+ case XSC_CMD_EVENT_RESP_CHANGE_LINK:
+ if (xdev->link_event_handler)
+ xdev->link_event_handler(xdev);
+ break;
+ case XSC_CMD_EVENT_RESP_TEMP_WARN:
+ pci_warn(xdev->pdev, "[Minor]nic chip temperature high warning\n");
+ break;
+ case XSC_CMD_EVENT_RESP_OVER_TEMP_PROTECTION:
+ pci_warn(xdev->pdev, "[Critical]nic chip was over-temperature\n");
+ break;
+ default:
+ pci_dbg(xdev->pdev, "unknown event cmdtype=0x%x\n",
+ out.ctx.resp_cmd_type);
+ break;
+ }
+}
+
+static irqreturn_t xsc_event_handler(int irq, void *arg)
+{
+ struct xsc_core_device *xdev = (struct xsc_core_device *)arg;
+
+ queue_work(xdev->event_wq, &xdev->event_work);
+
+ return IRQ_HANDLED;
+}
+
+/*
+ * Register the link event callback.
+ *
+ * The events are dispatched from the PCI layer's own workqueue, whose
+ * contents are not the client's business (it also carries the temperature
+ * warnings, which are logged here), so the client must not touch
+ * xdev->event_wq itself.
+ */
+void xsc_irq_event_register(struct xsc_core_device *xdev,
+ void (*handler)(void *xdev))
+{
+ xdev->link_event_handler = handler;
+}
+EXPORT_SYMBOL_GPL(xsc_irq_event_register);
+
+/*
+ * Detach the callback and wait for an event that is already in flight to
+ * finish with it, in the same way free_irq() both unregisters and waits.
+ * Must be called before the caller frees whatever its handler dereferences.
+ * Safe to call when no handler is registered.
+ */
+void xsc_irq_event_unregister(struct xsc_core_device *xdev)
+{
+ xdev->link_event_handler = NULL;
+
+ /* Only exists once MSI-X, which requests the event IRQ, is up. */
+ if (xdev->event_wq)
+ flush_workqueue(xdev->event_wq);
+}
+EXPORT_SYMBOL_GPL(xsc_irq_event_unregister);
+
+static int xsc_request_irq_for_event(struct xsc_core_device *xdev)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+
+ snprintf(dev_res->irq_info[XSC_VEC_CMD_EVENT].name,
+ XSC_MAX_IRQ_NAME, "%s@pci:%s",
+ "xsc_eth_event", pci_name(xdev->pdev));
+ return request_irq(pci_irq_vector(xdev->pdev, XSC_VEC_CMD_EVENT),
+ xsc_event_handler, 0,
+ dev_res->irq_info[XSC_VEC_CMD_EVENT].name, xdev);
+}
+
+static void xsc_free_irq_for_event(struct xsc_core_device *xdev)
+{
+ xsc_free_irq(xdev, XSC_VEC_CMD_EVENT);
+}
+
+static int xsc_cmd_enable_msix(struct xsc_core_device *xdev)
+{
+ struct xsc_msix_table_info_mbox_out out;
+ struct xsc_msix_table_info_mbox_in in;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_ENABLE_MSIX);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err) {
+ pci_err(xdev->pdev, "xsc_cmd_exec enable msix failed %d\n",
+ err);
+ return err;
+ }
+
+ return 0;
+}
+
+int xsc_irq_eq_create(struct xsc_core_device *xdev)
+{
+ int err;
+
+ if (xdev->caps.msix_enable == 0)
+ return 0;
+
+ err = xsc_alloc_irq_vectors(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "enable msix failed, err=%d\n", err);
+ goto err_out;
+ }
+
+ err = xsc_start_eqs(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "failed to start EQs, err=%d\n", err);
+ goto err_free_irq_vectors;
+ }
+
+ err = alloc_comp_eqs(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "failed to alloc comp EQs, err=%d\n", err);
+ goto err_stop_eqs;
+ }
+
+ err = xsc_request_irq_for_cmdq(xdev, XSC_VEC_CMD);
+ if (err) {
+ pci_err(xdev->pdev, "failed to request irq for cmdq, err=%d\n",
+ err);
+ goto err_free_comp_eqs;
+ }
+
+ xdev->event_wq = create_singlethread_workqueue("xsc_event");
+ if (!xdev->event_wq) {
+ pci_err(xdev->pdev, "failed to create event workqueue\n");
+ err = -ENOMEM;
+ goto err_free_irq_cmdq;
+ }
+ INIT_WORK(&xdev->event_work, xsc_event_work);
+
+ err = xsc_request_irq_for_event(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "failed to request irq for event, err=%d\n",
+ err);
+ goto err_destroy_event_wq;
+ }
+
+ err = set_comp_irq_affinity_hints(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "failed to alloc affinity hint cpumask, err=%d\n",
+ err);
+ goto err_free_irq_evnt;
+ }
+
+ xsc_cmd_use_events(xdev);
+ err = xsc_cmd_enable_msix(xdev);
+ if (err) {
+ pci_err(xdev->pdev, "xsc_cmd_enable_msix failed %d.\n", err);
+ xsc_cmd_use_polling(xdev);
+ goto err_free_irq_evnt;
+ }
+ return 0;
+
+err_free_irq_evnt:
+ xsc_free_irq_for_event(xdev);
+err_destroy_event_wq:
+ destroy_workqueue(xdev->event_wq);
+err_free_irq_cmdq:
+ xsc_free_irq_for_cmdq(xdev);
+err_free_comp_eqs:
+ free_comp_eqs(xdev);
+err_stop_eqs:
+ xsc_stop_eqs(xdev);
+err_free_irq_vectors:
+ xsc_free_irq_vectors(xdev);
+err_out:
+ return err;
+}
+
+void xsc_irq_eq_destroy(struct xsc_core_device *xdev)
+{
+ if (xdev->caps.msix_enable == 0)
+ return;
+
+ xsc_stop_eqs(xdev);
+ clear_comp_irq_affinity_hints(xdev);
+ free_comp_eqs(xdev);
+
+ /* xsc_free_irq() disables and synchronises the event IRQ, so no new
+ * work can be queued once this returns.
+ */
+ xsc_free_irq_for_event(xdev);
+ destroy_workqueue(xdev->event_wq);
+ xsc_free_irq_for_cmdq(xdev);
+ xsc_free_irq_vectors(xdev);
+}
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.h b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.h
new file mode 100644
index 0000000000000..750251d1b90e6
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/pci_irq.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_PCI_IRQ_H
+#define __XSC_PCI_IRQ_H
+
+#include "common/xsc_core.h"
+
+int xsc_irq_eq_create(struct xsc_core_device *xdev);
+void xsc_irq_eq_destroy(struct xsc_core_device *xdev);
+int xsc_core_vector2eqn(struct xsc_core_device *xdev, int vector, u32 *eqn,
+ unsigned int *irqn);
+int xsc_vector2eqn(struct xsc_core_device *xdev, int vector, int *eqn,
+ unsigned int *irqn);
+
+#endif
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/pd.c b/drivers/net/ethernet/yunsilicon/xsc/pci/pd.c
new file mode 100644
index 0000000000000..7969dfefc2e13
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/pd.c
@@ -0,0 +1,55 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+#include "pd.h"
+
+int xsc_core_alloc_pd(struct xsc_core_device *xdev, u32 *pdn)
+{
+ struct xsc_alloc_pd_mbox_in in;
+ struct xsc_alloc_pd_mbox_out out;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_ALLOC_PD);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err)
+ return err;
+
+ if (out.hdr.status)
+ return xsc_cmd_status_to_err(&out.hdr);
+
+ *pdn = be32_to_cpu(out.pdn) & 0xffffff;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_alloc_pd);
+
+int xsc_core_dealloc_pd(struct xsc_core_device *xdev, u32 pdn)
+{
+ struct xsc_dealloc_pd_mbox_in in;
+ struct xsc_dealloc_pd_mbox_out out;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DEALLOC_PD);
+ in.pdn = cpu_to_be32(pdn);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err)
+ return err;
+
+ if (out.hdr.status)
+ return xsc_cmd_status_to_err(&out.hdr);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_dealloc_pd);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/pd.h b/drivers/net/ethernet/yunsilicon/xsc/pci/pd.h
new file mode 100644
index 0000000000000..d9a99baea3f5b
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/pd.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_PD_H
+#define __XSC_PD_H
+
+#include "common/xsc_core.h"
+
+int xsc_core_alloc_pd(struct xsc_core_device *xdev, u32 *pdn);
+int xsc_core_dealloc_pd(struct xsc_core_device *xdev, u32 pdn);
+
+#endif /* __XSC_PD_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/qp.c b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.c
new file mode 100644
index 0000000000000..8ea57a09f6ceb
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.c
@@ -0,0 +1,356 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/gfp.h>
+#include <linux/time.h>
+#include <linux/export.h>
+#include <linux/kthread.h>
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+#include "qp.h"
+
+int xsc_core_create_resource_common(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp)
+{
+ struct xsc_qp_table *table = &xdev->dev_res->qp_table;
+ int err;
+
+ spin_lock_irq(&table->lock);
+ err = radix_tree_insert(&table->tree, qp->qpn, qp);
+ spin_unlock_irq(&table->lock);
+ if (err)
+ return err;
+
+ atomic_set(&qp->refcount, 1);
+ init_completion(&qp->free);
+ qp->pid = current->pid;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_create_resource_common);
+
+void xsc_core_destroy_resource_common(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp)
+{
+ struct xsc_qp_table *table = &xdev->dev_res->qp_table;
+ unsigned long flags;
+
+ spin_lock_irqsave(&table->lock, flags);
+ radix_tree_delete(&table->tree, qp->qpn);
+ spin_unlock_irqrestore(&table->lock, flags);
+
+ if (atomic_dec_and_test(&qp->refcount))
+ complete(&qp->free);
+ wait_for_completion(&qp->free);
+}
+EXPORT_SYMBOL_GPL(xsc_core_destroy_resource_common);
+
+void xsc_qp_event(struct xsc_core_device *xdev, u32 qpn, int event_type)
+{
+ struct xsc_qp_table *table = &xdev->dev_res->qp_table;
+ struct xsc_core_qp *qp;
+
+ spin_lock(&table->lock);
+
+ qp = radix_tree_lookup(&table->tree, qpn);
+ if (qp)
+ atomic_inc(&qp->refcount);
+
+ spin_unlock(&table->lock);
+
+ if (!qp) {
+ pci_err(xdev->pdev, "Async event for bogus QP 0x%x\n", qpn);
+ return;
+ }
+
+ qp->event(qp, event_type);
+
+ if (atomic_dec_and_test(&qp->refcount))
+ complete(&qp->free);
+}
+
+void xsc_init_qp_table(struct xsc_core_device *xdev)
+{
+ struct xsc_qp_table *table = &xdev->dev_res->qp_table;
+
+ spin_lock_init(&table->lock);
+ INIT_RADIX_TREE(&table->tree, GFP_ATOMIC);
+}
+
+int xsc_core_eth_create_qp(struct xsc_core_device *xdev,
+ struct xsc_create_qp_mbox_in *in,
+ int insize, u32 *p_qpn)
+{
+ struct xsc_create_qp_mbox_out out;
+ int ret;
+
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_QP);
+ ret = xsc_cmd_exec(xdev, in, insize, &out, sizeof(out));
+ if (ret) {
+ pci_err(xdev->pdev, "failed to create sq, err=%d out.status=%u\n",
+ ret, out.hdr.status);
+ return ret;
+ }
+
+ *p_qpn = be32_to_cpu(out.qpn) & 0xffffff;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eth_create_qp);
+
+int xsc_core_eth_modify_qp_status(struct xsc_core_device *xdev,
+ u32 qpn, u16 status)
+{
+ struct xsc_modify_qp_mbox_out out;
+ struct xsc_modify_qp_mbox_in in = {};
+ int ret = 0;
+
+ in.hdr.opcode = cpu_to_be16(status);
+ in.qpn = cpu_to_be32(qpn);
+ in.no_need_wait = 1;
+
+ ret = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (ret) {
+ pci_err(xdev->pdev, "failed to modify qp %u status=%u, err=%d out.status %u\n",
+ qpn, status, ret, out.hdr.status);
+ return ret;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eth_modify_qp_status);
+
+int xsc_core_eth_destroy_qp(struct xsc_core_device *xdev, u32 qpn)
+{
+ struct xsc_destroy_qp_mbox_out out;
+ struct xsc_destroy_qp_mbox_in in;
+ int err;
+
+ err = xsc_core_eth_modify_qp_status(xdev, qpn, XSC_CMD_OP_2RST_QP);
+ if (err) {
+ pci_err(xdev->pdev, "failed to set sq%d status=rst, err=%d\n",
+ qpn, err);
+ return err;
+ }
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_QP);
+ in.qpn = cpu_to_be32(qpn);
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err) {
+ pci_err(xdev->pdev, "failed to destroy sq%d, err=%d out.status=%u\n",
+ qpn, err, out.hdr.status);
+ return err;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eth_destroy_qp);
+
+int xsc_core_eth_modify_raw_qp(struct xsc_core_device *xdev,
+ struct xsc_modify_raw_qp_mbox_in *in)
+{
+ struct xsc_modify_raw_qp_mbox_out out;
+ int err;
+
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_MODIFY_RAW_QP);
+
+ err = xsc_cmd_exec(xdev, in, sizeof(struct xsc_modify_raw_qp_mbox_in),
+ &out, sizeof(struct xsc_modify_raw_qp_mbox_out));
+ if (err) {
+ pci_err(xdev->pdev, "failed to modify sq, err=%d out.status=%u\n",
+ err, out.hdr.status);
+ return err;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eth_modify_raw_qp);
+
+int xsc_core_eth_create_rss_qp_rqs(struct xsc_core_device *xdev,
+ struct xsc_create_multiqp_mbox_in *in,
+ int insize,
+ u32 *p_qpn_base)
+{
+ struct xsc_create_multiqp_mbox_out out;
+ int err;
+
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_MULTI_QP);
+ err = xsc_cmd_exec(xdev, in, insize, &out, sizeof(out));
+ if (err) {
+ pci_err(xdev->pdev,
+ "failed to create rss rq, qp_num=%d, type=%d, err=%d out.status=%u\n",
+ in->qp_num, in->qp_type, err, out.hdr.status);
+ return err;
+ }
+
+ *p_qpn_base = be32_to_cpu(out.qpn_base) & 0xffffff;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_eth_create_rss_qp_rqs);
+
+int xsc_core_create_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp,
+ struct xsc_create_qp_mbox_in *in,
+ int inlen)
+{
+ struct xsc_create_qp_mbox_out out;
+ struct xsc_destroy_qp_mbox_in din;
+ struct xsc_destroy_qp_mbox_out dout;
+ int err;
+
+ memset(&out, 0, sizeof(out));
+ in->hdr.opcode = cpu_to_be16(XSC_CMD_OP_CREATE_QP);
+
+ err = xsc_cmd_exec(xdev, in, inlen, &out, sizeof(out));
+ if (err)
+ return err;
+
+ if (out.hdr.status)
+ return xsc_cmd_status_to_err(&out.hdr);
+
+ qp->qpn = be32_to_cpu(out.qpn) & 0xffffff;
+
+ err = xsc_core_create_resource_common(xdev, qp);
+ if (err)
+ goto err_cmd;
+
+ atomic_inc(&xdev->num_qps);
+ return 0;
+
+err_cmd:
+ memset(&din, 0, sizeof(din));
+ memset(&dout, 0, sizeof(dout));
+ din.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_QP);
+ din.qpn = cpu_to_be32(qp->qpn);
+ xsc_cmd_exec(xdev, &din, sizeof(din), &dout, sizeof(dout));
+
+ return err;
+}
+EXPORT_SYMBOL_GPL(xsc_core_create_qp);
+
+int xsc_core_destroy_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp)
+{
+ struct xsc_destroy_qp_mbox_in in;
+ struct xsc_destroy_qp_mbox_out out;
+ int err;
+
+ xsc_core_destroy_resource_common(xdev, qp);
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_DESTROY_QP);
+ in.qpn = cpu_to_be32(qp->qpn);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err)
+ return err;
+
+ if (out.hdr.status)
+ return xsc_cmd_status_to_err(&out.hdr);
+
+ atomic_dec(&xdev->num_qps);
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_destroy_qp);
+
+int xsc_core_qp_modify(struct xsc_core_device *xdev, enum xsc_qp_state cur_state,
+ enum xsc_qp_state new_state,
+ struct xsc_modify_qp_mbox_in *in, int sqd_event,
+ struct xsc_core_qp *qp)
+{
+ static const u16 optab[XSC_QP_NUM_STATE][XSC_QP_NUM_STATE] = {
+ [XSC_QP_STATE_RST] = {
+ [XSC_QP_STATE_RST] = XSC_CMD_OP_2RST_QP,
+ [XSC_QP_STATE_ERR] = XSC_CMD_OP_2ERR_QP,
+ [XSC_QP_STATE_INIT] = XSC_CMD_OP_RST2INIT_QP,
+ },
+ [XSC_QP_STATE_INIT] = {
+ [XSC_QP_STATE_RST] = XSC_CMD_OP_2RST_QP,
+ [XSC_QP_STATE_ERR] = XSC_CMD_OP_2ERR_QP,
+ [XSC_QP_STATE_INIT] = XSC_CMD_OP_INIT2INIT_QP,
+ [XSC_QP_STATE_RTR] = XSC_CMD_OP_INIT2RTR_QP,
+ },
+ [XSC_QP_STATE_RTR] = {
+ [XSC_QP_STATE_RST] = XSC_CMD_OP_2RST_QP,
+ [XSC_QP_STATE_ERR] = XSC_CMD_OP_2ERR_QP,
+ [XSC_QP_STATE_RTS] = XSC_CMD_OP_RTR2RTS_QP,
+ },
+ [XSC_QP_STATE_RTS] = {
+ [XSC_QP_STATE_RST] = XSC_CMD_OP_2RST_QP,
+ [XSC_QP_STATE_ERR] = XSC_CMD_OP_2ERR_QP,
+ [XSC_QP_STATE_RTS] = XSC_CMD_OP_RTS2RTS_QP,
+ [XSC_QP_STATE_SQD] = XSC_CMD_OP_RTS2SQD_QP,
+ },
+ [XSC_QP_STATE_SQD] = {
+ [XSC_QP_STATE_RST] = XSC_CMD_OP_2RST_QP,
+ [XSC_QP_STATE_ERR] = XSC_CMD_OP_2ERR_QP,
+ [XSC_QP_STATE_RTS] = XSC_CMD_OP_SQD2RTS_QP,
+ [XSC_QP_STATE_SQD] = XSC_CMD_OP_SQD2SQD_QP,
+ },
+ [XSC_QP_STATE_SQER] = {
+ [XSC_QP_STATE_RST] = XSC_CMD_OP_2RST_QP,
+ [XSC_QP_STATE_ERR] = XSC_CMD_OP_2ERR_QP,
+ [XSC_QP_STATE_RTS] = XSC_CMD_OP_SQERR2RTS_QP,
+ },
+ [XSC_QP_STATE_ERR] = {
+ [XSC_QP_STATE_RST] = XSC_CMD_OP_2RST_QP,
+ [XSC_QP_STATE_ERR] = XSC_CMD_OP_2ERR_QP,
+ }
+ };
+
+ struct xsc_modify_qp_mbox_out out;
+ u16 op;
+
+ if (cur_state >= XSC_QP_NUM_STATE || new_state >= XSC_QP_NUM_STATE ||
+ !optab[cur_state][new_state])
+ return -EINVAL;
+
+ memset(&out, 0, sizeof(out));
+ op = optab[cur_state][new_state];
+
+ if (new_state == XSC_QP_STATE_RTR) {
+ in->ctx.pcie_no = xdev->pcie_no;
+ in->ctx.func_id = cpu_to_be16(xdev->glb_func_id);
+ }
+
+ in->hdr.opcode = cpu_to_be16(op);
+ in->qpn = cpu_to_be32(qp->qpn);
+ in->no_need_wait = 1;
+
+ if (xsc_cmd_exec(xdev, in, sizeof(*in), &out, sizeof(out)))
+ return -ENOEXEC;
+
+ if (out.hdr.status)
+ return xsc_cmd_status_to_err(&out.hdr);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_qp_modify);
+
+int xsc_core_qp_query(struct xsc_core_device *xdev, struct xsc_core_qp *qp,
+ struct xsc_query_qp_mbox_out *out, int outlen)
+{
+ struct xsc_query_qp_mbox_in in;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(out, 0, outlen);
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_QP);
+ in.qpn = cpu_to_be32(qp->qpn);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), out, outlen);
+ if (err)
+ return err;
+
+ if (out->hdr.status)
+ return xsc_cmd_status_to_err(&out->hdr);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_qp_query);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/qp.h b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.h
new file mode 100644
index 0000000000000..08af2008a22ff
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/qp.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_QP_H
+#define __XSC_QP_H
+
+#include "common/xsc_core.h"
+#include "common/xsc_hsi.h"
+
+void xsc_init_qp_table(struct xsc_core_device *xdev);
+void xsc_qp_event(struct xsc_core_device *xdev, u32 qpn, int event_type);
+
+int xsc_core_create_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp,
+ struct xsc_create_qp_mbox_in *in,
+ int inlen);
+int xsc_core_destroy_qp(struct xsc_core_device *xdev,
+ struct xsc_core_qp *qp);
+int xsc_core_qp_modify(struct xsc_core_device *xdev, enum xsc_qp_state cur_state,
+ enum xsc_qp_state new_state,
+ struct xsc_modify_qp_mbox_in *in, int sqd_event,
+ struct xsc_core_qp *qp);
+int xsc_core_qp_query(struct xsc_core_device *xdev, struct xsc_core_qp *qp,
+ struct xsc_query_qp_mbox_out *out, int outlen);
+
+#endif /* __XSC_QP_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/srq.c b/drivers/net/ethernet/yunsilicon/xsc/pci/srq.c
new file mode 100644
index 0000000000000..c2f9c1a88259f
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/srq.c
@@ -0,0 +1,92 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/vmalloc.h>
+
+#include "common/xsc_core.h"
+#include "common/srq.h"
+#include "srq.h"
+
+int xsc_create_srq_res(struct xsc_core_device *xdev)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+ u32 max_srq_num = XSC_MAX_SRQ_NUM;
+
+ dev_res->srq_tbl = bitmap_zalloc(max_srq_num, GFP_KERNEL);
+ if (!dev_res->srq_tbl)
+ return -ENOMEM;
+
+ dev_res->srq_cache_wr = vzalloc(max_srq_num *
+ sizeof(*dev_res->srq_cache_wr));
+ if (!dev_res->srq_cache_wr) {
+ bitmap_free(dev_res->srq_tbl);
+ dev_res->srq_tbl = NULL;
+ return -ENOMEM;
+ }
+
+ spin_lock_init(&dev_res->srq_lock);
+ dev_res->max_srq_num = max_srq_num;
+
+ return 0;
+}
+
+void xsc_destroy_srq_res(struct xsc_core_device *xdev)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+
+ vfree(dev_res->srq_cache_wr);
+ dev_res->srq_cache_wr = NULL;
+ bitmap_free(dev_res->srq_tbl);
+ dev_res->srq_tbl = NULL;
+ dev_res->max_srq_num = 0;
+}
+
+int xsc_core_create_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+ u32 srqn;
+
+ spin_lock(&dev_res->srq_lock);
+ srqn = find_first_zero_bit(dev_res->srq_tbl, dev_res->max_srq_num);
+ if (srqn == dev_res->max_srq_num) {
+ spin_unlock(&dev_res->srq_lock);
+ pci_err(xdev->pdev, "no free srq number\n");
+ return -ENOMEM;
+ }
+ set_bit(srqn, dev_res->srq_tbl);
+ spin_unlock(&dev_res->srq_lock);
+
+ srq->srqn = srqn;
+ /* array base: callers index it by srqn */
+ srq->cache_wr = dev_res->srq_cache_wr;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_create_srq);
+
+int xsc_core_destroy_srq(struct xsc_core_device *xdev, struct xsc_core_srq *srq)
+{
+ struct xsc_dev_resource *dev_res = xdev->dev_res;
+
+ if (srq->srqn >= dev_res->max_srq_num) {
+ pci_err(xdev->pdev, "invalid srqn %u\n", srq->srqn);
+ return -EINVAL;
+ }
+
+ spin_lock(&dev_res->srq_lock);
+ clear_bit(srq->srqn, dev_res->srq_tbl);
+ spin_unlock(&dev_res->srq_lock);
+
+ /*
+ * The caller still owns the cached recv WRs in cache_wr[srqn] and
+ * releases them right after this call, so the slot must be left
+ * untouched here.
+ */
+ return 0;
+}
+EXPORT_SYMBOL_GPL(xsc_core_destroy_srq);
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/srq.h b/drivers/net/ethernet/yunsilicon/xsc/pci/srq.h
new file mode 100644
index 0000000000000..a70faf9e70ef8
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/srq.h
@@ -0,0 +1,16 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_SRQ_H
+#define __XSC_SRQ_H
+
+#include "common/xsc_core.h"
+#include "common/srq.h"
+
+/* return 0 on success, negative errno otherwise */
+int xsc_create_srq_res(struct xsc_core_device *xdev);
+void xsc_destroy_srq_res(struct xsc_core_device *xdev);
+
+#endif /* __XSC_SRQ_H */
diff --git a/drivers/net/ethernet/yunsilicon/xsc/pci/vport.c b/drivers/net/ethernet/yunsilicon/xsc/pci/vport.c
new file mode 100644
index 0000000000000..5118ff36b8f38
--- /dev/null
+++ b/drivers/net/ethernet/yunsilicon/xsc/pci/vport.c
@@ -0,0 +1,34 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2021 - 2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#include "common/xsc_core.h"
+#include "common/xsc_cmd_api.h"
+
+u8 xsc_core_query_vport_state(struct xsc_core_device *xdev, u16 vport)
+{
+ struct xsc_query_vport_state_out out;
+ struct xsc_query_vport_state_in in;
+ u32 val = 0;
+ int err;
+
+ memset(&in, 0, sizeof(in));
+ memset(&out, 0, sizeof(out));
+
+ in.hdr.opcode = cpu_to_be16(XSC_CMD_OP_QUERY_VPORT_STATE);
+ val |= FIELD_PREP(XSC_QUERY_VPORT_VPORT_NUM_MASK, vport);
+ if (vport)
+ val |= XSC_QUERY_VPORT_OTHER_VPORT;
+ in.data0 = cpu_to_be32(val);
+
+ err = xsc_cmd_exec(xdev, &in, sizeof(in), &out, sizeof(out));
+ if (err) {
+ pci_err(xdev->pdev, "failed to query vport state, err=%d, status=%d\n",
+ err, out.hdr.status);
+ return err;
+ }
+
+ return out.state;
+}
+EXPORT_SYMBOL_GPL(xsc_core_query_vport_state);
diff --git a/include/uapi/rdma/ib_user_ioctl_verbs.h b/include/uapi/rdma/ib_user_ioctl_verbs.h
index 21f86cc7bb1f1..53fc990578200 100644
--- a/include/uapi/rdma/ib_user_ioctl_verbs.h
+++ b/include/uapi/rdma/ib_user_ioctl_verbs.h
@@ -257,6 +257,7 @@ enum rdma_driver_id {
RDMA_DRIVER_ERDMA,
RDMA_DRIVER_MANA,
RDMA_DRIVER_IONIC,
+ RDMA_DRIVER_XSC,
};
enum ib_uverbs_gid_type {
diff --git a/include/uapi/rdma/xsc-abi.h b/include/uapi/rdma/xsc-abi.h
new file mode 100644
index 0000000000000..c4d0db30e43c4
--- /dev/null
+++ b/include/uapi/rdma/xsc-abi.h
@@ -0,0 +1,295 @@
+/* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) */
+/*
+ * Copyright (c) 2021-2026, Shanghai Yunsilicon Technology Co., Ltd.
+ * All rights reserved.
+ */
+
+#ifndef __XSC_ABI_H__
+#define __XSC_ABI_H__
+
+#include <linux/types.h>
+#include <linux/if_ether.h> /* For ETH_ALEN. */
+
+/* Increment this value if any changes that break userspace ABI
+ * compatibility are made.
+ */
+#define XSC_IB_UVERBS_ABI_VERSION 1
+
+/* Make sure that all structs defined in this file remain laid out so
+ * that they pack the same way on 32-bit and 64-bit architectures (to
+ * avoid incompatibility between 32-bit userspace and 64-bit kernels).
+ * In particular do not use pointer types -- pass pointers in __u64
+ * instead.
+ */
+
+enum {
+ XSC_QP_FLAG_SIGNATURE = 1 << 0,
+ XSC_QP_FLAG_SCATTER_CQE = 1 << 1,
+ XSC_QP_FLAG_TUNNEL_OFFLOADS = 1 << 2,
+ XSC_QP_FLAG_BFREG_INDEX = 1 << 3,
+ XSC_QP_FLAG_TYPE_DCT = 1 << 4,
+ XSC_QP_FLAG_TYPE_DCI = 1 << 5,
+ XSC_QP_FLAG_TIR_ALLOW_SELF_LB_UC = 1 << 6,
+ XSC_QP_FLAG_TIR_ALLOW_SELF_LB_MC = 1 << 7,
+ XSC_QP_FLAG_ALLOW_SCATTER_CQE = 1 << 8,
+ XSC_QP_FLAG_RAWPACKET_TSO = 1 << 9,
+ XSC_QP_FLAG_RAWPACKET_TX = 1 << 10,
+ XSC_QP_FLAG_RAWPACKET_SNIFFER = 1 << 11,
+ XSC_QP_FLAG_WORC = 1 << 12,
+};
+
+#define ALLOC_UCONTEXT_VER_V1 1
+#define ALLOC_UCONTEXT_VER_V2 2
+
+#define CURRENT_UCONTEXT_ABI_VER ALLOC_UCONTEXT_VER_V2
+
+struct xsc_ib_alloc_ucontext_req {
+ __u32 abi_ver;
+ __u32 rsvd1;
+};
+
+enum xsc_user_cmds_supp_uhw {
+ XSC_USER_CMDS_SUPP_UHW_QUERY_DEVICE = 1 << 0,
+ XSC_USER_CMDS_SUPP_UHW_CREATE_AH = 1 << 1,
+};
+
+enum {
+ XSC_HW_FEATURE_FLAG_SUPPORT_CQE64 = 1 << 0,
+};
+
+struct xsc_ib_alloc_ucontext_resp {
+ __u32 qp_tab_size;
+ __u32 cache_line_size;
+ __u16 max_sq_desc_sz;
+ __u16 max_rq_desc_sz;
+ __u32 max_send_wqebb;
+ __u32 max_recv_wr;
+ __u16 num_ports;
+ __u16 device_id;
+ __u64 qpm_tx_db;
+ __u64 qpm_rx_db;
+ __u64 cqm_next_cid_reg;
+ __u64 cqm_armdb;
+ __u32 send_ds_num;
+ __u32 recv_ds_num;
+ __u32 cmds_supp_uhw;
+ __u32 pad;
+ __u32 multidb_num;
+ __u32 tx_multidb_base;
+ __u32 rdma_proto_mode;
+ __u32 hw_feature_flag;
+ __u32 reserved[32];
+};
+
+struct xsc_ib_create_qp {
+ __u64 buf_addr;
+ __u64 db_addr;
+ __u32 sq_wqe_count;
+ __u32 rq_wqe_count;
+ __u32 rq_wqe_shift;
+ __u32 flags;
+
+ int dmabuf_fd;
+ size_t dmabuf_sz;
+};
+
+struct xsc_ib_create_qp_resp {
+ __u32 uuar_index;
+ __u32 reserved;
+};
+
+struct xsc_ib_create_srq {
+ __u64 buf_addr;
+ __u64 db_addr;
+ __u32 flags;
+ __u32 reserved0;
+ __u32 uidx;
+ __u32 reserved1;
+};
+
+struct xsc_ib_create_srq_resp {
+ __u32 srqn;
+ __u32 reserved;
+};
+
+struct xsc_ib_create_cq {
+ __u64 buf_addr;
+ __u64 db_addr;
+ __u32 cqe_size;
+
+ int dmabuf_fd;
+ size_t dmabuf_sz;
+};
+
+struct xsc_ib_create_cq_resp {
+ __u32 cqn;
+ __u32 reserved;
+};
+
+struct xsc_ib_create_ah_resp {
+ __u32 response_length;
+ __u8 dmac[ETH_ALEN];
+ __u8 reserved[6];
+};
+
+struct xsc_ib_alloc_pd_resp {
+ __u32 pdn;
+};
+
+struct xsc_ib_tso_caps {
+ __u32 max_tso; /* Maximum tso payload size in bytes */
+
+ /* Corresponding bit will be set if qp type from
+ * 'enum ib_qp_type' is supported, e.g.
+ * supported_qpts |= 1 << IB_QPT_UD
+ */
+ __u32 supported_qpts;
+};
+
+/* RX Hash function flags */
+enum xsc_rx_hash_function_flags {
+ XSC_RX_HASH_FUNC_TOEPLITZ = 1 << 0,
+};
+
+enum xsc_rdma_link_speed {
+ XSC_RDMA_LINK_SPEED_2_5GB = 1 << 0,
+ XSC_RDMA_LINK_SPEED_5GB = 1 << 1,
+ XSC_RDMA_LINK_SPEED_10GB = 1 << 3,
+ XSC_RDMA_LINK_SPEED_14GB = 1 << 4,
+ XSC_RDMA_LINK_SPEED_25GB = 1 << 5,
+ XSC_RDMA_LINK_SPEED_50GB = 1 << 6,
+ XSC_RDMA_LINK_SPEED_100GB = 1 << 7,
+};
+
+enum xsc_rdma_phys_state {
+ XSC_RDMA_PHY_STATE_SLEEP = 1,
+ XSC_RDMA_PHY_STATE_POLLING,
+ XSC_RDMA_PHY_STATE_DISABLED,
+ XSC_RDMA_PHY_STATE_PORT_CONFIGURATION_TRAINNING,
+ XSC_RDMA_PHY_STATE_LINK_UP,
+ XSC_RDMA_PHY_STATE_LINK_ERROR_RECOVERY,
+ XSC_RDMA_PHY_STATE_PHY_TEST,
+};
+
+/*
+ * RX Hash flags, these flags allows to set which incoming packet's field should
+ * participates in RX Hash. Each flag represent certain packet's field,
+ * when the flag is set the field that is represented by the flag will
+ * participate in RX Hash calculation.
+ * Note: *IPV4 and *IPV6 flags can't be enabled together on the same QP
+ * and *TCP and *UDP flags can't be enabled together on the same QP.
+ */
+enum xsc_rx_hash_fields {
+ XSC_RX_HASH_SRC_IPV4 = 1 << 0,
+ XSC_RX_HASH_DST_IPV4 = 1 << 1,
+ XSC_RX_HASH_SRC_IPV6 = 1 << 2,
+ XSC_RX_HASH_DST_IPV6 = 1 << 3,
+ XSC_RX_HASH_SRC_PORT_TCP = 1 << 4,
+ XSC_RX_HASH_DST_PORT_TCP = 1 << 5,
+ XSC_RX_HASH_SRC_PORT_UDP = 1 << 6,
+ XSC_RX_HASH_DST_PORT_UDP = 1 << 7,
+ XSC_RX_HASH_IPSEC_SPI = 1 << 8,
+ /* Save bits for future fields */
+ XSC_RX_HASH_INNER = (1UL << 31),
+};
+
+struct xsc_ib_rss_caps {
+ __aligned_u64 rx_hash_fields_mask; /* enum xsc_rx_hash_fields */
+ __u8 rx_hash_function; /* enum xsc_rx_hash_function_flags */
+ __u8 reserved[7];
+};
+
+enum xsc_ib_cqe_comp_res_format {
+ XSC_IB_CQE_RES_FORMAT_HASH = 1 << 0,
+ XSC_IB_CQE_RES_FORMAT_CSUM = 1 << 1,
+ XSC_IB_CQE_RES_FORMAT_CSUM_STRIDX = 1 << 2,
+};
+
+struct xsc_ib_cqe_comp_caps {
+ __u32 max_num;
+ __u32 supported_format; /* enum xsc_ib_cqe_comp_res_format */
+};
+
+enum xsc_ib_packet_pacing_cap_flags {
+ XSC_IB_PP_SUPPORT_BURST = 1 << 0,
+};
+
+struct xsc_packet_pacing_caps {
+ __u32 qp_rate_limit_min;
+ __u32 qp_rate_limit_max; /* In kpbs */
+
+ /* Corresponding bit will be set if qp type from
+ * 'enum ib_qp_type' is supported, e.g.
+ * supported_qpts |= 1 << IB_QPT_RAW_PACKET
+ */
+ __u32 supported_qpts;
+ __u8 cap_flags; /* enum xsc_ib_packet_pacing_cap_flags */
+ __u8 reserved[3];
+};
+
+enum xsc_ib_mpw_caps {
+ MPW_RESERVED = 1 << 0,
+ XSC_IB_ALLOW_MPW = 1 << 1,
+ XSC_IB_SUPPORT_EMPW = 1 << 2,
+};
+
+enum xsc_ib_sw_parsing_offloads {
+ XSC_IB_SW_PARSING = 1 << 0,
+ XSC_IB_SW_PARSING_CSUM = 1 << 1,
+ XSC_IB_SW_PARSING_LSO = 1 << 2,
+};
+
+struct xsc_ib_sw_parsing_caps {
+ __u32 sw_parsing_offloads; /* enum xsc_ib_sw_parsing_offloads */
+
+ /* Corresponding bit will be set if qp type from
+ * 'enum ib_qp_type' is supported, e.g.
+ * supported_qpts |= 1 << IB_QPT_RAW_PACKET
+ */
+ __u32 supported_qpts;
+};
+
+struct xsc_ib_striding_rq_caps {
+ __u32 min_single_stride_log_num_of_bytes;
+ __u32 max_single_stride_log_num_of_bytes;
+ __u32 min_single_wqe_log_num_of_strides;
+ __u32 max_single_wqe_log_num_of_strides;
+
+ /* Corresponding bit will be set if qp type from
+ * 'enum ib_qp_type' is supported, e.g.
+ * supported_qpts |= 1 << IB_QPT_RAW_PACKET
+ */
+ __u32 supported_qpts;
+ __u32 reserved;
+};
+
+enum xsc_ib_query_dev_resp_flags {
+ /* Support 128B CQE compression */
+ XSC_IB_QUERY_DEV_RESP_FLAGS_CQE_128B_COMP = 1 << 0,
+ XSC_IB_QUERY_DEV_RESP_FLAGS_CQE_128B_PAD = 1 << 1,
+};
+
+enum xsc_ib_tunnel_offloads {
+ XSC_IB_TUNNELED_OFFLOADS_VXLAN = 1 << 0,
+ XSC_IB_TUNNELED_OFFLOADS_GRE = 1 << 1,
+ XSC_IB_TUNNELED_OFFLOADS_GENEVE = 1 << 2,
+ XSC_IB_TUNNELED_OFFLOADS_MPLS_GRE = 1 << 3,
+ XSC_IB_TUNNELED_OFFLOADS_MPLS_UDP = 1 << 4,
+};
+
+struct xsc_ib_query_device_resp {
+ __u32 comp_mask;
+ __u32 response_length;
+ struct xsc_ib_tso_caps tso_caps;
+ struct xsc_ib_rss_caps rss_caps;
+ struct xsc_ib_cqe_comp_caps cqe_comp_caps;
+ struct xsc_packet_pacing_caps packet_pacing_caps;
+ __u32 xsc_ib_support_multi_pkt_send_wqes;
+ __u32 flags; /* Use enum xsc_ib_query_dev_resp_flags */
+ struct xsc_ib_sw_parsing_caps sw_parsing_caps;
+ struct xsc_ib_striding_rq_caps striding_rq_caps;
+ __u32 tunnel_offloads_caps; /* enum xsc_ib_tunnel_offloads */
+ __u32 reserved;
+};
+
+#endif /* __XSC_ABI_H__ */
|
| 3/2 | 2026/10/12 12:01 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 240449 output: 130 thoughts: 2204 Thoughts: **Analysis of Yunsilicon XSC Driver for Fuzzing Suitability** Okay, so this patch introduces new drivers for Yunsilicon XSC devices, both for InfiniBand and Ethernet. My initial thought is to assess its reachability within standard virtualized environments, specifically QEMU and GCE, as per the guidelines. Given my experience, I know that hardware that isn't emulated is essentially off-limits for fuzzing in these environments. I've quickly identified that the driver interacts with devices having PCI vendor ID `0x1f67`, which belongs to Shanghai Yunsilicon Technology Co., Ltd. Diving into the code, especially `drivers/net/ethernet/yunsilicon/xsc/pci/main.c`, reveals it binds to specific PCI device IDs. Critically, the driver then creates "eth" and "ib" auxiliary devices, dependent on successful PCI probe and initialization. My immediate concern is whether QEMU or GCE offer emulation for this specific Yunsilicon hardware. I know they do not, and my expectation is that they would need to, in order for the driver to be considered suitable for fuzzing in syzkaller / syz-cluster. Without that, there's no way for the code to run in the target environment. I then check to see if there is any kind of fallback or mock driver, a software-based implementation. Unfortunately, after looking at the code for both the ethernet and infiniband drivers, it's clear the auxiliary devices are created in `xsc_adev_init()` which is only ever called after a successful PCI probe. Without a physical PCI device or appropriate emulation, the probe, initialization, and device creation never occur. The drivers are, in effect, structurally unreachable. Furthermore, I have checked the changes introduced by the patch. There is a UAPI addition in `include/uapi/rdma/ib_user_ioctl_verbs.h` which adds an `RDMA_DRIVER_XSC` enum, but this is a UAPI change and will not affect the core logic of the driver. Recalling the negative criteria from the guidelines, the Yunsilicon XSC driver fits the description of vendor-specific SmartNIC/PCIe driver, analogous to examples like mlxsw, pds_core, qed, and ionic. These require dedicated hardware not emulated by the standard environments. Therefore, because this driver requires a hardware that is not emulated in standard environments, I'm confident that `WorthFuzzing` must be `false`. The associated metadata confirms this. There are no focus symbols, and no additional configurations required, since the code is structurally unreachable. I am sure of my analysis. |
| 4/2 | 2026/10/12 12:01 | tool | set-results |
0mArgs:{
"EnableConfigs": [],
"FocusSymbols": [],
"Reasoning": "The patch series introduces support for Yunsilicon XSC PCIe devices, including the core PCI driver, ethernet driver, and RDMA (InfiniBand) driver. These drivers bind strictly to proprietary physical ASIC/PCIe hardware (Vendor ID 0x1f67) and associated auxiliary devices that are not emulated in standard virtualized QEMU or cloud environments (GCE). Because the modified code is structurally unreachable without specialized physical hardware, it is not worth fuzzing.",
"WorthFuzzing": false
}Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"Reasoning": "The patch series introduces support for Yunsilicon XSC PCIe devices, including the core PCI driver, ethernet driver, and RDMA (InfiniBand) driver. These drivers bind strictly to proprietary physical ASIC/PCIe hardware (Vendor ID 0x1f67) and associated auxiliary devices that are not emulated in standard virtualized QEMU or cloud environments (GCE). Because the modified code is structurally unreachable without specialized physical hardware, it is not worth fuzzing.",
"WorthFuzzing": false
} |
| 5/1 | 2026/10/12 12:01 | action | If |
0mArgs:{
"WorthFuzzing": false
}Results: null |
| Total Calls | Total Tokens | Avg Tokens | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|
| Total Calls | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|