| Seq | Timestamp | Type | Name | Duration |
|---|---|---|---|---|
| 0/0 | 2026/07/25 16:10 | flow | patch-triage |
4mArgs:null Results: {
"EnableConfigs": [
"KVM"
],
"FocusSymbols": [
"vgic_v5_parse_attr",
"vgic_v5_irs_attr_regs_access",
"vgic_v5_has_attr_regs",
"kvm_vgic_v5_irs_init",
"vgic_v5_register_irs_iodev",
"vgic_v5_cpu_sysregs_uaccess",
"vgic_v5_has_cpu_sysregs_attr",
"vgic_v5_irs_save_ists",
"vgic_v5_irs_restore_ists",
"vgic_v5_vmt_allocate",
"vgic_v5_vmt_free",
"vgic_v5_allocate_vm_id",
"vgic_v5_release_vm_id",
"vgic_v5_vmte_init",
"vgic_v5_vmte_release",
"vgic_v5_vmte_alloc_vpe",
"vgic_v5_vmte_free_vpe",
"vgic_v5_spi_ist_alloc",
"vgic_v5_lpi_ist_alloc",
"vgic_v5_lpi_ist_free",
"vgic_v5_save_spi_ist",
"vgic_v5_save_lpi_ist",
"vgic_v5_restore_spi_ist",
"vgic_v5_restore_lpi_ist"
],
"Reasoning": "The patch series introduces support for GICv5 IRS (Interrupt Routing Service), SPIs, and LPIs in KVM. It adds new KVM device attributes for GICv5, including IRS registers, CPU sysregs, and IST (Interrupt State Table) save/restore functionality. This is a significant functional change to the KVM ARM VGIC subsystem that introduces new reachable code paths and APIs, making it highly relevant for fuzzing.",
"WorthFuzzing": true
} |
| 1/1 | 2026/07/25 16:10 | action | read-patch-diff |
0mArgs:null Results: {
"PatchDiff": "commit eb3e8c4cd3bdcee09fe7080f97581a948e8892e5\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Sat Jul 25 16:10:13 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/Documentation/virt/kvm/devices/arm-vgic-v5.rst b/Documentation/virt/kvm/devices/arm-vgic-v5.rst\nindex 70b9162755c7e8..6813051789ac48 100644\n--- a/Documentation/virt/kvm/devices/arm-vgic-v5.rst\n+++ b/Documentation/virt/kvm/devices/arm-vgic-v5.rst\n@@ -12,12 +12,39 @@ Only one VGIC instance may be instantiated through this API. The created VGIC\n will act as the VM interrupt controller, requiring emulated user-space devices\n to inject interrupts to the VGIC instead of directly to CPUs.\n \n-Creating a guest GICv5 device requires a GICv5 host. The current VGICv5 device\n-only supports PPI interrupts. These can either be injected from emulated\n-in-kernel devices (such as the Arch Timer, or PMU), or via the KVM_IRQ_LINE\n-ioctl.\n+Creating a guest GICv5 device requires a GICv5 host. The VGICv5 device supports\n+PPI, SPI, and LPI interrupts. The PPI and SPI interrupts can either be injected\n+from emulated in-kernel devices (such as the Arch Timer, or PMU), or via the\n+KVM_IRQ_LINE ioctl. LPIs are not externally injected, but are handled in\n+hardware via the LPI IST. Their pending state is driven directly by the guest.\n \n Groups:\n+ KVM_DEV_ARM_VGIC_GRP_ADDR\n+ Attributes:\n+\n+ KVM_VGIC_V5_ADDR_TYPE_IRS (rw, 64-bit)\n+ Base address in the guest physical address space of the GICv5 IRS\n+ (Interrupt Routing Service) register mappings. Only valid for\n+ KVM_DEV_TYPE_ARM_VGIC_V5. This address needs to be 64K aligned and the\n+ region covers 128 KByte - the IRS has a CONFIG_FRAME and a SETLPI_FRAME,\n+ each of which is 64 KBytes in size.\n+\n+ Setting the address of the IRS in GPA space is mandatory before VGIC\n+ resources are mapped, as the IRS is responsible for handling SPIs and\n+ LPIs. Failure to set the IRS address before the first vCPU run results in\n+ an error.\n+\n+ KVM_DEV_ARM_VGIC_GRP_NR_IRQS\n+ Attributes:\n+\n+ A value describing the number of SPIs for this GIC instance. This is\n+ GICv5-specific: unlike GICv2/v3, the value does not include SGIs or PPIs.\n+ The value ranges from 32 to KVM's VGICv5 maximum of 65536 SPIs, in\n+ increments of 32. If userspace does not set this attribute, KVM uses 32\n+ SPIs by default.\n+\n+ kvm_device_attr.addr points to a __u32 value.\n+\n KVM_DEV_ARM_VGIC_GRP_CTRL\n Attributes:\n \n@@ -48,3 +75,232 @@ Groups:\n -EFAULT Invalid guest ram access\n -EBUSY One or more VCPUS are running\n ======= ========================================================\n+\n+ KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS\n+ Attributes:\n+\n+ The attr field of kvm_device_attr encodes two values::\n+\n+ bits: | 63 .... 32 | 31 .... 16 | 15 .... 0 |\n+ values: | mpidr | RES | instr |\n+\n+ The mpidr field encodes the CPU ID based on the affinity information in the\n+ architecture defined MPIDR, and the field is encoded as follows::\n+\n+ | 63 .... 56 | 55 .... 48 | 47 .... 40 | 39 .... 32 |\n+ | Aff3 | Aff2 | Aff1 | Aff0 |\n+\n+ The instr field encodes the system register to access based on the fields\n+ defined in the A64 instruction set encoding for system register access\n+ (RES means the bits are reserved for future use and should be zero)::\n+\n+ | 15 ... 14 | 13 ... 11 | 10 ... 7 | 6 ... 3 | 2 ... 0 |\n+ | Op 0 | Op1 | CRn | CRm | Op2 |\n+\n+ All system regs accessed through this API are (rw, 64-bit) and\n+ kvm_device_attr.addr points to a __u64 value.\n+\n+ KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS accesses the CPU interface registers for the\n+ CPU specified by the mpidr field.\n+\n+ The available registers are:\n+\n+ ======================= ===================================================\n+ ICC_ICSR_EL1\n+ ICC_PPI_ENABLER0_EL1\n+ ICC_PPI_ENABLER1_EL1\n+ ICC_PPI_SACTIVER0_EL1 ICC_PPI_CACTIVER0_EL1 is not supported. Writes to\n+ ICC_PPI_SACTIVER0_EL1 are treated as RAW writes of\n+ the underlying state.\n+ ICC_PPI_SACTIVER1_EL1 ICC_PPI_CACTIVER1_EL1 is not supported. Writes to\n+ ICC_PPI_SACTIVER1_EL1 are treated as RAW writes of\n+ the underlying state.\n+ ICC_PPI_SPENDR0_EL1 ICC_PPI_CPENDR0_EL1 is not supported. Writes to\n+ ICC_PPI_SPENDR0_EL1 are treated as RAW writes of\n+ the underlying state.\n+ ICC_PPI_SPENDR1_EL1 ICC_PPI_CPENDR1_EL1 is not supported. Writes to\n+ ICC_PPI_SPENDR1_EL1 are treated as RAW writes of\n+ the underlying state.\n+ ICC_PPI_PRIORITYR0_EL1\n+ ICC_PPI_PRIORITYR1_EL1\n+ ICC_PPI_PRIORITYR2_EL1\n+ ICC_PPI_PRIORITYR3_EL1\n+ ICC_PPI_PRIORITYR4_EL1\n+ ICC_PPI_PRIORITYR5_EL1\n+ ICC_PPI_PRIORITYR6_EL1\n+ ICC_PPI_PRIORITYR7_EL1\n+ ICC_PPI_PRIORITYR8_EL1\n+ ICC_PPI_PRIORITYR9_EL1\n+ ICC_PPI_PRIORITYR10_EL1\n+ ICC_PPI_PRIORITYR11_EL1\n+ ICC_PPI_PRIORITYR12_EL1\n+ ICC_PPI_PRIORITYR13_EL1\n+ ICC_PPI_PRIORITYR14_EL1\n+ ICC_PPI_PRIORITYR15_EL1\n+ ICC_APR_EL1\n+ ICC_CR0_EL1\n+ ICC_PCR_EL1\n+ ======================= ===================================================\n+\n+ Errors:\n+\n+ ======= =============================================================\n+ -ENXIO Getting or setting this register is not supported\n+ -EBUSY VCPU is running, or write attempted after a VCPU has run\n+ -EINVAL Invalid mpidr or register value supplied\n+ ======= =============================================================\n+\n+\n+KVM_DEV_ARM_VGIC_GRP_IRS_REGS\n+ Attributes:\n+ The attr field of kvm_device_attr encodes the offset of the IRS register,\n+ relative to the IRS CONFIG_FRAME base address. This is the address that\n+ was provided via KVM_VGIC_V5_ADDR_TYPE_IRS when creating VGICv5 in the\n+ first place.\n+\n+ kvm_device_attr.addr points to a __u64 value whatever the width\n+ of the addressed register (32/64 bits). 64 bit registers can only\n+ be accessed with full length.\n+\n+ Writes to read-only registers are ignored by the kernel except for\n+ IRS_IDR0 - IRS_IDR2 and IRS_IDR5 - IRS_IDR7, which are sanity checked to\n+ ensure that they match a sane config.\n+\n+ IRS_IDR3 and IRS_IDR4 are RAZ/WI, as nested virtualization is not\n+ supported.\n+\n+ For registers without dedicated userspace accessors, getting or setting a\n+ register uses the same emulated MMIO handlers as guest reads/writes.\n+ Dedicated userspace accessors may instead save or restore migration state\n+ without triggering guest-visible side effects. For example, restoring\n+ IRS_IST_BASER only restores the emulated register state; any host LPI IST\n+ allocation based on the restored IRS_IST_CFGR and IRS_IST_BASER state\n+ happens when KVM_DEV_ARM_VGIC_GRP_IST is restored.\n+\n+ Errors:\n+\n+ ======= =================================================================\n+ -ENXIO Offset does not correspond to any supported register\n+ -EFAULT Invalid user pointer for attr-\u003eaddr\n+ -EINVAL Offset is not 32-bit aligned for 32-bit MMIO registers, not\n+ 64-bit aligned for 64-bit registers, or the supplied register\n+ value is not compatible with the configured VGICv5 IRS state\n+ -EBUSY VGIC is not initialized, or one or more VCPUs are running\n+ ======= =================================================================\n+\n+ KVM_DEV_ARM_VGIC_GRP_IST\n+ Attributes:\n+ This interface is used to either save the state of the IRS's Interrupt\n+ State Tables (ISTs), or to restore them. A get operation saves IST state,\n+ and a set operation restores IST state. kvm_device_attr.attr is reserved\n+ and must be zero.\n+\n+ The VGIC must be initialized before using this interface. Restore must be\n+ performed before the VM has run. For restore, userspace must have already\n+ restored the IRS state needed to describe any guest LPI IST.\n+\n+ Saving first asks the IRS to save and quiesce the VM so that interrupt\n+ state has been written back to the ISTs. KVM checks that the VM remains\n+ quiesced while copying out the SPI and LPI IST state.\n+\n+ kvm_device_attr.addr points to a struct kvm_vgic_v5_ist::\n+\n+ struct kvm_vgic_v5_ist {\n+ __u64 spi_ist_addr;\n+ __u64 spi_ist_size;\n+ __u64 lpi_ist_addr;\n+ __u64 lpi_ist_size;\n+ };\n+\n+ The SPI and LPI IST buffers contain one little-endian 32-bit IST entry per\n+ interrupt, in interrupt number order. Only the architected 32-bit ISTE\n+ state is exposed to userspace; host IST layout and metadata are private to\n+ KVM.\n+\n+ The SPI IST buffer is required and its size must be:\n+\n+ nr_spis * sizeof(__u32)\n+\n+ where nr_spis is the value returned by KVM_DEV_ARM_VGIC_GRP_NR_IRQS for\n+ the VGICv5 device. For VGICv5 this value is the number of SPIs, not the\n+ total number of interrupts.\n+\n+ The LPI IST buffer is required if the guest's IRS_IST_CFGR and\n+ IRS_IST_BASER state describes a valid LPI IST, and its size must be:\n+\n+ BIT(lpi_id_bits) * sizeof(__u32)\n+\n+ where lpi_id_bits is the LPI ID width configured in IRS_IST_CFGR. If no\n+ LPI IST is configured, lpi_ist_addr and lpi_ist_size must both be zero.\n+\n+ Errors:\n+\n+ =========== ============================================================\n+ -EBUSY One or more VCPUs are running, the VGIC is not initialized,\n+ restore was requested after the VM has run, an LPI IST\n+ already exists, or the save operation completed but the VM\n+ did not remain quiesced\n+ -EINVAL attr-\u003eaddr is NULL, the userspace IST descriptor is missing\n+ a required buffer, supplies a non-zero LPI buffer when no\n+ LPI IST is configured, has a buffer size that does not match\n+ the configured VGICv5 IST state, or an internal VM table\n+ operation rejected the VM state\n+ -ENOENT A userspace SPI IST buffer was supplied, but there is no SPI\n+ IST to serialise/unserialise\n+ -EFAULT Invalid user pointer for attr-\u003eaddr, spi_ist_addr, or\n+ lpi_ist_addr\n+ -ENXIO Required per-VM VGICv5/IST backing state is missing or\n+ inconsistent\n+ -ENOMEM Restoring IST state failed while allocating the host LPI IST\n+ or tracking pending interrupts\n+ -ETIMEDOUT An IRS save/VM operation timed out\n+ =========== ============================================================\n+\n+IRS Save Sequence:\n+------------------\n+\n+The following operations are required when saving the virtual GICv5 IRS:\n+\n+a) Save the ISTs by issuing KVM_GET_DEVICE_ATTR on KVM_DEV_ARM_VGIC_GRP_IST.\n+b) Save the IRS MMIO register state by issuing KVM_GET_DEVICE_ATTR on\n+ KVM_DEV_ARM_VGIC_GRP_IRS_REGS.\n+\n+These two steps may be performed in either order. Saving the ISTs writes all\n+IST state to userspace-provided buffers and does not update guest memory.\n+\n+IRS Restore Sequence:\n+---------------------\n+\n+The following ordering must be followed when restoring the virtual GICv5 and\n+IRS:\n+\n+a) Create vCPUs.\n+b) Provide the IRS base address by issuing KVM_SET_DEVICE_ATTR on\n+ KVM_DEV_ARM_VGIC_GRP_ADDR\n+c) Restore the number of SPIs by issuing KVM_SET_DEVICE_ATTR on\n+ KVM_DEV_ARM_VGIC_GRP_NR_IRQS.\n+d) Initialise the GIC - this sets up the default state and creates the SPI\n+ IST - by issuing KVM_SET_DEVICE_ATTR on KVM_DEV_ARM_VGIC_GRP_CTRL with\n+ KVM_DEV_ARM_VGIC_CTRL_INIT\n+e) Restore the IRS MMIO register state by issuing KVM_SET_DEVICE_ATTR on\n+ KVM_DEV_ARM_VGIC_GRP_IRS_REGS. The IRS registers may be restored in any\n+ order, but any IRS_IDR* state and the IRS_IST_CFGR/IRS_IST_BASER state\n+ describing an LPI IST must be restored before restoring the ISTs. KVM uses\n+ the restored IRS_IST_CFGR and IRS_IST_BASER state to allocate the LPI IST\n+ during the following step.\n+f) Restore the ISTs by issuing KVM_SET_DEVICE_ATTR on\n+ KVM_DEV_ARM_VGIC_GRP_IST.\n+\n+The number of SPIs must be restored before VGIC initialization because\n+initialization allocates the SPI state and fixes the SPI range exposed by the\n+IRS ID registers.\n+\n+The IRS's MMIO registers must be restored prior to restoring the ISTs as these\n+are used to convey the number of LPIs the guest has configured to KVM.\n+\n+The various ``*_STATUSR`` registers are observational state in the current KVM\n+implementation. Userspace may save them for validation or debugging purposes,\n+but they are not required as restore input and do not need to be replayed during\n+restore.\n+\n+Then vCPUs can be started.\ndiff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h\nindex 043495f7fc78b9..36808ed0e9afe4 100644\n--- a/arch/arm64/include/asm/kvm_asm.h\n+++ b/arch/arm64/include/asm/kvm_asm.h\n@@ -87,6 +87,9 @@ enum __kvm_host_smccc_func {\n \t__KVM_HOST_SMCCC_FUNC___tracing_write_event,\n \t__KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs,\n \t__KVM_HOST_SMCCC_FUNC___vgic_v3_restore_vmcr_aprs,\n+\t__KVM_HOST_SMCCC_FUNC___vgic_v5_make_resident,\n+\t__KVM_HOST_SMCCC_FUNC___vgic_v5_make_non_resident,\n+\t__KVM_HOST_SMCCC_FUNC___vgic_v5_vdpend,\n \t__KVM_HOST_SMCCC_FUNC___vgic_v5_save_apr,\n \t__KVM_HOST_SMCCC_FUNC___vgic_v5_restore_vmcr_apr,\n \ndiff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h\nindex bae2c4f92ef5c0..b04dcc9a73c288 100644\n--- a/arch/arm64/include/asm/kvm_host.h\n+++ b/arch/arm64/include/asm/kvm_host.h\n@@ -55,6 +55,7 @@\n #define KVM_REQ_GUEST_HYP_IRQ_PENDING\tKVM_ARCH_REQ(9)\n #define KVM_REQ_MAP_L1_VNCR_EL2\t\tKVM_ARCH_REQ(10)\n #define KVM_REQ_VGIC_PROCESS_UPDATE\tKVM_ARCH_REQ(11)\n+#define KVM_REQ_RELOAD_GICv5\t\tKVM_ARCH_REQ(12)\n \n #define KVM_DIRTY_LOG_MANUAL_CAPS (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | \\\n \t\t\t\t KVM_DIRTY_LOG_INITIALLY_SET)\ndiff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h\nindex 4974492744cc8d..8e2f0280c0cc58 100644\n--- a/arch/arm64/include/asm/kvm_hyp.h\n+++ b/arch/arm64/include/asm/kvm_hyp.h\n@@ -88,6 +88,8 @@ void __vgic_v3_restore_vmcr_aprs(struct vgic_v3_cpu_if *cpu_if);\n int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);\n \n /* GICv5 */\n+void __vgic_v5_make_resident(struct vgic_v5_cpu_if *cpu_if);\n+void __vgic_v5_make_non_resident(struct vgic_v5_cpu_if *cpu_if);\n void __vgic_v5_save_apr(struct vgic_v5_cpu_if *cpu_if);\n void __vgic_v5_restore_vmcr_apr(struct vgic_v5_cpu_if *cpu_if);\n /* No hypercalls for the following */\n@@ -95,6 +97,7 @@ void __vgic_v5_save_ppi_state(struct vgic_v5_cpu_if *cpu_if);\n void __vgic_v5_restore_ppi_state(struct vgic_v5_cpu_if *cpu_if);\n void __vgic_v5_save_state(struct vgic_v5_cpu_if *cpu_if);\n void __vgic_v5_restore_state(struct vgic_v5_cpu_if *cpu_if);\n+void __vgic_v5_vdpend(u32 intid, bool pending, u16 vm);\n \n #ifdef __KVM_NVHE_HYPERVISOR__\n void __timer_enable_traps(struct kvm_vcpu *vcpu);\ndiff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h\nindex 7aa08d59d49448..c840b37adb2fb3 100644\n--- a/arch/arm64/include/asm/sysreg.h\n+++ b/arch/arm64/include/asm/sysreg.h\n@@ -1040,7 +1040,7 @@\n #define GCS_CAP(x)\t((((unsigned long)x) \u0026 GCS_CAP_ADDR_MASK) | \\\n \t\t\t\t\t GCS_CAP_VALID_TOKEN)\n /*\n- * Definitions for GICv5 instructions\n+ * Definitions for GICv5 instructions for the Current Domain\n */\n #define GICV5_OP_GIC_CDAFF\t\tsys_insn(1, 0, 12, 1, 3)\n #define GICV5_OP_GIC_CDDI\t\tsys_insn(1, 0, 12, 2, 0)\n@@ -1105,6 +1105,17 @@\n #define GICV5_GICR_CDNMIA_TYPE_MASK\tGENMASK_ULL(31, 29)\n #define GICV5_GICR_CDNMIA_ID_MASK\tGENMASK_ULL(23, 0)\n \n+/*\n+ * Definitions for GICv5 instructions for the Virtual Domain\n+ */\n+#define GICV5_OP_GIC_VDPEND\t\tsys_insn(1, 4, 12, 1, 4)\n+\n+/* Shift and mask definitions for GIC VDPEND */\n+#define GICV5_GIC_VDPEND_PENDING_MASK\tBIT_ULL(63)\n+#define GICV5_GIC_VDPEND_VM_MASK\tGENMASK_ULL(47, 32)\n+#define GICV5_GIC_VDPEND_TYPE_MASK\tGENMASK_ULL(31, 29)\n+#define GICV5_GIC_VDPEND_ID_MASK\tGENMASK_ULL(23, 0)\n+\n #define gicr_insn(insn)\t\t\tread_sysreg_s(GICV5_OP_GICR_##insn)\n #define gic_insn(v, insn)\t\twrite_sysreg_s(v, GICV5_OP_GIC_##insn)\n \ndiff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h\nindex 1c13bfa2d38aad..1b9bbeab18a4e0 100644\n--- a/arch/arm64/include/uapi/asm/kvm.h\n+++ b/arch/arm64/include/uapi/asm/kvm.h\n@@ -97,6 +97,11 @@ struct kvm_regs {\n #define KVM_VGIC_V3_REDIST_SIZE\t\t(2 * SZ_64K)\n #define KVM_VGIC_V3_ITS_SIZE\t\t(2 * SZ_64K)\n \n+/* Supported VGICv5 address types */\n+#define KVM_VGIC_V5_ADDR_TYPE_IRS\t6\n+\n+#define KVM_VGIC_V5_IRS_SIZE\t\t(2 * SZ_64K)\n+\n #define KVM_ARM_VCPU_POWER_OFF\t\t0 /* CPU is started in OFF state */\n #define KVM_ARM_VCPU_EL1_32BIT\t\t1 /* CPU running a 32bit VM */\n #define KVM_ARM_VCPU_PSCI_0_2\t\t2 /* CPU uses PSCI v0.2 */\n@@ -417,6 +422,8 @@ enum {\n #define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7\n #define KVM_DEV_ARM_VGIC_GRP_ITS_REGS 8\n #define KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ 9\n+#define KVM_DEV_ARM_VGIC_GRP_IRS_REGS\t10\n+#define KVM_DEV_ARM_VGIC_GRP_IST\t11\n #define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT\t10\n #define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \\\n \t\t\t(0x3fffffULL \u003c\u003c KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)\ndiff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile\nindex 59612d2f277c1d..7aaeeb84e788ec 100644\n--- a/arch/arm64/kvm/Makefile\n+++ b/arch/arm64/kvm/Makefile\n@@ -24,7 +24,8 @@ kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \\\n \t vgic/vgic-mmio.o vgic/vgic-mmio-v2.o \\\n \t vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \\\n \t vgic/vgic-its.o vgic/vgic-debug.o vgic/vgic-v3-nested.o \\\n-\t vgic/vgic-v5.o\n+\t vgic/vgic-v5.o vgic/vgic-v5-tables.o vgic/vgic-irs-v5.o \\\n+\t vgic-sys-reg-v5.o\n \n kvm-$(CONFIG_HW_PERF_EVENTS) += pmu-emul.o pmu.o\n kvm-$(CONFIG_ARM64_PTR_AUTH) += pauth.o\ndiff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c\nindex 50adfff75be82e..7c59f9b451049c 100644\n--- a/arch/arm64/kvm/arm.c\n+++ b/arch/arm64/kvm/arm.c\n@@ -961,7 +961,7 @@ int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu)\n \t\t\treturn ret;\n \t}\n \n-\tret = vgic_v5_finalize_ppi_state(kvm);\n+\tret = vgic_v5_finalize_ppi_state(vcpu);\n \tif (ret)\n \t\treturn ret;\n \n@@ -1138,6 +1138,14 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu)\n \t\t\tpreempt_enable();\n \t\t}\n \n+\t\tif (kvm_check_request(KVM_REQ_RELOAD_GICv5, vcpu)) {\n+\t\t\t/* The IRS enable bit was changed */\n+\t\t\tpreempt_disable();\n+\t\t\tvgic_v5_put(vcpu);\n+\t\t\tvgic_v5_load(vcpu);\n+\t\t\tpreempt_enable();\n+\t\t}\n+\n \t\tif (kvm_check_request(KVM_REQ_RELOAD_PMU, vcpu))\n \t\t\tkvm_vcpu_reload_pmu(vcpu);\n \ndiff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c\nindex d3c69de698f484..8f9cc3cd422aae 100644\n--- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c\n+++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c\n@@ -691,6 +691,20 @@ static void handle___tracing_write_event(struct kvm_cpu_context *host_ctxt)\n \ttrace_selftest(id);\n }\n \n+static void handle___vgic_v5_make_resident(struct kvm_cpu_context *host_ctxt)\n+{\n+\tDECLARE_REG(struct vgic_v5_cpu_if *, cpu_if, host_ctxt, 1);\n+\n+\t__vgic_v5_make_resident(kern_hyp_va(cpu_if));\n+}\n+\n+static void handle___vgic_v5_make_non_resident(struct kvm_cpu_context *host_ctxt)\n+{\n+\tDECLARE_REG(struct vgic_v5_cpu_if *, cpu_if, host_ctxt, 1);\n+\n+\t__vgic_v5_make_non_resident(kern_hyp_va(cpu_if));\n+}\n+\n static void handle___vgic_v5_save_apr(struct kvm_cpu_context *host_ctxt)\n {\n \tDECLARE_REG(struct vgic_v5_cpu_if *, cpu_if, host_ctxt, 1);\n@@ -705,6 +719,15 @@ static void handle___vgic_v5_restore_vmcr_apr(struct kvm_cpu_context *host_ctxt)\n \t__vgic_v5_restore_vmcr_apr(kern_hyp_va(cpu_if));\n }\n \n+static void handle___vgic_v5_vdpend(struct kvm_cpu_context *host_ctxt)\n+{\n+\tDECLARE_REG(u32, intid, host_ctxt, 1);\n+\tDECLARE_REG(bool, pending, host_ctxt, 2);\n+\tDECLARE_REG(u16, vm, host_ctxt, 3);\n+\n+\t__vgic_v5_vdpend(intid, pending, vm);\n+}\n+\n typedef void (*hcall_t)(struct kvm_cpu_context *);\n \n #define HANDLE_FUNC(x)\t[__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x\n@@ -738,6 +761,9 @@ static const hcall_t host_hcall[] = {\n \tHANDLE_FUNC(__tracing_write_event),\n \tHANDLE_FUNC(__vgic_v3_save_aprs),\n \tHANDLE_FUNC(__vgic_v3_restore_vmcr_aprs),\n+\tHANDLE_FUNC(__vgic_v5_make_resident),\n+\tHANDLE_FUNC(__vgic_v5_make_non_resident),\n+\tHANDLE_FUNC(__vgic_v5_vdpend),\n \tHANDLE_FUNC(__vgic_v5_save_apr),\n \tHANDLE_FUNC(__vgic_v5_restore_vmcr_apr),\n \ndiff --git a/arch/arm64/kvm/hyp/vgic-v5-sr.c b/arch/arm64/kvm/hyp/vgic-v5-sr.c\nindex 6d69dfe89a96cf..94746317ca79fc 100644\n--- a/arch/arm64/kvm/hyp/vgic-v5-sr.c\n+++ b/arch/arm64/kvm/hyp/vgic-v5-sr.c\n@@ -7,6 +7,41 @@\n \n #include \u003casm/kvm_hyp.h\u003e\n \n+void __vgic_v5_make_resident(struct vgic_v5_cpu_if *cpu_if)\n+{\n+\twrite_sysreg_s(cpu_if-\u003evgic_contextr, SYS_ICH_CONTEXTR_EL2);\n+\tisb();\n+\n+\t/* Catch any faults */\n+\tcpu_if-\u003evgic_contextr = read_sysreg_s(SYS_ICH_CONTEXTR_EL2);\n+\tif (!!FIELD_GET(ICH_CONTEXTR_EL2_F, cpu_if-\u003evgic_contextr))\n+\t\treturn;\n+\n+\tcpu_if-\u003egicv5_vpe.resident = true;\n+}\n+\n+void __vgic_v5_make_non_resident(struct vgic_v5_cpu_if *cpu_if)\n+{\n+\t/*\n+\t * Clear the db_fired state to ensure that we're ready for the next\n+\t * doorbell when it is requested. If a doorbell firing caused us to\n+\t * enter the guest, then we've already consumed that state at this\n+\t * point, so this is safe to clear. Use WRITE_ONCE() to ensure we're not\n+\t * racing with the doorbell firing and setting the state true again.\n+\t */\n+\tWRITE_ONCE(cpu_if-\u003egicv5_vpe.db_fired, false);\n+\n+\t/*\n+\t * Make as non-resident before actually making non-resident. Avoids race\n+\t * with doorbell arriving.\n+\t */\n+\tcpu_if-\u003egicv5_vpe.resident = false;\n+\tdsb(st);\n+\n+\twrite_sysreg_s(cpu_if-\u003evgic_contextr, SYS_ICH_CONTEXTR_EL2);\n+\tisb();\n+}\n+\n void __vgic_v5_save_apr(struct vgic_v5_cpu_if *cpu_if)\n {\n \tcpu_if-\u003evgic_apr = read_sysreg_s(SYS_ICH_APR_EL2);\n@@ -114,3 +149,13 @@ void __vgic_v5_restore_state(struct vgic_v5_cpu_if *cpu_if)\n {\n \twrite_sysreg_s(cpu_if-\u003evgic_icsr, SYS_ICC_ICSR_EL1);\n }\n+\n+void __vgic_v5_vdpend(u32 intid, bool pending, u16 vm)\n+{\n+\tu64 value;\n+\n+\tvalue = intid \u0026 (GICV5_GIC_VDPEND_ID_MASK | GICV5_GIC_VDPEND_TYPE_MASK);\n+\tvalue |= FIELD_PREP(GICV5_GIC_VDPEND_PENDING_MASK, pending);\n+\tvalue |= FIELD_PREP(GICV5_GIC_VDPEND_VM_MASK, vm);\n+\tgic_insn(value, VDPEND);\n+}\ndiff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c\nindex 5d5c579d457902..ba5918413f1645 100644\n--- a/arch/arm64/kvm/sys_regs.c\n+++ b/arch/arm64/kvm/sys_regs.c\n@@ -5827,7 +5827,7 @@ int kvm_finalize_sys_regs(struct kvm_vcpu *vcpu)\n \n int __init kvm_sys_reg_table_init(void)\n {\n-\tconst struct sys_reg_desc *gicv3_regs;\n+\tconst struct sys_reg_desc *gicv3_regs, *gicv5_regs;\n \tbool valid = true;\n \tunsigned int i, sz;\n \tint ret = 0;\n@@ -5840,8 +5840,12 @@ int __init kvm_sys_reg_table_init(void)\n \tvalid \u0026= check_sysreg_table(cp15_64_regs, ARRAY_SIZE(cp15_64_regs), false);\n \tvalid \u0026= check_sysreg_table(sys_insn_descs, ARRAY_SIZE(sys_insn_descs), false);\n \n+\t/* The GICv3 system registers... */\n \tgicv3_regs = vgic_v3_get_sysreg_table(\u0026sz);\n \tvalid \u0026= check_sysreg_table(gicv3_regs, sz, false);\n+\t/* ...and the GICv5 system registers. */\n+\tgicv5_regs = vgic_v5_get_sysreg_table(\u0026sz);\n+\tvalid \u0026= check_sysreg_table(gicv5_regs, sz, false);\n \n \tif (!valid)\n \t\treturn -EINVAL;\ndiff --git a/arch/arm64/kvm/vgic-sys-reg-v5.c b/arch/arm64/kvm/vgic-sys-reg-v5.c\nnew file mode 100644\nindex 00000000000000..6b255eacbf60d8\n--- /dev/null\n+++ b/arch/arm64/kvm/vgic-sys-reg-v5.c\n@@ -0,0 +1,519 @@\n+// SPDX-License-Identifier: GPL-2.0-only\n+/*\n+ * Copyright (C) 2025, 2026 Arm Ltd.\n+ */\n+\n+/*\n+ * VGICv5 system registers handling functions for AArch64 mode\n+ */\n+\n+#include \u003clinux/irqchip/arm-gic-v5.h\u003e\n+\n+#include \u003clinux/kvm.h\u003e\n+#include \u003clinux/kvm_host.h\u003e\n+#include \u003clinux/wordpart.h\u003e\n+\n+#include \u003casm/kvm_emulate.h\u003e\n+\n+#include \"vgic/vgic.h\"\n+#include \"sys_regs.h\"\n+\n+#define ICC_PPI_PRIORITYR_PRIORITY_MASK\t\tREPEAT_BYTE(0x1f)\n+\n+static int set_gic_apr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,\n+\t\t u64 val)\n+{\n+\tstruct vgic_v5_cpu_if *cpu_if = \u0026vcpu-\u003earch.vgic_cpu.vgic_v5;\n+\n+\t/* The upper 32 bits are RES0 */\n+\tcpu_if-\u003evgic_apr = val \u0026 ~ICC_APR_EL1_RES0;\n+\n+\treturn 0;\n+}\n+\n+static int get_gic_apr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,\n+\t\t u64 *val)\n+{\n+\tstruct vgic_v5_cpu_if *cpu_if = \u0026vcpu-\u003earch.vgic_cpu.vgic_v5;\n+\n+\t*val = cpu_if-\u003evgic_apr;\n+\n+\treturn 0;\n+}\n+\n+static int set_gic_cr0(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,\n+\t\t u64 val)\n+{\n+\tstruct vgic_v5_cpu_if *cpu_if = \u0026vcpu-\u003earch.vgic_cpu.vgic_v5;\n+\n+\t/*\n+\t * We only support setting the ICC_CR0_EL1.En bit, which is actually\n+\t * stored in the VMCR.\n+\t */\n+\tFIELD_MODIFY(FEAT_GCIE_ICH_VMCR_EL2_EN, \u0026cpu_if-\u003evgic_vmcr,\n+\t\t FIELD_GET(ICC_CR0_EL1_EN, val));\n+\n+\treturn 0;\n+}\n+\n+static int get_gic_cr0(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,\n+\t\t u64 *val)\n+{\n+\tstruct vgic_v5_cpu_if *cpu_if = \u0026vcpu-\u003earch.vgic_cpu.vgic_v5;\n+\n+\t/*\n+\t * PID only applies if EL3 is present. Same applies to IPPT. Hence,\n+\t * those fields are always presented as 0.\n+\t *\n+\t * We always present the link as connected and idle:\n+\t * (LINK = 1, LINK_IDLE = 1).\n+\t */\n+\t*val = FIELD_PREP(ICC_CR0_EL1_EN,\n+\t\t\t FIELD_GET(FEAT_GCIE_ICH_VMCR_EL2_EN, cpu_if-\u003evgic_vmcr));\n+\t*val |= ICC_CR0_EL1_LINK_MASK;\n+\t*val |= ICC_CR0_EL1_LINK_IDLE_MASK;\n+\n+\treturn 0;\n+}\n+\n+static int set_gic_pcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,\n+\t\t u64 val)\n+{\n+\tstruct vgic_v5_cpu_if *cpu_if = \u0026vcpu-\u003earch.vgic_cpu.vgic_v5;\n+\n+\t/* Set the VPMR field in the VMCR */\n+\tFIELD_MODIFY(FEAT_GCIE_ICH_VMCR_EL2_VPMR, \u0026cpu_if-\u003evgic_vmcr,\n+\t\t FIELD_GET(ICC_PCR_EL1_PRIORITY, val));\n+\n+\treturn 0;\n+}\n+\n+static int get_gic_pcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,\n+\t\t u64 *val)\n+{\n+\tstruct vgic_v5_cpu_if *cpu_if = \u0026vcpu-\u003earch.vgic_cpu.vgic_v5;\n+\n+\t*val = FIELD_PREP(ICC_PCR_EL1_PRIORITY,\n+\t\t\t FIELD_GET(FEAT_GCIE_ICH_VMCR_EL2_VPMR, cpu_if-\u003evgic_vmcr));\n+\n+\treturn 0;\n+}\n+\n+static int set_gic_icsr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,\n+\t\t\tu64 val)\n+{\n+\tstruct vgic_v5_cpu_if *cpu_if = \u0026vcpu-\u003earch.vgic_cpu.vgic_v5;\n+\n+\tcpu_if-\u003evgic_icsr = val \u0026 ~ICC_ICSR_EL1_RES0;\n+\n+\treturn 0;\n+}\n+\n+static int get_gic_icsr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,\n+\t\t\tu64 *val)\n+{\n+\tstruct vgic_v5_cpu_if *cpu_if = \u0026vcpu-\u003earch.vgic_cpu.vgic_v5;\n+\n+\t*val = cpu_if-\u003evgic_icsr;\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Helper macro to iterate over a range of PPIs and execute some code (to either\n+ * extract or set the vgic_irq state). This is used when `get`-ing the PPI\n+ * ENABLER, ACTIVER, PENDR and when setting the PRIORITYR state.\n+ *\n+ * vcpu: Pointer to struct kvm_vcpu (to which these PPIs belong)\n+ * r: The register index. 0 or 1 for all except PRIORITYR (which is 0-15)\n+ * nr: The number of PPIs iterated over. 64 for all but PRIORITYR (which is 8)\n+ * code: The code snippet to execute for each vgic_irq\n+ */\n+#define for_ppi_state(vcpu, r, nr, code)\t\t\t\t\\\n+\tdo {\t\t\t\t\t\t\t\t\\\n+\t\tstruct kvm_vcpu *__vcpu = (vcpu);\t\t\t\\\n+\t\tint __r = (r);\t\t\t\t\t\t\\\n+\t\tint __nr = (nr);\t\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\t\tfor (int i = 0; i \u003c __nr; i++) {\t\t\t\\\n+\t\t\tu32 id = vgic_v5_make_ppi(__r * __nr + i);\t\\\n+\t\t\tstruct vgic_irq *irq;\t\t\t\t\\\n+\t\t\t\t\t\t\t\t\t\\\n+\t\t\tirq = vgic_get_vcpu_irq(__vcpu, id);\t\t\\\n+\t\t\tscoped_guard(raw_spinlock_irqsave, \u0026irq-\u003eirq_lock) { \\\n+\t\t\t\tcode;\t\t\t\t\t\\\n+\t\t\t}\t\t\t\t\t\t\\\n+\t\t\tvgic_put_irq(__vcpu-\u003ekvm, irq);\t\t\t\\\n+\t\t}\t\t\t\t\t\t\t\\\n+\t} while (0)\n+\n+static int set_gic_ppi_enabler(struct kvm_vcpu *vcpu,\n+\t\t\t const struct sys_reg_desc *r, u64 val)\n+{\n+\tstruct vgic_v5_cpu_if *cpu_if = \u0026vcpu-\u003earch.vgic_cpu.vgic_v5;\n+\tint i, start, end, reg = r-\u003eOp2 % 2;\n+\n+\t/*\n+\t * If we're only handling architected PPIs and the guest writes to the\n+\t * enable for the non-architected PPIs, we just return as there's\n+\t * nothing to do at all. We don't even allocate the storage for them in\n+\t * this case.\n+\t */\n+\tif (VGIC_V5_NR_PRIVATE_IRQS == 64 \u0026\u0026 reg == 1)\n+\t\treturn 0;\n+\n+\t/*\n+\t * Merge the raw guest write into our bitmap at an offset of either 0 or\n+\t * 64.\n+\t *\n+\t * Note that there is *NO* masking applied - the enable state is written\n+\t * unfiltered. The assumption is that userspace uses this interface to\n+\t * set initial state before the guest runs, and then the exposed PPI\n+\t * mask is applied later, when vgic_v5_finalize_ppi_state() runs on\n+\t * first entry to each vCPU. If userspace chooses to set the enabler\n+\t * state later, it is fully capable of breaking the illusion we provided\n+\t * to the guest by exposing register state (and PPIs) to the guest that\n+\t * were not initially exposed. Good luck!\n+\t */\n+\tbitmap_write(cpu_if-\u003evgic_ppi_enabler, val, 64 * reg, 64);\n+\n+\t/*\n+\t * Sync the change in enable states to the vgic_irqs for the written\n+\t * register slice.\n+\t */\n+\tstart = reg * 64;\n+\tend = min(start + 64, VGIC_V5_NR_PRIVATE_IRQS);\n+\tfor (i = start; i \u003c end; i++) {\n+\t\tu32 intid = vgic_v5_make_ppi(i);\n+\t\tstruct vgic_irq *irq;\n+\n+\t\tirq = vgic_get_vcpu_irq(vcpu, intid);\n+\n+\t\tscoped_guard(raw_spinlock_irqsave, \u0026irq-\u003eirq_lock)\n+\t\t\tirq-\u003eenabled = test_bit(i, cpu_if-\u003evgic_ppi_enabler);\n+\n+\t\tvgic_put_irq(vcpu-\u003ekvm, irq);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int get_gic_ppi_enabler(struct kvm_vcpu *vcpu,\n+\t\t\t const struct sys_reg_desc *r, u64 *val)\n+{\n+\tunsigned long enabler = 0;\n+\tint reg = r-\u003eOp2 % 2;\n+\n+\t/* If we only support architected PPIs, return 0 */\n+\tif (VGIC_V5_NR_PRIVATE_IRQS == 64 \u0026\u0026 reg == 1) {\n+\t\t*val = 0;\n+\t\treturn 0;\n+\t}\n+\n+\t/* Iterate over each struct vgic_irq to build the ENABLER value. */\n+\tfor_ppi_state(vcpu, reg, 64, __assign_bit(i % 64, \u0026enabler, irq-\u003eenabled));\n+\n+\t*val = enabler;\n+\n+\treturn 0;\n+}\n+\n+static int set_gic_ppi_activer(struct kvm_vcpu *vcpu,\n+\t\t\t const struct sys_reg_desc *r, u64 val)\n+{\n+\tstruct vgic_v5_cpu_if *cpu_if = \u0026vcpu-\u003earch.vgic_cpu.vgic_v5;\n+\tint i, start, end, reg = r-\u003eOp2 % 2;\n+\n+\tif (VGIC_V5_NR_PRIVATE_IRQS == 64 \u0026\u0026 reg == 1)\n+\t\treturn 0;\n+\n+\t/*\n+\t * Store the raw guest write. The exposed PPI mask is applied later,\n+\t * when vgic_v5_finalize_ppi_state() runs on first entry to each\n+\t * vCPU. See comment on set_gic_ppi_enabler() for details.\n+\t */\n+\tbitmap_write(cpu_if-\u003evgic_ppi_activer, val, 64 * reg, 64);\n+\n+\tstart = reg * 64;\n+\tend = min(start + 64, VGIC_V5_NR_PRIVATE_IRQS);\n+\tfor (i = start; i \u003c end; i++) {\n+\t\tu32 intid = vgic_v5_make_ppi(i);\n+\t\tstruct vgic_irq *irq;\n+\n+\t\tirq = vgic_get_vcpu_irq(vcpu, intid);\n+\n+\t\tscoped_guard(raw_spinlock_irqsave, \u0026irq-\u003eirq_lock)\n+\t\t\tirq-\u003eactive = test_bit(i, cpu_if-\u003evgic_ppi_activer);\n+\n+\t\tvgic_put_irq(vcpu-\u003ekvm, irq);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int get_gic_ppi_activer(struct kvm_vcpu *vcpu,\n+\t\t\t const struct sys_reg_desc *r, u64 *val)\n+{\n+\tunsigned long activer = 0;\n+\tint reg = r-\u003eOp2 % 2;\n+\n+\t/* If we only support architected PPIs, return 0 */\n+\tif (VGIC_V5_NR_PRIVATE_IRQS == 64 \u0026\u0026 reg == 1) {\n+\t\t*val = 0;\n+\t\treturn 0;\n+\t}\n+\n+\t/* Iterate over each struct vgic_irq to build the ACTIVER value. */\n+\tfor_ppi_state(vcpu, reg, 64, __assign_bit(i % 64, \u0026activer, irq-\u003eactive));\n+\n+\t*val = activer;\n+\n+\treturn 0;\n+}\n+\n+static int set_gic_ppi_pendr(struct kvm_vcpu *vcpu,\n+\t\t\t const struct sys_reg_desc *r, u64 val)\n+{\n+\tint i, start, end, reg = r-\u003eOp2 % 2;\n+\n+\t/* If we only support architected PPIs, return */\n+\tif (VGIC_V5_NR_PRIVATE_IRQS == 64 \u0026\u0026 reg == 1)\n+\t\treturn 0;\n+\n+\t/*\n+\t * Update each struct vgic_irq with the pending state, treating Level\n+\t * and Edge interrupts differently. The exposed PPI mask is applied\n+\t * later, when vgic_v5_finalize_ppi_state() runs on first entry to each\n+\t * vCPU. See comment on set_gic_ppi_enabler() for details.\n+\t */\n+\tstart = reg * 64;\n+\tend = min(start + 64, VGIC_V5_NR_PRIVATE_IRQS);\n+\tfor (i = start; i \u003c end; i++) {\n+\t\tu32 intid = vgic_v5_make_ppi(i);\n+\t\tstruct vgic_irq *irq;\n+\n+\t\tirq = vgic_get_vcpu_irq(vcpu, intid);\n+\n+\t\tscoped_guard(raw_spinlock_irqsave, \u0026irq-\u003eirq_lock) {\n+\t\t\tbool level = !!(val \u0026 BIT_ULL(i - start));\n+\n+\t\t\tif (irq-\u003econfig == VGIC_CONFIG_LEVEL)\n+\t\t\t\tirq-\u003eline_level = level;\n+\t\t\telse\n+\t\t\t\tirq-\u003epending_latch = level;\n+\t\t}\n+\n+\t\tvgic_put_irq(vcpu-\u003ekvm, irq);\n+\t}\n+\n+\t/*\n+\t * The pending state is generated from the vgic_irqs on each guest\n+\t * entry. Therefore, we don't store the raw value written anywhere in\n+\t * the case of userspace PPI_PENDRx_EL1 writes.\n+\t */\n+\n+\treturn 0;\n+}\n+\n+static int get_gic_ppi_pendr(struct kvm_vcpu *vcpu,\n+\t\t\t const struct sys_reg_desc *r, u64 *val)\n+{\n+\tunsigned long pendr = 0;\n+\tint reg = r-\u003eOp2 % 2;\n+\n+\t/* If we only support architected PPIs, return 0 */\n+\tif (VGIC_V5_NR_PRIVATE_IRQS == 64 \u0026\u0026 reg == 1) {\n+\t\t*val = 0;\n+\t\treturn 0;\n+\t}\n+\n+\t/* Iterate over each struct vgic_irq to build the PENDR value. */\n+\tfor_ppi_state(vcpu, reg, 64, {\n+\t\tif (irq_is_pending(irq))\n+\t\t\t__assign_bit(i % 64, \u0026pendr, 1);\n+\t});\n+\n+\t*val = pendr;\n+\n+\treturn 0;\n+}\n+\n+static int set_gic_ppi_priorityr(struct kvm_vcpu *vcpu,\n+\t\t\t\t const struct sys_reg_desc *r, u64 val)\n+{\n+\tstruct vgic_v5_cpu_if *cpu_if = \u0026vcpu-\u003earch.vgic_cpu.vgic_v5;\n+\tint reg = ((r-\u003eCRm \u0026 0x1) \u003c\u003c 3) + r-\u003eOp2;\n+\n+\t/* If we only support architected PPIs, return */\n+\tif (VGIC_V5_NR_PRIVATE_IRQS == 64 \u0026\u0026 reg \u003e 7)\n+\t\treturn 0;\n+\n+\tval \u0026= ICC_PPI_PRIORITYR_PRIORITY_MASK;\n+\n+\t/*\n+\t * Although priorities are not regularly synced back to the vgic_irq\n+\t * state, they are explicitly synced back here. This is to ensure that\n+\t * any pending PPIs are evaluated correctly when first running the guest\n+\t * after setting the state.\n+\t */\n+\tfor_ppi_state(vcpu, reg, 8,\n+\t\t irq-\u003epriority = (u8)(val \u003e\u003e (8 * i));\n+\t\t);\n+\n+\t/*\n+\t * Update the state that will be written to the ICH_PPI_PRIORITYRx_EL2\n+\t * on next guest entry.\n+\t */\n+\tcpu_if-\u003evgic_ppi_priorityr[reg] = val;\n+\n+\treturn 0;\n+}\n+\n+static int get_gic_ppi_priorityr(struct kvm_vcpu *vcpu,\n+\t\t\t\t const struct sys_reg_desc *r, u64 *val)\n+{\n+\tstruct vgic_v5_cpu_if *cpu_if = \u0026vcpu-\u003earch.vgic_cpu.vgic_v5;\n+\tint reg = ((r-\u003eCRm \u0026 0x1) \u003c\u003c 3) + r-\u003eOp2;\n+\n+\t/* If we only support architected PPIs, return 0 */\n+\tif (VGIC_V5_NR_PRIVATE_IRQS == 64 \u0026\u0026 reg \u003e 7) {\n+\t\t*val = 0;\n+\t\treturn 0;\n+\t}\n+\n+\t/*\n+\t * The priorities are only synced back to the vgic_irq state when the\n+\t * vcpu is entering WFI (KVM only needs to know the priorities when\n+\t * evaluating if there are pending PPI interrupts for a vcpu). The raw\n+\t * register ICH_PPI_PRIORITYRx_EL1 state is simply saved and restored\n+\t * blindly. This state is just returned as it contains the most recent\n+\t * priorities written by the guest.\n+\t */\n+\t*val = cpu_if-\u003evgic_ppi_priorityr[reg];\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * The following registers are NOT supported:\n+ *\n+ * - ICC_HAPR_EL1\n+ *\tThe value of this is directly generated by the GICv5 hardware based on\n+ *\tthe ICC_APR_EL1 when the guest is running.\n+ * - ICC_IAFFIDR_EL1\n+ *\tThe IAFFID for a GICv5 VPE is the same as the VPE ID, which is the index\n+ *\tinto the in-memory VPE Table. This is not configurable, and instead we\n+ *\trely on userspace recreating the VPEs in the same order prior to\n+ *\trestoring guest state.\n+ * - ICC_PPI_CACTIVER\u003cn\u003e_EL1\n+ *\tOnly raw state writes are supported via the S(et) variant.\n+ * - ICC_PPI_CPENDR\u003cn\u003e_EL1\n+ *\tOnly raw state writes are supported via the S(et) variant.\n+ */\n+static const struct sys_reg_desc gic_v5_icc_reg_descs[] = {\n+\t{ SYS_DESC(SYS_ICC_ICSR_EL1),\n+\t .set_user = set_gic_icsr, .get_user = get_gic_icsr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_ENABLER0_EL1),\n+\t .set_user = set_gic_ppi_enabler, .get_user = get_gic_ppi_enabler, },\n+\t{ SYS_DESC(SYS_ICC_PPI_ENABLER1_EL1),\n+\t .set_user = set_gic_ppi_enabler, .get_user = get_gic_ppi_enabler, },\n+\t/*\n+\t * Only ICC_SACTIVER\u003cn\u003e_EL1 is exposed to the guest. This is treated as\n+\t * a *RAW* write of register state for writes.\n+\t */\n+\t{ SYS_DESC(SYS_ICC_PPI_SACTIVER0_EL1),\n+\t .set_user = set_gic_ppi_activer, .get_user = get_gic_ppi_activer, },\n+\t{ SYS_DESC(SYS_ICC_PPI_SACTIVER1_EL1),\n+\t .set_user = set_gic_ppi_activer, .get_user = get_gic_ppi_activer, },\n+\t/*\n+\t * Only ICC_SPENDR\u003cn\u003e_EL1 is exposed to the guest. This is treated as\n+\t * a *RAW* write of register state for writes.\n+\t */\n+\t{ SYS_DESC(SYS_ICC_PPI_SPENDR0_EL1),\n+\t .set_user = set_gic_ppi_pendr, .get_user = get_gic_ppi_pendr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_SPENDR1_EL1),\n+\t .set_user = set_gic_ppi_pendr, .get_user = get_gic_ppi_pendr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR0_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR1_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR2_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR3_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR4_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR5_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR6_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR7_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR8_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR9_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR10_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR11_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR12_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR13_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR14_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_PPI_PRIORITYR15_EL1),\n+\t .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },\n+\t{ SYS_DESC(SYS_ICC_APR_EL1),\n+\t .set_user = set_gic_apr, .get_user = get_gic_apr, },\n+\t{ SYS_DESC(SYS_ICC_CR0_EL1),\n+\t .set_user = set_gic_cr0, .get_user = get_gic_cr0, },\n+\t{ SYS_DESC(SYS_ICC_PCR_EL1),\n+\t .set_user = set_gic_pcr, .get_user = get_gic_pcr, },\n+};\n+\n+const struct sys_reg_desc *vgic_v5_get_sysreg_table(unsigned int *sz)\n+{\n+\t*sz = ARRAY_SIZE(gic_v5_icc_reg_descs);\n+\treturn gic_v5_icc_reg_descs;\n+}\n+\n+static u64 attr_to_id(u64 attr)\n+{\n+\treturn ARM64_SYS_REG(FIELD_GET(KVM_REG_ARM_VGIC_SYSREG_OP0_MASK, attr),\n+\t\t\t FIELD_GET(KVM_REG_ARM_VGIC_SYSREG_OP1_MASK, attr),\n+\t\t\t FIELD_GET(KVM_REG_ARM_VGIC_SYSREG_CRN_MASK, attr),\n+\t\t\t FIELD_GET(KVM_REG_ARM_VGIC_SYSREG_CRM_MASK, attr),\n+\t\t\t FIELD_GET(KVM_REG_ARM_VGIC_SYSREG_OP2_MASK, attr));\n+}\n+\n+int vgic_v5_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)\n+{\n+\tconst struct sys_reg_desc *r;\n+\n+\tr = get_reg_by_id(attr_to_id(attr-\u003eattr), gic_v5_icc_reg_descs,\n+\t\t\t ARRAY_SIZE(gic_v5_icc_reg_descs));\n+\n+\tif (r \u0026\u0026 !sysreg_hidden(vcpu, r))\n+\t\treturn 0;\n+\n+\treturn -ENXIO;\n+}\n+\n+int vgic_v5_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu,\n+\t\t\t\tstruct kvm_device_attr *attr,\n+\t\t\t\tbool is_write)\n+{\n+\tstruct kvm_one_reg reg = {\n+\t\t.id\t= attr_to_id(attr-\u003eattr),\n+\t\t.addr\t= attr-\u003eaddr,\n+\t};\n+\n+\tif (is_write)\n+\t\treturn kvm_sys_reg_set_user(vcpu, \u0026reg, gic_v5_icc_reg_descs,\n+\t\t\t\t\t ARRAY_SIZE(gic_v5_icc_reg_descs));\n+\telse\n+\t\treturn kvm_sys_reg_get_user(vcpu, \u0026reg, gic_v5_icc_reg_descs,\n+\t\t\t\t\t ARRAY_SIZE(gic_v5_icc_reg_descs));\n+}\ndiff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-init.c\nindex 907057881b26aa..8a38d915287742 100644\n--- a/arch/arm64/kvm/vgic/vgic-init.c\n+++ b/arch/arm64/kvm/vgic/vgic-init.c\n@@ -129,13 +129,17 @@ int kvm_vgic_create(struct kvm *kvm, u32 type)\n \t}\n \tret = 0;\n \n-\tif (type == KVM_DEV_TYPE_ARM_VGIC_V2)\n+\tswitch (type) {\n+\tcase KVM_DEV_TYPE_ARM_VGIC_V2:\n \t\tkvm-\u003emax_vcpus = VGIC_V2_MAX_CPUS;\n-\telse if (type == KVM_DEV_TYPE_ARM_VGIC_V3)\n+\t\tbreak;\n+\tcase KVM_DEV_TYPE_ARM_VGIC_V3:\n \t\tkvm-\u003emax_vcpus = VGIC_V3_MAX_CPUS;\n-\telse if (type == KVM_DEV_TYPE_ARM_VGIC_V5)\n-\t\tkvm-\u003emax_vcpus = min(VGIC_V5_MAX_CPUS,\n-\t\t\t\t kvm_vgic_global_state.max_gic_vcpus);\n+\t\tbreak;\n+\tcase KVM_DEV_TYPE_ARM_VGIC_V5:\n+\t\tkvm-\u003emax_vcpus = kvm_vgic_global_state.max_gicv5_vcpus;\n+\t\tbreak;\n+\t}\n \n \tif (atomic_read(\u0026kvm-\u003eonline_vcpus) \u003e kvm-\u003emax_vcpus) {\n \t\tret = -E2BIG;\n@@ -154,6 +158,8 @@ int kvm_vgic_create(struct kvm *kvm, u32 type)\n \tcase KVM_DEV_TYPE_ARM_VGIC_V3:\n \t\tINIT_LIST_HEAD(\u0026kvm-\u003earch.vgic.rd_regions);\n \t\tbreak;\n+\tcase KVM_DEV_TYPE_ARM_VGIC_V5:\n+\t\tkvm-\u003earch.vgic.gicv5_vm.vm_id = VGIC_V5_VM_ID_INVAL;\n \t}\n \n \t/*\n@@ -168,28 +174,48 @@ int kvm_vgic_create(struct kvm *kvm, u32 type)\n \t\t\tbreak;\n \t}\n \n-\tif (ret) {\n-\t\tkvm_for_each_vcpu(i, vcpu, kvm) {\n-\t\t\tstruct vgic_cpu *vgic_cpu = \u0026vcpu-\u003earch.vgic_cpu;\n-\t\t\tkfree(vgic_cpu-\u003eprivate_irqs);\n-\t\t\tvgic_cpu-\u003eprivate_irqs = NULL;\n-\t\t}\n-\n-\t\tkvm-\u003earch.vgic.vgic_model = 0;\n-\t\tgoto out_unlock;\n-\t}\n+\tif (ret)\n+\t\tgoto out_free_private_irqs;\n \n \tif (type == KVM_DEV_TYPE_ARM_VGIC_V3)\n \t\tkvm-\u003earch.vgic.nassgicap = system_supports_direct_sgis();\n \n-\t/*\n-\t * We now know that we have a GICv5. The Arch Timer PPI interrupts may\n-\t * have been initialised at this stage, but will have done so assuming\n-\t * that we have an older GIC, meaning that the IntIDs won't be\n-\t * correct. We init them again, and this time they will be correct.\n-\t */\n-\tif (type == KVM_DEV_TYPE_ARM_VGIC_V5)\n+\tif (type == KVM_DEV_TYPE_ARM_VGIC_V5) {\n+\t\t/* Allocate a vIRS for GICv5 systems */\n+\t\tkvm-\u003earch.vgic.vgic_v5_irs_data = kzalloc_obj(struct vgic_v5_irs,\n+\t\t\t\t\t\t\t GFP_KERNEL_ACCOUNT);\n+\t\tif (!kvm-\u003earch.vgic.vgic_v5_irs_data) {\n+\t\t\tret = -ENOMEM;\n+\t\t\tgoto out_free_private_irqs;\n+\t\t}\n+\n+\t\t/*\n+\t\t * Initialization happens later, for now just explicitly\n+\t\t * disable the device and undef its base address.\n+\t\t */\n+\t\tkvm-\u003earch.vgic.vgic_v5_irs_data-\u003evgic_v5_irs_base = VGIC_ADDR_UNDEF;\n+\n+\t\t/*\n+\t\t * We now know that we have a GICv5. The Arch Timer PPI\n+\t\t * interrupts may have been initialised at this stage, but will\n+\t\t * have done so assuming that we have an older GIC, meaning that\n+\t\t * the IntIDs won't be correct. We init them again, and this\n+\t\t * time they will be correct.\n+\t\t */\n \t\tkvm_timer_init_vm(kvm);\n+\t}\n+\n+\tgoto out_unlock;\n+\n+out_free_private_irqs:\n+\tkvm_for_each_vcpu(i, vcpu, kvm) {\n+\t\tstruct vgic_cpu *vgic_cpu = \u0026vcpu-\u003earch.vgic_cpu;\n+\n+\t\tkfree(vgic_cpu-\u003eprivate_irqs);\n+\t\tvgic_cpu-\u003eprivate_irqs = NULL;\n+\t}\n+\n+\tkvm-\u003earch.vgic.vgic_model = 0;\n \n out_unlock:\n \tmutex_unlock(\u0026kvm-\u003earch.config_lock);\n@@ -461,6 +487,9 @@ int vgic_init(struct kvm *kvm)\n \t\t\t\treturn ret;\n \t\t}\n \t} else {\n+\t\tif (!dist-\u003enr_spis)\n+\t\t\tdist-\u003enr_spis = VGIC_V5_DEFAULT_NR_SPIS;\n+\n \t\tret = vgic_v5_init(kvm);\n \t\tif (ret)\n \t\t\treturn ret;\n@@ -492,16 +521,24 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)\n \tdist-\u003enr_spis = 0;\n \tdist-\u003evgic_dist_base = VGIC_ADDR_UNDEF;\n \n-\tif (dist-\u003evgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) {\n+\tswitch (dist-\u003evgic_model) {\n+\tcase KVM_DEV_TYPE_ARM_VGIC_V2:\n+\t\tdist-\u003evgic_cpu_base = VGIC_ADDR_UNDEF;\n+\t\tbreak;\n+\tcase KVM_DEV_TYPE_ARM_VGIC_V3:\n \t\tlist_for_each_entry_safe(rdreg, next, \u0026dist-\u003erd_regions, list)\n \t\t\tvgic_v3_free_redist_region(kvm, rdreg);\n \t\tINIT_LIST_HEAD(\u0026dist-\u003erd_regions);\n-\t} else {\n-\t\tdist-\u003evgic_cpu_base = VGIC_ADDR_UNDEF;\n-\t}\n \n-\tif (vgic_supports_direct_irqs(kvm))\n-\t\tvgic_v4_teardown(kvm);\n+\t\tif (vgic_supports_direct_irqs(kvm))\n+\t\t\tvgic_v4_teardown(kvm);\n+\t\tbreak;\n+\tcase KVM_DEV_TYPE_ARM_VGIC_V5:\n+\t\tvgic_v5_teardown(kvm);\n+\t\tkfree(dist-\u003evgic_v5_irs_data);\n+\t\tdist-\u003evgic_v5_irs_data = NULL;\n+\t\tbreak;\n+\t}\n \n \txa_destroy(\u0026dist-\u003elpi_xa);\n }\n@@ -620,9 +657,8 @@ int vgic_lazy_init(struct kvm *kvm)\n int kvm_vgic_map_resources(struct kvm *kvm)\n {\n \tstruct vgic_dist *dist = \u0026kvm-\u003earch.vgic;\n-\tbool needs_dist = true;\n \tenum vgic_type type;\n-\tgpa_t dist_base;\n+\tgpa_t dist_base, irs_base;\n \tint ret = 0;\n \n \tif (likely(smp_load_acquire(\u0026dist-\u003eready)))\n@@ -645,13 +681,12 @@ int kvm_vgic_map_resources(struct kvm *kvm)\n \t} else {\n \t\tret = vgic_v5_map_resources(kvm);\n \t\ttype = VGIC_V5;\n-\t\tneeds_dist = false;\n \t}\n \n \tif (ret)\n \t\tgoto out;\n \n-\tif (needs_dist) {\n+\tif (type != VGIC_V5) {\n \t\tdist_base = dist-\u003evgic_dist_base;\n \t\tmutex_unlock(\u0026kvm-\u003earch.config_lock);\n \n@@ -661,7 +696,19 @@ int kvm_vgic_map_resources(struct kvm *kvm)\n \t\t\tgoto out_slots;\n \t\t}\n \t} else {\n+\t\tirs_base = dist-\u003evgic_v5_irs_data-\u003evgic_v5_irs_base;\n \t\tmutex_unlock(\u0026kvm-\u003earch.config_lock);\n+\n+\t\tif (IS_VGIC_ADDR_UNDEF(irs_base)) {\n+\t\t\tret = -ENXIO;\n+\t\t\tgoto out_slots;\n+\t\t}\n+\n+\t\tret = vgic_v5_register_irs_iodev(kvm, irs_base);\n+\t\tif (ret) {\n+\t\t\tkvm_err(\"Unable to register VGIC IRS MMIO regions\\n\");\n+\t\t\tgoto out_slots;\n+\t\t}\n \t}\n \n \tsmp_store_release(\u0026dist-\u003eready, true);\ndiff --git a/arch/arm64/kvm/vgic/vgic-irqfd.c b/arch/arm64/kvm/vgic/vgic-irqfd.c\nindex 19a1094536e6a8..83e84151d7aa26 100644\n--- a/arch/arm64/kvm/vgic/vgic-irqfd.c\n+++ b/arch/arm64/kvm/vgic/vgic-irqfd.c\n@@ -19,9 +19,14 @@ static int vgic_irqfd_set_irq(struct kvm_kernel_irq_routing_entry *e,\n \t\t\tstruct kvm *kvm, int irq_source_id,\n \t\t\tint level, bool line_status)\n {\n-\tunsigned int spi_id = e-\u003eirqchip.pin + VGIC_NR_PRIVATE_IRQS;\n+\tunsigned int spi_id;\n \tint ret;\n \n+\tif (kvm-\u003earch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V5)\n+\t\tspi_id = vgic_v5_make_spi(e-\u003eirqchip.pin);\n+\telse\n+\t\tspi_id = e-\u003eirqchip.pin + VGIC_NR_PRIVATE_IRQS;\n+\n \tif (!vgic_valid_spi(kvm, spi_id))\n \t\treturn -EINVAL;\n \n@@ -52,8 +57,8 @@ int kvm_set_routing_entry(struct kvm *kvm,\n \t\te-\u003eset = vgic_irqfd_set_irq;\n \t\te-\u003eirqchip.irqchip = ue-\u003eu.irqchip.irqchip;\n \t\te-\u003eirqchip.pin = ue-\u003eu.irqchip.pin;\n-\t\tif ((e-\u003eirqchip.pin \u003e= KVM_IRQCHIP_NUM_PINS) ||\n-\t\t (e-\u003eirqchip.irqchip \u003e= KVM_NR_IRQCHIPS))\n+\t\tif (e-\u003eirqchip.pin \u003e= KVM_IRQCHIP_NUM_PINS ||\n+\t\t e-\u003eirqchip.irqchip \u003e= KVM_NR_IRQCHIPS)\n \t\t\tgoto out;\n \t\tbreak;\n \tcase KVM_IRQ_ROUTING_MSI:\ndiff --git a/arch/arm64/kvm/vgic/vgic-irs-v5.c b/arch/arm64/kvm/vgic/vgic-irs-v5.c\nnew file mode 100644\nindex 00000000000000..be580d9a00cd58\n--- /dev/null\n+++ b/arch/arm64/kvm/vgic/vgic-irs-v5.c\n@@ -0,0 +1,1136 @@\n+// SPDX-License-Identifier: GPL-2.0-only\n+/*\n+ * Copyright (C) 2025 ARM Limited, All Rights Reserved.\n+ */\n+#include \u003clinux/bitops.h\u003e\n+#include \u003clinux/bsearch.h\u003e\n+#include \u003clinux/interrupt.h\u003e\n+#include \u003clinux/irq.h\u003e\n+#include \u003clinux/kvm.h\u003e\n+#include \u003clinux/kvm_host.h\u003e\n+#include \u003ckvm/iodev.h\u003e\n+#include \u003ckvm/arm_arch_timer.h\u003e\n+#include \u003ckvm/arm_vgic.h\u003e\n+\n+#include \"vgic.h\"\n+#include \"vgic-mmio.h\"\n+#include \"vgic-v5-tables.h\"\n+\n+#define irs_caps kvm_vgic_global_state.vgic_v5_irs_caps\n+\n+static struct vgic_dist *vgic_v5_get_vgic(struct kvm_vcpu *vcpu)\n+{\n+\treturn \u0026vcpu-\u003ekvm-\u003earch.vgic;\n+}\n+\n+static struct vgic_v5_irs *vgic_v5_get_irs(struct kvm_vcpu *vcpu)\n+{\n+\treturn vcpu-\u003ekvm-\u003earch.vgic.vgic_v5_irs_data;\n+}\n+\n+static unsigned long vgic_v5_mmio_read_irs_misc(struct kvm_vcpu *vcpu,\n+\t\t\t\t\t\tgpa_t addr, unsigned int len)\n+{\n+\tstruct vgic_v5_irs *irs = vgic_v5_get_irs(vcpu);\n+\tconst size_t offset = addr \u0026 (SZ_64K - 1);\n+\tstruct kvm_vcpu *target_vcpu;\n+\tu8 vpe_id_bits;\n+\tu64 value = 0;\n+\n+\tswitch (offset) {\n+\tcase GICV5_IRS_IDR0:\n+\t\tvalue = FIELD_PREP(GICV5_IRS_IDR0_INT_DOM, irs-\u003eidr0.domain);\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_IDR0_PA_RANGE, irs-\u003eidr0.pa_range);\n+\t\tif (irs-\u003eidr0.virt)\n+\t\t\tvalue |= GICV5_IRS_IDR0_VIRT;\n+\t\tif (irs-\u003eidr0.setlpi)\n+\t\t\tvalue |= GICV5_IRS_IDR0_SETLPI;\n+\t\tif (irs-\u003eidr0.mec)\n+\t\t\tvalue |= GICV5_IRS_IDR0_MEC;\n+\t\tif (irs-\u003eidr0.mpam)\n+\t\t\tvalue |= GICV5_IRS_IDR0_MPAM;\n+\t\tif (irs-\u003eidr0.swe)\n+\t\t\tvalue |= GICV5_IRS_IDR0_SWE;\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_IDR0_IRSID, irs-\u003eidr0.irs_id);\n+\t\tbreak;\n+\tcase GICV5_IRS_IDR1:\n+\t\tvalue = FIELD_PREP(GICV5_IRS_IDR1_PE_CNT,\n+\t\t\t\t atomic_read(\u0026vcpu-\u003ekvm-\u003eonline_vcpus));\n+\t\t/*\n+\t\t * IRS_IDR1 encodes IAFFID_BITS as N - 1.\n+\t\t */\n+\t\tvpe_id_bits = vgic_v5_vmte_vpe_id_bits(vcpu);\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_IDR1_IAFFID_BITS, vpe_id_bits - 1);\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_IDR1_PRIORITY_BITS, irs-\u003eidr1.priority_bits);\n+\t\tbreak;\n+\tcase GICV5_IRS_IDR2:\n+\t\tvalue = FIELD_PREP(GICV5_IRS_IDR2_ISTMD_SZ, irs-\u003eidr2.istmd_sz);\n+\t\tif (irs-\u003eidr2.istmd)\n+\t\t\tvalue |= GICV5_IRS_IDR2_ISTMD;\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_IDR2_IST_L2SZ, irs-\u003eidr2.ist_l2sz);\n+\t\tif (irs-\u003eidr2.ist_levels)\n+\t\t\tvalue |= GICV5_IRS_IDR2_IST_LEVELS;\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_IDR2_MIN_LPI_ID_BITS, irs-\u003eidr2.min_lpi_id_bits);\n+\t\tvalue |= GICV5_IRS_IDR2_LPI;\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_IDR2_ID_BITS, irs-\u003eidr2.id_bits);\n+\t\tbreak;\n+\tcase GICV5_IRS_IDR5:\n+\t\tvalue = FIELD_PREP(GICV5_IRS_IDR5_SPI_RANGE, irs-\u003eidr5.spi_range);\n+\t\tbreak;\n+\tcase GICV5_IRS_IDR6:\n+\t\tvalue = FIELD_PREP(GICV5_IRS_IDR6_SPI_IRS_RANGE, irs-\u003eidr6.spi_irs_range);\n+\t\tbreak;\n+\tcase GICV5_IRS_IDR7:\n+\t\tvalue = FIELD_PREP(GICV5_IRS_IDR7_SPI_BASE, irs-\u003eidr7.spi_base);\n+\t\tbreak;\n+\tcase GICV5_IRS_IIDR:\n+\t\t/* Revision, Variant, ProductID are implementation defined */\n+\t\tvalue = FIELD_PREP(GICV5_IRS_IIDR_PRODUCT_ID, PRODUCT_ID_KVM);\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_IIDR_VARIANT, 0);\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_IIDR_REVISION, 0);\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_IIDR_IMPLEMENTER, IMPLEMENTER_ARM);\n+\t\tbreak;\n+\tcase GICV5_IRS_AIDR:\n+\t\tvalue = FIELD_PREP(GICV5_IRS_AIDR_COMPONENT,\n+\t\t\t\t GICV5_AIDR_COMPONENT_IRS);\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_AIDR_ARCHMAJORREV,\n+\t\t\t\t GICV5_AIDR_ARCH_MAJ_REV_V5);\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_AIDR_ARCHMINORREV,\n+\t\t\t\t GICV5_AIDR_ARCH_MIN_REV_V0);\n+\t\tbreak;\n+\tcase GICV5_IRS_CR0:\n+\t\t/*\n+\t\t * The IRS is ALWAYS idle as we handle things instantaneously\n+\t\t * from a guest's viewpoint.\n+\t\t */\n+\t\tvalue = GICV5_IRS_CR0_IDLE;\n+\t\tif (READ_ONCE(vcpu-\u003ekvm-\u003earch.vgic.enabled))\n+\t\t\tvalue |= GICV5_IRS_CR0_IRSEN;\n+\t\tbreak;\n+\tcase GICV5_IRS_CR1:\n+\t\tif (irs-\u003ecr1.vped_wa)\n+\t\t\tvalue |= GICV5_IRS_CR1_VPED_WA;\n+\t\tif (irs-\u003ecr1.vped_ra)\n+\t\t\tvalue |= GICV5_IRS_CR1_VPED_RA;\n+\t\tif (irs-\u003ecr1.vmd_wa)\n+\t\t\tvalue |= GICV5_IRS_CR1_VMD_WA;\n+\t\tif (irs-\u003ecr1.vmd_ra)\n+\t\t\tvalue |= GICV5_IRS_CR1_VMD_RA;\n+\t\tif (irs-\u003ecr1.vpet_ra)\n+\t\t\tvalue |= GICV5_IRS_CR1_VPET_RA;\n+\t\tif (irs-\u003ecr1.vmt_ra)\n+\t\t\tvalue |= GICV5_IRS_CR1_VMT_RA;\n+\t\tif (irs-\u003ecr1.ist_wa)\n+\t\t\tvalue |= GICV5_IRS_CR1_IST_WA;\n+\t\tif (irs-\u003ecr1.ist_ra)\n+\t\t\tvalue |= GICV5_IRS_CR1_IST_RA;\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_CR1_IC, irs-\u003ecr1.ic);\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_CR1_OC, irs-\u003ecr1.oc);\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_CR1_SH, irs-\u003ecr1.sh);\n+\t\tbreak;\n+\tcase GICV5_IRS_SYNC_STATUSR:\n+\t\tvalue = GICV5_IRS_SYNC_STATUSR_IDLE;\n+\t\tbreak;\n+\tcase GICV5_IRS_PE_SELR:\n+\t\tvalue = FIELD_PREP(GICV5_IRS_PE_SELR_IAFFID, irs-\u003epe_selr.iaffid);\n+\t\tbreak;\n+\tcase GICV5_IRS_PE_STATUSR:\n+\t\t/* We assume that the PE is Online if present. Always IDLE too */\n+\t\tvalue = GICV5_IRS_PE_STATUSR_IDLE;\n+\n+\t\t/* Set ONLINE and V if IAFFID selects a present PE */\n+\t\tif (kvm_get_vcpu_by_id(vcpu-\u003ekvm, irs-\u003epe_selr.iaffid)) {\n+\t\t\tvalue |= GICV5_IRS_PE_STATUSR_ONLINE;\n+\t\t\tvalue |= GICV5_IRS_PE_STATUSR_V;\n+\t\t}\n+\t\tbreak;\n+\tcase GICV5_IRS_PE_CR0:\n+\t\t/*\n+\t\t * Make sure that we are doing something reasonable first.\n+\t\t * Remember, the IAFFID is the same as the VPE_ID\n+\t\t */\n+\t\ttarget_vcpu = kvm_get_vcpu_by_id(vcpu-\u003ekvm, irs-\u003epe_selr.iaffid);\n+\t\tif (!target_vcpu) {\n+\t\t\tkvm_debug(\"Guest programmed invalid IAFFID (0x%x) into the IRS_PE_SELR\\n\",\n+\t\t\t\t irs-\u003epe_selr.iaffid);\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tvalue = GICV5_IRS_PE_CR0_DPS;\n+\t\tbreak;\n+\tdefault:\n+\t\treturn 0;\n+\t}\n+\n+\treturn value;\n+}\n+\n+static void vgic_v5_mmio_write_irs_misc(struct kvm_vcpu *vcpu, gpa_t addr,\n+\t\t\t\t\tunsigned int len, unsigned long val)\n+{\n+\tstruct vgic_v5_irs *irs = vgic_v5_get_irs(vcpu);\n+\tstruct vgic_dist *vgic = vgic_v5_get_vgic(vcpu);\n+\tconst size_t offset = addr \u0026 (SZ_64K - 1);\n+\n+\tswitch (offset) {\n+\tcase GICV5_IRS_CR0:\n+\t\tmutex_lock(\u0026vcpu-\u003ekvm-\u003earch.config_lock);\n+\t\tWRITE_ONCE(vgic-\u003eenabled, !!(val \u0026 GICV5_IRS_CR0_IRSEN));\n+\t\tkvm_make_all_cpus_request(vcpu-\u003ekvm, KVM_REQ_RELOAD_GICv5);\n+\t\tmutex_unlock(\u0026vcpu-\u003ekvm-\u003earch.config_lock);\n+\n+\t\treturn;\n+\tcase GICV5_IRS_CR1:\n+\t\tirs-\u003ecr1.sh = FIELD_GET(GICV5_IRS_CR1_SH, val);\n+\t\tirs-\u003ecr1.oc = FIELD_GET(GICV5_IRS_CR1_OC, val);\n+\t\tirs-\u003ecr1.ic = FIELD_GET(GICV5_IRS_CR1_IC, val);\n+\t\tirs-\u003ecr1.ist_ra = !!(val \u0026 GICV5_IRS_CR1_IST_RA);\n+\t\tirs-\u003ecr1.ist_wa = !!(val \u0026 GICV5_IRS_CR1_IST_WA);\n+\t\tirs-\u003ecr1.vmt_ra = !!(val \u0026 GICV5_IRS_CR1_VMT_RA);\n+\t\tirs-\u003ecr1.vpet_ra = !!(val \u0026 GICV5_IRS_CR1_VPET_RA);\n+\t\tirs-\u003ecr1.vmd_ra = !!(val \u0026 GICV5_IRS_CR1_VMD_RA);\n+\t\tirs-\u003ecr1.vmd_wa = !!(val \u0026 GICV5_IRS_CR1_VMD_WA);\n+\t\tirs-\u003ecr1.vped_ra = !!(val \u0026 GICV5_IRS_CR1_VPED_RA);\n+\t\tirs-\u003ecr1.vped_wa = !!(val \u0026 GICV5_IRS_CR1_VPED_WA);\n+\t\treturn;\n+\tcase GICV5_IRS_PE_SELR:\n+\t\tirs-\u003epe_selr.iaffid = FIELD_GET(GICV5_IRS_PE_SELR_IAFFID, val);\n+\t\treturn;\n+\tcase GICV5_IRS_PE_CR0:\n+\t\t/*\n+\t\t * We actually have nothing to do here as we don't support\n+\t\t * 1-of-N routing. The only thing that the guest can correctly\n+\t\t * write here is 0x1. However, there's no way to fault if it\n+\t\t * writes something else. This is effectively a WI in our case,\n+\t\t * but we keep it here for the purposes of documenting it.\n+\t\t */\n+\t\treturn;\n+\tdefault:\n+\t\treturn;\n+\t}\n+}\n+\n+static bool vgic_v5_is_spi_selr_valid(struct vgic_v5_irs *irs, u32 id)\n+{\n+\t/* Invalid - we don't have any SPIs at all */\n+\tif (irs-\u003eidr5.spi_range == 0)\n+\t\treturn false;\n+\n+\t/* Invalid - we don't have any on this IRS */\n+\tif (irs-\u003eidr6.spi_irs_range == 0)\n+\t\treturn false;\n+\n+\t/* Invalid - ID is less than min */\n+\tif (id \u003c irs-\u003eidr7.spi_base)\n+\t\treturn false;\n+\n+\t/* Invalid - ID is greater than max */\n+\tif (id \u003e= (irs-\u003eidr7.spi_base + irs-\u003eidr6.spi_irs_range))\n+\t\treturn false;\n+\n+\treturn true;\n+}\n+\n+static unsigned long vgic_v5_mmio_read_irs_spi(struct kvm_vcpu *vcpu,\n+\t\t\t\t\t gpa_t addr, unsigned int len)\n+{\n+\tstruct vgic_v5_irs *irs = vgic_v5_get_irs(vcpu);\n+\tconst size_t offset = addr \u0026 (SZ_64K - 1);\n+\tstruct vgic_irq *irq;\n+\tu64 value = 0;\n+\n+\tswitch (offset) {\n+\tcase GICV5_IRS_SPI_SELR:\n+\t\t/* Return whatever was last written */\n+\t\tvalue = FIELD_PREP(GICV5_IRS_SPI_SELR_ID, READ_ONCE(irs-\u003espi_selr.id));\n+\t\tbreak;\n+\tcase GICV5_IRS_SPI_STATUSR:\n+\t\t/* We assume that we can always claim to be idle */\n+\t\tvalue = GICV5_IRS_SPI_STATUSR_IDLE;\n+\t\tif (vgic_v5_is_spi_selr_valid(irs, READ_ONCE(irs-\u003espi_selr.id)))\n+\t\t\tvalue |= GICV5_IRS_SPI_STATUSR_V;\n+\t\tbreak;\n+\tcase GICV5_IRS_SPI_DOMAINR:\n+\t\tvalue = FIELD_PREP(GICV5_IRS_SPI_DOMAINR_DOMAIN,\n+\t\t\t\t GICV5_IRS_SPI_DOMAINR_DOMAIN_NON_SECURE);\n+\t\tbreak;\n+\tcase GICV5_IRS_SPI_CFGR: {\n+\t\tu32 id = READ_ONCE(irs-\u003espi_selr.id);\n+\n+\t\tif (!vgic_v5_is_spi_selr_valid(irs, id)) {\n+\t\t\t/* Fault with IRS_SPI_SELR; return 0*/\n+\t\t\tvalue = 0;\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tirq = vgic_get_irq(vcpu-\u003ekvm, vgic_v5_make_spi(id));\n+\t\tif (!irq) {\n+\t\t\tkvm_debug_ratelimited(\"Guest trying to access SPI not backed by KVM\\n\");\n+\t\t\tvalue = 0;\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tscoped_guard(raw_spinlock_irqsave, \u0026irq-\u003eirq_lock) {\n+\t\t\tif (irq-\u003econfig == VGIC_CONFIG_LEVEL)\n+\t\t\t\tvalue = GICV5_IRS_SPI_CFGR_TM;\n+\t\t}\n+\n+\t\tvgic_put_irq(vcpu-\u003ekvm, irq);\n+\n+\t\tbreak;\n+\t}\n+\tdefault:\n+\t\treturn 0;\n+\t}\n+\n+\treturn value;\n+}\n+\n+static void vgic_v5_mmio_write_irs_spi(struct kvm_vcpu *vcpu, gpa_t addr,\n+\t\t\t\t unsigned int len, unsigned long val)\n+{\n+\tstruct vgic_v5_irs *irs = vgic_v5_get_irs(vcpu);\n+\tconst size_t offset = addr \u0026 (SZ_64K - 1);\n+\tstruct vgic_irq *irq;\n+\n+\tswitch (offset) {\n+\tcase GICV5_IRS_SPI_SELR:\n+\t\tWRITE_ONCE(irs-\u003espi_selr.id,\n+\t\t\t FIELD_GET(GICV5_IRS_SPI_SELR_ID, val));\n+\t\treturn;\n+\tcase GICV5_IRS_SPI_CFGR: {\n+\t\tu32 id = READ_ONCE(irs-\u003espi_selr.id);\n+\n+\t\tif (!vgic_v5_is_spi_selr_valid(irs, id))\n+\t\t\treturn;\n+\n+\t\t/*\n+\t\t * Find KVM's representation of the interrupt - we need to make\n+\t\t * sure that KVM's view agrees with the guest's, else interrupt\n+\t\t * injection won't work properly for level-triggered interrupts\n+\t\t * (we fail to handle the clearing of the pending state if KVM\n+\t\t * thinks that the interrupt is edge-triggered, which is the\n+\t\t * default.)\n+\t\t */\n+\t\tirq = vgic_get_irq(vcpu-\u003ekvm, vgic_v5_make_spi(id));\n+\t\tif (!irq)\n+\t\t\treturn;\n+\n+\t\tscoped_guard(raw_spinlock_irqsave, \u0026irq-\u003eirq_lock) {\n+\t\t\tif (val \u0026 GICV5_IRS_SPI_CFGR_TM)\n+\t\t\t\tirq-\u003econfig = VGIC_CONFIG_LEVEL;\n+\t\t\telse\n+\t\t\t\tirq-\u003econfig = VGIC_CONFIG_EDGE;\n+\t\t}\n+\n+\t\tvgic_put_irq(vcpu-\u003ekvm, irq);\n+\n+\t\treturn;\n+\t}\n+\tdefault:\n+\t\treturn;\n+\t}\n+}\n+\n+static bool vgic_v5_ist_cfgr_valid(struct vgic_v5_irs *irs)\n+{\n+\tunsigned int expected_istsz;\n+\n+\tif (irs-\u003eist_cfgr.lpi_id_bits \u003c irs-\u003eidr2.min_lpi_id_bits ||\n+\t irs-\u003eist_cfgr.lpi_id_bits \u003e irs-\u003eidr2.id_bits)\n+\t\treturn false;\n+\n+\tif (!irs-\u003eidr2.istmd)\n+\t\texpected_istsz = GICV5_IRS_IST_CFGR_ISTSZ_4;\n+\telse if (irs-\u003eist_cfgr.lpi_id_bits \u003e= irs-\u003eidr2.istmd_sz)\n+\t\texpected_istsz = GICV5_IRS_IST_CFGR_ISTSZ_16;\n+\telse\n+\t\texpected_istsz = GICV5_IRS_IST_CFGR_ISTSZ_8;\n+\n+\tif (irs-\u003eist_cfgr.istsz != expected_istsz)\n+\t\treturn false;\n+\n+\tif (irs-\u003eist_cfgr.structure \u0026\u0026 !irs-\u003eidr2.ist_levels)\n+\t\treturn false;\n+\n+\tif (!irs-\u003eist_cfgr.structure)\n+\t\treturn true;\n+\n+\treturn irs-\u003eist_cfgr.l2sz == irs-\u003eidr2.ist_l2sz;\n+}\n+\n+static unsigned long vgic_v5_mmio_read_irs_ist(struct kvm_vcpu *vcpu,\n+\t\t\t\t\t gpa_t addr, unsigned int len)\n+{\n+\tstruct vgic_v5_irs *irs = vgic_v5_get_irs(vcpu);\n+\tconst size_t offset = addr \u0026 (SZ_64K - 1);\n+\tu64 value = 0;\n+\n+\tswitch (offset) {\n+\tcase GICV5_IRS_IST_STATUSR:\n+\t\treturn GICV5_IRS_IST_STATUSR_IDLE;\n+\tcase GICV5_IRS_IST_CFGR:\n+\t\tif (irs-\u003eist_cfgr.structure)\n+\t\t\tvalue |= GICV5_IRS_IST_CFGR_STRUCTURE;\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_IST_CFGR_ISTSZ, irs-\u003eist_cfgr.istsz);\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_IST_CFGR_L2SZ, irs-\u003eist_cfgr.l2sz);\n+\t\tvalue |= FIELD_PREP(GICV5_IRS_IST_CFGR_LPI_ID_BITS, irs-\u003eist_cfgr.lpi_id_bits);\n+\t\tbreak;\n+\tcase GICV5_IRS_IST_BASER:\n+\t\tvalue = FIELD_PREP(GICV5_IRS_IST_BASER_ADDR_MASK,\n+\t\t\t\t irs-\u003eist_baser.addr \u003e\u003e GICV5_IRS_IST_BASER_ADDR_SHIFT);\n+\t\tif (irs-\u003eist_baser.valid)\n+\t\t\tvalue |= GICV5_IRS_IST_BASER_VALID;\n+\t\tbreak;\n+\tdefault:\n+\t\treturn 0;\n+\t}\n+\n+\treturn value;\n+}\n+\n+static void vgic_v5_update_irs_ist_baser(struct vgic_v5_irs *irs,\n+\t\t\t\t\t unsigned long val)\n+{\n+\tirs-\u003eist_baser.valid = !!(val \u0026 GICV5_IRS_IST_BASER_VALID);\n+\tirs-\u003eist_baser.addr = FIELD_GET(GICV5_IRS_IST_BASER_ADDR_MASK, val)\n+\t\t\u003c\u003c GICV5_IRS_IST_BASER_ADDR_SHIFT;\n+}\n+\n+static int vgic_v5_write_irs_ist_baser(struct kvm_vcpu *vcpu, unsigned long val)\n+{\n+\tstruct vgic_v5_irs *irs = vgic_v5_get_irs(vcpu);\n+\tenum gicv5_vcpu_cmd cmd = LPI_VIST_MAKE_INVALID;\n+\tbool valid = !!(val \u0026 GICV5_IRS_IST_BASER_VALID);\n+\tint rc;\n+\n+\t/* Valid -\u003e Invalid */\n+\tif (irs-\u003eist_baser.valid \u0026\u0026 !valid) {\n+\t\t/* Make the LPI IST invalid and then ... */\n+\t\trc = irq_set_vcpu_affinity(vgic_v5_vpe_db(vcpu), \u0026cmd);\n+\t\tif (rc)\n+\t\t\treturn rc;\n+\n+\t\tvgic_v5_update_irs_ist_baser(irs, val);\n+\n+\t\t/*\n+\t\t * ... reflect that in the emulated BASER before freeing the\n+\t\t * host IST. If the free fails, the guest-visible valid bit\n+\t\t * still matches the hardware state.\n+\t\t */\n+\t\treturn vgic_v5_lpi_ist_free(vcpu-\u003ekvm);\n+\t} else if (!irs-\u003eist_baser.valid \u0026\u0026 valid) { /* Invalid -\u003e Valid */\n+\t\tif (!vgic_v5_ist_cfgr_valid(irs)) {\n+\t\t\tpr_debug_ratelimited(\"Guest programmed invalid IRS_IST_CFGR\\n\");\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\n+\t\trc = vgic_v5_lpi_ist_alloc(vcpu-\u003ekvm, irs-\u003eist_cfgr.lpi_id_bits);\n+\t\tif (rc)\n+\t\t\treturn rc;\n+\t}\n+\n+\t/* Now that we've handled the edges, update the valid bit and addr */\n+\tvgic_v5_update_irs_ist_baser(irs, val);\n+\n+\treturn 0;\n+}\n+\n+static void vgic_v5_mmio_write_irs_ist(struct kvm_vcpu *vcpu, gpa_t addr,\n+\t\t\t\t unsigned int len, unsigned long val)\n+{\n+\tstruct vgic_v5_irs *irs = vgic_v5_get_irs(vcpu);\n+\tconst size_t offset = addr \u0026 (SZ_64K - 1);\n+\n+\tswitch (offset) {\n+\tcase GICV5_IRS_IST_CFGR:\n+\t\tscoped_guard(mutex, \u0026vcpu-\u003ekvm-\u003earch.config_lock) {\n+\t\t\tif (irs-\u003eist_baser.valid)\n+\t\t\t\treturn;\n+\n+\t\t\tirs-\u003eist_cfgr.lpi_id_bits = FIELD_GET(GICV5_IRS_IST_CFGR_LPI_ID_BITS, val);\n+\t\t\tirs-\u003eist_cfgr.l2sz = FIELD_GET(GICV5_IRS_IST_CFGR_L2SZ, val);\n+\t\t\tirs-\u003eist_cfgr.istsz = FIELD_GET(GICV5_IRS_IST_CFGR_ISTSZ, val);\n+\t\t\tirs-\u003eist_cfgr.structure = !!(val \u0026 GICV5_IRS_IST_CFGR_STRUCTURE);\n+\t\t}\n+\t\treturn;\n+\tcase GICV5_IRS_IST_BASER: {\n+\t\tguard(mutex)(\u0026vcpu-\u003ekvm-\u003earch.config_lock);\n+\t\tvgic_v5_write_irs_ist_baser(vcpu, val);\n+\t\treturn;\n+\t}\n+\tdefault:\n+\t\treturn;\n+\t}\n+}\n+\n+static unsigned long vgic_v5_coresight_read(struct kvm_vcpu *vcpu,\n+\t\t\t\t\t gpa_t addr, unsigned int len)\n+{\n+\tconst size_t offset = addr \u0026 (SZ_64K - 1);\n+\n+\tswitch (offset) {\n+\tcase GICV5_CORESIGHT_DEVARCH:\n+\t\treturn GICV5_CORESIGHT_DEVARCH_VAL;\n+\tcase GICV5_CORESIGHT_PIDR4:\n+\t\treturn GICV5_CORESIGHT_PIDR4_JEP106_CONT;\n+\tcase GICV5_CORESIGHT_PIDR5:\n+\t\treturn GICV5_CORESIGHT_PIDR5_RES0;\n+\tcase GICV5_CORESIGHT_PIDR6:\n+\t\treturn GICV5_CORESIGHT_PIDR6_RES0;\n+\tcase GICV5_CORESIGHT_PIDR7:\n+\t\treturn GICV5_CORESIGHT_PIDR7_RES0;\n+\tcase GICV5_CORESIGHT_PIDR0:\n+\t\treturn GICV5_CORESIGHT_PIDR0_PART_0;\n+\tcase GICV5_CORESIGHT_PIDR1:\n+\t\treturn GICV5_CORESIGHT_PIDR1_DES_0_PART_1;\n+\tcase GICV5_CORESIGHT_PIDR2:\n+\t\treturn GICV5_CORESIGHT_PIDR2_DES_1;\n+\tcase GICV5_CORESIGHT_PIDR3:\n+\t\treturn GICV5_CORESIGHT_PIDR3_REVAND_CMOD;\n+\tcase GICV5_CORESIGHT_CIDR0:\n+\t\treturn GICV5_CORESIGHT_CIDR0_VAL;\n+\tcase GICV5_CORESIGHT_CIDR1:\n+\t\treturn GICV5_CORESIGHT_CIDR1_VAL;\n+\tcase GICV5_CORESIGHT_CIDR2:\n+\t\treturn GICV5_CORESIGHT_CIDR2_VAL;\n+\tcase GICV5_CORESIGHT_CIDR3:\n+\t\treturn GICV5_CORESIGHT_CIDR3_VAL;\n+\tdefault:\n+\t\treturn 0;\n+\t}\n+}\n+\n+static unsigned long vgic_v5_mmio_uaccess_read_irs_status(struct kvm_vcpu *vcpu,\n+\t\t\t\t\t\t\t gpa_t addr,\n+\t\t\t\t\t\t\t unsigned int len)\n+{\n+\tconst size_t offset = addr \u0026 (SZ_64K - 1);\n+\n+\tswitch (offset) {\n+\tcase GICV5_IRS_SYNC_STATUSR:\n+\t\treturn GICV5_IRS_SYNC_STATUSR_IDLE;\n+\tcase GICV5_IRS_SPI_STATUSR:\n+\t\treturn GICV5_IRS_SPI_STATUSR_IDLE;\n+\tcase GICV5_IRS_PE_STATUSR:\n+\t\treturn GICV5_IRS_PE_STATUSR_IDLE;\n+\tcase GICV5_IRS_IST_STATUSR:\n+\t\treturn GICV5_IRS_IST_STATUSR_IDLE;\n+\tdefault:\n+\t\treturn 0;\n+\t}\n+}\n+\n+static int vgic_v5_mmio_uaccess_write_irs(struct kvm_vcpu *vcpu, gpa_t addr,\n+\t\t\t\t\t unsigned int len, unsigned long val)\n+{\n+\tstruct vgic_dist *vgic = \u0026vcpu-\u003ekvm-\u003earch.vgic;\n+\tstruct vgic_v5_irs *irs_data = vgic-\u003evgic_v5_irs_data;\n+\tsize_t offset = addr \u0026 (SZ_64K - 1);\n+\n+\t/*\n+\t * The following registers are ONLY settable via uaccesses. The guest\n+\t * cannot write them!\n+\t */\n+\n+\tswitch (offset) {\n+\tcase GICV5_IRS_IDR0:\n+\t\tif (FIELD_GET(GICV5_IRS_IDR0_INT_DOM, val) !=\n+\t\t GICV5_IRS_IDR0_INT_DOM_NON_SECURE)\n+\t\t\treturn -EINVAL;\n+\n+\t\tif ((val \u0026 GICV5_IRS_IDR0_VIRT) ||\n+\t\t (val \u0026 GICV5_IRS_IDR0_ONE_N) ||\n+\t\t (val \u0026 GICV5_IRS_IDR0_VIRT_ONE_N) ||\n+\t\t (val \u0026 GICV5_IRS_IDR0_SETLPI) ||\n+\t\t (val \u0026 GICV5_IRS_IDR0_MEC) ||\n+\t\t (val \u0026 GICV5_IRS_IDR0_MPAM) ||\n+\t\t (val \u0026 GICV5_IRS_IDR0_SWE))\n+\t\t\treturn -EINVAL;\n+\n+\t\tirs_data-\u003eidr0.domain = FIELD_GET(GICV5_IRS_IDR0_INT_DOM, val);\n+\t\tirs_data-\u003eidr0.pa_range = FIELD_GET(GICV5_IRS_IDR0_PA_RANGE, val);\n+\t\tirs_data-\u003eidr0.virt = !!(val \u0026 GICV5_IRS_IDR0_VIRT);\n+\t\tirs_data-\u003eidr0.setlpi = !!(val \u0026 GICV5_IRS_IDR0_SETLPI);\n+\t\tirs_data-\u003eidr0.mec = !!(val \u0026 GICV5_IRS_IDR0_MEC);\n+\t\tirs_data-\u003eidr0.mpam = !!(val \u0026 GICV5_IRS_IDR0_MPAM);\n+\t\tirs_data-\u003eidr0.swe = !!(val \u0026 GICV5_IRS_IDR0_SWE);\n+\t\tirs_data-\u003eidr0.irs_id = FIELD_GET(GICV5_IRS_IDR0_IRSID, val);\n+\t\tbreak;\n+\tcase GICV5_IRS_IDR1: {\n+\t\tunsigned int iaffid_bits, priority_bits;\n+\t\tu8 vpe_id_bits;\n+\n+\t\t/* Ignore writes to PE_CNT as this is populated from num vcpus */\n+\t\tiaffid_bits = FIELD_GET(GICV5_IRS_IDR1_IAFFID_BITS, val);\n+\t\tpriority_bits = FIELD_GET(GICV5_IRS_IDR1_PRIORITY_BITS, val);\n+\n+\t\t/* IAFFID_BITS is derived from the VMTE and encoded as N - 1. */\n+\t\tvpe_id_bits = vgic_v5_vmte_vpe_id_bits(vcpu);\n+\t\tif (iaffid_bits != vpe_id_bits - 1)\n+\t\t\treturn -EINVAL;\n+\n+\t\tif (priority_bits \u003e gicv5_global_data.irs_pri_bits - 1)\n+\t\t\treturn -EINVAL;\n+\n+\t\tirs_data-\u003eidr1.priority_bits = priority_bits;\n+\t\tbreak;\n+\t}\n+\tcase GICV5_IRS_IDR2:\n+\t\t/* We always support LPIs */\n+\t\tif (!(val \u0026 GICV5_IRS_IDR2_LPI))\n+\t\t\treturn -EINVAL;\n+\n+\t\t/* We only support LPIs with linear, non-metadata guest ISTs */\n+\t\tif (val \u0026 GICV5_IRS_IDR2_IST_LEVELS)\n+\t\t\treturn -EINVAL;\n+\n+\t\tif ((val \u0026 GICV5_IRS_IDR2_ISTMD) ||\n+\t\t FIELD_GET(GICV5_IRS_IDR2_ISTMD_SZ, val))\n+\t\t\treturn -EINVAL;\n+\n+\t\t/* We can't present more bits than we have support for in HW */\n+\t\tif (FIELD_GET(GICV5_IRS_IDR2_ID_BITS, val) \u003e irs_caps.ist_id_bits)\n+\t\t\treturn -EINVAL;\n+\n+\t\t/* Min LPI ID bits must be greater than or equal to the HW */\n+\t\tif (FIELD_GET(GICV5_IRS_IDR2_MIN_LPI_ID_BITS, val) \u003c\n+\t\t irs_caps.min_lpi_id_bits)\n+\t\t\treturn -EINVAL;\n+\n+\t\tif (FIELD_GET(GICV5_IRS_IDR2_MIN_LPI_ID_BITS, val) \u003e\n+\t\t FIELD_GET(GICV5_IRS_IDR2_ID_BITS, val))\n+\t\t\treturn -EINVAL;\n+\n+\t\tirs_data-\u003eidr2.istmd_sz = FIELD_GET(GICV5_IRS_IDR2_ISTMD_SZ, val);\n+\t\tirs_data-\u003eidr2.istmd = !!(val \u0026 GICV5_IRS_IDR2_ISTMD);\n+\t\tirs_data-\u003eidr2.ist_l2sz = FIELD_GET(GICV5_IRS_IDR2_IST_L2SZ, val);\n+\t\tirs_data-\u003eidr2.ist_levels = !!(val \u0026 GICV5_IRS_IDR2_IST_LEVELS);\n+\t\tirs_data-\u003eidr2.min_lpi_id_bits = FIELD_GET(GICV5_IRS_IDR2_MIN_LPI_ID_BITS, val);\n+\t\tirs_data-\u003eidr2.id_bits = FIELD_GET(GICV5_IRS_IDR2_ID_BITS, val);\n+\t\tbreak;\n+\tcase GICV5_IRS_IDR5:\n+\t\tif (FIELD_GET(GICV5_IRS_IDR5_SPI_RANGE, val) != irs_data-\u003eidr5.spi_range)\n+\t\t\treturn -EINVAL;\n+\t\tbreak;\n+\tcase GICV5_IRS_IDR6:\n+\t\tif (FIELD_GET(GICV5_IRS_IDR6_SPI_IRS_RANGE, val) != irs_data-\u003eidr6.spi_irs_range)\n+\t\t\treturn -EINVAL;\n+\t\tbreak;\n+\tcase GICV5_IRS_IDR7:\n+\t\tif (FIELD_GET(GICV5_IRS_IDR7_SPI_BASE, val) != irs_data-\u003eidr7.spi_base)\n+\t\t\treturn -EINVAL;\n+\t\tbreak;\n+\tcase GICV5_IRS_IST_BASER:\n+\t\tvgic_v5_update_irs_ist_baser(irs_data, val);\n+\t\tbreak;\n+\tcase GICV5_IRS_IST_CFGR:\n+\t\tirs_data-\u003eist_cfgr.lpi_id_bits = FIELD_GET(GICV5_IRS_IST_CFGR_LPI_ID_BITS, val);\n+\t\tirs_data-\u003eist_cfgr.l2sz = FIELD_GET(GICV5_IRS_IST_CFGR_L2SZ, val);\n+\t\tirs_data-\u003eist_cfgr.istsz = FIELD_GET(GICV5_IRS_IST_CFGR_ISTSZ, val);\n+\t\tirs_data-\u003eist_cfgr.structure = !!(val \u0026 GICV5_IRS_IST_CFGR_STRUCTURE);\n+\t\tbreak;\n+\tcase GICV5_IRS_CR0:\n+\t\tvgic-\u003eenabled = !!(val \u0026 GICV5_IRS_CR0_IRSEN);\n+\t\tbreak;\n+\tcase GICV5_IRS_SPI_CFGR:\n+\t\tbreak;\n+\tcase GICV5_IRS_IIDR:\n+\t\tfallthrough;\n+\tcase GICV5_IRS_AIDR:\n+\t\tbreak;\n+\tdefault:\n+\t\treturn -EINVAL;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static const struct vgic_register_region vgic_v5_irs_registers[] = {\n+\t/*\n+\t * This is the IRS_CONFIG_FRAME.\n+\t */\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IDR0, vgic_v5_mmio_read_irs_misc,\n+\t\t\t\t\t vgic_mmio_write_wi, NULL,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_write_irs, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IDR1, vgic_v5_mmio_read_irs_misc,\n+\t\t\t\t\t vgic_mmio_write_wi, NULL,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_write_irs, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IDR2, vgic_v5_mmio_read_irs_misc,\n+\t\t\t\t\t vgic_mmio_write_wi, NULL,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_write_irs, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_IDR3, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_IDR4, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IDR5, vgic_v5_mmio_read_irs_misc,\n+\t\t\t\t\t vgic_mmio_write_wi, NULL,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_write_irs, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IDR6, vgic_v5_mmio_read_irs_misc,\n+\t\t\t\t\t vgic_mmio_write_wi, NULL,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_write_irs, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IDR7, vgic_v5_mmio_read_irs_misc,\n+\t\t\t\t\t vgic_mmio_write_wi, NULL,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_write_irs, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IIDR, vgic_v5_mmio_read_irs_misc,\n+\t\t\t\t\t vgic_mmio_write_wi, NULL,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_write_irs, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_AIDR, vgic_v5_mmio_read_irs_misc,\n+\t\t\t\t\t vgic_mmio_write_wi, NULL,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_write_irs, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_CR0,\n+\t\t\t\t\t vgic_v5_mmio_read_irs_misc,\n+\t\t\t\t\t vgic_v5_mmio_write_irs_misc, NULL,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_write_irs, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_CR1, vgic_v5_mmio_read_irs_misc,\n+\t\t\t\t vgic_v5_mmio_write_irs_misc, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_SYNCR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_SYNC_STATUSR,\n+\t\t\t\t\t vgic_v5_mmio_read_irs_misc,\n+\t\t\t\t\t vgic_mmio_write_wi,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_read_irs_status,\n+\t\t\t\t\t vgic_mmio_uaccess_write_wi, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_SPI_VMR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 8,\n+\t\t\t\t VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_SPI_SELR, vgic_v5_mmio_read_irs_spi,\n+\t\t\t\t vgic_v5_mmio_write_irs_spi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_SPI_DOMAINR, vgic_v5_mmio_read_irs_spi,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_SPI_RESAMPLER, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_SPI_CFGR,\n+\t\t\t\t\t vgic_v5_mmio_read_irs_spi,\n+\t\t\t\t\t vgic_v5_mmio_write_irs_spi, NULL,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_write_irs, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_SPI_STATUSR,\n+\t\t\t\t\t vgic_v5_mmio_read_irs_spi,\n+\t\t\t\t\t vgic_mmio_write_wi,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_read_irs_status,\n+\t\t\t\t\t vgic_mmio_uaccess_write_wi, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_PE_SELR,\n+\t\t\t\t vgic_v5_mmio_read_irs_misc,\n+\t\t\t\t vgic_v5_mmio_write_irs_misc, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_PE_STATUSR,\n+\t\t\t\t\t vgic_v5_mmio_read_irs_misc,\n+\t\t\t\t\t vgic_mmio_write_wi,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_read_irs_status,\n+\t\t\t\t\t vgic_mmio_uaccess_write_wi, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_PE_CR0, vgic_v5_mmio_read_irs_misc,\n+\t\t\t\t vgic_v5_mmio_write_irs_misc, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IST_BASER,\n+\t\t\t\t\t vgic_v5_mmio_read_irs_ist,\n+\t\t\t\t\t vgic_v5_mmio_write_irs_ist, NULL,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_write_irs, 8,\n+\t\t\t\t\t VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IST_CFGR,\n+\t\t\t\t\t vgic_v5_mmio_read_irs_ist,\n+\t\t\t\t\t vgic_v5_mmio_write_irs_ist, NULL,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_write_irs, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IST_STATUSR,\n+\t\t\t\t\t vgic_v5_mmio_read_irs_ist,\n+\t\t\t\t\t vgic_mmio_write_wi,\n+\t\t\t\t\t vgic_v5_mmio_uaccess_read_irs_status,\n+\t\t\t\t\t vgic_mmio_uaccess_write_wi, 4,\n+\t\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_MAP_L2_ISTR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\n+\t/*\n+\t * The following registers are only for running VMs. They are not yet\n+\t * supported as we don't currently support nested, so expose them as\n+\t * read-as-zero/write-ignored.\n+\t */\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMT_BASER, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMT_CFGR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMT_STATUSR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VPE_SELR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VPE_DBR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VPE_HPPIR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VPE_CR0, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VPE_STATUSR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VM_DBR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VM_SELR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VM_STATUSR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMAP_L2_VMTR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMAP_VMR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMAP_VISTR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMAP_L2_VISTR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMAP_VPER, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_SAVE_VMR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_SAVE_VM_STATUSR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\n+\t/* MEC, MPAM, SWERR - all unimplemented */\n+\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_MEC_IDR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_MEC_MECID_R, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_MPAM_IDR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_MPAM_PARTID_R, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_SWERR_STATUSR, vgic_mmio_read_raz,\n+\t\t\t\t vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_SWERR_SYNDROMER0,\n+\t\t\t\t vgic_mmio_read_raz, vgic_mmio_write_wi, 8,\n+\t\t\t\t VGIC_ACCESS_64bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_IRS_SWERR_SYNDROMER1,\n+\t\t\t\t vgic_mmio_read_raz, vgic_mmio_write_wi, 8,\n+\t\t\t\t VGIC_ACCESS_64bit),\n+\n+\t/* CoreSight identification registers */\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_DEVARCH,\n+\t\t\t\t vgic_v5_coresight_read, vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR4,\n+\t\t\t\t vgic_v5_coresight_read, vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR5,\n+\t\t\t\t vgic_v5_coresight_read, vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR6,\n+\t\t\t\t vgic_v5_coresight_read, vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR7,\n+\t\t\t\t vgic_v5_coresight_read, vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR0,\n+\t\t\t\t vgic_v5_coresight_read, vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR1,\n+\t\t\t\t vgic_v5_coresight_read, vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR2,\n+\t\t\t\t vgic_v5_coresight_read, vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR3,\n+\t\t\t\t vgic_v5_coresight_read, vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR0,\n+\t\t\t\t vgic_v5_coresight_read, vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR1,\n+\t\t\t\t vgic_v5_coresight_read, vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR2,\n+\t\t\t\t vgic_v5_coresight_read, vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+\tREGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR3,\n+\t\t\t\t vgic_v5_coresight_read, vgic_mmio_write_wi, 4,\n+\t\t\t\t VGIC_ACCESS_32bit),\n+};\n+\n+unsigned int vgic_v5_init_irs_iodev(struct vgic_io_device *dev)\n+{\n+\tdev-\u003eregions = vgic_v5_irs_registers;\n+\tdev-\u003enr_regions = ARRAY_SIZE(vgic_v5_irs_registers);\n+\n+\tkvm_iodevice_init(\u0026dev-\u003edev, \u0026kvm_io_gic_ops);\n+\n+\t/* We represent both of the IRS frames back to back, so this is 128K */\n+\treturn KVM_VGIC_V5_IRS_SIZE;\n+}\n+\n+int vgic_v5_register_irs_iodev(struct kvm *kvm, gpa_t irs_base_address)\n+{\n+\tstruct vgic_io_device *io_device = \u0026kvm-\u003earch.vgic.vgic_v5_irs_data-\u003eiodev;\n+\tunsigned int len;\n+\n+\t/*\n+\t * Design choice: Force MMIO region to be 64k aligned. Simplifies\n+\t * pulling out registers.\n+\t */\n+\tif (!IS_ALIGNED(irs_base_address, SZ_64K))\n+\t\treturn -EINVAL;\n+\n+\tlen = vgic_v5_init_irs_iodev(io_device);\n+\n+\tio_device-\u003ebase_addr = irs_base_address;\n+\tio_device-\u003eiodev_type = IODEV_GICV5_IRS;\n+\tio_device-\u003eredist_vcpu = NULL;\n+\n+\treturn kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, irs_base_address, len,\n+\t\t\t\t \u0026io_device-\u003edev);\n+}\n+\n+/**\n+ * kvm_vgic_v5_irs_init: initialize the IRS data structures\n+ * @kvm: kvm struct pointer\n+ * @nr_spis: number of spis, frozen by caller\n+ */\n+int kvm_vgic_v5_irs_init(struct kvm *kvm, unsigned int nr_spis)\n+{\n+\tstruct vgic_dist *dist = \u0026kvm-\u003earch.vgic;\n+\tstruct vgic_v5_irs *irs = dist-\u003evgic_v5_irs_data;\n+\tstruct kvm_vcpu *vcpu0 = kvm_get_vcpu(kvm, 0);\n+\tsize_t nr_spi_bits;\n+\tu64 mmfr0;\n+\tint ret, i;\n+\n+\t/*\n+\t * We (KVM) allocate an Interrupt State Table (IST) for SPIs. The\n+\t * hardware mandates that lower 6 bits of the address are 0. Each ISTE\n+\t * is 4 bytes in size (or larger if metadata storage is required), so 16\n+\t * entries would be enough for alignment. Keep the minimum at 32 SPIs to\n+\t * match KVM's vGICv3 minimum and the VGICv5 device API.\n+\t */\n+\tif (nr_spis \u0026\u0026 nr_spis \u003c VGIC_V5_DEFAULT_NR_SPIS)\n+\t\tnr_spis = VGIC_V5_DEFAULT_NR_SPIS;\n+\n+\tif (nr_spis) {\n+\t\tdist-\u003espis = kcalloc(nr_spis, sizeof(struct vgic_irq),\n+\t\t\t\t GFP_KERNEL_ACCOUNT);\n+\t\tif (!dist-\u003espis)\n+\t\t\treturn -ENOMEM;\n+\n+\t\t/*\n+\t\t * In the following code we do not take the irq struct lock since\n+\t\t * no other action on irq structs can happen while the VGIC is\n+\t\t * not initialized yet.\n+\t\t */\n+\t\tfor (i = 0; i \u003c nr_spis; i++) {\n+\t\t\tstruct vgic_irq *irq = \u0026dist-\u003espis[i];\n+\n+\t\t\tirq-\u003eintid = vgic_v5_make_spi(i);\n+\t\t\tINIT_LIST_HEAD(\u0026irq-\u003eap_list);\n+\t\t\traw_spin_lock_init(\u0026irq-\u003eirq_lock);\n+\t\t\tirq-\u003evcpu = NULL;\n+\t\t\tirq-\u003etarget_vcpu = vcpu0;\n+\t\t\trefcount_set(\u0026irq-\u003erefcount, 0);\n+\t\t\t/*\n+\t\t\t * The guest controls the enable state, and again it is\n+\t\t\t * directly handled by the hardware. From our point of\n+\t\t\t * view it is always enabled.\n+\t\t\t */\n+\t\t\tirq-\u003eenabled = 1;\n+\t\t\tvgic_v5_set_spi_ops(irq);\n+\t\t}\n+\n+\t\tnr_spi_bits = fls(roundup_pow_of_two(nr_spis)) - 1;\n+\n+\t\tret = vgic_v5_spi_ist_alloc(kvm, nr_spi_bits);\n+\t\tif (ret) {\n+\t\t\tkfree(dist-\u003espis);\n+\t\t\tdist-\u003espis = NULL;\n+\t\t\treturn ret;\n+\t\t}\n+\t}\n+\n+\t/* Set sane initial state for the IRS MMIO registers */\n+\n+\tirs-\u003eidr0.domain = GICV5_IRS_IDR0_INT_DOM_NON_SECURE;\n+\n+\tmmfr0 = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1);\n+\tirs-\u003eidr0.pa_range = cpuid_feature_extract_unsigned_field(mmfr0,\n+\t\t\t\t\t\t\t\t ID_AA64MMFR0_EL1_PARANGE_SHIFT);\n+\n+\tirs-\u003eidr0.virt = 0;\n+\tirs-\u003eidr0.setlpi = 0;\n+\tirs-\u003eidr0.mec = 0;\n+\tirs-\u003eidr0.mpam = 0;\n+\tirs-\u003eidr0.swe = 0;\n+\tirs-\u003eidr0.irs_id = 0;\n+\n+\tirs-\u003eidr1.priority_bits = gicv5_global_data.irs_pri_bits - 1;\n+\n+\t/*\n+\t * Support 16-bits of ID space for the IRS. This should be sufficient\n+\t * for most applications, and the CPUIF is guaranteed to have at least\n+\t * 16-bits of ID space support (we actually present 16-bits there, even\n+\t * if the hardware supports more). Warn if the hardware doesn't support\n+\t * 16 bits, and use the smaller value. YMMV!\n+\t *\n+\t * As for the minimum number of ID bits, we match the hardware's\n+\t * capability.\n+\t */\n+\tif (irs_caps.ist_id_bits \u003c 16)\n+\t\tpr_warn(\"Host IRS supports fewer than 16 ID bits for ISTs (%u)\\n\",\n+\t\t\tirs_caps.ist_id_bits);\n+\n+\tirs-\u003eidr2.id_bits = min(16, irs_caps.ist_id_bits);\n+\tirs-\u003eidr2.min_lpi_id_bits = irs_caps.min_lpi_id_bits;\n+\n+\t/* Only allow the guest to create Linear ISTs - simplifies Save/Restore */\n+\tirs-\u003eidr2.ist_levels = 0;\n+\tirs-\u003eidr2.ist_l2sz = GICV5_IRS_IST_CFGR_L2SZ_4K;\n+\tirs-\u003eidr2.istmd = 0;\n+\tirs-\u003eidr2.istmd_sz = 0;\n+\n+\t/* We have a single IRS, only. All SPIs reside here! */\n+\tirs-\u003eidr5.spi_range = nr_spis;\n+\tirs-\u003eidr6.spi_irs_range = nr_spis;\n+\tirs-\u003eidr7.spi_base = 0;\n+\n+\tirs-\u003ecr1.sh = 0;\n+\tirs-\u003ecr1.oc = 0;\n+\tirs-\u003ecr1.ic = 0;\n+\tirs-\u003ecr1.ist_ra = 0;\n+\tirs-\u003ecr1.ist_wa = 0;\n+\tirs-\u003ecr1.vmt_ra = 0;\n+\tirs-\u003ecr1.vpet_ra = 0;\n+\tirs-\u003ecr1.vmd_ra = 0;\n+\tirs-\u003ecr1.vmd_wa = 0;\n+\tirs-\u003ecr1.vped_ra = 0;\n+\tirs-\u003ecr1.vped_wa = 0;\n+\n+\tirs-\u003espi_selr.id = -1;\n+\n+\tirs-\u003epe_selr.iaffid = -1;\n+\n+\tirs-\u003eist_cfgr.lpi_id_bits = 0;\n+\tirs-\u003eist_cfgr.l2sz = 0;\n+\tirs-\u003eist_cfgr.istsz = 0;\n+\tirs-\u003eist_cfgr.structure = 0;\n+\n+\tirs-\u003eist_baser.valid = 0;\n+\tirs-\u003eist_baser.addr = 0;\n+\n+\treturn 0;\n+}\n+\n+int vgic_v5_irs_lpi_ist_id_bits(struct kvm *kvm, unsigned int *id_bits)\n+{\n+\tstruct vgic_v5_irs *irs = kvm-\u003earch.vgic.vgic_v5_irs_data;\n+\n+\tif (!irs)\n+\t\treturn -ENXIO;\n+\n+\tif (!irs-\u003eist_baser.valid)\n+\t\treturn 0;\n+\n+\tif (!vgic_v5_ist_cfgr_valid(irs)) {\n+\t\tkvm_err(\"Guest programmed invalid IRS_IST_CFGR\\n\");\n+\t\treturn -EINVAL;\n+\t}\n+\n+\t*id_bits = irs-\u003eist_cfgr.lpi_id_bits;\n+\n+\treturn 1;\n+}\n+\n+int vgic_v5_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)\n+{\n+\tconst struct vgic_register_region *region;\n+\tstruct vgic_reg_attr reg_attr;\n+\tstruct kvm_vcpu *vcpu;\n+\tgpa_t addr, offset;\n+\tint ret, align;\n+\n+\tret = vgic_v5_parse_attr(dev, attr, \u0026reg_attr);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tvcpu = reg_attr.vcpu;\n+\taddr = reg_attr.addr;\n+\n+\tif (attr-\u003egroup == KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS)\n+\t\treturn vgic_v5_has_cpu_sysregs_attr(vcpu, attr);\n+\n+\toffset = attr-\u003eattr;\n+\n+\tregion = vgic_find_mmio_region(vgic_v5_irs_registers,\n+\t\t\t\t ARRAY_SIZE(vgic_v5_irs_registers),\n+\t\t\t\t offset);\n+\tif (!region)\n+\t\treturn -ENXIO;\n+\n+\talign = region-\u003eaccess_flags \u0026 VGIC_ACCESS_64bit ? 0x7 : 0x3;\n+\tif (offset \u0026 align)\n+\t\treturn -EINVAL;\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Access the IRS MMIO Regs. Relevant locks have been taken by the calling code.\n+ */\n+int vgic_v5_irs_attr_regs_access(struct kvm_device *dev,\n+\t\t\t\t struct kvm_device_attr *attr,\n+\t\t\t\t u64 *reg, bool is_write)\n+{\n+\tconst struct vgic_register_region *region;\n+\tgpa_t addr, offset;\n+\tunsigned int len;\n+\tint align, ret = 0;\n+\n+\toffset = attr-\u003eattr;\n+\n+\tif (IS_VGIC_ADDR_UNDEF(dev-\u003ekvm-\u003earch.vgic.vgic_v5_irs_data-\u003evgic_v5_irs_base))\n+\t\treturn -ENXIO;\n+\n+\tregion = vgic_find_mmio_region(vgic_v5_irs_registers,\n+\t\t\t\t ARRAY_SIZE(vgic_v5_irs_registers),\n+\t\t\t\t offset);\n+\tif (!region)\n+\t\treturn -ENXIO;\n+\n+\t/*\n+\t * Although the spec supports upper/lower 32-bit accesses to\n+\t * 64-bit IRS registers, the userspace ABI requires 64-bit\n+\t * accesses to all 64-bit wide registers. We therefore only\n+\t * support 32-bit accesses to 32-bit-wide registers.\n+\t */\n+\talign = region-\u003eaccess_flags \u0026 VGIC_ACCESS_64bit ? 0x7 : 0x3;\n+\tlen = region-\u003eaccess_flags \u0026 VGIC_ACCESS_64bit ? 8 : 4;\n+\n+\tif (offset \u0026 align)\n+\t\treturn -EINVAL;\n+\n+\taddr = dev-\u003ekvm-\u003earch.vgic.vgic_v5_irs_data-\u003evgic_v5_irs_base + offset;\n+\n+\tif (is_write) {\n+\t\tif (region-\u003euaccess_write)\n+\t\t\tret = region-\u003euaccess_write(kvm_get_vcpu(dev-\u003ekvm, 0),\n+\t\t\t\t\t\t addr, len, *reg);\n+\t\telse\n+\t\t\tregion-\u003ewrite(kvm_get_vcpu(dev-\u003ekvm, 0), addr, len, *reg);\n+\t} else {\n+\t\tif (region-\u003euaccess_read)\n+\t\t\t*reg = region-\u003euaccess_read(kvm_get_vcpu(dev-\u003ekvm, 0),\n+\t\t\t\t\t\t addr, len);\n+\t\telse\n+\t\t\t*reg = region-\u003eread(kvm_get_vcpu(dev-\u003ekvm, 0), addr, len);\n+\t}\n+\n+\treturn ret;\n+}\ndiff --git a/arch/arm64/kvm/vgic/vgic-kvm-device.c b/arch/arm64/kvm/vgic/vgic-kvm-device.c\nindex 90be99443df3bc..1c205cb1361fd5 100644\n--- a/arch/arm64/kvm/vgic/vgic-kvm-device.c\n+++ b/arch/arm64/kvm/vgic/vgic-kvm-device.c\n@@ -181,6 +181,14 @@ static int kvm_vgic_addr(struct kvm *kvm, struct kvm_device_attr *attr, bool wri\n \t\taddr |= (u64)rdreg-\u003ecount \u003c\u003c KVM_VGIC_V3_RDIST_COUNT_SHIFT;\n \t\tgoto out;\n \t}\n+\tcase KVM_VGIC_V5_ADDR_TYPE_IRS:\n+\t\tr = vgic_check_type(kvm, KVM_DEV_TYPE_ARM_VGIC_V5);\n+\t\tif (r)\n+\t\t\tbreak;\n+\t\taddr_ptr = \u0026vgic-\u003evgic_v5_irs_data-\u003evgic_v5_irs_base;\n+\t\talignment = SZ_64K;\n+\t\tsize = KVM_VGIC_V5_IRS_SIZE;\n+\t\tbreak;\n \tdefault:\n \t\tr = -ENODEV;\n \t}\n@@ -224,31 +232,48 @@ static int vgic_set_common_attr(struct kvm_device *dev,\n \t\tif (get_user(val, uaddr))\n \t\t\treturn -EFAULT;\n \n-\t\t/*\n-\t\t * We require:\n-\t\t * - at least 32 SPIs on top of the 16 SGIs and 16 PPIs\n-\t\t * - at most 1024 interrupts\n-\t\t * - a multiple of 32 interrupts\n-\t\t */\n-\t\tif (val \u003c (VGIC_NR_PRIVATE_IRQS + 32) ||\n-\t\t val \u003e VGIC_MAX_RESERVED ||\n-\t\t (val \u0026 31))\n-\t\t\treturn -EINVAL;\n+\t\tif (!vgic_is_v5(dev-\u003ekvm)) {\n+\t\t\t/*\n+\t\t\t * We require:\n+\t\t\t * - at least 32 SPIs on top of the 16 SGIs and 16 PPIs\n+\t\t\t * - at most 1024 interrupts\n+\t\t\t * - a multiple of 32 interrupts\n+\t\t\t */\n+\t\t\tif (val \u003c (VGIC_NR_PRIVATE_IRQS + 32) ||\n+\t\t\t val \u003e VGIC_MAX_RESERVED || (val \u0026 31))\n+\t\t\t\treturn -EINVAL;\n+\n+\t\t\tmutex_lock(\u0026dev-\u003ekvm-\u003earch.config_lock);\n \n-\t\tmutex_lock(\u0026dev-\u003ekvm-\u003earch.config_lock);\n+\t\t\t/*\n+\t\t\t * Either userspace has already configured NR_IRQS or\n+\t\t\t * the vgic has already been initialized and vgic_init()\n+\t\t\t * supplied a default amount of SPIs.\n+\t\t\t */\n+\t\t\tif (dev-\u003ekvm-\u003earch.vgic.nr_spis)\n+\t\t\t\tret = -EBUSY;\n+\t\t\telse\n+\t\t\t\tdev-\u003ekvm-\u003earch.vgic.nr_spis =\n+\t\t\t\t\tval - VGIC_NR_PRIVATE_IRQS;\n \n-\t\t/*\n-\t\t * Either userspace has already configured NR_IRQS or\n-\t\t * the vgic has already been initialized and vgic_init()\n-\t\t * supplied a default amount of SPIs.\n-\t\t */\n-\t\tif (dev-\u003ekvm-\u003earch.vgic.nr_spis)\n-\t\t\tret = -EBUSY;\n-\t\telse\n-\t\t\tdev-\u003ekvm-\u003earch.vgic.nr_spis =\n-\t\t\t\tval - VGIC_NR_PRIVATE_IRQS;\n+\t\t\tmutex_unlock(\u0026dev-\u003ekvm-\u003earch.config_lock);\n+\t\t} else {\n+\t\t\t/*\n+\t\t\t * GICv5 reports a number of SPIs, not a total number of\n+\t\t\t * interrupts. Require a multiple of 32 SPIs.\n+\t\t\t */\n+\t\t\tif (val \u003c VGIC_V5_DEFAULT_NR_SPIS ||\n+\t\t\t val \u003e VGIC_V5_MAX_NR_SPIS ||\n+\t\t\t (val \u0026 31))\n+\t\t\t\treturn -EINVAL;\n \n-\t\tmutex_unlock(\u0026dev-\u003ekvm-\u003earch.config_lock);\n+\t\t\tmutex_lock(\u0026dev-\u003ekvm-\u003earch.config_lock);\n+\t\t\tif (vgic_initialized(dev-\u003ekvm) || dev-\u003ekvm-\u003earch.vgic.nr_spis)\n+\t\t\t\tret = -EBUSY;\n+\t\t\telse\n+\t\t\t\tdev-\u003ekvm-\u003earch.vgic.nr_spis = val;\n+\t\t\tmutex_unlock(\u0026dev-\u003ekvm-\u003earch.config_lock);\n+\t\t}\n \n \t\treturn ret;\n \t}\n@@ -299,9 +324,14 @@ static int vgic_get_common_attr(struct kvm_device *dev,\n \t\treturn (r == -ENODEV) ? -ENXIO : r;\n \tcase KVM_DEV_ARM_VGIC_GRP_NR_IRQS: {\n \t\tu32 __user *uaddr = (u32 __user *)(long)attr-\u003eaddr;\n-\n-\t\tr = put_user(dev-\u003ekvm-\u003earch.vgic.nr_spis +\n-\t\t\t VGIC_NR_PRIVATE_IRQS, uaddr);\n+\t\t/* Older GICs */\n+\t\tif (!vgic_is_v5(dev-\u003ekvm)) {\n+\t\t\tr = put_user(dev-\u003ekvm-\u003earch.vgic.nr_spis +\n+\t\t\t\t\t VGIC_NR_PRIVATE_IRQS,\n+\t\t\t\t uaddr);\n+\t\t} else {\n+\t\t\tr = put_user(dev-\u003ekvm-\u003earch.vgic.nr_spis, uaddr);\n+\t\t}\n \t\tbreak;\n \t}\n \t}\n@@ -512,7 +542,7 @@ int vgic_v3_parse_attr(struct kvm_device *dev, struct kvm_device_attr *attr,\n * Allow access to certain ID-like registers prior to VGIC initialization,\n * thereby allowing the VMM to provision the features / sizing of the VGIC.\n */\n-static bool reg_allowed_pre_init(struct kvm_device_attr *attr)\n+static bool v3_reg_allowed_pre_init(struct kvm_device_attr *attr)\n {\n \tif (attr-\u003egroup != KVM_DEV_ARM_VGIC_GRP_DIST_REGS)\n \t\treturn false;\n@@ -575,7 +605,7 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,\n \n \tmutex_lock(\u0026dev-\u003ekvm-\u003earch.config_lock);\n \n-\tif (!(vgic_initialized(dev-\u003ekvm) || reg_allowed_pre_init(attr))) {\n+\tif (!(vgic_initialized(dev-\u003ekvm) || v3_reg_allowed_pre_init(attr))) {\n \t\tret = -EBUSY;\n \t\tgoto out;\n \t}\n@@ -743,26 +773,178 @@ static int vgic_v5_get_userspace_ppis(struct kvm_device *dev,\n \treturn ret;\n }\n \n+int vgic_v5_parse_attr(struct kvm_device *dev, struct kvm_device_attr *attr,\n+\t\t struct vgic_reg_attr *reg_attr)\n+{\n+\tunsigned long vgic_mpidr, mpidr_reg;\n+\n+\tswitch (attr-\u003egroup) {\n+\tcase KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:\n+\t\tvgic_mpidr = (attr-\u003eattr \u0026 KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) \u003e\u003e\n+\t\t\tKVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;\n+\n+\t\tmpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);\n+\t\treg_attr-\u003evcpu = kvm_mpidr_to_vcpu(dev-\u003ekvm, mpidr_reg);\n+\t\tbreak;\n+\tcase KVM_DEV_ARM_VGIC_GRP_IRS_REGS:\n+\t\treg_attr-\u003evcpu = kvm_get_vcpu(dev-\u003ekvm, 0);\n+\t\tbreak;\n+\tdefault:\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tif (!reg_attr-\u003evcpu)\n+\t\treturn -EINVAL;\n+\n+\treg_attr-\u003eaddr = attr-\u003eattr \u0026 KVM_DEV_ARM_VGIC_OFFSET_MASK;\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Some registers can potentially be read before the core GIC \u0026 IRS has been\n+ * initialised. Right now, everything is required to be post-init.\n+ */\n+static bool v5_reg_allowed_pre_init(struct kvm_device_attr *attr)\n+{\n+\treturn false;\n+}\n+\n+static bool vgic_v5_vm_has_run_once(struct kvm *kvm)\n+{\n+\tstruct kvm_vcpu *vcpu;\n+\tunsigned long i;\n+\n+\tkvm_for_each_vcpu(i, vcpu, kvm) {\n+\t\tif (vcpu_has_run_once(vcpu))\n+\t\t\treturn true;\n+\t}\n+\n+\treturn false;\n+}\n+\n+/*\n+ * vgic_v5_attr_regs_access - allows user space to access VGIC v5 state\n+ *\n+ * @dev: kvm device handle\n+ * @attr: kvm device attribute\n+ * @is_write: true if userspace is writing a register\n+ */\n+static int vgic_v5_attr_regs_access(struct kvm_device *dev,\n+\t\t\t\t struct kvm_device_attr *attr,\n+\t\t\t\t bool is_write)\n+{\n+\tu64 __user *uaddr = (u64 __user *)(unsigned long)attr-\u003eaddr;\n+\tstruct vgic_reg_attr reg_attr;\n+\tstruct kvm_vcpu *vcpu;\n+\tbool uaccess;\n+\tu64 val;\n+\tint ret;\n+\n+\tret = vgic_v5_parse_attr(dev, attr, \u0026reg_attr);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tvcpu = reg_attr.vcpu;\n+\n+\tswitch (attr-\u003egroup) {\n+\tcase KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:\n+\t\t/* Sysregs uaccess is performed by the sysreg handling code */\n+\t\tuaccess = false;\n+\t\tbreak;\n+\tcase KVM_DEV_ARM_VGIC_GRP_IRS_REGS:\n+\t\tfallthrough;\n+\tdefault:\n+\t\tuaccess = true;\n+\t}\n+\n+\tif (uaccess \u0026\u0026 is_write) {\n+\t\tif (get_user(val, uaddr))\n+\t\t\treturn -EFAULT;\n+\t}\n+\n+\tmutex_lock(\u0026dev-\u003ekvm-\u003elock);\n+\n+\tif (kvm_trylock_all_vcpus(dev-\u003ekvm)) {\n+\t\tmutex_unlock(\u0026dev-\u003ekvm-\u003elock);\n+\t\treturn -EBUSY;\n+\t}\n+\n+\tmutex_lock(\u0026dev-\u003ekvm-\u003earch.config_lock);\n+\n+\tif (!(vgic_initialized(dev-\u003ekvm) || v5_reg_allowed_pre_init(attr))) {\n+\t\tret = -EBUSY;\n+\t\tgoto out;\n+\t}\n+\n+\tif (is_write \u0026\u0026 vgic_v5_vm_has_run_once(dev-\u003ekvm)) {\n+\t\tret = -EBUSY;\n+\t\tgoto out;\n+\t}\n+\n+\tswitch (attr-\u003egroup) {\n+\tcase KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:\n+\t\tret = vgic_v5_cpu_sysregs_uaccess(vcpu, attr, is_write);\n+\t\tbreak;\n+\tcase KVM_DEV_ARM_VGIC_GRP_IRS_REGS:\n+\t\t/*\n+\t\t * The IRS registers are a mixture of 32-bit and 64-bit\n+\t\t * registers. Internally, we always perform the correctly sized\n+\t\t * access, but the UAPI is defined in such a way that we are\n+\t\t * always provided a __u64 by userspace. When userspace writes,\n+\t\t * the upper 32-bits are ignored for 32-bit accesses, and on a\n+\t\t * read any 32-bit accesses are written back to user memory\n+\t\t * using the full 64-bits.\n+\t\t */\n+\t\tret = vgic_v5_irs_attr_regs_access(dev, attr, \u0026val, is_write);\n+\t\tbreak;\n+\tdefault:\n+\t\tret = -EINVAL;\n+\t\tbreak;\n+\t}\n+\n+out:\n+\tmutex_unlock(\u0026dev-\u003ekvm-\u003earch.config_lock);\n+\tkvm_unlock_all_vcpus(dev-\u003ekvm);\n+\tmutex_unlock(\u0026dev-\u003ekvm-\u003elock);\n+\n+\tif (!ret \u0026\u0026 uaccess \u0026\u0026 !is_write)\n+\t\tret = put_user(val, uaddr);\n+\n+\treturn ret;\n+}\n+\n static int vgic_v5_set_attr(struct kvm_device *dev,\n \t\t\t struct kvm_device_attr *attr)\n {\n \tswitch (attr-\u003egroup) {\n \tcase KVM_DEV_ARM_VGIC_GRP_ADDR:\n+\t\tbreak;\n+\tcase KVM_DEV_ARM_VGIC_GRP_IST:\n+\t\tif (attr-\u003eattr)\n+\t\t\treturn -ENXIO;\n+\n+\t\treturn vgic_v5_irs_restore_ists(dev-\u003ekvm, attr);\n+\tcase KVM_DEV_ARM_VGIC_GRP_IRS_REGS:\n+\t\tfallthrough;\n \tcase KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:\n+\t\treturn vgic_v5_attr_regs_access(dev, attr, true);\n \tcase KVM_DEV_ARM_VGIC_GRP_NR_IRQS:\n-\t\treturn -ENXIO;\n+\t\tbreak;\n \tcase KVM_DEV_ARM_VGIC_GRP_CTRL:\n \t\tswitch (attr-\u003eattr) {\n \t\tcase KVM_DEV_ARM_VGIC_CTRL_INIT:\n-\t\t\treturn vgic_set_common_attr(dev, attr);\n+\t\t\tbreak;\n \t\tcase KVM_DEV_ARM_VGIC_USERSPACE_PPIS:\n \t\tdefault:\n \t\t\treturn -ENXIO;\n \t\t}\n+\t\tbreak;\n \tdefault:\n \t\treturn -ENXIO;\n \t}\n \n+\treturn vgic_set_common_attr(dev, attr);\n }\n \n static int vgic_v5_get_attr(struct kvm_device *dev,\n@@ -770,21 +952,33 @@ static int vgic_v5_get_attr(struct kvm_device *dev,\n {\n \tswitch (attr-\u003egroup) {\n \tcase KVM_DEV_ARM_VGIC_GRP_ADDR:\n+\t\tbreak;\n+\tcase KVM_DEV_ARM_VGIC_GRP_IST:\n+\t\tif (attr-\u003eattr)\n+\t\t\treturn -ENXIO;\n+\n+\t\treturn vgic_v5_irs_save_ists(dev-\u003ekvm, attr);\n+\tcase KVM_DEV_ARM_VGIC_GRP_IRS_REGS:\n+\t\tfallthrough;\n \tcase KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:\n+\t\treturn vgic_v5_attr_regs_access(dev, attr, false);\n \tcase KVM_DEV_ARM_VGIC_GRP_NR_IRQS:\n-\t\treturn -ENXIO;\n+\t\tbreak;\n \tcase KVM_DEV_ARM_VGIC_GRP_CTRL:\n \t\tswitch (attr-\u003eattr) {\n \t\tcase KVM_DEV_ARM_VGIC_CTRL_INIT:\n-\t\t\treturn vgic_get_common_attr(dev, attr);\n+\t\t\tbreak;\n \t\tcase KVM_DEV_ARM_VGIC_USERSPACE_PPIS:\n \t\t\treturn vgic_v5_get_userspace_ppis(dev, attr);\n \t\tdefault:\n \t\t\treturn -ENXIO;\n \t\t}\n+\t\tbreak;\n \tdefault:\n \t\treturn -ENXIO;\n \t}\n+\n+\treturn vgic_get_common_attr(dev, attr);\n }\n \n static int vgic_v5_has_attr(struct kvm_device *dev,\n@@ -792,18 +986,30 @@ static int vgic_v5_has_attr(struct kvm_device *dev,\n {\n \tswitch (attr-\u003egroup) {\n \tcase KVM_DEV_ARM_VGIC_GRP_ADDR:\n+\t\tswitch (attr-\u003eattr) {\n+\t\tcase KVM_VGIC_V5_ADDR_TYPE_IRS:\n+\t\t\treturn 0;\n+\t\t}\n+\t\treturn -ENXIO;\n+\tcase KVM_DEV_ARM_VGIC_GRP_IRS_REGS:\n+\t\tfallthrough;\n \tcase KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:\n+\t\treturn vgic_v5_has_attr_regs(dev, attr);\n \tcase KVM_DEV_ARM_VGIC_GRP_NR_IRQS:\n-\t\treturn -ENXIO;\n+\t\treturn 0;\n \tcase KVM_DEV_ARM_VGIC_GRP_CTRL:\n \t\tswitch (attr-\u003eattr) {\n \t\tcase KVM_DEV_ARM_VGIC_CTRL_INIT:\n \t\t\treturn 0;\n \t\tcase KVM_DEV_ARM_VGIC_USERSPACE_PPIS:\n \t\t\treturn 0;\n+\t\tcase KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES:\n \t\tdefault:\n \t\t\treturn -ENXIO;\n \t\t}\n+\t\tbreak;\n+\tcase KVM_DEV_ARM_VGIC_GRP_IST:\n+\t\treturn attr-\u003eattr ? -ENXIO : 0;\n \tdefault:\n \t\treturn -ENXIO;\n \t}\ndiff --git a/arch/arm64/kvm/vgic/vgic-mmio.c b/arch/arm64/kvm/vgic/vgic-mmio.c\nindex 74d76dec97304b..fddb9da0403d55 100644\n--- a/arch/arm64/kvm/vgic/vgic-mmio.c\n+++ b/arch/arm64/kvm/vgic/vgic-mmio.c\n@@ -1065,6 +1065,9 @@ static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,\n \tcase IODEV_ITS:\n \t\tdata = region-\u003eits_read(vcpu-\u003ekvm, iodev-\u003eits, addr, len);\n \t\tbreak;\n+\tcase IODEV_GICV5_IRS:\n+\t\tdata = region-\u003eread(vcpu, addr, len);\n+\t\tbreak;\n \t}\n \n \tvgic_data_host_to_mmio_bus(val, len, data);\n@@ -1095,6 +1098,9 @@ static int dispatch_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,\n \tcase IODEV_ITS:\n \t\tregion-\u003eits_write(vcpu-\u003ekvm, iodev-\u003eits, addr, len, data);\n \t\tbreak;\n+\tcase IODEV_GICV5_IRS:\n+\t\tregion-\u003ewrite(vcpu, addr, len, data);\n+\t\tbreak;\n \t}\n \n \treturn 0;\ndiff --git a/arch/arm64/kvm/vgic/vgic-mmio.h b/arch/arm64/kvm/vgic/vgic-mmio.h\nindex 50dc80220b0f37..38ed730d68ac38 100644\n--- a/arch/arm64/kvm/vgic/vgic-mmio.h\n+++ b/arch/arm64/kvm/vgic/vgic-mmio.h\n@@ -217,6 +217,8 @@ unsigned int vgic_v2_init_cpuif_iodev(struct vgic_io_device *dev);\n \n unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);\n \n+unsigned int vgic_v5_init_irs_iodev(struct vgic_io_device *dev);\n+\n u64 vgic_sanitise_outer_cacheability(u64 reg);\n u64 vgic_sanitise_inner_cacheability(u64 reg);\n u64 vgic_sanitise_shareability(u64 reg);\ndiff --git a/arch/arm64/kvm/vgic/vgic-v5-tables.c b/arch/arm64/kvm/vgic/vgic-v5-tables.c\nnew file mode 100644\nindex 00000000000000..f220267266c29d\n--- /dev/null\n+++ b/arch/arm64/kvm/vgic/vgic-v5-tables.c\n@@ -0,0 +1,1795 @@\n+// SPDX-License-Identifier: GPL-2.0-only\n+/*\n+ * Copyright (C) 2025, 2026 Arm Ltd.\n+ */\n+\n+#include \u003ckvm/arm_vgic.h\u003e\n+#include \u003clinux/kernel.h\u003e\n+#include \u003clinux/mm.h\u003e\n+#include \u003clinux/mutex.h\u003e\n+#include \u003clinux/sizes.h\u003e\n+#include \u003clinux/slab.h\u003e\n+#include \u003clinux/uaccess.h\u003e\n+#include \u003clinux/xarray.h\u003e\n+#include \u003casm/kvm_mmu.h\u003e\n+\n+#include \"vgic.h\"\n+#include \"vgic-v5-tables.h\"\n+\n+#define irs_caps\tkvm_vgic_global_state.vgic_v5_irs_caps\n+\n+static struct vgic_v5_vmt *vmt_info;\n+/* Serialises lazy installation of shared second-level VMTs. */\n+static DEFINE_MUTEX(vmt_l2_lock);\n+static DEFINE_XARRAY(vm_info);\n+\n+/* Level 1 Virtual Machine Table Entry */\n+#define GICV5_VMTEL1E_VALID\t\tBIT_ULL(0)\n+/* Note that there is no shift for the address by design */\n+#define GICV5_VMTEL1E_L2_ADDR\t\tGENMASK(51, 12)\n+\n+#define GICV5_VMTEL2E_SIZE\t\t32ULL\n+/* An L2 table (two-level VMT) is ALWAYS 4kB! */\n+#define GICV5_VMT_L2_TABLE_SIZE\t\t4096ULL\n+#define GICV5_VMT_L2_TABLE_ENTRIES\t(GICV5_VMT_L2_TABLE_SIZE / GICV5_VMTEL2E_SIZE)\n+\n+/*\n+ * As the L2 VMTE is a large data structure, we are splitting it into 4 parts.\n+ * We only mask and shift WITHIN each part for simplicity.\n+ */\n+/* First 64-bit chunk */\n+#define GICV5_VMTEL2E_VALID\t\tBIT_ULL(0)\n+#define GICV5_VMTEL2E_VMD_ADDR_SHIFT\t3ULL\n+#define GICV5_VMTEL2E_VMD_ADDR\t\tGENMASK_ULL(55, 3)\n+/* Second 64-bit chunk */\n+#define GICV5_VMTEL2E_VPET_ADDR_SHIFT\t3ULL\n+#define GICV5_VMTEL2E_VPET_ADDR\t\tGENMASK_ULL(55, 3)\n+#define GICV5_VMTEL2E_VPE_ID_BITS\tGENMASK_ULL(63, 59)\n+/* Third \u0026 fourth 64-bit chunks (the encodings are the same for each) */\n+#define GICV5_VMTEL2E_IST_VALID\t\tBIT_ULL(0)\n+#define GICV5_VMTEL2E_IST_L2SZ\t\tGENMASK_ULL(2, 1)\n+#define GICV5_VMTEL2E_IST_ADDR_SHIFT\t6ULL\n+#define GICV5_VMTEL2E_IST_ADDR\t\tGENMASK_ULL(55, 6)\n+#define GICV5_VMTEL2E_IST_ISTSZ\t\tGENMASK_ULL(57, 56)\n+#define GICV5_VMTEL2E_IST_STRUCTURE\tBIT_ULL(58)\n+#define GICV5_VMTEL2E_IST_ID_BITS\tGENMASK_ULL(63, 59)\n+\n+/* Virtual PE Table Entry */\n+#define GICV5_VPE_VALID\t\t\tBIT_ULL(0)\n+/* Note that there is no shift for the address by design. */\n+#define GICV5_VPED_ADDR_SHIFT\t\t3ULL\n+#define GICV5_VPED_ADDR\t\t\tGENMASK_ULL(55, 3)\n+\n+/* L2 Interrupt State Table Entry */\n+#define GICV5_ISTL2E_PENDING\t\tBIT(0)\n+#define GICV5_ISTL2E_ACTIVE\t\tBIT(1)\n+#define GICV5_ISTL2E_HM\t\t\tBIT(2)\n+#define GICV5_ISTL2E_ENABLE\t\tBIT(3)\n+#define GICV5_ISTL2E_IRM\t\tBIT(4)\n+#define GICV5_ISTL2E_HWU\t\tGENMASK(10, 9)\n+#define GICV5_ISTL2E_PRIORITY\t\tGENMASK(15, 11)\n+#define GICV5_ISTL2E_IAFFID\t\tGENMASK(31, 16)\n+\n+#define GICV5_ISTE_SIZE(istsz)\t\tBIT((istsz) + 2)\n+#define GICV5_LINEAR_IST_SIZE(id_bits, istsz)\t\\\n+\t(BIT(id_bits) * GICV5_ISTE_SIZE(istsz))\n+\n+/*\n+ * The LPI and SPI configuration is stored in the 2nd and 3rd 64-bit chunks of\n+ * the VMTE (0-based). We call this a section here in an attempt to simplify the\n+ * code.\n+ */\n+#define GICV5_VMTEL2_LPI_SECTION\t2\n+#define GICV5_VMTEL2_SPI_SECTION\t3\n+\n+struct vgic_v5_ist_desc {\n+\tstruct vgic_v5_vm_info\t*vmi;\n+\tvoid\t\t\t*base;\n+\tunsigned int\t\tid_bits;\n+\tunsigned int\t\tistsz;\n+\tunsigned int\t\tl2sz;\n+\tsize_t\t\t\tiste_size;\n+\tbool\t\t\tpresent;\n+};\n+\n+struct vgic_v5_two_level_ist_shape {\n+\tsize_t\tl1_entries;\n+\tsize_t\tl2_entries;\n+};\n+\n+struct vgic_v5_pending_irq {\n+\tu32\t\t\tirq;\n+\tstruct list_head\tnext;\n+};\n+\n+static int vgic_v5_alloc_linear_ist(struct kvm *kvm, bool spi_ist,\n+\t\t\t\t unsigned int id_bits,\n+\t\t\t\t unsigned int istsz);\n+static int vgic_v5_alloc_l1_ist(struct kvm *kvm, unsigned int id_bits,\n+\t\t\t\tunsigned int istsz, unsigned int l2_split);\n+static int vgic_v5_alloc_l2_ists(struct kvm *kvm, unsigned int id_bits,\n+\t\t\t\t unsigned int istsz, unsigned int l2_split);\n+static int vgic_v5_alloc_two_level_lpi_ist(struct kvm *kvm,\n+\t\t\t\t\t unsigned int id_bits,\n+\t\t\t\t\t unsigned int istsz,\n+\t\t\t\t\t unsigned int l2_split);\n+static int vgic_v5_linear_ist_free(struct kvm *kvm, bool spi);\n+static int vgic_v5_two_level_ist_free(struct kvm *kvm, bool spi);\n+static int vgic_v5_spi_ist_free(struct kvm *kvm);\n+\n+/*\n+ * Our IRS might be coherent or non-coherent. If coherent, we can just emit a\n+ * DSB to ensure that we're in sync. However, when non-coherent, we need to\n+ * manage our cached data explicitly.\n+ *\n+ * This helper is used to handle both coherent and non-coherent IRSes, and\n+ * handles all combinations of cleaning and invalidating to the PoC.\n+ */\n+static void vgic_v5_clean_inval(void *va, size_t size)\n+{\n+\tunsigned long base = (unsigned long)va;\n+\n+\tdsb(ishst);\n+\n+\tif (kvm_vgic_global_state.vgic_v5_irs_caps.non_coherent)\n+\t\tdcache_clean_inval_poc(base, base + size);\n+}\n+\n+static void vgic_v5_drain_pending_irqs(struct kvm *kvm,\n+\t\t\t\t struct vgic_v5_vm_info *vmi,\n+\t\t\t\t bool reinject)\n+{\n+\tstruct vgic_v5_pending_irq *pirq, *tmp;\n+\n+\tlist_for_each_entry_safe(pirq, tmp, \u0026vmi-\u003epending_irqs, next) {\n+\t\tif (reinject)\n+\t\t\tkvm_call_hyp(__vgic_v5_vdpend, pirq-\u003eirq, true,\n+\t\t\t\t vgic_v5_vm_id(kvm));\n+\n+\t\tlist_del(\u0026pirq-\u003enext);\n+\t\tkfree(pirq);\n+\t}\n+}\n+\n+/*\n+ * Create a linear VM Table. Directly using the number of entries supplied as\n+ * the size of an L2 VMTE (32 bytes) guarantees that our allocation is aligned per\n+ * the GICv5 requirements for the IRS_VMT_BASER.\n+ */\n+static int vgic_v5_alloc_vmt_linear(unsigned int num_entries)\n+{\n+\tvmt_info-\u003elinear.vmt_base = kzalloc_objs(*vmt_info-\u003elinear.vmt_base,\n+\t\t\t\t\t\t num_entries);\n+\tif (!vmt_info-\u003elinear.vmt_base)\n+\t\treturn -ENOMEM;\n+\n+\tvgic_v5_clean_inval(vmt_info-\u003elinear.vmt_base,\n+\t\t\t num_entries * sizeof(struct vmtl2_entry));\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Allocate the first level of a two-level VM table. The second-level VM tables\n+ * are allocated on demand (by vgic_v5_alloc_l2_vmt()).\n+ */\n+static int vgic_v5_alloc_vmt_two_level(unsigned int num_entries)\n+{\n+\t/*\n+\t * Each L2 VMT array is always 4k-sized (covering 128 VMs). This is\n+\t * mandated by the GICv5 specification (GICv5 EAC0 Specification rule\n+\t * D_LSPBK). Hence, round up the number of entries to be at least 128\n+\t * (or the next highest power of two as we give the HW the number of VM\n+\t * ID bits).\n+\t */\n+\tif (num_entries \u003c GICV5_VMT_L2_TABLE_ENTRIES)\n+\t\tnum_entries = GICV5_VMT_L2_TABLE_ENTRIES;\n+\tnum_entries = roundup_pow_of_two(num_entries);\n+\n+\tvmt_info-\u003el2.num_l1_ents = (num_entries / GICV5_VMT_L2_TABLE_ENTRIES);\n+\tvmt_info-\u003el2.vmt_base = kzalloc_objs(*vmt_info-\u003el2.vmt_base,\n+\t\t\t\t\t vmt_info-\u003el2.num_l1_ents);\n+\tif (!vmt_info-\u003el2.vmt_base)\n+\t\treturn -ENOMEM;\n+\n+\tvmt_info-\u003el2.l2ptrs = kzalloc_objs(*vmt_info-\u003el2.l2ptrs,\n+\t\t\t\t\t vmt_info-\u003el2.num_l1_ents,\n+\t\t\t\t\t GFP_KERNEL);\n+\tif (!vmt_info-\u003el2.l2ptrs) {\n+\t\tkfree(vmt_info-\u003el2.vmt_base);\n+\t\treturn -ENOMEM;\n+\t}\n+\n+\tvgic_v5_clean_inval(vmt_info-\u003el2.vmt_base,\n+\t\t\t vmt_info-\u003el2.num_l1_ents * sizeof(vmtl1_entry));\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Allocate a second level VMT, if required. This can be called eagerly, and\n+ * will only perform the allocation if required.\n+ */\n+static int vgic_v5_alloc_l2_vmt(struct kvm *kvm)\n+{\n+\tstruct kvm_vcpu *vcpu0 = kvm_get_vcpu(kvm, 0);\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tenum gicv5_vcpu_cmd cmd = VMT_L2_MAP;\n+\tstruct vmtl2_entry *l2_table;\n+\tunsigned int l1_index;\n+\tint ret;\n+\n+\t/* Nothing to do if we have linear tables! */\n+\tif (!vmt_info-\u003etwo_level)\n+\t\treturn 0;\n+\n+\tif (vm_id == VGIC_V5_VM_ID_INVAL)\n+\t\treturn -EINVAL;\n+\n+\t/*\n+\t * We have 4k-sized L2 tables - this is mandated by the spec for\n+\t * two-level VMTs (GICv5 EAC0 Specification rule D_LSPBK). This means\n+\t * that we have 128 entries per L1 VMTE.\n+\t */\n+\tl1_index = vm_id / GICV5_VMT_L2_TABLE_ENTRIES;\n+\n+\tguard(mutex)(\u0026vmt_l2_lock);\n+\n+\t/* Already valid? Great! */\n+\tif (vmt_info-\u003el2.l2ptrs[l1_index])\n+\t\treturn 0;\n+\n+\tl2_table = kzalloc_objs(*l2_table, GICV5_VMT_L2_TABLE_ENTRIES);\n+\tif (!l2_table)\n+\t\treturn -ENOMEM;\n+\n+\tvgic_v5_clean_inval(l2_table, GICV5_VMT_L2_TABLE_SIZE);\n+\n+\tvgic_v5_clean_inval(vmt_info-\u003el2.vmt_base + l1_index,\n+\t\t\t sizeof(vmtl1_entry));\n+\n+\tWRITE_ONCE(vmt_info-\u003el2.vmt_base[l1_index],\n+\t\t cpu_to_le64(virt_to_phys(l2_table)));\n+\n+\tvgic_v5_clean_inval(vmt_info-\u003el2.vmt_base + l1_index,\n+\t\t\t sizeof(vmtl1_entry));\n+\n+\t/*\n+\t * VMAP in the L2 VMT via the IRS. We use any of the VM's CPUs as a\n+\t * conduit for interacting with the host's IRS. In the current case,\n+\t * this lets us resolve the VM ID to pass to the hardware.\n+\t */\n+\tret = irq_set_vcpu_affinity(vgic_v5_vpe_db(vcpu0), \u0026cmd);\n+\n+\t/* We've failed to make the L2 VMT valid - things are very broken! */\n+\tif (ret) {\n+\t\t/* Remove the pointer from L1 table */\n+\t\tWRITE_ONCE(vmt_info-\u003el2.vmt_base[l1_index], 0);\n+\n+\t\tvgic_v5_clean_inval(vmt_info-\u003el2.vmt_base + l1_index,\n+\t\t\t\t sizeof(vmtl1_entry));\n+\n+\t\tkfree(l2_table);\n+\n+\t\treturn ret;\n+\t}\n+\n+\tvmt_info-\u003el2.l2ptrs[l1_index] = l2_table;\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Allocate the top-level VMT. This can either be linear or two-level.\n+ */\n+int vgic_v5_vmt_allocate(unsigned int max_vpes)\n+{\n+\tint ret;\n+\n+\t/* Allocate the tracking structure */\n+\tvmt_info = kzalloc_obj(*vmt_info, GFP_KERNEL);\n+\tif (!vmt_info)\n+\t\treturn -ENOMEM;\n+\n+\tida_init(\u0026vmt_info-\u003evm_id_ida);\n+\tvmt_info-\u003emax_vpes = max_vpes;\n+\tvmt_info-\u003evmd_size = irs_caps.vmd_size;\n+\tvmt_info-\u003evped_size = irs_caps.vped_size;\n+\tvmt_info-\u003etwo_level = irs_caps.two_level_vmt_support;\n+\tvmt_info-\u003enum_entries = irs_caps.max_vms;\n+\n+\tif (vmt_info-\u003etwo_level)\n+\t\tret = vgic_v5_alloc_vmt_two_level(vmt_info-\u003enum_entries);\n+\telse\n+\t\tret = vgic_v5_alloc_vmt_linear(vmt_info-\u003enum_entries);\n+\n+\t/* If anything failed, free our tracking structure before returning */\n+\tif (ret) {\n+\t\tkfree(vmt_info);\n+\t\tvmt_info = NULL;\n+\t}\n+\n+\treturn ret;\n+}\n+\n+/*\n+ * Free the VMT and associated tracking structures. This isn't strictly expected\n+ * to be called in general operation, but instead exists for completeness.\n+ */\n+int vgic_v5_vmt_free(void)\n+{\n+\tif (!vmt_info)\n+\t\treturn 0;\n+\n+\tif (!vmt_info-\u003etwo_level) {\n+\t\tkfree(vmt_info-\u003elinear.vmt_base);\n+\t} else {\n+\t\t/* Free the L2 tables; kfree(NULL) is safe */\n+\t\tfor (int i = 0; i \u003c vmt_info-\u003el2.num_l1_ents; ++i)\n+\t\t\tkfree(vmt_info-\u003el2.l2ptrs[i]);\n+\t\tkfree(vmt_info-\u003el2.l2ptrs);\n+\n+\t\t/* And now free the L1 table */\n+\t\tkfree(vmt_info-\u003el2.vmt_base);\n+\t}\n+\n+\tida_destroy(\u0026vmt_info-\u003evm_id_ida);\n+\tkfree(vmt_info);\n+\tvmt_info = NULL;\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Look up a VMT Entry by VM ID.\n+ */\n+static struct vmtl2_entry *vgic_v5_get_l2_vmte(u32 vm_id)\n+{\n+\tunsigned int l1_index, l2_index;\n+\tstruct vmtl2_entry *l2_table;\n+\n+\tif (vm_id == VGIC_V5_VM_ID_INVAL)\n+\t\treturn ERR_PTR(-EINVAL);\n+\n+\tif (!vmt_info-\u003etwo_level)\n+\t\treturn \u0026vmt_info-\u003elinear.vmt_base[vm_id];\n+\n+\tl1_index = vm_id / GICV5_VMT_L2_TABLE_ENTRIES;\n+\tl2_index = vm_id % GICV5_VMT_L2_TABLE_ENTRIES;\n+\n+\tif (l1_index \u003e= vmt_info-\u003el2.num_l1_ents)\n+\t\treturn ERR_PTR(-E2BIG);\n+\n+\tif (!vmt_info-\u003el2.l2ptrs[l1_index])\n+\t\treturn ERR_PTR(-EINVAL);\n+\n+\tl2_table = vmt_info-\u003el2.l2ptrs[l1_index];\n+\treturn \u0026l2_table[l2_index];\n+}\n+\n+/*\n+ * Zero a VMT Entry, and flush \u0026 invalidate to the PoC, if required.\n+ */\n+static int vgic_v5_reset_vmte(struct kvm *kvm)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vmtl2_entry *vmte;\n+\n+\tvmte = vgic_v5_get_l2_vmte(vm_id);\n+\tif (IS_ERR(vmte))\n+\t\treturn PTR_ERR(vmte);\n+\n+\t/*\n+\t * The VMT is normal memory shared with the IRS. Invalidate before\n+\t * rewriting the entry so that cacheline-granular maintenance cannot\n+\t * later push stale data for neighbouring IRS-visible state back to\n+\t * memory.\n+\t */\n+\tvgic_v5_clean_inval(vmte, sizeof(*vmte));\n+\n+\t/*\n+\t * Prevent the compiler from eliding the individual VMTE\n+\t * stores. Ordering and visibility to the IRS are provided by the\n+\t * surrounding cache maintenance and command protocol, not by\n+\t * WRITE_ONCE().\n+\t *\n+\t * The same compiler-access constraint applies to READ_ONCE() users in\n+\t * this file: when inspecting IRS-visible table entries, read the field\n+\t * exactly once and prevent the compiler from reusing, merging or\n+\t * tearing the access. Coherency and freshness for non-coherent IRSes\n+\t * still come from the surrounding cache maintenance.\n+\t */\n+\tWRITE_ONCE(vmte-\u003eval[0], cpu_to_le64(0ULL));\n+\tWRITE_ONCE(vmte-\u003eval[1], cpu_to_le64(0ULL));\n+\tWRITE_ONCE(vmte-\u003eval[2], cpu_to_le64(0ULL));\n+\tWRITE_ONCE(vmte-\u003eval[3], cpu_to_le64(0ULL));\n+\n+\t/* And make our write visible to the IRS (if non-coherent) */\n+\tvgic_v5_clean_inval(vmte, sizeof(*vmte));\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Use the IDA to allocate a new VM ID, and track it in the gicv5_vm data\n+ * structure. If we're out of VM IDs, the IDA catches that, and we return the\n+ * error (-ENOSPC). If we've previously allocated a VM ID, we catch that too and\n+ * return -EBUSY.\n+ */\n+int vgic_v5_allocate_vm_id(struct kvm *kvm)\n+{\n+\tint id;\n+\n+\tif (kvm-\u003earch.vgic.gicv5_vm.vm_id != VGIC_V5_VM_ID_INVAL)\n+\t\treturn -EBUSY;\n+\n+\tid = ida_alloc_max(\u0026vmt_info-\u003evm_id_ida, vmt_info-\u003enum_entries - 1u,\n+\t\t\t GFP_KERNEL);\n+\tif (id \u003c 0)\n+\t\treturn id;\n+\n+\tkvm-\u003earch.vgic.gicv5_vm.vm_id = id;\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Release the VM ID to allow it to be reallocated in the future.\n+ */\n+void vgic_v5_release_vm_id(struct kvm *kvm)\n+{\n+\tif (kvm-\u003earch.vgic.gicv5_vm.vm_id == VGIC_V5_VM_ID_INVAL)\n+\t\treturn;\n+\n+\tida_free(\u0026vmt_info-\u003evm_id_ida, kvm-\u003earch.vgic.gicv5_vm.vm_id);\n+\tkvm-\u003earch.vgic.gicv5_vm.vm_id = VGIC_V5_VM_ID_INVAL;\n+}\n+\n+/*\n+ * Initialise an entry in the VMT based on the index of the VM.\n+ *\n+ * Note: We don't mark the VMTE as valid as this needs to be done by\n+ * the hardware.\n+ */\n+int vgic_v5_vmte_init(struct kvm *kvm)\n+{\n+\tint nr_cpus = atomic_read(\u0026kvm-\u003eonline_vcpus);\n+\tstruct vgic_v5_vm_info *vmi = NULL;\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vmtl2_entry *vmte;\n+\tvoid **vped_ptrs = NULL;\n+\tvpe_entry *vpet = NULL;\n+\tvoid *vmd = NULL;\n+\tint ret;\n+\tu64 tmp;\n+\n+\tif (nr_cpus \u003e vmt_info-\u003emax_vpes)\n+\t\treturn -E2BIG;\n+\n+\t/*\n+\t * If we're using two-level VMTs, L2 is allocated on demand. For linear\n+\t * VMTs, this is a NOP.\n+\t */\n+\tret = vgic_v5_alloc_l2_vmt(kvm);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tvmte = vgic_v5_get_l2_vmte(vm_id);\n+\tif (IS_ERR(vmte))\n+\t\treturn PTR_ERR(vmte);\n+\n+\t/* If the entry is already valid, something went wrong */\n+\tvgic_v5_clean_inval(vmte, sizeof(*vmte));\n+\tif (le64_to_cpu(READ_ONCE(vmte-\u003eval[0])) \u0026 GICV5_VMTEL2E_VALID)\n+\t\treturn -EINVAL;\n+\n+\tret = vgic_v5_reset_vmte(kvm);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tvmi = kzalloc_obj(*vmi);\n+\tif (!vmi) {\n+\t\tret = -ENOMEM;\n+\t\tgoto out_fail;\n+\t}\n+\n+\tret = xa_insert(\u0026vm_info, vm_id, vmi, GFP_KERNEL);\n+\tif (ret)\n+\t\tgoto out_fail;\n+\n+\t/*\n+\t * If we are restoring the state of a guest, we need to re-inject any\n+\t * IRQs that were pending when the state of the guest was originally\n+\t * saved. We use the pending_irqs list for this.\n+\t */\n+\tINIT_LIST_HEAD(\u0026vmi-\u003epending_irqs);\n+\n+\t/* Allocate and assign the VM Descriptor, if required. */\n+\tif (vmt_info-\u003evmd_size != 0) {\n+\t\tvmd = kzalloc(vmt_info-\u003evmd_size, GFP_KERNEL);\n+\t\tif (!vmd) {\n+\t\t\tret = -ENOMEM;\n+\t\t\tgoto out_fail;\n+\t\t}\n+\n+\t\t/* Stash the VA so we can free it later */\n+\t\tvmi-\u003evmd_base = vmd;\n+\n+\t\ttmp = FIELD_PREP(GICV5_VMTEL2E_VMD_ADDR,\n+\t\t\t\t virt_to_phys(vmd) \u003e\u003e GICV5_VMTEL2E_VMD_ADDR_SHIFT);\n+\t\tWRITE_ONCE(vmte-\u003eval[0], cpu_to_le64(tmp));\n+\t}\n+\n+\t/*\n+\t * Allocate and assign the VPE Table. Round up the number of CPUs to a\n+\t * whole power of two as we cannot describe non-powers-of-two in the\n+\t * VMTE field as it conveys the number of ID bits used and not the\n+\t * number of vPEs. IRS_IDR1.IAFFID_BITS is encoded as N - 1, so expose\n+\t * at least one VPE ID bit even for a single-vCPU VM to keep the views\n+\t * consistent.\n+\t */\n+\tnr_cpus = max(2UL, roundup_pow_of_two(nr_cpus));\n+\tvmi-\u003evpe_id_bits = fls(nr_cpus) - 1;\n+\n+\tvpet = kzalloc_objs(*vpet, nr_cpus);\n+\tif (!vpet) {\n+\t\tret = -ENOMEM;\n+\t\tgoto out_fail;\n+\t}\n+\n+\t/* Stash the VA so we can free it later */\n+\tvmi-\u003evpet_base = vpet;\n+\n+\ttmp = FIELD_PREP(GICV5_VMTEL2E_VPET_ADDR,\n+\t\t\t virt_to_phys(vpet) \u003e\u003e GICV5_VMTEL2E_VPET_ADDR_SHIFT);\n+\ttmp |= FIELD_PREP(GICV5_VMTEL2E_VPE_ID_BITS, vmi-\u003evpe_id_bits);\n+\tWRITE_ONCE(vmte-\u003eval[1], cpu_to_le64(tmp));\n+\n+\tvped_ptrs = kzalloc_objs(*vped_ptrs, nr_cpus, GFP_KERNEL);\n+\tif (!vped_ptrs) {\n+\t\tret = -ENOMEM;\n+\t\tgoto out_fail;\n+\t}\n+\tvmi-\u003evped_ptrs = vped_ptrs;\n+\n+\tif (vmd)\n+\t\tvgic_v5_clean_inval(vmd, vmt_info-\u003evmd_size);\n+\tvgic_v5_clean_inval(vpet, sizeof(*vpet) * nr_cpus);\n+\tvgic_v5_clean_inval(vmte, sizeof(*vmte));\n+\n+\tkvm-\u003earch.vgic.gicv5_vm.vmte_allocated = true;\n+\n+\treturn 0;\n+\n+out_fail:\n+\t/* kfree(NULL) is safe so we can just kfree() at leisure */\n+\tkfree(vmd);\n+\tkfree(vpet);\n+\tkfree(vped_ptrs);\n+\tif (vmi)\n+\t\txa_erase(\u0026vm_info, vm_id);\n+\tkfree(vmi);\n+\n+\tvgic_v5_reset_vmte(kvm);\n+\n+\treturn ret;\n+}\n+\n+/*\n+ * Release the VMT Entry, freeing up any allocated data structures before\n+ * zeroing the VMTE.\n+ *\n+ * The VMTE must be marked as invalid before it is released.\n+ */\n+int vgic_v5_vmte_release(struct kvm *kvm)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vgic_v5_vm_info *vmi;\n+\tstruct vmtl2_entry *vmte;\n+\tint ret;\n+\n+\tvmte = vgic_v5_get_l2_vmte(vm_id);\n+\tif (IS_ERR(vmte))\n+\t\treturn PTR_ERR(vmte);\n+\n+\t/* Reject if the VMTE has not been marked as invalid! */\n+\tvgic_v5_clean_inval(vmte, sizeof(*vmte));\n+\tif (le64_to_cpu(READ_ONCE(vmte-\u003eval[0])) \u0026 GICV5_VMTEL2E_VALID)\n+\t\treturn -EINVAL;\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\tgoto no_vmi;\n+\n+\t/* Unlikely, but possible. Avoid leaking the memory. */\n+\tvgic_v5_drain_pending_irqs(kvm, vmi, false);\n+\n+\t/* If we have an LPI IST, free it */\n+\tif (vmi-\u003eh_lpi_ist) {\n+\t\tret = vgic_v5_lpi_ist_free(kvm);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\t}\n+\tvmi-\u003eh_lpi_ist = NULL;\n+\n+\t/* If we have an SPI IST, free it */\n+\tif (vmi-\u003eh_spi_ist) {\n+\t\tret = vgic_v5_spi_ist_free(kvm);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\t}\n+\tvmi-\u003eh_spi_ist = NULL;\n+\n+\tfor (int i = 0; i \u003c BIT(vmi-\u003evpe_id_bits); i++)\n+\t\tkfree(vmi-\u003evped_ptrs[i]);\n+\tkfree(vmi-\u003evped_ptrs);\n+\tkfree(vmi-\u003evpet_base);\n+\tkfree(vmi-\u003evmd_base);\n+\n+\txa_erase(\u0026vm_info, vm_id);\n+\tkfree(vmi);\n+\n+no_vmi:\n+\t/*\n+\t * If we didn't get far enough into allocating a VMTE to create the VM\n+\t * info structure, then we just zero the VMTE and move on. There's\n+\t * nothing else we can realistically do here.\n+\t */\n+\tret = vgic_v5_reset_vmte(kvm);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tkvm-\u003earch.vgic.gicv5_vm.vmte_allocated = false;\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Provide a way for the IRS MMIO emulation to correctly populate the number of\n+ * IAFFID bits (which correspond to our vpe_id_bits.\n+ */\n+u8 vgic_v5_vmte_vpe_id_bits(struct kvm_vcpu *vcpu)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(vcpu-\u003ekvm);\n+\tstruct vgic_v5_vm_info *vmi;\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\treturn 0;\n+\n+\treturn vmi-\u003evpe_id_bits;\n+}\n+\n+/*\n+ * Allocate a VPE descriptor and provide it to the hardware via the VPE Table.\n+ */\n+int vgic_v5_vmte_alloc_vpe(struct kvm_vcpu *vcpu)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(vcpu-\u003ekvm);\n+\tu16 vpe_id = vgic_v5_vpe_id(vcpu);\n+\tstruct vgic_v5_vm_info *vmi;\n+\tvpe_entry tmp, *vpet_base;\n+\tvoid *vped;\n+\n+\t/* Make sure we're not over what the hardware supports */\n+\tif (vpe_id \u003e= vmt_info-\u003emax_vpes)\n+\t\treturn -E2BIG;\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\treturn -EINVAL;\n+\n+\tif (vpe_id \u003e= 1 \u003c\u003c vmi-\u003evpe_id_bits)\n+\t\treturn -E2BIG;\n+\n+\tvpet_base = vmi-\u003evpet_base;\n+\n+\t/* If the VPETE for this CPU is already valid we've gone wrong */\n+\tvgic_v5_clean_inval(\u0026vpet_base[vpe_id], sizeof(*vpet_base));\n+\tif (le64_to_cpu(READ_ONCE(vpet_base[vpe_id])) \u0026 GICV5_VPE_VALID)\n+\t\treturn -EBUSY;\n+\n+\t/* Alloc VPE Descriptor. Only used by IRS. */\n+\tvped = kzalloc(vmt_info-\u003evped_size, GFP_KERNEL);\n+\tif (!vped)\n+\t\treturn -ENOMEM;\n+\n+\tvmi-\u003evped_ptrs[vpe_id] = vped;\n+\n+\ttmp = FIELD_PREP(GICV5_VPED_ADDR, virt_to_phys(vped) \u003e\u003e GICV5_VPED_ADDR_SHIFT);\n+\tWRITE_ONCE(vpet_base[vpe_id], cpu_to_le64(tmp));\n+\n+\tvgic_v5_clean_inval(vped, vmt_info-\u003evped_size);\n+\tvgic_v5_clean_inval(vpet_base + vpe_id, sizeof(vpe_entry));\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Free the memory allocated for the VPE descriptor.\n+ */\n+int vgic_v5_vmte_free_vpe(struct kvm_vcpu *vcpu)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(vcpu-\u003ekvm);\n+\tu16 vpe_id = vgic_v5_vpe_id(vcpu);\n+\tstruct vgic_v5_vm_info *vmi;\n+\tstruct vmtl2_entry *vmte;\n+\tvpe_entry *vpet_base;\n+\tvoid *vped;\n+\n+\tvmte = vgic_v5_get_l2_vmte(vm_id);\n+\tif (IS_ERR(vmte))\n+\t\treturn PTR_ERR(vmte);\n+\n+\tvgic_v5_clean_inval(vmte, sizeof(*vmte));\n+\tif (le64_to_cpu(READ_ONCE(vmte-\u003eval[0])) \u0026 GICV5_VMTEL2E_VALID)\n+\t\treturn -EBUSY;\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\treturn -EINVAL;\n+\n+\tif (vpe_id \u003e= 1 \u003c\u003c vmi-\u003evpe_id_bits)\n+\t\treturn -E2BIG;\n+\n+\tvpet_base = vmi-\u003evpet_base;\n+\tWRITE_ONCE(vpet_base[vpe_id], 0ULL);\n+\n+\tvgic_v5_clean_inval(vpet_base + vpe_id, sizeof(vpe_entry));\n+\n+\t/* Free VPE Descriptor. Only used by IRS. */\n+\tvped = vmi-\u003evped_ptrs[vpe_id];\n+\tvmi-\u003evped_ptrs[vpe_id] = NULL;\n+\tkfree(vped);\n+\n+\treturn 0;\n+}\n+\n+phys_addr_t vgic_v5_get_vmt_base(void)\n+{\n+\tphys_addr_t vmt_base;\n+\n+\tif (!vmt_info-\u003etwo_level)\n+\t\tvmt_base = virt_to_phys(vmt_info-\u003elinear.vmt_base);\n+\telse\n+\t\tvmt_base = virt_to_phys(vmt_info-\u003el2.vmt_base);\n+\n+\treturn vmt_base;\n+}\n+\n+/*\n+ * Assign an already allocated IST to the VM by populating the fields in the\n+ * corresponding VMTE. We re-use this code for both an SPI IST and LPI IST, even\n+ * if the paths to reach it might be vastly different.\n+ */\n+static int vgic_v5_vmte_assign_ist(struct kvm *kvm, phys_addr_t ist_base,\n+\t\t\t\t bool two_level, unsigned int id_bits,\n+\t\t\t\t unsigned int l2sz, unsigned int istsz,\n+\t\t\t\t bool spi_ist)\n+{\n+\tstruct kvm_vcpu *vcpu0 = kvm_get_vcpu(kvm, 0);\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tenum gicv5_vcpu_cmd cmd;\n+\tstruct vmtl2_entry *vmte;\n+\tunsigned int section;\n+\tu64 tmp;\n+\tint ret;\n+\n+\t/*\n+\t * The L2 VMTE comprises four 64-bit \"sections\", where sections 2 \u0026 3\n+\t * describe the LPI and SPI ISTs, respectively. Both the LPI and SPI\n+\t * sections have the same layout, and as we are either operating on SPIs\n+\t * or LPIs we pick a section of the VMTE to modify up-front.\n+\t *\n+\t * See the GICv5 EAC0 Specification 11.2.2 for more details about the\n+\t * VMTE layout.\n+\t */\n+\tsection = spi_ist ? GICV5_VMTEL2_SPI_SECTION : GICV5_VMTEL2_LPI_SECTION;\n+\n+\tif (ist_base \u0026 ~GICV5_VMTEL2E_IST_ADDR) {\n+\t\tkvm_err(\"IST alignment issue! Address: 0x%llx, Mask 0x%llx\\n\",\n+\t\t\tist_base, GICV5_VMTEL2E_IST_ADDR);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tvmte = vgic_v5_get_l2_vmte(vm_id);\n+\tif (IS_ERR(vmte))\n+\t\treturn PTR_ERR(vmte);\n+\n+\t/* Bail if already allocated */\n+\tvgic_v5_clean_inval(vmte, sizeof(*vmte));\n+\tif (le64_to_cpu(READ_ONCE(vmte-\u003eval[section])) \u0026 GICV5_VMTEL2E_IST_VALID)\n+\t\treturn -EINVAL;\n+\n+\ttmp = FIELD_PREP(GICV5_VMTEL2E_IST_L2SZ, l2sz);\n+\ttmp |= FIELD_PREP(GICV5_VMTEL2E_IST_ADDR,\n+\t\t\tist_base \u003e\u003e GICV5_VMTEL2E_IST_ADDR_SHIFT);\n+\ttmp |= FIELD_PREP(GICV5_VMTEL2E_IST_ISTSZ, istsz);\n+\ttmp |= FIELD_PREP(GICV5_VMTEL2E_IST_ID_BITS, id_bits);\n+\tif (two_level)\n+\t\ttmp |= GICV5_VMTEL2E_IST_STRUCTURE;\n+\n+\tWRITE_ONCE(vmte-\u003eval[section], cpu_to_le64(tmp));\n+\tvgic_v5_clean_inval(vmte, sizeof(*vmte));\n+\n+\t/* Finally, mark the entry as valid */\n+\tcmd = spi_ist ? SPI_VIST_MAKE_VALID : LPI_VIST_MAKE_VALID;\n+\tret = irq_set_vcpu_affinity(vgic_v5_vpe_db(vcpu0), \u0026cmd);\n+\tif (ret) {\n+\t\tWRITE_ONCE(vmte-\u003eval[section], 0ULL);\n+\t\tvgic_v5_clean_inval(vmte, sizeof(*vmte));\n+\t\treturn ret;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Allocate a Linear IST - always used for SPIs and potentially LPIs.\n+ *\n+ * The calculation for n has been taken from section 11.2.2 of the GICv5 EAC0\n+ * spec.\n+ *\n+ * NOTE: istsz is the FIELD used by GICv5, not the actual size (or log2() of the\n+ * size).\n+ */\n+static int vgic_v5_alloc_linear_ist(struct kvm *kvm, bool spi_ist,\n+\t\t\t\t unsigned int id_bits, unsigned int istsz)\n+{\n+\tconst size_t n = id_bits + 1 + istsz;\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vgic_v5_vm_info *vmi;\n+\t__le64 *ist;\n+\tu32 l1sz;\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\treturn -EINVAL;\n+\n+\t/*\n+\t * Allocate the IST. We only have one level, so we just use the L2 ISTE.\n+\t */\n+\tl1sz = BIT(n + 1);\n+\tist = kzalloc(l1sz, GFP_KERNEL_ACCOUNT);\n+\tif (!ist)\n+\t\treturn -ENOMEM;\n+\n+\tif (spi_ist) {\n+\t\tvmi-\u003eh_spi_ist = ist;\n+\t} else {\n+\t\tvmi-\u003eh_lpi_ist_structure = false;\n+\t\tvmi-\u003eh_lpi_ist = ist;\n+\t}\n+\n+\tvgic_v5_clean_inval(ist, l1sz);\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Allocate the first level of a two-level IST - LPI, only.\n+ *\n+ * The calculation for n has been taken from section 11.2.2 of the GICv5 EAC0\n+ * spec.\n+ *\n+ * NOTE: istsz and l2sz are the FIELDS used by GICv5, not the actual sizes (or\n+ * log2() of the sizes).\n+ */\n+static int vgic_v5_alloc_l1_ist(struct kvm *kvm, unsigned int id_bits,\n+\t\t\t\tunsigned int istsz, unsigned int l2sz)\n+{\n+\tconst size_t n = max(5, id_bits - ((10 - istsz) + (2 * l2sz)) + 3 - 1);\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tconst u32 l1_size = BIT(n + 1);\n+\tstruct vgic_v5_vm_info *vmi;\n+\t__le64 *ist;\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\treturn -EINVAL;\n+\n+\tist = kzalloc(l1_size, GFP_KERNEL_ACCOUNT);\n+\tif (!ist)\n+\t\treturn -ENOMEM;\n+\n+\tvmi-\u003eh_lpi_ist_structure = true;\n+\tvmi-\u003eh_lpi_ist = ist;\n+\n+\tvgic_v5_clean_inval(ist, l1_size);\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Allocate ALL of the second level ISTs for a two-level IST - LPI, only.\n+ *\n+ * The calculation for n has been taken from section 11.2.2 of the GICv5 EAC0\n+ * spec. The l2_size calculation is from section 11.2.3 of the same document.\n+ *\n+ * NOTE: istsz and l2sz are the FIELDS used by GICv5, not the actual sizes (or\n+ * log2() of the sizes).\n+ */\n+static int vgic_v5_alloc_l2_ists(struct kvm *kvm, unsigned int id_bits,\n+\t\t\t\t unsigned int istsz, unsigned int l2sz)\n+{\n+\tconst size_t n = max(5, id_bits - ((10 - istsz) + (2 * l2sz)) + 3 - 1);\n+\tconst int l1_entries = BIT(n + 1) / GICV5_IRS_ISTL1E_SIZE;\n+\tconst size_t l2_size = BIT(11 + (2 * l2sz) + 1);\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vgic_v5_vm_info *vmi;\n+\t__le64 *l2ist;\n+\t__le64 *l1ist;\n+\tint index;\n+\tu64 val;\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\treturn -EINVAL;\n+\n+\tl1ist = vmi-\u003eh_lpi_ist;\n+\n+\t/*\n+\t * Allocate the storage for the pointers to the L2 ISTs (used when\n+\t * freeing later).\n+\t */\n+\tvmi-\u003eh_lpi_l2_ists = kzalloc_objs(*vmi-\u003eh_lpi_l2_ists, l1_entries,\n+\t\t\t\t\t GFP_KERNEL_ACCOUNT);\n+\tif (!vmi-\u003eh_lpi_l2_ists)\n+\t\treturn -ENOMEM;\n+\n+\t/* Allocate the L2 IST for each L1 IST entry */\n+\tfor (index = 0; index \u003c l1_entries; ++index) {\n+\t\tl2ist = kzalloc(l2_size, GFP_KERNEL_ACCOUNT);\n+\t\tif (!l2ist) {\n+\t\t\twhile (--index \u003e= 0)\n+\t\t\t\tkfree(vmi-\u003eh_lpi_l2_ists[index]);\n+\n+\t\t\tkfree(vmi-\u003eh_lpi_l2_ists);\n+\t\t\tvmi-\u003eh_lpi_l2_ists = NULL;\n+\n+\t\t\treturn -ENOMEM;\n+\t\t}\n+\n+\t\t/*\n+\t\t * We are not doing on-demand allocation of the L2 ISTs, and are\n+\t\t * instead provisioning the whole IST up front. This means that\n+\t\t * we are able to mark the L2 ISTs as valid in the L1 ISTEs as\n+\t\t * the overall IST is not yet valid.\n+\t\t */\n+\t\tval = (virt_to_phys(l2ist) \u0026 GICV5_ISTL1E_L2_ADDR_MASK) |\n+\t\t GICV5_ISTL1E_VALID;\n+\t\tl1ist[index] = cpu_to_le64(val);\n+\n+\t\tvmi-\u003eh_lpi_l2_ists[index] = l2ist;\n+\n+\t\tvgic_v5_clean_inval(l2ist, l2_size);\n+\t}\n+\n+\t/* Handle CMOs for the whole L1 IST in one go */\n+\tvgic_v5_clean_inval(l1ist, l1_entries * sizeof(*l1ist));\n+\n+\treturn 0;\n+}\n+\n+/* Allocate a two-level IST - LPIs, only */\n+static int vgic_v5_alloc_two_level_lpi_ist(struct kvm *kvm, unsigned int id_bits,\n+\t\t\t\t\t unsigned int istsz, unsigned int l2sz)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vgic_v5_vm_info *vmi;\n+\tint ret;\n+\n+\t/*\n+\t * Allocate the L1 IST first, then all of the L2s. Everything\n+\t * is preallocated and we do no on-demand IST allocation. This\n+\t * is to avoid needing to track if and when the guest is doing\n+\t * on-demand IST allocation.\n+\t */\n+\tret = vgic_v5_alloc_l1_ist(kvm, id_bits, istsz, l2sz);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tret = vgic_v5_alloc_l2_ists(kvm, id_bits, istsz, l2sz);\n+\tif (ret) {\n+\t\t/* Free the L1 IST again */\n+\t\tvmi = xa_load(\u0026vm_info, vm_id);\n+\t\tkfree(vmi-\u003eh_lpi_ist);\n+\t\tvmi-\u003eh_lpi_ist = 0;\n+\n+\t\treturn ret;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static void vgic_v5_free_allocated_lpi_ist(struct vgic_v5_vm_info *vmi,\n+\t\t\t\t\t unsigned int id_bits,\n+\t\t\t\t\t unsigned int istsz,\n+\t\t\t\t\t unsigned int l2sz)\n+{\n+\tif (!vmi-\u003eh_lpi_ist_structure) {\n+\t\tkfree(vmi-\u003eh_lpi_ist);\n+\t\tvmi-\u003eh_lpi_ist = NULL;\n+\t\treturn;\n+\t}\n+\n+\tif (vmi-\u003eh_lpi_l2_ists) {\n+\t\tconst size_t n = max(5, id_bits - ((10 - istsz) + (2 * l2sz)) + 3 - 1);\n+\t\tconst int l1_entries = BIT(n + 1) / GICV5_IRS_ISTL1E_SIZE;\n+\t\tint index;\n+\n+\t\tfor (index = 0; index \u003c l1_entries; ++index)\n+\t\t\tkfree(vmi-\u003eh_lpi_l2_ists[index]);\n+\n+\t\tkfree(vmi-\u003eh_lpi_l2_ists);\n+\t\tvmi-\u003eh_lpi_l2_ists = NULL;\n+\t}\n+\n+\tkfree(vmi-\u003eh_lpi_ist);\n+\tvmi-\u003eh_lpi_ist = NULL;\n+}\n+\n+static void vgic_v5_free_allocated_spi_ist(struct kvm *kvm)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vgic_v5_vm_info *vmi;\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\treturn;\n+\n+\tkfree(vmi-\u003eh_spi_ist);\n+\tvmi-\u003eh_spi_ist = NULL;\n+}\n+\n+/*\n+ * Free a Linear IST. Can only happen once the VM is dead.\n+ */\n+static int vgic_v5_linear_ist_free(struct kvm *kvm, bool spi)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vmtl2_entry *vmte;\n+\tstruct vgic_v5_vm_info *vmi;\n+\tint section;\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\treturn -EINVAL;\n+\n+\tvmte = vgic_v5_get_l2_vmte(vm_id);\n+\tif (IS_ERR(vmte))\n+\t\treturn PTR_ERR(vmte);\n+\n+\tif (spi) {\n+\t\tsection = GICV5_VMTEL2_SPI_SECTION;\n+\t\tvgic_v5_free_allocated_spi_ist(kvm);\n+\t} else {\n+\t\tsection = GICV5_VMTEL2_LPI_SECTION;\n+\t\tvgic_v5_free_allocated_lpi_ist(vmi, 0, 0, 0);\n+\t}\n+\n+\t/* The VM should be dead here, so we can just zero the VMT section */\n+\tvmte-\u003eval[section] = cpu_to_le64(0);\n+\tvgic_v5_clean_inval(vmte, sizeof(*vmte));\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Free a Two-Level IST. Can only happen once the VM is dead.\n+ */\n+static int vgic_v5_two_level_ist_free(struct kvm *kvm, bool spi)\n+{\n+\tunsigned int id_bits, istsz, l2sz;\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vgic_v5_vm_info *vmi;\n+\tstruct vmtl2_entry *vmte;\n+\tu64 tmp;\n+\tint section;\n+\n+\t/* We don't create two-level SPI ISTs, so freeing is a bad idea! */\n+\tif (spi)\n+\t\treturn -EINVAL;\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\treturn -EINVAL;\n+\n+\tsection = GICV5_VMTEL2_LPI_SECTION;\n+\n+\tif (!vmi-\u003eh_lpi_ist_structure)\n+\t\treturn -EINVAL;\n+\n+\tvmte = vgic_v5_get_l2_vmte(vm_id);\n+\tif (IS_ERR(vmte))\n+\t\treturn PTR_ERR(vmte);\n+\n+\ttmp = le64_to_cpu(READ_ONCE(vmte-\u003eval[section]));\n+\n+\tid_bits = FIELD_GET(GICV5_VMTEL2E_IST_ID_BITS, tmp);\n+\tistsz = FIELD_GET(GICV5_VMTEL2E_IST_ISTSZ, tmp);\n+\tl2sz = FIELD_GET(GICV5_VMTEL2E_IST_L2SZ, tmp);\n+\n+\tvgic_v5_free_allocated_lpi_ist(vmi, id_bits, istsz, l2sz);\n+\n+\t/* The VM must be dead, so we can just zero the VMT section */\n+\tvmte-\u003eval[section] = cpu_to_le64(0);\n+\tvgic_v5_clean_inval(vmte, sizeof(*vmte));\n+\n+\treturn 0;\n+}\n+\n+/* Helper to determine ISTE size based on metadata requirements */\n+static unsigned int vgic_v5_ist_istsz(unsigned int id_bits)\n+{\n+\tif (!irs_caps.istmd)\n+\t\treturn GICV5_IRS_IST_CFGR_ISTSZ_4;\n+\n+\tif (id_bits \u003e= irs_caps.istmd_sz)\n+\t\treturn GICV5_IRS_IST_CFGR_ISTSZ_16;\n+\n+\treturn GICV5_IRS_IST_CFGR_ISTSZ_8;\n+}\n+\n+/*\n+ * Allocate an IST for SPIs.\n+ *\n+ * We don't anticipate a large number of SPIs being allocated. Therefore, we\n+ * always allocate a Linear IST for SPIs. This will need to be revisited should\n+ * that assumption no longer hold.\n+ */\n+int vgic_v5_spi_ist_alloc(struct kvm *kvm, unsigned int id_bits)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vgic_v5_vm_info *vmi;\n+\tphys_addr_t base_addr;\n+\tunsigned int istsz;\n+\tint ret;\n+\n+\tlockdep_assert_held(\u0026kvm-\u003earch.config_lock);\n+\n+\tistsz = vgic_v5_ist_istsz(id_bits);\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\treturn -EINVAL;\n+\n+\tif (vmi-\u003eh_spi_ist)\n+\t\treturn -EBUSY;\n+\n+\tret = vgic_v5_alloc_linear_ist(kvm, true, id_bits, istsz);\n+\tif (ret)\n+\t\treturn ret;\n+\tbase_addr = virt_to_phys(vmi-\u003eh_spi_ist);\n+\n+\tret = vgic_v5_vmte_assign_ist(kvm, base_addr, false, id_bits, 0, istsz,\n+\t\t\t\t true);\n+\tif (ret) {\n+\t\tvgic_v5_free_allocated_spi_ist(kvm);\n+\t\treturn ret;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Free the IST for SPIs. Should only happen once the VM is dead.\n+ */\n+static int vgic_v5_spi_ist_free(struct kvm *kvm)\n+{\n+\tlockdep_assert_held(\u0026kvm-\u003earch.config_lock);\n+\n+\treturn vgic_v5_linear_ist_free(kvm, true);\n+}\n+\n+int vgic_v5_lpi_ist_exists(struct kvm *kvm)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vgic_v5_vm_info *vmi;\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\treturn -ENXIO;\n+\n+\treturn !!vmi-\u003eh_lpi_ist;\n+}\n+\n+/*\n+ * Allocate an IST for LPIs.\n+ *\n+ * Unlike with SPIs, we anticipate that the guest will allocate a relatively\n+ * large number of LPIs. Therefore, while we support doing a linear LPI IST, it\n+ * is expected that LPI ISTs will be two-level.\n+ */\n+int vgic_v5_lpi_ist_alloc(struct kvm *kvm, unsigned int id_bits)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vgic_v5_vm_info *vmi;\n+\tunsigned int istsz, l2sz;\n+\tphys_addr_t phys_addr;\n+\tbool two_level;\n+\tint ret;\n+\n+\tlockdep_assert_held(\u0026kvm-\u003earch.config_lock);\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\treturn -EINVAL;\n+\n+\tif (vmi-\u003eh_lpi_ist)\n+\t\treturn -EBUSY;\n+\n+\tistsz = vgic_v5_ist_istsz(id_bits);\n+\tl2sz = gicv5_irs_l2_sz(irs_caps.ist_l2sz);\n+\n+\t/*\n+\t * Determine if we want to create a Linear or a Two-Level IST.\n+\t *\n+\t * A two-level IST is only required when a single L2 IST cannot cover\n+\t * the requested ID space. This depends on the L2 IST size selected for\n+\t * the IRS, not PAGE_SIZE. Using PAGE_SIZE here would switch to\n+\t * two-level too early when the selected L2 IST is larger than a page,\n+\t * and the allocation sizing arithmetic would underflow.\n+\t */\n+\ttwo_level = irs_caps.ist_levels \u0026\u0026\n+\t\tid_bits \u003e ((10 - istsz) + (2 * l2sz));\n+\n+\tif (!two_level)\n+\t\tret = vgic_v5_alloc_linear_ist(kvm, false /* LPIs, not SPIs */,\n+\t\t\t\t\t id_bits, istsz);\n+\telse\n+\t\tret = vgic_v5_alloc_two_level_lpi_ist(kvm, id_bits, istsz,\n+\t\t\t\t\t\t l2sz);\n+\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tphys_addr = virt_to_phys(vmi-\u003eh_lpi_ist);\n+\tret = vgic_v5_vmte_assign_ist(kvm, phys_addr, two_level, id_bits, l2sz,\n+\t\t\t\t istsz, false);\n+\tif (ret)\n+\t\tvgic_v5_free_allocated_lpi_ist(vmi, id_bits, istsz, l2sz);\n+\n+\treturn ret;\n+}\n+\n+/* Free the LPI IST again */\n+int vgic_v5_lpi_ist_free(struct kvm *kvm)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vgic_v5_vm_info *vmi;\n+\n+\tlockdep_assert_held(\u0026kvm-\u003earch.config_lock);\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\treturn -ENXIO;\n+\n+\tif (!vmi-\u003eh_lpi_ist_structure)\n+\t\treturn vgic_v5_linear_ist_free(kvm, false);\n+\telse\n+\t\treturn vgic_v5_two_level_ist_free(kvm, false);\n+}\n+\n+static struct vgic_v5_two_level_ist_shape\n+vgic_v5_two_level_ist_shape(const struct vgic_v5_ist_desc *ist)\n+{\n+\tstruct vgic_v5_two_level_ist_shape shape;\n+\tsize_t l2bits, n;\n+\n+\tl2bits = (10 - ist-\u003eistsz) + (2 * ist-\u003el2sz);\n+\tn = max(2, ist-\u003eid_bits - l2bits + 3 - 1);\n+\n+\tshape.l1_entries = BIT(n + 1) / GICV5_IRS_ISTL1E_SIZE;\n+\tshape.l2_entries = BIT(l2bits);\n+\n+\treturn shape;\n+}\n+\n+static int vgic_v5_read_vm_ist_desc(struct kvm *kvm, unsigned int section,\n+\t\t\t\t struct vgic_v5_ist_desc *ist)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vmtl2_entry *vmte;\n+\tu64 vmte_ist_section;\n+\n+\tvmte = vgic_v5_get_l2_vmte(vm_id);\n+\tif (IS_ERR(vmte))\n+\t\treturn PTR_ERR(vmte);\n+\n+\tvgic_v5_clean_inval(vmte, sizeof(*vmte));\n+\tvmte_ist_section = le64_to_cpu(READ_ONCE(vmte-\u003eval[section]));\n+\n+\tist-\u003eid_bits = FIELD_GET(GICV5_VMTEL2E_IST_ID_BITS, vmte_ist_section);\n+\tist-\u003eistsz = FIELD_GET(GICV5_VMTEL2E_IST_ISTSZ, vmte_ist_section);\n+\tist-\u003el2sz = FIELD_GET(GICV5_VMTEL2E_IST_L2SZ, vmte_ist_section);\n+\tist-\u003eiste_size = GICV5_ISTE_SIZE(ist-\u003eistsz);\n+\n+\treturn !!(vmte_ist_section \u0026 GICV5_VMTEL2E_IST_VALID);\n+}\n+\n+static int vgic_v5_get_spi_ist_desc(struct kvm *kvm, bool userspace_buf,\n+\t\t\t\t struct vgic_v5_ist_desc *ist)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tint ret;\n+\n+\tmemset(ist, 0, sizeof(*ist));\n+\n+\tist-\u003evmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!ist-\u003evmi)\n+\t\treturn -ENXIO;\n+\n+\tret = vgic_v5_read_vm_ist_desc(kvm, GICV5_VMTEL2_SPI_SECTION, ist);\n+\tif (ret \u003c 0)\n+\t\treturn ret;\n+\n+\tist-\u003ebase = ist-\u003evmi-\u003eh_spi_ist;\n+\n+\t/* We don't have SPIs, but userspace is trying to save/restore them. */\n+\tif (!ist-\u003ebase \u0026\u0026 userspace_buf)\n+\t\treturn -ENOENT;\n+\n+\t/* We have SPIs but userspace isn't trying to save/restore them. */\n+\tif (ist-\u003ebase \u0026\u0026 !userspace_buf)\n+\t\treturn -EINVAL;\n+\n+\t/* No SPIs and no userspace buffer: nothing to do. */\n+\tif (!ist-\u003ebase \u0026\u0026 !userspace_buf)\n+\t\treturn 0;\n+\n+\tist-\u003epresent = true;\n+\treturn 0;\n+}\n+\n+static int vgic_v5_get_lpi_ist_desc(struct kvm *kvm,\n+\t\t\t\t struct vgic_v5_ist_desc *ist)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tbool guest_valid, host_valid;\n+\tint ret;\n+\n+\tmemset(ist, 0, sizeof(*ist));\n+\n+\tist-\u003evmi = xa_load(\u0026vm_info, vm_id);\n+\tif (WARN_ON_ONCE(!ist-\u003evmi))\n+\t\treturn -ENXIO;\n+\n+\tret = vgic_v5_read_vm_ist_desc(kvm, GICV5_VMTEL2_LPI_SECTION, ist);\n+\tif (ret \u003c 0)\n+\t\treturn ret;\n+\n+\thost_valid = ret;\n+\tguest_valid = kvm-\u003earch.vgic.vgic_v5_irs_data-\u003eist_baser.valid;\n+\tist-\u003ebase = ist-\u003evmi-\u003eh_lpi_ist;\n+\n+\t/* If there is no IST to save/restore, return without error. */\n+\tif (!guest_valid \u0026\u0026 !host_valid \u0026\u0026 !ist-\u003ebase)\n+\t\treturn 0;\n+\n+\t/* Mismatched combination of valid state */\n+\tif (!guest_valid || !host_valid || !ist-\u003ebase)\n+\t\treturn -ENXIO;\n+\n+\tif (ist-\u003evmi-\u003eh_lpi_ist_structure \u0026\u0026 !ist-\u003evmi-\u003eh_lpi_l2_ists)\n+\t\treturn -ENXIO;\n+\n+\tist-\u003epresent = true;\n+\treturn 0;\n+}\n+\n+/*\n+ * Save a linear host IST to userspace memory.\n+ *\n+ * Only the architected 32-bit ISTE state is stored. Metadata is skipped when\n+ * striding through the host IST.\n+ */\n+static int vgic_v5_save_linear_ist(const struct vgic_v5_ist_desc *ist,\n+\t\t\t\t u32 __user *uaddr, size_t nr_entries)\n+{\n+\t__le32 h_iste;\n+\tsize_t index;\n+\tint ret;\n+\n+\tvgic_v5_clean_inval(ist-\u003ebase,\n+\t\t\t GICV5_LINEAR_IST_SIZE(ist-\u003eid_bits, ist-\u003eistsz));\n+\n+\tfor (index = 0; index \u003c nr_entries; index++) {\n+\t\t__le32 *h_iste_addr = ist-\u003ebase + index * ist-\u003eiste_size;\n+\n+\t\th_iste = READ_ONCE(*h_iste_addr);\n+\t\tret = put_user(h_iste, uaddr);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\n+\t\tuaddr++;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Save a two-level host IST to userspace memory.\n+ *\n+ * Only the architected 32-bit ISTE state is stored. Metadata is skipped when\n+ * striding through the host IST.\n+ */\n+static int vgic_v5_save_two_level_ist(const struct vgic_v5_ist_desc *ist,\n+\t\t\t\t u32 __user *uaddr)\n+{\n+\tstruct vgic_v5_two_level_ist_shape shape;\n+\tsize_t h_l1_index, h_l2_index;\n+\tvoid *h_l2_ist_base;\n+\t__le32 h_iste;\n+\tint ret;\n+\n+\tshape = vgic_v5_two_level_ist_shape(ist);\n+\n+\tvgic_v5_clean_inval(ist-\u003ebase,\n+\t\t\t shape.l1_entries * sizeof(*ist-\u003evmi-\u003eh_lpi_ist));\n+\n+\tfor (h_l1_index = 0; h_l1_index \u003c shape.l1_entries; h_l1_index++) {\n+\t\tu64 l1_iste;\n+\n+\t\t/*\n+\t\t * Host L2 ISTs are preallocated. Any invalid L1 entry means the\n+\t\t * host IST state is inconsistent.\n+\t\t */\n+\t\tl1_iste = le64_to_cpu(READ_ONCE(ist-\u003evmi-\u003eh_lpi_ist[h_l1_index]));\n+\t\tif (!FIELD_GET(GICV5_ISTL1E_VALID, l1_iste))\n+\t\t\treturn -ENXIO;\n+\n+\t\th_l2_ist_base = ist-\u003evmi-\u003eh_lpi_l2_ists[h_l1_index];\n+\t\tif (!h_l2_ist_base)\n+\t\t\treturn -ENXIO;\n+\n+\t\tvgic_v5_clean_inval(h_l2_ist_base,\n+\t\t\t\t shape.l2_entries * ist-\u003eiste_size);\n+\n+\t\tfor (h_l2_index = 0; h_l2_index \u003c shape.l2_entries; h_l2_index++) {\n+\t\t\th_iste = *(__le32 *)(h_l2_ist_base +\n+\t\t\t\t\t h_l2_index * ist-\u003eiste_size);\n+\n+\t\t\tret = put_user(h_iste, uaddr);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\n+\t\t\tuaddr++;\n+\t\t}\n+\t}\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Save the SPI IST to userspace-provided memory.\n+ */\n+int vgic_v5_save_spi_ist(struct kvm *kvm, struct kvm_vgic_v5_ist *ist_attr)\n+{\n+\tstruct vgic_v5_ist_desc ist;\n+\tu32 __user *uaddr;\n+\tint ret;\n+\n+\tret = vgic_v5_get_spi_ist_desc(kvm, !!ist_attr-\u003espi_ist_addr, \u0026ist);\n+\tif (ret || !ist.present)\n+\t\treturn ret;\n+\n+\tuaddr = (u32 __user *)(unsigned long)ist_attr-\u003espi_ist_addr;\n+\n+\t/* The host SPI IST is always linear. */\n+\treturn vgic_v5_save_linear_ist(\u0026ist, uaddr,\n+\t\t\t\t kvm-\u003earch.vgic.nr_spis);\n+}\n+\n+/*\n+ * Save the LPI IST to userspace memory.\n+ *\n+ * The LPI IST may be linear or two-level, so host iteration depends on the\n+ * allocated host shape.\n+ */\n+int vgic_v5_save_lpi_ist(struct kvm *kvm, struct kvm_vgic_v5_ist *ist_attr)\n+{\n+\tstruct vgic_v5_ist_desc ist;\n+\tu32 __user *uaddr;\n+\tint ret;\n+\n+\tret = vgic_v5_get_lpi_ist_desc(kvm, \u0026ist);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tif (!ist.present)\n+\t\treturn 0;\n+\n+\tuaddr = (u32 __user *)(unsigned long)ist_attr-\u003elpi_ist_addr;\n+\n+\tif (!ist.vmi-\u003eh_lpi_ist_structure)\n+\t\treturn vgic_v5_save_linear_ist(\u0026ist, uaddr,\n+\t\t\t\t\t BIT(ist.id_bits));\n+\n+\treturn vgic_v5_save_two_level_ist(\u0026ist, uaddr);\n+}\n+\n+/*\n+ * Track any SPIs and LPIs that were marked as pending at the point where the\n+ * IST was restored.\n+ *\n+ * Restored pending state is cleared from the host ISTE and replayed with VDPEND\n+ * before the VM first runs.\n+ */\n+static int vgic_v5_track_pending_irq(struct list_head *pending_irqs, u32 intid,\n+\t\t\t\t u32 type)\n+{\n+\tstruct vgic_v5_pending_irq *pirq;\n+\n+\tpirq = kzalloc_obj(*pirq, GFP_KERNEL);\n+\tif (!pirq)\n+\t\treturn -ENOMEM;\n+\n+\t/* Encode the interrupt as a GICv5 IntID. */\n+\tpirq-\u003eirq = FIELD_PREP(GICV5_HWIRQ_TYPE, type) |\n+\t\t FIELD_PREP(GICV5_HWIRQ_ID, intid);\n+\n+\tINIT_LIST_HEAD(\u0026pirq-\u003enext);\n+\tlist_add_tail(\u0026pirq-\u003enext, pending_irqs);\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Process and sanitise each restored ISTE.\n+ *\n+ * HWU is for hardware use and must not survive migration. Pending state is\n+ * tracked, cleared from the ISTE, and replayed before the VM first runs.\n+ */\n+static int vgic_v5_process_iste(__le32 *iste, struct list_head *pending_irqs,\n+\t\t\t\tu32 intid, u32 type)\n+{\n+\tu32 iste_data = le32_to_cpu(READ_ONCE(*iste));\n+\tint ret;\n+\n+\t/* Pending state is replayed later with VDPEND. */\n+\tif (iste_data \u0026 GICV5_ISTL2E_PENDING) {\n+\t\tret = vgic_v5_track_pending_irq(pending_irqs, intid, type);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\t}\n+\n+\tiste_data \u0026= ~GICV5_ISTL2E_PENDING;\n+\tiste_data \u0026= ~GICV5_ISTL2E_HWU;\n+\n+\tWRITE_ONCE(*iste, cpu_to_le32(iste_data));\n+\n+\treturn 0;\n+}\n+\n+static void vgic_v5_restore_spi_config(struct kvm *kvm, __le32 iste, u32 spi)\n+{\n+\tu32 iste_data = le32_to_cpu(iste);\n+\tbool pending = iste_data \u0026 GICV5_ISTL2E_PENDING;\n+\tstruct vgic_irq *irq;\n+\tunsigned long flags;\n+\n+\tirq = vgic_get_irq(kvm, vgic_v5_make_spi(spi));\n+\tif (WARN_ON_ONCE(!irq))\n+\t\treturn;\n+\n+\traw_spin_lock_irqsave(\u0026irq-\u003eirq_lock, flags);\n+\n+\tif (iste_data \u0026 GICV5_ISTL2E_HM)\n+\t\tirq-\u003econfig = VGIC_CONFIG_LEVEL;\n+\telse\n+\t\tirq-\u003econfig = VGIC_CONFIG_EDGE;\n+\n+\tif (irq-\u003econfig == VGIC_CONFIG_EDGE)\n+\t\tirq-\u003epending_latch = pending;\n+\telse if (pending)\n+\t\tirq-\u003epending_latch = true;\n+\telse if (!irq-\u003eactive)\n+\t\tirq-\u003epending_latch = false;\n+\n+\traw_spin_unlock_irqrestore(\u0026irq-\u003eirq_lock, flags);\n+\tvgic_put_irq(kvm, irq);\n+}\n+\n+static int vgic_v5_restore_ist_entry(struct kvm *kvm,\n+\t\t\t\t const struct vgic_v5_ist_desc *ist,\n+\t\t\t\t void *h_iste_addr, __le32 h_iste,\n+\t\t\t\t u32 intid, u32 intid_type)\n+{\n+\t__le32 raw_iste = h_iste;\n+\tint ret;\n+\n+\t/*\n+\t * Sanitise the IST, clearing HWU \u0026 pending fields. Pending state is\n+\t * later replayed via GIC VDPEND.\n+\t */\n+\tret = vgic_v5_process_iste(\u0026h_iste, \u0026ist-\u003evmi-\u003epending_irqs,\n+\t\t\t\t intid, intid_type);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tif (intid_type == GICV5_HWIRQ_TYPE_SPI)\n+\t\tvgic_v5_restore_spi_config(kvm, raw_iste, intid);\n+\n+\t/*\n+\t * Zero the full ISTE (incl metadata), and write back the non-metadata\n+\t * region, only.\n+\t */\n+\tmemset(h_iste_addr, 0, ist-\u003eiste_size);\n+\tWRITE_ONCE(*(__le32 *)h_iste_addr, h_iste);\n+\tvgic_v5_clean_inval(h_iste_addr, ist-\u003eiste_size);\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Restore a userspace IST image to a linear host IST.\n+ *\n+ * The userspace IST image is a linear array of 32-bit ISTEs.\n+ */\n+static int vgic_v5_restore_linear_ist(struct kvm *kvm,\n+\t\t\t\t const struct vgic_v5_ist_desc *ist,\n+\t\t\t\t u32 __user *uaddr, size_t nr_entries,\n+\t\t\t\t u32 intid_type)\n+{\n+\t__le32 h_iste;\n+\tsize_t index;\n+\tint ret;\n+\n+\tfor (index = 0; index \u003c nr_entries; index++) {\n+\t\tvoid *h_iste_addr = ist-\u003ebase + index * ist-\u003eiste_size;\n+\n+\t\tret = get_user(h_iste, uaddr);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\n+\t\tret = vgic_v5_restore_ist_entry(kvm, ist, h_iste_addr,\n+\t\t\t\t\th_iste, index, intid_type);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\n+\t\tuaddr++;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Restore a userspace IST image to a two-level host IST.\n+ *\n+ * The userspace IST image is a linear array of 32-bit ISTEs.\n+ */\n+static int vgic_v5_restore_two_level_ist(struct kvm *kvm,\n+\t\t\t\t\t const struct vgic_v5_ist_desc *ist,\n+\t\t\t\t\t u32 __user *uaddr, u32 intid_type)\n+{\n+\tstruct vgic_v5_two_level_ist_shape shape;\n+\tsize_t h_l1_index, h_l2_index;\n+\tvoid *h_l2_ist_base;\n+\t__le32 h_iste;\n+\tint ret;\n+\n+\tshape = vgic_v5_two_level_ist_shape(ist);\n+\n+\tvgic_v5_clean_inval(ist-\u003evmi-\u003eh_lpi_ist,\n+\t\t\t shape.l1_entries * sizeof(*ist-\u003evmi-\u003eh_lpi_ist));\n+\n+\tfor (h_l1_index = 0; h_l1_index \u003c shape.l1_entries; ++h_l1_index) {\n+\t\tu64 l1_iste;\n+\n+\t\t/*\n+\t\t * Host L2 ISTs are preallocated. Any invalid L1 entry means the\n+\t\t * host IST state is inconsistent.\n+\t\t */\n+\t\tl1_iste = le64_to_cpu(READ_ONCE(ist-\u003evmi-\u003eh_lpi_ist[h_l1_index]));\n+\t\tif (!FIELD_GET(GICV5_ISTL1E_VALID, l1_iste))\n+\t\t\treturn -ENXIO;\n+\n+\t\th_l2_ist_base = ist-\u003evmi-\u003eh_lpi_l2_ists[h_l1_index];\n+\t\tif (!h_l2_ist_base)\n+\t\t\treturn -ENXIO;\n+\n+\t\tfor (h_l2_index = 0; h_l2_index \u003c shape.l2_entries; h_l2_index++) {\n+\t\t\tvoid *h_iste_addr = h_l2_ist_base +\n+\t\t\t\t\t h_l2_index * ist-\u003eiste_size;\n+\t\t\tu32 intid = h_l1_index * shape.l2_entries + h_l2_index;\n+\n+\t\t\tret = get_user(h_iste, uaddr);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\n+\t\t\tret = vgic_v5_restore_ist_entry(kvm, ist, h_iste_addr,\n+\t\t\t\t\t\t\th_iste, intid,\n+\t\t\t\t\t\t\tintid_type);\n+\t\t\tif (ret)\n+\t\t\t\treturn ret;\n+\n+\t\t\tuaddr++;\n+\t\t}\n+\t}\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Restore the SPI IST from userspace-provided buffer to the host-allocated IST.\n+ */\n+int vgic_v5_restore_spi_ist(struct kvm *kvm, struct kvm_vgic_v5_ist *ist_attr)\n+{\n+\tu32 __user *uaddr;\n+\tstruct vgic_v5_ist_desc ist;\n+\tint ret;\n+\n+\tret = vgic_v5_get_spi_ist_desc(kvm, !!ist_attr-\u003espi_ist_addr, \u0026ist);\n+\tif (ret || !ist.present)\n+\t\treturn ret;\n+\n+\tuaddr = (u32 __user *)(unsigned long)ist_attr-\u003espi_ist_addr;\n+\n+\t/* The host SPI IST is always linear. */\n+\treturn vgic_v5_restore_linear_ist(kvm, \u0026ist, uaddr,\n+\t\t\t\t\t kvm-\u003earch.vgic.nr_spis,\n+\t\t\t\t\t GICV5_HWIRQ_TYPE_SPI);\n+}\n+\n+/*\n+ * Restore the LPI IST from userspace memory to the host-allocated LPI IST.\n+ *\n+ * The host LPI IST may be linear or two-level, so host iteration depends on the\n+ * host IST's shape.\n+ */\n+int vgic_v5_restore_lpi_ist(struct kvm *kvm, struct kvm_vgic_v5_ist *ist_attr)\n+{\n+\tu32 __user *uaddr;\n+\tstruct vgic_v5_ist_desc ist;\n+\tint ret;\n+\n+\tret = vgic_v5_get_lpi_ist_desc(kvm, \u0026ist);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tif (!ist.present)\n+\t\treturn 0;\n+\n+\tuaddr = (u32 __user *)(unsigned long)ist_attr-\u003elpi_ist_addr;\n+\n+\tif (!ist.vmi-\u003eh_lpi_ist_structure)\n+\t\treturn vgic_v5_restore_linear_ist(kvm, \u0026ist, uaddr,\n+\t\t\t\t\t\t BIT(ist.id_bits),\n+\t\t\t\t\t\t GICV5_HWIRQ_TYPE_LPI);\n+\n+\treturn vgic_v5_restore_two_level_ist(kvm, \u0026ist, uaddr,\n+\t\t\t\t\t GICV5_HWIRQ_TYPE_LPI);\n+}\n+\n+/*\n+ * Process the pending IRQs removing them from the list and optionally injecting\n+ * them.\n+ */\n+static int vgic_v5_process_pending_irqs(struct kvm *kvm, bool inject)\n+{\n+\tu32 vm_id = vgic_v5_vm_id(kvm);\n+\tstruct vgic_v5_vm_info *vmi;\n+\n+\tlockdep_assert_held(\u0026kvm-\u003earch.config_lock);\n+\n+\tvmi = xa_load(\u0026vm_info, vm_id);\n+\tif (!vmi)\n+\t\treturn -ENXIO;\n+\n+\tvgic_v5_drain_pending_irqs(kvm, vmi, inject);\n+\n+\treturn 0;\n+}\n+\n+/* Replay pending state that was cleared while restoring guest IST state. */\n+int vgic_v5_restore_pending_irqs(struct kvm *kvm)\n+{\n+\treturn vgic_v5_process_pending_irqs(kvm, true);\n+}\n+\n+/* Drop pending state collected by a failed IST restore. */\n+void vgic_v5_discard_pending_irqs(struct kvm *kvm)\n+{\n+\tvgic_v5_process_pending_irqs(kvm, false);\n+}\ndiff --git a/arch/arm64/kvm/vgic/vgic-v5-tables.h b/arch/arm64/kvm/vgic/vgic-v5-tables.h\nnew file mode 100644\nindex 00000000000000..9a272b8cc6c87e\n--- /dev/null\n+++ b/arch/arm64/kvm/vgic/vgic-v5-tables.h\n@@ -0,0 +1,116 @@\n+/* SPDX-License-Identifier: GPL-2.0-only */\n+/*\n+ * Copyright (C) 2025, 2026 Arm Ltd.\n+ */\n+\n+#ifndef __KVM_ARM_VGICV5_TABLES_H__\n+#define __KVM_ARM_VGICV5_TABLES_H__\n+\n+#include \u003clinux/idr.h\u003e\n+#include \u003clinux/irqchip/arm-gic-v5.h\u003e\n+#include \u003clinux/list.h\u003e\n+\n+/* Level 1 Virtual Machine Table Entry */\n+typedef __le64 vmtl1_entry;\n+\n+/* Level 2 Virtual Machine Table Entry */\n+struct vmtl2_entry {\n+\t__le64 val[4];\n+};\n+\n+/* Virtual PE Table Entry */\n+typedef __le64 vpe_entry;\n+\n+struct vgic_v5_vm_info {\n+\tvoid __iomem\t\t*vmd_base;\n+\tvpe_entry __iomem\t*vpet_base;\n+\tvoid __iomem\t\t**vped_ptrs;\n+\tu8\t\t\tvpe_id_bits;\n+\n+\t/*\n+\t * Both the LPI and SPI ISTs are allocated by the hypervisor. While it\n+\t * would be possible to track and access them by iterating over the ISTs\n+\t * themselves, it makes more sense to store pointers to the ISTs.\n+\t *\n+\t * The LPI IST can either be two-level or linear. Hence, we keep track\n+\t * of the structure. If it is two-level, we retain pointers to the L1\n+\t * IST and to each L2 IST array. If it is linear, we just store the base\n+\t * address of the IST array.\n+\t *\n+\t * The SPI IST is linear, and therefore we just store the base address\n+\t * of the SPI IST array.\n+\t */\n+\tbool\t\t\th_lpi_ist_structure;\n+\t__le64\t\t\t*h_lpi_ist;\n+\t__le64\t\t\t**h_lpi_l2_ists;\n+\t__le64\t\t\t*h_spi_ist;\n+\n+\t/* Tracking of pending interrupts as part of IST restore */\n+\tstruct list_head\tpending_irqs;\n+};\n+\n+struct vgic_v5_vmt {\n+\tunion {\n+\t\tstruct {\n+\t\t\tstruct vmtl2_entry *vmt_base;\n+\t\t\tunsigned int num_ents;\n+\t\t} linear;\n+\t\tstruct {\n+\t\t\tvmtl1_entry *vmt_base;\n+\t\t\tstruct vmtl2_entry **l2ptrs;\n+\t\t\tunsigned int num_l1_ents;\n+\t\t} l2;\n+\t};\n+\tbool\t\ttwo_level;\n+\tunsigned int\tnum_entries;\n+\tunsigned int\tmax_vpes;\n+\tsize_t\t\tvmd_size;\n+\tsize_t\t\tvped_size;\n+\tstruct ida\tvm_id_ida;\n+};\n+\n+static inline u32 vgic_v5_vm_id(struct kvm *kvm)\n+{\n+\treturn kvm-\u003earch.vgic.gicv5_vm.vm_id;\n+}\n+\n+static inline u16 vgic_v5_vpe_id(struct kvm_vcpu *vcpu)\n+{\n+\treturn vcpu-\u003evcpu_idx;\n+}\n+\n+static inline int vgic_v5_vpe_db(struct kvm_vcpu *vcpu)\n+{\n+\treturn vcpu-\u003earch.vgic_cpu.vgic_v5.gicv5_vpe.db;\n+}\n+\n+int vgic_v5_vmt_allocate(unsigned int max_vpes);\n+int vgic_v5_vmt_free(void);\n+phys_addr_t vgic_v5_get_vmt_base(void);\n+\n+int vgic_v5_allocate_vm_id(struct kvm *kvm);\n+void vgic_v5_release_vm_id(struct kvm *kvm);\n+\n+int vgic_v5_vmte_init(struct kvm *kvm);\n+int vgic_v5_vmte_release(struct kvm *kvm);\n+u8 vgic_v5_vmte_vpe_id_bits(struct kvm_vcpu *vcpu);\n+int vgic_v5_vmte_alloc_vpe(struct kvm_vcpu *vcpu);\n+int vgic_v5_vmte_free_vpe(struct kvm_vcpu *vcpu);\n+\n+int vgic_v5_spi_ist_alloc(struct kvm *kvm, unsigned int id_bits);\n+int vgic_v5_lpi_ist_exists(struct kvm *kvm);\n+int vgic_v5_lpi_ist_alloc(struct kvm *kvm, unsigned int id_bits);\n+int vgic_v5_lpi_ist_free(struct kvm *kvm);\n+\n+int vgic_v5_save_spi_ist(struct kvm *kvm,\n+\t\t\t struct kvm_vgic_v5_ist *ist_attr);\n+int vgic_v5_save_lpi_ist(struct kvm *kvm,\n+\t\t\t struct kvm_vgic_v5_ist *ist_attr);\n+int vgic_v5_restore_spi_ist(struct kvm *kvm,\n+\t\t\t struct kvm_vgic_v5_ist *ist_attr);\n+int vgic_v5_restore_lpi_ist(struct kvm *kvm,\n+\t\t\t struct kvm_vgic_v5_ist *ist_attr);\n+int vgic_v5_restore_pending_irqs(struct kvm *kvm);\n+void vgic_v5_discard_pending_irqs(struct kvm *kvm);\n+\n+#endif\ndiff --git a/arch/arm64/kvm/vgic/vgic-v5.c b/arch/arm64/kvm/vgic/vgic-v5.c\nindex d4789ff3e74026..153a8fcd4db7c1 100644\n--- a/arch/arm64/kvm/vgic/vgic-v5.c\n+++ b/arch/arm64/kvm/vgic/vgic-v5.c\n@@ -7,10 +7,18 @@\n \n #include \u003clinux/bitops.h\u003e\n #include \u003clinux/irqchip/arm-vgic-info.h\u003e\n+#include \u003clinux/irqdomain.h\u003e\n+#include \u003clinux/kvm_host.h\u003e\n+#include \u003clinux/uaccess.h\u003e\n \n #include \"vgic.h\"\n+#include \"vgic-v5-tables.h\"\n \n #define ppi_caps\tkvm_vgic_global_state.vgic_v5_ppi_caps\n+#define irs_caps\tkvm_vgic_global_state.vgic_v5_irs_caps\n+\n+static int vgic_v5_irs_assign_vmt(bool two_level, u8 vm_id_bits, phys_addr_t vmt_base);\n+static int vgic_v5_irs_clear_vmt(void);\n \n /*\n * Not all PPIs are guaranteed to be implemented for GICv5. Deterermine which\n@@ -34,6 +42,66 @@ static void vgic_v5_get_implemented_ppis(void)\n \t__assign_bit(GICV5_ARCH_PPI_PMUIRQ, ppi_caps.impl_ppi_mask, system_supports_pmuv3());\n }\n \n+/*\n+ * The IRS MMIO interface is shared between all VMs, so make sure we don't do\n+ * anything stupid!\n+ */\n+static DEFINE_RAW_SPINLOCK(global_irs_lock);\n+\n+static u32 irs_readl_relaxed(const u32 reg_offset)\n+{\n+\treturn readl_relaxed(irs_caps.irs_base + reg_offset);\n+}\n+\n+static void irs_writel_relaxed(const u32 val, const u32 reg_offset)\n+{\n+\twritel_relaxed(val, irs_caps.irs_base + reg_offset);\n+}\n+\n+static u64 irs_readq_relaxed(const u32 reg_offset)\n+{\n+\treturn readq_relaxed(irs_caps.irs_base + reg_offset);\n+}\n+\n+static void irs_writeq_relaxed(const u64 val, const u32 reg_offset)\n+{\n+\twriteq_relaxed(val, irs_caps.irs_base + reg_offset);\n+}\n+\n+static void vgic_v5_irs_extract_vm_caps(const struct gic_kvm_info *info)\n+{\n+\tu64 idr;\n+\n+\tirs_caps.irs_base = info-\u003egicv5_irs.base;\n+\tirs_caps.non_coherent = info-\u003egicv5_irs.non_coherent;\n+\n+\tidr = irs_readl_relaxed(GICV5_IRS_IDR2);\n+\n+\t/* We skip the LPI field as it only applies to physical LPIs */\n+\tirs_caps.ist_id_bits = FIELD_GET(GICV5_IRS_IDR2_ID_BITS, idr);\n+\tirs_caps.min_lpi_id_bits = FIELD_GET(GICV5_IRS_IDR2_MIN_LPI_ID_BITS, idr);\n+\tirs_caps.ist_levels = (idr \u0026 GICV5_IRS_IDR2_IST_LEVELS);\n+\tirs_caps.ist_l2sz = FIELD_GET(GICV5_IRS_IDR2_IST_L2SZ, idr);\n+\tirs_caps.istmd = (idr \u0026 GICV5_IRS_IDR2_ISTMD);\n+\tirs_caps.istmd_sz = FIELD_GET(GICV5_IRS_IDR2_ISTMD_SZ, idr);\n+\n+\tidr = irs_readl_relaxed(GICV5_IRS_IDR3);\n+\n+\tirs_caps.max_vms = BIT(FIELD_GET(GICV5_IRS_IDR3_VM_ID_BITS, idr));\n+\tirs_caps.two_level_vmt_support = (idr \u0026 GICV5_IRS_IDR3_VMT_LEVELS);\n+\n+\tif (idr \u0026 GICV5_IRS_IDR3_VMD)\n+\t\tirs_caps.vmd_size = BIT(FIELD_GET(GICV5_IRS_IDR3_VMD_SZ, idr));\n+\telse\n+\t\tirs_caps.vmd_size = 0;\n+\n+\tidr = irs_readl_relaxed(GICV5_IRS_IDR4);\n+\n+\tirs_caps.vped_size = BIT(FIELD_GET(GICV5_IRS_IDR4_VPED_SZ, idr));\n+\t/* Field stores VPE_ID_BITS - 1 */\n+\tirs_caps.max_vpes = BIT(FIELD_GET(GICV5_IRS_IDR4_VPE_ID_BITS, idr) + 1);\n+}\n+\n /*\n * Probe for a vGICv5 compatible interrupt controller, returning 0 on success.\n */\n@@ -44,6 +112,8 @@ int vgic_v5_probe(const struct gic_kvm_info *info)\n \tint ret;\n \n \tkvm_vgic_global_state.type = VGIC_V5;\n+\tkvm_vgic_global_state.max_gic_vcpus = 0;\n+\tkvm_vgic_global_state.max_gicv5_vcpus = 0;\n \n \tkvm_vgic_global_state.vcpu_base = 0;\n \tkvm_vgic_global_state.vctrl_base = NULL;\n@@ -61,17 +131,60 @@ int vgic_v5_probe(const struct gic_kvm_info *info)\n \t\tgoto skip_v5;\n \t}\n \n-\tkvm_vgic_global_state.max_gic_vcpus = VGIC_V5_MAX_CPUS;\n-\n+\tvgic_v5_irs_extract_vm_caps(info);\n \tvgic_v5_get_implemented_ppis();\n \n+\t/*\n+\t * Even if the HW supports more per-VM vCPUs, artificially cap as we\n+\t * can't use them all.\n+\t */\n+\tkvm_vgic_global_state.max_gicv5_vcpus = min(irs_caps.max_vpes,\n+\t\t\t\t\t\t VGIC_V5_MAX_CPUS);\n+\n+\t/*\n+\t * GICv5 requires a set of tables to be allocated in order to manage\n+\t * VMs. We allocate them in advance here, which alas means that we\n+\t * already have to make a decisions regarding the maximum number of VMs\n+\t * we want to run. For now, we match the maximum number offered by the\n+\t * hardware, but this might not be a wise choice in the long term.\n+\t */\n+\tret = vgic_v5_vmt_allocate(kvm_vgic_global_state.max_gicv5_vcpus);\n+\tif (ret) {\n+\t\tkvm_err(\"Failed to allocate the GICv5 VM tables; no GICv5 support\\n\");\n+\t\treturn -ENODEV;\n+\t}\n+\n+\t/*\n+\t * We've now allocated the VM table, but the host's IRS doesn't know\n+\t * about it yet. Provide the base address of the VMT to the IRS, as well\n+\t * as the number of ID bits that it covers and the structure used\n+\t * (linear/two-level).\n+\t */\n+\tret = vgic_v5_irs_assign_vmt(irs_caps.two_level_vmt_support,\n+\t\t\t\t ilog2(irs_caps.max_vms),\n+\t\t\t\t vgic_v5_get_vmt_base());\n+\tif (ret) {\n+\t\tkvm_err(\"Failed to assign the GICv5 VM tables to the IRS; no GICv5 support\\n\");\n+\t\tif (!vgic_v5_irs_clear_vmt())\n+\t\t\tvgic_v5_vmt_free();\n+\t\treturn -ENODEV;\n+\t}\n+\n \tret = kvm_register_vgic_device(KVM_DEV_TYPE_ARM_VGIC_V5);\n \tif (ret) {\n \t\tkvm_err(\"Cannot register GICv5 KVM device.\\n\");\n-\t\tgoto skip_v5;\n+\t\t/*\n+\t\t * Don't free the VMT itself if the hardware still has a valid\n+\t\t * pointer to it.\n+\t\t */\n+\t\tif (!vgic_v5_irs_clear_vmt())\n+\t\t\tvgic_v5_vmt_free();\n+\t\treturn -ENODEV;\n \t}\n \n \tv5_registered = true;\n+\tkvm_vgic_global_state.max_gic_vcpus =\n+\t\tkvm_vgic_global_state.max_gicv5_vcpus;\n \tkvm_info(\"GCIE system register CPU interface\\n\");\n \n skip_v5:\n@@ -95,12 +208,12 @@ int vgic_v5_probe(const struct gic_kvm_info *info)\n \tret = kvm_register_vgic_device(KVM_DEV_TYPE_ARM_VGIC_V3);\n \tif (ret) {\n \t\tkvm_err(\"Cannot register GICv3-legacy KVM device.\\n\");\n-\t\treturn ret;\n+\t\t/* vGICv5 should still work */\n+\t\treturn v5_registered ? 0 : ret;\n \t}\n \n-\t/* We potentially limit the max VCPUs further than we need to here */\n-\tkvm_vgic_global_state.max_gic_vcpus = min(VGIC_V3_MAX_CPUS,\n-\t\t\t\t\t\t VGIC_V5_MAX_CPUS);\n+\tkvm_vgic_global_state.max_gic_vcpus = max(kvm_vgic_global_state.max_gic_vcpus,\n+\t\t\t\t\t\t VGIC_V3_MAX_CPUS);\n \n \tstatic_branch_enable(\u0026kvm_vgic_global_state.gicv3_cpuif);\n \tkvm_info(\"GCIE legacy system register CPU interface\\n\");\n@@ -110,6 +223,403 @@ int vgic_v5_probe(const struct gic_kvm_info *info)\n \treturn 0;\n }\n \n+/*\n+ * Wait for completion of a change in any of IRS_VMT_BASER, IRS_VMAP_L2_VMTR,\n+ * IRS_VMAP_VMR, IRS_VMAP_VPER, IRS_VMAP_VISTR, IRS_VMAP_L2_VISTR.\n+ */\n+static int vgic_v5_irs_wait_for_vm_op(void)\n+{\n+\treturn gicv5_wait_for_op_atomic(irs_caps.irs_base,\n+\t\t\t\t\tGICV5_IRS_VMT_STATUSR,\n+\t\t\t\t\tGICV5_IRS_VMT_STATUSR_IDLE,\n+\t\t\t\t\tNULL);\n+}\n+\n+/*\n+ * Wait for completion of a change in any of IRS_VPE_SELR, IRS_VPE_DBR,\n+ * IRS_VPE_CR0.\n+ */\n+static int vgic_v5_irs_wait_for_vpe_op(void)\n+{\n+\treturn gicv5_wait_for_op_atomic(irs_caps.irs_base,\n+\t\t\t\t\tGICV5_IRS_VPE_STATUSR,\n+\t\t\t\t\tGICV5_IRS_VPE_STATUSR_IDLE,\n+\t\t\t\t\tNULL);\n+}\n+\n+/*\n+ * Wait for a write to IRS_SAVE_VMR to complete.\n+ */\n+static int vgic_v5_irs_wait_for_save_vm_op(u32 *statusr)\n+{\n+\treturn gicv5_wait_for_op_atomic(irs_caps.irs_base,\n+\t\t\t\t\tGICV5_IRS_SAVE_VM_STATUSR,\n+\t\t\t\t\tGICV5_IRS_SAVE_VM_STATUSR_IDLE,\n+\t\t\t\t\tstatusr);\n+}\n+\n+static int vgic_v5_irs_write_vm_mmio_reg(u64 val, u32 offset)\n+{\n+\tint ret;\n+\n+\tguard(raw_spinlock_irqsave)(\u0026global_irs_lock);\n+\n+\t/* Make sure that we are idle to begin with */\n+\tret = vgic_v5_irs_wait_for_vm_op();\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tirs_writeq_relaxed(val, offset);\n+\n+\treturn vgic_v5_irs_wait_for_vm_op();\n+}\n+\n+static int vgic_v5_irs_assign_vmt(bool two_level, u8 vm_id_bits,\n+\t\t\t\t phys_addr_t vmt_base)\n+{\n+\tu64 vmt_baser;\n+\tu32 vmt_cfgr;\n+\n+\tguard(raw_spinlock_irqsave)(\u0026global_irs_lock);\n+\n+\tvmt_baser = irs_readq_relaxed(GICV5_IRS_VMT_BASER);\n+\tif (!!FIELD_GET(GICV5_IRS_VMT_BASER_VALID, vmt_baser))\n+\t\treturn -EBUSY;\n+\n+\tvmt_cfgr = FIELD_PREP(GICV5_IRS_VMT_CFGR_VM_ID_BITS, vm_id_bits);\n+\tif (two_level)\n+\t\tvmt_cfgr |= FIELD_PREP(GICV5_IRS_VMT_CFGR_STRUCTURE,\n+\t\t\t\t GICV5_IRS_VMT_CFGR_STRUCTURE_TWO_LEVEL);\n+\n+\tirs_writel_relaxed(vmt_cfgr, GICV5_IRS_VMT_CFGR);\n+\n+\t/* The base address is intentionally only masked and not shifted */\n+\tvmt_baser = FIELD_PREP(GICV5_IRS_VMT_BASER_VALID, true) |\n+\t\t (vmt_base \u0026 GICV5_IRS_VMT_BASER_ADDR);\n+\tirs_writeq_relaxed(vmt_baser, GICV5_IRS_VMT_BASER);\n+\n+\treturn vgic_v5_irs_wait_for_vm_op();\n+}\n+\n+static int vgic_v5_irs_clear_vmt(void)\n+{\n+\treturn vgic_v5_irs_write_vm_mmio_reg(0, GICV5_IRS_VMT_BASER);\n+}\n+\n+static int vgic_v5_irs_vmap_l2_vmt(u16 vm_id)\n+{\n+\tu64 val = FIELD_PREP(GICV5_IRS_VMAP_L2_VMTR_VM_ID, vm_id) |\n+\t\tGICV5_IRS_VMAP_L2_VMTR_M;\n+\n+\treturn vgic_v5_irs_write_vm_mmio_reg(val, GICV5_IRS_VMAP_L2_VMTR);\n+}\n+\n+static int __vgic_v5_irs_vmap_vm(u16 vm_id, bool unmap)\n+{\n+\tu64 val = FIELD_PREP(GICV5_IRS_VMAP_VMR_VM_ID, vm_id) |\n+\t\tFIELD_PREP(GICV5_IRS_VMAP_VMR_U, unmap) |\n+\t\tGICV5_IRS_VMAP_VMR_M;\n+\n+\treturn vgic_v5_irs_write_vm_mmio_reg(val, GICV5_IRS_VMAP_VMR);\n+}\n+\n+static int vgic_v5_irs_set_vm_valid(u16 vm_id)\n+{\n+\treturn __vgic_v5_irs_vmap_vm(vm_id, false);\n+}\n+\n+static int vgic_v5_irs_set_vm_invalid(u16 vm_id)\n+{\n+\treturn __vgic_v5_irs_vmap_vm(vm_id, true);\n+}\n+\n+static int __vgic_v5_irs_update_vist_validity(u16 vm_id, bool spi_ist, bool unmap)\n+{\n+\tu8 type = spi_ist ? 0b011 : 0b010;\n+\tu64 val = FIELD_PREP(GICV5_IRS_VMAP_VISTR_TYPE, type) |\n+\t\tFIELD_PREP(GICV5_IRS_VMAP_VISTR_VM_ID, vm_id) |\n+\t\tFIELD_PREP(GICV5_IRS_VMAP_VISTR_U, unmap) |\n+\t\tGICV5_IRS_VMAP_VISTR_M;\n+\n+\treturn vgic_v5_irs_write_vm_mmio_reg(val, GICV5_IRS_VMAP_VISTR);\n+}\n+\n+static int vgic_v5_irs_set_vist_valid(u16 vm_id, bool spi_ist)\n+{\n+\treturn __vgic_v5_irs_update_vist_validity(vm_id, spi_ist, false);\n+}\n+\n+/*\n+ * LPI ISTs can be invalidated explicitly. SPI ISTs are invalidated by making\n+ * the VMTE invalid during teardown.\n+ */\n+static int vgic_v5_irs_set_vist_invalid(u16 vm_id, bool spi_ist)\n+{\n+\treturn __vgic_v5_irs_update_vist_validity(vm_id, spi_ist, true);\n+}\n+\n+static int vgic_v5_irs_set_up_vpe(u16 vm_id, u16 vpe_id,\n+\t\t\t\t irq_hw_number_t db_hwirq)\n+{\n+\tu64 vmap_vper, dbr, selr;\n+\tu32 statusr, cr0;\n+\tint ret;\n+\n+\tguard(raw_spinlock_irqsave)(\u0026global_irs_lock);\n+\n+\t/* Make sure that we are idle to begin with */\n+\tret = vgic_v5_irs_wait_for_vm_op();\n+\tif (ret)\n+\t\treturn ret;\n+\n+\t/* Mark the VPE as valid */\n+\tvmap_vper = FIELD_PREP(GICV5_IRS_VMAP_VPER_VPE_ID, vpe_id) |\n+\t\t FIELD_PREP(GICV5_IRS_VMAP_VPER_VM_ID, vm_id) |\n+\t\t GICV5_IRS_VMAP_VPER_M;\n+\tirs_writeq_relaxed(vmap_vper, GICV5_IRS_VMAP_VPER);\n+\n+\t/* Wait for the VPE to be marked valid in the VPET */\n+\tret = vgic_v5_irs_wait_for_vm_op();\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tselr = FIELD_PREP(GICV5_IRS_VPE_SELR_VPE_ID, vpe_id) |\n+\t FIELD_PREP(GICV5_IRS_VPE_SELR_VM_ID, vm_id) |\n+\t GICV5_IRS_VPE_SELR_S;\n+\tirs_writeq_relaxed(selr, GICV5_IRS_VPE_SELR);\n+\n+\tret = vgic_v5_irs_wait_for_vpe_op();\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tstatusr = irs_readl_relaxed(GICV5_IRS_VPE_STATUSR);\n+\tif (!FIELD_GET(GICV5_IRS_VPE_STATUSR_V, statusr))\n+\t\treturn -EINVAL;\n+\n+\t/* Set targeted only routing (disable 1ofN vPE selection) */\n+\tcr0 = GICV5_IRS_VPE_CR0_DPS;\n+\tirs_writel_relaxed(cr0, GICV5_IRS_VPE_CR0);\n+\n+\tret = vgic_v5_irs_wait_for_vpe_op();\n+\tif (ret)\n+\t\treturn ret;\n+\n+\t/*\n+\t * The VPE has not yet run. Therefore, make sure that all interrupts\n+\t * will generate a doorbell.\n+\t */\n+\tdbr = FIELD_PREP(GICV5_IRS_VPE_DBR_INTID, db_hwirq) |\n+\t GICV5_IRS_VPE_DBR_DBV;\n+\tirs_writeq_relaxed(dbr, GICV5_IRS_VPE_DBR);\n+\n+\tret = vgic_v5_irs_wait_for_vpe_op();\n+\tif (ret)\n+\t\treturn ret;\n+\n+\treturn 0;\n+}\n+\n+static int vgic_v5_irs_save_vm_op(u16 vm_id, bool save, u32 *statusr)\n+{\n+\tu64 save_vmr;\n+\tint ret;\n+\n+\tsave_vmr = FIELD_PREP(GICV5_IRS_SAVE_VMR_VM_ID, vm_id);\n+\tsave_vmr |= GICV5_IRS_SAVE_VMR_Q;\n+\tsave_vmr |= FIELD_PREP(GICV5_IRS_SAVE_VMR_S, save);\n+\n+\tguard(raw_spinlock_irqsave)(\u0026global_irs_lock);\n+\n+\t/* Make sure that we are idle to begin with. */\n+\tret = vgic_v5_irs_wait_for_save_vm_op(NULL);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tirs_writeq_relaxed(save_vmr, GICV5_IRS_SAVE_VMR);\n+\n+\treturn vgic_v5_irs_wait_for_save_vm_op(statusr);\n+}\n+\n+static irqreturn_t db_handler(int irq, void *data)\n+{\n+\tstruct kvm_vcpu *vcpu = data;\n+\n+\tWRITE_ONCE(vcpu-\u003earch.vgic_cpu.vgic_v5.gicv5_vpe.db_fired, true);\n+\n+\tkvm_make_request(KVM_REQ_IRQ_PENDING, vcpu);\n+\tkvm_vcpu_kick(vcpu);\n+\n+\treturn IRQ_HANDLED;\n+}\n+\n+static int vgic_v5_send_command(struct kvm_vcpu *vcpu, enum gicv5_vcpu_cmd cmd)\n+{\n+\treturn irq_set_vcpu_affinity(vgic_v5_vpe_db(vcpu), \u0026cmd);\n+}\n+\n+static int vgic_v5_db_set_vcpu_affinity(struct irq_data *data, void *vcpu_info)\n+{\n+\tstruct vgic_v5_vm *vm = data-\u003edomain-\u003ehost_data;\n+\tenum gicv5_vcpu_cmd *cmd = vcpu_info;\n+\t/* Our VPE ID is the index within the doorbell domain */\n+\tu16 vpe_id = data-\u003ehwirq;\n+\n+\tswitch (*cmd) {\n+\tcase VMT_L2_MAP:\n+\t\treturn vgic_v5_irs_vmap_l2_vmt(vm-\u003evm_id);\n+\tcase VMTE_MAKE_VALID:\n+\t\treturn vgic_v5_irs_set_vm_valid(vm-\u003evm_id);\n+\tcase VMTE_MAKE_INVALID:\n+\t\treturn vgic_v5_irs_set_vm_invalid(vm-\u003evm_id);\n+\tcase VPE_MAKE_VALID:\n+\t\t/*\n+\t\t * We need the actual LPI ID which lives in the top-most parent\n+\t\t * domain. This hwirq won't include the type (LPI) but that's\n+\t\t * not required for the IRS_VPE_DBR.\n+\t\t */\n+\t\twhile (data-\u003eparent_data)\n+\t\t\tdata = data-\u003eparent_data;\n+\t\treturn vgic_v5_irs_set_up_vpe(vm-\u003evm_id, vpe_id, data-\u003ehwirq);\n+\tcase SPI_VIST_MAKE_VALID:\n+\t\treturn vgic_v5_irs_set_vist_valid(vm-\u003evm_id, true);\n+\tcase LPI_VIST_MAKE_VALID:\n+\t\treturn vgic_v5_irs_set_vist_valid(vm-\u003evm_id, false);\n+\tcase LPI_VIST_MAKE_INVALID:\n+\t\treturn vgic_v5_irs_set_vist_invalid(vm-\u003evm_id, false);\n+\tdefault:\n+\t\treturn -EINVAL;\n+\t}\n+}\n+\n+/*\n+ * This set of irq_chip functions is specific for doorbells.\n+ */\n+static const struct irq_chip vgic_v5_db_irq_chip = {\n+\t.name = \"GICv5-DB\",\n+\t.irq_mask = irq_chip_mask_parent,\n+\t.irq_unmask = irq_chip_unmask_parent,\n+\t.irq_eoi = irq_chip_eoi_parent,\n+\t.irq_set_affinity = irq_chip_set_affinity_parent,\n+\t.irq_get_irqchip_state = irq_chip_get_parent_state,\n+\t.irq_set_irqchip_state = irq_chip_set_parent_state,\n+\t.irq_set_vcpu_affinity = vgic_v5_db_set_vcpu_affinity,\n+\t.flags = IRQCHIP_SET_TYPE_MASKED | IRQCHIP_SKIP_SET_WAKE |\n+\t\t IRQCHIP_MASK_ON_SUSPEND,\n+};\n+\n+static void vgic_v5_irq_db_domain_free(struct irq_domain *domain,\n+\t\t\t\t unsigned int virq, unsigned int nr_irqs)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i \u003c nr_irqs; i++) {\n+\t\tstruct irq_data *d = irq_domain_get_irq_data(domain, virq + i);\n+\n+\t\tirq_set_handler(virq + i, NULL);\n+\t\tirq_domain_reset_irq_data(d);\n+\t}\n+\n+\tirq_domain_free_irqs_parent(domain, virq, nr_irqs);\n+}\n+\n+static int vgic_v5_irq_db_domain_alloc(struct irq_domain *domain,\n+\t\t\t\t unsigned int virq, unsigned int nr_irqs,\n+\t\t\t\t void *arg)\n+{\n+\tconst struct irq_chip *chip = \u0026vgic_v5_db_irq_chip;\n+\tstruct vgic_v5_vm *vm = arg;\n+\tstruct irq_data *irqd;\n+\tint ret;\n+\n+\tif (!vm) {\n+\t\tkvm_err(\"invalid parameter for doorbell irq allocation\\n\");\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tret = irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, NULL);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tfor (int i = 0; i \u003c nr_irqs; i++) {\n+\t\tirq_domain_set_hwirq_and_chip(domain, virq + i, i, chip,\n+\t\t\t\t\t domain-\u003ehost_data);\n+\t\tirqd = irq_desc_get_irq_data(irq_to_desc(virq + i));\n+\t\tirqd_set_single_target(irqd);\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static const struct irq_domain_ops vgic_v5_irq_db_domain_ops = {\n+\t.alloc = vgic_v5_irq_db_domain_alloc,\n+\t.free = vgic_v5_irq_db_domain_free,\n+};\n+\n+static int vgic_v5_create_per_vm_domain(struct kvm *kvm)\n+{\n+\tstruct vgic_v5_vm *vm = \u0026kvm-\u003earch.vgic.gicv5_vm;\n+\tint nr_vcpus = atomic_read(\u0026kvm-\u003eonline_vcpus);\n+\tint id = task_pid_nr(current);\n+\tint ret, db_virq = 0;\n+\n+\tif (!gicv5_global_data.lpi_domain) {\n+\t\tkvm_err(\"LPI domain uninitialized, can't set up KVM Doorbells\\n\");\n+\t\treturn -ENODEV;\n+\t}\n+\n+\tvm-\u003efwnode = irq_domain_alloc_named_id_fwnode(\"GICv5-vpe-db\", id);\n+\tif (!vm-\u003efwnode)\n+\t\treturn -ENOMEM;\n+\n+\t/*\n+\t * KVM per-VM VPE DB domain; child of LPI domain; only ever handles\n+\t * doorbells. We know how many doorbells we have, and therefore we\n+\t * create a linear domain.\n+\t */\n+\tvm-\u003edomain = irq_domain_create_hierarchy(gicv5_global_data.lpi_domain,\n+\t\t\t\t\t\t 0, nr_vcpus, vm-\u003efwnode,\n+\t\t\t\t\t\t \u0026vgic_v5_irq_db_domain_ops, vm);\n+\tif (!vm-\u003edomain) {\n+\t\tret = -ENOMEM;\n+\t\tgoto err;\n+\t}\n+\n+\tdb_virq = irq_domain_alloc_irqs(vm-\u003edomain, nr_vcpus, NUMA_NO_NODE, vm);\n+\tif (db_virq \u003c= 0) {\n+\t\tret = db_virq;\n+\t\tgoto err;\n+\t}\n+\n+\tkvm-\u003earch.vgic.gicv5_vm.vpe_db_base = db_virq;\n+\n+\treturn 0;\n+\n+err:\n+\tif (db_virq \u003e 0)\n+\t\tirq_domain_free_irqs(db_virq, nr_vcpus);\n+\tif (vm-\u003edomain)\n+\t\tirq_domain_remove(vm-\u003edomain);\n+\tif (vm-\u003efwnode)\n+\t\tirq_domain_free_fwnode(vm-\u003efwnode);\n+\n+\tkvm-\u003earch.vgic.gicv5_vm.vpe_db_base = 0;\n+\tvm-\u003edomain = NULL;\n+\tvm-\u003efwnode = NULL;\n+\n+\treturn ret;\n+}\n+\n+static void vgic_v5_teardown_per_vm_domain(struct vgic_v5_vm *vm)\n+{\n+\tif (!vm-\u003edomain)\n+\t\treturn;\n+\n+\tirq_domain_remove(vm-\u003edomain);\n+\tirq_domain_free_fwnode(vm-\u003efwnode);\n+\tvm-\u003edomain = NULL;\n+\tvm-\u003efwnode = NULL;\n+}\n+\n void vgic_v5_reset(struct kvm_vcpu *vcpu)\n {\n \t/*\n@@ -123,17 +633,112 @@ void vgic_v5_reset(struct kvm_vcpu *vcpu)\n \t * CPUIF (but potentially fewer in the IRS).\n \t */\n \tvcpu-\u003earch.vgic_cpu.num_pri_bits = 5;\n+\n+\t/* Make the VPE valid in the VPET */\n+\tif (WARN_ON(vgic_v5_send_command(vcpu, VPE_MAKE_VALID)))\n+\t\treturn;\n }\n \n-int vgic_v5_init(struct kvm *kvm)\n+static void vgic_v5_free_doorbells(struct kvm *kvm, unsigned int nr_dbs)\n {\n+\tstruct vgic_v5_vm *vm = \u0026kvm-\u003earch.vgic.gicv5_vm;\n \tstruct kvm_vcpu *vcpu;\n-\tunsigned long idx;\n+\tunsigned long i;\n+\tint db;\n \n-\tif (vgic_initialized(kvm))\n-\t\treturn 0;\n+\tfor (i = 0; i \u003c nr_dbs; i++) {\n+\t\tvcpu = kvm_get_vcpu(kvm, i);\n+\t\tdb = vgic_v5_vpe_db(vcpu);\n+\t\tif (!db)\n+\t\t\tcontinue;\n+\n+\t\tfree_irq(db, vcpu);\n+\t\tvcpu-\u003earch.vgic_cpu.vgic_v5.gicv5_vpe.db = 0;\n+\t}\n+\n+\tif (vm-\u003evpe_db_base) {\n+\t\tirq_domain_free_irqs(vm-\u003evpe_db_base,\n+\t\t\t\t atomic_read(\u0026kvm-\u003eonline_vcpus));\n+\t\tvm-\u003evpe_db_base = 0;\n+\t}\n+}\n+\n+void vgic_v5_teardown(struct kvm *kvm)\n+{\n+\tstruct vgic_dist *dist = \u0026kvm-\u003earch.vgic;\n+\tstruct kvm_vcpu *vcpu, *vcpu0;\n+\tbool release_vm_id = true;\n+\tunsigned long i;\n+\tint rc;\n+\n+\t/*\n+\t * If the VM's ID isn't valid, then we either failed init very early or\n+\t * we've been called a second time. Nothing to do here in either case.\n+\t */\n+\tif (kvm-\u003earch.vgic.gicv5_vm.vm_id == VGIC_V5_VM_ID_INVAL)\n+\t\treturn;\n+\n+\tif (kvm-\u003earch.vgic.gicv5_vm.vmte_allocated) {\n+\t\t/* Make the VM invalid */\n+\t\tvcpu0 = kvm_get_vcpu(kvm, 0);\n+\t\trc = vgic_v5_send_command(vcpu0, VMTE_MAKE_INVALID);\n+\t\tif (rc) {\n+\t\t\tkvm_err(\"could not make VMTE invalid\\n\");\n+\t\t\trelease_vm_id = false;\n+\t\t\tgoto out_free_doorbells;\n+\t\t}\n \n-\tkvm_for_each_vcpu(idx, vcpu, kvm) {\n+\t\tkvm_for_each_vcpu(i, vcpu, kvm) {\n+\t\t\tif (vgic_v5_vmte_free_vpe(vcpu)) {\n+\t\t\t\tkvm_err(\"Failed to free VPE\\n\");\n+\t\t\t\trelease_vm_id = false;\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (vgic_v5_vmte_release(kvm)) {\n+\t\t\tkvm_err(\"Failed to release VM 0x%x\\n\", dist-\u003egicv5_vm.vm_id);\n+\t\t\trelease_vm_id = false;\n+\t\t}\n+\t}\n+\n+out_free_doorbells:\n+\tvgic_v5_free_doorbells(kvm, atomic_read(\u0026kvm-\u003eonline_vcpus));\n+\tvgic_v5_teardown_per_vm_domain(\u0026kvm-\u003earch.vgic.gicv5_vm);\n+\n+\t/*\n+\t * We only release the VM ID itself if we didn't fail earlier. It does\n+\t * mean that we might lose the VM ID (and associated VMTE, etc), but\n+\t * given that we've failed to tear them down correctly there's no way to\n+\t * safely reuse them. The VM ID allocating IDA will make sure we don't\n+\t * accidentally reuse this partially torn down state.\n+\t */\n+\tif (release_vm_id)\n+\t\tvgic_v5_release_vm_id(kvm);\n+}\n+\n+/*\n+ * Claim and populate a VMTE (optionally making a new L2 VMT valid), create VPE\n+ * doorbells, allocate VPET and populate for each VPE. Finally, we also init the\n+ * vIRS, which means allocating and making the virtual SPI IST valid.\n+ *\n+ * Note: We do need to put the cart before the horse here. The VPE doorbells are\n+ * our conduit for communication with the IRS, which means we need to have those\n+ * before making the VMTE valid.\n+ *\n+ * On failure, we clean up in the teardown path (vgic_v5_teardown()).\n+ */\n+int vgic_v5_init(struct kvm *kvm)\n+{\n+\tstruct kvm_vcpu *vcpu, *vcpu0;\n+\tint nr_vcpus, ret = 0;\n+\tunsigned int db_virq;\n+\tunsigned long i;\n+\n+\tnr_vcpus = atomic_read(\u0026kvm-\u003eonline_vcpus);\n+\tif (nr_vcpus == 0)\n+\t\treturn -ENODEV;\n+\n+\tkvm_for_each_vcpu(i, vcpu, kvm) {\n \t\tif (vcpu_has_nv(vcpu)) {\n \t\t\tkvm_err(\"Nested GICv5 VMs are currently unsupported\\n\");\n \t\t\treturn -EINVAL;\n@@ -148,7 +753,61 @@ int vgic_v5_init(struct kvm *kvm)\n \t\t kvm-\u003earch.vgic.gicv5_vm.userspace_ppis,\n \t\t ppi_caps.impl_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);\n \n+\tret = vgic_v5_allocate_vm_id(kvm);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tret = vgic_v5_create_per_vm_domain(kvm);\n+\tif (ret)\n+\t\tgoto err;\n+\n+\tdb_virq = kvm-\u003earch.vgic.gicv5_vm.vpe_db_base;\n+\tkvm_for_each_vcpu(i, vcpu, kvm) {\n+\t\tret = request_irq(db_virq + i, db_handler, 0, \"vcpu\", vcpu);\n+\t\tif (ret)\n+\t\t\tgoto err;\n+\n+\t\t/* Stash it with the VCPU for easy retrieval */\n+\t\tvcpu-\u003earch.vgic_cpu.vgic_v5.gicv5_vpe.db = db_virq + i;\n+\t}\n+\n+\t/* Populate VMTE (with VPET and VM descriptor) */\n+\tret = vgic_v5_vmte_init(kvm);\n+\tif (ret)\n+\t\tgoto err;\n+\n+\t/* We pick the first vcpu to make the VMTE valid - any would do */\n+\tvcpu0 = kvm_get_vcpu(kvm, 0);\n+\tret = vgic_v5_send_command(vcpu0, VMTE_MAKE_VALID);\n+\tif (ret)\n+\t\tgoto err;\n+\n+\t/* Loop over all VPEs, allocate/populate their data structures */\n+\tkvm_for_each_vcpu(i, vcpu, kvm) {\n+\t\tret = vgic_v5_vmte_alloc_vpe(vcpu);\n+\t\tif (ret)\n+\t\t\tgoto err;\n+\t}\n+\n+\t/* Init IRS (and alloc SPI IST) */\n+\tret = kvm_vgic_v5_irs_init(kvm, kvm-\u003earch.vgic.nr_spis);\n+\tif (ret)\n+\t\tgoto err;\n+\n \treturn 0;\n+\n+err:\n+\t/*\n+\t * Explicitly tear everything down on failure. The teardown function is\n+\t * written to handle any partial state we might have, so we don't need\n+\t * to do any clean-up first. Teardown will be called a second time on VM\n+\t * destruction, but that's fine - it is better to leave things in a\n+\t * clean state now, and doubly so because userspace could actually go\n+\t * and retry init.\n+\t */\n+\tvgic_v5_teardown(kvm);\n+\n+\treturn ret;\n }\n \n int vgic_v5_map_resources(struct kvm *kvm)\n@@ -159,9 +818,10 @@ int vgic_v5_map_resources(struct kvm *kvm)\n \treturn 0;\n }\n \n-int vgic_v5_finalize_ppi_state(struct kvm *kvm)\n+int vgic_v5_finalize_ppi_state(struct kvm_vcpu *vcpu)\n {\n-\tstruct kvm_vcpu *vcpu0;\n+\tstruct kvm *kvm\t= vcpu-\u003ekvm;\n+\tstruct vgic_v5_cpu_if *cpu_if = \u0026vcpu-\u003earch.vgic_cpu.vgic_v5;\n \tint i;\n \n \tif (!vgic_is_v5(kvm))\n@@ -170,35 +830,65 @@ int vgic_v5_finalize_ppi_state(struct kvm *kvm)\n \tguard(mutex)(\u0026kvm-\u003earch.config_lock);\n \n \t/*\n-\t * If SW_PPI has been advertised, then we know we already\n-\t * initialised the whole thing, and we can return early. Yes,\n-\t * this is pretty hackish as far as state tracking goes...\n+\t * Discover the set of PPIs that are exposed to the guest once per VM.\n+\t * Once known, apply that mask to each VCPU's restored PPI state as the\n+\t * VCPUs are first run.\n \t */\n-\tif (test_bit(GICV5_ARCH_PPI_SW_PPI, kvm-\u003earch.vgic.gicv5_vm.vgic_ppi_mask))\n-\t\treturn 0;\n-\n-\t/* The PPI state for all VCPUs should be the same. Pick the first. */\n-\tvcpu0 = kvm_get_vcpu(kvm, 0);\n+\tif (!test_bit(GICV5_ARCH_PPI_SW_PPI, kvm-\u003earch.vgic.gicv5_vm.vgic_ppi_mask)) {\n+\t\tbitmap_zero(kvm-\u003earch.vgic.gicv5_vm.vgic_ppi_mask,\n+\t\t\t VGIC_V5_NR_PRIVATE_IRQS);\n+\t\tbitmap_zero(kvm-\u003earch.vgic.gicv5_vm.vgic_ppi_hmr,\n+\t\t\t VGIC_V5_NR_PRIVATE_IRQS);\n+\n+\t\tfor_each_set_bit(i, ppi_caps.impl_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS) {\n+\t\t\tconst u32 intid = vgic_v5_make_ppi(i);\n+\t\t\tstruct vgic_irq *irq;\n+\n+\t\t\tirq = vgic_get_vcpu_irq(vcpu, intid);\n+\n+\t\t\t/* Expose PPIs with an owner or the SW_PPI, only */\n+\t\t\tscoped_guard(raw_spinlock_irqsave, \u0026irq-\u003eirq_lock) {\n+\t\t\t\tif (irq-\u003eowner || i == GICV5_ARCH_PPI_SW_PPI) {\n+\t\t\t\t\t__set_bit(i, kvm-\u003earch.vgic.gicv5_vm.vgic_ppi_mask);\n+\t\t\t\t\t__assign_bit(i, kvm-\u003earch.vgic.gicv5_vm.vgic_ppi_hmr,\n+\t\t\t\t\t\t irq-\u003econfig == VGIC_CONFIG_LEVEL);\n+\t\t\t\t}\n+\t\t\t}\n \n-\tbitmap_zero(kvm-\u003earch.vgic.gicv5_vm.vgic_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);\n-\tbitmap_zero(kvm-\u003earch.vgic.gicv5_vm.vgic_ppi_hmr, VGIC_V5_NR_PRIVATE_IRQS);\n+\t\t\tvgic_put_irq(kvm, irq);\n+\t\t}\n+\t}\n \n-\tfor_each_set_bit(i, ppi_caps.impl_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS) {\n+\t/*\n+\t * Apply the mask to Enable, Active. Skip pending as that's calculated\n+\t * on guest entry.\n+\t */\n+\tbitmap_and(cpu_if-\u003evgic_ppi_enabler, cpu_if-\u003evgic_ppi_enabler,\n+\t\t kvm-\u003earch.vgic.gicv5_vm.vgic_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);\n+\tbitmap_and(cpu_if-\u003evgic_ppi_activer, cpu_if-\u003evgic_ppi_activer,\n+\t\t kvm-\u003earch.vgic.gicv5_vm.vgic_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);\n+\n+\t/* Also update the vgic_irqs */\n+\tfor (i = 0; i \u003c VGIC_V5_NR_PRIVATE_IRQS; i++) {\n+\t\tbool visible = test_bit(i, kvm-\u003earch.vgic.gicv5_vm.vgic_ppi_mask);\n \t\tconst u32 intid = vgic_v5_make_ppi(i);\n \t\tstruct vgic_irq *irq;\n \n-\t\tirq = vgic_get_vcpu_irq(vcpu0, intid);\n+\t\tirq = vgic_get_vcpu_irq(vcpu, intid);\n \n-\t\t/* Expose PPIs with an owner or the SW_PPI, only */\n \t\tscoped_guard(raw_spinlock_irqsave, \u0026irq-\u003eirq_lock) {\n-\t\t\tif (irq-\u003eowner || i == GICV5_ARCH_PPI_SW_PPI) {\n-\t\t\t\t__set_bit(i, kvm-\u003earch.vgic.gicv5_vm.vgic_ppi_mask);\n-\t\t\t\t__assign_bit(i, kvm-\u003earch.vgic.gicv5_vm.vgic_ppi_hmr,\n-\t\t\t\t\t irq-\u003econfig == VGIC_CONFIG_LEVEL);\n+\t\t\tif (!visible) {\n+\t\t\t\tirq-\u003eenabled = false;\n+\t\t\t\tirq-\u003eactive = false;\n+\t\t\t\tirq-\u003epending_latch = false;\n+\t\t\t\tirq-\u003eline_level = false;\n+\t\t\t} else {\n+\t\t\t\tirq-\u003eenabled = test_bit(i, cpu_if-\u003evgic_ppi_enabler);\n+\t\t\t\tirq-\u003eactive = test_bit(i, cpu_if-\u003evgic_ppi_activer);\n \t\t\t}\n \t\t}\n \n-\t\tvgic_put_irq(vcpu0-\u003ekvm, irq);\n+\t\tvgic_put_irq(kvm, irq);\n \t}\n \n \treturn 0;\n@@ -439,9 +1129,53 @@ void vgic_v5_flush_ppi_state(struct kvm_vcpu *vcpu)\n \t\t VGIC_V5_NR_PRIVATE_IRQS);\n }\n \n+static bool vgic_v5_set_spi_pending_state(struct kvm_vcpu *vcpu,\n+\t\t\t\t\t struct vgic_irq *irq)\n+{\n+\tvgic_v5_set_irq_pend(irq-\u003etarget_vcpu, irq);\n+\treturn true;\n+}\n+\n+bool vgic_v5_spi_queue_irq_unlock(struct kvm *kvm,\n+\t\t\t\t struct vgic_irq *irq,\n+\t\t\t\t unsigned long flags)\n+\t__releases(\u0026irq-\u003eirq_lock)\n+{\n+\tlockdep_assert_held(\u0026irq-\u003eirq_lock);\n+\n+\traw_spin_unlock_irqrestore(\u0026irq-\u003eirq_lock, flags);\n+\treturn true;\n+}\n+\n+static const struct irq_ops vgic_v5_spi_irq_ops = {\n+\t.set_pending_state = vgic_v5_set_spi_pending_state,\n+\t.queue_irq_unlock = vgic_v5_spi_queue_irq_unlock,\n+};\n+\n+void vgic_v5_set_spi_ops(struct vgic_irq *irq)\n+{\n+\tif (WARN_ON(!irq) || WARN_ON(irq-\u003eops))\n+\t\treturn;\n+\n+\tirq-\u003eops = \u0026vgic_v5_spi_irq_ops;\n+}\n+\n+/* Set the pending state for GICv5 SPIs and LPIs */\n+void vgic_v5_set_irq_pend(struct kvm_vcpu *vcpu, struct vgic_irq *irq)\n+{\n+\tif (WARN_ON(__irq_is_ppi(KVM_DEV_TYPE_ARM_VGIC_V5, irq-\u003eintid)))\n+\t\treturn;\n+\n+\tkvm_call_hyp(__vgic_v5_vdpend, irq-\u003eintid, irq_is_pending(irq),\n+\t\t vcpu-\u003ekvm-\u003earch.vgic.gicv5_vm.vm_id);\n+}\n+\n void vgic_v5_load(struct kvm_vcpu *vcpu)\n {\n+\tbool irichppidis = !vcpu-\u003ekvm-\u003earch.vgic.enabled;\n \tstruct vgic_v5_cpu_if *cpu_if = \u0026vcpu-\u003earch.vgic_cpu.vgic_v5;\n+\tu16 vm = vgic_v5_vm_id(vcpu-\u003ekvm);\n+\tu16 vpe = vgic_v5_vpe_id(vcpu);\n \n \t/*\n \t * On the WFI path, vgic_load is called a second time. The first is when\n@@ -454,7 +1188,16 @@ void vgic_v5_load(struct kvm_vcpu *vcpu)\n \n \tkvm_call_hyp(__vgic_v5_restore_vmcr_apr, cpu_if);\n \n-\tcpu_if-\u003egicv5_vpe.resident = true;\n+\tcpu_if-\u003evgic_contextr = FIELD_PREP(ICH_CONTEXTR_EL2_V, true) |\n+\t\t\t\tFIELD_PREP(ICH_CONTEXTR_EL2_IRICHPPIDIS, irichppidis) |\n+\t\t\t\tFIELD_PREP(ICH_CONTEXTR_EL2_VPE, vpe) |\n+\t\t\t\tFIELD_PREP(ICH_CONTEXTR_EL2_VM, vm);\n+\n+\tkvm_call_hyp(__vgic_v5_make_resident, cpu_if);\n+\n+\t/* Failed to make the VPE resident? Bang! */\n+\tif (WARN_ON(!!FIELD_GET(ICH_CONTEXTR_EL2_F, cpu_if-\u003evgic_contextr)))\n+\t\tkvm_vm_dead(vcpu-\u003ekvm);\n }\n \n void vgic_v5_put(struct kvm_vcpu *vcpu)\n@@ -472,7 +1215,49 @@ void vgic_v5_put(struct kvm_vcpu *vcpu)\n \n \tkvm_call_hyp(__vgic_v5_save_apr, cpu_if);\n \n-\tcpu_if-\u003egicv5_vpe.resident = false;\n+\tcpu_if-\u003evgic_contextr = 0;\n+\tif (vcpu_get_flag(vcpu, IN_WFI)) {\n+\t\tu32 priority_mask;\n+\t\tint dbpm;\n+\n+\t\t/*\n+\t\t * Find the virtual running priority and use this to calculate\n+\t\t * the doorbell priority mask. We combine the highest active\n+\t\t * priority and the CPU's priority mask. The guest can't handle\n+\t\t * interrupts with priorities less than or equal to the virtual\n+\t\t * running priority, so there's literally no point in waking the\n+\t\t * guest for these.\n+\t\t *\n+\t\t * The priority needs to be higher than the mask to signal, so\n+\t\t * pick the next higher priority (subtract 1).\n+\t\t */\n+\t\tpriority_mask = vgic_v5_get_effective_priority_mask(vcpu);\n+\n+\t\t/*\n+\t\t * Request a doorbell *unless* the priority is 0, indicating\n+\t\t * that no interrupt can wake the CPU up.\n+\t\t */\n+\t\tif (priority_mask) {\n+\t\t\tint db_irq = vgic_v5_vpe_db(vcpu);\n+\t\t\tstruct irq_data *d = irq_get_irq_data(db_irq);\n+\t\t\tconst struct cpumask *aff = irq_data_get_effective_affinity_mask(d);\n+\t\t\tint cpu = smp_processor_id();\n+\n+\t\t\tdbpm = priority_mask - 1;\n+\t\t\tcpu_if-\u003evgic_contextr = FIELD_PREP(ICH_CONTEXTR_EL2_DB, 1) |\n+\t\t\t\t\t\tFIELD_PREP(ICH_CONTEXTR_EL2_DBPM, dbpm);\n+\n+\t\t\t/*\n+\t\t\t * Make the doorbell affine to this CPU, if it isn't\n+\t\t\t * already. Actively check the cpumask first as it is\n+\t\t\t * cheaper than changing the affinity every time.\n+\t\t\t */\n+\t\t\tif (!cpumask_test_cpu(cpu, aff))\n+\t\t\t\tWARN_ON(irq_set_affinity(db_irq, cpumask_of(cpu)));\n+\t\t}\n+\t}\n+\n+\tkvm_call_hyp(__vgic_v5_make_non_resident, cpu_if);\n \n \t/* The shadow priority is only updated on entering WFI */\n \tif (vcpu_get_flag(vcpu, IN_WFI))\n@@ -516,3 +1301,292 @@ void vgic_v5_save_state(struct kvm_vcpu *vcpu)\n \t__vgic_v5_save_ppi_state(cpu_if);\n \tdsb(sy);\n }\n+\n+static int vgic_v5_irs_status_is_quiesced(u32 statusr)\n+{\n+\tif (statusr \u0026 GICV5_IRS_SAVE_VM_STATUSR_Q)\n+\t\treturn 0;\n+\n+\treturn -EBUSY;\n+}\n+\n+static int vgic_v5_irs_is_quiesced(u16 vm_id)\n+{\n+\tu32 statusr;\n+\tint ret;\n+\n+\tret = vgic_v5_irs_save_vm_op(vm_id, false, \u0026statusr);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\treturn vgic_v5_irs_status_is_quiesced(statusr);\n+}\n+\n+static int vgic_v5_copy_ist_attr(struct kvm_device_attr *attr,\n+\t\t\t\t struct kvm_vgic_v5_ist *ist_attr)\n+{\n+\tvoid __user *uaddr = (void __user *)(unsigned long)attr-\u003eaddr;\n+\n+\tif (!uaddr)\n+\t\treturn -EINVAL;\n+\n+\tif (copy_from_user(ist_attr, uaddr, sizeof(*ist_attr)))\n+\t\treturn -EFAULT;\n+\n+\treturn 0;\n+}\n+\n+static int vgic_v5_validate_ist_user_buffer(__u64 addr, __u64 size,\n+\t\t\t\t\t size_t expected)\n+{\n+\tif (!addr || size != expected)\n+\t\treturn -EINVAL;\n+\n+\treturn 0;\n+}\n+\n+static int vgic_v5_validate_ist_attr(struct kvm *kvm,\n+\t\t\t\t const struct kvm_vgic_v5_ist *ist_attr)\n+{\n+\tunsigned int id_bits;\n+\tint ret;\n+\n+\t/* We always have SPIs to save */\n+\tret = vgic_v5_validate_ist_user_buffer(ist_attr-\u003espi_ist_addr,\n+\t\t\t\t\tist_attr-\u003espi_ist_size,\n+\t\t\t\t\tkvm-\u003earch.vgic.nr_spis * sizeof(__u32));\n+\tif (ret)\n+\t\treturn ret;\n+\n+\t/* We don't always have LPIs to save */\n+\tret = vgic_v5_irs_lpi_ist_id_bits(kvm, \u0026id_bits);\n+\tif (ret \u003c 0)\n+\t\treturn ret;\n+\n+\t/* No LPI IST */\n+\tif (!ret) {\n+\t\tif (ist_attr-\u003elpi_ist_addr || ist_attr-\u003elpi_ist_size)\n+\t\t\treturn -EINVAL;\n+\n+\t\treturn 0;\n+\t}\n+\n+\treturn vgic_v5_validate_ist_user_buffer(ist_attr-\u003elpi_ist_addr,\n+\t\t\t\t\t ist_attr-\u003elpi_ist_size,\n+\t\t\t\t\t BIT(id_bits) * sizeof(__u32));\n+}\n+\n+int vgic_v5_irs_save_ists(struct kvm *kvm, struct kvm_device_attr *attr)\n+{\n+\tstruct kvm_vgic_v5_ist ist_attr;\n+\tu16 vm_id = vgic_v5_vm_id(kvm);\n+\tu32 statusr;\n+\tint ret = 0;\n+\n+\tmutex_lock(\u0026kvm-\u003elock);\n+\n+\tif (kvm_trylock_all_vcpus(kvm)) {\n+\t\tmutex_unlock(\u0026kvm-\u003elock);\n+\t\treturn -EBUSY;\n+\t}\n+\n+\tmutex_lock(\u0026kvm-\u003earch.config_lock);\n+\n+\tif (!vgic_initialized(kvm)) {\n+\t\tret = -EBUSY;\n+\t\tgoto out_unlock;\n+\t}\n+\n+\tret = vgic_v5_copy_ist_attr(attr, \u0026ist_attr);\n+\tif (ret)\n+\t\tgoto out_unlock;\n+\n+\tret = vgic_v5_validate_ist_attr(kvm, \u0026ist_attr);\n+\tif (ret)\n+\t\tgoto out_unlock;\n+\n+\tret = vgic_v5_irs_save_vm_op(vm_id, true, \u0026statusr);\n+\tif (ret) {\n+\t\tkvm_err(\"Failed to save GICv5 IRS VM state: %d\\n\", ret);\n+\t\tgoto out_unlock;\n+\t}\n+\n+\tret = vgic_v5_irs_status_is_quiesced(statusr);\n+\tif (ret)\n+\t\tgoto out_unlock;\n+\n+\t/* Save the SPI IST to the userspace buffer. */\n+\tret = vgic_v5_save_spi_ist(kvm, \u0026ist_attr);\n+\tif (ret)\n+\t\tgoto out_unlock;\n+\n+\tret = vgic_v5_irs_is_quiesced(vm_id);\n+\tif (ret)\n+\t\tgoto out_unlock;\n+\n+\t/* Save the LPI IST to the userspace buffer. */\n+\tret = vgic_v5_save_lpi_ist(kvm, \u0026ist_attr);\n+\tif (ret)\n+\t\tgoto out_unlock;\n+\n+\tret = vgic_v5_irs_is_quiesced(vm_id);\n+\tif (ret)\n+\t\tgoto out_unlock;\n+\n+out_unlock:\n+\tmutex_unlock(\u0026kvm-\u003earch.config_lock);\n+\tkvm_unlock_all_vcpus(kvm);\n+\tmutex_unlock(\u0026kvm-\u003elock);\n+\n+\treturn ret;\n+}\n+\n+/* Allocate the LPI IST to restore into */\n+static int vgic_v5_restore_lpi_ist_alloc(struct kvm *kvm, bool *allocated)\n+{\n+\tunsigned int id_bits;\n+\tint ret;\n+\n+\t*allocated = false;\n+\n+\tret = vgic_v5_irs_lpi_ist_id_bits(kvm, \u0026id_bits);\n+\tif (ret \u003c= 0)\n+\t\treturn ret;\n+\n+\tret = vgic_v5_lpi_ist_alloc(kvm, id_bits);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\t*allocated = true;\n+\n+\treturn 0;\n+}\n+\n+/*\n+ * Clean up the LPI IST if we allocated it, and restore the VMTE to the\n+ * original, valid state.\n+ */\n+static void vgic_v5_restore_cleanup(struct kvm *kvm,\n+\t\t\t\t struct kvm_vcpu *vcpu,\n+\t\t\t\t bool lpi_ist_allocated)\n+{\n+\t/*\n+\t * We are on the restore failure path, so we do a best-effort\n+\t * cleanup. These commands might fail, but at this stage this is the\n+\t * best we can realistically do.\n+\t */\n+\tif (lpi_ist_allocated) {\n+\t\tif (!vgic_v5_send_command(vcpu, VMTE_MAKE_INVALID))\n+\t\t\tvgic_v5_lpi_ist_free(kvm);\n+\t}\n+\n+\tvgic_v5_send_command(vcpu, VMTE_MAKE_VALID);\n+}\n+\n+int vgic_v5_irs_restore_ists(struct kvm *kvm, struct kvm_device_attr *attr)\n+{\n+\tbool lpi_ist_allocated = false, vmte_invalid = false;\n+\tstruct kvm_vcpu *vcpu0 = kvm_get_vcpu(kvm, 0);\n+\tstruct kvm_vgic_v5_ist ist_attr;\n+\tint ret = 0;\n+\n+\tmutex_lock(\u0026kvm-\u003elock);\n+\n+\tif (kvm_trylock_all_vcpus(kvm)) {\n+\t\tmutex_unlock(\u0026kvm-\u003elock);\n+\t\treturn -EBUSY;\n+\t}\n+\n+\tmutex_lock(\u0026kvm-\u003earch.config_lock);\n+\n+\tif (!vgic_initialized(kvm)) {\n+\t\tret = -EBUSY;\n+\t\tgoto out_unlock;\n+\t}\n+\n+\tif (kvm_vm_has_ran_once(kvm)) {\n+\t\tret = -EBUSY;\n+\t\tgoto out_unlock;\n+\t}\n+\n+\tret = vgic_v5_copy_ist_attr(attr, \u0026ist_attr);\n+\tif (ret)\n+\t\tgoto out_unlock;\n+\n+\tret = vgic_v5_validate_ist_attr(kvm, \u0026ist_attr);\n+\tif (ret)\n+\t\tgoto out_unlock;\n+\n+\tret = vgic_v5_lpi_ist_exists(kvm);\n+\tif (ret) {\n+\t\tif (ret \u003e 0)\n+\t\t\tret = -EBUSY;\n+\t\tgoto out_unlock;\n+\t}\n+\n+\t/*\n+\t * If the guest has previously allocated an IST (which we check based on\n+\t * the IRS_IST_BASER), extract the number of LPI ID bits from the\n+\t * IRS_IST_CFGR. Else, do nothing.\n+\t *\n+\t * We do this before making the VMTE invalid as we rely on\n+\t * IRS_VMAP_VISTR to mark the IST as valid in the VMTE. This can only\n+\t * happen while the VMTE is valid.\n+\t */\n+\tret = vgic_v5_restore_lpi_ist_alloc(kvm, \u0026lpi_ist_allocated);\n+\tif (ret)\n+\t\tgoto out_unlock;\n+\n+\t/*\n+\t * Host ISTs are updated while the VMTE is invalid, so the GIC cannot\n+\t * observe partially restored state.\n+\t */\n+\tret = vgic_v5_send_command(vcpu0, VMTE_MAKE_INVALID);\n+\tif (ret) {\n+\t\t/*\n+\t\t * If invalidation fails, the restore cannot safely update host\n+\t\t * IST state.\n+\t\t */\n+\t\tgoto out_unlock;\n+\t}\n+\tvmte_invalid = true;\n+\n+\t/* Restore the SPI IST from the userspace buffer. */\n+\tret = vgic_v5_restore_spi_ist(kvm, \u0026ist_attr);\n+\tif (ret)\n+\t\tgoto out_unlock;\n+\n+\t/* Restore the LPI IST from the userspace buffer. */\n+\tif (lpi_ist_allocated) {\n+\t\tret = vgic_v5_restore_lpi_ist(kvm, \u0026ist_attr);\n+\t\tif (ret)\n+\t\t\tgoto out_unlock;\n+\t}\n+\n+\t/* And make the VM Valid again */\n+\tret = vgic_v5_send_command(vcpu0, VMTE_MAKE_VALID);\n+\tif (ret)\n+\t\tgoto out_unlock;\n+\tvmte_invalid = false;\n+\n+\t/*\n+\t * As part of restoring the ISTs, and previously pending interrupts have\n+\t * been tracked and made non-pending. Now that the ISTs have been\n+\t * restored, and the VM is valid again, restore the pending interrupts.\n+\t */\n+\tret = vgic_v5_restore_pending_irqs(kvm);\n+\tif (ret)\n+\t\tgoto out_unlock;\n+\n+out_unlock:\n+\tif (ret \u0026\u0026 (vmte_invalid || lpi_ist_allocated)) {\n+\t\tvgic_v5_discard_pending_irqs(kvm);\n+\t\tvgic_v5_restore_cleanup(kvm, vcpu0, lpi_ist_allocated);\n+\t}\n+\n+\tmutex_unlock(\u0026kvm-\u003earch.config_lock);\n+\tkvm_unlock_all_vcpus(kvm);\n+\tmutex_unlock(\u0026kvm-\u003elock);\n+\n+\treturn ret;\n+}\ndiff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c\nindex ccb7e3a90cd07a..a27895295ecbb8 100644\n--- a/arch/arm64/kvm/vgic/vgic.c\n+++ b/arch/arm64/kvm/vgic/vgic.c\n@@ -86,19 +86,31 @@ static struct vgic_irq *vgic_get_lpi(struct kvm *kvm, u32 intid)\n */\n struct vgic_irq *vgic_get_irq(struct kvm *kvm, u32 intid)\n {\n-\t/* Non-private IRQs are not yet implemented for GICv5 */\n-\tif (vgic_is_v5(kvm))\n-\t\treturn NULL;\n+\tenum kvm_device_type type = kvm-\u003earch.vgic.vgic_model;\n \n \t/* SPIs */\n-\tif (intid \u003e= VGIC_NR_PRIVATE_IRQS \u0026\u0026\n-\t intid \u003c (kvm-\u003earch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS)) {\n-\t\tintid = array_index_nospec(intid, kvm-\u003earch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS);\n-\t\treturn \u0026kvm-\u003earch.vgic.spis[intid - VGIC_NR_PRIVATE_IRQS];\n+\tif (__irq_is_spi(type, intid)) {\n+\t\tswitch (type) {\n+\t\tcase KVM_DEV_TYPE_ARM_VGIC_V5:\n+\t\t\tintid = vgic_v5_get_hwirq_id(intid);\n+\n+\t\t\tif (intid \u003e= kvm-\u003earch.vgic.nr_spis)\n+\t\t\t\treturn NULL;\n+\n+\t\t\tintid = array_index_nospec(intid, kvm-\u003earch.vgic.nr_spis);\n+\t\t\treturn \u0026kvm-\u003earch.vgic.spis[intid];\n+\t\tdefault: {\n+\t\t\tu32 max_intid = kvm-\u003earch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;\n+\n+\t\t\tif (intid \u003c max_intid) {\n+\t\t\t\tintid = array_index_nospec(intid, max_intid);\n+\t\t\t\treturn \u0026kvm-\u003earch.vgic.spis[intid - VGIC_NR_PRIVATE_IRQS];\n+\t\t\t}\n+\t\t}}\n \t}\n \n \t/* LPIs */\n-\tif (irq_is_lpi(kvm, intid))\n+\tif (__irq_is_lpi(type, intid))\n \t\treturn vgic_get_lpi(kvm, intid);\n \n \treturn NULL;\n@@ -564,6 +576,9 @@ int kvm_vgic_inject_irq(struct kvm *kvm, struct kvm_vcpu *vcpu,\n \telse\n \t\tirq-\u003epending_latch = true;\n \n+\tif (irq-\u003eops \u0026\u0026 irq-\u003eops-\u003eset_pending_state)\n+\t\tWARN_ON_ONCE(!irq-\u003eops-\u003eset_pending_state(vcpu, irq));\n+\n \tvgic_queue_irq_unlock(kvm, irq, flags);\n \tvgic_put_irq(kvm, irq);\n \n@@ -1235,8 +1250,12 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)\n \tunsigned long flags;\n \tstruct vgic_vmcr vmcr;\n \n-\tif (vgic_is_v5(vcpu-\u003ekvm))\n+\tif (vgic_is_v5(vcpu-\u003ekvm)) {\n+\t\tif (READ_ONCE(vcpu-\u003earch.vgic_cpu.vgic_v5.gicv5_vpe.db_fired))\n+\t\t\treturn true;\n+\n \t\treturn vgic_v5_has_pending_ppi(vcpu);\n+\t}\n \n \tif (!vcpu-\u003ekvm-\u003earch.vgic.enabled)\n \t\treturn false;\ndiff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h\nindex f45f7e3ec4d6ef..cb673da96ec55b 100644\n--- a/arch/arm64/kvm/vgic/vgic.h\n+++ b/arch/arm64/kvm/vgic/vgic.h\n@@ -252,6 +252,8 @@ struct ap_list_summary {\n #define irqs_active_outside_lrs(s)\t\t\\\n \t((s)-\u003enr_act \u0026\u0026\tirqs_outside_lrs(s))\n \n+int vgic_v5_parse_attr(struct kvm_device *dev, struct kvm_device_attr *attr,\n+\t\t struct vgic_reg_attr *reg_attr);\n int vgic_v3_parse_attr(struct kvm_device *dev, struct kvm_device_attr *attr,\n \t\t struct vgic_reg_attr *reg_attr);\n int vgic_v2_parse_attr(struct kvm_device *dev, struct kvm_device_attr *attr,\n@@ -366,8 +368,14 @@ void vgic_debug_destroy(struct kvm *kvm);\n int vgic_v5_probe(const struct gic_kvm_info *info);\n void vgic_v5_reset(struct kvm_vcpu *vcpu);\n int vgic_v5_init(struct kvm *kvm);\n+int kvm_vgic_v5_irs_init(struct kvm *kvm, unsigned int nr_spis);\n+void vgic_v5_teardown(struct kvm *kvm);\n int vgic_v5_map_resources(struct kvm *kvm);\n void vgic_v5_set_ppi_ops(struct kvm_vcpu *vcpu, u32 vintid);\n+void vgic_v5_set_spi_ops(struct vgic_irq *irq);\n+bool vgic_v5_spi_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq,\n+\t\t\t\t unsigned long flags);\n+void vgic_v5_set_irq_pend(struct kvm_vcpu *vcpu, struct vgic_irq *irq);\n bool vgic_v5_has_pending_ppi(struct kvm_vcpu *vcpu);\n void vgic_v5_flush_ppi_state(struct kvm_vcpu *vcpu);\n void vgic_v5_fold_ppi_state(struct kvm_vcpu *vcpu);\n@@ -377,6 +385,19 @@ void vgic_v5_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);\n void vgic_v5_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);\n void vgic_v5_restore_state(struct kvm_vcpu *vcpu);\n void vgic_v5_save_state(struct kvm_vcpu *vcpu);\n+int vgic_v5_register_irs_iodev(struct kvm *kvm, gpa_t irs_base_address);\n+int vgic_v5_irs_lpi_ist_id_bits(struct kvm *kvm, unsigned int *id_bits);\n+\n+int vgic_v5_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu,\n+\t\t\t\tstruct kvm_device_attr *attr, bool is_write);\n+int vgic_v5_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr);\n+const struct sys_reg_desc *vgic_v5_get_sysreg_table(unsigned int *sz);\n+int vgic_v5_irs_save_ists(struct kvm *kvm, struct kvm_device_attr *attr);\n+int vgic_v5_irs_restore_ists(struct kvm *kvm, struct kvm_device_attr *attr);\n+int vgic_v5_irs_attr_regs_access(struct kvm_device *dev,\n+\t\t\t\t struct kvm_device_attr *attr,\n+\t\t\t\t u64 *reg, bool is_write);\n+int vgic_v5_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr);\n \n #define for_each_visible_v5_ppi(__i, __k)\t\t\\\n \tfor_each_set_bit(__i, (__k)-\u003earch.vgic.gicv5_vm.vgic_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS)\ndiff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg\nindex 7cb61aca3797fa..03f5e1755a6333 100644\n--- a/arch/arm64/tools/sysreg\n+++ b/arch/arm64/tools/sysreg\n@@ -3736,7 +3736,9 @@ Sysreg\tICC_CR0_EL1\t3\t1\t12\t0\t1\n Res0\t63:39\n Field\t38\tPID\n Field\t37:32\tIPPT\n-Res0\t31:1\n+Res0\t31:3\n+Field\t2\tLINK_IDLE\n+Field\t1\tLINK\n Field\t0\tEN\n EndSysreg\n \ndiff --git a/drivers/irqchip/irq-gic-v5-irs.c b/drivers/irqchip/irq-gic-v5-irs.c\nindex f3fce0b1e25d96..70502b07ec8d72 100644\n--- a/drivers/irqchip/irq-gic-v5-irs.c\n+++ b/drivers/irqchip/irq-gic-v5-irs.c\n@@ -21,8 +21,6 @@\n */\n #define LPI_ID_BITS_LINEAR\t\t12\n \n-#define IRS_FLAGS_NON_COHERENT\t\tBIT(0)\n-\n static DEFINE_PER_CPU_READ_MOSTLY(struct gicv5_irs_chip_data *, per_cpu_irs_data);\n static LIST_HEAD(irs_nodes);\n \n@@ -50,6 +48,11 @@ static void irs_writeq_relaxed(struct gicv5_irs_chip_data *irs_data,\n \twriteq_relaxed(val, irs_data-\u003eirs_base + reg_offset);\n }\n \n+struct gicv5_irs_chip_data *gicv5_irs_get_chip_data(void)\n+{\n+\treturn per_cpu(per_cpu_irs_data, 0);\n+}\n+\n /*\n * The polling wait (in gicv5_wait_for_op_s_atomic()) on a GIC register\n * provides the memory barriers (through MMIO accessors)\n@@ -266,24 +269,24 @@ int gicv5_irs_iste_alloc(const u32 lpi)\n * itself is not supported) again serves to make it easier to find physically\n * contiguous blocks of memory.\n */\n-static unsigned int gicv5_irs_l2_sz(u32 idr2)\n+unsigned int gicv5_irs_l2_sz(u32 l2sz)\n {\n \tswitch (PAGE_SIZE) {\n \tcase SZ_64K:\n-\t\tif (GICV5_IRS_IST_L2SZ_SUPPORT_64KB(idr2))\n+\t\tif (GICV5_IRS_IST_L2SZ_SUPPORT_64KB(l2sz))\n \t\t\treturn GICV5_IRS_IST_CFGR_L2SZ_64K;\n \t\tfallthrough;\n \tcase SZ_4K:\n-\t\tif (GICV5_IRS_IST_L2SZ_SUPPORT_4KB(idr2))\n+\t\tif (GICV5_IRS_IST_L2SZ_SUPPORT_4KB(l2sz))\n \t\t\treturn GICV5_IRS_IST_CFGR_L2SZ_4K;\n \t\tfallthrough;\n \tcase SZ_16K:\n-\t\tif (GICV5_IRS_IST_L2SZ_SUPPORT_16KB(idr2))\n+\t\tif (GICV5_IRS_IST_L2SZ_SUPPORT_16KB(l2sz))\n \t\t\treturn GICV5_IRS_IST_CFGR_L2SZ_16K;\n \t\tbreak;\n \t}\n \n-\tif (GICV5_IRS_IST_L2SZ_SUPPORT_4KB(idr2))\n+\tif (GICV5_IRS_IST_L2SZ_SUPPORT_4KB(l2sz))\n \t\treturn GICV5_IRS_IST_CFGR_L2SZ_4K;\n \n \treturn GICV5_IRS_IST_CFGR_L2SZ_64K;\n@@ -331,7 +334,7 @@ static int __init gicv5_irs_init_ist(struct gicv5_irs_chip_data *irs_data)\n \tlpi_id_bits = min(lpi_id_bits, gicv5_global_data.cpuif_id_bits);\n \n \tif (two_levels)\n-\t\tl2sz = gicv5_irs_l2_sz(idr2);\n+\t\tl2sz = gicv5_irs_l2_sz(FIELD_GET(GICV5_IRS_IDR2_IST_L2SZ, idr2));\n \n \tistmd = !!FIELD_GET(GICV5_IRS_IDR2_ISTMD, idr2);\n \ndiff --git a/drivers/irqchip/irq-gic-v5.c b/drivers/irqchip/irq-gic-v5.c\nindex e9d1795235a66f..0d90675fb319bb 100644\n--- a/drivers/irqchip/irq-gic-v5.c\n+++ b/drivers/irqchip/irq-gic-v5.c\n@@ -1126,8 +1126,10 @@ static void gicv5_set_cpuif_idbits(void)\n #ifdef CONFIG_KVM\n static struct gic_kvm_info gic_v5_kvm_info __initdata;\n \n-static void __init gic_of_setup_kvm_info(struct device_node *node)\n+static void __init gic_setup_kvm_info(unsigned int maint_irq)\n {\n+\tstruct gicv5_irs_chip_data *irs_data = gicv5_irs_get_chip_data();\n+\n \t/*\n \t * If we don't have native GICv5 virtualisation support, then\n \t * we also don't have FEAT_GCIE_LEGACY - the architecture\n@@ -1135,25 +1137,127 @@ static void __init gic_of_setup_kvm_info(struct device_node *node)\n \t */\n \tif (!gicv5_global_data.virt_capable) {\n \t\tpr_info(\"GIC implementation is not virtualization capable\\n\");\n-\t\treturn;\n+\t\tgoto out_dispose_maint_irq;\n \t}\n \n+\tif (WARN_ON(!irs_data))\n+\t\tgoto out_dispose_maint_irq;\n+\n \tgic_v5_kvm_info.type = GIC_V5;\n+\tgic_v5_kvm_info.gicv5_irs.base = irs_data-\u003eirs_base;\n+\tgic_v5_kvm_info.gicv5_irs.non_coherent = !!(irs_data-\u003eflags \u0026 IRS_FLAGS_NON_COHERENT);\n \n \t/* GIC Virtual CPU interface maintenance interrupt */\n \tgic_v5_kvm_info.no_maint_irq_mask = false;\n-\tgic_v5_kvm_info.maint_irq = irq_of_parse_and_map(node, 0);\n-\tif (!gic_v5_kvm_info.maint_irq) {\n-\t\tpr_warn(\"cannot find GICv5 virtual CPU interface maintenance interrupt\\n\");\n+\tgic_v5_kvm_info.maint_irq = maint_irq;\n+\n+\t/*\n+\t * We require an MI if we have legacy support, but don't, otherwise.\n+\t * Given that there's an existing flag to convey that an MI isn't\n+\t * needed, we (ab)use it to tell KVM that the MI isn't needed if we\n+\t * don't support legacy.\n+\t *\n+\t * The check for ARM64_HAS_GICV5_LEGACY explicitly doesn't use\n+\t * cpus_have_final_cap() here as we run too early.\n+\t */\n+\tif (!cpus_have_cap(ARM64_HAS_GICV5_LEGACY) \u0026\u0026 !gic_v5_kvm_info.maint_irq)\n+\t\tgic_v5_kvm_info.no_maint_irq_mask = true;\n+\n+\tvgic_set_kvm_info(\u0026gic_v5_kvm_info);\n+\treturn;\n+\n+out_dispose_maint_irq:\n+\tirq_dispose_mapping(maint_irq);\n+}\n+\n+static void __init gic_of_setup_kvm_info(struct device_node *node)\n+{\n+\t/* GIC Virtual CPU interface maintenance interrupt */\n+\tgic_setup_kvm_info(irq_of_parse_and_map(node, 0));\n+}\n+\n+#ifdef CONFIG_ACPI\n+struct gicv5_acpi_kvm_info {\n+\tu32 maint_irq;\n+\tint maint_irq_mode;\n+};\n+\n+static struct gicv5_acpi_kvm_info acpi_v5_kvm_info __initdata;\n+\n+static int __init gic_acpi_parse_virt_madt_gicc(union acpi_subtable_headers *header,\n+\t\t\t\t\t\tconst unsigned long end)\n+{\n+\tstruct acpi_madt_generic_interrupt *gicc =\n+\t\t(struct acpi_madt_generic_interrupt *)header;\n+\tstatic int first_madt = true;\n+\tint maint_irq_mode;\n+\n+\tif (!(gicc-\u003eflags \u0026\n+\t (ACPI_MADT_ENABLED | ACPI_MADT_GICC_ONLINE_CAPABLE)))\n+\t\treturn 0;\n+\n+\tmaint_irq_mode = (gicc-\u003eflags \u0026 ACPI_MADT_VGIC_IRQ_MODE) ?\n+\t\t\t ACPI_EDGE_SENSITIVE : ACPI_LEVEL_SENSITIVE;\n+\n+\tif (first_madt) {\n+\t\tfirst_madt = false;\n+\n+\t\tacpi_v5_kvm_info.maint_irq = gicc-\u003evgic_interrupt;\n+\t\tacpi_v5_kvm_info.maint_irq_mode = maint_irq_mode;\n+\t\treturn 0;\n+\t}\n+\n+\t/* The maintenance interrupt must be the same for every GICC entry. */\n+\tif (acpi_v5_kvm_info.maint_irq != gicc-\u003evgic_interrupt ||\n+\t acpi_v5_kvm_info.maint_irq_mode != maint_irq_mode)\n+\t\treturn -EINVAL;\n+\n+\treturn 0;\n+}\n+\n+static bool __init gic_acpi_collect_virt_info(void)\n+{\n+\tint count;\n+\n+\tcount = acpi_table_parse_madt(ACPI_MADT_TYPE_GENERIC_INTERRUPT,\n+\t\t\t\t gic_acpi_parse_virt_madt_gicc, 0);\n+\n+\treturn count \u003e 0;\n+}\n+\n+static void __init gic_acpi_setup_kvm_info(void)\n+{\n+\tunsigned int maint_irq = 0;\n+\tint irq;\n+\n+\tif (!gic_acpi_collect_virt_info()) {\n+\t\tpr_warn(\"Unable to get hardware information used for virtualization\\n\");\n \t\treturn;\n \t}\n \n-\tvgic_set_kvm_info(\u0026gic_v5_kvm_info);\n+\tif (acpi_v5_kvm_info.maint_irq) {\n+\t\tirq = acpi_register_gsi(NULL, acpi_v5_kvm_info.maint_irq,\n+\t\t\t\t\tacpi_v5_kvm_info.maint_irq_mode,\n+\t\t\t\t\tACPI_ACTIVE_HIGH);\n+\t\tif (irq \u003e 0)\n+\t\t\tmaint_irq = irq;\n+\t\telse\n+\t\t\tpr_warn(\"Failed to register GSI for GICv5 maintenance IRQ\\n\");\n+\t}\n+\n+\tgic_setup_kvm_info(maint_irq);\n }\n+#endif\n #else\n static inline void __init gic_of_setup_kvm_info(struct device_node *node)\n {\n }\n+\n+#ifdef CONFIG_ACPI\n+static inline void __init gic_acpi_setup_kvm_info(void)\n+{\n+}\n+#endif\n #endif // CONFIG_KVM\n \n static int __init gicv5_init_common(struct fwnode_handle *parent_domain)\n@@ -1252,6 +1356,7 @@ static int __init gic_acpi_init(union acpi_subtable_headers *header, const unsig\n \t\tgoto out_irs;\n \n \tacpi_set_irq_model(ACPI_IRQ_MODEL_GIC_V5, gic_v5_get_gsi_domain_id);\n+\tgic_acpi_setup_kvm_info();\n \n \treturn 0;\n \ndiff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h\nindex fe49fb56dc3c92..d3dcf1598c4210 100644\n--- a/include/kvm/arm_vgic.h\n+++ b/include/kvm/arm_vgic.h\n@@ -40,6 +40,8 @@\n * in KVM for now. At a future stage, this can be bumped up to 128, if required.\n */\n #define VGIC_V5_NR_PRIVATE_IRQS\t64\n+#define VGIC_V5_DEFAULT_NR_SPIS\t32\n+#define VGIC_V5_MAX_NR_SPIS\tBIT(16)\n \n #define is_v5_type(t, i)\t(FIELD_GET(GICV5_HWIRQ_TYPE, (i)) == (t))\n \n@@ -157,9 +159,16 @@ struct vgic_global {\n \t/* Maintenance IRQ number */\n \tunsigned int\t\tmaint_irq;\n \n-\t/* maximum number of VCPUs allowed (GICv2 limits us to 8) */\n+\t/*\n+\t * Maximum number of VCPUs exposed before userspace has selected a\n+\t * VGIC model. Individual VGIC models can impose a lower limit\n+\t * (GICv2 limits us to 8).\n+\t */\n \tint\t\t\tmax_gic_vcpus;\n \n+\t/* Maximum number of VCPUs allowed for a GICv5 VM. */\n+\tint\t\t\tmax_gicv5_vcpus;\n+\n \t/* Only needed for the legacy KVM_CREATE_IRQCHIP */\n \tbool\t\t\tcan_emulate_gicv2;\n \n@@ -182,6 +191,32 @@ struct vgic_global {\n \tstruct {\n \t\tDECLARE_BITMAP(impl_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);\n \t} vgic_v5_ppi_caps;\n+\n+\t/* GICv5 IRS capabilities */\n+\tstruct {\n+\t\t/* Base address of the host IRS's CONFIG_FRAME */\n+\t\tvoid __iomem\t*irs_base;\n+\n+\t\t/* IST Caps */\n+\t\tu8\t\tist_id_bits;\n+\t\tbool\t\tist_levels;\n+\t\tu8\t\tist_l2sz;\n+\t\tbool\t\tistmd;\n+\t\tu8\t\tistmd_sz;\n+\n+\t\t/* LPI only */\n+\t\tu8\t\tmin_lpi_id_bits;\n+\n+\t\t/* VM Table, VPE Table */\n+\t\tbool\t\ttwo_level_vmt_support;\n+\t\tu32\t\tmax_vms;\n+\t\tu32\t\tmax_vpes;\n+\t\tu16\t\tvmd_size;\n+\t\tu16\t\tvped_size;\n+\n+\t\t/* Is the IRS coherent with us, or not? */\n+\t\tbool\t\tnon_coherent;\n+\t} vgic_v5_irs_caps;\n };\n \n extern struct vgic_global kvm_vgic_global_state;\n@@ -216,6 +251,12 @@ struct irq_ops {\n \t */\n \tbool (*get_input_level)(int vintid);\n \n+\t/*\n+\t * Function pointer to directly update hardware pending state after the\n+\t * VGIC shadow pending state has changed.\n+\t */\n+\tbool (*set_pending_state)(struct kvm_vcpu *vcpu, struct vgic_irq *irq);\n+\n \t/*\n \t * Function pointer to override the queuing of an IRQ.\n \t */\n@@ -290,7 +331,8 @@ enum iodev_type {\n \tIODEV_CPUIF,\n \tIODEV_DIST,\n \tIODEV_REDIST,\n-\tIODEV_ITS\n+\tIODEV_ITS,\n+\tIODEV_GICV5_IRS\n };\n \n struct vgic_io_device {\n@@ -348,6 +390,8 @@ struct vgic_redist_region {\n \tstruct list_head list;\n };\n \n+#define VGIC_V5_VM_ID_INVAL\t\t(-1)\n+\n struct vgic_v5_vm {\n \t/*\n \t * We only expose a subset of PPIs to the guest. This subset is a\n@@ -366,6 +410,93 @@ struct vgic_v5_vm {\n \t * convenient way to do that).\n \t */\n \tDECLARE_BITMAP(vgic_ppi_hmr, VGIC_V5_NR_PRIVATE_IRQS);\n+\n+\tstruct fwnode_handle\t*fwnode;\n+\tstruct irq_domain\t*domain;\n+\tint\t\t\tvpe_db_base;\n+\tu32\t\t\tvm_id;\n+\tbool\t\t\tvmte_allocated;\n+};\n+\n+/*** GICv5 ***/\n+struct vgic_v5_irs {\n+\t/* base addresses in guest physical address space: */\n+\tgpa_t vgic_v5_irs_base;\n+\n+\tstruct vgic_io_device iodev;\n+\tstruct kvm_device *dev;\n+\n+\t/* IRS state - used for registers etc */\n+\tstruct {\n+\t\tu8 domain;\n+\t\tu8 pa_range;\n+\t\tbool virt;\n+\t\tbool setlpi;\n+\t\tbool mec;\n+\t\tbool mpam;\n+\t\tbool swe;\n+\t\tu16 irs_id;\n+\t} idr0;\n+\n+\tstruct {\n+\t\t/* PE_CNT is populated from online_vcpus at runtime */\n+\t\tu8 priority_bits;\n+\t} idr1;\n+\n+\tstruct {\n+\t\tu8 id_bits;\n+\t\tu8 min_lpi_id_bits;\n+\t\tbool ist_levels;\n+\t\tu8 ist_l2sz;\n+\t\tbool istmd;\n+\t\tu8 istmd_sz;\n+\t} idr2;\n+\n+\tstruct {\n+\t\tu32 spi_range;\n+\t} idr5;\n+\n+\tstruct {\n+\t\tu32 spi_irs_range;\n+\t} idr6;\n+\n+\tstruct {\n+\t\tu32 spi_base;\n+\t} idr7;\n+\n+\tstruct {\n+\t\tu8 sh;\n+\t\tu8 oc;\n+\t\tu8 ic;\n+\t\tbool ist_ra;\n+\t\tbool ist_wa;\n+\t\tbool vmt_ra;\n+\t\tbool vpet_ra;\n+\t\tbool vmd_ra;\n+\t\tbool vmd_wa;\n+\t\tbool vped_ra;\n+\t\tbool vped_wa;\n+\t} cr1;\n+\n+\tstruct {\n+\t\tu32 id;\n+\t} spi_selr;\n+\n+\tstruct {\n+\t\tu32 iaffid;\n+\t} pe_selr;\n+\n+\tstruct {\n+\t\tu8 lpi_id_bits;\n+\t\tu8 l2sz;\n+\t\tu8 istsz;\n+\t\tbool structure;\n+\t} ist_cfgr;\n+\n+\tstruct {\n+\t\tbool valid;\n+\t\tu64 addr;\n+\t} ist_baser;\n };\n \n struct vgic_dist {\n@@ -445,6 +576,11 @@ struct vgic_dist {\n \t * GICv5 per-VM data.\n \t */\n \tstruct vgic_v5_vm\tgicv5_vm;\n+\n+\t/*\n+\t * GICv5 IRS data. Dynamically allocated due to the size.\n+\t */\n+\tstruct vgic_v5_irs\t*vgic_v5_irs_data;\n };\n \n struct vgic_v2_cpu_if {\n@@ -495,6 +631,9 @@ struct vgic_v5_cpu_if {\n \t */\n \tu64\tvgic_icsr;\n \n+\t/* The contextr used to make VPEs resident and non-resident */\n+\tu64\tvgic_contextr;\n+\n \tstruct gicv5_vpe gicv5_vpe;\n };\n \n@@ -601,10 +740,11 @@ void kvm_vgic_process_async_update(struct kvm_vcpu *vcpu);\n void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg, bool allow_group1);\n \n /**\n- * kvm_vgic_get_max_vcpus - Get the maximum number of VCPUs allowed by HW\n+ * kvm_vgic_get_max_vcpus - Get the pre-VGIC-selection VCPU limit\n *\n- * The host's GIC naturally limits the maximum amount of VCPUs a guest\n- * can use.\n+ * Userspace can query KVM_CAP_MAX_VCPUS before selecting a VGIC model, so\n+ * expose the highest model-specific limit and let kvm_vgic_create() enforce\n+ * the selected model's actual limit.\n */\n static inline int kvm_vgic_get_max_vcpus(void)\n {\n@@ -630,7 +770,7 @@ int vgic_v4_load(struct kvm_vcpu *vcpu);\n void vgic_v4_commit(struct kvm_vcpu *vcpu);\n int vgic_v4_put(struct kvm_vcpu *vcpu);\n \n-int vgic_v5_finalize_ppi_state(struct kvm *kvm);\n+int vgic_v5_finalize_ppi_state(struct kvm_vcpu *vcpu);\n bool vgic_v5_ppi_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq,\n \t\t\t\t unsigned long flags);\n void vgic_v5_set_ppi_dvi(struct kvm_vcpu *vcpu, struct vgic_irq *irq, bool dvi);\ndiff --git a/include/linux/irqchip/arm-gic-v5.h b/include/linux/irqchip/arm-gic-v5.h\nindex f78787e654f4c6..8b6c497d3b5902 100644\n--- a/include/linux/irqchip/arm-gic-v5.h\n+++ b/include/linux/irqchip/arm-gic-v5.h\n@@ -62,20 +62,34 @@\n #define GICV5_OUTER_SHARE\t\t0b10\n #define GICV5_INNER_SHARE\t\t0b11\n \n+#define GICV5_AIDR_COMPONENT_IRS\t0b00\n+#define GICV5_AIDR_COMPONENT_ITS\t0b01\n+#define GICV5_AIDR_COMPONENT_IWB\t0b10\n+\n+#define GICV5_AIDR_ARCH_MAJ_REV_V5\t0\n+#define GICV5_AIDR_ARCH_MIN_REV_V0\t0\n+\n /*\n * IRS registers and tables structures\n */\n #define GICV5_IRS_IDR0\t\t\t0x0000\n #define GICV5_IRS_IDR1\t\t\t0x0004\n #define GICV5_IRS_IDR2\t\t\t0x0008\n+#define GICV5_IRS_IDR3\t\t\t0x000c\n+#define GICV5_IRS_IDR4\t\t\t0x0010\n #define GICV5_IRS_IDR5\t\t\t0x0014\n #define GICV5_IRS_IDR6\t\t\t0x0018\n #define GICV5_IRS_IDR7\t\t\t0x001c\n+#define GICV5_IRS_IIDR\t\t\t0x0040\n+#define GICV5_IRS_AIDR\t\t\t0x0044\n #define GICV5_IRS_CR0\t\t\t0x0080\n #define GICV5_IRS_CR1\t\t\t0x0084\n #define GICV5_IRS_SYNCR\t\t\t0x00c0\n #define GICV5_IRS_SYNC_STATUSR\t\t0x00c4\n+#define GICV5_IRS_SPI_VMR\t\t0x0100\n #define GICV5_IRS_SPI_SELR\t\t0x0108\n+#define GICV5_IRS_SPI_DOMAINR\t\t0x010c\n+#define GICV5_IRS_SPI_RESAMPLER\t\t0x0110\n #define GICV5_IRS_SPI_CFGR\t\t0x0114\n #define GICV5_IRS_SPI_STATUSR\t\t0x0118\n #define GICV5_IRS_PE_SELR\t\t0x0140\n@@ -85,11 +99,51 @@\n #define GICV5_IRS_IST_CFGR\t\t0x0190\n #define GICV5_IRS_IST_STATUSR\t\t0x0194\n #define GICV5_IRS_MAP_L2_ISTR\t\t0x01c0\n-\n+#define GICV5_IRS_VMT_BASER\t\t0x0200\n+#define GICV5_IRS_VMT_CFGR\t\t0x0210\n+#define GICV5_IRS_VMT_STATUSR\t\t0x0214\n+#define GICV5_IRS_VPE_SELR\t\t0x0240\n+#define GICV5_IRS_VPE_DBR\t\t0x0248\n+#define GICV5_IRS_VPE_HPPIR\t\t0x0250\n+#define GICV5_IRS_VPE_CR0\t\t0x0258\n+#define GICV5_IRS_VPE_STATUSR\t\t0x025c\n+#define GICV5_IRS_VM_DBR\t\t0x0280\n+#define GICV5_IRS_VM_SELR\t\t0x0288\n+#define GICV5_IRS_VM_STATUSR\t\t0x028c\n+#define GICV5_IRS_VMAP_L2_VMTR\t\t0x02c0\n+#define GICV5_IRS_VMAP_VMR\t\t0x02c8\n+#define GICV5_IRS_VMAP_VISTR\t\t0x02d0\n+#define GICV5_IRS_VMAP_L2_VISTR\t\t0x02d8\n+#define GICV5_IRS_VMAP_VPER\t\t0x02e0\n+#define GICV5_IRS_SAVE_VMR\t\t0x0300\n+#define GICV5_IRS_SAVE_VM_STATUSR\t0x0308\n+#define GICV5_IRS_MEC_IDR\t\t0x0340\n+#define GICV5_IRS_MEC_MECID_R\t\t0x0344\n+#define GICV5_IRS_MPAM_IDR\t\t0x0380\n+#define GICV5_IRS_MPAM_PARTID_R\t\t0x0384\n+#define GICV5_IRS_SWERR_STATUSR\t\t0x03c0\n+#define GICV5_IRS_SWERR_SYNDROMER0\t0x03c8\n+#define GICV5_IRS_SWERR_SYNDROMER1\t0x03d0\n+\n+#define GICV5_IRS_IDR0_IRSID\t\tGENMASK(31, 16)\n+#define GICV5_IRS_IDR0_SWE\t\tBIT(12)\n+#define GICV5_IRS_IDR0_MPAM\t\tBIT(11)\n+#define GICV5_IRS_IDR0_MEC\t\tBIT(10)\n+#define GICV5_IRS_IDR0_SETLPI\t\tBIT(9)\n+#define GICV5_IRS_IDR0_VIRT_ONE_N\tBIT(8)\n+#define GICV5_IRS_IDR0_ONE_N\t\tBIT(7)\n #define GICV5_IRS_IDR0_VIRT\t\tBIT(6)\n+#define GICV5_IRS_IDR0_PA_RANGE\t\tGENMASK(5, 2)\n+#define GICV5_IRS_IDR0_INT_DOM\t\tGENMASK(1, 0)\n+\n+#define GICV5_IRS_IDR0_INT_DOM_SECURE\t\t0b00\n+#define GICV5_IRS_IDR0_INT_DOM_NON_SECURE\t0b01\n+#define GICV5_IRS_IDR0_INT_DOM_EL3\t\t0b10\n+#define GICV5_IRS_IDR0_INT_DOM_REALM\t\t0b11\n \n #define GICV5_IRS_IDR1_PRIORITY_BITS\tGENMASK(22, 20)\n #define GICV5_IRS_IDR1_IAFFID_BITS\tGENMASK(19, 16)\n+#define GICV5_IRS_IDR1_PE_CNT\t\tGENMASK(15, 0)\n \n #define GICV5_IRS_IDR1_PRIORITY_BITS_1BITS\t0b000\n #define GICV5_IRS_IDR1_PRIORITY_BITS_2BITS\t0b001\n@@ -105,13 +159,30 @@\n #define GICV5_IRS_IDR2_LPI\t\tBIT(5)\n #define GICV5_IRS_IDR2_ID_BITS\t\tGENMASK(4, 0)\n \n+#define GICV5_IRS_IST_L2SZ_SUPPORT_4KB(r)\tFIELD_GET(BIT(0), (r))\n+#define GICV5_IRS_IST_L2SZ_SUPPORT_16KB(r)\tFIELD_GET(BIT(1), (r))\n+#define GICV5_IRS_IST_L2SZ_SUPPORT_64KB(r)\tFIELD_GET(BIT(2), (r))\n+\n+#define GICV5_IRS_IDR3_VMT_LEVELS\tBIT(10)\n+#define GICV5_IRS_IDR3_VM_ID_BITS\tGENMASK(9, 5)\n+#define GICV5_IRS_IDR3_VMD_SZ\t\tGENMASK(4, 1)\n+#define GICV5_IRS_IDR3_VMD\t\tBIT(0)\n+\n+#define GICV5_IRS_IDR4_VPE_ID_BITS\tGENMASK(9, 6)\n+#define GICV5_IRS_IDR4_VPED_SZ\t\tGENMASK(5, 0)\n+\n #define GICV5_IRS_IDR5_SPI_RANGE\tGENMASK(24, 0)\n #define GICV5_IRS_IDR6_SPI_IRS_RANGE\tGENMASK(24, 0)\n #define GICV5_IRS_IDR7_SPI_BASE\t\tGENMASK(23, 0)\n \n-#define GICV5_IRS_IST_L2SZ_SUPPORT_4KB(r)\tFIELD_GET(BIT(11), (r))\n-#define GICV5_IRS_IST_L2SZ_SUPPORT_16KB(r)\tFIELD_GET(BIT(12), (r))\n-#define GICV5_IRS_IST_L2SZ_SUPPORT_64KB(r)\tFIELD_GET(BIT(13), (r))\n+#define GICV5_IRS_IIDR_PRODUCT_ID\tGENMASK(31, 20)\n+#define GICV5_IRS_IIDR_VARIANT\t\tGENMASK(19, 16)\n+#define GICV5_IRS_IIDR_REVISION\t\tGENMASK(15, 12)\n+#define GICV5_IRS_IIDR_IMPLEMENTER\tGENMASK(11, 0)\n+\n+#define GICV5_IRS_AIDR_COMPONENT\tGENMASK(11, 8)\n+#define GICV5_IRS_AIDR_ARCHMAJORREV\tGENMASK(7, 4)\n+#define GICV5_IRS_AIDR_ARCHMINORREV\tGENMASK(3, 0)\n \n #define GICV5_IRS_CR0_IDLE\t\tBIT(1)\n #define GICV5_IRS_CR0_IRSEN\t\tBIT(0)\n@@ -134,21 +205,39 @@\n \n #define GICV5_IRS_SYNC_STATUSR_IDLE\tBIT(0)\n \n-#define GICV5_IRS_SPI_STATUSR_V\t\tBIT(1)\n-#define GICV5_IRS_SPI_STATUSR_IDLE\tBIT(0)\n+#define GICV5_IRS_SPI_VMR_VIRT\t\tBIT_ULL(63)\n+#define GICV5_IRS_SPI_VMR_VM_ID\t\tGENMASK_ULL(15, 0)\n \n #define GICV5_IRS_SPI_SELR_ID\t\tGENMASK(23, 0)\n \n+#define GICV5_IRS_SPI_DOMAINR_DOMAIN\tGENMASK(1, 0)\n+\n+#define GICV5_IRS_SPI_DOMAINR_DOMAIN_SECURE\t0b00\n+#define GICV5_IRS_SPI_DOMAINR_DOMAIN_NON_SECURE\t0b01\n+#define GICV5_IRS_SPI_DOMAINR_DOMAIN_EL3\t0b10\n+#define GICV5_IRS_SPI_DOMAINR_DOMAIN_REALM\t0b11\n+\n+#define GICV5_IRS_SPI_RESAMPLER_ID\tGENMASK(23, 0)\n+\n #define GICV5_IRS_SPI_CFGR_TM\t\tBIT(0)\n \n+#define GICV5_IRS_SPI_CFGR_TM_EDGE\t0b0\n+#define GICV5_IRS_SPI_CFGR_TM_LEVEL\t0b1\n+\n+#define GICV5_IRS_SPI_STATUSR_V\t\tBIT(1)\n+#define GICV5_IRS_SPI_STATUSR_IDLE\tBIT(0)\n+\n #define GICV5_IRS_PE_SELR_IAFFID\tGENMASK(15, 0)\n \n+#define GICV5_IRS_PE_STATUSR_ONLINE\tBIT(2)\n #define GICV5_IRS_PE_STATUSR_V\t\tBIT(1)\n #define GICV5_IRS_PE_STATUSR_IDLE\tBIT(0)\n \n #define GICV5_IRS_PE_CR0_DPS\t\tBIT(0)\n \n-#define GICV5_IRS_IST_STATUSR_IDLE\tBIT(0)\n+#define GICV5_IRS_IST_BASER_ADDR_MASK\tGENMASK_ULL(55, 6)\n+#define GICV5_IRS_IST_BASER_VALID\tBIT_ULL(0)\n+#define GICV5_IRS_IST_BASER_ADDR_SHIFT\t6ULL\n \n #define GICV5_IRS_IST_CFGR_STRUCTURE\tBIT(16)\n #define GICV5_IRS_IST_CFGR_ISTSZ\tGENMASK(8, 7)\n@@ -166,15 +255,111 @@\n #define GICV5_IRS_IST_CFGR_L2SZ_16K\t0b01\n #define GICV5_IRS_IST_CFGR_L2SZ_64K\t0b10\n \n-#define GICV5_IRS_IST_BASER_ADDR_MASK\tGENMASK_ULL(55, 6)\n-#define GICV5_IRS_IST_BASER_VALID\tBIT_ULL(0)\n+#define GICV5_IRS_IST_STATUSR_IDLE\tBIT(0)\n \n #define GICV5_IRS_MAP_L2_ISTR_ID\tGENMASK(23, 0)\n \n+#define GICV5_IRS_VMT_BASER_ADDR\tGENMASK_ULL(55, 3)\n+#define GICV5_IRS_VMT_BASER_ADDR_SHIFT\t3ULL\n+#define GICV5_IRS_VMT_BASER_VALID\tBIT_ULL(0)\n+\n+#define GICV5_IRS_VMT_CFGR_STRUCTURE_TWO_LEVEL\t0b1\n+#define GICV5_IRS_VMT_CFGR_STRUCTURE_LINEAR\t0b0\n+\n+#define GICV5_IRS_VMT_CFGR_STRUCTURE\tBIT(16)\n+#define GICV5_IRS_VMT_CFGR_VM_ID_BITS\tGENMASK(4, 0)\n+\n+#define GICV5_IRS_VMT_STATUSR_IDLE\tBIT(0)\n+\n+#define GICV5_IRS_VPE_SELR_S\t\tBIT_ULL(63)\n+#define GICV5_IRS_VPE_SELR_VPE_ID\tGENMASK_ULL(47, 32)\n+#define GICV5_IRS_VPE_SELR_VM_ID\tGENMASK_ULL(15, 0)\n+\n+#define GICV5_IRS_VPE_DBR_DBV\t\tBIT_ULL(63)\n+#define GICV5_IRS_VPE_DBR_REQ_DB\tBIT_ULL(62)\n+#define GICV5_IRS_VPE_DBR_DBPM\t\tGENMASK_ULL(36, 32)\n+#define GICV5_IRS_VPE_DBR_INTID\tGENMASK_ULL(23, 0)\n+\n+#define GICV5_IRS_VPE_HPPIR_HPPIV\tBIT_ULL(32)\n+#define GICV5_IRS_VPE_HPPIR_TYPE\tGENMASK_ULL(31, 29)\n+#define GICV5_IRS_VPE_HPPIR_ID\t\tGENMASK_ULL(23, 0)\n+\n+#define GICV5_IRS_VPE_CR0_DPS\t\tBIT(0)\n+\n+#define GICV5_IRS_VPE_STATUSR_V\t\tBIT(1)\n+#define GICV5_IRS_VPE_STATUSR_IDLE\tBIT(0)\n+\n+#define GICV5_IRS_VM_DBR_EN\t\tBIT_ULL(63)\n+#define GICV5_IRS_VM_DBR_VPE_ID\t\tGENMASK_ULL(15, 0)\n+\n+#define GICV5_IRS_VM_SELR_VM_ID\t\tGENMASK(15, 0)\n+\n+#define GICV5_IRS_VM_STATUSR_V\t\tBIT(1)\n+#define GICV5_IRS_VM_STATUSR_IDLE\tBIT(0)\n+\n+#define GICV5_IRS_VMAP_L2_VMTR_M\tBIT_ULL(63)\n+#define GICV5_IRS_VMAP_L2_VMTR_VM_ID\tGENMASK_ULL(15, 0)\n+\n+#define GICV5_IRS_VMAP_VMR_M\t\tBIT_ULL(63)\n+#define GICV5_IRS_VMAP_VMR_U\t\tBIT_ULL(62)\n+#define GICV5_IRS_VMAP_VMR_VM_ID\tGENMASK_ULL(15, 0)\n+\n+#define GICV5_IRS_VMAP_VISTR_M\t\tBIT_ULL(63)\n+#define GICV5_IRS_VMAP_VISTR_U\t\tBIT_ULL(62)\n+#define GICV5_IRS_VMAP_VISTR_VM_ID\tGENMASK_ULL(47, 32)\n+#define GICV5_IRS_VMAP_VISTR_TYPE\tGENMASK_ULL(31, 29)\n+\n+#define GICV5_IRS_VMAP_L2_VISTR_M\tBIT_ULL(63)\n+#define GICV5_IRS_VMAP_L2_VISTR_VM_ID\tGENMASK_ULL(47, 32)\n+#define GICV5_IRS_VMAP_L2_VISTR_TYPE\tGENMASK_ULL(31, 29)\n+#define GICV5_IRS_VMAP_L2_VISTR_ID\tGENMASK_ULL(23, 0)\n+\n+#define GICV5_IRS_VMAP_VPER_M\t\tBIT_ULL(63)\n+#define GICV5_IRS_VMAP_VPER_VM_ID\tGENMASK_ULL(47, 32)\n+#define GICV5_IRS_VMAP_VPER_VPE_ID\tGENMASK_ULL(15, 0)\n+\n+#define GICV5_IRS_SAVE_VMR_VM_ID\tGENMASK_ULL(15, 0)\n+#define GICV5_IRS_SAVE_VMR_Q\t\tBIT_ULL(62)\n+#define GICV5_IRS_SAVE_VMR_S\t\tBIT_ULL(63)\n+\n+#define GICV5_IRS_SAVE_VM_STATUSR_IDLE\tBIT(0)\n+#define GICV5_IRS_SAVE_VM_STATUSR_Q\tBIT(1)\n+\n+#define GICV5_IRS_MEC_IDR_MECIDSIZE\tGENMASK(3, 0)\n+\n+#define GICV5_IRS_MEC_MECID_R_MECID\tGENMASK(15, 0)\n+\n+#define GICV5_IRS_MPAM_IDR_HAS_MPAM_SP\tBIT(24)\n+#define GICV5_IRS_MPAM_IDR_PMG_MAX\tGENMASK(23, 16)\n+#define GICV5_IRS_MPAM_IDR_PARTID_MAX\tGENMASK(15, 0)\n+\n+#define GICV5_IRS_MPAM_PARTID_R_IDLE\tBIT(31)\n+#define GICV5_IRS_MPAM_PARTID_R_MPAM_SP\tGENMASK(25, 24)\n+#define GICV5_IRS_MPAM_PARTID_R_PMG\tGENMASK(23, 16)\n+#define GICV5_IRS_MPAM_PARTID_R_PARTID\tGENMASK(15, 0)\n+\n+#define GICV5_IRS_SWERR_STATUSR_IMP_EC\tGENMASK_ULL(31, 24)\n+#define GICV5_IRS_SWERR_STATUSR_EC\tGENMASK_ULL(23, 16)\n+#define GICV5_IRS_SWERR_STATUSR_OF\tBIT_ULL(3)\n+#define GICV5_IRS_SWERR_STATUSR_S1V\tBIT_ULL(2)\n+#define GICV5_IRS_SWERR_STATUSR_S0V\tBIT_ULL(1)\n+#define GICV5_IRS_SWERR_STATUSR_V\tBIT_ULL(0)\n+\n+#define GICV5_IRS_SWERR_SYNDROMER0_VIRTUAL\tBIT_ULL(63)\n+#define GICV5_IRS_SWERR_SYNDROMER0_TYPE\t\tGENMASK_ULL(62, 60)\n+#define GICV5_IRS_SWERR_SYNDROMER0_ID\t\tGENMASK_ULL(55, 32)\n+#define GICV5_IRS_SWERR_SYNDROMER0_VM_ID\tGENMASK_ULL(15, 0)\n+\n+#define GICV5_IRS_SWERR_SYNDROMER1_ADDR\tGENMASK_ULL(55, 3)\n+\n #define GICV5_ISTL1E_VALID\t\tBIT_ULL(0)\n+#define GICV5_IRS_ISTL1E_SIZE\t\t8UL\n \n #define GICV5_ISTL1E_L2_ADDR_MASK\tGENMASK_ULL(55, 12)\n \n+#define GICV5_IRS_SETLPIR\t\t0x0000\n+#define GICV5_IRS_SETLPIR_ID\t\tGENMASK(23, 0)\n+\n /*\n * ITS registers and tables structures\n */\n@@ -296,6 +481,42 @@\n #define GICV5_GSI_IWB_FRAME_ID\t\t\tGENMASK(28, 16)\n #define GICV5_GSI_IWB_WIRE\t\t\tGENMASK(15, 0)\n \n+/*\n+ * CoreSight identification registers - ordered by increasing offset.\n+ */\n+#define GICV5_CORESIGHT_DEVARCH\t\t0xffbc\n+#define GICV5_CORESIGHT_PIDR4\t\t0xffd0\n+#define GICV5_CORESIGHT_PIDR5\t\t0xffd4\n+#define GICV5_CORESIGHT_PIDR6\t\t0xffd8\n+#define GICV5_CORESIGHT_PIDR7\t\t0xffdc\n+#define GICV5_CORESIGHT_PIDR0\t\t0xffe0\n+#define GICV5_CORESIGHT_PIDR1\t\t0xffe4\n+#define GICV5_CORESIGHT_PIDR2\t\t0xffe8\n+#define GICV5_CORESIGHT_PIDR3\t\t0xffec\n+#define GICV5_CORESIGHT_CIDR0\t\t0xfff0\n+#define GICV5_CORESIGHT_CIDR1\t\t0xfff4\n+#define GICV5_CORESIGHT_CIDR2\t\t0xfff8\n+#define GICV5_CORESIGHT_CIDR3\t\t0xfffc\n+\n+#define GICV5_CORESIGHT_DEVARCH_VAL\t\t\t\\\n+\t(FIELD_PREP(GENMASK(31, 21), 0x23b) |\t\t\\\n+\t BIT(20) |\t\t\t\t\t\\\n+\t 0x5a19)\n+\n+#define GICV5_CORESIGHT_PIDR4_JEP106_CONT\t0x04\n+#define GICV5_CORESIGHT_PIDR5_RES0\t\t0x00\n+#define GICV5_CORESIGHT_PIDR6_RES0\t\t0x00\n+#define GICV5_CORESIGHT_PIDR7_RES0\t\t0x00\n+#define GICV5_CORESIGHT_PIDR0_PART_0\t\t0x4b\n+#define GICV5_CORESIGHT_PIDR1_DES_0_PART_1\t0xb0\n+#define GICV5_CORESIGHT_PIDR2_DES_1\t\t0x0b\n+#define GICV5_CORESIGHT_PIDR3_REVAND_CMOD\t0x00\n+\n+#define GICV5_CORESIGHT_CIDR0_VAL\t\t0x0d\n+#define GICV5_CORESIGHT_CIDR1_VAL\t\t0xf0\n+#define GICV5_CORESIGHT_CIDR2_VAL\t\t0x05\n+#define GICV5_CORESIGHT_CIDR3_VAL\t\t0xb1\n+\n /*\n * Global Data structures and functions\n */\n@@ -330,6 +551,8 @@ struct gicv5_irs_chip_data {\n \traw_spinlock_t\t\tspi_config_lock;\n };\n \n+#define IRS_FLAGS_NON_COHERENT\t\tBIT(0)\n+\n static inline int gicv5_wait_for_op_s_atomic(void __iomem *addr, u32 offset,\n \t\t\t\t\t const char *reg_s, u32 mask,\n \t\t\t\t\t u32 *val)\n@@ -377,6 +600,7 @@ void __init gicv5_free_lpi_domain(void);\n \n int gicv5_irs_of_probe(struct device_node *parent);\n int gicv5_irs_acpi_probe(void);\n+struct gicv5_irs_chip_data *gicv5_irs_get_chip_data(void);\n void gicv5_irs_remove(void);\n int gicv5_irs_enable(void);\n void gicv5_irs_its_probe(void);\n@@ -385,10 +609,13 @@ int gicv5_irs_cpu_to_iaffid(int cpu_id, u16 *iaffid);\n struct gicv5_irs_chip_data *gicv5_irs_lookup_by_spi_id(u32 spi_id);\n int gicv5_spi_irq_set_type(struct irq_data *d, unsigned int type);\n int gicv5_irs_iste_alloc(u32 lpi);\n+unsigned int gicv5_irs_l2_sz(u32 l2sz);\n void gicv5_irs_syncr(void);\n \n /* Embedded in kvm.arch */\n struct gicv5_vpe {\n+\tint\t\t\tdb;\n+\tbool\t\t\tdb_fired;\n \tbool\t\t\tresident;\n };\n \n@@ -427,4 +654,15 @@ void gicv5_deinit_lpis(void);\n \n void __init gicv5_its_of_probe(struct device_node *parent);\n void __init gicv5_its_acpi_probe(void);\n+\n+enum gicv5_vcpu_cmd {\n+\tVMT_L2_MAP,\t\t/* Map in a L2 VMT - *may* happen on VM init */\n+\tVMTE_MAKE_VALID,\t/* Make the VMTE valid */\n+\tVMTE_MAKE_INVALID,\t/* Make the VMTE (et al.) invalid */\n+\tVPE_MAKE_VALID,\t\t/* No corresponding invalid */\n+\tSPI_VIST_MAKE_VALID,\t/* No corresponding invalid */\n+\tLPI_VIST_MAKE_VALID,\t/* Triggered by a guest */\n+\tLPI_VIST_MAKE_INVALID,\t/* Triggered by a guest */\n+};\n+\n #endif\ndiff --git a/include/linux/irqchip/arm-vgic-info.h b/include/linux/irqchip/arm-vgic-info.h\nindex 67d9d960273b99..f05370e2debf44 100644\n--- a/include/linux/irqchip/arm-vgic-info.h\n+++ b/include/linux/irqchip/arm-vgic-info.h\n@@ -38,6 +38,11 @@ struct gic_kvm_info {\n \tbool\t\thas_v4_1;\n \t/* Deactivation impared, subpar stuff */\n \tbool\t\tno_hw_deactivation;\n+\t/* GICv5 IRS base */\n+\tstruct {\n+\t\tvoid __iomem\t*base;\n+\t\tbool\t\tnon_coherent;\n+\t}\t\tgicv5_irs;\n };\n \n #ifdef CONFIG_KVM\ndiff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h\nindex 419011097fa8e2..df6356fb45b089 100644\n--- a/include/uapi/linux/kvm.h\n+++ b/include/uapi/linux/kvm.h\n@@ -1191,6 +1191,13 @@ struct kvm_device_attr {\n \t__u64\taddr;\t\t/* userspace address of attr data */\n };\n \n+struct kvm_vgic_v5_ist {\n+\t__u64\tspi_ist_addr;\n+\t__u64\tspi_ist_size;\n+\t__u64\tlpi_ist_addr;\n+\t__u64\tlpi_ist_size;\n+};\n+\n #define KVM_DEV_VFIO_FILE\t\t\t1\n \n #define KVM_DEV_VFIO_FILE_ADD\t\t\t1\ndiff --git a/tools/arch/arm64/include/uapi/asm/kvm.h b/tools/arch/arm64/include/uapi/asm/kvm.h\nindex 1c13bfa2d38aad..1b9bbeab18a4e0 100644\n--- a/tools/arch/arm64/include/uapi/asm/kvm.h\n+++ b/tools/arch/arm64/include/uapi/asm/kvm.h\n@@ -97,6 +97,11 @@ struct kvm_regs {\n #define KVM_VGIC_V3_REDIST_SIZE\t\t(2 * SZ_64K)\n #define KVM_VGIC_V3_ITS_SIZE\t\t(2 * SZ_64K)\n \n+/* Supported VGICv5 address types */\n+#define KVM_VGIC_V5_ADDR_TYPE_IRS\t6\n+\n+#define KVM_VGIC_V5_IRS_SIZE\t\t(2 * SZ_64K)\n+\n #define KVM_ARM_VCPU_POWER_OFF\t\t0 /* CPU is started in OFF state */\n #define KVM_ARM_VCPU_EL1_32BIT\t\t1 /* CPU running a 32bit VM */\n #define KVM_ARM_VCPU_PSCI_0_2\t\t2 /* CPU uses PSCI v0.2 */\n@@ -417,6 +422,8 @@ enum {\n #define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7\n #define KVM_DEV_ARM_VGIC_GRP_ITS_REGS 8\n #define KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ 9\n+#define KVM_DEV_ARM_VGIC_GRP_IRS_REGS\t10\n+#define KVM_DEV_ARM_VGIC_GRP_IST\t11\n #define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT\t10\n #define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \\\n \t\t\t(0x3fffffULL \u003c\u003c KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)\ndiff --git a/tools/testing/selftests/kvm/arm64/vgic_v5.c b/tools/testing/selftests/kvm/arm64/vgic_v5.c\nindex 96cfd6bb32f6f2..05d8be8d75b511 100644\n--- a/tools/testing/selftests/kvm/arm64/vgic_v5.c\n+++ b/tools/testing/selftests/kvm/arm64/vgic_v5.c\n@@ -13,6 +13,28 @@\n #include \"vgic.h\"\n \n #define NR_VCPUS\t\t1\n+#define VGIC_V5_LPI_NR_VCPUS\t2\n+#define VGIC_V5_LPI_MEMSLOT\t1\n+#define VGIC_V5_NR_PRIVATE_IRQS\t64\n+#define VGIC_V5_DEFAULT_NR_SPIS\t32\n+#define VGIC_V5_MAX_NR_SPIS\tBIT(16)\n+#define VGIC_V5_TEST_NR_SPIS\t\t64\n+#define VGIC_V5_IRS_SIZE\t\t0x20000\n+#define VGIC_V5_IRS_WAIT_RETRIES\t1000000\n+#define VGIC_V5_LPI_IST_BASE_GPA\t(GICV5_IRS_CONFIG_BASE_GPA + VGIC_V5_IRS_SIZE)\n+#define VGIC_V5_LPI_IST_SIZE\t\t0x10000\n+#define LPI_TEST_TO_VPE1\t\t0\n+#define LPI_TEST_TO_VPE0\t\t1\n+#define RESTORE_TEST_LPI_PENDING\t\t0\n+#define RESTORE_TEST_LPI_ENABLED\t\t1\n+#define RESTORE_TEST_SPI_PRIO_HIGH\t\t0\n+#define RESTORE_TEST_SPI_PRIO_LOW\t\t1\n+#define RESTORE_TEST_SPI_PENDING_DISABLED\t2\n+#define RESTORE_TEST_SPI_ENABLED\t\t3\n+#define VGIC_V5_IST_PRIO_HIGH\t\t0x08\n+#define VGIC_V5_IST_PRIO_LOW\t\t0x10\n+\n+static u64 max_phys_size;\n \n struct vm_gic {\n \tstruct kvm_vm *vm;\n@@ -24,74 +46,1469 @@ struct vm_gic {\n #define GUEST_CMD_IRQ_DIEOI\t11\n #define GUEST_CMD_IS_AWAKE\t12\n #define GUEST_CMD_IS_READY\t13\n+#define GUEST_CMD_LPI_SENT\t14\n+#define GUEST_CMD_LPI_REPLIED\t15\n+#define GUEST_CMD_IST_SOURCE_READY\t16\n+#define GUEST_CMD_IST_SPI_INJECT_READY\t17\n+#define GUEST_CMD_IST_LPI_INJECT_READY\t18\n+\n+static struct kvm_vgic_v5_ist vgic_v5_ist_attr(void *spi_ist, size_t spi_size,\n+\t\t\t\t\t void *lpi_ist, size_t lpi_size)\n+{\n+\treturn (struct kvm_vgic_v5_ist) {\n+\t\t.spi_ist_addr = (uintptr_t)spi_ist,\n+\t\t.spi_ist_size = spi_size,\n+\t\t.lpi_ist_addr = (uintptr_t)lpi_ist,\n+\t\t.lpi_ist_size = lpi_size,\n+\t};\n+}\n+\n+static u32 spi_line_expected;\n+static bool spi_line_level_sensitive;\n+static bool lpi_ist_ready;\n+static u32 ist_restore_irq_count[VGIC_V5_LPI_NR_VCPUS];\n+\n+static u64 gicv5_hwirq(u32 type, u32 intid)\n+{\n+\treturn FIELD_PREP(GICV5_HWIRQ_TYPE, type) |\n+\t FIELD_PREP(GICV5_HWIRQ_ID, intid);\n+}\n+\n+static u64 gicv5_lpi_hwirq(u32 lpi)\n+{\n+\treturn gicv5_hwirq(GICV5_HWIRQ_TYPE_LPI, lpi);\n+}\n+\n+static u64 gicv5_spi_hwirq(u32 spi)\n+{\n+\treturn gicv5_hwirq(GICV5_HWIRQ_TYPE_SPI, spi);\n+}\n+\n+static void gicv5_setup_and_enable_hwirq(u64 hwirq, u32 target_vpe)\n+{\n+\tu64 val;\n+\n+\tval = hwirq | FIELD_PREP(GICV5_GIC_CDPRI_PRIORITY_MASK,\n+\t\t\t\t GICV5_IRQ_DEFAULT_PRI);\n+\tgic_insn(val, CDPRI);\n+\n+\tval = hwirq | FIELD_PREP(GICV5_GIC_CDAFF_IAFFID_MASK, target_vpe);\n+\tgic_insn(val, CDAFF);\n+\n+\tgic_insn(hwirq, CDEN);\n+}\n+\n+static void gicv5_set_hwirq_priority(u64 hwirq, u32 priority)\n+{\n+\tu64 val;\n+\n+\tval = hwirq | FIELD_PREP(GICV5_GIC_CDPRI_PRIORITY_MASK, priority);\n+\tgic_insn(val, CDPRI);\n+}\n+\n+static void gicv5_set_hwirq_affinity(u64 hwirq, u32 target_vpe)\n+{\n+\tu64 val;\n+\n+\tval = hwirq | FIELD_PREP(GICV5_GIC_CDAFF_IAFFID_MASK, target_vpe);\n+\tgic_insn(val, CDAFF);\n+}\n+\n+static void gicv5_enable_hwirq(u64 hwirq)\n+{\n+\tgic_insn(hwirq, CDEN);\n+}\n+\n+static void gicv5_enable_spi(u32 spi, u32 target_vpe)\n+{\n+\tgicv5_setup_and_enable_hwirq(gicv5_spi_hwirq(spi), target_vpe);\n+}\n+\n+static void gicv5_enable_lpi(u32 lpi, u32 target_vpe)\n+{\n+\tgicv5_setup_and_enable_hwirq(gicv5_lpi_hwirq(lpi), target_vpe);\n+}\n+\n+static void gicv5_send_lpi(u32 lpi)\n+{\n+\tu64 hwirq = gicv5_lpi_hwirq(lpi);\n+\n+\tgic_insn(hwirq | GICV5_GIC_CDPEND_PENDING_MASK, CDPEND);\n+}\n+\n+static void gicv5_wait_for_irs_idle(u32 reg, u32 idle)\n+{\n+\tint i;\n+\n+\tfor (i = 0; i \u003c VGIC_V5_IRS_WAIT_RETRIES; i++) {\n+\t\tif (readl(GICV5_IRS_CONFIG_BASE_GVA + reg) \u0026 idle)\n+\t\t\treturn;\n+\n+\t\tcpu_relax();\n+\t}\n+\n+\tGUEST_FAIL(\"IRS operation did not become idle\");\n+}\n+\n+static void gicv5_configure_spi(u32 spi, bool level)\n+{\n+\tu32 val;\n+\n+\tval = FIELD_PREP(GICV5_IRS_SPI_SELR_ID, spi);\n+\twritel(val, GICV5_IRS_CONFIG_BASE_GVA + GICV5_IRS_SPI_SELR);\n+\tgicv5_wait_for_irs_idle(GICV5_IRS_SPI_STATUSR,\n+\t\t\t\tGICV5_IRS_SPI_STATUSR_IDLE);\n+\n+\tval = level ? GICV5_IRS_SPI_CFGR_TM : 0;\n+\twritel(val, GICV5_IRS_CONFIG_BASE_GVA + GICV5_IRS_SPI_CFGR);\n+\tgicv5_wait_for_irs_idle(GICV5_IRS_SPI_STATUSR,\n+\t\t\t\tGICV5_IRS_SPI_STATUSR_IDLE);\n+}\n+\n+static u32 gicv5_lpi_istsz(u32 idr2, u32 lpi_id_bits)\n+{\n+\tif (!(idr2 \u0026 GICV5_IRS_IDR2_ISTMD))\n+\t\treturn GICV5_IRS_IST_CFGR_ISTSZ_4;\n+\n+\tif (lpi_id_bits \u003e= FIELD_GET(GICV5_IRS_IDR2_ISTMD_SZ, idr2))\n+\t\treturn GICV5_IRS_IST_CFGR_ISTSZ_16;\n+\n+\treturn GICV5_IRS_IST_CFGR_ISTSZ_8;\n+}\n+\n+static void gicv5_configure_lpi_ist(void)\n+{\n+\tu32 idr2, min_lpi_id_bits, lpi_id_bits, id_bits, istsz;\n+\tu64 val;\n+\n+\tidr2 = readl(GICV5_IRS_CONFIG_BASE_GVA + GICV5_IRS_IDR2);\n+\tGUEST_ASSERT(idr2 \u0026 GICV5_IRS_IDR2_LPI);\n+\n+\tmin_lpi_id_bits = FIELD_GET(GICV5_IRS_IDR2_MIN_LPI_ID_BITS, idr2);\n+\tid_bits = FIELD_GET(GICV5_IRS_IDR2_ID_BITS, idr2);\n+\t/* Default to 32 LPIs, unless the min requirement is higher */\n+\tlpi_id_bits = max(5U, min_lpi_id_bits);\n+\tGUEST_ASSERT(lpi_id_bits \u003c= id_bits);\n+\n+\tistsz = gicv5_lpi_istsz(idr2, lpi_id_bits);\n+\t/*\n+\t * We allocate a fixed-size IST buffer on the host to avoid dynamic\n+\t * allocation from the guest. Make sure it fits the linear IST described\n+\t * by IDR2 before programming IRS_IST_BASER.\n+\t */\n+\tGUEST_ASSERT((BIT_ULL(lpi_id_bits) \u003c\u003c (istsz + 2)) \u003c= VGIC_V5_LPI_IST_SIZE);\n+\n+\tval = FIELD_PREP(GICV5_IRS_IST_CFGR_LPI_ID_BITS, lpi_id_bits) |\n+\t FIELD_PREP(GICV5_IRS_IST_CFGR_ISTSZ, istsz);\n+\twritel(val, GICV5_IRS_CONFIG_BASE_GVA + GICV5_IRS_IST_CFGR);\n+\n+\tval = FIELD_PREP(GICV5_IRS_IST_BASER_ADDR_MASK,\n+\t\t\t VGIC_V5_LPI_IST_BASE_GPA \u003e\u003e GICV5_IRS_IST_BASER_ADDR_SHIFT);\n+\tval |= GICV5_IRS_IST_BASER_VALID;\n+\twriteq(val, GICV5_IRS_CONFIG_BASE_GVA + GICV5_IRS_IST_BASER);\n+\n+\tGUEST_ASSERT_EQ(readl(GICV5_IRS_CONFIG_BASE_GVA + GICV5_IRS_IST_STATUSR),\n+\t\t\tGICV5_IRS_IST_STATUSR_IDLE);\n+}\n+\n+static void gicv5_enable_irs(void)\n+{\n+\twritel(GICV5_IRS_CR0_IRSEN,\n+\t GICV5_IRS_CONFIG_BASE_GVA + GICV5_IRS_CR0);\n+\tgicv5_wait_for_irs_idle(GICV5_IRS_CR0, GICV5_IRS_CR0_IDLE);\n+}\n+\n+static void gicv5_configure_test_lpis(void)\n+{\n+\tgicv5_enable_lpi(LPI_TEST_TO_VPE1, 1);\n+\tgicv5_enable_lpi(LPI_TEST_TO_VPE0, 0);\n+}\n+\n+static void gicv5_set_pcr(u32 priority)\n+{\n+\tu64 pcr;\n+\n+\tpcr = FIELD_PREP(ICC_PCR_EL1_PRIORITY, priority);\n+\twrite_sysreg_s(pcr, SYS_ICC_PCR_EL1);\n+\tisb();\n+}\n+\n+static void guest_ppi_irq_handler(struct ex_regs *regs)\n+{\n+\tbool valid;\n+\tu32 hwirq;\n+\tu64 ia;\n+\tstatic int count;\n+\n+\t/*\n+\t * We have pending interrupts. Should never actually enter WFI\n+\t * here!\n+\t */\n+\twfi();\n+\tGUEST_SYNC(GUEST_CMD_IS_AWAKE);\n+\n+\tia = gicr_insn(CDIA);\n+\tvalid = GICV5_GICR_CDIA_VALID(ia);\n+\n+\tGUEST_SYNC(GUEST_CMD_IRQ_CDIA);\n+\n+\tif (!valid)\n+\t\treturn;\n+\n+\tgsb_ack();\n+\tisb();\n+\n+\thwirq = FIELD_GET(GICV5_GICR_CDIA_INTID, ia);\n+\n+\tgic_insn(hwirq, CDDI);\n+\tgic_insn(0, CDEOI);\n+\n+\tGUEST_SYNC(GUEST_CMD_IRQ_DIEOI);\n+\n+\tif (++count \u003e= 2)\n+\t\tGUEST_DONE();\n+\n+\t/* Ask for the next interrupt to be injected */\n+\tGUEST_SYNC(GUEST_CMD_IS_READY);\n+}\n+\n+static void guest_code(void)\n+{\n+\tlocal_irq_disable();\n+\n+\tgicv5_cpu_enable_interrupts();\n+\tlocal_irq_enable();\n+\n+\t/* Enable the SW_PPI (3) */\n+\twrite_sysreg_s(BIT_ULL(3), SYS_ICC_PPI_ENABLER0_EL1);\n+\n+\t/* Ask for the first interrupt to be injected */\n+\tGUEST_SYNC(GUEST_CMD_IS_READY);\n+\n+\t/* Loop forever waiting for interrupts */\n+\twhile (1);\n+}\n+\n+static void guest_spi_irq_handler(struct ex_regs *regs)\n+{\n+\tbool valid;\n+\tu32 hwirq;\n+\tu64 ia;\n+\n+\tia = gicr_insn(CDIA);\n+\tvalid = GICV5_GICR_CDIA_VALID(ia);\n+\n+\tif (!valid)\n+\t\treturn;\n+\n+\tgsb_ack();\n+\tisb();\n+\n+\thwirq = FIELD_GET(GICV5_GICR_CDIA_INTID, ia);\n+\n+\tGUEST_ASSERT_EQ(hwirq, gicv5_spi_hwirq(READ_ONCE(spi_line_expected)));\n+\n+\tgic_insn(hwirq, CDDI);\n+\tgic_insn(0, CDEOI);\n+\n+\tGUEST_DONE();\n+}\n+\n+static void guest_spi_line_code(void)\n+{\n+\tlocal_irq_disable();\n+\n+\tgicv5_enable_irs();\n+\tgicv5_cpu_enable_interrupts();\n+\tgicv5_configure_spi(READ_ONCE(spi_line_expected),\n+\t\t\t READ_ONCE(spi_line_level_sensitive));\n+\tgicv5_enable_spi(READ_ONCE(spi_line_expected), 0);\n+\n+\tlocal_irq_enable();\n+\n+\tGUEST_SYNC(GUEST_CMD_IS_READY);\n+\n+\twhile (1)\n+\t\twfi();\n+}\n+\n+static void guest_lpi_irq_handler(struct ex_regs *regs)\n+{\n+\tu32 vcpu_id = guest_get_vcpuid();\n+\tu32 expected_lpi = vcpu_id ? LPI_TEST_TO_VPE1 : LPI_TEST_TO_VPE0;\n+\tu64 expected_hwirq = gicv5_lpi_hwirq(expected_lpi);\n+\tu32 hwirq;\n+\tu64 ia;\n+\n+\tia = gicr_insn(CDIA);\n+\tif (!GICV5_GICR_CDIA_VALID(ia))\n+\t\treturn;\n+\n+\tgsb_ack();\n+\tisb();\n+\n+\thwirq = FIELD_GET(GICV5_GICR_CDIA_INTID, ia);\n+\tGUEST_ASSERT_EQ(hwirq, expected_hwirq);\n+\n+\tgic_insn(hwirq, CDDI);\n+\tgic_insn(0, CDEOI);\n+\n+\tif (vcpu_id) {\n+\t\tgicv5_send_lpi(LPI_TEST_TO_VPE0);\n+\t\tGUEST_SYNC(GUEST_CMD_LPI_REPLIED);\n+\t\twhile (1)\n+\t\t\twfi();\n+\t}\n+\n+\tGUEST_DONE();\n+}\n+\n+static void guest_unexpected_irq_handler(struct ex_regs *regs)\n+{\n+\tGUEST_FAIL(\"Unexpected IRQ\");\n+}\n+\n+static void guest_lpi_code(void)\n+{\n+\tu32 vcpu_id = guest_get_vcpuid();\n+\n+\tlocal_irq_disable();\n+\n+\tif (!vcpu_id) {\n+\t\tgicv5_enable_irs();\n+\t\tgicv5_configure_lpi_ist();\n+\t\tgicv5_configure_test_lpis();\n+\t\tWRITE_ONCE(lpi_ist_ready, true);\n+\t}\n+\n+\t/* Go bang if we run VCPU1 before VCPU0 */\n+\tGUEST_ASSERT(READ_ONCE(lpi_ist_ready));\n+\n+\tgicv5_cpu_enable_interrupts();\n+\tlocal_irq_enable();\n+\n+\tGUEST_SYNC(GUEST_CMD_IS_READY);\n+\n+\tif (!vcpu_id) {\n+\t\tgicv5_send_lpi(LPI_TEST_TO_VPE1);\n+\t\tGUEST_SYNC(GUEST_CMD_LPI_SENT);\n+\t}\n+\n+\twhile (1)\n+\t\twfi();\n+}\n+\n+static void restore_test_setup_hwirq(u64 hwirq, u32 target_vpe,\n+\t\t\t\t u32 priority, bool enable)\n+{\n+\tgicv5_set_hwirq_priority(hwirq, priority);\n+\tgicv5_set_hwirq_affinity(hwirq, target_vpe);\n+\n+\tif (enable)\n+\t\tgicv5_enable_hwirq(hwirq);\n+}\n+\n+/*\n+ * The source VM configures SPI and LPI state for the IST save/restore test.\n+ * It leaves interrupts disabled, allowing userspace to save the state before\n+ * the restore VM enables interrupts and checks delivery.\n+ */\n+static void guest_ist_source_code(void)\n+{\n+\tlocal_irq_disable();\n+\n+\tgicv5_enable_irs();\n+\tgicv5_configure_lpi_ist();\n+\n+\trestore_test_setup_hwirq(gicv5_lpi_hwirq(RESTORE_TEST_LPI_PENDING),\n+\t\t\t\t 1, VGIC_V5_IST_PRIO_HIGH, true);\n+\trestore_test_setup_hwirq(gicv5_lpi_hwirq(RESTORE_TEST_LPI_ENABLED),\n+\t\t\t\t 1, VGIC_V5_IST_PRIO_HIGH, true);\n+\trestore_test_setup_hwirq(gicv5_spi_hwirq(RESTORE_TEST_SPI_PRIO_HIGH),\n+\t\t\t\t 0, VGIC_V5_IST_PRIO_HIGH, true);\n+\trestore_test_setup_hwirq(gicv5_spi_hwirq(RESTORE_TEST_SPI_PRIO_LOW),\n+\t\t\t\t 0, VGIC_V5_IST_PRIO_LOW, true);\n+\trestore_test_setup_hwirq(gicv5_spi_hwirq(RESTORE_TEST_SPI_ENABLED),\n+\t\t\t\t 0, VGIC_V5_IST_PRIO_HIGH, true);\n+\trestore_test_setup_hwirq(gicv5_spi_hwirq(RESTORE_TEST_SPI_PENDING_DISABLED),\n+\t\t\t\t 0, VGIC_V5_IST_PRIO_HIGH, false);\n+\n+\tgicv5_send_lpi(RESTORE_TEST_LPI_PENDING);\n+\n+\tGUEST_SYNC(GUEST_CMD_IST_SOURCE_READY);\n+\n+\twhile (1)\n+\t\twfi();\n+}\n+\n+/*\n+ * This is the restore VM for the IST save/restore test. The IRS regs and ISTs\n+ * have been restored before this code runs. Based on which VPE is running, a\n+ * different set of interrupts is expected. We check that we get what we\n+ * expected before terminating the test.\n+ *\n+ * The expected interrupts for VPE0 are:\n+ * - High-priority SPI, pending and enabled before save.\n+ * - Low-priority SPI, pending and enabled before save. This is only delivered\n+ * after the guest lowers the masked priority via ICC_PCR_EL1.\n+ * - SPI that was pending but disabled before save. This is only delivered\n+ * after the guest enables it after restore.\n+ * - SPI that was enabled but not pending before save. Userspace injects this\n+ * after restore to check that the enable state and affinity were restored.\n+ *\n+ * The expected interrupts for VPE1 are:\n+ * - LPI that was pending and enabled before save.\n+ * - LPI that was enabled but not pending before save. The guest injects this\n+ * after restore to check that the enable state and affinity were restored.\n+ */\n+static void guest_ist_restore_irq_handler(struct ex_regs *regs)\n+{\n+\tu32 vcpu_id = guest_get_vcpuid();\n+\tu32 count = ist_restore_irq_count[vcpu_id];\n+\tu64 hwirq, expected_hwirq = 0;\n+\tu64 ia;\n+\n+\tia = gicr_insn(CDIA);\n+\tif (!GICV5_GICR_CDIA_VALID(ia))\n+\t\treturn;\n+\n+\tgsb_ack();\n+\tisb();\n+\n+\thwirq = FIELD_GET(GICV5_GICR_CDIA_INTID, ia);\n+\n+\tif (!vcpu_id) {\n+\t\tswitch (count) {\n+\t\tcase 0:\n+\t\t\texpected_hwirq = gicv5_spi_hwirq(RESTORE_TEST_SPI_PRIO_HIGH);\n+\t\t\tbreak;\n+\t\tcase 1:\n+\t\t\texpected_hwirq = gicv5_spi_hwirq(RESTORE_TEST_SPI_PRIO_LOW);\n+\t\t\tbreak;\n+\t\tcase 2:\n+\t\t\texpected_hwirq = gicv5_spi_hwirq(RESTORE_TEST_SPI_PENDING_DISABLED);\n+\t\t\tbreak;\n+\t\tcase 3:\n+\t\t\texpected_hwirq = gicv5_spi_hwirq(RESTORE_TEST_SPI_ENABLED);\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tGUEST_FAIL(\"Unexpected VPE0 IST restore interrupt\");\n+\t\t}\n+\t} else {\n+\t\tswitch (count) {\n+\t\tcase 0:\n+\t\t\texpected_hwirq = gicv5_lpi_hwirq(RESTORE_TEST_LPI_PENDING);\n+\t\t\tbreak;\n+\t\tcase 1:\n+\t\t\texpected_hwirq = gicv5_lpi_hwirq(RESTORE_TEST_LPI_ENABLED);\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tGUEST_FAIL(\"Unexpected VPE1 IST restore interrupt\");\n+\t\t}\n+\t}\n+\n+\tGUEST_ASSERT_EQ(hwirq, expected_hwirq);\n+\n+\tgic_insn(hwirq, CDDI);\n+\tgic_insn(0, CDEOI);\n+\tist_restore_irq_count[vcpu_id]++;\n+\n+\tif (!vcpu_id) {\n+\t\tswitch (count) {\n+\t\tcase 0:\n+\t\t\tgicv5_set_pcr(VGIC_V5_IST_PRIO_LOW);\n+\t\t\tbreak;\n+\t\tcase 1:\n+\t\t\tgicv5_enable_hwirq(\n+\t\t\t\tgicv5_spi_hwirq(RESTORE_TEST_SPI_PENDING_DISABLED));\n+\t\t\tbreak;\n+\t\tcase 2:\n+\t\t\tGUEST_SYNC(GUEST_CMD_IST_SPI_INJECT_READY);\n+\t\t\tbreak;\n+\t\tcase 3:\n+\t\t\tGUEST_DONE();\n+\t\t}\n+\t} else {\n+\t\tswitch (count) {\n+\t\tcase 0:\n+\t\t\tgicv5_send_lpi(RESTORE_TEST_LPI_ENABLED);\n+\t\t\tGUEST_SYNC(GUEST_CMD_IST_LPI_INJECT_READY);\n+\t\t\tbreak;\n+\t\tcase 1:\n+\t\t\tGUEST_DONE();\n+\t\t}\n+\t}\n+}\n+\n+static void guest_ist_restore_code(void)\n+{\n+\tlocal_irq_disable();\n+\n+\tgicv5_cpu_enable_interrupts();\n+\n+\tif (!guest_get_vcpuid())\n+\t\tgicv5_set_pcr(VGIC_V5_IST_PRIO_HIGH);\n+\n+\tGUEST_SYNC(GUEST_CMD_IS_READY);\n+\n+\tlocal_irq_enable();\n+\n+\twhile (1)\n+\t\twfi();\n+}\n+\n+/* we don't want to assert on run execution, hence that helper */\n+static int run_vcpu(struct kvm_vcpu *vcpu)\n+{\n+\treturn __vcpu_run(vcpu) ? -errno : 0;\n+}\n+\n+static void vm_gic_destroy(struct vm_gic *v)\n+{\n+\tclose(v-\u003egic_fd);\n+\tkvm_vm_free(v-\u003evm);\n+}\n+\n+static void vgic_v5_map_irs(struct kvm_vm *vm)\n+{\n+\tunsigned int nr_irs_pages;\n+\n+\tnr_irs_pages = vm_calc_num_guest_pages(vm-\u003emode, VGIC_V5_IRS_SIZE);\n+\n+\t/* Map the IRS at VA == IPA so guest MMIO writes hit the IRS IODEV. */\n+\tvirt_map(vm, GICV5_IRS_CONFIG_BASE_GPA,\n+\t\t GICV5_IRS_CONFIG_BASE_GPA, nr_irs_pages);\n+}\n+\n+static u32 vgic_v5_irq_line_payload(u32 type, u32 num)\n+{\n+\treturn (type \u003c\u003c KVM_ARM_IRQ_TYPE_SHIFT) |\n+\t FIELD_PREP(KVM_ARM_IRQ_NUM_MASK, num);\n+}\n+\n+static int __vgic_v5_irq_line(struct kvm_vm *vm, u32 type, u32 num, int level)\n+{\n+\treturn _kvm_irq_line(vm, vgic_v5_irq_line_payload(type, num), level);\n+}\n+\n+static void vgic_v5_spi_line(struct kvm_vm *vm, u32 spi, int level)\n+{\n+\tint ret = __vgic_v5_irq_line(vm, KVM_ARM_IRQ_TYPE_SPI, spi, level);\n+\n+\tTEST_ASSERT(!ret, \"KVM_IRQ_LINE failed for SPI %u level %d\",\n+\t\t spi, level);\n+}\n+\n+static void vgic_v5_spi_line_vm_create(struct vm_gic *v,\n+\t\t\t\t struct kvm_vcpu **vcpu,\n+\t\t\t\t u32 nr_spis)\n+{\n+\tu64 attr;\n+\n+\tv-\u003egic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;\n+\tv-\u003evm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);\n+\tv-\u003egic_fd = kvm_create_device(v-\u003evm, v-\u003egic_dev_type);\n+\t*vcpu = vm_vcpu_add(v-\u003evm, 0, guest_spi_line_code);\n+\tTEST_ASSERT(*vcpu, \"Failed to create vCPU\");\n+\n+\tvm_init_descriptor_tables(v-\u003evm);\n+\tvm_install_exception_handler(v-\u003evm, VECTOR_IRQ_CURRENT,\n+\t\t\t\t guest_spi_irq_handler);\n+\tvcpu_init_descriptor_tables(*vcpu);\n+\n+\tkvm_device_attr_set(v-\u003egic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS, 0,\n+\t\t\t \u0026nr_spis);\n+\n+\tattr = GICV5_IRS_CONFIG_BASE_GPA;\n+\tkvm_device_attr_set(v-\u003egic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,\n+\t\t\t KVM_VGIC_V5_ADDR_TYPE_IRS, \u0026attr);\n+\tvgic_v5_map_irs(v-\u003evm);\n+\tkvm_device_attr_set(v-\u003egic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);\n+}\n+\n+static void vgic_v5_lpi_ist_vm_create(struct vm_gic *v,\n+\t\t\t\t struct kvm_vcpu *vcpus[VGIC_V5_LPI_NR_VCPUS],\n+\t\t\t\t void (*guest_code)(void),\n+\t\t\t\t void (*irq_handler)(struct ex_regs *))\n+{\n+\tunsigned int nr_lpi_ist_pages;\n+\tu64 attr;\n+\tint i;\n+\n+\tv-\u003egic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;\n+\tv-\u003evm = __vm_create(VM_SHAPE_DEFAULT, VGIC_V5_LPI_NR_VCPUS, 0);\n+\tv-\u003egic_fd = kvm_create_device(v-\u003evm, v-\u003egic_dev_type);\n+\n+\tfor (i = 0; i \u003c VGIC_V5_LPI_NR_VCPUS; i++) {\n+\t\tvcpus[i] = vm_vcpu_add(v-\u003evm, i, guest_code);\n+\t\tTEST_ASSERT(vcpus[i], \"Failed to create vCPU\");\n+\t}\n+\n+\tvm_init_descriptor_tables(v-\u003evm);\n+\tvm_install_exception_handler(v-\u003evm, VECTOR_IRQ_CURRENT,\n+\t\t\t\t irq_handler);\n+\n+\tfor (i = 0; i \u003c VGIC_V5_LPI_NR_VCPUS; i++)\n+\t\tvcpu_init_descriptor_tables(vcpus[i]);\n+\n+\tattr = GICV5_IRS_CONFIG_BASE_GPA;\n+\tkvm_device_attr_set(v-\u003egic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,\n+\t\t\t KVM_VGIC_V5_ADDR_TYPE_IRS, \u0026attr);\n+\tvgic_v5_map_irs(v-\u003evm);\n+\n+\tnr_lpi_ist_pages = vm_calc_num_guest_pages(v-\u003evm-\u003emode,\n+\t\t\t\t\t\t VGIC_V5_LPI_IST_SIZE);\n+\tvm_userspace_mem_region_add(v-\u003evm, VM_MEM_SRC_ANONYMOUS,\n+\t\t\t\t VGIC_V5_LPI_IST_BASE_GPA,\n+\t\t\t\t VGIC_V5_LPI_MEMSLOT,\n+\t\t\t\t nr_lpi_ist_pages, 0);\n+\t/*\n+\t * Map the IST at VA == IPA so the guest can program the same BASER\n+\t * address.\n+\t */\n+\tvirt_map(v-\u003evm, VGIC_V5_LPI_IST_BASE_GPA,\n+\t\t VGIC_V5_LPI_IST_BASE_GPA, nr_lpi_ist_pages);\n+\n+\tkvm_device_attr_set(v-\u003egic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);\n+}\n+\n+static void vgic_v5_lpi_vm_create(struct vm_gic *v,\n+\t\t\t\t struct kvm_vcpu *vcpus[VGIC_V5_LPI_NR_VCPUS])\n+{\n+\tvgic_v5_lpi_ist_vm_create(v, vcpus, guest_lpi_code,\n+\t\t\t\t guest_lpi_irq_handler);\n+}\n+\n+static void vgic_v5_ist_restore_vm_create(struct vm_gic *v,\n+\t\t\t\t\t struct kvm_vcpu *vcpus[VGIC_V5_LPI_NR_VCPUS],\n+\t\t\t\t\t void (*guest_code)(void),\n+\t\t\t\t\t void (*irq_handler)(struct ex_regs *))\n+{\n+\tvgic_v5_lpi_ist_vm_create(v, vcpus, guest_code, irq_handler);\n+}\n+\n+static void vgic_v5_run_spi_line_test(u32 nr_spis, u32 expected_spi,\n+\t\t\t\t bool level_sensitive,\n+\t\t\t\t bool lower_before_run)\n+{\n+\tstruct kvm_vcpu *vcpu;\n+\tstruct vm_gic v;\n+\tstruct ucall uc;\n+\tint ret;\n+\n+\tspi_line_expected = expected_spi;\n+\tspi_line_level_sensitive = level_sensitive;\n+\n+\tvgic_v5_spi_line_vm_create(\u0026v, \u0026vcpu, nr_spis);\n+\n+\tsync_global_to_guest(v.vm, spi_line_expected);\n+\tsync_global_to_guest(v.vm, spi_line_level_sensitive);\n+\n+\tret = run_vcpu(vcpu);\n+\tTEST_ASSERT(!ret, \"Failed to run GICv5 IRQ_LINE VM\");\n+\tTEST_ASSERT(get_ucall(vcpu, \u0026uc) == UCALL_SYNC \u0026\u0026\n+\t\t uc.args[1] == GUEST_CMD_IS_READY,\n+\t\t \"GICv5 IRQ_LINE guest did not become ready\");\n+\n+\tvgic_v5_spi_line(v.vm, expected_spi, 1);\n+\t/*\n+\t * For edge-triggered SPIs, a following low transition must be ignored:\n+\t * once the edge has made the SPI pending, it cannot be recalled. This\n+\t * allows us to test that.\n+\t */\n+\tif (lower_before_run)\n+\t\tvgic_v5_spi_line(v.vm, expected_spi, 0);\n+\n+\twhile (1) {\n+\t\tret = run_vcpu(vcpu);\n+\t\tTEST_ASSERT(!ret, \"Failed to run GICv5 IRQ_LINE VM\");\n+\n+\t\tswitch (get_ucall(vcpu, \u0026uc)) {\n+\t\tcase UCALL_ABORT:\n+\t\t\tREPORT_GUEST_ASSERT(uc);\n+\t\t\tbreak;\n+\t\tcase UCALL_DONE:\n+\t\t\tgoto done;\n+\t\tdefault:\n+\t\t\tTEST_FAIL(\"Unknown ucall %lu\", uc.cmd);\n+\t\t}\n+\t}\n+\n+done:\n+\tvm_gic_destroy(\u0026v);\n+}\n+\n+static void vgic_v5_expect_sync(struct kvm_vcpu *vcpu, u64 cmd,\n+\t\t\t\tconst char *msg)\n+{\n+\tstruct ucall uc;\n+\n+\tTEST_ASSERT(get_ucall(vcpu, \u0026uc) == UCALL_SYNC \u0026\u0026 uc.args[1] == cmd,\n+\t\t \"%s\", msg);\n+}\n+\n+struct vgic_region_attr {\n+\tu64 attr;\n+\tu64 size;\n+\tu64 alignment;\n+};\n+\n+static const struct vgic_region_attr gic_v5_irs_region = {\n+\t.attr = KVM_VGIC_V5_ADDR_TYPE_IRS,\n+\t.size = GICV5_IRS_SIZE,\n+\t.alignment = GICV5_IRS_ALIGN,\n+};\n+\n+struct vgic_sysreg_attr {\n+\tconst char\t*name;\n+\tu32\t\tencoding;\n+};\n+\n+struct vgic_irs_reg_attr {\n+\tconst char\t*name;\n+\tu64\t\tattr;\n+};\n+\n+#define PACK_SR(r)\t\t\t\t\t\t\\\n+\t((sys_reg_Op0(r) \u003c\u003c 14) |\t\t\t\t\\\n+\t (sys_reg_Op1(r) \u003c\u003c 11) |\t\t\t\t\\\n+\t (sys_reg_CRn(r) \u003c\u003c 7) |\t\t\t\t\\\n+\t (sys_reg_CRm(r) \u003c\u003c 3) |\t\t\t\t\\\n+\t (sys_reg_Op2(r)))\n+\n+#define SR(r)\t\t\t\t\t\t\t\\\n+\t{\t\t\t\t\t\t\t\\\n+\t\t.name\t\t= #r,\t\t\t\t\\\n+\t\t.encoding\t= r,\t\t\t\t\\\n+\t}\n+\n+#define IRS_REG(r)\t\t\t\t\t\t\\\n+\t{\t\t\t\t\t\t\t\\\n+\t\t.name\t\t= #r,\t\t\t\t\\\n+\t\t.attr\t\t= r,\t\t\t\t\\\n+\t}\n+\n+static const struct vgic_irs_reg_attr gic_v5_irs_regs[] = {\n+\tIRS_REG(GICV5_IRS_IDR0),\n+\tIRS_REG(GICV5_IRS_IDR1),\n+\tIRS_REG(GICV5_IRS_IDR2),\n+\tIRS_REG(GICV5_IRS_IDR3),\n+\tIRS_REG(GICV5_IRS_IDR4),\n+\tIRS_REG(GICV5_IRS_IDR5),\n+\tIRS_REG(GICV5_IRS_IDR6),\n+\tIRS_REG(GICV5_IRS_IDR7),\n+\tIRS_REG(GICV5_IRS_IIDR),\n+\tIRS_REG(GICV5_IRS_AIDR),\n+\tIRS_REG(GICV5_IRS_CR0),\n+\tIRS_REG(GICV5_IRS_CR1),\n+\tIRS_REG(GICV5_IRS_SYNCR),\n+\tIRS_REG(GICV5_IRS_SYNC_STATUSR),\n+\tIRS_REG(GICV5_IRS_SPI_VMR),\n+\tIRS_REG(GICV5_IRS_SPI_SELR),\n+\tIRS_REG(GICV5_IRS_SPI_DOMAINR),\n+\tIRS_REG(GICV5_IRS_SPI_RESAMPLER),\n+\tIRS_REG(GICV5_IRS_SPI_CFGR),\n+\tIRS_REG(GICV5_IRS_SPI_STATUSR),\n+\tIRS_REG(GICV5_IRS_PE_SELR),\n+\tIRS_REG(GICV5_IRS_PE_STATUSR),\n+\tIRS_REG(GICV5_IRS_PE_CR0),\n+\tIRS_REG(GICV5_IRS_IST_BASER),\n+\tIRS_REG(GICV5_IRS_IST_CFGR),\n+\tIRS_REG(GICV5_IRS_IST_STATUSR),\n+\tIRS_REG(GICV5_IRS_MAP_L2_ISTR),\n+\tIRS_REG(GICV5_IRS_VMT_BASER),\n+\tIRS_REG(GICV5_IRS_VMT_CFGR),\n+\tIRS_REG(GICV5_IRS_VMT_STATUSR),\n+\tIRS_REG(GICV5_IRS_VPE_SELR),\n+\tIRS_REG(GICV5_IRS_VPE_DBR),\n+\tIRS_REG(GICV5_IRS_VPE_HPPIR),\n+\tIRS_REG(GICV5_IRS_VPE_CR0),\n+\tIRS_REG(GICV5_IRS_VPE_STATUSR),\n+\tIRS_REG(GICV5_IRS_VM_DBR),\n+\tIRS_REG(GICV5_IRS_VM_SELR),\n+\tIRS_REG(GICV5_IRS_VM_STATUSR),\n+\tIRS_REG(GICV5_IRS_VMAP_L2_VMTR),\n+\tIRS_REG(GICV5_IRS_VMAP_VMR),\n+\tIRS_REG(GICV5_IRS_VMAP_VISTR),\n+\tIRS_REG(GICV5_IRS_VMAP_L2_VISTR),\n+\tIRS_REG(GICV5_IRS_VMAP_VPER),\n+\tIRS_REG(GICV5_IRS_SAVE_VMR),\n+\tIRS_REG(GICV5_IRS_SAVE_VM_STATUSR),\n+\tIRS_REG(GICV5_IRS_MEC_IDR),\n+\tIRS_REG(GICV5_IRS_MEC_MECID_R),\n+\tIRS_REG(GICV5_IRS_MPAM_IDR),\n+\tIRS_REG(GICV5_IRS_MPAM_PARTID_R),\n+\tIRS_REG(GICV5_IRS_SWERR_STATUSR),\n+\tIRS_REG(GICV5_IRS_SWERR_SYNDROMER0),\n+\tIRS_REG(GICV5_IRS_SWERR_SYNDROMER1),\n+};\n+\n+static const struct vgic_sysreg_attr gic_v5_cpu_sysregs[] = {\n+\tSR(SYS_ICC_ICSR_EL1),\n+\tSR(SYS_ICC_PPI_ENABLER0_EL1),\n+\tSR(SYS_ICC_PPI_ENABLER1_EL1),\n+\tSR(SYS_ICC_PPI_SACTIVER0_EL1),\n+\tSR(SYS_ICC_PPI_SACTIVER1_EL1),\n+\tSR(SYS_ICC_PPI_SPENDR0_EL1),\n+\tSR(SYS_ICC_PPI_SPENDR1_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR0_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR1_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR2_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR3_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR4_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR5_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR6_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR7_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR8_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR9_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR10_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR11_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR12_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR13_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR14_EL1),\n+\tSR(SYS_ICC_PPI_PRIORITYR15_EL1),\n+\tSR(SYS_ICC_APR_EL1),\n+\tSR(SYS_ICC_CR0_EL1),\n+\tSR(SYS_ICC_PCR_EL1),\n+};\n+\n+static void vgic_v5_save_irs_regs(struct vm_gic *v, u64 *regs)\n+{\n+\tint ret, i;\n+\n+\tfor (i = 0; i \u003c ARRAY_SIZE(gic_v5_irs_regs); i++) {\n+\t\tret = __kvm_device_attr_get(v-\u003egic_fd,\n+\t\t\t\t\t KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t\t gic_v5_irs_regs[i].attr,\n+\t\t\t\t\t \u0026regs[i]);\n+\t\tTEST_ASSERT(!ret, \"GICv5 IRS_REGS save failed for %s\",\n+\t\t\t gic_v5_irs_regs[i].name);\n+\t}\n+}\n+\n+static void vgic_v5_restore_irs_regs(struct vm_gic *v, u64 *regs)\n+{\n+\tint ret, i;\n+\n+\tfor (i = 0; i \u003c ARRAY_SIZE(gic_v5_irs_regs); i++) {\n+\t\tret = __kvm_device_attr_set(v-\u003egic_fd,\n+\t\t\t\t\t KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t\t gic_v5_irs_regs[i].attr,\n+\t\t\t\t\t \u0026regs[i]);\n+\t\tTEST_ASSERT(!ret, \"GICv5 IRS_REGS restore failed for %s\",\n+\t\t\t gic_v5_irs_regs[i].name);\n+\t}\n+}\n+\n+static u64 vgic_v5_saved_irs_reg(u64 *regs, u64 attr)\n+{\n+\tfor (int i = 0; i \u003c ARRAY_SIZE(gic_v5_irs_regs); i++) {\n+\t\tif (gic_v5_irs_regs[i].attr == attr)\n+\t\t\treturn regs[i];\n+\t}\n+\n+\tTEST_FAIL(\"GICv5 IRS register 0x%lx was not saved\", attr);\n+}\n+\n+static size_t vgic_v5_lpi_ist_state_size(u64 *irs_regs)\n+{\n+\tu64 cfgr = vgic_v5_saved_irs_reg(irs_regs, GICV5_IRS_IST_CFGR);\n+\tu32 lpi_id_bits = FIELD_GET(GICV5_IRS_IST_CFGR_LPI_ID_BITS, cfgr);\n+\n+\treturn BIT(lpi_id_bits) * sizeof(__u32);\n+}\n+\n+static void test_vgic_v5_addr_attrs(void)\n+{\n+\tstruct kvm_vcpu *vcpu;\n+\tstruct vm_gic v;\n+\tu64 addr;\n+\tint ret;\n+\n+\tv.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;\n+\tv.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);\n+\tv.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);\n+\n+\t/* Check existing group/attributes */\n+\tkvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR, gic_v5_irs_region.attr);\n+\n+\t/* check non existing attribute */\n+\tret = __kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR, -1);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == ENXIO, \"attribute not supported\");\n+\n+\t/* get IRS base address before setting*/\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,\n+\t\t\t\t KVM_VGIC_V5_ADDR_TYPE_IRS, \u0026addr);\n+\tTEST_ASSERT(!ret \u0026\u0026 addr == (-1ULL), \"GICv5 IRS returns VGIC_ADDR_UNDEF\");\n+\n+\t/* misaligned IRS address settings */\n+\taddr = gic_v5_irs_region.alignment / 0x10;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,\n+\t\t\t\t KVM_VGIC_V5_ADDR_TYPE_IRS, \u0026addr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GIC IRS base not aligned\");\n+\n+\t/* out of range address */\n+\taddr = max_phys_size;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,\n+\t\t\t\t KVM_VGIC_V5_ADDR_TYPE_IRS, \u0026addr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == E2BIG, \"IRS address beyond IPA limit\");\n+\n+\t/* Space for half an IRS (an IRS is: 2 * irs.alignment). */\n+\taddr = max_phys_size - gic_v5_irs_region.alignment;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,\n+\t\t\t\t KVM_VGIC_V5_ADDR_TYPE_IRS, \u0026addr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == E2BIG,\n+\t\t\t\"half of the IRS is beyond IPA limit\");\n+\n+\t/* set IRS base address @0x0*/\n+\taddr = 0x00000;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,\n+\t\t\t\t KVM_VGIC_V5_ADDR_TYPE_IRS, \u0026addr);\n+\tTEST_ASSERT(!ret, \"GICv5 IRS base correctly set\");\n+\n+\t/* get IRS base address */\n+\taddr = 0xbad;\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,\n+\t\t\t\t KVM_VGIC_V5_ADDR_TYPE_IRS, \u0026addr);\n+\tTEST_ASSERT(!ret \u0026\u0026 addr == 0, \"GICv5 IRS base correctly set\");\n+\n+\t/* Attempt to create a second IRS region */\n+\taddr = 0xE0000;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,\n+\t\t\t\t KVM_VGIC_V5_ADDR_TYPE_IRS, \u0026addr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EEXIST, \"GICv5 IRS base set again\");\n+\n+\tvm_gic_destroy(\u0026v);\n+\n+\t/* Try running a VM without ever setting the IRS base addr */\n+\tv.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);\n+\tv.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);\n+\tvcpu = vm_vcpu_add(v.vm, 0, NULL);\n+\tkvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);\n+\tret = run_vcpu(vcpu);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == ENXIO, \"GICv5 IRS base not set\");\n+\n+\tvm_gic_destroy(\u0026v);\n+}\n \n-static void guest_irq_handler(struct ex_regs *regs)\n+static void test_vgic_v5_nr_irqs_attrs(void)\n {\n-\tbool valid;\n-\tu32 hwirq;\n-\tu64 ia;\n-\tstatic int count;\n+\tstruct kvm_vcpu *vcpu;\n+\tstruct vm_gic v;\n+\tuint64_t attr;\n+\tu32 nr_irqs;\n+\tint ret;\n \n-\t/*\n-\t * We have pending interrupts. Should never actually enter WFI\n-\t * here!\n-\t */\n-\twfi();\n-\tGUEST_SYNC(GUEST_CMD_IS_AWAKE);\n+\tv.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;\n+\tv.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);\n+\tv.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);\n \n-\tia = gicr_insn(CDIA);\n-\tvalid = GICV5_GICR_CDIA_VALID(ia);\n+\t/* Check existing group/attribute */\n+\tkvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS, 0);\n \n-\tGUEST_SYNC(GUEST_CMD_IRQ_CDIA);\n+\t/* Before userspace sets NR_IRQS, no SPI count has been selected. */\n+\tnr_irqs = 0xbad;\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,\n+\t\t\t\t 0, \u0026nr_irqs);\n+\tTEST_ASSERT(!ret \u0026\u0026 nr_irqs == 0, \"GICv5 NR_IRQS defaults to 0 before init\");\n \n-\tif (!valid)\n-\t\treturn;\n+\t/* Too few SPIs */\n+\tnr_irqs = VGIC_V5_DEFAULT_NR_SPIS - 1;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,\n+\t\t\t\t 0, \u0026nr_irqs);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 NR_IRQS below minimum\");\n \n-\tgsb_ack();\n-\tisb();\n+\t/* Not a multiple of 32 */\n+\tnr_irqs = VGIC_V5_DEFAULT_NR_SPIS + 1;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,\n+\t\t\t\t 0, \u0026nr_irqs);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 NR_IRQS not 32-aligned\");\n \n-\thwirq = FIELD_GET(GICV5_GICR_CDIA_INTID, ia);\n+\t/* Larger than KVM's supported VGICv5 SPI count */\n+\tnr_irqs = VGIC_V5_MAX_NR_SPIS + 1;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,\n+\t\t\t\t 0, \u0026nr_irqs);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 NR_IRQS above maximum\");\n \n-\tgic_insn(hwirq, CDDI);\n-\tgic_insn(0, CDEOI);\n+\t/* Valid custom SPI count */\n+\tnr_irqs = VGIC_V5_DEFAULT_NR_SPIS * 2;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,\n+\t\t\t\t 0, \u0026nr_irqs);\n+\tTEST_ASSERT(!ret, \"GICv5 NR_IRQS accepts valid custom SPI count\");\n \n-\tGUEST_SYNC(GUEST_CMD_IRQ_DIEOI);\n+\tnr_irqs = 0xbad;\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,\n+\t\t\t\t 0, \u0026nr_irqs);\n+\tTEST_ASSERT(!ret \u0026\u0026 nr_irqs == VGIC_V5_DEFAULT_NR_SPIS * 2,\n+\t\t \"GICv5 NR_IRQS returns SPI count only\");\n \n-\tif (++count \u003e= 2)\n-\t\tGUEST_DONE();\n+\t/* A second successful configuration attempt must be rejected. */\n+\tnr_irqs = VGIC_V5_DEFAULT_NR_SPIS;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,\n+\t\t\t\t 0, \u0026nr_irqs);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EBUSY, \"GICv5 NR_IRQS set twice\");\n \n-\t/* Ask for the next interrupt to be injected */\n-\tGUEST_SYNC(GUEST_CMD_IS_READY);\n+\tvm_gic_destroy(\u0026v);\n+\n+\t/* If userspace does not set NR_IRQS, init selects the default. */\n+\tv.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);\n+\tv.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);\n+\tvcpu = vm_vcpu_add(v.vm, 0, NULL);\n+\tTEST_ASSERT(vcpu, \"Failed to create vCPU\");\n+\tkvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);\n+\n+\tnr_irqs = 0xbad;\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,\n+\t\t\t\t 0, \u0026nr_irqs);\n+\tTEST_ASSERT(!ret \u0026\u0026 nr_irqs == VGIC_V5_DEFAULT_NR_SPIS,\n+\t\t \"GICv5 NR_IRQS defaults to 32 SPIs after init\");\n+\n+\tnr_irqs = VGIC_V5_DEFAULT_NR_SPIS * 2;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,\n+\t\t\t\t 0, \u0026nr_irqs);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EBUSY, \"GICv5 NR_IRQS set after init\");\n+\n+\tvm_gic_destroy(\u0026v);\n+\n+\t/* NR_IRQS must remain immutable after the VM has run. */\n+\tv.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);\n+\tv.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);\n+\tvcpu = vm_vcpu_add(v.vm, 0, guest_code);\n+\n+\tattr = GICV5_IRS_CONFIG_BASE_GPA;\n+\tkvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,\n+\t\t\t KVM_VGIC_V5_ADDR_TYPE_IRS, \u0026attr);\n+\tkvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);\n+\n+\tret = run_vcpu(vcpu);\n+\tTEST_ASSERT(!ret, \"Failed to run GICv5 VM before NR_IRQS test\");\n+\n+\tnr_irqs = VGIC_V5_DEFAULT_NR_SPIS * 2;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,\n+\t\t\t\t 0, \u0026nr_irqs);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EBUSY, \"GICv5 NR_IRQS set after run\");\n+\n+\tvm_gic_destroy(\u0026v);\n }\n \n-static void guest_code(void)\n+static void test_vgic_v5_irs_regs_attrs(void)\n {\n-\tlocal_irq_disable();\n+\tstruct kvm_vcpu *vcpu;\n+\tstruct vm_gic v;\n+\tu64 attr, val;\n+\tint ret, i;\n \n-\tgicv5_cpu_enable_interrupts();\n-\tlocal_irq_enable();\n+\tv.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;\n+\tv.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);\n+\tv.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);\n+\tvcpu = vm_vcpu_add(v.vm, 0, NULL);\n+\tTEST_ASSERT(vcpu, \"Failed to create vCPU\");\n \n-\t/* Enable the SW_PPI (3) */\n-\twrite_sysreg_s(BIT_ULL(3), SYS_ICC_PPI_ENABLER0_EL1);\n+\t/* IRS_REGS attributes can be probed before the IRS base is set. */\n+\tkvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t GICV5_IRS_IDR0);\n \n-\t/* Ask for the first interrupt to be injected */\n-\tGUEST_SYNC(GUEST_CMD_IS_READY);\n+\tattr = GICV5_IRS_CONFIG_BASE_GPA;\n+\tkvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,\n+\t\t\t KVM_VGIC_V5_ADDR_TYPE_IRS, \u0026attr);\n \n-\t/* Loop forever waiting for interrupts */\n-\twhile (1);\n+\t/* Check existing group/attribute */\n+\tkvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t GICV5_IRS_IDR0);\n+\n+\t/* IRS_REGS are not accessible before the VGIC is initialized. */\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR0, \u0026val);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EBUSY, \"GICv5 IRS_REGS get before init\");\n+\n+\tval = 0;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR0, \u0026val);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EBUSY, \"GICv5 IRS_REGS set before init\");\n+\n+\tkvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);\n+\n+\t/* Read all supported IRS regs and write the value back. */\n+\tfor (i = 0; i \u003c ARRAY_SIZE(gic_v5_irs_regs); i++) {\n+\t\tattr = gic_v5_irs_regs[i].attr;\n+\t\tret = __kvm_has_device_attr(v.gic_fd,\n+\t\t\t\t\t KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t\t attr);\n+\t\tTEST_ASSERT(!ret, \"GICv5 IRS_REGS missing %s\",\n+\t\t\t gic_v5_irs_regs[i].name);\n+\n+\t\tval = 0xbad;\n+\t\tret = __kvm_device_attr_get(v.gic_fd,\n+\t\t\t\t\t KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t\t attr, \u0026val);\n+\t\tTEST_ASSERT(!ret, \"GICv5 IRS_REGS get failed for %s\",\n+\t\t\t gic_v5_irs_regs[i].name);\n+\n+\t\tret = __kvm_device_attr_set(v.gic_fd,\n+\t\t\t\t\t KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t\t attr, \u0026val);\n+\t\tTEST_ASSERT(!ret, \"GICv5 IRS_REGS set failed for %s\",\n+\t\t\t gic_v5_irs_regs[i].name);\n+\t}\n+\n+\t/* Check bad offsets */\n+\tattr = 0x10000;\n+\tret = __kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == ENXIO, \"GICv5 IRS_REGS accepted bad offset\");\n+\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t attr, \u0026val);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == ENXIO, \"GICv5 IRS_REGS get bad offset\");\n+\n+\t/* Check alignment for 32-bit and 64-bit IRS regs. */\n+\tattr = GICV5_IRS_IDR0 + 2;\n+\tret = __kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IRS_REGS accepted unaligned 32-bit attr\");\n+\n+\tattr = GICV5_IRS_IST_BASER + 4;\n+\tret = __kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IRS_REGS accepted unaligned 64-bit attr\");\n+\n+\t/* Check bad user pointers */\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR0, NULL);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EFAULT, \"GICv5 IRS_REGS get with bad pointer\");\n+\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR0, NULL);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EFAULT, \"GICv5 IRS_REGS set with bad pointer\");\n+\n+\t/* ID restore validation rejects unsupported values. */\n+\tval = FIELD_PREP(GICV5_IRS_IDR0_INT_DOM, GICV5_IRS_IDR0_INT_DOM_SECURE);\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR0, \u0026val);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IRS_REGS accepted bad IDR0 domain\");\n+\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR0, \u0026val);\n+\tTEST_ASSERT(!ret, \"GICv5 IRS_REGS get IDR0 failed\");\n+\tval |= GICV5_IRS_IDR0_SETLPI;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR0, \u0026val);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IRS_REGS accepted unsupported IDR0\");\n+\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR1, \u0026val);\n+\tTEST_ASSERT(!ret, \"GICv5 IRS_REGS get IDR1 failed\");\n+\tval |= FIELD_PREP(GICV5_IRS_IDR1_PRIORITY_BITS, 0x7);\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR1, \u0026val);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IRS_REGS accepted bad IDR1\");\n+\n+\tval = 0;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR2, \u0026val);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IRS_REGS accepted bad IDR2\");\n+\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR5, \u0026val);\n+\tTEST_ASSERT(!ret, \"GICv5 IRS_REGS get IDR5 failed\");\n+\tval++;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR5, \u0026val);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IRS_REGS accepted bad IDR5\");\n+\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR6, \u0026val);\n+\tTEST_ASSERT(!ret, \"GICv5 IRS_REGS get IDR6 failed\");\n+\tval++;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR6, \u0026val);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IRS_REGS accepted bad IDR6\");\n+\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR7, \u0026val);\n+\tTEST_ASSERT(!ret, \"GICv5 IRS_REGS get IDR7 failed\");\n+\tval++;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IDR7, \u0026val);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IRS_REGS accepted bad IDR7\");\n+\n+\t/* Status registers read as idle through userspace accessors. */\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_SYNC_STATUSR, \u0026val);\n+\tTEST_ASSERT(!ret \u0026\u0026 val == GICV5_IRS_SYNC_STATUSR_IDLE,\n+\t\t \"GICv5 IRS_REGS SYNC_STATUSR is not idle\");\n+\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_SPI_STATUSR, \u0026val);\n+\tTEST_ASSERT(!ret \u0026\u0026 val == GICV5_IRS_SPI_STATUSR_IDLE,\n+\t\t \"GICv5 IRS_REGS SPI_STATUSR is not idle\");\n+\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_PE_STATUSR, \u0026val);\n+\tTEST_ASSERT(!ret \u0026\u0026 val == GICV5_IRS_PE_STATUSR_IDLE,\n+\t\t \"GICv5 IRS_REGS PE_STATUSR is not idle\");\n+\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IST_STATUSR, \u0026val);\n+\tTEST_ASSERT(!ret \u0026\u0026 val == GICV5_IRS_IST_STATUSR_IDLE,\n+\t\t \"GICv5 IRS_REGS IST_STATUSR is not idle\");\n+\n+\t/* Userspace restore of IST_BASER updates state without allocating ISTs. */\n+\tval = FIELD_PREP(GICV5_IRS_IST_BASER_ADDR_MASK,\n+\t\t\t GICV5_IRS_CONFIG_BASE_GPA \u003e\u003e GICV5_IRS_IST_BASER_ADDR_SHIFT);\n+\tval |= GICV5_IRS_IST_BASER_VALID;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IST_BASER, \u0026val);\n+\tTEST_ASSERT(!ret, \"GICv5 IRS_REGS failed to restore IST_BASER\");\n+\n+\tattr = 0xbad;\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,\n+\t\t\t\t GICV5_IRS_IST_BASER, \u0026attr);\n+\tTEST_ASSERT(!ret \u0026\u0026 attr == val, \"GICv5 IRS_REGS IST_BASER restore mismatch\");\n+\n+\tvm_gic_destroy(\u0026v);\n }\n \n+static void test_vgic_v5_ist_attrs(void)\n+{\n+\tstruct kvm_vgic_v5_ist ist_attr;\n+\tstruct kvm_vcpu *vcpu;\n+\tstruct vm_gic v;\n+\tu32 spi_ist[VGIC_V5_DEFAULT_NR_SPIS];\n+\tu64 attr;\n+\tint ret;\n \n-/* we don't want to assert on run execution, hence that helper */\n-static int run_vcpu(struct kvm_vcpu *vcpu)\n+\tv.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;\n+\tv.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);\n+\tv.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);\n+\tvcpu = vm_vcpu_add(v.vm, 0, NULL);\n+\tTEST_ASSERT(vcpu, \"Failed to create vCPU\");\n+\n+\t/* Check existing group/attribute */\n+\tkvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST, 0);\n+\n+\t/* Check non-existing attribute */\n+\tret = __kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST, 1);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == ENXIO, \"GICv5 IST accepted bad attr\");\n+\n+\tist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), NULL, 0);\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 1, \u0026ist_attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == ENXIO, \"GICv5 IST get accepted bad attr\");\n+\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 1, \u0026ist_attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == ENXIO, \"GICv5 IST set accepted bad attr\");\n+\n+\tattr = GICV5_IRS_CONFIG_BASE_GPA;\n+\tkvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,\n+\t\t\t KVM_VGIC_V5_ADDR_TYPE_IRS, \u0026attr);\n+\n+\t/* IST save/restore is not accessible before the VGIC is initialized. */\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, \u0026ist_attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EBUSY, \"GICv5 IST get before init\");\n+\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, \u0026ist_attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EBUSY, \"GICv5 IST set before init\");\n+\n+\tkvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);\n+\n+\t/* A VM with SPIs must provide a userspace IST descriptor. */\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, NULL);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IST get accepted NULL descriptor\");\n+\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, NULL);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IST set accepted NULL descriptor\");\n+\n+\t/* Check bad userspace IST descriptors. */\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, (void *)1);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EFAULT, \"GICv5 IST get with bad descriptor\");\n+\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, (void *)1);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EFAULT, \"GICv5 IST set with bad descriptor\");\n+\n+\t/* Check missing and incorrectly sized SPI IST buffers. */\n+\tist_attr = vgic_v5_ist_attr(NULL, 0, NULL, 0);\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, \u0026ist_attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IST get accepted missing SPI buffer\");\n+\n+\tist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist) - sizeof(__u32),\n+\t\t\t\t\t NULL, 0);\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, \u0026ist_attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IST set accepted bad SPI size\");\n+\n+\t/* LPI storage must be absent when no LPI IST is configured. */\n+\tist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), spi_ist,\n+\t\t\t\t\t sizeof(__u32));\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, \u0026ist_attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IST get accepted unexpected LPI buffer\");\n+\n+\tvm_gic_destroy(\u0026v);\n+\n+\t/* IST restore is rejected after the VM has run. */\n+\tv.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);\n+\tv.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);\n+\tvcpu = vm_vcpu_add(v.vm, 0, guest_code);\n+\n+\tattr = GICV5_IRS_CONFIG_BASE_GPA;\n+\tkvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,\n+\t\t\t KVM_VGIC_V5_ADDR_TYPE_IRS, \u0026attr);\n+\tkvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);\n+\n+\tret = run_vcpu(vcpu);\n+\tTEST_ASSERT(!ret, \"Failed to run GICv5 VM before IST restore test\");\n+\n+\tist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), NULL, 0);\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, \u0026ist_attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EBUSY, \"GICv5 IST restore after run\");\n+\n+\tvm_gic_destroy(\u0026v);\n+}\n+\n+static void test_vgic_v5_userspace_ppis_attrs(void)\n {\n-\treturn __vcpu_run(vcpu) ? -errno : 0;\n+\tstruct kvm_vcpu *vcpu;\n+\tstruct vm_gic v;\n+\tu64 user_ppis[2];\n+\tint ret;\n+\n+\tv.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;\n+\tv.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);\n+\tv.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);\n+\tvcpu = vm_vcpu_add(v.vm, 0, NULL);\n+\tTEST_ASSERT(vcpu, \"Failed to create vCPU\");\n+\n+\tkvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t KVM_DEV_ARM_VGIC_CTRL_INIT);\n+\tkvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t KVM_DEV_ARM_VGIC_USERSPACE_PPIS);\n+\n+\tkvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);\n+\n+\tuser_ppis[0] = 0;\n+\tuser_ppis[1] = 0xbad;\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t\t KVM_DEV_ARM_VGIC_USERSPACE_PPIS, user_ppis);\n+\tTEST_ASSERT(!ret, \"GICv5 USERSPACE_PPIS get failed\");\n+\tTEST_ASSERT(user_ppis[0] \u0026 BIT(GICV5_ARCH_PPI_SW_PPI),\n+\t\t \"GICv5 USERSPACE_PPIS does not expose SW_PPI\");\n+\tTEST_ASSERT(!user_ppis[1], \"GICv5 USERSPACE_PPIS upper word is not zero\");\n+\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t\t KVM_DEV_ARM_VGIC_USERSPACE_PPIS, user_ppis);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == ENXIO, \"GICv5 USERSPACE_PPIS set accepted\");\n+\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t\t KVM_DEV_ARM_VGIC_USERSPACE_PPIS, NULL);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EFAULT, \"GICv5 USERSPACE_PPIS get with bad pointer\");\n+\n+\tvm_gic_destroy(\u0026v);\n }\n \n-static void vm_gic_destroy(struct vm_gic *v)\n+static void test_vgic_v5_cpu_sysreg_attrs(void)\n {\n-\tclose(v-\u003egic_fd);\n-\tkvm_vm_free(v-\u003evm);\n+\tstruct kvm_vcpu *vcpu;\n+\tstruct vm_gic v;\n+\tu64 attr, val;\n+\tint ret, i;\n+\n+\tv.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;\n+\tv.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);\n+\tv.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);\n+\tvcpu = vm_vcpu_add(v.vm, 0, NULL);\n+\tTEST_ASSERT(vcpu, \"Failed to create vCPU\");\n+\n+\t/* Check existing group/attribute */\n+\tattr = PACK_SR(SYS_ICC_CR0_EL1);\n+\tkvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS, attr);\n+\n+\t/* CPU sysregs are not accessible before the VGIC is initialized. */\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,\n+\t\t\t\t attr, \u0026val);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EBUSY, \"GICv5 CPU_SYSREGS get before init\");\n+\n+\tval = 0;\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,\n+\t\t\t\t attr, \u0026val);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EBUSY, \"GICv5 CPU_SYSREGS set before init\");\n+\n+\tkvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n+\t\t\t KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);\n+\n+\t/* Read all exposed CPU sysregs and write the value back. */\n+\tfor (i = 0; i \u003c ARRAY_SIZE(gic_v5_cpu_sysregs); i++) {\n+\t\tattr = PACK_SR(gic_v5_cpu_sysregs[i].encoding);\n+\t\tret = __kvm_has_device_attr(v.gic_fd,\n+\t\t\t\t\t KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,\n+\t\t\t\t\t attr);\n+\t\tTEST_ASSERT(!ret, \"GICv5 CPU_SYSREGS missing %s\",\n+\t\t\t gic_v5_cpu_sysregs[i].name);\n+\n+\t\tval = 0xbad;\n+\t\tret = __kvm_device_attr_get(v.gic_fd,\n+\t\t\t\t\t KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,\n+\t\t\t\t\t attr, \u0026val);\n+\t\tTEST_ASSERT(!ret, \"GICv5 CPU_SYSREGS get failed for %s\",\n+\t\t\t gic_v5_cpu_sysregs[i].name);\n+\n+\t\tret = __kvm_device_attr_set(v.gic_fd,\n+\t\t\t\t\t KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,\n+\t\t\t\t\t attr, \u0026val);\n+\t\tTEST_ASSERT(!ret, \"GICv5 CPU_SYSREGS set failed for %s\",\n+\t\t\t gic_v5_cpu_sysregs[i].name);\n+\t}\n+\n+\t/* Check non existent GICv3 sysreg */\n+\tattr = PACK_SR(SYS_ICC_CTLR_EL1);\n+\tret = __kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,\n+\t\t\t\t attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == ENXIO, \"GICv5 CPU_SYSREGS accepted bad sysreg\");\n+\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,\n+\t\t\t\t attr, \u0026val);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == ENOENT, \"GICv5 CPU_SYSREGS get bad sysreg\");\n+\n+\t/* Check non existing vCPU */\n+\tattr = PACK_SR(SYS_ICC_CR0_EL1) |\n+\t (1ULL \u003c\u003c KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT);\n+\tret = __kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,\n+\t\t\t\t attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 CPU_SYSREGS accepted bad MPIDR\");\n+\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,\n+\t\t\t\t attr, \u0026val);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 CPU_SYSREGS get bad MPIDR\");\n+\n+\t/* Check bad user pointers */\n+\tattr = PACK_SR(SYS_ICC_CR0_EL1);\n+\tret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,\n+\t\t\t\t attr, NULL);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EFAULT, \"GICv5 CPU_SYSREGS get with bad pointer\");\n+\n+\tret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,\n+\t\t\t\t attr, NULL);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EFAULT, \"GICv5 CPU_SYSREGS set with bad pointer\");\n+\n+\tvm_gic_destroy(\u0026v);\n }\n \n static void test_vgic_v5_ppis(u32 gic_dev_type)\n@@ -100,6 +1517,7 @@ static void test_vgic_v5_ppis(u32 gic_dev_type)\n \tstruct ucall uc;\n \tu64 user_ppis[2];\n \tstruct vm_gic v;\n+\tuint64_t attr;\n \tint ret, i;\n \n \tv.gic_dev_type = gic_dev_type;\n@@ -111,11 +1529,17 @@ static void test_vgic_v5_ppis(u32 gic_dev_type)\n \t\tvcpus[i] = vm_vcpu_add(v.vm, i, guest_code);\n \n \tvm_init_descriptor_tables(v.vm);\n-\tvm_install_exception_handler(v.vm, VECTOR_IRQ_CURRENT, guest_irq_handler);\n+\tvm_install_exception_handler(v.vm, VECTOR_IRQ_CURRENT,\n+\t\t\t\t guest_ppi_irq_handler);\n \n \tfor (i = 0; i \u003c NR_VCPUS; i++)\n \t\tvcpu_init_descriptor_tables(vcpus[i]);\n \n+\t/* Set the address of the IRS before initialising the GIC */\n+\tattr = GICV5_IRS_CONFIG_BASE_GPA;\n+\tkvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,\n+\t\t\t KVM_VGIC_V5_ADDR_TYPE_IRS, \u0026attr);\n+\n \tkvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,\n \t\t\t KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);\n \n@@ -127,6 +1551,20 @@ static void test_vgic_v5_ppis(u32 gic_dev_type)\n \tTEST_ASSERT(user_ppis[0] \u0026 BIT(GICV5_ARCH_PPI_SW_PPI),\n \t\t\"SW_PPI is not drivable by userspace\");\n \n+\t/* PPIs not explicitly exposed to userspace must be rejected. */\n+\tfor (i = 0; i \u003c VGIC_V5_NR_PRIVATE_IRQS; i++) {\n+\t\tif (user_ppis[i / 64] \u0026 BIT_ULL(i % 64))\n+\t\t\tcontinue;\n+\n+\t\tret = __vgic_v5_irq_line(v.vm, KVM_ARM_IRQ_TYPE_PPI, i, 1);\n+\t\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL,\n+\t\t\t \"GICv5 accepted non-userspace PPI %d\", i);\n+\t}\n+\n+\tret = __vgic_v5_irq_line(v.vm, KVM_ARM_IRQ_TYPE_PPI,\n+\t\t\t\t VGIC_V5_NR_PRIVATE_IRQS, 1);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 accepted out-of-range PPI\");\n+\n \twhile (1) {\n \t\tret = run_vcpu(vcpus[0]);\n \t\tif (ret)\n@@ -148,9 +1586,9 @@ static void test_vgic_v5_ppis(u32 gic_dev_type)\n \n \t\t\t\tkvm_irq_line(v.vm, irq, level);\n \t\t\t} else if (uc.args[1] == GUEST_CMD_IS_AWAKE) {\n-\t\t\t\tpr_info(\"Guest skipping WFI due to pending IRQ\\n\");\n+\t\t\t\tpr_debug(\"Guest skipping WFI due to pending IRQ\\n\");\n \t\t\t} else if (uc.args[1] == GUEST_CMD_IRQ_CDIA) {\n-\t\t\t\tpr_info(\"Guest acknowledged IRQ\\n\");\n+\t\t\t\tpr_debug(\"Guest acknowledged IRQ\\n\");\n \t\t\t}\n \n \t\t\tcontinue;\n@@ -170,6 +1608,247 @@ static void test_vgic_v5_ppis(u32 gic_dev_type)\n \tvm_gic_destroy(\u0026v);\n }\n \n+static void test_vgic_v5_spis(void)\n+{\n+\tstruct kvm_vcpu *vcpu;\n+\tstruct vm_gic v;\n+\tint ret;\n+\n+\t/* Default NR_IRQS exposes 32 SPIs, numbered 0..31 in KVM_IRQ_LINE. */\n+\tvgic_v5_spi_line_vm_create(\u0026v, \u0026vcpu, VGIC_V5_DEFAULT_NR_SPIS);\n+\tret = __vgic_v5_irq_line(v.vm, KVM_ARM_IRQ_TYPE_SPI,\n+\t\t\t\t VGIC_V5_DEFAULT_NR_SPIS, 1);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 accepted first invalid SPI\");\n+\n+\tvm_gic_destroy(\u0026v);\n+\n+\t/* Basic SPI injection through KVM_IRQ_LINE. */\n+\tvgic_v5_run_spi_line_test(VGIC_V5_DEFAULT_NR_SPIS, 0, false, false);\n+\n+\t/* The last valid SPI for a configured NR_IRQS is injectable. */\n+\tvgic_v5_run_spi_line_test(VGIC_V5_TEST_NR_SPIS,\n+\t\t\t\t VGIC_V5_TEST_NR_SPIS - 1, false, false);\n+\n+\tvgic_v5_spi_line_vm_create(\u0026v, \u0026vcpu, VGIC_V5_TEST_NR_SPIS);\n+\tret = __vgic_v5_irq_line(v.vm, KVM_ARM_IRQ_TYPE_SPI,\n+\t\t\t\t VGIC_V5_TEST_NR_SPIS, 1);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 accepted configured invalid SPI\");\n+\tvm_gic_destroy(\u0026v);\n+\n+\t/*\n+\t * Edge SPIs remain pending after the line is lowered. The low\n+\t * transition is injected before the guest runs and must be ignored:\n+\t * once an edge interrupt is pending, lowering the line cannot recall it.\n+\t */\n+\tvgic_v5_run_spi_line_test(VGIC_V5_DEFAULT_NR_SPIS, 1, false, true);\n+\n+\t/* Level SPIs can be raised and delivered through KVM_IRQ_LINE. */\n+\tvgic_v5_run_spi_line_test(VGIC_V5_DEFAULT_NR_SPIS, 2, true, false);\n+}\n+\n+static void test_vgic_v5_lpis(void)\n+{\n+\tstruct kvm_vcpu *vcpus[VGIC_V5_LPI_NR_VCPUS];\n+\tstruct ucall uc;\n+\tstruct vm_gic v;\n+\tint ret;\n+\n+\tlpi_ist_ready = false;\n+\tvgic_v5_lpi_vm_create(\u0026v, vcpus);\n+\tsync_global_to_guest(v.vm, lpi_ist_ready);\n+\n+\t/* VPE0 programs a linear LPI IST from the virtual IRS ID registers. */\n+\tret = run_vcpu(vcpus[0]);\n+\tTEST_ASSERT(!ret, \"Failed to run GICv5 LPI vCPU0\");\n+\tTEST_ASSERT(get_ucall(vcpus[0], \u0026uc) == UCALL_SYNC \u0026\u0026\n+\t\t uc.args[1] == GUEST_CMD_IS_READY,\n+\t\t \"GICv5 LPI vCPU0 did not become ready\");\n+\n+\t/* VPE1 observes the IST and enables the LPIs it will receive/send. */\n+\tret = run_vcpu(vcpus[1]);\n+\tTEST_ASSERT(!ret, \"Failed to run GICv5 LPI vCPU1\");\n+\tTEST_ASSERT(get_ucall(vcpus[1], \u0026uc) == UCALL_SYNC \u0026\u0026\n+\t\t uc.args[1] == GUEST_CMD_IS_READY,\n+\t\t \"GICv5 LPI vCPU1 did not become ready\");\n+\n+\t/* VPE0 sends an LPI to VPE1. */\n+\tret = run_vcpu(vcpus[0]);\n+\tTEST_ASSERT(!ret, \"Failed to run GICv5 LPI vCPU0 sender\");\n+\tTEST_ASSERT(get_ucall(vcpus[0], \u0026uc) == UCALL_SYNC \u0026\u0026\n+\t\t uc.args[1] == GUEST_CMD_LPI_SENT,\n+\t\t \"GICv5 LPI vCPU0 did not send LPI\");\n+\n+\t/* VPE1 consumes the LPI and replies with another LPI. */\n+\tret = run_vcpu(vcpus[1]);\n+\tTEST_ASSERT(!ret, \"Failed to run GICv5 LPI vCPU1 receiver\");\n+\tTEST_ASSERT(get_ucall(vcpus[1], \u0026uc) == UCALL_SYNC \u0026\u0026\n+\t\t uc.args[1] == GUEST_CMD_LPI_REPLIED,\n+\t\t \"GICv5 LPI vCPU1 did not reply\");\n+\n+\t/* VPE0 consumes the reply LPI and ends the test. */\n+\tret = run_vcpu(vcpus[0]);\n+\tTEST_ASSERT(!ret, \"Failed to run GICv5 LPI vCPU0 receiver\");\n+\tTEST_ASSERT(get_ucall(vcpus[0], \u0026uc) == UCALL_DONE,\n+\t\t \"GICv5 LPI vCPU0 did not complete\");\n+\n+\tvm_gic_destroy(\u0026v);\n+}\n+\n+static void test_vgic_v5_ist_save_restore(void)\n+{\n+\tstatic u32 lpi_ist[VGIC_V5_LPI_IST_SIZE / sizeof(u32)];\n+\tstatic u32 lpi_ist_resave[VGIC_V5_LPI_IST_SIZE / sizeof(u32)];\n+\tstatic u32 spi_ist[VGIC_V5_DEFAULT_NR_SPIS];\n+\tstatic u32 spi_ist_resave[VGIC_V5_DEFAULT_NR_SPIS];\n+\tstatic u64 irs_regs[ARRAY_SIZE(gic_v5_irs_regs)];\n+\tstruct kvm_vcpu *vcpus[VGIC_V5_LPI_NR_VCPUS];\n+\tstruct kvm_vgic_v5_ist ist_attr;\n+\tstruct vm_gic src, dst;\n+\tsize_t lpi_ist_size;\n+\tint ret;\n+\n+\tmemset(ist_restore_irq_count, 0, sizeof(ist_restore_irq_count));\n+\n+\tvgic_v5_ist_restore_vm_create(\u0026src, vcpus, guest_ist_source_code,\n+\t\t\t\t guest_unexpected_irq_handler);\n+\n+\tret = run_vcpu(vcpus[0]);\n+\tTEST_ASSERT(!ret, \"Failed to run GICv5 IST save source VM\");\n+\tvgic_v5_expect_sync(vcpus[0], GUEST_CMD_IST_SOURCE_READY,\n+\t\t\t \"GICv5 IST source guest did not become ready\");\n+\n+\t/*\n+\t * The guest configured enable/priority/affinity state. Make selected\n+\t * SPIs pending from userspace so the SPI IST save path sees them.\n+\t */\n+\tvgic_v5_spi_line(src.vm, RESTORE_TEST_SPI_PRIO_HIGH, 1);\n+\tvgic_v5_spi_line(src.vm, RESTORE_TEST_SPI_PRIO_LOW, 1);\n+\tvgic_v5_spi_line(src.vm, RESTORE_TEST_SPI_PENDING_DISABLED, 1);\n+\n+\tvgic_v5_save_irs_regs(\u0026src, irs_regs);\n+\tlpi_ist_size = vgic_v5_lpi_ist_state_size(irs_regs);\n+\n+\t/*\n+\t * VCPU0 has configured the initial state. KVM must reject missing or\n+\t * incorrectly sized LPI storage before accepting the complete IST image.\n+\t */\n+\tist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), NULL, 0);\n+\tret = __kvm_device_attr_get(src.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, \u0026ist_attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IST save accepted missing LPI buffer\");\n+\n+\tist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), lpi_ist,\n+\t\t\t\t\t lpi_ist_size - sizeof(__u32));\n+\tret = __kvm_device_attr_get(src.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, \u0026ist_attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IST save accepted bad LPI size\");\n+\n+\tist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), lpi_ist,\n+\t\t\t\t\t lpi_ist_size);\n+\tret = __kvm_device_attr_get(src.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, \u0026ist_attr);\n+\tTEST_ASSERT(!ret, \"GICv5 IST save failed\");\n+\n+\tvm_gic_destroy(\u0026src);\n+\n+\t/* Create the guest that we are restoring state into */\n+\tmemset(ist_restore_irq_count, 0, sizeof(ist_restore_irq_count));\n+\tvgic_v5_ist_restore_vm_create(\u0026dst, vcpus, guest_ist_restore_code,\n+\t\t\t\t guest_ist_restore_irq_handler);\n+\tsync_global_to_guest(dst.vm, ist_restore_irq_count);\n+\n+\tist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), lpi_ist,\n+\t\t\t\t\t lpi_ist_size);\n+\tret = __kvm_device_attr_set(dst.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, \u0026ist_attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IST restore before IRS state\");\n+\n+\tvgic_v5_restore_irs_regs(\u0026dst, irs_regs);\n+\n+\tist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), NULL, 0);\n+\tret = __kvm_device_attr_set(dst.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, \u0026ist_attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IST restore accepted missing LPI buffer\");\n+\n+\tist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), lpi_ist,\n+\t\t\t\t\t lpi_ist_size - sizeof(__u32));\n+\tret = __kvm_device_attr_set(dst.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, \u0026ist_attr);\n+\tTEST_ASSERT(ret \u0026\u0026 errno == EINVAL, \"GICv5 IST restore accepted bad LPI size\");\n+\n+\t/* Restore the VMM-owned SPI and LPI IST images. */\n+\tist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), lpi_ist,\n+\t\t\t\t\t lpi_ist_size);\n+\tret = __kvm_device_attr_set(dst.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, \u0026ist_attr);\n+\tTEST_ASSERT(!ret, \"GICv5 IST restore failed\");\n+\n+\t/*\n+\t * Re-save the restored IST image before the destination VM runs, so a\n+\t * dropped serialized field fails independently of later interrupt\n+\t * delivery.\n+\t */\n+\tist_attr = vgic_v5_ist_attr(spi_ist_resave, sizeof(spi_ist_resave),\n+\t\t\t\t lpi_ist_resave, lpi_ist_size);\n+\tret = __kvm_device_attr_get(dst.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,\n+\t\t\t\t 0, \u0026ist_attr);\n+\tTEST_ASSERT(!ret, \"GICv5 IST re-save failed\");\n+\tTEST_ASSERT(!memcmp(spi_ist, spi_ist_resave, sizeof(spi_ist)),\n+\t\t \"GICv5 SPI IST state changed across restore\");\n+\tTEST_ASSERT(!memcmp(lpi_ist, lpi_ist_resave, lpi_ist_size),\n+\t\t \"GICv5 LPI IST state changed across restore\");\n+\n+\t/*\n+\t * At this stage, the guest's IST state should match what was saved. Run\n+\t * each vcpu once to let it enable interrupts, etc.\n+\t */\n+\tret = run_vcpu(vcpus[0]);\n+\tTEST_ASSERT(!ret, \"Failed to run GICv5 IST restore vCPU0\");\n+\tvgic_v5_expect_sync(vcpus[0], GUEST_CMD_IS_READY,\n+\t\t\t \"GICv5 IST restore vCPU0 did not become ready\");\n+\n+\tret = run_vcpu(vcpus[1]);\n+\tTEST_ASSERT(!ret, \"Failed to run GICv5 IST restore vCPU1\");\n+\tvgic_v5_expect_sync(vcpus[1], GUEST_CMD_IS_READY,\n+\t\t\t \"GICv5 IST restore vCPU1 did not become ready\");\n+\n+\t/*\n+\t * VPE0 consumes two pending SPIs in restored priority order, then\n+\t * enables a restored pending-but-disabled SPI.\n+\t */\n+\tret = run_vcpu(vcpus[0]);\n+\tTEST_ASSERT(!ret, \"Failed to consume restored GICv5 SPIs\");\n+\tvgic_v5_expect_sync(vcpus[0], GUEST_CMD_IST_SPI_INJECT_READY,\n+\t\t\t \"GICv5 IST restore vCPU0 did not consume restored SPIs\");\n+\n+\t/*\n+\t * This SPI was enabled before save, but not pending. Inject it after\n+\t * restore to prove enable, priority and affinity were restored.\n+\t */\n+\tvgic_v5_spi_line(dst.vm, RESTORE_TEST_SPI_ENABLED, 1);\n+\n+\tret = run_vcpu(vcpus[0]);\n+\tTEST_ASSERT(!ret, \"Failed to consume post-restore GICv5 SPI\");\n+\tTEST_ASSERT(get_ucall(vcpus[0], NULL) == UCALL_DONE,\n+\t\t \"GICv5 IST restore vCPU0 did not complete\");\n+\n+\t/*\n+\t * VPE1 consumes a restored pending LPI, then pends another LPI that\n+\t * was enabled before save but not pending.\n+\t */\n+\tret = run_vcpu(vcpus[1]);\n+\tTEST_ASSERT(!ret, \"Failed to consume restored GICv5 LPI\");\n+\tvgic_v5_expect_sync(vcpus[1], GUEST_CMD_IST_LPI_INJECT_READY,\n+\t\t\t \"GICv5 IST restore vCPU1 did not consume restored LPI\");\n+\n+\tret = run_vcpu(vcpus[1]);\n+\tTEST_ASSERT(!ret, \"Failed to consume post-restore GICv5 LPI\");\n+\tTEST_ASSERT(get_ucall(vcpus[1], NULL) == UCALL_DONE,\n+\t\t \"GICv5 IST restore vCPU1 did not complete\");\n+\n+\tvm_gic_destroy(\u0026dst);\n+}\n+\n /*\n * Returns 0 if it's possible to create GIC device of a given type (V5).\n */\n@@ -201,13 +1880,43 @@ int test_kvm_device(u32 gic_dev_type)\n \n void run_tests(u32 gic_dev_type)\n {\n+\tpr_info(\"Test VGICv5 address attrs\\n\");\n+\ttest_vgic_v5_addr_attrs();\n+\n+\tpr_info(\"Test VGICv5 NR_IRQS attrs\\n\");\n+\ttest_vgic_v5_nr_irqs_attrs();\n+\n+\tpr_info(\"Test VGICv5 IRS_REGS attrs\\n\");\n+\ttest_vgic_v5_irs_regs_attrs();\n+\n+\tpr_info(\"Test VGICv5 IST attrs\\n\");\n+\ttest_vgic_v5_ist_attrs();\n+\n+\tpr_info(\"Test VGICv5 userspace PPI attrs\\n\");\n+\ttest_vgic_v5_userspace_ppis_attrs();\n+\n+\tpr_info(\"Test VGICv5 CPU sysreg attrs\\n\");\n+\ttest_vgic_v5_cpu_sysreg_attrs();\n+\n \tpr_info(\"Test VGICv5 PPIs\\n\");\n \ttest_vgic_v5_ppis(gic_dev_type);\n+\n+\tpr_info(\"Test VGICv5 SPIs\\n\");\n+\ttest_vgic_v5_spis();\n+\n+\tpr_info(\"Test VGICv5 LPIs\\n\");\n+\ttest_vgic_v5_lpis();\n+\n+\tpr_info(\"Test VGICv5 IST save/restore\\n\");\n+\ttest_vgic_v5_ist_save_restore();\n }\n \n int main(int ac, char **av)\n {\n-\tint ret;\n+\tint pa_bits, ret;\n+\n+\tpa_bits = vm_guest_mode_params[VM_MODE_DEFAULT].pa_bits;\n+\tmax_phys_size = 1ULL \u003c\u003c pa_bits;\n \n \ttest_disable_default_vgic();\n \ndiff --git a/tools/testing/selftests/kvm/include/arm64/gic_v5.h b/tools/testing/selftests/kvm/include/arm64/gic_v5.h\nindex eb523d9277cf1e..2d455515563e0a 100644\n--- a/tools/testing/selftests/kvm/include/arm64/gic_v5.h\n+++ b/tools/testing/selftests/kvm/include/arm64/gic_v5.h\n@@ -10,6 +10,106 @@\n \n #include \"processor.h\"\n \n+/* GIC component base address is guest PA space */\n+#define GICV5_IRS_CONFIG_BASE_GPA\t0x8000000ULL\n+#define GICV5_IRS_CONFIG_BASE_GVA\t((volatile void *)GICV5_IRS_CONFIG_BASE_GPA)\n+\n+#define GICV5_IRS_SIZE\t\t\t0x20000\n+#define GICV5_IRS_ALIGN\t\t\t0x10000\n+\n+/*\n+ * IRS register offsets. Keep this selftest copy scoped to the offsets and\n+ * fields exercised by the userspace API tests.\n+ */\n+#define GICV5_IRS_IDR0\t\t\t0x0000\n+#define GICV5_IRS_IDR1\t\t\t0x0004\n+#define GICV5_IRS_IDR2\t\t\t0x0008\n+#define GICV5_IRS_IDR3\t\t\t0x000c\n+#define GICV5_IRS_IDR4\t\t\t0x0010\n+#define GICV5_IRS_IDR5\t\t\t0x0014\n+#define GICV5_IRS_IDR6\t\t\t0x0018\n+#define GICV5_IRS_IDR7\t\t\t0x001c\n+#define GICV5_IRS_IIDR\t\t\t0x0040\n+#define GICV5_IRS_AIDR\t\t\t0x0044\n+#define GICV5_IRS_CR0\t\t\t0x0080\n+#define GICV5_IRS_CR1\t\t\t0x0084\n+#define GICV5_IRS_SYNCR\t\t\t0x00c0\n+#define GICV5_IRS_SYNC_STATUSR\t\t0x00c4\n+#define GICV5_IRS_SPI_VMR\t\t0x0100\n+#define GICV5_IRS_SPI_SELR\t\t0x0108\n+#define GICV5_IRS_SPI_DOMAINR\t\t0x010c\n+#define GICV5_IRS_SPI_RESAMPLER\t\t0x0110\n+#define GICV5_IRS_SPI_CFGR\t\t0x0114\n+#define GICV5_IRS_SPI_STATUSR\t\t0x0118\n+#define GICV5_IRS_PE_SELR\t\t0x0140\n+#define GICV5_IRS_PE_STATUSR\t\t0x0144\n+#define GICV5_IRS_PE_CR0\t\t0x0148\n+#define GICV5_IRS_IST_BASER\t\t0x0180\n+#define GICV5_IRS_IST_CFGR\t\t0x0190\n+#define GICV5_IRS_IST_STATUSR\t\t0x0194\n+#define GICV5_IRS_MAP_L2_ISTR\t\t0x01c0\n+#define GICV5_IRS_VMT_BASER\t\t0x0200\n+#define GICV5_IRS_VMT_CFGR\t\t0x0210\n+#define GICV5_IRS_VMT_STATUSR\t\t0x0214\n+#define GICV5_IRS_VPE_SELR\t\t0x0240\n+#define GICV5_IRS_VPE_DBR\t\t0x0248\n+#define GICV5_IRS_VPE_HPPIR\t\t0x0250\n+#define GICV5_IRS_VPE_CR0\t\t0x0258\n+#define GICV5_IRS_VPE_STATUSR\t\t0x025c\n+#define GICV5_IRS_VM_DBR\t\t0x0280\n+#define GICV5_IRS_VM_SELR\t\t0x0288\n+#define GICV5_IRS_VM_STATUSR\t\t0x028c\n+#define GICV5_IRS_VMAP_L2_VMTR\t\t0x02c0\n+#define GICV5_IRS_VMAP_VMR\t\t0x02c8\n+#define GICV5_IRS_VMAP_VISTR\t\t0x02d0\n+#define GICV5_IRS_VMAP_L2_VISTR\t\t0x02d8\n+#define GICV5_IRS_VMAP_VPER\t\t0x02e0\n+#define GICV5_IRS_SAVE_VMR\t\t0x0300\n+#define GICV5_IRS_SAVE_VM_STATUSR\t0x0308\n+#define GICV5_IRS_MEC_IDR\t\t0x0340\n+#define GICV5_IRS_MEC_MECID_R\t\t0x0344\n+#define GICV5_IRS_MPAM_IDR\t\t0x0380\n+#define GICV5_IRS_MPAM_PARTID_R\t\t0x0384\n+#define GICV5_IRS_SWERR_STATUSR\t\t0x03c0\n+#define GICV5_IRS_SWERR_SYNDROMER0\t0x03c8\n+#define GICV5_IRS_SWERR_SYNDROMER1\t0x03d0\n+\n+#define GICV5_IRS_IDR0_SETLPI\t\tBIT(9)\n+#define GICV5_IRS_IDR0_INT_DOM\t\tGENMASK(1, 0)\n+#define GICV5_IRS_IDR0_INT_DOM_SECURE\t\t0b00\n+\n+#define GICV5_IRS_IDR1_PRIORITY_BITS\tGENMASK(22, 20)\n+\n+#define GICV5_IRS_IDR2_ISTMD_SZ\tGENMASK(19, 15)\n+#define GICV5_IRS_IDR2_ISTMD\t\tBIT(14)\n+#define GICV5_IRS_IDR2_IST_L2SZ\tGENMASK(13, 11)\n+#define GICV5_IRS_IDR2_IST_LEVELS\tBIT(10)\n+#define GICV5_IRS_IDR2_MIN_LPI_ID_BITS\tGENMASK(9, 6)\n+#define GICV5_IRS_IDR2_LPI\t\tBIT(5)\n+#define GICV5_IRS_IDR2_ID_BITS\t\tGENMASK(4, 0)\n+\n+#define GICV5_IRS_CR0_IDLE\t\tBIT(1)\n+#define GICV5_IRS_CR0_IRSEN\t\tBIT(0)\n+\n+#define GICV5_IRS_SPI_SELR_ID\t\tGENMASK(23, 0)\n+#define GICV5_IRS_SPI_CFGR_TM\t\tBIT(0)\n+\n+#define GICV5_IRS_SYNC_STATUSR_IDLE\tBIT(0)\n+#define GICV5_IRS_SPI_STATUSR_IDLE\tBIT(0)\n+#define GICV5_IRS_PE_STATUSR_IDLE\tBIT(0)\n+#define GICV5_IRS_IST_STATUSR_IDLE\tBIT(0)\n+\n+#define GICV5_IRS_IST_BASER_ADDR_MASK\tGENMASK_ULL(55, 6)\n+#define GICV5_IRS_IST_BASER_VALID\tBIT_ULL(0)\n+#define GICV5_IRS_IST_BASER_ADDR_SHIFT\t6ULL\n+#define GICV5_IRS_IST_CFGR_STRUCTURE\tBIT(16)\n+#define GICV5_IRS_IST_CFGR_ISTSZ\tGENMASK(8, 7)\n+#define GICV5_IRS_IST_CFGR_L2SZ\tGENMASK(6, 5)\n+#define GICV5_IRS_IST_CFGR_LPI_ID_BITS\tGENMASK(4, 0)\n+#define GICV5_IRS_IST_CFGR_ISTSZ_4\t0b00\n+#define GICV5_IRS_IST_CFGR_ISTSZ_8\t0b01\n+#define GICV5_IRS_IST_CFGR_ISTSZ_16\t0b10\n+\n /*\n * Definitions for GICv5 instructions for the Current Domain\n */\n@@ -77,6 +177,11 @@\n #define GICV5_GICR_CDNMIA_TYPE_MASK\tGENMASK_ULL(31, 29)\n #define GICV5_GICR_CDNMIA_ID_MASK\tGENMASK_ULL(23, 0)\n \n+#define GICV5_HWIRQ_TYPE\t\tGENMASK_ULL(31, 29)\n+#define GICV5_HWIRQ_ID\t\t\tGENMASK_ULL(23, 0)\n+#define GICV5_HWIRQ_TYPE_LPI\t\t0x2\n+#define GICV5_HWIRQ_TYPE_SPI\t\t0x3\n+\n #define gicr_insn(insn)\t\t\tread_sysreg_s(GICV5_OP_GICR_##insn)\n #define gic_insn(v, insn)\t\twrite_sysreg_s(v, GICV5_OP_GIC_##insn)\n \n"
} |
| 2/1 | 2026/07/25 16:10 | agent | patch-evaluator |
4mModel:best-expensive Args: null Results: {
"EnableConfigs": [
"KVM"
],
"FocusSymbols": [
"vgic_v5_parse_attr",
"vgic_v5_irs_attr_regs_access",
"vgic_v5_has_attr_regs",
"kvm_vgic_v5_irs_init",
"vgic_v5_register_irs_iodev",
"vgic_v5_cpu_sysregs_uaccess",
"vgic_v5_has_cpu_sysregs_attr",
"vgic_v5_irs_save_ists",
"vgic_v5_irs_restore_ists",
"vgic_v5_vmt_allocate",
"vgic_v5_vmt_free",
"vgic_v5_allocate_vm_id",
"vgic_v5_release_vm_id",
"vgic_v5_vmte_init",
"vgic_v5_vmte_release",
"vgic_v5_vmte_alloc_vpe",
"vgic_v5_vmte_free_vpe",
"vgic_v5_spi_ist_alloc",
"vgic_v5_lpi_ist_alloc",
"vgic_v5_lpi_ist_free",
"vgic_v5_save_spi_ist",
"vgic_v5_save_lpi_ist",
"vgic_v5_restore_spi_ist",
"vgic_v5_restore_lpi_ist"
],
"Reasoning": "The patch series introduces support for GICv5 IRS (Interrupt Routing Service), SPIs, and LPIs in KVM. It adds new KVM device attributes for GICv5, including IRS registers, CPU sysregs, and IST (Interrupt State Table) save/restore functionality. This is a significant functional change to the KVM ARM VGIC subsystem that introduces new reachable code paths and APIs, making it highly relevant for fuzzing.",
"WorthFuzzing": true
}Instruction: You are an expert Linux kernel maintainer. Your job is to review a provided patch series and determine if it makes functional changes to the kernel that should be fuzzed. IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in your workspace. Do NOT rely on your internal knowledge of the kernel. You must actively use your code access tools to examine the actual source code and confirm any assumptions. Return WorthFuzzing=false if the patch only contains: - Modifications to Documentation/, Kconfig files, or code comments. - Purely decorative changes, such as logging (e.g., pr_err, printk) or tracepoints. - Changes to numeric constants or macros that do not functionally alter execution flow. - Code paths that are impossible to reach in virtualized environments like GCE or QEMU, even when utilizing software-emulated hardware (e.g., usb gadget, mac80211_hwsim). - Code in vendor-specific PCIe switch, SmartNIC, or GPU drivers (e.g., mlxsw, pds_core, qed, ionic, amdgpu) that require physical PCIe hardware cards not emulated in standard QEMU. - Driver .remove, .shutdown, or pci_unregister_driver teardown callbacks (e.g., igb_remove) that are executed only during PCI hot-unplug or sysfs driver unbind operations. If it modifies reachable core kernel logic, drivers, or architectures, use your code search tools to verify the code can be executed, then return WorthFuzzing=true. When returning WorthFuzzing=true, you MUST ALSO: 1. Extract any specific kernel functions that should be heavily fuzzed into FocusSymbols. Avoid listing generic hot-path functions to prevent skewed test distributions. Prefer non-static, non-inlined API entrypoint functions over internal static helper functions (which are inlined by the compiler and do not have distinct symbol addresses). 2. Identify any specific CONFIG_ options required to properly test this new/modified feature. Go and look into the Kconfig files and check for ifdefs around the code, do not make assumptions. Also check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed for Kbuild to compile the code statically into vmlinux. List them in the EnableConfigs output array, and DO NOT add a 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4"). Prefer calling several tools at the same time to save round-trips. Use set-results tool to provide results of the analysis. It must be called exactly once before the final reply. Ignore results of this tool. Prompt: For your convenience, here is the diff of the changes:
commit eb3e8c4cd3bdcee09fe7080f97581a948e8892e5
Author: syz-cluster <triage@syzkaller.com>
Date: Sat Jul 25 16:10:13 2026 +0000
syz-cluster: applied patch under review
diff --git a/Documentation/virt/kvm/devices/arm-vgic-v5.rst b/Documentation/virt/kvm/devices/arm-vgic-v5.rst
index 70b9162755c7e8..6813051789ac48 100644
--- a/Documentation/virt/kvm/devices/arm-vgic-v5.rst
+++ b/Documentation/virt/kvm/devices/arm-vgic-v5.rst
@@ -12,12 +12,39 @@ Only one VGIC instance may be instantiated through this API. The created VGIC
will act as the VM interrupt controller, requiring emulated user-space devices
to inject interrupts to the VGIC instead of directly to CPUs.
-Creating a guest GICv5 device requires a GICv5 host. The current VGICv5 device
-only supports PPI interrupts. These can either be injected from emulated
-in-kernel devices (such as the Arch Timer, or PMU), or via the KVM_IRQ_LINE
-ioctl.
+Creating a guest GICv5 device requires a GICv5 host. The VGICv5 device supports
+PPI, SPI, and LPI interrupts. The PPI and SPI interrupts can either be injected
+from emulated in-kernel devices (such as the Arch Timer, or PMU), or via the
+KVM_IRQ_LINE ioctl. LPIs are not externally injected, but are handled in
+hardware via the LPI IST. Their pending state is driven directly by the guest.
Groups:
+ KVM_DEV_ARM_VGIC_GRP_ADDR
+ Attributes:
+
+ KVM_VGIC_V5_ADDR_TYPE_IRS (rw, 64-bit)
+ Base address in the guest physical address space of the GICv5 IRS
+ (Interrupt Routing Service) register mappings. Only valid for
+ KVM_DEV_TYPE_ARM_VGIC_V5. This address needs to be 64K aligned and the
+ region covers 128 KByte - the IRS has a CONFIG_FRAME and a SETLPI_FRAME,
+ each of which is 64 KBytes in size.
+
+ Setting the address of the IRS in GPA space is mandatory before VGIC
+ resources are mapped, as the IRS is responsible for handling SPIs and
+ LPIs. Failure to set the IRS address before the first vCPU run results in
+ an error.
+
+ KVM_DEV_ARM_VGIC_GRP_NR_IRQS
+ Attributes:
+
+ A value describing the number of SPIs for this GIC instance. This is
+ GICv5-specific: unlike GICv2/v3, the value does not include SGIs or PPIs.
+ The value ranges from 32 to KVM's VGICv5 maximum of 65536 SPIs, in
+ increments of 32. If userspace does not set this attribute, KVM uses 32
+ SPIs by default.
+
+ kvm_device_attr.addr points to a __u32 value.
+
KVM_DEV_ARM_VGIC_GRP_CTRL
Attributes:
@@ -48,3 +75,232 @@ Groups:
-EFAULT Invalid guest ram access
-EBUSY One or more VCPUS are running
======= ========================================================
+
+ KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS
+ Attributes:
+
+ The attr field of kvm_device_attr encodes two values::
+
+ bits: | 63 .... 32 | 31 .... 16 | 15 .... 0 |
+ values: | mpidr | RES | instr |
+
+ The mpidr field encodes the CPU ID based on the affinity information in the
+ architecture defined MPIDR, and the field is encoded as follows::
+
+ | 63 .... 56 | 55 .... 48 | 47 .... 40 | 39 .... 32 |
+ | Aff3 | Aff2 | Aff1 | Aff0 |
+
+ The instr field encodes the system register to access based on the fields
+ defined in the A64 instruction set encoding for system register access
+ (RES means the bits are reserved for future use and should be zero)::
+
+ | 15 ... 14 | 13 ... 11 | 10 ... 7 | 6 ... 3 | 2 ... 0 |
+ | Op 0 | Op1 | CRn | CRm | Op2 |
+
+ All system regs accessed through this API are (rw, 64-bit) and
+ kvm_device_attr.addr points to a __u64 value.
+
+ KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS accesses the CPU interface registers for the
+ CPU specified by the mpidr field.
+
+ The available registers are:
+
+ ======================= ===================================================
+ ICC_ICSR_EL1
+ ICC_PPI_ENABLER0_EL1
+ ICC_PPI_ENABLER1_EL1
+ ICC_PPI_SACTIVER0_EL1 ICC_PPI_CACTIVER0_EL1 is not supported. Writes to
+ ICC_PPI_SACTIVER0_EL1 are treated as RAW writes of
+ the underlying state.
+ ICC_PPI_SACTIVER1_EL1 ICC_PPI_CACTIVER1_EL1 is not supported. Writes to
+ ICC_PPI_SACTIVER1_EL1 are treated as RAW writes of
+ the underlying state.
+ ICC_PPI_SPENDR0_EL1 ICC_PPI_CPENDR0_EL1 is not supported. Writes to
+ ICC_PPI_SPENDR0_EL1 are treated as RAW writes of
+ the underlying state.
+ ICC_PPI_SPENDR1_EL1 ICC_PPI_CPENDR1_EL1 is not supported. Writes to
+ ICC_PPI_SPENDR1_EL1 are treated as RAW writes of
+ the underlying state.
+ ICC_PPI_PRIORITYR0_EL1
+ ICC_PPI_PRIORITYR1_EL1
+ ICC_PPI_PRIORITYR2_EL1
+ ICC_PPI_PRIORITYR3_EL1
+ ICC_PPI_PRIORITYR4_EL1
+ ICC_PPI_PRIORITYR5_EL1
+ ICC_PPI_PRIORITYR6_EL1
+ ICC_PPI_PRIORITYR7_EL1
+ ICC_PPI_PRIORITYR8_EL1
+ ICC_PPI_PRIORITYR9_EL1
+ ICC_PPI_PRIORITYR10_EL1
+ ICC_PPI_PRIORITYR11_EL1
+ ICC_PPI_PRIORITYR12_EL1
+ ICC_PPI_PRIORITYR13_EL1
+ ICC_PPI_PRIORITYR14_EL1
+ ICC_PPI_PRIORITYR15_EL1
+ ICC_APR_EL1
+ ICC_CR0_EL1
+ ICC_PCR_EL1
+ ======================= ===================================================
+
+ Errors:
+
+ ======= =============================================================
+ -ENXIO Getting or setting this register is not supported
+ -EBUSY VCPU is running, or write attempted after a VCPU has run
+ -EINVAL Invalid mpidr or register value supplied
+ ======= =============================================================
+
+
+KVM_DEV_ARM_VGIC_GRP_IRS_REGS
+ Attributes:
+ The attr field of kvm_device_attr encodes the offset of the IRS register,
+ relative to the IRS CONFIG_FRAME base address. This is the address that
+ was provided via KVM_VGIC_V5_ADDR_TYPE_IRS when creating VGICv5 in the
+ first place.
+
+ kvm_device_attr.addr points to a __u64 value whatever the width
+ of the addressed register (32/64 bits). 64 bit registers can only
+ be accessed with full length.
+
+ Writes to read-only registers are ignored by the kernel except for
+ IRS_IDR0 - IRS_IDR2 and IRS_IDR5 - IRS_IDR7, which are sanity checked to
+ ensure that they match a sane config.
+
+ IRS_IDR3 and IRS_IDR4 are RAZ/WI, as nested virtualization is not
+ supported.
+
+ For registers without dedicated userspace accessors, getting or setting a
+ register uses the same emulated MMIO handlers as guest reads/writes.
+ Dedicated userspace accessors may instead save or restore migration state
+ without triggering guest-visible side effects. For example, restoring
+ IRS_IST_BASER only restores the emulated register state; any host LPI IST
+ allocation based on the restored IRS_IST_CFGR and IRS_IST_BASER state
+ happens when KVM_DEV_ARM_VGIC_GRP_IST is restored.
+
+ Errors:
+
+ ======= =================================================================
+ -ENXIO Offset does not correspond to any supported register
+ -EFAULT Invalid user pointer for attr->addr
+ -EINVAL Offset is not 32-bit aligned for 32-bit MMIO registers, not
+ 64-bit aligned for 64-bit registers, or the supplied register
+ value is not compatible with the configured VGICv5 IRS state
+ -EBUSY VGIC is not initialized, or one or more VCPUs are running
+ ======= =================================================================
+
+ KVM_DEV_ARM_VGIC_GRP_IST
+ Attributes:
+ This interface is used to either save the state of the IRS's Interrupt
+ State Tables (ISTs), or to restore them. A get operation saves IST state,
+ and a set operation restores IST state. kvm_device_attr.attr is reserved
+ and must be zero.
+
+ The VGIC must be initialized before using this interface. Restore must be
+ performed before the VM has run. For restore, userspace must have already
+ restored the IRS state needed to describe any guest LPI IST.
+
+ Saving first asks the IRS to save and quiesce the VM so that interrupt
+ state has been written back to the ISTs. KVM checks that the VM remains
+ quiesced while copying out the SPI and LPI IST state.
+
+ kvm_device_attr.addr points to a struct kvm_vgic_v5_ist::
+
+ struct kvm_vgic_v5_ist {
+ __u64 spi_ist_addr;
+ __u64 spi_ist_size;
+ __u64 lpi_ist_addr;
+ __u64 lpi_ist_size;
+ };
+
+ The SPI and LPI IST buffers contain one little-endian 32-bit IST entry per
+ interrupt, in interrupt number order. Only the architected 32-bit ISTE
+ state is exposed to userspace; host IST layout and metadata are private to
+ KVM.
+
+ The SPI IST buffer is required and its size must be:
+
+ nr_spis * sizeof(__u32)
+
+ where nr_spis is the value returned by KVM_DEV_ARM_VGIC_GRP_NR_IRQS for
+ the VGICv5 device. For VGICv5 this value is the number of SPIs, not the
+ total number of interrupts.
+
+ The LPI IST buffer is required if the guest's IRS_IST_CFGR and
+ IRS_IST_BASER state describes a valid LPI IST, and its size must be:
+
+ BIT(lpi_id_bits) * sizeof(__u32)
+
+ where lpi_id_bits is the LPI ID width configured in IRS_IST_CFGR. If no
+ LPI IST is configured, lpi_ist_addr and lpi_ist_size must both be zero.
+
+ Errors:
+
+ =========== ============================================================
+ -EBUSY One or more VCPUs are running, the VGIC is not initialized,
+ restore was requested after the VM has run, an LPI IST
+ already exists, or the save operation completed but the VM
+ did not remain quiesced
+ -EINVAL attr->addr is NULL, the userspace IST descriptor is missing
+ a required buffer, supplies a non-zero LPI buffer when no
+ LPI IST is configured, has a buffer size that does not match
+ the configured VGICv5 IST state, or an internal VM table
+ operation rejected the VM state
+ -ENOENT A userspace SPI IST buffer was supplied, but there is no SPI
+ IST to serialise/unserialise
+ -EFAULT Invalid user pointer for attr->addr, spi_ist_addr, or
+ lpi_ist_addr
+ -ENXIO Required per-VM VGICv5/IST backing state is missing or
+ inconsistent
+ -ENOMEM Restoring IST state failed while allocating the host LPI IST
+ or tracking pending interrupts
+ -ETIMEDOUT An IRS save/VM operation timed out
+ =========== ============================================================
+
+IRS Save Sequence:
+------------------
+
+The following operations are required when saving the virtual GICv5 IRS:
+
+a) Save the ISTs by issuing KVM_GET_DEVICE_ATTR on KVM_DEV_ARM_VGIC_GRP_IST.
+b) Save the IRS MMIO register state by issuing KVM_GET_DEVICE_ATTR on
+ KVM_DEV_ARM_VGIC_GRP_IRS_REGS.
+
+These two steps may be performed in either order. Saving the ISTs writes all
+IST state to userspace-provided buffers and does not update guest memory.
+
+IRS Restore Sequence:
+---------------------
+
+The following ordering must be followed when restoring the virtual GICv5 and
+IRS:
+
+a) Create vCPUs.
+b) Provide the IRS base address by issuing KVM_SET_DEVICE_ATTR on
+ KVM_DEV_ARM_VGIC_GRP_ADDR
+c) Restore the number of SPIs by issuing KVM_SET_DEVICE_ATTR on
+ KVM_DEV_ARM_VGIC_GRP_NR_IRQS.
+d) Initialise the GIC - this sets up the default state and creates the SPI
+ IST - by issuing KVM_SET_DEVICE_ATTR on KVM_DEV_ARM_VGIC_GRP_CTRL with
+ KVM_DEV_ARM_VGIC_CTRL_INIT
+e) Restore the IRS MMIO register state by issuing KVM_SET_DEVICE_ATTR on
+ KVM_DEV_ARM_VGIC_GRP_IRS_REGS. The IRS registers may be restored in any
+ order, but any IRS_IDR* state and the IRS_IST_CFGR/IRS_IST_BASER state
+ describing an LPI IST must be restored before restoring the ISTs. KVM uses
+ the restored IRS_IST_CFGR and IRS_IST_BASER state to allocate the LPI IST
+ during the following step.
+f) Restore the ISTs by issuing KVM_SET_DEVICE_ATTR on
+ KVM_DEV_ARM_VGIC_GRP_IST.
+
+The number of SPIs must be restored before VGIC initialization because
+initialization allocates the SPI state and fixes the SPI range exposed by the
+IRS ID registers.
+
+The IRS's MMIO registers must be restored prior to restoring the ISTs as these
+are used to convey the number of LPIs the guest has configured to KVM.
+
+The various ``*_STATUSR`` registers are observational state in the current KVM
+implementation. Userspace may save them for validation or debugging purposes,
+but they are not required as restore input and do not need to be replayed during
+restore.
+
+Then vCPUs can be started.
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index 043495f7fc78b9..36808ed0e9afe4 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -87,6 +87,9 @@ enum __kvm_host_smccc_func {
__KVM_HOST_SMCCC_FUNC___tracing_write_event,
__KVM_HOST_SMCCC_FUNC___vgic_v3_save_aprs,
__KVM_HOST_SMCCC_FUNC___vgic_v3_restore_vmcr_aprs,
+ __KVM_HOST_SMCCC_FUNC___vgic_v5_make_resident,
+ __KVM_HOST_SMCCC_FUNC___vgic_v5_make_non_resident,
+ __KVM_HOST_SMCCC_FUNC___vgic_v5_vdpend,
__KVM_HOST_SMCCC_FUNC___vgic_v5_save_apr,
__KVM_HOST_SMCCC_FUNC___vgic_v5_restore_vmcr_apr,
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index bae2c4f92ef5c0..b04dcc9a73c288 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -55,6 +55,7 @@
#define KVM_REQ_GUEST_HYP_IRQ_PENDING KVM_ARCH_REQ(9)
#define KVM_REQ_MAP_L1_VNCR_EL2 KVM_ARCH_REQ(10)
#define KVM_REQ_VGIC_PROCESS_UPDATE KVM_ARCH_REQ(11)
+#define KVM_REQ_RELOAD_GICv5 KVM_ARCH_REQ(12)
#define KVM_DIRTY_LOG_MANUAL_CAPS (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE | \
KVM_DIRTY_LOG_INITIALLY_SET)
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 4974492744cc8d..8e2f0280c0cc58 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -88,6 +88,8 @@ void __vgic_v3_restore_vmcr_aprs(struct vgic_v3_cpu_if *cpu_if);
int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);
/* GICv5 */
+void __vgic_v5_make_resident(struct vgic_v5_cpu_if *cpu_if);
+void __vgic_v5_make_non_resident(struct vgic_v5_cpu_if *cpu_if);
void __vgic_v5_save_apr(struct vgic_v5_cpu_if *cpu_if);
void __vgic_v5_restore_vmcr_apr(struct vgic_v5_cpu_if *cpu_if);
/* No hypercalls for the following */
@@ -95,6 +97,7 @@ void __vgic_v5_save_ppi_state(struct vgic_v5_cpu_if *cpu_if);
void __vgic_v5_restore_ppi_state(struct vgic_v5_cpu_if *cpu_if);
void __vgic_v5_save_state(struct vgic_v5_cpu_if *cpu_if);
void __vgic_v5_restore_state(struct vgic_v5_cpu_if *cpu_if);
+void __vgic_v5_vdpend(u32 intid, bool pending, u16 vm);
#ifdef __KVM_NVHE_HYPERVISOR__
void __timer_enable_traps(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 7aa08d59d49448..c840b37adb2fb3 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -1040,7 +1040,7 @@
#define GCS_CAP(x) ((((unsigned long)x) & GCS_CAP_ADDR_MASK) | \
GCS_CAP_VALID_TOKEN)
/*
- * Definitions for GICv5 instructions
+ * Definitions for GICv5 instructions for the Current Domain
*/
#define GICV5_OP_GIC_CDAFF sys_insn(1, 0, 12, 1, 3)
#define GICV5_OP_GIC_CDDI sys_insn(1, 0, 12, 2, 0)
@@ -1105,6 +1105,17 @@
#define GICV5_GICR_CDNMIA_TYPE_MASK GENMASK_ULL(31, 29)
#define GICV5_GICR_CDNMIA_ID_MASK GENMASK_ULL(23, 0)
+/*
+ * Definitions for GICv5 instructions for the Virtual Domain
+ */
+#define GICV5_OP_GIC_VDPEND sys_insn(1, 4, 12, 1, 4)
+
+/* Shift and mask definitions for GIC VDPEND */
+#define GICV5_GIC_VDPEND_PENDING_MASK BIT_ULL(63)
+#define GICV5_GIC_VDPEND_VM_MASK GENMASK_ULL(47, 32)
+#define GICV5_GIC_VDPEND_TYPE_MASK GENMASK_ULL(31, 29)
+#define GICV5_GIC_VDPEND_ID_MASK GENMASK_ULL(23, 0)
+
#define gicr_insn(insn) read_sysreg_s(GICV5_OP_GICR_##insn)
#define gic_insn(v, insn) write_sysreg_s(v, GICV5_OP_GIC_##insn)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 1c13bfa2d38aad..1b9bbeab18a4e0 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -97,6 +97,11 @@ struct kvm_regs {
#define KVM_VGIC_V3_REDIST_SIZE (2 * SZ_64K)
#define KVM_VGIC_V3_ITS_SIZE (2 * SZ_64K)
+/* Supported VGICv5 address types */
+#define KVM_VGIC_V5_ADDR_TYPE_IRS 6
+
+#define KVM_VGIC_V5_IRS_SIZE (2 * SZ_64K)
+
#define KVM_ARM_VCPU_POWER_OFF 0 /* CPU is started in OFF state */
#define KVM_ARM_VCPU_EL1_32BIT 1 /* CPU running a 32bit VM */
#define KVM_ARM_VCPU_PSCI_0_2 2 /* CPU uses PSCI v0.2 */
@@ -417,6 +422,8 @@ enum {
#define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7
#define KVM_DEV_ARM_VGIC_GRP_ITS_REGS 8
#define KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ 9
+#define KVM_DEV_ARM_VGIC_GRP_IRS_REGS 10
+#define KVM_DEV_ARM_VGIC_GRP_IST 11
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT 10
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \
(0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 59612d2f277c1d..7aaeeb84e788ec 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -24,7 +24,8 @@ kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \
vgic/vgic-mmio.o vgic/vgic-mmio-v2.o \
vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \
vgic/vgic-its.o vgic/vgic-debug.o vgic/vgic-v3-nested.o \
- vgic/vgic-v5.o
+ vgic/vgic-v5.o vgic/vgic-v5-tables.o vgic/vgic-irs-v5.o \
+ vgic-sys-reg-v5.o
kvm-$(CONFIG_HW_PERF_EVENTS) += pmu-emul.o pmu.o
kvm-$(CONFIG_ARM64_PTR_AUTH) += pauth.o
diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
index 50adfff75be82e..7c59f9b451049c 100644
--- a/arch/arm64/kvm/arm.c
+++ b/arch/arm64/kvm/arm.c
@@ -961,7 +961,7 @@ int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu)
return ret;
}
- ret = vgic_v5_finalize_ppi_state(kvm);
+ ret = vgic_v5_finalize_ppi_state(vcpu);
if (ret)
return ret;
@@ -1138,6 +1138,14 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu)
preempt_enable();
}
+ if (kvm_check_request(KVM_REQ_RELOAD_GICv5, vcpu)) {
+ /* The IRS enable bit was changed */
+ preempt_disable();
+ vgic_v5_put(vcpu);
+ vgic_v5_load(vcpu);
+ preempt_enable();
+ }
+
if (kvm_check_request(KVM_REQ_RELOAD_PMU, vcpu))
kvm_vcpu_reload_pmu(vcpu);
diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
index d3c69de698f484..8f9cc3cd422aae 100644
--- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c
+++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c
@@ -691,6 +691,20 @@ static void handle___tracing_write_event(struct kvm_cpu_context *host_ctxt)
trace_selftest(id);
}
+static void handle___vgic_v5_make_resident(struct kvm_cpu_context *host_ctxt)
+{
+ DECLARE_REG(struct vgic_v5_cpu_if *, cpu_if, host_ctxt, 1);
+
+ __vgic_v5_make_resident(kern_hyp_va(cpu_if));
+}
+
+static void handle___vgic_v5_make_non_resident(struct kvm_cpu_context *host_ctxt)
+{
+ DECLARE_REG(struct vgic_v5_cpu_if *, cpu_if, host_ctxt, 1);
+
+ __vgic_v5_make_non_resident(kern_hyp_va(cpu_if));
+}
+
static void handle___vgic_v5_save_apr(struct kvm_cpu_context *host_ctxt)
{
DECLARE_REG(struct vgic_v5_cpu_if *, cpu_if, host_ctxt, 1);
@@ -705,6 +719,15 @@ static void handle___vgic_v5_restore_vmcr_apr(struct kvm_cpu_context *host_ctxt)
__vgic_v5_restore_vmcr_apr(kern_hyp_va(cpu_if));
}
+static void handle___vgic_v5_vdpend(struct kvm_cpu_context *host_ctxt)
+{
+ DECLARE_REG(u32, intid, host_ctxt, 1);
+ DECLARE_REG(bool, pending, host_ctxt, 2);
+ DECLARE_REG(u16, vm, host_ctxt, 3);
+
+ __vgic_v5_vdpend(intid, pending, vm);
+}
+
typedef void (*hcall_t)(struct kvm_cpu_context *);
#define HANDLE_FUNC(x) [__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x
@@ -738,6 +761,9 @@ static const hcall_t host_hcall[] = {
HANDLE_FUNC(__tracing_write_event),
HANDLE_FUNC(__vgic_v3_save_aprs),
HANDLE_FUNC(__vgic_v3_restore_vmcr_aprs),
+ HANDLE_FUNC(__vgic_v5_make_resident),
+ HANDLE_FUNC(__vgic_v5_make_non_resident),
+ HANDLE_FUNC(__vgic_v5_vdpend),
HANDLE_FUNC(__vgic_v5_save_apr),
HANDLE_FUNC(__vgic_v5_restore_vmcr_apr),
diff --git a/arch/arm64/kvm/hyp/vgic-v5-sr.c b/arch/arm64/kvm/hyp/vgic-v5-sr.c
index 6d69dfe89a96cf..94746317ca79fc 100644
--- a/arch/arm64/kvm/hyp/vgic-v5-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v5-sr.c
@@ -7,6 +7,41 @@
#include <asm/kvm_hyp.h>
+void __vgic_v5_make_resident(struct vgic_v5_cpu_if *cpu_if)
+{
+ write_sysreg_s(cpu_if->vgic_contextr, SYS_ICH_CONTEXTR_EL2);
+ isb();
+
+ /* Catch any faults */
+ cpu_if->vgic_contextr = read_sysreg_s(SYS_ICH_CONTEXTR_EL2);
+ if (!!FIELD_GET(ICH_CONTEXTR_EL2_F, cpu_if->vgic_contextr))
+ return;
+
+ cpu_if->gicv5_vpe.resident = true;
+}
+
+void __vgic_v5_make_non_resident(struct vgic_v5_cpu_if *cpu_if)
+{
+ /*
+ * Clear the db_fired state to ensure that we're ready for the next
+ * doorbell when it is requested. If a doorbell firing caused us to
+ * enter the guest, then we've already consumed that state at this
+ * point, so this is safe to clear. Use WRITE_ONCE() to ensure we're not
+ * racing with the doorbell firing and setting the state true again.
+ */
+ WRITE_ONCE(cpu_if->gicv5_vpe.db_fired, false);
+
+ /*
+ * Make as non-resident before actually making non-resident. Avoids race
+ * with doorbell arriving.
+ */
+ cpu_if->gicv5_vpe.resident = false;
+ dsb(st);
+
+ write_sysreg_s(cpu_if->vgic_contextr, SYS_ICH_CONTEXTR_EL2);
+ isb();
+}
+
void __vgic_v5_save_apr(struct vgic_v5_cpu_if *cpu_if)
{
cpu_if->vgic_apr = read_sysreg_s(SYS_ICH_APR_EL2);
@@ -114,3 +149,13 @@ void __vgic_v5_restore_state(struct vgic_v5_cpu_if *cpu_if)
{
write_sysreg_s(cpu_if->vgic_icsr, SYS_ICC_ICSR_EL1);
}
+
+void __vgic_v5_vdpend(u32 intid, bool pending, u16 vm)
+{
+ u64 value;
+
+ value = intid & (GICV5_GIC_VDPEND_ID_MASK | GICV5_GIC_VDPEND_TYPE_MASK);
+ value |= FIELD_PREP(GICV5_GIC_VDPEND_PENDING_MASK, pending);
+ value |= FIELD_PREP(GICV5_GIC_VDPEND_VM_MASK, vm);
+ gic_insn(value, VDPEND);
+}
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 5d5c579d457902..ba5918413f1645 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -5827,7 +5827,7 @@ int kvm_finalize_sys_regs(struct kvm_vcpu *vcpu)
int __init kvm_sys_reg_table_init(void)
{
- const struct sys_reg_desc *gicv3_regs;
+ const struct sys_reg_desc *gicv3_regs, *gicv5_regs;
bool valid = true;
unsigned int i, sz;
int ret = 0;
@@ -5840,8 +5840,12 @@ int __init kvm_sys_reg_table_init(void)
valid &= check_sysreg_table(cp15_64_regs, ARRAY_SIZE(cp15_64_regs), false);
valid &= check_sysreg_table(sys_insn_descs, ARRAY_SIZE(sys_insn_descs), false);
+ /* The GICv3 system registers... */
gicv3_regs = vgic_v3_get_sysreg_table(&sz);
valid &= check_sysreg_table(gicv3_regs, sz, false);
+ /* ...and the GICv5 system registers. */
+ gicv5_regs = vgic_v5_get_sysreg_table(&sz);
+ valid &= check_sysreg_table(gicv5_regs, sz, false);
if (!valid)
return -EINVAL;
diff --git a/arch/arm64/kvm/vgic-sys-reg-v5.c b/arch/arm64/kvm/vgic-sys-reg-v5.c
new file mode 100644
index 00000000000000..6b255eacbf60d8
--- /dev/null
+++ b/arch/arm64/kvm/vgic-sys-reg-v5.c
@@ -0,0 +1,519 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2025, 2026 Arm Ltd.
+ */
+
+/*
+ * VGICv5 system registers handling functions for AArch64 mode
+ */
+
+#include <linux/irqchip/arm-gic-v5.h>
+
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+#include <linux/wordpart.h>
+
+#include <asm/kvm_emulate.h>
+
+#include "vgic/vgic.h"
+#include "sys_regs.h"
+
+#define ICC_PPI_PRIORITYR_PRIORITY_MASK REPEAT_BYTE(0x1f)
+
+static int set_gic_apr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,
+ u64 val)
+{
+ struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
+
+ /* The upper 32 bits are RES0 */
+ cpu_if->vgic_apr = val & ~ICC_APR_EL1_RES0;
+
+ return 0;
+}
+
+static int get_gic_apr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,
+ u64 *val)
+{
+ struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
+
+ *val = cpu_if->vgic_apr;
+
+ return 0;
+}
+
+static int set_gic_cr0(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,
+ u64 val)
+{
+ struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
+
+ /*
+ * We only support setting the ICC_CR0_EL1.En bit, which is actually
+ * stored in the VMCR.
+ */
+ FIELD_MODIFY(FEAT_GCIE_ICH_VMCR_EL2_EN, &cpu_if->vgic_vmcr,
+ FIELD_GET(ICC_CR0_EL1_EN, val));
+
+ return 0;
+}
+
+static int get_gic_cr0(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,
+ u64 *val)
+{
+ struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
+
+ /*
+ * PID only applies if EL3 is present. Same applies to IPPT. Hence,
+ * those fields are always presented as 0.
+ *
+ * We always present the link as connected and idle:
+ * (LINK = 1, LINK_IDLE = 1).
+ */
+ *val = FIELD_PREP(ICC_CR0_EL1_EN,
+ FIELD_GET(FEAT_GCIE_ICH_VMCR_EL2_EN, cpu_if->vgic_vmcr));
+ *val |= ICC_CR0_EL1_LINK_MASK;
+ *val |= ICC_CR0_EL1_LINK_IDLE_MASK;
+
+ return 0;
+}
+
+static int set_gic_pcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,
+ u64 val)
+{
+ struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
+
+ /* Set the VPMR field in the VMCR */
+ FIELD_MODIFY(FEAT_GCIE_ICH_VMCR_EL2_VPMR, &cpu_if->vgic_vmcr,
+ FIELD_GET(ICC_PCR_EL1_PRIORITY, val));
+
+ return 0;
+}
+
+static int get_gic_pcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,
+ u64 *val)
+{
+ struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
+
+ *val = FIELD_PREP(ICC_PCR_EL1_PRIORITY,
+ FIELD_GET(FEAT_GCIE_ICH_VMCR_EL2_VPMR, cpu_if->vgic_vmcr));
+
+ return 0;
+}
+
+static int set_gic_icsr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,
+ u64 val)
+{
+ struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
+
+ cpu_if->vgic_icsr = val & ~ICC_ICSR_EL1_RES0;
+
+ return 0;
+}
+
+static int get_gic_icsr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r,
+ u64 *val)
+{
+ struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
+
+ *val = cpu_if->vgic_icsr;
+
+ return 0;
+}
+
+/*
+ * Helper macro to iterate over a range of PPIs and execute some code (to either
+ * extract or set the vgic_irq state). This is used when `get`-ing the PPI
+ * ENABLER, ACTIVER, PENDR and when setting the PRIORITYR state.
+ *
+ * vcpu: Pointer to struct kvm_vcpu (to which these PPIs belong)
+ * r: The register index. 0 or 1 for all except PRIORITYR (which is 0-15)
+ * nr: The number of PPIs iterated over. 64 for all but PRIORITYR (which is 8)
+ * code: The code snippet to execute for each vgic_irq
+ */
+#define for_ppi_state(vcpu, r, nr, code) \
+ do { \
+ struct kvm_vcpu *__vcpu = (vcpu); \
+ int __r = (r); \
+ int __nr = (nr); \
+ \
+ for (int i = 0; i < __nr; i++) { \
+ u32 id = vgic_v5_make_ppi(__r * __nr + i); \
+ struct vgic_irq *irq; \
+ \
+ irq = vgic_get_vcpu_irq(__vcpu, id); \
+ scoped_guard(raw_spinlock_irqsave, &irq->irq_lock) { \
+ code; \
+ } \
+ vgic_put_irq(__vcpu->kvm, irq); \
+ } \
+ } while (0)
+
+static int set_gic_ppi_enabler(struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *r, u64 val)
+{
+ struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
+ int i, start, end, reg = r->Op2 % 2;
+
+ /*
+ * If we're only handling architected PPIs and the guest writes to the
+ * enable for the non-architected PPIs, we just return as there's
+ * nothing to do at all. We don't even allocate the storage for them in
+ * this case.
+ */
+ if (VGIC_V5_NR_PRIVATE_IRQS == 64 && reg == 1)
+ return 0;
+
+ /*
+ * Merge the raw guest write into our bitmap at an offset of either 0 or
+ * 64.
+ *
+ * Note that there is *NO* masking applied - the enable state is written
+ * unfiltered. The assumption is that userspace uses this interface to
+ * set initial state before the guest runs, and then the exposed PPI
+ * mask is applied later, when vgic_v5_finalize_ppi_state() runs on
+ * first entry to each vCPU. If userspace chooses to set the enabler
+ * state later, it is fully capable of breaking the illusion we provided
+ * to the guest by exposing register state (and PPIs) to the guest that
+ * were not initially exposed. Good luck!
+ */
+ bitmap_write(cpu_if->vgic_ppi_enabler, val, 64 * reg, 64);
+
+ /*
+ * Sync the change in enable states to the vgic_irqs for the written
+ * register slice.
+ */
+ start = reg * 64;
+ end = min(start + 64, VGIC_V5_NR_PRIVATE_IRQS);
+ for (i = start; i < end; i++) {
+ u32 intid = vgic_v5_make_ppi(i);
+ struct vgic_irq *irq;
+
+ irq = vgic_get_vcpu_irq(vcpu, intid);
+
+ scoped_guard(raw_spinlock_irqsave, &irq->irq_lock)
+ irq->enabled = test_bit(i, cpu_if->vgic_ppi_enabler);
+
+ vgic_put_irq(vcpu->kvm, irq);
+ }
+
+ return 0;
+}
+
+static int get_gic_ppi_enabler(struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *r, u64 *val)
+{
+ unsigned long enabler = 0;
+ int reg = r->Op2 % 2;
+
+ /* If we only support architected PPIs, return 0 */
+ if (VGIC_V5_NR_PRIVATE_IRQS == 64 && reg == 1) {
+ *val = 0;
+ return 0;
+ }
+
+ /* Iterate over each struct vgic_irq to build the ENABLER value. */
+ for_ppi_state(vcpu, reg, 64, __assign_bit(i % 64, &enabler, irq->enabled));
+
+ *val = enabler;
+
+ return 0;
+}
+
+static int set_gic_ppi_activer(struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *r, u64 val)
+{
+ struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
+ int i, start, end, reg = r->Op2 % 2;
+
+ if (VGIC_V5_NR_PRIVATE_IRQS == 64 && reg == 1)
+ return 0;
+
+ /*
+ * Store the raw guest write. The exposed PPI mask is applied later,
+ * when vgic_v5_finalize_ppi_state() runs on first entry to each
+ * vCPU. See comment on set_gic_ppi_enabler() for details.
+ */
+ bitmap_write(cpu_if->vgic_ppi_activer, val, 64 * reg, 64);
+
+ start = reg * 64;
+ end = min(start + 64, VGIC_V5_NR_PRIVATE_IRQS);
+ for (i = start; i < end; i++) {
+ u32 intid = vgic_v5_make_ppi(i);
+ struct vgic_irq *irq;
+
+ irq = vgic_get_vcpu_irq(vcpu, intid);
+
+ scoped_guard(raw_spinlock_irqsave, &irq->irq_lock)
+ irq->active = test_bit(i, cpu_if->vgic_ppi_activer);
+
+ vgic_put_irq(vcpu->kvm, irq);
+ }
+
+ return 0;
+}
+
+static int get_gic_ppi_activer(struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *r, u64 *val)
+{
+ unsigned long activer = 0;
+ int reg = r->Op2 % 2;
+
+ /* If we only support architected PPIs, return 0 */
+ if (VGIC_V5_NR_PRIVATE_IRQS == 64 && reg == 1) {
+ *val = 0;
+ return 0;
+ }
+
+ /* Iterate over each struct vgic_irq to build the ACTIVER value. */
+ for_ppi_state(vcpu, reg, 64, __assign_bit(i % 64, &activer, irq->active));
+
+ *val = activer;
+
+ return 0;
+}
+
+static int set_gic_ppi_pendr(struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *r, u64 val)
+{
+ int i, start, end, reg = r->Op2 % 2;
+
+ /* If we only support architected PPIs, return */
+ if (VGIC_V5_NR_PRIVATE_IRQS == 64 && reg == 1)
+ return 0;
+
+ /*
+ * Update each struct vgic_irq with the pending state, treating Level
+ * and Edge interrupts differently. The exposed PPI mask is applied
+ * later, when vgic_v5_finalize_ppi_state() runs on first entry to each
+ * vCPU. See comment on set_gic_ppi_enabler() for details.
+ */
+ start = reg * 64;
+ end = min(start + 64, VGIC_V5_NR_PRIVATE_IRQS);
+ for (i = start; i < end; i++) {
+ u32 intid = vgic_v5_make_ppi(i);
+ struct vgic_irq *irq;
+
+ irq = vgic_get_vcpu_irq(vcpu, intid);
+
+ scoped_guard(raw_spinlock_irqsave, &irq->irq_lock) {
+ bool level = !!(val & BIT_ULL(i - start));
+
+ if (irq->config == VGIC_CONFIG_LEVEL)
+ irq->line_level = level;
+ else
+ irq->pending_latch = level;
+ }
+
+ vgic_put_irq(vcpu->kvm, irq);
+ }
+
+ /*
+ * The pending state is generated from the vgic_irqs on each guest
+ * entry. Therefore, we don't store the raw value written anywhere in
+ * the case of userspace PPI_PENDRx_EL1 writes.
+ */
+
+ return 0;
+}
+
+static int get_gic_ppi_pendr(struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *r, u64 *val)
+{
+ unsigned long pendr = 0;
+ int reg = r->Op2 % 2;
+
+ /* If we only support architected PPIs, return 0 */
+ if (VGIC_V5_NR_PRIVATE_IRQS == 64 && reg == 1) {
+ *val = 0;
+ return 0;
+ }
+
+ /* Iterate over each struct vgic_irq to build the PENDR value. */
+ for_ppi_state(vcpu, reg, 64, {
+ if (irq_is_pending(irq))
+ __assign_bit(i % 64, &pendr, 1);
+ });
+
+ *val = pendr;
+
+ return 0;
+}
+
+static int set_gic_ppi_priorityr(struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *r, u64 val)
+{
+ struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
+ int reg = ((r->CRm & 0x1) << 3) + r->Op2;
+
+ /* If we only support architected PPIs, return */
+ if (VGIC_V5_NR_PRIVATE_IRQS == 64 && reg > 7)
+ return 0;
+
+ val &= ICC_PPI_PRIORITYR_PRIORITY_MASK;
+
+ /*
+ * Although priorities are not regularly synced back to the vgic_irq
+ * state, they are explicitly synced back here. This is to ensure that
+ * any pending PPIs are evaluated correctly when first running the guest
+ * after setting the state.
+ */
+ for_ppi_state(vcpu, reg, 8,
+ irq->priority = (u8)(val >> (8 * i));
+ );
+
+ /*
+ * Update the state that will be written to the ICH_PPI_PRIORITYRx_EL2
+ * on next guest entry.
+ */
+ cpu_if->vgic_ppi_priorityr[reg] = val;
+
+ return 0;
+}
+
+static int get_gic_ppi_priorityr(struct kvm_vcpu *vcpu,
+ const struct sys_reg_desc *r, u64 *val)
+{
+ struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
+ int reg = ((r->CRm & 0x1) << 3) + r->Op2;
+
+ /* If we only support architected PPIs, return 0 */
+ if (VGIC_V5_NR_PRIVATE_IRQS == 64 && reg > 7) {
+ *val = 0;
+ return 0;
+ }
+
+ /*
+ * The priorities are only synced back to the vgic_irq state when the
+ * vcpu is entering WFI (KVM only needs to know the priorities when
+ * evaluating if there are pending PPI interrupts for a vcpu). The raw
+ * register ICH_PPI_PRIORITYRx_EL1 state is simply saved and restored
+ * blindly. This state is just returned as it contains the most recent
+ * priorities written by the guest.
+ */
+ *val = cpu_if->vgic_ppi_priorityr[reg];
+
+ return 0;
+}
+
+/*
+ * The following registers are NOT supported:
+ *
+ * - ICC_HAPR_EL1
+ * The value of this is directly generated by the GICv5 hardware based on
+ * the ICC_APR_EL1 when the guest is running.
+ * - ICC_IAFFIDR_EL1
+ * The IAFFID for a GICv5 VPE is the same as the VPE ID, which is the index
+ * into the in-memory VPE Table. This is not configurable, and instead we
+ * rely on userspace recreating the VPEs in the same order prior to
+ * restoring guest state.
+ * - ICC_PPI_CACTIVER<n>_EL1
+ * Only raw state writes are supported via the S(et) variant.
+ * - ICC_PPI_CPENDR<n>_EL1
+ * Only raw state writes are supported via the S(et) variant.
+ */
+static const struct sys_reg_desc gic_v5_icc_reg_descs[] = {
+ { SYS_DESC(SYS_ICC_ICSR_EL1),
+ .set_user = set_gic_icsr, .get_user = get_gic_icsr, },
+ { SYS_DESC(SYS_ICC_PPI_ENABLER0_EL1),
+ .set_user = set_gic_ppi_enabler, .get_user = get_gic_ppi_enabler, },
+ { SYS_DESC(SYS_ICC_PPI_ENABLER1_EL1),
+ .set_user = set_gic_ppi_enabler, .get_user = get_gic_ppi_enabler, },
+ /*
+ * Only ICC_SACTIVER<n>_EL1 is exposed to the guest. This is treated as
+ * a *RAW* write of register state for writes.
+ */
+ { SYS_DESC(SYS_ICC_PPI_SACTIVER0_EL1),
+ .set_user = set_gic_ppi_activer, .get_user = get_gic_ppi_activer, },
+ { SYS_DESC(SYS_ICC_PPI_SACTIVER1_EL1),
+ .set_user = set_gic_ppi_activer, .get_user = get_gic_ppi_activer, },
+ /*
+ * Only ICC_SPENDR<n>_EL1 is exposed to the guest. This is treated as
+ * a *RAW* write of register state for writes.
+ */
+ { SYS_DESC(SYS_ICC_PPI_SPENDR0_EL1),
+ .set_user = set_gic_ppi_pendr, .get_user = get_gic_ppi_pendr, },
+ { SYS_DESC(SYS_ICC_PPI_SPENDR1_EL1),
+ .set_user = set_gic_ppi_pendr, .get_user = get_gic_ppi_pendr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR0_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR1_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR2_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR3_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR4_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR5_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR6_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR7_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR8_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR9_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR10_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR11_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR12_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR13_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR14_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_PPI_PRIORITYR15_EL1),
+ .set_user = set_gic_ppi_priorityr, .get_user = get_gic_ppi_priorityr, },
+ { SYS_DESC(SYS_ICC_APR_EL1),
+ .set_user = set_gic_apr, .get_user = get_gic_apr, },
+ { SYS_DESC(SYS_ICC_CR0_EL1),
+ .set_user = set_gic_cr0, .get_user = get_gic_cr0, },
+ { SYS_DESC(SYS_ICC_PCR_EL1),
+ .set_user = set_gic_pcr, .get_user = get_gic_pcr, },
+};
+
+const struct sys_reg_desc *vgic_v5_get_sysreg_table(unsigned int *sz)
+{
+ *sz = ARRAY_SIZE(gic_v5_icc_reg_descs);
+ return gic_v5_icc_reg_descs;
+}
+
+static u64 attr_to_id(u64 attr)
+{
+ return ARM64_SYS_REG(FIELD_GET(KVM_REG_ARM_VGIC_SYSREG_OP0_MASK, attr),
+ FIELD_GET(KVM_REG_ARM_VGIC_SYSREG_OP1_MASK, attr),
+ FIELD_GET(KVM_REG_ARM_VGIC_SYSREG_CRN_MASK, attr),
+ FIELD_GET(KVM_REG_ARM_VGIC_SYSREG_CRM_MASK, attr),
+ FIELD_GET(KVM_REG_ARM_VGIC_SYSREG_OP2_MASK, attr));
+}
+
+int vgic_v5_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)
+{
+ const struct sys_reg_desc *r;
+
+ r = get_reg_by_id(attr_to_id(attr->attr), gic_v5_icc_reg_descs,
+ ARRAY_SIZE(gic_v5_icc_reg_descs));
+
+ if (r && !sysreg_hidden(vcpu, r))
+ return 0;
+
+ return -ENXIO;
+}
+
+int vgic_v5_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr,
+ bool is_write)
+{
+ struct kvm_one_reg reg = {
+ .id = attr_to_id(attr->attr),
+ .addr = attr->addr,
+ };
+
+ if (is_write)
+ return kvm_sys_reg_set_user(vcpu, ®, gic_v5_icc_reg_descs,
+ ARRAY_SIZE(gic_v5_icc_reg_descs));
+ else
+ return kvm_sys_reg_get_user(vcpu, ®, gic_v5_icc_reg_descs,
+ ARRAY_SIZE(gic_v5_icc_reg_descs));
+}
diff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-init.c
index 907057881b26aa..8a38d915287742 100644
--- a/arch/arm64/kvm/vgic/vgic-init.c
+++ b/arch/arm64/kvm/vgic/vgic-init.c
@@ -129,13 +129,17 @@ int kvm_vgic_create(struct kvm *kvm, u32 type)
}
ret = 0;
- if (type == KVM_DEV_TYPE_ARM_VGIC_V2)
+ switch (type) {
+ case KVM_DEV_TYPE_ARM_VGIC_V2:
kvm->max_vcpus = VGIC_V2_MAX_CPUS;
- else if (type == KVM_DEV_TYPE_ARM_VGIC_V3)
+ break;
+ case KVM_DEV_TYPE_ARM_VGIC_V3:
kvm->max_vcpus = VGIC_V3_MAX_CPUS;
- else if (type == KVM_DEV_TYPE_ARM_VGIC_V5)
- kvm->max_vcpus = min(VGIC_V5_MAX_CPUS,
- kvm_vgic_global_state.max_gic_vcpus);
+ break;
+ case KVM_DEV_TYPE_ARM_VGIC_V5:
+ kvm->max_vcpus = kvm_vgic_global_state.max_gicv5_vcpus;
+ break;
+ }
if (atomic_read(&kvm->online_vcpus) > kvm->max_vcpus) {
ret = -E2BIG;
@@ -154,6 +158,8 @@ int kvm_vgic_create(struct kvm *kvm, u32 type)
case KVM_DEV_TYPE_ARM_VGIC_V3:
INIT_LIST_HEAD(&kvm->arch.vgic.rd_regions);
break;
+ case KVM_DEV_TYPE_ARM_VGIC_V5:
+ kvm->arch.vgic.gicv5_vm.vm_id = VGIC_V5_VM_ID_INVAL;
}
/*
@@ -168,28 +174,48 @@ int kvm_vgic_create(struct kvm *kvm, u32 type)
break;
}
- if (ret) {
- kvm_for_each_vcpu(i, vcpu, kvm) {
- struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
- kfree(vgic_cpu->private_irqs);
- vgic_cpu->private_irqs = NULL;
- }
-
- kvm->arch.vgic.vgic_model = 0;
- goto out_unlock;
- }
+ if (ret)
+ goto out_free_private_irqs;
if (type == KVM_DEV_TYPE_ARM_VGIC_V3)
kvm->arch.vgic.nassgicap = system_supports_direct_sgis();
- /*
- * We now know that we have a GICv5. The Arch Timer PPI interrupts may
- * have been initialised at this stage, but will have done so assuming
- * that we have an older GIC, meaning that the IntIDs won't be
- * correct. We init them again, and this time they will be correct.
- */
- if (type == KVM_DEV_TYPE_ARM_VGIC_V5)
+ if (type == KVM_DEV_TYPE_ARM_VGIC_V5) {
+ /* Allocate a vIRS for GICv5 systems */
+ kvm->arch.vgic.vgic_v5_irs_data = kzalloc_obj(struct vgic_v5_irs,
+ GFP_KERNEL_ACCOUNT);
+ if (!kvm->arch.vgic.vgic_v5_irs_data) {
+ ret = -ENOMEM;
+ goto out_free_private_irqs;
+ }
+
+ /*
+ * Initialization happens later, for now just explicitly
+ * disable the device and undef its base address.
+ */
+ kvm->arch.vgic.vgic_v5_irs_data->vgic_v5_irs_base = VGIC_ADDR_UNDEF;
+
+ /*
+ * We now know that we have a GICv5. The Arch Timer PPI
+ * interrupts may have been initialised at this stage, but will
+ * have done so assuming that we have an older GIC, meaning that
+ * the IntIDs won't be correct. We init them again, and this
+ * time they will be correct.
+ */
kvm_timer_init_vm(kvm);
+ }
+
+ goto out_unlock;
+
+out_free_private_irqs:
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+
+ kfree(vgic_cpu->private_irqs);
+ vgic_cpu->private_irqs = NULL;
+ }
+
+ kvm->arch.vgic.vgic_model = 0;
out_unlock:
mutex_unlock(&kvm->arch.config_lock);
@@ -461,6 +487,9 @@ int vgic_init(struct kvm *kvm)
return ret;
}
} else {
+ if (!dist->nr_spis)
+ dist->nr_spis = VGIC_V5_DEFAULT_NR_SPIS;
+
ret = vgic_v5_init(kvm);
if (ret)
return ret;
@@ -492,16 +521,24 @@ static void kvm_vgic_dist_destroy(struct kvm *kvm)
dist->nr_spis = 0;
dist->vgic_dist_base = VGIC_ADDR_UNDEF;
- if (dist->vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) {
+ switch (dist->vgic_model) {
+ case KVM_DEV_TYPE_ARM_VGIC_V2:
+ dist->vgic_cpu_base = VGIC_ADDR_UNDEF;
+ break;
+ case KVM_DEV_TYPE_ARM_VGIC_V3:
list_for_each_entry_safe(rdreg, next, &dist->rd_regions, list)
vgic_v3_free_redist_region(kvm, rdreg);
INIT_LIST_HEAD(&dist->rd_regions);
- } else {
- dist->vgic_cpu_base = VGIC_ADDR_UNDEF;
- }
- if (vgic_supports_direct_irqs(kvm))
- vgic_v4_teardown(kvm);
+ if (vgic_supports_direct_irqs(kvm))
+ vgic_v4_teardown(kvm);
+ break;
+ case KVM_DEV_TYPE_ARM_VGIC_V5:
+ vgic_v5_teardown(kvm);
+ kfree(dist->vgic_v5_irs_data);
+ dist->vgic_v5_irs_data = NULL;
+ break;
+ }
xa_destroy(&dist->lpi_xa);
}
@@ -620,9 +657,8 @@ int vgic_lazy_init(struct kvm *kvm)
int kvm_vgic_map_resources(struct kvm *kvm)
{
struct vgic_dist *dist = &kvm->arch.vgic;
- bool needs_dist = true;
enum vgic_type type;
- gpa_t dist_base;
+ gpa_t dist_base, irs_base;
int ret = 0;
if (likely(smp_load_acquire(&dist->ready)))
@@ -645,13 +681,12 @@ int kvm_vgic_map_resources(struct kvm *kvm)
} else {
ret = vgic_v5_map_resources(kvm);
type = VGIC_V5;
- needs_dist = false;
}
if (ret)
goto out;
- if (needs_dist) {
+ if (type != VGIC_V5) {
dist_base = dist->vgic_dist_base;
mutex_unlock(&kvm->arch.config_lock);
@@ -661,7 +696,19 @@ int kvm_vgic_map_resources(struct kvm *kvm)
goto out_slots;
}
} else {
+ irs_base = dist->vgic_v5_irs_data->vgic_v5_irs_base;
mutex_unlock(&kvm->arch.config_lock);
+
+ if (IS_VGIC_ADDR_UNDEF(irs_base)) {
+ ret = -ENXIO;
+ goto out_slots;
+ }
+
+ ret = vgic_v5_register_irs_iodev(kvm, irs_base);
+ if (ret) {
+ kvm_err("Unable to register VGIC IRS MMIO regions\n");
+ goto out_slots;
+ }
}
smp_store_release(&dist->ready, true);
diff --git a/arch/arm64/kvm/vgic/vgic-irqfd.c b/arch/arm64/kvm/vgic/vgic-irqfd.c
index 19a1094536e6a8..83e84151d7aa26 100644
--- a/arch/arm64/kvm/vgic/vgic-irqfd.c
+++ b/arch/arm64/kvm/vgic/vgic-irqfd.c
@@ -19,9 +19,14 @@ static int vgic_irqfd_set_irq(struct kvm_kernel_irq_routing_entry *e,
struct kvm *kvm, int irq_source_id,
int level, bool line_status)
{
- unsigned int spi_id = e->irqchip.pin + VGIC_NR_PRIVATE_IRQS;
+ unsigned int spi_id;
int ret;
+ if (kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V5)
+ spi_id = vgic_v5_make_spi(e->irqchip.pin);
+ else
+ spi_id = e->irqchip.pin + VGIC_NR_PRIVATE_IRQS;
+
if (!vgic_valid_spi(kvm, spi_id))
return -EINVAL;
@@ -52,8 +57,8 @@ int kvm_set_routing_entry(struct kvm *kvm,
e->set = vgic_irqfd_set_irq;
e->irqchip.irqchip = ue->u.irqchip.irqchip;
e->irqchip.pin = ue->u.irqchip.pin;
- if ((e->irqchip.pin >= KVM_IRQCHIP_NUM_PINS) ||
- (e->irqchip.irqchip >= KVM_NR_IRQCHIPS))
+ if (e->irqchip.pin >= KVM_IRQCHIP_NUM_PINS ||
+ e->irqchip.irqchip >= KVM_NR_IRQCHIPS)
goto out;
break;
case KVM_IRQ_ROUTING_MSI:
diff --git a/arch/arm64/kvm/vgic/vgic-irs-v5.c b/arch/arm64/kvm/vgic/vgic-irs-v5.c
new file mode 100644
index 00000000000000..be580d9a00cd58
--- /dev/null
+++ b/arch/arm64/kvm/vgic/vgic-irs-v5.c
@@ -0,0 +1,1136 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2025 ARM Limited, All Rights Reserved.
+ */
+#include <linux/bitops.h>
+#include <linux/bsearch.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+#include <kvm/iodev.h>
+#include <kvm/arm_arch_timer.h>
+#include <kvm/arm_vgic.h>
+
+#include "vgic.h"
+#include "vgic-mmio.h"
+#include "vgic-v5-tables.h"
+
+#define irs_caps kvm_vgic_global_state.vgic_v5_irs_caps
+
+static struct vgic_dist *vgic_v5_get_vgic(struct kvm_vcpu *vcpu)
+{
+ return &vcpu->kvm->arch.vgic;
+}
+
+static struct vgic_v5_irs *vgic_v5_get_irs(struct kvm_vcpu *vcpu)
+{
+ return vcpu->kvm->arch.vgic.vgic_v5_irs_data;
+}
+
+static unsigned long vgic_v5_mmio_read_irs_misc(struct kvm_vcpu *vcpu,
+ gpa_t addr, unsigned int len)
+{
+ struct vgic_v5_irs *irs = vgic_v5_get_irs(vcpu);
+ const size_t offset = addr & (SZ_64K - 1);
+ struct kvm_vcpu *target_vcpu;
+ u8 vpe_id_bits;
+ u64 value = 0;
+
+ switch (offset) {
+ case GICV5_IRS_IDR0:
+ value = FIELD_PREP(GICV5_IRS_IDR0_INT_DOM, irs->idr0.domain);
+ value |= FIELD_PREP(GICV5_IRS_IDR0_PA_RANGE, irs->idr0.pa_range);
+ if (irs->idr0.virt)
+ value |= GICV5_IRS_IDR0_VIRT;
+ if (irs->idr0.setlpi)
+ value |= GICV5_IRS_IDR0_SETLPI;
+ if (irs->idr0.mec)
+ value |= GICV5_IRS_IDR0_MEC;
+ if (irs->idr0.mpam)
+ value |= GICV5_IRS_IDR0_MPAM;
+ if (irs->idr0.swe)
+ value |= GICV5_IRS_IDR0_SWE;
+ value |= FIELD_PREP(GICV5_IRS_IDR0_IRSID, irs->idr0.irs_id);
+ break;
+ case GICV5_IRS_IDR1:
+ value = FIELD_PREP(GICV5_IRS_IDR1_PE_CNT,
+ atomic_read(&vcpu->kvm->online_vcpus));
+ /*
+ * IRS_IDR1 encodes IAFFID_BITS as N - 1.
+ */
+ vpe_id_bits = vgic_v5_vmte_vpe_id_bits(vcpu);
+ value |= FIELD_PREP(GICV5_IRS_IDR1_IAFFID_BITS, vpe_id_bits - 1);
+ value |= FIELD_PREP(GICV5_IRS_IDR1_PRIORITY_BITS, irs->idr1.priority_bits);
+ break;
+ case GICV5_IRS_IDR2:
+ value = FIELD_PREP(GICV5_IRS_IDR2_ISTMD_SZ, irs->idr2.istmd_sz);
+ if (irs->idr2.istmd)
+ value |= GICV5_IRS_IDR2_ISTMD;
+ value |= FIELD_PREP(GICV5_IRS_IDR2_IST_L2SZ, irs->idr2.ist_l2sz);
+ if (irs->idr2.ist_levels)
+ value |= GICV5_IRS_IDR2_IST_LEVELS;
+ value |= FIELD_PREP(GICV5_IRS_IDR2_MIN_LPI_ID_BITS, irs->idr2.min_lpi_id_bits);
+ value |= GICV5_IRS_IDR2_LPI;
+ value |= FIELD_PREP(GICV5_IRS_IDR2_ID_BITS, irs->idr2.id_bits);
+ break;
+ case GICV5_IRS_IDR5:
+ value = FIELD_PREP(GICV5_IRS_IDR5_SPI_RANGE, irs->idr5.spi_range);
+ break;
+ case GICV5_IRS_IDR6:
+ value = FIELD_PREP(GICV5_IRS_IDR6_SPI_IRS_RANGE, irs->idr6.spi_irs_range);
+ break;
+ case GICV5_IRS_IDR7:
+ value = FIELD_PREP(GICV5_IRS_IDR7_SPI_BASE, irs->idr7.spi_base);
+ break;
+ case GICV5_IRS_IIDR:
+ /* Revision, Variant, ProductID are implementation defined */
+ value = FIELD_PREP(GICV5_IRS_IIDR_PRODUCT_ID, PRODUCT_ID_KVM);
+ value |= FIELD_PREP(GICV5_IRS_IIDR_VARIANT, 0);
+ value |= FIELD_PREP(GICV5_IRS_IIDR_REVISION, 0);
+ value |= FIELD_PREP(GICV5_IRS_IIDR_IMPLEMENTER, IMPLEMENTER_ARM);
+ break;
+ case GICV5_IRS_AIDR:
+ value = FIELD_PREP(GICV5_IRS_AIDR_COMPONENT,
+ GICV5_AIDR_COMPONENT_IRS);
+ value |= FIELD_PREP(GICV5_IRS_AIDR_ARCHMAJORREV,
+ GICV5_AIDR_ARCH_MAJ_REV_V5);
+ value |= FIELD_PREP(GICV5_IRS_AIDR_ARCHMINORREV,
+ GICV5_AIDR_ARCH_MIN_REV_V0);
+ break;
+ case GICV5_IRS_CR0:
+ /*
+ * The IRS is ALWAYS idle as we handle things instantaneously
+ * from a guest's viewpoint.
+ */
+ value = GICV5_IRS_CR0_IDLE;
+ if (READ_ONCE(vcpu->kvm->arch.vgic.enabled))
+ value |= GICV5_IRS_CR0_IRSEN;
+ break;
+ case GICV5_IRS_CR1:
+ if (irs->cr1.vped_wa)
+ value |= GICV5_IRS_CR1_VPED_WA;
+ if (irs->cr1.vped_ra)
+ value |= GICV5_IRS_CR1_VPED_RA;
+ if (irs->cr1.vmd_wa)
+ value |= GICV5_IRS_CR1_VMD_WA;
+ if (irs->cr1.vmd_ra)
+ value |= GICV5_IRS_CR1_VMD_RA;
+ if (irs->cr1.vpet_ra)
+ value |= GICV5_IRS_CR1_VPET_RA;
+ if (irs->cr1.vmt_ra)
+ value |= GICV5_IRS_CR1_VMT_RA;
+ if (irs->cr1.ist_wa)
+ value |= GICV5_IRS_CR1_IST_WA;
+ if (irs->cr1.ist_ra)
+ value |= GICV5_IRS_CR1_IST_RA;
+ value |= FIELD_PREP(GICV5_IRS_CR1_IC, irs->cr1.ic);
+ value |= FIELD_PREP(GICV5_IRS_CR1_OC, irs->cr1.oc);
+ value |= FIELD_PREP(GICV5_IRS_CR1_SH, irs->cr1.sh);
+ break;
+ case GICV5_IRS_SYNC_STATUSR:
+ value = GICV5_IRS_SYNC_STATUSR_IDLE;
+ break;
+ case GICV5_IRS_PE_SELR:
+ value = FIELD_PREP(GICV5_IRS_PE_SELR_IAFFID, irs->pe_selr.iaffid);
+ break;
+ case GICV5_IRS_PE_STATUSR:
+ /* We assume that the PE is Online if present. Always IDLE too */
+ value = GICV5_IRS_PE_STATUSR_IDLE;
+
+ /* Set ONLINE and V if IAFFID selects a present PE */
+ if (kvm_get_vcpu_by_id(vcpu->kvm, irs->pe_selr.iaffid)) {
+ value |= GICV5_IRS_PE_STATUSR_ONLINE;
+ value |= GICV5_IRS_PE_STATUSR_V;
+ }
+ break;
+ case GICV5_IRS_PE_CR0:
+ /*
+ * Make sure that we are doing something reasonable first.
+ * Remember, the IAFFID is the same as the VPE_ID
+ */
+ target_vcpu = kvm_get_vcpu_by_id(vcpu->kvm, irs->pe_selr.iaffid);
+ if (!target_vcpu) {
+ kvm_debug("Guest programmed invalid IAFFID (0x%x) into the IRS_PE_SELR\n",
+ irs->pe_selr.iaffid);
+ break;
+ }
+
+ value = GICV5_IRS_PE_CR0_DPS;
+ break;
+ default:
+ return 0;
+ }
+
+ return value;
+}
+
+static void vgic_v5_mmio_write_irs_misc(struct kvm_vcpu *vcpu, gpa_t addr,
+ unsigned int len, unsigned long val)
+{
+ struct vgic_v5_irs *irs = vgic_v5_get_irs(vcpu);
+ struct vgic_dist *vgic = vgic_v5_get_vgic(vcpu);
+ const size_t offset = addr & (SZ_64K - 1);
+
+ switch (offset) {
+ case GICV5_IRS_CR0:
+ mutex_lock(&vcpu->kvm->arch.config_lock);
+ WRITE_ONCE(vgic->enabled, !!(val & GICV5_IRS_CR0_IRSEN));
+ kvm_make_all_cpus_request(vcpu->kvm, KVM_REQ_RELOAD_GICv5);
+ mutex_unlock(&vcpu->kvm->arch.config_lock);
+
+ return;
+ case GICV5_IRS_CR1:
+ irs->cr1.sh = FIELD_GET(GICV5_IRS_CR1_SH, val);
+ irs->cr1.oc = FIELD_GET(GICV5_IRS_CR1_OC, val);
+ irs->cr1.ic = FIELD_GET(GICV5_IRS_CR1_IC, val);
+ irs->cr1.ist_ra = !!(val & GICV5_IRS_CR1_IST_RA);
+ irs->cr1.ist_wa = !!(val & GICV5_IRS_CR1_IST_WA);
+ irs->cr1.vmt_ra = !!(val & GICV5_IRS_CR1_VMT_RA);
+ irs->cr1.vpet_ra = !!(val & GICV5_IRS_CR1_VPET_RA);
+ irs->cr1.vmd_ra = !!(val & GICV5_IRS_CR1_VMD_RA);
+ irs->cr1.vmd_wa = !!(val & GICV5_IRS_CR1_VMD_WA);
+ irs->cr1.vped_ra = !!(val & GICV5_IRS_CR1_VPED_RA);
+ irs->cr1.vped_wa = !!(val & GICV5_IRS_CR1_VPED_WA);
+ return;
+ case GICV5_IRS_PE_SELR:
+ irs->pe_selr.iaffid = FIELD_GET(GICV5_IRS_PE_SELR_IAFFID, val);
+ return;
+ case GICV5_IRS_PE_CR0:
+ /*
+ * We actually have nothing to do here as we don't support
+ * 1-of-N routing. The only thing that the guest can correctly
+ * write here is 0x1. However, there's no way to fault if it
+ * writes something else. This is effectively a WI in our case,
+ * but we keep it here for the purposes of documenting it.
+ */
+ return;
+ default:
+ return;
+ }
+}
+
+static bool vgic_v5_is_spi_selr_valid(struct vgic_v5_irs *irs, u32 id)
+{
+ /* Invalid - we don't have any SPIs at all */
+ if (irs->idr5.spi_range == 0)
+ return false;
+
+ /* Invalid - we don't have any on this IRS */
+ if (irs->idr6.spi_irs_range == 0)
+ return false;
+
+ /* Invalid - ID is less than min */
+ if (id < irs->idr7.spi_base)
+ return false;
+
+ /* Invalid - ID is greater than max */
+ if (id >= (irs->idr7.spi_base + irs->idr6.spi_irs_range))
+ return false;
+
+ return true;
+}
+
+static unsigned long vgic_v5_mmio_read_irs_spi(struct kvm_vcpu *vcpu,
+ gpa_t addr, unsigned int len)
+{
+ struct vgic_v5_irs *irs = vgic_v5_get_irs(vcpu);
+ const size_t offset = addr & (SZ_64K - 1);
+ struct vgic_irq *irq;
+ u64 value = 0;
+
+ switch (offset) {
+ case GICV5_IRS_SPI_SELR:
+ /* Return whatever was last written */
+ value = FIELD_PREP(GICV5_IRS_SPI_SELR_ID, READ_ONCE(irs->spi_selr.id));
+ break;
+ case GICV5_IRS_SPI_STATUSR:
+ /* We assume that we can always claim to be idle */
+ value = GICV5_IRS_SPI_STATUSR_IDLE;
+ if (vgic_v5_is_spi_selr_valid(irs, READ_ONCE(irs->spi_selr.id)))
+ value |= GICV5_IRS_SPI_STATUSR_V;
+ break;
+ case GICV5_IRS_SPI_DOMAINR:
+ value = FIELD_PREP(GICV5_IRS_SPI_DOMAINR_DOMAIN,
+ GICV5_IRS_SPI_DOMAINR_DOMAIN_NON_SECURE);
+ break;
+ case GICV5_IRS_SPI_CFGR: {
+ u32 id = READ_ONCE(irs->spi_selr.id);
+
+ if (!vgic_v5_is_spi_selr_valid(irs, id)) {
+ /* Fault with IRS_SPI_SELR; return 0*/
+ value = 0;
+ break;
+ }
+
+ irq = vgic_get_irq(vcpu->kvm, vgic_v5_make_spi(id));
+ if (!irq) {
+ kvm_debug_ratelimited("Guest trying to access SPI not backed by KVM\n");
+ value = 0;
+ break;
+ }
+
+ scoped_guard(raw_spinlock_irqsave, &irq->irq_lock) {
+ if (irq->config == VGIC_CONFIG_LEVEL)
+ value = GICV5_IRS_SPI_CFGR_TM;
+ }
+
+ vgic_put_irq(vcpu->kvm, irq);
+
+ break;
+ }
+ default:
+ return 0;
+ }
+
+ return value;
+}
+
+static void vgic_v5_mmio_write_irs_spi(struct kvm_vcpu *vcpu, gpa_t addr,
+ unsigned int len, unsigned long val)
+{
+ struct vgic_v5_irs *irs = vgic_v5_get_irs(vcpu);
+ const size_t offset = addr & (SZ_64K - 1);
+ struct vgic_irq *irq;
+
+ switch (offset) {
+ case GICV5_IRS_SPI_SELR:
+ WRITE_ONCE(irs->spi_selr.id,
+ FIELD_GET(GICV5_IRS_SPI_SELR_ID, val));
+ return;
+ case GICV5_IRS_SPI_CFGR: {
+ u32 id = READ_ONCE(irs->spi_selr.id);
+
+ if (!vgic_v5_is_spi_selr_valid(irs, id))
+ return;
+
+ /*
+ * Find KVM's representation of the interrupt - we need to make
+ * sure that KVM's view agrees with the guest's, else interrupt
+ * injection won't work properly for level-triggered interrupts
+ * (we fail to handle the clearing of the pending state if KVM
+ * thinks that the interrupt is edge-triggered, which is the
+ * default.)
+ */
+ irq = vgic_get_irq(vcpu->kvm, vgic_v5_make_spi(id));
+ if (!irq)
+ return;
+
+ scoped_guard(raw_spinlock_irqsave, &irq->irq_lock) {
+ if (val & GICV5_IRS_SPI_CFGR_TM)
+ irq->config = VGIC_CONFIG_LEVEL;
+ else
+ irq->config = VGIC_CONFIG_EDGE;
+ }
+
+ vgic_put_irq(vcpu->kvm, irq);
+
+ return;
+ }
+ default:
+ return;
+ }
+}
+
+static bool vgic_v5_ist_cfgr_valid(struct vgic_v5_irs *irs)
+{
+ unsigned int expected_istsz;
+
+ if (irs->ist_cfgr.lpi_id_bits < irs->idr2.min_lpi_id_bits ||
+ irs->ist_cfgr.lpi_id_bits > irs->idr2.id_bits)
+ return false;
+
+ if (!irs->idr2.istmd)
+ expected_istsz = GICV5_IRS_IST_CFGR_ISTSZ_4;
+ else if (irs->ist_cfgr.lpi_id_bits >= irs->idr2.istmd_sz)
+ expected_istsz = GICV5_IRS_IST_CFGR_ISTSZ_16;
+ else
+ expected_istsz = GICV5_IRS_IST_CFGR_ISTSZ_8;
+
+ if (irs->ist_cfgr.istsz != expected_istsz)
+ return false;
+
+ if (irs->ist_cfgr.structure && !irs->idr2.ist_levels)
+ return false;
+
+ if (!irs->ist_cfgr.structure)
+ return true;
+
+ return irs->ist_cfgr.l2sz == irs->idr2.ist_l2sz;
+}
+
+static unsigned long vgic_v5_mmio_read_irs_ist(struct kvm_vcpu *vcpu,
+ gpa_t addr, unsigned int len)
+{
+ struct vgic_v5_irs *irs = vgic_v5_get_irs(vcpu);
+ const size_t offset = addr & (SZ_64K - 1);
+ u64 value = 0;
+
+ switch (offset) {
+ case GICV5_IRS_IST_STATUSR:
+ return GICV5_IRS_IST_STATUSR_IDLE;
+ case GICV5_IRS_IST_CFGR:
+ if (irs->ist_cfgr.structure)
+ value |= GICV5_IRS_IST_CFGR_STRUCTURE;
+ value |= FIELD_PREP(GICV5_IRS_IST_CFGR_ISTSZ, irs->ist_cfgr.istsz);
+ value |= FIELD_PREP(GICV5_IRS_IST_CFGR_L2SZ, irs->ist_cfgr.l2sz);
+ value |= FIELD_PREP(GICV5_IRS_IST_CFGR_LPI_ID_BITS, irs->ist_cfgr.lpi_id_bits);
+ break;
+ case GICV5_IRS_IST_BASER:
+ value = FIELD_PREP(GICV5_IRS_IST_BASER_ADDR_MASK,
+ irs->ist_baser.addr >> GICV5_IRS_IST_BASER_ADDR_SHIFT);
+ if (irs->ist_baser.valid)
+ value |= GICV5_IRS_IST_BASER_VALID;
+ break;
+ default:
+ return 0;
+ }
+
+ return value;
+}
+
+static void vgic_v5_update_irs_ist_baser(struct vgic_v5_irs *irs,
+ unsigned long val)
+{
+ irs->ist_baser.valid = !!(val & GICV5_IRS_IST_BASER_VALID);
+ irs->ist_baser.addr = FIELD_GET(GICV5_IRS_IST_BASER_ADDR_MASK, val)
+ << GICV5_IRS_IST_BASER_ADDR_SHIFT;
+}
+
+static int vgic_v5_write_irs_ist_baser(struct kvm_vcpu *vcpu, unsigned long val)
+{
+ struct vgic_v5_irs *irs = vgic_v5_get_irs(vcpu);
+ enum gicv5_vcpu_cmd cmd = LPI_VIST_MAKE_INVALID;
+ bool valid = !!(val & GICV5_IRS_IST_BASER_VALID);
+ int rc;
+
+ /* Valid -> Invalid */
+ if (irs->ist_baser.valid && !valid) {
+ /* Make the LPI IST invalid and then ... */
+ rc = irq_set_vcpu_affinity(vgic_v5_vpe_db(vcpu), &cmd);
+ if (rc)
+ return rc;
+
+ vgic_v5_update_irs_ist_baser(irs, val);
+
+ /*
+ * ... reflect that in the emulated BASER before freeing the
+ * host IST. If the free fails, the guest-visible valid bit
+ * still matches the hardware state.
+ */
+ return vgic_v5_lpi_ist_free(vcpu->kvm);
+ } else if (!irs->ist_baser.valid && valid) { /* Invalid -> Valid */
+ if (!vgic_v5_ist_cfgr_valid(irs)) {
+ pr_debug_ratelimited("Guest programmed invalid IRS_IST_CFGR\n");
+ return -EINVAL;
+ }
+
+ rc = vgic_v5_lpi_ist_alloc(vcpu->kvm, irs->ist_cfgr.lpi_id_bits);
+ if (rc)
+ return rc;
+ }
+
+ /* Now that we've handled the edges, update the valid bit and addr */
+ vgic_v5_update_irs_ist_baser(irs, val);
+
+ return 0;
+}
+
+static void vgic_v5_mmio_write_irs_ist(struct kvm_vcpu *vcpu, gpa_t addr,
+ unsigned int len, unsigned long val)
+{
+ struct vgic_v5_irs *irs = vgic_v5_get_irs(vcpu);
+ const size_t offset = addr & (SZ_64K - 1);
+
+ switch (offset) {
+ case GICV5_IRS_IST_CFGR:
+ scoped_guard(mutex, &vcpu->kvm->arch.config_lock) {
+ if (irs->ist_baser.valid)
+ return;
+
+ irs->ist_cfgr.lpi_id_bits = FIELD_GET(GICV5_IRS_IST_CFGR_LPI_ID_BITS, val);
+ irs->ist_cfgr.l2sz = FIELD_GET(GICV5_IRS_IST_CFGR_L2SZ, val);
+ irs->ist_cfgr.istsz = FIELD_GET(GICV5_IRS_IST_CFGR_ISTSZ, val);
+ irs->ist_cfgr.structure = !!(val & GICV5_IRS_IST_CFGR_STRUCTURE);
+ }
+ return;
+ case GICV5_IRS_IST_BASER: {
+ guard(mutex)(&vcpu->kvm->arch.config_lock);
+ vgic_v5_write_irs_ist_baser(vcpu, val);
+ return;
+ }
+ default:
+ return;
+ }
+}
+
+static unsigned long vgic_v5_coresight_read(struct kvm_vcpu *vcpu,
+ gpa_t addr, unsigned int len)
+{
+ const size_t offset = addr & (SZ_64K - 1);
+
+ switch (offset) {
+ case GICV5_CORESIGHT_DEVARCH:
+ return GICV5_CORESIGHT_DEVARCH_VAL;
+ case GICV5_CORESIGHT_PIDR4:
+ return GICV5_CORESIGHT_PIDR4_JEP106_CONT;
+ case GICV5_CORESIGHT_PIDR5:
+ return GICV5_CORESIGHT_PIDR5_RES0;
+ case GICV5_CORESIGHT_PIDR6:
+ return GICV5_CORESIGHT_PIDR6_RES0;
+ case GICV5_CORESIGHT_PIDR7:
+ return GICV5_CORESIGHT_PIDR7_RES0;
+ case GICV5_CORESIGHT_PIDR0:
+ return GICV5_CORESIGHT_PIDR0_PART_0;
+ case GICV5_CORESIGHT_PIDR1:
+ return GICV5_CORESIGHT_PIDR1_DES_0_PART_1;
+ case GICV5_CORESIGHT_PIDR2:
+ return GICV5_CORESIGHT_PIDR2_DES_1;
+ case GICV5_CORESIGHT_PIDR3:
+ return GICV5_CORESIGHT_PIDR3_REVAND_CMOD;
+ case GICV5_CORESIGHT_CIDR0:
+ return GICV5_CORESIGHT_CIDR0_VAL;
+ case GICV5_CORESIGHT_CIDR1:
+ return GICV5_CORESIGHT_CIDR1_VAL;
+ case GICV5_CORESIGHT_CIDR2:
+ return GICV5_CORESIGHT_CIDR2_VAL;
+ case GICV5_CORESIGHT_CIDR3:
+ return GICV5_CORESIGHT_CIDR3_VAL;
+ default:
+ return 0;
+ }
+}
+
+static unsigned long vgic_v5_mmio_uaccess_read_irs_status(struct kvm_vcpu *vcpu,
+ gpa_t addr,
+ unsigned int len)
+{
+ const size_t offset = addr & (SZ_64K - 1);
+
+ switch (offset) {
+ case GICV5_IRS_SYNC_STATUSR:
+ return GICV5_IRS_SYNC_STATUSR_IDLE;
+ case GICV5_IRS_SPI_STATUSR:
+ return GICV5_IRS_SPI_STATUSR_IDLE;
+ case GICV5_IRS_PE_STATUSR:
+ return GICV5_IRS_PE_STATUSR_IDLE;
+ case GICV5_IRS_IST_STATUSR:
+ return GICV5_IRS_IST_STATUSR_IDLE;
+ default:
+ return 0;
+ }
+}
+
+static int vgic_v5_mmio_uaccess_write_irs(struct kvm_vcpu *vcpu, gpa_t addr,
+ unsigned int len, unsigned long val)
+{
+ struct vgic_dist *vgic = &vcpu->kvm->arch.vgic;
+ struct vgic_v5_irs *irs_data = vgic->vgic_v5_irs_data;
+ size_t offset = addr & (SZ_64K - 1);
+
+ /*
+ * The following registers are ONLY settable via uaccesses. The guest
+ * cannot write them!
+ */
+
+ switch (offset) {
+ case GICV5_IRS_IDR0:
+ if (FIELD_GET(GICV5_IRS_IDR0_INT_DOM, val) !=
+ GICV5_IRS_IDR0_INT_DOM_NON_SECURE)
+ return -EINVAL;
+
+ if ((val & GICV5_IRS_IDR0_VIRT) ||
+ (val & GICV5_IRS_IDR0_ONE_N) ||
+ (val & GICV5_IRS_IDR0_VIRT_ONE_N) ||
+ (val & GICV5_IRS_IDR0_SETLPI) ||
+ (val & GICV5_IRS_IDR0_MEC) ||
+ (val & GICV5_IRS_IDR0_MPAM) ||
+ (val & GICV5_IRS_IDR0_SWE))
+ return -EINVAL;
+
+ irs_data->idr0.domain = FIELD_GET(GICV5_IRS_IDR0_INT_DOM, val);
+ irs_data->idr0.pa_range = FIELD_GET(GICV5_IRS_IDR0_PA_RANGE, val);
+ irs_data->idr0.virt = !!(val & GICV5_IRS_IDR0_VIRT);
+ irs_data->idr0.setlpi = !!(val & GICV5_IRS_IDR0_SETLPI);
+ irs_data->idr0.mec = !!(val & GICV5_IRS_IDR0_MEC);
+ irs_data->idr0.mpam = !!(val & GICV5_IRS_IDR0_MPAM);
+ irs_data->idr0.swe = !!(val & GICV5_IRS_IDR0_SWE);
+ irs_data->idr0.irs_id = FIELD_GET(GICV5_IRS_IDR0_IRSID, val);
+ break;
+ case GICV5_IRS_IDR1: {
+ unsigned int iaffid_bits, priority_bits;
+ u8 vpe_id_bits;
+
+ /* Ignore writes to PE_CNT as this is populated from num vcpus */
+ iaffid_bits = FIELD_GET(GICV5_IRS_IDR1_IAFFID_BITS, val);
+ priority_bits = FIELD_GET(GICV5_IRS_IDR1_PRIORITY_BITS, val);
+
+ /* IAFFID_BITS is derived from the VMTE and encoded as N - 1. */
+ vpe_id_bits = vgic_v5_vmte_vpe_id_bits(vcpu);
+ if (iaffid_bits != vpe_id_bits - 1)
+ return -EINVAL;
+
+ if (priority_bits > gicv5_global_data.irs_pri_bits - 1)
+ return -EINVAL;
+
+ irs_data->idr1.priority_bits = priority_bits;
+ break;
+ }
+ case GICV5_IRS_IDR2:
+ /* We always support LPIs */
+ if (!(val & GICV5_IRS_IDR2_LPI))
+ return -EINVAL;
+
+ /* We only support LPIs with linear, non-metadata guest ISTs */
+ if (val & GICV5_IRS_IDR2_IST_LEVELS)
+ return -EINVAL;
+
+ if ((val & GICV5_IRS_IDR2_ISTMD) ||
+ FIELD_GET(GICV5_IRS_IDR2_ISTMD_SZ, val))
+ return -EINVAL;
+
+ /* We can't present more bits than we have support for in HW */
+ if (FIELD_GET(GICV5_IRS_IDR2_ID_BITS, val) > irs_caps.ist_id_bits)
+ return -EINVAL;
+
+ /* Min LPI ID bits must be greater than or equal to the HW */
+ if (FIELD_GET(GICV5_IRS_IDR2_MIN_LPI_ID_BITS, val) <
+ irs_caps.min_lpi_id_bits)
+ return -EINVAL;
+
+ if (FIELD_GET(GICV5_IRS_IDR2_MIN_LPI_ID_BITS, val) >
+ FIELD_GET(GICV5_IRS_IDR2_ID_BITS, val))
+ return -EINVAL;
+
+ irs_data->idr2.istmd_sz = FIELD_GET(GICV5_IRS_IDR2_ISTMD_SZ, val);
+ irs_data->idr2.istmd = !!(val & GICV5_IRS_IDR2_ISTMD);
+ irs_data->idr2.ist_l2sz = FIELD_GET(GICV5_IRS_IDR2_IST_L2SZ, val);
+ irs_data->idr2.ist_levels = !!(val & GICV5_IRS_IDR2_IST_LEVELS);
+ irs_data->idr2.min_lpi_id_bits = FIELD_GET(GICV5_IRS_IDR2_MIN_LPI_ID_BITS, val);
+ irs_data->idr2.id_bits = FIELD_GET(GICV5_IRS_IDR2_ID_BITS, val);
+ break;
+ case GICV5_IRS_IDR5:
+ if (FIELD_GET(GICV5_IRS_IDR5_SPI_RANGE, val) != irs_data->idr5.spi_range)
+ return -EINVAL;
+ break;
+ case GICV5_IRS_IDR6:
+ if (FIELD_GET(GICV5_IRS_IDR6_SPI_IRS_RANGE, val) != irs_data->idr6.spi_irs_range)
+ return -EINVAL;
+ break;
+ case GICV5_IRS_IDR7:
+ if (FIELD_GET(GICV5_IRS_IDR7_SPI_BASE, val) != irs_data->idr7.spi_base)
+ return -EINVAL;
+ break;
+ case GICV5_IRS_IST_BASER:
+ vgic_v5_update_irs_ist_baser(irs_data, val);
+ break;
+ case GICV5_IRS_IST_CFGR:
+ irs_data->ist_cfgr.lpi_id_bits = FIELD_GET(GICV5_IRS_IST_CFGR_LPI_ID_BITS, val);
+ irs_data->ist_cfgr.l2sz = FIELD_GET(GICV5_IRS_IST_CFGR_L2SZ, val);
+ irs_data->ist_cfgr.istsz = FIELD_GET(GICV5_IRS_IST_CFGR_ISTSZ, val);
+ irs_data->ist_cfgr.structure = !!(val & GICV5_IRS_IST_CFGR_STRUCTURE);
+ break;
+ case GICV5_IRS_CR0:
+ vgic->enabled = !!(val & GICV5_IRS_CR0_IRSEN);
+ break;
+ case GICV5_IRS_SPI_CFGR:
+ break;
+ case GICV5_IRS_IIDR:
+ fallthrough;
+ case GICV5_IRS_AIDR:
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const struct vgic_register_region vgic_v5_irs_registers[] = {
+ /*
+ * This is the IRS_CONFIG_FRAME.
+ */
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IDR0, vgic_v5_mmio_read_irs_misc,
+ vgic_mmio_write_wi, NULL,
+ vgic_v5_mmio_uaccess_write_irs, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IDR1, vgic_v5_mmio_read_irs_misc,
+ vgic_mmio_write_wi, NULL,
+ vgic_v5_mmio_uaccess_write_irs, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IDR2, vgic_v5_mmio_read_irs_misc,
+ vgic_mmio_write_wi, NULL,
+ vgic_v5_mmio_uaccess_write_irs, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_IDR3, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_IDR4, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IDR5, vgic_v5_mmio_read_irs_misc,
+ vgic_mmio_write_wi, NULL,
+ vgic_v5_mmio_uaccess_write_irs, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IDR6, vgic_v5_mmio_read_irs_misc,
+ vgic_mmio_write_wi, NULL,
+ vgic_v5_mmio_uaccess_write_irs, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IDR7, vgic_v5_mmio_read_irs_misc,
+ vgic_mmio_write_wi, NULL,
+ vgic_v5_mmio_uaccess_write_irs, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IIDR, vgic_v5_mmio_read_irs_misc,
+ vgic_mmio_write_wi, NULL,
+ vgic_v5_mmio_uaccess_write_irs, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_AIDR, vgic_v5_mmio_read_irs_misc,
+ vgic_mmio_write_wi, NULL,
+ vgic_v5_mmio_uaccess_write_irs, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_CR0,
+ vgic_v5_mmio_read_irs_misc,
+ vgic_v5_mmio_write_irs_misc, NULL,
+ vgic_v5_mmio_uaccess_write_irs, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_CR1, vgic_v5_mmio_read_irs_misc,
+ vgic_v5_mmio_write_irs_misc, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_SYNCR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_SYNC_STATUSR,
+ vgic_v5_mmio_read_irs_misc,
+ vgic_mmio_write_wi,
+ vgic_v5_mmio_uaccess_read_irs_status,
+ vgic_mmio_uaccess_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_SPI_VMR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 8,
+ VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_SPI_SELR, vgic_v5_mmio_read_irs_spi,
+ vgic_v5_mmio_write_irs_spi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_SPI_DOMAINR, vgic_v5_mmio_read_irs_spi,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_SPI_RESAMPLER, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_SPI_CFGR,
+ vgic_v5_mmio_read_irs_spi,
+ vgic_v5_mmio_write_irs_spi, NULL,
+ vgic_v5_mmio_uaccess_write_irs, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_SPI_STATUSR,
+ vgic_v5_mmio_read_irs_spi,
+ vgic_mmio_write_wi,
+ vgic_v5_mmio_uaccess_read_irs_status,
+ vgic_mmio_uaccess_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_PE_SELR,
+ vgic_v5_mmio_read_irs_misc,
+ vgic_v5_mmio_write_irs_misc, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_PE_STATUSR,
+ vgic_v5_mmio_read_irs_misc,
+ vgic_mmio_write_wi,
+ vgic_v5_mmio_uaccess_read_irs_status,
+ vgic_mmio_uaccess_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_PE_CR0, vgic_v5_mmio_read_irs_misc,
+ vgic_v5_mmio_write_irs_misc, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IST_BASER,
+ vgic_v5_mmio_read_irs_ist,
+ vgic_v5_mmio_write_irs_ist, NULL,
+ vgic_v5_mmio_uaccess_write_irs, 8,
+ VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IST_CFGR,
+ vgic_v5_mmio_read_irs_ist,
+ vgic_v5_mmio_write_irs_ist, NULL,
+ vgic_v5_mmio_uaccess_write_irs, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICV5_IRS_IST_STATUSR,
+ vgic_v5_mmio_read_irs_ist,
+ vgic_mmio_write_wi,
+ vgic_v5_mmio_uaccess_read_irs_status,
+ vgic_mmio_uaccess_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_MAP_L2_ISTR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+
+ /*
+ * The following registers are only for running VMs. They are not yet
+ * supported as we don't currently support nested, so expose them as
+ * read-as-zero/write-ignored.
+ */
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMT_BASER, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMT_CFGR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMT_STATUSR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VPE_SELR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VPE_DBR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VPE_HPPIR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VPE_CR0, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VPE_STATUSR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VM_DBR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VM_SELR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VM_STATUSR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMAP_L2_VMTR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMAP_VMR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMAP_VISTR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMAP_L2_VISTR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_VMAP_VPER, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_SAVE_VMR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_SAVE_VM_STATUSR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+
+ /* MEC, MPAM, SWERR - all unimplemented */
+
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_MEC_IDR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_MEC_MECID_R, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_MPAM_IDR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_MPAM_PARTID_R, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 4, VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_SWERR_STATUSR, vgic_mmio_read_raz,
+ vgic_mmio_write_wi, 8, VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_SWERR_SYNDROMER0,
+ vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
+ VGIC_ACCESS_64bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_IRS_SWERR_SYNDROMER1,
+ vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
+ VGIC_ACCESS_64bit),
+
+ /* CoreSight identification registers */
+ REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_DEVARCH,
+ vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR4,
+ vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR5,
+ vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR6,
+ vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR7,
+ vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR0,
+ vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR1,
+ vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR2,
+ vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_PIDR3,
+ vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR0,
+ vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR1,
+ vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR2,
+ vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICV5_CORESIGHT_CIDR3,
+ vgic_v5_coresight_read, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
+};
+
+unsigned int vgic_v5_init_irs_iodev(struct vgic_io_device *dev)
+{
+ dev->regions = vgic_v5_irs_registers;
+ dev->nr_regions = ARRAY_SIZE(vgic_v5_irs_registers);
+
+ kvm_iodevice_init(&dev->dev, &kvm_io_gic_ops);
+
+ /* We represent both of the IRS frames back to back, so this is 128K */
+ return KVM_VGIC_V5_IRS_SIZE;
+}
+
+int vgic_v5_register_irs_iodev(struct kvm *kvm, gpa_t irs_base_address)
+{
+ struct vgic_io_device *io_device = &kvm->arch.vgic.vgic_v5_irs_data->iodev;
+ unsigned int len;
+
+ /*
+ * Design choice: Force MMIO region to be 64k aligned. Simplifies
+ * pulling out registers.
+ */
+ if (!IS_ALIGNED(irs_base_address, SZ_64K))
+ return -EINVAL;
+
+ len = vgic_v5_init_irs_iodev(io_device);
+
+ io_device->base_addr = irs_base_address;
+ io_device->iodev_type = IODEV_GICV5_IRS;
+ io_device->redist_vcpu = NULL;
+
+ return kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, irs_base_address, len,
+ &io_device->dev);
+}
+
+/**
+ * kvm_vgic_v5_irs_init: initialize the IRS data structures
+ * @kvm: kvm struct pointer
+ * @nr_spis: number of spis, frozen by caller
+ */
+int kvm_vgic_v5_irs_init(struct kvm *kvm, unsigned int nr_spis)
+{
+ struct vgic_dist *dist = &kvm->arch.vgic;
+ struct vgic_v5_irs *irs = dist->vgic_v5_irs_data;
+ struct kvm_vcpu *vcpu0 = kvm_get_vcpu(kvm, 0);
+ size_t nr_spi_bits;
+ u64 mmfr0;
+ int ret, i;
+
+ /*
+ * We (KVM) allocate an Interrupt State Table (IST) for SPIs. The
+ * hardware mandates that lower 6 bits of the address are 0. Each ISTE
+ * is 4 bytes in size (or larger if metadata storage is required), so 16
+ * entries would be enough for alignment. Keep the minimum at 32 SPIs to
+ * match KVM's vGICv3 minimum and the VGICv5 device API.
+ */
+ if (nr_spis && nr_spis < VGIC_V5_DEFAULT_NR_SPIS)
+ nr_spis = VGIC_V5_DEFAULT_NR_SPIS;
+
+ if (nr_spis) {
+ dist->spis = kcalloc(nr_spis, sizeof(struct vgic_irq),
+ GFP_KERNEL_ACCOUNT);
+ if (!dist->spis)
+ return -ENOMEM;
+
+ /*
+ * In the following code we do not take the irq struct lock since
+ * no other action on irq structs can happen while the VGIC is
+ * not initialized yet.
+ */
+ for (i = 0; i < nr_spis; i++) {
+ struct vgic_irq *irq = &dist->spis[i];
+
+ irq->intid = vgic_v5_make_spi(i);
+ INIT_LIST_HEAD(&irq->ap_list);
+ raw_spin_lock_init(&irq->irq_lock);
+ irq->vcpu = NULL;
+ irq->target_vcpu = vcpu0;
+ refcount_set(&irq->refcount, 0);
+ /*
+ * The guest controls the enable state, and again it is
+ * directly handled by the hardware. From our point of
+ * view it is always enabled.
+ */
+ irq->enabled = 1;
+ vgic_v5_set_spi_ops(irq);
+ }
+
+ nr_spi_bits = fls(roundup_pow_of_two(nr_spis)) - 1;
+
+ ret = vgic_v5_spi_ist_alloc(kvm, nr_spi_bits);
+ if (ret) {
+ kfree(dist->spis);
+ dist->spis = NULL;
+ return ret;
+ }
+ }
+
+ /* Set sane initial state for the IRS MMIO registers */
+
+ irs->idr0.domain = GICV5_IRS_IDR0_INT_DOM_NON_SECURE;
+
+ mmfr0 = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1);
+ irs->idr0.pa_range = cpuid_feature_extract_unsigned_field(mmfr0,
+ ID_AA64MMFR0_EL1_PARANGE_SHIFT);
+
+ irs->idr0.virt = 0;
+ irs->idr0.setlpi = 0;
+ irs->idr0.mec = 0;
+ irs->idr0.mpam = 0;
+ irs->idr0.swe = 0;
+ irs->idr0.irs_id = 0;
+
+ irs->idr1.priority_bits = gicv5_global_data.irs_pri_bits - 1;
+
+ /*
+ * Support 16-bits of ID space for the IRS. This should be sufficient
+ * for most applications, and the CPUIF is guaranteed to have at least
+ * 16-bits of ID space support (we actually present 16-bits there, even
+ * if the hardware supports more). Warn if the hardware doesn't support
+ * 16 bits, and use the smaller value. YMMV!
+ *
+ * As for the minimum number of ID bits, we match the hardware's
+ * capability.
+ */
+ if (irs_caps.ist_id_bits < 16)
+ pr_warn("Host IRS supports fewer than 16 ID bits for ISTs (%u)\n",
+ irs_caps.ist_id_bits);
+
+ irs->idr2.id_bits = min(16, irs_caps.ist_id_bits);
+ irs->idr2.min_lpi_id_bits = irs_caps.min_lpi_id_bits;
+
+ /* Only allow the guest to create Linear ISTs - simplifies Save/Restore */
+ irs->idr2.ist_levels = 0;
+ irs->idr2.ist_l2sz = GICV5_IRS_IST_CFGR_L2SZ_4K;
+ irs->idr2.istmd = 0;
+ irs->idr2.istmd_sz = 0;
+
+ /* We have a single IRS, only. All SPIs reside here! */
+ irs->idr5.spi_range = nr_spis;
+ irs->idr6.spi_irs_range = nr_spis;
+ irs->idr7.spi_base = 0;
+
+ irs->cr1.sh = 0;
+ irs->cr1.oc = 0;
+ irs->cr1.ic = 0;
+ irs->cr1.ist_ra = 0;
+ irs->cr1.ist_wa = 0;
+ irs->cr1.vmt_ra = 0;
+ irs->cr1.vpet_ra = 0;
+ irs->cr1.vmd_ra = 0;
+ irs->cr1.vmd_wa = 0;
+ irs->cr1.vped_ra = 0;
+ irs->cr1.vped_wa = 0;
+
+ irs->spi_selr.id = -1;
+
+ irs->pe_selr.iaffid = -1;
+
+ irs->ist_cfgr.lpi_id_bits = 0;
+ irs->ist_cfgr.l2sz = 0;
+ irs->ist_cfgr.istsz = 0;
+ irs->ist_cfgr.structure = 0;
+
+ irs->ist_baser.valid = 0;
+ irs->ist_baser.addr = 0;
+
+ return 0;
+}
+
+int vgic_v5_irs_lpi_ist_id_bits(struct kvm *kvm, unsigned int *id_bits)
+{
+ struct vgic_v5_irs *irs = kvm->arch.vgic.vgic_v5_irs_data;
+
+ if (!irs)
+ return -ENXIO;
+
+ if (!irs->ist_baser.valid)
+ return 0;
+
+ if (!vgic_v5_ist_cfgr_valid(irs)) {
+ kvm_err("Guest programmed invalid IRS_IST_CFGR\n");
+ return -EINVAL;
+ }
+
+ *id_bits = irs->ist_cfgr.lpi_id_bits;
+
+ return 1;
+}
+
+int vgic_v5_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
+{
+ const struct vgic_register_region *region;
+ struct vgic_reg_attr reg_attr;
+ struct kvm_vcpu *vcpu;
+ gpa_t addr, offset;
+ int ret, align;
+
+ ret = vgic_v5_parse_attr(dev, attr, ®_attr);
+ if (ret)
+ return ret;
+
+ vcpu = reg_attr.vcpu;
+ addr = reg_attr.addr;
+
+ if (attr->group == KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS)
+ return vgic_v5_has_cpu_sysregs_attr(vcpu, attr);
+
+ offset = attr->attr;
+
+ region = vgic_find_mmio_region(vgic_v5_irs_registers,
+ ARRAY_SIZE(vgic_v5_irs_registers),
+ offset);
+ if (!region)
+ return -ENXIO;
+
+ align = region->access_flags & VGIC_ACCESS_64bit ? 0x7 : 0x3;
+ if (offset & align)
+ return -EINVAL;
+
+ return 0;
+}
+
+/*
+ * Access the IRS MMIO Regs. Relevant locks have been taken by the calling code.
+ */
+int vgic_v5_irs_attr_regs_access(struct kvm_device *dev,
+ struct kvm_device_attr *attr,
+ u64 *reg, bool is_write)
+{
+ const struct vgic_register_region *region;
+ gpa_t addr, offset;
+ unsigned int len;
+ int align, ret = 0;
+
+ offset = attr->attr;
+
+ if (IS_VGIC_ADDR_UNDEF(dev->kvm->arch.vgic.vgic_v5_irs_data->vgic_v5_irs_base))
+ return -ENXIO;
+
+ region = vgic_find_mmio_region(vgic_v5_irs_registers,
+ ARRAY_SIZE(vgic_v5_irs_registers),
+ offset);
+ if (!region)
+ return -ENXIO;
+
+ /*
+ * Although the spec supports upper/lower 32-bit accesses to
+ * 64-bit IRS registers, the userspace ABI requires 64-bit
+ * accesses to all 64-bit wide registers. We therefore only
+ * support 32-bit accesses to 32-bit-wide registers.
+ */
+ align = region->access_flags & VGIC_ACCESS_64bit ? 0x7 : 0x3;
+ len = region->access_flags & VGIC_ACCESS_64bit ? 8 : 4;
+
+ if (offset & align)
+ return -EINVAL;
+
+ addr = dev->kvm->arch.vgic.vgic_v5_irs_data->vgic_v5_irs_base + offset;
+
+ if (is_write) {
+ if (region->uaccess_write)
+ ret = region->uaccess_write(kvm_get_vcpu(dev->kvm, 0),
+ addr, len, *reg);
+ else
+ region->write(kvm_get_vcpu(dev->kvm, 0), addr, len, *reg);
+ } else {
+ if (region->uaccess_read)
+ *reg = region->uaccess_read(kvm_get_vcpu(dev->kvm, 0),
+ addr, len);
+ else
+ *reg = region->read(kvm_get_vcpu(dev->kvm, 0), addr, len);
+ }
+
+ return ret;
+}
diff --git a/arch/arm64/kvm/vgic/vgic-kvm-device.c b/arch/arm64/kvm/vgic/vgic-kvm-device.c
index 90be99443df3bc..1c205cb1361fd5 100644
--- a/arch/arm64/kvm/vgic/vgic-kvm-device.c
+++ b/arch/arm64/kvm/vgic/vgic-kvm-device.c
@@ -181,6 +181,14 @@ static int kvm_vgic_addr(struct kvm *kvm, struct kvm_device_attr *attr, bool wri
addr |= (u64)rdreg->count << KVM_VGIC_V3_RDIST_COUNT_SHIFT;
goto out;
}
+ case KVM_VGIC_V5_ADDR_TYPE_IRS:
+ r = vgic_check_type(kvm, KVM_DEV_TYPE_ARM_VGIC_V5);
+ if (r)
+ break;
+ addr_ptr = &vgic->vgic_v5_irs_data->vgic_v5_irs_base;
+ alignment = SZ_64K;
+ size = KVM_VGIC_V5_IRS_SIZE;
+ break;
default:
r = -ENODEV;
}
@@ -224,31 +232,48 @@ static int vgic_set_common_attr(struct kvm_device *dev,
if (get_user(val, uaddr))
return -EFAULT;
- /*
- * We require:
- * - at least 32 SPIs on top of the 16 SGIs and 16 PPIs
- * - at most 1024 interrupts
- * - a multiple of 32 interrupts
- */
- if (val < (VGIC_NR_PRIVATE_IRQS + 32) ||
- val > VGIC_MAX_RESERVED ||
- (val & 31))
- return -EINVAL;
+ if (!vgic_is_v5(dev->kvm)) {
+ /*
+ * We require:
+ * - at least 32 SPIs on top of the 16 SGIs and 16 PPIs
+ * - at most 1024 interrupts
+ * - a multiple of 32 interrupts
+ */
+ if (val < (VGIC_NR_PRIVATE_IRQS + 32) ||
+ val > VGIC_MAX_RESERVED || (val & 31))
+ return -EINVAL;
+
+ mutex_lock(&dev->kvm->arch.config_lock);
- mutex_lock(&dev->kvm->arch.config_lock);
+ /*
+ * Either userspace has already configured NR_IRQS or
+ * the vgic has already been initialized and vgic_init()
+ * supplied a default amount of SPIs.
+ */
+ if (dev->kvm->arch.vgic.nr_spis)
+ ret = -EBUSY;
+ else
+ dev->kvm->arch.vgic.nr_spis =
+ val - VGIC_NR_PRIVATE_IRQS;
- /*
- * Either userspace has already configured NR_IRQS or
- * the vgic has already been initialized and vgic_init()
- * supplied a default amount of SPIs.
- */
- if (dev->kvm->arch.vgic.nr_spis)
- ret = -EBUSY;
- else
- dev->kvm->arch.vgic.nr_spis =
- val - VGIC_NR_PRIVATE_IRQS;
+ mutex_unlock(&dev->kvm->arch.config_lock);
+ } else {
+ /*
+ * GICv5 reports a number of SPIs, not a total number of
+ * interrupts. Require a multiple of 32 SPIs.
+ */
+ if (val < VGIC_V5_DEFAULT_NR_SPIS ||
+ val > VGIC_V5_MAX_NR_SPIS ||
+ (val & 31))
+ return -EINVAL;
- mutex_unlock(&dev->kvm->arch.config_lock);
+ mutex_lock(&dev->kvm->arch.config_lock);
+ if (vgic_initialized(dev->kvm) || dev->kvm->arch.vgic.nr_spis)
+ ret = -EBUSY;
+ else
+ dev->kvm->arch.vgic.nr_spis = val;
+ mutex_unlock(&dev->kvm->arch.config_lock);
+ }
return ret;
}
@@ -299,9 +324,14 @@ static int vgic_get_common_attr(struct kvm_device *dev,
return (r == -ENODEV) ? -ENXIO : r;
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS: {
u32 __user *uaddr = (u32 __user *)(long)attr->addr;
-
- r = put_user(dev->kvm->arch.vgic.nr_spis +
- VGIC_NR_PRIVATE_IRQS, uaddr);
+ /* Older GICs */
+ if (!vgic_is_v5(dev->kvm)) {
+ r = put_user(dev->kvm->arch.vgic.nr_spis +
+ VGIC_NR_PRIVATE_IRQS,
+ uaddr);
+ } else {
+ r = put_user(dev->kvm->arch.vgic.nr_spis, uaddr);
+ }
break;
}
}
@@ -512,7 +542,7 @@ int vgic_v3_parse_attr(struct kvm_device *dev, struct kvm_device_attr *attr,
* Allow access to certain ID-like registers prior to VGIC initialization,
* thereby allowing the VMM to provision the features / sizing of the VGIC.
*/
-static bool reg_allowed_pre_init(struct kvm_device_attr *attr)
+static bool v3_reg_allowed_pre_init(struct kvm_device_attr *attr)
{
if (attr->group != KVM_DEV_ARM_VGIC_GRP_DIST_REGS)
return false;
@@ -575,7 +605,7 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
mutex_lock(&dev->kvm->arch.config_lock);
- if (!(vgic_initialized(dev->kvm) || reg_allowed_pre_init(attr))) {
+ if (!(vgic_initialized(dev->kvm) || v3_reg_allowed_pre_init(attr))) {
ret = -EBUSY;
goto out;
}
@@ -743,26 +773,178 @@ static int vgic_v5_get_userspace_ppis(struct kvm_device *dev,
return ret;
}
+int vgic_v5_parse_attr(struct kvm_device *dev, struct kvm_device_attr *attr,
+ struct vgic_reg_attr *reg_attr)
+{
+ unsigned long vgic_mpidr, mpidr_reg;
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:
+ vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
+ KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
+
+ mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
+ reg_attr->vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
+ break;
+ case KVM_DEV_ARM_VGIC_GRP_IRS_REGS:
+ reg_attr->vcpu = kvm_get_vcpu(dev->kvm, 0);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (!reg_attr->vcpu)
+ return -EINVAL;
+
+ reg_attr->addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+
+ return 0;
+}
+
+/*
+ * Some registers can potentially be read before the core GIC & IRS has been
+ * initialised. Right now, everything is required to be post-init.
+ */
+static bool v5_reg_allowed_pre_init(struct kvm_device_attr *attr)
+{
+ return false;
+}
+
+static bool vgic_v5_vm_has_run_once(struct kvm *kvm)
+{
+ struct kvm_vcpu *vcpu;
+ unsigned long i;
+
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ if (vcpu_has_run_once(vcpu))
+ return true;
+ }
+
+ return false;
+}
+
+/*
+ * vgic_v5_attr_regs_access - allows user space to access VGIC v5 state
+ *
+ * @dev: kvm device handle
+ * @attr: kvm device attribute
+ * @is_write: true if userspace is writing a register
+ */
+static int vgic_v5_attr_regs_access(struct kvm_device *dev,
+ struct kvm_device_attr *attr,
+ bool is_write)
+{
+ u64 __user *uaddr = (u64 __user *)(unsigned long)attr->addr;
+ struct vgic_reg_attr reg_attr;
+ struct kvm_vcpu *vcpu;
+ bool uaccess;
+ u64 val;
+ int ret;
+
+ ret = vgic_v5_parse_attr(dev, attr, ®_attr);
+ if (ret)
+ return ret;
+
+ vcpu = reg_attr.vcpu;
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:
+ /* Sysregs uaccess is performed by the sysreg handling code */
+ uaccess = false;
+ break;
+ case KVM_DEV_ARM_VGIC_GRP_IRS_REGS:
+ fallthrough;
+ default:
+ uaccess = true;
+ }
+
+ if (uaccess && is_write) {
+ if (get_user(val, uaddr))
+ return -EFAULT;
+ }
+
+ mutex_lock(&dev->kvm->lock);
+
+ if (kvm_trylock_all_vcpus(dev->kvm)) {
+ mutex_unlock(&dev->kvm->lock);
+ return -EBUSY;
+ }
+
+ mutex_lock(&dev->kvm->arch.config_lock);
+
+ if (!(vgic_initialized(dev->kvm) || v5_reg_allowed_pre_init(attr))) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ if (is_write && vgic_v5_vm_has_run_once(dev->kvm)) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:
+ ret = vgic_v5_cpu_sysregs_uaccess(vcpu, attr, is_write);
+ break;
+ case KVM_DEV_ARM_VGIC_GRP_IRS_REGS:
+ /*
+ * The IRS registers are a mixture of 32-bit and 64-bit
+ * registers. Internally, we always perform the correctly sized
+ * access, but the UAPI is defined in such a way that we are
+ * always provided a __u64 by userspace. When userspace writes,
+ * the upper 32-bits are ignored for 32-bit accesses, and on a
+ * read any 32-bit accesses are written back to user memory
+ * using the full 64-bits.
+ */
+ ret = vgic_v5_irs_attr_regs_access(dev, attr, &val, is_write);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+out:
+ mutex_unlock(&dev->kvm->arch.config_lock);
+ kvm_unlock_all_vcpus(dev->kvm);
+ mutex_unlock(&dev->kvm->lock);
+
+ if (!ret && uaccess && !is_write)
+ ret = put_user(val, uaddr);
+
+ return ret;
+}
+
static int vgic_v5_set_attr(struct kvm_device *dev,
struct kvm_device_attr *attr)
{
switch (attr->group) {
case KVM_DEV_ARM_VGIC_GRP_ADDR:
+ break;
+ case KVM_DEV_ARM_VGIC_GRP_IST:
+ if (attr->attr)
+ return -ENXIO;
+
+ return vgic_v5_irs_restore_ists(dev->kvm, attr);
+ case KVM_DEV_ARM_VGIC_GRP_IRS_REGS:
+ fallthrough;
case KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:
+ return vgic_v5_attr_regs_access(dev, attr, true);
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
- return -ENXIO;
+ break;
case KVM_DEV_ARM_VGIC_GRP_CTRL:
switch (attr->attr) {
case KVM_DEV_ARM_VGIC_CTRL_INIT:
- return vgic_set_common_attr(dev, attr);
+ break;
case KVM_DEV_ARM_VGIC_USERSPACE_PPIS:
default:
return -ENXIO;
}
+ break;
default:
return -ENXIO;
}
+ return vgic_set_common_attr(dev, attr);
}
static int vgic_v5_get_attr(struct kvm_device *dev,
@@ -770,21 +952,33 @@ static int vgic_v5_get_attr(struct kvm_device *dev,
{
switch (attr->group) {
case KVM_DEV_ARM_VGIC_GRP_ADDR:
+ break;
+ case KVM_DEV_ARM_VGIC_GRP_IST:
+ if (attr->attr)
+ return -ENXIO;
+
+ return vgic_v5_irs_save_ists(dev->kvm, attr);
+ case KVM_DEV_ARM_VGIC_GRP_IRS_REGS:
+ fallthrough;
case KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:
+ return vgic_v5_attr_regs_access(dev, attr, false);
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
- return -ENXIO;
+ break;
case KVM_DEV_ARM_VGIC_GRP_CTRL:
switch (attr->attr) {
case KVM_DEV_ARM_VGIC_CTRL_INIT:
- return vgic_get_common_attr(dev, attr);
+ break;
case KVM_DEV_ARM_VGIC_USERSPACE_PPIS:
return vgic_v5_get_userspace_ppis(dev, attr);
default:
return -ENXIO;
}
+ break;
default:
return -ENXIO;
}
+
+ return vgic_get_common_attr(dev, attr);
}
static int vgic_v5_has_attr(struct kvm_device *dev,
@@ -792,18 +986,30 @@ static int vgic_v5_has_attr(struct kvm_device *dev,
{
switch (attr->group) {
case KVM_DEV_ARM_VGIC_GRP_ADDR:
+ switch (attr->attr) {
+ case KVM_VGIC_V5_ADDR_TYPE_IRS:
+ return 0;
+ }
+ return -ENXIO;
+ case KVM_DEV_ARM_VGIC_GRP_IRS_REGS:
+ fallthrough;
case KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:
+ return vgic_v5_has_attr_regs(dev, attr);
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
- return -ENXIO;
+ return 0;
case KVM_DEV_ARM_VGIC_GRP_CTRL:
switch (attr->attr) {
case KVM_DEV_ARM_VGIC_CTRL_INIT:
return 0;
case KVM_DEV_ARM_VGIC_USERSPACE_PPIS:
return 0;
+ case KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES:
default:
return -ENXIO;
}
+ break;
+ case KVM_DEV_ARM_VGIC_GRP_IST:
+ return attr->attr ? -ENXIO : 0;
default:
return -ENXIO;
}
diff --git a/arch/arm64/kvm/vgic/vgic-mmio.c b/arch/arm64/kvm/vgic/vgic-mmio.c
index 74d76dec97304b..fddb9da0403d55 100644
--- a/arch/arm64/kvm/vgic/vgic-mmio.c
+++ b/arch/arm64/kvm/vgic/vgic-mmio.c
@@ -1065,6 +1065,9 @@ static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
case IODEV_ITS:
data = region->its_read(vcpu->kvm, iodev->its, addr, len);
break;
+ case IODEV_GICV5_IRS:
+ data = region->read(vcpu, addr, len);
+ break;
}
vgic_data_host_to_mmio_bus(val, len, data);
@@ -1095,6 +1098,9 @@ static int dispatch_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
case IODEV_ITS:
region->its_write(vcpu->kvm, iodev->its, addr, len, data);
break;
+ case IODEV_GICV5_IRS:
+ region->write(vcpu, addr, len, data);
+ break;
}
return 0;
diff --git a/arch/arm64/kvm/vgic/vgic-mmio.h b/arch/arm64/kvm/vgic/vgic-mmio.h
index 50dc80220b0f37..38ed730d68ac38 100644
--- a/arch/arm64/kvm/vgic/vgic-mmio.h
+++ b/arch/arm64/kvm/vgic/vgic-mmio.h
@@ -217,6 +217,8 @@ unsigned int vgic_v2_init_cpuif_iodev(struct vgic_io_device *dev);
unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
+unsigned int vgic_v5_init_irs_iodev(struct vgic_io_device *dev);
+
u64 vgic_sanitise_outer_cacheability(u64 reg);
u64 vgic_sanitise_inner_cacheability(u64 reg);
u64 vgic_sanitise_shareability(u64 reg);
diff --git a/arch/arm64/kvm/vgic/vgic-v5-tables.c b/arch/arm64/kvm/vgic/vgic-v5-tables.c
new file mode 100644
index 00000000000000..f220267266c29d
--- /dev/null
+++ b/arch/arm64/kvm/vgic/vgic-v5-tables.c
@@ -0,0 +1,1795 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2025, 2026 Arm Ltd.
+ */
+
+#include <kvm/arm_vgic.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/mutex.h>
+#include <linux/sizes.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+#include <linux/xarray.h>
+#include <asm/kvm_mmu.h>
+
+#include "vgic.h"
+#include "vgic-v5-tables.h"
+
+#define irs_caps kvm_vgic_global_state.vgic_v5_irs_caps
+
+static struct vgic_v5_vmt *vmt_info;
+/* Serialises lazy installation of shared second-level VMTs. */
+static DEFINE_MUTEX(vmt_l2_lock);
+static DEFINE_XARRAY(vm_info);
+
+/* Level 1 Virtual Machine Table Entry */
+#define GICV5_VMTEL1E_VALID BIT_ULL(0)
+/* Note that there is no shift for the address by design */
+#define GICV5_VMTEL1E_L2_ADDR GENMASK(51, 12)
+
+#define GICV5_VMTEL2E_SIZE 32ULL
+/* An L2 table (two-level VMT) is ALWAYS 4kB! */
+#define GICV5_VMT_L2_TABLE_SIZE 4096ULL
+#define GICV5_VMT_L2_TABLE_ENTRIES (GICV5_VMT_L2_TABLE_SIZE / GICV5_VMTEL2E_SIZE)
+
+/*
+ * As the L2 VMTE is a large data structure, we are splitting it into 4 parts.
+ * We only mask and shift WITHIN each part for simplicity.
+ */
+/* First 64-bit chunk */
+#define GICV5_VMTEL2E_VALID BIT_ULL(0)
+#define GICV5_VMTEL2E_VMD_ADDR_SHIFT 3ULL
+#define GICV5_VMTEL2E_VMD_ADDR GENMASK_ULL(55, 3)
+/* Second 64-bit chunk */
+#define GICV5_VMTEL2E_VPET_ADDR_SHIFT 3ULL
+#define GICV5_VMTEL2E_VPET_ADDR GENMASK_ULL(55, 3)
+#define GICV5_VMTEL2E_VPE_ID_BITS GENMASK_ULL(63, 59)
+/* Third & fourth 64-bit chunks (the encodings are the same for each) */
+#define GICV5_VMTEL2E_IST_VALID BIT_ULL(0)
+#define GICV5_VMTEL2E_IST_L2SZ GENMASK_ULL(2, 1)
+#define GICV5_VMTEL2E_IST_ADDR_SHIFT 6ULL
+#define GICV5_VMTEL2E_IST_ADDR GENMASK_ULL(55, 6)
+#define GICV5_VMTEL2E_IST_ISTSZ GENMASK_ULL(57, 56)
+#define GICV5_VMTEL2E_IST_STRUCTURE BIT_ULL(58)
+#define GICV5_VMTEL2E_IST_ID_BITS GENMASK_ULL(63, 59)
+
+/* Virtual PE Table Entry */
+#define GICV5_VPE_VALID BIT_ULL(0)
+/* Note that there is no shift for the address by design. */
+#define GICV5_VPED_ADDR_SHIFT 3ULL
+#define GICV5_VPED_ADDR GENMASK_ULL(55, 3)
+
+/* L2 Interrupt State Table Entry */
+#define GICV5_ISTL2E_PENDING BIT(0)
+#define GICV5_ISTL2E_ACTIVE BIT(1)
+#define GICV5_ISTL2E_HM BIT(2)
+#define GICV5_ISTL2E_ENABLE BIT(3)
+#define GICV5_ISTL2E_IRM BIT(4)
+#define GICV5_ISTL2E_HWU GENMASK(10, 9)
+#define GICV5_ISTL2E_PRIORITY GENMASK(15, 11)
+#define GICV5_ISTL2E_IAFFID GENMASK(31, 16)
+
+#define GICV5_ISTE_SIZE(istsz) BIT((istsz) + 2)
+#define GICV5_LINEAR_IST_SIZE(id_bits, istsz) \
+ (BIT(id_bits) * GICV5_ISTE_SIZE(istsz))
+
+/*
+ * The LPI and SPI configuration is stored in the 2nd and 3rd 64-bit chunks of
+ * the VMTE (0-based). We call this a section here in an attempt to simplify the
+ * code.
+ */
+#define GICV5_VMTEL2_LPI_SECTION 2
+#define GICV5_VMTEL2_SPI_SECTION 3
+
+struct vgic_v5_ist_desc {
+ struct vgic_v5_vm_info *vmi;
+ void *base;
+ unsigned int id_bits;
+ unsigned int istsz;
+ unsigned int l2sz;
+ size_t iste_size;
+ bool present;
+};
+
+struct vgic_v5_two_level_ist_shape {
+ size_t l1_entries;
+ size_t l2_entries;
+};
+
+struct vgic_v5_pending_irq {
+ u32 irq;
+ struct list_head next;
+};
+
+static int vgic_v5_alloc_linear_ist(struct kvm *kvm, bool spi_ist,
+ unsigned int id_bits,
+ unsigned int istsz);
+static int vgic_v5_alloc_l1_ist(struct kvm *kvm, unsigned int id_bits,
+ unsigned int istsz, unsigned int l2_split);
+static int vgic_v5_alloc_l2_ists(struct kvm *kvm, unsigned int id_bits,
+ unsigned int istsz, unsigned int l2_split);
+static int vgic_v5_alloc_two_level_lpi_ist(struct kvm *kvm,
+ unsigned int id_bits,
+ unsigned int istsz,
+ unsigned int l2_split);
+static int vgic_v5_linear_ist_free(struct kvm *kvm, bool spi);
+static int vgic_v5_two_level_ist_free(struct kvm *kvm, bool spi);
+static int vgic_v5_spi_ist_free(struct kvm *kvm);
+
+/*
+ * Our IRS might be coherent or non-coherent. If coherent, we can just emit a
+ * DSB to ensure that we're in sync. However, when non-coherent, we need to
+ * manage our cached data explicitly.
+ *
+ * This helper is used to handle both coherent and non-coherent IRSes, and
+ * handles all combinations of cleaning and invalidating to the PoC.
+ */
+static void vgic_v5_clean_inval(void *va, size_t size)
+{
+ unsigned long base = (unsigned long)va;
+
+ dsb(ishst);
+
+ if (kvm_vgic_global_state.vgic_v5_irs_caps.non_coherent)
+ dcache_clean_inval_poc(base, base + size);
+}
+
+static void vgic_v5_drain_pending_irqs(struct kvm *kvm,
+ struct vgic_v5_vm_info *vmi,
+ bool reinject)
+{
+ struct vgic_v5_pending_irq *pirq, *tmp;
+
+ list_for_each_entry_safe(pirq, tmp, &vmi->pending_irqs, next) {
+ if (reinject)
+ kvm_call_hyp(__vgic_v5_vdpend, pirq->irq, true,
+ vgic_v5_vm_id(kvm));
+
+ list_del(&pirq->next);
+ kfree(pirq);
+ }
+}
+
+/*
+ * Create a linear VM Table. Directly using the number of entries supplied as
+ * the size of an L2 VMTE (32 bytes) guarantees that our allocation is aligned per
+ * the GICv5 requirements for the IRS_VMT_BASER.
+ */
+static int vgic_v5_alloc_vmt_linear(unsigned int num_entries)
+{
+ vmt_info->linear.vmt_base = kzalloc_objs(*vmt_info->linear.vmt_base,
+ num_entries);
+ if (!vmt_info->linear.vmt_base)
+ return -ENOMEM;
+
+ vgic_v5_clean_inval(vmt_info->linear.vmt_base,
+ num_entries * sizeof(struct vmtl2_entry));
+
+ return 0;
+}
+
+/*
+ * Allocate the first level of a two-level VM table. The second-level VM tables
+ * are allocated on demand (by vgic_v5_alloc_l2_vmt()).
+ */
+static int vgic_v5_alloc_vmt_two_level(unsigned int num_entries)
+{
+ /*
+ * Each L2 VMT array is always 4k-sized (covering 128 VMs). This is
+ * mandated by the GICv5 specification (GICv5 EAC0 Specification rule
+ * D_LSPBK). Hence, round up the number of entries to be at least 128
+ * (or the next highest power of two as we give the HW the number of VM
+ * ID bits).
+ */
+ if (num_entries < GICV5_VMT_L2_TABLE_ENTRIES)
+ num_entries = GICV5_VMT_L2_TABLE_ENTRIES;
+ num_entries = roundup_pow_of_two(num_entries);
+
+ vmt_info->l2.num_l1_ents = (num_entries / GICV5_VMT_L2_TABLE_ENTRIES);
+ vmt_info->l2.vmt_base = kzalloc_objs(*vmt_info->l2.vmt_base,
+ vmt_info->l2.num_l1_ents);
+ if (!vmt_info->l2.vmt_base)
+ return -ENOMEM;
+
+ vmt_info->l2.l2ptrs = kzalloc_objs(*vmt_info->l2.l2ptrs,
+ vmt_info->l2.num_l1_ents,
+ GFP_KERNEL);
+ if (!vmt_info->l2.l2ptrs) {
+ kfree(vmt_info->l2.vmt_base);
+ return -ENOMEM;
+ }
+
+ vgic_v5_clean_inval(vmt_info->l2.vmt_base,
+ vmt_info->l2.num_l1_ents * sizeof(vmtl1_entry));
+
+ return 0;
+}
+
+/*
+ * Allocate a second level VMT, if required. This can be called eagerly, and
+ * will only perform the allocation if required.
+ */
+static int vgic_v5_alloc_l2_vmt(struct kvm *kvm)
+{
+ struct kvm_vcpu *vcpu0 = kvm_get_vcpu(kvm, 0);
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ enum gicv5_vcpu_cmd cmd = VMT_L2_MAP;
+ struct vmtl2_entry *l2_table;
+ unsigned int l1_index;
+ int ret;
+
+ /* Nothing to do if we have linear tables! */
+ if (!vmt_info->two_level)
+ return 0;
+
+ if (vm_id == VGIC_V5_VM_ID_INVAL)
+ return -EINVAL;
+
+ /*
+ * We have 4k-sized L2 tables - this is mandated by the spec for
+ * two-level VMTs (GICv5 EAC0 Specification rule D_LSPBK). This means
+ * that we have 128 entries per L1 VMTE.
+ */
+ l1_index = vm_id / GICV5_VMT_L2_TABLE_ENTRIES;
+
+ guard(mutex)(&vmt_l2_lock);
+
+ /* Already valid? Great! */
+ if (vmt_info->l2.l2ptrs[l1_index])
+ return 0;
+
+ l2_table = kzalloc_objs(*l2_table, GICV5_VMT_L2_TABLE_ENTRIES);
+ if (!l2_table)
+ return -ENOMEM;
+
+ vgic_v5_clean_inval(l2_table, GICV5_VMT_L2_TABLE_SIZE);
+
+ vgic_v5_clean_inval(vmt_info->l2.vmt_base + l1_index,
+ sizeof(vmtl1_entry));
+
+ WRITE_ONCE(vmt_info->l2.vmt_base[l1_index],
+ cpu_to_le64(virt_to_phys(l2_table)));
+
+ vgic_v5_clean_inval(vmt_info->l2.vmt_base + l1_index,
+ sizeof(vmtl1_entry));
+
+ /*
+ * VMAP in the L2 VMT via the IRS. We use any of the VM's CPUs as a
+ * conduit for interacting with the host's IRS. In the current case,
+ * this lets us resolve the VM ID to pass to the hardware.
+ */
+ ret = irq_set_vcpu_affinity(vgic_v5_vpe_db(vcpu0), &cmd);
+
+ /* We've failed to make the L2 VMT valid - things are very broken! */
+ if (ret) {
+ /* Remove the pointer from L1 table */
+ WRITE_ONCE(vmt_info->l2.vmt_base[l1_index], 0);
+
+ vgic_v5_clean_inval(vmt_info->l2.vmt_base + l1_index,
+ sizeof(vmtl1_entry));
+
+ kfree(l2_table);
+
+ return ret;
+ }
+
+ vmt_info->l2.l2ptrs[l1_index] = l2_table;
+
+ return 0;
+}
+
+/*
+ * Allocate the top-level VMT. This can either be linear or two-level.
+ */
+int vgic_v5_vmt_allocate(unsigned int max_vpes)
+{
+ int ret;
+
+ /* Allocate the tracking structure */
+ vmt_info = kzalloc_obj(*vmt_info, GFP_KERNEL);
+ if (!vmt_info)
+ return -ENOMEM;
+
+ ida_init(&vmt_info->vm_id_ida);
+ vmt_info->max_vpes = max_vpes;
+ vmt_info->vmd_size = irs_caps.vmd_size;
+ vmt_info->vped_size = irs_caps.vped_size;
+ vmt_info->two_level = irs_caps.two_level_vmt_support;
+ vmt_info->num_entries = irs_caps.max_vms;
+
+ if (vmt_info->two_level)
+ ret = vgic_v5_alloc_vmt_two_level(vmt_info->num_entries);
+ else
+ ret = vgic_v5_alloc_vmt_linear(vmt_info->num_entries);
+
+ /* If anything failed, free our tracking structure before returning */
+ if (ret) {
+ kfree(vmt_info);
+ vmt_info = NULL;
+ }
+
+ return ret;
+}
+
+/*
+ * Free the VMT and associated tracking structures. This isn't strictly expected
+ * to be called in general operation, but instead exists for completeness.
+ */
+int vgic_v5_vmt_free(void)
+{
+ if (!vmt_info)
+ return 0;
+
+ if (!vmt_info->two_level) {
+ kfree(vmt_info->linear.vmt_base);
+ } else {
+ /* Free the L2 tables; kfree(NULL) is safe */
+ for (int i = 0; i < vmt_info->l2.num_l1_ents; ++i)
+ kfree(vmt_info->l2.l2ptrs[i]);
+ kfree(vmt_info->l2.l2ptrs);
+
+ /* And now free the L1 table */
+ kfree(vmt_info->l2.vmt_base);
+ }
+
+ ida_destroy(&vmt_info->vm_id_ida);
+ kfree(vmt_info);
+ vmt_info = NULL;
+
+ return 0;
+}
+
+/*
+ * Look up a VMT Entry by VM ID.
+ */
+static struct vmtl2_entry *vgic_v5_get_l2_vmte(u32 vm_id)
+{
+ unsigned int l1_index, l2_index;
+ struct vmtl2_entry *l2_table;
+
+ if (vm_id == VGIC_V5_VM_ID_INVAL)
+ return ERR_PTR(-EINVAL);
+
+ if (!vmt_info->two_level)
+ return &vmt_info->linear.vmt_base[vm_id];
+
+ l1_index = vm_id / GICV5_VMT_L2_TABLE_ENTRIES;
+ l2_index = vm_id % GICV5_VMT_L2_TABLE_ENTRIES;
+
+ if (l1_index >= vmt_info->l2.num_l1_ents)
+ return ERR_PTR(-E2BIG);
+
+ if (!vmt_info->l2.l2ptrs[l1_index])
+ return ERR_PTR(-EINVAL);
+
+ l2_table = vmt_info->l2.l2ptrs[l1_index];
+ return &l2_table[l2_index];
+}
+
+/*
+ * Zero a VMT Entry, and flush & invalidate to the PoC, if required.
+ */
+static int vgic_v5_reset_vmte(struct kvm *kvm)
+{
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vmtl2_entry *vmte;
+
+ vmte = vgic_v5_get_l2_vmte(vm_id);
+ if (IS_ERR(vmte))
+ return PTR_ERR(vmte);
+
+ /*
+ * The VMT is normal memory shared with the IRS. Invalidate before
+ * rewriting the entry so that cacheline-granular maintenance cannot
+ * later push stale data for neighbouring IRS-visible state back to
+ * memory.
+ */
+ vgic_v5_clean_inval(vmte, sizeof(*vmte));
+
+ /*
+ * Prevent the compiler from eliding the individual VMTE
+ * stores. Ordering and visibility to the IRS are provided by the
+ * surrounding cache maintenance and command protocol, not by
+ * WRITE_ONCE().
+ *
+ * The same compiler-access constraint applies to READ_ONCE() users in
+ * this file: when inspecting IRS-visible table entries, read the field
+ * exactly once and prevent the compiler from reusing, merging or
+ * tearing the access. Coherency and freshness for non-coherent IRSes
+ * still come from the surrounding cache maintenance.
+ */
+ WRITE_ONCE(vmte->val[0], cpu_to_le64(0ULL));
+ WRITE_ONCE(vmte->val[1], cpu_to_le64(0ULL));
+ WRITE_ONCE(vmte->val[2], cpu_to_le64(0ULL));
+ WRITE_ONCE(vmte->val[3], cpu_to_le64(0ULL));
+
+ /* And make our write visible to the IRS (if non-coherent) */
+ vgic_v5_clean_inval(vmte, sizeof(*vmte));
+
+ return 0;
+}
+
+/*
+ * Use the IDA to allocate a new VM ID, and track it in the gicv5_vm data
+ * structure. If we're out of VM IDs, the IDA catches that, and we return the
+ * error (-ENOSPC). If we've previously allocated a VM ID, we catch that too and
+ * return -EBUSY.
+ */
+int vgic_v5_allocate_vm_id(struct kvm *kvm)
+{
+ int id;
+
+ if (kvm->arch.vgic.gicv5_vm.vm_id != VGIC_V5_VM_ID_INVAL)
+ return -EBUSY;
+
+ id = ida_alloc_max(&vmt_info->vm_id_ida, vmt_info->num_entries - 1u,
+ GFP_KERNEL);
+ if (id < 0)
+ return id;
+
+ kvm->arch.vgic.gicv5_vm.vm_id = id;
+
+ return 0;
+}
+
+/*
+ * Release the VM ID to allow it to be reallocated in the future.
+ */
+void vgic_v5_release_vm_id(struct kvm *kvm)
+{
+ if (kvm->arch.vgic.gicv5_vm.vm_id == VGIC_V5_VM_ID_INVAL)
+ return;
+
+ ida_free(&vmt_info->vm_id_ida, kvm->arch.vgic.gicv5_vm.vm_id);
+ kvm->arch.vgic.gicv5_vm.vm_id = VGIC_V5_VM_ID_INVAL;
+}
+
+/*
+ * Initialise an entry in the VMT based on the index of the VM.
+ *
+ * Note: We don't mark the VMTE as valid as this needs to be done by
+ * the hardware.
+ */
+int vgic_v5_vmte_init(struct kvm *kvm)
+{
+ int nr_cpus = atomic_read(&kvm->online_vcpus);
+ struct vgic_v5_vm_info *vmi = NULL;
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vmtl2_entry *vmte;
+ void **vped_ptrs = NULL;
+ vpe_entry *vpet = NULL;
+ void *vmd = NULL;
+ int ret;
+ u64 tmp;
+
+ if (nr_cpus > vmt_info->max_vpes)
+ return -E2BIG;
+
+ /*
+ * If we're using two-level VMTs, L2 is allocated on demand. For linear
+ * VMTs, this is a NOP.
+ */
+ ret = vgic_v5_alloc_l2_vmt(kvm);
+ if (ret)
+ return ret;
+
+ vmte = vgic_v5_get_l2_vmte(vm_id);
+ if (IS_ERR(vmte))
+ return PTR_ERR(vmte);
+
+ /* If the entry is already valid, something went wrong */
+ vgic_v5_clean_inval(vmte, sizeof(*vmte));
+ if (le64_to_cpu(READ_ONCE(vmte->val[0])) & GICV5_VMTEL2E_VALID)
+ return -EINVAL;
+
+ ret = vgic_v5_reset_vmte(kvm);
+ if (ret)
+ return ret;
+
+ vmi = kzalloc_obj(*vmi);
+ if (!vmi) {
+ ret = -ENOMEM;
+ goto out_fail;
+ }
+
+ ret = xa_insert(&vm_info, vm_id, vmi, GFP_KERNEL);
+ if (ret)
+ goto out_fail;
+
+ /*
+ * If we are restoring the state of a guest, we need to re-inject any
+ * IRQs that were pending when the state of the guest was originally
+ * saved. We use the pending_irqs list for this.
+ */
+ INIT_LIST_HEAD(&vmi->pending_irqs);
+
+ /* Allocate and assign the VM Descriptor, if required. */
+ if (vmt_info->vmd_size != 0) {
+ vmd = kzalloc(vmt_info->vmd_size, GFP_KERNEL);
+ if (!vmd) {
+ ret = -ENOMEM;
+ goto out_fail;
+ }
+
+ /* Stash the VA so we can free it later */
+ vmi->vmd_base = vmd;
+
+ tmp = FIELD_PREP(GICV5_VMTEL2E_VMD_ADDR,
+ virt_to_phys(vmd) >> GICV5_VMTEL2E_VMD_ADDR_SHIFT);
+ WRITE_ONCE(vmte->val[0], cpu_to_le64(tmp));
+ }
+
+ /*
+ * Allocate and assign the VPE Table. Round up the number of CPUs to a
+ * whole power of two as we cannot describe non-powers-of-two in the
+ * VMTE field as it conveys the number of ID bits used and not the
+ * number of vPEs. IRS_IDR1.IAFFID_BITS is encoded as N - 1, so expose
+ * at least one VPE ID bit even for a single-vCPU VM to keep the views
+ * consistent.
+ */
+ nr_cpus = max(2UL, roundup_pow_of_two(nr_cpus));
+ vmi->vpe_id_bits = fls(nr_cpus) - 1;
+
+ vpet = kzalloc_objs(*vpet, nr_cpus);
+ if (!vpet) {
+ ret = -ENOMEM;
+ goto out_fail;
+ }
+
+ /* Stash the VA so we can free it later */
+ vmi->vpet_base = vpet;
+
+ tmp = FIELD_PREP(GICV5_VMTEL2E_VPET_ADDR,
+ virt_to_phys(vpet) >> GICV5_VMTEL2E_VPET_ADDR_SHIFT);
+ tmp |= FIELD_PREP(GICV5_VMTEL2E_VPE_ID_BITS, vmi->vpe_id_bits);
+ WRITE_ONCE(vmte->val[1], cpu_to_le64(tmp));
+
+ vped_ptrs = kzalloc_objs(*vped_ptrs, nr_cpus, GFP_KERNEL);
+ if (!vped_ptrs) {
+ ret = -ENOMEM;
+ goto out_fail;
+ }
+ vmi->vped_ptrs = vped_ptrs;
+
+ if (vmd)
+ vgic_v5_clean_inval(vmd, vmt_info->vmd_size);
+ vgic_v5_clean_inval(vpet, sizeof(*vpet) * nr_cpus);
+ vgic_v5_clean_inval(vmte, sizeof(*vmte));
+
+ kvm->arch.vgic.gicv5_vm.vmte_allocated = true;
+
+ return 0;
+
+out_fail:
+ /* kfree(NULL) is safe so we can just kfree() at leisure */
+ kfree(vmd);
+ kfree(vpet);
+ kfree(vped_ptrs);
+ if (vmi)
+ xa_erase(&vm_info, vm_id);
+ kfree(vmi);
+
+ vgic_v5_reset_vmte(kvm);
+
+ return ret;
+}
+
+/*
+ * Release the VMT Entry, freeing up any allocated data structures before
+ * zeroing the VMTE.
+ *
+ * The VMTE must be marked as invalid before it is released.
+ */
+int vgic_v5_vmte_release(struct kvm *kvm)
+{
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vgic_v5_vm_info *vmi;
+ struct vmtl2_entry *vmte;
+ int ret;
+
+ vmte = vgic_v5_get_l2_vmte(vm_id);
+ if (IS_ERR(vmte))
+ return PTR_ERR(vmte);
+
+ /* Reject if the VMTE has not been marked as invalid! */
+ vgic_v5_clean_inval(vmte, sizeof(*vmte));
+ if (le64_to_cpu(READ_ONCE(vmte->val[0])) & GICV5_VMTEL2E_VALID)
+ return -EINVAL;
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ goto no_vmi;
+
+ /* Unlikely, but possible. Avoid leaking the memory. */
+ vgic_v5_drain_pending_irqs(kvm, vmi, false);
+
+ /* If we have an LPI IST, free it */
+ if (vmi->h_lpi_ist) {
+ ret = vgic_v5_lpi_ist_free(kvm);
+ if (ret)
+ return ret;
+ }
+ vmi->h_lpi_ist = NULL;
+
+ /* If we have an SPI IST, free it */
+ if (vmi->h_spi_ist) {
+ ret = vgic_v5_spi_ist_free(kvm);
+ if (ret)
+ return ret;
+ }
+ vmi->h_spi_ist = NULL;
+
+ for (int i = 0; i < BIT(vmi->vpe_id_bits); i++)
+ kfree(vmi->vped_ptrs[i]);
+ kfree(vmi->vped_ptrs);
+ kfree(vmi->vpet_base);
+ kfree(vmi->vmd_base);
+
+ xa_erase(&vm_info, vm_id);
+ kfree(vmi);
+
+no_vmi:
+ /*
+ * If we didn't get far enough into allocating a VMTE to create the VM
+ * info structure, then we just zero the VMTE and move on. There's
+ * nothing else we can realistically do here.
+ */
+ ret = vgic_v5_reset_vmte(kvm);
+ if (ret)
+ return ret;
+
+ kvm->arch.vgic.gicv5_vm.vmte_allocated = false;
+
+ return 0;
+}
+
+/*
+ * Provide a way for the IRS MMIO emulation to correctly populate the number of
+ * IAFFID bits (which correspond to our vpe_id_bits.
+ */
+u8 vgic_v5_vmte_vpe_id_bits(struct kvm_vcpu *vcpu)
+{
+ u32 vm_id = vgic_v5_vm_id(vcpu->kvm);
+ struct vgic_v5_vm_info *vmi;
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ return 0;
+
+ return vmi->vpe_id_bits;
+}
+
+/*
+ * Allocate a VPE descriptor and provide it to the hardware via the VPE Table.
+ */
+int vgic_v5_vmte_alloc_vpe(struct kvm_vcpu *vcpu)
+{
+ u32 vm_id = vgic_v5_vm_id(vcpu->kvm);
+ u16 vpe_id = vgic_v5_vpe_id(vcpu);
+ struct vgic_v5_vm_info *vmi;
+ vpe_entry tmp, *vpet_base;
+ void *vped;
+
+ /* Make sure we're not over what the hardware supports */
+ if (vpe_id >= vmt_info->max_vpes)
+ return -E2BIG;
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ return -EINVAL;
+
+ if (vpe_id >= 1 << vmi->vpe_id_bits)
+ return -E2BIG;
+
+ vpet_base = vmi->vpet_base;
+
+ /* If the VPETE for this CPU is already valid we've gone wrong */
+ vgic_v5_clean_inval(&vpet_base[vpe_id], sizeof(*vpet_base));
+ if (le64_to_cpu(READ_ONCE(vpet_base[vpe_id])) & GICV5_VPE_VALID)
+ return -EBUSY;
+
+ /* Alloc VPE Descriptor. Only used by IRS. */
+ vped = kzalloc(vmt_info->vped_size, GFP_KERNEL);
+ if (!vped)
+ return -ENOMEM;
+
+ vmi->vped_ptrs[vpe_id] = vped;
+
+ tmp = FIELD_PREP(GICV5_VPED_ADDR, virt_to_phys(vped) >> GICV5_VPED_ADDR_SHIFT);
+ WRITE_ONCE(vpet_base[vpe_id], cpu_to_le64(tmp));
+
+ vgic_v5_clean_inval(vped, vmt_info->vped_size);
+ vgic_v5_clean_inval(vpet_base + vpe_id, sizeof(vpe_entry));
+
+ return 0;
+}
+
+/*
+ * Free the memory allocated for the VPE descriptor.
+ */
+int vgic_v5_vmte_free_vpe(struct kvm_vcpu *vcpu)
+{
+ u32 vm_id = vgic_v5_vm_id(vcpu->kvm);
+ u16 vpe_id = vgic_v5_vpe_id(vcpu);
+ struct vgic_v5_vm_info *vmi;
+ struct vmtl2_entry *vmte;
+ vpe_entry *vpet_base;
+ void *vped;
+
+ vmte = vgic_v5_get_l2_vmte(vm_id);
+ if (IS_ERR(vmte))
+ return PTR_ERR(vmte);
+
+ vgic_v5_clean_inval(vmte, sizeof(*vmte));
+ if (le64_to_cpu(READ_ONCE(vmte->val[0])) & GICV5_VMTEL2E_VALID)
+ return -EBUSY;
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ return -EINVAL;
+
+ if (vpe_id >= 1 << vmi->vpe_id_bits)
+ return -E2BIG;
+
+ vpet_base = vmi->vpet_base;
+ WRITE_ONCE(vpet_base[vpe_id], 0ULL);
+
+ vgic_v5_clean_inval(vpet_base + vpe_id, sizeof(vpe_entry));
+
+ /* Free VPE Descriptor. Only used by IRS. */
+ vped = vmi->vped_ptrs[vpe_id];
+ vmi->vped_ptrs[vpe_id] = NULL;
+ kfree(vped);
+
+ return 0;
+}
+
+phys_addr_t vgic_v5_get_vmt_base(void)
+{
+ phys_addr_t vmt_base;
+
+ if (!vmt_info->two_level)
+ vmt_base = virt_to_phys(vmt_info->linear.vmt_base);
+ else
+ vmt_base = virt_to_phys(vmt_info->l2.vmt_base);
+
+ return vmt_base;
+}
+
+/*
+ * Assign an already allocated IST to the VM by populating the fields in the
+ * corresponding VMTE. We re-use this code for both an SPI IST and LPI IST, even
+ * if the paths to reach it might be vastly different.
+ */
+static int vgic_v5_vmte_assign_ist(struct kvm *kvm, phys_addr_t ist_base,
+ bool two_level, unsigned int id_bits,
+ unsigned int l2sz, unsigned int istsz,
+ bool spi_ist)
+{
+ struct kvm_vcpu *vcpu0 = kvm_get_vcpu(kvm, 0);
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ enum gicv5_vcpu_cmd cmd;
+ struct vmtl2_entry *vmte;
+ unsigned int section;
+ u64 tmp;
+ int ret;
+
+ /*
+ * The L2 VMTE comprises four 64-bit "sections", where sections 2 & 3
+ * describe the LPI and SPI ISTs, respectively. Both the LPI and SPI
+ * sections have the same layout, and as we are either operating on SPIs
+ * or LPIs we pick a section of the VMTE to modify up-front.
+ *
+ * See the GICv5 EAC0 Specification 11.2.2 for more details about the
+ * VMTE layout.
+ */
+ section = spi_ist ? GICV5_VMTEL2_SPI_SECTION : GICV5_VMTEL2_LPI_SECTION;
+
+ if (ist_base & ~GICV5_VMTEL2E_IST_ADDR) {
+ kvm_err("IST alignment issue! Address: 0x%llx, Mask 0x%llx\n",
+ ist_base, GICV5_VMTEL2E_IST_ADDR);
+ return -EINVAL;
+ }
+
+ vmte = vgic_v5_get_l2_vmte(vm_id);
+ if (IS_ERR(vmte))
+ return PTR_ERR(vmte);
+
+ /* Bail if already allocated */
+ vgic_v5_clean_inval(vmte, sizeof(*vmte));
+ if (le64_to_cpu(READ_ONCE(vmte->val[section])) & GICV5_VMTEL2E_IST_VALID)
+ return -EINVAL;
+
+ tmp = FIELD_PREP(GICV5_VMTEL2E_IST_L2SZ, l2sz);
+ tmp |= FIELD_PREP(GICV5_VMTEL2E_IST_ADDR,
+ ist_base >> GICV5_VMTEL2E_IST_ADDR_SHIFT);
+ tmp |= FIELD_PREP(GICV5_VMTEL2E_IST_ISTSZ, istsz);
+ tmp |= FIELD_PREP(GICV5_VMTEL2E_IST_ID_BITS, id_bits);
+ if (two_level)
+ tmp |= GICV5_VMTEL2E_IST_STRUCTURE;
+
+ WRITE_ONCE(vmte->val[section], cpu_to_le64(tmp));
+ vgic_v5_clean_inval(vmte, sizeof(*vmte));
+
+ /* Finally, mark the entry as valid */
+ cmd = spi_ist ? SPI_VIST_MAKE_VALID : LPI_VIST_MAKE_VALID;
+ ret = irq_set_vcpu_affinity(vgic_v5_vpe_db(vcpu0), &cmd);
+ if (ret) {
+ WRITE_ONCE(vmte->val[section], 0ULL);
+ vgic_v5_clean_inval(vmte, sizeof(*vmte));
+ return ret;
+ }
+
+ return 0;
+}
+
+/*
+ * Allocate a Linear IST - always used for SPIs and potentially LPIs.
+ *
+ * The calculation for n has been taken from section 11.2.2 of the GICv5 EAC0
+ * spec.
+ *
+ * NOTE: istsz is the FIELD used by GICv5, not the actual size (or log2() of the
+ * size).
+ */
+static int vgic_v5_alloc_linear_ist(struct kvm *kvm, bool spi_ist,
+ unsigned int id_bits, unsigned int istsz)
+{
+ const size_t n = id_bits + 1 + istsz;
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vgic_v5_vm_info *vmi;
+ __le64 *ist;
+ u32 l1sz;
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ return -EINVAL;
+
+ /*
+ * Allocate the IST. We only have one level, so we just use the L2 ISTE.
+ */
+ l1sz = BIT(n + 1);
+ ist = kzalloc(l1sz, GFP_KERNEL_ACCOUNT);
+ if (!ist)
+ return -ENOMEM;
+
+ if (spi_ist) {
+ vmi->h_spi_ist = ist;
+ } else {
+ vmi->h_lpi_ist_structure = false;
+ vmi->h_lpi_ist = ist;
+ }
+
+ vgic_v5_clean_inval(ist, l1sz);
+
+ return 0;
+}
+
+/*
+ * Allocate the first level of a two-level IST - LPI, only.
+ *
+ * The calculation for n has been taken from section 11.2.2 of the GICv5 EAC0
+ * spec.
+ *
+ * NOTE: istsz and l2sz are the FIELDS used by GICv5, not the actual sizes (or
+ * log2() of the sizes).
+ */
+static int vgic_v5_alloc_l1_ist(struct kvm *kvm, unsigned int id_bits,
+ unsigned int istsz, unsigned int l2sz)
+{
+ const size_t n = max(5, id_bits - ((10 - istsz) + (2 * l2sz)) + 3 - 1);
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ const u32 l1_size = BIT(n + 1);
+ struct vgic_v5_vm_info *vmi;
+ __le64 *ist;
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ return -EINVAL;
+
+ ist = kzalloc(l1_size, GFP_KERNEL_ACCOUNT);
+ if (!ist)
+ return -ENOMEM;
+
+ vmi->h_lpi_ist_structure = true;
+ vmi->h_lpi_ist = ist;
+
+ vgic_v5_clean_inval(ist, l1_size);
+
+ return 0;
+}
+
+/*
+ * Allocate ALL of the second level ISTs for a two-level IST - LPI, only.
+ *
+ * The calculation for n has been taken from section 11.2.2 of the GICv5 EAC0
+ * spec. The l2_size calculation is from section 11.2.3 of the same document.
+ *
+ * NOTE: istsz and l2sz are the FIELDS used by GICv5, not the actual sizes (or
+ * log2() of the sizes).
+ */
+static int vgic_v5_alloc_l2_ists(struct kvm *kvm, unsigned int id_bits,
+ unsigned int istsz, unsigned int l2sz)
+{
+ const size_t n = max(5, id_bits - ((10 - istsz) + (2 * l2sz)) + 3 - 1);
+ const int l1_entries = BIT(n + 1) / GICV5_IRS_ISTL1E_SIZE;
+ const size_t l2_size = BIT(11 + (2 * l2sz) + 1);
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vgic_v5_vm_info *vmi;
+ __le64 *l2ist;
+ __le64 *l1ist;
+ int index;
+ u64 val;
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ return -EINVAL;
+
+ l1ist = vmi->h_lpi_ist;
+
+ /*
+ * Allocate the storage for the pointers to the L2 ISTs (used when
+ * freeing later).
+ */
+ vmi->h_lpi_l2_ists = kzalloc_objs(*vmi->h_lpi_l2_ists, l1_entries,
+ GFP_KERNEL_ACCOUNT);
+ if (!vmi->h_lpi_l2_ists)
+ return -ENOMEM;
+
+ /* Allocate the L2 IST for each L1 IST entry */
+ for (index = 0; index < l1_entries; ++index) {
+ l2ist = kzalloc(l2_size, GFP_KERNEL_ACCOUNT);
+ if (!l2ist) {
+ while (--index >= 0)
+ kfree(vmi->h_lpi_l2_ists[index]);
+
+ kfree(vmi->h_lpi_l2_ists);
+ vmi->h_lpi_l2_ists = NULL;
+
+ return -ENOMEM;
+ }
+
+ /*
+ * We are not doing on-demand allocation of the L2 ISTs, and are
+ * instead provisioning the whole IST up front. This means that
+ * we are able to mark the L2 ISTs as valid in the L1 ISTEs as
+ * the overall IST is not yet valid.
+ */
+ val = (virt_to_phys(l2ist) & GICV5_ISTL1E_L2_ADDR_MASK) |
+ GICV5_ISTL1E_VALID;
+ l1ist[index] = cpu_to_le64(val);
+
+ vmi->h_lpi_l2_ists[index] = l2ist;
+
+ vgic_v5_clean_inval(l2ist, l2_size);
+ }
+
+ /* Handle CMOs for the whole L1 IST in one go */
+ vgic_v5_clean_inval(l1ist, l1_entries * sizeof(*l1ist));
+
+ return 0;
+}
+
+/* Allocate a two-level IST - LPIs, only */
+static int vgic_v5_alloc_two_level_lpi_ist(struct kvm *kvm, unsigned int id_bits,
+ unsigned int istsz, unsigned int l2sz)
+{
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vgic_v5_vm_info *vmi;
+ int ret;
+
+ /*
+ * Allocate the L1 IST first, then all of the L2s. Everything
+ * is preallocated and we do no on-demand IST allocation. This
+ * is to avoid needing to track if and when the guest is doing
+ * on-demand IST allocation.
+ */
+ ret = vgic_v5_alloc_l1_ist(kvm, id_bits, istsz, l2sz);
+ if (ret)
+ return ret;
+
+ ret = vgic_v5_alloc_l2_ists(kvm, id_bits, istsz, l2sz);
+ if (ret) {
+ /* Free the L1 IST again */
+ vmi = xa_load(&vm_info, vm_id);
+ kfree(vmi->h_lpi_ist);
+ vmi->h_lpi_ist = 0;
+
+ return ret;
+ }
+
+ return 0;
+}
+
+static void vgic_v5_free_allocated_lpi_ist(struct vgic_v5_vm_info *vmi,
+ unsigned int id_bits,
+ unsigned int istsz,
+ unsigned int l2sz)
+{
+ if (!vmi->h_lpi_ist_structure) {
+ kfree(vmi->h_lpi_ist);
+ vmi->h_lpi_ist = NULL;
+ return;
+ }
+
+ if (vmi->h_lpi_l2_ists) {
+ const size_t n = max(5, id_bits - ((10 - istsz) + (2 * l2sz)) + 3 - 1);
+ const int l1_entries = BIT(n + 1) / GICV5_IRS_ISTL1E_SIZE;
+ int index;
+
+ for (index = 0; index < l1_entries; ++index)
+ kfree(vmi->h_lpi_l2_ists[index]);
+
+ kfree(vmi->h_lpi_l2_ists);
+ vmi->h_lpi_l2_ists = NULL;
+ }
+
+ kfree(vmi->h_lpi_ist);
+ vmi->h_lpi_ist = NULL;
+}
+
+static void vgic_v5_free_allocated_spi_ist(struct kvm *kvm)
+{
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vgic_v5_vm_info *vmi;
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ return;
+
+ kfree(vmi->h_spi_ist);
+ vmi->h_spi_ist = NULL;
+}
+
+/*
+ * Free a Linear IST. Can only happen once the VM is dead.
+ */
+static int vgic_v5_linear_ist_free(struct kvm *kvm, bool spi)
+{
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vmtl2_entry *vmte;
+ struct vgic_v5_vm_info *vmi;
+ int section;
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ return -EINVAL;
+
+ vmte = vgic_v5_get_l2_vmte(vm_id);
+ if (IS_ERR(vmte))
+ return PTR_ERR(vmte);
+
+ if (spi) {
+ section = GICV5_VMTEL2_SPI_SECTION;
+ vgic_v5_free_allocated_spi_ist(kvm);
+ } else {
+ section = GICV5_VMTEL2_LPI_SECTION;
+ vgic_v5_free_allocated_lpi_ist(vmi, 0, 0, 0);
+ }
+
+ /* The VM should be dead here, so we can just zero the VMT section */
+ vmte->val[section] = cpu_to_le64(0);
+ vgic_v5_clean_inval(vmte, sizeof(*vmte));
+
+ return 0;
+}
+
+/*
+ * Free a Two-Level IST. Can only happen once the VM is dead.
+ */
+static int vgic_v5_two_level_ist_free(struct kvm *kvm, bool spi)
+{
+ unsigned int id_bits, istsz, l2sz;
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vgic_v5_vm_info *vmi;
+ struct vmtl2_entry *vmte;
+ u64 tmp;
+ int section;
+
+ /* We don't create two-level SPI ISTs, so freeing is a bad idea! */
+ if (spi)
+ return -EINVAL;
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ return -EINVAL;
+
+ section = GICV5_VMTEL2_LPI_SECTION;
+
+ if (!vmi->h_lpi_ist_structure)
+ return -EINVAL;
+
+ vmte = vgic_v5_get_l2_vmte(vm_id);
+ if (IS_ERR(vmte))
+ return PTR_ERR(vmte);
+
+ tmp = le64_to_cpu(READ_ONCE(vmte->val[section]));
+
+ id_bits = FIELD_GET(GICV5_VMTEL2E_IST_ID_BITS, tmp);
+ istsz = FIELD_GET(GICV5_VMTEL2E_IST_ISTSZ, tmp);
+ l2sz = FIELD_GET(GICV5_VMTEL2E_IST_L2SZ, tmp);
+
+ vgic_v5_free_allocated_lpi_ist(vmi, id_bits, istsz, l2sz);
+
+ /* The VM must be dead, so we can just zero the VMT section */
+ vmte->val[section] = cpu_to_le64(0);
+ vgic_v5_clean_inval(vmte, sizeof(*vmte));
+
+ return 0;
+}
+
+/* Helper to determine ISTE size based on metadata requirements */
+static unsigned int vgic_v5_ist_istsz(unsigned int id_bits)
+{
+ if (!irs_caps.istmd)
+ return GICV5_IRS_IST_CFGR_ISTSZ_4;
+
+ if (id_bits >= irs_caps.istmd_sz)
+ return GICV5_IRS_IST_CFGR_ISTSZ_16;
+
+ return GICV5_IRS_IST_CFGR_ISTSZ_8;
+}
+
+/*
+ * Allocate an IST for SPIs.
+ *
+ * We don't anticipate a large number of SPIs being allocated. Therefore, we
+ * always allocate a Linear IST for SPIs. This will need to be revisited should
+ * that assumption no longer hold.
+ */
+int vgic_v5_spi_ist_alloc(struct kvm *kvm, unsigned int id_bits)
+{
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vgic_v5_vm_info *vmi;
+ phys_addr_t base_addr;
+ unsigned int istsz;
+ int ret;
+
+ lockdep_assert_held(&kvm->arch.config_lock);
+
+ istsz = vgic_v5_ist_istsz(id_bits);
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ return -EINVAL;
+
+ if (vmi->h_spi_ist)
+ return -EBUSY;
+
+ ret = vgic_v5_alloc_linear_ist(kvm, true, id_bits, istsz);
+ if (ret)
+ return ret;
+ base_addr = virt_to_phys(vmi->h_spi_ist);
+
+ ret = vgic_v5_vmte_assign_ist(kvm, base_addr, false, id_bits, 0, istsz,
+ true);
+ if (ret) {
+ vgic_v5_free_allocated_spi_ist(kvm);
+ return ret;
+ }
+
+ return 0;
+}
+
+/*
+ * Free the IST for SPIs. Should only happen once the VM is dead.
+ */
+static int vgic_v5_spi_ist_free(struct kvm *kvm)
+{
+ lockdep_assert_held(&kvm->arch.config_lock);
+
+ return vgic_v5_linear_ist_free(kvm, true);
+}
+
+int vgic_v5_lpi_ist_exists(struct kvm *kvm)
+{
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vgic_v5_vm_info *vmi;
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ return -ENXIO;
+
+ return !!vmi->h_lpi_ist;
+}
+
+/*
+ * Allocate an IST for LPIs.
+ *
+ * Unlike with SPIs, we anticipate that the guest will allocate a relatively
+ * large number of LPIs. Therefore, while we support doing a linear LPI IST, it
+ * is expected that LPI ISTs will be two-level.
+ */
+int vgic_v5_lpi_ist_alloc(struct kvm *kvm, unsigned int id_bits)
+{
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vgic_v5_vm_info *vmi;
+ unsigned int istsz, l2sz;
+ phys_addr_t phys_addr;
+ bool two_level;
+ int ret;
+
+ lockdep_assert_held(&kvm->arch.config_lock);
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ return -EINVAL;
+
+ if (vmi->h_lpi_ist)
+ return -EBUSY;
+
+ istsz = vgic_v5_ist_istsz(id_bits);
+ l2sz = gicv5_irs_l2_sz(irs_caps.ist_l2sz);
+
+ /*
+ * Determine if we want to create a Linear or a Two-Level IST.
+ *
+ * A two-level IST is only required when a single L2 IST cannot cover
+ * the requested ID space. This depends on the L2 IST size selected for
+ * the IRS, not PAGE_SIZE. Using PAGE_SIZE here would switch to
+ * two-level too early when the selected L2 IST is larger than a page,
+ * and the allocation sizing arithmetic would underflow.
+ */
+ two_level = irs_caps.ist_levels &&
+ id_bits > ((10 - istsz) + (2 * l2sz));
+
+ if (!two_level)
+ ret = vgic_v5_alloc_linear_ist(kvm, false /* LPIs, not SPIs */,
+ id_bits, istsz);
+ else
+ ret = vgic_v5_alloc_two_level_lpi_ist(kvm, id_bits, istsz,
+ l2sz);
+
+ if (ret)
+ return ret;
+
+ phys_addr = virt_to_phys(vmi->h_lpi_ist);
+ ret = vgic_v5_vmte_assign_ist(kvm, phys_addr, two_level, id_bits, l2sz,
+ istsz, false);
+ if (ret)
+ vgic_v5_free_allocated_lpi_ist(vmi, id_bits, istsz, l2sz);
+
+ return ret;
+}
+
+/* Free the LPI IST again */
+int vgic_v5_lpi_ist_free(struct kvm *kvm)
+{
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vgic_v5_vm_info *vmi;
+
+ lockdep_assert_held(&kvm->arch.config_lock);
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ return -ENXIO;
+
+ if (!vmi->h_lpi_ist_structure)
+ return vgic_v5_linear_ist_free(kvm, false);
+ else
+ return vgic_v5_two_level_ist_free(kvm, false);
+}
+
+static struct vgic_v5_two_level_ist_shape
+vgic_v5_two_level_ist_shape(const struct vgic_v5_ist_desc *ist)
+{
+ struct vgic_v5_two_level_ist_shape shape;
+ size_t l2bits, n;
+
+ l2bits = (10 - ist->istsz) + (2 * ist->l2sz);
+ n = max(2, ist->id_bits - l2bits + 3 - 1);
+
+ shape.l1_entries = BIT(n + 1) / GICV5_IRS_ISTL1E_SIZE;
+ shape.l2_entries = BIT(l2bits);
+
+ return shape;
+}
+
+static int vgic_v5_read_vm_ist_desc(struct kvm *kvm, unsigned int section,
+ struct vgic_v5_ist_desc *ist)
+{
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vmtl2_entry *vmte;
+ u64 vmte_ist_section;
+
+ vmte = vgic_v5_get_l2_vmte(vm_id);
+ if (IS_ERR(vmte))
+ return PTR_ERR(vmte);
+
+ vgic_v5_clean_inval(vmte, sizeof(*vmte));
+ vmte_ist_section = le64_to_cpu(READ_ONCE(vmte->val[section]));
+
+ ist->id_bits = FIELD_GET(GICV5_VMTEL2E_IST_ID_BITS, vmte_ist_section);
+ ist->istsz = FIELD_GET(GICV5_VMTEL2E_IST_ISTSZ, vmte_ist_section);
+ ist->l2sz = FIELD_GET(GICV5_VMTEL2E_IST_L2SZ, vmte_ist_section);
+ ist->iste_size = GICV5_ISTE_SIZE(ist->istsz);
+
+ return !!(vmte_ist_section & GICV5_VMTEL2E_IST_VALID);
+}
+
+static int vgic_v5_get_spi_ist_desc(struct kvm *kvm, bool userspace_buf,
+ struct vgic_v5_ist_desc *ist)
+{
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ int ret;
+
+ memset(ist, 0, sizeof(*ist));
+
+ ist->vmi = xa_load(&vm_info, vm_id);
+ if (!ist->vmi)
+ return -ENXIO;
+
+ ret = vgic_v5_read_vm_ist_desc(kvm, GICV5_VMTEL2_SPI_SECTION, ist);
+ if (ret < 0)
+ return ret;
+
+ ist->base = ist->vmi->h_spi_ist;
+
+ /* We don't have SPIs, but userspace is trying to save/restore them. */
+ if (!ist->base && userspace_buf)
+ return -ENOENT;
+
+ /* We have SPIs but userspace isn't trying to save/restore them. */
+ if (ist->base && !userspace_buf)
+ return -EINVAL;
+
+ /* No SPIs and no userspace buffer: nothing to do. */
+ if (!ist->base && !userspace_buf)
+ return 0;
+
+ ist->present = true;
+ return 0;
+}
+
+static int vgic_v5_get_lpi_ist_desc(struct kvm *kvm,
+ struct vgic_v5_ist_desc *ist)
+{
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ bool guest_valid, host_valid;
+ int ret;
+
+ memset(ist, 0, sizeof(*ist));
+
+ ist->vmi = xa_load(&vm_info, vm_id);
+ if (WARN_ON_ONCE(!ist->vmi))
+ return -ENXIO;
+
+ ret = vgic_v5_read_vm_ist_desc(kvm, GICV5_VMTEL2_LPI_SECTION, ist);
+ if (ret < 0)
+ return ret;
+
+ host_valid = ret;
+ guest_valid = kvm->arch.vgic.vgic_v5_irs_data->ist_baser.valid;
+ ist->base = ist->vmi->h_lpi_ist;
+
+ /* If there is no IST to save/restore, return without error. */
+ if (!guest_valid && !host_valid && !ist->base)
+ return 0;
+
+ /* Mismatched combination of valid state */
+ if (!guest_valid || !host_valid || !ist->base)
+ return -ENXIO;
+
+ if (ist->vmi->h_lpi_ist_structure && !ist->vmi->h_lpi_l2_ists)
+ return -ENXIO;
+
+ ist->present = true;
+ return 0;
+}
+
+/*
+ * Save a linear host IST to userspace memory.
+ *
+ * Only the architected 32-bit ISTE state is stored. Metadata is skipped when
+ * striding through the host IST.
+ */
+static int vgic_v5_save_linear_ist(const struct vgic_v5_ist_desc *ist,
+ u32 __user *uaddr, size_t nr_entries)
+{
+ __le32 h_iste;
+ size_t index;
+ int ret;
+
+ vgic_v5_clean_inval(ist->base,
+ GICV5_LINEAR_IST_SIZE(ist->id_bits, ist->istsz));
+
+ for (index = 0; index < nr_entries; index++) {
+ __le32 *h_iste_addr = ist->base + index * ist->iste_size;
+
+ h_iste = READ_ONCE(*h_iste_addr);
+ ret = put_user(h_iste, uaddr);
+ if (ret)
+ return ret;
+
+ uaddr++;
+ }
+
+ return 0;
+}
+
+/*
+ * Save a two-level host IST to userspace memory.
+ *
+ * Only the architected 32-bit ISTE state is stored. Metadata is skipped when
+ * striding through the host IST.
+ */
+static int vgic_v5_save_two_level_ist(const struct vgic_v5_ist_desc *ist,
+ u32 __user *uaddr)
+{
+ struct vgic_v5_two_level_ist_shape shape;
+ size_t h_l1_index, h_l2_index;
+ void *h_l2_ist_base;
+ __le32 h_iste;
+ int ret;
+
+ shape = vgic_v5_two_level_ist_shape(ist);
+
+ vgic_v5_clean_inval(ist->base,
+ shape.l1_entries * sizeof(*ist->vmi->h_lpi_ist));
+
+ for (h_l1_index = 0; h_l1_index < shape.l1_entries; h_l1_index++) {
+ u64 l1_iste;
+
+ /*
+ * Host L2 ISTs are preallocated. Any invalid L1 entry means the
+ * host IST state is inconsistent.
+ */
+ l1_iste = le64_to_cpu(READ_ONCE(ist->vmi->h_lpi_ist[h_l1_index]));
+ if (!FIELD_GET(GICV5_ISTL1E_VALID, l1_iste))
+ return -ENXIO;
+
+ h_l2_ist_base = ist->vmi->h_lpi_l2_ists[h_l1_index];
+ if (!h_l2_ist_base)
+ return -ENXIO;
+
+ vgic_v5_clean_inval(h_l2_ist_base,
+ shape.l2_entries * ist->iste_size);
+
+ for (h_l2_index = 0; h_l2_index < shape.l2_entries; h_l2_index++) {
+ h_iste = *(__le32 *)(h_l2_ist_base +
+ h_l2_index * ist->iste_size);
+
+ ret = put_user(h_iste, uaddr);
+ if (ret)
+ return ret;
+
+ uaddr++;
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * Save the SPI IST to userspace-provided memory.
+ */
+int vgic_v5_save_spi_ist(struct kvm *kvm, struct kvm_vgic_v5_ist *ist_attr)
+{
+ struct vgic_v5_ist_desc ist;
+ u32 __user *uaddr;
+ int ret;
+
+ ret = vgic_v5_get_spi_ist_desc(kvm, !!ist_attr->spi_ist_addr, &ist);
+ if (ret || !ist.present)
+ return ret;
+
+ uaddr = (u32 __user *)(unsigned long)ist_attr->spi_ist_addr;
+
+ /* The host SPI IST is always linear. */
+ return vgic_v5_save_linear_ist(&ist, uaddr,
+ kvm->arch.vgic.nr_spis);
+}
+
+/*
+ * Save the LPI IST to userspace memory.
+ *
+ * The LPI IST may be linear or two-level, so host iteration depends on the
+ * allocated host shape.
+ */
+int vgic_v5_save_lpi_ist(struct kvm *kvm, struct kvm_vgic_v5_ist *ist_attr)
+{
+ struct vgic_v5_ist_desc ist;
+ u32 __user *uaddr;
+ int ret;
+
+ ret = vgic_v5_get_lpi_ist_desc(kvm, &ist);
+ if (ret)
+ return ret;
+
+ if (!ist.present)
+ return 0;
+
+ uaddr = (u32 __user *)(unsigned long)ist_attr->lpi_ist_addr;
+
+ if (!ist.vmi->h_lpi_ist_structure)
+ return vgic_v5_save_linear_ist(&ist, uaddr,
+ BIT(ist.id_bits));
+
+ return vgic_v5_save_two_level_ist(&ist, uaddr);
+}
+
+/*
+ * Track any SPIs and LPIs that were marked as pending at the point where the
+ * IST was restored.
+ *
+ * Restored pending state is cleared from the host ISTE and replayed with VDPEND
+ * before the VM first runs.
+ */
+static int vgic_v5_track_pending_irq(struct list_head *pending_irqs, u32 intid,
+ u32 type)
+{
+ struct vgic_v5_pending_irq *pirq;
+
+ pirq = kzalloc_obj(*pirq, GFP_KERNEL);
+ if (!pirq)
+ return -ENOMEM;
+
+ /* Encode the interrupt as a GICv5 IntID. */
+ pirq->irq = FIELD_PREP(GICV5_HWIRQ_TYPE, type) |
+ FIELD_PREP(GICV5_HWIRQ_ID, intid);
+
+ INIT_LIST_HEAD(&pirq->next);
+ list_add_tail(&pirq->next, pending_irqs);
+
+ return 0;
+}
+
+/*
+ * Process and sanitise each restored ISTE.
+ *
+ * HWU is for hardware use and must not survive migration. Pending state is
+ * tracked, cleared from the ISTE, and replayed before the VM first runs.
+ */
+static int vgic_v5_process_iste(__le32 *iste, struct list_head *pending_irqs,
+ u32 intid, u32 type)
+{
+ u32 iste_data = le32_to_cpu(READ_ONCE(*iste));
+ int ret;
+
+ /* Pending state is replayed later with VDPEND. */
+ if (iste_data & GICV5_ISTL2E_PENDING) {
+ ret = vgic_v5_track_pending_irq(pending_irqs, intid, type);
+ if (ret)
+ return ret;
+ }
+
+ iste_data &= ~GICV5_ISTL2E_PENDING;
+ iste_data &= ~GICV5_ISTL2E_HWU;
+
+ WRITE_ONCE(*iste, cpu_to_le32(iste_data));
+
+ return 0;
+}
+
+static void vgic_v5_restore_spi_config(struct kvm *kvm, __le32 iste, u32 spi)
+{
+ u32 iste_data = le32_to_cpu(iste);
+ bool pending = iste_data & GICV5_ISTL2E_PENDING;
+ struct vgic_irq *irq;
+ unsigned long flags;
+
+ irq = vgic_get_irq(kvm, vgic_v5_make_spi(spi));
+ if (WARN_ON_ONCE(!irq))
+ return;
+
+ raw_spin_lock_irqsave(&irq->irq_lock, flags);
+
+ if (iste_data & GICV5_ISTL2E_HM)
+ irq->config = VGIC_CONFIG_LEVEL;
+ else
+ irq->config = VGIC_CONFIG_EDGE;
+
+ if (irq->config == VGIC_CONFIG_EDGE)
+ irq->pending_latch = pending;
+ else if (pending)
+ irq->pending_latch = true;
+ else if (!irq->active)
+ irq->pending_latch = false;
+
+ raw_spin_unlock_irqrestore(&irq->irq_lock, flags);
+ vgic_put_irq(kvm, irq);
+}
+
+static int vgic_v5_restore_ist_entry(struct kvm *kvm,
+ const struct vgic_v5_ist_desc *ist,
+ void *h_iste_addr, __le32 h_iste,
+ u32 intid, u32 intid_type)
+{
+ __le32 raw_iste = h_iste;
+ int ret;
+
+ /*
+ * Sanitise the IST, clearing HWU & pending fields. Pending state is
+ * later replayed via GIC VDPEND.
+ */
+ ret = vgic_v5_process_iste(&h_iste, &ist->vmi->pending_irqs,
+ intid, intid_type);
+ if (ret)
+ return ret;
+
+ if (intid_type == GICV5_HWIRQ_TYPE_SPI)
+ vgic_v5_restore_spi_config(kvm, raw_iste, intid);
+
+ /*
+ * Zero the full ISTE (incl metadata), and write back the non-metadata
+ * region, only.
+ */
+ memset(h_iste_addr, 0, ist->iste_size);
+ WRITE_ONCE(*(__le32 *)h_iste_addr, h_iste);
+ vgic_v5_clean_inval(h_iste_addr, ist->iste_size);
+
+ return 0;
+}
+
+/*
+ * Restore a userspace IST image to a linear host IST.
+ *
+ * The userspace IST image is a linear array of 32-bit ISTEs.
+ */
+static int vgic_v5_restore_linear_ist(struct kvm *kvm,
+ const struct vgic_v5_ist_desc *ist,
+ u32 __user *uaddr, size_t nr_entries,
+ u32 intid_type)
+{
+ __le32 h_iste;
+ size_t index;
+ int ret;
+
+ for (index = 0; index < nr_entries; index++) {
+ void *h_iste_addr = ist->base + index * ist->iste_size;
+
+ ret = get_user(h_iste, uaddr);
+ if (ret)
+ return ret;
+
+ ret = vgic_v5_restore_ist_entry(kvm, ist, h_iste_addr,
+ h_iste, index, intid_type);
+ if (ret)
+ return ret;
+
+ uaddr++;
+ }
+
+ return 0;
+}
+
+/*
+ * Restore a userspace IST image to a two-level host IST.
+ *
+ * The userspace IST image is a linear array of 32-bit ISTEs.
+ */
+static int vgic_v5_restore_two_level_ist(struct kvm *kvm,
+ const struct vgic_v5_ist_desc *ist,
+ u32 __user *uaddr, u32 intid_type)
+{
+ struct vgic_v5_two_level_ist_shape shape;
+ size_t h_l1_index, h_l2_index;
+ void *h_l2_ist_base;
+ __le32 h_iste;
+ int ret;
+
+ shape = vgic_v5_two_level_ist_shape(ist);
+
+ vgic_v5_clean_inval(ist->vmi->h_lpi_ist,
+ shape.l1_entries * sizeof(*ist->vmi->h_lpi_ist));
+
+ for (h_l1_index = 0; h_l1_index < shape.l1_entries; ++h_l1_index) {
+ u64 l1_iste;
+
+ /*
+ * Host L2 ISTs are preallocated. Any invalid L1 entry means the
+ * host IST state is inconsistent.
+ */
+ l1_iste = le64_to_cpu(READ_ONCE(ist->vmi->h_lpi_ist[h_l1_index]));
+ if (!FIELD_GET(GICV5_ISTL1E_VALID, l1_iste))
+ return -ENXIO;
+
+ h_l2_ist_base = ist->vmi->h_lpi_l2_ists[h_l1_index];
+ if (!h_l2_ist_base)
+ return -ENXIO;
+
+ for (h_l2_index = 0; h_l2_index < shape.l2_entries; h_l2_index++) {
+ void *h_iste_addr = h_l2_ist_base +
+ h_l2_index * ist->iste_size;
+ u32 intid = h_l1_index * shape.l2_entries + h_l2_index;
+
+ ret = get_user(h_iste, uaddr);
+ if (ret)
+ return ret;
+
+ ret = vgic_v5_restore_ist_entry(kvm, ist, h_iste_addr,
+ h_iste, intid,
+ intid_type);
+ if (ret)
+ return ret;
+
+ uaddr++;
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * Restore the SPI IST from userspace-provided buffer to the host-allocated IST.
+ */
+int vgic_v5_restore_spi_ist(struct kvm *kvm, struct kvm_vgic_v5_ist *ist_attr)
+{
+ u32 __user *uaddr;
+ struct vgic_v5_ist_desc ist;
+ int ret;
+
+ ret = vgic_v5_get_spi_ist_desc(kvm, !!ist_attr->spi_ist_addr, &ist);
+ if (ret || !ist.present)
+ return ret;
+
+ uaddr = (u32 __user *)(unsigned long)ist_attr->spi_ist_addr;
+
+ /* The host SPI IST is always linear. */
+ return vgic_v5_restore_linear_ist(kvm, &ist, uaddr,
+ kvm->arch.vgic.nr_spis,
+ GICV5_HWIRQ_TYPE_SPI);
+}
+
+/*
+ * Restore the LPI IST from userspace memory to the host-allocated LPI IST.
+ *
+ * The host LPI IST may be linear or two-level, so host iteration depends on the
+ * host IST's shape.
+ */
+int vgic_v5_restore_lpi_ist(struct kvm *kvm, struct kvm_vgic_v5_ist *ist_attr)
+{
+ u32 __user *uaddr;
+ struct vgic_v5_ist_desc ist;
+ int ret;
+
+ ret = vgic_v5_get_lpi_ist_desc(kvm, &ist);
+ if (ret)
+ return ret;
+
+ if (!ist.present)
+ return 0;
+
+ uaddr = (u32 __user *)(unsigned long)ist_attr->lpi_ist_addr;
+
+ if (!ist.vmi->h_lpi_ist_structure)
+ return vgic_v5_restore_linear_ist(kvm, &ist, uaddr,
+ BIT(ist.id_bits),
+ GICV5_HWIRQ_TYPE_LPI);
+
+ return vgic_v5_restore_two_level_ist(kvm, &ist, uaddr,
+ GICV5_HWIRQ_TYPE_LPI);
+}
+
+/*
+ * Process the pending IRQs removing them from the list and optionally injecting
+ * them.
+ */
+static int vgic_v5_process_pending_irqs(struct kvm *kvm, bool inject)
+{
+ u32 vm_id = vgic_v5_vm_id(kvm);
+ struct vgic_v5_vm_info *vmi;
+
+ lockdep_assert_held(&kvm->arch.config_lock);
+
+ vmi = xa_load(&vm_info, vm_id);
+ if (!vmi)
+ return -ENXIO;
+
+ vgic_v5_drain_pending_irqs(kvm, vmi, inject);
+
+ return 0;
+}
+
+/* Replay pending state that was cleared while restoring guest IST state. */
+int vgic_v5_restore_pending_irqs(struct kvm *kvm)
+{
+ return vgic_v5_process_pending_irqs(kvm, true);
+}
+
+/* Drop pending state collected by a failed IST restore. */
+void vgic_v5_discard_pending_irqs(struct kvm *kvm)
+{
+ vgic_v5_process_pending_irqs(kvm, false);
+}
diff --git a/arch/arm64/kvm/vgic/vgic-v5-tables.h b/arch/arm64/kvm/vgic/vgic-v5-tables.h
new file mode 100644
index 00000000000000..9a272b8cc6c87e
--- /dev/null
+++ b/arch/arm64/kvm/vgic/vgic-v5-tables.h
@@ -0,0 +1,116 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2025, 2026 Arm Ltd.
+ */
+
+#ifndef __KVM_ARM_VGICV5_TABLES_H__
+#define __KVM_ARM_VGICV5_TABLES_H__
+
+#include <linux/idr.h>
+#include <linux/irqchip/arm-gic-v5.h>
+#include <linux/list.h>
+
+/* Level 1 Virtual Machine Table Entry */
+typedef __le64 vmtl1_entry;
+
+/* Level 2 Virtual Machine Table Entry */
+struct vmtl2_entry {
+ __le64 val[4];
+};
+
+/* Virtual PE Table Entry */
+typedef __le64 vpe_entry;
+
+struct vgic_v5_vm_info {
+ void __iomem *vmd_base;
+ vpe_entry __iomem *vpet_base;
+ void __iomem **vped_ptrs;
+ u8 vpe_id_bits;
+
+ /*
+ * Both the LPI and SPI ISTs are allocated by the hypervisor. While it
+ * would be possible to track and access them by iterating over the ISTs
+ * themselves, it makes more sense to store pointers to the ISTs.
+ *
+ * The LPI IST can either be two-level or linear. Hence, we keep track
+ * of the structure. If it is two-level, we retain pointers to the L1
+ * IST and to each L2 IST array. If it is linear, we just store the base
+ * address of the IST array.
+ *
+ * The SPI IST is linear, and therefore we just store the base address
+ * of the SPI IST array.
+ */
+ bool h_lpi_ist_structure;
+ __le64 *h_lpi_ist;
+ __le64 **h_lpi_l2_ists;
+ __le64 *h_spi_ist;
+
+ /* Tracking of pending interrupts as part of IST restore */
+ struct list_head pending_irqs;
+};
+
+struct vgic_v5_vmt {
+ union {
+ struct {
+ struct vmtl2_entry *vmt_base;
+ unsigned int num_ents;
+ } linear;
+ struct {
+ vmtl1_entry *vmt_base;
+ struct vmtl2_entry **l2ptrs;
+ unsigned int num_l1_ents;
+ } l2;
+ };
+ bool two_level;
+ unsigned int num_entries;
+ unsigned int max_vpes;
+ size_t vmd_size;
+ size_t vped_size;
+ struct ida vm_id_ida;
+};
+
+static inline u32 vgic_v5_vm_id(struct kvm *kvm)
+{
+ return kvm->arch.vgic.gicv5_vm.vm_id;
+}
+
+static inline u16 vgic_v5_vpe_id(struct kvm_vcpu *vcpu)
+{
+ return vcpu->vcpu_idx;
+}
+
+static inline int vgic_v5_vpe_db(struct kvm_vcpu *vcpu)
+{
+ return vcpu->arch.vgic_cpu.vgic_v5.gicv5_vpe.db;
+}
+
+int vgic_v5_vmt_allocate(unsigned int max_vpes);
+int vgic_v5_vmt_free(void);
+phys_addr_t vgic_v5_get_vmt_base(void);
+
+int vgic_v5_allocate_vm_id(struct kvm *kvm);
+void vgic_v5_release_vm_id(struct kvm *kvm);
+
+int vgic_v5_vmte_init(struct kvm *kvm);
+int vgic_v5_vmte_release(struct kvm *kvm);
+u8 vgic_v5_vmte_vpe_id_bits(struct kvm_vcpu *vcpu);
+int vgic_v5_vmte_alloc_vpe(struct kvm_vcpu *vcpu);
+int vgic_v5_vmte_free_vpe(struct kvm_vcpu *vcpu);
+
+int vgic_v5_spi_ist_alloc(struct kvm *kvm, unsigned int id_bits);
+int vgic_v5_lpi_ist_exists(struct kvm *kvm);
+int vgic_v5_lpi_ist_alloc(struct kvm *kvm, unsigned int id_bits);
+int vgic_v5_lpi_ist_free(struct kvm *kvm);
+
+int vgic_v5_save_spi_ist(struct kvm *kvm,
+ struct kvm_vgic_v5_ist *ist_attr);
+int vgic_v5_save_lpi_ist(struct kvm *kvm,
+ struct kvm_vgic_v5_ist *ist_attr);
+int vgic_v5_restore_spi_ist(struct kvm *kvm,
+ struct kvm_vgic_v5_ist *ist_attr);
+int vgic_v5_restore_lpi_ist(struct kvm *kvm,
+ struct kvm_vgic_v5_ist *ist_attr);
+int vgic_v5_restore_pending_irqs(struct kvm *kvm);
+void vgic_v5_discard_pending_irqs(struct kvm *kvm);
+
+#endif
diff --git a/arch/arm64/kvm/vgic/vgic-v5.c b/arch/arm64/kvm/vgic/vgic-v5.c
index d4789ff3e74026..153a8fcd4db7c1 100644
--- a/arch/arm64/kvm/vgic/vgic-v5.c
+++ b/arch/arm64/kvm/vgic/vgic-v5.c
@@ -7,10 +7,18 @@
#include <linux/bitops.h>
#include <linux/irqchip/arm-vgic-info.h>
+#include <linux/irqdomain.h>
+#include <linux/kvm_host.h>
+#include <linux/uaccess.h>
#include "vgic.h"
+#include "vgic-v5-tables.h"
#define ppi_caps kvm_vgic_global_state.vgic_v5_ppi_caps
+#define irs_caps kvm_vgic_global_state.vgic_v5_irs_caps
+
+static int vgic_v5_irs_assign_vmt(bool two_level, u8 vm_id_bits, phys_addr_t vmt_base);
+static int vgic_v5_irs_clear_vmt(void);
/*
* Not all PPIs are guaranteed to be implemented for GICv5. Deterermine which
@@ -34,6 +42,66 @@ static void vgic_v5_get_implemented_ppis(void)
__assign_bit(GICV5_ARCH_PPI_PMUIRQ, ppi_caps.impl_ppi_mask, system_supports_pmuv3());
}
+/*
+ * The IRS MMIO interface is shared between all VMs, so make sure we don't do
+ * anything stupid!
+ */
+static DEFINE_RAW_SPINLOCK(global_irs_lock);
+
+static u32 irs_readl_relaxed(const u32 reg_offset)
+{
+ return readl_relaxed(irs_caps.irs_base + reg_offset);
+}
+
+static void irs_writel_relaxed(const u32 val, const u32 reg_offset)
+{
+ writel_relaxed(val, irs_caps.irs_base + reg_offset);
+}
+
+static u64 irs_readq_relaxed(const u32 reg_offset)
+{
+ return readq_relaxed(irs_caps.irs_base + reg_offset);
+}
+
+static void irs_writeq_relaxed(const u64 val, const u32 reg_offset)
+{
+ writeq_relaxed(val, irs_caps.irs_base + reg_offset);
+}
+
+static void vgic_v5_irs_extract_vm_caps(const struct gic_kvm_info *info)
+{
+ u64 idr;
+
+ irs_caps.irs_base = info->gicv5_irs.base;
+ irs_caps.non_coherent = info->gicv5_irs.non_coherent;
+
+ idr = irs_readl_relaxed(GICV5_IRS_IDR2);
+
+ /* We skip the LPI field as it only applies to physical LPIs */
+ irs_caps.ist_id_bits = FIELD_GET(GICV5_IRS_IDR2_ID_BITS, idr);
+ irs_caps.min_lpi_id_bits = FIELD_GET(GICV5_IRS_IDR2_MIN_LPI_ID_BITS, idr);
+ irs_caps.ist_levels = (idr & GICV5_IRS_IDR2_IST_LEVELS);
+ irs_caps.ist_l2sz = FIELD_GET(GICV5_IRS_IDR2_IST_L2SZ, idr);
+ irs_caps.istmd = (idr & GICV5_IRS_IDR2_ISTMD);
+ irs_caps.istmd_sz = FIELD_GET(GICV5_IRS_IDR2_ISTMD_SZ, idr);
+
+ idr = irs_readl_relaxed(GICV5_IRS_IDR3);
+
+ irs_caps.max_vms = BIT(FIELD_GET(GICV5_IRS_IDR3_VM_ID_BITS, idr));
+ irs_caps.two_level_vmt_support = (idr & GICV5_IRS_IDR3_VMT_LEVELS);
+
+ if (idr & GICV5_IRS_IDR3_VMD)
+ irs_caps.vmd_size = BIT(FIELD_GET(GICV5_IRS_IDR3_VMD_SZ, idr));
+ else
+ irs_caps.vmd_size = 0;
+
+ idr = irs_readl_relaxed(GICV5_IRS_IDR4);
+
+ irs_caps.vped_size = BIT(FIELD_GET(GICV5_IRS_IDR4_VPED_SZ, idr));
+ /* Field stores VPE_ID_BITS - 1 */
+ irs_caps.max_vpes = BIT(FIELD_GET(GICV5_IRS_IDR4_VPE_ID_BITS, idr) + 1);
+}
+
/*
* Probe for a vGICv5 compatible interrupt controller, returning 0 on success.
*/
@@ -44,6 +112,8 @@ int vgic_v5_probe(const struct gic_kvm_info *info)
int ret;
kvm_vgic_global_state.type = VGIC_V5;
+ kvm_vgic_global_state.max_gic_vcpus = 0;
+ kvm_vgic_global_state.max_gicv5_vcpus = 0;
kvm_vgic_global_state.vcpu_base = 0;
kvm_vgic_global_state.vctrl_base = NULL;
@@ -61,17 +131,60 @@ int vgic_v5_probe(const struct gic_kvm_info *info)
goto skip_v5;
}
- kvm_vgic_global_state.max_gic_vcpus = VGIC_V5_MAX_CPUS;
-
+ vgic_v5_irs_extract_vm_caps(info);
vgic_v5_get_implemented_ppis();
+ /*
+ * Even if the HW supports more per-VM vCPUs, artificially cap as we
+ * can't use them all.
+ */
+ kvm_vgic_global_state.max_gicv5_vcpus = min(irs_caps.max_vpes,
+ VGIC_V5_MAX_CPUS);
+
+ /*
+ * GICv5 requires a set of tables to be allocated in order to manage
+ * VMs. We allocate them in advance here, which alas means that we
+ * already have to make a decisions regarding the maximum number of VMs
+ * we want to run. For now, we match the maximum number offered by the
+ * hardware, but this might not be a wise choice in the long term.
+ */
+ ret = vgic_v5_vmt_allocate(kvm_vgic_global_state.max_gicv5_vcpus);
+ if (ret) {
+ kvm_err("Failed to allocate the GICv5 VM tables; no GICv5 support\n");
+ return -ENODEV;
+ }
+
+ /*
+ * We've now allocated the VM table, but the host's IRS doesn't know
+ * about it yet. Provide the base address of the VMT to the IRS, as well
+ * as the number of ID bits that it covers and the structure used
+ * (linear/two-level).
+ */
+ ret = vgic_v5_irs_assign_vmt(irs_caps.two_level_vmt_support,
+ ilog2(irs_caps.max_vms),
+ vgic_v5_get_vmt_base());
+ if (ret) {
+ kvm_err("Failed to assign the GICv5 VM tables to the IRS; no GICv5 support\n");
+ if (!vgic_v5_irs_clear_vmt())
+ vgic_v5_vmt_free();
+ return -ENODEV;
+ }
+
ret = kvm_register_vgic_device(KVM_DEV_TYPE_ARM_VGIC_V5);
if (ret) {
kvm_err("Cannot register GICv5 KVM device.\n");
- goto skip_v5;
+ /*
+ * Don't free the VMT itself if the hardware still has a valid
+ * pointer to it.
+ */
+ if (!vgic_v5_irs_clear_vmt())
+ vgic_v5_vmt_free();
+ return -ENODEV;
}
v5_registered = true;
+ kvm_vgic_global_state.max_gic_vcpus =
+ kvm_vgic_global_state.max_gicv5_vcpus;
kvm_info("GCIE system register CPU interface\n");
skip_v5:
@@ -95,12 +208,12 @@ int vgic_v5_probe(const struct gic_kvm_info *info)
ret = kvm_register_vgic_device(KVM_DEV_TYPE_ARM_VGIC_V3);
if (ret) {
kvm_err("Cannot register GICv3-legacy KVM device.\n");
- return ret;
+ /* vGICv5 should still work */
+ return v5_registered ? 0 : ret;
}
- /* We potentially limit the max VCPUs further than we need to here */
- kvm_vgic_global_state.max_gic_vcpus = min(VGIC_V3_MAX_CPUS,
- VGIC_V5_MAX_CPUS);
+ kvm_vgic_global_state.max_gic_vcpus = max(kvm_vgic_global_state.max_gic_vcpus,
+ VGIC_V3_MAX_CPUS);
static_branch_enable(&kvm_vgic_global_state.gicv3_cpuif);
kvm_info("GCIE legacy system register CPU interface\n");
@@ -110,6 +223,403 @@ int vgic_v5_probe(const struct gic_kvm_info *info)
return 0;
}
+/*
+ * Wait for completion of a change in any of IRS_VMT_BASER, IRS_VMAP_L2_VMTR,
+ * IRS_VMAP_VMR, IRS_VMAP_VPER, IRS_VMAP_VISTR, IRS_VMAP_L2_VISTR.
+ */
+static int vgic_v5_irs_wait_for_vm_op(void)
+{
+ return gicv5_wait_for_op_atomic(irs_caps.irs_base,
+ GICV5_IRS_VMT_STATUSR,
+ GICV5_IRS_VMT_STATUSR_IDLE,
+ NULL);
+}
+
+/*
+ * Wait for completion of a change in any of IRS_VPE_SELR, IRS_VPE_DBR,
+ * IRS_VPE_CR0.
+ */
+static int vgic_v5_irs_wait_for_vpe_op(void)
+{
+ return gicv5_wait_for_op_atomic(irs_caps.irs_base,
+ GICV5_IRS_VPE_STATUSR,
+ GICV5_IRS_VPE_STATUSR_IDLE,
+ NULL);
+}
+
+/*
+ * Wait for a write to IRS_SAVE_VMR to complete.
+ */
+static int vgic_v5_irs_wait_for_save_vm_op(u32 *statusr)
+{
+ return gicv5_wait_for_op_atomic(irs_caps.irs_base,
+ GICV5_IRS_SAVE_VM_STATUSR,
+ GICV5_IRS_SAVE_VM_STATUSR_IDLE,
+ statusr);
+}
+
+static int vgic_v5_irs_write_vm_mmio_reg(u64 val, u32 offset)
+{
+ int ret;
+
+ guard(raw_spinlock_irqsave)(&global_irs_lock);
+
+ /* Make sure that we are idle to begin with */
+ ret = vgic_v5_irs_wait_for_vm_op();
+ if (ret)
+ return ret;
+
+ irs_writeq_relaxed(val, offset);
+
+ return vgic_v5_irs_wait_for_vm_op();
+}
+
+static int vgic_v5_irs_assign_vmt(bool two_level, u8 vm_id_bits,
+ phys_addr_t vmt_base)
+{
+ u64 vmt_baser;
+ u32 vmt_cfgr;
+
+ guard(raw_spinlock_irqsave)(&global_irs_lock);
+
+ vmt_baser = irs_readq_relaxed(GICV5_IRS_VMT_BASER);
+ if (!!FIELD_GET(GICV5_IRS_VMT_BASER_VALID, vmt_baser))
+ return -EBUSY;
+
+ vmt_cfgr = FIELD_PREP(GICV5_IRS_VMT_CFGR_VM_ID_BITS, vm_id_bits);
+ if (two_level)
+ vmt_cfgr |= FIELD_PREP(GICV5_IRS_VMT_CFGR_STRUCTURE,
+ GICV5_IRS_VMT_CFGR_STRUCTURE_TWO_LEVEL);
+
+ irs_writel_relaxed(vmt_cfgr, GICV5_IRS_VMT_CFGR);
+
+ /* The base address is intentionally only masked and not shifted */
+ vmt_baser = FIELD_PREP(GICV5_IRS_VMT_BASER_VALID, true) |
+ (vmt_base & GICV5_IRS_VMT_BASER_ADDR);
+ irs_writeq_relaxed(vmt_baser, GICV5_IRS_VMT_BASER);
+
+ return vgic_v5_irs_wait_for_vm_op();
+}
+
+static int vgic_v5_irs_clear_vmt(void)
+{
+ return vgic_v5_irs_write_vm_mmio_reg(0, GICV5_IRS_VMT_BASER);
+}
+
+static int vgic_v5_irs_vmap_l2_vmt(u16 vm_id)
+{
+ u64 val = FIELD_PREP(GICV5_IRS_VMAP_L2_VMTR_VM_ID, vm_id) |
+ GICV5_IRS_VMAP_L2_VMTR_M;
+
+ return vgic_v5_irs_write_vm_mmio_reg(val, GICV5_IRS_VMAP_L2_VMTR);
+}
+
+static int __vgic_v5_irs_vmap_vm(u16 vm_id, bool unmap)
+{
+ u64 val = FIELD_PREP(GICV5_IRS_VMAP_VMR_VM_ID, vm_id) |
+ FIELD_PREP(GICV5_IRS_VMAP_VMR_U, unmap) |
+ GICV5_IRS_VMAP_VMR_M;
+
+ return vgic_v5_irs_write_vm_mmio_reg(val, GICV5_IRS_VMAP_VMR);
+}
+
+static int vgic_v5_irs_set_vm_valid(u16 vm_id)
+{
+ return __vgic_v5_irs_vmap_vm(vm_id, false);
+}
+
+static int vgic_v5_irs_set_vm_invalid(u16 vm_id)
+{
+ return __vgic_v5_irs_vmap_vm(vm_id, true);
+}
+
+static int __vgic_v5_irs_update_vist_validity(u16 vm_id, bool spi_ist, bool unmap)
+{
+ u8 type = spi_ist ? 0b011 : 0b010;
+ u64 val = FIELD_PREP(GICV5_IRS_VMAP_VISTR_TYPE, type) |
+ FIELD_PREP(GICV5_IRS_VMAP_VISTR_VM_ID, vm_id) |
+ FIELD_PREP(GICV5_IRS_VMAP_VISTR_U, unmap) |
+ GICV5_IRS_VMAP_VISTR_M;
+
+ return vgic_v5_irs_write_vm_mmio_reg(val, GICV5_IRS_VMAP_VISTR);
+}
+
+static int vgic_v5_irs_set_vist_valid(u16 vm_id, bool spi_ist)
+{
+ return __vgic_v5_irs_update_vist_validity(vm_id, spi_ist, false);
+}
+
+/*
+ * LPI ISTs can be invalidated explicitly. SPI ISTs are invalidated by making
+ * the VMTE invalid during teardown.
+ */
+static int vgic_v5_irs_set_vist_invalid(u16 vm_id, bool spi_ist)
+{
+ return __vgic_v5_irs_update_vist_validity(vm_id, spi_ist, true);
+}
+
+static int vgic_v5_irs_set_up_vpe(u16 vm_id, u16 vpe_id,
+ irq_hw_number_t db_hwirq)
+{
+ u64 vmap_vper, dbr, selr;
+ u32 statusr, cr0;
+ int ret;
+
+ guard(raw_spinlock_irqsave)(&global_irs_lock);
+
+ /* Make sure that we are idle to begin with */
+ ret = vgic_v5_irs_wait_for_vm_op();
+ if (ret)
+ return ret;
+
+ /* Mark the VPE as valid */
+ vmap_vper = FIELD_PREP(GICV5_IRS_VMAP_VPER_VPE_ID, vpe_id) |
+ FIELD_PREP(GICV5_IRS_VMAP_VPER_VM_ID, vm_id) |
+ GICV5_IRS_VMAP_VPER_M;
+ irs_writeq_relaxed(vmap_vper, GICV5_IRS_VMAP_VPER);
+
+ /* Wait for the VPE to be marked valid in the VPET */
+ ret = vgic_v5_irs_wait_for_vm_op();
+ if (ret)
+ return ret;
+
+ selr = FIELD_PREP(GICV5_IRS_VPE_SELR_VPE_ID, vpe_id) |
+ FIELD_PREP(GICV5_IRS_VPE_SELR_VM_ID, vm_id) |
+ GICV5_IRS_VPE_SELR_S;
+ irs_writeq_relaxed(selr, GICV5_IRS_VPE_SELR);
+
+ ret = vgic_v5_irs_wait_for_vpe_op();
+ if (ret)
+ return ret;
+
+ statusr = irs_readl_relaxed(GICV5_IRS_VPE_STATUSR);
+ if (!FIELD_GET(GICV5_IRS_VPE_STATUSR_V, statusr))
+ return -EINVAL;
+
+ /* Set targeted only routing (disable 1ofN vPE selection) */
+ cr0 = GICV5_IRS_VPE_CR0_DPS;
+ irs_writel_relaxed(cr0, GICV5_IRS_VPE_CR0);
+
+ ret = vgic_v5_irs_wait_for_vpe_op();
+ if (ret)
+ return ret;
+
+ /*
+ * The VPE has not yet run. Therefore, make sure that all interrupts
+ * will generate a doorbell.
+ */
+ dbr = FIELD_PREP(GICV5_IRS_VPE_DBR_INTID, db_hwirq) |
+ GICV5_IRS_VPE_DBR_DBV;
+ irs_writeq_relaxed(dbr, GICV5_IRS_VPE_DBR);
+
+ ret = vgic_v5_irs_wait_for_vpe_op();
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int vgic_v5_irs_save_vm_op(u16 vm_id, bool save, u32 *statusr)
+{
+ u64 save_vmr;
+ int ret;
+
+ save_vmr = FIELD_PREP(GICV5_IRS_SAVE_VMR_VM_ID, vm_id);
+ save_vmr |= GICV5_IRS_SAVE_VMR_Q;
+ save_vmr |= FIELD_PREP(GICV5_IRS_SAVE_VMR_S, save);
+
+ guard(raw_spinlock_irqsave)(&global_irs_lock);
+
+ /* Make sure that we are idle to begin with. */
+ ret = vgic_v5_irs_wait_for_save_vm_op(NULL);
+ if (ret)
+ return ret;
+
+ irs_writeq_relaxed(save_vmr, GICV5_IRS_SAVE_VMR);
+
+ return vgic_v5_irs_wait_for_save_vm_op(statusr);
+}
+
+static irqreturn_t db_handler(int irq, void *data)
+{
+ struct kvm_vcpu *vcpu = data;
+
+ WRITE_ONCE(vcpu->arch.vgic_cpu.vgic_v5.gicv5_vpe.db_fired, true);
+
+ kvm_make_request(KVM_REQ_IRQ_PENDING, vcpu);
+ kvm_vcpu_kick(vcpu);
+
+ return IRQ_HANDLED;
+}
+
+static int vgic_v5_send_command(struct kvm_vcpu *vcpu, enum gicv5_vcpu_cmd cmd)
+{
+ return irq_set_vcpu_affinity(vgic_v5_vpe_db(vcpu), &cmd);
+}
+
+static int vgic_v5_db_set_vcpu_affinity(struct irq_data *data, void *vcpu_info)
+{
+ struct vgic_v5_vm *vm = data->domain->host_data;
+ enum gicv5_vcpu_cmd *cmd = vcpu_info;
+ /* Our VPE ID is the index within the doorbell domain */
+ u16 vpe_id = data->hwirq;
+
+ switch (*cmd) {
+ case VMT_L2_MAP:
+ return vgic_v5_irs_vmap_l2_vmt(vm->vm_id);
+ case VMTE_MAKE_VALID:
+ return vgic_v5_irs_set_vm_valid(vm->vm_id);
+ case VMTE_MAKE_INVALID:
+ return vgic_v5_irs_set_vm_invalid(vm->vm_id);
+ case VPE_MAKE_VALID:
+ /*
+ * We need the actual LPI ID which lives in the top-most parent
+ * domain. This hwirq won't include the type (LPI) but that's
+ * not required for the IRS_VPE_DBR.
+ */
+ while (data->parent_data)
+ data = data->parent_data;
+ return vgic_v5_irs_set_up_vpe(vm->vm_id, vpe_id, data->hwirq);
+ case SPI_VIST_MAKE_VALID:
+ return vgic_v5_irs_set_vist_valid(vm->vm_id, true);
+ case LPI_VIST_MAKE_VALID:
+ return vgic_v5_irs_set_vist_valid(vm->vm_id, false);
+ case LPI_VIST_MAKE_INVALID:
+ return vgic_v5_irs_set_vist_invalid(vm->vm_id, false);
+ default:
+ return -EINVAL;
+ }
+}
+
+/*
+ * This set of irq_chip functions is specific for doorbells.
+ */
+static const struct irq_chip vgic_v5_db_irq_chip = {
+ .name = "GICv5-DB",
+ .irq_mask = irq_chip_mask_parent,
+ .irq_unmask = irq_chip_unmask_parent,
+ .irq_eoi = irq_chip_eoi_parent,
+ .irq_set_affinity = irq_chip_set_affinity_parent,
+ .irq_get_irqchip_state = irq_chip_get_parent_state,
+ .irq_set_irqchip_state = irq_chip_set_parent_state,
+ .irq_set_vcpu_affinity = vgic_v5_db_set_vcpu_affinity,
+ .flags = IRQCHIP_SET_TYPE_MASKED | IRQCHIP_SKIP_SET_WAKE |
+ IRQCHIP_MASK_ON_SUSPEND,
+};
+
+static void vgic_v5_irq_db_domain_free(struct irq_domain *domain,
+ unsigned int virq, unsigned int nr_irqs)
+{
+ int i;
+
+ for (i = 0; i < nr_irqs; i++) {
+ struct irq_data *d = irq_domain_get_irq_data(domain, virq + i);
+
+ irq_set_handler(virq + i, NULL);
+ irq_domain_reset_irq_data(d);
+ }
+
+ irq_domain_free_irqs_parent(domain, virq, nr_irqs);
+}
+
+static int vgic_v5_irq_db_domain_alloc(struct irq_domain *domain,
+ unsigned int virq, unsigned int nr_irqs,
+ void *arg)
+{
+ const struct irq_chip *chip = &vgic_v5_db_irq_chip;
+ struct vgic_v5_vm *vm = arg;
+ struct irq_data *irqd;
+ int ret;
+
+ if (!vm) {
+ kvm_err("invalid parameter for doorbell irq allocation\n");
+ return -EINVAL;
+ }
+
+ ret = irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, NULL);
+ if (ret)
+ return ret;
+
+ for (int i = 0; i < nr_irqs; i++) {
+ irq_domain_set_hwirq_and_chip(domain, virq + i, i, chip,
+ domain->host_data);
+ irqd = irq_desc_get_irq_data(irq_to_desc(virq + i));
+ irqd_set_single_target(irqd);
+ }
+
+ return 0;
+}
+
+static const struct irq_domain_ops vgic_v5_irq_db_domain_ops = {
+ .alloc = vgic_v5_irq_db_domain_alloc,
+ .free = vgic_v5_irq_db_domain_free,
+};
+
+static int vgic_v5_create_per_vm_domain(struct kvm *kvm)
+{
+ struct vgic_v5_vm *vm = &kvm->arch.vgic.gicv5_vm;
+ int nr_vcpus = atomic_read(&kvm->online_vcpus);
+ int id = task_pid_nr(current);
+ int ret, db_virq = 0;
+
+ if (!gicv5_global_data.lpi_domain) {
+ kvm_err("LPI domain uninitialized, can't set up KVM Doorbells\n");
+ return -ENODEV;
+ }
+
+ vm->fwnode = irq_domain_alloc_named_id_fwnode("GICv5-vpe-db", id);
+ if (!vm->fwnode)
+ return -ENOMEM;
+
+ /*
+ * KVM per-VM VPE DB domain; child of LPI domain; only ever handles
+ * doorbells. We know how many doorbells we have, and therefore we
+ * create a linear domain.
+ */
+ vm->domain = irq_domain_create_hierarchy(gicv5_global_data.lpi_domain,
+ 0, nr_vcpus, vm->fwnode,
+ &vgic_v5_irq_db_domain_ops, vm);
+ if (!vm->domain) {
+ ret = -ENOMEM;
+ goto err;
+ }
+
+ db_virq = irq_domain_alloc_irqs(vm->domain, nr_vcpus, NUMA_NO_NODE, vm);
+ if (db_virq <= 0) {
+ ret = db_virq;
+ goto err;
+ }
+
+ kvm->arch.vgic.gicv5_vm.vpe_db_base = db_virq;
+
+ return 0;
+
+err:
+ if (db_virq > 0)
+ irq_domain_free_irqs(db_virq, nr_vcpus);
+ if (vm->domain)
+ irq_domain_remove(vm->domain);
+ if (vm->fwnode)
+ irq_domain_free_fwnode(vm->fwnode);
+
+ kvm->arch.vgic.gicv5_vm.vpe_db_base = 0;
+ vm->domain = NULL;
+ vm->fwnode = NULL;
+
+ return ret;
+}
+
+static void vgic_v5_teardown_per_vm_domain(struct vgic_v5_vm *vm)
+{
+ if (!vm->domain)
+ return;
+
+ irq_domain_remove(vm->domain);
+ irq_domain_free_fwnode(vm->fwnode);
+ vm->domain = NULL;
+ vm->fwnode = NULL;
+}
+
void vgic_v5_reset(struct kvm_vcpu *vcpu)
{
/*
@@ -123,17 +633,112 @@ void vgic_v5_reset(struct kvm_vcpu *vcpu)
* CPUIF (but potentially fewer in the IRS).
*/
vcpu->arch.vgic_cpu.num_pri_bits = 5;
+
+ /* Make the VPE valid in the VPET */
+ if (WARN_ON(vgic_v5_send_command(vcpu, VPE_MAKE_VALID)))
+ return;
}
-int vgic_v5_init(struct kvm *kvm)
+static void vgic_v5_free_doorbells(struct kvm *kvm, unsigned int nr_dbs)
{
+ struct vgic_v5_vm *vm = &kvm->arch.vgic.gicv5_vm;
struct kvm_vcpu *vcpu;
- unsigned long idx;
+ unsigned long i;
+ int db;
- if (vgic_initialized(kvm))
- return 0;
+ for (i = 0; i < nr_dbs; i++) {
+ vcpu = kvm_get_vcpu(kvm, i);
+ db = vgic_v5_vpe_db(vcpu);
+ if (!db)
+ continue;
+
+ free_irq(db, vcpu);
+ vcpu->arch.vgic_cpu.vgic_v5.gicv5_vpe.db = 0;
+ }
+
+ if (vm->vpe_db_base) {
+ irq_domain_free_irqs(vm->vpe_db_base,
+ atomic_read(&kvm->online_vcpus));
+ vm->vpe_db_base = 0;
+ }
+}
+
+void vgic_v5_teardown(struct kvm *kvm)
+{
+ struct vgic_dist *dist = &kvm->arch.vgic;
+ struct kvm_vcpu *vcpu, *vcpu0;
+ bool release_vm_id = true;
+ unsigned long i;
+ int rc;
+
+ /*
+ * If the VM's ID isn't valid, then we either failed init very early or
+ * we've been called a second time. Nothing to do here in either case.
+ */
+ if (kvm->arch.vgic.gicv5_vm.vm_id == VGIC_V5_VM_ID_INVAL)
+ return;
+
+ if (kvm->arch.vgic.gicv5_vm.vmte_allocated) {
+ /* Make the VM invalid */
+ vcpu0 = kvm_get_vcpu(kvm, 0);
+ rc = vgic_v5_send_command(vcpu0, VMTE_MAKE_INVALID);
+ if (rc) {
+ kvm_err("could not make VMTE invalid\n");
+ release_vm_id = false;
+ goto out_free_doorbells;
+ }
- kvm_for_each_vcpu(idx, vcpu, kvm) {
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ if (vgic_v5_vmte_free_vpe(vcpu)) {
+ kvm_err("Failed to free VPE\n");
+ release_vm_id = false;
+ }
+ }
+
+ if (vgic_v5_vmte_release(kvm)) {
+ kvm_err("Failed to release VM 0x%x\n", dist->gicv5_vm.vm_id);
+ release_vm_id = false;
+ }
+ }
+
+out_free_doorbells:
+ vgic_v5_free_doorbells(kvm, atomic_read(&kvm->online_vcpus));
+ vgic_v5_teardown_per_vm_domain(&kvm->arch.vgic.gicv5_vm);
+
+ /*
+ * We only release the VM ID itself if we didn't fail earlier. It does
+ * mean that we might lose the VM ID (and associated VMTE, etc), but
+ * given that we've failed to tear them down correctly there's no way to
+ * safely reuse them. The VM ID allocating IDA will make sure we don't
+ * accidentally reuse this partially torn down state.
+ */
+ if (release_vm_id)
+ vgic_v5_release_vm_id(kvm);
+}
+
+/*
+ * Claim and populate a VMTE (optionally making a new L2 VMT valid), create VPE
+ * doorbells, allocate VPET and populate for each VPE. Finally, we also init the
+ * vIRS, which means allocating and making the virtual SPI IST valid.
+ *
+ * Note: We do need to put the cart before the horse here. The VPE doorbells are
+ * our conduit for communication with the IRS, which means we need to have those
+ * before making the VMTE valid.
+ *
+ * On failure, we clean up in the teardown path (vgic_v5_teardown()).
+ */
+int vgic_v5_init(struct kvm *kvm)
+{
+ struct kvm_vcpu *vcpu, *vcpu0;
+ int nr_vcpus, ret = 0;
+ unsigned int db_virq;
+ unsigned long i;
+
+ nr_vcpus = atomic_read(&kvm->online_vcpus);
+ if (nr_vcpus == 0)
+ return -ENODEV;
+
+ kvm_for_each_vcpu(i, vcpu, kvm) {
if (vcpu_has_nv(vcpu)) {
kvm_err("Nested GICv5 VMs are currently unsupported\n");
return -EINVAL;
@@ -148,7 +753,61 @@ int vgic_v5_init(struct kvm *kvm)
kvm->arch.vgic.gicv5_vm.userspace_ppis,
ppi_caps.impl_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);
+ ret = vgic_v5_allocate_vm_id(kvm);
+ if (ret)
+ return ret;
+
+ ret = vgic_v5_create_per_vm_domain(kvm);
+ if (ret)
+ goto err;
+
+ db_virq = kvm->arch.vgic.gicv5_vm.vpe_db_base;
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ ret = request_irq(db_virq + i, db_handler, 0, "vcpu", vcpu);
+ if (ret)
+ goto err;
+
+ /* Stash it with the VCPU for easy retrieval */
+ vcpu->arch.vgic_cpu.vgic_v5.gicv5_vpe.db = db_virq + i;
+ }
+
+ /* Populate VMTE (with VPET and VM descriptor) */
+ ret = vgic_v5_vmte_init(kvm);
+ if (ret)
+ goto err;
+
+ /* We pick the first vcpu to make the VMTE valid - any would do */
+ vcpu0 = kvm_get_vcpu(kvm, 0);
+ ret = vgic_v5_send_command(vcpu0, VMTE_MAKE_VALID);
+ if (ret)
+ goto err;
+
+ /* Loop over all VPEs, allocate/populate their data structures */
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ ret = vgic_v5_vmte_alloc_vpe(vcpu);
+ if (ret)
+ goto err;
+ }
+
+ /* Init IRS (and alloc SPI IST) */
+ ret = kvm_vgic_v5_irs_init(kvm, kvm->arch.vgic.nr_spis);
+ if (ret)
+ goto err;
+
return 0;
+
+err:
+ /*
+ * Explicitly tear everything down on failure. The teardown function is
+ * written to handle any partial state we might have, so we don't need
+ * to do any clean-up first. Teardown will be called a second time on VM
+ * destruction, but that's fine - it is better to leave things in a
+ * clean state now, and doubly so because userspace could actually go
+ * and retry init.
+ */
+ vgic_v5_teardown(kvm);
+
+ return ret;
}
int vgic_v5_map_resources(struct kvm *kvm)
@@ -159,9 +818,10 @@ int vgic_v5_map_resources(struct kvm *kvm)
return 0;
}
-int vgic_v5_finalize_ppi_state(struct kvm *kvm)
+int vgic_v5_finalize_ppi_state(struct kvm_vcpu *vcpu)
{
- struct kvm_vcpu *vcpu0;
+ struct kvm *kvm = vcpu->kvm;
+ struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
int i;
if (!vgic_is_v5(kvm))
@@ -170,35 +830,65 @@ int vgic_v5_finalize_ppi_state(struct kvm *kvm)
guard(mutex)(&kvm->arch.config_lock);
/*
- * If SW_PPI has been advertised, then we know we already
- * initialised the whole thing, and we can return early. Yes,
- * this is pretty hackish as far as state tracking goes...
+ * Discover the set of PPIs that are exposed to the guest once per VM.
+ * Once known, apply that mask to each VCPU's restored PPI state as the
+ * VCPUs are first run.
*/
- if (test_bit(GICV5_ARCH_PPI_SW_PPI, kvm->arch.vgic.gicv5_vm.vgic_ppi_mask))
- return 0;
-
- /* The PPI state for all VCPUs should be the same. Pick the first. */
- vcpu0 = kvm_get_vcpu(kvm, 0);
+ if (!test_bit(GICV5_ARCH_PPI_SW_PPI, kvm->arch.vgic.gicv5_vm.vgic_ppi_mask)) {
+ bitmap_zero(kvm->arch.vgic.gicv5_vm.vgic_ppi_mask,
+ VGIC_V5_NR_PRIVATE_IRQS);
+ bitmap_zero(kvm->arch.vgic.gicv5_vm.vgic_ppi_hmr,
+ VGIC_V5_NR_PRIVATE_IRQS);
+
+ for_each_set_bit(i, ppi_caps.impl_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS) {
+ const u32 intid = vgic_v5_make_ppi(i);
+ struct vgic_irq *irq;
+
+ irq = vgic_get_vcpu_irq(vcpu, intid);
+
+ /* Expose PPIs with an owner or the SW_PPI, only */
+ scoped_guard(raw_spinlock_irqsave, &irq->irq_lock) {
+ if (irq->owner || i == GICV5_ARCH_PPI_SW_PPI) {
+ __set_bit(i, kvm->arch.vgic.gicv5_vm.vgic_ppi_mask);
+ __assign_bit(i, kvm->arch.vgic.gicv5_vm.vgic_ppi_hmr,
+ irq->config == VGIC_CONFIG_LEVEL);
+ }
+ }
- bitmap_zero(kvm->arch.vgic.gicv5_vm.vgic_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);
- bitmap_zero(kvm->arch.vgic.gicv5_vm.vgic_ppi_hmr, VGIC_V5_NR_PRIVATE_IRQS);
+ vgic_put_irq(kvm, irq);
+ }
+ }
- for_each_set_bit(i, ppi_caps.impl_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS) {
+ /*
+ * Apply the mask to Enable, Active. Skip pending as that's calculated
+ * on guest entry.
+ */
+ bitmap_and(cpu_if->vgic_ppi_enabler, cpu_if->vgic_ppi_enabler,
+ kvm->arch.vgic.gicv5_vm.vgic_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);
+ bitmap_and(cpu_if->vgic_ppi_activer, cpu_if->vgic_ppi_activer,
+ kvm->arch.vgic.gicv5_vm.vgic_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);
+
+ /* Also update the vgic_irqs */
+ for (i = 0; i < VGIC_V5_NR_PRIVATE_IRQS; i++) {
+ bool visible = test_bit(i, kvm->arch.vgic.gicv5_vm.vgic_ppi_mask);
const u32 intid = vgic_v5_make_ppi(i);
struct vgic_irq *irq;
- irq = vgic_get_vcpu_irq(vcpu0, intid);
+ irq = vgic_get_vcpu_irq(vcpu, intid);
- /* Expose PPIs with an owner or the SW_PPI, only */
scoped_guard(raw_spinlock_irqsave, &irq->irq_lock) {
- if (irq->owner || i == GICV5_ARCH_PPI_SW_PPI) {
- __set_bit(i, kvm->arch.vgic.gicv5_vm.vgic_ppi_mask);
- __assign_bit(i, kvm->arch.vgic.gicv5_vm.vgic_ppi_hmr,
- irq->config == VGIC_CONFIG_LEVEL);
+ if (!visible) {
+ irq->enabled = false;
+ irq->active = false;
+ irq->pending_latch = false;
+ irq->line_level = false;
+ } else {
+ irq->enabled = test_bit(i, cpu_if->vgic_ppi_enabler);
+ irq->active = test_bit(i, cpu_if->vgic_ppi_activer);
}
}
- vgic_put_irq(vcpu0->kvm, irq);
+ vgic_put_irq(kvm, irq);
}
return 0;
@@ -439,9 +1129,53 @@ void vgic_v5_flush_ppi_state(struct kvm_vcpu *vcpu)
VGIC_V5_NR_PRIVATE_IRQS);
}
+static bool vgic_v5_set_spi_pending_state(struct kvm_vcpu *vcpu,
+ struct vgic_irq *irq)
+{
+ vgic_v5_set_irq_pend(irq->target_vcpu, irq);
+ return true;
+}
+
+bool vgic_v5_spi_queue_irq_unlock(struct kvm *kvm,
+ struct vgic_irq *irq,
+ unsigned long flags)
+ __releases(&irq->irq_lock)
+{
+ lockdep_assert_held(&irq->irq_lock);
+
+ raw_spin_unlock_irqrestore(&irq->irq_lock, flags);
+ return true;
+}
+
+static const struct irq_ops vgic_v5_spi_irq_ops = {
+ .set_pending_state = vgic_v5_set_spi_pending_state,
+ .queue_irq_unlock = vgic_v5_spi_queue_irq_unlock,
+};
+
+void vgic_v5_set_spi_ops(struct vgic_irq *irq)
+{
+ if (WARN_ON(!irq) || WARN_ON(irq->ops))
+ return;
+
+ irq->ops = &vgic_v5_spi_irq_ops;
+}
+
+/* Set the pending state for GICv5 SPIs and LPIs */
+void vgic_v5_set_irq_pend(struct kvm_vcpu *vcpu, struct vgic_irq *irq)
+{
+ if (WARN_ON(__irq_is_ppi(KVM_DEV_TYPE_ARM_VGIC_V5, irq->intid)))
+ return;
+
+ kvm_call_hyp(__vgic_v5_vdpend, irq->intid, irq_is_pending(irq),
+ vcpu->kvm->arch.vgic.gicv5_vm.vm_id);
+}
+
void vgic_v5_load(struct kvm_vcpu *vcpu)
{
+ bool irichppidis = !vcpu->kvm->arch.vgic.enabled;
struct vgic_v5_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v5;
+ u16 vm = vgic_v5_vm_id(vcpu->kvm);
+ u16 vpe = vgic_v5_vpe_id(vcpu);
/*
* On the WFI path, vgic_load is called a second time. The first is when
@@ -454,7 +1188,16 @@ void vgic_v5_load(struct kvm_vcpu *vcpu)
kvm_call_hyp(__vgic_v5_restore_vmcr_apr, cpu_if);
- cpu_if->gicv5_vpe.resident = true;
+ cpu_if->vgic_contextr = FIELD_PREP(ICH_CONTEXTR_EL2_V, true) |
+ FIELD_PREP(ICH_CONTEXTR_EL2_IRICHPPIDIS, irichppidis) |
+ FIELD_PREP(ICH_CONTEXTR_EL2_VPE, vpe) |
+ FIELD_PREP(ICH_CONTEXTR_EL2_VM, vm);
+
+ kvm_call_hyp(__vgic_v5_make_resident, cpu_if);
+
+ /* Failed to make the VPE resident? Bang! */
+ if (WARN_ON(!!FIELD_GET(ICH_CONTEXTR_EL2_F, cpu_if->vgic_contextr)))
+ kvm_vm_dead(vcpu->kvm);
}
void vgic_v5_put(struct kvm_vcpu *vcpu)
@@ -472,7 +1215,49 @@ void vgic_v5_put(struct kvm_vcpu *vcpu)
kvm_call_hyp(__vgic_v5_save_apr, cpu_if);
- cpu_if->gicv5_vpe.resident = false;
+ cpu_if->vgic_contextr = 0;
+ if (vcpu_get_flag(vcpu, IN_WFI)) {
+ u32 priority_mask;
+ int dbpm;
+
+ /*
+ * Find the virtual running priority and use this to calculate
+ * the doorbell priority mask. We combine the highest active
+ * priority and the CPU's priority mask. The guest can't handle
+ * interrupts with priorities less than or equal to the virtual
+ * running priority, so there's literally no point in waking the
+ * guest for these.
+ *
+ * The priority needs to be higher than the mask to signal, so
+ * pick the next higher priority (subtract 1).
+ */
+ priority_mask = vgic_v5_get_effective_priority_mask(vcpu);
+
+ /*
+ * Request a doorbell *unless* the priority is 0, indicating
+ * that no interrupt can wake the CPU up.
+ */
+ if (priority_mask) {
+ int db_irq = vgic_v5_vpe_db(vcpu);
+ struct irq_data *d = irq_get_irq_data(db_irq);
+ const struct cpumask *aff = irq_data_get_effective_affinity_mask(d);
+ int cpu = smp_processor_id();
+
+ dbpm = priority_mask - 1;
+ cpu_if->vgic_contextr = FIELD_PREP(ICH_CONTEXTR_EL2_DB, 1) |
+ FIELD_PREP(ICH_CONTEXTR_EL2_DBPM, dbpm);
+
+ /*
+ * Make the doorbell affine to this CPU, if it isn't
+ * already. Actively check the cpumask first as it is
+ * cheaper than changing the affinity every time.
+ */
+ if (!cpumask_test_cpu(cpu, aff))
+ WARN_ON(irq_set_affinity(db_irq, cpumask_of(cpu)));
+ }
+ }
+
+ kvm_call_hyp(__vgic_v5_make_non_resident, cpu_if);
/* The shadow priority is only updated on entering WFI */
if (vcpu_get_flag(vcpu, IN_WFI))
@@ -516,3 +1301,292 @@ void vgic_v5_save_state(struct kvm_vcpu *vcpu)
__vgic_v5_save_ppi_state(cpu_if);
dsb(sy);
}
+
+static int vgic_v5_irs_status_is_quiesced(u32 statusr)
+{
+ if (statusr & GICV5_IRS_SAVE_VM_STATUSR_Q)
+ return 0;
+
+ return -EBUSY;
+}
+
+static int vgic_v5_irs_is_quiesced(u16 vm_id)
+{
+ u32 statusr;
+ int ret;
+
+ ret = vgic_v5_irs_save_vm_op(vm_id, false, &statusr);
+ if (ret)
+ return ret;
+
+ return vgic_v5_irs_status_is_quiesced(statusr);
+}
+
+static int vgic_v5_copy_ist_attr(struct kvm_device_attr *attr,
+ struct kvm_vgic_v5_ist *ist_attr)
+{
+ void __user *uaddr = (void __user *)(unsigned long)attr->addr;
+
+ if (!uaddr)
+ return -EINVAL;
+
+ if (copy_from_user(ist_attr, uaddr, sizeof(*ist_attr)))
+ return -EFAULT;
+
+ return 0;
+}
+
+static int vgic_v5_validate_ist_user_buffer(__u64 addr, __u64 size,
+ size_t expected)
+{
+ if (!addr || size != expected)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int vgic_v5_validate_ist_attr(struct kvm *kvm,
+ const struct kvm_vgic_v5_ist *ist_attr)
+{
+ unsigned int id_bits;
+ int ret;
+
+ /* We always have SPIs to save */
+ ret = vgic_v5_validate_ist_user_buffer(ist_attr->spi_ist_addr,
+ ist_attr->spi_ist_size,
+ kvm->arch.vgic.nr_spis * sizeof(__u32));
+ if (ret)
+ return ret;
+
+ /* We don't always have LPIs to save */
+ ret = vgic_v5_irs_lpi_ist_id_bits(kvm, &id_bits);
+ if (ret < 0)
+ return ret;
+
+ /* No LPI IST */
+ if (!ret) {
+ if (ist_attr->lpi_ist_addr || ist_attr->lpi_ist_size)
+ return -EINVAL;
+
+ return 0;
+ }
+
+ return vgic_v5_validate_ist_user_buffer(ist_attr->lpi_ist_addr,
+ ist_attr->lpi_ist_size,
+ BIT(id_bits) * sizeof(__u32));
+}
+
+int vgic_v5_irs_save_ists(struct kvm *kvm, struct kvm_device_attr *attr)
+{
+ struct kvm_vgic_v5_ist ist_attr;
+ u16 vm_id = vgic_v5_vm_id(kvm);
+ u32 statusr;
+ int ret = 0;
+
+ mutex_lock(&kvm->lock);
+
+ if (kvm_trylock_all_vcpus(kvm)) {
+ mutex_unlock(&kvm->lock);
+ return -EBUSY;
+ }
+
+ mutex_lock(&kvm->arch.config_lock);
+
+ if (!vgic_initialized(kvm)) {
+ ret = -EBUSY;
+ goto out_unlock;
+ }
+
+ ret = vgic_v5_copy_ist_attr(attr, &ist_attr);
+ if (ret)
+ goto out_unlock;
+
+ ret = vgic_v5_validate_ist_attr(kvm, &ist_attr);
+ if (ret)
+ goto out_unlock;
+
+ ret = vgic_v5_irs_save_vm_op(vm_id, true, &statusr);
+ if (ret) {
+ kvm_err("Failed to save GICv5 IRS VM state: %d\n", ret);
+ goto out_unlock;
+ }
+
+ ret = vgic_v5_irs_status_is_quiesced(statusr);
+ if (ret)
+ goto out_unlock;
+
+ /* Save the SPI IST to the userspace buffer. */
+ ret = vgic_v5_save_spi_ist(kvm, &ist_attr);
+ if (ret)
+ goto out_unlock;
+
+ ret = vgic_v5_irs_is_quiesced(vm_id);
+ if (ret)
+ goto out_unlock;
+
+ /* Save the LPI IST to the userspace buffer. */
+ ret = vgic_v5_save_lpi_ist(kvm, &ist_attr);
+ if (ret)
+ goto out_unlock;
+
+ ret = vgic_v5_irs_is_quiesced(vm_id);
+ if (ret)
+ goto out_unlock;
+
+out_unlock:
+ mutex_unlock(&kvm->arch.config_lock);
+ kvm_unlock_all_vcpus(kvm);
+ mutex_unlock(&kvm->lock);
+
+ return ret;
+}
+
+/* Allocate the LPI IST to restore into */
+static int vgic_v5_restore_lpi_ist_alloc(struct kvm *kvm, bool *allocated)
+{
+ unsigned int id_bits;
+ int ret;
+
+ *allocated = false;
+
+ ret = vgic_v5_irs_lpi_ist_id_bits(kvm, &id_bits);
+ if (ret <= 0)
+ return ret;
+
+ ret = vgic_v5_lpi_ist_alloc(kvm, id_bits);
+ if (ret)
+ return ret;
+
+ *allocated = true;
+
+ return 0;
+}
+
+/*
+ * Clean up the LPI IST if we allocated it, and restore the VMTE to the
+ * original, valid state.
+ */
+static void vgic_v5_restore_cleanup(struct kvm *kvm,
+ struct kvm_vcpu *vcpu,
+ bool lpi_ist_allocated)
+{
+ /*
+ * We are on the restore failure path, so we do a best-effort
+ * cleanup. These commands might fail, but at this stage this is the
+ * best we can realistically do.
+ */
+ if (lpi_ist_allocated) {
+ if (!vgic_v5_send_command(vcpu, VMTE_MAKE_INVALID))
+ vgic_v5_lpi_ist_free(kvm);
+ }
+
+ vgic_v5_send_command(vcpu, VMTE_MAKE_VALID);
+}
+
+int vgic_v5_irs_restore_ists(struct kvm *kvm, struct kvm_device_attr *attr)
+{
+ bool lpi_ist_allocated = false, vmte_invalid = false;
+ struct kvm_vcpu *vcpu0 = kvm_get_vcpu(kvm, 0);
+ struct kvm_vgic_v5_ist ist_attr;
+ int ret = 0;
+
+ mutex_lock(&kvm->lock);
+
+ if (kvm_trylock_all_vcpus(kvm)) {
+ mutex_unlock(&kvm->lock);
+ return -EBUSY;
+ }
+
+ mutex_lock(&kvm->arch.config_lock);
+
+ if (!vgic_initialized(kvm)) {
+ ret = -EBUSY;
+ goto out_unlock;
+ }
+
+ if (kvm_vm_has_ran_once(kvm)) {
+ ret = -EBUSY;
+ goto out_unlock;
+ }
+
+ ret = vgic_v5_copy_ist_attr(attr, &ist_attr);
+ if (ret)
+ goto out_unlock;
+
+ ret = vgic_v5_validate_ist_attr(kvm, &ist_attr);
+ if (ret)
+ goto out_unlock;
+
+ ret = vgic_v5_lpi_ist_exists(kvm);
+ if (ret) {
+ if (ret > 0)
+ ret = -EBUSY;
+ goto out_unlock;
+ }
+
+ /*
+ * If the guest has previously allocated an IST (which we check based on
+ * the IRS_IST_BASER), extract the number of LPI ID bits from the
+ * IRS_IST_CFGR. Else, do nothing.
+ *
+ * We do this before making the VMTE invalid as we rely on
+ * IRS_VMAP_VISTR to mark the IST as valid in the VMTE. This can only
+ * happen while the VMTE is valid.
+ */
+ ret = vgic_v5_restore_lpi_ist_alloc(kvm, &lpi_ist_allocated);
+ if (ret)
+ goto out_unlock;
+
+ /*
+ * Host ISTs are updated while the VMTE is invalid, so the GIC cannot
+ * observe partially restored state.
+ */
+ ret = vgic_v5_send_command(vcpu0, VMTE_MAKE_INVALID);
+ if (ret) {
+ /*
+ * If invalidation fails, the restore cannot safely update host
+ * IST state.
+ */
+ goto out_unlock;
+ }
+ vmte_invalid = true;
+
+ /* Restore the SPI IST from the userspace buffer. */
+ ret = vgic_v5_restore_spi_ist(kvm, &ist_attr);
+ if (ret)
+ goto out_unlock;
+
+ /* Restore the LPI IST from the userspace buffer. */
+ if (lpi_ist_allocated) {
+ ret = vgic_v5_restore_lpi_ist(kvm, &ist_attr);
+ if (ret)
+ goto out_unlock;
+ }
+
+ /* And make the VM Valid again */
+ ret = vgic_v5_send_command(vcpu0, VMTE_MAKE_VALID);
+ if (ret)
+ goto out_unlock;
+ vmte_invalid = false;
+
+ /*
+ * As part of restoring the ISTs, and previously pending interrupts have
+ * been tracked and made non-pending. Now that the ISTs have been
+ * restored, and the VM is valid again, restore the pending interrupts.
+ */
+ ret = vgic_v5_restore_pending_irqs(kvm);
+ if (ret)
+ goto out_unlock;
+
+out_unlock:
+ if (ret && (vmte_invalid || lpi_ist_allocated)) {
+ vgic_v5_discard_pending_irqs(kvm);
+ vgic_v5_restore_cleanup(kvm, vcpu0, lpi_ist_allocated);
+ }
+
+ mutex_unlock(&kvm->arch.config_lock);
+ kvm_unlock_all_vcpus(kvm);
+ mutex_unlock(&kvm->lock);
+
+ return ret;
+}
diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c
index ccb7e3a90cd07a..a27895295ecbb8 100644
--- a/arch/arm64/kvm/vgic/vgic.c
+++ b/arch/arm64/kvm/vgic/vgic.c
@@ -86,19 +86,31 @@ static struct vgic_irq *vgic_get_lpi(struct kvm *kvm, u32 intid)
*/
struct vgic_irq *vgic_get_irq(struct kvm *kvm, u32 intid)
{
- /* Non-private IRQs are not yet implemented for GICv5 */
- if (vgic_is_v5(kvm))
- return NULL;
+ enum kvm_device_type type = kvm->arch.vgic.vgic_model;
/* SPIs */
- if (intid >= VGIC_NR_PRIVATE_IRQS &&
- intid < (kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS)) {
- intid = array_index_nospec(intid, kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS);
- return &kvm->arch.vgic.spis[intid - VGIC_NR_PRIVATE_IRQS];
+ if (__irq_is_spi(type, intid)) {
+ switch (type) {
+ case KVM_DEV_TYPE_ARM_VGIC_V5:
+ intid = vgic_v5_get_hwirq_id(intid);
+
+ if (intid >= kvm->arch.vgic.nr_spis)
+ return NULL;
+
+ intid = array_index_nospec(intid, kvm->arch.vgic.nr_spis);
+ return &kvm->arch.vgic.spis[intid];
+ default: {
+ u32 max_intid = kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
+
+ if (intid < max_intid) {
+ intid = array_index_nospec(intid, max_intid);
+ return &kvm->arch.vgic.spis[intid - VGIC_NR_PRIVATE_IRQS];
+ }
+ }}
}
/* LPIs */
- if (irq_is_lpi(kvm, intid))
+ if (__irq_is_lpi(type, intid))
return vgic_get_lpi(kvm, intid);
return NULL;
@@ -564,6 +576,9 @@ int kvm_vgic_inject_irq(struct kvm *kvm, struct kvm_vcpu *vcpu,
else
irq->pending_latch = true;
+ if (irq->ops && irq->ops->set_pending_state)
+ WARN_ON_ONCE(!irq->ops->set_pending_state(vcpu, irq));
+
vgic_queue_irq_unlock(kvm, irq, flags);
vgic_put_irq(kvm, irq);
@@ -1235,8 +1250,12 @@ int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)
unsigned long flags;
struct vgic_vmcr vmcr;
- if (vgic_is_v5(vcpu->kvm))
+ if (vgic_is_v5(vcpu->kvm)) {
+ if (READ_ONCE(vcpu->arch.vgic_cpu.vgic_v5.gicv5_vpe.db_fired))
+ return true;
+
return vgic_v5_has_pending_ppi(vcpu);
+ }
if (!vcpu->kvm->arch.vgic.enabled)
return false;
diff --git a/arch/arm64/kvm/vgic/vgic.h b/arch/arm64/kvm/vgic/vgic.h
index f45f7e3ec4d6ef..cb673da96ec55b 100644
--- a/arch/arm64/kvm/vgic/vgic.h
+++ b/arch/arm64/kvm/vgic/vgic.h
@@ -252,6 +252,8 @@ struct ap_list_summary {
#define irqs_active_outside_lrs(s) \
((s)->nr_act && irqs_outside_lrs(s))
+int vgic_v5_parse_attr(struct kvm_device *dev, struct kvm_device_attr *attr,
+ struct vgic_reg_attr *reg_attr);
int vgic_v3_parse_attr(struct kvm_device *dev, struct kvm_device_attr *attr,
struct vgic_reg_attr *reg_attr);
int vgic_v2_parse_attr(struct kvm_device *dev, struct kvm_device_attr *attr,
@@ -366,8 +368,14 @@ void vgic_debug_destroy(struct kvm *kvm);
int vgic_v5_probe(const struct gic_kvm_info *info);
void vgic_v5_reset(struct kvm_vcpu *vcpu);
int vgic_v5_init(struct kvm *kvm);
+int kvm_vgic_v5_irs_init(struct kvm *kvm, unsigned int nr_spis);
+void vgic_v5_teardown(struct kvm *kvm);
int vgic_v5_map_resources(struct kvm *kvm);
void vgic_v5_set_ppi_ops(struct kvm_vcpu *vcpu, u32 vintid);
+void vgic_v5_set_spi_ops(struct vgic_irq *irq);
+bool vgic_v5_spi_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq,
+ unsigned long flags);
+void vgic_v5_set_irq_pend(struct kvm_vcpu *vcpu, struct vgic_irq *irq);
bool vgic_v5_has_pending_ppi(struct kvm_vcpu *vcpu);
void vgic_v5_flush_ppi_state(struct kvm_vcpu *vcpu);
void vgic_v5_fold_ppi_state(struct kvm_vcpu *vcpu);
@@ -377,6 +385,19 @@ void vgic_v5_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
void vgic_v5_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
void vgic_v5_restore_state(struct kvm_vcpu *vcpu);
void vgic_v5_save_state(struct kvm_vcpu *vcpu);
+int vgic_v5_register_irs_iodev(struct kvm *kvm, gpa_t irs_base_address);
+int vgic_v5_irs_lpi_ist_id_bits(struct kvm *kvm, unsigned int *id_bits);
+
+int vgic_v5_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu,
+ struct kvm_device_attr *attr, bool is_write);
+int vgic_v5_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr);
+const struct sys_reg_desc *vgic_v5_get_sysreg_table(unsigned int *sz);
+int vgic_v5_irs_save_ists(struct kvm *kvm, struct kvm_device_attr *attr);
+int vgic_v5_irs_restore_ists(struct kvm *kvm, struct kvm_device_attr *attr);
+int vgic_v5_irs_attr_regs_access(struct kvm_device *dev,
+ struct kvm_device_attr *attr,
+ u64 *reg, bool is_write);
+int vgic_v5_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr);
#define for_each_visible_v5_ppi(__i, __k) \
for_each_set_bit(__i, (__k)->arch.vgic.gicv5_vm.vgic_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS)
diff --git a/arch/arm64/tools/sysreg b/arch/arm64/tools/sysreg
index 7cb61aca3797fa..03f5e1755a6333 100644
--- a/arch/arm64/tools/sysreg
+++ b/arch/arm64/tools/sysreg
@@ -3736,7 +3736,9 @@ Sysreg ICC_CR0_EL1 3 1 12 0 1
Res0 63:39
Field 38 PID
Field 37:32 IPPT
-Res0 31:1
+Res0 31:3
+Field 2 LINK_IDLE
+Field 1 LINK
Field 0 EN
EndSysreg
diff --git a/drivers/irqchip/irq-gic-v5-irs.c b/drivers/irqchip/irq-gic-v5-irs.c
index f3fce0b1e25d96..70502b07ec8d72 100644
--- a/drivers/irqchip/irq-gic-v5-irs.c
+++ b/drivers/irqchip/irq-gic-v5-irs.c
@@ -21,8 +21,6 @@
*/
#define LPI_ID_BITS_LINEAR 12
-#define IRS_FLAGS_NON_COHERENT BIT(0)
-
static DEFINE_PER_CPU_READ_MOSTLY(struct gicv5_irs_chip_data *, per_cpu_irs_data);
static LIST_HEAD(irs_nodes);
@@ -50,6 +48,11 @@ static void irs_writeq_relaxed(struct gicv5_irs_chip_data *irs_data,
writeq_relaxed(val, irs_data->irs_base + reg_offset);
}
+struct gicv5_irs_chip_data *gicv5_irs_get_chip_data(void)
+{
+ return per_cpu(per_cpu_irs_data, 0);
+}
+
/*
* The polling wait (in gicv5_wait_for_op_s_atomic()) on a GIC register
* provides the memory barriers (through MMIO accessors)
@@ -266,24 +269,24 @@ int gicv5_irs_iste_alloc(const u32 lpi)
* itself is not supported) again serves to make it easier to find physically
* contiguous blocks of memory.
*/
-static unsigned int gicv5_irs_l2_sz(u32 idr2)
+unsigned int gicv5_irs_l2_sz(u32 l2sz)
{
switch (PAGE_SIZE) {
case SZ_64K:
- if (GICV5_IRS_IST_L2SZ_SUPPORT_64KB(idr2))
+ if (GICV5_IRS_IST_L2SZ_SUPPORT_64KB(l2sz))
return GICV5_IRS_IST_CFGR_L2SZ_64K;
fallthrough;
case SZ_4K:
- if (GICV5_IRS_IST_L2SZ_SUPPORT_4KB(idr2))
+ if (GICV5_IRS_IST_L2SZ_SUPPORT_4KB(l2sz))
return GICV5_IRS_IST_CFGR_L2SZ_4K;
fallthrough;
case SZ_16K:
- if (GICV5_IRS_IST_L2SZ_SUPPORT_16KB(idr2))
+ if (GICV5_IRS_IST_L2SZ_SUPPORT_16KB(l2sz))
return GICV5_IRS_IST_CFGR_L2SZ_16K;
break;
}
- if (GICV5_IRS_IST_L2SZ_SUPPORT_4KB(idr2))
+ if (GICV5_IRS_IST_L2SZ_SUPPORT_4KB(l2sz))
return GICV5_IRS_IST_CFGR_L2SZ_4K;
return GICV5_IRS_IST_CFGR_L2SZ_64K;
@@ -331,7 +334,7 @@ static int __init gicv5_irs_init_ist(struct gicv5_irs_chip_data *irs_data)
lpi_id_bits = min(lpi_id_bits, gicv5_global_data.cpuif_id_bits);
if (two_levels)
- l2sz = gicv5_irs_l2_sz(idr2);
+ l2sz = gicv5_irs_l2_sz(FIELD_GET(GICV5_IRS_IDR2_IST_L2SZ, idr2));
istmd = !!FIELD_GET(GICV5_IRS_IDR2_ISTMD, idr2);
diff --git a/drivers/irqchip/irq-gic-v5.c b/drivers/irqchip/irq-gic-v5.c
index e9d1795235a66f..0d90675fb319bb 100644
--- a/drivers/irqchip/irq-gic-v5.c
+++ b/drivers/irqchip/irq-gic-v5.c
@@ -1126,8 +1126,10 @@ static void gicv5_set_cpuif_idbits(void)
#ifdef CONFIG_KVM
static struct gic_kvm_info gic_v5_kvm_info __initdata;
-static void __init gic_of_setup_kvm_info(struct device_node *node)
+static void __init gic_setup_kvm_info(unsigned int maint_irq)
{
+ struct gicv5_irs_chip_data *irs_data = gicv5_irs_get_chip_data();
+
/*
* If we don't have native GICv5 virtualisation support, then
* we also don't have FEAT_GCIE_LEGACY - the architecture
@@ -1135,25 +1137,127 @@ static void __init gic_of_setup_kvm_info(struct device_node *node)
*/
if (!gicv5_global_data.virt_capable) {
pr_info("GIC implementation is not virtualization capable\n");
- return;
+ goto out_dispose_maint_irq;
}
+ if (WARN_ON(!irs_data))
+ goto out_dispose_maint_irq;
+
gic_v5_kvm_info.type = GIC_V5;
+ gic_v5_kvm_info.gicv5_irs.base = irs_data->irs_base;
+ gic_v5_kvm_info.gicv5_irs.non_coherent = !!(irs_data->flags & IRS_FLAGS_NON_COHERENT);
/* GIC Virtual CPU interface maintenance interrupt */
gic_v5_kvm_info.no_maint_irq_mask = false;
- gic_v5_kvm_info.maint_irq = irq_of_parse_and_map(node, 0);
- if (!gic_v5_kvm_info.maint_irq) {
- pr_warn("cannot find GICv5 virtual CPU interface maintenance interrupt\n");
+ gic_v5_kvm_info.maint_irq = maint_irq;
+
+ /*
+ * We require an MI if we have legacy support, but don't, otherwise.
+ * Given that there's an existing flag to convey that an MI isn't
+ * needed, we (ab)use it to tell KVM that the MI isn't needed if we
+ * don't support legacy.
+ *
+ * The check for ARM64_HAS_GICV5_LEGACY explicitly doesn't use
+ * cpus_have_final_cap() here as we run too early.
+ */
+ if (!cpus_have_cap(ARM64_HAS_GICV5_LEGACY) && !gic_v5_kvm_info.maint_irq)
+ gic_v5_kvm_info.no_maint_irq_mask = true;
+
+ vgic_set_kvm_info(&gic_v5_kvm_info);
+ return;
+
+out_dispose_maint_irq:
+ irq_dispose_mapping(maint_irq);
+}
+
+static void __init gic_of_setup_kvm_info(struct device_node *node)
+{
+ /* GIC Virtual CPU interface maintenance interrupt */
+ gic_setup_kvm_info(irq_of_parse_and_map(node, 0));
+}
+
+#ifdef CONFIG_ACPI
+struct gicv5_acpi_kvm_info {
+ u32 maint_irq;
+ int maint_irq_mode;
+};
+
+static struct gicv5_acpi_kvm_info acpi_v5_kvm_info __initdata;
+
+static int __init gic_acpi_parse_virt_madt_gicc(union acpi_subtable_headers *header,
+ const unsigned long end)
+{
+ struct acpi_madt_generic_interrupt *gicc =
+ (struct acpi_madt_generic_interrupt *)header;
+ static int first_madt = true;
+ int maint_irq_mode;
+
+ if (!(gicc->flags &
+ (ACPI_MADT_ENABLED | ACPI_MADT_GICC_ONLINE_CAPABLE)))
+ return 0;
+
+ maint_irq_mode = (gicc->flags & ACPI_MADT_VGIC_IRQ_MODE) ?
+ ACPI_EDGE_SENSITIVE : ACPI_LEVEL_SENSITIVE;
+
+ if (first_madt) {
+ first_madt = false;
+
+ acpi_v5_kvm_info.maint_irq = gicc->vgic_interrupt;
+ acpi_v5_kvm_info.maint_irq_mode = maint_irq_mode;
+ return 0;
+ }
+
+ /* The maintenance interrupt must be the same for every GICC entry. */
+ if (acpi_v5_kvm_info.maint_irq != gicc->vgic_interrupt ||
+ acpi_v5_kvm_info.maint_irq_mode != maint_irq_mode)
+ return -EINVAL;
+
+ return 0;
+}
+
+static bool __init gic_acpi_collect_virt_info(void)
+{
+ int count;
+
+ count = acpi_table_parse_madt(ACPI_MADT_TYPE_GENERIC_INTERRUPT,
+ gic_acpi_parse_virt_madt_gicc, 0);
+
+ return count > 0;
+}
+
+static void __init gic_acpi_setup_kvm_info(void)
+{
+ unsigned int maint_irq = 0;
+ int irq;
+
+ if (!gic_acpi_collect_virt_info()) {
+ pr_warn("Unable to get hardware information used for virtualization\n");
return;
}
- vgic_set_kvm_info(&gic_v5_kvm_info);
+ if (acpi_v5_kvm_info.maint_irq) {
+ irq = acpi_register_gsi(NULL, acpi_v5_kvm_info.maint_irq,
+ acpi_v5_kvm_info.maint_irq_mode,
+ ACPI_ACTIVE_HIGH);
+ if (irq > 0)
+ maint_irq = irq;
+ else
+ pr_warn("Failed to register GSI for GICv5 maintenance IRQ\n");
+ }
+
+ gic_setup_kvm_info(maint_irq);
}
+#endif
#else
static inline void __init gic_of_setup_kvm_info(struct device_node *node)
{
}
+
+#ifdef CONFIG_ACPI
+static inline void __init gic_acpi_setup_kvm_info(void)
+{
+}
+#endif
#endif // CONFIG_KVM
static int __init gicv5_init_common(struct fwnode_handle *parent_domain)
@@ -1252,6 +1356,7 @@ static int __init gic_acpi_init(union acpi_subtable_headers *header, const unsig
goto out_irs;
acpi_set_irq_model(ACPI_IRQ_MODEL_GIC_V5, gic_v5_get_gsi_domain_id);
+ gic_acpi_setup_kvm_info();
return 0;
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index fe49fb56dc3c92..d3dcf1598c4210 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -40,6 +40,8 @@
* in KVM for now. At a future stage, this can be bumped up to 128, if required.
*/
#define VGIC_V5_NR_PRIVATE_IRQS 64
+#define VGIC_V5_DEFAULT_NR_SPIS 32
+#define VGIC_V5_MAX_NR_SPIS BIT(16)
#define is_v5_type(t, i) (FIELD_GET(GICV5_HWIRQ_TYPE, (i)) == (t))
@@ -157,9 +159,16 @@ struct vgic_global {
/* Maintenance IRQ number */
unsigned int maint_irq;
- /* maximum number of VCPUs allowed (GICv2 limits us to 8) */
+ /*
+ * Maximum number of VCPUs exposed before userspace has selected a
+ * VGIC model. Individual VGIC models can impose a lower limit
+ * (GICv2 limits us to 8).
+ */
int max_gic_vcpus;
+ /* Maximum number of VCPUs allowed for a GICv5 VM. */
+ int max_gicv5_vcpus;
+
/* Only needed for the legacy KVM_CREATE_IRQCHIP */
bool can_emulate_gicv2;
@@ -182,6 +191,32 @@ struct vgic_global {
struct {
DECLARE_BITMAP(impl_ppi_mask, VGIC_V5_NR_PRIVATE_IRQS);
} vgic_v5_ppi_caps;
+
+ /* GICv5 IRS capabilities */
+ struct {
+ /* Base address of the host IRS's CONFIG_FRAME */
+ void __iomem *irs_base;
+
+ /* IST Caps */
+ u8 ist_id_bits;
+ bool ist_levels;
+ u8 ist_l2sz;
+ bool istmd;
+ u8 istmd_sz;
+
+ /* LPI only */
+ u8 min_lpi_id_bits;
+
+ /* VM Table, VPE Table */
+ bool two_level_vmt_support;
+ u32 max_vms;
+ u32 max_vpes;
+ u16 vmd_size;
+ u16 vped_size;
+
+ /* Is the IRS coherent with us, or not? */
+ bool non_coherent;
+ } vgic_v5_irs_caps;
};
extern struct vgic_global kvm_vgic_global_state;
@@ -216,6 +251,12 @@ struct irq_ops {
*/
bool (*get_input_level)(int vintid);
+ /*
+ * Function pointer to directly update hardware pending state after the
+ * VGIC shadow pending state has changed.
+ */
+ bool (*set_pending_state)(struct kvm_vcpu *vcpu, struct vgic_irq *irq);
+
/*
* Function pointer to override the queuing of an IRQ.
*/
@@ -290,7 +331,8 @@ enum iodev_type {
IODEV_CPUIF,
IODEV_DIST,
IODEV_REDIST,
- IODEV_ITS
+ IODEV_ITS,
+ IODEV_GICV5_IRS
};
struct vgic_io_device {
@@ -348,6 +390,8 @@ struct vgic_redist_region {
struct list_head list;
};
+#define VGIC_V5_VM_ID_INVAL (-1)
+
struct vgic_v5_vm {
/*
* We only expose a subset of PPIs to the guest. This subset is a
@@ -366,6 +410,93 @@ struct vgic_v5_vm {
* convenient way to do that).
*/
DECLARE_BITMAP(vgic_ppi_hmr, VGIC_V5_NR_PRIVATE_IRQS);
+
+ struct fwnode_handle *fwnode;
+ struct irq_domain *domain;
+ int vpe_db_base;
+ u32 vm_id;
+ bool vmte_allocated;
+};
+
+/*** GICv5 ***/
+struct vgic_v5_irs {
+ /* base addresses in guest physical address space: */
+ gpa_t vgic_v5_irs_base;
+
+ struct vgic_io_device iodev;
+ struct kvm_device *dev;
+
+ /* IRS state - used for registers etc */
+ struct {
+ u8 domain;
+ u8 pa_range;
+ bool virt;
+ bool setlpi;
+ bool mec;
+ bool mpam;
+ bool swe;
+ u16 irs_id;
+ } idr0;
+
+ struct {
+ /* PE_CNT is populated from online_vcpus at runtime */
+ u8 priority_bits;
+ } idr1;
+
+ struct {
+ u8 id_bits;
+ u8 min_lpi_id_bits;
+ bool ist_levels;
+ u8 ist_l2sz;
+ bool istmd;
+ u8 istmd_sz;
+ } idr2;
+
+ struct {
+ u32 spi_range;
+ } idr5;
+
+ struct {
+ u32 spi_irs_range;
+ } idr6;
+
+ struct {
+ u32 spi_base;
+ } idr7;
+
+ struct {
+ u8 sh;
+ u8 oc;
+ u8 ic;
+ bool ist_ra;
+ bool ist_wa;
+ bool vmt_ra;
+ bool vpet_ra;
+ bool vmd_ra;
+ bool vmd_wa;
+ bool vped_ra;
+ bool vped_wa;
+ } cr1;
+
+ struct {
+ u32 id;
+ } spi_selr;
+
+ struct {
+ u32 iaffid;
+ } pe_selr;
+
+ struct {
+ u8 lpi_id_bits;
+ u8 l2sz;
+ u8 istsz;
+ bool structure;
+ } ist_cfgr;
+
+ struct {
+ bool valid;
+ u64 addr;
+ } ist_baser;
};
struct vgic_dist {
@@ -445,6 +576,11 @@ struct vgic_dist {
* GICv5 per-VM data.
*/
struct vgic_v5_vm gicv5_vm;
+
+ /*
+ * GICv5 IRS data. Dynamically allocated due to the size.
+ */
+ struct vgic_v5_irs *vgic_v5_irs_data;
};
struct vgic_v2_cpu_if {
@@ -495,6 +631,9 @@ struct vgic_v5_cpu_if {
*/
u64 vgic_icsr;
+ /* The contextr used to make VPEs resident and non-resident */
+ u64 vgic_contextr;
+
struct gicv5_vpe gicv5_vpe;
};
@@ -601,10 +740,11 @@ void kvm_vgic_process_async_update(struct kvm_vcpu *vcpu);
void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg, bool allow_group1);
/**
- * kvm_vgic_get_max_vcpus - Get the maximum number of VCPUs allowed by HW
+ * kvm_vgic_get_max_vcpus - Get the pre-VGIC-selection VCPU limit
*
- * The host's GIC naturally limits the maximum amount of VCPUs a guest
- * can use.
+ * Userspace can query KVM_CAP_MAX_VCPUS before selecting a VGIC model, so
+ * expose the highest model-specific limit and let kvm_vgic_create() enforce
+ * the selected model's actual limit.
*/
static inline int kvm_vgic_get_max_vcpus(void)
{
@@ -630,7 +770,7 @@ int vgic_v4_load(struct kvm_vcpu *vcpu);
void vgic_v4_commit(struct kvm_vcpu *vcpu);
int vgic_v4_put(struct kvm_vcpu *vcpu);
-int vgic_v5_finalize_ppi_state(struct kvm *kvm);
+int vgic_v5_finalize_ppi_state(struct kvm_vcpu *vcpu);
bool vgic_v5_ppi_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq,
unsigned long flags);
void vgic_v5_set_ppi_dvi(struct kvm_vcpu *vcpu, struct vgic_irq *irq, bool dvi);
diff --git a/include/linux/irqchip/arm-gic-v5.h b/include/linux/irqchip/arm-gic-v5.h
index f78787e654f4c6..8b6c497d3b5902 100644
--- a/include/linux/irqchip/arm-gic-v5.h
+++ b/include/linux/irqchip/arm-gic-v5.h
@@ -62,20 +62,34 @@
#define GICV5_OUTER_SHARE 0b10
#define GICV5_INNER_SHARE 0b11
+#define GICV5_AIDR_COMPONENT_IRS 0b00
+#define GICV5_AIDR_COMPONENT_ITS 0b01
+#define GICV5_AIDR_COMPONENT_IWB 0b10
+
+#define GICV5_AIDR_ARCH_MAJ_REV_V5 0
+#define GICV5_AIDR_ARCH_MIN_REV_V0 0
+
/*
* IRS registers and tables structures
*/
#define GICV5_IRS_IDR0 0x0000
#define GICV5_IRS_IDR1 0x0004
#define GICV5_IRS_IDR2 0x0008
+#define GICV5_IRS_IDR3 0x000c
+#define GICV5_IRS_IDR4 0x0010
#define GICV5_IRS_IDR5 0x0014
#define GICV5_IRS_IDR6 0x0018
#define GICV5_IRS_IDR7 0x001c
+#define GICV5_IRS_IIDR 0x0040
+#define GICV5_IRS_AIDR 0x0044
#define GICV5_IRS_CR0 0x0080
#define GICV5_IRS_CR1 0x0084
#define GICV5_IRS_SYNCR 0x00c0
#define GICV5_IRS_SYNC_STATUSR 0x00c4
+#define GICV5_IRS_SPI_VMR 0x0100
#define GICV5_IRS_SPI_SELR 0x0108
+#define GICV5_IRS_SPI_DOMAINR 0x010c
+#define GICV5_IRS_SPI_RESAMPLER 0x0110
#define GICV5_IRS_SPI_CFGR 0x0114
#define GICV5_IRS_SPI_STATUSR 0x0118
#define GICV5_IRS_PE_SELR 0x0140
@@ -85,11 +99,51 @@
#define GICV5_IRS_IST_CFGR 0x0190
#define GICV5_IRS_IST_STATUSR 0x0194
#define GICV5_IRS_MAP_L2_ISTR 0x01c0
-
+#define GICV5_IRS_VMT_BASER 0x0200
+#define GICV5_IRS_VMT_CFGR 0x0210
+#define GICV5_IRS_VMT_STATUSR 0x0214
+#define GICV5_IRS_VPE_SELR 0x0240
+#define GICV5_IRS_VPE_DBR 0x0248
+#define GICV5_IRS_VPE_HPPIR 0x0250
+#define GICV5_IRS_VPE_CR0 0x0258
+#define GICV5_IRS_VPE_STATUSR 0x025c
+#define GICV5_IRS_VM_DBR 0x0280
+#define GICV5_IRS_VM_SELR 0x0288
+#define GICV5_IRS_VM_STATUSR 0x028c
+#define GICV5_IRS_VMAP_L2_VMTR 0x02c0
+#define GICV5_IRS_VMAP_VMR 0x02c8
+#define GICV5_IRS_VMAP_VISTR 0x02d0
+#define GICV5_IRS_VMAP_L2_VISTR 0x02d8
+#define GICV5_IRS_VMAP_VPER 0x02e0
+#define GICV5_IRS_SAVE_VMR 0x0300
+#define GICV5_IRS_SAVE_VM_STATUSR 0x0308
+#define GICV5_IRS_MEC_IDR 0x0340
+#define GICV5_IRS_MEC_MECID_R 0x0344
+#define GICV5_IRS_MPAM_IDR 0x0380
+#define GICV5_IRS_MPAM_PARTID_R 0x0384
+#define GICV5_IRS_SWERR_STATUSR 0x03c0
+#define GICV5_IRS_SWERR_SYNDROMER0 0x03c8
+#define GICV5_IRS_SWERR_SYNDROMER1 0x03d0
+
+#define GICV5_IRS_IDR0_IRSID GENMASK(31, 16)
+#define GICV5_IRS_IDR0_SWE BIT(12)
+#define GICV5_IRS_IDR0_MPAM BIT(11)
+#define GICV5_IRS_IDR0_MEC BIT(10)
+#define GICV5_IRS_IDR0_SETLPI BIT(9)
+#define GICV5_IRS_IDR0_VIRT_ONE_N BIT(8)
+#define GICV5_IRS_IDR0_ONE_N BIT(7)
#define GICV5_IRS_IDR0_VIRT BIT(6)
+#define GICV5_IRS_IDR0_PA_RANGE GENMASK(5, 2)
+#define GICV5_IRS_IDR0_INT_DOM GENMASK(1, 0)
+
+#define GICV5_IRS_IDR0_INT_DOM_SECURE 0b00
+#define GICV5_IRS_IDR0_INT_DOM_NON_SECURE 0b01
+#define GICV5_IRS_IDR0_INT_DOM_EL3 0b10
+#define GICV5_IRS_IDR0_INT_DOM_REALM 0b11
#define GICV5_IRS_IDR1_PRIORITY_BITS GENMASK(22, 20)
#define GICV5_IRS_IDR1_IAFFID_BITS GENMASK(19, 16)
+#define GICV5_IRS_IDR1_PE_CNT GENMASK(15, 0)
#define GICV5_IRS_IDR1_PRIORITY_BITS_1BITS 0b000
#define GICV5_IRS_IDR1_PRIORITY_BITS_2BITS 0b001
@@ -105,13 +159,30 @@
#define GICV5_IRS_IDR2_LPI BIT(5)
#define GICV5_IRS_IDR2_ID_BITS GENMASK(4, 0)
+#define GICV5_IRS_IST_L2SZ_SUPPORT_4KB(r) FIELD_GET(BIT(0), (r))
+#define GICV5_IRS_IST_L2SZ_SUPPORT_16KB(r) FIELD_GET(BIT(1), (r))
+#define GICV5_IRS_IST_L2SZ_SUPPORT_64KB(r) FIELD_GET(BIT(2), (r))
+
+#define GICV5_IRS_IDR3_VMT_LEVELS BIT(10)
+#define GICV5_IRS_IDR3_VM_ID_BITS GENMASK(9, 5)
+#define GICV5_IRS_IDR3_VMD_SZ GENMASK(4, 1)
+#define GICV5_IRS_IDR3_VMD BIT(0)
+
+#define GICV5_IRS_IDR4_VPE_ID_BITS GENMASK(9, 6)
+#define GICV5_IRS_IDR4_VPED_SZ GENMASK(5, 0)
+
#define GICV5_IRS_IDR5_SPI_RANGE GENMASK(24, 0)
#define GICV5_IRS_IDR6_SPI_IRS_RANGE GENMASK(24, 0)
#define GICV5_IRS_IDR7_SPI_BASE GENMASK(23, 0)
-#define GICV5_IRS_IST_L2SZ_SUPPORT_4KB(r) FIELD_GET(BIT(11), (r))
-#define GICV5_IRS_IST_L2SZ_SUPPORT_16KB(r) FIELD_GET(BIT(12), (r))
-#define GICV5_IRS_IST_L2SZ_SUPPORT_64KB(r) FIELD_GET(BIT(13), (r))
+#define GICV5_IRS_IIDR_PRODUCT_ID GENMASK(31, 20)
+#define GICV5_IRS_IIDR_VARIANT GENMASK(19, 16)
+#define GICV5_IRS_IIDR_REVISION GENMASK(15, 12)
+#define GICV5_IRS_IIDR_IMPLEMENTER GENMASK(11, 0)
+
+#define GICV5_IRS_AIDR_COMPONENT GENMASK(11, 8)
+#define GICV5_IRS_AIDR_ARCHMAJORREV GENMASK(7, 4)
+#define GICV5_IRS_AIDR_ARCHMINORREV GENMASK(3, 0)
#define GICV5_IRS_CR0_IDLE BIT(1)
#define GICV5_IRS_CR0_IRSEN BIT(0)
@@ -134,21 +205,39 @@
#define GICV5_IRS_SYNC_STATUSR_IDLE BIT(0)
-#define GICV5_IRS_SPI_STATUSR_V BIT(1)
-#define GICV5_IRS_SPI_STATUSR_IDLE BIT(0)
+#define GICV5_IRS_SPI_VMR_VIRT BIT_ULL(63)
+#define GICV5_IRS_SPI_VMR_VM_ID GENMASK_ULL(15, 0)
#define GICV5_IRS_SPI_SELR_ID GENMASK(23, 0)
+#define GICV5_IRS_SPI_DOMAINR_DOMAIN GENMASK(1, 0)
+
+#define GICV5_IRS_SPI_DOMAINR_DOMAIN_SECURE 0b00
+#define GICV5_IRS_SPI_DOMAINR_DOMAIN_NON_SECURE 0b01
+#define GICV5_IRS_SPI_DOMAINR_DOMAIN_EL3 0b10
+#define GICV5_IRS_SPI_DOMAINR_DOMAIN_REALM 0b11
+
+#define GICV5_IRS_SPI_RESAMPLER_ID GENMASK(23, 0)
+
#define GICV5_IRS_SPI_CFGR_TM BIT(0)
+#define GICV5_IRS_SPI_CFGR_TM_EDGE 0b0
+#define GICV5_IRS_SPI_CFGR_TM_LEVEL 0b1
+
+#define GICV5_IRS_SPI_STATUSR_V BIT(1)
+#define GICV5_IRS_SPI_STATUSR_IDLE BIT(0)
+
#define GICV5_IRS_PE_SELR_IAFFID GENMASK(15, 0)
+#define GICV5_IRS_PE_STATUSR_ONLINE BIT(2)
#define GICV5_IRS_PE_STATUSR_V BIT(1)
#define GICV5_IRS_PE_STATUSR_IDLE BIT(0)
#define GICV5_IRS_PE_CR0_DPS BIT(0)
-#define GICV5_IRS_IST_STATUSR_IDLE BIT(0)
+#define GICV5_IRS_IST_BASER_ADDR_MASK GENMASK_ULL(55, 6)
+#define GICV5_IRS_IST_BASER_VALID BIT_ULL(0)
+#define GICV5_IRS_IST_BASER_ADDR_SHIFT 6ULL
#define GICV5_IRS_IST_CFGR_STRUCTURE BIT(16)
#define GICV5_IRS_IST_CFGR_ISTSZ GENMASK(8, 7)
@@ -166,15 +255,111 @@
#define GICV5_IRS_IST_CFGR_L2SZ_16K 0b01
#define GICV5_IRS_IST_CFGR_L2SZ_64K 0b10
-#define GICV5_IRS_IST_BASER_ADDR_MASK GENMASK_ULL(55, 6)
-#define GICV5_IRS_IST_BASER_VALID BIT_ULL(0)
+#define GICV5_IRS_IST_STATUSR_IDLE BIT(0)
#define GICV5_IRS_MAP_L2_ISTR_ID GENMASK(23, 0)
+#define GICV5_IRS_VMT_BASER_ADDR GENMASK_ULL(55, 3)
+#define GICV5_IRS_VMT_BASER_ADDR_SHIFT 3ULL
+#define GICV5_IRS_VMT_BASER_VALID BIT_ULL(0)
+
+#define GICV5_IRS_VMT_CFGR_STRUCTURE_TWO_LEVEL 0b1
+#define GICV5_IRS_VMT_CFGR_STRUCTURE_LINEAR 0b0
+
+#define GICV5_IRS_VMT_CFGR_STRUCTURE BIT(16)
+#define GICV5_IRS_VMT_CFGR_VM_ID_BITS GENMASK(4, 0)
+
+#define GICV5_IRS_VMT_STATUSR_IDLE BIT(0)
+
+#define GICV5_IRS_VPE_SELR_S BIT_ULL(63)
+#define GICV5_IRS_VPE_SELR_VPE_ID GENMASK_ULL(47, 32)
+#define GICV5_IRS_VPE_SELR_VM_ID GENMASK_ULL(15, 0)
+
+#define GICV5_IRS_VPE_DBR_DBV BIT_ULL(63)
+#define GICV5_IRS_VPE_DBR_REQ_DB BIT_ULL(62)
+#define GICV5_IRS_VPE_DBR_DBPM GENMASK_ULL(36, 32)
+#define GICV5_IRS_VPE_DBR_INTID GENMASK_ULL(23, 0)
+
+#define GICV5_IRS_VPE_HPPIR_HPPIV BIT_ULL(32)
+#define GICV5_IRS_VPE_HPPIR_TYPE GENMASK_ULL(31, 29)
+#define GICV5_IRS_VPE_HPPIR_ID GENMASK_ULL(23, 0)
+
+#define GICV5_IRS_VPE_CR0_DPS BIT(0)
+
+#define GICV5_IRS_VPE_STATUSR_V BIT(1)
+#define GICV5_IRS_VPE_STATUSR_IDLE BIT(0)
+
+#define GICV5_IRS_VM_DBR_EN BIT_ULL(63)
+#define GICV5_IRS_VM_DBR_VPE_ID GENMASK_ULL(15, 0)
+
+#define GICV5_IRS_VM_SELR_VM_ID GENMASK(15, 0)
+
+#define GICV5_IRS_VM_STATUSR_V BIT(1)
+#define GICV5_IRS_VM_STATUSR_IDLE BIT(0)
+
+#define GICV5_IRS_VMAP_L2_VMTR_M BIT_ULL(63)
+#define GICV5_IRS_VMAP_L2_VMTR_VM_ID GENMASK_ULL(15, 0)
+
+#define GICV5_IRS_VMAP_VMR_M BIT_ULL(63)
+#define GICV5_IRS_VMAP_VMR_U BIT_ULL(62)
+#define GICV5_IRS_VMAP_VMR_VM_ID GENMASK_ULL(15, 0)
+
+#define GICV5_IRS_VMAP_VISTR_M BIT_ULL(63)
+#define GICV5_IRS_VMAP_VISTR_U BIT_ULL(62)
+#define GICV5_IRS_VMAP_VISTR_VM_ID GENMASK_ULL(47, 32)
+#define GICV5_IRS_VMAP_VISTR_TYPE GENMASK_ULL(31, 29)
+
+#define GICV5_IRS_VMAP_L2_VISTR_M BIT_ULL(63)
+#define GICV5_IRS_VMAP_L2_VISTR_VM_ID GENMASK_ULL(47, 32)
+#define GICV5_IRS_VMAP_L2_VISTR_TYPE GENMASK_ULL(31, 29)
+#define GICV5_IRS_VMAP_L2_VISTR_ID GENMASK_ULL(23, 0)
+
+#define GICV5_IRS_VMAP_VPER_M BIT_ULL(63)
+#define GICV5_IRS_VMAP_VPER_VM_ID GENMASK_ULL(47, 32)
+#define GICV5_IRS_VMAP_VPER_VPE_ID GENMASK_ULL(15, 0)
+
+#define GICV5_IRS_SAVE_VMR_VM_ID GENMASK_ULL(15, 0)
+#define GICV5_IRS_SAVE_VMR_Q BIT_ULL(62)
+#define GICV5_IRS_SAVE_VMR_S BIT_ULL(63)
+
+#define GICV5_IRS_SAVE_VM_STATUSR_IDLE BIT(0)
+#define GICV5_IRS_SAVE_VM_STATUSR_Q BIT(1)
+
+#define GICV5_IRS_MEC_IDR_MECIDSIZE GENMASK(3, 0)
+
+#define GICV5_IRS_MEC_MECID_R_MECID GENMASK(15, 0)
+
+#define GICV5_IRS_MPAM_IDR_HAS_MPAM_SP BIT(24)
+#define GICV5_IRS_MPAM_IDR_PMG_MAX GENMASK(23, 16)
+#define GICV5_IRS_MPAM_IDR_PARTID_MAX GENMASK(15, 0)
+
+#define GICV5_IRS_MPAM_PARTID_R_IDLE BIT(31)
+#define GICV5_IRS_MPAM_PARTID_R_MPAM_SP GENMASK(25, 24)
+#define GICV5_IRS_MPAM_PARTID_R_PMG GENMASK(23, 16)
+#define GICV5_IRS_MPAM_PARTID_R_PARTID GENMASK(15, 0)
+
+#define GICV5_IRS_SWERR_STATUSR_IMP_EC GENMASK_ULL(31, 24)
+#define GICV5_IRS_SWERR_STATUSR_EC GENMASK_ULL(23, 16)
+#define GICV5_IRS_SWERR_STATUSR_OF BIT_ULL(3)
+#define GICV5_IRS_SWERR_STATUSR_S1V BIT_ULL(2)
+#define GICV5_IRS_SWERR_STATUSR_S0V BIT_ULL(1)
+#define GICV5_IRS_SWERR_STATUSR_V BIT_ULL(0)
+
+#define GICV5_IRS_SWERR_SYNDROMER0_VIRTUAL BIT_ULL(63)
+#define GICV5_IRS_SWERR_SYNDROMER0_TYPE GENMASK_ULL(62, 60)
+#define GICV5_IRS_SWERR_SYNDROMER0_ID GENMASK_ULL(55, 32)
+#define GICV5_IRS_SWERR_SYNDROMER0_VM_ID GENMASK_ULL(15, 0)
+
+#define GICV5_IRS_SWERR_SYNDROMER1_ADDR GENMASK_ULL(55, 3)
+
#define GICV5_ISTL1E_VALID BIT_ULL(0)
+#define GICV5_IRS_ISTL1E_SIZE 8UL
#define GICV5_ISTL1E_L2_ADDR_MASK GENMASK_ULL(55, 12)
+#define GICV5_IRS_SETLPIR 0x0000
+#define GICV5_IRS_SETLPIR_ID GENMASK(23, 0)
+
/*
* ITS registers and tables structures
*/
@@ -296,6 +481,42 @@
#define GICV5_GSI_IWB_FRAME_ID GENMASK(28, 16)
#define GICV5_GSI_IWB_WIRE GENMASK(15, 0)
+/*
+ * CoreSight identification registers - ordered by increasing offset.
+ */
+#define GICV5_CORESIGHT_DEVARCH 0xffbc
+#define GICV5_CORESIGHT_PIDR4 0xffd0
+#define GICV5_CORESIGHT_PIDR5 0xffd4
+#define GICV5_CORESIGHT_PIDR6 0xffd8
+#define GICV5_CORESIGHT_PIDR7 0xffdc
+#define GICV5_CORESIGHT_PIDR0 0xffe0
+#define GICV5_CORESIGHT_PIDR1 0xffe4
+#define GICV5_CORESIGHT_PIDR2 0xffe8
+#define GICV5_CORESIGHT_PIDR3 0xffec
+#define GICV5_CORESIGHT_CIDR0 0xfff0
+#define GICV5_CORESIGHT_CIDR1 0xfff4
+#define GICV5_CORESIGHT_CIDR2 0xfff8
+#define GICV5_CORESIGHT_CIDR3 0xfffc
+
+#define GICV5_CORESIGHT_DEVARCH_VAL \
+ (FIELD_PREP(GENMASK(31, 21), 0x23b) | \
+ BIT(20) | \
+ 0x5a19)
+
+#define GICV5_CORESIGHT_PIDR4_JEP106_CONT 0x04
+#define GICV5_CORESIGHT_PIDR5_RES0 0x00
+#define GICV5_CORESIGHT_PIDR6_RES0 0x00
+#define GICV5_CORESIGHT_PIDR7_RES0 0x00
+#define GICV5_CORESIGHT_PIDR0_PART_0 0x4b
+#define GICV5_CORESIGHT_PIDR1_DES_0_PART_1 0xb0
+#define GICV5_CORESIGHT_PIDR2_DES_1 0x0b
+#define GICV5_CORESIGHT_PIDR3_REVAND_CMOD 0x00
+
+#define GICV5_CORESIGHT_CIDR0_VAL 0x0d
+#define GICV5_CORESIGHT_CIDR1_VAL 0xf0
+#define GICV5_CORESIGHT_CIDR2_VAL 0x05
+#define GICV5_CORESIGHT_CIDR3_VAL 0xb1
+
/*
* Global Data structures and functions
*/
@@ -330,6 +551,8 @@ struct gicv5_irs_chip_data {
raw_spinlock_t spi_config_lock;
};
+#define IRS_FLAGS_NON_COHERENT BIT(0)
+
static inline int gicv5_wait_for_op_s_atomic(void __iomem *addr, u32 offset,
const char *reg_s, u32 mask,
u32 *val)
@@ -377,6 +600,7 @@ void __init gicv5_free_lpi_domain(void);
int gicv5_irs_of_probe(struct device_node *parent);
int gicv5_irs_acpi_probe(void);
+struct gicv5_irs_chip_data *gicv5_irs_get_chip_data(void);
void gicv5_irs_remove(void);
int gicv5_irs_enable(void);
void gicv5_irs_its_probe(void);
@@ -385,10 +609,13 @@ int gicv5_irs_cpu_to_iaffid(int cpu_id, u16 *iaffid);
struct gicv5_irs_chip_data *gicv5_irs_lookup_by_spi_id(u32 spi_id);
int gicv5_spi_irq_set_type(struct irq_data *d, unsigned int type);
int gicv5_irs_iste_alloc(u32 lpi);
+unsigned int gicv5_irs_l2_sz(u32 l2sz);
void gicv5_irs_syncr(void);
/* Embedded in kvm.arch */
struct gicv5_vpe {
+ int db;
+ bool db_fired;
bool resident;
};
@@ -427,4 +654,15 @@ void gicv5_deinit_lpis(void);
void __init gicv5_its_of_probe(struct device_node *parent);
void __init gicv5_its_acpi_probe(void);
+
+enum gicv5_vcpu_cmd {
+ VMT_L2_MAP, /* Map in a L2 VMT - *may* happen on VM init */
+ VMTE_MAKE_VALID, /* Make the VMTE valid */
+ VMTE_MAKE_INVALID, /* Make the VMTE (et al.) invalid */
+ VPE_MAKE_VALID, /* No corresponding invalid */
+ SPI_VIST_MAKE_VALID, /* No corresponding invalid */
+ LPI_VIST_MAKE_VALID, /* Triggered by a guest */
+ LPI_VIST_MAKE_INVALID, /* Triggered by a guest */
+};
+
#endif
diff --git a/include/linux/irqchip/arm-vgic-info.h b/include/linux/irqchip/arm-vgic-info.h
index 67d9d960273b99..f05370e2debf44 100644
--- a/include/linux/irqchip/arm-vgic-info.h
+++ b/include/linux/irqchip/arm-vgic-info.h
@@ -38,6 +38,11 @@ struct gic_kvm_info {
bool has_v4_1;
/* Deactivation impared, subpar stuff */
bool no_hw_deactivation;
+ /* GICv5 IRS base */
+ struct {
+ void __iomem *base;
+ bool non_coherent;
+ } gicv5_irs;
};
#ifdef CONFIG_KVM
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 419011097fa8e2..df6356fb45b089 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1191,6 +1191,13 @@ struct kvm_device_attr {
__u64 addr; /* userspace address of attr data */
};
+struct kvm_vgic_v5_ist {
+ __u64 spi_ist_addr;
+ __u64 spi_ist_size;
+ __u64 lpi_ist_addr;
+ __u64 lpi_ist_size;
+};
+
#define KVM_DEV_VFIO_FILE 1
#define KVM_DEV_VFIO_FILE_ADD 1
diff --git a/tools/arch/arm64/include/uapi/asm/kvm.h b/tools/arch/arm64/include/uapi/asm/kvm.h
index 1c13bfa2d38aad..1b9bbeab18a4e0 100644
--- a/tools/arch/arm64/include/uapi/asm/kvm.h
+++ b/tools/arch/arm64/include/uapi/asm/kvm.h
@@ -97,6 +97,11 @@ struct kvm_regs {
#define KVM_VGIC_V3_REDIST_SIZE (2 * SZ_64K)
#define KVM_VGIC_V3_ITS_SIZE (2 * SZ_64K)
+/* Supported VGICv5 address types */
+#define KVM_VGIC_V5_ADDR_TYPE_IRS 6
+
+#define KVM_VGIC_V5_IRS_SIZE (2 * SZ_64K)
+
#define KVM_ARM_VCPU_POWER_OFF 0 /* CPU is started in OFF state */
#define KVM_ARM_VCPU_EL1_32BIT 1 /* CPU running a 32bit VM */
#define KVM_ARM_VCPU_PSCI_0_2 2 /* CPU uses PSCI v0.2 */
@@ -417,6 +422,8 @@ enum {
#define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7
#define KVM_DEV_ARM_VGIC_GRP_ITS_REGS 8
#define KVM_DEV_ARM_VGIC_GRP_MAINT_IRQ 9
+#define KVM_DEV_ARM_VGIC_GRP_IRS_REGS 10
+#define KVM_DEV_ARM_VGIC_GRP_IST 11
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT 10
#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \
(0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)
diff --git a/tools/testing/selftests/kvm/arm64/vgic_v5.c b/tools/testing/selftests/kvm/arm64/vgic_v5.c
index 96cfd6bb32f6f2..05d8be8d75b511 100644
--- a/tools/testing/selftests/kvm/arm64/vgic_v5.c
+++ b/tools/testing/selftests/kvm/arm64/vgic_v5.c
@@ -13,6 +13,28 @@
#include "vgic.h"
#define NR_VCPUS 1
+#define VGIC_V5_LPI_NR_VCPUS 2
+#define VGIC_V5_LPI_MEMSLOT 1
+#define VGIC_V5_NR_PRIVATE_IRQS 64
+#define VGIC_V5_DEFAULT_NR_SPIS 32
+#define VGIC_V5_MAX_NR_SPIS BIT(16)
+#define VGIC_V5_TEST_NR_SPIS 64
+#define VGIC_V5_IRS_SIZE 0x20000
+#define VGIC_V5_IRS_WAIT_RETRIES 1000000
+#define VGIC_V5_LPI_IST_BASE_GPA (GICV5_IRS_CONFIG_BASE_GPA + VGIC_V5_IRS_SIZE)
+#define VGIC_V5_LPI_IST_SIZE 0x10000
+#define LPI_TEST_TO_VPE1 0
+#define LPI_TEST_TO_VPE0 1
+#define RESTORE_TEST_LPI_PENDING 0
+#define RESTORE_TEST_LPI_ENABLED 1
+#define RESTORE_TEST_SPI_PRIO_HIGH 0
+#define RESTORE_TEST_SPI_PRIO_LOW 1
+#define RESTORE_TEST_SPI_PENDING_DISABLED 2
+#define RESTORE_TEST_SPI_ENABLED 3
+#define VGIC_V5_IST_PRIO_HIGH 0x08
+#define VGIC_V5_IST_PRIO_LOW 0x10
+
+static u64 max_phys_size;
struct vm_gic {
struct kvm_vm *vm;
@@ -24,74 +46,1469 @@ struct vm_gic {
#define GUEST_CMD_IRQ_DIEOI 11
#define GUEST_CMD_IS_AWAKE 12
#define GUEST_CMD_IS_READY 13
+#define GUEST_CMD_LPI_SENT 14
+#define GUEST_CMD_LPI_REPLIED 15
+#define GUEST_CMD_IST_SOURCE_READY 16
+#define GUEST_CMD_IST_SPI_INJECT_READY 17
+#define GUEST_CMD_IST_LPI_INJECT_READY 18
+
+static struct kvm_vgic_v5_ist vgic_v5_ist_attr(void *spi_ist, size_t spi_size,
+ void *lpi_ist, size_t lpi_size)
+{
+ return (struct kvm_vgic_v5_ist) {
+ .spi_ist_addr = (uintptr_t)spi_ist,
+ .spi_ist_size = spi_size,
+ .lpi_ist_addr = (uintptr_t)lpi_ist,
+ .lpi_ist_size = lpi_size,
+ };
+}
+
+static u32 spi_line_expected;
+static bool spi_line_level_sensitive;
+static bool lpi_ist_ready;
+static u32 ist_restore_irq_count[VGIC_V5_LPI_NR_VCPUS];
+
+static u64 gicv5_hwirq(u32 type, u32 intid)
+{
+ return FIELD_PREP(GICV5_HWIRQ_TYPE, type) |
+ FIELD_PREP(GICV5_HWIRQ_ID, intid);
+}
+
+static u64 gicv5_lpi_hwirq(u32 lpi)
+{
+ return gicv5_hwirq(GICV5_HWIRQ_TYPE_LPI, lpi);
+}
+
+static u64 gicv5_spi_hwirq(u32 spi)
+{
+ return gicv5_hwirq(GICV5_HWIRQ_TYPE_SPI, spi);
+}
+
+static void gicv5_setup_and_enable_hwirq(u64 hwirq, u32 target_vpe)
+{
+ u64 val;
+
+ val = hwirq | FIELD_PREP(GICV5_GIC_CDPRI_PRIORITY_MASK,
+ GICV5_IRQ_DEFAULT_PRI);
+ gic_insn(val, CDPRI);
+
+ val = hwirq | FIELD_PREP(GICV5_GIC_CDAFF_IAFFID_MASK, target_vpe);
+ gic_insn(val, CDAFF);
+
+ gic_insn(hwirq, CDEN);
+}
+
+static void gicv5_set_hwirq_priority(u64 hwirq, u32 priority)
+{
+ u64 val;
+
+ val = hwirq | FIELD_PREP(GICV5_GIC_CDPRI_PRIORITY_MASK, priority);
+ gic_insn(val, CDPRI);
+}
+
+static void gicv5_set_hwirq_affinity(u64 hwirq, u32 target_vpe)
+{
+ u64 val;
+
+ val = hwirq | FIELD_PREP(GICV5_GIC_CDAFF_IAFFID_MASK, target_vpe);
+ gic_insn(val, CDAFF);
+}
+
+static void gicv5_enable_hwirq(u64 hwirq)
+{
+ gic_insn(hwirq, CDEN);
+}
+
+static void gicv5_enable_spi(u32 spi, u32 target_vpe)
+{
+ gicv5_setup_and_enable_hwirq(gicv5_spi_hwirq(spi), target_vpe);
+}
+
+static void gicv5_enable_lpi(u32 lpi, u32 target_vpe)
+{
+ gicv5_setup_and_enable_hwirq(gicv5_lpi_hwirq(lpi), target_vpe);
+}
+
+static void gicv5_send_lpi(u32 lpi)
+{
+ u64 hwirq = gicv5_lpi_hwirq(lpi);
+
+ gic_insn(hwirq | GICV5_GIC_CDPEND_PENDING_MASK, CDPEND);
+}
+
+static void gicv5_wait_for_irs_idle(u32 reg, u32 idle)
+{
+ int i;
+
+ for (i = 0; i < VGIC_V5_IRS_WAIT_RETRIES; i++) {
+ if (readl(GICV5_IRS_CONFIG_BASE_GVA + reg) & idle)
+ return;
+
+ cpu_relax();
+ }
+
+ GUEST_FAIL("IRS operation did not become idle");
+}
+
+static void gicv5_configure_spi(u32 spi, bool level)
+{
+ u32 val;
+
+ val = FIELD_PREP(GICV5_IRS_SPI_SELR_ID, spi);
+ writel(val, GICV5_IRS_CONFIG_BASE_GVA + GICV5_IRS_SPI_SELR);
+ gicv5_wait_for_irs_idle(GICV5_IRS_SPI_STATUSR,
+ GICV5_IRS_SPI_STATUSR_IDLE);
+
+ val = level ? GICV5_IRS_SPI_CFGR_TM : 0;
+ writel(val, GICV5_IRS_CONFIG_BASE_GVA + GICV5_IRS_SPI_CFGR);
+ gicv5_wait_for_irs_idle(GICV5_IRS_SPI_STATUSR,
+ GICV5_IRS_SPI_STATUSR_IDLE);
+}
+
+static u32 gicv5_lpi_istsz(u32 idr2, u32 lpi_id_bits)
+{
+ if (!(idr2 & GICV5_IRS_IDR2_ISTMD))
+ return GICV5_IRS_IST_CFGR_ISTSZ_4;
+
+ if (lpi_id_bits >= FIELD_GET(GICV5_IRS_IDR2_ISTMD_SZ, idr2))
+ return GICV5_IRS_IST_CFGR_ISTSZ_16;
+
+ return GICV5_IRS_IST_CFGR_ISTSZ_8;
+}
+
+static void gicv5_configure_lpi_ist(void)
+{
+ u32 idr2, min_lpi_id_bits, lpi_id_bits, id_bits, istsz;
+ u64 val;
+
+ idr2 = readl(GICV5_IRS_CONFIG_BASE_GVA + GICV5_IRS_IDR2);
+ GUEST_ASSERT(idr2 & GICV5_IRS_IDR2_LPI);
+
+ min_lpi_id_bits = FIELD_GET(GICV5_IRS_IDR2_MIN_LPI_ID_BITS, idr2);
+ id_bits = FIELD_GET(GICV5_IRS_IDR2_ID_BITS, idr2);
+ /* Default to 32 LPIs, unless the min requirement is higher */
+ lpi_id_bits = max(5U, min_lpi_id_bits);
+ GUEST_ASSERT(lpi_id_bits <= id_bits);
+
+ istsz = gicv5_lpi_istsz(idr2, lpi_id_bits);
+ /*
+ * We allocate a fixed-size IST buffer on the host to avoid dynamic
+ * allocation from the guest. Make sure it fits the linear IST described
+ * by IDR2 before programming IRS_IST_BASER.
+ */
+ GUEST_ASSERT((BIT_ULL(lpi_id_bits) << (istsz + 2)) <= VGIC_V5_LPI_IST_SIZE);
+
+ val = FIELD_PREP(GICV5_IRS_IST_CFGR_LPI_ID_BITS, lpi_id_bits) |
+ FIELD_PREP(GICV5_IRS_IST_CFGR_ISTSZ, istsz);
+ writel(val, GICV5_IRS_CONFIG_BASE_GVA + GICV5_IRS_IST_CFGR);
+
+ val = FIELD_PREP(GICV5_IRS_IST_BASER_ADDR_MASK,
+ VGIC_V5_LPI_IST_BASE_GPA >> GICV5_IRS_IST_BASER_ADDR_SHIFT);
+ val |= GICV5_IRS_IST_BASER_VALID;
+ writeq(val, GICV5_IRS_CONFIG_BASE_GVA + GICV5_IRS_IST_BASER);
+
+ GUEST_ASSERT_EQ(readl(GICV5_IRS_CONFIG_BASE_GVA + GICV5_IRS_IST_STATUSR),
+ GICV5_IRS_IST_STATUSR_IDLE);
+}
+
+static void gicv5_enable_irs(void)
+{
+ writel(GICV5_IRS_CR0_IRSEN,
+ GICV5_IRS_CONFIG_BASE_GVA + GICV5_IRS_CR0);
+ gicv5_wait_for_irs_idle(GICV5_IRS_CR0, GICV5_IRS_CR0_IDLE);
+}
+
+static void gicv5_configure_test_lpis(void)
+{
+ gicv5_enable_lpi(LPI_TEST_TO_VPE1, 1);
+ gicv5_enable_lpi(LPI_TEST_TO_VPE0, 0);
+}
+
+static void gicv5_set_pcr(u32 priority)
+{
+ u64 pcr;
+
+ pcr = FIELD_PREP(ICC_PCR_EL1_PRIORITY, priority);
+ write_sysreg_s(pcr, SYS_ICC_PCR_EL1);
+ isb();
+}
+
+static void guest_ppi_irq_handler(struct ex_regs *regs)
+{
+ bool valid;
+ u32 hwirq;
+ u64 ia;
+ static int count;
+
+ /*
+ * We have pending interrupts. Should never actually enter WFI
+ * here!
+ */
+ wfi();
+ GUEST_SYNC(GUEST_CMD_IS_AWAKE);
+
+ ia = gicr_insn(CDIA);
+ valid = GICV5_GICR_CDIA_VALID(ia);
+
+ GUEST_SYNC(GUEST_CMD_IRQ_CDIA);
+
+ if (!valid)
+ return;
+
+ gsb_ack();
+ isb();
+
+ hwirq = FIELD_GET(GICV5_GICR_CDIA_INTID, ia);
+
+ gic_insn(hwirq, CDDI);
+ gic_insn(0, CDEOI);
+
+ GUEST_SYNC(GUEST_CMD_IRQ_DIEOI);
+
+ if (++count >= 2)
+ GUEST_DONE();
+
+ /* Ask for the next interrupt to be injected */
+ GUEST_SYNC(GUEST_CMD_IS_READY);
+}
+
+static void guest_code(void)
+{
+ local_irq_disable();
+
+ gicv5_cpu_enable_interrupts();
+ local_irq_enable();
+
+ /* Enable the SW_PPI (3) */
+ write_sysreg_s(BIT_ULL(3), SYS_ICC_PPI_ENABLER0_EL1);
+
+ /* Ask for the first interrupt to be injected */
+ GUEST_SYNC(GUEST_CMD_IS_READY);
+
+ /* Loop forever waiting for interrupts */
+ while (1);
+}
+
+static void guest_spi_irq_handler(struct ex_regs *regs)
+{
+ bool valid;
+ u32 hwirq;
+ u64 ia;
+
+ ia = gicr_insn(CDIA);
+ valid = GICV5_GICR_CDIA_VALID(ia);
+
+ if (!valid)
+ return;
+
+ gsb_ack();
+ isb();
+
+ hwirq = FIELD_GET(GICV5_GICR_CDIA_INTID, ia);
+
+ GUEST_ASSERT_EQ(hwirq, gicv5_spi_hwirq(READ_ONCE(spi_line_expected)));
+
+ gic_insn(hwirq, CDDI);
+ gic_insn(0, CDEOI);
+
+ GUEST_DONE();
+}
+
+static void guest_spi_line_code(void)
+{
+ local_irq_disable();
+
+ gicv5_enable_irs();
+ gicv5_cpu_enable_interrupts();
+ gicv5_configure_spi(READ_ONCE(spi_line_expected),
+ READ_ONCE(spi_line_level_sensitive));
+ gicv5_enable_spi(READ_ONCE(spi_line_expected), 0);
+
+ local_irq_enable();
+
+ GUEST_SYNC(GUEST_CMD_IS_READY);
+
+ while (1)
+ wfi();
+}
+
+static void guest_lpi_irq_handler(struct ex_regs *regs)
+{
+ u32 vcpu_id = guest_get_vcpuid();
+ u32 expected_lpi = vcpu_id ? LPI_TEST_TO_VPE1 : LPI_TEST_TO_VPE0;
+ u64 expected_hwirq = gicv5_lpi_hwirq(expected_lpi);
+ u32 hwirq;
+ u64 ia;
+
+ ia = gicr_insn(CDIA);
+ if (!GICV5_GICR_CDIA_VALID(ia))
+ return;
+
+ gsb_ack();
+ isb();
+
+ hwirq = FIELD_GET(GICV5_GICR_CDIA_INTID, ia);
+ GUEST_ASSERT_EQ(hwirq, expected_hwirq);
+
+ gic_insn(hwirq, CDDI);
+ gic_insn(0, CDEOI);
+
+ if (vcpu_id) {
+ gicv5_send_lpi(LPI_TEST_TO_VPE0);
+ GUEST_SYNC(GUEST_CMD_LPI_REPLIED);
+ while (1)
+ wfi();
+ }
+
+ GUEST_DONE();
+}
+
+static void guest_unexpected_irq_handler(struct ex_regs *regs)
+{
+ GUEST_FAIL("Unexpected IRQ");
+}
+
+static void guest_lpi_code(void)
+{
+ u32 vcpu_id = guest_get_vcpuid();
+
+ local_irq_disable();
+
+ if (!vcpu_id) {
+ gicv5_enable_irs();
+ gicv5_configure_lpi_ist();
+ gicv5_configure_test_lpis();
+ WRITE_ONCE(lpi_ist_ready, true);
+ }
+
+ /* Go bang if we run VCPU1 before VCPU0 */
+ GUEST_ASSERT(READ_ONCE(lpi_ist_ready));
+
+ gicv5_cpu_enable_interrupts();
+ local_irq_enable();
+
+ GUEST_SYNC(GUEST_CMD_IS_READY);
+
+ if (!vcpu_id) {
+ gicv5_send_lpi(LPI_TEST_TO_VPE1);
+ GUEST_SYNC(GUEST_CMD_LPI_SENT);
+ }
+
+ while (1)
+ wfi();
+}
+
+static void restore_test_setup_hwirq(u64 hwirq, u32 target_vpe,
+ u32 priority, bool enable)
+{
+ gicv5_set_hwirq_priority(hwirq, priority);
+ gicv5_set_hwirq_affinity(hwirq, target_vpe);
+
+ if (enable)
+ gicv5_enable_hwirq(hwirq);
+}
+
+/*
+ * The source VM configures SPI and LPI state for the IST save/restore test.
+ * It leaves interrupts disabled, allowing userspace to save the state before
+ * the restore VM enables interrupts and checks delivery.
+ */
+static void guest_ist_source_code(void)
+{
+ local_irq_disable();
+
+ gicv5_enable_irs();
+ gicv5_configure_lpi_ist();
+
+ restore_test_setup_hwirq(gicv5_lpi_hwirq(RESTORE_TEST_LPI_PENDING),
+ 1, VGIC_V5_IST_PRIO_HIGH, true);
+ restore_test_setup_hwirq(gicv5_lpi_hwirq(RESTORE_TEST_LPI_ENABLED),
+ 1, VGIC_V5_IST_PRIO_HIGH, true);
+ restore_test_setup_hwirq(gicv5_spi_hwirq(RESTORE_TEST_SPI_PRIO_HIGH),
+ 0, VGIC_V5_IST_PRIO_HIGH, true);
+ restore_test_setup_hwirq(gicv5_spi_hwirq(RESTORE_TEST_SPI_PRIO_LOW),
+ 0, VGIC_V5_IST_PRIO_LOW, true);
+ restore_test_setup_hwirq(gicv5_spi_hwirq(RESTORE_TEST_SPI_ENABLED),
+ 0, VGIC_V5_IST_PRIO_HIGH, true);
+ restore_test_setup_hwirq(gicv5_spi_hwirq(RESTORE_TEST_SPI_PENDING_DISABLED),
+ 0, VGIC_V5_IST_PRIO_HIGH, false);
+
+ gicv5_send_lpi(RESTORE_TEST_LPI_PENDING);
+
+ GUEST_SYNC(GUEST_CMD_IST_SOURCE_READY);
+
+ while (1)
+ wfi();
+}
+
+/*
+ * This is the restore VM for the IST save/restore test. The IRS regs and ISTs
+ * have been restored before this code runs. Based on which VPE is running, a
+ * different set of interrupts is expected. We check that we get what we
+ * expected before terminating the test.
+ *
+ * The expected interrupts for VPE0 are:
+ * - High-priority SPI, pending and enabled before save.
+ * - Low-priority SPI, pending and enabled before save. This is only delivered
+ * after the guest lowers the masked priority via ICC_PCR_EL1.
+ * - SPI that was pending but disabled before save. This is only delivered
+ * after the guest enables it after restore.
+ * - SPI that was enabled but not pending before save. Userspace injects this
+ * after restore to check that the enable state and affinity were restored.
+ *
+ * The expected interrupts for VPE1 are:
+ * - LPI that was pending and enabled before save.
+ * - LPI that was enabled but not pending before save. The guest injects this
+ * after restore to check that the enable state and affinity were restored.
+ */
+static void guest_ist_restore_irq_handler(struct ex_regs *regs)
+{
+ u32 vcpu_id = guest_get_vcpuid();
+ u32 count = ist_restore_irq_count[vcpu_id];
+ u64 hwirq, expected_hwirq = 0;
+ u64 ia;
+
+ ia = gicr_insn(CDIA);
+ if (!GICV5_GICR_CDIA_VALID(ia))
+ return;
+
+ gsb_ack();
+ isb();
+
+ hwirq = FIELD_GET(GICV5_GICR_CDIA_INTID, ia);
+
+ if (!vcpu_id) {
+ switch (count) {
+ case 0:
+ expected_hwirq = gicv5_spi_hwirq(RESTORE_TEST_SPI_PRIO_HIGH);
+ break;
+ case 1:
+ expected_hwirq = gicv5_spi_hwirq(RESTORE_TEST_SPI_PRIO_LOW);
+ break;
+ case 2:
+ expected_hwirq = gicv5_spi_hwirq(RESTORE_TEST_SPI_PENDING_DISABLED);
+ break;
+ case 3:
+ expected_hwirq = gicv5_spi_hwirq(RESTORE_TEST_SPI_ENABLED);
+ break;
+ default:
+ GUEST_FAIL("Unexpected VPE0 IST restore interrupt");
+ }
+ } else {
+ switch (count) {
+ case 0:
+ expected_hwirq = gicv5_lpi_hwirq(RESTORE_TEST_LPI_PENDING);
+ break;
+ case 1:
+ expected_hwirq = gicv5_lpi_hwirq(RESTORE_TEST_LPI_ENABLED);
+ break;
+ default:
+ GUEST_FAIL("Unexpected VPE1 IST restore interrupt");
+ }
+ }
+
+ GUEST_ASSERT_EQ(hwirq, expected_hwirq);
+
+ gic_insn(hwirq, CDDI);
+ gic_insn(0, CDEOI);
+ ist_restore_irq_count[vcpu_id]++;
+
+ if (!vcpu_id) {
+ switch (count) {
+ case 0:
+ gicv5_set_pcr(VGIC_V5_IST_PRIO_LOW);
+ break;
+ case 1:
+ gicv5_enable_hwirq(
+ gicv5_spi_hwirq(RESTORE_TEST_SPI_PENDING_DISABLED));
+ break;
+ case 2:
+ GUEST_SYNC(GUEST_CMD_IST_SPI_INJECT_READY);
+ break;
+ case 3:
+ GUEST_DONE();
+ }
+ } else {
+ switch (count) {
+ case 0:
+ gicv5_send_lpi(RESTORE_TEST_LPI_ENABLED);
+ GUEST_SYNC(GUEST_CMD_IST_LPI_INJECT_READY);
+ break;
+ case 1:
+ GUEST_DONE();
+ }
+ }
+}
+
+static void guest_ist_restore_code(void)
+{
+ local_irq_disable();
+
+ gicv5_cpu_enable_interrupts();
+
+ if (!guest_get_vcpuid())
+ gicv5_set_pcr(VGIC_V5_IST_PRIO_HIGH);
+
+ GUEST_SYNC(GUEST_CMD_IS_READY);
+
+ local_irq_enable();
+
+ while (1)
+ wfi();
+}
+
+/* we don't want to assert on run execution, hence that helper */
+static int run_vcpu(struct kvm_vcpu *vcpu)
+{
+ return __vcpu_run(vcpu) ? -errno : 0;
+}
+
+static void vm_gic_destroy(struct vm_gic *v)
+{
+ close(v->gic_fd);
+ kvm_vm_free(v->vm);
+}
+
+static void vgic_v5_map_irs(struct kvm_vm *vm)
+{
+ unsigned int nr_irs_pages;
+
+ nr_irs_pages = vm_calc_num_guest_pages(vm->mode, VGIC_V5_IRS_SIZE);
+
+ /* Map the IRS at VA == IPA so guest MMIO writes hit the IRS IODEV. */
+ virt_map(vm, GICV5_IRS_CONFIG_BASE_GPA,
+ GICV5_IRS_CONFIG_BASE_GPA, nr_irs_pages);
+}
+
+static u32 vgic_v5_irq_line_payload(u32 type, u32 num)
+{
+ return (type << KVM_ARM_IRQ_TYPE_SHIFT) |
+ FIELD_PREP(KVM_ARM_IRQ_NUM_MASK, num);
+}
+
+static int __vgic_v5_irq_line(struct kvm_vm *vm, u32 type, u32 num, int level)
+{
+ return _kvm_irq_line(vm, vgic_v5_irq_line_payload(type, num), level);
+}
+
+static void vgic_v5_spi_line(struct kvm_vm *vm, u32 spi, int level)
+{
+ int ret = __vgic_v5_irq_line(vm, KVM_ARM_IRQ_TYPE_SPI, spi, level);
+
+ TEST_ASSERT(!ret, "KVM_IRQ_LINE failed for SPI %u level %d",
+ spi, level);
+}
+
+static void vgic_v5_spi_line_vm_create(struct vm_gic *v,
+ struct kvm_vcpu **vcpu,
+ u32 nr_spis)
+{
+ u64 attr;
+
+ v->gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+ v->vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
+ v->gic_fd = kvm_create_device(v->vm, v->gic_dev_type);
+ *vcpu = vm_vcpu_add(v->vm, 0, guest_spi_line_code);
+ TEST_ASSERT(*vcpu, "Failed to create vCPU");
+
+ vm_init_descriptor_tables(v->vm);
+ vm_install_exception_handler(v->vm, VECTOR_IRQ_CURRENT,
+ guest_spi_irq_handler);
+ vcpu_init_descriptor_tables(*vcpu);
+
+ kvm_device_attr_set(v->gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS, 0,
+ &nr_spis);
+
+ attr = GICV5_IRS_CONFIG_BASE_GPA;
+ kvm_device_attr_set(v->gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+ KVM_VGIC_V5_ADDR_TYPE_IRS, &attr);
+ vgic_v5_map_irs(v->vm);
+ kvm_device_attr_set(v->gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);
+}
+
+static void vgic_v5_lpi_ist_vm_create(struct vm_gic *v,
+ struct kvm_vcpu *vcpus[VGIC_V5_LPI_NR_VCPUS],
+ void (*guest_code)(void),
+ void (*irq_handler)(struct ex_regs *))
+{
+ unsigned int nr_lpi_ist_pages;
+ u64 attr;
+ int i;
+
+ v->gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+ v->vm = __vm_create(VM_SHAPE_DEFAULT, VGIC_V5_LPI_NR_VCPUS, 0);
+ v->gic_fd = kvm_create_device(v->vm, v->gic_dev_type);
+
+ for (i = 0; i < VGIC_V5_LPI_NR_VCPUS; i++) {
+ vcpus[i] = vm_vcpu_add(v->vm, i, guest_code);
+ TEST_ASSERT(vcpus[i], "Failed to create vCPU");
+ }
+
+ vm_init_descriptor_tables(v->vm);
+ vm_install_exception_handler(v->vm, VECTOR_IRQ_CURRENT,
+ irq_handler);
+
+ for (i = 0; i < VGIC_V5_LPI_NR_VCPUS; i++)
+ vcpu_init_descriptor_tables(vcpus[i]);
+
+ attr = GICV5_IRS_CONFIG_BASE_GPA;
+ kvm_device_attr_set(v->gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+ KVM_VGIC_V5_ADDR_TYPE_IRS, &attr);
+ vgic_v5_map_irs(v->vm);
+
+ nr_lpi_ist_pages = vm_calc_num_guest_pages(v->vm->mode,
+ VGIC_V5_LPI_IST_SIZE);
+ vm_userspace_mem_region_add(v->vm, VM_MEM_SRC_ANONYMOUS,
+ VGIC_V5_LPI_IST_BASE_GPA,
+ VGIC_V5_LPI_MEMSLOT,
+ nr_lpi_ist_pages, 0);
+ /*
+ * Map the IST at VA == IPA so the guest can program the same BASER
+ * address.
+ */
+ virt_map(v->vm, VGIC_V5_LPI_IST_BASE_GPA,
+ VGIC_V5_LPI_IST_BASE_GPA, nr_lpi_ist_pages);
+
+ kvm_device_attr_set(v->gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);
+}
+
+static void vgic_v5_lpi_vm_create(struct vm_gic *v,
+ struct kvm_vcpu *vcpus[VGIC_V5_LPI_NR_VCPUS])
+{
+ vgic_v5_lpi_ist_vm_create(v, vcpus, guest_lpi_code,
+ guest_lpi_irq_handler);
+}
+
+static void vgic_v5_ist_restore_vm_create(struct vm_gic *v,
+ struct kvm_vcpu *vcpus[VGIC_V5_LPI_NR_VCPUS],
+ void (*guest_code)(void),
+ void (*irq_handler)(struct ex_regs *))
+{
+ vgic_v5_lpi_ist_vm_create(v, vcpus, guest_code, irq_handler);
+}
+
+static void vgic_v5_run_spi_line_test(u32 nr_spis, u32 expected_spi,
+ bool level_sensitive,
+ bool lower_before_run)
+{
+ struct kvm_vcpu *vcpu;
+ struct vm_gic v;
+ struct ucall uc;
+ int ret;
+
+ spi_line_expected = expected_spi;
+ spi_line_level_sensitive = level_sensitive;
+
+ vgic_v5_spi_line_vm_create(&v, &vcpu, nr_spis);
+
+ sync_global_to_guest(v.vm, spi_line_expected);
+ sync_global_to_guest(v.vm, spi_line_level_sensitive);
+
+ ret = run_vcpu(vcpu);
+ TEST_ASSERT(!ret, "Failed to run GICv5 IRQ_LINE VM");
+ TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC &&
+ uc.args[1] == GUEST_CMD_IS_READY,
+ "GICv5 IRQ_LINE guest did not become ready");
+
+ vgic_v5_spi_line(v.vm, expected_spi, 1);
+ /*
+ * For edge-triggered SPIs, a following low transition must be ignored:
+ * once the edge has made the SPI pending, it cannot be recalled. This
+ * allows us to test that.
+ */
+ if (lower_before_run)
+ vgic_v5_spi_line(v.vm, expected_spi, 0);
+
+ while (1) {
+ ret = run_vcpu(vcpu);
+ TEST_ASSERT(!ret, "Failed to run GICv5 IRQ_LINE VM");
+
+ switch (get_ucall(vcpu, &uc)) {
+ case UCALL_ABORT:
+ REPORT_GUEST_ASSERT(uc);
+ break;
+ case UCALL_DONE:
+ goto done;
+ default:
+ TEST_FAIL("Unknown ucall %lu", uc.cmd);
+ }
+ }
+
+done:
+ vm_gic_destroy(&v);
+}
+
+static void vgic_v5_expect_sync(struct kvm_vcpu *vcpu, u64 cmd,
+ const char *msg)
+{
+ struct ucall uc;
+
+ TEST_ASSERT(get_ucall(vcpu, &uc) == UCALL_SYNC && uc.args[1] == cmd,
+ "%s", msg);
+}
+
+struct vgic_region_attr {
+ u64 attr;
+ u64 size;
+ u64 alignment;
+};
+
+static const struct vgic_region_attr gic_v5_irs_region = {
+ .attr = KVM_VGIC_V5_ADDR_TYPE_IRS,
+ .size = GICV5_IRS_SIZE,
+ .alignment = GICV5_IRS_ALIGN,
+};
+
+struct vgic_sysreg_attr {
+ const char *name;
+ u32 encoding;
+};
+
+struct vgic_irs_reg_attr {
+ const char *name;
+ u64 attr;
+};
+
+#define PACK_SR(r) \
+ ((sys_reg_Op0(r) << 14) | \
+ (sys_reg_Op1(r) << 11) | \
+ (sys_reg_CRn(r) << 7) | \
+ (sys_reg_CRm(r) << 3) | \
+ (sys_reg_Op2(r)))
+
+#define SR(r) \
+ { \
+ .name = #r, \
+ .encoding = r, \
+ }
+
+#define IRS_REG(r) \
+ { \
+ .name = #r, \
+ .attr = r, \
+ }
+
+static const struct vgic_irs_reg_attr gic_v5_irs_regs[] = {
+ IRS_REG(GICV5_IRS_IDR0),
+ IRS_REG(GICV5_IRS_IDR1),
+ IRS_REG(GICV5_IRS_IDR2),
+ IRS_REG(GICV5_IRS_IDR3),
+ IRS_REG(GICV5_IRS_IDR4),
+ IRS_REG(GICV5_IRS_IDR5),
+ IRS_REG(GICV5_IRS_IDR6),
+ IRS_REG(GICV5_IRS_IDR7),
+ IRS_REG(GICV5_IRS_IIDR),
+ IRS_REG(GICV5_IRS_AIDR),
+ IRS_REG(GICV5_IRS_CR0),
+ IRS_REG(GICV5_IRS_CR1),
+ IRS_REG(GICV5_IRS_SYNCR),
+ IRS_REG(GICV5_IRS_SYNC_STATUSR),
+ IRS_REG(GICV5_IRS_SPI_VMR),
+ IRS_REG(GICV5_IRS_SPI_SELR),
+ IRS_REG(GICV5_IRS_SPI_DOMAINR),
+ IRS_REG(GICV5_IRS_SPI_RESAMPLER),
+ IRS_REG(GICV5_IRS_SPI_CFGR),
+ IRS_REG(GICV5_IRS_SPI_STATUSR),
+ IRS_REG(GICV5_IRS_PE_SELR),
+ IRS_REG(GICV5_IRS_PE_STATUSR),
+ IRS_REG(GICV5_IRS_PE_CR0),
+ IRS_REG(GICV5_IRS_IST_BASER),
+ IRS_REG(GICV5_IRS_IST_CFGR),
+ IRS_REG(GICV5_IRS_IST_STATUSR),
+ IRS_REG(GICV5_IRS_MAP_L2_ISTR),
+ IRS_REG(GICV5_IRS_VMT_BASER),
+ IRS_REG(GICV5_IRS_VMT_CFGR),
+ IRS_REG(GICV5_IRS_VMT_STATUSR),
+ IRS_REG(GICV5_IRS_VPE_SELR),
+ IRS_REG(GICV5_IRS_VPE_DBR),
+ IRS_REG(GICV5_IRS_VPE_HPPIR),
+ IRS_REG(GICV5_IRS_VPE_CR0),
+ IRS_REG(GICV5_IRS_VPE_STATUSR),
+ IRS_REG(GICV5_IRS_VM_DBR),
+ IRS_REG(GICV5_IRS_VM_SELR),
+ IRS_REG(GICV5_IRS_VM_STATUSR),
+ IRS_REG(GICV5_IRS_VMAP_L2_VMTR),
+ IRS_REG(GICV5_IRS_VMAP_VMR),
+ IRS_REG(GICV5_IRS_VMAP_VISTR),
+ IRS_REG(GICV5_IRS_VMAP_L2_VISTR),
+ IRS_REG(GICV5_IRS_VMAP_VPER),
+ IRS_REG(GICV5_IRS_SAVE_VMR),
+ IRS_REG(GICV5_IRS_SAVE_VM_STATUSR),
+ IRS_REG(GICV5_IRS_MEC_IDR),
+ IRS_REG(GICV5_IRS_MEC_MECID_R),
+ IRS_REG(GICV5_IRS_MPAM_IDR),
+ IRS_REG(GICV5_IRS_MPAM_PARTID_R),
+ IRS_REG(GICV5_IRS_SWERR_STATUSR),
+ IRS_REG(GICV5_IRS_SWERR_SYNDROMER0),
+ IRS_REG(GICV5_IRS_SWERR_SYNDROMER1),
+};
+
+static const struct vgic_sysreg_attr gic_v5_cpu_sysregs[] = {
+ SR(SYS_ICC_ICSR_EL1),
+ SR(SYS_ICC_PPI_ENABLER0_EL1),
+ SR(SYS_ICC_PPI_ENABLER1_EL1),
+ SR(SYS_ICC_PPI_SACTIVER0_EL1),
+ SR(SYS_ICC_PPI_SACTIVER1_EL1),
+ SR(SYS_ICC_PPI_SPENDR0_EL1),
+ SR(SYS_ICC_PPI_SPENDR1_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR0_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR1_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR2_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR3_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR4_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR5_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR6_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR7_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR8_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR9_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR10_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR11_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR12_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR13_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR14_EL1),
+ SR(SYS_ICC_PPI_PRIORITYR15_EL1),
+ SR(SYS_ICC_APR_EL1),
+ SR(SYS_ICC_CR0_EL1),
+ SR(SYS_ICC_PCR_EL1),
+};
+
+static void vgic_v5_save_irs_regs(struct vm_gic *v, u64 *regs)
+{
+ int ret, i;
+
+ for (i = 0; i < ARRAY_SIZE(gic_v5_irs_regs); i++) {
+ ret = __kvm_device_attr_get(v->gic_fd,
+ KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ gic_v5_irs_regs[i].attr,
+ ®s[i]);
+ TEST_ASSERT(!ret, "GICv5 IRS_REGS save failed for %s",
+ gic_v5_irs_regs[i].name);
+ }
+}
+
+static void vgic_v5_restore_irs_regs(struct vm_gic *v, u64 *regs)
+{
+ int ret, i;
+
+ for (i = 0; i < ARRAY_SIZE(gic_v5_irs_regs); i++) {
+ ret = __kvm_device_attr_set(v->gic_fd,
+ KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ gic_v5_irs_regs[i].attr,
+ ®s[i]);
+ TEST_ASSERT(!ret, "GICv5 IRS_REGS restore failed for %s",
+ gic_v5_irs_regs[i].name);
+ }
+}
+
+static u64 vgic_v5_saved_irs_reg(u64 *regs, u64 attr)
+{
+ for (int i = 0; i < ARRAY_SIZE(gic_v5_irs_regs); i++) {
+ if (gic_v5_irs_regs[i].attr == attr)
+ return regs[i];
+ }
+
+ TEST_FAIL("GICv5 IRS register 0x%lx was not saved", attr);
+}
+
+static size_t vgic_v5_lpi_ist_state_size(u64 *irs_regs)
+{
+ u64 cfgr = vgic_v5_saved_irs_reg(irs_regs, GICV5_IRS_IST_CFGR);
+ u32 lpi_id_bits = FIELD_GET(GICV5_IRS_IST_CFGR_LPI_ID_BITS, cfgr);
+
+ return BIT(lpi_id_bits) * sizeof(__u32);
+}
+
+static void test_vgic_v5_addr_attrs(void)
+{
+ struct kvm_vcpu *vcpu;
+ struct vm_gic v;
+ u64 addr;
+ int ret;
+
+ v.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+ v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
+ v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
+
+ /* Check existing group/attributes */
+ kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR, gic_v5_irs_region.attr);
+
+ /* check non existing attribute */
+ ret = __kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR, -1);
+ TEST_ASSERT(ret && errno == ENXIO, "attribute not supported");
+
+ /* get IRS base address before setting*/
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+ KVM_VGIC_V5_ADDR_TYPE_IRS, &addr);
+ TEST_ASSERT(!ret && addr == (-1ULL), "GICv5 IRS returns VGIC_ADDR_UNDEF");
+
+ /* misaligned IRS address settings */
+ addr = gic_v5_irs_region.alignment / 0x10;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+ KVM_VGIC_V5_ADDR_TYPE_IRS, &addr);
+ TEST_ASSERT(ret && errno == EINVAL, "GIC IRS base not aligned");
+
+ /* out of range address */
+ addr = max_phys_size;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+ KVM_VGIC_V5_ADDR_TYPE_IRS, &addr);
+ TEST_ASSERT(ret && errno == E2BIG, "IRS address beyond IPA limit");
+
+ /* Space for half an IRS (an IRS is: 2 * irs.alignment). */
+ addr = max_phys_size - gic_v5_irs_region.alignment;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+ KVM_VGIC_V5_ADDR_TYPE_IRS, &addr);
+ TEST_ASSERT(ret && errno == E2BIG,
+ "half of the IRS is beyond IPA limit");
+
+ /* set IRS base address @0x0*/
+ addr = 0x00000;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+ KVM_VGIC_V5_ADDR_TYPE_IRS, &addr);
+ TEST_ASSERT(!ret, "GICv5 IRS base correctly set");
+
+ /* get IRS base address */
+ addr = 0xbad;
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+ KVM_VGIC_V5_ADDR_TYPE_IRS, &addr);
+ TEST_ASSERT(!ret && addr == 0, "GICv5 IRS base correctly set");
+
+ /* Attempt to create a second IRS region */
+ addr = 0xE0000;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+ KVM_VGIC_V5_ADDR_TYPE_IRS, &addr);
+ TEST_ASSERT(ret && errno == EEXIST, "GICv5 IRS base set again");
+
+ vm_gic_destroy(&v);
+
+ /* Try running a VM without ever setting the IRS base addr */
+ v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
+ v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
+ vcpu = vm_vcpu_add(v.vm, 0, NULL);
+ kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);
+ ret = run_vcpu(vcpu);
+ TEST_ASSERT(ret && errno == ENXIO, "GICv5 IRS base not set");
+
+ vm_gic_destroy(&v);
+}
-static void guest_irq_handler(struct ex_regs *regs)
+static void test_vgic_v5_nr_irqs_attrs(void)
{
- bool valid;
- u32 hwirq;
- u64 ia;
- static int count;
+ struct kvm_vcpu *vcpu;
+ struct vm_gic v;
+ uint64_t attr;
+ u32 nr_irqs;
+ int ret;
- /*
- * We have pending interrupts. Should never actually enter WFI
- * here!
- */
- wfi();
- GUEST_SYNC(GUEST_CMD_IS_AWAKE);
+ v.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+ v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
+ v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
- ia = gicr_insn(CDIA);
- valid = GICV5_GICR_CDIA_VALID(ia);
+ /* Check existing group/attribute */
+ kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS, 0);
- GUEST_SYNC(GUEST_CMD_IRQ_CDIA);
+ /* Before userspace sets NR_IRQS, no SPI count has been selected. */
+ nr_irqs = 0xbad;
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,
+ 0, &nr_irqs);
+ TEST_ASSERT(!ret && nr_irqs == 0, "GICv5 NR_IRQS defaults to 0 before init");
- if (!valid)
- return;
+ /* Too few SPIs */
+ nr_irqs = VGIC_V5_DEFAULT_NR_SPIS - 1;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,
+ 0, &nr_irqs);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 NR_IRQS below minimum");
- gsb_ack();
- isb();
+ /* Not a multiple of 32 */
+ nr_irqs = VGIC_V5_DEFAULT_NR_SPIS + 1;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,
+ 0, &nr_irqs);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 NR_IRQS not 32-aligned");
- hwirq = FIELD_GET(GICV5_GICR_CDIA_INTID, ia);
+ /* Larger than KVM's supported VGICv5 SPI count */
+ nr_irqs = VGIC_V5_MAX_NR_SPIS + 1;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,
+ 0, &nr_irqs);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 NR_IRQS above maximum");
- gic_insn(hwirq, CDDI);
- gic_insn(0, CDEOI);
+ /* Valid custom SPI count */
+ nr_irqs = VGIC_V5_DEFAULT_NR_SPIS * 2;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,
+ 0, &nr_irqs);
+ TEST_ASSERT(!ret, "GICv5 NR_IRQS accepts valid custom SPI count");
- GUEST_SYNC(GUEST_CMD_IRQ_DIEOI);
+ nr_irqs = 0xbad;
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,
+ 0, &nr_irqs);
+ TEST_ASSERT(!ret && nr_irqs == VGIC_V5_DEFAULT_NR_SPIS * 2,
+ "GICv5 NR_IRQS returns SPI count only");
- if (++count >= 2)
- GUEST_DONE();
+ /* A second successful configuration attempt must be rejected. */
+ nr_irqs = VGIC_V5_DEFAULT_NR_SPIS;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,
+ 0, &nr_irqs);
+ TEST_ASSERT(ret && errno == EBUSY, "GICv5 NR_IRQS set twice");
- /* Ask for the next interrupt to be injected */
- GUEST_SYNC(GUEST_CMD_IS_READY);
+ vm_gic_destroy(&v);
+
+ /* If userspace does not set NR_IRQS, init selects the default. */
+ v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
+ v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
+ vcpu = vm_vcpu_add(v.vm, 0, NULL);
+ TEST_ASSERT(vcpu, "Failed to create vCPU");
+ kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);
+
+ nr_irqs = 0xbad;
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,
+ 0, &nr_irqs);
+ TEST_ASSERT(!ret && nr_irqs == VGIC_V5_DEFAULT_NR_SPIS,
+ "GICv5 NR_IRQS defaults to 32 SPIs after init");
+
+ nr_irqs = VGIC_V5_DEFAULT_NR_SPIS * 2;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,
+ 0, &nr_irqs);
+ TEST_ASSERT(ret && errno == EBUSY, "GICv5 NR_IRQS set after init");
+
+ vm_gic_destroy(&v);
+
+ /* NR_IRQS must remain immutable after the VM has run. */
+ v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
+ v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
+ vcpu = vm_vcpu_add(v.vm, 0, guest_code);
+
+ attr = GICV5_IRS_CONFIG_BASE_GPA;
+ kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+ KVM_VGIC_V5_ADDR_TYPE_IRS, &attr);
+ kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);
+
+ ret = run_vcpu(vcpu);
+ TEST_ASSERT(!ret, "Failed to run GICv5 VM before NR_IRQS test");
+
+ nr_irqs = VGIC_V5_DEFAULT_NR_SPIS * 2;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_NR_IRQS,
+ 0, &nr_irqs);
+ TEST_ASSERT(ret && errno == EBUSY, "GICv5 NR_IRQS set after run");
+
+ vm_gic_destroy(&v);
}
-static void guest_code(void)
+static void test_vgic_v5_irs_regs_attrs(void)
{
- local_irq_disable();
+ struct kvm_vcpu *vcpu;
+ struct vm_gic v;
+ u64 attr, val;
+ int ret, i;
- gicv5_cpu_enable_interrupts();
- local_irq_enable();
+ v.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+ v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
+ v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
+ vcpu = vm_vcpu_add(v.vm, 0, NULL);
+ TEST_ASSERT(vcpu, "Failed to create vCPU");
- /* Enable the SW_PPI (3) */
- write_sysreg_s(BIT_ULL(3), SYS_ICC_PPI_ENABLER0_EL1);
+ /* IRS_REGS attributes can be probed before the IRS base is set. */
+ kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR0);
- /* Ask for the first interrupt to be injected */
- GUEST_SYNC(GUEST_CMD_IS_READY);
+ attr = GICV5_IRS_CONFIG_BASE_GPA;
+ kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+ KVM_VGIC_V5_ADDR_TYPE_IRS, &attr);
- /* Loop forever waiting for interrupts */
- while (1);
+ /* Check existing group/attribute */
+ kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR0);
+
+ /* IRS_REGS are not accessible before the VGIC is initialized. */
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR0, &val);
+ TEST_ASSERT(ret && errno == EBUSY, "GICv5 IRS_REGS get before init");
+
+ val = 0;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR0, &val);
+ TEST_ASSERT(ret && errno == EBUSY, "GICv5 IRS_REGS set before init");
+
+ kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);
+
+ /* Read all supported IRS regs and write the value back. */
+ for (i = 0; i < ARRAY_SIZE(gic_v5_irs_regs); i++) {
+ attr = gic_v5_irs_regs[i].attr;
+ ret = __kvm_has_device_attr(v.gic_fd,
+ KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ attr);
+ TEST_ASSERT(!ret, "GICv5 IRS_REGS missing %s",
+ gic_v5_irs_regs[i].name);
+
+ val = 0xbad;
+ ret = __kvm_device_attr_get(v.gic_fd,
+ KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ attr, &val);
+ TEST_ASSERT(!ret, "GICv5 IRS_REGS get failed for %s",
+ gic_v5_irs_regs[i].name);
+
+ ret = __kvm_device_attr_set(v.gic_fd,
+ KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ attr, &val);
+ TEST_ASSERT(!ret, "GICv5 IRS_REGS set failed for %s",
+ gic_v5_irs_regs[i].name);
+ }
+
+ /* Check bad offsets */
+ attr = 0x10000;
+ ret = __kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ attr);
+ TEST_ASSERT(ret && errno == ENXIO, "GICv5 IRS_REGS accepted bad offset");
+
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ attr, &val);
+ TEST_ASSERT(ret && errno == ENXIO, "GICv5 IRS_REGS get bad offset");
+
+ /* Check alignment for 32-bit and 64-bit IRS regs. */
+ attr = GICV5_IRS_IDR0 + 2;
+ ret = __kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ attr);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IRS_REGS accepted unaligned 32-bit attr");
+
+ attr = GICV5_IRS_IST_BASER + 4;
+ ret = __kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ attr);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IRS_REGS accepted unaligned 64-bit attr");
+
+ /* Check bad user pointers */
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR0, NULL);
+ TEST_ASSERT(ret && errno == EFAULT, "GICv5 IRS_REGS get with bad pointer");
+
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR0, NULL);
+ TEST_ASSERT(ret && errno == EFAULT, "GICv5 IRS_REGS set with bad pointer");
+
+ /* ID restore validation rejects unsupported values. */
+ val = FIELD_PREP(GICV5_IRS_IDR0_INT_DOM, GICV5_IRS_IDR0_INT_DOM_SECURE);
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR0, &val);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IRS_REGS accepted bad IDR0 domain");
+
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR0, &val);
+ TEST_ASSERT(!ret, "GICv5 IRS_REGS get IDR0 failed");
+ val |= GICV5_IRS_IDR0_SETLPI;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR0, &val);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IRS_REGS accepted unsupported IDR0");
+
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR1, &val);
+ TEST_ASSERT(!ret, "GICv5 IRS_REGS get IDR1 failed");
+ val |= FIELD_PREP(GICV5_IRS_IDR1_PRIORITY_BITS, 0x7);
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR1, &val);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IRS_REGS accepted bad IDR1");
+
+ val = 0;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR2, &val);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IRS_REGS accepted bad IDR2");
+
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR5, &val);
+ TEST_ASSERT(!ret, "GICv5 IRS_REGS get IDR5 failed");
+ val++;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR5, &val);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IRS_REGS accepted bad IDR5");
+
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR6, &val);
+ TEST_ASSERT(!ret, "GICv5 IRS_REGS get IDR6 failed");
+ val++;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR6, &val);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IRS_REGS accepted bad IDR6");
+
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR7, &val);
+ TEST_ASSERT(!ret, "GICv5 IRS_REGS get IDR7 failed");
+ val++;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IDR7, &val);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IRS_REGS accepted bad IDR7");
+
+ /* Status registers read as idle through userspace accessors. */
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_SYNC_STATUSR, &val);
+ TEST_ASSERT(!ret && val == GICV5_IRS_SYNC_STATUSR_IDLE,
+ "GICv5 IRS_REGS SYNC_STATUSR is not idle");
+
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_SPI_STATUSR, &val);
+ TEST_ASSERT(!ret && val == GICV5_IRS_SPI_STATUSR_IDLE,
+ "GICv5 IRS_REGS SPI_STATUSR is not idle");
+
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_PE_STATUSR, &val);
+ TEST_ASSERT(!ret && val == GICV5_IRS_PE_STATUSR_IDLE,
+ "GICv5 IRS_REGS PE_STATUSR is not idle");
+
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IST_STATUSR, &val);
+ TEST_ASSERT(!ret && val == GICV5_IRS_IST_STATUSR_IDLE,
+ "GICv5 IRS_REGS IST_STATUSR is not idle");
+
+ /* Userspace restore of IST_BASER updates state without allocating ISTs. */
+ val = FIELD_PREP(GICV5_IRS_IST_BASER_ADDR_MASK,
+ GICV5_IRS_CONFIG_BASE_GPA >> GICV5_IRS_IST_BASER_ADDR_SHIFT);
+ val |= GICV5_IRS_IST_BASER_VALID;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IST_BASER, &val);
+ TEST_ASSERT(!ret, "GICv5 IRS_REGS failed to restore IST_BASER");
+
+ attr = 0xbad;
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IRS_REGS,
+ GICV5_IRS_IST_BASER, &attr);
+ TEST_ASSERT(!ret && attr == val, "GICv5 IRS_REGS IST_BASER restore mismatch");
+
+ vm_gic_destroy(&v);
}
+static void test_vgic_v5_ist_attrs(void)
+{
+ struct kvm_vgic_v5_ist ist_attr;
+ struct kvm_vcpu *vcpu;
+ struct vm_gic v;
+ u32 spi_ist[VGIC_V5_DEFAULT_NR_SPIS];
+ u64 attr;
+ int ret;
-/* we don't want to assert on run execution, hence that helper */
-static int run_vcpu(struct kvm_vcpu *vcpu)
+ v.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+ v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
+ v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
+ vcpu = vm_vcpu_add(v.vm, 0, NULL);
+ TEST_ASSERT(vcpu, "Failed to create vCPU");
+
+ /* Check existing group/attribute */
+ kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST, 0);
+
+ /* Check non-existing attribute */
+ ret = __kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST, 1);
+ TEST_ASSERT(ret && errno == ENXIO, "GICv5 IST accepted bad attr");
+
+ ist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), NULL, 0);
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 1, &ist_attr);
+ TEST_ASSERT(ret && errno == ENXIO, "GICv5 IST get accepted bad attr");
+
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 1, &ist_attr);
+ TEST_ASSERT(ret && errno == ENXIO, "GICv5 IST set accepted bad attr");
+
+ attr = GICV5_IRS_CONFIG_BASE_GPA;
+ kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+ KVM_VGIC_V5_ADDR_TYPE_IRS, &attr);
+
+ /* IST save/restore is not accessible before the VGIC is initialized. */
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, &ist_attr);
+ TEST_ASSERT(ret && errno == EBUSY, "GICv5 IST get before init");
+
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, &ist_attr);
+ TEST_ASSERT(ret && errno == EBUSY, "GICv5 IST set before init");
+
+ kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);
+
+ /* A VM with SPIs must provide a userspace IST descriptor. */
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, NULL);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IST get accepted NULL descriptor");
+
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, NULL);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IST set accepted NULL descriptor");
+
+ /* Check bad userspace IST descriptors. */
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, (void *)1);
+ TEST_ASSERT(ret && errno == EFAULT, "GICv5 IST get with bad descriptor");
+
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, (void *)1);
+ TEST_ASSERT(ret && errno == EFAULT, "GICv5 IST set with bad descriptor");
+
+ /* Check missing and incorrectly sized SPI IST buffers. */
+ ist_attr = vgic_v5_ist_attr(NULL, 0, NULL, 0);
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, &ist_attr);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IST get accepted missing SPI buffer");
+
+ ist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist) - sizeof(__u32),
+ NULL, 0);
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, &ist_attr);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IST set accepted bad SPI size");
+
+ /* LPI storage must be absent when no LPI IST is configured. */
+ ist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), spi_ist,
+ sizeof(__u32));
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, &ist_attr);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IST get accepted unexpected LPI buffer");
+
+ vm_gic_destroy(&v);
+
+ /* IST restore is rejected after the VM has run. */
+ v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
+ v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
+ vcpu = vm_vcpu_add(v.vm, 0, guest_code);
+
+ attr = GICV5_IRS_CONFIG_BASE_GPA;
+ kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+ KVM_VGIC_V5_ADDR_TYPE_IRS, &attr);
+ kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);
+
+ ret = run_vcpu(vcpu);
+ TEST_ASSERT(!ret, "Failed to run GICv5 VM before IST restore test");
+
+ ist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), NULL, 0);
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, &ist_attr);
+ TEST_ASSERT(ret && errno == EBUSY, "GICv5 IST restore after run");
+
+ vm_gic_destroy(&v);
+}
+
+static void test_vgic_v5_userspace_ppis_attrs(void)
{
- return __vcpu_run(vcpu) ? -errno : 0;
+ struct kvm_vcpu *vcpu;
+ struct vm_gic v;
+ u64 user_ppis[2];
+ int ret;
+
+ v.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+ v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
+ v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
+ vcpu = vm_vcpu_add(v.vm, 0, NULL);
+ TEST_ASSERT(vcpu, "Failed to create vCPU");
+
+ kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_CTRL_INIT);
+ kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_USERSPACE_PPIS);
+
+ kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);
+
+ user_ppis[0] = 0;
+ user_ppis[1] = 0xbad;
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_USERSPACE_PPIS, user_ppis);
+ TEST_ASSERT(!ret, "GICv5 USERSPACE_PPIS get failed");
+ TEST_ASSERT(user_ppis[0] & BIT(GICV5_ARCH_PPI_SW_PPI),
+ "GICv5 USERSPACE_PPIS does not expose SW_PPI");
+ TEST_ASSERT(!user_ppis[1], "GICv5 USERSPACE_PPIS upper word is not zero");
+
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_USERSPACE_PPIS, user_ppis);
+ TEST_ASSERT(ret && errno == ENXIO, "GICv5 USERSPACE_PPIS set accepted");
+
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_USERSPACE_PPIS, NULL);
+ TEST_ASSERT(ret && errno == EFAULT, "GICv5 USERSPACE_PPIS get with bad pointer");
+
+ vm_gic_destroy(&v);
}
-static void vm_gic_destroy(struct vm_gic *v)
+static void test_vgic_v5_cpu_sysreg_attrs(void)
{
- close(v->gic_fd);
- kvm_vm_free(v->vm);
+ struct kvm_vcpu *vcpu;
+ struct vm_gic v;
+ u64 attr, val;
+ int ret, i;
+
+ v.gic_dev_type = KVM_DEV_TYPE_ARM_VGIC_V5;
+ v.vm = __vm_create(VM_SHAPE_DEFAULT, NR_VCPUS, 0);
+ v.gic_fd = kvm_create_device(v.vm, v.gic_dev_type);
+ vcpu = vm_vcpu_add(v.vm, 0, NULL);
+ TEST_ASSERT(vcpu, "Failed to create vCPU");
+
+ /* Check existing group/attribute */
+ attr = PACK_SR(SYS_ICC_CR0_EL1);
+ kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS, attr);
+
+ /* CPU sysregs are not accessible before the VGIC is initialized. */
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,
+ attr, &val);
+ TEST_ASSERT(ret && errno == EBUSY, "GICv5 CPU_SYSREGS get before init");
+
+ val = 0;
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,
+ attr, &val);
+ TEST_ASSERT(ret && errno == EBUSY, "GICv5 CPU_SYSREGS set before init");
+
+ kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+ KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);
+
+ /* Read all exposed CPU sysregs and write the value back. */
+ for (i = 0; i < ARRAY_SIZE(gic_v5_cpu_sysregs); i++) {
+ attr = PACK_SR(gic_v5_cpu_sysregs[i].encoding);
+ ret = __kvm_has_device_attr(v.gic_fd,
+ KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,
+ attr);
+ TEST_ASSERT(!ret, "GICv5 CPU_SYSREGS missing %s",
+ gic_v5_cpu_sysregs[i].name);
+
+ val = 0xbad;
+ ret = __kvm_device_attr_get(v.gic_fd,
+ KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,
+ attr, &val);
+ TEST_ASSERT(!ret, "GICv5 CPU_SYSREGS get failed for %s",
+ gic_v5_cpu_sysregs[i].name);
+
+ ret = __kvm_device_attr_set(v.gic_fd,
+ KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,
+ attr, &val);
+ TEST_ASSERT(!ret, "GICv5 CPU_SYSREGS set failed for %s",
+ gic_v5_cpu_sysregs[i].name);
+ }
+
+ /* Check non existent GICv3 sysreg */
+ attr = PACK_SR(SYS_ICC_CTLR_EL1);
+ ret = __kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,
+ attr);
+ TEST_ASSERT(ret && errno == ENXIO, "GICv5 CPU_SYSREGS accepted bad sysreg");
+
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,
+ attr, &val);
+ TEST_ASSERT(ret && errno == ENOENT, "GICv5 CPU_SYSREGS get bad sysreg");
+
+ /* Check non existing vCPU */
+ attr = PACK_SR(SYS_ICC_CR0_EL1) |
+ (1ULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT);
+ ret = __kvm_has_device_attr(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,
+ attr);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 CPU_SYSREGS accepted bad MPIDR");
+
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,
+ attr, &val);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 CPU_SYSREGS get bad MPIDR");
+
+ /* Check bad user pointers */
+ attr = PACK_SR(SYS_ICC_CR0_EL1);
+ ret = __kvm_device_attr_get(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,
+ attr, NULL);
+ TEST_ASSERT(ret && errno == EFAULT, "GICv5 CPU_SYSREGS get with bad pointer");
+
+ ret = __kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS,
+ attr, NULL);
+ TEST_ASSERT(ret && errno == EFAULT, "GICv5 CPU_SYSREGS set with bad pointer");
+
+ vm_gic_destroy(&v);
}
static void test_vgic_v5_ppis(u32 gic_dev_type)
@@ -100,6 +1517,7 @@ static void test_vgic_v5_ppis(u32 gic_dev_type)
struct ucall uc;
u64 user_ppis[2];
struct vm_gic v;
+ uint64_t attr;
int ret, i;
v.gic_dev_type = gic_dev_type;
@@ -111,11 +1529,17 @@ static void test_vgic_v5_ppis(u32 gic_dev_type)
vcpus[i] = vm_vcpu_add(v.vm, i, guest_code);
vm_init_descriptor_tables(v.vm);
- vm_install_exception_handler(v.vm, VECTOR_IRQ_CURRENT, guest_irq_handler);
+ vm_install_exception_handler(v.vm, VECTOR_IRQ_CURRENT,
+ guest_ppi_irq_handler);
for (i = 0; i < NR_VCPUS; i++)
vcpu_init_descriptor_tables(vcpus[i]);
+ /* Set the address of the IRS before initialising the GIC */
+ attr = GICV5_IRS_CONFIG_BASE_GPA;
+ kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_ADDR,
+ KVM_VGIC_V5_ADDR_TYPE_IRS, &attr);
+
kvm_device_attr_set(v.gic_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
KVM_DEV_ARM_VGIC_CTRL_INIT, NULL);
@@ -127,6 +1551,20 @@ static void test_vgic_v5_ppis(u32 gic_dev_type)
TEST_ASSERT(user_ppis[0] & BIT(GICV5_ARCH_PPI_SW_PPI),
"SW_PPI is not drivable by userspace");
+ /* PPIs not explicitly exposed to userspace must be rejected. */
+ for (i = 0; i < VGIC_V5_NR_PRIVATE_IRQS; i++) {
+ if (user_ppis[i / 64] & BIT_ULL(i % 64))
+ continue;
+
+ ret = __vgic_v5_irq_line(v.vm, KVM_ARM_IRQ_TYPE_PPI, i, 1);
+ TEST_ASSERT(ret && errno == EINVAL,
+ "GICv5 accepted non-userspace PPI %d", i);
+ }
+
+ ret = __vgic_v5_irq_line(v.vm, KVM_ARM_IRQ_TYPE_PPI,
+ VGIC_V5_NR_PRIVATE_IRQS, 1);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 accepted out-of-range PPI");
+
while (1) {
ret = run_vcpu(vcpus[0]);
if (ret)
@@ -148,9 +1586,9 @@ static void test_vgic_v5_ppis(u32 gic_dev_type)
kvm_irq_line(v.vm, irq, level);
} else if (uc.args[1] == GUEST_CMD_IS_AWAKE) {
- pr_info("Guest skipping WFI due to pending IRQ\n");
+ pr_debug("Guest skipping WFI due to pending IRQ\n");
} else if (uc.args[1] == GUEST_CMD_IRQ_CDIA) {
- pr_info("Guest acknowledged IRQ\n");
+ pr_debug("Guest acknowledged IRQ\n");
}
continue;
@@ -170,6 +1608,247 @@ static void test_vgic_v5_ppis(u32 gic_dev_type)
vm_gic_destroy(&v);
}
+static void test_vgic_v5_spis(void)
+{
+ struct kvm_vcpu *vcpu;
+ struct vm_gic v;
+ int ret;
+
+ /* Default NR_IRQS exposes 32 SPIs, numbered 0..31 in KVM_IRQ_LINE. */
+ vgic_v5_spi_line_vm_create(&v, &vcpu, VGIC_V5_DEFAULT_NR_SPIS);
+ ret = __vgic_v5_irq_line(v.vm, KVM_ARM_IRQ_TYPE_SPI,
+ VGIC_V5_DEFAULT_NR_SPIS, 1);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 accepted first invalid SPI");
+
+ vm_gic_destroy(&v);
+
+ /* Basic SPI injection through KVM_IRQ_LINE. */
+ vgic_v5_run_spi_line_test(VGIC_V5_DEFAULT_NR_SPIS, 0, false, false);
+
+ /* The last valid SPI for a configured NR_IRQS is injectable. */
+ vgic_v5_run_spi_line_test(VGIC_V5_TEST_NR_SPIS,
+ VGIC_V5_TEST_NR_SPIS - 1, false, false);
+
+ vgic_v5_spi_line_vm_create(&v, &vcpu, VGIC_V5_TEST_NR_SPIS);
+ ret = __vgic_v5_irq_line(v.vm, KVM_ARM_IRQ_TYPE_SPI,
+ VGIC_V5_TEST_NR_SPIS, 1);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 accepted configured invalid SPI");
+ vm_gic_destroy(&v);
+
+ /*
+ * Edge SPIs remain pending after the line is lowered. The low
+ * transition is injected before the guest runs and must be ignored:
+ * once an edge interrupt is pending, lowering the line cannot recall it.
+ */
+ vgic_v5_run_spi_line_test(VGIC_V5_DEFAULT_NR_SPIS, 1, false, true);
+
+ /* Level SPIs can be raised and delivered through KVM_IRQ_LINE. */
+ vgic_v5_run_spi_line_test(VGIC_V5_DEFAULT_NR_SPIS, 2, true, false);
+}
+
+static void test_vgic_v5_lpis(void)
+{
+ struct kvm_vcpu *vcpus[VGIC_V5_LPI_NR_VCPUS];
+ struct ucall uc;
+ struct vm_gic v;
+ int ret;
+
+ lpi_ist_ready = false;
+ vgic_v5_lpi_vm_create(&v, vcpus);
+ sync_global_to_guest(v.vm, lpi_ist_ready);
+
+ /* VPE0 programs a linear LPI IST from the virtual IRS ID registers. */
+ ret = run_vcpu(vcpus[0]);
+ TEST_ASSERT(!ret, "Failed to run GICv5 LPI vCPU0");
+ TEST_ASSERT(get_ucall(vcpus[0], &uc) == UCALL_SYNC &&
+ uc.args[1] == GUEST_CMD_IS_READY,
+ "GICv5 LPI vCPU0 did not become ready");
+
+ /* VPE1 observes the IST and enables the LPIs it will receive/send. */
+ ret = run_vcpu(vcpus[1]);
+ TEST_ASSERT(!ret, "Failed to run GICv5 LPI vCPU1");
+ TEST_ASSERT(get_ucall(vcpus[1], &uc) == UCALL_SYNC &&
+ uc.args[1] == GUEST_CMD_IS_READY,
+ "GICv5 LPI vCPU1 did not become ready");
+
+ /* VPE0 sends an LPI to VPE1. */
+ ret = run_vcpu(vcpus[0]);
+ TEST_ASSERT(!ret, "Failed to run GICv5 LPI vCPU0 sender");
+ TEST_ASSERT(get_ucall(vcpus[0], &uc) == UCALL_SYNC &&
+ uc.args[1] == GUEST_CMD_LPI_SENT,
+ "GICv5 LPI vCPU0 did not send LPI");
+
+ /* VPE1 consumes the LPI and replies with another LPI. */
+ ret = run_vcpu(vcpus[1]);
+ TEST_ASSERT(!ret, "Failed to run GICv5 LPI vCPU1 receiver");
+ TEST_ASSERT(get_ucall(vcpus[1], &uc) == UCALL_SYNC &&
+ uc.args[1] == GUEST_CMD_LPI_REPLIED,
+ "GICv5 LPI vCPU1 did not reply");
+
+ /* VPE0 consumes the reply LPI and ends the test. */
+ ret = run_vcpu(vcpus[0]);
+ TEST_ASSERT(!ret, "Failed to run GICv5 LPI vCPU0 receiver");
+ TEST_ASSERT(get_ucall(vcpus[0], &uc) == UCALL_DONE,
+ "GICv5 LPI vCPU0 did not complete");
+
+ vm_gic_destroy(&v);
+}
+
+static void test_vgic_v5_ist_save_restore(void)
+{
+ static u32 lpi_ist[VGIC_V5_LPI_IST_SIZE / sizeof(u32)];
+ static u32 lpi_ist_resave[VGIC_V5_LPI_IST_SIZE / sizeof(u32)];
+ static u32 spi_ist[VGIC_V5_DEFAULT_NR_SPIS];
+ static u32 spi_ist_resave[VGIC_V5_DEFAULT_NR_SPIS];
+ static u64 irs_regs[ARRAY_SIZE(gic_v5_irs_regs)];
+ struct kvm_vcpu *vcpus[VGIC_V5_LPI_NR_VCPUS];
+ struct kvm_vgic_v5_ist ist_attr;
+ struct vm_gic src, dst;
+ size_t lpi_ist_size;
+ int ret;
+
+ memset(ist_restore_irq_count, 0, sizeof(ist_restore_irq_count));
+
+ vgic_v5_ist_restore_vm_create(&src, vcpus, guest_ist_source_code,
+ guest_unexpected_irq_handler);
+
+ ret = run_vcpu(vcpus[0]);
+ TEST_ASSERT(!ret, "Failed to run GICv5 IST save source VM");
+ vgic_v5_expect_sync(vcpus[0], GUEST_CMD_IST_SOURCE_READY,
+ "GICv5 IST source guest did not become ready");
+
+ /*
+ * The guest configured enable/priority/affinity state. Make selected
+ * SPIs pending from userspace so the SPI IST save path sees them.
+ */
+ vgic_v5_spi_line(src.vm, RESTORE_TEST_SPI_PRIO_HIGH, 1);
+ vgic_v5_spi_line(src.vm, RESTORE_TEST_SPI_PRIO_LOW, 1);
+ vgic_v5_spi_line(src.vm, RESTORE_TEST_SPI_PENDING_DISABLED, 1);
+
+ vgic_v5_save_irs_regs(&src, irs_regs);
+ lpi_ist_size = vgic_v5_lpi_ist_state_size(irs_regs);
+
+ /*
+ * VCPU0 has configured the initial state. KVM must reject missing or
+ * incorrectly sized LPI storage before accepting the complete IST image.
+ */
+ ist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), NULL, 0);
+ ret = __kvm_device_attr_get(src.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, &ist_attr);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IST save accepted missing LPI buffer");
+
+ ist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), lpi_ist,
+ lpi_ist_size - sizeof(__u32));
+ ret = __kvm_device_attr_get(src.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, &ist_attr);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IST save accepted bad LPI size");
+
+ ist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), lpi_ist,
+ lpi_ist_size);
+ ret = __kvm_device_attr_get(src.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, &ist_attr);
+ TEST_ASSERT(!ret, "GICv5 IST save failed");
+
+ vm_gic_destroy(&src);
+
+ /* Create the guest that we are restoring state into */
+ memset(ist_restore_irq_count, 0, sizeof(ist_restore_irq_count));
+ vgic_v5_ist_restore_vm_create(&dst, vcpus, guest_ist_restore_code,
+ guest_ist_restore_irq_handler);
+ sync_global_to_guest(dst.vm, ist_restore_irq_count);
+
+ ist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), lpi_ist,
+ lpi_ist_size);
+ ret = __kvm_device_attr_set(dst.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, &ist_attr);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IST restore before IRS state");
+
+ vgic_v5_restore_irs_regs(&dst, irs_regs);
+
+ ist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), NULL, 0);
+ ret = __kvm_device_attr_set(dst.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, &ist_attr);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IST restore accepted missing LPI buffer");
+
+ ist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), lpi_ist,
+ lpi_ist_size - sizeof(__u32));
+ ret = __kvm_device_attr_set(dst.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, &ist_attr);
+ TEST_ASSERT(ret && errno == EINVAL, "GICv5 IST restore accepted bad LPI size");
+
+ /* Restore the VMM-owned SPI and LPI IST images. */
+ ist_attr = vgic_v5_ist_attr(spi_ist, sizeof(spi_ist), lpi_ist,
+ lpi_ist_size);
+ ret = __kvm_device_attr_set(dst.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, &ist_attr);
+ TEST_ASSERT(!ret, "GICv5 IST restore failed");
+
+ /*
+ * Re-save the restored IST image before the destination VM runs, so a
+ * dropped serialized field fails independently of later interrupt
+ * delivery.
+ */
+ ist_attr = vgic_v5_ist_attr(spi_ist_resave, sizeof(spi_ist_resave),
+ lpi_ist_resave, lpi_ist_size);
+ ret = __kvm_device_attr_get(dst.gic_fd, KVM_DEV_ARM_VGIC_GRP_IST,
+ 0, &ist_attr);
+ TEST_ASSERT(!ret, "GICv5 IST re-save failed");
+ TEST_ASSERT(!memcmp(spi_ist, spi_ist_resave, sizeof(spi_ist)),
+ "GICv5 SPI IST state changed across restore");
+ TEST_ASSERT(!memcmp(lpi_ist, lpi_ist_resave, lpi_ist_size),
+ "GICv5 LPI IST state changed across restore");
+
+ /*
+ * At this stage, the guest's IST state should match what was saved. Run
+ * each vcpu once to let it enable interrupts, etc.
+ */
+ ret = run_vcpu(vcpus[0]);
+ TEST_ASSERT(!ret, "Failed to run GICv5 IST restore vCPU0");
+ vgic_v5_expect_sync(vcpus[0], GUEST_CMD_IS_READY,
+ "GICv5 IST restore vCPU0 did not become ready");
+
+ ret = run_vcpu(vcpus[1]);
+ TEST_ASSERT(!ret, "Failed to run GICv5 IST restore vCPU1");
+ vgic_v5_expect_sync(vcpus[1], GUEST_CMD_IS_READY,
+ "GICv5 IST restore vCPU1 did not become ready");
+
+ /*
+ * VPE0 consumes two pending SPIs in restored priority order, then
+ * enables a restored pending-but-disabled SPI.
+ */
+ ret = run_vcpu(vcpus[0]);
+ TEST_ASSERT(!ret, "Failed to consume restored GICv5 SPIs");
+ vgic_v5_expect_sync(vcpus[0], GUEST_CMD_IST_SPI_INJECT_READY,
+ "GICv5 IST restore vCPU0 did not consume restored SPIs");
+
+ /*
+ * This SPI was enabled before save, but not pending. Inject it after
+ * restore to prove enable, priority and affinity were restored.
+ */
+ vgic_v5_spi_line(dst.vm, RESTORE_TEST_SPI_ENABLED, 1);
+
+ ret = run_vcpu(vcpus[0]);
+ TEST_ASSERT(!ret, "Failed to consume post-restore GICv5 SPI");
+ TEST_ASSERT(get_ucall(vcpus[0], NULL) == UCALL_DONE,
+ "GICv5 IST restore vCPU0 did not complete");
+
+ /*
+ * VPE1 consumes a restored pending LPI, then pends another LPI that
+ * was enabled before save but not pending.
+ */
+ ret = run_vcpu(vcpus[1]);
+ TEST_ASSERT(!ret, "Failed to consume restored GICv5 LPI");
+ vgic_v5_expect_sync(vcpus[1], GUEST_CMD_IST_LPI_INJECT_READY,
+ "GICv5 IST restore vCPU1 did not consume restored LPI");
+
+ ret = run_vcpu(vcpus[1]);
+ TEST_ASSERT(!ret, "Failed to consume post-restore GICv5 LPI");
+ TEST_ASSERT(get_ucall(vcpus[1], NULL) == UCALL_DONE,
+ "GICv5 IST restore vCPU1 did not complete");
+
+ vm_gic_destroy(&dst);
+}
+
/*
* Returns 0 if it's possible to create GIC device of a given type (V5).
*/
@@ -201,13 +1880,43 @@ int test_kvm_device(u32 gic_dev_type)
void run_tests(u32 gic_dev_type)
{
+ pr_info("Test VGICv5 address attrs\n");
+ test_vgic_v5_addr_attrs();
+
+ pr_info("Test VGICv5 NR_IRQS attrs\n");
+ test_vgic_v5_nr_irqs_attrs();
+
+ pr_info("Test VGICv5 IRS_REGS attrs\n");
+ test_vgic_v5_irs_regs_attrs();
+
+ pr_info("Test VGICv5 IST attrs\n");
+ test_vgic_v5_ist_attrs();
+
+ pr_info("Test VGICv5 userspace PPI attrs\n");
+ test_vgic_v5_userspace_ppis_attrs();
+
+ pr_info("Test VGICv5 CPU sysreg attrs\n");
+ test_vgic_v5_cpu_sysreg_attrs();
+
pr_info("Test VGICv5 PPIs\n");
test_vgic_v5_ppis(gic_dev_type);
+
+ pr_info("Test VGICv5 SPIs\n");
+ test_vgic_v5_spis();
+
+ pr_info("Test VGICv5 LPIs\n");
+ test_vgic_v5_lpis();
+
+ pr_info("Test VGICv5 IST save/restore\n");
+ test_vgic_v5_ist_save_restore();
}
int main(int ac, char **av)
{
- int ret;
+ int pa_bits, ret;
+
+ pa_bits = vm_guest_mode_params[VM_MODE_DEFAULT].pa_bits;
+ max_phys_size = 1ULL << pa_bits;
test_disable_default_vgic();
diff --git a/tools/testing/selftests/kvm/include/arm64/gic_v5.h b/tools/testing/selftests/kvm/include/arm64/gic_v5.h
index eb523d9277cf1e..2d455515563e0a 100644
--- a/tools/testing/selftests/kvm/include/arm64/gic_v5.h
+++ b/tools/testing/selftests/kvm/include/arm64/gic_v5.h
@@ -10,6 +10,106 @@
#include "processor.h"
+/* GIC component base address is guest PA space */
+#define GICV5_IRS_CONFIG_BASE_GPA 0x8000000ULL
+#define GICV5_IRS_CONFIG_BASE_GVA ((volatile void *)GICV5_IRS_CONFIG_BASE_GPA)
+
+#define GICV5_IRS_SIZE 0x20000
+#define GICV5_IRS_ALIGN 0x10000
+
+/*
+ * IRS register offsets. Keep this selftest copy scoped to the offsets and
+ * fields exercised by the userspace API tests.
+ */
+#define GICV5_IRS_IDR0 0x0000
+#define GICV5_IRS_IDR1 0x0004
+#define GICV5_IRS_IDR2 0x0008
+#define GICV5_IRS_IDR3 0x000c
+#define GICV5_IRS_IDR4 0x0010
+#define GICV5_IRS_IDR5 0x0014
+#define GICV5_IRS_IDR6 0x0018
+#define GICV5_IRS_IDR7 0x001c
+#define GICV5_IRS_IIDR 0x0040
+#define GICV5_IRS_AIDR 0x0044
+#define GICV5_IRS_CR0 0x0080
+#define GICV5_IRS_CR1 0x0084
+#define GICV5_IRS_SYNCR 0x00c0
+#define GICV5_IRS_SYNC_STATUSR 0x00c4
+#define GICV5_IRS_SPI_VMR 0x0100
+#define GICV5_IRS_SPI_SELR 0x0108
+#define GICV5_IRS_SPI_DOMAINR 0x010c
+#define GICV5_IRS_SPI_RESAMPLER 0x0110
+#define GICV5_IRS_SPI_CFGR 0x0114
+#define GICV5_IRS_SPI_STATUSR 0x0118
+#define GICV5_IRS_PE_SELR 0x0140
+#define GICV5_IRS_PE_STATUSR 0x0144
+#define GICV5_IRS_PE_CR0 0x0148
+#define GICV5_IRS_IST_BASER 0x0180
+#define GICV5_IRS_IST_CFGR 0x0190
+#define GICV5_IRS_IST_STATUSR 0x0194
+#define GICV5_IRS_MAP_L2_ISTR 0x01c0
+#define GICV5_IRS_VMT_BASER 0x0200
+#define GICV5_IRS_VMT_CFGR 0x0210
+#define GICV5_IRS_VMT_STATUSR 0x0214
+#define GICV5_IRS_VPE_SELR 0x0240
+#define GICV5_IRS_VPE_DBR 0x0248
+#define GICV5_IRS_VPE_HPPIR 0x0250
+#define GICV5_IRS_VPE_CR0 0x0258
+#define GICV5_IRS_VPE_STATUSR 0x025c
+#define GICV5_IRS_VM_DBR 0x0280
+#define GICV5_IRS_VM_SELR 0x0288
+#define GICV5_IRS_VM_STATUSR 0x028c
+#define GICV5_IRS_VMAP_L2_VMTR 0x02c0
+#define GICV5_IRS_VMAP_VMR 0x02c8
+#define GICV5_IRS_VMAP_VISTR 0x02d0
+#define GICV5_IRS_VMAP_L2_VISTR 0x02d8
+#define GICV5_IRS_VMAP_VPER 0x02e0
+#define GICV5_IRS_SAVE_VMR 0x0300
+#define GICV5_IRS_SAVE_VM_STATUSR 0x0308
+#define GICV5_IRS_MEC_IDR 0x0340
+#define GICV5_IRS_MEC_MECID_R 0x0344
+#define GICV5_IRS_MPAM_IDR 0x0380
+#define GICV5_IRS_MPAM_PARTID_R 0x0384
+#define GICV5_IRS_SWERR_STATUSR 0x03c0
+#define GICV5_IRS_SWERR_SYNDROMER0 0x03c8
+#define GICV5_IRS_SWERR_SYNDROMER1 0x03d0
+
+#define GICV5_IRS_IDR0_SETLPI BIT(9)
+#define GICV5_IRS_IDR0_INT_DOM GENMASK(1, 0)
+#define GICV5_IRS_IDR0_INT_DOM_SECURE 0b00
+
+#define GICV5_IRS_IDR1_PRIORITY_BITS GENMASK(22, 20)
+
+#define GICV5_IRS_IDR2_ISTMD_SZ GENMASK(19, 15)
+#define GICV5_IRS_IDR2_ISTMD BIT(14)
+#define GICV5_IRS_IDR2_IST_L2SZ GENMASK(13, 11)
+#define GICV5_IRS_IDR2_IST_LEVELS BIT(10)
+#define GICV5_IRS_IDR2_MIN_LPI_ID_BITS GENMASK(9, 6)
+#define GICV5_IRS_IDR2_LPI BIT(5)
+#define GICV5_IRS_IDR2_ID_BITS GENMASK(4, 0)
+
+#define GICV5_IRS_CR0_IDLE BIT(1)
+#define GICV5_IRS_CR0_IRSEN BIT(0)
+
+#define GICV5_IRS_SPI_SELR_ID GENMASK(23, 0)
+#define GICV5_IRS_SPI_CFGR_TM BIT(0)
+
+#define GICV5_IRS_SYNC_STATUSR_IDLE BIT(0)
+#define GICV5_IRS_SPI_STATUSR_IDLE BIT(0)
+#define GICV5_IRS_PE_STATUSR_IDLE BIT(0)
+#define GICV5_IRS_IST_STATUSR_IDLE BIT(0)
+
+#define GICV5_IRS_IST_BASER_ADDR_MASK GENMASK_ULL(55, 6)
+#define GICV5_IRS_IST_BASER_VALID BIT_ULL(0)
+#define GICV5_IRS_IST_BASER_ADDR_SHIFT 6ULL
+#define GICV5_IRS_IST_CFGR_STRUCTURE BIT(16)
+#define GICV5_IRS_IST_CFGR_ISTSZ GENMASK(8, 7)
+#define GICV5_IRS_IST_CFGR_L2SZ GENMASK(6, 5)
+#define GICV5_IRS_IST_CFGR_LPI_ID_BITS GENMASK(4, 0)
+#define GICV5_IRS_IST_CFGR_ISTSZ_4 0b00
+#define GICV5_IRS_IST_CFGR_ISTSZ_8 0b01
+#define GICV5_IRS_IST_CFGR_ISTSZ_16 0b10
+
/*
* Definitions for GICv5 instructions for the Current Domain
*/
@@ -77,6 +177,11 @@
#define GICV5_GICR_CDNMIA_TYPE_MASK GENMASK_ULL(31, 29)
#define GICV5_GICR_CDNMIA_ID_MASK GENMASK_ULL(23, 0)
+#define GICV5_HWIRQ_TYPE GENMASK_ULL(31, 29)
+#define GICV5_HWIRQ_ID GENMASK_ULL(23, 0)
+#define GICV5_HWIRQ_TYPE_LPI 0x2
+#define GICV5_HWIRQ_TYPE_SPI 0x3
+
#define gicr_insn(insn) read_sysreg_s(GICV5_OP_GICR_##insn)
#define gic_insn(v, insn) write_sysreg_s(v, GICV5_OP_GIC_##insn)
|
| 3/2 | 2026/07/25 16:10 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 132623 output: 32 thoughts: 1126 Thoughts: **Fuzzing Target Identification: KVM GICv5 IRS Implementation** Okay, let's break this down. My initial assessment is that this patch series *definitely* introduces functional changes to the kernel's KVM subsystem that warrant fuzzing. We're talking about the implementation of GICv5 IRS, SPIs, and LPIs within KVM. This is a significant expansion of functionality. The addition of new KVM device attributes like `KVM_DEV_ARM_VGIC_GRP_ADDR`, `KVM_DEV_ARM_VGIC_GRP_NR_IRQS`, `KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS`, `KVM_DEV_ARM_VGIC_GRP_IRS_REGS`, and `KVM_DEV_ARM_VGIC_GRP_IST` is a pretty clear indicator of this. The modifications to `arch/arm64/kvm/vgic/vgic-kvm-device.c` to handle these new attributes, coupled with the new files - `vgic-irs-v5.c`, `vgic-v5-tables.c`, and `vgic-sys-reg-v5.c` - all point to a core implementation change. Modifying `vgic-v5.c` to initialize the IRS and handle SPIs/LPIs further reinforces this. Now, let's identify the specific symbols to target with our fuzzer. We need to find the critical functions that handle the new IRS/SPI/LPI functionality. Looking through the provided list, the following functions stand out: * In `vgic-kvm-device.c`: `vgic_v5_has_attr`, `vgic_v5_set_attr`, `vgic_v5_get_attr`, `vgic_v5_attr_regs_access`, `vgic_v5_parse_attr`. These functions handle the attribute management, setting, getting, and parsing which will be core for the fuzzer. * In `vgic-irs-v5.c`: This file is gold. I see `kvm_vgic_v5_irs_init`, and all the `vgic_v5_mmio_*` functions. These handle memory-mapped I/O, registration, and initialization of the IRS, which are prime fuzzing targets. We'll be hitting `vgic_v5_irs_attr_regs_access`, `vgic_v5_has_attr_regs`, `vgic_v5_register_irs_iodev`, `vgic_v5_mmio_write_irs_misc`, `vgic_v5_mmio_read_irs_misc`, `vgic_v5_mmio_write_irs_spi`, `vgic_v5_mmio_read_irs_spi`, `vgic_v5_mmio_write_irs_ist`, `vgic_v5_mmio_read_irs_ist`, `vgic_v5_mmio_uaccess_write_irs` and `vgic_v5_mmio_uaccess_read_irs_status`. * In `vgic-v5-tables.c`: I see a lot of memory allocation functions here which will be critical to attack with a fuzzer: `vgic_v5_vmt_allocate`, `vgic_v5_vmt_free`, `vgic_v5_allocate_vm_id`, `vgic_v5_release_vm_id`, `vgic_v5_vmte_init`, `vgic_v5_vmte_release`, `vgic_v5_vmte_alloc_vpe`, `vgic_v5_vmte_free_vpe`, `vgic_v5_spi_ist_alloc`, `vgic_v5_lpi_ist_alloc`, `vgic_v5_lpi_ist_free`, `vgic_v5_save_spi_ist`, `vgic_v5_save_lpi_ist`, `vgic_v5_restore_spi_ist`, `vgic_v5_restore_lpi_ist`. * In `vgic-sys-reg-v5.c`: `vgic_v5_has_cpu_sysregs_attr`, `vgic_v5_cpu_sysregs_uaccess`. Handling system register access could also be a source of vulnerabilities. Now, a quick check for configuration options. I'll need to know if there's a specific `CONFIG_` option |
| 4/2 | 2026/07/25 16:10 | tool | grepper |
0mError:no matches Args: {
"Expression": "GICV5",
"PathPrefix": "arch/arm64/Kconfig"
}Results: {
"error": "no matches"
} |
| 5/2 | 2026/07/25 16:10 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 134820 output: 33 thoughts: 23 Thoughts: Okay, here's my summary, presented as a stream of thought: **Checking for GICv5 Support** Alright, time to get this sorted. I need to figure out if GICv5 support is present in this build configuration. My first step is to hunt down the Kconfig file that likely controls this. Given the architecture, `arch/arm64/` seems the obvious place to start. And within that, the Kconfig file should define features related to the kernel. Let's start with searching within `arch/arm64/Kconfig` for the string `GIC_V5`. This seems like the most direct approach. I'm expecting to find a definition, a dependency, or at least some mention of GICv5 within the Arm64 architecture's configuration file. This search will be the key to understanding if it's enabled and how. |
| 6/2 | 2026/07/25 16:10 | tool | grepper |
0mArgs:{
"Expression": "GIC_V5",
"PathPrefix": "arch/arm64/Kconfig"
}Results: {
"Output": "arch/arm64/Kconfig=2=config ARM64\n--\narch/arm64/Kconfig-106-\tselect ARM_GIC_V3_ITS if PCI\narch/arm64/Kconfig:107:\tselect ARM_GIC_V5\narch/arm64/Kconfig-108-\tselect ARM_PSCI_FW\n"
} |
| 7/2 | 2026/07/25 16:10 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 135166 output: 35 thoughts: 43 Thoughts: Okay, here's my take on those thoughts, framed for an expert audience: **Investigating the ARM GICv5 Configuration** Right, so `ARM_GIC_V5` is enabled, and it looks like that's happening because of `ARM64`. Okay, typical. But I need to understand the specifics of *how* that enablement works. My first step is to dive into the kernel configuration files, specifically the `drivers/irqchip/Kconfig` file. That's the logical place to start looking for the details on the `ARM_GIC_V5` configuration. I need to see the dependencies, any default settings, and any conditional logic associated with this selection. Knowing the Kconfig details is key to tracing this and understand the full implications and options regarding the GICv5 implementation on this platform. This will help me figure out how the GICv5 is being brought into the system and how to configure it correctly. |
| 8/2 | 2026/07/25 16:10 | tool | grepper |
0mArgs:{
"Expression": "ARM_GIC_V5",
"PathPrefix": "drivers/irqchip/Kconfig"
}Results: {
"Output": "drivers/irqchip/Kconfig=46=config ARM_GIC_V3_ITS\n--\ndrivers/irqchip/Kconfig-53-\ndrivers/irqchip/Kconfig:54:config ARM_GIC_V5\ndrivers/irqchip/Kconfig-55-\tbool\n"
} |
| 9/2 | 2026/07/25 16:10 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 135534 output: 35 thoughts: 67 Thoughts: **Analyzing the ARM64 KVM Build Process** Okay, so I'm looking at this build configuration, and it's pretty straightforward. It looks like `ARM_GIC_V5` is defined as a boolean, and the `ARM64` architecture selects it. That makes sense, given the focus on ARM64. Now, the key dependency here is `KVM`, because `arch/arm64/kvm/vgic/vgic-v5.c` relies on it during the compilation process. To understand the relationship more clearly, I should really dig into the `arch/arm64/kvm/Makefile` and see how everything is tied together. Let's see what that file reveals... |
| 10/2 | 2026/07/25 16:10 | tool | grepper |
0mArgs:{
"Expression": "vgic-v5",
"PathPrefix": "arch/arm64/kvm/Makefile"
}Results: {
"Output": "arch/arm64/kvm/Makefile=16=kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \\\n--\narch/arm64/kvm/Makefile-26-\t vgic/vgic-its.o vgic/vgic-debug.o vgic/vgic-v3-nested.o \\\narch/arm64/kvm/Makefile:27:\t vgic/vgic-v5.o vgic/vgic-v5-tables.o vgic/vgic-irs-v5.o \\\narch/arm64/kvm/Makefile-28-\t vgic-sys-reg-v5.o\n"
} |
| 11/2 | 2026/07/25 16:10 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 135975 output: 50 thoughts: 232 Thoughts: **KVM Compilation Configuration** Okay, so the code's designed to be compiled conditionally – it seems that if `KVM` is enabled, then the relevant parts of the code get compiled. Right, that makes perfect sense. In that case, we definitely need to make sure that `KVM` is included in `EnableConfigs`. It's a key dependency. Now, let's nail down the focus symbols. We're going to need a list to work with, and I've got a good starting set here: `vgic_v5_has_attr`, `vgic_v5_set_attr`, `vgic_v5_get_attr`, `vgic_v5_attr_regs_access`, `vgic_v5_parse_attr`, `vgic_v5_irs_attr_regs_access`, `vgic_v5_has_attr_regs`, `kvm_vgic_v5_irs_init`, `vgic_v5_register_irs_iodev`, `vgic_v5_cpu_sysregs_uaccess`, `vgic_v5_has_cpu_sysregs_attr`, `vgic_v5_irs_save_ists`, and `vgic_v5_irs_restore_ists`. Seems like a thorough set, hitting all the key aspects of the V5 interrupt controller and its integration with KVM. Before we go any further, and to ensure we don't trip over static or inline issues later, a quick check is in order. It's time to verify if these functions are declared as static. This will prevent any unintended linking of them. |
| 12/2 | 2026/07/25 16:11 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_has_attr",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-kvm-device.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-kvm-device.c=950=static int vgic_v5_get_attr(struct kvm_device *dev,\n--\narch/arm64/kvm/vgic/vgic-kvm-device.c-983-\narch/arm64/kvm/vgic/vgic-kvm-device.c:984:static int vgic_v5_has_attr(struct kvm_device *dev,\narch/arm64/kvm/vgic/vgic-kvm-device.c-985-\t\t\t struct kvm_device_attr *attr)\n--\narch/arm64/kvm/vgic/vgic-kvm-device.c-996-\tcase KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS:\narch/arm64/kvm/vgic/vgic-kvm-device.c:997:\t\treturn vgic_v5_has_attr_regs(dev, attr);\narch/arm64/kvm/vgic/vgic-kvm-device.c-998-\tcase KVM_DEV_ARM_VGIC_GRP_NR_IRQS:\n--\narch/arm64/kvm/vgic/vgic-kvm-device.c=1018=struct kvm_device_ops kvm_arm_vgic_v5_ops = {\n--\narch/arm64/kvm/vgic/vgic-kvm-device.c-1023-\t.get_attr = vgic_v5_get_attr,\narch/arm64/kvm/vgic/vgic-kvm-device.c:1024:\t.has_attr = vgic_v5_has_attr,\narch/arm64/kvm/vgic/vgic-kvm-device.c-1025-};\n"
} |
| 13/2 | 2026/07/25 16:11 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 137102 output: 50 thoughts: 42 Thoughts: Okay, here's my take on those thoughts, framed from my perspective as an expert in this area: **Analyzing Vgic_v5 Attribute Management** Alright, so `vgic_v5_has_attr` is a static function, which means its behavior is fixed at compile time and doesn't rely on any external state, which is typical for this type of check. Now, where the real action happens, the key to understanding this architecture, is in how the attributes are *set* and *retrieved*. I need to immediately look into `vgic_v5_set_attr` and `vgic_v5_get_attr`. Those are the functions that will define how the attributes for the Virtual Generic Interrupt Controller version 5 are being configured, used, or modified. They are the core of the attribute management logic; I need to understand what's happening at those two points. Those two functions are definitely where the interesting interactions and potential complexities will reside. |
| 14/2 | 2026/07/25 16:11 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_set_attr",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-kvm-device.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-kvm-device.c=833=static int vgic_v5_attr_regs_access(struct kvm_device *dev,\n--\narch/arm64/kvm/vgic/vgic-kvm-device.c-916-\narch/arm64/kvm/vgic/vgic-kvm-device.c:917:static int vgic_v5_set_attr(struct kvm_device *dev,\narch/arm64/kvm/vgic/vgic-kvm-device.c-918-\t\t\t struct kvm_device_attr *attr)\n--\narch/arm64/kvm/vgic/vgic-kvm-device.c=1018=struct kvm_device_ops kvm_arm_vgic_v5_ops = {\n--\narch/arm64/kvm/vgic/vgic-kvm-device.c-1021-\t.destroy = vgic_destroy,\narch/arm64/kvm/vgic/vgic-kvm-device.c:1022:\t.set_attr = vgic_v5_set_attr,\narch/arm64/kvm/vgic/vgic-kvm-device.c-1023-\t.get_attr = vgic_v5_get_attr,\n"
} |
| 15/2 | 2026/07/25 16:11 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 137754 output: 50 thoughts: 28 Thoughts: Okay, here's my interpretation of those thoughts, crafted with the understanding that I'm speaking to a fellow expert: **Navigating the Static Labyrinth** Hmm, things feel a bit…*static* right now. I need to break free from these stagnant points, and I know that means finding some dynamic function calls. My focus shifts to the volatile nature of the system. I'm going to dig deeper. Let's see if `vgic_v5_parse_attr` offers a more dynamic entry point to the system. |
| 16/2 | 2026/07/25 16:11 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_parse_attr",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-kvm-device.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-kvm-device.c=753=static int vgic_v5_get_userspace_ppis(struct kvm_device *dev,\n--\narch/arm64/kvm/vgic/vgic-kvm-device.c-775-\narch/arm64/kvm/vgic/vgic-kvm-device.c:776:int vgic_v5_parse_attr(struct kvm_device *dev, struct kvm_device_attr *attr,\narch/arm64/kvm/vgic/vgic-kvm-device.c-777-\t\t struct vgic_reg_attr *reg_attr)\n--\narch/arm64/kvm/vgic/vgic-kvm-device.c=833=static int vgic_v5_attr_regs_access(struct kvm_device *dev,\n--\narch/arm64/kvm/vgic/vgic-kvm-device.c-843-\narch/arm64/kvm/vgic/vgic-kvm-device.c:844:\tret = vgic_v5_parse_attr(dev, attr, \u0026reg_attr);\narch/arm64/kvm/vgic/vgic-kvm-device.c-845-\tif (ret)\n"
} |
| 17/2 | 2026/07/25 16:11 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 138297 output: 54 thoughts: 36 Thoughts: Here's my summary, as though I'm thinking through this process: **Diving into vgic_v5_parse_attr** Okay, so `vgic_v5_parse_attr` is non-static, which is an important first observation. That means it might be called from various places in the code and isn't isolated to a single compilation unit. I need to understand its context. Let's start by looking at `vgic_v5_irs_attr_regs_access`. I'm guessing this function likely provides some input for attribute parsing or consumes the output. This seems like the logical next step to figure out how `vgic_v5_parse_attr` fits into the wider picture of Vgic v5's interrupt routing and attribute handling. I'm going to look at the interactions between these two functions. |
| 18/2 | 2026/07/25 16:11 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_irs_attr_regs_access",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-irs-v5.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-irs-v5.c=1051=int vgic_v5_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)\n--\narch/arm64/kvm/vgic/vgic-irs-v5.c-1086- */\narch/arm64/kvm/vgic/vgic-irs-v5.c:1087:int vgic_v5_irs_attr_regs_access(struct kvm_device *dev,\narch/arm64/kvm/vgic/vgic-irs-v5.c-1088-\t\t\t\t struct kvm_device_attr *attr,\n"
} |
| 19/2 | 2026/07/25 16:11 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 138762 output: 52 thoughts: 38 Thoughts: Okay, here's my take on those thoughts, framed as an expert's internal monologue: **Investigating Vgic_v5 Attribute Register Access** Alright, so `vgic_v5_irs_attr_regs_access` isn't static, which is interesting and potentially important for how we manage access to those attribute registers. This immediately raises a question: *why* isn't it static? Are we dealing with something that requires dynamic initialization or perhaps multiple instances? The next logical step is to quickly confirm the obvious related check, `vgic_v5_has_attr_regs`. Let's establish that the *basic* prerequisites are met first. We need to know if the hardware even *supports* these attribute registers before we dig any deeper into the access method. This foundational check will dictate the validity of any further investigation. If it's *not* present, we don't need to waste time investigating register access methods. If it *is* present, well, then the investigation continues! |
| 20/2 | 2026/07/25 16:11 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_has_attr_regs",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-irs-v5.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-irs-v5.c=1031=int vgic_v5_irs_lpi_ist_id_bits(struct kvm *kvm, unsigned int *id_bits)\n--\narch/arm64/kvm/vgic/vgic-irs-v5.c-1050-\narch/arm64/kvm/vgic/vgic-irs-v5.c:1051:int vgic_v5_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)\narch/arm64/kvm/vgic/vgic-irs-v5.c-1052-{\n"
} |
| 21/2 | 2026/07/25 16:11 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 139257 output: 53 thoughts: 37 Thoughts: Here's my analysis of the provided snippet, framed as my own thought process: **Diving into the Vgic_v5 Landscape** Okay, so `vgic_v5_has_attr_regs` isn't static, meaning it's likely accessible from outside its immediate scope. That's a good starting point. Now, to understand how it's used, my next step is to examine `kvm_vgic_v5_irs_init`. That function call is a key piece in this puzzle. I'm assuming IRS stands for Interrupt Routing Set - it's a good bet the initialization of the interrupt routing structure is where I'll see how `vgic_v5_has_attr_regs` plays its role, if any. Digging into `kvm_vgic_v5_irs_init` should reveal if the non-static nature of that attribute plays a pivotal role in the Vgic_v5 initialization flow. I'm anticipating that this is where the conditional logic, potentially dependent on the existence or absence of the attribute registers, is implemented. Let's see what we find. |
| 22/2 | 2026/07/25 16:11 | tool | grepper |
0mArgs:{
"Expression": "kvm_vgic_v5_irs_init",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-irs-v5.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-irs-v5.c=874=int vgic_v5_register_irs_iodev(struct kvm *kvm, gpa_t irs_base_address)\n--\narch/arm64/kvm/vgic/vgic-irs-v5.c-896-/**\narch/arm64/kvm/vgic/vgic-irs-v5.c:897: * kvm_vgic_v5_irs_init: initialize the IRS data structures\narch/arm64/kvm/vgic/vgic-irs-v5.c-898- * @kvm: kvm struct pointer\n--\narch/arm64/kvm/vgic/vgic-irs-v5.c-900- */\narch/arm64/kvm/vgic/vgic-irs-v5.c:901:int kvm_vgic_v5_irs_init(struct kvm *kvm, unsigned int nr_spis)\narch/arm64/kvm/vgic/vgic-irs-v5.c-902-{\n"
} |
| 23/2 | 2026/07/25 16:11 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 139885 output: 54 thoughts: 39 Thoughts: Here's my analysis of the provided snippet, framed as my own thought process: **Diving into the KVM vGIC v5 IRS Initialization** Okay, so I see that `kvm_vgic_v5_irs_init` isn't declared `static`. That's the first thing I notice. This means it's likely accessible from outside the specific compilation unit it's defined in. This is important to understand for any potential inter-module dependencies or if other parts of the system are directly calling it. Now, given that observation, I need to understand *why* it's non-static. The next logical step is to dig into where and how this function is used. Let's see... Let's check `vgic_v5_register_irs_iodev` to see if that's where the function is being used. That's likely the key. By examining how this function is *called* or *referenced*, I can deduce its role in the larger vGIC v5 initialization sequence and its overall architecture. |
| 24/2 | 2026/07/25 16:11 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_register_irs_iodev",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-irs-v5.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-irs-v5.c=863=unsigned int vgic_v5_init_irs_iodev(struct vgic_io_device *dev)\n--\narch/arm64/kvm/vgic/vgic-irs-v5.c-873-\narch/arm64/kvm/vgic/vgic-irs-v5.c:874:int vgic_v5_register_irs_iodev(struct kvm *kvm, gpa_t irs_base_address)\narch/arm64/kvm/vgic/vgic-irs-v5.c-875-{\n"
} |
| 25/2 | 2026/07/25 16:11 | llm | patch-evaluator |
1mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 140386 output: 53 thoughts: 40 Thoughts: Here's my take on the thought process, as requested: **Navigating the Vgic_v5 Register and CPU Sysreg Landscape** Okay, so `vgic_v5_register_irs_iodev` isn't static. That immediately piques my interest – a non-static function in a driver usually means it's designed to be called from multiple places, or potentially from outside the core module itself. That means I need to understand its access patterns very clearly. The fact that it's part of the vgic_v5 subsystem tells me it's likely responsible for registering interrupts or managing I/O devices related to the GICv5. I’ll make a mental note to keep this in mind. Now, my next logical step is to dive into how this function is *used*. The text mentions checking `vgic_v5_cpu_sysregs_uaccess`, which likely relates to CPU system register access, specifically within the user space context. This gives me a good pointer, it’s going to be the next place I need to dig into. I know the GICv5 architecture well, and I suspect there's a connection between how the system registers and the IRS (Interrupt Routing Set) are managed when handling interrupts or I/O. Perhaps `vgic_v5_register_irs_iodev` interacts with `vgic_v5_cpu_sysregs_uaccess` to handle privileged access, or security features. This is the likely area of interest to investigate further. |
| 26/2 | 2026/07/25 16:12 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_cpu_sysregs_uaccess",
"PathPrefix": "arch/arm64/kvm/vgic-sys-reg-v5.c"
}Results: {
"Output": "arch/arm64/kvm/vgic-sys-reg-v5.c=491=int vgic_v5_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)\n--\narch/arm64/kvm/vgic-sys-reg-v5.c-503-\narch/arm64/kvm/vgic-sys-reg-v5.c:504:int vgic_v5_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu,\narch/arm64/kvm/vgic-sys-reg-v5.c-505-\t\t\t\tstruct kvm_device_attr *attr,\n"
} |
| 27/2 | 2026/07/25 16:12 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 140994 output: 54 thoughts: 41 Thoughts: Okay, here's my interpretation of those thoughts, tailored for an expert audience, presented in the first person: **Analyzing `vgic_v5_cpu_sysregs_uaccess`** Alright, so I'm looking at `vgic_v5_cpu_sysregs_uaccess`. The first thing that jumps out is that this function isn't declared static. That immediately raises a question in my mind about its scope and potential external usage – something to keep in mind when assessing its overall design and impact. Now, I need to understand this function's context a bit better. My next logical step is to quickly check `vgic_v5_has_cpu_sysregs_attr`. That'll likely tell me whether we even *need* this function in the first place, or if its functionality is conditional. This attribute check could be the gatekeeper for its use, or might just hint at the architectural support and presence of the underlying system registers the function interacts with. It's all about verifying assumptions before digging deeper. |
| 28/2 | 2026/07/25 16:13 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_has_cpu_sysregs_attr",
"PathPrefix": "arch/arm64/kvm/vgic-sys-reg-v5.c"
}Results: {
"Output": "arch/arm64/kvm/vgic-sys-reg-v5.c=482=static u64 attr_to_id(u64 attr)\n--\narch/arm64/kvm/vgic-sys-reg-v5.c-490-\narch/arm64/kvm/vgic-sys-reg-v5.c:491:int vgic_v5_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr)\narch/arm64/kvm/vgic-sys-reg-v5.c-492-{\n"
} |
| 29/2 | 2026/07/25 16:13 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 141479 output: 52 thoughts: 39 Thoughts: Okay, here's my take on those thoughts, framed as my own reasoning: **Analyzing `vgic_v5_has_cpu_sysregs_attr` and `vgic_v5_irs_save_ists`** Right, `vgic_v5_has_cpu_sysregs_attr` isn't static. That immediately makes me think about how it's being used across the code base and the potential implications for reentrancy or concurrency. If it *were* static, the scope and lifetime would be much easier to control. The fact that it's not indicates a potential for interactions with other parts of the system that need to be carefully considered. Next, I need to understand how it's used and that naturally leads me to think of checking `vgic_v5_irs_save_ists`. I'm curious to see how the code is saving or restoring information related to Interrupt Routing Sets (IRS). The IRS likely uses system registers (given the `sysregs` part of the attribute name), so how this data is handled, whether there's atomicity, and the overall save/restore mechanism in relation to `vgic_v5_has_cpu_sysregs_attr` are all critical. The relationship between those two data points is the key to understanding this. |
| 30/2 | 2026/07/25 16:13 | tool | grepper |
0mError:no matches Args: {
"Expression": "vgic_v5_irs_save_ists",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-v5-tables.c"
}Results: {
"error": "no matches"
} |
| 31/2 | 2026/07/25 16:13 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 141863 output: 50 thoughts: 43 Thoughts: Here's my attempt at summarizing the thought process, tailored for an expert audience: **Hunting Down vgic_v5_irs_save_ists** Okay, hold on a second. Where did I last see that `vgic_v5_irs_save_ists` function? Seems important, and I can't quite place it off the top of my head. Right, let's go straight to the source, as usual. Given the naming convention, it's a pretty safe bet it's tucked away in the Vgic-v5 code, so let's navigate to `arch/arm64/kvm/vgic/vgic-v5.c`. That should be the definitive spot to check. Let's see what's in there... |
| 32/2 | 2026/07/25 16:13 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_irs_save_ists",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-v5.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-v5.c=1348=static int vgic_v5_validate_ist_attr(struct kvm *kvm,\n--\narch/arm64/kvm/vgic/vgic-v5.c-1378-\narch/arm64/kvm/vgic/vgic-v5.c:1379:int vgic_v5_irs_save_ists(struct kvm *kvm, struct kvm_device_attr *attr)\narch/arm64/kvm/vgic/vgic-v5.c-1380-{\n"
} |
| 33/2 | 2026/07/25 16:13 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 142292 output: 50 thoughts: 36 Thoughts: Here's my take on the thought process, as if it were my own: **Investigating ISTS Register Management in VGICv5** Okay, so `vgic_v5_irs_save_ists` is *not* static. That's the first thing to note. Now, based on that observation, my next logical step is to delve into the companion function, `vgic_v5_irs_restore_ists`. Understanding how the Intermediate State Translation Store (ISTS) registers are handled during *restoration* is key, given that `vgic_v5_irs_save_ists` is non-static which implies there is a process of saving and potentially restoring the ISTS registers, thus, I need to know how the restoration works. This tells a good story about the process. |
| 34/2 | 2026/07/25 16:13 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_irs_restore_ists",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-v5.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-v5.c=1469=static void vgic_v5_restore_cleanup(struct kvm *kvm,\n--\narch/arm64/kvm/vgic/vgic-v5.c-1485-\narch/arm64/kvm/vgic/vgic-v5.c:1486:int vgic_v5_irs_restore_ists(struct kvm *kvm, struct kvm_device_attr *attr)\narch/arm64/kvm/vgic/vgic-v5.c-1487-{\n"
} |
| 35/2 | 2026/07/25 16:13 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 142717 output: 51 thoughts: 438 Thoughts: **Analyzing Non-Static Function Usage in the vgic_v5 Module** Okay, so I've been looking at `vgic_v5_irs_restore_ists` and it's non-static. This means it's likely called from outside its defining compilation unit, and I need to understand its dependencies. That brings me to these focus symbols: `vgic_v5_parse_attr`, `vgic_v5_irs_attr_regs_access`, `vgic_v5_has_attr_regs`, `kvm_vgic_v5_irs_init`, `vgic_v5_register_irs_iodev`, `vgic_v5_cpu_sysregs_uaccess`, `vgic_v5_has_cpu_sysregs_attr`, `vgic_v5_irs_save_ists`, and of course `vgic_v5_irs_restore_ists` itself. I need to trace the call graph and see how these interact, especially focusing on how the IRQ state is being handled. Just to be thorough, I'm going to quickly check if there are any other non-static functions floating around in the `vgic_v5` implementation that could be relevant to this IRQ handling. Specifically, I'm looking at `arch/arm64/kvm/vgic/vgic-v5-tables.c`, and I see a whole slew of them: `vgic_v5_vmt_allocate`, `vgic_v5_vmt_free`, `vgic_v5_allocate_vm_id`, `vgic_v5_release_vm_id`, `vgic_v5_vmte_init`, `vgic_v5_vmte_release`, `vgic_v5_vmte_alloc_vpe`, `vgic_v5_vmte_free_vpe`, `vgic_v5_spi_ist_alloc`, `vgic_v5_lpi_ist_alloc`, `vgic_v5_lpi_ist_free`, `vgic_v5_save_spi_ist`, `vgic_v5_save_lpi_ist`, `vgic_v5_restore_spi_ist`, and `vgic_v5_restore_lpi_ist`. These could be involved in setting up the virtual interrupt state, and I need to keep them in mind as I trace the flow. This list of functions points toward memory management and initialization of the virtual interrupt tables and state. Now I need to consider how these functions are being called, and if they have any interaction with `vgic_v5_irs_restore_ists`. It appears that these functions could represent a significant degree of the KVM vgic code base. |
| 36/2 | 2026/07/25 16:13 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_vmt_allocate",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-v5-tables.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-v5-tables.c=213=static int vgic_v5_alloc_l2_vmt(struct kvm *kvm)\n--\narch/arm64/kvm/vgic/vgic-v5-tables.c-284- */\narch/arm64/kvm/vgic/vgic-v5-tables.c:285:int vgic_v5_vmt_allocate(unsigned int max_vpes)\narch/arm64/kvm/vgic/vgic-v5-tables.c-286-{\n"
} |
| 37/2 | 2026/07/25 16:13 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 143982 output: 51 thoughts: 34 Thoughts: Here's my take on the thought process: **Diving into the Vgic Code** Okay, so `vgic_v5_vmt_allocate` isn't static. That's a key observation. Now, where does it get used? Given its name, it's obviously involved in VM-related allocation within the Generic Interrupt Controller v5 (VGIC). I need to understand the initialization sequence to get a grip on how this allocation fits in. Let's see what `vgic_v5_vmte_init` does and how it interacts with the allocation process. That function likely initializes the Virtual Machine Translation Engine (VMTE) data structures, so examining that is the logical next step. |
| 38/2 | 2026/07/25 16:14 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_vmte_init",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-v5-tables.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-v5-tables.c=439=void vgic_v5_release_vm_id(struct kvm *kvm)\n--\narch/arm64/kvm/vgic/vgic-v5-tables.c-453- */\narch/arm64/kvm/vgic/vgic-v5-tables.c:454:int vgic_v5_vmte_init(struct kvm *kvm)\narch/arm64/kvm/vgic/vgic-v5-tables.c-455-{\n"
} |
| 39/2 | 2026/07/25 16:14 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 144381 output: 52 thoughts: 35 Thoughts: Here's a summary of my immediate thought process, based on the provided snippets, geared towards someone already familiar with the subject: **Navigating Vgic_v5 Initialization** Okay, so `vgic_v5_vmte_init` isn't declared `static`, which is an immediate data point. Now, that being the case, the next logical area to probe would be `vgic_v5_spi_ist_alloc`. My hunch is this is the next level of initialization or potentially a related allocation function. Let me see where this leads, and what we find. |
| 40/2 | 2026/07/25 16:14 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_spi_ist_alloc",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-v5-tables.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-v5-tables.c=1123=static unsigned int vgic_v5_ist_istsz(unsigned int id_bits)\n--\narch/arm64/kvm/vgic/vgic-v5-tables.c-1140- */\narch/arm64/kvm/vgic/vgic-v5-tables.c:1141:int vgic_v5_spi_ist_alloc(struct kvm *kvm, unsigned int id_bits)\narch/arm64/kvm/vgic/vgic-v5-tables.c-1142-{\n"
} |
| 41/2 | 2026/07/25 16:14 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 144766 output: 53 thoughts: 37 Thoughts: Here's my analysis of the provided information, as if these were my internal thoughts: **Analyzing IST Allocation Functions** Okay, so `vgic_v5_spi_ist_alloc` is defined with external linkage, which means it's accessible from other compilation units. That immediately sparks my interest; I wonder if it has any dependencies or relationships I need to be aware of. Let's delve deeper and contrast it with `vgic_v5_lpi_ist_alloc`. I need to understand the characteristics of both of them. |
| 42/2 | 2026/07/25 16:14 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_lpi_ist_alloc",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-v5-tables.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-v5-tables.c=1185=int vgic_v5_lpi_ist_exists(struct kvm *kvm)\n--\narch/arm64/kvm/vgic/vgic-v5-tables.c-1203- */\narch/arm64/kvm/vgic/vgic-v5-tables.c:1204:int vgic_v5_lpi_ist_alloc(struct kvm *kvm, unsigned int id_bits)\narch/arm64/kvm/vgic/vgic-v5-tables.c-1205-{\n"
} |
| 43/2 | 2026/07/25 16:14 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 145147 output: 52 thoughts: 37 Thoughts: Here's my thought process, as if I were analyzing this code: **Analyzing `vgic_v5_lpi_ist_alloc` and `vgic_v5_save_spi_ist`** Okay, so `vgic_v5_lpi_ist_alloc` isn't declared as `static`. That immediately grabs my attention. Knowing this is a crucial piece of the puzzle, because it dictates its linkage and scope. Now, I need to understand its relationship to other functions, especially in terms of data sharing and potential side effects. Given that it's *not* static, I'm going to quickly check `vgic_v5_save_spi_ist`. My immediate question is: does `vgic_v5_save_spi_ist` call, or potentially interact with, `vgic_v5_lpi_ist_alloc`? Are they part of a tightly coupled module, or are they more loosely related? I need to understand their control flow to see if there's any implicit dependency or interaction between the two routines. |
| 44/2 | 2026/07/25 16:14 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_save_spi_ist",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-v5-tables.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-v5-tables.c=1417=static int vgic_v5_save_two_level_ist(const struct vgic_v5_ist_desc *ist,\n--\narch/arm64/kvm/vgic/vgic-v5-tables.c-1466- */\narch/arm64/kvm/vgic/vgic-v5-tables.c:1467:int vgic_v5_save_spi_ist(struct kvm *kvm, struct kvm_vgic_v5_ist *ist_attr)\narch/arm64/kvm/vgic/vgic-v5-tables.c-1468-{\n"
} |
| 45/2 | 2026/07/25 16:14 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 145661 output: 53 thoughts: 37 Thoughts: Okay, here's my interpretation of those thoughts, framed as my own: **Investigating Interrupt State Save Functions** Alright, so `vgic_v5_save_spi_ist` – that one isn't static. Good to know. Let's make sure we document that. Now, given that it's non-static, I should take a look at `vgic_v5_save_lpi_ist`. We'll see how that one is defined and that should give us some insight into how they have decided to implement these state saves and, more importantly, if the implementation of the second interrupt saving function is logically consistent with the first. A comparative analysis might surface some design decisions or potential areas where we could refactor for greater consistency or clarity. My initial thought is that these functions likely have some similarities and differences that could be worth exploring. |
| 46/2 | 2026/07/25 16:14 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_save_lpi_ist",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-v5-tables.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-v5-tables.c=1467=int vgic_v5_save_spi_ist(struct kvm *kvm, struct kvm_vgic_v5_ist *ist_attr)\n--\narch/arm64/kvm/vgic/vgic-v5-tables.c-1489- */\narch/arm64/kvm/vgic/vgic-v5-tables.c:1490:int vgic_v5_save_lpi_ist(struct kvm *kvm, struct kvm_vgic_v5_ist *ist_attr)\narch/arm64/kvm/vgic/vgic-v5-tables.c-1491-{\n"
} |
| 47/2 | 2026/07/25 16:14 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 146134 output: 52 thoughts: 37 Thoughts: Here's my analysis of the provided snippet, framed as my own thought process: **Diving into the vgic_v5 Landscape** Okay, so `vgic_v5_save_lpi_ist` is flagged as non-static. Interesting. That suggests it might be handling some state that needs to be accessible outside of its immediate scope, potentially global state or shared data. Now, to get a better handle on its context, I need to look at its counterpart for restoration. Let's see... I'll check `vgic_v5_restore_spi_ist`. Understanding how these save and restore functions work together is crucial to grasping the larger picture of the interrupt controller's operation within this version of the architecture. Knowing the relationship between `lpi_ist` and `spi_ist` is essential to identify the underlying purpose of this code block. |
| 48/2 | 2026/07/25 16:14 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_restore_spi_ist",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-v5-tables.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-v5-tables.c=1662=static int vgic_v5_restore_two_level_ist(struct kvm *kvm,\n--\narch/arm64/kvm/vgic/vgic-v5-tables.c-1716- */\narch/arm64/kvm/vgic/vgic-v5-tables.c:1717:int vgic_v5_restore_spi_ist(struct kvm *kvm, struct kvm_vgic_v5_ist *ist_attr)\narch/arm64/kvm/vgic/vgic-v5-tables.c-1718-{\n"
} |
| 49/2 | 2026/07/25 16:14 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 146594 output: 53 thoughts: 37 Thoughts: Okay, here's my interpretation of those thoughts, tailored for an expert audience, presented in the first person: **Investigating VGICv5 Restore Procedures** Right, so `vgic_v5_restore_spi_ist` isn't static. That's a key observation. Now, since we're looking at VGICv5 restore procedures, it makes logical sense to immediately jump over to `vgic_v5_restore_lpi_ist`. We've established one isn't static, so we need to see if the other is. This suggests a pattern of related restore functionality, and the non-static attribute of one informs how we approach the other. The fact that the code is structured in such a way merits further scrutiny to see if there is potential for optimizations or if something is missing. Understanding this dynamic will be key to understanding the full picture of the VGICv5 restore sequence. My expectation is that the calling context or usage of `vgic_v5_restore_lpi_ist` will be tightly coupled with the findings related to `vgic_v5_restore_spi_ist`. |
| 50/2 | 2026/07/25 16:14 | tool | grepper |
0mArgs:{
"Expression": "vgic_v5_restore_lpi_ist",
"PathPrefix": "arch/arm64/kvm/vgic/vgic-v5-tables.c"
}Results: {
"Output": "arch/arm64/kvm/vgic/vgic-v5-tables.c=1717=int vgic_v5_restore_spi_ist(struct kvm *kvm, struct kvm_vgic_v5_ist *ist_attr)\n--\narch/arm64/kvm/vgic/vgic-v5-tables.c-1740- */\narch/arm64/kvm/vgic/vgic-v5-tables.c:1741:int vgic_v5_restore_lpi_ist(struct kvm *kvm, struct kvm_vgic_v5_ist *ist_attr)\narch/arm64/kvm/vgic/vgic-v5-tables.c-1742-{\n"
} |
| 51/2 | 2026/07/25 16:14 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 147124 output: 473 thoughts: 502 Thoughts: **My Analysis of `vgic_v5_restore_lpi_ist` and Related Symbols** Okay, so I've identified that `vgic_v5_restore_lpi_ist` isn't static. This means it's accessible externally, which is important to consider for its interactions. To fully understand its context, I need to focus on the surrounding symbols it interacts with. My thinking is, the key players are: * `vgic_v5_parse_attr` * `vgic_v5_irs_attr_regs_access` * `vgic_v5_has_attr_regs` * `kvm_vgic_v5_irs_init` * `vgic_v5_register_irs_iodev` * `vgic_v5_cpu_sysregs_uaccess` * `vgic_v5_has_cpu_sysregs_attr` * `vgic_v5_irs_save_ists` * `vgic_v5_irs_restore_ists` * `vgic_v5_vmt_allocate` * `vgic_v5_vmt_free` * `vgic_v5_allocate_vm_id` * `vgic_v5_release_vm_id` * `vgic_v5_vmte_init` * `vgic_v5_vmte_release` * `vgic_v5_vmte_alloc_vpe` * `vgic_v5_vmte_free_vpe` * `vgic_v5_spi_ist_alloc` * `vgic_v5_lpi_ist_alloc` * `vgic_v5_lpi_ist_free` * `vgic_v5_save_spi_ist` * `vgic_v5_save_lpi_ist` * `vgic_v5_restore_spi_ist` * `vgic_v5_restore_lpi_ist` I will need to ensure that the code I'm working with has the appropriate understanding of these other symbols. Now, to satisfy the configuration requirements, I'm fairly certain `KVM` is the only relevant config. However, I want to confirm. The patch itself touches `arch/arm64/kvm/vgic/vgic-v5.c`. Given my knowledge of the kernel, I know that this file is only compiled when `KVM` is enabled, and in this architecture, `ARM_GIC_V5` is selected by `ARM64`. So, I'm confident `KVM` is indeed sufficient. I will proceed with this as the assumption. Now that I have confirmed the analysis, I'm going to call `set-results`. |
| 52/2 | 2026/07/25 16:15 | tool | set-results |
0mArgs:{
"EnableConfigs": [
"KVM"
],
"FocusSymbols": [
"vgic_v5_parse_attr",
"vgic_v5_irs_attr_regs_access",
"vgic_v5_has_attr_regs",
"kvm_vgic_v5_irs_init",
"vgic_v5_register_irs_iodev",
"vgic_v5_cpu_sysregs_uaccess",
"vgic_v5_has_cpu_sysregs_attr",
"vgic_v5_irs_save_ists",
"vgic_v5_irs_restore_ists",
"vgic_v5_vmt_allocate",
"vgic_v5_vmt_free",
"vgic_v5_allocate_vm_id",
"vgic_v5_release_vm_id",
"vgic_v5_vmte_init",
"vgic_v5_vmte_release",
"vgic_v5_vmte_alloc_vpe",
"vgic_v5_vmte_free_vpe",
"vgic_v5_spi_ist_alloc",
"vgic_v5_lpi_ist_alloc",
"vgic_v5_lpi_ist_free",
"vgic_v5_save_spi_ist",
"vgic_v5_save_lpi_ist",
"vgic_v5_restore_spi_ist",
"vgic_v5_restore_lpi_ist"
],
"Reasoning": "The patch series introduces support for GICv5 IRS (Interrupt Routing Service), SPIs, and LPIs in KVM. It adds new KVM device attributes for GICv5, including IRS registers, CPU sysregs, and IST (Interrupt State Table) save/restore functionality. This is a significant functional change to the KVM ARM VGIC subsystem that introduces new reachable code paths and APIs, making it highly relevant for fuzzing.",
"WorthFuzzing": true
}Results: {
"EnableConfigs": [
"KVM"
],
"FocusSymbols": [
"vgic_v5_parse_attr",
"vgic_v5_irs_attr_regs_access",
"vgic_v5_has_attr_regs",
"kvm_vgic_v5_irs_init",
"vgic_v5_register_irs_iodev",
"vgic_v5_cpu_sysregs_uaccess",
"vgic_v5_has_cpu_sysregs_attr",
"vgic_v5_irs_save_ists",
"vgic_v5_irs_restore_ists",
"vgic_v5_vmt_allocate",
"vgic_v5_vmt_free",
"vgic_v5_allocate_vm_id",
"vgic_v5_release_vm_id",
"vgic_v5_vmte_init",
"vgic_v5_vmte_release",
"vgic_v5_vmte_alloc_vpe",
"vgic_v5_vmte_free_vpe",
"vgic_v5_spi_ist_alloc",
"vgic_v5_lpi_ist_alloc",
"vgic_v5_lpi_ist_free",
"vgic_v5_save_spi_ist",
"vgic_v5_save_lpi_ist",
"vgic_v5_restore_spi_ist",
"vgic_v5_restore_lpi_ist"
],
"Reasoning": "The patch series introduces support for GICv5 IRS (Interrupt Routing Service), SPIs, and LPIs in KVM. It adds new KVM device attributes for GICv5, including IRS registers, CPU sysregs, and IST (Interrupt State Table) save/restore functionality. This is a significant functional change to the KVM ARM VGIC subsystem that introduces new reachable code paths and APIs, making it highly relevant for fuzzing.",
"WorthFuzzing": true
} |
| Total Calls | Total Tokens | Avg Tokens | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|
| Total Calls | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|