AI Trajectory

Trajectory: (Expand/Collapse All)
Seq	Timestamp	Type	Name	Duration
0/0	2026/09/11 21:49	flow	patch-triage	5m Args: null Results: { "EnableConfigs": [ "KVM", "KVM_INTEL", "X86_FRED" ], "FocusSymbols": [ "vmx_inject_exception", "vmx_set_msr", "vmx_get_msr", "kvm_deliver_exception_payload", "kvm_vcpu_ioctl_x86_set_vcpu_events", "kvm_vcpu_ioctl_x86_get_vcpu_events", "nested_check_vm_entry_controls", "nested_vmx_check_guest_state", "load_vmcs12_host_state", "this_cpu_fred_rsp" ], "KMSANReasoning": "The patch series adds support for Intel FRED (Flexible Return and Event Delivery) in x86 KVM, including vCPU events ABI updates, FRED MSR emulation/context-switching, nested VMX FRED state support, and CPU entry area exception stack refactoring.\n\nKMSAN vs KASAN Applicability Analysis:\n1. Kernel-to-Userspace Leaks:\n - `struct kvm_vcpu_events`: The newly added fields (`exception_event_data` and `exception_is_nested`) replace part of the existing 26-byte reserved array. In `kvm_vcpu_ioctl_x86_get_vcpu_events()` (and the `store_regs()` path), `memset(events, 0, sizeof(*events))` is called prior to populating any fields, ensuring all data, reserved bytes, and padding are fully zeroed before copying to userspace via `copy_to_user()`.\n - `KVM_GET_MSRS`: FRED MSR values returned to userspace originate from guest VMCS fields initialized to 0 in `init_vmcs()`, or from vCPU struct fields (`msr_guest_fred_rsp0`, `fred_ssp0_fallback`) allocated via `kmem_cache_zalloc()`.\n - Nested VMX (`struct vmcs12`): The `cached_vmcs12` buffers copied to userspace via `KVM_GET_NESTED_STATE` are allocated with `kzalloc()`.\n2. Uninitialized Memory in Control Flow:\n - The new fields in `struct kvm_queued_exception` (`is_nested`, `event_data`) are zero-initialized on vCPU allocation, cleared explicitly in `kvm_clear_exception_queue()` and `kvm_arch_vcpu_ioctl_run()`, and explicitly assigned whenever exceptions are queued or requeued (`kvm_multiple_exception()`, `kvm_requeue_exception()`).\n - The CPU entry area exception stacks are static per-CPU allocations mapped with guard pages.\n\nAny potential bugs in this series involve canonical address verification, reserved bit validation, or nested virtualization state machine logic, which are covered by functional testing and standard sanitizers (KASAN). There are no uninitialized memory usage or leak risks requiring KMSAN.", "NeedsKMSAN": false, "Reasoning": "The patch introduces support for Intel FRED (Flexible Return and Event Delivery) across x86 core exception stacks, KVM x86 emulation, VMX, and nested VMX virtualization. It modifies reachable ioctl entrypoints (KVM_CAP_X86_FRED_EVENT, KVM_SET_VCPU_EVENTS, KVM_GET_VCPU_EVENTS), MSR get/set handling, CR4 validation and emulation, exception payload delivery and injection, and nested VMX VM-entry/exit checks and state transition emulation (with newly added WARN_ON_ONCE assertions and consistency checks). These paths are reachable via /dev/kvm ioctls from userspace in virtualized environments.", "WorthFuzzing": true }
1/1	2026/09/11 21:49	action	read-patch-diff	0m Args: null Results: { "PatchDiff": "commit 77863fb69ef8d49cac01c1c41901061a73cc1032\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Fri Sep 11 21:49:27 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst\nindex 4eb7e75a7473f..22ef94f3eb3a1 100644\n--- a/Documentation/virt/kvm/api.rst\n+++ b/Documentation/virt/kvm/api.rst\n@@ -1199,6 +1199,10 @@ The following bits are defined in the flags field:\n triple_fault_pending field contains a valid state. This bit will\n be set whenever KVM_CAP_X86_TRIPLE_FAULT_EVENT is enabled.\n \n+- KVM_VCPUEVENT_VALID_FRED_STATE may be set to inform that the exception\n+ state includes FRED state (specifically, the event nested flag and data).\n+ This bit will be set whenever KVM_CAP_X86_FRED_EVENT is enabled.\n+\n ARM64:\n ^^^^^^\n \n@@ -1301,6 +1305,11 @@ If KVM_CAP_X86_TRIPLE_FAULT_EVENT is enabled, KVM_VCPUEVENT_VALID_TRIPLE_FAULT\n can be set in flags field to signal that the triple_fault field contains\n a valid state and shall be written into the VCPU.\n \n+If KVM_CAP_X86_FRED_EVENT is enabled, KVM_VCPUEVENT_VALID_FRED_STATE can be set\n+in the flags field to inform that the exception state contains FRED state\n+(specifically, the event nested flag and data), which shall be written into the\n+VCPU.\n+\n ARM64:\n ^^^^^^\n \n@@ -8979,6 +8988,19 @@ enabled, cmma can't be enabled anymore and pfmfi and the storage key\n interpretation are disabled. If cmma has already been enabled or the\n hpage_2g module parameter is not set to 1, -EINVAL is returned.\n \n+7.48 KVM_CAP_X86_FRED_EVENT\n+---------------------------\n+\n+:Architectures: x86\n+:Parameters: args[0] whether feature should be enabled or not\n+:Returns: 0 on success; -EINVAL if KVM cannot virtualize FRED.\n+\n+With this capability enabled, KVM allows exception save and restore operations\n+to include FRED event context (specifically, the event nested flag and data).\n+When injecting a FRED exception during VM entry, FRED event delivery relies\n+on this information to select the correct event stack level and apply proper\n+event data.\n+\n 8. Other capabilities.\n ======================\n \ndiff --git a/arch/x86/coco/sev/noinstr.c b/arch/x86/coco/sev/noinstr.c\nindex e1e03f12fc7bb..8a695ab67821b 100644\n--- a/arch/x86/coco/sev/noinstr.c\n+++ b/arch/x86/coco/sev/noinstr.c\n@@ -31,7 +31,7 @@ static __always_inline bool on_vc_stack(struct pt_regs regs)\n \tif (ip_within_syscall_gap(regs))\n \t\treturn false;\n \n-\treturn ((sp \u003e= __this_cpu_ist_bottom_va(VC)) \u0026\u0026 (sp \u003c __this_cpu_ist_top_va(VC)));\n+\treturn ((sp \u003e= __this_cpu_ist_bottom_va(ESTACK_VC)) \u0026\u0026 (sp \u003c __this_cpu_ist_top_va(ESTACK_VC)));\n }\n \n /\n@@ -83,7 +83,7 @@ void noinstr __sev_es_ist_exit(void)\n \t/* Read IST entry /\n \tist = __this_cpu_read(cpu_tss_rw.x86_tss.ist[IST_INDEX_VC]);\n \n-\tif (WARN_ON(ist == __this_cpu_ist_top_va(VC)))\n+\tif (WARN_ON(ist == __this_cpu_ist_top_va(ESTACK_VC)))\n \t\treturn;\n \n \t/ Read back old IST entry and write it to the TSS /\ndiff --git a/arch/x86/coco/sev/vc-handle.c b/arch/x86/coco/sev/vc-handle.c\nindex 96b62b49b2b53..134c14b489ef8 100644\n--- a/arch/x86/coco/sev/vc-handle.c\n+++ b/arch/x86/coco/sev/vc-handle.c\n@@ -871,7 +871,7 @@ static enum es_result vc_handle_exitcode(struct es_em_ctxt ctxt,\n \n static __always_inline bool is_vc2_stack(unsigned long sp)\n {\n-\treturn (sp \u003e= __this_cpu_ist_bottom_va(VC2) \u0026\u0026 sp \u003c __this_cpu_ist_top_va(VC2));\n+\treturn (sp \u003e= __this_cpu_ist_bottom_va(ESTACK_VC2) \u0026\u0026 sp \u003c __this_cpu_ist_top_va(ESTACK_VC2));\n }\n \n static __always_inline bool vc_from_invalid_context(struct pt_regs regs)\ndiff --git a/arch/x86/include/asm/cpu_entry_area.h b/arch/x86/include/asm/cpu_entry_area.h\nindex 462fc34f13176..509e52fc3a0f6 100644\n--- a/arch/x86/include/asm/cpu_entry_area.h\n+++ b/arch/x86/include/asm/cpu_entry_area.h\n@@ -16,32 +16,6 @@\n #define VC_EXCEPTION_STKSZ\t0\n #endif\n \n-/ Macro to enforce the same ordering and stack sizes /\n-#define ESTACKS_MEMBERS(guardsize, optional_stack_size)\t\t\\\n-\tchar\tDF_stack_guard[guardsize];\t\t\t\\\n-\tchar\tDF_stack[EXCEPTION_STKSZ];\t\t\t\\\n-\tchar\tNMI_stack_guard[guardsize];\t\t\t\\\n-\tchar\tNMI_stack[EXCEPTION_STKSZ];\t\t\t\\\n-\tchar\tDB_stack_guard[guardsize];\t\t\t\\\n-\tchar\tDB_stack[EXCEPTION_STKSZ];\t\t\t\\\n-\tchar\tMCE_stack_guard[guardsize];\t\t\t\\\n-\tchar\tMCE_stack[EXCEPTION_STKSZ];\t\t\t\\\n-\tchar\tVC_stack_guard[guardsize];\t\t\t\\\n-\tchar\tVC_stack[optional_stack_size];\t\t\t\\\n-\tchar\tVC2_stack_guard[guardsize];\t\t\t\\\n-\tchar\tVC2_stack[optional_stack_size];\t\t\t\\\n-\tchar\tIST_top_guard[guardsize];\t\t\t\\\n-\n-/ The exception stacks' physical storage. No guard pages required /\n-struct exception_stacks {\n-\tESTACKS_MEMBERS(0, VC_EXCEPTION_STKSZ)\n-};\n-\n-/ The effective cpu entry area mapping with guard pages. /\n-struct cea_exception_stacks {\n-\tESTACKS_MEMBERS(PAGE_SIZE, EXCEPTION_STKSZ)\n-};\n-\n /\n * The exception stack ordering in [cea_]exception_stacks\n /\n@@ -55,21 +29,45 @@ enum exception_stack_ordering {\n \tN_EXCEPTION_STACKS\n };\n \n-#define CEA_ESTACK_SIZE(st)\t\t\t\t\t\\\n-\tsizeof(((struct cea_exception_stacks )0)-\u003est## _stack)\n+/* Macro to enforce the same ordering and stack sizes /\n+#define ESTACKS_MEMBERS(guardsize, optional_stack_size)\t\t\\\n+\tchar\tESTACK_DF_stack_guard[guardsize];\t\t\\\n+\tchar\tESTACK_DF_stack[EXCEPTION_STKSZ];\t\t\\\n+\tchar\tESTACK_NMI_stack_guard[guardsize];\t\t\\\n+\tchar\tESTACK_NMI_stack[EXCEPTION_STKSZ];\t\t\\\n+\tchar\tESTACK_DB_stack_guard[guardsize];\t\t\\\n+\tchar\tESTACK_DB_stack[EXCEPTION_STKSZ];\t\t\\\n+\tchar\tESTACK_MCE_stack_guard[guardsize];\t\t\\\n+\tchar\tESTACK_MCE_stack[EXCEPTION_STKSZ];\t\t\\\n+\tchar\tESTACK_VC_stack_guard[guardsize];\t\t\\\n+\tchar\tESTACK_VC_stack[optional_stack_size];\t\t\\\n+\tchar\tESTACK_VC2_stack_guard[guardsize];\t\t\\\n+\tchar\tESTACK_VC2_stack[optional_stack_size];\t\t\\\n+\tchar\tESTACK_IST_top_guard[guardsize];\t\t\\\n \n-#define CEA_ESTACK_BOT(ceastp, st)\t\t\t\t\\\n-\t((unsigned long)\u0026(ceastp)-\u003est## _stack)\n+/ The exception stacks' physical storage. No guard pages required /\n+struct exception_stacks {\n+\tESTACKS_MEMBERS(0, VC_EXCEPTION_STKSZ)\n+};\n \n-#define CEA_ESTACK_TOP(ceastp, st)\t\t\t\t\\\n-\t(CEA_ESTACK_BOT(ceastp, st) + CEA_ESTACK_SIZE(st))\n+/ The effective cpu entry area mapping with guard pages. /\n+struct cea_exception_stacks {\n+\tstruct {\n+\t\tchar stack_guard[PAGE_SIZE];\n+\t\tchar stack[EXCEPTION_STKSZ];\n+\t} event_stacks[N_EXCEPTION_STACKS];\n+\tchar IST_top_guard[PAGE_SIZE];\n+};\n \n #define CEA_ESTACK_OFFS(st)\t\t\t\t\t\\\n-\toffsetof(struct cea_exception_stacks, st## _stack)\n+\toffsetof(struct cea_exception_stacks, event_stacks[st].stack)\n \n #define CEA_ESTACK_PAGES\t\t\t\t\t\\\n \t(sizeof(struct cea_exception_stacks) / PAGE_SIZE)\n \n+extern unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack);\n+extern unsigned long __this_cpu_ist_bottom_va(enum exception_stack_ordering stack);\n+\n #endif\n \n #ifdef CONFIG_X86_32\n@@ -144,10 +142,4 @@ static __always_inline struct entry_stack cpu_entry_stack(int cpu)\n \treturn \u0026get_cpu_entry_area(cpu)-\u003eentry_stack_page.stack;\n }\n \n-#define __this_cpu_ist_top_va(name)\t\t\t\t\t\\\n-\tCEA_ESTACK_TOP(__this_cpu_read(cea_exception_stacks), name)\n-\n-#define __this_cpu_ist_bottom_va(name)\t\t\t\t\t\\\n-\tCEA_ESTACK_BOT(__this_cpu_read(cea_exception_stacks), name)\n-\n #endif\ndiff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h\nindex 18a2f811c3580..85b851f16baea 100644\n--- a/arch/x86/include/asm/fred.h\n+++ b/arch/x86/include/asm/fred.h\n@@ -35,6 +35,13 @@\n \n #ifndef __ASSEMBLER__\n \n+enum fred_stack_level {\n+\tFRED_STACK_LEVEL_0,\n+\tFRED_STACK_LEVEL_1,\n+\tFRED_STACK_LEVEL_2,\n+\tFRED_STACK_LEVEL_3\n+};\n+\n #ifdef CONFIG_X86_FRED\n #include \u003clinux/kernel.h\u003e\n #include \u003clinux/sched/task_stack.h\u003e\n@@ -105,6 +112,8 @@ static __always_inline void fred_update_rsp0(void)\n \t\t__this_cpu_write(fred_rsp0, rsp0);\n \t}\n }\n+\n+unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl);\n #else /* CONFIG_X86_FRED /\n static __always_inline unsigned long fred_event_data(struct pt_regs regs) { return 0; }\n static inline void cpu_init_fred_exceptions(void) { }\n@@ -112,6 +121,7 @@ static inline void cpu_init_fred_rsps(void) { }\n static inline void fred_complete_exception_setup(void) { }\n static inline void fred_sync_rsp0(unsigned long rsp0) { }\n static inline void fred_update_rsp0(void) { }\n+static inline unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl) { return 0; }\n #endif /* CONFIG_X86_FRED /\n #endif / !__ASSEMBLER__ /\n \ndiff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h\nindex 683bb8bf43a94..21d12d86a18b0 100644\n--- a/arch/x86/include/asm/kvm_host.h\n+++ b/arch/x86/include/asm/kvm_host.h\n@@ -294,6 +294,9 @@ struct kvm_host_values {\n \tu64 xss;\n \tu64 s_cet;\n \tu64 arch_capabilities;\n+\n+\tu64 fred_config;\n+\tu64 fred_stklvls;\n };\n extern struct kvm_host_values kvm_host;\n \n@@ -692,6 +695,8 @@ struct kvm_queued_exception {\n \tu32 error_code;\n \tunsigned long payload;\n \tbool has_payload;\n+\tbool is_nested;\n+\tu64 event_data;\n };\n \n /\n@@ -1029,6 +1034,11 @@ struct kvm_vcpu_arch {\n #if IS_ENABLED(CONFIG_HYPERV)\n \thpa_t hv_root_tdp;\n #endif\n+\t/\n+\t Stores the FRED SSP0 MSR when CET is not supported, prompting KVM\n+\t * to intercept its accesses.\n+\t /\n+\tu64 fred_ssp0_fallback;\n };\n \n struct kvm_lpage_info {\n@@ -1278,6 +1288,7 @@ struct kvm_arch {\n \tbool has_mapped_host_mmio;\n \tbool guest_can_read_msr_platform_info;\n \tbool exception_payload_enabled;\n+\tbool exception_fred_state_enabled;\n \n \tbool triple_fault_event;\n \ndiff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h\nindex 18c4be75e9271..b386709515c47 100644\n--- a/arch/x86/include/asm/msr-index.h\n+++ b/arch/x86/include/asm/msr-index.h\n@@ -64,6 +64,8 @@\n #define MSR_IA32_FRED_SSP3\t0x1d3\t\t\t/ Level 3 shadow stack pointer /\n #define MSR_IA32_FRED_CONFIG\t0x1d4\t\t\t/ Entrypoint and interrupt stack level /\n \n+#define FRED_CONFIG_RESERVED\t(BIT_ULL(2) \| GENMASK_ULL(5, 4) \| BIT_ULL(11))\n+\n / Intel MSRs. Some also available on other CPUs /\n #define MSR_TEST_CTRL\t\t\t\t0x00000033\n #define MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT\t29\n@@ -1274,6 +1276,7 @@\n #define MSR_IA32_VMX_TRUE_ENTRY_CTLS 0x00000490\n #define MSR_IA32_VMX_VMFUNC 0x00000491\n #define MSR_IA32_VMX_PROCBASED_CTLS3\t0x00000492\n+#define MSR_IA32_VMX_EXIT_CTLS2\t\t0x00000493\n \n #define MSR_IA32_MCU_STAGING_MBOX_ADDR\t0x000007a5\n \ndiff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h\nindex 3f1b3096ff040..6633988a84da3 100644\n--- a/arch/x86/include/asm/vmx.h\n+++ b/arch/x86/include/asm/vmx.h\n@@ -119,6 +119,10 @@ struct vmcs {\n #define VM_EXIT_CLEAR_IA32_RTIT_CTL\t\t0x02000000\n #define VM_EXIT_LOAD_CET_STATE 0x10000000\n #define VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL\t0x40000000\n+#define VM_EXIT_ACTIVATE_SECONDARY_CONTROLS\t0x80000000\n+\n+#define SECONDARY_VM_EXIT_SAVE_IA32_FRED\tBIT_ULL(0)\n+#define SECONDARY_VM_EXIT_LOAD_IA32_FRED\tBIT_ULL(1)\n \n #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR\t0x00036dff\n \n@@ -133,6 +137,7 @@ struct vmcs {\n #define VM_ENTRY_PT_CONCEAL_PIP\t\t\t0x00020000\n #define VM_ENTRY_LOAD_IA32_RTIT_CTL\t\t0x00040000\n #define VM_ENTRY_LOAD_CET_STATE 0x00100000\n+#define VM_ENTRY_LOAD_IA32_FRED\t\t\t0x00800000\n \n #define VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR\t0x000011ff\n \n@@ -147,6 +152,7 @@ struct vmcs {\n #define VMX_BASIC_INOUT\t\t\t\tBIT_ULL(54)\n #define VMX_BASIC_TRUE_CTLS\t\t\tBIT_ULL(55)\n #define VMX_BASIC_NO_HW_ERROR_CODE_CC\t\tBIT_ULL(56)\n+#define VMX_BASIC_NESTED_EXCEPTION\t\tBIT_ULL(58)\n #define VMX_BASIC_NO_SEAMRET_INVD_VMCS\t\tBIT_ULL(60)\n \n static inline u32 vmx_basic_vmcs_revision_id(u64 vmx_basic)\n@@ -275,8 +281,14 @@ enum vmcs_field {\n \tSHARED_EPT_POINTER\t\t= 0x0000203C,\n \tPID_POINTER_TABLE\t\t= 0x00002042,\n \tPID_POINTER_TABLE_HIGH\t\t= 0x00002043,\n+\tSECONDARY_VM_EXIT_CONTROLS\t= 0x00002044,\n+\tSECONDARY_VM_EXIT_CONTROLS_HIGH\t= 0x00002045,\n+\tINJECTED_EVENT_DATA\t\t= 0x00002052,\n+\tINJECTED_EVENT_DATA_HIGH\t= 0x00002053,\n \tGUEST_PHYSICAL_ADDRESS = 0x00002400,\n \tGUEST_PHYSICAL_ADDRESS_HIGH = 0x00002401,\n+\tORIGINAL_EVENT_DATA\t\t= 0x00002404,\n+\tORIGINAL_EVENT_DATA_HIGH\t= 0x00002405,\n \tVMCS_LINK_POINTER = 0x00002800,\n \tVMCS_LINK_POINTER_HIGH = 0x00002801,\n \tGUEST_IA32_DEBUGCTL = 0x00002802,\n@@ -299,12 +311,44 @@ enum vmcs_field {\n \tGUEST_BNDCFGS_HIGH = 0x00002813,\n \tGUEST_IA32_RTIT_CTL\t\t= 0x00002814,\n \tGUEST_IA32_RTIT_CTL_HIGH\t= 0x00002815,\n+\tGUEST_IA32_FRED_CONFIG\t\t= 0x0000281a,\n+\tGUEST_IA32_FRED_CONFIG_HIGH\t= 0x0000281b,\n+\tGUEST_IA32_FRED_RSP1\t\t= 0x0000281c,\n+\tGUEST_IA32_FRED_RSP1_HIGH\t= 0x0000281d,\n+\tGUEST_IA32_FRED_RSP2\t\t= 0x0000281e,\n+\tGUEST_IA32_FRED_RSP2_HIGH\t= 0x0000281f,\n+\tGUEST_IA32_FRED_RSP3\t\t= 0x00002820,\n+\tGUEST_IA32_FRED_RSP3_HIGH\t= 0x00002821,\n+\tGUEST_IA32_FRED_STKLVLS\t\t= 0x00002822,\n+\tGUEST_IA32_FRED_STKLVLS_HIGH\t= 0x00002823,\n+\tGUEST_IA32_FRED_SSP1\t\t= 0x00002824,\n+\tGUEST_IA32_FRED_SSP1_HIGH\t= 0x00002825,\n+\tGUEST_IA32_FRED_SSP2\t\t= 0x00002826,\n+\tGUEST_IA32_FRED_SSP2_HIGH\t= 0x00002827,\n+\tGUEST_IA32_FRED_SSP3\t\t= 0x00002828,\n+\tGUEST_IA32_FRED_SSP3_HIGH\t= 0x00002829,\n \tHOST_IA32_PAT\t\t\t= 0x00002c00,\n \tHOST_IA32_PAT_HIGH\t\t= 0x00002c01,\n \tHOST_IA32_EFER\t\t\t= 0x00002c02,\n \tHOST_IA32_EFER_HIGH\t\t= 0x00002c03,\n \tHOST_IA32_PERF_GLOBAL_CTRL\t= 0x00002c04,\n \tHOST_IA32_PERF_GLOBAL_CTRL_HIGH\t= 0x00002c05,\n+\tHOST_IA32_FRED_CONFIG\t\t= 0x00002c08,\n+\tHOST_IA32_FRED_CONFIG_HIGH\t= 0x00002c09,\n+\tHOST_IA32_FRED_RSP1\t\t= 0x00002c0a,\n+\tHOST_IA32_FRED_RSP1_HIGH\t= 0x00002c0b,\n+\tHOST_IA32_FRED_RSP2\t\t= 0x00002c0c,\n+\tHOST_IA32_FRED_RSP2_HIGH\t= 0x00002c0d,\n+\tHOST_IA32_FRED_RSP3\t\t= 0x00002c0e,\n+\tHOST_IA32_FRED_RSP3_HIGH\t= 0x00002c0f,\n+\tHOST_IA32_FRED_STKLVLS\t\t= 0x00002c10,\n+\tHOST_IA32_FRED_STKLVLS_HIGH\t= 0x00002c11,\n+\tHOST_IA32_FRED_SSP1\t\t= 0x00002c12,\n+\tHOST_IA32_FRED_SSP1_HIGH\t= 0x00002c13,\n+\tHOST_IA32_FRED_SSP2\t\t= 0x00002c14,\n+\tHOST_IA32_FRED_SSP2_HIGH\t= 0x00002c15,\n+\tHOST_IA32_FRED_SSP3\t\t= 0x00002c16,\n+\tHOST_IA32_FRED_SSP3_HIGH\t= 0x00002c17,\n \tPIN_BASED_VM_EXEC_CONTROL = 0x00004000,\n \tCPU_BASED_VM_EXEC_CONTROL = 0x00004002,\n \tEXCEPTION_BITMAP = 0x00004004,\n@@ -412,13 +456,15 @@ enum vmcs_field {\n #define INTR_INFO_INTR_TYPE_MASK 0x700 / 10:8 /\n #define INTR_INFO_DELIVER_CODE_MASK 0x800 / 11 /\n #define INTR_INFO_UNBLOCK_NMI\t\t0x1000\t\t/ 12 /\n+#define INTR_INFO_NESTED_EXCEPTION_MASK\t0x2000\t\t/ 13 /\n #define INTR_INFO_VALID_MASK 0x80000000 / 31 /\n-#define INTR_INFO_RESVD_BITS_MASK 0x7ffff000\n+#define INTR_INFO_RESVD_BITS_MASK 0x7fffd000\n \n #define VECTORING_INFO_VECTOR_MASK \tINTR_INFO_VECTOR_MASK\n #define VECTORING_INFO_TYPE_MASK \tINTR_INFO_INTR_TYPE_MASK\n #define VECTORING_INFO_DELIVER_CODE_MASK \tINTR_INFO_DELIVER_CODE_MASK\n #define VECTORING_INFO_VALID_MASK \tINTR_INFO_VALID_MASK\n+#define VECTORING_INFO_NESTED_EXCEPTION_MASK\tINTR_INFO_NESTED_EXCEPTION_MASK\n \n #define INTR_TYPE_EXT_INTR\t\t(EVENT_TYPE_EXTINT \u003c\u003c 8)\t/ external interrupt /\n #define INTR_TYPE_RESERVED\t\t(EVENT_TYPE_RESERVED \u003c\u003c 8)\t/ reserved /\ndiff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h\nindex 1585ec8040666..c8a6149fcf632 100644\n--- a/arch/x86/include/uapi/asm/kvm.h\n+++ b/arch/x86/include/uapi/asm/kvm.h\n@@ -331,6 +331,7 @@ struct kvm_reinject_control {\n #define KVM_VCPUEVENT_VALID_SMM\t\t0x00000008\n #define KVM_VCPUEVENT_VALID_PAYLOAD\t0x00000010\n #define KVM_VCPUEVENT_VALID_TRIPLE_FAULT\t0x00000020\n+#define KVM_VCPUEVENT_VALID_FRED_STATE\t0x00000040\n \n / Interrupt shadow states /\n #define KVM_X86_SHADOW_INT_MOV_SS\t0x01\n@@ -368,7 +369,11 @@ struct kvm_vcpu_events {\n \tstruct {\n \t\t__u8 pending;\n \t} triple_fault;\n-\t__u8 reserved[26];\n+\t__u8 reserved0[11];\n+\t/ Aligned to a 64-bit boundary /\n+\t__u64 exception_event_data;\n+\t__u8 reserved1[6];\n+\t__u8 exception_is_nested;\n \t__u8 exception_has_payload;\n \t__u64 exception_payload;\n };\ndiff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c\nindex a3df21d264606..99ec284fb8293 100644\n--- a/arch/x86/kernel/cpu/common.c\n+++ b/arch/x86/kernel/cpu/common.c\n@@ -2372,12 +2372,12 @@ static inline void setup_getcpu(int cpu)\n static inline void tss_setup_ist(struct tss_struct tss)\n {\n \t/* Set up the per-CPU TSS IST stacks /\n-\ttss-\u003ex86_tss.ist[IST_INDEX_DF] = __this_cpu_ist_top_va(DF);\n-\ttss-\u003ex86_tss.ist[IST_INDEX_NMI] = __this_cpu_ist_top_va(NMI);\n-\ttss-\u003ex86_tss.ist[IST_INDEX_DB] = __this_cpu_ist_top_va(DB);\n-\ttss-\u003ex86_tss.ist[IST_INDEX_MCE] = __this_cpu_ist_top_va(MCE);\n+\ttss-\u003ex86_tss.ist[IST_INDEX_DF]\t= __this_cpu_ist_top_va(ESTACK_DF);\n+\ttss-\u003ex86_tss.ist[IST_INDEX_NMI]\t= __this_cpu_ist_top_va(ESTACK_NMI);\n+\ttss-\u003ex86_tss.ist[IST_INDEX_DB]\t= __this_cpu_ist_top_va(ESTACK_DB);\n+\ttss-\u003ex86_tss.ist[IST_INDEX_MCE]\t= __this_cpu_ist_top_va(ESTACK_MCE);\n \t/ Only mapped when SEV-ES is active /\n-\ttss-\u003ex86_tss.ist[IST_INDEX_VC] = __this_cpu_ist_top_va(VC);\n+\ttss-\u003ex86_tss.ist[IST_INDEX_VC]\t= __this_cpu_ist_top_va(ESTACK_VC);\n }\n #else / CONFIG_X86_64 /\n static inline void tss_setup_ist(struct tss_struct tss) { }\ndiff --git a/arch/x86/kernel/dumpstack_64.c b/arch/x86/kernel/dumpstack_64.c\nindex 6c5defd6569a3..93b10b264e53b 100644\n--- a/arch/x86/kernel/dumpstack_64.c\n+++ b/arch/x86/kernel/dumpstack_64.c\n@@ -70,10 +70,10 @@ struct estack_pages {\n \n #define EPAGERANGE(st)\t\t\t\t\t\t\t\\\n \t[PFN_DOWN(CEA_ESTACK_OFFS(st)) ...\t\t\t\t\\\n-\t PFN_DOWN(CEA_ESTACK_OFFS(st) + CEA_ESTACK_SIZE(st) - 1)] = {\t\\\n+\t PFN_DOWN(CEA_ESTACK_OFFS(st) + EXCEPTION_STKSZ - 1)] = {\t\\\n \t\t.offs\t= CEA_ESTACK_OFFS(st),\t\t\t\t\\\n-\t\t.size\t= CEA_ESTACK_SIZE(st),\t\t\t\t\\\n-\t\t.type\t= STACK_TYPE_EXCEPTION + ESTACK_ ##st, }\n+\t\t.size\t= EXCEPTION_STKSZ,\t\t\t\t\\\n+\t\t.type\t= STACK_TYPE_EXCEPTION + st, }\n \n /\n Array of exception stack page descriptors. If the stack is larger than\n@@ -83,12 +83,12 @@ struct estack_pages {\n /\n static const\n struct estack_pages estack_pages[CEA_ESTACK_PAGES] ____cacheline_aligned = {\n-\tEPAGERANGE(DF),\n-\tEPAGERANGE(NMI),\n-\tEPAGERANGE(DB),\n-\tEPAGERANGE(MCE),\n-\tEPAGERANGE(VC),\n-\tEPAGERANGE(VC2),\n+\tEPAGERANGE(ESTACK_DF),\n+\tEPAGERANGE(ESTACK_NMI),\n+\tEPAGERANGE(ESTACK_DB),\n+\tEPAGERANGE(ESTACK_MCE),\n+\tEPAGERANGE(ESTACK_VC),\n+\tEPAGERANGE(ESTACK_VC2),\n };\n \n static __always_inline bool in_exception_stack(unsigned long stack, struct stack_info info)\ndiff --git a/arch/x86/kernel/fred.c b/arch/x86/kernel/fred.c\nindex 117aa06d25ca9..68947c53a4941 100644\n--- a/arch/x86/kernel/fred.c\n+++ b/arch/x86/kernel/fred.c\n@@ -1,5 +1,6 @@\n / SPDX-License-Identifier: GPL-2.0 /\n #include \u003clinux/kernel.h\u003e\n+#include \u003clinux/kvm_types.h\u003e\n \n #include \u003casm/desc.h\u003e\n #include \u003casm/fred.h\u003e\n@@ -69,6 +70,23 @@ void cpu_init_fred_exceptions(void)\n \tsetup_clear_cpu_cap(X86_FEATURE_SYSCALL32);\n }\n \n+unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl)\n+{\n+\tswitch (lvl) {\n+\tcase FRED_STACK_LEVEL_0:\n+\t\treturn __this_cpu_read(fred_rsp0);\n+\tcase FRED_STACK_LEVEL_1:\n+\t\treturn __this_cpu_ist_top_va(ESTACK_DB);\n+\tcase FRED_STACK_LEVEL_2:\n+\t\treturn __this_cpu_ist_top_va(ESTACK_NMI);\n+\tcase FRED_STACK_LEVEL_3:\n+\t\treturn __this_cpu_ist_top_va(ESTACK_DF);\n+\tdefault:\n+\t\tBUG();\n+\t}\n+}\n+EXPORT_SYMBOL_FOR_KVM(this_cpu_fred_rsp);\n+\n / Must be called after setup_cpu_entry_areas() /\n void cpu_init_fred_rsps(void)\n {\n@@ -84,7 +102,7 @@ void cpu_init_fred_rsps(void)\n \t FRED_STKLVL(X86_TRAP_DF, FRED_DF_STACK_LEVEL));\n \n \t/ The FRED equivalents to IST stacks... /\n-\twrmsrq(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB));\n-\twrmsrq(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI));\n-\twrmsrq(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF));\n+\twrmsrq(MSR_IA32_FRED_RSP1, this_cpu_fred_rsp(FRED_STACK_LEVEL_1));\n+\twrmsrq(MSR_IA32_FRED_RSP2, this_cpu_fred_rsp(FRED_STACK_LEVEL_2));\n+\twrmsrq(MSR_IA32_FRED_RSP3, this_cpu_fred_rsp(FRED_STACK_LEVEL_3));\n }\ndiff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c\nindex 30aa8369957e9..8afd607a58730 100644\n--- a/arch/x86/kernel/traps.c\n+++ b/arch/x86/kernel/traps.c\n@@ -1075,7 +1075,7 @@ asmlinkage __visible noinstr struct pt_regs vc_switch_off_ist(struct pt_regs r\n \n \tif (!get_stack_info_noinstr(stack, current, \u0026info) \|\| info.type == STACK_TYPE_ENTRY \|\|\n \t info.type \u003e STACK_TYPE_EXCEPTION_LAST)\n-\t\tsp = __this_cpu_ist_top_va(VC2);\n+\t\tsp = __this_cpu_ist_top_va(ESTACK_VC2);\n \n sync:\n \t/\ndiff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c\nindex ddb022cb203a2..5682bacdfd32f 100644\n--- a/arch/x86/kvm/cpuid.c\n+++ b/arch/x86/kvm/cpuid.c\n@@ -1034,6 +1034,7 @@ void kvm_initialize_cpu_caps(void)\n \t\tF(FSRS),\n \t\tF(FSRC),\n \t\tF(WRMSRNS),\n+\t\tX86_64_F(FRED),\n \t\tX86_64_F(LKGS),\n \t\tF(AMX_FP16),\n \t\tF(AVX_IFMA),\ndiff --git a/arch/x86/kvm/msrs.c b/arch/x86/kvm/msrs.c\nindex 66fa7140d65d9..3260bddb5f526 100644\n--- a/arch/x86/kvm/msrs.c\n+++ b/arch/x86/kvm/msrs.c\n@@ -193,6 +193,9 @@ static const u32 msrs_to_save_base[] = {\n \tMSR_STAR,\n #ifdef CONFIG_X86_64\n \tMSR_CSTAR, MSR_KERNEL_GS_BASE, MSR_SYSCALL_MASK, MSR_LSTAR,\n+\tMSR_IA32_FRED_RSP0, MSR_IA32_FRED_RSP1, MSR_IA32_FRED_RSP2,\n+\tMSR_IA32_FRED_RSP3, MSR_IA32_FRED_STKLVLS, MSR_IA32_FRED_SSP1,\n+\tMSR_IA32_FRED_SSP2, MSR_IA32_FRED_SSP3, MSR_IA32_FRED_CONFIG,\n #endif\n \tMSR_IA32_TSC, MSR_IA32_CR_PAT, MSR_VM_HSAVE_PA,\n \tMSR_IA32_FEAT_CTL, MSR_IA32_BNDCFGS, MSR_TSC_AUX,\n@@ -314,6 +317,7 @@ static const u32 emulated_msrs_all[] = {\n \tMSR_IA32_VMX_PROCBASED_CTLS2,\n \tMSR_IA32_VMX_EPT_VPID_CAP,\n \tMSR_IA32_VMX_VMFUNC,\n+\tMSR_IA32_VMX_EXIT_CTLS2,\n \n \tMSR_K7_HWCR,\n \tMSR_KVM_POLL_CONTROL,\n@@ -772,7 +776,7 @@ static int __kvm_set_msr(struct kvm_vcpu vcpu, u32 index, u64 data,\n \t\t architecture. Intercepting XRSTORS/XSAVES for this\n \t\t * special case isn't deemed worthwhile.\n \t\t /\n-\tcase MSR_IA32_PL0_SSP ... MSR_IA32_INT_SSP_TAB:\n+\tcase MSR_IA32_PL1_SSP ... MSR_IA32_INT_SSP_TAB:\n \t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK))\n \t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n \t\t/\n@@ -787,6 +791,52 @@ static int __kvm_set_msr(struct kvm_vcpu vcpu, u32 index, u64 data,\n \t\tif (index != MSR_IA32_INT_SSP_TAB \u0026\u0026 !IS_ALIGNED(data, 4))\n \t\t\treturn 1;\n \t\tbreak;\n+\tcase MSR_IA32_FRED_STKLVLS:\n+\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))\n+\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n+\t\tbreak;\n+\tcase MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_RSP3:\n+\tcase MSR_IA32_FRED_SSP1 ... MSR_IA32_FRED_CONFIG: {\n+\t\tu64 reserved_bits = 0;\n+\n+\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))\n+\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n+\n+\t\tif (is_noncanonical_msr_address(data, vcpu))\n+\t\t\treturn 1;\n+\n+\t\tswitch (index) {\n+\t\tcase MSR_IA32_FRED_CONFIG:\n+\t\t\treserved_bits = FRED_CONFIG_RESERVED;\n+\t\t\tbreak;\n+\t\tcase MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_RSP3:\n+\t\t\treserved_bits = GENMASK_ULL(5, 0);\n+\t\t\tbreak;\n+\t\tcase MSR_IA32_FRED_SSP1 ... MSR_IA32_FRED_SSP3:\n+\t\t\treserved_bits = GENMASK_ULL(2, 0);\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tWARN_ON_ONCE(1);\n+\t\t\treturn 1;\n+\t\t}\n+\n+\t\tif (data \u0026 reserved_bits)\n+\t\t\treturn 1;\n+\n+\t\tbreak;\n+\t}\n+\tcase MSR_IA32_PL0_SSP: / I.e., MSR_IA32_FRED_SSP0 /\n+\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) \u0026\u0026\n+\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))\n+\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n+\n+\t\tif (is_noncanonical_msr_address(data, vcpu))\n+\t\t\treturn 1;\n+\n+\t\tif (!IS_ALIGNED(data, 4))\n+\t\t\treturn 1;\n+\n+\t\tbreak;\n \t}\n \n \tmsr.data = data;\n@@ -841,10 +891,19 @@ static int __kvm_get_msr(struct kvm_vcpu vcpu, u32 index, u64 data,\n \t\tif (!host_initiated)\n \t\t\treturn 1;\n \t\tfallthrough;\n-\tcase MSR_IA32_PL0_SSP ... MSR_IA32_INT_SSP_TAB:\n+\tcase MSR_IA32_PL1_SSP ... MSR_IA32_INT_SSP_TAB:\n \t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK))\n \t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n \t\tbreak;\n+\tcase MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_CONFIG:\n+\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))\n+\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n+\t\tbreak;\n+\tcase MSR_IA32_PL0_SSP: / I.e., MSR_IA32_FRED_SSP0 /\n+\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) \u0026\u0026\n+\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))\n+\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n+\t\tbreak;\n \t}\n \n \tmsr.index = index;\n@@ -1833,7 +1892,15 @@ int kvm_set_msr_common(struct kvm_vcpu vcpu, struct msr_data msr_info)\n \t\tbreak;\n #endif\n \tcase MSR_IA32_U_CET:\n+\t\tkvm_set_xstate_msr(vcpu, msr_info);\n+\t\tbreak;\n \tcase MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:\n+\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) {\n+\t\t\tWARN_ON_ONCE(msr != MSR_IA32_FRED_SSP0);\n+\t\t\tvcpu-\u003earch.fred_ssp0_fallback = data;\n+\t\t\tbreak;\n+\t\t}\n+\n \t\tkvm_set_xstate_msr(vcpu, msr_info);\n \t\tbreak;\n \tdefault:\n@@ -2186,7 +2253,15 @@ int kvm_get_msr_common(struct kvm_vcpu vcpu, struct msr_data msr_info)\n \t\tbreak;\n #endif\n \tcase MSR_IA32_U_CET:\n+\t\tkvm_get_xstate_msr(vcpu, msr_info);\n+\t\tbreak;\n \tcase MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:\n+\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) {\n+\t\t\tWARN_ON_ONCE(msr_info-\u003eindex != MSR_IA32_FRED_SSP0);\n+\t\t\tmsr_info-\u003edata = vcpu-\u003earch.fred_ssp0_fallback;\n+\t\t\tbreak;\n+\t\t}\n+\n \t\tkvm_get_xstate_msr(vcpu, msr_info);\n \t\tbreak;\n \tdefault:\n@@ -2656,10 +2731,19 @@ static void kvm_probe_msr_to_save(u32 msr_index)\n \t\tif (!kvm_cpu_cap_has(X86_FEATURE_LM))\n \t\t\treturn;\n \t\tfallthrough;\n-\tcase MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:\n+\tcase MSR_IA32_PL1_SSP ... MSR_IA32_PL3_SSP:\n \t\tif (!kvm_cpu_cap_has(X86_FEATURE_SHSTK))\n \t\t\treturn;\n \t\tbreak;\n+\tcase MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_CONFIG:\n+\t\tif (!kvm_cpu_cap_has(X86_FEATURE_FRED))\n+\t\t\treturn;\n+\t\tbreak;\n+\tcase MSR_IA32_PL0_SSP: / I.e., MSR_IA32_FRED_SSP0 /\n+\t\tif (!kvm_cpu_cap_has(X86_FEATURE_SHSTK) \u0026\u0026\n+\t\t !kvm_cpu_cap_has(X86_FEATURE_FRED))\n+\t\t\treturn;\n+\t\tbreak;\n \tdefault:\n \t\tbreak;\n \t}\ndiff --git a/arch/x86/kvm/msrs.h b/arch/x86/kvm/msrs.h\nindex 7cc182a15b3b3..845d2cbe80e5e 100644\n--- a/arch/x86/kvm/msrs.h\n+++ b/arch/x86/kvm/msrs.h\n@@ -31,7 +31,7 @@ static inline void kvm_pr_unimpl_rdmsr(struct kvm_vcpu vcpu, u32 msr)\n * associated feature that KVM supports for nested virtualization.\n /\n #define KVM_FIRST_EMULATED_VMX_MSR\tMSR_IA32_VMX_BASIC\n-#define KVM_LAST_EMULATED_VMX_MSR\tMSR_IA32_VMX_VMFUNC\n+#define KVM_LAST_EMULATED_VMX_MSR\tMSR_IA32_VMX_EXIT_CTLS2\n \n /\n * KVM's internal, non-ABI indices for synthetic MSRs. The values themselves\ndiff --git a/arch/x86/kvm/regs.c b/arch/x86/kvm/regs.c\nindex 8f66438989e47..ba6c8856b4ffa 100644\n--- a/arch/x86/kvm/regs.c\n+++ b/arch/x86/kvm/regs.c\n@@ -429,6 +429,13 @@ int kvm_set_cr4(struct kvm_vcpu vcpu, unsigned long cr4)\n \t\t\treturn 1;\n \t}\n \n+\t/\n+\t * FRED can not be enabled when EFER.LMA=0. Note, MOV to CR4 outside\n+\t * 64-bit mode clears CR4[63:32] so only emulation can get here.\n+\t /\n+\tif ((cr4 \u0026 X86_CR4_FRED) \u0026\u0026 !is_long_mode(vcpu))\n+\t\treturn 1;\n+\n \tif ((cr4 \u0026 X86_CR4_CET) \u0026\u0026 !kvm_is_cr0_bit_set(vcpu, X86_CR0_WP))\n \t\treturn 1;\n \ndiff --git a/arch/x86/kvm/regs.h b/arch/x86/kvm/regs.h\nindex 447f0ec3e63e4..5db01693ad07a 100644\n--- a/arch/x86/kvm/regs.h\n+++ b/arch/x86/kvm/regs.h\n@@ -8,7 +8,7 @@\n #define KVM_POSSIBLE_CR4_GUEST_BITS\t\t\t\t \\\n \t(X86_CR4_PVI \| X86_CR4_DE \| X86_CR4_PCE \| X86_CR4_OSFXSR \\\n \t \| X86_CR4_OSXMMEXCPT \| X86_CR4_PGE \| X86_CR4_TSD \| X86_CR4_FSGSBASE \\\n-\t \| X86_CR4_CET)\n+\t \| X86_CR4_CET \| X86_CR4_FRED)\n \n #define X86_CR0_PDPTR_BITS (X86_CR0_CD \| X86_CR0_NW \| X86_CR0_PG)\n #define X86_CR4_TLBFLUSH_BITS (X86_CR4_PGE \| X86_CR4_PCIDE \| X86_CR4_PAE \| X86_CR4_SMEP)\n@@ -28,7 +28,7 @@ static_assert(!(KVM_POSSIBLE_CR0_GUEST_BITS \u0026 X86_CR0_PDPTR_BITS));\n \t\t\t \| X86_CR4_OSXSAVE \| X86_CR4_SMEP \| X86_CR4_FSGSBASE \\\n \t\t\t \| X86_CR4_OSXMMEXCPT \| X86_CR4_LA57 \| X86_CR4_VMXE \\\n \t\t\t \| X86_CR4_SMAP \| X86_CR4_PKE \| X86_CR4_UMIP \\\n-\t\t\t \| X86_CR4_LAM_SUP \| X86_CR4_CET))\n+\t\t\t \| X86_CR4_LAM_SUP \| X86_CR4_CET \| X86_CR4_FRED))\n \n #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR)\n \n@@ -364,6 +364,21 @@ static __always_inline bool kvm_is_cr4_bit_set(struct kvm_vcpu vcpu,\n \treturn !!kvm_read_cr4_bits(vcpu, cr4_bit);\n }\n \n+/\n+ It's enough to check just CR4.FRED (X86_CR4_FRED) to tell if\n+ * a vCPU is running with FRED enabled, because:\n+ * 1) CR4.FRED can be set to 1 only _after_ IA32_EFER.LMA = 1.\n+ * 2) To leave IA-32e mode, CR4.FRED must be cleared first.\n+ /\n+static inline bool is_fred_enabled(struct kvm_vcpu vcpu)\n+{\n+#ifdef CONFIG_X86_64\n+\treturn kvm_is_cr4_bit_set(vcpu, X86_CR4_FRED);\n+#else\n+\treturn false;\n+#endif\n+}\n+\n static inline ulong kvm_read_cr3(struct kvm_vcpu vcpu)\n {\n \tif (!kvm_register_is_available(vcpu, VCPU_REG_CR3))\n@@ -408,6 +423,8 @@ static inline bool __kvm_is_valid_cr4(struct kvm_vcpu vcpu, unsigned long cr4)\n \tif (!__cpu_has(__c, X86_FEATURE_SHSTK) \u0026\u0026 \\\n \t !__cpu_has(__c, X86_FEATURE_IBT)) \\\n \t\t__reserved_bits \|= X86_CR4_CET; \\\n+\tif (!__cpu_has(__c, X86_FEATURE_FRED)) \\\n+\t\t__reserved_bits \|= X86_CR4_FRED; \\\n \t__reserved_bits; \\\n })\n \ndiff --git a/arch/x86/kvm/smm.c b/arch/x86/kvm/smm.c\nindex 656a38dad7e7a..a504e93fc5f5f 100644\n--- a/arch/x86/kvm/smm.c\n+++ b/arch/x86/kvm/smm.c\n@@ -435,10 +435,10 @@ static int rsm_enter_protected_mode(struct kvm_vcpu vcpu,\n \n \t/\n \t * First enable PAE, long mode needs it before CR0.PG = 1 is set.\n-\t * Then enable protected mode.\tHowever, PCID cannot be enabled\n-\t * if EFER.LMA=0, so set it separately.\n+\t * Then enable protected mode. However, PCID and FRED cannot be\n+\t * enabled if EFER.LMA=0, so set them separately.\n \t /\n-\tbad = kvm_set_cr4(vcpu, cr4 \u0026 ~X86_CR4_PCIDE);\n+\tbad = kvm_set_cr4(vcpu, cr4 \u0026 ~(X86_CR4_PCIDE \| X86_CR4_FRED));\n \tif (bad)\n \t\treturn X86EMUL_UNHANDLEABLE;\n \n@@ -446,7 +446,7 @@ static int rsm_enter_protected_mode(struct kvm_vcpu vcpu,\n \tif (bad)\n \t\treturn X86EMUL_UNHANDLEABLE;\n \n-\tif (cr4 \u0026 X86_CR4_PCIDE) {\n+\tif (cr4 \u0026 (X86_CR4_PCIDE \| X86_CR4_FRED)) {\n \t\tbad = kvm_set_cr4(vcpu, cr4);\n \t\tif (bad)\n \t\t\treturn X86EMUL_UNHANDLEABLE;\n@@ -599,10 +599,14 @@ int emulator_leave_smm(struct x86_emulate_ctxt ctxt)\n \t\tstruct kvm_segment cs_desc;\n \t\tunsigned long cr4;\n \n-\t\t/ Zero CR4.PCIDE before CR0.PG. /\n+\t\t/\n+\t\t * Zero CR4.PCIDE and CR4.FRED before CR0.PG, as neither can\n+\t\t * be set while EFER.LMA is 0, and clearing CR0.PG clears\n+\t\t * EFER.LMA.\n+\t\t /\n \t\tcr4 = kvm_read_cr4(vcpu);\n-\t\tif (cr4 \u0026 X86_CR4_PCIDE)\n-\t\t\tkvm_set_cr4(vcpu, cr4 \u0026 ~X86_CR4_PCIDE);\n+\t\tif (cr4 \u0026 (X86_CR4_PCIDE \| X86_CR4_FRED))\n+\t\t\tkvm_set_cr4(vcpu, cr4 \u0026 ~(X86_CR4_PCIDE \| X86_CR4_FRED));\n \n \t\t/ A 32-bit code segment is required to clear EFER.LMA. /\n \t\tmemset(\u0026cs_desc, 0, sizeof(cs_desc));\ndiff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c\nindex ea647938a2a65..dd19c7b4e9047 100644\n--- a/arch/x86/kvm/svm/svm.c\n+++ b/arch/x86/kvm/svm/svm.c\n@@ -4392,7 +4392,7 @@ static void svm_complete_interrupts(struct kvm_vcpu vcpu)\n \n \t\tkvm_requeue_exception(vcpu, vector,\n \t\t\t\t exitintinfo \u0026 SVM_EXITINTINFO_VALID_ERR,\n-\t\t\t\t error_code);\n+\t\t\t\t error_code, false, 0);\n \t\tbreak;\n \t}\n \tcase SVM_EXITINTINFO_TYPE_INTR:\n@@ -5518,6 +5518,9 @@ static __init void svm_set_cpu_caps(void)\n \n \tkvm_cpu_cap_clear(X86_FEATURE_IBT);\n \n+\t/* SVM FRED virtualization not implemented yet /\n+\tkvm_cpu_cap_clear(X86_FEATURE_FRED);\n+\n \t/ CPUID 0x80000001 and 0x8000000A (SVM features) /\n \tif (nested) {\n \t\tkvm_cpu_cap_set(X86_FEATURE_SVM);\ndiff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilities.h\nindex 810119167f798..74456f4533e9d 100644\n--- a/arch/x86/kvm/vmx/capabilities.h\n+++ b/arch/x86/kvm/vmx/capabilities.h\n@@ -36,6 +36,7 @@ struct nested_vmx_msrs {\n \tu32 pinbased_ctls_high;\n \tu32 exit_ctls_low;\n \tu32 exit_ctls_high;\n+\tu64 secondary_exit_ctls;\n \tu32 entry_ctls_low;\n \tu32 entry_ctls_high;\n \tu32 misc_low;\n@@ -57,8 +58,9 @@ struct vmcs_config {\n \tu32 cpu_based_exec_ctrl;\n \tu32 cpu_based_2nd_exec_ctrl;\n \tu64 cpu_based_3rd_exec_ctrl;\n-\tu32 vmexit_ctrl;\n \tu32 vmentry_ctrl;\n+\tu32 vmexit_ctrl;\n+\tu64 vmexit_2nd_ctrl;\n \tu64 misc;\n \tstruct nested_vmx_msrs nested;\n };\n@@ -80,6 +82,11 @@ static inline bool cpu_has_vmx_basic_no_hw_errcode_cc(void)\n \treturn\tvmcs_config.basic \u0026 VMX_BASIC_NO_HW_ERROR_CODE_CC;\n }\n \n+static inline bool cpu_has_vmx_nested_exception(void)\n+{\n+\treturn vmcs_config.basic \u0026 VMX_BASIC_NESTED_EXCEPTION;\n+}\n+\n static inline bool cpu_has_virtual_nmis(void)\n {\n \treturn vmcs_config.pin_based_exec_ctrl \u0026 PIN_BASED_VIRTUAL_NMIS \u0026\u0026\n@@ -149,6 +156,12 @@ static inline bool cpu_has_tertiary_exec_ctrls(void)\n \t\tCPU_BASED_ACTIVATE_TERTIARY_CONTROLS;\n }\n \n+static inline bool cpu_has_secondary_vmexit_ctrls(void)\n+{\n+\treturn vmcs_config.vmexit_ctrl \u0026\n+\t\tVM_EXIT_ACTIVATE_SECONDARY_CONTROLS;\n+}\n+\n static inline bool cpu_has_vmx_virtualize_apic_accesses(void)\n {\n \treturn vmcs_config.cpu_based_2nd_exec_ctrl \u0026\n@@ -402,6 +415,11 @@ static inline bool vmx_pebs_supported(void)\n \t !enable_mediated_pmu;\n }\n \n+static inline bool cpu_has_vmx_fred(void)\n+{\n+\treturn vmcs_config.vmentry_ctrl \u0026 VM_ENTRY_LOAD_IA32_FRED;\n+}\n+\n static inline bool cpu_has_notify_vmexit(void)\n {\n \treturn vmcs_config.cpu_based_2nd_exec_ctrl \u0026\ndiff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c\nindex 151873407abd3..4fad9728a2ea0 100644\n--- a/arch/x86/kvm/vmx/nested.c\n+++ b/arch/x86/kvm/vmx/nested.c\n@@ -812,6 +812,15 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu vcpu,\n \tnested_vmx_merge_msr_bitmaps_rw(MSR_FS_BASE);\n \tnested_vmx_merge_msr_bitmaps_rw(MSR_GS_BASE);\n \tnested_vmx_merge_msr_bitmaps_rw(MSR_KERNEL_GS_BASE);\n+\tnested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_RSP0);\n+\tnested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_RSP1);\n+\tnested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_RSP2);\n+\tnested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_RSP3);\n+\tnested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_STKLVLS);\n+\tnested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_SSP1);\n+\tnested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_SSP2);\n+\tnested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_SSP3);\n+\tnested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_CONFIG);\n #endif\n \tnested_vmx_merge_msr_bitmaps_rw(MSR_IA32_SPEC_CTRL);\n \tnested_vmx_merge_msr_bitmaps_write(MSR_IA32_PRED_CMD);\n@@ -1356,9 +1365,11 @@ static int vmx_restore_vmx_basic(struct vcpu_vmx vmx, u64 data)\n \tconst u64 feature_bits = VMX_BASIC_DUAL_MONITOR_TREATMENT \|\n \t\t\t\t VMX_BASIC_INOUT \|\n \t\t\t\t VMX_BASIC_TRUE_CTLS \|\n-\t\t\t\t VMX_BASIC_NO_HW_ERROR_CODE_CC;\n+\t\t\t\t VMX_BASIC_NO_HW_ERROR_CODE_CC \|\n+\t\t\t\t VMX_BASIC_NESTED_EXCEPTION;\n \n-\tconst u64 reserved_bits = GENMASK_ULL(63, 57) \|\n+\tconst u64 reserved_bits = GENMASK_ULL(63, 59) \|\n+\t\t\t\t BIT_ULL(57) \|\n \t\t\t\t GENMASK_ULL(47, 45) \|\n \t\t\t\t BIT_ULL(31);\n \n@@ -1596,6 +1607,11 @@ int vmx_set_vmx_msr(struct kvm_vcpu vcpu, u32 msr_index, u64 data)\n \t\t\treturn -EINVAL;\n \t\tvmx-\u003enested.msrs.vmfunc_controls = data;\n \t\treturn 0;\n+\tcase MSR_IA32_VMX_EXIT_CTLS2:\n+\t\tif (data \u0026 ~vmcs_config.nested.secondary_exit_ctls)\n+\t\t\treturn -EINVAL;\n+\t\tvmx-\u003enested.msrs.secondary_exit_ctls = data;\n+\t\treturn 0;\n \tdefault:\n \t\t/\n \t\t The rest of the VMX capability MSRs do not support restore.\n@@ -1635,6 +1651,9 @@ int vmx_get_vmx_msr(struct nested_vmx_msrs msrs, u32 msr_index, u64 pdata)\n \t\tif (msr_index == MSR_IA32_VMX_EXIT_CTLS)\n \t\t\tpdata \|= VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR;\n \t\tbreak;\n+\tcase MSR_IA32_VMX_EXIT_CTLS2:\n+\t\tpdata = msrs-\u003esecondary_exit_ctls;\n+\t\tbreak;\n \tcase MSR_IA32_VMX_TRUE_ENTRY_CTLS:\n \tcase MSR_IA32_VMX_ENTRY_CTLS:\n \t\tpdata = vmx_control_msr(\n@@ -2576,6 +2595,9 @@ static void prepare_vmcs02_early(struct vcpu_vmx vmx, struct loaded_vmcs vmcs0\n \t\texec_control \u0026= ~VM_EXIT_LOAD_IA32_EFER;\n \tvm_exit_controls_set(vmx, exec_control);\n \n+\tif (exec_control \u0026 VM_EXIT_ACTIVATE_SECONDARY_CONTROLS)\n+\t\tsecondary_vm_exit_controls_set(vmx, __secondary_vm_exit_controls_get(vmcs01));\n+\n \t/\n \t * Interrupt/Exception Fields\n \t /\n@@ -2588,6 +2610,8 @@ static void prepare_vmcs02_early(struct vcpu_vmx vmx, struct loaded_vmcs vmcs0\n \t\t\t vmcs12-\u003evm_entry_instruction_len);\n \t\tvmcs_write32(GUEST_INTERRUPTIBILITY_INFO,\n \t\t\t vmcs12-\u003eguest_interruptibility_info);\n+\t\tif (cpu_has_vmx_fred())\n+\t\t\tvmcs_write64(INJECTED_EVENT_DATA, vmcs12-\u003einjected_event_data);\n \t\tvmx-\u003eloaded_vmcs-\u003enmi_known_unmasked =\n \t\t\t!(vmcs12-\u003eguest_interruptibility_info \u0026 GUEST_INTR_STATE_NMI);\n \t} else {\n@@ -2621,6 +2645,30 @@ static void vmcs_write_cet_state(struct kvm_vcpu vcpu, u64 s_cet,\n \t}\n }\n \n+static void vmcs_read_fred_msrs(struct vmcs_fred_msrs msrs)\n+{\n+\tmsrs-\u003efred_config = vmcs_read64(GUEST_IA32_FRED_CONFIG);\n+\tmsrs-\u003efred_rsp1 = vmcs_read64(GUEST_IA32_FRED_RSP1);\n+\tmsrs-\u003efred_rsp2 = vmcs_read64(GUEST_IA32_FRED_RSP2);\n+\tmsrs-\u003efred_rsp3 = vmcs_read64(GUEST_IA32_FRED_RSP3);\n+\tmsrs-\u003efred_stklvls = vmcs_read64(GUEST_IA32_FRED_STKLVLS);\n+\tmsrs-\u003efred_ssp1 = vmcs_read64(GUEST_IA32_FRED_SSP1);\n+\tmsrs-\u003efred_ssp2 = vmcs_read64(GUEST_IA32_FRED_SSP2);\n+\tmsrs-\u003efred_ssp3 = vmcs_read64(GUEST_IA32_FRED_SSP3);\n+}\n+\n+static void vmcs_write_fred_msrs(struct vmcs_fred_msrs msrs)\n+{\n+\tvmcs_write64(GUEST_IA32_FRED_CONFIG, msrs-\u003efred_config);\n+\tvmcs_write64(GUEST_IA32_FRED_RSP1, msrs-\u003efred_rsp1);\n+\tvmcs_write64(GUEST_IA32_FRED_RSP2, msrs-\u003efred_rsp2);\n+\tvmcs_write64(GUEST_IA32_FRED_RSP3, msrs-\u003efred_rsp3);\n+\tvmcs_write64(GUEST_IA32_FRED_STKLVLS, msrs-\u003efred_stklvls);\n+\tvmcs_write64(GUEST_IA32_FRED_SSP1, msrs-\u003efred_ssp1);\n+\tvmcs_write64(GUEST_IA32_FRED_SSP2, msrs-\u003efred_ssp2);\n+\tvmcs_write64(GUEST_IA32_FRED_SSP3, msrs-\u003efred_ssp3);\n+}\n+\n static void prepare_vmcs02_rare(struct vcpu_vmx vmx, struct vmcs12 vmcs12)\n {\n \tstruct hv_enlightened_vmcs hv_evmcs = nested_vmx_evmcs(vmx);\n@@ -2744,6 +2792,10 @@ static void prepare_vmcs02_rare(struct vcpu_vmx vmx, struct vmcs12 vmcs12)\n \t\t\t\t vmcs12-\u003eguest_ssp, vmcs12-\u003eguest_ssp_tbl);\n \n \tset_cr4_guest_host_mask(vmx);\n+\n+\tif (guest_cpu_cap_has(\u0026vmx-\u003evcpu, X86_FEATURE_FRED) \u0026\u0026\n+\t nested_cpu_load_guest_fred_state(vmcs12))\n+\t\tvmcs_write_fred_msrs(\u0026vmcs12-\u003eguest_fred_msrs);\n }\n \n /\n@@ -2810,6 +2862,10 @@ static int prepare_vmcs02(struct kvm_vcpu vcpu, struct vmcs12 vmcs12,\n \t\tvmcs_write64(GUEST_IA32_PAT, vcpu-\u003earch.pat);\n \t}\n \n+\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED) \u0026\u0026\n+\t (!vmx-\u003evcpu.arch.nested_run_pending \|\| !nested_cpu_load_guest_fred_state(vmcs12)))\n+\t\tvmcs_write_fred_msrs(\u0026vmx-\u003enested.pre_vmenter_fred_msrs);\n+\n \tvcpu-\u003earch.tsc_offset = kvm_calc_nested_tsc_offset(\n \t\t\tvcpu-\u003earch.l1_tsc_offset,\n \t\t\tvmx_get_l2_tsc_offset(vcpu),\n@@ -3040,6 +3096,11 @@ static int nested_check_vm_exit_controls(struct kvm_vcpu vcpu,\n \t CC(nested_vmx_check_exit_msr_switch_controls(vcpu, vmcs12)))\n \t\treturn -EINVAL;\n \n+\tif (nested_cpu_has_secondary_vm_exit_controls(vmcs12) \u0026\u0026\n+\t CC(vmcs12-\u003esecondary_vm_exit_controls \u0026\n+\t ~vmx-\u003enested.msrs.secondary_exit_ctls))\n+\t\treturn -EINVAL;\n+\n \treturn 0;\n }\n \n@@ -3050,6 +3111,8 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu vcpu,\n \t\t\t\t\t struct vmcs12 vmcs12)\n {\n \tstruct vcpu_vmx vmx = to_vmx(vcpu);\n+\tbool fred_enabled = (vmcs12-\u003evm_entry_controls \u0026 VM_ENTRY_IA32E_MODE) \u0026\u0026\n+\t\t\t (vmcs12-\u003eguest_cr4 \u0026 X86_CR4_FRED);\n \n \tif (CC(!vmx_control_verify(vmcs12-\u003evm_entry_controls,\n \t\t\t\t vmx-\u003enested.msrs.entry_ctls_low,\n@@ -3067,22 +3130,11 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu vcpu,\n \t\tu8 vector = intr_info \u0026 INTR_INFO_VECTOR_MASK;\n \t\tu32 intr_type = intr_info \u0026 INTR_INFO_INTR_TYPE_MASK;\n \t\tbool has_error_code = intr_info \u0026 INTR_INFO_DELIVER_CODE_MASK;\n+\t\tbool has_nested_exception = intr_info \u0026 INTR_INFO_NESTED_EXCEPTION_MASK;\n \t\tbool urg = nested_cpu_has2(vmcs12,\n \t\t\t\t\t SECONDARY_EXEC_UNRESTRICTED_GUEST);\n \t\tbool prot_mode = !urg \|\| vmcs12-\u003eguest_cr0 \u0026 X86_CR0_PE;\n \n-\t\t/ VM-entry interruption-info field: interruption type /\n-\t\tif (CC(intr_type == INTR_TYPE_RESERVED) \|\|\n-\t\t CC(intr_type == INTR_TYPE_OTHER_EVENT \u0026\u0026\n-\t\t !nested_cpu_supports_monitor_trap_flag(vcpu)))\n-\t\t\treturn -EINVAL;\n-\n-\t\t/ VM-entry interruption-info field: vector /\n-\t\tif (CC(intr_type == INTR_TYPE_NMI_INTR \u0026\u0026 vector != NMI_VECTOR) \|\|\n-\t\t CC(intr_type == INTR_TYPE_HARD_EXCEPTION \u0026\u0026 vector \u003e 31) \|\|\n-\t\t CC(intr_type == INTR_TYPE_OTHER_EVENT \u0026\u0026 vector != 0))\n-\t\t\treturn -EINVAL;\n-\n \t\t/\n \t\t * Cannot deliver error code in real mode or if the interrupt\n \t\t * type is not hardware exception. For other cases, do the\n@@ -3106,8 +3158,28 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu vcpu,\n \t\tif (CC(intr_info \u0026 INTR_INFO_RESVD_BITS_MASK))\n \t\t\treturn -EINVAL;\n \n-\t\t/ VM-entry instruction length /\n+\t\tif (CC(intr_type == INTR_TYPE_RESERVED))\n+\t\t\treturn -EINVAL;\n+\n+\t\t/\n+\t\t * Only for hardware exceptions and when the CPU enumerates\n+\t\t * VMX nested-exception support, bit 13 (indicating a nested\n+\t\t * exception) has value 1. Otherwise it is reserved.\n+\t\t /\n+\t\tif (CC(has_nested_exception \u0026\u0026\n+\t\t (intr_type != INTR_TYPE_HARD_EXCEPTION \|\|\n+\t\t\t!nested_cpu_has_nested_exception(vcpu))))\n+\t\t\treturn -EINVAL;\n+\n \t\tswitch (intr_type) {\n+\t\tcase INTR_TYPE_NMI_INTR:\n+\t\t\tif (CC(vector != NMI_VECTOR))\n+\t\t\t\treturn -EINVAL;\n+\t\t\tbreak;\n+\t\tcase INTR_TYPE_HARD_EXCEPTION:\n+\t\t\tif (CC(vector \u003e 31))\n+\t\t\t\treturn -EINVAL;\n+\t\t\tbreak;\n \t\tcase INTR_TYPE_SOFT_EXCEPTION:\n \t\tcase INTR_TYPE_SOFT_INTR:\n \t\tcase INTR_TYPE_PRIV_SW_EXCEPTION:\n@@ -3115,6 +3187,28 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu vcpu,\n \t\t\t CC(vmcs12-\u003evm_entry_instruction_len == 0 \u0026\u0026\n \t\t\t CC(!nested_cpu_has_zero_length_injection(vcpu))))\n \t\t\t\treturn -EINVAL;\n+\t\t\tbreak;\n+\t\tcase INTR_TYPE_OTHER_EVENT:\n+\t\t\tif (CC(vector \u003e 2))\n+\t\t\t\treturn -EINVAL;\n+\n+\t\t\tswitch (vector) {\n+\t\t\tcase 0:\n+\t\t\t\tif (CC(!nested_cpu_supports_monitor_trap_flag(vcpu)))\n+\t\t\t\t\treturn -EINVAL;\n+\t\t\t\tbreak;\n+\t\t\tcase 1:\n+\t\t\tcase 2:\n+\t\t\t\tif (CC(!fred_enabled))\n+\t\t\t\t\treturn -EINVAL;\n+\t\t\t\tif (CC(vmcs12-\u003evm_entry_instruction_len \u003e X86_MAX_INSTRUCTION_LENGTH))\n+\t\t\t\t\treturn -EINVAL;\n+\t\t\t\tif (CC(vmcs12-\u003evm_entry_instruction_len == 0 \u0026\u0026\n+\t\t\t\t !nested_cpu_has_zero_length_injection(vcpu)))\n+\t\t\t\t\treturn -EINVAL;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tbreak;\n \t\t}\n \t}\n \n@@ -3201,9 +3295,27 @@ static int nested_vmx_check_host_state(struct kvm_vcpu vcpu,\n \tif (ia32e) {\n \t\tif (CC(!(vmcs12-\u003ehost_cr4 \u0026 X86_CR4_PAE)))\n \t\t\treturn -EINVAL;\n+\t\tif (nested_cpu_load_host_fred_state(vmcs12)) {\n+\t\t\tif (CC(vmcs12-\u003ehost_ia32_fred_config \u0026 FRED_CONFIG_RESERVED) \|\|\n+\t\t\t CC(vmcs12-\u003ehost_ia32_fred_rsp1 \u0026 GENMASK_ULL(5, 0)) \|\|\n+\t\t\t CC(vmcs12-\u003ehost_ia32_fred_rsp2 \u0026 GENMASK_ULL(5, 0)) \|\|\n+\t\t\t CC(vmcs12-\u003ehost_ia32_fred_rsp3 \u0026 GENMASK_ULL(5, 0)) \|\|\n+\t\t\t CC(vmcs12-\u003ehost_ia32_fred_ssp1 \u0026 GENMASK_ULL(2, 0)) \|\|\n+\t\t\t CC(vmcs12-\u003ehost_ia32_fred_ssp2 \u0026 GENMASK_ULL(2, 0)) \|\|\n+\t\t\t CC(vmcs12-\u003ehost_ia32_fred_ssp3 \u0026 GENMASK_ULL(2, 0)) \|\|\n+\t\t\t CC(is_noncanonical_msr_address(vmcs12-\u003ehost_ia32_fred_config \u0026 PAGE_MASK, vcpu)) \|\|\n+\t\t\t CC(is_noncanonical_msr_address(vmcs12-\u003ehost_ia32_fred_rsp1, vcpu)) \|\|\n+\t\t\t CC(is_noncanonical_msr_address(vmcs12-\u003ehost_ia32_fred_rsp2, vcpu)) \|\|\n+\t\t\t CC(is_noncanonical_msr_address(vmcs12-\u003ehost_ia32_fred_rsp3, vcpu)) \|\|\n+\t\t\t CC(is_noncanonical_msr_address(vmcs12-\u003ehost_ia32_fred_ssp1, vcpu)) \|\|\n+\t\t\t CC(is_noncanonical_msr_address(vmcs12-\u003ehost_ia32_fred_ssp2, vcpu)) \|\|\n+\t\t\t CC(is_noncanonical_msr_address(vmcs12-\u003ehost_ia32_fred_ssp3, vcpu)))\n+\t\t\t\treturn -EINVAL;\n+\t\t}\n \t} else {\n \t\tif (CC(vmcs12-\u003evm_entry_controls \u0026 VM_ENTRY_IA32E_MODE) \|\|\n \t\t CC(vmcs12-\u003ehost_cr4 \u0026 X86_CR4_PCIDE) \|\|\n+\t\t CC(vmcs12-\u003ehost_cr4 \u0026 X86_CR4_FRED) \|\|\n \t\t CC((vmcs12-\u003ehost_rip) \u003e\u003e 32))\n \t\t\treturn -EINVAL;\n \t}\n@@ -3386,6 +3498,48 @@ static int nested_vmx_check_guest_state(struct kvm_vcpu vcpu,\n \t CC((vmcs12-\u003eguest_bndcfgs \u0026 MSR_IA32_BNDCFGS_RSVD))))\n \t\treturn -EINVAL;\n \n+\tif (ia32e) {\n+\t\tif (nested_cpu_load_guest_fred_state(vmcs12)) {\n+\t\t\tif (CC(vmcs12-\u003eguest_ia32_fred_config \u0026 FRED_CONFIG_RESERVED) \|\|\n+\t\t\t CC(vmcs12-\u003eguest_ia32_fred_rsp1 \u0026 GENMASK_ULL(5, 0)) \|\|\n+\t\t\t CC(vmcs12-\u003eguest_ia32_fred_rsp2 \u0026 GENMASK_ULL(5, 0)) \|\|\n+\t\t\t CC(vmcs12-\u003eguest_ia32_fred_rsp3 \u0026 GENMASK_ULL(5, 0)) \|\|\n+\t\t\t CC(vmcs12-\u003eguest_ia32_fred_ssp1 \u0026 GENMASK_ULL(2, 0)) \|\|\n+\t\t\t CC(vmcs12-\u003eguest_ia32_fred_ssp2 \u0026 GENMASK_ULL(2, 0)) \|\|\n+\t\t\t CC(vmcs12-\u003eguest_ia32_fred_ssp3 \u0026 GENMASK_ULL(2, 0)) \|\|\n+\t\t\t CC(is_noncanonical_msr_address(vmcs12-\u003eguest_ia32_fred_config \u0026 PAGE_MASK, vcpu)) \|\|\n+\t\t\t CC(is_noncanonical_msr_address(vmcs12-\u003eguest_ia32_fred_rsp1, vcpu)) \|\|\n+\t\t\t CC(is_noncanonical_msr_address(vmcs12-\u003eguest_ia32_fred_rsp2, vcpu)) \|\|\n+\t\t\t CC(is_noncanonical_msr_address(vmcs12-\u003eguest_ia32_fred_rsp3, vcpu)) \|\|\n+\t\t\t CC(is_noncanonical_msr_address(vmcs12-\u003eguest_ia32_fred_ssp1, vcpu)) \|\|\n+\t\t\t CC(is_noncanonical_msr_address(vmcs12-\u003eguest_ia32_fred_ssp2, vcpu)) \|\|\n+\t\t\t CC(is_noncanonical_msr_address(vmcs12-\u003eguest_ia32_fred_ssp3, vcpu)))\n+\t\t\t\treturn -EINVAL;\n+\t\t}\n+\t\tif (vmcs12-\u003eguest_cr4 \u0026 X86_CR4_FRED) {\n+\t\t\tunsigned int ss_dpl = VMX_AR_DPL(vmcs12-\u003eguest_ss_ar_bytes);\n+\n+\t\t\tif (CC(ss_dpl == 1 \|\| ss_dpl == 2))\n+\t\t\t\treturn -EINVAL;\n+\n+\t\t\tswitch (ss_dpl) {\n+\t\t\tcase 0:\n+\t\t\t\tif (CC(!(vmcs12-\u003eguest_cs_ar_bytes \u0026 VMX_AR_L_MASK)))\n+\t\t\t\t\treturn -EINVAL;\n+\t\t\t\tbreak;\n+\t\t\tcase 3:\n+\t\t\t\tif (CC(vmcs12-\u003eguest_rflags \u0026 X86_EFLAGS_IOPL))\n+\t\t\t\t\treturn -EINVAL;\n+\t\t\t\tif (CC(vmcs12-\u003eguest_interruptibility_info \u0026 GUEST_INTR_STATE_STI))\n+\t\t\t\t\treturn -EINVAL;\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\t} else {\n+\t\tif (CC(vmcs12-\u003eguest_cr4 \u0026 X86_CR4_FRED))\n+\t\t\treturn -EINVAL;\n+\t}\n+\n \tif (vmcs12-\u003evm_entry_controls \u0026 VM_ENTRY_LOAD_CET_STATE) {\n \t\tif (nested_vmx_check_cet_state_common(vcpu, vmcs12-\u003eguest_s_cet,\n \t\t\t\t\t\t vmcs12-\u003eguest_ssp,\n@@ -3614,7 +3768,8 @@ static int nested_vmx_check_permission(struct kvm_vcpu vcpu)\n }\n \n static void load_vmcs12_host_state(struct kvm_vcpu vcpu,\n-\t\t\t\t struct vmcs12 vmcs12);\n+\t\t\t\t struct vmcs12 vmcs12,\n+\t\t\t\t bool from_failed_vmentry);\n \n /\n If from_vmentry is false, this is being called from state restore (either RSM\n@@ -3664,6 +3819,10 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu vcpu,\n \t\t\t\t \u0026vmx-\u003enested.pre_vmenter_ssp,\n \t\t\t\t \u0026vmx-\u003enested.pre_vmenter_ssp_tbl);\n \n+\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED) \u0026\u0026\n+\t (!vmx-\u003evcpu.arch.nested_run_pending \|\| !nested_cpu_load_guest_fred_state(vmcs12)))\n+\t\tvmcs_read_fred_msrs(\u0026vmx-\u003enested.pre_vmenter_fred_msrs);\n+\n \t/\n \t * Stash L1's CR3, so that in the event of a \"late\" VM-Fail, i.e. a\n \t * VM-Fail detected by hardware but not KVM, KVM can unwind its\n@@ -3778,7 +3937,7 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu vcpu,\n \n \tnested_put_vmcs12_pages(vcpu);\n \n-\tload_vmcs12_host_state(vcpu, vmcs12);\n+\tload_vmcs12_host_state(vcpu, vmcs12, true);\n \tvmcs12-\u003evm_exit_reason = exit_reason.full;\n \tif (enable_shadow_vmcs \|\| nested_vmx_is_evmptr12_valid(vmx))\n \t\tvmx-\u003enested.need_vmcs12_to_shadow_sync = true;\n@@ -3967,6 +4126,8 @@ static void vmcs12_save_pending_event(struct kvm_vcpu vcpu,\n \tu32 idt_vectoring;\n \tunsigned int nr;\n \n+\tvmcs12-\u003eoriginal_event_data = 0;\n+\n \t/\n \t Per the SDM, VM-Exits due to double and triple faults are never\n \t * considered to occur during event delivery, even if the double/triple\n@@ -4005,6 +4166,13 @@ static void vmcs12_save_pending_event(struct kvm_vcpu vcpu,\n \t\t\t\tvcpu-\u003earch.exception.error_code;\n \t\t}\n \n+\t\tif ((vmcs12-\u003evm_entry_controls \u0026 VM_ENTRY_IA32E_MODE) \u0026\u0026\n+\t\t (vmcs12-\u003eguest_cr4 \u0026 X86_CR4_FRED) \u0026\u0026\n+\t\t (vcpu-\u003earch.exception.is_nested))\n+\t\t\tidt_vectoring \|= VECTORING_INFO_NESTED_EXCEPTION_MASK;\n+\n+\t\tvmcs12-\u003eoriginal_event_data = vcpu-\u003earch.exception.event_data;\n+\n \t\tvmcs12-\u003eidt_vectoring_info_field = idt_vectoring;\n \t} else if (vcpu-\u003earch.nmi_injected) {\n \t\tvmcs12-\u003eidt_vectoring_info_field =\n@@ -4728,6 +4896,15 @@ static void sync_vmcs02_to_vmcs12(struct kvm_vcpu vcpu, struct vmcs12 vmcs12)\n \tvmcs_read_cet_state(\u0026vmx-\u003evcpu, \u0026vmcs12-\u003eguest_s_cet,\n \t\t\t \u0026vmcs12-\u003eguest_ssp,\n \t\t\t \u0026vmcs12-\u003eguest_ssp_tbl);\n+\n+\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) {\n+\t\tvmcs_read_fred_msrs(\u0026vmx-\u003enested.at_vmexit_fred_msrs);\n+\n+\t\tif (nested_cpu_save_guest_fred_state(vmcs12))\n+\t\t\tmemcpy(\u0026vmcs12-\u003eguest_fred_msrs,\n+\t\t\t \u0026vmx-\u003enested.at_vmexit_fred_msrs,\n+\t\t\t sizeof(struct vmcs_fred_msrs));\n+\t}\n }\n \n /\n@@ -4772,6 +4949,21 @@ static void prepare_vmcs12(struct kvm_vcpu vcpu, struct vmcs12 vmcs12,\n \n \t\tvmcs12-\u003evm_exit_intr_info = exit_intr_info;\n \t\tvmcs12-\u003evm_exit_instruction_len = exit_insn_len;\n+\n+\t\t/\n+\t\t When there is a valid original event, the exiting event is a nested\n+\t\t * event during delivery of the earlier original event.\n+\t\t \n+\t\t FRED event delivery reflects this relationship by setting the value\n+\t\t * of the nested exception bit of VM-exit interruption information\n+\t\t * (aka exiting-event identification) to that of the valid bit of the\n+\t\t * IDT-vectoring information (aka original-event identification).\n+\t\t /\n+\t\tif ((vmcs12-\u003eidt_vectoring_info_field \u0026 VECTORING_INFO_VALID_MASK) \u0026\u0026\n+\t\t (vmcs12-\u003evm_entry_controls \u0026 VM_ENTRY_IA32E_MODE) \u0026\u0026\n+\t\t (vmcs12-\u003eguest_cr4 \u0026 X86_CR4_FRED))\n+\t\t\tvmcs12-\u003evm_exit_intr_info \|= INTR_INFO_NESTED_EXCEPTION_MASK;\n+\n \t\tvmcs12-\u003evmx_instruction_info = vmcs_read32(VMX_INSTRUCTION_INFO);\n \n \t\t/\n@@ -4798,8 +4990,10 @@ static void prepare_vmcs12(struct kvm_vcpu vcpu, struct vmcs12 vmcs12,\n * This function should be called when the active VMCS is L1's (vmcs01).\n /\n static void load_vmcs12_host_state(struct kvm_vcpu vcpu,\n-\t\t\t\t struct vmcs12 vmcs12)\n+\t\t\t\t struct vmcs12 vmcs12,\n+\t\t\t\t bool from_failed_vmentry)\n {\n+\tstruct vcpu_vmx vmx = to_vmx(vcpu);\n \tenum vm_entry_failure_code ignored;\n \tstruct kvm_segment seg;\n \n@@ -4874,6 +5068,14 @@ static void load_vmcs12_host_state(struct kvm_vcpu vcpu,\n \t\tWARN_ON_ONCE(__kvm_emulate_msr_write(vcpu, MSR_CORE_PERF_GLOBAL_CTRL,\n \t\t\t\t\t\t vmcs12-\u003ehost_ia32_perf_global_ctrl));\n \n+\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) {\n+\t\tif (nested_cpu_load_host_fred_state(vmcs12)) {\n+\t\t\tvmcs_write_fred_msrs(\u0026vmcs12-\u003ehost_fred_msrs);\n+\t\t} else if (!from_failed_vmentry) {\n+\t\t\tvmcs_write_fred_msrs(\u0026vmx-\u003enested.at_vmexit_fred_msrs);\n+\t\t}\n+\t}\n+\n \t/* Set L1 segment info according to Intel SDM\n \t 27.5.2 Loading Host Segment and Descriptor-Table Registers /\n \tseg = (struct kvm_segment) {\n@@ -5193,7 +5395,7 @@ void __nested_vmx_vmexit(struct kvm_vcpu vcpu, u32 vm_exit_reason,\n \t\t\t\t\t\t vmcs12-\u003evm_exit_intr_error_code,\n \t\t\t\t\t\t KVM_ISA_VMX);\n \n-\t\tload_vmcs12_host_state(vcpu, vmcs12);\n+\t\tload_vmcs12_host_state(vcpu, vmcs12, false);\n \n \t\t/\n \t\t Process events if an injectable IRQ or NMI is pending, even\n@@ -7152,7 +7354,8 @@ static void nested_vmx_setup_exit_ctls(struct vmcs_config vmcs_conf,\n \t\tVM_EXIT_HOST_ADDR_SPACE_SIZE \|\n #endif\n \t\tVM_EXIT_LOAD_IA32_PAT \| VM_EXIT_SAVE_IA32_PAT \|\n-\t\tVM_EXIT_CLEAR_BNDCFGS \| VM_EXIT_LOAD_CET_STATE;\n+\t\tVM_EXIT_CLEAR_BNDCFGS \| VM_EXIT_LOAD_CET_STATE \|\n+\t\tVM_EXIT_ACTIVATE_SECONDARY_CONTROLS;\n \tmsrs-\u003eexit_ctls_high \|=\n \t\tVM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR \|\n \t\tVM_EXIT_LOAD_IA32_EFER \| VM_EXIT_SAVE_IA32_EFER \|\n@@ -7165,6 +7368,12 @@ static void nested_vmx_setup_exit_ctls(struct vmcs_config vmcs_conf,\n \n \t/* We support free control of debug control saving. /\n \tmsrs-\u003eexit_ctls_low \u0026= ~VM_EXIT_SAVE_DEBUG_CONTROLS;\n+\n+\tif (msrs-\u003eexit_ctls_high \u0026 VM_EXIT_ACTIVATE_SECONDARY_CONTROLS) {\n+\t\tmsrs-\u003esecondary_exit_ctls = vmcs_conf-\u003evmexit_2nd_ctrl;\n+\t\tmsrs-\u003esecondary_exit_ctls \u0026= SECONDARY_VM_EXIT_SAVE_IA32_FRED \|\n+\t\t\t\t\t SECONDARY_VM_EXIT_LOAD_IA32_FRED;\n+\t}\n }\n \n static void nested_vmx_setup_entry_ctls(struct vmcs_config vmcs_conf,\n@@ -7179,7 +7388,7 @@ static void nested_vmx_setup_entry_ctls(struct vmcs_config vmcs_conf,\n \t\tVM_ENTRY_IA32E_MODE \|\n #endif\n \t\tVM_ENTRY_LOAD_IA32_PAT \| VM_ENTRY_LOAD_BNDCFGS \|\n-\t\tVM_ENTRY_LOAD_CET_STATE;\n+\t\tVM_ENTRY_LOAD_CET_STATE \| VM_ENTRY_LOAD_IA32_FRED;\n \tmsrs-\u003eentry_ctls_high \|=\n \t\t(VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR \| VM_ENTRY_LOAD_IA32_EFER \|\n \t\t VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL);\n@@ -7343,6 +7552,8 @@ static void nested_vmx_setup_basic(struct nested_vmx_msrs msrs)\n \t\tmsrs-\u003ebasic \|= VMX_BASIC_INOUT;\n \tif (cpu_has_vmx_basic_no_hw_errcode_cc())\n \t\tmsrs-\u003ebasic \|= VMX_BASIC_NO_HW_ERROR_CODE_CC;\n+\tif (cpu_has_vmx_nested_exception())\n+\t\tmsrs-\u003ebasic \|= VMX_BASIC_NESTED_EXCEPTION;\n }\n \n static void nested_vmx_setup_cr_fixed(struct nested_vmx_msrs msrs)\ndiff --git a/arch/x86/kvm/vmx/nested.h b/arch/x86/kvm/vmx/nested.h\nindex c6de848bd9ced..5511218241281 100644\n--- a/arch/x86/kvm/vmx/nested.h\n+++ b/arch/x86/kvm/vmx/nested.h\n@@ -247,6 +247,11 @@ static inline bool nested_cpu_has_save_preemption_timer(struct vmcs12 vmcs12)\n \t VM_EXIT_SAVE_VMX_PREEMPTION_TIMER;\n }\n \n+static inline bool nested_cpu_has_secondary_vm_exit_controls(struct vmcs12 vmcs12)\n+{\n+\treturn vmcs12-\u003evm_exit_controls \u0026 VM_EXIT_ACTIVATE_SECONDARY_CONTROLS;\n+}\n+\n static inline bool nested_exit_on_nmi(struct kvm_vcpu vcpu)\n {\n \treturn nested_cpu_has_nmi_exiting(get_vmcs12(vcpu));\n@@ -267,6 +272,23 @@ static inline bool nested_cpu_has_encls_exit(struct vmcs12 vmcs12)\n \treturn nested_cpu_has2(vmcs12, SECONDARY_EXEC_ENCLS_EXITING);\n }\n \n+static inline bool nested_cpu_load_guest_fred_state(struct vmcs12 vmcs12)\n+{\n+\treturn vmcs12-\u003evm_entry_controls \u0026 VM_ENTRY_LOAD_IA32_FRED;\n+}\n+\n+static inline bool nested_cpu_save_guest_fred_state(struct vmcs12 vmcs12)\n+{\n+\treturn nested_cpu_has_secondary_vm_exit_controls(vmcs12) \u0026\u0026\n+\t vmcs12-\u003esecondary_vm_exit_controls \u0026 SECONDARY_VM_EXIT_SAVE_IA32_FRED;\n+}\n+\n+static inline bool nested_cpu_load_host_fred_state(struct vmcs12 vmcs12)\n+{\n+\treturn nested_cpu_has_secondary_vm_exit_controls(vmcs12) \u0026\u0026\n+\t vmcs12-\u003esecondary_vm_exit_controls \u0026 SECONDARY_VM_EXIT_LOAD_IA32_FRED;\n+}\n+\n /\n if fixed0[i] == 1: val[i] must be 1\n * if fixed1[i] == 0: val[i] must be 0\n@@ -312,6 +334,11 @@ static inline bool nested_cpu_has_no_hw_errcode_cc(struct kvm_vcpu vcpu)\n \treturn to_vmx(vcpu)-\u003enested.msrs.basic \u0026 VMX_BASIC_NO_HW_ERROR_CODE_CC;\n }\n \n+static inline bool nested_cpu_has_nested_exception(struct kvm_vcpu vcpu)\n+{\n+\treturn to_vmx(vcpu)-\u003enested.msrs.basic \u0026 VMX_BASIC_NESTED_EXCEPTION;\n+}\n+\n /* No difference in the restrictions on guest and host CR4 in VMX operation. /\n #define nested_guest_cr4_valid\tnested_cr4_valid\n #define nested_host_cr4_valid\tnested_cr4_valid\ndiff --git a/arch/x86/kvm/vmx/vmcs.h b/arch/x86/kvm/vmx/vmcs.h\nindex 1f16ddeae9cb1..58af10f73789e 100644\n--- a/arch/x86/kvm/vmx/vmcs.h\n+++ b/arch/x86/kvm/vmx/vmcs.h\n@@ -45,12 +45,24 @@ struct vmcs_host_state {\n struct vmcs_controls_shadow {\n \tu32 vm_entry;\n \tu32 vm_exit;\n+\tu64 secondary_vm_exit;\n \tu32 pin;\n \tu32 exec;\n \tu32 secondary_exec;\n \tu64 tertiary_exec;\n };\n \n+struct vmcs_fred_msrs {\n+\tu64 fred_config;\n+\tu64 fred_rsp1;\n+\tu64 fred_rsp2;\n+\tu64 fred_rsp3;\n+\tu64 fred_stklvls;\n+\tu64 fred_ssp1;\n+\tu64 fred_ssp2;\n+\tu64 fred_ssp3;\n+};\n+\n /\n * Track a VMCS that may be loaded on a certain CPU. If it is (cpu!=-1), also\n * remember whether it was VMLAUNCHed, and maintain a linked list of all VMCSs\ndiff --git a/arch/x86/kvm/vmx/vmcs12.c b/arch/x86/kvm/vmx/vmcs12.c\nindex 1ebe67c384ad4..67c730e314655 100644\n--- a/arch/x86/kvm/vmx/vmcs12.c\n+++ b/arch/x86/kvm/vmx/vmcs12.c\n@@ -66,6 +66,25 @@ static const u16 kvm_supported_vmcs12_field_offsets[] __initconst = {\n \tFIELD64(HOST_IA32_PAT, host_ia32_pat),\n \tFIELD64(HOST_IA32_EFER, host_ia32_efer),\n \tFIELD64(HOST_IA32_PERF_GLOBAL_CTRL, host_ia32_perf_global_ctrl),\n+\tFIELD64(SECONDARY_VM_EXIT_CONTROLS, secondary_vm_exit_controls),\n+\tFIELD64(INJECTED_EVENT_DATA, injected_event_data),\n+\tFIELD64(ORIGINAL_EVENT_DATA, original_event_data),\n+\tFIELD64(GUEST_IA32_FRED_CONFIG, guest_ia32_fred_config),\n+\tFIELD64(GUEST_IA32_FRED_RSP1, guest_ia32_fred_rsp1),\n+\tFIELD64(GUEST_IA32_FRED_RSP2, guest_ia32_fred_rsp2),\n+\tFIELD64(GUEST_IA32_FRED_RSP3, guest_ia32_fred_rsp3),\n+\tFIELD64(GUEST_IA32_FRED_STKLVLS, guest_ia32_fred_stklvls),\n+\tFIELD64(GUEST_IA32_FRED_SSP1, guest_ia32_fred_ssp1),\n+\tFIELD64(GUEST_IA32_FRED_SSP2, guest_ia32_fred_ssp2),\n+\tFIELD64(GUEST_IA32_FRED_SSP3, guest_ia32_fred_ssp3),\n+\tFIELD64(HOST_IA32_FRED_CONFIG, host_ia32_fred_config),\n+\tFIELD64(HOST_IA32_FRED_RSP1, host_ia32_fred_rsp1),\n+\tFIELD64(HOST_IA32_FRED_RSP2, host_ia32_fred_rsp2),\n+\tFIELD64(HOST_IA32_FRED_RSP3, host_ia32_fred_rsp3),\n+\tFIELD64(HOST_IA32_FRED_STKLVLS, host_ia32_fred_stklvls),\n+\tFIELD64(HOST_IA32_FRED_SSP1, host_ia32_fred_ssp1),\n+\tFIELD64(HOST_IA32_FRED_SSP2, host_ia32_fred_ssp2),\n+\tFIELD64(HOST_IA32_FRED_SSP3, host_ia32_fred_ssp3),\n \tFIELD(PIN_BASED_VM_EXEC_CONTROL, pin_based_vm_exec_control),\n \tFIELD(CPU_BASED_VM_EXEC_CONTROL, cpu_based_vm_exec_control),\n \tFIELD(EXCEPTION_BITMAP, exception_bitmap),\n@@ -203,6 +222,9 @@ static __init bool cpu_has_vmcs12_field(unsigned int idx)\n \tcase HOST_SSP:\n \tcase HOST_INTR_SSP_TABLE:\n \t\treturn cpu_has_load_cet_ctrl();\n+\tVMCS12_CASE64(ORIGINAL_EVENT_DATA):\n+\tVMCS12_CASE64(INJECTED_EVENT_DATA):\n+\t\treturn cpu_has_vmx_fred();\n \n \t/* KVM always emulates PML and the VMX preemption timer in software. /\n \tcase GUEST_PML_INDEX:\ndiff --git a/arch/x86/kvm/vmx/vmcs12.h b/arch/x86/kvm/vmx/vmcs12.h\nindex 21cd1b75e4fdf..c4c5cd3722a5b 100644\n--- a/arch/x86/kvm/vmx/vmcs12.h\n+++ b/arch/x86/kvm/vmx/vmcs12.h\n@@ -71,7 +71,7 @@ struct __packed vmcs12 {\n \tu64 pml_address;\n \tu64 encls_exiting_bitmap;\n \tu64 tsc_multiplier;\n-\tu64 padding64[1]; / room for future expansion /\n+\tu64 secondary_vm_exit_controls;\n \t/\n \t * To allow migration of L1 (complete with its L2 guests) between\n \t * machines of different natural widths (32 or 64 bit), we cannot have\n@@ -191,6 +191,27 @@ struct __packed vmcs12 {\n \tu16 host_gs_selector;\n \tu16 host_tr_selector;\n \tu16 guest_pml_index;\n+\tu16 padding16[1]; /* align to 64-bit boundary /\n+\tstruct vmcs_fred_msrs guest_fred_msrs;\n+#define guest_ia32_fred_config\tguest_fred_msrs.fred_config\n+#define guest_ia32_fred_rsp1\tguest_fred_msrs.fred_rsp1\n+#define guest_ia32_fred_rsp2\tguest_fred_msrs.fred_rsp2\n+#define guest_ia32_fred_rsp3\tguest_fred_msrs.fred_rsp3\n+#define guest_ia32_fred_stklvls\tguest_fred_msrs.fred_stklvls\n+#define guest_ia32_fred_ssp1\tguest_fred_msrs.fred_ssp1\n+#define guest_ia32_fred_ssp2\tguest_fred_msrs.fred_ssp2\n+#define guest_ia32_fred_ssp3\tguest_fred_msrs.fred_ssp3\n+\tstruct vmcs_fred_msrs host_fred_msrs;\n+#define host_ia32_fred_config\thost_fred_msrs.fred_config\n+#define host_ia32_fred_rsp1\thost_fred_msrs.fred_rsp1\n+#define host_ia32_fred_rsp2\thost_fred_msrs.fred_rsp2\n+#define host_ia32_fred_rsp3\thost_fred_msrs.fred_rsp3\n+#define host_ia32_fred_stklvls\thost_fred_msrs.fred_stklvls\n+#define host_ia32_fred_ssp1\thost_fred_msrs.fred_ssp1\n+#define host_ia32_fred_ssp2\thost_fred_msrs.fred_ssp2\n+#define host_ia32_fred_ssp3\thost_fred_msrs.fred_ssp3\n+\tu64 injected_event_data;\n+\tu64 original_event_data;\n };\n \n /\n@@ -261,6 +282,7 @@ static inline void vmx_check_vmcs12_offsets(void)\n \tCHECK_OFFSET(pml_address, 312);\n \tCHECK_OFFSET(encls_exiting_bitmap, 320);\n \tCHECK_OFFSET(tsc_multiplier, 328);\n+\tCHECK_OFFSET(secondary_vm_exit_controls, 336);\n \tCHECK_OFFSET(cr0_guest_host_mask, 344);\n \tCHECK_OFFSET(cr4_guest_host_mask, 352);\n \tCHECK_OFFSET(cr0_read_shadow, 360);\n@@ -372,6 +394,24 @@ static inline void vmx_check_vmcs12_offsets(void)\n \tCHECK_OFFSET(host_gs_selector, 992);\n \tCHECK_OFFSET(host_tr_selector, 994);\n \tCHECK_OFFSET(guest_pml_index, 996);\n+\tCHECK_OFFSET(guest_ia32_fred_config, 1000);\n+\tCHECK_OFFSET(guest_ia32_fred_rsp1, 1008);\n+\tCHECK_OFFSET(guest_ia32_fred_rsp2, 1016);\n+\tCHECK_OFFSET(guest_ia32_fred_rsp3, 1024);\n+\tCHECK_OFFSET(guest_ia32_fred_stklvls, 1032);\n+\tCHECK_OFFSET(guest_ia32_fred_ssp1, 1040);\n+\tCHECK_OFFSET(guest_ia32_fred_ssp2, 1048);\n+\tCHECK_OFFSET(guest_ia32_fred_ssp3, 1056);\n+\tCHECK_OFFSET(host_ia32_fred_config, 1064);\n+\tCHECK_OFFSET(host_ia32_fred_rsp1, 1072);\n+\tCHECK_OFFSET(host_ia32_fred_rsp2, 1080);\n+\tCHECK_OFFSET(host_ia32_fred_rsp3, 1088);\n+\tCHECK_OFFSET(host_ia32_fred_stklvls, 1096);\n+\tCHECK_OFFSET(host_ia32_fred_ssp1, 1104);\n+\tCHECK_OFFSET(host_ia32_fred_ssp2, 1112);\n+\tCHECK_OFFSET(host_ia32_fred_ssp3, 1120);\n+\tCHECK_OFFSET(injected_event_data, 1128);\n+\tCHECK_OFFSET(original_event_data, 1136);\n }\n \n extern u16 vmcs12_field_offsets[] __ro_after_init;\ndiff --git a/arch/x86/kvm/vmx/vmcs_shadow_fields.h b/arch/x86/kvm/vmx/vmcs_shadow_fields.h\nindex 67e821c2be6d5..bdd56f3ca6c30 100644\n--- a/arch/x86/kvm/vmx/vmcs_shadow_fields.h\n+++ b/arch/x86/kvm/vmx/vmcs_shadow_fields.h\n@@ -74,6 +74,10 @@ SHADOW_FIELD_RW(HOST_GS_BASE, host_gs_base)\n /* 64-bit /\n SHADOW_FIELD_RO(GUEST_PHYSICAL_ADDRESS, guest_physical_address)\n SHADOW_FIELD_RO(GUEST_PHYSICAL_ADDRESS_HIGH, guest_physical_address)\n+SHADOW_FIELD_RO(ORIGINAL_EVENT_DATA, original_event_data)\n+SHADOW_FIELD_RO(ORIGINAL_EVENT_DATA_HIGH, original_event_data)\n+SHADOW_FIELD_RW(INJECTED_EVENT_DATA, injected_event_data)\n+SHADOW_FIELD_RW(INJECTED_EVENT_DATA_HIGH, injected_event_data)\n #endif\n \n #undef SHADOW_FIELD_RO\ndiff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c\nindex 504630f0eb40f..f673cd4e8956e 100644\n--- a/arch/x86/kvm/vmx/vmx.c\n+++ b/arch/x86/kvm/vmx/vmx.c\n@@ -1395,6 +1395,9 @@ void vmx_prepare_switch_to_guest(struct kvm_vcpu vcpu)\n \t}\n \n \twrmsrq(MSR_KERNEL_GS_BASE, vmx-\u003emsr_guest_kernel_gs_base);\n+\n+\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))\n+\t\twrmsrns(MSR_IA32_FRED_RSP0, vmx-\u003emsr_guest_fred_rsp0);\n #else\n \tsavesegment(fs, fs_sel);\n \tsavesegment(gs, gs_sel);\n@@ -1439,6 +1442,16 @@ static void vmx_prepare_switch_to_host(struct vcpu_vmx vmx)\n \tinvalidate_tss_limit();\n #ifdef CONFIG_X86_64\n \twrmsrq(MSR_KERNEL_GS_BASE, vmx-\u003evt.msr_host_kernel_gs_base);\n+\n+\tif (guest_cpu_cap_has(\u0026vmx-\u003evcpu, X86_FEATURE_FRED)) {\n+\t\tvmx-\u003emsr_guest_fred_rsp0 = read_msr(MSR_IA32_FRED_RSP0);\n+\t\t/\n+\t\t * Synchronize the current value in hardware to the kernel's\n+\t\t * local cache. The desired host RSP0 will be set when the\n+\t\t * CPU exits to userspace (RSP0 is a per-task value).\n+\t\t /\n+\t\tfred_sync_rsp0(vmx-\u003emsr_guest_fred_rsp0);\n+\t}\n #endif\n \tload_fixmap_gdt(raw_smp_processor_id());\n \tvmx-\u003evt.guest_state_loaded = false;\n@@ -1476,6 +1489,21 @@ static void vmx_write_guest_kernel_gs_base(struct vcpu_vmx vmx, u64 data)\n \tvmx_write_guest_host_msr(vmx, MSR_KERNEL_GS_BASE, data,\n \t\t\t\t \u0026vmx-\u003emsr_guest_kernel_gs_base);\n }\n+\n+static u64 vmx_read_guest_fred_rsp0(struct vcpu_vmx vmx)\n+{\n+\treturn vmx_read_guest_host_msr(vmx, MSR_IA32_FRED_RSP0,\n+\t\t\t\t \u0026vmx-\u003emsr_guest_fred_rsp0);\n+}\n+\n+static void vmx_write_guest_fred_rsp0(struct vcpu_vmx vmx, u64 data)\n+{\n+\tvmx_write_guest_host_msr(vmx, MSR_IA32_FRED_RSP0, data,\n+\t\t\t\t \u0026vmx-\u003emsr_guest_fred_rsp0);\n+}\n+#else\n+/* Make sure it builds on 32-bit /\n+static u64 vmx_read_guest_fred_rsp0(struct vcpu_vmx vmx) { return 0; }\n #endif\n \n static void grow_ple_window(struct kvm_vcpu vcpu)\n@@ -1562,6 +1590,12 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu vcpu, int cpu)\n \t\t\t\t (unsigned long)(cpu_entry_stack(cpu) + 1));\n \t\t}\n \n+\t\tif (cpu_feature_enabled(X86_FEATURE_FRED) \u0026\u0026 kvm_cpu_cap_has(X86_FEATURE_FRED)) {\n+\t\t\tvmcs_write64(HOST_IA32_FRED_RSP1, this_cpu_fred_rsp(FRED_STACK_LEVEL_1));\n+\t\t\tvmcs_write64(HOST_IA32_FRED_RSP2, this_cpu_fred_rsp(FRED_STACK_LEVEL_2));\n+\t\t\tvmcs_write64(HOST_IA32_FRED_RSP3, this_cpu_fred_rsp(FRED_STACK_LEVEL_3));\n+\t\t}\n+\n \t\tvmx-\u003eloaded_vmcs-\u003ecpu = cpu;\n \t}\n }\n@@ -1973,12 +2007,41 @@ void vmx_inject_exception(struct kvm_vcpu vcpu)\n \n \tWARN_ON_ONCE(vmx-\u003evt.emulation_required);\n \n+\t/\n+\t * Event data is strictly required only for #PF, #DB, and #NM (due to\n+\t * extended feature disable) hardware exceptions under FRED, making it\n+\t * straightforward to correctly populate for just these vectors.\n+\t \n+\t Conversely, trying to guarantee that event data is cleanly managed or\n+\t * zeroed out across all other exception paths is highly complicated\n+\t * and error-prone. A blanket fallback to 0 handles those cases safely.\n+\t \n+\t Therefore, set INJECTED_EVENT_DATA only if all the following are true:\n+\t * - Guest has FRED enabled.\n+\t * - Exception type is a hardware exception.\n+\t * - Exception vector is #PF, #DB, or #NM.\n+\t \n+\t Otherwise, clear it to 0 to prevent stale data leakage.\n+\t /\n \tif (kvm_exception_is_soft(ex-\u003evector)) {\n \t\tvmcs_write32(VM_ENTRY_INSTRUCTION_LEN,\n \t\t\t vmx-\u003evcpu.arch.event_exit_inst_len);\n \t\tintr_info \|= INTR_TYPE_SOFT_EXCEPTION;\n-\t} else\n+\t\tif (is_fred_enabled(vcpu))\n+\t\t\tvmcs_write64(INJECTED_EVENT_DATA, 0);\n+\t} else {\n \t\tintr_info \|= INTR_TYPE_HARD_EXCEPTION;\n+\t\tif (is_fred_enabled(vcpu)) {\n+\t\t\tif (ex-\u003eis_nested)\n+\t\t\t\tintr_info \|= INTR_INFO_NESTED_EXCEPTION_MASK;\n+\t\t\tif (ex-\u003evector == PF_VECTOR \|\|\n+\t\t\t ex-\u003evector == DB_VECTOR \|\|\n+\t\t\t ex-\u003evector == NM_VECTOR)\n+\t\t\t\tvmcs_write64(INJECTED_EVENT_DATA, ex-\u003eevent_data);\n+\t\t\telse\n+\t\t\t\tvmcs_write64(INJECTED_EVENT_DATA, 0);\n+\t\t}\n+\t}\n \n \tvmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr_info);\n \n@@ -2121,6 +2184,27 @@ int vmx_get_feature_msr(u32 msr, u64 data)\n \t}\n }\n \n+#ifdef CONFIG_X86_64\n+static const u32 fred_msr_vmcs_fields[] = {\n+\tGUEST_IA32_FRED_RSP1,\n+\tGUEST_IA32_FRED_RSP2,\n+\tGUEST_IA32_FRED_RSP3,\n+\tGUEST_IA32_FRED_STKLVLS,\n+\tGUEST_IA32_FRED_SSP1,\n+\tGUEST_IA32_FRED_SSP2,\n+\tGUEST_IA32_FRED_SSP3,\n+\tGUEST_IA32_FRED_CONFIG,\n+};\n+\n+static_assert(MSR_IA32_FRED_CONFIG - MSR_IA32_FRED_RSP1 ==\n+\t ARRAY_SIZE(fred_msr_vmcs_fields) - 1);\n+\n+static u32 fred_msr_to_vmcs(u32 msr)\n+{\n+\treturn fred_msr_vmcs_fields[msr - MSR_IA32_FRED_RSP1];\n+}\n+#endif\n+\n /\n Reads an msr value (of 'msr_info-\u003eindex') into 'msr_info-\u003edata'.\n * Returns 0 on success, non-0 otherwise.\n@@ -2143,6 +2227,12 @@ int vmx_get_msr(struct kvm_vcpu vcpu, struct msr_data msr_info)\n \tcase MSR_KERNEL_GS_BASE:\n \t\tmsr_info-\u003edata = vmx_read_guest_kernel_gs_base(vmx);\n \t\tbreak;\n+\tcase MSR_IA32_FRED_RSP0:\n+\t\tmsr_info-\u003edata = vmx_read_guest_fred_rsp0(vmx);\n+\t\tbreak;\n+\tcase MSR_IA32_FRED_RSP1 ... MSR_IA32_FRED_CONFIG:\n+\t\tmsr_info-\u003edata = vmcs_read64(fred_msr_to_vmcs(msr_info-\u003eindex));\n+\t\tbreak;\n #endif\n \tcase MSR_EFER:\n \t\treturn kvm_get_msr_common(vcpu, msr_info);\n@@ -2375,6 +2465,12 @@ int vmx_set_msr(struct kvm_vcpu vcpu, struct msr_data msr_info)\n \t\t\tvmx_update_exception_bitmap(vcpu);\n \t\t}\n \t\tbreak;\n+\tcase MSR_IA32_FRED_RSP0:\n+\t\tvmx_write_guest_fred_rsp0(vmx, data);\n+\t\tbreak;\n+\tcase MSR_IA32_FRED_RSP1 ... MSR_IA32_FRED_CONFIG:\n+\t\tvmcs_write64(fred_msr_to_vmcs(msr_index), data);\n+\t\tbreak;\n #endif\n \tcase MSR_IA32_SYSENTER_CS:\n \t\tif (is_guest_mode(vcpu))\n@@ -2751,8 +2847,9 @@ static int setup_vmcs_config(struct vmcs_config vmcs_conf,\n \tu32 _cpu_based_exec_control = 0;\n \tu32 _cpu_based_2nd_exec_control = 0;\n \tu64 _cpu_based_3rd_exec_control = 0;\n-\tu32 _vmexit_control = 0;\n \tu32 _vmentry_control = 0;\n+\tu32 _vmexit_control = 0;\n+\tu64 _vmexit2_control = 0;\n \tu64 basic_msr;\n \tu64 misc_msr;\n \n@@ -2773,6 +2870,14 @@ static int setup_vmcs_config(struct vmcs_config vmcs_conf,\n \t\t{ VM_ENTRY_LOAD_CET_STATE,\t\tVM_EXIT_LOAD_CET_STATE },\n \t};\n \n+\tstruct {\n+\t\tu32 entry_control;\n+\t\tu64 exit_control;\n+\t} const vmcs_entry_exit2_pairs[] = {\n+\t\t{ VM_ENTRY_LOAD_IA32_FRED,\n+\t\t\tSECONDARY_VM_EXIT_SAVE_IA32_FRED \| SECONDARY_VM_EXIT_LOAD_IA32_FRED },\n+\t};\n+\n \tmemset(vmcs_conf, 0, sizeof(vmcs_conf));\n \n \tif (adjust_vmx_controls(KVM_REQUIRED_VMX_CPU_BASED_VM_EXEC_CONTROL,\n@@ -2870,10 +2975,19 @@ static int setup_vmcs_config(struct vmcs_config vmcs_conf,\n \t\t\t\t\u0026_vmentry_control))\n \t\treturn -EIO;\n \n+\tif (_vmexit_control \u0026 VM_EXIT_ACTIVATE_SECONDARY_CONTROLS)\n+\t\t_vmexit2_control =\n+\t\t\tadjust_vmx_controls64(KVM_OPTIONAL_VMX_SECONDARY_VM_EXIT_CONTROLS,\n+\t\t\t\t\t MSR_IA32_VMX_EXIT_CTLS2);\n+\n \tif (vmx_check_entry_exit_pairs(vmcs_entry_exit_pairs,\n \t\t\t\t _vmentry_control, _vmexit_control))\n \t\treturn -EIO;\n \n+\tif (vmx_check_entry_exit_pairs(vmcs_entry_exit2_pairs,\n+\t\t\t\t _vmentry_control, _vmexit2_control))\n+\t\treturn -EIO;\n+\n \t/\n \t Some cpus support VM_{ENTRY,EXIT}_IA32_PERF_GLOBAL_CTRL but they\n \t * can't be used due to an errata where VM Exit may incorrectly clear\n@@ -2922,8 +3036,9 @@ static int setup_vmcs_config(struct vmcs_config vmcs_conf,\n \tvmcs_conf-\u003ecpu_based_exec_ctrl = _cpu_based_exec_control;\n \tvmcs_conf-\u003ecpu_based_2nd_exec_ctrl = _cpu_based_2nd_exec_control;\n \tvmcs_conf-\u003ecpu_based_3rd_exec_ctrl = _cpu_based_3rd_exec_control;\n-\tvmcs_conf-\u003evmexit_ctrl = _vmexit_control;\n \tvmcs_conf-\u003evmentry_ctrl = _vmentry_control;\n+\tvmcs_conf-\u003evmexit_ctrl = _vmexit_control;\n+\tvmcs_conf-\u003evmexit_2nd_ctrl = _vmexit2_control;\n \tvmcs_conf-\u003emisc\t= misc_msr;\n \n #if IS_ENABLED(CONFIG_HYPERV)\n@@ -4279,6 +4394,72 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu vcpu)\n \t\t\t\t MSR_TYPE_RW, intercept);\n }\n \n+static void vmx_set_intercept_for_fred_msr(struct kvm_vcpu vcpu)\n+{\n+\tbool intercept = !guest_cpu_cap_has(vcpu, X86_FEATURE_FRED);\n+\n+\tif (!kvm_cpu_cap_has(X86_FEATURE_FRED))\n+\t\treturn;\n+\n+\t/\n+\t * Because the following eight FRED MSRs,\n+\t * \tMSR_IA32_FRED_RSP[123], MSR_IA32_FRED_STKLVLS,\n+\t * \tMSR_IA32_FRED_SSP[123], MSR_IA32_FRED_CONFIG,\n+\t * are used by the kernel itself to take an exception at any time, they\n+\t * should be context-switched by Intel VT-x automatically in order to\n+\t * preserve the FRED architectural invariant that there should NEVER be\n+\t * a \"gap\" during which it is unsafe to take an exception.\n+\t \n+\t KVM leverages Intel VT-x hardware to automatically context switch the\n+\t * eight FRED MSRs using:\n+\t \n+\t 1) Dedicated host and guest VMCS fields for each MSR.\n+\t \n+\t 2) VM-entry/exit controls to manage the automated loading and saving\n+\t * of the eight FRED MSRs.\n+\t \n+\t Consequently, passing these MSRs through to the guest would only add\n+\t * unnecessary handling code without benefit.\n+\t /\n+\tvmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP1, MSR_TYPE_RW, intercept);\n+\tvmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP2, MSR_TYPE_RW, intercept);\n+\tvmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP3, MSR_TYPE_RW, intercept);\n+\tvmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_STKLVLS, MSR_TYPE_RW, intercept);\n+\tvmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP1, MSR_TYPE_RW, intercept);\n+\tvmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP2, MSR_TYPE_RW, intercept);\n+\tvmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP3, MSR_TYPE_RW, intercept);\n+\tvmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_CONFIG, MSR_TYPE_RW, intercept);\n+\n+\t/\n+\t * MSR_IA32_FRED_RSP0 and MSR_IA32_PL0_SSP (aka MSR_IA32_FRED_SSP0) are\n+\t * designed for event delivery while executing in userspace. Since KVM\n+\t * operates entirely in kernel mode (CPL is always 0 after any VM exit),\n+\t * it can safely retain and operate with guest-defined values for these\n+\t * MSRs.\n+\t \n+\t Disabling interception of the two MSRs offers two advantages:\n+\t * 1) Simplicity: Eliminates dedicated MSR handling code.\n+\t * 2) Performance: Avoids frequent VM-exits since the two MSRs are\n+\t * per user thread variables and frequently accessed.\n+\t \n+\t MSR_IA32_PL0_SSP (aka MSR_IA32_FRED_SSP0) is part of CET supervisor\n+\t * state, but all four FRED SSP MSRs are architecturally visible on any\n+\t * processor that enumerates FRED. Even if CET is absent, these MSRs\n+\t * remain accessible via RDMSR/WRMSR, though FRED transitions will not\n+\t * use them.\n+\t \n+\t Intercept MSR_IA32_PL0_SSP if CET shadow stacks are unsupported (even\n+\t * with FRED present). Since this MSR is rarely accessed and ignored by\n+\t * XSAVES in this configuration, interception avoids the overhead of\n+\t * manually context switching the hardware MSR during vcpu_load/put.\n+\t \n+\t This behavior is consistent with the current setup in\n+\t * vmx_recalc_msr_intercepts(), so no change is needed to the interception\n+\t * logic for MSR_IA32_PL0_SSP.\n+\t /\n+\tvmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP0, MSR_TYPE_RW, intercept);\n+}\n+\n static void vmx_recalc_msr_intercepts(struct kvm_vcpu vcpu)\n {\n \tbool intercept;\n@@ -4346,6 +4527,7 @@ static void vmx_recalc_msr_intercepts(struct kvm_vcpu vcpu)\n \t}\n \n \tvmx_recalc_pmu_msr_intercepts(vcpu);\n+\tvmx_set_intercept_for_fred_msr(vcpu);\n \n \t/\n \t * x2APIC and LBR MSR intercepts are modified on-demand and cannot be\n@@ -4481,6 +4663,17 @@ void vmx_set_constant_host_state(struct vcpu_vmx vmx)\n \t /\n \tvmcs_write16(HOST_DS_SELECTOR, 0);\n \tvmcs_write16(HOST_ES_SELECTOR, 0);\n+\n+\tif (cpu_feature_enabled(X86_FEATURE_FRED) \u0026\u0026 kvm_cpu_cap_has(X86_FEATURE_FRED)) {\n+\t\t/* FRED CONFIG and STKLVLS are the same on all CPUs /\n+\t\tvmcs_write64(HOST_IA32_FRED_CONFIG, kvm_host.fred_config);\n+\t\tvmcs_write64(HOST_IA32_FRED_STKLVLS, kvm_host.fred_stklvls);\n+\n+\t\t/ Linux doesn't support kernel shadow stacks, thus SSPs are 0s /\n+\t\tvmcs_write64(HOST_IA32_FRED_SSP1, 0);\n+\t\tvmcs_write64(HOST_IA32_FRED_SSP2, 0);\n+\t\tvmcs_write64(HOST_IA32_FRED_SSP3, 0);\n+\t}\n #else\n \tvmcs_write16(HOST_DS_SELECTOR, __KERNEL_DS); / 22.2.4 /\n \tvmcs_write16(HOST_ES_SELECTOR, __KERNEL_DS); / 22.2.4 /\n@@ -4617,6 +4810,11 @@ static u32 vmx_get_initial_vmexit_ctrl(void)\n \t\t VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL);\n }\n \n+static u64 vmx_secondary_vmexit_ctrl(void)\n+{\n+\treturn vmcs_config.vmexit_2nd_ctrl;\n+}\n+\n void vmx_refresh_apicv_exec_ctrl(struct kvm_vcpu vcpu)\n {\n \tstruct vcpu_vmx vmx = to_vmx(vcpu);\n@@ -4960,6 +5158,9 @@ static void init_vmcs(struct vcpu_vmx vmx)\n \n \tvm_exit_controls_set(vmx, vmx_get_initial_vmexit_ctrl());\n \n+\tif (cpu_has_secondary_vmexit_ctrls())\n+\t\tsecondary_vm_exit_controls_set(vmx, vmx_secondary_vmexit_ctrl());\n+\n \t/* 22.2.1, 20.8.1 /\n \tvm_entry_controls_set(vmx, vmx_get_initial_vmentry_ctrl());\n \n@@ -5003,6 +5204,17 @@ static void init_vmcs(struct vcpu_vmx vmx)\n \t}\n \n \tvmx_setup_uret_msrs(vmx);\n+\n+\tif (cpu_feature_enabled(X86_FEATURE_FRED) \u0026\u0026 kvm_cpu_cap_has(X86_FEATURE_FRED)) {\n+\t\tvmcs_write64(GUEST_IA32_FRED_CONFIG, 0);\n+\t\tvmcs_write64(GUEST_IA32_FRED_RSP1, 0);\n+\t\tvmcs_write64(GUEST_IA32_FRED_RSP2, 0);\n+\t\tvmcs_write64(GUEST_IA32_FRED_RSP3, 0);\n+\t\tvmcs_write64(GUEST_IA32_FRED_STKLVLS, 0);\n+\t\tvmcs_write64(GUEST_IA32_FRED_SSP1, 0);\n+\t\tvmcs_write64(GUEST_IA32_FRED_SSP2, 0);\n+\t\tvmcs_write64(GUEST_IA32_FRED_SSP3, 0);\n+\t}\n }\n \n static void __vmx_vcpu_reset(struct kvm_vcpu vcpu)\n@@ -5143,6 +5355,14 @@ void vmx_inject_irq(struct kvm_vcpu vcpu, bool reinjected)\n \t\t\t vmx-\u003evcpu.arch.event_exit_inst_len);\n \t} else\n \t\tintr \|= INTR_TYPE_EXT_INTR;\n+\n+\t/\n+\t Interrupts define no event data, so clear any value left behind\n+\t * by a previously injected exception.\n+\t /\n+\tif (is_fred_enabled(vcpu))\n+\t\tvmcs_write64(INJECTED_EVENT_DATA, 0);\n+\n \tvmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr);\n \n \tvmx_clear_hlt(vcpu);\n@@ -5173,6 +5393,13 @@ void vmx_inject_nmi(struct kvm_vcpu vcpu)\n \t\treturn;\n \t}\n \n+\t/\n+\t KVM doesn't virtualize the NMI-source reporting feature. So clear\n+\t * any value left behind by a previously injected exception.\n+\t /\n+\tif (is_fred_enabled(vcpu))\n+\t\tvmcs_write64(INJECTED_EVENT_DATA, 0);\n+\n \tvmcs_write32(VM_ENTRY_INTR_INFO_FIELD,\n \t\t\tINTR_TYPE_NMI_INTR \| INTR_INFO_VALID_MASK \| NMI_VECTOR);\n \n@@ -6503,7 +6730,7 @@ void dump_vmcs(struct kvm_vcpu vcpu)\n \tstruct vcpu_vmx vmx = to_vmx(vcpu);\n \tu32 vmentry_ctl, vmexit_ctl;\n \tu32 cpu_based_exec_ctrl, pin_based_exec_ctrl, secondary_exec_control;\n-\tu64 tertiary_exec_control;\n+\tu64 tertiary_exec_control, secondary_vmexit_ctl;\n \tunsigned long cr4;\n \tint efer_slot;\n \n@@ -6514,6 +6741,8 @@ void dump_vmcs(struct kvm_vcpu vcpu)\n \n \tvmentry_ctl = vmcs_read32(VM_ENTRY_CONTROLS);\n \tvmexit_ctl = vmcs_read32(VM_EXIT_CONTROLS);\n+\tsecondary_vmexit_ctl = cpu_has_secondary_vmexit_ctrls() ?\n+\t\t\t vmcs_read64(SECONDARY_VM_EXIT_CONTROLS) : 0;\n \tcpu_based_exec_ctrl = vmcs_read32(CPU_BASED_VM_EXEC_CONTROL);\n \tpin_based_exec_ctrl = vmcs_read32(PIN_BASED_VM_EXEC_CONTROL);\n \tcr4 = vmcs_readl(GUEST_CR4);\n@@ -6560,6 +6789,16 @@ void dump_vmcs(struct kvm_vcpu vcpu)\n \tvmx_dump_sel(\"LDTR:\", GUEST_LDTR_SELECTOR);\n \tvmx_dump_dtsel(\"IDTR:\", GUEST_IDTR_LIMIT);\n \tvmx_dump_sel(\"TR: \", GUEST_TR_SELECTOR);\n+\tif (vmentry_ctl \u0026 VM_ENTRY_LOAD_IA32_FRED)\n+\t\tpr_err(\"FRED guest: config=0x%016llx, stack_levels=0x%016llx\\n\"\n+\t\t \"RSP0=0x%016llx, RSP1=0x%016llx\\n\"\n+\t\t \"RSP2=0x%016llx, RSP3=0x%016llx\\n\",\n+\t\t vmcs_read64(GUEST_IA32_FRED_CONFIG),\n+\t\t vmcs_read64(GUEST_IA32_FRED_STKLVLS),\n+\t\t vmx_read_guest_fred_rsp0(vmx),\n+\t\t vmcs_read64(GUEST_IA32_FRED_RSP1),\n+\t\t vmcs_read64(GUEST_IA32_FRED_RSP2),\n+\t\t vmcs_read64(GUEST_IA32_FRED_RSP3));\n \tefer_slot = vmx_find_loadstore_msr_slot(\u0026vmx-\u003emsr_autoload.guest, MSR_EFER);\n \tif (vmentry_ctl \u0026 VM_ENTRY_LOAD_IA32_EFER)\n \t\tpr_err(\"EFER= 0x%016llx\\n\", vmcs_read64(GUEST_IA32_EFER));\n@@ -6611,6 +6850,16 @@ void dump_vmcs(struct kvm_vcpu vcpu)\n \t vmcs_readl(HOST_TR_BASE));\n \tpr_err(\"GDTBase=%016lx IDTBase=%016lx\\n\",\n \t vmcs_readl(HOST_GDTR_BASE), vmcs_readl(HOST_IDTR_BASE));\n+\tif (secondary_vmexit_ctl \u0026 SECONDARY_VM_EXIT_LOAD_IA32_FRED)\n+\t\tpr_err(\"FRED host: config=0x%016llx, stack_levels=0x%016llx\\n\"\n+\t\t \"RSP0=0x%016lx, RSP1=0x%016llx\\n\"\n+\t\t \"RSP2=0x%016llx, RSP3=0x%016llx\\n\",\n+\t\t vmcs_read64(HOST_IA32_FRED_CONFIG),\n+\t\t vmcs_read64(HOST_IA32_FRED_STKLVLS),\n+\t\t (unsigned long)task_stack_page(current) + THREAD_SIZE,\n+\t\t vmcs_read64(HOST_IA32_FRED_RSP1),\n+\t\t vmcs_read64(HOST_IA32_FRED_RSP2),\n+\t\t vmcs_read64(HOST_IA32_FRED_RSP3));\n \tpr_err(\"CR0=%016lx CR3=%016lx CR4=%016lx\\n\",\n \t vmcs_readl(HOST_CR0), vmcs_readl(HOST_CR3),\n \t vmcs_readl(HOST_CR4));\n@@ -6636,25 +6885,29 @@ void dump_vmcs(struct kvm_vcpu vcpu)\n \tpr_err(\" Control State \\n\");\n \tpr_err(\"CPUBased=0x%08x SecondaryExec=0x%08x TertiaryExec=0x%016llx\\n\",\n \t cpu_based_exec_ctrl, secondary_exec_control, tertiary_exec_control);\n-\tpr_err(\"PinBased=0x%08x EntryControls=%08x ExitControls=%08x\\n\",\n-\t pin_based_exec_ctrl, vmentry_ctl, vmexit_ctl);\n+\tpr_err(\"PinBased=0x%08x EntryControls=0x%08x\\n\",\n+\t pin_based_exec_ctrl, vmentry_ctl);\n+\tpr_err(\"ExitControls=0x%08x SecondaryExitControls=0x%016llx\\n\",\n+\t vmexit_ctl, secondary_vmexit_ctl);\n \tpr_err(\"ExceptionBitmap=%08x PFECmask=%08x PFECmatch=%08x\\n\",\n \t vmcs_read32(EXCEPTION_BITMAP),\n \t vmcs_read32(PAGE_FAULT_ERROR_CODE_MASK),\n \t vmcs_read32(PAGE_FAULT_ERROR_CODE_MATCH));\n-\tpr_err(\"VMEntry: intr_info=%08x errcode=%08x ilen=%08x\\n\",\n+\tpr_err(\"VMEntry: intr_info=%08x errcode=%08x ilen=%08x event_data=%016llx\\n\",\n \t vmcs_read32(VM_ENTRY_INTR_INFO_FIELD),\n \t vmcs_read32(VM_ENTRY_EXCEPTION_ERROR_CODE),\n-\t vmcs_read32(VM_ENTRY_INSTRUCTION_LEN));\n+\t vmcs_read32(VM_ENTRY_INSTRUCTION_LEN),\n+\t kvm_cpu_cap_has(X86_FEATURE_FRED) ? vmcs_read64(INJECTED_EVENT_DATA) : 0);\n \tpr_err(\"VMExit: intr_info=%08x errcode=%08x ilen=%08x\\n\",\n \t vmcs_read32(VM_EXIT_INTR_INFO),\n \t vmcs_read32(VM_EXIT_INTR_ERROR_CODE),\n \t vmcs_read32(VM_EXIT_INSTRUCTION_LEN));\n \tpr_err(\" reason=%08x qualification=%016lx\\n\",\n \t vmcs_read32(VM_EXIT_REASON), vmcs_readl(EXIT_QUALIFICATION));\n-\tpr_err(\"IDTVectoring: info=%08x errcode=%08x\\n\",\n+\tpr_err(\"IDTVectoring: info=%08x errcode=%08x event_data=%016llx\\n\",\n \t vmcs_read32(IDT_VECTORING_INFO_FIELD),\n-\t vmcs_read32(IDT_VECTORING_ERROR_CODE));\n+\t vmcs_read32(IDT_VECTORING_ERROR_CODE),\n+\t kvm_cpu_cap_has(X86_FEATURE_FRED) ? vmcs_read64(ORIGINAL_EVENT_DATA) : 0);\n \tpr_err(\"TSC Offset = 0x%016llx\\n\", vmcs_read64(TSC_OFFSET));\n \tif (secondary_exec_control \u0026 SECONDARY_EXEC_TSC_SCALING)\n \t\tpr_err(\"TSC Multiplier = 0x%016llx\\n\",\n@@ -7280,7 +7533,8 @@ static void vmx_recover_nmi_blocking(struct vcpu_vmx vmx)\n static void __vmx_complete_interrupts(struct kvm_vcpu vcpu,\n \t\t\t\t u32 idt_vectoring_info,\n \t\t\t\t int instr_len_field,\n-\t\t\t\t int error_code_field)\n+\t\t\t\t int error_code_field,\n+\t\t\t\t int event_data_field)\n {\n \tu8 vector;\n \tint type;\n@@ -7315,13 +7569,18 @@ static void __vmx_complete_interrupts(struct kvm_vcpu vcpu,\n \t\tfallthrough;\n \tcase INTR_TYPE_HARD_EXCEPTION: {\n \t\tu32 error_code = 0;\n+\t\tu64 event_data = 0;\n \n \t\tif (idt_vectoring_info \u0026 VECTORING_INFO_DELIVER_CODE_MASK)\n \t\t\terror_code = vmcs_read32(error_code_field);\n+\t\tif (is_fred_enabled(vcpu))\n+\t\t\tevent_data = vmcs_read64(event_data_field);\n \n \t\tkvm_requeue_exception(vcpu, vector,\n \t\t\t\t idt_vectoring_info \u0026 VECTORING_INFO_DELIVER_CODE_MASK,\n-\t\t\t\t error_code);\n+\t\t\t\t error_code,\n+\t\t\t\t idt_vectoring_info \u0026 VECTORING_INFO_NESTED_EXCEPTION_MASK,\n+\t\t\t\t event_data);\n \t\tbreak;\n \t}\n \tcase INTR_TYPE_SOFT_INTR:\n@@ -7339,7 +7598,8 @@ static void vmx_complete_interrupts(struct vcpu_vmx vmx)\n {\n \t__vmx_complete_interrupts(\u0026vmx-\u003evcpu, vmx-\u003eidt_vectoring_info,\n \t\t\t\t VM_EXIT_INSTRUCTION_LEN,\n-\t\t\t\t IDT_VECTORING_ERROR_CODE);\n+\t\t\t\t IDT_VECTORING_ERROR_CODE,\n+\t\t\t\t ORIGINAL_EVENT_DATA);\n }\n \n void vmx_cancel_injection(struct kvm_vcpu vcpu)\n@@ -7347,7 +7607,8 @@ void vmx_cancel_injection(struct kvm_vcpu vcpu)\n \t__vmx_complete_interrupts(vcpu,\n \t\t\t\t vmcs_read32(VM_ENTRY_INTR_INFO_FIELD),\n \t\t\t\t VM_ENTRY_INSTRUCTION_LEN,\n-\t\t\t\t VM_ENTRY_EXCEPTION_ERROR_CODE);\n+\t\t\t\t VM_ENTRY_EXCEPTION_ERROR_CODE,\n+\t\t\t\t INJECTED_EVENT_DATA);\n \n \tvmcs_write32(VM_ENTRY_INTR_INFO_FIELD, 0);\n }\n@@ -7459,6 +7720,10 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu vcpu,\n \n \tvmx_disable_fb_clear(vmx);\n \n+\t/\n+\t Note, even though FRED delivers the faulting linear address via the\n+\t * event data field on the stack, CR2 is still updated.\n+\t /\n \tif (vcpu-\u003earch.cr2 != native_read_cr2())\n \t\tnative_write_cr2(vcpu-\u003earch.cr2);\n \n@@ -7885,6 +8150,7 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu vcpu)\n \n \tentry = kvm_find_cpuid_entry_index(vcpu, 0x7, 1);\n \tcr4_fixed1_update(X86_CR4_LAM_SUP, eax, feature_bit(LAM));\n+\tcr4_fixed1_update(X86_CR4_FRED, eax, feature_bit(FRED));\n \n #undef cr4_fixed1_update\n }\n@@ -8095,6 +8361,9 @@ static __init void vmx_set_cpu_caps(void)\n \t\tkvm_cpu_cap_check_and_set(X86_FEATURE_DTES64);\n \t}\n \n+\tif (!cpu_has_vmx_fred())\n+\t\tkvm_cpu_cap_clear(X86_FEATURE_FRED);\n+\n \tif (!enable_pmu)\n \t\tkvm_cpu_cap_clear(X86_FEATURE_PDCM);\n \tkvm_caps.supported_perf_cap = vmx_get_perf_capabilities();\n@@ -8853,6 +9122,11 @@ __init int vmx_hardware_setup(void)\n \n \tkvm_caps.inapplicable_quirks \u0026= ~KVM_X86_QUIRK_IGNORE_GUEST_PAT;\n \n+\tif (cpu_feature_enabled(X86_FEATURE_FRED) \u0026\u0026 kvm_cpu_cap_has(X86_FEATURE_FRED)) {\n+\t\trdmsrq(MSR_IA32_FRED_CONFIG, kvm_host.fred_config);\n+\t\trdmsrq(MSR_IA32_FRED_STKLVLS, kvm_host.fred_stklvls);\n+\t}\n+\n \treturn 0;\n }\n \ndiff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h\nindex dc8517f15bc46..cc1e4038cd441 100644\n--- a/arch/x86/kvm/vmx/vmx.h\n+++ b/arch/x86/kvm/vmx/vmx.h\n@@ -182,6 +182,29 @@ struct nested_vmx {\n \tu64 pre_vmenter_ssp;\n \tu64 pre_vmenter_ssp_tbl;\n \n+\t/\n+\t Used to snapshot FRED MSRs that may NOT be saved to vmcs12 as specified\n+\t * in the VM-Exit controls of vmcs12 configured by L1 VMM.\n+\t \n+\t FRED MSRs are always saved into vmcs02 because KVM always sets\n+\t * SECONDARY_VM_EXIT_SAVE_IA32_FRED. However an L1 VMM may choose to clear\n+\t * this bit, resulting in FRED MSRs not being propagated to vmcs12 from\n+\t * vmcs02. When the L1 VMM sets SECONDARY_VM_EXIT_LOAD_IA32_FRED, this is\n+\t * not a problem, since KVM then immediately loads the host FRED MSRs of\n+\t * vmcs12 to the guest FRED MSRs of vmcs01.\n+\t \n+\t But if the L1 VMM clears SECONDARY_VM_EXIT_LOAD_IA32_FRED, KVM should\n+\t * retain the FRED MSRs, i.e., propagate the guest FRED MSRs of vmcs02 to\n+\t * the guest FRED MSRs of vmcs01.\n+\t \n+\t This structure stores guest FRED MSRs that an L1 VMM opts not to save\n+\t * during VM-Exits from L2 to L1. These MSRs may still be retained for\n+\t * running the L1 VMM if SECONDARY_VM_EXIT_LOAD_IA32_FRED is cleared in\n+\t * vmcs12.\n+\t /\n+\tstruct vmcs_fred_msrs pre_vmenter_fred_msrs;\n+\tstruct vmcs_fred_msrs at_vmexit_fred_msrs;\n+\n \tu16 vpid02;\n \tu16 last_vpid;\n \n@@ -222,6 +245,7 @@ struct vcpu_vmx {\n \tbool guest_uret_msrs_loaded;\n #ifdef CONFIG_X86_64\n \tu64\t\t msr_guest_kernel_gs_base;\n+\tu64\t\t msr_guest_fred_rsp0;\n #endif\n \n \tu64\t\t spec_ctrl;\n@@ -477,7 +501,8 @@ static inline u8 vmx_get_rvi(void)\n \t VM_ENTRY_LOAD_BNDCFGS \|\t\t\t\t\t\\\n \t VM_ENTRY_PT_CONCEAL_PIP \|\t\t\t\t\t\\\n \t VM_ENTRY_LOAD_IA32_RTIT_CTL \|\t\t\t\t\t\\\n-\t VM_ENTRY_LOAD_CET_STATE)\n+\t VM_ENTRY_LOAD_CET_STATE \|\t\t\t\t\t\\\n+\t VM_ENTRY_LOAD_IA32_FRED)\n \n #define __KVM_REQUIRED_VMX_VM_EXIT_CONTROLS\t\t\t\t\\\n \t(VM_EXIT_SAVE_DEBUG_CONTROLS \|\t\t\t\t\t\\\n@@ -501,7 +526,13 @@ static inline u8 vmx_get_rvi(void)\n \t VM_EXIT_PT_CONCEAL_PIP \|\t\t\t\t\t\\\n \t VM_EXIT_CLEAR_IA32_RTIT_CTL \|\t\t\t\t\\\n \t VM_EXIT_LOAD_CET_STATE \|\t\t\t\t\t\\\n-\t VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL)\n+\t VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL \|\t\t\t\\\n+\t VM_EXIT_ACTIVATE_SECONDARY_CONTROLS)\n+\n+#define KVM_REQUIRED_VMX_SECONDARY_VM_EXIT_CONTROLS (0)\n+#define KVM_OPTIONAL_VMX_SECONDARY_VM_EXIT_CONTROLS\t\t\t\\\n+\t (SECONDARY_VM_EXIT_SAVE_IA32_FRED \|\t\t\t\\\n+\t SECONDARY_VM_EXIT_LOAD_IA32_FRED)\n \n #define KVM_REQUIRED_VMX_PIN_BASED_VM_EXEC_CONTROL\t\t\t\\\n \t(PIN_BASED_EXT_INTR_MASK \|\t\t\t\t\t\\\n@@ -615,6 +646,7 @@ static __always_inline void lname##_controls_changebit(struct vcpu_vmx vmx, u##\n }\n BUILD_CONTROLS_SHADOW(vm_entry, VM_ENTRY_CONTROLS, 32)\n BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS, 32)\n+BUILD_CONTROLS_SHADOW(secondary_vm_exit, SECONDARY_VM_EXIT_CONTROLS, 64)\n BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL, 32)\n BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL, 32)\n BUILD_CONTROLS_SHADOW(secondary_exec, SECONDARY_VM_EXEC_CONTROL, 32)\ndiff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c\nindex 4b3681796c752..f264bc9c35f9d 100644\n--- a/arch/x86/kvm/x86.c\n+++ b/arch/x86/kvm/x86.c\n@@ -431,9 +431,22 @@ void kvm_deliver_exception_payload(struct kvm_vcpu vcpu,\n \t\t breakpoint), it is reserved and must be zero in DR6.\n \t\t /\n \t\tvcpu-\u003earch.dr6 \u0026= ~BIT(12);\n+\n+\t\t/\n+\t\t * FRED #DB event data matches DR6, but follows the polarity of\n+\t\t * VMX's pending debug exceptions, not DR6.\n+\t\t /\n+\t\tex-\u003eevent_data = ex-\u003epayload \u0026 ~BIT(12);\n+\t\tbreak;\n+\tcase NM_VECTOR:\n+\t\tex-\u003eevent_data = ex-\u003epayload;\n \t\tbreak;\n \tcase PF_VECTOR:\n \t\tvcpu-\u003earch.cr2 = ex-\u003epayload;\n+\t\tex-\u003eevent_data = ex-\u003epayload;\n+\t\tbreak;\n+\tdefault:\n+\t\tex-\u003eevent_data = 0;\n \t\tbreak;\n \t}\n \n@@ -461,6 +474,7 @@ static void kvm_multiple_exception(struct kvm_vcpu vcpu, unsigned int nr,\n \t\t\t\t bool has_error, u32 error_code,\n \t\t\t\t bool has_payload, unsigned long payload)\n {\n+\tbool is_nested = false;\n \tu32 prev_nr;\n \tint class1, class2;\n \n@@ -487,6 +501,15 @@ static void kvm_multiple_exception(struct kvm_vcpu vcpu, unsigned int nr,\n \t\tvcpu-\u003earch.exception.error_code = error_code;\n \t\tvcpu-\u003earch.exception.has_payload = has_payload;\n \t\tvcpu-\u003earch.exception.payload = payload;\n+\t\t/ #DF is NOT a nested event, per its definition. /\n+\t\tvcpu-\u003earch.exception.is_nested = (nr != DF_VECTOR) \u0026\u0026\n+\t\t\t\t\t\t (is_nested \|\|\n+\t\t\t\t\t\t vcpu-\u003earch.nmi_injected \|\|\n+\t\t\t\t\t\t vcpu-\u003earch.interrupt.injected);\n+\t\t/\n+\t\t * Do not update the event data, as the current value may have\n+\t\t * just been restored during a VM save/restore or live migration.\n+\t\t /\n \t\treturn;\n \t}\n \n@@ -510,6 +533,9 @@ static void kvm_multiple_exception(struct kvm_vcpu vcpu, unsigned int nr,\n \n \t\tkvm_queue_exception_e(vcpu, DF_VECTOR, 0);\n \t} else {\n+\t\t/* The new exception arrived while delivering the previous one. /\n+\t\tis_nested = true;\n+\n \t\t/ replace previous exception with a new one in a hope\n \t\t that instruction re-execution will regenerate lost\n \t\t exception /\n@@ -538,7 +564,8 @@ static void kvm_queue_exception_e_p(struct kvm_vcpu vcpu, unsigned nr,\n }\n \n void kvm_requeue_exception(struct kvm_vcpu vcpu, unsigned int nr,\n-\t\t\t bool has_error_code, u32 error_code)\n+\t\t\t bool has_error_code, u32 error_code,\n+\t\t\t bool is_nested, u64 event_data)\n {\n \n \t/\n@@ -563,6 +590,8 @@ void kvm_requeue_exception(struct kvm_vcpu vcpu, unsigned int nr,\n \tvcpu-\u003earch.exception.error_code = error_code;\n \tvcpu-\u003earch.exception.has_payload = false;\n \tvcpu-\u003earch.exception.payload = 0;\n+\tvcpu-\u003earch.exception.is_nested = is_nested;\n+\tvcpu-\u003earch.exception.event_data = event_data;\n }\n EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_requeue_exception);\n \n@@ -2288,6 +2317,9 @@ int kvm_vm_ioctl_check_extension(struct kvm kvm, long ext)\n \tcase KVM_CAP_PRE_FAULT_MEMORY:\n \t\tr = tdp_enabled;\n \t\tbreak;\n+\tcase KVM_CAP_X86_FRED_EVENT:\n+\t\tr = kvm_cpu_cap_has(X86_FEATURE_FRED);\n+\t\tbreak;\n \tcase KVM_CAP_X86_APIC_BUS_CYCLES_NS:\n \t\tr = kvm ? kvm-\u003earch.apic_bus_cycle_ns : APIC_BUS_CYCLE_NS_DEFAULT;\n \t\tbreak;\n@@ -2997,6 +3029,8 @@ static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu vcpu,\n \tevents-\u003eexception.error_code = ex-\u003eerror_code;\n \tevents-\u003eexception_has_payload = ex-\u003ehas_payload;\n \tevents-\u003eexception_payload = ex-\u003epayload;\n+\tevents-\u003eexception_is_nested = ex-\u003eis_nested;\n+\tevents-\u003eexception_event_data = ex-\u003eevent_data;\n \n \tevents-\u003einterrupt.injected =\n \t\tvcpu-\u003earch.interrupt.injected \u0026\u0026 !vcpu-\u003earch.interrupt.soft;\n@@ -3022,6 +3056,8 @@ static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu vcpu,\n \t\t\t \| KVM_VCPUEVENT_VALID_SMM);\n \tif (vcpu-\u003ekvm-\u003earch.exception_payload_enabled)\n \t\tevents-\u003eflags \|= KVM_VCPUEVENT_VALID_PAYLOAD;\n+\tif (vcpu-\u003ekvm-\u003earch.exception_fred_state_enabled)\n+\t\tevents-\u003eflags \|= KVM_VCPUEVENT_VALID_FRED_STATE;\n \tif (vcpu-\u003ekvm-\u003earch.triple_fault_event) {\n \t\tevents-\u003etriple_fault.pending = kvm_test_request(KVM_REQ_TRIPLE_FAULT, vcpu);\n \t\tevents-\u003eflags \|= KVM_VCPUEVENT_VALID_TRIPLE_FAULT;\n@@ -3036,6 +3072,7 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu vcpu,\n \t\t\t \| KVM_VCPUEVENT_VALID_SHADOW\n \t\t\t \| KVM_VCPUEVENT_VALID_SMM\n \t\t\t \| KVM_VCPUEVENT_VALID_PAYLOAD\n+\t\t\t \| KVM_VCPUEVENT_VALID_FRED_STATE\n \t\t\t \| KVM_VCPUEVENT_VALID_TRIPLE_FAULT))\n \t\treturn -EINVAL;\n \n@@ -3051,6 +3088,14 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu vcpu,\n \t\tevents-\u003eexception_has_payload = 0;\n \t}\n \n+\tif (events-\u003eflags \u0026 KVM_VCPUEVENT_VALID_FRED_STATE) {\n+\t\tif (!vcpu-\u003ekvm-\u003earch.exception_fred_state_enabled)\n+\t\t\treturn -EINVAL;\n+\t} else {\n+\t\tevents-\u003eexception_is_nested = 0;\n+\t\tevents-\u003eexception_event_data = 0;\n+\t}\n+\n \tif ((events-\u003eexception.injected \|\| events-\u003eexception.pending) \u0026\u0026\n \t (events-\u003eexception.nr \u003e 31 \|\| events-\u003eexception.nr == NMI_VECTOR))\n \t\treturn -EINVAL;\n@@ -3076,6 +3121,8 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu vcpu,\n \tvcpu-\u003earch.exception.error_code = events-\u003eexception.error_code;\n \tvcpu-\u003earch.exception.has_payload = events-\u003eexception_has_payload;\n \tvcpu-\u003earch.exception.payload = events-\u003eexception_payload;\n+\tvcpu-\u003earch.exception.is_nested = events-\u003eexception_is_nested;\n+\tvcpu-\u003earch.exception.event_data = events-\u003eexception_event_data;\n \n \tvcpu-\u003earch.interrupt.injected = events-\u003einterrupt.injected;\n \tvcpu-\u003earch.interrupt.nr = events-\u003einterrupt.nr;\n@@ -4038,6 +4085,13 @@ int kvm_vm_ioctl_enable_cap(struct kvm kvm,\n \t\tkvm-\u003earch.exception_payload_enabled = cap-\u003eargs[0];\n \t\tr = 0;\n \t\tbreak;\n+\tcase KVM_CAP_X86_FRED_EVENT:\n+\t\tr = -EINVAL;\n+\t\tif (!kvm_cpu_cap_has(X86_FEATURE_FRED))\n+\t\t\tbreak;\n+\t\tkvm-\u003earch.exception_fred_state_enabled = cap-\u003eargs[0];\n+\t\tr = 0;\n+\t\tbreak;\n \tcase KVM_CAP_X86_TRIPLE_FAULT_EVENT:\n \t\tkvm-\u003earch.triple_fault_event = cap-\u003eargs[0];\n \t\tr = 0;\n@@ -5498,11 +5552,17 @@ static int emulator_set_dr(struct x86_emulate_ctxt ctxt, int dr,\n \treturn kvm_set_dr(emul_to_vcpu(ctxt), dr, value);\n }\n \n-static u64 mk_cr_64(u64 curr_cr, u32 new_val)\n+static u64 mk_cr0_64(u64 curr_cr, u32 new_val)\n {\n \treturn (curr_cr \u0026 ~((1ULL \u003c\u003c 32) - 1)) \| new_val;\n }\n \n+static u64 mk_cr4_64(struct kvm_vcpu vcpu, u64 curr_cr, u64 new_val)\n+{\n+\tu32 shift = guest_cpu_cap_has(vcpu, X86_FEATURE_FRED) ? 33 : 32;\n+\treturn (curr_cr \u0026 ~((1ULL \u003c\u003c shift) - 1)) \| new_val;\n+}\n+\n static unsigned long emulator_get_cr(struct x86_emulate_ctxt ctxt, int cr)\n {\n \tstruct kvm_vcpu vcpu = emul_to_vcpu(ctxt);\n@@ -5539,7 +5599,7 @@ static int emulator_set_cr(struct x86_emulate_ctxt ctxt, int cr, ulong val)\n \n \tswitch (cr) {\n \tcase 0:\n-\t\tres = kvm_set_cr0(vcpu, mk_cr_64(kvm_read_cr0(vcpu), val));\n+\t\tres = kvm_set_cr0(vcpu, mk_cr0_64(kvm_read_cr0(vcpu), val));\n \t\tbreak;\n \tcase 2:\n \t\tvcpu-\u003earch.cr2 = val;\n@@ -5548,7 +5608,7 @@ static int emulator_set_cr(struct x86_emulate_ctxt ctxt, int cr, ulong val)\n \t\tres = kvm_set_cr3(vcpu, val);\n \t\tbreak;\n \tcase 4:\n-\t\tres = kvm_set_cr4(vcpu, mk_cr_64(kvm_read_cr4(vcpu), val));\n+\t\tres = kvm_set_cr4(vcpu, mk_cr4_64(vcpu, kvm_read_cr4(vcpu), val));\n \t\tbreak;\n \tcase 8:\n \t\tres = kvm_set_cr8(vcpu, val);\n@@ -8960,6 +9020,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu vcpu)\n \t\t\t\t\t ex-\u003ehas_payload, ex-\u003epayload);\n \t\tex-\u003einjected = false;\n \t\tex-\u003epending = false;\n+\t\tex-\u003eis_nested = false;\n \t}\n \tvcpu-\u003earch.exception_from_userspace = false;\n \ndiff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h\nindex 0f5919b092e47..43c5c08f5d5ca 100644\n--- a/arch/x86/kvm/x86.h\n+++ b/arch/x86/kvm/x86.h\n@@ -158,6 +158,8 @@ static inline void kvm_clear_exception_queue(struct kvm_vcpu vcpu)\n {\n \tvcpu-\u003earch.exception.pending = false;\n \tvcpu-\u003earch.exception.injected = false;\n+\tvcpu-\u003earch.exception.is_nested = false;\n+\tvcpu-\u003earch.exception.event_data = 0;\n \tvcpu-\u003earch.exception_vmexit.pending = false;\n }\n \n@@ -502,7 +504,8 @@ void kvm_queue_exception(struct kvm_vcpu vcpu, unsigned nr);\n void kvm_queue_exception_e(struct kvm_vcpu vcpu, unsigned nr, u32 error_code);\n void kvm_queue_exception_p(struct kvm_vcpu vcpu, unsigned nr, unsigned long payload);\n void kvm_requeue_exception(struct kvm_vcpu vcpu, unsigned int nr,\n-\t\t\t bool has_error_code, u32 error_code);\n+\t\t\t bool has_error_code, u32 error_code,\n+\t\t\t bool is_nested, u64 event_data);\n void kvm_inject_page_fault(struct kvm_vcpu vcpu, struct x86_exception fault,\n \t\t\t bool from_hardware);\n void __kvm_inject_emulated_page_fault(struct kvm_vcpu vcpu,\ndiff --git a/arch/x86/mm/cpu_entry_area.c b/arch/x86/mm/cpu_entry_area.c\nindex 575f863f3c75e..a47dfd686bd33 100644\n--- a/arch/x86/mm/cpu_entry_area.c\n+++ b/arch/x86/mm/cpu_entry_area.c\n@@ -18,6 +18,26 @@ static DEFINE_PER_CPU_PAGE_ALIGNED(struct entry_stack_page, entry_stack_storage)\n static DEFINE_PER_CPU_PAGE_ALIGNED(struct exception_stacks, exception_stacks);\n DEFINE_PER_CPU(struct cea_exception_stacks, cea_exception_stacks);\n \n+/\n+ Typically invoked by entry code, so must be noinstr.\n+ /\n+noinstr unsigned long __this_cpu_ist_bottom_va(enum exception_stack_ordering stack)\n+{\n+\tstruct cea_exception_stacks s;\n+\n+\tBUILD_BUG_ON(ESTACK_DF != 0);\n+\tBUG_ON(stack \u003e= N_EXCEPTION_STACKS);\n+\n+\ts = __this_cpu_read(cea_exception_stacks);\n+\n+\treturn (unsigned long)\u0026s-\u003eevent_stacks[stack].stack;\n+}\n+\n+noinstr unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack)\n+{\n+\treturn __this_cpu_ist_bottom_va(stack) + EXCEPTION_STKSZ;\n+}\n+\n static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, _cea_offset);\n \n static __always_inline unsigned int cea_offset(unsigned int cpu)\n@@ -132,7 +152,7 @@ static void __init percpu_setup_debug_store(unsigned int cpu)\n \n #define cea_map_stack(name) do {\t\t\t\t\t\\\n \tnpages = sizeof(estacks-\u003ename## _stack) / PAGE_SIZE;\t\t\\\n-\tcea_map_percpu_pages(cea-\u003eestacks.name## _stack,\t\t\\\n+\tcea_map_percpu_pages(cea-\u003eestacks.event_stacks[name].stack,\t\\\n \t\t\testacks-\u003ename## _stack, npages, PAGE_KERNEL);\t\\\n \t} while (0)\n \n@@ -151,15 +171,15 @@ static void __init percpu_setup_exception_stacks(unsigned int cpu)\n \t * by guard pages so each stack must be mapped separately. DB2 is\n \t * not mapped; it just exists to catch triple nesting of #DB.\n \t /\n-\tcea_map_stack(DF);\n-\tcea_map_stack(NMI);\n-\tcea_map_stack(DB);\n-\tcea_map_stack(MCE);\n+\tcea_map_stack(ESTACK_DF);\n+\tcea_map_stack(ESTACK_NMI);\n+\tcea_map_stack(ESTACK_DB);\n+\tcea_map_stack(ESTACK_MCE);\n \n \tif (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) {\n \t\tif (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {\n-\t\t\tcea_map_stack(VC);\n-\t\t\tcea_map_stack(VC2);\n+\t\t\tcea_map_stack(ESTACK_VC);\n+\t\t\tcea_map_stack(ESTACK_VC2);\n \t\t}\n \t}\n }\ndiff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c\nindex 45b99c3b1442d..f6ee1cc3f125c 100644\n--- a/arch/x86/mm/fault.c\n+++ b/arch/x86/mm/fault.c\n@@ -677,7 +677,7 @@ page_fault_oops(struct pt_regs regs, unsigned long error_code,\n \t\t * and then double-fault, though, because we're likely to\n \t\t * break the console driver and lose most of the stack dump.\n \t\t /\n-\t\tcall_on_stack(__this_cpu_ist_top_va(DF) - sizeof(void),\n+\t\tcall_on_stack(__this_cpu_ist_top_va(ESTACK_DF) - sizeof(void),\n \t\t\t handle_stack_overflow,\n \t\t\t ASM_CALL_ARG3,\n \t\t\t , [arg1] \"r\" (regs), [arg2] \"r\" (address), [arg3] \"r\" (\u0026info));\ndiff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h\nindex 9fc8dfdfd65ff..c5ffc4e7ddb14 100644\n--- a/include/uapi/linux/kvm.h\n+++ b/include/uapi/linux/kvm.h\n@@ -998,6 +998,7 @@ struct kvm_enable_cap {\n #define KVM_CAP_S390_VSIE_ESAMODE 248\n #define KVM_CAP_S390_HPAGE_2G 249\n #define KVM_CAP_ARM_PMU_V3_STRICT 250\n+#define KVM_CAP_X86_FRED_EVENT 251\n \n struct kvm_irq_routing_irqchip {\n \t__u32 irqchip;\ndiff --git a/tools/arch/x86/include/asm/msr-index.h b/tools/arch/x86/include/asm/msr-index.h\nindex 18c4be75e9271..14632e29f9814 100644\n--- a/tools/arch/x86/include/asm/msr-index.h\n+++ b/tools/arch/x86/include/asm/msr-index.h\n@@ -64,6 +64,8 @@\n #define MSR_IA32_FRED_SSP3\t0x1d3\t\t\t/ Level 3 shadow stack pointer /\n #define MSR_IA32_FRED_CONFIG\t0x1d4\t\t\t/ Entrypoint and interrupt stack level /\n \n+#define FRED_CONFIG_RESERVED\t\t(BIT_ULL(2) \| GENMASK_ULL(5, 4) \| BIT_ULL(11))\n+\n / Intel MSRs. Some also available on other CPUs /\n #define MSR_TEST_CTRL\t\t\t\t0x00000033\n #define MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT\t29\ndiff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm\nindex 96bab7002d39e..73aed018f4a38 100644\n--- a/tools/testing/selftests/kvm/Makefile.kvm\n+++ b/tools/testing/selftests/kvm/Makefile.kvm\n@@ -78,6 +78,7 @@ TEST_GEN_PROGS_x86 += x86/feature_msrs_test\n TEST_GEN_PROGS_x86 += x86/evmcs_smm_controls_test\n TEST_GEN_PROGS_x86 += x86/exit_on_emulation_failure_test\n TEST_GEN_PROGS_x86 += x86/fastops_test\n+TEST_GEN_PROGS_x86 += x86/fred_test\n TEST_GEN_PROGS_x86 += x86/fix_hypercall_test\n TEST_GEN_PROGS_x86 += x86/hwcr_msr_test\n TEST_GEN_PROGS_x86 += x86/hyperv_clock\ndiff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h\nindex 5a7a455b53875..981adfcbd5fa2 100644\n--- a/tools/testing/selftests/kvm/include/kvm_util.h\n+++ b/tools/testing/selftests/kvm/include/kvm_util.h\n@@ -191,6 +191,7 @@ enum vm_guest_mode {\n \tVM_MODE_P40V48_16K,\n \tVM_MODE_P40V48_64K,\n \tVM_MODE_PXXVYY_4K,\t/ For 48-bit or 57-bit VA, depending on host support /\n+\tVM_MODE_PXXVYY_4K_USER,\t/ For 48bits VA but ANY bits PA with USER bit set /\n \tVM_MODE_P47V64_4K,\n \tVM_MODE_P44V64_4K,\n \tVM_MODE_P36V48_4K,\ndiff --git a/tools/testing/selftests/kvm/include/x86/processor.h b/tools/testing/selftests/kvm/include/x86/processor.h\nindex 6e6f70035508a..988c45e326894 100644\n--- a/tools/testing/selftests/kvm/include/x86/processor.h\n+++ b/tools/testing/selftests/kvm/include/x86/processor.h\n@@ -79,6 +79,7 @@ const char ex_str(int vector);\n #define X86_CR4_SMEP\t\t(1ul \u003c\u003c 20)\n #define X86_CR4_SMAP\t\t(1ul \u003c\u003c 21)\n #define X86_CR4_PKE\t\t(1ul \u003c\u003c 22)\n+#define X86_CR4_FRED\t\t(1ul \u003c\u003c 32)\n \n struct xstate_header {\n \tu64\t\t\t\txstate_bv;\n@@ -195,6 +196,7 @@ struct kvm_x86_cpu_feature {\n #define\tX86_FEATURE_SPEC_CTRL\t\tKVM_X86_CPU_FEATURE(0x7, 0, EDX, 26)\n #define\tX86_FEATURE_ARCH_CAPABILITIES\tKVM_X86_CPU_FEATURE(0x7, 0, EDX, 29)\n #define\tX86_FEATURE_PKS\t\t\tKVM_X86_CPU_FEATURE(0x7, 0, ECX, 31)\n+#define\tX86_FEATURE_FRED\t\tKVM_X86_CPU_FEATURE(0x7, 1, EAX, 17)\n #define\tX86_FEATURE_XTILECFG\t\tKVM_X86_CPU_FEATURE(0xD, 0, EAX, 17)\n #define\tX86_FEATURE_XTILEDATA\t\tKVM_X86_CPU_FEATURE(0xD, 0, EAX, 18)\n #define\tX86_FEATURE_XSAVES\t\tKVM_X86_CPU_FEATURE(0xD, 1, EAX, 3)\n@@ -1645,4 +1647,36 @@ u64 tdp_get_pte(struct kvm_vm vm, u64 l2_gpa);\n \n bool sys_clocksource_is_based_on_tsc(void);\n \n+/\n+ FRED related data structures and functions\n+ /\n+\n+#define FRED_SSX_NMI\t\tBIT_ULL(18)\n+\n+struct fred_stack {\n+\tu64 r15;\n+\tu64 r14;\n+\tu64 r13;\n+\tu64 r12;\n+\tu64 bp;\n+\tu64 bx;\n+\tu64 r11;\n+\tu64 r10;\n+\tu64 r9;\n+\tu64 r8;\n+\tu64 ax;\n+\tu64 cx;\n+\tu64 dx;\n+\tu64 si;\n+\tu64 di;\n+\tu64 error_code;\n+\tu64 ip;\n+\tu64 csx;\n+\tu64 flags;\n+\tu64 sp;\n+\tu64 ssx;\n+\tu64 event_data;\n+\tu64 reserved;\n+};\n+\n #endif / SELFTEST_KVM_PROCESSOR_H /\ndiff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c\nindex 9ddc047d5c275..e79fddedaf396 100644\n--- a/tools/testing/selftests/kvm/lib/kvm_util.c\n+++ b/tools/testing/selftests/kvm/lib/kvm_util.c\n@@ -210,6 +210,7 @@ const char vm_guest_mode_string(u32 i)\n \t\t[VM_MODE_P40V48_16K]\t= \"PA-bits:40, VA-bits:48, 16K pages\",\n \t\t[VM_MODE_P40V48_64K]\t= \"PA-bits:40, VA-bits:48, 64K pages\",\n \t\t[VM_MODE_PXXVYY_4K]\t= \"PA-bits:ANY, VA-bits:48 or 57, 4K pages\",\n+\t\t[VM_MODE_PXXVYY_4K_USER]= \"PA-bits:ANY, VA-bits:48, 4K user pages\",\n \t\t[VM_MODE_P47V64_4K]\t= \"PA-bits:47, VA-bits:64, 4K pages\",\n \t\t[VM_MODE_P44V64_4K]\t= \"PA-bits:44, VA-bits:64, 4K pages\",\n \t\t[VM_MODE_P36V48_4K]\t= \"PA-bits:36, VA-bits:48, 4K pages\",\n@@ -246,6 +247,7 @@ const struct vm_guest_mode_params vm_guest_mode_params[] = {\n \t[VM_MODE_P40V48_16K]\t= { 40, 48, 0x4000, 14 },\n \t[VM_MODE_P40V48_64K]\t= { 40, 48, 0x10000, 16 },\n \t[VM_MODE_PXXVYY_4K]\t= { 0, 0, 0x1000, 12 },\n+\t[VM_MODE_PXXVYY_4K_USER]= { 0, 0, 0x1000, 12 },\n \t[VM_MODE_P47V64_4K]\t= { 47, 64, 0x1000, 12 },\n \t[VM_MODE_P44V64_4K]\t= { 44, 64, 0x1000, 12 },\n \t[VM_MODE_P36V48_4K]\t= { 36, 48, 0x1000, 12 },\n@@ -337,6 +339,7 @@ struct kvm_vm ____vm_create(struct vm_shape shape)\n \t\tvm-\u003emmu.pgtable_levels = 3;\n \t\tbreak;\n \tcase VM_MODE_PXXVYY_4K:\n+\tcase VM_MODE_PXXVYY_4K_USER:\n #ifdef __x86_64__\n \t\tkvm_get_cpu_address_width(\u0026vm-\u003epa_bits, \u0026vm-\u003eva_bits);\n \t\tkvm_init_vm_address_properties(vm);\n@@ -355,7 +358,7 @@ struct kvm_vm ____vm_create(struct vm_shape shape)\n \t\t\tvm-\u003emmu.pgtable_levels = 4;\n \t\t}\n #else\n-\t\tTEST_FAIL(\"VM_MODE_PXXVYY_4K not supported on non-x86 platforms\");\n+\t\tTEST_FAIL(\"VM_MODE_PXXVYY_4K(_USER) not supported on non-x86 platforms\");\n #endif\n \t\tbreak;\n \tcase VM_MODE_P47V64_4K:\ndiff --git a/tools/testing/selftests/kvm/lib/x86/processor.c b/tools/testing/selftests/kvm/lib/x86/processor.c\nindex d31fa81ea0756..51d713c4c9de1 100644\n--- a/tools/testing/selftests/kvm/lib/x86/processor.c\n+++ b/tools/testing/selftests/kvm/lib/x86/processor.c\n@@ -167,6 +167,9 @@ bool kvm_is_tdp_enabled(void)\n static void virt_mmu_init(struct kvm_vm vm, struct kvm_mmu mmu,\n \t\t\t struct pte_masks pte_masks)\n {\n+\tTEST_ASSERT(vm-\u003emode == VM_MODE_PXXVYY_4K \|\| vm-\u003emode == VM_MODE_PXXVYY_4K_USER,\n+\t\t \"Unknown or unsupported guest mode: 0x%x\", vm-\u003emode);\n+\n \t/ If needed, create the top-level page table. /\n \tif (!mmu-\u003epgd_created) {\n \t\tmmu-\u003epgd = vm_alloc_page_table(vm);\n@@ -181,7 +184,7 @@ static void virt_mmu_init(struct kvm_vm vm, struct kvm_mmu mmu,\n \n void virt_arch_pgd_alloc(struct kvm_vm vm)\n {\n-\tTEST_ASSERT(vm-\u003emode == VM_MODE_PXXVYY_4K,\n+\tTEST_ASSERT(vm-\u003emode == VM_MODE_PXXVYY_4K \|\| vm-\u003emode == VM_MODE_PXXVYY_4K_USER,\n \t\t \"Unknown or unsupported guest mode: 0x%x\", vm-\u003emode);\n \n \tstruct pte_masks pte_masks = (struct pte_masks){\n@@ -239,6 +242,8 @@ static u64 virt_create_upper_pte(struct kvm_vm vm,\n \t\tpte = PTE_PRESENT_MASK(mmu) \| PTE_READABLE_MASK(mmu) \|\n \t\t PTE_WRITABLE_MASK(mmu) \| PTE_EXECUTABLE_MASK(mmu) \|\n \t\t PTE_ALWAYS_SET_MASK(mmu);\n+\t\tif (vm-\u003emode == VM_MODE_PXXVYY_4K_USER)\n+\t\t\tpte \|= PTE_USER_MASK(mmu);\n \t\tif (current_level == target_level)\n \t\t\tpte \|= PTE_HUGE_MASK(mmu) \| (gpa \u0026 PHYSICAL_PAGE_MASK);\n \t\telse\n@@ -266,7 +271,7 @@ void __virt_pg_map(struct kvm_vm vm, struct kvm_mmu mmu, gva_t gva,\n \tu64 pte = \u0026mmu-\u003epgd;\n \tint current_level;\n \n-\tTEST_ASSERT(vm-\u003emode == VM_MODE_PXXVYY_4K,\n+\tTEST_ASSERT(vm-\u003emode == VM_MODE_PXXVYY_4K \|\| vm-\u003emode == VM_MODE_PXXVYY_4K_USER,\n \t\t \"Unknown or unsupported guest mode: 0x%x\", vm-\u003emode);\n \n \tTEST_ASSERT((gva % pg_size) == 0,\n@@ -316,6 +321,9 @@ void __virt_pg_map(struct kvm_vm vm, struct kvm_mmu mmu, gva_t gva,\n \t\tpte \|= PTE_C_BIT_MASK(mmu);\n \telse\n \t\tpte \|= PTE_S_BIT_MASK(mmu);\n+\n+\tif (vm-\u003emode == VM_MODE_PXXVYY_4K_USER)\n+\t\tpte \|= PTE_USER_MASK(mmu);\n }\n \n void virt_arch_pg_map(struct kvm_vm vm, gva_t gva, gpa_t gpa)\n@@ -372,7 +380,7 @@ static u64 __vm_get_page_table_entry(struct kvm_vm vm,\n \tTEST_ASSERT(level \u003e= PG_LEVEL_NONE \u0026\u0026 level \u003c= mmu-\u003epgtable_levels,\n \t\t \"Invalid PG_LEVEL_* '%d'\", level);\n \n-\tTEST_ASSERT(vm-\u003emode == VM_MODE_PXXVYY_4K,\n+\tTEST_ASSERT(vm-\u003emode == VM_MODE_PXXVYY_4K \|\| vm-\u003emode == VM_MODE_PXXVYY_4K_USER,\n \t\t \"Unknown or unsupported guest mode: 0x%x\", vm-\u003emode);\n \tTEST_ASSERT(sparsebit_is_set(vm-\u003evpages_valid, (gva \u003e\u003e vm-\u003epage_shift)),\n \t\t \"Invalid virtual address, gva: 0x%lx\", gva);\n@@ -645,7 +653,7 @@ static void vcpu_init_sregs(struct kvm_vm vm, struct kvm_vcpu vcpu)\n {\n \tstruct kvm_sregs sregs;\n \n-\tTEST_ASSERT(vm-\u003emode == VM_MODE_PXXVYY_4K,\n+\tTEST_ASSERT(vm-\u003emode == VM_MODE_PXXVYY_4K \|\| vm-\u003emode == VM_MODE_PXXVYY_4K_USER,\n \t\t \"Unknown or unsupported guest mode: 0x%x\", vm-\u003emode);\n \n \t/ Set mode specific system register values. /\ndiff --git a/tools/testing/selftests/kvm/x86/fred_test.c b/tools/testing/selftests/kvm/x86/fred_test.c\nnew file mode 100644\nindex 0000000000000..c7d08e5253b98\n--- /dev/null\n+++ b/tools/testing/selftests/kvm/x86/fred_test.c\n@@ -0,0 +1,291 @@\n+// SPDX-License-Identifier: GPL-2.0-only\n+/\n+ * FRED nested exception tests\n+ \n+ Copyright (C) 2023, Intel, Inc.\n+ /\n+#define _GNU_SOURCE / for program_invocation_short_name /\n+#include \u003cfcntl.h\u003e\n+#include \u003cstdio.h\u003e\n+#include \u003cstdlib.h\u003e\n+#include \u003cstring.h\u003e\n+#include \u003csys/ioctl.h\u003e\n+#include \u003casm/msr-index.h\u003e\n+\n+#include \"apic.h\"\n+#include \"kvm_util.h\"\n+#include \"test_util.h\"\n+#include \"guest_modes.h\"\n+#include \"processor.h\"\n+\n+#define IRQ_VECTOR 0xAA\n+\n+#define FRED_STKLVL(v,l)\t\t(_AT(unsigned long, l) \u003c\u003c (2 (v)))\n+#define FRED_CONFIG_ENTRYPOINT(p)\t_AT(unsigned long, (p))\n+\n+/* This address is already mapped in guest page table. /\n+#define FRED_VALID_RSP\t\t\t0x8000\n+\n+/\n+ * The following addresses are not yet mapped in both EPT and guest page\n+ * tables at the beginning. As a result, it causes an EPT violation VM\n+ * exit with an original guest #PF to access any of them for the first\n+ * time.\n+ \n+ Use these addresses as guest FRED RSP0 to generate nested #PFs to test\n+ * if event data are properly virtualized.\n+ /\n+static unsigned long fred_invalid_rsp[4] = {\n+\t0x0,\n+\t0xf0000000,\n+\t0xe0000000,\n+\t0xd0000000,\n+};\n+\n+extern char asm_user_nop[];\n+extern char asm_user_ud[];\n+extern char asm_done_fault[];\n+\n+extern void asm_test_fault(int test);\n+\n+/\n+ * user level code for triggering faults.\n+ /\n+asm(\".pushsection .text\\n\"\n+ \".align 4096\\n\"\n+\n+ \".type asm_user_nop, @function\\n\"\n+ \"asm_user_nop:\\n\"\n+ \"1: .byte 0x90\\n\"\n+ \"jmp 1b\\n\"\n+\n+ \".org asm_user_nop + 16, 0xcc\\n\"\n+ \".type asm_user_ud, @function\\n\"\n+ \"asm_user_ud:\\n\"\n+ / Trigger a #UD /\n+ \"ud2\\n\"\n+\n+ \".align 4096, 0xcc\\n\"\n+ \".popsection\");\n+\n+/ Send current stack level and #PF address /\n+#define GUEST_SYNC_CSL_FA(__stage, __pf_address)\t\t\\\n+\tGUEST_SYNC_ARGS(__stage, __pf_address, 0, 0, 0)\n+\n+void fred_entry_from_user(struct fred_stack stack)\n+{\n+\tu32 current_stack_level = rdmsr(MSR_IA32_FRED_CONFIG) \u0026 0x3;\n+\n+\tGUEST_SYNC_CSL_FA(current_stack_level, stack-\u003eevent_data);\n+\n+\t/* Do NOT go back to user level, continue the next test instead /\n+\tstack-\u003essx = 0x18;\n+\tstack-\u003ecsx = 0x10;\n+\tstack-\u003eip = (u64)\u0026asm_done_fault;\n+}\n+\n+void fred_entry_from_kernel(struct fred_stack stack)\n+{\n+\t/\n+\t Keep NMI blocked to delay the delivery of the next NMI until\n+\t * returning to user level.\n+\t * /\n+\tstack-\u003essx \u0026= ~FRED_SSX_NMI;\n+}\n+\n+#define PUSH_REGS\t\\\n+\t\"push %rdi\\n\"\t\\\n+\t\"push %rsi\\n\"\t\\\n+\t\"push %rdx\\n\"\t\\\n+\t\"push %rcx\\n\"\t\\\n+\t\"push %rax\\n\"\t\\\n+\t\"push %r8\\n\"\t\\\n+\t\"push %r9\\n\"\t\\\n+\t\"push %r10\\n\"\t\\\n+\t\"push %r11\\n\"\t\\\n+\t\"push %rbx\\n\"\t\\\n+\t\"push %rbp\\n\"\t\\\n+\t\"push %r12\\n\"\t\\\n+\t\"push %r13\\n\"\t\\\n+\t\"push %r14\\n\"\t\\\n+\t\"push %r15\\n\"\n+\n+#define POP_REGS\t\\\n+\t\"pop %r15\\n\"\t\\\n+\t\"pop %r14\\n\"\t\\\n+\t\"pop %r13\\n\"\t\\\n+\t\"pop %r12\\n\"\t\\\n+\t\"pop %rbp\\n\"\t\\\n+\t\"pop %rbx\\n\"\t\\\n+\t\"pop %r11\\n\"\t\\\n+\t\"pop %r10\\n\"\t\\\n+\t\"pop %r9\\n\"\t\\\n+\t\"pop %r8\\n\"\t\\\n+\t\"pop %rax\\n\"\t\\\n+\t\"pop %rcx\\n\"\t\\\n+\t\"pop %rdx\\n\"\t\\\n+\t\"pop %rsi\\n\"\t\\\n+\t\"pop %rdi\\n\"\n+\n+/\n+ * FRED entry points.\n+ /\n+asm(\".pushsection .text\\n\"\n+ \".type asm_fred_entrypoint_user, @function\\n\"\n+ \".align 4096\\n\"\n+ \"asm_fred_entrypoint_user:\\n\"\n+ PUSH_REGS\n+ \"movq %rsp, %rdi\\n\"\n+ \"call fred_entry_from_user\\n\"\n+ POP_REGS\n+ / Do NOT go back to user level, continue the next test instead /\n+ \".byte 0xf2,0x0f,0x01,0xca\\n\"\t/ ERETS /\n+\n+ \".org asm_fred_entrypoint_user + 256, 0xcc\\n\"\n+ \".type asm_fred_entrypoint_kernel, @function\\n\"\n+ \"asm_fred_entrypoint_kernel:\\n\"\n+ PUSH_REGS\n+ \"movq %rsp, %rdi\\n\"\n+ \"call fred_entry_from_kernel\\n\"\n+ POP_REGS\n+ \".byte 0xf2,0x0f,0x01,0xca\\n\"\t/ ERETS /\n+ \".align 4096, 0xcc\\n\"\n+ \".popsection\");\n+\n+extern char asm_fred_entrypoint_user[];\n+\n+/\n+ * Prepare a FRED stack frame for ERETU to return to user level code,\n+ * nop or ud2.\n+ \n+ Because FRED RSP0 is deliberately not mapped in guest page table,\n+ * the delivery of interrupt/NMI or #UD from ring 3 causes a nested\n+ * #PF, which is then delivered on FRED RSPx (x is 1, 2 or 3,\n+ * determinated by MSR FRED_STKLVL[PF_VECTOR]).\n+ /\n+asm(\".pushsection .text\\n\"\n+ \".type asm_test_fault, @function\\n\"\n+ \".align 4096\\n\"\n+ \"asm_test_fault:\\n\"\n+ \"endbr64\\n\"\n+ \"push %rbp\\n\"\n+ \"mov %rsp, %rbp\\n\"\n+ \"and $(~0x3f), %rsp\\n\"\n+ \"push $0\\n\"\n+ \"push $0\\n\"\n+ \"mov $0x2b, %rax\\n\"\n+ / Unblock NMI /\n+ \"bts $18, %rax\\n\"\n+ / Set long mode bit /\n+ \"bts $57, %rax\\n\"\n+ \"push %rax\\n\"\n+ / No stack required for the FRED user level test code /\n+ \"push $0\\n\"\n+ \"pushf\\n\"\n+ \"pop %rax\\n\"\n+ / Allow external interrupts /\n+ \"bts $9, %rax\\n\"\n+ \"push %rax\\n\"\n+ \"mov $0x33, %rax\\n\"\n+ \"push %rax\\n\"\n+ \"cmp $0, %edi\\n\"\n+ \"jne 1f\\n\"\n+ \"lea asm_user_nop(%rip), %rax\\n\"\n+ \"jmp 2f\\n\"\n+ \"1: lea asm_user_ud(%rip), %rax\\n\"\n+ \"2: push %rax\\n\"\n+ \"push $0\\n\"\n+ / ERETU to user level code to allow event delivery immediately /\n+ \".byte 0xf3,0x0f,0x01,0xca\\n\"\n+ \"asm_done_fault:\\n\"\n+ \"mov %rbp, %rsp\\n\"\n+ \"pop %rbp\\n\"\n+ \"ret\\n\"\n+ \".align 4096, 0xcc\\n\"\n+ \".popsection\");\n+\n+/\n+ * To fully test the underlying FRED VMX code, this test should be run one\n+ * more round with EPT disabled to inject page faults as nested exceptions.\n+ /\n+static void guest_code(void)\n+{\n+\twrmsr(MSR_IA32_FRED_CONFIG,\n+\t FRED_CONFIG_ENTRYPOINT(asm_fred_entrypoint_user));\n+\n+\twrmsr(MSR_IA32_FRED_RSP1, FRED_VALID_RSP);\n+\twrmsr(MSR_IA32_FRED_RSP2, FRED_VALID_RSP);\n+\twrmsr(MSR_IA32_FRED_RSP3, FRED_VALID_RSP);\n+\n+\t/ Enable FRED /\n+\tset_cr4(get_cr4() \| X86_CR4_FRED);\n+\n+\tx2apic_enable();\n+\n+\twrmsr(MSR_IA32_FRED_STKLVLS, FRED_STKLVL(PF_VECTOR, 1));\n+\twrmsr(MSR_IA32_FRED_RSP0, fred_invalid_rsp[1]);\n+\t/ 1: ud2 to generate #UD /\n+\tasm_test_fault(1);\n+\n+\twrmsr(MSR_IA32_FRED_STKLVLS, FRED_STKLVL(PF_VECTOR, 2));\n+\twrmsr(MSR_IA32_FRED_RSP0, fred_invalid_rsp[2]);\n+\tasm volatile(\"cli\");\n+\t/ Create a pending interrupt on current vCPU /\n+\tx2apic_write_reg(APIC_ICR, APIC_DEST_SELF \| APIC_INT_ASSERT \|\n+\t\t\t APIC_DM_FIXED \| IRQ_VECTOR);\n+\t/ Return to ring 3 /\n+\tasm_test_fault(0);\n+\tx2apic_write_reg(APIC_EOI, 0);\n+\n+\twrmsr(MSR_IA32_FRED_STKLVLS, FRED_STKLVL(PF_VECTOR, 3));\n+\twrmsr(MSR_IA32_FRED_RSP0, fred_invalid_rsp[3]);\n+\t/\n+\t * The first NMI is just to have NMI blocked in ring 0, because\n+\t * fred_entry_from_kernel() deliberately clears the NMI bit in\n+\t * FRED stack frame.\n+\t /\n+\tx2apic_write_reg(APIC_ICR, APIC_DEST_SELF \| APIC_INT_ASSERT \|\n+\t\t\t APIC_DM_NMI \| NMI_VECTOR);\n+\t/ The second NMI will be delivered after returning to ring 3 /\n+\tx2apic_write_reg(APIC_ICR, APIC_DEST_SELF \| APIC_INT_ASSERT \|\n+\t\t\t APIC_DM_NMI \| NMI_VECTOR);\n+\t/ Return to ring 3 /\n+\tasm_test_fault(0);\n+\n+\tGUEST_DONE();\n+}\n+\n+int main(int argc, char argv[])\n+{\n+\tstruct kvm_vcpu vcpu;\n+\tstruct kvm_vm vm;\n+\tstruct ucall uc;\n+\tuint64_t expected_current_stack_level = 1;\n+\n+\tTEST_REQUIRE(kvm_cpu_has(X86_FEATURE_FRED));\n+\n+\tvm = __vm_create_with_vcpus(VM_SHAPE(VM_MODE_PXXVYY_4K_USER), 1, 0,\n+\t\t\t\t guest_code, \u0026vcpu);\n+\n+\twhile (true) {\n+\t\tuint64_t r;\n+\n+\t\tvcpu_run(vcpu);\n+\n+\t\tr = get_ucall(vcpu, \u0026uc);\n+\n+\t\tif (r == UCALL_DONE)\n+\t\t\tbreak;\n+\n+\t\tif (r == UCALL_SYNC) {\n+\t\t\tTEST_ASSERT((uc.args[1] == expected_current_stack_level) \u0026\u0026\n+\t\t\t\t (uc.args[2] == fred_invalid_rsp[expected_current_stack_level] - 8),\n+\t\t\t\t \"Incorrect stack level %lx and #PF address %lx\\n\",\n+\t\t\t\t uc.args[1], uc.args[2]);\n+\t\t\texpected_current_stack_level++;\n+\t\t}\n+\t}\n+\n+\tkvm_vm_free(vm);\n+\treturn 0;\n+}\ndiff --git a/tools/testing/selftests/kvm/x86/msrs_test.c b/tools/testing/selftests/kvm/x86/msrs_test.c\nindex f7e39bf887ad3..6e2d8b8f1bd34 100644\n--- a/tools/testing/selftests/kvm/x86/msrs_test.c\n+++ b/tools/testing/selftests/kvm/x86/msrs_test.c\n@@ -60,9 +60,23 @@ static const u64 u64_val = 0xaaaa5555aaaa5555ull;\n #define MSR_TEST_CANONICAL(msr, feat)\t\t\t\t\t\\\n \t__MSR_TEST(msr, #msr, canonical_val, NONCANONICAL, 0, feat)\n \n+#define MSR_TEST_CANONICAL2(msr, feat, f2)\t\t\t\t\\\n+\t____MSR_TEST(msr, #msr, canonical_val, NONCANONICAL, 0, feat, f2, false)\n+\n #define MSR_TEST_KVM(msr, val, rsvd, feat)\t\t\t\t\\\n \t____MSR_TEST(KVM_REG_ ##msr, #msr, val, rsvd, 0, feat, feat, true)\n \n+/* FRED RSPs are 64-byte aligned, thus bits 5:0 are all zero /\n+static const u64 fred_rsp_canonical_val = canonical_val + 0xbc0;\n+\n+/\n+ * FRED SSP[123] are 8-byte aligned, thus bits 2:0 are all zero.\n+ \n+ For legacy reasons, FRED SSP0 is allowed to be 4-byte aligned, but treat\n+ * it as 8-byte aligned to simplify the tests.\n+ /\n+static const u64 fred_ssp_canonical_val = canonical_val + 0xbc8;\n+\n /\n * The main struct must be scoped to a function due to the use of structures to\n * define features. For the global structure, allocate enough space for the\n@@ -388,8 +402,9 @@ static void test_msrs(void)\n \t\tMSR_TEST2(MSR_IA32_S_CET, CET_ENDBR_EN, CET_RESERVED, IBT, SHSTK),\n \t\tMSR_TEST2(MSR_IA32_U_CET, CET_SHSTK_EN, CET_RESERVED, SHSTK, IBT),\n \t\tMSR_TEST2(MSR_IA32_U_CET, CET_ENDBR_EN, CET_RESERVED, IBT, SHSTK),\n-\t\tMSR_TEST_CANONICAL(MSR_IA32_PL0_SSP, SHSTK),\n-\t\tMSR_TEST(MSR_IA32_PL0_SSP, canonical_val, canonical_val \| 1, SHSTK),\n+\t\tMSR_TEST_CANONICAL2(MSR_IA32_PL0_SSP, SHSTK, FRED),\n+\t\tMSR_TEST2(MSR_IA32_PL0_SSP, canonical_val, canonical_val \| 1, SHSTK, FRED),\n+\t\tMSR_TEST2(MSR_IA32_PL0_SSP, canonical_val, canonical_val \| 1, FRED, SHSTK),\n \t\tMSR_TEST_CANONICAL(MSR_IA32_PL1_SSP, SHSTK),\n \t\tMSR_TEST(MSR_IA32_PL1_SSP, canonical_val, canonical_val \| 1, SHSTK),\n \t\tMSR_TEST_CANONICAL(MSR_IA32_PL2_SSP, SHSTK),\n@@ -398,10 +413,35 @@ static void test_msrs(void)\n \t\tMSR_TEST(MSR_IA32_PL3_SSP, canonical_val, canonical_val \| 1, SHSTK),\n \n \t\tMSR_TEST_KVM(GUEST_SSP, canonical_val, NONCANONICAL, SHSTK),\n+\n+\t\tMSR_TEST_CANONICAL(MSR_IA32_FRED_RSP0, FRED),\n+\t\tMSR_TEST(MSR_IA32_FRED_RSP0, fred_rsp_canonical_val, fred_rsp_canonical_val \| 1, FRED),\n+\t\tMSR_TEST_CANONICAL(MSR_IA32_FRED_RSP1, FRED),\n+\t\tMSR_TEST(MSR_IA32_FRED_RSP1, fred_rsp_canonical_val, fred_rsp_canonical_val \| 2, FRED),\n+\t\tMSR_TEST_CANONICAL(MSR_IA32_FRED_RSP2, FRED),\n+\t\tMSR_TEST(MSR_IA32_FRED_RSP2, fred_rsp_canonical_val, fred_rsp_canonical_val \| 4, FRED),\n+\t\tMSR_TEST_CANONICAL(MSR_IA32_FRED_RSP3, FRED),\n+\t\tMSR_TEST(MSR_IA32_FRED_RSP3, fred_rsp_canonical_val, fred_rsp_canonical_val \| 7, FRED),\n+\n+\t\tMSR_TEST_CANONICAL2(MSR_IA32_FRED_SSP0, FRED, SHSTK),\n+\t\tMSR_TEST2(MSR_IA32_FRED_SSP0, fred_ssp_canonical_val, fred_ssp_canonical_val \| 1, SHSTK, FRED),\n+\t\tMSR_TEST2(MSR_IA32_FRED_SSP0, fred_ssp_canonical_val, fred_ssp_canonical_val \| 1, FRED, SHSTK),\n+\t\tMSR_TEST_CANONICAL(MSR_IA32_FRED_SSP1, FRED),\n+\t\tMSR_TEST(MSR_IA32_FRED_SSP1, fred_ssp_canonical_val, fred_ssp_canonical_val \| 2, FRED),\n+\t\tMSR_TEST_CANONICAL(MSR_IA32_FRED_SSP2, FRED),\n+\t\tMSR_TEST(MSR_IA32_FRED_SSP2, fred_ssp_canonical_val, fred_ssp_canonical_val \| 4, FRED),\n+\t\tMSR_TEST_CANONICAL(MSR_IA32_FRED_SSP3, FRED),\n+\t\tMSR_TEST(MSR_IA32_FRED_SSP3, fred_ssp_canonical_val, fred_ssp_canonical_val \| 7, FRED),\n+\n+\t\tMSR_TEST(MSR_IA32_FRED_STKLVLS, 0xe41be41be41be41b, 0, FRED),\n+\n+\t\tMSR_TEST_CANONICAL(MSR_IA32_FRED_CONFIG, FRED),\n+\t\tMSR_TEST(MSR_IA32_FRED_CONFIG, canonical_val, canonical_val \| FRED_CONFIG_RESERVED, FRED),\n \t};\n \n \tconst struct kvm_x86_cpu_feature feat_none = X86_FEATURE_NONE;\n \tconst struct kvm_x86_cpu_feature feat_lm = X86_FEATURE_LM;\n+\tconst struct kvm_x86_cpu_feature feat_fred = X86_FEATURE_FRED;\n \n \t/\n \t Create three vCPUs, but run them on the same task, to validate KVM's\n@@ -432,6 +472,14 @@ static void test_msrs(void)\n \t * immutable once the vCPU has been run.\n \t /\n \tfor (idx = 0; idx \u003c ARRAY_SIZE(__msrs); idx++) {\n+\t\t/\n+\t\t * Feature word 0x7:1:EAX used by FRED doesn't exist on some CPU\n+\t\t * models, simply skip clearing FRED feature bit.\n+\t\t /\n+\t\tif (!memcmp(\u0026msrs[idx].feature, \u0026feat_fred, sizeof(feat_fred)) \u0026\u0026\n+\t\t !kvm_cpu_has(X86_FEATURE_FRED))\n+\t\t\tcontinue;\n+\n \t\t/\n \t\t * Don't clear LM; selftests are 64-bit only, and KVM doesn't\n \t\t * honor LM=0 for MSRs that are supposed to exist if and only\n" }
2/1	2026/09/11 21:49	agent	patch-evaluator	1m Model: core Args: null Results: { "EnableConfigs": [ "KVM", "KVM_INTEL", "X86_FRED" ], "FocusSymbols": [ "vmx_inject_exception", "vmx_set_msr", "vmx_get_msr", "kvm_deliver_exception_payload", "kvm_vcpu_ioctl_x86_set_vcpu_events", "kvm_vcpu_ioctl_x86_get_vcpu_events", "nested_check_vm_entry_controls", "nested_vmx_check_guest_state", "load_vmcs12_host_state", "this_cpu_fred_rsp" ], "Reasoning": "The patch introduces support for Intel FRED (Flexible Return and Event Delivery) across x86 core exception stacks, KVM x86 emulation, VMX, and nested VMX virtualization. It modifies reachable ioctl entrypoints (KVM_CAP_X86_FRED_EVENT, KVM_SET_VCPU_EVENTS, KVM_GET_VCPU_EVENTS), MSR get/set handling, CR4 validation and emulation, exception payload delivery and injection, and nested VMX VM-entry/exit checks and state transition emulation (with newly added WARN_ON_ONCE assertions and consistency checks). These paths are reachable via /dev/kvm ioctls from userspace in virtualized environments.", "WorthFuzzing": true } Instruction: You are an expert Linux kernel maintainer and security engineer. Your job is to review a provided patch series and evaluate whether it warrants fuzzing with syzkaller. IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in your workspace. Do NOT rely on internal assumptions. You must actively use your code access tools to inspect the actual source code, callers, and surrounding context. ================================================================================ 1. CORE TRIAGE PHILOSOPHY ================================================================================ The goal of patch fuzzing is to discover crashes, regressions, exposed latent bugs, and newly triggered assertions introduced by the patch series. - REACHABILITY IS THE PRIMARY GATE: Fuzzing can only discover bugs in code that can actually execute in standard virtualized environments (GCE or QEMU, utilizing software-emulated devices like USB gadgets, netdev, tun/tap). If the modified code is structurally unreachable (see Section 2), it MUST NOT be fuzzed, regardless of whether it adds assertions or complex logic. - DO NOT BLINDLY TRUST "NO FUNCTIONAL CHANGE" (NFCI) OR "REFACTORING" CLAIMS: Patch authors routinely label changes as "cleanups", "refactorings", or state "No functional change intended". Do NOT take these claims at face value. Code refactorings that rearrange logic, introduce helper functions, or alter state management in core subsystems frequently introduce subtle semantic shifts or uncover latent kernel bugs. If reachable executable code is modified or refactored, it MUST be fuzzed. - NEW OR MODIFIED ASSERTIONS IN REACHABLE CODE MUST BE FUZZED: When a patch introduces or modifies runtime checks or assertions (e.g., WARN_ON, VM_WARN_ON, BUG_ON, lockdep_assert) in reachable code paths, it enforces new or stricter invariants. Even if the author believes the invariant always holds, fuzzing is essential to verify whether an unusual sequence of operations can violate it. ================================================================================ 2. WHEN TO RETURN WorthFuzzing=false (NEGATIVE CRITERIA) ================================================================================ Return WorthFuzzing=false ONLY IF all modified code falls strictly into one or more of these categories: - Non-kernel and non-executable changes: * Modifications to Documentation/, comments, or spelling fixes. * User-space directories, self-tests, samples, or scripts (e.g., tools/, samples/, scripts/, usr/) that do not affect the compiled kernel image (vmlinux) or kernel modules. * Purely decorative logging (e.g., message strings in pr_err, printk, dev_info) or tracepoints that do not alter control flow or data structures. * Build system or Kconfig changes that do not alter compiled C logic. - Structurally unreachable hardware: * Vendor-specific PCIe switches, SmartNICs, or GPU drivers (e.g., mlxsw, pds_core, qed, ionic, amdgpu) requiring physical ASIC/PCIe cards not emulated in standard QEMU. - Unreachable execution paths: * Driver teardown callbacks (.remove, .shutdown, pci_unregister_driver) executed only during physical PCI hot-unplug or manual sysfs driver unbinding. * Code paths exclusive to architectures other than the target architecture. ================================================================================ 3. WHEN TO RETURN WorthFuzzing=true (POSITIVE CRITERIA) ================================================================================ Return WorthFuzzing=true whenever the patch touches reachable executable code, including: - Core Subsystems: * Any logic modifications in memory management (mm/), synchronization/locking (kernel/locking/), BPF, scheduler, core networking, VFS, or syscall handling. - Refactorings and Code Cleanups: * Any restructuring of reachable data structures, helper abstractions, or algorithm flows. - Runtime Assertions and Defensive Checks: * Any introduction or alteration of assertions (WARN_ON, VM_WARN_ON, BUG_ON, etc.) in reachable paths. - Reachable Drivers and Protocols: Drivers accessible via virtual buses (virtio, USB gadget, loopback, netlink, binder, sockets, etc.). ================================================================================ 4. EXTRACTING FocusSymbols (PREVENTING DILUTION) ================================================================================ When WorthFuzzing=true, you must extract specific kernel functions into FocusSymbols to guide the fuzzer: - AVOID UBIQUITOUS LIFECYCLE HOT-PATHS: Do NOT list generic, ubiquitous functions called by almost every program in the corpus (including, but not limited to: general memory allocators and deallocators, page fault and trap handlers, or core synchronization primitives; this is not an exhaustive list). Listing ubiquitous functions causes the fuzzer to classify thousands of unrelated tests as "focused", which severely dilutes fuzzing effort away from the actual changes. - TARGET SPECIFIC FEATURE LOGIC AND ENTRYPOINTS: List functions that specifically implement the logic being added or altered, or direct API entrypoints for the subsystem feature under review. - HANDLING STATIC INLINE FUNCTIONS IN HEADERS (.h): Compiler-inlined static functions (such as static inlines in mm/.h or include/linux/.h) lack distinct symbol addresses in vmlinux and cannot be targeted directly by symbol coverage filters. If the changes are primarily in static inline helpers, identify non-static, feature-specific caller functions in .c files that exercise them (avoiding ubiquitous lifecycle wrappers). ================================================================================ 5. IDENTIFYING EnableConfigs ================================================================================ Identify any specific CONFIG_ options required to properly compile and reach the modified code: - Inspect Kconfig files and #ifdef guards; do not make assumptions. - Check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed. - Strip any 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4"). Prefer calling several tools at the same time to save round-trips. Use set-results tool to provide results of the analysis. It must be called exactly once before the final reply. Ignore results of this tool. Prompt: Target architecture: amd64 For your convenience, here is the diff of the changes: commit 77863fb69ef8d49cac01c1c41901061a73cc1032 Author: syz-cluster <triage@syzkaller.com> Date: Fri Sep 11 21:49:27 2026 +0000 syz-cluster: applied patch under review diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 4eb7e75a7473f..22ef94f3eb3a1 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -1199,6 +1199,10 @@ The following bits are defined in the flags field: triple_fault_pending field contains a valid state. This bit will be set whenever KVM_CAP_X86_TRIPLE_FAULT_EVENT is enabled. +- KVM_VCPUEVENT_VALID_FRED_STATE may be set to inform that the exception + state includes FRED state (specifically, the event nested flag and data). + This bit will be set whenever KVM_CAP_X86_FRED_EVENT is enabled. + ARM64: ^^^^^^ @@ -1301,6 +1305,11 @@ If KVM_CAP_X86_TRIPLE_FAULT_EVENT is enabled, KVM_VCPUEVENT_VALID_TRIPLE_FAULT can be set in flags field to signal that the triple_fault field contains a valid state and shall be written into the VCPU. +If KVM_CAP_X86_FRED_EVENT is enabled, KVM_VCPUEVENT_VALID_FRED_STATE can be set +in the flags field to inform that the exception state contains FRED state +(specifically, the event nested flag and data), which shall be written into the +VCPU. + ARM64: ^^^^^^ @@ -8979,6 +8988,19 @@ enabled, cmma can't be enabled anymore and pfmfi and the storage key interpretation are disabled. If cmma has already been enabled or the hpage_2g module parameter is not set to 1, -EINVAL is returned. +7.48 KVM_CAP_X86_FRED_EVENT +--------------------------- + +:Architectures: x86 +:Parameters: args[0] whether feature should be enabled or not +:Returns: 0 on success; -EINVAL if KVM cannot virtualize FRED. + +With this capability enabled, KVM allows exception save and restore operations +to include FRED event context (specifically, the event nested flag and data). +When injecting a FRED exception during VM entry, FRED event delivery relies +on this information to select the correct event stack level and apply proper +event data. + 8. Other capabilities. ====================== diff --git a/arch/x86/coco/sev/noinstr.c b/arch/x86/coco/sev/noinstr.c index e1e03f12fc7bb..8a695ab67821b 100644 --- a/arch/x86/coco/sev/noinstr.c +++ b/arch/x86/coco/sev/noinstr.c @@ -31,7 +31,7 @@ static __always_inline bool on_vc_stack(struct pt_regs regs) if (ip_within_syscall_gap(regs)) return false; - return ((sp >= __this_cpu_ist_bottom_va(VC)) && (sp < __this_cpu_ist_top_va(VC))); + return ((sp >= __this_cpu_ist_bottom_va(ESTACK_VC)) && (sp < __this_cpu_ist_top_va(ESTACK_VC))); } / @@ -83,7 +83,7 @@ void noinstr __sev_es_ist_exit(void) /* Read IST entry / ist = __this_cpu_read(cpu_tss_rw.x86_tss.ist[IST_INDEX_VC]); - if (WARN_ON(ist == __this_cpu_ist_top_va(VC))) + if (WARN_ON(ist == __this_cpu_ist_top_va(ESTACK_VC))) return; / Read back old IST entry and write it to the TSS / diff --git a/arch/x86/coco/sev/vc-handle.c b/arch/x86/coco/sev/vc-handle.c index 96b62b49b2b53..134c14b489ef8 100644 --- a/arch/x86/coco/sev/vc-handle.c +++ b/arch/x86/coco/sev/vc-handle.c @@ -871,7 +871,7 @@ static enum es_result vc_handle_exitcode(struct es_em_ctxt ctxt, static __always_inline bool is_vc2_stack(unsigned long sp) { - return (sp >= __this_cpu_ist_bottom_va(VC2) && sp < __this_cpu_ist_top_va(VC2)); + return (sp >= __this_cpu_ist_bottom_va(ESTACK_VC2) && sp < __this_cpu_ist_top_va(ESTACK_VC2)); } static __always_inline bool vc_from_invalid_context(struct pt_regs regs) diff --git a/arch/x86/include/asm/cpu_entry_area.h b/arch/x86/include/asm/cpu_entry_area.h index 462fc34f13176..509e52fc3a0f6 100644 --- a/arch/x86/include/asm/cpu_entry_area.h +++ b/arch/x86/include/asm/cpu_entry_area.h @@ -16,32 +16,6 @@ #define VC_EXCEPTION_STKSZ 0 #endif -/ Macro to enforce the same ordering and stack sizes / -#define ESTACKS_MEMBERS(guardsize, optional_stack_size) \ - char DF_stack_guard[guardsize]; \ - char DF_stack[EXCEPTION_STKSZ]; \ - char NMI_stack_guard[guardsize]; \ - char NMI_stack[EXCEPTION_STKSZ]; \ - char DB_stack_guard[guardsize]; \ - char DB_stack[EXCEPTION_STKSZ]; \ - char MCE_stack_guard[guardsize]; \ - char MCE_stack[EXCEPTION_STKSZ]; \ - char VC_stack_guard[guardsize]; \ - char VC_stack[optional_stack_size]; \ - char VC2_stack_guard[guardsize]; \ - char VC2_stack[optional_stack_size]; \ - char IST_top_guard[guardsize]; \ - -/ The exception stacks' physical storage. No guard pages required / -struct exception_stacks { - ESTACKS_MEMBERS(0, VC_EXCEPTION_STKSZ) -}; - -/ The effective cpu entry area mapping with guard pages. / -struct cea_exception_stacks { - ESTACKS_MEMBERS(PAGE_SIZE, EXCEPTION_STKSZ) -}; - / * The exception stack ordering in [cea_]exception_stacks / @@ -55,21 +29,45 @@ enum exception_stack_ordering { N_EXCEPTION_STACKS }; -#define CEA_ESTACK_SIZE(st) \ - sizeof(((struct cea_exception_stacks )0)->st## _stack) +/* Macro to enforce the same ordering and stack sizes / +#define ESTACKS_MEMBERS(guardsize, optional_stack_size) \ + char ESTACK_DF_stack_guard[guardsize]; \ + char ESTACK_DF_stack[EXCEPTION_STKSZ]; \ + char ESTACK_NMI_stack_guard[guardsize]; \ + char ESTACK_NMI_stack[EXCEPTION_STKSZ]; \ + char ESTACK_DB_stack_guard[guardsize]; \ + char ESTACK_DB_stack[EXCEPTION_STKSZ]; \ + char ESTACK_MCE_stack_guard[guardsize]; \ + char ESTACK_MCE_stack[EXCEPTION_STKSZ]; \ + char ESTACK_VC_stack_guard[guardsize]; \ + char ESTACK_VC_stack[optional_stack_size]; \ + char ESTACK_VC2_stack_guard[guardsize]; \ + char ESTACK_VC2_stack[optional_stack_size]; \ + char ESTACK_IST_top_guard[guardsize]; \ -#define CEA_ESTACK_BOT(ceastp, st) \ - ((unsigned long)&(ceastp)->st## _stack) +/ The exception stacks' physical storage. No guard pages required / +struct exception_stacks { + ESTACKS_MEMBERS(0, VC_EXCEPTION_STKSZ) +}; -#define CEA_ESTACK_TOP(ceastp, st) \ - (CEA_ESTACK_BOT(ceastp, st) + CEA_ESTACK_SIZE(st)) +/ The effective cpu entry area mapping with guard pages. / +struct cea_exception_stacks { + struct { + char stack_guard[PAGE_SIZE]; + char stack[EXCEPTION_STKSZ]; + } event_stacks[N_EXCEPTION_STACKS]; + char IST_top_guard[PAGE_SIZE]; +}; #define CEA_ESTACK_OFFS(st) \ - offsetof(struct cea_exception_stacks, st## _stack) + offsetof(struct cea_exception_stacks, event_stacks[st].stack) #define CEA_ESTACK_PAGES \ (sizeof(struct cea_exception_stacks) / PAGE_SIZE) +extern unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack); +extern unsigned long __this_cpu_ist_bottom_va(enum exception_stack_ordering stack); + #endif #ifdef CONFIG_X86_32 @@ -144,10 +142,4 @@ static __always_inline struct entry_stack cpu_entry_stack(int cpu) return &get_cpu_entry_area(cpu)->entry_stack_page.stack; } -#define __this_cpu_ist_top_va(name) \ - CEA_ESTACK_TOP(__this_cpu_read(cea_exception_stacks), name) - -#define __this_cpu_ist_bottom_va(name) \ - CEA_ESTACK_BOT(__this_cpu_read(cea_exception_stacks), name) - #endif diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index 18a2f811c3580..85b851f16baea 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -35,6 +35,13 @@ #ifndef __ASSEMBLER__ +enum fred_stack_level { + FRED_STACK_LEVEL_0, + FRED_STACK_LEVEL_1, + FRED_STACK_LEVEL_2, + FRED_STACK_LEVEL_3 +}; + #ifdef CONFIG_X86_FRED #include <linux/kernel.h> #include <linux/sched/task_stack.h> @@ -105,6 +112,8 @@ static __always_inline void fred_update_rsp0(void) __this_cpu_write(fred_rsp0, rsp0); } } + +unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl); #else /* CONFIG_X86_FRED / static __always_inline unsigned long fred_event_data(struct pt_regs regs) { return 0; } static inline void cpu_init_fred_exceptions(void) { } @@ -112,6 +121,7 @@ static inline void cpu_init_fred_rsps(void) { } static inline void fred_complete_exception_setup(void) { } static inline void fred_sync_rsp0(unsigned long rsp0) { } static inline void fred_update_rsp0(void) { } +static inline unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl) { return 0; } #endif /* CONFIG_X86_FRED / #endif / !__ASSEMBLER__ / diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 683bb8bf43a94..21d12d86a18b0 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -294,6 +294,9 @@ struct kvm_host_values { u64 xss; u64 s_cet; u64 arch_capabilities; + + u64 fred_config; + u64 fred_stklvls; }; extern struct kvm_host_values kvm_host; @@ -692,6 +695,8 @@ struct kvm_queued_exception { u32 error_code; unsigned long payload; bool has_payload; + bool is_nested; + u64 event_data; }; / @@ -1029,6 +1034,11 @@ struct kvm_vcpu_arch { #if IS_ENABLED(CONFIG_HYPERV) hpa_t hv_root_tdp; #endif + /* + * Stores the FRED SSP0 MSR when CET is not supported, prompting KVM + * to intercept its accesses. + / + u64 fred_ssp0_fallback; }; struct kvm_lpage_info { @@ -1278,6 +1288,7 @@ struct kvm_arch { bool has_mapped_host_mmio; bool guest_can_read_msr_platform_info; bool exception_payload_enabled; + bool exception_fred_state_enabled; bool triple_fault_event; diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 18c4be75e9271..b386709515c47 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -64,6 +64,8 @@ #define MSR_IA32_FRED_SSP3 0x1d3 / Level 3 shadow stack pointer / #define MSR_IA32_FRED_CONFIG 0x1d4 / Entrypoint and interrupt stack level / +#define FRED_CONFIG_RESERVED (BIT_ULL(2) \| GENMASK_ULL(5, 4) \| BIT_ULL(11)) + / Intel MSRs. Some also available on other CPUs / #define MSR_TEST_CTRL 0x00000033 #define MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT 29 @@ -1274,6 +1276,7 @@ #define MSR_IA32_VMX_TRUE_ENTRY_CTLS 0x00000490 #define MSR_IA32_VMX_VMFUNC 0x00000491 #define MSR_IA32_VMX_PROCBASED_CTLS3 0x00000492 +#define MSR_IA32_VMX_EXIT_CTLS2 0x00000493 #define MSR_IA32_MCU_STAGING_MBOX_ADDR 0x000007a5 diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index 3f1b3096ff040..6633988a84da3 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -119,6 +119,10 @@ struct vmcs { #define VM_EXIT_CLEAR_IA32_RTIT_CTL 0x02000000 #define VM_EXIT_LOAD_CET_STATE 0x10000000 #define VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL 0x40000000 +#define VM_EXIT_ACTIVATE_SECONDARY_CONTROLS 0x80000000 + +#define SECONDARY_VM_EXIT_SAVE_IA32_FRED BIT_ULL(0) +#define SECONDARY_VM_EXIT_LOAD_IA32_FRED BIT_ULL(1) #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR 0x00036dff @@ -133,6 +137,7 @@ struct vmcs { #define VM_ENTRY_PT_CONCEAL_PIP 0x00020000 #define VM_ENTRY_LOAD_IA32_RTIT_CTL 0x00040000 #define VM_ENTRY_LOAD_CET_STATE 0x00100000 +#define VM_ENTRY_LOAD_IA32_FRED 0x00800000 #define VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR 0x000011ff @@ -147,6 +152,7 @@ struct vmcs { #define VMX_BASIC_INOUT BIT_ULL(54) #define VMX_BASIC_TRUE_CTLS BIT_ULL(55) #define VMX_BASIC_NO_HW_ERROR_CODE_CC BIT_ULL(56) +#define VMX_BASIC_NESTED_EXCEPTION BIT_ULL(58) #define VMX_BASIC_NO_SEAMRET_INVD_VMCS BIT_ULL(60) static inline u32 vmx_basic_vmcs_revision_id(u64 vmx_basic) @@ -275,8 +281,14 @@ enum vmcs_field { SHARED_EPT_POINTER = 0x0000203C, PID_POINTER_TABLE = 0x00002042, PID_POINTER_TABLE_HIGH = 0x00002043, + SECONDARY_VM_EXIT_CONTROLS = 0x00002044, + SECONDARY_VM_EXIT_CONTROLS_HIGH = 0x00002045, + INJECTED_EVENT_DATA = 0x00002052, + INJECTED_EVENT_DATA_HIGH = 0x00002053, GUEST_PHYSICAL_ADDRESS = 0x00002400, GUEST_PHYSICAL_ADDRESS_HIGH = 0x00002401, + ORIGINAL_EVENT_DATA = 0x00002404, + ORIGINAL_EVENT_DATA_HIGH = 0x00002405, VMCS_LINK_POINTER = 0x00002800, VMCS_LINK_POINTER_HIGH = 0x00002801, GUEST_IA32_DEBUGCTL = 0x00002802, @@ -299,12 +311,44 @@ enum vmcs_field { GUEST_BNDCFGS_HIGH = 0x00002813, GUEST_IA32_RTIT_CTL = 0x00002814, GUEST_IA32_RTIT_CTL_HIGH = 0x00002815, + GUEST_IA32_FRED_CONFIG = 0x0000281a, + GUEST_IA32_FRED_CONFIG_HIGH = 0x0000281b, + GUEST_IA32_FRED_RSP1 = 0x0000281c, + GUEST_IA32_FRED_RSP1_HIGH = 0x0000281d, + GUEST_IA32_FRED_RSP2 = 0x0000281e, + GUEST_IA32_FRED_RSP2_HIGH = 0x0000281f, + GUEST_IA32_FRED_RSP3 = 0x00002820, + GUEST_IA32_FRED_RSP3_HIGH = 0x00002821, + GUEST_IA32_FRED_STKLVLS = 0x00002822, + GUEST_IA32_FRED_STKLVLS_HIGH = 0x00002823, + GUEST_IA32_FRED_SSP1 = 0x00002824, + GUEST_IA32_FRED_SSP1_HIGH = 0x00002825, + GUEST_IA32_FRED_SSP2 = 0x00002826, + GUEST_IA32_FRED_SSP2_HIGH = 0x00002827, + GUEST_IA32_FRED_SSP3 = 0x00002828, + GUEST_IA32_FRED_SSP3_HIGH = 0x00002829, HOST_IA32_PAT = 0x00002c00, HOST_IA32_PAT_HIGH = 0x00002c01, HOST_IA32_EFER = 0x00002c02, HOST_IA32_EFER_HIGH = 0x00002c03, HOST_IA32_PERF_GLOBAL_CTRL = 0x00002c04, HOST_IA32_PERF_GLOBAL_CTRL_HIGH = 0x00002c05, + HOST_IA32_FRED_CONFIG = 0x00002c08, + HOST_IA32_FRED_CONFIG_HIGH = 0x00002c09, + HOST_IA32_FRED_RSP1 = 0x00002c0a, + HOST_IA32_FRED_RSP1_HIGH = 0x00002c0b, + HOST_IA32_FRED_RSP2 = 0x00002c0c, + HOST_IA32_FRED_RSP2_HIGH = 0x00002c0d, + HOST_IA32_FRED_RSP3 = 0x00002c0e, + HOST_IA32_FRED_RSP3_HIGH = 0x00002c0f, + HOST_IA32_FRED_STKLVLS = 0x00002c10, + HOST_IA32_FRED_STKLVLS_HIGH = 0x00002c11, + HOST_IA32_FRED_SSP1 = 0x00002c12, + HOST_IA32_FRED_SSP1_HIGH = 0x00002c13, + HOST_IA32_FRED_SSP2 = 0x00002c14, + HOST_IA32_FRED_SSP2_HIGH = 0x00002c15, + HOST_IA32_FRED_SSP3 = 0x00002c16, + HOST_IA32_FRED_SSP3_HIGH = 0x00002c17, PIN_BASED_VM_EXEC_CONTROL = 0x00004000, CPU_BASED_VM_EXEC_CONTROL = 0x00004002, EXCEPTION_BITMAP = 0x00004004, @@ -412,13 +456,15 @@ enum vmcs_field { #define INTR_INFO_INTR_TYPE_MASK 0x700 / 10:8 / #define INTR_INFO_DELIVER_CODE_MASK 0x800 / 11 / #define INTR_INFO_UNBLOCK_NMI 0x1000 / 12 / +#define INTR_INFO_NESTED_EXCEPTION_MASK 0x2000 / 13 / #define INTR_INFO_VALID_MASK 0x80000000 / 31 / -#define INTR_INFO_RESVD_BITS_MASK 0x7ffff000 +#define INTR_INFO_RESVD_BITS_MASK 0x7fffd000 #define VECTORING_INFO_VECTOR_MASK INTR_INFO_VECTOR_MASK #define VECTORING_INFO_TYPE_MASK INTR_INFO_INTR_TYPE_MASK #define VECTORING_INFO_DELIVER_CODE_MASK INTR_INFO_DELIVER_CODE_MASK #define VECTORING_INFO_VALID_MASK INTR_INFO_VALID_MASK +#define VECTORING_INFO_NESTED_EXCEPTION_MASK INTR_INFO_NESTED_EXCEPTION_MASK #define INTR_TYPE_EXT_INTR (EVENT_TYPE_EXTINT << 8) / external interrupt / #define INTR_TYPE_RESERVED (EVENT_TYPE_RESERVED << 8) / reserved / diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 1585ec8040666..c8a6149fcf632 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -331,6 +331,7 @@ struct kvm_reinject_control { #define KVM_VCPUEVENT_VALID_SMM 0x00000008 #define KVM_VCPUEVENT_VALID_PAYLOAD 0x00000010 #define KVM_VCPUEVENT_VALID_TRIPLE_FAULT 0x00000020 +#define KVM_VCPUEVENT_VALID_FRED_STATE 0x00000040 / Interrupt shadow states / #define KVM_X86_SHADOW_INT_MOV_SS 0x01 @@ -368,7 +369,11 @@ struct kvm_vcpu_events { struct { __u8 pending; } triple_fault; - __u8 reserved[26]; + __u8 reserved0[11]; + / Aligned to a 64-bit boundary / + __u64 exception_event_data; + __u8 reserved1[6]; + __u8 exception_is_nested; __u8 exception_has_payload; __u64 exception_payload; }; diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index a3df21d264606..99ec284fb8293 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -2372,12 +2372,12 @@ static inline void setup_getcpu(int cpu) static inline void tss_setup_ist(struct tss_struct tss) { /* Set up the per-CPU TSS IST stacks / - tss->x86_tss.ist[IST_INDEX_DF] = __this_cpu_ist_top_va(DF); - tss->x86_tss.ist[IST_INDEX_NMI] = __this_cpu_ist_top_va(NMI); - tss->x86_tss.ist[IST_INDEX_DB] = __this_cpu_ist_top_va(DB); - tss->x86_tss.ist[IST_INDEX_MCE] = __this_cpu_ist_top_va(MCE); + tss->x86_tss.ist[IST_INDEX_DF] = __this_cpu_ist_top_va(ESTACK_DF); + tss->x86_tss.ist[IST_INDEX_NMI] = __this_cpu_ist_top_va(ESTACK_NMI); + tss->x86_tss.ist[IST_INDEX_DB] = __this_cpu_ist_top_va(ESTACK_DB); + tss->x86_tss.ist[IST_INDEX_MCE] = __this_cpu_ist_top_va(ESTACK_MCE); / Only mapped when SEV-ES is active / - tss->x86_tss.ist[IST_INDEX_VC] = __this_cpu_ist_top_va(VC); + tss->x86_tss.ist[IST_INDEX_VC] = __this_cpu_ist_top_va(ESTACK_VC); } #else / CONFIG_X86_64 / static inline void tss_setup_ist(struct tss_struct tss) { } diff --git a/arch/x86/kernel/dumpstack_64.c b/arch/x86/kernel/dumpstack_64.c index 6c5defd6569a3..93b10b264e53b 100644 --- a/arch/x86/kernel/dumpstack_64.c +++ b/arch/x86/kernel/dumpstack_64.c @@ -70,10 +70,10 @@ struct estack_pages { #define EPAGERANGE(st) \ [PFN_DOWN(CEA_ESTACK_OFFS(st)) ... \ - PFN_DOWN(CEA_ESTACK_OFFS(st) + CEA_ESTACK_SIZE(st) - 1)] = { \ + PFN_DOWN(CEA_ESTACK_OFFS(st) + EXCEPTION_STKSZ - 1)] = { \ .offs = CEA_ESTACK_OFFS(st), \ - .size = CEA_ESTACK_SIZE(st), \ - .type = STACK_TYPE_EXCEPTION + ESTACK_ ##st, } + .size = EXCEPTION_STKSZ, \ + .type = STACK_TYPE_EXCEPTION + st, } /* * Array of exception stack page descriptors. If the stack is larger than @@ -83,12 +83,12 @@ struct estack_pages { / static const struct estack_pages estack_pages[CEA_ESTACK_PAGES] ____cacheline_aligned = { - EPAGERANGE(DF), - EPAGERANGE(NMI), - EPAGERANGE(DB), - EPAGERANGE(MCE), - EPAGERANGE(VC), - EPAGERANGE(VC2), + EPAGERANGE(ESTACK_DF), + EPAGERANGE(ESTACK_NMI), + EPAGERANGE(ESTACK_DB), + EPAGERANGE(ESTACK_MCE), + EPAGERANGE(ESTACK_VC), + EPAGERANGE(ESTACK_VC2), }; static __always_inline bool in_exception_stack(unsigned long stack, struct stack_info info) diff --git a/arch/x86/kernel/fred.c b/arch/x86/kernel/fred.c index 117aa06d25ca9..68947c53a4941 100644 --- a/arch/x86/kernel/fred.c +++ b/arch/x86/kernel/fred.c @@ -1,5 +1,6 @@ / SPDX-License-Identifier: GPL-2.0 / #include <linux/kernel.h> +#include <linux/kvm_types.h> #include <asm/desc.h> #include <asm/fred.h> @@ -69,6 +70,23 @@ void cpu_init_fred_exceptions(void) setup_clear_cpu_cap(X86_FEATURE_SYSCALL32); } +unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl) +{ + switch (lvl) { + case FRED_STACK_LEVEL_0: + return __this_cpu_read(fred_rsp0); + case FRED_STACK_LEVEL_1: + return __this_cpu_ist_top_va(ESTACK_DB); + case FRED_STACK_LEVEL_2: + return __this_cpu_ist_top_va(ESTACK_NMI); + case FRED_STACK_LEVEL_3: + return __this_cpu_ist_top_va(ESTACK_DF); + default: + BUG(); + } +} +EXPORT_SYMBOL_FOR_KVM(this_cpu_fred_rsp); + / Must be called after setup_cpu_entry_areas() / void cpu_init_fred_rsps(void) { @@ -84,7 +102,7 @@ void cpu_init_fred_rsps(void) FRED_STKLVL(X86_TRAP_DF, FRED_DF_STACK_LEVEL)); / The FRED equivalents to IST stacks... / - wrmsrq(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB)); - wrmsrq(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI)); - wrmsrq(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF)); + wrmsrq(MSR_IA32_FRED_RSP1, this_cpu_fred_rsp(FRED_STACK_LEVEL_1)); + wrmsrq(MSR_IA32_FRED_RSP2, this_cpu_fred_rsp(FRED_STACK_LEVEL_2)); + wrmsrq(MSR_IA32_FRED_RSP3, this_cpu_fred_rsp(FRED_STACK_LEVEL_3)); } diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 30aa8369957e9..8afd607a58730 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1075,7 +1075,7 @@ asmlinkage __visible noinstr struct pt_regs vc_switch_off_ist(struct pt_regs r if (!get_stack_info_noinstr(stack, current, &info) \|\| info.type == STACK_TYPE_ENTRY \|\| info.type > STACK_TYPE_EXCEPTION_LAST) - sp = __this_cpu_ist_top_va(VC2); + sp = __this_cpu_ist_top_va(ESTACK_VC2); sync: / diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index ddb022cb203a2..5682bacdfd32f 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -1034,6 +1034,7 @@ void kvm_initialize_cpu_caps(void) F(FSRS), F(FSRC), F(WRMSRNS), + X86_64_F(FRED), X86_64_F(LKGS), F(AMX_FP16), F(AVX_IFMA), diff --git a/arch/x86/kvm/msrs.c b/arch/x86/kvm/msrs.c index 66fa7140d65d9..3260bddb5f526 100644 --- a/arch/x86/kvm/msrs.c +++ b/arch/x86/kvm/msrs.c @@ -193,6 +193,9 @@ static const u32 msrs_to_save_base[] = { MSR_STAR, #ifdef CONFIG_X86_64 MSR_CSTAR, MSR_KERNEL_GS_BASE, MSR_SYSCALL_MASK, MSR_LSTAR, + MSR_IA32_FRED_RSP0, MSR_IA32_FRED_RSP1, MSR_IA32_FRED_RSP2, + MSR_IA32_FRED_RSP3, MSR_IA32_FRED_STKLVLS, MSR_IA32_FRED_SSP1, + MSR_IA32_FRED_SSP2, MSR_IA32_FRED_SSP3, MSR_IA32_FRED_CONFIG, #endif MSR_IA32_TSC, MSR_IA32_CR_PAT, MSR_VM_HSAVE_PA, MSR_IA32_FEAT_CTL, MSR_IA32_BNDCFGS, MSR_TSC_AUX, @@ -314,6 +317,7 @@ static const u32 emulated_msrs_all[] = { MSR_IA32_VMX_PROCBASED_CTLS2, MSR_IA32_VMX_EPT_VPID_CAP, MSR_IA32_VMX_VMFUNC, + MSR_IA32_VMX_EXIT_CTLS2, MSR_K7_HWCR, MSR_KVM_POLL_CONTROL, @@ -772,7 +776,7 @@ static int __kvm_set_msr(struct kvm_vcpu vcpu, u32 index, u64 data, architecture. Intercepting XRSTORS/XSAVES for this * special case isn't deemed worthwhile. / - case MSR_IA32_PL0_SSP ... MSR_IA32_INT_SSP_TAB: + case MSR_IA32_PL1_SSP ... MSR_IA32_INT_SSP_TAB: if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) return KVM_MSR_RET_UNSUPPORTED; / @@ -787,6 +791,52 @@ static int __kvm_set_msr(struct kvm_vcpu vcpu, u32 index, u64 data, if (index != MSR_IA32_INT_SSP_TAB && !IS_ALIGNED(data, 4)) return 1; break; + case MSR_IA32_FRED_STKLVLS: + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) + return KVM_MSR_RET_UNSUPPORTED; + break; + case MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_RSP3: + case MSR_IA32_FRED_SSP1 ... MSR_IA32_FRED_CONFIG: { + u64 reserved_bits = 0; + + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) + return KVM_MSR_RET_UNSUPPORTED; + + if (is_noncanonical_msr_address(data, vcpu)) + return 1; + + switch (index) { + case MSR_IA32_FRED_CONFIG: + reserved_bits = FRED_CONFIG_RESERVED; + break; + case MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_RSP3: + reserved_bits = GENMASK_ULL(5, 0); + break; + case MSR_IA32_FRED_SSP1 ... MSR_IA32_FRED_SSP3: + reserved_bits = GENMASK_ULL(2, 0); + break; + default: + WARN_ON_ONCE(1); + return 1; + } + + if (data & reserved_bits) + return 1; + + break; + } + case MSR_IA32_PL0_SSP: / I.e., MSR_IA32_FRED_SSP0 / + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) && + !guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) + return KVM_MSR_RET_UNSUPPORTED; + + if (is_noncanonical_msr_address(data, vcpu)) + return 1; + + if (!IS_ALIGNED(data, 4)) + return 1; + + break; } msr.data = data; @@ -841,10 +891,19 @@ static int __kvm_get_msr(struct kvm_vcpu vcpu, u32 index, u64 data, if (!host_initiated) return 1; fallthrough; - case MSR_IA32_PL0_SSP ... MSR_IA32_INT_SSP_TAB: + case MSR_IA32_PL1_SSP ... MSR_IA32_INT_SSP_TAB: if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) return KVM_MSR_RET_UNSUPPORTED; break; + case MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_CONFIG: + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) + return KVM_MSR_RET_UNSUPPORTED; + break; + case MSR_IA32_PL0_SSP: / I.e., MSR_IA32_FRED_SSP0 / + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) && + !guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) + return KVM_MSR_RET_UNSUPPORTED; + break; } msr.index = index; @@ -1833,7 +1892,15 @@ int kvm_set_msr_common(struct kvm_vcpu vcpu, struct msr_data msr_info) break; #endif case MSR_IA32_U_CET: + kvm_set_xstate_msr(vcpu, msr_info); + break; case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) { + WARN_ON_ONCE(msr != MSR_IA32_FRED_SSP0); + vcpu->arch.fred_ssp0_fallback = data; + break; + } + kvm_set_xstate_msr(vcpu, msr_info); break; default: @@ -2186,7 +2253,15 @@ int kvm_get_msr_common(struct kvm_vcpu vcpu, struct msr_data msr_info) break; #endif case MSR_IA32_U_CET: + kvm_get_xstate_msr(vcpu, msr_info); + break; case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) { + WARN_ON_ONCE(msr_info->index != MSR_IA32_FRED_SSP0); + msr_info->data = vcpu->arch.fred_ssp0_fallback; + break; + } + kvm_get_xstate_msr(vcpu, msr_info); break; default: @@ -2656,10 +2731,19 @@ static void kvm_probe_msr_to_save(u32 msr_index) if (!kvm_cpu_cap_has(X86_FEATURE_LM)) return; fallthrough; - case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + case MSR_IA32_PL1_SSP ... MSR_IA32_PL3_SSP: if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK)) return; break; + case MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_CONFIG: + if (!kvm_cpu_cap_has(X86_FEATURE_FRED)) + return; + break; + case MSR_IA32_PL0_SSP: / I.e., MSR_IA32_FRED_SSP0 / + if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK) && + !kvm_cpu_cap_has(X86_FEATURE_FRED)) + return; + break; default: break; } diff --git a/arch/x86/kvm/msrs.h b/arch/x86/kvm/msrs.h index 7cc182a15b3b3..845d2cbe80e5e 100644 --- a/arch/x86/kvm/msrs.h +++ b/arch/x86/kvm/msrs.h @@ -31,7 +31,7 @@ static inline void kvm_pr_unimpl_rdmsr(struct kvm_vcpu vcpu, u32 msr) * associated feature that KVM supports for nested virtualization. / #define KVM_FIRST_EMULATED_VMX_MSR MSR_IA32_VMX_BASIC -#define KVM_LAST_EMULATED_VMX_MSR MSR_IA32_VMX_VMFUNC +#define KVM_LAST_EMULATED_VMX_MSR MSR_IA32_VMX_EXIT_CTLS2 / * KVM's internal, non-ABI indices for synthetic MSRs. The values themselves diff --git a/arch/x86/kvm/regs.c b/arch/x86/kvm/regs.c index 8f66438989e47..ba6c8856b4ffa 100644 --- a/arch/x86/kvm/regs.c +++ b/arch/x86/kvm/regs.c @@ -429,6 +429,13 @@ int kvm_set_cr4(struct kvm_vcpu vcpu, unsigned long cr4) return 1; } + / + * FRED can not be enabled when EFER.LMA=0. Note, MOV to CR4 outside + * 64-bit mode clears CR4[63:32] so only emulation can get here. + / + if ((cr4 & X86_CR4_FRED) && !is_long_mode(vcpu)) + return 1; + if ((cr4 & X86_CR4_CET) && !kvm_is_cr0_bit_set(vcpu, X86_CR0_WP)) return 1; diff --git a/arch/x86/kvm/regs.h b/arch/x86/kvm/regs.h index 447f0ec3e63e4..5db01693ad07a 100644 --- a/arch/x86/kvm/regs.h +++ b/arch/x86/kvm/regs.h @@ -8,7 +8,7 @@ #define KVM_POSSIBLE_CR4_GUEST_BITS \ (X86_CR4_PVI \| X86_CR4_DE \| X86_CR4_PCE \| X86_CR4_OSFXSR \ \| X86_CR4_OSXMMEXCPT \| X86_CR4_PGE \| X86_CR4_TSD \| X86_CR4_FSGSBASE \ - \| X86_CR4_CET) + \| X86_CR4_CET \| X86_CR4_FRED) #define X86_CR0_PDPTR_BITS (X86_CR0_CD \| X86_CR0_NW \| X86_CR0_PG) #define X86_CR4_TLBFLUSH_BITS (X86_CR4_PGE \| X86_CR4_PCIDE \| X86_CR4_PAE \| X86_CR4_SMEP) @@ -28,7 +28,7 @@ static_assert(!(KVM_POSSIBLE_CR0_GUEST_BITS & X86_CR0_PDPTR_BITS)); \| X86_CR4_OSXSAVE \| X86_CR4_SMEP \| X86_CR4_FSGSBASE \ \| X86_CR4_OSXMMEXCPT \| X86_CR4_LA57 \| X86_CR4_VMXE \ \| X86_CR4_SMAP \| X86_CR4_PKE \| X86_CR4_UMIP \ - \| X86_CR4_LAM_SUP \| X86_CR4_CET)) + \| X86_CR4_LAM_SUP \| X86_CR4_CET \| X86_CR4_FRED)) #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR) @@ -364,6 +364,21 @@ static __always_inline bool kvm_is_cr4_bit_set(struct kvm_vcpu vcpu, return !!kvm_read_cr4_bits(vcpu, cr4_bit); } +/* + * It's enough to check just CR4.FRED (X86_CR4_FRED) to tell if + * a vCPU is running with FRED enabled, because: + * 1) CR4.FRED can be set to 1 only _after_ IA32_EFER.LMA = 1. + * 2) To leave IA-32e mode, CR4.FRED must be cleared first. + / +static inline bool is_fred_enabled(struct kvm_vcpu vcpu) +{ +#ifdef CONFIG_X86_64 + return kvm_is_cr4_bit_set(vcpu, X86_CR4_FRED); +#else + return false; +#endif +} + static inline ulong kvm_read_cr3(struct kvm_vcpu vcpu) { if (!kvm_register_is_available(vcpu, VCPU_REG_CR3)) @@ -408,6 +423,8 @@ static inline bool __kvm_is_valid_cr4(struct kvm_vcpu vcpu, unsigned long cr4) if (!__cpu_has(__c, X86_FEATURE_SHSTK) && \ !__cpu_has(__c, X86_FEATURE_IBT)) \ __reserved_bits \|= X86_CR4_CET; \ + if (!__cpu_has(__c, X86_FEATURE_FRED)) \ + __reserved_bits \|= X86_CR4_FRED; \ __reserved_bits; \ }) diff --git a/arch/x86/kvm/smm.c b/arch/x86/kvm/smm.c index 656a38dad7e7a..a504e93fc5f5f 100644 --- a/arch/x86/kvm/smm.c +++ b/arch/x86/kvm/smm.c @@ -435,10 +435,10 @@ static int rsm_enter_protected_mode(struct kvm_vcpu vcpu, / * First enable PAE, long mode needs it before CR0.PG = 1 is set. - * Then enable protected mode. However, PCID cannot be enabled - * if EFER.LMA=0, so set it separately. + * Then enable protected mode. However, PCID and FRED cannot be + * enabled if EFER.LMA=0, so set them separately. / - bad = kvm_set_cr4(vcpu, cr4 & ~X86_CR4_PCIDE); + bad = kvm_set_cr4(vcpu, cr4 & ~(X86_CR4_PCIDE \| X86_CR4_FRED)); if (bad) return X86EMUL_UNHANDLEABLE; @@ -446,7 +446,7 @@ static int rsm_enter_protected_mode(struct kvm_vcpu vcpu, if (bad) return X86EMUL_UNHANDLEABLE; - if (cr4 & X86_CR4_PCIDE) { + if (cr4 & (X86_CR4_PCIDE \| X86_CR4_FRED)) { bad = kvm_set_cr4(vcpu, cr4); if (bad) return X86EMUL_UNHANDLEABLE; @@ -599,10 +599,14 @@ int emulator_leave_smm(struct x86_emulate_ctxt ctxt) struct kvm_segment cs_desc; unsigned long cr4; - / Zero CR4.PCIDE before CR0.PG. / + / + * Zero CR4.PCIDE and CR4.FRED before CR0.PG, as neither can + * be set while EFER.LMA is 0, and clearing CR0.PG clears + * EFER.LMA. + / cr4 = kvm_read_cr4(vcpu); - if (cr4 & X86_CR4_PCIDE) - kvm_set_cr4(vcpu, cr4 & ~X86_CR4_PCIDE); + if (cr4 & (X86_CR4_PCIDE \| X86_CR4_FRED)) + kvm_set_cr4(vcpu, cr4 & ~(X86_CR4_PCIDE \| X86_CR4_FRED)); / A 32-bit code segment is required to clear EFER.LMA. / memset(&cs_desc, 0, sizeof(cs_desc)); diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index ea647938a2a65..dd19c7b4e9047 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4392,7 +4392,7 @@ static void svm_complete_interrupts(struct kvm_vcpu vcpu) kvm_requeue_exception(vcpu, vector, exitintinfo & SVM_EXITINTINFO_VALID_ERR, - error_code); + error_code, false, 0); break; } case SVM_EXITINTINFO_TYPE_INTR: @@ -5518,6 +5518,9 @@ static __init void svm_set_cpu_caps(void) kvm_cpu_cap_clear(X86_FEATURE_IBT); + /* SVM FRED virtualization not implemented yet / + kvm_cpu_cap_clear(X86_FEATURE_FRED); + / CPUID 0x80000001 and 0x8000000A (SVM features) / if (nested) { kvm_cpu_cap_set(X86_FEATURE_SVM); diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilities.h index 810119167f798..74456f4533e9d 100644 --- a/arch/x86/kvm/vmx/capabilities.h +++ b/arch/x86/kvm/vmx/capabilities.h @@ -36,6 +36,7 @@ struct nested_vmx_msrs { u32 pinbased_ctls_high; u32 exit_ctls_low; u32 exit_ctls_high; + u64 secondary_exit_ctls; u32 entry_ctls_low; u32 entry_ctls_high; u32 misc_low; @@ -57,8 +58,9 @@ struct vmcs_config { u32 cpu_based_exec_ctrl; u32 cpu_based_2nd_exec_ctrl; u64 cpu_based_3rd_exec_ctrl; - u32 vmexit_ctrl; u32 vmentry_ctrl; + u32 vmexit_ctrl; + u64 vmexit_2nd_ctrl; u64 misc; struct nested_vmx_msrs nested; }; @@ -80,6 +82,11 @@ static inline bool cpu_has_vmx_basic_no_hw_errcode_cc(void) return vmcs_config.basic & VMX_BASIC_NO_HW_ERROR_CODE_CC; } +static inline bool cpu_has_vmx_nested_exception(void) +{ + return vmcs_config.basic & VMX_BASIC_NESTED_EXCEPTION; +} + static inline bool cpu_has_virtual_nmis(void) { return vmcs_config.pin_based_exec_ctrl & PIN_BASED_VIRTUAL_NMIS && @@ -149,6 +156,12 @@ static inline bool cpu_has_tertiary_exec_ctrls(void) CPU_BASED_ACTIVATE_TERTIARY_CONTROLS; } +static inline bool cpu_has_secondary_vmexit_ctrls(void) +{ + return vmcs_config.vmexit_ctrl & + VM_EXIT_ACTIVATE_SECONDARY_CONTROLS; +} + static inline bool cpu_has_vmx_virtualize_apic_accesses(void) { return vmcs_config.cpu_based_2nd_exec_ctrl & @@ -402,6 +415,11 @@ static inline bool vmx_pebs_supported(void) !enable_mediated_pmu; } +static inline bool cpu_has_vmx_fred(void) +{ + return vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_IA32_FRED; +} + static inline bool cpu_has_notify_vmexit(void) { return vmcs_config.cpu_based_2nd_exec_ctrl & diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 151873407abd3..4fad9728a2ea0 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -812,6 +812,15 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu vcpu, nested_vmx_merge_msr_bitmaps_rw(MSR_FS_BASE); nested_vmx_merge_msr_bitmaps_rw(MSR_GS_BASE); nested_vmx_merge_msr_bitmaps_rw(MSR_KERNEL_GS_BASE); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_RSP0); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_RSP1); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_RSP2); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_RSP3); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_STKLVLS); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_SSP1); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_SSP2); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_SSP3); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_CONFIG); #endif nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_SPEC_CTRL); nested_vmx_merge_msr_bitmaps_write(MSR_IA32_PRED_CMD); @@ -1356,9 +1365,11 @@ static int vmx_restore_vmx_basic(struct vcpu_vmx vmx, u64 data) const u64 feature_bits = VMX_BASIC_DUAL_MONITOR_TREATMENT \| VMX_BASIC_INOUT \| VMX_BASIC_TRUE_CTLS \| - VMX_BASIC_NO_HW_ERROR_CODE_CC; + VMX_BASIC_NO_HW_ERROR_CODE_CC \| + VMX_BASIC_NESTED_EXCEPTION; - const u64 reserved_bits = GENMASK_ULL(63, 57) \| + const u64 reserved_bits = GENMASK_ULL(63, 59) \| + BIT_ULL(57) \| GENMASK_ULL(47, 45) \| BIT_ULL(31); @@ -1596,6 +1607,11 @@ int vmx_set_vmx_msr(struct kvm_vcpu vcpu, u32 msr_index, u64 data) return -EINVAL; vmx->nested.msrs.vmfunc_controls = data; return 0; + case MSR_IA32_VMX_EXIT_CTLS2: + if (data & ~vmcs_config.nested.secondary_exit_ctls) + return -EINVAL; + vmx->nested.msrs.secondary_exit_ctls = data; + return 0; default: /* * The rest of the VMX capability MSRs do not support restore. @@ -1635,6 +1651,9 @@ int vmx_get_vmx_msr(struct nested_vmx_msrs msrs, u32 msr_index, u64 pdata) if (msr_index == MSR_IA32_VMX_EXIT_CTLS) pdata \|= VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR; break; + case MSR_IA32_VMX_EXIT_CTLS2: + pdata = msrs->secondary_exit_ctls; + break; case MSR_IA32_VMX_TRUE_ENTRY_CTLS: case MSR_IA32_VMX_ENTRY_CTLS: pdata = vmx_control_msr( @@ -2576,6 +2595,9 @@ static void prepare_vmcs02_early(struct vcpu_vmx vmx, struct loaded_vmcs vmcs0 exec_control &= ~VM_EXIT_LOAD_IA32_EFER; vm_exit_controls_set(vmx, exec_control); + if (exec_control & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS) + secondary_vm_exit_controls_set(vmx, __secondary_vm_exit_controls_get(vmcs01)); + / * Interrupt/Exception Fields / @@ -2588,6 +2610,8 @@ static void prepare_vmcs02_early(struct vcpu_vmx vmx, struct loaded_vmcs vmcs0 vmcs12->vm_entry_instruction_len); vmcs_write32(GUEST_INTERRUPTIBILITY_INFO, vmcs12->guest_interruptibility_info); + if (cpu_has_vmx_fred()) + vmcs_write64(INJECTED_EVENT_DATA, vmcs12->injected_event_data); vmx->loaded_vmcs->nmi_known_unmasked = !(vmcs12->guest_interruptibility_info & GUEST_INTR_STATE_NMI); } else { @@ -2621,6 +2645,30 @@ static void vmcs_write_cet_state(struct kvm_vcpu vcpu, u64 s_cet, } } +static void vmcs_read_fred_msrs(struct vmcs_fred_msrs msrs) +{ + msrs->fred_config = vmcs_read64(GUEST_IA32_FRED_CONFIG); + msrs->fred_rsp1 = vmcs_read64(GUEST_IA32_FRED_RSP1); + msrs->fred_rsp2 = vmcs_read64(GUEST_IA32_FRED_RSP2); + msrs->fred_rsp3 = vmcs_read64(GUEST_IA32_FRED_RSP3); + msrs->fred_stklvls = vmcs_read64(GUEST_IA32_FRED_STKLVLS); + msrs->fred_ssp1 = vmcs_read64(GUEST_IA32_FRED_SSP1); + msrs->fred_ssp2 = vmcs_read64(GUEST_IA32_FRED_SSP2); + msrs->fred_ssp3 = vmcs_read64(GUEST_IA32_FRED_SSP3); +} + +static void vmcs_write_fred_msrs(struct vmcs_fred_msrs msrs) +{ + vmcs_write64(GUEST_IA32_FRED_CONFIG, msrs->fred_config); + vmcs_write64(GUEST_IA32_FRED_RSP1, msrs->fred_rsp1); + vmcs_write64(GUEST_IA32_FRED_RSP2, msrs->fred_rsp2); + vmcs_write64(GUEST_IA32_FRED_RSP3, msrs->fred_rsp3); + vmcs_write64(GUEST_IA32_FRED_STKLVLS, msrs->fred_stklvls); + vmcs_write64(GUEST_IA32_FRED_SSP1, msrs->fred_ssp1); + vmcs_write64(GUEST_IA32_FRED_SSP2, msrs->fred_ssp2); + vmcs_write64(GUEST_IA32_FRED_SSP3, msrs->fred_ssp3); +} + static void prepare_vmcs02_rare(struct vcpu_vmx vmx, struct vmcs12 vmcs12) { struct hv_enlightened_vmcs hv_evmcs = nested_vmx_evmcs(vmx); @@ -2744,6 +2792,10 @@ static void prepare_vmcs02_rare(struct vcpu_vmx vmx, struct vmcs12 vmcs12) vmcs12->guest_ssp, vmcs12->guest_ssp_tbl); set_cr4_guest_host_mask(vmx); + + if (guest_cpu_cap_has(&vmx->vcpu, X86_FEATURE_FRED) && + nested_cpu_load_guest_fred_state(vmcs12)) + vmcs_write_fred_msrs(&vmcs12->guest_fred_msrs); } / @@ -2810,6 +2862,10 @@ static int prepare_vmcs02(struct kvm_vcpu vcpu, struct vmcs12 vmcs12, vmcs_write64(GUEST_IA32_PAT, vcpu->arch.pat); } + if (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED) && + (!vmx->vcpu.arch.nested_run_pending \|\| !nested_cpu_load_guest_fred_state(vmcs12))) + vmcs_write_fred_msrs(&vmx->nested.pre_vmenter_fred_msrs); + vcpu->arch.tsc_offset = kvm_calc_nested_tsc_offset( vcpu->arch.l1_tsc_offset, vmx_get_l2_tsc_offset(vcpu), @@ -3040,6 +3096,11 @@ static int nested_check_vm_exit_controls(struct kvm_vcpu vcpu, CC(nested_vmx_check_exit_msr_switch_controls(vcpu, vmcs12))) return -EINVAL; + if (nested_cpu_has_secondary_vm_exit_controls(vmcs12) && + CC(vmcs12->secondary_vm_exit_controls & + ~vmx->nested.msrs.secondary_exit_ctls)) + return -EINVAL; + return 0; } @@ -3050,6 +3111,8 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu vcpu, struct vmcs12 vmcs12) { struct vcpu_vmx vmx = to_vmx(vcpu); + bool fred_enabled = (vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE) && + (vmcs12->guest_cr4 & X86_CR4_FRED); if (CC(!vmx_control_verify(vmcs12->vm_entry_controls, vmx->nested.msrs.entry_ctls_low, @@ -3067,22 +3130,11 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu vcpu, u8 vector = intr_info & INTR_INFO_VECTOR_MASK; u32 intr_type = intr_info & INTR_INFO_INTR_TYPE_MASK; bool has_error_code = intr_info & INTR_INFO_DELIVER_CODE_MASK; + bool has_nested_exception = intr_info & INTR_INFO_NESTED_EXCEPTION_MASK; bool urg = nested_cpu_has2(vmcs12, SECONDARY_EXEC_UNRESTRICTED_GUEST); bool prot_mode = !urg \|\| vmcs12->guest_cr0 & X86_CR0_PE; - / VM-entry interruption-info field: interruption type / - if (CC(intr_type == INTR_TYPE_RESERVED) \|\| - CC(intr_type == INTR_TYPE_OTHER_EVENT && - !nested_cpu_supports_monitor_trap_flag(vcpu))) - return -EINVAL; - - / VM-entry interruption-info field: vector / - if (CC(intr_type == INTR_TYPE_NMI_INTR && vector != NMI_VECTOR) \|\| - CC(intr_type == INTR_TYPE_HARD_EXCEPTION && vector > 31) \|\| - CC(intr_type == INTR_TYPE_OTHER_EVENT && vector != 0)) - return -EINVAL; - / * Cannot deliver error code in real mode or if the interrupt * type is not hardware exception. For other cases, do the @@ -3106,8 +3158,28 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu vcpu, if (CC(intr_info & INTR_INFO_RESVD_BITS_MASK)) return -EINVAL; - / VM-entry instruction length / + if (CC(intr_type == INTR_TYPE_RESERVED)) + return -EINVAL; + + / + * Only for hardware exceptions and when the CPU enumerates + * VMX nested-exception support, bit 13 (indicating a nested + * exception) has value 1. Otherwise it is reserved. + / + if (CC(has_nested_exception && + (intr_type != INTR_TYPE_HARD_EXCEPTION \|\| + !nested_cpu_has_nested_exception(vcpu)))) + return -EINVAL; + switch (intr_type) { + case INTR_TYPE_NMI_INTR: + if (CC(vector != NMI_VECTOR)) + return -EINVAL; + break; + case INTR_TYPE_HARD_EXCEPTION: + if (CC(vector > 31)) + return -EINVAL; + break; case INTR_TYPE_SOFT_EXCEPTION: case INTR_TYPE_SOFT_INTR: case INTR_TYPE_PRIV_SW_EXCEPTION: @@ -3115,6 +3187,28 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu vcpu, CC(vmcs12->vm_entry_instruction_len == 0 && CC(!nested_cpu_has_zero_length_injection(vcpu)))) return -EINVAL; + break; + case INTR_TYPE_OTHER_EVENT: + if (CC(vector > 2)) + return -EINVAL; + + switch (vector) { + case 0: + if (CC(!nested_cpu_supports_monitor_trap_flag(vcpu))) + return -EINVAL; + break; + case 1: + case 2: + if (CC(!fred_enabled)) + return -EINVAL; + if (CC(vmcs12->vm_entry_instruction_len > X86_MAX_INSTRUCTION_LENGTH)) + return -EINVAL; + if (CC(vmcs12->vm_entry_instruction_len == 0 && + !nested_cpu_has_zero_length_injection(vcpu))) + return -EINVAL; + break; + } + break; } } @@ -3201,9 +3295,27 @@ static int nested_vmx_check_host_state(struct kvm_vcpu vcpu, if (ia32e) { if (CC(!(vmcs12->host_cr4 & X86_CR4_PAE))) return -EINVAL; + if (nested_cpu_load_host_fred_state(vmcs12)) { + if (CC(vmcs12->host_ia32_fred_config & FRED_CONFIG_RESERVED) \|\| + CC(vmcs12->host_ia32_fred_rsp1 & GENMASK_ULL(5, 0)) \|\| + CC(vmcs12->host_ia32_fred_rsp2 & GENMASK_ULL(5, 0)) \|\| + CC(vmcs12->host_ia32_fred_rsp3 & GENMASK_ULL(5, 0)) \|\| + CC(vmcs12->host_ia32_fred_ssp1 & GENMASK_ULL(2, 0)) \|\| + CC(vmcs12->host_ia32_fred_ssp2 & GENMASK_ULL(2, 0)) \|\| + CC(vmcs12->host_ia32_fred_ssp3 & GENMASK_ULL(2, 0)) \|\| + CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_config & PAGE_MASK, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp1, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp2, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp3, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp1, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp2, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp3, vcpu))) + return -EINVAL; + } } else { if (CC(vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE) \|\| CC(vmcs12->host_cr4 & X86_CR4_PCIDE) \|\| + CC(vmcs12->host_cr4 & X86_CR4_FRED) \|\| CC((vmcs12->host_rip) >> 32)) return -EINVAL; } @@ -3386,6 +3498,48 @@ static int nested_vmx_check_guest_state(struct kvm_vcpu vcpu, CC((vmcs12->guest_bndcfgs & MSR_IA32_BNDCFGS_RSVD)))) return -EINVAL; + if (ia32e) { + if (nested_cpu_load_guest_fred_state(vmcs12)) { + if (CC(vmcs12->guest_ia32_fred_config & FRED_CONFIG_RESERVED) \|\| + CC(vmcs12->guest_ia32_fred_rsp1 & GENMASK_ULL(5, 0)) \|\| + CC(vmcs12->guest_ia32_fred_rsp2 & GENMASK_ULL(5, 0)) \|\| + CC(vmcs12->guest_ia32_fred_rsp3 & GENMASK_ULL(5, 0)) \|\| + CC(vmcs12->guest_ia32_fred_ssp1 & GENMASK_ULL(2, 0)) \|\| + CC(vmcs12->guest_ia32_fred_ssp2 & GENMASK_ULL(2, 0)) \|\| + CC(vmcs12->guest_ia32_fred_ssp3 & GENMASK_ULL(2, 0)) \|\| + CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_config & PAGE_MASK, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp1, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp2, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp3, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp1, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp2, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp3, vcpu))) + return -EINVAL; + } + if (vmcs12->guest_cr4 & X86_CR4_FRED) { + unsigned int ss_dpl = VMX_AR_DPL(vmcs12->guest_ss_ar_bytes); + + if (CC(ss_dpl == 1 \|\| ss_dpl == 2)) + return -EINVAL; + + switch (ss_dpl) { + case 0: + if (CC(!(vmcs12->guest_cs_ar_bytes & VMX_AR_L_MASK))) + return -EINVAL; + break; + case 3: + if (CC(vmcs12->guest_rflags & X86_EFLAGS_IOPL)) + return -EINVAL; + if (CC(vmcs12->guest_interruptibility_info & GUEST_INTR_STATE_STI)) + return -EINVAL; + break; + } + } + } else { + if (CC(vmcs12->guest_cr4 & X86_CR4_FRED)) + return -EINVAL; + } + if (vmcs12->vm_entry_controls & VM_ENTRY_LOAD_CET_STATE) { if (nested_vmx_check_cet_state_common(vcpu, vmcs12->guest_s_cet, vmcs12->guest_ssp, @@ -3614,7 +3768,8 @@ static int nested_vmx_check_permission(struct kvm_vcpu vcpu) } static void load_vmcs12_host_state(struct kvm_vcpu vcpu, - struct vmcs12 vmcs12); + struct vmcs12 vmcs12, + bool from_failed_vmentry); /* * If from_vmentry is false, this is being called from state restore (either RSM @@ -3664,6 +3819,10 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu vcpu, &vmx->nested.pre_vmenter_ssp, &vmx->nested.pre_vmenter_ssp_tbl); + if (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED) && + (!vmx->vcpu.arch.nested_run_pending \|\| !nested_cpu_load_guest_fred_state(vmcs12))) + vmcs_read_fred_msrs(&vmx->nested.pre_vmenter_fred_msrs); + / * Stash L1's CR3, so that in the event of a "late" VM-Fail, i.e. a * VM-Fail detected by hardware but not KVM, KVM can unwind its @@ -3778,7 +3937,7 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu vcpu, nested_put_vmcs12_pages(vcpu); - load_vmcs12_host_state(vcpu, vmcs12); + load_vmcs12_host_state(vcpu, vmcs12, true); vmcs12->vm_exit_reason = exit_reason.full; if (enable_shadow_vmcs \|\| nested_vmx_is_evmptr12_valid(vmx)) vmx->nested.need_vmcs12_to_shadow_sync = true; @@ -3967,6 +4126,8 @@ static void vmcs12_save_pending_event(struct kvm_vcpu vcpu, u32 idt_vectoring; unsigned int nr; + vmcs12->original_event_data = 0; + /* * Per the SDM, VM-Exits due to double and triple faults are never * considered to occur during event delivery, even if the double/triple @@ -4005,6 +4166,13 @@ static void vmcs12_save_pending_event(struct kvm_vcpu vcpu, vcpu->arch.exception.error_code; } + if ((vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE) && + (vmcs12->guest_cr4 & X86_CR4_FRED) && + (vcpu->arch.exception.is_nested)) + idt_vectoring \|= VECTORING_INFO_NESTED_EXCEPTION_MASK; + + vmcs12->original_event_data = vcpu->arch.exception.event_data; + vmcs12->idt_vectoring_info_field = idt_vectoring; } else if (vcpu->arch.nmi_injected) { vmcs12->idt_vectoring_info_field = @@ -4728,6 +4896,15 @@ static void sync_vmcs02_to_vmcs12(struct kvm_vcpu vcpu, struct vmcs12 vmcs12) vmcs_read_cet_state(&vmx->vcpu, &vmcs12->guest_s_cet, &vmcs12->guest_ssp, &vmcs12->guest_ssp_tbl); + + if (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) { + vmcs_read_fred_msrs(&vmx->nested.at_vmexit_fred_msrs); + + if (nested_cpu_save_guest_fred_state(vmcs12)) + memcpy(&vmcs12->guest_fred_msrs, + &vmx->nested.at_vmexit_fred_msrs, + sizeof(struct vmcs_fred_msrs)); + } } / @@ -4772,6 +4949,21 @@ static void prepare_vmcs12(struct kvm_vcpu vcpu, struct vmcs12 vmcs12, vmcs12->vm_exit_intr_info = exit_intr_info; vmcs12->vm_exit_instruction_len = exit_insn_len; + + /* + * When there is a valid original event, the exiting event is a nested + * event during delivery of the earlier original event. + * + * FRED event delivery reflects this relationship by setting the value + * of the nested exception bit of VM-exit interruption information + * (aka exiting-event identification) to that of the valid bit of the + * IDT-vectoring information (aka original-event identification). + / + if ((vmcs12->idt_vectoring_info_field & VECTORING_INFO_VALID_MASK) && + (vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE) && + (vmcs12->guest_cr4 & X86_CR4_FRED)) + vmcs12->vm_exit_intr_info \|= INTR_INFO_NESTED_EXCEPTION_MASK; + vmcs12->vmx_instruction_info = vmcs_read32(VMX_INSTRUCTION_INFO); / @@ -4798,8 +4990,10 @@ static void prepare_vmcs12(struct kvm_vcpu vcpu, struct vmcs12 vmcs12, * This function should be called when the active VMCS is L1's (vmcs01). / static void load_vmcs12_host_state(struct kvm_vcpu vcpu, - struct vmcs12 vmcs12) + struct vmcs12 vmcs12, + bool from_failed_vmentry) { + struct vcpu_vmx vmx = to_vmx(vcpu); enum vm_entry_failure_code ignored; struct kvm_segment seg; @@ -4874,6 +5068,14 @@ static void load_vmcs12_host_state(struct kvm_vcpu vcpu, WARN_ON_ONCE(__kvm_emulate_msr_write(vcpu, MSR_CORE_PERF_GLOBAL_CTRL, vmcs12->host_ia32_perf_global_ctrl)); + if (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) { + if (nested_cpu_load_host_fred_state(vmcs12)) { + vmcs_write_fred_msrs(&vmcs12->host_fred_msrs); + } else if (!from_failed_vmentry) { + vmcs_write_fred_msrs(&vmx->nested.at_vmexit_fred_msrs); + } + } + /* Set L1 segment info according to Intel SDM 27.5.2 Loading Host Segment and Descriptor-Table Registers / seg = (struct kvm_segment) { @@ -5193,7 +5395,7 @@ void __nested_vmx_vmexit(struct kvm_vcpu vcpu, u32 vm_exit_reason, vmcs12->vm_exit_intr_error_code, KVM_ISA_VMX); - load_vmcs12_host_state(vcpu, vmcs12); + load_vmcs12_host_state(vcpu, vmcs12, false); /* * Process events if an injectable IRQ or NMI is pending, even @@ -7152,7 +7354,8 @@ static void nested_vmx_setup_exit_ctls(struct vmcs_config vmcs_conf, VM_EXIT_HOST_ADDR_SPACE_SIZE \| #endif VM_EXIT_LOAD_IA32_PAT \| VM_EXIT_SAVE_IA32_PAT \| - VM_EXIT_CLEAR_BNDCFGS \| VM_EXIT_LOAD_CET_STATE; + VM_EXIT_CLEAR_BNDCFGS \| VM_EXIT_LOAD_CET_STATE \| + VM_EXIT_ACTIVATE_SECONDARY_CONTROLS; msrs->exit_ctls_high \|= VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR \| VM_EXIT_LOAD_IA32_EFER \| VM_EXIT_SAVE_IA32_EFER \| @@ -7165,6 +7368,12 @@ static void nested_vmx_setup_exit_ctls(struct vmcs_config vmcs_conf, /* We support free control of debug control saving. / msrs->exit_ctls_low &= ~VM_EXIT_SAVE_DEBUG_CONTROLS; + + if (msrs->exit_ctls_high & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS) { + msrs->secondary_exit_ctls = vmcs_conf->vmexit_2nd_ctrl; + msrs->secondary_exit_ctls &= SECONDARY_VM_EXIT_SAVE_IA32_FRED \| + SECONDARY_VM_EXIT_LOAD_IA32_FRED; + } } static void nested_vmx_setup_entry_ctls(struct vmcs_config vmcs_conf, @@ -7179,7 +7388,7 @@ static void nested_vmx_setup_entry_ctls(struct vmcs_config vmcs_conf, VM_ENTRY_IA32E_MODE \| #endif VM_ENTRY_LOAD_IA32_PAT \| VM_ENTRY_LOAD_BNDCFGS \| - VM_ENTRY_LOAD_CET_STATE; + VM_ENTRY_LOAD_CET_STATE \| VM_ENTRY_LOAD_IA32_FRED; msrs->entry_ctls_high \|= (VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR \| VM_ENTRY_LOAD_IA32_EFER \| VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL); @@ -7343,6 +7552,8 @@ static void nested_vmx_setup_basic(struct nested_vmx_msrs msrs) msrs->basic \|= VMX_BASIC_INOUT; if (cpu_has_vmx_basic_no_hw_errcode_cc()) msrs->basic \|= VMX_BASIC_NO_HW_ERROR_CODE_CC; + if (cpu_has_vmx_nested_exception()) + msrs->basic \|= VMX_BASIC_NESTED_EXCEPTION; } static void nested_vmx_setup_cr_fixed(struct nested_vmx_msrs msrs) diff --git a/arch/x86/kvm/vmx/nested.h b/arch/x86/kvm/vmx/nested.h index c6de848bd9ced..5511218241281 100644 --- a/arch/x86/kvm/vmx/nested.h +++ b/arch/x86/kvm/vmx/nested.h @@ -247,6 +247,11 @@ static inline bool nested_cpu_has_save_preemption_timer(struct vmcs12 vmcs12) VM_EXIT_SAVE_VMX_PREEMPTION_TIMER; } +static inline bool nested_cpu_has_secondary_vm_exit_controls(struct vmcs12 vmcs12) +{ + return vmcs12->vm_exit_controls & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS; +} + static inline bool nested_exit_on_nmi(struct kvm_vcpu vcpu) { return nested_cpu_has_nmi_exiting(get_vmcs12(vcpu)); @@ -267,6 +272,23 @@ static inline bool nested_cpu_has_encls_exit(struct vmcs12 vmcs12) return nested_cpu_has2(vmcs12, SECONDARY_EXEC_ENCLS_EXITING); } +static inline bool nested_cpu_load_guest_fred_state(struct vmcs12 vmcs12) +{ + return vmcs12->vm_entry_controls & VM_ENTRY_LOAD_IA32_FRED; +} + +static inline bool nested_cpu_save_guest_fred_state(struct vmcs12 vmcs12) +{ + return nested_cpu_has_secondary_vm_exit_controls(vmcs12) && + vmcs12->secondary_vm_exit_controls & SECONDARY_VM_EXIT_SAVE_IA32_FRED; +} + +static inline bool nested_cpu_load_host_fred_state(struct vmcs12 vmcs12) +{ + return nested_cpu_has_secondary_vm_exit_controls(vmcs12) && + vmcs12->secondary_vm_exit_controls & SECONDARY_VM_EXIT_LOAD_IA32_FRED; +} + /* * if fixed0[i] == 1: val[i] must be 1 * if fixed1[i] == 0: val[i] must be 0 @@ -312,6 +334,11 @@ static inline bool nested_cpu_has_no_hw_errcode_cc(struct kvm_vcpu vcpu) return to_vmx(vcpu)->nested.msrs.basic & VMX_BASIC_NO_HW_ERROR_CODE_CC; } +static inline bool nested_cpu_has_nested_exception(struct kvm_vcpu vcpu) +{ + return to_vmx(vcpu)->nested.msrs.basic & VMX_BASIC_NESTED_EXCEPTION; +} + /* No difference in the restrictions on guest and host CR4 in VMX operation. / #define nested_guest_cr4_valid nested_cr4_valid #define nested_host_cr4_valid nested_cr4_valid diff --git a/arch/x86/kvm/vmx/vmcs.h b/arch/x86/kvm/vmx/vmcs.h index 1f16ddeae9cb1..58af10f73789e 100644 --- a/arch/x86/kvm/vmx/vmcs.h +++ b/arch/x86/kvm/vmx/vmcs.h @@ -45,12 +45,24 @@ struct vmcs_host_state { struct vmcs_controls_shadow { u32 vm_entry; u32 vm_exit; + u64 secondary_vm_exit; u32 pin; u32 exec; u32 secondary_exec; u64 tertiary_exec; }; +struct vmcs_fred_msrs { + u64 fred_config; + u64 fred_rsp1; + u64 fred_rsp2; + u64 fred_rsp3; + u64 fred_stklvls; + u64 fred_ssp1; + u64 fred_ssp2; + u64 fred_ssp3; +}; + / * Track a VMCS that may be loaded on a certain CPU. If it is (cpu!=-1), also * remember whether it was VMLAUNCHed, and maintain a linked list of all VMCSs diff --git a/arch/x86/kvm/vmx/vmcs12.c b/arch/x86/kvm/vmx/vmcs12.c index 1ebe67c384ad4..67c730e314655 100644 --- a/arch/x86/kvm/vmx/vmcs12.c +++ b/arch/x86/kvm/vmx/vmcs12.c @@ -66,6 +66,25 @@ static const u16 kvm_supported_vmcs12_field_offsets[] __initconst = { FIELD64(HOST_IA32_PAT, host_ia32_pat), FIELD64(HOST_IA32_EFER, host_ia32_efer), FIELD64(HOST_IA32_PERF_GLOBAL_CTRL, host_ia32_perf_global_ctrl), + FIELD64(SECONDARY_VM_EXIT_CONTROLS, secondary_vm_exit_controls), + FIELD64(INJECTED_EVENT_DATA, injected_event_data), + FIELD64(ORIGINAL_EVENT_DATA, original_event_data), + FIELD64(GUEST_IA32_FRED_CONFIG, guest_ia32_fred_config), + FIELD64(GUEST_IA32_FRED_RSP1, guest_ia32_fred_rsp1), + FIELD64(GUEST_IA32_FRED_RSP2, guest_ia32_fred_rsp2), + FIELD64(GUEST_IA32_FRED_RSP3, guest_ia32_fred_rsp3), + FIELD64(GUEST_IA32_FRED_STKLVLS, guest_ia32_fred_stklvls), + FIELD64(GUEST_IA32_FRED_SSP1, guest_ia32_fred_ssp1), + FIELD64(GUEST_IA32_FRED_SSP2, guest_ia32_fred_ssp2), + FIELD64(GUEST_IA32_FRED_SSP3, guest_ia32_fred_ssp3), + FIELD64(HOST_IA32_FRED_CONFIG, host_ia32_fred_config), + FIELD64(HOST_IA32_FRED_RSP1, host_ia32_fred_rsp1), + FIELD64(HOST_IA32_FRED_RSP2, host_ia32_fred_rsp2), + FIELD64(HOST_IA32_FRED_RSP3, host_ia32_fred_rsp3), + FIELD64(HOST_IA32_FRED_STKLVLS, host_ia32_fred_stklvls), + FIELD64(HOST_IA32_FRED_SSP1, host_ia32_fred_ssp1), + FIELD64(HOST_IA32_FRED_SSP2, host_ia32_fred_ssp2), + FIELD64(HOST_IA32_FRED_SSP3, host_ia32_fred_ssp3), FIELD(PIN_BASED_VM_EXEC_CONTROL, pin_based_vm_exec_control), FIELD(CPU_BASED_VM_EXEC_CONTROL, cpu_based_vm_exec_control), FIELD(EXCEPTION_BITMAP, exception_bitmap), @@ -203,6 +222,9 @@ static __init bool cpu_has_vmcs12_field(unsigned int idx) case HOST_SSP: case HOST_INTR_SSP_TABLE: return cpu_has_load_cet_ctrl(); + VMCS12_CASE64(ORIGINAL_EVENT_DATA): + VMCS12_CASE64(INJECTED_EVENT_DATA): + return cpu_has_vmx_fred(); /* KVM always emulates PML and the VMX preemption timer in software. / case GUEST_PML_INDEX: diff --git a/arch/x86/kvm/vmx/vmcs12.h b/arch/x86/kvm/vmx/vmcs12.h index 21cd1b75e4fdf..c4c5cd3722a5b 100644 --- a/arch/x86/kvm/vmx/vmcs12.h +++ b/arch/x86/kvm/vmx/vmcs12.h @@ -71,7 +71,7 @@ struct __packed vmcs12 { u64 pml_address; u64 encls_exiting_bitmap; u64 tsc_multiplier; - u64 padding64[1]; / room for future expansion / + u64 secondary_vm_exit_controls; / * To allow migration of L1 (complete with its L2 guests) between * machines of different natural widths (32 or 64 bit), we cannot have @@ -191,6 +191,27 @@ struct __packed vmcs12 { u16 host_gs_selector; u16 host_tr_selector; u16 guest_pml_index; + u16 padding16[1]; /* align to 64-bit boundary / + struct vmcs_fred_msrs guest_fred_msrs; +#define guest_ia32_fred_config guest_fred_msrs.fred_config +#define guest_ia32_fred_rsp1 guest_fred_msrs.fred_rsp1 +#define guest_ia32_fred_rsp2 guest_fred_msrs.fred_rsp2 +#define guest_ia32_fred_rsp3 guest_fred_msrs.fred_rsp3 +#define guest_ia32_fred_stklvls guest_fred_msrs.fred_stklvls +#define guest_ia32_fred_ssp1 guest_fred_msrs.fred_ssp1 +#define guest_ia32_fred_ssp2 guest_fred_msrs.fred_ssp2 +#define guest_ia32_fred_ssp3 guest_fred_msrs.fred_ssp3 + struct vmcs_fred_msrs host_fred_msrs; +#define host_ia32_fred_config host_fred_msrs.fred_config +#define host_ia32_fred_rsp1 host_fred_msrs.fred_rsp1 +#define host_ia32_fred_rsp2 host_fred_msrs.fred_rsp2 +#define host_ia32_fred_rsp3 host_fred_msrs.fred_rsp3 +#define host_ia32_fred_stklvls host_fred_msrs.fred_stklvls +#define host_ia32_fred_ssp1 host_fred_msrs.fred_ssp1 +#define host_ia32_fred_ssp2 host_fred_msrs.fred_ssp2 +#define host_ia32_fred_ssp3 host_fred_msrs.fred_ssp3 + u64 injected_event_data; + u64 original_event_data; }; / @@ -261,6 +282,7 @@ static inline void vmx_check_vmcs12_offsets(void) CHECK_OFFSET(pml_address, 312); CHECK_OFFSET(encls_exiting_bitmap, 320); CHECK_OFFSET(tsc_multiplier, 328); + CHECK_OFFSET(secondary_vm_exit_controls, 336); CHECK_OFFSET(cr0_guest_host_mask, 344); CHECK_OFFSET(cr4_guest_host_mask, 352); CHECK_OFFSET(cr0_read_shadow, 360); @@ -372,6 +394,24 @@ static inline void vmx_check_vmcs12_offsets(void) CHECK_OFFSET(host_gs_selector, 992); CHECK_OFFSET(host_tr_selector, 994); CHECK_OFFSET(guest_pml_index, 996); + CHECK_OFFSET(guest_ia32_fred_config, 1000); + CHECK_OFFSET(guest_ia32_fred_rsp1, 1008); + CHECK_OFFSET(guest_ia32_fred_rsp2, 1016); + CHECK_OFFSET(guest_ia32_fred_rsp3, 1024); + CHECK_OFFSET(guest_ia32_fred_stklvls, 1032); + CHECK_OFFSET(guest_ia32_fred_ssp1, 1040); + CHECK_OFFSET(guest_ia32_fred_ssp2, 1048); + CHECK_OFFSET(guest_ia32_fred_ssp3, 1056); + CHECK_OFFSET(host_ia32_fred_config, 1064); + CHECK_OFFSET(host_ia32_fred_rsp1, 1072); + CHECK_OFFSET(host_ia32_fred_rsp2, 1080); + CHECK_OFFSET(host_ia32_fred_rsp3, 1088); + CHECK_OFFSET(host_ia32_fred_stklvls, 1096); + CHECK_OFFSET(host_ia32_fred_ssp1, 1104); + CHECK_OFFSET(host_ia32_fred_ssp2, 1112); + CHECK_OFFSET(host_ia32_fred_ssp3, 1120); + CHECK_OFFSET(injected_event_data, 1128); + CHECK_OFFSET(original_event_data, 1136); } extern u16 vmcs12_field_offsets[] __ro_after_init; diff --git a/arch/x86/kvm/vmx/vmcs_shadow_fields.h b/arch/x86/kvm/vmx/vmcs_shadow_fields.h index 67e821c2be6d5..bdd56f3ca6c30 100644 --- a/arch/x86/kvm/vmx/vmcs_shadow_fields.h +++ b/arch/x86/kvm/vmx/vmcs_shadow_fields.h @@ -74,6 +74,10 @@ SHADOW_FIELD_RW(HOST_GS_BASE, host_gs_base) /* 64-bit / SHADOW_FIELD_RO(GUEST_PHYSICAL_ADDRESS, guest_physical_address) SHADOW_FIELD_RO(GUEST_PHYSICAL_ADDRESS_HIGH, guest_physical_address) +SHADOW_FIELD_RO(ORIGINAL_EVENT_DATA, original_event_data) +SHADOW_FIELD_RO(ORIGINAL_EVENT_DATA_HIGH, original_event_data) +SHADOW_FIELD_RW(INJECTED_EVENT_DATA, injected_event_data) +SHADOW_FIELD_RW(INJECTED_EVENT_DATA_HIGH, injected_event_data) #endif #undef SHADOW_FIELD_RO diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 504630f0eb40f..f673cd4e8956e 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1395,6 +1395,9 @@ void vmx_prepare_switch_to_guest(struct kvm_vcpu vcpu) } wrmsrq(MSR_KERNEL_GS_BASE, vmx->msr_guest_kernel_gs_base); + + if (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) + wrmsrns(MSR_IA32_FRED_RSP0, vmx->msr_guest_fred_rsp0); #else savesegment(fs, fs_sel); savesegment(gs, gs_sel); @@ -1439,6 +1442,16 @@ static void vmx_prepare_switch_to_host(struct vcpu_vmx vmx) invalidate_tss_limit(); #ifdef CONFIG_X86_64 wrmsrq(MSR_KERNEL_GS_BASE, vmx->vt.msr_host_kernel_gs_base); + + if (guest_cpu_cap_has(&vmx->vcpu, X86_FEATURE_FRED)) { + vmx->msr_guest_fred_rsp0 = read_msr(MSR_IA32_FRED_RSP0); + / + * Synchronize the current value in hardware to the kernel's + * local cache. The desired host RSP0 will be set when the + * CPU exits to userspace (RSP0 is a per-task value). + / + fred_sync_rsp0(vmx->msr_guest_fred_rsp0); + } #endif load_fixmap_gdt(raw_smp_processor_id()); vmx->vt.guest_state_loaded = false; @@ -1476,6 +1489,21 @@ static void vmx_write_guest_kernel_gs_base(struct vcpu_vmx vmx, u64 data) vmx_write_guest_host_msr(vmx, MSR_KERNEL_GS_BASE, data, &vmx->msr_guest_kernel_gs_base); } + +static u64 vmx_read_guest_fred_rsp0(struct vcpu_vmx vmx) +{ + return vmx_read_guest_host_msr(vmx, MSR_IA32_FRED_RSP0, + &vmx->msr_guest_fred_rsp0); +} + +static void vmx_write_guest_fred_rsp0(struct vcpu_vmx vmx, u64 data) +{ + vmx_write_guest_host_msr(vmx, MSR_IA32_FRED_RSP0, data, + &vmx->msr_guest_fred_rsp0); +} +#else +/* Make sure it builds on 32-bit / +static u64 vmx_read_guest_fred_rsp0(struct vcpu_vmx vmx) { return 0; } #endif static void grow_ple_window(struct kvm_vcpu vcpu) @@ -1562,6 +1590,12 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu vcpu, int cpu) (unsigned long)(cpu_entry_stack(cpu) + 1)); } + if (cpu_feature_enabled(X86_FEATURE_FRED) && kvm_cpu_cap_has(X86_FEATURE_FRED)) { + vmcs_write64(HOST_IA32_FRED_RSP1, this_cpu_fred_rsp(FRED_STACK_LEVEL_1)); + vmcs_write64(HOST_IA32_FRED_RSP2, this_cpu_fred_rsp(FRED_STACK_LEVEL_2)); + vmcs_write64(HOST_IA32_FRED_RSP3, this_cpu_fred_rsp(FRED_STACK_LEVEL_3)); + } + vmx->loaded_vmcs->cpu = cpu; } } @@ -1973,12 +2007,41 @@ void vmx_inject_exception(struct kvm_vcpu vcpu) WARN_ON_ONCE(vmx->vt.emulation_required); + / + * Event data is strictly required only for #PF, #DB, and #NM (due to + * extended feature disable) hardware exceptions under FRED, making it + * straightforward to correctly populate for just these vectors. + * + * Conversely, trying to guarantee that event data is cleanly managed or + * zeroed out across all other exception paths is highly complicated + * and error-prone. A blanket fallback to 0 handles those cases safely. + * + * Therefore, set INJECTED_EVENT_DATA only if all the following are true: + * - Guest has FRED enabled. + * - Exception type is a hardware exception. + * - Exception vector is #PF, #DB, or #NM. + * + * Otherwise, clear it to 0 to prevent stale data leakage. + / if (kvm_exception_is_soft(ex->vector)) { vmcs_write32(VM_ENTRY_INSTRUCTION_LEN, vmx->vcpu.arch.event_exit_inst_len); intr_info \|= INTR_TYPE_SOFT_EXCEPTION; - } else + if (is_fred_enabled(vcpu)) + vmcs_write64(INJECTED_EVENT_DATA, 0); + } else { intr_info \|= INTR_TYPE_HARD_EXCEPTION; + if (is_fred_enabled(vcpu)) { + if (ex->is_nested) + intr_info \|= INTR_INFO_NESTED_EXCEPTION_MASK; + if (ex->vector == PF_VECTOR \|\| + ex->vector == DB_VECTOR \|\| + ex->vector == NM_VECTOR) + vmcs_write64(INJECTED_EVENT_DATA, ex->event_data); + else + vmcs_write64(INJECTED_EVENT_DATA, 0); + } + } vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr_info); @@ -2121,6 +2184,27 @@ int vmx_get_feature_msr(u32 msr, u64 data) } } +#ifdef CONFIG_X86_64 +static const u32 fred_msr_vmcs_fields[] = { + GUEST_IA32_FRED_RSP1, + GUEST_IA32_FRED_RSP2, + GUEST_IA32_FRED_RSP3, + GUEST_IA32_FRED_STKLVLS, + GUEST_IA32_FRED_SSP1, + GUEST_IA32_FRED_SSP2, + GUEST_IA32_FRED_SSP3, + GUEST_IA32_FRED_CONFIG, +}; + +static_assert(MSR_IA32_FRED_CONFIG - MSR_IA32_FRED_RSP1 == + ARRAY_SIZE(fred_msr_vmcs_fields) - 1); + +static u32 fred_msr_to_vmcs(u32 msr) +{ + return fred_msr_vmcs_fields[msr - MSR_IA32_FRED_RSP1]; +} +#endif + /* * Reads an msr value (of 'msr_info->index') into 'msr_info->data'. * Returns 0 on success, non-0 otherwise. @@ -2143,6 +2227,12 @@ int vmx_get_msr(struct kvm_vcpu vcpu, struct msr_data msr_info) case MSR_KERNEL_GS_BASE: msr_info->data = vmx_read_guest_kernel_gs_base(vmx); break; + case MSR_IA32_FRED_RSP0: + msr_info->data = vmx_read_guest_fred_rsp0(vmx); + break; + case MSR_IA32_FRED_RSP1 ... MSR_IA32_FRED_CONFIG: + msr_info->data = vmcs_read64(fred_msr_to_vmcs(msr_info->index)); + break; #endif case MSR_EFER: return kvm_get_msr_common(vcpu, msr_info); @@ -2375,6 +2465,12 @@ int vmx_set_msr(struct kvm_vcpu vcpu, struct msr_data msr_info) vmx_update_exception_bitmap(vcpu); } break; + case MSR_IA32_FRED_RSP0: + vmx_write_guest_fred_rsp0(vmx, data); + break; + case MSR_IA32_FRED_RSP1 ... MSR_IA32_FRED_CONFIG: + vmcs_write64(fred_msr_to_vmcs(msr_index), data); + break; #endif case MSR_IA32_SYSENTER_CS: if (is_guest_mode(vcpu)) @@ -2751,8 +2847,9 @@ static int setup_vmcs_config(struct vmcs_config vmcs_conf, u32 _cpu_based_exec_control = 0; u32 _cpu_based_2nd_exec_control = 0; u64 _cpu_based_3rd_exec_control = 0; - u32 _vmexit_control = 0; u32 _vmentry_control = 0; + u32 _vmexit_control = 0; + u64 _vmexit2_control = 0; u64 basic_msr; u64 misc_msr; @@ -2773,6 +2870,14 @@ static int setup_vmcs_config(struct vmcs_config vmcs_conf, { VM_ENTRY_LOAD_CET_STATE, VM_EXIT_LOAD_CET_STATE }, }; + struct { + u32 entry_control; + u64 exit_control; + } const vmcs_entry_exit2_pairs[] = { + { VM_ENTRY_LOAD_IA32_FRED, + SECONDARY_VM_EXIT_SAVE_IA32_FRED \| SECONDARY_VM_EXIT_LOAD_IA32_FRED }, + }; + memset(vmcs_conf, 0, sizeof(vmcs_conf)); if (adjust_vmx_controls(KVM_REQUIRED_VMX_CPU_BASED_VM_EXEC_CONTROL, @@ -2870,10 +2975,19 @@ static int setup_vmcs_config(struct vmcs_config vmcs_conf, &_vmentry_control)) return -EIO; + if (_vmexit_control & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS) + _vmexit2_control = + adjust_vmx_controls64(KVM_OPTIONAL_VMX_SECONDARY_VM_EXIT_CONTROLS, + MSR_IA32_VMX_EXIT_CTLS2); + if (vmx_check_entry_exit_pairs(vmcs_entry_exit_pairs, _vmentry_control, _vmexit_control)) return -EIO; + if (vmx_check_entry_exit_pairs(vmcs_entry_exit2_pairs, + _vmentry_control, _vmexit2_control)) + return -EIO; + /* * Some cpus support VM_{ENTRY,EXIT}_IA32_PERF_GLOBAL_CTRL but they * can't be used due to an errata where VM Exit may incorrectly clear @@ -2922,8 +3036,9 @@ static int setup_vmcs_config(struct vmcs_config vmcs_conf, vmcs_conf->cpu_based_exec_ctrl = _cpu_based_exec_control; vmcs_conf->cpu_based_2nd_exec_ctrl = _cpu_based_2nd_exec_control; vmcs_conf->cpu_based_3rd_exec_ctrl = _cpu_based_3rd_exec_control; - vmcs_conf->vmexit_ctrl = _vmexit_control; vmcs_conf->vmentry_ctrl = _vmentry_control; + vmcs_conf->vmexit_ctrl = _vmexit_control; + vmcs_conf->vmexit_2nd_ctrl = _vmexit2_control; vmcs_conf->misc = misc_msr; #if IS_ENABLED(CONFIG_HYPERV) @@ -4279,6 +4394,72 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu vcpu) MSR_TYPE_RW, intercept); } +static void vmx_set_intercept_for_fred_msr(struct kvm_vcpu vcpu) +{ + bool intercept = !guest_cpu_cap_has(vcpu, X86_FEATURE_FRED); + + if (!kvm_cpu_cap_has(X86_FEATURE_FRED)) + return; + + / + * Because the following eight FRED MSRs, + * MSR_IA32_FRED_RSP[123], MSR_IA32_FRED_STKLVLS, + * MSR_IA32_FRED_SSP[123], MSR_IA32_FRED_CONFIG, + * are used by the kernel itself to take an exception at any time, they + * should be context-switched by Intel VT-x automatically in order to + * preserve the FRED architectural invariant that there should NEVER be + * a "gap" during which it is unsafe to take an exception. + * + * KVM leverages Intel VT-x hardware to automatically context switch the + * eight FRED MSRs using: + * + * 1) Dedicated host and guest VMCS fields for each MSR. + * + * 2) VM-entry/exit controls to manage the automated loading and saving + * of the eight FRED MSRs. + * + * Consequently, passing these MSRs through to the guest would only add + * unnecessary handling code without benefit. + / + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP1, MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP2, MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP3, MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_STKLVLS, MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP1, MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP2, MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP3, MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_CONFIG, MSR_TYPE_RW, intercept); + + / + * MSR_IA32_FRED_RSP0 and MSR_IA32_PL0_SSP (aka MSR_IA32_FRED_SSP0) are + * designed for event delivery while executing in userspace. Since KVM + * operates entirely in kernel mode (CPL is always 0 after any VM exit), + * it can safely retain and operate with guest-defined values for these + * MSRs. + * + * Disabling interception of the two MSRs offers two advantages: + * 1) Simplicity: Eliminates dedicated MSR handling code. + * 2) Performance: Avoids frequent VM-exits since the two MSRs are + * per user thread variables and frequently accessed. + * + * MSR_IA32_PL0_SSP (aka MSR_IA32_FRED_SSP0) is part of CET supervisor + * state, but all four FRED SSP MSRs are architecturally visible on any + * processor that enumerates FRED. Even if CET is absent, these MSRs + * remain accessible via RDMSR/WRMSR, though FRED transitions will not + * use them. + * + * Intercept MSR_IA32_PL0_SSP if CET shadow stacks are unsupported (even + * with FRED present). Since this MSR is rarely accessed and ignored by + * XSAVES in this configuration, interception avoids the overhead of + * manually context switching the hardware MSR during vcpu_load/put. + * + * This behavior is consistent with the current setup in + * vmx_recalc_msr_intercepts(), so no change is needed to the interception + * logic for MSR_IA32_PL0_SSP. + / + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP0, MSR_TYPE_RW, intercept); +} + static void vmx_recalc_msr_intercepts(struct kvm_vcpu vcpu) { bool intercept; @@ -4346,6 +4527,7 @@ static void vmx_recalc_msr_intercepts(struct kvm_vcpu vcpu) } vmx_recalc_pmu_msr_intercepts(vcpu); + vmx_set_intercept_for_fred_msr(vcpu); / * x2APIC and LBR MSR intercepts are modified on-demand and cannot be @@ -4481,6 +4663,17 @@ void vmx_set_constant_host_state(struct vcpu_vmx vmx) / vmcs_write16(HOST_DS_SELECTOR, 0); vmcs_write16(HOST_ES_SELECTOR, 0); + + if (cpu_feature_enabled(X86_FEATURE_FRED) && kvm_cpu_cap_has(X86_FEATURE_FRED)) { + /* FRED CONFIG and STKLVLS are the same on all CPUs / + vmcs_write64(HOST_IA32_FRED_CONFIG, kvm_host.fred_config); + vmcs_write64(HOST_IA32_FRED_STKLVLS, kvm_host.fred_stklvls); + + / Linux doesn't support kernel shadow stacks, thus SSPs are 0s / + vmcs_write64(HOST_IA32_FRED_SSP1, 0); + vmcs_write64(HOST_IA32_FRED_SSP2, 0); + vmcs_write64(HOST_IA32_FRED_SSP3, 0); + } #else vmcs_write16(HOST_DS_SELECTOR, __KERNEL_DS); / 22.2.4 / vmcs_write16(HOST_ES_SELECTOR, __KERNEL_DS); / 22.2.4 / @@ -4617,6 +4810,11 @@ static u32 vmx_get_initial_vmexit_ctrl(void) VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL); } +static u64 vmx_secondary_vmexit_ctrl(void) +{ + return vmcs_config.vmexit_2nd_ctrl; +} + void vmx_refresh_apicv_exec_ctrl(struct kvm_vcpu vcpu) { struct vcpu_vmx vmx = to_vmx(vcpu); @@ -4960,6 +5158,9 @@ static void init_vmcs(struct vcpu_vmx vmx) vm_exit_controls_set(vmx, vmx_get_initial_vmexit_ctrl()); + if (cpu_has_secondary_vmexit_ctrls()) + secondary_vm_exit_controls_set(vmx, vmx_secondary_vmexit_ctrl()); + /* 22.2.1, 20.8.1 / vm_entry_controls_set(vmx, vmx_get_initial_vmentry_ctrl()); @@ -5003,6 +5204,17 @@ static void init_vmcs(struct vcpu_vmx vmx) } vmx_setup_uret_msrs(vmx); + + if (cpu_feature_enabled(X86_FEATURE_FRED) && kvm_cpu_cap_has(X86_FEATURE_FRED)) { + vmcs_write64(GUEST_IA32_FRED_CONFIG, 0); + vmcs_write64(GUEST_IA32_FRED_RSP1, 0); + vmcs_write64(GUEST_IA32_FRED_RSP2, 0); + vmcs_write64(GUEST_IA32_FRED_RSP3, 0); + vmcs_write64(GUEST_IA32_FRED_STKLVLS, 0); + vmcs_write64(GUEST_IA32_FRED_SSP1, 0); + vmcs_write64(GUEST_IA32_FRED_SSP2, 0); + vmcs_write64(GUEST_IA32_FRED_SSP3, 0); + } } static void __vmx_vcpu_reset(struct kvm_vcpu vcpu) @@ -5143,6 +5355,14 @@ void vmx_inject_irq(struct kvm_vcpu vcpu, bool reinjected) vmx->vcpu.arch.event_exit_inst_len); } else intr \|= INTR_TYPE_EXT_INTR; + + /* + * Interrupts define no event data, so clear any value left behind + * by a previously injected exception. + / + if (is_fred_enabled(vcpu)) + vmcs_write64(INJECTED_EVENT_DATA, 0); + vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr); vmx_clear_hlt(vcpu); @@ -5173,6 +5393,13 @@ void vmx_inject_nmi(struct kvm_vcpu vcpu) return; } + /* + * KVM doesn't virtualize the NMI-source reporting feature. So clear + * any value left behind by a previously injected exception. + / + if (is_fred_enabled(vcpu)) + vmcs_write64(INJECTED_EVENT_DATA, 0); + vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, INTR_TYPE_NMI_INTR \| INTR_INFO_VALID_MASK \| NMI_VECTOR); @@ -6503,7 +6730,7 @@ void dump_vmcs(struct kvm_vcpu vcpu) struct vcpu_vmx vmx = to_vmx(vcpu); u32 vmentry_ctl, vmexit_ctl; u32 cpu_based_exec_ctrl, pin_based_exec_ctrl, secondary_exec_control; - u64 tertiary_exec_control; + u64 tertiary_exec_control, secondary_vmexit_ctl; unsigned long cr4; int efer_slot; @@ -6514,6 +6741,8 @@ void dump_vmcs(struct kvm_vcpu vcpu) vmentry_ctl = vmcs_read32(VM_ENTRY_CONTROLS); vmexit_ctl = vmcs_read32(VM_EXIT_CONTROLS); + secondary_vmexit_ctl = cpu_has_secondary_vmexit_ctrls() ? + vmcs_read64(SECONDARY_VM_EXIT_CONTROLS) : 0; cpu_based_exec_ctrl = vmcs_read32(CPU_BASED_VM_EXEC_CONTROL); pin_based_exec_ctrl = vmcs_read32(PIN_BASED_VM_EXEC_CONTROL); cr4 = vmcs_readl(GUEST_CR4); @@ -6560,6 +6789,16 @@ void dump_vmcs(struct kvm_vcpu vcpu) vmx_dump_sel("LDTR:", GUEST_LDTR_SELECTOR); vmx_dump_dtsel("IDTR:", GUEST_IDTR_LIMIT); vmx_dump_sel("TR: ", GUEST_TR_SELECTOR); + if (vmentry_ctl & VM_ENTRY_LOAD_IA32_FRED) + pr_err("FRED guest: config=0x%016llx, stack_levels=0x%016llx\n" + "RSP0=0x%016llx, RSP1=0x%016llx\n" + "RSP2=0x%016llx, RSP3=0x%016llx\n", + vmcs_read64(GUEST_IA32_FRED_CONFIG), + vmcs_read64(GUEST_IA32_FRED_STKLVLS), + vmx_read_guest_fred_rsp0(vmx), + vmcs_read64(GUEST_IA32_FRED_RSP1), + vmcs_read64(GUEST_IA32_FRED_RSP2), + vmcs_read64(GUEST_IA32_FRED_RSP3)); efer_slot = vmx_find_loadstore_msr_slot(&vmx->msr_autoload.guest, MSR_EFER); if (vmentry_ctl & VM_ENTRY_LOAD_IA32_EFER) pr_err("EFER= 0x%016llx\n", vmcs_read64(GUEST_IA32_EFER)); @@ -6611,6 +6850,16 @@ void dump_vmcs(struct kvm_vcpu vcpu) vmcs_readl(HOST_TR_BASE)); pr_err("GDTBase=%016lx IDTBase=%016lx\n", vmcs_readl(HOST_GDTR_BASE), vmcs_readl(HOST_IDTR_BASE)); + if (secondary_vmexit_ctl & SECONDARY_VM_EXIT_LOAD_IA32_FRED) + pr_err("FRED host: config=0x%016llx, stack_levels=0x%016llx\n" + "RSP0=0x%016lx, RSP1=0x%016llx\n" + "RSP2=0x%016llx, RSP3=0x%016llx\n", + vmcs_read64(HOST_IA32_FRED_CONFIG), + vmcs_read64(HOST_IA32_FRED_STKLVLS), + (unsigned long)task_stack_page(current) + THREAD_SIZE, + vmcs_read64(HOST_IA32_FRED_RSP1), + vmcs_read64(HOST_IA32_FRED_RSP2), + vmcs_read64(HOST_IA32_FRED_RSP3)); pr_err("CR0=%016lx CR3=%016lx CR4=%016lx\n", vmcs_readl(HOST_CR0), vmcs_readl(HOST_CR3), vmcs_readl(HOST_CR4)); @@ -6636,25 +6885,29 @@ void dump_vmcs(struct kvm_vcpu vcpu) pr_err(" Control State \n"); pr_err("CPUBased=0x%08x SecondaryExec=0x%08x TertiaryExec=0x%016llx\n", cpu_based_exec_ctrl, secondary_exec_control, tertiary_exec_control); - pr_err("PinBased=0x%08x EntryControls=%08x ExitControls=%08x\n", - pin_based_exec_ctrl, vmentry_ctl, vmexit_ctl); + pr_err("PinBased=0x%08x EntryControls=0x%08x\n", + pin_based_exec_ctrl, vmentry_ctl); + pr_err("ExitControls=0x%08x SecondaryExitControls=0x%016llx\n", + vmexit_ctl, secondary_vmexit_ctl); pr_err("ExceptionBitmap=%08x PFECmask=%08x PFECmatch=%08x\n", vmcs_read32(EXCEPTION_BITMAP), vmcs_read32(PAGE_FAULT_ERROR_CODE_MASK), vmcs_read32(PAGE_FAULT_ERROR_CODE_MATCH)); - pr_err("VMEntry: intr_info=%08x errcode=%08x ilen=%08x\n", + pr_err("VMEntry: intr_info=%08x errcode=%08x ilen=%08x event_data=%016llx\n", vmcs_read32(VM_ENTRY_INTR_INFO_FIELD), vmcs_read32(VM_ENTRY_EXCEPTION_ERROR_CODE), - vmcs_read32(VM_ENTRY_INSTRUCTION_LEN)); + vmcs_read32(VM_ENTRY_INSTRUCTION_LEN), + kvm_cpu_cap_has(X86_FEATURE_FRED) ? vmcs_read64(INJECTED_EVENT_DATA) : 0); pr_err("VMExit: intr_info=%08x errcode=%08x ilen=%08x\n", vmcs_read32(VM_EXIT_INTR_INFO), vmcs_read32(VM_EXIT_INTR_ERROR_CODE), vmcs_read32(VM_EXIT_INSTRUCTION_LEN)); pr_err(" reason=%08x qualification=%016lx\n", vmcs_read32(VM_EXIT_REASON), vmcs_readl(EXIT_QUALIFICATION)); - pr_err("IDTVectoring: info=%08x errcode=%08x\n", + pr_err("IDTVectoring: info=%08x errcode=%08x event_data=%016llx\n", vmcs_read32(IDT_VECTORING_INFO_FIELD), - vmcs_read32(IDT_VECTORING_ERROR_CODE)); + vmcs_read32(IDT_VECTORING_ERROR_CODE), + kvm_cpu_cap_has(X86_FEATURE_FRED) ? vmcs_read64(ORIGINAL_EVENT_DATA) : 0); pr_err("TSC Offset = 0x%016llx\n", vmcs_read64(TSC_OFFSET)); if (secondary_exec_control & SECONDARY_EXEC_TSC_SCALING) pr_err("TSC Multiplier = 0x%016llx\n", @@ -7280,7 +7533,8 @@ static void vmx_recover_nmi_blocking(struct vcpu_vmx vmx) static void __vmx_complete_interrupts(struct kvm_vcpu vcpu, u32 idt_vectoring_info, int instr_len_field, - int error_code_field) + int error_code_field, + int event_data_field) { u8 vector; int type; @@ -7315,13 +7569,18 @@ static void __vmx_complete_interrupts(struct kvm_vcpu vcpu, fallthrough; case INTR_TYPE_HARD_EXCEPTION: { u32 error_code = 0; + u64 event_data = 0; if (idt_vectoring_info & VECTORING_INFO_DELIVER_CODE_MASK) error_code = vmcs_read32(error_code_field); + if (is_fred_enabled(vcpu)) + event_data = vmcs_read64(event_data_field); kvm_requeue_exception(vcpu, vector, idt_vectoring_info & VECTORING_INFO_DELIVER_CODE_MASK, - error_code); + error_code, + idt_vectoring_info & VECTORING_INFO_NESTED_EXCEPTION_MASK, + event_data); break; } case INTR_TYPE_SOFT_INTR: @@ -7339,7 +7598,8 @@ static void vmx_complete_interrupts(struct vcpu_vmx vmx) { __vmx_complete_interrupts(&vmx->vcpu, vmx->idt_vectoring_info, VM_EXIT_INSTRUCTION_LEN, - IDT_VECTORING_ERROR_CODE); + IDT_VECTORING_ERROR_CODE, + ORIGINAL_EVENT_DATA); } void vmx_cancel_injection(struct kvm_vcpu vcpu) @@ -7347,7 +7607,8 @@ void vmx_cancel_injection(struct kvm_vcpu vcpu) __vmx_complete_interrupts(vcpu, vmcs_read32(VM_ENTRY_INTR_INFO_FIELD), VM_ENTRY_INSTRUCTION_LEN, - VM_ENTRY_EXCEPTION_ERROR_CODE); + VM_ENTRY_EXCEPTION_ERROR_CODE, + INJECTED_EVENT_DATA); vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, 0); } @@ -7459,6 +7720,10 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu vcpu, vmx_disable_fb_clear(vmx); + /* + * Note, even though FRED delivers the faulting linear address via the + * event data field on the stack, CR2 is still updated. + / if (vcpu->arch.cr2 != native_read_cr2()) native_write_cr2(vcpu->arch.cr2); @@ -7885,6 +8150,7 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu vcpu) entry = kvm_find_cpuid_entry_index(vcpu, 0x7, 1); cr4_fixed1_update(X86_CR4_LAM_SUP, eax, feature_bit(LAM)); + cr4_fixed1_update(X86_CR4_FRED, eax, feature_bit(FRED)); #undef cr4_fixed1_update } @@ -8095,6 +8361,9 @@ static __init void vmx_set_cpu_caps(void) kvm_cpu_cap_check_and_set(X86_FEATURE_DTES64); } + if (!cpu_has_vmx_fred()) + kvm_cpu_cap_clear(X86_FEATURE_FRED); + if (!enable_pmu) kvm_cpu_cap_clear(X86_FEATURE_PDCM); kvm_caps.supported_perf_cap = vmx_get_perf_capabilities(); @@ -8853,6 +9122,11 @@ __init int vmx_hardware_setup(void) kvm_caps.inapplicable_quirks &= ~KVM_X86_QUIRK_IGNORE_GUEST_PAT; + if (cpu_feature_enabled(X86_FEATURE_FRED) && kvm_cpu_cap_has(X86_FEATURE_FRED)) { + rdmsrq(MSR_IA32_FRED_CONFIG, kvm_host.fred_config); + rdmsrq(MSR_IA32_FRED_STKLVLS, kvm_host.fred_stklvls); + } + return 0; } diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index dc8517f15bc46..cc1e4038cd441 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -182,6 +182,29 @@ struct nested_vmx { u64 pre_vmenter_ssp; u64 pre_vmenter_ssp_tbl; + /* + * Used to snapshot FRED MSRs that may NOT be saved to vmcs12 as specified + * in the VM-Exit controls of vmcs12 configured by L1 VMM. + * + * FRED MSRs are always saved into vmcs02 because KVM always sets + * SECONDARY_VM_EXIT_SAVE_IA32_FRED. However an L1 VMM may choose to clear + * this bit, resulting in FRED MSRs not being propagated to vmcs12 from + * vmcs02. When the L1 VMM sets SECONDARY_VM_EXIT_LOAD_IA32_FRED, this is + * not a problem, since KVM then immediately loads the host FRED MSRs of + * vmcs12 to the guest FRED MSRs of vmcs01. + * + * But if the L1 VMM clears SECONDARY_VM_EXIT_LOAD_IA32_FRED, KVM should + * retain the FRED MSRs, i.e., propagate the guest FRED MSRs of vmcs02 to + * the guest FRED MSRs of vmcs01. + * + * This structure stores guest FRED MSRs that an L1 VMM opts not to save + * during VM-Exits from L2 to L1. These MSRs may still be retained for + * running the L1 VMM if SECONDARY_VM_EXIT_LOAD_IA32_FRED is cleared in + * vmcs12. + / + struct vmcs_fred_msrs pre_vmenter_fred_msrs; + struct vmcs_fred_msrs at_vmexit_fred_msrs; + u16 vpid02; u16 last_vpid; @@ -222,6 +245,7 @@ struct vcpu_vmx { bool guest_uret_msrs_loaded; #ifdef CONFIG_X86_64 u64 msr_guest_kernel_gs_base; + u64 msr_guest_fred_rsp0; #endif u64 spec_ctrl; @@ -477,7 +501,8 @@ static inline u8 vmx_get_rvi(void) VM_ENTRY_LOAD_BNDCFGS \| \ VM_ENTRY_PT_CONCEAL_PIP \| \ VM_ENTRY_LOAD_IA32_RTIT_CTL \| \ - VM_ENTRY_LOAD_CET_STATE) + VM_ENTRY_LOAD_CET_STATE \| \ + VM_ENTRY_LOAD_IA32_FRED) #define __KVM_REQUIRED_VMX_VM_EXIT_CONTROLS \ (VM_EXIT_SAVE_DEBUG_CONTROLS \| \ @@ -501,7 +526,13 @@ static inline u8 vmx_get_rvi(void) VM_EXIT_PT_CONCEAL_PIP \| \ VM_EXIT_CLEAR_IA32_RTIT_CTL \| \ VM_EXIT_LOAD_CET_STATE \| \ - VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL) + VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL \| \ + VM_EXIT_ACTIVATE_SECONDARY_CONTROLS) + +#define KVM_REQUIRED_VMX_SECONDARY_VM_EXIT_CONTROLS (0) +#define KVM_OPTIONAL_VMX_SECONDARY_VM_EXIT_CONTROLS \ + (SECONDARY_VM_EXIT_SAVE_IA32_FRED \| \ + SECONDARY_VM_EXIT_LOAD_IA32_FRED) #define KVM_REQUIRED_VMX_PIN_BASED_VM_EXEC_CONTROL \ (PIN_BASED_EXT_INTR_MASK \| \ @@ -615,6 +646,7 @@ static __always_inline void lname##_controls_changebit(struct vcpu_vmx vmx, u## } BUILD_CONTROLS_SHADOW(vm_entry, VM_ENTRY_CONTROLS, 32) BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS, 32) +BUILD_CONTROLS_SHADOW(secondary_vm_exit, SECONDARY_VM_EXIT_CONTROLS, 64) BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL, 32) BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL, 32) BUILD_CONTROLS_SHADOW(secondary_exec, SECONDARY_VM_EXEC_CONTROL, 32) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 4b3681796c752..f264bc9c35f9d 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -431,9 +431,22 @@ void kvm_deliver_exception_payload(struct kvm_vcpu vcpu, breakpoint), it is reserved and must be zero in DR6. / vcpu->arch.dr6 &= ~BIT(12); + + / + * FRED #DB event data matches DR6, but follows the polarity of + * VMX's pending debug exceptions, not DR6. + / + ex->event_data = ex->payload & ~BIT(12); + break; + case NM_VECTOR: + ex->event_data = ex->payload; break; case PF_VECTOR: vcpu->arch.cr2 = ex->payload; + ex->event_data = ex->payload; + break; + default: + ex->event_data = 0; break; } @@ -461,6 +474,7 @@ static void kvm_multiple_exception(struct kvm_vcpu vcpu, unsigned int nr, bool has_error, u32 error_code, bool has_payload, unsigned long payload) { + bool is_nested = false; u32 prev_nr; int class1, class2; @@ -487,6 +501,15 @@ static void kvm_multiple_exception(struct kvm_vcpu vcpu, unsigned int nr, vcpu->arch.exception.error_code = error_code; vcpu->arch.exception.has_payload = has_payload; vcpu->arch.exception.payload = payload; + / #DF is NOT a nested event, per its definition. / + vcpu->arch.exception.is_nested = (nr != DF_VECTOR) && + (is_nested \|\| + vcpu->arch.nmi_injected \|\| + vcpu->arch.interrupt.injected); + / + * Do not update the event data, as the current value may have + * just been restored during a VM save/restore or live migration. + / return; } @@ -510,6 +533,9 @@ static void kvm_multiple_exception(struct kvm_vcpu vcpu, unsigned int nr, kvm_queue_exception_e(vcpu, DF_VECTOR, 0); } else { + /* The new exception arrived while delivering the previous one. / + is_nested = true; + / replace previous exception with a new one in a hope that instruction re-execution will regenerate lost exception / @@ -538,7 +564,8 @@ static void kvm_queue_exception_e_p(struct kvm_vcpu vcpu, unsigned nr, } void kvm_requeue_exception(struct kvm_vcpu vcpu, unsigned int nr, - bool has_error_code, u32 error_code) + bool has_error_code, u32 error_code, + bool is_nested, u64 event_data) { / @@ -563,6 +590,8 @@ void kvm_requeue_exception(struct kvm_vcpu vcpu, unsigned int nr, vcpu->arch.exception.error_code = error_code; vcpu->arch.exception.has_payload = false; vcpu->arch.exception.payload = 0; + vcpu->arch.exception.is_nested = is_nested; + vcpu->arch.exception.event_data = event_data; } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_requeue_exception); @@ -2288,6 +2317,9 @@ int kvm_vm_ioctl_check_extension(struct kvm kvm, long ext) case KVM_CAP_PRE_FAULT_MEMORY: r = tdp_enabled; break; + case KVM_CAP_X86_FRED_EVENT: + r = kvm_cpu_cap_has(X86_FEATURE_FRED); + break; case KVM_CAP_X86_APIC_BUS_CYCLES_NS: r = kvm ? kvm->arch.apic_bus_cycle_ns : APIC_BUS_CYCLE_NS_DEFAULT; break; @@ -2997,6 +3029,8 @@ static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu vcpu, events->exception.error_code = ex->error_code; events->exception_has_payload = ex->has_payload; events->exception_payload = ex->payload; + events->exception_is_nested = ex->is_nested; + events->exception_event_data = ex->event_data; events->interrupt.injected = vcpu->arch.interrupt.injected && !vcpu->arch.interrupt.soft; @@ -3022,6 +3056,8 @@ static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu vcpu, \| KVM_VCPUEVENT_VALID_SMM); if (vcpu->kvm->arch.exception_payload_enabled) events->flags \|= KVM_VCPUEVENT_VALID_PAYLOAD; + if (vcpu->kvm->arch.exception_fred_state_enabled) + events->flags \|= KVM_VCPUEVENT_VALID_FRED_STATE; if (vcpu->kvm->arch.triple_fault_event) { events->triple_fault.pending = kvm_test_request(KVM_REQ_TRIPLE_FAULT, vcpu); events->flags \|= KVM_VCPUEVENT_VALID_TRIPLE_FAULT; @@ -3036,6 +3072,7 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu vcpu, \| KVM_VCPUEVENT_VALID_SHADOW \| KVM_VCPUEVENT_VALID_SMM \| KVM_VCPUEVENT_VALID_PAYLOAD + \| KVM_VCPUEVENT_VALID_FRED_STATE \| KVM_VCPUEVENT_VALID_TRIPLE_FAULT)) return -EINVAL; @@ -3051,6 +3088,14 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu vcpu, events->exception_has_payload = 0; } + if (events->flags & KVM_VCPUEVENT_VALID_FRED_STATE) { + if (!vcpu->kvm->arch.exception_fred_state_enabled) + return -EINVAL; + } else { + events->exception_is_nested = 0; + events->exception_event_data = 0; + } + if ((events->exception.injected \|\| events->exception.pending) && (events->exception.nr > 31 \|\| events->exception.nr == NMI_VECTOR)) return -EINVAL; @@ -3076,6 +3121,8 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu vcpu, vcpu->arch.exception.error_code = events->exception.error_code; vcpu->arch.exception.has_payload = events->exception_has_payload; vcpu->arch.exception.payload = events->exception_payload; + vcpu->arch.exception.is_nested = events->exception_is_nested; + vcpu->arch.exception.event_data = events->exception_event_data; vcpu->arch.interrupt.injected = events->interrupt.injected; vcpu->arch.interrupt.nr = events->interrupt.nr; @@ -4038,6 +4085,13 @@ int kvm_vm_ioctl_enable_cap(struct kvm kvm, kvm->arch.exception_payload_enabled = cap->args[0]; r = 0; break; + case KVM_CAP_X86_FRED_EVENT: + r = -EINVAL; + if (!kvm_cpu_cap_has(X86_FEATURE_FRED)) + break; + kvm->arch.exception_fred_state_enabled = cap->args[0]; + r = 0; + break; case KVM_CAP_X86_TRIPLE_FAULT_EVENT: kvm->arch.triple_fault_event = cap->args[0]; r = 0; @@ -5498,11 +5552,17 @@ static int emulator_set_dr(struct x86_emulate_ctxt ctxt, int dr, return kvm_set_dr(emul_to_vcpu(ctxt), dr, value); } -static u64 mk_cr_64(u64 curr_cr, u32 new_val) +static u64 mk_cr0_64(u64 curr_cr, u32 new_val) { return (curr_cr & ~((1ULL << 32) - 1)) \| new_val; } +static u64 mk_cr4_64(struct kvm_vcpu vcpu, u64 curr_cr, u64 new_val) +{ + u32 shift = guest_cpu_cap_has(vcpu, X86_FEATURE_FRED) ? 33 : 32; + return (curr_cr & ~((1ULL << shift) - 1)) \| new_val; +} + static unsigned long emulator_get_cr(struct x86_emulate_ctxt ctxt, int cr) { struct kvm_vcpu vcpu = emul_to_vcpu(ctxt); @@ -5539,7 +5599,7 @@ static int emulator_set_cr(struct x86_emulate_ctxt ctxt, int cr, ulong val) switch (cr) { case 0: - res = kvm_set_cr0(vcpu, mk_cr_64(kvm_read_cr0(vcpu), val)); + res = kvm_set_cr0(vcpu, mk_cr0_64(kvm_read_cr0(vcpu), val)); break; case 2: vcpu->arch.cr2 = val; @@ -5548,7 +5608,7 @@ static int emulator_set_cr(struct x86_emulate_ctxt ctxt, int cr, ulong val) res = kvm_set_cr3(vcpu, val); break; case 4: - res = kvm_set_cr4(vcpu, mk_cr_64(kvm_read_cr4(vcpu), val)); + res = kvm_set_cr4(vcpu, mk_cr4_64(vcpu, kvm_read_cr4(vcpu), val)); break; case 8: res = kvm_set_cr8(vcpu, val); @@ -8960,6 +9020,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu vcpu) ex->has_payload, ex->payload); ex->injected = false; ex->pending = false; + ex->is_nested = false; } vcpu->arch.exception_from_userspace = false; diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 0f5919b092e47..43c5c08f5d5ca 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -158,6 +158,8 @@ static inline void kvm_clear_exception_queue(struct kvm_vcpu vcpu) { vcpu->arch.exception.pending = false; vcpu->arch.exception.injected = false; + vcpu->arch.exception.is_nested = false; + vcpu->arch.exception.event_data = 0; vcpu->arch.exception_vmexit.pending = false; } @@ -502,7 +504,8 @@ void kvm_queue_exception(struct kvm_vcpu vcpu, unsigned nr); void kvm_queue_exception_e(struct kvm_vcpu vcpu, unsigned nr, u32 error_code); void kvm_queue_exception_p(struct kvm_vcpu vcpu, unsigned nr, unsigned long payload); void kvm_requeue_exception(struct kvm_vcpu vcpu, unsigned int nr, - bool has_error_code, u32 error_code); + bool has_error_code, u32 error_code, + bool is_nested, u64 event_data); void kvm_inject_page_fault(struct kvm_vcpu vcpu, struct x86_exception fault, bool from_hardware); void __kvm_inject_emulated_page_fault(struct kvm_vcpu vcpu, diff --git a/arch/x86/mm/cpu_entry_area.c b/arch/x86/mm/cpu_entry_area.c index 575f863f3c75e..a47dfd686bd33 100644 --- a/arch/x86/mm/cpu_entry_area.c +++ b/arch/x86/mm/cpu_entry_area.c @@ -18,6 +18,26 @@ static DEFINE_PER_CPU_PAGE_ALIGNED(struct entry_stack_page, entry_stack_storage) static DEFINE_PER_CPU_PAGE_ALIGNED(struct exception_stacks, exception_stacks); DEFINE_PER_CPU(struct cea_exception_stacks, cea_exception_stacks); +/* + * Typically invoked by entry code, so must be noinstr. + / +noinstr unsigned long __this_cpu_ist_bottom_va(enum exception_stack_ordering stack) +{ + struct cea_exception_stacks s; + + BUILD_BUG_ON(ESTACK_DF != 0); + BUG_ON(stack >= N_EXCEPTION_STACKS); + + s = __this_cpu_read(cea_exception_stacks); + + return (unsigned long)&s->event_stacks[stack].stack; +} + +noinstr unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack) +{ + return __this_cpu_ist_bottom_va(stack) + EXCEPTION_STKSZ; +} + static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, _cea_offset); static __always_inline unsigned int cea_offset(unsigned int cpu) @@ -132,7 +152,7 @@ static void __init percpu_setup_debug_store(unsigned int cpu) #define cea_map_stack(name) do { \ npages = sizeof(estacks->name## _stack) / PAGE_SIZE; \ - cea_map_percpu_pages(cea->estacks.name## _stack, \ + cea_map_percpu_pages(cea->estacks.event_stacks[name].stack, \ estacks->name## _stack, npages, PAGE_KERNEL); \ } while (0) @@ -151,15 +171,15 @@ static void __init percpu_setup_exception_stacks(unsigned int cpu) * by guard pages so each stack must be mapped separately. DB2 is * not mapped; it just exists to catch triple nesting of #DB. / - cea_map_stack(DF); - cea_map_stack(NMI); - cea_map_stack(DB); - cea_map_stack(MCE); + cea_map_stack(ESTACK_DF); + cea_map_stack(ESTACK_NMI); + cea_map_stack(ESTACK_DB); + cea_map_stack(ESTACK_MCE); if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) { - cea_map_stack(VC); - cea_map_stack(VC2); + cea_map_stack(ESTACK_VC); + cea_map_stack(ESTACK_VC2); } } } diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 45b99c3b1442d..f6ee1cc3f125c 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -677,7 +677,7 @@ page_fault_oops(struct pt_regs regs, unsigned long error_code, * and then double-fault, though, because we're likely to * break the console driver and lose most of the stack dump. / - call_on_stack(__this_cpu_ist_top_va(DF) - sizeof(void), + call_on_stack(__this_cpu_ist_top_va(ESTACK_DF) - sizeof(void), handle_stack_overflow, ASM_CALL_ARG3, , [arg1] "r" (regs), [arg2] "r" (address), [arg3] "r" (&info)); diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 9fc8dfdfd65ff..c5ffc4e7ddb14 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -998,6 +998,7 @@ struct kvm_enable_cap { #define KVM_CAP_S390_VSIE_ESAMODE 248 #define KVM_CAP_S390_HPAGE_2G 249 #define KVM_CAP_ARM_PMU_V3_STRICT 250 +#define KVM_CAP_X86_FRED_EVENT 251 struct kvm_irq_routing_irqchip { __u32 irqchip; diff --git a/tools/arch/x86/include/asm/msr-index.h b/tools/arch/x86/include/asm/msr-index.h index 18c4be75e9271..14632e29f9814 100644 --- a/tools/arch/x86/include/asm/msr-index.h +++ b/tools/arch/x86/include/asm/msr-index.h @@ -64,6 +64,8 @@ #define MSR_IA32_FRED_SSP3 0x1d3 / Level 3 shadow stack pointer / #define MSR_IA32_FRED_CONFIG 0x1d4 / Entrypoint and interrupt stack level / +#define FRED_CONFIG_RESERVED (BIT_ULL(2) \| GENMASK_ULL(5, 4) \| BIT_ULL(11)) + / Intel MSRs. Some also available on other CPUs / #define MSR_TEST_CTRL 0x00000033 #define MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT 29 diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm index 96bab7002d39e..73aed018f4a38 100644 --- a/tools/testing/selftests/kvm/Makefile.kvm +++ b/tools/testing/selftests/kvm/Makefile.kvm @@ -78,6 +78,7 @@ TEST_GEN_PROGS_x86 += x86/feature_msrs_test TEST_GEN_PROGS_x86 += x86/evmcs_smm_controls_test TEST_GEN_PROGS_x86 += x86/exit_on_emulation_failure_test TEST_GEN_PROGS_x86 += x86/fastops_test +TEST_GEN_PROGS_x86 += x86/fred_test TEST_GEN_PROGS_x86 += x86/fix_hypercall_test TEST_GEN_PROGS_x86 += x86/hwcr_msr_test TEST_GEN_PROGS_x86 += x86/hyperv_clock diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h index 5a7a455b53875..981adfcbd5fa2 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -191,6 +191,7 @@ enum vm_guest_mode { VM_MODE_P40V48_16K, VM_MODE_P40V48_64K, VM_MODE_PXXVYY_4K, / For 48-bit or 57-bit VA, depending on host support / + VM_MODE_PXXVYY_4K_USER, / For 48bits VA but ANY bits PA with USER bit set / VM_MODE_P47V64_4K, VM_MODE_P44V64_4K, VM_MODE_P36V48_4K, diff --git a/tools/testing/selftests/kvm/include/x86/processor.h b/tools/testing/selftests/kvm/include/x86/processor.h index 6e6f70035508a..988c45e326894 100644 --- a/tools/testing/selftests/kvm/include/x86/processor.h +++ b/tools/testing/selftests/kvm/include/x86/processor.h @@ -79,6 +79,7 @@ const char ex_str(int vector); #define X86_CR4_SMEP (1ul << 20) #define X86_CR4_SMAP (1ul << 21) #define X86_CR4_PKE (1ul << 22) +#define X86_CR4_FRED (1ul << 32) struct xstate_header { u64 xstate_bv; @@ -195,6 +196,7 @@ struct kvm_x86_cpu_feature { #define X86_FEATURE_SPEC_CTRL KVM_X86_CPU_FEATURE(0x7, 0, EDX, 26) #define X86_FEATURE_ARCH_CAPABILITIES KVM_X86_CPU_FEATURE(0x7, 0, EDX, 29) #define X86_FEATURE_PKS KVM_X86_CPU_FEATURE(0x7, 0, ECX, 31) +#define X86_FEATURE_FRED KVM_X86_CPU_FEATURE(0x7, 1, EAX, 17) #define X86_FEATURE_XTILECFG KVM_X86_CPU_FEATURE(0xD, 0, EAX, 17) #define X86_FEATURE_XTILEDATA KVM_X86_CPU_FEATURE(0xD, 0, EAX, 18) #define X86_FEATURE_XSAVES KVM_X86_CPU_FEATURE(0xD, 1, EAX, 3) @@ -1645,4 +1647,36 @@ u64 tdp_get_pte(struct kvm_vm vm, u64 l2_gpa); bool sys_clocksource_is_based_on_tsc(void); +/* + * FRED related data structures and functions + / + +#define FRED_SSX_NMI BIT_ULL(18) + +struct fred_stack { + u64 r15; + u64 r14; + u64 r13; + u64 r12; + u64 bp; + u64 bx; + u64 r11; + u64 r10; + u64 r9; + u64 r8; + u64 ax; + u64 cx; + u64 dx; + u64 si; + u64 di; + u64 error_code; + u64 ip; + u64 csx; + u64 flags; + u64 sp; + u64 ssx; + u64 event_data; + u64 reserved; +}; + #endif / SELFTEST_KVM_PROCESSOR_H / diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c index 9ddc047d5c275..e79fddedaf396 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -210,6 +210,7 @@ const char vm_guest_mode_string(u32 i) [VM_MODE_P40V48_16K] = "PA-bits:40, VA-bits:48, 16K pages", [VM_MODE_P40V48_64K] = "PA-bits:40, VA-bits:48, 64K pages", [VM_MODE_PXXVYY_4K] = "PA-bits:ANY, VA-bits:48 or 57, 4K pages", + [VM_MODE_PXXVYY_4K_USER]= "PA-bits:ANY, VA-bits:48, 4K user pages", [VM_MODE_P47V64_4K] = "PA-bits:47, VA-bits:64, 4K pages", [VM_MODE_P44V64_4K] = "PA-bits:44, VA-bits:64, 4K pages", [VM_MODE_P36V48_4K] = "PA-bits:36, VA-bits:48, 4K pages", @@ -246,6 +247,7 @@ const struct vm_guest_mode_params vm_guest_mode_params[] = { [VM_MODE_P40V48_16K] = { 40, 48, 0x4000, 14 }, [VM_MODE_P40V48_64K] = { 40, 48, 0x10000, 16 }, [VM_MODE_PXXVYY_4K] = { 0, 0, 0x1000, 12 }, + [VM_MODE_PXXVYY_4K_USER]= { 0, 0, 0x1000, 12 }, [VM_MODE_P47V64_4K] = { 47, 64, 0x1000, 12 }, [VM_MODE_P44V64_4K] = { 44, 64, 0x1000, 12 }, [VM_MODE_P36V48_4K] = { 36, 48, 0x1000, 12 }, @@ -337,6 +339,7 @@ struct kvm_vm ____vm_create(struct vm_shape shape) vm->mmu.pgtable_levels = 3; break; case VM_MODE_PXXVYY_4K: + case VM_MODE_PXXVYY_4K_USER: #ifdef __x86_64__ kvm_get_cpu_address_width(&vm->pa_bits, &vm->va_bits); kvm_init_vm_address_properties(vm); @@ -355,7 +358,7 @@ struct kvm_vm ____vm_create(struct vm_shape shape) vm->mmu.pgtable_levels = 4; } #else - TEST_FAIL("VM_MODE_PXXVYY_4K not supported on non-x86 platforms"); + TEST_FAIL("VM_MODE_PXXVYY_4K(_USER) not supported on non-x86 platforms"); #endif break; case VM_MODE_P47V64_4K: diff --git a/tools/testing/selftests/kvm/lib/x86/processor.c b/tools/testing/selftests/kvm/lib/x86/processor.c index d31fa81ea0756..51d713c4c9de1 100644 --- a/tools/testing/selftests/kvm/lib/x86/processor.c +++ b/tools/testing/selftests/kvm/lib/x86/processor.c @@ -167,6 +167,9 @@ bool kvm_is_tdp_enabled(void) static void virt_mmu_init(struct kvm_vm vm, struct kvm_mmu mmu, struct pte_masks pte_masks) { + TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K \|\| vm->mode == VM_MODE_PXXVYY_4K_USER, + "Unknown or unsupported guest mode: 0x%x", vm->mode); + / If needed, create the top-level page table. / if (!mmu->pgd_created) { mmu->pgd = vm_alloc_page_table(vm); @@ -181,7 +184,7 @@ static void virt_mmu_init(struct kvm_vm vm, struct kvm_mmu mmu, void virt_arch_pgd_alloc(struct kvm_vm vm) { - TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K, + TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K \|\| vm->mode == VM_MODE_PXXVYY_4K_USER, "Unknown or unsupported guest mode: 0x%x", vm->mode); struct pte_masks pte_masks = (struct pte_masks){ @@ -239,6 +242,8 @@ static u64 virt_create_upper_pte(struct kvm_vm vm, pte = PTE_PRESENT_MASK(mmu) \| PTE_READABLE_MASK(mmu) \| PTE_WRITABLE_MASK(mmu) \| PTE_EXECUTABLE_MASK(mmu) \| PTE_ALWAYS_SET_MASK(mmu); + if (vm->mode == VM_MODE_PXXVYY_4K_USER) + pte \|= PTE_USER_MASK(mmu); if (current_level == target_level) pte \|= PTE_HUGE_MASK(mmu) \| (gpa & PHYSICAL_PAGE_MASK); else @@ -266,7 +271,7 @@ void __virt_pg_map(struct kvm_vm vm, struct kvm_mmu mmu, gva_t gva, u64 pte = &mmu->pgd; int current_level; - TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K, + TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K \|\| vm->mode == VM_MODE_PXXVYY_4K_USER, "Unknown or unsupported guest mode: 0x%x", vm->mode); TEST_ASSERT((gva % pg_size) == 0, @@ -316,6 +321,9 @@ void __virt_pg_map(struct kvm_vm vm, struct kvm_mmu mmu, gva_t gva, pte \|= PTE_C_BIT_MASK(mmu); else pte \|= PTE_S_BIT_MASK(mmu); + + if (vm->mode == VM_MODE_PXXVYY_4K_USER) + pte \|= PTE_USER_MASK(mmu); } void virt_arch_pg_map(struct kvm_vm vm, gva_t gva, gpa_t gpa) @@ -372,7 +380,7 @@ static u64 __vm_get_page_table_entry(struct kvm_vm vm, TEST_ASSERT(level >= PG_LEVEL_NONE && level <= mmu->pgtable_levels, "Invalid PG_LEVEL_* '%d'", level); - TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K, + TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K \|\| vm->mode == VM_MODE_PXXVYY_4K_USER, "Unknown or unsupported guest mode: 0x%x", vm->mode); TEST_ASSERT(sparsebit_is_set(vm->vpages_valid, (gva >> vm->page_shift)), "Invalid virtual address, gva: 0x%lx", gva); @@ -645,7 +653,7 @@ static void vcpu_init_sregs(struct kvm_vm vm, struct kvm_vcpu vcpu) { struct kvm_sregs sregs; - TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K, + TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K \|\| vm->mode == VM_MODE_PXXVYY_4K_USER, "Unknown or unsupported guest mode: 0x%x", vm->mode); / Set mode specific system register values. / diff --git a/tools/testing/selftests/kvm/x86/fred_test.c b/tools/testing/selftests/kvm/x86/fred_test.c new file mode 100644 index 0000000000000..c7d08e5253b98 --- /dev/null +++ b/tools/testing/selftests/kvm/x86/fred_test.c @@ -0,0 +1,291 @@ +// SPDX-License-Identifier: GPL-2.0-only +/ + * FRED nested exception tests + * + * Copyright (C) 2023, Intel, Inc. + / +#define _GNU_SOURCE / for program_invocation_short_name / +#include <fcntl.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <sys/ioctl.h> +#include <asm/msr-index.h> + +#include "apic.h" +#include "kvm_util.h" +#include "test_util.h" +#include "guest_modes.h" +#include "processor.h" + +#define IRQ_VECTOR 0xAA + +#define FRED_STKLVL(v,l) (_AT(unsigned long, l) << (2 (v))) +#define FRED_CONFIG_ENTRYPOINT(p) _AT(unsigned long, (p)) + +/* This address is already mapped in guest page table. / +#define FRED_VALID_RSP 0x8000 + +/ + * The following addresses are not yet mapped in both EPT and guest page + * tables at the beginning. As a result, it causes an EPT violation VM + * exit with an original guest #PF to access any of them for the first + * time. + * + * Use these addresses as guest FRED RSP0 to generate nested #PFs to test + * if event data are properly virtualized. + / +static unsigned long fred_invalid_rsp[4] = { + 0x0, + 0xf0000000, + 0xe0000000, + 0xd0000000, +}; + +extern char asm_user_nop[]; +extern char asm_user_ud[]; +extern char asm_done_fault[]; + +extern void asm_test_fault(int test); + +/ + * user level code for triggering faults. + / +asm(".pushsection .text\n" + ".align 4096\n" + + ".type asm_user_nop, @function\n" + "asm_user_nop:\n" + "1: .byte 0x90\n" + "jmp 1b\n" + + ".org asm_user_nop + 16, 0xcc\n" + ".type asm_user_ud, @function\n" + "asm_user_ud:\n" + / Trigger a #UD / + "ud2\n" + + ".align 4096, 0xcc\n" + ".popsection"); + +/ Send current stack level and #PF address / +#define GUEST_SYNC_CSL_FA(__stage, __pf_address) \ + GUEST_SYNC_ARGS(__stage, __pf_address, 0, 0, 0) + +void fred_entry_from_user(struct fred_stack stack) +{ + u32 current_stack_level = rdmsr(MSR_IA32_FRED_CONFIG) & 0x3; + + GUEST_SYNC_CSL_FA(current_stack_level, stack->event_data); + + /* Do NOT go back to user level, continue the next test instead / + stack->ssx = 0x18; + stack->csx = 0x10; + stack->ip = (u64)&asm_done_fault; +} + +void fred_entry_from_kernel(struct fred_stack stack) +{ + /* + * Keep NMI blocked to delay the delivery of the next NMI until + * returning to user level. + * / + stack->ssx &= ~FRED_SSX_NMI; +} + +#define PUSH_REGS \ + "push %rdi\n" \ + "push %rsi\n" \ + "push %rdx\n" \ + "push %rcx\n" \ + "push %rax\n" \ + "push %r8\n" \ + "push %r9\n" \ + "push %r10\n" \ + "push %r11\n" \ + "push %rbx\n" \ + "push %rbp\n" \ + "push %r12\n" \ + "push %r13\n" \ + "push %r14\n" \ + "push %r15\n" + +#define POP_REGS \ + "pop %r15\n" \ + "pop %r14\n" \ + "pop %r13\n" \ + "pop %r12\n" \ + "pop %rbp\n" \ + "pop %rbx\n" \ + "pop %r11\n" \ + "pop %r10\n" \ + "pop %r9\n" \ + "pop %r8\n" \ + "pop %rax\n" \ + "pop %rcx\n" \ + "pop %rdx\n" \ + "pop %rsi\n" \ + "pop %rdi\n" + +/ + * FRED entry points. + / +asm(".pushsection .text\n" + ".type asm_fred_entrypoint_user, @function\n" + ".align 4096\n" + "asm_fred_entrypoint_user:\n" + PUSH_REGS + "movq %rsp, %rdi\n" + "call fred_entry_from_user\n" + POP_REGS + / Do NOT go back to user level, continue the next test instead / + ".byte 0xf2,0x0f,0x01,0xca\n" / ERETS / + + ".org asm_fred_entrypoint_user + 256, 0xcc\n" + ".type asm_fred_entrypoint_kernel, @function\n" + "asm_fred_entrypoint_kernel:\n" + PUSH_REGS + "movq %rsp, %rdi\n" + "call fred_entry_from_kernel\n" + POP_REGS + ".byte 0xf2,0x0f,0x01,0xca\n" / ERETS / + ".align 4096, 0xcc\n" + ".popsection"); + +extern char asm_fred_entrypoint_user[]; + +/ + * Prepare a FRED stack frame for ERETU to return to user level code, + * nop or ud2. + * + * Because FRED RSP0 is deliberately not mapped in guest page table, + * the delivery of interrupt/NMI or #UD from ring 3 causes a nested + * #PF, which is then delivered on FRED RSPx (x is 1, 2 or 3, + * determinated by MSR FRED_STKLVL[PF_VECTOR]). + / +asm(".pushsection .text\n" + ".type asm_test_fault, @function\n" + ".align 4096\n" + "asm_test_fault:\n" + "endbr64\n" + "push %rbp\n" + "mov %rsp, %rbp\n" + "and $(~0x3f), %rsp\n" + "push $0\n" + "push $0\n" + "mov $0x2b, %rax\n" + / Unblock NMI / + "bts $18, %rax\n" + / Set long mode bit / + "bts $57, %rax\n" + "push %rax\n" + / No stack required for the FRED user level test code / + "push $0\n" + "pushf\n" + "pop %rax\n" + / Allow external interrupts / + "bts $9, %rax\n" + "push %rax\n" + "mov $0x33, %rax\n" + "push %rax\n" + "cmp $0, %edi\n" + "jne 1f\n" + "lea asm_user_nop(%rip), %rax\n" + "jmp 2f\n" + "1: lea asm_user_ud(%rip), %rax\n" + "2: push %rax\n" + "push $0\n" + / ERETU to user level code to allow event delivery immediately / + ".byte 0xf3,0x0f,0x01,0xca\n" + "asm_done_fault:\n" + "mov %rbp, %rsp\n" + "pop %rbp\n" + "ret\n" + ".align 4096, 0xcc\n" + ".popsection"); + +/ + * To fully test the underlying FRED VMX code, this test should be run one + * more round with EPT disabled to inject page faults as nested exceptions. + / +static void guest_code(void) +{ + wrmsr(MSR_IA32_FRED_CONFIG, + FRED_CONFIG_ENTRYPOINT(asm_fred_entrypoint_user)); + + wrmsr(MSR_IA32_FRED_RSP1, FRED_VALID_RSP); + wrmsr(MSR_IA32_FRED_RSP2, FRED_VALID_RSP); + wrmsr(MSR_IA32_FRED_RSP3, FRED_VALID_RSP); + + / Enable FRED / + set_cr4(get_cr4() \| X86_CR4_FRED); + + x2apic_enable(); + + wrmsr(MSR_IA32_FRED_STKLVLS, FRED_STKLVL(PF_VECTOR, 1)); + wrmsr(MSR_IA32_FRED_RSP0, fred_invalid_rsp[1]); + / 1: ud2 to generate #UD / + asm_test_fault(1); + + wrmsr(MSR_IA32_FRED_STKLVLS, FRED_STKLVL(PF_VECTOR, 2)); + wrmsr(MSR_IA32_FRED_RSP0, fred_invalid_rsp[2]); + asm volatile("cli"); + / Create a pending interrupt on current vCPU / + x2apic_write_reg(APIC_ICR, APIC_DEST_SELF \| APIC_INT_ASSERT \| + APIC_DM_FIXED \| IRQ_VECTOR); + / Return to ring 3 / + asm_test_fault(0); + x2apic_write_reg(APIC_EOI, 0); + + wrmsr(MSR_IA32_FRED_STKLVLS, FRED_STKLVL(PF_VECTOR, 3)); + wrmsr(MSR_IA32_FRED_RSP0, fred_invalid_rsp[3]); + / + * The first NMI is just to have NMI blocked in ring 0, because + * fred_entry_from_kernel() deliberately clears the NMI bit in + * FRED stack frame. + / + x2apic_write_reg(APIC_ICR, APIC_DEST_SELF \| APIC_INT_ASSERT \| + APIC_DM_NMI \| NMI_VECTOR); + / The second NMI will be delivered after returning to ring 3 / + x2apic_write_reg(APIC_ICR, APIC_DEST_SELF \| APIC_INT_ASSERT \| + APIC_DM_NMI \| NMI_VECTOR); + / Return to ring 3 / + asm_test_fault(0); + + GUEST_DONE(); +} + +int main(int argc, char argv[]) +{ + struct kvm_vcpu vcpu; + struct kvm_vm vm; + struct ucall uc; + uint64_t expected_current_stack_level = 1; + + TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_FRED)); + + vm = __vm_create_with_vcpus(VM_SHAPE(VM_MODE_PXXVYY_4K_USER), 1, 0, + guest_code, &vcpu); + + while (true) { + uint64_t r; + + vcpu_run(vcpu); + + r = get_ucall(vcpu, &uc); + + if (r == UCALL_DONE) + break; + + if (r == UCALL_SYNC) { + TEST_ASSERT((uc.args[1] == expected_current_stack_level) && + (uc.args[2] == fred_invalid_rsp[expected_current_stack_level] - 8), + "Incorrect stack level %lx and #PF address %lx\n", + uc.args[1], uc.args[2]); + expected_current_stack_level++; + } + } + + kvm_vm_free(vm); + return 0; +} diff --git a/tools/testing/selftests/kvm/x86/msrs_test.c b/tools/testing/selftests/kvm/x86/msrs_test.c index f7e39bf887ad3..6e2d8b8f1bd34 100644 --- a/tools/testing/selftests/kvm/x86/msrs_test.c +++ b/tools/testing/selftests/kvm/x86/msrs_test.c @@ -60,9 +60,23 @@ static const u64 u64_val = 0xaaaa5555aaaa5555ull; #define MSR_TEST_CANONICAL(msr, feat) \ __MSR_TEST(msr, #msr, canonical_val, NONCANONICAL, 0, feat) +#define MSR_TEST_CANONICAL2(msr, feat, f2) \ + ____MSR_TEST(msr, #msr, canonical_val, NONCANONICAL, 0, feat, f2, false) + #define MSR_TEST_KVM(msr, val, rsvd, feat) \ ____MSR_TEST(KVM_REG_ ##msr, #msr, val, rsvd, 0, feat, feat, true) +/* FRED RSPs are 64-byte aligned, thus bits 5:0 are all zero / +static const u64 fred_rsp_canonical_val = canonical_val + 0xbc0; + +/ + * FRED SSP[123] are 8-byte aligned, thus bits 2:0 are all zero. + * + * For legacy reasons, FRED SSP0 is allowed to be 4-byte aligned, but treat + * it as 8-byte aligned to simplify the tests. + / +static const u64 fred_ssp_canonical_val = canonical_val + 0xbc8; + / * The main struct must be scoped to a function due to the use of structures to * define features. For the global structure, allocate enough space for the @@ -388,8 +402,9 @@ static void test_msrs(void) MSR_TEST2(MSR_IA32_S_CET, CET_ENDBR_EN, CET_RESERVED, IBT, SHSTK), MSR_TEST2(MSR_IA32_U_CET, CET_SHSTK_EN, CET_RESERVED, SHSTK, IBT), MSR_TEST2(MSR_IA32_U_CET, CET_ENDBR_EN, CET_RESERVED, IBT, SHSTK), - MSR_TEST_CANONICAL(MSR_IA32_PL0_SSP, SHSTK), - MSR_TEST(MSR_IA32_PL0_SSP, canonical_val, canonical_val \| 1, SHSTK), + MSR_TEST_CANONICAL2(MSR_IA32_PL0_SSP, SHSTK, FRED), + MSR_TEST2(MSR_IA32_PL0_SSP, canonical_val, canonical_val \| 1, SHSTK, FRED), + MSR_TEST2(MSR_IA32_PL0_SSP, canonical_val, canonical_val \| 1, FRED, SHSTK), MSR_TEST_CANONICAL(MSR_IA32_PL1_SSP, SHSTK), MSR_TEST(MSR_IA32_PL1_SSP, canonical_val, canonical_val \| 1, SHSTK), MSR_TEST_CANONICAL(MSR_IA32_PL2_SSP, SHSTK), @@ -398,10 +413,35 @@ static void test_msrs(void) MSR_TEST(MSR_IA32_PL3_SSP, canonical_val, canonical_val \| 1, SHSTK), MSR_TEST_KVM(GUEST_SSP, canonical_val, NONCANONICAL, SHSTK), + + MSR_TEST_CANONICAL(MSR_IA32_FRED_RSP0, FRED), + MSR_TEST(MSR_IA32_FRED_RSP0, fred_rsp_canonical_val, fred_rsp_canonical_val \| 1, FRED), + MSR_TEST_CANONICAL(MSR_IA32_FRED_RSP1, FRED), + MSR_TEST(MSR_IA32_FRED_RSP1, fred_rsp_canonical_val, fred_rsp_canonical_val \| 2, FRED), + MSR_TEST_CANONICAL(MSR_IA32_FRED_RSP2, FRED), + MSR_TEST(MSR_IA32_FRED_RSP2, fred_rsp_canonical_val, fred_rsp_canonical_val \| 4, FRED), + MSR_TEST_CANONICAL(MSR_IA32_FRED_RSP3, FRED), + MSR_TEST(MSR_IA32_FRED_RSP3, fred_rsp_canonical_val, fred_rsp_canonical_val \| 7, FRED), + + MSR_TEST_CANONICAL2(MSR_IA32_FRED_SSP0, FRED, SHSTK), + MSR_TEST2(MSR_IA32_FRED_SSP0, fred_ssp_canonical_val, fred_ssp_canonical_val \| 1, SHSTK, FRED), + MSR_TEST2(MSR_IA32_FRED_SSP0, fred_ssp_canonical_val, fred_ssp_canonical_val \| 1, FRED, SHSTK), + MSR_TEST_CANONICAL(MSR_IA32_FRED_SSP1, FRED), + MSR_TEST(MSR_IA32_FRED_SSP1, fred_ssp_canonical_val, fred_ssp_canonical_val \| 2, FRED), + MSR_TEST_CANONICAL(MSR_IA32_FRED_SSP2, FRED), + MSR_TEST(MSR_IA32_FRED_SSP2, fred_ssp_canonical_val, fred_ssp_canonical_val \| 4, FRED), + MSR_TEST_CANONICAL(MSR_IA32_FRED_SSP3, FRED), + MSR_TEST(MSR_IA32_FRED_SSP3, fred_ssp_canonical_val, fred_ssp_canonical_val \| 7, FRED), + + MSR_TEST(MSR_IA32_FRED_STKLVLS, 0xe41be41be41be41b, 0, FRED), + + MSR_TEST_CANONICAL(MSR_IA32_FRED_CONFIG, FRED), + MSR_TEST(MSR_IA32_FRED_CONFIG, canonical_val, canonical_val \| FRED_CONFIG_RESERVED, FRED), }; const struct kvm_x86_cpu_feature feat_none = X86_FEATURE_NONE; const struct kvm_x86_cpu_feature feat_lm = X86_FEATURE_LM; + const struct kvm_x86_cpu_feature feat_fred = X86_FEATURE_FRED; /* * Create three vCPUs, but run them on the same task, to validate KVM's @@ -432,6 +472,14 @@ static void test_msrs(void) * immutable once the vCPU has been run. / for (idx = 0; idx < ARRAY_SIZE(__msrs); idx++) { + / + * Feature word 0x7:1:EAX used by FRED doesn't exist on some CPU + * models, simply skip clearing FRED feature bit. + / + if (!memcmp(&msrs[idx].feature, &feat_fred, sizeof(feat_fred)) && + !kvm_cpu_has(X86_FEATURE_FRED)) + continue; + / * Don't clear LM; selftests are 64-bit only, and KVM doesn't * honor LM=0 for MSRs that are supposed to exist if and only
3/2	2026/09/11 21:49	llm	patch-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 57215 output: 38 thoughts: 1883 Thoughts: Analyzing FRED Support in KVM x86 Alright, let's dive into this patch introducing FRED (Flexible Return and Event Delivery) support for KVM on x86, specifically targeting VMX, nested VMX, and related areas. This is clearly an amd64 architecture change, focusing on a relatively new x86 feature. My initial thought is to apply a "reachability-first" approach. Can this even be tested in a standard virtualized environment like QEMU or GCE? I need to ascertain whether the standard setup supports FRED. The crucial point here is understanding if FRED is hardware-dependent. If so, which CPUs are even capable? Intel Meteor Lake is out, but perhaps Arrow Lake or Lunar Lake. The immediate questions are: Does standard QEMU emulate FRED, or is it a true hardware feature? The check on `kvm_cpu_cap_has(X86_FEATURE_FRED)` is a good start. The presence of `X86_64_F(FRED)` in `kvm_initialize_cpu_caps()` seems to suggest it's supported if the host CPU has the feature and KVM is properly set up. So, does QEMU or GCE even have the FRED feature? If not, could we still reach parts of the code via syzkaller fuzzing of KVM? Thankfully, syzkaller does target /dev/kvm constantly, so that's a key area to explore. Can we reach the modified ioctls even without host FRED support? Now, let's carefully dissect the code changes. I need to understand what's altered. The changes to `Documentation/virt/kvm/api.rst` suggest a user-facing API change. Modifications to `arch/x86/coco/sev/noinstr.c` and `arch/x86/coco/sev/vc-handle.c`, alongside the restructuring of `struct cea_exception_stacks` in `arch/x86/include/asm/cpu_entry_area.h` are critical, as they seem to modify the CPU entry area and exception stack layout. The changes to the `cea_exception_stacks` structure and related functions in `arch/x86/mm/cpu_entry_area.c` are clearly low-level. And the fact that these are used in the TSS setup and fault handling is very interesting. That `call_on_stack` usage in `fault.c` is significant. The `percpu_setup_exception_stacks` changes in `cpu_entry_area.c` are also central to exception stack mapping. Now, for the KVM-specific code. The MSR (Model-Specific Register) changes are telling. The conditional checks on `guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)` determine the behavior when handling specific MSRs related to FRED (like `MSR_IA32_PL1_SSP` and `MSR_IA32_FRED_STKLVLS`, etc.). The key is this: the conditional check for `X86_FEATURE_SHSTK` combined with FRED and the `WARN_ON_ONCE` within the `kvm_set_msr_common` function. Specifically, `WARN_ON_ONCE(msr != MSR_IA32_FRED_SSP0)` is screaming for further examination! This is likely the first potential bug. I need to figure out when this warning could be triggered. I need to trace back callers of `kvm_set_msr_common` and understand how the MSRs are dispatched. This is probably where the interesting edge cases are. The code paths in `arch/x86/kvm/x86.c` and `arch/x86/kvm/vmx/vmx.c` are prime targets for a deeper dive.
4/2	2026/09/11 21:49	tool	grepper	0m Args: { "Expression": "kvm_set_msr_common", "PathPrefix": "arch/x86/kvm/" } Results: { "Output": "arch/x86/kvm/msrs.c=1495=static inline u64 kvm_guest_supported_xfd(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/msrs.c-1500-\narch/x86/kvm/msrs.c:1501:int kvm_set_msr_common(struct kvm_vcpu vcpu, struct msr_data msr_info)\narch/x86/kvm/msrs.c-1502-{\n--\narch/x86/kvm/msrs.c-1913-}\narch/x86/kvm/msrs.c:1914:EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_set_msr_common);\narch/x86/kvm/msrs.c-1915-\n--\narch/x86/kvm/msrs.h=76=int kvm_get_msr_common(struct kvm_vcpu vcpu, struct msr_data msr);\narch/x86/kvm/msrs.h:77:int kvm_set_msr_common(struct kvm_vcpu vcpu, struct msr_data msr);\narch/x86/kvm/msrs.h-78-\n--\narch/x86/kvm/svm/svm.c=2752=static int efer_trap(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/svm/svm.c-2765-\tmsr_info.data = to_svm(vcpu)-\u003evmcb-\u003econtrol.exit_info_1 \u0026 ~EFER_SVME;\narch/x86/kvm/svm/svm.c:2766:\tret = kvm_set_msr_common(vcpu, \u0026msr_info);\narch/x86/kvm/svm/svm.c-2767-\n--\narch/x86/kvm/svm/svm.c=2990=static int svm_set_msr(struct kvm_vcpu vcpu, struct msr_data msr)\n--\narch/x86/kvm/svm/svm.c-3039-\narch/x86/kvm/svm/svm.c:3040:\t\tret = kvm_set_msr_common(vcpu, msr);\narch/x86/kvm/svm/svm.c-3041-\t\tif (ret)\n--\narch/x86/kvm/svm/svm.c-3228-\tdefault:\narch/x86/kvm/svm/svm.c:3229:\t\treturn kvm_set_msr_common(vcpu, msr);\narch/x86/kvm/svm/svm.c-3230-\t}\n--\narch/x86/kvm/vmx/tdx.c=2241=int tdx_set_msr(struct kvm_vcpu vcpu, struct msr_data msr)\n--\narch/x86/kvm/vmx/tdx.c-2256-\narch/x86/kvm/vmx/tdx.c:2257:\t\treturn kvm_set_msr_common(vcpu, msr);\narch/x86/kvm/vmx/tdx.c-2258-\t}\n--\narch/x86/kvm/vmx/vmx.c=2425=int vmx_set_msr(struct kvm_vcpu vcpu, struct msr_data msr_info)\n--\narch/x86/kvm/vmx/vmx.c-2435-\tcase MSR_EFER:\narch/x86/kvm/vmx/vmx.c:2436:\t\tret = kvm_set_msr_common(vcpu, msr_info);\narch/x86/kvm/vmx/vmx.c-2437-\t\tbreak;\n--\narch/x86/kvm/vmx/vmx.c-2450-\tcase MSR_IA32_XFD:\narch/x86/kvm/vmx/vmx.c:2451:\t\tret = kvm_set_msr_common(vcpu, msr_info);\narch/x86/kvm/vmx/vmx.c-2452-\t\t/\n--\narch/x86/kvm/vmx/vmx.c-2571-\tcase MSR_IA32_CR_PAT:\narch/x86/kvm/vmx/vmx.c:2572:\t\tret = kvm_set_msr_common(vcpu, msr_info);\narch/x86/kvm/vmx/vmx.c-2573-\t\tif (ret)\n--\narch/x86/kvm/vmx/vmx.c-2714-\t\t}\narch/x86/kvm/vmx/vmx.c:2715:\t\tret = kvm_set_msr_common(vcpu, msr_info);\narch/x86/kvm/vmx/vmx.c-2716-\t\tbreak;\n--\narch/x86/kvm/vmx/vmx.c-2723-\t\telse\narch/x86/kvm/vmx/vmx.c:2724:\t\t\tret = kvm_set_msr_common(vcpu, msr_info);\narch/x86/kvm/vmx/vmx.c-2725-\t}\n--\narch/x86/kvm/x86.c=10643=void kvm_arch_async_page_present_queued(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/x86.c-10646-\narch/x86/kvm/x86.c:10647:\t/* Pairs with smp_store_mb() in kvm_set_msr_common(). */\narch/x86/kvm/x86.c-10648-\tsmp_mb__after_atomic();\n" }
5/2	2026/09/11 21:49	llm	patch-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 61576 output: 37 thoughts: 48 Thoughts: Here's my summary, as if I were thinking it: Digging into KVM's MSR Handling Okay, so I need to understand how KVM interacts with Model Specific Registers (MSRs) at the architectural level. My immediate thought is to zero in on the core functions. I need to refresh my memory on `kvm_set_msr` and `kvm_get_msr`. That's where I should find the heart of the MSR manipulation logic within the KVM hypervisor. I'm pretty sure those are in `arch/x86/kvm/x86.c`, or at least closely related. But just to be safe, I'll trace their call sites to be sure. I'm going to track down every place those functions are referenced to get a complete picture of the control flow.
6/2	2026/09/11 21:49	tool	grepper	0m Args: { "Expression": "kvm_set_msr\\(", "PathPrefix": "arch/x86/kvm/" } Results: { "Output": "arch/x86/kvm/msrs.c=695=EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_msr_allowed);\n--\narch/x86/kvm/msrs.c-702- /\narch/x86/kvm/msrs.c:703:static int __kvm_set_msr(struct kvm_vcpu vcpu, u32 index, u64 data,\narch/x86/kvm/msrs.c-704-\t\t\t bool host_initiated)\n--\narch/x86/kvm/msrs.c-848-\narch/x86/kvm/msrs.c:849:static int _kvm_set_msr(struct kvm_vcpu vcpu, u32 index, u64 data,\narch/x86/kvm/msrs.c-850-\t\t\tbool host_initiated)\narch/x86/kvm/msrs.c-851-{\narch/x86/kvm/msrs.c:852:\treturn __kvm_set_msr(vcpu, index, data, host_initiated);\narch/x86/kvm/msrs.c-853-}\n--\narch/x86/kvm/msrs.c=925=int kvm_msr_write(struct kvm_vcpu vcpu, u32 index, u64 data)\narch/x86/kvm/msrs.c-926-{\narch/x86/kvm/msrs.c:927:\treturn __kvm_set_msr(vcpu, index, data, true);\narch/x86/kvm/msrs.c-928-}\n" }
7/2	2026/09/11 21:49	llm	patch-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 62295 output: 41 thoughts: 44 Thoughts: Here's my summary, as though I'm thinking through this process: Diving into the `__kvm_set_msr` Implementation Okay, time to dig into the nitty-gritty of how MSR writes are handled in KVM. I'm going to jump into `arch/x86/kvm/msrs.c`, specifically around lines 770 to 850. That should give me the core logic for `__kvm_set_msr`. I need to refresh my memory on the interactions here, especially given some recent performance tweaks I've been considering. I'm keen to see how the code handles the actual mechanics of writing to the model-specific registers – what checks are done, what data structures are involved, and, crucially, how it interacts with the virtual CPU state. This is fundamental; understanding the write path is key to optimizing any MSR-related operations within a KVM context. I'll be looking for any potential bottlenecks or areas where we might be able to improve efficiency, particularly in terms of context switching or data movement. Let's see what gems this section of code reveals.
8/2	2026/09/11 21:49	tool	read-file	0m Args: { "File": "arch/x86/kvm/msrs.c", "FirstLine": 765, "LineCount": 60 } Results: { "Contents": " 765:\t\t\tif (!host_initiated)\n 766:\t\t\t\treturn 1;\n 767:\t\t\tfallthrough;\n 768:\t\t\t/\n 769:\t\t\t Note that the MSR emulation here is flawed when a vCPU\n 770:\t\t\t * doesn't support the Intel 64 architecture. The expected\n 771:\t\t\t * architectural behavior in this case is that the upper 32\n 772:\t\t\t * bits do not exist and should always read '0'. However,\n 773:\t\t\t * because the actual hardware on which the virtual CPU is\n 774:\t\t\t * running does support Intel 64, XRSTORS/XSAVES in the\n 775:\t\t\t * guest could observe behavior that violates the\n 776:\t\t\t * architecture. Intercepting XRSTORS/XSAVES for this\n 777:\t\t\t * special case isn't deemed worthwhile.\n 778:\t\t\t /\n 779:\t\tcase MSR_IA32_PL1_SSP ... MSR_IA32_INT_SSP_TAB:\n 780:\t\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK))\n 781:\t\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n 782:\t\t\t/\n 783:\t\t\t * MSR_IA32_INT_SSP_TAB is not present on processors that do\n 784:\t\t\t * not support Intel 64 architecture.\n 785:\t\t\t /\n 786:\t\t\tif (index == MSR_IA32_INT_SSP_TAB \u0026\u0026 !guest_cpu_cap_has(vcpu, X86_FEATURE_LM))\n 787:\t\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n 788:\t\t\tif (is_noncanonical_msr_address(data, vcpu))\n 789:\t\t\t\treturn 1;\n 790:\t\t\t/ All SSP MSRs except MSR_IA32_INT_SSP_TAB must be 4-byte aligned */\n 791:\t\t\tif (index != MSR_IA32_INT_SSP_TAB \u0026\u0026 !IS_ALIGNED(data, 4))\n 792:\t\t\t\treturn 1;\n 793:\t\t\tbreak;\n 794:\t\tcase MSR_IA32_FRED_STKLVLS:\n 795:\t\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))\n 796:\t\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n 797:\t\t\tbreak;\n 798:\t\tcase MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_RSP3:\n 799:\t\tcase MSR_IA32_FRED_SSP1 ... MSR_IA32_FRED_CONFIG: {\n 800:\t\t\tu64 reserved_bits = 0;\n 801:\t\n 802:\t\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))\n 803:\t\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n 804:\t\n 805:\t\t\tif (is_noncanonical_msr_address(data, vcpu))\n 806:\t\t\t\treturn 1;\n 807:\t\n 808:\t\t\tswitch (index) {\n 809:\t\t\tcase MSR_IA32_FRED_CONFIG:\n 810:\t\t\t\treserved_bits = FRED_CONFIG_RESERVED;\n 811:\t\t\t\tbreak;\n 812:\t\t\tcase MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_RSP3:\n 813:\t\t\t\treserved_bits = GENMASK_ULL(5, 0);\n 814:\t\t\t\tbreak;\n 815:\t\t\tcase MSR_IA32_FRED_SSP1 ... MSR_IA32_FRED_SSP3:\n 816:\t\t\t\treserved_bits = GENMASK_ULL(2, 0);\n 817:\t\t\t\tbreak;\n 818:\t\t\tdefault:\n 819:\t\t\t\tWARN_ON_ONCE(1);\n 820:\t\t\t\treturn 1;\n 821:\t\t\t}\n 822:\t\n 823:\t\t\tif (data \u0026 reserved_bits)\n 824:\t\t\t\treturn 1;\n" }
9/2	2026/09/11 21:49	llm	patch-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 63725 output: 42 thoughts: 3172 Thoughts: KVM FRED Virtualization Analysis: A Deep Dive Alright, let's break this down systematically. My initial thought process revolves around tracing the flow of MSR (Model Specific Register) writes within the KVM hypervisor, particularly concerning FRED (Fast Return Exception Delivery). I'm starting with the `__kvm_set_msr` function and its interactions with `vmx_set_msr` and `kvm_set_msr_common`. The immediate question is, what happens when specific MSRs, like `MSR_IA32_PL0_SSP`, are targeted? Because there isn't a direct case for it, I see that the code defaults to a common path, `kvm_set_msr_common`, which contains some logic dependent on `X86_FEATURE_SHSTK`. The next logical step is to see where the filter is, given that there's a guard to that check, the code may get called another way, such as `KVM_SET_CPUID2`. I then investigate the `svm_set_msr` path and quickly notice that SVM (Secure Virtual Machine) doesn't seem to have explicit FRED virtualization. It seems to defer to `kvm_set_msr_common` for MSR writes, and the `X86_FEATURE_FRED` is cleared. The core of my concern now is whether this code is even reachable, which means can a fuzzer such as syzkaller trigger these code paths? My thought process then expands to consider the broader attack surface. I confirm that KVM code is reachable via standard syzkaller fuzzing because it utilizes `/dev/kvm` ioctls like `KVM_CREATE_VM`, `KVM_CREATE_VCPU`, `KVM_SET_CPUID2`, `KVM_ENABLE_CAP`, `KVM_SET_MSRS`, etc. This means any discovered vulnerability could potentially be exploited by creating malicious KVM VMs with carefully configured CPUID data. The subsequent phase of my analysis becomes focused on understanding the implications of patch changes related to `FRED_EVENT` capability and how the kernel handles CPUID capabilities and events. I examine the newly introduced `KVM_CAP_X86_FRED_EVENT`, `KVM_SET_VCPU_EVENTS`, and changes to `mk_cr4_64`, specifically noticing the inclusion of the FRED bit in CR4 manipulation. It's a critical point to realize that modifying CR4 directly impacts the processor's behavior, and how CR4 is changed is essential to security. I now need to consider how these changes interact with the x86 instruction emulator and SMM (System Management Mode) emulation. The emulator's interaction with `CR4` and SMM's use of FRED are now relevant attack vectors. The analysis is then expanded to include core kernel changes outside KVM. I'm now looking at `cpu_entry_area.c` and associated files, where there have been changes to exception stack handling. The introduction of `__this_cpu_ist_bottom_va` and `__this_cpu_ist_top_va` functions and their use within the FRED stack framework is another point of concern. The critical insight here is how these routines interact with exception handling, especially in the context of FRED and the potentially modified exception stack layouts, and how this is used within routines such as `dumpstack_64.c`. I need to fully understand how these new function interactions could be exploited. This involves tracing how these new functions are used during boot, exception handling, and stack dumps, looking for any potential vulnerabilities.
10/2	2026/09/11 21:49	tool	read-file	0m Args: { "File": "arch/x86/kernel/dumpstack_64.c", "FirstLine": 50, "LineCount": 55 } Results: { "Contents": " 50:\t\t\treturn \"ENTRY_TRAMPOLINE\";\n 51:\t\t}\n 52:\t\n 53:\t\tif (type \u003e= STACK_TYPE_EXCEPTION \u0026\u0026 type \u003c= STACK_TYPE_EXCEPTION_LAST)\n 54:\t\t\treturn exception_stack_names[type - STACK_TYPE_EXCEPTION];\n 55:\t\n 56:\t\treturn NULL;\n 57:\t}\n 58:\t\n 59:\t/*\n 60:\t struct estack_pages - Page descriptor for exception stacks\n 61:\t * @offs:\tOffset from the start of the exception stack area\n 62:\t * @size:\tSize of the exception stack\n 63:\t * @type:\tType to store in the stack_info struct\n 64:\t /\n 65:\tstruct estack_pages {\n 66:\t\tu32\toffs;\n 67:\t\tu16\tsize;\n 68:\t\tu16\ttype;\n 69:\t};\n 70:\t\n 71:\t#define EPAGERANGE(st)\t\t\t\t\t\t\t\\\n 72:\t\t[PFN_DOWN(CEA_ESTACK_OFFS(st)) ...\t\t\t\t\\\n 73:\t\t PFN_DOWN(CEA_ESTACK_OFFS(st) + EXCEPTION_STKSZ - 1)] = {\t\\\n 74:\t\t\t.offs\t= CEA_ESTACK_OFFS(st),\t\t\t\t\\\n 75:\t\t\t.size\t= EXCEPTION_STKSZ,\t\t\t\t\\\n 76:\t\t\t.type\t= STACK_TYPE_EXCEPTION + st, }\n 77:\t\n 78:\t/\n 79:\t * Array of exception stack page descriptors. If the stack is larger than\n 80:\t * PAGE_SIZE, all pages covering a particular stack will have the same\n 81:\t * info. The guard pages including the not mapped DB2 stack are zeroed\n 82:\t * out.\n 83:\t /\n 84:\tstatic const\n 85:\tstruct estack_pages estack_pages[CEA_ESTACK_PAGES] ____cacheline_aligned = {\n 86:\t\tEPAGERANGE(ESTACK_DF),\n 87:\t\tEPAGERANGE(ESTACK_NMI),\n 88:\t\tEPAGERANGE(ESTACK_DB),\n 89:\t\tEPAGERANGE(ESTACK_MCE),\n 90:\t\tEPAGERANGE(ESTACK_VC),\n 91:\t\tEPAGERANGE(ESTACK_VC2),\n 92:\t};\n 93:\t\n 94:\tstatic __always_inline bool in_exception_stack(unsigned long stack, struct stack_info info)\n 95:\t{\n 96:\t\tunsigned long begin, end, stk = (unsigned long)stack;\n 97:\t\tconst struct estack_pages ep;\n 98:\t\tstruct pt_regs regs;\n 99:\t\tunsigned int k;\n 100:\t\n 101:\t\tBUILD_BUG_ON(N_EXCEPTION_STACKS != 6);\n 102:\t\n 103:\t\tbegin = (unsigned long)__this_cpu_read(cea_exception_stacks);\n 104:\t\t/\n" }
11/2	2026/09/11 21:49	llm	patch-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 68524 output: 36 thoughts: 818 Thoughts: Analyzing KVM FRED Support: Reachability and Fuzzing Potential Okay, so I'm looking at this patch and my initial thought is: what's the deal with `estack_pages`? I see how the exception stack ordering changed, but my real concern is: Can this KVM FRED stuff even be fuzzed? That's the crucial question. Section 1 and Section 2 in that document were very clear: "REACHABILITY IS THE PRIMARY GATE." If the code isn't reachable in a standard environment like QEMU or GCE, then fuzzing is a waste of time, even with all the assertions in the world! First, let's establish: Can KVM FRED actually run in QEMU or GCE? Or, is any part of this patch reachable in a standard QEMU / GCE environment? Let's break this down further with a key question: What if the host CPU doesn't have FRED? Can the KVM code even execute in that scenario? Looking at `kvm_set_cr4` is a good starting point. I see the code checks for `X86_CR4_FRED` and `is_long_mode(vcpu)`. It looks like it could return 1 if `cr4 & X86_CR4_FRED` and we're not in long mode. Now, when is `kvm_set_cr4` called? During `KVM_SET_SREGS`, `KVM_SET_SREGS2` or instruction emulation. That's a lot of potential paths! But let's keep digging, I need to know the conditions. Okay, let's go deeper. `__kvm_is_valid_cr4(vcpu, cr4)` calls the `__cpu_has` macro. It looks like it checks `X86_FEATURE_FRED`. And if the host doesn't have FRED, that macro sets `__reserved_bits` to `X86_CR4_FRED`. I'm starting to see a picture. If the host CPU lacks FRED, and `X86_64_F(FRED)` is only set if the host supports it, is there any way a guest CPU could claim to have FRED via `KVM_SET_CPUID2`, even if the host doesn't? That's the real test of reachability. If a malicious guest can set the FRED bit, and the code path is taken then it could be a real problem. So, the critical question is: Does KVM allow userspace to set CPUID bits that the host doesn't support? Time to analyze how `guest_cpu_cap_has` works in KVM and find out!
12/2	2026/09/11 21:50	tool	grepper	0m Args: { "Expression": "guest_cpu_cap_has", "PathPrefix": "arch/x86/kvm/" } Results: { "Output": "\nFull output is too long, showing 500 out of 883 lines.\nUse more precise expression if possible.\n\narch/x86/kvm/cpuid.c=376=void kvm_vcpu_after_set_cpuid(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/cpuid.c-428-\tallow_gbpages = tdp_enabled ? boot_cpu_has(X86_FEATURE_GBPAGES) :\narch/x86/kvm/cpuid.c:429:\t\t\t\t guest_cpu_cap_has(vcpu, X86_FEATURE_GBPAGES);\narch/x86/kvm/cpuid.c-430-\tguest_cpu_cap_change(vcpu, X86_FEATURE_GBPAGES, allow_gbpages);\n--\narch/x86/kvm/cpuid.c-454-\tvcpu-\u003earch.cr4_guest_rsvd_bits = __cr4_reserved_bits(__kvm_cpu_cap_has, UNUSED_) \|\narch/x86/kvm/cpuid.c:455:\t\t\t\t\t __cr4_reserved_bits(guest_cpu_cap_has, vcpu);\narch/x86/kvm/cpuid.c-456-#undef __kvm_cpu_cap_has\n--\narch/x86/kvm/cpuid.h=258=static __always_inline void guest_cpu_cap_change(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/cpuid.h-267-\narch/x86/kvm/cpuid.h:268:static __always_inline bool guest_cpu_cap_has(struct kvm_vcpu vcpu,\narch/x86/kvm/cpuid.h-269-\t\t\t\t\t unsigned int x86_feature)\n--\narch/x86/kvm/cpuid.h=284=static inline bool kvm_vcpu_is_legal_cr3(struct kvm_vcpu vcpu, unsigned long cr3)\narch/x86/kvm/cpuid.h-285-{\narch/x86/kvm/cpuid.h:286:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_LAM))\narch/x86/kvm/cpuid.h-287-\t\tcr3 \u0026= ~(X86_CR3_LAM_U48 \| X86_CR3_LAM_U57);\n--\narch/x86/kvm/cpuid.h=292=static inline bool guest_has_spec_ctrl_msr(struct kvm_vcpu vcpu)\narch/x86/kvm/cpuid.h-293-{\narch/x86/kvm/cpuid.h:294:\treturn (guest_cpu_cap_has(vcpu, X86_FEATURE_SPEC_CTRL) \|\|\narch/x86/kvm/cpuid.h:295:\t\tguest_cpu_cap_has(vcpu, X86_FEATURE_AMD_STIBP) \|\|\narch/x86/kvm/cpuid.h:296:\t\tguest_cpu_cap_has(vcpu, X86_FEATURE_AMD_IBRS) \|\|\narch/x86/kvm/cpuid.h:297:\t\tguest_cpu_cap_has(vcpu, X86_FEATURE_AMD_SSBD));\narch/x86/kvm/cpuid.h-298-}\n--\narch/x86/kvm/cpuid.h=300=static inline bool guest_has_pred_cmd_msr(struct kvm_vcpu vcpu)\narch/x86/kvm/cpuid.h-301-{\narch/x86/kvm/cpuid.h:302:\treturn (guest_cpu_cap_has(vcpu, X86_FEATURE_SPEC_CTRL) \|\|\narch/x86/kvm/cpuid.h:303:\t\tguest_cpu_cap_has(vcpu, X86_FEATURE_AMD_IBPB) \|\|\narch/x86/kvm/cpuid.h:304:\t\tguest_cpu_cap_has(vcpu, X86_FEATURE_SBPB));\narch/x86/kvm/cpuid.h-305-}\n--\narch/x86/kvm/hyperv.c=1358=static void __kvm_hv_xsaves_xsavec_maybe_warn(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/hyperv.c-1377-\tif (guest_cpuid_has(vcpu, X86_FEATURE_XSAVES) \|\|\narch/x86/kvm/hyperv.c:1378:\t !guest_cpu_cap_has(vcpu, X86_FEATURE_XSAVEC))\narch/x86/kvm/hyperv.c-1379-\t\treturn;\n--\narch/x86/kvm/lapic.c=614=void kvm_apic_set_version(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/lapic.c-624-\narch/x86/kvm/lapic.c:625:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_X2APIC) \u0026\u0026\narch/x86/kvm/lapic.c-626-\t kvm_lapic_advertise_suppress_eoi_broadcast(vcpu-\u003ekvm))\n--\narch/x86/kvm/lapic.c=2849=int kvm_apic_set_base(struct kvm_vcpu vcpu, u64 value, bool host_initiated)\n--\narch/x86/kvm/lapic.c-2857-\tu64 reserved_bits = kvm_vcpu_reserved_gpa_bits_raw(vcpu) \| 0x2ff \|\narch/x86/kvm/lapic.c:2858:\t\t(guest_cpu_cap_has(vcpu, X86_FEATURE_X2APIC) ? 0 : X2APIC_ENABLE);\narch/x86/kvm/lapic.c-2859-\n--\narch/x86/kvm/mmu.h=229=static inline unsigned long kvm_get_active_cr3_lam_bits(struct kvm_vcpu vcpu)\narch/x86/kvm/mmu.h-230-{\narch/x86/kvm/mmu.h:231:\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_LAM))\narch/x86/kvm/mmu.h-232-\t\treturn 0;\n--\narch/x86/kvm/mmu/mmu.c=5513=static void reset_guest_rsvds_bits_mask(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/mmu/mmu.c-5518-\t\t\t\tw-\u003ecpu_role.base.level, is_efer_nx(w),\narch/x86/kvm/mmu/mmu.c:5519:\t\t\t\tguest_cpu_cap_has(vcpu, X86_FEATURE_GBPAGES),\narch/x86/kvm/mmu/mmu.c-5520-\t\t\t\tis_cr4_pse(w),\n--\narch/x86/kvm/mmu/mmu.c=5580=static void reset_shadow_zero_bits_mask(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/mmu/mmu.c-5595-\t\t\t\tcontext-\u003eroot_role.efer_nx,\narch/x86/kvm/mmu/mmu.c:5596:\t\t\t\tguest_cpu_cap_has(vcpu, X86_FEATURE_GBPAGES),\narch/x86/kvm/mmu/mmu.c-5597-\t\t\t\tis_pse, is_amd);\n--\narch/x86/kvm/msrs.c=587=static bool __kvm_valid_efer(struct kvm_vcpu vcpu, u64 efer)\narch/x86/kvm/msrs.c-588-{\narch/x86/kvm/msrs.c:589:\tif (efer \u0026 EFER_AUTOIBRS \u0026\u0026 !guest_cpu_cap_has(vcpu, X86_FEATURE_AUTOIBRS))\narch/x86/kvm/msrs.c-590-\t\treturn false;\narch/x86/kvm/msrs.c-591-\narch/x86/kvm/msrs.c:592:\tif (efer \u0026 EFER_FFXSR \u0026\u0026 !guest_cpu_cap_has(vcpu, X86_FEATURE_FXSR_OPT))\narch/x86/kvm/msrs.c-593-\t\treturn false;\narch/x86/kvm/msrs.c-594-\narch/x86/kvm/msrs.c:595:\tif (efer \u0026 EFER_SVME \u0026\u0026 !guest_cpu_cap_has(vcpu, X86_FEATURE_SVM))\narch/x86/kvm/msrs.c-596-\t\treturn false;\n--\narch/x86/kvm/msrs.c-598-\tif (efer \u0026 (EFER_LME \| EFER_LMA) \u0026\u0026\narch/x86/kvm/msrs.c:599:\t !guest_cpu_cap_has(vcpu, X86_FEATURE_LM))\narch/x86/kvm/msrs.c-600-\t\treturn false;\narch/x86/kvm/msrs.c-601-\narch/x86/kvm/msrs.c:602:\tif (efer \u0026 EFER_NX \u0026\u0026 !guest_cpu_cap_has(vcpu, X86_FEATURE_NX))\narch/x86/kvm/msrs.c-603-\t\treturn false;\n--\narch/x86/kvm/msrs.c=703=static int __kvm_set_msr(struct kvm_vcpu vcpu, u32 index, u64 data,\n--\narch/x86/kvm/msrs.c-737-\t\tif (!host_initiated \u0026\u0026\narch/x86/kvm/msrs.c:738:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_RDTSCP) \u0026\u0026\narch/x86/kvm/msrs.c:739:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_RDPID))\narch/x86/kvm/msrs.c-740-\t\t\treturn 1;\n--\narch/x86/kvm/msrs.c-757-\tcase MSR_IA32_S_CET:\narch/x86/kvm/msrs.c:758:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) \u0026\u0026\narch/x86/kvm/msrs.c:759:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_IBT))\narch/x86/kvm/msrs.c-760-\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n--\narch/x86/kvm/msrs.c-779-\tcase MSR_IA32_PL1_SSP ... MSR_IA32_INT_SSP_TAB:\narch/x86/kvm/msrs.c:780:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK))\narch/x86/kvm/msrs.c-781-\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n--\narch/x86/kvm/msrs.c-785-\t\t /\narch/x86/kvm/msrs.c:786:\t\tif (index == MSR_IA32_INT_SSP_TAB \u0026\u0026 !guest_cpu_cap_has(vcpu, X86_FEATURE_LM))\narch/x86/kvm/msrs.c-787-\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n--\narch/x86/kvm/msrs.c-794-\tcase MSR_IA32_FRED_STKLVLS:\narch/x86/kvm/msrs.c:795:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))\narch/x86/kvm/msrs.c-796-\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n--\narch/x86/kvm/msrs.c-801-\narch/x86/kvm/msrs.c:802:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))\narch/x86/kvm/msrs.c-803-\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n--\narch/x86/kvm/msrs.c-828-\tcase MSR_IA32_PL0_SSP: / I.e., MSR_IA32_FRED_SSP0 /\narch/x86/kvm/msrs.c:829:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) \u0026\u0026\narch/x86/kvm/msrs.c:830:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))\narch/x86/kvm/msrs.c-831-\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n--\narch/x86/kvm/msrs.c=868=static int __kvm_get_msr(struct kvm_vcpu vcpu, u32 index, u64 data,\n--\narch/x86/kvm/msrs.c-879-\t\tif (!host_initiated \u0026\u0026\narch/x86/kvm/msrs.c:880:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_RDTSCP) \u0026\u0026\narch/x86/kvm/msrs.c:881:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_RDPID))\narch/x86/kvm/msrs.c-882-\t\t\treturn 1;\n--\narch/x86/kvm/msrs.c-885-\tcase MSR_IA32_S_CET:\narch/x86/kvm/msrs.c:886:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) \u0026\u0026\narch/x86/kvm/msrs.c:887:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_IBT))\narch/x86/kvm/msrs.c-888-\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n--\narch/x86/kvm/msrs.c-894-\tcase MSR_IA32_PL1_SSP ... MSR_IA32_INT_SSP_TAB:\narch/x86/kvm/msrs.c:895:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK))\narch/x86/kvm/msrs.c-896-\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n--\narch/x86/kvm/msrs.c-898-\tcase MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_CONFIG:\narch/x86/kvm/msrs.c:899:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))\narch/x86/kvm/msrs.c-900-\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n--\narch/x86/kvm/msrs.c-902-\tcase MSR_IA32_PL0_SSP: / I.e., MSR_IA32_FRED_SSP0 /\narch/x86/kvm/msrs.c:903:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) \u0026\u0026\narch/x86/kvm/msrs.c:904:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_FRED))\narch/x86/kvm/msrs.c-905-\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n--\narch/x86/kvm/msrs.c=1229=static bool is_xstate_managed_msr(struct kvm_vcpu vcpu, u32 msr)\n--\narch/x86/kvm/msrs.c-1235-\tcase MSR_IA32_U_CET:\narch/x86/kvm/msrs.c:1236:\t\treturn guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) \|\|\narch/x86/kvm/msrs.c:1237:\t\t guest_cpu_cap_has(vcpu, X86_FEATURE_IBT);\narch/x86/kvm/msrs.c-1238-\tcase MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:\narch/x86/kvm/msrs.c:1239:\t\treturn guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK);\narch/x86/kvm/msrs.c-1240-\tdefault:\n--\narch/x86/kvm/msrs.c=1501=int kvm_set_msr_common(struct kvm_vcpu vcpu, struct msr_data msr_info)\n--\narch/x86/kvm/msrs.c-1531-\t\tif (!msr_info-\u003ehost_initiated \|\|\narch/x86/kvm/msrs.c:1532:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_ARCH_CAPABILITIES))\narch/x86/kvm/msrs.c-1533-\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n--\narch/x86/kvm/msrs.c-1537-\t\tif (!msr_info-\u003ehost_initiated \|\|\narch/x86/kvm/msrs.c:1538:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_PDCM))\narch/x86/kvm/msrs.c-1539-\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n--\narch/x86/kvm/msrs.c-1562-\narch/x86/kvm/msrs.c:1563:\t\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SPEC_CTRL) \u0026\u0026\narch/x86/kvm/msrs.c:1564:\t\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_AMD_IBPB))\narch/x86/kvm/msrs.c-1565-\t\t\t\treserved_bits \|= PRED_CMD_IBPB;\narch/x86/kvm/msrs.c-1566-\narch/x86/kvm/msrs.c:1567:\t\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SBPB))\narch/x86/kvm/msrs.c-1568-\t\t\t\treserved_bits \|= PRED_CMD_SBPB;\n--\narch/x86/kvm/msrs.c-1587-\t\tif (!msr_info-\u003ehost_initiated \u0026\u0026\narch/x86/kvm/msrs.c:1588:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_FLUSH_L1D))\narch/x86/kvm/msrs.c-1589-\t\t\treturn 1;\n--\narch/x86/kvm/msrs.c-1611-\narch/x86/kvm/msrs.c:1612:\t\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_GP_ON_USER_CPUID))\narch/x86/kvm/msrs.c-1613-\t\t\tvalid \|= MSR_K7_HWCR_CPUID_USER_DIS;\n--\narch/x86/kvm/msrs.c-1644-\tcase MSR_IA32_TSC_ADJUST:\narch/x86/kvm/msrs.c:1645:\t\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_TSC_ADJUST)) {\narch/x86/kvm/msrs.c-1646-\t\t\tif (!msr_info-\u003ehost_initiated) {\n--\narch/x86/kvm/msrs.c-1671-\t\t ((old_val ^ data) \u0026 MSR_IA32_MISC_ENABLE_MWAIT)) {\narch/x86/kvm/msrs.c:1672:\t\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_XMM3))\narch/x86/kvm/msrs.c-1673-\t\t\t\treturn 1;\n--\narch/x86/kvm/msrs.c-1850-\tcase MSR_AMD64_OSVW_ID_LENGTH:\narch/x86/kvm/msrs.c:1851:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_OSVW))\narch/x86/kvm/msrs.c-1852-\t\t\treturn 1;\n--\narch/x86/kvm/msrs.c-1855-\tcase MSR_AMD64_OSVW_STATUS:\narch/x86/kvm/msrs.c:1856:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_OSVW))\narch/x86/kvm/msrs.c-1857-\t\t\treturn 1;\n--\narch/x86/kvm/msrs.c-1874-\t\tif (!msr_info-\u003ehost_initiated \u0026\u0026\narch/x86/kvm/msrs.c:1875:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_XFD))\narch/x86/kvm/msrs.c-1876-\t\t\treturn 1;\n--\narch/x86/kvm/msrs.c-1884-\t\tif (!msr_info-\u003ehost_initiated \u0026\u0026\narch/x86/kvm/msrs.c:1885:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_XFD))\narch/x86/kvm/msrs.c-1886-\t\t\treturn 1;\n--\narch/x86/kvm/msrs.c-1897-\tcase MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:\narch/x86/kvm/msrs.c:1898:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) {\narch/x86/kvm/msrs.c-1899-\t\t\tWARN_ON_ONCE(msr != MSR_IA32_FRED_SSP0);\n--\narch/x86/kvm/msrs.c=1966=int kvm_get_msr_common(struct kvm_vcpu vcpu, struct msr_data msr_info)\n--\narch/x86/kvm/msrs.c-2011-\tcase MSR_IA32_ARCH_CAPABILITIES:\narch/x86/kvm/msrs.c:2012:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_ARCH_CAPABILITIES))\narch/x86/kvm/msrs.c-2013-\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n--\narch/x86/kvm/msrs.c-2016-\tcase MSR_IA32_PERF_CAPABILITIES:\narch/x86/kvm/msrs.c:2017:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_PDCM))\narch/x86/kvm/msrs.c-2018-\t\t\treturn KVM_MSR_RET_UNSUPPORTED;\n--\narch/x86/kvm/msrs.c-2217-\tcase MSR_AMD64_OSVW_ID_LENGTH:\narch/x86/kvm/msrs.c:2218:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_OSVW))\narch/x86/kvm/msrs.c-2219-\t\t\treturn 1;\n--\narch/x86/kvm/msrs.c-2222-\tcase MSR_AMD64_OSVW_STATUS:\narch/x86/kvm/msrs.c:2223:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_OSVW))\narch/x86/kvm/msrs.c-2224-\t\t\treturn 1;\n--\narch/x86/kvm/msrs.c-2241-\t\tif (!msr_info-\u003ehost_initiated \u0026\u0026\narch/x86/kvm/msrs.c:2242:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_XFD))\narch/x86/kvm/msrs.c-2243-\t\t\treturn 1;\n--\narch/x86/kvm/msrs.c-2248-\t\tif (!msr_info-\u003ehost_initiated \u0026\u0026\narch/x86/kvm/msrs.c:2249:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_XFD))\narch/x86/kvm/msrs.c-2250-\t\t\treturn 1;\n--\narch/x86/kvm/msrs.c-2258-\tcase MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP:\narch/x86/kvm/msrs.c:2259:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) {\narch/x86/kvm/msrs.c-2260-\t\t\tWARN_ON_ONCE(msr_info-\u003eindex != MSR_IA32_FRED_SSP0);\n--\narch/x86/kvm/msrs.c=2428=static int kvm_translate_kvm_reg(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/msrs.c-2438-\t\t /\narch/x86/kvm/msrs.c:2439:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK))\narch/x86/kvm/msrs.c-2440-\t\t\treturn -EINVAL;\n--\narch/x86/kvm/msrs.c=2499=int kvm_get_reg_list(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/msrs.c-2501-{\narch/x86/kvm/msrs.c:2502:\tu64 nr_regs = guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) ? 1 : 0;\narch/x86/kvm/msrs.c-2503-\tu64 user_nr_regs;\n--\narch/x86/kvm/msrs.h=136=static inline bool kvm_is_valid_u_s_cet(struct kvm_vcpu vcpu, u64 data)\n--\narch/x86/kvm/msrs.h-139-\t\treturn false;\narch/x86/kvm/msrs.h:140:\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) \u0026\u0026\narch/x86/kvm/msrs.h-141-\t (data \u0026 CET_US_SHSTK_MASK_BITS))\narch/x86/kvm/msrs.h-142-\t\treturn false;\narch/x86/kvm/msrs.h:143:\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_IBT) \u0026\u0026\narch/x86/kvm/msrs.h-144-\t (data \u0026 CET_US_IBT_MASK_BITS))\n--\narch/x86/kvm/regs.c=776=static u64 kvm_dr6_fixed(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/regs.c-779-\narch/x86/kvm/regs.c:780:\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_RTM))\narch/x86/kvm/regs.c-781-\t\tfixed \|= DR6_RTM;\narch/x86/kvm/regs.c-782-\narch/x86/kvm/regs.c:783:\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_BUS_LOCK_DETECT))\narch/x86/kvm/regs.c-784-\t\tfixed \|= DR6_BUS_LOCK;\n--\narch/x86/kvm/smm.c=228=static void enter_smm_save_state_64(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/smm.c-272-\narch/x86/kvm/smm.c:273:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) \u0026\u0026\narch/x86/kvm/smm.c-274-\t kvm_msr_read(vcpu, MSR_KVM_INTERNAL_GUEST_SSP, \u0026smram-\u003essp))\n--\narch/x86/kvm/smm.c=279=void enter_smm(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/smm.c-290-#ifdef CONFIG_X86_64\narch/x86/kvm/smm.c:291:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_LM))\narch/x86/kvm/smm.c-292-\t\tenter_smm_save_state_64(vcpu, \u0026smram.smram64);\n--\narch/x86/kvm/smm.c-360-#ifdef CONFIG_X86_64\narch/x86/kvm/smm.c:361:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_LM))\narch/x86/kvm/smm.c-362-\t\tif (kvm_x86_call(set_efer)(vcpu, 0))\n--\narch/x86/kvm/smm.c=516=static int rsm_load_state_64(struct x86_emulate_ctxt ctxt,\n--\narch/x86/kvm/smm.c-564-\narch/x86/kvm/smm.c:565:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) \u0026\u0026\narch/x86/kvm/smm.c-566-\t kvm_msr_write(vcpu, MSR_KVM_INTERNAL_GUEST_SSP, smstate-\u003essp))\n--\narch/x86/kvm/smm.c=573=int emulator_leave_smm(struct x86_emulate_ctxt ctxt)\n--\narch/x86/kvm/smm.c-597-#ifdef CONFIG_X86_64\narch/x86/kvm/smm.c:598:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_LM)) {\narch/x86/kvm/smm.c-599-\t\tstruct kvm_segment cs_desc;\n--\narch/x86/kvm/smm.c-624-#ifdef CONFIG_X86_64\narch/x86/kvm/smm.c:625:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_LM)) {\narch/x86/kvm/smm.c-626-\t\tunsigned long cr4, efer;\n--\narch/x86/kvm/smm.c-649-#ifdef CONFIG_X86_64\narch/x86/kvm/smm.c:650:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_LM))\narch/x86/kvm/smm.c-651-\t\tret = rsm_load_state_64(ctxt, \u0026smram.smram64);\n--\narch/x86/kvm/svm/nested.c=127=static bool nested_vmcb_needs_vls_intercept(struct vcpu_svm svm)\narch/x86/kvm/svm/nested.c-128-{\narch/x86/kvm/svm/nested.c:129:\tif (!guest_cpu_cap_has(\u0026svm-\u003evcpu, X86_FEATURE_V_VMSAVE_VMLOAD))\narch/x86/kvm/svm/nested.c-130-\t\treturn true;\n--\narch/x86/kvm/svm/nested.c=358=static bool nested_svm_event_inj_valid_exept(struct kvm_vcpu vcpu, u8 vector)\n--\narch/x86/kvm/svm/nested.c-373-\tcase CP_VECTOR:\narch/x86/kvm/svm/nested.c:374:\t\treturn guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK);\narch/x86/kvm/svm/nested.c-375-\tcase VC_VECTOR:\narch/x86/kvm/svm/nested.c:376:\t\treturn guest_cpu_cap_has(vcpu, X86_FEATURE_SEV_ES);\narch/x86/kvm/svm/nested.c-377-\t}\n--\narch/x86/kvm/svm/nested.c=504=do {\t\t\t\t\t\t\t\t\t\t\\\narch/x86/kvm/svm/nested.c:505:\tif (!guest_cpu_cap_has(__vcpu, X86_FEATURE_##fname))\t\t\t\\\narch/x86/kvm/svm/nested.c-506-\t\tvmcb12_clr_intercept(__control, INTERCEPT_##iname);\t\t\\\n--\narch/x86/kvm/svm/nested.c=513=void __nested_copy_vmcb_control_to_cache(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/svm/nested.c-529-\tto-\u003emisc_ctl = from-\u003emisc_ctl;\narch/x86/kvm/svm/nested.c:530:\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_NPT))\narch/x86/kvm/svm/nested.c-531-\t\tto-\u003emisc_ctl \u0026= ~SVM_MISC_ENABLE_NP;\narch/x86/kvm/svm/nested.c-532-\narch/x86/kvm/svm/nested.c:533:\tif (!gmet_enabled \|\| !guest_cpu_cap_has(vcpu, X86_FEATURE_GMET))\narch/x86/kvm/svm/nested.c-534-\t\tto-\u003emisc_ctl \u0026= ~SVM_MISC_ENABLE_GMET;\n--\narch/x86/kvm/svm/nested.c=738=static bool nested_vmcb12_has_lbrv(struct kvm_vcpu vcpu)\narch/x86/kvm/svm/nested.c-739-{\narch/x86/kvm/svm/nested.c:740:\treturn guest_cpu_cap_has(vcpu, X86_FEATURE_LBRV) \u0026\u0026\narch/x86/kvm/svm/nested.c-741-\t\t(to_svm(vcpu)-\u003enested.ctl.misc_ctl2 \u0026 SVM_MISC2_ENABLE_V_LBR);\n--\narch/x86/kvm/svm/nested.c=744=static void nested_vmcb02_prepare_save(struct vcpu_svm svm)\n--\narch/x86/kvm/svm/nested.c-774-\narch/x86/kvm/svm/nested.c:775:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) \u0026\u0026\narch/x86/kvm/svm/nested.c-776-\t (unlikely(new_vmcb12 \|\| vmcb12_is_dirty(control, VMCB_CET)))) {\n--\narch/x86/kvm/svm/nested.c=852=static void nested_vmcb02_prepare_control(struct vcpu_svm svm)\n--\narch/x86/kvm/svm/nested.c-872-\narch/x86/kvm/svm/nested.c:873:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_VGIF) \u0026\u0026\narch/x86/kvm/svm/nested.c-874-\t (vmcb12_ctrl-\u003eint_ctl \u0026 V_GIF_ENABLE_MASK))\n--\narch/x86/kvm/svm/nested.c-949-\narch/x86/kvm/svm/nested.c:950:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_TSCRATEMSR) \u0026\u0026\narch/x86/kvm/svm/nested.c-951-\t svm-\u003etsc_ratio_msr != kvm_caps.default_tsc_scaling_ratio)\n--\narch/x86/kvm/svm/nested.c-969-\t /\narch/x86/kvm/svm/nested.c:970:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_NRIPS) \|\|\narch/x86/kvm/svm/nested.c-971-\t !vcpu-\u003earch.nested_run_pending)\n--\narch/x86/kvm/svm/nested.c-981-\t\tsvm-\u003esoft_int_injected = true;\narch/x86/kvm/svm/nested.c:982:\t\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_NRIPS) \|\|\narch/x86/kvm/svm/nested.c-983-\t\t !vcpu-\u003earch.nested_run_pending)\n--\narch/x86/kvm/svm/nested.c-991-\narch/x86/kvm/svm/nested.c:992:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_PAUSEFILTER))\narch/x86/kvm/svm/nested.c-993-\t\tvmcb02-\u003econtrol.pause_filter_count = vmcb12_ctrl-\u003epause_filter_count;\n--\narch/x86/kvm/svm/nested.c-995-\t\tvmcb02-\u003econtrol.pause_filter_count = 0;\narch/x86/kvm/svm/nested.c:996:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_PFTHRESHOLD))\narch/x86/kvm/svm/nested.c-997-\t\tvmcb02-\u003econtrol.pause_filter_thresh = vmcb12_ctrl-\u003epause_filter_thresh;\n--\narch/x86/kvm/svm/nested.c-1008-\t /\narch/x86/kvm/svm/nested.c:1009:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS))\narch/x86/kvm/svm/nested.c-1010-\t\tvmcb02-\u003econtrol.erap_ctl = (vmcb12_ctrl-\u003eerap_ctl \u0026\n--\narch/x86/kvm/svm/nested.c=1249=static int nested_svm_vmexit_update_vmcb12(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/svm/nested.c-1282-\narch/x86/kvm/svm/nested.c:1283:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) {\narch/x86/kvm/svm/nested.c-1284-\t\tvmcb12-\u003esave.s_cet\t= vmcb02-\u003esave.s_cet;\n--\narch/x86/kvm/svm/nested.c-1296-\narch/x86/kvm/svm/nested.c:1297:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_NRIPS))\narch/x86/kvm/svm/nested.c-1298-\t\tvmcb12-\u003econtrol.next_rip = vmcb02-\u003econtrol.next_rip;\n--\narch/x86/kvm/svm/nested.c=1317=void nested_svm_vmexit(struct vcpu_svm svm)\n--\narch/x86/kvm/svm/nested.c-1349-\narch/x86/kvm/svm/nested.c:1350:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS))\narch/x86/kvm/svm/nested.c-1351-\t\tvmcb01-\u003econtrol.erap_ctl \|= ERAP_CONTROL_CLEAR_RAP;\n--\narch/x86/kvm/svm/pmu.c=38=static inline struct kvm_pmc get_gp_pmc_amd(struct kvm_pmu pmu, u32 msr,\n--\narch/x86/kvm/svm/pmu.c-48-\tcase MSR_F15H_PERF_CTL0 ... MSR_F15H_PERF_CTR5:\narch/x86/kvm/svm/pmu.c:49:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE))\narch/x86/kvm/svm/pmu.c-50-\t\t\treturn NULL;\n--\narch/x86/kvm/svm/pmu.c=104=static bool amd_is_valid_msr(struct kvm_vcpu vcpu, u32 msr)\n--\narch/x86/kvm/svm/pmu.c-111-\tcase MSR_F15H_PERF_CTL0 ... MSR_F15H_PERF_CTR5:\narch/x86/kvm/svm/pmu.c:112:\t\treturn guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE);\narch/x86/kvm/svm/pmu.c-113-\tcase MSR_AMD64_PERF_CNTR_GLOBAL_STATUS:\n--\narch/x86/kvm/svm/pmu.c=185=static void amd_pmu_refresh(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/svm/pmu.c-190-\tpmu-\u003eversion = 1;\narch/x86/kvm/svm/pmu.c:191:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_PERFMON_V2)) {\narch/x86/kvm/svm/pmu.c-192-\t\tpmu-\u003eversion = 2;\n--\narch/x86/kvm/svm/pmu.c-200-\t\tpmu-\u003enr_arch_gp_counters = ebx.split.num_core_pmc;\narch/x86/kvm/svm/pmu.c:201:\t} else if (guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE)) {\narch/x86/kvm/svm/pmu.c-202-\t\tpmu-\u003enr_arch_gp_counters = AMD64_NUM_COUNTERS_CORE;\n--\narch/x86/kvm/svm/pmu.c-217-\tpmu-\u003ereserved_bits = 0xfffffff000280000ull;\narch/x86/kvm/svm/pmu.c:218:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_SVM) \u0026\u0026 kvm_vcpu_has_mediated_pmu(vcpu))\narch/x86/kvm/svm/pmu.c-219-\t\tpmu-\u003ereserved_bits \u0026= ~AMD64_EVENTSEL_HOST_GUEST_MASK;\n--\narch/x86/kvm/svm/sev.c=4727=void sev_es_recalc_msr_intercepts(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/svm/sev.c-4735-\t\tsvm_set_intercept_for_msr(vcpu, MSR_TSC_AUX, MSR_TYPE_RW,\narch/x86/kvm/svm/sev.c:4736:\t\t\t\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_RDTSCP) \u0026\u0026\narch/x86/kvm/svm/sev.c:4737:\t\t\t\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_RDPID));\narch/x86/kvm/svm/sev.c-4738-\n--\narch/x86/kvm/svm/sev.c-4754-\tsvm_set_intercept_for_msr(vcpu, MSR_IA32_XSS, MSR_TYPE_RW,\narch/x86/kvm/svm/sev.c:4755:\t\t\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_XSAVES) \|\|\narch/x86/kvm/svm/sev.c-4756-\t\t\t\t !guest_cpuid_has(vcpu, X86_FEATURE_XSAVES));\n--\narch/x86/kvm/svm/svm.c=757=static void svm_recalc_pmu_msr_intercepts(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/svm/svm.c-770-\narch/x86/kvm/svm/svm.c:771:\tintercept \|= !guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE);\narch/x86/kvm/svm/svm.c-772-\tfor (i = 0; i \u003c pmu-\u003enr_arch_gp_counters; i++)\n--\narch/x86/kvm/svm/svm.c=791=static void svm_recalc_msr_intercepts(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/svm/svm.c-815-\t\tsvm_set_intercept_for_msr(vcpu, MSR_IA32_FLUSH_CMD, MSR_TYPE_W,\narch/x86/kvm/svm/svm.c:816:\t\t\t\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_FLUSH_L1D));\narch/x86/kvm/svm/svm.c-817-\n--\narch/x86/kvm/svm/svm.c-845-\tif (kvm_cpu_cap_has(X86_FEATURE_SHSTK)) {\narch/x86/kvm/svm/svm.c:846:\t\tbool shstk_enabled = guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK);\narch/x86/kvm/svm/svm.c-847-\n--\narch/x86/kvm/svm/svm.c=884=void svm_update_lbrv(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/svm/svm.c-888-\tbool enable_lbrv = (svm-\u003evmcb-\u003esave.dbgctl \u0026 DEBUGCTLMSR_LBR) \|\|\narch/x86/kvm/svm/svm.c:889:\t\t\t (is_guest_mode(vcpu) \u0026\u0026 guest_cpu_cap_has(vcpu, X86_FEATURE_LBRV) \u0026\u0026\narch/x86/kvm/svm/svm.c-890-\t\t\t (svm-\u003enested.ctl.misc_ctl2 \u0026 SVM_MISC2_ENABLE_V_LBR));\n--\narch/x86/kvm/svm/svm.c=1044=static void svm_recalc_instruction_intercepts(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/svm/svm.c-1053-\t\tif (!npt_enabled \|\|\narch/x86/kvm/svm/svm.c:1054:\t\t !guest_cpu_cap_has(\u0026svm-\u003evcpu, X86_FEATURE_INVPCID))\narch/x86/kvm/svm/svm.c-1055-\t\t\tsvm_set_intercept(svm, INTERCEPT_INVPCID);\n--\narch/x86/kvm/svm/svm.c-1060-\tif (kvm_cpu_cap_has(X86_FEATURE_RDTSCP)) {\narch/x86/kvm/svm/svm.c:1061:\t\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_RDTSCP))\narch/x86/kvm/svm/svm.c-1062-\t\t\tsvm_clr_intercept(svm, INTERCEPT_RDTSCP);\n--\narch/x86/kvm/svm/svm.c=1126=static void init_vmcb(struct kvm_vcpu vcpu, bool init_event)\n--\narch/x86/kvm/svm/svm.c-1251-\narch/x86/kvm/svm/svm.c:1252:\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS))\narch/x86/kvm/svm/svm.c-1253-\t\tsvm-\u003evmcb-\u003econtrol.erap_ctl \|= ERAP_CONTROL_ALLOW_LARGER_RAP;\n--\narch/x86/kvm/svm/svm.c=2827=static int svm_get_msr(struct kvm_vcpu vcpu, struct msr_data msr_info)\n--\narch/x86/kvm/svm/svm.c-2838-\t\tif (!msr_info-\u003ehost_initiated \u0026\u0026\narch/x86/kvm/svm/svm.c:2839:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_TSCRATEMSR))\narch/x86/kvm/svm/svm.c-2840-\t\t\treturn 1;\n--\narch/x86/kvm/svm/svm.c-2917-\t\tif (!msr_info-\u003ehost_initiated \u0026\u0026\narch/x86/kvm/svm/svm.c:2918:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_VIRT_SSBD))\narch/x86/kvm/svm/svm.c-2919-\t\t\treturn 1;\n--\narch/x86/kvm/svm/svm.c=2990=static int svm_set_msr(struct kvm_vcpu vcpu, struct msr_data msr)\n--\narch/x86/kvm/svm/svm.c-3003-\narch/x86/kvm/svm/svm.c:3004:\t\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_TSCRATEMSR)) {\narch/x86/kvm/svm/svm.c-3005-\n--\narch/x86/kvm/svm/svm.c-3025-\narch/x86/kvm/svm/svm.c:3026:\t\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_TSCRATEMSR) \u0026\u0026\narch/x86/kvm/svm/svm.c-3027-\t\t is_guest_mode(vcpu))\n--\narch/x86/kvm/svm/svm.c-3079-\t\tif (!msr-\u003ehost_initiated \u0026\u0026\narch/x86/kvm/svm/svm.c:3080:\t\t !guest_cpu_cap_has(vcpu, X86_FEATURE_VIRT_SSBD))\narch/x86/kvm/svm/svm.c-3081-\t\t\treturn 1;\n--\narch/x86/kvm/svm/svm.c=3267=static int invpcid_interception(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/svm/svm.c-3272-\narch/x86/kvm/svm/svm.c:3273:\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_INVPCID)) {\narch/x86/kvm/svm/svm.c-3274-\t\tkvm_queue_exception(vcpu, UD_VECTOR);\n--\narch/x86/kvm/svm/svm.c=3752=static void svm_set_nested_run_soft_int_state(struct kvm_vcpu *vcpu)\n--\narch/x86/kvm/svm/svm.c-3757-\tsvm-\u003esoft_int_old_rip = kvm_rip_read(vcpu);\narch/x86/kvm/svm/svm.c:3758:\tif (!guest_cpu_cap_has(vcpu, X86_FEATURE_NRIPS))\narch/x86/kvm/svm/svm.c-3759-\t\tsvm-\u003esoft_int_next_rip = kvm_rip_read(vcpu);\n--\n\n" }
13/2	2026/09/11 21:50	llm	patch-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 84159 output: 41 thoughts: 44 Thoughts: Okay, let's dive into this. I need to understand how `guest_cpu_cap_has` is defined. Where would I find that? Ah, yes, it's a KVM-related definition, so it's most likely in the architecture-specific headers under `arch/x86`. More specifically, since it's related to CPUID features, my best bet is `arch/x86/kvm/cpuid.h`. Now, I'll need to pinpoint its exact location in that file; around line 268 should be a good starting point. Let's get to it.
14/2	2026/09/11 21:50	tool	read-file	0m Args: { "File": "arch/x86/kvm/cpuid.h", "FirstLine": 255, "LineCount": 35 } Results: { "Contents": " 255:\t\tvcpu-\u003earch.cpu_caps[x86_leaf] \u0026= ~__feature_bit(x86_feature);\n 256:\t}\n 257:\t\n 258:\tstatic __always_inline void guest_cpu_cap_change(struct kvm_vcpu vcpu,\n 259:\t\t\t\t\t\t\t unsigned int x86_feature,\n 260:\t\t\t\t\t\t\t bool guest_has_cap)\n 261:\t{\n 262:\t\tif (guest_has_cap)\n 263:\t\t\tguest_cpu_cap_set(vcpu, x86_feature);\n 264:\t\telse\n 265:\t\t\tguest_cpu_cap_clear(vcpu, x86_feature);\n 266:\t}\n 267:\t\n 268:\tstatic __always_inline bool guest_cpu_cap_has(struct kvm_vcpu vcpu,\n 269:\t\t\t\t\t\t unsigned int x86_feature)\n 270:\t{\n 271:\t\tunsigned int x86_leaf = __feature_leaf(x86_feature);\n 272:\t\n 273:\t\t/\n 274:\t\t Except for MWAIT, querying dynamic feature bits is disallowed, so\n 275:\t\t * that KVM can defer runtime updates until the next CPUID emulation.\n 276:\t\t /\n 277:\t\tBUILD_BUG_ON(x86_feature == X86_FEATURE_APIC \|\|\n 278:\t\t\t x86_feature == X86_FEATURE_OSXSAVE \|\|\n 279:\t\t\t x86_feature == X86_FEATURE_OSPKE);\n 280:\t\n 281:\t\treturn vcpu-\u003earch.cpu_caps[x86_leaf] \u0026 __feature_bit(x86_feature);\n 282:\t}\n 283:\t\n 284:\tstatic inline bool kvm_vcpu_is_legal_cr3(struct kvm_vcpu vcpu, unsigned long cr3)\n 285:\t{\n 286:\t\tif (guest_cpu_cap_has(vcpu, X86_FEATURE_LAM))\n 287:\t\t\tcr3 \u0026= ~(X86_CR3_LAM_U48 \| X86_CR3_LAM_U57);\n 288:\t\n 289:\t\treturn kvm_vcpu_is_legal_gpa(vcpu, cr3);\n" }
15/2	2026/09/11 21:50	llm	patch-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 85019 output: 34 thoughts: 63 Thoughts: Here's my thought process, as if I were considering this code: Diving into CPUID Capabilities Population Okay, so I need to track down where `vcpu->arch.cpu_caps` gets populated. That's the key to understanding how these CPU capabilities are being set within the KVM x86 environment. My first instinct, given it's architecture-specific, is to zero in on the x86 code. Let's start by looking into `arch/x86/kvm/cpuid.c`. Specifically, I'll be checking around the functions that seem most likely to be involved in CPUID handling: `kvm_set_cpuid` and `kvm_vcpu_after_set_cpuid`. These functions are likely the entry points for setting up the CPUID information which might involve initializing the `vcpu->arch.cpu_caps` structure, or calling functions that do. I'll need to examine the surrounding code in those functions, and trace any calls related to updating or initializing CPUID data.
16/2	2026/09/11 21:50	tool	grepper	0m Args: { "Expression": "arch\\.cpu_caps", "PathPrefix": "arch/x86/kvm/" } Results: { "Output": "arch/x86/kvm/cpuid.c=376=void kvm_vcpu_after_set_cpuid(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/cpuid.c-383-\narch/x86/kvm/cpuid.c:384:\tmemset(vcpu-\u003earch.cpu_caps, 0, sizeof(vcpu-\u003earch.cpu_caps));\narch/x86/kvm/cpuid.c-385-\tBUILD_BUG_ON(ARRAY_SIZE(reverse_cpuid) != NR_KVM_CPU_CAPS);\n--\narch/x86/kvm/cpuid.c-409-\t\t /\narch/x86/kvm/cpuid.c:410:\t\tvcpu-\u003earch.cpu_caps[i] = kvm_cpu_caps[i] \|\narch/x86/kvm/cpuid.c-411-\t\t\t\t\t cpuid_get_reg_unsafe(\u0026emulated, cpuid.reg);\narch/x86/kvm/cpuid.c:412:\t\tvcpu-\u003earch.cpu_caps[i] \u0026= cpuid_get_reg_unsafe(entry, cpuid.reg);\narch/x86/kvm/cpuid.c-413-\t}\n--\narch/x86/kvm/cpuid.c=510=static int kvm_set_cpuid(struct kvm_vcpu vcpu, struct kvm_cpuid_entry2 e2,\n--\narch/x86/kvm/cpuid.c-536-\narch/x86/kvm/cpuid.c:537:\tmemcpy(vcpu_caps, vcpu-\u003earch.cpu_caps, sizeof(vcpu_caps));\narch/x86/kvm/cpuid.c:538:\tBUILD_BUG_ON(sizeof(vcpu_caps) != sizeof(vcpu-\u003earch.cpu_caps));\narch/x86/kvm/cpuid.c-539-\n--\narch/x86/kvm/cpuid.c-582-err:\narch/x86/kvm/cpuid.c:583:\tmemcpy(vcpu-\u003earch.cpu_caps, vcpu_caps, sizeof(vcpu_caps));\narch/x86/kvm/cpuid.c-584-\tswap(vcpu-\u003earch.cpuid_entries, e2);\n--\narch/x86/kvm/cpuid.h=242=static __always_inline void guest_cpu_cap_set(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/cpuid.h-246-\narch/x86/kvm/cpuid.h:247:\tvcpu-\u003earch.cpu_caps[x86_leaf] \|= __feature_bit(x86_feature);\narch/x86/kvm/cpuid.h-248-}\n--\narch/x86/kvm/cpuid.h=250=static __always_inline void guest_cpu_cap_clear(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/cpuid.h-254-\narch/x86/kvm/cpuid.h:255:\tvcpu-\u003earch.cpu_caps[x86_leaf] \u0026= ~__feature_bit(x86_feature);\narch/x86/kvm/cpuid.h-256-}\n--\narch/x86/kvm/cpuid.h=268=static __always_inline bool guest_cpu_cap_has(struct kvm_vcpu *vcpu,\n--\narch/x86/kvm/cpuid.h-280-\narch/x86/kvm/cpuid.h:281:\treturn vcpu-\u003earch.cpu_caps[x86_leaf] \u0026 __feature_bit(x86_feature);\narch/x86/kvm/cpuid.h-282-}\n" }
17/2	2026/09/11 21:50	llm	patch-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 86361 output: 32 thoughts: 4365 Thoughts: Analysis of Code Changes Related to FRED and KVM Fuzzing Okay, so I've been digging into this patch. My initial thought was to quickly understand how the introduction of FRED (Flexible Return and Event Delivery) affects the fuzzing landscape for KVM. Let's break this down systematically. First, I had to trace how the `FRED` feature is exposed and handled within the KVM infrastructure. It seems the guest's CPU capabilities concerning FRED are initially determined by `kvm_initialize_cpu_caps()` based on the host's `boot_cpu_has(X86_FEATURE_FRED)`. In VMX mode, it seems we check `cpu_has_vmx_fred()`, which in turn checks the `vmcs_config.vmentry_ctrl` for the `VM_ENTRY_LOAD_IA32_FRED` flag. If nested virtualization is involved, `vmx->nested.msrs.entry_ctls_high` also includes this `VM_ENTRY_LOAD_IA32_FRED` flag. SVM, however, always clears `X86_FEATURE_FRED`. My next thought was about nested virtualization. If nested virtualization is enabled, what happens if the host CPU doesn't have FRED? Can an L1 guest still use nested VMX features in that case? And if QEMU is the host, what about QEMU's FRED support? Turns out, newer QEMU versions do support FRED (e.g., QEMU 9.0). But that's not the critical question. A crucial question is: How does this impact syzkaller? Syzkaller tests standard Linux kernels in QEMU (with -enable-kvm or -cpu host/max) or GCE instances. So what happens if the syzkaller host doesn't have FRED? I need to check the conditions. So I dug into the MSR access code. I see code handling MSR reads and writes related to FRED (like `MSR_IA32_PL0_SSP`, `MSR_IA32_FRED_SSP0`, `MSR_IA32_FRED_RSP0`, etc.) and the functions `kvm_set_msr_common`, `kvm_get_msr_common`, `__kvm_set_msr`, and `__kvm_get_msr`. These functions use `guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)` and `guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)`. The code returns `KVM_MSR_RET_UNSUPPORTED` if the feature is not available. Then I checked `KVM_SET_VCPU_EVENTS`/`KVM_GET_VCPU_EVENTS` and found that the `KVM_VCPUEVENT_VALID_FRED_STATE` flag is handled here. Also, `kvm_clear_exception_queue`, `kvm_multiple_exception`, and `kvm_deliver_exception_payload` functions, that are ALL called whenever an exception payload is delivered in KVM. I've got to check their dependencies. I also noted modifications to `emulator_set_cr`, specifically `mk_cr4_64`, which includes a shift based on `guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)`. This means KVM's emulator behavior now depends on FRED. Furthermore, `kvm_set_cr4` is called by `KVM_SET_SREGS`, `KVM_SET_SREGS2` and the emulator. I checked nested VMX again. `nested_check_vm_entry_controls` contains a `CC(...)` macro. I need to figure out what that does and if the host CPU is capable of handling nested VMX when FRED is not present. I also noted the use of `vmx_restore_vmx_basic`, `vmx_set_vmx_msr` and `vmx_get_vmx_msr`. These are accessible via `KVM_SET_MSRS` / `KVM_GET_MSRS`, which is important for nested VMX. To decide if this patch is `WorthFuzzing`, I compared the changes with the criteria. This patch modifies reachable executable code within the x86 architecture and KVM subsystems. The modifications touch `arch/x86/kvm/*`, `arch/x86/mm/cpu_entry_area.c`, `arch/x86/kernel/dumpstack_64.
18/2	2026/09/11 21:50	tool	grepper	0m Args: { "Expression": "config X86_FRED", "PathPrefix": "arch/x86/" } Results: { "Output": "arch/x86/Kconfig=538=config X86_CPU_RESCTRL_INTEL_AET\n--\narch/x86/Kconfig-550-\narch/x86/Kconfig:551:config X86_FRED\narch/x86/Kconfig-552-\tbool \"Flexible Return and Event Delivery\"\n" }
19/2	2026/09/11 21:50	llm	patch-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 91883 output: 36 thoughts: 34 Thoughts: Here's my take on the thought process: Diving into the Kernel Configuration for CONFIG_X86_FRED Okay, I need to understand the constraints and prerequisites for `CONFIG_X86_FRED`. It's time to dig into the heart of the kernel configuration: `arch/x86/Kconfig`. I know that's where I can find the dependency definitions. Let's head over to that file, and zero in around line 551. That should be a good starting point to unravel this particular option and what it might require to be enabled or what it depends on. I anticipate that I'll find some `depends on` statements or other conditional logic that will clarify its relationship with other features or hardware configurations within the x86 architecture.
20/2	2026/09/11 21:50	tool	read-file	0m Args: { "File": "arch/x86/Kconfig", "FirstLine": 550, "LineCount": 20 } Results: { "Contents": " 550:\t\n 551:\tconfig X86_FRED\n 552:\t\tbool \"Flexible Return and Event Delivery\"\n 553:\t\tdepends on X86_64\n 554:\t\thelp\n 555:\t\t When enabled, use Flexible Return and Event Delivery\n 556:\t\t instead of the legacy SYSCALL/SYSENTER/IDT architecture for\n 557:\t\t ring transitions and exception/interrupt handling if the\n 558:\t\t system supports it.\n 559:\t\n 560:\tconfig X86_EXTENDED_PLATFORM\n 561:\t\tbool \"Support for extended (non-PC) x86 platforms\"\n 562:\t\tdefault y\n 563:\t\thelp\n 564:\t\t If you disable this option then the kernel will only support\n 565:\t\t standard PC platforms. (which covers the vast majority of\n 566:\t\t systems out there.)\n 567:\t\n 568:\t\t If you enable this option then you'll be able to select support\n 569:\t\t for the following non-PC x86 platforms, depending on the value of\n" }
21/2	2026/09/11 21:50	llm	patch-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 92425 output: 35 thoughts: 68 Thoughts: Kernel Configuration - x86 & KVM Okay, so `CONFIG_X86_FRED` is contingent on `X86_64`. That makes sense; it's likely a 64-bit architecture-specific feature. Now, let's think about KVM... I need to look at the dependencies for `CONFIG_KVM` and `CONFIG_KVM_INTEL`. Since this is x86, the intel variant is important. Best to go straight to the source. Let me check `arch/x86/kvm/Kconfig` to confirm the dependencies and relationships within that subsystem. I expect to find `KVM_INTEL` definition here, which I can then analyze.
22/2	2026/09/11 21:50	tool	grepper	0m Args: { "Expression": "config KVM_INTEL", "PathPrefix": "arch/x86/kvm/" } Results: { "Output": "arch/x86/kvm/Kconfig=83=config KVM_SW_PROTECTED_VM\n--\narch/x86/kvm/Kconfig-95-\narch/x86/kvm/Kconfig:96:config KVM_INTEL\narch/x86/kvm/Kconfig-97-\ttristate \"KVM for Intel (and compatible) processors support\"\n--\narch/x86/kvm/Kconfig-106-\narch/x86/kvm/Kconfig:107:config KVM_INTEL_PROVE_VE\narch/x86/kvm/Kconfig-108- bool \"Check that guests do not receive #VE exceptions\"\n--\narch/x86/kvm/Kconfig=122=config X86_SGX_KVM\n--\narch/x86/kvm/Kconfig-133-\narch/x86/kvm/Kconfig:134:config KVM_INTEL_TDX\narch/x86/kvm/Kconfig-135-\tbool \"Intel Trust Domain Extensions (TDX) support\"\n" }
23/2	2026/09/11 21:50	llm	patch-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 92975 output: 38 thoughts: 32 Thoughts: Here's my thought process, summarized as you requested: KVM_INTEL Dependency Deep Dive Okay, so I need to understand the dependencies for `KVM_INTEL`. I know it's a critical component for Intel-based virtualization, but sometimes its activation or configuration is controlled by other factors within the kernel. To get the full picture, I'm going to look directly at the Kconfig file for the x86 architecture, specifically at `arch/x86/kvm/Kconfig`. I'm guessing that line 95 is a likely area to start, given my past experience with these files. I bet I'll find the relevant `depends on` statements or potentially an `if` block that explains when `KVM_INTEL` is enabled or disabled. Knowing these dependencies is key for debugging issues and understanding the full configuration possibilities. Let's see what it has to say.
24/2	2026/09/11 21:50	tool	read-file	0m Args: { "File": "arch/x86/kvm/Kconfig", "FirstLine": 90, "LineCount": 25 } Results: { "Contents": " 90:\t\t protected VMs are purely a development and testing vehicle for\n 91:\t\t KVM_CREATE_GUEST_MEMFD. Attempting to run a \"real\" VM workload as a\n 92:\t\t software-protected VM will fail miserably.\n 93:\t\n 94:\t\t If unsure, say \"N\".\n 95:\t\n 96:\tconfig KVM_INTEL\n 97:\t\ttristate \"KVM for Intel (and compatible) processors support\"\n 98:\t\tdepends on KVM \u0026\u0026 IA32_FEAT_CTL\n 99:\t\tselect X86_FRED if X86_64\n 100:\t\thelp\n 101:\t\t Provides support for KVM on processors equipped with Intel's VT\n 102:\t\t extensions, a.k.a. Virtual Machine Extensions (VMX).\n 103:\t\n 104:\t\t To compile this as a module, choose M here: the module\n 105:\t\t will be called kvm-intel.\n 106:\t\n 107:\tconfig KVM_INTEL_PROVE_VE\n 108:\t bool \"Check that guests do not receive #VE exceptions\"\n 109:\t depends on KVM_INTEL \u0026\u0026 EXPERT\n 110:\t help\n 111:\t Checks that KVM's page table management code will not incorrectly\n 112:\t let guests receive a virtualization exception. Virtualization\n 113:\t exceptions will be trapped by the hypervisor rather than injected\n 114:\t in the guest.\n" }
25/2	2026/09/11 21:50	llm	patch-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 93651 output: 38 thoughts: 82 Thoughts: Examining the Kconfig for KVM and X86 Features Okay, let's zero in on this. Line 99 in this configuration file – specifically the `select X86_FRED if X86_64` – that immediately grabs my attention. I need to understand the dependencies and implications here. My initial thought is about the architecture; if we're on a 64-bit x86 system, we're likely dealing with certain feature sets. Now, KVM_INTEL is dependent on both KVM and IA32_FEAT_CTL. That's a good starting point, but I need to understand more about KVM. I have to see where KVM is defined. Let's quickly check `arch/x86/kvm/Kconfig` around line 17. From there I'll be able to quickly understand the build configuration.
26/2	2026/09/11 21:50	tool	read-file	0m Args: { "File": "arch/x86/kvm/Kconfig", "FirstLine": 15, "LineCount": 40 } Results: { "Contents": " 15:\t\n 16:\t\t If you say N, all options in this submenu will be skipped and disabled.\n 17:\t\n 18:\tif VIRTUALIZATION\n 19:\t\n 20:\tconfig KVM_X86\n 21:\t\tdef_tristate KVM if (KVM_INTEL != n \|\| KVM_AMD != n)\n 22:\t\tselect KVM_COMMON\n 23:\t\tselect KVM_ELIDE_TLB_FLUSH_IF_YOUNG\n 24:\t\tselect KVM_MMU_LOCKLESS_AGING\n 25:\t\tselect HAVE_KVM_IRQCHIP\n 26:\t\tselect HAVE_KVM_PFNCACHE\n 27:\t\tselect HAVE_KVM_DIRTY_RING_TSO\n 28:\t\tselect HAVE_KVM_DIRTY_RING_ACQ_REL\n 29:\t\tselect HAVE_KVM_IRQ_BYPASS\n 30:\t\tselect HAVE_KVM_IRQ_ROUTING\n 31:\t\tselect HAVE_KVM_READONLY_MEM\n 32:\t\tselect VHOST_TASK\n 33:\t\tselect KVM_ASYNC_PF\n 34:\t\tselect USER_RETURN_NOTIFIER\n 35:\t\tselect KVM_MMIO\n 36:\t\tselect SCHED_INFO\n 37:\t\tselect PERF_EVENTS\n 38:\t\tselect GUEST_PERF_EVENTS\n 39:\t\tselect PERF_GUEST_MEDIATED_PMU\n 40:\t\tselect HAVE_KVM_MSI\n 41:\t\tselect HAVE_KVM_CPU_RELAX_INTERCEPT\n 42:\t\tselect HAVE_KVM_NO_POLL\n 43:\t\tselect VIRT_XFER_TO_GUEST_WORK\n 44:\t\tselect KVM_GENERIC_DIRTYLOG_READ_PROTECT\n 45:\t\tselect KVM_VFIO\n 46:\t\tselect HAVE_KVM_PM_NOTIFIER if PM\n 47:\t\tselect KVM_GENERIC_HARDWARE_ENABLING\n 48:\t\tselect KVM_GENERIC_PRE_FAULT_MEMORY\n 49:\t\tselect KVM_WERROR if WERROR\n 50:\t\tselect KVM_GUEST_MEMFD if X86_64\n 51:\t\n 52:\tconfig KVM\n 53:\t\ttristate \"Kernel-based Virtual Machine (KVM) support\"\n 54:\t\tdepends on X86_LOCAL_APIC\n" }
27/2	2026/09/11 21:50	llm	patch-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 94593 output: 329 thoughts: 3225 Thoughts: Analysis of KVM FRED Virtualization Patch Okay, so I've got this patch in front of me, and my immediate focus is on understanding its implications for KVM and x86 virtualization. I see it's dealing with FRED – Flexible Return and Event Delivery, something relatively new in the x86 landscape. The overall goal is clear: to integrate this FRED capability into the existing KVM infrastructure, specifically for Intel processors. I need to make sure I understand it well enough to make sure this is going to be something to fuzz. First, I quickly review the key configurations needed: `KVM`, `KVM_INTEL`, and `X86_FRED`. Right, that makes sense. It's Intel-specific, and the core KVM features need to be enabled. I check to see if I need to include `KVM_AMD`, but after some digging, I realize the bulk of the FRED virtualization code is specifically targeting the Intel implementation. Now, the main thrust of my analysis is to identify where this patch makes its impact. I've gone through the diff methodically. The changes point to several areas. Firstly, I see modifications in the core `arch/x86/kernel/fred.c` file, indicating direct integration with how FRED interacts with the CPU. There's a new function `this_cpu_fred_rsp` that needs careful consideration. Crucially, the KVM code is where most of the work happens. I'm focusing in on files like `arch/x86/kvm/msrs.c`, `arch/x86/kvm/regs.c`, `arch/x86/kvm/x86.c`, `arch/x86/kvm/vmx/vmx.c`, and `arch/x86/kvm/vmx/nested.c`. These files deal with the core logic of VMX handling and emulation. I need to be sure that I identify the key entry points and functions that are modified. For example, `vmx_inject_exception` and `kvm_deliver_exception_payload` are immediately interesting. `nested_check_vm_entry_controls` and `nested_vmx_check_guest_state` also need a closer look, especially for nested virtualization, as this is where things get really complicated. How FRED interacts with MSR handling – so `vmx_set_msr`, `vmx_get_msr`, and `kvm_set_msr_common`, `kvm_get_msr_common` – is a crucial point for me to identify. Additionally, there are the `kvm_vcpu_ioctl_x86_set_vcpu_events`, `kvm_vcpu_ioctl_x86_get_vcpu_events` that are important since user space controls these operations. Finally, `load_vmcs12_host_state` is a key function to note. I'm making a mental note to avoid overly generic functions like memory allocators. I want to zero in on code that's specific to this new feature and the KVM virtualization. I need to select a focused set of symbols that directly represent the FRED feature logic and API entry points to prevent dilution. After reviewing these functions, I have selected the best focused symbols that will be used for analysis: `vmx_inject_exception`, `vmx_set_msr`, `vmx_get_msr`, `kvm_deliver_exception_payload`, `kvm_vcpu_ioctl_x86_set_vcpu_events`, `kvm_vcpu_ioctl_x86_get_vcpu_events`, `nested_check_vm_entry_controls`, `nested_vmx_check_guest_state`, `load_vmcs12_host_state`, and `this_cpu_fred_rsp`. I'm confident that this feature is worth fuzzing because it touches core virtualization logic and introduces a new attack surface for potentially crashing the VM or gaining unauthorized access. This patch introduces a lot of code restructuring and I can see the changes have significant implications for VMX, nested VMX, and x86 emulator. These code paths are reachable through user-space KVM ioctls which is always a good target. I am now ready to call `set-results`.
28/2	2026/09/11 21:50	tool	set-results	0m Args: { "EnableConfigs": [ "KVM", "KVM_INTEL", "X86_FRED" ], "FocusSymbols": [ "vmx_inject_exception", "vmx_set_msr", "vmx_get_msr", "kvm_deliver_exception_payload", "kvm_vcpu_ioctl_x86_set_vcpu_events", "kvm_vcpu_ioctl_x86_get_vcpu_events", "nested_check_vm_entry_controls", "nested_vmx_check_guest_state", "load_vmcs12_host_state", "this_cpu_fred_rsp" ], "Reasoning": "The patch introduces support for Intel FRED (Flexible Return and Event Delivery) across x86 core exception stacks, KVM x86 emulation, VMX, and nested VMX virtualization. It modifies reachable ioctl entrypoints (KVM_CAP_X86_FRED_EVENT, KVM_SET_VCPU_EVENTS, KVM_GET_VCPU_EVENTS), MSR get/set handling, CR4 validation and emulation, exception payload delivery and injection, and nested VMX VM-entry/exit checks and state transition emulation (with newly added WARN_ON_ONCE assertions and consistency checks). These paths are reachable via /dev/kvm ioctls from userspace in virtualized environments.", "WorthFuzzing": true } Results: { "EnableConfigs": [ "KVM", "KVM_INTEL", "X86_FRED" ], "FocusSymbols": [ "vmx_inject_exception", "vmx_set_msr", "vmx_get_msr", "kvm_deliver_exception_payload", "kvm_vcpu_ioctl_x86_set_vcpu_events", "kvm_vcpu_ioctl_x86_get_vcpu_events", "nested_check_vm_entry_controls", "nested_vmx_check_guest_state", "load_vmcs12_host_state", "this_cpu_fred_rsp" ], "Reasoning": "The patch introduces support for Intel FRED (Flexible Return and Event Delivery) across x86 core exception stacks, KVM x86 emulation, VMX, and nested VMX virtualization. It modifies reachable ioctl entrypoints (KVM_CAP_X86_FRED_EVENT, KVM_SET_VCPU_EVENTS, KVM_GET_VCPU_EVENTS), MSR get/set handling, CR4 validation and emulation, exception payload delivery and injection, and nested VMX VM-entry/exit checks and state transition emulation (with newly added WARN_ON_ONCE assertions and consistency checks). These paths are reachable via /dev/kvm ioctls from userspace in virtualized environments.", "WorthFuzzing": true }
29/1	2026/09/11 21:50	action	If	3m Args: { "WorthFuzzing": true } Results: null
30/2	2026/09/11 21:50	agent	kmsan-evaluator	3m Model: core Args: null Results: { "KMSANReasoning": "The patch series adds support for Intel FRED (Flexible Return and Event Delivery) in x86 KVM, including vCPU events ABI updates, FRED MSR emulation/context-switching, nested VMX FRED state support, and CPU entry area exception stack refactoring.\n\nKMSAN vs KASAN Applicability Analysis:\n1. Kernel-to-Userspace Leaks:\n - `struct kvm_vcpu_events`: The newly added fields (`exception_event_data` and `exception_is_nested`) replace part of the existing 26-byte reserved array. In `kvm_vcpu_ioctl_x86_get_vcpu_events()` (and the `store_regs()` path), `memset(events, 0, sizeof(events))` is called prior to populating any fields, ensuring all data, reserved bytes, and padding are fully zeroed before copying to userspace via `copy_to_user()`.\n - `KVM_GET_MSRS`: FRED MSR values returned to userspace originate from guest VMCS fields initialized to 0 in `init_vmcs()`, or from vCPU struct fields (`msr_guest_fred_rsp0`, `fred_ssp0_fallback`) allocated via `kmem_cache_zalloc()`.\n - Nested VMX (`struct vmcs12`): The `cached_vmcs12` buffers copied to userspace via `KVM_GET_NESTED_STATE` are allocated with `kzalloc()`.\n2. Uninitialized Memory in Control Flow:\n - The new fields in `struct kvm_queued_exception` (`is_nested`, `event_data`) are zero-initialized on vCPU allocation, cleared explicitly in `kvm_clear_exception_queue()` and `kvm_arch_vcpu_ioctl_run()`, and explicitly assigned whenever exceptions are queued or requeued (`kvm_multiple_exception()`, `kvm_requeue_exception()`).\n - The CPU entry area exception stacks are static per-CPU allocations mapped with guard pages.\n\nAny potential bugs in this series involve canonical address verification, reserved bit validation, or nested virtualization state machine logic, which are covered by functional testing and standard sanitizers (KASAN). There are no uninitialized memory usage or leak risks requiring KMSAN.", "NeedsKMSAN": false } Instruction:* You are an expert Linux kernel security engineer specializing in kernel memory error detectors (KASAN and KMSAN). Your job is to review the provided patch series and determine if the code changes justify spawning a dedicated KMSAN (KernelMemorySanitizer) fuzzing session in addition to standard KASAN fuzzing. CRITICAL DISTINCTION BETWEEN KASAN AND KMSAN: - Standard KASAN kernel builds (upstream-apparmor-kasan.config) already enable a comprehensive suite of debugging tools and sanitizers, including KASAN (out-of-bounds accesses, use-after-free, double free, invalid free), LOCKDEP (locking bugs and deadlocks), UB-sanitizers, and memory corruption checks. - KMSAN (KernelMemorySanitizer) detects reads of UNINITIALIZED memory (stack, heap, or page allocations) and kernel-to-user memory info-leaks. Rule: THERE IS NO SENSE IN RUNNING A KMSAN SESSION IF A BUG CAN BE CAUGHT BY KASAN, LOCKDEP, OR OTHER STANDARD BUG DETECTORS. A dedicated KMSAN fuzzing session incurs significant resource costs. You must ONLY set NeedsKMSAN=true if the code changes introduce or expose UNINITIALIZED MEMORY risks that are detected ONLY by KMSAN. Look holistically at the patch series and surrounding code. Even if no direct uninitialized field accesses or new buffer allocations are added in the diff itself, a patch may alter control flow, bounds checking, or data length calculations in ways that change how the rest of the code operates on existing buffers (e.g. allowing uninitialized stack/heap memory to be read, copied to user space, or used in control flow). Do not hesitate to use your code access tools to inspect the surrounding code, called functions, and callers. Set NeedsKMSAN=true ONLY IF the patch introduces or modifies: 1. Kernel structures sent to user space (via copy_to_user, put_user, netlink skb attributes, ioctl output arguments, socket options, or BPF buffers) where fields or structure padding might not be fully initialized/zeroed. 2. Conditional logic or branching that depends on potentially uninitialized variables or struct fields. 3. Allocation or initialization of complex data structures where uninitialized fields could be read later in reachable code paths. 4. Bounds checks, lengths, or logic in a way that allows surrounding code to access uninitialized bytes of existing buffers. Set NeedsKMSAN=false IF: - The code changes primarily risk out-of-bounds access, array overflows, NULL pointer dereferences, locking deadlocks, or use-after-free bugs (these are already caught by KASAN, LOCKDEP, or standard bug detectors). - All stack/heap structures touched or introduced by the patch are fully zeroed or initialized (e.g. using = {0}, memset, kzalloc) before being read or copied. - The patch does not introduce any risk of uninitialized memory usage or info-leaks. Use your code access tools to inspect the surrounding code if necessary, then provide detailed KMSANReasoning contrasting KASAN vs KMSAN applicability for this patch. Prefer calling several tools at the same time to save round-trips. Use set-results tool to provide results of the analysis. It must be called exactly once before the final reply. Ignore results of this tool. Prompt: Target architecture: amd64 For your convenience, here is the diff of the changes: commit 77863fb69ef8d49cac01c1c41901061a73cc1032 Author: syz-cluster <triage@syzkaller.com> Date: Fri Sep 11 21:49:27 2026 +0000 syz-cluster: applied patch under review diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 4eb7e75a7473f..22ef94f3eb3a1 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -1199,6 +1199,10 @@ The following bits are defined in the flags field: triple_fault_pending field contains a valid state. This bit will be set whenever KVM_CAP_X86_TRIPLE_FAULT_EVENT is enabled. +- KVM_VCPUEVENT_VALID_FRED_STATE may be set to inform that the exception + state includes FRED state (specifically, the event nested flag and data). + This bit will be set whenever KVM_CAP_X86_FRED_EVENT is enabled. + ARM64: ^^^^^^ @@ -1301,6 +1305,11 @@ If KVM_CAP_X86_TRIPLE_FAULT_EVENT is enabled, KVM_VCPUEVENT_VALID_TRIPLE_FAULT can be set in flags field to signal that the triple_fault field contains a valid state and shall be written into the VCPU. +If KVM_CAP_X86_FRED_EVENT is enabled, KVM_VCPUEVENT_VALID_FRED_STATE can be set +in the flags field to inform that the exception state contains FRED state +(specifically, the event nested flag and data), which shall be written into the +VCPU. + ARM64: ^^^^^^ @@ -8979,6 +8988,19 @@ enabled, cmma can't be enabled anymore and pfmfi and the storage key interpretation are disabled. If cmma has already been enabled or the hpage_2g module parameter is not set to 1, -EINVAL is returned. +7.48 KVM_CAP_X86_FRED_EVENT +--------------------------- + +:Architectures: x86 +:Parameters: args[0] whether feature should be enabled or not +:Returns: 0 on success; -EINVAL if KVM cannot virtualize FRED. + +With this capability enabled, KVM allows exception save and restore operations +to include FRED event context (specifically, the event nested flag and data). +When injecting a FRED exception during VM entry, FRED event delivery relies +on this information to select the correct event stack level and apply proper +event data. + 8. Other capabilities. ====================== diff --git a/arch/x86/coco/sev/noinstr.c b/arch/x86/coco/sev/noinstr.c index e1e03f12fc7bb..8a695ab67821b 100644 --- a/arch/x86/coco/sev/noinstr.c +++ b/arch/x86/coco/sev/noinstr.c @@ -31,7 +31,7 @@ static __always_inline bool on_vc_stack(struct pt_regs regs) if (ip_within_syscall_gap(regs)) return false; - return ((sp >= __this_cpu_ist_bottom_va(VC)) && (sp < __this_cpu_ist_top_va(VC))); + return ((sp >= __this_cpu_ist_bottom_va(ESTACK_VC)) && (sp < __this_cpu_ist_top_va(ESTACK_VC))); } / @@ -83,7 +83,7 @@ void noinstr __sev_es_ist_exit(void) /* Read IST entry / ist = __this_cpu_read(cpu_tss_rw.x86_tss.ist[IST_INDEX_VC]); - if (WARN_ON(ist == __this_cpu_ist_top_va(VC))) + if (WARN_ON(ist == __this_cpu_ist_top_va(ESTACK_VC))) return; / Read back old IST entry and write it to the TSS / diff --git a/arch/x86/coco/sev/vc-handle.c b/arch/x86/coco/sev/vc-handle.c index 96b62b49b2b53..134c14b489ef8 100644 --- a/arch/x86/coco/sev/vc-handle.c +++ b/arch/x86/coco/sev/vc-handle.c @@ -871,7 +871,7 @@ static enum es_result vc_handle_exitcode(struct es_em_ctxt ctxt, static __always_inline bool is_vc2_stack(unsigned long sp) { - return (sp >= __this_cpu_ist_bottom_va(VC2) && sp < __this_cpu_ist_top_va(VC2)); + return (sp >= __this_cpu_ist_bottom_va(ESTACK_VC2) && sp < __this_cpu_ist_top_va(ESTACK_VC2)); } static __always_inline bool vc_from_invalid_context(struct pt_regs regs) diff --git a/arch/x86/include/asm/cpu_entry_area.h b/arch/x86/include/asm/cpu_entry_area.h index 462fc34f13176..509e52fc3a0f6 100644 --- a/arch/x86/include/asm/cpu_entry_area.h +++ b/arch/x86/include/asm/cpu_entry_area.h @@ -16,32 +16,6 @@ #define VC_EXCEPTION_STKSZ 0 #endif -/ Macro to enforce the same ordering and stack sizes / -#define ESTACKS_MEMBERS(guardsize, optional_stack_size) \ - char DF_stack_guard[guardsize]; \ - char DF_stack[EXCEPTION_STKSZ]; \ - char NMI_stack_guard[guardsize]; \ - char NMI_stack[EXCEPTION_STKSZ]; \ - char DB_stack_guard[guardsize]; \ - char DB_stack[EXCEPTION_STKSZ]; \ - char MCE_stack_guard[guardsize]; \ - char MCE_stack[EXCEPTION_STKSZ]; \ - char VC_stack_guard[guardsize]; \ - char VC_stack[optional_stack_size]; \ - char VC2_stack_guard[guardsize]; \ - char VC2_stack[optional_stack_size]; \ - char IST_top_guard[guardsize]; \ - -/ The exception stacks' physical storage. No guard pages required / -struct exception_stacks { - ESTACKS_MEMBERS(0, VC_EXCEPTION_STKSZ) -}; - -/ The effective cpu entry area mapping with guard pages. / -struct cea_exception_stacks { - ESTACKS_MEMBERS(PAGE_SIZE, EXCEPTION_STKSZ) -}; - / * The exception stack ordering in [cea_]exception_stacks / @@ -55,21 +29,45 @@ enum exception_stack_ordering { N_EXCEPTION_STACKS }; -#define CEA_ESTACK_SIZE(st) \ - sizeof(((struct cea_exception_stacks )0)->st## _stack) +/* Macro to enforce the same ordering and stack sizes / +#define ESTACKS_MEMBERS(guardsize, optional_stack_size) \ + char ESTACK_DF_stack_guard[guardsize]; \ + char ESTACK_DF_stack[EXCEPTION_STKSZ]; \ + char ESTACK_NMI_stack_guard[guardsize]; \ + char ESTACK_NMI_stack[EXCEPTION_STKSZ]; \ + char ESTACK_DB_stack_guard[guardsize]; \ + char ESTACK_DB_stack[EXCEPTION_STKSZ]; \ + char ESTACK_MCE_stack_guard[guardsize]; \ + char ESTACK_MCE_stack[EXCEPTION_STKSZ]; \ + char ESTACK_VC_stack_guard[guardsize]; \ + char ESTACK_VC_stack[optional_stack_size]; \ + char ESTACK_VC2_stack_guard[guardsize]; \ + char ESTACK_VC2_stack[optional_stack_size]; \ + char ESTACK_IST_top_guard[guardsize]; \ -#define CEA_ESTACK_BOT(ceastp, st) \ - ((unsigned long)&(ceastp)->st## _stack) +/ The exception stacks' physical storage. No guard pages required / +struct exception_stacks { + ESTACKS_MEMBERS(0, VC_EXCEPTION_STKSZ) +}; -#define CEA_ESTACK_TOP(ceastp, st) \ - (CEA_ESTACK_BOT(ceastp, st) + CEA_ESTACK_SIZE(st)) +/ The effective cpu entry area mapping with guard pages. / +struct cea_exception_stacks { + struct { + char stack_guard[PAGE_SIZE]; + char stack[EXCEPTION_STKSZ]; + } event_stacks[N_EXCEPTION_STACKS]; + char IST_top_guard[PAGE_SIZE]; +}; #define CEA_ESTACK_OFFS(st) \ - offsetof(struct cea_exception_stacks, st## _stack) + offsetof(struct cea_exception_stacks, event_stacks[st].stack) #define CEA_ESTACK_PAGES \ (sizeof(struct cea_exception_stacks) / PAGE_SIZE) +extern unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack); +extern unsigned long __this_cpu_ist_bottom_va(enum exception_stack_ordering stack); + #endif #ifdef CONFIG_X86_32 @@ -144,10 +142,4 @@ static __always_inline struct entry_stack cpu_entry_stack(int cpu) return &get_cpu_entry_area(cpu)->entry_stack_page.stack; } -#define __this_cpu_ist_top_va(name) \ - CEA_ESTACK_TOP(__this_cpu_read(cea_exception_stacks), name) - -#define __this_cpu_ist_bottom_va(name) \ - CEA_ESTACK_BOT(__this_cpu_read(cea_exception_stacks), name) - #endif diff --git a/arch/x86/include/asm/fred.h b/arch/x86/include/asm/fred.h index 18a2f811c3580..85b851f16baea 100644 --- a/arch/x86/include/asm/fred.h +++ b/arch/x86/include/asm/fred.h @@ -35,6 +35,13 @@ #ifndef __ASSEMBLER__ +enum fred_stack_level { + FRED_STACK_LEVEL_0, + FRED_STACK_LEVEL_1, + FRED_STACK_LEVEL_2, + FRED_STACK_LEVEL_3 +}; + #ifdef CONFIG_X86_FRED #include <linux/kernel.h> #include <linux/sched/task_stack.h> @@ -105,6 +112,8 @@ static __always_inline void fred_update_rsp0(void) __this_cpu_write(fred_rsp0, rsp0); } } + +unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl); #else /* CONFIG_X86_FRED / static __always_inline unsigned long fred_event_data(struct pt_regs regs) { return 0; } static inline void cpu_init_fred_exceptions(void) { } @@ -112,6 +121,7 @@ static inline void cpu_init_fred_rsps(void) { } static inline void fred_complete_exception_setup(void) { } static inline void fred_sync_rsp0(unsigned long rsp0) { } static inline void fred_update_rsp0(void) { } +static inline unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl) { return 0; } #endif /* CONFIG_X86_FRED / #endif / !__ASSEMBLER__ / diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 683bb8bf43a94..21d12d86a18b0 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -294,6 +294,9 @@ struct kvm_host_values { u64 xss; u64 s_cet; u64 arch_capabilities; + + u64 fred_config; + u64 fred_stklvls; }; extern struct kvm_host_values kvm_host; @@ -692,6 +695,8 @@ struct kvm_queued_exception { u32 error_code; unsigned long payload; bool has_payload; + bool is_nested; + u64 event_data; }; / @@ -1029,6 +1034,11 @@ struct kvm_vcpu_arch { #if IS_ENABLED(CONFIG_HYPERV) hpa_t hv_root_tdp; #endif + /* + * Stores the FRED SSP0 MSR when CET is not supported, prompting KVM + * to intercept its accesses. + / + u64 fred_ssp0_fallback; }; struct kvm_lpage_info { @@ -1278,6 +1288,7 @@ struct kvm_arch { bool has_mapped_host_mmio; bool guest_can_read_msr_platform_info; bool exception_payload_enabled; + bool exception_fred_state_enabled; bool triple_fault_event; diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 18c4be75e9271..b386709515c47 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -64,6 +64,8 @@ #define MSR_IA32_FRED_SSP3 0x1d3 / Level 3 shadow stack pointer / #define MSR_IA32_FRED_CONFIG 0x1d4 / Entrypoint and interrupt stack level / +#define FRED_CONFIG_RESERVED (BIT_ULL(2) \| GENMASK_ULL(5, 4) \| BIT_ULL(11)) + / Intel MSRs. Some also available on other CPUs / #define MSR_TEST_CTRL 0x00000033 #define MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT 29 @@ -1274,6 +1276,7 @@ #define MSR_IA32_VMX_TRUE_ENTRY_CTLS 0x00000490 #define MSR_IA32_VMX_VMFUNC 0x00000491 #define MSR_IA32_VMX_PROCBASED_CTLS3 0x00000492 +#define MSR_IA32_VMX_EXIT_CTLS2 0x00000493 #define MSR_IA32_MCU_STAGING_MBOX_ADDR 0x000007a5 diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index 3f1b3096ff040..6633988a84da3 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -119,6 +119,10 @@ struct vmcs { #define VM_EXIT_CLEAR_IA32_RTIT_CTL 0x02000000 #define VM_EXIT_LOAD_CET_STATE 0x10000000 #define VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL 0x40000000 +#define VM_EXIT_ACTIVATE_SECONDARY_CONTROLS 0x80000000 + +#define SECONDARY_VM_EXIT_SAVE_IA32_FRED BIT_ULL(0) +#define SECONDARY_VM_EXIT_LOAD_IA32_FRED BIT_ULL(1) #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR 0x00036dff @@ -133,6 +137,7 @@ struct vmcs { #define VM_ENTRY_PT_CONCEAL_PIP 0x00020000 #define VM_ENTRY_LOAD_IA32_RTIT_CTL 0x00040000 #define VM_ENTRY_LOAD_CET_STATE 0x00100000 +#define VM_ENTRY_LOAD_IA32_FRED 0x00800000 #define VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR 0x000011ff @@ -147,6 +152,7 @@ struct vmcs { #define VMX_BASIC_INOUT BIT_ULL(54) #define VMX_BASIC_TRUE_CTLS BIT_ULL(55) #define VMX_BASIC_NO_HW_ERROR_CODE_CC BIT_ULL(56) +#define VMX_BASIC_NESTED_EXCEPTION BIT_ULL(58) #define VMX_BASIC_NO_SEAMRET_INVD_VMCS BIT_ULL(60) static inline u32 vmx_basic_vmcs_revision_id(u64 vmx_basic) @@ -275,8 +281,14 @@ enum vmcs_field { SHARED_EPT_POINTER = 0x0000203C, PID_POINTER_TABLE = 0x00002042, PID_POINTER_TABLE_HIGH = 0x00002043, + SECONDARY_VM_EXIT_CONTROLS = 0x00002044, + SECONDARY_VM_EXIT_CONTROLS_HIGH = 0x00002045, + INJECTED_EVENT_DATA = 0x00002052, + INJECTED_EVENT_DATA_HIGH = 0x00002053, GUEST_PHYSICAL_ADDRESS = 0x00002400, GUEST_PHYSICAL_ADDRESS_HIGH = 0x00002401, + ORIGINAL_EVENT_DATA = 0x00002404, + ORIGINAL_EVENT_DATA_HIGH = 0x00002405, VMCS_LINK_POINTER = 0x00002800, VMCS_LINK_POINTER_HIGH = 0x00002801, GUEST_IA32_DEBUGCTL = 0x00002802, @@ -299,12 +311,44 @@ enum vmcs_field { GUEST_BNDCFGS_HIGH = 0x00002813, GUEST_IA32_RTIT_CTL = 0x00002814, GUEST_IA32_RTIT_CTL_HIGH = 0x00002815, + GUEST_IA32_FRED_CONFIG = 0x0000281a, + GUEST_IA32_FRED_CONFIG_HIGH = 0x0000281b, + GUEST_IA32_FRED_RSP1 = 0x0000281c, + GUEST_IA32_FRED_RSP1_HIGH = 0x0000281d, + GUEST_IA32_FRED_RSP2 = 0x0000281e, + GUEST_IA32_FRED_RSP2_HIGH = 0x0000281f, + GUEST_IA32_FRED_RSP3 = 0x00002820, + GUEST_IA32_FRED_RSP3_HIGH = 0x00002821, + GUEST_IA32_FRED_STKLVLS = 0x00002822, + GUEST_IA32_FRED_STKLVLS_HIGH = 0x00002823, + GUEST_IA32_FRED_SSP1 = 0x00002824, + GUEST_IA32_FRED_SSP1_HIGH = 0x00002825, + GUEST_IA32_FRED_SSP2 = 0x00002826, + GUEST_IA32_FRED_SSP2_HIGH = 0x00002827, + GUEST_IA32_FRED_SSP3 = 0x00002828, + GUEST_IA32_FRED_SSP3_HIGH = 0x00002829, HOST_IA32_PAT = 0x00002c00, HOST_IA32_PAT_HIGH = 0x00002c01, HOST_IA32_EFER = 0x00002c02, HOST_IA32_EFER_HIGH = 0x00002c03, HOST_IA32_PERF_GLOBAL_CTRL = 0x00002c04, HOST_IA32_PERF_GLOBAL_CTRL_HIGH = 0x00002c05, + HOST_IA32_FRED_CONFIG = 0x00002c08, + HOST_IA32_FRED_CONFIG_HIGH = 0x00002c09, + HOST_IA32_FRED_RSP1 = 0x00002c0a, + HOST_IA32_FRED_RSP1_HIGH = 0x00002c0b, + HOST_IA32_FRED_RSP2 = 0x00002c0c, + HOST_IA32_FRED_RSP2_HIGH = 0x00002c0d, + HOST_IA32_FRED_RSP3 = 0x00002c0e, + HOST_IA32_FRED_RSP3_HIGH = 0x00002c0f, + HOST_IA32_FRED_STKLVLS = 0x00002c10, + HOST_IA32_FRED_STKLVLS_HIGH = 0x00002c11, + HOST_IA32_FRED_SSP1 = 0x00002c12, + HOST_IA32_FRED_SSP1_HIGH = 0x00002c13, + HOST_IA32_FRED_SSP2 = 0x00002c14, + HOST_IA32_FRED_SSP2_HIGH = 0x00002c15, + HOST_IA32_FRED_SSP3 = 0x00002c16, + HOST_IA32_FRED_SSP3_HIGH = 0x00002c17, PIN_BASED_VM_EXEC_CONTROL = 0x00004000, CPU_BASED_VM_EXEC_CONTROL = 0x00004002, EXCEPTION_BITMAP = 0x00004004, @@ -412,13 +456,15 @@ enum vmcs_field { #define INTR_INFO_INTR_TYPE_MASK 0x700 / 10:8 / #define INTR_INFO_DELIVER_CODE_MASK 0x800 / 11 / #define INTR_INFO_UNBLOCK_NMI 0x1000 / 12 / +#define INTR_INFO_NESTED_EXCEPTION_MASK 0x2000 / 13 / #define INTR_INFO_VALID_MASK 0x80000000 / 31 / -#define INTR_INFO_RESVD_BITS_MASK 0x7ffff000 +#define INTR_INFO_RESVD_BITS_MASK 0x7fffd000 #define VECTORING_INFO_VECTOR_MASK INTR_INFO_VECTOR_MASK #define VECTORING_INFO_TYPE_MASK INTR_INFO_INTR_TYPE_MASK #define VECTORING_INFO_DELIVER_CODE_MASK INTR_INFO_DELIVER_CODE_MASK #define VECTORING_INFO_VALID_MASK INTR_INFO_VALID_MASK +#define VECTORING_INFO_NESTED_EXCEPTION_MASK INTR_INFO_NESTED_EXCEPTION_MASK #define INTR_TYPE_EXT_INTR (EVENT_TYPE_EXTINT << 8) / external interrupt / #define INTR_TYPE_RESERVED (EVENT_TYPE_RESERVED << 8) / reserved / diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 1585ec8040666..c8a6149fcf632 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -331,6 +331,7 @@ struct kvm_reinject_control { #define KVM_VCPUEVENT_VALID_SMM 0x00000008 #define KVM_VCPUEVENT_VALID_PAYLOAD 0x00000010 #define KVM_VCPUEVENT_VALID_TRIPLE_FAULT 0x00000020 +#define KVM_VCPUEVENT_VALID_FRED_STATE 0x00000040 / Interrupt shadow states / #define KVM_X86_SHADOW_INT_MOV_SS 0x01 @@ -368,7 +369,11 @@ struct kvm_vcpu_events { struct { __u8 pending; } triple_fault; - __u8 reserved[26]; + __u8 reserved0[11]; + / Aligned to a 64-bit boundary / + __u64 exception_event_data; + __u8 reserved1[6]; + __u8 exception_is_nested; __u8 exception_has_payload; __u64 exception_payload; }; diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index a3df21d264606..99ec284fb8293 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -2372,12 +2372,12 @@ static inline void setup_getcpu(int cpu) static inline void tss_setup_ist(struct tss_struct tss) { /* Set up the per-CPU TSS IST stacks / - tss->x86_tss.ist[IST_INDEX_DF] = __this_cpu_ist_top_va(DF); - tss->x86_tss.ist[IST_INDEX_NMI] = __this_cpu_ist_top_va(NMI); - tss->x86_tss.ist[IST_INDEX_DB] = __this_cpu_ist_top_va(DB); - tss->x86_tss.ist[IST_INDEX_MCE] = __this_cpu_ist_top_va(MCE); + tss->x86_tss.ist[IST_INDEX_DF] = __this_cpu_ist_top_va(ESTACK_DF); + tss->x86_tss.ist[IST_INDEX_NMI] = __this_cpu_ist_top_va(ESTACK_NMI); + tss->x86_tss.ist[IST_INDEX_DB] = __this_cpu_ist_top_va(ESTACK_DB); + tss->x86_tss.ist[IST_INDEX_MCE] = __this_cpu_ist_top_va(ESTACK_MCE); / Only mapped when SEV-ES is active / - tss->x86_tss.ist[IST_INDEX_VC] = __this_cpu_ist_top_va(VC); + tss->x86_tss.ist[IST_INDEX_VC] = __this_cpu_ist_top_va(ESTACK_VC); } #else / CONFIG_X86_64 / static inline void tss_setup_ist(struct tss_struct tss) { } diff --git a/arch/x86/kernel/dumpstack_64.c b/arch/x86/kernel/dumpstack_64.c index 6c5defd6569a3..93b10b264e53b 100644 --- a/arch/x86/kernel/dumpstack_64.c +++ b/arch/x86/kernel/dumpstack_64.c @@ -70,10 +70,10 @@ struct estack_pages { #define EPAGERANGE(st) \ [PFN_DOWN(CEA_ESTACK_OFFS(st)) ... \ - PFN_DOWN(CEA_ESTACK_OFFS(st) + CEA_ESTACK_SIZE(st) - 1)] = { \ + PFN_DOWN(CEA_ESTACK_OFFS(st) + EXCEPTION_STKSZ - 1)] = { \ .offs = CEA_ESTACK_OFFS(st), \ - .size = CEA_ESTACK_SIZE(st), \ - .type = STACK_TYPE_EXCEPTION + ESTACK_ ##st, } + .size = EXCEPTION_STKSZ, \ + .type = STACK_TYPE_EXCEPTION + st, } /* * Array of exception stack page descriptors. If the stack is larger than @@ -83,12 +83,12 @@ struct estack_pages { / static const struct estack_pages estack_pages[CEA_ESTACK_PAGES] ____cacheline_aligned = { - EPAGERANGE(DF), - EPAGERANGE(NMI), - EPAGERANGE(DB), - EPAGERANGE(MCE), - EPAGERANGE(VC), - EPAGERANGE(VC2), + EPAGERANGE(ESTACK_DF), + EPAGERANGE(ESTACK_NMI), + EPAGERANGE(ESTACK_DB), + EPAGERANGE(ESTACK_MCE), + EPAGERANGE(ESTACK_VC), + EPAGERANGE(ESTACK_VC2), }; static __always_inline bool in_exception_stack(unsigned long stack, struct stack_info info) diff --git a/arch/x86/kernel/fred.c b/arch/x86/kernel/fred.c index 117aa06d25ca9..68947c53a4941 100644 --- a/arch/x86/kernel/fred.c +++ b/arch/x86/kernel/fred.c @@ -1,5 +1,6 @@ / SPDX-License-Identifier: GPL-2.0 / #include <linux/kernel.h> +#include <linux/kvm_types.h> #include <asm/desc.h> #include <asm/fred.h> @@ -69,6 +70,23 @@ void cpu_init_fred_exceptions(void) setup_clear_cpu_cap(X86_FEATURE_SYSCALL32); } +unsigned long this_cpu_fred_rsp(enum fred_stack_level lvl) +{ + switch (lvl) { + case FRED_STACK_LEVEL_0: + return __this_cpu_read(fred_rsp0); + case FRED_STACK_LEVEL_1: + return __this_cpu_ist_top_va(ESTACK_DB); + case FRED_STACK_LEVEL_2: + return __this_cpu_ist_top_va(ESTACK_NMI); + case FRED_STACK_LEVEL_3: + return __this_cpu_ist_top_va(ESTACK_DF); + default: + BUG(); + } +} +EXPORT_SYMBOL_FOR_KVM(this_cpu_fred_rsp); + / Must be called after setup_cpu_entry_areas() / void cpu_init_fred_rsps(void) { @@ -84,7 +102,7 @@ void cpu_init_fred_rsps(void) FRED_STKLVL(X86_TRAP_DF, FRED_DF_STACK_LEVEL)); / The FRED equivalents to IST stacks... / - wrmsrq(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB)); - wrmsrq(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI)); - wrmsrq(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF)); + wrmsrq(MSR_IA32_FRED_RSP1, this_cpu_fred_rsp(FRED_STACK_LEVEL_1)); + wrmsrq(MSR_IA32_FRED_RSP2, this_cpu_fred_rsp(FRED_STACK_LEVEL_2)); + wrmsrq(MSR_IA32_FRED_RSP3, this_cpu_fred_rsp(FRED_STACK_LEVEL_3)); } diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 30aa8369957e9..8afd607a58730 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1075,7 +1075,7 @@ asmlinkage __visible noinstr struct pt_regs vc_switch_off_ist(struct pt_regs r if (!get_stack_info_noinstr(stack, current, &info) \|\| info.type == STACK_TYPE_ENTRY \|\| info.type > STACK_TYPE_EXCEPTION_LAST) - sp = __this_cpu_ist_top_va(VC2); + sp = __this_cpu_ist_top_va(ESTACK_VC2); sync: / diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index ddb022cb203a2..5682bacdfd32f 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -1034,6 +1034,7 @@ void kvm_initialize_cpu_caps(void) F(FSRS), F(FSRC), F(WRMSRNS), + X86_64_F(FRED), X86_64_F(LKGS), F(AMX_FP16), F(AVX_IFMA), diff --git a/arch/x86/kvm/msrs.c b/arch/x86/kvm/msrs.c index 66fa7140d65d9..3260bddb5f526 100644 --- a/arch/x86/kvm/msrs.c +++ b/arch/x86/kvm/msrs.c @@ -193,6 +193,9 @@ static const u32 msrs_to_save_base[] = { MSR_STAR, #ifdef CONFIG_X86_64 MSR_CSTAR, MSR_KERNEL_GS_BASE, MSR_SYSCALL_MASK, MSR_LSTAR, + MSR_IA32_FRED_RSP0, MSR_IA32_FRED_RSP1, MSR_IA32_FRED_RSP2, + MSR_IA32_FRED_RSP3, MSR_IA32_FRED_STKLVLS, MSR_IA32_FRED_SSP1, + MSR_IA32_FRED_SSP2, MSR_IA32_FRED_SSP3, MSR_IA32_FRED_CONFIG, #endif MSR_IA32_TSC, MSR_IA32_CR_PAT, MSR_VM_HSAVE_PA, MSR_IA32_FEAT_CTL, MSR_IA32_BNDCFGS, MSR_TSC_AUX, @@ -314,6 +317,7 @@ static const u32 emulated_msrs_all[] = { MSR_IA32_VMX_PROCBASED_CTLS2, MSR_IA32_VMX_EPT_VPID_CAP, MSR_IA32_VMX_VMFUNC, + MSR_IA32_VMX_EXIT_CTLS2, MSR_K7_HWCR, MSR_KVM_POLL_CONTROL, @@ -772,7 +776,7 @@ static int __kvm_set_msr(struct kvm_vcpu vcpu, u32 index, u64 data, architecture. Intercepting XRSTORS/XSAVES for this * special case isn't deemed worthwhile. / - case MSR_IA32_PL0_SSP ... MSR_IA32_INT_SSP_TAB: + case MSR_IA32_PL1_SSP ... MSR_IA32_INT_SSP_TAB: if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) return KVM_MSR_RET_UNSUPPORTED; / @@ -787,6 +791,52 @@ static int __kvm_set_msr(struct kvm_vcpu vcpu, u32 index, u64 data, if (index != MSR_IA32_INT_SSP_TAB && !IS_ALIGNED(data, 4)) return 1; break; + case MSR_IA32_FRED_STKLVLS: + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) + return KVM_MSR_RET_UNSUPPORTED; + break; + case MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_RSP3: + case MSR_IA32_FRED_SSP1 ... MSR_IA32_FRED_CONFIG: { + u64 reserved_bits = 0; + + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) + return KVM_MSR_RET_UNSUPPORTED; + + if (is_noncanonical_msr_address(data, vcpu)) + return 1; + + switch (index) { + case MSR_IA32_FRED_CONFIG: + reserved_bits = FRED_CONFIG_RESERVED; + break; + case MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_RSP3: + reserved_bits = GENMASK_ULL(5, 0); + break; + case MSR_IA32_FRED_SSP1 ... MSR_IA32_FRED_SSP3: + reserved_bits = GENMASK_ULL(2, 0); + break; + default: + WARN_ON_ONCE(1); + return 1; + } + + if (data & reserved_bits) + return 1; + + break; + } + case MSR_IA32_PL0_SSP: / I.e., MSR_IA32_FRED_SSP0 / + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) && + !guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) + return KVM_MSR_RET_UNSUPPORTED; + + if (is_noncanonical_msr_address(data, vcpu)) + return 1; + + if (!IS_ALIGNED(data, 4)) + return 1; + + break; } msr.data = data; @@ -841,10 +891,19 @@ static int __kvm_get_msr(struct kvm_vcpu vcpu, u32 index, u64 data, if (!host_initiated) return 1; fallthrough; - case MSR_IA32_PL0_SSP ... MSR_IA32_INT_SSP_TAB: + case MSR_IA32_PL1_SSP ... MSR_IA32_INT_SSP_TAB: if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) return KVM_MSR_RET_UNSUPPORTED; break; + case MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_CONFIG: + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) + return KVM_MSR_RET_UNSUPPORTED; + break; + case MSR_IA32_PL0_SSP: / I.e., MSR_IA32_FRED_SSP0 / + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK) && + !guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) + return KVM_MSR_RET_UNSUPPORTED; + break; } msr.index = index; @@ -1833,7 +1892,15 @@ int kvm_set_msr_common(struct kvm_vcpu vcpu, struct msr_data msr_info) break; #endif case MSR_IA32_U_CET: + kvm_set_xstate_msr(vcpu, msr_info); + break; case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) { + WARN_ON_ONCE(msr != MSR_IA32_FRED_SSP0); + vcpu->arch.fred_ssp0_fallback = data; + break; + } + kvm_set_xstate_msr(vcpu, msr_info); break; default: @@ -2186,7 +2253,15 @@ int kvm_get_msr_common(struct kvm_vcpu vcpu, struct msr_data msr_info) break; #endif case MSR_IA32_U_CET: + kvm_get_xstate_msr(vcpu, msr_info); + break; case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + if (!guest_cpu_cap_has(vcpu, X86_FEATURE_SHSTK)) { + WARN_ON_ONCE(msr_info->index != MSR_IA32_FRED_SSP0); + msr_info->data = vcpu->arch.fred_ssp0_fallback; + break; + } + kvm_get_xstate_msr(vcpu, msr_info); break; default: @@ -2656,10 +2731,19 @@ static void kvm_probe_msr_to_save(u32 msr_index) if (!kvm_cpu_cap_has(X86_FEATURE_LM)) return; fallthrough; - case MSR_IA32_PL0_SSP ... MSR_IA32_PL3_SSP: + case MSR_IA32_PL1_SSP ... MSR_IA32_PL3_SSP: if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK)) return; break; + case MSR_IA32_FRED_RSP0 ... MSR_IA32_FRED_CONFIG: + if (!kvm_cpu_cap_has(X86_FEATURE_FRED)) + return; + break; + case MSR_IA32_PL0_SSP: / I.e., MSR_IA32_FRED_SSP0 / + if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK) && + !kvm_cpu_cap_has(X86_FEATURE_FRED)) + return; + break; default: break; } diff --git a/arch/x86/kvm/msrs.h b/arch/x86/kvm/msrs.h index 7cc182a15b3b3..845d2cbe80e5e 100644 --- a/arch/x86/kvm/msrs.h +++ b/arch/x86/kvm/msrs.h @@ -31,7 +31,7 @@ static inline void kvm_pr_unimpl_rdmsr(struct kvm_vcpu vcpu, u32 msr) * associated feature that KVM supports for nested virtualization. / #define KVM_FIRST_EMULATED_VMX_MSR MSR_IA32_VMX_BASIC -#define KVM_LAST_EMULATED_VMX_MSR MSR_IA32_VMX_VMFUNC +#define KVM_LAST_EMULATED_VMX_MSR MSR_IA32_VMX_EXIT_CTLS2 / * KVM's internal, non-ABI indices for synthetic MSRs. The values themselves diff --git a/arch/x86/kvm/regs.c b/arch/x86/kvm/regs.c index 8f66438989e47..ba6c8856b4ffa 100644 --- a/arch/x86/kvm/regs.c +++ b/arch/x86/kvm/regs.c @@ -429,6 +429,13 @@ int kvm_set_cr4(struct kvm_vcpu vcpu, unsigned long cr4) return 1; } + / + * FRED can not be enabled when EFER.LMA=0. Note, MOV to CR4 outside + * 64-bit mode clears CR4[63:32] so only emulation can get here. + / + if ((cr4 & X86_CR4_FRED) && !is_long_mode(vcpu)) + return 1; + if ((cr4 & X86_CR4_CET) && !kvm_is_cr0_bit_set(vcpu, X86_CR0_WP)) return 1; diff --git a/arch/x86/kvm/regs.h b/arch/x86/kvm/regs.h index 447f0ec3e63e4..5db01693ad07a 100644 --- a/arch/x86/kvm/regs.h +++ b/arch/x86/kvm/regs.h @@ -8,7 +8,7 @@ #define KVM_POSSIBLE_CR4_GUEST_BITS \ (X86_CR4_PVI \| X86_CR4_DE \| X86_CR4_PCE \| X86_CR4_OSFXSR \ \| X86_CR4_OSXMMEXCPT \| X86_CR4_PGE \| X86_CR4_TSD \| X86_CR4_FSGSBASE \ - \| X86_CR4_CET) + \| X86_CR4_CET \| X86_CR4_FRED) #define X86_CR0_PDPTR_BITS (X86_CR0_CD \| X86_CR0_NW \| X86_CR0_PG) #define X86_CR4_TLBFLUSH_BITS (X86_CR4_PGE \| X86_CR4_PCIDE \| X86_CR4_PAE \| X86_CR4_SMEP) @@ -28,7 +28,7 @@ static_assert(!(KVM_POSSIBLE_CR0_GUEST_BITS & X86_CR0_PDPTR_BITS)); \| X86_CR4_OSXSAVE \| X86_CR4_SMEP \| X86_CR4_FSGSBASE \ \| X86_CR4_OSXMMEXCPT \| X86_CR4_LA57 \| X86_CR4_VMXE \ \| X86_CR4_SMAP \| X86_CR4_PKE \| X86_CR4_UMIP \ - \| X86_CR4_LAM_SUP \| X86_CR4_CET)) + \| X86_CR4_LAM_SUP \| X86_CR4_CET \| X86_CR4_FRED)) #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR) @@ -364,6 +364,21 @@ static __always_inline bool kvm_is_cr4_bit_set(struct kvm_vcpu vcpu, return !!kvm_read_cr4_bits(vcpu, cr4_bit); } +/* + * It's enough to check just CR4.FRED (X86_CR4_FRED) to tell if + * a vCPU is running with FRED enabled, because: + * 1) CR4.FRED can be set to 1 only _after_ IA32_EFER.LMA = 1. + * 2) To leave IA-32e mode, CR4.FRED must be cleared first. + / +static inline bool is_fred_enabled(struct kvm_vcpu vcpu) +{ +#ifdef CONFIG_X86_64 + return kvm_is_cr4_bit_set(vcpu, X86_CR4_FRED); +#else + return false; +#endif +} + static inline ulong kvm_read_cr3(struct kvm_vcpu vcpu) { if (!kvm_register_is_available(vcpu, VCPU_REG_CR3)) @@ -408,6 +423,8 @@ static inline bool __kvm_is_valid_cr4(struct kvm_vcpu vcpu, unsigned long cr4) if (!__cpu_has(__c, X86_FEATURE_SHSTK) && \ !__cpu_has(__c, X86_FEATURE_IBT)) \ __reserved_bits \|= X86_CR4_CET; \ + if (!__cpu_has(__c, X86_FEATURE_FRED)) \ + __reserved_bits \|= X86_CR4_FRED; \ __reserved_bits; \ }) diff --git a/arch/x86/kvm/smm.c b/arch/x86/kvm/smm.c index 656a38dad7e7a..a504e93fc5f5f 100644 --- a/arch/x86/kvm/smm.c +++ b/arch/x86/kvm/smm.c @@ -435,10 +435,10 @@ static int rsm_enter_protected_mode(struct kvm_vcpu vcpu, / * First enable PAE, long mode needs it before CR0.PG = 1 is set. - * Then enable protected mode. However, PCID cannot be enabled - * if EFER.LMA=0, so set it separately. + * Then enable protected mode. However, PCID and FRED cannot be + * enabled if EFER.LMA=0, so set them separately. / - bad = kvm_set_cr4(vcpu, cr4 & ~X86_CR4_PCIDE); + bad = kvm_set_cr4(vcpu, cr4 & ~(X86_CR4_PCIDE \| X86_CR4_FRED)); if (bad) return X86EMUL_UNHANDLEABLE; @@ -446,7 +446,7 @@ static int rsm_enter_protected_mode(struct kvm_vcpu vcpu, if (bad) return X86EMUL_UNHANDLEABLE; - if (cr4 & X86_CR4_PCIDE) { + if (cr4 & (X86_CR4_PCIDE \| X86_CR4_FRED)) { bad = kvm_set_cr4(vcpu, cr4); if (bad) return X86EMUL_UNHANDLEABLE; @@ -599,10 +599,14 @@ int emulator_leave_smm(struct x86_emulate_ctxt ctxt) struct kvm_segment cs_desc; unsigned long cr4; - / Zero CR4.PCIDE before CR0.PG. / + / + * Zero CR4.PCIDE and CR4.FRED before CR0.PG, as neither can + * be set while EFER.LMA is 0, and clearing CR0.PG clears + * EFER.LMA. + / cr4 = kvm_read_cr4(vcpu); - if (cr4 & X86_CR4_PCIDE) - kvm_set_cr4(vcpu, cr4 & ~X86_CR4_PCIDE); + if (cr4 & (X86_CR4_PCIDE \| X86_CR4_FRED)) + kvm_set_cr4(vcpu, cr4 & ~(X86_CR4_PCIDE \| X86_CR4_FRED)); / A 32-bit code segment is required to clear EFER.LMA. / memset(&cs_desc, 0, sizeof(cs_desc)); diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index ea647938a2a65..dd19c7b4e9047 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4392,7 +4392,7 @@ static void svm_complete_interrupts(struct kvm_vcpu vcpu) kvm_requeue_exception(vcpu, vector, exitintinfo & SVM_EXITINTINFO_VALID_ERR, - error_code); + error_code, false, 0); break; } case SVM_EXITINTINFO_TYPE_INTR: @@ -5518,6 +5518,9 @@ static __init void svm_set_cpu_caps(void) kvm_cpu_cap_clear(X86_FEATURE_IBT); + /* SVM FRED virtualization not implemented yet / + kvm_cpu_cap_clear(X86_FEATURE_FRED); + / CPUID 0x80000001 and 0x8000000A (SVM features) / if (nested) { kvm_cpu_cap_set(X86_FEATURE_SVM); diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilities.h index 810119167f798..74456f4533e9d 100644 --- a/arch/x86/kvm/vmx/capabilities.h +++ b/arch/x86/kvm/vmx/capabilities.h @@ -36,6 +36,7 @@ struct nested_vmx_msrs { u32 pinbased_ctls_high; u32 exit_ctls_low; u32 exit_ctls_high; + u64 secondary_exit_ctls; u32 entry_ctls_low; u32 entry_ctls_high; u32 misc_low; @@ -57,8 +58,9 @@ struct vmcs_config { u32 cpu_based_exec_ctrl; u32 cpu_based_2nd_exec_ctrl; u64 cpu_based_3rd_exec_ctrl; - u32 vmexit_ctrl; u32 vmentry_ctrl; + u32 vmexit_ctrl; + u64 vmexit_2nd_ctrl; u64 misc; struct nested_vmx_msrs nested; }; @@ -80,6 +82,11 @@ static inline bool cpu_has_vmx_basic_no_hw_errcode_cc(void) return vmcs_config.basic & VMX_BASIC_NO_HW_ERROR_CODE_CC; } +static inline bool cpu_has_vmx_nested_exception(void) +{ + return vmcs_config.basic & VMX_BASIC_NESTED_EXCEPTION; +} + static inline bool cpu_has_virtual_nmis(void) { return vmcs_config.pin_based_exec_ctrl & PIN_BASED_VIRTUAL_NMIS && @@ -149,6 +156,12 @@ static inline bool cpu_has_tertiary_exec_ctrls(void) CPU_BASED_ACTIVATE_TERTIARY_CONTROLS; } +static inline bool cpu_has_secondary_vmexit_ctrls(void) +{ + return vmcs_config.vmexit_ctrl & + VM_EXIT_ACTIVATE_SECONDARY_CONTROLS; +} + static inline bool cpu_has_vmx_virtualize_apic_accesses(void) { return vmcs_config.cpu_based_2nd_exec_ctrl & @@ -402,6 +415,11 @@ static inline bool vmx_pebs_supported(void) !enable_mediated_pmu; } +static inline bool cpu_has_vmx_fred(void) +{ + return vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_IA32_FRED; +} + static inline bool cpu_has_notify_vmexit(void) { return vmcs_config.cpu_based_2nd_exec_ctrl & diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 151873407abd3..4fad9728a2ea0 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -812,6 +812,15 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu vcpu, nested_vmx_merge_msr_bitmaps_rw(MSR_FS_BASE); nested_vmx_merge_msr_bitmaps_rw(MSR_GS_BASE); nested_vmx_merge_msr_bitmaps_rw(MSR_KERNEL_GS_BASE); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_RSP0); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_RSP1); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_RSP2); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_RSP3); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_STKLVLS); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_SSP1); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_SSP2); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_SSP3); + nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_FRED_CONFIG); #endif nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_SPEC_CTRL); nested_vmx_merge_msr_bitmaps_write(MSR_IA32_PRED_CMD); @@ -1356,9 +1365,11 @@ static int vmx_restore_vmx_basic(struct vcpu_vmx vmx, u64 data) const u64 feature_bits = VMX_BASIC_DUAL_MONITOR_TREATMENT \| VMX_BASIC_INOUT \| VMX_BASIC_TRUE_CTLS \| - VMX_BASIC_NO_HW_ERROR_CODE_CC; + VMX_BASIC_NO_HW_ERROR_CODE_CC \| + VMX_BASIC_NESTED_EXCEPTION; - const u64 reserved_bits = GENMASK_ULL(63, 57) \| + const u64 reserved_bits = GENMASK_ULL(63, 59) \| + BIT_ULL(57) \| GENMASK_ULL(47, 45) \| BIT_ULL(31); @@ -1596,6 +1607,11 @@ int vmx_set_vmx_msr(struct kvm_vcpu vcpu, u32 msr_index, u64 data) return -EINVAL; vmx->nested.msrs.vmfunc_controls = data; return 0; + case MSR_IA32_VMX_EXIT_CTLS2: + if (data & ~vmcs_config.nested.secondary_exit_ctls) + return -EINVAL; + vmx->nested.msrs.secondary_exit_ctls = data; + return 0; default: /* * The rest of the VMX capability MSRs do not support restore. @@ -1635,6 +1651,9 @@ int vmx_get_vmx_msr(struct nested_vmx_msrs msrs, u32 msr_index, u64 pdata) if (msr_index == MSR_IA32_VMX_EXIT_CTLS) pdata \|= VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR; break; + case MSR_IA32_VMX_EXIT_CTLS2: + pdata = msrs->secondary_exit_ctls; + break; case MSR_IA32_VMX_TRUE_ENTRY_CTLS: case MSR_IA32_VMX_ENTRY_CTLS: pdata = vmx_control_msr( @@ -2576,6 +2595,9 @@ static void prepare_vmcs02_early(struct vcpu_vmx vmx, struct loaded_vmcs vmcs0 exec_control &= ~VM_EXIT_LOAD_IA32_EFER; vm_exit_controls_set(vmx, exec_control); + if (exec_control & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS) + secondary_vm_exit_controls_set(vmx, __secondary_vm_exit_controls_get(vmcs01)); + / * Interrupt/Exception Fields / @@ -2588,6 +2610,8 @@ static void prepare_vmcs02_early(struct vcpu_vmx vmx, struct loaded_vmcs vmcs0 vmcs12->vm_entry_instruction_len); vmcs_write32(GUEST_INTERRUPTIBILITY_INFO, vmcs12->guest_interruptibility_info); + if (cpu_has_vmx_fred()) + vmcs_write64(INJECTED_EVENT_DATA, vmcs12->injected_event_data); vmx->loaded_vmcs->nmi_known_unmasked = !(vmcs12->guest_interruptibility_info & GUEST_INTR_STATE_NMI); } else { @@ -2621,6 +2645,30 @@ static void vmcs_write_cet_state(struct kvm_vcpu vcpu, u64 s_cet, } } +static void vmcs_read_fred_msrs(struct vmcs_fred_msrs msrs) +{ + msrs->fred_config = vmcs_read64(GUEST_IA32_FRED_CONFIG); + msrs->fred_rsp1 = vmcs_read64(GUEST_IA32_FRED_RSP1); + msrs->fred_rsp2 = vmcs_read64(GUEST_IA32_FRED_RSP2); + msrs->fred_rsp3 = vmcs_read64(GUEST_IA32_FRED_RSP3); + msrs->fred_stklvls = vmcs_read64(GUEST_IA32_FRED_STKLVLS); + msrs->fred_ssp1 = vmcs_read64(GUEST_IA32_FRED_SSP1); + msrs->fred_ssp2 = vmcs_read64(GUEST_IA32_FRED_SSP2); + msrs->fred_ssp3 = vmcs_read64(GUEST_IA32_FRED_SSP3); +} + +static void vmcs_write_fred_msrs(struct vmcs_fred_msrs msrs) +{ + vmcs_write64(GUEST_IA32_FRED_CONFIG, msrs->fred_config); + vmcs_write64(GUEST_IA32_FRED_RSP1, msrs->fred_rsp1); + vmcs_write64(GUEST_IA32_FRED_RSP2, msrs->fred_rsp2); + vmcs_write64(GUEST_IA32_FRED_RSP3, msrs->fred_rsp3); + vmcs_write64(GUEST_IA32_FRED_STKLVLS, msrs->fred_stklvls); + vmcs_write64(GUEST_IA32_FRED_SSP1, msrs->fred_ssp1); + vmcs_write64(GUEST_IA32_FRED_SSP2, msrs->fred_ssp2); + vmcs_write64(GUEST_IA32_FRED_SSP3, msrs->fred_ssp3); +} + static void prepare_vmcs02_rare(struct vcpu_vmx vmx, struct vmcs12 vmcs12) { struct hv_enlightened_vmcs hv_evmcs = nested_vmx_evmcs(vmx); @@ -2744,6 +2792,10 @@ static void prepare_vmcs02_rare(struct vcpu_vmx vmx, struct vmcs12 vmcs12) vmcs12->guest_ssp, vmcs12->guest_ssp_tbl); set_cr4_guest_host_mask(vmx); + + if (guest_cpu_cap_has(&vmx->vcpu, X86_FEATURE_FRED) && + nested_cpu_load_guest_fred_state(vmcs12)) + vmcs_write_fred_msrs(&vmcs12->guest_fred_msrs); } / @@ -2810,6 +2862,10 @@ static int prepare_vmcs02(struct kvm_vcpu vcpu, struct vmcs12 vmcs12, vmcs_write64(GUEST_IA32_PAT, vcpu->arch.pat); } + if (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED) && + (!vmx->vcpu.arch.nested_run_pending \|\| !nested_cpu_load_guest_fred_state(vmcs12))) + vmcs_write_fred_msrs(&vmx->nested.pre_vmenter_fred_msrs); + vcpu->arch.tsc_offset = kvm_calc_nested_tsc_offset( vcpu->arch.l1_tsc_offset, vmx_get_l2_tsc_offset(vcpu), @@ -3040,6 +3096,11 @@ static int nested_check_vm_exit_controls(struct kvm_vcpu vcpu, CC(nested_vmx_check_exit_msr_switch_controls(vcpu, vmcs12))) return -EINVAL; + if (nested_cpu_has_secondary_vm_exit_controls(vmcs12) && + CC(vmcs12->secondary_vm_exit_controls & + ~vmx->nested.msrs.secondary_exit_ctls)) + return -EINVAL; + return 0; } @@ -3050,6 +3111,8 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu vcpu, struct vmcs12 vmcs12) { struct vcpu_vmx vmx = to_vmx(vcpu); + bool fred_enabled = (vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE) && + (vmcs12->guest_cr4 & X86_CR4_FRED); if (CC(!vmx_control_verify(vmcs12->vm_entry_controls, vmx->nested.msrs.entry_ctls_low, @@ -3067,22 +3130,11 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu vcpu, u8 vector = intr_info & INTR_INFO_VECTOR_MASK; u32 intr_type = intr_info & INTR_INFO_INTR_TYPE_MASK; bool has_error_code = intr_info & INTR_INFO_DELIVER_CODE_MASK; + bool has_nested_exception = intr_info & INTR_INFO_NESTED_EXCEPTION_MASK; bool urg = nested_cpu_has2(vmcs12, SECONDARY_EXEC_UNRESTRICTED_GUEST); bool prot_mode = !urg \|\| vmcs12->guest_cr0 & X86_CR0_PE; - / VM-entry interruption-info field: interruption type / - if (CC(intr_type == INTR_TYPE_RESERVED) \|\| - CC(intr_type == INTR_TYPE_OTHER_EVENT && - !nested_cpu_supports_monitor_trap_flag(vcpu))) - return -EINVAL; - - / VM-entry interruption-info field: vector / - if (CC(intr_type == INTR_TYPE_NMI_INTR && vector != NMI_VECTOR) \|\| - CC(intr_type == INTR_TYPE_HARD_EXCEPTION && vector > 31) \|\| - CC(intr_type == INTR_TYPE_OTHER_EVENT && vector != 0)) - return -EINVAL; - / * Cannot deliver error code in real mode or if the interrupt * type is not hardware exception. For other cases, do the @@ -3106,8 +3158,28 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu vcpu, if (CC(intr_info & INTR_INFO_RESVD_BITS_MASK)) return -EINVAL; - / VM-entry instruction length / + if (CC(intr_type == INTR_TYPE_RESERVED)) + return -EINVAL; + + / + * Only for hardware exceptions and when the CPU enumerates + * VMX nested-exception support, bit 13 (indicating a nested + * exception) has value 1. Otherwise it is reserved. + / + if (CC(has_nested_exception && + (intr_type != INTR_TYPE_HARD_EXCEPTION \|\| + !nested_cpu_has_nested_exception(vcpu)))) + return -EINVAL; + switch (intr_type) { + case INTR_TYPE_NMI_INTR: + if (CC(vector != NMI_VECTOR)) + return -EINVAL; + break; + case INTR_TYPE_HARD_EXCEPTION: + if (CC(vector > 31)) + return -EINVAL; + break; case INTR_TYPE_SOFT_EXCEPTION: case INTR_TYPE_SOFT_INTR: case INTR_TYPE_PRIV_SW_EXCEPTION: @@ -3115,6 +3187,28 @@ static int nested_check_vm_entry_controls(struct kvm_vcpu vcpu, CC(vmcs12->vm_entry_instruction_len == 0 && CC(!nested_cpu_has_zero_length_injection(vcpu)))) return -EINVAL; + break; + case INTR_TYPE_OTHER_EVENT: + if (CC(vector > 2)) + return -EINVAL; + + switch (vector) { + case 0: + if (CC(!nested_cpu_supports_monitor_trap_flag(vcpu))) + return -EINVAL; + break; + case 1: + case 2: + if (CC(!fred_enabled)) + return -EINVAL; + if (CC(vmcs12->vm_entry_instruction_len > X86_MAX_INSTRUCTION_LENGTH)) + return -EINVAL; + if (CC(vmcs12->vm_entry_instruction_len == 0 && + !nested_cpu_has_zero_length_injection(vcpu))) + return -EINVAL; + break; + } + break; } } @@ -3201,9 +3295,27 @@ static int nested_vmx_check_host_state(struct kvm_vcpu vcpu, if (ia32e) { if (CC(!(vmcs12->host_cr4 & X86_CR4_PAE))) return -EINVAL; + if (nested_cpu_load_host_fred_state(vmcs12)) { + if (CC(vmcs12->host_ia32_fred_config & FRED_CONFIG_RESERVED) \|\| + CC(vmcs12->host_ia32_fred_rsp1 & GENMASK_ULL(5, 0)) \|\| + CC(vmcs12->host_ia32_fred_rsp2 & GENMASK_ULL(5, 0)) \|\| + CC(vmcs12->host_ia32_fred_rsp3 & GENMASK_ULL(5, 0)) \|\| + CC(vmcs12->host_ia32_fred_ssp1 & GENMASK_ULL(2, 0)) \|\| + CC(vmcs12->host_ia32_fred_ssp2 & GENMASK_ULL(2, 0)) \|\| + CC(vmcs12->host_ia32_fred_ssp3 & GENMASK_ULL(2, 0)) \|\| + CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_config & PAGE_MASK, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp1, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp2, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_rsp3, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp1, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp2, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->host_ia32_fred_ssp3, vcpu))) + return -EINVAL; + } } else { if (CC(vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE) \|\| CC(vmcs12->host_cr4 & X86_CR4_PCIDE) \|\| + CC(vmcs12->host_cr4 & X86_CR4_FRED) \|\| CC((vmcs12->host_rip) >> 32)) return -EINVAL; } @@ -3386,6 +3498,48 @@ static int nested_vmx_check_guest_state(struct kvm_vcpu vcpu, CC((vmcs12->guest_bndcfgs & MSR_IA32_BNDCFGS_RSVD)))) return -EINVAL; + if (ia32e) { + if (nested_cpu_load_guest_fred_state(vmcs12)) { + if (CC(vmcs12->guest_ia32_fred_config & FRED_CONFIG_RESERVED) \|\| + CC(vmcs12->guest_ia32_fred_rsp1 & GENMASK_ULL(5, 0)) \|\| + CC(vmcs12->guest_ia32_fred_rsp2 & GENMASK_ULL(5, 0)) \|\| + CC(vmcs12->guest_ia32_fred_rsp3 & GENMASK_ULL(5, 0)) \|\| + CC(vmcs12->guest_ia32_fred_ssp1 & GENMASK_ULL(2, 0)) \|\| + CC(vmcs12->guest_ia32_fred_ssp2 & GENMASK_ULL(2, 0)) \|\| + CC(vmcs12->guest_ia32_fred_ssp3 & GENMASK_ULL(2, 0)) \|\| + CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_config & PAGE_MASK, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp1, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp2, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_rsp3, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp1, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp2, vcpu)) \|\| + CC(is_noncanonical_msr_address(vmcs12->guest_ia32_fred_ssp3, vcpu))) + return -EINVAL; + } + if (vmcs12->guest_cr4 & X86_CR4_FRED) { + unsigned int ss_dpl = VMX_AR_DPL(vmcs12->guest_ss_ar_bytes); + + if (CC(ss_dpl == 1 \|\| ss_dpl == 2)) + return -EINVAL; + + switch (ss_dpl) { + case 0: + if (CC(!(vmcs12->guest_cs_ar_bytes & VMX_AR_L_MASK))) + return -EINVAL; + break; + case 3: + if (CC(vmcs12->guest_rflags & X86_EFLAGS_IOPL)) + return -EINVAL; + if (CC(vmcs12->guest_interruptibility_info & GUEST_INTR_STATE_STI)) + return -EINVAL; + break; + } + } + } else { + if (CC(vmcs12->guest_cr4 & X86_CR4_FRED)) + return -EINVAL; + } + if (vmcs12->vm_entry_controls & VM_ENTRY_LOAD_CET_STATE) { if (nested_vmx_check_cet_state_common(vcpu, vmcs12->guest_s_cet, vmcs12->guest_ssp, @@ -3614,7 +3768,8 @@ static int nested_vmx_check_permission(struct kvm_vcpu vcpu) } static void load_vmcs12_host_state(struct kvm_vcpu vcpu, - struct vmcs12 vmcs12); + struct vmcs12 vmcs12, + bool from_failed_vmentry); /* * If from_vmentry is false, this is being called from state restore (either RSM @@ -3664,6 +3819,10 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu vcpu, &vmx->nested.pre_vmenter_ssp, &vmx->nested.pre_vmenter_ssp_tbl); + if (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED) && + (!vmx->vcpu.arch.nested_run_pending \|\| !nested_cpu_load_guest_fred_state(vmcs12))) + vmcs_read_fred_msrs(&vmx->nested.pre_vmenter_fred_msrs); + / * Stash L1's CR3, so that in the event of a "late" VM-Fail, i.e. a * VM-Fail detected by hardware but not KVM, KVM can unwind its @@ -3778,7 +3937,7 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu vcpu, nested_put_vmcs12_pages(vcpu); - load_vmcs12_host_state(vcpu, vmcs12); + load_vmcs12_host_state(vcpu, vmcs12, true); vmcs12->vm_exit_reason = exit_reason.full; if (enable_shadow_vmcs \|\| nested_vmx_is_evmptr12_valid(vmx)) vmx->nested.need_vmcs12_to_shadow_sync = true; @@ -3967,6 +4126,8 @@ static void vmcs12_save_pending_event(struct kvm_vcpu vcpu, u32 idt_vectoring; unsigned int nr; + vmcs12->original_event_data = 0; + /* * Per the SDM, VM-Exits due to double and triple faults are never * considered to occur during event delivery, even if the double/triple @@ -4005,6 +4166,13 @@ static void vmcs12_save_pending_event(struct kvm_vcpu vcpu, vcpu->arch.exception.error_code; } + if ((vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE) && + (vmcs12->guest_cr4 & X86_CR4_FRED) && + (vcpu->arch.exception.is_nested)) + idt_vectoring \|= VECTORING_INFO_NESTED_EXCEPTION_MASK; + + vmcs12->original_event_data = vcpu->arch.exception.event_data; + vmcs12->idt_vectoring_info_field = idt_vectoring; } else if (vcpu->arch.nmi_injected) { vmcs12->idt_vectoring_info_field = @@ -4728,6 +4896,15 @@ static void sync_vmcs02_to_vmcs12(struct kvm_vcpu vcpu, struct vmcs12 vmcs12) vmcs_read_cet_state(&vmx->vcpu, &vmcs12->guest_s_cet, &vmcs12->guest_ssp, &vmcs12->guest_ssp_tbl); + + if (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) { + vmcs_read_fred_msrs(&vmx->nested.at_vmexit_fred_msrs); + + if (nested_cpu_save_guest_fred_state(vmcs12)) + memcpy(&vmcs12->guest_fred_msrs, + &vmx->nested.at_vmexit_fred_msrs, + sizeof(struct vmcs_fred_msrs)); + } } / @@ -4772,6 +4949,21 @@ static void prepare_vmcs12(struct kvm_vcpu vcpu, struct vmcs12 vmcs12, vmcs12->vm_exit_intr_info = exit_intr_info; vmcs12->vm_exit_instruction_len = exit_insn_len; + + /* + * When there is a valid original event, the exiting event is a nested + * event during delivery of the earlier original event. + * + * FRED event delivery reflects this relationship by setting the value + * of the nested exception bit of VM-exit interruption information + * (aka exiting-event identification) to that of the valid bit of the + * IDT-vectoring information (aka original-event identification). + / + if ((vmcs12->idt_vectoring_info_field & VECTORING_INFO_VALID_MASK) && + (vmcs12->vm_entry_controls & VM_ENTRY_IA32E_MODE) && + (vmcs12->guest_cr4 & X86_CR4_FRED)) + vmcs12->vm_exit_intr_info \|= INTR_INFO_NESTED_EXCEPTION_MASK; + vmcs12->vmx_instruction_info = vmcs_read32(VMX_INSTRUCTION_INFO); / @@ -4798,8 +4990,10 @@ static void prepare_vmcs12(struct kvm_vcpu vcpu, struct vmcs12 vmcs12, * This function should be called when the active VMCS is L1's (vmcs01). / static void load_vmcs12_host_state(struct kvm_vcpu vcpu, - struct vmcs12 vmcs12) + struct vmcs12 vmcs12, + bool from_failed_vmentry) { + struct vcpu_vmx vmx = to_vmx(vcpu); enum vm_entry_failure_code ignored; struct kvm_segment seg; @@ -4874,6 +5068,14 @@ static void load_vmcs12_host_state(struct kvm_vcpu vcpu, WARN_ON_ONCE(__kvm_emulate_msr_write(vcpu, MSR_CORE_PERF_GLOBAL_CTRL, vmcs12->host_ia32_perf_global_ctrl)); + if (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) { + if (nested_cpu_load_host_fred_state(vmcs12)) { + vmcs_write_fred_msrs(&vmcs12->host_fred_msrs); + } else if (!from_failed_vmentry) { + vmcs_write_fred_msrs(&vmx->nested.at_vmexit_fred_msrs); + } + } + /* Set L1 segment info according to Intel SDM 27.5.2 Loading Host Segment and Descriptor-Table Registers / seg = (struct kvm_segment) { @@ -5193,7 +5395,7 @@ void __nested_vmx_vmexit(struct kvm_vcpu vcpu, u32 vm_exit_reason, vmcs12->vm_exit_intr_error_code, KVM_ISA_VMX); - load_vmcs12_host_state(vcpu, vmcs12); + load_vmcs12_host_state(vcpu, vmcs12, false); /* * Process events if an injectable IRQ or NMI is pending, even @@ -7152,7 +7354,8 @@ static void nested_vmx_setup_exit_ctls(struct vmcs_config vmcs_conf, VM_EXIT_HOST_ADDR_SPACE_SIZE \| #endif VM_EXIT_LOAD_IA32_PAT \| VM_EXIT_SAVE_IA32_PAT \| - VM_EXIT_CLEAR_BNDCFGS \| VM_EXIT_LOAD_CET_STATE; + VM_EXIT_CLEAR_BNDCFGS \| VM_EXIT_LOAD_CET_STATE \| + VM_EXIT_ACTIVATE_SECONDARY_CONTROLS; msrs->exit_ctls_high \|= VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR \| VM_EXIT_LOAD_IA32_EFER \| VM_EXIT_SAVE_IA32_EFER \| @@ -7165,6 +7368,12 @@ static void nested_vmx_setup_exit_ctls(struct vmcs_config vmcs_conf, /* We support free control of debug control saving. / msrs->exit_ctls_low &= ~VM_EXIT_SAVE_DEBUG_CONTROLS; + + if (msrs->exit_ctls_high & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS) { + msrs->secondary_exit_ctls = vmcs_conf->vmexit_2nd_ctrl; + msrs->secondary_exit_ctls &= SECONDARY_VM_EXIT_SAVE_IA32_FRED \| + SECONDARY_VM_EXIT_LOAD_IA32_FRED; + } } static void nested_vmx_setup_entry_ctls(struct vmcs_config vmcs_conf, @@ -7179,7 +7388,7 @@ static void nested_vmx_setup_entry_ctls(struct vmcs_config vmcs_conf, VM_ENTRY_IA32E_MODE \| #endif VM_ENTRY_LOAD_IA32_PAT \| VM_ENTRY_LOAD_BNDCFGS \| - VM_ENTRY_LOAD_CET_STATE; + VM_ENTRY_LOAD_CET_STATE \| VM_ENTRY_LOAD_IA32_FRED; msrs->entry_ctls_high \|= (VM_ENTRY_ALWAYSON_WITHOUT_TRUE_MSR \| VM_ENTRY_LOAD_IA32_EFER \| VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL); @@ -7343,6 +7552,8 @@ static void nested_vmx_setup_basic(struct nested_vmx_msrs msrs) msrs->basic \|= VMX_BASIC_INOUT; if (cpu_has_vmx_basic_no_hw_errcode_cc()) msrs->basic \|= VMX_BASIC_NO_HW_ERROR_CODE_CC; + if (cpu_has_vmx_nested_exception()) + msrs->basic \|= VMX_BASIC_NESTED_EXCEPTION; } static void nested_vmx_setup_cr_fixed(struct nested_vmx_msrs msrs) diff --git a/arch/x86/kvm/vmx/nested.h b/arch/x86/kvm/vmx/nested.h index c6de848bd9ced..5511218241281 100644 --- a/arch/x86/kvm/vmx/nested.h +++ b/arch/x86/kvm/vmx/nested.h @@ -247,6 +247,11 @@ static inline bool nested_cpu_has_save_preemption_timer(struct vmcs12 vmcs12) VM_EXIT_SAVE_VMX_PREEMPTION_TIMER; } +static inline bool nested_cpu_has_secondary_vm_exit_controls(struct vmcs12 vmcs12) +{ + return vmcs12->vm_exit_controls & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS; +} + static inline bool nested_exit_on_nmi(struct kvm_vcpu vcpu) { return nested_cpu_has_nmi_exiting(get_vmcs12(vcpu)); @@ -267,6 +272,23 @@ static inline bool nested_cpu_has_encls_exit(struct vmcs12 vmcs12) return nested_cpu_has2(vmcs12, SECONDARY_EXEC_ENCLS_EXITING); } +static inline bool nested_cpu_load_guest_fred_state(struct vmcs12 vmcs12) +{ + return vmcs12->vm_entry_controls & VM_ENTRY_LOAD_IA32_FRED; +} + +static inline bool nested_cpu_save_guest_fred_state(struct vmcs12 vmcs12) +{ + return nested_cpu_has_secondary_vm_exit_controls(vmcs12) && + vmcs12->secondary_vm_exit_controls & SECONDARY_VM_EXIT_SAVE_IA32_FRED; +} + +static inline bool nested_cpu_load_host_fred_state(struct vmcs12 vmcs12) +{ + return nested_cpu_has_secondary_vm_exit_controls(vmcs12) && + vmcs12->secondary_vm_exit_controls & SECONDARY_VM_EXIT_LOAD_IA32_FRED; +} + /* * if fixed0[i] == 1: val[i] must be 1 * if fixed1[i] == 0: val[i] must be 0 @@ -312,6 +334,11 @@ static inline bool nested_cpu_has_no_hw_errcode_cc(struct kvm_vcpu vcpu) return to_vmx(vcpu)->nested.msrs.basic & VMX_BASIC_NO_HW_ERROR_CODE_CC; } +static inline bool nested_cpu_has_nested_exception(struct kvm_vcpu vcpu) +{ + return to_vmx(vcpu)->nested.msrs.basic & VMX_BASIC_NESTED_EXCEPTION; +} + /* No difference in the restrictions on guest and host CR4 in VMX operation. / #define nested_guest_cr4_valid nested_cr4_valid #define nested_host_cr4_valid nested_cr4_valid diff --git a/arch/x86/kvm/vmx/vmcs.h b/arch/x86/kvm/vmx/vmcs.h index 1f16ddeae9cb1..58af10f73789e 100644 --- a/arch/x86/kvm/vmx/vmcs.h +++ b/arch/x86/kvm/vmx/vmcs.h @@ -45,12 +45,24 @@ struct vmcs_host_state { struct vmcs_controls_shadow { u32 vm_entry; u32 vm_exit; + u64 secondary_vm_exit; u32 pin; u32 exec; u32 secondary_exec; u64 tertiary_exec; }; +struct vmcs_fred_msrs { + u64 fred_config; + u64 fred_rsp1; + u64 fred_rsp2; + u64 fred_rsp3; + u64 fred_stklvls; + u64 fred_ssp1; + u64 fred_ssp2; + u64 fred_ssp3; +}; + / * Track a VMCS that may be loaded on a certain CPU. If it is (cpu!=-1), also * remember whether it was VMLAUNCHed, and maintain a linked list of all VMCSs diff --git a/arch/x86/kvm/vmx/vmcs12.c b/arch/x86/kvm/vmx/vmcs12.c index 1ebe67c384ad4..67c730e314655 100644 --- a/arch/x86/kvm/vmx/vmcs12.c +++ b/arch/x86/kvm/vmx/vmcs12.c @@ -66,6 +66,25 @@ static const u16 kvm_supported_vmcs12_field_offsets[] __initconst = { FIELD64(HOST_IA32_PAT, host_ia32_pat), FIELD64(HOST_IA32_EFER, host_ia32_efer), FIELD64(HOST_IA32_PERF_GLOBAL_CTRL, host_ia32_perf_global_ctrl), + FIELD64(SECONDARY_VM_EXIT_CONTROLS, secondary_vm_exit_controls), + FIELD64(INJECTED_EVENT_DATA, injected_event_data), + FIELD64(ORIGINAL_EVENT_DATA, original_event_data), + FIELD64(GUEST_IA32_FRED_CONFIG, guest_ia32_fred_config), + FIELD64(GUEST_IA32_FRED_RSP1, guest_ia32_fred_rsp1), + FIELD64(GUEST_IA32_FRED_RSP2, guest_ia32_fred_rsp2), + FIELD64(GUEST_IA32_FRED_RSP3, guest_ia32_fred_rsp3), + FIELD64(GUEST_IA32_FRED_STKLVLS, guest_ia32_fred_stklvls), + FIELD64(GUEST_IA32_FRED_SSP1, guest_ia32_fred_ssp1), + FIELD64(GUEST_IA32_FRED_SSP2, guest_ia32_fred_ssp2), + FIELD64(GUEST_IA32_FRED_SSP3, guest_ia32_fred_ssp3), + FIELD64(HOST_IA32_FRED_CONFIG, host_ia32_fred_config), + FIELD64(HOST_IA32_FRED_RSP1, host_ia32_fred_rsp1), + FIELD64(HOST_IA32_FRED_RSP2, host_ia32_fred_rsp2), + FIELD64(HOST_IA32_FRED_RSP3, host_ia32_fred_rsp3), + FIELD64(HOST_IA32_FRED_STKLVLS, host_ia32_fred_stklvls), + FIELD64(HOST_IA32_FRED_SSP1, host_ia32_fred_ssp1), + FIELD64(HOST_IA32_FRED_SSP2, host_ia32_fred_ssp2), + FIELD64(HOST_IA32_FRED_SSP3, host_ia32_fred_ssp3), FIELD(PIN_BASED_VM_EXEC_CONTROL, pin_based_vm_exec_control), FIELD(CPU_BASED_VM_EXEC_CONTROL, cpu_based_vm_exec_control), FIELD(EXCEPTION_BITMAP, exception_bitmap), @@ -203,6 +222,9 @@ static __init bool cpu_has_vmcs12_field(unsigned int idx) case HOST_SSP: case HOST_INTR_SSP_TABLE: return cpu_has_load_cet_ctrl(); + VMCS12_CASE64(ORIGINAL_EVENT_DATA): + VMCS12_CASE64(INJECTED_EVENT_DATA): + return cpu_has_vmx_fred(); /* KVM always emulates PML and the VMX preemption timer in software. / case GUEST_PML_INDEX: diff --git a/arch/x86/kvm/vmx/vmcs12.h b/arch/x86/kvm/vmx/vmcs12.h index 21cd1b75e4fdf..c4c5cd3722a5b 100644 --- a/arch/x86/kvm/vmx/vmcs12.h +++ b/arch/x86/kvm/vmx/vmcs12.h @@ -71,7 +71,7 @@ struct __packed vmcs12 { u64 pml_address; u64 encls_exiting_bitmap; u64 tsc_multiplier; - u64 padding64[1]; / room for future expansion / + u64 secondary_vm_exit_controls; / * To allow migration of L1 (complete with its L2 guests) between * machines of different natural widths (32 or 64 bit), we cannot have @@ -191,6 +191,27 @@ struct __packed vmcs12 { u16 host_gs_selector; u16 host_tr_selector; u16 guest_pml_index; + u16 padding16[1]; /* align to 64-bit boundary / + struct vmcs_fred_msrs guest_fred_msrs; +#define guest_ia32_fred_config guest_fred_msrs.fred_config +#define guest_ia32_fred_rsp1 guest_fred_msrs.fred_rsp1 +#define guest_ia32_fred_rsp2 guest_fred_msrs.fred_rsp2 +#define guest_ia32_fred_rsp3 guest_fred_msrs.fred_rsp3 +#define guest_ia32_fred_stklvls guest_fred_msrs.fred_stklvls +#define guest_ia32_fred_ssp1 guest_fred_msrs.fred_ssp1 +#define guest_ia32_fred_ssp2 guest_fred_msrs.fred_ssp2 +#define guest_ia32_fred_ssp3 guest_fred_msrs.fred_ssp3 + struct vmcs_fred_msrs host_fred_msrs; +#define host_ia32_fred_config host_fred_msrs.fred_config +#define host_ia32_fred_rsp1 host_fred_msrs.fred_rsp1 +#define host_ia32_fred_rsp2 host_fred_msrs.fred_rsp2 +#define host_ia32_fred_rsp3 host_fred_msrs.fred_rsp3 +#define host_ia32_fred_stklvls host_fred_msrs.fred_stklvls +#define host_ia32_fred_ssp1 host_fred_msrs.fred_ssp1 +#define host_ia32_fred_ssp2 host_fred_msrs.fred_ssp2 +#define host_ia32_fred_ssp3 host_fred_msrs.fred_ssp3 + u64 injected_event_data; + u64 original_event_data; }; / @@ -261,6 +282,7 @@ static inline void vmx_check_vmcs12_offsets(void) CHECK_OFFSET(pml_address, 312); CHECK_OFFSET(encls_exiting_bitmap, 320); CHECK_OFFSET(tsc_multiplier, 328); + CHECK_OFFSET(secondary_vm_exit_controls, 336); CHECK_OFFSET(cr0_guest_host_mask, 344); CHECK_OFFSET(cr4_guest_host_mask, 352); CHECK_OFFSET(cr0_read_shadow, 360); @@ -372,6 +394,24 @@ static inline void vmx_check_vmcs12_offsets(void) CHECK_OFFSET(host_gs_selector, 992); CHECK_OFFSET(host_tr_selector, 994); CHECK_OFFSET(guest_pml_index, 996); + CHECK_OFFSET(guest_ia32_fred_config, 1000); + CHECK_OFFSET(guest_ia32_fred_rsp1, 1008); + CHECK_OFFSET(guest_ia32_fred_rsp2, 1016); + CHECK_OFFSET(guest_ia32_fred_rsp3, 1024); + CHECK_OFFSET(guest_ia32_fred_stklvls, 1032); + CHECK_OFFSET(guest_ia32_fred_ssp1, 1040); + CHECK_OFFSET(guest_ia32_fred_ssp2, 1048); + CHECK_OFFSET(guest_ia32_fred_ssp3, 1056); + CHECK_OFFSET(host_ia32_fred_config, 1064); + CHECK_OFFSET(host_ia32_fred_rsp1, 1072); + CHECK_OFFSET(host_ia32_fred_rsp2, 1080); + CHECK_OFFSET(host_ia32_fred_rsp3, 1088); + CHECK_OFFSET(host_ia32_fred_stklvls, 1096); + CHECK_OFFSET(host_ia32_fred_ssp1, 1104); + CHECK_OFFSET(host_ia32_fred_ssp2, 1112); + CHECK_OFFSET(host_ia32_fred_ssp3, 1120); + CHECK_OFFSET(injected_event_data, 1128); + CHECK_OFFSET(original_event_data, 1136); } extern u16 vmcs12_field_offsets[] __ro_after_init; diff --git a/arch/x86/kvm/vmx/vmcs_shadow_fields.h b/arch/x86/kvm/vmx/vmcs_shadow_fields.h index 67e821c2be6d5..bdd56f3ca6c30 100644 --- a/arch/x86/kvm/vmx/vmcs_shadow_fields.h +++ b/arch/x86/kvm/vmx/vmcs_shadow_fields.h @@ -74,6 +74,10 @@ SHADOW_FIELD_RW(HOST_GS_BASE, host_gs_base) /* 64-bit / SHADOW_FIELD_RO(GUEST_PHYSICAL_ADDRESS, guest_physical_address) SHADOW_FIELD_RO(GUEST_PHYSICAL_ADDRESS_HIGH, guest_physical_address) +SHADOW_FIELD_RO(ORIGINAL_EVENT_DATA, original_event_data) +SHADOW_FIELD_RO(ORIGINAL_EVENT_DATA_HIGH, original_event_data) +SHADOW_FIELD_RW(INJECTED_EVENT_DATA, injected_event_data) +SHADOW_FIELD_RW(INJECTED_EVENT_DATA_HIGH, injected_event_data) #endif #undef SHADOW_FIELD_RO diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 504630f0eb40f..f673cd4e8956e 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1395,6 +1395,9 @@ void vmx_prepare_switch_to_guest(struct kvm_vcpu vcpu) } wrmsrq(MSR_KERNEL_GS_BASE, vmx->msr_guest_kernel_gs_base); + + if (guest_cpu_cap_has(vcpu, X86_FEATURE_FRED)) + wrmsrns(MSR_IA32_FRED_RSP0, vmx->msr_guest_fred_rsp0); #else savesegment(fs, fs_sel); savesegment(gs, gs_sel); @@ -1439,6 +1442,16 @@ static void vmx_prepare_switch_to_host(struct vcpu_vmx vmx) invalidate_tss_limit(); #ifdef CONFIG_X86_64 wrmsrq(MSR_KERNEL_GS_BASE, vmx->vt.msr_host_kernel_gs_base); + + if (guest_cpu_cap_has(&vmx->vcpu, X86_FEATURE_FRED)) { + vmx->msr_guest_fred_rsp0 = read_msr(MSR_IA32_FRED_RSP0); + / + * Synchronize the current value in hardware to the kernel's + * local cache. The desired host RSP0 will be set when the + * CPU exits to userspace (RSP0 is a per-task value). + / + fred_sync_rsp0(vmx->msr_guest_fred_rsp0); + } #endif load_fixmap_gdt(raw_smp_processor_id()); vmx->vt.guest_state_loaded = false; @@ -1476,6 +1489,21 @@ static void vmx_write_guest_kernel_gs_base(struct vcpu_vmx vmx, u64 data) vmx_write_guest_host_msr(vmx, MSR_KERNEL_GS_BASE, data, &vmx->msr_guest_kernel_gs_base); } + +static u64 vmx_read_guest_fred_rsp0(struct vcpu_vmx vmx) +{ + return vmx_read_guest_host_msr(vmx, MSR_IA32_FRED_RSP0, + &vmx->msr_guest_fred_rsp0); +} + +static void vmx_write_guest_fred_rsp0(struct vcpu_vmx vmx, u64 data) +{ + vmx_write_guest_host_msr(vmx, MSR_IA32_FRED_RSP0, data, + &vmx->msr_guest_fred_rsp0); +} +#else +/* Make sure it builds on 32-bit / +static u64 vmx_read_guest_fred_rsp0(struct vcpu_vmx vmx) { return 0; } #endif static void grow_ple_window(struct kvm_vcpu vcpu) @@ -1562,6 +1590,12 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu vcpu, int cpu) (unsigned long)(cpu_entry_stack(cpu) + 1)); } + if (cpu_feature_enabled(X86_FEATURE_FRED) && kvm_cpu_cap_has(X86_FEATURE_FRED)) { + vmcs_write64(HOST_IA32_FRED_RSP1, this_cpu_fred_rsp(FRED_STACK_LEVEL_1)); + vmcs_write64(HOST_IA32_FRED_RSP2, this_cpu_fred_rsp(FRED_STACK_LEVEL_2)); + vmcs_write64(HOST_IA32_FRED_RSP3, this_cpu_fred_rsp(FRED_STACK_LEVEL_3)); + } + vmx->loaded_vmcs->cpu = cpu; } } @@ -1973,12 +2007,41 @@ void vmx_inject_exception(struct kvm_vcpu vcpu) WARN_ON_ONCE(vmx->vt.emulation_required); + / + * Event data is strictly required only for #PF, #DB, and #NM (due to + * extended feature disable) hardware exceptions under FRED, making it + * straightforward to correctly populate for just these vectors. + * + * Conversely, trying to guarantee that event data is cleanly managed or + * zeroed out across all other exception paths is highly complicated + * and error-prone. A blanket fallback to 0 handles those cases safely. + * + * Therefore, set INJECTED_EVENT_DATA only if all the following are true: + * - Guest has FRED enabled. + * - Exception type is a hardware exception. + * - Exception vector is #PF, #DB, or #NM. + * + * Otherwise, clear it to 0 to prevent stale data leakage. + / if (kvm_exception_is_soft(ex->vector)) { vmcs_write32(VM_ENTRY_INSTRUCTION_LEN, vmx->vcpu.arch.event_exit_inst_len); intr_info \|= INTR_TYPE_SOFT_EXCEPTION; - } else + if (is_fred_enabled(vcpu)) + vmcs_write64(INJECTED_EVENT_DATA, 0); + } else { intr_info \|= INTR_TYPE_HARD_EXCEPTION; + if (is_fred_enabled(vcpu)) { + if (ex->is_nested) + intr_info \|= INTR_INFO_NESTED_EXCEPTION_MASK; + if (ex->vector == PF_VECTOR \|\| + ex->vector == DB_VECTOR \|\| + ex->vector == NM_VECTOR) + vmcs_write64(INJECTED_EVENT_DATA, ex->event_data); + else + vmcs_write64(INJECTED_EVENT_DATA, 0); + } + } vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr_info); @@ -2121,6 +2184,27 @@ int vmx_get_feature_msr(u32 msr, u64 data) } } +#ifdef CONFIG_X86_64 +static const u32 fred_msr_vmcs_fields[] = { + GUEST_IA32_FRED_RSP1, + GUEST_IA32_FRED_RSP2, + GUEST_IA32_FRED_RSP3, + GUEST_IA32_FRED_STKLVLS, + GUEST_IA32_FRED_SSP1, + GUEST_IA32_FRED_SSP2, + GUEST_IA32_FRED_SSP3, + GUEST_IA32_FRED_CONFIG, +}; + +static_assert(MSR_IA32_FRED_CONFIG - MSR_IA32_FRED_RSP1 == + ARRAY_SIZE(fred_msr_vmcs_fields) - 1); + +static u32 fred_msr_to_vmcs(u32 msr) +{ + return fred_msr_vmcs_fields[msr - MSR_IA32_FRED_RSP1]; +} +#endif + /* * Reads an msr value (of 'msr_info->index') into 'msr_info->data'. * Returns 0 on success, non-0 otherwise. @@ -2143,6 +2227,12 @@ int vmx_get_msr(struct kvm_vcpu vcpu, struct msr_data msr_info) case MSR_KERNEL_GS_BASE: msr_info->data = vmx_read_guest_kernel_gs_base(vmx); break; + case MSR_IA32_FRED_RSP0: + msr_info->data = vmx_read_guest_fred_rsp0(vmx); + break; + case MSR_IA32_FRED_RSP1 ... MSR_IA32_FRED_CONFIG: + msr_info->data = vmcs_read64(fred_msr_to_vmcs(msr_info->index)); + break; #endif case MSR_EFER: return kvm_get_msr_common(vcpu, msr_info); @@ -2375,6 +2465,12 @@ int vmx_set_msr(struct kvm_vcpu vcpu, struct msr_data msr_info) vmx_update_exception_bitmap(vcpu); } break; + case MSR_IA32_FRED_RSP0: + vmx_write_guest_fred_rsp0(vmx, data); + break; + case MSR_IA32_FRED_RSP1 ... MSR_IA32_FRED_CONFIG: + vmcs_write64(fred_msr_to_vmcs(msr_index), data); + break; #endif case MSR_IA32_SYSENTER_CS: if (is_guest_mode(vcpu)) @@ -2751,8 +2847,9 @@ static int setup_vmcs_config(struct vmcs_config vmcs_conf, u32 _cpu_based_exec_control = 0; u32 _cpu_based_2nd_exec_control = 0; u64 _cpu_based_3rd_exec_control = 0; - u32 _vmexit_control = 0; u32 _vmentry_control = 0; + u32 _vmexit_control = 0; + u64 _vmexit2_control = 0; u64 basic_msr; u64 misc_msr; @@ -2773,6 +2870,14 @@ static int setup_vmcs_config(struct vmcs_config vmcs_conf, { VM_ENTRY_LOAD_CET_STATE, VM_EXIT_LOAD_CET_STATE }, }; + struct { + u32 entry_control; + u64 exit_control; + } const vmcs_entry_exit2_pairs[] = { + { VM_ENTRY_LOAD_IA32_FRED, + SECONDARY_VM_EXIT_SAVE_IA32_FRED \| SECONDARY_VM_EXIT_LOAD_IA32_FRED }, + }; + memset(vmcs_conf, 0, sizeof(vmcs_conf)); if (adjust_vmx_controls(KVM_REQUIRED_VMX_CPU_BASED_VM_EXEC_CONTROL, @@ -2870,10 +2975,19 @@ static int setup_vmcs_config(struct vmcs_config vmcs_conf, &_vmentry_control)) return -EIO; + if (_vmexit_control & VM_EXIT_ACTIVATE_SECONDARY_CONTROLS) + _vmexit2_control = + adjust_vmx_controls64(KVM_OPTIONAL_VMX_SECONDARY_VM_EXIT_CONTROLS, + MSR_IA32_VMX_EXIT_CTLS2); + if (vmx_check_entry_exit_pairs(vmcs_entry_exit_pairs, _vmentry_control, _vmexit_control)) return -EIO; + if (vmx_check_entry_exit_pairs(vmcs_entry_exit2_pairs, + _vmentry_control, _vmexit2_control)) + return -EIO; + /* * Some cpus support VM_{ENTRY,EXIT}_IA32_PERF_GLOBAL_CTRL but they * can't be used due to an errata where VM Exit may incorrectly clear @@ -2922,8 +3036,9 @@ static int setup_vmcs_config(struct vmcs_config vmcs_conf, vmcs_conf->cpu_based_exec_ctrl = _cpu_based_exec_control; vmcs_conf->cpu_based_2nd_exec_ctrl = _cpu_based_2nd_exec_control; vmcs_conf->cpu_based_3rd_exec_ctrl = _cpu_based_3rd_exec_control; - vmcs_conf->vmexit_ctrl = _vmexit_control; vmcs_conf->vmentry_ctrl = _vmentry_control; + vmcs_conf->vmexit_ctrl = _vmexit_control; + vmcs_conf->vmexit_2nd_ctrl = _vmexit2_control; vmcs_conf->misc = misc_msr; #if IS_ENABLED(CONFIG_HYPERV) @@ -4279,6 +4394,72 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu vcpu) MSR_TYPE_RW, intercept); } +static void vmx_set_intercept_for_fred_msr(struct kvm_vcpu vcpu) +{ + bool intercept = !guest_cpu_cap_has(vcpu, X86_FEATURE_FRED); + + if (!kvm_cpu_cap_has(X86_FEATURE_FRED)) + return; + + / + * Because the following eight FRED MSRs, + * MSR_IA32_FRED_RSP[123], MSR_IA32_FRED_STKLVLS, + * MSR_IA32_FRED_SSP[123], MSR_IA32_FRED_CONFIG, + * are used by the kernel itself to take an exception at any time, they + * should be context-switched by Intel VT-x automatically in order to + * preserve the FRED architectural invariant that there should NEVER be + * a "gap" during which it is unsafe to take an exception. + * + * KVM leverages Intel VT-x hardware to automatically context switch the + * eight FRED MSRs using: + * + * 1) Dedicated host and guest VMCS fields for each MSR. + * + * 2) VM-entry/exit controls to manage the automated loading and saving + * of the eight FRED MSRs. + * + * Consequently, passing these MSRs through to the guest would only add + * unnecessary handling code without benefit. + / + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP1, MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP2, MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP3, MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_STKLVLS, MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP1, MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP2, MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_SSP3, MSR_TYPE_RW, intercept); + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_CONFIG, MSR_TYPE_RW, intercept); + + / + * MSR_IA32_FRED_RSP0 and MSR_IA32_PL0_SSP (aka MSR_IA32_FRED_SSP0) are + * designed for event delivery while executing in userspace. Since KVM + * operates entirely in kernel mode (CPL is always 0 after any VM exit), + * it can safely retain and operate with guest-defined values for these + * MSRs. + * + * Disabling interception of the two MSRs offers two advantages: + * 1) Simplicity: Eliminates dedicated MSR handling code. + * 2) Performance: Avoids frequent VM-exits since the two MSRs are + * per user thread variables and frequently accessed. + * + * MSR_IA32_PL0_SSP (aka MSR_IA32_FRED_SSP0) is part of CET supervisor + * state, but all four FRED SSP MSRs are architecturally visible on any + * processor that enumerates FRED. Even if CET is absent, these MSRs + * remain accessible via RDMSR/WRMSR, though FRED transitions will not + * use them. + * + * Intercept MSR_IA32_PL0_SSP if CET shadow stacks are unsupported (even + * with FRED present). Since this MSR is rarely accessed and ignored by + * XSAVES in this configuration, interception avoids the overhead of + * manually context switching the hardware MSR during vcpu_load/put. + * + * This behavior is consistent with the current setup in + * vmx_recalc_msr_intercepts(), so no change is needed to the interception + * logic for MSR_IA32_PL0_SSP. + / + vmx_set_intercept_for_msr(vcpu, MSR_IA32_FRED_RSP0, MSR_TYPE_RW, intercept); +} + static void vmx_recalc_msr_intercepts(struct kvm_vcpu vcpu) { bool intercept; @@ -4346,6 +4527,7 @@ static void vmx_recalc_msr_intercepts(struct kvm_vcpu vcpu) } vmx_recalc_pmu_msr_intercepts(vcpu); + vmx_set_intercept_for_fred_msr(vcpu); / * x2APIC and LBR MSR intercepts are modified on-demand and cannot be @@ -4481,6 +4663,17 @@ void vmx_set_constant_host_state(struct vcpu_vmx vmx) / vmcs_write16(HOST_DS_SELECTOR, 0); vmcs_write16(HOST_ES_SELECTOR, 0); + + if (cpu_feature_enabled(X86_FEATURE_FRED) && kvm_cpu_cap_has(X86_FEATURE_FRED)) { + /* FRED CONFIG and STKLVLS are the same on all CPUs / + vmcs_write64(HOST_IA32_FRED_CONFIG, kvm_host.fred_config); + vmcs_write64(HOST_IA32_FRED_STKLVLS, kvm_host.fred_stklvls); + + / Linux doesn't support kernel shadow stacks, thus SSPs are 0s / + vmcs_write64(HOST_IA32_FRED_SSP1, 0); + vmcs_write64(HOST_IA32_FRED_SSP2, 0); + vmcs_write64(HOST_IA32_FRED_SSP3, 0); + } #else vmcs_write16(HOST_DS_SELECTOR, __KERNEL_DS); / 22.2.4 / vmcs_write16(HOST_ES_SELECTOR, __KERNEL_DS); / 22.2.4 / @@ -4617,6 +4810,11 @@ static u32 vmx_get_initial_vmexit_ctrl(void) VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL); } +static u64 vmx_secondary_vmexit_ctrl(void) +{ + return vmcs_config.vmexit_2nd_ctrl; +} + void vmx_refresh_apicv_exec_ctrl(struct kvm_vcpu vcpu) { struct vcpu_vmx vmx = to_vmx(vcpu); @@ -4960,6 +5158,9 @@ static void init_vmcs(struct vcpu_vmx vmx) vm_exit_controls_set(vmx, vmx_get_initial_vmexit_ctrl()); + if (cpu_has_secondary_vmexit_ctrls()) + secondary_vm_exit_controls_set(vmx, vmx_secondary_vmexit_ctrl()); + /* 22.2.1, 20.8.1 / vm_entry_controls_set(vmx, vmx_get_initial_vmentry_ctrl()); @@ -5003,6 +5204,17 @@ static void init_vmcs(struct vcpu_vmx vmx) } vmx_setup_uret_msrs(vmx); + + if (cpu_feature_enabled(X86_FEATURE_FRED) && kvm_cpu_cap_has(X86_FEATURE_FRED)) { + vmcs_write64(GUEST_IA32_FRED_CONFIG, 0); + vmcs_write64(GUEST_IA32_FRED_RSP1, 0); + vmcs_write64(GUEST_IA32_FRED_RSP2, 0); + vmcs_write64(GUEST_IA32_FRED_RSP3, 0); + vmcs_write64(GUEST_IA32_FRED_STKLVLS, 0); + vmcs_write64(GUEST_IA32_FRED_SSP1, 0); + vmcs_write64(GUEST_IA32_FRED_SSP2, 0); + vmcs_write64(GUEST_IA32_FRED_SSP3, 0); + } } static void __vmx_vcpu_reset(struct kvm_vcpu vcpu) @@ -5143,6 +5355,14 @@ void vmx_inject_irq(struct kvm_vcpu vcpu, bool reinjected) vmx->vcpu.arch.event_exit_inst_len); } else intr \|= INTR_TYPE_EXT_INTR; + + /* + * Interrupts define no event data, so clear any value left behind + * by a previously injected exception. + / + if (is_fred_enabled(vcpu)) + vmcs_write64(INJECTED_EVENT_DATA, 0); + vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, intr); vmx_clear_hlt(vcpu); @@ -5173,6 +5393,13 @@ void vmx_inject_nmi(struct kvm_vcpu vcpu) return; } + /* + * KVM doesn't virtualize the NMI-source reporting feature. So clear + * any value left behind by a previously injected exception. + / + if (is_fred_enabled(vcpu)) + vmcs_write64(INJECTED_EVENT_DATA, 0); + vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, INTR_TYPE_NMI_INTR \| INTR_INFO_VALID_MASK \| NMI_VECTOR); @@ -6503,7 +6730,7 @@ void dump_vmcs(struct kvm_vcpu vcpu) struct vcpu_vmx vmx = to_vmx(vcpu); u32 vmentry_ctl, vmexit_ctl; u32 cpu_based_exec_ctrl, pin_based_exec_ctrl, secondary_exec_control; - u64 tertiary_exec_control; + u64 tertiary_exec_control, secondary_vmexit_ctl; unsigned long cr4; int efer_slot; @@ -6514,6 +6741,8 @@ void dump_vmcs(struct kvm_vcpu vcpu) vmentry_ctl = vmcs_read32(VM_ENTRY_CONTROLS); vmexit_ctl = vmcs_read32(VM_EXIT_CONTROLS); + secondary_vmexit_ctl = cpu_has_secondary_vmexit_ctrls() ? + vmcs_read64(SECONDARY_VM_EXIT_CONTROLS) : 0; cpu_based_exec_ctrl = vmcs_read32(CPU_BASED_VM_EXEC_CONTROL); pin_based_exec_ctrl = vmcs_read32(PIN_BASED_VM_EXEC_CONTROL); cr4 = vmcs_readl(GUEST_CR4); @@ -6560,6 +6789,16 @@ void dump_vmcs(struct kvm_vcpu vcpu) vmx_dump_sel("LDTR:", GUEST_LDTR_SELECTOR); vmx_dump_dtsel("IDTR:", GUEST_IDTR_LIMIT); vmx_dump_sel("TR: ", GUEST_TR_SELECTOR); + if (vmentry_ctl & VM_ENTRY_LOAD_IA32_FRED) + pr_err("FRED guest: config=0x%016llx, stack_levels=0x%016llx\n" + "RSP0=0x%016llx, RSP1=0x%016llx\n" + "RSP2=0x%016llx, RSP3=0x%016llx\n", + vmcs_read64(GUEST_IA32_FRED_CONFIG), + vmcs_read64(GUEST_IA32_FRED_STKLVLS), + vmx_read_guest_fred_rsp0(vmx), + vmcs_read64(GUEST_IA32_FRED_RSP1), + vmcs_read64(GUEST_IA32_FRED_RSP2), + vmcs_read64(GUEST_IA32_FRED_RSP3)); efer_slot = vmx_find_loadstore_msr_slot(&vmx->msr_autoload.guest, MSR_EFER); if (vmentry_ctl & VM_ENTRY_LOAD_IA32_EFER) pr_err("EFER= 0x%016llx\n", vmcs_read64(GUEST_IA32_EFER)); @@ -6611,6 +6850,16 @@ void dump_vmcs(struct kvm_vcpu vcpu) vmcs_readl(HOST_TR_BASE)); pr_err("GDTBase=%016lx IDTBase=%016lx\n", vmcs_readl(HOST_GDTR_BASE), vmcs_readl(HOST_IDTR_BASE)); + if (secondary_vmexit_ctl & SECONDARY_VM_EXIT_LOAD_IA32_FRED) + pr_err("FRED host: config=0x%016llx, stack_levels=0x%016llx\n" + "RSP0=0x%016lx, RSP1=0x%016llx\n" + "RSP2=0x%016llx, RSP3=0x%016llx\n", + vmcs_read64(HOST_IA32_FRED_CONFIG), + vmcs_read64(HOST_IA32_FRED_STKLVLS), + (unsigned long)task_stack_page(current) + THREAD_SIZE, + vmcs_read64(HOST_IA32_FRED_RSP1), + vmcs_read64(HOST_IA32_FRED_RSP2), + vmcs_read64(HOST_IA32_FRED_RSP3)); pr_err("CR0=%016lx CR3=%016lx CR4=%016lx\n", vmcs_readl(HOST_CR0), vmcs_readl(HOST_CR3), vmcs_readl(HOST_CR4)); @@ -6636,25 +6885,29 @@ void dump_vmcs(struct kvm_vcpu vcpu) pr_err(" Control State \n"); pr_err("CPUBased=0x%08x SecondaryExec=0x%08x TertiaryExec=0x%016llx\n", cpu_based_exec_ctrl, secondary_exec_control, tertiary_exec_control); - pr_err("PinBased=0x%08x EntryControls=%08x ExitControls=%08x\n", - pin_based_exec_ctrl, vmentry_ctl, vmexit_ctl); + pr_err("PinBased=0x%08x EntryControls=0x%08x\n", + pin_based_exec_ctrl, vmentry_ctl); + pr_err("ExitControls=0x%08x SecondaryExitControls=0x%016llx\n", + vmexit_ctl, secondary_vmexit_ctl); pr_err("ExceptionBitmap=%08x PFECmask=%08x PFECmatch=%08x\n", vmcs_read32(EXCEPTION_BITMAP), vmcs_read32(PAGE_FAULT_ERROR_CODE_MASK), vmcs_read32(PAGE_FAULT_ERROR_CODE_MATCH)); - pr_err("VMEntry: intr_info=%08x errcode=%08x ilen=%08x\n", + pr_err("VMEntry: intr_info=%08x errcode=%08x ilen=%08x event_data=%016llx\n", vmcs_read32(VM_ENTRY_INTR_INFO_FIELD), vmcs_read32(VM_ENTRY_EXCEPTION_ERROR_CODE), - vmcs_read32(VM_ENTRY_INSTRUCTION_LEN)); + vmcs_read32(VM_ENTRY_INSTRUCTION_LEN), + kvm_cpu_cap_has(X86_FEATURE_FRED) ? vmcs_read64(INJECTED_EVENT_DATA) : 0); pr_err("VMExit: intr_info=%08x errcode=%08x ilen=%08x\n", vmcs_read32(VM_EXIT_INTR_INFO), vmcs_read32(VM_EXIT_INTR_ERROR_CODE), vmcs_read32(VM_EXIT_INSTRUCTION_LEN)); pr_err(" reason=%08x qualification=%016lx\n", vmcs_read32(VM_EXIT_REASON), vmcs_readl(EXIT_QUALIFICATION)); - pr_err("IDTVectoring: info=%08x errcode=%08x\n", + pr_err("IDTVectoring: info=%08x errcode=%08x event_data=%016llx\n", vmcs_read32(IDT_VECTORING_INFO_FIELD), - vmcs_read32(IDT_VECTORING_ERROR_CODE)); + vmcs_read32(IDT_VECTORING_ERROR_CODE), + kvm_cpu_cap_has(X86_FEATURE_FRED) ? vmcs_read64(ORIGINAL_EVENT_DATA) : 0); pr_err("TSC Offset = 0x%016llx\n", vmcs_read64(TSC_OFFSET)); if (secondary_exec_control & SECONDARY_EXEC_TSC_SCALING) pr_err("TSC Multiplier = 0x%016llx\n", @@ -7280,7 +7533,8 @@ static void vmx_recover_nmi_blocking(struct vcpu_vmx vmx) static void __vmx_complete_interrupts(struct kvm_vcpu vcpu, u32 idt_vectoring_info, int instr_len_field, - int error_code_field) + int error_code_field, + int event_data_field) { u8 vector; int type; @@ -7315,13 +7569,18 @@ static void __vmx_complete_interrupts(struct kvm_vcpu vcpu, fallthrough; case INTR_TYPE_HARD_EXCEPTION: { u32 error_code = 0; + u64 event_data = 0; if (idt_vectoring_info & VECTORING_INFO_DELIVER_CODE_MASK) error_code = vmcs_read32(error_code_field); + if (is_fred_enabled(vcpu)) + event_data = vmcs_read64(event_data_field); kvm_requeue_exception(vcpu, vector, idt_vectoring_info & VECTORING_INFO_DELIVER_CODE_MASK, - error_code); + error_code, + idt_vectoring_info & VECTORING_INFO_NESTED_EXCEPTION_MASK, + event_data); break; } case INTR_TYPE_SOFT_INTR: @@ -7339,7 +7598,8 @@ static void vmx_complete_interrupts(struct vcpu_vmx vmx) { __vmx_complete_interrupts(&vmx->vcpu, vmx->idt_vectoring_info, VM_EXIT_INSTRUCTION_LEN, - IDT_VECTORING_ERROR_CODE); + IDT_VECTORING_ERROR_CODE, + ORIGINAL_EVENT_DATA); } void vmx_cancel_injection(struct kvm_vcpu vcpu) @@ -7347,7 +7607,8 @@ void vmx_cancel_injection(struct kvm_vcpu vcpu) __vmx_complete_interrupts(vcpu, vmcs_read32(VM_ENTRY_INTR_INFO_FIELD), VM_ENTRY_INSTRUCTION_LEN, - VM_ENTRY_EXCEPTION_ERROR_CODE); + VM_ENTRY_EXCEPTION_ERROR_CODE, + INJECTED_EVENT_DATA); vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, 0); } @@ -7459,6 +7720,10 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu vcpu, vmx_disable_fb_clear(vmx); + /* + * Note, even though FRED delivers the faulting linear address via the + * event data field on the stack, CR2 is still updated. + / if (vcpu->arch.cr2 != native_read_cr2()) native_write_cr2(vcpu->arch.cr2); @@ -7885,6 +8150,7 @@ static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu vcpu) entry = kvm_find_cpuid_entry_index(vcpu, 0x7, 1); cr4_fixed1_update(X86_CR4_LAM_SUP, eax, feature_bit(LAM)); + cr4_fixed1_update(X86_CR4_FRED, eax, feature_bit(FRED)); #undef cr4_fixed1_update } @@ -8095,6 +8361,9 @@ static __init void vmx_set_cpu_caps(void) kvm_cpu_cap_check_and_set(X86_FEATURE_DTES64); } + if (!cpu_has_vmx_fred()) + kvm_cpu_cap_clear(X86_FEATURE_FRED); + if (!enable_pmu) kvm_cpu_cap_clear(X86_FEATURE_PDCM); kvm_caps.supported_perf_cap = vmx_get_perf_capabilities(); @@ -8853,6 +9122,11 @@ __init int vmx_hardware_setup(void) kvm_caps.inapplicable_quirks &= ~KVM_X86_QUIRK_IGNORE_GUEST_PAT; + if (cpu_feature_enabled(X86_FEATURE_FRED) && kvm_cpu_cap_has(X86_FEATURE_FRED)) { + rdmsrq(MSR_IA32_FRED_CONFIG, kvm_host.fred_config); + rdmsrq(MSR_IA32_FRED_STKLVLS, kvm_host.fred_stklvls); + } + return 0; } diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index dc8517f15bc46..cc1e4038cd441 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -182,6 +182,29 @@ struct nested_vmx { u64 pre_vmenter_ssp; u64 pre_vmenter_ssp_tbl; + /* + * Used to snapshot FRED MSRs that may NOT be saved to vmcs12 as specified + * in the VM-Exit controls of vmcs12 configured by L1 VMM. + * + * FRED MSRs are always saved into vmcs02 because KVM always sets + * SECONDARY_VM_EXIT_SAVE_IA32_FRED. However an L1 VMM may choose to clear + * this bit, resulting in FRED MSRs not being propagated to vmcs12 from + * vmcs02. When the L1 VMM sets SECONDARY_VM_EXIT_LOAD_IA32_FRED, this is + * not a problem, since KVM then immediately loads the host FRED MSRs of + * vmcs12 to the guest FRED MSRs of vmcs01. + * + * But if the L1 VMM clears SECONDARY_VM_EXIT_LOAD_IA32_FRED, KVM should + * retain the FRED MSRs, i.e., propagate the guest FRED MSRs of vmcs02 to + * the guest FRED MSRs of vmcs01. + * + * This structure stores guest FRED MSRs that an L1 VMM opts not to save + * during VM-Exits from L2 to L1. These MSRs may still be retained for + * running the L1 VMM if SECONDARY_VM_EXIT_LOAD_IA32_FRED is cleared in + * vmcs12. + / + struct vmcs_fred_msrs pre_vmenter_fred_msrs; + struct vmcs_fred_msrs at_vmexit_fred_msrs; + u16 vpid02; u16 last_vpid; @@ -222,6 +245,7 @@ struct vcpu_vmx { bool guest_uret_msrs_loaded; #ifdef CONFIG_X86_64 u64 msr_guest_kernel_gs_base; + u64 msr_guest_fred_rsp0; #endif u64 spec_ctrl; @@ -477,7 +501,8 @@ static inline u8 vmx_get_rvi(void) VM_ENTRY_LOAD_BNDCFGS \| \ VM_ENTRY_PT_CONCEAL_PIP \| \ VM_ENTRY_LOAD_IA32_RTIT_CTL \| \ - VM_ENTRY_LOAD_CET_STATE) + VM_ENTRY_LOAD_CET_STATE \| \ + VM_ENTRY_LOAD_IA32_FRED) #define __KVM_REQUIRED_VMX_VM_EXIT_CONTROLS \ (VM_EXIT_SAVE_DEBUG_CONTROLS \| \ @@ -501,7 +526,13 @@ static inline u8 vmx_get_rvi(void) VM_EXIT_PT_CONCEAL_PIP \| \ VM_EXIT_CLEAR_IA32_RTIT_CTL \| \ VM_EXIT_LOAD_CET_STATE \| \ - VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL) + VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL \| \ + VM_EXIT_ACTIVATE_SECONDARY_CONTROLS) + +#define KVM_REQUIRED_VMX_SECONDARY_VM_EXIT_CONTROLS (0) +#define KVM_OPTIONAL_VMX_SECONDARY_VM_EXIT_CONTROLS \ + (SECONDARY_VM_EXIT_SAVE_IA32_FRED \| \ + SECONDARY_VM_EXIT_LOAD_IA32_FRED) #define KVM_REQUIRED_VMX_PIN_BASED_VM_EXEC_CONTROL \ (PIN_BASED_EXT_INTR_MASK \| \ @@ -615,6 +646,7 @@ static __always_inline void lname##_controls_changebit(struct vcpu_vmx vmx, u## } BUILD_CONTROLS_SHADOW(vm_entry, VM_ENTRY_CONTROLS, 32) BUILD_CONTROLS_SHADOW(vm_exit, VM_EXIT_CONTROLS, 32) +BUILD_CONTROLS_SHADOW(secondary_vm_exit, SECONDARY_VM_EXIT_CONTROLS, 64) BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL, 32) BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL, 32) BUILD_CONTROLS_SHADOW(secondary_exec, SECONDARY_VM_EXEC_CONTROL, 32) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 4b3681796c752..f264bc9c35f9d 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -431,9 +431,22 @@ void kvm_deliver_exception_payload(struct kvm_vcpu vcpu, breakpoint), it is reserved and must be zero in DR6. / vcpu->arch.dr6 &= ~BIT(12); + + / + * FRED #DB event data matches DR6, but follows the polarity of + * VMX's pending debug exceptions, not DR6. + / + ex->event_data = ex->payload & ~BIT(12); + break; + case NM_VECTOR: + ex->event_data = ex->payload; break; case PF_VECTOR: vcpu->arch.cr2 = ex->payload; + ex->event_data = ex->payload; + break; + default: + ex->event_data = 0; break; } @@ -461,6 +474,7 @@ static void kvm_multiple_exception(struct kvm_vcpu vcpu, unsigned int nr, bool has_error, u32 error_code, bool has_payload, unsigned long payload) { + bool is_nested = false; u32 prev_nr; int class1, class2; @@ -487,6 +501,15 @@ static void kvm_multiple_exception(struct kvm_vcpu vcpu, unsigned int nr, vcpu->arch.exception.error_code = error_code; vcpu->arch.exception.has_payload = has_payload; vcpu->arch.exception.payload = payload; + / #DF is NOT a nested event, per its definition. / + vcpu->arch.exception.is_nested = (nr != DF_VECTOR) && + (is_nested \|\| + vcpu->arch.nmi_injected \|\| + vcpu->arch.interrupt.injected); + / + * Do not update the event data, as the current value may have + * just been restored during a VM save/restore or live migration. + / return; } @@ -510,6 +533,9 @@ static void kvm_multiple_exception(struct kvm_vcpu vcpu, unsigned int nr, kvm_queue_exception_e(vcpu, DF_VECTOR, 0); } else { + /* The new exception arrived while delivering the previous one. / + is_nested = true; + / replace previous exception with a new one in a hope that instruction re-execution will regenerate lost exception / @@ -538,7 +564,8 @@ static void kvm_queue_exception_e_p(struct kvm_vcpu vcpu, unsigned nr, } void kvm_requeue_exception(struct kvm_vcpu vcpu, unsigned int nr, - bool has_error_code, u32 error_code) + bool has_error_code, u32 error_code, + bool is_nested, u64 event_data) { / @@ -563,6 +590,8 @@ void kvm_requeue_exception(struct kvm_vcpu vcpu, unsigned int nr, vcpu->arch.exception.error_code = error_code; vcpu->arch.exception.has_payload = false; vcpu->arch.exception.payload = 0; + vcpu->arch.exception.is_nested = is_nested; + vcpu->arch.exception.event_data = event_data; } EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_requeue_exception); @@ -2288,6 +2317,9 @@ int kvm_vm_ioctl_check_extension(struct kvm kvm, long ext) case KVM_CAP_PRE_FAULT_MEMORY: r = tdp_enabled; break; + case KVM_CAP_X86_FRED_EVENT: + r = kvm_cpu_cap_has(X86_FEATURE_FRED); + break; case KVM_CAP_X86_APIC_BUS_CYCLES_NS: r = kvm ? kvm->arch.apic_bus_cycle_ns : APIC_BUS_CYCLE_NS_DEFAULT; break; @@ -2997,6 +3029,8 @@ static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu vcpu, events->exception.error_code = ex->error_code; events->exception_has_payload = ex->has_payload; events->exception_payload = ex->payload; + events->exception_is_nested = ex->is_nested; + events->exception_event_data = ex->event_data; events->interrupt.injected = vcpu->arch.interrupt.injected && !vcpu->arch.interrupt.soft; @@ -3022,6 +3056,8 @@ static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu vcpu, \| KVM_VCPUEVENT_VALID_SMM); if (vcpu->kvm->arch.exception_payload_enabled) events->flags \|= KVM_VCPUEVENT_VALID_PAYLOAD; + if (vcpu->kvm->arch.exception_fred_state_enabled) + events->flags \|= KVM_VCPUEVENT_VALID_FRED_STATE; if (vcpu->kvm->arch.triple_fault_event) { events->triple_fault.pending = kvm_test_request(KVM_REQ_TRIPLE_FAULT, vcpu); events->flags \|= KVM_VCPUEVENT_VALID_TRIPLE_FAULT; @@ -3036,6 +3072,7 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu vcpu, \| KVM_VCPUEVENT_VALID_SHADOW \| KVM_VCPUEVENT_VALID_SMM \| KVM_VCPUEVENT_VALID_PAYLOAD + \| KVM_VCPUEVENT_VALID_FRED_STATE \| KVM_VCPUEVENT_VALID_TRIPLE_FAULT)) return -EINVAL; @@ -3051,6 +3088,14 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu vcpu, events->exception_has_payload = 0; } + if (events->flags & KVM_VCPUEVENT_VALID_FRED_STATE) { + if (!vcpu->kvm->arch.exception_fred_state_enabled) + return -EINVAL; + } else { + events->exception_is_nested = 0; + events->exception_event_data = 0; + } + if ((events->exception.injected \|\| events->exception.pending) && (events->exception.nr > 31 \|\| events->exception.nr == NMI_VECTOR)) return -EINVAL; @@ -3076,6 +3121,8 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu vcpu, vcpu->arch.exception.error_code = events->exception.error_code; vcpu->arch.exception.has_payload = events->exception_has_payload; vcpu->arch.exception.payload = events->exception_payload; + vcpu->arch.exception.is_nested = events->exception_is_nested; + vcpu->arch.exception.event_data = events->exception_event_data; vcpu->arch.interrupt.injected = events->interrupt.injected; vcpu->arch.interrupt.nr = events->interrupt.nr; @@ -4038,6 +4085,13 @@ int kvm_vm_ioctl_enable_cap(struct kvm kvm, kvm->arch.exception_payload_enabled = cap->args[0]; r = 0; break; + case KVM_CAP_X86_FRED_EVENT: + r = -EINVAL; + if (!kvm_cpu_cap_has(X86_FEATURE_FRED)) + break; + kvm->arch.exception_fred_state_enabled = cap->args[0]; + r = 0; + break; case KVM_CAP_X86_TRIPLE_FAULT_EVENT: kvm->arch.triple_fault_event = cap->args[0]; r = 0; @@ -5498,11 +5552,17 @@ static int emulator_set_dr(struct x86_emulate_ctxt ctxt, int dr, return kvm_set_dr(emul_to_vcpu(ctxt), dr, value); } -static u64 mk_cr_64(u64 curr_cr, u32 new_val) +static u64 mk_cr0_64(u64 curr_cr, u32 new_val) { return (curr_cr & ~((1ULL << 32) - 1)) \| new_val; } +static u64 mk_cr4_64(struct kvm_vcpu vcpu, u64 curr_cr, u64 new_val) +{ + u32 shift = guest_cpu_cap_has(vcpu, X86_FEATURE_FRED) ? 33 : 32; + return (curr_cr & ~((1ULL << shift) - 1)) \| new_val; +} + static unsigned long emulator_get_cr(struct x86_emulate_ctxt ctxt, int cr) { struct kvm_vcpu vcpu = emul_to_vcpu(ctxt); @@ -5539,7 +5599,7 @@ static int emulator_set_cr(struct x86_emulate_ctxt ctxt, int cr, ulong val) switch (cr) { case 0: - res = kvm_set_cr0(vcpu, mk_cr_64(kvm_read_cr0(vcpu), val)); + res = kvm_set_cr0(vcpu, mk_cr0_64(kvm_read_cr0(vcpu), val)); break; case 2: vcpu->arch.cr2 = val; @@ -5548,7 +5608,7 @@ static int emulator_set_cr(struct x86_emulate_ctxt ctxt, int cr, ulong val) res = kvm_set_cr3(vcpu, val); break; case 4: - res = kvm_set_cr4(vcpu, mk_cr_64(kvm_read_cr4(vcpu), val)); + res = kvm_set_cr4(vcpu, mk_cr4_64(vcpu, kvm_read_cr4(vcpu), val)); break; case 8: res = kvm_set_cr8(vcpu, val); @@ -8960,6 +9020,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu vcpu) ex->has_payload, ex->payload); ex->injected = false; ex->pending = false; + ex->is_nested = false; } vcpu->arch.exception_from_userspace = false; diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 0f5919b092e47..43c5c08f5d5ca 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -158,6 +158,8 @@ static inline void kvm_clear_exception_queue(struct kvm_vcpu vcpu) { vcpu->arch.exception.pending = false; vcpu->arch.exception.injected = false; + vcpu->arch.exception.is_nested = false; + vcpu->arch.exception.event_data = 0; vcpu->arch.exception_vmexit.pending = false; } @@ -502,7 +504,8 @@ void kvm_queue_exception(struct kvm_vcpu vcpu, unsigned nr); void kvm_queue_exception_e(struct kvm_vcpu vcpu, unsigned nr, u32 error_code); void kvm_queue_exception_p(struct kvm_vcpu vcpu, unsigned nr, unsigned long payload); void kvm_requeue_exception(struct kvm_vcpu vcpu, unsigned int nr, - bool has_error_code, u32 error_code); + bool has_error_code, u32 error_code, + bool is_nested, u64 event_data); void kvm_inject_page_fault(struct kvm_vcpu vcpu, struct x86_exception fault, bool from_hardware); void __kvm_inject_emulated_page_fault(struct kvm_vcpu vcpu, diff --git a/arch/x86/mm/cpu_entry_area.c b/arch/x86/mm/cpu_entry_area.c index 575f863f3c75e..a47dfd686bd33 100644 --- a/arch/x86/mm/cpu_entry_area.c +++ b/arch/x86/mm/cpu_entry_area.c @@ -18,6 +18,26 @@ static DEFINE_PER_CPU_PAGE_ALIGNED(struct entry_stack_page, entry_stack_storage) static DEFINE_PER_CPU_PAGE_ALIGNED(struct exception_stacks, exception_stacks); DEFINE_PER_CPU(struct cea_exception_stacks, cea_exception_stacks); +/* + * Typically invoked by entry code, so must be noinstr. + / +noinstr unsigned long __this_cpu_ist_bottom_va(enum exception_stack_ordering stack) +{ + struct cea_exception_stacks s; + + BUILD_BUG_ON(ESTACK_DF != 0); + BUG_ON(stack >= N_EXCEPTION_STACKS); + + s = __this_cpu_read(cea_exception_stacks); + + return (unsigned long)&s->event_stacks[stack].stack; +} + +noinstr unsigned long __this_cpu_ist_top_va(enum exception_stack_ordering stack) +{ + return __this_cpu_ist_bottom_va(stack) + EXCEPTION_STKSZ; +} + static DEFINE_PER_CPU_READ_MOSTLY(unsigned long, _cea_offset); static __always_inline unsigned int cea_offset(unsigned int cpu) @@ -132,7 +152,7 @@ static void __init percpu_setup_debug_store(unsigned int cpu) #define cea_map_stack(name) do { \ npages = sizeof(estacks->name## _stack) / PAGE_SIZE; \ - cea_map_percpu_pages(cea->estacks.name## _stack, \ + cea_map_percpu_pages(cea->estacks.event_stacks[name].stack, \ estacks->name## _stack, npages, PAGE_KERNEL); \ } while (0) @@ -151,15 +171,15 @@ static void __init percpu_setup_exception_stacks(unsigned int cpu) * by guard pages so each stack must be mapped separately. DB2 is * not mapped; it just exists to catch triple nesting of #DB. / - cea_map_stack(DF); - cea_map_stack(NMI); - cea_map_stack(DB); - cea_map_stack(MCE); + cea_map_stack(ESTACK_DF); + cea_map_stack(ESTACK_NMI); + cea_map_stack(ESTACK_DB); + cea_map_stack(ESTACK_MCE); if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) { - cea_map_stack(VC); - cea_map_stack(VC2); + cea_map_stack(ESTACK_VC); + cea_map_stack(ESTACK_VC2); } } } diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 45b99c3b1442d..f6ee1cc3f125c 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -677,7 +677,7 @@ page_fault_oops(struct pt_regs regs, unsigned long error_code, * and then double-fault, though, because we're likely to * break the console driver and lose most of the stack dump. / - call_on_stack(__this_cpu_ist_top_va(DF) - sizeof(void), + call_on_stack(__this_cpu_ist_top_va(ESTACK_DF) - sizeof(void), handle_stack_overflow, ASM_CALL_ARG3, , [arg1] "r" (regs), [arg2] "r" (address), [arg3] "r" (&info)); diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 9fc8dfdfd65ff..c5ffc4e7ddb14 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -998,6 +998,7 @@ struct kvm_enable_cap { #define KVM_CAP_S390_VSIE_ESAMODE 248 #define KVM_CAP_S390_HPAGE_2G 249 #define KVM_CAP_ARM_PMU_V3_STRICT 250 +#define KVM_CAP_X86_FRED_EVENT 251 struct kvm_irq_routing_irqchip { __u32 irqchip; diff --git a/tools/arch/x86/include/asm/msr-index.h b/tools/arch/x86/include/asm/msr-index.h index 18c4be75e9271..14632e29f9814 100644 --- a/tools/arch/x86/include/asm/msr-index.h +++ b/tools/arch/x86/include/asm/msr-index.h @@ -64,6 +64,8 @@ #define MSR_IA32_FRED_SSP3 0x1d3 / Level 3 shadow stack pointer / #define MSR_IA32_FRED_CONFIG 0x1d4 / Entrypoint and interrupt stack level / +#define FRED_CONFIG_RESERVED (BIT_ULL(2) \| GENMASK_ULL(5, 4) \| BIT_ULL(11)) + / Intel MSRs. Some also available on other CPUs / #define MSR_TEST_CTRL 0x00000033 #define MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT 29 diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm index 96bab7002d39e..73aed018f4a38 100644 --- a/tools/testing/selftests/kvm/Makefile.kvm +++ b/tools/testing/selftests/kvm/Makefile.kvm @@ -78,6 +78,7 @@ TEST_GEN_PROGS_x86 += x86/feature_msrs_test TEST_GEN_PROGS_x86 += x86/evmcs_smm_controls_test TEST_GEN_PROGS_x86 += x86/exit_on_emulation_failure_test TEST_GEN_PROGS_x86 += x86/fastops_test +TEST_GEN_PROGS_x86 += x86/fred_test TEST_GEN_PROGS_x86 += x86/fix_hypercall_test TEST_GEN_PROGS_x86 += x86/hwcr_msr_test TEST_GEN_PROGS_x86 += x86/hyperv_clock diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h index 5a7a455b53875..981adfcbd5fa2 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -191,6 +191,7 @@ enum vm_guest_mode { VM_MODE_P40V48_16K, VM_MODE_P40V48_64K, VM_MODE_PXXVYY_4K, / For 48-bit or 57-bit VA, depending on host support / + VM_MODE_PXXVYY_4K_USER, / For 48bits VA but ANY bits PA with USER bit set / VM_MODE_P47V64_4K, VM_MODE_P44V64_4K, VM_MODE_P36V48_4K, diff --git a/tools/testing/selftests/kvm/include/x86/processor.h b/tools/testing/selftests/kvm/include/x86/processor.h index 6e6f70035508a..988c45e326894 100644 --- a/tools/testing/selftests/kvm/include/x86/processor.h +++ b/tools/testing/selftests/kvm/include/x86/processor.h @@ -79,6 +79,7 @@ const char ex_str(int vector); #define X86_CR4_SMEP (1ul << 20) #define X86_CR4_SMAP (1ul << 21) #define X86_CR4_PKE (1ul << 22) +#define X86_CR4_FRED (1ul << 32) struct xstate_header { u64 xstate_bv; @@ -195,6 +196,7 @@ struct kvm_x86_cpu_feature { #define X86_FEATURE_SPEC_CTRL KVM_X86_CPU_FEATURE(0x7, 0, EDX, 26) #define X86_FEATURE_ARCH_CAPABILITIES KVM_X86_CPU_FEATURE(0x7, 0, EDX, 29) #define X86_FEATURE_PKS KVM_X86_CPU_FEATURE(0x7, 0, ECX, 31) +#define X86_FEATURE_FRED KVM_X86_CPU_FEATURE(0x7, 1, EAX, 17) #define X86_FEATURE_XTILECFG KVM_X86_CPU_FEATURE(0xD, 0, EAX, 17) #define X86_FEATURE_XTILEDATA KVM_X86_CPU_FEATURE(0xD, 0, EAX, 18) #define X86_FEATURE_XSAVES KVM_X86_CPU_FEATURE(0xD, 1, EAX, 3) @@ -1645,4 +1647,36 @@ u64 tdp_get_pte(struct kvm_vm vm, u64 l2_gpa); bool sys_clocksource_is_based_on_tsc(void); +/* + * FRED related data structures and functions + / + +#define FRED_SSX_NMI BIT_ULL(18) + +struct fred_stack { + u64 r15; + u64 r14; + u64 r13; + u64 r12; + u64 bp; + u64 bx; + u64 r11; + u64 r10; + u64 r9; + u64 r8; + u64 ax; + u64 cx; + u64 dx; + u64 si; + u64 di; + u64 error_code; + u64 ip; + u64 csx; + u64 flags; + u64 sp; + u64 ssx; + u64 event_data; + u64 reserved; +}; + #endif / SELFTEST_KVM_PROCESSOR_H / diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c index 9ddc047d5c275..e79fddedaf396 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -210,6 +210,7 @@ const char vm_guest_mode_string(u32 i) [VM_MODE_P40V48_16K] = "PA-bits:40, VA-bits:48, 16K pages", [VM_MODE_P40V48_64K] = "PA-bits:40, VA-bits:48, 64K pages", [VM_MODE_PXXVYY_4K] = "PA-bits:ANY, VA-bits:48 or 57, 4K pages", + [VM_MODE_PXXVYY_4K_USER]= "PA-bits:ANY, VA-bits:48, 4K user pages", [VM_MODE_P47V64_4K] = "PA-bits:47, VA-bits:64, 4K pages", [VM_MODE_P44V64_4K] = "PA-bits:44, VA-bits:64, 4K pages", [VM_MODE_P36V48_4K] = "PA-bits:36, VA-bits:48, 4K pages", @@ -246,6 +247,7 @@ const struct vm_guest_mode_params vm_guest_mode_params[] = { [VM_MODE_P40V48_16K] = { 40, 48, 0x4000, 14 }, [VM_MODE_P40V48_64K] = { 40, 48, 0x10000, 16 }, [VM_MODE_PXXVYY_4K] = { 0, 0, 0x1000, 12 }, + [VM_MODE_PXXVYY_4K_USER]= { 0, 0, 0x1000, 12 }, [VM_MODE_P47V64_4K] = { 47, 64, 0x1000, 12 }, [VM_MODE_P44V64_4K] = { 44, 64, 0x1000, 12 }, [VM_MODE_P36V48_4K] = { 36, 48, 0x1000, 12 }, @@ -337,6 +339,7 @@ struct kvm_vm ____vm_create(struct vm_shape shape) vm->mmu.pgtable_levels = 3; break; case VM_MODE_PXXVYY_4K: + case VM_MODE_PXXVYY_4K_USER: #ifdef __x86_64__ kvm_get_cpu_address_width(&vm->pa_bits, &vm->va_bits); kvm_init_vm_address_properties(vm); @@ -355,7 +358,7 @@ struct kvm_vm ____vm_create(struct vm_shape shape) vm->mmu.pgtable_levels = 4; } #else - TEST_FAIL("VM_MODE_PXXVYY_4K not supported on non-x86 platforms"); + TEST_FAIL("VM_MODE_PXXVYY_4K(_USER) not supported on non-x86 platforms"); #endif break; case VM_MODE_P47V64_4K: diff --git a/tools/testing/selftests/kvm/lib/x86/processor.c b/tools/testing/selftests/kvm/lib/x86/processor.c index d31fa81ea0756..51d713c4c9de1 100644 --- a/tools/testing/selftests/kvm/lib/x86/processor.c +++ b/tools/testing/selftests/kvm/lib/x86/processor.c @@ -167,6 +167,9 @@ bool kvm_is_tdp_enabled(void) static void virt_mmu_init(struct kvm_vm vm, struct kvm_mmu mmu, struct pte_masks pte_masks) { + TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K \|\| vm->mode == VM_MODE_PXXVYY_4K_USER, + "Unknown or unsupported guest mode: 0x%x", vm->mode); + / If needed, create the top-level page table. / if (!mmu->pgd_created) { mmu->pgd = vm_alloc_page_table(vm); @@ -181,7 +184,7 @@ static void virt_mmu_init(struct kvm_vm vm, struct kvm_mmu mmu, void virt_arch_pgd_alloc(struct kvm_vm vm) { - TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K, + TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K \|\| vm->mode == VM_MODE_PXXVYY_4K_USER, "Unknown or unsupported guest mode: 0x%x", vm->mode); struct pte_masks pte_masks = (struct pte_masks){ @@ -239,6 +242,8 @@ static u64 virt_create_upper_pte(struct kvm_vm vm, pte = PTE_PRESENT_MASK(mmu) \| PTE_READABLE_MASK(mmu) \| PTE_WRITABLE_MASK(mmu) \| PTE_EXECUTABLE_MASK(mmu) \| PTE_ALWAYS_SET_MASK(mmu); + if (vm->mode == VM_MODE_PXXVYY_4K_USER) + pte \|= PTE_USER_MASK(mmu); if (current_level == target_level) pte \|= PTE_HUGE_MASK(mmu) \| (gpa & PHYSICAL_PAGE_MASK); else @@ -266,7 +271,7 @@ void __virt_pg_map(struct kvm_vm vm, struct kvm_mmu mmu, gva_t gva, u64 pte = &mmu->pgd; int current_level; - TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K, + TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K \|\| vm->mode == VM_MODE_PXXVYY_4K_USER, "Unknown or unsupported guest mode: 0x%x", vm->mode); TEST_ASSERT((gva % pg_size) == 0, @@ -316,6 +321,9 @@ void __virt_pg_map(struct kvm_vm vm, struct kvm_mmu mmu, gva_t gva, pte \|= PTE_C_BIT_MASK(mmu); else pte \|= PTE_S_BIT_MASK(mmu); + + if (vm->mode == VM_MODE_PXXVYY_4K_USER) + pte \|= PTE_USER_MASK(mmu); } void virt_arch_pg_map(struct kvm_vm vm, gva_t gva, gpa_t gpa) @@ -372,7 +380,7 @@ static u64 __vm_get_page_table_entry(struct kvm_vm vm, TEST_ASSERT(level >= PG_LEVEL_NONE && level <= mmu->pgtable_levels, "Invalid PG_LEVEL_* '%d'", level); - TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K, + TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K \|\| vm->mode == VM_MODE_PXXVYY_4K_USER, "Unknown or unsupported guest mode: 0x%x", vm->mode); TEST_ASSERT(sparsebit_is_set(vm->vpages_valid, (gva >> vm->page_shift)), "Invalid virtual address, gva: 0x%lx", gva); @@ -645,7 +653,7 @@ static void vcpu_init_sregs(struct kvm_vm vm, struct kvm_vcpu vcpu) { struct kvm_sregs sregs; - TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K, + TEST_ASSERT(vm->mode == VM_MODE_PXXVYY_4K \|\| vm->mode == VM_MODE_PXXVYY_4K_USER, "Unknown or unsupported guest mode: 0x%x", vm->mode); / Set mode specific system register values. / diff --git a/tools/testing/selftests/kvm/x86/fred_test.c b/tools/testing/selftests/kvm/x86/fred_test.c new file mode 100644 index 0000000000000..c7d08e5253b98 --- /dev/null +++ b/tools/testing/selftests/kvm/x86/fred_test.c @@ -0,0 +1,291 @@ +// SPDX-License-Identifier: GPL-2.0-only +/ + * FRED nested exception tests + * + * Copyright (C) 2023, Intel, Inc. + / +#define _GNU_SOURCE / for program_invocation_short_name / +#include <fcntl.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <sys/ioctl.h> +#include <asm/msr-index.h> + +#include "apic.h" +#include "kvm_util.h" +#include "test_util.h" +#include "guest_modes.h" +#include "processor.h" + +#define IRQ_VECTOR 0xAA + +#define FRED_STKLVL(v,l) (_AT(unsigned long, l) << (2 (v))) +#define FRED_CONFIG_ENTRYPOINT(p) _AT(unsigned long, (p)) + +/* This address is already mapped in guest page table. / +#define FRED_VALID_RSP 0x8000 + +/ + * The following addresses are not yet mapped in both EPT and guest page + * tables at the beginning. As a result, it causes an EPT violation VM + * exit with an original guest #PF to access any of them for the first + * time. + * + * Use these addresses as guest FRED RSP0 to generate nested #PFs to test + * if event data are properly virtualized. + / +static unsigned long fred_invalid_rsp[4] = { + 0x0, + 0xf0000000, + 0xe0000000, + 0xd0000000, +}; + +extern char asm_user_nop[]; +extern char asm_user_ud[]; +extern char asm_done_fault[]; + +extern void asm_test_fault(int test); + +/ + * user level code for triggering faults. + / +asm(".pushsection .text\n" + ".align 4096\n" + + ".type asm_user_nop, @function\n" + "asm_user_nop:\n" + "1: .byte 0x90\n" + "jmp 1b\n" + + ".org asm_user_nop + 16, 0xcc\n" + ".type asm_user_ud, @function\n" + "asm_user_ud:\n" + / Trigger a #UD / + "ud2\n" + + ".align 4096, 0xcc\n" + ".popsection"); + +/ Send current stack level and #PF address / +#define GUEST_SYNC_CSL_FA(__stage, __pf_address) \ + GUEST_SYNC_ARGS(__stage, __pf_address, 0, 0, 0) + +void fred_entry_from_user(struct fred_stack stack) +{ + u32 current_stack_level = rdmsr(MSR_IA32_FRED_CONFIG) & 0x3; + + GUEST_SYNC_CSL_FA(current_stack_level, stack->event_data); + + /* Do NOT go back to user level, continue the next test instead / + stack->ssx = 0x18; + stack->csx = 0x10; + stack->ip = (u64)&asm_done_fault; +} + +void fred_entry_from_kernel(struct fred_stack stack) +{ + /* + * Keep NMI blocked to delay the delivery of the next NMI until + * returning to user level. + * / + stack->ssx &= ~FRED_SSX_NMI; +} + +#define PUSH_REGS \ + "push %rdi\n" \ + "push %rsi\n" \ + "push %rdx\n" \ + "push %rcx\n" \ + "push %rax\n" \ + "push %r8\n" \ + "push %r9\n" \ + "push %r10\n" \ + "push %r11\n" \ + "push %rbx\n" \ + "push %rbp\n" \ + "push %r12\n" \ + "push %r13\n" \ + "push %r14\n" \ + "push %r15\n" + +#define POP_REGS \ + "pop %r15\n" \ + "pop %r14\n" \ + "pop %r13\n" \ + "pop %r12\n" \ + "pop %rbp\n" \ + "pop %rbx\n" \ + "pop %r11\n" \ + "pop %r10\n" \ + "pop %r9\n" \ + "pop %r8\n" \ + "pop %rax\n" \ + "pop %rcx\n" \ + "pop %rdx\n" \ + "pop %rsi\n" \ + "pop %rdi\n" + +/ + * FRED entry points. + / +asm(".pushsection .text\n" + ".type asm_fred_entrypoint_user, @function\n" + ".align 4096\n" + "asm_fred_entrypoint_user:\n" + PUSH_REGS + "movq %rsp, %rdi\n" + "call fred_entry_from_user\n" + POP_REGS + / Do NOT go back to user level, continue the next test instead / + ".byte 0xf2,0x0f,0x01,0xca\n" / ERETS / + + ".org asm_fred_entrypoint_user + 256, 0xcc\n" + ".type asm_fred_entrypoint_kernel, @function\n" + "asm_fred_entrypoint_kernel:\n" + PUSH_REGS + "movq %rsp, %rdi\n" + "call fred_entry_from_kernel\n" + POP_REGS + ".byte 0xf2,0x0f,0x01,0xca\n" / ERETS / + ".align 4096, 0xcc\n" + ".popsection"); + +extern char asm_fred_entrypoint_user[]; + +/ + * Prepare a FRED stack frame for ERETU to return to user level code, + * nop or ud2. + * + * Because FRED RSP0 is deliberately not mapped in guest page table, + * the delivery of interrupt/NMI or #UD from ring 3 causes a nested + * #PF, which is then delivered on FRED RSPx (x is 1, 2 or 3, + * determinated by MSR FRED_STKLVL[PF_VECTOR]). + / +asm(".pushsection .text\n" + ".type asm_test_fault, @function\n" + ".align 4096\n" + "asm_test_fault:\n" + "endbr64\n" + "push %rbp\n" + "mov %rsp, %rbp\n" + "and $(~0x3f), %rsp\n" + "push $0\n" + "push $0\n" + "mov $0x2b, %rax\n" + / Unblock NMI / + "bts $18, %rax\n" + / Set long mode bit / + "bts $57, %rax\n" + "push %rax\n" + / No stack required for the FRED user level test code / + "push $0\n" + "pushf\n" + "pop %rax\n" + / Allow external interrupts / + "bts $9, %rax\n" + "push %rax\n" + "mov $0x33, %rax\n" + "push %rax\n" + "cmp $0, %edi\n" + "jne 1f\n" + "lea asm_user_nop(%rip), %rax\n" + "jmp 2f\n" + "1: lea asm_user_ud(%rip), %rax\n" + "2: push %rax\n" + "push $0\n" + / ERETU to user level code to allow event delivery immediately / + ".byte 0xf3,0x0f,0x01,0xca\n" + "asm_done_fault:\n" + "mov %rbp, %rsp\n" + "pop %rbp\n" + "ret\n" + ".align 4096, 0xcc\n" + ".popsection"); + +/ + * To fully test the underlying FRED VMX code, this test should be run one + * more round with EPT disabled to inject page faults as nested exceptions. + / +static void guest_code(void) +{ + wrmsr(MSR_IA32_FRED_CONFIG, + FRED_CONFIG_ENTRYPOINT(asm_fred_entrypoint_user)); + + wrmsr(MSR_IA32_FRED_RSP1, FRED_VALID_RSP); + wrmsr(MSR_IA32_FRED_RSP2, FRED_VALID_RSP); + wrmsr(MSR_IA32_FRED_RSP3, FRED_VALID_RSP); + + / Enable FRED / + set_cr4(get_cr4() \| X86_CR4_FRED); + + x2apic_enable(); + + wrmsr(MSR_IA32_FRED_STKLVLS, FRED_STKLVL(PF_VECTOR, 1)); + wrmsr(MSR_IA32_FRED_RSP0, fred_invalid_rsp[1]); + / 1: ud2 to generate #UD / + asm_test_fault(1); + + wrmsr(MSR_IA32_FRED_STKLVLS, FRED_STKLVL(PF_VECTOR, 2)); + wrmsr(MSR_IA32_FRED_RSP0, fred_invalid_rsp[2]); + asm volatile("cli"); + / Create a pending interrupt on current vCPU / + x2apic_write_reg(APIC_ICR, APIC_DEST_SELF \| APIC_INT_ASSERT \| + APIC_DM_FIXED \| IRQ_VECTOR); + / Return to ring 3 / + asm_test_fault(0); + x2apic_write_reg(APIC_EOI, 0); + + wrmsr(MSR_IA32_FRED_STKLVLS, FRED_STKLVL(PF_VECTOR, 3)); + wrmsr(MSR_IA32_FRED_RSP0, fred_invalid_rsp[3]); + / + * The first NMI is just to have NMI blocked in ring 0, because + * fred_entry_from_kernel() deliberately clears the NMI bit in + * FRED stack frame. + / + x2apic_write_reg(APIC_ICR, APIC_DEST_SELF \| APIC_INT_ASSERT \| + APIC_DM_NMI \| NMI_VECTOR); + / The second NMI will be delivered after returning to ring 3 / + x2apic_write_reg(APIC_ICR, APIC_DEST_SELF \| APIC_INT_ASSERT \| + APIC_DM_NMI \| NMI_VECTOR); + / Return to ring 3 / + asm_test_fault(0); + + GUEST_DONE(); +} + +int main(int argc, char argv[]) +{ + struct kvm_vcpu vcpu; + struct kvm_vm vm; + struct ucall uc; + uint64_t expected_current_stack_level = 1; + + TEST_REQUIRE(kvm_cpu_has(X86_FEATURE_FRED)); + + vm = __vm_create_with_vcpus(VM_SHAPE(VM_MODE_PXXVYY_4K_USER), 1, 0, + guest_code, &vcpu); + + while (true) { + uint64_t r; + + vcpu_run(vcpu); + + r = get_ucall(vcpu, &uc); + + if (r == UCALL_DONE) + break; + + if (r == UCALL_SYNC) { + TEST_ASSERT((uc.args[1] == expected_current_stack_level) && + (uc.args[2] == fred_invalid_rsp[expected_current_stack_level] - 8), + "Incorrect stack level %lx and #PF address %lx\n", + uc.args[1], uc.args[2]); + expected_current_stack_level++; + } + } + + kvm_vm_free(vm); + return 0; +} diff --git a/tools/testing/selftests/kvm/x86/msrs_test.c b/tools/testing/selftests/kvm/x86/msrs_test.c index f7e39bf887ad3..6e2d8b8f1bd34 100644 --- a/tools/testing/selftests/kvm/x86/msrs_test.c +++ b/tools/testing/selftests/kvm/x86/msrs_test.c @@ -60,9 +60,23 @@ static const u64 u64_val = 0xaaaa5555aaaa5555ull; #define MSR_TEST_CANONICAL(msr, feat) \ __MSR_TEST(msr, #msr, canonical_val, NONCANONICAL, 0, feat) +#define MSR_TEST_CANONICAL2(msr, feat, f2) \ + ____MSR_TEST(msr, #msr, canonical_val, NONCANONICAL, 0, feat, f2, false) + #define MSR_TEST_KVM(msr, val, rsvd, feat) \ ____MSR_TEST(KVM_REG_ ##msr, #msr, val, rsvd, 0, feat, feat, true) +/* FRED RSPs are 64-byte aligned, thus bits 5:0 are all zero / +static const u64 fred_rsp_canonical_val = canonical_val + 0xbc0; + +/ + * FRED SSP[123] are 8-byte aligned, thus bits 2:0 are all zero. + * + * For legacy reasons, FRED SSP0 is allowed to be 4-byte aligned, but treat + * it as 8-byte aligned to simplify the tests. + / +static const u64 fred_ssp_canonical_val = canonical_val + 0xbc8; + / * The main struct must be scoped to a function due to the use of structures to * define features. For the global structure, allocate enough space for the @@ -388,8 +402,9 @@ static void test_msrs(void) MSR_TEST2(MSR_IA32_S_CET, CET_ENDBR_EN, CET_RESERVED, IBT, SHSTK), MSR_TEST2(MSR_IA32_U_CET, CET_SHSTK_EN, CET_RESERVED, SHSTK, IBT), MSR_TEST2(MSR_IA32_U_CET, CET_ENDBR_EN, CET_RESERVED, IBT, SHSTK), - MSR_TEST_CANONICAL(MSR_IA32_PL0_SSP, SHSTK), - MSR_TEST(MSR_IA32_PL0_SSP, canonical_val, canonical_val \| 1, SHSTK), + MSR_TEST_CANONICAL2(MSR_IA32_PL0_SSP, SHSTK, FRED), + MSR_TEST2(MSR_IA32_PL0_SSP, canonical_val, canonical_val \| 1, SHSTK, FRED), + MSR_TEST2(MSR_IA32_PL0_SSP, canonical_val, canonical_val \| 1, FRED, SHSTK), MSR_TEST_CANONICAL(MSR_IA32_PL1_SSP, SHSTK), MSR_TEST(MSR_IA32_PL1_SSP, canonical_val, canonical_val \| 1, SHSTK), MSR_TEST_CANONICAL(MSR_IA32_PL2_SSP, SHSTK), @@ -398,10 +413,35 @@ static void test_msrs(void) MSR_TEST(MSR_IA32_PL3_SSP, canonical_val, canonical_val \| 1, SHSTK), MSR_TEST_KVM(GUEST_SSP, canonical_val, NONCANONICAL, SHSTK), + + MSR_TEST_CANONICAL(MSR_IA32_FRED_RSP0, FRED), + MSR_TEST(MSR_IA32_FRED_RSP0, fred_rsp_canonical_val, fred_rsp_canonical_val \| 1, FRED), + MSR_TEST_CANONICAL(MSR_IA32_FRED_RSP1, FRED), + MSR_TEST(MSR_IA32_FRED_RSP1, fred_rsp_canonical_val, fred_rsp_canonical_val \| 2, FRED), + MSR_TEST_CANONICAL(MSR_IA32_FRED_RSP2, FRED), + MSR_TEST(MSR_IA32_FRED_RSP2, fred_rsp_canonical_val, fred_rsp_canonical_val \| 4, FRED), + MSR_TEST_CANONICAL(MSR_IA32_FRED_RSP3, FRED), + MSR_TEST(MSR_IA32_FRED_RSP3, fred_rsp_canonical_val, fred_rsp_canonical_val \| 7, FRED), + + MSR_TEST_CANONICAL2(MSR_IA32_FRED_SSP0, FRED, SHSTK), + MSR_TEST2(MSR_IA32_FRED_SSP0, fred_ssp_canonical_val, fred_ssp_canonical_val \| 1, SHSTK, FRED), + MSR_TEST2(MSR_IA32_FRED_SSP0, fred_ssp_canonical_val, fred_ssp_canonical_val \| 1, FRED, SHSTK), + MSR_TEST_CANONICAL(MSR_IA32_FRED_SSP1, FRED), + MSR_TEST(MSR_IA32_FRED_SSP1, fred_ssp_canonical_val, fred_ssp_canonical_val \| 2, FRED), + MSR_TEST_CANONICAL(MSR_IA32_FRED_SSP2, FRED), + MSR_TEST(MSR_IA32_FRED_SSP2, fred_ssp_canonical_val, fred_ssp_canonical_val \| 4, FRED), + MSR_TEST_CANONICAL(MSR_IA32_FRED_SSP3, FRED), + MSR_TEST(MSR_IA32_FRED_SSP3, fred_ssp_canonical_val, fred_ssp_canonical_val \| 7, FRED), + + MSR_TEST(MSR_IA32_FRED_STKLVLS, 0xe41be41be41be41b, 0, FRED), + + MSR_TEST_CANONICAL(MSR_IA32_FRED_CONFIG, FRED), + MSR_TEST(MSR_IA32_FRED_CONFIG, canonical_val, canonical_val \| FRED_CONFIG_RESERVED, FRED), }; const struct kvm_x86_cpu_feature feat_none = X86_FEATURE_NONE; const struct kvm_x86_cpu_feature feat_lm = X86_FEATURE_LM; + const struct kvm_x86_cpu_feature feat_fred = X86_FEATURE_FRED; /* * Create three vCPUs, but run them on the same task, to validate KVM's @@ -432,6 +472,14 @@ static void test_msrs(void) * immutable once the vCPU has been run. / for (idx = 0; idx < ARRAY_SIZE(__msrs); idx++) { + / + * Feature word 0x7:1:EAX used by FRED doesn't exist on some CPU + * models, simply skip clearing FRED feature bit. + / + if (!memcmp(&msrs[idx].feature, &feat_fred, sizeof(feat_fred)) && + !kvm_cpu_has(X86_FEATURE_FRED)) + continue; + / * Don't clear LM; selftests are 64-bit only, and KVM doesn't * honor LM=0 for MSRs that are supposed to exist if and only
31/3	2026/09/11 21:50	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 56427 output: 32 thoughts: 736 Thoughts: Analyzing the Patch for KMSAN Fuzzing Needs Okay, so the task at hand is to determine if this patch series, targeting amd64, warrants a dedicated KMSAN fuzzing session. Let's dig in. I need to understand exactly what's being changed here. The core seems to be about adding support for Intel FRED (Flexible Return and Event Delivery) within KVM, with some related refactoring in arch/x86 for exception stack handling. Specifically, I see changes impacting: 1. Documentation/virt/kvm/api.rst: Introduces new KVM capabilities – `KVM_CAP_X86_FRED_EVENT`, `KVM_VCPUEVENT_VALID_FRED_STATE`. Straightforward additions, but still important context. 2. arch/x86/coco/sev/noinstr.c, vc-handle.c: Re-naming VC and VC2 references to ESTACK\_VC and ESTACK\_VC2. Refactoring, again, but needs to be followed. 3. arch/x86/include/asm/cpu\_entry\_area.h: The `struct cea_exception_stacks` is being refactored to use `event_stacks[N_EXCEPTION_STACKS]`, along with new functions `__this_cpu_ist_top_va`. This touches memory structures. I need to understand why this change is happening. What's the impact on existing code and data structures? Are there new memory regions being introduced that KMSAN needs to be aware of? 4. arch/x86/include/asm/fred.h: Declaration of `this_cpu_fred_rsp`. Fine, seems like a necessary component for FRED support. 5. arch/x86/include/asm/kvm\_host.h: Key additions here. I see `fred_config`, `fred_stklvls`, `is_nested`, `event_data`, and `fred_ssp0_fallback` being added to several structs: `kvm_host_values`, `kvm_queued_exception`, and `kvm_vcpu_arch`. Also, `exception_fred_state_enabled` in `struct kvm_arch`. These additions are significant, implying new states and data paths related to FRED. These fields are high-priority. How are they used? How are they initialized? Are there any potential overflow or use-after-free issues? 6. arch/x86/include/asm/msr-index.h: FRED MSR macros. Standard stuff for MSR access, but worth noting for potential fuzzing targets that deal with MSRs. 7. arch/x86/include/asm/vmx.h: VMCS fields for FRED. VMX is always an interesting area for fuzzing, especially when new features are added. The changes here definitely increase the attack surface. 8. arch/x86/include/uapi/asm/kvm.h: This is where it gets interesting. There are modifications to `struct kvm_vcpu_events`, specifically, the addition of a `exception_event_data`, `exception_is_nested`, and adjustments of `reserved` fields and `exception_payload`. WAIT! `kvm_vcpu_events`...is that copied to user space? I need to check. If it's exposed to userspace, then this is a huge KMSAN concern. Uninitialized or incorrectly handled data in these fields could be exposed to untrusted code. My first task now is to verify how `kvm_vcpu_ioctl_x86_get_vcpu_events` is used, and more importantly, how the `events` member within `kvm_vcpu_events` is initialized and copied to user space. I'm going to search the codebase for that exact function name right now. Any uninitialized data copied to userspace would be a high-priority bug and a definite KMSAN trigger. This patch has a significant probability of needing a dedicated KMSAN session because of the interaction of hardware features with user-accessible data structures.
32/3	2026/09/11 21:50	tool	grepper	0m Args: { "Expression": "kvm_vcpu_ioctl_x86_get_vcpu_events" } Results: { "Output": "arch/x86/kvm/x86.c=2972=void kvm_handle_exception_payload_quirk(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/x86.c-2993-\narch/x86/kvm/x86.c:2994:static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu vcpu,\narch/x86/kvm/x86.c-2995-\t\t\t\t\t struct kvm_vcpu_events events)\n--\narch/x86/kvm/x86.c=3463=long kvm_arch_vcpu_ioctl(struct file filp,\n--\narch/x86/kvm/x86.c-3630-\narch/x86/kvm/x86.c:3631:\t\tkvm_vcpu_ioctl_x86_get_vcpu_events(vcpu, \u0026events);\narch/x86/kvm/x86.c-3632-\n--\narch/x86/kvm/x86.c=9333=static void store_regs(struct kvm_vcpu *vcpu)\n--\narch/x86/kvm/x86.c-9337-\tif (vcpu-\u003erun-\u003ekvm_valid_regs \u0026 KVM_SYNC_X86_EVENTS)\narch/x86/kvm/x86.c:9338:\t\tkvm_vcpu_ioctl_x86_get_vcpu_events(\narch/x86/kvm/x86.c-9339-\t\t\t\tvcpu, \u0026vcpu-\u003erun-\u003es.regs.events);\n" }
33/3	2026/09/11 21:50	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 58572 output: 43 thoughts: 40 Thoughts: Here's my summary, as if I were thinking these thoughts: Diving Deep into KVM's Architecture Alright, time to get my hands dirty. I want to zero in on a couple of key areas within KVM's architecture to understand how it's handling virtual CPU register manipulation. My focus will be on `kvm_arch_vcpu_ioctl` around line 3630 – that's a prime spot for examining the interaction between the user-space IOCTL calls and the underlying virtualization mechanism. Specifically, I'm interested in how it's handling the transfer of virtual CPU state, the data flow, and any potential points of interest for performance optimization or security hardening. Then I'll shift gears to `store_regs` around line 9333. This function likely deals with saving the virtual CPU's registers, which is crucial for context switching, migration, and handling various events. I need to understand the specifics of what registers are being saved, how the data is being stored, and the overall efficiency of this operation. Given the performance-critical nature of these functions, I'll pay close attention to the data structures involved, memory access patterns, and any optimizations that might be employed. I want to see this as it is being read in the file, line by line. Let's see how `read-file` handles these two points of interest!
34/3	2026/09/11 21:51	tool	read-file	0m Args: { "File": "arch/x86/kvm/x86.c", "FirstLine": 3625, "LineCount": 25 } Results: { "Contents": "3625:\t\t\tr = kvm_vcpu_ioctl_x86_set_mce(vcpu, \u0026mce);\n3626:\t\t\tbreak;\n3627:\t\t}\n3628:\t\tcase KVM_GET_VCPU_EVENTS: {\n3629:\t\t\tstruct kvm_vcpu_events events;\n3630:\t\n3631:\t\t\tkvm_vcpu_ioctl_x86_get_vcpu_events(vcpu, \u0026events);\n3632:\t\n3633:\t\t\tr = -EFAULT;\n3634:\t\t\tif (copy_to_user(argp, \u0026events, sizeof(struct kvm_vcpu_events)))\n3635:\t\t\t\tbreak;\n3636:\t\t\tr = 0;\n3637:\t\t\tbreak;\n3638:\t\t}\n3639:\t\tcase KVM_SET_VCPU_EVENTS: {\n3640:\t\t\tstruct kvm_vcpu_events events;\n3641:\t\n3642:\t\t\tr = -EFAULT;\n3643:\t\t\tif (copy_from_user(\u0026events, argp, sizeof(struct kvm_vcpu_events)))\n3644:\t\t\t\tbreak;\n3645:\t\n3646:\t\t\tkvm_vcpu_srcu_read_lock(vcpu);\n3647:\t\t\tr = kvm_vcpu_ioctl_x86_set_vcpu_events(vcpu, \u0026events);\n3648:\t\t\tkvm_vcpu_srcu_read_unlock(vcpu);\n3649:\t\t\tbreak;\n" }
35/3	2026/09/11 21:51	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 59368 output: 43 thoughts: 188 Thoughts: Examining Potential KVM Vulnerability Okay, here we are, diving into the KVM code. Line 3629... let's see, it's handling the `KVM_GET_VCPU_EVENTS` ioctl. Right, so we grab a `struct kvm_vcpu_events` called `events`, then call `kvm_vcpu_ioctl_x86_get_vcpu_events` and pass in the address of `events`. My immediate concern, as always, is memory safety. I see that `events` gets copied to user space using `copy_to_user`, and the return code `r` gets assigned `-EFAULT` if that copy fails, which makes sense. But wait a minute... looking closer at the `events` variable, I immediately wonder, is it initialized before that copy? Does `kvm_vcpu_ioctl_x86_get_vcpu_events` initialize it within the function? This is a critical point. If `events` isn't initialized before the `copy_to_user` call, we potentially have an uninitialized memory read vulnerability. We could be leaking kernel memory to userspace. Alright, I'm going to jump straight to the definition of `kvm_vcpu_ioctl_x86_get_vcpu_events`. I need to confirm whether or not it explicitly zeros out `events` with a `memset` or assigns meaningful values to each field before the structure is returned. No assumptions, I need to see the code. A missing initialization here is a glaring issue.
36/3	2026/09/11 21:51	tool	read-file	0m Args: { "File": "arch/x86/kvm/x86.c", "FirstLine": 2990, "LineCount": 75 } Results: { "Contents": "2990:\t\t ex-\u003epending \u0026\u0026 ex-\u003ehas_payload)\n2991:\t\t\tkvm_deliver_exception_payload(vcpu, ex);\n2992:\t}\n2993:\t\n2994:\tstatic void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu vcpu,\n2995:\t\t\t\t\t\t struct kvm_vcpu_events events)\n2996:\t{\n2997:\t\tstruct kvm_queued_exception ex = kvm_get_exception_to_save(vcpu);\n2998:\t\n2999:\t\tprocess_nmi(vcpu);\n3000:\t\n3001:\t#ifdef CONFIG_KVM_SMM\n3002:\t\tif (kvm_check_request(KVM_REQ_SMI, vcpu))\n3003:\t\t\tprocess_smi(vcpu);\n3004:\t#endif\n3005:\t\n3006:\t\tkvm_handle_exception_payload_quirk(vcpu);\n3007:\t\n3008:\t\tmemset(events, 0, sizeof(events));\n3009:\t\n3010:\t\t/\n3011:\t\t The API doesn't provide the instruction length for software\n3012:\t\t * exceptions, so don't report them. As long as the guest RIP\n3013:\t\t * isn't advanced, we should expect to encounter the exception\n3014:\t\t * again.\n3015:\t\t /\n3016:\t\tif (!kvm_exception_is_soft(ex-\u003evector)) {\n3017:\t\t\tevents-\u003eexception.injected = ex-\u003einjected;\n3018:\t\t\tevents-\u003eexception.pending = ex-\u003epending;\n3019:\t\t\t/\n3020:\t\t\t * For ABI compatibility, deliberately conflate\n3021:\t\t\t * pending and injected exceptions when\n3022:\t\t\t * KVM_CAP_EXCEPTION_PAYLOAD isn't enabled.\n3023:\t\t\t /\n3024:\t\t\tif (!vcpu-\u003ekvm-\u003earch.exception_payload_enabled)\n3025:\t\t\t\tevents-\u003eexception.injected \|= ex-\u003epending;\n3026:\t\t}\n3027:\t\tevents-\u003eexception.nr = ex-\u003evector;\n3028:\t\tevents-\u003eexception.has_error_code = ex-\u003ehas_error_code;\n3029:\t\tevents-\u003eexception.error_code = ex-\u003eerror_code;\n3030:\t\tevents-\u003eexception_has_payload = ex-\u003ehas_payload;\n3031:\t\tevents-\u003eexception_payload = ex-\u003epayload;\n3032:\t\tevents-\u003eexception_is_nested = ex-\u003eis_nested;\n3033:\t\tevents-\u003eexception_event_data = ex-\u003eevent_data;\n3034:\t\n3035:\t\tevents-\u003einterrupt.injected =\n3036:\t\t\tvcpu-\u003earch.interrupt.injected \u0026\u0026 !vcpu-\u003earch.interrupt.soft;\n3037:\t\tevents-\u003einterrupt.nr = vcpu-\u003earch.interrupt.nr;\n3038:\t\tevents-\u003einterrupt.shadow = kvm_x86_call(get_interrupt_shadow)(vcpu);\n3039:\t\n3040:\t\tevents-\u003enmi.injected = vcpu-\u003earch.nmi_injected;\n3041:\t\tevents-\u003enmi.pending = kvm_get_nr_pending_nmis(vcpu);\n3042:\t\tevents-\u003enmi.masked = kvm_x86_call(get_nmi_mask)(vcpu);\n3043:\t\n3044:\t\t/ events-\u003esipi_vector is never valid when reporting to user space */\n3045:\t\n3046:\t#ifdef CONFIG_KVM_SMM\n3047:\t\tevents-\u003esmi.smm = is_smm(vcpu);\n3048:\t\tevents-\u003esmi.pending = vcpu-\u003earch.smi_pending;\n3049:\t\tevents-\u003esmi.smm_inside_nmi =\n3050:\t\t\t!!(vcpu-\u003earch.hflags \u0026 HF_SMM_INSIDE_NMI_MASK);\n3051:\t#endif\n3052:\t\tevents-\u003esmi.latched_init = kvm_lapic_latched_init(vcpu);\n3053:\t\n3054:\t\tevents-\u003eflags = (KVM_VCPUEVENT_VALID_NMI_PENDING\n3055:\t\t\t\t \| KVM_VCPUEVENT_VALID_SHADOW\n3056:\t\t\t\t \| KVM_VCPUEVENT_VALID_SMM);\n3057:\t\tif (vcpu-\u003ekvm-\u003earch.exception_payload_enabled)\n3058:\t\t\tevents-\u003eflags \|= KVM_VCPUEVENT_VALID_PAYLOAD;\n3059:\t\tif (vcpu-\u003ekvm-\u003earch.exception_fred_state_enabled)\n3060:\t\t\tevents-\u003eflags \|= KVM_VCPUEVENT_VALID_FRED_STATE;\n3061:\t\tif (vcpu-\u003ekvm-\u003earch.triple_fault_event) {\n3062:\t\t\tevents-\u003etriple_fault.pending = kvm_test_request(KVM_REQ_TRIPLE_FAULT, vcpu);\n3063:\t\t\tevents-\u003eflags \|= KVM_VCPUEVENT_VALID_TRIPLE_FAULT;\n3064:\t\t}\n" }
37/3	2026/09/11 21:51	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 61320 output: 20 thoughts: 251 Thoughts: My Analysis of Potential Uninitialized Reads Okay, so I see `memset(events, 0, sizeof(events))` being called at line 3008. That's good, it explicitly zeroes out the entire `events` structure, including all padding. That’s standard practice for structures where we need to ensure a known initial state, before any further assignment. Line 9338 also calls `memset(events, 0, sizeof(events))` which is the call `kvm_vcpu_ioctl_x86_get_vcpu_events(vcpu, &vcpu->run->s.regs.events);`. No big deal. But hold on, my concern is around the potential for uninitialized reads, specifically related to how `is_nested` and `event_data` get set. We know that data in those fields is ultimately assigned to `events` structure. We are setting up to save those values. I need to trace back where the `struct kvm_queued_exception` object, represented by `ex` in the code, is allocated and initialized. I need to be sure that `ex->is_nested` and `ex->event_data` are properly initialized before they're potentially used. I'm going to look for any uninitialized reads of `ex->is_nested` or `ex->event_data`, as those can lead to significant vulnerabilities. I need to search the code base for allocations or initializations involving `struct kvm_queued_exception` or instances like `vcpu->arch.exception`, to understand the control flow. If these values don't get initialized before they are assigned to, it's a potential bug.
38/3	2026/09/11 21:51	tool	grepper	0m Args: { "Expression": "vcpu-\u003earch.exception" } Results: { "Output": "arch/loongarch/include/asm/kvm_vcpu.h=123=static inline int kvm_queue_exception(struct kvm_vcpu vcpu,\n--\narch/loongarch/include/asm/kvm_vcpu.h-126-\t/ only one exception can be injected /\narch/loongarch/include/asm/kvm_vcpu.h:127:\tif (!vcpu-\u003earch.exception_pending) {\narch/loongarch/include/asm/kvm_vcpu.h:128:\t\tset_bit(code, \u0026vcpu-\u003earch.exception_pending);\narch/loongarch/include/asm/kvm_vcpu.h-129-\t\tvcpu-\u003earch.esubcode = subcode;\n--\narch/loongarch/kvm/interrupt.c=136=void kvm_deliver_exception(struct kvm_vcpu vcpu)\n--\narch/loongarch/kvm/interrupt.c-138-\tunsigned int code;\narch/loongarch/kvm/interrupt.c:139:\tunsigned long pending = \u0026vcpu-\u003earch.exception_pending;\narch/loongarch/kvm/interrupt.c-140-\n--\narch/x86/kvm/regs.c=102=static void __set_regs(struct kvm_vcpu vcpu, struct kvm_regs regs)\n--\narch/x86/kvm/regs.c-128-\narch/x86/kvm/regs.c:129:\tvcpu-\u003earch.exception.pending = false;\narch/x86/kvm/regs.c:130:\tvcpu-\u003earch.exception_vmexit.pending = false;\narch/x86/kvm/regs.c-131-\n--\narch/x86/kvm/svm/nested.c=655=static void nested_save_pending_event_to_vmcb12(struct vcpu_svm svm,\n--\narch/x86/kvm/svm/nested.c-661-\narch/x86/kvm/svm/nested.c:662:\tif (vcpu-\u003earch.exception.injected) {\narch/x86/kvm/svm/nested.c:663:\t\tnr = vcpu-\u003earch.exception.vector;\narch/x86/kvm/svm/nested.c-664-\t\texit_int_info = nr \| SVM_EVTINJ_VALID \| SVM_EVTINJ_TYPE_EXEPT;\narch/x86/kvm/svm/nested.c-665-\narch/x86/kvm/svm/nested.c:666:\t\tif (vcpu-\u003earch.exception.has_error_code) {\narch/x86/kvm/svm/nested.c-667-\t\t\texit_int_info \|= SVM_EVTINJ_VALID_ERR;\narch/x86/kvm/svm/nested.c-668-\t\t\tvmcb12-\u003econtrol.exit_int_info_err =\narch/x86/kvm/svm/nested.c:669:\t\t\t\tvcpu-\u003earch.exception.error_code;\narch/x86/kvm/svm/nested.c-670-\t\t}\n--\narch/x86/kvm/svm/nested.c=1681=static void nested_svm_inject_exception_vmexit(struct kvm_vcpu vcpu)\narch/x86/kvm/svm/nested.c-1682-{\narch/x86/kvm/svm/nested.c:1683:\tstruct kvm_queued_exception ex = \u0026vcpu-\u003earch.exception_vmexit;\narch/x86/kvm/svm/nested.c-1684-\tstruct vcpu_svm svm = to_svm(vcpu);\n--\narch/x86/kvm/svm/nested.c=1721=static int svm_check_nested_events(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/svm/nested.c-1749-\narch/x86/kvm/svm/nested.c:1750:\tif (vcpu-\u003earch.exception_vmexit.pending) {\narch/x86/kvm/svm/nested.c-1751-\t\tif (block_nested_exceptions)\n--\narch/x86/kvm/svm/nested.c-1756-\narch/x86/kvm/svm/nested.c:1757:\tif (vcpu-\u003earch.exception.pending) {\narch/x86/kvm/svm/nested.c-1758-\t\tif (block_nested_exceptions)\n--\narch/x86/kvm/svm/svm.c=389=static void svm_inject_exception(struct kvm_vcpu vcpu)\narch/x86/kvm/svm/svm.c-390-{\narch/x86/kvm/svm/svm.c:391:\tstruct kvm_queued_exception ex = \u0026vcpu-\u003earch.exception;\narch/x86/kvm/svm/svm.c-392-\tstruct vcpu_svm svm = to_svm(vcpu);\n--\narch/x86/kvm/vmx/nested.c=4122=static void vmcs12_save_pending_event(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/vmx/nested.c-4151-\t\tvmcs12-\u003eidt_vectoring_info_field = 0;\narch/x86/kvm/vmx/nested.c:4152:\t} else if (vcpu-\u003earch.exception.injected) {\narch/x86/kvm/vmx/nested.c:4153:\t\tnr = vcpu-\u003earch.exception.vector;\narch/x86/kvm/vmx/nested.c-4154-\t\tidt_vectoring = nr \| VECTORING_INFO_VALID_MASK;\n--\narch/x86/kvm/vmx/nested.c-4162-\narch/x86/kvm/vmx/nested.c:4163:\t\tif (vcpu-\u003earch.exception.has_error_code) {\narch/x86/kvm/vmx/nested.c-4164-\t\t\tidt_vectoring \|= VECTORING_INFO_DELIVER_CODE_MASK;\narch/x86/kvm/vmx/nested.c-4165-\t\t\tvmcs12-\u003eidt_vectoring_error_code =\narch/x86/kvm/vmx/nested.c:4166:\t\t\t\tvcpu-\u003earch.exception.error_code;\narch/x86/kvm/vmx/nested.c-4167-\t\t}\n--\narch/x86/kvm/vmx/nested.c-4170-\t\t (vmcs12-\u003eguest_cr4 \u0026 X86_CR4_FRED) \u0026\u0026\narch/x86/kvm/vmx/nested.c:4171:\t\t (vcpu-\u003earch.exception.is_nested))\narch/x86/kvm/vmx/nested.c-4172-\t\t\tidt_vectoring \|= VECTORING_INFO_NESTED_EXCEPTION_MASK;\narch/x86/kvm/vmx/nested.c-4173-\narch/x86/kvm/vmx/nested.c:4174:\t\tvmcs12-\u003eoriginal_event_data = vcpu-\u003earch.exception.event_data;\narch/x86/kvm/vmx/nested.c-4175-\n--\narch/x86/kvm/vmx/nested.c=4240=static void nested_vmx_inject_exception_vmexit(struct kvm_vcpu vcpu)\narch/x86/kvm/vmx/nested.c-4241-{\narch/x86/kvm/vmx/nested.c:4242:\tstruct kvm_queued_exception ex = \u0026vcpu-\u003earch.exception_vmexit;\narch/x86/kvm/vmx/nested.c-4243-\tu32 intr_info = ex-\u003evector \| INTR_INFO_VALID_MASK;\n--\narch/x86/kvm/vmx/nested.c=4333=static void nested_vmx_update_pending_dbg(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-4336-\narch/x86/kvm/vmx/nested.c:4337:\tpending_dbg = vmx_get_pending_dbg_trap(\u0026vcpu-\u003earch.exception);\narch/x86/kvm/vmx/nested.c-4338-\tif (pending_dbg)\n--\narch/x86/kvm/vmx/nested.c=4473=static int vmx_check_nested_events(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-4541-\t /\narch/x86/kvm/vmx/nested.c:4542:\tif (vcpu-\u003earch.exception_vmexit.pending \u0026\u0026\narch/x86/kvm/vmx/nested.c:4543:\t !vmx_is_low_priority_db_trap(\u0026vcpu-\u003earch.exception_vmexit)) {\narch/x86/kvm/vmx/nested.c-4544-\t\tif (block_nested_exceptions)\n--\narch/x86/kvm/vmx/nested.c-4550-\narch/x86/kvm/vmx/nested.c:4551:\tif (vcpu-\u003earch.exception.pending \u0026\u0026\narch/x86/kvm/vmx/nested.c:4552:\t !vmx_is_low_priority_db_trap(\u0026vcpu-\u003earch.exception)) {\narch/x86/kvm/vmx/nested.c-4553-\t\tif (block_nested_exceptions)\n--\narch/x86/kvm/vmx/nested.c-4565-\narch/x86/kvm/vmx/nested.c:4566:\tif (vcpu-\u003earch.exception_vmexit.pending) {\narch/x86/kvm/vmx/nested.c-4567-\t\tif (block_nested_exceptions)\n--\narch/x86/kvm/vmx/nested.c-4573-\narch/x86/kvm/vmx/nested.c:4574:\tif (vcpu-\u003earch.exception.pending) {\narch/x86/kvm/vmx/nested.c-4575-\t\tif (block_nested_exceptions)\n--\narch/x86/kvm/vmx/vmx.c=1909=void vmx_update_emulated_instruction(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/vmx.c-1928-\tif (nested_cpu_has_mtf(vmcs12) \u0026\u0026\narch/x86/kvm/vmx/vmx.c:1929:\t (!vcpu-\u003earch.exception.pending \|\|\narch/x86/kvm/vmx/vmx.c:1930:\t vcpu-\u003earch.exception.vector == DB_VECTOR) \u0026\u0026\narch/x86/kvm/vmx/vmx.c:1931:\t (!vcpu-\u003earch.exception_vmexit.pending \|\|\narch/x86/kvm/vmx/vmx.c:1932:\t vcpu-\u003earch.exception_vmexit.vector == DB_VECTOR)) {\narch/x86/kvm/vmx/vmx.c-1933-\t\tvmx-\u003enested.mtf_pending = true;\n--\narch/x86/kvm/vmx/vmx.c=1959=void vmx_inject_exception(struct kvm_vcpu vcpu)\narch/x86/kvm/vmx/vmx.c-1960-{\narch/x86/kvm/vmx/vmx.c:1961:\tstruct kvm_queued_exception ex = \u0026vcpu-\u003earch.exception;\narch/x86/kvm/vmx/vmx.c-1962-\tu32 intr_info = ex-\u003evector \| INTR_INFO_VALID_MASK;\n--\narch/x86/kvm/vmx/vmx.c=6268=bool vmx_unhandleable_emulation_required(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/vmx.c-6291-\treturn !vmx-\u003ermode.vm86_active \u0026\u0026\narch/x86/kvm/vmx/vmx.c:6292:\t (kvm_is_exception_pending(vcpu) \|\| vcpu-\u003earch.exception.injected);\narch/x86/kvm/vmx/vmx.c-6293-}\n--\narch/x86/kvm/x86.c=458=static void kvm_queue_exception_vmexit(struct kvm_vcpu vcpu, unsigned int vector,\n--\narch/x86/kvm/x86.c-461-{\narch/x86/kvm/x86.c:462:\tstruct kvm_queued_exception ex = \u0026vcpu-\u003earch.exception_vmexit;\narch/x86/kvm/x86.c-463-\n--\narch/x86/kvm/x86.c=473=static void kvm_multiple_exception(struct kvm_vcpu vcpu, unsigned int nr,\n--\narch/x86/kvm/x86.c-493-\narch/x86/kvm/x86.c:494:\tif (!vcpu-\u003earch.exception.pending \u0026\u0026 !vcpu-\u003earch.exception.injected) {\narch/x86/kvm/x86.c-495-\tqueue:\narch/x86/kvm/x86.c:496:\t\tvcpu-\u003earch.exception.pending = true;\narch/x86/kvm/x86.c:497:\t\tvcpu-\u003earch.exception.injected = false;\narch/x86/kvm/x86.c-498-\narch/x86/kvm/x86.c:499:\t\tvcpu-\u003earch.exception.has_error_code = has_error;\narch/x86/kvm/x86.c:500:\t\tvcpu-\u003earch.exception.vector = nr;\narch/x86/kvm/x86.c:501:\t\tvcpu-\u003earch.exception.error_code = error_code;\narch/x86/kvm/x86.c:502:\t\tvcpu-\u003earch.exception.has_payload = has_payload;\narch/x86/kvm/x86.c:503:\t\tvcpu-\u003earch.exception.payload = payload;\narch/x86/kvm/x86.c-504-\t\t/* #DF is NOT a nested event, per its definition. /\narch/x86/kvm/x86.c:505:\t\tvcpu-\u003earch.exception.is_nested = (nr != DF_VECTOR) \u0026\u0026\narch/x86/kvm/x86.c-506-\t\t\t\t\t\t (is_nested \|\|\n--\narch/x86/kvm/x86.c-516-\t/ to check exception /\narch/x86/kvm/x86.c:517:\tprev_nr = vcpu-\u003earch.exception.vector;\narch/x86/kvm/x86.c-518-\tif (prev_nr == DF_VECTOR) {\n--\narch/x86/kvm/x86.c-530-\t\t /\narch/x86/kvm/x86.c:531:\t\tvcpu-\u003earch.exception.injected = false;\narch/x86/kvm/x86.c:532:\t\tvcpu-\u003earch.exception.pending = false;\narch/x86/kvm/x86.c-533-\n--\narch/x86/kvm/x86.c=566=void kvm_requeue_exception(struct kvm_vcpu vcpu, unsigned int nr,\n--\narch/x86/kvm/x86.c-586-\narch/x86/kvm/x86.c:587:\tvcpu-\u003earch.exception.injected = true;\narch/x86/kvm/x86.c:588:\tvcpu-\u003earch.exception.has_error_code = has_error_code;\narch/x86/kvm/x86.c:589:\tvcpu-\u003earch.exception.vector = nr;\narch/x86/kvm/x86.c:590:\tvcpu-\u003earch.exception.error_code = error_code;\narch/x86/kvm/x86.c:591:\tvcpu-\u003earch.exception.has_payload = false;\narch/x86/kvm/x86.c:592:\tvcpu-\u003earch.exception.payload = 0;\narch/x86/kvm/x86.c:593:\tvcpu-\u003earch.exception.is_nested = is_nested;\narch/x86/kvm/x86.c:594:\tvcpu-\u003earch.exception.event_data = event_data;\narch/x86/kvm/x86.c-595-}\n--\narch/x86/kvm/x86.c=2955=static struct kvm_queued_exception kvm_get_exception_to_save(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/x86.c-2963-\t /\narch/x86/kvm/x86.c:2964:\tif (vcpu-\u003earch.exception_vmexit.pending \u0026\u0026\narch/x86/kvm/x86.c:2965:\t !vcpu-\u003earch.exception.pending \u0026\u0026\narch/x86/kvm/x86.c:2966:\t !vcpu-\u003earch.exception.injected)\narch/x86/kvm/x86.c:2967:\t\treturn \u0026vcpu-\u003earch.exception_vmexit;\narch/x86/kvm/x86.c-2968-\narch/x86/kvm/x86.c:2969:\treturn \u0026vcpu-\u003earch.exception;\narch/x86/kvm/x86.c-2970-}\n--\narch/x86/kvm/x86.c=3067=static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/x86.c-3112-\t /\narch/x86/kvm/x86.c:3113:\tvcpu-\u003earch.exception_from_userspace = events-\u003eexception.pending;\narch/x86/kvm/x86.c-3114-\narch/x86/kvm/x86.c:3115:\tvcpu-\u003earch.exception_vmexit.pending = false;\narch/x86/kvm/x86.c-3116-\narch/x86/kvm/x86.c:3117:\tvcpu-\u003earch.exception.injected = events-\u003eexception.injected;\narch/x86/kvm/x86.c:3118:\tvcpu-\u003earch.exception.pending = events-\u003eexception.pending;\narch/x86/kvm/x86.c:3119:\tvcpu-\u003earch.exception.vector = events-\u003eexception.nr;\narch/x86/kvm/x86.c:3120:\tvcpu-\u003earch.exception.has_error_code = events-\u003eexception.has_error_code;\narch/x86/kvm/x86.c:3121:\tvcpu-\u003earch.exception.error_code = events-\u003eexception.error_code;\narch/x86/kvm/x86.c:3122:\tvcpu-\u003earch.exception.has_payload = events-\u003eexception_has_payload;\narch/x86/kvm/x86.c:3123:\tvcpu-\u003earch.exception.payload = events-\u003eexception_payload;\narch/x86/kvm/x86.c:3124:\tvcpu-\u003earch.exception.is_nested = events-\u003eexception_is_nested;\narch/x86/kvm/x86.c:3125:\tvcpu-\u003earch.exception.event_data = events-\u003eexception_event_data;\narch/x86/kvm/x86.c-3126-\n--\narch/x86/kvm/x86.c=7574=static void kvm_inject_exception(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/x86.c-7582-\t /\narch/x86/kvm/x86.c:7583:\tvcpu-\u003earch.exception.has_error_code \u0026= is_protmode(vcpu);\narch/x86/kvm/x86.c-7584-\narch/x86/kvm/x86.c:7585:\ttrace_kvm_inj_exception(vcpu-\u003earch.exception.vector,\narch/x86/kvm/x86.c:7586:\t\t\t\tvcpu-\u003earch.exception.has_error_code,\narch/x86/kvm/x86.c:7587:\t\t\t\tvcpu-\u003earch.exception.error_code,\narch/x86/kvm/x86.c:7588:\t\t\t\tvcpu-\u003earch.exception.injected);\narch/x86/kvm/x86.c-7589-\n--\narch/x86/kvm/x86.c=7632=static int kvm_check_and_inject_events(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/x86.c-7670-\t /\narch/x86/kvm/x86.c:7671:\tif (vcpu-\u003earch.exception.injected)\narch/x86/kvm/x86.c-7672-\t\tkvm_inject_exception(vcpu);\n--\narch/x86/kvm/x86.c-7684-\t /\narch/x86/kvm/x86.c:7685:\tWARN_ON_ONCE(vcpu-\u003earch.exception.injected \u0026\u0026\narch/x86/kvm/x86.c:7686:\t\t vcpu-\u003earch.exception.pending);\narch/x86/kvm/x86.c-7687-\n--\narch/x86/kvm/x86.c-7704-\t /\narch/x86/kvm/x86.c:7705:\tWARN_ON_ONCE(vcpu-\u003earch.exception_vmexit.injected \|\|\narch/x86/kvm/x86.c:7706:\t\t vcpu-\u003earch.exception_vmexit.pending);\narch/x86/kvm/x86.c-7707-\n--\narch/x86/kvm/x86.c-7714-\narch/x86/kvm/x86.c:7715:\tif (vcpu-\u003earch.exception.pending) {\narch/x86/kvm/x86.c-7716-\t\t/\n--\narch/x86/kvm/x86.c-7725-\t\t /\narch/x86/kvm/x86.c:7726:\t\tif (exception_type(vcpu-\u003earch.exception.vector) == EXCPT_FAULT)\narch/x86/kvm/x86.c-7727-\t\t\t__kvm_set_rflags(vcpu, kvm_get_rflags(vcpu) \|\n--\narch/x86/kvm/x86.c-7729-\narch/x86/kvm/x86.c:7730:\t\tif (vcpu-\u003earch.exception.vector == DB_VECTOR \u0026\u0026\narch/x86/kvm/x86.c-7731-\t\t vcpu-\u003earch.dr7 \u0026 DR7_GD) {\n--\narch/x86/kvm/x86.c-7737-\narch/x86/kvm/x86.c:7738:\t\tvcpu-\u003earch.exception.pending = false;\narch/x86/kvm/x86.c:7739:\t\tvcpu-\u003earch.exception.injected = true;\narch/x86/kvm/x86.c-7740-\n--\narch/x86/kvm/x86.c-7825-\t /\narch/x86/kvm/x86.c:7826:\tWARN_ON_ONCE(vcpu-\u003earch.exception.pending \|\|\narch/x86/kvm/x86.c:7827:\t\t vcpu-\u003earch.exception_vmexit.pending);\narch/x86/kvm/x86.c-7828-\treturn 0;\n--\narch/x86/kvm/x86.c=8941=int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu vcpu)\narch/x86/kvm/x86.c-8942-{\narch/x86/kvm/x86.c:8943:\tstruct kvm_queued_exception ex = \u0026vcpu-\u003earch.exception;\narch/x86/kvm/x86.c-8944-\tstruct kvm_run kvm_run = vcpu-\u003erun;\n--\narch/x86/kvm/x86.c-9015-\t /\narch/x86/kvm/x86.c:9016:\tif (vcpu-\u003earch.exception_from_userspace \u0026\u0026 is_guest_mode(vcpu) \u0026\u0026\narch/x86/kvm/x86.c-9017-\t kvm_nested_call(is_exception_vmexit)(vcpu, ex-\u003evector, ex-\u003eerror_code)) {\n--\narch/x86/kvm/x86.c-9024-\t}\narch/x86/kvm/x86.c:9025:\tvcpu-\u003earch.exception_from_userspace = false;\narch/x86/kvm/x86.c-9026-\n--\narch/x86/kvm/x86.c=9201=int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/x86.c-9213-\t\tr = -EBUSY;\narch/x86/kvm/x86.c:9214:\t\tif (kvm_is_exception_pending(vcpu) \|\| vcpu-\u003earch.exception.injected)\narch/x86/kvm/x86.c-9215-\t\t\tgoto out;\n--\narch/x86/kvm/x86.h=150=static inline bool kvm_is_exception_pending(struct kvm_vcpu vcpu)\narch/x86/kvm/x86.h-151-{\narch/x86/kvm/x86.h:152:\treturn vcpu-\u003earch.exception.pending \|\|\narch/x86/kvm/x86.h:153:\t vcpu-\u003earch.exception_vmexit.pending \|\|\narch/x86/kvm/x86.h-154-\t kvm_test_request(KVM_REQ_TRIPLE_FAULT, vcpu);\n--\narch/x86/kvm/x86.h=157=static inline void kvm_clear_exception_queue(struct kvm_vcpu vcpu)\narch/x86/kvm/x86.h-158-{\narch/x86/kvm/x86.h:159:\tvcpu-\u003earch.exception.pending = false;\narch/x86/kvm/x86.h:160:\tvcpu-\u003earch.exception.injected = false;\narch/x86/kvm/x86.h:161:\tvcpu-\u003earch.exception.is_nested = false;\narch/x86/kvm/x86.h:162:\tvcpu-\u003earch.exception.event_data = 0;\narch/x86/kvm/x86.h:163:\tvcpu-\u003earch.exception_vmexit.pending = false;\narch/x86/kvm/x86.h-164-}\n--\narch/x86/kvm/x86.h=179=static inline bool kvm_event_needs_reinjection(struct kvm_vcpu *vcpu)\narch/x86/kvm/x86.h-180-{\narch/x86/kvm/x86.h:181:\treturn vcpu-\u003earch.exception.injected \|\| vcpu-\u003earch.interrupt.injected \|\|\narch/x86/kvm/x86.h-182-\t\tvcpu-\u003earch.nmi_injected;\n" }
39/3	2026/09/11 21:51	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 69620 output: 23 thoughts: 59 Thoughts: Here's my summary, as though I were thinking through the problem: Delving into Virtual CPU Allocation in KVM Okay, so I need to figure out the memory allocation strategy for the Virtual CPUs (`vcpu`) within KVM. My understanding is that it's being handled by either `kmem_cache_zalloc` or `kzalloc`, but I need to verify this and understand the exact mechanics. Let's dig in and pinpoint exactly how and where these `vcpu` structures are allocated. My first instinct is to locate the function responsible for creating these `vcpu` instances. Let's use `grep` and search for `kvm_arch_vcpu_create`. That should give me a good starting point and direct me to the code that matters. From there, I can trace back and understand what allocation strategy is being employed.
40/3	2026/09/11 21:51	tool	grepper	0m Args: { "Expression": "kvm_arch_vcpu_create" } Results: { "Output": "arch/arm64/kvm/arm.c=526=int kvm_arch_vcpu_precreate(struct kvm kvm, unsigned int id)\n--\narch/arm64/kvm/arm.c-536-\narch/arm64/kvm/arm.c:537:int kvm_arch_vcpu_create(struct kvm_vcpu vcpu)\narch/arm64/kvm/arm.c-538-{\n--\narch/arm64/kvm/vgic/vgic-init.c=71=int kvm_vgic_create(struct kvm kvm, u32 type)\n--\narch/arm64/kvm/vgic/vgic-init.c-112-\t - Bailing on the entire thing if a vCPU is in the middle of creation,\narch/arm64/kvm/vgic/vgic-init.c:113:\t * dropped the kvm-\u003elock, but hasn't reached kvm_arch_vcpu_create().\narch/arm64/kvm/vgic/vgic-init.c-114-\t \n--\narch/loongarch/kvm/vcpu.c=1508=int kvm_arch_vcpu_precreate(struct kvm kvm, unsigned int id)\n--\narch/loongarch/kvm/vcpu.c-1512-\narch/loongarch/kvm/vcpu.c:1513:int kvm_arch_vcpu_create(struct kvm_vcpu vcpu)\narch/loongarch/kvm/vcpu.c-1514-{\n--\narch/mips/kvm/mips.c=273=int kvm_arch_vcpu_precreate(struct kvm kvm, unsigned int id)\n--\narch/mips/kvm/mips.c-277-\narch/mips/kvm/mips.c:278:int kvm_arch_vcpu_create(struct kvm_vcpu vcpu)\narch/mips/kvm/mips.c-279-{\n--\narch/powerpc/kvm/powerpc.c=746=static enum hrtimer_restart kvmppc_decrementer_wakeup(struct hrtimer timer)\n--\narch/powerpc/kvm/powerpc.c-755-\narch/powerpc/kvm/powerpc.c:756:int kvm_arch_vcpu_create(struct kvm_vcpu vcpu)\narch/powerpc/kvm/powerpc.c-757-{\n--\narch/riscv/kvm/vcpu.c=139=int kvm_arch_vcpu_precreate(struct kvm kvm, unsigned int id)\n--\narch/riscv/kvm/vcpu.c-143-\narch/riscv/kvm/vcpu.c:144:int kvm_arch_vcpu_create(struct kvm_vcpu vcpu)\narch/riscv/kvm/vcpu.c-145-{\n--\narch/s390/kvm/s390/s390.c=3779=DEFINE_FREE(sie_page, struct sie_page , if (_T) free_page((unsigned long)(_T)))\narch/s390/kvm/s390/s390.c-3780-\narch/s390/kvm/s390/s390.c:3781:int kvm_arch_vcpu_create(struct kvm_vcpu vcpu)\narch/s390/kvm/s390/s390.c-3782-{\n--\narch/x86/kvm/vmx/tdx.c=827=void tdx_vcpu_put(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/tdx.c-842- * 3.1 tdx_vcpu_create() transitions vCPU state to VCPU_TD_STATE_UNINITIALIZED.\narch/x86/kvm/vmx/tdx.c:843: * 3.2 vcpu_load() and vcpu_put() in kvm_arch_vcpu_create().\narch/x86/kvm/vmx/tdx.c:844: * 3.3 (conditional) if any error encountered after kvm_arch_vcpu_create()\narch/x86/kvm/vmx/tdx.c-845- * kvm_arch_vcpu_destroy() --\u003e tdx_vcpu_free().\n--\narch/x86/kvm/x86.c=9362=int kvm_arch_vcpu_precreate(struct kvm kvm, unsigned int id)\n--\narch/x86/kvm/x86.c-9400-\narch/x86/kvm/x86.c:9401:int kvm_arch_vcpu_create(struct kvm_vcpu vcpu)\narch/x86/kvm/x86.c-9402-{\n--\ninclude/linux/kvm_host.h=1649=int kvm_arch_vcpu_precreate(struct kvm kvm, unsigned int id);\ninclude/linux/kvm_host.h:1650:int kvm_arch_vcpu_create(struct kvm_vcpu vcpu);\ninclude/linux/kvm_host.h-1651-void kvm_arch_vcpu_postcreate(struct kvm_vcpu vcpu);\n--\nvirt/kvm/kvm_main.c=4156=static int kvm_vm_ioctl_create_vcpu(struct kvm kvm, unsigned long id)\n--\nvirt/kvm/kvm_main.c-4212-\nvirt/kvm/kvm_main.c:4213:\tr = kvm_arch_vcpu_create(vcpu);\nvirt/kvm/kvm_main.c-4214-\tif (r)\n" }
41/3	2026/09/11 21:51	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 71442 output: 40 thoughts: 33 Thoughts: Okay, here's my take on those thoughts, framed as an expert's internal monologue: Investigating KVM VCPU Allocation Alright, let's dive into this. I'm thinking, specifically, about how the `vcpu` structure gets instantiated within the KVM core. My gut says the details are probably in `virt/kvm/kvm_main.c`, and I'm pretty sure it's around line 4212, give or take a few lines. That seems like a likely spot for the main VCPU allocation routine. The real question is, what's the mechanics? Is it `kmalloc`? `kzalloc`? Maybe some custom allocator tied to the VM's memory context? I need to get a clear picture of that allocation process to understand how resources are managed. I'm expecting some kind of slab allocator usage there.
42/3	2026/09/11 21:51	tool	read-file	0m Args: { "File": "virt/kvm/kvm_main.c", "FirstLine": 4180, "LineCount": 40 } Results: { "Contents": "4180:\t\tif (test_bit(id, kvm-\u003evcpu_ids)) {\n4181:\t\t\tmutex_unlock(\u0026kvm-\u003elock);\n4182:\t\t\treturn -EEXIST;\n4183:\t\t}\n4184:\t\n4185:\t\tr = kvm_arch_vcpu_precreate(kvm, id);\n4186:\t\tif (r) {\n4187:\t\t\tmutex_unlock(\u0026kvm-\u003elock);\n4188:\t\t\treturn r;\n4189:\t\t}\n4190:\t\n4191:\t\tkvm-\u003ecreated_vcpus++;\n4192:\t\t__set_bit(id, kvm-\u003evcpu_ids);\n4193:\t\tmutex_unlock(\u0026kvm-\u003elock);\n4194:\t\n4195:\t\tvcpu = kmem_cache_zalloc(kvm_vcpu_cache, GFP_KERNEL_ACCOUNT);\n4196:\t\tif (!vcpu) {\n4197:\t\t\tr = -ENOMEM;\n4198:\t\t\tgoto vcpu_decrement;\n4199:\t\t}\n4200:\t\n4201:\t\tvcpu-\u003evcpu_idx = -1;\n4202:\t\n4203:\t\tBUILD_BUG_ON(sizeof(struct kvm_run) \u003e PAGE_SIZE);\n4204:\t\tpage = alloc_page(GFP_KERNEL_ACCOUNT \| __GFP_ZERO);\n4205:\t\tif (!page) {\n4206:\t\t\tr = -ENOMEM;\n4207:\t\t\tgoto vcpu_free;\n4208:\t\t}\n4209:\t\tvcpu-\u003erun = page_address(page);\n4210:\t\n4211:\t\tkvm_vcpu_init(vcpu, kvm, id);\n4212:\t\n4213:\t\tr = kvm_arch_vcpu_create(vcpu);\n4214:\t\tif (r)\n4215:\t\t\tgoto vcpu_free_run_page;\n4216:\t\n4217:\t\tif (kvm-\u003edirty_ring_size) {\n4218:\t\t\tr = kvm_dirty_ring_alloc(kvm, \u0026vcpu-\u003edirty_ring,\n4219:\t\t\t\t\t\t id, kvm-\u003edirty_ring_size);\n" }
43/3	2026/09/11 21:51	llm	kmsan-evaluator	1m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 72310 output: 39 thoughts: 424 Thoughts: My Analysis of Exception Handling and Nested VMX in KVM Okay, let's break this down. The `vcpu` itself is zero-initialized using `kmem_cache_zalloc`, which is good to know. Similarly, `vcpu->run` gets zeroed with `__GFP_ZERO`. The `kvm_clear_exception_queue` function ensures `is_nested` is `false` and `event_data` is cleared. This gives us a known starting state. Now, looking at exception nesting, `kvm_multiple_exception` sets `vcpu->arch.exception.is_nested` based on a few conditions, including the current state of `is_nested`. `kvm_requeue_exception` explicitly sets `vcpu->arch.exception.is_nested` and `event_data`, which seems straightforward. `kvm_deliver_exception_payload` is interesting because it conditionally modifies `ex->event_data`, masking out a bit, or using the entire payload, or setting it to zero. This could be where some subtleties are. The `KVM_VCPUEVENT_VALID_FRED_STATE` flag in `kvm_vcpu_ioctl_x86_set_vcpu_events` controls whether we zero `exception_is_nested` and `exception_event_data` or read them from userspace. The `kvm_vcpu_ioctl_x86_get_vcpu_events` function gets the state of nested and event data. This ioctl gives the state of the registers. Now, let's shift gears to nested VMX. I need to understand how nested VMX interacts with this exception handling. My immediate questions are around `vmcs12`. How is this structure allocated? Is it copied, accessed, or managed? Let's check `struct vmcs12` and, specifically, the `nested.c` file. I'll search for how `alloc_loaded_vmcs` is used or other code related to allocation or copying of `vmcs12`. This should reveal how the nested VMCS is initialized and managed.
44/3	2026/09/11 21:52	tool	grepper	0m Args: { "Expression": "vmcs12", "PathPrefix": "arch/x86/kvm/vmx/nested.c" } Results: { "Output": "\nFull output is too long, showing 500 out of 2760 lines.\nUse more precise expression if possible.\n\narch/x86/kvm/vmx/nested.c=59=static struct shadow_vmcs_field shadow_read_only_fields[] = {\narch/x86/kvm/vmx/nested.c:60:#define SHADOW_FIELD_RO(x, y) { x, offsetof(struct vmcs12, y) },\narch/x86/kvm/vmx/nested.c-61-#include \"vmcs_shadow_fields.h\"\n--\narch/x86/kvm/vmx/nested.c=66=static struct shadow_vmcs_field shadow_read_write_fields[] = {\narch/x86/kvm/vmx/nested.c:67:#define SHADOW_FIELD_RW(x, y) { x, offsetof(struct vmcs12, y) },\narch/x86/kvm/vmx/nested.c-68-#include \"vmcs_shadow_fields.h\"\n--\narch/x86/kvm/vmx/nested.c=73=static void init_vmcs_shadow_fields(void)\n--\narch/x86/kvm/vmx/nested.c-89-\narch/x86/kvm/vmx/nested.c:90:\t\tif (get_vmcs12_field_offset(field) \u003c 0)\narch/x86/kvm/vmx/nested.c-91-\t\t\tcontinue;\n--\narch/x86/kvm/vmx/nested.c-115-\t\t\t field \u003c= GUEST_TR_AR_BYTES,\narch/x86/kvm/vmx/nested.c:116:\t\t\t \"Update vmcs12_write_any() to drop reserved bits from AR_BYTES\");\narch/x86/kvm/vmx/nested.c-117-\narch/x86/kvm/vmx/nested.c:118:\t\tif (get_vmcs12_field_offset(field) \u003c 0)\narch/x86/kvm/vmx/nested.c-119-\t\t\tcontinue;\n--\narch/x86/kvm/vmx/nested.c=176=static int nested_vmx_failValid(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/vmx/nested.c-182-\t\t\t\| X86_EFLAGS_ZF);\narch/x86/kvm/vmx/nested.c:183:\tget_vmcs12(vcpu)-\u003evm_instruction_error = vm_instruction_error;\narch/x86/kvm/vmx/nested.c-184-\t/\n--\narch/x86/kvm/vmx/nested.c-189-\tif (nested_vmx_is_evmptr12_set(to_vmx(vcpu)))\narch/x86/kvm/vmx/nested.c:190:\t\tto_vmx(vcpu)-\u003enested.need_vmcs12_to_shadow_sync = true;\narch/x86/kvm/vmx/nested.c-191-\n--\narch/x86/kvm/vmx/nested.c=227=static void vmx_disable_shadow_vmcs(struct vcpu_vmx vmx)\n--\narch/x86/kvm/vmx/nested.c-230-\tvmcs_write64(VMCS_LINK_POINTER, INVALID_GPA);\narch/x86/kvm/vmx/nested.c:231:\tvmx-\u003enested.need_vmcs12_to_shadow_sync = false;\narch/x86/kvm/vmx/nested.c-232-}\n--\narch/x86/kvm/vmx/nested.c=298=static void vmx_switch_vmcs(struct kvm_vcpu vcpu, struct loaded_vmcs vmcs)\n--\narch/x86/kvm/vmx/nested.c-322-\narch/x86/kvm/vmx/nested.c:323:static void nested_put_vmcs12_pages(struct kvm_vcpu vcpu)\narch/x86/kvm/vmx/nested.c-324-{\n--\narch/x86/kvm/vmx/nested.c=337=static void free_nested(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-367-\t}\narch/x86/kvm/vmx/nested.c:368:\tkfree(vmx-\u003enested.cached_vmcs12);\narch/x86/kvm/vmx/nested.c:369:\tvmx-\u003enested.cached_vmcs12 = NULL;\narch/x86/kvm/vmx/nested.c:370:\tkfree(vmx-\u003enested.cached_shadow_vmcs12);\narch/x86/kvm/vmx/nested.c:371:\tvmx-\u003enested.cached_shadow_vmcs12 = NULL;\narch/x86/kvm/vmx/nested.c-372-\narch/x86/kvm/vmx/nested.c:373:\tnested_put_vmcs12_pages(vcpu);\narch/x86/kvm/vmx/nested.c-374-\n--\narch/x86/kvm/vmx/nested.c=421=static void nested_ept_inject_page_fault(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/vmx/nested.c-424-{\narch/x86/kvm/vmx/nested.c:425:\tstruct vmcs12 vmcs12 = get_vmcs12(vcpu);\narch/x86/kvm/vmx/nested.c-426-\tstruct vcpu_vmx vmx = to_vmx(vcpu);\n--\narch/x86/kvm/vmx/nested.c-488-\t\t /\narch/x86/kvm/vmx/nested.c:489:\t\tnested_ept_invalidate_addr(vcpu, vmcs12-\u003eept_pointer,\narch/x86/kvm/vmx/nested.c-490-\t\t\t\t\t fault-\u003eaddress);\n--\narch/x86/kvm/vmx/nested.c-493-\tnested_vmx_vmexit(vcpu, vm_exit_reason, 0, exit_qualification);\narch/x86/kvm/vmx/nested.c:494:\tvmcs12-\u003eguest_physical_address = fault-\u003eaddress;\narch/x86/kvm/vmx/nested.c-495-}\n--\narch/x86/kvm/vmx/nested.c=497=static inline bool nested_ept_mbec_enabled(struct kvm_vcpu vcpu)\narch/x86/kvm/vmx/nested.c-498-{\narch/x86/kvm/vmx/nested.c:499:\tstruct vmcs12 vmcs12 = get_vmcs12(vcpu);\narch/x86/kvm/vmx/nested.c-500-\narch/x86/kvm/vmx/nested.c:501:\treturn nested_cpu_has2(vmcs12, SECONDARY_EXEC_MODE_BASED_EPT_EXEC);\narch/x86/kvm/vmx/nested.c-502-}\n--\narch/x86/kvm/vmx/nested.c=528=static void nested_ept_uninit_mmu_context(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-532-\narch/x86/kvm/vmx/nested.c:533:static bool nested_vmx_is_page_fault_vmexit(struct vmcs12 vmcs12,\narch/x86/kvm/vmx/nested.c-534-\t\t\t\t\t u16 error_code)\n--\narch/x86/kvm/vmx/nested.c-537-\narch/x86/kvm/vmx/nested.c:538:\tbit = (vmcs12-\u003eexception_bitmap \u0026 (1u \u003c\u003c PF_VECTOR)) != 0;\narch/x86/kvm/vmx/nested.c-539-\tinequality =\narch/x86/kvm/vmx/nested.c:540:\t\t(error_code \u0026 vmcs12-\u003epage_fault_error_code_mask) !=\narch/x86/kvm/vmx/nested.c:541:\t\t vmcs12-\u003epage_fault_error_code_match;\narch/x86/kvm/vmx/nested.c-542-\treturn inequality ^ bit;\n--\narch/x86/kvm/vmx/nested.c=545=static bool nested_vmx_is_exception_vmexit(struct kvm_vcpu vcpu, u8 vector,\n--\narch/x86/kvm/vmx/nested.c-547-{\narch/x86/kvm/vmx/nested.c:548:\tstruct vmcs12 vmcs12 = get_vmcs12(vcpu);\narch/x86/kvm/vmx/nested.c-549-\n--\narch/x86/kvm/vmx/nested.c-557-\tif (vector == PF_VECTOR)\narch/x86/kvm/vmx/nested.c:558:\t\treturn nested_vmx_is_page_fault_vmexit(vmcs12, (u16)error_code);\narch/x86/kvm/vmx/nested.c-559-\narch/x86/kvm/vmx/nested.c:560:\treturn (vmcs12-\u003eexception_bitmap \u0026 (1u \u003c\u003c vector));\narch/x86/kvm/vmx/nested.c-561-}\n--\narch/x86/kvm/vmx/nested.c=563=static int nested_vmx_check_io_bitmap_controls(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:564:\t\t\t\t\t struct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-565-{\narch/x86/kvm/vmx/nested.c:566:\tif (!nested_cpu_has(vmcs12, CPU_BASED_USE_IO_BITMAPS))\narch/x86/kvm/vmx/nested.c-567-\t\treturn 0;\narch/x86/kvm/vmx/nested.c-568-\narch/x86/kvm/vmx/nested.c:569:\tif (CC(!page_address_valid(vcpu, vmcs12-\u003eio_bitmap_a)) \|\|\narch/x86/kvm/vmx/nested.c:570:\t CC(!page_address_valid(vcpu, vmcs12-\u003eio_bitmap_b)))\narch/x86/kvm/vmx/nested.c-571-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=576=static int nested_vmx_check_msr_bitmap_controls(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:577:\t\t\t\t\t\tstruct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-578-{\narch/x86/kvm/vmx/nested.c:579:\tif (!nested_cpu_has(vmcs12, CPU_BASED_USE_MSR_BITMAPS))\narch/x86/kvm/vmx/nested.c-580-\t\treturn 0;\narch/x86/kvm/vmx/nested.c-581-\narch/x86/kvm/vmx/nested.c:582:\tif (CC(!page_address_valid(vcpu, vmcs12-\u003emsr_bitmap)))\narch/x86/kvm/vmx/nested.c-583-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=588=static int nested_vmx_check_tpr_shadow_controls(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:589:\t\t\t\t\t\tstruct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-590-{\narch/x86/kvm/vmx/nested.c:591:\tgpa_t vtpr_gpa = vmcs12-\u003evirtual_apic_page_addr + APIC_TASKPRI;\narch/x86/kvm/vmx/nested.c-592-\tu32 vtpr;\narch/x86/kvm/vmx/nested.c-593-\narch/x86/kvm/vmx/nested.c:594:\tif (!nested_cpu_has(vmcs12, CPU_BASED_TPR_SHADOW))\narch/x86/kvm/vmx/nested.c-595-\t\treturn 0;\narch/x86/kvm/vmx/nested.c-596-\narch/x86/kvm/vmx/nested.c:597:\tif (CC(!page_address_valid(vcpu, vmcs12-\u003evirtual_apic_page_addr)))\narch/x86/kvm/vmx/nested.c-598-\t\treturn -EINVAL;\narch/x86/kvm/vmx/nested.c-599-\narch/x86/kvm/vmx/nested.c:600:\tif (CC(!nested_cpu_has_vid(vmcs12) \u0026\u0026 vmcs12-\u003etpr_threshold \u003e\u003e 4))\narch/x86/kvm/vmx/nested.c-601-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c-621-\tif (warn_on_missed_cc \u0026\u0026 vcpu-\u003ewants_to_run \u0026\u0026\narch/x86/kvm/vmx/nested.c:622:\t nested_cpu_has(vmcs12, CPU_BASED_TPR_SHADOW) \u0026\u0026\narch/x86/kvm/vmx/nested.c:623:\t !nested_cpu_has_vid(vmcs12) \u0026\u0026\narch/x86/kvm/vmx/nested.c:624:\t !nested_cpu_has2(vmcs12, SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES) \u0026\u0026\narch/x86/kvm/vmx/nested.c-625-\t !kvm_read_guest(vcpu-\u003ekvm, vtpr_gpa, \u0026vtpr, sizeof(vtpr)) \u0026\u0026\narch/x86/kvm/vmx/nested.c:626:\t CC((vmcs12-\u003etpr_threshold \u0026 GENMASK(3, 0)) \u003e ((vtpr \u003e\u003e 4) \u0026 GENMASK(3, 0))))\narch/x86/kvm/vmx/nested.c-627-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=733=static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:734:\t\t\t\t\t\t struct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-735-{\n--\narch/x86/kvm/vmx/nested.c-742-\tif (!cpu_has_vmx_msr_bitmap() \|\|\narch/x86/kvm/vmx/nested.c:743:\t !nested_cpu_has(vmcs12, CPU_BASED_USE_MSR_BITMAPS))\narch/x86/kvm/vmx/nested.c-744-\t\treturn false;\n--\narch/x86/kvm/vmx/nested.c-761-\narch/x86/kvm/vmx/nested.c:762:\tCLASS(kvm_vcpu_map_local_readonly, m)(vcpu, gpa_to_gfn(vmcs12-\u003emsr_bitmap));\narch/x86/kvm/vmx/nested.c-763-\tif (m.ret)\n--\narch/x86/kvm/vmx/nested.c-774-\narch/x86/kvm/vmx/nested.c:775:\tif (nested_cpu_has_virt_x2apic_mode(vmcs12)) {\narch/x86/kvm/vmx/nested.c:776:\t\tif (nested_cpu_has_apic_reg_virt(vmcs12)) {\narch/x86/kvm/vmx/nested.c-777-\t\t\t/\n--\narch/x86/kvm/vmx/nested.c-794-\narch/x86/kvm/vmx/nested.c:795:\t\tif (nested_cpu_has_vid(vmcs12)) {\narch/x86/kvm/vmx/nested.c-796-\t\t\tnested_vmx_disable_intercept_for_x2apic_msr(\n--\narch/x86/kvm/vmx/nested.c-859-\narch/x86/kvm/vmx/nested.c:860:static void nested_cache_shadow_vmcs12(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:861:\t\t\t\t struct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-862-{\narch/x86/kvm/vmx/nested.c-863-\tstruct vcpu_vmx vmx = to_vmx(vcpu);\narch/x86/kvm/vmx/nested.c:864:\tstruct gfn_to_hva_cache ghc = \u0026vmx-\u003enested.shadow_vmcs12_cache;\narch/x86/kvm/vmx/nested.c-865-\narch/x86/kvm/vmx/nested.c:866:\tif (!nested_cpu_has_shadow_vmcs(vmcs12) \|\|\narch/x86/kvm/vmx/nested.c:867:\t vmcs12-\u003evmcs_link_pointer == INVALID_GPA)\narch/x86/kvm/vmx/nested.c-868-\t\treturn;\narch/x86/kvm/vmx/nested.c-869-\narch/x86/kvm/vmx/nested.c:870:\tif (ghc-\u003egpa != vmcs12-\u003evmcs_link_pointer \u0026\u0026\narch/x86/kvm/vmx/nested.c-871-\t kvm_gfn_to_hva_cache_init(vcpu-\u003ekvm, ghc,\narch/x86/kvm/vmx/nested.c:872:\t\t\t\t vmcs12-\u003evmcs_link_pointer, VMCS12_SIZE))\narch/x86/kvm/vmx/nested.c-873-\t\treturn;\narch/x86/kvm/vmx/nested.c-874-\narch/x86/kvm/vmx/nested.c:875:\tkvm_read_guest_cached(vcpu-\u003ekvm, ghc, get_shadow_vmcs12(vcpu),\narch/x86/kvm/vmx/nested.c-876-\t\t\t VMCS12_SIZE);\n--\narch/x86/kvm/vmx/nested.c-878-\narch/x86/kvm/vmx/nested.c:879:static void nested_flush_cached_shadow_vmcs12(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:880:\t\t\t\t\t struct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-881-{\narch/x86/kvm/vmx/nested.c-882-\tstruct vcpu_vmx vmx = to_vmx(vcpu);\narch/x86/kvm/vmx/nested.c:883:\tstruct gfn_to_hva_cache ghc = \u0026vmx-\u003enested.shadow_vmcs12_cache;\narch/x86/kvm/vmx/nested.c-884-\narch/x86/kvm/vmx/nested.c:885:\tif (!nested_cpu_has_shadow_vmcs(vmcs12) \|\|\narch/x86/kvm/vmx/nested.c:886:\t vmcs12-\u003evmcs_link_pointer == INVALID_GPA)\narch/x86/kvm/vmx/nested.c-887-\t\treturn;\narch/x86/kvm/vmx/nested.c-888-\narch/x86/kvm/vmx/nested.c:889:\tif (ghc-\u003egpa != vmcs12-\u003evmcs_link_pointer \u0026\u0026\narch/x86/kvm/vmx/nested.c-890-\t kvm_gfn_to_hva_cache_init(vcpu-\u003ekvm, ghc,\narch/x86/kvm/vmx/nested.c:891:\t\t\t\t vmcs12-\u003evmcs_link_pointer, VMCS12_SIZE))\narch/x86/kvm/vmx/nested.c-892-\t\treturn;\narch/x86/kvm/vmx/nested.c-893-\narch/x86/kvm/vmx/nested.c:894:\tkvm_write_guest_cached(vcpu-\u003ekvm, ghc, get_shadow_vmcs12(vcpu),\narch/x86/kvm/vmx/nested.c-895-\t\t\t VMCS12_SIZE);\n--\narch/x86/kvm/vmx/nested.c=902=static bool nested_exit_intr_ack_set(struct kvm_vcpu vcpu)\narch/x86/kvm/vmx/nested.c-903-{\narch/x86/kvm/vmx/nested.c:904:\treturn get_vmcs12(vcpu)-\u003evm_exit_controls \u0026\narch/x86/kvm/vmx/nested.c-905-\t\tVM_EXIT_ACK_INTR_ON_EXIT;\n--\narch/x86/kvm/vmx/nested.c=908=static int nested_vmx_check_apic_access_controls(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:909:\t\t\t\t\t struct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-910-{\narch/x86/kvm/vmx/nested.c:911:\tif (nested_cpu_has2(vmcs12, SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES) \u0026\u0026\narch/x86/kvm/vmx/nested.c:912:\t CC(!page_address_valid(vcpu, vmcs12-\u003eapic_access_addr)))\narch/x86/kvm/vmx/nested.c-913-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=918=static int nested_vmx_check_apicv_controls(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:919:\t\t\t\t\t struct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-920-{\narch/x86/kvm/vmx/nested.c:921:\tif (!nested_cpu_has_virt_x2apic_mode(vmcs12) \u0026\u0026\narch/x86/kvm/vmx/nested.c:922:\t !nested_cpu_has_apic_reg_virt(vmcs12) \u0026\u0026\narch/x86/kvm/vmx/nested.c:923:\t !nested_cpu_has_vid(vmcs12) \u0026\u0026\narch/x86/kvm/vmx/nested.c:924:\t !nested_cpu_has_posted_intr(vmcs12))\narch/x86/kvm/vmx/nested.c-925-\t\treturn 0;\n--\narch/x86/kvm/vmx/nested.c-930-\t /\narch/x86/kvm/vmx/nested.c:931:\tif (CC(nested_cpu_has_virt_x2apic_mode(vmcs12) \u0026\u0026\narch/x86/kvm/vmx/nested.c:932:\t nested_cpu_has2(vmcs12, SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES)))\narch/x86/kvm/vmx/nested.c-933-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c-938-\t /\narch/x86/kvm/vmx/nested.c:939:\tif (CC(nested_cpu_has_vid(vmcs12) \u0026\u0026 !nested_exit_on_intr(vcpu)))\narch/x86/kvm/vmx/nested.c-940-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c-944-\t the descriptor address has been already checked\narch/x86/kvm/vmx/nested.c:945:\t * in nested_get_vmcs12_pages.\narch/x86/kvm/vmx/nested.c-946-\t \n--\narch/x86/kvm/vmx/nested.c-948-\t /\narch/x86/kvm/vmx/nested.c:949:\tif (nested_cpu_has_posted_intr(vmcs12) \u0026\u0026\narch/x86/kvm/vmx/nested.c:950:\t (CC(!nested_cpu_has_vid(vmcs12)) \|\|\narch/x86/kvm/vmx/nested.c-951-\t CC(!nested_exit_intr_ack_set(vcpu)) \|\|\narch/x86/kvm/vmx/nested.c:952:\t CC((vmcs12-\u003eposted_intr_nv \u0026 0xff00)) \|\|\narch/x86/kvm/vmx/nested.c:953:\t CC(!kvm_vcpu_is_legal_aligned_gpa(vcpu, vmcs12-\u003eposted_intr_desc_addr, 64))))\narch/x86/kvm/vmx/nested.c-954-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c-956-\t/* tpr shadow is needed by all apicv features. /\narch/x86/kvm/vmx/nested.c:957:\tif (CC(!nested_cpu_has(vmcs12, CPU_BASED_TPR_SHADOW)))\narch/x86/kvm/vmx/nested.c-958-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=994=static int nested_vmx_check_exit_msr_switch_controls(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:995:\t\t\t\t\t\t struct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-996-{\narch/x86/kvm/vmx/nested.c-997-\tif (CC(nested_vmx_check_msr_switch(vcpu,\narch/x86/kvm/vmx/nested.c:998:\t\t\t\t\t vmcs12-\u003evm_exit_msr_load_count,\narch/x86/kvm/vmx/nested.c:999:\t\t\t\t\t vmcs12-\u003evm_exit_msr_load_addr)) \|\|\narch/x86/kvm/vmx/nested.c-1000-\t CC(nested_vmx_check_msr_switch(vcpu,\narch/x86/kvm/vmx/nested.c:1001:\t\t\t\t\t vmcs12-\u003evm_exit_msr_store_count,\narch/x86/kvm/vmx/nested.c:1002:\t\t\t\t\t vmcs12-\u003evm_exit_msr_store_addr)))\narch/x86/kvm/vmx/nested.c-1003-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=1008=static int nested_vmx_check_entry_msr_switch_controls(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:1009: struct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-1010-{\narch/x86/kvm/vmx/nested.c-1011-\tif (CC(nested_vmx_check_msr_switch(vcpu,\narch/x86/kvm/vmx/nested.c:1012:\t\t\t\t\t vmcs12-\u003evm_entry_msr_load_count,\narch/x86/kvm/vmx/nested.c:1013:\t\t\t\t\t vmcs12-\u003evm_entry_msr_load_addr)))\narch/x86/kvm/vmx/nested.c-1014- return -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=1019=static int nested_vmx_check_pml_controls(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:1020:\t\t\t\t\t struct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-1021-{\narch/x86/kvm/vmx/nested.c:1022:\tif (!nested_cpu_has_pml(vmcs12))\narch/x86/kvm/vmx/nested.c-1023-\t\treturn 0;\narch/x86/kvm/vmx/nested.c-1024-\narch/x86/kvm/vmx/nested.c:1025:\tif (CC(!nested_cpu_has_ept(vmcs12)) \|\|\narch/x86/kvm/vmx/nested.c:1026:\t CC(!page_address_valid(vcpu, vmcs12-\u003epml_address)))\narch/x86/kvm/vmx/nested.c-1027-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=1032=static int nested_vmx_check_unrestricted_guest_controls(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:1033:\t\t\t\t\t\t\tstruct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-1034-{\narch/x86/kvm/vmx/nested.c:1035:\tif (CC(nested_cpu_has2(vmcs12, SECONDARY_EXEC_UNRESTRICTED_GUEST) \u0026\u0026\narch/x86/kvm/vmx/nested.c:1036:\t !nested_cpu_has_ept(vmcs12)))\narch/x86/kvm/vmx/nested.c-1037-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=1041=static int nested_vmx_check_mode_based_ept_exec_controls(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:1042:\t\t\t\t\t\t\t struct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-1043-{\narch/x86/kvm/vmx/nested.c:1044:\tif (CC(nested_cpu_has2(vmcs12, SECONDARY_EXEC_MODE_BASED_EPT_EXEC) \u0026\u0026\narch/x86/kvm/vmx/nested.c:1045:\t !nested_cpu_has_ept(vmcs12)))\narch/x86/kvm/vmx/nested.c-1046-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=1050=static int nested_vmx_check_shadow_vmcs_controls(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:1051:\t\t\t\t\t\t struct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-1052-{\narch/x86/kvm/vmx/nested.c:1053:\tif (!nested_cpu_has_shadow_vmcs(vmcs12))\narch/x86/kvm/vmx/nested.c-1054-\t\treturn 0;\narch/x86/kvm/vmx/nested.c-1055-\narch/x86/kvm/vmx/nested.c:1056:\tif (CC(!page_address_valid(vcpu, vmcs12-\u003evmread_bitmap)) \|\|\narch/x86/kvm/vmx/nested.c:1057:\t CC(!page_address_valid(vcpu, vmcs12-\u003evmwrite_bitmap)))\narch/x86/kvm/vmx/nested.c-1058-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=1219=static bool nested_msr_store_list_has_msr(struct kvm_vcpu vcpu, u32 msr_index)\narch/x86/kvm/vmx/nested.c-1220-{\narch/x86/kvm/vmx/nested.c:1221:\tstruct vmcs12 vmcs12 = get_vmcs12(vcpu);\narch/x86/kvm/vmx/nested.c:1222:\tu32 count = vmcs12-\u003evm_exit_msr_store_count;\narch/x86/kvm/vmx/nested.c:1223:\tu64 gpa = vmcs12-\u003evm_exit_msr_store_addr;\narch/x86/kvm/vmx/nested.c-1224-\tstruct vmx_msr_entry e;\n--\narch/x86/kvm/vmx/nested.c=1287=static bool nested_has_guest_tlb_tag(struct kvm_vcpu vcpu)\narch/x86/kvm/vmx/nested.c-1288-{\narch/x86/kvm/vmx/nested.c:1289:\tstruct vmcs12 vmcs12 = get_vmcs12(vcpu);\narch/x86/kvm/vmx/nested.c-1290-\narch/x86/kvm/vmx/nested.c-1291-\treturn enable_ept \|\|\narch/x86/kvm/vmx/nested.c:1292:\t (nested_cpu_has_vpid(vmcs12) \u0026\u0026 to_vmx(vcpu)-\u003enested.vpid02);\narch/x86/kvm/vmx/nested.c-1293-}\n--\narch/x86/kvm/vmx/nested.c=1295=static void nested_vmx_transition_tlb_flush(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c:1296:\t\t\t\t\t struct vmcs12 vmcs12,\narch/x86/kvm/vmx/nested.c-1297-\t\t\t\t\t bool is_vmenter)\n--\narch/x86/kvm/vmx/nested.c-1320-\t /\narch/x86/kvm/vmx/nested.c:1321:\tif (!nested_cpu_has_vpid(vmcs12)) {\narch/x86/kvm/vmx/nested.c-1322-\t\tkvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu);\n--\narch/x86/kvm/vmx/nested.c-1329-\t/\narch/x86/kvm/vmx/nested.c:1330:\t VPID is enabled and in use by vmcs12. If vpid12 is changing, then\narch/x86/kvm/vmx/nested.c-1331-\t * emulate a guest TLB flush as KVM does not track vpid12 history nor\n--\narch/x86/kvm/vmx/nested.c-1338-\t /\narch/x86/kvm/vmx/nested.c:1339:\tif (is_vmenter \u0026\u0026 vmcs12-\u003evirtual_processor_id != vmx-\u003enested.last_vpid) {\narch/x86/kvm/vmx/nested.c:1340:\t\tvmx-\u003enested.last_vpid = vmcs12-\u003evirtual_processor_id;\narch/x86/kvm/vmx/nested.c-1341-\t\tkvm_make_request(KVM_REQ_TLB_FLUSH_GUEST, vcpu);\n--\narch/x86/kvm/vmx/nested.c=1624=int vmx_get_vmx_msr(struct nested_vmx_msrs msrs, u32 msr_index, u64 pdata)\n--\narch/x86/kvm/vmx/nested.c-1711- /\narch/x86/kvm/vmx/nested.c:1712:static void copy_shadow_to_vmcs12(struct vcpu_vmx vmx)\narch/x86/kvm/vmx/nested.c-1713-{\narch/x86/kvm/vmx/nested.c-1714-\tstruct vmcs shadow_vmcs = vmx-\u003evmcs01.shadow_vmcs;\narch/x86/kvm/vmx/nested.c:1715:\tstruct vmcs12 vmcs12 = get_vmcs12(\u0026vmx-\u003evcpu);\narch/x86/kvm/vmx/nested.c-1716-\tstruct shadow_vmcs_field field;\n--\narch/x86/kvm/vmx/nested.c-1729-\t\tval = __vmcs_readl(field.encoding);\narch/x86/kvm/vmx/nested.c:1730:\t\tvmcs12_write_any(vmcs12, field.encoding, field.offset, val);\narch/x86/kvm/vmx/nested.c-1731-\t}\n--\narch/x86/kvm/vmx/nested.c-1738-\narch/x86/kvm/vmx/nested.c:1739:static void copy_vmcs12_to_shadow(struct vcpu_vmx vmx)\narch/x86/kvm/vmx/nested.c-1740-{\n--\narch/x86/kvm/vmx/nested.c-1749-\tstruct vmcs shadow_vmcs = vmx-\u003evmcs01.shadow_vmcs;\narch/x86/kvm/vmx/nested.c:1750:\tstruct vmcs12 vmcs12 = get_vmcs12(\u0026vmx-\u003evcpu);\narch/x86/kvm/vmx/nested.c-1751-\tstruct shadow_vmcs_field field;\n--\narch/x86/kvm/vmx/nested.c-1762-\t\t\tfield = fields[q][i];\narch/x86/kvm/vmx/nested.c:1763:\t\t\tval = vmcs12_read_any(vmcs12, field.encoding,\narch/x86/kvm/vmx/nested.c-1764-\t\t\t\t\t field.offset);\n--\narch/x86/kvm/vmx/nested.c-1772-\narch/x86/kvm/vmx/nested.c:1773:static void copy_enlightened_to_vmcs12(struct vcpu_vmx vmx, u32 hv_clean_fields)\narch/x86/kvm/vmx/nested.c-1774-{\narch/x86/kvm/vmx/nested.c-1775-#ifdef CONFIG_KVM_HYPERV\narch/x86/kvm/vmx/nested.c:1776:\tstruct vmcs12 vmcs12 = vmx-\u003enested.cached_vmcs12;\narch/x86/kvm/vmx/nested.c-1777-\tstruct hv_enlightened_vmcs evmcs = nested_vmx_evmcs(vmx);\n--\narch/x86/kvm/vmx/nested.c-1780-\t/ HV_VMX_ENLIGHTENED_CLEAN_FIELD_NONE /\narch/x86/kvm/vmx/nested.c:1781:\tvmcs12-\u003etpr_threshold = evmcs-\u003etpr_threshold;\narch/x86/kvm/vmx/nested.c:1782:\tvmcs12-\u003eguest_rip = evmcs-\u003eguest_rip;\narch/x86/kvm/vmx/nested.c-1783-\n--\narch/x86/kvm/vmx/nested.c-1792-\t\t HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_BASIC))) {\narch/x86/kvm/vmx/nested.c:1793:\t\tvmcs12-\u003eguest_rsp = evmcs-\u003eguest_rsp;\narch/x86/kvm/vmx/nested.c:1794:\t\tvmcs12-\u003eguest_rflags = evmcs-\u003eguest_rflags;\narch/x86/kvm/vmx/nested.c:1795:\t\tvmcs12-\u003eguest_interruptibility_info =\narch/x86/kvm/vmx/nested.c-1796-\t\t\tevmcs-\u003eguest_interruptibility_info;\narch/x86/kvm/vmx/nested.c-1797-\t\t/\narch/x86/kvm/vmx/nested.c:1798:\t\t * Not present in struct vmcs12:\narch/x86/kvm/vmx/nested.c:1799:\t\t * vmcs12-\u003eguest_ssp = evmcs-\u003eguest_ssp;\narch/x86/kvm/vmx/nested.c-1800-\t\t /\n--\narch/x86/kvm/vmx/nested.c-1804-\t\t HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_PROC))) {\narch/x86/kvm/vmx/nested.c:1805:\t\tvmcs12-\u003ecpu_based_vm_exec_control =\narch/x86/kvm/vmx/nested.c-1806-\t\t\tevmcs-\u003ecpu_based_vm_exec_control;\n--\narch/x86/kvm/vmx/nested.c-1810-\t\t HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_EXCPN))) {\narch/x86/kvm/vmx/nested.c:1811:\t\tvmcs12-\u003eexception_bitmap = evmcs-\u003eexception_bitmap;\narch/x86/kvm/vmx/nested.c-1812-\t}\n--\narch/x86/kvm/vmx/nested.c-1815-\t\t HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_ENTRY))) {\narch/x86/kvm/vmx/nested.c:1816:\t\tvmcs12-\u003evm_entry_controls = evmcs-\u003evm_entry_controls;\narch/x86/kvm/vmx/nested.c-1817-\t}\n--\narch/x86/kvm/vmx/nested.c-1820-\t\t HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_EVENT))) {\narch/x86/kvm/vmx/nested.c:1821:\t\tvmcs12-\u003evm_entry_intr_info_field =\narch/x86/kvm/vmx/nested.c-1822-\t\t\tevmcs-\u003evm_entry_intr_info_field;\narch/x86/kvm/vmx/nested.c:1823:\t\tvmcs12-\u003evm_entry_exception_error_code =\narch/x86/kvm/vmx/nested.c-1824-\t\t\tevmcs-\u003evm_entry_exception_error_code;\narch/x86/kvm/vmx/nested.c:1825:\t\tvmcs12-\u003evm_entry_instruction_len =\narch/x86/kvm/vmx/nested.c-1826-\t\t\tevmcs-\u003evm_entry_instruction_len;\n--\narch/x86/kvm/vmx/nested.c-1830-\t\t HV_VMX_ENLIGHTENED_CLEAN_FIELD_HOST_GRP1))) {\narch/x86/kvm/vmx/nested.c:1831:\t\tvmcs12-\u003ehost_ia32_pat = evmcs-\u003ehost_ia32_pat;\narch/x86/kvm/vmx/nested.c:1832:\t\tvmcs12-\u003ehost_ia32_efer = evmcs-\u003ehost_ia32_efer;\narch/x86/kvm/vmx/nested.c:1833:\t\tvmcs12-\u003ehost_cr0 = evmcs-\u003ehost_cr0;\narch/x86/kvm/vmx/nested.c:1834:\t\tvmcs12-\u003ehost_cr3 = evmcs-\u003ehost_cr3;\narch/x86/kvm/vmx/nested.c:1835:\t\tvmcs12-\u003ehost_cr4 = evmcs-\u003ehost_cr4;\narch/x86/kvm/vmx/nested.c:1836:\t\tvmcs12-\u003ehost_ia32_sysenter_esp = evmcs-\u003ehost_ia32_sysenter_esp;\narch/x86/kvm/vmx/nested.c:1837:\t\tvmcs12-\u003ehost_ia32_sysenter_eip = evmcs-\u003ehost_ia32_sysenter_eip;\narch/x86/kvm/vmx/nested.c:1838:\t\tvmcs12-\u003ehost_rip = evmcs-\u003ehost_rip;\narch/x86/kvm/vmx/nested.c:1839:\t\tvmcs12-\u003ehost_ia32_sysenter_cs = evmcs-\u003ehost_ia32_sysenter_cs;\narch/x86/kvm/vmx/nested.c:1840:\t\tvmcs12-\u003ehost_es_selector = evmcs-\u003ehost_es_selector;\narch/x86/kvm/vmx/nested.c:1841:\t\tvmcs12-\u003ehost_cs_selector = evmcs-\u003ehost_cs_selector;\narch/x86/kvm/vmx/nested.c:1842:\t\tvmcs12-\u003ehost_ss_selector = evmcs-\u003ehost_ss_selector;\narch/x86/kvm/vmx/nested.c:1843:\t\tvmcs12-\u003ehost_ds_selector = evmcs-\u003ehost_ds_selector;\narch/x86/kvm/vmx/nested.c:1844:\t\tvmcs12-\u003ehost_fs_selector = evmcs-\u003ehost_fs_selector;\narch/x86/kvm/vmx/nested.c:1845:\t\tvmcs12-\u003ehost_gs_selector = evmcs-\u003ehost_gs_selector;\narch/x86/kvm/vmx/nested.c:1846:\t\tvmcs12-\u003ehost_tr_selector = evmcs-\u003ehost_tr_selector;\narch/x86/kvm/vmx/nested.c:1847:\t\tvmcs12-\u003ehost_ia32_perf_global_ctrl = evmcs-\u003ehost_ia32_perf_global_ctrl;\narch/x86/kvm/vmx/nested.c-1848-\t\t/\narch/x86/kvm/vmx/nested.c:1849:\t\t * Not present in struct vmcs12:\narch/x86/kvm/vmx/nested.c:1850:\t\t * vmcs12-\u003ehost_ia32_s_cet = evmcs-\u003ehost_ia32_s_cet;\narch/x86/kvm/vmx/nested.c:1851:\t\t * vmcs12-\u003ehost_ssp = evmcs-\u003ehost_ssp;\narch/x86/kvm/vmx/nested.c:1852:\t\t * vmcs12-\u003ehost_ia32_int_ssp_table_addr = evmcs-\u003ehost_ia32_int_ssp_table_addr;\narch/x86/kvm/vmx/nested.c-1853-\t\t */\n--\narch/x86/kvm/vmx/nested.c-1857-\t\t HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_GRP1))) {\narch/x86/kvm/vmx/nested.c:1858:\t\tvmcs12-\u003epin_based_vm_exec_control =\narch/x86/kvm/vmx/nested.c-1859-\t\t\tevmcs-\u003epin_based_vm_exec_control;\narch/x86/kvm/vmx/nested.c:1860:\t\tvmcs12-\u003evm_exit_controls = evmcs-\u003evm_exit_controls;\narch/x86/kvm/vmx/nested.c:1861:\t\tvmcs12-\u003esecondary_vm_exec_control =\narch/x86/kvm/vmx/nested.c-1862-\t\t\tevmcs-\u003esecondary_vm_exec_control;\n--\narch/x86/kvm/vmx/nested.c-1866-\t\t HV_VMX_ENLIGHTENED_CLEAN_FIELD_IO_BITMAP))) {\narch/x86/kvm/vmx/nested.c:1867:\t\tvmcs12-\u003eio_bitmap_a = evmcs-\u003eio_bitmap_a;\narch/x86/kvm/vmx/nested.c:1868:\t\tvmcs12-\u003eio_bitmap_b = evmcs-\u003eio_bitmap_b;\narch/x86/kvm/vmx/nested.c-1869-\t}\n--\narch/x86/kvm/vmx/nested.c-1872-\t\t HV_VMX_ENLIGHTENED_CLEAN_FIELD_MSR_BITMAP))) {\narch/x86/kvm/vmx/nested.c:1873:\t\tvmcs12-\u003emsr_bitmap = evmcs-\u003emsr_bitmap;\narch/x86/kvm/vmx/nested.c-1874-\t}\n--\narch/x86/kvm/vmx/nested.c-1877-\t\t HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_GRP2))) {\narch/x86/kvm/vmx/nested.c:1878:\t\tvmcs12-\u003eguest_es_base = evmcs-\u003eguest_es_base;\narch/x86/kvm/vmx/nested.c:1879:\t\tvmcs12-\u003eguest_cs_base = evmcs-\u003eguest_cs_base;\narch/x86/kvm/vmx/nested.c:1880:\t\tvmcs12-\u003eguest_ss_base = evmcs-\u003eguest_ss_base;\narch/x86/kvm/vmx/nested.c:1881:\t\tvmcs12-\u003eguest_ds_base = evmcs-\u003eguest_ds_base;\narch/x86/kvm/vmx/nested.c:1882:\t\tvmcs12-\u003eguest_fs_base = evmcs-\u003eguest_fs_base;\narch/x86/kvm/vmx/nested.c:1883:\t\tvmcs12-\u003eguest_gs_base = evmcs-\u003eguest_gs_base;\narch/x86/kvm/vmx/nested.c:1884:\t\tvmcs12-\u003eguest_ldtr_base = evmcs-\u003eguest_ldtr_base;\narch/x86/kvm/vmx/nested.c:1885:\t\tvmcs12-\u003eguest_tr_base = evmcs-\u003eguest_tr_base;\narch/x86/kvm/vmx/nested.c:1886:\t\tvmcs12-\u003eguest_gdtr_base = evmcs-\u003eguest_gdtr_base;\narch/x86/kvm/vmx/nested.c:1887:\t\tvmcs12-\u003eguest_idtr_base = evmcs-\u003eguest_idtr_base;\narch/x86/kvm/vmx/nested.c:1888:\t\tvmcs12-\u003eguest_es_limit = evmcs-\u003eguest_es_limit;\narch/x86/kvm/vmx/nested.c:1889:\t\tvmcs12-\u003eguest_cs_limit = evmcs-\u003eguest_cs_limit;\narch/x86/kvm/vmx/nested.c:1890:\t\tvmcs12-\u003eguest_ss_limit = evmcs-\u003eguest_ss_limit;\narch/x86/kvm/vmx/nested.c:1891:\t\tvmcs12-\u003eguest_ds_limit = evmcs-\u003eguest_ds_limit;\narch/x86/kvm/vmx/nested.c:1892:\t\tvmcs12-\u003eguest_fs_limit = evmcs-\u003eguest_fs_limit;\narch/x86/kvm/vmx/nested.c:1893:\t\tvmcs12-\u003eguest_gs_limit = evmcs-\u003eguest_gs_limit;\narch/x86/kvm/vmx/nested.c:1894:\t\tvmcs12-\u003eguest_ldtr_limit = evmcs-\u003eguest_ldtr_limit;\narch/x86/kvm/vmx/nested.c:1895:\t\tvmcs12-\u003eguest_tr_limit = evmcs-\u003eguest_tr_limit;\narch/x86/kvm/vmx/nested.c:1896:\t\tvmcs12-\u003eguest_gdtr_limit = evmcs-\u003eguest_gdtr_limit;\narch/x86/kvm/vmx/nested.c:1897:\t\tvmcs12-\u003eguest_idtr_limit = evmcs-\u003eguest_idtr_limit;\narch/x86/kvm/vmx/nested.c:1898:\t\tvmcs12-\u003eguest_es_ar_bytes = evmcs-\u003eguest_es_ar_bytes;\narch/x86/kvm/vmx/nested.c:1899:\t\tvmcs12-\u003eguest_cs_ar_bytes = evmcs-\u003eguest_cs_ar_bytes;\narch/x86/kvm/vmx/nested.c:1900:\t\tvmcs12-\u003eguest_ss_ar_bytes = evmcs-\u003eguest_ss_ar_bytes;\narch/x86/kvm/vmx/nested.c:1901:\t\tvmcs12-\u003eguest_ds_ar_bytes = evmcs-\u003eguest_ds_ar_bytes;\narch/x86/kvm/vmx/nested.c:1902:\t\tvmcs12-\u003eguest_fs_ar_bytes = evmcs-\u003eguest_fs_ar_bytes;\narch/x86/kvm/vmx/nested.c:1903:\t\tvmcs12-\u003eguest_gs_ar_bytes = evmcs-\u003eguest_gs_ar_bytes;\narch/x86/kvm/vmx/nested.c:1904:\t\tvmcs12-\u003eguest_ldtr_ar_bytes = evmcs-\u003eguest_ldtr_ar_bytes;\narch/x86/kvm/vmx/nested.c:1905:\t\tvmcs12-\u003eguest_tr_ar_bytes = evmcs-\u003eguest_tr_ar_bytes;\narch/x86/kvm/vmx/nested.c:1906:\t\tvmcs12-\u003eguest_es_selector = evmcs-\u003eguest_es_selector;\narch/x86/kvm/vmx/nested.c:1907:\t\tvmcs12-\u003eguest_cs_selector = evmcs-\u003eguest_cs_selector;\narch/x86/kvm/vmx/nested.c:1908:\t\tvmcs12-\u003eguest_ss_selector = evmcs-\u003eguest_ss_selector;\narch/x86/kvm/vmx/nested.c:1909:\t\tvmcs12-\u003eguest_ds_selector = evmcs-\u003eguest_ds_selector;\narch/x86/kvm/vmx/nested.c:1910:\t\tvmcs12-\u003eguest_fs_selector = evmcs-\u003eguest_fs_selector;\narch/x86/kvm/vmx/nested.c:1911:\t\tvmcs12-\u003eguest_gs_selector = evmcs-\u003eguest_gs_selector;\narch/x86/kvm/vmx/nested.c:1912:\t\tvmcs12-\u003eguest_ldtr_selector = evmcs-\u003eguest_ldtr_selector;\narch/x86/kvm/vmx/nested.c:1913:\t\tvmcs12-\u003eguest_tr_selector = evmcs-\u003eguest_tr_selector;\narch/x86/kvm/vmx/nested.c-1914-\t}\n--\narch/x86/kvm/vmx/nested.c-1917-\t\t HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_GRP2))) {\narch/x86/kvm/vmx/nested.c:1918:\t\tvmcs12-\u003etsc_offset = evmcs-\u003etsc_offset;\narch/x86/kvm/vmx/nested.c:1919:\t\tvmcs12-\u003evirtual_apic_page_addr = evmcs-\u003evirtual_apic_page_addr;\narch/x86/kvm/vmx/nested.c:1920:\t\tvmcs12-\u003exss_exit_bitmap = evmcs-\u003exss_exit_bitmap;\narch/x86/kvm/vmx/nested.c:1921:\t\tvmcs12-\u003eencls_exiting_bitmap = evmcs-\u003eencls_exiting_bitmap;\narch/x86/kvm/vmx/nested.c:1922:\t\tvmcs12-\u003etsc_multiplier = evmcs-\u003etsc_multiplier;\narch/x86/kvm/vmx/nested.c-1923-\t}\n--\narch/x86/kvm/vmx/nested.c-1926-\t\t HV_VMX_ENLIGHTENED_CLEAN_FIELD_CRDR))) {\narch/x86/kvm/vmx/nested.c:1927:\t\tvmcs12-\u003ecr0_guest_host_mask = evmcs-\u003ecr0_guest_host_mask;\narch/x86/kvm/vmx/nested.c:1928:\t\tvmcs12-\u003ecr4_guest_host_mask = evmcs-\u003ecr4_guest_host_mask;\narch/x86/kvm/vmx/nested.c:1929:\t\tvmcs12-\u003ecr0_read_shadow = evmcs-\u003ecr0_read_shadow;\narch/x86/kvm/vmx/nested.c:1930:\t\tvmcs12-\u003ecr4_read_shadow = evmcs-\u003ecr4_read_shadow;\narch/x86/kvm/vmx/nested.c:1931:\t\tvmcs12-\u003eguest_cr0 = evmcs-\u003eguest_cr0;\narch/x86/kvm/vmx/nested.c:1932:\t\tvmcs12-\u003eguest_cr3 = evmcs-\u003eguest_cr3;\narch/x86/kvm/vmx/nested.c:1933:\t\tvmcs12-\u003eguest_cr4 = evmcs-\u003eguest_cr4;\narch/x86/kvm/vmx/nested.c:1934:\t\tvmcs12-\u003eguest_dr7 = evmcs-\u003eguest_dr7;\narch/x86/kvm/vmx/nested.c-1935-\t}\n--\narch/x86/kvm/vmx/nested.c-1938-\t\t HV_VMX_ENLIGHTENED_CLEAN_FIELD_HOST_POINTER))) {\narch/x86/kvm/vmx/nested.c:1939:\t\tvmcs12-\u003ehost_fs_base = evmcs-\u003ehost_fs_base;\narch/x86/kvm/vmx/nested.c:1940:\t\tvmcs12-\u003ehost_gs_base = evmcs-\u003ehost_gs_base;\n\n" }
45/3	2026/09/11 21:52	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 89193 output: 20 thoughts: 87 Thoughts: Here's my thought process, as it stands: Navigating the VMCS12 Landscape Okay, so I'm digging into how `vmcs12` is handled. Right now, my focus is on understanding its allocation. I see it's nestled within `cached_vmcs12`, and that this `cached_vmcs12` is brought to life using `kzalloc` with the `VMCS12_SIZE` and `GFP_KERNEL_ACCOUNT` flags. Now, I need to trace back from `cached_vmcs12`. I should be able to see how this crucial structure is actually instantiated somewhere in a function like `nested_vmx_alloc_kvm_area`, or a similar function in the codebase. My next move is to run a quick `grep` for `cached_vmcs12` to pinpoint where this object is allocated and initialized to understand the overall architecture.
46/3	2026/09/11 21:52	tool	grepper	0m Args: { "Expression": "cached_vmcs12" } Results: { "Output": "arch/x86/kvm/vmx/nested.c=337=static void free_nested(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-367-\t}\narch/x86/kvm/vmx/nested.c:368:\tkfree(vmx-\u003enested.cached_vmcs12);\narch/x86/kvm/vmx/nested.c:369:\tvmx-\u003enested.cached_vmcs12 = NULL;\narch/x86/kvm/vmx/nested.c-370-\tkfree(vmx-\u003enested.cached_shadow_vmcs12);\n--\narch/x86/kvm/vmx/nested.c=1773=static void copy_enlightened_to_vmcs12(struct vcpu_vmx vmx, u32 hv_clean_fields)\n--\narch/x86/kvm/vmx/nested.c-1775-#ifdef CONFIG_KVM_HYPERV\narch/x86/kvm/vmx/nested.c:1776:\tstruct vmcs12 vmcs12 = vmx-\u003enested.cached_vmcs12;\narch/x86/kvm/vmx/nested.c-1777-\tstruct hv_enlightened_vmcs evmcs = nested_vmx_evmcs(vmx);\n--\narch/x86/kvm/vmx/nested.c=2021=static void copy_vmcs12_to_enlightened(struct vcpu_vmx vmx)\n--\narch/x86/kvm/vmx/nested.c-2023-#ifdef CONFIG_KVM_HYPERV\narch/x86/kvm/vmx/nested.c:2024:\tstruct vmcs12 vmcs12 = vmx-\u003enested.cached_vmcs12;\narch/x86/kvm/vmx/nested.c-2025-\tstruct hv_enlightened_vmcs evmcs = nested_vmx_evmcs(vmx);\n--\narch/x86/kvm/vmx/nested.c=5622=static int enter_vmx_operation(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-5630-\narch/x86/kvm/vmx/nested.c:5631:\tvmx-\u003enested.cached_vmcs12 = kzalloc(VMCS12_SIZE, GFP_KERNEL_ACCOUNT);\narch/x86/kvm/vmx/nested.c:5632:\tif (!vmx-\u003enested.cached_vmcs12)\narch/x86/kvm/vmx/nested.c:5633:\t\tgoto out_cached_vmcs12;\narch/x86/kvm/vmx/nested.c-5634-\n--\narch/x86/kvm/vmx/nested.c-5668-out_cached_shadow_vmcs12:\narch/x86/kvm/vmx/nested.c:5669:\tkfree(vmx-\u003enested.cached_vmcs12);\narch/x86/kvm/vmx/nested.c-5670-\narch/x86/kvm/vmx/nested.c:5671:out_cached_vmcs12:\narch/x86/kvm/vmx/nested.c-5672-\tfree_loaded_vmcs(\u0026vmx-\u003enested.vmcs02);\n--\narch/x86/kvm/vmx/nested.c=5770=static inline void nested_release_vmcs12(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-5789-\t\t\t\t vmx-\u003enested.current_vmptr \u003e\u003e PAGE_SHIFT,\narch/x86/kvm/vmx/nested.c:5790:\t\t\t\t vmx-\u003enested.cached_vmcs12, 0, VMCS12_SIZE);\narch/x86/kvm/vmx/nested.c-5791-\n--\narch/x86/kvm/vmx/nested.c=6093=static int handle_vmptrld(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-6149-\t\t /\narch/x86/kvm/vmx/nested.c:6150:\t\tif (kvm_read_guest_cached(vcpu-\u003ekvm, ghc, vmx-\u003enested.cached_vmcs12,\narch/x86/kvm/vmx/nested.c-6151-\t\t\t\t\t VMCS12_SIZE)) {\n--\narch/x86/kvm/vmx/nested.h=58=static inline struct vmcs12 get_vmcs12(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.h-61-\narch/x86/kvm/vmx/nested.h:62:\treturn to_vmx(vcpu)-\u003enested.cached_vmcs12;\narch/x86/kvm/vmx/nested.h-63-}\n--\narch/x86/kvm/vmx/vmx.h=73=struct nested_vmx {\n--\narch/x86/kvm/vmx/vmx.h-85-\t /\narch/x86/kvm/vmx/vmx.h:86:\tstruct vmcs12 cached_vmcs12;\narch/x86/kvm/vmx/vmx.h-87-\t/\n--\ntools/testing/selftests/kvm/x86/evmcs_smm_controls_test.c=68=int main(int argc, char argv[])\n--\ntools/testing/selftests/kvm/x86/evmcs_smm_controls_test.c-122-\t Trigger copy_enlightened_to_vmcs12() via KVM_GET_NESTED_STATE,\ntools/testing/selftests/kvm/x86/evmcs_smm_controls_test.c:123:\t * copying the invalid pin_based_vm_exec_control into cached_vmcs12.\ntools/testing/selftests/kvm/x86/evmcs_smm_controls_test.c-124-\t */\n" }
47/3	2026/09/11 21:52	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 91160 output: 20 thoughts: 74 Thoughts: My Analysis of the `cached_vmcs12` Structure Okay, so I'm dealing with `cached_vmcs12` here. The code allocates this using `kzalloc`. Right, so it's initialized to zero. Got it. Now, when the guest reads from it using `kvm_read_guest_cached`, the entire `VMCS12_SIZE` is read into this `cached_vmcs12` structure. That sounds straightforward enough, a full read to populate the cache. But hang on a second... what is `VMCS12_SIZE` exactly? I can’t proceed further in my analysis until I know the size of this `VMCS12_SIZE`. I need to verify that the assumptions being made about the cache size are valid. Time to go check the definition.
48/3	2026/09/11 21:52	tool	grepper	0m Args: { "Expression": "VMCS12_SIZE" } Results: { "Output": "arch/x86/kvm/vmx/nested.c=860=static void nested_cache_shadow_vmcs12(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/vmx/nested.c-871-\t kvm_gfn_to_hva_cache_init(vcpu-\u003ekvm, ghc,\narch/x86/kvm/vmx/nested.c:872:\t\t\t\t vmcs12-\u003evmcs_link_pointer, VMCS12_SIZE))\narch/x86/kvm/vmx/nested.c-873-\t\treturn;\n--\narch/x86/kvm/vmx/nested.c-875-\tkvm_read_guest_cached(vcpu-\u003ekvm, ghc, get_shadow_vmcs12(vcpu),\narch/x86/kvm/vmx/nested.c:876:\t\t\t VMCS12_SIZE);\narch/x86/kvm/vmx/nested.c-877-}\n--\narch/x86/kvm/vmx/nested.c=879=static void nested_flush_cached_shadow_vmcs12(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/vmx/nested.c-890-\t kvm_gfn_to_hva_cache_init(vcpu-\u003ekvm, ghc,\narch/x86/kvm/vmx/nested.c:891:\t\t\t\t vmcs12-\u003evmcs_link_pointer, VMCS12_SIZE))\narch/x86/kvm/vmx/nested.c-892-\t\treturn;\n--\narch/x86/kvm/vmx/nested.c-894-\tkvm_write_guest_cached(vcpu-\u003ekvm, ghc, get_shadow_vmcs12(vcpu),\narch/x86/kvm/vmx/nested.c:895:\t\t\t VMCS12_SIZE);\narch/x86/kvm/vmx/nested.c-896-}\n--\narch/x86/kvm/vmx/nested.c=3380=static int nested_vmx_check_vmcs_link_ptr(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/vmx/nested.c-3394-\t CC(kvm_gfn_to_hva_cache_init(vcpu-\u003ekvm, ghc,\narch/x86/kvm/vmx/nested.c:3395:\t\t\t\t\t vmcs12-\u003evmcs_link_pointer, VMCS12_SIZE)))\narch/x86/kvm/vmx/nested.c-3396- return -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=5622=static int enter_vmx_operation(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-5630-\narch/x86/kvm/vmx/nested.c:5631:\tvmx-\u003enested.cached_vmcs12 = kzalloc(VMCS12_SIZE, GFP_KERNEL_ACCOUNT);\narch/x86/kvm/vmx/nested.c-5632-\tif (!vmx-\u003enested.cached_vmcs12)\n--\narch/x86/kvm/vmx/nested.c-5635-\tvmx-\u003enested.shadow_vmcs12_cache.gpa = INVALID_GPA;\narch/x86/kvm/vmx/nested.c:5636:\tvmx-\u003enested.cached_shadow_vmcs12 = kzalloc(VMCS12_SIZE, GFP_KERNEL_ACCOUNT);\narch/x86/kvm/vmx/nested.c-5637-\tif (!vmx-\u003enested.cached_shadow_vmcs12)\n--\narch/x86/kvm/vmx/nested.c=5770=static inline void nested_release_vmcs12(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-5789-\t\t\t\t vmx-\u003enested.current_vmptr \u003e\u003e PAGE_SHIFT,\narch/x86/kvm/vmx/nested.c:5790:\t\t\t\t vmx-\u003enested.cached_vmcs12, 0, VMCS12_SIZE);\narch/x86/kvm/vmx/nested.c-5791-\n--\narch/x86/kvm/vmx/nested.c=6093=static int handle_vmptrld(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-6118-\narch/x86/kvm/vmx/nested.c:6119:\t\tif (kvm_gfn_to_hva_cache_init(vcpu-\u003ekvm, ghc, vmptr, VMCS12_SIZE)) {\narch/x86/kvm/vmx/nested.c-6120-\t\t\t/\n--\narch/x86/kvm/vmx/nested.c-6150-\t\tif (kvm_read_guest_cached(vcpu-\u003ekvm, ghc, vmx-\u003enested.cached_vmcs12,\narch/x86/kvm/vmx/nested.c:6151:\t\t\t\t\t VMCS12_SIZE)) {\narch/x86/kvm/vmx/nested.c-6152-\t\t\treturn nested_vmx_fail(vcpu,\n--\narch/x86/kvm/vmx/nested.c=6956=static int vmx_get_nested_state(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/vmx/nested.c-7058-\narch/x86/kvm/vmx/nested.c:7059:\tBUILD_BUG_ON(sizeof(user_vmx_nested_state-\u003evmcs12) \u003c VMCS12_SIZE);\narch/x86/kvm/vmx/nested.c:7060:\tBUILD_BUG_ON(sizeof(user_vmx_nested_state-\u003eshadow_vmcs12) \u003c VMCS12_SIZE);\narch/x86/kvm/vmx/nested.c-7061-\n--\narch/x86/kvm/vmx/nested.c-7065-\t /\narch/x86/kvm/vmx/nested.c:7066:\tif (copy_to_user(user_vmx_nested_state-\u003evmcs12, vmcs12, VMCS12_SIZE))\narch/x86/kvm/vmx/nested.c-7067-\t\treturn -EFAULT;\n--\narch/x86/kvm/vmx/nested.c-7071-\t\tif (copy_to_user(user_vmx_nested_state-\u003eshadow_vmcs12,\narch/x86/kvm/vmx/nested.c:7072:\t\t\t\t get_shadow_vmcs12(vcpu), VMCS12_SIZE))\narch/x86/kvm/vmx/nested.c-7073-\t\t\treturn -EFAULT;\n--\narch/x86/kvm/vmx/nested.c=7539=static void nested_vmx_setup_basic(struct nested_vmx_msrs msrs)\n--\narch/x86/kvm/vmx/nested.c-7546-\t /\narch/x86/kvm/vmx/nested.c:7547:\tmsrs-\u003ebasic = vmx_basic_encode_vmcs_info(VMCS12_REVISION, VMCS12_SIZE,\narch/x86/kvm/vmx/nested.c-7548-\t\t\t\t\t\t X86_MEMTYPE_WB);\n--\narch/x86/kvm/vmx/vmcs12.h=27=struct __packed vmcs12 {\n--\narch/x86/kvm/vmx/vmcs12.h-228-/\narch/x86/kvm/vmx/vmcs12.h:229: * VMCS12_SIZE is the number of bytes L1 should allocate for the VMXON region\narch/x86/kvm/vmx/vmcs12.h-230- * and any VMCS region. Although only sizeof(struct vmcs12) are used by the\n--\narch/x86/kvm/vmx/vmcs12.h-233- */\narch/x86/kvm/vmx/vmcs12.h:234:#define VMCS12_SIZE\t\tKVM_STATE_NESTED_VMX_VMCS_SIZE\narch/x86/kvm/vmx/vmcs12.h-235-\n" }
49/3	2026/09/11 21:52	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 93709 output: 44 thoughts: 78 Thoughts: Here's my summary, as though I were thinking through the problem: Diving into the VMX Nested State Retrieval Okay, I need to understand how the nested VM state is being retrieved. Let's start by looking at `arch/x86/kvm/vmx/nested.c`. My immediate target is lines 7058-7075. Ah, `vmx_get_nested_state` is the function responsible for getting the nested state. And, crucially, it looks like it copies `vmcs12` to user space. Interesting, so that's the mechanism being used for this transfer. To fully understand what's happening I should read around line 7055 of `vmx_get_nested_state`. I need to see how this `vmcs12` is being copied, what checks are being done, and any potential vulnerabilities in that process. Let's see...
50/3	2026/09/11 21:52	tool	read-file	0m Args: { "File": "arch/x86/kvm/vmx/nested.c", "FirstLine": 7040, "LineCount": 40 } Results: { "Contents": "7040:\t\t\tsync_vmcs02_to_vmcs12(vcpu, vmcs12);\n7041:\t\t\tsync_vmcs02_to_vmcs12_rare(vcpu, vmcs12);\n7042:\t\t} else {\n7043:\t\t\tcopy_vmcs02_to_vmcs12_rare(vcpu, get_vmcs12(vcpu));\n7044:\t\t\tif (!vmx-\u003enested.need_vmcs12_to_shadow_sync) {\n7045:\t\t\t\tif (nested_vmx_is_evmptr12_valid(vmx))\n7046:\t\t\t\t\t/\n7047:\t\t\t\t\t L1 hypervisor is not obliged to keep eVMCS\n7048:\t\t\t\t\t * clean fields data always up-to-date while\n7049:\t\t\t\t\t * not in guest mode, 'hv_clean_fields' is only\n7050:\t\t\t\t\t * supposed to be actual upon vmentry so we need\n7051:\t\t\t\t\t * to ignore it here and do full copy.\n7052:\t\t\t\t\t /\n7053:\t\t\t\t\tcopy_enlightened_to_vmcs12(vmx, 0);\n7054:\t\t\t\telse if (enable_shadow_vmcs)\n7055:\t\t\t\t\tcopy_shadow_to_vmcs12(vmx);\n7056:\t\t\t}\n7057:\t\t}\n7058:\t\n7059:\t\tBUILD_BUG_ON(sizeof(user_vmx_nested_state-\u003evmcs12) \u003c VMCS12_SIZE);\n7060:\t\tBUILD_BUG_ON(sizeof(user_vmx_nested_state-\u003eshadow_vmcs12) \u003c VMCS12_SIZE);\n7061:\t\n7062:\t\t/\n7063:\t\t * Copy over the full allocated size of vmcs12 rather than just the size\n7064:\t\t * of the struct.\n7065:\t\t /\n7066:\t\tif (copy_to_user(user_vmx_nested_state-\u003evmcs12, vmcs12, VMCS12_SIZE))\n7067:\t\t\treturn -EFAULT;\n7068:\t\n7069:\t\tif (nested_cpu_has_shadow_vmcs(vmcs12) \u0026\u0026\n7070:\t\t vmcs12-\u003evmcs_link_pointer != INVALID_GPA) {\n7071:\t\t\tif (copy_to_user(user_vmx_nested_state-\u003eshadow_vmcs12,\n7072:\t\t\t\t\t get_shadow_vmcs12(vcpu), VMCS12_SIZE))\n7073:\t\t\t\treturn -EFAULT;\n7074:\t\t}\n7075:\tout:\n7076:\t\treturn kvm_state.size;\n7077:\t}\n7078:\t\n7079:\tvoid vmx_leave_nested(struct kvm_vcpu vcpu)\n" }
51/3	2026/09/11 21:52	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 94796 output: 46 thoughts: 111 Thoughts: Deconstructing the VMCS12 Structure Okay, so I'm diving into this `vmcs12` variable. It looks like it's actually referring to `cached_vmcs12`, which was allocated using `kzalloc`. Right, `GFP_KERNEL_ACCOUNT`... standard allocation flags. Now, I need to understand exactly what `vmcs12` represents. My gut tells me it's more than just a simple integer. It's time to dig into the definition. What's the underlying `struct __packed vmcs12` look like, and how big is it? Let me jump over to the `arch/x86/kvm/vmx/vmcs12.h` header file. I need to get a clear picture of its composition and the memory layout. Let me check the lines around 190 to 240 in `vmcs12.h`. That should give me the details I need to start dissecting this further.
52/3	2026/09/11 21:52	tool	read-file	0m Args: { "File": "arch/x86/kvm/vmx/vmcs12.h", "FirstLine": 190, "LineCount": 50 } Results: { "Contents": " 190:\t\tu16 host_fs_selector;\n 191:\t\tu16 host_gs_selector;\n 192:\t\tu16 host_tr_selector;\n 193:\t\tu16 guest_pml_index;\n 194:\t\tu16 padding16[1]; /* align to 64-bit boundary /\n 195:\t\tstruct vmcs_fred_msrs guest_fred_msrs;\n 196:\t#define guest_ia32_fred_config\tguest_fred_msrs.fred_config\n 197:\t#define guest_ia32_fred_rsp1\tguest_fred_msrs.fred_rsp1\n 198:\t#define guest_ia32_fred_rsp2\tguest_fred_msrs.fred_rsp2\n 199:\t#define guest_ia32_fred_rsp3\tguest_fred_msrs.fred_rsp3\n 200:\t#define guest_ia32_fred_stklvls\tguest_fred_msrs.fred_stklvls\n 201:\t#define guest_ia32_fred_ssp1\tguest_fred_msrs.fred_ssp1\n 202:\t#define guest_ia32_fred_ssp2\tguest_fred_msrs.fred_ssp2\n 203:\t#define guest_ia32_fred_ssp3\tguest_fred_msrs.fred_ssp3\n 204:\t\tstruct vmcs_fred_msrs host_fred_msrs;\n 205:\t#define host_ia32_fred_config\thost_fred_msrs.fred_config\n 206:\t#define host_ia32_fred_rsp1\thost_fred_msrs.fred_rsp1\n 207:\t#define host_ia32_fred_rsp2\thost_fred_msrs.fred_rsp2\n 208:\t#define host_ia32_fred_rsp3\thost_fred_msrs.fred_rsp3\n 209:\t#define host_ia32_fred_stklvls\thost_fred_msrs.fred_stklvls\n 210:\t#define host_ia32_fred_ssp1\thost_fred_msrs.fred_ssp1\n 211:\t#define host_ia32_fred_ssp2\thost_fred_msrs.fred_ssp2\n 212:\t#define host_ia32_fred_ssp3\thost_fred_msrs.fred_ssp3\n 213:\t\tu64 injected_event_data;\n 214:\t\tu64 original_event_data;\n 215:\t};\n 216:\t\n 217:\t/\n 218:\t * VMCS12_REVISION is KVM's arbitrary ID for the layout of struct vmcs12. KVM\n 219:\t * enumerates this value to L1 via MSR_IA32_VMX_BASIC, and checks the revision\n 220:\t * ID during nested VMPTRLD to verify that L1 is loading a VMCS that adhere's\n 221:\t * to KVM's virtual CPU definition.\n 222:\t \n 223:\t DO NOT change this value, as it will break save/restore compatibility with\n 224:\t * older KVM releases.\n 225:\t /\n 226:\t#define VMCS12_REVISION 0x11e57ed0\n 227:\t\n 228:\t/\n 229:\t * VMCS12_SIZE is the number of bytes L1 should allocate for the VMXON region\n 230:\t * and any VMCS region. Although only sizeof(struct vmcs12) are used by the\n 231:\t * current implementation, 4K are reserved to avoid future complications and\n 232:\t * to preserve userspace ABI.\n 233:\t /\n 234:\t#define VMCS12_SIZE\t\tKVM_STATE_NESTED_VMX_VMCS_SIZE\n 235:\t\n 236:\t/\n 237:\t * For save/restore compatibility, the vmcs12 field offsets must not change,\n 238:\t * although appending fields and/or filling gaps is obviously allowed.\n 239:\t */\n" }
53/3	2026/09/11 21:52	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 96291 output: 48 thoughts: 1495 Thoughts: Analyzing FRED State Handling in Nested VMX Okay, so I'm diving into this nested VMX code, focusing on the FRED (Fast Return from Exception Delivery) feature. The first thing I note is how `struct __packed vmcs12` is defined. The `__packed` attribute is key here; it guarantees no padding, which is critical for the memory layout, and this is important as `cached_vmcs12` is allocated with `kzalloc`, and later filled from guest memory via `kvm_read_guest_cached` during L1 VMPTRLD. This sets the stage. Now, I'm tracing the execution path, particularly during non-root mode entry. I see several code snippets dealing with FRED state. Let's start with what happens when entering non-root mode. When entering non-root mode, if FRED is available, and `nested_run_pending` is false, or `nested_cpu_load_guest_fred_state` returns false, then the code calls `vmcs_read_fred_msrs` to fetch `pre_vmenter_fred_msrs`. Then, there's `sync_vmcs02_to_vmcs12`, which is crucial. It saves the guest FRED state via `nested_cpu_save_guest_fred_state` and copies the `at_vmexit_fred_msrs` into `vmcs12->guest_fred_msrs`. So, we're definitely transitioning data here. Next, I look at `vmcs12_save_pending_event`. This is where `original_event_data` is saved, but also there is a check for `X86_CR4_FRED` in `vmcs12->guest_cr4`, and if an exception is nested, it sets `VECTORING_INFO_NESTED_EXCEPTION_MASK`. It appears related to exception handling within the nested environment and can modify `original_event_data`, but it doesn't seem to affect FRED MSRs directly. Then, I'm checking `prepare_vmcs02_rare` and `prepare_vmcs02`. Both of them, if FRED is enabled and `nested_cpu_load_guest_fred_state` returns true in `prepare_vmcs02_rare`, or `nested_run_pending` is false, or `nested_cpu_load_guest_fred_state` returns false in `prepare_vmcs02`, write `guest_fred_msrs` or `pre_vmenter_fred_msrs`. This is all about preparing VMCS02 for an upcoming vmenter and VMX. Now, `load_vmcs12_host_state` is the most interesting part. Here, if FRED is available, the code tries to load the host FRED state via `nested_cpu_load_host_fred_state`. If this succeeds, it writes the host FRED MSRs to the VMCS. However, if it fails, and critically if `!from_failed_vmentry` (which, let's keep in mind, we're assuming is false here), then it'll use `at_vmexit_fred_msrs` instead. I'm wondering if `at_vmexit_fred_msrs` is uninitialized. The conditional there is what I want to understand. If `from_failed_vmentry` is true and `nested_cpu_load_host_fred_state` fails, then nothing is loaded, which makes sense, as the active VMCS01 host FRED state hasn't changed. Now, I'm tracking where `load_vmcs12_host_state` gets called, as that'll shed light on the `from_failed_vmentry` flag. It is called from two places: `nested_vmx_enter_non_root_mode` and `__nested_vmx_vmexit`. In `nested_vmx_enter_non_root_mode`, `from_failed_vmentry` is set to `true`. This means the code branch will not use `at_vmexit_fred_msrs` because the `!from_failed_vmentry` check will fail. This is intentional to keep it consistent with the host VMCS01 state on a failed entry. In `__nested_vmx_vmexit`, `from_failed_vmentry` is false. This will write `at_vmexit_fred_msrs` when `nested_cpu_load_host_fred_state` fails. That means we should check when `sync_vmcs02_to_vmcs1
54/3	2026/09/11 21:52	tool	grepper	0m Args: { "Expression": "sync_vmcs02_to_vmcs12", "PathPrefix": "arch/x86/kvm/vmx/nested.c" } Results: { "Output": "arch/x86/kvm/vmx/nested.c=2021=static void copy_vmcs12_to_enlightened(struct vcpu_vmx vmx)\n--\narch/x86/kvm/vmx/nested.c-2051-\t evmcs-\u003ehost_rsp = vmcs12-\u003ehost_rsp;\narch/x86/kvm/vmx/nested.c:2052:\t * sync_vmcs02_to_vmcs12() doesn't read these:\narch/x86/kvm/vmx/nested.c-2053-\t * evmcs-\u003eio_bitmap_a = vmcs12-\u003eio_bitmap_a;\n--\narch/x86/kvm/vmx/nested.c=4700=static bool is_vmcs12_ext_field(unsigned long field)\n--\narch/x86/kvm/vmx/nested.c-4746-\narch/x86/kvm/vmx/nested.c:4747:static void sync_vmcs02_to_vmcs12_rare(struct kvm_vcpu vcpu,\narch/x86/kvm/vmx/nested.c-4748-\t\t\t\t struct vmcs12 vmcs12)\n--\narch/x86/kvm/vmx/nested.c-4788-\narch/x86/kvm/vmx/nested.c:4789:\tvmx-\u003enested.need_sync_vmcs02_to_vmcs12_rare = false;\narch/x86/kvm/vmx/nested.c-4790-}\n--\narch/x86/kvm/vmx/nested.c=4792=static void copy_vmcs02_to_vmcs12_rare(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/vmx/nested.c-4797-\narch/x86/kvm/vmx/nested.c:4798:\tif (!vmx-\u003enested.need_sync_vmcs02_to_vmcs12_rare)\narch/x86/kvm/vmx/nested.c-4799-\t\treturn;\n--\narch/x86/kvm/vmx/nested.c-4807-\narch/x86/kvm/vmx/nested.c:4808:\tsync_vmcs02_to_vmcs12_rare(vcpu, vmcs12);\narch/x86/kvm/vmx/nested.c-4809-\n--\narch/x86/kvm/vmx/nested.c-4820- /\narch/x86/kvm/vmx/nested.c:4821:static void sync_vmcs02_to_vmcs12(struct kvm_vcpu vcpu, struct vmcs12 vmcs12)\narch/x86/kvm/vmx/nested.c-4822-{\n--\narch/x86/kvm/vmx/nested.c-4825-\tif (nested_vmx_is_evmptr12_valid(vmx))\narch/x86/kvm/vmx/nested.c:4826:\t\tsync_vmcs02_to_vmcs12_rare(vcpu, vmcs12);\narch/x86/kvm/vmx/nested.c-4827-\narch/x86/kvm/vmx/nested.c:4828:\tvmx-\u003enested.need_sync_vmcs02_to_vmcs12_rare =\narch/x86/kvm/vmx/nested.c-4829-\t\t!nested_vmx_is_evmptr12_valid(vmx);\n--\narch/x86/kvm/vmx/nested.c=5275=void __nested_vmx_vmexit(struct kvm_vcpu vcpu, u32 vm_exit_reason,\n--\narch/x86/kvm/vmx/nested.c-5323-\tif (likely(!vmx-\u003efail)) {\narch/x86/kvm/vmx/nested.c:5324:\t\tsync_vmcs02_to_vmcs12(vcpu, vmcs12);\narch/x86/kvm/vmx/nested.c-5325-\n--\narch/x86/kvm/vmx/nested.c-5331-\t\t/\narch/x86/kvm/vmx/nested.c:5332:\t\t * Must happen outside of sync_vmcs02_to_vmcs12() as it will\narch/x86/kvm/vmx/nested.c-5333-\t\t * also be used to capture vmcs12 cache as part of\n--\narch/x86/kvm/vmx/nested.c=6956=static int vmx_get_nested_state(struct kvm_vcpu *vcpu,\n--\narch/x86/kvm/vmx/nested.c-7039-\tif (is_guest_mode(vcpu)) {\narch/x86/kvm/vmx/nested.c:7040:\t\tsync_vmcs02_to_vmcs12(vcpu, vmcs12);\narch/x86/kvm/vmx/nested.c:7041:\t\tsync_vmcs02_to_vmcs12_rare(vcpu, vmcs12);\narch/x86/kvm/vmx/nested.c-7042-\t} else {\n" }
55/3	2026/09/11 21:52	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 100306 output: 44 thoughts: 122 Thoughts: Investigating a Potential Failure Scenario in Nested VMX VM Exit Okay, here we go, back in `__nested_vmx_vmexit`. Let's zero in on line 5324. We're assuming the happy path, where `vmx->fail` is unlikely. We're syncing `vmcs02` to `vmcs12`, then loading the host state. But what happens if `vmx->fail` is true? That's what I need to understand. Something might be going awry. Let's backtrack a few lines, right around line 5320, and re-examine that conditional. I need to figure out the code path when a VMX failure occurs within this exit. Time to dig in and see what's happening.
56/3	2026/09/11 21:52	tool	read-file	0m Args: { "File": "arch/x86/kvm/vmx/nested.c", "FirstLine": 5320, "LineCount": 85 } Results: { "Contents": "5320:\t\t\t\tvcpu-\u003earch.tsc_scaling_ratio = vcpu-\u003earch.l1_tsc_scaling_ratio;\n5321:\t\t}\n5322:\t\n5323:\t\tif (likely(!vmx-\u003efail)) {\n5324:\t\t\tsync_vmcs02_to_vmcs12(vcpu, vmcs12);\n5325:\t\n5326:\t\t\tif (vm_exit_reason != -1)\n5327:\t\t\t\tprepare_vmcs12(vcpu, vmcs12, vm_exit_reason,\n5328:\t\t\t\t\t exit_intr_info, exit_qualification,\n5329:\t\t\t\t\t exit_insn_len);\n5330:\t\n5331:\t\t\t/\n5332:\t\t\t Must happen outside of sync_vmcs02_to_vmcs12() as it will\n5333:\t\t\t * also be used to capture vmcs12 cache as part of\n5334:\t\t\t * capturing nVMX state for snapshot (migration).\n5335:\t\t\t \n5336:\t\t\t Otherwise, this flush will dirty guest memory at a\n5337:\t\t\t * point it is already assumed by user-space to be\n5338:\t\t\t * immutable.\n5339:\t\t\t /\n5340:\t\t\tnested_flush_cached_shadow_vmcs12(vcpu, vmcs12);\n5341:\t\t} else {\n5342:\t\t\t/\n5343:\t\t\t * The only expected VM-instruction error is \"VM entry with\n5344:\t\t\t * invalid control field(s).\" Anything else indicates a\n5345:\t\t\t * problem with L0.\n5346:\t\t\t /\n5347:\t\t\tWARN_ON_ONCE(vmcs_read32(VM_INSTRUCTION_ERROR) !=\n5348:\t\t\t\t VMXERR_ENTRY_INVALID_CONTROL_FIELD);\n5349:\t\n5350:\t\t\t/ VM-Fail at VM-Entry means KVM missed a consistency check. /\n5351:\t\t\tWARN_ON_ONCE(warn_on_missed_cc);\n5352:\t\t}\n5353:\t\n5354:\t\t/\n5355:\t\t * Drop events/exceptions that were queued for re-injection to L2\n5356:\t\t * (picked up via vmx_complete_interrupts()), as well as exceptions\n5357:\t\t * that were pending for L2. Note, this must NOT be hoisted above\n5358:\t\t * prepare_vmcs12(), events/exceptions queued for re-injection need to\n5359:\t\t * be captured in vmcs12 (see vmcs12_save_pending_event()).\n5360:\t\t /\n5361:\t\tvcpu-\u003earch.nmi_injected = false;\n5362:\t\tkvm_clear_exception_queue(vcpu);\n5363:\t\tkvm_clear_interrupt_queue(vcpu);\n5364:\t\n5365:\t\tvmx_switch_vmcs(vcpu, \u0026vmx-\u003evmcs01);\n5366:\t\n5367:\t\tkvm_nested_vmexit_handle_ibrs(vcpu);\n5368:\t\n5369:\t\t/\n5370:\t\t * Update any VMCS fields that might have changed while vmcs02 was the\n5371:\t\t * active VMCS. The tracking is per-vCPU, not per-VMCS.\n5372:\t\t /\n5373:\t\tvmcs_write32(VM_EXIT_MSR_STORE_COUNT, vmx-\u003emsr_autostore.nr);\n5374:\t\tvmcs_write32(VM_EXIT_MSR_LOAD_COUNT, vmx-\u003emsr_autoload.host.nr);\n5375:\t\tvmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, vmx-\u003emsr_autoload.guest.nr);\n5376:\t\tvmcs_write64(TSC_OFFSET, vcpu-\u003earch.tsc_offset);\n5377:\t\tif (kvm_caps.has_tsc_control)\n5378:\t\t\tvmcs_write64(TSC_MULTIPLIER, vcpu-\u003earch.tsc_scaling_ratio);\n5379:\t\n5380:\t\tnested_put_vmcs12_pages(vcpu);\n5381:\t\n5382:\t\tif ((vm_exit_reason != -1) \u0026\u0026\n5383:\t\t (enable_shadow_vmcs \|\| nested_vmx_is_evmptr12_valid(vmx)))\n5384:\t\t\tvmx-\u003enested.need_vmcs12_to_shadow_sync = true;\n5385:\t\n5386:\t\t/ in case we halted in L2 /\n5387:\t\tkvm_set_mp_state(vcpu, KVM_MP_STATE_RUNNABLE);\n5388:\t\n5389:\t\tif (likely(!vmx-\u003efail)) {\n5390:\t\t\tif (vm_exit_reason != -1)\n5391:\t\t\t\ttrace_kvm_nested_vmexit_inject(vmcs12-\u003evm_exit_reason,\n5392:\t\t\t\t\t\t\t vmcs12-\u003eexit_qualification,\n5393:\t\t\t\t\t\t\t vmcs12-\u003eidt_vectoring_info_field,\n5394:\t\t\t\t\t\t\t vmcs12-\u003evm_exit_intr_info,\n5395:\t\t\t\t\t\t\t vmcs12-\u003evm_exit_intr_error_code,\n5396:\t\t\t\t\t\t\t KVM_ISA_VMX);\n5397:\t\n5398:\t\t\tload_vmcs12_host_state(vcpu, vmcs12, false);\n5399:\t\n5400:\t\t\t/\n5401:\t\t\t * Process events if an injectable IRQ or NMI is pending, even\n5402:\t\t\t * if the event is blocked (RFLAGS.IF is cleared on VM-Exit).\n5403:\t\t\t * If an event became pending while L2 was active, KVM needs to\n5404:\t\t\t * either inject the event or request an IRQ/NMI window. SMIs\n" }
57/3	2026/09/11 21:52	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 102215 output: 25 thoughts: 3450 Thoughts: FRED MSRs and VMCS State Synchronization Analysis Okay, let's break this down systematically. My initial observation focuses on the sequencing of operations involving FRED (Fast Return Error Detection) MSRs and the VMCS (Virtual Machine Control Structure). The critical point seems to be that `vmx->nested.at_vmexit_fred_msrs` must be written before `load_vmcs12_host_state` is called, and, if `vmx->fail` is true, then `load_vmcs12_host_state` isn't called, at all. So we're covered there. Now, let's circle back to the CEA (CPU Entry Area) exception stacks. I want to be sure there are no uninitialized memory issues related to changes there. The diffs in `arch/x86/include/asm/cpu_entry_area.h` and `arch/x86/mm/cpu_entry_area.c` look like they're purely address computations, and no actual data, so there's nothing there that would lead to uninitialized memory issues. I'm now going to focus on the static constant tables for stack dumping in `dumpstack_64.c`. I need to ensure they are using the correct sizes and offsets for the exception stacks. Upon review of the code, everything there seems to be consistent. I've got to consider how `this_cpu_fred_rsp()` obtains the top of the stack. We're getting that from IST (Interrupt Stack Table) stacks which are the exception stacks. Now, let's make sure that FRED stacks are being initialized properly. I'm focusing on `vmx_vcpu_load_vmcs`, `vmx_set_constant_host_state`, and `init_vmcs`. `vmx_vcpu_load_vmcs` writes FRED RSP values from the IST, and `vmx_set_constant_host_state` writes FRED configuration and STKLVLS. It's important to guard these writes with a FRED feature check. I want to make sure it runs correctly even if FRED is not enabled. Good, the `if` guard prevents this function from running when FRED is not enabled. Also, `kvm_host.fred_config` comes from the MSR when FRED is enabled. And of course, `init_vmcs` initializes the FRED MSRs to zero. That's a solid baseline. Let's look at `setup_vmcs_config` and `vmx_get_msr`, and `vmx_set_msr`. The critical thing here is how MSRs are read and written within the VMX context. `vmx_get_msr` and `vmx_set_msr` look sane. The access to `MSR_IA32_FRED_RSP1` and so on, uses the helper function `fred_msr_to_vmcs`. So, it all comes down to `vmx_read_guest_fred_rsp0(vmx)` and how it handles reading the MSR. That is not something that would affect this. I've got to investigate `vmx_read_guest_host_msr`.
58/3	2026/09/11 21:53	tool	grepper	0m Args: { "Expression": "vmx_read_guest_host_msr" } Results: { "Output": "arch/x86/kvm/vmx/vmx.c=1412=static void vmx_prepare_switch_to_host(struct vcpu_vmx vmx)\n--\narch/x86/kvm/vmx/vmx.c-1461-#ifdef CONFIG_X86_64\narch/x86/kvm/vmx/vmx.c:1462:static u64 vmx_read_guest_host_msr(struct vcpu_vmx vmx, u32 msr, u64 cache)\narch/x86/kvm/vmx/vmx.c-1463-{\n--\narch/x86/kvm/vmx/vmx.c=1481=static u64 vmx_read_guest_kernel_gs_base(struct vcpu_vmx vmx)\narch/x86/kvm/vmx/vmx.c-1482-{\narch/x86/kvm/vmx/vmx.c:1483:\treturn vmx_read_guest_host_msr(vmx, MSR_KERNEL_GS_BASE,\narch/x86/kvm/vmx/vmx.c-1484-\t\t\t\t \u0026vmx-\u003emsr_guest_kernel_gs_base);\n--\narch/x86/kvm/vmx/vmx.c=1493=static u64 vmx_read_guest_fred_rsp0(struct vcpu_vmx *vmx)\narch/x86/kvm/vmx/vmx.c-1494-{\narch/x86/kvm/vmx/vmx.c:1495:\treturn vmx_read_guest_host_msr(vmx, MSR_IA32_FRED_RSP0,\narch/x86/kvm/vmx/vmx.c-1496-\t\t\t\t \u0026vmx-\u003emsr_guest_fred_rsp0);\n" }
59/3	2026/09/11 21:53	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 106904 output: 45 thoughts: 37 Thoughts: Okay, here's my interpretation of that thought process, as if I'm the one having it, and assuming my expertise in the field: Diving into the VMX MSR Read Right, time to get my hands dirty. I need to understand how KVM handles reading Model-Specific Registers (MSRs) from the guest when running within VMX. Specifically, I'm heading straight to the heart of the matter – the `vmx_read_guest_host_msr` function. I know this is the place where the magic happens, or at least, the core logic for the VMX side of the MSR read operation resides. I'm going to find this function in `arch/x86/kvm/vmx/vmx.c`. This is where the intricacies of VMX's interaction with MSR reads would be implemented. Let's see how the hypervisor intercepts, emulates, or passes through these requests. It is important to know how it manages any privilege escalation considerations too, and whether any sanitization is needed. Time to dig in!
60/3	2026/09/11 21:53	tool	read-file	0m Args: { "File": "arch/x86/kvm/vmx/vmx.c", "FirstLine": 1460, "LineCount": 30 } Results: { "Contents": "1460:\t\n1461:\t#ifdef CONFIG_X86_64\n1462:\tstatic u64 vmx_read_guest_host_msr(struct vcpu_vmx vmx, u32 msr, u64 cache)\n1463:\t{\n1464:\t\tpreempt_disable();\n1465:\t\tif (vmx-\u003evt.guest_state_loaded)\n1466:\t\t\tcache = read_msr(msr);\n1467:\t\tpreempt_enable();\n1468:\t\treturn cache;\n1469:\t}\n1470:\t\n1471:\tstatic void vmx_write_guest_host_msr(struct vcpu_vmx vmx, u32 msr, u64 data,\n1472:\t\t\t\t\t u64 cache)\n1473:\t{\n1474:\t\tpreempt_disable();\n1475:\t\tif (vmx-\u003evt.guest_state_loaded)\n1476:\t\t\twrmsrns(msr, data);\n1477:\t\tpreempt_enable();\n1478:\t\tcache = data;\n1479:\t}\n1480:\t\n1481:\tstatic u64 vmx_read_guest_kernel_gs_base(struct vcpu_vmx vmx)\n1482:\t{\n1483:\t\treturn vmx_read_guest_host_msr(vmx, MSR_KERNEL_GS_BASE,\n1484:\t\t\t\t\t \u0026vmx-\u003emsr_guest_kernel_gs_base);\n1485:\t}\n1486:\t\n1487:\tstatic void vmx_write_guest_kernel_gs_base(struct vcpu_vmx *vmx, u64 data)\n1488:\t{\n1489:\t\tvmx_write_guest_host_msr(vmx, MSR_KERNEL_GS_BASE, data,\n" }
61/3	2026/09/11 21:53	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 107739 output: 45 thoughts: 668 Thoughts: My Initial Thoughts on MSR Initialization and Event Handling Okay, so I'm digging into this code, and it looks like `vmx` gets allocated via `kmem_cache_zalloc`. That means `msr_guest_fred_rsp0` should be initialized to zero. But wait a minute, what about `vcpu->arch.fred_ssp0_fallback`? I need to trace this back. Alright, I see it in `arch/x86/kvm/msrs.c`. If the guest doesn't support Shadow Stacks (SHSTK), the code in the MSR handling routines uses `vcpu->arch.fred_ssp0_fallback` for SSP0. And, `vcpu` itself is initialized to zero... so `fred_ssp0_fallback` starts at 0. Makes sense, so far. Now I'm starting to think about event handling, specifically `KVM_GET_VCPU_EVENTS` and `KVM_SET_VCPU_EVENTS`. Let me look at the `struct kvm_vcpu_events` definition. Hmm, this is interesting. I see a `triple_fault` structure, followed by what looks like a fair amount of reserved space. Then there's `exception_has_payload` and `exception_payload`. But... hang on. I need to pay closer attention to the alignment here. It looks like there have been some changes to this structure. Let's dig deeper and figure out the exact layout and sizes of the fields in this struct, paying particular attention to alignment, and see what the implications of the changes are. I need to look at the include file to determine the true layout and what has changed.
62/3	2026/09/11 21:53	tool	read-file	0m Args: { "File": "arch/x86/include/uapi/asm/kvm.h", "FirstLine": 340, "LineCount": 45 } Results: { "Contents": " 340:\t/* for KVM_GET/SET_VCPU_EVENTS /\n 341:\tstruct kvm_vcpu_events {\n 342:\t\tstruct {\n 343:\t\t\t__u8 injected;\n 344:\t\t\t__u8 nr;\n 345:\t\t\t__u8 has_error_code;\n 346:\t\t\t__u8 pending;\n 347:\t\t\t__u32 error_code;\n 348:\t\t} exception;\n 349:\t\tstruct {\n 350:\t\t\t__u8 injected;\n 351:\t\t\t__u8 nr;\n 352:\t\t\t__u8 soft;\n 353:\t\t\t__u8 shadow;\n 354:\t\t} interrupt;\n 355:\t\tstruct {\n 356:\t\t\t__u8 injected;\n 357:\t\t\t__u8 pending;\n 358:\t\t\t__u8 masked;\n 359:\t\t\t__u8 pad;\n 360:\t\t} nmi;\n 361:\t\t__u32 sipi_vector;\n 362:\t\t__u32 flags;\n 363:\t\tstruct {\n 364:\t\t\t__u8 smm;\n 365:\t\t\t__u8 pending;\n 366:\t\t\t__u8 smm_inside_nmi;\n 367:\t\t\t__u8 latched_init;\n 368:\t\t} smi;\n 369:\t\tstruct {\n 370:\t\t\t__u8 pending;\n 371:\t\t} triple_fault;\n 372:\t\t__u8 reserved0[11];\n 373:\t\t/ Aligned to a 64-bit boundary /\n 374:\t\t__u64 exception_event_data;\n 375:\t\t__u8 reserved1[6];\n 376:\t\t__u8 exception_is_nested;\n 377:\t\t__u8 exception_has_payload;\n 378:\t\t__u64 exception_payload;\n 379:\t};\n 380:\t\n 381:\t/ for KVM_GET/SET_DEBUGREGS */\n 382:\tstruct kvm_debugregs {\n 383:\t\t__u64 db[4];\n 384:\t\t__u64 dr6;\n" }
63/3	2026/09/11 21:53	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 109427 output: 22 thoughts: 2253 Thoughts: Analyzing `kvm_vcpu_events` and Associated Structures Okay, let's break down this structure, `kvm_vcpu_events`. I need to ensure there are no surprises with the offsets and how the data is handled. First, I verified the offsets of each member within the `kvm_vcpu_events` structure: `exception`, `interrupt`, `nmi`, `sipi_vector`, `flags`, `smi`, `triple_fault`, `reserved0`, `exception_event_data`, `reserved1`, `exception_is_nested`, `exception_has_payload`, and `exception_payload`. I meticulously checked the byte sizes and offsets to ensure alignment and understand the total size of the structure. I see the total size is 64 bytes, and the structure elements are aligned on 8-byte boundaries. Next, I looked at the before and after states with a specific patch in place. Before the patch, there was a larger `reserved` section that was adjusted to accommodate `exception_event_data`, `reserved1`, `exception_is_nested`, `exception_has_payload` and a new `exception_payload` at the end of the structure. I ensured the structure maintained its 64-byte size, even with the new fields, and that no padding was introduced. Good, no surprises there. I then considered how this structure is used in KVM. I confirmed that the `kvm_vcpu_ioctl_x86_get_vcpu_events` ioctl zeroes the entire 64 bytes before populating it. This is a very good starting point for consistency. I also examined `KVM_SET_VCPU_EVENTS`. The critical part here is that all fields are copied from userspace, which is concerning. The code checks for valid flags when setting events, but any invalid bits passed could still have an effect. This is made safer by clearing `is_nested` and `event_data` if `KVM_VCPUEVENT_VALID_FRED_STATE` is not set. And if the feature is not supported it should return `-EINVAL`. I also wondered: What happens if `copy_from_user` fails? I saw that a non-zero return results in a `-EFAULT`, which is good. I then checked `svm_complete_interrupts`. Nested exceptions and event data are passed as 0. Great, that guarantees no uninitialized data when SVM is used. And I saw that FRED is never enabled on SVM. I then checked `vmx_complete_interrupts` for VMX. I saw that `event_data` is initialized to 0, and read only when FRED is enabled. `is_nested` is properly initialized too. Finally I looked at `vmx_cancel_injection`, which calls `__vmx_complete_interrupts` with 0, which is exactly as expected. I was also checking `dump_vmcs` where `secondary_vmexit_ctl` is initialized or read, which is safe. And the initialization of controls in `setup_vmcs_config` look good to me. I need to confirm all the paths and ensure no uninitialized data is ever read.
64/3	2026/09/11 21:53	tool	grepper	0m Args: { "Expression": "adjust_vmx_controls64" } Results: { "Output": "arch/x86/kvm/vmx/vmx.c=2788=static int adjust_vmx_controls(u32 ctl_min, u32 ctl_opt, u32 msr, u32 result)\n--\narch/x86/kvm/vmx/vmx.c-2805-\narch/x86/kvm/vmx/vmx.c:2806:static u64 adjust_vmx_controls64(u64 ctl_opt, u32 msr)\narch/x86/kvm/vmx/vmx.c-2807-{\n--\narch/x86/kvm/vmx/vmx.c=2843=static int setup_vmcs_config(struct vmcs_config vmcs_conf,\n--\narch/x86/kvm/vmx/vmx.c-2950-\t\t_cpu_based_3rd_exec_control =\narch/x86/kvm/vmx/vmx.c:2951:\t\t\tadjust_vmx_controls64(KVM_OPTIONAL_VMX_TERTIARY_VM_EXEC_CONTROL,\narch/x86/kvm/vmx/vmx.c-2952-\t\t\t\t\t MSR_IA32_VMX_PROCBASED_CTLS3);\n--\narch/x86/kvm/vmx/vmx.c-2979-\t\t_vmexit2_control =\narch/x86/kvm/vmx/vmx.c:2980:\t\t\tadjust_vmx_controls64(KVM_OPTIONAL_VMX_SECONDARY_VM_EXIT_CONTROLS,\narch/x86/kvm/vmx/vmx.c-2981-\t\t\t\t\t MSR_IA32_VMX_EXIT_CTLS2);\n" }
65/3	2026/09/11 21:53	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 112865 output: 18 thoughts: 510 Thoughts: Analyzing Secondary VM-Exit Controls Initialization Okay, let's break this down. I'm looking at how secondary VM-exit controls are handled, particularly their initialization and potential for misuse. I'm focusing on the interaction between a few key functions and data structures. First, `nested_vmx_setup_exit_ctls` catches my eye. It appears we're enabling secondary controls based on the `VM_EXIT_ACTIVATE_SECONDARY_CONTROLS` bit. If that bit is set in `msrs->exit_ctls_high`, we copy `vmcs_conf->vmexit_2nd_ctrl` into `msrs->secondary_exit_ctls`, but then immediately mask it down to only allow `SECONDARY_VM_EXIT_SAVE_IA32_FRED` and `SECONDARY_VM_EXIT_LOAD_IA32_FRED`. Good, that seems like a reasonable restriction. Then, `nested_check_vm_exit_controls` comes into play. It checks if the guest is attempting to set secondary VM-exit controls that are not permitted by the host's `msrs->secondary_exit_ctls`. The `CC()` macro here suggests some sort of compile-time check or assertion – that's something to investigate further, I suppose. It's returning `-EINVAL` if there's a disallowed bit. Makes sense; that's the expected error code. Next, `vmx_get_vmx_msr` is straightforward. It's simply reading out the value of `msrs->secondary_exit_ctls` when the MSR is `MSR_IA32_VMX_EXIT_CTLS2`. Now, the critical question: If `VM_EXIT_ACTIVATE_SECONDARY_CONTROLS` isn't set, then what's the default value of `msrs->secondary_exit_ctls`? Ah, I see: `msrs` is part of `vmcs_config.nested`. And `vmcs_conf` gets zeroed in `setup_vmcs_config` using `memset`. That means `secondary_exit_ctls` starts as zero. Okay, that's important. But wait! What about writes? How are these controls set? I need to check `vmx_set_vmx_msr`. Here it is. It seems to check that the data being written doesn't set any bits that aren't already permitted by `vmcs_config.nested.secondary_exit_ctls`. If there are disallowed bits, it returns `-EINVAL`. This looks like a defense in depth, which is great. It reinforces that whatever is written to the secondary controls must be a subset of the allowed controls. I'm still missing something, though. When is `vmx->nested.msrs` initialized in the first place? I need to track down where the initial values of `msrs` are set, so I'm going to look for `nested_vmx_setup_ctls_msrs` or anywhere else that `vmx->nested.msrs` is initialized. I need to understand the full picture, from the initial zeroing to the permitted modifications. Gotta get a handle on the creation flow and initialization.
66/3	2026/09/11 21:53	tool	grepper	0m Args: { "Expression": "nested\\.msrs" } Results: { "Output": "arch/x86/kvm/vmx/nested.c=421=static void nested_ept_inject_page_fault(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/vmx/nested.c-457-\narch/x86/kvm/vmx/nested.c:458:\t\t\tif (vmx-\u003enested.msrs.ept_caps \u0026 VMX_EPT_ADVANCED_VMEXIT_INFO_BIT)\narch/x86/kvm/vmx/nested.c-459-\t\t\t\tmask \|= EPT_VIOLATION_GVA_USER \|\n--\narch/x86/kvm/vmx/nested.c=504=static void nested_ept_new_eptp(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-506-\tstruct vcpu_vmx vmx = to_vmx(vcpu);\narch/x86/kvm/vmx/nested.c:507:\tbool execonly = vmx-\u003enested.msrs.ept_caps \u0026 VMX_EPT_EXECUTE_ONLY_BIT;\narch/x86/kvm/vmx/nested.c:508:\tint ept_lpage_level = ept_caps_to_lpage_level(vmx-\u003enested.msrs.ept_caps);\narch/x86/kvm/vmx/nested.c-509-\n--\narch/x86/kvm/vmx/nested.c=963=static u32 nested_vmx_max_atomic_switch_msrs(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-965-\tstruct vcpu_vmx vmx = to_vmx(vcpu);\narch/x86/kvm/vmx/nested.c:966:\tu64 vmx_misc = vmx_control_msr(vmx-\u003enested.msrs.misc_low,\narch/x86/kvm/vmx/nested.c:967:\t\t\t\t vmx-\u003enested.msrs.misc_high);\narch/x86/kvm/vmx/nested.c-968-\n--\narch/x86/kvm/vmx/nested.c=1363=static int vmx_restore_vmx_basic(struct vcpu_vmx vmx, u64 data)\n--\narch/x86/kvm/vmx/nested.c-1402-\narch/x86/kvm/vmx/nested.c:1403:\tvmx-\u003enested.msrs.basic = data;\narch/x86/kvm/vmx/nested.c-1404-\treturn 0;\n--\narch/x86/kvm/vmx/nested.c=1437=vmx_restore_control_msr(struct vcpu_vmx vmx, u32 msr_index, u64 data)\n--\narch/x86/kvm/vmx/nested.c-1453-\narch/x86/kvm/vmx/nested.c:1454:\tvmx_get_control_msr(\u0026vmx-\u003enested.msrs, msr_index, \u0026lowp, \u0026highp);\narch/x86/kvm/vmx/nested.c-1455-\tlowp = data;\n--\narch/x86/kvm/vmx/nested.c=1460=static int vmx_restore_vmx_misc(struct vcpu_vmx vmx, u64 data)\n--\narch/x86/kvm/vmx/nested.c-1486-\narch/x86/kvm/vmx/nested.c:1487:\tif ((vmx-\u003enested.msrs.pinbased_ctls_high \u0026\narch/x86/kvm/vmx/nested.c-1488-\t PIN_BASED_VMX_PREEMPTION_TIMER) \u0026\u0026\n--\narch/x86/kvm/vmx/nested.c-1501-\narch/x86/kvm/vmx/nested.c:1502:\tvmx-\u003enested.msrs.misc_low = data;\narch/x86/kvm/vmx/nested.c:1503:\tvmx-\u003enested.msrs.misc_high = data \u003e\u003e 32;\narch/x86/kvm/vmx/nested.c-1504-\n--\narch/x86/kvm/vmx/nested.c=1508=static int vmx_restore_vmx_ept_vpid_cap(struct vcpu_vmx vmx, u64 data)\n--\narch/x86/kvm/vmx/nested.c-1516-\narch/x86/kvm/vmx/nested.c:1517:\tvmx-\u003enested.msrs.ept_caps = data;\narch/x86/kvm/vmx/nested.c:1518:\tvmx-\u003enested.msrs.vpid_caps = data \u003e\u003e 32;\narch/x86/kvm/vmx/nested.c-1519-\treturn 0;\n--\narch/x86/kvm/vmx/nested.c=1534=static int vmx_restore_fixed0_msr(struct vcpu_vmx vmx, u32 msr_index, u64 data)\n--\narch/x86/kvm/vmx/nested.c-1544-\narch/x86/kvm/vmx/nested.c:1545:\tvmx_get_fixed0_msr(\u0026vmx-\u003enested.msrs, msr_index) = data;\narch/x86/kvm/vmx/nested.c-1546-\treturn 0;\n--\narch/x86/kvm/vmx/nested.c=1554=int vmx_set_vmx_msr(struct kvm_vcpu vcpu, u32 msr_index, u64 data)\n--\narch/x86/kvm/vmx/nested.c-1602-\tcase MSR_IA32_VMX_VMCS_ENUM:\narch/x86/kvm/vmx/nested.c:1603:\t\tvmx-\u003enested.msrs.vmcs_enum = data;\narch/x86/kvm/vmx/nested.c-1604-\t\treturn 0;\n--\narch/x86/kvm/vmx/nested.c-1607-\t\t\treturn -EINVAL;\narch/x86/kvm/vmx/nested.c:1608:\t\tvmx-\u003enested.msrs.vmfunc_controls = data;\narch/x86/kvm/vmx/nested.c-1609-\t\treturn 0;\n--\narch/x86/kvm/vmx/nested.c-1612-\t\t\treturn -EINVAL;\narch/x86/kvm/vmx/nested.c:1613:\t\tvmx-\u003enested.msrs.secondary_exit_ctls = data;\narch/x86/kvm/vmx/nested.c-1614-\t\treturn 0;\n--\narch/x86/kvm/vmx/nested.c=2977=static bool nested_vmx_check_eptp(struct kvm_vcpu vcpu, u64 new_eptp)\n--\narch/x86/kvm/vmx/nested.c-2983-\tcase VMX_EPTP_MT_UC:\narch/x86/kvm/vmx/nested.c:2984:\t\tif (CC(!(vmx-\u003enested.msrs.ept_caps \u0026 VMX_EPTP_UC_BIT)))\narch/x86/kvm/vmx/nested.c-2985-\t\t\treturn false;\n--\narch/x86/kvm/vmx/nested.c-2987-\tcase VMX_EPTP_MT_WB:\narch/x86/kvm/vmx/nested.c:2988:\t\tif (CC(!(vmx-\u003enested.msrs.ept_caps \u0026 VMX_EPTP_WB_BIT)))\narch/x86/kvm/vmx/nested.c-2989-\t\t\treturn false;\n--\narch/x86/kvm/vmx/nested.c-2997-\tcase VMX_EPTP_PWL_5:\narch/x86/kvm/vmx/nested.c:2998:\t\tif (CC(!(vmx-\u003enested.msrs.ept_caps \u0026 VMX_EPT_PAGE_WALK_5_BIT)))\narch/x86/kvm/vmx/nested.c-2999-\t\t\treturn false;\n--\narch/x86/kvm/vmx/nested.c-3001-\tcase VMX_EPTP_PWL_4:\narch/x86/kvm/vmx/nested.c:3002:\t\tif (CC(!(vmx-\u003enested.msrs.ept_caps \u0026 VMX_EPT_PAGE_WALK_4_BIT)))\narch/x86/kvm/vmx/nested.c-3003-\t\t\treturn false;\n--\narch/x86/kvm/vmx/nested.c-3014-\tif (new_eptp \u0026 VMX_EPTP_AD_ENABLE_BIT) {\narch/x86/kvm/vmx/nested.c:3015:\t\tif (CC(!(vmx-\u003enested.msrs.ept_caps \u0026 VMX_EPT_AD_BIT)))\narch/x86/kvm/vmx/nested.c-3016-\t\t\treturn false;\n--\narch/x86/kvm/vmx/nested.c=3025=static int nested_check_vm_execution_controls(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/vmx/nested.c-3030-\tif (CC(!vmx_control_verify(vmcs12-\u003epin_based_vm_exec_control,\narch/x86/kvm/vmx/nested.c:3031:\t\t\t\t vmx-\u003enested.msrs.pinbased_ctls_low,\narch/x86/kvm/vmx/nested.c:3032:\t\t\t\t vmx-\u003enested.msrs.pinbased_ctls_high)) \|\|\narch/x86/kvm/vmx/nested.c-3033-\t CC(!vmx_control_verify(vmcs12-\u003ecpu_based_vm_exec_control,\narch/x86/kvm/vmx/nested.c:3034:\t\t\t\t vmx-\u003enested.msrs.procbased_ctls_low,\narch/x86/kvm/vmx/nested.c:3035:\t\t\t\t vmx-\u003enested.msrs.procbased_ctls_high)))\narch/x86/kvm/vmx/nested.c-3036-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c-3039-\t CC(!vmx_control_verify(vmcs12-\u003esecondary_vm_exec_control,\narch/x86/kvm/vmx/nested.c:3040:\t\t\t\t vmx-\u003enested.msrs.secondary_ctls_low,\narch/x86/kvm/vmx/nested.c:3041:\t\t\t\t vmx-\u003enested.msrs.secondary_ctls_high)))\narch/x86/kvm/vmx/nested.c-3042-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c-3067-\t\tif (CC(vmcs12-\u003evm_function_control \u0026\narch/x86/kvm/vmx/nested.c:3068:\t\t ~vmx-\u003enested.msrs.vmfunc_controls))\narch/x86/kvm/vmx/nested.c-3069-\t\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=3088=static int nested_check_vm_exit_controls(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/vmx/nested.c-3093-\tif (CC(!vmx_control_verify(vmcs12-\u003evm_exit_controls,\narch/x86/kvm/vmx/nested.c:3094:\t\t\t\t vmx-\u003enested.msrs.exit_ctls_low,\narch/x86/kvm/vmx/nested.c:3095:\t\t\t\t vmx-\u003enested.msrs.exit_ctls_high)) \|\|\narch/x86/kvm/vmx/nested.c-3096-\t CC(nested_vmx_check_exit_msr_switch_controls(vcpu, vmcs12)))\n--\narch/x86/kvm/vmx/nested.c-3100-\t CC(vmcs12-\u003esecondary_vm_exit_controls \u0026\narch/x86/kvm/vmx/nested.c:3101:\t ~vmx-\u003enested.msrs.secondary_exit_ctls))\narch/x86/kvm/vmx/nested.c-3102-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=3110=static int nested_check_vm_entry_controls(struct kvm_vcpu vcpu,\n--\narch/x86/kvm/vmx/nested.c-3117-\tif (CC(!vmx_control_verify(vmcs12-\u003evm_entry_controls,\narch/x86/kvm/vmx/nested.c:3118:\t\t\t\t vmx-\u003enested.msrs.entry_ctls_low,\narch/x86/kvm/vmx/nested.c:3119:\t\t\t\t vmx-\u003enested.msrs.entry_ctls_high)))\narch/x86/kvm/vmx/nested.c-3120-\t\treturn -EINVAL;\n--\narch/x86/kvm/vmx/nested.c=6191=static int handle_invept(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-6203-\narch/x86/kvm/vmx/nested.c:6204:\tif (!(vmx-\u003enested.msrs.secondary_ctls_high \u0026\narch/x86/kvm/vmx/nested.c-6205-\t SECONDARY_EXEC_ENABLE_EPT) \|\|\narch/x86/kvm/vmx/nested.c:6206:\t !(vmx-\u003enested.msrs.ept_caps \u0026 VMX_EPT_INVEPT_BIT)) {\narch/x86/kvm/vmx/nested.c-6207-\t\tkvm_queue_exception(vcpu, UD_VECTOR);\n--\narch/x86/kvm/vmx/nested.c-6217-\narch/x86/kvm/vmx/nested.c:6218:\ttypes = (vmx-\u003enested.msrs.ept_caps \u003e\u003e VMX_EPT_EXTENT_SHIFT) \u0026 6;\narch/x86/kvm/vmx/nested.c-6219-\n--\narch/x86/kvm/vmx/nested.c=6271=static int handle_invvpid(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/nested.c-6284-\narch/x86/kvm/vmx/nested.c:6285:\tif (!(vmx-\u003enested.msrs.secondary_ctls_high \u0026\narch/x86/kvm/vmx/nested.c-6286-\t SECONDARY_EXEC_ENABLE_VPID) \|\|\narch/x86/kvm/vmx/nested.c:6287:\t\t\t!(vmx-\u003enested.msrs.vpid_caps \u0026 VMX_VPID_INVVPID_BIT)) {\narch/x86/kvm/vmx/nested.c-6288-\t\tkvm_queue_exception(vcpu, UD_VECTOR);\n--\narch/x86/kvm/vmx/nested.c-6298-\narch/x86/kvm/vmx/nested.c:6299:\ttypes = (vmx-\u003enested.msrs.vpid_caps \u0026\narch/x86/kvm/vmx/nested.c-6300-\t\t\tVMX_VPID_EXTENT_SUPPORTED_MASK) \u003e\u003e 8;\n--\narch/x86/kvm/vmx/nested.h=121=static inline unsigned nested_cpu_vmx_misc_cr3_count(struct kvm_vcpu vcpu)\narch/x86/kvm/vmx/nested.h-122-{\narch/x86/kvm/vmx/nested.h:123:\treturn vmx_misc_cr3_count(to_vmx(vcpu)-\u003enested.msrs.misc_low);\narch/x86/kvm/vmx/nested.h-124-}\n--\narch/x86/kvm/vmx/nested.h=131=static inline bool nested_cpu_has_vmwrite_any_field(struct kvm_vcpu vcpu)\narch/x86/kvm/vmx/nested.h-132-{\narch/x86/kvm/vmx/nested.h:133:\treturn to_vmx(vcpu)-\u003enested.msrs.misc_low \u0026\narch/x86/kvm/vmx/nested.h-134-\t\tVMX_MISC_VMWRITE_SHADOW_RO_FIELDS;\n--\narch/x86/kvm/vmx/nested.h=137=static inline bool nested_cpu_has_zero_length_injection(struct kvm_vcpu vcpu)\narch/x86/kvm/vmx/nested.h-138-{\narch/x86/kvm/vmx/nested.h:139:\treturn to_vmx(vcpu)-\u003enested.msrs.misc_low \u0026 VMX_MISC_ZERO_LEN_INS;\narch/x86/kvm/vmx/nested.h-140-}\n--\narch/x86/kvm/vmx/nested.h=142=static inline bool nested_cpu_supports_monitor_trap_flag(struct kvm_vcpu vcpu)\narch/x86/kvm/vmx/nested.h-143-{\narch/x86/kvm/vmx/nested.h:144:\treturn to_vmx(vcpu)-\u003enested.msrs.procbased_ctls_high \u0026\narch/x86/kvm/vmx/nested.h-145-\t\t\tCPU_BASED_MONITOR_TRAP_FLAG;\n--\narch/x86/kvm/vmx/nested.h=148=static inline bool nested_cpu_has_vmx_shadow_vmcs(struct kvm_vcpu vcpu)\narch/x86/kvm/vmx/nested.h-149-{\narch/x86/kvm/vmx/nested.h:150:\treturn to_vmx(vcpu)-\u003enested.msrs.secondary_ctls_high \u0026\narch/x86/kvm/vmx/nested.h-151-\t\tSECONDARY_EXEC_SHADOW_VMCS;\n--\narch/x86/kvm/vmx/nested.h=301=static inline bool nested_guest_cr0_valid(struct kvm_vcpu vcpu, unsigned long val)\narch/x86/kvm/vmx/nested.h-302-{\narch/x86/kvm/vmx/nested.h:303:\tu64 fixed0 = to_vmx(vcpu)-\u003enested.msrs.cr0_fixed0;\narch/x86/kvm/vmx/nested.h:304:\tu64 fixed1 = to_vmx(vcpu)-\u003enested.msrs.cr0_fixed1;\narch/x86/kvm/vmx/nested.h-305-\tstruct vmcs12 vmcs12 = get_vmcs12(vcpu);\narch/x86/kvm/vmx/nested.h-306-\narch/x86/kvm/vmx/nested.h:307:\tif (to_vmx(vcpu)-\u003enested.msrs.secondary_ctls_high \u0026\narch/x86/kvm/vmx/nested.h-308-\t\tSECONDARY_EXEC_UNRESTRICTED_GUEST \u0026\u0026\n--\narch/x86/kvm/vmx/nested.h=315=static inline bool nested_host_cr0_valid(struct kvm_vcpu vcpu, unsigned long val)\narch/x86/kvm/vmx/nested.h-316-{\narch/x86/kvm/vmx/nested.h:317:\tu64 fixed0 = to_vmx(vcpu)-\u003enested.msrs.cr0_fixed0;\narch/x86/kvm/vmx/nested.h:318:\tu64 fixed1 = to_vmx(vcpu)-\u003enested.msrs.cr0_fixed1;\narch/x86/kvm/vmx/nested.h-319-\n--\narch/x86/kvm/vmx/nested.h=323=static inline bool nested_cr4_valid(struct kvm_vcpu vcpu, unsigned long val)\narch/x86/kvm/vmx/nested.h-324-{\narch/x86/kvm/vmx/nested.h:325:\tu64 fixed0 = to_vmx(vcpu)-\u003enested.msrs.cr4_fixed0;\narch/x86/kvm/vmx/nested.h:326:\tu64 fixed1 = to_vmx(vcpu)-\u003enested.msrs.cr4_fixed1;\narch/x86/kvm/vmx/nested.h-327-\n--\narch/x86/kvm/vmx/nested.h=332=static inline bool nested_cpu_has_no_hw_errcode_cc(struct kvm_vcpu vcpu)\narch/x86/kvm/vmx/nested.h-333-{\narch/x86/kvm/vmx/nested.h:334:\treturn to_vmx(vcpu)-\u003enested.msrs.basic \u0026 VMX_BASIC_NO_HW_ERROR_CODE_CC;\narch/x86/kvm/vmx/nested.h-335-}\n--\narch/x86/kvm/vmx/nested.h=337=static inline bool nested_cpu_has_nested_exception(struct kvm_vcpu vcpu)\narch/x86/kvm/vmx/nested.h-338-{\narch/x86/kvm/vmx/nested.h:339:\treturn to_vmx(vcpu)-\u003enested.msrs.basic \u0026 VMX_BASIC_NESTED_EXCEPTION;\narch/x86/kvm/vmx/nested.h-340-}\n--\narch/x86/kvm/vmx/vmx.c=2213=int vmx_get_msr(struct kvm_vcpu vcpu, struct msr_data msr_info)\n--\narch/x86/kvm/vmx/vmx.c-2292-\t\t\treturn 1;\narch/x86/kvm/vmx/vmx.c:2293:\t\tif (vmx_get_vmx_msr(\u0026vmx-\u003enested.msrs, msr_info-\u003eindex,\narch/x86/kvm/vmx/vmx.c-2294-\t\t\t\t \u0026msr_info-\u003edata))\n--\narch/x86/kvm/vmx/vmx.c=2425=int vmx_set_msr(struct kvm_vcpu vcpu, struct msr_data msr_info)\n--\narch/x86/kvm/vmx/vmx.c-2519-\t\tif (is_guest_mode(vcpu) \u0026\u0026\narch/x86/kvm/vmx/vmx.c:2520:\t\t ((vmx-\u003enested.msrs.entry_ctls_high \u0026 VM_ENTRY_LOAD_BNDCFGS) \|\|\narch/x86/kvm/vmx/vmx.c:2521:\t\t (vmx-\u003enested.msrs.exit_ctls_high \u0026 VM_EXIT_CLEAR_BNDCFGS)))\narch/x86/kvm/vmx/vmx.c-2522-\t\t\tget_vmcs12(vcpu)-\u003eguest_bndcfgs = data;\n--\narch/x86/kvm/vmx/vmx.c=4901=vmx_adjust_secondary_exec_control(struct vcpu_vmx vmx, u32 exec_control,\n--\narch/x86/kvm/vmx/vmx.c-4928-\t\tif (enabled)\narch/x86/kvm/vmx/vmx.c:4929:\t\t\tvmx-\u003enested.msrs.secondary_ctls_high \|= control;\narch/x86/kvm/vmx/vmx.c-4930-\t\telse\narch/x86/kvm/vmx/vmx.c:4931:\t\t\tvmx-\u003enested.msrs.secondary_ctls_high \u0026= ~control;\narch/x86/kvm/vmx/vmx.c-4932-\t}\n--\narch/x86/kvm/vmx/vmx.c=5220=static void __vmx_vcpu_reset(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/vmx.c-5227-\t kvm_check_has_quirk(vcpu-\u003ekvm, KVM_X86_QUIRK_STUFF_FEATURE_MSRS))\narch/x86/kvm/vmx/vmx.c:5228:\t\tmemcpy(\u0026vmx-\u003enested.msrs, \u0026vmcs_config.nested, sizeof(vmx-\u003enested.msrs));\narch/x86/kvm/vmx/vmx.c-5229-\n--\narch/x86/kvm/vmx/vmx.c=8112=static void nested_vmx_cr_fixed1_bits_update(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/vmx/vmx.c-8116-\narch/x86/kvm/vmx/vmx.c:8117:\tvmx-\u003enested.msrs.cr0_fixed1 = 0xffffffff;\narch/x86/kvm/vmx/vmx.c:8118:\tvmx-\u003enested.msrs.cr4_fixed1 = X86_CR4_PCE;\narch/x86/kvm/vmx/vmx.c-8119-\n--\narch/x86/kvm/vmx/vmx.c-8121-\tif (entry \u0026\u0026 (entry-\u003e_reg \u0026 (_cpuid_mask)))\t\t\t\\\narch/x86/kvm/vmx/vmx.c:8122:\t\tvmx-\u003enested.msrs.cr4_fixed1 \|= (_cr4_mask);\t\\\narch/x86/kvm/vmx/vmx.c-8123-} while (0)\n" }
67/3	2026/09/11 21:53	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 121171 output: 30 thoughts: 2245 Thoughts: Analyzing Potential FRED-Related Vulnerabilities in KVM Okay, let's break this down systematically. Line 5228, the `memcpy` looks fine; nested MSRs are being properly initialized. That's a good starting point. Next, I need to check how `prepare_vmcs02_early` deals with secondary controls and the `INJECTED_EVENT_DATA`. Right. I see that `vmcs12->injected_event_data` originates from either a VMWRITE or VMPTRLD from the guest memory. Guest memory is inherently untrusted, so is there any possibility that uninitialized kernel memory is ever exposed? Nope, it's written to hardware VMCS. Good. Now, checking `sync_vmcs02_to_vmcs12`, it seems FRED MSRs are read using `vmcs_read_fred_msrs`, which itself reads from the VMCS. All looks clean in `vmcs_read_fred_msrs`. All fields of struct `vmcs_fred_msrs` are written. Next is `prepare_vmcs12`: It deals with the `VECTORING_INFO_VALID_MASK` and the CR4 FRED bit. If nested exception injection is enabled, and if IA32e mode is enabled, it sets `vm_exit_intr_info` with `INTR_INFO_NESTED_EXCEPTION_MASK`. Then, let's look at `vmcs12_save_pending_event`. It looks like `original_event_data` is initialized to zero, but it's assigned to `vcpu->arch.exception.event_data`, which is from guest. If nested exception injection enabled then `VECTORING_INFO_NESTED_EXCEPTION_MASK` is also set. Then `vmx_inject_exception` has different behavior based on whether the exception is soft or hard and whether FRED is enabled. If FRED is enabled, `INJECTED_EVENT_DATA` is set to zero if the exception isn't one of the known exceptions. It seems okay. Now, `mk_cr4_64`, in `arch/x86/kvm/x86.c`. This is about building CR4 values. The function `mk_cr4_64` has a shift based on whether X86_FEATURE_FRED is enabled. If FRED is supported, the MSB is set. `curr_cr` and `new_val` are coming from `kvm_read_cr4(vcpu)` and `val`. Finally, the MSRs. `msrs_to_save_base[]` includes FRED-related MSRs: RSP0 through CONFIG. Userspace can probe them using `KVM_GET_MSR_INDEX_LIST`. If supported by the CPU, userspace can use `KVM_GET_MSRS` to get the values. `kvm_probe_msr_to_save` then filters these. If FRED isn't supported, those specific MSRs are skipped. For MSRs that aren't FRED related, they will return zero. In `KVM_GET_MSRS`, for RSP0, `vmx_read_guest_fred_rsp0()` returns `vmx->msr_guest_fred_rsp0`, which is initialized to zero. For RSP1 and through CONFIG, `vmcs_read64()` is used to read values from the VMCS, and VMCS has them initialized to zero. And PL0_SSP (i.e. SSP0) it's reading the value from `vcpu->arch.fred_ssp0_fallback`, which is also initialized to 0. It's safe.
68/3	2026/09/11 21:53	tool	grepper	0m Error: no matches Args: { "Expression": "kvm_vcpu_ioctl_x86_get_msrs" } Results: { "error": "no matches" }
69/3	2026/09/11 21:53	llm	kmsan-evaluator-compressor	0m Model: gemini-3.8-flash Args: null Results: null Reply: ### Summary of Investigation #### 1. Goal and Core Distinction - Objective: Determine whether the patch introducing Intel FRED (Flexible Return and Event Delivery) support in KVM for x86/amd64 warrants a dedicated KMSAN fuzzing session (`NeedsKMSAN=true` vs `false`). - Guiding principle: KMSAN is strictly required only if the changes introduce or modify uninitialized memory use or kernel-to-userspace memory leaks (uninitialized fields/padding) not covered by KASAN/LOCKDEP. #### 2. Key Code Changes in the Patch Series - Documentation/virt/kvm/api.rst & include/uapi/linux/kvm.h: Added `KVM_CAP_X86_FRED_EVENT` (cap 251) and `KVM_VCPUEVENT_VALID_FRED_STATE` (flag `0x40`). - arch/x86/include/uapi/asm/kvm.h: In `struct kvm_vcpu_events`, replaced 26 reserved bytes with: ```c __u8 reserved0[11]; __u64 exception_event_data; __u8 reserved1[6]; __u8 exception_is_nested; __u8 exception_has_payload; __u64 exception_payload; ``` - arch/x86/kvm/x86.c: - `kvm_vcpu_ioctl_x86_get_vcpu_events()`: copies `is_nested` and `event_data` to `events->exception_is_nested` and `events->exception_event_data`. Sets `KVM_VCPUEVENT_VALID_FRED_STATE` in `events->flags` if `exception_fred_state_enabled`. - `kvm_vcpu_ioctl_x86_set_vcpu_events()`: sets `vcpu->arch.exception.is_nested` and `event_data` from user input; validates `KVM_VCPUEVENT_VALID_FRED_STATE`. - `kvm_multiple_exception()`, `kvm_requeue_exception()`, `kvm_deliver_exception_payload()`: initializes and tracks `is_nested` and `event_data`. - CR4 manipulation helpers: `mk_cr4_64()` allowing bit 32 (X86_CR4_FRED). - Nested VMX (arch/x86/kvm/vmx/nested.c, nested.h, vmcs12.h, vmcs12.c): - Added FRED MSR fields (`guest_fred_msrs`, `host_fred_msrs`) and event data (`injected_event_data`, `original_event_data`) to `struct vmcs12`. - Added nested controls: `secondary_vm_exit_controls` handling `SECONDARY_VM_EXIT_SAVE_IA32_FRED` and `SECONDARY_VM_EXIT_LOAD_IA32_FRED`. - Added tracking fields to `struct nested_vmx`: `pre_vmenter_fred_msrs` and `at_vmexit_fred_msrs`. - Nested VMCS read/writes: `vmcs_read_fred_msrs()` and `vmcs_write_fred_msrs()`. - CPU Entry Area & Traps (`arch/x86/include/asm/cpu_entry_area.h`, `arch/x86/mm/cpu_entry_area.c`, etc.): - Refactored `cea_exception_stacks` into an array of `event_stacks[N_EXCEPTION_STACKS]` with guard pages. - Implemented `__this_cpu_ist_bottom_va()` and `__this_cpu_ist_top_va()`. - Added `this_cpu_fred_rsp()` in `arch/x86/kernel/fred.c`. #### 3. Approaches Tried and Findings ##### A. Checking User-Space Memory Leak via `KVM_GET_VCPU_EVENTS` - Location: `arch/x86/kvm/x86.c:3628` (`case KVM_GET_VCPU_EVENTS`). - Trace: - `struct kvm_vcpu_events events;` allocated on stack in `kvm_arch_vcpu_ioctl()`. - Passed to `kvm_vcpu_ioctl_x86_get_vcpu_events(vcpu, &events)`. - In `kvm_vcpu_ioctl_x86_get_vcpu_events()`, line 3008: ```c memset(events, 0, sizeof(events)); ``` The entire struct (including `reserved0`, `reserved1`, and all padding) is unconditionally zero-initialized before any fields are populated. - Furthermore, `vcpu->run->s.regs.events` in `store_regs()` is similarly passed to `kvm_vcpu_ioctl_x86_get_vcpu_events()` which memsets it first. - Verdict: No uninitialized data or padding can leak to userspace via `KVM_GET_VCPU_EVENTS` or sync regs. ##### B. Initialization and Lifecycle of `vcpu->arch.exception` (`is_nested` and `event_data`) - Structure: `struct kvm_queued_exception` in `kvm_host.h`. - Allocation: - `vcpu` allocated in `virt/kvm/kvm_main.c:4195` via `kmem_cache_zalloc(kvm_vcpu_cache, GFP_KERNEL_ACCOUNT)`. - Zero-initialized at creation time. - State Transitions: - `kvm_clear_exception_queue(vcpu)` explicitly sets `is_nested = false; event_data = 0;`. - `kvm_requeue_exception()` explicitly sets `vcpu->arch.exception.is_nested = is_nested; vcpu->arch.exception.event_data = event_data;`. - `kvm_multiple_exception()` explicitly assigns `is_nested` and does not leak uninitialized values. - In `kvm_arch_vcpu_ioctl_run()`, `ex->is_nested = false;` is set. - Verdict: All fields in `kvm_queued_exception` are deterministically initialized. ##### C. Nested VMX (`struct vmcs12`) and `KVM_GET_NESTED_STATE` - Location: `arch/x86/kvm/vmx/nested.c:7066` (`copy_to_user(user_vmx_nested_state->vmcs12, vmcs12, VMCS12_SIZE)`). - Allocation: - `cached_vmcs12` and `cached_shadow_vmcs12` are allocated via `kzalloc(VMCS12_SIZE, GFP_KERNEL_ACCOUNT)` in `enter_vmx_operation()` (`nested.c:5631`). - Read from guest memory via `kvm_read_guest_cached()`. - `struct __packed vmcs12` is defined with `__packed` and explicit padding (`padding16[1]`). - Fields added to `vmcs12`: `guest_fred_msrs`, `host_fred_msrs`, `injected_event_data`, `original_event_data`, `secondary_vm_exit_controls`. - When VM exit occurs (`__nested_vmx_vmexit()`): - `sync_vmcs02_to_vmcs12()` is called if `!vmx->fail`. - If FRED enabled, `vmcs_read_fred_msrs(&vmx->nested.at_vmexit_fred_msrs)` reads from hardware/VMCS02, and `memcpy(&vmcs12->guest_fred_msrs, ...)` copies it if configured. - If `vmx->fail`, `load_vmcs12_host_state()` is skipped in `__nested_vmx_vmexit()`. - `load_vmcs12_host_state(..., false)` only accesses `at_vmexit_fred_msrs` when `!from_failed_vmentry` and `sync_vmcs02_to_vmcs12()` has already populated `at_vmexit_fred_msrs`. - Since `vmcs12` originates from `kzalloc`, copying `VMCS12_SIZE` to userspace via `copy_to_user` does not expose uninitialized kernel stack or heap memory. ##### D. CPU Entry Area & IST Stack Changes - Replaces macros with inlined/noinstr accessor functions `__this_cpu_ist_bottom_va()` and `__this_cpu_ist_top_va()`. - Static per-cpu declarations and page mappings (`cea_map_percpu_pages`). - No new dynamic uninitialized heap or stack allocations introduced. #### 4. Current Hypotheses and Conclusion - Hypothesis: All paths touching user-facing structures (`kvm_vcpu_events`, `vmcs12`) either explicitly zero memory via `memset()` or allocate via `kzalloc()`. - Sanitizer Suitability: - KASAN / existing sanitizers: Address checks, canonical address checking, bounds checking on MSR indexes, and VMCS structures are all standard logic testable by KASAN. - KMSAN: No new uninitialized memory usage, uninitialized branching, or kernel-to-user information leaks are introduced. - Verdict to return: `NeedsKMSAN = false`. Tokens:* input: 103137 output: 2240 thoughts: 0
70/3	2026/09/11 21:53	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 71569 output: 21 thoughts: 0
71/3	2026/09/11 21:53	tool	grepper	0m Args: { "Expression": "KVM_GET_MSRS" } Results: { "Output": "Documentation/virt/kvm/api.rst=253=KVM_GET_MSR_FEATURE_INDEX_LIST returns the list of MSRs that can be passed\nDocumentation/virt/kvm/api.rst:254:to the KVM_GET_MSRS system ioctl. This lets userspace probe host capabilities\nDocumentation/virt/kvm/api.rst-255-and processor features that are exposed via MSRs (e.g., VMX capabilities).\n--\nDocumentation/virt/kvm/api.rst=640=This is an asynchronous vcpu ioctl and can be invoked from any thread.\n--\nDocumentation/virt/kvm/api.rst-642-\nDocumentation/virt/kvm/api.rst:643:4.18 KVM_GET_MSRS\nDocumentation/virt/kvm/api.rst-644------------------\n--\nDocumentation/virt/kvm/api.rst=680=kvm will fill in the 'data' member.\n--\nDocumentation/virt/kvm/api.rst-691-\nDocumentation/virt/kvm/api.rst:692:Writes model-specific registers to the vcpu. See KVM_GET_MSRS for the\nDocumentation/virt/kvm/api.rst-693-data structures.\n--\narch/x86/include/uapi/asm/kvm.h=189=struct kvm_msr_entry {\n--\narch/x86/include/uapi/asm/kvm.h-194-\narch/x86/include/uapi/asm/kvm.h:195:/* for KVM_GET_MSRS and KVM_SET_MSRS /\narch/x86/include/uapi/asm/kvm.h-196-struct kvm_msrs {\n--\narch/x86/kvm/msrs.c=172=void drop_user_return_notifiers(void)\n--\narch/x86/kvm/msrs.c-181- The three MSR lists(msrs_to_save, emulated_msrs, msr_based_features) track\narch/x86/kvm/msrs.c:182: * the set of MSRs that KVM exposes to userspace through KVM_GET_MSRS,\narch/x86/kvm/msrs.c-183- * KVM_SET_MSRS, and KVM_GET_MSR_INDEX_LIST. msrs_to_save holds MSRs that\n--\narch/x86/kvm/x86.c=2488=long kvm_arch_dev_ioctl(struct file filp,\n--\narch/x86/kvm/x86.c-2527-\t\tbreak;\narch/x86/kvm/x86.c:2528:\tcase KVM_GET_MSRS:\narch/x86/kvm/x86.c-2529-\t\tr = kvm_get_feature_msrs(argp);\n--\narch/x86/kvm/x86.c=3463=long kvm_arch_vcpu_ioctl(struct file filp,\n--\narch/x86/kvm/x86.c-3566-\t}\narch/x86/kvm/x86.c:3567:\tcase KVM_GET_MSRS:\narch/x86/kvm/x86.c-3568-\t\tr = kvm_get_msrs(vcpu, argp);\n--\ninclude/uapi/linux/kvm.h=1254=struct kvm_s390_keyop {\n--\ninclude/uapi/linux/kvm.h-1364-#define KVM_INTERRUPT _IOW(KVMIO, 0x86, struct kvm_interrupt)\ninclude/uapi/linux/kvm.h:1365:#define KVM_GET_MSRS _IOWR(KVMIO, 0x88, struct kvm_msrs)\ninclude/uapi/linux/kvm.h-1366-#define KVM_SET_MSRS _IOW(KVMIO, 0x89, struct kvm_msrs)\n--\ntools/arch/x86/include/uapi/asm/kvm.h=189=struct kvm_msr_entry {\n--\ntools/arch/x86/include/uapi/asm/kvm.h-194-\ntools/arch/x86/include/uapi/asm/kvm.h:195:/* for KVM_GET_MSRS and KVM_SET_MSRS /\ntools/arch/x86/include/uapi/asm/kvm.h-196-struct kvm_msrs {\n--\ntools/include/uapi/linux/kvm.h=1252=struct kvm_s390_keyop {\n--\ntools/include/uapi/linux/kvm.h-1362-#define KVM_INTERRUPT _IOW(KVMIO, 0x86, struct kvm_interrupt)\ntools/include/uapi/linux/kvm.h:1363:#define KVM_GET_MSRS _IOWR(KVMIO, 0x88, struct kvm_msrs)\ntools/include/uapi/linux/kvm.h-1364-#define KVM_SET_MSRS _IOW(KVMIO, 0x89, struct kvm_msrs)\n--\ntools/testing/selftests/kvm/include/x86/processor.h=973=static inline void vcpu_msrs_get(struct kvm_vcpu vcpu,\n--\ntools/testing/selftests/kvm/include/x86/processor.h-975-{\ntools/testing/selftests/kvm/include/x86/processor.h:976:\tint r = __vcpu_ioctl(vcpu, KVM_GET_MSRS, msrs);\ntools/testing/selftests/kvm/include/x86/processor.h-977-\ntools/testing/selftests/kvm/include/x86/processor.h-978-\tTEST_ASSERT(r == msrs-\u003enmsrs,\ntools/testing/selftests/kvm/include/x86/processor.h:979:\t\t \"KVM_GET_MSRS failed, r: %i (failed on MSR %x)\",\ntools/testing/selftests/kvm/include/x86/processor.h-980-\t\t r, r \u003c 0 \|\| r \u003e= msrs-\u003enmsrs ? -1 : msrs-\u003eentries[r].index);\n--\ntools/testing/selftests/kvm/lib/x86/processor.c=967=u64 kvm_get_feature_msr(u64 msr_index)\n--\ntools/testing/selftests/kvm/lib/x86/processor.c-978-\ntools/testing/selftests/kvm/lib/x86/processor.c:979:\tr = __kvm_ioctl(kvm_fd, KVM_GET_MSRS, \u0026buffer.header);\ntools/testing/selftests/kvm/lib/x86/processor.c:980:\tTEST_ASSERT(r == 1, KVM_IOCTL_ERROR(KVM_GET_MSRS, r));\ntools/testing/selftests/kvm/lib/x86/processor.c-981-\n" }
72/3	2026/09/11 21:53	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 73494 output: 43 thoughts: 30 Thoughts: Here's my take on the thought process, as if it were my own: Investigating MSR Handling in KVM for x86 Alright, time to dive into this. Let's start by pinpointing the function responsible for retrieving Model Specific Registers (MSRs) within the KVM framework, specifically for the x86 architecture. My first instinct is to check `kvm_get_msrs`. This feels like the most likely candidate. Now, where would that function reside? Given this is x86-specific, the obvious place to begin the search is within the `arch/x86/kvm/` directory, and likely the `x86.c` file since it sounds like a core function. Let's confirm my hunch. I'll head over there and take a look. I'm expecting to find the implementation details for the `kvm_get_msrs` function, the one I hope is responsible for returning the content of a requested MSR, or MSRs, to the KVM host. I need to understand its behavior and how it relates to my current problem. I'm especially interested in how it handles permissions and any potential error conditions.
73/3	2026/09/11 21:53	tool	read-file	0m Args: { "File": "arch/x86/kvm/x86.c", "FirstLine": 4615, "LineCount": 40 } Results: { "Contents": "4615:\t\t\tr = -EFAULT;\n4616:\t\t\tif (copy_to_user(argp, \u0026u.ps2, sizeof(u.ps2)))\n4617:\t\t\t\tgoto out;\n4618:\t\t\tr = 0;\n4619:\t\t\tbreak;\n4620:\t\t}\n4621:\t\tcase KVM_SET_PIT2: {\n4622:\t\t\tr = -EFAULT;\n4623:\t\t\tif (copy_from_user(\u0026u.ps2, argp, sizeof(u.ps2)))\n4624:\t\t\t\tgoto out;\n4625:\t\t\tmutex_lock(\u0026kvm-\u003elock);\n4626:\t\t\tr = -ENXIO;\n4627:\t\t\tif (!kvm-\u003earch.vpit)\n4628:\t\t\t\tgoto set_pit2_out;\n4629:\t\t\tr = kvm_vm_ioctl_set_pit2(kvm, \u0026u.ps2);\n4630:\tset_pit2_out:\n4631:\t\t\tmutex_unlock(\u0026kvm-\u003elock);\n4632:\t\t\tbreak;\n4633:\t\t}\n4634:\t\tcase KVM_REINJECT_CONTROL: {\n4635:\t\t\tstruct kvm_reinject_control control;\n4636:\t\t\tr = -EFAULT;\n4637:\t\t\tif (copy_from_user(\u0026control, argp, sizeof(control)))\n4638:\t\t\t\tgoto out;\n4639:\t\t\tr = -ENXIO;\n4640:\t\t\tif (!kvm-\u003earch.vpit)\n4641:\t\t\t\tgoto out;\n4642:\t\t\tr = kvm_vm_ioctl_reinject(kvm, \u0026control);\n4643:\t\t\tbreak;\n4644:\t\t}\n4645:\t#endif\n4646:\t\tcase KVM_SET_BOOT_CPU_ID:\n4647:\t\t\tr = 0;\n4648:\t\t\tmutex_lock(\u0026kvm-\u003elock);\n4649:\t\t\tif (kvm-\u003ecreated_vcpus)\n4650:\t\t\t\tr = -EBUSY;\n4651:\t\t\telse if (arg \u003e KVM_MAX_VCPU_IDS \|\|\n4652:\t\t\t\t (kvm-\u003earch.max_vcpu_ids \u0026\u0026 arg \u003e kvm-\u003earch.max_vcpu_ids))\n4653:\t\t\t\tr = -EINVAL;\n4654:\t\t\telse\n" }
74/3	2026/09/11 21:53	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 74459 output: 23 thoughts: 20 Thoughts: Here's my attempt at summarizing the thought process, tailored for an expert audience: Investigating `kvm_get_msrs` Okay, so I need to refresh my memory on what `kvm_get_msrs` actually does. It's been a while since I dug into that part of the KVM code. Let's see...I'm thinking the quickest way to get a solid definition, especially with the likely current version, is to use `grepper`. I can quickly search the kernel sources and related documentation that way. I'm expecting to find the function definition, but ideally, I want more – ideally some context from the surrounding comments. I'm hoping to get a clearer sense of the intended purpose and how it's used within the KVM infrastructure. I'll need to remember to look for its relationship to other MSR related functions and how it interacts with the VM's state. Grepping is probably the fastest way to understand the immediate function details and usage.
75/3	2026/09/11 21:53	tool	grepper	0m Args: { "Expression": "int kvm_get_msrs\\(" } Results: { "Output": "arch/x86/kvm/msrs.c=2374=int kvm_get_feature_msrs(struct kvm_msrs __user user_msrs)\n--\narch/x86/kvm/msrs.c-2378-\narch/x86/kvm/msrs.c:2379:int kvm_get_msrs(struct kvm_vcpu vcpu, struct kvm_msrs __user user_msrs)\narch/x86/kvm/msrs.c-2380-{\n--\narch/x86/kvm/msrs.h=51=int kvm_get_feature_msrs(struct kvm_msrs __user user_msrs);\narch/x86/kvm/msrs.h-52-\narch/x86/kvm/msrs.h:53:int kvm_get_msrs(struct kvm_vcpu vcpu, struct kvm_msrs __user user_msrs);\narch/x86/kvm/msrs.h-54-int kvm_set_msrs(struct kvm_vcpu vcpu, struct kvm_msrs __user user_msrs);\n" }
76/3	2026/09/11 21:53	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 75034 output: 42 thoughts: 29 Thoughts: Here's my summary, as if I were thinking these thoughts: Diving into KVM MSR Handling Okay, time to dig into the heart of how KVM handles Model Specific Registers, or MSRs, on x86. I'm focusing on the `kvm_get_msrs` function, specifically in `arch/x86/kvm/msrs.c`. I need to understand exactly how the guest's requests for MSR data are processed and fulfilled. I expect this function will be responsible for fetching the values of specific MSRs from the guest's virtual CPU context. It's probably interacting with the hardware through the hypervisor's MMU to gain access. I'll be looking for how it handles different MSR types, any potential optimizations, and any potential security implications related to the data transfer. I want to see if this function is a key bottleneck in performance or a security vulnerability. Time to get my hands dirty.
77/3	2026/09/11 21:54	tool	read-file	0m Args: { "File": "arch/x86/kvm/msrs.c", "FirstLine": 2378, "LineCount": 40 } Results: { "Contents": "2378:\t\n2379:\tint kvm_get_msrs(struct kvm_vcpu vcpu, struct kvm_msrs __user user_msrs)\n2380:\t{\n2381:\t\tguard(srcu)(\u0026vcpu-\u003ekvm-\u003esrcu);\n2382:\t\n2383:\t\treturn msr_io(vcpu, user_msrs, do_get_msr, 1);\n2384:\t}\n2385:\t\n2386:\tint kvm_set_msrs(struct kvm_vcpu vcpu, struct kvm_msrs __user user_msrs)\n2387:\t{\n2388:\t\tguard(srcu)(\u0026vcpu-\u003ekvm-\u003esrcu);\n2389:\t\n2390:\t\treturn msr_io(vcpu, user_msrs, do_set_msr, 0);\n2391:\t}\n2392:\t\n2393:\tstatic int kvm_get_one_msr(struct kvm_vcpu vcpu, u32 msr, u64 __user user_val)\n2394:\t{\n2395:\t\tu64 val;\n2396:\t\n2397:\t\tif (do_get_msr(vcpu, msr, \u0026val))\n2398:\t\t\treturn -EINVAL;\n2399:\t\n2400:\t\tif (put_user(val, user_val))\n2401:\t\t\treturn -EFAULT;\n2402:\t\n2403:\t\treturn 0;\n2404:\t}\n2405:\t\n2406:\tstatic int kvm_set_one_msr(struct kvm_vcpu vcpu, u32 msr, u64 __user user_val)\n2407:\t{\n2408:\t\tu64 val;\n2409:\t\n2410:\t\tif (get_user(val, user_val))\n2411:\t\t\treturn -EFAULT;\n2412:\t\n2413:\t\tif (do_set_msr(vcpu, msr, \u0026val))\n2414:\t\t\treturn -EINVAL;\n2415:\t\n2416:\t\treturn 0;\n2417:\t}\n" }
78/3	2026/09/11 21:54	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 75929 output: 45 thoughts: 1318 Thoughts: Analyzing MSR and Event Structures in KVM/VMX Alright, let's break this down systematically. I'm focusing on the data flow and initialization of specific MSRs and the layout of the `kvm_vcpu_events` structure. First, I'm tracing the journey of data used in `msr_io`, specifically regarding FRED MSRs. It's clear `do_get_msr` is used to get the data, and that data is then sent to userspace via `put_user`. I need to figure out the source for that data. The trace begins in `vmx_get_msr`. For the FRED MSRs, the data appears to originate from a couple of places. `MSR_IA32_FRED_RSP0` gets its data via a function call, `vmx_read_guest_fred_rsp0`. The others, `MSR_IA32_FRED_RSP1` through `MSR_IA32_FRED_CONFIG` come from `vmcs_read64`. So, I must now see how these get initialized in the VMCS. I'm now looking at the `init_vmcs` function. There I can see, if the CPU has FRED support enabled, these registers are explicitly initialized to zero within the VMCS. So, `GUEST_IA32_FRED_CONFIG`, `GUEST_IA32_FRED_RSP1`, and so on, get a clean start. Also, I see `vmx->msr_guest_fred_rsp0` which is a member of the `struct vcpu_vmx`, and that is allocated via `kmem_cache_zalloc` which means they will be zeroed initially. The same goes for `vcpu->arch.fred_ssp0_fallback`. So, the FRED MSRs are properly initialized in the VMCS and/or the VCPU structure. Now, let's shift to `MSR_IA32_VMX_EXIT_CTLS2`. I can trace it via `vmx_get_vmx_msr`. This MSR gets its value from `msrs->secondary_exit_ctls`, which is set by `nested_vmx_setup_exit_ctls`. That in turn checks the `VM_EXIT_ACTIVATE_SECONDARY_CONTROLS` bit. If it's set, it copies data from `vmcs_conf->vmexit_2nd_ctrl` and applies a mask. The value is then copied into `vmx->nested.msrs` during `__vmx_vcpu_reset()` and initialized in `setup_vmcs_config`. I can see how `_vmexit2_control` is set and passed to `vmcs_conf->vmexit_2nd_ctrl`. The initial checks here are ensuring the register gets an appropriate value. So, initialization is correct. Okay, let's change gears. I am now looking at the `struct kvm_vcpu_events`. Specifically, I'm checking the layout and padding. I want to be certain that there's no subtle misalignment, especially around potentially sensitive fields. The first issue is the `triple_fault` struct. I see a `pending` byte. The following variables are: `reserved0`, `exception_event_data`, `reserved1`, `exception_is_nested`, `exception_has_payload`, and `exception_payload`. This raises a question: Is there padding around the triple fault struct? I need to check the offsets. I see that the `reserved` was 26 bytes, and then `exception_has_payload` which means implicit padding to an 8-byte boundary! Since `exception_payload` is 8 bytes, this explains the padding! No, wait...Previously, there was 26 bytes for reserved, and then a 1 byte field, but there was 4 bytes of implicit padding between `exception_has_payload` and `exception_payload`. I need to verify all the offsets to be absolutely certain.
79/3	2026/09/11 21:54	tool	read-file	0m Args: { "File": "arch/x86/include/uapi/asm/kvm.h", "FirstLine": 360, "LineCount": 25 } Results: { "Contents": " 360:\t\t} nmi;\n 361:\t\t__u32 sipi_vector;\n 362:\t\t__u32 flags;\n 363:\t\tstruct {\n 364:\t\t\t__u8 smm;\n 365:\t\t\t__u8 pending;\n 366:\t\t\t__u8 smm_inside_nmi;\n 367:\t\t\t__u8 latched_init;\n 368:\t\t} smi;\n 369:\t\tstruct {\n 370:\t\t\t__u8 pending;\n 371:\t\t} triple_fault;\n 372:\t\t__u8 reserved0[11];\n 373:\t\t/* Aligned to a 64-bit boundary /\n 374:\t\t__u64 exception_event_data;\n 375:\t\t__u8 reserved1[6];\n 376:\t\t__u8 exception_is_nested;\n 377:\t\t__u8 exception_has_payload;\n 378:\t\t__u64 exception_payload;\n 379:\t};\n 380:\t\n 381:\t/ for KVM_GET/SET_DEBUGREGS */\n 382:\tstruct kvm_debugregs {\n 383:\t\t__u64 db[4];\n 384:\t\t__u64 dr6;\n" }
80/3	2026/09/11 21:54	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 78528 output: 22 thoughts: 732 Thoughts: Analyzing the Layout and Safety of `struct kvm_vcpu_events` Okay, let's break down the memory layout of `struct kvm_vcpu_events`. I'm starting by meticulously calculating the offsets of each member. First, we have the `exception` struct with its nested members: `injected`, `nr`, `has_error_code`, `pending` (each 1 byte), and `error_code` (4 bytes). That gives us a total of 8 bytes for `exception`, occupying offsets 0 through 7. Then we have `interrupt`, `nmi` and `smi` and their 4 `__u8` members each, giving 4 bytes, spanning offsets 8-11, 12-15 and 24-27 respectively. `sipi_vector` and `flags`, both `__u32` are 4 bytes each, taking offsets 16-19 and 20-23. `triple_fault` takes 1 byte, offset 28 and `reserved0` takes 11 bytes, offset 29-39. This gives us 28 + 1 + 11 = 40! So `exception_event_data` starts at offset 40, a multiple of 8, so it's perfectly aligned (good!). This will then give an offset of 40-47 (8 bytes). Moving on, `reserved1` takes 6 bytes (48-53), followed by `exception_is_nested` and `exception_has_payload`, each 1 byte (54 and 55, respectively). Now, 54 + 1 + 1 = 56! That's also a multiple of 8, which means the next member, `exception_payload` (`__u64`), can start at offset 56 and occupy 8 bytes (56-63). Therefore, the total size of the struct is 64 bytes. Now, let's compare this to the pre-patch layout. `triple_fault.pending` was at offset 28. Then, `reserved[26]` was from 29 to 54, and `exception_has_payload` was at 55 and `exception_payload` was at 56! So, no padding and the author carefully verified the alignment, which is good. The key point here is that before copying this structure to userspace in `kvm_vcpu_ioctl_x86_get_vcpu_events()`, a `memset(events, 0, sizeof(events))` is performed. This zeroes out the entire struct, including all data, reserved fields, and any padding that might* have been present (though we've confirmed there isn't any). This is a crucial defense against potential information leaks. Therefore, I need to check for other places in the code where this struct is passed to userspace and if a `memset` is called. So, I will grep the codebase for `kvm_vcpu_events`.
81/3	2026/09/11 21:54	tool	grepper	0m Args: { "Expression": "struct kvm_vcpu_events" } Results: { "Output": "Documentation/virt/kvm/api.rst=1120=Other flags returned by ``KVM_GET_CLOCK`` are accepted but ignored.\n--\nDocumentation/virt/kvm/api.rst-1140-:Type: vcpu ioctl\nDocumentation/virt/kvm/api.rst:1141::Parameters: struct kvm_vcpu_events (out)\nDocumentation/virt/kvm/api.rst-1142-:Returns: 0 on success, -1 on error\n--\nDocumentation/virt/kvm/api.rst=1148=states of the vcpu.\n--\nDocumentation/virt/kvm/api.rst-1151-\nDocumentation/virt/kvm/api.rst:1152: struct kvm_vcpu_events {\nDocumentation/virt/kvm/api.rst-1153-\tstruct {\n--\nDocumentation/virt/kvm/api.rst=1247=Calling this ioctl on a vCPU that hasn't been initialized will return\n--\nDocumentation/virt/kvm/api.rst-1251-\nDocumentation/virt/kvm/api.rst:1252: struct kvm_vcpu_events {\nDocumentation/virt/kvm/api.rst-1253-\tstruct {\n--\nDocumentation/virt/kvm/api.rst-1270-:Type: vcpu ioctl\nDocumentation/virt/kvm/api.rst:1271::Parameters: struct kvm_vcpu_events (in)\nDocumentation/virt/kvm/api.rst-1272-:Returns: 0 on success, -1 on error\n--\nDocumentation/virt/kvm/api.rst=7749=Unused bitfields in the bitarrays must be set to zero.\n--\nDocumentation/virt/kvm/api.rst-7755- struct kvm_sregs sregs;\nDocumentation/virt/kvm/api.rst:7756: struct kvm_vcpu_events events;\nDocumentation/virt/kvm/api.rst-7757- };\n--\narch/arm64/include/asm/kvm_host.h=1250=int __kvm_arm_vcpu_get_events(struct kvm_vcpu vcpu,\narch/arm64/include/asm/kvm_host.h:1251:\t\t\t struct kvm_vcpu_events events);\narch/arm64/include/asm/kvm_host.h-1252-\narch/arm64/include/asm/kvm_host.h=1253=int __kvm_arm_vcpu_set_events(struct kvm_vcpu vcpu,\narch/arm64/include/asm/kvm_host.h:1254:\t\t\t struct kvm_vcpu_events events);\narch/arm64/include/asm/kvm_host.h-1255-\n--\narch/arm64/include/uapi/asm/kvm.h=171=struct kvm_pmu_event_filter {\n--\narch/arm64/include/uapi/asm/kvm.h-182-/* for KVM_GET/SET_VCPU_EVENTS /\narch/arm64/include/uapi/asm/kvm.h:183:struct kvm_vcpu_events {\narch/arm64/include/uapi/asm/kvm.h-184-\tstruct {\n--\narch/arm64/kvm/arm.c=1829=static int kvm_arm_vcpu_get_events(struct kvm_vcpu vcpu,\narch/arm64/kvm/arm.c:1830:\t\t\t\t struct kvm_vcpu_events events)\narch/arm64/kvm/arm.c-1831-{\n--\narch/arm64/kvm/arm.c=1837=static int kvm_arm_vcpu_set_events(struct kvm_vcpu vcpu,\narch/arm64/kvm/arm.c:1838:\t\t\t\t struct kvm_vcpu_events events)\narch/arm64/kvm/arm.c-1839-{\n--\narch/arm64/kvm/arm.c=1855=long kvm_arch_vcpu_ioctl(struct file filp,\n--\narch/arm64/kvm/arm.c-1947-\tcase KVM_GET_VCPU_EVENTS: {\narch/arm64/kvm/arm.c:1948:\t\tstruct kvm_vcpu_events events;\narch/arm64/kvm/arm.c-1949-\n--\narch/arm64/kvm/arm.c-1961-\tcase KVM_SET_VCPU_EVENTS: {\narch/arm64/kvm/arm.c:1962:\t\tstruct kvm_vcpu_events events;\narch/arm64/kvm/arm.c-1963-\n--\narch/arm64/kvm/guest.c=748=int __kvm_arm_vcpu_get_events(struct kvm_vcpu vcpu,\narch/arm64/kvm/guest.c:749:\t\t\t struct kvm_vcpu_events events)\narch/arm64/kvm/guest.c-750-{\n--\narch/arm64/kvm/guest.c=780=int __kvm_arm_vcpu_set_events(struct kvm_vcpu vcpu,\narch/arm64/kvm/guest.c:781:\t\t\t struct kvm_vcpu_events events)\narch/arm64/kvm/guest.c-782-{\n--\narch/x86/include/uapi/asm/kvm.h=322=struct kvm_reinject_control {\n--\narch/x86/include/uapi/asm/kvm.h-340-/* for KVM_GET/SET_VCPU_EVENTS /\narch/x86/include/uapi/asm/kvm.h:341:struct kvm_vcpu_events {\narch/x86/include/uapi/asm/kvm.h-342-\tstruct {\n--\narch/x86/include/uapi/asm/kvm.h=463=struct kvm_sync_regs {\n--\narch/x86/include/uapi/asm/kvm.h-470-\tstruct kvm_sregs sregs;\narch/x86/include/uapi/asm/kvm.h:471:\tstruct kvm_vcpu_events events;\narch/x86/include/uapi/asm/kvm.h-472-};\n--\narch/x86/kvm/x86.c=2994=static void kvm_vcpu_ioctl_x86_get_vcpu_events(struct kvm_vcpu vcpu,\narch/x86/kvm/x86.c:2995:\t\t\t\t\t struct kvm_vcpu_events events)\narch/x86/kvm/x86.c-2996-{\n--\narch/x86/kvm/x86.c=3067=static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu vcpu,\narch/x86/kvm/x86.c:3068:\t\t\t\t\t struct kvm_vcpu_events events)\narch/x86/kvm/x86.c-3069-{\n--\narch/x86/kvm/x86.c=3463=long kvm_arch_vcpu_ioctl(struct file filp,\n--\narch/x86/kvm/x86.c-3628-\tcase KVM_GET_VCPU_EVENTS: {\narch/x86/kvm/x86.c:3629:\t\tstruct kvm_vcpu_events events;\narch/x86/kvm/x86.c-3630-\n--\narch/x86/kvm/x86.c-3633-\t\tr = -EFAULT;\narch/x86/kvm/x86.c:3634:\t\tif (copy_to_user(argp, \u0026events, sizeof(struct kvm_vcpu_events)))\narch/x86/kvm/x86.c-3635-\t\t\tbreak;\n--\narch/x86/kvm/x86.c-3639-\tcase KVM_SET_VCPU_EVENTS: {\narch/x86/kvm/x86.c:3640:\t\tstruct kvm_vcpu_events events;\narch/x86/kvm/x86.c-3641-\narch/x86/kvm/x86.c-3642-\t\tr = -EFAULT;\narch/x86/kvm/x86.c:3643:\t\tif (copy_from_user(\u0026events, argp, sizeof(struct kvm_vcpu_events)))\narch/x86/kvm/x86.c-3644-\t\t\tbreak;\n--\narch/x86/kvm/x86.c=9342=static int sync_regs(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/x86.c-9347-\tif (vcpu-\u003erun-\u003ekvm_dirty_regs \u0026 KVM_SYNC_X86_EVENTS) {\narch/x86/kvm/x86.c:9348:\t\tstruct kvm_vcpu_events events = vcpu-\u003erun-\u003es.regs.events;\narch/x86/kvm/x86.c-9349-\n--\ninclude/uapi/linux/kvm.h=1254=struct kvm_s390_keyop {\n--\ninclude/uapi/linux/kvm.h-1399-/ Available with KVM_CAP_VCPU_EVENTS /\ninclude/uapi/linux/kvm.h:1400:#define KVM_GET_VCPU_EVENTS _IOR(KVMIO, 0x9f, struct kvm_vcpu_events)\ninclude/uapi/linux/kvm.h:1401:#define KVM_SET_VCPU_EVENTS _IOW(KVMIO, 0xa0, struct kvm_vcpu_events)\ninclude/uapi/linux/kvm.h-1402-/ Available with KVM_CAP_DEBUGREGS /\n--\ntools/arch/arm64/include/uapi/asm/kvm.h=170=struct kvm_pmu_event_filter {\n--\ntools/arch/arm64/include/uapi/asm/kvm.h-181-/ for KVM_GET/SET_VCPU_EVENTS /\ntools/arch/arm64/include/uapi/asm/kvm.h:182:struct kvm_vcpu_events {\ntools/arch/arm64/include/uapi/asm/kvm.h-183-\tstruct {\n--\ntools/arch/x86/include/uapi/asm/kvm.h=322=struct kvm_reinject_control {\n--\ntools/arch/x86/include/uapi/asm/kvm.h-339-/ for KVM_GET/SET_VCPU_EVENTS /\ntools/arch/x86/include/uapi/asm/kvm.h:340:struct kvm_vcpu_events {\ntools/arch/x86/include/uapi/asm/kvm.h-341-\tstruct {\n--\ntools/arch/x86/include/uapi/asm/kvm.h=458=struct kvm_sync_regs {\n--\ntools/arch/x86/include/uapi/asm/kvm.h-465-\tstruct kvm_sregs sregs;\ntools/arch/x86/include/uapi/asm/kvm.h:466:\tstruct kvm_vcpu_events events;\ntools/arch/x86/include/uapi/asm/kvm.h-467-};\n--\ntools/include/uapi/linux/kvm.h=1252=struct kvm_s390_keyop {\n--\ntools/include/uapi/linux/kvm.h-1397-/ Available with KVM_CAP_VCPU_EVENTS /\ntools/include/uapi/linux/kvm.h:1398:#define KVM_GET_VCPU_EVENTS _IOR(KVMIO, 0x9f, struct kvm_vcpu_events)\ntools/include/uapi/linux/kvm.h:1399:#define KVM_SET_VCPU_EVENTS _IOW(KVMIO, 0xa0, struct kvm_vcpu_events)\ntools/include/uapi/linux/kvm.h-1400-/ Available with KVM_CAP_DEBUGREGS /\n--\ntools/testing/selftests/kvm/arm64/external_aborts.c=45=static void vcpu_inject_sea(struct kvm_vcpu vcpu)\ntools/testing/selftests/kvm/arm64/external_aborts.c-46-{\ntools/testing/selftests/kvm/arm64/external_aborts.c:47:\tstruct kvm_vcpu_events events = {};\ntools/testing/selftests/kvm/arm64/external_aborts.c-48-\n--\ntools/testing/selftests/kvm/arm64/external_aborts.c=65=static void vcpu_inject_serror(struct kvm_vcpu vcpu)\ntools/testing/selftests/kvm/arm64/external_aborts.c-66-{\ntools/testing/selftests/kvm/arm64/external_aborts.c:67:\tstruct kvm_vcpu_events events = {};\ntools/testing/selftests/kvm/arm64/external_aborts.c-68-\n--\ntools/testing/selftests/kvm/arm64/sea_to_user.c=177=static void vcpu_inject_sea(struct kvm_vcpu vcpu)\ntools/testing/selftests/kvm/arm64/sea_to_user.c-178-{\ntools/testing/selftests/kvm/arm64/sea_to_user.c:179:\tstruct kvm_vcpu_events events = {};\ntools/testing/selftests/kvm/arm64/sea_to_user.c-180-\n--\ntools/testing/selftests/kvm/include/kvm_util.h=847=static inline void vcpu_events_get(struct kvm_vcpu vcpu,\ntools/testing/selftests/kvm/include/kvm_util.h:848:\t\t\t\t struct kvm_vcpu_events events)\ntools/testing/selftests/kvm/include/kvm_util.h-849-{\n--\ntools/testing/selftests/kvm/include/kvm_util.h=852=static inline void vcpu_events_set(struct kvm_vcpu vcpu,\ntools/testing/selftests/kvm/include/kvm_util.h:853:\t\t\t\t struct kvm_vcpu_events events)\ntools/testing/selftests/kvm/include/kvm_util.h-854-{\n--\ntools/testing/selftests/kvm/include/x86/processor.h=467=struct kvm_x86_state {\ntools/testing/selftests/kvm/include/x86/processor.h-468-\tstruct kvm_xsave xsave;\ntools/testing/selftests/kvm/include/x86/processor.h:469:\tstruct kvm_vcpu_events events;\ntools/testing/selftests/kvm/include/x86/processor.h-470-\tstruct kvm_mp_state mp_state;\n--\ntools/testing/selftests/kvm/lib/x86/processor.c=1490=void inject_smi(struct kvm_vcpu vcpu)\ntools/testing/selftests/kvm/lib/x86/processor.c-1491-{\ntools/testing/selftests/kvm/lib/x86/processor.c:1492:\tstruct kvm_vcpu_events events;\ntools/testing/selftests/kvm/lib/x86/processor.c-1493-\n--\ntools/testing/selftests/kvm/x86/hyperv_evmcs.c=189=void inject_nmi(struct kvm_vcpu vcpu)\ntools/testing/selftests/kvm/x86/hyperv_evmcs.c-190-{\ntools/testing/selftests/kvm/x86/hyperv_evmcs.c:191:\tstruct kvm_vcpu_events events;\ntools/testing/selftests/kvm/x86/hyperv_evmcs.c-192-\n--\ntools/testing/selftests/kvm/x86/nested_exceptions_test.c=185=static void queue_ss_exception(struct kvm_vcpu vcpu, bool inject)\ntools/testing/selftests/kvm/x86/nested_exceptions_test.c-186-{\ntools/testing/selftests/kvm/x86/nested_exceptions_test.c:187:\tstruct kvm_vcpu_events events;\ntools/testing/selftests/kvm/x86/nested_exceptions_test.c-188-\n--\ntools/testing/selftests/kvm/x86/nested_exceptions_test.c=212=int main(int argc, char argv[])\n--\ntools/testing/selftests/kvm/x86/nested_exceptions_test.c-214-\tgva_t nested_test_data_gva;\ntools/testing/selftests/kvm/x86/nested_exceptions_test.c:215:\tstruct kvm_vcpu_events events;\ntools/testing/selftests/kvm/x86/nested_exceptions_test.c-216-\tstruct kvm_vcpu vcpu;\n--\ntools/testing/selftests/kvm/x86/sync_regs_test.c=73=static void compare_sregs(struct kvm_sregs left, struct kvm_sregs right)\n--\ntools/testing/selftests/kvm/x86/sync_regs_test.c-76-\ntools/testing/selftests/kvm/x86/sync_regs_test.c:77:static void compare_vcpu_events(struct kvm_vcpu_events left,\ntools/testing/selftests/kvm/x86/sync_regs_test.c:78:\t\t\t\tstruct kvm_vcpu_events right)\ntools/testing/selftests/kvm/x86/sync_regs_test.c-79-{\n--\ntools/testing/selftests/kvm/x86/sync_regs_test.c=90=static void race_events_inj_pen(void arg)\n--\ntools/testing/selftests/kvm/x86/sync_regs_test.c-92-\tstruct kvm_run run = (struct kvm_run )arg;\ntools/testing/selftests/kvm/x86/sync_regs_test.c:93:\tstruct kvm_vcpu_events events = \u0026run-\u003es.regs.events;\ntools/testing/selftests/kvm/x86/sync_regs_test.c-94-\n--\ntools/testing/selftests/kvm/x86/sync_regs_test.c=113=static void race_events_exc(void arg)\n--\ntools/testing/selftests/kvm/x86/sync_regs_test.c-115-\tstruct kvm_run run = (struct kvm_run )arg;\ntools/testing/selftests/kvm/x86/sync_regs_test.c:116:\tstruct kvm_vcpu_events events = \u0026run-\u003es.regs.events;\ntools/testing/selftests/kvm/x86/sync_regs_test.c-117-\n--\ntools/testing/selftests/kvm/x86/sync_regs_test.c=249=KVM_ONE_VCPU_TEST(sync_regs_test, req_and_verify_all_valid, guest_code)\n--\ntools/testing/selftests/kvm/x86/sync_regs_test.c-251-\tstruct kvm_run run = vcpu-\u003erun;\ntools/testing/selftests/kvm/x86/sync_regs_test.c:252:\tstruct kvm_vcpu_events events;\ntools/testing/selftests/kvm/x86/sync_regs_test.c-253-\tstruct kvm_sregs sregs;\n--\ntools/testing/selftests/kvm/x86/sync_regs_test.c=271=KVM_ONE_VCPU_TEST(sync_regs_test, set_and_verify_various, guest_code)\n--\ntools/testing/selftests/kvm/x86/sync_regs_test.c-273-\tstruct kvm_run run = vcpu-\u003erun;\ntools/testing/selftests/kvm/x86/sync_regs_test.c:274:\tstruct kvm_vcpu_events events;\ntools/testing/selftests/kvm/x86/sync_regs_test.c-275-\tstruct kvm_sregs sregs;\n--\ntools/testing/selftests/kvm/x86/triple_fault_event_test.c=54=int main(void)\n--\ntools/testing/selftests/kvm/x86/triple_fault_event_test.c-57-\tstruct kvm_run run;\ntools/testing/selftests/kvm/x86/triple_fault_event_test.c:58:\tstruct kvm_vcpu_events events;\ntools/testing/selftests/kvm/x86/triple_fault_event_test.c-59-\tstruct ucall uc;\n--\ntools/testing/selftests/kvm/x86/vmx_exception_with_invalid_guest_state.c=85=static void sigalrm_handler(int sig)\n--\ntools/testing/selftests/kvm/x86/vmx_exception_with_invalid_guest_state.c-87-\tstruct kvm_vcpu vcpu = get_set_sigalrm_vcpu(NULL);\ntools/testing/selftests/kvm/x86/vmx_exception_with_invalid_guest_state.c:88:\tstruct kvm_vcpu_events events;\ntools/testing/selftests/kvm/x86/vmx_exception_with_invalid_guest_state.c-89-\n" }
82/3	2026/09/11 21:54	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 85867 output: 18 thoughts: 25 Thoughts: Here's a summary of my thought process, as if I were the one thinking it: Digging into Register State Updates Okay, so I need to understand how register events are being updated within the virtual CPU's run structure. My first instinct is to trace where the `vcpu->run->s.regs.events` field is written to. I need to find the function that's modifying it, so I can see what's triggering those changes. I'll start by checking the `store_regs` function, as that seems like a likely candidate given the context. I'll examine the code within that function to see if the write to `vcpu->run->s.regs.events` occurs, and if so, understand the conditions that lead to that write. This will give me crucial insights into the register state management within the VCPU context.
83/3	2026/09/11 21:54	tool	grepper	0m Args: { "Expression": "store_regs\\(" } Results: { "Output": "arch/powerpc/platforms/52xx/lite5200_pm.c=98=static void lite5200_save_regs(void)\n--\narch/powerpc/platforms/52xx/lite5200_pm.c-110-\narch/powerpc/platforms/52xx/lite5200_pm.c:111:static void lite5200_restore_regs(void)\narch/powerpc/platforms/52xx/lite5200_pm.c-112-{\n--\narch/powerpc/platforms/52xx/lite5200_pm.c=204=static int lite5200_pm_enter(suspend_state_t state)\n--\narch/powerpc/platforms/52xx/lite5200_pm.c-217-\narch/powerpc/platforms/52xx/lite5200_pm.c:218:\tlite5200_restore_regs();\narch/powerpc/platforms/52xx/lite5200_pm.c-219-\n--\narch/powerpc/platforms/83xx/suspend.c=143=static irqreturn_t pmc_irq_handler(int irq, void dev_id)\n--\narch/powerpc/platforms/83xx/suspend.c-158-\narch/powerpc/platforms/83xx/suspend.c:159:static void mpc83xx_suspend_restore_regs(void)\narch/powerpc/platforms/83xx/suspend.c-160-{\n--\narch/powerpc/platforms/83xx/suspend.c=173=static int mpc83xx_suspend_enter(suspend_state_t state)\n--\narch/powerpc/platforms/83xx/suspend.c-218-\narch/powerpc/platforms/83xx/suspend.c:219:\t\tmpc83xx_suspend_restore_regs();\narch/powerpc/platforms/83xx/suspend.c-220-\t} else {\n--\narch/s390/kvm/s390/s390.c=4959=static void store_regs_fmt2(struct kvm_vcpu vcpu)\n--\narch/s390/kvm/s390/s390.c-4982-\narch/s390/kvm/s390/s390.c:4983:static void store_regs(struct kvm_vcpu vcpu)\narch/s390/kvm/s390/s390.c-4984-{\n--\narch/s390/kvm/s390/s390.c=5004=int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu vcpu)\n--\narch/s390/kvm/s390/s390.c-5076-\tdisable_cpu_timer_accounting(vcpu);\narch/s390/kvm/s390/s390.c:5077:\tstore_regs(vcpu);\narch/s390/kvm/s390/s390.c-5078-\tkernel_fpu_end(\u0026fpu, KERNEL_FPC \| KERNEL_VXR);\n--\narch/s390/net/bpf_jit_comp.c=445=static void save_regs(struct bpf_jit jit, u32 rs, u32 re)\n--\narch/s390/net/bpf_jit_comp.c-459- /\narch/s390/net/bpf_jit_comp.c:460:static void restore_regs(struct bpf_jit jit, u32 rs, u32 re)\narch/s390/net/bpf_jit_comp.c-461-{\n--\narch/s390/net/bpf_jit_comp.c=489=static int get_end(u16 seen_regs, int start)\n--\narch/s390/net/bpf_jit_comp.c-505- /\narch/s390/net/bpf_jit_comp.c:506:static void save_restore_regs(struct bpf_jit jit, int op, u16 extra_regs)\narch/s390/net/bpf_jit_comp.c-507-{\n--\narch/s390/net/bpf_jit_comp.c-528-\t\telse\narch/s390/net/bpf_jit_comp.c:529:\t\t\trestore_regs(jit, rs, re);\narch/s390/net/bpf_jit_comp.c-530-\t\tre++;\n--\narch/s390/net/bpf_jit_comp.c=596=static void bpf_jit_prologue(struct bpf_jit jit, struct bpf_prog fp)\n--\narch/s390/net/bpf_jit_comp.c-631-\t\t/ Save registers /\narch/s390/net/bpf_jit_comp.c:632:\t\tsave_restore_regs(jit, REGS_SAVE,\narch/s390/net/bpf_jit_comp.c-633-\t\t\t\t fp-\u003eaux-\u003eexception_boundary ? NVREGS : 0);\n--\narch/s390/net/bpf_jit_comp.c=680=static void bpf_jit_epilogue(struct bpf_jit jit)\n--\narch/s390/net/bpf_jit_comp.c-685-\t/* Restore registers /\narch/s390/net/bpf_jit_comp.c:686:\tsave_restore_regs(jit, REGS_RESTORE, 0);\narch/s390/net/bpf_jit_comp.c-687-\tEMIT_JUMP_REG(14);\n--\narch/s390/net/bpf_jit_comp.c=876=static noinline int bpf_jit_insn(struct bpf_jit jit, struct bpf_prog fp,\n--\narch/s390/net/bpf_jit_comp.c-1940-\t\t /\narch/s390/net/bpf_jit_comp.c:1941:\t\tsave_restore_regs(jit, REGS_RESTORE, 0);\narch/s390/net/bpf_jit_comp.c-1942-\n--\narch/sh/kernel/cpu/sh3/entry.S=188=ENTRY(sh_bios_handler)\n--\narch/sh/kernel/cpu/sh3/entry.S-204-\narch/sh/kernel/cpu/sh3/entry.S:205:! restore_regs()\narch/sh/kernel/cpu/sh3/entry.S-206-! - restore r0, r1, r2, r3, r4, r5, r6, r7 from the stack\n--\narch/x86/kvm/x86.c=159=static void process_nmi(struct kvm_vcpu vcpu);\narch/x86/kvm/x86.c:160:static void store_regs(struct kvm_vcpu vcpu);\narch/x86/kvm/x86.c-161-static int sync_regs(struct kvm_vcpu vcpu);\n--\narch/x86/kvm/x86.c=8941=int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu vcpu)\n--\narch/x86/kvm/x86.c-9051-\tif (kvm_run-\u003ekvm_valid_regs \u0026\u0026 likely(!vcpu-\u003earch.guest_state_protected))\narch/x86/kvm/x86.c:9052:\t\tstore_regs(vcpu);\narch/x86/kvm/x86.c-9053-\tpost_kvm_run_save(vcpu);\n--\narch/x86/kvm/x86.c=9309=int kvm_arch_vcpu_ioctl_set_fpu(struct kvm_vcpu vcpu, struct kvm_fpu fpu)\n--\narch/x86/kvm/x86.c-9332-\narch/x86/kvm/x86.c:9333:static void store_regs(struct kvm_vcpu vcpu)\narch/x86/kvm/x86.c-9334-{\n--\narch/x86/net/bpf_jit_comp.c=2996=static void save_args(const struct btf_func_model m, u8 *prog,\n--\narch/x86/net/bpf_jit_comp.c-3073-\narch/x86/net/bpf_jit_comp.c:3074:static void restore_regs(const struct btf_func_model m, u8 *prog,\narch/x86/net/bpf_jit_comp.c-3075-\t\t\t int stack_size)\n--\narch/x86/net/bpf_jit_comp.c=3346=static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image im, void rw_image,\n--\narch/x86/net/bpf_jit_comp.c-3549-\tif (flags \u0026 BPF_TRAMP_F_CALL_ORIG) {\narch/x86/net/bpf_jit_comp.c:3550:\t\trestore_regs(m, \u0026prog, regs_off);\narch/x86/net/bpf_jit_comp.c-3551-\t\tsave_args(m, \u0026prog, arg_stack_off, true, flags);\n--\narch/x86/net/bpf_jit_comp.c-3605-\tif (flags \u0026 BPF_TRAMP_F_RESTORE_REGS)\narch/x86/net/bpf_jit_comp.c:3606:\t\trestore_regs(m, \u0026prog, regs_off);\narch/x86/net/bpf_jit_comp.c-3607-\n--\ndrivers/dma/pch_dma.c=733=static void __maybe_unused pch_dma_save_regs(struct pch_dma pd)\n--\ndrivers/dma/pch_dma.c-755-\ndrivers/dma/pch_dma.c:756:static void __maybe_unused pch_dma_restore_regs(struct pch_dma pd)\ndrivers/dma/pch_dma.c-757-{\n--\ndrivers/dma/pch_dma.c=789=static int __maybe_unused pch_dma_resume(struct device dev)\n--\ndrivers/dma/pch_dma.c-793-\tif (pd)\ndrivers/dma/pch_dma.c:794:\t\tpch_dma_restore_regs(pd);\ndrivers/dma/pch_dma.c-795-\n--\ndrivers/gpio/gpio-mxc.c=542=static void mxc_gpio_save_regs(struct mxc_gpio_port port)\n--\ndrivers/gpio/gpio-mxc.c-554-\ndrivers/gpio/gpio-mxc.c:555:static void mxc_gpio_restore_regs(struct mxc_gpio_port port)\ndrivers/gpio/gpio-mxc.c-556-{\n--\ndrivers/gpio/gpio-mxc.c=633=static int mxc_gpio_runtime_resume(struct device dev)\n--\ndrivers/gpio/gpio-mxc.c-644-\ndrivers/gpio/gpio-mxc.c:645:\tmxc_gpio_restore_regs(port);\ndrivers/gpio/gpio-mxc.c-646-\n--\ndrivers/gpio/gpio-mxc.c=695=static void mxc_gpio_syscore_resume(void data)\n--\ndrivers/gpio/gpio-mxc.c-706-\t\t}\ndrivers/gpio/gpio-mxc.c:707:\t\tmxc_gpio_restore_regs(port);\ndrivers/gpio/gpio-mxc.c-708-\t\tclk_disable_unprepare(port-\u003eclk);\n--\ndrivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c=547=static void gfxhub_v2_1_save_regs(struct amdgpu_device adev)\n--\ndrivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c-581-\ndrivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c:582:static void gfxhub_v2_1_restore_regs(struct amdgpu_device adev)\ndrivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c-583-{\n--\ndrivers/gpu/drm/amd/amdgpu/sienna_cichlid.c=149=static int sienna_cichlid_mode2_restore_ip(struct amdgpu_device adev)\n--\ndrivers/gpu/drm/amd/amdgpu/sienna_cichlid.c-161-\tif (adev-\u003egfxhub.funcs-\u003emode2_restore_regs)\ndrivers/gpu/drm/amd/amdgpu/sienna_cichlid.c:162:\t\tadev-\u003egfxhub.funcs-\u003emode2_restore_regs(adev);\ndrivers/gpu/drm/amd/amdgpu/sienna_cichlid.c-163-\tadev-\u003egfxhub.funcs-\u003einit(adev);\n--\ndrivers/gpu/drm/gma500/power.c=110=static void gma_resume_display(struct pci_dev pdev)\n--\ndrivers/gpu/drm/gma500/power.c-124-\tpsb_gem_mm_resume(dev);\ndrivers/gpu/drm/gma500/power.c:125:\tdev_priv-\u003eops-\u003erestore_regs(dev);\ndrivers/gpu/drm/gma500/power.c-126-}\n--\ndrivers/i2c/busses/i2c-i801.c=1522=static void i801_setup_hstcfg(struct i801_priv priv)\n--\ndrivers/i2c/busses/i2c-i801.c-1530-\ndrivers/i2c/busses/i2c-i801.c:1531:static void i801_restore_regs(struct i801_priv priv)\ndrivers/i2c/busses/i2c-i801.c-1532-{\n--\ndrivers/i2c/busses/i2c-i801.c=1537=static int i801_probe(struct pci_dev dev, const struct pci_device_id id)\n--\ndrivers/i2c/busses/i2c-i801.c-1668-\t\ti801_acpi_remove(priv);\ndrivers/i2c/busses/i2c-i801.c:1669:\t\ti801_restore_regs(priv);\ndrivers/i2c/busses/i2c-i801.c-1670-\t\treturn err;\n--\ndrivers/i2c/busses/i2c-i801.c=1690=static void i801_remove(struct pci_dev dev)\n--\ndrivers/i2c/busses/i2c-i801.c-1704-\ndrivers/i2c/busses/i2c-i801.c:1705:\ti801_restore_regs(priv);\ndrivers/i2c/busses/i2c-i801.c-1706-\n--\ndrivers/i2c/busses/i2c-i801.c=1713=static void i801_shutdown(struct pci_dev dev)\n--\ndrivers/i2c/busses/i2c-i801.c-1718-\t/* Restore config registers to avoid hard hang on some systems /\ndrivers/i2c/busses/i2c-i801.c:1719:\ti801_restore_regs(priv);\ndrivers/i2c/busses/i2c-i801.c-1720-}\n--\ndrivers/i2c/busses/i2c-i801.c=1722=static int i801_suspend(struct device dev)\n--\ndrivers/i2c/busses/i2c-i801.c-1726-\ti2c_mark_adapter_suspended(\u0026priv-\u003eadapter);\ndrivers/i2c/busses/i2c-i801.c:1727:\ti801_restore_regs(priv);\ndrivers/i2c/busses/i2c-i801.c-1728-\n--\ndrivers/i3c/master/svc-i3c-master.c=2096=static void svc_i3c_save_regs(struct svc_i3c_master master)\n--\ndrivers/i3c/master/svc-i3c-master.c-2101-\ndrivers/i3c/master/svc-i3c-master.c:2102:static void svc_i3c_restore_regs(struct svc_i3c_master master)\ndrivers/i3c/master/svc-i3c-master.c-2103-{\n--\ndrivers/i3c/master/svc-i3c-master.c=2124=static int __maybe_unused svc_i3c_runtime_resume(struct device dev)\n--\ndrivers/i3c/master/svc-i3c-master.c-2133-\ndrivers/i3c/master/svc-i3c-master.c:2134:\tsvc_i3c_restore_regs(master);\ndrivers/i3c/master/svc-i3c-master.c-2135-\n--\ndrivers/irqchip/irq-imx-irqsteer.c=293=static void imx_irqsteer_save_regs(struct irqsteer_data data)\n--\ndrivers/irqchip/irq-imx-irqsteer.c-301-\ndrivers/irqchip/irq-imx-irqsteer.c:302:static void imx_irqsteer_restore_regs(struct irqsteer_data data)\ndrivers/irqchip/irq-imx-irqsteer.c-303-{\n--\ndrivers/irqchip/irq-imx-irqsteer.c=324=static int imx_irqsteer_resume(struct device dev)\n--\ndrivers/irqchip/irq-imx-irqsteer.c-333-\t}\ndrivers/irqchip/irq-imx-irqsteer.c:334:\timx_irqsteer_restore_regs(irqsteer_data);\ndrivers/irqchip/irq-imx-irqsteer.c-335-\n--\ndrivers/leds/leds-cht-wcove.c=312=static int cht_wc_led_save_regs(struct cht_wc_led led,\n--\ndrivers/leds/leds-cht-wcove.c-327-\ndrivers/leds/leds-cht-wcove.c:328:static void cht_wc_led_restore_regs(struct cht_wc_led led,\ndrivers/leds/leds-cht-wcove.c-329-\t\t\t\t const struct cht_wc_led_saved_regs saved_regs)\n--\ndrivers/leds/leds-cht-wcove.c=405=static void cht_wc_leds_remove(struct platform_device pdev)\n--\ndrivers/leds/leds-cht-wcove.c-410-\tif (!(leds-\u003eled1_initial_regs.ctrl \u0026 CHT_WC_LED1_SWCTL))\ndrivers/leds/leds-cht-wcove.c:411:\t\tcht_wc_led_restore_regs(\u0026leds-\u003eleds[0], \u0026leds-\u003eled1_initial_regs);\ndrivers/leds/leds-cht-wcove.c-412-}\n--\ndrivers/leds/leds-cht-wcove.c=414=static void cht_wc_leds_disable(struct platform_device pdev)\n--\ndrivers/leds/leds-cht-wcove.c-423-\tif (!(leds-\u003eled1_initial_regs.ctrl \u0026 CHT_WC_LED1_SWCTL))\ndrivers/leds/leds-cht-wcove.c:424:\t\tcht_wc_led_restore_regs(\u0026leds-\u003eleds[0], \u0026leds-\u003eled1_initial_regs);\ndrivers/leds/leds-cht-wcove.c-425-}\n--\ndrivers/leds/leds-cht-wcove.c=444=static int cht_wc_leds_resume(struct device dev)\n--\ndrivers/leds/leds-cht-wcove.c-449-\tfor (i = 0; i \u003c CHT_WC_LED_COUNT; i++)\ndrivers/leds/leds-cht-wcove.c:450:\t\tcht_wc_led_restore_regs(\u0026leds-\u003eleds[i], \u0026leds-\u003eleds[i].saved_regs);\ndrivers/leds/leds-cht-wcove.c-451-\n--\ndrivers/macintosh/therm_windtunnel.c=263=static void\ndrivers/macintosh/therm_windtunnel.c:264:restore_regs( void )\ndrivers/macintosh/therm_windtunnel.c-265-{\n--\ndrivers/macintosh/therm_windtunnel.c=276=static int control_loop(void dummy)\n--\ndrivers/macintosh/therm_windtunnel.c-292-\tmutex_lock(\u0026x.lock);\ndrivers/macintosh/therm_windtunnel.c:293:\trestore_regs();\ndrivers/macintosh/therm_windtunnel.c-294-\tmutex_unlock(\u0026x.lock);\n--\ndrivers/mmc/host/sdhci-brcmstb.c=84=static void sdhci_brcmstb_save_regs(struct mmc_host mmc, enum cfg_core_ver ver)\n--\ndrivers/mmc/host/sdhci-brcmstb.c-105-\ndrivers/mmc/host/sdhci-brcmstb.c:106:static void sdhci_brcmstb_restore_regs(struct mmc_host mmc, enum cfg_core_ver ver)\ndrivers/mmc/host/sdhci-brcmstb.c-107-{\n--\ndrivers/mmc/host/sdhci-brcmstb.c=128=static void sdhci_brcmstb_save_restore_regs_v1(struct mmc_host mmc, int save)\n--\ndrivers/mmc/host/sdhci-brcmstb.c-132-\telse\ndrivers/mmc/host/sdhci-brcmstb.c:133:\t\tsdhci_brcmstb_restore_regs(mmc, SDIO_CFG_CORE_V1);\ndrivers/mmc/host/sdhci-brcmstb.c-134-}\n--\ndrivers/mmc/host/sdhci-brcmstb.c=136=static void sdhci_brcmstb_save_restore_regs_v2(struct mmc_host mmc, int save)\n--\ndrivers/mmc/host/sdhci-brcmstb.c-140-\telse\ndrivers/mmc/host/sdhci-brcmstb.c:141:\t\tsdhci_brcmstb_restore_regs(mmc, SDIO_CFG_CORE_V2);\ndrivers/mmc/host/sdhci-brcmstb.c-142-}\n--\ndrivers/mmc/host/sdhci-brcmstb.c=626=static int sdhci_brcmstb_suspend(struct device dev)\n--\ndrivers/mmc/host/sdhci-brcmstb.c-635-\tif (match_priv-\u003esave_restore_regs)\ndrivers/mmc/host/sdhci-brcmstb.c:636:\t\tmatch_priv-\u003esave_restore_regs(host-\u003emmc, 1);\ndrivers/mmc/host/sdhci-brcmstb.c-637-\n--\ndrivers/mmc/host/sdhci-brcmstb.c=648=static int sdhci_brcmstb_resume(struct device dev)\n--\ndrivers/mmc/host/sdhci-brcmstb.c-670-\tif (match_priv-\u003esave_restore_regs)\ndrivers/mmc/host/sdhci-brcmstb.c:671:\t\tmatch_priv-\u003esave_restore_regs(host-\u003emmc, 0);\ndrivers/mmc/host/sdhci-brcmstb.c-672-\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8188e.c=750=static void rtl8188eu_phy_iqcalibrate(struct rtl8xxxu_priv priv,\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8188e.c-887-\t/* Reload ADDA power saving parameters /\ndrivers/net/wireless/realtek/rtl8xxxu/8188e.c:888:\trtl8xxxu_restore_regs(priv, adda_regs, priv-\u003eadda_backup,\ndrivers/net/wireless/realtek/rtl8xxxu/8188e.c-889-\t\t\t RTL8XXXU_ADDA_REGS);\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8188e.c-894-\t/ Reload BB parameters /\ndrivers/net/wireless/realtek/rtl8xxxu/8188e.c:895:\trtl8xxxu_restore_regs(priv, iqk_bb_regs,\ndrivers/net/wireless/realtek/rtl8xxxu/8188e.c-896-\t\t\t priv-\u003ebb_backup, RTL8XXXU_BB_REGS);\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8188f.c=1061=static void rtl8188fu_phy_iqcalibrate(struct rtl8xxxu_priv priv,\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8188f.c-1188-\t/* Reload ADDA power saving parameters /\ndrivers/net/wireless/realtek/rtl8xxxu/8188f.c:1189:\trtl8xxxu_restore_regs(priv, adda_regs, priv-\u003eadda_backup,\ndrivers/net/wireless/realtek/rtl8xxxu/8188f.c-1190-\t\t\t RTL8XXXU_ADDA_REGS);\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8188f.c-1195-\t/ Reload BB parameters /\ndrivers/net/wireless/realtek/rtl8xxxu/8188f.c:1196:\trtl8xxxu_restore_regs(priv, iqk_bb_regs,\ndrivers/net/wireless/realtek/rtl8xxxu/8188f.c-1197-\t\t\t priv-\u003ebb_backup, RTL8XXXU_BB_REGS);\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8192e.c=1050=static void rtl8192eu_phy_iqcalibrate(struct rtl8xxxu_priv priv,\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8192e.c-1204-\t\t/* Reload ADDA power saving parameters /\ndrivers/net/wireless/realtek/rtl8xxxu/8192e.c:1205:\t\trtl8xxxu_restore_regs(priv, adda_regs, priv-\u003eadda_backup,\ndrivers/net/wireless/realtek/rtl8xxxu/8192e.c-1206-\t\t\t\t RTL8XXXU_ADDA_REGS);\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8192e.c-1211-\t\t/ Reload BB parameters /\ndrivers/net/wireless/realtek/rtl8xxxu/8192e.c:1212:\t\trtl8xxxu_restore_regs(priv, iqk_bb_regs,\ndrivers/net/wireless/realtek/rtl8xxxu/8192e.c-1213-\t\t\t\t priv-\u003ebb_backup, RTL8XXXU_BB_REGS);\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8192f.c=1255=static void rtl8192fu_phy_iqcalibrate(struct rtl8xxxu_priv priv,\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8192f.c-1418-\t/* Reload ADDA power saving parameters /\ndrivers/net/wireless/realtek/rtl8xxxu/8192f.c:1419:\trtl8xxxu_restore_regs(priv, adda_regs, priv-\u003eadda_backup,\ndrivers/net/wireless/realtek/rtl8xxxu/8192f.c-1420-\t\t\t ARRAY_SIZE(adda_regs));\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8192f.c-1425-\t/ Reload BB parameters /\ndrivers/net/wireless/realtek/rtl8xxxu/8192f.c:1426:\trtl8xxxu_restore_regs(priv, iqk_bb_regs, priv-\u003ebb_backup, RTL8XXXU_BB_REGS);\ndrivers/net/wireless/realtek/rtl8xxxu/8192f.c-1427-\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8710b.c=1254=static void rtl8710bu_phy_iqcalibrate(struct rtl8xxxu_priv priv,\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8710b.c-1380-\t/* Reload ADDA power saving parameters /\ndrivers/net/wireless/realtek/rtl8xxxu/8710b.c:1381:\trtl8xxxu_restore_regs(priv, adda_regs, priv-\u003eadda_backup, RTL8XXXU_ADDA_REGS);\ndrivers/net/wireless/realtek/rtl8xxxu/8710b.c-1382-\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8710b.c-1386-\t/ Reload BB parameters /\ndrivers/net/wireless/realtek/rtl8xxxu/8710b.c:1387:\trtl8xxxu_restore_regs(priv, iqk_bb_regs, priv-\u003ebb_backup, RTL8XXXU_BB_REGS);\ndrivers/net/wireless/realtek/rtl8xxxu/8710b.c-1388-\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8723b.c=894=static void rtl8723bu_phy_iqcalibrate(struct rtl8xxxu_priv priv,\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8723b.c-1070-\t\t/* Reload ADDA power saving parameters /\ndrivers/net/wireless/realtek/rtl8xxxu/8723b.c:1071:\t\trtl8xxxu_restore_regs(priv, adda_regs, priv-\u003eadda_backup,\ndrivers/net/wireless/realtek/rtl8xxxu/8723b.c-1072-\t\t\t\t RTL8XXXU_ADDA_REGS);\n--\ndrivers/net/wireless/realtek/rtl8xxxu/8723b.c-1077-\t\t/ Reload BB parameters /\ndrivers/net/wireless/realtek/rtl8xxxu/8723b.c:1078:\t\trtl8xxxu_restore_regs(priv, iqk_bb_regs,\ndrivers/net/wireless/realtek/rtl8xxxu/8723b.c-1079-\t\t\t\t priv-\u003ebb_backup, RTL8XXXU_BB_REGS);\n--\ndrivers/net/wireless/realtek/rtl8xxxu/core.c=3038=void rtl8xxxu_save_regs(struct rtl8xxxu_priv priv, const u32 regs,\n--\ndrivers/net/wireless/realtek/rtl8xxxu/core.c-3046-\ndrivers/net/wireless/realtek/rtl8xxxu/core.c:3047:void rtl8xxxu_restore_regs(struct rtl8xxxu_priv priv, const u32 regs,\ndrivers/net/wireless/realtek/rtl8xxxu/core.c-3048-\t\t\t u32 backup, int count)\n--\ndrivers/net/wireless/realtek/rtl8xxxu/core.c=3183=static void rtl8xxxu_phy_iqcalibrate(struct rtl8xxxu_priv priv,\n--\ndrivers/net/wireless/realtek/rtl8xxxu/core.c-3363-\t\t/ Reload ADDA power saving parameters /\ndrivers/net/wireless/realtek/rtl8xxxu/core.c:3364:\t\trtl8xxxu_restore_regs(priv, adda_regs, priv-\u003eadda_backup,\ndrivers/net/wireless/realtek/rtl8xxxu/core.c-3365-\t\t\t\t RTL8XXXU_ADDA_REGS);\n--\ndrivers/net/wireless/realtek/rtl8xxxu/core.c-3370-\t\t/ Reload BB parameters /\ndrivers/net/wireless/realtek/rtl8xxxu/core.c:3371:\t\trtl8xxxu_restore_regs(priv, iqk_bb_regs,\ndrivers/net/wireless/realtek/rtl8xxxu/core.c-3372-\t\t\t\t priv-\u003ebb_backup, RTL8XXXU_BB_REGS);\n--\ndrivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h=2065=void rtl8xxxu_save_regs(struct rtl8xxxu_priv priv, const u32 regs,\ndrivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h-2066-\t\t\tu32 backup, int count);\ndrivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h:2067:void rtl8xxxu_restore_regs(struct rtl8xxxu_priv priv, const u32 regs,\ndrivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h-2068-\t\t\t u32 backup, int count);\n--\ndrivers/net/wireless/realtek/rtw88/rtw8703b.c=1321=static void rtw8703b_phy_calibration(struct rtw_dev rtwdev)\n--\ndrivers/net/wireless/realtek/rtw88/rtw8703b.c-1349-\t\tif (i \u003e IQK_ROUND_0)\ndrivers/net/wireless/realtek/rtw88/rtw8703b.c:1350:\t\t\trtw8723x_iqk_restore_regs(rtwdev, \u0026backup);\ndrivers/net/wireless/realtek/rtw88/rtw8703b.c-1351-\t\trtw8723x_iqk_restore_lte_path_gnt(rtwdev, \u0026backup);\n--\ndrivers/net/wireless/realtek/rtw88/rtw8723d.c=925=static void rtw8723d_phy_calibration(struct rtw_dev rtwdev)\n--\ndrivers/net/wireless/realtek/rtw88/rtw8723d.c-948-\t\tif (i \u003e IQK_ROUND_0)\ndrivers/net/wireless/realtek/rtw88/rtw8723d.c:949:\t\t\trtw8723x_iqk_restore_regs(rtwdev, \u0026backup);\ndrivers/net/wireless/realtek/rtw88/rtw8723d.c-950-\t\trtw8723x_iqk_restore_lte_path_gnt(rtwdev, \u0026backup);\n--\ndrivers/net/wireless/realtek/rtw88/rtw8723x.c=535=static\ndrivers/net/wireless/realtek/rtw88/rtw8723x.c:536:void __rtw8723x_iqk_restore_regs(struct rtw_dev rtwdev,\ndrivers/net/wireless/realtek/rtw88/rtw8723x.c-537-\t\t\t\t const struct rtw8723x_iqk_backup_regs backup)\n--\ndrivers/net/wireless/realtek/rtw88/rtw8723x.h=419=static inline\ndrivers/net/wireless/realtek/rtw88/rtw8723x.h:420:void rtw8723x_iqk_restore_regs(struct rtw_dev rtwdev,\ndrivers/net/wireless/realtek/rtw88/rtw8723x.h-421-\t\t\t const struct rtw8723x_iqk_backup_regs backup)\ndrivers/net/wireless/realtek/rtw88/rtw8723x.h-422-{\ndrivers/net/wireless/realtek/rtw88/rtw8723x.h:423:\trtw8723x_common.iqk_restore_regs(rtwdev, backup);\ndrivers/net/wireless/realtek/rtw88/rtw8723x.h-424-}\n--\ndrivers/pinctrl/pinctrl-stmfx.c=756=static int stmfx_pinctrl_backup_regs(struct stmfx_pinctrl pctl)\n--\ndrivers/pinctrl/pinctrl-stmfx.c-779-\ndrivers/pinctrl/pinctrl-stmfx.c:780:static int stmfx_pinctrl_restore_regs(struct stmfx_pinctrl pctl)\ndrivers/pinctrl/pinctrl-stmfx.c-781-{\n--\ndrivers/pinctrl/pinctrl-stmfx.c=830=static int stmfx_pinctrl_resume(struct device dev)\n--\ndrivers/pinctrl/pinctrl-stmfx.c-834-\ndrivers/pinctrl/pinctrl-stmfx.c:835:\tret = stmfx_pinctrl_restore_regs(pctl);\ndrivers/pinctrl/pinctrl-stmfx.c-836-\tif (ret) {\n--\ndrivers/tty/serial/8250/8250_omap.c=174=static u32 uart_read(struct omap8250_priv priv, u32 reg)\n--\ndrivers/tty/serial/8250/8250_omap.c-179-/\ndrivers/tty/serial/8250/8250_omap.c:180: * Called on runtime PM resume path from omap8250_restore_regs(), and\ndrivers/tty/serial/8250/8250_omap.c-181- * omap8250_set_mctrl().\n--\ndrivers/tty/serial/8250/8250_omap.c=286=static void omap8250_update_mdr1(struct uart_8250_port up,\n--\ndrivers/tty/serial/8250/8250_omap.c-294-\ndrivers/tty/serial/8250/8250_omap.c:295:static void omap8250_restore_regs(struct uart_8250_port up)\ndrivers/tty/serial/8250/8250_omap.c-296-{\n--\ndrivers/tty/serial/8250/8250_omap.c=363=static void omap_8250_set_termios_atomic(struct uart_port port, struct ktermios termios,\n--\ndrivers/tty/serial/8250/8250_omap.c-494-\t}\ndrivers/tty/serial/8250/8250_omap.c:495:\tomap8250_restore_regs(up);\ndrivers/tty/serial/8250/8250_omap.c-496-}\n--\ndrivers/tty/serial/8250/8250_omap.c=1078=static void omap_8250_dma_tx_complete(void param)\n--\ndrivers/tty/serial/8250/8250_omap.c-1096-\t\tpriv-\u003edelayed_restore = 0;\ndrivers/tty/serial/8250/8250_omap.c:1097:\t\tomap8250_restore_regs(p);\ndrivers/tty/serial/8250/8250_omap.c-1098-\t}\n--\ndrivers/tty/serial/8250/8250_omap.c=1804=static int omap8250_runtime_resume(struct device dev)\n--\ndrivers/tty/serial/8250/8250_omap.c-1817-\t\tguard(uart_port_lock_irq)(\u0026up-\u003eport);\ndrivers/tty/serial/8250/8250_omap.c:1818:\t\tomap8250_restore_regs(up);\ndrivers/tty/serial/8250/8250_omap.c-1819-\t}\n--\ndrivers/video/fbdev/aty/radeon_pm.c=561=static void radeon_pm_save_regs(struct radeonfb_info rinfo, int saving_for_d3)\n--\ndrivers/video/fbdev/aty/radeon_pm.c-686-\ndrivers/video/fbdev/aty/radeon_pm.c:687:static void radeon_pm_restore_regs(struct radeonfb_info rinfo)\ndrivers/video/fbdev/aty/radeon_pm.c-688-{\n--\ndrivers/video/fbdev/aty/radeon_pm.c=2537=static void radeon_set_suspend(struct radeonfb_info rinfo, int suspend)\n--\ndrivers/video/fbdev/aty/radeon_pm.c-2602-\t\t\t/ Restore some registers /\ndrivers/video/fbdev/aty/radeon_pm.c:2603:\t\t\tradeon_pm_restore_regs(rinfo);\ndrivers/video/fbdev/aty/radeon_pm.c-2604-\t\t} else {\ndrivers/video/fbdev/aty/radeon_pm.c-2605-\t\t\t/ Restore registers first /\ndrivers/video/fbdev/aty/radeon_pm.c:2606:\t\t\tradeon_pm_restore_regs(rinfo);\ndrivers/video/fbdev/aty/radeon_pm.c-2607-\t\t\t/ init sdram controller /\n--\ndrivers/video/fbdev/geode/lxfb_ops.c=729=static void lx_restore_video_proc(struct lxfb_par par)\n--\ndrivers/video/fbdev/geode/lxfb_ops.c-763-\ndrivers/video/fbdev/geode/lxfb_ops.c:764:static void lx_restore_regs(struct lxfb_par par)\ndrivers/video/fbdev/geode/lxfb_ops.c-765-{\n--\ndrivers/video/fbdev/geode/lxfb_ops.c=828=int lx_powerup(struct fb_info info)\n--\ndrivers/video/fbdev/geode/lxfb_ops.c-834-\ndrivers/video/fbdev/geode/lxfb_ops.c:835:\tlx_restore_regs(par);\ndrivers/video/fbdev/geode/lxfb_ops.c-836-\n--\ndrivers/video/fbdev/geode/suspend_gx.c=132=static void gx_restore_video_proc(struct gxfb_par par)\n--\ndrivers/video/fbdev/geode/suspend_gx.c-168-\ndrivers/video/fbdev/geode/suspend_gx.c:169:static void gx_restore_regs(struct gxfb_par par)\ndrivers/video/fbdev/geode/suspend_gx.c-170-{\n--\ndrivers/video/fbdev/geode/suspend_gx.c=247=int gx_powerup(struct fb_info info)\n--\ndrivers/video/fbdev/geode/suspend_gx.c-253-\ndrivers/video/fbdev/geode/suspend_gx.c:254:\tgx_restore_regs(par);\ndrivers/video/fbdev/geode/suspend_gx.c-255-\tgx_enable_graphics(par);\n--\nsound/isa/gus/interwave.c=386=static void snd_interwave_detect_memory(struct snd_gus_card gus)\n--\nsound/isa/gus/interwave.c-472-\nsound/isa/gus/interwave.c:473:static void __snd_interwave_restore_regs(struct snd_gus_card gus)\nsound/isa/gus/interwave.c-474-{\n--\nsound/isa/gus/interwave.c=483=static void snd_interwave_init(int dev, struct snd_gus_card gus)\n--\nsound/isa/gus/interwave.c-487-\t\tsnd_gf1_write8(gus, SNDRV_GF1_GB_SOUND_BLASTER_CONTROL, 0x00);\nsound/isa/gus/interwave.c:488:\t\t__snd_interwave_restore_regs(gus);\nsound/isa/gus/interwave.c-489-\t}\n--\nsound/isa/gus/interwave.c=794=static int snd_interwave_isa_probe(struct device pdev,\n--\nsound/isa/gus/interwave.c-852-#ifdef CONFIG_PM\nsound/isa/gus/interwave.c:853:static void snd_interwave_restore_regs(struct snd_gus_card gus)\nsound/isa/gus/interwave.c-854-{\nsound/isa/gus/interwave.c-855-\tscoped_guard(spinlock_irqsave, \u0026gus-\u003ereg_lock)\nsound/isa/gus/interwave.c:856:\t\t__snd_interwave_restore_regs(gus);\nsound/isa/gus/interwave.c-857-}\n--\nsound/isa/gus/interwave.c=899=static int snd_interwave_card_resume(struct snd_card card)\n--\nsound/isa/gus/interwave.c-907-\nsound/isa/gus/interwave.c:908:\tsnd_interwave_restore_regs(iwcard-\u003egus);\nsound/isa/gus/interwave.c-909-\tsnd_interwave_restore_memory(iwcard-\u003egus);\n--\ntools/bpf/bpf_dbg.c=581=static void bpf_safe_regs(void)\n--\ntools/bpf/bpf_dbg.c-585-\ntools/bpf/bpf_dbg.c:586:static bool bpf_restore_regs(int off)\ntools/bpf/bpf_dbg.c-587-{\n--\ntools/bpf/bpf_dbg.c=1093=static int cmd_step(char num)\n--\ntools/bpf/bpf_dbg.c-1104-\tif (steps \u003c 0) {\ntools/bpf/bpf_dbg.c:1105:\t\tif (!bpf_restore_regs(steps))\ntools/bpf/bpf_dbg.c-1106-\t\t\treturn CMD_ERR;\n" }
84/3	2026/09/11 21:54	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 98596 output: 43 thoughts: 31 Thoughts: Here's my summary, as though I'm reading the code: Diving into `store_regs` Alright, let's get down to brass tacks. I'm going to take a look at the `store_regs` function, specifically in `arch/x86/kvm/x86.c:9333`. My gut tells me this function is responsible for, well, storing register contents – probably within the context of KVM and x86 architecture. Given the location in the KVM source, it's likely involved in saving the guest's register state, perhaps during a VM exit or context switch. I'll need to see precisely what registers are being saved, where they're being saved to (memory, perhaps a shadow structure?), and what trigger this operation. I'm anticipating some intricate interplay between the guest's registers and the host's representation of them. I'll be looking for any potential optimizations, error handling, and the overall efficiency of this register-saving process, since performance is paramount in virtualization. Hopefully, this function is well-commented so I can understand any nuanced design choices.
85/3	2026/09/11 21:54	tool	read-file	0m Args: { "File": "arch/x86/kvm/x86.c", "FirstLine": 9330, "LineCount": 35 } Results: { "Contents": "9330:\t\treturn 0;\n9331:\t}\n9332:\t\n9333:\tstatic void store_regs(struct kvm_vcpu vcpu)\n9334:\t{\n9335:\t\tkvm_run_sync_regs_to_user(vcpu);\n9336:\t\n9337:\t\tif (vcpu-\u003erun-\u003ekvm_valid_regs \u0026 KVM_SYNC_X86_EVENTS)\n9338:\t\t\tkvm_vcpu_ioctl_x86_get_vcpu_events(\n9339:\t\t\t\t\tvcpu, \u0026vcpu-\u003erun-\u003es.regs.events);\n9340:\t}\n9341:\t\n9342:\tstatic int sync_regs(struct kvm_vcpu vcpu)\n9343:\t{\n9344:\t\tif (kvm_run_sync_regs_from_user(vcpu))\n9345:\t\t\treturn -EINVAL;\n9346:\t\n9347:\t\tif (vcpu-\u003erun-\u003ekvm_dirty_regs \u0026 KVM_SYNC_X86_EVENTS) {\n9348:\t\t\tstruct kvm_vcpu_events events = vcpu-\u003erun-\u003es.regs.events;\n9349:\t\n9350:\t\t\tif (kvm_vcpu_ioctl_x86_set_vcpu_events(vcpu, \u0026events))\n9351:\t\t\t\treturn -EINVAL;\n9352:\t\n9353:\t\t\tvcpu-\u003erun-\u003ekvm_dirty_regs \u0026= ~KVM_SYNC_X86_EVENTS;\n9354:\t\t}\n9355:\t\n9356:\t\treturn 0;\n9357:\t}\n9358:\t\n9359:\t#define PERF_MEDIATED_PMU_MSG \\\n9360:\t\t\"Failed to enable mediated vPMU, try disabling system wide perf events and nmi_watchdog.\\n\"\n9361:\t\n9362:\tint kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)\n9363:\t{\n9364:\t\tint r;\n" }
86/3	2026/09/11 21:54	llm	kmsan-evaluator	0m Model: gemini-3.8-flash Args: null Results: null Tokens: input: 99487 output: 527 thoughts: 1391 Thoughts: KMSAN Assessment: FRED Implementation in KVM x86 Okay, so I've meticulously reviewed this patch series implementing Intel FRED support in KVM/x86. My analysis focused on the potential for uninitialized memory usage, particularly as it relates to kernel-to-user information leaks and internal kernel vulnerabilities. First, the core of my examination revolves around the new user ABI elements (`struct kvm_vcpu_events`, `KVM_CAP_X86_FRED_EVENT`), FRED MSR handling, nested VMX FRED virtualization, and exception stack refactoring. The key question is, are we leaking uninitialized kernel memory to user space, or are we relying on uninitialized values within the kernel itself in such a way that it could lead to vulnerabilities? I started with the most obvious area: information that is explicitly sent to user space. I confirmed that in `kvm_vcpu_ioctl_x86_get_vcpu_events()`, and consequently through `store_regs()`, the `struct kvm_vcpu_events` structure is always zeroed with `memset()` before any fields are populated. This means that the new fields (`exception_event_data`, `exception_is_nested`) added to this structure are guaranteed to be initialized before being sent out. Moreover, I specifically checked that these new fields replace existing reserved bytes, avoiding any padding holes. No uninitialized memory is being sent to user space here. Similarly, `KVM_GET_MSRS` accesses either zero-initialized VMCS fields or zero-initialized vCPU state for FRED MSR values. For nested VMX state (`KVM_GET_NESTED_STATE`), I verified that the `struct vmcs12` structure is allocated using `kzalloc()`, which guarantees zero initialization before any user space copies are performed. This also indicates that the struct is initialized and avoids leaking uninitialized memory. Next, I looked for conditional logic that might depend on uninitialized variables. The new `is_nested` and `event_data` fields within the `struct kvm_queued_exception` are properly initialized during vCPU allocation and are cleared and repopulated in the various exception handling paths. This is consistent across `kvm_clear_exception_queue()`, `kvm_multiple_exception()`, `kvm_requeue_exception()`, and `kvm_arch_vcpu_ioctl_run()`. It's all consistent and avoids potential logic errors stemming from uninitialized values. The exception stacks, which are per-CPU static data structures, are also properly initialized. Finally, I considered potential issues with allocated data structures, which are also handled carefully. `pre_vmenter_fred_msrs` and `at_vmexit_fred_msrs` are part of `struct nested_vmx`, allocated with `kmem_cache_zalloc`, ensuring zero-initialization. So, in the end, it boils down to: Are we reading uninitialized memory, or are there logic errors? No, this patch series doesn't introduce any new risks related to uninitialized memory reads or information leaks to user space. I went through and specifically checked potential areas such as canonical address checks, VMCS validation, and CR4 interaction. I've concluded that these are functional and correctness bugs, not uninitialized memory bugs. KASAN and standard testing are more appropriate for these issues. Therefore, no KMSAN fuzzing is necessary here. The patch series is safe from a KMSAN perspective.
87/3	2026/09/11 21:54	tool	set-results	0m Args: { "KMSANReasoning": "The patch series adds support for Intel FRED (Flexible Return and Event Delivery) in x86 KVM, including vCPU events ABI updates, FRED MSR emulation/context-switching, nested VMX FRED state support, and CPU entry area exception stack refactoring.\n\nKMSAN vs KASAN Applicability Analysis:\n1. Kernel-to-Userspace Leaks:\n - `struct kvm_vcpu_events`: The newly added fields (`exception_event_data` and `exception_is_nested`) replace part of the existing 26-byte reserved array. In `kvm_vcpu_ioctl_x86_get_vcpu_events()` (and the `store_regs()` path), `memset(events, 0, sizeof(events))` is called prior to populating any fields, ensuring all data, reserved bytes, and padding are fully zeroed before copying to userspace via `copy_to_user()`.\n - `KVM_GET_MSRS`: FRED MSR values returned to userspace originate from guest VMCS fields initialized to 0 in `init_vmcs()`, or from vCPU struct fields (`msr_guest_fred_rsp0`, `fred_ssp0_fallback`) allocated via `kmem_cache_zalloc()`.\n - Nested VMX (`struct vmcs12`): The `cached_vmcs12` buffers copied to userspace via `KVM_GET_NESTED_STATE` are allocated with `kzalloc()`.\n2. Uninitialized Memory in Control Flow:\n - The new fields in `struct kvm_queued_exception` (`is_nested`, `event_data`) are zero-initialized on vCPU allocation, cleared explicitly in `kvm_clear_exception_queue()` and `kvm_arch_vcpu_ioctl_run()`, and explicitly assigned whenever exceptions are queued or requeued (`kvm_multiple_exception()`, `kvm_requeue_exception()`).\n - The CPU entry area exception stacks are static per-CPU allocations mapped with guard pages.\n\nAny potential bugs in this series involve canonical address verification, reserved bit validation, or nested virtualization state machine logic, which are covered by functional testing and standard sanitizers (KASAN). There are no uninitialized memory usage or leak risks requiring KMSAN.", "NeedsKMSAN": false } Results:* { "KMSANReasoning": "The patch series adds support for Intel FRED (Flexible Return and Event Delivery) in x86 KVM, including vCPU events ABI updates, FRED MSR emulation/context-switching, nested VMX FRED state support, and CPU entry area exception stack refactoring.\n\nKMSAN vs KASAN Applicability Analysis:\n1. Kernel-to-Userspace Leaks:\n - `struct kvm_vcpu_events`: The newly added fields (`exception_event_data` and `exception_is_nested`) replace part of the existing 26-byte reserved array. In `kvm_vcpu_ioctl_x86_get_vcpu_events()` (and the `store_regs()` path), `memset(events, 0, sizeof(*events))` is called prior to populating any fields, ensuring all data, reserved bytes, and padding are fully zeroed before copying to userspace via `copy_to_user()`.\n - `KVM_GET_MSRS`: FRED MSR values returned to userspace originate from guest VMCS fields initialized to 0 in `init_vmcs()`, or from vCPU struct fields (`msr_guest_fred_rsp0`, `fred_ssp0_fallback`) allocated via `kmem_cache_zalloc()`.\n - Nested VMX (`struct vmcs12`): The `cached_vmcs12` buffers copied to userspace via `KVM_GET_NESTED_STATE` are allocated with `kzalloc()`.\n2. Uninitialized Memory in Control Flow:\n - The new fields in `struct kvm_queued_exception` (`is_nested`, `event_data`) are zero-initialized on vCPU allocation, cleared explicitly in `kvm_clear_exception_queue()` and `kvm_arch_vcpu_ioctl_run()`, and explicitly assigned whenever exceptions are queued or requeued (`kvm_multiple_exception()`, `kvm_requeue_exception()`).\n - The CPU entry area exception stacks are static per-CPU allocations mapped with guard pages.\n\nAny potential bugs in this series involve canonical address verification, reserved bit validation, or nested virtualization state machine logic, which are covered by functional testing and standard sanitizers (KASAN). There are no uninitialized memory usage or leak risks requiring KMSAN.", "NeedsKMSAN": false }

LLM Calls Summary:
Total Calls	Total Tokens	Avg Tokens	Total Duration (Seconds)	Avg Duration (Seconds)

Tool Calls Summary:
Total Calls	Total Duration (Seconds)	Avg Duration (Seconds)