trampoline_start executes WBINVD in 16-bit real mode before switching to protected mode. This WBINVD was originally added in the 1990s for Sequent NUMA-Q hardware (as noted by the comment in trampoline_32.S: "Needed for NUMA-Q should be harmless for others"), whose cross-quad caches came up non-coherent prior to OS initialization. Sequent NUMA-Q support (CONFIG_X86_NUMAQ) was removed in commit b5660ba76b41 ("x86, platforms: Remove NUMAQ"). Per the Intel SDM (Vol. 3A, Table 10-1, note 2) and AMD APM, the INIT IPI used to wake secondary CPUs preserves cache contents and leaves CR0.CD and CR0.NW unchanged (0, as already enabled by firmware or the kernel). Neither sev_es_trampoline_start nor trampoline_start64 executes WBINVD. On modern multi-core processors with large shared last-level caches (such as 504 MB shared L3 per socket on Intel Xeon 6985P-C), WBINVD is not harmless: each execution flushes and invalidates the entire coherence domain's caches and takes ~85-115 ms per CPU while disturbing active workloads on online CPUs. Remove the obsolete WBINVD from trampoline_start. 2-socket, 480-CPU Intel Xeon 6985P-C (N=479, mean of 10 runs): Offline: 40.40 s -> 40.46 s Online: 65.83 s -> 11.14 s (-83.1%) Total: 106.23 s -> 51.60 s (-51.4%) Signed-off-by: Pasha Tatashin --- arch/x86/realmode/rm/trampoline_32.S | 2 -- arch/x86/realmode/rm/trampoline_64.S | 1 - 2 files changed, 3 deletions(-) diff --git a/arch/x86/realmode/rm/trampoline_32.S b/arch/x86/realmode/rm/trampoline_32.S index 3fad907a179f..14e56ea95416 100644 --- a/arch/x86/realmode/rm/trampoline_32.S +++ b/arch/x86/realmode/rm/trampoline_32.S @@ -30,8 +30,6 @@ .balign PAGE_SIZE SYM_CODE_START(trampoline_start) - wbinvd # Needed for NUMA-Q should be harmless for others - LJMPW_RM(1f) 1: mov %cs, %ax # Code and data in the same place diff --git a/arch/x86/realmode/rm/trampoline_64.S b/arch/x86/realmode/rm/trampoline_64.S index 14d9c7daf90f..e5a3f03d6221 100644 --- a/arch/x86/realmode/rm/trampoline_64.S +++ b/arch/x86/realmode/rm/trampoline_64.S @@ -60,7 +60,6 @@ .balign PAGE_SIZE SYM_CODE_START(trampoline_start) cli # We should be safe anyway - wbinvd LJMPW_RM(1f) 1: -- 2.56.0.176.g3575e65682-goog