5.15-stable review patch. If anyone has any objections, please let me know. ------------------ From: Stephen Dolan This is a stable-specific fix. There is no single upstream commit to cherry-pick; the equivalent mainline fix is commit 83b0177a6c48 ("x86/mm: Fix SMP ordering in switch_mm_irqs_off()"), which is in v6.18 but does not apply to these trees because the surrounding code was refactored. The backports of commit fea4e317f9e7 ("x86/mm: Eliminate window where TLB flushes may be inadvertently skipped") -- the fix for CVE-2025-37964 -- to the 6.1, 6.6 and 6.12 trees ended up with two code blocks in the wrong order relative to mainline. The fix depends on setting cpu_tlbstate.loaded_mm to LOADED_MM_SWITCHING *before* reading tlb_gen, so that a concurrent TLB shootdown either sees LOADED_MM_SWITCHING and sends the IPI, or the switching CPU sees the updated tlb_gen. In the stable trees the write of LOADED_MM_SWITCHING was placed after the tlb_gen read, so the race window fea4e317f9e7 was meant to close is still open and CVE-2025-37964 is unfixed there: a process can be left running with stale TLB entries, which typically manifests as rare, hard-to-bisect memory corruption or segfaults. This has been confirmed by several independent parties: - reproduced on 6.1.y/6.6.y/6.12.y with the test program in [1], and confirmed fixed by this patch - Seth Forshee saw segfaults on 6.12.y within ~30 minutes of running a test workload; with this patch it ran 18 hours cleanly [2] - Greg Thelen reports 6.6.y- and 6.12.y-based test failures fixed by this patch [3] Dave Hansen acked the patch [4] and has no objection to it going into stable [5]. The patch below is against 6.12.y; the identical change applies to 6.1.y and 6.6.y. (The cpumask_test_cpu()/smp_mb() portion of the mainline fix is not needed here because commit 209954cbc7d0 ("x86/mm/tlb: Update mm_cpumask lazily") was never backported to these trees.) [1] https://lore.kernel.org/lkml/CAHDw0oGd0B4=uuv8NGqbUQ_ZVmSheU2bN70e4QhFXWvuAZdt2w@mail.gmail.com/ [2] https://lore.kernel.org/lkml/aZYWXe739XUJrBld@do-x1carbon/ [3] https://lore.kernel.org/lkml/CAHH2K0brx9omC9QYyB6Lio3t_1Lf8v=VaFoiaG23UgQ-aec89Q@mail.gmail.com/ [4] https://lore.kernel.org/lkml/281e8018-5506-4a79-8775-e0de7e58b95f@intel.com/ [5] https://lore.kernel.org/lkml/86421ee5-5332-46c2-bb48-d40310b818be@intel.com/ Fixes: fea4e317f9e7 ("x86/mm: Eliminate window where TLB flushes may be inadvertently skipped") # 6.1.y/6.6.y/6.12.y backports Acked-by: Dave Hansen Tested-by: Seth Forshee Tested-by: Eric Hagberg Signed-off-by: Stephen Dolan Signed-off-by: Sasha Levin [Lee: The v5.15.y backport of fea4e317f9e7 (12f703811af0) placed the LOADED_MM_SWITCHING write after the tlb_gen read too, so v5.15.y suffers from the same issue. Applies cleanly] Signed-off-by: Lee Jones Signed-off-by: Sasha Levin --- arch/x86/mm/tlb.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 11a43d373baee..f6e541c24ae2d 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -596,6 +596,14 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, */ cond_mitigation(tsk); + /* + * Indicate that CR3 is about to change. nmi_uaccess_okay() + * and others are sensitive to the window where mm_cpumask(), + * CR3 and cpu_tlbstate.loaded_mm are not all in sync. + */ + this_cpu_write(cpu_tlbstate.loaded_mm, LOADED_MM_SWITCHING); + barrier(); + /* * Stop remote flushes for the previous mm. * Skip kernel threads; we never send init_mm TLB flushing IPIs, @@ -615,14 +623,6 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, next_tlb_gen = atomic64_read(&next->context.tlb_gen); choose_new_asid(next, next_tlb_gen, &new_asid, &need_flush); - - /* - * Indicate that CR3 is about to change. nmi_uaccess_okay() - * and others are sensitive to the window where mm_cpumask(), - * CR3 and cpu_tlbstate.loaded_mm are not all in sync. - */ - this_cpu_write(cpu_tlbstate.loaded_mm, LOADED_MM_SWITCHING); - barrier(); } if (need_flush) { -- 2.53.0