From: Ridong Chen lru_gen_use_mm() is called from context_switch() on every switch to a user task and unconditionally sets mm->lru_gen.bitmap to -1. The bitmap is only cleared by the page table walkers in get_next_mm() during aging, so between aging passes it is already all ones and the store is redundant. The store is not free when the mm_struct is shared: every CPU running a thread of the same process must take the cacheline in exclusive state, so the line bounces between CPUs on every context switch. On x86, switch_mm_irqs_off() returns early when switching between threads of the same mm, which can make this the only shared write on that path. Read the bitmap first and only write it when it is not already set. The semantics are unchanged: the plain WRITE_ONCE() had no ordering guarantees, and a walker clearing a bit concurrently can be missed in either version. Tested on an i7-13700HX (8P+8E cores, 24 threads), with turbo disabled and the performance governor. On each CPU two tasks sched_yield() to each other, either as threads of one process (shared mm) or as one process per CPU (private mm, control). Context switches per second per CPU, median of 5 runs, averaged over 2 boots of each kernel: CPUs shared mm private mm base patched base patched 1 1114766 1106870 -0.7% 1104512 1109284 +0.4% 2 1086513 1110204 +2.2% 1110334 1112209 +0.2% 4 1085872 1112258 +2.4% 1113146 1112467 -0.1% 8 1085945 1111371 +2.3% 1111662 1111876 +0.0% 16 660870 666748 +0.9% 665470 667058 +0.2% 24 579382 596472 +2.9% 596177 598500 +0.4% The 16 CPU case uses both SMT siblings of each P-core, which share the L1/L2, so less of the traffic crosses cores. With the patch, the shared mm case is within noise of the private mm case. will-it-scale context_switch1 and sched_yield do not show a difference (within 1% for both threads and processes): sched_yield runs one task per CPU and never switches, and context_switch1 leaves the pipe peer unpinned, so most switches stay on one CPU and the line does not move. Signed-off-by: Ridong Chen --- include/linux/mm_types.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 1ed35d258d2e..3680cc0fd714 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -1553,7 +1553,8 @@ static inline void lru_gen_use_mm(struct mm_struct *mm) * used since the last time it cleared the bitmap. So it might be worth * walking the page tables of this mm_struct to clear the accessed bit. */ - WRITE_ONCE(mm->lru_gen.bitmap, -1); + if (READ_ONCE(mm->lru_gen.bitmap) != -1UL) + WRITE_ONCE(mm->lru_gen.bitmap, -1UL); } #else /* !CONFIG_LRU_GEN_WALKS_MMU */ -- 2.34.1