Commit 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB has been flushed") introduced delayed rmap removal of dirty pages until after the TLB flush, so writeback cannot clean a page that is still writable through a stale TLB entry. It skips this on UP, because on UP there are no remote TLBs (single CPU) and because the PTL prevents preemption. The PTL prevents preemption while removing the rmap, but the local TLB flush is usually only performed after dropping the PTL. Consequently, after dropping the PTL we could reschedule to another thread of the same mm, which does not flush the TLB. A kworker could then run into the original race with writeback: it cleans the folio, but the other thread can still modify it through the stale TLB entries, and these writes get lost. Fix it by simply not special-casing UP. Now, only architectures that select MMU_GATHER_NO_GATHER (s390) will skip delay_rmap handling. Fixes: 5df397dec7c4 ("mm: delay page_remove_rmap() until after the TLB has been flushed") Cc: stable@vger.kernel.org Reported-by: Lorenzo Stoakes (ARM) Link: https://lore.kernel.org/r/avrxOt_GyXLXNqtR@gremlin Signed-off-by: David Hildenbrand (Arm) --- include/asm-generic/tlb.h | 6 +----- mm/mmu_gather.c | 2 -- 2 files changed, 1 insertion(+), 7 deletions(-) diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h index 4bba02fd5d543..cb087315b4e72 100644 --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -249,23 +249,19 @@ extern bool __tlb_remove_page_size(struct mmu_gather *tlb, struct page *page, in bool __tlb_remove_folio_pages(struct mmu_gather *tlb, struct page *page, unsigned int nr_pages, bool delay_rmap); -#ifdef CONFIG_SMP /* * This both sets 'delayed_rmap', and returns true. It would be an inline * function, except we define it before the 'struct mmu_gather'. */ #define tlb_delay_rmap(tlb) (((tlb)->delayed_rmap = 1), true) extern void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma); -#endif #endif /* * We have a no-op version of the rmap removal that doesn't * delay anything. That is used on S390, which flushes remote - * TLBs synchronously, and on UP, which doesn't have any - * remote TLBs to flush and is not preemptible due to this - * all happening under the page table lock. + * TLBs synchronously. */ #ifndef tlb_delay_rmap #define tlb_delay_rmap(tlb) (false) diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c index e98397ac577c4..c022aeb7c84d0 100644 --- a/mm/mmu_gather.c +++ b/mm/mmu_gather.c @@ -49,7 +49,6 @@ static bool tlb_next_batch(struct mmu_gather *tlb) return true; } -#ifdef CONFIG_SMP static void tlb_flush_rmap_batch(struct mmu_gather_batch *batch, struct vm_area_struct *vma) { struct encoded_page **pages = batch->encoded_pages; @@ -91,7 +90,6 @@ void tlb_flush_rmaps(struct mmu_gather *tlb, struct vm_area_struct *vma) tlb_flush_rmap_batch(tlb->active, vma); tlb->delayed_rmap = 0; } -#endif /* * We might end up freeing a lot of pages. Reschedule on a regular -- 2.56.1