From: Serapheim Dimitropoulos POSIX_FADV_DONTNEED retries invalidation after a global LRU drain whenever mapping_try_invalidate() reports a failed eviction. This includes failures for mapped, dirty or writeback folios, which dropping LRU batch references cannot make evictable while those conditions persist. Filter those failures in mapping_try_invalidate(), while the folio is still locked and before deactivation can enqueue another batch reference. Leave mapping_evict_folio() and its eviction safety checks unchanged. Use a boolean retry flag instead of a failure count: generic_fadvise() only needs to decide whether to drain and retry once. This remains a heuristic, not a test for remote LRU references. Preserve the retry for other failures on clean, unmapped folios, including failures from filemap_release_folio() and remove_mapping(), rather than limiting it to the early refcount check. This follows the problem identified in fujunjie's earlier proposal, with the filtering kept in mapping_try_invalidate() and a conservative fallback for other eviction failures. Link: https://lkml.rescloud.iu.edu/2605.0/07547.html Link: https://lkml.iu.edu/2605.1/03284.html Link: https://lkml.iu.edu/2605.1/03732.html Signed-off-by: Serapheim Dimitropoulos --- Tested baseline and patched kernels in QEMU on ext4, XFS and OverlayFS. For clean mapped files on each filesystem, 32 POSIX_FADV_DONTNEED calls produced 32 lru_add_drain_all() calls before the patch and none afterwards. A separate dirty, unmapped test using ext4 data=journal showed the same reduction. Cross-CPU and mixed-range tests confirmed that clean, unmapped folios were still evicted and the global-drain fallback remained available. Writeback and concurrent pwrite/mmap/fadvise tests also passed, with file contents verified after fsync, eviction and refault. Skipping the retry can miss opportunistic eviction if a folio becomes eligible immediately afterwards; POSIX_FADV_DONTNEED remains advisory. --- mm/fadvise.c | 12 ++++++------ mm/internal.h | 2 +- mm/truncate.c | 22 +++++++++++++--------- 3 files changed, 20 insertions(+), 16 deletions(-) diff --git a/mm/fadvise.c b/mm/fadvise.c index b63fe2141..daf655846 100644 --- a/mm/fadvise.c +++ b/mm/fadvise.c @@ -141,7 +141,7 @@ int generic_fadvise(struct file *file, loff_t offset, loff_t len, int advice) } if (end_index >= start_index) { - unsigned long nr_failed = 0; + bool need_drain = false; /* * It's common to FADV_DONTNEED right after @@ -155,14 +155,14 @@ int generic_fadvise(struct file *file, loff_t offset, loff_t len, int advice) lru_add_drain(); mapping_try_invalidate(mapping, start_index, end_index, - &nr_failed); + &need_drain); /* - * The failures may be due to the folio being - * in the LRU cache of a remote CPU. Drain all - * caches and try again. + * Clean, unmapped folios may still have references in + * remote LRU batches. Drain and retry only if a failure + * might be resolved by dropping those references. */ - if (nr_failed) { + if (need_drain) { lru_add_drain_all(); invalidate_mapping_pages(mapping, start_index, end_index); diff --git a/mm/internal.h b/mm/internal.h index 0ca863f26..1637d4275 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -628,7 +628,7 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t lstart, loff_t lend, pgoff_t *pstart, pgoff_t *pend); long mapping_evict_folio(struct address_space *mapping, struct folio *folio); unsigned long mapping_try_invalidate(struct address_space *mapping, - pgoff_t start, pgoff_t end, unsigned long *nr_failed); + pgoff_t start, pgoff_t end, bool *need_drain); /** * folio_evictable - Test whether a folio is evictable. diff --git a/mm/truncate.c b/mm/truncate.c index 5b1b13cf8..42b6a1b63 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -567,13 +567,14 @@ EXPORT_SYMBOL(truncate_inode_pages_final); * @mapping: the address_space which holds the folios to invalidate * @start: the offset 'from' which to invalidate * @end: the offset 'to' which to invalidate (inclusive) - * @nr_failed: How many folio invalidations failed + * @need_drain: Optional flag, set if a remote LRU drain may help eviction * - * This function is similar to invalidate_mapping_pages(), except that it - * returns the number of folios which could not be evicted in @nr_failed. + * This function is similar to invalidate_mapping_pages(), except that it can + * indicate whether a remote LRU drain may allow a failed eviction to succeed. + * Callers using @need_drain must initialize it to false before the first call. */ unsigned long mapping_try_invalidate(struct address_space *mapping, - pgoff_t start, pgoff_t end, unsigned long *nr_failed) + pgoff_t start, pgoff_t end, bool *need_drain) { pgoff_t indices[FOLIO_BATCH_SIZE]; struct folio_batch fbatch; @@ -599,17 +600,20 @@ unsigned long mapping_try_invalidate(struct address_space *mapping, } ret = mapping_evict_folio(mapping, folio); + /* + * Draining LRU batches cannot unmap a folio or clean it. + * Other failures may be due to remote batch references. + */ + if (!ret && need_drain && !folio_mapped(folio) && + !folio_test_dirty(folio) && !folio_test_writeback(folio)) + *need_drain = true; folio_unlock(folio); /* * Invalidation is a hint that the folio is no longer * of interest and try to speed up its reclaim. */ - if (!ret) { + if (!ret) deactivate_file_folio(folio); - /* Likely in the lru cache of a remote CPU */ - if (nr_failed) - (*nr_failed)++; - } count += ret; } --- base-commit: 0aaec43576cd41474a8e90c919631c0c9fc88417 change-id: 20261005-fadvise-lru-drain-d5b75785c9fd Best regards, -- Serapheim Dimitropoulos