truncate_inode_pages_range() takes the folios it finds out of the page cache with delete_from_page_cache_batch(), which takes the inode's i_lock and the i_pages lock once for each batch of up to FOLIO_BATCH_SIZE folios. shmem_undo_range() calls truncate_inode_folio() on each folio instead, so filemap_remove_folio() takes and drops both locks once per folio. Truncating, punching or freeing a tmpfs file or memfd made of small folios, such as a large /dev/shm segment, pays for that on every page. Clean each folio up as truncate_inode_folio() does and, once the swap entries are gone, pass the batch to delete_from_page_cache_batch(), which needs truncate_cleanup_folio() outside mm/truncate.c. Undoing a failed fallocate() only removes the folios that are not uptodate, so keep removing those one at a time. Dropping an 8 GiB file of 4K folios in a 32 vCPU VM takes (median of five runs, averaged over two boots of each kernel): operation before (ms) after (ms) change ftruncate() to 0 1144 901 -21% hole punch 1148 906 -21% close() of a memfd 1139 912 -20% unlink() 1142 914 -20% Files of PMD-sized folios take 18-31 ms with or without this change. Signed-off-by: Qiliang Yuan --- V2 -> V3: - Delete the folios from fbatch directly instead of collecting them in a second folio_batch, which pushed the shmem_undo_range() stack frame past 1024 bytes on mips with clang (reported by the kernel test robot) - Rebase onto bf87d3fe8b35 in core.git shmem-7.5.misc and update the numbers from a new run - Drop the Reviewed-by tags from Jan and Baolin, since the loop changed V1 -> V2: - Unlock the folios after folio_batch_remove_exceptionals(), as Jan suggested - Rebase onto v7.4-rc1-132-g05441d697f7f and update the numbers from a new run v2: https://lore.kernel.org/r/20261105-bug-mm-shmem-undo-batch-v2-1-ae5f89ced044@gmail.com v1: https://lore.kernel.org/r/20261029-bug-mm-shmem-undo-batch-v1-1-59b34212fe78@gmail.com To see it, fill and remove a large /dev/shm file: fallocate -l 8G /dev/shm/f && time rm /dev/shm/f --- mm/internal.h | 1 + mm/shmem.c | 9 +++++++-- mm/truncate.c | 2 +- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 566cbd2dce485..abe64e49a4170 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -628,6 +628,7 @@ unsigned find_lock_entries(struct address_space *mapping, pgoff_t *start, pgoff_t end, struct folio_batch *fbatch, pgoff_t *indices); unsigned find_get_entries(struct address_space *mapping, pgoff_t *start, pgoff_t end, struct folio_batch *fbatch, pgoff_t *indices); +void truncate_cleanup_folio(struct folio *folio); int truncate_inode_folio(struct address_space *mapping, struct folio *folio); bool truncate_inode_partial_folio(struct folio *folio, loff_t lstart, loff_t lend, pgoff_t *pstart, pgoff_t *pend); diff --git a/mm/shmem.c b/mm/shmem.c index ce92e74baeb75..5bcbdd6078a73 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1363,11 +1363,16 @@ static void shmem_undo_range(struct inode *inode, loff_t lstart, uoff_t lend, continue; } - if (!unfalloc || !folio_test_uptodate(folio)) + if (!unfalloc) + truncate_cleanup_folio(folio); + else if (!folio_test_uptodate(folio)) truncate_inode_folio(mapping, folio); - folio_unlock(folio); } folio_batch_remove_exceptionals(&fbatch); + if (!unfalloc) + delete_from_page_cache_batch(mapping, &fbatch); + for (i = 0; i < folio_batch_count(&fbatch); i++) + folio_unlock(fbatch.folios[i]); folio_batch_release(&fbatch); cond_resched_tasks_rcu_qs(); } diff --git a/mm/truncate.c b/mm/truncate.c index 733d2713a875a..c6bf4aa0b91e8 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -151,7 +151,7 @@ EXPORT_SYMBOL_GPL(folio_invalidate); * its lock, b) when a concurrent invalidate_mapping_pages got there first and * c) when tmpfs swizzles a page between a tmpfs inode and swapper_space. */ -static void truncate_cleanup_folio(struct folio *folio) +void truncate_cleanup_folio(struct folio *folio) { if (folio_mapped(folio)) unmap_mapping_folio(folio); --- base-commit: bf87d3fe8b35087c2c043642e76e9f100d61328b change-id: 20261029-bug-mm-shmem-undo-batch-de001d9c6f1e Best regards, -- Qiliang Yuan