Commit 5d8edfb900d5 ("iomap: Copy larger chunks from userspace") introduced high-order folio allocations in the buffered write path. When memory is fragmented, each failed allocation triggers compaction and drain_all_pages() via __alloc_pages_slowpath(), causing a 0.75x throughput drop on pgbench (simple-update) with 1024 clients on a 96-vCPU arm64 system. Strip __GFP_DIRECT_RECLAIM from folio allocations in iomap_get_folio() when the order exceeds PAGE_ALLOC_COSTLY_ORDER, making them purely opportunistic. Fixes: 5d8edfb900d5 ("iomap: Copy larger chunks from userspace") Cc: stable@vger.kernel.org Signed-off-by: Salvatore Dipietro --- fs/iomap/buffered-io.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 92a831cf4bf1..cb843d54b4d9 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -715,6 +715,7 @@ EXPORT_SYMBOL_GPL(iomap_is_partially_uptodate); struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len) { fgf_t fgp = FGP_WRITEBEGIN | FGP_NOFS; + gfp_t gfp; if (iter->flags & IOMAP_NOWAIT) fgp |= FGP_NOWAIT; @@ -722,8 +723,20 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len) fgp |= FGP_DONTCACHE; fgp |= fgf_set_order(len); + gfp = mapping_gfp_mask(iter->inode->i_mapping); + + /* + * If the folio order hint exceeds PAGE_ALLOC_COSTLY_ORDER, + * strip __GFP_DIRECT_RECLAIM to make the allocation purely + * opportunistic. This avoids compaction + drain_all_pages() + * in __alloc_pages_slowpath() that devastate throughput + * on large systems during buffered writes. + */ + if (FGF_GET_ORDER(fgp) > PAGE_ALLOC_COSTLY_ORDER) + gfp &= ~__GFP_DIRECT_RECLAIM; + return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT, - fgp, mapping_gfp_mask(iter->inode->i_mapping)); + fgp, gfp); } EXPORT_SYMBOL_GPL(iomap_get_folio); -- 2.50.1 (Apple Git-155) AMAZON DEVELOPMENT CENTER ITALY SRL, viale Monte Grappa 3/5, 20124 Milano, Italia, Registro delle Imprese di Milano Monza Brianza Lodi REA n. 2504859, Capitale Sociale: 10.000 EUR i.v., Cod. Fisc. e P.IVA 10100050961, Societa con Socio Unico