Sometimes, vm_area_alloc_pages() will want many pages from the buddy allocator. Rather than making requests to the buddy allocator for at most 100 pages at a time, we can eagerly request large order pages a smaller number of times. We still split the large order pages down to order-0 as the rest of the vmalloc code (and some callers) depend on it. We still defer to the bulk allocator and fallback path in case of order-0 pages or failure. Running 1000 iterations of allocations on a small 4GB system finds: 1000 2mb allocations: [Baseline] [This patch] real 46.310s real 0m34.582 user 0.001s user 0.006s sys 46.058s sys 0m34.365s 10000 200kb allocations: [Baseline] [This patch] real 56.104s real 0m43.696 user 0.001s user 0.003s sys 55.375s sys 0m42.995s Signed-off-by: Vishal Moola (Oracle) ----- RFC: https://lore.kernel.org/linux-mm/20251014182754.4329-1-vishal.moola@gmail.com/ Changes since rfc: - Mask off NO_FAIL in large_gfp - Mask off GFP_COMP in large_gfp There was discussion about warning on and rejecting unsupported GFP flags in vmalloc, I'll have a separate patch for that. - Introduce nr_remaining variable to track total pages - Calculate large order as (min(max_order, ilog2()) - Attempt lower orders on failure before falling back to original path - Drop unnecessary fallback comment change --- mm/vmalloc.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index adde450ddf5e..0832f944544c 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -3619,8 +3619,44 @@ vm_area_alloc_pages(gfp_t gfp, int nid, unsigned int order, unsigned int nr_pages, struct page **pages) { unsigned int nr_allocated = 0; + unsigned int nr_remaining = nr_pages; + unsigned int max_attempt_order = MAX_PAGE_ORDER; struct page *page; int i; + gfp_t large_gfp = (gfp & + ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL | __GFP_COMP)) + | __GFP_NOWARN; + unsigned int large_order = ilog2(nr_remaining); + + large_order = min(max_attempt_order, large_order); + + /* + * Initially, attempt to have the page allocator give us large order + * pages. Do not attempt allocating smaller than order chunks since + * __vmap_pages_range() expects physically contigous pages of exactly + * order long chunks. + */ + while (large_order > order && nr_remaining) { + if (nid == NUMA_NO_NODE) + page = alloc_pages_noprof(large_gfp, large_order); + else + page = alloc_pages_node_noprof(nid, large_gfp, large_order); + + if (unlikely(!page)) { + max_attempt_order = --large_order; + continue; + } + + split_page(page, large_order); + for (i = 0; i < (1U << large_order); i++) + pages[nr_allocated + i] = page + i; + + nr_allocated += 1U << large_order; + nr_remaining = nr_pages - nr_allocated; + + large_order = ilog2(nr_remaining); + large_order = min(max_attempt_order, large_order); + } /* * For order-0 pages we make use of bulk allocator, if -- 2.51.0