From: Yili Zhang The head page used during HugeTLB vmemmap optimization is allocated with a bare GFP_KERNEL mask. When boot-time hugepage allocation drains free memory, parallel optimization workers can enter OOM handling before userspace exists, leaving the kernel with no killable process and causing a panic. hugetlb_pages_alloc_boot_node() invokes HVO on the folios accumulated so far when available memory reaches zero, allowing the released vmemmap pages to support further hugepage allocations. If allocating the HVO head page returns -ENOMEM, the bulk HVO path frees any vmemmap pages it has already accumulated and retries, while callers can continue with an unoptimized folio if the retry still fails. Therefore, neither an allocation warning nor OOM handling helps this best-effort allocation. Use GFP_KERNEL | __GFP_NOWARN | __GFP_RETRY_MAYFAIL to allow reclaim retries while avoiding both. Fixes: c0b495b91a47 ("mm/hugetlb: refactor code around vmemmap_walk") Cc: stable@vger.kernel.org Signed-off-by: Yili Zhang Co-developed-by: Muchun Song Signed-off-by: Muchun Song --- v4: - Split this patch out from patch 4. - After splitting it out, the resulting change was almost identical to the patch at: https://lore.kernel.org/r/20261104133938.50974-1-zhangyili01@baidu.com/ Retain the author of that patch and only rework the subject and commit message. --- mm/hugetlb_vmemmap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index b8ce014ba6e6..7fcc97b12773 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -501,6 +501,7 @@ static int __hugetlb_vmemmap_optimize_folio(const struct hstate *h, { unsigned long vmemmap_start, vmemmap_end; struct page *vmemmap_head, *vmemmap_tail; + gfp_t gfp_mask = GFP_KERNEL | __GFP_NOWARN | __GFP_RETRY_MAYFAIL; int nid, ret = 0; VM_WARN_ON_ONCE_FOLIO(!folio_test_hugetlb(folio), folio); @@ -527,7 +528,7 @@ static int __hugetlb_vmemmap_optimize_folio(const struct hstate *h, */ folio_set_hugetlb_vmemmap_optimized(folio); - vmemmap_head = alloc_pages_node(nid, GFP_KERNEL, 0); + vmemmap_head = alloc_pages_node(nid, gfp_mask, 0); if (!vmemmap_head) { ret = -ENOMEM; goto out; -- 2.54.0