An anonymous write fault on a VMA registered for missing userfaultfd faults currently allocates, zeroes and memcg-charges a folio before checking the PTE and handing the fault to userspace. The folio is then immediately discarded without ever being mapped. Check the PTE and address-space stability under the PTE lock before allocating a folio when missing faults are registered. Deliver the fault to userfaultfd without allocating and discarding a temporary folio. In a four-boot native B/P/B/P comparison, the mean UFFD fault median fell from 13,997.75 ns to 13,093.55 ns (-904.2 ns, -6.46%). A guest allocation trace saw 256 fewer allocations in the faulting task for 256 registered missing faults. Note: this changes OOM ordering. In a fixed 18 MiB memcg test, the baseline OOM-killed only the faulting client before delivering a UFFD event. With this change, the event reached an unprotected shared handler, and its UFFDIO_COPY allocation could trigger an additional OOM kill of the handler. A separate surviving client consequently lost its handler and failed a page-content check after the UFFD registration was released. The same handler-death mechanism was reproducible on the unmodified (mainline) kernel when memory pressure was imposed after event delivery but before UFFDIO_COPY, although the fixed-limit outcomes differed. Signed-off-by: Avi Weiss --- mm/memory.c | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 0fb676d..63b39fa 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5547,6 +5547,22 @@ static vm_fault_t do_anonymous_page(struct vm_fault *vmf) ret = vmf_anon_prepare(vmf); if (ret) return ret; + + if (unlikely(userfaultfd_missing(vma))) { + vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, + vmf->address, &vmf->ptl); + if (!vmf->pte) + goto unlock; + if (vmf_pte_changed(vmf)) { + update_mmu_tlb(vma, vmf->address, vmf->pte); + goto unlock; + } + ret = check_stable_address_space(vma->vm_mm); + if (ret) + goto unlock; + pte_unmap_unlock(vmf->pte, vmf->ptl); + return handle_userfault(vmf, VM_UFFD_MISSING); + } /* Returns NULL on OOM or ERR_PTR(-EAGAIN) if we must retry the fault */ folio = alloc_anon_folio(vmf); if (IS_ERR(folio)) @@ -5579,12 +5595,6 @@ static vm_fault_t do_anonymous_page(struct vm_fault *vmf) if (ret) goto release; - /* Deliver the page fault to userland, check inside PT lock */ - if (userfaultfd_missing(vma)) { - pte_unmap_unlock(vmf->pte, vmf->ptl); - folio_put(folio); - return handle_userfault(vmf, VM_UFFD_MISSING); - } map_anon_folio_pte_pf(folio, vmf->pte, vma, addr, vmf_orig_pte_uffd_wp(vmf)); unlock: -- 2.43.0