From: "Kiryl Shutsemau (Meta)" PAGEMAP_SCAN reports an unpopulated PTE in a uffd-wp VMA as written (pagemap_page_category() and the PAGE_IS_WRITTEN fast path), but a range with no page table at all -- a PMD hole -- is skipped. pagemap_scan_pte_hole() evaluates the hole against p->cur_vma_category, which pagemap_scan_test_walk() builds from only PAGE_IS_WPALLOWED and PAGE_IS_SOFT_DIRTY, so PAGE_IS_WRITTEN is never set: the hole is neither reported nor, under PM_SCAN_WP_MATCHING, armed. This is reachable. An anonymous THP is write-protected in place as a huge PMD (change_huge_pmd(), anon is not split), and a full-PMD MADV_DONTNEED clears it to pmd_none. A WP-async consumer such as CRIU then misses the 2MB drop -- the range is not reported written and the next incremental dump keeps stale data. (A file/shmem THP is split on write-protect, so a later DONTNEED leaves a populated page table of pte_none entries, which are already reported; only anon THP reaches the hole path.) Add PAGE_IS_WRITTEN to the categories evaluated for a hole in a non-hugetlb uffd-wp VMA, matching the pte_none handling in pagemap_page_category(). The existing PM_SCAN_WP_MATCHING path then also arms the range: uffd_wp_range() allocates the page table and installs markers under WP_UNPOPULATED, so the next scan sees it clean until re-written. hugetlb is excluded on purpose: an allocated-but-empty huge entry reads as not-written via pagemap_hugetlb_category(), so reporting an unallocated hugetlb hole (which also reaches this path) as written would be inconsistent within the same VMA. hugetlb hole handling is left as-is. Add a pagemap_ioctl selftest that forms an anon THP, drops it with MADV_DONTNEED and checks the resulting PMD hole is reported written. Fixes: 2bad466cc9d9 ("mm/uffd: UFFD_FEATURE_WP_UNPOPULATED") Cc: Muhammad Usama Anjum Cc: Peter Xu Cc: stable@vger.kernel.org Signed-off-by: Kiryl Shutsemau Assisted-by: Claude:claude-fable-5 --- Changes since v1 [1], addressing Sashiko AI review: - Exclude hugetlb from the hole PAGE_IS_WRITTEN report. An unallocated hugetlb slot also reaches pagemap_scan_pte_hole(), and reporting it written while an allocated-but-empty huge entry reads not-written (pagemap_hugetlb_category()) would be inconsistent within the VMA. This also keeps the pre-existing pagemap_scan_hugetlb_hole_wp() range concern out of scope here. Based on the pte_none fix [2] (same selftest file). [1] https://lore.kernel.org/all/20260708093444.145566-1-kirill@shutemov.name/ [2] https://lore.kernel.org/all/20260707151349.92143-1-kirill@shutemov.name/ fs/proc/task_mmu.c | 25 ++++++++- tools/testing/selftests/mm/pagemap_ioctl.c | 60 +++++++++++++++++++++- 2 files changed, 82 insertions(+), 3 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index d45c729ab6bb..fad648f9a40c 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -3049,12 +3049,33 @@ static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end, { struct pagemap_scan_private *p = walk->private; struct vm_area_struct *vma = walk->vma; + unsigned long categories; int ret, err; - if (!vma || !pagemap_scan_is_interesting_page(p->cur_vma_category, p)) + if (!vma) return 0; - ret = pagemap_scan_output(p->cur_vma_category, p, addr, &end); + /* + * An unpopulated range with no page table -- e.g. a 2MB anon THP + * dropped via MADV_DONTNEED, which pagemap_page_category() never sees + * -- reads as written on a uffd-wp VMA, matching the pte_none case + * there. Reporting it also lets the PM_SCAN_WP_MATCHING arming below + * install markers (uffd_wp_range() allocates the page table under + * WP_UNPOPULATED), so the next scan sees it clean until re-written. + * + * hugetlb is excluded: an allocated-but-empty huge entry reads as + * not-written via pagemap_hugetlb_category(), so reporting an + * unallocated hugetlb hole as written here would be inconsistent + * within the same VMA. + */ + categories = p->cur_vma_category; + if (userfaultfd_wp(vma) && !is_vm_hugetlb_page(vma)) + categories |= PAGE_IS_WRITTEN; + + if (!pagemap_scan_is_interesting_page(categories, p)) + return 0; + + ret = pagemap_scan_output(categories, p, addr, &end); if (addr == end) return ret; diff --git a/tools/testing/selftests/mm/pagemap_ioctl.c b/tools/testing/selftests/mm/pagemap_ioctl.c index 550d1f2c059b..c0e45d0f7478 100644 --- a/tools/testing/selftests/mm/pagemap_ioctl.c +++ b/tools/testing/selftests/mm/pagemap_ioctl.c @@ -25,6 +25,10 @@ #include "kselftest.h" #include "hugepage_settings.h" +#ifndef MADV_COLLAPSE +#define MADV_COLLAPSE 25 +#endif + #define PAGEMAP_BITS_ALL (PAGE_IS_WPALLOWED | PAGE_IS_WRITTEN | \ PAGE_IS_FILE | PAGE_IS_PRESENT | \ PAGE_IS_SWAPPED | PAGE_IS_PFNZERO | \ @@ -1102,6 +1106,59 @@ static void unpopulated_scan_test(void) munmap(mem, mem_size); } +/* + * A 2MB anon THP dropped with MADV_DONTNEED leaves a pmd_none hole with no + * page table, which pagemap_page_category() never sees. PAGEMAP_SCAN must + * still report it as written on a uffd-wp VMA, via pagemap_scan_pte_hole(). + */ +static void unpopulated_thp_hole_test(void) +{ + long npages, written = 0, ret, i; + struct page_region regions[16]; + char *area, *mem; + + if (!hpage_size) { + ksft_test_result_skip("%s THP not supported\n", __func__); + return; + } + npages = hpage_size / page_size; + + /* Get a PMD-aligned range so the range can be a single THP. */ + area = mmap(NULL, 2 * hpage_size, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (area == MAP_FAILED) + ksft_exit_fail_msg("%s mmap failed\n", __func__); + mem = (char *)(((unsigned long)area + hpage_size - 1) & ~(hpage_size - 1)); + + memset(mem, 1, hpage_size); + if (madvise(mem, hpage_size, MADV_COLLAPSE) || + !check_huge_anon(mem, 1, hpage_size)) { + ksft_test_result_skip("%s could not form a THP\n", __func__); + munmap(area, 2 * hpage_size); + return; + } + + wp_init(mem, hpage_size); + + /* Drop the whole PMD: it is cleared to a pmd_none hole. */ + if (madvise(mem, hpage_size, MADV_DONTNEED)) + ksft_exit_fail_msg("%s MADV_DONTNEED failed\n", __func__); + + ret = pagemap_ioctl(mem, hpage_size, regions, 16, 0, 0, + PAGE_IS_WRITTEN, 0, 0, PAGE_IS_WRITTEN); + if (ret < 0) + ksft_exit_fail_msg("%s scan failed\n", __func__); + for (i = 0; i < ret; i++) + written += LEN(regions[i]); + + ksft_test_result(written == npages, + "%s pmd-hole reported written (%ld of %ld)\n", + __func__, written, npages); + + wp_free(mem, hpage_size); + munmap(area, 2 * hpage_size); +} + int sanity_tests(void) { unsigned long long mem_size, vec_size; @@ -1610,7 +1667,7 @@ int main(int __attribute__((unused)) argc, char *argv[]) if (!hugetlb_setup_default(4)) ksft_print_msg("HugeTLB test will be skipped\n"); - ksft_set_plan(118); + ksft_set_plan(119); page_size = getpagesize(); hpage_size = read_pmd_pagesize(); @@ -1790,6 +1847,7 @@ int main(int __attribute__((unused)) argc, char *argv[]) /* 18. Unpopulated pte scan-path consistency */ unpopulated_scan_test(); + unpopulated_thp_hole_test(); close(pagemap_fd); ksft_finished(); base-commit: 9795ad96d277c4af049fe30de1cebd4e39d7bcbe -- 2.54.0