From: "Kiryl Shutsemau (Meta)" PAGEMAP_SCAN reports an unpopulated pte differently depending on which path serves the request. The PAGE_IS_WRITTEN fast path in pagemap_scan_pmd_entry() reports a pte_none as written (and, under PM_SCAN_WP_MATCHING, arms a marker); pagemap_page_category() returns 0 for the same pte_none. A request that cannot take the fast path (an extra category bit, category_anyof_mask or category_inverted) therefore reports the pte as clean and skips arming it. A range that was populated and then MADV_DONTNEED'd reads as written via one mask and clean via another, and in the latter case is not re-armed for the next round -- an incremental-dump consumer (e.g. CRIU) using a richer mask drops the zapped range and stops tracking writes to it. Report pte_none as written in pagemap_page_category() too. A pte_none carries no uffd-wp marker, i.e. it is not write-protected -- the same condition under which the present and swap cases already report PAGE_IS_WRITTEN. The fast path applies no VMA test, so neither does this. The hugetlb and fully-unpopulated-PMD (no page table) scans have no PAGE_IS_WRITTEN fast path, so they do not exhibit the per-entry divergence and are left unchanged. Add a pagemap_ioctl selftest that populates a range, drops it with MADV_DONTNEED, and checks that the fast path and the generic (category_anyof_mask) path both report every page written. Fixes: 12f6b01a0bcb ("fs/proc/task_mmu: add fast paths to get/clear PAGE_IS_WRITTEN flag") Cc: Muhammad Usama Anjum Cc: Peter Xu Cc: stable@vger.kernel.org Signed-off-by: Kiryl Shutsemau Assisted-by: Claude:claude-fable-5 --- Changes since v1 [1], addressing Sashiko AI review: - Report pte_none unconditionally instead of only on userfaultfd_wp VMAs. v1 left the fast vs generic path mismatch in place for non-uffd-wp VMAs, where the fast path applies no VMA test. - Strengthen the selftest to assert both paths report every page written (v1 only checked the two counts were equal, which passes vacuously if both are 0). The hugetlb and fully-unpopulated-PMD scans have no PAGE_IS_WRITTEN fast path, so they stay self-consistent and are out of scope here. [1] https://lore.kernel.org/all/20260706104308.34741-1-kirill@shutemov.name/ fs/proc/task_mmu.c | 14 +++++- tools/testing/selftests/mm/pagemap_ioctl.c | 56 +++++++++++++++++++++- 2 files changed, 67 insertions(+), 3 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index d32408f7cd5e..d45c729ab6bb 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -2432,8 +2432,18 @@ static unsigned long pagemap_page_category(struct pagemap_scan_private *p, { unsigned long categories; - if (pte_none(pte)) - return 0; + if (pte_none(pte)) { + /* + * An unpopulated pte carries no uffd-wp marker, i.e. it is not + * write-protected, the same condition under which the present + * and swap cases below report PAGE_IS_WRITTEN. Report it here + * too so this generic path agrees with the PAGE_IS_WRITTEN fast + * path in pagemap_scan_pmd_entry(), which reports pte_none as + * written and, under PM_SCAN_WP_MATCHING, arms a marker. The + * fast path applies no VMA test, so neither does this. + */ + return PAGE_IS_WRITTEN; + } if (pte_present(pte)) { struct page *page; diff --git a/tools/testing/selftests/mm/pagemap_ioctl.c b/tools/testing/selftests/mm/pagemap_ioctl.c index 762306177ad8..550d1f2c059b 100644 --- a/tools/testing/selftests/mm/pagemap_ioctl.c +++ b/tools/testing/selftests/mm/pagemap_ioctl.c @@ -1051,6 +1051,57 @@ static void test_simple(void) ksft_test_result(i == TEST_ITERATIONS, "Test %s\n", __func__); } +/* + * A range that was populated and then MADV_DONTNEED'd is genuine pte_none + * with no uffd-wp marker. Such a pte must read the same regardless of which + * PAGEMAP_SCAN path serves the request: both the PAGE_IS_WRITTEN fast path and + * the generic path (reached e.g. via category_anyof_mask) must report every + * page written. + */ +static void unpopulated_scan_test(void) +{ + int npages = 16, i; + long mem_size = npages * page_size; + struct page_region regions[16]; + long fast = 0, slow = 0, ret; + char *mem; + + mem = mmap(NULL, mem_size, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (mem == MAP_FAILED) + ksft_exit_fail_msg("%s mmap failed\n", __func__); + + wp_init(mem, mem_size); + + /* Populate, then drop: the ptes become pte_none without a marker. */ + memset(mem, 1, mem_size); + if (madvise(mem, mem_size, MADV_DONTNEED)) + ksft_exit_fail_msg("%s MADV_DONTNEED failed\n", __func__); + + /* Fast path: category_mask == return_mask == PAGE_IS_WRITTEN. */ + ret = pagemap_ioctl(mem, mem_size, regions, npages, 0, 0, + PAGE_IS_WRITTEN, 0, 0, PAGE_IS_WRITTEN); + if (ret < 0) + ksft_exit_fail_msg("%s fast scan failed\n", __func__); + for (i = 0; i < ret; i++) + fast += LEN(regions[i]); + + /* Generic path: same query expressed via category_anyof_mask. */ + ret = pagemap_ioctl(mem, mem_size, regions, npages, 0, 0, + 0, PAGE_IS_WRITTEN, 0, PAGE_IS_WRITTEN); + if (ret < 0) + ksft_exit_fail_msg("%s generic scan failed\n", __func__); + for (i = 0; i < ret; i++) + slow += LEN(regions[i]); + + ksft_test_result(fast == npages && slow == npages, + "%s unpopulated ptes reported written by both paths (%ld, %ld of %d)\n", + __func__, fast, slow, npages); + + wp_free(mem, mem_size); + munmap(mem, mem_size); +} + int sanity_tests(void) { unsigned long long mem_size, vec_size; @@ -1559,7 +1610,7 @@ int main(int __attribute__((unused)) argc, char *argv[]) if (!hugetlb_setup_default(4)) ksft_print_msg("HugeTLB test will be skipped\n"); - ksft_set_plan(117); + ksft_set_plan(118); page_size = getpagesize(); hpage_size = read_pmd_pagesize(); @@ -1737,6 +1788,9 @@ int main(int __attribute__((unused)) argc, char *argv[]) /* 17. ZEROPFN tests */ zeropfn_tests(); + /* 18. Unpopulated pte scan-path consistency */ + unpopulated_scan_test(); + close(pagemap_fd); ksft_finished(); } base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482 -- 2.54.0