| Seq | Timestamp | Type | Name | Duration |
|---|---|---|---|---|
| 0/0 | 2026/08/12 14:59 | flow | patch-triage |
0mArgs:null Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"KMSANReasoning": "",
"NeedsKMSAN": false,
"Reasoning": "The patch only modifies userspace selftests in tools/testing/selftests/mm/, which do not affect the kernel's functional execution flow.",
"WorthFuzzing": false
} |
| 1/1 | 2026/08/12 14:59 | action | read-patch-diff |
0mArgs:null Results: {
"PatchDiff": "commit 67a19e17a91b615d0071c3d8410382e81c0e9088\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Wed Aug 12 14:59:10 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile\nindex 2d5366196e309..308bbad73c11a 100644\n--- a/tools/testing/selftests/mm/Makefile\n+++ b/tools/testing/selftests/mm/Makefile\n@@ -104,6 +104,9 @@ TEST_GEN_FILES += guard-regions\n TEST_GEN_FILES += merge\n TEST_GEN_FILES += rmap\n TEST_GEN_FILES += folio_split_race_test\n+TEST_GEN_FILES += folio_order_check\n+TEST_GEN_FILES += khugepaged_sync_check\n+TEST_GEN_FILES += khugepaged_race\n \n ifneq ($(ARCH),arm64)\n TEST_GEN_FILES += soft-dirty\ndiff --git a/tools/testing/selftests/mm/folio_order_check.c b/tools/testing/selftests/mm/folio_order_check.c\nnew file mode 100644\nindex 0000000000000..93030a42c3ccd\n--- /dev/null\n+++ b/tools/testing/selftests/mm/folio_order_check.c\n@@ -0,0 +1,137 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Self-check for the vm_util folio-order detection helpers,\n+ * is_backed_by_folio() and is_range_backed_by_folio_orders().\n+ *\n+ * For every anon THP order the kernel supports, fault memory in with only\n+ * that order enabled and verify the helpers report exactly that order:\n+ * not a neighbouring order, and plain 4K memory as order 0. The helpers\n+ * are what the khugepaged mTHP tests use to detect collapse results, so\n+ * they must agree with the kernel's own idea of the backing before any\n+ * collapse test relies on them.\n+ */\n+#define _GNU_SOURCE\n+#include \u003cfcntl.h\u003e\n+#include \u003cstdio.h\u003e\n+#include \u003cstdlib.h\u003e\n+#include \u003csys/mman.h\u003e\n+#include \u003cunistd.h\u003e\n+\n+#include \"kselftest.h\"\n+#include \"vm_util.h\"\n+#include \"hugepage_settings.h\"\n+\n+static int pagemap_fd;\n+static int kpageflags_fd;\n+\n+/* mmap an anon VMA of exactly @size bytes at a @size-aligned address. */\n+static char *alloc_aligned(size_t size)\n+{\n+\tsize_t len = size * 2;\n+\tuintptr_t aligned;\n+\tchar *p;\n+\n+\tp = mmap(NULL, len, PROT_READ | PROT_WRITE,\n+\t\t MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);\n+\tif (p == MAP_FAILED)\n+\t\tksft_exit_fail_perror(\"mmap()\");\n+\n+\taligned = ALIGN((uintptr_t)p, size);\n+\tif (aligned != (uintptr_t)p)\n+\t\tmunmap(p, aligned - (uintptr_t)p);\n+\tif (aligned + size != (uintptr_t)p + len)\n+\t\tmunmap((char *)aligned + size,\n+\t\t (uintptr_t)p + len - aligned - size);\n+\n+\treturn (char *)aligned;\n+}\n+\n+/*\n+ * Enable only @order (order 0: nothing), fault one aligned window in and\n+ * check the helpers see exactly @order.\n+ */\n+static void check_order(int order)\n+{\n+\tstruct thp_settings settings = *thp_current_settings();\n+\tsize_t size = psize() \u003c\u003c order;\n+\tbool ok = true;\n+\tchar *p;\n+\tint i;\n+\n+\tfor (i = 0; i \u003c NR_ORDERS; i++)\n+\t\tsettings.hugepages[i].enabled = THP_NEVER;\n+\tif (order)\n+\t\tsettings.hugepages[order].enabled = THP_ALWAYS;\n+\tthp_push_settings(\u0026settings);\n+\n+\tp = alloc_aligned(size);\n+\t*p = 1;\n+\n+\tif (!is_range_backed_by_folio_orders(p, size, order,\n+\t\t\t\t\t pagemap_fd, kpageflags_fd)) {\n+\t\tksft_print_msg(\"order %d not detected after fault\\n\", order);\n+\t\tok = false;\n+\t}\n+\n+\t/* A lower order must be rejected: the folio is larger. */\n+\tif (order \u0026\u0026 is_range_backed_by_folio_orders(p, size, order - 1,\n+\t\t\t\t\t\t pagemap_fd,\n+\t\t\t\t\t\t kpageflags_fd)) {\n+\t\tksft_print_msg(\"order %d also reported as order %d\\n\",\n+\t\t\t order, order - 1);\n+\t\tok = false;\n+\t}\n+\n+\t/* Order 0 pages must not look like any large folio, and vice versa. */\n+\tif (order \u0026\u0026 is_range_backed_by_folio_orders(p, size, 0,\n+\t\t\t\t\t\t pagemap_fd,\n+\t\t\t\t\t\t kpageflags_fd)) {\n+\t\tksft_print_msg(\"order %d also reported as order 0\\n\", order);\n+\t\tok = false;\n+\t}\n+\n+\tmunmap(p, size);\n+\tthp_pop_settings();\n+\n+\tksft_test_result(ok, \"order %d classified\\n\", order);\n+}\n+\n+int main(void)\n+{\n+\tstruct thp_settings settings;\n+\tunsigned long orders;\n+\tint order;\n+\n+\tksft_print_header();\n+\n+\tif (!thp_available())\n+\t\tksft_exit_skip(\"Transparent Hugepages not available\\n\");\n+\n+\tpagemap_fd = open(\"/proc/self/pagemap\", O_RDONLY);\n+\tif (pagemap_fd \u003c 0)\n+\t\tksft_exit_fail_perror(\"open(/proc/self/pagemap)\");\n+\tkpageflags_fd = open(\"/proc/kpageflags\", O_RDONLY);\n+\tif (kpageflags_fd \u003c 0)\n+\t\tksft_exit_skip(\"open(\\\"/proc/kpageflags\\\") requires root\\n\");\n+\n+\torders = thp_supported_orders();\n+\tif (!orders)\n+\t\tksft_exit_skip(\"No supported THP orders\\n\");\n+\n+\tksft_set_plan(__builtin_popcountl(orders) + 1);\n+\n+\tthp_save_settings();\n+\tthp_read_settings(\u0026settings);\n+\t/* Base of the settings stack; the bottom entry is never popped. */\n+\tthp_push_settings(\u0026settings);\n+\n+\tcheck_order(0);\n+\tfor (order = 1; order \u003c NR_ORDERS; order++) {\n+\t\tif (!(orders \u0026 (1UL \u003c\u003c order)))\n+\t\t\tcontinue;\n+\t\tcheck_order(order);\n+\t}\n+\n+\n+\tksft_finished();\n+}\ndiff --git a/tools/testing/selftests/mm/hmm-tests.c b/tools/testing/selftests/mm/hmm-tests.c\nindex e2642eca0d02b..df426f9218e71 100644\n--- a/tools/testing/selftests/mm/hmm-tests.c\n+++ b/tools/testing/selftests/mm/hmm-tests.c\n@@ -65,7 +65,6 @@ enum {\n #define HMM_PATH_MAX 64\n #define NTIMES\t\t10\n \n-#define ALIGN(x, a) (((x) + (a - 1)) \u0026 (~((a) - 1)))\n /* Just the flags we need, copied from mm.h: */\n \n #ifndef FOLL_WRITE\ndiff --git a/tools/testing/selftests/mm/hugepage_settings.c b/tools/testing/selftests/mm/hugepage_settings.c\nindex d7917dce3abac..8afcdf9793bb7 100644\n--- a/tools/testing/selftests/mm/hugepage_settings.c\n+++ b/tools/testing/selftests/mm/hugepage_settings.c\n@@ -183,6 +183,17 @@ void thp_read_settings(struct thp_settings *settings)\n \t}\n }\n \n+/*\n+ * Write only on change: any store to a khugepaged sysfs knob wakes the\n+ * daemon, and settings pushes/pops must not start scan passes nobody\n+ * asked for -- khugepaged_full_pass() is the only sanctioned wake.\n+ */\n+void thp_update_num(const char *name, unsigned long num)\n+{\n+\tif (thp_read_num(name) != num)\n+\t\tthp_write_num(name, num);\n+}\n+\n void thp_write_settings(struct thp_settings *settings)\n {\n \tstruct khugepaged_settings *khugepaged = \u0026settings-\u003ekhugepaged;\n@@ -198,15 +209,15 @@ void thp_write_settings(struct thp_settings *settings)\n \t\t\tshmem_enabled_strings[settings-\u003eshmem_enabled]);\n \tthp_write_num(\"use_zero_page\", settings-\u003euse_zero_page);\n \n-\tthp_write_num(\"khugepaged/defrag\", khugepaged-\u003edefrag);\n-\tthp_write_num(\"khugepaged/alloc_sleep_millisecs\",\n-\t\t\tkhugepaged-\u003ealloc_sleep_millisecs);\n-\tthp_write_num(\"khugepaged/scan_sleep_millisecs\",\n-\t\t\tkhugepaged-\u003escan_sleep_millisecs);\n-\tthp_write_num(\"khugepaged/max_ptes_none\", khugepaged-\u003emax_ptes_none);\n-\tthp_write_num(\"khugepaged/max_ptes_swap\", khugepaged-\u003emax_ptes_swap);\n-\tthp_write_num(\"khugepaged/max_ptes_shared\", khugepaged-\u003emax_ptes_shared);\n-\tthp_write_num(\"khugepaged/pages_to_scan\", khugepaged-\u003epages_to_scan);\n+\tthp_update_num(\"khugepaged/defrag\", khugepaged-\u003edefrag);\n+\tthp_update_num(\"khugepaged/alloc_sleep_millisecs\",\n+\t\t khugepaged-\u003ealloc_sleep_millisecs);\n+\tthp_update_num(\"khugepaged/scan_sleep_millisecs\",\n+\t\t khugepaged-\u003escan_sleep_millisecs);\n+\tthp_update_num(\"khugepaged/max_ptes_none\", khugepaged-\u003emax_ptes_none);\n+\tthp_update_num(\"khugepaged/max_ptes_swap\", khugepaged-\u003emax_ptes_swap);\n+\tthp_update_num(\"khugepaged/max_ptes_shared\", khugepaged-\u003emax_ptes_shared);\n+\tthp_update_num(\"khugepaged/pages_to_scan\", khugepaged-\u003epages_to_scan);\n \n \tif (dev_queue_read_ahead_path[0])\n \t\twrite_num(dev_queue_read_ahead_path, settings-\u003eread_ahead_kb);\n@@ -230,6 +241,49 @@ void thp_write_settings(struct thp_settings *settings)\n \t}\n }\n \n+/*\n+ * Completion barrier for khugepaged: wait until a full scan pass that\n+ * started after this call has finished. full_scans must advance by two;\n+ * a +1 step may complete a pass that examined this mm before the\n+ * caller's setup was in place.\n+ *\n+ * Any store to scan_sleep_millisecs wakes the daemon, so the barrier works\n+ * whatever the configured scan cadence -- but a store can be lost.\n+ * __sleep_millisecs_store() clears khugepaged_sleep_expire and wakes the\n+ * queue; if the daemon is between scans rather than sleeping, it sets\n+ * khugepaged_sleep_expire itself on the way into khugepaged_wait_work() and\n+ * then sleeps for the full interval, having never seen the store. So keep\n+ * storing until the pass lands; a store while the daemon is awake costs\n+ * nothing and does not queue an extra pass.\n+ *\n+ * One wake completes one full pass only if the whole mm list fits in\n+ * one scan batch, so callers must pair this with a large\n+ * pages_to_scan.\n+ */\n+bool khugepaged_full_pass(unsigned int timeout_s)\n+{\n+\tunsigned long deadline_ms = timeout_s * 1000UL;\n+\tunsigned long sleep_ms =\n+\t\tthp_read_num(\"khugepaged/scan_sleep_millisecs\");\n+\tunsigned long elapsed_ms = 0;\n+\tint pass;\n+\n+\tfor (pass = 0; pass \u003c 2; pass++) {\n+\t\tunsigned long target =\n+\t\t\tthp_read_num(\"khugepaged/full_scans\") + 1;\n+\n+\t\twhile (thp_read_num(\"khugepaged/full_scans\") \u003c target) {\n+\t\t\tif (elapsed_ms \u003e= deadline_ms)\n+\t\t\t\treturn false;\n+\t\t\tthp_write_num(\"khugepaged/scan_sleep_millisecs\",\n+\t\t\t\t sleep_ms);\n+\t\t\tusleep(10 * 1000);\n+\t\t\telapsed_ms += 10;\n+\t\t}\n+\t}\n+\treturn true;\n+}\n+\n struct thp_settings *thp_current_settings(void)\n {\n \tif (!settings_index) {\ndiff --git a/tools/testing/selftests/mm/hugepage_settings.h b/tools/testing/selftests/mm/hugepage_settings.h\nindex 726c73c43c05b..ba7d38370d433 100644\n--- a/tools/testing/selftests/mm/hugepage_settings.h\n+++ b/tools/testing/selftests/mm/hugepage_settings.h\n@@ -70,6 +70,7 @@ int thp_read_string(const char *name, const char * const strings[]);\n void thp_write_string(const char *name, const char *val);\n unsigned long thp_read_num(const char *name);\n void thp_write_num(const char *name, unsigned long num);\n+void thp_update_num(const char *name, unsigned long num);\n \n void thp_write_settings(struct thp_settings *settings);\n void thp_read_settings(struct thp_settings *settings);\n@@ -83,6 +84,8 @@ static inline void thp_save_settings(void)\n \thugepage_save_settings(/* thp = */ true, /* hugetlb = */ false);\n }\n \n+bool khugepaged_full_pass(unsigned int timeout_s);\n+\n void thp_set_read_ahead_path(char *path);\n unsigned long thp_supported_orders(void);\n unsigned long thp_shmem_supported_orders(void);\ndiff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c\nindex 0d6c71ed2fae3..f5773d4475427 100644\n--- a/tools/testing/selftests/mm/khugepaged.c\n+++ b/tools/testing/selftests/mm/khugepaged.c\n@@ -31,6 +31,11 @@ static unsigned long page_size;\n static int hpage_pmd_nr;\n static int anon_order;\n static int collapse_order;\n+static bool collapse_order_given;\n+static int collapse_orders[NR_ORDERS];\n+static int nr_collapse_orders;\n+static int pagemap_fd = -1;\n+static int kpageflags_fd = -1;\n \n #define PID_SMAPS \"/proc/self/smaps\"\n #define TEST_FILE \"collapse_test_file\"\n@@ -241,6 +246,41 @@ static bool check_swap(void *addr, unsigned long size)\n \treturn swap;\n }\n \n+/*\n+ * Page the range out and wait for the swap count to say so.\n+ *\n+ * Two things get in the way. MADV_PAGEOUT is best effort:\n+ * shrink_folio_list() leaves a folio alone when it cannot reclaim it right\n+ * away, and one still under writeback from an earlier pageout is the common\n+ * case, so the count the caller asks for arrives a moment later. And a range\n+ * an earlier collapse left MADV_HUGEPAGE is one khugepaged is still working\n+ * on: collapsing a range with up to max_ptes_swap pages swapped out means\n+ * reading those pages back in, so the daemon undoes the pageout as fast as it\n+ * is asked for. Keep the range out of its reach; the collapse the caller runs\n+ * next puts MADV_HUGEPAGE back.\n+ *\n+ * Failing to get the pages out is the machine's answer, not the kernel's --\n+ * swap too small, swap full, a memcg cap, a folio still under writeback -- so\n+ * callers skip rather than fail. An error from madvise() is different, and\n+ * ends the run here.\n+ */\n+static bool swapout_range(void *p, unsigned long size)\n+{\n+\tint i;\n+\n+\tif (madvise(p, size, MADV_NOHUGEPAGE))\n+\t\tksft_exit_fail_perror(\"madvise(MADV_NOHUGEPAGE)\");\n+\n+\tfor (i = 0; i \u003c 40; i++) {\n+\t\tif (madvise(p, size, MADV_PAGEOUT))\n+\t\t\tksft_exit_fail_perror(\"madvise(MADV_PAGEOUT)\");\n+\t\tif (check_swap(p, size))\n+\t\t\treturn true;\n+\t\tusleep(50 * 1000);\n+\t}\n+\treturn false;\n+}\n+\n static void *alloc_mapping(int nr)\n {\n \tvoid *p;\n@@ -583,8 +623,10 @@ static bool wait_for_scan(const char *msg, char *p, size_t len,\n \t\tint nr_hpages, int collap_order, struct mem_ops *ops)\n {\n \tunsigned long hpage_size = page_size \u003c\u003c collap_order;\n+\t/* Three seconds as a floor, plus a second per 128M to collapse */\n+\tconst unsigned long bytes = (unsigned long)nr_hpages * hpage_size;\n+\tint timeout = 6 + 2 * (bytes / (128UL \u003c\u003c 20));\n \tint full_scans;\n-\tint timeout = 6; /* 3 seconds */\n \n \t/* Sanity check */\n \tif (!ops-\u003echeck_huge(p, len, 0, hpage_size))\n@@ -853,12 +895,10 @@ static void collapse_swapin_single_pte(struct collapse_context *c, struct mem_op\n \tp = ops-\u003esetup_area(1);\n \tops-\u003efault(p, 0, hpage_pmd_size);\n \n-\tif (madvise(p, page_size, MADV_PAGEOUT))\n-\t\tksft_exit_fail_perror(\"madvise(MADV_PAGEOUT)\");\n-\tif (check_swap(p, page_size)) {\n+\tif (swapout_range(p, page_size)) {\n \t\tsuccess(\"OK\");\n \t} else {\n-\t\tfail(\"Fail\");\n+\t\tskip(\"Could not swap out\");\n \t\tgoto out;\n \t}\n \n@@ -885,12 +925,10 @@ static void collapse_max_ptes_swap(struct collapse_context *c, struct mem_ops *o\n \tp = ops-\u003esetup_area(1);\n \tops-\u003efault(p, 0, hpage_pmd_size);\n \n-\tif (madvise(p, (max_ptes_swap + 1) * page_size, MADV_PAGEOUT))\n-\t\tksft_exit_fail_perror(\"madvise(MADV_PAGEOUT)\");\n-\tif (check_swap(p, (max_ptes_swap + 1) * page_size)) {\n+\tif (swapout_range(p, (max_ptes_swap + 1) * page_size)) {\n \t\tsuccess(\"OK\");\n \t} else {\n-\t\tfail(\"Fail\");\n+\t\tskip(\"Could not swap out\");\n \t\tgoto out;\n \t}\n \n@@ -902,12 +940,10 @@ static void collapse_max_ptes_swap(struct collapse_context *c, struct mem_ops *o\n \t\tops-\u003efault(p, 0, hpage_pmd_size);\n \t\tksft_print_msg(\"Swapout %d of %d pages...\", max_ptes_swap,\n \t\t hpage_pmd_nr);\n-\t\tif (madvise(p, max_ptes_swap * page_size, MADV_PAGEOUT))\n-\t\t\tksft_exit_fail_perror(\"madvise(MADV_PAGEOUT)\");\n-\t\tif (check_swap(p, max_ptes_swap * page_size)) {\n+\t\tif (swapout_range(p, max_ptes_swap * page_size)) {\n \t\t\tsuccess(\"OK\");\n \t\t} else {\n-\t\t\tfail(\"Fail\");\n+\t\t\tskip(\"Could not swap out\");\n \t\t\tgoto out;\n \t\t}\n \n@@ -974,6 +1010,16 @@ static void collapse_compound_extreme(struct collapse_context *c, struct mem_ops\n \tvoid *p;\n \tint i;\n \n+\t/*\n+\t * The test needs hpage_pmd_nr PMD-order allocations, which is likely to\n+\t * fail for large PMD sizes. Skip if the PMD size is over 32M.\n+\t */\n+\tif (hpage_pmd_size \u003e (32UL \u003c\u003c 20)) {\n+\t\tksft_test_result_skip(\"%s: PMD too large for fault-time THP construction\\n\",\n+\t\t\t\t __func__);\n+\t\treturn;\n+\t}\n+\n \tp = ops-\u003esetup_area(1);\n \tksft_print_msg(\"Construct PTE page table full of different PTE-mapped compound pages\\n\");\n \tfor (i = 0; i \u003c hpage_pmd_nr; i++) {\n@@ -1164,6 +1210,72 @@ static void collapse_max_ptes_shared(struct collapse_context *c, struct mem_ops\n \tksft_test_result_report(exit_status, \"%s\\n\", __func__);\n }\n \n+/*\n+ * Content stays isolated while a co-sharer writes concurrently. A shared\n+ * source is copied live (not frozen), relying on it being CoW - immutable\n+ * for the duration of the copy; a co-sharer's write goes to a CoW copy. The\n+ * collapsing child must see the pre-fork content, the writing parent only\n+ * its own writes.\n+ */\n+static void collapse_fork_cow_race(struct collapse_context *c, struct mem_ops *ops)\n+{\n+\tconst unsigned long shared = 64 * page_size;\n+\tconst int stride = page_size / sizeof(int);\n+\tint wstatus, child_status, i, n = shared / page_size;\n+\t/* volatile: the loop below must really store, on every iteration */\n+\tvolatile int *ip;\n+\tvoid *p;\n+\n+\tp = ops-\u003esetup_area(1);\n+\tip = p;\n+\tops-\u003efault(p, 0, shared);\t\t/* shared prefix, pre-fork pattern */\n+\n+\tksft_print_msg(\"Fork, collapse in the child while the parent rewrites...\");\n+\tif (!fork()) {\n+\t\tint collapse_status;\n+\n+\t\tops-\u003efault(p, shared, hpage_pmd_size);\t/* private remainder */\n+\t\tc-\u003ecollapse(\"Collapse a range shared with a writing co-sharer\",\n+\t\t\t p, 1, ops, true);\n+\t\tcollapse_status = exit_status;\n+\t\tfor (i = 0; i \u003c n; i++)\n+\t\t\tif (ip[i * stride] != i + 0xdead0000)\n+\t\t\t\tbreak;\n+\t\tif (i == n)\n+\t\t\tsuccess(\"OK\");\n+\t\telse\n+\t\t\tfail(\"Fail: child content\");\n+\t\t/* The content check must not bury a failed collapse. */\n+\t\tif (exit_status != KSFT_FAIL)\n+\t\t\texit_status = collapse_status;\n+\t\tops-\u003ecleanup_area(p, hpage_pmd_size);\n+\t\t_exit(exit_status);\n+\t}\n+\n+\t/* Hammer the parent's own writes over the shared prefix. */\n+\tfor (int it = 0; it \u003c 200000; it++)\n+\t\tfor (i = 0; i \u003c n; i++)\n+\t\t\tip[i * stride] = i + 0xbeef0000;\n+\n+\twait(\u0026wstatus);\n+\t/* A child that died reading the racing pages is a failure, not a zero. */\n+\tchild_status = WIFEXITED(wstatus) ? WEXITSTATUS(wstatus) : KSFT_FAIL;\n+\n+\tksft_print_msg(\"Check the parent sees only its own writes...\");\n+\tfor (i = 0; i \u003c n; i++)\n+\t\tif (ip[i * stride] != i + 0xbeef0000)\n+\t\t\tbreak;\n+\tif (i == n)\n+\t\tsuccess(\"OK\");\n+\telse\n+\t\tfail(\"Fail: parent content\");\n+\tops-\u003ecleanup_area(p, hpage_pmd_size);\n+\t/* Same again: our own check must not bury the child's verdict. */\n+\tif (exit_status != KSFT_FAIL)\n+\t\texit_status = child_status;\n+\tksft_test_result_report(exit_status, \"%s\\n\", __func__);\n+}\n+\n static void madvise_collapse_existing_thps(struct collapse_context *c,\n \t\t\t\t\t struct mem_ops *ops)\n {\n@@ -1209,6 +1321,219 @@ static void madvise_retracted_page_tables(struct collapse_context *c,\n \tksft_test_result_report(exit_status, \"%s\\n\", __func__);\n }\n \n+/* Smallest order khugepaged will consider for mTHP collapse. */\n+#define MIN_MTHP_ORDER 2\n+\n+/*\n+ * Order-parameterized collapse cases for the mthp_khugepaged context. What\n+ * they add over the generic cases run under that context is per-window\n+ * detection: which aligned window collapsed, and which of its neighbours did\n+ * not. check_huge() answers how many folios of the order the range holds,\n+ * which cannot tell one window from another.\n+ *\n+ * The region is faulted before MADV_HUGEPAGE, and the target order is only\n+ * enabled for madvise, so the sources are always order 0 and the collapse\n+ * product can only have come from khugepaged.\n+ */\n+static size_t mthp_window_size(void)\n+{\n+\treturn page_size \u003c\u003c collapse_order;\n+}\n+\n+static void mthp_push_target_order(void)\n+{\n+\tstruct thp_settings settings = *thp_current_settings();\n+\tint i;\n+\n+\t/*\n+\t * The target order, for madvise only, and nothing else enabled: the\n+\t * cases fault their region before MADV_HUGEPAGE, so the sources are\n+\t * order 0 whatever -s asked the fault path for. That matters for the\n+\t * cases built around a hole -- a large source folio would fill it in\n+\t * and the window would collapse after all.\n+\t * collapse_order_mixed_sources enables the source order it wants on\n+\t * top of this.\n+\t */\n+\tsettings.thp_enabled = THP_NEVER;\n+\tfor (i = 0; i \u003c NR_ORDERS; i++)\n+\t\tsettings.hugepages[i].enabled = THP_NEVER;\n+\tsettings.hugepages[collapse_order].enabled = THP_MADVISE;\n+\tthp_push_settings(\u0026settings);\n+}\n+\n+static bool window_collapsed(void *p, size_t len)\n+{\n+\treturn is_range_backed_by_folio_orders(p, len, collapse_order,\n+\t\t\t\t\t pagemap_fd, kpageflags_fd);\n+}\n+\n+/* No aligned window in [p, p + len) is backed at the target order. */\n+static bool window_not_collapsed(void *p, size_t len)\n+{\n+\tsize_t window = mthp_window_size();\n+\tchar *addr = p;\n+\n+\tfor (; len \u003e= window; addr += window, len -= window) {\n+\t\tif (window_collapsed(addr, window))\n+\t\t\treturn false;\n+\t}\n+\treturn true;\n+}\n+\n+static bool khugepaged_wait_full_pass(void)\n+{\n+\t/* Wait up to 30 seconds for the pass to complete. */\n+\treturn khugepaged_full_pass(30);\n+}\n+\n+static void collapse_order_single_window(struct collapse_context *c,\n+\t\t\t\t\t struct mem_ops *ops)\n+{\n+\tsize_t window = mthp_window_size();\n+\tvoid *p;\n+\n+\tmthp_push_target_order();\n+\n+\tp = ops-\u003esetup_area(1);\n+\tops-\u003efault(p, window, 2 * window);\n+\tif (!window_not_collapsed(p, hpage_pmd_size))\n+\t\tksft_exit_fail_msg(\"Unexpected large folio after fault\\n\");\n+\n+\tmadvise(p, hpage_pmd_size, MADV_HUGEPAGE);\n+\tksft_print_msg(\"Collapse one fully populated window...\");\n+\tif (!khugepaged_wait_full_pass())\n+\t\tfail(\"Timeout\");\n+\telse if (window_collapsed(p + window, window) \u0026\u0026\n+\t\t window_not_collapsed(p, window) \u0026\u0026\n+\t\t window_not_collapsed(p + 2 * window,\n+\t\t\t\t hpage_pmd_size - 2 * window))\n+\t\tsuccess(\"OK\");\n+\telse\n+\t\tfail(\"Fail\");\n+\n+\tvalidate_memory(p, window, 2 * window);\n+\tops-\u003ecleanup_area(p, hpage_pmd_size);\n+\tthp_pop_settings();\n+\tksft_test_result_report(exit_status, \"%s\\n\", __func__);\n+}\n+\n+static void collapse_order_partial_window(struct collapse_context *c,\n+\t\t\t\t\t struct mem_ops *ops)\n+{\n+\tvoid *p;\n+\n+\tmthp_push_target_order();\n+\n+\tp = ops-\u003esetup_area(1);\n+\tops-\u003efault(p, 0, page_size);\n+\tif (!window_not_collapsed(p, hpage_pmd_size))\n+\t\tksft_exit_fail_msg(\"Unexpected large folio after fault\\n\");\n+\n+\tmadvise(p, hpage_pmd_size, MADV_HUGEPAGE);\n+\tksft_print_msg(\"Collapse window with single PTE entry present...\");\n+\tif (!khugepaged_wait_full_pass())\n+\t\tfail(\"Timeout\");\n+\telse if (window_collapsed(p, mthp_window_size()))\n+\t\tsuccess(\"OK\");\n+\telse\n+\t\tfail(\"Fail\");\n+\n+\tvalidate_memory(p, 0, page_size);\n+\tops-\u003ecleanup_area(p, hpage_pmd_size);\n+\tthp_pop_settings();\n+\tksft_test_result_report(exit_status, \"%s\\n\", __func__);\n+}\n+\n+static void collapse_order_max_ptes_none(struct collapse_context *c,\n+\t\t\t\t\t struct mem_ops *ops)\n+{\n+\tstruct thp_settings settings;\n+\tsize_t window = mthp_window_size();\n+\tvoid *p;\n+\n+\tmthp_push_target_order();\n+\tsettings = *thp_current_settings();\n+\tsettings.khugepaged.max_ptes_none = 0;\n+\tthp_push_settings(\u0026settings);\n+\n+\tp = ops-\u003esetup_area(1);\n+\tops-\u003efault(p, 0, 2 * window - page_size);\n+\tif (!window_not_collapsed(p, hpage_pmd_size))\n+\t\tksft_exit_fail_msg(\"Unexpected large folio after fault\\n\");\n+\n+\tmadvise(p, hpage_pmd_size, MADV_HUGEPAGE);\n+\tksft_print_msg(\"Collapse full window, not the one missing a page...\");\n+\tif (!khugepaged_wait_full_pass())\n+\t\tfail(\"Timeout\");\n+\telse if (window_collapsed(p, window) \u0026\u0026\n+\t\t window_not_collapsed(p + window, window))\n+\t\tsuccess(\"OK\");\n+\telse\n+\t\tfail(\"Fail\");\n+\n+\tvalidate_memory(p, 0, 2 * window - page_size);\n+\tops-\u003ecleanup_area(p, hpage_pmd_size);\n+\tthp_pop_settings();\n+\tthp_pop_settings();\n+\tksft_test_result_report(exit_status, \"%s\\n\", __func__);\n+}\n+\n+static void collapse_order_mixed_sources(struct collapse_context *c,\n+\t\t\t\t\t struct mem_ops *ops)\n+{\n+\tint source_order = anon_order ? anon_order : MIN_MTHP_ORDER;\n+\tstruct thp_settings settings;\n+\tvoid *p;\n+\n+\t/* Sources must be a supported mTHP order strictly below the target. */\n+\tif (source_order \u003e= collapse_order ||\n+\t !(thp_supported_orders() \u0026 (1UL \u003c\u003c source_order))) {\n+\t\tksft_test_result_skip(\"%s: no source order below target\\n\",\n+\t\t\t\t __func__);\n+\t\treturn;\n+\t}\n+\n+\tmthp_push_target_order();\n+\n+\t/* Fault the whole region as order-@source_order folios. */\n+\tsettings = *thp_current_settings();\n+\tsettings.hugepages[source_order].enabled = THP_ALWAYS;\n+\tthp_push_settings(\u0026settings);\n+\tp = ops-\u003esetup_area(1);\n+\tops-\u003efault(p, 0, hpage_pmd_size);\n+\tthp_pop_settings();\n+\n+\t/*\n+\t * The order is enabled and supported, but the allocator can still fall\n+\t * back under fragmentation. That leaves nothing to collapse from,\n+\t * which is the machine's answer rather than a reason to end the run.\n+\t */\n+\tif (!is_range_backed_by_folio_orders(p, hpage_pmd_size, source_order,\n+\t\t\t\t\t pagemap_fd, kpageflags_fd)) {\n+\t\tksft_print_msg(\"No order-%d sources to collapse...\", source_order);\n+\t\tskip(\"Skip\");\n+\t\tops-\u003ecleanup_area(p, hpage_pmd_size);\n+\t\tthp_pop_settings();\n+\t\tksft_test_result_report(exit_status, \"%s\\n\", __func__);\n+\t\treturn;\n+\t}\n+\n+\tmadvise(p, hpage_pmd_size, MADV_HUGEPAGE);\n+\tksft_print_msg(\"Collapse region backed by order-%d sources...\",\n+\t\t source_order);\n+\tif (!khugepaged_wait_full_pass())\n+\t\tfail(\"Timeout\");\n+\telse if (window_collapsed(p, hpage_pmd_size))\n+\t\tsuccess(\"OK\");\n+\telse\n+\t\tfail(\"Fail\");\n+\n+\tvalidate_memory(p, 0, hpage_pmd_size);\n+\tops-\u003ecleanup_area(p, hpage_pmd_size);\n+\tthp_pop_settings();\n+\tksft_test_result_report(exit_status, \"%s\\n\", __func__);\n+}\n+\n static void usage(void)\n {\n \tfprintf(stderr, \"\\nUsage: ./khugepaged [OPTIONS] \u003ctest type\u003e [dir]\\n\\n\");\n@@ -1226,11 +1551,15 @@ static void usage(void)\n \tfprintf(stderr,\t\"\\t\\t-s: mTHP size, expressed as page order.\\n\");\n \tfprintf(stderr,\t\"\\t\\t Defaults to 0. Use this size for anon or shmem allocations.\\n\");\n \tfprintf(stderr,\t\"\\t\\t-c: collapse order for mTHP collapse, expressed as page order.\\n\");\n+\tfprintf(stderr,\t\"\\t\\t Defaults to every supported order below the PMD.\\n\");\n+\tfprintf(stderr,\t\"\\t\\t With -s, -s names the mTHP source order for the\\n\");\n+\tfprintf(stderr,\t\"\\t\\t mixed-source case (source order below the target).\\n\");\n \texit(1);\n }\n \n static void parse_test_type(int argc, char **argv)\n {\n+\tbool mthp_context_implied = false;\n \tint opt;\n \tchar *buf;\n \tconst char *token;\n@@ -1242,6 +1571,7 @@ static void parse_test_type(int argc, char **argv)\n \t\t\tbreak;\n \t\tcase 'c':\n \t\t\tcollapse_order = atoi(optarg);\n+\t\t\tcollapse_order_given = true;\n \t\t\tbreak;\n \t\tcase 'h':\n \t\tdefault:\n@@ -1249,12 +1579,25 @@ static void parse_test_type(int argc, char **argv)\n \t\t}\n \t}\n \n+\t/*\n+\t * Both orders end up as array indices and shift counts, so neither\n+\t * can be negative, and a zero collapse order asks for base pages.\n+\t */\n+\tif (anon_order \u003c 0 || anon_order \u003e hpage_pmd_order)\n+\t\tksft_exit_fail_msg(\"-s takes an order in 0..%d, not %d\\n\",\n+\t\t\t\t hpage_pmd_order, anon_order);\n+\tif (collapse_order_given \u0026\u0026\n+\t (collapse_order \u003c= 0 || collapse_order \u003e hpage_pmd_order))\n+\t\tksft_exit_fail_msg(\"-c takes an order in 1..%d, not %d\\n\",\n+\t\t\t\t hpage_pmd_order, collapse_order);\n+\n \targv += optind;\n \targc -= optind;\n \n \tif (argc == 0) {\n-\t\t/* Backwards compatibility */\n+\t\t/* Everything that needs no argument of its own: anon, every context */\n \t\tkhugepaged_context = \u0026__khugepaged_context;\n+\t\tmthp_khugepaged_context = \u0026__mthp_khugepaged_context;\n \t\tmadvise_context = \u0026__madvise_context;\n \t\tanon_ops = \u0026__anon_ops;\n \t\treturn;\n@@ -1265,13 +1608,19 @@ static void parse_test_type(int argc, char **argv)\n \n \tif (!strcmp(token, \"all\")) {\n \t\tkhugepaged_context = \u0026__khugepaged_context;\n+\t\tmthp_khugepaged_context = \u0026__mthp_khugepaged_context;\n \t\tmadvise_context = \u0026__madvise_context;\n+\n+\t\t/*\n+\t\t * \"all\" sweeps the mTHP context in, but it only has anon\n+\t\t * cases: step it aside for the other mem_types rather than\n+\t\t * refusing the whole run.\n+\t\t */\n+\t\tmthp_context_implied = true;\n \t} else if (!strcmp(token, \"khugepaged\")) {\n \t\tkhugepaged_context = \u0026__khugepaged_context;\n \t} else if (!strcmp(token, \"mthp_khugepaged\")) {\n \t\tmthp_khugepaged_context = \u0026__mthp_khugepaged_context;\n-\t\tif (collapse_order \u003c= 0 || collapse_order \u003e= hpage_pmd_order)\n-\t\t\tusage();\n \t} else if (!strcmp(token, \"madvise\")) {\n \t\tmadvise_context = \u0026__madvise_context;\n \t} else {\n@@ -1287,20 +1636,20 @@ static void parse_test_type(int argc, char **argv)\n \t\tread_write_file_write_ops = \u0026__read_write_file_write_ops;\n \t\tanon_ops = \u0026__anon_ops;\n \t\tshmem_ops = \u0026__shmem_ops;\n-\t\tif (mthp_khugepaged_context)\n-\t\t\tusage();\n \t} else if (!strcmp(buf, \"anon\")) {\n \t\tanon_ops = \u0026__anon_ops;\n \t} else if (!strcmp(buf, \"file\")) {\n \t\tread_only_file_ops = \u0026__read_only_file_ops;\n \t\tread_write_file_read_ops = \u0026__read_write_file_read_ops;\n \t\tread_write_file_write_ops = \u0026__read_write_file_write_ops;\n-\t\tif (mthp_khugepaged_context)\n+\t\tif (mthp_khugepaged_context \u0026\u0026 !mthp_context_implied)\n \t\t\tusage();\n+\t\tmthp_khugepaged_context = NULL;\n \t} else if (!strcmp(buf, \"shmem\")) {\n \t\tshmem_ops = \u0026__shmem_ops;\n-\t\tif (mthp_khugepaged_context)\n+\t\tif (mthp_khugepaged_context \u0026\u0026 !mthp_context_implied)\n \t\t\tusage();\n+\t\tmthp_khugepaged_context = NULL;\n \t} else {\n \t\tusage();\n \t}\n@@ -1322,9 +1671,14 @@ struct test_case {\n \tstruct mem_ops *ops;\n \tconst char *desc;\n \ttest_fn fn;\n+\tint order;\t\t/* mTHP contexts: the collapse order */\n };\n \n-#define MAX_TEST_CASES 64\n+/*\n+ * Enough for every case at every order the kernel offers: the mTHP context\n+ * runs its cases once per supported order below the PMD.\n+ */\n+#define MAX_TEST_CASES 256\n static struct test_case test_cases[MAX_TEST_CASES];\n static int nr_test_cases;\n \n@@ -1337,6 +1691,7 @@ static int nr_test_cases;\n \t\t\t.ops\t= o,\t\t\t\t\t\\\n \t\t\t.desc\t= #t,\t\t\t\t\t\\\n \t\t\t.fn\t= t,\t\t\t\t\t\\\n+\t\t\t.order\t= collapse_order,\t\t\t\\\n \t\t};\t\t\t\t\t\t\t\\\n \t}\t\t\t\t\t\t\t\t\\\n \t} while (0)\n@@ -1377,8 +1732,83 @@ int main(int argc, char **argv)\n \n \tparse_test_type(argc, argv);\n \n+\tif (mthp_khugepaged_context) {\n+\t\tunsigned long orders = thp_supported_orders();\n+\n+\t\tif (collapse_order_given) {\n+\t\t\t/* -c pins one order; it has to be one we can build */\n+\t\t\tif (collapse_order \u003e= hpage_pmd_order)\n+\t\t\t\tksft_exit_fail_msg(\"-c takes an order below the PMD order (%d)\\n\",\n+\t\t\t\t\t\t hpage_pmd_order);\n+\t\t\tif (!(orders \u0026 (1UL \u003c\u003c collapse_order)))\n+\t\t\t\tksft_exit_skip(\"Order %d is not a supported anon THP order\\n\",\n+\t\t\t\t\t collapse_order);\n+\t\t\tif (collapse_order \u003c= anon_order)\n+\t\t\t\tksft_exit_skip(\"-c %d needs a source order below it, -s says %d\\n\",\n+\t\t\t\t\t collapse_order, anon_order);\n+\t\t\tcollapse_orders[nr_collapse_orders++] = collapse_order;\n+\t\t} else {\n+\t\t\t/*\n+\t\t\t * Otherwise every order a collapse could produce. -s\n+\t\t\t * makes the fault path hand out folios of that order,\n+\t\t\t * so a target at or below it has nothing to collapse:\n+\t\t\t * the sources are already the size being asked for.\n+\t\t\t */\n+\t\t\tint first = anon_order ? anon_order + 1 : MIN_MTHP_ORDER;\n+\n+\t\t\tif (first \u003c MIN_MTHP_ORDER)\n+\t\t\t\tfirst = MIN_MTHP_ORDER;\n+\t\t\tfor (int i = first; i \u003c hpage_pmd_order; i++) {\n+\t\t\t\tif (orders \u0026 (1UL \u003c\u003c i))\n+\t\t\t\t\tcollapse_orders[nr_collapse_orders++] = i;\n+\t\t\t}\n+\t\t\tif (!nr_collapse_orders)\n+\t\t\t\tksft_print_msg(\"mTHP cases skipped: no order above the source\\n\");\n+\t\t}\n+\t}\n+\n+\tif (mthp_khugepaged_context) {\n+\t\tpagemap_fd = open(\"/proc/self/pagemap\", O_RDONLY);\n+\t\tif (pagemap_fd \u003c 0)\n+\t\t\tksft_exit_fail_perror(\"open(/proc/self/pagemap)\");\n+\t\tkpageflags_fd = open(\"/proc/kpageflags\", O_RDONLY);\n+\t\tif (kpageflags_fd \u003c 0)\n+\t\t\tksft_exit_fail_perror(\"open(/proc/kpageflags)\");\n+\t}\n+\n \tsetbuf(stdout, NULL);\n \n+\t/*\n+\t * The page cache caps folio order at MAX_PAGECACHE_ORDER, which\n+\t * xas_split_alloc() puts below the PMD order on arm64 with 64K pages.\n+\t * A PMD-sized page cache folio is then impossible, so the kernel\n+\t * refuses these collapses by design and there is nothing to test.\n+\t *\n+\t * The cap is one global, so it rules out every file mapping, not just\n+\t * shmem: shmem_huge_global_enabled() drops the PMD order from what it\n+\t * allows, and file_thp_enabled() refuses a regular file whose mapping\n+\t * cannot hold a PMD folio.\n+\t *\n+\t * The per-order shmem_enabled control below is what makes the cap\n+\t * readable: it is created for the orders in THP_ORDERS_ALL_FILE_DEFAULT,\n+\t * which is the cap and nothing else, so whether the PMD order has one\n+\t * answers for a regular file as much as for shmem.\n+\t */\n+\tif (!(thp_shmem_supported_orders() \u0026 (1UL \u003c\u003c hpage_pmd_order))) {\n+\t\tif (shmem_ops) {\n+\t\t\tksft_print_msg(\"no PMD-order page cache folio: skipping shmem\\n\");\n+\t\t\tshmem_ops = NULL;\n+\t\t}\n+\t\tif (read_only_file_ops) {\n+\t\t\tksft_print_msg(\"no PMD-order page cache folio: skipping file\\n\");\n+\t\t\tread_only_file_ops = NULL;\n+\t\t\tread_write_file_read_ops = NULL;\n+\t\t\tread_write_file_write_ops = NULL;\n+\t\t}\n+\t\tif (!anon_ops \u0026\u0026 !shmem_ops \u0026\u0026 !read_only_file_ops)\n+\t\t\tksft_exit_skip(\"Nothing left to collapse into\\n\");\n+\t}\n+\n \tdefault_settings.khugepaged.max_ptes_none = hpage_pmd_nr - 1;\n \tdefault_settings.khugepaged.max_ptes_swap = hpage_pmd_nr / 8;\n \tdefault_settings.khugepaged.max_ptes_shared = hpage_pmd_nr / 2;\n@@ -1396,7 +1826,17 @@ int main(int argc, char **argv)\n \tTEST(collapse_full, khugepaged_context, read_write_file_read_ops);\n \tTEST(collapse_full, khugepaged_context, read_write_file_write_ops);\n \tTEST(collapse_full, khugepaged_context, shmem_ops);\n-\tTEST(collapse_full, mthp_khugepaged_context, anon_ops);\n+\tfor (int i = 0; i \u003c nr_collapse_orders; i++) {\n+\t\tcollapse_order = collapse_orders[i];\n+\t\tTEST(collapse_full, mthp_khugepaged_context, anon_ops);\n+\t\tTEST(collapse_empty, mthp_khugepaged_context, anon_ops);\n+\t\tTEST(collapse_single_mthp, mthp_khugepaged_context, anon_ops);\n+\t\tTEST(collapse_order_single_window, mthp_khugepaged_context, anon_ops);\n+\t\tTEST(collapse_order_partial_window, mthp_khugepaged_context, anon_ops);\n+\t\tTEST(collapse_order_max_ptes_none, mthp_khugepaged_context, anon_ops);\n+\t\tTEST(collapse_order_mixed_sources, mthp_khugepaged_context, anon_ops);\n+\t}\n+\n \tTEST(collapse_full, madvise_context, anon_ops);\n \tTEST(collapse_full, madvise_context, read_only_file_ops);\n \tTEST(collapse_full, madvise_context, read_write_file_read_ops);\n@@ -1404,10 +1844,8 @@ int main(int argc, char **argv)\n \tTEST(collapse_full, madvise_context, shmem_ops);\n \n \tTEST(collapse_empty, khugepaged_context, anon_ops);\n-\tTEST(collapse_empty, mthp_khugepaged_context, anon_ops);\n \tTEST(collapse_empty, madvise_context, anon_ops);\n \n-\tTEST(collapse_single_mthp, mthp_khugepaged_context, anon_ops);\n \n \tTEST(collapse_single_pte_entry, khugepaged_context, anon_ops);\n \tTEST(collapse_single_pte_entry, khugepaged_context, read_only_file_ops);\n@@ -1463,6 +1901,9 @@ int main(int argc, char **argv)\n \tTEST(collapse_max_ptes_shared, khugepaged_context, anon_ops);\n \tTEST(collapse_max_ptes_shared, madvise_context, anon_ops);\n \n+\tTEST(collapse_fork_cow_race, khugepaged_context, anon_ops);\n+\tTEST(collapse_fork_cow_race, madvise_context, anon_ops);\n+\n \tTEST(madvise_collapse_existing_thps, madvise_context, anon_ops);\n \tTEST(madvise_collapse_existing_thps, madvise_context, read_only_file_ops);\n \tTEST(madvise_collapse_existing_thps, madvise_context, read_write_file_read_ops);\n@@ -1478,7 +1919,15 @@ int main(int argc, char **argv)\n \tfor (int i = 0; i \u003c nr_test_cases; i++) {\n \t\tstruct test_case *t = \u0026test_cases[i];\n \n-\t\tksft_print_msg(\"\\n# Run test: %s (%s:%s)\\n\", t-\u003edesc, t-\u003ectx-\u003ename, t-\u003eops-\u003ename);\n+\t\tif (t-\u003ectx == \u0026__mthp_khugepaged_context) {\n+\t\t\tcollapse_order = t-\u003eorder;\n+\t\t\tksft_print_msg(\"\\n# Run test: %s (%s:%s, order %d)\\n\",\n+\t\t\t\t t-\u003edesc, t-\u003ectx-\u003ename, t-\u003eops-\u003ename,\n+\t\t\t\t t-\u003eorder);\n+\t\t} else {\n+\t\t\tksft_print_msg(\"\\n# Run test: %s (%s:%s)\\n\", t-\u003edesc,\n+\t\t\t\t t-\u003ectx-\u003ename, t-\u003eops-\u003ename);\n+\t\t}\n \t\tt-\u003efn(t-\u003ectx, t-\u003eops);\n \t}\n \ndiff --git a/tools/testing/selftests/mm/khugepaged_race.c b/tools/testing/selftests/mm/khugepaged_race.c\nnew file mode 100644\nindex 0000000000000..3601c030d43e5\n--- /dev/null\n+++ b/tools/testing/selftests/mm/khugepaged_race.c\n@@ -0,0 +1,610 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * khugepaged race harness.\n+ *\n+ * Runs collapse against concurrent faults, transient GUP pins\n+ * (gup_test), fork, mremap and MADV_DONTNEED over the same ranges, in\n+ * one of three driver modes:\n+ *\n+ * stepped\tkhugepaged, one full pass at a time through\n+ *\t\tkhugepaged_full_pass(), so a step covers a known extent;\n+ * free\tkhugepaged left to run (scan_sleep_millisecs=0), for soak;\n+ * madvise\tMADV_COLLAPSE in a loop.\n+ *\n+ * All anon THP orders are enabled (inherit). Occupancy runs at both ends\n+ * of what mTHP collapse supports: max_ptes_none 0, where a window must be\n+ * fully populated, and HPAGE_PMD_NR - 1, where a window full of holes\n+ * collapses too. The holes are not copied from anywhere -- they are\n+ * zero-filled, and re-checked under the page table lock at install time in\n+ * case a racing fault got there first.\n+ *\n+ * -p adds memory pressure to any of the above: MADV_PAGEOUT cycling\n+ * on a dedicated neighbor region (swap traffic and LRU churn; skipped\n+ * with a note when the host has no swap) and a compact_memory trigger\n+ * loop (compaction migrates source folios, racing collapse's freeze\n+ * with refcount elevation and migration entries of its own).\n+ *\n+ * Correctness signals: every racing page must read as its pattern or\n+ * zero (MADV_DONTNEED), never anything else. The faulters and the fork\n+ * children check that continuously, a final sweep checks it once more, plus\n+ * whatever DEBUG_VM / page_table_check / KASAN / lockdep report in\n+ * dmesg, which the caller is expected to inspect.\n+ */\n+#define _GNU_SOURCE\n+#include \u003cerrno.h\u003e\n+#include \u003cfcntl.h\u003e\n+#include \u003cpthread.h\u003e\n+#include \u003cstdio.h\u003e\n+#include \u003cstdlib.h\u003e\n+#include \u003cstring.h\u003e\n+#include \u003csys/ioctl.h\u003e\n+#include \u003csys/mman.h\u003e\n+#include \u003csys/time.h\u003e\n+#include \u003csys/wait.h\u003e\n+#include \u003cunistd.h\u003e\n+\n+#include \"kselftest.h\"\n+#include \"vm_util.h\"\n+#include \"hugepage_settings.h\"\n+#include \"../../../../mm/gup_test.h\"\n+\n+#define BASE_ADDR\t((void *)(1UL \u003c\u003c 30))\n+\n+/*\n+ * Shared playground for faults/pins/fork/dontneed: several PMD-sized\n+ * areas the racing threads spread across, plus one area owned by the\n+ * mremap thread. More areas means more independent regions collapsing\n+ * at once; the default suits a normal machine. On a memory-constrained\n+ * host -- or under emulation, where a 512M PMD (arm64/64K) makes the\n+ * default playground multi-gigabyte -- pass -a to shrink it.\n+ */\n+#define DEFAULT_SHARED_AREAS\t3\n+static int nr_shared_areas;\n+static int nr_areas;\n+\n+static unsigned long hpage_pmd_size;\n+static unsigned long page_size;\n+static char *region;\t\t/* NR_AREAS * hpage_pmd_size */\n+static char *mremap_area;\t/* region + NR_SHARED_AREAS areas */\n+static char *mremap_scratch;\t/* well above the region */\n+static char *pageout_area;\t/* -p: dedicated pressure region */\n+static size_t pageout_size;\n+static int gup_fd = -1;\n+static volatile int stop;\n+static volatile int corrupted;\n+\n+static unsigned int pattern(unsigned long page_idx)\n+{\n+\tunsigned int val = (unsigned int)page_idx * 2654435761U;\n+\n+\treturn val ? val : 1;\t/* never collides with the zero-fill */\n+}\n+\n+/* Zero means never written; anything else must be this page's pattern */\n+static bool page_is_corrupt(unsigned long page_idx, unsigned int *val)\n+{\n+\t*val = *(unsigned int *)(region + page_idx * page_size);\n+\n+\treturn *val \u0026\u0026 *val != pattern(page_idx);\n+}\n+\n+static void check_page(unsigned long page_idx)\n+{\n+\tunsigned int val;\n+\n+\tif (page_is_corrupt(page_idx, \u0026val)) {\n+\t\tcorrupted = 1;\n+\t\tksft_print_msg(\"Corruption at page %lu: %#x != %#x\\n\",\n+\t\t\t page_idx, val, pattern(page_idx));\n+\t}\n+}\n+\n+static unsigned long shared_pages(void)\n+{\n+\treturn nr_shared_areas * hpage_pmd_size / page_size;\n+}\n+\n+static unsigned long rand_page(unsigned int *seed)\n+{\n+\treturn (unsigned long)rand_r(seed) % shared_pages();\n+}\n+\n+/* Pages left from @page_idx, so a range never reaches the mremap thread's area */\n+static unsigned long room_from(unsigned long page_idx, unsigned long want)\n+{\n+\tunsigned long left = shared_pages() - page_idx;\n+\n+\treturn want \u003c left ? want : left;\n+}\n+\n+static void *faulter_fn(void *arg)\n+{\n+\tunsigned int seed = (unsigned long)arg;\n+\n+\twhile (!stop) {\n+\t\tunsigned long page_idx = rand_page(\u0026seed);\n+\n+\t\tif (rand_r(\u0026seed) \u0026 1)\n+\t\t\t*(unsigned int *)(region + page_idx * page_size) =\n+\t\t\t\tpattern(page_idx);\n+\t\telse\n+\t\t\tcheck_page(page_idx);\n+\t}\n+\treturn NULL;\n+}\n+\n+static void *dontneed_fn(void *arg)\n+{\n+\tunsigned int seed = (unsigned long)arg;\n+\n+\twhile (!stop) {\n+\t\tunsigned long page_idx = rand_page(\u0026seed);\n+\t\tunsigned long nr = 1UL \u003c\u003c (rand_r(\u0026seed) % 6);\t/* 1..32 pages */\n+\n+\t\t/*\n+\t\t * Once in a while zap a whole PMD-aligned area: only a zap\n+\t\t * spanning the full table triggers the empty-table reclaim\n+\t\t * (CONFIG_PT_RECLAIM), which can free the table under a\n+\t\t * collapse that is midway through it. Sub-table zaps never\n+\t\t * reach that path.\n+\t\t */\n+\t\tif (!(rand_r(\u0026seed) % 64)) {\n+\t\t\tunsigned long area = page_idx /\n+\t\t\t\t\t(hpage_pmd_size / page_size);\n+\n+\t\t\tmadvise(region + area * hpage_pmd_size,\n+\t\t\t\thpage_pmd_size, MADV_DONTNEED);\n+\t\t} else {\n+\t\t\tmadvise(region + page_idx * page_size,\n+\t\t\t\troom_from(page_idx, nr) * page_size,\n+\t\t\t\tMADV_DONTNEED);\n+\t\t}\n+\t\tusleep(rand_r(\u0026seed) % 500);\n+\t}\n+\treturn NULL;\n+}\n+\n+static void *pinner_fn(void *arg)\n+{\n+\tunsigned int seed = (unsigned long)arg;\n+\n+\twhile (!stop) {\n+\t\tstruct gup_test gup = {};\n+\t\tunsigned long page_idx = rand_page(\u0026seed);\n+\n+\t\tunsigned long nr = room_from(page_idx, 16);\n+\n+\t\tgup.addr = (unsigned long)(region + page_idx * page_size);\n+\t\tgup.size = nr * page_size;\n+\t\tgup.nr_pages_per_call = nr;\n+\t\tgup.gup_flags = 1;\t/* FOLL_WRITE */\n+\t\t/* Racing MADV_DONTNEED makes transient failures expected. */\n+\t\tioctl(gup_fd, PIN_FAST_BENCHMARK, \u0026gup);\n+\t\tusleep(rand_r(\u0026seed) % 200);\n+\t}\n+\treturn NULL;\n+}\n+\n+static void *forker_fn(void *arg)\n+{\n+\tunsigned int seed = (unsigned long)arg;\n+\n+\twhile (!stop) {\n+\t\tpid_t pid = fork();\n+\n+\t\tif (pid == 0) {\n+\t\t\tunsigned int val;\n+\t\t\tint bad = 0;\n+\n+\t\t\t/*\n+\t\t\t * No stdio in the child: a thread may have held\n+\t\t\t * stdout's lock when we forked, and printing under an\n+\t\t\t * inherited lock hangs. The parent reports what the\n+\t\t\t * exit status says.\n+\t\t\t */\n+\t\t\tfor (int i = 0; i \u003c 16; i++)\n+\t\t\t\tbad |= page_is_corrupt(rand_page(\u0026seed), \u0026val);\n+\t\t\t_exit(bad);\n+\t\t}\n+\t\tif (pid \u003e 0) {\n+\t\t\tint wstatus;\n+\n+\t\t\tif (waitpid(pid, \u0026wstatus, 0) \u003c 0)\n+\t\t\t\tksft_exit_fail_perror(\"waitpid()\");\n+\t\t\t/* A child killed on the read counts too, not just its exit code. */\n+\t\t\tif (!WIFEXITED(wstatus) || WEXITSTATUS(wstatus))\n+\t\t\t\tcorrupted = 1;\n+\t\t}\n+\t\tusleep(rand_r(\u0026seed) % 2000);\n+\t}\n+\treturn NULL;\n+}\n+\n+static void *mremapper_fn(void *arg)\n+{\n+\tunsigned int seed = (unsigned long)arg;\n+\n+\twhile (!stop) {\n+\t\tvoid *p;\n+\n+\t\tp = mremap(mremap_area, hpage_pmd_size, hpage_pmd_size,\n+\t\t\t MREMAP_MAYMOVE | MREMAP_FIXED, mremap_scratch);\n+\t\tif (p == MAP_FAILED)\n+\t\t\tksft_exit_fail_perror(\"mremap() away\");\n+\t\tfor (int i = 0; i \u003c 8; i++)\n+\t\t\tmremap_scratch[(rand_r(\u0026seed) %\n+\t\t\t\t(hpage_pmd_size / page_size)) * page_size] = 1;\n+\t\tp = mremap(mremap_scratch, hpage_pmd_size, hpage_pmd_size,\n+\t\t\t MREMAP_MAYMOVE | MREMAP_FIXED, mremap_area);\n+\t\tif (p == MAP_FAILED)\n+\t\t\tksft_exit_fail_perror(\"mremap() back\");\n+\t\tusleep(rand_r(\u0026seed) % 2000);\n+\t}\n+\treturn NULL;\n+}\n+\n+/*\n+ * -p: swap traffic and LRU churn on a region of our own. The content\n+ * check is exact: a page out and back through swap must preserve the\n+ * pattern, and nothing else ever writes here.\n+ */\n+static void *pageout_fn(void *arg)\n+{\n+\tunsigned int seed = (unsigned long)arg;\n+\tunsigned long nr = pageout_size / page_size;\n+\tunsigned long i;\n+\n+\tfor (i = 0; i \u003c nr; i++)\n+\t\t*(unsigned int *)(pageout_area + i * page_size) = pattern(i);\n+\n+\twhile (!stop) {\n+\t\tmadvise(pageout_area, pageout_size, MADV_PAGEOUT);\n+\t\tfor (i = 0; i \u003c nr \u0026\u0026 !stop; i++) {\n+\t\t\tunsigned int val = *(unsigned int *)(pageout_area +\n+\t\t\t\t\t\t\t i * page_size);\n+\n+\t\t\tif (val != pattern(i)) {\n+\t\t\t\tcorrupted = 1;\n+\t\t\t\tksft_print_msg(\"Pageout corruption at page %lu: %#x != %#x\\n\",\n+\t\t\t\t\t i, val, pattern(i));\n+\t\t\t}\n+\t\t}\n+\t\tusleep(rand_r(\u0026seed) % 2000);\n+\t}\n+\treturn NULL;\n+}\n+\n+/* -p: compaction migrates the collapse sources out from under us. */\n+static void *compactor_fn(void *arg)\n+{\n+\tunsigned int seed = (unsigned long)arg;\n+\tint fd = open(\"/proc/sys/vm/compact_memory\", O_WRONLY);\n+\n+\tif (fd \u003c 0) {\n+\t\tksft_print_msg(\"No compact_memory; compactor idle\\n\");\n+\t\treturn NULL;\n+\t}\n+\twhile (!stop) {\n+\t\tif (write(fd, \"1\", 1) \u003c 0)\n+\t\t\tbreak;\n+\t\tusleep(10000 + rand_r(\u0026seed) % 100000);\n+\t}\n+\tclose(fd);\n+\treturn NULL;\n+}\n+\n+static bool swap_available(void)\n+{\n+\tchar line[256];\n+\tint lines = 0;\n+\tFILE *fp = fopen(\"/proc/swaps\", \"r\");\n+\n+\tif (!fp)\n+\t\treturn false;\n+\twhile (fgets(line, sizeof(line), fp))\n+\t\tlines++;\n+\tfclose(fp);\n+\treturn lines \u003e 1;\n+}\n+\n+static unsigned long now_ms(void)\n+{\n+\tstruct timeval tv;\n+\n+\tgettimeofday(\u0026tv, NULL);\n+\treturn tv.tv_sec * 1000UL + tv.tv_usec / 1000;\n+}\n+\n+static void usage(void)\n+{\n+\tfprintf(stderr,\n+\t\t\"Usage: khugepaged_race [-d seconds] [-m stepped|free|madvise] [-z] [-p] [-a areas]\\n\"\n+\t\t\"\\tWithout -m, every mode runs in turn.\\n\"\n+\t\t\"\\t-d: seconds per mode (default 5)\\n\"\n+\t\t\"\\tBoth occupancy limits run unless -z asks for holes only.\\n\"\n+\t\t\"\\t-z: only max_ptes_none = HPAGE_PMD_NR - 1 (hole-heavy)\\n\"\n+\t\t\"\\tRuns with and without memory pressure unless -p asks for\\n\"\n+\t\t\"\\tpressure only.\\n\"\n+\t\t\"\\t-p: only with the pageout and compaction threads\\n\"\n+\t\t\"\\t-a: number of shared PMD-sized playground areas (default 3)\\n\");\n+\texit(1);\n+}\n+\n+int main(int argc, char **argv)\n+{\n+\tstatic const char * const thread_names[] = {\n+\t\t\"faulter\", \"faulter2\", \"dontneed\", \"pinner\", \"forker\",\n+\t\t\"mremapper\", \"pageout\", \"compactor\",\n+\t};\n+\tvoid *(*const thread_fns[])(void *) = {\n+\t\tfaulter_fn, faulter_fn, dontneed_fn, pinner_fn, forker_fn,\n+\t\tmremapper_fn, pageout_fn, compactor_fn,\n+\t};\n+\tconst unsigned long pageout_bit = 1UL \u003c\u003c 6, compactor_bit = 1UL \u003c\u003c 7;\n+\tconst int nr_threads = ARRAY_SIZE(thread_names);\n+\tpthread_t threads[ARRAY_SIZE(thread_names)];\n+\tstatic const char * const all_modes[] = { \"stepped\", \"free\", \"madvise\" };\n+\tstatic const int all_nones[] = { 0, 1 };\t/* strict, holes */\n+\tstatic const int all_press[] = { 0, 1 };\t/* quiet, under pressure */\n+\tconst int *nones = all_nones;\n+\tconst int *press = all_press;\n+\tint nr_nones = ARRAY_SIZE(all_nones);\n+\tint nr_press = ARRAY_SIZE(all_press);\n+\tconst char *one_mode[1];\n+\tconst char * const *modes = all_modes;\n+\tint nr_modes = ARRAY_SIZE(all_modes);\n+\tconst char *mode_arg = NULL;\n+\tstruct thp_settings settings;\n+\tunsigned long end_ms;\n+\tint duration_s = 5;\n+\tunsigned long thread_mask = ~0UL;\n+\tunsigned long base_mask;\n+\tint nr_areas_arg = 0;\n+\tbool holes_only = false;\n+\tbool pressure_only = false;\n+\tunsigned long i;\n+\tint steps = 0;\n+\tint opt;\n+\n+\twhile ((opt = getopt(argc, argv, \"a:d:m:t:zph\")) != -1) {\n+\t\tswitch (opt) {\n+\t\tcase 'a':\n+\t\t\tnr_areas_arg = atoi(optarg);\n+\t\t\tbreak;\n+\t\tcase 'd':\n+\t\t\tduration_s = atoi(optarg);\n+\t\t\tbreak;\n+\t\tcase 'm':\n+\t\t\tmode_arg = optarg;\n+\t\t\tbreak;\n+\t\tcase 't':\n+\t\t\t/* debug: bitmask of racing threads to start */\n+\t\t\tthread_mask = strtoul(optarg, NULL, 0);\n+\t\t\tbreak;\n+\t\tcase 'z':\n+\t\t\tholes_only = true;\n+\t\t\tbreak;\n+\t\tcase 'p':\n+\t\t\tpressure_only = true;\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\tusage();\n+\t\t}\n+\t}\n+\tif (holes_only) {\n+\t\tnones = all_nones + 1;\n+\t\tnr_nones = 1;\n+\t}\n+\n+\tif (pressure_only) {\n+\t\tpress = all_press + 1;\n+\t\tnr_press = 1;\n+\t}\n+\n+\tif (mode_arg) {\n+\t\tif (strcmp(mode_arg, \"stepped\") \u0026\u0026 strcmp(mode_arg, \"free\") \u0026\u0026\n+\t\t strcmp(mode_arg, \"madvise\"))\n+\t\t\tusage();\n+\t\tone_mode[0] = mode_arg;\n+\t\tmodes = one_mode;\n+\t\tnr_modes = 1;\n+\t}\n+\n+\tksft_print_header();\n+\tif (!thp_available())\n+\t\tksft_exit_skip(\"Transparent Hugepages not available\\n\");\n+\n+\tpage_size = getpagesize();\n+\thpage_pmd_size = read_pmd_pagesize();\n+\tif (!hpage_pmd_size)\n+\t\tksft_exit_fail_msg(\"Reading PMD pagesize failed\\n\");\n+\n+\tgup_fd = open(\"/sys/kernel/debug/gup_test\", O_RDWR);\n+\tif (gup_fd \u003c 0)\n+\t\tksft_exit_skip(\"/sys/kernel/debug/gup_test requires CONFIG_GUP_TEST and root\\n\");\n+\n+\tnr_shared_areas = nr_areas_arg \u003e 0 ? nr_areas_arg : DEFAULT_SHARED_AREAS;\n+\tnr_areas = nr_shared_areas + 1;\n+\n+\t/*\n+\t * The mremap thread moves its area to this address and back, and\n+\t * MREMAP_FIXED unmaps whatever is in the way without saying so. Claim\n+\t * the address here, so a layout that does not match this assumption\n+\t * fails now instead of losing a mapping later. Nothing else in the\n+\t * process maps this low: thread stacks and malloc arenas come from the\n+\t * top-down mmap area, well above.\n+\t */\n+\tmremap_scratch = (char *)BASE_ADDR + 2 * nr_areas * hpage_pmd_size;\n+\tif (mmap(mremap_scratch, hpage_pmd_size, PROT_NONE,\n+\t\t MAP_ANONYMOUS | MAP_PRIVATE | MAP_FIXED_NOREPLACE,\n+\t\t -1, 0) != (void *)mremap_scratch)\n+\t\tksft_exit_fail_perror(\"mmap() mremap scratch\");\n+\n+\tbase_mask = thread_mask;\n+\tif (!swap_available())\n+\t\t/* No swap, no anon reclaim: compaction-only pressure. */\n+\t\tksft_print_msg(\"no swap: the pageout thread stays idle\\n\");\n+\n+\tksft_set_plan(nr_modes * nr_nones * nr_press);\n+\n+\tthp_save_settings();\n+\tthp_read_settings(\u0026settings);\n+\n+\t/*\n+\t * A base entry for the stack, so that the pop at the end of a mode\n+\t * always has something to write back: thp_pop_settings() on an empty\n+\t * stack has no settings to apply and gives up.\n+\t */\n+\tthp_push_settings(\u0026settings);\n+\n+\tfor (int run = 0; run \u003c nr_modes * nr_nones * nr_press; run++) {\n+\t\tint rem = run % (nr_nones * nr_press);\n+\t\tconst char *mode = modes[run / (nr_nones * nr_press)];\n+\t\tbool holes = nones[rem / nr_press];\n+\t\tbool pressure = press[rem % nr_press];\n+\n+\t\tthread_mask = base_mask;\n+\t\tif (!pressure)\n+\t\t\tthread_mask \u0026= ~(pageout_bit | compactor_bit);\n+\t\telse if (!swap_available())\n+\t\t\tthread_mask \u0026= ~pageout_bit;\n+\n+\t\tthp_read_settings(\u0026settings);\n+\t\tsettings.thp_enabled = THP_MADVISE;\n+\t\tsettings.thp_defrag = THP_DEFRAG_ALWAYS;\n+\t\tsettings.shmem_enabled = SHMEM_NEVER;\n+\t\tsettings.khugepaged.defrag = 1;\n+\t\tsettings.khugepaged.scan_sleep_millisecs =\n+\t\t\tstrcmp(mode, \"free\") ? 1000 : 0;\n+\t\tsettings.khugepaged.alloc_sleep_millisecs = 10;\n+\n+\t\t/*\n+\t\t * mTHP collapse only supports the two ends of the occupancy\n+\t\t * scale: 0 or HPAGE_PMD_NR - 1 (anything else coerces to 0).\n+\t\t * Strict needs a fully populated window, which is rare under\n+\t\t * racing MADV_DONTNEED; hole-heavy windows collapse instead,\n+\t\t * so the two ends race different paths.\n+\t\t */\n+\t\tsettings.khugepaged.max_ptes_none = holes ?\n+\t\t\t(hpage_pmd_size / page_size) - 1 : 0;\n+\t\tsettings.khugepaged.pages_to_scan =\n+\t\t\tnr_areas * (hpage_pmd_size / page_size) * 8;\n+\t\tfor (i = 0; i \u003c NR_ORDERS; i++) {\n+\t\t\tif (thp_supported_orders() \u0026 (1UL \u003c\u003c i))\n+\t\t\t\tsettings.hugepages[i].enabled = THP_INHERIT;\n+\t\t}\n+\t\t/* Popped at the end of this mode, before the next one. */\n+\t\tthp_push_settings(\u0026settings);\n+\n+\t\tregion = mmap(BASE_ADDR, nr_areas * hpage_pmd_size,\n+\t\t\t PROT_READ | PROT_WRITE, MAP_ANONYMOUS |\n+\t\t\t MAP_PRIVATE | MAP_FIXED_NOREPLACE, -1, 0);\n+\t\tif (region != BASE_ADDR)\n+\t\t\tksft_exit_fail_perror(\"mmap() playground\");\n+\t\tmremap_area = region + nr_shared_areas * hpage_pmd_size;\n+\n+\t\tif (thread_mask \u0026 pageout_bit) {\n+\t\t\t/*\n+\t\t\t * Big enough to cycle real reclaim, small enough not\n+\t\t\t * to dominate a TCG guest: 4 PMD areas, clamped to\n+\t\t\t * [16M, 64M].\n+\t\t\t */\n+\t\t\tpageout_size = 4 * hpage_pmd_size;\n+\t\t\tpageout_size = pageout_size \u003c (16UL \u003c\u003c 20) ?\n+\t\t\t\t (16UL \u003c\u003c 20) :\n+\t\t\t\t pageout_size \u003e (64UL \u003c\u003c 20) ?\n+\t\t\t\t (64UL \u003c\u003c 20) : pageout_size;\n+\t\t\tpageout_area = mmap(NULL, pageout_size,\n+\t\t\t\t\t PROT_READ | PROT_WRITE,\n+\t\t\t\t\t MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);\n+\t\t\tif (pageout_area == MAP_FAILED)\n+\t\t\t\tksft_exit_fail_perror(\"mmap() pageout area\");\n+\t\t}\n+\n+\t\t/* Populate so the first pass has something to collapse. */\n+\t\tfor (i = 0; i \u003c nr_shared_areas * hpage_pmd_size / page_size; i++)\n+\t\t\t*(unsigned int *)(region + i * page_size) = pattern(i);\n+\t\tmemset(mremap_area, 1, hpage_pmd_size);\n+\t\tmadvise(region, nr_areas * hpage_pmd_size, MADV_HUGEPAGE);\n+\n+\t\tfor (i = 0; i \u003c nr_threads; i++) {\n+\t\t\tif (!(thread_mask \u0026 (1UL \u003c\u003c i))) {\n+\t\t\t\tthreads[i] = 0;\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\t\t\tif (pthread_create(\u0026threads[i], NULL, thread_fns[i],\n+\t\t\t\t\t (void *)(i + 1)))\n+\t\t\t\tksft_exit_fail_perror(\"pthread_create()\");\n+\t\t}\n+\n+\t\tend_ms = now_ms() + duration_s * 1000UL;\n+\t\tif (!strcmp(mode, \"stepped\")) {\n+\t\t\twhile (now_ms() \u003c end_ms \u0026\u0026 !corrupted) {\n+\t\t\t\tif (!khugepaged_full_pass(600))\n+\t\t\t\t\tksft_exit_fail_msg(\"khugepaged pass timed out\\n\");\n+\t\t\t\tsteps++;\n+\t\t\t}\n+\t\t} else if (!strcmp(mode, \"free\")) {\n+\t\t\twhile (now_ms() \u003c end_ms \u0026\u0026 !corrupted)\n+\t\t\t\tusleep(100 * 1000);\n+\t\t} else {\t/* madvise */\n+\t\t\twhile (now_ms() \u003c end_ms \u0026\u0026 !corrupted) {\n+\t\t\t\tfor (i = 0; i \u003c nr_shared_areas; i++) {\n+\t\t\t\t\tmadvise(region + i * hpage_pmd_size,\n+\t\t\t\t\t\thpage_pmd_size, MADV_COLLAPSE);\n+\t\t\t\t}\n+\t\t\t\tmadvise(region, nr_shared_areas * hpage_pmd_size,\n+\t\t\t\t\tMADV_DONTNEED);\n+\t\t\t\tsteps++;\n+\t\t\t}\n+\t\t}\n+\n+\t\tstop = 1;\n+\t\tfor (i = 0; i \u003c nr_threads; i++) {\n+\t\t\tif (threads[i])\n+\t\t\t\tpthread_join(threads[i], NULL);\n+\t\t}\n+\n+\t\t/* Final integrity sweep. */\n+\t\tfor (i = 0; i \u003c nr_shared_areas * hpage_pmd_size / page_size; i++)\n+\t\t\tcheck_page(i);\n+\n+\t\tksft_test_result(!corrupted,\n+\t\t\t\t \"%s/%s%s: %ds, %d steps, no corruption\\n\",\n+\t\t\t\t mode, holes ? \"holes\" : \"strict\",\n+\t\t\t\t pressure ? \"/pressure\" : \"\",\n+\t\t\t\t duration_s, steps);\n+\n+\t\t/*\n+\t\t * Hand the address space and the settings back before the\n+\t\t * next mode: it maps the region at the same fixed address,\n+\t\t * and its scan cadence differs.\n+\t\t */\n+\t\tmunmap(region, nr_areas * hpage_pmd_size);\n+\t\tif (pageout_area) {\n+\t\t\tmunmap(pageout_area, pageout_size);\n+\t\t\tpageout_area = NULL;\n+\t\t}\n+\t\tthp_pop_settings();\n+\t\tstop = 0;\n+\t\tsteps = 0;\n+\n+\t\tif (corrupted) {\n+\t\t\t/* Memory is suspect; the rest would prove nothing. */\n+\t\t\twhile (++run \u003c nr_modes * nr_nones * nr_press) {\n+\t\t\t\trem = run % (nr_nones * nr_press);\n+\n+\t\t\t\tksft_test_result_skip(\"%s/%s%s: skipped after corruption\\n\",\n+\t\t\t\t\t\t modes[run / (nr_nones * nr_press)],\n+\t\t\t\t\t\t nones[rem / nr_press] ?\n+\t\t\t\t\t\t \"holes\" : \"strict\",\n+\t\t\t\t\t\t press[rem % nr_press] ?\n+\t\t\t\t\t\t \"/pressure\" : \"\");\n+\t\t\t}\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\n+\tthp_restore_settings();\n+\tksft_finished();\n+}\ndiff --git a/tools/testing/selftests/mm/khugepaged_sync_check.c b/tools/testing/selftests/mm/khugepaged_sync_check.c\nnew file mode 100644\nindex 0000000000000..30d3fb519fb2c\n--- /dev/null\n+++ b/tools/testing/selftests/mm/khugepaged_sync_check.c\n@@ -0,0 +1,217 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Synchronous khugepaged driving check.\n+ *\n+ * Race tests drive khugepaged through the existing sysfs controls: a\n+ * store to scan_sleep_millisecs wakes the daemon, and full_scans\n+ * advancing by two is a completion barrier for one full pass that\n+ * started after setup (khugepaged_full_pass()). Verify the pair gives\n+ * deterministic, attributable results: one barrier step over one\n+ * prepared window produces exactly one collapse attempt on that\n+ * window's source pages (mm_collapse_huge_page_isolate events filtered\n+ * by source PFN and order) and the window is collapsed\n+ * afterwards, repeatably.\n+ *\n+ * scan_sleep_millisecs is set to 60s to prove the wake path: without\n+ * the wake, one barrier step would sleep multiples of that and blow\n+ * the timeout. It also keeps the daemon from free-running between\n+ * steps, per the khugepaged_full_pass() discipline.\n+ */\n+#define _GNU_SOURCE\n+#include \u003cfcntl.h\u003e\n+#include \u003cstdio.h\u003e\n+#include \u003cstdlib.h\u003e\n+#include \u003cstring.h\u003e\n+#include \u003csys/mman.h\u003e\n+#include \u003cunistd.h\u003e\n+\n+#include \"kselftest.h\"\n+#include \"vm_util.h\"\n+#include \"hugepage_settings.h\"\n+\n+#define BASE_ADDR ((void *)(1UL \u003c\u003c 30))\n+#define TARGET_ORDER 2\t/* smallest order khugepaged considers */\n+#define NR_ITERATIONS 5\n+\n+static int pagemap_fd;\n+static int kpageflags_fd;\n+static int trace_events_fd = -1;\n+static unsigned long hpage_pmd_size;\n+\n+/*\n+ * Each step switches the events off again, but a helper can still give up\n+ * on us in between (a failing sysfs write ends the test from inside\n+ * thp_write_num()), and huge_memory events left on are the whole machine's\n+ * problem, not this test's.\n+ */\n+static void trace_events_off(void)\n+{\n+\tif (trace_events_fd \u003e= 0)\n+\t\ttracing_events_enable(trace_events_fd, false);\n+}\n+\n+/*\n+ * Count collapse attempts attributable to our window: legacy-engine\n+ * isolate events whose scan_pfn is one of the window's source PFNs,\n+ * plus batch-engine per-candidate install events at the window's\n+ * address. Either engine reports exactly once per attempt.\n+ */\n+static int count_attributed(unsigned long *pfns, int nr_pfns,\n+\t\t\t unsigned long addr, unsigned int order)\n+{\n+\tchar line[1024];\n+\tint count = 0;\n+\tFILE *fp;\n+\n+\tfp = tracing_open_trace();\n+\tif (!fp)\n+\t\tksft_exit_fail_msg(\"Cannot open trace buffer\\n\");\n+\n+\twhile (fgets(line, sizeof(line), fp)) {\n+\t\tchar *s;\n+\t\tunsigned long val;\n+\t\tunsigned int ord;\n+\t\tchar *o;\n+\t\tint i;\n+\n+\t\ts = strstr(line, \"mm_collapse_huge_page_isolate:\");\n+\t\tif (s) {\n+\t\t\tif (sscanf(s, \"mm_collapse_huge_page_isolate: scan_pfn=0x%lx\",\n+\t\t\t\t \u0026val) != 1)\n+\t\t\t\tcontinue;\n+\t\t\to = strstr(s, \"order=\");\n+\t\t\tif (!o || sscanf(o, \"order=%u\", \u0026ord) != 1 ||\n+\t\t\t ord != order)\n+\t\t\t\tcontinue;\n+\t\t\tfor (i = 0; i \u003c nr_pfns; i++) {\n+\t\t\t\tif (val == pfns[i]) {\n+\t\t\t\t\tcount++;\n+\t\t\t\t\tbreak;\n+\t\t\t\t}\n+\t\t\t}\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\ts = strstr(line, \"mm_collapse_candidate:\");\n+\t\tif (s) {\n+\t\t\tif (!strstr(s, \"pass=install\") ||\n+\t\t\t !strstr(s, \"result=succeeded\"))\n+\t\t\t\tcontinue;\n+\t\t\to = strstr(s, \"addr=\");\n+\t\t\tif (!o || sscanf(o, \"addr=0x%lx\", \u0026val) != 1 ||\n+\t\t\t val != addr)\n+\t\t\t\tcontinue;\n+\t\t\to = strstr(s, \"order=\");\n+\t\t\tif (!o || sscanf(o, \"order=%u\", \u0026ord) != 1 ||\n+\t\t\t ord != order)\n+\t\t\t\tcontinue;\n+\t\t\tcount++;\n+\t\t}\n+\t}\n+\tfclose(fp);\n+\treturn count;\n+}\n+\n+static void one_step(int iteration)\n+{\n+\tconst size_t window = getpagesize() \u003c\u003c TARGET_ORDER;\n+\tconst int nr_pages = 1 \u003c\u003c TARGET_ORDER;\n+\tunsigned long pfns[1 \u003c\u003c TARGET_ORDER];\n+\tbool collapsed, passed;\n+\tint attributed;\n+\tchar *p;\n+\tint i;\n+\n+\tp = mmap(BASE_ADDR, hpage_pmd_size, PROT_READ | PROT_WRITE,\n+\t\t MAP_ANONYMOUS | MAP_PRIVATE | MAP_FIXED_NOREPLACE, -1, 0);\n+\tif (p != BASE_ADDR)\n+\t\tksft_exit_fail_perror(\"mmap() window\");\n+\n+\t/* Prepare one window; record its source PFNs. */\n+\tfor (i = 0; i \u003c nr_pages; i++) {\n+\t\tp[i * getpagesize()] = i + 1;\n+\t\tpfns[i] = pagemap_get_pfn(pagemap_fd, p + i * getpagesize());\n+\t\tif (pfns[i] == -1UL)\n+\t\t\tksft_exit_fail_msg(\"Source page not present\\n\");\n+\t}\n+\n+\t/* Clear first: with the events still off there is nothing to undo. */\n+\tif (tracing_clear_trace())\n+\t\tksft_exit_fail_msg(\"Cannot clear the trace buffer\\n\");\n+\tif (tracing_events_enable(trace_events_fd, true))\n+\t\tksft_exit_fail_msg(\"Cannot enable huge_memory events\\n\");\n+\n+\tmadvise(p, hpage_pmd_size, MADV_HUGEPAGE);\n+\t/* Wait up to 120 seconds for the pass to complete. */\n+\tpassed = khugepaged_full_pass(120);\n+\n+\t/* Off before anything that can give up: the events are system-wide. */\n+\tif (tracing_events_enable(trace_events_fd, false))\n+\t\tksft_exit_fail_msg(\"Cannot disable huge_memory events\\n\");\n+\tif (!passed)\n+\t\tksft_exit_fail_msg(\"khugepaged did not complete a full pass\\n\");\n+\n+\tcollapsed = is_range_backed_by_folio_orders(p, window, TARGET_ORDER,\n+\t\t\t\t\t\t pagemap_fd, kpageflags_fd);\n+\tattributed = count_attributed(pfns, nr_pages, (unsigned long)p,\n+\t\t\t\t TARGET_ORDER);\n+\n+\tksft_test_result(collapsed \u0026\u0026 attributed == 1,\n+\t\t\t \"step %d: window collapsed, %d attributed result(s)\\n\",\n+\t\t\t iteration, attributed);\n+\n+\tmunmap(p, hpage_pmd_size);\n+}\n+\n+int main(void)\n+{\n+\tstruct thp_settings settings;\n+\tint i;\n+\n+\tksft_print_header();\n+\n+\tif (!thp_available())\n+\t\tksft_exit_skip(\"Transparent Hugepages not available\\n\");\n+\tif (!(thp_supported_orders() \u0026 (1UL \u003c\u003c TARGET_ORDER)))\n+\t\tksft_exit_skip(\"Order %d is not a supported anon THP order\\n\",\n+\t\t\t TARGET_ORDER);\n+\n+\thpage_pmd_size = read_pmd_pagesize();\n+\tif (!hpage_pmd_size)\n+\t\tksft_exit_fail_msg(\"Reading PMD pagesize failed\\n\");\n+\tpagemap_fd = open(\"/proc/self/pagemap\", O_RDONLY);\n+\tif (pagemap_fd \u003c 0)\n+\t\tksft_exit_fail_perror(\"open(/proc/self/pagemap)\");\n+\tkpageflags_fd = open(\"/proc/kpageflags\", O_RDONLY);\n+\tif (kpageflags_fd \u003c 0)\n+\t\tksft_exit_skip(\"open(\\\"/proc/kpageflags\\\") requires root\\n\");\n+\ttrace_events_fd = tracing_events_open(\"huge_memory\");\n+\tif (trace_events_fd \u003c 0)\n+\t\tksft_exit_skip(\"huge_memory events require tracefs and root\\n\");\n+\tatexit(trace_events_off);\n+\n+\tksft_set_plan(NR_ITERATIONS);\n+\n+\tthp_save_settings();\n+\tthp_read_settings(\u0026settings);\n+\tsettings.thp_enabled = THP_MADVISE;\n+\tsettings.thp_defrag = THP_DEFRAG_ALWAYS;\n+\tsettings.khugepaged.defrag = 1;\n+\tsettings.khugepaged.scan_sleep_millisecs = 60000;\n+\tsettings.khugepaged.alloc_sleep_millisecs = 60000;\n+\tsettings.khugepaged.max_ptes_none = (hpage_pmd_size / getpagesize()) - 1;\n+\t/* One wake must complete one full pass; see khugepaged_full_pass(). */\n+\tsettings.khugepaged.pages_to_scan = 1UL \u003c\u003c 24;\n+\tfor (i = 0; i \u003c NR_ORDERS; i++)\n+\t\tsettings.hugepages[i].enabled = THP_NEVER;\n+\tsettings.hugepages[TARGET_ORDER].enabled = THP_INHERIT;\n+\t/* Base of the settings stack; the bottom entry is never popped. */\n+\tthp_push_settings(\u0026settings);\n+\n+\tfor (i = 0; i \u003c NR_ITERATIONS; i++)\n+\t\tone_step(i);\n+\n+\tthp_restore_settings();\n+\n+\tksft_finished();\n+}\ndiff --git a/tools/testing/selftests/mm/migration.c b/tools/testing/selftests/mm/migration.c\nindex f19d53c695764..fd35f8a7b5b83 100644\n--- a/tools/testing/selftests/mm/migration.c\n+++ b/tools/testing/selftests/mm/migration.c\n@@ -20,7 +20,6 @@\n \n #define TWOMEG\t\t(2\u003c\u003c20)\n #define RUNTIME\t\t(20)\n-#define ALIGN(x, a)\t(((x) + (a - 1)) \u0026 (~((a) - 1)))\n \n HUGETLB_SETUP_DEFAULT_PAGES(1)\n \ndiff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh\nindex d09f9f6a384ee..fc61907aa3b2a 100755\n--- a/tools/testing/selftests/mm/run_vmtests.sh\n+++ b/tools/testing/selftests/mm/run_vmtests.sh\n@@ -402,6 +402,12 @@ CATEGORY=\"pfnmap\" run_test ./pfnmap\n # COW tests\n CATEGORY=\"cow\" run_test ./cow\n \n+CATEGORY=\"thp\" run_test ./folio_order_check\n+\n+CATEGORY=\"thp\" run_test ./khugepaged_sync_check\n+\n+CATEGORY=\"thp\" run_test ./khugepaged_race\n+\n CATEGORY=\"thp\" run_test ./khugepaged\n \n CATEGORY=\"thp\" run_test ./khugepaged -s 2\n@@ -410,8 +416,6 @@ CATEGORY=\"thp\" run_test ./khugepaged all:shmem\n \n CATEGORY=\"thp\" run_test ./khugepaged -s 4 all:shmem\n \n-CATEGORY=\"thp\" run_test ./khugepaged -c 4 mthp_khugepaged:anon\n-\n # Try to create XFS if not provided\n if [ -z \"${SPLIT_HUGE_PAGE_TEST_XFS_PATH}\" ]; then\n if test_selected \"thp\"; then\ndiff --git a/tools/testing/selftests/mm/split_huge_page_test.c b/tools/testing/selftests/mm/split_huge_page_test.c\nindex 86a6036928261..0adfe7dde7e59 100644\n--- a/tools/testing/selftests/mm/split_huge_page_test.c\n+++ b/tools/testing/selftests/mm/split_huge_page_test.c\n@@ -42,68 +42,6 @@ const char *kpageflags_proc = \"/proc/kpageflags\";\n int pagemap_fd;\n int kpageflags_fd;\n \n-static bool is_backed_by_folio(char *vaddr, int order, int pagemap_fd,\n-\t\tint kpageflags_fd)\n-{\n-\tconst uint64_t folio_head_flags = KPF_THP | KPF_COMPOUND_HEAD;\n-\tconst uint64_t folio_tail_flags = KPF_THP | KPF_COMPOUND_TAIL;\n-\tconst unsigned long nr_pages = 1UL \u003c\u003c order;\n-\tunsigned long pfn_head;\n-\tuint64_t pfn_flags;\n-\tunsigned long pfn;\n-\tunsigned long i;\n-\n-\tpfn = pagemap_get_pfn(pagemap_fd, vaddr);\n-\n-\t/* non present page */\n-\tif (pfn == -1UL)\n-\t\treturn false;\n-\n-\tif (pageflags_get(pfn, kpageflags_fd, \u0026pfn_flags))\n-\t\tgoto fail;\n-\n-\t/* check for order-0 pages */\n-\tif (!order) {\n-\t\tif (pfn_flags \u0026 (folio_head_flags | folio_tail_flags))\n-\t\t\treturn false;\n-\t\treturn true;\n-\t}\n-\n-\t/* non THP folio */\n-\tif (!(pfn_flags \u0026 KPF_THP))\n-\t\treturn false;\n-\n-\tpfn_head = pfn \u0026 ~(nr_pages - 1);\n-\n-\tif (pageflags_get(pfn_head, kpageflags_fd, \u0026pfn_flags))\n-\t\tgoto fail;\n-\n-\t/* head PFN has no compound_head flag set */\n-\tif ((pfn_flags \u0026 folio_head_flags) != folio_head_flags)\n-\t\treturn false;\n-\n-\t/* check all tail PFN flags */\n-\tfor (i = 1; i \u003c nr_pages; i++) {\n-\t\tif (pageflags_get(pfn_head + i, kpageflags_fd, \u0026pfn_flags))\n-\t\t\tgoto fail;\n-\t\tif ((pfn_flags \u0026 folio_tail_flags) != folio_tail_flags)\n-\t\t\treturn false;\n-\t}\n-\n-\t/*\n-\t * check the PFN after this folio, but if its flags cannot be obtained,\n-\t * assume this folio has the expected order\n-\t */\n-\tif (pageflags_get(pfn_head + nr_pages, kpageflags_fd, \u0026pfn_flags))\n-\t\treturn true;\n-\n-\t/* If we find another tail page, then the folio is larger. */\n-\treturn (pfn_flags \u0026 folio_tail_flags) != folio_tail_flags;\n-fail:\n-\tksft_exit_fail_msg(\"Failed to get folio info\\n\");\n-\treturn false;\n-}\n-\n static int check_after_split_folio_orders(char *vaddr_start, size_t len,\n \t\tint pagemap_fd, int kpageflags_fd, int orders[], int nr_orders)\n {\ndiff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c\nindex 80bc9f597b521..ee1334778391f 100644\n--- a/tools/testing/selftests/mm/vm_util.c\n+++ b/tools/testing/selftests/mm/vm_util.c\n@@ -494,6 +494,151 @@ int pageflags_get(unsigned long pfn, int kpageflags_fd, uint64_t *flags)\n \treturn 0;\n }\n \n+bool is_backed_by_folio(char *vaddr, int order, int pagemap_fd,\n+\t\t\tint kpageflags_fd)\n+{\n+\tconst uint64_t folio_head_flags = KPF_THP | KPF_COMPOUND_HEAD;\n+\tconst uint64_t folio_tail_flags = KPF_THP | KPF_COMPOUND_TAIL;\n+\tconst unsigned long nr_pages = 1UL \u003c\u003c order;\n+\tunsigned long pfn_head;\n+\tuint64_t pfn_flags;\n+\tunsigned long pfn;\n+\tunsigned long i;\n+\n+\tpfn = pagemap_get_pfn(pagemap_fd, vaddr);\n+\n+\t/* non present page */\n+\tif (pfn == -1UL)\n+\t\treturn false;\n+\n+\tif (pageflags_get(pfn, kpageflags_fd, \u0026pfn_flags))\n+\t\tgoto fail;\n+\n+\t/* check for order-0 pages */\n+\tif (!order) {\n+\t\tif (pfn_flags \u0026 (folio_head_flags | folio_tail_flags))\n+\t\t\treturn false;\n+\t\treturn true;\n+\t}\n+\n+\t/* non THP folio */\n+\tif (!(pfn_flags \u0026 KPF_THP))\n+\t\treturn false;\n+\n+\tpfn_head = pfn \u0026 ~(nr_pages - 1);\n+\n+\tif (pageflags_get(pfn_head, kpageflags_fd, \u0026pfn_flags))\n+\t\tgoto fail;\n+\n+\t/* head PFN has no compound_head flag set */\n+\tif ((pfn_flags \u0026 folio_head_flags) != folio_head_flags)\n+\t\treturn false;\n+\n+\t/* check all tail PFN flags */\n+\tfor (i = 1; i \u003c nr_pages; i++) {\n+\t\tif (pageflags_get(pfn_head + i, kpageflags_fd, \u0026pfn_flags))\n+\t\t\tgoto fail;\n+\t\tif ((pfn_flags \u0026 folio_tail_flags) != folio_tail_flags)\n+\t\t\treturn false;\n+\t}\n+\n+\t/*\n+\t * check the PFN after this folio, but if its flags cannot be obtained,\n+\t * assume this folio has the expected order\n+\t */\n+\tif (pageflags_get(pfn_head + nr_pages, kpageflags_fd, \u0026pfn_flags))\n+\t\treturn true;\n+\n+\t/* If we find another tail page, then the folio is larger. */\n+\treturn (pfn_flags \u0026 folio_tail_flags) != folio_tail_flags;\n+fail:\n+\tksft_exit_fail_msg(\"Failed to get folio info\\n\");\n+\treturn false;\n+}\n+\n+/*\n+ * Check whether every order-@order window of [start, len) maps exactly one\n+ * folio of that order, head to tail. The address range must be naturally\n+ * aligned, each window's PFN run must be contiguous, and a window's first\n+ * PFN must be the folio head.\n+ *\n+ * This is the check \"did this range collapse into order-@order folios\": a\n+ * window assembled from parts of several folios, or mapping a folio shifted\n+ * from its natural position, fails.\n+ */\n+bool is_range_backed_by_folio_orders(char *start, size_t len, int order,\n+\t\t\t\t int pagemap_fd, int kpageflags_fd)\n+{\n+\tconst unsigned long nr_pages = 1UL \u003c\u003c order;\n+\tconst size_t window = nr_pages * psize();\n+\tchar *vaddr;\n+\n+\tif ((uintptr_t)start % window || len % window)\n+\t\treturn false;\n+\n+\tfor (vaddr = start; vaddr \u003c start + len; vaddr += window) {\n+\t\tunsigned long pfn = pagemap_get_pfn(pagemap_fd, vaddr);\n+\t\tunsigned long i;\n+\n+\t\t/* Not present, or not mapping the folio head. */\n+\t\tif (pfn == -1UL || pfn % nr_pages)\n+\t\t\treturn false;\n+\n+\t\tfor (i = 1; i \u003c nr_pages; i++) {\n+\t\t\tif (pagemap_get_pfn(pagemap_fd, vaddr + i * psize()) !=\n+\t\t\t pfn + i)\n+\t\t\t\treturn false;\n+\t\t}\n+\n+\t\tif (!is_backed_by_folio(vaddr, order, pagemap_fd,\n+\t\t\t\t\tkpageflags_fd))\n+\t\t\treturn false;\n+\t}\n+\n+\treturn true;\n+}\n+\n+#define TRACEFS_ROOT \"/sys/kernel/tracing\"\n+\n+/*\n+ * Open the enable file of one ftrace event subsystem (e.g. \"huge_memory\").\n+ * Returns a descriptor for tracing_events_enable(), or -1 if tracefs or the\n+ * subsystem is not there. The events are system-wide state: whoever\n+ * switches them on owns them until it switches them off, including on the\n+ * paths where the test gives up.\n+ */\n+int tracing_events_open(const char *subsys)\n+{\n+\tchar path[256];\n+\n+\tsnprintf(path, sizeof(path), TRACEFS_ROOT \"/events/%s/enable\",\n+\t\t subsys);\n+\treturn open(path, O_WRONLY);\n+}\n+\n+int tracing_events_enable(int fd, bool enable)\n+{\n+\tif (pwrite(fd, enable ? \"1\" : \"0\", 1, 0) != 1)\n+\t\treturn -1;\n+\treturn 0;\n+}\n+\n+/* Drop what the trace buffer holds so far. */\n+int tracing_clear_trace(void)\n+{\n+\tint fd = open(TRACEFS_ROOT \"/trace\", O_WRONLY | O_TRUNC);\n+\n+\tif (fd \u003c 0)\n+\t\treturn -1;\n+\tclose(fd);\n+\treturn 0;\n+}\n+\n+FILE *tracing_open_trace(void)\n+{\n+\treturn fopen(TRACEFS_ROOT \"/trace\", \"r\");\n+}\n+\n /* If `ioctls' non-NULL, the allowed ioctls will be returned into the var */\n int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,\n \t\t\t bool miss, bool wp, bool minor, uint64_t *ioctls)\ndiff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h\nindex 9a49af88702e4..10c7be46e44ca 100644\n--- a/tools/testing/selftests/mm/vm_util.h\n+++ b/tools/testing/selftests/mm/vm_util.h\n@@ -10,6 +10,8 @@\n #include \u003clinux/fs.h\u003e\n \n #define BIT_ULL(nr) (1ULL \u003c\u003c (nr))\n+#define ALIGN(x, a) (((x) + (a) - 1) \u0026 ~((a) - 1))\n+\n #define PM_SOFT_DIRTY BIT_ULL(55)\n #define PM_MMAP_EXCLUSIVE BIT_ULL(56)\n #define PM_UFFD_WP BIT_ULL(57)\n@@ -97,6 +99,10 @@ int64_t allocate_transhuge(void *ptr, int pagemap_fd);\n int pageflags_get(unsigned long pfn, int kpageflags_fd, uint64_t *flags);\n int gather_folio_orders(char *vaddr_start, size_t len,\n \t\tint pagemap_fd, int kpageflags_fd, int orders[], int nr_orders);\n+bool is_backed_by_folio(char *vaddr, int order, int pagemap_fd,\n+\t\t\tint kpageflags_fd);\n+bool is_range_backed_by_folio_orders(char *start, size_t len, int order,\n+\t\t\t\t int pagemap_fd, int kpageflags_fd);\n \n int uffd_register(int uffd, void *addr, uint64_t len,\n \t\t bool miss, bool wp, bool minor);\n@@ -113,6 +119,10 @@ int close_procmap(struct procmap_fd *procmap);\n int write_sysfs(const char *file_path, unsigned long val);\n int read_sysfs(const char *file_path, unsigned long *val);\n bool softdirty_supported(void);\n+int tracing_events_open(const char *subsys);\n+int tracing_events_enable(int fd, bool enable);\n+int tracing_clear_trace(void);\n+FILE *tracing_open_trace(void);\n \n static inline int open_self_procmap(struct procmap_fd *procmap_out)\n {\n"
} |
| 2/1 | 2026/08/12 14:59 | agent | patch-evaluator |
0mModel:best-expensive Args: null Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"Reasoning": "The patch only modifies userspace selftests in tools/testing/selftests/mm/, which do not affect the kernel's functional execution flow.",
"WorthFuzzing": false
}Instruction: You are an expert Linux kernel maintainer. Your job is to review a provided patch series and determine if it makes functional changes to the kernel that should be fuzzed. IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in your workspace. Do NOT rely on your internal knowledge of the kernel. You must actively use your code access tools to examine the actual source code and confirm any assumptions. Return WorthFuzzing=false if the patch only contains: - Modifications to Documentation/, Kconfig files, or code comments. - Purely decorative changes, such as logging (e.g., pr_err, printk) or tracepoints. - Changes to numeric constants or macros that do not functionally alter execution flow. - Code paths that are impossible to reach in virtualized environments like GCE or QEMU, even when utilizing software-emulated hardware (e.g., usb gadget, mac80211_hwsim). - Code in vendor-specific PCIe switch, SmartNIC, or GPU drivers (e.g., mlxsw, pds_core, qed, ionic, amdgpu) that require physical PCIe hardware cards not emulated in standard QEMU. - Driver .remove, .shutdown, or pci_unregister_driver teardown callbacks (e.g., igb_remove) that are executed only during PCI hot-unplug or sysfs driver unbind operations. If it modifies reachable core kernel logic, drivers, or architectures, use your code search tools to verify the code can be executed, then return WorthFuzzing=true. When returning WorthFuzzing=true, you MUST ALSO: 1. Extract any specific kernel functions that should be heavily fuzzed into FocusSymbols. Avoid listing generic hot-path functions to prevent skewed test distributions. Prefer non-static, non-inlined API entrypoint functions over internal static helper functions (which are inlined by the compiler and do not have distinct symbol addresses). 2. Identify any specific CONFIG_ options required to properly test this new/modified feature. Go and look into the Kconfig files and check for ifdefs around the code, do not make assumptions. Also check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed for Kbuild to compile the code statically into vmlinux. List them in the EnableConfigs output array, and DO NOT add a 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4"). Prefer calling several tools at the same time to save round-trips. Use set-results tool to provide results of the analysis. It must be called exactly once before the final reply. Ignore results of this tool. Prompt: For your convenience, here is the diff of the changes:
commit 67a19e17a91b615d0071c3d8410382e81c0e9088
Author: syz-cluster <triage@syzkaller.com>
Date: Wed Aug 12 14:59:10 2026 +0000
syz-cluster: applied patch under review
diff --git a/tools/testing/selftests/mm/Makefile b/tools/testing/selftests/mm/Makefile
index 2d5366196e309..308bbad73c11a 100644
--- a/tools/testing/selftests/mm/Makefile
+++ b/tools/testing/selftests/mm/Makefile
@@ -104,6 +104,9 @@ TEST_GEN_FILES += guard-regions
TEST_GEN_FILES += merge
TEST_GEN_FILES += rmap
TEST_GEN_FILES += folio_split_race_test
+TEST_GEN_FILES += folio_order_check
+TEST_GEN_FILES += khugepaged_sync_check
+TEST_GEN_FILES += khugepaged_race
ifneq ($(ARCH),arm64)
TEST_GEN_FILES += soft-dirty
diff --git a/tools/testing/selftests/mm/folio_order_check.c b/tools/testing/selftests/mm/folio_order_check.c
new file mode 100644
index 0000000000000..93030a42c3ccd
--- /dev/null
+++ b/tools/testing/selftests/mm/folio_order_check.c
@@ -0,0 +1,137 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Self-check for the vm_util folio-order detection helpers,
+ * is_backed_by_folio() and is_range_backed_by_folio_orders().
+ *
+ * For every anon THP order the kernel supports, fault memory in with only
+ * that order enabled and verify the helpers report exactly that order:
+ * not a neighbouring order, and plain 4K memory as order 0. The helpers
+ * are what the khugepaged mTHP tests use to detect collapse results, so
+ * they must agree with the kernel's own idea of the backing before any
+ * collapse test relies on them.
+ */
+#define _GNU_SOURCE
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/mman.h>
+#include <unistd.h>
+
+#include "kselftest.h"
+#include "vm_util.h"
+#include "hugepage_settings.h"
+
+static int pagemap_fd;
+static int kpageflags_fd;
+
+/* mmap an anon VMA of exactly @size bytes at a @size-aligned address. */
+static char *alloc_aligned(size_t size)
+{
+ size_t len = size * 2;
+ uintptr_t aligned;
+ char *p;
+
+ p = mmap(NULL, len, PROT_READ | PROT_WRITE,
+ MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+ if (p == MAP_FAILED)
+ ksft_exit_fail_perror("mmap()");
+
+ aligned = ALIGN((uintptr_t)p, size);
+ if (aligned != (uintptr_t)p)
+ munmap(p, aligned - (uintptr_t)p);
+ if (aligned + size != (uintptr_t)p + len)
+ munmap((char *)aligned + size,
+ (uintptr_t)p + len - aligned - size);
+
+ return (char *)aligned;
+}
+
+/*
+ * Enable only @order (order 0: nothing), fault one aligned window in and
+ * check the helpers see exactly @order.
+ */
+static void check_order(int order)
+{
+ struct thp_settings settings = *thp_current_settings();
+ size_t size = psize() << order;
+ bool ok = true;
+ char *p;
+ int i;
+
+ for (i = 0; i < NR_ORDERS; i++)
+ settings.hugepages[i].enabled = THP_NEVER;
+ if (order)
+ settings.hugepages[order].enabled = THP_ALWAYS;
+ thp_push_settings(&settings);
+
+ p = alloc_aligned(size);
+ *p = 1;
+
+ if (!is_range_backed_by_folio_orders(p, size, order,
+ pagemap_fd, kpageflags_fd)) {
+ ksft_print_msg("order %d not detected after fault\n", order);
+ ok = false;
+ }
+
+ /* A lower order must be rejected: the folio is larger. */
+ if (order && is_range_backed_by_folio_orders(p, size, order - 1,
+ pagemap_fd,
+ kpageflags_fd)) {
+ ksft_print_msg("order %d also reported as order %d\n",
+ order, order - 1);
+ ok = false;
+ }
+
+ /* Order 0 pages must not look like any large folio, and vice versa. */
+ if (order && is_range_backed_by_folio_orders(p, size, 0,
+ pagemap_fd,
+ kpageflags_fd)) {
+ ksft_print_msg("order %d also reported as order 0\n", order);
+ ok = false;
+ }
+
+ munmap(p, size);
+ thp_pop_settings();
+
+ ksft_test_result(ok, "order %d classified\n", order);
+}
+
+int main(void)
+{
+ struct thp_settings settings;
+ unsigned long orders;
+ int order;
+
+ ksft_print_header();
+
+ if (!thp_available())
+ ksft_exit_skip("Transparent Hugepages not available\n");
+
+ pagemap_fd = open("/proc/self/pagemap", O_RDONLY);
+ if (pagemap_fd < 0)
+ ksft_exit_fail_perror("open(/proc/self/pagemap)");
+ kpageflags_fd = open("/proc/kpageflags", O_RDONLY);
+ if (kpageflags_fd < 0)
+ ksft_exit_skip("open(\"/proc/kpageflags\") requires root\n");
+
+ orders = thp_supported_orders();
+ if (!orders)
+ ksft_exit_skip("No supported THP orders\n");
+
+ ksft_set_plan(__builtin_popcountl(orders) + 1);
+
+ thp_save_settings();
+ thp_read_settings(&settings);
+ /* Base of the settings stack; the bottom entry is never popped. */
+ thp_push_settings(&settings);
+
+ check_order(0);
+ for (order = 1; order < NR_ORDERS; order++) {
+ if (!(orders & (1UL << order)))
+ continue;
+ check_order(order);
+ }
+
+
+ ksft_finished();
+}
diff --git a/tools/testing/selftests/mm/hmm-tests.c b/tools/testing/selftests/mm/hmm-tests.c
index e2642eca0d02b..df426f9218e71 100644
--- a/tools/testing/selftests/mm/hmm-tests.c
+++ b/tools/testing/selftests/mm/hmm-tests.c
@@ -65,7 +65,6 @@ enum {
#define HMM_PATH_MAX 64
#define NTIMES 10
-#define ALIGN(x, a) (((x) + (a - 1)) & (~((a) - 1)))
/* Just the flags we need, copied from mm.h: */
#ifndef FOLL_WRITE
diff --git a/tools/testing/selftests/mm/hugepage_settings.c b/tools/testing/selftests/mm/hugepage_settings.c
index d7917dce3abac..8afcdf9793bb7 100644
--- a/tools/testing/selftests/mm/hugepage_settings.c
+++ b/tools/testing/selftests/mm/hugepage_settings.c
@@ -183,6 +183,17 @@ void thp_read_settings(struct thp_settings *settings)
}
}
+/*
+ * Write only on change: any store to a khugepaged sysfs knob wakes the
+ * daemon, and settings pushes/pops must not start scan passes nobody
+ * asked for -- khugepaged_full_pass() is the only sanctioned wake.
+ */
+void thp_update_num(const char *name, unsigned long num)
+{
+ if (thp_read_num(name) != num)
+ thp_write_num(name, num);
+}
+
void thp_write_settings(struct thp_settings *settings)
{
struct khugepaged_settings *khugepaged = &settings->khugepaged;
@@ -198,15 +209,15 @@ void thp_write_settings(struct thp_settings *settings)
shmem_enabled_strings[settings->shmem_enabled]);
thp_write_num("use_zero_page", settings->use_zero_page);
- thp_write_num("khugepaged/defrag", khugepaged->defrag);
- thp_write_num("khugepaged/alloc_sleep_millisecs",
- khugepaged->alloc_sleep_millisecs);
- thp_write_num("khugepaged/scan_sleep_millisecs",
- khugepaged->scan_sleep_millisecs);
- thp_write_num("khugepaged/max_ptes_none", khugepaged->max_ptes_none);
- thp_write_num("khugepaged/max_ptes_swap", khugepaged->max_ptes_swap);
- thp_write_num("khugepaged/max_ptes_shared", khugepaged->max_ptes_shared);
- thp_write_num("khugepaged/pages_to_scan", khugepaged->pages_to_scan);
+ thp_update_num("khugepaged/defrag", khugepaged->defrag);
+ thp_update_num("khugepaged/alloc_sleep_millisecs",
+ khugepaged->alloc_sleep_millisecs);
+ thp_update_num("khugepaged/scan_sleep_millisecs",
+ khugepaged->scan_sleep_millisecs);
+ thp_update_num("khugepaged/max_ptes_none", khugepaged->max_ptes_none);
+ thp_update_num("khugepaged/max_ptes_swap", khugepaged->max_ptes_swap);
+ thp_update_num("khugepaged/max_ptes_shared", khugepaged->max_ptes_shared);
+ thp_update_num("khugepaged/pages_to_scan", khugepaged->pages_to_scan);
if (dev_queue_read_ahead_path[0])
write_num(dev_queue_read_ahead_path, settings->read_ahead_kb);
@@ -230,6 +241,49 @@ void thp_write_settings(struct thp_settings *settings)
}
}
+/*
+ * Completion barrier for khugepaged: wait until a full scan pass that
+ * started after this call has finished. full_scans must advance by two;
+ * a +1 step may complete a pass that examined this mm before the
+ * caller's setup was in place.
+ *
+ * Any store to scan_sleep_millisecs wakes the daemon, so the barrier works
+ * whatever the configured scan cadence -- but a store can be lost.
+ * __sleep_millisecs_store() clears khugepaged_sleep_expire and wakes the
+ * queue; if the daemon is between scans rather than sleeping, it sets
+ * khugepaged_sleep_expire itself on the way into khugepaged_wait_work() and
+ * then sleeps for the full interval, having never seen the store. So keep
+ * storing until the pass lands; a store while the daemon is awake costs
+ * nothing and does not queue an extra pass.
+ *
+ * One wake completes one full pass only if the whole mm list fits in
+ * one scan batch, so callers must pair this with a large
+ * pages_to_scan.
+ */
+bool khugepaged_full_pass(unsigned int timeout_s)
+{
+ unsigned long deadline_ms = timeout_s * 1000UL;
+ unsigned long sleep_ms =
+ thp_read_num("khugepaged/scan_sleep_millisecs");
+ unsigned long elapsed_ms = 0;
+ int pass;
+
+ for (pass = 0; pass < 2; pass++) {
+ unsigned long target =
+ thp_read_num("khugepaged/full_scans") + 1;
+
+ while (thp_read_num("khugepaged/full_scans") < target) {
+ if (elapsed_ms >= deadline_ms)
+ return false;
+ thp_write_num("khugepaged/scan_sleep_millisecs",
+ sleep_ms);
+ usleep(10 * 1000);
+ elapsed_ms += 10;
+ }
+ }
+ return true;
+}
+
struct thp_settings *thp_current_settings(void)
{
if (!settings_index) {
diff --git a/tools/testing/selftests/mm/hugepage_settings.h b/tools/testing/selftests/mm/hugepage_settings.h
index 726c73c43c05b..ba7d38370d433 100644
--- a/tools/testing/selftests/mm/hugepage_settings.h
+++ b/tools/testing/selftests/mm/hugepage_settings.h
@@ -70,6 +70,7 @@ int thp_read_string(const char *name, const char * const strings[]);
void thp_write_string(const char *name, const char *val);
unsigned long thp_read_num(const char *name);
void thp_write_num(const char *name, unsigned long num);
+void thp_update_num(const char *name, unsigned long num);
void thp_write_settings(struct thp_settings *settings);
void thp_read_settings(struct thp_settings *settings);
@@ -83,6 +84,8 @@ static inline void thp_save_settings(void)
hugepage_save_settings(/* thp = */ true, /* hugetlb = */ false);
}
+bool khugepaged_full_pass(unsigned int timeout_s);
+
void thp_set_read_ahead_path(char *path);
unsigned long thp_supported_orders(void);
unsigned long thp_shmem_supported_orders(void);
diff --git a/tools/testing/selftests/mm/khugepaged.c b/tools/testing/selftests/mm/khugepaged.c
index 0d6c71ed2fae3..f5773d4475427 100644
--- a/tools/testing/selftests/mm/khugepaged.c
+++ b/tools/testing/selftests/mm/khugepaged.c
@@ -31,6 +31,11 @@ static unsigned long page_size;
static int hpage_pmd_nr;
static int anon_order;
static int collapse_order;
+static bool collapse_order_given;
+static int collapse_orders[NR_ORDERS];
+static int nr_collapse_orders;
+static int pagemap_fd = -1;
+static int kpageflags_fd = -1;
#define PID_SMAPS "/proc/self/smaps"
#define TEST_FILE "collapse_test_file"
@@ -241,6 +246,41 @@ static bool check_swap(void *addr, unsigned long size)
return swap;
}
+/*
+ * Page the range out and wait for the swap count to say so.
+ *
+ * Two things get in the way. MADV_PAGEOUT is best effort:
+ * shrink_folio_list() leaves a folio alone when it cannot reclaim it right
+ * away, and one still under writeback from an earlier pageout is the common
+ * case, so the count the caller asks for arrives a moment later. And a range
+ * an earlier collapse left MADV_HUGEPAGE is one khugepaged is still working
+ * on: collapsing a range with up to max_ptes_swap pages swapped out means
+ * reading those pages back in, so the daemon undoes the pageout as fast as it
+ * is asked for. Keep the range out of its reach; the collapse the caller runs
+ * next puts MADV_HUGEPAGE back.
+ *
+ * Failing to get the pages out is the machine's answer, not the kernel's --
+ * swap too small, swap full, a memcg cap, a folio still under writeback -- so
+ * callers skip rather than fail. An error from madvise() is different, and
+ * ends the run here.
+ */
+static bool swapout_range(void *p, unsigned long size)
+{
+ int i;
+
+ if (madvise(p, size, MADV_NOHUGEPAGE))
+ ksft_exit_fail_perror("madvise(MADV_NOHUGEPAGE)");
+
+ for (i = 0; i < 40; i++) {
+ if (madvise(p, size, MADV_PAGEOUT))
+ ksft_exit_fail_perror("madvise(MADV_PAGEOUT)");
+ if (check_swap(p, size))
+ return true;
+ usleep(50 * 1000);
+ }
+ return false;
+}
+
static void *alloc_mapping(int nr)
{
void *p;
@@ -583,8 +623,10 @@ static bool wait_for_scan(const char *msg, char *p, size_t len,
int nr_hpages, int collap_order, struct mem_ops *ops)
{
unsigned long hpage_size = page_size << collap_order;
+ /* Three seconds as a floor, plus a second per 128M to collapse */
+ const unsigned long bytes = (unsigned long)nr_hpages * hpage_size;
+ int timeout = 6 + 2 * (bytes / (128UL << 20));
int full_scans;
- int timeout = 6; /* 3 seconds */
/* Sanity check */
if (!ops->check_huge(p, len, 0, hpage_size))
@@ -853,12 +895,10 @@ static void collapse_swapin_single_pte(struct collapse_context *c, struct mem_op
p = ops->setup_area(1);
ops->fault(p, 0, hpage_pmd_size);
- if (madvise(p, page_size, MADV_PAGEOUT))
- ksft_exit_fail_perror("madvise(MADV_PAGEOUT)");
- if (check_swap(p, page_size)) {
+ if (swapout_range(p, page_size)) {
success("OK");
} else {
- fail("Fail");
+ skip("Could not swap out");
goto out;
}
@@ -885,12 +925,10 @@ static void collapse_max_ptes_swap(struct collapse_context *c, struct mem_ops *o
p = ops->setup_area(1);
ops->fault(p, 0, hpage_pmd_size);
- if (madvise(p, (max_ptes_swap + 1) * page_size, MADV_PAGEOUT))
- ksft_exit_fail_perror("madvise(MADV_PAGEOUT)");
- if (check_swap(p, (max_ptes_swap + 1) * page_size)) {
+ if (swapout_range(p, (max_ptes_swap + 1) * page_size)) {
success("OK");
} else {
- fail("Fail");
+ skip("Could not swap out");
goto out;
}
@@ -902,12 +940,10 @@ static void collapse_max_ptes_swap(struct collapse_context *c, struct mem_ops *o
ops->fault(p, 0, hpage_pmd_size);
ksft_print_msg("Swapout %d of %d pages...", max_ptes_swap,
hpage_pmd_nr);
- if (madvise(p, max_ptes_swap * page_size, MADV_PAGEOUT))
- ksft_exit_fail_perror("madvise(MADV_PAGEOUT)");
- if (check_swap(p, max_ptes_swap * page_size)) {
+ if (swapout_range(p, max_ptes_swap * page_size)) {
success("OK");
} else {
- fail("Fail");
+ skip("Could not swap out");
goto out;
}
@@ -974,6 +1010,16 @@ static void collapse_compound_extreme(struct collapse_context *c, struct mem_ops
void *p;
int i;
+ /*
+ * The test needs hpage_pmd_nr PMD-order allocations, which is likely to
+ * fail for large PMD sizes. Skip if the PMD size is over 32M.
+ */
+ if (hpage_pmd_size > (32UL << 20)) {
+ ksft_test_result_skip("%s: PMD too large for fault-time THP construction\n",
+ __func__);
+ return;
+ }
+
p = ops->setup_area(1);
ksft_print_msg("Construct PTE page table full of different PTE-mapped compound pages\n");
for (i = 0; i < hpage_pmd_nr; i++) {
@@ -1164,6 +1210,72 @@ static void collapse_max_ptes_shared(struct collapse_context *c, struct mem_ops
ksft_test_result_report(exit_status, "%s\n", __func__);
}
+/*
+ * Content stays isolated while a co-sharer writes concurrently. A shared
+ * source is copied live (not frozen), relying on it being CoW - immutable
+ * for the duration of the copy; a co-sharer's write goes to a CoW copy. The
+ * collapsing child must see the pre-fork content, the writing parent only
+ * its own writes.
+ */
+static void collapse_fork_cow_race(struct collapse_context *c, struct mem_ops *ops)
+{
+ const unsigned long shared = 64 * page_size;
+ const int stride = page_size / sizeof(int);
+ int wstatus, child_status, i, n = shared / page_size;
+ /* volatile: the loop below must really store, on every iteration */
+ volatile int *ip;
+ void *p;
+
+ p = ops->setup_area(1);
+ ip = p;
+ ops->fault(p, 0, shared); /* shared prefix, pre-fork pattern */
+
+ ksft_print_msg("Fork, collapse in the child while the parent rewrites...");
+ if (!fork()) {
+ int collapse_status;
+
+ ops->fault(p, shared, hpage_pmd_size); /* private remainder */
+ c->collapse("Collapse a range shared with a writing co-sharer",
+ p, 1, ops, true);
+ collapse_status = exit_status;
+ for (i = 0; i < n; i++)
+ if (ip[i * stride] != i + 0xdead0000)
+ break;
+ if (i == n)
+ success("OK");
+ else
+ fail("Fail: child content");
+ /* The content check must not bury a failed collapse. */
+ if (exit_status != KSFT_FAIL)
+ exit_status = collapse_status;
+ ops->cleanup_area(p, hpage_pmd_size);
+ _exit(exit_status);
+ }
+
+ /* Hammer the parent's own writes over the shared prefix. */
+ for (int it = 0; it < 200000; it++)
+ for (i = 0; i < n; i++)
+ ip[i * stride] = i + 0xbeef0000;
+
+ wait(&wstatus);
+ /* A child that died reading the racing pages is a failure, not a zero. */
+ child_status = WIFEXITED(wstatus) ? WEXITSTATUS(wstatus) : KSFT_FAIL;
+
+ ksft_print_msg("Check the parent sees only its own writes...");
+ for (i = 0; i < n; i++)
+ if (ip[i * stride] != i + 0xbeef0000)
+ break;
+ if (i == n)
+ success("OK");
+ else
+ fail("Fail: parent content");
+ ops->cleanup_area(p, hpage_pmd_size);
+ /* Same again: our own check must not bury the child's verdict. */
+ if (exit_status != KSFT_FAIL)
+ exit_status = child_status;
+ ksft_test_result_report(exit_status, "%s\n", __func__);
+}
+
static void madvise_collapse_existing_thps(struct collapse_context *c,
struct mem_ops *ops)
{
@@ -1209,6 +1321,219 @@ static void madvise_retracted_page_tables(struct collapse_context *c,
ksft_test_result_report(exit_status, "%s\n", __func__);
}
+/* Smallest order khugepaged will consider for mTHP collapse. */
+#define MIN_MTHP_ORDER 2
+
+/*
+ * Order-parameterized collapse cases for the mthp_khugepaged context. What
+ * they add over the generic cases run under that context is per-window
+ * detection: which aligned window collapsed, and which of its neighbours did
+ * not. check_huge() answers how many folios of the order the range holds,
+ * which cannot tell one window from another.
+ *
+ * The region is faulted before MADV_HUGEPAGE, and the target order is only
+ * enabled for madvise, so the sources are always order 0 and the collapse
+ * product can only have come from khugepaged.
+ */
+static size_t mthp_window_size(void)
+{
+ return page_size << collapse_order;
+}
+
+static void mthp_push_target_order(void)
+{
+ struct thp_settings settings = *thp_current_settings();
+ int i;
+
+ /*
+ * The target order, for madvise only, and nothing else enabled: the
+ * cases fault their region before MADV_HUGEPAGE, so the sources are
+ * order 0 whatever -s asked the fault path for. That matters for the
+ * cases built around a hole -- a large source folio would fill it in
+ * and the window would collapse after all.
+ * collapse_order_mixed_sources enables the source order it wants on
+ * top of this.
+ */
+ settings.thp_enabled = THP_NEVER;
+ for (i = 0; i < NR_ORDERS; i++)
+ settings.hugepages[i].enabled = THP_NEVER;
+ settings.hugepages[collapse_order].enabled = THP_MADVISE;
+ thp_push_settings(&settings);
+}
+
+static bool window_collapsed(void *p, size_t len)
+{
+ return is_range_backed_by_folio_orders(p, len, collapse_order,
+ pagemap_fd, kpageflags_fd);
+}
+
+/* No aligned window in [p, p + len) is backed at the target order. */
+static bool window_not_collapsed(void *p, size_t len)
+{
+ size_t window = mthp_window_size();
+ char *addr = p;
+
+ for (; len >= window; addr += window, len -= window) {
+ if (window_collapsed(addr, window))
+ return false;
+ }
+ return true;
+}
+
+static bool khugepaged_wait_full_pass(void)
+{
+ /* Wait up to 30 seconds for the pass to complete. */
+ return khugepaged_full_pass(30);
+}
+
+static void collapse_order_single_window(struct collapse_context *c,
+ struct mem_ops *ops)
+{
+ size_t window = mthp_window_size();
+ void *p;
+
+ mthp_push_target_order();
+
+ p = ops->setup_area(1);
+ ops->fault(p, window, 2 * window);
+ if (!window_not_collapsed(p, hpage_pmd_size))
+ ksft_exit_fail_msg("Unexpected large folio after fault\n");
+
+ madvise(p, hpage_pmd_size, MADV_HUGEPAGE);
+ ksft_print_msg("Collapse one fully populated window...");
+ if (!khugepaged_wait_full_pass())
+ fail("Timeout");
+ else if (window_collapsed(p + window, window) &&
+ window_not_collapsed(p, window) &&
+ window_not_collapsed(p + 2 * window,
+ hpage_pmd_size - 2 * window))
+ success("OK");
+ else
+ fail("Fail");
+
+ validate_memory(p, window, 2 * window);
+ ops->cleanup_area(p, hpage_pmd_size);
+ thp_pop_settings();
+ ksft_test_result_report(exit_status, "%s\n", __func__);
+}
+
+static void collapse_order_partial_window(struct collapse_context *c,
+ struct mem_ops *ops)
+{
+ void *p;
+
+ mthp_push_target_order();
+
+ p = ops->setup_area(1);
+ ops->fault(p, 0, page_size);
+ if (!window_not_collapsed(p, hpage_pmd_size))
+ ksft_exit_fail_msg("Unexpected large folio after fault\n");
+
+ madvise(p, hpage_pmd_size, MADV_HUGEPAGE);
+ ksft_print_msg("Collapse window with single PTE entry present...");
+ if (!khugepaged_wait_full_pass())
+ fail("Timeout");
+ else if (window_collapsed(p, mthp_window_size()))
+ success("OK");
+ else
+ fail("Fail");
+
+ validate_memory(p, 0, page_size);
+ ops->cleanup_area(p, hpage_pmd_size);
+ thp_pop_settings();
+ ksft_test_result_report(exit_status, "%s\n", __func__);
+}
+
+static void collapse_order_max_ptes_none(struct collapse_context *c,
+ struct mem_ops *ops)
+{
+ struct thp_settings settings;
+ size_t window = mthp_window_size();
+ void *p;
+
+ mthp_push_target_order();
+ settings = *thp_current_settings();
+ settings.khugepaged.max_ptes_none = 0;
+ thp_push_settings(&settings);
+
+ p = ops->setup_area(1);
+ ops->fault(p, 0, 2 * window - page_size);
+ if (!window_not_collapsed(p, hpage_pmd_size))
+ ksft_exit_fail_msg("Unexpected large folio after fault\n");
+
+ madvise(p, hpage_pmd_size, MADV_HUGEPAGE);
+ ksft_print_msg("Collapse full window, not the one missing a page...");
+ if (!khugepaged_wait_full_pass())
+ fail("Timeout");
+ else if (window_collapsed(p, window) &&
+ window_not_collapsed(p + window, window))
+ success("OK");
+ else
+ fail("Fail");
+
+ validate_memory(p, 0, 2 * window - page_size);
+ ops->cleanup_area(p, hpage_pmd_size);
+ thp_pop_settings();
+ thp_pop_settings();
+ ksft_test_result_report(exit_status, "%s\n", __func__);
+}
+
+static void collapse_order_mixed_sources(struct collapse_context *c,
+ struct mem_ops *ops)
+{
+ int source_order = anon_order ? anon_order : MIN_MTHP_ORDER;
+ struct thp_settings settings;
+ void *p;
+
+ /* Sources must be a supported mTHP order strictly below the target. */
+ if (source_order >= collapse_order ||
+ !(thp_supported_orders() & (1UL << source_order))) {
+ ksft_test_result_skip("%s: no source order below target\n",
+ __func__);
+ return;
+ }
+
+ mthp_push_target_order();
+
+ /* Fault the whole region as order-@source_order folios. */
+ settings = *thp_current_settings();
+ settings.hugepages[source_order].enabled = THP_ALWAYS;
+ thp_push_settings(&settings);
+ p = ops->setup_area(1);
+ ops->fault(p, 0, hpage_pmd_size);
+ thp_pop_settings();
+
+ /*
+ * The order is enabled and supported, but the allocator can still fall
+ * back under fragmentation. That leaves nothing to collapse from,
+ * which is the machine's answer rather than a reason to end the run.
+ */
+ if (!is_range_backed_by_folio_orders(p, hpage_pmd_size, source_order,
+ pagemap_fd, kpageflags_fd)) {
+ ksft_print_msg("No order-%d sources to collapse...", source_order);
+ skip("Skip");
+ ops->cleanup_area(p, hpage_pmd_size);
+ thp_pop_settings();
+ ksft_test_result_report(exit_status, "%s\n", __func__);
+ return;
+ }
+
+ madvise(p, hpage_pmd_size, MADV_HUGEPAGE);
+ ksft_print_msg("Collapse region backed by order-%d sources...",
+ source_order);
+ if (!khugepaged_wait_full_pass())
+ fail("Timeout");
+ else if (window_collapsed(p, hpage_pmd_size))
+ success("OK");
+ else
+ fail("Fail");
+
+ validate_memory(p, 0, hpage_pmd_size);
+ ops->cleanup_area(p, hpage_pmd_size);
+ thp_pop_settings();
+ ksft_test_result_report(exit_status, "%s\n", __func__);
+}
+
static void usage(void)
{
fprintf(stderr, "\nUsage: ./khugepaged [OPTIONS] <test type> [dir]\n\n");
@@ -1226,11 +1551,15 @@ static void usage(void)
fprintf(stderr, "\t\t-s: mTHP size, expressed as page order.\n");
fprintf(stderr, "\t\t Defaults to 0. Use this size for anon or shmem allocations.\n");
fprintf(stderr, "\t\t-c: collapse order for mTHP collapse, expressed as page order.\n");
+ fprintf(stderr, "\t\t Defaults to every supported order below the PMD.\n");
+ fprintf(stderr, "\t\t With -s, -s names the mTHP source order for the\n");
+ fprintf(stderr, "\t\t mixed-source case (source order below the target).\n");
exit(1);
}
static void parse_test_type(int argc, char **argv)
{
+ bool mthp_context_implied = false;
int opt;
char *buf;
const char *token;
@@ -1242,6 +1571,7 @@ static void parse_test_type(int argc, char **argv)
break;
case 'c':
collapse_order = atoi(optarg);
+ collapse_order_given = true;
break;
case 'h':
default:
@@ -1249,12 +1579,25 @@ static void parse_test_type(int argc, char **argv)
}
}
+ /*
+ * Both orders end up as array indices and shift counts, so neither
+ * can be negative, and a zero collapse order asks for base pages.
+ */
+ if (anon_order < 0 || anon_order > hpage_pmd_order)
+ ksft_exit_fail_msg("-s takes an order in 0..%d, not %d\n",
+ hpage_pmd_order, anon_order);
+ if (collapse_order_given &&
+ (collapse_order <= 0 || collapse_order > hpage_pmd_order))
+ ksft_exit_fail_msg("-c takes an order in 1..%d, not %d\n",
+ hpage_pmd_order, collapse_order);
+
argv += optind;
argc -= optind;
if (argc == 0) {
- /* Backwards compatibility */
+ /* Everything that needs no argument of its own: anon, every context */
khugepaged_context = &__khugepaged_context;
+ mthp_khugepaged_context = &__mthp_khugepaged_context;
madvise_context = &__madvise_context;
anon_ops = &__anon_ops;
return;
@@ -1265,13 +1608,19 @@ static void parse_test_type(int argc, char **argv)
if (!strcmp(token, "all")) {
khugepaged_context = &__khugepaged_context;
+ mthp_khugepaged_context = &__mthp_khugepaged_context;
madvise_context = &__madvise_context;
+
+ /*
+ * "all" sweeps the mTHP context in, but it only has anon
+ * cases: step it aside for the other mem_types rather than
+ * refusing the whole run.
+ */
+ mthp_context_implied = true;
} else if (!strcmp(token, "khugepaged")) {
khugepaged_context = &__khugepaged_context;
} else if (!strcmp(token, "mthp_khugepaged")) {
mthp_khugepaged_context = &__mthp_khugepaged_context;
- if (collapse_order <= 0 || collapse_order >= hpage_pmd_order)
- usage();
} else if (!strcmp(token, "madvise")) {
madvise_context = &__madvise_context;
} else {
@@ -1287,20 +1636,20 @@ static void parse_test_type(int argc, char **argv)
read_write_file_write_ops = &__read_write_file_write_ops;
anon_ops = &__anon_ops;
shmem_ops = &__shmem_ops;
- if (mthp_khugepaged_context)
- usage();
} else if (!strcmp(buf, "anon")) {
anon_ops = &__anon_ops;
} else if (!strcmp(buf, "file")) {
read_only_file_ops = &__read_only_file_ops;
read_write_file_read_ops = &__read_write_file_read_ops;
read_write_file_write_ops = &__read_write_file_write_ops;
- if (mthp_khugepaged_context)
+ if (mthp_khugepaged_context && !mthp_context_implied)
usage();
+ mthp_khugepaged_context = NULL;
} else if (!strcmp(buf, "shmem")) {
shmem_ops = &__shmem_ops;
- if (mthp_khugepaged_context)
+ if (mthp_khugepaged_context && !mthp_context_implied)
usage();
+ mthp_khugepaged_context = NULL;
} else {
usage();
}
@@ -1322,9 +1671,14 @@ struct test_case {
struct mem_ops *ops;
const char *desc;
test_fn fn;
+ int order; /* mTHP contexts: the collapse order */
};
-#define MAX_TEST_CASES 64
+/*
+ * Enough for every case at every order the kernel offers: the mTHP context
+ * runs its cases once per supported order below the PMD.
+ */
+#define MAX_TEST_CASES 256
static struct test_case test_cases[MAX_TEST_CASES];
static int nr_test_cases;
@@ -1337,6 +1691,7 @@ static int nr_test_cases;
.ops = o, \
.desc = #t, \
.fn = t, \
+ .order = collapse_order, \
}; \
} \
} while (0)
@@ -1377,8 +1732,83 @@ int main(int argc, char **argv)
parse_test_type(argc, argv);
+ if (mthp_khugepaged_context) {
+ unsigned long orders = thp_supported_orders();
+
+ if (collapse_order_given) {
+ /* -c pins one order; it has to be one we can build */
+ if (collapse_order >= hpage_pmd_order)
+ ksft_exit_fail_msg("-c takes an order below the PMD order (%d)\n",
+ hpage_pmd_order);
+ if (!(orders & (1UL << collapse_order)))
+ ksft_exit_skip("Order %d is not a supported anon THP order\n",
+ collapse_order);
+ if (collapse_order <= anon_order)
+ ksft_exit_skip("-c %d needs a source order below it, -s says %d\n",
+ collapse_order, anon_order);
+ collapse_orders[nr_collapse_orders++] = collapse_order;
+ } else {
+ /*
+ * Otherwise every order a collapse could produce. -s
+ * makes the fault path hand out folios of that order,
+ * so a target at or below it has nothing to collapse:
+ * the sources are already the size being asked for.
+ */
+ int first = anon_order ? anon_order + 1 : MIN_MTHP_ORDER;
+
+ if (first < MIN_MTHP_ORDER)
+ first = MIN_MTHP_ORDER;
+ for (int i = first; i < hpage_pmd_order; i++) {
+ if (orders & (1UL << i))
+ collapse_orders[nr_collapse_orders++] = i;
+ }
+ if (!nr_collapse_orders)
+ ksft_print_msg("mTHP cases skipped: no order above the source\n");
+ }
+ }
+
+ if (mthp_khugepaged_context) {
+ pagemap_fd = open("/proc/self/pagemap", O_RDONLY);
+ if (pagemap_fd < 0)
+ ksft_exit_fail_perror("open(/proc/self/pagemap)");
+ kpageflags_fd = open("/proc/kpageflags", O_RDONLY);
+ if (kpageflags_fd < 0)
+ ksft_exit_fail_perror("open(/proc/kpageflags)");
+ }
+
setbuf(stdout, NULL);
+ /*
+ * The page cache caps folio order at MAX_PAGECACHE_ORDER, which
+ * xas_split_alloc() puts below the PMD order on arm64 with 64K pages.
+ * A PMD-sized page cache folio is then impossible, so the kernel
+ * refuses these collapses by design and there is nothing to test.
+ *
+ * The cap is one global, so it rules out every file mapping, not just
+ * shmem: shmem_huge_global_enabled() drops the PMD order from what it
+ * allows, and file_thp_enabled() refuses a regular file whose mapping
+ * cannot hold a PMD folio.
+ *
+ * The per-order shmem_enabled control below is what makes the cap
+ * readable: it is created for the orders in THP_ORDERS_ALL_FILE_DEFAULT,
+ * which is the cap and nothing else, so whether the PMD order has one
+ * answers for a regular file as much as for shmem.
+ */
+ if (!(thp_shmem_supported_orders() & (1UL << hpage_pmd_order))) {
+ if (shmem_ops) {
+ ksft_print_msg("no PMD-order page cache folio: skipping shmem\n");
+ shmem_ops = NULL;
+ }
+ if (read_only_file_ops) {
+ ksft_print_msg("no PMD-order page cache folio: skipping file\n");
+ read_only_file_ops = NULL;
+ read_write_file_read_ops = NULL;
+ read_write_file_write_ops = NULL;
+ }
+ if (!anon_ops && !shmem_ops && !read_only_file_ops)
+ ksft_exit_skip("Nothing left to collapse into\n");
+ }
+
default_settings.khugepaged.max_ptes_none = hpage_pmd_nr - 1;
default_settings.khugepaged.max_ptes_swap = hpage_pmd_nr / 8;
default_settings.khugepaged.max_ptes_shared = hpage_pmd_nr / 2;
@@ -1396,7 +1826,17 @@ int main(int argc, char **argv)
TEST(collapse_full, khugepaged_context, read_write_file_read_ops);
TEST(collapse_full, khugepaged_context, read_write_file_write_ops);
TEST(collapse_full, khugepaged_context, shmem_ops);
- TEST(collapse_full, mthp_khugepaged_context, anon_ops);
+ for (int i = 0; i < nr_collapse_orders; i++) {
+ collapse_order = collapse_orders[i];
+ TEST(collapse_full, mthp_khugepaged_context, anon_ops);
+ TEST(collapse_empty, mthp_khugepaged_context, anon_ops);
+ TEST(collapse_single_mthp, mthp_khugepaged_context, anon_ops);
+ TEST(collapse_order_single_window, mthp_khugepaged_context, anon_ops);
+ TEST(collapse_order_partial_window, mthp_khugepaged_context, anon_ops);
+ TEST(collapse_order_max_ptes_none, mthp_khugepaged_context, anon_ops);
+ TEST(collapse_order_mixed_sources, mthp_khugepaged_context, anon_ops);
+ }
+
TEST(collapse_full, madvise_context, anon_ops);
TEST(collapse_full, madvise_context, read_only_file_ops);
TEST(collapse_full, madvise_context, read_write_file_read_ops);
@@ -1404,10 +1844,8 @@ int main(int argc, char **argv)
TEST(collapse_full, madvise_context, shmem_ops);
TEST(collapse_empty, khugepaged_context, anon_ops);
- TEST(collapse_empty, mthp_khugepaged_context, anon_ops);
TEST(collapse_empty, madvise_context, anon_ops);
- TEST(collapse_single_mthp, mthp_khugepaged_context, anon_ops);
TEST(collapse_single_pte_entry, khugepaged_context, anon_ops);
TEST(collapse_single_pte_entry, khugepaged_context, read_only_file_ops);
@@ -1463,6 +1901,9 @@ int main(int argc, char **argv)
TEST(collapse_max_ptes_shared, khugepaged_context, anon_ops);
TEST(collapse_max_ptes_shared, madvise_context, anon_ops);
+ TEST(collapse_fork_cow_race, khugepaged_context, anon_ops);
+ TEST(collapse_fork_cow_race, madvise_context, anon_ops);
+
TEST(madvise_collapse_existing_thps, madvise_context, anon_ops);
TEST(madvise_collapse_existing_thps, madvise_context, read_only_file_ops);
TEST(madvise_collapse_existing_thps, madvise_context, read_write_file_read_ops);
@@ -1478,7 +1919,15 @@ int main(int argc, char **argv)
for (int i = 0; i < nr_test_cases; i++) {
struct test_case *t = &test_cases[i];
- ksft_print_msg("\n# Run test: %s (%s:%s)\n", t->desc, t->ctx->name, t->ops->name);
+ if (t->ctx == &__mthp_khugepaged_context) {
+ collapse_order = t->order;
+ ksft_print_msg("\n# Run test: %s (%s:%s, order %d)\n",
+ t->desc, t->ctx->name, t->ops->name,
+ t->order);
+ } else {
+ ksft_print_msg("\n# Run test: %s (%s:%s)\n", t->desc,
+ t->ctx->name, t->ops->name);
+ }
t->fn(t->ctx, t->ops);
}
diff --git a/tools/testing/selftests/mm/khugepaged_race.c b/tools/testing/selftests/mm/khugepaged_race.c
new file mode 100644
index 0000000000000..3601c030d43e5
--- /dev/null
+++ b/tools/testing/selftests/mm/khugepaged_race.c
@@ -0,0 +1,610 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * khugepaged race harness.
+ *
+ * Runs collapse against concurrent faults, transient GUP pins
+ * (gup_test), fork, mremap and MADV_DONTNEED over the same ranges, in
+ * one of three driver modes:
+ *
+ * stepped khugepaged, one full pass at a time through
+ * khugepaged_full_pass(), so a step covers a known extent;
+ * free khugepaged left to run (scan_sleep_millisecs=0), for soak;
+ * madvise MADV_COLLAPSE in a loop.
+ *
+ * All anon THP orders are enabled (inherit). Occupancy runs at both ends
+ * of what mTHP collapse supports: max_ptes_none 0, where a window must be
+ * fully populated, and HPAGE_PMD_NR - 1, where a window full of holes
+ * collapses too. The holes are not copied from anywhere -- they are
+ * zero-filled, and re-checked under the page table lock at install time in
+ * case a racing fault got there first.
+ *
+ * -p adds memory pressure to any of the above: MADV_PAGEOUT cycling
+ * on a dedicated neighbor region (swap traffic and LRU churn; skipped
+ * with a note when the host has no swap) and a compact_memory trigger
+ * loop (compaction migrates source folios, racing collapse's freeze
+ * with refcount elevation and migration entries of its own).
+ *
+ * Correctness signals: every racing page must read as its pattern or
+ * zero (MADV_DONTNEED), never anything else. The faulters and the fork
+ * children check that continuously, a final sweep checks it once more, plus
+ * whatever DEBUG_VM / page_table_check / KASAN / lockdep report in
+ * dmesg, which the caller is expected to inspect.
+ */
+#define _GNU_SOURCE
+#include <errno.h>
+#include <fcntl.h>
+#include <pthread.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+#include <sys/time.h>
+#include <sys/wait.h>
+#include <unistd.h>
+
+#include "kselftest.h"
+#include "vm_util.h"
+#include "hugepage_settings.h"
+#include "../../../../mm/gup_test.h"
+
+#define BASE_ADDR ((void *)(1UL << 30))
+
+/*
+ * Shared playground for faults/pins/fork/dontneed: several PMD-sized
+ * areas the racing threads spread across, plus one area owned by the
+ * mremap thread. More areas means more independent regions collapsing
+ * at once; the default suits a normal machine. On a memory-constrained
+ * host -- or under emulation, where a 512M PMD (arm64/64K) makes the
+ * default playground multi-gigabyte -- pass -a to shrink it.
+ */
+#define DEFAULT_SHARED_AREAS 3
+static int nr_shared_areas;
+static int nr_areas;
+
+static unsigned long hpage_pmd_size;
+static unsigned long page_size;
+static char *region; /* NR_AREAS * hpage_pmd_size */
+static char *mremap_area; /* region + NR_SHARED_AREAS areas */
+static char *mremap_scratch; /* well above the region */
+static char *pageout_area; /* -p: dedicated pressure region */
+static size_t pageout_size;
+static int gup_fd = -1;
+static volatile int stop;
+static volatile int corrupted;
+
+static unsigned int pattern(unsigned long page_idx)
+{
+ unsigned int val = (unsigned int)page_idx * 2654435761U;
+
+ return val ? val : 1; /* never collides with the zero-fill */
+}
+
+/* Zero means never written; anything else must be this page's pattern */
+static bool page_is_corrupt(unsigned long page_idx, unsigned int *val)
+{
+ *val = *(unsigned int *)(region + page_idx * page_size);
+
+ return *val && *val != pattern(page_idx);
+}
+
+static void check_page(unsigned long page_idx)
+{
+ unsigned int val;
+
+ if (page_is_corrupt(page_idx, &val)) {
+ corrupted = 1;
+ ksft_print_msg("Corruption at page %lu: %#x != %#x\n",
+ page_idx, val, pattern(page_idx));
+ }
+}
+
+static unsigned long shared_pages(void)
+{
+ return nr_shared_areas * hpage_pmd_size / page_size;
+}
+
+static unsigned long rand_page(unsigned int *seed)
+{
+ return (unsigned long)rand_r(seed) % shared_pages();
+}
+
+/* Pages left from @page_idx, so a range never reaches the mremap thread's area */
+static unsigned long room_from(unsigned long page_idx, unsigned long want)
+{
+ unsigned long left = shared_pages() - page_idx;
+
+ return want < left ? want : left;
+}
+
+static void *faulter_fn(void *arg)
+{
+ unsigned int seed = (unsigned long)arg;
+
+ while (!stop) {
+ unsigned long page_idx = rand_page(&seed);
+
+ if (rand_r(&seed) & 1)
+ *(unsigned int *)(region + page_idx * page_size) =
+ pattern(page_idx);
+ else
+ check_page(page_idx);
+ }
+ return NULL;
+}
+
+static void *dontneed_fn(void *arg)
+{
+ unsigned int seed = (unsigned long)arg;
+
+ while (!stop) {
+ unsigned long page_idx = rand_page(&seed);
+ unsigned long nr = 1UL << (rand_r(&seed) % 6); /* 1..32 pages */
+
+ /*
+ * Once in a while zap a whole PMD-aligned area: only a zap
+ * spanning the full table triggers the empty-table reclaim
+ * (CONFIG_PT_RECLAIM), which can free the table under a
+ * collapse that is midway through it. Sub-table zaps never
+ * reach that path.
+ */
+ if (!(rand_r(&seed) % 64)) {
+ unsigned long area = page_idx /
+ (hpage_pmd_size / page_size);
+
+ madvise(region + area * hpage_pmd_size,
+ hpage_pmd_size, MADV_DONTNEED);
+ } else {
+ madvise(region + page_idx * page_size,
+ room_from(page_idx, nr) * page_size,
+ MADV_DONTNEED);
+ }
+ usleep(rand_r(&seed) % 500);
+ }
+ return NULL;
+}
+
+static void *pinner_fn(void *arg)
+{
+ unsigned int seed = (unsigned long)arg;
+
+ while (!stop) {
+ struct gup_test gup = {};
+ unsigned long page_idx = rand_page(&seed);
+
+ unsigned long nr = room_from(page_idx, 16);
+
+ gup.addr = (unsigned long)(region + page_idx * page_size);
+ gup.size = nr * page_size;
+ gup.nr_pages_per_call = nr;
+ gup.gup_flags = 1; /* FOLL_WRITE */
+ /* Racing MADV_DONTNEED makes transient failures expected. */
+ ioctl(gup_fd, PIN_FAST_BENCHMARK, &gup);
+ usleep(rand_r(&seed) % 200);
+ }
+ return NULL;
+}
+
+static void *forker_fn(void *arg)
+{
+ unsigned int seed = (unsigned long)arg;
+
+ while (!stop) {
+ pid_t pid = fork();
+
+ if (pid == 0) {
+ unsigned int val;
+ int bad = 0;
+
+ /*
+ * No stdio in the child: a thread may have held
+ * stdout's lock when we forked, and printing under an
+ * inherited lock hangs. The parent reports what the
+ * exit status says.
+ */
+ for (int i = 0; i < 16; i++)
+ bad |= page_is_corrupt(rand_page(&seed), &val);
+ _exit(bad);
+ }
+ if (pid > 0) {
+ int wstatus;
+
+ if (waitpid(pid, &wstatus, 0) < 0)
+ ksft_exit_fail_perror("waitpid()");
+ /* A child killed on the read counts too, not just its exit code. */
+ if (!WIFEXITED(wstatus) || WEXITSTATUS(wstatus))
+ corrupted = 1;
+ }
+ usleep(rand_r(&seed) % 2000);
+ }
+ return NULL;
+}
+
+static void *mremapper_fn(void *arg)
+{
+ unsigned int seed = (unsigned long)arg;
+
+ while (!stop) {
+ void *p;
+
+ p = mremap(mremap_area, hpage_pmd_size, hpage_pmd_size,
+ MREMAP_MAYMOVE | MREMAP_FIXED, mremap_scratch);
+ if (p == MAP_FAILED)
+ ksft_exit_fail_perror("mremap() away");
+ for (int i = 0; i < 8; i++)
+ mremap_scratch[(rand_r(&seed) %
+ (hpage_pmd_size / page_size)) * page_size] = 1;
+ p = mremap(mremap_scratch, hpage_pmd_size, hpage_pmd_size,
+ MREMAP_MAYMOVE | MREMAP_FIXED, mremap_area);
+ if (p == MAP_FAILED)
+ ksft_exit_fail_perror("mremap() back");
+ usleep(rand_r(&seed) % 2000);
+ }
+ return NULL;
+}
+
+/*
+ * -p: swap traffic and LRU churn on a region of our own. The content
+ * check is exact: a page out and back through swap must preserve the
+ * pattern, and nothing else ever writes here.
+ */
+static void *pageout_fn(void *arg)
+{
+ unsigned int seed = (unsigned long)arg;
+ unsigned long nr = pageout_size / page_size;
+ unsigned long i;
+
+ for (i = 0; i < nr; i++)
+ *(unsigned int *)(pageout_area + i * page_size) = pattern(i);
+
+ while (!stop) {
+ madvise(pageout_area, pageout_size, MADV_PAGEOUT);
+ for (i = 0; i < nr && !stop; i++) {
+ unsigned int val = *(unsigned int *)(pageout_area +
+ i * page_size);
+
+ if (val != pattern(i)) {
+ corrupted = 1;
+ ksft_print_msg("Pageout corruption at page %lu: %#x != %#x\n",
+ i, val, pattern(i));
+ }
+ }
+ usleep(rand_r(&seed) % 2000);
+ }
+ return NULL;
+}
+
+/* -p: compaction migrates the collapse sources out from under us. */
+static void *compactor_fn(void *arg)
+{
+ unsigned int seed = (unsigned long)arg;
+ int fd = open("/proc/sys/vm/compact_memory", O_WRONLY);
+
+ if (fd < 0) {
+ ksft_print_msg("No compact_memory; compactor idle\n");
+ return NULL;
+ }
+ while (!stop) {
+ if (write(fd, "1", 1) < 0)
+ break;
+ usleep(10000 + rand_r(&seed) % 100000);
+ }
+ close(fd);
+ return NULL;
+}
+
+static bool swap_available(void)
+{
+ char line[256];
+ int lines = 0;
+ FILE *fp = fopen("/proc/swaps", "r");
+
+ if (!fp)
+ return false;
+ while (fgets(line, sizeof(line), fp))
+ lines++;
+ fclose(fp);
+ return lines > 1;
+}
+
+static unsigned long now_ms(void)
+{
+ struct timeval tv;
+
+ gettimeofday(&tv, NULL);
+ return tv.tv_sec * 1000UL + tv.tv_usec / 1000;
+}
+
+static void usage(void)
+{
+ fprintf(stderr,
+ "Usage: khugepaged_race [-d seconds] [-m stepped|free|madvise] [-z] [-p] [-a areas]\n"
+ "\tWithout -m, every mode runs in turn.\n"
+ "\t-d: seconds per mode (default 5)\n"
+ "\tBoth occupancy limits run unless -z asks for holes only.\n"
+ "\t-z: only max_ptes_none = HPAGE_PMD_NR - 1 (hole-heavy)\n"
+ "\tRuns with and without memory pressure unless -p asks for\n"
+ "\tpressure only.\n"
+ "\t-p: only with the pageout and compaction threads\n"
+ "\t-a: number of shared PMD-sized playground areas (default 3)\n");
+ exit(1);
+}
+
+int main(int argc, char **argv)
+{
+ static const char * const thread_names[] = {
+ "faulter", "faulter2", "dontneed", "pinner", "forker",
+ "mremapper", "pageout", "compactor",
+ };
+ void *(*const thread_fns[])(void *) = {
+ faulter_fn, faulter_fn, dontneed_fn, pinner_fn, forker_fn,
+ mremapper_fn, pageout_fn, compactor_fn,
+ };
+ const unsigned long pageout_bit = 1UL << 6, compactor_bit = 1UL << 7;
+ const int nr_threads = ARRAY_SIZE(thread_names);
+ pthread_t threads[ARRAY_SIZE(thread_names)];
+ static const char * const all_modes[] = { "stepped", "free", "madvise" };
+ static const int all_nones[] = { 0, 1 }; /* strict, holes */
+ static const int all_press[] = { 0, 1 }; /* quiet, under pressure */
+ const int *nones = all_nones;
+ const int *press = all_press;
+ int nr_nones = ARRAY_SIZE(all_nones);
+ int nr_press = ARRAY_SIZE(all_press);
+ const char *one_mode[1];
+ const char * const *modes = all_modes;
+ int nr_modes = ARRAY_SIZE(all_modes);
+ const char *mode_arg = NULL;
+ struct thp_settings settings;
+ unsigned long end_ms;
+ int duration_s = 5;
+ unsigned long thread_mask = ~0UL;
+ unsigned long base_mask;
+ int nr_areas_arg = 0;
+ bool holes_only = false;
+ bool pressure_only = false;
+ unsigned long i;
+ int steps = 0;
+ int opt;
+
+ while ((opt = getopt(argc, argv, "a:d:m:t:zph")) != -1) {
+ switch (opt) {
+ case 'a':
+ nr_areas_arg = atoi(optarg);
+ break;
+ case 'd':
+ duration_s = atoi(optarg);
+ break;
+ case 'm':
+ mode_arg = optarg;
+ break;
+ case 't':
+ /* debug: bitmask of racing threads to start */
+ thread_mask = strtoul(optarg, NULL, 0);
+ break;
+ case 'z':
+ holes_only = true;
+ break;
+ case 'p':
+ pressure_only = true;
+ break;
+ default:
+ usage();
+ }
+ }
+ if (holes_only) {
+ nones = all_nones + 1;
+ nr_nones = 1;
+ }
+
+ if (pressure_only) {
+ press = all_press + 1;
+ nr_press = 1;
+ }
+
+ if (mode_arg) {
+ if (strcmp(mode_arg, "stepped") && strcmp(mode_arg, "free") &&
+ strcmp(mode_arg, "madvise"))
+ usage();
+ one_mode[0] = mode_arg;
+ modes = one_mode;
+ nr_modes = 1;
+ }
+
+ ksft_print_header();
+ if (!thp_available())
+ ksft_exit_skip("Transparent Hugepages not available\n");
+
+ page_size = getpagesize();
+ hpage_pmd_size = read_pmd_pagesize();
+ if (!hpage_pmd_size)
+ ksft_exit_fail_msg("Reading PMD pagesize failed\n");
+
+ gup_fd = open("/sys/kernel/debug/gup_test", O_RDWR);
+ if (gup_fd < 0)
+ ksft_exit_skip("/sys/kernel/debug/gup_test requires CONFIG_GUP_TEST and root\n");
+
+ nr_shared_areas = nr_areas_arg > 0 ? nr_areas_arg : DEFAULT_SHARED_AREAS;
+ nr_areas = nr_shared_areas + 1;
+
+ /*
+ * The mremap thread moves its area to this address and back, and
+ * MREMAP_FIXED unmaps whatever is in the way without saying so. Claim
+ * the address here, so a layout that does not match this assumption
+ * fails now instead of losing a mapping later. Nothing else in the
+ * process maps this low: thread stacks and malloc arenas come from the
+ * top-down mmap area, well above.
+ */
+ mremap_scratch = (char *)BASE_ADDR + 2 * nr_areas * hpage_pmd_size;
+ if (mmap(mremap_scratch, hpage_pmd_size, PROT_NONE,
+ MAP_ANONYMOUS | MAP_PRIVATE | MAP_FIXED_NOREPLACE,
+ -1, 0) != (void *)mremap_scratch)
+ ksft_exit_fail_perror("mmap() mremap scratch");
+
+ base_mask = thread_mask;
+ if (!swap_available())
+ /* No swap, no anon reclaim: compaction-only pressure. */
+ ksft_print_msg("no swap: the pageout thread stays idle\n");
+
+ ksft_set_plan(nr_modes * nr_nones * nr_press);
+
+ thp_save_settings();
+ thp_read_settings(&settings);
+
+ /*
+ * A base entry for the stack, so that the pop at the end of a mode
+ * always has something to write back: thp_pop_settings() on an empty
+ * stack has no settings to apply and gives up.
+ */
+ thp_push_settings(&settings);
+
+ for (int run = 0; run < nr_modes * nr_nones * nr_press; run++) {
+ int rem = run % (nr_nones * nr_press);
+ const char *mode = modes[run / (nr_nones * nr_press)];
+ bool holes = nones[rem / nr_press];
+ bool pressure = press[rem % nr_press];
+
+ thread_mask = base_mask;
+ if (!pressure)
+ thread_mask &= ~(pageout_bit | compactor_bit);
+ else if (!swap_available())
+ thread_mask &= ~pageout_bit;
+
+ thp_read_settings(&settings);
+ settings.thp_enabled = THP_MADVISE;
+ settings.thp_defrag = THP_DEFRAG_ALWAYS;
+ settings.shmem_enabled = SHMEM_NEVER;
+ settings.khugepaged.defrag = 1;
+ settings.khugepaged.scan_sleep_millisecs =
+ strcmp(mode, "free") ? 1000 : 0;
+ settings.khugepaged.alloc_sleep_millisecs = 10;
+
+ /*
+ * mTHP collapse only supports the two ends of the occupancy
+ * scale: 0 or HPAGE_PMD_NR - 1 (anything else coerces to 0).
+ * Strict needs a fully populated window, which is rare under
+ * racing MADV_DONTNEED; hole-heavy windows collapse instead,
+ * so the two ends race different paths.
+ */
+ settings.khugepaged.max_ptes_none = holes ?
+ (hpage_pmd_size / page_size) - 1 : 0;
+ settings.khugepaged.pages_to_scan =
+ nr_areas * (hpage_pmd_size / page_size) * 8;
+ for (i = 0; i < NR_ORDERS; i++) {
+ if (thp_supported_orders() & (1UL << i))
+ settings.hugepages[i].enabled = THP_INHERIT;
+ }
+ /* Popped at the end of this mode, before the next one. */
+ thp_push_settings(&settings);
+
+ region = mmap(BASE_ADDR, nr_areas * hpage_pmd_size,
+ PROT_READ | PROT_WRITE, MAP_ANONYMOUS |
+ MAP_PRIVATE | MAP_FIXED_NOREPLACE, -1, 0);
+ if (region != BASE_ADDR)
+ ksft_exit_fail_perror("mmap() playground");
+ mremap_area = region + nr_shared_areas * hpage_pmd_size;
+
+ if (thread_mask & pageout_bit) {
+ /*
+ * Big enough to cycle real reclaim, small enough not
+ * to dominate a TCG guest: 4 PMD areas, clamped to
+ * [16M, 64M].
+ */
+ pageout_size = 4 * hpage_pmd_size;
+ pageout_size = pageout_size < (16UL << 20) ?
+ (16UL << 20) :
+ pageout_size > (64UL << 20) ?
+ (64UL << 20) : pageout_size;
+ pageout_area = mmap(NULL, pageout_size,
+ PROT_READ | PROT_WRITE,
+ MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
+ if (pageout_area == MAP_FAILED)
+ ksft_exit_fail_perror("mmap() pageout area");
+ }
+
+ /* Populate so the first pass has something to collapse. */
+ for (i = 0; i < nr_shared_areas * hpage_pmd_size / page_size; i++)
+ *(unsigned int *)(region + i * page_size) = pattern(i);
+ memset(mremap_area, 1, hpage_pmd_size);
+ madvise(region, nr_areas * hpage_pmd_size, MADV_HUGEPAGE);
+
+ for (i = 0; i < nr_threads; i++) {
+ if (!(thread_mask & (1UL << i))) {
+ threads[i] = 0;
+ continue;
+ }
+ if (pthread_create(&threads[i], NULL, thread_fns[i],
+ (void *)(i + 1)))
+ ksft_exit_fail_perror("pthread_create()");
+ }
+
+ end_ms = now_ms() + duration_s * 1000UL;
+ if (!strcmp(mode, "stepped")) {
+ while (now_ms() < end_ms && !corrupted) {
+ if (!khugepaged_full_pass(600))
+ ksft_exit_fail_msg("khugepaged pass timed out\n");
+ steps++;
+ }
+ } else if (!strcmp(mode, "free")) {
+ while (now_ms() < end_ms && !corrupted)
+ usleep(100 * 1000);
+ } else { /* madvise */
+ while (now_ms() < end_ms && !corrupted) {
+ for (i = 0; i < nr_shared_areas; i++) {
+ madvise(region + i * hpage_pmd_size,
+ hpage_pmd_size, MADV_COLLAPSE);
+ }
+ madvise(region, nr_shared_areas * hpage_pmd_size,
+ MADV_DONTNEED);
+ steps++;
+ }
+ }
+
+ stop = 1;
+ for (i = 0; i < nr_threads; i++) {
+ if (threads[i])
+ pthread_join(threads[i], NULL);
+ }
+
+ /* Final integrity sweep. */
+ for (i = 0; i < nr_shared_areas * hpage_pmd_size / page_size; i++)
+ check_page(i);
+
+ ksft_test_result(!corrupted,
+ "%s/%s%s: %ds, %d steps, no corruption\n",
+ mode, holes ? "holes" : "strict",
+ pressure ? "/pressure" : "",
+ duration_s, steps);
+
+ /*
+ * Hand the address space and the settings back before the
+ * next mode: it maps the region at the same fixed address,
+ * and its scan cadence differs.
+ */
+ munmap(region, nr_areas * hpage_pmd_size);
+ if (pageout_area) {
+ munmap(pageout_area, pageout_size);
+ pageout_area = NULL;
+ }
+ thp_pop_settings();
+ stop = 0;
+ steps = 0;
+
+ if (corrupted) {
+ /* Memory is suspect; the rest would prove nothing. */
+ while (++run < nr_modes * nr_nones * nr_press) {
+ rem = run % (nr_nones * nr_press);
+
+ ksft_test_result_skip("%s/%s%s: skipped after corruption\n",
+ modes[run / (nr_nones * nr_press)],
+ nones[rem / nr_press] ?
+ "holes" : "strict",
+ press[rem % nr_press] ?
+ "/pressure" : "");
+ }
+ break;
+ }
+ }
+
+ thp_restore_settings();
+ ksft_finished();
+}
diff --git a/tools/testing/selftests/mm/khugepaged_sync_check.c b/tools/testing/selftests/mm/khugepaged_sync_check.c
new file mode 100644
index 0000000000000..30d3fb519fb2c
--- /dev/null
+++ b/tools/testing/selftests/mm/khugepaged_sync_check.c
@@ -0,0 +1,217 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Synchronous khugepaged driving check.
+ *
+ * Race tests drive khugepaged through the existing sysfs controls: a
+ * store to scan_sleep_millisecs wakes the daemon, and full_scans
+ * advancing by two is a completion barrier for one full pass that
+ * started after setup (khugepaged_full_pass()). Verify the pair gives
+ * deterministic, attributable results: one barrier step over one
+ * prepared window produces exactly one collapse attempt on that
+ * window's source pages (mm_collapse_huge_page_isolate events filtered
+ * by source PFN and order) and the window is collapsed
+ * afterwards, repeatably.
+ *
+ * scan_sleep_millisecs is set to 60s to prove the wake path: without
+ * the wake, one barrier step would sleep multiples of that and blow
+ * the timeout. It also keeps the daemon from free-running between
+ * steps, per the khugepaged_full_pass() discipline.
+ */
+#define _GNU_SOURCE
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <unistd.h>
+
+#include "kselftest.h"
+#include "vm_util.h"
+#include "hugepage_settings.h"
+
+#define BASE_ADDR ((void *)(1UL << 30))
+#define TARGET_ORDER 2 /* smallest order khugepaged considers */
+#define NR_ITERATIONS 5
+
+static int pagemap_fd;
+static int kpageflags_fd;
+static int trace_events_fd = -1;
+static unsigned long hpage_pmd_size;
+
+/*
+ * Each step switches the events off again, but a helper can still give up
+ * on us in between (a failing sysfs write ends the test from inside
+ * thp_write_num()), and huge_memory events left on are the whole machine's
+ * problem, not this test's.
+ */
+static void trace_events_off(void)
+{
+ if (trace_events_fd >= 0)
+ tracing_events_enable(trace_events_fd, false);
+}
+
+/*
+ * Count collapse attempts attributable to our window: legacy-engine
+ * isolate events whose scan_pfn is one of the window's source PFNs,
+ * plus batch-engine per-candidate install events at the window's
+ * address. Either engine reports exactly once per attempt.
+ */
+static int count_attributed(unsigned long *pfns, int nr_pfns,
+ unsigned long addr, unsigned int order)
+{
+ char line[1024];
+ int count = 0;
+ FILE *fp;
+
+ fp = tracing_open_trace();
+ if (!fp)
+ ksft_exit_fail_msg("Cannot open trace buffer\n");
+
+ while (fgets(line, sizeof(line), fp)) {
+ char *s;
+ unsigned long val;
+ unsigned int ord;
+ char *o;
+ int i;
+
+ s = strstr(line, "mm_collapse_huge_page_isolate:");
+ if (s) {
+ if (sscanf(s, "mm_collapse_huge_page_isolate: scan_pfn=0x%lx",
+ &val) != 1)
+ continue;
+ o = strstr(s, "order=");
+ if (!o || sscanf(o, "order=%u", &ord) != 1 ||
+ ord != order)
+ continue;
+ for (i = 0; i < nr_pfns; i++) {
+ if (val == pfns[i]) {
+ count++;
+ break;
+ }
+ }
+ continue;
+ }
+
+ s = strstr(line, "mm_collapse_candidate:");
+ if (s) {
+ if (!strstr(s, "pass=install") ||
+ !strstr(s, "result=succeeded"))
+ continue;
+ o = strstr(s, "addr=");
+ if (!o || sscanf(o, "addr=0x%lx", &val) != 1 ||
+ val != addr)
+ continue;
+ o = strstr(s, "order=");
+ if (!o || sscanf(o, "order=%u", &ord) != 1 ||
+ ord != order)
+ continue;
+ count++;
+ }
+ }
+ fclose(fp);
+ return count;
+}
+
+static void one_step(int iteration)
+{
+ const size_t window = getpagesize() << TARGET_ORDER;
+ const int nr_pages = 1 << TARGET_ORDER;
+ unsigned long pfns[1 << TARGET_ORDER];
+ bool collapsed, passed;
+ int attributed;
+ char *p;
+ int i;
+
+ p = mmap(BASE_ADDR, hpage_pmd_size, PROT_READ | PROT_WRITE,
+ MAP_ANONYMOUS | MAP_PRIVATE | MAP_FIXED_NOREPLACE, -1, 0);
+ if (p != BASE_ADDR)
+ ksft_exit_fail_perror("mmap() window");
+
+ /* Prepare one window; record its source PFNs. */
+ for (i = 0; i < nr_pages; i++) {
+ p[i * getpagesize()] = i + 1;
+ pfns[i] = pagemap_get_pfn(pagemap_fd, p + i * getpagesize());
+ if (pfns[i] == -1UL)
+ ksft_exit_fail_msg("Source page not present\n");
+ }
+
+ /* Clear first: with the events still off there is nothing to undo. */
+ if (tracing_clear_trace())
+ ksft_exit_fail_msg("Cannot clear the trace buffer\n");
+ if (tracing_events_enable(trace_events_fd, true))
+ ksft_exit_fail_msg("Cannot enable huge_memory events\n");
+
+ madvise(p, hpage_pmd_size, MADV_HUGEPAGE);
+ /* Wait up to 120 seconds for the pass to complete. */
+ passed = khugepaged_full_pass(120);
+
+ /* Off before anything that can give up: the events are system-wide. */
+ if (tracing_events_enable(trace_events_fd, false))
+ ksft_exit_fail_msg("Cannot disable huge_memory events\n");
+ if (!passed)
+ ksft_exit_fail_msg("khugepaged did not complete a full pass\n");
+
+ collapsed = is_range_backed_by_folio_orders(p, window, TARGET_ORDER,
+ pagemap_fd, kpageflags_fd);
+ attributed = count_attributed(pfns, nr_pages, (unsigned long)p,
+ TARGET_ORDER);
+
+ ksft_test_result(collapsed && attributed == 1,
+ "step %d: window collapsed, %d attributed result(s)\n",
+ iteration, attributed);
+
+ munmap(p, hpage_pmd_size);
+}
+
+int main(void)
+{
+ struct thp_settings settings;
+ int i;
+
+ ksft_print_header();
+
+ if (!thp_available())
+ ksft_exit_skip("Transparent Hugepages not available\n");
+ if (!(thp_supported_orders() & (1UL << TARGET_ORDER)))
+ ksft_exit_skip("Order %d is not a supported anon THP order\n",
+ TARGET_ORDER);
+
+ hpage_pmd_size = read_pmd_pagesize();
+ if (!hpage_pmd_size)
+ ksft_exit_fail_msg("Reading PMD pagesize failed\n");
+ pagemap_fd = open("/proc/self/pagemap", O_RDONLY);
+ if (pagemap_fd < 0)
+ ksft_exit_fail_perror("open(/proc/self/pagemap)");
+ kpageflags_fd = open("/proc/kpageflags", O_RDONLY);
+ if (kpageflags_fd < 0)
+ ksft_exit_skip("open(\"/proc/kpageflags\") requires root\n");
+ trace_events_fd = tracing_events_open("huge_memory");
+ if (trace_events_fd < 0)
+ ksft_exit_skip("huge_memory events require tracefs and root\n");
+ atexit(trace_events_off);
+
+ ksft_set_plan(NR_ITERATIONS);
+
+ thp_save_settings();
+ thp_read_settings(&settings);
+ settings.thp_enabled = THP_MADVISE;
+ settings.thp_defrag = THP_DEFRAG_ALWAYS;
+ settings.khugepaged.defrag = 1;
+ settings.khugepaged.scan_sleep_millisecs = 60000;
+ settings.khugepaged.alloc_sleep_millisecs = 60000;
+ settings.khugepaged.max_ptes_none = (hpage_pmd_size / getpagesize()) - 1;
+ /* One wake must complete one full pass; see khugepaged_full_pass(). */
+ settings.khugepaged.pages_to_scan = 1UL << 24;
+ for (i = 0; i < NR_ORDERS; i++)
+ settings.hugepages[i].enabled = THP_NEVER;
+ settings.hugepages[TARGET_ORDER].enabled = THP_INHERIT;
+ /* Base of the settings stack; the bottom entry is never popped. */
+ thp_push_settings(&settings);
+
+ for (i = 0; i < NR_ITERATIONS; i++)
+ one_step(i);
+
+ thp_restore_settings();
+
+ ksft_finished();
+}
diff --git a/tools/testing/selftests/mm/migration.c b/tools/testing/selftests/mm/migration.c
index f19d53c695764..fd35f8a7b5b83 100644
--- a/tools/testing/selftests/mm/migration.c
+++ b/tools/testing/selftests/mm/migration.c
@@ -20,7 +20,6 @@
#define TWOMEG (2<<20)
#define RUNTIME (20)
-#define ALIGN(x, a) (((x) + (a - 1)) & (~((a) - 1)))
HUGETLB_SETUP_DEFAULT_PAGES(1)
diff --git a/tools/testing/selftests/mm/run_vmtests.sh b/tools/testing/selftests/mm/run_vmtests.sh
index d09f9f6a384ee..fc61907aa3b2a 100755
--- a/tools/testing/selftests/mm/run_vmtests.sh
+++ b/tools/testing/selftests/mm/run_vmtests.sh
@@ -402,6 +402,12 @@ CATEGORY="pfnmap" run_test ./pfnmap
# COW tests
CATEGORY="cow" run_test ./cow
+CATEGORY="thp" run_test ./folio_order_check
+
+CATEGORY="thp" run_test ./khugepaged_sync_check
+
+CATEGORY="thp" run_test ./khugepaged_race
+
CATEGORY="thp" run_test ./khugepaged
CATEGORY="thp" run_test ./khugepaged -s 2
@@ -410,8 +416,6 @@ CATEGORY="thp" run_test ./khugepaged all:shmem
CATEGORY="thp" run_test ./khugepaged -s 4 all:shmem
-CATEGORY="thp" run_test ./khugepaged -c 4 mthp_khugepaged:anon
-
# Try to create XFS if not provided
if [ -z "${SPLIT_HUGE_PAGE_TEST_XFS_PATH}" ]; then
if test_selected "thp"; then
diff --git a/tools/testing/selftests/mm/split_huge_page_test.c b/tools/testing/selftests/mm/split_huge_page_test.c
index 86a6036928261..0adfe7dde7e59 100644
--- a/tools/testing/selftests/mm/split_huge_page_test.c
+++ b/tools/testing/selftests/mm/split_huge_page_test.c
@@ -42,68 +42,6 @@ const char *kpageflags_proc = "/proc/kpageflags";
int pagemap_fd;
int kpageflags_fd;
-static bool is_backed_by_folio(char *vaddr, int order, int pagemap_fd,
- int kpageflags_fd)
-{
- const uint64_t folio_head_flags = KPF_THP | KPF_COMPOUND_HEAD;
- const uint64_t folio_tail_flags = KPF_THP | KPF_COMPOUND_TAIL;
- const unsigned long nr_pages = 1UL << order;
- unsigned long pfn_head;
- uint64_t pfn_flags;
- unsigned long pfn;
- unsigned long i;
-
- pfn = pagemap_get_pfn(pagemap_fd, vaddr);
-
- /* non present page */
- if (pfn == -1UL)
- return false;
-
- if (pageflags_get(pfn, kpageflags_fd, &pfn_flags))
- goto fail;
-
- /* check for order-0 pages */
- if (!order) {
- if (pfn_flags & (folio_head_flags | folio_tail_flags))
- return false;
- return true;
- }
-
- /* non THP folio */
- if (!(pfn_flags & KPF_THP))
- return false;
-
- pfn_head = pfn & ~(nr_pages - 1);
-
- if (pageflags_get(pfn_head, kpageflags_fd, &pfn_flags))
- goto fail;
-
- /* head PFN has no compound_head flag set */
- if ((pfn_flags & folio_head_flags) != folio_head_flags)
- return false;
-
- /* check all tail PFN flags */
- for (i = 1; i < nr_pages; i++) {
- if (pageflags_get(pfn_head + i, kpageflags_fd, &pfn_flags))
- goto fail;
- if ((pfn_flags & folio_tail_flags) != folio_tail_flags)
- return false;
- }
-
- /*
- * check the PFN after this folio, but if its flags cannot be obtained,
- * assume this folio has the expected order
- */
- if (pageflags_get(pfn_head + nr_pages, kpageflags_fd, &pfn_flags))
- return true;
-
- /* If we find another tail page, then the folio is larger. */
- return (pfn_flags & folio_tail_flags) != folio_tail_flags;
-fail:
- ksft_exit_fail_msg("Failed to get folio info\n");
- return false;
-}
-
static int check_after_split_folio_orders(char *vaddr_start, size_t len,
int pagemap_fd, int kpageflags_fd, int orders[], int nr_orders)
{
diff --git a/tools/testing/selftests/mm/vm_util.c b/tools/testing/selftests/mm/vm_util.c
index 80bc9f597b521..ee1334778391f 100644
--- a/tools/testing/selftests/mm/vm_util.c
+++ b/tools/testing/selftests/mm/vm_util.c
@@ -494,6 +494,151 @@ int pageflags_get(unsigned long pfn, int kpageflags_fd, uint64_t *flags)
return 0;
}
+bool is_backed_by_folio(char *vaddr, int order, int pagemap_fd,
+ int kpageflags_fd)
+{
+ const uint64_t folio_head_flags = KPF_THP | KPF_COMPOUND_HEAD;
+ const uint64_t folio_tail_flags = KPF_THP | KPF_COMPOUND_TAIL;
+ const unsigned long nr_pages = 1UL << order;
+ unsigned long pfn_head;
+ uint64_t pfn_flags;
+ unsigned long pfn;
+ unsigned long i;
+
+ pfn = pagemap_get_pfn(pagemap_fd, vaddr);
+
+ /* non present page */
+ if (pfn == -1UL)
+ return false;
+
+ if (pageflags_get(pfn, kpageflags_fd, &pfn_flags))
+ goto fail;
+
+ /* check for order-0 pages */
+ if (!order) {
+ if (pfn_flags & (folio_head_flags | folio_tail_flags))
+ return false;
+ return true;
+ }
+
+ /* non THP folio */
+ if (!(pfn_flags & KPF_THP))
+ return false;
+
+ pfn_head = pfn & ~(nr_pages - 1);
+
+ if (pageflags_get(pfn_head, kpageflags_fd, &pfn_flags))
+ goto fail;
+
+ /* head PFN has no compound_head flag set */
+ if ((pfn_flags & folio_head_flags) != folio_head_flags)
+ return false;
+
+ /* check all tail PFN flags */
+ for (i = 1; i < nr_pages; i++) {
+ if (pageflags_get(pfn_head + i, kpageflags_fd, &pfn_flags))
+ goto fail;
+ if ((pfn_flags & folio_tail_flags) != folio_tail_flags)
+ return false;
+ }
+
+ /*
+ * check the PFN after this folio, but if its flags cannot be obtained,
+ * assume this folio has the expected order
+ */
+ if (pageflags_get(pfn_head + nr_pages, kpageflags_fd, &pfn_flags))
+ return true;
+
+ /* If we find another tail page, then the folio is larger. */
+ return (pfn_flags & folio_tail_flags) != folio_tail_flags;
+fail:
+ ksft_exit_fail_msg("Failed to get folio info\n");
+ return false;
+}
+
+/*
+ * Check whether every order-@order window of [start, len) maps exactly one
+ * folio of that order, head to tail. The address range must be naturally
+ * aligned, each window's PFN run must be contiguous, and a window's first
+ * PFN must be the folio head.
+ *
+ * This is the check "did this range collapse into order-@order folios": a
+ * window assembled from parts of several folios, or mapping a folio shifted
+ * from its natural position, fails.
+ */
+bool is_range_backed_by_folio_orders(char *start, size_t len, int order,
+ int pagemap_fd, int kpageflags_fd)
+{
+ const unsigned long nr_pages = 1UL << order;
+ const size_t window = nr_pages * psize();
+ char *vaddr;
+
+ if ((uintptr_t)start % window || len % window)
+ return false;
+
+ for (vaddr = start; vaddr < start + len; vaddr += window) {
+ unsigned long pfn = pagemap_get_pfn(pagemap_fd, vaddr);
+ unsigned long i;
+
+ /* Not present, or not mapping the folio head. */
+ if (pfn == -1UL || pfn % nr_pages)
+ return false;
+
+ for (i = 1; i < nr_pages; i++) {
+ if (pagemap_get_pfn(pagemap_fd, vaddr + i * psize()) !=
+ pfn + i)
+ return false;
+ }
+
+ if (!is_backed_by_folio(vaddr, order, pagemap_fd,
+ kpageflags_fd))
+ return false;
+ }
+
+ return true;
+}
+
+#define TRACEFS_ROOT "/sys/kernel/tracing"
+
+/*
+ * Open the enable file of one ftrace event subsystem (e.g. "huge_memory").
+ * Returns a descriptor for tracing_events_enable(), or -1 if tracefs or the
+ * subsystem is not there. The events are system-wide state: whoever
+ * switches them on owns them until it switches them off, including on the
+ * paths where the test gives up.
+ */
+int tracing_events_open(const char *subsys)
+{
+ char path[256];
+
+ snprintf(path, sizeof(path), TRACEFS_ROOT "/events/%s/enable",
+ subsys);
+ return open(path, O_WRONLY);
+}
+
+int tracing_events_enable(int fd, bool enable)
+{
+ if (pwrite(fd, enable ? "1" : "0", 1, 0) != 1)
+ return -1;
+ return 0;
+}
+
+/* Drop what the trace buffer holds so far. */
+int tracing_clear_trace(void)
+{
+ int fd = open(TRACEFS_ROOT "/trace", O_WRONLY | O_TRUNC);
+
+ if (fd < 0)
+ return -1;
+ close(fd);
+ return 0;
+}
+
+FILE *tracing_open_trace(void)
+{
+ return fopen(TRACEFS_ROOT "/trace", "r");
+}
+
/* If `ioctls' non-NULL, the allowed ioctls will be returned into the var */
int uffd_register_with_ioctls(int uffd, void *addr, uint64_t len,
bool miss, bool wp, bool minor, uint64_t *ioctls)
diff --git a/tools/testing/selftests/mm/vm_util.h b/tools/testing/selftests/mm/vm_util.h
index 9a49af88702e4..10c7be46e44ca 100644
--- a/tools/testing/selftests/mm/vm_util.h
+++ b/tools/testing/selftests/mm/vm_util.h
@@ -10,6 +10,8 @@
#include <linux/fs.h>
#define BIT_ULL(nr) (1ULL << (nr))
+#define ALIGN(x, a) (((x) + (a) - 1) & ~((a) - 1))
+
#define PM_SOFT_DIRTY BIT_ULL(55)
#define PM_MMAP_EXCLUSIVE BIT_ULL(56)
#define PM_UFFD_WP BIT_ULL(57)
@@ -97,6 +99,10 @@ int64_t allocate_transhuge(void *ptr, int pagemap_fd);
int pageflags_get(unsigned long pfn, int kpageflags_fd, uint64_t *flags);
int gather_folio_orders(char *vaddr_start, size_t len,
int pagemap_fd, int kpageflags_fd, int orders[], int nr_orders);
+bool is_backed_by_folio(char *vaddr, int order, int pagemap_fd,
+ int kpageflags_fd);
+bool is_range_backed_by_folio_orders(char *start, size_t len, int order,
+ int pagemap_fd, int kpageflags_fd);
int uffd_register(int uffd, void *addr, uint64_t len,
bool miss, bool wp, bool minor);
@@ -113,6 +119,10 @@ int close_procmap(struct procmap_fd *procmap);
int write_sysfs(const char *file_path, unsigned long val);
int read_sysfs(const char *file_path, unsigned long *val);
bool softdirty_supported(void);
+int tracing_events_open(const char *subsys);
+int tracing_events_enable(int fd, bool enable);
+int tracing_clear_trace(void);
+FILE *tracing_open_trace(void);
static inline int open_self_procmap(struct procmap_fd *procmap_out)
{
|
| 3/2 | 2026/08/12 14:59 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 28927 output: 63 thoughts: 725 Thoughts: **WorthFuzzing Analysis: Selftest Patch** Okay, so I've got this patch that's adding and modifying tests within `tools/testing/selftests/mm/`. I'm going through the files – `Makefile`, `folio_order_check.c`, `hmm-tests.c`, the hugepage stuff, `khugepaged`, migration, and so on. It seems like a pretty comprehensive addition to the selftests suite. My initial thought is to quickly assess whether this warrants fuzzing attention. The instructions I have lay out a specific set of scenarios where we should return `WorthFuzzing=false`. It lists things like documentation changes, comment updates, decorative logging, constant adjustments that don't change execution, and unreachable code paths. There's also some important caveats about specific hardware and driver contexts. Ah, but wait a second... it *doesn't* explicitly mention selftests in that "WorthFuzzing=false" list. However, selftests are not kernel code. They live in userspace; they're *tools* used to validate the kernel's behavior. The kernel itself *is* what Syzkaller fuzzes using syscalls. The selftests are part of how we measure its performance and behavior, but they aren't part of the system call flow that's the primary target of fuzzing. Let me just do a quick, very explicit, double-check to confirm. Yes, all the changes are squarely within `tools/testing/selftests/mm/`. Nothing in any kernel subsystems here. Therefore, because this patch *only* touches userspace selftests, and does not alter the execution flow of the kernel itself, I am going to set `WorthFuzzing=false`. The reasoning is straightforward: these changes are isolated to the testing framework and are not part of the kernel execution path that Syzkaller fuzzes. Let's call `set-results` to complete this task. |
| 4/2 | 2026/08/12 14:59 | tool | set-results |
0mArgs:{
"EnableConfigs": [],
"FocusSymbols": [],
"Reasoning": "The patch only modifies userspace selftests in tools/testing/selftests/mm/, which do not affect the kernel's functional execution flow.",
"WorthFuzzing": false
}Results: {
"EnableConfigs": null,
"FocusSymbols": [],
"Reasoning": "The patch only modifies userspace selftests in tools/testing/selftests/mm/, which do not affect the kernel's functional execution flow.",
"WorthFuzzing": false
} |
| 5/1 | 2026/08/12 14:59 | action | If |
0mArgs:{
"WorthFuzzing": false
}Results: null |
| Total Calls | Total Tokens | Avg Tokens | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|
| Total Calls | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|