| Seq | Timestamp | Type | Name | Duration |
|---|---|---|---|---|
| 0/0 | 2026/07/17 22:47 | flow | patch-triage |
1mArgs:null Results: {
"EnableConfigs": [
"KSM",
"USERFAULTFD",
"MEMORY_FAILURE",
"TRANSPARENT_HUGEPAGE",
"MIGRATION",
"FS_DAX"
],
"FocusSymbols": [
"mmap_zero_prepare",
"try_to_merge_with_ksm_page",
"rmap_walk_ksm",
"remove_migration_pte",
"copy_vma_and_data",
"move_present_ptes",
"relocate_vma_down"
],
"Reasoning": "The patch modifies the core memory management subsystem to handle MAP_PRIVATE mappings of /dev/zero as truly anonymous mappings, rather than file-backed mappings with a special flag. It introduces virtual page offsets for VMAs to correctly track and merge these mappings. This affects various mm subsystems including KSM, migration, mremap, userfaultfd, and memory failure handling. The changes are functional and reachable from userspace via mmap, mremap, userfaultfd, etc., so they should be fuzzed.",
"WorthFuzzing": true
} |
| 1/1 | 2026/07/17 22:47 | action | read-patch-diff |
0mArgs:null Results: {
"PatchDiff": "commit 6133f60c76a07fbed1d030d1492bac8c961453f5\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Fri Jul 17 22:47:34 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/drivers/char/mem.c b/drivers/char/mem.c\nindex 63253d1de5d70b..dcfd896b733d8f 100644\n--- a/drivers/char/mem.c\n+++ b/drivers/char/mem.c\n@@ -506,11 +506,7 @@ static int mmap_zero_prepare(struct vm_area_desc *desc)\n \tif (vma_desc_test(desc, VMA_SHARED_BIT))\n \t\treturn shmem_zero_setup_desc(desc);\n \n-\t/*\n-\t * This is a highly unique situation where we mark a MAP_PRIVATE mapping\n-\t * of /dev/zero anonymous, despite it not being.\n-\t */\n-\tvma_desc_set_anonymous(desc);\n+\t/* MAP_PRIVATE semantics are taken care for us by core mm. */\n \treturn 0;\n }\n \n@@ -698,7 +694,7 @@ static const struct memdev {\n #ifdef CONFIG_DEVPORT\n \t[4] = { \"port\", \u0026port_fops, 0, 0 },\n #endif\n-\t[5] = { \"zero\", \u0026zero_fops, FMODE_NOWAIT, 0666 },\n+\t[DEVZERO_MINOR] = { \"zero\", \u0026zero_fops, FMODE_NOWAIT, 0666 },\n \t[7] = { \"full\", \u0026full_fops, 0, 0666 },\n \t[8] = { \"random\", \u0026random_fops, FMODE_NOWAIT, 0666 },\n \t[9] = { \"urandom\", \u0026urandom_fops, FMODE_NOWAIT, 0666 },\ndiff --git a/include/linux/mm.h b/include/linux/mm.h\nindex 87feaa5a2b780e..7fabe6c66b4b73 100644\n--- a/include/linux/mm.h\n+++ b/include/linux/mm.h\n@@ -740,6 +740,9 @@ static inline bool fault_flag_allow_retry_first(enum fault_flag flags)\n \t{ FAULT_FLAG_INTERRUPTIBLE,\t\"INTERRUPTIBLE\" }, \\\n \t{ FAULT_FLAG_VMA_LOCK,\t\t\"VMA_LOCK\" }\n \n+/* /dev/zero minor device number. Special due to MAP_PRIVATE semantics. */\n+#define DEVZERO_MINOR\t5\n+\n /*\n * vm_fault is filled by the pagefault handler and passed to the vma's\n * -\u003efault function. The vma's -\u003efault is responsible for returning a bitmask\n@@ -1551,12 +1554,7 @@ static inline void vma_set_anonymous(struct vm_area_struct *vma)\n \tvma-\u003evm_ops = NULL;\n }\n \n-static inline void vma_desc_set_anonymous(struct vm_area_desc *desc)\n-{\n-\tdesc-\u003evm_ops = NULL;\n-}\n-\n-static inline bool vma_is_anonymous(struct vm_area_struct *vma)\n+static inline bool vma_is_anonymous(const struct vm_area_struct *vma)\n {\n \treturn !vma-\u003evm_ops;\n }\n@@ -4349,9 +4347,8 @@ static inline unsigned long vma_pages(const struct vm_area_struct *vma)\n * If @vma is a MAP_PRIVATE file-backed mapping, then this returns the\n * page offset within the file.\n *\n- * Edge cases: nommu does not abide by these, MAP_PRIVATE-/dev/zero satisfies\n- * vma_is_anonymous() but has file-backed page offset, and MAP_PRIVATE-pfnmap\n- * regions have their page offset set to the first PFN in the range.\n+ * Edge cases: nommu does not abide by these and CoW MAP_PRIVATE-pfnmap regions\n+ * have their page offset set to the first PFN in the range.\n *\n * Returns: The page offset of the start of @vma.\n */\n@@ -4393,6 +4390,65 @@ static inline pgoff_t vma_last_pgoff(const struct vm_area_struct *vma)\n \treturn vma_end_pgoff(vma) - 1;\n }\n \n+/**\n+ * vma_start_virt_pgoff() - Get the virtual page offset of the start of @vma\n+ * @vma: The VMA whose virtual page offset is required.\n+ *\n+ * If unfaulted, then this is vma-\u003evm_start \u003e\u003e PAGE_SHIFT, if faulted then the\n+ * virtual page offset at the time of first fault.\n+ *\n+ * If the VMA is anonymous, this returns the same value as vma_start_pgoff().\n+ *\n+ * This value is used for tracking MAP_PRIVATE file-backed mappings by their\n+ * virtual page offset.\n+ *\n+ * Returns: The virtual page offset of the start of @vma.\n+ */\n+static inline pgoff_t vma_start_virt_pgoff(const struct vm_area_struct *vma)\n+{\n+\tpgoff_t pgoff = 0;\n+\n+#ifdef CONFIG_64BIT\n+\tpgoff += vma-\u003e__vm_virt_pgoff_hi;\n+\tpgoff \u003c\u003c= 32;\n+#endif\n+\tpgoff += vma-\u003e__vm_virt_pgoff_lo;\n+\treturn pgoff;\n+}\n+\n+/**\n+ * vma_end_virt_pgoff() - Get the virtual page offset of the exclusive end of\n+ * @vma.\n+ * @vma: The VMA whose end virtual page offset is required.\n+ *\n+ * This returns the virtual exclusive end page offset of @vma, which is useful\n+ * for expressing page offset ranges.\n+ *\n+ * See the description of vma_start_virt_pgoff() for a description of VMA\n+ * virtual page offsets.\n+ *\n+ * Returns: The exclusive end virtual page offset of @vma.\n+ */\n+static inline pgoff_t vma_end_virt_pgoff(const struct vm_area_struct *vma)\n+{\n+\treturn vma_start_virt_pgoff(vma) + vma_pages(vma);\n+}\n+\n+/**\n+ * vma_last_virt_pgoff() - Get the virtual page offset of the last page in\n+ * @vma.\n+ * @vma: The VMA whose last virtual page offset is required.\n+ *\n+ * See the description of vma_start_virt_pgoff() for a description of VMA\n+ * virtual page offsets.\n+ *\n+ * Returns: The last virtual page offset of @vma.\n+ */\n+static inline pgoff_t vma_last_virt_pgoff(const struct vm_area_struct *vma)\n+{\n+\treturn vma_end_virt_pgoff(vma) - 1;\n+}\n+\n static inline unsigned long vma_desc_size(const struct vm_area_desc *desc)\n {\n \treturn desc-\u003eend - desc-\u003estart;\ndiff --git a/include/linux/mm_types.h b/include/linux/mm_types.h\nindex 939b5ea8c9e0be..2710628059b15b 100644\n--- a/include/linux/mm_types.h\n+++ b/include/linux/mm_types.h\n@@ -967,6 +967,7 @@ struct vm_area_struct {\n \t */\n \tunsigned int vm_lock_seq;\n #endif\n+\tunsigned int __vm_virt_pgoff_lo; /* Low 32-bits of virtual pgoff. */\n \t/*\n \t * A file's MAP_PRIVATE vma can be in both i_mmap tree and anon_vma\n \t * list, after a COW of one of the file pages.\tA MAP_SHARED vma\n@@ -1041,6 +1042,9 @@ struct vm_area_struct {\n #ifdef CONFIG_DEBUG_LOCK_ALLOC\n \tstruct lockdep_map vmlock_dep_map;\n #endif\n+#endif\n+#ifdef CONFIG_64BIT\n+\tunsigned int __vm_virt_pgoff_hi; /* High 32-bits of virtual pgoff. */\n #endif\n \t/*\n \t * For areas with an address space and backing store,\ndiff --git a/include/linux/pagemap.h b/include/linux/pagemap.h\nindex c6fc783aaee576..90130f28e7a87d 100644\n--- a/include/linux/pagemap.h\n+++ b/include/linux/pagemap.h\n@@ -1101,6 +1101,72 @@ static inline pgoff_t linear_page_index(const struct vm_area_struct *vma,\n \treturn pgoff;\n }\n \n+static inline pgoff_t __linear_virt_page_index(const struct vm_area_struct *vma,\n+\t\t\t\t\t const unsigned long address)\n+{\n+\tpgoff_t pgoff;\n+\n+\tpgoff = linear_page_delta(vma, address);\n+\tpgoff += vma_start_virt_pgoff(vma);\n+\treturn pgoff;\n+}\n+\n+/**\n+ * linear_virt_page_index() - Determine the absolute virtual page offset of\n+ * @address within @vma.\n+ * @vma: An anonymous or MAP_PRIVATE file-backed VMA in which @address resides.\n+ * @address: The address whose absolute page offset is required.\n+ *\n+ * This returns the virtual page offset of @address, which is the page offset\n+ * the address possessed at the time the VMA was first faulted.\n+ *\n+ * For anonymous mappings, this returns the same value as linear_page_index().\n+ *\n+ * For MAP_PRIVATE file-backed mappings, this returns the virtual page offset of\n+ * @address, which is the page offset the address possessed at the time the VMA\n+ * was first faulted.\n+ *\n+ * It is not valid to call this function for shared file-backed mappings.\n+ *\n+ * Returns: The absolute virtual page offset of @address within @vma.\n+ */\n+static inline pgoff_t linear_virt_page_index(const struct vm_area_struct *vma,\n+\t\t\t\t\t const unsigned long address)\n+{\n+\tconst pgoff_t pgoff = __linear_virt_page_index(vma, address);\n+\n+\tVM_WARN_ON_ONCE(vma_test(vma, VMA_SHARED_BIT));\n+\tif (!vma-\u003evm_file) /* Is anonymous except MAP_PRIVATE-/dev/zero */\n+\t\tVM_WARN_ON_ONCE(pgoff != linear_page_index(vma, address));\n+\n+\treturn pgoff;\n+}\n+\n+/**\n+ * linear_folio_page_index() - Determine the absolute page offset of\n+ * @address within @vma from @folio.\n+ * @folio: The folio whose linear page index is sought.\n+ * @vma: The VMA in which @address resides.\n+ * @address: The address whose absolute page offset is required.\n+ *\n+ * Determines whether to obtain the virtual linear page index based on whether\n+ * @folio is anonymous or not.\n+ *\n+ * See the descriptions of linear_virt_page_index() and linear_page_index() for\n+ * details of each.\n+ *\n+ * Returns: The absolute page offset of @address within @vma.\n+ */\n+static inline pgoff_t linear_folio_page_index(const struct folio *folio,\n+\t\t\t\t\t const struct vm_area_struct *vma,\n+\t\t\t\t\t const unsigned long address)\n+{\n+\tif (folio_test_anon(folio))\n+\t\treturn linear_virt_page_index(vma, address);\n+\n+\treturn linear_page_index(vma, address);\n+}\n+\n struct wait_page_key {\n \tstruct folio *folio;\n \tint bit_nr;\ndiff --git a/include/linux/rmap.h b/include/linux/rmap.h\nindex 8dc0871e5f0011..a48ae9575bd262 100644\n--- a/include/linux/rmap.h\n+++ b/include/linux/rmap.h\n@@ -871,6 +871,7 @@ struct page_vma_mapped_walk {\n \tpte_t *pte;\n \tspinlock_t *ptl;\n \tunsigned int flags;\n+\tbool is_anon_walk;\n };\n \n #define DEFINE_FOLIO_VMA_WALK(name, _folio, _vma, _address, _flags)\t\\\n@@ -881,6 +882,7 @@ struct page_vma_mapped_walk {\n \t\t.vma = _vma,\t\t\t\t\t\t\\\n \t\t.address = _address,\t\t\t\t\t\\\n \t\t.flags = _flags,\t\t\t\t\t\\\n+\t\t.is_anon_walk = folio_test_anon(_folio),\t\t\\\n \t}\n \n static inline void page_vma_mapped_walk_done(struct page_vma_mapped_walk *pvmw)\ndiff --git a/mm/huge_memory.c b/mm/huge_memory.c\nindex 9b1f3b24f7e0d0..abc65d608c2392 100644\n--- a/mm/huge_memory.c\n+++ b/mm/huge_memory.c\n@@ -2887,7 +2887,8 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pm\n \t\t}\n \n \t\tfolio_move_anon_rmap(src_folio, dst_vma);\n-\t\tsrc_folio-\u003eindex = linear_page_index(dst_vma, dst_addr);\n+\t\tsrc_folio-\u003eindex = linear_folio_page_index(src_folio, dst_vma,\n+\t\t\t\t\t\t\t dst_addr);\n \n \t\t_dst_pmd = folio_mk_pmd(src_folio, dst_vma-\u003evm_page_prot);\n \t\t/* Follow mremap() behavior and treat the entry dirty after the move */\ndiff --git a/mm/internal.h b/mm/internal.h\nindex f26423de4ca287..41561fdeb56d0a 100644\n--- a/mm/internal.h\n+++ b/mm/internal.h\n@@ -240,6 +240,10 @@ static inline int mmap_file(struct file *file, struct vm_area_struct *vma)\n {\n \tint err = vfs_mmap(file, vma);\n \n+\t/* Hooks cannot mark themselves anonymous. */\n+\tif (WARN_ON_ONCE(vma_is_anonymous(vma)))\n+\t\terr = -EINVAL;\n+\n \tif (likely(!err))\n \t\treturn 0;\n \n@@ -933,7 +937,8 @@ folio_within_range(struct folio *folio, struct vm_area_struct *vma,\n \t\treturn false;\n \n \tpgoff_folio = folio_pgoff(folio);\n-\tpgoff_vma_start = vma_start_pgoff(vma);\n+\tpgoff_vma_start = folio_test_anon(folio) ?\n+\t\tvma_start_virt_pgoff(vma) : vma_start_pgoff(vma);\n \n \tif (start \u003c vma-\u003evm_start)\n \t\tstart = vma-\u003evm_start;\n@@ -1005,19 +1010,9 @@ void mlock_drain_remote(int cpu);\n \n extern pmd_t maybe_pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma);\n \n-/**\n- * vma_address - Find the virtual address a page range is mapped at\n- * @vma: The vma which maps this object.\n- * @pgoff: The page offset within its object.\n- * @nr_pages: The number of pages to consider.\n- *\n- * If any page in this range is mapped by this VMA, return the first address\n- * where any of these pages appear. Otherwise, return -EFAULT.\n- */\n-static inline unsigned long vma_address(const struct vm_area_struct *vma,\n-\t\tpgoff_t pgoff, unsigned long nr_pages)\n+static inline unsigned long __vma_address(const struct vm_area_struct *vma,\n+\t\tpgoff_t pgoff, pgoff_t pgoff_start, unsigned long nr_pages)\n {\n-\tconst pgoff_t pgoff_start = vma_start_pgoff(vma);\n \tunsigned long address;\n \n \tif (pgoff \u003e= pgoff_start) {\n@@ -1035,23 +1030,68 @@ static inline unsigned long vma_address(const struct vm_area_struct *vma,\n \treturn address;\n }\n \n+/**\n+ * vma_filebacked_address - Find the virtual address a file-backed page range is\n+ * mapped at.\n+ * @vma: The vma which maps this object.\n+ * @pgoff: The page offset within its object.\n+ * @nr_pages: The number of pages to consider.\n+ *\n+ * Returns: If any page in this range is mapped by this VMA, return the first\n+ * address where any of these pages appear. Otherwise, return -EFAULT.\n+ */\n+static inline unsigned long vma_filebacked_address(const struct vm_area_struct *vma,\n+\t\tpgoff_t pgoff, unsigned long nr_pages)\n+{\n+\tVM_WARN_ON_ONCE(vma_is_anonymous(vma));\n+\n+\treturn __vma_address(vma, pgoff, vma_start_pgoff(vma), nr_pages);\n+}\n+\n+/**\n+ * vma_anon_address - Find the virtual address an anonymous page range is mapped\n+ * at.\n+ * @vma: The vma which maps this object.\n+ * @pgoff_virt: The virtual page index belonging to the folio.\n+ * @nr_pages: The number of pages to consider.\n+ *\n+ * This is only valid for anonymous or MAP_PRIVATE-mapped file-backed VMAs.\n+ *\n+ * Returns: If any page in this range is mapped by this VMA, return the first address\n+ * where any of these pages appear. Otherwise, return -EFAULT.\n+ */\n+static inline unsigned long vma_anon_address(const struct vm_area_struct *vma,\n+\t\tpgoff_t pgoff_virt, unsigned long nr_pages)\n+{\n+\tVM_WARN_ON_ONCE(!vma_is_anonymous(vma) \u0026\u0026 vma_test(vma, VMA_SHARED_BIT));\n+\n+\treturn __vma_address(vma, pgoff_virt, vma_start_virt_pgoff(vma), nr_pages);\n+}\n+\n /*\n- * Then at what user virtual address will none of the range be found in vma?\n+ * At what user virtual address will none of the range be found in vma?\n * Assumes that vma_address() already returned a good starting address.\n */\n static inline unsigned long vma_address_end(struct page_vma_mapped_walk *pvmw)\n {\n-\tstruct vm_area_struct *vma = pvmw-\u003evma;\n-\tpgoff_t pgoff;\n+\tconst struct vm_area_struct *vma = pvmw-\u003evma;\n+\tconst pgoff_t pgoff = pvmw-\u003epgoff;\n+\tpgoff_t pgoff_vma_start;\n \tunsigned long address;\n+\tpgoff_t pgoff_end;\n \n \t/* Common case, plus -\u003epgoff is invalid for KSM */\n \tif (pvmw-\u003enr_pages == 1)\n \t\treturn pvmw-\u003eaddress + PAGE_SIZE;\n \n-\tpgoff = pvmw-\u003epgoff + pvmw-\u003enr_pages;\n+\tif (pvmw-\u003eis_anon_walk)\n+\t\tpgoff_vma_start = vma_start_virt_pgoff(vma);\n+\telse\n+\t\tpgoff_vma_start = vma_start_pgoff(vma);\n+\n+\tpgoff_end = pgoff + pvmw-\u003enr_pages;\n \taddress = vma-\u003evm_start +\n-\t\t((pgoff - vma_start_pgoff(vma)) \u003c\u003c PAGE_SHIFT);\n+\t\t((pgoff_end - pgoff_vma_start) \u003c\u003c PAGE_SHIFT);\n \t/* Check for address beyond vma (or wrapped through 0?) */\n \tif (address \u003c vma-\u003evm_start || address \u003e vma-\u003evm_end)\n \t\taddress = vma-\u003evm_end;\ndiff --git a/mm/interval_tree.c b/mm/interval_tree.c\nindex 3ae9e106d3afd8..26b8437e3b1b13 100644\n--- a/mm/interval_tree.c\n+++ b/mm/interval_tree.c\n@@ -83,12 +83,12 @@ mapping_rmap_tree_iter_next(struct vm_area_struct *vma,\n \n static pgoff_t avc_start_pgoff(struct anon_vma_chain *avc)\n {\n-\treturn vma_start_pgoff(avc-\u003evma);\n+\treturn vma_start_virt_pgoff(avc-\u003evma);\n }\n \n static pgoff_t avc_last_pgoff(struct anon_vma_chain *avc)\n {\n-\treturn vma_last_pgoff(avc-\u003evma);\n+\treturn vma_last_virt_pgoff(avc-\u003evma);\n }\n \n INTERVAL_TREE_DEFINE(struct anon_vma_chain, rb, pgoff_t, rb_subtree_last,\ndiff --git a/mm/ksm.c b/mm/ksm.c\nindex 47006f494fcb1b..b9b32f7bfb691e 100644\n--- a/mm/ksm.c\n+++ b/mm/ksm.c\n@@ -1625,7 +1625,8 @@ static int try_to_merge_with_ksm_page(struct ksm_rmap_item *rmap_item,\n \t * stable_tree, break_cow() will clean it up.\n \t */\n \trmap_item-\u003eanon_vma = vma-\u003eanon_vma;\n-\trmap_item-\u003elinear_page_index = linear_page_index(vma, rmap_item-\u003eaddress);\n+\t/* The VMA is always anon/MAP_PRIVATE-file backed so use anon index. */\n+\trmap_item-\u003elinear_page_index = linear_virt_page_index(vma, rmap_item-\u003eaddress);\n \tget_anon_vma(vma-\u003eanon_vma);\n out:\n \tmmap_read_unlock(mm);\n@@ -3152,7 +3153,7 @@ struct folio *ksm_might_need_to_copy(struct folio *folio,\n \t\t\treturn folio;\t/* no need to copy it */\n \t} else if (!anon_vma) {\n \t\treturn folio;\t\t/* no need to copy it */\n-\t} else if (folio-\u003eindex == linear_page_index(vma, addr) \u0026\u0026\n+\t} else if (folio-\u003eindex == linear_virt_page_index(vma, addr) \u0026\u0026\n \t\t\tanon_vma-\u003eroot == vma-\u003eanon_vma-\u003eroot) {\n \t\treturn folio;\t\t/* still no need to copy it */\n \t}\n@@ -3222,7 +3223,7 @@ void rmap_walk_ksm(struct folio *folio, struct rmap_walk_control *rwc)\n \t\t/*\n \t\t * Currently, KSM folios are always small folios, so it's\n \t\t * sufficient to search for a single page. We can simply use\n-\t\t * the linear_page_index of the original de-duplicate\n+\t\t * the linear_virt_page_index of the original de-duplicate\n \t\t * anonymous page that we remembered in the rmap_item while\n \t\t * de-duplicating. Note that mremap() always de-duplicates KSM\n \t\t * folios: so if there was mremap() in our parent or our child,\ndiff --git a/mm/memory-failure.c b/mm/memory-failure.c\nindex aaf14608b30e2d..a8b03e2920ba8a 100644\n--- a/mm/memory-failure.c\n+++ b/mm/memory-failure.c\n@@ -620,7 +620,7 @@ static void add_to_kill_fsdax(struct task_struct *tsk, const struct page *p,\n \t\t\t struct vm_area_struct *vma,\n \t\t\t struct list_head *to_kill, pgoff_t pgoff)\n {\n-\tunsigned long addr = vma_address(vma, pgoff, 1);\n+\tunsigned long addr = vma_filebacked_address(vma, pgoff, 1);\n \t__add_to_kill(tsk, p, vma, to_kill, addr);\n }\n \n@@ -2265,7 +2265,7 @@ static void add_to_kill_pgoff(struct task_struct *tsk,\n \t}\n \n \t/* Check for pgoff not backed by struct page */\n-\ttk-\u003eaddr = vma_address(vma, pgoff, 1);\n+\ttk-\u003eaddr = vma_filebacked_address(vma, pgoff, 1);\n \ttk-\u003esize_shift = PAGE_SHIFT;\n \n \tif (tk-\u003eaddr == -EFAULT)\ndiff --git a/mm/memory.c b/mm/memory.c\nindex d5e87624f69205..56b244552f1394 100644\n--- a/mm/memory.c\n+++ b/mm/memory.c\n@@ -631,13 +631,14 @@ static void print_bad_page_map(struct vm_area_struct *vma,\n {\n \tstruct address_space *mapping;\n \tchar entry_str[PTVAL_STR_MAX];\n-\tpgoff_t index;\n+\tpgoff_t index, virt_index;\n \n \tif (is_bad_page_map_ratelimited())\n \t\treturn;\n \n \tmapping = vma-\u003evm_file ? vma-\u003evm_file-\u003ef_mapping : NULL;\n \tindex = linear_page_index(vma, addr);\n+\tvirt_index = __linear_virt_page_index(vma, addr);\n \n \tptval_bytes_to_hex_str(entry_str, sizeof(entry_str), entry, entry_size);\n \tpr_alert(\"BUG: Bad page map in process %s %s:%s\", current-\u003ecomm,\n@@ -645,8 +646,9 @@ static void print_bad_page_map(struct vm_area_struct *vma,\n \t__print_bad_page_map_pgtable(vma-\u003evm_mm, addr);\n \tif (page)\n \t\tdump_page(page, \"bad page map\");\n-\tpr_alert(\"addr:%px vm_flags:%08lx anon_vma:%px mapping:%px index:%lx\\n\",\n-\t\t (void *)addr, vma-\u003evm_flags, vma-\u003eanon_vma, mapping, index);\n+\tpr_alert(\"addr:%px vm_flags:%08lx anon_vma:%px mapping:%px index:%lx virt_index:%lx\\n\",\n+\t\t (void *)addr, vma-\u003evm_flags, vma-\u003eanon_vma, mapping, index,\n+\t\t virt_index);\n \tpr_alert(\"file:%pD fault:%ps mmap:%ps mmap_prepare: %ps read_folio:%ps\\n\",\n \t\t vma-\u003evm_file,\n \t\t vma-\u003evm_ops ? vma-\u003evm_ops-\u003efault : NULL,\ndiff --git a/mm/migrate.c b/mm/migrate.c\nindex 222c8c15f782f7..37fe7a9b3facc8 100644\n--- a/mm/migrate.c\n+++ b/mm/migrate.c\n@@ -363,8 +363,10 @@ static bool remove_migration_pte(struct folio *folio,\n \t\tunsigned long idx = 0;\n \n \t\t/* pgoff is invalid for ksm pages, but they are never large */\n-\t\tif (folio_test_large(folio) \u0026\u0026 !folio_test_hugetlb(folio))\n-\t\t\tidx = linear_page_index(vma, pvmw.address) - pvmw.pgoff;\n+\t\tif (folio_test_large(folio) \u0026\u0026 !folio_test_hugetlb(folio)) {\n+\t\t\tidx += linear_folio_page_index(folio, vma, pvmw.address);\n+\t\t\tidx -= pvmw.pgoff;\n+\t\t}\n \t\tnew = folio_page(folio, idx);\n \n #ifdef CONFIG_ARCH_HAS_PMD_SOFTLEAVES\ndiff --git a/mm/mremap.c b/mm/mremap.c\nindex b64aa1f6e07ede..f07fc4e3ef2e6c 100644\n--- a/mm/mremap.c\n+++ b/mm/mremap.c\n@@ -1265,7 +1265,9 @@ static void unmap_source_vma(struct vma_remap_struct *vrm)\n static int copy_vma_and_data(struct vma_remap_struct *vrm,\n \t\t\t struct vm_area_struct **new_vma_ptr)\n {\n-\tconst unsigned long new_pgoff = linear_page_index(vrm-\u003evma, vrm-\u003eaddr);\n+\tconst pgoff_t new_pgoff = linear_page_index(vrm-\u003evma, vrm-\u003eaddr);\n+\tconst pgoff_t new_virt_pgoff =\n+\t\t__linear_virt_page_index(vrm-\u003evma, vrm-\u003eaddr);\n \tstruct vm_area_struct *vma = vrm-\u003evma;\n \tstruct vm_area_struct *new_vma;\n \tunsigned long moved_len;\n@@ -1273,7 +1275,7 @@ static int copy_vma_and_data(struct vma_remap_struct *vrm,\n \tPAGETABLE_MOVE(pmc, NULL, NULL, vrm-\u003eaddr, vrm-\u003enew_addr, vrm-\u003eold_len);\n \n \tnew_vma = copy_vma(\u0026vma, vrm-\u003enew_addr, vrm-\u003enew_len, new_pgoff,\n-\t\t\t \u0026pmc.need_rmap_locks);\n+\t\t\t new_virt_pgoff, \u0026pmc.need_rmap_locks);\n \tif (!new_vma) {\n \t\tvrm_uncharge(vrm);\n \t\t*new_vma_ptr = NULL;\ndiff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c\nindex d7670ba4147bff..4e964545e5e85a 100644\n--- a/mm/page_vma_mapped.c\n+++ b/mm/page_vma_mapped.c\n@@ -356,6 +356,7 @@ unsigned long page_mapped_in_vma(const struct page *page,\n \t\tstruct vm_area_struct *vma)\n {\n \tconst struct folio *folio = page_folio(page);\n+\tconst pgoff_t pgoff = page_pgoff(folio, page);\n \tstruct page_vma_mapped_walk pvmw = {\n \t\t.pfn = page_to_pfn(page),\n \t\t.nr_pages = 1,\n@@ -363,7 +364,10 @@ unsigned long page_mapped_in_vma(const struct page *page,\n \t\t.flags = PVMW_SYNC,\n \t};\n \n-\tpvmw.address = vma_address(vma, page_pgoff(folio, page), 1);\n+\tif (folio_test_anon(folio))\n+\t\tpvmw.address = vma_anon_address(vma, pgoff, 1);\n+\telse\n+\t\tpvmw.address = vma_filebacked_address(vma, pgoff, 1);\n \tif (pvmw.address == -EFAULT)\n \t\tgoto out;\n \tif (!page_vma_mapped_walk(\u0026pvmw))\ndiff --git a/mm/rmap.c b/mm/rmap.c\nindex ad820fe86f7d8c..6854baf4b8778f 100644\n--- a/mm/rmap.c\n+++ b/mm/rmap.c\n@@ -865,14 +865,15 @@ unsigned long page_address_in_vma(const struct folio *folio,\n \t\tif (!vma-\u003eanon_vma || !anon_vma ||\n \t\t vma-\u003eanon_vma-\u003eroot != anon_vma-\u003eroot)\n \t\t\treturn -EFAULT;\n+\t\t/* KSM folios don't reach here because of the !anon_vma check */\n+\t\treturn vma_anon_address(vma, page_pgoff(folio, page), 1);\n \t} else if (!vma-\u003evm_file) {\n \t\treturn -EFAULT;\n \t} else if (vma-\u003evm_file-\u003ef_mapping != folio-\u003emapping) {\n \t\treturn -EFAULT;\n \t}\n \n-\t/* KSM folios don't reach here because of the !anon_vma check */\n-\treturn vma_address(vma, page_pgoff(folio, page), 1);\n+\treturn vma_filebacked_address(vma, page_pgoff(folio, page), 1);\n }\n \n /*\n@@ -1239,8 +1240,10 @@ static bool mapping_wrprotect_range_one(struct folio *folio,\n \t\t.vma\t\t= vma,\n \t\t.address\t= address,\n \t\t.flags\t\t= PVMW_SYNC,\n+\t\t.is_anon_walk = false,\n \t};\n \n+\tVM_WARN_ON_ONCE(folio_test_anon(folio));\n \tstate-\u003ecleaned += page_vma_mkclean_one(\u0026pvmw);\n \n \treturn true;\n@@ -1316,12 +1319,13 @@ int pfn_mkclean_range(unsigned long pfn, unsigned long nr_pages, pgoff_t pgoff,\n \t\t.pgoff\t\t= pgoff,\n \t\t.vma\t\t= vma,\n \t\t.flags\t\t= PVMW_SYNC,\n+\t\t.is_anon_walk = false,\n \t};\n \n \tif (invalid_mkclean_vma(vma, NULL))\n \t\treturn 0;\n \n-\tpvmw.address = vma_address(vma, pgoff, nr_pages);\n+\tpvmw.address = vma_filebacked_address(vma, pgoff, nr_pages);\n \tVM_BUG_ON_VMA(pvmw.address == -EFAULT, vma);\n \n \treturn page_vma_mkclean_one(\u0026pvmw);\n@@ -1482,7 +1486,7 @@ static void __folio_set_anon(struct folio *folio, struct vm_area_struct *vma,\n \t */\n \tanon_vma = (void *) anon_vma + FOLIO_MAPPING_ANON;\n \tWRITE_ONCE(folio-\u003emapping, (struct address_space *) anon_vma);\n-\tfolio-\u003eindex = linear_page_index(vma, address);\n+\tfolio-\u003eindex = linear_virt_page_index(vma, address);\n }\n \n /**\n@@ -1509,8 +1513,8 @@ static void __page_check_anon_rmap(const struct folio *folio,\n \t */\n \tVM_BUG_ON_FOLIO(folio_anon_vma(folio)-\u003eroot != vma-\u003eanon_vma-\u003eroot,\n \t\t\tfolio);\n-\tVM_BUG_ON_PAGE(page_pgoff(folio, page) != linear_page_index(vma, address),\n-\t\t page);\n+\tVM_BUG_ON_PAGE(page_pgoff(folio, page) !=\n+\t\t linear_virt_page_index(vma, address), page);\n }\n \n static __always_inline void __folio_add_anon_rmap(struct folio *folio,\n@@ -3037,10 +3041,10 @@ static void rmap_walk_anon(struct folio *folio,\n \tpgoff_end = pgoff_start + folio_nr_pages(folio) - 1;\n \tanon_rmap_tree_foreach(avc, anon_vma, pgoff_start, pgoff_end) {\n \t\tstruct vm_area_struct *vma = avc-\u003evma;\n-\t\tunsigned long address = vma_address(vma, pgoff_start,\n+\t\tconst unsigned long address = vma_anon_address(vma, pgoff_start,\n \t\t\t\tfolio_nr_pages(folio));\n \n-\t\tVM_BUG_ON_VMA(address == -EFAULT, vma);\n+\t\tVM_WARN_ON_ONCE_VMA(address == -EFAULT, vma);\n \t\tcond_resched();\n \n \t\tif (rwc-\u003einvalid_vma \u0026\u0026 rwc-\u003einvalid_vma(vma, rwc-\u003earg))\n@@ -3100,7 +3104,8 @@ static void __rmap_walk_file(struct folio *folio, struct address_space *mapping,\n \t}\n lookup:\n \tmapping_rmap_tree_foreach(vma, mapping, pgoff_start, pgoff_end) {\n-\t\tunsigned long address = vma_address(vma, pgoff_start, nr_pages);\n+\t\tunsigned long address = vma_filebacked_address(vma, pgoff_start,\n+\t\t\t\t\t\t\t nr_pages);\n \n \t\tVM_BUG_ON_VMA(address == -EFAULT, vma);\n \t\tcond_resched();\ndiff --git a/mm/userfaultfd.c b/mm/userfaultfd.c\nindex 8fd24c8b428e1b..258b03182a780b 100644\n--- a/mm/userfaultfd.c\n+++ b/mm/userfaultfd.c\n@@ -1352,7 +1352,8 @@ static long move_present_ptes(struct mm_struct *mm,\n \t\t}\n \n \t\tfolio_move_anon_rmap(src_folio, dst_vma);\n-\t\tsrc_folio-\u003eindex = linear_page_index(dst_vma, dst_addr);\n+\t\tsrc_folio-\u003eindex = linear_folio_page_index(src_folio, dst_vma,\n+\t\t\t\t\t\t\t dst_addr);\n \n \t\torig_dst_pte = folio_mk_pte(src_folio, dst_vma-\u003evm_page_prot);\n \t\t/* Set soft dirty bit so userspace can notice the pte was moved */\n@@ -1428,7 +1429,8 @@ static int move_swap_pte(struct mm_struct *mm, struct vm_area_struct *dst_vma,\n \t */\n \tif (src_folio) {\n \t\tfolio_move_anon_rmap(src_folio, dst_vma);\n-\t\tsrc_folio-\u003eindex = linear_page_index(dst_vma, dst_addr);\n+\t\tsrc_folio-\u003eindex = linear_folio_page_index(src_folio, dst_vma,\n+\t\t\t\t\t\t\t dst_addr);\n \t} else {\n \t\t/*\n \t\t * Check if the swap entry is cached after acquiring the src_pte\ndiff --git a/mm/vma.c b/mm/vma.c\nindex b5bc3eec961c79..f8001d5c23aad1 100644\n--- a/mm/vma.c\n+++ b/mm/vma.c\n@@ -18,6 +18,7 @@ struct mmap_state {\n \tunsigned long addr;\n \tunsigned long end;\n \tpgoff_t pgoff;\n+\tpgoff_t virt_pgoff;\n \tunsigned long pglen;\n \tunion {\n \t\tvm_flags_t vm_flags;\n@@ -46,13 +47,22 @@ struct mmap_state {\n \tbool file_doesnt_need_get :1;\n };\n \n-#define MMAP_STATE(name, mm_, vmi_, addr_, len_, pgoff_, vma_flags_, file_) \\\n+static inline pgoff_t map_anon_pgoff(const struct mmap_state *map)\n+{\n+\tif (vma_flags_test(\u0026map-\u003evma_flags, VMA_SHARED_BIT))\n+\t\treturn map-\u003epgoff;\n+\n+\treturn map-\u003evirt_pgoff;\n+}\n+\n+#define MMAP_STATE(name, mm_, vmi_, addr_, len_, pgoff_, virt_pgoff_, vma_flags_, file_) \\\n \tstruct mmap_state name = {\t\t\t\t\t\\\n \t\t.mm = mm_,\t\t\t\t\t\t\\\n \t\t.vmi = vmi_,\t\t\t\t\t\t\\\n \t\t.addr = addr_,\t\t\t\t\t\t\\\n \t\t.end = (addr_) + (len_),\t\t\t\t\\\n \t\t.pgoff = pgoff_,\t\t\t\t\t\\\n+\t\t.virt_pgoff = virt_pgoff_,\t\t\t\t\\\n \t\t.pglen = PHYS_PFN(len_),\t\t\t\t\\\n \t\t.vma_flags = vma_flags_,\t\t\t\t\\\n \t\t.file = file_,\t\t\t\t\t\t\\\n@@ -67,6 +77,7 @@ struct mmap_state {\n \t\t.end = (map_)-\u003eend,\t\t\t\t\t\\\n \t\t.vma_flags = (map_)-\u003evma_flags,\t\t\t\t\\\n \t\t.pgoff = (map_)-\u003epgoff,\t\t\t\t\t\\\n+\t\t.anon_pgoff = map_anon_pgoff(map_),\t\t\t\\\n \t\t.file = (map_)-\u003efile,\t\t\t\t\t\\\n \t\t.prev = (map_)-\u003eprev,\t\t\t\t\t\\\n \t\t.middle = vma_,\t\t\t\t\t\t\\\n@@ -82,10 +93,11 @@ static void __vma_set_range(struct vm_area_struct *vma, unsigned long start,\n }\n \n static void vma_set_range(struct vm_area_struct *vma, unsigned long start,\n-\t\t\t unsigned long end, pgoff_t pgoff)\n+\t\t\t unsigned long end, pgoff_t pgoff, pgoff_t virt_pgoff)\n {\n \t__vma_set_range(vma, start, end);\n \tvma_set_pgoff(vma, pgoff);\n+\tvma_set_virt_pgoff(vma, virt_pgoff);\n }\n \n /* Was this VMA ever forked from a parent, i.e. maybe contains CoW mappings? */\n@@ -221,6 +233,8 @@ static bool can_vma_merge_before(struct vma_merge_struct *vmg)\n \t\treturn false;\n \tif (vmg_end_pgoff(vmg) != vma_start_pgoff(vmg-\u003enext))\n \t\treturn false;\n+\tif (vmg_end_anon_pgoff(vmg) != vma_start_anon_pgoff(vmg-\u003enext))\n+\t\treturn false;\n \treturn true;\n }\n \n@@ -241,6 +255,8 @@ static bool can_vma_merge_after(struct vma_merge_struct *vmg)\n \t\treturn false;\n \tif (vma_end_pgoff(vmg-\u003eprev) != vmg_start_pgoff(vmg))\n \t\treturn false;\n+\tif (vma_end_anon_pgoff(vmg-\u003eprev) != vmg_start_anon_pgoff(vmg))\n+\t\treturn false;\n \treturn true;\n }\n \n@@ -812,7 +828,8 @@ static int commit_merge(struct vma_merge_struct *vmg)\n \t */\n \tvma_adjust_trans_huge(vma, vmg-\u003estart, vmg-\u003eend,\n \t\t\t vmg-\u003e__adjust_middle_start ? vmg-\u003emiddle : NULL);\n-\tvma_set_range(vma, vmg-\u003estart, vmg-\u003eend, vmg_start_pgoff(vmg));\n+\tvma_set_range(vma, vmg-\u003estart, vmg-\u003eend, vmg_start_pgoff(vmg),\n+\t\t vmg_start_anon_pgoff(vmg));\n \tvmg_adjust_set_range(vmg);\n \tvma_iter_store_overwrite(vmg-\u003evmi, vmg-\u003etarget);\n \n@@ -982,6 +999,7 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(\n \t\tvmg-\u003estart = prev-\u003evm_start;\n \t\tvmg-\u003eend = next-\u003evm_end;\n \t\tvmg-\u003epgoff = vma_start_pgoff(prev);\n+\t\tvmg-\u003eanon_pgoff = vma_start_anon_pgoff(prev);\n \n \t\t/*\n \t\t * We already ensured anon_vma compatibility above, so now it's\n@@ -1000,6 +1018,7 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(\n \t\t */\n \t\tvmg-\u003estart = prev-\u003evm_start;\n \t\tvmg-\u003epgoff = vma_start_pgoff(prev);\n+\t\tvmg-\u003eanon_pgoff = vma_start_anon_pgoff(prev);\n \n \t\tif (!vmg-\u003e__remove_middle)\n \t\t\tvmg-\u003e__adjust_middle_start = true;\n@@ -1022,12 +1041,14 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(\n \t\tif (vmg-\u003e__remove_middle) {\n \t\t\tvmg-\u003eend = next-\u003evm_end;\n \t\t\tvmg-\u003epgoff = vma_start_pgoff(next) - pglen;\n+\t\t\tvmg-\u003eanon_pgoff = vma_start_anon_pgoff(next) - pglen;\n \t\t} else {\n \t\t\t/* We shrink middle and expand next. */\n \t\t\tvmg-\u003e__adjust_next_start = true;\n \t\t\tvmg-\u003estart = middle-\u003evm_start;\n \t\t\tvmg-\u003eend = start;\n \t\t\tvmg-\u003epgoff = vma_start_pgoff(middle);\n+\t\t\tvmg-\u003eanon_pgoff = vma_start_anon_pgoff(middle);\n \t\t}\n \n \t\terr = dup_anon_vma(next, middle, \u0026anon_dup);\n@@ -1137,6 +1158,7 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)\n \t\tvmg-\u003estart = prev-\u003evm_start;\n \t\tvmg-\u003etarget = prev;\n \t\tvmg-\u003epgoff = vma_start_pgoff(prev);\n+\t\tvmg-\u003eanon_pgoff = vma_start_anon_pgoff(prev);\n \n \t\t/*\n \t\t * If this merge would result in removal of the next VMA but we\n@@ -1908,9 +1930,10 @@ static int vma_link(struct mm_struct *mm, struct vm_area_struct *vma)\n */\n struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,\n \tunsigned long addr, unsigned long len, pgoff_t pgoff,\n-\tbool *need_rmap_locks)\n+\tpgoff_t virt_pgoff, bool *need_rmap_locks)\n {\n \tstruct vm_area_struct *vma = *vmap;\n+\tconst bool is_shared = vma_test(vma, VMA_SHARED_BIT);\n \tunsigned long vma_start = vma-\u003evm_start;\n \tstruct mm_struct *mm = vma-\u003evm_mm;\n \tstruct vm_area_struct *new_vma;\n@@ -1919,11 +1942,14 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,\n \tVMG_VMA_STATE(vmg, \u0026vmi, NULL, vma, addr, addr + len);\n \n \t/*\n-\t * If anonymous vma has not yet been faulted, update new pgoff\n-\t * to match new location, to increase its chance of merging.\n+\t * If a vma has not yet been faulted, update its virtual pgoff to match\n+\t * the new location to increase its chance of merging.\n \t */\n-\tif (unlikely(vma_is_anonymous(vma) \u0026\u0026 !vma-\u003eanon_vma)) {\n-\t\tpgoff = addr \u003e\u003e PAGE_SHIFT;\n+\tif (!vma-\u003eanon_vma \u0026\u0026 !is_shared) {\n+\t\tvirt_pgoff = addr \u003e\u003e PAGE_SHIFT;\n+\n+\t\tif (vma_is_anonymous(vma))\n+\t\t\tpgoff = virt_pgoff;\n \t\tfaulted_in_anon_vma = false;\n \t}\n \n@@ -1940,6 +1966,7 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,\n \t\treturn NULL;\t/* should never get here */\n \n \tvmg.pgoff = pgoff;\n+\tvmg.anon_pgoff = is_shared ? pgoff : virt_pgoff;\n \tvmg.next = vma_iter_next_rewind(\u0026vmi, NULL);\n \tnew_vma = vma_merge_copied_range(\u0026vmg);\n \n@@ -1961,16 +1988,17 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,\n \t\t\t * safe. It is only safe to keep the vm_pgoff\n \t\t\t * linear if there are no pages mapped yet.\n \t\t\t */\n-\t\t\tVM_BUG_ON_VMA(faulted_in_anon_vma, new_vma);\n+\t\t\tVM_WARN_ON_ONCE_VMA(faulted_in_anon_vma, new_vma);\n \t\t\t*vmap = vma = new_vma;\n \t\t}\n \t\t*need_rmap_locks =\n-\t\t\t(vma_start_pgoff(new_vma) \u003c= vma_start_pgoff(vma));\n+\t\t\t(vma_start_pgoff(new_vma) \u003c= vma_start_pgoff(vma)) ||\n+\t\t\t(vma_start_anon_pgoff(new_vma) \u003c= vma_start_anon_pgoff(vma));\n \t} else {\n \t\tnew_vma = vm_area_dup(vma);\n \t\tif (!new_vma)\n \t\t\tgoto out;\n-\t\tvma_set_range(new_vma, addr, addr + len, pgoff);\n+\t\tvma_set_range(new_vma, addr, addr + len, pgoff, virt_pgoff);\n \t\tif (vma_dup_policy(vma, new_vma))\n \t\t\tgoto out_free_vma;\n \t\tif (anon_vma_clone(new_vma, vma, VMA_OP_REMAP))\n@@ -2036,7 +2064,12 @@ static int anon_vma_compatible(struct vm_area_struct *a, struct vm_area_struct *\n \tif (!vma_flags_empty(\u0026diff))\n \t\treturn false;\n \t/* Page offset must align. */\n-\treturn vma_end_pgoff(a) == vma_start_pgoff(b);\n+\tif (vma_end_pgoff(a) != vma_start_pgoff(b))\n+\t\treturn false;\n+\t/* Anon page offset must align. */\n+\tif (vma_end_anon_pgoff(a) != vma_start_anon_pgoff(b))\n+\t\treturn false;\n+\treturn true;\n }\n \n /*\n@@ -2579,6 +2612,32 @@ static int __mmap_new_file_vma(struct mmap_state *map,\n \treturn 0;\n }\n \n+static bool map_is_dev_zero(const struct mmap_state *map)\n+{\n+\tconst struct file *file = map-\u003efile;\n+\tconst struct inode *inode = file_inode(file);\n+\n+\treturn imajor(inode) == MEM_MAJOR \u0026\u0026 iminor(inode) == DEVZERO_MINOR;\n+}\n+\n+static void map_set_anon(struct mmap_state *map)\n+{\n+\tmap-\u003efile = NULL;\n+\tmap-\u003efile_doesnt_need_get = false;\n+\tmap-\u003epgoff = map-\u003eaddr \u003e\u003e PAGE_SHIFT;\n+\tmap-\u003evm_ops = NULL;\n+}\n+\n+static bool map_is_private(const struct mmap_state *map)\n+{\n+\treturn !vma_flags_test(\u0026map-\u003evma_flags, VMA_SHARED_BIT);\n+}\n+\n+static bool map_is_anon(const struct mmap_state *map)\n+{\n+\treturn map_is_private(map) \u0026\u0026 !map-\u003efile;\n+}\n+\n /*\n * __mmap_new_vma() - Allocate a new VMA for the region, as merging was not\n * possible.\n@@ -2592,8 +2651,7 @@ static int __mmap_new_file_vma(struct mmap_state *map,\n static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap,\n \tstruct mmap_action *action)\n {\n-\tconst bool is_anon = !map-\u003efile \u0026\u0026\n-\t\t!vma_flags_test(\u0026map-\u003evma_flags, VMA_SHARED_BIT);\n+\tconst bool is_anon = map_is_anon(map);\n \tstruct vma_iterator *vmi = map-\u003evmi;\n \tint error = 0;\n \tstruct vm_area_struct *vma;\n@@ -2612,7 +2670,7 @@ static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap,\n \tif (is_anon)\n \t\tvma_set_anonymous(vma);\n \n-\tvma_set_range(vma, map-\u003eaddr, map-\u003eend, map-\u003epgoff);\n+\tvma_set_range(vma, map-\u003eaddr, map-\u003eend, map-\u003epgoff, map-\u003evirt_pgoff);\n \tvma-\u003eflags = map-\u003evma_flags;\n \tvma-\u003evm_page_prot = map-\u003epage_prot;\n \n@@ -2735,6 +2793,10 @@ static int call_mmap_prepare(struct mmap_state *map,\n \tif (err)\n \t\treturn err;\n \n+\t/* Hooks cannot mark themselves anonymous. */\n+\tif (!desc-\u003evm_ops)\n+\t\treturn -EINVAL;\n+\n \terr = call_action_prepare(map, desc);\n \tif (err)\n \t\treturn err;\n@@ -2751,16 +2813,21 @@ static int call_mmap_prepare(struct mmap_state *map,\n \tmap-\u003evm_ops = desc-\u003evm_ops;\n \tmap-\u003evm_private_data = desc-\u003eprivate_data;\n \n+\t/*\n+\t * MAP_PRIVATE-/dev/zero mappings are an ancient way of getting\n+\t * anonymous mappings. Rather than allowing these mappings to be odd\n+\t * outliers, simply make them truly anonymous.\n+\t */\n+\tif (map_is_private(map) \u0026\u0026 map_is_dev_zero(map))\n+\t\tmap_set_anon(map);\n+\n \treturn 0;\n }\n \n static void set_vma_user_defined_fields(struct vm_area_struct *vma,\n \t\tstruct mmap_state *map)\n {\n-\tif (map-\u003evm_ops)\n-\t\tvma-\u003evm_ops = map-\u003evm_ops;\n-\telse\t/* Only /dev/zero should do this. */\n-\t\tvma_set_anonymous(vma);\n+\tvma-\u003evm_ops = map-\u003evm_ops;\n \tvma-\u003evm_private_data = map-\u003evm_private_data;\n }\n \n@@ -2801,7 +2868,8 @@ static unsigned long __mmap_region(struct file *file, unsigned long addr,\n \tstruct vm_area_struct *vma = NULL;\n \tbool have_mmap_prepare = file \u0026\u0026 file-\u003ef_op-\u003emmap_prepare;\n \tVMA_ITERATOR(vmi, mm, addr);\n-\tMMAP_STATE(map, mm, \u0026vmi, addr, len, pgoff, vma_flags, file);\n+\tconst pgoff_t virt_pgoff = addr \u003e\u003e PAGE_SHIFT;\n+\tMMAP_STATE(map, mm, \u0026vmi, addr, len, pgoff, virt_pgoff, vma_flags, file);\n \tstruct vm_area_desc desc = {\n \t\t.mm = mm,\n \t\t.file = file,\n@@ -2946,6 +3014,7 @@ int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *vma,\n \t\t unsigned long addr, unsigned long len, vma_flags_t vma_flags)\n {\n \tstruct mm_struct *mm = current-\u003emm;\n+\tconst pgoff_t pgoff = addr \u003e\u003e PAGE_SHIFT;\n \n \t/*\n \t * Check against address space limits by the changed size\n@@ -2970,7 +3039,7 @@ int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *vma,\n \t * occur after forking, so the expand will only happen on new VMAs.\n \t */\n \tif (vma \u0026\u0026 vma-\u003evm_end == addr) {\n-\t\tVMG_STATE(vmg, mm, vmi, addr, addr + len, vma_flags, PHYS_PFN(addr));\n+\t\tVMG_STATE(vmg, mm, vmi, addr, addr + len, vma_flags, pgoff, pgoff);\n \n \t\tvmg.prev = vma;\n \t\t/* vmi is positioned at prev, which this mode expects. */\n@@ -2990,7 +3059,7 @@ int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *vma,\n \t\tgoto unacct_fail;\n \n \tvma_set_anonymous(vma);\n-\tvma_set_range(vma, addr, addr + len, addr \u003e\u003e PAGE_SHIFT);\n+\tvma_set_range(vma, addr, addr + len, pgoff, pgoff);\n \tvma-\u003eflags = vma_flags;\n \tvma-\u003evm_page_prot = vm_get_page_prot(vma_flags_to_legacy(vma_flags));\n \tvma_start_write(vma);\n@@ -3382,6 +3451,7 @@ int insert_vm_struct(struct mm_struct *mm, struct vm_area_struct *vma)\n \t\tWARN_ON_ONCE(vma-\u003eanon_vma);\n \t\tvma_set_pgoff(vma, vma-\u003evm_start \u003e\u003e PAGE_SHIFT);\n \t}\n+\tvma_set_virt_pgoff(vma, vma-\u003evm_start \u003e\u003e PAGE_SHIFT);\n \n \tif (vma_link(mm, vma)) {\n \t\tif (vma_test(vma, VMA_ACCOUNT_BIT))\n@@ -3437,7 +3507,7 @@ struct vm_area_struct *__install_special_mapping(\n \n \tvma-\u003evm_ops = ops;\n \tvma-\u003evm_private_data = priv;\n-\tvma_set_range(vma, addr, addr + len, 0);\n+\tvma_set_range(vma, addr, addr + len, 0, addr \u003e\u003e PAGE_SHIFT);\n \n \tret = insert_vm_struct(mm, vma);\n \tif (ret)\ndiff --git a/mm/vma.h b/mm/vma.h\nindex 0bc7d521e97677..8caee3b4ba2671 100644\n--- a/mm/vma.h\n+++ b/mm/vma.h\n@@ -104,6 +104,7 @@ struct vma_merge_struct {\n \tunsigned long start;\n \tunsigned long end;\n \tpgoff_t pgoff;\n+\tpgoff_t anon_pgoff;\n \n \tunion {\n \t\t/* Temporary while VMA flags are being converted. */\n@@ -237,11 +238,6 @@ static inline bool vmg_nomem(struct vma_merge_struct *vmg)\n \treturn vmg-\u003estate == VMA_MERGE_ERROR_NOMEM;\n }\n \n-static inline pgoff_t vmg_start_pgoff(const struct vma_merge_struct *vmg)\n-{\n-\treturn vmg-\u003epgoff;\n-}\n-\n static inline pgoff_t vmg_pages(const struct vma_merge_struct *vmg)\n {\n \tconst unsigned long size = vmg-\u003eend - vmg-\u003estart;\n@@ -249,6 +245,11 @@ static inline pgoff_t vmg_pages(const struct vma_merge_struct *vmg)\n \treturn size \u003e\u003e PAGE_SHIFT;\n }\n \n+static inline pgoff_t vmg_start_pgoff(const struct vma_merge_struct *vmg)\n+{\n+\treturn vmg-\u003epgoff;\n+}\n+\n static inline pgoff_t vmg_end_pgoff(const struct vma_merge_struct *vmg)\n {\n \treturn vmg_start_pgoff(vmg) + vmg_pages(vmg);\n@@ -266,9 +267,6 @@ static inline void assert_sane_pgoff(struct vm_area_struct *vma, pgoff_t pgoff)\n \t */\n \tif (!vma_is_anonymous(vma))\n \t\treturn;\n-\t/* MAP_PRIVATE-/dev/zero is anon, non-NULL vm_file, but has file pgoff. */\n-\tif (vma-\u003evm_file)\n-\t\treturn;\n \t/* If faulted in, could have been remapped. */\n \tif (vma-\u003eanon_vma)\n \t\treturn;\n@@ -283,48 +281,130 @@ static inline void vma_set_pgoff(struct vm_area_struct *vma, pgoff_t pgoff)\n \tvma-\u003evm_pgoff = pgoff;\n }\n \n+static inline pgoff_t vmg_start_anon_pgoff(const struct vma_merge_struct *vmg)\n+{\n+\treturn vmg-\u003eanon_pgoff;\n+}\n+\n+static inline pgoff_t vmg_end_anon_pgoff(const struct vma_merge_struct *vmg)\n+{\n+\treturn vmg_start_anon_pgoff(vmg) + vmg_pages(vmg);\n+}\n+\n+static inline void __vma_set_virt_pgoff(struct vm_area_struct *vma, pgoff_t pgoff)\n+{\n+#ifdef CONFIG_64BIT\n+\tvma-\u003e__vm_virt_pgoff_hi = pgoff \u003e\u003e 32;\n+#endif\n+\tvma-\u003e__vm_virt_pgoff_lo = pgoff \u0026 GENMASK(31, 0);\n+}\n+\n+static inline void vma_set_virt_pgoff(struct vm_area_struct *vma, pgoff_t pgoff)\n+{\n+\tvma_assert_can_modify(vma);\n+\t__vma_set_virt_pgoff(vma, pgoff);\n+}\n+\n static inline void vma_add_pgoff(struct vm_area_struct *vma, pgoff_t delta)\n {\n \tvma_assert_can_modify(vma);\n \tvma_set_pgoff(vma, vma_start_pgoff(vma) + delta);\n+\tvma_set_virt_pgoff(vma, vma_start_virt_pgoff(vma) + delta);\n }\n \n static inline void vma_sub_pgoff(struct vm_area_struct *vma, pgoff_t delta)\n {\n \tvma_assert_can_modify(vma);\n \tvma_set_pgoff(vma, vma_start_pgoff(vma) - delta);\n+\tvma_set_virt_pgoff(vma, vma_start_virt_pgoff(vma) - delta);\n+}\n+\n+/**\n+ * vma_anon_pgoff_addr() - Calculates the absolute anonymous page offset of\n+ * @address.\n+ * @vma: The VMA whose anonymous page offset is required.\n+ * @address: The address whose absolute page offset is required.\n+ *\n+ * If the VMA is a shared file-backed mapping, then the file-based page offset\n+ * is returned.\n+ *\n+ * Otherwise, the virtual page offset is returned.\n+ *\n+ * This means that shared file-backed mappings are correctly merged based on\n+ * their file page offset compatibility.\n+ *\n+ * Returns: The absolute anonymous page offset of @address within @vma.\n+ */\n+static inline pgoff_t vma_anon_pgoff_addr(const struct vm_area_struct *vma,\n+\t\t\t\t\t unsigned long address)\n+{\n+\tif (vma_test(vma, VMA_SHARED_BIT))\n+\t\treturn linear_page_index(vma, address);\n+\n+\treturn linear_virt_page_index(vma, address);\n }\n \n-#define VMG_STATE(name, mm_, vmi_, start_, end_, vma_flags_, pgoff_)\t\\\n+/**\n+ * vma_start_anon_pgoff() - Calculates the absolute anonymous page offset used\n+ * for purposes of merge compatibility.\n+ * @vma: The VMA whose anonymous page offset is required.\n+ *\n+ * See vma_anon_pgoff_addr().\n+ *\n+ * Returns: The absolute anonymous page offset of @vma for purposes of merging.\n+ */\n+static inline pgoff_t vma_start_anon_pgoff(const struct vm_area_struct *vma)\n+{\n+\treturn vma_anon_pgoff_addr(vma, vma-\u003evm_start);\n+}\n+\n+/**\n+ * vma_end_anon_pgoff() - Calculates the absolute exclusive end anonymous page\n+ * offset used for purposes of merge compatibility.\n+ * @vma: The VMA whosse anonymous end page offset is required.\n+ *\n+ * See vma_start_anon_pgoff().\n+ *\n+ * Returns: The absolute exclusive end anonymous page offset of @vma for\n+ * purposes of merging.\n+ */\n+static inline pgoff_t vma_end_anon_pgoff(const struct vm_area_struct *vma)\n+{\n+\treturn vma_start_anon_pgoff(vma) + vma_pages(vma);\n+}\n+\n+#define VMG_STATE(name, mm_, vmi_, start_, end_, vma_flags_, pgoff_, anon_pgoff_) \\\n+\tstruct vma_merge_struct name = {\t\t\t\t\t \\\n+\t\t.mm = mm_,\t\t\t\t\t\t\t \\\n+\t\t.vmi = vmi_,\t\t\t\t\t\t\t \\\n+\t\t.start = start_,\t\t\t\t\t\t \\\n+\t\t.end = end_,\t\t\t\t\t\t\t \\\n+\t\t.vma_flags = vma_flags_,\t\t\t\t\t \\\n+\t\t.pgoff = pgoff_,\t\t\t\t\t\t \\\n+\t\t.anon_pgoff = anon_pgoff_,\t\t\t\t\t \\\n+\t\t.state = VMA_MERGE_START,\t\t\t\t\t \\\n+\t}\n+\n+#define VMG_VMA_STATE(name, vmi_, prev_, vma_, start_, end_)\t\t\\\n \tstruct vma_merge_struct name = {\t\t\t\t\\\n-\t\t.mm = mm_,\t\t\t\t\t\t\\\n+\t\t.mm = vma_-\u003evm_mm,\t\t\t\t\t\\\n \t\t.vmi = vmi_,\t\t\t\t\t\t\\\n+\t\t.prev = prev_,\t\t\t\t\t\t\\\n+\t\t.middle = vma_,\t\t\t\t\t\t\\\n+\t\t.next = NULL,\t\t\t\t\t\t\\\n \t\t.start = start_,\t\t\t\t\t\\\n \t\t.end = end_,\t\t\t\t\t\t\\\n-\t\t.vma_flags = vma_flags_,\t\t\t\t\\\n-\t\t.pgoff = pgoff_,\t\t\t\t\t\\\n+\t\t.vm_flags = vma_-\u003evm_flags,\t\t\t\t\\\n+\t\t.pgoff = linear_page_index(vma_, start_),\t\t\\\n+\t\t.anon_pgoff = vma_anon_pgoff_addr(vma_, start_),\t\\\n+\t\t.file = vma_-\u003evm_file,\t\t\t\t\t\\\n+\t\t.anon_vma = vma_-\u003eanon_vma,\t\t\t\t\\\n+\t\t.policy = vma_policy(vma_),\t\t\t\t\\\n+\t\t.uffd_ctx = vma_-\u003evm_userfaultfd_ctx,\t\t\t\\\n+\t\t.anon_name = anon_vma_name(vma_),\t\t\t\\\n \t\t.state = VMA_MERGE_START,\t\t\t\t\\\n \t}\n \n-#define VMG_VMA_STATE(name, vmi_, prev_, vma_, start_, end_)\t\\\n-\tstruct vma_merge_struct name = {\t\t\t\\\n-\t\t.mm = vma_-\u003evm_mm,\t\t\t\t\\\n-\t\t.vmi = vmi_,\t\t\t\t\t\\\n-\t\t.prev = prev_,\t\t\t\t\t\\\n-\t\t.middle = vma_,\t\t\t\t\t\\\n-\t\t.next = NULL,\t\t\t\t\t\\\n-\t\t.start = start_,\t\t\t\t\\\n-\t\t.end = end_,\t\t\t\t\t\\\n-\t\t.vm_flags = vma_-\u003evm_flags,\t\t\t\\\n-\t\t.pgoff = linear_page_index(vma_, start_),\t\\\n-\t\t.file = vma_-\u003evm_file,\t\t\t\t\\\n-\t\t.anon_vma = vma_-\u003eanon_vma,\t\t\t\\\n-\t\t.policy = vma_policy(vma_),\t\t\t\\\n-\t\t.uffd_ctx = vma_-\u003evm_userfaultfd_ctx,\t\t\\\n-\t\t.anon_name = anon_vma_name(vma_),\t\t\\\n-\t\t.state = VMA_MERGE_START,\t\t\t\\\n-\t}\n-\n #ifdef CONFIG_DEBUG_VM_MAPLE_TREE\n void validate_mm(struct mm_struct *mm);\n #else\n@@ -506,7 +586,7 @@ void unlink_file_vma_batch_add(struct unlink_vma_file_batch *vb,\n \n struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,\n \tunsigned long addr, unsigned long len, pgoff_t pgoff,\n-\tbool *need_rmap_locks);\n+\tpgoff_t anon_pgoff, bool *need_rmap_locks);\n \n struct anon_vma *find_mergeable_anon_vma(struct vm_area_struct *vma);\n \ndiff --git a/mm/vma_exec.c b/mm/vma_exec.c\nindex 7af1260689b97e..586c5215594245 100644\n--- a/mm/vma_exec.c\n+++ b/mm/vma_exec.c\n@@ -41,7 +41,7 @@ int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift)\n \tunsigned long new_end = old_end - shift;\n \tVMA_ITERATOR(vmi, mm, new_start);\n \tVMG_STATE(vmg, mm, \u0026vmi, new_start, old_end, EMPTY_VMA_FLAGS,\n-\t\t vma_start_pgoff(vma));\n+\t\t vma_start_pgoff(vma), vma_start_anon_pgoff(vma));\n \tstruct vm_area_struct *next;\n \tstruct mmu_gather tlb;\n \tPAGETABLE_MOVE(pmc, vma, vma, old_start, new_start, length);\ndiff --git a/mm/vma_init.c b/mm/vma_init.c\nindex 715feee283f0fb..710b18849a3696 100644\n--- a/mm/vma_init.c\n+++ b/mm/vma_init.c\n@@ -51,6 +51,7 @@ static void vm_area_init_from(const struct vm_area_struct *src,\n \tdest-\u003evm_end = src-\u003evm_end;\n \tdest-\u003eanon_vma = src-\u003eanon_vma;\n \tdest-\u003evm_pgoff = vma_start_pgoff(src);\n+\t__vma_set_virt_pgoff(dest, vma_start_virt_pgoff(src));\n \tdest-\u003evm_file = src-\u003evm_file;\n \tdest-\u003evm_private_data = src-\u003evm_private_data;\n \tvm_flags_init(dest, src-\u003evm_flags);\ndiff --git a/mm/vma_internal.h b/mm/vma_internal.h\nindex 4d300e7bbaf4c2..385c0ab1377748 100644\n--- a/mm/vma_internal.h\n+++ b/mm/vma_internal.h\n@@ -23,6 +23,7 @@\n #include \u003clinux/ksm.h\u003e\n #include \u003clinux/khugepaged.h\u003e\n #include \u003clinux/list.h\u003e\n+#include \u003clinux/major.h\u003e\n #include \u003clinux/maple_tree.h\u003e\n #include \u003clinux/mempolicy.h\u003e\n #include \u003clinux/mm.h\u003e\ndiff --git a/tools/testing/selftests/mm/merge.c b/tools/testing/selftests/mm/merge.c\nindex 519e5ac02db78c..edfbd39ae58eb4 100644\n--- a/tools/testing/selftests/mm/merge.c\n+++ b/tools/testing/selftests/mm/merge.c\n@@ -1305,6 +1305,167 @@ TEST_F(merge, merge_vmas_with_mseal)\n \tASSERT_EQ(procmap-\u003equery.vma_end, (unsigned long)ptr + 2 * page_size);\n }\n \n+TEST_F(merge, virt_and_page_offset_mismatch_memfd)\n+{\n+\tstruct procmap_fd *procmap = \u0026self-\u003eprocmap;\n+\tunsigned int page_size = self-\u003epage_size;\n+\tchar *carveout = self-\u003ecarveout;\n+\tchar *ptr, *ptr2;\n+\tint fd;\n+\n+\t/* Create a 10 page memfd descriptor. */\n+\tfd = memfd_create(\"virt_page_offset_test\", MFD_CLOEXEC);\n+\tASSERT_NE(fd, -1);\n+\tASSERT_EQ(ftruncate(fd, 10 * page_size), 0);\n+\n+\t/* Map a region using the memfd at page offset 0. */\n+\tptr = mmap(carveout, 5 * page_size, PROT_READ | PROT_WRITE,\n+\t\t MAP_FIXED | MAP_PRIVATE, fd, 0);\n+\tASSERT_NE(ptr, MAP_FAILED);\n+\n+\t/*\n+\t * Map another separately, and fault in, at page offset 5:\n+\t *\n+\t * |-----------| |---------|\n+\t * | unfaulted | | faulted |\n+\t * |-----------| |---------|\n+\t */\n+\tptr2 = mmap(\u0026carveout[10 * page_size], 5 * page_size,\n+\t\t PROT_READ | PROT_WRITE, MAP_FIXED | MAP_PRIVATE,\n+\t\t fd, 5 * page_size);\n+\tASSERT_NE(ptr2, MAP_FAILED);\n+\tptr2[0] = 'x';\n+\n+\t/*\n+\t * Now move it in place:\n+\t *\n+\t * |----------|\n+\t * | |\n+\t * v |\n+\t * |-----------| |---------|\n+\t * | unfaulted | | faulted |\n+\t * |-----------| |---------|\n+\t *\n+\t * Because virtual page offset of the faulted region is now\n+\t * \u0026carveout[10 * page_size], despite the two regions being mergeable\n+\t * due to file page offset, they are NOT mergeable due to virtual page\n+\t * offset.\n+\t */\n+\tptr2 = sys_mremap(ptr2, 5 * page_size, 5 * page_size,\n+\t\t\t MREMAP_MAYMOVE | MREMAP_FIXED,\n+\t\t\t \u0026carveout[5 * page_size]);\n+\tASSERT_NE(ptr2, MAP_FAILED);\n+\n+\t/* Assert that they did not merge. */\n+\tASSERT_TRUE(find_vma_procmap(procmap, ptr));\n+\tASSERT_EQ(procmap-\u003equery.vma_start, (unsigned long)ptr);\n+\tASSERT_EQ(procmap-\u003equery.vma_end, (unsigned long)ptr + 5 * page_size);\n+}\n+\n+TEST_F(merge, merge_map_private_dev_zero_unfaulted)\n+{\n+\tstruct procmap_fd *procmap = \u0026self-\u003eprocmap;\n+\tunsigned int page_size = self-\u003epage_size;\n+\tchar *carveout = self-\u003ecarveout;\n+\tchar *ptr, *ptr2;\n+\tint fd_zero;\n+\n+\tif (access(\"/dev/zero\", F_OK))\n+\t\tSKIP(return, \"No /dev/zero.\");\n+\tfd_zero = open(\"/dev/zero\", O_RDWR);\n+\tASSERT_NE(fd_zero, -1);\n+\n+\t/*\n+\t * Map two MAP_PRIVATE-/dev/zero VMAs next to one another with offset 0\n+\t * each.\n+\t *\n+\t * With these being made truly anonymous upon mapping, they will\n+\t * merge. If they were file-backed VMAs the page offsets would prevent\n+\t * merge:\n+\t *\n+\t * |-----||------| |-------------|\n+\t * | ptr || ptr2 | -\u003e | ptr |\n+\t * |-----||------| |-------------|\n+\t */\n+\tptr = mmap(carveout, 5 * page_size, PROT_READ | PROT_WRITE,\n+\t\t MAP_FIXED | MAP_PRIVATE, fd_zero, 0);\n+\tif (ptr == MAP_FAILED) {\n+\t\tclose(fd_zero);\n+\t\tASSERT_TRUE(false);\n+\t}\n+\tptr2 = mmap(\u0026carveout[5 * page_size], 5 * page_size,\n+\t\t PROT_READ | PROT_WRITE, MAP_FIXED | MAP_PRIVATE, fd_zero, 0);\n+\tif (ptr2 == MAP_FAILED) {\n+\t\tclose(fd_zero);\n+\t\tASSERT_TRUE(false);\n+\t}\n+\tclose(fd_zero);\n+\n+\t/* Assert that they merged. */\n+\tASSERT_TRUE(find_vma_procmap(procmap, ptr));\n+\tASSERT_EQ(procmap-\u003equery.vma_start, (unsigned long)ptr);\n+\tASSERT_EQ(procmap-\u003equery.vma_end, (unsigned long)ptr + 10 * page_size);\n+}\n+\n+TEST_F(merge, merge_map_private_dev_zero_faulted_unfaulted)\n+{\n+\tstruct procmap_fd *procmap = \u0026self-\u003eprocmap;\n+\tunsigned int page_size = self-\u003epage_size;\n+\tchar *carveout = self-\u003ecarveout;\n+\tchar *ptr, *ptr2;\n+\tint fd_zero;\n+\n+\tif (access(\"/dev/zero\", F_OK))\n+\t\tSKIP(return, \"No /dev/zero.\");\n+\tfd_zero = open(\"/dev/zero\", O_RDWR);\n+\tASSERT_NE(fd_zero, -1);\n+\n+\t/*\n+\t * Map a MAP_PRIVATE mapping of /dev/zero with page offset 0, then fault\n+\t * it in:\n+\t *\n+\t * |-------------------------------|\n+\t * | faulted |\n+\t * |-------------------------------|\n+\t */\n+\tptr = mmap(carveout, 15 * page_size, PROT_READ | PROT_WRITE,\n+\t\t MAP_FIXED | MAP_PRIVATE, fd_zero, 0);\n+\tif (ptr == MAP_FAILED) {\n+\t\tclose(fd_zero);\n+\t\tASSERT_TRUE(false);\n+\t}\n+\tmemset(ptr, 'x', 15 * page_size);\n+\n+\t/*\n+\t * Unmap the middle:\n+\t *\n+\t * |---------| |---------|\n+\t * | faulted | | faulted |\n+\t * |---------| |---------|\n+\t */\n+\tASSERT_EQ(munmap(\u0026ptr[5 * page_size], 5 * page_size), 0);\n+\n+\t/*\n+\t * Map in a new unfaulted mapping in the middle with page offset 0 -\n+\t * this should merge and would not if it were treated as a file rather\n+\t * than pure anon:\n+\t *\n+\t * |---------|-----------|---------|\n+\t * | faulted | unfaulted | faulted |\n+\t * |---------|-----------|---------|\n+\t */\n+\tptr2 = mmap(\u0026carveout[5 * page_size], 5 * page_size,\n+\t\t PROT_READ | PROT_WRITE, MAP_FIXED | MAP_PRIVATE,\n+\t\t fd_zero, 0);\n+\tclose(fd_zero);\n+\tASSERT_NE(ptr2, MAP_FAILED);\n+\n+\t/* Assert that they merged. */\n+\tASSERT_TRUE(find_vma_procmap(procmap, ptr));\n+\tASSERT_EQ(procmap-\u003equery.vma_start, (unsigned long)ptr);\n+\tASSERT_EQ(procmap-\u003equery.vma_end, (unsigned long)ptr + 15 * page_size);\n+}\n+\n TEST_F(merge_with_fork, mremap_faulted_to_unfaulted_prev)\n {\n \tstruct procmap_fd *procmap = \u0026self-\u003eprocmap;\ndiff --git a/tools/testing/vma/include/dup.h b/tools/testing/vma/include/dup.h\nindex c52b23773cd263..b4f0f7c24fcb8a 100644\n--- a/tools/testing/vma/include/dup.h\n+++ b/tools/testing/vma/include/dup.h\n@@ -15,6 +15,16 @@ struct task_struct *get_current(void);\n #define MMF_HAS_MDWE\t28\n #define current get_current()\n \n+#define MINORBITS\t20\n+#define MINORMASK\t((1U \u003c\u003c MINORBITS) - 1)\n+\n+#define MAJOR(dev)\t((unsigned int) ((dev) \u003e\u003e MINORBITS))\n+#define MINOR(dev)\t((unsigned int) ((dev) \u0026 MINORMASK))\n+#define MKDEV(ma, mi)\t(((ma) \u003c\u003c MINORBITS) | (mi))\n+\n+#define MEM_MAJOR\t\t1\n+#define DEVZERO_MINOR\t5\n+\n /*\n * Define the task command name length as enum, then it can be visible to\n * BPF programs.\n@@ -45,6 +55,9 @@ struct address_space {\n \tunsigned long\t\tflags;\n \tatomic_t\t\ti_mmap_writable;\n };\n+struct inode {\n+\tdev_t\t\t\ti_rdev;\n+};\n struct file_operations {\n \tint (*mmap)(struct file *, struct vm_area_struct *);\n \tint (*mmap_prepare)(struct vm_area_desc *);\n@@ -52,6 +65,7 @@ struct file_operations {\n struct file {\n \tstruct address_space\t*f_mapping;\n \tconst struct file_operations\t*f_op;\n+\tstruct inode\t\t\t*f_inode;\n };\n struct anon_vma_chain {\n \tstruct anon_vma *anon_vma;\n@@ -577,6 +591,7 @@ struct vm_area_struct {\n \t */\n \tunsigned int vm_lock_seq;\n #endif\n+\tunsigned int __vm_virt_pgoff_lo;\n \n \t/*\n \t * A file's MAP_PRIVATE vma can be in both i_mmap tree and anon_vma\n@@ -612,6 +627,9 @@ struct vm_area_struct {\n #ifdef CONFIG_PER_VMA_LOCK\n \t/* Unstable RCU readers are allowed to read this. */\n \trefcount_t vm_refcnt;\n+#endif\n+#ifdef CONFIG_64BIT\n+\tunsigned int __vm_virt_pgoff_hi;\n #endif\n \t/*\n \t * For areas with an address space and backing store,\n@@ -1320,6 +1338,28 @@ static inline pgoff_t vma_end_pgoff(const struct vm_area_struct *vma)\n \treturn vma_start_pgoff(vma) + vma_pages(vma);\n }\n \n+static inline pgoff_t vma_start_virt_pgoff(const struct vm_area_struct *vma)\n+{\n+\tpgoff_t pgoff = 0;\n+\n+#ifdef CONFIG_64BIT\n+\tpgoff += vma-\u003e__vm_virt_pgoff_hi;\n+\tpgoff \u003c\u003c= 32;\n+#endif\n+\tpgoff += vma-\u003e__vm_virt_pgoff_lo;\n+\treturn pgoff;\n+}\n+\n+static inline pgoff_t vma_end_virt_pgoff(const struct vm_area_struct *vma)\n+{\n+\treturn vma_start_virt_pgoff(vma) + vma_pages(vma);\n+}\n+\n+static inline pgoff_t vma_last_virt_pgoff(const struct vm_area_struct *vma)\n+{\n+\treturn vma_end_virt_pgoff(vma) - 1;\n+}\n+\n static inline int vfs_mmap_prepare(struct file *file, struct vm_area_desc *desc)\n {\n \treturn file-\u003ef_op-\u003emmap_prepare(desc);\n@@ -1584,3 +1624,40 @@ static inline pgprot_t vma_get_page_prot(const struct vm_area_struct *vma)\n {\n \treturn vma_flags_to_page_prot(vma-\u003eflags);\n }\n+\n+static inline pgoff_t __linear_virt_page_index(const struct vm_area_struct *vma,\n+\t\t\t\t\t const unsigned long address)\n+{\n+\tpgoff_t pgoff;\n+\n+\tpgoff = linear_page_delta(vma, address);\n+\tpgoff += vma_start_virt_pgoff(vma);\n+\treturn pgoff;\n+}\n+\n+static inline pgoff_t linear_virt_page_index(const struct vm_area_struct *vma,\n+\t\t\t\t\t const unsigned long address)\n+{\n+\tconst pgoff_t pgoff = __linear_virt_page_index(vma, address);\n+\n+\tVM_WARN_ON_ONCE(vma_test(vma, VMA_SHARED_BIT));\n+\tif (!vma-\u003evm_file) /* Is anonymous except MAP_PRIVATE-/dev/zero */\n+\t\tVM_WARN_ON_ONCE(pgoff != linear_page_index(vma, address));\n+\n+\treturn pgoff;\n+}\n+\n+static inline struct inode *file_inode(const struct file *f)\n+{\n+\treturn f-\u003ef_inode;\n+}\n+\n+static inline unsigned iminor(const struct inode *inode)\n+{\n+\treturn MINOR(inode-\u003ei_rdev);\n+}\n+\n+static inline unsigned imajor(const struct inode *inode)\n+{\n+\treturn MAJOR(inode-\u003ei_rdev);\n+}\ndiff --git a/tools/testing/vma/shared.c b/tools/testing/vma/shared.c\nindex bea9ea6db02a4c..f410bb6f858e11 100644\n--- a/tools/testing/vma/shared.c\n+++ b/tools/testing/vma/shared.c\n@@ -23,7 +23,8 @@ struct vm_area_struct *alloc_vma(struct mm_struct *mm,\n \n \tvma-\u003evm_start = start;\n \tvma-\u003evm_end = end;\n-\tvma-\u003evm_pgoff = pgoff;\n+\tvma_set_pgoff(vma, pgoff);\n+\tvma_set_virt_pgoff(vma, start \u003e\u003e PAGE_SHIFT);\n \tvma-\u003eflags = vma_flags;\n \tvma_assert_detached(vma);\n \ndiff --git a/tools/testing/vma/tests/merge.c b/tools/testing/vma/tests/merge.c\nindex e357accc849905..a01107ab8e2134 100644\n--- a/tools/testing/vma/tests/merge.c\n+++ b/tools/testing/vma/tests/merge.c\n@@ -45,6 +45,7 @@ void vmg_set_range(struct vma_merge_struct *vmg, unsigned long start,\n \tvmg-\u003estart = start;\n \tvmg-\u003eend = end;\n \tvmg-\u003epgoff = pgoff;\n+\tvmg-\u003eanon_pgoff = start \u003e\u003e PAGE_SHIFT;\n \tvmg-\u003evma_flags = vma_flags;\n \n \tvmg-\u003ejust_expand = false;\n@@ -108,6 +109,7 @@ static bool test_simple_merge(void)\n \t\t.end = 0x2000,\n \t\t.vma_flags = vma_flags,\n \t\t.pgoff = 1,\n+\t\t.anon_pgoff = 1,\n \t};\n \n \tASSERT_FALSE(attach_vma(\u0026mm, vma_left));\n@@ -119,6 +121,7 @@ static bool test_simple_merge(void)\n \tASSERT_EQ(vma-\u003evm_start, 0);\n \tASSERT_EQ(vma-\u003evm_end, 0x3000);\n \tASSERT_EQ(vma_start_pgoff(vma), 0);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 0);\n \tASSERT_FLAGS_SAME_MASK(\u0026vma-\u003eflags, vma_flags);\n \n \tdetach_free_vma(vma);\n@@ -151,6 +154,7 @@ static bool test_simple_modify(void)\n \tASSERT_EQ(vma-\u003evm_start, 0x1000);\n \tASSERT_EQ(vma-\u003evm_end, 0x2000);\n \tASSERT_EQ(vma_start_pgoff(vma), 1);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 1);\n \n \t/*\n \t * Now walk through the three split VMAs and make sure they are as\n@@ -163,6 +167,7 @@ static bool test_simple_modify(void)\n \tASSERT_EQ(vma-\u003evm_start, 0);\n \tASSERT_EQ(vma-\u003evm_end, 0x1000);\n \tASSERT_EQ(vma_start_pgoff(vma), 0);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 0);\n \n \tdetach_free_vma(vma);\n \tvma_iter_clear(\u0026vmi);\n@@ -172,6 +177,7 @@ static bool test_simple_modify(void)\n \tASSERT_EQ(vma-\u003evm_start, 0x1000);\n \tASSERT_EQ(vma-\u003evm_end, 0x2000);\n \tASSERT_EQ(vma_start_pgoff(vma), 1);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 1);\n \n \tdetach_free_vma(vma);\n \tvma_iter_clear(\u0026vmi);\n@@ -181,6 +187,7 @@ static bool test_simple_modify(void)\n \tASSERT_EQ(vma-\u003evm_start, 0x2000);\n \tASSERT_EQ(vma-\u003evm_end, 0x3000);\n \tASSERT_EQ(vma_start_pgoff(vma), 2);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 2);\n \n \tdetach_free_vma(vma);\n \tmtree_destroy(\u0026mm.mm_mt);\n@@ -210,6 +217,7 @@ static bool test_simple_expand(void)\n \tASSERT_EQ(vma-\u003evm_start, 0);\n \tASSERT_EQ(vma-\u003evm_end, 0x3000);\n \tASSERT_EQ(vma_start_pgoff(vma), 0);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 0);\n \n \tdetach_free_vma(vma);\n \tmtree_destroy(\u0026mm.mm_mt);\n@@ -232,6 +240,7 @@ static bool test_simple_shrink(void)\n \tASSERT_EQ(vma-\u003evm_start, 0);\n \tASSERT_EQ(vma-\u003evm_end, 0x1000);\n \tASSERT_EQ(vma_start_pgoff(vma), 0);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 0);\n \n \tdetach_free_vma(vma);\n \tmtree_destroy(\u0026mm.mm_mt);\n@@ -344,6 +353,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,\n \tASSERT_EQ(vma-\u003evm_start, 0);\n \tASSERT_EQ(vma-\u003evm_end, 0x5000);\n \tASSERT_EQ(vma_start_pgoff(vma), 0);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 0);\n \tASSERT_EQ(vma-\u003eanon_vma, \u0026dummy_anon_vma);\n \tASSERT_TRUE(vma_write_started(vma));\n \tASSERT_EQ(mm.map_count, 3);\n@@ -365,6 +375,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,\n \tASSERT_EQ(vma-\u003evm_start, 0x6000);\n \tASSERT_EQ(vma-\u003evm_end, 0x9000);\n \tASSERT_EQ(vma_start_pgoff(vma), 6);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 6);\n \tASSERT_EQ(vma-\u003eanon_vma, \u0026dummy_anon_vma);\n \tASSERT_TRUE(vma_write_started(vma));\n \tASSERT_EQ(mm.map_count, 3);\n@@ -385,6 +396,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,\n \tASSERT_EQ(vma-\u003evm_start, 0);\n \tASSERT_EQ(vma-\u003evm_end, 0x9000);\n \tASSERT_EQ(vma_start_pgoff(vma), 0);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 0);\n \tASSERT_EQ(vma-\u003eanon_vma, \u0026dummy_anon_vma);\n \tASSERT_TRUE(vma_write_started(vma));\n \tASSERT_EQ(mm.map_count, 2);\n@@ -405,6 +417,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,\n \tASSERT_EQ(vma-\u003evm_start, 0xa000);\n \tASSERT_EQ(vma-\u003evm_end, 0xc000);\n \tASSERT_EQ(vma_start_pgoff(vma), 0xa);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 0xa);\n \tASSERT_EQ(vma-\u003eanon_vma, \u0026dummy_anon_vma);\n \tASSERT_TRUE(vma_write_started(vma));\n \tASSERT_EQ(mm.map_count, 2);\n@@ -424,6 +437,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,\n \tASSERT_EQ(vma-\u003evm_start, 0);\n \tASSERT_EQ(vma-\u003evm_end, 0xc000);\n \tASSERT_EQ(vma_start_pgoff(vma), 0);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 0);\n \tASSERT_EQ(vma-\u003eanon_vma, \u0026dummy_anon_vma);\n \tASSERT_TRUE(vma_write_started(vma));\n \tASSERT_EQ(mm.map_count, 1);\n@@ -444,6 +458,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,\n \t\tASSERT_EQ(vma-\u003evm_start, 0);\n \t\tASSERT_EQ(vma-\u003evm_end, 0xc000);\n \t\tASSERT_EQ(vma_start_pgoff(vma), 0);\n+\t\tASSERT_EQ(vma_start_virt_pgoff(vma), 0);\n \t\tASSERT_EQ(vma-\u003eanon_vma, \u0026dummy_anon_vma);\n \n \t\tdetach_free_vma(vma);\n@@ -806,6 +821,7 @@ static bool test_vma_merge_new_with_close(void)\n \tASSERT_EQ(vma-\u003evm_start, 0);\n \tASSERT_EQ(vma-\u003evm_end, 0x5000);\n \tASSERT_EQ(vma_start_pgoff(vma), 0);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 0);\n \tASSERT_EQ(vma-\u003evm_ops, \u0026vm_ops);\n \tASSERT_TRUE(vma_write_started(vma));\n \tASSERT_EQ(mm.map_count, 2);\n@@ -866,6 +882,7 @@ static bool __test_merge_existing(bool prev_is_sticky, bool middle_is_sticky, bo\n \tASSERT_EQ(vma-\u003evm_start, 0x2000);\n \tASSERT_EQ(vma-\u003evm_end, 0x3000);\n \tASSERT_EQ(vma_start_pgoff(vma), 2);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 2);\n \tASSERT_TRUE(vma_write_started(vma));\n \tASSERT_TRUE(vma_write_started(vma_next));\n \tASSERT_EQ(mm.map_count, 2);\n@@ -932,6 +949,7 @@ static bool __test_merge_existing(bool prev_is_sticky, bool middle_is_sticky, bo\n \tASSERT_EQ(vma-\u003evm_start, 0x6000);\n \tASSERT_EQ(vma-\u003evm_end, 0x7000);\n \tASSERT_EQ(vma_start_pgoff(vma), 6);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 6);\n \tASSERT_TRUE(vma_write_started(vma_prev));\n \tASSERT_TRUE(vma_write_started(vma));\n \tASSERT_EQ(mm.map_count, 2);\n@@ -1417,6 +1435,7 @@ static bool test_merge_extend(void)\n \tASSERT_EQ(vma-\u003evm_start, 0);\n \tASSERT_EQ(vma-\u003evm_end, 0x4000);\n \tASSERT_EQ(vma_start_pgoff(vma), 0);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 0);\n \tASSERT_TRUE(vma_write_started(vma));\n \tASSERT_EQ(mm.map_count, 1);\n \n@@ -1431,7 +1450,7 @@ static bool test_expand_only_mode(void)\n \tstruct mm_struct mm = {};\n \tVMA_ITERATOR(vmi, \u0026mm, 0);\n \tstruct vm_area_struct *vma_prev, *vma;\n-\tVMG_STATE(vmg, \u0026mm, \u0026vmi, 0x5000, 0x9000, vma_flags, 5);\n+\tVMG_STATE(vmg, \u0026mm, \u0026vmi, 0x5000, 0x9000, vma_flags, 5, 5);\n \n \t/*\n \t * Place a VMA prior to the one we're expanding so we assert that we do\n@@ -1457,6 +1476,7 @@ static bool test_expand_only_mode(void)\n \tASSERT_EQ(vma-\u003evm_start, 0x3000);\n \tASSERT_EQ(vma-\u003evm_end, 0x9000);\n \tASSERT_EQ(vma_start_pgoff(vma), 3);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 3);\n \tASSERT_TRUE(vma_write_started(vma));\n \tASSERT_EQ(vma_iter_addr(\u0026vmi), 0x3000);\n \tvma_assert_attached(vma);\ndiff --git a/tools/testing/vma/tests/mmap.c b/tools/testing/vma/tests/mmap.c\nindex c85bc000d1cb7a..0c8a3446b90502 100644\n--- a/tools/testing/vma/tests/mmap.c\n+++ b/tools/testing/vma/tests/mmap.c\n@@ -45,7 +45,56 @@ static bool test_mmap_region_basic(void)\n \treturn true;\n }\n \n+static int dummy_mmap_prepare(struct vm_area_desc *desc)\n+{\n+\treturn 0;\n+}\n+\n+static bool test_pure_anon_dev_zero(void)\n+{\n+\tconst vma_flags_t vma_flags = mk_vma_flags(VMA_READ_BIT, VMA_WRITE_BIT,\n+\t\t\tVMA_MAYREAD_BIT, VMA_MAYWRITE_BIT);\n+\tconst struct file_operations f_op = {\n+\t\t.mmap_prepare = dummy_mmap_prepare,\n+\t};\n+\tstruct inode inode = {\n+\t\t.i_rdev = MKDEV(MEM_MAJOR, DEVZERO_MINOR),\n+\t};\n+\tstruct file file = {\n+\t\t.f_inode = \u0026inode,\n+\t\t.f_op = \u0026f_op,\n+\t};\n+\tstruct mm_struct mm = {};\n+\tstruct vm_area_struct *vma;\n+\tunsigned long addr;\n+\tVMA_ITERATOR(vmi, \u0026mm, 0);\n+\n+\tcurrent-\u003emm = \u0026mm;\n+\n+\t/*\n+\t * Map a MAP_PRIVATE-/dev/zero mapping at address 0x300000 with a page\n+\t * offset of 0x10, which we expect to be reset to the virtual page\n+\t * offset.\n+\t */\n+\taddr = __mmap_region(\u0026file, 0x300000, 0x3000, vma_flags, 0x10, NULL);\n+\tASSERT_EQ(addr, 0x300000);\n+\n+\t/* Assert that it truly is an anonymous mapping. */\n+\tvma = vma_lookup(\u0026mm, addr);\n+\tASSERT_NE(vma, NULL);\n+\tASSERT_TRUE(vma_is_anonymous(vma));\n+\tASSERT_EQ(vma-\u003evm_file, NULL);\n+\tASSERT_EQ(vma-\u003evm_private_data, NULL);\n+\t/* Expect virtual page offsets. */\n+\tASSERT_EQ(vma-\u003evm_pgoff, 0x300);\n+\tASSERT_EQ(vma_start_virt_pgoff(vma), 0x300);\n+\n+\tcleanup_mm(\u0026mm, \u0026vmi);\n+\treturn true;\n+}\n+\n static void run_mmap_tests(int *num_tests, int *num_fail)\n {\n \tTEST(mmap_region_basic);\n+\tTEST(pure_anon_dev_zero);\n }\ndiff --git a/tools/testing/vma/tests/vma.c b/tools/testing/vma/tests/vma.c\nindex 754a2da0632157..7ca5289e0f9593 100644\n--- a/tools/testing/vma/tests/vma.c\n+++ b/tools/testing/vma/tests/vma.c\n@@ -38,7 +38,7 @@ static bool test_copy_vma(void)\n \t/* Move backwards and do not merge. */\n \n \tvma = alloc_and_link_vma(\u0026mm, 0x3000, 0x5000, 3, vma_flags);\n-\tvma_new = copy_vma(\u0026vma, 0, 0x2000, 0, \u0026need_locks);\n+\tvma_new = copy_vma(\u0026vma, 0, 0x2000, 0, 3, \u0026need_locks);\n \tASSERT_NE(vma_new, vma);\n \tASSERT_EQ(vma_new-\u003evm_start, 0);\n \tASSERT_EQ(vma_new-\u003evm_end, 0x2000);\n@@ -51,7 +51,7 @@ static bool test_copy_vma(void)\n \n \tvma = alloc_and_link_vma(\u0026mm, 0, 0x2000, 0, vma_flags);\n \tvma_next = alloc_and_link_vma(\u0026mm, 0x6000, 0x8000, 6, vma_flags);\n-\tvma_new = copy_vma(\u0026vma, 0x4000, 0x2000, 4, \u0026need_locks);\n+\tvma_new = copy_vma(\u0026vma, 0x4000, 0x2000, 4, 4, \u0026need_locks);\n \tvma_assert_attached(vma_new);\n \n \tASSERT_EQ(vma_new, vma_next);\ndiff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_internal.h\nindex 4f6c5666ac07d3..8a48b231aa7abf 100644\n--- a/tools/testing/vma/vma_internal.h\n+++ b/tools/testing/vma/vma_internal.h\n@@ -53,6 +53,7 @@ typedef __bitwise unsigned int vm_fault_t;\n \n #define VM_WARN_ON(_expr) (WARN_ON(_expr))\n #define VM_WARN_ON_ONCE(_expr) (WARN_ON_ONCE(_expr))\n+#define VM_WARN_ON_ONCE_VMA(_expr, _vma) (WARN_ON_ONCE(_expr))\n #define VM_WARN_ON_VMG(_expr, _vmg) (WARN_ON(_expr))\n #define VM_BUG_ON(_expr) (BUG_ON(_expr))\n #define VM_BUG_ON_VMA(_expr, _vma) (BUG_ON(_expr))\n"
} |
| 2/1 | 2026/07/17 22:47 | agent | patch-evaluator |
1mModel:gemini-3.1-pro-preview Args: null Results: {
"EnableConfigs": [
"KSM",
"USERFAULTFD",
"MEMORY_FAILURE",
"TRANSPARENT_HUGEPAGE",
"MIGRATION",
"FS_DAX"
],
"FocusSymbols": [
"mmap_zero_prepare",
"try_to_merge_with_ksm_page",
"rmap_walk_ksm",
"remove_migration_pte",
"copy_vma_and_data",
"move_present_ptes",
"relocate_vma_down"
],
"Reasoning": "The patch modifies the core memory management subsystem to handle MAP_PRIVATE mappings of /dev/zero as truly anonymous mappings, rather than file-backed mappings with a special flag. It introduces virtual page offsets for VMAs to correctly track and merge these mappings. This affects various mm subsystems including KSM, migration, mremap, userfaultfd, and memory failure handling. The changes are functional and reachable from userspace via mmap, mremap, userfaultfd, etc., so they should be fuzzed.",
"WorthFuzzing": true
}Instruction: You are an expert Linux kernel maintainer. Your job is to review a provided patch series and determine if it makes functional changes to the kernel that should be fuzzed. IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in your workspace. Do NOT rely on your internal knowledge of the kernel. You must actively use your code access tools to examine the actual source code and confirm any assumptions. Return WorthFuzzing=false if the patch only contains: - Modifications to Documentation/, Kconfig files, or code comments. - Purely decorative changes, such as logging (e.g., pr_err, printk) or tracepoints. - Changes to numeric constants or macros that do not functionally alter execution flow. - Code paths that are impossible to reach in virtualized environments like GCE or QEMU, even when utilizing software-emulated hardware (e.g., usb gadget, mac80211_hwsim). If it modifies reachable core kernel logic, drivers, or architectures, use your code search tools to verify the code can be executed, then return WorthFuzzing=true. When returning WorthFuzzing=true, you MUST ALSO: 1. Extract any specific kernel functions that should be heavily fuzzed into FocusSymbols. Avoid listing generic hot-path functions to prevent skewed test distributions. 2. Identify any specific CONFIG_ options required to properly test this new/modified feature. Go and look into the Kconfig files and check for ifdefs around the code, do not make assumptions. Do not list too generic configs (we already have them enabled). Only list those that specifically cover the modified code. List them in the EnableConfigs output array, and DO NOT add a 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4"). Prefer calling several tools at the same time to save round-trips. Use set-results tool to provide results of the analysis. It must be called exactly once before the final reply. Ignore results of this tool. Prompt: For your convenience, here is the diff of the changes:
commit 6133f60c76a07fbed1d030d1492bac8c961453f5
Author: syz-cluster <triage@syzkaller.com>
Date: Fri Jul 17 22:47:34 2026 +0000
syz-cluster: applied patch under review
diff --git a/drivers/char/mem.c b/drivers/char/mem.c
index 63253d1de5d70b..dcfd896b733d8f 100644
--- a/drivers/char/mem.c
+++ b/drivers/char/mem.c
@@ -506,11 +506,7 @@ static int mmap_zero_prepare(struct vm_area_desc *desc)
if (vma_desc_test(desc, VMA_SHARED_BIT))
return shmem_zero_setup_desc(desc);
- /*
- * This is a highly unique situation where we mark a MAP_PRIVATE mapping
- * of /dev/zero anonymous, despite it not being.
- */
- vma_desc_set_anonymous(desc);
+ /* MAP_PRIVATE semantics are taken care for us by core mm. */
return 0;
}
@@ -698,7 +694,7 @@ static const struct memdev {
#ifdef CONFIG_DEVPORT
[4] = { "port", &port_fops, 0, 0 },
#endif
- [5] = { "zero", &zero_fops, FMODE_NOWAIT, 0666 },
+ [DEVZERO_MINOR] = { "zero", &zero_fops, FMODE_NOWAIT, 0666 },
[7] = { "full", &full_fops, 0, 0666 },
[8] = { "random", &random_fops, FMODE_NOWAIT, 0666 },
[9] = { "urandom", &urandom_fops, FMODE_NOWAIT, 0666 },
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 87feaa5a2b780e..7fabe6c66b4b73 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -740,6 +740,9 @@ static inline bool fault_flag_allow_retry_first(enum fault_flag flags)
{ FAULT_FLAG_INTERRUPTIBLE, "INTERRUPTIBLE" }, \
{ FAULT_FLAG_VMA_LOCK, "VMA_LOCK" }
+/* /dev/zero minor device number. Special due to MAP_PRIVATE semantics. */
+#define DEVZERO_MINOR 5
+
/*
* vm_fault is filled by the pagefault handler and passed to the vma's
* ->fault function. The vma's ->fault is responsible for returning a bitmask
@@ -1551,12 +1554,7 @@ static inline void vma_set_anonymous(struct vm_area_struct *vma)
vma->vm_ops = NULL;
}
-static inline void vma_desc_set_anonymous(struct vm_area_desc *desc)
-{
- desc->vm_ops = NULL;
-}
-
-static inline bool vma_is_anonymous(struct vm_area_struct *vma)
+static inline bool vma_is_anonymous(const struct vm_area_struct *vma)
{
return !vma->vm_ops;
}
@@ -4349,9 +4347,8 @@ static inline unsigned long vma_pages(const struct vm_area_struct *vma)
* If @vma is a MAP_PRIVATE file-backed mapping, then this returns the
* page offset within the file.
*
- * Edge cases: nommu does not abide by these, MAP_PRIVATE-/dev/zero satisfies
- * vma_is_anonymous() but has file-backed page offset, and MAP_PRIVATE-pfnmap
- * regions have their page offset set to the first PFN in the range.
+ * Edge cases: nommu does not abide by these and CoW MAP_PRIVATE-pfnmap regions
+ * have their page offset set to the first PFN in the range.
*
* Returns: The page offset of the start of @vma.
*/
@@ -4393,6 +4390,65 @@ static inline pgoff_t vma_last_pgoff(const struct vm_area_struct *vma)
return vma_end_pgoff(vma) - 1;
}
+/**
+ * vma_start_virt_pgoff() - Get the virtual page offset of the start of @vma
+ * @vma: The VMA whose virtual page offset is required.
+ *
+ * If unfaulted, then this is vma->vm_start >> PAGE_SHIFT, if faulted then the
+ * virtual page offset at the time of first fault.
+ *
+ * If the VMA is anonymous, this returns the same value as vma_start_pgoff().
+ *
+ * This value is used for tracking MAP_PRIVATE file-backed mappings by their
+ * virtual page offset.
+ *
+ * Returns: The virtual page offset of the start of @vma.
+ */
+static inline pgoff_t vma_start_virt_pgoff(const struct vm_area_struct *vma)
+{
+ pgoff_t pgoff = 0;
+
+#ifdef CONFIG_64BIT
+ pgoff += vma->__vm_virt_pgoff_hi;
+ pgoff <<= 32;
+#endif
+ pgoff += vma->__vm_virt_pgoff_lo;
+ return pgoff;
+}
+
+/**
+ * vma_end_virt_pgoff() - Get the virtual page offset of the exclusive end of
+ * @vma.
+ * @vma: The VMA whose end virtual page offset is required.
+ *
+ * This returns the virtual exclusive end page offset of @vma, which is useful
+ * for expressing page offset ranges.
+ *
+ * See the description of vma_start_virt_pgoff() for a description of VMA
+ * virtual page offsets.
+ *
+ * Returns: The exclusive end virtual page offset of @vma.
+ */
+static inline pgoff_t vma_end_virt_pgoff(const struct vm_area_struct *vma)
+{
+ return vma_start_virt_pgoff(vma) + vma_pages(vma);
+}
+
+/**
+ * vma_last_virt_pgoff() - Get the virtual page offset of the last page in
+ * @vma.
+ * @vma: The VMA whose last virtual page offset is required.
+ *
+ * See the description of vma_start_virt_pgoff() for a description of VMA
+ * virtual page offsets.
+ *
+ * Returns: The last virtual page offset of @vma.
+ */
+static inline pgoff_t vma_last_virt_pgoff(const struct vm_area_struct *vma)
+{
+ return vma_end_virt_pgoff(vma) - 1;
+}
+
static inline unsigned long vma_desc_size(const struct vm_area_desc *desc)
{
return desc->end - desc->start;
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 939b5ea8c9e0be..2710628059b15b 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -967,6 +967,7 @@ struct vm_area_struct {
*/
unsigned int vm_lock_seq;
#endif
+ unsigned int __vm_virt_pgoff_lo; /* Low 32-bits of virtual pgoff. */
/*
* A file's MAP_PRIVATE vma can be in both i_mmap tree and anon_vma
* list, after a COW of one of the file pages. A MAP_SHARED vma
@@ -1041,6 +1042,9 @@ struct vm_area_struct {
#ifdef CONFIG_DEBUG_LOCK_ALLOC
struct lockdep_map vmlock_dep_map;
#endif
+#endif
+#ifdef CONFIG_64BIT
+ unsigned int __vm_virt_pgoff_hi; /* High 32-bits of virtual pgoff. */
#endif
/*
* For areas with an address space and backing store,
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index c6fc783aaee576..90130f28e7a87d 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -1101,6 +1101,72 @@ static inline pgoff_t linear_page_index(const struct vm_area_struct *vma,
return pgoff;
}
+static inline pgoff_t __linear_virt_page_index(const struct vm_area_struct *vma,
+ const unsigned long address)
+{
+ pgoff_t pgoff;
+
+ pgoff = linear_page_delta(vma, address);
+ pgoff += vma_start_virt_pgoff(vma);
+ return pgoff;
+}
+
+/**
+ * linear_virt_page_index() - Determine the absolute virtual page offset of
+ * @address within @vma.
+ * @vma: An anonymous or MAP_PRIVATE file-backed VMA in which @address resides.
+ * @address: The address whose absolute page offset is required.
+ *
+ * This returns the virtual page offset of @address, which is the page offset
+ * the address possessed at the time the VMA was first faulted.
+ *
+ * For anonymous mappings, this returns the same value as linear_page_index().
+ *
+ * For MAP_PRIVATE file-backed mappings, this returns the virtual page offset of
+ * @address, which is the page offset the address possessed at the time the VMA
+ * was first faulted.
+ *
+ * It is not valid to call this function for shared file-backed mappings.
+ *
+ * Returns: The absolute virtual page offset of @address within @vma.
+ */
+static inline pgoff_t linear_virt_page_index(const struct vm_area_struct *vma,
+ const unsigned long address)
+{
+ const pgoff_t pgoff = __linear_virt_page_index(vma, address);
+
+ VM_WARN_ON_ONCE(vma_test(vma, VMA_SHARED_BIT));
+ if (!vma->vm_file) /* Is anonymous except MAP_PRIVATE-/dev/zero */
+ VM_WARN_ON_ONCE(pgoff != linear_page_index(vma, address));
+
+ return pgoff;
+}
+
+/**
+ * linear_folio_page_index() - Determine the absolute page offset of
+ * @address within @vma from @folio.
+ * @folio: The folio whose linear page index is sought.
+ * @vma: The VMA in which @address resides.
+ * @address: The address whose absolute page offset is required.
+ *
+ * Determines whether to obtain the virtual linear page index based on whether
+ * @folio is anonymous or not.
+ *
+ * See the descriptions of linear_virt_page_index() and linear_page_index() for
+ * details of each.
+ *
+ * Returns: The absolute page offset of @address within @vma.
+ */
+static inline pgoff_t linear_folio_page_index(const struct folio *folio,
+ const struct vm_area_struct *vma,
+ const unsigned long address)
+{
+ if (folio_test_anon(folio))
+ return linear_virt_page_index(vma, address);
+
+ return linear_page_index(vma, address);
+}
+
struct wait_page_key {
struct folio *folio;
int bit_nr;
diff --git a/include/linux/rmap.h b/include/linux/rmap.h
index 8dc0871e5f0011..a48ae9575bd262 100644
--- a/include/linux/rmap.h
+++ b/include/linux/rmap.h
@@ -871,6 +871,7 @@ struct page_vma_mapped_walk {
pte_t *pte;
spinlock_t *ptl;
unsigned int flags;
+ bool is_anon_walk;
};
#define DEFINE_FOLIO_VMA_WALK(name, _folio, _vma, _address, _flags) \
@@ -881,6 +882,7 @@ struct page_vma_mapped_walk {
.vma = _vma, \
.address = _address, \
.flags = _flags, \
+ .is_anon_walk = folio_test_anon(_folio), \
}
static inline void page_vma_mapped_walk_done(struct page_vma_mapped_walk *pvmw)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 9b1f3b24f7e0d0..abc65d608c2392 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2887,7 +2887,8 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, pm
}
folio_move_anon_rmap(src_folio, dst_vma);
- src_folio->index = linear_page_index(dst_vma, dst_addr);
+ src_folio->index = linear_folio_page_index(src_folio, dst_vma,
+ dst_addr);
_dst_pmd = folio_mk_pmd(src_folio, dst_vma->vm_page_prot);
/* Follow mremap() behavior and treat the entry dirty after the move */
diff --git a/mm/internal.h b/mm/internal.h
index f26423de4ca287..41561fdeb56d0a 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -240,6 +240,10 @@ static inline int mmap_file(struct file *file, struct vm_area_struct *vma)
{
int err = vfs_mmap(file, vma);
+ /* Hooks cannot mark themselves anonymous. */
+ if (WARN_ON_ONCE(vma_is_anonymous(vma)))
+ err = -EINVAL;
+
if (likely(!err))
return 0;
@@ -933,7 +937,8 @@ folio_within_range(struct folio *folio, struct vm_area_struct *vma,
return false;
pgoff_folio = folio_pgoff(folio);
- pgoff_vma_start = vma_start_pgoff(vma);
+ pgoff_vma_start = folio_test_anon(folio) ?
+ vma_start_virt_pgoff(vma) : vma_start_pgoff(vma);
if (start < vma->vm_start)
start = vma->vm_start;
@@ -1005,19 +1010,9 @@ void mlock_drain_remote(int cpu);
extern pmd_t maybe_pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma);
-/**
- * vma_address - Find the virtual address a page range is mapped at
- * @vma: The vma which maps this object.
- * @pgoff: The page offset within its object.
- * @nr_pages: The number of pages to consider.
- *
- * If any page in this range is mapped by this VMA, return the first address
- * where any of these pages appear. Otherwise, return -EFAULT.
- */
-static inline unsigned long vma_address(const struct vm_area_struct *vma,
- pgoff_t pgoff, unsigned long nr_pages)
+static inline unsigned long __vma_address(const struct vm_area_struct *vma,
+ pgoff_t pgoff, pgoff_t pgoff_start, unsigned long nr_pages)
{
- const pgoff_t pgoff_start = vma_start_pgoff(vma);
unsigned long address;
if (pgoff >= pgoff_start) {
@@ -1035,23 +1030,68 @@ static inline unsigned long vma_address(const struct vm_area_struct *vma,
return address;
}
+/**
+ * vma_filebacked_address - Find the virtual address a file-backed page range is
+ * mapped at.
+ * @vma: The vma which maps this object.
+ * @pgoff: The page offset within its object.
+ * @nr_pages: The number of pages to consider.
+ *
+ * Returns: If any page in this range is mapped by this VMA, return the first
+ * address where any of these pages appear. Otherwise, return -EFAULT.
+ */
+static inline unsigned long vma_filebacked_address(const struct vm_area_struct *vma,
+ pgoff_t pgoff, unsigned long nr_pages)
+{
+ VM_WARN_ON_ONCE(vma_is_anonymous(vma));
+
+ return __vma_address(vma, pgoff, vma_start_pgoff(vma), nr_pages);
+}
+
+/**
+ * vma_anon_address - Find the virtual address an anonymous page range is mapped
+ * at.
+ * @vma: The vma which maps this object.
+ * @pgoff_virt: The virtual page index belonging to the folio.
+ * @nr_pages: The number of pages to consider.
+ *
+ * This is only valid for anonymous or MAP_PRIVATE-mapped file-backed VMAs.
+ *
+ * Returns: If any page in this range is mapped by this VMA, return the first address
+ * where any of these pages appear. Otherwise, return -EFAULT.
+ */
+static inline unsigned long vma_anon_address(const struct vm_area_struct *vma,
+ pgoff_t pgoff_virt, unsigned long nr_pages)
+{
+ VM_WARN_ON_ONCE(!vma_is_anonymous(vma) && vma_test(vma, VMA_SHARED_BIT));
+
+ return __vma_address(vma, pgoff_virt, vma_start_virt_pgoff(vma), nr_pages);
+}
+
/*
- * Then at what user virtual address will none of the range be found in vma?
+ * At what user virtual address will none of the range be found in vma?
* Assumes that vma_address() already returned a good starting address.
*/
static inline unsigned long vma_address_end(struct page_vma_mapped_walk *pvmw)
{
- struct vm_area_struct *vma = pvmw->vma;
- pgoff_t pgoff;
+ const struct vm_area_struct *vma = pvmw->vma;
+ const pgoff_t pgoff = pvmw->pgoff;
+ pgoff_t pgoff_vma_start;
unsigned long address;
+ pgoff_t pgoff_end;
/* Common case, plus ->pgoff is invalid for KSM */
if (pvmw->nr_pages == 1)
return pvmw->address + PAGE_SIZE;
- pgoff = pvmw->pgoff + pvmw->nr_pages;
+ if (pvmw->is_anon_walk)
+ pgoff_vma_start = vma_start_virt_pgoff(vma);
+ else
+ pgoff_vma_start = vma_start_pgoff(vma);
+
+ pgoff_end = pgoff + pvmw->nr_pages;
address = vma->vm_start +
- ((pgoff - vma_start_pgoff(vma)) << PAGE_SHIFT);
+ ((pgoff_end - pgoff_vma_start) << PAGE_SHIFT);
/* Check for address beyond vma (or wrapped through 0?) */
if (address < vma->vm_start || address > vma->vm_end)
address = vma->vm_end;
diff --git a/mm/interval_tree.c b/mm/interval_tree.c
index 3ae9e106d3afd8..26b8437e3b1b13 100644
--- a/mm/interval_tree.c
+++ b/mm/interval_tree.c
@@ -83,12 +83,12 @@ mapping_rmap_tree_iter_next(struct vm_area_struct *vma,
static pgoff_t avc_start_pgoff(struct anon_vma_chain *avc)
{
- return vma_start_pgoff(avc->vma);
+ return vma_start_virt_pgoff(avc->vma);
}
static pgoff_t avc_last_pgoff(struct anon_vma_chain *avc)
{
- return vma_last_pgoff(avc->vma);
+ return vma_last_virt_pgoff(avc->vma);
}
INTERVAL_TREE_DEFINE(struct anon_vma_chain, rb, pgoff_t, rb_subtree_last,
diff --git a/mm/ksm.c b/mm/ksm.c
index 47006f494fcb1b..b9b32f7bfb691e 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -1625,7 +1625,8 @@ static int try_to_merge_with_ksm_page(struct ksm_rmap_item *rmap_item,
* stable_tree, break_cow() will clean it up.
*/
rmap_item->anon_vma = vma->anon_vma;
- rmap_item->linear_page_index = linear_page_index(vma, rmap_item->address);
+ /* The VMA is always anon/MAP_PRIVATE-file backed so use anon index. */
+ rmap_item->linear_page_index = linear_virt_page_index(vma, rmap_item->address);
get_anon_vma(vma->anon_vma);
out:
mmap_read_unlock(mm);
@@ -3152,7 +3153,7 @@ struct folio *ksm_might_need_to_copy(struct folio *folio,
return folio; /* no need to copy it */
} else if (!anon_vma) {
return folio; /* no need to copy it */
- } else if (folio->index == linear_page_index(vma, addr) &&
+ } else if (folio->index == linear_virt_page_index(vma, addr) &&
anon_vma->root == vma->anon_vma->root) {
return folio; /* still no need to copy it */
}
@@ -3222,7 +3223,7 @@ void rmap_walk_ksm(struct folio *folio, struct rmap_walk_control *rwc)
/*
* Currently, KSM folios are always small folios, so it's
* sufficient to search for a single page. We can simply use
- * the linear_page_index of the original de-duplicate
+ * the linear_virt_page_index of the original de-duplicate
* anonymous page that we remembered in the rmap_item while
* de-duplicating. Note that mremap() always de-duplicates KSM
* folios: so if there was mremap() in our parent or our child,
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index aaf14608b30e2d..a8b03e2920ba8a 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -620,7 +620,7 @@ static void add_to_kill_fsdax(struct task_struct *tsk, const struct page *p,
struct vm_area_struct *vma,
struct list_head *to_kill, pgoff_t pgoff)
{
- unsigned long addr = vma_address(vma, pgoff, 1);
+ unsigned long addr = vma_filebacked_address(vma, pgoff, 1);
__add_to_kill(tsk, p, vma, to_kill, addr);
}
@@ -2265,7 +2265,7 @@ static void add_to_kill_pgoff(struct task_struct *tsk,
}
/* Check for pgoff not backed by struct page */
- tk->addr = vma_address(vma, pgoff, 1);
+ tk->addr = vma_filebacked_address(vma, pgoff, 1);
tk->size_shift = PAGE_SHIFT;
if (tk->addr == -EFAULT)
diff --git a/mm/memory.c b/mm/memory.c
index d5e87624f69205..56b244552f1394 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -631,13 +631,14 @@ static void print_bad_page_map(struct vm_area_struct *vma,
{
struct address_space *mapping;
char entry_str[PTVAL_STR_MAX];
- pgoff_t index;
+ pgoff_t index, virt_index;
if (is_bad_page_map_ratelimited())
return;
mapping = vma->vm_file ? vma->vm_file->f_mapping : NULL;
index = linear_page_index(vma, addr);
+ virt_index = __linear_virt_page_index(vma, addr);
ptval_bytes_to_hex_str(entry_str, sizeof(entry_str), entry, entry_size);
pr_alert("BUG: Bad page map in process %s %s:%s", current->comm,
@@ -645,8 +646,9 @@ static void print_bad_page_map(struct vm_area_struct *vma,
__print_bad_page_map_pgtable(vma->vm_mm, addr);
if (page)
dump_page(page, "bad page map");
- pr_alert("addr:%px vm_flags:%08lx anon_vma:%px mapping:%px index:%lx\n",
- (void *)addr, vma->vm_flags, vma->anon_vma, mapping, index);
+ pr_alert("addr:%px vm_flags:%08lx anon_vma:%px mapping:%px index:%lx virt_index:%lx\n",
+ (void *)addr, vma->vm_flags, vma->anon_vma, mapping, index,
+ virt_index);
pr_alert("file:%pD fault:%ps mmap:%ps mmap_prepare: %ps read_folio:%ps\n",
vma->vm_file,
vma->vm_ops ? vma->vm_ops->fault : NULL,
diff --git a/mm/migrate.c b/mm/migrate.c
index 222c8c15f782f7..37fe7a9b3facc8 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -363,8 +363,10 @@ static bool remove_migration_pte(struct folio *folio,
unsigned long idx = 0;
/* pgoff is invalid for ksm pages, but they are never large */
- if (folio_test_large(folio) && !folio_test_hugetlb(folio))
- idx = linear_page_index(vma, pvmw.address) - pvmw.pgoff;
+ if (folio_test_large(folio) && !folio_test_hugetlb(folio)) {
+ idx += linear_folio_page_index(folio, vma, pvmw.address);
+ idx -= pvmw.pgoff;
+ }
new = folio_page(folio, idx);
#ifdef CONFIG_ARCH_HAS_PMD_SOFTLEAVES
diff --git a/mm/mremap.c b/mm/mremap.c
index b64aa1f6e07ede..f07fc4e3ef2e6c 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -1265,7 +1265,9 @@ static void unmap_source_vma(struct vma_remap_struct *vrm)
static int copy_vma_and_data(struct vma_remap_struct *vrm,
struct vm_area_struct **new_vma_ptr)
{
- const unsigned long new_pgoff = linear_page_index(vrm->vma, vrm->addr);
+ const pgoff_t new_pgoff = linear_page_index(vrm->vma, vrm->addr);
+ const pgoff_t new_virt_pgoff =
+ __linear_virt_page_index(vrm->vma, vrm->addr);
struct vm_area_struct *vma = vrm->vma;
struct vm_area_struct *new_vma;
unsigned long moved_len;
@@ -1273,7 +1275,7 @@ static int copy_vma_and_data(struct vma_remap_struct *vrm,
PAGETABLE_MOVE(pmc, NULL, NULL, vrm->addr, vrm->new_addr, vrm->old_len);
new_vma = copy_vma(&vma, vrm->new_addr, vrm->new_len, new_pgoff,
- &pmc.need_rmap_locks);
+ new_virt_pgoff, &pmc.need_rmap_locks);
if (!new_vma) {
vrm_uncharge(vrm);
*new_vma_ptr = NULL;
diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
index d7670ba4147bff..4e964545e5e85a 100644
--- a/mm/page_vma_mapped.c
+++ b/mm/page_vma_mapped.c
@@ -356,6 +356,7 @@ unsigned long page_mapped_in_vma(const struct page *page,
struct vm_area_struct *vma)
{
const struct folio *folio = page_folio(page);
+ const pgoff_t pgoff = page_pgoff(folio, page);
struct page_vma_mapped_walk pvmw = {
.pfn = page_to_pfn(page),
.nr_pages = 1,
@@ -363,7 +364,10 @@ unsigned long page_mapped_in_vma(const struct page *page,
.flags = PVMW_SYNC,
};
- pvmw.address = vma_address(vma, page_pgoff(folio, page), 1);
+ if (folio_test_anon(folio))
+ pvmw.address = vma_anon_address(vma, pgoff, 1);
+ else
+ pvmw.address = vma_filebacked_address(vma, pgoff, 1);
if (pvmw.address == -EFAULT)
goto out;
if (!page_vma_mapped_walk(&pvmw))
diff --git a/mm/rmap.c b/mm/rmap.c
index ad820fe86f7d8c..6854baf4b8778f 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -865,14 +865,15 @@ unsigned long page_address_in_vma(const struct folio *folio,
if (!vma->anon_vma || !anon_vma ||
vma->anon_vma->root != anon_vma->root)
return -EFAULT;
+ /* KSM folios don't reach here because of the !anon_vma check */
+ return vma_anon_address(vma, page_pgoff(folio, page), 1);
} else if (!vma->vm_file) {
return -EFAULT;
} else if (vma->vm_file->f_mapping != folio->mapping) {
return -EFAULT;
}
- /* KSM folios don't reach here because of the !anon_vma check */
- return vma_address(vma, page_pgoff(folio, page), 1);
+ return vma_filebacked_address(vma, page_pgoff(folio, page), 1);
}
/*
@@ -1239,8 +1240,10 @@ static bool mapping_wrprotect_range_one(struct folio *folio,
.vma = vma,
.address = address,
.flags = PVMW_SYNC,
+ .is_anon_walk = false,
};
+ VM_WARN_ON_ONCE(folio_test_anon(folio));
state->cleaned += page_vma_mkclean_one(&pvmw);
return true;
@@ -1316,12 +1319,13 @@ int pfn_mkclean_range(unsigned long pfn, unsigned long nr_pages, pgoff_t pgoff,
.pgoff = pgoff,
.vma = vma,
.flags = PVMW_SYNC,
+ .is_anon_walk = false,
};
if (invalid_mkclean_vma(vma, NULL))
return 0;
- pvmw.address = vma_address(vma, pgoff, nr_pages);
+ pvmw.address = vma_filebacked_address(vma, pgoff, nr_pages);
VM_BUG_ON_VMA(pvmw.address == -EFAULT, vma);
return page_vma_mkclean_one(&pvmw);
@@ -1482,7 +1486,7 @@ static void __folio_set_anon(struct folio *folio, struct vm_area_struct *vma,
*/
anon_vma = (void *) anon_vma + FOLIO_MAPPING_ANON;
WRITE_ONCE(folio->mapping, (struct address_space *) anon_vma);
- folio->index = linear_page_index(vma, address);
+ folio->index = linear_virt_page_index(vma, address);
}
/**
@@ -1509,8 +1513,8 @@ static void __page_check_anon_rmap(const struct folio *folio,
*/
VM_BUG_ON_FOLIO(folio_anon_vma(folio)->root != vma->anon_vma->root,
folio);
- VM_BUG_ON_PAGE(page_pgoff(folio, page) != linear_page_index(vma, address),
- page);
+ VM_BUG_ON_PAGE(page_pgoff(folio, page) !=
+ linear_virt_page_index(vma, address), page);
}
static __always_inline void __folio_add_anon_rmap(struct folio *folio,
@@ -3037,10 +3041,10 @@ static void rmap_walk_anon(struct folio *folio,
pgoff_end = pgoff_start + folio_nr_pages(folio) - 1;
anon_rmap_tree_foreach(avc, anon_vma, pgoff_start, pgoff_end) {
struct vm_area_struct *vma = avc->vma;
- unsigned long address = vma_address(vma, pgoff_start,
+ const unsigned long address = vma_anon_address(vma, pgoff_start,
folio_nr_pages(folio));
- VM_BUG_ON_VMA(address == -EFAULT, vma);
+ VM_WARN_ON_ONCE_VMA(address == -EFAULT, vma);
cond_resched();
if (rwc->invalid_vma && rwc->invalid_vma(vma, rwc->arg))
@@ -3100,7 +3104,8 @@ static void __rmap_walk_file(struct folio *folio, struct address_space *mapping,
}
lookup:
mapping_rmap_tree_foreach(vma, mapping, pgoff_start, pgoff_end) {
- unsigned long address = vma_address(vma, pgoff_start, nr_pages);
+ unsigned long address = vma_filebacked_address(vma, pgoff_start,
+ nr_pages);
VM_BUG_ON_VMA(address == -EFAULT, vma);
cond_resched();
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 8fd24c8b428e1b..258b03182a780b 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -1352,7 +1352,8 @@ static long move_present_ptes(struct mm_struct *mm,
}
folio_move_anon_rmap(src_folio, dst_vma);
- src_folio->index = linear_page_index(dst_vma, dst_addr);
+ src_folio->index = linear_folio_page_index(src_folio, dst_vma,
+ dst_addr);
orig_dst_pte = folio_mk_pte(src_folio, dst_vma->vm_page_prot);
/* Set soft dirty bit so userspace can notice the pte was moved */
@@ -1428,7 +1429,8 @@ static int move_swap_pte(struct mm_struct *mm, struct vm_area_struct *dst_vma,
*/
if (src_folio) {
folio_move_anon_rmap(src_folio, dst_vma);
- src_folio->index = linear_page_index(dst_vma, dst_addr);
+ src_folio->index = linear_folio_page_index(src_folio, dst_vma,
+ dst_addr);
} else {
/*
* Check if the swap entry is cached after acquiring the src_pte
diff --git a/mm/vma.c b/mm/vma.c
index b5bc3eec961c79..f8001d5c23aad1 100644
--- a/mm/vma.c
+++ b/mm/vma.c
@@ -18,6 +18,7 @@ struct mmap_state {
unsigned long addr;
unsigned long end;
pgoff_t pgoff;
+ pgoff_t virt_pgoff;
unsigned long pglen;
union {
vm_flags_t vm_flags;
@@ -46,13 +47,22 @@ struct mmap_state {
bool file_doesnt_need_get :1;
};
-#define MMAP_STATE(name, mm_, vmi_, addr_, len_, pgoff_, vma_flags_, file_) \
+static inline pgoff_t map_anon_pgoff(const struct mmap_state *map)
+{
+ if (vma_flags_test(&map->vma_flags, VMA_SHARED_BIT))
+ return map->pgoff;
+
+ return map->virt_pgoff;
+}
+
+#define MMAP_STATE(name, mm_, vmi_, addr_, len_, pgoff_, virt_pgoff_, vma_flags_, file_) \
struct mmap_state name = { \
.mm = mm_, \
.vmi = vmi_, \
.addr = addr_, \
.end = (addr_) + (len_), \
.pgoff = pgoff_, \
+ .virt_pgoff = virt_pgoff_, \
.pglen = PHYS_PFN(len_), \
.vma_flags = vma_flags_, \
.file = file_, \
@@ -67,6 +77,7 @@ struct mmap_state {
.end = (map_)->end, \
.vma_flags = (map_)->vma_flags, \
.pgoff = (map_)->pgoff, \
+ .anon_pgoff = map_anon_pgoff(map_), \
.file = (map_)->file, \
.prev = (map_)->prev, \
.middle = vma_, \
@@ -82,10 +93,11 @@ static void __vma_set_range(struct vm_area_struct *vma, unsigned long start,
}
static void vma_set_range(struct vm_area_struct *vma, unsigned long start,
- unsigned long end, pgoff_t pgoff)
+ unsigned long end, pgoff_t pgoff, pgoff_t virt_pgoff)
{
__vma_set_range(vma, start, end);
vma_set_pgoff(vma, pgoff);
+ vma_set_virt_pgoff(vma, virt_pgoff);
}
/* Was this VMA ever forked from a parent, i.e. maybe contains CoW mappings? */
@@ -221,6 +233,8 @@ static bool can_vma_merge_before(struct vma_merge_struct *vmg)
return false;
if (vmg_end_pgoff(vmg) != vma_start_pgoff(vmg->next))
return false;
+ if (vmg_end_anon_pgoff(vmg) != vma_start_anon_pgoff(vmg->next))
+ return false;
return true;
}
@@ -241,6 +255,8 @@ static bool can_vma_merge_after(struct vma_merge_struct *vmg)
return false;
if (vma_end_pgoff(vmg->prev) != vmg_start_pgoff(vmg))
return false;
+ if (vma_end_anon_pgoff(vmg->prev) != vmg_start_anon_pgoff(vmg))
+ return false;
return true;
}
@@ -812,7 +828,8 @@ static int commit_merge(struct vma_merge_struct *vmg)
*/
vma_adjust_trans_huge(vma, vmg->start, vmg->end,
vmg->__adjust_middle_start ? vmg->middle : NULL);
- vma_set_range(vma, vmg->start, vmg->end, vmg_start_pgoff(vmg));
+ vma_set_range(vma, vmg->start, vmg->end, vmg_start_pgoff(vmg),
+ vmg_start_anon_pgoff(vmg));
vmg_adjust_set_range(vmg);
vma_iter_store_overwrite(vmg->vmi, vmg->target);
@@ -982,6 +999,7 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(
vmg->start = prev->vm_start;
vmg->end = next->vm_end;
vmg->pgoff = vma_start_pgoff(prev);
+ vmg->anon_pgoff = vma_start_anon_pgoff(prev);
/*
* We already ensured anon_vma compatibility above, so now it's
@@ -1000,6 +1018,7 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(
*/
vmg->start = prev->vm_start;
vmg->pgoff = vma_start_pgoff(prev);
+ vmg->anon_pgoff = vma_start_anon_pgoff(prev);
if (!vmg->__remove_middle)
vmg->__adjust_middle_start = true;
@@ -1022,12 +1041,14 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(
if (vmg->__remove_middle) {
vmg->end = next->vm_end;
vmg->pgoff = vma_start_pgoff(next) - pglen;
+ vmg->anon_pgoff = vma_start_anon_pgoff(next) - pglen;
} else {
/* We shrink middle and expand next. */
vmg->__adjust_next_start = true;
vmg->start = middle->vm_start;
vmg->end = start;
vmg->pgoff = vma_start_pgoff(middle);
+ vmg->anon_pgoff = vma_start_anon_pgoff(middle);
}
err = dup_anon_vma(next, middle, &anon_dup);
@@ -1137,6 +1158,7 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
vmg->start = prev->vm_start;
vmg->target = prev;
vmg->pgoff = vma_start_pgoff(prev);
+ vmg->anon_pgoff = vma_start_anon_pgoff(prev);
/*
* If this merge would result in removal of the next VMA but we
@@ -1908,9 +1930,10 @@ static int vma_link(struct mm_struct *mm, struct vm_area_struct *vma)
*/
struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,
unsigned long addr, unsigned long len, pgoff_t pgoff,
- bool *need_rmap_locks)
+ pgoff_t virt_pgoff, bool *need_rmap_locks)
{
struct vm_area_struct *vma = *vmap;
+ const bool is_shared = vma_test(vma, VMA_SHARED_BIT);
unsigned long vma_start = vma->vm_start;
struct mm_struct *mm = vma->vm_mm;
struct vm_area_struct *new_vma;
@@ -1919,11 +1942,14 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,
VMG_VMA_STATE(vmg, &vmi, NULL, vma, addr, addr + len);
/*
- * If anonymous vma has not yet been faulted, update new pgoff
- * to match new location, to increase its chance of merging.
+ * If a vma has not yet been faulted, update its virtual pgoff to match
+ * the new location to increase its chance of merging.
*/
- if (unlikely(vma_is_anonymous(vma) && !vma->anon_vma)) {
- pgoff = addr >> PAGE_SHIFT;
+ if (!vma->anon_vma && !is_shared) {
+ virt_pgoff = addr >> PAGE_SHIFT;
+
+ if (vma_is_anonymous(vma))
+ pgoff = virt_pgoff;
faulted_in_anon_vma = false;
}
@@ -1940,6 +1966,7 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,
return NULL; /* should never get here */
vmg.pgoff = pgoff;
+ vmg.anon_pgoff = is_shared ? pgoff : virt_pgoff;
vmg.next = vma_iter_next_rewind(&vmi, NULL);
new_vma = vma_merge_copied_range(&vmg);
@@ -1961,16 +1988,17 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,
* safe. It is only safe to keep the vm_pgoff
* linear if there are no pages mapped yet.
*/
- VM_BUG_ON_VMA(faulted_in_anon_vma, new_vma);
+ VM_WARN_ON_ONCE_VMA(faulted_in_anon_vma, new_vma);
*vmap = vma = new_vma;
}
*need_rmap_locks =
- (vma_start_pgoff(new_vma) <= vma_start_pgoff(vma));
+ (vma_start_pgoff(new_vma) <= vma_start_pgoff(vma)) ||
+ (vma_start_anon_pgoff(new_vma) <= vma_start_anon_pgoff(vma));
} else {
new_vma = vm_area_dup(vma);
if (!new_vma)
goto out;
- vma_set_range(new_vma, addr, addr + len, pgoff);
+ vma_set_range(new_vma, addr, addr + len, pgoff, virt_pgoff);
if (vma_dup_policy(vma, new_vma))
goto out_free_vma;
if (anon_vma_clone(new_vma, vma, VMA_OP_REMAP))
@@ -2036,7 +2064,12 @@ static int anon_vma_compatible(struct vm_area_struct *a, struct vm_area_struct *
if (!vma_flags_empty(&diff))
return false;
/* Page offset must align. */
- return vma_end_pgoff(a) == vma_start_pgoff(b);
+ if (vma_end_pgoff(a) != vma_start_pgoff(b))
+ return false;
+ /* Anon page offset must align. */
+ if (vma_end_anon_pgoff(a) != vma_start_anon_pgoff(b))
+ return false;
+ return true;
}
/*
@@ -2579,6 +2612,32 @@ static int __mmap_new_file_vma(struct mmap_state *map,
return 0;
}
+static bool map_is_dev_zero(const struct mmap_state *map)
+{
+ const struct file *file = map->file;
+ const struct inode *inode = file_inode(file);
+
+ return imajor(inode) == MEM_MAJOR && iminor(inode) == DEVZERO_MINOR;
+}
+
+static void map_set_anon(struct mmap_state *map)
+{
+ map->file = NULL;
+ map->file_doesnt_need_get = false;
+ map->pgoff = map->addr >> PAGE_SHIFT;
+ map->vm_ops = NULL;
+}
+
+static bool map_is_private(const struct mmap_state *map)
+{
+ return !vma_flags_test(&map->vma_flags, VMA_SHARED_BIT);
+}
+
+static bool map_is_anon(const struct mmap_state *map)
+{
+ return map_is_private(map) && !map->file;
+}
+
/*
* __mmap_new_vma() - Allocate a new VMA for the region, as merging was not
* possible.
@@ -2592,8 +2651,7 @@ static int __mmap_new_file_vma(struct mmap_state *map,
static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap,
struct mmap_action *action)
{
- const bool is_anon = !map->file &&
- !vma_flags_test(&map->vma_flags, VMA_SHARED_BIT);
+ const bool is_anon = map_is_anon(map);
struct vma_iterator *vmi = map->vmi;
int error = 0;
struct vm_area_struct *vma;
@@ -2612,7 +2670,7 @@ static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap,
if (is_anon)
vma_set_anonymous(vma);
- vma_set_range(vma, map->addr, map->end, map->pgoff);
+ vma_set_range(vma, map->addr, map->end, map->pgoff, map->virt_pgoff);
vma->flags = map->vma_flags;
vma->vm_page_prot = map->page_prot;
@@ -2735,6 +2793,10 @@ static int call_mmap_prepare(struct mmap_state *map,
if (err)
return err;
+ /* Hooks cannot mark themselves anonymous. */
+ if (!desc->vm_ops)
+ return -EINVAL;
+
err = call_action_prepare(map, desc);
if (err)
return err;
@@ -2751,16 +2813,21 @@ static int call_mmap_prepare(struct mmap_state *map,
map->vm_ops = desc->vm_ops;
map->vm_private_data = desc->private_data;
+ /*
+ * MAP_PRIVATE-/dev/zero mappings are an ancient way of getting
+ * anonymous mappings. Rather than allowing these mappings to be odd
+ * outliers, simply make them truly anonymous.
+ */
+ if (map_is_private(map) && map_is_dev_zero(map))
+ map_set_anon(map);
+
return 0;
}
static void set_vma_user_defined_fields(struct vm_area_struct *vma,
struct mmap_state *map)
{
- if (map->vm_ops)
- vma->vm_ops = map->vm_ops;
- else /* Only /dev/zero should do this. */
- vma_set_anonymous(vma);
+ vma->vm_ops = map->vm_ops;
vma->vm_private_data = map->vm_private_data;
}
@@ -2801,7 +2868,8 @@ static unsigned long __mmap_region(struct file *file, unsigned long addr,
struct vm_area_struct *vma = NULL;
bool have_mmap_prepare = file && file->f_op->mmap_prepare;
VMA_ITERATOR(vmi, mm, addr);
- MMAP_STATE(map, mm, &vmi, addr, len, pgoff, vma_flags, file);
+ const pgoff_t virt_pgoff = addr >> PAGE_SHIFT;
+ MMAP_STATE(map, mm, &vmi, addr, len, pgoff, virt_pgoff, vma_flags, file);
struct vm_area_desc desc = {
.mm = mm,
.file = file,
@@ -2946,6 +3014,7 @@ int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *vma,
unsigned long addr, unsigned long len, vma_flags_t vma_flags)
{
struct mm_struct *mm = current->mm;
+ const pgoff_t pgoff = addr >> PAGE_SHIFT;
/*
* Check against address space limits by the changed size
@@ -2970,7 +3039,7 @@ int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *vma,
* occur after forking, so the expand will only happen on new VMAs.
*/
if (vma && vma->vm_end == addr) {
- VMG_STATE(vmg, mm, vmi, addr, addr + len, vma_flags, PHYS_PFN(addr));
+ VMG_STATE(vmg, mm, vmi, addr, addr + len, vma_flags, pgoff, pgoff);
vmg.prev = vma;
/* vmi is positioned at prev, which this mode expects. */
@@ -2990,7 +3059,7 @@ int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *vma,
goto unacct_fail;
vma_set_anonymous(vma);
- vma_set_range(vma, addr, addr + len, addr >> PAGE_SHIFT);
+ vma_set_range(vma, addr, addr + len, pgoff, pgoff);
vma->flags = vma_flags;
vma->vm_page_prot = vm_get_page_prot(vma_flags_to_legacy(vma_flags));
vma_start_write(vma);
@@ -3382,6 +3451,7 @@ int insert_vm_struct(struct mm_struct *mm, struct vm_area_struct *vma)
WARN_ON_ONCE(vma->anon_vma);
vma_set_pgoff(vma, vma->vm_start >> PAGE_SHIFT);
}
+ vma_set_virt_pgoff(vma, vma->vm_start >> PAGE_SHIFT);
if (vma_link(mm, vma)) {
if (vma_test(vma, VMA_ACCOUNT_BIT))
@@ -3437,7 +3507,7 @@ struct vm_area_struct *__install_special_mapping(
vma->vm_ops = ops;
vma->vm_private_data = priv;
- vma_set_range(vma, addr, addr + len, 0);
+ vma_set_range(vma, addr, addr + len, 0, addr >> PAGE_SHIFT);
ret = insert_vm_struct(mm, vma);
if (ret)
diff --git a/mm/vma.h b/mm/vma.h
index 0bc7d521e97677..8caee3b4ba2671 100644
--- a/mm/vma.h
+++ b/mm/vma.h
@@ -104,6 +104,7 @@ struct vma_merge_struct {
unsigned long start;
unsigned long end;
pgoff_t pgoff;
+ pgoff_t anon_pgoff;
union {
/* Temporary while VMA flags are being converted. */
@@ -237,11 +238,6 @@ static inline bool vmg_nomem(struct vma_merge_struct *vmg)
return vmg->state == VMA_MERGE_ERROR_NOMEM;
}
-static inline pgoff_t vmg_start_pgoff(const struct vma_merge_struct *vmg)
-{
- return vmg->pgoff;
-}
-
static inline pgoff_t vmg_pages(const struct vma_merge_struct *vmg)
{
const unsigned long size = vmg->end - vmg->start;
@@ -249,6 +245,11 @@ static inline pgoff_t vmg_pages(const struct vma_merge_struct *vmg)
return size >> PAGE_SHIFT;
}
+static inline pgoff_t vmg_start_pgoff(const struct vma_merge_struct *vmg)
+{
+ return vmg->pgoff;
+}
+
static inline pgoff_t vmg_end_pgoff(const struct vma_merge_struct *vmg)
{
return vmg_start_pgoff(vmg) + vmg_pages(vmg);
@@ -266,9 +267,6 @@ static inline void assert_sane_pgoff(struct vm_area_struct *vma, pgoff_t pgoff)
*/
if (!vma_is_anonymous(vma))
return;
- /* MAP_PRIVATE-/dev/zero is anon, non-NULL vm_file, but has file pgoff. */
- if (vma->vm_file)
- return;
/* If faulted in, could have been remapped. */
if (vma->anon_vma)
return;
@@ -283,48 +281,130 @@ static inline void vma_set_pgoff(struct vm_area_struct *vma, pgoff_t pgoff)
vma->vm_pgoff = pgoff;
}
+static inline pgoff_t vmg_start_anon_pgoff(const struct vma_merge_struct *vmg)
+{
+ return vmg->anon_pgoff;
+}
+
+static inline pgoff_t vmg_end_anon_pgoff(const struct vma_merge_struct *vmg)
+{
+ return vmg_start_anon_pgoff(vmg) + vmg_pages(vmg);
+}
+
+static inline void __vma_set_virt_pgoff(struct vm_area_struct *vma, pgoff_t pgoff)
+{
+#ifdef CONFIG_64BIT
+ vma->__vm_virt_pgoff_hi = pgoff >> 32;
+#endif
+ vma->__vm_virt_pgoff_lo = pgoff & GENMASK(31, 0);
+}
+
+static inline void vma_set_virt_pgoff(struct vm_area_struct *vma, pgoff_t pgoff)
+{
+ vma_assert_can_modify(vma);
+ __vma_set_virt_pgoff(vma, pgoff);
+}
+
static inline void vma_add_pgoff(struct vm_area_struct *vma, pgoff_t delta)
{
vma_assert_can_modify(vma);
vma_set_pgoff(vma, vma_start_pgoff(vma) + delta);
+ vma_set_virt_pgoff(vma, vma_start_virt_pgoff(vma) + delta);
}
static inline void vma_sub_pgoff(struct vm_area_struct *vma, pgoff_t delta)
{
vma_assert_can_modify(vma);
vma_set_pgoff(vma, vma_start_pgoff(vma) - delta);
+ vma_set_virt_pgoff(vma, vma_start_virt_pgoff(vma) - delta);
+}
+
+/**
+ * vma_anon_pgoff_addr() - Calculates the absolute anonymous page offset of
+ * @address.
+ * @vma: The VMA whose anonymous page offset is required.
+ * @address: The address whose absolute page offset is required.
+ *
+ * If the VMA is a shared file-backed mapping, then the file-based page offset
+ * is returned.
+ *
+ * Otherwise, the virtual page offset is returned.
+ *
+ * This means that shared file-backed mappings are correctly merged based on
+ * their file page offset compatibility.
+ *
+ * Returns: The absolute anonymous page offset of @address within @vma.
+ */
+static inline pgoff_t vma_anon_pgoff_addr(const struct vm_area_struct *vma,
+ unsigned long address)
+{
+ if (vma_test(vma, VMA_SHARED_BIT))
+ return linear_page_index(vma, address);
+
+ return linear_virt_page_index(vma, address);
}
-#define VMG_STATE(name, mm_, vmi_, start_, end_, vma_flags_, pgoff_) \
+/**
+ * vma_start_anon_pgoff() - Calculates the absolute anonymous page offset used
+ * for purposes of merge compatibility.
+ * @vma: The VMA whose anonymous page offset is required.
+ *
+ * See vma_anon_pgoff_addr().
+ *
+ * Returns: The absolute anonymous page offset of @vma for purposes of merging.
+ */
+static inline pgoff_t vma_start_anon_pgoff(const struct vm_area_struct *vma)
+{
+ return vma_anon_pgoff_addr(vma, vma->vm_start);
+}
+
+/**
+ * vma_end_anon_pgoff() - Calculates the absolute exclusive end anonymous page
+ * offset used for purposes of merge compatibility.
+ * @vma: The VMA whosse anonymous end page offset is required.
+ *
+ * See vma_start_anon_pgoff().
+ *
+ * Returns: The absolute exclusive end anonymous page offset of @vma for
+ * purposes of merging.
+ */
+static inline pgoff_t vma_end_anon_pgoff(const struct vm_area_struct *vma)
+{
+ return vma_start_anon_pgoff(vma) + vma_pages(vma);
+}
+
+#define VMG_STATE(name, mm_, vmi_, start_, end_, vma_flags_, pgoff_, anon_pgoff_) \
+ struct vma_merge_struct name = { \
+ .mm = mm_, \
+ .vmi = vmi_, \
+ .start = start_, \
+ .end = end_, \
+ .vma_flags = vma_flags_, \
+ .pgoff = pgoff_, \
+ .anon_pgoff = anon_pgoff_, \
+ .state = VMA_MERGE_START, \
+ }
+
+#define VMG_VMA_STATE(name, vmi_, prev_, vma_, start_, end_) \
struct vma_merge_struct name = { \
- .mm = mm_, \
+ .mm = vma_->vm_mm, \
.vmi = vmi_, \
+ .prev = prev_, \
+ .middle = vma_, \
+ .next = NULL, \
.start = start_, \
.end = end_, \
- .vma_flags = vma_flags_, \
- .pgoff = pgoff_, \
+ .vm_flags = vma_->vm_flags, \
+ .pgoff = linear_page_index(vma_, start_), \
+ .anon_pgoff = vma_anon_pgoff_addr(vma_, start_), \
+ .file = vma_->vm_file, \
+ .anon_vma = vma_->anon_vma, \
+ .policy = vma_policy(vma_), \
+ .uffd_ctx = vma_->vm_userfaultfd_ctx, \
+ .anon_name = anon_vma_name(vma_), \
.state = VMA_MERGE_START, \
}
-#define VMG_VMA_STATE(name, vmi_, prev_, vma_, start_, end_) \
- struct vma_merge_struct name = { \
- .mm = vma_->vm_mm, \
- .vmi = vmi_, \
- .prev = prev_, \
- .middle = vma_, \
- .next = NULL, \
- .start = start_, \
- .end = end_, \
- .vm_flags = vma_->vm_flags, \
- .pgoff = linear_page_index(vma_, start_), \
- .file = vma_->vm_file, \
- .anon_vma = vma_->anon_vma, \
- .policy = vma_policy(vma_), \
- .uffd_ctx = vma_->vm_userfaultfd_ctx, \
- .anon_name = anon_vma_name(vma_), \
- .state = VMA_MERGE_START, \
- }
-
#ifdef CONFIG_DEBUG_VM_MAPLE_TREE
void validate_mm(struct mm_struct *mm);
#else
@@ -506,7 +586,7 @@ void unlink_file_vma_batch_add(struct unlink_vma_file_batch *vb,
struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,
unsigned long addr, unsigned long len, pgoff_t pgoff,
- bool *need_rmap_locks);
+ pgoff_t anon_pgoff, bool *need_rmap_locks);
struct anon_vma *find_mergeable_anon_vma(struct vm_area_struct *vma);
diff --git a/mm/vma_exec.c b/mm/vma_exec.c
index 7af1260689b97e..586c5215594245 100644
--- a/mm/vma_exec.c
+++ b/mm/vma_exec.c
@@ -41,7 +41,7 @@ int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift)
unsigned long new_end = old_end - shift;
VMA_ITERATOR(vmi, mm, new_start);
VMG_STATE(vmg, mm, &vmi, new_start, old_end, EMPTY_VMA_FLAGS,
- vma_start_pgoff(vma));
+ vma_start_pgoff(vma), vma_start_anon_pgoff(vma));
struct vm_area_struct *next;
struct mmu_gather tlb;
PAGETABLE_MOVE(pmc, vma, vma, old_start, new_start, length);
diff --git a/mm/vma_init.c b/mm/vma_init.c
index 715feee283f0fb..710b18849a3696 100644
--- a/mm/vma_init.c
+++ b/mm/vma_init.c
@@ -51,6 +51,7 @@ static void vm_area_init_from(const struct vm_area_struct *src,
dest->vm_end = src->vm_end;
dest->anon_vma = src->anon_vma;
dest->vm_pgoff = vma_start_pgoff(src);
+ __vma_set_virt_pgoff(dest, vma_start_virt_pgoff(src));
dest->vm_file = src->vm_file;
dest->vm_private_data = src->vm_private_data;
vm_flags_init(dest, src->vm_flags);
diff --git a/mm/vma_internal.h b/mm/vma_internal.h
index 4d300e7bbaf4c2..385c0ab1377748 100644
--- a/mm/vma_internal.h
+++ b/mm/vma_internal.h
@@ -23,6 +23,7 @@
#include <linux/ksm.h>
#include <linux/khugepaged.h>
#include <linux/list.h>
+#include <linux/major.h>
#include <linux/maple_tree.h>
#include <linux/mempolicy.h>
#include <linux/mm.h>
diff --git a/tools/testing/selftests/mm/merge.c b/tools/testing/selftests/mm/merge.c
index 519e5ac02db78c..edfbd39ae58eb4 100644
--- a/tools/testing/selftests/mm/merge.c
+++ b/tools/testing/selftests/mm/merge.c
@@ -1305,6 +1305,167 @@ TEST_F(merge, merge_vmas_with_mseal)
ASSERT_EQ(procmap->query.vma_end, (unsigned long)ptr + 2 * page_size);
}
+TEST_F(merge, virt_and_page_offset_mismatch_memfd)
+{
+ struct procmap_fd *procmap = &self->procmap;
+ unsigned int page_size = self->page_size;
+ char *carveout = self->carveout;
+ char *ptr, *ptr2;
+ int fd;
+
+ /* Create a 10 page memfd descriptor. */
+ fd = memfd_create("virt_page_offset_test", MFD_CLOEXEC);
+ ASSERT_NE(fd, -1);
+ ASSERT_EQ(ftruncate(fd, 10 * page_size), 0);
+
+ /* Map a region using the memfd at page offset 0. */
+ ptr = mmap(carveout, 5 * page_size, PROT_READ | PROT_WRITE,
+ MAP_FIXED | MAP_PRIVATE, fd, 0);
+ ASSERT_NE(ptr, MAP_FAILED);
+
+ /*
+ * Map another separately, and fault in, at page offset 5:
+ *
+ * |-----------| |---------|
+ * | unfaulted | | faulted |
+ * |-----------| |---------|
+ */
+ ptr2 = mmap(&carveout[10 * page_size], 5 * page_size,
+ PROT_READ | PROT_WRITE, MAP_FIXED | MAP_PRIVATE,
+ fd, 5 * page_size);
+ ASSERT_NE(ptr2, MAP_FAILED);
+ ptr2[0] = 'x';
+
+ /*
+ * Now move it in place:
+ *
+ * |----------|
+ * | |
+ * v |
+ * |-----------| |---------|
+ * | unfaulted | | faulted |
+ * |-----------| |---------|
+ *
+ * Because virtual page offset of the faulted region is now
+ * &carveout[10 * page_size], despite the two regions being mergeable
+ * due to file page offset, they are NOT mergeable due to virtual page
+ * offset.
+ */
+ ptr2 = sys_mremap(ptr2, 5 * page_size, 5 * page_size,
+ MREMAP_MAYMOVE | MREMAP_FIXED,
+ &carveout[5 * page_size]);
+ ASSERT_NE(ptr2, MAP_FAILED);
+
+ /* Assert that they did not merge. */
+ ASSERT_TRUE(find_vma_procmap(procmap, ptr));
+ ASSERT_EQ(procmap->query.vma_start, (unsigned long)ptr);
+ ASSERT_EQ(procmap->query.vma_end, (unsigned long)ptr + 5 * page_size);
+}
+
+TEST_F(merge, merge_map_private_dev_zero_unfaulted)
+{
+ struct procmap_fd *procmap = &self->procmap;
+ unsigned int page_size = self->page_size;
+ char *carveout = self->carveout;
+ char *ptr, *ptr2;
+ int fd_zero;
+
+ if (access("/dev/zero", F_OK))
+ SKIP(return, "No /dev/zero.");
+ fd_zero = open("/dev/zero", O_RDWR);
+ ASSERT_NE(fd_zero, -1);
+
+ /*
+ * Map two MAP_PRIVATE-/dev/zero VMAs next to one another with offset 0
+ * each.
+ *
+ * With these being made truly anonymous upon mapping, they will
+ * merge. If they were file-backed VMAs the page offsets would prevent
+ * merge:
+ *
+ * |-----||------| |-------------|
+ * | ptr || ptr2 | -> | ptr |
+ * |-----||------| |-------------|
+ */
+ ptr = mmap(carveout, 5 * page_size, PROT_READ | PROT_WRITE,
+ MAP_FIXED | MAP_PRIVATE, fd_zero, 0);
+ if (ptr == MAP_FAILED) {
+ close(fd_zero);
+ ASSERT_TRUE(false);
+ }
+ ptr2 = mmap(&carveout[5 * page_size], 5 * page_size,
+ PROT_READ | PROT_WRITE, MAP_FIXED | MAP_PRIVATE, fd_zero, 0);
+ if (ptr2 == MAP_FAILED) {
+ close(fd_zero);
+ ASSERT_TRUE(false);
+ }
+ close(fd_zero);
+
+ /* Assert that they merged. */
+ ASSERT_TRUE(find_vma_procmap(procmap, ptr));
+ ASSERT_EQ(procmap->query.vma_start, (unsigned long)ptr);
+ ASSERT_EQ(procmap->query.vma_end, (unsigned long)ptr + 10 * page_size);
+}
+
+TEST_F(merge, merge_map_private_dev_zero_faulted_unfaulted)
+{
+ struct procmap_fd *procmap = &self->procmap;
+ unsigned int page_size = self->page_size;
+ char *carveout = self->carveout;
+ char *ptr, *ptr2;
+ int fd_zero;
+
+ if (access("/dev/zero", F_OK))
+ SKIP(return, "No /dev/zero.");
+ fd_zero = open("/dev/zero", O_RDWR);
+ ASSERT_NE(fd_zero, -1);
+
+ /*
+ * Map a MAP_PRIVATE mapping of /dev/zero with page offset 0, then fault
+ * it in:
+ *
+ * |-------------------------------|
+ * | faulted |
+ * |-------------------------------|
+ */
+ ptr = mmap(carveout, 15 * page_size, PROT_READ | PROT_WRITE,
+ MAP_FIXED | MAP_PRIVATE, fd_zero, 0);
+ if (ptr == MAP_FAILED) {
+ close(fd_zero);
+ ASSERT_TRUE(false);
+ }
+ memset(ptr, 'x', 15 * page_size);
+
+ /*
+ * Unmap the middle:
+ *
+ * |---------| |---------|
+ * | faulted | | faulted |
+ * |---------| |---------|
+ */
+ ASSERT_EQ(munmap(&ptr[5 * page_size], 5 * page_size), 0);
+
+ /*
+ * Map in a new unfaulted mapping in the middle with page offset 0 -
+ * this should merge and would not if it were treated as a file rather
+ * than pure anon:
+ *
+ * |---------|-----------|---------|
+ * | faulted | unfaulted | faulted |
+ * |---------|-----------|---------|
+ */
+ ptr2 = mmap(&carveout[5 * page_size], 5 * page_size,
+ PROT_READ | PROT_WRITE, MAP_FIXED | MAP_PRIVATE,
+ fd_zero, 0);
+ close(fd_zero);
+ ASSERT_NE(ptr2, MAP_FAILED);
+
+ /* Assert that they merged. */
+ ASSERT_TRUE(find_vma_procmap(procmap, ptr));
+ ASSERT_EQ(procmap->query.vma_start, (unsigned long)ptr);
+ ASSERT_EQ(procmap->query.vma_end, (unsigned long)ptr + 15 * page_size);
+}
+
TEST_F(merge_with_fork, mremap_faulted_to_unfaulted_prev)
{
struct procmap_fd *procmap = &self->procmap;
diff --git a/tools/testing/vma/include/dup.h b/tools/testing/vma/include/dup.h
index c52b23773cd263..b4f0f7c24fcb8a 100644
--- a/tools/testing/vma/include/dup.h
+++ b/tools/testing/vma/include/dup.h
@@ -15,6 +15,16 @@ struct task_struct *get_current(void);
#define MMF_HAS_MDWE 28
#define current get_current()
+#define MINORBITS 20
+#define MINORMASK ((1U << MINORBITS) - 1)
+
+#define MAJOR(dev) ((unsigned int) ((dev) >> MINORBITS))
+#define MINOR(dev) ((unsigned int) ((dev) & MINORMASK))
+#define MKDEV(ma, mi) (((ma) << MINORBITS) | (mi))
+
+#define MEM_MAJOR 1
+#define DEVZERO_MINOR 5
+
/*
* Define the task command name length as enum, then it can be visible to
* BPF programs.
@@ -45,6 +55,9 @@ struct address_space {
unsigned long flags;
atomic_t i_mmap_writable;
};
+struct inode {
+ dev_t i_rdev;
+};
struct file_operations {
int (*mmap)(struct file *, struct vm_area_struct *);
int (*mmap_prepare)(struct vm_area_desc *);
@@ -52,6 +65,7 @@ struct file_operations {
struct file {
struct address_space *f_mapping;
const struct file_operations *f_op;
+ struct inode *f_inode;
};
struct anon_vma_chain {
struct anon_vma *anon_vma;
@@ -577,6 +591,7 @@ struct vm_area_struct {
*/
unsigned int vm_lock_seq;
#endif
+ unsigned int __vm_virt_pgoff_lo;
/*
* A file's MAP_PRIVATE vma can be in both i_mmap tree and anon_vma
@@ -612,6 +627,9 @@ struct vm_area_struct {
#ifdef CONFIG_PER_VMA_LOCK
/* Unstable RCU readers are allowed to read this. */
refcount_t vm_refcnt;
+#endif
+#ifdef CONFIG_64BIT
+ unsigned int __vm_virt_pgoff_hi;
#endif
/*
* For areas with an address space and backing store,
@@ -1320,6 +1338,28 @@ static inline pgoff_t vma_end_pgoff(const struct vm_area_struct *vma)
return vma_start_pgoff(vma) + vma_pages(vma);
}
+static inline pgoff_t vma_start_virt_pgoff(const struct vm_area_struct *vma)
+{
+ pgoff_t pgoff = 0;
+
+#ifdef CONFIG_64BIT
+ pgoff += vma->__vm_virt_pgoff_hi;
+ pgoff <<= 32;
+#endif
+ pgoff += vma->__vm_virt_pgoff_lo;
+ return pgoff;
+}
+
+static inline pgoff_t vma_end_virt_pgoff(const struct vm_area_struct *vma)
+{
+ return vma_start_virt_pgoff(vma) + vma_pages(vma);
+}
+
+static inline pgoff_t vma_last_virt_pgoff(const struct vm_area_struct *vma)
+{
+ return vma_end_virt_pgoff(vma) - 1;
+}
+
static inline int vfs_mmap_prepare(struct file *file, struct vm_area_desc *desc)
{
return file->f_op->mmap_prepare(desc);
@@ -1584,3 +1624,40 @@ static inline pgprot_t vma_get_page_prot(const struct vm_area_struct *vma)
{
return vma_flags_to_page_prot(vma->flags);
}
+
+static inline pgoff_t __linear_virt_page_index(const struct vm_area_struct *vma,
+ const unsigned long address)
+{
+ pgoff_t pgoff;
+
+ pgoff = linear_page_delta(vma, address);
+ pgoff += vma_start_virt_pgoff(vma);
+ return pgoff;
+}
+
+static inline pgoff_t linear_virt_page_index(const struct vm_area_struct *vma,
+ const unsigned long address)
+{
+ const pgoff_t pgoff = __linear_virt_page_index(vma, address);
+
+ VM_WARN_ON_ONCE(vma_test(vma, VMA_SHARED_BIT));
+ if (!vma->vm_file) /* Is anonymous except MAP_PRIVATE-/dev/zero */
+ VM_WARN_ON_ONCE(pgoff != linear_page_index(vma, address));
+
+ return pgoff;
+}
+
+static inline struct inode *file_inode(const struct file *f)
+{
+ return f->f_inode;
+}
+
+static inline unsigned iminor(const struct inode *inode)
+{
+ return MINOR(inode->i_rdev);
+}
+
+static inline unsigned imajor(const struct inode *inode)
+{
+ return MAJOR(inode->i_rdev);
+}
diff --git a/tools/testing/vma/shared.c b/tools/testing/vma/shared.c
index bea9ea6db02a4c..f410bb6f858e11 100644
--- a/tools/testing/vma/shared.c
+++ b/tools/testing/vma/shared.c
@@ -23,7 +23,8 @@ struct vm_area_struct *alloc_vma(struct mm_struct *mm,
vma->vm_start = start;
vma->vm_end = end;
- vma->vm_pgoff = pgoff;
+ vma_set_pgoff(vma, pgoff);
+ vma_set_virt_pgoff(vma, start >> PAGE_SHIFT);
vma->flags = vma_flags;
vma_assert_detached(vma);
diff --git a/tools/testing/vma/tests/merge.c b/tools/testing/vma/tests/merge.c
index e357accc849905..a01107ab8e2134 100644
--- a/tools/testing/vma/tests/merge.c
+++ b/tools/testing/vma/tests/merge.c
@@ -45,6 +45,7 @@ void vmg_set_range(struct vma_merge_struct *vmg, unsigned long start,
vmg->start = start;
vmg->end = end;
vmg->pgoff = pgoff;
+ vmg->anon_pgoff = start >> PAGE_SHIFT;
vmg->vma_flags = vma_flags;
vmg->just_expand = false;
@@ -108,6 +109,7 @@ static bool test_simple_merge(void)
.end = 0x2000,
.vma_flags = vma_flags,
.pgoff = 1,
+ .anon_pgoff = 1,
};
ASSERT_FALSE(attach_vma(&mm, vma_left));
@@ -119,6 +121,7 @@ static bool test_simple_merge(void)
ASSERT_EQ(vma->vm_start, 0);
ASSERT_EQ(vma->vm_end, 0x3000);
ASSERT_EQ(vma_start_pgoff(vma), 0);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
ASSERT_FLAGS_SAME_MASK(&vma->flags, vma_flags);
detach_free_vma(vma);
@@ -151,6 +154,7 @@ static bool test_simple_modify(void)
ASSERT_EQ(vma->vm_start, 0x1000);
ASSERT_EQ(vma->vm_end, 0x2000);
ASSERT_EQ(vma_start_pgoff(vma), 1);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 1);
/*
* Now walk through the three split VMAs and make sure they are as
@@ -163,6 +167,7 @@ static bool test_simple_modify(void)
ASSERT_EQ(vma->vm_start, 0);
ASSERT_EQ(vma->vm_end, 0x1000);
ASSERT_EQ(vma_start_pgoff(vma), 0);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
detach_free_vma(vma);
vma_iter_clear(&vmi);
@@ -172,6 +177,7 @@ static bool test_simple_modify(void)
ASSERT_EQ(vma->vm_start, 0x1000);
ASSERT_EQ(vma->vm_end, 0x2000);
ASSERT_EQ(vma_start_pgoff(vma), 1);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 1);
detach_free_vma(vma);
vma_iter_clear(&vmi);
@@ -181,6 +187,7 @@ static bool test_simple_modify(void)
ASSERT_EQ(vma->vm_start, 0x2000);
ASSERT_EQ(vma->vm_end, 0x3000);
ASSERT_EQ(vma_start_pgoff(vma), 2);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 2);
detach_free_vma(vma);
mtree_destroy(&mm.mm_mt);
@@ -210,6 +217,7 @@ static bool test_simple_expand(void)
ASSERT_EQ(vma->vm_start, 0);
ASSERT_EQ(vma->vm_end, 0x3000);
ASSERT_EQ(vma_start_pgoff(vma), 0);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
detach_free_vma(vma);
mtree_destroy(&mm.mm_mt);
@@ -232,6 +240,7 @@ static bool test_simple_shrink(void)
ASSERT_EQ(vma->vm_start, 0);
ASSERT_EQ(vma->vm_end, 0x1000);
ASSERT_EQ(vma_start_pgoff(vma), 0);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
detach_free_vma(vma);
mtree_destroy(&mm.mm_mt);
@@ -344,6 +353,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,
ASSERT_EQ(vma->vm_start, 0);
ASSERT_EQ(vma->vm_end, 0x5000);
ASSERT_EQ(vma_start_pgoff(vma), 0);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
ASSERT_EQ(vma->anon_vma, &dummy_anon_vma);
ASSERT_TRUE(vma_write_started(vma));
ASSERT_EQ(mm.map_count, 3);
@@ -365,6 +375,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,
ASSERT_EQ(vma->vm_start, 0x6000);
ASSERT_EQ(vma->vm_end, 0x9000);
ASSERT_EQ(vma_start_pgoff(vma), 6);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 6);
ASSERT_EQ(vma->anon_vma, &dummy_anon_vma);
ASSERT_TRUE(vma_write_started(vma));
ASSERT_EQ(mm.map_count, 3);
@@ -385,6 +396,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,
ASSERT_EQ(vma->vm_start, 0);
ASSERT_EQ(vma->vm_end, 0x9000);
ASSERT_EQ(vma_start_pgoff(vma), 0);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
ASSERT_EQ(vma->anon_vma, &dummy_anon_vma);
ASSERT_TRUE(vma_write_started(vma));
ASSERT_EQ(mm.map_count, 2);
@@ -405,6 +417,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,
ASSERT_EQ(vma->vm_start, 0xa000);
ASSERT_EQ(vma->vm_end, 0xc000);
ASSERT_EQ(vma_start_pgoff(vma), 0xa);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 0xa);
ASSERT_EQ(vma->anon_vma, &dummy_anon_vma);
ASSERT_TRUE(vma_write_started(vma));
ASSERT_EQ(mm.map_count, 2);
@@ -424,6 +437,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,
ASSERT_EQ(vma->vm_start, 0);
ASSERT_EQ(vma->vm_end, 0xc000);
ASSERT_EQ(vma_start_pgoff(vma), 0);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
ASSERT_EQ(vma->anon_vma, &dummy_anon_vma);
ASSERT_TRUE(vma_write_started(vma));
ASSERT_EQ(mm.map_count, 1);
@@ -444,6 +458,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,
ASSERT_EQ(vma->vm_start, 0);
ASSERT_EQ(vma->vm_end, 0xc000);
ASSERT_EQ(vma_start_pgoff(vma), 0);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
ASSERT_EQ(vma->anon_vma, &dummy_anon_vma);
detach_free_vma(vma);
@@ -806,6 +821,7 @@ static bool test_vma_merge_new_with_close(void)
ASSERT_EQ(vma->vm_start, 0);
ASSERT_EQ(vma->vm_end, 0x5000);
ASSERT_EQ(vma_start_pgoff(vma), 0);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
ASSERT_EQ(vma->vm_ops, &vm_ops);
ASSERT_TRUE(vma_write_started(vma));
ASSERT_EQ(mm.map_count, 2);
@@ -866,6 +882,7 @@ static bool __test_merge_existing(bool prev_is_sticky, bool middle_is_sticky, bo
ASSERT_EQ(vma->vm_start, 0x2000);
ASSERT_EQ(vma->vm_end, 0x3000);
ASSERT_EQ(vma_start_pgoff(vma), 2);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 2);
ASSERT_TRUE(vma_write_started(vma));
ASSERT_TRUE(vma_write_started(vma_next));
ASSERT_EQ(mm.map_count, 2);
@@ -932,6 +949,7 @@ static bool __test_merge_existing(bool prev_is_sticky, bool middle_is_sticky, bo
ASSERT_EQ(vma->vm_start, 0x6000);
ASSERT_EQ(vma->vm_end, 0x7000);
ASSERT_EQ(vma_start_pgoff(vma), 6);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 6);
ASSERT_TRUE(vma_write_started(vma_prev));
ASSERT_TRUE(vma_write_started(vma));
ASSERT_EQ(mm.map_count, 2);
@@ -1417,6 +1435,7 @@ static bool test_merge_extend(void)
ASSERT_EQ(vma->vm_start, 0);
ASSERT_EQ(vma->vm_end, 0x4000);
ASSERT_EQ(vma_start_pgoff(vma), 0);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 0);
ASSERT_TRUE(vma_write_started(vma));
ASSERT_EQ(mm.map_count, 1);
@@ -1431,7 +1450,7 @@ static bool test_expand_only_mode(void)
struct mm_struct mm = {};
VMA_ITERATOR(vmi, &mm, 0);
struct vm_area_struct *vma_prev, *vma;
- VMG_STATE(vmg, &mm, &vmi, 0x5000, 0x9000, vma_flags, 5);
+ VMG_STATE(vmg, &mm, &vmi, 0x5000, 0x9000, vma_flags, 5, 5);
/*
* Place a VMA prior to the one we're expanding so we assert that we do
@@ -1457,6 +1476,7 @@ static bool test_expand_only_mode(void)
ASSERT_EQ(vma->vm_start, 0x3000);
ASSERT_EQ(vma->vm_end, 0x9000);
ASSERT_EQ(vma_start_pgoff(vma), 3);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 3);
ASSERT_TRUE(vma_write_started(vma));
ASSERT_EQ(vma_iter_addr(&vmi), 0x3000);
vma_assert_attached(vma);
diff --git a/tools/testing/vma/tests/mmap.c b/tools/testing/vma/tests/mmap.c
index c85bc000d1cb7a..0c8a3446b90502 100644
--- a/tools/testing/vma/tests/mmap.c
+++ b/tools/testing/vma/tests/mmap.c
@@ -45,7 +45,56 @@ static bool test_mmap_region_basic(void)
return true;
}
+static int dummy_mmap_prepare(struct vm_area_desc *desc)
+{
+ return 0;
+}
+
+static bool test_pure_anon_dev_zero(void)
+{
+ const vma_flags_t vma_flags = mk_vma_flags(VMA_READ_BIT, VMA_WRITE_BIT,
+ VMA_MAYREAD_BIT, VMA_MAYWRITE_BIT);
+ const struct file_operations f_op = {
+ .mmap_prepare = dummy_mmap_prepare,
+ };
+ struct inode inode = {
+ .i_rdev = MKDEV(MEM_MAJOR, DEVZERO_MINOR),
+ };
+ struct file file = {
+ .f_inode = &inode,
+ .f_op = &f_op,
+ };
+ struct mm_struct mm = {};
+ struct vm_area_struct *vma;
+ unsigned long addr;
+ VMA_ITERATOR(vmi, &mm, 0);
+
+ current->mm = &mm;
+
+ /*
+ * Map a MAP_PRIVATE-/dev/zero mapping at address 0x300000 with a page
+ * offset of 0x10, which we expect to be reset to the virtual page
+ * offset.
+ */
+ addr = __mmap_region(&file, 0x300000, 0x3000, vma_flags, 0x10, NULL);
+ ASSERT_EQ(addr, 0x300000);
+
+ /* Assert that it truly is an anonymous mapping. */
+ vma = vma_lookup(&mm, addr);
+ ASSERT_NE(vma, NULL);
+ ASSERT_TRUE(vma_is_anonymous(vma));
+ ASSERT_EQ(vma->vm_file, NULL);
+ ASSERT_EQ(vma->vm_private_data, NULL);
+ /* Expect virtual page offsets. */
+ ASSERT_EQ(vma->vm_pgoff, 0x300);
+ ASSERT_EQ(vma_start_virt_pgoff(vma), 0x300);
+
+ cleanup_mm(&mm, &vmi);
+ return true;
+}
+
static void run_mmap_tests(int *num_tests, int *num_fail)
{
TEST(mmap_region_basic);
+ TEST(pure_anon_dev_zero);
}
diff --git a/tools/testing/vma/tests/vma.c b/tools/testing/vma/tests/vma.c
index 754a2da0632157..7ca5289e0f9593 100644
--- a/tools/testing/vma/tests/vma.c
+++ b/tools/testing/vma/tests/vma.c
@@ -38,7 +38,7 @@ static bool test_copy_vma(void)
/* Move backwards and do not merge. */
vma = alloc_and_link_vma(&mm, 0x3000, 0x5000, 3, vma_flags);
- vma_new = copy_vma(&vma, 0, 0x2000, 0, &need_locks);
+ vma_new = copy_vma(&vma, 0, 0x2000, 0, 3, &need_locks);
ASSERT_NE(vma_new, vma);
ASSERT_EQ(vma_new->vm_start, 0);
ASSERT_EQ(vma_new->vm_end, 0x2000);
@@ -51,7 +51,7 @@ static bool test_copy_vma(void)
vma = alloc_and_link_vma(&mm, 0, 0x2000, 0, vma_flags);
vma_next = alloc_and_link_vma(&mm, 0x6000, 0x8000, 6, vma_flags);
- vma_new = copy_vma(&vma, 0x4000, 0x2000, 4, &need_locks);
+ vma_new = copy_vma(&vma, 0x4000, 0x2000, 4, 4, &need_locks);
vma_assert_attached(vma_new);
ASSERT_EQ(vma_new, vma_next);
diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_internal.h
index 4f6c5666ac07d3..8a48b231aa7abf 100644
--- a/tools/testing/vma/vma_internal.h
+++ b/tools/testing/vma/vma_internal.h
@@ -53,6 +53,7 @@ typedef __bitwise unsigned int vm_fault_t;
#define VM_WARN_ON(_expr) (WARN_ON(_expr))
#define VM_WARN_ON_ONCE(_expr) (WARN_ON_ONCE(_expr))
+#define VM_WARN_ON_ONCE_VMA(_expr, _vma) (WARN_ON_ONCE(_expr))
#define VM_WARN_ON_VMG(_expr, _vmg) (WARN_ON(_expr))
#define VM_BUG_ON(_expr) (BUG_ON(_expr))
#define VM_BUG_ON_VMA(_expr, _vma) (BUG_ON(_expr))
|
| 3/2 | 2026/07/17 22:47 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 29504 output: 32 thoughts: 1274 Thoughts: **Analyzing a Memory Management Patch** Okay, so I've got a patch here that's tweaking the memory management code, specifically related to how `/dev/zero` is handled with `MAP_PRIVATE` mappings. The commit message just says it's under review from a syz-cluster, so let's dig in. Looking at the diff, it seems like `drivers/char/mem.c:mmap_zero_prepare` is the starting point. It's dropped that `vma_desc_set_anonymous` call, which means we're now relying on the core MM to manage `MAP_PRIVATE` semantics. Intriguing. There are a bunch of new defines and fields in the include files, particularly around virtual page offsets (`vma_start_virt_pgoff`, `vma_end_virt_pgoff`, and the `__vm_virt_pgoff_` fields). Plus, `linear_virt_page_index` and `linear_folio_page_index` are new. This is a pretty fundamental change. The patch touches a ton of files: `mm/huge_memory.c`, `mm/ksm.c`, `mm/memory-failure.c`, `mm/memory.c`, `mm/migrate.c`, `mm/mremap.c`, `mm/page_vma_mapped.c`, `mm/rmap.c`, `mm/userfaultfd.c`, `mm/vma.c`, `mm/vma.h`, `mm/vma_exec.c`, and `mm/vma_init.c`. This is definitely a candidate for thorough fuzzing; `WorthFuzzing = true`. I need to nail down the specific functions to focus on. Let's see, besides that `mmap_zero_prepare`, it's impacting: * `move_pages_huge_pmd` in `mm/huge_memory.c` * `try_to_merge_with_ksm_page`, `ksm_might_need_to_copy`, `rmap_walk_ksm` in `mm/ksm.c` * `add_to_kill_fsdax`, `add_to_kill_pgoff` in `mm/memory-failure.c` * `print_bad_page_map` in `mm/memory.c` * `remove_migration_pte` in `mm/migrate.c` * `copy_vma_and_data` in `mm/mremap.c` * `page_mapped_in_vma` in `mm/page_vma_mapped.c` * `page_address_in_vma`, `mapping_wrprotect_range_one`, `pfn_mkclean_range`, `__folio_set_anon`, `__page_check_anon_rmap`, `rmap_walk_anon`, `__rmap_walk_file` in `mm/rmap.c` * `move_present_ptes`, `move_swap_pte` in `mm/userfaultfd.c` * `commit_merge`, `vma_merge_existing_range`, `vma_merge_new_range`, `copy_vma`, `anon_vma_compatible`, `__mmap_new_file_vma`, `__mmap_new_vma`, `call_mmap_prepare`, `set_vma_user_defined_fields`, `__mmap_region`, `do_brk_flags`, `insert_vm_struct`, `__install_special_mapping` in `mm/vma.c` * `relocate_vma_down` in `mm/vma_exec.c` * `vm_area_init_from` in `mm/vma_init.c` Okay, I've got a lot to consider. Now, I want to filter out the high-frequency functions. I will avoid hot-path functions. Good targets for fuzzing focus are: * `mmap_zero_prepare` * `move_pages_huge_pmd` * `try_to_merge_with_ksm_page` * `ksm_might_need_to_copy` * `rmap_walk_ksm` * `add_to_kill_fsdax` * `add_to_kill_pgoff` * `remove_migration_pte` * `copy_vma_ |
| 4/2 | 2026/07/17 22:47 | tool | grepper |
0mArgs:{
"Expression": "^(menuconfig|config) KSM",
"PathPrefix": "mm/Kconfig"
}Results: {
"Output": "mm/Kconfig=722=config MMU_NOTIFIER\n--\nmm/Kconfig-725-\nmm/Kconfig:726:config KSM\nmm/Kconfig-727-\tbool \"Enable KSM for page merging\"\n"
} |
| 5/2 | 2026/07/17 22:47 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 31908 output: 33 thoughts: 10 Thoughts: Okay, here's my summary, tailored as requested: **KSM and USERFAULTFD: A Quick Mental Run-Through** Alright, KSM. Always a solid choice, you know? Good, reliable kernel-level memory deduplication. Effective at what it does, especially in virtualized environments. I'm usually happy to see it in the mix. Now, USERFAULTFD... ah, *that's* where things get interesting. I start thinking about its potential. It's not just a tool; it's more of a framework, isn't it? A way to manage page faults in userspace. And the implications... they're significant. Fine-grained control over memory management, that’s key. Think about the possibilities for advanced memory allocation, custom memory policies, and even emulating certain aspects of hardware directly. My brain jumps to the implications for high-performance computing, specialized memory subsystems, and even security hardening. USERFAULTFD really lets you get down to the metal, and bypass some of the layers of abstraction we are often forced to work with. There are trade-offs of course, the complexity can be a pain in the arse. I start to wonder about the implementation details. What's the overhead? How does it affect context switching? How does it interact with the other memory management subsystems? Those are the key questions. You really need to understand those things, to see what this thing is really going to be useful for. |
| 6/2 | 2026/07/17 22:47 | tool | grepper |
0mError:no matches Args: {
"Expression": "^(menuconfig|config) USERFAULTFD",
"PathPrefix": "init/Kconfig"
}Results: {
"error": "no matches"
} |
| 7/2 | 2026/07/17 22:47 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 32274 output: 29 thoughts: 8 Thoughts: Okay, here's my attempt at summarizing the thought process, tailored for an expert user and presented in the first person: **Finding USERFAULTFD** Alright, let's get down to brass tacks. My initial thought process is laser-focused: I need to locate the details on `USERFAULTFD`. This is a crucial area, and I need to quickly establish a framework for understanding it. The goal is pinpointing specifics - its behavior, its relationship to other system mechanisms, potential implications... I'm immediately running through the various resources in my mind: man pages, kernel documentation, past experiences, perhaps even discussions from mailing lists or relevant projects I've been following. The context is implied - I understand the architecture involved, and am ready to dive deep into specifics. |
| 8/2 | 2026/07/17 22:47 | tool | grepper |
0mArgs:{
"Expression": "^(menuconfig|config) USERFAULTFD",
"PathPrefix": ""
}Results: {
"Output": "mm/Kconfig=1368=config HAVE_ARCH_USERFAULTFD_MINOR\n--\nmm/Kconfig-1372-\nmm/Kconfig:1373:config USERFAULTFD_RWP\nmm/Kconfig-1374-\tdef_bool y\n--\nmm/Kconfig-1381-\nmm/Kconfig:1382:menuconfig USERFAULTFD\nmm/Kconfig-1383-\tbool \"Enable userfaultfd() system call\"\n"
} |
| 9/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 32604 output: 34 thoughts: 11 Thoughts: Here's my summary, as though I'm thinking these thoughts: **USERFAULTFD and Memory Failures: A Quick Assessment** Okay, so I'm thinking about `USERFAULTFD`. It's definitely a valuable tool in the kit, especially when it comes to memory management and handling potential page faults at a granular level. I see its utility, particularly in applications where you need fine-grained control over how memory is allocated, mapped, and handled. I'm thinking of scenarios like custom memory allocators, virtual memory implementations within userspace, and even potentially security-focused applications where you need to carefully monitor and respond to memory accesses. Now, that said, the context is MEMORY_FAILURE, which immediately flags some deeper considerations. I'm wondering what the specific nature of the failure is. Is it a hardware issue, a page table corruption, a resource exhaustion problem, a permissions issue, or something else entirely? The type of failure will drive the appropriate response. Thinking about how `USERFAULTFD` can assist, I'll need to remember to correctly implement signal handling and the right flags. With `USERFAULTFD`, I can design a custom error handling mechanism for this issue. What's clear to me is that `USERFAULTFD` gives me the ability to intercept, analyze, and *attempt* to rectify memory errors, though I'll have to consider if an attempt is wise depending on the root cause and whether recovery is even possible or desirable. It opens doors, but it also carries the responsibility of properly managing the process's memory and state. I need to keep in mind, of course, that if the memory issue is truly critical, the safest approach might be to terminate the process gracefully. |
| 10/2 | 2026/07/17 22:48 | tool | grepper |
0mArgs:{
"Expression": "^(menuconfig|config) MEMORY_FAILURE",
"PathPrefix": "mm/Kconfig"
}Results: {
"Output": "mm/Kconfig=760=config ARCH_SUPPORTS_MEMORY_FAILURE\n--\nmm/Kconfig-762-\nmm/Kconfig:763:config MEMORY_FAILURE\nmm/Kconfig-764-\tdepends on MMU\n"
} |
| 11/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 33081 output: 37 thoughts: 16 Thoughts: Okay, here's my interpretation of those thoughts, crafted with the understanding that I'm speaking to a fellow expert: **Reflecting on Memory Management Strategies** Ah, `MEMORY_FAILURE`. That's actually a positive signal, isn't it? It means the system is detecting and handling potential memory allocation issues, preventing a crash. It's a key defense mechanism. Better to fail early and informatively than to have a silent, catastrophic failure down the line. We can analyze the context of this failure and implement a robust strategy. And then we have `TRANSPARENT_HUGEPAGE`. I'm immediately thinking about its implications for performance. We're talking about significantly larger page sizes here, which can reduce TLB misses and potentially boost memory access times, particularly for large datasets. But of course, it's not a silver bullet. There are tradeoffs. The question that comes to mind is: are we properly evaluating the potential fragmentation issues and the impact on memory utilization, especially in our specific workload? We need to keep a close eye on the balance between performance gains and memory overhead, especially when we are dealing with systems that need both performance and efficiency. |
| 12/2 | 2026/07/17 22:48 | tool | grepper |
0mArgs:{
"Expression": "^(menuconfig|config) TRANSPARENT_HUGEPAGE",
"PathPrefix": "mm/Kconfig"
}Results: {
"Output": "mm/Kconfig=829=config MM_ID\n--\nmm/Kconfig-831-\nmm/Kconfig:832:menuconfig TRANSPARENT_HUGEPAGE\nmm/Kconfig-833-\tbool \"Transparent Hugepage Support\"\n"
} |
| 13/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 33454 output: 29 thoughts: 343 Thoughts: **Refining the Focus on Transparent Hugepages** Okay, Transparent Hugepages are definitely a key area of interest. Let's sharpen the focus on specific symbols to investigate further. I've got a list here: `mmap_zero_prepare`, `move_pages_huge_pmd`, `try_to_merge_with_ksm_page`, `ksm_might_need_to_copy`, `rmap_walk_ksm`, `add_to_kill_fsdax`, `add_to_kill_pgoff`, `remove_migration_pte`, `copy_vma_and_data`, `move_present_ptes`, `move_swap_pte`, and `relocate_vma_down`. That seems like a good starting point. Now, let's see how reachable these functions are. I see that `mmap_zero_prepare` is within `drivers/char/mem.c`. Ah, right, that's called when you map `/dev/zero`. That's *very* reachable, given how ubiquitous that is. `move_pages_huge_pmd` is in `mm/huge_memory.c`, and that's triggered during mremap of Transparent Hugepages, which is plausible. `try_to_merge_with_ksm_page` is in `mm/ksm.c`, so it's KSM related. `copy_vma_and_data` is in `mm/mremap.c`. And `move_present_ptes` and `move_swap_pte` are in `mm/userfaultfd.c`. Finally, `relocate_vma_down` is in `mm/vma_exec.c`. Let's quickly check on the reachability of `relocate_vma_down`. I'll have to investigate that further, as it's less obvious. |
| 14/2 | 2026/07/17 22:48 | tool | grepper |
0mArgs:{
"Expression": "relocate_vma_down",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/vma.h=839=int create_init_stack_vma(struct mm_struct *mm, struct vm_area_struct **vmap,\nmm/vma.h-840-\t\t\t unsigned long *top_mem_p);\nmm/vma.h:841:int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift);\nmm/vma.h-842-#endif\n--\nmm/vma_exec.c-22- */\nmm/vma_exec.c:23:int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift)\nmm/vma_exec.c-24-{\n--\nmm/vma_exec.c-102- *\nmm/vma_exec.c:103: * We later relocate this downwards in relocate_vma_down().\nmm/vma_exec.c-104- *\n"
} |
| 15/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 34485 output: 33 thoughts: 23 Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience: **Analyzing the `execve` Stack Setup** Alright, so we're diving into the nitty-gritty of `execve` stack setup, specifically looking at how things get arranged during the process of creating a new program image. The context strongly implies we're dealing with the low-level mechanics. My initial thought is to zero in on a critical function, `mmap_zero_prepare`, to further the process. From experience, I suspect this function plays a crucial role in initializing or preparing memory regions, particularly those associated with the stack frame that will contain our arguments, environment variables, and auxiliary vector. That function's involvement in the process flow is key. Let's see what's actually going on in there. |
| 16/2 | 2026/07/17 22:48 | tool | grepper |
0mArgs:{
"Expression": "mmap_zero_prepare",
"PathPrefix": "drivers/char/mem.c"
}Results: {
"Output": "drivers/char/mem.c=474=static ssize_t read_zero(struct file *file, char __user *buf,\n--\ndrivers/char/mem.c-500-\ndrivers/char/mem.c:501:static int mmap_zero_prepare(struct vm_area_desc *desc)\ndrivers/char/mem.c-502-{\n--\ndrivers/char/mem.c=521=static unsigned long get_unmapped_area_zero(struct file *file,\n--\ndrivers/char/mem.c-526-\t\t/*\ndrivers/char/mem.c:527:\t\t * mmap_zero_prepare() will call shmem_zero_setup() to create a\ndrivers/char/mem.c-528-\t\t * file, so use shmem's get_unmapped_area in case it can be\n--\ndrivers/char/mem.c=662=static const struct file_operations zero_fops = {\n--\ndrivers/char/mem.c-669-\t.splice_write\t= splice_write_zero,\ndrivers/char/mem.c:670:\t.mmap_prepare\t= mmap_zero_prepare,\ndrivers/char/mem.c-671-\t.get_unmapped_area = get_unmapped_area_zero,\n"
} |
| 17/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 35047 output: 35 thoughts: 34 Thoughts: Okay, here's my interpretation of those thoughts, presented as a first-person expert summary: **The `/dev/zero` Mmap Preparation** Right, that confirms it. This is definitely about the `mmap_prepare` function, specifically when it's being used with `/dev/zero`. That's a common enough use case, but the nuances can be tricky. Now, my focus should shift to understanding how the `copy_vma_and_data` function interacts in this context. It is important to know if there's any implicit data copying, memory allocation or any other data handling steps in there. This is especially important for `/dev/zero` because we expect zeroed memory, so I need to check how the VMAs and data are handled to ensure that the expected behavior is maintained. I need to delve into the details of the function to understand the precise mechanics involved. I need to verify that everything is aligned with the expected outcome: zeroed pages when mapping `/dev/zero`. |
| 18/2 | 2026/07/17 22:48 | tool | grepper |
0mArgs:{
"Expression": "copy_vma_and_data",
"PathPrefix": "mm/mremap.c"
}Results: {
"Output": "mm/mremap.c=1160=static void unmap_source_vma(struct vma_remap_struct *vrm)\n--\nmm/mremap.c-1264- */\nmm/mremap.c:1265:static int copy_vma_and_data(struct vma_remap_struct *vrm,\nmm/mremap.c-1266-\t\t\t struct vm_area_struct **new_vma_ptr)\n--\nmm/mremap.c=1350=static unsigned long move_vma(struct vma_remap_struct *vrm)\n--\nmm/mremap.c-1371-\t/* Perform copy step. */\nmm/mremap.c:1372:\terr = copy_vma_and_data(vrm, \u0026new_vma);\nmm/mremap.c-1373-\t/*\n"
} |
| 19/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 35571 output: 33 thoughts: 22 Thoughts: Here's my take on the thought process, as if I were the one thinking it: **Digging into mremap** Okay, I've been called during `mremap`. That's the key. I need to understand what's going on at this point. Time to dive into the core logic. Since I'm in `mremap`, the main focus is likely going to be around handling the mapping changes and potentially moving or adjusting page table entries. Let's start by scrutinizing `move_present_ptes`. That function has a high likelihood of telling me a lot about how these memory regions are being rearranged. I need to confirm whether we're shifting around existing physical pages, creating new ones, or updating virtual addresses. |
| 20/2 | 2026/07/17 22:48 | tool | grepper |
0mArgs:{
"Expression": "move_present_ptes",
"PathPrefix": "mm/userfaultfd.c"
}Results: {
"Output": "mm/userfaultfd.c=1280=static struct folio *check_ptes_for_batched_move(struct vm_area_struct *src_vma,\n--\nmm/userfaultfd.c-1308- */\nmm/userfaultfd.c:1309:static long move_present_ptes(struct mm_struct *mm,\nmm/userfaultfd.c-1310-\t\t\t struct vm_area_struct *dst_vma,\n--\nmm/userfaultfd.c=1503=static long move_pages_ptes(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd,\n--\nmm/userfaultfd.c-1676-\nmm/userfaultfd.c:1677:\t\tret = move_present_ptes(mm, dst_vma, src_vma,\nmm/userfaultfd.c-1678-\t\t\t\t\tdst_addr, src_addr, dst_pte, src_pte,\n"
} |
| 21/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 36050 output: 38 thoughts: 29 Thoughts: Okay, here's my interpretation of those thoughts, crafted with the understanding that I'm speaking to a fellow expert: **Investigating a UFFDIO_MOVE Call** Alright, so the call came in during `UFFDIO_MOVE`. My immediate instinct is to check `try_to_merge_with_ksm_page`. Given the context, this is the logical place to start. I need to understand why this merge attempt is being triggered during a move operation. Is this expected behavior? Are we optimizing something? Or is there a potential corner case or issue that needs addressing here? I'll probably be focusing on the conditions under which this merge is being attempted, the data being passed, and the outcome of the function. Time to dive in and see what's going on. |
| 22/2 | 2026/07/17 22:48 | tool | grepper |
0mArgs:{
"Expression": "try_to_merge_with_ksm_page",
"PathPrefix": "mm/ksm.c"
}Results: {
"Output": "mm/ksm.c=1561=static int try_to_merge_with_zero_page(struct ksm_rmap_item *rmap_item,\n--\nmm/ksm.c-1595-/*\nmm/ksm.c:1596: * try_to_merge_with_ksm_page - like try_to_merge_two_pages,\nmm/ksm.c-1597- * but no new kernel page is allocated: kpage must already be a ksm page.\n--\nmm/ksm.c-1600- */\nmm/ksm.c:1601:static int try_to_merge_with_ksm_page(struct ksm_rmap_item *rmap_item,\nmm/ksm.c-1602-\t\t\t\t struct page *page, struct page *kpage)\n--\nmm/ksm.c-1645- * Note that this function upgrades page to ksm page: if one of the pages\nmm/ksm.c:1646: * is already a ksm page, try_to_merge_with_ksm_page should be used.\nmm/ksm.c-1647- */\nmm/ksm.c=1648=static struct folio *try_to_merge_two_pages(struct ksm_rmap_item *rmap_item,\n--\nmm/ksm.c-1654-\nmm/ksm.c:1655:\terr = try_to_merge_with_ksm_page(rmap_item, page, NULL);\nmm/ksm.c-1656-\tif (!err) {\nmm/ksm.c:1657:\t\terr = try_to_merge_with_ksm_page(tree_rmap_item,\nmm/ksm.c-1658-\t\t\t\t\t\t\ttree_page, page);\n--\nmm/ksm.c=2278=static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_item)\n--\nmm/ksm.c-2338-\nmm/ksm.c:2339:\t\terr = try_to_merge_with_ksm_page(rmap_item, page, \u0026kfolio-\u003epage);\nmm/ksm.c-2340-\t\tif (!err) {\n"
} |
| 23/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 36867 output: 36 thoughts: 24 Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience: **Analyzing the KSM Merge Context** Right, that phone call just came in regarding the KSM merging process. It seems this arose during a crucial point. My first instinct is to dive into the `add_to_kill_fsdax` function. That's the likely spot to start looking for interactions with the specific KSM merge activity. We're dealing with filesystem DAX here, so any unexpected behavior there during merging is definitely worth immediate investigation. It's time to check what's going on with how we're killing or disabling FSDax pages and see if that's where the problem is. |
| 24/2 | 2026/07/17 22:48 | tool | grepper |
0mArgs:{
"Expression": "add_to_kill_fsdax",
"PathPrefix": "mm/memory-failure.c"
}Results: {
"Output": "mm/memory-failure.c=582=static void collect_procs_file(const struct folio *folio,\n--\nmm/memory-failure.c-618-#ifdef CONFIG_FS_DAX\nmm/memory-failure.c:619:static void add_to_kill_fsdax(struct task_struct *tsk, const struct page *p,\nmm/memory-failure.c-620-\t\t\t struct vm_area_struct *vma,\n--\nmm/memory-failure.c=630=static void collect_procs_fsdax(const struct page *page,\n--\nmm/memory-failure.c-652-\t\t\tif (vma-\u003evm_mm == t-\u003emm)\nmm/memory-failure.c:653:\t\t\t\tadd_to_kill_fsdax(t, page, vma, to_kill, pgoff);\nmm/memory-failure.c-654-\t\t}\n"
} |
| 25/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 37316 output: 36 thoughts: 27 Thoughts: Okay, here's my interpretation of those thoughts, crafted with the understanding that I'm talking to a fellow expert: **Investigating a Filesystem DAX Memory Failure** Right, so I got the call; it's a memory failure on FS DAX. Okay, let's get down to brass tacks. First things first, let's take a look at the `add_to_kill_pgoff` function. That seems like the logical place to start poking around when dealing with memory errors, specifically in this DAX context. We're obviously in a critical situation, so let's start with the basics of figuring out what caused the memory failure. |
| 26/2 | 2026/07/17 22:48 | tool | grepper |
0mArgs:{
"Expression": "add_to_kill_pgoff",
"PathPrefix": "mm/memory-failure.c"
}Results: {
"Output": "mm/memory-failure.c=2252=EXPORT_SYMBOL_GPL(unregister_pfn_address_space);\nmm/memory-failure.c-2253-\nmm/memory-failure.c:2254:static void add_to_kill_pgoff(struct task_struct *tsk,\nmm/memory-failure.c-2255-\t\t\t struct vm_area_struct *vma,\n--\nmm/memory-failure.c=2283=static void collect_procs_pfn(struct pfn_address_space *pfn_space,\n--\nmm/memory-failure.c-2302-\t\t\t !pfn_space-\u003epfn_to_vma_pgoff(vma, pfn, \u0026pgoff))\nmm/memory-failure.c:2303:\t\t\t\tadd_to_kill_pgoff(t, vma, to_kill, pgoff);\nmm/memory-failure.c-2304-\t\t}\n"
} |
| 27/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 37778 output: 30 thoughts: 23 Thoughts: Okay, here's my interpretation of those thoughts, crafted with the understanding that I'm discussing this with a fellow expert: **Investigating a PFN Mapping Memory Failure** Right, so we've hit an error during memory management, specifically, we've been called due to a failure involving Page Frame Number (PFN) mappings. My immediate instinct is to pinpoint the exact routine involved in removing these mappings. Given the context, my initial thought is that the function `remove_migration_pte` is the likely culprit, or at least a key component in the problem. I need to take a closer look at that function's implementation and how it's interacting with the PFN mappings to understand what's gone wrong. |
| 28/2 | 2026/07/17 22:48 | tool | grepper |
0mArgs:{
"Expression": "remove_migration_pte",
"PathPrefix": "mm/migrate.c"
}Results: {
"Output": "mm/migrate.c=299=static bool try_to_map_unused_to_zeropage(struct page_vma_mapped_walk *pvmw,\n--\nmm/migrate.c-332-\nmm/migrate.c:333:\t/* See remove_migration_pte(): restore PAGE_NONE for RWP */\nmm/migrate.c-334-\tif (pte_swp_uffd(old_pte) \u0026\u0026 userfaultfd_rwp(pvmw-\u003evma))\n--\nmm/migrate.c=343=struct rmap_walk_arg {\n--\nmm/migrate.c-350- */\nmm/migrate.c:351:static bool remove_migration_pte(struct folio *folio,\nmm/migrate.c-352-\t\tstruct vm_area_struct *vma, unsigned long addr, void *arg)\n--\nmm/migrate.c-455-\nmm/migrate.c:456:\t\ttrace_remove_migration_pte(pvmw.address, pte_val(pte),\nmm/migrate.c-457-\t\t\t\t\t compound_order(new));\n--\nmm/migrate.c-469- */\nmm/migrate.c:470:void remove_migration_ptes(struct folio *src, struct folio *dst,\nmm/migrate.c-471-\t\tenum ttu_flags flags)\n--\nmm/migrate.c-478-\tstruct rmap_walk_control rwc = {\nmm/migrate.c:479:\t\t.rmap_one = remove_migration_pte,\nmm/migrate.c-480-\t\t.arg = \u0026rmap_walk_arg,\n--\nmm/migrate.c=1177=static void migrate_folio_undo_src(struct folio *src, int was_mapped,\n--\nmm/migrate.c-1180-\tif (was_mapped)\nmm/migrate.c:1181:\t\tremove_migration_ptes(src, src, 0);\nmm/migrate.c-1182-\t/* Drop an anon_vma reference if we took one */\n--\nmm/migrate.c=1367=static int migrate_folio_move(free_folio_t put_new_folio, unsigned long private,\n--\nmm/migrate.c-1410-\t * When successful, push dst to LRU immediately: so that if it\nmm/migrate.c:1411:\t * turns out to be an mlocked page, remove_migration_ptes() will\nmm/migrate.c-1412-\t * automatically build up the correct dst-\u003emlock_count for it.\n--\nmm/migrate.c-1422-\tif (old_folio_state \u0026 FOLIO_WAS_MAPPED)\nmm/migrate.c:1423:\t\tremove_migration_ptes(src, dst, 0);\nmm/migrate.c-1424-\n--\nmm/migrate.c=1484=static int unmap_and_move_hugetlb_folio(new_folio_t get_new_folio,\n--\nmm/migrate.c-1556-\tif (was_mapped)\nmm/migrate.c:1557:\t\tremove_migration_ptes(src, !rc ? dst : src, ttu);\nmm/migrate.c-1558-\n"
} |
| 29/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 38814 output: 32 thoughts: 20 Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience: **Analyzing the Migration Callback** Alright, that's right, the trigger happened during a page migration, I'm noting that down. So, let's dive straight into `__install_special_mapping`. That function is critical here. It has to be - this is where we'll find the core logic for how the system handles the memory remapping during the page migration process. Understanding its behavior will be key to deciphering what’s happening. Let's see if we can identify any potential gotchas or edge cases, depending on how this installation works. |
| 30/2 | 2026/07/17 22:48 | tool | grepper |
0mArgs:{
"Expression": "__install_special_mapping",
"PathPrefix": "mm/vma.c"
}Results: {
"Output": "mm/vma.c=3481=__weak unsigned long vma_mmu_pagesize(struct vm_area_struct *vma)\n--\nmm/vma.c-3485-\nmm/vma.c:3486:struct vm_area_struct *__install_special_mapping(\nmm/vma.c-3487-\tstruct mm_struct *mm,\n"
} |
| 31/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 39114 output: 33 thoughts: 222 Thoughts: **My Analysis of Kernel VM Setup and Related Configurations** Okay, so I'm thinking through this kernel setup and recall that this is all happening during the `vdso/vvar` setup. Let's get right into it, I've got a list of focus symbols here. I'm seeing: `mmap_zero_prepare`, which makes sense in the context of memory mapping. Then we've got a cluster of symbols related to huge pages and page migration: `move_pages_huge_pmd`, `try_to_merge_with_ksm_page`, `ksm_might_need_to_copy`, `rmap_walk_ksm`, `add_to_kill_fsdax`, `add_to_kill_pgoff`, `remove_migration_pte`, `copy_vma_and_data`, `move_present_ptes`, `move_swap_pte`, `relocate_vma_down`, and finally, the rather generic-sounding, but probably critical, `__install_special_mapping`. Seems like a rather comprehensive list of functions involved in handling memory management tasks. Now, let's consider the relevant configurations. I see `KSM`, `USERFAULTFD`, `MEMORY_FAILURE`, `TRANSPARENT_HUGEPAGE`, and `MIGRATION`. Given the focus symbols, these make a lot of sense, right? KSM is obviously linked to the KSM functions. Userfaultfd is an interesting inclusion, it may be used to handle transparent page faults and memory faults and tie into the copy-on-write functionality. Memory failures are going to be key, and of course transparent huge pages, and migration seem to be intertwined with the focus symbols related to moving pages around. Let me just double check `MIGRATION` config real quick. It's crucial for the symbols that seem to be doing the heavy lifting around memory manipulation. |
| 32/2 | 2026/07/17 22:48 | tool | grepper |
0mArgs:{
"Expression": "^(menuconfig|config) MIGRATION",
"PathPrefix": "mm/Kconfig"
}Results: {
"Output": "mm/Kconfig=671=config NUMA_MIGRATION\n--\nmm/Kconfig-681-\nmm/Kconfig:682:config MIGRATION\nmm/Kconfig-683-\tbool\n"
} |
| 33/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 39844 output: 37 thoughts: 32 Thoughts: Okay, here's my interpretation of those thoughts, framed as my own: **Migration and FS_DAX Configuration** Right, migration is definitely a positive thing. It's a key strategy to implement here. I need to make sure I am doing a good job. But before I proceed too far down that path, let me double-check the configuration of `FS_DAX`. Specifically, I need to look at how we're handling `add_to_kill_fsdax`. I've probably worked with this many times before but it's important to make sure it is configured correctly, because the configuration will affect how effective this migration will be. So let's make sure it's set appropriately. Gotta dot the i's and cross the t's before we make any moves. |
| 34/2 | 2026/07/17 22:48 | tool | grepper |
0mError:no matches Args: {
"Expression": "^(menuconfig|config) FS_DAX",
"PathPrefix": "fs/dax/Kconfig"
}Results: {
"error": "no matches"
} |
| 35/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 40099 output: 30 thoughts: 10 Thoughts: Okay, here's my interpretation of that thought process, tailored for an expert audience, and written in the first person: **Locating the `FS_DAX` Key** Alright, the initial prompt is pretty straightforward. I'm essentially being asked to locate or identify something, and the immediate focus is on `FS_DAX`. Given my expertise, I'm already mentally running through the various contexts where this identifier is likely to appear. It's almost certainly related to file systems, possibly data access, and likely involves a specific implementation or abstraction layer. The immediate goal is clear: pinpoint what `FS_DAX` represents within the current context. I need to figure out the significance and intended use of this key. My approach would be to start building a mental map; what are the likely components this identifier references? Is it a configuration parameter, a macro, a symbolic constant, or maybe even a specific function name? The context is what's important here. The search begins. |
| 36/2 | 2026/07/17 22:48 | tool | grepper |
0mArgs:{
"Expression": "^(menuconfig|config) FS_DAX",
"PathPrefix": ""
}Results: {
"Output": "fs/Kconfig=56=endif # BLOCK\nfs/Kconfig-57-\nfs/Kconfig:58:config FS_DAX\nfs/Kconfig-59-\tbool \"File system based Direct Access (DAX) support\"\n--\nfs/Kconfig-89-\nfs/Kconfig:90:config FS_DAX_PMD\nfs/Kconfig-91-\tbool\n"
} |
| 37/2 | 2026/07/17 22:48 | llm | patch-evaluator |
0mModel:gemini-3.1-pro-preview Args: null Results: null Tokens: input: 40455 output: 247 thoughts: 400 Thoughts: **My Assessment of the Memory Management Patch** Okay, this looks like a solid patch. `FS_DAX` is a good sign – that's central to how this works. Let's start with a configuration summary: I'm seeing interactions with `KSM`, `USERFAULTFD`, `MEMORY_FAILURE`, `TRANSPARENT_HUGEPAGE`, `MIGRATION`, and, of course, `FS_DAX`. I skimmed for other potentially relevant configs, just in case. `DEVPORT` and `CONFIG_64BIT` don't seem directly connected to the changes, so I'll ignore those. Now, let's refine the "FocusSymbols" – the key function calls to pinpoint: * `mmap_zero_prepare`: That's the core of the `/dev/zero` mapping alteration, so it's a great choice. * `try_to_merge_with_ksm_page` and `rmap_walk_ksm`: I expect KSM interactions to be important to understand. * `remove_migration_pte`: Migration changes are likely related to how the mappings are handled. * `copy_vma_and_data`: That sounds like mremap is involved, so this needs to be investigated. * `move_present_ptes`: Sounds like it will relate to userfaultfd. * `relocate_vma_down`: This might be involved with execve, so worth looking at. This FocusSymbols list is really well-defined. I'm happy with this. The core of the patch is modifying the memory management to treat `MAP_PRIVATE` mappings of `/dev/zero` as *truly* anonymous memory, instead of file-backed. It's using virtual page offsets to track and merge these mappings. This touches a lot of other areas: KSM, migration, mremap, userfaultfd, and memory failure handling. The changes are all reachable from userspace, specifically through functions like `mmap`, `mremap`, `userfaultfd`, etc., which means fuzzing is going to be important. Okay, this has gone very well. Let's call `set-results`. |
| 38/2 | 2026/07/17 22:49 | tool | set-results |
0mArgs:{
"EnableConfigs": [
"KSM",
"USERFAULTFD",
"MEMORY_FAILURE",
"TRANSPARENT_HUGEPAGE",
"MIGRATION",
"FS_DAX"
],
"FocusSymbols": [
"mmap_zero_prepare",
"try_to_merge_with_ksm_page",
"rmap_walk_ksm",
"remove_migration_pte",
"copy_vma_and_data",
"move_present_ptes",
"relocate_vma_down"
],
"Reasoning": "The patch modifies the core memory management subsystem to handle MAP_PRIVATE mappings of /dev/zero as truly anonymous mappings, rather than file-backed mappings with a special flag. It introduces virtual page offsets for VMAs to correctly track and merge these mappings. This affects various mm subsystems including KSM, migration, mremap, userfaultfd, and memory failure handling. The changes are functional and reachable from userspace via mmap, mremap, userfaultfd, etc., so they should be fuzzed.",
"WorthFuzzing": true
}Results: {
"EnableConfigs": [
"KSM",
"USERFAULTFD",
"MEMORY_FAILURE",
"TRANSPARENT_HUGEPAGE",
"MIGRATION",
"FS_DAX"
],
"FocusSymbols": [
"mmap_zero_prepare",
"try_to_merge_with_ksm_page",
"rmap_walk_ksm",
"remove_migration_pte",
"copy_vma_and_data",
"move_present_ptes",
"relocate_vma_down"
],
"Reasoning": "The patch modifies the core memory management subsystem to handle MAP_PRIVATE mappings of /dev/zero as truly anonymous mappings, rather than file-backed mappings with a special flag. It introduces virtual page offsets for VMAs to correctly track and merge these mappings. This affects various mm subsystems including KSM, migration, mremap, userfaultfd, and memory failure handling. The changes are functional and reachable from userspace via mmap, mremap, userfaultfd, etc., so they should be fuzzed.",
"WorthFuzzing": true
} |
| Total Calls | Total Tokens | Avg Tokens | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|
| Total Calls | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|