| Seq | Timestamp | Type | Name | Duration |
|---|---|---|---|---|
| 0/0 | 2026/09/14 19:11 | flow | patch-triage |
4mArgs:null Results: {
"EnableConfigs": [
"CHR_DEV_SG",
"SCSI",
"USB_MON",
"USB",
"SND_PCM",
"HUGETLB_PAGE",
"USERFAULTFD"
],
"FocusSymbols": [
"map_discontig_kernel_pages_prepare",
"map_discontig_kernel_pages_complete",
"mmap_prepare_validate",
"mmap_hook_validate",
"mmap_action_prepare",
"mmap_action_complete",
"sg_mmap_prepare",
"mon_bin_mmap_prepare",
"snd_pcm_mmap_status"
],
"KMSANReasoning": "The patch series refactors VMA flag helpers (e.g. replacing is_vm_hugetlb_page() with vma_is_hugetlb(), introducing vma_is_kernel_owned(), vma_can_merge(), vma_is_persistent(), etc.), adds descriptor-based mmap validation (mmap_prepare_validate(), mmap_hook_validate()), introduces a new discontiguous kernel page mapping mechanism (struct discontig_kernel_page_state, struct discontig_kernel_page_ops), and migrates several drivers (cmt_speech, hfi1, sg, usb mon, snd_pcm) to the new mmap / discontig mapping infrastructure.\n\nKASAN vs KMSAN applicability:\n- KMSAN detects reads of uninitialized memory (stack, heap, or page allocations) and info-leaks to user space. In this patch series, no new kernel-to-user copy interfaces (copy_to_user, put_user, netlink, ioctls) are introduced or modified. All newly introduced structures, such as struct discontig_kernel_page_state in map_discontig_kernel_pages_complete() and prev_desc in __compat_vma_mmap(), are fully zero-initialized via designated initializers or memset() prior to use. The mapped pages originate from existing driver buffers or zero-allocated pages (e.g. __GFP_ZERO in sg).\n- Potential risks in these changes relate to VMA lifecycle, reference counting (fput/get_file), page table manipulations, locking, or out-of-bounds page array indexing. These types of bugs are covered by standard KASAN, page table debugging, and LOCKDEP builds.\n\nTherefore, a dedicated KMSAN fuzzing session is not warranted.",
"NeedsKMSAN": false,
"Reasoning": "The patch modifies core memory management (mm) and reachable drivers by introducing a new discontiguous kernel page mapping mechanism (map_discontig_kernel_pages_*), adding validation hooks for mmap and mmap_prepare (mmap_prepare_validate, mmap_hook_validate), refactoring VMA classifications across mm (vma_is_kernel_owned, vma_is_fixed_mapping, vma_can_merge, vma_is_persistent), altering mlock tracking logic during page table walks, and migrating reachable drivers (drivers/scsi/sg.c, drivers/usb/mon/mon_bin.c, sound/core/pcm_native.c) to these new APIs. These changes alter VMA invariant verification and mapping setup in reachable core code, warranting fuzzing for assertion failures and subtle mm regressions.",
"WorthFuzzing": true
} |
| 1/1 | 2026/09/14 19:11 | action | read-patch-diff |
0mArgs:null Results: {
"PatchDiff": "commit dfb57cd7fb217657931dcb005b6452b86d63b7db\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Mon Sep 14 19:11:06 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/Documentation/filesystems/mmap_prepare.rst b/Documentation/filesystems/mmap_prepare.rst\nindex 82c99c95ad854..a476e1006bf12 100644\n--- a/Documentation/filesystems/mmap_prepare.rst\n+++ b/Documentation/filesystems/mmap_prepare.rst\n@@ -164,5 +164,86 @@ pointer. These are:\n sufficient entries in the page array to cover the entire range of the\n described VMA.\n \n+* mmap_action_map_discontig_kernel_pages() - Maps a discontiguous range of\n+ `struct page` pointers over the VMA. They must span from the start of the VMA,\n+ but may terminate prior to the end (leaving the remainder unmapped).\n+\n **NOTE:** The ``action`` field should never normally be manipulated directly,\n rather you ought to use one of these helpers.\n+\n+Discontiguous Actions\n+=====================\n+\n+Some actions can be performed across discontiguous ranges.\n+\n+Map kernel pages\n+----------------\n+\n+To map kernel pages discontiguously, you must provide hooks using ``struct\n+discontig_kernel_page_ops``:\n+\n+.. code-block:: C\n+\n+ struct discontig_kernel_page_ops {\n+ int (*init)(void *vm_private_data, void **private);\n+ int (*get)(struct discontig_kernel_page_state *state);\n+ };\n+\n+The ``init`` hook is optional and allows state to be established before the\n+operation starts, for instance taking a reference count. Nothing is invoked\n+after the operation, so ``init`` must not leave locks held, and state that must\n+be released once the mapping goes away should be released in\n+``vm_ops-\u003eclose``.\n+\n+The ``init`` hook, if provided, is invoked prior to the operation starting. It\n+may update what is pointed to by ``vm_private_data`` and/or ``private``. If an\n+error is returned, then the operation is aborted. The ``private`` field can be\n+reassigned.\n+\n+**NOTE:** The operation may sleep between invocations of ``get``, so locks\n+needed to stabilise state must be taken and released within each hook.\n+\n+The ``get`` handler is the key means through which the operation is\n+executed. The current state of the operation is provided through ``struct\n+discontig_kernel_page_state``:\n+\n+.. code-block:: C\n+\n+ struct discontig_kernel_page_state {\n+ /* Map state. */\n+ unsigned long start; /* Start address of VMA. */\n+ unsigned long end; /* End address of VMA. */\n+ unsigned long addr; /* The current address to be mapped. */\n+ pgoff_t pgoff; /* The current pgoff to be mapped. */\n+ unsigned long nr_pages_mapped; /* The number of pages mapped. */\n+ unsigned long nr_pages_remain; /* The number of pages remaining. */\n+\n+ /* User-defined state. */\n+ void *vm_private_data; /* VMA private data. */\n+ void *private; /* Mapping private data. */\n+\n+ /* Users should not touch these, use discontig_kernel_map_*() helpers. */\n+ ... internal fields ...\n+ };\n+\n+With ``private`` being an additional user-controllable state variable,\n+initialised via ``mmap_action_map_discontig_kernel_pages()``, and\n+``vm_private_data`` being equal to the ``desc-\u003eprivate_data`` field set in\n+the ``mmap_prepare()`` hook.\n+\n+In the ``get`` hook, the user must choose how to map kernel pages:\n+\n+* ``discontig_kernel_map_abort()`` - Call this to abort the operation, whatever\n+ has been mapped so far will be retained, the rest of the mapping will SIGBUS\n+ if accessed.\n+* ``discontig_kernel_map_page()`` - Maps a single page, correctly handling\n+ compound pages (if the compound page is bigger than the remaining pages in the\n+ VMA, then only those pages that fit will be mapped). For a compound page, the\n+ head page must be passed.\n+* ``discontig_kernel_map_page_range()`` - Map an array of pages of a specified\n+ size. Note that if the number of pages specified exceeds the VMA size then an\n+ error will arise.\n+\n+If an error arises after ``init`` succeeded, the core unmaps the VMA, invoking\n+``vm_ops-\u003eclose`` if set, which is therefore the place to release any state\n+that ``init`` established.\ndiff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c\nindex 9ba86450fe4af..3c1240ffc38df 100644\n--- a/arch/arm64/kvm/mmu.c\n+++ b/arch/arm64/kvm/mmu.c\n@@ -1463,14 +1463,12 @@ static int get_vma_page_shift(struct vm_area_struct *vma, unsigned long hva)\n {\n \tunsigned long pa;\n \n-\tif (is_vm_hugetlb_page(vma) \u0026\u0026 !(vma-\u003evm_flags \u0026 VM_PFNMAP))\n+\tif (vma_is_hugetlb(vma))\n \t\treturn huge_page_shift(hstate_vma(vma));\n \n \tif (!(vma-\u003evm_flags \u0026 VM_PFNMAP))\n \t\treturn PAGE_SHIFT;\n \n-\tVM_BUG_ON(is_vm_hugetlb_page(vma));\n-\n \tpa = (vma-\u003evm_pgoff \u003c\u003c PAGE_SHIFT) + (hva - vma-\u003evm_start);\n \n #ifndef __PAGETABLE_PMD_FOLDED\ndiff --git a/arch/powerpc/mm/book3s64/radix_tlb.c b/arch/powerpc/mm/book3s64/radix_tlb.c\nindex 7de5760164a90..b4603a98224b3 100644\n--- a/arch/powerpc/mm/book3s64/radix_tlb.c\n+++ b/arch/powerpc/mm/book3s64/radix_tlb.c\n@@ -627,7 +627,7 @@ void radix__local_flush_tlb_page(struct vm_area_struct *vma, unsigned long vmadd\n {\n #ifdef CONFIG_HUGETLB_PAGE\n \t/* need the return fix for nohash.c */\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\treturn radix__local_flush_hugetlb_page(vma, vmaddr);\n #endif\n \tradix__local_flush_tlb_page_psize(vma-\u003evm_mm, vmaddr, mmu_virtual_psize);\n@@ -945,7 +945,7 @@ void radix__flush_tlb_page_psize(struct mm_struct *mm, unsigned long vmaddr,\n void radix__flush_tlb_page(struct vm_area_struct *vma, unsigned long vmaddr)\n {\n #ifdef CONFIG_HUGETLB_PAGE\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\treturn radix__flush_hugetlb_page(vma, vmaddr);\n #endif\n \tradix__flush_tlb_page_psize(vma-\u003evm_mm, vmaddr, mmu_virtual_psize);\n@@ -1113,7 +1113,7 @@ void radix__flush_tlb_range(struct vm_area_struct *vma, unsigned long start,\n \n {\n #ifdef CONFIG_HUGETLB_PAGE\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\treturn radix__flush_hugetlb_tlb_range(vma, start, end);\n #endif\n \ndiff --git a/arch/powerpc/mm/nohash/e500_hugetlbpage.c b/arch/powerpc/mm/nohash/e500_hugetlbpage.c\nindex a134d28a0e4d3..b87623f04be53 100644\n--- a/arch/powerpc/mm/nohash/e500_hugetlbpage.c\n+++ b/arch/powerpc/mm/nohash/e500_hugetlbpage.c\n@@ -180,7 +180,7 @@ book3e_hugetlb_preload(struct vm_area_struct *vma, unsigned long ea, pte_t pte)\n */\n void __update_mmu_cache(struct vm_area_struct *vma, unsigned long address, pte_t *ptep)\n {\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\tbook3e_hugetlb_preload(vma, address, *ptep);\n }\n \ndiff --git a/arch/powerpc/mm/nohash/tlb.c b/arch/powerpc/mm/nohash/tlb.c\nindex 0a650742f3a00..07a2db16c2b15 100644\n--- a/arch/powerpc/mm/nohash/tlb.c\n+++ b/arch/powerpc/mm/nohash/tlb.c\n@@ -278,7 +278,7 @@ void __flush_tlb_page(struct mm_struct *mm, unsigned long vmaddr,\n void flush_tlb_page(struct vm_area_struct *vma, unsigned long vmaddr)\n {\n #ifdef CONFIG_HUGETLB_PAGE\n-\tif (vma \u0026\u0026 is_vm_hugetlb_page(vma))\n+\tif (vma \u0026\u0026 vma_is_hugetlb(vma))\n \t\tflush_hugetlb_page(vma, vmaddr);\n #endif\n \ndiff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c\nindex 6035b5ec95039..5c5c77f98bf0f 100644\n--- a/arch/riscv/kvm/mmu.c\n+++ b/arch/riscv/kvm/mmu.c\n@@ -664,7 +664,7 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,\n \t\treturn -EFAULT;\n \t}\n \n-\tis_hugetlb = is_vm_hugetlb_page(vma);\n+\tis_hugetlb = vma_is_hugetlb(vma);\n \tif (is_hugetlb)\n \t\tvma_pageshift = huge_page_shift(hstate_vma(vma));\n \telse\ndiff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c\nindex 962db300a1665..a74a7d5258aa1 100644\n--- a/arch/riscv/mm/tlbflush.c\n+++ b/arch/riscv/mm/tlbflush.c\n@@ -149,7 +149,7 @@ void flush_tlb_range(struct vm_area_struct *vma, unsigned long start,\n {\n \tunsigned long stride_size;\n \n-\tif (!is_vm_hugetlb_page(vma)) {\n+\tif (!vma_is_hugetlb(vma)) {\n \t\tstride_size = PAGE_SIZE;\n \t} else {\n \t\tstride_size = huge_page_size(hstate_vma(vma));\ndiff --git a/arch/s390/mm/gmap_helpers.c b/arch/s390/mm/gmap_helpers.c\nindex ff63ffb1dbd29..3f6783b93e679 100644\n--- a/arch/s390/mm/gmap_helpers.c\n+++ b/arch/s390/mm/gmap_helpers.c\n@@ -102,7 +102,7 @@ __context_unsafe(/* pte_unmap_unlock() not instrumented */)\n \n \t/* Find the vm address for the guest address */\n \tvma = vma_lookup(mm, vmaddr);\n-\tif (!vma || is_vm_hugetlb_page(vma))\n+\tif (!vma || vma_is_hugetlb(vma))\n \t\treturn;\n \n \t/* Get pointer to the page table entry */\n@@ -139,7 +139,7 @@ void gmap_helper_discard(struct mm_struct *mm, unsigned long vmaddr, unsigned lo\n \t\tvma = find_vma_intersection(mm, vmaddr, end);\n \t\tif (!vma)\n \t\t\treturn;\n-\t\tif (!is_vm_hugetlb_page(vma))\n+\t\tif (!vma_is_hugetlb(vma))\n \t\t\tzap_vma_range(vma, vmaddr, min(end, vma-\u003evm_end) - vmaddr);\n \t\tvmaddr = vma-\u003evm_end;\n \t}\n@@ -247,7 +247,7 @@ static int __gmap_helper_unshare_zeropages(struct mm_struct *mm)\n \t\t * proof to catch unexpected zeropages in other mappings and\n \t\t * fail.\n \t\t */\n-\t\tif ((vma-\u003evm_flags \u0026 VM_PFNMAP) || is_vm_hugetlb_page(vma))\n+\t\tif ((vma-\u003evm_flags \u0026 VM_PFNMAP) || vma_is_hugetlb(vma))\n \t\t\tcontinue;\n \t\taddr = vma-\u003evm_start;\n \ndiff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c\nindex 103db4683b165..9bbccb5d23a8f 100644\n--- a/arch/sparc/mm/init_64.c\n+++ b/arch/sparc/mm/init_64.c\n@@ -413,7 +413,7 @@ void update_mmu_cache_range(struct vm_fault *vmf, struct vm_area_struct *vma,\n \tif (mm-\u003econtext.hugetlb_pte_count || mm-\u003econtext.thp_pte_count) {\n \t\tunsigned long hugepage_size = PAGE_SIZE;\n \n-\t\tif (is_vm_hugetlb_page(vma))\n+\t\tif (vma_is_hugetlb(vma))\n \t\t\thugepage_size = huge_page_size(hstate_vma(vma));\n \n \t\tif (hugepage_size \u003e= PUD_SIZE) {\ndiff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c\nindex 65a2de82ecd29..0f60c0d076b62 100644\n--- a/arch/x86/kernel/uprobes.c\n+++ b/arch/x86/kernel/uprobes.c\n@@ -715,7 +715,7 @@ static struct vm_area_struct *get_uprobe_trampoline(struct mm_struct *mm, unsign\n \n \t*new_mapping = true;\n \treturn _install_special_mapping(mm, vaddr, PAGE_SIZE,\n-\t\t\t\tVM_READ|VM_EXEC|VM_MAYEXEC|VM_MAYREAD|VM_IO,\n+\t\t\t\tVM_READ|VM_EXEC|VM_MAYEXEC|VM_MAYREAD|VM_MIXEDMAP,\n \t\t\t\t\u0026tramp_mapping);\n }\n \ndiff --git a/drivers/gpu/drm/drm_gpusvm.c b/drivers/gpu/drm/drm_gpusvm.c\nindex a93eee7ddb9e9..fab34fea99c2f 100644\n--- a/drivers/gpu/drm/drm_gpusvm.c\n+++ b/drivers/gpu/drm/drm_gpusvm.c\n@@ -9,9 +9,9 @@\n #include \u003clinux/dma-mapping.h\u003e\n #include \u003clinux/export.h\u003e\n #include \u003clinux/hmm.h\u003e\n-#include \u003clinux/hugetlb_inline.h\u003e\n #include \u003clinux/memremap.h\u003e\n #include \u003clinux/mm_types.h\u003e\n+#include \u003clinux/mm.h\u003e\n #include \u003clinux/slab.h\u003e\n \n #include \u003cdrm/drm_device.h\u003e\n@@ -1141,8 +1141,7 @@ drm_gpusvm_range_find_or_insert(struct drm_gpusvm *gpusvm,\n \t * limitations. If/when migrate_vma_* add more support, this logic will\n \t * have to change.\n \t */\n-\tmigrate_devmem = ctx-\u003edevmem_possible \u0026\u0026\n-\t\tvma_is_anonymous(vas) \u0026\u0026 !is_vm_hugetlb_page(vas);\n+\tmigrate_devmem = ctx-\u003edevmem_possible \u0026\u0026 vma_is_anonymous(vas);\n \n \tchunk_size = drm_gpusvm_range_chunk_size(gpusvm, notifier, vas,\n \t\t\t\t\t\t fault_addr, gpuva_start,\ndiff --git a/drivers/hsi/clients/cmt_speech.c b/drivers/hsi/clients/cmt_speech.c\nindex 7226677ebde7a..801697b74d4f8 100644\n--- a/drivers/hsi/clients/cmt_speech.c\n+++ b/drivers/hsi/clients/cmt_speech.c\n@@ -1084,22 +1084,6 @@ static void cs_hsi_stop(struct cs_hsi_iface *hi)\n \tkfree(hi);\n }\n \n-static vm_fault_t cs_char_vma_fault(struct vm_fault *vmf)\n-{\n-\tstruct cs_char *csdata = vmf-\u003evma-\u003evm_private_data;\n-\tstruct page *page;\n-\n-\tpage = virt_to_page((void *)csdata-\u003emmap_base);\n-\tget_page(page);\n-\tvmf-\u003epage = page;\n-\n-\treturn 0;\n-}\n-\n-static const struct vm_operations_struct cs_char_vm_ops = {\n-\t.fault\t= cs_char_vma_fault,\n-};\n-\n static int cs_char_fasync(int fd, struct file *file, int on)\n {\n \tstruct cs_char *csdata = file-\u003eprivate_data;\n@@ -1256,18 +1240,19 @@ static long cs_char_ioctl(struct file *file, unsigned int cmd,\n \treturn r;\n }\n \n-static int cs_char_mmap(struct file *file, struct vm_area_struct *vma)\n+static int cs_char_mmap_prepare(struct vm_area_desc *desc)\n {\n-\tif (vma-\u003evm_end \u003c vma-\u003evm_start)\n-\t\treturn -EINVAL;\n+\tstruct file *file = desc-\u003efile;\n+\tstruct cs_char *csdata = file-\u003eprivate_data;\n+\tstruct page **pages = (struct page **)\u0026desc-\u003eprivate_data;\n \n-\tif (vma_pages(vma) != 1)\n+\tif (vma_desc_pages(desc) != 1)\n \t\treturn -EINVAL;\n \n-\tvm_flags_set(vma, VM_IO | VM_DONTDUMP | VM_DONTEXPAND);\n-\tvma-\u003evm_ops = \u0026cs_char_vm_ops;\n-\tvma-\u003evm_private_data = file-\u003eprivate_data;\n+\tvma_desc_set_flags(desc, VMA_DONTDUMP_BIT, VMA_DONTEXPAND_BIT);\n \n+\t*pages = virt_to_page((void *)csdata-\u003emmap_base);\n+\tmmap_action_map_kernel_pages_full(desc, pages);\n \treturn 0;\n }\n \n@@ -1353,7 +1338,7 @@ static const struct file_operations cs_char_fops = {\n \t.write\t\t= cs_char_write,\n \t.poll\t\t= cs_char_poll,\n \t.unlocked_ioctl\t= cs_char_ioctl,\n-\t.mmap\t\t= cs_char_mmap,\n+\t.mmap_prepare\t= cs_char_mmap_prepare,\n \t.open\t\t= cs_char_open,\n \t.release\t= cs_char_release,\n \t.fasync\t\t= cs_char_fasync,\ndiff --git a/drivers/infiniband/hw/hfi1/file_ops.c b/drivers/infiniband/hw/hfi1/file_ops.c\nindex dc548e6802e24..b02d1f1dbb27b 100644\n--- a/drivers/infiniband/hw/hfi1/file_ops.c\n+++ b/drivers/infiniband/hw/hfi1/file_ops.c\n@@ -70,7 +70,6 @@ static int set_ctxt_pkey(struct hfi1_ctxtdata *uctxt, unsigned long arg);\n static int ctxt_reset(struct hfi1_ctxtdata *uctxt);\n static int manage_rcvq(struct hfi1_ctxtdata *uctxt, u16 subctxt,\n \t\t unsigned long arg);\n-static vm_fault_t vma_fault(struct vm_fault *vmf);\n static long hfi1_file_ioctl(struct file *fp, unsigned int cmd,\n \t\t\t unsigned long arg);\n \n@@ -85,10 +84,6 @@ static const struct file_operations hfi1_file_ops = {\n \t.llseek = noop_llseek,\n };\n \n-static const struct vm_operations_struct vm_ops = {\n-\t.fault = vma_fault,\n-};\n-\n /*\n * Types of memories mapped into user processes' space\n */\n@@ -304,13 +299,13 @@ static ssize_t hfi1_write_iter(struct kiocb *kiocb, struct iov_iter *from)\n \treturn reqs;\n }\n \n-static inline void mmap_cdbg(u16 ctxt, u8 subctxt, u8 type, u8 mapio, u8 vmf,\n+static inline void mmap_cdbg(u16 ctxt, u8 subctxt, u8 type, u8 mapio, u8 is_vmalloc,\n \t\t\t u64 memaddr, void *memvirt, dma_addr_t memdma,\n \t\t\t ssize_t memlen, struct vm_area_struct *vma)\n {\n \thfi1_cdbg(PROC,\n-\t\t \"%u:%u type:%u io/vf/dma:%d/%d/%d, addr:0x%llx, len:%lu(%lu), flags:0x%lx\",\n-\t\t ctxt, subctxt, type, mapio, vmf, !!memdma,\n+\t\t \"%u:%u type:%u io/vmalloc/dma:%d/%d/%d, addr:0x%llx, len:%lu(%lu), flags:0x%lx\",\n+\t\t ctxt, subctxt, type, mapio, is_vmalloc, !!memdma,\n \t\t memaddr ?: (u64)memvirt, memlen,\n \t\t vma-\u003evm_end - vma-\u003evm_start, vma-\u003evm_flags);\n }\n@@ -325,7 +320,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)\n \t\tmemaddr = 0;\n \tvoid *memvirt = NULL;\n \tdma_addr_t memdma = 0;\n-\tu8 subctxt, mapio = 0, vmf = 0, type;\n+\tu8 subctxt, mapio = 0, is_vmalloc = 0, type;\n \tssize_t memlen = 0;\n \tint ret = 0;\n \tu16 ctxt;\n@@ -347,7 +342,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)\n \t/*\n \t * vm_pgoff is used as a buffer selector cookie. Always mmap from\n \t * the beginning.\n-\t */ \n+\t */\n \tvma-\u003evm_pgoff = 0;\n \tflags = vma-\u003evm_flags;\n \n@@ -366,7 +361,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)\n \t\t */\n \t\tmemlen = PAGE_ALIGN(uctxt-\u003esc-\u003ecredits * PIO_BLOCK_SIZE);\n \t\tflags \u0026= ~VM_MAYREAD;\n-\t\tflags |= VM_DONTCOPY | VM_DONTEXPAND;\n+\t\tflags |= VM_DONTCOPY;\n \t\tvma-\u003evm_page_prot = pgprot_writecombine(vma-\u003evm_page_prot);\n \t\tmapio = 1;\n \t\tbreak;\n@@ -401,6 +396,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)\n \t\tmemlen = rcvhdrq_size(uctxt);\n \t\tmemvirt = uctxt-\u003ercvhdrq;\n \t\tmemdma = uctxt-\u003ercvhdrq_dma;\n+\t\tflags |= VM_DONTEXPAND;\n \t\tbreak;\n \tcase RCV_EGRBUF: {\n \t\tunsigned long vm_start_save;\n@@ -422,7 +418,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)\n \t\t\tret = -EPERM;\n \t\t\tgoto done;\n \t\t}\n-\t\tvm_flags_clear(vma, VM_MAYWRITE);\n+\t\tvm_flags_mod(vma, VM_DONTEXPAND, VM_MAYWRITE);\n \t\t/*\n \t\t * Mmap multiple separate allocations into a single vma. From\n \t\t * here, dma_mmap_coherent() calls dma_direct_mmap(), which\n@@ -438,7 +434,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)\n \t\t\tmemvirt = uctxt-\u003eegrbufs.buffers[i].addr;\n \t\t\tmemdma = uctxt-\u003eegrbufs.buffers[i].dma;\n \t\t\tvma-\u003evm_end += memlen;\n-\t\t\tmmap_cdbg(ctxt, subctxt, type, mapio, vmf, memaddr,\n+\t\t\tmmap_cdbg(ctxt, subctxt, type, mapio, is_vmalloc, memaddr,\n \t\t\t\t memvirt, memdma, memlen, vma);\n \t\t\tret = dma_mmap_coherent(\u0026dd-\u003epcidev-\u003edev, vma,\n \t\t\t\t\t\tmemvirt, memdma, memlen);\n@@ -467,7 +463,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)\n \t\t * user registers.\n \t\t */\n \t\tmemlen = PAGE_SIZE;\n-\t\tflags |= VM_DONTCOPY | VM_DONTEXPAND;\n+\t\tflags |= VM_DONTCOPY;\n \t\tvma-\u003evm_page_prot = pgprot_noncached(vma-\u003evm_page_prot);\n \t\tmapio = 1;\n \t\tbreak;\n@@ -476,15 +472,10 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)\n \t\t * Use the page where this context's flags are. User level\n \t\t * knows where it's own bitmap is within the page.\n \t\t */\n-\t\tmemaddr = (unsigned long)\n-\t\t\t(dd-\u003eevents + uctxt_offset(uctxt)) \u0026 PAGE_MASK;\n+\t\tmemvirt = dd-\u003eevents + uctxt_offset(uctxt);\n+\t\tmemvirt = (void *)(((uintptr_t)memvirt) \u0026 PAGE_MASK);\n \t\tmemlen = PAGE_SIZE;\n-\t\t/*\n-\t\t * v3.7 removes VM_RESERVED but the effect is kept by\n-\t\t * using VM_IO.\n-\t\t */\n-\t\tflags |= VM_IO | VM_DONTEXPAND;\n-\t\tvmf = 1;\n+\t\tis_vmalloc = 1;\n \t\tbreak;\n \tcase STATUS:\n \t\tif (flags \u0026 VM_WRITE) {\n@@ -493,7 +484,6 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)\n \t\t}\n \t\tmemaddr = kvirt_to_phys((void *)dd-\u003estatus);\n \t\tmemlen = PAGE_SIZE;\n-\t\tflags |= VM_IO | VM_DONTEXPAND;\n \t\tbreak;\n \tcase RTAIL:\n \t\tif (!HFI1_CAP_IS_USET(DMA_RTAIL)) {\n@@ -512,25 +502,23 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)\n \t\tmemvirt = (void *)hfi1_rcvhdrtail_kvaddr(uctxt);\n \t\tmemdma = uctxt-\u003ercvhdrqtailaddr_dma;\n \t\tflags \u0026= ~VM_MAYWRITE;\n+\t\tflags |= VM_DONTEXPAND;\n \t\tbreak;\n \tcase SUBCTXT_UREGS:\n-\t\tmemaddr = (u64)uctxt-\u003esubctxt_uregbase;\n+\t\tmemvirt = uctxt-\u003esubctxt_uregbase;\n \t\tmemlen = PAGE_SIZE;\n-\t\tflags |= VM_IO | VM_DONTEXPAND;\n-\t\tvmf = 1;\n+\t\tis_vmalloc = 1;\n \t\tbreak;\n \tcase SUBCTXT_RCV_HDRQ:\n-\t\tmemaddr = (u64)uctxt-\u003esubctxt_rcvhdr_base;\n+\t\tmemvirt = uctxt-\u003esubctxt_rcvhdr_base;\n \t\tmemlen = rcvhdrq_size(uctxt) * uctxt-\u003esubctxt_cnt;\n-\t\tflags |= VM_IO | VM_DONTEXPAND;\n-\t\tvmf = 1;\n+\t\tis_vmalloc = 1;\n \t\tbreak;\n \tcase SUBCTXT_EGRBUF:\n-\t\tmemaddr = (u64)uctxt-\u003esubctxt_rcvegrbuf;\n+\t\tmemvirt = uctxt-\u003esubctxt_rcvegrbuf;\n \t\tmemlen = uctxt-\u003eegrbufs.size * uctxt-\u003esubctxt_cnt;\n-\t\tflags |= VM_IO | VM_DONTEXPAND;\n \t\tflags \u0026= ~VM_MAYWRITE;\n-\t\tvmf = 1;\n+\t\tis_vmalloc = 1;\n \t\tbreak;\n \tcase SDMA_COMP: {\n \t\tstruct hfi1_user_sdma_comp_q *cq = fd-\u003ecq;\n@@ -539,10 +527,9 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)\n \t\t\tret = -EFAULT;\n \t\t\tgoto done;\n \t\t}\n-\t\tmemaddr = (u64)cq-\u003ecomps;\n+\t\tmemvirt = cq-\u003ecomps;\n \t\tmemlen = PAGE_ALIGN(sizeof(*cq-\u003ecomps) * cq-\u003enentries);\n-\t\tflags |= VM_IO | VM_DONTEXPAND;\n-\t\tvmf = 1;\n+\t\tis_vmalloc = 1;\n \t\tbreak;\n \t}\n \tdefault:\n@@ -559,12 +546,10 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)\n \t}\n \n \tvm_flags_reset(vma, flags);\n-\tmmap_cdbg(ctxt, subctxt, type, mapio, vmf, memaddr, memvirt, memdma, \n+\tmmap_cdbg(ctxt, subctxt, type, mapio, is_vmalloc, memaddr, memvirt, memdma,\n \t\t memlen, vma);\n-\tif (vmf) {\n-\t\tvma-\u003evm_pgoff = PFN_DOWN(memaddr);\n-\t\tvma-\u003evm_ops = \u0026vm_ops;\n-\t\tret = 0;\n+\tif (is_vmalloc) {\n+\t\tret = remap_vmalloc_range(vma, memvirt, 0);\n \t} else if (memdma) {\n \t\tret = dma_mmap_coherent(\u0026dd-\u003epcidev-\u003edev, vma,\n \t\t\t\t\tmemvirt, memdma, memlen);\n@@ -588,24 +573,6 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)\n \treturn ret;\n }\n \n-/*\n- * Local (non-chip) user memory is not mapped right away but as it is\n- * accessed by the user-level code.\n- */\n-static vm_fault_t vma_fault(struct vm_fault *vmf)\n-{\n-\tstruct page *page;\n-\n-\tpage = vmalloc_to_page((void *)(vmf-\u003epgoff \u003c\u003c PAGE_SHIFT));\n-\tif (!page)\n-\t\treturn VM_FAULT_SIGBUS;\n-\n-\tget_page(page);\n-\tvmf-\u003epage = page;\n-\n-\treturn 0;\n-}\n-\n static __poll_t hfi1_poll(struct file *fp, struct poll_table_struct *pt)\n {\n \tstruct hfi1_ctxtdata *uctxt;\ndiff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c\nindex 5408f002e6c01..3f9e08725602c 100644\n--- a/drivers/scsi/sg.c\n+++ b/drivers/scsi/sg.c\n@@ -1212,85 +1212,72 @@ sg_fasync(int fd, struct file *filp, int mode)\n \treturn fasync_helper(fd, filp, mode, \u0026sfp-\u003easync_qp);\n }\n \n-static vm_fault_t\n-sg_vma_fault(struct vm_fault *vmf)\n+static int sg_discontig_init(void *vm_private_data, void **private)\n {\n-\tstruct vm_area_struct *vma = vmf-\u003evma;\n-\tSg_fd *sfp;\n-\tunsigned long offset, len, sa;\n-\tSg_scatter_hold *rsv_schp;\n-\tint k, length;\n-\n-\tif ((NULL == vma) || (!(sfp = (Sg_fd *) vma-\u003evm_private_data)))\n-\t\treturn VM_FAULT_SIGBUS;\n-\trsv_schp = \u0026sfp-\u003ereserve;\n-\toffset = vmf-\u003epgoff \u003c\u003c PAGE_SHIFT;\n-\tif (offset \u003e= rsv_schp-\u003ebufflen)\n-\t\treturn VM_FAULT_SIGBUS;\n-\tSCSI_LOG_TIMEOUT(3, sg_printk(KERN_INFO, sfp-\u003eparentdp,\n-\t\t\t\t \"sg_vma_fault: offset=%lu, scatg=%d\\n\",\n-\t\t\t\t offset, rsv_schp-\u003ek_use_sg));\n-\tsa = vma-\u003evm_start;\n-\tlength = 1 \u003c\u003c (PAGE_SHIFT + rsv_schp-\u003epage_order);\n-\tfor (k = 0; k \u003c rsv_schp-\u003ek_use_sg \u0026\u0026 sa \u003c vma-\u003evm_end; k++) {\n-\t\tlen = vma-\u003evm_end - sa;\n-\t\tlen = (len \u003c length) ? len : length;\n-\t\tif (offset \u003c len) {\n-\t\t\tstruct page *page = rsv_schp-\u003epages[k] + (offset \u003e\u003e PAGE_SHIFT);\n-\t\t\tget_page(page);\t/* increment page count */\n-\t\t\tvmf-\u003epage = page;\n-\t\t\treturn 0; /* success */\n-\t\t}\n-\t\tsa += len;\n-\t\toffset -= len;\n+\tconst unsigned long req_sz = (unsigned long)*private;\n+\tSg_fd *sfp = vm_private_data;\n+\tSg_scatter_hold *rsv_schp = \u0026sfp-\u003ereserve;\n+\tint err = 0;\n+\n+\tmutex_lock(\u0026sfp-\u003ef_mutex);\n+\tif (req_sz \u003e rsv_schp-\u003ebufflen) {\n+\t\terr = -ENOMEM;\t/* cannot map more than reserved buffer */\n+\t\tgoto out;\n+\t}\n+\tsfp-\u003emmap_called = 1; /* Prevents changes to buffer size. */\n+out:\n+\tmutex_unlock(\u0026sfp-\u003ef_mutex);\n+\treturn err;\n+}\n+\n+static int\n+sg_discontig_get(struct discontig_kernel_page_state *state)\n+{\n+\tSg_fd *sfp = state-\u003evm_private_data;\n+\tSg_scatter_hold *rsv_schp = \u0026sfp-\u003ereserve;\n+\tconst unsigned int order = rsv_schp-\u003epage_order;\n+\tconst pgoff_t nr_pages = state-\u003enr_pages_mapped;\n+\n+\tif (nr_pages \u003e= (rsv_schp-\u003ebufflen \u003e\u003e PAGE_SHIFT)) {\n+\t\tdiscontig_kernel_map_abort(state);\n+\t\treturn 0;\n \t}\n \n-\treturn VM_FAULT_SIGBUS;\n+\tSCSI_LOG_TIMEOUT(3, sg_printk(KERN_INFO, sfp-\u003eparentdp,\n+\t\t\t\t \"%s: offset=%lu, scatg=%d\\n\", __func__,\n+\t\t\t\t nr_pages \u003c\u003c PAGE_SHIFT, rsv_schp-\u003ek_use_sg));\n+\n+\tdiscontig_kernel_map_page(state, rsv_schp-\u003epages[nr_pages \u003e\u003e order]);\n+\treturn 0;\n }\n \n-static const struct vm_operations_struct sg_mmap_vm_ops = {\n-\t.fault = sg_vma_fault,\n+static const struct discontig_kernel_page_ops sg_discontig_ops = {\n+\t.init = sg_discontig_init,\n+\t.get = sg_discontig_get,\n };\n \n static int\n-sg_mmap(struct file *filp, struct vm_area_struct *vma)\n+sg_mmap_prepare(struct vm_area_desc *desc)\n {\n-\tSg_fd *sfp;\n-\tunsigned long req_sz, len, sa;\n-\tSg_scatter_hold *rsv_schp;\n-\tint k, length;\n-\tint ret = 0;\n+\tSg_fd *sfp = desc-\u003efile-\u003eprivate_data;\n+\tconst unsigned long req_sz = vma_desc_size(desc);\n \n-\tif ((!filp) || (!vma) || (!(sfp = (Sg_fd *) filp-\u003eprivate_data)))\n+\tif (!sfp)\n \t\treturn -ENXIO;\n-\treq_sz = vma-\u003evm_end - vma-\u003evm_start;\n+\n \tSCSI_LOG_TIMEOUT(3, sg_printk(KERN_INFO, sfp-\u003eparentdp,\n \t\t\t\t \"sg_mmap starting, vm_start=%p, len=%d\\n\",\n-\t\t\t\t (void *) vma-\u003evm_start, (int) req_sz));\n-\tif (vma-\u003evm_pgoff)\n+\t\t\t\t (void *) desc-\u003estart, (int) req_sz));\n+\n+\tif (desc-\u003epgoff)\n \t\treturn -EINVAL;\t/* want no offset */\n-\trsv_schp = \u0026sfp-\u003ereserve;\n-\tmutex_lock(\u0026sfp-\u003ef_mutex);\n-\tif (req_sz \u003e rsv_schp-\u003ebufflen) {\n-\t\tret = -ENOMEM;\t/* cannot map more than reserved buffer */\n-\t\tgoto out;\n-\t}\n \n-\tsa = vma-\u003evm_start;\n-\tlength = 1 \u003c\u003c (PAGE_SHIFT + rsv_schp-\u003epage_order);\n-\tfor (k = 0; k \u003c rsv_schp-\u003ek_use_sg \u0026\u0026 sa \u003c vma-\u003evm_end; k++) {\n-\t\tlen = vma-\u003evm_end - sa;\n-\t\tlen = (len \u003c length) ? len : length;\n-\t\tsa += len;\n-\t}\n+\tvma_desc_set_flags(desc, VMA_DONTEXPAND_BIT, VMA_DONTDUMP_BIT);\n+\tdesc-\u003eprivate_data = sfp;\n \n-\tsfp-\u003emmap_called = 1;\n-\tvm_flags_set(vma, VM_IO | VM_DONTEXPAND | VM_DONTDUMP);\n-\tvma-\u003evm_private_data = sfp;\n-\tvma-\u003evm_ops = \u0026sg_mmap_vm_ops;\n-out:\n-\tmutex_unlock(\u0026sfp-\u003ef_mutex);\n-\treturn ret;\n+\tmmap_action_map_discontig_kernel_pages(desc, (void *)req_sz,\n+\t\t\t\t\t \u0026sg_discontig_ops);\n+\treturn 0;\n }\n \n static void\n@@ -1415,7 +1402,7 @@ static const struct file_operations sg_fops = {\n \t.unlocked_ioctl = sg_ioctl,\n \t.compat_ioctl = compat_ptr_ioctl,\n \t.open = sg_open,\n-\t.mmap = sg_mmap,\n+\t.mmap_prepare = sg_mmap_prepare,\n \t.release = sg_release,\n \t.fasync = sg_fasync,\n };\ndiff --git a/drivers/usb/mon/mon_bin.c b/drivers/usb/mon/mon_bin.c\nindex 687f6a8981f34..9d00b21a8153b 100644\n--- a/drivers/usb/mon/mon_bin.c\n+++ b/drivers/usb/mon/mon_bin.c\n@@ -1219,6 +1219,15 @@ mon_bin_poll(struct file *file, struct poll_table_struct *wait)\n \treturn mask;\n }\n \n+static void __mon_bin_vma_open(struct mon_reader_bin *rp)\n+{\n+\tunsigned long flags;\n+\n+\tspin_lock_irqsave(\u0026rp-\u003eb_lock, flags);\n+\trp-\u003emmap_active++;\n+\tspin_unlock_irqrestore(\u0026rp-\u003eb_lock, flags);\n+}\n+\n /*\n * open and close: just keep track of how many times the device is\n * mapped, to use the proper memory allocation function.\n@@ -1226,64 +1235,79 @@ mon_bin_poll(struct file *file, struct poll_table_struct *wait)\n static void mon_bin_vma_open(struct vm_area_struct *vma)\n {\n \tstruct mon_reader_bin *rp = vma-\u003evm_private_data;\n-\tunsigned long flags;\n \n-\tspin_lock_irqsave(\u0026rp-\u003eb_lock, flags);\n-\trp-\u003emmap_active++;\n-\tspin_unlock_irqrestore(\u0026rp-\u003eb_lock, flags);\n+\t__mon_bin_vma_open(rp);\n }\n \n-static void mon_bin_vma_close(struct vm_area_struct *vma)\n+static void __mon_bin_vma_close(struct mon_reader_bin *rp)\n {\n \tunsigned long flags;\n \n-\tstruct mon_reader_bin *rp = vma-\u003evm_private_data;\n \tspin_lock_irqsave(\u0026rp-\u003eb_lock, flags);\n \trp-\u003emmap_active--;\n \tspin_unlock_irqrestore(\u0026rp-\u003eb_lock, flags);\n }\n \n-/*\n- * Map ring pages to user space.\n- */\n-static vm_fault_t mon_bin_vma_fault(struct vm_fault *vmf)\n+static void mon_bin_vma_close(struct vm_area_struct *vma)\n {\n-\tstruct mon_reader_bin *rp = vmf-\u003evma-\u003evm_private_data;\n+\tstruct mon_reader_bin *rp = vma-\u003evm_private_data;\n+\n+\t__mon_bin_vma_close(rp);\n+}\n+\n+static const struct vm_operations_struct mon_bin_vm_ops = {\n+\t.open = mon_bin_vma_open,\n+\t.close = mon_bin_vma_close,\n+};\n+\n+static int mon_bin_discontig_init(void *vm_private_data, void **private)\n+{\n+\tstruct mon_reader_bin *rp = vm_private_data;\n+\n+\t/* Dropped by mon_bin_vma_close() on unmap, including on error. */\n+\t__mon_bin_vma_open(rp);\n+\treturn 0;\n+}\n+\n+static int mon_bin_discontig_get(struct discontig_kernel_page_state *state)\n+{\n+\tstruct mon_reader_bin *rp = state-\u003evm_private_data;\n \tunsigned long offset, chunk_idx;\n-\tstruct page *pageptr;\n \tunsigned long flags;\n \n \tspin_lock_irqsave(\u0026rp-\u003eb_lock, flags);\n-\toffset = vmf-\u003epgoff \u003c\u003c PAGE_SHIFT;\n+\n+\toffset = state-\u003epgoff \u003c\u003c PAGE_SHIFT;\n \tif (offset \u003e= rp-\u003eb_size) {\n \t\tspin_unlock_irqrestore(\u0026rp-\u003eb_lock, flags);\n-\t\treturn VM_FAULT_SIGBUS;\n+\t\tdiscontig_kernel_map_abort(state);\n+\t\treturn 0;\n \t}\n \tchunk_idx = offset / CHUNK_SIZE;\n-\tpageptr = rp-\u003eb_vec[chunk_idx].pg;\n-\tget_page(pageptr);\n-\tvmf-\u003epage = pageptr;\n+\tdiscontig_kernel_map_page(state, rp-\u003eb_vec[chunk_idx].pg);\n+\n \tspin_unlock_irqrestore(\u0026rp-\u003eb_lock, flags);\n \treturn 0;\n }\n \n-static const struct vm_operations_struct mon_bin_vm_ops = {\n-\t.open = mon_bin_vma_open,\n-\t.close = mon_bin_vma_close,\n-\t.fault = mon_bin_vma_fault,\n+static const struct discontig_kernel_page_ops mon_discontig_ops = {\n+\t.init = mon_bin_discontig_init,\n+\t.get = mon_bin_discontig_get,\n };\n \n-static int mon_bin_mmap(struct file *filp, struct vm_area_struct *vma)\n+static int mon_bin_mmap_prepare(struct vm_area_desc *desc)\n {\n-\t/* don't do anything here: \"fault\" will set up page table entries */\n-\tvma-\u003evm_ops = \u0026mon_bin_vm_ops;\n+\tconst struct file *filp = desc-\u003efile;\n \n-\tif (vma-\u003evm_flags \u0026 VM_WRITE)\n+\tif (vma_desc_test(desc, VMA_WRITE_BIT))\n \t\treturn -EPERM;\n \n-\tvm_flags_mod(vma, VM_DONTEXPAND | VM_DONTDUMP, VM_MAYWRITE);\n-\tvma-\u003evm_private_data = filp-\u003eprivate_data;\n-\tmon_bin_vma_open(vma);\n+\tdesc-\u003evm_ops = \u0026mon_bin_vm_ops;\n+\tvma_desc_clear_flags(desc, VMA_MAYWRITE_BIT);\n+\tvma_desc_set_flags(desc, VMA_DONTEXPAND_BIT, VMA_DONTDUMP_BIT);\n+\tdesc-\u003eprivate_data = filp-\u003eprivate_data;\n+\n+\tmmap_action_map_discontig_kernel_pages(desc, NULL, \u0026mon_discontig_ops);\n \treturn 0;\n }\n \n@@ -1298,7 +1322,7 @@ static const struct file_operations mon_fops_binary = {\n \t.compat_ioctl =\tmon_bin_compat_ioctl,\n #endif\n \t.release =\tmon_bin_release,\n-\t.mmap =\t\tmon_bin_mmap,\n+\t.mmap_prepare = mon_bin_mmap_prepare,\n };\n \n static int mon_bin_wait_event(struct file *file, struct mon_reader_bin *rp)\ndiff --git a/drivers/video/fbdev/core/fb_defio.c b/drivers/video/fbdev/core/fb_defio.c\nindex fd00b86e1ae60..fb359ecc39661 100644\n--- a/drivers/video/fbdev/core/fb_defio.c\n+++ b/drivers/video/fbdev/core/fb_defio.c\n@@ -366,13 +366,13 @@ int fb_deferred_io_mmap(struct fb_info *info, struct vm_area_struct *vma)\n {\n \tvma-\u003evm_page_prot = pgprot_decrypted(vma-\u003evm_page_prot);\n \n+\tif (WARN_ON_ONCE(!(info-\u003eflags \u0026 FBINFO_VIRTFB)))\n+\t\treturn -EINVAL;\n \tif (!try_module_get(THIS_MODULE))\n \t\treturn -EINVAL;\n \n \tvma-\u003evm_ops = \u0026fb_deferred_io_vm_ops;\n-\tvm_flags_set(vma, VM_DONTEXPAND | VM_DONTDUMP);\n-\tif (!(info-\u003eflags \u0026 FBINFO_VIRTFB))\n-\t\tvm_flags_set(vma, VM_IO);\n+\tvm_flags_set(vma, VM_MIXEDMAP | VM_DONTEXPAND | VM_DONTDUMP);\n \tvma-\u003evm_private_data = info-\u003efbdefio_state;\n \n \tfb_deferred_io_state_get(info-\u003efbdefio_state); /* released in vma-\u003evm_ops-\u003eclose() */\ndiff --git a/drivers/video/fbdev/ssd1307fb.c b/drivers/video/fbdev/ssd1307fb.c\nindex c4fdecafd8560..958514a354338 100644\n--- a/drivers/video/fbdev/ssd1307fb.c\n+++ b/drivers/video/fbdev/ssd1307fb.c\n@@ -763,6 +763,8 @@ static int ssd1307fb_probe(struct i2c_client *client)\n \tinfo-\u003efix.smem_start = __pa(vmem);\n \tinfo-\u003efix.smem_len = vmem_size;\n \n+\tinfo-\u003eflags = FBINFO_VIRTFB;\n+\n \tfb_deferred_io_init(info);\n \n \ti2c_set_clientdata(client, info);\ndiff --git a/fs/coredump.c b/fs/coredump.c\nindex ac3cd74808c64..9f729c594c47e 100644\n--- a/fs/coredump.c\n+++ b/fs/coredump.c\n@@ -1608,7 +1608,7 @@ static unsigned long vma_dump_size(struct vm_area_struct *vma,\n \t}\n \n \t/* Hugetlb memory check */\n-\tif (is_vm_hugetlb_page(vma)) {\n+\tif (vma_is_hugetlb(vma)) {\n \t\tif ((vma-\u003evm_flags \u0026 VM_SHARED) \u0026\u0026 FILTER(HUGETLB_SHARED))\n \t\t\tgoto whole;\n \t\tif (!(vma-\u003evm_flags \u0026 VM_SHARED) \u0026\u0026 FILTER(HUGETLB_PRIVATE))\n@@ -1616,8 +1616,8 @@ static unsigned long vma_dump_size(struct vm_area_struct *vma,\n \t\treturn 0;\n \t}\n \n-\t/* Do not dump I/O mapped devices or special mappings */\n-\tif (vma-\u003evm_flags \u0026 VM_IO)\n+\t/* Do not dump memory-mapped I/O, which may have side effects on read. */\n+\tif (vma_test(vma, VMA_IO_BIT))\n \t\treturn 0;\n \n \t/* By default, dump shared memory if mapped from an anonymous file. */\ndiff --git a/fs/fuse/dax.c b/fs/fuse/dax.c\nindex 85cdf0199bc0b..a5994f1c637d9 100644\n--- a/fs/fuse/dax.c\n+++ b/fs/fuse/dax.c\n@@ -826,7 +826,7 @@ int fuse_dax_mmap(struct file *file, struct vm_area_struct *vma)\n {\n \tfile_accessed(file);\n \tvma-\u003evm_ops = \u0026fuse_dax_vm_ops;\n-\tvm_flags_set(vma, VM_MIXEDMAP | VM_HUGEPAGE);\n+\tvma_set_flags(vma, VMA_HUGEPAGE_BIT);\n \treturn 0;\n }\n \ndiff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c\nindex 7611a8470ea26..ba7097d5720c0 100644\n--- a/fs/hugetlbfs/inode.c\n+++ b/fs/hugetlbfs/inode.c\n@@ -108,7 +108,7 @@ static int hugetlbfs_file_mmap(struct file *file, struct vm_area_struct *vma)\n \t * vma address alignment (but not the pgoff alignment) has\n \t * already been checked by prepare_hugepage_range. If you add\n \t * any error returns here, do so after setting VM_HUGETLB, so\n-\t * is_vm_hugetlb_page tests below unmap_region go the right\n+\t * vma_is_hugetlb tests below unmap_region go the right\n \t * way when do_mmap unwinds (may be important on powerpc\n \t * and ia64).\n \t */\ndiff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c\nindex e671b4fd8dedd..565e6446bd312 100644\n--- a/fs/proc/task_mmu.c\n+++ b/fs/proc/task_mmu.c\n@@ -3015,7 +3015,7 @@ static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end,\n \t * hugetlb differs, see pagemap_hugetlb_category().\n \t */\n \tcategories = p-\u003ecur_vma_category;\n-\tif (userfaultfd_wp(vma) \u0026\u0026 !is_vm_hugetlb_page(vma))\n+\tif (userfaultfd_wp(vma) \u0026\u0026 !vma_is_hugetlb(vma))\n \t\tcategories |= PAGE_IS_WRITTEN;\n \n \tif (!pagemap_scan_is_interesting_page(categories, p))\n@@ -3028,7 +3028,7 @@ static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end,\n \tif (~p-\u003earg.flags \u0026 PM_SCAN_WP_MATCHING)\n \t\treturn ret;\n \n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\terr = pagemap_scan_hugetlb_hole_wp(vma, addr, end);\n \telse\n \t\terr = uffd_wp_range(vma, addr, end - addr, true);\n@@ -3470,7 +3470,7 @@ static int show_numa_map(struct seq_file *m, void *v)\n \t\tseq_puts(m, \" stack\");\n \t}\n \n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\tseq_puts(m, \" huge\");\n \n \t/* Skip walking pages if gate VMA */\n@@ -3499,7 +3499,7 @@ static int show_numa_map(struct seq_file *m, void *v)\n \tif (md-\u003eswapcache)\n \t\tseq_printf(m, \" swapcache=%lu\", md-\u003eswapcache);\n \n-\tif (md-\u003eactive \u003c md-\u003epages \u0026\u0026 !is_vm_hugetlb_page(vma))\n+\tif (md-\u003eactive \u003c md-\u003epages \u0026\u0026 !vma_is_hugetlb(vma))\n \t\tseq_printf(m, \" active=%lu\", md-\u003eactive);\n \n \tif (md-\u003ewriteback)\ndiff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h\nindex 044dabc1fe9cb..48d47b34cc777 100644\n--- a/include/asm-generic/tlb.h\n+++ b/include/asm-generic/tlb.h\n@@ -11,9 +11,9 @@\n #ifndef _ASM_GENERIC__TLB_H\n #define _ASM_GENERIC__TLB_H\n \n+#include \u003clinux/mm.h\u003e\n #include \u003clinux/mmu_notifier.h\u003e\n #include \u003clinux/swap.h\u003e\n-#include \u003clinux/hugetlb_inline.h\u003e\n #include \u003casm/tlbflush.h\u003e\n #include \u003casm/cacheflush.h\u003e\n \n@@ -438,7 +438,7 @@ tlb_update_vma_flags(struct mmu_gather *tlb, struct vm_area_struct *vma)\n \t * We rely on tlb_end_vma() to issue a flush, such that when we reset\n \t * these values the batch is empty.\n \t */\n-\ttlb-\u003evma_huge = is_vm_hugetlb_page(vma);\n+\ttlb-\u003evma_huge = vma_is_hugetlb(vma);\n \ttlb-\u003evma_exec = !!(vma-\u003evm_flags \u0026 VM_EXEC);\n \n \t/*\ndiff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h\nindex 80a5a03e9cee7..24727ece20fe5 100644\n--- a/include/linux/hugetlb.h\n+++ b/include/linux/hugetlb.h\n@@ -7,7 +7,6 @@\n #include \u003clinux/mm_types.h\u003e\n #include \u003clinux/mmdebug.h\u003e\n #include \u003clinux/fs.h\u003e\n-#include \u003clinux/hugetlb_inline.h\u003e\n #include \u003clinux/cgroup.h\u003e\n #include \u003clinux/page_ref.h\u003e\n #include \u003clinux/list.h\u003e\n@@ -252,14 +251,14 @@ extern void __hugetlb_zap_end(struct vm_area_struct *vma,\n static inline void hugetlb_zap_begin(struct vm_area_struct *vma,\n \t\t\t\t unsigned long *start, unsigned long *end)\n {\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\t__hugetlb_zap_begin(vma, start, end);\n }\n \n static inline void hugetlb_zap_end(struct vm_area_struct *vma,\n \t\t\t\t struct zap_details *details)\n {\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\t__hugetlb_zap_end(vma, details);\n }\n \ndiff --git a/include/linux/hugetlb_inline.h b/include/linux/hugetlb_inline.h\ndeleted file mode 100644\nindex 5c29cd3223a1e..0000000000000\n--- a/include/linux/hugetlb_inline.h\n+++ /dev/null\n@@ -1,28 +0,0 @@\n-/* SPDX-License-Identifier: GPL-2.0 */\n-#ifndef _LINUX_HUGETLB_INLINE_H\n-#define _LINUX_HUGETLB_INLINE_H\n-\n-#include \u003clinux/mm.h\u003e\n-\n-#ifdef CONFIG_HUGETLB_PAGE\n-\n-static inline bool is_vma_hugetlb_flags(const vma_flags_t *flags)\n-{\n-\treturn vma_flags_test(flags, VMA_HUGETLB_BIT);\n-}\n-\n-#else\n-\n-static inline bool is_vma_hugetlb_flags(const vma_flags_t *flags)\n-{\n-\treturn false;\n-}\n-\n-#endif\n-\n-static inline bool is_vm_hugetlb_page(const struct vm_area_struct *vma)\n-{\n-\treturn is_vma_hugetlb_flags(\u0026vma-\u003eflags);\n-}\n-\n-#endif\ndiff --git a/include/linux/mm.h b/include/linux/mm.h\nindex 969594074fd2d..1249e04d7b980 100644\n--- a/include/linux/mm.h\n+++ b/include/linux/mm.h\n@@ -576,14 +576,6 @@ enum {\n #define VM_ACCESS_FLAGS (VM_READ | VM_WRITE | VM_EXEC)\n #define VMA_ACCESS_FLAGS mk_vma_flags(VMA_READ_BIT, VMA_WRITE_BIT, VMA_EXEC_BIT)\n \n-/*\n- * Special vmas that are non-mergable, non-mlock()able.\n- */\n-\n-#define VMA_SPECIAL_FLAGS mk_vma_flags(VMA_IO_BIT, VMA_DONTEXPAND_BIT, \\\n-\t\t\t\t VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT)\n-#define VM_SPECIAL vma_flags_to_legacy(VMA_SPECIAL_FLAGS)\n-\n /*\n * Physically remapped pages are special. Tell the\n * rest of the world about it:\n@@ -600,9 +592,6 @@ enum {\n #define VMA_REMAP_FLAGS mk_vma_flags(VMA_IO_BIT, VMA_PFNMAP_BIT,\t\\\n \t\t\t\t VMA_DONTEXPAND_BIT, VMA_DONTDUMP_BIT)\n \n-/* This mask prevents VMA from being scanned with khugepaged */\n-#define VM_NO_KHUGEPAGED (VM_SPECIAL | VM_HUGETLB)\n-\n /* This mask defines which mm-\u003edef_flags a process can inherit its parent */\n #define VM_INIT_DEF_MASK\tVM_NOHUGEPAGE\n \n@@ -1612,6 +1601,211 @@ static inline bool vma_is_shared_maywrite(const struct vm_area_struct *vma)\n \treturn is_shared_maywrite(\u0026vma-\u003eflags);\n }\n \n+/**\n+ * vma_flags_is_hugetlb() - Do the specified VMA flags indicate that the\n+ * VMA is a hugetlb mapping?\n+ * @flags: The VMA flags to test.\n+ *\n+ * Returns: true if the flags indicate a hugetlb mapping, false otherwise.\n+ */\n+static inline bool vma_flags_is_hugetlb(const vma_flags_t *flags)\n+{\n+\treturn IS_ENABLED(CONFIG_HUGETLB_PAGE) \u0026\u0026\n+\t vma_flags_test(flags, VMA_HUGETLB_BIT);\n+}\n+\n+/**\n+ * vma_is_hugetlb() - Is @vma a hugetlb mapping?\n+ * @vma: The VMA to test.\n+ *\n+ * Returns: true if @vma is a hugetlb mapping, false otherwise.\n+ */\n+static inline bool vma_is_hugetlb(const struct vm_area_struct *vma)\n+{\n+\treturn vma_flags_is_hugetlb(\u0026vma-\u003eflags);\n+}\n+\n+/**\n+ * vma_flags_is_kernel_owned() - Do the specified VMA flags indicate that the\n+ * contents of the VMA are owned by the kernel rather than the core mm?\n+ * @flags: The VMA flags to test.\n+ *\n+ * A kernel-owned mapping is one whose contents are established and controlled\n+ * by the kernel, typically a driver, rather than by the core mm's fault and\n+ * rmap machinery.\n+ *\n+ * The mapping may be memory-mapped I/O, kernel-allocated pages or ordinary\n+ * pages the owner has chosen to map itself (shmem via a PFN map, for instance).\n+ *\n+ * In all cases the core mm must not populate, reclaim, migrate, copy-on-write\n+ * or merge it of its own accord.\n+ *\n+ * Pages mapped this way are not necessarily reference counted or map counted.\n+ *\n+ * Returns: true if the flags indicate a kernel-owned mapping.\n+ */\n+static inline bool vma_flags_is_kernel_owned(const vma_flags_t *flags)\n+{\n+\treturn vma_flags_test_any(flags, VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT);\n+}\n+\n+/**\n+ * vma_is_kernel_owned() - Are the contents of @vma owned by the kernel?\n+ * @vma: The VMA to test.\n+ *\n+ * See vma_flags_is_kernel_owned() for a description of this property.\n+ *\n+ * Returns: true if the VMA is kernel-owned.\n+ */\n+static inline bool vma_is_kernel_owned(const struct vm_area_struct *vma)\n+{\n+\treturn vma_flags_is_kernel_owned(\u0026vma-\u003eflags);\n+}\n+\n+/**\n+ * vma_flags_is_fixed_mapping() - Do the specified VMA flags indicate that this\n+ * is a fixed mapping that cannot be expanded or merged?\n+ * @flags: The VMA flags to test.\n+ *\n+ * Fixed mappings are those whose size is set at the point of mmap (for\n+ * instance, a kernel-owned mapping of a fixed range of memory), and thus\n+ * cannot be expanded or merged.\n+ *\n+ * Returns: true if the flags indicate a fixed mapping.\n+ */\n+static inline bool vma_flags_is_fixed_mapping(const vma_flags_t *flags)\n+{\n+\t/*\n+\t * VMA_PFNMAP_BIT should imply VMA_DONTEXPAND_BIT, but some callers set\n+\t * only the former.\n+\t */\n+\treturn vma_flags_test_any(flags, VMA_PFNMAP_BIT, VMA_DONTEXPAND_BIT);\n+}\n+\n+/**\n+ * vma_is_fixed_mapping() - Is this VMA a fixed mapping that cannot be\n+ * expanded or merged?\n+ * @vma: The VMA to test.\n+ *\n+ * See vma_flags_is_fixed_mapping() for a description of this property.\n+ *\n+ * Returns: true if the VMA maps a fixed mapping.\n+ */\n+static inline bool vma_is_fixed_mapping(const struct vm_area_struct *vma)\n+{\n+\treturn vma_flags_is_fixed_mapping(\u0026vma-\u003eflags);\n+}\n+\n+/**\n+ * vma_flags_can_merge() - Do the specified VMA flags permit the VMA to be\n+ * merged with another?\n+ * @flags: The VMA flags to test.\n+ * Returns: true if the flags permit merging, false otherwise.\n+ */\n+static inline bool vma_flags_can_merge(const vma_flags_t *flags)\n+{\n+\t/*\n+\t * VMA merging assumes that a VMA's flags and fields completely describe\n+\t * its state.\n+\t *\n+\t * However, kernel-owned mappings may have established state upon mapping\n+\t * not embodied in any attribute of the VMA.\n+\t *\n+\t * Additionally, private (CoW) PFN maps encode the source PFN of the\n+\t * range in vma-\u003evm_pgoff, which may otherwise cause spurious merges.\n+\t */\n+\tif (vma_flags_is_kernel_owned(flags))\n+\t\treturn false;\n+\t/* VMA explicitly marked as being unmergeable. */\n+\tif (vma_flags_is_fixed_mapping(flags))\n+\t\treturn false;\n+\n+\treturn true;\n+}\n+\n+/**\n+ * vma_can_merge() - Do @vma's flags permit it to be merged with another VMA?\n+ * @vma: The VMA to test.\n+ * Returns: true if the flags permit merging, otherwise false.\n+ */\n+static inline bool vma_can_merge(const struct vm_area_struct *vma)\n+{\n+\treturn vma_flags_can_merge(\u0026vma-\u003eflags);\n+}\n+\n+/**\n+ * vma_flags_is_persistent() - Do the specified VMA flags imply that the VMA\n+ * contains persistent data?\n+ * @flags: The VMA flags to test.\n+ *\n+ * Persistent in the sense that - if you write bytes to the mapping - do they\n+ * stay written?\n+ *\n+ * If the kernel or a device could write to the memory independently of\n+ * userland, or the kernel could arbitrarily discard it, then it is not\n+ * persistent.\n+ *\n+ * Returns: true if the flags imply this VMA is persistent, otherwise false.\n+ */\n+static inline bool vma_flags_is_persistent(const vma_flags_t *flags)\n+{\n+\t/* hugetlb is a fixed mapping, but its contents are the user's own. */\n+\tif (vma_flags_is_hugetlb(flags))\n+\t\treturn true;\n+\t/*\n+\t * MMIO mappings may not store what is written and may be changed by the\n+\t * device. Kernel-owned and fixed mappings may be changed by their owner\n+\t * without the user having initiated it.\n+\t */\n+\tif (vma_flags_is_kernel_owned(flags) ||\n+\t vma_flags_is_fixed_mapping(flags))\n+\t\treturn false;\n+\t/* Droppable memory is discardable by definition. */\n+\treturn !vma_flags_test_single_mask(flags, VMA_DROPPABLE);\n+}\n+\n+/**\n+ * vma_is_persistent() - Does the VMA contain persistent data?\n+ * @vma: The VMA to test.\n+ *\n+ * See vma_flags_is_persistent() for details.\n+ *\n+ * Returns: true if the VMA is persistent, otherwise false.\n+ */\n+static inline bool vma_is_persistent(const struct vm_area_struct *vma)\n+{\n+\treturn vma_flags_is_persistent(\u0026vma-\u003eflags);\n+}\n+\n+/**\n+ * vma_flags_can_gup() - Do the specified VMA flags permit GUP to access the\n+ * mapping's pages?\n+ * @flags: The VMA flags to test.\n+ *\n+ * GUP cannot obtain pages from a PFN map (VMA_PFNMAP_BIT), which may have no\n+ * struct pages behind it, and must not provide access to memory-mapped I/O\n+ * (VMA_IO_BIT).\n+ *\n+ * Returns: true if GUP may access pages from the mapping, otherwise false.\n+ */\n+static inline bool vma_flags_can_gup(const vma_flags_t *flags)\n+{\n+\treturn !vma_flags_test_any(flags, VMA_IO_BIT, VMA_PFNMAP_BIT);\n+}\n+\n+/**\n+ * vma_can_gup() - May GUP obtain pages from @vma?\n+ * @vma: The VMA to test.\n+ *\n+ * See vma_flags_can_gup() for details.\n+ *\n+ * Returns: true if GUP may access pages from the mapping, otherwise false.\n+ */\n+static inline bool vma_can_gup(const struct vm_area_struct *vma)\n+{\n+\treturn vma_flags_can_gup(\u0026vma-\u003eflags);\n+}\n+\n /**\n * vma_kernel_pagesize - Default page size granularity for this VMA.\n * @vma: The user mapping.\n@@ -4602,7 +4796,7 @@ static inline void mmap_action_map_kernel_pages(struct vm_area_desc *desc,\n {\n \tstruct mmap_action *action = \u0026desc-\u003eaction;\n \n-\taction-\u003etype = MMAP_MAP_KERNEL_PAGES;\n+\taction-\u003etype = MMAP_KERNEL_PAGES;\n \taction-\u003emap_kernel.start = start;\n \taction-\u003emap_kernel.pages = pages;\n \taction-\u003emap_kernel.nr_pages = nr_pages;\n@@ -4626,10 +4820,55 @@ static inline void mmap_action_map_kernel_pages_full(struct vm_area_desc *desc,\n \t\t\t\t vma_desc_pages(desc));\n }\n \n+static inline\n+void mmap_action_map_discontig_kernel_pages(struct vm_area_desc *desc,\n+\t\tvoid *init_private, const struct discontig_kernel_page_ops *ops)\n+{\n+\tstruct mmap_action *action = \u0026desc-\u003eaction;\n+\n+\taction-\u003etype = MMAP_DISCONTIG_KERNEL_PAGES;\n+\taction-\u003emap_kernel_discontig.init_private = init_private;\n+\taction-\u003emap_kernel_discontig.ops = ops;\n+}\n+\n int mmap_action_prepare(struct vm_area_desc *desc);\n int mmap_action_complete(struct vm_area_struct *vma,\n \t\t\t struct mmap_action *action, bool is_compat);\n \n+static inline void\n+discontig_kernel_map_abort(struct discontig_kernel_page_state *state)\n+{\n+\tstate-\u003eaction = DISCONTIG_KERNEL_PAGE_ABORT;\n+}\n+\n+static inline void\n+discontig_kernel_map_page(struct discontig_kernel_page_state *state,\n+\t\t\t struct page *page)\n+{\n+\tstruct folio *folio = page_folio(page);\n+\n+\tif (folio_test_large(folio)) {\n+\t\tVM_WARN_ON_ONCE(page != folio_page(folio, 0));\n+\t\tstate-\u003eaction = DISCONTIG_KERNEL_PAGE_MAP_COMPOUND_PAGE;\n+\t\tstate-\u003e__folio = folio;\n+\t\tstate-\u003e__nr_pages = min(state-\u003enr_pages_remain,\n+\t\t\t\t\tfolio_nr_pages(folio));\n+\t} else {\n+\t\tstate-\u003eaction = DISCONTIG_KERNEL_PAGE_MAP_PAGE;\n+\t\tstate-\u003e__page = page;\n+\t\tstate-\u003e__nr_pages = 1;\n+\t}\n+}\n+\n+static inline void\n+discontig_kernel_map_page_range(struct discontig_kernel_page_state *state,\n+\t\t\t\tstruct page **page_arr, unsigned long nr_pages)\n+{\n+\tstate-\u003eaction = DISCONTIG_KERNEL_PAGE_MAP_PAGE_RANGE;\n+\tstate-\u003e__page_arr = page_arr;\n+\tstate-\u003e__nr_pages = nr_pages;\n+}\n+\n /* Look up the first VMA which exactly match the interval vm_start ... vm_end */\n static inline struct vm_area_struct *find_exact_vma(struct mm_struct *mm,\n \t\t\t\tunsigned long vm_start, unsigned long vm_end)\n@@ -4747,9 +4986,6 @@ int remap_pfn_range(struct vm_area_struct *vma, unsigned long addr,\n int vm_insert_page(struct vm_area_struct *, unsigned long addr, struct page *);\n int vm_insert_pages(struct vm_area_struct *vma, unsigned long addr,\n \t\t\tstruct page **pages, unsigned long *num);\n-int map_kernel_pages_prepare(struct vm_area_desc *desc);\n-int map_kernel_pages_complete(struct vm_area_struct *vma,\n-\t\t\t struct mmap_action *action);\n int vm_map_pages(struct vm_area_struct *vma, struct page **pages,\n \t\t\t\tunsigned long num);\n int vm_map_pages_zero(struct vm_area_struct *vma, struct page **pages,\ndiff --git a/include/linux/mm_types.h b/include/linux/mm_types.h\nindex 5413bd10fff2c..0cb4f96039568 100644\n--- a/include/linux/mm_types.h\n+++ b/include/linux/mm_types.h\n@@ -815,11 +815,47 @@ struct pfnmap_track_ctx {\n \n /* What action should be taken after an .mmap_prepare call is complete? */\n enum mmap_action_type {\n-\tMMAP_NOTHING,\t\t/* Mapping is complete, no further action. */\n-\tMMAP_REMAP_PFN,\t\t/* Remap PFN range. */\n-\tMMAP_IO_REMAP_PFN,\t/* I/O remap PFN range. */\n-\tMMAP_SIMPLE_IO_REMAP,\t/* I/O remap with guardrails. */\n-\tMMAP_MAP_KERNEL_PAGES,\t/* Map kernel page range from array. */\n+\tMMAP_NOTHING,\n+\tMMAP_REMAP_PFN,\n+\tMMAP_IO_REMAP_PFN,\n+\tMMAP_SIMPLE_IO_REMAP,\t\t/* I/O remap with guardrails. */\n+\tMMAP_KERNEL_PAGES,\t\t/* Map kernel page range from array. */\n+\tMMAP_DISCONTIG_KERNEL_PAGES,\t/* Map kernel discontig page range. */\n+};\n+\n+enum discontig_kernel_page_action {\n+\tDISCONTIG_KERNEL_PAGE_ABORT,\n+\tDISCONTIG_KERNEL_PAGE_MAP_PAGE,\n+\tDISCONTIG_KERNEL_PAGE_MAP_COMPOUND_PAGE,\n+\tDISCONTIG_KERNEL_PAGE_MAP_PAGE_RANGE,\n+};\n+\n+struct discontig_kernel_page_state {\n+\t/* Map state. */\n+\tconst unsigned long start;\t/* Start address of VMA. */\n+\tconst unsigned long end;\t/* End address of VMA. */\n+\tunsigned long addr;\t\t/* The current address to be mapped. */\n+\tpgoff_t pgoff;\t\t\t/* The current pgoff to be mapped. */\n+\tunsigned long nr_pages_mapped;\t/* The number of pages mapped. */\n+\tunsigned long nr_pages_remain;\t/* The number of pages remaining. */\n+\n+\t/* User-defined state. */\n+\tvoid *vm_private_data;\t\t/* VMA private data. */\n+\tvoid *private;\t\t\t/* Mapping private data. */\n+\n+\t/* Users should not touch these, use discontig_kernel_map_*() helpers. */\n+\tenum discontig_kernel_page_action action;\n+\tunion {\n+\t\tstruct page *__page;\n+\t\tstruct folio *__folio;\n+\t\tstruct page **__page_arr;\n+\t};\n+\tunsigned long __nr_pages;\n+};\n+\n+struct discontig_kernel_page_ops {\n+\tint (*init)(void *vm_private_data, void **private);\n+\tint (*get)(struct discontig_kernel_page_state *state);\n };\n \n /*\n@@ -844,6 +880,10 @@ struct mmap_action {\n \t\t\tunsigned long nr_pages;\n \t\t\tpgoff_t pgoff;\n \t\t} map_kernel;\n+\t\tstruct {\n+\t\t\tvoid *init_private;\n+\t\t\tconst struct discontig_kernel_page_ops *ops;\n+\t\t} map_kernel_discontig;\n \t};\n \tenum mmap_action_type type;\n \ndiff --git a/include/linux/pagemap.h b/include/linux/pagemap.h\nindex 939f3a5e973f6..d7d8b312466c2 100644\n--- a/include/linux/pagemap.h\n+++ b/include/linux/pagemap.h\n@@ -14,7 +14,6 @@\n #include \u003clinux/gfp.h\u003e\n #include \u003clinux/bitops.h\u003e\n #include \u003clinux/hardirq.h\u003e /* for in_interrupt() */\n-#include \u003clinux/hugetlb_inline.h\u003e\n \n struct folio_batch;\n \ndiff --git a/include/linux/rmap.h b/include/linux/rmap.h\nindex 0b332770abeed..74cca0e3c7264 100644\n--- a/include/linux/rmap.h\n+++ b/include/linux/rmap.h\n@@ -888,7 +888,7 @@ struct page_vma_mapped_walk {\n static inline void page_vma_mapped_walk_done(struct page_vma_mapped_walk *pvmw)\n {\n \t/* HugeTLB pte is set to the relevant page table entry without pte_mapped. */\n-\tif (pvmw-\u003epte \u0026\u0026 !is_vm_hugetlb_page(pvmw-\u003evma))\n+\tif (pvmw-\u003epte \u0026\u0026 !vma_is_hugetlb(pvmw-\u003evma))\n \t\tpte_unmap(pvmw-\u003epte);\n \tif (pvmw-\u003eptl)\n \t\tspin_unlock(pvmw-\u003eptl);\ndiff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h\nindex a4351cffc60ce..a14b8a9ffb7b1 100644\n--- a/include/linux/userfaultfd_k.h\n+++ b/include/linux/userfaultfd_k.h\n@@ -18,7 +18,6 @@\n #include \u003clinux/swap.h\u003e\n #include \u003clinux/leafops.h\u003e\n #include \u003casm-generic/pgtable_uffd.h\u003e\n-#include \u003clinux/hugetlb_inline.h\u003e\n \n /* The set of all possible UFFD-related VM flags. */\n #define __VM_UFFD_FLAGS (VM_UFFD_MISSING | VM_UFFD_MINOR | \\\ndiff --git a/kernel/bpf/arena.c b/kernel/bpf/arena.c\nindex 7b6847200b431..b69fe5e343393 100644\n--- a/kernel/bpf/arena.c\n+++ b/kernel/bpf/arena.c\n@@ -620,8 +620,9 @@ static int arena_map_mmap(struct bpf_map *map, struct vm_area_struct *vma)\n \t * clears VM_MAYEXEC. Set VM_DONTEXPAND to avoid potential change\n \t * of user_vm_start. Set VM_DONTCOPY to prevent arena VMA from\n \t * being copied into the child process on fork.\n+\t * This is a kernel page so set VM_MIXEDMAP.\n \t */\n-\tvm_flags_set(vma, VM_DONTEXPAND | VM_DONTCOPY);\n+\tvm_flags_set(vma, VM_MIXEDMAP | VM_DONTEXPAND | VM_DONTCOPY);\n \tvma-\u003evm_ops = \u0026arena_vm_ops;\n \treturn 0;\n }\ndiff --git a/kernel/events/core.c b/kernel/events/core.c\nindex a6c8e38a31104..8ca8a68429242 100644\n--- a/kernel/events/core.c\n+++ b/kernel/events/core.c\n@@ -9808,7 +9808,7 @@ static void perf_event_mmap_event(struct perf_mmap_event *mmap_event)\n \n \tif (vma-\u003evm_flags \u0026 VM_LOCKED)\n \t\tflags |= MAP_LOCKED;\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\tflags |= MAP_HUGETLB;\n \n \tif (file) {\ndiff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c\nindex 7709ea8824778..b89cc5cee0027 100644\n--- a/kernel/events/uprobes.c\n+++ b/kernel/events/uprobes.c\n@@ -1726,8 +1726,8 @@ static int xol_add_vma(struct mm_struct *mm, struct xol_area *area)\n \t}\n \n \tvma = _install_special_mapping(mm, area-\u003evaddr, PAGE_SIZE,\n-\t\t\t\tVM_EXEC|VM_MAYEXEC|VM_DONTCOPY|VM_IO|\n-\t\t\t\tVM_SEALED_SYSMAP,\n+\t\t\t\tVM_EXEC|VM_MAYEXEC|VM_DONTCOPY|\n+\t\t\t\tVM_MIXEDMAP|VM_SEALED_SYSMAP,\n \t\t\t\t\u0026xol_mapping);\n \tif (IS_ERR(vma)) {\n \t\tret = PTR_ERR(vma);\ndiff --git a/kernel/sched/fair.c b/kernel/sched/fair.c\nindex 8dff37059faf7..ae6c1a606eb5d 100644\n--- a/kernel/sched/fair.c\n+++ b/kernel/sched/fair.c\n@@ -22,7 +22,6 @@\n */\n #include \u003clinux/energy_model.h\u003e\n #include \u003clinux/mmap_lock.h\u003e\n-#include \u003clinux/hugetlb_inline.h\u003e\n #include \u003clinux/jiffies.h\u003e\n #include \u003clinux/mm_api.h\u003e\n #include \u003clinux/highmem.h\u003e\n@@ -4212,7 +4211,7 @@ static void task_numa_work(struct callback_head *work)\n \n \tfor (; vma; vma = vma_next(\u0026vmi)) {\n \t\tif (!vma_migratable(vma) || !vma_policy_mof(vma) ||\n-\t\t\tis_vm_hugetlb_page(vma) || (vma-\u003evm_flags \u0026 VM_MIXEDMAP)) {\n+\t\t\tvma_is_hugetlb(vma) || vma_is_kernel_owned(vma)) {\n \t\t\ttrace_sched_skip_vma_numa(mm, vma, NUMAB_SKIP_UNSUITABLE);\n \t\t\tcontinue;\n \t\t}\ndiff --git a/mm/folio.c b/mm/folio.c\nindex 47a437e0f7fde..35e242b48870b 100644\n--- a/mm/folio.c\n+++ b/mm/folio.c\n@@ -505,7 +505,7 @@ void folio_add_lru_vma(struct folio *folio, struct vm_area_struct *vma)\n {\n \tVM_BUG_ON_FOLIO(folio_test_lru(folio), folio);\n \n-\tif (unlikely((vma-\u003evm_flags \u0026 (VM_LOCKED | VM_SPECIAL)) == VM_LOCKED))\n+\tif (vma_test(vma, VMA_LOCKED_BIT))\n \t\tmlock_new_folio(folio);\n \telse\n \t\tfolio_add_lru(folio);\ndiff --git a/mm/gup.c b/mm/gup.c\nindex c2dfcb4744bc3..8e9ef5ee7498c 100644\n--- a/mm/gup.c\n+++ b/mm/gup.c\n@@ -621,7 +621,7 @@ static struct page *no_page_table(struct vm_area_struct *vma,\n \t * But we can only make this optimization where a hole would surely\n \t * be zero-filled if handle_mm_fault() actually did handle it.\n \t */\n-\tif (is_vm_hugetlb_page(vma)) {\n+\tif (vma_is_hugetlb(vma)) {\n \t\tstruct hstate *h = hstate_vma(vma);\n \n \t\tif (!hugetlbfs_pagecache_present(h, vma, address))\n@@ -1204,7 +1204,7 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags)\n \tint foreign = (gup_flags \u0026 FOLL_REMOTE);\n \tbool vma_anon = vma_is_anonymous(vma);\n \n-\tif (vm_flags \u0026 (VM_IO | VM_PFNMAP))\n+\tif (!vma_can_gup(vma))\n \t\treturn -EFAULT;\n \n \tif ((gup_flags \u0026 FOLL_ANON) \u0026\u0026 !vma_anon)\n@@ -1213,7 +1213,7 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags)\n \tif ((gup_flags \u0026 FOLL_LONGTERM) \u0026\u0026 vma_is_fsdax(vma))\n \t\treturn -EOPNOTSUPP;\n \n-\tif ((gup_flags \u0026 FOLL_SPLIT_PMD) \u0026\u0026 is_vm_hugetlb_page(vma))\n+\tif ((gup_flags \u0026 FOLL_SPLIT_PMD) \u0026\u0026 vma_is_hugetlb(vma))\n \t\treturn -EOPNOTSUPP;\n \n \tif (vma_is_secretmem(vma))\n@@ -1836,6 +1836,10 @@ long populate_vma_page_range(struct vm_area_struct *vma,\n \tif (!vma_is_accessible(vma))\n \t\treturn -EFAULT;\n \n+\t/* Unreadable VMAs also cannot be faulted in. */\n+\tif (!vma_test(vma, VMA_MAYREAD_BIT))\n+\t\treturn -EFAULT;\n+\n \tgup_flags = FOLL_TOUCH;\n \t/*\n \t * We want to touch writable mappings with a write fault in order\n@@ -1951,7 +1955,7 @@ int __mm_populate(unsigned long start, unsigned long len, int ignore_errors)\n \t\t * range with the first VMA. Also, skip undesirable VMA types.\n \t\t */\n \t\tnend = min(end, vma-\u003evm_end);\n-\t\tif (vma-\u003evm_flags \u0026 (VM_IO | VM_PFNMAP))\n+\t\tif (!vma_can_gup(vma))\n \t\t\tcontinue;\n \t\tif (nstart \u003c vma-\u003evm_start)\n \t\t\tnstart = vma-\u003evm_start;\n@@ -2013,8 +2017,7 @@ static long __get_user_pages_locked(struct mm_struct *mm, unsigned long start,\n \t\t\tbreak;\n \n \t\t/* protect what we can, including chardevs */\n-\t\tif ((vma-\u003evm_flags \u0026 (VM_IO | VM_PFNMAP)) ||\n-\t\t !(vm_flags \u0026 vma-\u003evm_flags))\n+\t\tif (!vma_can_gup(vma) || !(vm_flags \u0026 vma-\u003evm_flags))\n \t\t\tbreak;\n \n \t\tif (pages) {\ndiff --git a/mm/hmm.c b/mm/hmm.c\nindex 2f1e98c6b6440..e9569b82a1f0c 100644\n--- a/mm/hmm.c\n+++ b/mm/hmm.c\n@@ -595,8 +595,7 @@ static int hmm_vma_walk_test(unsigned long start, unsigned long end,\n \tstruct hmm_range *range = hmm_vma_walk-\u003erange;\n \tstruct vm_area_struct *vma = walk-\u003evma;\n \n-\tif (!(vma-\u003evm_flags \u0026 (VM_IO | VM_PFNMAP)) \u0026\u0026\n-\t vma-\u003evm_flags \u0026 VM_READ)\n+\tif (vma_can_gup(vma) \u0026\u0026 vma_test(vma, VMA_READ_BIT))\n \t\treturn 0;\n \n \t/*\ndiff --git a/mm/huge_memory.c b/mm/huge_memory.c\nindex 4cd917f77f3f7..3cb8e2d4d65cb 100644\n--- a/mm/huge_memory.c\n+++ b/mm/huge_memory.c\n@@ -110,14 +110,6 @@ static inline bool file_thp_enabled(const struct vm_area_struct *vma)\n \treturn S_ISREG(inode-\u003ei_mode);\n }\n \n-/* If returns true, we are unable to access the VMA's folios. */\n-static bool vma_is_special_huge(const struct vm_area_struct *vma)\n-{\n-\tif (vma_is_dax(vma))\n-\t\treturn false;\n-\treturn vma_test_any(vma, VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT);\n-}\n-\n static bool vma_file_bypass_thp_tuneables(const struct vm_area_struct *vma,\n \t\tenum tva_type type)\n {\n@@ -192,7 +184,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,\n \t/* Check the intersection of requested and supported orders. */\n \tif (vma_is_anonymous(vma))\n \t\tsupported_orders = THP_ORDERS_ALL_ANON;\n-\telse if (vma_is_dax(vma) || vma_is_special_huge(vma))\n+\telse if (vma_is_dax(vma) || vma_is_kernel_owned(vma))\n \t\tsupported_orders = THP_ORDERS_ALL_SPECIAL_DAX;\n \telse\n \t\tsupported_orders = THP_ORDERS_ALL_FILE_DEFAULT;\n@@ -212,11 +204,14 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,\n \t\treturn in_pf ? orders : 0;\n \n \t/*\n-\t * khugepaged special VMA and hugetlb VMA.\n-\t * Must be checked after dax since some dax mappings may have\n-\t * VM_MIXEDMAP set.\n+\t * khugepaged moves data from VMAs once collapsed, after they have been\n+\t * faulted in, relying on refaulting for file-backed memory.\n+\t *\n+\t * Kernel-owned mappings cannot be reliably reconstructed from page\n+\t * faults, and fixed mappings (including hugetlb) may not be marked as\n+\t * kernel-owned - precisely the mappings which cannot be merged.\n \t */\n-\tif (!in_pf \u0026\u0026 !smaps \u0026\u0026 (vm_flags \u0026 VM_NO_KHUGEPAGED))\n+\tif (!in_pf \u0026\u0026 !smaps \u0026\u0026 !vma_can_merge(vma))\n \t\treturn 0;\n \n \t/*\n@@ -3063,7 +3058,7 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma,\n \torig_pud = pudp_huge_get_and_clear_full(vma, addr, pud, tlb-\u003efullmm);\n \tarch_check_zapped_pud(vma, orig_pud);\n \ttlb_remove_pud_tlb_entry(tlb, pud, addr);\n-\tif (vma_is_special_huge(vma)) {\n+\tif (vma_is_kernel_owned(vma)) {\n \t\tspin_unlock(ptl);\n \t\t/* No zero page support yet */\n \t} else {\n@@ -3219,7 +3214,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,\n \t\t */\n \t\tif (arch_needs_pgtable_deposit())\n \t\t\tzap_deposited_table(mm, pmd);\n-\t\tif (vma_is_special_huge(vma))\n+\t\tif (vma_is_kernel_owned(vma))\n \t\t\treturn;\n \t\tif (unlikely(pmd_is_migration_entry(old_pmd))) {\n \t\t\tconst softleaf_t old_entry = softleaf_from_pmd(old_pmd);\n@@ -4762,11 +4757,9 @@ static inline bool vma_not_suitable_for_thp_split(struct vm_area_struct *vma)\n {\n \tif (vma_is_dax(vma))\n \t\treturn true;\n-\tif (vma_is_special_huge(vma))\n-\t\treturn true;\n-\tif (vma_test(vma, VMA_IO_BIT))\n+\tif (vma_is_kernel_owned(vma))\n \t\treturn true;\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\treturn true;\n \n \treturn false;\ndiff --git a/mm/hugetlb.c b/mm/hugetlb.c\nindex d3a0650ff6905..817f57f13b09d 100644\n--- a/mm/hugetlb.c\n+++ b/mm/hugetlb.c\n@@ -1147,7 +1147,7 @@ static inline struct resv_map *inode_resv_map(struct inode *inode)\n \n static struct resv_map *vma_resv_map(struct vm_area_struct *vma)\n {\n-\tVM_BUG_ON_VMA(!is_vm_hugetlb_page(vma), vma);\n+\tVM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);\n \tif (vma-\u003evm_flags \u0026 VM_MAYSHARE) {\n \t\tstruct address_space *mapping = vma-\u003evm_file-\u003ef_mapping;\n \t\tstruct inode *inode = mapping-\u003ehost;\n@@ -1162,7 +1162,7 @@ static struct resv_map *vma_resv_map(struct vm_area_struct *vma)\n \n static void set_vma_resv_map(struct vm_area_struct *vma, struct resv_map *map)\n {\n-\tVM_WARN_ON_ONCE_VMA(!is_vm_hugetlb_page(vma), vma);\n+\tVM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);\n \tVM_WARN_ON_ONCE_VMA(vma_test(vma, VMA_MAYSHARE_BIT), vma);\n \n \tset_vma_private_data(vma, (unsigned long)map);\n@@ -1170,7 +1170,7 @@ static void set_vma_resv_map(struct vm_area_struct *vma, struct resv_map *map)\n \n static void set_vma_resv_flags(struct vm_area_struct *vma, unsigned long flags)\n {\n-\tVM_WARN_ON_ONCE_VMA(!is_vm_hugetlb_page(vma), vma);\n+\tVM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);\n \tVM_WARN_ON_ONCE_VMA(vma_test(vma, VMA_MAYSHARE_BIT), vma);\n \n \tset_vma_private_data(vma, get_vma_private_data(vma) | flags);\n@@ -1178,7 +1178,7 @@ static void set_vma_resv_flags(struct vm_area_struct *vma, unsigned long flags)\n \n static int is_vma_resv_set(struct vm_area_struct *vma, unsigned long flag)\n {\n-\tVM_BUG_ON_VMA(!is_vm_hugetlb_page(vma), vma);\n+\tVM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);\n \n \treturn (get_vma_private_data(vma) \u0026 flag) != 0;\n }\n@@ -1192,7 +1192,7 @@ bool __vma_private_lock(struct vm_area_struct *vma)\n \n void hugetlb_dup_vma_private(struct vm_area_struct *vma)\n {\n-\tVM_BUG_ON_VMA(!is_vm_hugetlb_page(vma), vma);\n+\tVM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);\n \t/*\n \t * Clear vm_private_data\n \t * - For shared mappings this is a per-vma semaphore that may be\n@@ -5279,7 +5279,7 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,\n \tunsigned long last_addr_mask;\n \n \ti_mmap_assert_write_locked(vma-\u003evm_file-\u003ef_mapping);\n-\tWARN_ON(!is_vm_hugetlb_page(vma));\n+\tWARN_ON(!vma_is_hugetlb(vma));\n \tBUG_ON(start \u0026 ~huge_page_mask(h));\n \tBUG_ON(end \u0026 ~huge_page_mask(h));\n \n@@ -7505,6 +7505,6 @@ void hugetlb_unshare_all_pmds(struct vm_area_struct *vma)\n */\n void fixup_hugetlb_reservations(struct vm_area_struct *vma)\n {\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\tclear_vma_resv_huge_pages(vma);\n }\ndiff --git a/mm/internal.h b/mm/internal.h\nindex 0dca33db068f6..83a4ba52aaebc 100644\n--- a/mm/internal.h\n+++ b/mm/internal.h\n@@ -7,6 +7,7 @@\n #ifndef __MM_INTERNAL_H\n #define __MM_INTERNAL_H\n \n+#include \u003clinux/file.h\u003e\n #include \u003clinux/fs.h\u003e\n #include \u003clinux/khugepaged.h\u003e\n #include \u003clinux/mm.h\u003e\n@@ -212,6 +213,24 @@ static inline void *folio_raw_mapping(const struct folio *folio)\n \treturn (void *)(mapping \u0026 ~FOLIO_MAPPING_FLAGS);\n }\n \n+/*\n+ * If the VMA has a close hook then close it, and since closing it might leave\n+ * it in an inconsistent state which makes the use of any hooks suspect, clear\n+ * them down by installing dummy empty hooks.\n+ */\n+static inline void vma_close(struct vm_area_struct *vma)\n+{\n+\tif (vma-\u003evm_ops \u0026\u0026 vma-\u003evm_ops-\u003eclose) {\n+\t\tvma-\u003evm_ops-\u003eclose(vma);\n+\n+\t\t/*\n+\t\t * The mapping is in an inconsistent state, and no further hooks\n+\t\t * may be invoked upon it.\n+\t\t */\n+\t\tvma-\u003evm_ops = \u0026vma_dummy_vm_ops;\n+\t}\n+}\n+\n /*\n * This is a file-backed mapping, and is about to be memory mapped - invoke its\n * mmap hook and safely handle error conditions. On error, VMA hooks will be\n@@ -224,8 +243,11 @@ static inline void *folio_raw_mapping(const struct folio *folio)\n */\n static inline int mmap_file(struct file *file, struct vm_area_struct *vma)\n {\n-\tint err = vfs_mmap(file, vma);\n+\tconst unsigned long prev_start = vma-\u003evm_start;\n+\tconst vma_flags_t prev_flags = vma-\u003eflags;\n+\tint err;\n \n+\terr = vfs_mmap(file, vma);\n \t/*\n \t * Either we tried to call the file hook for mmap() and an error arose\n \t * or a driver set vma-\u003evm_ops = NULL intending there to be no VMA\n@@ -238,26 +260,16 @@ static inline int mmap_file(struct file *file, struct vm_area_struct *vma)\n \t */\n \tif (unlikely(err || !vma-\u003evm_ops))\n \t\tvma-\u003evm_ops = \u0026vma_dummy_vm_ops;\n+\tif (unlikely(err))\n+\t\treturn err;\n \n-\treturn err;\n-}\n-\n-/*\n- * If the VMA has a close hook then close it, and since closing it might leave\n- * it in an inconsistent state which makes the use of any hooks suspect, clear\n- * them down by installing dummy empty hooks.\n- */\n-static inline void vma_close(struct vm_area_struct *vma)\n-{\n-\tif (vma-\u003evm_ops \u0026\u0026 vma-\u003evm_ops-\u003eclose) {\n-\t\tvma-\u003evm_ops-\u003eclose(vma);\n-\n-\t\t/*\n-\t\t * The mapping is in an inconsistent state, and no further hooks\n-\t\t * may be invoked upon it.\n-\t\t */\n-\t\tvma-\u003evm_ops = \u0026vma_dummy_vm_ops;\n+\terr = mmap_hook_validate(prev_start, \u0026prev_flags, vma);\n+\tif (unlikely(err)) {\n+\t\tvma-\u003evm_start = prev_start;\n+\t\tvma_close(vma);\n \t}\n+\n+\treturn err;\n }\n \n /* unmap_vmas is in mm/memory.c */\n@@ -957,15 +969,7 @@ void mlock_folio(struct folio *folio);\n static inline void mlock_vma_folio(struct folio *folio,\n \t\t\t\tstruct vm_area_struct *vma)\n {\n-\t/*\n-\t * The VM_SPECIAL check here serves two purposes.\n-\t * 1) VM_IO check prevents migration from double-counting during mlock.\n-\t * 2) Although mmap_region() and mlock_fixup() take care that VM_LOCKED\n-\t * is never left set on a VM_SPECIAL vma, there is an interval while\n-\t * file-\u003ef_op-\u003emmap() is using vm_insert_page(s), when VM_LOCKED may\n-\t * still be set while VM_SPECIAL bits are added: so ignore it then.\n-\t */\n-\tif (unlikely((vma-\u003evm_flags \u0026 (VM_LOCKED|VM_SPECIAL)) == VM_LOCKED))\n+\tif (vma_test(vma, VMA_LOCKED_BIT))\n \t\tmlock_folio(folio);\n }\n \n@@ -982,7 +986,12 @@ static inline void munlock_vma_folio(struct folio *folio,\n \t * always munlock the folio and page reclaim will correct it\n \t * if it's wrong.\n \t */\n-\tif (unlikely(vma-\u003evm_flags \u0026 VM_LOCKED))\n+\t/*\n+\t * VMA_LOCKONFAULT_BIT alone marks an mlock walk in progress, see\n+\t * mlock_vma_pages_range(). An unmap racing with the walk must still\n+\t * munlock folios the walk has already counted.\n+\t */\n+\tif (unlikely(vma_test_any_mask(vma, VMA_LOCKED_MASK)))\n \t\tmunlock_folio(folio);\n }\n \n@@ -1102,11 +1111,9 @@ static inline struct file *maybe_unlock_mmap_for_io(struct vm_fault *vmf,\n \n static inline bool vma_supports_mlock(const struct vm_area_struct *vma)\n {\n-\tif (vma_test_any_mask(vma, VMA_SPECIAL_FLAGS))\n-\t\treturn false;\n-\tif (vma_test_single_mask(vma, VMA_DROPPABLE))\n+\tif (!vma_is_persistent(vma))\n \t\treturn false;\n-\tif (vma_is_dax(vma) || is_vm_hugetlb_page(vma))\n+\tif (vma_is_dax(vma) || vma_is_hugetlb(vma))\n \t\treturn false;\n \treturn vma != get_gate_vma(current-\u003emm);\n }\n@@ -1499,6 +1506,12 @@ int remap_pfn_range_prepare(struct vm_area_desc *desc);\n int remap_pfn_range_complete(struct vm_area_struct *vma,\n \t\t\t struct mmap_action *action);\n int simple_ioremap_prepare(struct vm_area_desc *desc);\n+int map_kernel_pages_prepare(struct vm_area_desc *desc);\n+int map_kernel_pages_complete(struct vm_area_struct *vma,\n+\t\t\t struct mmap_action *action);\n+int map_discontig_kernel_pages_prepare(struct vm_area_desc *desc);\n+int map_discontig_kernel_pages_complete(struct vm_area_struct *vma,\n+\t\t\t\t\tstruct mmap_action *action);\n \n static inline int io_remap_pfn_range_prepare(struct vm_area_desc *desc)\n {\ndiff --git a/mm/ksm.c b/mm/ksm.c\nindex 624f37975e129..f80372bfd4b2f 100644\n--- a/mm/ksm.c\n+++ b/mm/ksm.c\n@@ -747,9 +747,7 @@ static bool ksm_compatible(const struct file *file, vma_flags_t vma_flags)\n \tif (vma_flags_test_any(\u0026vma_flags, VMA_SHARED_BIT, VMA_MAYSHARE_BIT,\n \t\t\t VMA_HUGETLB_BIT))\n \t\treturn false;\n-\tif (vma_flags_test_single_mask(\u0026vma_flags, VMA_DROPPABLE))\n-\t\treturn false;\n-\tif (vma_flags_test_any_mask(\u0026vma_flags, VMA_SPECIAL_FLAGS))\n+\tif (!vma_flags_is_persistent(\u0026vma_flags))\n \t\treturn false;\n \tif (file_is_dax(file))\n \t\treturn false;\ndiff --git a/mm/madvise.c b/mm/madvise.c\nindex fbb72ab49aa64..1af82b044d235 100644\n--- a/mm/madvise.c\n+++ b/mm/madvise.c\n@@ -881,7 +881,7 @@ bool madvise_dontneed_free_valid_vma(struct madvise_behavior *madv_behavior)\n \tint behavior = madv_behavior-\u003ebehavior;\n \tstruct madvise_behavior_range *range = \u0026madv_behavior-\u003erange;\n \n-\tif (!is_vm_hugetlb_page(vma)) {\n+\tif (!vma_is_hugetlb(vma)) {\n \t\tunsigned int forbidden = VM_PFNMAP;\n \n \t\tif (behavior != MADV_DONTNEED_LOCKED)\n@@ -1221,19 +1221,25 @@ static long madvise_remove(struct madvise_behavior *madv_behavior)\n \treturn error;\n }\n \n-static bool is_valid_guard_vma(struct vm_area_struct *vma, bool allow_locked)\n+static bool is_valid_guard_vma(const struct vm_area_struct *vma,\n+\t\t\t bool allow_locked)\n {\n-\tvm_flags_t disallowed = VM_SPECIAL | VM_HUGETLB;\n-\n \t/*\n-\t * A user could lock after setting a guard range but that's fine, as\n+\t * A user could lock after setting a guard range but that's fine as\n \t * they'd not be able to fault in. The issue arises when we try to zap\n \t * existing locked VMAs. We don't want to do that.\n \t */\n-\tif (!allow_locked)\n-\t\tdisallowed |= VM_LOCKED;\n+\tif (!allow_locked \u0026\u0026 vma_test(vma, VMA_LOCKED_BIT))\n+\t\treturn false;\n+\t/*\n+\t * Guard regions require a VMA whose page tables are managed solely by\n+\t * the core, which is also what merging requires, so disallow any flags\n+\t * that would prevent a merge.\n+\t */\n+\tif (!vma_can_merge(vma))\n+\t\treturn false;\n \n-\treturn !(vma-\u003evm_flags \u0026 disallowed);\n+\treturn true;\n }\n \n static bool is_guard_pte_marker(pte_t ptent)\n@@ -1559,7 +1565,7 @@ static int madvise_vma_behavior(struct madvise_behavior *madv_behavior)\n \t\tnew_flags |= VM_DONTCOPY;\n \t\tbreak;\n \tcase MADV_DOFORK:\n-\t\tif (new_flags \u0026 VM_SPECIAL)\n+\t\tif (!vma_can_merge(vma))\n \t\t\treturn -EINVAL;\n \t\tnew_flags \u0026= ~VM_DONTCOPY;\n \t\tbreak;\n@@ -1578,8 +1584,8 @@ static int madvise_vma_behavior(struct madvise_behavior *madv_behavior)\n \t\tnew_flags |= VM_DONTDUMP;\n \t\tbreak;\n \tcase MADV_DODUMP:\n-\t\tif ((!is_vm_hugetlb_page(vma) \u0026\u0026 (new_flags \u0026 VM_SPECIAL)) ||\n-\t\t (new_flags \u0026 VM_DROPPABLE))\n+\t\t/* Non-persistent memory cannot be dumped. */\n+\t\tif (!vma_is_persistent(vma))\n \t\t\treturn -EINVAL;\n \t\tnew_flags \u0026= ~VM_DONTDUMP;\n \t\tbreak;\ndiff --git a/mm/memory.c b/mm/memory.c\nindex 926276d419202..9e4a70421a6b8 100644\n--- a/mm/memory.c\n+++ b/mm/memory.c\n@@ -1564,7 +1564,7 @@ copy_page_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma)\n \tif (!vma_needs_copy(dst_vma, src_vma))\n \t\treturn 0;\n \n-\tif (is_vm_hugetlb_page(src_vma))\n+\tif (vma_is_hugetlb(src_vma))\n \t\treturn copy_hugetlb_page_range(dst_mm, src_mm, dst_vma, src_vma);\n \n \t/*\n@@ -2178,7 +2178,7 @@ static void __zap_vma_range(struct mmu_gather *tlb, struct vm_area_struct *vma,\n \tif (vma-\u003evm_file \u0026\u0026 !reaping)\n \t\tuprobe_munmap(vma, start, end);\n \n-\tif (unlikely(is_vm_hugetlb_page(vma))) {\n+\tif (unlikely(vma_is_hugetlb(vma))) {\n \t\tzap_flags_t zap_flags = details ? details-\u003ezap_flags : 0;\n \n \t\tVM_WARN_ON_ONCE(reaping);\n@@ -2313,7 +2313,7 @@ void zap_vma_range_batched(struct mmu_gather *tlb,\n \t */\n \t__zap_vma_range(tlb, vma, address, end, details);\n \tmmu_notifier_invalidate_range_end(\u0026range);\n-\tif (is_vm_hugetlb_page(vma)) {\n+\tif (vma_is_hugetlb(vma)) {\n \t\t/*\n \t\t * flush tlb and free resources before hugetlb_zap_end(), to\n \t\t * avoid concurrent page faults' allocation failure.\n@@ -2343,19 +2343,19 @@ void zap_vma_range(struct vm_area_struct *vma, unsigned long address,\n }\n \n /**\n- * zap_special_vma_range - zap all page table entries in a special vma range\n+ * zap_special_vma_range - zap all page table entries in a kernel-owned VMA\n * @vma: the vma covering the range to zap\n * @address: starting address of the range to zap\n * @size: number of bytes to zap\n *\n * This function does nothing when the provided address range is not fully\n- * contained in @vma, or when the @vma is not VM_PFNMAP or VM_MIXEDMAP.\n+ * contained in @vma, or when @vma is not kernel-owned.\n */\n void zap_special_vma_range(struct vm_area_struct *vma, unsigned long address,\n \t\tunsigned long size)\n {\n \tif (!range_in_vma(vma, address, address + size) ||\n-\t !(vma-\u003evm_flags \u0026 (VM_PFNMAP | VM_MIXEDMAP)))\n+\t !vma_is_kernel_owned(vma))\n \t\treturn;\n \n \tzap_vma_range(vma, address, size);\n@@ -2417,11 +2417,11 @@ static bool vm_mixed_zeropage_allowed(struct vm_area_struct *vma)\n \t * be problematic as soon as the zeropage gets replaced by a different\n \t * page due to vma-\u003evm_ops-\u003epfn_mkwrite, because what's mapped would\n \t * now differ to what GUP looked up. FSDAX is incompatible to\n-\t * FOLL_LONGTERM and VM_IO is incompatible to GUP completely (see\n-\t * check_vma_flags).\n+\t * FOLL_LONGTERM and memory-mapped I/O is incompatible to GUP completely\n+\t * (see vma_can_gup()).\n \t */\n \treturn vma-\u003evm_ops \u0026\u0026 vma-\u003evm_ops-\u003epfn_mkwrite \u0026\u0026\n-\t (vma_is_fsdax(vma) || vma-\u003evm_flags \u0026 VM_IO);\n+\t (vma_is_fsdax(vma) || vma_test(vma, VMA_IO_BIT));\n }\n \n static int validate_page_before_insert(struct vm_area_struct *vma,\n@@ -2609,17 +2609,23 @@ int vm_insert_pages(struct vm_area_struct *vma, unsigned long addr,\n }\n EXPORT_SYMBOL(vm_insert_pages);\n \n+static void __map_kernel_pages_prepare(struct vm_area_desc *desc)\n+{\n+\tif (vma_desc_test(desc, VMA_MIXEDMAP_BIT))\n+\t\treturn;\n+\n+\tVM_WARN_ON_ONCE(mmap_read_trylock(desc-\u003emm));\n+\tVM_WARN_ON_ONCE(vma_desc_test(desc, VMA_PFNMAP_BIT));\n+\tvma_desc_set_flags(desc, VMA_MIXEDMAP_BIT);\n+}\n+\n int map_kernel_pages_prepare(struct vm_area_desc *desc)\n {\n \tconst struct mmap_action *action = \u0026desc-\u003eaction;\n \tconst unsigned long addr = action-\u003emap_kernel.start;\n \tunsigned long nr_pages, end;\n \n-\tif (!vma_desc_test(desc, VMA_MIXEDMAP_BIT)) {\n-\t\tVM_WARN_ON_ONCE(mmap_read_trylock(desc-\u003emm));\n-\t\tVM_WARN_ON_ONCE(vma_desc_test(desc, VMA_PFNMAP_BIT));\n-\t\tvma_desc_set_flags(desc, VMA_MIXEDMAP_BIT);\n-\t}\n+\t__map_kernel_pages_prepare(desc);\n \n \tnr_pages = action-\u003emap_kernel.nr_pages;\n \tend = addr + PAGE_SIZE * nr_pages;\n@@ -2628,7 +2634,6 @@ int map_kernel_pages_prepare(struct vm_area_desc *desc)\n \n \treturn 0;\n }\n-EXPORT_SYMBOL(map_kernel_pages_prepare);\n \n int map_kernel_pages_complete(struct vm_area_struct *vma,\n \t\t\t struct mmap_action *action)\n@@ -2640,7 +2645,98 @@ int map_kernel_pages_complete(struct vm_area_struct *vma,\n \t\t\t action-\u003emap_kernel.pages,\n \t\t\t \u0026nr_pages, vma-\u003evm_page_prot);\n }\n-EXPORT_SYMBOL(map_kernel_pages_complete);\n+\n+int map_discontig_kernel_pages_prepare(struct vm_area_desc *desc)\n+{\n+\tconst struct mmap_action *action = \u0026desc-\u003eaction;\n+\tconst struct discontig_kernel_page_ops *ops =\n+\t\taction-\u003emap_kernel_discontig.ops;\n+\n+\t/* At minimum need to be able to get pages. */\n+\tif (WARN_ON_ONCE(!ops-\u003eget))\n+\t\treturn -EINVAL;\n+\n+\t__map_kernel_pages_prepare(desc);\n+\treturn 0;\n+}\n+\n+static int apply_discontig_action(struct vm_area_struct *vma,\n+\t\t\t\t struct discontig_kernel_page_state *state)\n+{\n+\tunsigned long nr_pages = state-\u003e__nr_pages;\n+\tunsigned long addr = state-\u003eaddr;\n+\tunsigned long i;\n+\n+\tif (state-\u003eaction == DISCONTIG_KERNEL_PAGE_MAP_PAGE)\n+\t\treturn insert_page(vma, addr, state-\u003e__page,\n+\t\t\t\t vma-\u003evm_page_prot, /*mkwrite=*/false);\n+\tif (state-\u003eaction == DISCONTIG_KERNEL_PAGE_MAP_PAGE_RANGE)\n+\t\treturn insert_pages(vma, addr, state-\u003e__page_arr,\n+\t\t\t\t \u0026nr_pages, vma-\u003evm_page_prot);\n+\n+\t/* Compound folio - have to iterate through each page. */\n+\tfor (i = 0; i \u003c nr_pages; i++, addr += PAGE_SIZE) {\n+\t\tstruct page *page = folio_page(state-\u003e__folio, i);\n+\t\tint err;\n+\n+\t\terr = insert_page(vma, addr, page, vma-\u003evm_page_prot,\n+\t\t\t\t /*mkwrite=*/false);\n+\t\tif (err)\n+\t\t\treturn err;\n+\t}\n+\treturn 0;\n+}\n+\n+int map_discontig_kernel_pages_complete(struct vm_area_struct *vma,\n+\t\t\t\t\tstruct mmap_action *action)\n+{\n+\tconst struct discontig_kernel_page_ops *ops =\n+\t\taction-\u003emap_kernel_discontig.ops;\n+\tstruct discontig_kernel_page_state state = {\n+\t\t.start = vma-\u003evm_start,\n+\t\t.end = vma-\u003evm_end,\n+\t\t.addr = vma-\u003evm_start,\n+\t\t.pgoff = vma-\u003evm_pgoff,\n+\t\t.nr_pages_mapped = 0,\n+\t\t.nr_pages_remain = vma_pages(vma),\n+\t\t.vm_private_data = vma-\u003evm_private_data,\n+\t\t.private = action-\u003emap_kernel_discontig.init_private,\n+\t};\n+\tint err = 0;\n+\n+\tif (ops-\u003einit)\n+\t\terr = ops-\u003einit(vma-\u003evm_private_data, \u0026state.private);\n+\tif (err)\n+\t\treturn err;\n+\n+\tdo {\n+\t\tunsigned long end, pgoff_end;\n+\t\tunsigned long nr_pages;\n+\n+\t\t/* Default to abort. */\n+\t\tstate.action = DISCONTIG_KERNEL_PAGE_ABORT;\n+\t\terr = ops-\u003eget(\u0026state);\n+\t\tif (err || state.action == DISCONTIG_KERNEL_PAGE_ABORT)\n+\t\t\treturn err;\n+\t\tnr_pages = state.__nr_pages;\n+\n+\t\tif (!nr_pages || nr_pages \u003e state.nr_pages_remain)\n+\t\t\treturn -EINVAL;\n+\t\tend = state.addr + PAGE_SIZE * nr_pages;\n+\t\tpgoff_end = state.pgoff + nr_pages;\n+\n+\t\terr = apply_discontig_action(vma, \u0026state);\n+\t\tif (err)\n+\t\t\treturn err;\n+\n+\t\tstate.addr = end;\n+\t\tstate.pgoff = pgoff_end;\n+\t\tstate.nr_pages_mapped += nr_pages;\n+\t\tstate.nr_pages_remain -= nr_pages;\n+\t} while (state.addr \u003c vma-\u003evm_end);\n+\n+\treturn 0;\n+}\n \n /**\n * vm_insert_page - insert single page into user vma\n@@ -6837,7 +6933,7 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,\n \n \tlru_gen_enter_fault(vma);\n \n-\tif (unlikely(is_vm_hugetlb_page(vma)))\n+\tif (unlikely(vma_is_hugetlb(vma)))\n \t\tret = hugetlb_fault(vma-\u003evm_mm, vma, address, flags);\n \telse\n \t\tret = __handle_mm_fault(vma, address, flags);\n@@ -7020,7 +7116,8 @@ int follow_pfnmap_start(struct follow_pfnmap_args *args)\n \tif (unlikely(address \u003c vma-\u003evm_start || address \u003e= vma-\u003evm_end))\n \t\tgoto out;\n \n-\tif (!(vma-\u003evm_flags \u0026 (VM_IO | VM_PFNMAP)))\n+\t/* Only mappings GUP cannot handle are followed here. */\n+\tif (vma_can_gup(vma))\n \t\tgoto out;\n retry:\n \tpgdp = pgd_offset(mm, address);\n@@ -7220,8 +7317,9 @@ static int __access_remote_vm(struct mm_struct *mm, unsigned long addr,\n \t\t\t}\n \n \t\t\t/*\n-\t\t\t * Check if this is a VM_IO | VM_PFNMAP VMA, which\n-\t\t\t * we can access using slightly different code.\n+\t\t\t * GUP failed, perhaps because this is a mapping it\n+\t\t\t * cannot handle (see vma_can_gup()) - such mappings may\n+\t\t\t * provide access via vm_ops-\u003eaccess() instead.\n \t\t\t */\n \t\t\tbytes = 0;\n #ifdef CONFIG_HAVE_IOREMAP_PROT\n@@ -7707,12 +7805,12 @@ void ptlock_free(struct ptdesc *ptdesc)\n \n void vma_pgtable_walk_begin(struct vm_area_struct *vma)\n {\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\thugetlb_vma_lock_read(vma);\n }\n \n void vma_pgtable_walk_end(struct vm_area_struct *vma)\n {\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\thugetlb_vma_unlock_read(vma);\n }\ndiff --git a/mm/mempolicy.c b/mm/mempolicy.c\nindex 8fc8a975657e6..2fd759e348ca1 100644\n--- a/mm/mempolicy.c\n+++ b/mm/mempolicy.c\n@@ -2013,7 +2013,8 @@ SYSCALL_DEFINE5(get_mempolicy, int __user *, policy,\n \n bool vma_migratable(struct vm_area_struct *vma)\n {\n-\tif (vma-\u003evm_flags \u0026 (VM_IO | VM_PFNMAP))\n+\t/* Pages which GUP cannot obtain cannot be migrated either. */\n+\tif (!vma_can_gup(vma))\n \t\treturn false;\n \n \t/*\n@@ -2023,7 +2024,7 @@ bool vma_migratable(struct vm_area_struct *vma)\n \tif (vma_is_dax(vma))\n \t\treturn false;\n \n-\tif (is_vm_hugetlb_page(vma) \u0026\u0026\n+\tif (vma_is_hugetlb(vma) \u0026\u0026\n \t\t!hugepage_migration_supported(hstate_vma(vma)))\n \t\treturn false;\n \ndiff --git a/mm/migrate_device.c b/mm/migrate_device.c\nindex 0c437004329d9..b74c0ae427682 100644\n--- a/mm/migrate_device.c\n+++ b/mm/migrate_device.c\n@@ -739,19 +739,21 @@ static void migrate_vma_unmap(struct migrate_vma *migrate)\n */\n int migrate_vma_setup(struct migrate_vma *args)\n {\n+\tconst struct vm_area_struct *vma = args-\u003evma;\n \tlong nr_pages = (args-\u003eend - args-\u003estart) \u003e\u003e PAGE_SHIFT;\n \n \targs-\u003estart \u0026= PAGE_MASK;\n \targs-\u003eend \u0026= PAGE_MASK;\n-\tif (!args-\u003evma || is_vm_hugetlb_page(args-\u003evma) ||\n-\t (args-\u003evma-\u003evm_flags \u0026 VM_SPECIAL) || vma_is_dax(args-\u003evma))\n+\tif (!vma)\n+\t\treturn -EINVAL;\n+\tif (vma_is_kernel_owned(vma) || vma_is_fixed_mapping(vma) ||\n+\t vma_is_dax(vma))\n \t\treturn -EINVAL;\n \tif (nr_pages \u003c= 0)\n \t\treturn -EINVAL;\n-\tif (args-\u003estart \u003c args-\u003evma-\u003evm_start ||\n-\t args-\u003estart \u003e= args-\u003evma-\u003evm_end)\n+\tif (args-\u003estart \u003c vma-\u003evm_start || args-\u003estart \u003e= vma-\u003evm_end)\n \t\treturn -EINVAL;\n-\tif (args-\u003eend \u003c= args-\u003evma-\u003evm_start || args-\u003eend \u003e args-\u003evma-\u003evm_end)\n+\tif (args-\u003eend \u003c= vma-\u003evm_start || args-\u003eend \u003e vma-\u003evm_end)\n \t\treturn -EINVAL;\n \tif (!args-\u003esrc || !args-\u003edst)\n \t\treturn -EINVAL;\ndiff --git a/mm/mlock.c b/mm/mlock.c\nindex 39215a3eab1fb..4235a1518fc9e 100644\n--- a/mm/mlock.c\n+++ b/mm/mlock.c\n@@ -316,22 +316,10 @@ static inline unsigned int folio_mlock_step(struct folio *folio,\n \treturn folio_pte_batch(folio, pte, ptent, count);\n }\n \n-static inline bool allow_mlock_munlock(struct folio *folio,\n+static inline bool allow_mlock(struct folio *folio,\n \t\tstruct vm_area_struct *vma, unsigned long start,\n \t\tunsigned long end, unsigned int step)\n {\n-\t/*\n-\t * For unlock, allow munlock large folio which is partially\n-\t * mapped to VMA. As it's possible that large folio is\n-\t * mlocked and VMA is split later.\n-\t *\n-\t * During memory pressure, such kind of large folio can\n-\t * be split. And the pages are not in VM_LOCKed VMA\n-\t * can be reclaimed.\n-\t */\n-\tif (!vma_test(vma, VMA_LOCKED_BIT))\n-\t\treturn true;\n-\n \t/* folio_within_range() cannot take KSM, but any small folio is OK */\n \tif (!folio_test_large(folio))\n \t\treturn true;\n@@ -352,6 +340,7 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr,\n \n {\n \tstruct vm_area_struct *vma = walk-\u003evma;\n+\tconst bool lock = walk-\u003eprivate;\n \tspinlock_t *ptl;\n \tpte_t *start_pte, *pte;\n \tpte_t ptent;\n@@ -368,7 +357,7 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr,\n \t\tfolio = pmd_folio(*pmd);\n \t\tif (folio_is_zone_device(folio))\n \t\t\tgoto out;\n-\t\tif (vma_test(vma, VMA_LOCKED_BIT))\n+\t\tif (lock)\n \t\t\tmlock_folio(folio);\n \t\telse\n \t\t\tmunlock_folio(folio);\n@@ -390,10 +379,10 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr,\n \t\t\tcontinue;\n \n \t\tstep = folio_mlock_step(folio, pte, addr, end);\n-\t\tif (!allow_mlock_munlock(folio, vma, start, end, step))\n+\t\tif (lock \u0026\u0026 !allow_mlock(folio, vma, start, end, step))\n \t\t\tgoto next_entry;\n \n-\t\tif (vma_test(vma, VMA_LOCKED_BIT))\n+\t\tif (lock)\n \t\t\tmlock_folio(folio);\n \t\telse\n \t\t\tmunlock_folio(folio);\n@@ -428,31 +417,29 @@ static void mlock_vma_pages_range(struct vm_area_struct *vma,\n \t\t.pmd_entry = mlock_pte_range,\n \t\t.walk_lock = PGWALK_WRLOCK_VERIFY,\n \t};\n+\tconst bool lock = vma_flags_test(new_vma_flags, VMA_LOCKED_BIT);\n+\tvma_flags_t walk_flags = *new_vma_flags;\n \n \t/*\n-\t * There is a slight chance that concurrent page migration,\n-\t * or page reclaim finding a page of this now-VMA_LOCKED_BIT vma,\n-\t * will call mlock_vma_folio() and raise page's mlock_count:\n-\t * double counting, leaving the page unevictable indefinitely.\n-\t * Communicate this danger to mlock_vma_folio() with VMA_IO_BIT,\n-\t * which is a VMA_SPECIAL_FLAGS flag not allowed on VMA_LOCKED_BIT vmas.\n-\t * mmap_lock is held in write mode here, so this weird\n-\t * combination should not be visible to other mmap_lock users;\n-\t * but WRITE_ONCE so rmap walkers must see VMA_IO_BIT if VMA_LOCKED_BIT.\n+\t * LOCKONFAULT without LOCKED never otherwise occurs: it marks a walk in\n+\t * progress so that rmap-side callers, which test VMA_LOCKED_BIT, do not\n+\t * count folios, while try_to_unmap_one(), which tests VMA_LOCKED_MASK,\n+\t * still refuses to unmap them.\n \t */\n-\tif (vma_flags_test(new_vma_flags, VMA_LOCKED_BIT))\n-\t\tvma_flags_set(new_vma_flags, VMA_IO_BIT);\n+\tif (lock) {\n+\t\tvma_flags_clear(\u0026walk_flags, VMA_LOCKED_BIT);\n+\t\tvma_flags_set(\u0026walk_flags, VMA_LOCKONFAULT_BIT);\n+\t}\n+\n \tvma_start_write(vma);\n-\tvma_flags_reset_once(vma, new_vma_flags);\n+\tvma_flags_reset_once(vma, \u0026walk_flags);\n \n \tlru_add_drain();\n-\twalk_page_range_vma(vma, start, end, \u0026mlock_walk_ops, NULL);\n+\twalk_page_range_vma(vma, start, end, \u0026mlock_walk_ops, (void *)lock);\n \tlru_add_drain();\n \n-\tif (vma_flags_test(new_vma_flags, VMA_IO_BIT)) {\n-\t\tvma_flags_clear(new_vma_flags, VMA_IO_BIT);\n+\tif (lock)\n \t\tvma_flags_reset_once(vma, new_vma_flags);\n-\t}\n }\n \n /*\ndiff --git a/mm/mmap.c b/mm/mmap.c\nindex 4bf26b0f1e6e3..98449f364af1c 100644\n--- a/mm/mmap.c\n+++ b/mm/mmap.c\n@@ -1786,7 +1786,7 @@ __latent_entropy int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)\n \t\t/*\n \t\t * Copy/update hugetlb private vma information.\n \t\t */\n-\t\tif (is_vm_hugetlb_page(tmp))\n+\t\tif (vma_is_hugetlb(tmp))\n \t\t\thugetlb_dup_vma_private(tmp);\n \n \t\t/*\ndiff --git a/mm/mmu_gather.c b/mm/mmu_gather.c\nindex 2a72a9686773a..9f353f0e2ef4d 100644\n--- a/mm/mmu_gather.c\n+++ b/mm/mmu_gather.c\n@@ -480,7 +480,7 @@ void tlb_gather_mmu_vma(struct mmu_gather *tlb, struct vm_area_struct *vma)\n {\n \ttlb_gather_mmu(tlb, vma-\u003evm_mm);\n \ttlb_update_vma_flags(tlb, vma);\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\t/* All entries have the same size. */\n \t\ttlb_change_page_size(tlb, huge_page_size(hstate_vma(vma)));\n }\ndiff --git a/mm/mprotect.c b/mm/mprotect.c\nindex 2888ee638d872..a1b6d29bf0390 100644\n--- a/mm/mprotect.c\n+++ b/mm/mprotect.c\n@@ -717,7 +717,7 @@ long change_protection(struct mmu_gather *tlb,\n \t (cp_flags \u0026 MM_CP_UFFD_RWP))\n \t\tnewprot = PAGE_NONE;\n \n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\tpages = hugetlb_change_protection(vma, start, end, newprot,\n \t\t\t\t\t\t cp_flags);\n \telse\n@@ -783,8 +783,7 @@ mprotect_fixup(struct vma_iterator *vmi, struct mmu_gather *tlb,\n \t * uncommon case, so doesn't need to be very optimized.\n \t */\n \tif (arch_has_pfn_modify_check() \u0026\u0026\n-\t vma_flags_test_any(\u0026old_vma_flags, VMA_PFNMAP_BIT,\n-\t\t\t VMA_MIXEDMAP_BIT) \u0026\u0026\n+\t vma_flags_is_kernel_owned(\u0026old_vma_flags) \u0026\u0026\n \t !vma_flags_test_any_mask(\u0026new_vma_flags, VMA_ACCESS_FLAGS)) {\n \t\tpgprot_t new_pgprot = vm_get_page_prot(newflags);\n \ndiff --git a/mm/mremap.c b/mm/mremap.c\nindex 7c368440fafe2..1122282a1d6ab 100644\n--- a/mm/mremap.c\n+++ b/mm/mremap.c\n@@ -812,7 +812,7 @@ unsigned long move_page_tables(struct pagetable_move_control *pmc)\n \tif (!pmc-\u003elen_in)\n \t\treturn 0;\n \n-\tif (is_vm_hugetlb_page(pmc-\u003eold))\n+\tif (vma_is_hugetlb(pmc-\u003eold))\n \t\treturn move_hugetlb_page_tables(pmc-\u003eold, pmc-\u003enew, pmc-\u003eold_addr,\n \t\t\t\t\t\tpmc-\u003enew_addr, pmc-\u003elen_in);\n \n@@ -1735,7 +1735,7 @@ static bool vma_multi_allowed(struct vm_area_struct *vma)\n \t/* Known good. */\n \tif (vma_is_shmem(vma))\n \t\treturn true;\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\treturn true;\n \tif (file-\u003ef_op-\u003eget_unmapped_area == thp_get_unmapped_area)\n \t\treturn true;\n@@ -1758,7 +1758,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm)\n \t\treturn -EPERM;\n \n \t/* Align to hugetlb page size, if required. */\n-\tif (is_vm_hugetlb_page(vma) \u0026\u0026 !align_hugetlb(vrm))\n+\tif (vma_is_hugetlb(vma) \u0026\u0026 !align_hugetlb(vrm))\n \t\treturn -EINVAL;\n \n \tvrm_set_delta(vrm);\n@@ -1788,8 +1788,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm)\n \t\treturn -EINVAL;\n \t}\n \n-\tif ((vrm-\u003eflags \u0026 MREMAP_DONTUNMAP) \u0026\u0026\n-\t vma_test_any(vma, VMA_DONTEXPAND_BIT, VMA_PFNMAP_BIT))\n+\tif ((vrm-\u003eflags \u0026 MREMAP_DONTUNMAP) \u0026\u0026 vma_is_fixed_mapping(vma))\n \t\treturn -EINVAL;\n \n \t/*\n@@ -1827,7 +1826,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm)\n \tif (pgoff + (new_len \u003e\u003e PAGE_SHIFT) \u003c pgoff)\n \t\treturn -EINVAL;\n \n-\tif (vma_test_any(vma, VMA_DONTEXPAND_BIT, VMA_PFNMAP_BIT))\n+\tif (vma_is_fixed_mapping(vma))\n \t\treturn -EFAULT;\n \n \tif (!mlock_future_ok(mm, vma_test(vma, VMA_LOCKED_BIT), vrm-\u003edelta))\ndiff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c\nindex 28e306fdb3a5b..8408aee7571b5 100644\n--- a/mm/page_vma_mapped.c\n+++ b/mm/page_vma_mapped.c\n@@ -109,7 +109,7 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw, unsigned long pte_nr)\n \tunsigned long pfn;\n \tpte_t ptent;\n \n-\tif (is_vm_hugetlb_page(pvmw-\u003evma))\n+\tif (vma_is_hugetlb(pvmw-\u003evma))\n \t\tptent = huge_ptep_get(pvmw-\u003evma-\u003evm_mm, pvmw-\u003eaddress,\n \t\t\t\t pvmw-\u003epte);\n \telse\n@@ -206,7 +206,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)\n \tif (pvmw-\u003epmd \u0026\u0026 !pvmw-\u003epte)\n \t\treturn not_found(pvmw);\n \n-\tif (unlikely(is_vm_hugetlb_page(vma))) {\n+\tif (unlikely(vma_is_hugetlb(vma))) {\n \t\tstruct hstate *hstate = hstate_vma(vma);\n \t\tunsigned long size = huge_page_size(hstate);\n \t\t/* The only possible mapping was handled on last iteration */\ndiff --git a/mm/pagewalk.c b/mm/pagewalk.c\nindex 7411702a37f58..e6493bbe6919e 100644\n--- a/mm/pagewalk.c\n+++ b/mm/pagewalk.c\n@@ -408,7 +408,7 @@ static int __walk_page_range(unsigned long start, unsigned long end,\n \tint err = 0;\n \tstruct vm_area_struct *vma = walk-\u003evma;\n \tconst struct mm_walk_ops *ops = walk-\u003eops;\n-\tbool is_hugetlb = is_vm_hugetlb_page(vma);\n+\tbool is_hugetlb = vma_is_hugetlb(vma);\n \n \t/* We do not support hugetlb PTE installation. */\n \tif (ops-\u003einstall_pte \u0026\u0026 is_hugetlb)\ndiff --git a/mm/rmap.c b/mm/rmap.c\nindex 5332c52909be1..6661bc11ce658 100644\n--- a/mm/rmap.c\n+++ b/mm/rmap.c\n@@ -2239,9 +2239,11 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,\n \n \t\t/*\n \t\t * If the folio is in an mlock()d vma, we must not swap it out.\n+\t\t * VMA_LOCKONFAULT_BIT alone marks an mlock walk in progress, see\n+\t\t * mlock_vma_pages_range().\n \t\t */\n \t\tif (!(flags \u0026 TTU_IGNORE_MLOCK) \u0026\u0026\n-\t\t (vma-\u003evm_flags \u0026 VM_LOCKED)) {\n+\t\t vma_test_any_mask(vma, VMA_LOCKED_MASK)) {\n \t\t\tptes++;\n \n \t\t\t/*\ndiff --git a/mm/swapfile.c b/mm/swapfile.c\nindex 2cd0d0ba966c3..c1c5fbb3c909d 100644\n--- a/mm/swapfile.c\n+++ b/mm/swapfile.c\n@@ -2707,7 +2707,7 @@ static int unuse_mm(struct mm_struct *mm, unsigned int type)\n \tif (check_stable_address_space(mm))\n \t\tgoto unlock;\n \tfor_each_vma(vmi, vma) {\n-\t\tif (vma-\u003eanon_vma \u0026\u0026 !is_vm_hugetlb_page(vma)) {\n+\t\tif (vma-\u003eanon_vma \u0026\u0026 !vma_is_hugetlb(vma)) {\n \t\t\tret = unuse_vma(vma, type);\n \t\t\tif (ret)\n \t\t\t\tbreak;\ndiff --git a/mm/userfaultfd.c b/mm/userfaultfd.c\nindex 79cc7b546f130..ddf0a4a3d3997 100644\n--- a/mm/userfaultfd.c\n+++ b/mm/userfaultfd.c\n@@ -237,7 +237,7 @@ static int mfill_get_vma(struct mfill_state *state)\n \tif ((flags \u0026 MFILL_ATOMIC_WP) \u0026\u0026 !(dst_vma-\u003evm_flags \u0026 VM_UFFD_WP))\n \t\tgoto out_unlock;\n \n-\tif (is_vm_hugetlb_page(dst_vma))\n+\tif (vma_is_hugetlb(dst_vma))\n \t\treturn 0;\n \n \tops = vma_uffd_ops(dst_vma);\n@@ -804,7 +804,7 @@ static __always_inline ssize_t mfill_atomic_hugetlb(\n \t\t}\n \n \t\terr = -ENOENT;\n-\t\tif (!is_vm_hugetlb_page(dst_vma))\n+\t\tif (!vma_is_hugetlb(dst_vma))\n \t\t\tgoto out_unlock_vma;\n \n \t\terr = -EINVAL;\n@@ -967,7 +967,7 @@ static __always_inline ssize_t mfill_atomic(struct userfaultfd_ctx *ctx,\n \t/*\n \t * If this is a HUGETLB vma, pass off to appropriate routine\n \t */\n-\tif (is_vm_hugetlb_page(state.vma))\n+\tif (vma_is_hugetlb(state.vma))\n \t\treturn mfill_atomic_hugetlb(ctx, state.vma, dst_start,\n \t\t\t\t\t src_start, len, flags);\n \n@@ -1114,7 +1114,7 @@ static int mwriteprotect_range(struct userfaultfd_ctx *ctx, unsigned long start,\n \t\t\tbreak;\n \t\t}\n \n-\t\tif (is_vm_hugetlb_page(dst_vma)) {\n+\t\tif (vma_is_hugetlb(dst_vma)) {\n \t\t\terr = -EINVAL;\n \t\t\tpage_mask = vma_kernel_pagesize(dst_vma) - 1;\n \t\t\tif ((start \u0026 page_mask) || (len \u0026 page_mask))\n@@ -1172,7 +1172,7 @@ int mrwprotect_range(struct userfaultfd_ctx *ctx, unsigned long start,\n \t\tif (!userfaultfd_rwp(dst_vma))\n \t\t\treturn -ENOENT;\n \n-\t\tif (is_vm_hugetlb_page(dst_vma)) {\n+\t\tif (vma_is_hugetlb(dst_vma)) {\n \t\t\tunsigned long page_mask;\n \n \t\t\tpage_mask = vma_kernel_pagesize(dst_vma) - 1;\n@@ -1754,10 +1754,18 @@ static inline bool move_splits_huge_pmd(unsigned long dst_addr,\n }\n #endif\n \n-static inline bool vma_move_compatible(struct vm_area_struct *vma)\n+static inline bool vma_move_compatible(const struct vm_area_struct *vma)\n {\n-\treturn !(vma-\u003evm_flags \u0026 (VM_PFNMAP | VM_IO | VM_HUGETLB |\n-\t\t\t\t VM_MIXEDMAP | VM_SHADOW_STACK));\n+\t/* uffd is generally incompatible with kernel-owned mappings. */\n+\tif (vma_is_kernel_owned(vma))\n+\t\treturn false;\n+\t/* The shadow stack should not be written to by userspace. */\n+\tif (vma_test_single_mask(vma, VMA_SHADOW_STACK))\n+\t\treturn false;\n+\t/* hugetlb mappings cannot be safely moved. */\n+\tif (vma_is_hugetlb(vma))\n+\t\treturn false;\n+\treturn true;\n }\n \n static int validate_move_areas(struct userfaultfd_ctx *ctx,\n@@ -2146,10 +2154,11 @@ static bool vma_can_userfault(struct vm_area_struct *vma, vm_flags_t vm_flags,\n {\n \tconst struct vm_uffd_ops *ops = vma_uffd_ops(vma);\n \n-\tif (vma-\u003evm_flags \u0026 (VM_DROPPABLE | VM_SHADOW_STACK))\n+\t/* Non-persistent memory is inherently not controllable by userspace. */\n+\tif (!vma_is_persistent(vma))\n \t\treturn false;\n-\n-\tif (!is_vm_hugetlb_page(vma) \u0026\u0026 (vma-\u003evm_flags \u0026 VM_SPECIAL))\n+\t/* The shadow stack should not be written to by userspace. */\n+\tif (vma_test_single_mask(vma, VMA_SHADOW_STACK))\n \t\treturn false;\n \n \tvm_flags \u0026= __VM_UFFD_FLAGS;\n@@ -2319,7 +2328,7 @@ static int userfaultfd_register_range(struct userfaultfd_ctx *ctx,\n \t\t */\n \t\tuserfaultfd_set_ctx(vma, ctx, vm_flags);\n \n-\t\tif (is_vm_hugetlb_page(vma) \u0026\u0026 uffd_disable_huge_pmd_share(vma))\n+\t\tif (vma_is_hugetlb(vma) \u0026\u0026 uffd_disable_huge_pmd_share(vma))\n \t\t\thugetlb_unshare_all_pmds(vma);\n \n skip:\n@@ -2895,7 +2904,7 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason)\n \t * (sleepable) vma lock can modify the current task state, that\n \t * must be before explicitly calling set_current_state().\n \t */\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\thugetlb_vma_lock_read(vma);\n \n \tspin_lock_irq(\u0026ctx-\u003efault_pending_wqh.lock);\n@@ -2912,7 +2921,7 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason)\n \tset_current_state(blocking_state);\n \tspin_unlock_irq(\u0026ctx-\u003efault_pending_wqh.lock);\n \n-\tif (is_vm_hugetlb_page(vma)) {\n+\tif (vma_is_hugetlb(vma)) {\n \t\tmust_wait = userfaultfd_huge_must_wait(ctx, vmf, reason);\n \t\thugetlb_vma_unlock_read(vma);\n \t} else {\n@@ -3744,7 +3753,7 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,\n \t * If the first vma contains huge pages, make sure start address\n \t * is aligned to huge page size.\n \t */\n-\tif (is_vm_hugetlb_page(vma)) {\n+\tif (vma_is_hugetlb(vma)) {\n \t\tunsigned long vma_hpagesize = vma_kernel_pagesize(vma);\n \n \t\tif (start \u0026 (vma_hpagesize - 1))\n@@ -3795,7 +3804,7 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,\n \t\t * If this vma contains ending address, and huge pages\n \t\t * check alignment.\n \t\t */\n-\t\tif (is_vm_hugetlb_page(cur) \u0026\u0026 end \u003c= cur-\u003evm_end \u0026\u0026\n+\t\tif (vma_is_hugetlb(cur) \u0026\u0026 end \u003c= cur-\u003evm_end \u0026\u0026\n \t\t end \u003e cur-\u003evm_start) {\n \t\t\tunsigned long vma_hpagesize = vma_kernel_pagesize(cur);\n \n@@ -3831,7 +3840,7 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,\n \t\t/*\n \t\t * Note vmas containing huge pages\n \t\t */\n-\t\tif (is_vm_hugetlb_page(cur))\n+\t\tif (vma_is_hugetlb(cur))\n \t\t\tbasic_ioctls = true;\n \n \t\tfound = true;\n@@ -3917,7 +3926,7 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx,\n \t * If the first vma contains huge pages, make sure start address\n \t * is aligned to huge page size.\n \t */\n-\tif (is_vm_hugetlb_page(vma)) {\n+\tif (vma_is_hugetlb(vma)) {\n \t\tunsigned long vma_hpagesize = vma_kernel_pagesize(vma);\n \n \t\tif (start \u0026 (vma_hpagesize - 1))\ndiff --git a/mm/util.c b/mm/util.c\nindex bf0513d1d3d08..c5ee52aede1e4 100644\n--- a/mm/util.c\n+++ b/mm/util.c\n@@ -1224,16 +1224,28 @@ EXPORT_SYMBOL(compat_set_desc_from_vma);\n int __compat_vma_mmap(struct vm_area_desc *desc,\n \t\t struct vm_area_struct *vma)\n {\n+\tstruct vm_area_desc prev_desc;\n \tint err;\n \n+\t/* Derive state prior to mmap_prepare hook. */\n+\tcompat_set_desc_from_vma(\u0026prev_desc, desc-\u003efile, vma);\n \t/* Perform any preparatory tasks for mmap action. */\n \terr = mmap_action_prepare(desc);\n \tif (err)\n-\t\treturn err;\n+\t\tgoto err_put;\n+\t/* Check the caller did nothing crazy. */\n+\terr = mmap_prepare_validate(\u0026prev_desc, desc);\n+\tif (err)\n+\t\tgoto err_put;\n \t/* Update the VMA from the descriptor. */\n \tcompat_set_vma_from_desc(vma, desc);\n \t/* Complete any specified mmap actions. */\n \treturn mmap_action_complete(vma, \u0026desc-\u003eaction, /*is_compat=*/true);\n+\n+err_put:\n+\tif (desc-\u003evm_file != vma-\u003evm_file)\n+\t\tfput(desc-\u003evm_file);\n+\treturn err;\n }\n EXPORT_SYMBOL(__compat_vma_mmap);\n \n@@ -1455,8 +1467,10 @@ int mmap_action_prepare(struct vm_area_desc *desc)\n \t\treturn io_remap_pfn_range_prepare(desc);\n \tcase MMAP_SIMPLE_IO_REMAP:\n \t\treturn simple_ioremap_prepare(desc);\n-\tcase MMAP_MAP_KERNEL_PAGES:\n+\tcase MMAP_KERNEL_PAGES:\n \t\treturn map_kernel_pages_prepare(desc);\n+\tcase MMAP_DISCONTIG_KERNEL_PAGES:\n+\t\treturn map_discontig_kernel_pages_prepare(desc);\n \t}\n \n \tWARN_ON_ONCE(1);\n@@ -1486,9 +1500,12 @@ int mmap_action_complete(struct vm_area_struct *vma,\n \tcase MMAP_REMAP_PFN:\n \t\terr = remap_pfn_range_complete(vma, action);\n \t\tbreak;\n-\tcase MMAP_MAP_KERNEL_PAGES:\n+\tcase MMAP_KERNEL_PAGES:\n \t\terr = map_kernel_pages_complete(vma, action);\n \t\tbreak;\n+\tcase MMAP_DISCONTIG_KERNEL_PAGES:\n+\t\terr = map_discontig_kernel_pages_complete(vma, action);\n+\t\tbreak;\n \tcase MMAP_IO_REMAP_PFN:\n \tcase MMAP_SIMPLE_IO_REMAP:\n \t\t/* Should have been delegated. */\n@@ -1509,7 +1526,8 @@ int mmap_action_prepare(struct vm_area_desc *desc)\n \tcase MMAP_REMAP_PFN:\n \tcase MMAP_IO_REMAP_PFN:\n \tcase MMAP_SIMPLE_IO_REMAP:\n-\tcase MMAP_MAP_KERNEL_PAGES:\n+\tcase MMAP_KERNEL_PAGES:\n+\tcase MMAP_DISCONTIG_KERNEL_PAGES:\n \t\tWARN_ON_ONCE(1); /* nommu cannot handle these. */\n \t\tbreak;\n \t}\n@@ -1530,7 +1548,8 @@ int mmap_action_complete(struct vm_area_struct *vma,\n \tcase MMAP_REMAP_PFN:\n \tcase MMAP_IO_REMAP_PFN:\n \tcase MMAP_SIMPLE_IO_REMAP:\n-\tcase MMAP_MAP_KERNEL_PAGES:\n+\tcase MMAP_KERNEL_PAGES:\n+\tcase MMAP_DISCONTIG_KERNEL_PAGES:\n \t\tWARN_ON_ONCE(1); /* nommu cannot handle this. */\n \n \t\terr = -EINVAL;\ndiff --git a/mm/vma.c b/mm/vma.c\nindex 55917d0979339..777656306705d 100644\n--- a/mm/vma.c\n+++ b/mm/vma.c\n@@ -24,7 +24,8 @@ struct mmap_state {\n \t\tvm_flags_t vm_flags;\n \t\tvma_flags_t vma_flags;\n \t};\n-\tstruct file *file;\n+\tstruct file *file;\t/* mmap()-specified file. */\n+\tstruct file *vm_file;\t/* May be updated by mmap_prepare. */\n \tpgprot_t page_prot;\n \n \t/* User-defined fields, perhaps updated by .mmap_prepare(). */\n@@ -43,8 +44,6 @@ struct mmap_state {\n \n \t/* Determine if we can check KSM flags early in mmap() logic. */\n \tbool check_ksm_early :1;\n-\t/* If .mmap_prepare changed the file, we don't need to pin. */\n-\tbool file_doesnt_need_get :1;\n };\n \n #define MMAP_STATE(name, mm_, vmi_, addr_, len_, pgoff_, anon_pgoff_, vma_flags_, file_) \\\n@@ -58,6 +57,7 @@ struct mmap_state {\n \t\t.pglen = PHYS_PFN(len_),\t\t\t\t\\\n \t\t.vma_flags = vma_flags_,\t\t\t\t\\\n \t\t.file = file_,\t\t\t\t\t\t\\\n+\t\t.vm_file = file_,\t\t\t\t\t\\\n \t\t.page_prot = vma_flags_to_page_prot(vma_flags_),\t\\\n \t}\n \n@@ -70,7 +70,7 @@ struct mmap_state {\n \t\t.vma_flags = (map_)-\u003evma_flags,\t\t\t\t\\\n \t\t.pgoff = (map_)-\u003epgoff,\t\t\t\t\t\\\n \t\t.anon_pgoff = (map_)-\u003eanon_pgoff,\t\t\t\\\n-\t\t.file = (map_)-\u003efile,\t\t\t\t\t\\\n+\t\t.file = (map_)-\u003evm_file,\t\t\t\t\\\n \t\t.prev = (map_)-\u003eprev,\t\t\t\t\t\\\n \t\t.middle = vma_,\t\t\t\t\t\t\\\n \t\t.next = (vma_) ? NULL : (map_)-\u003enext,\t\t\t\\\n@@ -599,7 +599,7 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,\n \t * boundary.\n \t */\n \tvma_adjust_trans_huge(vma, vma-\u003evm_start, addr, NULL);\n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\thugetlb_split(vma, addr);\n \n \tif (new_below) {\n@@ -924,13 +924,14 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(\n \n \tvmg-\u003estate = VMA_MERGE_NOMERGE;\n \n+\tif (!vma_flags_can_merge(\u0026vmg-\u003evma_flags))\n+\t\treturn NULL;\n \t/*\n-\t * If a special mapping or if the range being modified is neither at the\n-\t * furthermost left or right side of the VMA, then we have no chance of\n-\t * merging and should abort.\n+\t * If the range being modified is neither at the furthermost left or\n+\t * right side of the VMA, then we have no chance of merging and should\n+\t * abort.\n \t */\n-\tif (vma_flags_test_any_mask(\u0026vmg-\u003evma_flags, VMA_SPECIAL_FLAGS) ||\n-\t (!left_side \u0026\u0026 !right_side))\n+\tif (!left_side \u0026\u0026 !right_side)\n \t\treturn NULL;\n \n \tif (left_side)\n@@ -1152,9 +1153,11 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)\n \n \tvmg-\u003estate = VMA_MERGE_NOMERGE;\n \n-\t/* Special VMAs are unmergeable, also if no prev/next. */\n-\tif (vma_flags_test_any_mask(\u0026vmg-\u003evma_flags, VMA_SPECIAL_FLAGS) ||\n-\t (!prev \u0026\u0026 !next))\n+\tif (!vma_flags_can_merge(\u0026vmg-\u003evma_flags))\n+\t\treturn NULL;\n+\n+\t/* VMAs with no prev/next are unmergeable. */\n+\tif (!prev \u0026\u0026 !next)\n \t\treturn NULL;\n \n \tcan_merge_left = can_vma_merge_left(vmg);\n@@ -2233,7 +2236,7 @@ bool vma_wants_writenotify(struct vm_area_struct *vma, pgprot_t vm_page_prot)\n \t * Do we need to track softdirty? hugetlb does not support softdirty\n \t * tracking yet.\n \t */\n-\tif (vma_soft_dirty_enabled(vma) \u0026\u0026 !is_vm_hugetlb_page(vma))\n+\tif (vma_soft_dirty_enabled(vma) \u0026\u0026 !vma_is_hugetlb(vma))\n \t\treturn true;\n \n \t/* Do we need write faults for uffd-wp tracking? */\n@@ -2352,7 +2355,7 @@ int mm_take_all_locks(struct mm_struct *mm)\n \t\tif (signal_pending(current))\n \t\t\tgoto out_unlock;\n \t\tif (vma-\u003evm_file \u0026\u0026 vma-\u003evm_file-\u003ef_mapping \u0026\u0026\n-\t\t\t\tis_vm_hugetlb_page(vma))\n+\t\t\t\tvma_is_hugetlb(vma))\n \t\t\tvm_lock_mapping(mm, vma-\u003evm_file-\u003ef_mapping);\n \t}\n \n@@ -2361,7 +2364,7 @@ int mm_take_all_locks(struct mm_struct *mm)\n \t\tif (signal_pending(current))\n \t\t\tgoto out_unlock;\n \t\tif (vma-\u003evm_file \u0026\u0026 vma-\u003evm_file-\u003ef_mapping \u0026\u0026\n-\t\t\t\t!is_vm_hugetlb_page(vma))\n+\t\t\t\t!vma_is_hugetlb(vma))\n \t\t\tvm_lock_mapping(mm, vma-\u003evm_file-\u003ef_mapping);\n \t}\n \n@@ -2447,7 +2450,7 @@ void mm_drop_all_locks(struct mm_struct *mm)\n */\n static bool accountable_mapping(struct mmap_state *map)\n {\n-\tconst struct file *file = map-\u003efile;\n+\tconst struct file *file = map-\u003evm_file;\n \n \t/*\n \t * hugetlb has its own accounting separate from the core VM\n@@ -2496,7 +2499,7 @@ static void vms_abort_munmap_vmas(struct vma_munmap_struct *vms,\n \n static void update_ksm_flags(struct mmap_state *map)\n {\n-\tmap-\u003evma_flags = ksm_vma_flags(map-\u003emm, map-\u003efile, map-\u003evma_flags);\n+\tmap-\u003evma_flags = ksm_vma_flags(map-\u003emm, map-\u003evm_file, map-\u003evma_flags);\n }\n \n static void set_desc_from_map(struct vm_area_desc *desc,\n@@ -2506,7 +2509,7 @@ static void set_desc_from_map(struct vm_area_desc *desc,\n \tdesc-\u003eend = map-\u003eend;\n \n \tdesc-\u003epgoff = map-\u003epgoff;\n-\tdesc-\u003evm_file = map-\u003efile;\n+\tdesc-\u003evm_file = map-\u003evm_file;\n \tdesc-\u003evma_flags = map-\u003evma_flags;\n \tdesc-\u003epage_prot = map-\u003epage_prot;\n }\n@@ -2586,6 +2589,10 @@ static int __mmap_setup(struct mmap_state *map, struct vm_area_desc *desc,\n \treturn 0;\n }\n \n+static bool map_same_file(struct mmap_state *map)\n+{\n+\treturn map-\u003evm_file == map-\u003efile;\n+}\n \n static int __mmap_new_file_vma(struct mmap_state *map,\n \t\t\t struct vm_area_struct *vma)\n@@ -2593,37 +2600,43 @@ static int __mmap_new_file_vma(struct mmap_state *map,\n \tstruct vma_iterator *vmi = map-\u003evmi;\n \tint error;\n \n-\tvma-\u003evm_file = map-\u003efile;\n-\tif (!map-\u003efile_doesnt_need_get)\n-\t\tget_file(map-\u003efile);\n+\tvma-\u003evm_file = map-\u003evm_file;\n+\tif (map_same_file(map))\n+\t\tget_file(map-\u003evm_file);\n \n-\tif (!map-\u003efile-\u003ef_op-\u003emmap)\n+\tif (!map-\u003evm_file-\u003ef_op-\u003emmap)\n \t\treturn 0;\n \n+\t/*\n+\t * Driver-specified flags may make the lock flags invalid, so clear\n+\t * VMA_LOCKED_MASK and reinstate it afterwards if appropriate.\n+\t */\n+\tvma_clear_flags_mask(vma, VMA_LOCKED_MASK);\n \terror = mmap_file(vma-\u003evm_file, vma);\n+\tmap-\u003evm_file = vma-\u003evm_file;\n+\n \tif (error) {\n \t\tUNMAP_STATE(unmap, vmi, vma, vma-\u003evm_start, vma-\u003evm_end,\n \t\t\t map-\u003eprev, map-\u003enext);\n-\t\tfput(vma-\u003evm_file);\n-\t\tvma-\u003evm_file = NULL;\n+\t\tif (map_same_file(map))\n+\t\t\tfput(map-\u003evm_file);\n \n+\t\tvma-\u003evm_file = NULL;\n \t\tvma_iter_set(vmi, vma-\u003evm_end);\n \t\t/* Undo any partial mapping done by a device driver. */\n \t\tunmap_region(\u0026unmap);\n \t\treturn error;\n \t}\n \n-\t/* Drivers cannot alter the address of the VMA. */\n-\tWARN_ON_ONCE(map-\u003eaddr != vma-\u003evm_start);\n-\t/*\n-\t * Drivers should not permit writability when previously it was\n-\t * disallowed.\n-\t */\n-\tVM_WARN_ON_ONCE(!vma_flags_same_pair(\u0026map-\u003evma_flags, \u0026vma-\u003eflags) \u0026\u0026\n-\t\t\t!vma_flags_test(\u0026map-\u003evma_flags, VMA_MAYWRITE_BIT) \u0026\u0026\n-\t\t\tvma_test(vma, VMA_MAYWRITE_BIT));\n+\t/* If VMA flags still valid for locked mask, reinstate. */\n+\tif (vma_supports_mlock(vma)) {\n+\t\tconst vma_flags_t mask =\n+\t\t\tvma_flags_and_mask(\u0026map-\u003evma_flags,\n+\t\t\t\t\t VMA_LOCKED_MASK);\n+\n+\t\tvma_set_flags_mask(vma, mask);\n+\t}\n \n-\tmap-\u003efile = vma-\u003evm_file;\n \tmap-\u003evma_flags = vma-\u003eflags;\n \n \treturn 0;\n@@ -2631,7 +2644,7 @@ static int __mmap_new_file_vma(struct mmap_state *map,\n \n static void map_set_anon(struct mmap_state *map)\n {\n-\tmap-\u003efile = NULL;\n+\tmap-\u003evm_file = NULL;\n \tmap-\u003evm_ops = NULL;\n \tmap-\u003epgoff = map-\u003eaddr \u003e\u003e PAGE_SHIFT;\n }\n@@ -2643,7 +2656,7 @@ static bool map_is_private(const struct mmap_state *map)\n \n static bool map_is_anon(const struct mmap_state *map)\n {\n-\treturn map_is_private(map) \u0026\u0026 !map-\u003efile;\n+\treturn map_is_private(map) \u0026\u0026 !map-\u003evm_file;\n }\n \n /*\n@@ -2688,7 +2701,7 @@ static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap,\n \t}\n \n \t/* Invoke callbacks. */\n-\tif (map-\u003efile)\n+\tif (map-\u003evm_file)\n \t\terror = __mmap_new_file_vma(map, vma);\n \telse if (!is_anon)\n \t\terror = shmem_zero_setup(vma);\n@@ -2701,11 +2714,6 @@ static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap,\n \t\tvma-\u003eflags = map-\u003evma_flags;\n \t}\n \n-#ifdef CONFIG_SPARC64\n-\t/* TODO: Fix SPARC ADI! */\n-\tWARN_ON_ONCE(!arch_validate_flags(map-\u003evm_flags));\n-#endif\n-\n \t/* Lock the VMA since it is modified after insertion into VMA tree */\n \tvma_start_write(vma);\n \tvma_iter_store_new(vmi, vma);\n@@ -2768,6 +2776,96 @@ static void __mmap_complete(struct mmap_state *map, struct vm_area_struct *vma)\n \tvma_set_page_prot(vma);\n }\n \n+/* Check to ensure that the VMA flags of a newly mapped VMA are sane. */\n+static int mmap_validate_vma_flags(const vma_flags_t *flags)\n+{\n+#ifdef CONFIG_SPARC64\n+\tconst vm_flags_t legacy_flags = vma_flags_to_legacy(*flags);\n+\n+\t/* TODO: Fix SPARC ADI! */\n+\tif (WARN_ON_ONCE(!arch_validate_flags(legacy_flags)))\n+\t\treturn -EINVAL;\n+#endif\n+\n+\tif (!vma_flags_is_kernel_owned(flags)) {\n+\t\t/* Only kernel-owned mappings may set VMA_IO_BIT. */\n+\t\tif (WARN_ON_ONCE(vma_flags_test(flags, VMA_IO_BIT)))\n+\t\t\treturn -EINVAL;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+/* Check to ensure a driver hasn't done something crazy. */\n+static int mmap_validate(unsigned long prev_start,\n+\t\t\t unsigned long curr_start,\n+\t\t\t const vma_flags_t *prev_flags,\n+\t\t\t const vma_flags_t *curr_flags)\n+{\n+\tbool was_maywrite, is_maywrite;\n+\n+\t/* Drivers cannot alter the address of the VMA. */\n+\tif (WARN_ON_ONCE(prev_start != curr_start))\n+\t\treturn -EINVAL;\n+\n+\twas_maywrite = vma_flags_test(prev_flags, VMA_MAYWRITE_BIT);\n+\tis_maywrite = vma_flags_test(curr_flags, VMA_MAYWRITE_BIT);\n+\n+\t/* A driver may not make a previously unwritable mapping writable. */\n+\tif (WARN_ON_ONCE(!was_maywrite \u0026\u0026 is_maywrite))\n+\t\treturn -EINVAL;\n+\n+\t/* Only kernel-owned mappings may clear VMA_MAYWRITE_BIT. */\n+\tif (!vma_flags_is_kernel_owned(curr_flags) \u0026\u0026\n+\t WARN_ON_ONCE(was_maywrite \u0026\u0026 !is_maywrite))\n+\t\treturn -EINVAL;\n+\n+\treturn mmap_validate_vma_flags(curr_flags);\n+}\n+\n+/**\n+ * mmap_prepare_validate() - Ensure the driver hasn't violated invariants in its\n+ * f_op-\u003emmap_prepare hook.\n+ * @prev_desc: The VMA descriptor prior to the mmap_prepare hook being called.\n+ * @desc: The VMA descriptor after the mmap_prepare hook has been called.\n+ *\n+ * Returns: 0 on success, otherwise an error.\n+ */\n+int mmap_prepare_validate(const struct vm_area_desc *prev_desc,\n+\t\t\t const struct vm_area_desc *desc)\n+{\n+\t/*\n+\t * It is not valid to execute mmap actions for VMAs which can be merged,\n+\t * as any such merge would leave portions of the mapping incorrectly\n+\t * unmapped.\n+\t */\n+\tif (vma_flags_can_merge(\u0026desc-\u003evma_flags) \u0026\u0026\n+\t WARN_ON_ONCE(desc-\u003eaction.type != MMAP_NOTHING))\n+\t\treturn -EINVAL;\n+\n+\treturn mmap_validate(prev_desc-\u003estart, desc-\u003estart,\n+\t\t\t \u0026prev_desc-\u003evma_flags, \u0026desc-\u003evma_flags);\n+}\n+\n+/**\n+ * mmap_hook_validate() - Ensure the driver hasn't violated invariants in\n+ * its f_op-\u003emmap hook.\n+ * @prev_start: The start of the mapping prior to the mmap hook.\n+ * @prev_flags: The VMA flags set for the VMA prior to the mmap hook.\n+ * @vma: The VMA after the hook has been applied.\n+ *\n+ * Returns: 0 on success, otherwise an error.\n+ */\n+int mmap_hook_validate(unsigned long prev_start,\n+\t\t const vma_flags_t *prev_flags,\n+\t\t const struct vm_area_struct *vma)\n+{\n+\tconst unsigned long start = vma-\u003evm_start;\n+\tconst vma_flags_t *flags = \u0026vma-\u003eflags;\n+\n+\treturn mmap_validate(prev_start, start, prev_flags, flags);\n+}\n+\n static int call_action_prepare(struct mmap_state *map,\n \t\t\t struct vm_area_desc *desc)\n {\n@@ -2794,39 +2892,43 @@ static int call_action_prepare(struct mmap_state *map,\n static int call_mmap_prepare(struct mmap_state *map,\n \t\tstruct vm_area_desc *desc)\n {\n+\tconst struct vm_area_desc prev_desc = *desc;\n \tint err;\n \n \t/* Invoke the hook. */\n-\terr = vfs_mmap_prepare(map-\u003efile, desc);\n-\tif (err)\n-\t\treturn err;\n-\n-\t/* It's invalid for mmap_preprare hooks to clear vm_ops. */\n-\tif (!desc-\u003evm_ops)\n-\t\treturn -EINVAL;\n-\n-\terr = call_action_prepare(map, desc);\n+\terr = vfs_mmap_prepare(map-\u003evm_file, desc);\n \tif (err)\n \t\treturn err;\n \n \t/* Update fields permitted to be changed. */\n \tmap-\u003epgoff = desc-\u003epgoff;\n-\tif (desc-\u003evm_file != map-\u003efile) {\n-\t\tmap-\u003efile_doesnt_need_get = true;\n-\t\tmap-\u003efile = desc-\u003evm_file;\n-\t}\n+\tif (desc-\u003evm_file != map-\u003evm_file)\n+\t\tmap-\u003evm_file = desc-\u003evm_file;\n \tmap-\u003evma_flags = desc-\u003evma_flags;\n \tmap-\u003epage_prot = desc-\u003epage_prot;\n \t/* User-defined fields. */\n \tmap-\u003evm_ops = desc-\u003evm_ops;\n \tmap-\u003evm_private_data = desc-\u003eprivate_data;\n \n+\t/* It's invalid for mmap_prepare hooks to clear vm_ops. */\n+\tif (!desc-\u003evm_ops)\n+\t\treturn -EINVAL;\n+\n+\terr = call_action_prepare(map, desc);\n+\tif (err)\n+\t\treturn err;\n+\n+\t/* Check the caller did nothing crazy. */\n+\terr = mmap_prepare_validate(\u0026prev_desc, desc);\n+\tif (err)\n+\t\treturn err;\n+\n \t/*\n \t * MAP_PRIVATE-/dev/zero mappings are an ancient way of getting\n \t * anonymous mappings. Rather than allowing these mappings to be odd\n \t * outliers, simply make them truly anonymous.\n \t */\n-\tif (map_is_private(map) \u0026\u0026 file_is_dev_zero(map-\u003efile))\n+\tif (map_is_private(map) \u0026\u0026 file_is_dev_zero(map-\u003evm_file))\n \t\tmap_set_anon(map);\n \n \treturn 0;\n@@ -2845,7 +2947,7 @@ static void set_vma_user_defined_fields(struct vm_area_struct *vma,\n */\n static bool can_set_ksm_flags_early(struct mmap_state *map)\n {\n-\tstruct file *file = map-\u003efile;\n+\tstruct file *file = map-\u003evm_file;\n \n \t/* Anonymous mappings have no driver which can change them. */\n \tif (!file)\n@@ -2868,13 +2970,27 @@ static bool can_set_ksm_flags_early(struct mmap_state *map)\n \treturn false;\n }\n \n+static void put_map(struct mmap_state *map)\n+{\n+\t/*\n+\t * An error occurred or the VMA was merged.\n+\t *\n+\t * If the file was changed by the driver (which is required to increment\n+\t * the replacement file's reference count), drop its reference count.\n+\t *\n+\t * On error, the caller always drops the original file regardless.\n+\t */\n+\tif (map-\u003evm_file \u0026\u0026 !map_same_file(map))\n+\t\tfput(map-\u003evm_file);\n+}\n+\n static unsigned long __mmap_region(struct file *file, unsigned long addr,\n \t\tunsigned long len, vma_flags_t vma_flags,\n \t\tunsigned long pgoff, struct list_head *uf)\n {\n \tstruct mm_struct *mm = current-\u003emm;\n \tstruct vm_area_struct *vma = NULL;\n-\tbool have_mmap_prepare = file \u0026\u0026 file-\u003ef_op-\u003emmap_prepare;\n+\tconst bool have_mmap_prepare = file \u0026\u0026 file-\u003ef_op-\u003emmap_prepare;\n \tVMA_ITERATOR(vmi, mm, addr);\n \tconst pgoff_t anon_pgoff = addr \u003e\u003e PAGE_SHIFT;\n \tMMAP_STATE(map, mm, \u0026vmi, addr, len, pgoff, anon_pgoff, vma_flags, file);\n@@ -2917,12 +3033,15 @@ static unsigned long __mmap_region(struct file *file, unsigned long addr,\n \t\tallocated_new = true;\n \t}\n \n-\tif (have_mmap_prepare \u0026\u0026 !map_is_anon(\u0026map))\n+\tif (have_mmap_prepare \u0026\u0026 allocated_new \u0026\u0026 !map_is_anon(\u0026map))\n \t\tset_vma_user_defined_fields(vma, \u0026map);\n \n \t__mmap_complete(\u0026map, vma);\n \n-\tif (have_mmap_prepare \u0026\u0026 allocated_new) {\n+\tif (!allocated_new) {\n+\t\t/* Merged, so need to drop refcount. */\n+\t\tput_map(\u0026map);\n+\t} else if (have_mmap_prepare) {\n \t\terror = mmap_action_complete(vma, \u0026desc.action,\n \t\t\t\t\t /*is_compat=*/false);\n \t\tif (error)\n@@ -2936,13 +3055,7 @@ static unsigned long __mmap_region(struct file *file, unsigned long addr,\n \tif (map.charged)\n \t\tvm_unacct_memory(map.charged);\n abort_munmap:\n-\t/*\n-\t * This indicates that .mmap_prepare has set a new file, differing from\n-\t * desc-\u003evm_file. But since we're aborting the operation, only the\n-\t * original file will be cleaned up. Ensure we clean up both.\n-\t */\n-\tif (map.file_doesnt_need_get)\n-\t\tfput(map.file);\n+\tput_map(\u0026map);\n \tvms_abort_munmap_vmas(\u0026map.vms, \u0026map.mas_detach);\n \treturn error;\n }\n@@ -3437,10 +3550,15 @@ int __vm_munmap(unsigned long start, size_t len, bool unlock)\n int insert_vm_struct(struct mm_struct *mm, struct vm_area_struct *vma)\n {\n \tunsigned long charged = vma_pages(vma);\n+\tint err;\n \n \tif (find_vma_intersection(mm, vma-\u003evm_start, vma-\u003evm_end))\n \t\treturn -ENOMEM;\n \n+\terr = mmap_validate_vma_flags(\u0026vma-\u003eflags);\n+\tif (err)\n+\t\treturn err;\n+\n \tif (vma_test(vma, VMA_ACCOUNT_BIT) \u0026\u0026\n \t security_vm_enough_memory_mm(mm, charged))\n \t\treturn -ENOMEM;\ndiff --git a/mm/vma.h b/mm/vma.h\nindex e97bd2dfa786d..77d395b8b1032 100644\n--- a/mm/vma.h\n+++ b/mm/vma.h\n@@ -394,8 +394,10 @@ static inline void compat_set_vma_from_desc(struct vm_area_struct *vma,\n \n \t/* Mutable fields. Populated with initial state. */\n \tvma_set_pgoff(vma, desc-\u003epgoff);\n-\tif (desc-\u003evm_file != vma-\u003evm_file)\n-\t\tvma_set_file(vma, desc-\u003evm_file);\n+\tif (desc-\u003evm_file != vma-\u003evm_file) {\n+\t\tfput(vma-\u003evm_file);\n+\t\tvma-\u003evm_file = desc-\u003evm_file;\n+\t}\n \tvma-\u003eflags = desc-\u003evma_flags;\n \tvma-\u003evm_page_prot = desc-\u003epage_prot;\n \n@@ -780,14 +782,19 @@ struct vm_area_struct *vm_area_alloc(struct mm_struct *mm);\n struct vm_area_struct *vm_area_dup(struct vm_area_struct *orig);\n void vm_area_free(struct vm_area_struct *vma);\n \n-/* vma_exec.c */\n #ifdef CONFIG_MMU\n+int mmap_prepare_validate(const struct vm_area_desc *prev_desc,\n+\t\t\t const struct vm_area_desc *desc);\n+\n+int mmap_hook_validate(unsigned long prev_start,\n+\t\t const vma_flags_t *prev_flags,\n+\t\t const struct vm_area_struct *vma);\n+\n+/* vma_exec.c */\n int create_init_stack_vma(struct mm_struct *mm, struct vm_area_struct **vmap,\n \t\t\t unsigned long *top_mem_p);\n int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift);\n-#endif\n \n-#ifdef CONFIG_MMU\n /*\n * Denies creating a writable executable mapping or gaining executable permissions.\n *\n@@ -836,6 +843,19 @@ static inline bool map_deny_write_exec(const vma_flags_t *old,\n \n \treturn false;\n }\n+#else\n+static inline int mmap_prepare_validate(const struct vm_area_desc *prev_desc,\n+\t\t\t\t\tconst struct vm_area_desc *desc)\n+{\n+\treturn 0;\n+}\n+\n+static inline int mmap_hook_validate(unsigned long prev_start,\n+\t\t\t\t const vma_flags_t *prev_flags,\n+\t\t\t\t const struct vm_area_struct *vma)\n+{\n+\treturn 0;\n+}\n #endif\n \n struct vm_area_struct *__install_special_mapping(struct mm_struct *mm,\ndiff --git a/mm/vma_internal.h b/mm/vma_internal.h\nindex 4d300e7bbaf4c..4f73f0a4db796 100644\n--- a/mm/vma_internal.h\n+++ b/mm/vma_internal.h\n@@ -18,7 +18,6 @@\n #include \u003clinux/fs.h\u003e\n #include \u003clinux/huge_mm.h\u003e\n #include \u003clinux/hugetlb.h\u003e\n-#include \u003clinux/hugetlb_inline.h\u003e\n #include \u003clinux/kernel.h\u003e\n #include \u003clinux/ksm.h\u003e\n #include \u003clinux/khugepaged.h\u003e\ndiff --git a/mm/vmscan.c b/mm/vmscan.c\nindex aaceed4759eeb..001f8b760266b 100644\n--- a/mm/vmscan.c\n+++ b/mm/vmscan.c\n@@ -3471,13 +3471,14 @@ static int should_skip_vma(unsigned long start, unsigned long end, struct mm_wal\n \tif (!vma_is_accessible(vma))\n \t\treturn true;\n \n-\tif (is_vm_hugetlb_page(vma))\n+\tif (vma_is_hugetlb(vma))\n \t\treturn true;\n \n \tif (!vma_has_recency(vma))\n \t\treturn true;\n \n-\tif (vma-\u003evm_flags \u0026 (VM_LOCKED | VM_SPECIAL))\n+\tif (vma_test(vma, VMA_LOCKED_BIT) || vma_is_kernel_owned(vma) ||\n+\t vma_is_fixed_mapping(vma))\n \t\treturn true;\n \n \tif (vma == get_gate_vma(vma-\u003evm_mm))\n@@ -4417,8 +4418,8 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr)\n \tif (spin_is_contended(pvmw-\u003eptl))\n \t\treturn true;\n \n-\t/* exclude special VMAs containing anon pages from COW */\n-\tif (vma-\u003evm_flags \u0026 VM_SPECIAL)\n+\t/* exclude kernel-owned and fixed VMAs containing anon pages from COW */\n+\tif (vma_is_kernel_owned(vma) || vma_is_fixed_mapping(vma))\n \t\treturn true;\n \n \t/* avoid taking the LRU lock under the PTL when possible */\ndiff --git a/security/selinux/selinuxfs.c b/security/selinux/selinuxfs.c\nindex c7d91476971cb..545a6f89f9e76 100644\n--- a/security/selinux/selinuxfs.c\n+++ b/security/selinux/selinuxfs.c\n@@ -340,6 +340,9 @@ static int sel_open_policy(struct inode *inode, struct file *filp)\n \tstruct policy_load_memory *plm = NULL;\n \tint rc;\n \n+\tif (filp-\u003ef_mode \u0026 FMODE_WRITE)\n+\t\treturn -EACCES;\n+\n \trc = avc_has_perm(current_sid(), SECINITSID_SECURITY,\n \t\t\t SECCLASS_SECURITY, SECURITY__READ_POLICY, NULL);\n \tif (rc)\n@@ -424,14 +427,6 @@ static const struct vm_operations_struct sel_mmap_policy_ops = {\n \n static int sel_mmap_policy(struct file *filp, struct vm_area_struct *vma)\n {\n-\tif (vma-\u003evm_flags \u0026 VM_SHARED) {\n-\t\t/* do not allow mprotect to make mapping writable */\n-\t\tvm_flags_clear(vma, VM_MAYWRITE);\n-\n-\t\tif (vma-\u003evm_flags \u0026 VM_WRITE)\n-\t\t\treturn -EACCES;\n-\t}\n-\n \tvm_flags_set(vma, VM_DONTEXPAND | VM_DONTDUMP);\n \tvma-\u003evm_ops = \u0026sel_mmap_policy_ops;\n \ndiff --git a/sound/core/pcm_native.c b/sound/core/pcm_native.c\nindex 62324282fcae9..c98fb3df14f34 100644\n--- a/sound/core/pcm_native.c\n+++ b/sound/core/pcm_native.c\n@@ -3760,39 +3760,27 @@ static __poll_t snd_pcm_poll(struct file *file, poll_table *wait)\n /*\n * mmap status record\n */\n-static vm_fault_t snd_pcm_mmap_status_fault(struct vm_fault *vmf)\n+static int snd_pcm_mmap_status(struct snd_pcm_substream *substream, struct file *file,\n+\t\t\t struct vm_area_struct *vma)\n {\n-\tstruct snd_pcm_substream *substream = vmf-\u003evma-\u003evm_private_data;\n+\tconst unsigned long size = vma-\u003evm_end - vma-\u003evm_start;\n \tstruct snd_pcm_runtime *runtime;\n-\t\n-\tif (substream == NULL)\n-\t\treturn VM_FAULT_SIGBUS;\n-\truntime = substream-\u003eruntime;\n-\tvmf-\u003epage = virt_to_page(runtime-\u003estatus);\n-\tget_page(vmf-\u003epage);\n-\treturn 0;\n-}\n+\tstruct page *page;\n \n-static const struct vm_operations_struct snd_pcm_vm_ops_status =\n-{\n-\t.fault =\tsnd_pcm_mmap_status_fault,\n-};\n+\tBUILD_BUG_ON(sizeof(struct snd_pcm_mmap_status) \u003e PAGE_SIZE);\n \n-static int snd_pcm_mmap_status(struct snd_pcm_substream *substream, struct file *file,\n-\t\t\t struct vm_area_struct *area)\n-{\n-\tlong size;\n-\tif (!(area-\u003evm_flags \u0026 VM_READ))\n+\tif (!(vma-\u003evm_flags \u0026 VM_READ))\n \t\treturn -EINVAL;\n-\tsize = area-\u003evm_end - area-\u003evm_start;\n-\tif (size != PAGE_ALIGN(sizeof(struct snd_pcm_mmap_status)))\n+\tif (size != PAGE_SIZE)\n \t\treturn -EINVAL;\n-\tarea-\u003evm_ops = \u0026snd_pcm_vm_ops_status;\n-\tarea-\u003evm_private_data = substream;\n-\tvm_flags_mod(area, VM_DONTEXPAND | VM_DONTDUMP,\n+\n+\tvm_flags_mod(vma, VM_DONTEXPAND | VM_DONTDUMP,\n \t\t VM_WRITE | VM_MAYWRITE);\n+\tvma-\u003evm_page_prot = vm_get_page_prot(vma-\u003evm_flags);\n \n-\treturn 0;\n+\truntime = substream-\u003eruntime;\n+\tpage = virt_to_page(runtime-\u003estatus);\n+\treturn vm_insert_page(vma, vma-\u003evm_start, page);\n }\n \n /*\ndiff --git a/tools/testing/vma/include/dup.h b/tools/testing/vma/include/dup.h\nindex 16c09dac59d9b..dc24f43a9394c 100644\n--- a/tools/testing/vma/include/dup.h\n+++ b/tools/testing/vma/include/dup.h\n@@ -352,14 +352,6 @@ enum {\n #define VM_ACCESS_FLAGS (VM_READ | VM_WRITE | VM_EXEC)\n #define VMA_ACCESS_FLAGS mk_vma_flags(VMA_READ_BIT, VMA_WRITE_BIT, VMA_EXEC_BIT)\n \n-/*\n- * Special vmas that are non-mergable, non-mlock()able.\n- */\n-#define VM_SPECIAL (VM_IO | VM_DONTEXPAND | VM_PFNMAP | VM_MIXEDMAP)\n-\n-#define VMA_SPECIAL_FLAGS mk_vma_flags(VMA_IO_BIT, VMA_DONTEXPAND_BIT, \\\n-\t\t\t\t VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT)\n-\n #define VMA_REMAP_FLAGS mk_vma_flags(VMA_IO_BIT, VMA_PFNMAP_BIT,\t\\\n \t\t\t\t VMA_DONTEXPAND_BIT, VMA_DONTDUMP_BIT)\n \n@@ -454,17 +446,20 @@ static __always_inline bool vma_flags_empty(const vma_flags_t *flags)\n \n /* What action should be taken after an .mmap_prepare call is complete? */\n enum mmap_action_type {\n-\tMMAP_NOTHING,\t\t/* Mapping is complete, no further action. */\n-\tMMAP_REMAP_PFN,\t\t/* Remap PFN range. */\n-\tMMAP_IO_REMAP_PFN,\t/* I/O remap PFN range. */\n-\tMMAP_SIMPLE_IO_REMAP,\t/* I/O remap with guardrails. */\n-\tMMAP_MAP_KERNEL_PAGES,\t/* Map kernel page range from an array. */\n+\tMMAP_NOTHING,\n+\tMMAP_REMAP_PFN,\n+\tMMAP_IO_REMAP_PFN,\n+\tMMAP_SIMPLE_IO_REMAP,\t\t/* I/O remap with guardrails. */\n+\tMMAP_KERNEL_PAGES,\t\t/* Map kernel page range from array. */\n+\tMMAP_DISCONTIG_KERNEL_PAGES,\t/* Map kernel discontig page range. */\n };\n \n /*\n * Describes an action an mmap_prepare hook can instruct to be taken to complete\n * the mapping of a VMA. Specified in vm_area_desc.\n */\n+struct discontig_kernel_page_ops;\n+\n struct mmap_action {\n \tunion {\n \t\tstruct {\n@@ -483,6 +478,10 @@ struct mmap_action {\n \t\t\tunsigned long nr_pages;\n \t\t\tpgoff_t pgoff;\n \t\t} map_kernel;\n+\t\tstruct {\n+\t\t\tvoid *init_private;\n+\t\t\tconst struct discontig_kernel_page_ops *ops;\n+\t\t} map_kernel_discontig;\n \t};\n \tenum mmap_action_type type;\n \n@@ -1359,13 +1358,23 @@ static inline int vfs_mmap_prepare(struct file *file, struct vm_area_desc *desc)\n \treturn file-\u003ef_op-\u003emmap_prepare(desc);\n }\n \n+int mmap_prepare_validate(const struct vm_area_desc *prev_desc,\n+\t\t\t const struct vm_area_desc *desc);\n+\n static inline int __compat_vma_mmap(struct vm_area_desc *desc,\n \t\tstruct vm_area_struct *vma)\n {\n+\tstruct vm_area_desc prev_desc;\n \tint err;\n \n+\t/* Derive state prior to mmap_prepare hook. */\n+\tcompat_set_desc_from_vma(\u0026prev_desc, desc-\u003efile, vma);\n \t/* Perform any preparatory tasks for mmap action. */\n \terr = mmap_action_prepare(desc);\n+\tif (err)\n+\t\treturn err;\n+\t/* Check the caller did nothing crazy. */\n+\terr = mmap_prepare_validate(\u0026prev_desc, desc);\n \tif (err)\n \t\treturn err;\n \t/* Update the VMA from the descriptor. */\n@@ -1647,3 +1656,34 @@ static inline bool file_is_dev_zero(const struct file *file)\n {\n \treturn file \u0026\u0026 file-\u003ef_op == \u0026zero_fops;\n }\n+\n+static inline bool vma_flags_is_kernel_owned(const vma_flags_t *flags)\n+{\n+\treturn vma_flags_test_any(flags, VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT);\n+}\n+\n+static inline bool vma_is_kernel_owned(const struct vm_area_struct *vma)\n+{\n+\treturn vma_flags_is_kernel_owned(\u0026vma-\u003eflags);\n+}\n+\n+static inline bool vma_flags_can_merge(const vma_flags_t *flags)\n+{\n+\t/*\n+\t * VMA merging assumes that the properties of a VMA completely describe\n+\t * the properties of that VMA.\n+\t *\n+\t * However, kernel-owned mappings may have established state upon mapping\n+\t * not embodied in any attribute of the VMA.\n+\t *\n+\t * Additionally, PFN maps encode the source PFN of the range in\n+\t * vma-\u003evm_pgoff, which may otherwise cause spurious merges.\n+\t */\n+\tif (vma_flags_is_kernel_owned(flags))\n+\t\treturn false;\n+\t/* VMA explicitly marked as being unmergeable. */\n+\tif (vma_flags_test(flags, VMA_DONTEXPAND_BIT))\n+\t\treturn false;\n+\n+\treturn true;\n+}\ndiff --git a/tools/testing/vma/include/stubs.h b/tools/testing/vma/include/stubs.h\nindex d6136e19a8af3..48d1dc53df42c 100644\n--- a/tools/testing/vma/include/stubs.h\n+++ b/tools/testing/vma/include/stubs.h\n@@ -193,7 +193,7 @@ static inline bool mapping_can_writeback(struct address_space *mapping)\n \treturn true;\n }\n \n-static inline bool is_vm_hugetlb_page(struct vm_area_struct *vma)\n+static inline bool vma_is_hugetlb(struct vm_area_struct *vma)\n {\n \treturn false;\n }\ndiff --git a/tools/testing/vma/tests/merge.c b/tools/testing/vma/tests/merge.c\nindex acaab282939c0..b26f1a66a1707 100644\n--- a/tools/testing/vma/tests/merge.c\n+++ b/tools/testing/vma/tests/merge.c\n@@ -496,17 +496,11 @@ static bool test_vma_merge_special_flags(void)\n \t\t.mm = \u0026mm,\n \t\t.vmi = \u0026vmi,\n \t};\n-\tvma_flag_t special_flags[] = { VMA_IO_BIT, VMA_DONTEXPAND_BIT,\n+\tvma_flag_t special_flags[] = { VMA_DONTEXPAND_BIT,\n \t\tVMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT };\n-\tvma_flags_t all_special_flags = EMPTY_VMA_FLAGS;\n \tint i;\n \tstruct vm_area_struct *vma_left, *vma;\n \n-\t/* Make sure there aren't new VM_SPECIAL flags. */\n-\tfor (i = 0; i \u003c ARRAY_SIZE(special_flags); i++)\n-\t\tvma_flags_set(\u0026all_special_flags, special_flags[i]);\n-\tASSERT_FLAGS_SAME_MASK(\u0026all_special_flags, VMA_SPECIAL_FLAGS);\n-\n \t/*\n \t * 01234\n \t * AAA\n@@ -520,7 +514,7 @@ static bool test_vma_merge_special_flags(void)\n \t * 01234\n \t * AAA*\n \t *\n-\t * This should merge if not for the VM_SPECIAL flag.\n+\t * This should merge if not for the 'special' flag.\n \t */\n \tvmg_set_range(\u0026vmg, 0x3000, 0x4000, 3, vma_flags);\n \tfor (i = 0; i \u003c ARRAY_SIZE(special_flags); i++) {\n"
} |
| 2/1 | 2026/09/14 19:11 | agent | patch-evaluator |
1mModel:core Args: null Results: {
"EnableConfigs": [
"CHR_DEV_SG",
"SCSI",
"USB_MON",
"USB",
"SND_PCM",
"HUGETLB_PAGE",
"USERFAULTFD"
],
"FocusSymbols": [
"map_discontig_kernel_pages_prepare",
"map_discontig_kernel_pages_complete",
"mmap_prepare_validate",
"mmap_hook_validate",
"mmap_action_prepare",
"mmap_action_complete",
"sg_mmap_prepare",
"mon_bin_mmap_prepare",
"snd_pcm_mmap_status"
],
"Reasoning": "The patch modifies core memory management (mm) and reachable drivers by introducing a new discontiguous kernel page mapping mechanism (map_discontig_kernel_pages_*), adding validation hooks for mmap and mmap_prepare (mmap_prepare_validate, mmap_hook_validate), refactoring VMA classifications across mm (vma_is_kernel_owned, vma_is_fixed_mapping, vma_can_merge, vma_is_persistent), altering mlock tracking logic during page table walks, and migrating reachable drivers (drivers/scsi/sg.c, drivers/usb/mon/mon_bin.c, sound/core/pcm_native.c) to these new APIs. These changes alter VMA invariant verification and mapping setup in reachable core code, warranting fuzzing for assertion failures and subtle mm regressions.",
"WorthFuzzing": true
}Instruction: You are an expert Linux kernel maintainer and security engineer.
Your job is to review a provided patch series and evaluate whether it warrants fuzzing with syzkaller.
IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in
your workspace. Do NOT rely on internal assumptions. You must actively use your code access
tools to inspect the actual source code, callers, and surrounding context.
================================================================================
1. CORE TRIAGE PHILOSOPHY
================================================================================
The goal of patch fuzzing is to discover crashes, regressions, exposed latent bugs,
and newly triggered assertions introduced by the patch series.
- REACHABILITY IS THE PRIMARY GATE:
Fuzzing can only discover bugs in code that can actually execute in standard virtualized
environments (GCE or QEMU, utilizing software-emulated devices like USB gadgets, netdev, tun/tap).
If the modified code is structurally unreachable (see Section 2), it MUST NOT be fuzzed,
regardless of whether it adds assertions or complex logic.
- DO NOT BLINDLY TRUST "NO FUNCTIONAL CHANGE" (NFCI) OR "REFACTORING" CLAIMS:
Patch authors routinely label changes as "cleanups", "refactorings", or state
"No functional change intended". Do NOT take these claims at face value.
Code refactorings that rearrange logic, introduce helper functions, or alter state management
in core subsystems frequently introduce subtle semantic shifts or uncover latent kernel bugs.
If reachable executable code is modified or refactored, it MUST be fuzzed.
- NEW OR MODIFIED ASSERTIONS IN REACHABLE CODE MUST BE FUZZED:
When a patch introduces or modifies runtime checks or assertions (e.g., WARN_ON*, VM_WARN_ON*,
BUG_ON*, lockdep_assert*) in reachable code paths, it enforces new or stricter invariants.
Even if the author believes the invariant always holds, fuzzing is essential to verify whether
an unusual sequence of operations can violate it.
================================================================================
2. WHEN TO RETURN WorthFuzzing=false (NEGATIVE CRITERIA)
================================================================================
Return WorthFuzzing=false ONLY IF all modified code falls strictly into one or more of these categories:
- Non-kernel and non-executable changes:
* Modifications to Documentation/, comments, or spelling fixes.
* User-space directories, self-tests, samples, or scripts (e.g., tools/, samples/, scripts/, usr/)
that do not affect the compiled kernel image (vmlinux) or kernel modules.
* Purely decorative logging (e.g., message strings in pr_err, printk, dev_info) or tracepoints
that do not alter control flow or data structures.
* Build system or Kconfig changes that do not alter compiled C logic.
- Structurally unreachable hardware:
* Vendor-specific PCIe switches, SmartNICs, or GPU drivers (e.g., mlxsw, pds_core, qed,
ionic, amdgpu) requiring physical ASIC/PCIe cards not emulated in standard QEMU.
- Unreachable execution paths:
* Driver teardown callbacks (.remove, .shutdown, pci_unregister_driver) executed only during
physical PCI hot-unplug or manual sysfs driver unbinding.
* Code paths exclusive to architectures other than the target architecture.
================================================================================
3. WHEN TO RETURN WorthFuzzing=true (POSITIVE CRITERIA)
================================================================================
Return WorthFuzzing=true whenever the patch touches reachable executable code, including:
- Core Subsystems:
* Any logic modifications in memory management (mm/), synchronization/locking (kernel/locking/),
BPF, scheduler, core networking, VFS, or syscall handling.
- Refactorings and Code Cleanups:
* Any restructuring of reachable data structures, helper abstractions, or algorithm flows.
- Runtime Assertions and Defensive Checks:
* Any introduction or alteration of assertions (WARN_ON*, VM_WARN_ON*, BUG_ON*, etc.) in reachable paths.
- Reachable Drivers and Protocols:
* Drivers accessible via virtual buses (virtio, USB gadget, loopback, netlink, binder, sockets, etc.).
================================================================================
4. EXTRACTING FocusSymbols (PREVENTING DILUTION)
================================================================================
When WorthFuzzing=true, you must extract specific kernel functions into FocusSymbols to guide the fuzzer:
- AVOID UBIQUITOUS LIFECYCLE HOT-PATHS:
Do NOT list generic, ubiquitous functions called by almost every program in the corpus
(including, but not limited to: general memory allocators and deallocators, page fault
and trap handlers, or core synchronization primitives; this is not an exhaustive list).
Listing ubiquitous functions causes the fuzzer to classify thousands of unrelated tests as "focused",
which severely dilutes fuzzing effort away from the actual changes.
- TARGET SPECIFIC FEATURE LOGIC AND ENTRYPOINTS:
List functions that specifically implement the logic being added or altered, or direct API entrypoints
for the subsystem feature under review.
- HANDLING STATIC INLINE FUNCTIONS IN HEADERS (.h):
Compiler-inlined static functions (such as static inlines in mm/*.h or include/linux/*.h) lack
distinct symbol addresses in vmlinux and cannot be targeted directly by symbol coverage filters.
If the changes are primarily in static inline helpers, identify non-static, feature-specific caller
functions in .c files that exercise them (avoiding ubiquitous lifecycle wrappers).
================================================================================
5. IDENTIFYING EnableConfigs
================================================================================
Identify any specific CONFIG_ options required to properly compile and reach the modified code:
- Inspect Kconfig files and #ifdef guards; do not make assumptions.
- Check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed.
- Strip any 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4").
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Target architecture: amd64
For your convenience, here is the diff of the changes:
commit dfb57cd7fb217657931dcb005b6452b86d63b7db
Author: syz-cluster <triage@syzkaller.com>
Date: Mon Sep 14 19:11:06 2026 +0000
syz-cluster: applied patch under review
diff --git a/Documentation/filesystems/mmap_prepare.rst b/Documentation/filesystems/mmap_prepare.rst
index 82c99c95ad854..a476e1006bf12 100644
--- a/Documentation/filesystems/mmap_prepare.rst
+++ b/Documentation/filesystems/mmap_prepare.rst
@@ -164,5 +164,86 @@ pointer. These are:
sufficient entries in the page array to cover the entire range of the
described VMA.
+* mmap_action_map_discontig_kernel_pages() - Maps a discontiguous range of
+ `struct page` pointers over the VMA. They must span from the start of the VMA,
+ but may terminate prior to the end (leaving the remainder unmapped).
+
**NOTE:** The ``action`` field should never normally be manipulated directly,
rather you ought to use one of these helpers.
+
+Discontiguous Actions
+=====================
+
+Some actions can be performed across discontiguous ranges.
+
+Map kernel pages
+----------------
+
+To map kernel pages discontiguously, you must provide hooks using ``struct
+discontig_kernel_page_ops``:
+
+.. code-block:: C
+
+ struct discontig_kernel_page_ops {
+ int (*init)(void *vm_private_data, void **private);
+ int (*get)(struct discontig_kernel_page_state *state);
+ };
+
+The ``init`` hook is optional and allows state to be established before the
+operation starts, for instance taking a reference count. Nothing is invoked
+after the operation, so ``init`` must not leave locks held, and state that must
+be released once the mapping goes away should be released in
+``vm_ops->close``.
+
+The ``init`` hook, if provided, is invoked prior to the operation starting. It
+may update what is pointed to by ``vm_private_data`` and/or ``private``. If an
+error is returned, then the operation is aborted. The ``private`` field can be
+reassigned.
+
+**NOTE:** The operation may sleep between invocations of ``get``, so locks
+needed to stabilise state must be taken and released within each hook.
+
+The ``get`` handler is the key means through which the operation is
+executed. The current state of the operation is provided through ``struct
+discontig_kernel_page_state``:
+
+.. code-block:: C
+
+ struct discontig_kernel_page_state {
+ /* Map state. */
+ unsigned long start; /* Start address of VMA. */
+ unsigned long end; /* End address of VMA. */
+ unsigned long addr; /* The current address to be mapped. */
+ pgoff_t pgoff; /* The current pgoff to be mapped. */
+ unsigned long nr_pages_mapped; /* The number of pages mapped. */
+ unsigned long nr_pages_remain; /* The number of pages remaining. */
+
+ /* User-defined state. */
+ void *vm_private_data; /* VMA private data. */
+ void *private; /* Mapping private data. */
+
+ /* Users should not touch these, use discontig_kernel_map_*() helpers. */
+ ... internal fields ...
+ };
+
+With ``private`` being an additional user-controllable state variable,
+initialised via ``mmap_action_map_discontig_kernel_pages()``, and
+``vm_private_data`` being equal to the ``desc->private_data`` field set in
+the ``mmap_prepare()`` hook.
+
+In the ``get`` hook, the user must choose how to map kernel pages:
+
+* ``discontig_kernel_map_abort()`` - Call this to abort the operation, whatever
+ has been mapped so far will be retained, the rest of the mapping will SIGBUS
+ if accessed.
+* ``discontig_kernel_map_page()`` - Maps a single page, correctly handling
+ compound pages (if the compound page is bigger than the remaining pages in the
+ VMA, then only those pages that fit will be mapped). For a compound page, the
+ head page must be passed.
+* ``discontig_kernel_map_page_range()`` - Map an array of pages of a specified
+ size. Note that if the number of pages specified exceeds the VMA size then an
+ error will arise.
+
+If an error arises after ``init`` succeeded, the core unmaps the VMA, invoking
+``vm_ops->close`` if set, which is therefore the place to release any state
+that ``init`` established.
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 9ba86450fe4af..3c1240ffc38df 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1463,14 +1463,12 @@ static int get_vma_page_shift(struct vm_area_struct *vma, unsigned long hva)
{
unsigned long pa;
- if (is_vm_hugetlb_page(vma) && !(vma->vm_flags & VM_PFNMAP))
+ if (vma_is_hugetlb(vma))
return huge_page_shift(hstate_vma(vma));
if (!(vma->vm_flags & VM_PFNMAP))
return PAGE_SHIFT;
- VM_BUG_ON(is_vm_hugetlb_page(vma));
-
pa = (vma->vm_pgoff << PAGE_SHIFT) + (hva - vma->vm_start);
#ifndef __PAGETABLE_PMD_FOLDED
diff --git a/arch/powerpc/mm/book3s64/radix_tlb.c b/arch/powerpc/mm/book3s64/radix_tlb.c
index 7de5760164a90..b4603a98224b3 100644
--- a/arch/powerpc/mm/book3s64/radix_tlb.c
+++ b/arch/powerpc/mm/book3s64/radix_tlb.c
@@ -627,7 +627,7 @@ void radix__local_flush_tlb_page(struct vm_area_struct *vma, unsigned long vmadd
{
#ifdef CONFIG_HUGETLB_PAGE
/* need the return fix for nohash.c */
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
return radix__local_flush_hugetlb_page(vma, vmaddr);
#endif
radix__local_flush_tlb_page_psize(vma->vm_mm, vmaddr, mmu_virtual_psize);
@@ -945,7 +945,7 @@ void radix__flush_tlb_page_psize(struct mm_struct *mm, unsigned long vmaddr,
void radix__flush_tlb_page(struct vm_area_struct *vma, unsigned long vmaddr)
{
#ifdef CONFIG_HUGETLB_PAGE
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
return radix__flush_hugetlb_page(vma, vmaddr);
#endif
radix__flush_tlb_page_psize(vma->vm_mm, vmaddr, mmu_virtual_psize);
@@ -1113,7 +1113,7 @@ void radix__flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
{
#ifdef CONFIG_HUGETLB_PAGE
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
return radix__flush_hugetlb_tlb_range(vma, start, end);
#endif
diff --git a/arch/powerpc/mm/nohash/e500_hugetlbpage.c b/arch/powerpc/mm/nohash/e500_hugetlbpage.c
index a134d28a0e4d3..b87623f04be53 100644
--- a/arch/powerpc/mm/nohash/e500_hugetlbpage.c
+++ b/arch/powerpc/mm/nohash/e500_hugetlbpage.c
@@ -180,7 +180,7 @@ book3e_hugetlb_preload(struct vm_area_struct *vma, unsigned long ea, pte_t pte)
*/
void __update_mmu_cache(struct vm_area_struct *vma, unsigned long address, pte_t *ptep)
{
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
book3e_hugetlb_preload(vma, address, *ptep);
}
diff --git a/arch/powerpc/mm/nohash/tlb.c b/arch/powerpc/mm/nohash/tlb.c
index 0a650742f3a00..07a2db16c2b15 100644
--- a/arch/powerpc/mm/nohash/tlb.c
+++ b/arch/powerpc/mm/nohash/tlb.c
@@ -278,7 +278,7 @@ void __flush_tlb_page(struct mm_struct *mm, unsigned long vmaddr,
void flush_tlb_page(struct vm_area_struct *vma, unsigned long vmaddr)
{
#ifdef CONFIG_HUGETLB_PAGE
- if (vma && is_vm_hugetlb_page(vma))
+ if (vma && vma_is_hugetlb(vma))
flush_hugetlb_page(vma, vmaddr);
#endif
diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
index 6035b5ec95039..5c5c77f98bf0f 100644
--- a/arch/riscv/kvm/mmu.c
+++ b/arch/riscv/kvm/mmu.c
@@ -664,7 +664,7 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
return -EFAULT;
}
- is_hugetlb = is_vm_hugetlb_page(vma);
+ is_hugetlb = vma_is_hugetlb(vma);
if (is_hugetlb)
vma_pageshift = huge_page_shift(hstate_vma(vma));
else
diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c
index 962db300a1665..a74a7d5258aa1 100644
--- a/arch/riscv/mm/tlbflush.c
+++ b/arch/riscv/mm/tlbflush.c
@@ -149,7 +149,7 @@ void flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
{
unsigned long stride_size;
- if (!is_vm_hugetlb_page(vma)) {
+ if (!vma_is_hugetlb(vma)) {
stride_size = PAGE_SIZE;
} else {
stride_size = huge_page_size(hstate_vma(vma));
diff --git a/arch/s390/mm/gmap_helpers.c b/arch/s390/mm/gmap_helpers.c
index ff63ffb1dbd29..3f6783b93e679 100644
--- a/arch/s390/mm/gmap_helpers.c
+++ b/arch/s390/mm/gmap_helpers.c
@@ -102,7 +102,7 @@ __context_unsafe(/* pte_unmap_unlock() not instrumented */)
/* Find the vm address for the guest address */
vma = vma_lookup(mm, vmaddr);
- if (!vma || is_vm_hugetlb_page(vma))
+ if (!vma || vma_is_hugetlb(vma))
return;
/* Get pointer to the page table entry */
@@ -139,7 +139,7 @@ void gmap_helper_discard(struct mm_struct *mm, unsigned long vmaddr, unsigned lo
vma = find_vma_intersection(mm, vmaddr, end);
if (!vma)
return;
- if (!is_vm_hugetlb_page(vma))
+ if (!vma_is_hugetlb(vma))
zap_vma_range(vma, vmaddr, min(end, vma->vm_end) - vmaddr);
vmaddr = vma->vm_end;
}
@@ -247,7 +247,7 @@ static int __gmap_helper_unshare_zeropages(struct mm_struct *mm)
* proof to catch unexpected zeropages in other mappings and
* fail.
*/
- if ((vma->vm_flags & VM_PFNMAP) || is_vm_hugetlb_page(vma))
+ if ((vma->vm_flags & VM_PFNMAP) || vma_is_hugetlb(vma))
continue;
addr = vma->vm_start;
diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c
index 103db4683b165..9bbccb5d23a8f 100644
--- a/arch/sparc/mm/init_64.c
+++ b/arch/sparc/mm/init_64.c
@@ -413,7 +413,7 @@ void update_mmu_cache_range(struct vm_fault *vmf, struct vm_area_struct *vma,
if (mm->context.hugetlb_pte_count || mm->context.thp_pte_count) {
unsigned long hugepage_size = PAGE_SIZE;
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
hugepage_size = huge_page_size(hstate_vma(vma));
if (hugepage_size >= PUD_SIZE) {
diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
index 65a2de82ecd29..0f60c0d076b62 100644
--- a/arch/x86/kernel/uprobes.c
+++ b/arch/x86/kernel/uprobes.c
@@ -715,7 +715,7 @@ static struct vm_area_struct *get_uprobe_trampoline(struct mm_struct *mm, unsign
*new_mapping = true;
return _install_special_mapping(mm, vaddr, PAGE_SIZE,
- VM_READ|VM_EXEC|VM_MAYEXEC|VM_MAYREAD|VM_IO,
+ VM_READ|VM_EXEC|VM_MAYEXEC|VM_MAYREAD|VM_MIXEDMAP,
&tramp_mapping);
}
diff --git a/drivers/gpu/drm/drm_gpusvm.c b/drivers/gpu/drm/drm_gpusvm.c
index a93eee7ddb9e9..fab34fea99c2f 100644
--- a/drivers/gpu/drm/drm_gpusvm.c
+++ b/drivers/gpu/drm/drm_gpusvm.c
@@ -9,9 +9,9 @@
#include <linux/dma-mapping.h>
#include <linux/export.h>
#include <linux/hmm.h>
-#include <linux/hugetlb_inline.h>
#include <linux/memremap.h>
#include <linux/mm_types.h>
+#include <linux/mm.h>
#include <linux/slab.h>
#include <drm/drm_device.h>
@@ -1141,8 +1141,7 @@ drm_gpusvm_range_find_or_insert(struct drm_gpusvm *gpusvm,
* limitations. If/when migrate_vma_* add more support, this logic will
* have to change.
*/
- migrate_devmem = ctx->devmem_possible &&
- vma_is_anonymous(vas) && !is_vm_hugetlb_page(vas);
+ migrate_devmem = ctx->devmem_possible && vma_is_anonymous(vas);
chunk_size = drm_gpusvm_range_chunk_size(gpusvm, notifier, vas,
fault_addr, gpuva_start,
diff --git a/drivers/hsi/clients/cmt_speech.c b/drivers/hsi/clients/cmt_speech.c
index 7226677ebde7a..801697b74d4f8 100644
--- a/drivers/hsi/clients/cmt_speech.c
+++ b/drivers/hsi/clients/cmt_speech.c
@@ -1084,22 +1084,6 @@ static void cs_hsi_stop(struct cs_hsi_iface *hi)
kfree(hi);
}
-static vm_fault_t cs_char_vma_fault(struct vm_fault *vmf)
-{
- struct cs_char *csdata = vmf->vma->vm_private_data;
- struct page *page;
-
- page = virt_to_page((void *)csdata->mmap_base);
- get_page(page);
- vmf->page = page;
-
- return 0;
-}
-
-static const struct vm_operations_struct cs_char_vm_ops = {
- .fault = cs_char_vma_fault,
-};
-
static int cs_char_fasync(int fd, struct file *file, int on)
{
struct cs_char *csdata = file->private_data;
@@ -1256,18 +1240,19 @@ static long cs_char_ioctl(struct file *file, unsigned int cmd,
return r;
}
-static int cs_char_mmap(struct file *file, struct vm_area_struct *vma)
+static int cs_char_mmap_prepare(struct vm_area_desc *desc)
{
- if (vma->vm_end < vma->vm_start)
- return -EINVAL;
+ struct file *file = desc->file;
+ struct cs_char *csdata = file->private_data;
+ struct page **pages = (struct page **)&desc->private_data;
- if (vma_pages(vma) != 1)
+ if (vma_desc_pages(desc) != 1)
return -EINVAL;
- vm_flags_set(vma, VM_IO | VM_DONTDUMP | VM_DONTEXPAND);
- vma->vm_ops = &cs_char_vm_ops;
- vma->vm_private_data = file->private_data;
+ vma_desc_set_flags(desc, VMA_DONTDUMP_BIT, VMA_DONTEXPAND_BIT);
+ *pages = virt_to_page((void *)csdata->mmap_base);
+ mmap_action_map_kernel_pages_full(desc, pages);
return 0;
}
@@ -1353,7 +1338,7 @@ static const struct file_operations cs_char_fops = {
.write = cs_char_write,
.poll = cs_char_poll,
.unlocked_ioctl = cs_char_ioctl,
- .mmap = cs_char_mmap,
+ .mmap_prepare = cs_char_mmap_prepare,
.open = cs_char_open,
.release = cs_char_release,
.fasync = cs_char_fasync,
diff --git a/drivers/infiniband/hw/hfi1/file_ops.c b/drivers/infiniband/hw/hfi1/file_ops.c
index dc548e6802e24..b02d1f1dbb27b 100644
--- a/drivers/infiniband/hw/hfi1/file_ops.c
+++ b/drivers/infiniband/hw/hfi1/file_ops.c
@@ -70,7 +70,6 @@ static int set_ctxt_pkey(struct hfi1_ctxtdata *uctxt, unsigned long arg);
static int ctxt_reset(struct hfi1_ctxtdata *uctxt);
static int manage_rcvq(struct hfi1_ctxtdata *uctxt, u16 subctxt,
unsigned long arg);
-static vm_fault_t vma_fault(struct vm_fault *vmf);
static long hfi1_file_ioctl(struct file *fp, unsigned int cmd,
unsigned long arg);
@@ -85,10 +84,6 @@ static const struct file_operations hfi1_file_ops = {
.llseek = noop_llseek,
};
-static const struct vm_operations_struct vm_ops = {
- .fault = vma_fault,
-};
-
/*
* Types of memories mapped into user processes' space
*/
@@ -304,13 +299,13 @@ static ssize_t hfi1_write_iter(struct kiocb *kiocb, struct iov_iter *from)
return reqs;
}
-static inline void mmap_cdbg(u16 ctxt, u8 subctxt, u8 type, u8 mapio, u8 vmf,
+static inline void mmap_cdbg(u16 ctxt, u8 subctxt, u8 type, u8 mapio, u8 is_vmalloc,
u64 memaddr, void *memvirt, dma_addr_t memdma,
ssize_t memlen, struct vm_area_struct *vma)
{
hfi1_cdbg(PROC,
- "%u:%u type:%u io/vf/dma:%d/%d/%d, addr:0x%llx, len:%lu(%lu), flags:0x%lx",
- ctxt, subctxt, type, mapio, vmf, !!memdma,
+ "%u:%u type:%u io/vmalloc/dma:%d/%d/%d, addr:0x%llx, len:%lu(%lu), flags:0x%lx",
+ ctxt, subctxt, type, mapio, is_vmalloc, !!memdma,
memaddr ?: (u64)memvirt, memlen,
vma->vm_end - vma->vm_start, vma->vm_flags);
}
@@ -325,7 +320,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
memaddr = 0;
void *memvirt = NULL;
dma_addr_t memdma = 0;
- u8 subctxt, mapio = 0, vmf = 0, type;
+ u8 subctxt, mapio = 0, is_vmalloc = 0, type;
ssize_t memlen = 0;
int ret = 0;
u16 ctxt;
@@ -347,7 +342,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
/*
* vm_pgoff is used as a buffer selector cookie. Always mmap from
* the beginning.
- */
+ */
vma->vm_pgoff = 0;
flags = vma->vm_flags;
@@ -366,7 +361,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
*/
memlen = PAGE_ALIGN(uctxt->sc->credits * PIO_BLOCK_SIZE);
flags &= ~VM_MAYREAD;
- flags |= VM_DONTCOPY | VM_DONTEXPAND;
+ flags |= VM_DONTCOPY;
vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
mapio = 1;
break;
@@ -401,6 +396,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
memlen = rcvhdrq_size(uctxt);
memvirt = uctxt->rcvhdrq;
memdma = uctxt->rcvhdrq_dma;
+ flags |= VM_DONTEXPAND;
break;
case RCV_EGRBUF: {
unsigned long vm_start_save;
@@ -422,7 +418,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
ret = -EPERM;
goto done;
}
- vm_flags_clear(vma, VM_MAYWRITE);
+ vm_flags_mod(vma, VM_DONTEXPAND, VM_MAYWRITE);
/*
* Mmap multiple separate allocations into a single vma. From
* here, dma_mmap_coherent() calls dma_direct_mmap(), which
@@ -438,7 +434,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
memvirt = uctxt->egrbufs.buffers[i].addr;
memdma = uctxt->egrbufs.buffers[i].dma;
vma->vm_end += memlen;
- mmap_cdbg(ctxt, subctxt, type, mapio, vmf, memaddr,
+ mmap_cdbg(ctxt, subctxt, type, mapio, is_vmalloc, memaddr,
memvirt, memdma, memlen, vma);
ret = dma_mmap_coherent(&dd->pcidev->dev, vma,
memvirt, memdma, memlen);
@@ -467,7 +463,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
* user registers.
*/
memlen = PAGE_SIZE;
- flags |= VM_DONTCOPY | VM_DONTEXPAND;
+ flags |= VM_DONTCOPY;
vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
mapio = 1;
break;
@@ -476,15 +472,10 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
* Use the page where this context's flags are. User level
* knows where it's own bitmap is within the page.
*/
- memaddr = (unsigned long)
- (dd->events + uctxt_offset(uctxt)) & PAGE_MASK;
+ memvirt = dd->events + uctxt_offset(uctxt);
+ memvirt = (void *)(((uintptr_t)memvirt) & PAGE_MASK);
memlen = PAGE_SIZE;
- /*
- * v3.7 removes VM_RESERVED but the effect is kept by
- * using VM_IO.
- */
- flags |= VM_IO | VM_DONTEXPAND;
- vmf = 1;
+ is_vmalloc = 1;
break;
case STATUS:
if (flags & VM_WRITE) {
@@ -493,7 +484,6 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
}
memaddr = kvirt_to_phys((void *)dd->status);
memlen = PAGE_SIZE;
- flags |= VM_IO | VM_DONTEXPAND;
break;
case RTAIL:
if (!HFI1_CAP_IS_USET(DMA_RTAIL)) {
@@ -512,25 +502,23 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
memvirt = (void *)hfi1_rcvhdrtail_kvaddr(uctxt);
memdma = uctxt->rcvhdrqtailaddr_dma;
flags &= ~VM_MAYWRITE;
+ flags |= VM_DONTEXPAND;
break;
case SUBCTXT_UREGS:
- memaddr = (u64)uctxt->subctxt_uregbase;
+ memvirt = uctxt->subctxt_uregbase;
memlen = PAGE_SIZE;
- flags |= VM_IO | VM_DONTEXPAND;
- vmf = 1;
+ is_vmalloc = 1;
break;
case SUBCTXT_RCV_HDRQ:
- memaddr = (u64)uctxt->subctxt_rcvhdr_base;
+ memvirt = uctxt->subctxt_rcvhdr_base;
memlen = rcvhdrq_size(uctxt) * uctxt->subctxt_cnt;
- flags |= VM_IO | VM_DONTEXPAND;
- vmf = 1;
+ is_vmalloc = 1;
break;
case SUBCTXT_EGRBUF:
- memaddr = (u64)uctxt->subctxt_rcvegrbuf;
+ memvirt = uctxt->subctxt_rcvegrbuf;
memlen = uctxt->egrbufs.size * uctxt->subctxt_cnt;
- flags |= VM_IO | VM_DONTEXPAND;
flags &= ~VM_MAYWRITE;
- vmf = 1;
+ is_vmalloc = 1;
break;
case SDMA_COMP: {
struct hfi1_user_sdma_comp_q *cq = fd->cq;
@@ -539,10 +527,9 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
ret = -EFAULT;
goto done;
}
- memaddr = (u64)cq->comps;
+ memvirt = cq->comps;
memlen = PAGE_ALIGN(sizeof(*cq->comps) * cq->nentries);
- flags |= VM_IO | VM_DONTEXPAND;
- vmf = 1;
+ is_vmalloc = 1;
break;
}
default:
@@ -559,12 +546,10 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
}
vm_flags_reset(vma, flags);
- mmap_cdbg(ctxt, subctxt, type, mapio, vmf, memaddr, memvirt, memdma,
+ mmap_cdbg(ctxt, subctxt, type, mapio, is_vmalloc, memaddr, memvirt, memdma,
memlen, vma);
- if (vmf) {
- vma->vm_pgoff = PFN_DOWN(memaddr);
- vma->vm_ops = &vm_ops;
- ret = 0;
+ if (is_vmalloc) {
+ ret = remap_vmalloc_range(vma, memvirt, 0);
} else if (memdma) {
ret = dma_mmap_coherent(&dd->pcidev->dev, vma,
memvirt, memdma, memlen);
@@ -588,24 +573,6 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
return ret;
}
-/*
- * Local (non-chip) user memory is not mapped right away but as it is
- * accessed by the user-level code.
- */
-static vm_fault_t vma_fault(struct vm_fault *vmf)
-{
- struct page *page;
-
- page = vmalloc_to_page((void *)(vmf->pgoff << PAGE_SHIFT));
- if (!page)
- return VM_FAULT_SIGBUS;
-
- get_page(page);
- vmf->page = page;
-
- return 0;
-}
-
static __poll_t hfi1_poll(struct file *fp, struct poll_table_struct *pt)
{
struct hfi1_ctxtdata *uctxt;
diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
index 5408f002e6c01..3f9e08725602c 100644
--- a/drivers/scsi/sg.c
+++ b/drivers/scsi/sg.c
@@ -1212,85 +1212,72 @@ sg_fasync(int fd, struct file *filp, int mode)
return fasync_helper(fd, filp, mode, &sfp->async_qp);
}
-static vm_fault_t
-sg_vma_fault(struct vm_fault *vmf)
+static int sg_discontig_init(void *vm_private_data, void **private)
{
- struct vm_area_struct *vma = vmf->vma;
- Sg_fd *sfp;
- unsigned long offset, len, sa;
- Sg_scatter_hold *rsv_schp;
- int k, length;
-
- if ((NULL == vma) || (!(sfp = (Sg_fd *) vma->vm_private_data)))
- return VM_FAULT_SIGBUS;
- rsv_schp = &sfp->reserve;
- offset = vmf->pgoff << PAGE_SHIFT;
- if (offset >= rsv_schp->bufflen)
- return VM_FAULT_SIGBUS;
- SCSI_LOG_TIMEOUT(3, sg_printk(KERN_INFO, sfp->parentdp,
- "sg_vma_fault: offset=%lu, scatg=%d\n",
- offset, rsv_schp->k_use_sg));
- sa = vma->vm_start;
- length = 1 << (PAGE_SHIFT + rsv_schp->page_order);
- for (k = 0; k < rsv_schp->k_use_sg && sa < vma->vm_end; k++) {
- len = vma->vm_end - sa;
- len = (len < length) ? len : length;
- if (offset < len) {
- struct page *page = rsv_schp->pages[k] + (offset >> PAGE_SHIFT);
- get_page(page); /* increment page count */
- vmf->page = page;
- return 0; /* success */
- }
- sa += len;
- offset -= len;
+ const unsigned long req_sz = (unsigned long)*private;
+ Sg_fd *sfp = vm_private_data;
+ Sg_scatter_hold *rsv_schp = &sfp->reserve;
+ int err = 0;
+
+ mutex_lock(&sfp->f_mutex);
+ if (req_sz > rsv_schp->bufflen) {
+ err = -ENOMEM; /* cannot map more than reserved buffer */
+ goto out;
+ }
+ sfp->mmap_called = 1; /* Prevents changes to buffer size. */
+out:
+ mutex_unlock(&sfp->f_mutex);
+ return err;
+}
+
+static int
+sg_discontig_get(struct discontig_kernel_page_state *state)
+{
+ Sg_fd *sfp = state->vm_private_data;
+ Sg_scatter_hold *rsv_schp = &sfp->reserve;
+ const unsigned int order = rsv_schp->page_order;
+ const pgoff_t nr_pages = state->nr_pages_mapped;
+
+ if (nr_pages >= (rsv_schp->bufflen >> PAGE_SHIFT)) {
+ discontig_kernel_map_abort(state);
+ return 0;
}
- return VM_FAULT_SIGBUS;
+ SCSI_LOG_TIMEOUT(3, sg_printk(KERN_INFO, sfp->parentdp,
+ "%s: offset=%lu, scatg=%d\n", __func__,
+ nr_pages << PAGE_SHIFT, rsv_schp->k_use_sg));
+
+ discontig_kernel_map_page(state, rsv_schp->pages[nr_pages >> order]);
+ return 0;
}
-static const struct vm_operations_struct sg_mmap_vm_ops = {
- .fault = sg_vma_fault,
+static const struct discontig_kernel_page_ops sg_discontig_ops = {
+ .init = sg_discontig_init,
+ .get = sg_discontig_get,
};
static int
-sg_mmap(struct file *filp, struct vm_area_struct *vma)
+sg_mmap_prepare(struct vm_area_desc *desc)
{
- Sg_fd *sfp;
- unsigned long req_sz, len, sa;
- Sg_scatter_hold *rsv_schp;
- int k, length;
- int ret = 0;
+ Sg_fd *sfp = desc->file->private_data;
+ const unsigned long req_sz = vma_desc_size(desc);
- if ((!filp) || (!vma) || (!(sfp = (Sg_fd *) filp->private_data)))
+ if (!sfp)
return -ENXIO;
- req_sz = vma->vm_end - vma->vm_start;
+
SCSI_LOG_TIMEOUT(3, sg_printk(KERN_INFO, sfp->parentdp,
"sg_mmap starting, vm_start=%p, len=%d\n",
- (void *) vma->vm_start, (int) req_sz));
- if (vma->vm_pgoff)
+ (void *) desc->start, (int) req_sz));
+
+ if (desc->pgoff)
return -EINVAL; /* want no offset */
- rsv_schp = &sfp->reserve;
- mutex_lock(&sfp->f_mutex);
- if (req_sz > rsv_schp->bufflen) {
- ret = -ENOMEM; /* cannot map more than reserved buffer */
- goto out;
- }
- sa = vma->vm_start;
- length = 1 << (PAGE_SHIFT + rsv_schp->page_order);
- for (k = 0; k < rsv_schp->k_use_sg && sa < vma->vm_end; k++) {
- len = vma->vm_end - sa;
- len = (len < length) ? len : length;
- sa += len;
- }
+ vma_desc_set_flags(desc, VMA_DONTEXPAND_BIT, VMA_DONTDUMP_BIT);
+ desc->private_data = sfp;
- sfp->mmap_called = 1;
- vm_flags_set(vma, VM_IO | VM_DONTEXPAND | VM_DONTDUMP);
- vma->vm_private_data = sfp;
- vma->vm_ops = &sg_mmap_vm_ops;
-out:
- mutex_unlock(&sfp->f_mutex);
- return ret;
+ mmap_action_map_discontig_kernel_pages(desc, (void *)req_sz,
+ &sg_discontig_ops);
+ return 0;
}
static void
@@ -1415,7 +1402,7 @@ static const struct file_operations sg_fops = {
.unlocked_ioctl = sg_ioctl,
.compat_ioctl = compat_ptr_ioctl,
.open = sg_open,
- .mmap = sg_mmap,
+ .mmap_prepare = sg_mmap_prepare,
.release = sg_release,
.fasync = sg_fasync,
};
diff --git a/drivers/usb/mon/mon_bin.c b/drivers/usb/mon/mon_bin.c
index 687f6a8981f34..9d00b21a8153b 100644
--- a/drivers/usb/mon/mon_bin.c
+++ b/drivers/usb/mon/mon_bin.c
@@ -1219,6 +1219,15 @@ mon_bin_poll(struct file *file, struct poll_table_struct *wait)
return mask;
}
+static void __mon_bin_vma_open(struct mon_reader_bin *rp)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&rp->b_lock, flags);
+ rp->mmap_active++;
+ spin_unlock_irqrestore(&rp->b_lock, flags);
+}
+
/*
* open and close: just keep track of how many times the device is
* mapped, to use the proper memory allocation function.
@@ -1226,64 +1235,79 @@ mon_bin_poll(struct file *file, struct poll_table_struct *wait)
static void mon_bin_vma_open(struct vm_area_struct *vma)
{
struct mon_reader_bin *rp = vma->vm_private_data;
- unsigned long flags;
- spin_lock_irqsave(&rp->b_lock, flags);
- rp->mmap_active++;
- spin_unlock_irqrestore(&rp->b_lock, flags);
+ __mon_bin_vma_open(rp);
}
-static void mon_bin_vma_close(struct vm_area_struct *vma)
+static void __mon_bin_vma_close(struct mon_reader_bin *rp)
{
unsigned long flags;
- struct mon_reader_bin *rp = vma->vm_private_data;
spin_lock_irqsave(&rp->b_lock, flags);
rp->mmap_active--;
spin_unlock_irqrestore(&rp->b_lock, flags);
}
-/*
- * Map ring pages to user space.
- */
-static vm_fault_t mon_bin_vma_fault(struct vm_fault *vmf)
+static void mon_bin_vma_close(struct vm_area_struct *vma)
{
- struct mon_reader_bin *rp = vmf->vma->vm_private_data;
+ struct mon_reader_bin *rp = vma->vm_private_data;
+
+ __mon_bin_vma_close(rp);
+}
+
+static const struct vm_operations_struct mon_bin_vm_ops = {
+ .open = mon_bin_vma_open,
+ .close = mon_bin_vma_close,
+};
+
+static int mon_bin_discontig_init(void *vm_private_data, void **private)
+{
+ struct mon_reader_bin *rp = vm_private_data;
+
+ /* Dropped by mon_bin_vma_close() on unmap, including on error. */
+ __mon_bin_vma_open(rp);
+ return 0;
+}
+
+static int mon_bin_discontig_get(struct discontig_kernel_page_state *state)
+{
+ struct mon_reader_bin *rp = state->vm_private_data;
unsigned long offset, chunk_idx;
- struct page *pageptr;
unsigned long flags;
spin_lock_irqsave(&rp->b_lock, flags);
- offset = vmf->pgoff << PAGE_SHIFT;
+
+ offset = state->pgoff << PAGE_SHIFT;
if (offset >= rp->b_size) {
spin_unlock_irqrestore(&rp->b_lock, flags);
- return VM_FAULT_SIGBUS;
+ discontig_kernel_map_abort(state);
+ return 0;
}
chunk_idx = offset / CHUNK_SIZE;
- pageptr = rp->b_vec[chunk_idx].pg;
- get_page(pageptr);
- vmf->page = pageptr;
+ discontig_kernel_map_page(state, rp->b_vec[chunk_idx].pg);
+
spin_unlock_irqrestore(&rp->b_lock, flags);
return 0;
}
-static const struct vm_operations_struct mon_bin_vm_ops = {
- .open = mon_bin_vma_open,
- .close = mon_bin_vma_close,
- .fault = mon_bin_vma_fault,
+static const struct discontig_kernel_page_ops mon_discontig_ops = {
+ .init = mon_bin_discontig_init,
+ .get = mon_bin_discontig_get,
};
-static int mon_bin_mmap(struct file *filp, struct vm_area_struct *vma)
+static int mon_bin_mmap_prepare(struct vm_area_desc *desc)
{
- /* don't do anything here: "fault" will set up page table entries */
- vma->vm_ops = &mon_bin_vm_ops;
+ const struct file *filp = desc->file;
- if (vma->vm_flags & VM_WRITE)
+ if (vma_desc_test(desc, VMA_WRITE_BIT))
return -EPERM;
- vm_flags_mod(vma, VM_DONTEXPAND | VM_DONTDUMP, VM_MAYWRITE);
- vma->vm_private_data = filp->private_data;
- mon_bin_vma_open(vma);
+ desc->vm_ops = &mon_bin_vm_ops;
+ vma_desc_clear_flags(desc, VMA_MAYWRITE_BIT);
+ vma_desc_set_flags(desc, VMA_DONTEXPAND_BIT, VMA_DONTDUMP_BIT);
+ desc->private_data = filp->private_data;
+
+ mmap_action_map_discontig_kernel_pages(desc, NULL, &mon_discontig_ops);
return 0;
}
@@ -1298,7 +1322,7 @@ static const struct file_operations mon_fops_binary = {
.compat_ioctl = mon_bin_compat_ioctl,
#endif
.release = mon_bin_release,
- .mmap = mon_bin_mmap,
+ .mmap_prepare = mon_bin_mmap_prepare,
};
static int mon_bin_wait_event(struct file *file, struct mon_reader_bin *rp)
diff --git a/drivers/video/fbdev/core/fb_defio.c b/drivers/video/fbdev/core/fb_defio.c
index fd00b86e1ae60..fb359ecc39661 100644
--- a/drivers/video/fbdev/core/fb_defio.c
+++ b/drivers/video/fbdev/core/fb_defio.c
@@ -366,13 +366,13 @@ int fb_deferred_io_mmap(struct fb_info *info, struct vm_area_struct *vma)
{
vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
+ if (WARN_ON_ONCE(!(info->flags & FBINFO_VIRTFB)))
+ return -EINVAL;
if (!try_module_get(THIS_MODULE))
return -EINVAL;
vma->vm_ops = &fb_deferred_io_vm_ops;
- vm_flags_set(vma, VM_DONTEXPAND | VM_DONTDUMP);
- if (!(info->flags & FBINFO_VIRTFB))
- vm_flags_set(vma, VM_IO);
+ vm_flags_set(vma, VM_MIXEDMAP | VM_DONTEXPAND | VM_DONTDUMP);
vma->vm_private_data = info->fbdefio_state;
fb_deferred_io_state_get(info->fbdefio_state); /* released in vma->vm_ops->close() */
diff --git a/drivers/video/fbdev/ssd1307fb.c b/drivers/video/fbdev/ssd1307fb.c
index c4fdecafd8560..958514a354338 100644
--- a/drivers/video/fbdev/ssd1307fb.c
+++ b/drivers/video/fbdev/ssd1307fb.c
@@ -763,6 +763,8 @@ static int ssd1307fb_probe(struct i2c_client *client)
info->fix.smem_start = __pa(vmem);
info->fix.smem_len = vmem_size;
+ info->flags = FBINFO_VIRTFB;
+
fb_deferred_io_init(info);
i2c_set_clientdata(client, info);
diff --git a/fs/coredump.c b/fs/coredump.c
index ac3cd74808c64..9f729c594c47e 100644
--- a/fs/coredump.c
+++ b/fs/coredump.c
@@ -1608,7 +1608,7 @@ static unsigned long vma_dump_size(struct vm_area_struct *vma,
}
/* Hugetlb memory check */
- if (is_vm_hugetlb_page(vma)) {
+ if (vma_is_hugetlb(vma)) {
if ((vma->vm_flags & VM_SHARED) && FILTER(HUGETLB_SHARED))
goto whole;
if (!(vma->vm_flags & VM_SHARED) && FILTER(HUGETLB_PRIVATE))
@@ -1616,8 +1616,8 @@ static unsigned long vma_dump_size(struct vm_area_struct *vma,
return 0;
}
- /* Do not dump I/O mapped devices or special mappings */
- if (vma->vm_flags & VM_IO)
+ /* Do not dump memory-mapped I/O, which may have side effects on read. */
+ if (vma_test(vma, VMA_IO_BIT))
return 0;
/* By default, dump shared memory if mapped from an anonymous file. */
diff --git a/fs/fuse/dax.c b/fs/fuse/dax.c
index 85cdf0199bc0b..a5994f1c637d9 100644
--- a/fs/fuse/dax.c
+++ b/fs/fuse/dax.c
@@ -826,7 +826,7 @@ int fuse_dax_mmap(struct file *file, struct vm_area_struct *vma)
{
file_accessed(file);
vma->vm_ops = &fuse_dax_vm_ops;
- vm_flags_set(vma, VM_MIXEDMAP | VM_HUGEPAGE);
+ vma_set_flags(vma, VMA_HUGEPAGE_BIT);
return 0;
}
diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index 7611a8470ea26..ba7097d5720c0 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -108,7 +108,7 @@ static int hugetlbfs_file_mmap(struct file *file, struct vm_area_struct *vma)
* vma address alignment (but not the pgoff alignment) has
* already been checked by prepare_hugepage_range. If you add
* any error returns here, do so after setting VM_HUGETLB, so
- * is_vm_hugetlb_page tests below unmap_region go the right
+ * vma_is_hugetlb tests below unmap_region go the right
* way when do_mmap unwinds (may be important on powerpc
* and ia64).
*/
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index e671b4fd8dedd..565e6446bd312 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -3015,7 +3015,7 @@ static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end,
* hugetlb differs, see pagemap_hugetlb_category().
*/
categories = p->cur_vma_category;
- if (userfaultfd_wp(vma) && !is_vm_hugetlb_page(vma))
+ if (userfaultfd_wp(vma) && !vma_is_hugetlb(vma))
categories |= PAGE_IS_WRITTEN;
if (!pagemap_scan_is_interesting_page(categories, p))
@@ -3028,7 +3028,7 @@ static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end,
if (~p->arg.flags & PM_SCAN_WP_MATCHING)
return ret;
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
err = pagemap_scan_hugetlb_hole_wp(vma, addr, end);
else
err = uffd_wp_range(vma, addr, end - addr, true);
@@ -3470,7 +3470,7 @@ static int show_numa_map(struct seq_file *m, void *v)
seq_puts(m, " stack");
}
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
seq_puts(m, " huge");
/* Skip walking pages if gate VMA */
@@ -3499,7 +3499,7 @@ static int show_numa_map(struct seq_file *m, void *v)
if (md->swapcache)
seq_printf(m, " swapcache=%lu", md->swapcache);
- if (md->active < md->pages && !is_vm_hugetlb_page(vma))
+ if (md->active < md->pages && !vma_is_hugetlb(vma))
seq_printf(m, " active=%lu", md->active);
if (md->writeback)
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index 044dabc1fe9cb..48d47b34cc777 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -11,9 +11,9 @@
#ifndef _ASM_GENERIC__TLB_H
#define _ASM_GENERIC__TLB_H
+#include <linux/mm.h>
#include <linux/mmu_notifier.h>
#include <linux/swap.h>
-#include <linux/hugetlb_inline.h>
#include <asm/tlbflush.h>
#include <asm/cacheflush.h>
@@ -438,7 +438,7 @@ tlb_update_vma_flags(struct mmu_gather *tlb, struct vm_area_struct *vma)
* We rely on tlb_end_vma() to issue a flush, such that when we reset
* these values the batch is empty.
*/
- tlb->vma_huge = is_vm_hugetlb_page(vma);
+ tlb->vma_huge = vma_is_hugetlb(vma);
tlb->vma_exec = !!(vma->vm_flags & VM_EXEC);
/*
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 80a5a03e9cee7..24727ece20fe5 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -7,7 +7,6 @@
#include <linux/mm_types.h>
#include <linux/mmdebug.h>
#include <linux/fs.h>
-#include <linux/hugetlb_inline.h>
#include <linux/cgroup.h>
#include <linux/page_ref.h>
#include <linux/list.h>
@@ -252,14 +251,14 @@ extern void __hugetlb_zap_end(struct vm_area_struct *vma,
static inline void hugetlb_zap_begin(struct vm_area_struct *vma,
unsigned long *start, unsigned long *end)
{
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
__hugetlb_zap_begin(vma, start, end);
}
static inline void hugetlb_zap_end(struct vm_area_struct *vma,
struct zap_details *details)
{
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
__hugetlb_zap_end(vma, details);
}
diff --git a/include/linux/hugetlb_inline.h b/include/linux/hugetlb_inline.h
deleted file mode 100644
index 5c29cd3223a1e..0000000000000
--- a/include/linux/hugetlb_inline.h
+++ /dev/null
@@ -1,28 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _LINUX_HUGETLB_INLINE_H
-#define _LINUX_HUGETLB_INLINE_H
-
-#include <linux/mm.h>
-
-#ifdef CONFIG_HUGETLB_PAGE
-
-static inline bool is_vma_hugetlb_flags(const vma_flags_t *flags)
-{
- return vma_flags_test(flags, VMA_HUGETLB_BIT);
-}
-
-#else
-
-static inline bool is_vma_hugetlb_flags(const vma_flags_t *flags)
-{
- return false;
-}
-
-#endif
-
-static inline bool is_vm_hugetlb_page(const struct vm_area_struct *vma)
-{
- return is_vma_hugetlb_flags(&vma->flags);
-}
-
-#endif
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 969594074fd2d..1249e04d7b980 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -576,14 +576,6 @@ enum {
#define VM_ACCESS_FLAGS (VM_READ | VM_WRITE | VM_EXEC)
#define VMA_ACCESS_FLAGS mk_vma_flags(VMA_READ_BIT, VMA_WRITE_BIT, VMA_EXEC_BIT)
-/*
- * Special vmas that are non-mergable, non-mlock()able.
- */
-
-#define VMA_SPECIAL_FLAGS mk_vma_flags(VMA_IO_BIT, VMA_DONTEXPAND_BIT, \
- VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT)
-#define VM_SPECIAL vma_flags_to_legacy(VMA_SPECIAL_FLAGS)
-
/*
* Physically remapped pages are special. Tell the
* rest of the world about it:
@@ -600,9 +592,6 @@ enum {
#define VMA_REMAP_FLAGS mk_vma_flags(VMA_IO_BIT, VMA_PFNMAP_BIT, \
VMA_DONTEXPAND_BIT, VMA_DONTDUMP_BIT)
-/* This mask prevents VMA from being scanned with khugepaged */
-#define VM_NO_KHUGEPAGED (VM_SPECIAL | VM_HUGETLB)
-
/* This mask defines which mm->def_flags a process can inherit its parent */
#define VM_INIT_DEF_MASK VM_NOHUGEPAGE
@@ -1612,6 +1601,211 @@ static inline bool vma_is_shared_maywrite(const struct vm_area_struct *vma)
return is_shared_maywrite(&vma->flags);
}
+/**
+ * vma_flags_is_hugetlb() - Do the specified VMA flags indicate that the
+ * VMA is a hugetlb mapping?
+ * @flags: The VMA flags to test.
+ *
+ * Returns: true if the flags indicate a hugetlb mapping, false otherwise.
+ */
+static inline bool vma_flags_is_hugetlb(const vma_flags_t *flags)
+{
+ return IS_ENABLED(CONFIG_HUGETLB_PAGE) &&
+ vma_flags_test(flags, VMA_HUGETLB_BIT);
+}
+
+/**
+ * vma_is_hugetlb() - Is @vma a hugetlb mapping?
+ * @vma: The VMA to test.
+ *
+ * Returns: true if @vma is a hugetlb mapping, false otherwise.
+ */
+static inline bool vma_is_hugetlb(const struct vm_area_struct *vma)
+{
+ return vma_flags_is_hugetlb(&vma->flags);
+}
+
+/**
+ * vma_flags_is_kernel_owned() - Do the specified VMA flags indicate that the
+ * contents of the VMA are owned by the kernel rather than the core mm?
+ * @flags: The VMA flags to test.
+ *
+ * A kernel-owned mapping is one whose contents are established and controlled
+ * by the kernel, typically a driver, rather than by the core mm's fault and
+ * rmap machinery.
+ *
+ * The mapping may be memory-mapped I/O, kernel-allocated pages or ordinary
+ * pages the owner has chosen to map itself (shmem via a PFN map, for instance).
+ *
+ * In all cases the core mm must not populate, reclaim, migrate, copy-on-write
+ * or merge it of its own accord.
+ *
+ * Pages mapped this way are not necessarily reference counted or map counted.
+ *
+ * Returns: true if the flags indicate a kernel-owned mapping.
+ */
+static inline bool vma_flags_is_kernel_owned(const vma_flags_t *flags)
+{
+ return vma_flags_test_any(flags, VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT);
+}
+
+/**
+ * vma_is_kernel_owned() - Are the contents of @vma owned by the kernel?
+ * @vma: The VMA to test.
+ *
+ * See vma_flags_is_kernel_owned() for a description of this property.
+ *
+ * Returns: true if the VMA is kernel-owned.
+ */
+static inline bool vma_is_kernel_owned(const struct vm_area_struct *vma)
+{
+ return vma_flags_is_kernel_owned(&vma->flags);
+}
+
+/**
+ * vma_flags_is_fixed_mapping() - Do the specified VMA flags indicate that this
+ * is a fixed mapping that cannot be expanded or merged?
+ * @flags: The VMA flags to test.
+ *
+ * Fixed mappings are those whose size is set at the point of mmap (for
+ * instance, a kernel-owned mapping of a fixed range of memory), and thus
+ * cannot be expanded or merged.
+ *
+ * Returns: true if the flags indicate a fixed mapping.
+ */
+static inline bool vma_flags_is_fixed_mapping(const vma_flags_t *flags)
+{
+ /*
+ * VMA_PFNMAP_BIT should imply VMA_DONTEXPAND_BIT, but some callers set
+ * only the former.
+ */
+ return vma_flags_test_any(flags, VMA_PFNMAP_BIT, VMA_DONTEXPAND_BIT);
+}
+
+/**
+ * vma_is_fixed_mapping() - Is this VMA a fixed mapping that cannot be
+ * expanded or merged?
+ * @vma: The VMA to test.
+ *
+ * See vma_flags_is_fixed_mapping() for a description of this property.
+ *
+ * Returns: true if the VMA maps a fixed mapping.
+ */
+static inline bool vma_is_fixed_mapping(const struct vm_area_struct *vma)
+{
+ return vma_flags_is_fixed_mapping(&vma->flags);
+}
+
+/**
+ * vma_flags_can_merge() - Do the specified VMA flags permit the VMA to be
+ * merged with another?
+ * @flags: The VMA flags to test.
+ * Returns: true if the flags permit merging, false otherwise.
+ */
+static inline bool vma_flags_can_merge(const vma_flags_t *flags)
+{
+ /*
+ * VMA merging assumes that a VMA's flags and fields completely describe
+ * its state.
+ *
+ * However, kernel-owned mappings may have established state upon mapping
+ * not embodied in any attribute of the VMA.
+ *
+ * Additionally, private (CoW) PFN maps encode the source PFN of the
+ * range in vma->vm_pgoff, which may otherwise cause spurious merges.
+ */
+ if (vma_flags_is_kernel_owned(flags))
+ return false;
+ /* VMA explicitly marked as being unmergeable. */
+ if (vma_flags_is_fixed_mapping(flags))
+ return false;
+
+ return true;
+}
+
+/**
+ * vma_can_merge() - Do @vma's flags permit it to be merged with another VMA?
+ * @vma: The VMA to test.
+ * Returns: true if the flags permit merging, otherwise false.
+ */
+static inline bool vma_can_merge(const struct vm_area_struct *vma)
+{
+ return vma_flags_can_merge(&vma->flags);
+}
+
+/**
+ * vma_flags_is_persistent() - Do the specified VMA flags imply that the VMA
+ * contains persistent data?
+ * @flags: The VMA flags to test.
+ *
+ * Persistent in the sense that - if you write bytes to the mapping - do they
+ * stay written?
+ *
+ * If the kernel or a device could write to the memory independently of
+ * userland, or the kernel could arbitrarily discard it, then it is not
+ * persistent.
+ *
+ * Returns: true if the flags imply this VMA is persistent, otherwise false.
+ */
+static inline bool vma_flags_is_persistent(const vma_flags_t *flags)
+{
+ /* hugetlb is a fixed mapping, but its contents are the user's own. */
+ if (vma_flags_is_hugetlb(flags))
+ return true;
+ /*
+ * MMIO mappings may not store what is written and may be changed by the
+ * device. Kernel-owned and fixed mappings may be changed by their owner
+ * without the user having initiated it.
+ */
+ if (vma_flags_is_kernel_owned(flags) ||
+ vma_flags_is_fixed_mapping(flags))
+ return false;
+ /* Droppable memory is discardable by definition. */
+ return !vma_flags_test_single_mask(flags, VMA_DROPPABLE);
+}
+
+/**
+ * vma_is_persistent() - Does the VMA contain persistent data?
+ * @vma: The VMA to test.
+ *
+ * See vma_flags_is_persistent() for details.
+ *
+ * Returns: true if the VMA is persistent, otherwise false.
+ */
+static inline bool vma_is_persistent(const struct vm_area_struct *vma)
+{
+ return vma_flags_is_persistent(&vma->flags);
+}
+
+/**
+ * vma_flags_can_gup() - Do the specified VMA flags permit GUP to access the
+ * mapping's pages?
+ * @flags: The VMA flags to test.
+ *
+ * GUP cannot obtain pages from a PFN map (VMA_PFNMAP_BIT), which may have no
+ * struct pages behind it, and must not provide access to memory-mapped I/O
+ * (VMA_IO_BIT).
+ *
+ * Returns: true if GUP may access pages from the mapping, otherwise false.
+ */
+static inline bool vma_flags_can_gup(const vma_flags_t *flags)
+{
+ return !vma_flags_test_any(flags, VMA_IO_BIT, VMA_PFNMAP_BIT);
+}
+
+/**
+ * vma_can_gup() - May GUP obtain pages from @vma?
+ * @vma: The VMA to test.
+ *
+ * See vma_flags_can_gup() for details.
+ *
+ * Returns: true if GUP may access pages from the mapping, otherwise false.
+ */
+static inline bool vma_can_gup(const struct vm_area_struct *vma)
+{
+ return vma_flags_can_gup(&vma->flags);
+}
+
/**
* vma_kernel_pagesize - Default page size granularity for this VMA.
* @vma: The user mapping.
@@ -4602,7 +4796,7 @@ static inline void mmap_action_map_kernel_pages(struct vm_area_desc *desc,
{
struct mmap_action *action = &desc->action;
- action->type = MMAP_MAP_KERNEL_PAGES;
+ action->type = MMAP_KERNEL_PAGES;
action->map_kernel.start = start;
action->map_kernel.pages = pages;
action->map_kernel.nr_pages = nr_pages;
@@ -4626,10 +4820,55 @@ static inline void mmap_action_map_kernel_pages_full(struct vm_area_desc *desc,
vma_desc_pages(desc));
}
+static inline
+void mmap_action_map_discontig_kernel_pages(struct vm_area_desc *desc,
+ void *init_private, const struct discontig_kernel_page_ops *ops)
+{
+ struct mmap_action *action = &desc->action;
+
+ action->type = MMAP_DISCONTIG_KERNEL_PAGES;
+ action->map_kernel_discontig.init_private = init_private;
+ action->map_kernel_discontig.ops = ops;
+}
+
int mmap_action_prepare(struct vm_area_desc *desc);
int mmap_action_complete(struct vm_area_struct *vma,
struct mmap_action *action, bool is_compat);
+static inline void
+discontig_kernel_map_abort(struct discontig_kernel_page_state *state)
+{
+ state->action = DISCONTIG_KERNEL_PAGE_ABORT;
+}
+
+static inline void
+discontig_kernel_map_page(struct discontig_kernel_page_state *state,
+ struct page *page)
+{
+ struct folio *folio = page_folio(page);
+
+ if (folio_test_large(folio)) {
+ VM_WARN_ON_ONCE(page != folio_page(folio, 0));
+ state->action = DISCONTIG_KERNEL_PAGE_MAP_COMPOUND_PAGE;
+ state->__folio = folio;
+ state->__nr_pages = min(state->nr_pages_remain,
+ folio_nr_pages(folio));
+ } else {
+ state->action = DISCONTIG_KERNEL_PAGE_MAP_PAGE;
+ state->__page = page;
+ state->__nr_pages = 1;
+ }
+}
+
+static inline void
+discontig_kernel_map_page_range(struct discontig_kernel_page_state *state,
+ struct page **page_arr, unsigned long nr_pages)
+{
+ state->action = DISCONTIG_KERNEL_PAGE_MAP_PAGE_RANGE;
+ state->__page_arr = page_arr;
+ state->__nr_pages = nr_pages;
+}
+
/* Look up the first VMA which exactly match the interval vm_start ... vm_end */
static inline struct vm_area_struct *find_exact_vma(struct mm_struct *mm,
unsigned long vm_start, unsigned long vm_end)
@@ -4747,9 +4986,6 @@ int remap_pfn_range(struct vm_area_struct *vma, unsigned long addr,
int vm_insert_page(struct vm_area_struct *, unsigned long addr, struct page *);
int vm_insert_pages(struct vm_area_struct *vma, unsigned long addr,
struct page **pages, unsigned long *num);
-int map_kernel_pages_prepare(struct vm_area_desc *desc);
-int map_kernel_pages_complete(struct vm_area_struct *vma,
- struct mmap_action *action);
int vm_map_pages(struct vm_area_struct *vma, struct page **pages,
unsigned long num);
int vm_map_pages_zero(struct vm_area_struct *vma, struct page **pages,
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 5413bd10fff2c..0cb4f96039568 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -815,11 +815,47 @@ struct pfnmap_track_ctx {
/* What action should be taken after an .mmap_prepare call is complete? */
enum mmap_action_type {
- MMAP_NOTHING, /* Mapping is complete, no further action. */
- MMAP_REMAP_PFN, /* Remap PFN range. */
- MMAP_IO_REMAP_PFN, /* I/O remap PFN range. */
- MMAP_SIMPLE_IO_REMAP, /* I/O remap with guardrails. */
- MMAP_MAP_KERNEL_PAGES, /* Map kernel page range from array. */
+ MMAP_NOTHING,
+ MMAP_REMAP_PFN,
+ MMAP_IO_REMAP_PFN,
+ MMAP_SIMPLE_IO_REMAP, /* I/O remap with guardrails. */
+ MMAP_KERNEL_PAGES, /* Map kernel page range from array. */
+ MMAP_DISCONTIG_KERNEL_PAGES, /* Map kernel discontig page range. */
+};
+
+enum discontig_kernel_page_action {
+ DISCONTIG_KERNEL_PAGE_ABORT,
+ DISCONTIG_KERNEL_PAGE_MAP_PAGE,
+ DISCONTIG_KERNEL_PAGE_MAP_COMPOUND_PAGE,
+ DISCONTIG_KERNEL_PAGE_MAP_PAGE_RANGE,
+};
+
+struct discontig_kernel_page_state {
+ /* Map state. */
+ const unsigned long start; /* Start address of VMA. */
+ const unsigned long end; /* End address of VMA. */
+ unsigned long addr; /* The current address to be mapped. */
+ pgoff_t pgoff; /* The current pgoff to be mapped. */
+ unsigned long nr_pages_mapped; /* The number of pages mapped. */
+ unsigned long nr_pages_remain; /* The number of pages remaining. */
+
+ /* User-defined state. */
+ void *vm_private_data; /* VMA private data. */
+ void *private; /* Mapping private data. */
+
+ /* Users should not touch these, use discontig_kernel_map_*() helpers. */
+ enum discontig_kernel_page_action action;
+ union {
+ struct page *__page;
+ struct folio *__folio;
+ struct page **__page_arr;
+ };
+ unsigned long __nr_pages;
+};
+
+struct discontig_kernel_page_ops {
+ int (*init)(void *vm_private_data, void **private);
+ int (*get)(struct discontig_kernel_page_state *state);
};
/*
@@ -844,6 +880,10 @@ struct mmap_action {
unsigned long nr_pages;
pgoff_t pgoff;
} map_kernel;
+ struct {
+ void *init_private;
+ const struct discontig_kernel_page_ops *ops;
+ } map_kernel_discontig;
};
enum mmap_action_type type;
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 939f3a5e973f6..d7d8b312466c2 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -14,7 +14,6 @@
#include <linux/gfp.h>
#include <linux/bitops.h>
#include <linux/hardirq.h> /* for in_interrupt() */
-#include <linux/hugetlb_inline.h>
struct folio_batch;
diff --git a/include/linux/rmap.h b/include/linux/rmap.h
index 0b332770abeed..74cca0e3c7264 100644
--- a/include/linux/rmap.h
+++ b/include/linux/rmap.h
@@ -888,7 +888,7 @@ struct page_vma_mapped_walk {
static inline void page_vma_mapped_walk_done(struct page_vma_mapped_walk *pvmw)
{
/* HugeTLB pte is set to the relevant page table entry without pte_mapped. */
- if (pvmw->pte && !is_vm_hugetlb_page(pvmw->vma))
+ if (pvmw->pte && !vma_is_hugetlb(pvmw->vma))
pte_unmap(pvmw->pte);
if (pvmw->ptl)
spin_unlock(pvmw->ptl);
diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h
index a4351cffc60ce..a14b8a9ffb7b1 100644
--- a/include/linux/userfaultfd_k.h
+++ b/include/linux/userfaultfd_k.h
@@ -18,7 +18,6 @@
#include <linux/swap.h>
#include <linux/leafops.h>
#include <asm-generic/pgtable_uffd.h>
-#include <linux/hugetlb_inline.h>
/* The set of all possible UFFD-related VM flags. */
#define __VM_UFFD_FLAGS (VM_UFFD_MISSING | VM_UFFD_MINOR | \
diff --git a/kernel/bpf/arena.c b/kernel/bpf/arena.c
index 7b6847200b431..b69fe5e343393 100644
--- a/kernel/bpf/arena.c
+++ b/kernel/bpf/arena.c
@@ -620,8 +620,9 @@ static int arena_map_mmap(struct bpf_map *map, struct vm_area_struct *vma)
* clears VM_MAYEXEC. Set VM_DONTEXPAND to avoid potential change
* of user_vm_start. Set VM_DONTCOPY to prevent arena VMA from
* being copied into the child process on fork.
+ * This is a kernel page so set VM_MIXEDMAP.
*/
- vm_flags_set(vma, VM_DONTEXPAND | VM_DONTCOPY);
+ vm_flags_set(vma, VM_MIXEDMAP | VM_DONTEXPAND | VM_DONTCOPY);
vma->vm_ops = &arena_vm_ops;
return 0;
}
diff --git a/kernel/events/core.c b/kernel/events/core.c
index a6c8e38a31104..8ca8a68429242 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -9808,7 +9808,7 @@ static void perf_event_mmap_event(struct perf_mmap_event *mmap_event)
if (vma->vm_flags & VM_LOCKED)
flags |= MAP_LOCKED;
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
flags |= MAP_HUGETLB;
if (file) {
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index 7709ea8824778..b89cc5cee0027 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -1726,8 +1726,8 @@ static int xol_add_vma(struct mm_struct *mm, struct xol_area *area)
}
vma = _install_special_mapping(mm, area->vaddr, PAGE_SIZE,
- VM_EXEC|VM_MAYEXEC|VM_DONTCOPY|VM_IO|
- VM_SEALED_SYSMAP,
+ VM_EXEC|VM_MAYEXEC|VM_DONTCOPY|
+ VM_MIXEDMAP|VM_SEALED_SYSMAP,
&xol_mapping);
if (IS_ERR(vma)) {
ret = PTR_ERR(vma);
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 8dff37059faf7..ae6c1a606eb5d 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -22,7 +22,6 @@
*/
#include <linux/energy_model.h>
#include <linux/mmap_lock.h>
-#include <linux/hugetlb_inline.h>
#include <linux/jiffies.h>
#include <linux/mm_api.h>
#include <linux/highmem.h>
@@ -4212,7 +4211,7 @@ static void task_numa_work(struct callback_head *work)
for (; vma; vma = vma_next(&vmi)) {
if (!vma_migratable(vma) || !vma_policy_mof(vma) ||
- is_vm_hugetlb_page(vma) || (vma->vm_flags & VM_MIXEDMAP)) {
+ vma_is_hugetlb(vma) || vma_is_kernel_owned(vma)) {
trace_sched_skip_vma_numa(mm, vma, NUMAB_SKIP_UNSUITABLE);
continue;
}
diff --git a/mm/folio.c b/mm/folio.c
index 47a437e0f7fde..35e242b48870b 100644
--- a/mm/folio.c
+++ b/mm/folio.c
@@ -505,7 +505,7 @@ void folio_add_lru_vma(struct folio *folio, struct vm_area_struct *vma)
{
VM_BUG_ON_FOLIO(folio_test_lru(folio), folio);
- if (unlikely((vma->vm_flags & (VM_LOCKED | VM_SPECIAL)) == VM_LOCKED))
+ if (vma_test(vma, VMA_LOCKED_BIT))
mlock_new_folio(folio);
else
folio_add_lru(folio);
diff --git a/mm/gup.c b/mm/gup.c
index c2dfcb4744bc3..8e9ef5ee7498c 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -621,7 +621,7 @@ static struct page *no_page_table(struct vm_area_struct *vma,
* But we can only make this optimization where a hole would surely
* be zero-filled if handle_mm_fault() actually did handle it.
*/
- if (is_vm_hugetlb_page(vma)) {
+ if (vma_is_hugetlb(vma)) {
struct hstate *h = hstate_vma(vma);
if (!hugetlbfs_pagecache_present(h, vma, address))
@@ -1204,7 +1204,7 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags)
int foreign = (gup_flags & FOLL_REMOTE);
bool vma_anon = vma_is_anonymous(vma);
- if (vm_flags & (VM_IO | VM_PFNMAP))
+ if (!vma_can_gup(vma))
return -EFAULT;
if ((gup_flags & FOLL_ANON) && !vma_anon)
@@ -1213,7 +1213,7 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags)
if ((gup_flags & FOLL_LONGTERM) && vma_is_fsdax(vma))
return -EOPNOTSUPP;
- if ((gup_flags & FOLL_SPLIT_PMD) && is_vm_hugetlb_page(vma))
+ if ((gup_flags & FOLL_SPLIT_PMD) && vma_is_hugetlb(vma))
return -EOPNOTSUPP;
if (vma_is_secretmem(vma))
@@ -1836,6 +1836,10 @@ long populate_vma_page_range(struct vm_area_struct *vma,
if (!vma_is_accessible(vma))
return -EFAULT;
+ /* Unreadable VMAs also cannot be faulted in. */
+ if (!vma_test(vma, VMA_MAYREAD_BIT))
+ return -EFAULT;
+
gup_flags = FOLL_TOUCH;
/*
* We want to touch writable mappings with a write fault in order
@@ -1951,7 +1955,7 @@ int __mm_populate(unsigned long start, unsigned long len, int ignore_errors)
* range with the first VMA. Also, skip undesirable VMA types.
*/
nend = min(end, vma->vm_end);
- if (vma->vm_flags & (VM_IO | VM_PFNMAP))
+ if (!vma_can_gup(vma))
continue;
if (nstart < vma->vm_start)
nstart = vma->vm_start;
@@ -2013,8 +2017,7 @@ static long __get_user_pages_locked(struct mm_struct *mm, unsigned long start,
break;
/* protect what we can, including chardevs */
- if ((vma->vm_flags & (VM_IO | VM_PFNMAP)) ||
- !(vm_flags & vma->vm_flags))
+ if (!vma_can_gup(vma) || !(vm_flags & vma->vm_flags))
break;
if (pages) {
diff --git a/mm/hmm.c b/mm/hmm.c
index 2f1e98c6b6440..e9569b82a1f0c 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -595,8 +595,7 @@ static int hmm_vma_walk_test(unsigned long start, unsigned long end,
struct hmm_range *range = hmm_vma_walk->range;
struct vm_area_struct *vma = walk->vma;
- if (!(vma->vm_flags & (VM_IO | VM_PFNMAP)) &&
- vma->vm_flags & VM_READ)
+ if (vma_can_gup(vma) && vma_test(vma, VMA_READ_BIT))
return 0;
/*
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 4cd917f77f3f7..3cb8e2d4d65cb 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -110,14 +110,6 @@ static inline bool file_thp_enabled(const struct vm_area_struct *vma)
return S_ISREG(inode->i_mode);
}
-/* If returns true, we are unable to access the VMA's folios. */
-static bool vma_is_special_huge(const struct vm_area_struct *vma)
-{
- if (vma_is_dax(vma))
- return false;
- return vma_test_any(vma, VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT);
-}
-
static bool vma_file_bypass_thp_tuneables(const struct vm_area_struct *vma,
enum tva_type type)
{
@@ -192,7 +184,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
/* Check the intersection of requested and supported orders. */
if (vma_is_anonymous(vma))
supported_orders = THP_ORDERS_ALL_ANON;
- else if (vma_is_dax(vma) || vma_is_special_huge(vma))
+ else if (vma_is_dax(vma) || vma_is_kernel_owned(vma))
supported_orders = THP_ORDERS_ALL_SPECIAL_DAX;
else
supported_orders = THP_ORDERS_ALL_FILE_DEFAULT;
@@ -212,11 +204,14 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
return in_pf ? orders : 0;
/*
- * khugepaged special VMA and hugetlb VMA.
- * Must be checked after dax since some dax mappings may have
- * VM_MIXEDMAP set.
+ * khugepaged moves data from VMAs once collapsed, after they have been
+ * faulted in, relying on refaulting for file-backed memory.
+ *
+ * Kernel-owned mappings cannot be reliably reconstructed from page
+ * faults, and fixed mappings (including hugetlb) may not be marked as
+ * kernel-owned - precisely the mappings which cannot be merged.
*/
- if (!in_pf && !smaps && (vm_flags & VM_NO_KHUGEPAGED))
+ if (!in_pf && !smaps && !vma_can_merge(vma))
return 0;
/*
@@ -3063,7 +3058,7 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma,
orig_pud = pudp_huge_get_and_clear_full(vma, addr, pud, tlb->fullmm);
arch_check_zapped_pud(vma, orig_pud);
tlb_remove_pud_tlb_entry(tlb, pud, addr);
- if (vma_is_special_huge(vma)) {
+ if (vma_is_kernel_owned(vma)) {
spin_unlock(ptl);
/* No zero page support yet */
} else {
@@ -3219,7 +3214,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,
*/
if (arch_needs_pgtable_deposit())
zap_deposited_table(mm, pmd);
- if (vma_is_special_huge(vma))
+ if (vma_is_kernel_owned(vma))
return;
if (unlikely(pmd_is_migration_entry(old_pmd))) {
const softleaf_t old_entry = softleaf_from_pmd(old_pmd);
@@ -4762,11 +4757,9 @@ static inline bool vma_not_suitable_for_thp_split(struct vm_area_struct *vma)
{
if (vma_is_dax(vma))
return true;
- if (vma_is_special_huge(vma))
- return true;
- if (vma_test(vma, VMA_IO_BIT))
+ if (vma_is_kernel_owned(vma))
return true;
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
return true;
return false;
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index d3a0650ff6905..817f57f13b09d 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1147,7 +1147,7 @@ static inline struct resv_map *inode_resv_map(struct inode *inode)
static struct resv_map *vma_resv_map(struct vm_area_struct *vma)
{
- VM_BUG_ON_VMA(!is_vm_hugetlb_page(vma), vma);
+ VM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);
if (vma->vm_flags & VM_MAYSHARE) {
struct address_space *mapping = vma->vm_file->f_mapping;
struct inode *inode = mapping->host;
@@ -1162,7 +1162,7 @@ static struct resv_map *vma_resv_map(struct vm_area_struct *vma)
static void set_vma_resv_map(struct vm_area_struct *vma, struct resv_map *map)
{
- VM_WARN_ON_ONCE_VMA(!is_vm_hugetlb_page(vma), vma);
+ VM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);
VM_WARN_ON_ONCE_VMA(vma_test(vma, VMA_MAYSHARE_BIT), vma);
set_vma_private_data(vma, (unsigned long)map);
@@ -1170,7 +1170,7 @@ static void set_vma_resv_map(struct vm_area_struct *vma, struct resv_map *map)
static void set_vma_resv_flags(struct vm_area_struct *vma, unsigned long flags)
{
- VM_WARN_ON_ONCE_VMA(!is_vm_hugetlb_page(vma), vma);
+ VM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);
VM_WARN_ON_ONCE_VMA(vma_test(vma, VMA_MAYSHARE_BIT), vma);
set_vma_private_data(vma, get_vma_private_data(vma) | flags);
@@ -1178,7 +1178,7 @@ static void set_vma_resv_flags(struct vm_area_struct *vma, unsigned long flags)
static int is_vma_resv_set(struct vm_area_struct *vma, unsigned long flag)
{
- VM_BUG_ON_VMA(!is_vm_hugetlb_page(vma), vma);
+ VM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);
return (get_vma_private_data(vma) & flag) != 0;
}
@@ -1192,7 +1192,7 @@ bool __vma_private_lock(struct vm_area_struct *vma)
void hugetlb_dup_vma_private(struct vm_area_struct *vma)
{
- VM_BUG_ON_VMA(!is_vm_hugetlb_page(vma), vma);
+ VM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);
/*
* Clear vm_private_data
* - For shared mappings this is a per-vma semaphore that may be
@@ -5279,7 +5279,7 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
unsigned long last_addr_mask;
i_mmap_assert_write_locked(vma->vm_file->f_mapping);
- WARN_ON(!is_vm_hugetlb_page(vma));
+ WARN_ON(!vma_is_hugetlb(vma));
BUG_ON(start & ~huge_page_mask(h));
BUG_ON(end & ~huge_page_mask(h));
@@ -7505,6 +7505,6 @@ void hugetlb_unshare_all_pmds(struct vm_area_struct *vma)
*/
void fixup_hugetlb_reservations(struct vm_area_struct *vma)
{
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
clear_vma_resv_huge_pages(vma);
}
diff --git a/mm/internal.h b/mm/internal.h
index 0dca33db068f6..83a4ba52aaebc 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -7,6 +7,7 @@
#ifndef __MM_INTERNAL_H
#define __MM_INTERNAL_H
+#include <linux/file.h>
#include <linux/fs.h>
#include <linux/khugepaged.h>
#include <linux/mm.h>
@@ -212,6 +213,24 @@ static inline void *folio_raw_mapping(const struct folio *folio)
return (void *)(mapping & ~FOLIO_MAPPING_FLAGS);
}
+/*
+ * If the VMA has a close hook then close it, and since closing it might leave
+ * it in an inconsistent state which makes the use of any hooks suspect, clear
+ * them down by installing dummy empty hooks.
+ */
+static inline void vma_close(struct vm_area_struct *vma)
+{
+ if (vma->vm_ops && vma->vm_ops->close) {
+ vma->vm_ops->close(vma);
+
+ /*
+ * The mapping is in an inconsistent state, and no further hooks
+ * may be invoked upon it.
+ */
+ vma->vm_ops = &vma_dummy_vm_ops;
+ }
+}
+
/*
* This is a file-backed mapping, and is about to be memory mapped - invoke its
* mmap hook and safely handle error conditions. On error, VMA hooks will be
@@ -224,8 +243,11 @@ static inline void *folio_raw_mapping(const struct folio *folio)
*/
static inline int mmap_file(struct file *file, struct vm_area_struct *vma)
{
- int err = vfs_mmap(file, vma);
+ const unsigned long prev_start = vma->vm_start;
+ const vma_flags_t prev_flags = vma->flags;
+ int err;
+ err = vfs_mmap(file, vma);
/*
* Either we tried to call the file hook for mmap() and an error arose
* or a driver set vma->vm_ops = NULL intending there to be no VMA
@@ -238,26 +260,16 @@ static inline int mmap_file(struct file *file, struct vm_area_struct *vma)
*/
if (unlikely(err || !vma->vm_ops))
vma->vm_ops = &vma_dummy_vm_ops;
+ if (unlikely(err))
+ return err;
- return err;
-}
-
-/*
- * If the VMA has a close hook then close it, and since closing it might leave
- * it in an inconsistent state which makes the use of any hooks suspect, clear
- * them down by installing dummy empty hooks.
- */
-static inline void vma_close(struct vm_area_struct *vma)
-{
- if (vma->vm_ops && vma->vm_ops->close) {
- vma->vm_ops->close(vma);
-
- /*
- * The mapping is in an inconsistent state, and no further hooks
- * may be invoked upon it.
- */
- vma->vm_ops = &vma_dummy_vm_ops;
+ err = mmap_hook_validate(prev_start, &prev_flags, vma);
+ if (unlikely(err)) {
+ vma->vm_start = prev_start;
+ vma_close(vma);
}
+
+ return err;
}
/* unmap_vmas is in mm/memory.c */
@@ -957,15 +969,7 @@ void mlock_folio(struct folio *folio);
static inline void mlock_vma_folio(struct folio *folio,
struct vm_area_struct *vma)
{
- /*
- * The VM_SPECIAL check here serves two purposes.
- * 1) VM_IO check prevents migration from double-counting during mlock.
- * 2) Although mmap_region() and mlock_fixup() take care that VM_LOCKED
- * is never left set on a VM_SPECIAL vma, there is an interval while
- * file->f_op->mmap() is using vm_insert_page(s), when VM_LOCKED may
- * still be set while VM_SPECIAL bits are added: so ignore it then.
- */
- if (unlikely((vma->vm_flags & (VM_LOCKED|VM_SPECIAL)) == VM_LOCKED))
+ if (vma_test(vma, VMA_LOCKED_BIT))
mlock_folio(folio);
}
@@ -982,7 +986,12 @@ static inline void munlock_vma_folio(struct folio *folio,
* always munlock the folio and page reclaim will correct it
* if it's wrong.
*/
- if (unlikely(vma->vm_flags & VM_LOCKED))
+ /*
+ * VMA_LOCKONFAULT_BIT alone marks an mlock walk in progress, see
+ * mlock_vma_pages_range(). An unmap racing with the walk must still
+ * munlock folios the walk has already counted.
+ */
+ if (unlikely(vma_test_any_mask(vma, VMA_LOCKED_MASK)))
munlock_folio(folio);
}
@@ -1102,11 +1111,9 @@ static inline struct file *maybe_unlock_mmap_for_io(struct vm_fault *vmf,
static inline bool vma_supports_mlock(const struct vm_area_struct *vma)
{
- if (vma_test_any_mask(vma, VMA_SPECIAL_FLAGS))
- return false;
- if (vma_test_single_mask(vma, VMA_DROPPABLE))
+ if (!vma_is_persistent(vma))
return false;
- if (vma_is_dax(vma) || is_vm_hugetlb_page(vma))
+ if (vma_is_dax(vma) || vma_is_hugetlb(vma))
return false;
return vma != get_gate_vma(current->mm);
}
@@ -1499,6 +1506,12 @@ int remap_pfn_range_prepare(struct vm_area_desc *desc);
int remap_pfn_range_complete(struct vm_area_struct *vma,
struct mmap_action *action);
int simple_ioremap_prepare(struct vm_area_desc *desc);
+int map_kernel_pages_prepare(struct vm_area_desc *desc);
+int map_kernel_pages_complete(struct vm_area_struct *vma,
+ struct mmap_action *action);
+int map_discontig_kernel_pages_prepare(struct vm_area_desc *desc);
+int map_discontig_kernel_pages_complete(struct vm_area_struct *vma,
+ struct mmap_action *action);
static inline int io_remap_pfn_range_prepare(struct vm_area_desc *desc)
{
diff --git a/mm/ksm.c b/mm/ksm.c
index 624f37975e129..f80372bfd4b2f 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -747,9 +747,7 @@ static bool ksm_compatible(const struct file *file, vma_flags_t vma_flags)
if (vma_flags_test_any(&vma_flags, VMA_SHARED_BIT, VMA_MAYSHARE_BIT,
VMA_HUGETLB_BIT))
return false;
- if (vma_flags_test_single_mask(&vma_flags, VMA_DROPPABLE))
- return false;
- if (vma_flags_test_any_mask(&vma_flags, VMA_SPECIAL_FLAGS))
+ if (!vma_flags_is_persistent(&vma_flags))
return false;
if (file_is_dax(file))
return false;
diff --git a/mm/madvise.c b/mm/madvise.c
index fbb72ab49aa64..1af82b044d235 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -881,7 +881,7 @@ bool madvise_dontneed_free_valid_vma(struct madvise_behavior *madv_behavior)
int behavior = madv_behavior->behavior;
struct madvise_behavior_range *range = &madv_behavior->range;
- if (!is_vm_hugetlb_page(vma)) {
+ if (!vma_is_hugetlb(vma)) {
unsigned int forbidden = VM_PFNMAP;
if (behavior != MADV_DONTNEED_LOCKED)
@@ -1221,19 +1221,25 @@ static long madvise_remove(struct madvise_behavior *madv_behavior)
return error;
}
-static bool is_valid_guard_vma(struct vm_area_struct *vma, bool allow_locked)
+static bool is_valid_guard_vma(const struct vm_area_struct *vma,
+ bool allow_locked)
{
- vm_flags_t disallowed = VM_SPECIAL | VM_HUGETLB;
-
/*
- * A user could lock after setting a guard range but that's fine, as
+ * A user could lock after setting a guard range but that's fine as
* they'd not be able to fault in. The issue arises when we try to zap
* existing locked VMAs. We don't want to do that.
*/
- if (!allow_locked)
- disallowed |= VM_LOCKED;
+ if (!allow_locked && vma_test(vma, VMA_LOCKED_BIT))
+ return false;
+ /*
+ * Guard regions require a VMA whose page tables are managed solely by
+ * the core, which is also what merging requires, so disallow any flags
+ * that would prevent a merge.
+ */
+ if (!vma_can_merge(vma))
+ return false;
- return !(vma->vm_flags & disallowed);
+ return true;
}
static bool is_guard_pte_marker(pte_t ptent)
@@ -1559,7 +1565,7 @@ static int madvise_vma_behavior(struct madvise_behavior *madv_behavior)
new_flags |= VM_DONTCOPY;
break;
case MADV_DOFORK:
- if (new_flags & VM_SPECIAL)
+ if (!vma_can_merge(vma))
return -EINVAL;
new_flags &= ~VM_DONTCOPY;
break;
@@ -1578,8 +1584,8 @@ static int madvise_vma_behavior(struct madvise_behavior *madv_behavior)
new_flags |= VM_DONTDUMP;
break;
case MADV_DODUMP:
- if ((!is_vm_hugetlb_page(vma) && (new_flags & VM_SPECIAL)) ||
- (new_flags & VM_DROPPABLE))
+ /* Non-persistent memory cannot be dumped. */
+ if (!vma_is_persistent(vma))
return -EINVAL;
new_flags &= ~VM_DONTDUMP;
break;
diff --git a/mm/memory.c b/mm/memory.c
index 926276d419202..9e4a70421a6b8 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -1564,7 +1564,7 @@ copy_page_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma)
if (!vma_needs_copy(dst_vma, src_vma))
return 0;
- if (is_vm_hugetlb_page(src_vma))
+ if (vma_is_hugetlb(src_vma))
return copy_hugetlb_page_range(dst_mm, src_mm, dst_vma, src_vma);
/*
@@ -2178,7 +2178,7 @@ static void __zap_vma_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
if (vma->vm_file && !reaping)
uprobe_munmap(vma, start, end);
- if (unlikely(is_vm_hugetlb_page(vma))) {
+ if (unlikely(vma_is_hugetlb(vma))) {
zap_flags_t zap_flags = details ? details->zap_flags : 0;
VM_WARN_ON_ONCE(reaping);
@@ -2313,7 +2313,7 @@ void zap_vma_range_batched(struct mmu_gather *tlb,
*/
__zap_vma_range(tlb, vma, address, end, details);
mmu_notifier_invalidate_range_end(&range);
- if (is_vm_hugetlb_page(vma)) {
+ if (vma_is_hugetlb(vma)) {
/*
* flush tlb and free resources before hugetlb_zap_end(), to
* avoid concurrent page faults' allocation failure.
@@ -2343,19 +2343,19 @@ void zap_vma_range(struct vm_area_struct *vma, unsigned long address,
}
/**
- * zap_special_vma_range - zap all page table entries in a special vma range
+ * zap_special_vma_range - zap all page table entries in a kernel-owned VMA
* @vma: the vma covering the range to zap
* @address: starting address of the range to zap
* @size: number of bytes to zap
*
* This function does nothing when the provided address range is not fully
- * contained in @vma, or when the @vma is not VM_PFNMAP or VM_MIXEDMAP.
+ * contained in @vma, or when @vma is not kernel-owned.
*/
void zap_special_vma_range(struct vm_area_struct *vma, unsigned long address,
unsigned long size)
{
if (!range_in_vma(vma, address, address + size) ||
- !(vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP)))
+ !vma_is_kernel_owned(vma))
return;
zap_vma_range(vma, address, size);
@@ -2417,11 +2417,11 @@ static bool vm_mixed_zeropage_allowed(struct vm_area_struct *vma)
* be problematic as soon as the zeropage gets replaced by a different
* page due to vma->vm_ops->pfn_mkwrite, because what's mapped would
* now differ to what GUP looked up. FSDAX is incompatible to
- * FOLL_LONGTERM and VM_IO is incompatible to GUP completely (see
- * check_vma_flags).
+ * FOLL_LONGTERM and memory-mapped I/O is incompatible to GUP completely
+ * (see vma_can_gup()).
*/
return vma->vm_ops && vma->vm_ops->pfn_mkwrite &&
- (vma_is_fsdax(vma) || vma->vm_flags & VM_IO);
+ (vma_is_fsdax(vma) || vma_test(vma, VMA_IO_BIT));
}
static int validate_page_before_insert(struct vm_area_struct *vma,
@@ -2609,17 +2609,23 @@ int vm_insert_pages(struct vm_area_struct *vma, unsigned long addr,
}
EXPORT_SYMBOL(vm_insert_pages);
+static void __map_kernel_pages_prepare(struct vm_area_desc *desc)
+{
+ if (vma_desc_test(desc, VMA_MIXEDMAP_BIT))
+ return;
+
+ VM_WARN_ON_ONCE(mmap_read_trylock(desc->mm));
+ VM_WARN_ON_ONCE(vma_desc_test(desc, VMA_PFNMAP_BIT));
+ vma_desc_set_flags(desc, VMA_MIXEDMAP_BIT);
+}
+
int map_kernel_pages_prepare(struct vm_area_desc *desc)
{
const struct mmap_action *action = &desc->action;
const unsigned long addr = action->map_kernel.start;
unsigned long nr_pages, end;
- if (!vma_desc_test(desc, VMA_MIXEDMAP_BIT)) {
- VM_WARN_ON_ONCE(mmap_read_trylock(desc->mm));
- VM_WARN_ON_ONCE(vma_desc_test(desc, VMA_PFNMAP_BIT));
- vma_desc_set_flags(desc, VMA_MIXEDMAP_BIT);
- }
+ __map_kernel_pages_prepare(desc);
nr_pages = action->map_kernel.nr_pages;
end = addr + PAGE_SIZE * nr_pages;
@@ -2628,7 +2634,6 @@ int map_kernel_pages_prepare(struct vm_area_desc *desc)
return 0;
}
-EXPORT_SYMBOL(map_kernel_pages_prepare);
int map_kernel_pages_complete(struct vm_area_struct *vma,
struct mmap_action *action)
@@ -2640,7 +2645,98 @@ int map_kernel_pages_complete(struct vm_area_struct *vma,
action->map_kernel.pages,
&nr_pages, vma->vm_page_prot);
}
-EXPORT_SYMBOL(map_kernel_pages_complete);
+
+int map_discontig_kernel_pages_prepare(struct vm_area_desc *desc)
+{
+ const struct mmap_action *action = &desc->action;
+ const struct discontig_kernel_page_ops *ops =
+ action->map_kernel_discontig.ops;
+
+ /* At minimum need to be able to get pages. */
+ if (WARN_ON_ONCE(!ops->get))
+ return -EINVAL;
+
+ __map_kernel_pages_prepare(desc);
+ return 0;
+}
+
+static int apply_discontig_action(struct vm_area_struct *vma,
+ struct discontig_kernel_page_state *state)
+{
+ unsigned long nr_pages = state->__nr_pages;
+ unsigned long addr = state->addr;
+ unsigned long i;
+
+ if (state->action == DISCONTIG_KERNEL_PAGE_MAP_PAGE)
+ return insert_page(vma, addr, state->__page,
+ vma->vm_page_prot, /*mkwrite=*/false);
+ if (state->action == DISCONTIG_KERNEL_PAGE_MAP_PAGE_RANGE)
+ return insert_pages(vma, addr, state->__page_arr,
+ &nr_pages, vma->vm_page_prot);
+
+ /* Compound folio - have to iterate through each page. */
+ for (i = 0; i < nr_pages; i++, addr += PAGE_SIZE) {
+ struct page *page = folio_page(state->__folio, i);
+ int err;
+
+ err = insert_page(vma, addr, page, vma->vm_page_prot,
+ /*mkwrite=*/false);
+ if (err)
+ return err;
+ }
+ return 0;
+}
+
+int map_discontig_kernel_pages_complete(struct vm_area_struct *vma,
+ struct mmap_action *action)
+{
+ const struct discontig_kernel_page_ops *ops =
+ action->map_kernel_discontig.ops;
+ struct discontig_kernel_page_state state = {
+ .start = vma->vm_start,
+ .end = vma->vm_end,
+ .addr = vma->vm_start,
+ .pgoff = vma->vm_pgoff,
+ .nr_pages_mapped = 0,
+ .nr_pages_remain = vma_pages(vma),
+ .vm_private_data = vma->vm_private_data,
+ .private = action->map_kernel_discontig.init_private,
+ };
+ int err = 0;
+
+ if (ops->init)
+ err = ops->init(vma->vm_private_data, &state.private);
+ if (err)
+ return err;
+
+ do {
+ unsigned long end, pgoff_end;
+ unsigned long nr_pages;
+
+ /* Default to abort. */
+ state.action = DISCONTIG_KERNEL_PAGE_ABORT;
+ err = ops->get(&state);
+ if (err || state.action == DISCONTIG_KERNEL_PAGE_ABORT)
+ return err;
+ nr_pages = state.__nr_pages;
+
+ if (!nr_pages || nr_pages > state.nr_pages_remain)
+ return -EINVAL;
+ end = state.addr + PAGE_SIZE * nr_pages;
+ pgoff_end = state.pgoff + nr_pages;
+
+ err = apply_discontig_action(vma, &state);
+ if (err)
+ return err;
+
+ state.addr = end;
+ state.pgoff = pgoff_end;
+ state.nr_pages_mapped += nr_pages;
+ state.nr_pages_remain -= nr_pages;
+ } while (state.addr < vma->vm_end);
+
+ return 0;
+}
/**
* vm_insert_page - insert single page into user vma
@@ -6837,7 +6933,7 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
lru_gen_enter_fault(vma);
- if (unlikely(is_vm_hugetlb_page(vma)))
+ if (unlikely(vma_is_hugetlb(vma)))
ret = hugetlb_fault(vma->vm_mm, vma, address, flags);
else
ret = __handle_mm_fault(vma, address, flags);
@@ -7020,7 +7116,8 @@ int follow_pfnmap_start(struct follow_pfnmap_args *args)
if (unlikely(address < vma->vm_start || address >= vma->vm_end))
goto out;
- if (!(vma->vm_flags & (VM_IO | VM_PFNMAP)))
+ /* Only mappings GUP cannot handle are followed here. */
+ if (vma_can_gup(vma))
goto out;
retry:
pgdp = pgd_offset(mm, address);
@@ -7220,8 +7317,9 @@ static int __access_remote_vm(struct mm_struct *mm, unsigned long addr,
}
/*
- * Check if this is a VM_IO | VM_PFNMAP VMA, which
- * we can access using slightly different code.
+ * GUP failed, perhaps because this is a mapping it
+ * cannot handle (see vma_can_gup()) - such mappings may
+ * provide access via vm_ops->access() instead.
*/
bytes = 0;
#ifdef CONFIG_HAVE_IOREMAP_PROT
@@ -7707,12 +7805,12 @@ void ptlock_free(struct ptdesc *ptdesc)
void vma_pgtable_walk_begin(struct vm_area_struct *vma)
{
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
hugetlb_vma_lock_read(vma);
}
void vma_pgtable_walk_end(struct vm_area_struct *vma)
{
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
hugetlb_vma_unlock_read(vma);
}
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 8fc8a975657e6..2fd759e348ca1 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -2013,7 +2013,8 @@ SYSCALL_DEFINE5(get_mempolicy, int __user *, policy,
bool vma_migratable(struct vm_area_struct *vma)
{
- if (vma->vm_flags & (VM_IO | VM_PFNMAP))
+ /* Pages which GUP cannot obtain cannot be migrated either. */
+ if (!vma_can_gup(vma))
return false;
/*
@@ -2023,7 +2024,7 @@ bool vma_migratable(struct vm_area_struct *vma)
if (vma_is_dax(vma))
return false;
- if (is_vm_hugetlb_page(vma) &&
+ if (vma_is_hugetlb(vma) &&
!hugepage_migration_supported(hstate_vma(vma)))
return false;
diff --git a/mm/migrate_device.c b/mm/migrate_device.c
index 0c437004329d9..b74c0ae427682 100644
--- a/mm/migrate_device.c
+++ b/mm/migrate_device.c
@@ -739,19 +739,21 @@ static void migrate_vma_unmap(struct migrate_vma *migrate)
*/
int migrate_vma_setup(struct migrate_vma *args)
{
+ const struct vm_area_struct *vma = args->vma;
long nr_pages = (args->end - args->start) >> PAGE_SHIFT;
args->start &= PAGE_MASK;
args->end &= PAGE_MASK;
- if (!args->vma || is_vm_hugetlb_page(args->vma) ||
- (args->vma->vm_flags & VM_SPECIAL) || vma_is_dax(args->vma))
+ if (!vma)
+ return -EINVAL;
+ if (vma_is_kernel_owned(vma) || vma_is_fixed_mapping(vma) ||
+ vma_is_dax(vma))
return -EINVAL;
if (nr_pages <= 0)
return -EINVAL;
- if (args->start < args->vma->vm_start ||
- args->start >= args->vma->vm_end)
+ if (args->start < vma->vm_start || args->start >= vma->vm_end)
return -EINVAL;
- if (args->end <= args->vma->vm_start || args->end > args->vma->vm_end)
+ if (args->end <= vma->vm_start || args->end > vma->vm_end)
return -EINVAL;
if (!args->src || !args->dst)
return -EINVAL;
diff --git a/mm/mlock.c b/mm/mlock.c
index 39215a3eab1fb..4235a1518fc9e 100644
--- a/mm/mlock.c
+++ b/mm/mlock.c
@@ -316,22 +316,10 @@ static inline unsigned int folio_mlock_step(struct folio *folio,
return folio_pte_batch(folio, pte, ptent, count);
}
-static inline bool allow_mlock_munlock(struct folio *folio,
+static inline bool allow_mlock(struct folio *folio,
struct vm_area_struct *vma, unsigned long start,
unsigned long end, unsigned int step)
{
- /*
- * For unlock, allow munlock large folio which is partially
- * mapped to VMA. As it's possible that large folio is
- * mlocked and VMA is split later.
- *
- * During memory pressure, such kind of large folio can
- * be split. And the pages are not in VM_LOCKed VMA
- * can be reclaimed.
- */
- if (!vma_test(vma, VMA_LOCKED_BIT))
- return true;
-
/* folio_within_range() cannot take KSM, but any small folio is OK */
if (!folio_test_large(folio))
return true;
@@ -352,6 +340,7 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr,
{
struct vm_area_struct *vma = walk->vma;
+ const bool lock = walk->private;
spinlock_t *ptl;
pte_t *start_pte, *pte;
pte_t ptent;
@@ -368,7 +357,7 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr,
folio = pmd_folio(*pmd);
if (folio_is_zone_device(folio))
goto out;
- if (vma_test(vma, VMA_LOCKED_BIT))
+ if (lock)
mlock_folio(folio);
else
munlock_folio(folio);
@@ -390,10 +379,10 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr,
continue;
step = folio_mlock_step(folio, pte, addr, end);
- if (!allow_mlock_munlock(folio, vma, start, end, step))
+ if (lock && !allow_mlock(folio, vma, start, end, step))
goto next_entry;
- if (vma_test(vma, VMA_LOCKED_BIT))
+ if (lock)
mlock_folio(folio);
else
munlock_folio(folio);
@@ -428,31 +417,29 @@ static void mlock_vma_pages_range(struct vm_area_struct *vma,
.pmd_entry = mlock_pte_range,
.walk_lock = PGWALK_WRLOCK_VERIFY,
};
+ const bool lock = vma_flags_test(new_vma_flags, VMA_LOCKED_BIT);
+ vma_flags_t walk_flags = *new_vma_flags;
/*
- * There is a slight chance that concurrent page migration,
- * or page reclaim finding a page of this now-VMA_LOCKED_BIT vma,
- * will call mlock_vma_folio() and raise page's mlock_count:
- * double counting, leaving the page unevictable indefinitely.
- * Communicate this danger to mlock_vma_folio() with VMA_IO_BIT,
- * which is a VMA_SPECIAL_FLAGS flag not allowed on VMA_LOCKED_BIT vmas.
- * mmap_lock is held in write mode here, so this weird
- * combination should not be visible to other mmap_lock users;
- * but WRITE_ONCE so rmap walkers must see VMA_IO_BIT if VMA_LOCKED_BIT.
+ * LOCKONFAULT without LOCKED never otherwise occurs: it marks a walk in
+ * progress so that rmap-side callers, which test VMA_LOCKED_BIT, do not
+ * count folios, while try_to_unmap_one(), which tests VMA_LOCKED_MASK,
+ * still refuses to unmap them.
*/
- if (vma_flags_test(new_vma_flags, VMA_LOCKED_BIT))
- vma_flags_set(new_vma_flags, VMA_IO_BIT);
+ if (lock) {
+ vma_flags_clear(&walk_flags, VMA_LOCKED_BIT);
+ vma_flags_set(&walk_flags, VMA_LOCKONFAULT_BIT);
+ }
+
vma_start_write(vma);
- vma_flags_reset_once(vma, new_vma_flags);
+ vma_flags_reset_once(vma, &walk_flags);
lru_add_drain();
- walk_page_range_vma(vma, start, end, &mlock_walk_ops, NULL);
+ walk_page_range_vma(vma, start, end, &mlock_walk_ops, (void *)lock);
lru_add_drain();
- if (vma_flags_test(new_vma_flags, VMA_IO_BIT)) {
- vma_flags_clear(new_vma_flags, VMA_IO_BIT);
+ if (lock)
vma_flags_reset_once(vma, new_vma_flags);
- }
}
/*
diff --git a/mm/mmap.c b/mm/mmap.c
index 4bf26b0f1e6e3..98449f364af1c 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1786,7 +1786,7 @@ __latent_entropy int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)
/*
* Copy/update hugetlb private vma information.
*/
- if (is_vm_hugetlb_page(tmp))
+ if (vma_is_hugetlb(tmp))
hugetlb_dup_vma_private(tmp);
/*
diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c
index 2a72a9686773a..9f353f0e2ef4d 100644
--- a/mm/mmu_gather.c
+++ b/mm/mmu_gather.c
@@ -480,7 +480,7 @@ void tlb_gather_mmu_vma(struct mmu_gather *tlb, struct vm_area_struct *vma)
{
tlb_gather_mmu(tlb, vma->vm_mm);
tlb_update_vma_flags(tlb, vma);
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
/* All entries have the same size. */
tlb_change_page_size(tlb, huge_page_size(hstate_vma(vma)));
}
diff --git a/mm/mprotect.c b/mm/mprotect.c
index 2888ee638d872..a1b6d29bf0390 100644
--- a/mm/mprotect.c
+++ b/mm/mprotect.c
@@ -717,7 +717,7 @@ long change_protection(struct mmu_gather *tlb,
(cp_flags & MM_CP_UFFD_RWP))
newprot = PAGE_NONE;
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
pages = hugetlb_change_protection(vma, start, end, newprot,
cp_flags);
else
@@ -783,8 +783,7 @@ mprotect_fixup(struct vma_iterator *vmi, struct mmu_gather *tlb,
* uncommon case, so doesn't need to be very optimized.
*/
if (arch_has_pfn_modify_check() &&
- vma_flags_test_any(&old_vma_flags, VMA_PFNMAP_BIT,
- VMA_MIXEDMAP_BIT) &&
+ vma_flags_is_kernel_owned(&old_vma_flags) &&
!vma_flags_test_any_mask(&new_vma_flags, VMA_ACCESS_FLAGS)) {
pgprot_t new_pgprot = vm_get_page_prot(newflags);
diff --git a/mm/mremap.c b/mm/mremap.c
index 7c368440fafe2..1122282a1d6ab 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -812,7 +812,7 @@ unsigned long move_page_tables(struct pagetable_move_control *pmc)
if (!pmc->len_in)
return 0;
- if (is_vm_hugetlb_page(pmc->old))
+ if (vma_is_hugetlb(pmc->old))
return move_hugetlb_page_tables(pmc->old, pmc->new, pmc->old_addr,
pmc->new_addr, pmc->len_in);
@@ -1735,7 +1735,7 @@ static bool vma_multi_allowed(struct vm_area_struct *vma)
/* Known good. */
if (vma_is_shmem(vma))
return true;
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
return true;
if (file->f_op->get_unmapped_area == thp_get_unmapped_area)
return true;
@@ -1758,7 +1758,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm)
return -EPERM;
/* Align to hugetlb page size, if required. */
- if (is_vm_hugetlb_page(vma) && !align_hugetlb(vrm))
+ if (vma_is_hugetlb(vma) && !align_hugetlb(vrm))
return -EINVAL;
vrm_set_delta(vrm);
@@ -1788,8 +1788,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm)
return -EINVAL;
}
- if ((vrm->flags & MREMAP_DONTUNMAP) &&
- vma_test_any(vma, VMA_DONTEXPAND_BIT, VMA_PFNMAP_BIT))
+ if ((vrm->flags & MREMAP_DONTUNMAP) && vma_is_fixed_mapping(vma))
return -EINVAL;
/*
@@ -1827,7 +1826,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm)
if (pgoff + (new_len >> PAGE_SHIFT) < pgoff)
return -EINVAL;
- if (vma_test_any(vma, VMA_DONTEXPAND_BIT, VMA_PFNMAP_BIT))
+ if (vma_is_fixed_mapping(vma))
return -EFAULT;
if (!mlock_future_ok(mm, vma_test(vma, VMA_LOCKED_BIT), vrm->delta))
diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
index 28e306fdb3a5b..8408aee7571b5 100644
--- a/mm/page_vma_mapped.c
+++ b/mm/page_vma_mapped.c
@@ -109,7 +109,7 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw, unsigned long pte_nr)
unsigned long pfn;
pte_t ptent;
- if (is_vm_hugetlb_page(pvmw->vma))
+ if (vma_is_hugetlb(pvmw->vma))
ptent = huge_ptep_get(pvmw->vma->vm_mm, pvmw->address,
pvmw->pte);
else
@@ -206,7 +206,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
if (pvmw->pmd && !pvmw->pte)
return not_found(pvmw);
- if (unlikely(is_vm_hugetlb_page(vma))) {
+ if (unlikely(vma_is_hugetlb(vma))) {
struct hstate *hstate = hstate_vma(vma);
unsigned long size = huge_page_size(hstate);
/* The only possible mapping was handled on last iteration */
diff --git a/mm/pagewalk.c b/mm/pagewalk.c
index 7411702a37f58..e6493bbe6919e 100644
--- a/mm/pagewalk.c
+++ b/mm/pagewalk.c
@@ -408,7 +408,7 @@ static int __walk_page_range(unsigned long start, unsigned long end,
int err = 0;
struct vm_area_struct *vma = walk->vma;
const struct mm_walk_ops *ops = walk->ops;
- bool is_hugetlb = is_vm_hugetlb_page(vma);
+ bool is_hugetlb = vma_is_hugetlb(vma);
/* We do not support hugetlb PTE installation. */
if (ops->install_pte && is_hugetlb)
diff --git a/mm/rmap.c b/mm/rmap.c
index 5332c52909be1..6661bc11ce658 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -2239,9 +2239,11 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
/*
* If the folio is in an mlock()d vma, we must not swap it out.
+ * VMA_LOCKONFAULT_BIT alone marks an mlock walk in progress, see
+ * mlock_vma_pages_range().
*/
if (!(flags & TTU_IGNORE_MLOCK) &&
- (vma->vm_flags & VM_LOCKED)) {
+ vma_test_any_mask(vma, VMA_LOCKED_MASK)) {
ptes++;
/*
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 2cd0d0ba966c3..c1c5fbb3c909d 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -2707,7 +2707,7 @@ static int unuse_mm(struct mm_struct *mm, unsigned int type)
if (check_stable_address_space(mm))
goto unlock;
for_each_vma(vmi, vma) {
- if (vma->anon_vma && !is_vm_hugetlb_page(vma)) {
+ if (vma->anon_vma && !vma_is_hugetlb(vma)) {
ret = unuse_vma(vma, type);
if (ret)
break;
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 79cc7b546f130..ddf0a4a3d3997 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -237,7 +237,7 @@ static int mfill_get_vma(struct mfill_state *state)
if ((flags & MFILL_ATOMIC_WP) && !(dst_vma->vm_flags & VM_UFFD_WP))
goto out_unlock;
- if (is_vm_hugetlb_page(dst_vma))
+ if (vma_is_hugetlb(dst_vma))
return 0;
ops = vma_uffd_ops(dst_vma);
@@ -804,7 +804,7 @@ static __always_inline ssize_t mfill_atomic_hugetlb(
}
err = -ENOENT;
- if (!is_vm_hugetlb_page(dst_vma))
+ if (!vma_is_hugetlb(dst_vma))
goto out_unlock_vma;
err = -EINVAL;
@@ -967,7 +967,7 @@ static __always_inline ssize_t mfill_atomic(struct userfaultfd_ctx *ctx,
/*
* If this is a HUGETLB vma, pass off to appropriate routine
*/
- if (is_vm_hugetlb_page(state.vma))
+ if (vma_is_hugetlb(state.vma))
return mfill_atomic_hugetlb(ctx, state.vma, dst_start,
src_start, len, flags);
@@ -1114,7 +1114,7 @@ static int mwriteprotect_range(struct userfaultfd_ctx *ctx, unsigned long start,
break;
}
- if (is_vm_hugetlb_page(dst_vma)) {
+ if (vma_is_hugetlb(dst_vma)) {
err = -EINVAL;
page_mask = vma_kernel_pagesize(dst_vma) - 1;
if ((start & page_mask) || (len & page_mask))
@@ -1172,7 +1172,7 @@ int mrwprotect_range(struct userfaultfd_ctx *ctx, unsigned long start,
if (!userfaultfd_rwp(dst_vma))
return -ENOENT;
- if (is_vm_hugetlb_page(dst_vma)) {
+ if (vma_is_hugetlb(dst_vma)) {
unsigned long page_mask;
page_mask = vma_kernel_pagesize(dst_vma) - 1;
@@ -1754,10 +1754,18 @@ static inline bool move_splits_huge_pmd(unsigned long dst_addr,
}
#endif
-static inline bool vma_move_compatible(struct vm_area_struct *vma)
+static inline bool vma_move_compatible(const struct vm_area_struct *vma)
{
- return !(vma->vm_flags & (VM_PFNMAP | VM_IO | VM_HUGETLB |
- VM_MIXEDMAP | VM_SHADOW_STACK));
+ /* uffd is generally incompatible with kernel-owned mappings. */
+ if (vma_is_kernel_owned(vma))
+ return false;
+ /* The shadow stack should not be written to by userspace. */
+ if (vma_test_single_mask(vma, VMA_SHADOW_STACK))
+ return false;
+ /* hugetlb mappings cannot be safely moved. */
+ if (vma_is_hugetlb(vma))
+ return false;
+ return true;
}
static int validate_move_areas(struct userfaultfd_ctx *ctx,
@@ -2146,10 +2154,11 @@ static bool vma_can_userfault(struct vm_area_struct *vma, vm_flags_t vm_flags,
{
const struct vm_uffd_ops *ops = vma_uffd_ops(vma);
- if (vma->vm_flags & (VM_DROPPABLE | VM_SHADOW_STACK))
+ /* Non-persistent memory is inherently not controllable by userspace. */
+ if (!vma_is_persistent(vma))
return false;
-
- if (!is_vm_hugetlb_page(vma) && (vma->vm_flags & VM_SPECIAL))
+ /* The shadow stack should not be written to by userspace. */
+ if (vma_test_single_mask(vma, VMA_SHADOW_STACK))
return false;
vm_flags &= __VM_UFFD_FLAGS;
@@ -2319,7 +2328,7 @@ static int userfaultfd_register_range(struct userfaultfd_ctx *ctx,
*/
userfaultfd_set_ctx(vma, ctx, vm_flags);
- if (is_vm_hugetlb_page(vma) && uffd_disable_huge_pmd_share(vma))
+ if (vma_is_hugetlb(vma) && uffd_disable_huge_pmd_share(vma))
hugetlb_unshare_all_pmds(vma);
skip:
@@ -2895,7 +2904,7 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason)
* (sleepable) vma lock can modify the current task state, that
* must be before explicitly calling set_current_state().
*/
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
hugetlb_vma_lock_read(vma);
spin_lock_irq(&ctx->fault_pending_wqh.lock);
@@ -2912,7 +2921,7 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason)
set_current_state(blocking_state);
spin_unlock_irq(&ctx->fault_pending_wqh.lock);
- if (is_vm_hugetlb_page(vma)) {
+ if (vma_is_hugetlb(vma)) {
must_wait = userfaultfd_huge_must_wait(ctx, vmf, reason);
hugetlb_vma_unlock_read(vma);
} else {
@@ -3744,7 +3753,7 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,
* If the first vma contains huge pages, make sure start address
* is aligned to huge page size.
*/
- if (is_vm_hugetlb_page(vma)) {
+ if (vma_is_hugetlb(vma)) {
unsigned long vma_hpagesize = vma_kernel_pagesize(vma);
if (start & (vma_hpagesize - 1))
@@ -3795,7 +3804,7 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,
* If this vma contains ending address, and huge pages
* check alignment.
*/
- if (is_vm_hugetlb_page(cur) && end <= cur->vm_end &&
+ if (vma_is_hugetlb(cur) && end <= cur->vm_end &&
end > cur->vm_start) {
unsigned long vma_hpagesize = vma_kernel_pagesize(cur);
@@ -3831,7 +3840,7 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,
/*
* Note vmas containing huge pages
*/
- if (is_vm_hugetlb_page(cur))
+ if (vma_is_hugetlb(cur))
basic_ioctls = true;
found = true;
@@ -3917,7 +3926,7 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx,
* If the first vma contains huge pages, make sure start address
* is aligned to huge page size.
*/
- if (is_vm_hugetlb_page(vma)) {
+ if (vma_is_hugetlb(vma)) {
unsigned long vma_hpagesize = vma_kernel_pagesize(vma);
if (start & (vma_hpagesize - 1))
diff --git a/mm/util.c b/mm/util.c
index bf0513d1d3d08..c5ee52aede1e4 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -1224,16 +1224,28 @@ EXPORT_SYMBOL(compat_set_desc_from_vma);
int __compat_vma_mmap(struct vm_area_desc *desc,
struct vm_area_struct *vma)
{
+ struct vm_area_desc prev_desc;
int err;
+ /* Derive state prior to mmap_prepare hook. */
+ compat_set_desc_from_vma(&prev_desc, desc->file, vma);
/* Perform any preparatory tasks for mmap action. */
err = mmap_action_prepare(desc);
if (err)
- return err;
+ goto err_put;
+ /* Check the caller did nothing crazy. */
+ err = mmap_prepare_validate(&prev_desc, desc);
+ if (err)
+ goto err_put;
/* Update the VMA from the descriptor. */
compat_set_vma_from_desc(vma, desc);
/* Complete any specified mmap actions. */
return mmap_action_complete(vma, &desc->action, /*is_compat=*/true);
+
+err_put:
+ if (desc->vm_file != vma->vm_file)
+ fput(desc->vm_file);
+ return err;
}
EXPORT_SYMBOL(__compat_vma_mmap);
@@ -1455,8 +1467,10 @@ int mmap_action_prepare(struct vm_area_desc *desc)
return io_remap_pfn_range_prepare(desc);
case MMAP_SIMPLE_IO_REMAP:
return simple_ioremap_prepare(desc);
- case MMAP_MAP_KERNEL_PAGES:
+ case MMAP_KERNEL_PAGES:
return map_kernel_pages_prepare(desc);
+ case MMAP_DISCONTIG_KERNEL_PAGES:
+ return map_discontig_kernel_pages_prepare(desc);
}
WARN_ON_ONCE(1);
@@ -1486,9 +1500,12 @@ int mmap_action_complete(struct vm_area_struct *vma,
case MMAP_REMAP_PFN:
err = remap_pfn_range_complete(vma, action);
break;
- case MMAP_MAP_KERNEL_PAGES:
+ case MMAP_KERNEL_PAGES:
err = map_kernel_pages_complete(vma, action);
break;
+ case MMAP_DISCONTIG_KERNEL_PAGES:
+ err = map_discontig_kernel_pages_complete(vma, action);
+ break;
case MMAP_IO_REMAP_PFN:
case MMAP_SIMPLE_IO_REMAP:
/* Should have been delegated. */
@@ -1509,7 +1526,8 @@ int mmap_action_prepare(struct vm_area_desc *desc)
case MMAP_REMAP_PFN:
case MMAP_IO_REMAP_PFN:
case MMAP_SIMPLE_IO_REMAP:
- case MMAP_MAP_KERNEL_PAGES:
+ case MMAP_KERNEL_PAGES:
+ case MMAP_DISCONTIG_KERNEL_PAGES:
WARN_ON_ONCE(1); /* nommu cannot handle these. */
break;
}
@@ -1530,7 +1548,8 @@ int mmap_action_complete(struct vm_area_struct *vma,
case MMAP_REMAP_PFN:
case MMAP_IO_REMAP_PFN:
case MMAP_SIMPLE_IO_REMAP:
- case MMAP_MAP_KERNEL_PAGES:
+ case MMAP_KERNEL_PAGES:
+ case MMAP_DISCONTIG_KERNEL_PAGES:
WARN_ON_ONCE(1); /* nommu cannot handle this. */
err = -EINVAL;
diff --git a/mm/vma.c b/mm/vma.c
index 55917d0979339..777656306705d 100644
--- a/mm/vma.c
+++ b/mm/vma.c
@@ -24,7 +24,8 @@ struct mmap_state {
vm_flags_t vm_flags;
vma_flags_t vma_flags;
};
- struct file *file;
+ struct file *file; /* mmap()-specified file. */
+ struct file *vm_file; /* May be updated by mmap_prepare. */
pgprot_t page_prot;
/* User-defined fields, perhaps updated by .mmap_prepare(). */
@@ -43,8 +44,6 @@ struct mmap_state {
/* Determine if we can check KSM flags early in mmap() logic. */
bool check_ksm_early :1;
- /* If .mmap_prepare changed the file, we don't need to pin. */
- bool file_doesnt_need_get :1;
};
#define MMAP_STATE(name, mm_, vmi_, addr_, len_, pgoff_, anon_pgoff_, vma_flags_, file_) \
@@ -58,6 +57,7 @@ struct mmap_state {
.pglen = PHYS_PFN(len_), \
.vma_flags = vma_flags_, \
.file = file_, \
+ .vm_file = file_, \
.page_prot = vma_flags_to_page_prot(vma_flags_), \
}
@@ -70,7 +70,7 @@ struct mmap_state {
.vma_flags = (map_)->vma_flags, \
.pgoff = (map_)->pgoff, \
.anon_pgoff = (map_)->anon_pgoff, \
- .file = (map_)->file, \
+ .file = (map_)->vm_file, \
.prev = (map_)->prev, \
.middle = vma_, \
.next = (vma_) ? NULL : (map_)->next, \
@@ -599,7 +599,7 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
* boundary.
*/
vma_adjust_trans_huge(vma, vma->vm_start, addr, NULL);
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
hugetlb_split(vma, addr);
if (new_below) {
@@ -924,13 +924,14 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(
vmg->state = VMA_MERGE_NOMERGE;
+ if (!vma_flags_can_merge(&vmg->vma_flags))
+ return NULL;
/*
- * If a special mapping or if the range being modified is neither at the
- * furthermost left or right side of the VMA, then we have no chance of
- * merging and should abort.
+ * If the range being modified is neither at the furthermost left or
+ * right side of the VMA, then we have no chance of merging and should
+ * abort.
*/
- if (vma_flags_test_any_mask(&vmg->vma_flags, VMA_SPECIAL_FLAGS) ||
- (!left_side && !right_side))
+ if (!left_side && !right_side)
return NULL;
if (left_side)
@@ -1152,9 +1153,11 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
vmg->state = VMA_MERGE_NOMERGE;
- /* Special VMAs are unmergeable, also if no prev/next. */
- if (vma_flags_test_any_mask(&vmg->vma_flags, VMA_SPECIAL_FLAGS) ||
- (!prev && !next))
+ if (!vma_flags_can_merge(&vmg->vma_flags))
+ return NULL;
+
+ /* VMAs with no prev/next are unmergeable. */
+ if (!prev && !next)
return NULL;
can_merge_left = can_vma_merge_left(vmg);
@@ -2233,7 +2236,7 @@ bool vma_wants_writenotify(struct vm_area_struct *vma, pgprot_t vm_page_prot)
* Do we need to track softdirty? hugetlb does not support softdirty
* tracking yet.
*/
- if (vma_soft_dirty_enabled(vma) && !is_vm_hugetlb_page(vma))
+ if (vma_soft_dirty_enabled(vma) && !vma_is_hugetlb(vma))
return true;
/* Do we need write faults for uffd-wp tracking? */
@@ -2352,7 +2355,7 @@ int mm_take_all_locks(struct mm_struct *mm)
if (signal_pending(current))
goto out_unlock;
if (vma->vm_file && vma->vm_file->f_mapping &&
- is_vm_hugetlb_page(vma))
+ vma_is_hugetlb(vma))
vm_lock_mapping(mm, vma->vm_file->f_mapping);
}
@@ -2361,7 +2364,7 @@ int mm_take_all_locks(struct mm_struct *mm)
if (signal_pending(current))
goto out_unlock;
if (vma->vm_file && vma->vm_file->f_mapping &&
- !is_vm_hugetlb_page(vma))
+ !vma_is_hugetlb(vma))
vm_lock_mapping(mm, vma->vm_file->f_mapping);
}
@@ -2447,7 +2450,7 @@ void mm_drop_all_locks(struct mm_struct *mm)
*/
static bool accountable_mapping(struct mmap_state *map)
{
- const struct file *file = map->file;
+ const struct file *file = map->vm_file;
/*
* hugetlb has its own accounting separate from the core VM
@@ -2496,7 +2499,7 @@ static void vms_abort_munmap_vmas(struct vma_munmap_struct *vms,
static void update_ksm_flags(struct mmap_state *map)
{
- map->vma_flags = ksm_vma_flags(map->mm, map->file, map->vma_flags);
+ map->vma_flags = ksm_vma_flags(map->mm, map->vm_file, map->vma_flags);
}
static void set_desc_from_map(struct vm_area_desc *desc,
@@ -2506,7 +2509,7 @@ static void set_desc_from_map(struct vm_area_desc *desc,
desc->end = map->end;
desc->pgoff = map->pgoff;
- desc->vm_file = map->file;
+ desc->vm_file = map->vm_file;
desc->vma_flags = map->vma_flags;
desc->page_prot = map->page_prot;
}
@@ -2586,6 +2589,10 @@ static int __mmap_setup(struct mmap_state *map, struct vm_area_desc *desc,
return 0;
}
+static bool map_same_file(struct mmap_state *map)
+{
+ return map->vm_file == map->file;
+}
static int __mmap_new_file_vma(struct mmap_state *map,
struct vm_area_struct *vma)
@@ -2593,37 +2600,43 @@ static int __mmap_new_file_vma(struct mmap_state *map,
struct vma_iterator *vmi = map->vmi;
int error;
- vma->vm_file = map->file;
- if (!map->file_doesnt_need_get)
- get_file(map->file);
+ vma->vm_file = map->vm_file;
+ if (map_same_file(map))
+ get_file(map->vm_file);
- if (!map->file->f_op->mmap)
+ if (!map->vm_file->f_op->mmap)
return 0;
+ /*
+ * Driver-specified flags may make the lock flags invalid, so clear
+ * VMA_LOCKED_MASK and reinstate it afterwards if appropriate.
+ */
+ vma_clear_flags_mask(vma, VMA_LOCKED_MASK);
error = mmap_file(vma->vm_file, vma);
+ map->vm_file = vma->vm_file;
+
if (error) {
UNMAP_STATE(unmap, vmi, vma, vma->vm_start, vma->vm_end,
map->prev, map->next);
- fput(vma->vm_file);
- vma->vm_file = NULL;
+ if (map_same_file(map))
+ fput(map->vm_file);
+ vma->vm_file = NULL;
vma_iter_set(vmi, vma->vm_end);
/* Undo any partial mapping done by a device driver. */
unmap_region(&unmap);
return error;
}
- /* Drivers cannot alter the address of the VMA. */
- WARN_ON_ONCE(map->addr != vma->vm_start);
- /*
- * Drivers should not permit writability when previously it was
- * disallowed.
- */
- VM_WARN_ON_ONCE(!vma_flags_same_pair(&map->vma_flags, &vma->flags) &&
- !vma_flags_test(&map->vma_flags, VMA_MAYWRITE_BIT) &&
- vma_test(vma, VMA_MAYWRITE_BIT));
+ /* If VMA flags still valid for locked mask, reinstate. */
+ if (vma_supports_mlock(vma)) {
+ const vma_flags_t mask =
+ vma_flags_and_mask(&map->vma_flags,
+ VMA_LOCKED_MASK);
+
+ vma_set_flags_mask(vma, mask);
+ }
- map->file = vma->vm_file;
map->vma_flags = vma->flags;
return 0;
@@ -2631,7 +2644,7 @@ static int __mmap_new_file_vma(struct mmap_state *map,
static void map_set_anon(struct mmap_state *map)
{
- map->file = NULL;
+ map->vm_file = NULL;
map->vm_ops = NULL;
map->pgoff = map->addr >> PAGE_SHIFT;
}
@@ -2643,7 +2656,7 @@ static bool map_is_private(const struct mmap_state *map)
static bool map_is_anon(const struct mmap_state *map)
{
- return map_is_private(map) && !map->file;
+ return map_is_private(map) && !map->vm_file;
}
/*
@@ -2688,7 +2701,7 @@ static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap,
}
/* Invoke callbacks. */
- if (map->file)
+ if (map->vm_file)
error = __mmap_new_file_vma(map, vma);
else if (!is_anon)
error = shmem_zero_setup(vma);
@@ -2701,11 +2714,6 @@ static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap,
vma->flags = map->vma_flags;
}
-#ifdef CONFIG_SPARC64
- /* TODO: Fix SPARC ADI! */
- WARN_ON_ONCE(!arch_validate_flags(map->vm_flags));
-#endif
-
/* Lock the VMA since it is modified after insertion into VMA tree */
vma_start_write(vma);
vma_iter_store_new(vmi, vma);
@@ -2768,6 +2776,96 @@ static void __mmap_complete(struct mmap_state *map, struct vm_area_struct *vma)
vma_set_page_prot(vma);
}
+/* Check to ensure that the VMA flags of a newly mapped VMA are sane. */
+static int mmap_validate_vma_flags(const vma_flags_t *flags)
+{
+#ifdef CONFIG_SPARC64
+ const vm_flags_t legacy_flags = vma_flags_to_legacy(*flags);
+
+ /* TODO: Fix SPARC ADI! */
+ if (WARN_ON_ONCE(!arch_validate_flags(legacy_flags)))
+ return -EINVAL;
+#endif
+
+ if (!vma_flags_is_kernel_owned(flags)) {
+ /* Only kernel-owned mappings may set VMA_IO_BIT. */
+ if (WARN_ON_ONCE(vma_flags_test(flags, VMA_IO_BIT)))
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+/* Check to ensure a driver hasn't done something crazy. */
+static int mmap_validate(unsigned long prev_start,
+ unsigned long curr_start,
+ const vma_flags_t *prev_flags,
+ const vma_flags_t *curr_flags)
+{
+ bool was_maywrite, is_maywrite;
+
+ /* Drivers cannot alter the address of the VMA. */
+ if (WARN_ON_ONCE(prev_start != curr_start))
+ return -EINVAL;
+
+ was_maywrite = vma_flags_test(prev_flags, VMA_MAYWRITE_BIT);
+ is_maywrite = vma_flags_test(curr_flags, VMA_MAYWRITE_BIT);
+
+ /* A driver may not make a previously unwritable mapping writable. */
+ if (WARN_ON_ONCE(!was_maywrite && is_maywrite))
+ return -EINVAL;
+
+ /* Only kernel-owned mappings may clear VMA_MAYWRITE_BIT. */
+ if (!vma_flags_is_kernel_owned(curr_flags) &&
+ WARN_ON_ONCE(was_maywrite && !is_maywrite))
+ return -EINVAL;
+
+ return mmap_validate_vma_flags(curr_flags);
+}
+
+/**
+ * mmap_prepare_validate() - Ensure the driver hasn't violated invariants in its
+ * f_op->mmap_prepare hook.
+ * @prev_desc: The VMA descriptor prior to the mmap_prepare hook being called.
+ * @desc: The VMA descriptor after the mmap_prepare hook has been called.
+ *
+ * Returns: 0 on success, otherwise an error.
+ */
+int mmap_prepare_validate(const struct vm_area_desc *prev_desc,
+ const struct vm_area_desc *desc)
+{
+ /*
+ * It is not valid to execute mmap actions for VMAs which can be merged,
+ * as any such merge would leave portions of the mapping incorrectly
+ * unmapped.
+ */
+ if (vma_flags_can_merge(&desc->vma_flags) &&
+ WARN_ON_ONCE(desc->action.type != MMAP_NOTHING))
+ return -EINVAL;
+
+ return mmap_validate(prev_desc->start, desc->start,
+ &prev_desc->vma_flags, &desc->vma_flags);
+}
+
+/**
+ * mmap_hook_validate() - Ensure the driver hasn't violated invariants in
+ * its f_op->mmap hook.
+ * @prev_start: The start of the mapping prior to the mmap hook.
+ * @prev_flags: The VMA flags set for the VMA prior to the mmap hook.
+ * @vma: The VMA after the hook has been applied.
+ *
+ * Returns: 0 on success, otherwise an error.
+ */
+int mmap_hook_validate(unsigned long prev_start,
+ const vma_flags_t *prev_flags,
+ const struct vm_area_struct *vma)
+{
+ const unsigned long start = vma->vm_start;
+ const vma_flags_t *flags = &vma->flags;
+
+ return mmap_validate(prev_start, start, prev_flags, flags);
+}
+
static int call_action_prepare(struct mmap_state *map,
struct vm_area_desc *desc)
{
@@ -2794,39 +2892,43 @@ static int call_action_prepare(struct mmap_state *map,
static int call_mmap_prepare(struct mmap_state *map,
struct vm_area_desc *desc)
{
+ const struct vm_area_desc prev_desc = *desc;
int err;
/* Invoke the hook. */
- err = vfs_mmap_prepare(map->file, desc);
- if (err)
- return err;
-
- /* It's invalid for mmap_preprare hooks to clear vm_ops. */
- if (!desc->vm_ops)
- return -EINVAL;
-
- err = call_action_prepare(map, desc);
+ err = vfs_mmap_prepare(map->vm_file, desc);
if (err)
return err;
/* Update fields permitted to be changed. */
map->pgoff = desc->pgoff;
- if (desc->vm_file != map->file) {
- map->file_doesnt_need_get = true;
- map->file = desc->vm_file;
- }
+ if (desc->vm_file != map->vm_file)
+ map->vm_file = desc->vm_file;
map->vma_flags = desc->vma_flags;
map->page_prot = desc->page_prot;
/* User-defined fields. */
map->vm_ops = desc->vm_ops;
map->vm_private_data = desc->private_data;
+ /* It's invalid for mmap_prepare hooks to clear vm_ops. */
+ if (!desc->vm_ops)
+ return -EINVAL;
+
+ err = call_action_prepare(map, desc);
+ if (err)
+ return err;
+
+ /* Check the caller did nothing crazy. */
+ err = mmap_prepare_validate(&prev_desc, desc);
+ if (err)
+ return err;
+
/*
* MAP_PRIVATE-/dev/zero mappings are an ancient way of getting
* anonymous mappings. Rather than allowing these mappings to be odd
* outliers, simply make them truly anonymous.
*/
- if (map_is_private(map) && file_is_dev_zero(map->file))
+ if (map_is_private(map) && file_is_dev_zero(map->vm_file))
map_set_anon(map);
return 0;
@@ -2845,7 +2947,7 @@ static void set_vma_user_defined_fields(struct vm_area_struct *vma,
*/
static bool can_set_ksm_flags_early(struct mmap_state *map)
{
- struct file *file = map->file;
+ struct file *file = map->vm_file;
/* Anonymous mappings have no driver which can change them. */
if (!file)
@@ -2868,13 +2970,27 @@ static bool can_set_ksm_flags_early(struct mmap_state *map)
return false;
}
+static void put_map(struct mmap_state *map)
+{
+ /*
+ * An error occurred or the VMA was merged.
+ *
+ * If the file was changed by the driver (which is required to increment
+ * the replacement file's reference count), drop its reference count.
+ *
+ * On error, the caller always drops the original file regardless.
+ */
+ if (map->vm_file && !map_same_file(map))
+ fput(map->vm_file);
+}
+
static unsigned long __mmap_region(struct file *file, unsigned long addr,
unsigned long len, vma_flags_t vma_flags,
unsigned long pgoff, struct list_head *uf)
{
struct mm_struct *mm = current->mm;
struct vm_area_struct *vma = NULL;
- bool have_mmap_prepare = file && file->f_op->mmap_prepare;
+ const bool have_mmap_prepare = file && file->f_op->mmap_prepare;
VMA_ITERATOR(vmi, mm, addr);
const pgoff_t anon_pgoff = addr >> PAGE_SHIFT;
MMAP_STATE(map, mm, &vmi, addr, len, pgoff, anon_pgoff, vma_flags, file);
@@ -2917,12 +3033,15 @@ static unsigned long __mmap_region(struct file *file, unsigned long addr,
allocated_new = true;
}
- if (have_mmap_prepare && !map_is_anon(&map))
+ if (have_mmap_prepare && allocated_new && !map_is_anon(&map))
set_vma_user_defined_fields(vma, &map);
__mmap_complete(&map, vma);
- if (have_mmap_prepare && allocated_new) {
+ if (!allocated_new) {
+ /* Merged, so need to drop refcount. */
+ put_map(&map);
+ } else if (have_mmap_prepare) {
error = mmap_action_complete(vma, &desc.action,
/*is_compat=*/false);
if (error)
@@ -2936,13 +3055,7 @@ static unsigned long __mmap_region(struct file *file, unsigned long addr,
if (map.charged)
vm_unacct_memory(map.charged);
abort_munmap:
- /*
- * This indicates that .mmap_prepare has set a new file, differing from
- * desc->vm_file. But since we're aborting the operation, only the
- * original file will be cleaned up. Ensure we clean up both.
- */
- if (map.file_doesnt_need_get)
- fput(map.file);
+ put_map(&map);
vms_abort_munmap_vmas(&map.vms, &map.mas_detach);
return error;
}
@@ -3437,10 +3550,15 @@ int __vm_munmap(unsigned long start, size_t len, bool unlock)
int insert_vm_struct(struct mm_struct *mm, struct vm_area_struct *vma)
{
unsigned long charged = vma_pages(vma);
+ int err;
if (find_vma_intersection(mm, vma->vm_start, vma->vm_end))
return -ENOMEM;
+ err = mmap_validate_vma_flags(&vma->flags);
+ if (err)
+ return err;
+
if (vma_test(vma, VMA_ACCOUNT_BIT) &&
security_vm_enough_memory_mm(mm, charged))
return -ENOMEM;
diff --git a/mm/vma.h b/mm/vma.h
index e97bd2dfa786d..77d395b8b1032 100644
--- a/mm/vma.h
+++ b/mm/vma.h
@@ -394,8 +394,10 @@ static inline void compat_set_vma_from_desc(struct vm_area_struct *vma,
/* Mutable fields. Populated with initial state. */
vma_set_pgoff(vma, desc->pgoff);
- if (desc->vm_file != vma->vm_file)
- vma_set_file(vma, desc->vm_file);
+ if (desc->vm_file != vma->vm_file) {
+ fput(vma->vm_file);
+ vma->vm_file = desc->vm_file;
+ }
vma->flags = desc->vma_flags;
vma->vm_page_prot = desc->page_prot;
@@ -780,14 +782,19 @@ struct vm_area_struct *vm_area_alloc(struct mm_struct *mm);
struct vm_area_struct *vm_area_dup(struct vm_area_struct *orig);
void vm_area_free(struct vm_area_struct *vma);
-/* vma_exec.c */
#ifdef CONFIG_MMU
+int mmap_prepare_validate(const struct vm_area_desc *prev_desc,
+ const struct vm_area_desc *desc);
+
+int mmap_hook_validate(unsigned long prev_start,
+ const vma_flags_t *prev_flags,
+ const struct vm_area_struct *vma);
+
+/* vma_exec.c */
int create_init_stack_vma(struct mm_struct *mm, struct vm_area_struct **vmap,
unsigned long *top_mem_p);
int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift);
-#endif
-#ifdef CONFIG_MMU
/*
* Denies creating a writable executable mapping or gaining executable permissions.
*
@@ -836,6 +843,19 @@ static inline bool map_deny_write_exec(const vma_flags_t *old,
return false;
}
+#else
+static inline int mmap_prepare_validate(const struct vm_area_desc *prev_desc,
+ const struct vm_area_desc *desc)
+{
+ return 0;
+}
+
+static inline int mmap_hook_validate(unsigned long prev_start,
+ const vma_flags_t *prev_flags,
+ const struct vm_area_struct *vma)
+{
+ return 0;
+}
#endif
struct vm_area_struct *__install_special_mapping(struct mm_struct *mm,
diff --git a/mm/vma_internal.h b/mm/vma_internal.h
index 4d300e7bbaf4c..4f73f0a4db796 100644
--- a/mm/vma_internal.h
+++ b/mm/vma_internal.h
@@ -18,7 +18,6 @@
#include <linux/fs.h>
#include <linux/huge_mm.h>
#include <linux/hugetlb.h>
-#include <linux/hugetlb_inline.h>
#include <linux/kernel.h>
#include <linux/ksm.h>
#include <linux/khugepaged.h>
diff --git a/mm/vmscan.c b/mm/vmscan.c
index aaceed4759eeb..001f8b760266b 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3471,13 +3471,14 @@ static int should_skip_vma(unsigned long start, unsigned long end, struct mm_wal
if (!vma_is_accessible(vma))
return true;
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
return true;
if (!vma_has_recency(vma))
return true;
- if (vma->vm_flags & (VM_LOCKED | VM_SPECIAL))
+ if (vma_test(vma, VMA_LOCKED_BIT) || vma_is_kernel_owned(vma) ||
+ vma_is_fixed_mapping(vma))
return true;
if (vma == get_gate_vma(vma->vm_mm))
@@ -4417,8 +4418,8 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr)
if (spin_is_contended(pvmw->ptl))
return true;
- /* exclude special VMAs containing anon pages from COW */
- if (vma->vm_flags & VM_SPECIAL)
+ /* exclude kernel-owned and fixed VMAs containing anon pages from COW */
+ if (vma_is_kernel_owned(vma) || vma_is_fixed_mapping(vma))
return true;
/* avoid taking the LRU lock under the PTL when possible */
diff --git a/security/selinux/selinuxfs.c b/security/selinux/selinuxfs.c
index c7d91476971cb..545a6f89f9e76 100644
--- a/security/selinux/selinuxfs.c
+++ b/security/selinux/selinuxfs.c
@@ -340,6 +340,9 @@ static int sel_open_policy(struct inode *inode, struct file *filp)
struct policy_load_memory *plm = NULL;
int rc;
+ if (filp->f_mode & FMODE_WRITE)
+ return -EACCES;
+
rc = avc_has_perm(current_sid(), SECINITSID_SECURITY,
SECCLASS_SECURITY, SECURITY__READ_POLICY, NULL);
if (rc)
@@ -424,14 +427,6 @@ static const struct vm_operations_struct sel_mmap_policy_ops = {
static int sel_mmap_policy(struct file *filp, struct vm_area_struct *vma)
{
- if (vma->vm_flags & VM_SHARED) {
- /* do not allow mprotect to make mapping writable */
- vm_flags_clear(vma, VM_MAYWRITE);
-
- if (vma->vm_flags & VM_WRITE)
- return -EACCES;
- }
-
vm_flags_set(vma, VM_DONTEXPAND | VM_DONTDUMP);
vma->vm_ops = &sel_mmap_policy_ops;
diff --git a/sound/core/pcm_native.c b/sound/core/pcm_native.c
index 62324282fcae9..c98fb3df14f34 100644
--- a/sound/core/pcm_native.c
+++ b/sound/core/pcm_native.c
@@ -3760,39 +3760,27 @@ static __poll_t snd_pcm_poll(struct file *file, poll_table *wait)
/*
* mmap status record
*/
-static vm_fault_t snd_pcm_mmap_status_fault(struct vm_fault *vmf)
+static int snd_pcm_mmap_status(struct snd_pcm_substream *substream, struct file *file,
+ struct vm_area_struct *vma)
{
- struct snd_pcm_substream *substream = vmf->vma->vm_private_data;
+ const unsigned long size = vma->vm_end - vma->vm_start;
struct snd_pcm_runtime *runtime;
-
- if (substream == NULL)
- return VM_FAULT_SIGBUS;
- runtime = substream->runtime;
- vmf->page = virt_to_page(runtime->status);
- get_page(vmf->page);
- return 0;
-}
+ struct page *page;
-static const struct vm_operations_struct snd_pcm_vm_ops_status =
-{
- .fault = snd_pcm_mmap_status_fault,
-};
+ BUILD_BUG_ON(sizeof(struct snd_pcm_mmap_status) > PAGE_SIZE);
-static int snd_pcm_mmap_status(struct snd_pcm_substream *substream, struct file *file,
- struct vm_area_struct *area)
-{
- long size;
- if (!(area->vm_flags & VM_READ))
+ if (!(vma->vm_flags & VM_READ))
return -EINVAL;
- size = area->vm_end - area->vm_start;
- if (size != PAGE_ALIGN(sizeof(struct snd_pcm_mmap_status)))
+ if (size != PAGE_SIZE)
return -EINVAL;
- area->vm_ops = &snd_pcm_vm_ops_status;
- area->vm_private_data = substream;
- vm_flags_mod(area, VM_DONTEXPAND | VM_DONTDUMP,
+
+ vm_flags_mod(vma, VM_DONTEXPAND | VM_DONTDUMP,
VM_WRITE | VM_MAYWRITE);
+ vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
- return 0;
+ runtime = substream->runtime;
+ page = virt_to_page(runtime->status);
+ return vm_insert_page(vma, vma->vm_start, page);
}
/*
diff --git a/tools/testing/vma/include/dup.h b/tools/testing/vma/include/dup.h
index 16c09dac59d9b..dc24f43a9394c 100644
--- a/tools/testing/vma/include/dup.h
+++ b/tools/testing/vma/include/dup.h
@@ -352,14 +352,6 @@ enum {
#define VM_ACCESS_FLAGS (VM_READ | VM_WRITE | VM_EXEC)
#define VMA_ACCESS_FLAGS mk_vma_flags(VMA_READ_BIT, VMA_WRITE_BIT, VMA_EXEC_BIT)
-/*
- * Special vmas that are non-mergable, non-mlock()able.
- */
-#define VM_SPECIAL (VM_IO | VM_DONTEXPAND | VM_PFNMAP | VM_MIXEDMAP)
-
-#define VMA_SPECIAL_FLAGS mk_vma_flags(VMA_IO_BIT, VMA_DONTEXPAND_BIT, \
- VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT)
-
#define VMA_REMAP_FLAGS mk_vma_flags(VMA_IO_BIT, VMA_PFNMAP_BIT, \
VMA_DONTEXPAND_BIT, VMA_DONTDUMP_BIT)
@@ -454,17 +446,20 @@ static __always_inline bool vma_flags_empty(const vma_flags_t *flags)
/* What action should be taken after an .mmap_prepare call is complete? */
enum mmap_action_type {
- MMAP_NOTHING, /* Mapping is complete, no further action. */
- MMAP_REMAP_PFN, /* Remap PFN range. */
- MMAP_IO_REMAP_PFN, /* I/O remap PFN range. */
- MMAP_SIMPLE_IO_REMAP, /* I/O remap with guardrails. */
- MMAP_MAP_KERNEL_PAGES, /* Map kernel page range from an array. */
+ MMAP_NOTHING,
+ MMAP_REMAP_PFN,
+ MMAP_IO_REMAP_PFN,
+ MMAP_SIMPLE_IO_REMAP, /* I/O remap with guardrails. */
+ MMAP_KERNEL_PAGES, /* Map kernel page range from array. */
+ MMAP_DISCONTIG_KERNEL_PAGES, /* Map kernel discontig page range. */
};
/*
* Describes an action an mmap_prepare hook can instruct to be taken to complete
* the mapping of a VMA. Specified in vm_area_desc.
*/
+struct discontig_kernel_page_ops;
+
struct mmap_action {
union {
struct {
@@ -483,6 +478,10 @@ struct mmap_action {
unsigned long nr_pages;
pgoff_t pgoff;
} map_kernel;
+ struct {
+ void *init_private;
+ const struct discontig_kernel_page_ops *ops;
+ } map_kernel_discontig;
};
enum mmap_action_type type;
@@ -1359,13 +1358,23 @@ static inline int vfs_mmap_prepare(struct file *file, struct vm_area_desc *desc)
return file->f_op->mmap_prepare(desc);
}
+int mmap_prepare_validate(const struct vm_area_desc *prev_desc,
+ const struct vm_area_desc *desc);
+
static inline int __compat_vma_mmap(struct vm_area_desc *desc,
struct vm_area_struct *vma)
{
+ struct vm_area_desc prev_desc;
int err;
+ /* Derive state prior to mmap_prepare hook. */
+ compat_set_desc_from_vma(&prev_desc, desc->file, vma);
/* Perform any preparatory tasks for mmap action. */
err = mmap_action_prepare(desc);
+ if (err)
+ return err;
+ /* Check the caller did nothing crazy. */
+ err = mmap_prepare_validate(&prev_desc, desc);
if (err)
return err;
/* Update the VMA from the descriptor. */
@@ -1647,3 +1656,34 @@ static inline bool file_is_dev_zero(const struct file *file)
{
return file && file->f_op == &zero_fops;
}
+
+static inline bool vma_flags_is_kernel_owned(const vma_flags_t *flags)
+{
+ return vma_flags_test_any(flags, VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT);
+}
+
+static inline bool vma_is_kernel_owned(const struct vm_area_struct *vma)
+{
+ return vma_flags_is_kernel_owned(&vma->flags);
+}
+
+static inline bool vma_flags_can_merge(const vma_flags_t *flags)
+{
+ /*
+ * VMA merging assumes that the properties of a VMA completely describe
+ * the properties of that VMA.
+ *
+ * However, kernel-owned mappings may have established state upon mapping
+ * not embodied in any attribute of the VMA.
+ *
+ * Additionally, PFN maps encode the source PFN of the range in
+ * vma->vm_pgoff, which may otherwise cause spurious merges.
+ */
+ if (vma_flags_is_kernel_owned(flags))
+ return false;
+ /* VMA explicitly marked as being unmergeable. */
+ if (vma_flags_test(flags, VMA_DONTEXPAND_BIT))
+ return false;
+
+ return true;
+}
diff --git a/tools/testing/vma/include/stubs.h b/tools/testing/vma/include/stubs.h
index d6136e19a8af3..48d1dc53df42c 100644
--- a/tools/testing/vma/include/stubs.h
+++ b/tools/testing/vma/include/stubs.h
@@ -193,7 +193,7 @@ static inline bool mapping_can_writeback(struct address_space *mapping)
return true;
}
-static inline bool is_vm_hugetlb_page(struct vm_area_struct *vma)
+static inline bool vma_is_hugetlb(struct vm_area_struct *vma)
{
return false;
}
diff --git a/tools/testing/vma/tests/merge.c b/tools/testing/vma/tests/merge.c
index acaab282939c0..b26f1a66a1707 100644
--- a/tools/testing/vma/tests/merge.c
+++ b/tools/testing/vma/tests/merge.c
@@ -496,17 +496,11 @@ static bool test_vma_merge_special_flags(void)
.mm = &mm,
.vmi = &vmi,
};
- vma_flag_t special_flags[] = { VMA_IO_BIT, VMA_DONTEXPAND_BIT,
+ vma_flag_t special_flags[] = { VMA_DONTEXPAND_BIT,
VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT };
- vma_flags_t all_special_flags = EMPTY_VMA_FLAGS;
int i;
struct vm_area_struct *vma_left, *vma;
- /* Make sure there aren't new VM_SPECIAL flags. */
- for (i = 0; i < ARRAY_SIZE(special_flags); i++)
- vma_flags_set(&all_special_flags, special_flags[i]);
- ASSERT_FLAGS_SAME_MASK(&all_special_flags, VMA_SPECIAL_FLAGS);
-
/*
* 01234
* AAA
@@ -520,7 +514,7 @@ static bool test_vma_merge_special_flags(void)
* 01234
* AAA*
*
- * This should merge if not for the VM_SPECIAL flag.
+ * This should merge if not for the 'special' flag.
*/
vmg_set_range(&vmg, 0x3000, 0x4000, 3, vma_flags);
for (i = 0; i < ARRAY_SIZE(special_flags); i++) {
|
| 3/2 | 2026/09/14 19:11 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 54979 output: 33 thoughts: 2021 Thoughts: **Fuzzing Target Assessment: A Deep Dive** Okay, let's break this down. My initial assessment is: "WorthFuzzing = true." This patch screams fuzzing target. It's touching core memory management (mm/), which immediately flags it as high-value, considering the impact surface. Plus, it appears to hit multiple drivers too! First, the breadth of changes is significant. We're talking about a comprehensive modification of the memory management subsystem. Key subsystems affected, so definitely in a reachable spot. The patch modifies critical core subsystems. Specifically: mm, drivers (USB, SCSI, sound, BPF, uprobes, and more). Second, the "WHEN TO RETURN WorthFuzzing=true" criteria are clearly met. We see major changes to memory management, which includes new validation functions, discontiguous kernel page mapping, and substantial changes to `mlock`. We have new validation functions (`mmap_prepare_validate`, `mmap_hook_validate`, `mmap_validate_vma_flags`). Discontiguous kernel page mapping (`map_discontig_kernel_pages_prepare`, `map_discontig_kernel_pages_complete`), and changes to drivers that use them (like USB/SCSI) are worth looking at. Assertions and defensive checks are being introduced or modified. All of that is a gold mine for fuzzing. Third, identifying *FocusSymbols* is the next critical step. I need functions that are specific and non-ubiquitous to narrow the fuzzing scope. I've compiled a list of modified or added functions: Functions like `mmap_prepare_validate`, `mmap_hook_validate`, `map_discontig_kernel_pages_prepare`, and `map_discontig_kernel_pages_complete` in the mm/ subsystem jump out as good starting points. We need to go verify visibility of those functions. Furthermore, functions like `sg_mmap_prepare` and `mon_bin_mmap_prepare` within their respective driver contexts are good too. Next, I need to look into how these symbols are used to create the right test configs, to maximize our fuzzing efficiency. I need to make sure the fuzzing is efficient, and doesn't get stuck doing something that won't surface any issues. |
| 4/2 | 2026/09/14 19:11 | tool | grepper |
0mArgs:{
"Expression": "^int map_discontig_kernel_pages",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/internal.h=1510=int map_kernel_pages_complete(struct vm_area_struct *vma,\nmm/internal.h-1511-\t\t\t struct mmap_action *action);\nmm/internal.h:1512:int map_discontig_kernel_pages_prepare(struct vm_area_desc *desc);\nmm/internal.h:1513:int map_discontig_kernel_pages_complete(struct vm_area_struct *vma,\nmm/internal.h-1514-\t\t\t\t\tstruct mmap_action *action);\n--\nmm/memory.c=2638=int map_kernel_pages_complete(struct vm_area_struct *vma,\n--\nmm/memory.c-2648-\nmm/memory.c:2649:int map_discontig_kernel_pages_prepare(struct vm_area_desc *desc)\nmm/memory.c-2650-{\n--\nmm/memory.c=2663=static int apply_discontig_action(struct vm_area_struct *vma,\n--\nmm/memory.c-2689-\nmm/memory.c:2690:int map_discontig_kernel_pages_complete(struct vm_area_struct *vma,\nmm/memory.c-2691-\t\t\t\t\tstruct mmap_action *action)\n"
} |
| 5/2 | 2026/09/14 19:11 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 57874 output: 29 thoughts: 70 Thoughts: Here's my summary, as though I'm thinking through this process: **Dissecting Kernel Memory Mapping** Okay, so I'm digging into how the kernel handles mapping of discontiguous kernel pages. Specifically, I'm noting that `map_discontig_kernel_pages_prepare` and `map_discontig_kernel_pages_complete` are right there in `mm/memory.c`. That's where I'd expect to find the core logic for this kind of allocation and mapping. Now, let's back up a bit and consider the validation. I'll need to double check the mmap validation routines. I'm going to shift focus and look at `mm/vma.c` - I want to see how this relates to the virtual memory area (VMA) structures. I suspect these two functions will be critical for understanding mmap in the context of kernel memory. I'll focus on `mmap_prepare_validate` and `mmap_hook_validate`. They will give me critical info on how mmap is used in kernel memory management, and how the kernel validates mmaps and related hooks. |
| 6/2 | 2026/09/14 19:11 | tool | grepper |
0mArgs:{
"Expression": "^int mmap_.*validate",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/vma.c=2800=static int mmap_validate(unsigned long prev_start,\n--\nmm/vma.c-2833- */\nmm/vma.c:2834:int mmap_prepare_validate(const struct vm_area_desc *prev_desc,\nmm/vma.c-2835-\t\t\t const struct vm_area_desc *desc)\n--\nmm/vma.c-2858- */\nmm/vma.c:2859:int mmap_hook_validate(unsigned long prev_start,\nmm/vma.c-2860-\t\t const vma_flags_t *prev_flags,\n--\nmm/vma.h=783=void vm_area_free(struct vm_area_struct *vma);\n--\nmm/vma.h-785-#ifdef CONFIG_MMU\nmm/vma.h:786:int mmap_prepare_validate(const struct vm_area_desc *prev_desc,\nmm/vma.h-787-\t\t\t const struct vm_area_desc *desc);\nmm/vma.h-788-\nmm/vma.h:789:int mmap_hook_validate(unsigned long prev_start,\nmm/vma.h-790-\t\t const vma_flags_t *prev_flags,\n"
} |
| 7/2 | 2026/09/14 19:11 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 58571 output: 29 thoughts: 61 Thoughts: Here's my take on the thought process, as if it were my own: **Investigating Memory Mapping Functionality** Okay, so I'm digging into memory mapping, specifically, how it's prepared and completed. I've noted that `mmap_prepare_validate` and `mmap_hook_validate` – those aren't static functions! They're hanging out there in `mm/vma.c`, just begging to be understood better. Now, to expand my understanding, I need to look at what sets them up and what the general workflow around memory mapping looks like, so I need to check out `mmap_action_prepare` and `mmap_action_complete`. They're probably coordinating the preparation and completion, respectively, so I'm heading over to `mm/util.c` to see how those are implemented and how they interact. |
| 8/2 | 2026/09/14 19:11 | tool | grepper |
0mArgs:{
"Expression": "^int mmap_action_",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/util.c=1435=static int check_mmap_action(struct mmap_action *action)\n--\nmm/util.c-1451- */\nmm/util.c:1452:int mmap_action_prepare(struct vm_area_desc *desc)\nmm/util.c-1453-{\n--\nmm/util.c=1479=EXPORT_SYMBOL(mmap_action_prepare);\n--\nmm/util.c-1491- */\nmm/util.c:1492:int mmap_action_complete(struct vm_area_struct *vma,\nmm/util.c-1493-\t\t\t struct mmap_action *action, bool is_compat)\n--\nmm/util.c=1519=EXPORT_SYMBOL(mmap_action_complete);\nmm/util.c-1520-#else\nmm/util.c:1521:int mmap_action_prepare(struct vm_area_desc *desc)\nmm/util.c-1522-{\n--\nmm/util.c=1537=EXPORT_SYMBOL(mmap_action_prepare);\nmm/util.c-1538-\nmm/util.c:1539:int mmap_action_complete(struct vm_area_struct *vma,\nmm/util.c-1540-\t\t\t struct mmap_action *action,\n"
} |
| 9/2 | 2026/09/14 19:11 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 59209 output: 31 thoughts: 43 Thoughts: Okay, let's dive into this. I need to make sure this is all correct. **Checking the Driver Implementation: `sg_mmap_prepare`** Right, so I need to check the driver code specifically. I'm going to focus in on `sg_mmap_prepare`. Let me quickly look at `drivers/scsi/sg.c`, specifically around lines 1250-1300. I need to get a better handle on this. I need to see what's happening there to make sure everything lines up with what I'd expect given the context. |
| 10/2 | 2026/09/14 19:11 | tool | grepper |
0mArgs:{
"Expression": "sg_mmap_prepare",
"PathPrefix": "drivers/scsi/"
}Results: {
"Output": "drivers/scsi/sg.c=1259=static int\ndrivers/scsi/sg.c:1260:sg_mmap_prepare(struct vm_area_desc *desc)\ndrivers/scsi/sg.c-1261-{\n--\ndrivers/scsi/sg.c=1397=static const struct file_operations sg_fops = {\n--\ndrivers/scsi/sg.c-1404-\t.open = sg_open,\ndrivers/scsi/sg.c:1405:\t.mmap_prepare = sg_mmap_prepare,\ndrivers/scsi/sg.c-1406-\t.release = sg_release,\n"
} |
| 11/2 | 2026/09/14 19:11 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 59593 output: 32 thoughts: 65 Thoughts: Okay, here's my take on those thoughts, framed as my own reasoning: **Analyzing Potential MMAP Implementation Issues** Right, so I'm thinking about memory mapping, specifically, I'm eyeing potential areas of concern in a couple of places. I've got my sights set on `drivers/usb/mon/mon_bin.c`, specifically the `mon_bin_mmap_prepare` function. That seems like a prime spot to double-check, given its relationship with USB monitoring and how it handles memory allocation for binary data. I need to make sure the MMAP implementation is sound there, particularly with USB data transfer rates in mind. Are we handling potential race conditions? Is the buffer alignment correct, given how USB hardware might interface? Memory-mapping errors can be especially tricky to debug in the kernel, so this is critical. Also, I've got my eye on `sound/core/pcm_native.c`, specifically `snd_pcm_mmap_status`. MMAP status updates can be a point of contention and a source of subtle issues within the audio subsystem. I really need to know what code paths call `snd_pcm_mmap_status` and the context to understand the exact use case. Is it called in an interrupt handler? What locks are involved? Are we efficiently synchronizing access to the shared memory? I'm going to follow the call chain to understand the bigger picture of where and how we're updating status via shared memory. I'm imagining scenarios involving buffer underruns, interrupts, and multi-threaded access. Let's see how those status updates are handled in reality. |
| 12/2 | 2026/09/14 19:11 | tool | grepper |
0mArgs:{
"Expression": "snd_pcm_mmap_status",
"PathPrefix": "sound/core/"
}Results: {
"Output": "sound/core/pcm.c=873=int snd_pcm_attach_substream(struct snd_pcm *pcm, int stream,\n--\nsound/core/pcm.c-939-\nsound/core/pcm.c:940:\tsize = PAGE_ALIGN(sizeof(struct snd_pcm_mmap_status));\nsound/core/pcm.c-941-\truntime-\u003estatus = alloc_pages_exact(size, GFP_KERNEL);\n--\nsound/core/pcm.c-951-\t\tfree_pages_exact(runtime-\u003estatus,\nsound/core/pcm.c:952:\t\t\t PAGE_ALIGN(sizeof(struct snd_pcm_mmap_status)));\nsound/core/pcm.c-953-\t\tkfree(runtime);\n--\nsound/core/pcm.c=978=void snd_pcm_detach_substream(struct snd_pcm_substream *substream)\n--\nsound/core/pcm.c-987-\tfree_pages_exact(runtime-\u003estatus,\nsound/core/pcm.c:988:\t\t PAGE_ALIGN(sizeof(struct snd_pcm_mmap_status)));\nsound/core/pcm.c-989-\tfree_pages_exact(runtime-\u003econtrol,\n--\nsound/core/pcm_compat.c=329=static int snd_pcm_ioctl_xfern_compat(struct snd_pcm_substream *substream,\n--\nsound/core/pcm_compat.c-375-/* X32 ABI has 64bit timespec and 64bit alignment */\nsound/core/pcm_compat.c:376:struct snd_pcm_mmap_status_x32 {\nsound/core/pcm_compat.c-377-\tsnd_pcm_state_t state;\n--\nsound/core/pcm_compat.c=392=struct snd_pcm_sync_ptr_x32 {\n--\nsound/core/pcm_compat.c-395-\tunion {\nsound/core/pcm_compat.c:396:\t\tstruct snd_pcm_mmap_status_x32 status;\nsound/core/pcm_compat.c-397-\t\tunsigned char reserved[64];\n--\nsound/core/pcm_compat.c=405=static int snd_pcm_ioctl_sync_ptr_x32(struct snd_pcm_substream *substream,\n--\nsound/core/pcm_compat.c-408-\tstruct snd_pcm_runtime *runtime = substream-\u003eruntime;\nsound/core/pcm_compat.c:409:\tvolatile struct snd_pcm_mmap_status *status;\nsound/core/pcm_compat.c-410-\tvolatile struct snd_pcm_mmap_control *control;\n--\nsound/core/pcm_compat.c-412-\tstruct snd_pcm_mmap_control scontrol;\nsound/core/pcm_compat.c:413:\tstruct snd_pcm_mmap_status sstatus;\nsound/core/pcm_compat.c-414-\tsnd_pcm_uframes_t boundary;\n--\nsound/core/pcm_compat.c=481=static int snd_pcm_ioctl_sync_ptr_buggy(struct snd_pcm_substream *substream,\n--\nsound/core/pcm_compat.c-486-\tstruct __snd_pcm_mmap_control64_buggy *sync_cp;\nsound/core/pcm_compat.c:487:\tvolatile struct snd_pcm_mmap_status *status;\nsound/core/pcm_compat.c-488-\tvolatile struct snd_pcm_mmap_control *control;\n--\nsound/core/pcm_native.c=3161=static int snd_pcm_sync_ptr(struct snd_pcm_substream *substream,\n--\nsound/core/pcm_native.c-3164-\tstruct snd_pcm_runtime *runtime = substream-\u003eruntime;\nsound/core/pcm_native.c:3165:\tvolatile struct snd_pcm_mmap_status *status;\nsound/core/pcm_native.c-3166-\tvolatile struct snd_pcm_mmap_control *control;\n--\nsound/core/pcm_native.c-3168-\tstruct snd_pcm_mmap_control scontrol;\nsound/core/pcm_native.c:3169:\tstruct snd_pcm_mmap_status sstatus;\nsound/core/pcm_native.c-3170-\tint err;\n--\nsound/core/pcm_native.c-3205-\nsound/core/pcm_native.c:3206:struct snd_pcm_mmap_status32 {\nsound/core/pcm_native.c-3207-\tsnd_pcm_state_t state;\n--\nsound/core/pcm_native.c=3220=struct snd_pcm_sync_ptr32 {\n--\nsound/core/pcm_native.c-3222-\tunion {\nsound/core/pcm_native.c:3223:\t\tstruct snd_pcm_mmap_status32 status;\nsound/core/pcm_native.c-3224-\t\tunsigned char reserved[64];\n--\nsound/core/pcm_native.c=3255=static int snd_pcm_ioctl_sync_ptr_compat(struct snd_pcm_substream *substream,\n--\nsound/core/pcm_native.c-3258-\tstruct snd_pcm_runtime *runtime = substream-\u003eruntime;\nsound/core/pcm_native.c:3259:\tvolatile struct snd_pcm_mmap_status *status;\nsound/core/pcm_native.c-3260-\tvolatile struct snd_pcm_mmap_control *control;\n--\nsound/core/pcm_native.c-3262-\tstruct snd_pcm_mmap_control scontrol;\nsound/core/pcm_native.c:3263:\tstruct snd_pcm_mmap_status sstatus;\nsound/core/pcm_native.c-3264-\tsnd_pcm_uframes_t boundary;\n--\nsound/core/pcm_native.c=3703=static __poll_t snd_pcm_poll(struct file *file, poll_table *wait)\n--\nsound/core/pcm_native.c-3762- */\nsound/core/pcm_native.c:3763:static int snd_pcm_mmap_status(struct snd_pcm_substream *substream, struct file *file,\nsound/core/pcm_native.c-3764-\t\t\t struct vm_area_struct *vma)\n--\nsound/core/pcm_native.c-3769-\nsound/core/pcm_native.c:3770:\tBUILD_BUG_ON(sizeof(struct snd_pcm_mmap_status) \u003e PAGE_SIZE);\nsound/core/pcm_native.c-3771-\n--\nsound/core/pcm_native.c=3840=static bool pcm_control_mmap_allowed(struct snd_pcm_file *pcm_file)\n--\nsound/core/pcm_native.c-3862-\nsound/core/pcm_native.c:3863:static int snd_pcm_mmap_status(struct snd_pcm_substream *substream, struct file *file,\nsound/core/pcm_native.c-3864-\t\t\t struct vm_area_struct *area)\n--\nsound/core/pcm_native.c=4056=static int snd_pcm_mmap(struct file *file, struct vm_area_struct *area)\n--\nsound/core/pcm_native.c-4077-\t\t\treturn -ENXIO;\nsound/core/pcm_native.c:4078:\t\treturn snd_pcm_mmap_status(substream, file, area);\nsound/core/pcm_native.c-4079-\tcase SNDRV_PCM_MMAP_OFFSET_CONTROL_OLD:\n"
} |
| 13/2 | 2026/09/14 19:11 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 62016 output: 31 thoughts: 93 Thoughts: Here's my thought process, as an expert in this area: **Analyzing the `snd_pcm_mmap_status` Modification** Okay, so they've switched `snd_pcm_mmap_status` to directly map using `vm_insert_page(vma, vma->vm_start, page)`. That's interesting. I need to understand the implications of this change. It sounds like they're trying to optimize the memory locking and mapping process. Direct page insertion often implies bypassing some of the higher-level VM abstractions, likely for performance. I need to delve deeper into the locking and security implications here. My next immediate concern is around memory locking, so let's cross-reference the change with `mlock_vma_pages_range`. I need to figure out how this impacts the virtual memory area (VMA) locking. I'm going to quickly check in `mm/mlock.c`. The question is, is `mlock_vma_pages_range` static to the file or exported? Its visibility is crucial. If it's static, it likely only impacts other functions within the file, while exported functions would have broader implications. This is important to determine how the direct mapping interacts with existing memory protection mechanisms. This will provide some clues for the overall impact of this change, and I'll see if it has some side-effects. |
| 14/2 | 2026/09/14 19:11 | tool | grepper |
0mArgs:{
"Expression": "mlock_vma_pages_range",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/internal.h=977=static inline void munlock_vma_folio(struct folio *folio,\n--\nmm/internal.h-990-\t * VMA_LOCKONFAULT_BIT alone marks an mlock walk in progress, see\nmm/internal.h:991:\t * mlock_vma_pages_range(). An unmap racing with the walk must still\nmm/internal.h-992-\t * munlock folios the walk has already counted.\n--\nmm/mlock.c=338=static int mlock_pte_range(pmd_t *pmd, unsigned long addr,\n--\nmm/mlock.c-401-/*\nmm/mlock.c:402: * mlock_vma_pages_range() - mlock any pages already in the range,\nmm/mlock.c-403- * or munlock all pages in the range.\n--\nmm/mlock.c-411- */\nmm/mlock.c:412:static void mlock_vma_pages_range(struct vm_area_struct *vma,\nmm/mlock.c-413-\tunsigned long start, unsigned long end,\n--\nmm/mlock.c=454=static int mlock_fixup(struct vma_iterator *vmi, struct vm_area_struct *vma,\n--\nmm/mlock.c-498-\t} else {\nmm/mlock.c:499:\t\tmlock_vma_pages_range(vma, start, end, new_vma_flags);\nmm/mlock.c-500-\t}\n--\nmm/rmap.c=2203=static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,\n--\nmm/rmap.c-2242-\t\t * VMA_LOCKONFAULT_BIT alone marks an mlock walk in progress, see\nmm/rmap.c:2243:\t\t * mlock_vma_pages_range().\nmm/rmap.c-2244-\t\t */\n"
} |
| 15/2 | 2026/09/14 19:11 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 62960 output: 27 thoughts: 49 Thoughts: Here's my thought process, as if I were analyzing this code: **Analyzing the Memory Locking Functions** Okay, so `mlock_vma_pages_range` is a static function, which means its scope is limited to the file it's defined in. That's a good starting point for understanding how it's used. Since `mlock_vma_pages_range` is called by `mlock_fixup`, my next question is whether `mlock_fixup` itself is also static. Knowing its scope will help me trace the call path more effectively. Let me quickly check the definition of `mlock_fixup`. Then, I'll need to understand `apply_mlock`, as it's likely involved in the memory locking process and, based on the naming, perhaps where the actual locking operation happens. Let's delve in. |
| 16/2 | 2026/09/14 19:11 | tool | grepper |
0mArgs:{
"Expression": "mlock_fixup",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/mlock.c=412=static void mlock_vma_pages_range(struct vm_area_struct *vma,\n--\nmm/mlock.c-445-/*\nmm/mlock.c:446: * mlock_fixup - handle mlock[all]/munlock[all] requests.\nmm/mlock.c-447- *\n--\nmm/mlock.c-453- */\nmm/mlock.c:454:static int mlock_fixup(struct vma_iterator *vmi, struct vm_area_struct *vma,\nmm/mlock.c-455-\t struct vm_area_struct **prev, unsigned long start,\n--\nmm/mlock.c=506=static int apply_vma_lock_flags(unsigned long start, size_t len,\n--\nmm/mlock.c-544-\t\t\ttmp = end;\nmm/mlock.c:545:\t\terror = mlock_fixup(\u0026vmi, vma, \u0026prev, nstart, tmp, \u0026newflags);\nmm/mlock.c-546-\t\tif (error)\n--\nmm/mlock.c=703=static int apply_mlockall_flags(int flags)\n--\nmm/mlock.c-733-\nmm/mlock.c:734:\t\terror = mlock_fixup(\u0026vmi, vma, \u0026prev, vma-\u003evm_start, vma-\u003evm_end,\nmm/mlock.c-735-\t\t\t\t \u0026newflags);\n"
} |
| 17/2 | 2026/09/14 19:11 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 63622 output: 30 thoughts: 48 Thoughts: Okay, let's dive into this. I need to understand the entry points or non-static functions within `mm/mlock.c`. I'll start by taking a look at `do_mlock`. Then, I'll need to figure out what functions are calling `apply_vma_lock_flags` because that feels like a crucial function to follow for understanding the locking mechanisms in place. That will give me a good lay of the land to then fully evaluate the file. |
| 18/2 | 2026/09/14 19:11 | tool | grepper |
0mArgs:{
"Expression": "apply_vma_lock_flags",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/mlock.c=454=static int mlock_fixup(struct vma_iterator *vmi, struct vm_area_struct *vma,\n--\nmm/mlock.c-505-\nmm/mlock.c:506:static int apply_vma_lock_flags(unsigned long start, size_t len,\nmm/mlock.c-507-\t\t\t\tconst vma_flags_t *flags)\n--\nmm/mlock.c=606=static __must_check int do_mlock(unsigned long start, size_t len,\n--\nmm/mlock.c-641-\tif ((locked \u003c= lock_limit) || capable(CAP_IPC_LOCK))\nmm/mlock.c:642:\t\terror = apply_vma_lock_flags(start, len, flags);\nmm/mlock.c-643-\n--\nmm/mlock.c=674=SYSCALL_DEFINE2(munlock, unsigned long, start, size_t, len)\n--\nmm/mlock.c-685-\t\treturn -EINTR;\nmm/mlock.c:686:\tret = apply_vma_lock_flags(start, len, \u0026flags);\nmm/mlock.c-687-\tmmap_write_unlock(current-\u003emm);\n"
} |
| 19/2 | 2026/09/14 19:11 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 64141 output: 30 thoughts: 121 Thoughts: **Analyzing Potential Issues in Memory Management** Okay, let's break this down. First, I'm taking a look at the `mlock_pte_range` function – that's static, which could have some implications. Now, what about `vma_merge_new_range`? I need to check its scope to see if it's the right function to be looking at. I'll dive into the source code, specifically `mm/vma.c`, and see how `vma_merge_new_range` is defined. Alright, I see the declaration: `struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)`. Okay, that's what I expected. Now the question is, is it static? Let's quickly verify that by checking the header file `mm/vma.h`. Okay, perfect, it's not. I've got my answer. Now, let's see what comes next. |
| 20/2 | 2026/09/14 19:11 | tool | grepper |
0mArgs:{
"Expression": "vma_merge_new_range",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/vma.c=889=static __must_check struct vm_area_struct *vma_merge_existing_range(\n--\nmm/vma.c-1097-/*\nmm/vma.c:1098: * vma_merge_new_range - Attempt to merge a new VMA into address space\nmm/vma.c-1099- *\n--\nmm/vma.c-1140- */\nmm/vma.c:1141:struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)\nmm/vma.c-1142-{\n--\nmm/vma.c-1220- *\nmm/vma.c:1221: * ASSUMPTIONS: Same as vma_merge_new_range(), except vmg-\u003emiddle must contain\nmm/vma.c-1222- * the copied-from VMA.\n--\nmm/vma.c=1224=static struct vm_area_struct *vma_merge_copied_range(struct vma_merge_struct *vmg)\n--\nmm/vma.c-1230-\tvmg-\u003emiddle = NULL;\nmm/vma.c:1231:\treturn vma_merge_new_range(vmg);\nmm/vma.c-1232-}\n--\nmm/vma.c=1860=struct vm_area_struct *vma_merge_extend(struct vma_iterator *vmi,\n--\nmm/vma.c-1868-\nmm/vma.c:1869:\treturn vma_merge_new_range(\u0026vmg);\nmm/vma.c-1870-}\n--\nmm/vma.c=2672=static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap,\n--\nmm/vma.c-2723-\t/*\nmm/vma.c:2724:\t * vma_merge_new_range() calls khugepaged_enter_vma() too, the below\nmm/vma.c-2725-\t * call covers the non-merge case.\n--\nmm/vma.c=2987=static unsigned long __mmap_region(struct file *file, unsigned long addr,\n--\nmm/vma.c-3024-\nmm/vma.c:3025:\t\tvma = vma_merge_new_range(\u0026vmg);\nmm/vma.c-3026-\t}\n--\nmm/vma.c=3136=int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *vma,\n--\nmm/vma.c-3170-\nmm/vma.c:3171:\t\tif (vma_merge_new_range(\u0026vmg))\nmm/vma.c-3172-\t\t\tgoto out;\n--\nmm/vma.h=518=__must_check struct vm_area_struct *vma_modify_flags_uffd(struct vma_iterator *vmi,\n--\nmm/vma.h-522-\nmm/vma.h:523:__must_check struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg);\nmm/vma.h-524-\n"
} |
| 21/2 | 2026/09/14 19:11 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 65391 output: 27 thoughts: 1371 Thoughts: **Analyzing the Patch for Fuzzing Strategy** Okay, so I'm looking at this patch series, and the question is, how do we effectively fuzz this, specifically considering syzkaller's coverage? First off, `vma_merge_new_range` is definitely ubiquitous and probably best to ignore for the purposes of focusing the fuzzer. The guidelines are clear about that. Right, let's break down the actual changes. The patch series seems to be a significant refactoring of mmap handling. It's replacing a lot of ad-hoc checks around `VM_SPECIAL` flags with semantic helper functions like `vma_is_hugetlb()`, `vma_is_kernel_owned()`, and so on. That's a good move for maintainability, and it provides a clear set of primitives. There's also a new mechanism, `.mmap_prepare`, and actions using that - `mmap_action_map_discontig_kernel_pages()` with implementations like `map_discontig_kernel_pages_prepare`, `map_discontig_kernel_pages_complete` and helpers. This seems like the key area for focused fuzzing. Additionally, `mmap_prepare_validate` and `mmap_hook_validate` are added. It's converting drivers to use this new `.mmap_prepare` path – `sg.c`, `mon_bin.c`, `cmt_speech.c`, `pcm_native.c`, and `file_ops.c`. These driver conversions are clearly important targets. The patch also refactors `mlock_vma_pages_range` with `VMA_LOCKONFAULT_BIT` for mlock walks, which is a significant change to how memory locking works. Finally, it adds validations in `insert_vm_struct()` and the validation hooks. Now, for fuzzing, I want to focus on the key, non-ubiquitous entry points. Based on the patch, the primary new or modified functions are: 1. `map_discontig_kernel_pages_prepare` 2. `map_discontig_kernel_pages_complete` 3. `mmap_prepare_validate` 4. `mmap_hook_validate` 5. `sg_mmap_prepare` 6. `mon_bin_mmap_prepare` 7. `snd_pcm_mmap_status` 8. `insert_vm_struct` 9. `fb_deferred_io_mmap` 10. `__compat_vma_mmap` I need to determine if these functions are static or exported, and whether they have symbols in `kallsyms`. The important thing here is that syzkaller *can* trace static functions if they appear in `kallsyms`, and functions in `.c` files in function pointer structs (like these new `mmap_prepare` callbacks) *do* have distinct addresses, which is great. The instruction explicitly warns about inlined static functions in headers, but not those defined in `.c` files. The non-static functions are even better targets, as those are already in the list. Now, let's consider `mmap_action_complete` and `mmap_action_prepare`. These aren’t ubiquitous, which is perfect! They are specific to the mmap preparation and completion actions, and that is what we are targeting here. They are only called by drivers providing `mmap_prepare`, and other `compat mmap` actions, so we've found our entry points. |
| 22/2 | 2026/09/14 19:12 | tool | grepper |
0mArgs:{
"Expression": "mmap_action_",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/char-mem.c=327=static int mmap_mem_prepare(struct vm_area_desc *desc)\n--\nmm/char-mem.c-360-\t/* Remap-pfn-range will mark the range with the I/O flag. */\nmm/char-mem.c:361:\tmmap_action_remap_full(desc, desc-\u003epgoff);\nmm/char-mem.c-362-\tdesc-\u003eaction.error_override = -EAGAIN;\n--\nmm/memory.c=3450=int simple_ioremap_prepare(struct vm_area_desc *desc)\n--\nmm/memory.c-3463-\t/* The I/O remap logic does the heavy lifting. */\nmm/memory.c:3464:\tmmap_action_ioremap_full(desc, pfn);\nmm/memory.c-3465-\treturn io_remap_pfn_range_prepare(desc);\n--\nmm/util.c=1224=int __compat_vma_mmap(struct vm_area_desc *desc,\n--\nmm/util.c-1232-\t/* Perform any preparatory tasks for mmap action. */\nmm/util.c:1233:\terr = mmap_action_prepare(desc);\nmm/util.c-1234-\tif (err)\n--\nmm/util.c-1242-\t/* Complete any specified mmap actions. */\nmm/util.c:1243:\treturn mmap_action_complete(vma, \u0026desc-\u003eaction, /*is_compat=*/true);\nmm/util.c-1244-\n--\nmm/util.c=1385=static int call_vma_mapped(struct vm_area_struct *vma)\n--\nmm/util.c-1403-\nmm/util.c:1404:static int mmap_action_finish(struct vm_area_struct *vma,\nmm/util.c-1405-\t\t\t struct mmap_action *action, int err,\n--\nmm/util.c=1435=static int check_mmap_action(struct mmap_action *action)\n--\nmm/util.c-1445-/**\nmm/util.c:1446: * mmap_action_prepare - Perform preparatory setup for an VMA descriptor\nmm/util.c-1447- * action which need to be performed.\n--\nmm/util.c-1451- */\nmm/util.c:1452:int mmap_action_prepare(struct vm_area_desc *desc)\nmm/util.c-1453-{\n--\nmm/util.c-1478-}\nmm/util.c:1479:EXPORT_SYMBOL(mmap_action_prepare);\nmm/util.c-1480-\nmm/util.c-1481-/**\nmm/util.c:1482: * mmap_action_complete - Execute VMA descriptor action.\nmm/util.c-1483- * @vma: The VMA to perform the action upon.\n--\nmm/util.c-1486- *\nmm/util.c:1487: * Similar to mmap_action_prepare().\nmm/util.c-1488- *\n--\nmm/util.c-1491- */\nmm/util.c:1492:int mmap_action_complete(struct vm_area_struct *vma,\nmm/util.c-1493-\t\t\t struct mmap_action *action, bool is_compat)\n--\nmm/util.c-1516-\nmm/util.c:1517:\treturn mmap_action_finish(vma, action, err, is_compat);\nmm/util.c-1518-}\nmm/util.c:1519:EXPORT_SYMBOL(mmap_action_complete);\nmm/util.c-1520-#else\nmm/util.c:1521:int mmap_action_prepare(struct vm_area_desc *desc)\nmm/util.c-1522-{\n--\nmm/util.c-1536-}\nmm/util.c:1537:EXPORT_SYMBOL(mmap_action_prepare);\nmm/util.c-1538-\nmm/util.c:1539:int mmap_action_complete(struct vm_area_struct *vma,\nmm/util.c-1540-\t\t\t struct mmap_action *action,\n--\nmm/util.c-1558-\nmm/util.c:1559:\treturn mmap_action_finish(vma, action, err, is_compat);\nmm/util.c-1560-}\nmm/util.c:1561:EXPORT_SYMBOL(mmap_action_complete);\nmm/util.c-1562-#endif\n--\nmm/vma.c=2869=static int call_action_prepare(struct mmap_state *map,\n--\nmm/vma.c-2873-\nmm/vma.c:2874:\terr = mmap_action_prepare(desc);\nmm/vma.c-2875-\tif (err)\n--\nmm/vma.c=2987=static unsigned long __mmap_region(struct file *file, unsigned long addr,\n--\nmm/vma.c-3044-\t} else if (have_mmap_prepare) {\nmm/vma.c:3045:\t\terror = mmap_action_complete(vma, \u0026desc.action,\nmm/vma.c-3046-\t\t\t\t\t /*is_compat=*/false);\n"
} |
| 23/2 | 2026/09/14 19:12 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 69020 output: 32 thoughts: 622 Thoughts: **Analyzing Memory Mapping and Validation** Okay, so I'm looking closely at how memory mapping is being handled, particularly concerning the `f_op->mmap_prepare` actions. It's clear that `mmap_action_prepare` and `mmap_action_complete` are specifically tailored for these operations. I see the code is handling `MMAP_KERNEL_PAGES` and `MMAP_DISCONTIG_KERNEL_PAGES`, preparing the mappings with `map_kernel_pages_prepare` and `map_discontig_kernel_pages_prepare`, and completing them with their respective `_complete` functions within the `mmap_action_complete` flow. Now, let's trace this back to the core `mmap_file` function. I'm noting the use of `vfs_mmap` and the critical inclusion of `mmap_hook_validate`. This is important. This validation step is there to protect against rogue driver `mmap` and `mmap_prepare` hooks. Essentially, `mmap_hook_validate` and `mmap_prepare_validate` are acting as gatekeepers, making sure drivers don't mess with memory region starting addresses, change permissions, or violate any other memory management invariants. It's about safeguarding the integrity of memory mapping, ensuring that drivers don't inadvertently create security holes or instability by mismanaging memory. Now, I need to consider any configuration dependencies. Since the changes touch core memory management files (`mm/vma.c`, `mm/memory.c`, `mm/util.c`), `CONFIG_MMU` is standard on amd64, so that's probably not a concern. I'll need to check the relevant Kconfig options in the driver files, such as `drivers/scsi/sg.c` to be sure. It's important to make sure all appropriate features are enabled for this functionality to work as intended. |
| 24/2 | 2026/09/14 19:12 | tool | grepper |
0mArgs:{
"Expression": "config CHR_DEV_SG",
"PathPrefix": "drivers/scsi/"
}Results: {
"Output": "drivers/scsi/Kconfig=116=config BLK_DEV_SR\n--\ndrivers/scsi/Kconfig-130-\ndrivers/scsi/Kconfig:131:config CHR_DEV_SG\ndrivers/scsi/Kconfig-132-\ttristate \"SCSI generic support\"\n"
} |
| 25/2 | 2026/09/14 19:12 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 70179 output: 35 thoughts: 20 Thoughts: Here's my summary, as if I were thinking these thoughts: **Examining SCSI Driver Configuration Options** Okay, let's dive into this. I need to take a look at the SCSI driver configuration specifically. I know there's a good chance what I need is defined in `drivers/scsi/Kconfig`. It's a common location for these types of settings, so that's where I'll start. Now, I'm thinking line 131 should be a good place to begin my investigation; hopefully, the relevant options or dependencies are documented or defined around that area. Let's see what's there... I have a suspicion that... |
| 26/2 | 2026/09/14 19:12 | tool | read-file |
0mArgs:{
"File": "drivers/scsi/Kconfig",
"FirstLine": 130,
"LineCount": 20
}Results: {
"Contents": " 130:\t\n 131:\tconfig CHR_DEV_SG\n 132:\t\ttristate \"SCSI generic support\"\n 133:\t\tdepends on SCSI\n 134:\t\thelp\n 135:\t\t If you want to use SCSI scanners, synthesizers or CD-writers or just\n 136:\t\t about anything having \"SCSI\" in its name other than hard disks,\n 137:\t\t CD-ROMs or tapes, say Y here. These won't be supported by the kernel\n 138:\t\t directly, so you need some additional software which knows how to\n 139:\t\t talk to these devices using the SCSI protocol:\n 140:\t\n 141:\t\t For scanners, look at SANE (\u003chttp://www.sane-project.org/\u003e). For CD\n 142:\t\t writer software look at Cdrtools\n 143:\t\t (\u003chttp://cdrtools.sourceforge.net/\u003e)\n 144:\t\t and for burning a \"disk at once\": CDRDAO\n 145:\t\t (\u003chttp://cdrdao.sourceforge.net/\u003e). Cdparanoia is a high\n 146:\t\t quality digital reader of audio CDs (\u003chttp://www.xiph.org/paranoia/\u003e).\n 147:\t\t For other devices, it's possible that you'll have to write the\n 148:\t\t driver software yourself. Please read the file\n 149:\t\t \u003cfile:Documentation/scsi/scsi-generic.rst\u003e for more information.\n"
} |
| 27/2 | 2026/09/14 19:12 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 70767 output: 27 thoughts: 34 Thoughts: Here's my summary, as though I'm thinking through this process: **Investigating a SCSI Dependency in a Kernel Configuration** Okay, so `CONFIG_CHR_DEV_SG` depends on `SCSI`. That's the starting point. I need to understand why and where this is coming into play. My next step is to examine the code that's likely interacting with this, given that I'm seeing it referenced in this particular build configuration. Let's see... a likely candidate... I'll check `drivers/usb/mon/mon_bin.c`. That file seems like a good place to start, given the "mon" (monitor) aspect and USB interaction. I expect it is related somehow. I'll see what I can turn up. |
| 28/2 | 2026/09/14 19:12 | tool | grepper |
0mArgs:{
"Expression": "mon_bin",
"PathPrefix": "drivers/usb/"
}Results: {
"Output": "drivers/usb/mon/Makefile-5-\ndrivers/usb/mon/Makefile:6:usbmon-y := mon_main.o mon_stat.o mon_text.o mon_bin.o\ndrivers/usb/mon/Makefile-7-\n--\ndrivers/usb/mon/mon_bin.c-38-/* #2 used to be MON_IOCX_URB, removed before it got into Linus tree */\ndrivers/usb/mon/mon_bin.c:39:#define MON_IOCG_STATS _IOR(MON_IOC_MAGIC, 3, struct mon_bin_stats)\ndrivers/usb/mon/mon_bin.c-40-#define MON_IOCT_RING_SIZE _IO(MON_IOC_MAGIC, 4)\ndrivers/usb/mon/mon_bin.c-41-#define MON_IOCQ_RING_SIZE _IO(MON_IOC_MAGIC, 5)\ndrivers/usb/mon/mon_bin.c:42:#define MON_IOCX_GET _IOW(MON_IOC_MAGIC, 6, struct mon_bin_get)\ndrivers/usb/mon/mon_bin.c:43:#define MON_IOCX_MFETCH _IOWR(MON_IOC_MAGIC, 7, struct mon_bin_mfetch)\ndrivers/usb/mon/mon_bin.c-44-#define MON_IOCH_MFLUSH _IO(MON_IOC_MAGIC, 8)\ndrivers/usb/mon/mon_bin.c-45-/* #9 was MON_IOCT_SETAPI */\ndrivers/usb/mon/mon_bin.c:46:#define MON_IOCX_GETX _IOW(MON_IOC_MAGIC, 10, struct mon_bin_get)\ndrivers/usb/mon/mon_bin.c-47-\ndrivers/usb/mon/mon_bin.c-48-#ifdef CONFIG_COMPAT\ndrivers/usb/mon/mon_bin.c:49:#define MON_IOCX_GET32 _IOW(MON_IOC_MAGIC, 6, struct mon_bin_get32)\ndrivers/usb/mon/mon_bin.c:50:#define MON_IOCX_MFETCH32 _IOWR(MON_IOC_MAGIC, 7, struct mon_bin_mfetch32)\ndrivers/usb/mon/mon_bin.c:51:#define MON_IOCX_GETX32 _IOW(MON_IOC_MAGIC, 10, struct mon_bin_get32)\ndrivers/usb/mon/mon_bin.c-52-#endif\n--\ndrivers/usb/mon/mon_bin.c-90- */\ndrivers/usb/mon/mon_bin.c:91:struct mon_bin_hdr {\ndrivers/usb/mon/mon_bin.c-92-\tu64 id;\t\t\t/* URB ID - from submission to callback */\n--\ndrivers/usb/mon/mon_bin.c-122- */\ndrivers/usb/mon/mon_bin.c:123:struct mon_bin_isodesc {\ndrivers/usb/mon/mon_bin.c-124-\tint iso_status;\n--\ndrivers/usb/mon/mon_bin.c-130-/* per file statistic */\ndrivers/usb/mon/mon_bin.c:131:struct mon_bin_stats {\ndrivers/usb/mon/mon_bin.c-132-\tu32 queued;\n--\ndrivers/usb/mon/mon_bin.c-135-\ndrivers/usb/mon/mon_bin.c:136:struct mon_bin_get {\ndrivers/usb/mon/mon_bin.c:137:\tstruct mon_bin_hdr __user *hdr;\t/* Can be 48 bytes or 64. */\ndrivers/usb/mon/mon_bin.c-138-\tvoid __user *data;\n--\ndrivers/usb/mon/mon_bin.c-141-\ndrivers/usb/mon/mon_bin.c:142:struct mon_bin_mfetch {\ndrivers/usb/mon/mon_bin.c-143-\tu32 __user *offvec;\t/* Vector of events fetched */\n--\ndrivers/usb/mon/mon_bin.c-148-#ifdef CONFIG_COMPAT\ndrivers/usb/mon/mon_bin.c:149:struct mon_bin_get32 {\ndrivers/usb/mon/mon_bin.c-150-\tu32 hdr32;\n--\ndrivers/usb/mon/mon_bin.c-154-\ndrivers/usb/mon/mon_bin.c:155:struct mon_bin_mfetch32 {\ndrivers/usb/mon/mon_bin.c-156- u32 offvec32;\n--\ndrivers/usb/mon/mon_bin.c-161-\ndrivers/usb/mon/mon_bin.c:162:/* Having these two values same prevents wrapping of the mon_bin_hdr */\ndrivers/usb/mon/mon_bin.c-163-#define PKT_ALIGN 64\n--\ndrivers/usb/mon/mon_bin.c=185=struct mon_reader_bin {\n--\ndrivers/usb/mon/mon_bin.c-204-\ndrivers/usb/mon/mon_bin.c:205:static inline struct mon_bin_hdr *MON_OFF2HDR(const struct mon_reader_bin *rp,\ndrivers/usb/mon/mon_bin.c-206- unsigned int offset)\ndrivers/usb/mon/mon_bin.c-207-{\ndrivers/usb/mon/mon_bin.c:208:\treturn (struct mon_bin_hdr *)\ndrivers/usb/mon/mon_bin.c-209-\t (rp-\u003eb_vec[offset / CHUNK_SIZE].ptr + offset % CHUNK_SIZE);\n--\ndrivers/usb/mon/mon_bin.c=214=static unsigned char xfer_to_pipe[4] = {\n--\ndrivers/usb/mon/mon_bin.c-217-\ndrivers/usb/mon/mon_bin.c:218:static const struct class mon_bin_class = {\ndrivers/usb/mon/mon_bin.c-219-\t.name = \"usbmon\",\n--\ndrivers/usb/mon/mon_bin.c-221-\ndrivers/usb/mon/mon_bin.c:222:static dev_t mon_bin_dev0;\ndrivers/usb/mon/mon_bin.c:223:static struct cdev mon_bin_cdev;\ndrivers/usb/mon/mon_bin.c-224-\ndrivers/usb/mon/mon_bin.c=225=static void mon_buff_area_fill(const struct mon_reader_bin *rp,\ndrivers/usb/mon/mon_bin.c-226- unsigned int offset, unsigned int size);\ndrivers/usb/mon/mon_bin.c:227:static int mon_bin_wait_event(struct file *file, struct mon_reader_bin *rp);\ndrivers/usb/mon/mon_bin.c-228-static int mon_alloc_buff(struct mon_pgmap *map, int npages);\n--\ndrivers/usb/mon/mon_bin.c=387=static void mon_buff_area_fill(const struct mon_reader_bin *rp,\n--\ndrivers/usb/mon/mon_bin.c-389-{\ndrivers/usb/mon/mon_bin.c:390:\tstruct mon_bin_hdr *ep;\ndrivers/usb/mon/mon_bin.c-391-\n--\ndrivers/usb/mon/mon_bin.c-397-\ndrivers/usb/mon/mon_bin.c:398:static inline char mon_bin_get_setup(unsigned char *setupb,\ndrivers/usb/mon/mon_bin.c-399- const struct urb *urb, char ev_type)\n--\ndrivers/usb/mon/mon_bin.c-407-\ndrivers/usb/mon/mon_bin.c:408:static unsigned int mon_bin_get_data(const struct mon_reader_bin *rp,\ndrivers/usb/mon/mon_bin.c-409- unsigned int offset, struct urb *urb, unsigned int length,\n--\ndrivers/usb/mon/mon_bin.c-451- */\ndrivers/usb/mon/mon_bin.c:452:static unsigned int mon_bin_collate_isodesc(const struct mon_reader_bin *rp,\ndrivers/usb/mon/mon_bin.c-453- struct urb *urb, unsigned int ndesc)\n--\ndrivers/usb/mon/mon_bin.c-469-\ndrivers/usb/mon/mon_bin.c:470:static void mon_bin_get_isodesc(const struct mon_reader_bin *rp,\ndrivers/usb/mon/mon_bin.c-471- unsigned int offset, struct urb *urb, char ev_type, unsigned int ndesc)\ndrivers/usb/mon/mon_bin.c-472-{\ndrivers/usb/mon/mon_bin.c:473:\tstruct mon_bin_isodesc *dp;\ndrivers/usb/mon/mon_bin.c-474-\tstruct usb_iso_packet_descriptor *fp;\n--\ndrivers/usb/mon/mon_bin.c-477-\twhile (ndesc-- != 0) {\ndrivers/usb/mon/mon_bin.c:478:\t\tdp = (struct mon_bin_isodesc *)\ndrivers/usb/mon/mon_bin.c-479-\t\t (rp-\u003eb_vec[offset / CHUNK_SIZE].ptr + offset % CHUNK_SIZE);\n--\ndrivers/usb/mon/mon_bin.c-483-\t\tdp-\u003e_pad = 0;\ndrivers/usb/mon/mon_bin.c:484:\t\tif ((offset += sizeof(struct mon_bin_isodesc)) \u003e= rp-\u003eb_size)\ndrivers/usb/mon/mon_bin.c-485-\t\t\toffset = 0;\n--\ndrivers/usb/mon/mon_bin.c-489-\ndrivers/usb/mon/mon_bin.c:490:static void mon_bin_event(struct mon_reader_bin *rp, struct urb *urb,\ndrivers/usb/mon/mon_bin.c-491- char ev_type, int status)\n--\ndrivers/usb/mon/mon_bin.c-501-\tunsigned char dir;\ndrivers/usb/mon/mon_bin.c:502:\tstruct mon_bin_hdr *ep;\ndrivers/usb/mon/mon_bin.c-503-\tchar data_tag = 0;\n--\ndrivers/usb/mon/mon_bin.c-524-\t\tif (ev_type == 'C' \u0026\u0026 usb_urb_dir_in(urb))\ndrivers/usb/mon/mon_bin.c:525:\t\t\tlength = mon_bin_collate_isodesc(rp, urb, ndesc);\ndrivers/usb/mon/mon_bin.c-526-\t} else {\n--\ndrivers/usb/mon/mon_bin.c-528-\t}\ndrivers/usb/mon/mon_bin.c:529:\tlendesc = ndesc*sizeof(struct mon_bin_isodesc);\ndrivers/usb/mon/mon_bin.c-530-\n--\ndrivers/usb/mon/mon_bin.c-594-\tif (usb_endpoint_xfer_control(epd) \u0026\u0026 ev_type == 'S') {\ndrivers/usb/mon/mon_bin.c:595:\t\tep-\u003eflag_setup = mon_bin_get_setup(ep-\u003es.setup, urb, ev_type);\ndrivers/usb/mon/mon_bin.c-596-\t} else {\n--\ndrivers/usb/mon/mon_bin.c-601-\t\tep-\u003endesc = ndesc;\ndrivers/usb/mon/mon_bin.c:602:\t\tmon_bin_get_isodesc(rp, offset, urb, ev_type, ndesc);\ndrivers/usb/mon/mon_bin.c-603-\t\tif ((offset += lendesc) \u003e= rp-\u003eb_size)\n--\ndrivers/usb/mon/mon_bin.c-607-\tif (length != 0) {\ndrivers/usb/mon/mon_bin.c:608:\t\tlength = mon_bin_get_data(rp, offset, urb, length,\ndrivers/usb/mon/mon_bin.c-609-\t\t\t\t\u0026ep-\u003eflag_data);\n--\ndrivers/usb/mon/mon_bin.c-624-\ndrivers/usb/mon/mon_bin.c:625:static void mon_bin_submit(void *data, struct urb *urb)\ndrivers/usb/mon/mon_bin.c-626-{\ndrivers/usb/mon/mon_bin.c-627-\tstruct mon_reader_bin *rp = data;\ndrivers/usb/mon/mon_bin.c:628:\tmon_bin_event(rp, urb, 'S', -EINPROGRESS);\ndrivers/usb/mon/mon_bin.c-629-}\ndrivers/usb/mon/mon_bin.c-630-\ndrivers/usb/mon/mon_bin.c:631:static void mon_bin_complete(void *data, struct urb *urb, int status)\ndrivers/usb/mon/mon_bin.c-632-{\ndrivers/usb/mon/mon_bin.c-633-\tstruct mon_reader_bin *rp = data;\ndrivers/usb/mon/mon_bin.c:634:\tmon_bin_event(rp, urb, 'C', status);\ndrivers/usb/mon/mon_bin.c-635-}\ndrivers/usb/mon/mon_bin.c-636-\ndrivers/usb/mon/mon_bin.c:637:static void mon_bin_error(void *data, struct urb *urb, int error)\ndrivers/usb/mon/mon_bin.c-638-{\n--\ndrivers/usb/mon/mon_bin.c-642-\tunsigned int offset;\ndrivers/usb/mon/mon_bin.c:643:\tstruct mon_bin_hdr *ep;\ndrivers/usb/mon/mon_bin.c-644-\n--\ndrivers/usb/mon/mon_bin.c-677-\ndrivers/usb/mon/mon_bin.c:678:static int mon_bin_open(struct inode *inode, struct file *file)\ndrivers/usb/mon/mon_bin.c-679-{\n--\ndrivers/usb/mon/mon_bin.c-717-\trp-\u003er.r_data = rp;\ndrivers/usb/mon/mon_bin.c:718:\trp-\u003er.rnf_submit = mon_bin_submit;\ndrivers/usb/mon/mon_bin.c:719:\trp-\u003er.rnf_error = mon_bin_error;\ndrivers/usb/mon/mon_bin.c:720:\trp-\u003er.rnf_complete = mon_bin_complete;\ndrivers/usb/mon/mon_bin.c-721-\n--\ndrivers/usb/mon/mon_bin.c-741- */\ndrivers/usb/mon/mon_bin.c:742:static int mon_bin_get_event(struct file *file, struct mon_reader_bin *rp,\ndrivers/usb/mon/mon_bin.c:743: struct mon_bin_hdr __user *hdr, unsigned int hdrbytes,\ndrivers/usb/mon/mon_bin.c-744- void __user *data, unsigned int nbytes)\n--\ndrivers/usb/mon/mon_bin.c-746-\tunsigned long flags;\ndrivers/usb/mon/mon_bin.c:747:\tstruct mon_bin_hdr *ep;\ndrivers/usb/mon/mon_bin.c-748-\tsize_t step_len;\n--\ndrivers/usb/mon/mon_bin.c-753-\ndrivers/usb/mon/mon_bin.c:754:\tif ((rc = mon_bin_wait_event(file, rp)) \u003c 0) {\ndrivers/usb/mon/mon_bin.c-755-\t\tmutex_unlock(\u0026rp-\u003efetch_lock);\n--\ndrivers/usb/mon/mon_bin.c-782-\ndrivers/usb/mon/mon_bin.c:783:static int mon_bin_release(struct inode *inode, struct file *file)\ndrivers/usb/mon/mon_bin.c-784-{\n--\ndrivers/usb/mon/mon_bin.c-804-\ndrivers/usb/mon/mon_bin.c:805:static ssize_t mon_bin_read(struct file *file, char __user *buf,\ndrivers/usb/mon/mon_bin.c-806- size_t nbytes, loff_t *ppos)\n--\ndrivers/usb/mon/mon_bin.c-810-\tunsigned long flags;\ndrivers/usb/mon/mon_bin.c:811:\tstruct mon_bin_hdr *ep;\ndrivers/usb/mon/mon_bin.c-812-\tunsigned int offset;\n--\ndrivers/usb/mon/mon_bin.c-819-\ndrivers/usb/mon/mon_bin.c:820:\tif ((rc = mon_bin_wait_event(file, rp)) \u003c 0) {\ndrivers/usb/mon/mon_bin.c-821-\t\tmutex_unlock(\u0026rp-\u003efetch_lock);\n--\ndrivers/usb/mon/mon_bin.c-876- */\ndrivers/usb/mon/mon_bin.c:877:static int mon_bin_flush(struct mon_reader_bin *rp, unsigned nevents)\ndrivers/usb/mon/mon_bin.c-878-{\ndrivers/usb/mon/mon_bin.c-879-\tunsigned long flags;\ndrivers/usb/mon/mon_bin.c:880:\tstruct mon_bin_hdr *ep;\ndrivers/usb/mon/mon_bin.c-881-\tint i;\n--\ndrivers/usb/mon/mon_bin.c-899- * Fetch at most max event offsets into the buffer and put them into vec.\ndrivers/usb/mon/mon_bin.c:900: * The events are usually freed later with mon_bin_flush.\ndrivers/usb/mon/mon_bin.c-901- * Return the effective number of events fetched.\ndrivers/usb/mon/mon_bin.c-902- */\ndrivers/usb/mon/mon_bin.c:903:static int mon_bin_fetch(struct file *file, struct mon_reader_bin *rp,\ndrivers/usb/mon/mon_bin.c-904- u32 __user *vec, unsigned int max)\n--\ndrivers/usb/mon/mon_bin.c-909-\tunsigned int nevents;\ndrivers/usb/mon/mon_bin.c:910:\tstruct mon_bin_hdr *ep;\ndrivers/usb/mon/mon_bin.c-911-\tunsigned long flags;\n--\ndrivers/usb/mon/mon_bin.c-915-\ndrivers/usb/mon/mon_bin.c:916:\tif ((rc = mon_bin_wait_event(file, rp)) \u003c 0) {\ndrivers/usb/mon/mon_bin.c-917-\t\tmutex_unlock(\u0026rp-\u003efetch_lock);\n--\ndrivers/usb/mon/mon_bin.c-950-/*\ndrivers/usb/mon/mon_bin.c:951: * Count events. This is almost the same as the above mon_bin_fetch,\ndrivers/usb/mon/mon_bin.c-952- * only we do not store offsets into user vector, and we have no limit.\ndrivers/usb/mon/mon_bin.c-953- */\ndrivers/usb/mon/mon_bin.c:954:static int mon_bin_queued(struct mon_reader_bin *rp)\ndrivers/usb/mon/mon_bin.c-955-{\n--\ndrivers/usb/mon/mon_bin.c-959-\tunsigned int nevents;\ndrivers/usb/mon/mon_bin.c:960:\tstruct mon_bin_hdr *ep;\ndrivers/usb/mon/mon_bin.c-961-\tunsigned long flags;\n--\ndrivers/usb/mon/mon_bin.c-988- */\ndrivers/usb/mon/mon_bin.c:989:static long mon_bin_ioctl(struct file *file, unsigned int cmd, unsigned long arg)\ndrivers/usb/mon/mon_bin.c-990-{\n--\ndrivers/usb/mon/mon_bin.c-993-\tint ret = 0;\ndrivers/usb/mon/mon_bin.c:994:\tstruct mon_bin_hdr *ep;\ndrivers/usb/mon/mon_bin.c-995-\tunsigned long flags;\n--\ndrivers/usb/mon/mon_bin.c-1063-\tcase MON_IOCH_MFLUSH:\ndrivers/usb/mon/mon_bin.c:1064:\t\tret = mon_bin_flush(rp, arg);\ndrivers/usb/mon/mon_bin.c-1065-\t\tbreak;\n--\ndrivers/usb/mon/mon_bin.c-1069-\t\t{\ndrivers/usb/mon/mon_bin.c:1070:\t\tstruct mon_bin_get getb;\ndrivers/usb/mon/mon_bin.c-1071-\ndrivers/usb/mon/mon_bin.c-1072-\t\tif (copy_from_user(\u0026getb, (void __user *)arg,\ndrivers/usb/mon/mon_bin.c:1073:\t\t\t\t\t sizeof(struct mon_bin_get)))\ndrivers/usb/mon/mon_bin.c-1074-\t\t\treturn -EFAULT;\n--\ndrivers/usb/mon/mon_bin.c-1077-\t\t\treturn -EINVAL;\ndrivers/usb/mon/mon_bin.c:1078:\t\tret = mon_bin_get_event(file, rp, getb.hdr,\ndrivers/usb/mon/mon_bin.c-1079-\t\t (cmd == MON_IOCX_GET)? PKT_SZ_API0: PKT_SZ_API1,\n--\ndrivers/usb/mon/mon_bin.c-1085-\t\t{\ndrivers/usb/mon/mon_bin.c:1086:\t\tstruct mon_bin_mfetch mfetch;\ndrivers/usb/mon/mon_bin.c:1087:\t\tstruct mon_bin_mfetch __user *uptr;\ndrivers/usb/mon/mon_bin.c-1088-\ndrivers/usb/mon/mon_bin.c:1089:\t\tuptr = (struct mon_bin_mfetch __user *)arg;\ndrivers/usb/mon/mon_bin.c-1090-\n--\ndrivers/usb/mon/mon_bin.c-1094-\t\tif (mfetch.nflush) {\ndrivers/usb/mon/mon_bin.c:1095:\t\t\tret = mon_bin_flush(rp, mfetch.nflush);\ndrivers/usb/mon/mon_bin.c-1096-\t\t\tif (ret \u003c 0)\n--\ndrivers/usb/mon/mon_bin.c-1100-\t\t}\ndrivers/usb/mon/mon_bin.c:1101:\t\tret = mon_bin_fetch(file, rp, mfetch.offvec, mfetch.nfetch);\ndrivers/usb/mon/mon_bin.c-1102-\t\tif (ret \u003c 0)\n--\ndrivers/usb/mon/mon_bin.c-1110-\tcase MON_IOCG_STATS: {\ndrivers/usb/mon/mon_bin.c:1111:\t\tstruct mon_bin_stats __user *sp;\ndrivers/usb/mon/mon_bin.c-1112-\t\tunsigned int nevents;\n--\ndrivers/usb/mon/mon_bin.c-1118-\t\tspin_unlock_irqrestore(\u0026rp-\u003eb_lock, flags);\ndrivers/usb/mon/mon_bin.c:1119:\t\tnevents = mon_bin_queued(rp);\ndrivers/usb/mon/mon_bin.c-1120-\ndrivers/usb/mon/mon_bin.c:1121:\t\tsp = (struct mon_bin_stats __user *)arg;\ndrivers/usb/mon/mon_bin.c-1122-\t\tif (put_user(ndropped, \u0026sp-\u003edropped))\n--\ndrivers/usb/mon/mon_bin.c-1137-#ifdef CONFIG_COMPAT\ndrivers/usb/mon/mon_bin.c:1138:static long mon_bin_compat_ioctl(struct file *file,\ndrivers/usb/mon/mon_bin.c-1139- unsigned int cmd, unsigned long arg)\n--\ndrivers/usb/mon/mon_bin.c-1148-\t\t{\ndrivers/usb/mon/mon_bin.c:1149:\t\tstruct mon_bin_get32 getb;\ndrivers/usb/mon/mon_bin.c-1150-\ndrivers/usb/mon/mon_bin.c-1151-\t\tif (copy_from_user(\u0026getb, (void __user *)arg,\ndrivers/usb/mon/mon_bin.c:1152:\t\t\t\t\t sizeof(struct mon_bin_get32)))\ndrivers/usb/mon/mon_bin.c-1153-\t\t\treturn -EFAULT;\ndrivers/usb/mon/mon_bin.c-1154-\ndrivers/usb/mon/mon_bin.c:1155:\t\tret = mon_bin_get_event(file, rp, compat_ptr(getb.hdr32),\ndrivers/usb/mon/mon_bin.c-1156-\t\t (cmd == MON_IOCX_GET32)? PKT_SZ_API0: PKT_SZ_API1,\n--\ndrivers/usb/mon/mon_bin.c-1164-\t\t{\ndrivers/usb/mon/mon_bin.c:1165:\t\tstruct mon_bin_mfetch32 mfetch;\ndrivers/usb/mon/mon_bin.c:1166:\t\tstruct mon_bin_mfetch32 __user *uptr;\ndrivers/usb/mon/mon_bin.c-1167-\ndrivers/usb/mon/mon_bin.c:1168:\t\tuptr = (struct mon_bin_mfetch32 __user *) compat_ptr(arg);\ndrivers/usb/mon/mon_bin.c-1169-\n--\ndrivers/usb/mon/mon_bin.c-1173-\t\tif (mfetch.nflush32) {\ndrivers/usb/mon/mon_bin.c:1174:\t\t\tret = mon_bin_flush(rp, mfetch.nflush32);\ndrivers/usb/mon/mon_bin.c-1175-\t\t\tif (ret \u003c 0)\n--\ndrivers/usb/mon/mon_bin.c-1179-\t\t}\ndrivers/usb/mon/mon_bin.c:1180:\t\tret = mon_bin_fetch(file, rp, compat_ptr(mfetch.offvec32),\ndrivers/usb/mon/mon_bin.c-1181-\t\t mfetch.nfetch32);\n--\ndrivers/usb/mon/mon_bin.c-1189-\tcase MON_IOCG_STATS:\ndrivers/usb/mon/mon_bin.c:1190:\t\treturn mon_bin_ioctl(file, cmd, (unsigned long) compat_ptr(arg));\ndrivers/usb/mon/mon_bin.c-1191-\n--\ndrivers/usb/mon/mon_bin.c-1195-\tcase MON_IOCH_MFLUSH:\ndrivers/usb/mon/mon_bin.c:1196:\t\treturn mon_bin_ioctl(file, cmd, arg);\ndrivers/usb/mon/mon_bin.c-1197-\n--\ndrivers/usb/mon/mon_bin.c=1205=static __poll_t\ndrivers/usb/mon/mon_bin.c:1206:mon_bin_poll(struct file *file, struct poll_table_struct *wait)\ndrivers/usb/mon/mon_bin.c-1207-{\n--\ndrivers/usb/mon/mon_bin.c-1221-\ndrivers/usb/mon/mon_bin.c:1222:static void __mon_bin_vma_open(struct mon_reader_bin *rp)\ndrivers/usb/mon/mon_bin.c-1223-{\n--\ndrivers/usb/mon/mon_bin.c-1234- */\ndrivers/usb/mon/mon_bin.c:1235:static void mon_bin_vma_open(struct vm_area_struct *vma)\ndrivers/usb/mon/mon_bin.c-1236-{\n--\ndrivers/usb/mon/mon_bin.c-1238-\ndrivers/usb/mon/mon_bin.c:1239:\t__mon_bin_vma_open(rp);\ndrivers/usb/mon/mon_bin.c-1240-}\ndrivers/usb/mon/mon_bin.c-1241-\ndrivers/usb/mon/mon_bin.c:1242:static void __mon_bin_vma_close(struct mon_reader_bin *rp)\ndrivers/usb/mon/mon_bin.c-1243-{\n--\ndrivers/usb/mon/mon_bin.c-1250-\ndrivers/usb/mon/mon_bin.c:1251:static void mon_bin_vma_close(struct vm_area_struct *vma)\ndrivers/usb/mon/mon_bin.c-1252-{\n--\ndrivers/usb/mon/mon_bin.c-1254-\ndrivers/usb/mon/mon_bin.c:1255:\t__mon_bin_vma_close(rp);\ndrivers/usb/mon/mon_bin.c-1256-}\ndrivers/usb/mon/mon_bin.c-1257-\ndrivers/usb/mon/mon_bin.c:1258:static const struct vm_operations_struct mon_bin_vm_ops = {\ndrivers/usb/mon/mon_bin.c:1259:\t.open = mon_bin_vma_open,\ndrivers/usb/mon/mon_bin.c:1260:\t.close = mon_bin_vma_close,\ndrivers/usb/mon/mon_bin.c-1261-};\ndrivers/usb/mon/mon_bin.c-1262-\ndrivers/usb/mon/mon_bin.c:1263:static int mon_bin_discontig_init(void *vm_private_data, void **private)\ndrivers/usb/mon/mon_bin.c-1264-{\n--\ndrivers/usb/mon/mon_bin.c-1266-\ndrivers/usb/mon/mon_bin.c:1267:\t/* Dropped by mon_bin_vma_close() on unmap, including on error. */\ndrivers/usb/mon/mon_bin.c:1268:\t__mon_bin_vma_open(rp);\ndrivers/usb/mon/mon_bin.c-1269-\treturn 0;\n--\ndrivers/usb/mon/mon_bin.c-1271-\ndrivers/usb/mon/mon_bin.c:1272:static int mon_bin_discontig_get(struct discontig_kernel_page_state *state)\ndrivers/usb/mon/mon_bin.c-1273-{\n--\ndrivers/usb/mon/mon_bin.c=1293=static const struct discontig_kernel_page_ops mon_discontig_ops = {\ndrivers/usb/mon/mon_bin.c:1294:\t.init = mon_bin_discontig_init,\ndrivers/usb/mon/mon_bin.c:1295:\t.get = mon_bin_discontig_get,\ndrivers/usb/mon/mon_bin.c-1296-};\ndrivers/usb/mon/mon_bin.c-1297-\ndrivers/usb/mon/mon_bin.c:1298:static int mon_bin_mmap_prepare(struct vm_area_desc *desc)\ndrivers/usb/mon/mon_bin.c-1299-{\n--\ndrivers/usb/mon/mon_bin.c-1304-\ndrivers/usb/mon/mon_bin.c:1305:\tdesc-\u003evm_ops = \u0026mon_bin_vm_ops;\ndrivers/usb/mon/mon_bin.c-1306-\tvma_desc_clear_flags(desc, VMA_MAYWRITE_BIT);\n--\ndrivers/usb/mon/mon_bin.c=1314=static const struct file_operations mon_fops_binary = {\ndrivers/usb/mon/mon_bin.c-1315-\t.owner =\tTHIS_MODULE,\ndrivers/usb/mon/mon_bin.c:1316:\t.open =\t\tmon_bin_open,\ndrivers/usb/mon/mon_bin.c:1317:\t.read =\t\tmon_bin_read,\ndrivers/usb/mon/mon_bin.c-1318-\t/* .write =\tmon_text_write, */\ndrivers/usb/mon/mon_bin.c:1319:\t.poll =\t\tmon_bin_poll,\ndrivers/usb/mon/mon_bin.c:1320:\t.unlocked_ioctl = mon_bin_ioctl,\ndrivers/usb/mon/mon_bin.c-1321-#ifdef CONFIG_COMPAT\ndrivers/usb/mon/mon_bin.c:1322:\t.compat_ioctl =\tmon_bin_compat_ioctl,\ndrivers/usb/mon/mon_bin.c-1323-#endif\ndrivers/usb/mon/mon_bin.c:1324:\t.release =\tmon_bin_release,\ndrivers/usb/mon/mon_bin.c:1325:\t.mmap_prepare = mon_bin_mmap_prepare,\ndrivers/usb/mon/mon_bin.c-1326-};\ndrivers/usb/mon/mon_bin.c-1327-\ndrivers/usb/mon/mon_bin.c:1328:static int mon_bin_wait_event(struct file *file, struct mon_reader_bin *rp)\ndrivers/usb/mon/mon_bin.c-1329-{\n--\ndrivers/usb/mon/mon_bin.c=1379=static void mon_free_buff(struct mon_pgmap *map, int npages)\n--\ndrivers/usb/mon/mon_bin.c-1386-\ndrivers/usb/mon/mon_bin.c:1387:int mon_bin_add(struct mon_bus *mbus, const struct usb_bus *ubus)\ndrivers/usb/mon/mon_bin.c-1388-{\n--\ndrivers/usb/mon/mon_bin.c-1394-\ndrivers/usb/mon/mon_bin.c:1395:\tdev = device_create(\u0026mon_bin_class, ubus ? ubus-\u003econtroller : NULL,\ndrivers/usb/mon/mon_bin.c:1396:\t\t\t MKDEV(MAJOR(mon_bin_dev0), minor), NULL,\ndrivers/usb/mon/mon_bin.c-1397-\t\t\t \"usbmon%d\", minor);\n--\ndrivers/usb/mon/mon_bin.c-1404-\ndrivers/usb/mon/mon_bin.c:1405:void mon_bin_del(struct mon_bus *mbus)\ndrivers/usb/mon/mon_bin.c-1406-{\ndrivers/usb/mon/mon_bin.c:1407:\tdevice_destroy(\u0026mon_bin_class, mbus-\u003eclassdev-\u003edevt);\ndrivers/usb/mon/mon_bin.c-1408-}\ndrivers/usb/mon/mon_bin.c-1409-\ndrivers/usb/mon/mon_bin.c:1410:int __init mon_bin_init(void)\ndrivers/usb/mon/mon_bin.c-1411-{\n--\ndrivers/usb/mon/mon_bin.c-1413-\ndrivers/usb/mon/mon_bin.c:1414:\trc = class_register(\u0026mon_bin_class);\ndrivers/usb/mon/mon_bin.c-1415-\tif (rc)\n--\ndrivers/usb/mon/mon_bin.c-1417-\ndrivers/usb/mon/mon_bin.c:1418:\trc = alloc_chrdev_region(\u0026mon_bin_dev0, 0, MON_BIN_MAX_MINOR, \"usbmon\");\ndrivers/usb/mon/mon_bin.c-1419-\tif (rc \u003c 0)\n--\ndrivers/usb/mon/mon_bin.c-1421-\ndrivers/usb/mon/mon_bin.c:1422:\tcdev_init(\u0026mon_bin_cdev, \u0026mon_fops_binary);\ndrivers/usb/mon/mon_bin.c:1423:\tmon_bin_cdev.owner = THIS_MODULE;\ndrivers/usb/mon/mon_bin.c-1424-\ndrivers/usb/mon/mon_bin.c:1425:\trc = cdev_add(\u0026mon_bin_cdev, mon_bin_dev0, MON_BIN_MAX_MINOR);\ndrivers/usb/mon/mon_bin.c-1426-\tif (rc \u003c 0)\n--\ndrivers/usb/mon/mon_bin.c-1431-err_add:\ndrivers/usb/mon/mon_bin.c:1432:\tunregister_chrdev_region(mon_bin_dev0, MON_BIN_MAX_MINOR);\ndrivers/usb/mon/mon_bin.c-1433-err_dev:\ndrivers/usb/mon/mon_bin.c:1434:\tclass_unregister(\u0026mon_bin_class);\ndrivers/usb/mon/mon_bin.c-1435-err_class:\n--\ndrivers/usb/mon/mon_bin.c-1438-\ndrivers/usb/mon/mon_bin.c:1439:void mon_bin_exit(void)\ndrivers/usb/mon/mon_bin.c-1440-{\ndrivers/usb/mon/mon_bin.c:1441:\tcdev_del(\u0026mon_bin_cdev);\ndrivers/usb/mon/mon_bin.c:1442:\tunregister_chrdev_region(mon_bin_dev0, MON_BIN_MAX_MINOR);\ndrivers/usb/mon/mon_bin.c:1443:\tclass_unregister(\u0026mon_bin_class);\ndrivers/usb/mon/mon_bin.c-1444-}\n--\ndrivers/usb/mon/mon_main.c=198=static void mon_bus_remove(struct usb_bus *ubus)\n--\ndrivers/usb/mon/mon_main.c-206-\tif (mbus-\u003ebin_inited)\ndrivers/usb/mon/mon_main.c:207:\t\tmon_bin_del(mbus);\ndrivers/usb/mon/mon_main.c-208-\n--\ndrivers/usb/mon/mon_main.c=272=static void mon_bus_init(struct usb_bus *ubus)\n--\ndrivers/usb/mon/mon_main.c-290-\tmbus-\u003etext_inited = mon_text_add(mbus, ubus);\ndrivers/usb/mon/mon_main.c:291:\tmbus-\u003ebin_inited = mon_bin_add(mbus, ubus);\ndrivers/usb/mon/mon_main.c-292-\n--\ndrivers/usb/mon/mon_main.c=302=static void mon_bus0_init(void)\n--\ndrivers/usb/mon/mon_main.c-310-\tmbus-\u003etext_inited = mon_text_add(mbus, NULL);\ndrivers/usb/mon/mon_main.c:311:\tmbus-\u003ebin_inited = mon_bin_add(mbus, NULL);\ndrivers/usb/mon/mon_main.c-312-}\n--\ndrivers/usb/mon/mon_main.c=337=static int __init mon_init(void)\n--\ndrivers/usb/mon/mon_main.c-343-\t\tgoto err_text;\ndrivers/usb/mon/mon_main.c:344:\tif ((rc = mon_bin_init()) != 0)\ndrivers/usb/mon/mon_main.c-345-\t\tgoto err_bin;\n--\ndrivers/usb/mon/mon_main.c-363-err_reg:\ndrivers/usb/mon/mon_main.c:364:\tmon_bin_exit();\ndrivers/usb/mon/mon_main.c-365-err_bin:\n--\ndrivers/usb/mon/mon_main.c=371=static void __exit mon_exit(void)\n--\ndrivers/usb/mon/mon_main.c-388-\t\tif (mbus-\u003ebin_inited)\ndrivers/usb/mon/mon_main.c:389:\t\t\tmon_bin_del(mbus);\ndrivers/usb/mon/mon_main.c-390-\n--\ndrivers/usb/mon/mon_main.c-410-\tif (mbus-\u003ebin_inited)\ndrivers/usb/mon/mon_main.c:411:\t\tmon_bin_del(mbus);\ndrivers/usb/mon/mon_main.c-412-\n--\ndrivers/usb/mon/mon_main.c-415-\tmon_text_exit();\ndrivers/usb/mon/mon_main.c:416:\tmon_bin_exit();\ndrivers/usb/mon/mon_main.c-417-}\n--\ndrivers/usb/mon/usb_mon.h=59=void mon_text_del(struct mon_bus *mbus);\ndrivers/usb/mon/usb_mon.h:60:int /*bool*/ mon_bin_add(struct mon_bus *mbus, const struct usb_bus *ubus);\ndrivers/usb/mon/usb_mon.h:61:void mon_bin_del(struct mon_bus *mbus);\ndrivers/usb/mon/usb_mon.h-62-\n--\ndrivers/usb/mon/usb_mon.h=64=void mon_text_exit(void);\ndrivers/usb/mon/usb_mon.h:65:int __init mon_bin_init(void);\ndrivers/usb/mon/usb_mon.h:66:void mon_bin_exit(void);\ndrivers/usb/mon/usb_mon.h-67-\n"
} |
| 29/2 | 2026/09/14 19:12 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 81839 output: 28 thoughts: 10 Thoughts: Here's my attempt at summarizing the thought process, tailored for an expert user: **Investigating USBmon in Kconfig** Alright, let's dive in. First, I need to check the Kconfig files for USBmon. My hunch is it's probably controlled by a feature flag somewhere, maybe dependent on a specific USB subsystem configuration or even architecture. I need to figure out where the build-time configuration happens. I'll need to trace the dependency chain if it's not immediately obvious. Knowing the precise Kconfig options is crucial, especially if there's any conditional compilation or options related to the level of detail captured by the monitoring. I'll be looking for potential dependencies on other modules, kernel versions, or hardware platforms. Might be worth checking to see if there's any documentation about specific Kconfig settings related to USBmon performance or tuning – are there any hidden options I should be aware of? I'll be looking for how to enable, disable, and configure this in Kconfig. |
| 30/2 | 2026/09/14 19:12 | tool | grepper |
0mArgs:{
"Expression": "config USB_MON",
"PathPrefix": "drivers/usb/"
}Results: {
"Output": "drivers/usb/mon/Kconfig-5-\ndrivers/usb/mon/Kconfig:6:config USB_MON\ndrivers/usb/mon/Kconfig-7-\ttristate \"USB Monitor\"\n"
} |
| 31/2 | 2026/09/14 19:12 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 82148 output: 34 thoughts: 14 Thoughts: Here's a summary of my thought process, as if I were considering that Kconfig file: **Analyzing the USB Monitor Kconfig File** Okay, time to dive into `drivers/usb/mon/Kconfig`. Right, it's a Kconfig file, so I know it's all about configuration options within the USB monitor subsystem. My immediate goal is to understand what options are available, how they're structured, and what dependencies might exist. I'm looking for the core functionality first – what defines the USB monitor itself? Are there variations, specific features that can be enabled or disabled? I'm anticipating finding things like options for enabling specific data capture methods, filtering capabilities, perhaps even support for different USB protocol versions or hardware monitoring interfaces. I'll be particularly interested in dependencies; does the monitor require any specific kernel features, other drivers, or perhaps a particular USB host controller configuration to be functional? I need to quickly grasp the scope of what can be configured here. I’ll start by scanning for the main `config` symbols and their associated `bool`, `tristate`, or `string` options and how they are grouped, looking for the `depends on` statements. After that, I will be thinking of any possible limitations of the monitor and how one can configure it to be effective. Finally, I will be thinking how I can enhance or integrate this Kconfig file to the overall codebase. |
| 32/2 | 2026/09/14 19:12 | tool | read-file |
0mArgs:{
"File": "drivers/usb/mon/Kconfig",
"FirstLine": 1,
"LineCount": 20
}Results: {
"Contents": " 1:\t# SPDX-License-Identifier: GPL-2.0\n 2:\t#\n 3:\t# USB Monitor configuration\n 4:\t#\n 5:\t\n 6:\tconfig USB_MON\n 7:\t\ttristate \"USB Monitor\"\n 8:\t\thelp\n 9:\t\t If you select this option, a component which captures the USB traffic\n 10:\t\t between peripheral-specific drivers and HC drivers will be built.\n 11:\t\t For more information, see \u003cfile:Documentation/usb/usbmon.rst\u003e.\n 12:\t\n 13:\t\t If unsure, say Y, if allowed, otherwise M.\n"
} |
| 33/2 | 2026/09/14 19:12 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 82660 output: 31 thoughts: 64 Thoughts: **My Initial Thoughts on Dependencies** Okay, so I'm looking at this configuration. `USB_MON` relies on `USB_SUPPORT` and the core `USB` configuration being enabled. That's straightforward enough, basic dependency logic. Now I need to see how a specific module utilizes this dependency. Let's dive into `drivers/video/fbdev/core/fb_defio.c`. Ah, right, the frame buffer deferred I/O code. Let's make sure that that is enabled and how it interacts with the configurations we're interested in. Time to do a quick grep for `CONFIG_FB_DEFERRED_IO` to see how it's defined and used in the broader picture. |
| 34/2 | 2026/09/14 19:12 | tool | grepper |
0mArgs:{
"Expression": "config FB_DEFERRED_IO",
"PathPrefix": "drivers/video/"
}Results: {
"Output": "drivers/video/fbdev/core/Kconfig=124=config FB_SYSMEM_FOPS\n--\ndrivers/video/fbdev/core/Kconfig-127-\ndrivers/video/fbdev/core/Kconfig:128:config FB_DEFERRED_IO\ndrivers/video/fbdev/core/Kconfig-129-\tbool\n"
} |
| 35/2 | 2026/09/14 19:12 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 83010 output: 29 thoughts: 1148 Thoughts: **Focusing on Key Functions for Fuzzing** Alright, let's break down this patch and figure out the most effective *FocusSymbols* for fuzzing, keeping in mind the guidelines. The goal is to identify functions that implement *specific* new logic or have their entry points modified, while avoiding the ubiquitous ones that would dilute our fuzzing efforts. First, I checked a few configuration options just to get a lay of the land: `CONFIG_FB_SSD1307` leads to `FB_DEFERRED_IO`, `CONFIG_SND_PCM` is relevant, `CONFIG_INFINIBAND_HFI1` requires specialized hardware which QEMU doesn't have, `CONFIG_HSI_CHAR` and related options seem specific to OMAP/Nokia hardware, `CONFIG_UPROBES` is always good to know about, `CONFIG_BPF_SYSCALL` is a key area, `CONFIG_USERFAULTFD` is interesting, and `CONFIG_HUGETLB_PAGE` is potentially related. `CONFIG_MMU` enables the always built `mlock`. These aren't the primary focus, but they establish context. Now, looking at the actual code changes, the following functions seem most relevant: 1. `map_discontig_kernel_pages_prepare` and `map_discontig_kernel_pages_complete` (new in `mm/memory.c`): These are entirely new functions. 2. `mmap_prepare_validate` and `mmap_hook_validate` (new in `mm/vma.c`): Another set of new functions, these are prime targets. 3. `insert_vm_struct` (modified in `mm/vma.c`): The modification, calling `mmap_prepare_validate`, makes this a good choice. This is where the new functionality is being integrated. 4. `sg_mmap_prepare` (modified in `drivers/scsi/sg.c`): Rewritten to use discontig kernel pages mmap action – this points to significant functional changes. 5. `mon_bin_mmap_prepare` (modified in `drivers/usb/mon/mon_bin.c`): Same as above. 6. `snd_pcm_mmap_status` (modified in `sound/core/pcm_native.c`): Also a significant change, using `vm_insert_page`. 7. `fb_deferred_io_mmap` (modified in `drivers/video/fbdev/core/fb_defio.c`): Adding a `WARN_ON_ONCE` and flag changes might expose interesting edge cases. 8. `__compat_vma_mmap` (modified in `mm/util.c`): Modified to call `mmap_prepare_validate`, so this is also a key target. 9. `mmap_action_prepare` and `mmap_action_complete` (in `mm/util.c`): These functions are a way into the discontig kernel pages mmap action, so they can't be missed. Finally, regarding `mlock`, I need to assess whether the relevant changes are suitable. The `mlock` syscalls, `sys_mlock`, `sys_munlock`, `sys_mlockall`, `sys_munlockall`, call `do_mlock`, which calls `apply_vma_lock_flags`. `apply_vma_lock_flags` is static, so I will need to check the symbols in `mm/mlock.c`. The core functionality is definitely in that area, but I need to avoid listing the `mlock` syscall directly, or its ubiquitous calls within. In summary, I'll prioritize fuzzing the new functions and the modified functions, particularly those that integrate or call into the new discontig kernel pages mechanism. The `mlock` syscalls and the related functions will need careful consideration to determine if they are suitably specific to the changes, and if so, how to best target them. The goal is focused, high-impact fuzzing. |
| 36/2 | 2026/09/14 19:12 | tool | grepper |
0mArgs:{
"Expression": "^int ",
"PathPrefix": "mm/mlock.c"
}Results: {
"Output": "mm/mlock.c=789=static DEFINE_SPINLOCK(shmlock_user_lock);\nmm/mlock.c-790-\nmm/mlock.c:791:int user_shm_lock(size_t size, struct ucounts *ucounts)\nmm/mlock.c-792-{\n"
} |
| 37/2 | 2026/09/14 19:12 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 85178 output: 28 thoughts: 1297 Thoughts: **Analyzing Kernel Memory Management and Related Functionalities** Okay, let's dive into this. I need to get a handle on the inlining and scope of these memory management functions, particularly as they relate to these new mechanisms. First, I examined `apply_vma_lock_flags`. I wanted to know if `mlock_vma_pages_range` in `mm/mlock.c` was static and could be inlined. I found that while it is static, I can't be sure about inlining. Given the function pointer in `struct mm_walk_ops`, `mlock_pte_range` *definitely* won't be inlined. It's a vmlinux symbol. I then started thinking about a set of global function targets: `map_discontig_kernel_pages_prepare`, `map_discontig_kernel_pages_complete`, `mmap_prepare_validate`, `mmap_hook_validate`, `mmap_action_prepare`, and `mmap_action_complete`. They're global, so, definitely not inlined! Then I saw the function pointers in `file_operations` for `sg_mmap_prepare` and `mon_bin_mmap_prepare`, so they're definitely not inlined either. I note that `apply_discontig_action` is static. `snd_pcm_mmap_status` is static in the ALSA PCM driver, called from `snd_pcm_mmap`. I moved on to related functions in `mm/`, starting with `zap_special_vma_range`, which was changed to use `vma_is_kernel_owned(vma)`. The inlines, like `vma_flags_can_merge`, `vma_is_kernel_owned`, and `vma_is_hugetlb`, are static inlines in `include/linux/mm.h`. I'll need to see where they're being used: `vma_merge_new_range` calls `vma_flags_can_merge`. I see `vma_is_kernel_owned` called by `zap_huge_pud`, `__split_huge_pmd_locked`, `vma_not_suitable_for_thp_split`, and `migrate_vma_setup`. I also noticed that `vma_can_userfault` calls `vma_is_persistent`, `vma_move_compatible` calls `vma_is_kernel_owned`, `is_valid_guard_vma` calls `vma_can_merge`, and `check_prep_vma` in `mm/mremap.c` calls `vma_is_fixed_mapping`. I then made a note of `is_valid_guard_vma` and `madvise_vma_behavior` as both are static. I also noted that `migrate_vma_setup` is an exported symbol. I also see `userfaultfd_register` in `mm/userfaultfd.c`. Now, I'm going to narrow the focus to the most crucial functions: 1. `map_discontig_kernel_pages_prepare` - For contiguous kernel page mapping core logic 2. `map_discontig_kernel_pages_complete` - For discontiguous kernel page insertion core logic 3. `mmap_prepare_validate` - New validation check 4. `mmap_hook_validate` - New validation check 5. `mmap_action_prepare` - Action dispatcher 6. `mmap_action_complete` - Action dispatcher 7. `sg_mmap_prepare` - SCSI generic driver adaptation 8. `mon_bin_mmap_prepare` - USB mon driver adaptation 9. `snd_pcm_mmap_status` - ALSA PCM status mmap rewrite Finally, let's identify any necessary configuration flags: * SCSI generic driver (`sg.c`): `CHR_DEV_SG`, `SCSI` * USB monitor (`mon_bin.c`): `USB_MON`, `USB_SUPPORT`, `USB` * ALSA PCM (`pcm_native.c`): `SND_PCM`, `SOUND` * FB defio: `FB_DEFERRED_IO`, `FB` * Core MM features: `MMU`, `HUGETLB_PAGE` I need to confirm if these configurations are standard or if there are dependencies. Checking Kconfig files, I see: * `CHR_DEV_SG` depends on `SCSI` * `USB_MON |
| 38/2 | 2026/09/14 19:12 | tool | grepper |
0mArgs:{
"Expression": "config SND_PCM\\b",
"PathPrefix": "sound/"
}Results: {
"Output": "sound/core/Kconfig=3=config SND_TIMER\n--\nsound/core/Kconfig-5-\nsound/core/Kconfig:6:config SND_PCM\nsound/core/Kconfig-7-\ttristate\n"
} |
| 39/2 | 2026/09/14 19:12 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 87594 output: 32 thoughts: 12 Thoughts: Here's my take on the thought process, as requested: **Diving into `sound/core/Kconfig`** Alright, let's get down to brass tacks. I'm going to start by taking a look at `sound/core/Kconfig`. This means I'm probably trying to understand the configuration options for the core sound subsystem within the Linux kernel. Given my expertise, I'm likely interested in how these options are structured, what dependencies exist, and what kind of granular control they provide over the audio device configuration. I anticipate this file is going to reveal the various features that can be compiled in or out, and the conditional logic used to determine when a particular sound-related feature is available. I'm expecting to see things like: definitions of audio driver capabilities, hardware support details, and potential conflict resolution between different sound modules. Ultimately, this file should paint a picture of how the sound core is configured and how it interacts with the rest of the kernel and the sound hardware. Time to get to work. |
| 40/2 | 2026/09/14 19:12 | tool | read-file |
0mArgs:{
"File": "sound/core/Kconfig",
"FirstLine": 1,
"LineCount": 20
}Results: {
"Contents": " 1:\t# SPDX-License-Identifier: GPL-2.0-only\n 2:\t# ALSA soundcard-configuration\n 3:\tconfig SND_TIMER\n 4:\t\ttristate\n 5:\t\n 6:\tconfig SND_PCM\n 7:\t\ttristate\n 8:\t\tselect SND_TIMER if SND_PCM_TIMER\n 9:\t\n 10:\tconfig SND_PCM_ELD\n 11:\t\tbool\n 12:\t\n 13:\tconfig SND_PCM_IEC958\n 14:\t\tbool\n 15:\t\n 16:\tconfig SND_DMAENGINE_PCM\n 17:\t\ttristate\n 18:\t\n 19:\tconfig SND_HWDEP\n 20:\t\ttristate\n"
} |
| 41/2 | 2026/09/14 19:12 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 88058 output: 28 thoughts: 35 Thoughts: **My Initial Thoughts on SND_PCM Driver Selection** Okay, so I'm thinking about which drivers specifically utilize `SND_PCM`. The first thing that pops into my head is definitely virtual sound cards, things like `SND_DUMMY`. That's a good place to start, as they often implement the PCM interface in a non-standard way. Now, I'm going to quickly check `sound/drivers/Kconfig` to get a clearer picture of the landscape. That Kconfig file should define all the driver options, and it’s the definitive source for this kind of information. I need to see what dependencies and relationships are defined in there. |
| 42/2 | 2026/09/14 19:12 | tool | grepper |
0mArgs:{
"Expression": "config SND_DUMMY",
"PathPrefix": "sound/"
}Results: {
"Output": "sound/drivers/Kconfig=49=config SND_PCSP\n--\nsound/drivers/Kconfig-78-\nsound/drivers/Kconfig:79:config SND_DUMMY\nsound/drivers/Kconfig-80-\ttristate \"Dummy (/dev/null) soundcard\"\n"
} |
| 43/2 | 2026/09/14 19:12 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 88346 output: 346 thoughts: 1390 Thoughts: **Analysis of the Discontiguous Kernel Page Mapping Patch** Okay, so the `SND_DUMMY` symbol selects `SND_PCM`, that's noted. Now, let's consider the implications regarding `HUGETLB_PAGE`. The patch heavily uses `vma_is_hugetlb(vma)` instead of the old `is_vm_hugetlb_page(vma)`. This is key, as it's tied to `IS_ENABLED(CONFIG_HUGETLB_PAGE) && vma_flags_test(flags, VMA_HUGETLB_BIT)`. Therefore, `HUGETLB_PAGE` is definitely relevant to my analysis. What about `USERFAULTFD`? Significant changes are also visible in `mm/userfaultfd.c` and `mm/huge_memory.c`, so I'll keep that in mind. To be thorough, let's do a quick sweep of all files touched in this diff. The breadth is interesting. We have: documentation, architecture-specific code across ARM64, PowerPC, RISC-V, s390, SPARC, and x86, a few drivers (gpu, hsi, infiniband, SCSI, USB, video, and fbdev), plus core memory management and file system code (coredump, fuse, hugetlbfs, proc, includes for various MM and related structures, bpf arena, events, scheduler, folio, gup, hmm, huge_memory, hugetlb, ksm, madvise, memory, mempolicy, migrate_device, mlock, mmap, mmu_gather, mprotect, mremap, page_vma_mapped, pagewalk, rmap, swapfile, userfaultfd, util, vma, and vmscan, and internal structures, selinux, and sound). This reinforces the wide reach of the changes. Based on the files affected, I should be focused on the following configurations: `CHR_DEV_SG`, `SCSI`, `USB_MON`, `USB`, `SND_PCM`, `HUGETLB_PAGE`, and `USERFAULTFD`. Let's drill down and look at the more specific FocusSymbols: `map_discontig_kernel_pages_prepare`, `map_discontig_kernel_pages_complete`, `mmap_prepare_validate`, `mmap_hook_validate`, `mmap_action_prepare`, `mmap_action_complete`, `sg_mmap_prepare`, `mon_bin_mmap_prepare`, and `snd_pcm_mmap_status`. Are there any ubiquitous symbols here? No, none of these are memory allocators or deallocators, or page fault handlers, or other core primitives. My understanding is that `mmap_action_prepare` and `mmap_action_complete` are linked to the VMA descriptor's "action" for specific drivers, such as `mmap_prepare`. The `map_discontig_kernel_pages_*` symbols are new and specifically handle this new discontiguous page mapping feature. `mmap_prepare_validate` and `mmap_hook_validate` are the new validation functions. `sg_mmap_prepare`, `mon_bin_mmap_prepare`, and `snd_pcm_mmap_status` are the entry points to the specific drivers. This gives a good, clean list to focus on. To summarise clearly, the patch introduces new MM infrastructure for mapping discontiguous kernel pages via `mmap_prepare`. It adds validation hooks via `mmap_prepare_validate` and `mmap_hook_validate`. It also refactors the legacy heuristics from the VMA flags and replaces them with new VMA classifications, such as `vma_is_kernel_owned`, `vma_is_fixed_mapping`, `vma_can_merge`, and `vma_is_persistent`. Additionally, there has been a refactoring of mlock state tracking, and drivers that implement mmap handlers, like SCSI sg, USB mon, and ALSA pcm, are converted to use the new interfaces. This is a good candidate for fuzzing. The patch makes significant changes to core memory management, specifically VMA merging and validation rules, how mlock is accounted for, and driver mmap handlers. This raises the possibility of race conditions, assertion failures from validation (like `VM_WARN_ON_ONCE`), and potential memory management issues. Everything looks good. I'm ready to `set-results`. |
| 44/2 | 2026/09/14 19:12 | tool | set-results |
0mArgs:{
"EnableConfigs": [
"CHR_DEV_SG",
"SCSI",
"USB_MON",
"USB",
"SND_PCM",
"HUGETLB_PAGE",
"USERFAULTFD"
],
"FocusSymbols": [
"map_discontig_kernel_pages_prepare",
"map_discontig_kernel_pages_complete",
"mmap_prepare_validate",
"mmap_hook_validate",
"mmap_action_prepare",
"mmap_action_complete",
"sg_mmap_prepare",
"mon_bin_mmap_prepare",
"snd_pcm_mmap_status"
],
"Reasoning": "The patch modifies core memory management (mm) and reachable drivers by introducing a new discontiguous kernel page mapping mechanism (map_discontig_kernel_pages_*), adding validation hooks for mmap and mmap_prepare (mmap_prepare_validate, mmap_hook_validate), refactoring VMA classifications across mm (vma_is_kernel_owned, vma_is_fixed_mapping, vma_can_merge, vma_is_persistent), altering mlock tracking logic during page table walks, and migrating reachable drivers (drivers/scsi/sg.c, drivers/usb/mon/mon_bin.c, sound/core/pcm_native.c) to these new APIs. These changes alter VMA invariant verification and mapping setup in reachable core code, warranting fuzzing for assertion failures and subtle mm regressions.",
"WorthFuzzing": true
}Results: {
"EnableConfigs": [
"CHR_DEV_SG",
"SCSI",
"USB_MON",
"USB",
"SND_PCM",
"HUGETLB_PAGE",
"USERFAULTFD"
],
"FocusSymbols": [
"map_discontig_kernel_pages_prepare",
"map_discontig_kernel_pages_complete",
"mmap_prepare_validate",
"mmap_hook_validate",
"mmap_action_prepare",
"mmap_action_complete",
"sg_mmap_prepare",
"mon_bin_mmap_prepare",
"snd_pcm_mmap_status"
],
"Reasoning": "The patch modifies core memory management (mm) and reachable drivers by introducing a new discontiguous kernel page mapping mechanism (map_discontig_kernel_pages_*), adding validation hooks for mmap and mmap_prepare (mmap_prepare_validate, mmap_hook_validate), refactoring VMA classifications across mm (vma_is_kernel_owned, vma_is_fixed_mapping, vma_can_merge, vma_is_persistent), altering mlock tracking logic during page table walks, and migrating reachable drivers (drivers/scsi/sg.c, drivers/usb/mon/mon_bin.c, sound/core/pcm_native.c) to these new APIs. These changes alter VMA invariant verification and mapping setup in reachable core code, warranting fuzzing for assertion failures and subtle mm regressions.",
"WorthFuzzing": true
} |
| 45/1 | 2026/09/14 19:12 | action | If |
2mArgs:{
"WorthFuzzing": true
}Results: null |
| 46/2 | 2026/09/14 19:12 | agent | kmsan-evaluator |
2mModel:core Args: null Results: {
"KMSANReasoning": "The patch series refactors VMA flag helpers (e.g. replacing is_vm_hugetlb_page() with vma_is_hugetlb(), introducing vma_is_kernel_owned(), vma_can_merge(), vma_is_persistent(), etc.), adds descriptor-based mmap validation (mmap_prepare_validate(), mmap_hook_validate()), introduces a new discontiguous kernel page mapping mechanism (struct discontig_kernel_page_state, struct discontig_kernel_page_ops), and migrates several drivers (cmt_speech, hfi1, sg, usb mon, snd_pcm) to the new mmap / discontig mapping infrastructure.\n\nKASAN vs KMSAN applicability:\n- KMSAN detects reads of uninitialized memory (stack, heap, or page allocations) and info-leaks to user space. In this patch series, no new kernel-to-user copy interfaces (copy_to_user, put_user, netlink, ioctls) are introduced or modified. All newly introduced structures, such as struct discontig_kernel_page_state in map_discontig_kernel_pages_complete() and prev_desc in __compat_vma_mmap(), are fully zero-initialized via designated initializers or memset() prior to use. The mapped pages originate from existing driver buffers or zero-allocated pages (e.g. __GFP_ZERO in sg).\n- Potential risks in these changes relate to VMA lifecycle, reference counting (fput/get_file), page table manipulations, locking, or out-of-bounds page array indexing. These types of bugs are covered by standard KASAN, page table debugging, and LOCKDEP builds.\n\nTherefore, a dedicated KMSAN fuzzing session is not warranted.",
"NeedsKMSAN": false
}Instruction: You are an expert Linux kernel security engineer specializing in kernel memory
error detectors (KASAN and KMSAN). Your job is to review the provided patch series and
determine if the code changes justify spawning a dedicated KMSAN (KernelMemorySanitizer)
fuzzing session in addition to standard KASAN fuzzing.
CRITICAL DISTINCTION BETWEEN KASAN AND KMSAN:
- Standard KASAN kernel builds (upstream-apparmor-kasan.config) already enable
a comprehensive suite of debugging tools and sanitizers, including KASAN
(out-of-bounds accesses, use-after-free, double free, invalid free), LOCKDEP
(locking bugs and deadlocks), UB-sanitizers, and memory corruption checks.
- KMSAN (KernelMemorySanitizer) detects reads of UNINITIALIZED memory (stack, heap,
or page allocations) and kernel-to-user memory info-leaks.
Rule: THERE IS NO SENSE IN RUNNING A KMSAN SESSION IF A BUG CAN BE CAUGHT BY KASAN,
LOCKDEP, OR OTHER STANDARD BUG DETECTORS.
A dedicated KMSAN fuzzing session incurs significant resource costs. You must ONLY
set NeedsKMSAN=true if the code changes introduce or expose UNINITIALIZED MEMORY risks
that are detected ONLY by KMSAN.
Look holistically at the patch series and surrounding code. Even if no direct
uninitialized field accesses or new buffer allocations are added in the diff itself,
a patch may alter control flow, bounds checking, or data length calculations in ways
that change how the rest of the code operates on existing buffers (e.g. allowing
uninitialized stack/heap memory to be read, copied to user space, or used in control
flow). Do not hesitate to use your code access tools to inspect the surrounding code,
called functions, and callers.
Set NeedsKMSAN=true ONLY IF the patch introduces or modifies:
1. Kernel structures sent to user space (via copy_to_user, put_user, netlink skb
attributes, ioctl output arguments, socket options, or BPF buffers) where fields
or structure padding might not be fully initialized/zeroed.
2. Conditional logic or branching that depends on potentially uninitialized variables
or struct fields.
3. Allocation or initialization of complex data structures where uninitialized fields
could be read later in reachable code paths.
4. Bounds checks, lengths, or logic in a way that allows surrounding code to access
uninitialized bytes of existing buffers.
Set NeedsKMSAN=false IF:
- The code changes primarily risk out-of-bounds access, array overflows, NULL pointer
dereferences, locking deadlocks, or use-after-free bugs (these are already caught
by KASAN, LOCKDEP, or standard bug detectors).
- All stack/heap structures touched or introduced by the patch are fully zeroed
or initialized (e.g. using = {0}, memset, kzalloc) before being read or copied.
- The patch does not introduce any risk of uninitialized memory usage or info-leaks.
Use your code access tools to inspect the surrounding code if necessary, then provide
detailed KMSANReasoning contrasting KASAN vs KMSAN applicability for this patch.
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Target architecture: amd64
For your convenience, here is the diff of the changes:
commit dfb57cd7fb217657931dcb005b6452b86d63b7db
Author: syz-cluster <triage@syzkaller.com>
Date: Mon Sep 14 19:11:06 2026 +0000
syz-cluster: applied patch under review
diff --git a/Documentation/filesystems/mmap_prepare.rst b/Documentation/filesystems/mmap_prepare.rst
index 82c99c95ad854..a476e1006bf12 100644
--- a/Documentation/filesystems/mmap_prepare.rst
+++ b/Documentation/filesystems/mmap_prepare.rst
@@ -164,5 +164,86 @@ pointer. These are:
sufficient entries in the page array to cover the entire range of the
described VMA.
+* mmap_action_map_discontig_kernel_pages() - Maps a discontiguous range of
+ `struct page` pointers over the VMA. They must span from the start of the VMA,
+ but may terminate prior to the end (leaving the remainder unmapped).
+
**NOTE:** The ``action`` field should never normally be manipulated directly,
rather you ought to use one of these helpers.
+
+Discontiguous Actions
+=====================
+
+Some actions can be performed across discontiguous ranges.
+
+Map kernel pages
+----------------
+
+To map kernel pages discontiguously, you must provide hooks using ``struct
+discontig_kernel_page_ops``:
+
+.. code-block:: C
+
+ struct discontig_kernel_page_ops {
+ int (*init)(void *vm_private_data, void **private);
+ int (*get)(struct discontig_kernel_page_state *state);
+ };
+
+The ``init`` hook is optional and allows state to be established before the
+operation starts, for instance taking a reference count. Nothing is invoked
+after the operation, so ``init`` must not leave locks held, and state that must
+be released once the mapping goes away should be released in
+``vm_ops->close``.
+
+The ``init`` hook, if provided, is invoked prior to the operation starting. It
+may update what is pointed to by ``vm_private_data`` and/or ``private``. If an
+error is returned, then the operation is aborted. The ``private`` field can be
+reassigned.
+
+**NOTE:** The operation may sleep between invocations of ``get``, so locks
+needed to stabilise state must be taken and released within each hook.
+
+The ``get`` handler is the key means through which the operation is
+executed. The current state of the operation is provided through ``struct
+discontig_kernel_page_state``:
+
+.. code-block:: C
+
+ struct discontig_kernel_page_state {
+ /* Map state. */
+ unsigned long start; /* Start address of VMA. */
+ unsigned long end; /* End address of VMA. */
+ unsigned long addr; /* The current address to be mapped. */
+ pgoff_t pgoff; /* The current pgoff to be mapped. */
+ unsigned long nr_pages_mapped; /* The number of pages mapped. */
+ unsigned long nr_pages_remain; /* The number of pages remaining. */
+
+ /* User-defined state. */
+ void *vm_private_data; /* VMA private data. */
+ void *private; /* Mapping private data. */
+
+ /* Users should not touch these, use discontig_kernel_map_*() helpers. */
+ ... internal fields ...
+ };
+
+With ``private`` being an additional user-controllable state variable,
+initialised via ``mmap_action_map_discontig_kernel_pages()``, and
+``vm_private_data`` being equal to the ``desc->private_data`` field set in
+the ``mmap_prepare()`` hook.
+
+In the ``get`` hook, the user must choose how to map kernel pages:
+
+* ``discontig_kernel_map_abort()`` - Call this to abort the operation, whatever
+ has been mapped so far will be retained, the rest of the mapping will SIGBUS
+ if accessed.
+* ``discontig_kernel_map_page()`` - Maps a single page, correctly handling
+ compound pages (if the compound page is bigger than the remaining pages in the
+ VMA, then only those pages that fit will be mapped). For a compound page, the
+ head page must be passed.
+* ``discontig_kernel_map_page_range()`` - Map an array of pages of a specified
+ size. Note that if the number of pages specified exceeds the VMA size then an
+ error will arise.
+
+If an error arises after ``init`` succeeded, the core unmaps the VMA, invoking
+``vm_ops->close`` if set, which is therefore the place to release any state
+that ``init`` established.
diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c
index 9ba86450fe4af..3c1240ffc38df 100644
--- a/arch/arm64/kvm/mmu.c
+++ b/arch/arm64/kvm/mmu.c
@@ -1463,14 +1463,12 @@ static int get_vma_page_shift(struct vm_area_struct *vma, unsigned long hva)
{
unsigned long pa;
- if (is_vm_hugetlb_page(vma) && !(vma->vm_flags & VM_PFNMAP))
+ if (vma_is_hugetlb(vma))
return huge_page_shift(hstate_vma(vma));
if (!(vma->vm_flags & VM_PFNMAP))
return PAGE_SHIFT;
- VM_BUG_ON(is_vm_hugetlb_page(vma));
-
pa = (vma->vm_pgoff << PAGE_SHIFT) + (hva - vma->vm_start);
#ifndef __PAGETABLE_PMD_FOLDED
diff --git a/arch/powerpc/mm/book3s64/radix_tlb.c b/arch/powerpc/mm/book3s64/radix_tlb.c
index 7de5760164a90..b4603a98224b3 100644
--- a/arch/powerpc/mm/book3s64/radix_tlb.c
+++ b/arch/powerpc/mm/book3s64/radix_tlb.c
@@ -627,7 +627,7 @@ void radix__local_flush_tlb_page(struct vm_area_struct *vma, unsigned long vmadd
{
#ifdef CONFIG_HUGETLB_PAGE
/* need the return fix for nohash.c */
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
return radix__local_flush_hugetlb_page(vma, vmaddr);
#endif
radix__local_flush_tlb_page_psize(vma->vm_mm, vmaddr, mmu_virtual_psize);
@@ -945,7 +945,7 @@ void radix__flush_tlb_page_psize(struct mm_struct *mm, unsigned long vmaddr,
void radix__flush_tlb_page(struct vm_area_struct *vma, unsigned long vmaddr)
{
#ifdef CONFIG_HUGETLB_PAGE
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
return radix__flush_hugetlb_page(vma, vmaddr);
#endif
radix__flush_tlb_page_psize(vma->vm_mm, vmaddr, mmu_virtual_psize);
@@ -1113,7 +1113,7 @@ void radix__flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
{
#ifdef CONFIG_HUGETLB_PAGE
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
return radix__flush_hugetlb_tlb_range(vma, start, end);
#endif
diff --git a/arch/powerpc/mm/nohash/e500_hugetlbpage.c b/arch/powerpc/mm/nohash/e500_hugetlbpage.c
index a134d28a0e4d3..b87623f04be53 100644
--- a/arch/powerpc/mm/nohash/e500_hugetlbpage.c
+++ b/arch/powerpc/mm/nohash/e500_hugetlbpage.c
@@ -180,7 +180,7 @@ book3e_hugetlb_preload(struct vm_area_struct *vma, unsigned long ea, pte_t pte)
*/
void __update_mmu_cache(struct vm_area_struct *vma, unsigned long address, pte_t *ptep)
{
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
book3e_hugetlb_preload(vma, address, *ptep);
}
diff --git a/arch/powerpc/mm/nohash/tlb.c b/arch/powerpc/mm/nohash/tlb.c
index 0a650742f3a00..07a2db16c2b15 100644
--- a/arch/powerpc/mm/nohash/tlb.c
+++ b/arch/powerpc/mm/nohash/tlb.c
@@ -278,7 +278,7 @@ void __flush_tlb_page(struct mm_struct *mm, unsigned long vmaddr,
void flush_tlb_page(struct vm_area_struct *vma, unsigned long vmaddr)
{
#ifdef CONFIG_HUGETLB_PAGE
- if (vma && is_vm_hugetlb_page(vma))
+ if (vma && vma_is_hugetlb(vma))
flush_hugetlb_page(vma, vmaddr);
#endif
diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
index 6035b5ec95039..5c5c77f98bf0f 100644
--- a/arch/riscv/kvm/mmu.c
+++ b/arch/riscv/kvm/mmu.c
@@ -664,7 +664,7 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
return -EFAULT;
}
- is_hugetlb = is_vm_hugetlb_page(vma);
+ is_hugetlb = vma_is_hugetlb(vma);
if (is_hugetlb)
vma_pageshift = huge_page_shift(hstate_vma(vma));
else
diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c
index 962db300a1665..a74a7d5258aa1 100644
--- a/arch/riscv/mm/tlbflush.c
+++ b/arch/riscv/mm/tlbflush.c
@@ -149,7 +149,7 @@ void flush_tlb_range(struct vm_area_struct *vma, unsigned long start,
{
unsigned long stride_size;
- if (!is_vm_hugetlb_page(vma)) {
+ if (!vma_is_hugetlb(vma)) {
stride_size = PAGE_SIZE;
} else {
stride_size = huge_page_size(hstate_vma(vma));
diff --git a/arch/s390/mm/gmap_helpers.c b/arch/s390/mm/gmap_helpers.c
index ff63ffb1dbd29..3f6783b93e679 100644
--- a/arch/s390/mm/gmap_helpers.c
+++ b/arch/s390/mm/gmap_helpers.c
@@ -102,7 +102,7 @@ __context_unsafe(/* pte_unmap_unlock() not instrumented */)
/* Find the vm address for the guest address */
vma = vma_lookup(mm, vmaddr);
- if (!vma || is_vm_hugetlb_page(vma))
+ if (!vma || vma_is_hugetlb(vma))
return;
/* Get pointer to the page table entry */
@@ -139,7 +139,7 @@ void gmap_helper_discard(struct mm_struct *mm, unsigned long vmaddr, unsigned lo
vma = find_vma_intersection(mm, vmaddr, end);
if (!vma)
return;
- if (!is_vm_hugetlb_page(vma))
+ if (!vma_is_hugetlb(vma))
zap_vma_range(vma, vmaddr, min(end, vma->vm_end) - vmaddr);
vmaddr = vma->vm_end;
}
@@ -247,7 +247,7 @@ static int __gmap_helper_unshare_zeropages(struct mm_struct *mm)
* proof to catch unexpected zeropages in other mappings and
* fail.
*/
- if ((vma->vm_flags & VM_PFNMAP) || is_vm_hugetlb_page(vma))
+ if ((vma->vm_flags & VM_PFNMAP) || vma_is_hugetlb(vma))
continue;
addr = vma->vm_start;
diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c
index 103db4683b165..9bbccb5d23a8f 100644
--- a/arch/sparc/mm/init_64.c
+++ b/arch/sparc/mm/init_64.c
@@ -413,7 +413,7 @@ void update_mmu_cache_range(struct vm_fault *vmf, struct vm_area_struct *vma,
if (mm->context.hugetlb_pte_count || mm->context.thp_pte_count) {
unsigned long hugepage_size = PAGE_SIZE;
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
hugepage_size = huge_page_size(hstate_vma(vma));
if (hugepage_size >= PUD_SIZE) {
diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
index 65a2de82ecd29..0f60c0d076b62 100644
--- a/arch/x86/kernel/uprobes.c
+++ b/arch/x86/kernel/uprobes.c
@@ -715,7 +715,7 @@ static struct vm_area_struct *get_uprobe_trampoline(struct mm_struct *mm, unsign
*new_mapping = true;
return _install_special_mapping(mm, vaddr, PAGE_SIZE,
- VM_READ|VM_EXEC|VM_MAYEXEC|VM_MAYREAD|VM_IO,
+ VM_READ|VM_EXEC|VM_MAYEXEC|VM_MAYREAD|VM_MIXEDMAP,
&tramp_mapping);
}
diff --git a/drivers/gpu/drm/drm_gpusvm.c b/drivers/gpu/drm/drm_gpusvm.c
index a93eee7ddb9e9..fab34fea99c2f 100644
--- a/drivers/gpu/drm/drm_gpusvm.c
+++ b/drivers/gpu/drm/drm_gpusvm.c
@@ -9,9 +9,9 @@
#include <linux/dma-mapping.h>
#include <linux/export.h>
#include <linux/hmm.h>
-#include <linux/hugetlb_inline.h>
#include <linux/memremap.h>
#include <linux/mm_types.h>
+#include <linux/mm.h>
#include <linux/slab.h>
#include <drm/drm_device.h>
@@ -1141,8 +1141,7 @@ drm_gpusvm_range_find_or_insert(struct drm_gpusvm *gpusvm,
* limitations. If/when migrate_vma_* add more support, this logic will
* have to change.
*/
- migrate_devmem = ctx->devmem_possible &&
- vma_is_anonymous(vas) && !is_vm_hugetlb_page(vas);
+ migrate_devmem = ctx->devmem_possible && vma_is_anonymous(vas);
chunk_size = drm_gpusvm_range_chunk_size(gpusvm, notifier, vas,
fault_addr, gpuva_start,
diff --git a/drivers/hsi/clients/cmt_speech.c b/drivers/hsi/clients/cmt_speech.c
index 7226677ebde7a..801697b74d4f8 100644
--- a/drivers/hsi/clients/cmt_speech.c
+++ b/drivers/hsi/clients/cmt_speech.c
@@ -1084,22 +1084,6 @@ static void cs_hsi_stop(struct cs_hsi_iface *hi)
kfree(hi);
}
-static vm_fault_t cs_char_vma_fault(struct vm_fault *vmf)
-{
- struct cs_char *csdata = vmf->vma->vm_private_data;
- struct page *page;
-
- page = virt_to_page((void *)csdata->mmap_base);
- get_page(page);
- vmf->page = page;
-
- return 0;
-}
-
-static const struct vm_operations_struct cs_char_vm_ops = {
- .fault = cs_char_vma_fault,
-};
-
static int cs_char_fasync(int fd, struct file *file, int on)
{
struct cs_char *csdata = file->private_data;
@@ -1256,18 +1240,19 @@ static long cs_char_ioctl(struct file *file, unsigned int cmd,
return r;
}
-static int cs_char_mmap(struct file *file, struct vm_area_struct *vma)
+static int cs_char_mmap_prepare(struct vm_area_desc *desc)
{
- if (vma->vm_end < vma->vm_start)
- return -EINVAL;
+ struct file *file = desc->file;
+ struct cs_char *csdata = file->private_data;
+ struct page **pages = (struct page **)&desc->private_data;
- if (vma_pages(vma) != 1)
+ if (vma_desc_pages(desc) != 1)
return -EINVAL;
- vm_flags_set(vma, VM_IO | VM_DONTDUMP | VM_DONTEXPAND);
- vma->vm_ops = &cs_char_vm_ops;
- vma->vm_private_data = file->private_data;
+ vma_desc_set_flags(desc, VMA_DONTDUMP_BIT, VMA_DONTEXPAND_BIT);
+ *pages = virt_to_page((void *)csdata->mmap_base);
+ mmap_action_map_kernel_pages_full(desc, pages);
return 0;
}
@@ -1353,7 +1338,7 @@ static const struct file_operations cs_char_fops = {
.write = cs_char_write,
.poll = cs_char_poll,
.unlocked_ioctl = cs_char_ioctl,
- .mmap = cs_char_mmap,
+ .mmap_prepare = cs_char_mmap_prepare,
.open = cs_char_open,
.release = cs_char_release,
.fasync = cs_char_fasync,
diff --git a/drivers/infiniband/hw/hfi1/file_ops.c b/drivers/infiniband/hw/hfi1/file_ops.c
index dc548e6802e24..b02d1f1dbb27b 100644
--- a/drivers/infiniband/hw/hfi1/file_ops.c
+++ b/drivers/infiniband/hw/hfi1/file_ops.c
@@ -70,7 +70,6 @@ static int set_ctxt_pkey(struct hfi1_ctxtdata *uctxt, unsigned long arg);
static int ctxt_reset(struct hfi1_ctxtdata *uctxt);
static int manage_rcvq(struct hfi1_ctxtdata *uctxt, u16 subctxt,
unsigned long arg);
-static vm_fault_t vma_fault(struct vm_fault *vmf);
static long hfi1_file_ioctl(struct file *fp, unsigned int cmd,
unsigned long arg);
@@ -85,10 +84,6 @@ static const struct file_operations hfi1_file_ops = {
.llseek = noop_llseek,
};
-static const struct vm_operations_struct vm_ops = {
- .fault = vma_fault,
-};
-
/*
* Types of memories mapped into user processes' space
*/
@@ -304,13 +299,13 @@ static ssize_t hfi1_write_iter(struct kiocb *kiocb, struct iov_iter *from)
return reqs;
}
-static inline void mmap_cdbg(u16 ctxt, u8 subctxt, u8 type, u8 mapio, u8 vmf,
+static inline void mmap_cdbg(u16 ctxt, u8 subctxt, u8 type, u8 mapio, u8 is_vmalloc,
u64 memaddr, void *memvirt, dma_addr_t memdma,
ssize_t memlen, struct vm_area_struct *vma)
{
hfi1_cdbg(PROC,
- "%u:%u type:%u io/vf/dma:%d/%d/%d, addr:0x%llx, len:%lu(%lu), flags:0x%lx",
- ctxt, subctxt, type, mapio, vmf, !!memdma,
+ "%u:%u type:%u io/vmalloc/dma:%d/%d/%d, addr:0x%llx, len:%lu(%lu), flags:0x%lx",
+ ctxt, subctxt, type, mapio, is_vmalloc, !!memdma,
memaddr ?: (u64)memvirt, memlen,
vma->vm_end - vma->vm_start, vma->vm_flags);
}
@@ -325,7 +320,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
memaddr = 0;
void *memvirt = NULL;
dma_addr_t memdma = 0;
- u8 subctxt, mapio = 0, vmf = 0, type;
+ u8 subctxt, mapio = 0, is_vmalloc = 0, type;
ssize_t memlen = 0;
int ret = 0;
u16 ctxt;
@@ -347,7 +342,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
/*
* vm_pgoff is used as a buffer selector cookie. Always mmap from
* the beginning.
- */
+ */
vma->vm_pgoff = 0;
flags = vma->vm_flags;
@@ -366,7 +361,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
*/
memlen = PAGE_ALIGN(uctxt->sc->credits * PIO_BLOCK_SIZE);
flags &= ~VM_MAYREAD;
- flags |= VM_DONTCOPY | VM_DONTEXPAND;
+ flags |= VM_DONTCOPY;
vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
mapio = 1;
break;
@@ -401,6 +396,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
memlen = rcvhdrq_size(uctxt);
memvirt = uctxt->rcvhdrq;
memdma = uctxt->rcvhdrq_dma;
+ flags |= VM_DONTEXPAND;
break;
case RCV_EGRBUF: {
unsigned long vm_start_save;
@@ -422,7 +418,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
ret = -EPERM;
goto done;
}
- vm_flags_clear(vma, VM_MAYWRITE);
+ vm_flags_mod(vma, VM_DONTEXPAND, VM_MAYWRITE);
/*
* Mmap multiple separate allocations into a single vma. From
* here, dma_mmap_coherent() calls dma_direct_mmap(), which
@@ -438,7 +434,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
memvirt = uctxt->egrbufs.buffers[i].addr;
memdma = uctxt->egrbufs.buffers[i].dma;
vma->vm_end += memlen;
- mmap_cdbg(ctxt, subctxt, type, mapio, vmf, memaddr,
+ mmap_cdbg(ctxt, subctxt, type, mapio, is_vmalloc, memaddr,
memvirt, memdma, memlen, vma);
ret = dma_mmap_coherent(&dd->pcidev->dev, vma,
memvirt, memdma, memlen);
@@ -467,7 +463,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
* user registers.
*/
memlen = PAGE_SIZE;
- flags |= VM_DONTCOPY | VM_DONTEXPAND;
+ flags |= VM_DONTCOPY;
vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
mapio = 1;
break;
@@ -476,15 +472,10 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
* Use the page where this context's flags are. User level
* knows where it's own bitmap is within the page.
*/
- memaddr = (unsigned long)
- (dd->events + uctxt_offset(uctxt)) & PAGE_MASK;
+ memvirt = dd->events + uctxt_offset(uctxt);
+ memvirt = (void *)(((uintptr_t)memvirt) & PAGE_MASK);
memlen = PAGE_SIZE;
- /*
- * v3.7 removes VM_RESERVED but the effect is kept by
- * using VM_IO.
- */
- flags |= VM_IO | VM_DONTEXPAND;
- vmf = 1;
+ is_vmalloc = 1;
break;
case STATUS:
if (flags & VM_WRITE) {
@@ -493,7 +484,6 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
}
memaddr = kvirt_to_phys((void *)dd->status);
memlen = PAGE_SIZE;
- flags |= VM_IO | VM_DONTEXPAND;
break;
case RTAIL:
if (!HFI1_CAP_IS_USET(DMA_RTAIL)) {
@@ -512,25 +502,23 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
memvirt = (void *)hfi1_rcvhdrtail_kvaddr(uctxt);
memdma = uctxt->rcvhdrqtailaddr_dma;
flags &= ~VM_MAYWRITE;
+ flags |= VM_DONTEXPAND;
break;
case SUBCTXT_UREGS:
- memaddr = (u64)uctxt->subctxt_uregbase;
+ memvirt = uctxt->subctxt_uregbase;
memlen = PAGE_SIZE;
- flags |= VM_IO | VM_DONTEXPAND;
- vmf = 1;
+ is_vmalloc = 1;
break;
case SUBCTXT_RCV_HDRQ:
- memaddr = (u64)uctxt->subctxt_rcvhdr_base;
+ memvirt = uctxt->subctxt_rcvhdr_base;
memlen = rcvhdrq_size(uctxt) * uctxt->subctxt_cnt;
- flags |= VM_IO | VM_DONTEXPAND;
- vmf = 1;
+ is_vmalloc = 1;
break;
case SUBCTXT_EGRBUF:
- memaddr = (u64)uctxt->subctxt_rcvegrbuf;
+ memvirt = uctxt->subctxt_rcvegrbuf;
memlen = uctxt->egrbufs.size * uctxt->subctxt_cnt;
- flags |= VM_IO | VM_DONTEXPAND;
flags &= ~VM_MAYWRITE;
- vmf = 1;
+ is_vmalloc = 1;
break;
case SDMA_COMP: {
struct hfi1_user_sdma_comp_q *cq = fd->cq;
@@ -539,10 +527,9 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
ret = -EFAULT;
goto done;
}
- memaddr = (u64)cq->comps;
+ memvirt = cq->comps;
memlen = PAGE_ALIGN(sizeof(*cq->comps) * cq->nentries);
- flags |= VM_IO | VM_DONTEXPAND;
- vmf = 1;
+ is_vmalloc = 1;
break;
}
default:
@@ -559,12 +546,10 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
}
vm_flags_reset(vma, flags);
- mmap_cdbg(ctxt, subctxt, type, mapio, vmf, memaddr, memvirt, memdma,
+ mmap_cdbg(ctxt, subctxt, type, mapio, is_vmalloc, memaddr, memvirt, memdma,
memlen, vma);
- if (vmf) {
- vma->vm_pgoff = PFN_DOWN(memaddr);
- vma->vm_ops = &vm_ops;
- ret = 0;
+ if (is_vmalloc) {
+ ret = remap_vmalloc_range(vma, memvirt, 0);
} else if (memdma) {
ret = dma_mmap_coherent(&dd->pcidev->dev, vma,
memvirt, memdma, memlen);
@@ -588,24 +573,6 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma)
return ret;
}
-/*
- * Local (non-chip) user memory is not mapped right away but as it is
- * accessed by the user-level code.
- */
-static vm_fault_t vma_fault(struct vm_fault *vmf)
-{
- struct page *page;
-
- page = vmalloc_to_page((void *)(vmf->pgoff << PAGE_SHIFT));
- if (!page)
- return VM_FAULT_SIGBUS;
-
- get_page(page);
- vmf->page = page;
-
- return 0;
-}
-
static __poll_t hfi1_poll(struct file *fp, struct poll_table_struct *pt)
{
struct hfi1_ctxtdata *uctxt;
diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
index 5408f002e6c01..3f9e08725602c 100644
--- a/drivers/scsi/sg.c
+++ b/drivers/scsi/sg.c
@@ -1212,85 +1212,72 @@ sg_fasync(int fd, struct file *filp, int mode)
return fasync_helper(fd, filp, mode, &sfp->async_qp);
}
-static vm_fault_t
-sg_vma_fault(struct vm_fault *vmf)
+static int sg_discontig_init(void *vm_private_data, void **private)
{
- struct vm_area_struct *vma = vmf->vma;
- Sg_fd *sfp;
- unsigned long offset, len, sa;
- Sg_scatter_hold *rsv_schp;
- int k, length;
-
- if ((NULL == vma) || (!(sfp = (Sg_fd *) vma->vm_private_data)))
- return VM_FAULT_SIGBUS;
- rsv_schp = &sfp->reserve;
- offset = vmf->pgoff << PAGE_SHIFT;
- if (offset >= rsv_schp->bufflen)
- return VM_FAULT_SIGBUS;
- SCSI_LOG_TIMEOUT(3, sg_printk(KERN_INFO, sfp->parentdp,
- "sg_vma_fault: offset=%lu, scatg=%d\n",
- offset, rsv_schp->k_use_sg));
- sa = vma->vm_start;
- length = 1 << (PAGE_SHIFT + rsv_schp->page_order);
- for (k = 0; k < rsv_schp->k_use_sg && sa < vma->vm_end; k++) {
- len = vma->vm_end - sa;
- len = (len < length) ? len : length;
- if (offset < len) {
- struct page *page = rsv_schp->pages[k] + (offset >> PAGE_SHIFT);
- get_page(page); /* increment page count */
- vmf->page = page;
- return 0; /* success */
- }
- sa += len;
- offset -= len;
+ const unsigned long req_sz = (unsigned long)*private;
+ Sg_fd *sfp = vm_private_data;
+ Sg_scatter_hold *rsv_schp = &sfp->reserve;
+ int err = 0;
+
+ mutex_lock(&sfp->f_mutex);
+ if (req_sz > rsv_schp->bufflen) {
+ err = -ENOMEM; /* cannot map more than reserved buffer */
+ goto out;
+ }
+ sfp->mmap_called = 1; /* Prevents changes to buffer size. */
+out:
+ mutex_unlock(&sfp->f_mutex);
+ return err;
+}
+
+static int
+sg_discontig_get(struct discontig_kernel_page_state *state)
+{
+ Sg_fd *sfp = state->vm_private_data;
+ Sg_scatter_hold *rsv_schp = &sfp->reserve;
+ const unsigned int order = rsv_schp->page_order;
+ const pgoff_t nr_pages = state->nr_pages_mapped;
+
+ if (nr_pages >= (rsv_schp->bufflen >> PAGE_SHIFT)) {
+ discontig_kernel_map_abort(state);
+ return 0;
}
- return VM_FAULT_SIGBUS;
+ SCSI_LOG_TIMEOUT(3, sg_printk(KERN_INFO, sfp->parentdp,
+ "%s: offset=%lu, scatg=%d\n", __func__,
+ nr_pages << PAGE_SHIFT, rsv_schp->k_use_sg));
+
+ discontig_kernel_map_page(state, rsv_schp->pages[nr_pages >> order]);
+ return 0;
}
-static const struct vm_operations_struct sg_mmap_vm_ops = {
- .fault = sg_vma_fault,
+static const struct discontig_kernel_page_ops sg_discontig_ops = {
+ .init = sg_discontig_init,
+ .get = sg_discontig_get,
};
static int
-sg_mmap(struct file *filp, struct vm_area_struct *vma)
+sg_mmap_prepare(struct vm_area_desc *desc)
{
- Sg_fd *sfp;
- unsigned long req_sz, len, sa;
- Sg_scatter_hold *rsv_schp;
- int k, length;
- int ret = 0;
+ Sg_fd *sfp = desc->file->private_data;
+ const unsigned long req_sz = vma_desc_size(desc);
- if ((!filp) || (!vma) || (!(sfp = (Sg_fd *) filp->private_data)))
+ if (!sfp)
return -ENXIO;
- req_sz = vma->vm_end - vma->vm_start;
+
SCSI_LOG_TIMEOUT(3, sg_printk(KERN_INFO, sfp->parentdp,
"sg_mmap starting, vm_start=%p, len=%d\n",
- (void *) vma->vm_start, (int) req_sz));
- if (vma->vm_pgoff)
+ (void *) desc->start, (int) req_sz));
+
+ if (desc->pgoff)
return -EINVAL; /* want no offset */
- rsv_schp = &sfp->reserve;
- mutex_lock(&sfp->f_mutex);
- if (req_sz > rsv_schp->bufflen) {
- ret = -ENOMEM; /* cannot map more than reserved buffer */
- goto out;
- }
- sa = vma->vm_start;
- length = 1 << (PAGE_SHIFT + rsv_schp->page_order);
- for (k = 0; k < rsv_schp->k_use_sg && sa < vma->vm_end; k++) {
- len = vma->vm_end - sa;
- len = (len < length) ? len : length;
- sa += len;
- }
+ vma_desc_set_flags(desc, VMA_DONTEXPAND_BIT, VMA_DONTDUMP_BIT);
+ desc->private_data = sfp;
- sfp->mmap_called = 1;
- vm_flags_set(vma, VM_IO | VM_DONTEXPAND | VM_DONTDUMP);
- vma->vm_private_data = sfp;
- vma->vm_ops = &sg_mmap_vm_ops;
-out:
- mutex_unlock(&sfp->f_mutex);
- return ret;
+ mmap_action_map_discontig_kernel_pages(desc, (void *)req_sz,
+ &sg_discontig_ops);
+ return 0;
}
static void
@@ -1415,7 +1402,7 @@ static const struct file_operations sg_fops = {
.unlocked_ioctl = sg_ioctl,
.compat_ioctl = compat_ptr_ioctl,
.open = sg_open,
- .mmap = sg_mmap,
+ .mmap_prepare = sg_mmap_prepare,
.release = sg_release,
.fasync = sg_fasync,
};
diff --git a/drivers/usb/mon/mon_bin.c b/drivers/usb/mon/mon_bin.c
index 687f6a8981f34..9d00b21a8153b 100644
--- a/drivers/usb/mon/mon_bin.c
+++ b/drivers/usb/mon/mon_bin.c
@@ -1219,6 +1219,15 @@ mon_bin_poll(struct file *file, struct poll_table_struct *wait)
return mask;
}
+static void __mon_bin_vma_open(struct mon_reader_bin *rp)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&rp->b_lock, flags);
+ rp->mmap_active++;
+ spin_unlock_irqrestore(&rp->b_lock, flags);
+}
+
/*
* open and close: just keep track of how many times the device is
* mapped, to use the proper memory allocation function.
@@ -1226,64 +1235,79 @@ mon_bin_poll(struct file *file, struct poll_table_struct *wait)
static void mon_bin_vma_open(struct vm_area_struct *vma)
{
struct mon_reader_bin *rp = vma->vm_private_data;
- unsigned long flags;
- spin_lock_irqsave(&rp->b_lock, flags);
- rp->mmap_active++;
- spin_unlock_irqrestore(&rp->b_lock, flags);
+ __mon_bin_vma_open(rp);
}
-static void mon_bin_vma_close(struct vm_area_struct *vma)
+static void __mon_bin_vma_close(struct mon_reader_bin *rp)
{
unsigned long flags;
- struct mon_reader_bin *rp = vma->vm_private_data;
spin_lock_irqsave(&rp->b_lock, flags);
rp->mmap_active--;
spin_unlock_irqrestore(&rp->b_lock, flags);
}
-/*
- * Map ring pages to user space.
- */
-static vm_fault_t mon_bin_vma_fault(struct vm_fault *vmf)
+static void mon_bin_vma_close(struct vm_area_struct *vma)
{
- struct mon_reader_bin *rp = vmf->vma->vm_private_data;
+ struct mon_reader_bin *rp = vma->vm_private_data;
+
+ __mon_bin_vma_close(rp);
+}
+
+static const struct vm_operations_struct mon_bin_vm_ops = {
+ .open = mon_bin_vma_open,
+ .close = mon_bin_vma_close,
+};
+
+static int mon_bin_discontig_init(void *vm_private_data, void **private)
+{
+ struct mon_reader_bin *rp = vm_private_data;
+
+ /* Dropped by mon_bin_vma_close() on unmap, including on error. */
+ __mon_bin_vma_open(rp);
+ return 0;
+}
+
+static int mon_bin_discontig_get(struct discontig_kernel_page_state *state)
+{
+ struct mon_reader_bin *rp = state->vm_private_data;
unsigned long offset, chunk_idx;
- struct page *pageptr;
unsigned long flags;
spin_lock_irqsave(&rp->b_lock, flags);
- offset = vmf->pgoff << PAGE_SHIFT;
+
+ offset = state->pgoff << PAGE_SHIFT;
if (offset >= rp->b_size) {
spin_unlock_irqrestore(&rp->b_lock, flags);
- return VM_FAULT_SIGBUS;
+ discontig_kernel_map_abort(state);
+ return 0;
}
chunk_idx = offset / CHUNK_SIZE;
- pageptr = rp->b_vec[chunk_idx].pg;
- get_page(pageptr);
- vmf->page = pageptr;
+ discontig_kernel_map_page(state, rp->b_vec[chunk_idx].pg);
+
spin_unlock_irqrestore(&rp->b_lock, flags);
return 0;
}
-static const struct vm_operations_struct mon_bin_vm_ops = {
- .open = mon_bin_vma_open,
- .close = mon_bin_vma_close,
- .fault = mon_bin_vma_fault,
+static const struct discontig_kernel_page_ops mon_discontig_ops = {
+ .init = mon_bin_discontig_init,
+ .get = mon_bin_discontig_get,
};
-static int mon_bin_mmap(struct file *filp, struct vm_area_struct *vma)
+static int mon_bin_mmap_prepare(struct vm_area_desc *desc)
{
- /* don't do anything here: "fault" will set up page table entries */
- vma->vm_ops = &mon_bin_vm_ops;
+ const struct file *filp = desc->file;
- if (vma->vm_flags & VM_WRITE)
+ if (vma_desc_test(desc, VMA_WRITE_BIT))
return -EPERM;
- vm_flags_mod(vma, VM_DONTEXPAND | VM_DONTDUMP, VM_MAYWRITE);
- vma->vm_private_data = filp->private_data;
- mon_bin_vma_open(vma);
+ desc->vm_ops = &mon_bin_vm_ops;
+ vma_desc_clear_flags(desc, VMA_MAYWRITE_BIT);
+ vma_desc_set_flags(desc, VMA_DONTEXPAND_BIT, VMA_DONTDUMP_BIT);
+ desc->private_data = filp->private_data;
+
+ mmap_action_map_discontig_kernel_pages(desc, NULL, &mon_discontig_ops);
return 0;
}
@@ -1298,7 +1322,7 @@ static const struct file_operations mon_fops_binary = {
.compat_ioctl = mon_bin_compat_ioctl,
#endif
.release = mon_bin_release,
- .mmap = mon_bin_mmap,
+ .mmap_prepare = mon_bin_mmap_prepare,
};
static int mon_bin_wait_event(struct file *file, struct mon_reader_bin *rp)
diff --git a/drivers/video/fbdev/core/fb_defio.c b/drivers/video/fbdev/core/fb_defio.c
index fd00b86e1ae60..fb359ecc39661 100644
--- a/drivers/video/fbdev/core/fb_defio.c
+++ b/drivers/video/fbdev/core/fb_defio.c
@@ -366,13 +366,13 @@ int fb_deferred_io_mmap(struct fb_info *info, struct vm_area_struct *vma)
{
vma->vm_page_prot = pgprot_decrypted(vma->vm_page_prot);
+ if (WARN_ON_ONCE(!(info->flags & FBINFO_VIRTFB)))
+ return -EINVAL;
if (!try_module_get(THIS_MODULE))
return -EINVAL;
vma->vm_ops = &fb_deferred_io_vm_ops;
- vm_flags_set(vma, VM_DONTEXPAND | VM_DONTDUMP);
- if (!(info->flags & FBINFO_VIRTFB))
- vm_flags_set(vma, VM_IO);
+ vm_flags_set(vma, VM_MIXEDMAP | VM_DONTEXPAND | VM_DONTDUMP);
vma->vm_private_data = info->fbdefio_state;
fb_deferred_io_state_get(info->fbdefio_state); /* released in vma->vm_ops->close() */
diff --git a/drivers/video/fbdev/ssd1307fb.c b/drivers/video/fbdev/ssd1307fb.c
index c4fdecafd8560..958514a354338 100644
--- a/drivers/video/fbdev/ssd1307fb.c
+++ b/drivers/video/fbdev/ssd1307fb.c
@@ -763,6 +763,8 @@ static int ssd1307fb_probe(struct i2c_client *client)
info->fix.smem_start = __pa(vmem);
info->fix.smem_len = vmem_size;
+ info->flags = FBINFO_VIRTFB;
+
fb_deferred_io_init(info);
i2c_set_clientdata(client, info);
diff --git a/fs/coredump.c b/fs/coredump.c
index ac3cd74808c64..9f729c594c47e 100644
--- a/fs/coredump.c
+++ b/fs/coredump.c
@@ -1608,7 +1608,7 @@ static unsigned long vma_dump_size(struct vm_area_struct *vma,
}
/* Hugetlb memory check */
- if (is_vm_hugetlb_page(vma)) {
+ if (vma_is_hugetlb(vma)) {
if ((vma->vm_flags & VM_SHARED) && FILTER(HUGETLB_SHARED))
goto whole;
if (!(vma->vm_flags & VM_SHARED) && FILTER(HUGETLB_PRIVATE))
@@ -1616,8 +1616,8 @@ static unsigned long vma_dump_size(struct vm_area_struct *vma,
return 0;
}
- /* Do not dump I/O mapped devices or special mappings */
- if (vma->vm_flags & VM_IO)
+ /* Do not dump memory-mapped I/O, which may have side effects on read. */
+ if (vma_test(vma, VMA_IO_BIT))
return 0;
/* By default, dump shared memory if mapped from an anonymous file. */
diff --git a/fs/fuse/dax.c b/fs/fuse/dax.c
index 85cdf0199bc0b..a5994f1c637d9 100644
--- a/fs/fuse/dax.c
+++ b/fs/fuse/dax.c
@@ -826,7 +826,7 @@ int fuse_dax_mmap(struct file *file, struct vm_area_struct *vma)
{
file_accessed(file);
vma->vm_ops = &fuse_dax_vm_ops;
- vm_flags_set(vma, VM_MIXEDMAP | VM_HUGEPAGE);
+ vma_set_flags(vma, VMA_HUGEPAGE_BIT);
return 0;
}
diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index 7611a8470ea26..ba7097d5720c0 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -108,7 +108,7 @@ static int hugetlbfs_file_mmap(struct file *file, struct vm_area_struct *vma)
* vma address alignment (but not the pgoff alignment) has
* already been checked by prepare_hugepage_range. If you add
* any error returns here, do so after setting VM_HUGETLB, so
- * is_vm_hugetlb_page tests below unmap_region go the right
+ * vma_is_hugetlb tests below unmap_region go the right
* way when do_mmap unwinds (may be important on powerpc
* and ia64).
*/
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index e671b4fd8dedd..565e6446bd312 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -3015,7 +3015,7 @@ static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end,
* hugetlb differs, see pagemap_hugetlb_category().
*/
categories = p->cur_vma_category;
- if (userfaultfd_wp(vma) && !is_vm_hugetlb_page(vma))
+ if (userfaultfd_wp(vma) && !vma_is_hugetlb(vma))
categories |= PAGE_IS_WRITTEN;
if (!pagemap_scan_is_interesting_page(categories, p))
@@ -3028,7 +3028,7 @@ static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end,
if (~p->arg.flags & PM_SCAN_WP_MATCHING)
return ret;
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
err = pagemap_scan_hugetlb_hole_wp(vma, addr, end);
else
err = uffd_wp_range(vma, addr, end - addr, true);
@@ -3470,7 +3470,7 @@ static int show_numa_map(struct seq_file *m, void *v)
seq_puts(m, " stack");
}
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
seq_puts(m, " huge");
/* Skip walking pages if gate VMA */
@@ -3499,7 +3499,7 @@ static int show_numa_map(struct seq_file *m, void *v)
if (md->swapcache)
seq_printf(m, " swapcache=%lu", md->swapcache);
- if (md->active < md->pages && !is_vm_hugetlb_page(vma))
+ if (md->active < md->pages && !vma_is_hugetlb(vma))
seq_printf(m, " active=%lu", md->active);
if (md->writeback)
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index 044dabc1fe9cb..48d47b34cc777 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -11,9 +11,9 @@
#ifndef _ASM_GENERIC__TLB_H
#define _ASM_GENERIC__TLB_H
+#include <linux/mm.h>
#include <linux/mmu_notifier.h>
#include <linux/swap.h>
-#include <linux/hugetlb_inline.h>
#include <asm/tlbflush.h>
#include <asm/cacheflush.h>
@@ -438,7 +438,7 @@ tlb_update_vma_flags(struct mmu_gather *tlb, struct vm_area_struct *vma)
* We rely on tlb_end_vma() to issue a flush, such that when we reset
* these values the batch is empty.
*/
- tlb->vma_huge = is_vm_hugetlb_page(vma);
+ tlb->vma_huge = vma_is_hugetlb(vma);
tlb->vma_exec = !!(vma->vm_flags & VM_EXEC);
/*
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 80a5a03e9cee7..24727ece20fe5 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -7,7 +7,6 @@
#include <linux/mm_types.h>
#include <linux/mmdebug.h>
#include <linux/fs.h>
-#include <linux/hugetlb_inline.h>
#include <linux/cgroup.h>
#include <linux/page_ref.h>
#include <linux/list.h>
@@ -252,14 +251,14 @@ extern void __hugetlb_zap_end(struct vm_area_struct *vma,
static inline void hugetlb_zap_begin(struct vm_area_struct *vma,
unsigned long *start, unsigned long *end)
{
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
__hugetlb_zap_begin(vma, start, end);
}
static inline void hugetlb_zap_end(struct vm_area_struct *vma,
struct zap_details *details)
{
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
__hugetlb_zap_end(vma, details);
}
diff --git a/include/linux/hugetlb_inline.h b/include/linux/hugetlb_inline.h
deleted file mode 100644
index 5c29cd3223a1e..0000000000000
--- a/include/linux/hugetlb_inline.h
+++ /dev/null
@@ -1,28 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef _LINUX_HUGETLB_INLINE_H
-#define _LINUX_HUGETLB_INLINE_H
-
-#include <linux/mm.h>
-
-#ifdef CONFIG_HUGETLB_PAGE
-
-static inline bool is_vma_hugetlb_flags(const vma_flags_t *flags)
-{
- return vma_flags_test(flags, VMA_HUGETLB_BIT);
-}
-
-#else
-
-static inline bool is_vma_hugetlb_flags(const vma_flags_t *flags)
-{
- return false;
-}
-
-#endif
-
-static inline bool is_vm_hugetlb_page(const struct vm_area_struct *vma)
-{
- return is_vma_hugetlb_flags(&vma->flags);
-}
-
-#endif
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 969594074fd2d..1249e04d7b980 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -576,14 +576,6 @@ enum {
#define VM_ACCESS_FLAGS (VM_READ | VM_WRITE | VM_EXEC)
#define VMA_ACCESS_FLAGS mk_vma_flags(VMA_READ_BIT, VMA_WRITE_BIT, VMA_EXEC_BIT)
-/*
- * Special vmas that are non-mergable, non-mlock()able.
- */
-
-#define VMA_SPECIAL_FLAGS mk_vma_flags(VMA_IO_BIT, VMA_DONTEXPAND_BIT, \
- VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT)
-#define VM_SPECIAL vma_flags_to_legacy(VMA_SPECIAL_FLAGS)
-
/*
* Physically remapped pages are special. Tell the
* rest of the world about it:
@@ -600,9 +592,6 @@ enum {
#define VMA_REMAP_FLAGS mk_vma_flags(VMA_IO_BIT, VMA_PFNMAP_BIT, \
VMA_DONTEXPAND_BIT, VMA_DONTDUMP_BIT)
-/* This mask prevents VMA from being scanned with khugepaged */
-#define VM_NO_KHUGEPAGED (VM_SPECIAL | VM_HUGETLB)
-
/* This mask defines which mm->def_flags a process can inherit its parent */
#define VM_INIT_DEF_MASK VM_NOHUGEPAGE
@@ -1612,6 +1601,211 @@ static inline bool vma_is_shared_maywrite(const struct vm_area_struct *vma)
return is_shared_maywrite(&vma->flags);
}
+/**
+ * vma_flags_is_hugetlb() - Do the specified VMA flags indicate that the
+ * VMA is a hugetlb mapping?
+ * @flags: The VMA flags to test.
+ *
+ * Returns: true if the flags indicate a hugetlb mapping, false otherwise.
+ */
+static inline bool vma_flags_is_hugetlb(const vma_flags_t *flags)
+{
+ return IS_ENABLED(CONFIG_HUGETLB_PAGE) &&
+ vma_flags_test(flags, VMA_HUGETLB_BIT);
+}
+
+/**
+ * vma_is_hugetlb() - Is @vma a hugetlb mapping?
+ * @vma: The VMA to test.
+ *
+ * Returns: true if @vma is a hugetlb mapping, false otherwise.
+ */
+static inline bool vma_is_hugetlb(const struct vm_area_struct *vma)
+{
+ return vma_flags_is_hugetlb(&vma->flags);
+}
+
+/**
+ * vma_flags_is_kernel_owned() - Do the specified VMA flags indicate that the
+ * contents of the VMA are owned by the kernel rather than the core mm?
+ * @flags: The VMA flags to test.
+ *
+ * A kernel-owned mapping is one whose contents are established and controlled
+ * by the kernel, typically a driver, rather than by the core mm's fault and
+ * rmap machinery.
+ *
+ * The mapping may be memory-mapped I/O, kernel-allocated pages or ordinary
+ * pages the owner has chosen to map itself (shmem via a PFN map, for instance).
+ *
+ * In all cases the core mm must not populate, reclaim, migrate, copy-on-write
+ * or merge it of its own accord.
+ *
+ * Pages mapped this way are not necessarily reference counted or map counted.
+ *
+ * Returns: true if the flags indicate a kernel-owned mapping.
+ */
+static inline bool vma_flags_is_kernel_owned(const vma_flags_t *flags)
+{
+ return vma_flags_test_any(flags, VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT);
+}
+
+/**
+ * vma_is_kernel_owned() - Are the contents of @vma owned by the kernel?
+ * @vma: The VMA to test.
+ *
+ * See vma_flags_is_kernel_owned() for a description of this property.
+ *
+ * Returns: true if the VMA is kernel-owned.
+ */
+static inline bool vma_is_kernel_owned(const struct vm_area_struct *vma)
+{
+ return vma_flags_is_kernel_owned(&vma->flags);
+}
+
+/**
+ * vma_flags_is_fixed_mapping() - Do the specified VMA flags indicate that this
+ * is a fixed mapping that cannot be expanded or merged?
+ * @flags: The VMA flags to test.
+ *
+ * Fixed mappings are those whose size is set at the point of mmap (for
+ * instance, a kernel-owned mapping of a fixed range of memory), and thus
+ * cannot be expanded or merged.
+ *
+ * Returns: true if the flags indicate a fixed mapping.
+ */
+static inline bool vma_flags_is_fixed_mapping(const vma_flags_t *flags)
+{
+ /*
+ * VMA_PFNMAP_BIT should imply VMA_DONTEXPAND_BIT, but some callers set
+ * only the former.
+ */
+ return vma_flags_test_any(flags, VMA_PFNMAP_BIT, VMA_DONTEXPAND_BIT);
+}
+
+/**
+ * vma_is_fixed_mapping() - Is this VMA a fixed mapping that cannot be
+ * expanded or merged?
+ * @vma: The VMA to test.
+ *
+ * See vma_flags_is_fixed_mapping() for a description of this property.
+ *
+ * Returns: true if the VMA maps a fixed mapping.
+ */
+static inline bool vma_is_fixed_mapping(const struct vm_area_struct *vma)
+{
+ return vma_flags_is_fixed_mapping(&vma->flags);
+}
+
+/**
+ * vma_flags_can_merge() - Do the specified VMA flags permit the VMA to be
+ * merged with another?
+ * @flags: The VMA flags to test.
+ * Returns: true if the flags permit merging, false otherwise.
+ */
+static inline bool vma_flags_can_merge(const vma_flags_t *flags)
+{
+ /*
+ * VMA merging assumes that a VMA's flags and fields completely describe
+ * its state.
+ *
+ * However, kernel-owned mappings may have established state upon mapping
+ * not embodied in any attribute of the VMA.
+ *
+ * Additionally, private (CoW) PFN maps encode the source PFN of the
+ * range in vma->vm_pgoff, which may otherwise cause spurious merges.
+ */
+ if (vma_flags_is_kernel_owned(flags))
+ return false;
+ /* VMA explicitly marked as being unmergeable. */
+ if (vma_flags_is_fixed_mapping(flags))
+ return false;
+
+ return true;
+}
+
+/**
+ * vma_can_merge() - Do @vma's flags permit it to be merged with another VMA?
+ * @vma: The VMA to test.
+ * Returns: true if the flags permit merging, otherwise false.
+ */
+static inline bool vma_can_merge(const struct vm_area_struct *vma)
+{
+ return vma_flags_can_merge(&vma->flags);
+}
+
+/**
+ * vma_flags_is_persistent() - Do the specified VMA flags imply that the VMA
+ * contains persistent data?
+ * @flags: The VMA flags to test.
+ *
+ * Persistent in the sense that - if you write bytes to the mapping - do they
+ * stay written?
+ *
+ * If the kernel or a device could write to the memory independently of
+ * userland, or the kernel could arbitrarily discard it, then it is not
+ * persistent.
+ *
+ * Returns: true if the flags imply this VMA is persistent, otherwise false.
+ */
+static inline bool vma_flags_is_persistent(const vma_flags_t *flags)
+{
+ /* hugetlb is a fixed mapping, but its contents are the user's own. */
+ if (vma_flags_is_hugetlb(flags))
+ return true;
+ /*
+ * MMIO mappings may not store what is written and may be changed by the
+ * device. Kernel-owned and fixed mappings may be changed by their owner
+ * without the user having initiated it.
+ */
+ if (vma_flags_is_kernel_owned(flags) ||
+ vma_flags_is_fixed_mapping(flags))
+ return false;
+ /* Droppable memory is discardable by definition. */
+ return !vma_flags_test_single_mask(flags, VMA_DROPPABLE);
+}
+
+/**
+ * vma_is_persistent() - Does the VMA contain persistent data?
+ * @vma: The VMA to test.
+ *
+ * See vma_flags_is_persistent() for details.
+ *
+ * Returns: true if the VMA is persistent, otherwise false.
+ */
+static inline bool vma_is_persistent(const struct vm_area_struct *vma)
+{
+ return vma_flags_is_persistent(&vma->flags);
+}
+
+/**
+ * vma_flags_can_gup() - Do the specified VMA flags permit GUP to access the
+ * mapping's pages?
+ * @flags: The VMA flags to test.
+ *
+ * GUP cannot obtain pages from a PFN map (VMA_PFNMAP_BIT), which may have no
+ * struct pages behind it, and must not provide access to memory-mapped I/O
+ * (VMA_IO_BIT).
+ *
+ * Returns: true if GUP may access pages from the mapping, otherwise false.
+ */
+static inline bool vma_flags_can_gup(const vma_flags_t *flags)
+{
+ return !vma_flags_test_any(flags, VMA_IO_BIT, VMA_PFNMAP_BIT);
+}
+
+/**
+ * vma_can_gup() - May GUP obtain pages from @vma?
+ * @vma: The VMA to test.
+ *
+ * See vma_flags_can_gup() for details.
+ *
+ * Returns: true if GUP may access pages from the mapping, otherwise false.
+ */
+static inline bool vma_can_gup(const struct vm_area_struct *vma)
+{
+ return vma_flags_can_gup(&vma->flags);
+}
+
/**
* vma_kernel_pagesize - Default page size granularity for this VMA.
* @vma: The user mapping.
@@ -4602,7 +4796,7 @@ static inline void mmap_action_map_kernel_pages(struct vm_area_desc *desc,
{
struct mmap_action *action = &desc->action;
- action->type = MMAP_MAP_KERNEL_PAGES;
+ action->type = MMAP_KERNEL_PAGES;
action->map_kernel.start = start;
action->map_kernel.pages = pages;
action->map_kernel.nr_pages = nr_pages;
@@ -4626,10 +4820,55 @@ static inline void mmap_action_map_kernel_pages_full(struct vm_area_desc *desc,
vma_desc_pages(desc));
}
+static inline
+void mmap_action_map_discontig_kernel_pages(struct vm_area_desc *desc,
+ void *init_private, const struct discontig_kernel_page_ops *ops)
+{
+ struct mmap_action *action = &desc->action;
+
+ action->type = MMAP_DISCONTIG_KERNEL_PAGES;
+ action->map_kernel_discontig.init_private = init_private;
+ action->map_kernel_discontig.ops = ops;
+}
+
int mmap_action_prepare(struct vm_area_desc *desc);
int mmap_action_complete(struct vm_area_struct *vma,
struct mmap_action *action, bool is_compat);
+static inline void
+discontig_kernel_map_abort(struct discontig_kernel_page_state *state)
+{
+ state->action = DISCONTIG_KERNEL_PAGE_ABORT;
+}
+
+static inline void
+discontig_kernel_map_page(struct discontig_kernel_page_state *state,
+ struct page *page)
+{
+ struct folio *folio = page_folio(page);
+
+ if (folio_test_large(folio)) {
+ VM_WARN_ON_ONCE(page != folio_page(folio, 0));
+ state->action = DISCONTIG_KERNEL_PAGE_MAP_COMPOUND_PAGE;
+ state->__folio = folio;
+ state->__nr_pages = min(state->nr_pages_remain,
+ folio_nr_pages(folio));
+ } else {
+ state->action = DISCONTIG_KERNEL_PAGE_MAP_PAGE;
+ state->__page = page;
+ state->__nr_pages = 1;
+ }
+}
+
+static inline void
+discontig_kernel_map_page_range(struct discontig_kernel_page_state *state,
+ struct page **page_arr, unsigned long nr_pages)
+{
+ state->action = DISCONTIG_KERNEL_PAGE_MAP_PAGE_RANGE;
+ state->__page_arr = page_arr;
+ state->__nr_pages = nr_pages;
+}
+
/* Look up the first VMA which exactly match the interval vm_start ... vm_end */
static inline struct vm_area_struct *find_exact_vma(struct mm_struct *mm,
unsigned long vm_start, unsigned long vm_end)
@@ -4747,9 +4986,6 @@ int remap_pfn_range(struct vm_area_struct *vma, unsigned long addr,
int vm_insert_page(struct vm_area_struct *, unsigned long addr, struct page *);
int vm_insert_pages(struct vm_area_struct *vma, unsigned long addr,
struct page **pages, unsigned long *num);
-int map_kernel_pages_prepare(struct vm_area_desc *desc);
-int map_kernel_pages_complete(struct vm_area_struct *vma,
- struct mmap_action *action);
int vm_map_pages(struct vm_area_struct *vma, struct page **pages,
unsigned long num);
int vm_map_pages_zero(struct vm_area_struct *vma, struct page **pages,
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 5413bd10fff2c..0cb4f96039568 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -815,11 +815,47 @@ struct pfnmap_track_ctx {
/* What action should be taken after an .mmap_prepare call is complete? */
enum mmap_action_type {
- MMAP_NOTHING, /* Mapping is complete, no further action. */
- MMAP_REMAP_PFN, /* Remap PFN range. */
- MMAP_IO_REMAP_PFN, /* I/O remap PFN range. */
- MMAP_SIMPLE_IO_REMAP, /* I/O remap with guardrails. */
- MMAP_MAP_KERNEL_PAGES, /* Map kernel page range from array. */
+ MMAP_NOTHING,
+ MMAP_REMAP_PFN,
+ MMAP_IO_REMAP_PFN,
+ MMAP_SIMPLE_IO_REMAP, /* I/O remap with guardrails. */
+ MMAP_KERNEL_PAGES, /* Map kernel page range from array. */
+ MMAP_DISCONTIG_KERNEL_PAGES, /* Map kernel discontig page range. */
+};
+
+enum discontig_kernel_page_action {
+ DISCONTIG_KERNEL_PAGE_ABORT,
+ DISCONTIG_KERNEL_PAGE_MAP_PAGE,
+ DISCONTIG_KERNEL_PAGE_MAP_COMPOUND_PAGE,
+ DISCONTIG_KERNEL_PAGE_MAP_PAGE_RANGE,
+};
+
+struct discontig_kernel_page_state {
+ /* Map state. */
+ const unsigned long start; /* Start address of VMA. */
+ const unsigned long end; /* End address of VMA. */
+ unsigned long addr; /* The current address to be mapped. */
+ pgoff_t pgoff; /* The current pgoff to be mapped. */
+ unsigned long nr_pages_mapped; /* The number of pages mapped. */
+ unsigned long nr_pages_remain; /* The number of pages remaining. */
+
+ /* User-defined state. */
+ void *vm_private_data; /* VMA private data. */
+ void *private; /* Mapping private data. */
+
+ /* Users should not touch these, use discontig_kernel_map_*() helpers. */
+ enum discontig_kernel_page_action action;
+ union {
+ struct page *__page;
+ struct folio *__folio;
+ struct page **__page_arr;
+ };
+ unsigned long __nr_pages;
+};
+
+struct discontig_kernel_page_ops {
+ int (*init)(void *vm_private_data, void **private);
+ int (*get)(struct discontig_kernel_page_state *state);
};
/*
@@ -844,6 +880,10 @@ struct mmap_action {
unsigned long nr_pages;
pgoff_t pgoff;
} map_kernel;
+ struct {
+ void *init_private;
+ const struct discontig_kernel_page_ops *ops;
+ } map_kernel_discontig;
};
enum mmap_action_type type;
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 939f3a5e973f6..d7d8b312466c2 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -14,7 +14,6 @@
#include <linux/gfp.h>
#include <linux/bitops.h>
#include <linux/hardirq.h> /* for in_interrupt() */
-#include <linux/hugetlb_inline.h>
struct folio_batch;
diff --git a/include/linux/rmap.h b/include/linux/rmap.h
index 0b332770abeed..74cca0e3c7264 100644
--- a/include/linux/rmap.h
+++ b/include/linux/rmap.h
@@ -888,7 +888,7 @@ struct page_vma_mapped_walk {
static inline void page_vma_mapped_walk_done(struct page_vma_mapped_walk *pvmw)
{
/* HugeTLB pte is set to the relevant page table entry without pte_mapped. */
- if (pvmw->pte && !is_vm_hugetlb_page(pvmw->vma))
+ if (pvmw->pte && !vma_is_hugetlb(pvmw->vma))
pte_unmap(pvmw->pte);
if (pvmw->ptl)
spin_unlock(pvmw->ptl);
diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h
index a4351cffc60ce..a14b8a9ffb7b1 100644
--- a/include/linux/userfaultfd_k.h
+++ b/include/linux/userfaultfd_k.h
@@ -18,7 +18,6 @@
#include <linux/swap.h>
#include <linux/leafops.h>
#include <asm-generic/pgtable_uffd.h>
-#include <linux/hugetlb_inline.h>
/* The set of all possible UFFD-related VM flags. */
#define __VM_UFFD_FLAGS (VM_UFFD_MISSING | VM_UFFD_MINOR | \
diff --git a/kernel/bpf/arena.c b/kernel/bpf/arena.c
index 7b6847200b431..b69fe5e343393 100644
--- a/kernel/bpf/arena.c
+++ b/kernel/bpf/arena.c
@@ -620,8 +620,9 @@ static int arena_map_mmap(struct bpf_map *map, struct vm_area_struct *vma)
* clears VM_MAYEXEC. Set VM_DONTEXPAND to avoid potential change
* of user_vm_start. Set VM_DONTCOPY to prevent arena VMA from
* being copied into the child process on fork.
+ * This is a kernel page so set VM_MIXEDMAP.
*/
- vm_flags_set(vma, VM_DONTEXPAND | VM_DONTCOPY);
+ vm_flags_set(vma, VM_MIXEDMAP | VM_DONTEXPAND | VM_DONTCOPY);
vma->vm_ops = &arena_vm_ops;
return 0;
}
diff --git a/kernel/events/core.c b/kernel/events/core.c
index a6c8e38a31104..8ca8a68429242 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -9808,7 +9808,7 @@ static void perf_event_mmap_event(struct perf_mmap_event *mmap_event)
if (vma->vm_flags & VM_LOCKED)
flags |= MAP_LOCKED;
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
flags |= MAP_HUGETLB;
if (file) {
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index 7709ea8824778..b89cc5cee0027 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -1726,8 +1726,8 @@ static int xol_add_vma(struct mm_struct *mm, struct xol_area *area)
}
vma = _install_special_mapping(mm, area->vaddr, PAGE_SIZE,
- VM_EXEC|VM_MAYEXEC|VM_DONTCOPY|VM_IO|
- VM_SEALED_SYSMAP,
+ VM_EXEC|VM_MAYEXEC|VM_DONTCOPY|
+ VM_MIXEDMAP|VM_SEALED_SYSMAP,
&xol_mapping);
if (IS_ERR(vma)) {
ret = PTR_ERR(vma);
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 8dff37059faf7..ae6c1a606eb5d 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -22,7 +22,6 @@
*/
#include <linux/energy_model.h>
#include <linux/mmap_lock.h>
-#include <linux/hugetlb_inline.h>
#include <linux/jiffies.h>
#include <linux/mm_api.h>
#include <linux/highmem.h>
@@ -4212,7 +4211,7 @@ static void task_numa_work(struct callback_head *work)
for (; vma; vma = vma_next(&vmi)) {
if (!vma_migratable(vma) || !vma_policy_mof(vma) ||
- is_vm_hugetlb_page(vma) || (vma->vm_flags & VM_MIXEDMAP)) {
+ vma_is_hugetlb(vma) || vma_is_kernel_owned(vma)) {
trace_sched_skip_vma_numa(mm, vma, NUMAB_SKIP_UNSUITABLE);
continue;
}
diff --git a/mm/folio.c b/mm/folio.c
index 47a437e0f7fde..35e242b48870b 100644
--- a/mm/folio.c
+++ b/mm/folio.c
@@ -505,7 +505,7 @@ void folio_add_lru_vma(struct folio *folio, struct vm_area_struct *vma)
{
VM_BUG_ON_FOLIO(folio_test_lru(folio), folio);
- if (unlikely((vma->vm_flags & (VM_LOCKED | VM_SPECIAL)) == VM_LOCKED))
+ if (vma_test(vma, VMA_LOCKED_BIT))
mlock_new_folio(folio);
else
folio_add_lru(folio);
diff --git a/mm/gup.c b/mm/gup.c
index c2dfcb4744bc3..8e9ef5ee7498c 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -621,7 +621,7 @@ static struct page *no_page_table(struct vm_area_struct *vma,
* But we can only make this optimization where a hole would surely
* be zero-filled if handle_mm_fault() actually did handle it.
*/
- if (is_vm_hugetlb_page(vma)) {
+ if (vma_is_hugetlb(vma)) {
struct hstate *h = hstate_vma(vma);
if (!hugetlbfs_pagecache_present(h, vma, address))
@@ -1204,7 +1204,7 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags)
int foreign = (gup_flags & FOLL_REMOTE);
bool vma_anon = vma_is_anonymous(vma);
- if (vm_flags & (VM_IO | VM_PFNMAP))
+ if (!vma_can_gup(vma))
return -EFAULT;
if ((gup_flags & FOLL_ANON) && !vma_anon)
@@ -1213,7 +1213,7 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags)
if ((gup_flags & FOLL_LONGTERM) && vma_is_fsdax(vma))
return -EOPNOTSUPP;
- if ((gup_flags & FOLL_SPLIT_PMD) && is_vm_hugetlb_page(vma))
+ if ((gup_flags & FOLL_SPLIT_PMD) && vma_is_hugetlb(vma))
return -EOPNOTSUPP;
if (vma_is_secretmem(vma))
@@ -1836,6 +1836,10 @@ long populate_vma_page_range(struct vm_area_struct *vma,
if (!vma_is_accessible(vma))
return -EFAULT;
+ /* Unreadable VMAs also cannot be faulted in. */
+ if (!vma_test(vma, VMA_MAYREAD_BIT))
+ return -EFAULT;
+
gup_flags = FOLL_TOUCH;
/*
* We want to touch writable mappings with a write fault in order
@@ -1951,7 +1955,7 @@ int __mm_populate(unsigned long start, unsigned long len, int ignore_errors)
* range with the first VMA. Also, skip undesirable VMA types.
*/
nend = min(end, vma->vm_end);
- if (vma->vm_flags & (VM_IO | VM_PFNMAP))
+ if (!vma_can_gup(vma))
continue;
if (nstart < vma->vm_start)
nstart = vma->vm_start;
@@ -2013,8 +2017,7 @@ static long __get_user_pages_locked(struct mm_struct *mm, unsigned long start,
break;
/* protect what we can, including chardevs */
- if ((vma->vm_flags & (VM_IO | VM_PFNMAP)) ||
- !(vm_flags & vma->vm_flags))
+ if (!vma_can_gup(vma) || !(vm_flags & vma->vm_flags))
break;
if (pages) {
diff --git a/mm/hmm.c b/mm/hmm.c
index 2f1e98c6b6440..e9569b82a1f0c 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -595,8 +595,7 @@ static int hmm_vma_walk_test(unsigned long start, unsigned long end,
struct hmm_range *range = hmm_vma_walk->range;
struct vm_area_struct *vma = walk->vma;
- if (!(vma->vm_flags & (VM_IO | VM_PFNMAP)) &&
- vma->vm_flags & VM_READ)
+ if (vma_can_gup(vma) && vma_test(vma, VMA_READ_BIT))
return 0;
/*
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 4cd917f77f3f7..3cb8e2d4d65cb 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -110,14 +110,6 @@ static inline bool file_thp_enabled(const struct vm_area_struct *vma)
return S_ISREG(inode->i_mode);
}
-/* If returns true, we are unable to access the VMA's folios. */
-static bool vma_is_special_huge(const struct vm_area_struct *vma)
-{
- if (vma_is_dax(vma))
- return false;
- return vma_test_any(vma, VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT);
-}
-
static bool vma_file_bypass_thp_tuneables(const struct vm_area_struct *vma,
enum tva_type type)
{
@@ -192,7 +184,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
/* Check the intersection of requested and supported orders. */
if (vma_is_anonymous(vma))
supported_orders = THP_ORDERS_ALL_ANON;
- else if (vma_is_dax(vma) || vma_is_special_huge(vma))
+ else if (vma_is_dax(vma) || vma_is_kernel_owned(vma))
supported_orders = THP_ORDERS_ALL_SPECIAL_DAX;
else
supported_orders = THP_ORDERS_ALL_FILE_DEFAULT;
@@ -212,11 +204,14 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
return in_pf ? orders : 0;
/*
- * khugepaged special VMA and hugetlb VMA.
- * Must be checked after dax since some dax mappings may have
- * VM_MIXEDMAP set.
+ * khugepaged moves data from VMAs once collapsed, after they have been
+ * faulted in, relying on refaulting for file-backed memory.
+ *
+ * Kernel-owned mappings cannot be reliably reconstructed from page
+ * faults, and fixed mappings (including hugetlb) may not be marked as
+ * kernel-owned - precisely the mappings which cannot be merged.
*/
- if (!in_pf && !smaps && (vm_flags & VM_NO_KHUGEPAGED))
+ if (!in_pf && !smaps && !vma_can_merge(vma))
return 0;
/*
@@ -3063,7 +3058,7 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma,
orig_pud = pudp_huge_get_and_clear_full(vma, addr, pud, tlb->fullmm);
arch_check_zapped_pud(vma, orig_pud);
tlb_remove_pud_tlb_entry(tlb, pud, addr);
- if (vma_is_special_huge(vma)) {
+ if (vma_is_kernel_owned(vma)) {
spin_unlock(ptl);
/* No zero page support yet */
} else {
@@ -3219,7 +3214,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,
*/
if (arch_needs_pgtable_deposit())
zap_deposited_table(mm, pmd);
- if (vma_is_special_huge(vma))
+ if (vma_is_kernel_owned(vma))
return;
if (unlikely(pmd_is_migration_entry(old_pmd))) {
const softleaf_t old_entry = softleaf_from_pmd(old_pmd);
@@ -4762,11 +4757,9 @@ static inline bool vma_not_suitable_for_thp_split(struct vm_area_struct *vma)
{
if (vma_is_dax(vma))
return true;
- if (vma_is_special_huge(vma))
- return true;
- if (vma_test(vma, VMA_IO_BIT))
+ if (vma_is_kernel_owned(vma))
return true;
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
return true;
return false;
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index d3a0650ff6905..817f57f13b09d 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1147,7 +1147,7 @@ static inline struct resv_map *inode_resv_map(struct inode *inode)
static struct resv_map *vma_resv_map(struct vm_area_struct *vma)
{
- VM_BUG_ON_VMA(!is_vm_hugetlb_page(vma), vma);
+ VM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);
if (vma->vm_flags & VM_MAYSHARE) {
struct address_space *mapping = vma->vm_file->f_mapping;
struct inode *inode = mapping->host;
@@ -1162,7 +1162,7 @@ static struct resv_map *vma_resv_map(struct vm_area_struct *vma)
static void set_vma_resv_map(struct vm_area_struct *vma, struct resv_map *map)
{
- VM_WARN_ON_ONCE_VMA(!is_vm_hugetlb_page(vma), vma);
+ VM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);
VM_WARN_ON_ONCE_VMA(vma_test(vma, VMA_MAYSHARE_BIT), vma);
set_vma_private_data(vma, (unsigned long)map);
@@ -1170,7 +1170,7 @@ static void set_vma_resv_map(struct vm_area_struct *vma, struct resv_map *map)
static void set_vma_resv_flags(struct vm_area_struct *vma, unsigned long flags)
{
- VM_WARN_ON_ONCE_VMA(!is_vm_hugetlb_page(vma), vma);
+ VM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);
VM_WARN_ON_ONCE_VMA(vma_test(vma, VMA_MAYSHARE_BIT), vma);
set_vma_private_data(vma, get_vma_private_data(vma) | flags);
@@ -1178,7 +1178,7 @@ static void set_vma_resv_flags(struct vm_area_struct *vma, unsigned long flags)
static int is_vma_resv_set(struct vm_area_struct *vma, unsigned long flag)
{
- VM_BUG_ON_VMA(!is_vm_hugetlb_page(vma), vma);
+ VM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);
return (get_vma_private_data(vma) & flag) != 0;
}
@@ -1192,7 +1192,7 @@ bool __vma_private_lock(struct vm_area_struct *vma)
void hugetlb_dup_vma_private(struct vm_area_struct *vma)
{
- VM_BUG_ON_VMA(!is_vm_hugetlb_page(vma), vma);
+ VM_WARN_ON_ONCE_VMA(!vma_is_hugetlb(vma), vma);
/*
* Clear vm_private_data
* - For shared mappings this is a per-vma semaphore that may be
@@ -5279,7 +5279,7 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
unsigned long last_addr_mask;
i_mmap_assert_write_locked(vma->vm_file->f_mapping);
- WARN_ON(!is_vm_hugetlb_page(vma));
+ WARN_ON(!vma_is_hugetlb(vma));
BUG_ON(start & ~huge_page_mask(h));
BUG_ON(end & ~huge_page_mask(h));
@@ -7505,6 +7505,6 @@ void hugetlb_unshare_all_pmds(struct vm_area_struct *vma)
*/
void fixup_hugetlb_reservations(struct vm_area_struct *vma)
{
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
clear_vma_resv_huge_pages(vma);
}
diff --git a/mm/internal.h b/mm/internal.h
index 0dca33db068f6..83a4ba52aaebc 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -7,6 +7,7 @@
#ifndef __MM_INTERNAL_H
#define __MM_INTERNAL_H
+#include <linux/file.h>
#include <linux/fs.h>
#include <linux/khugepaged.h>
#include <linux/mm.h>
@@ -212,6 +213,24 @@ static inline void *folio_raw_mapping(const struct folio *folio)
return (void *)(mapping & ~FOLIO_MAPPING_FLAGS);
}
+/*
+ * If the VMA has a close hook then close it, and since closing it might leave
+ * it in an inconsistent state which makes the use of any hooks suspect, clear
+ * them down by installing dummy empty hooks.
+ */
+static inline void vma_close(struct vm_area_struct *vma)
+{
+ if (vma->vm_ops && vma->vm_ops->close) {
+ vma->vm_ops->close(vma);
+
+ /*
+ * The mapping is in an inconsistent state, and no further hooks
+ * may be invoked upon it.
+ */
+ vma->vm_ops = &vma_dummy_vm_ops;
+ }
+}
+
/*
* This is a file-backed mapping, and is about to be memory mapped - invoke its
* mmap hook and safely handle error conditions. On error, VMA hooks will be
@@ -224,8 +243,11 @@ static inline void *folio_raw_mapping(const struct folio *folio)
*/
static inline int mmap_file(struct file *file, struct vm_area_struct *vma)
{
- int err = vfs_mmap(file, vma);
+ const unsigned long prev_start = vma->vm_start;
+ const vma_flags_t prev_flags = vma->flags;
+ int err;
+ err = vfs_mmap(file, vma);
/*
* Either we tried to call the file hook for mmap() and an error arose
* or a driver set vma->vm_ops = NULL intending there to be no VMA
@@ -238,26 +260,16 @@ static inline int mmap_file(struct file *file, struct vm_area_struct *vma)
*/
if (unlikely(err || !vma->vm_ops))
vma->vm_ops = &vma_dummy_vm_ops;
+ if (unlikely(err))
+ return err;
- return err;
-}
-
-/*
- * If the VMA has a close hook then close it, and since closing it might leave
- * it in an inconsistent state which makes the use of any hooks suspect, clear
- * them down by installing dummy empty hooks.
- */
-static inline void vma_close(struct vm_area_struct *vma)
-{
- if (vma->vm_ops && vma->vm_ops->close) {
- vma->vm_ops->close(vma);
-
- /*
- * The mapping is in an inconsistent state, and no further hooks
- * may be invoked upon it.
- */
- vma->vm_ops = &vma_dummy_vm_ops;
+ err = mmap_hook_validate(prev_start, &prev_flags, vma);
+ if (unlikely(err)) {
+ vma->vm_start = prev_start;
+ vma_close(vma);
}
+
+ return err;
}
/* unmap_vmas is in mm/memory.c */
@@ -957,15 +969,7 @@ void mlock_folio(struct folio *folio);
static inline void mlock_vma_folio(struct folio *folio,
struct vm_area_struct *vma)
{
- /*
- * The VM_SPECIAL check here serves two purposes.
- * 1) VM_IO check prevents migration from double-counting during mlock.
- * 2) Although mmap_region() and mlock_fixup() take care that VM_LOCKED
- * is never left set on a VM_SPECIAL vma, there is an interval while
- * file->f_op->mmap() is using vm_insert_page(s), when VM_LOCKED may
- * still be set while VM_SPECIAL bits are added: so ignore it then.
- */
- if (unlikely((vma->vm_flags & (VM_LOCKED|VM_SPECIAL)) == VM_LOCKED))
+ if (vma_test(vma, VMA_LOCKED_BIT))
mlock_folio(folio);
}
@@ -982,7 +986,12 @@ static inline void munlock_vma_folio(struct folio *folio,
* always munlock the folio and page reclaim will correct it
* if it's wrong.
*/
- if (unlikely(vma->vm_flags & VM_LOCKED))
+ /*
+ * VMA_LOCKONFAULT_BIT alone marks an mlock walk in progress, see
+ * mlock_vma_pages_range(). An unmap racing with the walk must still
+ * munlock folios the walk has already counted.
+ */
+ if (unlikely(vma_test_any_mask(vma, VMA_LOCKED_MASK)))
munlock_folio(folio);
}
@@ -1102,11 +1111,9 @@ static inline struct file *maybe_unlock_mmap_for_io(struct vm_fault *vmf,
static inline bool vma_supports_mlock(const struct vm_area_struct *vma)
{
- if (vma_test_any_mask(vma, VMA_SPECIAL_FLAGS))
- return false;
- if (vma_test_single_mask(vma, VMA_DROPPABLE))
+ if (!vma_is_persistent(vma))
return false;
- if (vma_is_dax(vma) || is_vm_hugetlb_page(vma))
+ if (vma_is_dax(vma) || vma_is_hugetlb(vma))
return false;
return vma != get_gate_vma(current->mm);
}
@@ -1499,6 +1506,12 @@ int remap_pfn_range_prepare(struct vm_area_desc *desc);
int remap_pfn_range_complete(struct vm_area_struct *vma,
struct mmap_action *action);
int simple_ioremap_prepare(struct vm_area_desc *desc);
+int map_kernel_pages_prepare(struct vm_area_desc *desc);
+int map_kernel_pages_complete(struct vm_area_struct *vma,
+ struct mmap_action *action);
+int map_discontig_kernel_pages_prepare(struct vm_area_desc *desc);
+int map_discontig_kernel_pages_complete(struct vm_area_struct *vma,
+ struct mmap_action *action);
static inline int io_remap_pfn_range_prepare(struct vm_area_desc *desc)
{
diff --git a/mm/ksm.c b/mm/ksm.c
index 624f37975e129..f80372bfd4b2f 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -747,9 +747,7 @@ static bool ksm_compatible(const struct file *file, vma_flags_t vma_flags)
if (vma_flags_test_any(&vma_flags, VMA_SHARED_BIT, VMA_MAYSHARE_BIT,
VMA_HUGETLB_BIT))
return false;
- if (vma_flags_test_single_mask(&vma_flags, VMA_DROPPABLE))
- return false;
- if (vma_flags_test_any_mask(&vma_flags, VMA_SPECIAL_FLAGS))
+ if (!vma_flags_is_persistent(&vma_flags))
return false;
if (file_is_dax(file))
return false;
diff --git a/mm/madvise.c b/mm/madvise.c
index fbb72ab49aa64..1af82b044d235 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -881,7 +881,7 @@ bool madvise_dontneed_free_valid_vma(struct madvise_behavior *madv_behavior)
int behavior = madv_behavior->behavior;
struct madvise_behavior_range *range = &madv_behavior->range;
- if (!is_vm_hugetlb_page(vma)) {
+ if (!vma_is_hugetlb(vma)) {
unsigned int forbidden = VM_PFNMAP;
if (behavior != MADV_DONTNEED_LOCKED)
@@ -1221,19 +1221,25 @@ static long madvise_remove(struct madvise_behavior *madv_behavior)
return error;
}
-static bool is_valid_guard_vma(struct vm_area_struct *vma, bool allow_locked)
+static bool is_valid_guard_vma(const struct vm_area_struct *vma,
+ bool allow_locked)
{
- vm_flags_t disallowed = VM_SPECIAL | VM_HUGETLB;
-
/*
- * A user could lock after setting a guard range but that's fine, as
+ * A user could lock after setting a guard range but that's fine as
* they'd not be able to fault in. The issue arises when we try to zap
* existing locked VMAs. We don't want to do that.
*/
- if (!allow_locked)
- disallowed |= VM_LOCKED;
+ if (!allow_locked && vma_test(vma, VMA_LOCKED_BIT))
+ return false;
+ /*
+ * Guard regions require a VMA whose page tables are managed solely by
+ * the core, which is also what merging requires, so disallow any flags
+ * that would prevent a merge.
+ */
+ if (!vma_can_merge(vma))
+ return false;
- return !(vma->vm_flags & disallowed);
+ return true;
}
static bool is_guard_pte_marker(pte_t ptent)
@@ -1559,7 +1565,7 @@ static int madvise_vma_behavior(struct madvise_behavior *madv_behavior)
new_flags |= VM_DONTCOPY;
break;
case MADV_DOFORK:
- if (new_flags & VM_SPECIAL)
+ if (!vma_can_merge(vma))
return -EINVAL;
new_flags &= ~VM_DONTCOPY;
break;
@@ -1578,8 +1584,8 @@ static int madvise_vma_behavior(struct madvise_behavior *madv_behavior)
new_flags |= VM_DONTDUMP;
break;
case MADV_DODUMP:
- if ((!is_vm_hugetlb_page(vma) && (new_flags & VM_SPECIAL)) ||
- (new_flags & VM_DROPPABLE))
+ /* Non-persistent memory cannot be dumped. */
+ if (!vma_is_persistent(vma))
return -EINVAL;
new_flags &= ~VM_DONTDUMP;
break;
diff --git a/mm/memory.c b/mm/memory.c
index 926276d419202..9e4a70421a6b8 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -1564,7 +1564,7 @@ copy_page_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma)
if (!vma_needs_copy(dst_vma, src_vma))
return 0;
- if (is_vm_hugetlb_page(src_vma))
+ if (vma_is_hugetlb(src_vma))
return copy_hugetlb_page_range(dst_mm, src_mm, dst_vma, src_vma);
/*
@@ -2178,7 +2178,7 @@ static void __zap_vma_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
if (vma->vm_file && !reaping)
uprobe_munmap(vma, start, end);
- if (unlikely(is_vm_hugetlb_page(vma))) {
+ if (unlikely(vma_is_hugetlb(vma))) {
zap_flags_t zap_flags = details ? details->zap_flags : 0;
VM_WARN_ON_ONCE(reaping);
@@ -2313,7 +2313,7 @@ void zap_vma_range_batched(struct mmu_gather *tlb,
*/
__zap_vma_range(tlb, vma, address, end, details);
mmu_notifier_invalidate_range_end(&range);
- if (is_vm_hugetlb_page(vma)) {
+ if (vma_is_hugetlb(vma)) {
/*
* flush tlb and free resources before hugetlb_zap_end(), to
* avoid concurrent page faults' allocation failure.
@@ -2343,19 +2343,19 @@ void zap_vma_range(struct vm_area_struct *vma, unsigned long address,
}
/**
- * zap_special_vma_range - zap all page table entries in a special vma range
+ * zap_special_vma_range - zap all page table entries in a kernel-owned VMA
* @vma: the vma covering the range to zap
* @address: starting address of the range to zap
* @size: number of bytes to zap
*
* This function does nothing when the provided address range is not fully
- * contained in @vma, or when the @vma is not VM_PFNMAP or VM_MIXEDMAP.
+ * contained in @vma, or when @vma is not kernel-owned.
*/
void zap_special_vma_range(struct vm_area_struct *vma, unsigned long address,
unsigned long size)
{
if (!range_in_vma(vma, address, address + size) ||
- !(vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP)))
+ !vma_is_kernel_owned(vma))
return;
zap_vma_range(vma, address, size);
@@ -2417,11 +2417,11 @@ static bool vm_mixed_zeropage_allowed(struct vm_area_struct *vma)
* be problematic as soon as the zeropage gets replaced by a different
* page due to vma->vm_ops->pfn_mkwrite, because what's mapped would
* now differ to what GUP looked up. FSDAX is incompatible to
- * FOLL_LONGTERM and VM_IO is incompatible to GUP completely (see
- * check_vma_flags).
+ * FOLL_LONGTERM and memory-mapped I/O is incompatible to GUP completely
+ * (see vma_can_gup()).
*/
return vma->vm_ops && vma->vm_ops->pfn_mkwrite &&
- (vma_is_fsdax(vma) || vma->vm_flags & VM_IO);
+ (vma_is_fsdax(vma) || vma_test(vma, VMA_IO_BIT));
}
static int validate_page_before_insert(struct vm_area_struct *vma,
@@ -2609,17 +2609,23 @@ int vm_insert_pages(struct vm_area_struct *vma, unsigned long addr,
}
EXPORT_SYMBOL(vm_insert_pages);
+static void __map_kernel_pages_prepare(struct vm_area_desc *desc)
+{
+ if (vma_desc_test(desc, VMA_MIXEDMAP_BIT))
+ return;
+
+ VM_WARN_ON_ONCE(mmap_read_trylock(desc->mm));
+ VM_WARN_ON_ONCE(vma_desc_test(desc, VMA_PFNMAP_BIT));
+ vma_desc_set_flags(desc, VMA_MIXEDMAP_BIT);
+}
+
int map_kernel_pages_prepare(struct vm_area_desc *desc)
{
const struct mmap_action *action = &desc->action;
const unsigned long addr = action->map_kernel.start;
unsigned long nr_pages, end;
- if (!vma_desc_test(desc, VMA_MIXEDMAP_BIT)) {
- VM_WARN_ON_ONCE(mmap_read_trylock(desc->mm));
- VM_WARN_ON_ONCE(vma_desc_test(desc, VMA_PFNMAP_BIT));
- vma_desc_set_flags(desc, VMA_MIXEDMAP_BIT);
- }
+ __map_kernel_pages_prepare(desc);
nr_pages = action->map_kernel.nr_pages;
end = addr + PAGE_SIZE * nr_pages;
@@ -2628,7 +2634,6 @@ int map_kernel_pages_prepare(struct vm_area_desc *desc)
return 0;
}
-EXPORT_SYMBOL(map_kernel_pages_prepare);
int map_kernel_pages_complete(struct vm_area_struct *vma,
struct mmap_action *action)
@@ -2640,7 +2645,98 @@ int map_kernel_pages_complete(struct vm_area_struct *vma,
action->map_kernel.pages,
&nr_pages, vma->vm_page_prot);
}
-EXPORT_SYMBOL(map_kernel_pages_complete);
+
+int map_discontig_kernel_pages_prepare(struct vm_area_desc *desc)
+{
+ const struct mmap_action *action = &desc->action;
+ const struct discontig_kernel_page_ops *ops =
+ action->map_kernel_discontig.ops;
+
+ /* At minimum need to be able to get pages. */
+ if (WARN_ON_ONCE(!ops->get))
+ return -EINVAL;
+
+ __map_kernel_pages_prepare(desc);
+ return 0;
+}
+
+static int apply_discontig_action(struct vm_area_struct *vma,
+ struct discontig_kernel_page_state *state)
+{
+ unsigned long nr_pages = state->__nr_pages;
+ unsigned long addr = state->addr;
+ unsigned long i;
+
+ if (state->action == DISCONTIG_KERNEL_PAGE_MAP_PAGE)
+ return insert_page(vma, addr, state->__page,
+ vma->vm_page_prot, /*mkwrite=*/false);
+ if (state->action == DISCONTIG_KERNEL_PAGE_MAP_PAGE_RANGE)
+ return insert_pages(vma, addr, state->__page_arr,
+ &nr_pages, vma->vm_page_prot);
+
+ /* Compound folio - have to iterate through each page. */
+ for (i = 0; i < nr_pages; i++, addr += PAGE_SIZE) {
+ struct page *page = folio_page(state->__folio, i);
+ int err;
+
+ err = insert_page(vma, addr, page, vma->vm_page_prot,
+ /*mkwrite=*/false);
+ if (err)
+ return err;
+ }
+ return 0;
+}
+
+int map_discontig_kernel_pages_complete(struct vm_area_struct *vma,
+ struct mmap_action *action)
+{
+ const struct discontig_kernel_page_ops *ops =
+ action->map_kernel_discontig.ops;
+ struct discontig_kernel_page_state state = {
+ .start = vma->vm_start,
+ .end = vma->vm_end,
+ .addr = vma->vm_start,
+ .pgoff = vma->vm_pgoff,
+ .nr_pages_mapped = 0,
+ .nr_pages_remain = vma_pages(vma),
+ .vm_private_data = vma->vm_private_data,
+ .private = action->map_kernel_discontig.init_private,
+ };
+ int err = 0;
+
+ if (ops->init)
+ err = ops->init(vma->vm_private_data, &state.private);
+ if (err)
+ return err;
+
+ do {
+ unsigned long end, pgoff_end;
+ unsigned long nr_pages;
+
+ /* Default to abort. */
+ state.action = DISCONTIG_KERNEL_PAGE_ABORT;
+ err = ops->get(&state);
+ if (err || state.action == DISCONTIG_KERNEL_PAGE_ABORT)
+ return err;
+ nr_pages = state.__nr_pages;
+
+ if (!nr_pages || nr_pages > state.nr_pages_remain)
+ return -EINVAL;
+ end = state.addr + PAGE_SIZE * nr_pages;
+ pgoff_end = state.pgoff + nr_pages;
+
+ err = apply_discontig_action(vma, &state);
+ if (err)
+ return err;
+
+ state.addr = end;
+ state.pgoff = pgoff_end;
+ state.nr_pages_mapped += nr_pages;
+ state.nr_pages_remain -= nr_pages;
+ } while (state.addr < vma->vm_end);
+
+ return 0;
+}
/**
* vm_insert_page - insert single page into user vma
@@ -6837,7 +6933,7 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
lru_gen_enter_fault(vma);
- if (unlikely(is_vm_hugetlb_page(vma)))
+ if (unlikely(vma_is_hugetlb(vma)))
ret = hugetlb_fault(vma->vm_mm, vma, address, flags);
else
ret = __handle_mm_fault(vma, address, flags);
@@ -7020,7 +7116,8 @@ int follow_pfnmap_start(struct follow_pfnmap_args *args)
if (unlikely(address < vma->vm_start || address >= vma->vm_end))
goto out;
- if (!(vma->vm_flags & (VM_IO | VM_PFNMAP)))
+ /* Only mappings GUP cannot handle are followed here. */
+ if (vma_can_gup(vma))
goto out;
retry:
pgdp = pgd_offset(mm, address);
@@ -7220,8 +7317,9 @@ static int __access_remote_vm(struct mm_struct *mm, unsigned long addr,
}
/*
- * Check if this is a VM_IO | VM_PFNMAP VMA, which
- * we can access using slightly different code.
+ * GUP failed, perhaps because this is a mapping it
+ * cannot handle (see vma_can_gup()) - such mappings may
+ * provide access via vm_ops->access() instead.
*/
bytes = 0;
#ifdef CONFIG_HAVE_IOREMAP_PROT
@@ -7707,12 +7805,12 @@ void ptlock_free(struct ptdesc *ptdesc)
void vma_pgtable_walk_begin(struct vm_area_struct *vma)
{
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
hugetlb_vma_lock_read(vma);
}
void vma_pgtable_walk_end(struct vm_area_struct *vma)
{
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
hugetlb_vma_unlock_read(vma);
}
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 8fc8a975657e6..2fd759e348ca1 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -2013,7 +2013,8 @@ SYSCALL_DEFINE5(get_mempolicy, int __user *, policy,
bool vma_migratable(struct vm_area_struct *vma)
{
- if (vma->vm_flags & (VM_IO | VM_PFNMAP))
+ /* Pages which GUP cannot obtain cannot be migrated either. */
+ if (!vma_can_gup(vma))
return false;
/*
@@ -2023,7 +2024,7 @@ bool vma_migratable(struct vm_area_struct *vma)
if (vma_is_dax(vma))
return false;
- if (is_vm_hugetlb_page(vma) &&
+ if (vma_is_hugetlb(vma) &&
!hugepage_migration_supported(hstate_vma(vma)))
return false;
diff --git a/mm/migrate_device.c b/mm/migrate_device.c
index 0c437004329d9..b74c0ae427682 100644
--- a/mm/migrate_device.c
+++ b/mm/migrate_device.c
@@ -739,19 +739,21 @@ static void migrate_vma_unmap(struct migrate_vma *migrate)
*/
int migrate_vma_setup(struct migrate_vma *args)
{
+ const struct vm_area_struct *vma = args->vma;
long nr_pages = (args->end - args->start) >> PAGE_SHIFT;
args->start &= PAGE_MASK;
args->end &= PAGE_MASK;
- if (!args->vma || is_vm_hugetlb_page(args->vma) ||
- (args->vma->vm_flags & VM_SPECIAL) || vma_is_dax(args->vma))
+ if (!vma)
+ return -EINVAL;
+ if (vma_is_kernel_owned(vma) || vma_is_fixed_mapping(vma) ||
+ vma_is_dax(vma))
return -EINVAL;
if (nr_pages <= 0)
return -EINVAL;
- if (args->start < args->vma->vm_start ||
- args->start >= args->vma->vm_end)
+ if (args->start < vma->vm_start || args->start >= vma->vm_end)
return -EINVAL;
- if (args->end <= args->vma->vm_start || args->end > args->vma->vm_end)
+ if (args->end <= vma->vm_start || args->end > vma->vm_end)
return -EINVAL;
if (!args->src || !args->dst)
return -EINVAL;
diff --git a/mm/mlock.c b/mm/mlock.c
index 39215a3eab1fb..4235a1518fc9e 100644
--- a/mm/mlock.c
+++ b/mm/mlock.c
@@ -316,22 +316,10 @@ static inline unsigned int folio_mlock_step(struct folio *folio,
return folio_pte_batch(folio, pte, ptent, count);
}
-static inline bool allow_mlock_munlock(struct folio *folio,
+static inline bool allow_mlock(struct folio *folio,
struct vm_area_struct *vma, unsigned long start,
unsigned long end, unsigned int step)
{
- /*
- * For unlock, allow munlock large folio which is partially
- * mapped to VMA. As it's possible that large folio is
- * mlocked and VMA is split later.
- *
- * During memory pressure, such kind of large folio can
- * be split. And the pages are not in VM_LOCKed VMA
- * can be reclaimed.
- */
- if (!vma_test(vma, VMA_LOCKED_BIT))
- return true;
-
/* folio_within_range() cannot take KSM, but any small folio is OK */
if (!folio_test_large(folio))
return true;
@@ -352,6 +340,7 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr,
{
struct vm_area_struct *vma = walk->vma;
+ const bool lock = walk->private;
spinlock_t *ptl;
pte_t *start_pte, *pte;
pte_t ptent;
@@ -368,7 +357,7 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr,
folio = pmd_folio(*pmd);
if (folio_is_zone_device(folio))
goto out;
- if (vma_test(vma, VMA_LOCKED_BIT))
+ if (lock)
mlock_folio(folio);
else
munlock_folio(folio);
@@ -390,10 +379,10 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr,
continue;
step = folio_mlock_step(folio, pte, addr, end);
- if (!allow_mlock_munlock(folio, vma, start, end, step))
+ if (lock && !allow_mlock(folio, vma, start, end, step))
goto next_entry;
- if (vma_test(vma, VMA_LOCKED_BIT))
+ if (lock)
mlock_folio(folio);
else
munlock_folio(folio);
@@ -428,31 +417,29 @@ static void mlock_vma_pages_range(struct vm_area_struct *vma,
.pmd_entry = mlock_pte_range,
.walk_lock = PGWALK_WRLOCK_VERIFY,
};
+ const bool lock = vma_flags_test(new_vma_flags, VMA_LOCKED_BIT);
+ vma_flags_t walk_flags = *new_vma_flags;
/*
- * There is a slight chance that concurrent page migration,
- * or page reclaim finding a page of this now-VMA_LOCKED_BIT vma,
- * will call mlock_vma_folio() and raise page's mlock_count:
- * double counting, leaving the page unevictable indefinitely.
- * Communicate this danger to mlock_vma_folio() with VMA_IO_BIT,
- * which is a VMA_SPECIAL_FLAGS flag not allowed on VMA_LOCKED_BIT vmas.
- * mmap_lock is held in write mode here, so this weird
- * combination should not be visible to other mmap_lock users;
- * but WRITE_ONCE so rmap walkers must see VMA_IO_BIT if VMA_LOCKED_BIT.
+ * LOCKONFAULT without LOCKED never otherwise occurs: it marks a walk in
+ * progress so that rmap-side callers, which test VMA_LOCKED_BIT, do not
+ * count folios, while try_to_unmap_one(), which tests VMA_LOCKED_MASK,
+ * still refuses to unmap them.
*/
- if (vma_flags_test(new_vma_flags, VMA_LOCKED_BIT))
- vma_flags_set(new_vma_flags, VMA_IO_BIT);
+ if (lock) {
+ vma_flags_clear(&walk_flags, VMA_LOCKED_BIT);
+ vma_flags_set(&walk_flags, VMA_LOCKONFAULT_BIT);
+ }
+
vma_start_write(vma);
- vma_flags_reset_once(vma, new_vma_flags);
+ vma_flags_reset_once(vma, &walk_flags);
lru_add_drain();
- walk_page_range_vma(vma, start, end, &mlock_walk_ops, NULL);
+ walk_page_range_vma(vma, start, end, &mlock_walk_ops, (void *)lock);
lru_add_drain();
- if (vma_flags_test(new_vma_flags, VMA_IO_BIT)) {
- vma_flags_clear(new_vma_flags, VMA_IO_BIT);
+ if (lock)
vma_flags_reset_once(vma, new_vma_flags);
- }
}
/*
diff --git a/mm/mmap.c b/mm/mmap.c
index 4bf26b0f1e6e3..98449f364af1c 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1786,7 +1786,7 @@ __latent_entropy int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)
/*
* Copy/update hugetlb private vma information.
*/
- if (is_vm_hugetlb_page(tmp))
+ if (vma_is_hugetlb(tmp))
hugetlb_dup_vma_private(tmp);
/*
diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c
index 2a72a9686773a..9f353f0e2ef4d 100644
--- a/mm/mmu_gather.c
+++ b/mm/mmu_gather.c
@@ -480,7 +480,7 @@ void tlb_gather_mmu_vma(struct mmu_gather *tlb, struct vm_area_struct *vma)
{
tlb_gather_mmu(tlb, vma->vm_mm);
tlb_update_vma_flags(tlb, vma);
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
/* All entries have the same size. */
tlb_change_page_size(tlb, huge_page_size(hstate_vma(vma)));
}
diff --git a/mm/mprotect.c b/mm/mprotect.c
index 2888ee638d872..a1b6d29bf0390 100644
--- a/mm/mprotect.c
+++ b/mm/mprotect.c
@@ -717,7 +717,7 @@ long change_protection(struct mmu_gather *tlb,
(cp_flags & MM_CP_UFFD_RWP))
newprot = PAGE_NONE;
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
pages = hugetlb_change_protection(vma, start, end, newprot,
cp_flags);
else
@@ -783,8 +783,7 @@ mprotect_fixup(struct vma_iterator *vmi, struct mmu_gather *tlb,
* uncommon case, so doesn't need to be very optimized.
*/
if (arch_has_pfn_modify_check() &&
- vma_flags_test_any(&old_vma_flags, VMA_PFNMAP_BIT,
- VMA_MIXEDMAP_BIT) &&
+ vma_flags_is_kernel_owned(&old_vma_flags) &&
!vma_flags_test_any_mask(&new_vma_flags, VMA_ACCESS_FLAGS)) {
pgprot_t new_pgprot = vm_get_page_prot(newflags);
diff --git a/mm/mremap.c b/mm/mremap.c
index 7c368440fafe2..1122282a1d6ab 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -812,7 +812,7 @@ unsigned long move_page_tables(struct pagetable_move_control *pmc)
if (!pmc->len_in)
return 0;
- if (is_vm_hugetlb_page(pmc->old))
+ if (vma_is_hugetlb(pmc->old))
return move_hugetlb_page_tables(pmc->old, pmc->new, pmc->old_addr,
pmc->new_addr, pmc->len_in);
@@ -1735,7 +1735,7 @@ static bool vma_multi_allowed(struct vm_area_struct *vma)
/* Known good. */
if (vma_is_shmem(vma))
return true;
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
return true;
if (file->f_op->get_unmapped_area == thp_get_unmapped_area)
return true;
@@ -1758,7 +1758,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm)
return -EPERM;
/* Align to hugetlb page size, if required. */
- if (is_vm_hugetlb_page(vma) && !align_hugetlb(vrm))
+ if (vma_is_hugetlb(vma) && !align_hugetlb(vrm))
return -EINVAL;
vrm_set_delta(vrm);
@@ -1788,8 +1788,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm)
return -EINVAL;
}
- if ((vrm->flags & MREMAP_DONTUNMAP) &&
- vma_test_any(vma, VMA_DONTEXPAND_BIT, VMA_PFNMAP_BIT))
+ if ((vrm->flags & MREMAP_DONTUNMAP) && vma_is_fixed_mapping(vma))
return -EINVAL;
/*
@@ -1827,7 +1826,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm)
if (pgoff + (new_len >> PAGE_SHIFT) < pgoff)
return -EINVAL;
- if (vma_test_any(vma, VMA_DONTEXPAND_BIT, VMA_PFNMAP_BIT))
+ if (vma_is_fixed_mapping(vma))
return -EFAULT;
if (!mlock_future_ok(mm, vma_test(vma, VMA_LOCKED_BIT), vrm->delta))
diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c
index 28e306fdb3a5b..8408aee7571b5 100644
--- a/mm/page_vma_mapped.c
+++ b/mm/page_vma_mapped.c
@@ -109,7 +109,7 @@ static bool check_pte(struct page_vma_mapped_walk *pvmw, unsigned long pte_nr)
unsigned long pfn;
pte_t ptent;
- if (is_vm_hugetlb_page(pvmw->vma))
+ if (vma_is_hugetlb(pvmw->vma))
ptent = huge_ptep_get(pvmw->vma->vm_mm, pvmw->address,
pvmw->pte);
else
@@ -206,7 +206,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw)
if (pvmw->pmd && !pvmw->pte)
return not_found(pvmw);
- if (unlikely(is_vm_hugetlb_page(vma))) {
+ if (unlikely(vma_is_hugetlb(vma))) {
struct hstate *hstate = hstate_vma(vma);
unsigned long size = huge_page_size(hstate);
/* The only possible mapping was handled on last iteration */
diff --git a/mm/pagewalk.c b/mm/pagewalk.c
index 7411702a37f58..e6493bbe6919e 100644
--- a/mm/pagewalk.c
+++ b/mm/pagewalk.c
@@ -408,7 +408,7 @@ static int __walk_page_range(unsigned long start, unsigned long end,
int err = 0;
struct vm_area_struct *vma = walk->vma;
const struct mm_walk_ops *ops = walk->ops;
- bool is_hugetlb = is_vm_hugetlb_page(vma);
+ bool is_hugetlb = vma_is_hugetlb(vma);
/* We do not support hugetlb PTE installation. */
if (ops->install_pte && is_hugetlb)
diff --git a/mm/rmap.c b/mm/rmap.c
index 5332c52909be1..6661bc11ce658 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -2239,9 +2239,11 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
/*
* If the folio is in an mlock()d vma, we must not swap it out.
+ * VMA_LOCKONFAULT_BIT alone marks an mlock walk in progress, see
+ * mlock_vma_pages_range().
*/
if (!(flags & TTU_IGNORE_MLOCK) &&
- (vma->vm_flags & VM_LOCKED)) {
+ vma_test_any_mask(vma, VMA_LOCKED_MASK)) {
ptes++;
/*
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 2cd0d0ba966c3..c1c5fbb3c909d 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -2707,7 +2707,7 @@ static int unuse_mm(struct mm_struct *mm, unsigned int type)
if (check_stable_address_space(mm))
goto unlock;
for_each_vma(vmi, vma) {
- if (vma->anon_vma && !is_vm_hugetlb_page(vma)) {
+ if (vma->anon_vma && !vma_is_hugetlb(vma)) {
ret = unuse_vma(vma, type);
if (ret)
break;
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 79cc7b546f130..ddf0a4a3d3997 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -237,7 +237,7 @@ static int mfill_get_vma(struct mfill_state *state)
if ((flags & MFILL_ATOMIC_WP) && !(dst_vma->vm_flags & VM_UFFD_WP))
goto out_unlock;
- if (is_vm_hugetlb_page(dst_vma))
+ if (vma_is_hugetlb(dst_vma))
return 0;
ops = vma_uffd_ops(dst_vma);
@@ -804,7 +804,7 @@ static __always_inline ssize_t mfill_atomic_hugetlb(
}
err = -ENOENT;
- if (!is_vm_hugetlb_page(dst_vma))
+ if (!vma_is_hugetlb(dst_vma))
goto out_unlock_vma;
err = -EINVAL;
@@ -967,7 +967,7 @@ static __always_inline ssize_t mfill_atomic(struct userfaultfd_ctx *ctx,
/*
* If this is a HUGETLB vma, pass off to appropriate routine
*/
- if (is_vm_hugetlb_page(state.vma))
+ if (vma_is_hugetlb(state.vma))
return mfill_atomic_hugetlb(ctx, state.vma, dst_start,
src_start, len, flags);
@@ -1114,7 +1114,7 @@ static int mwriteprotect_range(struct userfaultfd_ctx *ctx, unsigned long start,
break;
}
- if (is_vm_hugetlb_page(dst_vma)) {
+ if (vma_is_hugetlb(dst_vma)) {
err = -EINVAL;
page_mask = vma_kernel_pagesize(dst_vma) - 1;
if ((start & page_mask) || (len & page_mask))
@@ -1172,7 +1172,7 @@ int mrwprotect_range(struct userfaultfd_ctx *ctx, unsigned long start,
if (!userfaultfd_rwp(dst_vma))
return -ENOENT;
- if (is_vm_hugetlb_page(dst_vma)) {
+ if (vma_is_hugetlb(dst_vma)) {
unsigned long page_mask;
page_mask = vma_kernel_pagesize(dst_vma) - 1;
@@ -1754,10 +1754,18 @@ static inline bool move_splits_huge_pmd(unsigned long dst_addr,
}
#endif
-static inline bool vma_move_compatible(struct vm_area_struct *vma)
+static inline bool vma_move_compatible(const struct vm_area_struct *vma)
{
- return !(vma->vm_flags & (VM_PFNMAP | VM_IO | VM_HUGETLB |
- VM_MIXEDMAP | VM_SHADOW_STACK));
+ /* uffd is generally incompatible with kernel-owned mappings. */
+ if (vma_is_kernel_owned(vma))
+ return false;
+ /* The shadow stack should not be written to by userspace. */
+ if (vma_test_single_mask(vma, VMA_SHADOW_STACK))
+ return false;
+ /* hugetlb mappings cannot be safely moved. */
+ if (vma_is_hugetlb(vma))
+ return false;
+ return true;
}
static int validate_move_areas(struct userfaultfd_ctx *ctx,
@@ -2146,10 +2154,11 @@ static bool vma_can_userfault(struct vm_area_struct *vma, vm_flags_t vm_flags,
{
const struct vm_uffd_ops *ops = vma_uffd_ops(vma);
- if (vma->vm_flags & (VM_DROPPABLE | VM_SHADOW_STACK))
+ /* Non-persistent memory is inherently not controllable by userspace. */
+ if (!vma_is_persistent(vma))
return false;
-
- if (!is_vm_hugetlb_page(vma) && (vma->vm_flags & VM_SPECIAL))
+ /* The shadow stack should not be written to by userspace. */
+ if (vma_test_single_mask(vma, VMA_SHADOW_STACK))
return false;
vm_flags &= __VM_UFFD_FLAGS;
@@ -2319,7 +2328,7 @@ static int userfaultfd_register_range(struct userfaultfd_ctx *ctx,
*/
userfaultfd_set_ctx(vma, ctx, vm_flags);
- if (is_vm_hugetlb_page(vma) && uffd_disable_huge_pmd_share(vma))
+ if (vma_is_hugetlb(vma) && uffd_disable_huge_pmd_share(vma))
hugetlb_unshare_all_pmds(vma);
skip:
@@ -2895,7 +2904,7 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason)
* (sleepable) vma lock can modify the current task state, that
* must be before explicitly calling set_current_state().
*/
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
hugetlb_vma_lock_read(vma);
spin_lock_irq(&ctx->fault_pending_wqh.lock);
@@ -2912,7 +2921,7 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason)
set_current_state(blocking_state);
spin_unlock_irq(&ctx->fault_pending_wqh.lock);
- if (is_vm_hugetlb_page(vma)) {
+ if (vma_is_hugetlb(vma)) {
must_wait = userfaultfd_huge_must_wait(ctx, vmf, reason);
hugetlb_vma_unlock_read(vma);
} else {
@@ -3744,7 +3753,7 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,
* If the first vma contains huge pages, make sure start address
* is aligned to huge page size.
*/
- if (is_vm_hugetlb_page(vma)) {
+ if (vma_is_hugetlb(vma)) {
unsigned long vma_hpagesize = vma_kernel_pagesize(vma);
if (start & (vma_hpagesize - 1))
@@ -3795,7 +3804,7 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,
* If this vma contains ending address, and huge pages
* check alignment.
*/
- if (is_vm_hugetlb_page(cur) && end <= cur->vm_end &&
+ if (vma_is_hugetlb(cur) && end <= cur->vm_end &&
end > cur->vm_start) {
unsigned long vma_hpagesize = vma_kernel_pagesize(cur);
@@ -3831,7 +3840,7 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,
/*
* Note vmas containing huge pages
*/
- if (is_vm_hugetlb_page(cur))
+ if (vma_is_hugetlb(cur))
basic_ioctls = true;
found = true;
@@ -3917,7 +3926,7 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx,
* If the first vma contains huge pages, make sure start address
* is aligned to huge page size.
*/
- if (is_vm_hugetlb_page(vma)) {
+ if (vma_is_hugetlb(vma)) {
unsigned long vma_hpagesize = vma_kernel_pagesize(vma);
if (start & (vma_hpagesize - 1))
diff --git a/mm/util.c b/mm/util.c
index bf0513d1d3d08..c5ee52aede1e4 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -1224,16 +1224,28 @@ EXPORT_SYMBOL(compat_set_desc_from_vma);
int __compat_vma_mmap(struct vm_area_desc *desc,
struct vm_area_struct *vma)
{
+ struct vm_area_desc prev_desc;
int err;
+ /* Derive state prior to mmap_prepare hook. */
+ compat_set_desc_from_vma(&prev_desc, desc->file, vma);
/* Perform any preparatory tasks for mmap action. */
err = mmap_action_prepare(desc);
if (err)
- return err;
+ goto err_put;
+ /* Check the caller did nothing crazy. */
+ err = mmap_prepare_validate(&prev_desc, desc);
+ if (err)
+ goto err_put;
/* Update the VMA from the descriptor. */
compat_set_vma_from_desc(vma, desc);
/* Complete any specified mmap actions. */
return mmap_action_complete(vma, &desc->action, /*is_compat=*/true);
+
+err_put:
+ if (desc->vm_file != vma->vm_file)
+ fput(desc->vm_file);
+ return err;
}
EXPORT_SYMBOL(__compat_vma_mmap);
@@ -1455,8 +1467,10 @@ int mmap_action_prepare(struct vm_area_desc *desc)
return io_remap_pfn_range_prepare(desc);
case MMAP_SIMPLE_IO_REMAP:
return simple_ioremap_prepare(desc);
- case MMAP_MAP_KERNEL_PAGES:
+ case MMAP_KERNEL_PAGES:
return map_kernel_pages_prepare(desc);
+ case MMAP_DISCONTIG_KERNEL_PAGES:
+ return map_discontig_kernel_pages_prepare(desc);
}
WARN_ON_ONCE(1);
@@ -1486,9 +1500,12 @@ int mmap_action_complete(struct vm_area_struct *vma,
case MMAP_REMAP_PFN:
err = remap_pfn_range_complete(vma, action);
break;
- case MMAP_MAP_KERNEL_PAGES:
+ case MMAP_KERNEL_PAGES:
err = map_kernel_pages_complete(vma, action);
break;
+ case MMAP_DISCONTIG_KERNEL_PAGES:
+ err = map_discontig_kernel_pages_complete(vma, action);
+ break;
case MMAP_IO_REMAP_PFN:
case MMAP_SIMPLE_IO_REMAP:
/* Should have been delegated. */
@@ -1509,7 +1526,8 @@ int mmap_action_prepare(struct vm_area_desc *desc)
case MMAP_REMAP_PFN:
case MMAP_IO_REMAP_PFN:
case MMAP_SIMPLE_IO_REMAP:
- case MMAP_MAP_KERNEL_PAGES:
+ case MMAP_KERNEL_PAGES:
+ case MMAP_DISCONTIG_KERNEL_PAGES:
WARN_ON_ONCE(1); /* nommu cannot handle these. */
break;
}
@@ -1530,7 +1548,8 @@ int mmap_action_complete(struct vm_area_struct *vma,
case MMAP_REMAP_PFN:
case MMAP_IO_REMAP_PFN:
case MMAP_SIMPLE_IO_REMAP:
- case MMAP_MAP_KERNEL_PAGES:
+ case MMAP_KERNEL_PAGES:
+ case MMAP_DISCONTIG_KERNEL_PAGES:
WARN_ON_ONCE(1); /* nommu cannot handle this. */
err = -EINVAL;
diff --git a/mm/vma.c b/mm/vma.c
index 55917d0979339..777656306705d 100644
--- a/mm/vma.c
+++ b/mm/vma.c
@@ -24,7 +24,8 @@ struct mmap_state {
vm_flags_t vm_flags;
vma_flags_t vma_flags;
};
- struct file *file;
+ struct file *file; /* mmap()-specified file. */
+ struct file *vm_file; /* May be updated by mmap_prepare. */
pgprot_t page_prot;
/* User-defined fields, perhaps updated by .mmap_prepare(). */
@@ -43,8 +44,6 @@ struct mmap_state {
/* Determine if we can check KSM flags early in mmap() logic. */
bool check_ksm_early :1;
- /* If .mmap_prepare changed the file, we don't need to pin. */
- bool file_doesnt_need_get :1;
};
#define MMAP_STATE(name, mm_, vmi_, addr_, len_, pgoff_, anon_pgoff_, vma_flags_, file_) \
@@ -58,6 +57,7 @@ struct mmap_state {
.pglen = PHYS_PFN(len_), \
.vma_flags = vma_flags_, \
.file = file_, \
+ .vm_file = file_, \
.page_prot = vma_flags_to_page_prot(vma_flags_), \
}
@@ -70,7 +70,7 @@ struct mmap_state {
.vma_flags = (map_)->vma_flags, \
.pgoff = (map_)->pgoff, \
.anon_pgoff = (map_)->anon_pgoff, \
- .file = (map_)->file, \
+ .file = (map_)->vm_file, \
.prev = (map_)->prev, \
.middle = vma_, \
.next = (vma_) ? NULL : (map_)->next, \
@@ -599,7 +599,7 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,
* boundary.
*/
vma_adjust_trans_huge(vma, vma->vm_start, addr, NULL);
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
hugetlb_split(vma, addr);
if (new_below) {
@@ -924,13 +924,14 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(
vmg->state = VMA_MERGE_NOMERGE;
+ if (!vma_flags_can_merge(&vmg->vma_flags))
+ return NULL;
/*
- * If a special mapping or if the range being modified is neither at the
- * furthermost left or right side of the VMA, then we have no chance of
- * merging and should abort.
+ * If the range being modified is neither at the furthermost left or
+ * right side of the VMA, then we have no chance of merging and should
+ * abort.
*/
- if (vma_flags_test_any_mask(&vmg->vma_flags, VMA_SPECIAL_FLAGS) ||
- (!left_side && !right_side))
+ if (!left_side && !right_side)
return NULL;
if (left_side)
@@ -1152,9 +1153,11 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)
vmg->state = VMA_MERGE_NOMERGE;
- /* Special VMAs are unmergeable, also if no prev/next. */
- if (vma_flags_test_any_mask(&vmg->vma_flags, VMA_SPECIAL_FLAGS) ||
- (!prev && !next))
+ if (!vma_flags_can_merge(&vmg->vma_flags))
+ return NULL;
+
+ /* VMAs with no prev/next are unmergeable. */
+ if (!prev && !next)
return NULL;
can_merge_left = can_vma_merge_left(vmg);
@@ -2233,7 +2236,7 @@ bool vma_wants_writenotify(struct vm_area_struct *vma, pgprot_t vm_page_prot)
* Do we need to track softdirty? hugetlb does not support softdirty
* tracking yet.
*/
- if (vma_soft_dirty_enabled(vma) && !is_vm_hugetlb_page(vma))
+ if (vma_soft_dirty_enabled(vma) && !vma_is_hugetlb(vma))
return true;
/* Do we need write faults for uffd-wp tracking? */
@@ -2352,7 +2355,7 @@ int mm_take_all_locks(struct mm_struct *mm)
if (signal_pending(current))
goto out_unlock;
if (vma->vm_file && vma->vm_file->f_mapping &&
- is_vm_hugetlb_page(vma))
+ vma_is_hugetlb(vma))
vm_lock_mapping(mm, vma->vm_file->f_mapping);
}
@@ -2361,7 +2364,7 @@ int mm_take_all_locks(struct mm_struct *mm)
if (signal_pending(current))
goto out_unlock;
if (vma->vm_file && vma->vm_file->f_mapping &&
- !is_vm_hugetlb_page(vma))
+ !vma_is_hugetlb(vma))
vm_lock_mapping(mm, vma->vm_file->f_mapping);
}
@@ -2447,7 +2450,7 @@ void mm_drop_all_locks(struct mm_struct *mm)
*/
static bool accountable_mapping(struct mmap_state *map)
{
- const struct file *file = map->file;
+ const struct file *file = map->vm_file;
/*
* hugetlb has its own accounting separate from the core VM
@@ -2496,7 +2499,7 @@ static void vms_abort_munmap_vmas(struct vma_munmap_struct *vms,
static void update_ksm_flags(struct mmap_state *map)
{
- map->vma_flags = ksm_vma_flags(map->mm, map->file, map->vma_flags);
+ map->vma_flags = ksm_vma_flags(map->mm, map->vm_file, map->vma_flags);
}
static void set_desc_from_map(struct vm_area_desc *desc,
@@ -2506,7 +2509,7 @@ static void set_desc_from_map(struct vm_area_desc *desc,
desc->end = map->end;
desc->pgoff = map->pgoff;
- desc->vm_file = map->file;
+ desc->vm_file = map->vm_file;
desc->vma_flags = map->vma_flags;
desc->page_prot = map->page_prot;
}
@@ -2586,6 +2589,10 @@ static int __mmap_setup(struct mmap_state *map, struct vm_area_desc *desc,
return 0;
}
+static bool map_same_file(struct mmap_state *map)
+{
+ return map->vm_file == map->file;
+}
static int __mmap_new_file_vma(struct mmap_state *map,
struct vm_area_struct *vma)
@@ -2593,37 +2600,43 @@ static int __mmap_new_file_vma(struct mmap_state *map,
struct vma_iterator *vmi = map->vmi;
int error;
- vma->vm_file = map->file;
- if (!map->file_doesnt_need_get)
- get_file(map->file);
+ vma->vm_file = map->vm_file;
+ if (map_same_file(map))
+ get_file(map->vm_file);
- if (!map->file->f_op->mmap)
+ if (!map->vm_file->f_op->mmap)
return 0;
+ /*
+ * Driver-specified flags may make the lock flags invalid, so clear
+ * VMA_LOCKED_MASK and reinstate it afterwards if appropriate.
+ */
+ vma_clear_flags_mask(vma, VMA_LOCKED_MASK);
error = mmap_file(vma->vm_file, vma);
+ map->vm_file = vma->vm_file;
+
if (error) {
UNMAP_STATE(unmap, vmi, vma, vma->vm_start, vma->vm_end,
map->prev, map->next);
- fput(vma->vm_file);
- vma->vm_file = NULL;
+ if (map_same_file(map))
+ fput(map->vm_file);
+ vma->vm_file = NULL;
vma_iter_set(vmi, vma->vm_end);
/* Undo any partial mapping done by a device driver. */
unmap_region(&unmap);
return error;
}
- /* Drivers cannot alter the address of the VMA. */
- WARN_ON_ONCE(map->addr != vma->vm_start);
- /*
- * Drivers should not permit writability when previously it was
- * disallowed.
- */
- VM_WARN_ON_ONCE(!vma_flags_same_pair(&map->vma_flags, &vma->flags) &&
- !vma_flags_test(&map->vma_flags, VMA_MAYWRITE_BIT) &&
- vma_test(vma, VMA_MAYWRITE_BIT));
+ /* If VMA flags still valid for locked mask, reinstate. */
+ if (vma_supports_mlock(vma)) {
+ const vma_flags_t mask =
+ vma_flags_and_mask(&map->vma_flags,
+ VMA_LOCKED_MASK);
+
+ vma_set_flags_mask(vma, mask);
+ }
- map->file = vma->vm_file;
map->vma_flags = vma->flags;
return 0;
@@ -2631,7 +2644,7 @@ static int __mmap_new_file_vma(struct mmap_state *map,
static void map_set_anon(struct mmap_state *map)
{
- map->file = NULL;
+ map->vm_file = NULL;
map->vm_ops = NULL;
map->pgoff = map->addr >> PAGE_SHIFT;
}
@@ -2643,7 +2656,7 @@ static bool map_is_private(const struct mmap_state *map)
static bool map_is_anon(const struct mmap_state *map)
{
- return map_is_private(map) && !map->file;
+ return map_is_private(map) && !map->vm_file;
}
/*
@@ -2688,7 +2701,7 @@ static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap,
}
/* Invoke callbacks. */
- if (map->file)
+ if (map->vm_file)
error = __mmap_new_file_vma(map, vma);
else if (!is_anon)
error = shmem_zero_setup(vma);
@@ -2701,11 +2714,6 @@ static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap,
vma->flags = map->vma_flags;
}
-#ifdef CONFIG_SPARC64
- /* TODO: Fix SPARC ADI! */
- WARN_ON_ONCE(!arch_validate_flags(map->vm_flags));
-#endif
-
/* Lock the VMA since it is modified after insertion into VMA tree */
vma_start_write(vma);
vma_iter_store_new(vmi, vma);
@@ -2768,6 +2776,96 @@ static void __mmap_complete(struct mmap_state *map, struct vm_area_struct *vma)
vma_set_page_prot(vma);
}
+/* Check to ensure that the VMA flags of a newly mapped VMA are sane. */
+static int mmap_validate_vma_flags(const vma_flags_t *flags)
+{
+#ifdef CONFIG_SPARC64
+ const vm_flags_t legacy_flags = vma_flags_to_legacy(*flags);
+
+ /* TODO: Fix SPARC ADI! */
+ if (WARN_ON_ONCE(!arch_validate_flags(legacy_flags)))
+ return -EINVAL;
+#endif
+
+ if (!vma_flags_is_kernel_owned(flags)) {
+ /* Only kernel-owned mappings may set VMA_IO_BIT. */
+ if (WARN_ON_ONCE(vma_flags_test(flags, VMA_IO_BIT)))
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+/* Check to ensure a driver hasn't done something crazy. */
+static int mmap_validate(unsigned long prev_start,
+ unsigned long curr_start,
+ const vma_flags_t *prev_flags,
+ const vma_flags_t *curr_flags)
+{
+ bool was_maywrite, is_maywrite;
+
+ /* Drivers cannot alter the address of the VMA. */
+ if (WARN_ON_ONCE(prev_start != curr_start))
+ return -EINVAL;
+
+ was_maywrite = vma_flags_test(prev_flags, VMA_MAYWRITE_BIT);
+ is_maywrite = vma_flags_test(curr_flags, VMA_MAYWRITE_BIT);
+
+ /* A driver may not make a previously unwritable mapping writable. */
+ if (WARN_ON_ONCE(!was_maywrite && is_maywrite))
+ return -EINVAL;
+
+ /* Only kernel-owned mappings may clear VMA_MAYWRITE_BIT. */
+ if (!vma_flags_is_kernel_owned(curr_flags) &&
+ WARN_ON_ONCE(was_maywrite && !is_maywrite))
+ return -EINVAL;
+
+ return mmap_validate_vma_flags(curr_flags);
+}
+
+/**
+ * mmap_prepare_validate() - Ensure the driver hasn't violated invariants in its
+ * f_op->mmap_prepare hook.
+ * @prev_desc: The VMA descriptor prior to the mmap_prepare hook being called.
+ * @desc: The VMA descriptor after the mmap_prepare hook has been called.
+ *
+ * Returns: 0 on success, otherwise an error.
+ */
+int mmap_prepare_validate(const struct vm_area_desc *prev_desc,
+ const struct vm_area_desc *desc)
+{
+ /*
+ * It is not valid to execute mmap actions for VMAs which can be merged,
+ * as any such merge would leave portions of the mapping incorrectly
+ * unmapped.
+ */
+ if (vma_flags_can_merge(&desc->vma_flags) &&
+ WARN_ON_ONCE(desc->action.type != MMAP_NOTHING))
+ return -EINVAL;
+
+ return mmap_validate(prev_desc->start, desc->start,
+ &prev_desc->vma_flags, &desc->vma_flags);
+}
+
+/**
+ * mmap_hook_validate() - Ensure the driver hasn't violated invariants in
+ * its f_op->mmap hook.
+ * @prev_start: The start of the mapping prior to the mmap hook.
+ * @prev_flags: The VMA flags set for the VMA prior to the mmap hook.
+ * @vma: The VMA after the hook has been applied.
+ *
+ * Returns: 0 on success, otherwise an error.
+ */
+int mmap_hook_validate(unsigned long prev_start,
+ const vma_flags_t *prev_flags,
+ const struct vm_area_struct *vma)
+{
+ const unsigned long start = vma->vm_start;
+ const vma_flags_t *flags = &vma->flags;
+
+ return mmap_validate(prev_start, start, prev_flags, flags);
+}
+
static int call_action_prepare(struct mmap_state *map,
struct vm_area_desc *desc)
{
@@ -2794,39 +2892,43 @@ static int call_action_prepare(struct mmap_state *map,
static int call_mmap_prepare(struct mmap_state *map,
struct vm_area_desc *desc)
{
+ const struct vm_area_desc prev_desc = *desc;
int err;
/* Invoke the hook. */
- err = vfs_mmap_prepare(map->file, desc);
- if (err)
- return err;
-
- /* It's invalid for mmap_preprare hooks to clear vm_ops. */
- if (!desc->vm_ops)
- return -EINVAL;
-
- err = call_action_prepare(map, desc);
+ err = vfs_mmap_prepare(map->vm_file, desc);
if (err)
return err;
/* Update fields permitted to be changed. */
map->pgoff = desc->pgoff;
- if (desc->vm_file != map->file) {
- map->file_doesnt_need_get = true;
- map->file = desc->vm_file;
- }
+ if (desc->vm_file != map->vm_file)
+ map->vm_file = desc->vm_file;
map->vma_flags = desc->vma_flags;
map->page_prot = desc->page_prot;
/* User-defined fields. */
map->vm_ops = desc->vm_ops;
map->vm_private_data = desc->private_data;
+ /* It's invalid for mmap_prepare hooks to clear vm_ops. */
+ if (!desc->vm_ops)
+ return -EINVAL;
+
+ err = call_action_prepare(map, desc);
+ if (err)
+ return err;
+
+ /* Check the caller did nothing crazy. */
+ err = mmap_prepare_validate(&prev_desc, desc);
+ if (err)
+ return err;
+
/*
* MAP_PRIVATE-/dev/zero mappings are an ancient way of getting
* anonymous mappings. Rather than allowing these mappings to be odd
* outliers, simply make them truly anonymous.
*/
- if (map_is_private(map) && file_is_dev_zero(map->file))
+ if (map_is_private(map) && file_is_dev_zero(map->vm_file))
map_set_anon(map);
return 0;
@@ -2845,7 +2947,7 @@ static void set_vma_user_defined_fields(struct vm_area_struct *vma,
*/
static bool can_set_ksm_flags_early(struct mmap_state *map)
{
- struct file *file = map->file;
+ struct file *file = map->vm_file;
/* Anonymous mappings have no driver which can change them. */
if (!file)
@@ -2868,13 +2970,27 @@ static bool can_set_ksm_flags_early(struct mmap_state *map)
return false;
}
+static void put_map(struct mmap_state *map)
+{
+ /*
+ * An error occurred or the VMA was merged.
+ *
+ * If the file was changed by the driver (which is required to increment
+ * the replacement file's reference count), drop its reference count.
+ *
+ * On error, the caller always drops the original file regardless.
+ */
+ if (map->vm_file && !map_same_file(map))
+ fput(map->vm_file);
+}
+
static unsigned long __mmap_region(struct file *file, unsigned long addr,
unsigned long len, vma_flags_t vma_flags,
unsigned long pgoff, struct list_head *uf)
{
struct mm_struct *mm = current->mm;
struct vm_area_struct *vma = NULL;
- bool have_mmap_prepare = file && file->f_op->mmap_prepare;
+ const bool have_mmap_prepare = file && file->f_op->mmap_prepare;
VMA_ITERATOR(vmi, mm, addr);
const pgoff_t anon_pgoff = addr >> PAGE_SHIFT;
MMAP_STATE(map, mm, &vmi, addr, len, pgoff, anon_pgoff, vma_flags, file);
@@ -2917,12 +3033,15 @@ static unsigned long __mmap_region(struct file *file, unsigned long addr,
allocated_new = true;
}
- if (have_mmap_prepare && !map_is_anon(&map))
+ if (have_mmap_prepare && allocated_new && !map_is_anon(&map))
set_vma_user_defined_fields(vma, &map);
__mmap_complete(&map, vma);
- if (have_mmap_prepare && allocated_new) {
+ if (!allocated_new) {
+ /* Merged, so need to drop refcount. */
+ put_map(&map);
+ } else if (have_mmap_prepare) {
error = mmap_action_complete(vma, &desc.action,
/*is_compat=*/false);
if (error)
@@ -2936,13 +3055,7 @@ static unsigned long __mmap_region(struct file *file, unsigned long addr,
if (map.charged)
vm_unacct_memory(map.charged);
abort_munmap:
- /*
- * This indicates that .mmap_prepare has set a new file, differing from
- * desc->vm_file. But since we're aborting the operation, only the
- * original file will be cleaned up. Ensure we clean up both.
- */
- if (map.file_doesnt_need_get)
- fput(map.file);
+ put_map(&map);
vms_abort_munmap_vmas(&map.vms, &map.mas_detach);
return error;
}
@@ -3437,10 +3550,15 @@ int __vm_munmap(unsigned long start, size_t len, bool unlock)
int insert_vm_struct(struct mm_struct *mm, struct vm_area_struct *vma)
{
unsigned long charged = vma_pages(vma);
+ int err;
if (find_vma_intersection(mm, vma->vm_start, vma->vm_end))
return -ENOMEM;
+ err = mmap_validate_vma_flags(&vma->flags);
+ if (err)
+ return err;
+
if (vma_test(vma, VMA_ACCOUNT_BIT) &&
security_vm_enough_memory_mm(mm, charged))
return -ENOMEM;
diff --git a/mm/vma.h b/mm/vma.h
index e97bd2dfa786d..77d395b8b1032 100644
--- a/mm/vma.h
+++ b/mm/vma.h
@@ -394,8 +394,10 @@ static inline void compat_set_vma_from_desc(struct vm_area_struct *vma,
/* Mutable fields. Populated with initial state. */
vma_set_pgoff(vma, desc->pgoff);
- if (desc->vm_file != vma->vm_file)
- vma_set_file(vma, desc->vm_file);
+ if (desc->vm_file != vma->vm_file) {
+ fput(vma->vm_file);
+ vma->vm_file = desc->vm_file;
+ }
vma->flags = desc->vma_flags;
vma->vm_page_prot = desc->page_prot;
@@ -780,14 +782,19 @@ struct vm_area_struct *vm_area_alloc(struct mm_struct *mm);
struct vm_area_struct *vm_area_dup(struct vm_area_struct *orig);
void vm_area_free(struct vm_area_struct *vma);
-/* vma_exec.c */
#ifdef CONFIG_MMU
+int mmap_prepare_validate(const struct vm_area_desc *prev_desc,
+ const struct vm_area_desc *desc);
+
+int mmap_hook_validate(unsigned long prev_start,
+ const vma_flags_t *prev_flags,
+ const struct vm_area_struct *vma);
+
+/* vma_exec.c */
int create_init_stack_vma(struct mm_struct *mm, struct vm_area_struct **vmap,
unsigned long *top_mem_p);
int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift);
-#endif
-#ifdef CONFIG_MMU
/*
* Denies creating a writable executable mapping or gaining executable permissions.
*
@@ -836,6 +843,19 @@ static inline bool map_deny_write_exec(const vma_flags_t *old,
return false;
}
+#else
+static inline int mmap_prepare_validate(const struct vm_area_desc *prev_desc,
+ const struct vm_area_desc *desc)
+{
+ return 0;
+}
+
+static inline int mmap_hook_validate(unsigned long prev_start,
+ const vma_flags_t *prev_flags,
+ const struct vm_area_struct *vma)
+{
+ return 0;
+}
#endif
struct vm_area_struct *__install_special_mapping(struct mm_struct *mm,
diff --git a/mm/vma_internal.h b/mm/vma_internal.h
index 4d300e7bbaf4c..4f73f0a4db796 100644
--- a/mm/vma_internal.h
+++ b/mm/vma_internal.h
@@ -18,7 +18,6 @@
#include <linux/fs.h>
#include <linux/huge_mm.h>
#include <linux/hugetlb.h>
-#include <linux/hugetlb_inline.h>
#include <linux/kernel.h>
#include <linux/ksm.h>
#include <linux/khugepaged.h>
diff --git a/mm/vmscan.c b/mm/vmscan.c
index aaceed4759eeb..001f8b760266b 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3471,13 +3471,14 @@ static int should_skip_vma(unsigned long start, unsigned long end, struct mm_wal
if (!vma_is_accessible(vma))
return true;
- if (is_vm_hugetlb_page(vma))
+ if (vma_is_hugetlb(vma))
return true;
if (!vma_has_recency(vma))
return true;
- if (vma->vm_flags & (VM_LOCKED | VM_SPECIAL))
+ if (vma_test(vma, VMA_LOCKED_BIT) || vma_is_kernel_owned(vma) ||
+ vma_is_fixed_mapping(vma))
return true;
if (vma == get_gate_vma(vma->vm_mm))
@@ -4417,8 +4418,8 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr)
if (spin_is_contended(pvmw->ptl))
return true;
- /* exclude special VMAs containing anon pages from COW */
- if (vma->vm_flags & VM_SPECIAL)
+ /* exclude kernel-owned and fixed VMAs containing anon pages from COW */
+ if (vma_is_kernel_owned(vma) || vma_is_fixed_mapping(vma))
return true;
/* avoid taking the LRU lock under the PTL when possible */
diff --git a/security/selinux/selinuxfs.c b/security/selinux/selinuxfs.c
index c7d91476971cb..545a6f89f9e76 100644
--- a/security/selinux/selinuxfs.c
+++ b/security/selinux/selinuxfs.c
@@ -340,6 +340,9 @@ static int sel_open_policy(struct inode *inode, struct file *filp)
struct policy_load_memory *plm = NULL;
int rc;
+ if (filp->f_mode & FMODE_WRITE)
+ return -EACCES;
+
rc = avc_has_perm(current_sid(), SECINITSID_SECURITY,
SECCLASS_SECURITY, SECURITY__READ_POLICY, NULL);
if (rc)
@@ -424,14 +427,6 @@ static const struct vm_operations_struct sel_mmap_policy_ops = {
static int sel_mmap_policy(struct file *filp, struct vm_area_struct *vma)
{
- if (vma->vm_flags & VM_SHARED) {
- /* do not allow mprotect to make mapping writable */
- vm_flags_clear(vma, VM_MAYWRITE);
-
- if (vma->vm_flags & VM_WRITE)
- return -EACCES;
- }
-
vm_flags_set(vma, VM_DONTEXPAND | VM_DONTDUMP);
vma->vm_ops = &sel_mmap_policy_ops;
diff --git a/sound/core/pcm_native.c b/sound/core/pcm_native.c
index 62324282fcae9..c98fb3df14f34 100644
--- a/sound/core/pcm_native.c
+++ b/sound/core/pcm_native.c
@@ -3760,39 +3760,27 @@ static __poll_t snd_pcm_poll(struct file *file, poll_table *wait)
/*
* mmap status record
*/
-static vm_fault_t snd_pcm_mmap_status_fault(struct vm_fault *vmf)
+static int snd_pcm_mmap_status(struct snd_pcm_substream *substream, struct file *file,
+ struct vm_area_struct *vma)
{
- struct snd_pcm_substream *substream = vmf->vma->vm_private_data;
+ const unsigned long size = vma->vm_end - vma->vm_start;
struct snd_pcm_runtime *runtime;
-
- if (substream == NULL)
- return VM_FAULT_SIGBUS;
- runtime = substream->runtime;
- vmf->page = virt_to_page(runtime->status);
- get_page(vmf->page);
- return 0;
-}
+ struct page *page;
-static const struct vm_operations_struct snd_pcm_vm_ops_status =
-{
- .fault = snd_pcm_mmap_status_fault,
-};
+ BUILD_BUG_ON(sizeof(struct snd_pcm_mmap_status) > PAGE_SIZE);
-static int snd_pcm_mmap_status(struct snd_pcm_substream *substream, struct file *file,
- struct vm_area_struct *area)
-{
- long size;
- if (!(area->vm_flags & VM_READ))
+ if (!(vma->vm_flags & VM_READ))
return -EINVAL;
- size = area->vm_end - area->vm_start;
- if (size != PAGE_ALIGN(sizeof(struct snd_pcm_mmap_status)))
+ if (size != PAGE_SIZE)
return -EINVAL;
- area->vm_ops = &snd_pcm_vm_ops_status;
- area->vm_private_data = substream;
- vm_flags_mod(area, VM_DONTEXPAND | VM_DONTDUMP,
+
+ vm_flags_mod(vma, VM_DONTEXPAND | VM_DONTDUMP,
VM_WRITE | VM_MAYWRITE);
+ vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
- return 0;
+ runtime = substream->runtime;
+ page = virt_to_page(runtime->status);
+ return vm_insert_page(vma, vma->vm_start, page);
}
/*
diff --git a/tools/testing/vma/include/dup.h b/tools/testing/vma/include/dup.h
index 16c09dac59d9b..dc24f43a9394c 100644
--- a/tools/testing/vma/include/dup.h
+++ b/tools/testing/vma/include/dup.h
@@ -352,14 +352,6 @@ enum {
#define VM_ACCESS_FLAGS (VM_READ | VM_WRITE | VM_EXEC)
#define VMA_ACCESS_FLAGS mk_vma_flags(VMA_READ_BIT, VMA_WRITE_BIT, VMA_EXEC_BIT)
-/*
- * Special vmas that are non-mergable, non-mlock()able.
- */
-#define VM_SPECIAL (VM_IO | VM_DONTEXPAND | VM_PFNMAP | VM_MIXEDMAP)
-
-#define VMA_SPECIAL_FLAGS mk_vma_flags(VMA_IO_BIT, VMA_DONTEXPAND_BIT, \
- VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT)
-
#define VMA_REMAP_FLAGS mk_vma_flags(VMA_IO_BIT, VMA_PFNMAP_BIT, \
VMA_DONTEXPAND_BIT, VMA_DONTDUMP_BIT)
@@ -454,17 +446,20 @@ static __always_inline bool vma_flags_empty(const vma_flags_t *flags)
/* What action should be taken after an .mmap_prepare call is complete? */
enum mmap_action_type {
- MMAP_NOTHING, /* Mapping is complete, no further action. */
- MMAP_REMAP_PFN, /* Remap PFN range. */
- MMAP_IO_REMAP_PFN, /* I/O remap PFN range. */
- MMAP_SIMPLE_IO_REMAP, /* I/O remap with guardrails. */
- MMAP_MAP_KERNEL_PAGES, /* Map kernel page range from an array. */
+ MMAP_NOTHING,
+ MMAP_REMAP_PFN,
+ MMAP_IO_REMAP_PFN,
+ MMAP_SIMPLE_IO_REMAP, /* I/O remap with guardrails. */
+ MMAP_KERNEL_PAGES, /* Map kernel page range from array. */
+ MMAP_DISCONTIG_KERNEL_PAGES, /* Map kernel discontig page range. */
};
/*
* Describes an action an mmap_prepare hook can instruct to be taken to complete
* the mapping of a VMA. Specified in vm_area_desc.
*/
+struct discontig_kernel_page_ops;
+
struct mmap_action {
union {
struct {
@@ -483,6 +478,10 @@ struct mmap_action {
unsigned long nr_pages;
pgoff_t pgoff;
} map_kernel;
+ struct {
+ void *init_private;
+ const struct discontig_kernel_page_ops *ops;
+ } map_kernel_discontig;
};
enum mmap_action_type type;
@@ -1359,13 +1358,23 @@ static inline int vfs_mmap_prepare(struct file *file, struct vm_area_desc *desc)
return file->f_op->mmap_prepare(desc);
}
+int mmap_prepare_validate(const struct vm_area_desc *prev_desc,
+ const struct vm_area_desc *desc);
+
static inline int __compat_vma_mmap(struct vm_area_desc *desc,
struct vm_area_struct *vma)
{
+ struct vm_area_desc prev_desc;
int err;
+ /* Derive state prior to mmap_prepare hook. */
+ compat_set_desc_from_vma(&prev_desc, desc->file, vma);
/* Perform any preparatory tasks for mmap action. */
err = mmap_action_prepare(desc);
+ if (err)
+ return err;
+ /* Check the caller did nothing crazy. */
+ err = mmap_prepare_validate(&prev_desc, desc);
if (err)
return err;
/* Update the VMA from the descriptor. */
@@ -1647,3 +1656,34 @@ static inline bool file_is_dev_zero(const struct file *file)
{
return file && file->f_op == &zero_fops;
}
+
+static inline bool vma_flags_is_kernel_owned(const vma_flags_t *flags)
+{
+ return vma_flags_test_any(flags, VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT);
+}
+
+static inline bool vma_is_kernel_owned(const struct vm_area_struct *vma)
+{
+ return vma_flags_is_kernel_owned(&vma->flags);
+}
+
+static inline bool vma_flags_can_merge(const vma_flags_t *flags)
+{
+ /*
+ * VMA merging assumes that the properties of a VMA completely describe
+ * the properties of that VMA.
+ *
+ * However, kernel-owned mappings may have established state upon mapping
+ * not embodied in any attribute of the VMA.
+ *
+ * Additionally, PFN maps encode the source PFN of the range in
+ * vma->vm_pgoff, which may otherwise cause spurious merges.
+ */
+ if (vma_flags_is_kernel_owned(flags))
+ return false;
+ /* VMA explicitly marked as being unmergeable. */
+ if (vma_flags_test(flags, VMA_DONTEXPAND_BIT))
+ return false;
+
+ return true;
+}
diff --git a/tools/testing/vma/include/stubs.h b/tools/testing/vma/include/stubs.h
index d6136e19a8af3..48d1dc53df42c 100644
--- a/tools/testing/vma/include/stubs.h
+++ b/tools/testing/vma/include/stubs.h
@@ -193,7 +193,7 @@ static inline bool mapping_can_writeback(struct address_space *mapping)
return true;
}
-static inline bool is_vm_hugetlb_page(struct vm_area_struct *vma)
+static inline bool vma_is_hugetlb(struct vm_area_struct *vma)
{
return false;
}
diff --git a/tools/testing/vma/tests/merge.c b/tools/testing/vma/tests/merge.c
index acaab282939c0..b26f1a66a1707 100644
--- a/tools/testing/vma/tests/merge.c
+++ b/tools/testing/vma/tests/merge.c
@@ -496,17 +496,11 @@ static bool test_vma_merge_special_flags(void)
.mm = &mm,
.vmi = &vmi,
};
- vma_flag_t special_flags[] = { VMA_IO_BIT, VMA_DONTEXPAND_BIT,
+ vma_flag_t special_flags[] = { VMA_DONTEXPAND_BIT,
VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT };
- vma_flags_t all_special_flags = EMPTY_VMA_FLAGS;
int i;
struct vm_area_struct *vma_left, *vma;
- /* Make sure there aren't new VM_SPECIAL flags. */
- for (i = 0; i < ARRAY_SIZE(special_flags); i++)
- vma_flags_set(&all_special_flags, special_flags[i]);
- ASSERT_FLAGS_SAME_MASK(&all_special_flags, VMA_SPECIAL_FLAGS);
-
/*
* 01234
* AAA
@@ -520,7 +514,7 @@ static bool test_vma_merge_special_flags(void)
* 01234
* AAA*
*
- * This should merge if not for the VM_SPECIAL flag.
+ * This should merge if not for the 'special' flag.
*/
vmg_set_range(&vmg, 0x3000, 0x4000, 3, vma_flags);
for (i = 0; i < ARRAY_SIZE(special_flags); i++) {
|
| 47/3 | 2026/09/14 19:12 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 54191 output: 28 thoughts: 3320 Thoughts: **KMSAN Fuzzing Session Evaluation** As an expert in kernel memory management, I need to determine if this kernel patch series warrants a dedicated KMSAN fuzzing session, beyond the existing KASAN coverage. The primary concern is the introduction or exposure of uninitialized memory risks, which KMSAN is specifically designed to detect. My analysis follows these principles: KMSAN is crucial when KASAN, LOCKDEP, or other standard bug detectors are insufficient, particularly if uninitialized memory reads or information leaks are introduced. The significant resource cost of a dedicated KMSAN session necessitates careful evaluation. We only enable it if the code changes genuinely introduce risks detectable *only* by KMSAN. I'm systematically examining each patch, focusing on potential areas where kernel structures are exposed to userspace, conditional logic may rely on uninitialized variables, or complex data structures are allocated without proper initialization. I'm also scrutinizing changes that may lead to information leaks via uninitialized memory content. I need to make certain that if there are any risks, they can not be detected by the current sanitizers. After reading the patch overview, the most concerning code changes appear to be those involving memory mapping and VMA (Virtual Memory Area) manipulations, because they handle memory shared between kernel and user space. I'm paying close attention to: * The conversion of `cs_char_mmap` to `cs_char_mmap_prepare` within `drivers/hsi/clients/cmt_speech.c`, specifically how pages are prepared and mapped, including `mmap_action_map_kernel_pages_full` and related functions. I'm carefully tracing where the `desc` is stored through the `mmap_prepare` to `mmap_action_complete`. I suspect this is where uninitialized memory could be exposed to userspace. * The `snd_pcm_mmap_status` code within `sound/core/pcm_native.c`, which now directly inserts a page into the VMA at mmap time. I have to make certain `runtime->status` is properly initialized. * The `map_discontig_kernel_pages_complete` in `mm/memory.c`, which handles the mapping of discontiguous kernel pages. This requires careful inspection for any potential uninitialized reads. So far, I see no immediate red flags, but I will need to trace all of the VMA flags and how they are stored with extreme care. The modifications across the various architectures and subsystems seem mainly to be cleanups and modernizations. However, I need to trace the data flow closely, particularly around the `mmap_action_...` helper functions to make sure there are no leaks. There is a lot of code, and with that comes more potential, so this will take more detailed work. Based on my early analysis, it is still possible a KMSAN session is not necessary, but I have a few areas that need more inspection. |
| 48/3 | 2026/09/14 19:13 | tool | grepper |
0mArgs:{
"Expression": "mmap_action_complete",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/util.c=1224=int __compat_vma_mmap(struct vm_area_desc *desc,\n--\nmm/util.c-1242-\t/* Complete any specified mmap actions. */\nmm/util.c:1243:\treturn mmap_action_complete(vma, \u0026desc-\u003eaction, /*is_compat=*/true);\nmm/util.c-1244-\n--\nmm/util.c=1479=EXPORT_SYMBOL(mmap_action_prepare);\n--\nmm/util.c-1481-/**\nmm/util.c:1482: * mmap_action_complete - Execute VMA descriptor action.\nmm/util.c-1483- * @vma: The VMA to perform the action upon.\n--\nmm/util.c-1491- */\nmm/util.c:1492:int mmap_action_complete(struct vm_area_struct *vma,\nmm/util.c-1493-\t\t\t struct mmap_action *action, bool is_compat)\n--\nmm/util.c-1518-}\nmm/util.c:1519:EXPORT_SYMBOL(mmap_action_complete);\nmm/util.c-1520-#else\n--\nmm/util.c=1537=EXPORT_SYMBOL(mmap_action_prepare);\nmm/util.c-1538-\nmm/util.c:1539:int mmap_action_complete(struct vm_area_struct *vma,\nmm/util.c-1540-\t\t\t struct mmap_action *action,\n--\nmm/util.c-1560-}\nmm/util.c:1561:EXPORT_SYMBOL(mmap_action_complete);\nmm/util.c-1562-#endif\n--\nmm/vma.c=2987=static unsigned long __mmap_region(struct file *file, unsigned long addr,\n--\nmm/vma.c-3044-\t} else if (have_mmap_prepare) {\nmm/vma.c:3045:\t\terror = mmap_action_complete(vma, \u0026desc.action,\nmm/vma.c-3046-\t\t\t\t\t /*is_compat=*/false);\n"
} |
| 49/3 | 2026/09/14 19:13 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 58738 output: 35 thoughts: 34 Thoughts: Here's a summary of my thought process, as though I were considering this myself: **Diving into Memory Mapping Implementation** Okay, so I'm digging into `__mmap_region`. My gut feeling is that it's handling the core logic of mapping a new virtual memory region. I need to get a solid grasp of how this function interacts with the virtual memory area (VMA) structures and how it establishes the new mapping. I'm going to head over to `mm/vma.c`, specifically around line numbers 2980 to 3050. This range should encompass the key parts of the implementation related to setting up the VMAs, dealing with permissions, and hopefully some tidbits about the underlying page table manipulations, based on how the kernel usually handles these operations. I expect this section to elucidate how the new region interacts with the existing address space, how it handles overlaps, and how it potentially interacts with the page tables. I'm hoping to understand the complete sequence of operations to see if there are any subtle optimization opportunities within the function. |
| 50/3 | 2026/09/14 19:13 | tool | read-file |
0mArgs:{
"File": "mm/vma.c",
"FirstLine": 2980,
"LineCount": 70
}Results: {
"Contents": "2980:\t\t *\n2981:\t\t * On error, the caller always drops the original file regardless.\n2982:\t\t */\n2983:\t\tif (map-\u003evm_file \u0026\u0026 !map_same_file(map))\n2984:\t\t\tfput(map-\u003evm_file);\n2985:\t}\n2986:\t\n2987:\tstatic unsigned long __mmap_region(struct file *file, unsigned long addr,\n2988:\t\t\tunsigned long len, vma_flags_t vma_flags,\n2989:\t\t\tunsigned long pgoff, struct list_head *uf)\n2990:\t{\n2991:\t\tstruct mm_struct *mm = current-\u003emm;\n2992:\t\tstruct vm_area_struct *vma = NULL;\n2993:\t\tconst bool have_mmap_prepare = file \u0026\u0026 file-\u003ef_op-\u003emmap_prepare;\n2994:\t\tVMA_ITERATOR(vmi, mm, addr);\n2995:\t\tconst pgoff_t anon_pgoff = addr \u003e\u003e PAGE_SHIFT;\n2996:\t\tMMAP_STATE(map, mm, \u0026vmi, addr, len, pgoff, anon_pgoff, vma_flags, file);\n2997:\t\tstruct vm_area_desc desc = {\n2998:\t\t\t.mm = mm,\n2999:\t\t\t.file = file,\n3000:\t\t\t.action = {\n3001:\t\t\t\t.type = MMAP_NOTHING, /* Default to no further action. */\n3002:\t\t\t},\n3003:\t\t\t.vm_ops = \u0026vma_dummy_vm_ops,\n3004:\t\t};\n3005:\t\tbool allocated_new = false;\n3006:\t\tint error;\n3007:\t\n3008:\t\tmap.check_ksm_early = can_set_ksm_flags_early(\u0026map);\n3009:\t\n3010:\t\terror = __mmap_setup(\u0026map, \u0026desc, uf);\n3011:\t\tif (error)\n3012:\t\t\tgoto abort_munmap;\n3013:\t\tif (have_mmap_prepare)\n3014:\t\t\terror = call_mmap_prepare(\u0026map, \u0026desc);\n3015:\t\tif (error)\n3016:\t\t\tgoto unacct_error;\n3017:\t\n3018:\t\tif (map.check_ksm_early)\n3019:\t\t\tupdate_ksm_flags(\u0026map);\n3020:\t\n3021:\t\t/* Attempt to merge with adjacent VMAs... */\n3022:\t\tif (map.prev || map.next) {\n3023:\t\t\tVMG_MMAP_STATE(vmg, \u0026map, /* vma = */ NULL);\n3024:\t\n3025:\t\t\tvma = vma_merge_new_range(\u0026vmg);\n3026:\t\t}\n3027:\t\n3028:\t\t/* ...but if we can't, allocate a new VMA. */\n3029:\t\tif (!vma) {\n3030:\t\t\terror = __mmap_new_vma(\u0026map, \u0026vma, \u0026desc.action);\n3031:\t\t\tif (error)\n3032:\t\t\t\tgoto unacct_error;\n3033:\t\t\tallocated_new = true;\n3034:\t\t}\n3035:\t\n3036:\t\tif (have_mmap_prepare \u0026\u0026 allocated_new \u0026\u0026 !map_is_anon(\u0026map))\n3037:\t\t\tset_vma_user_defined_fields(vma, \u0026map);\n3038:\t\n3039:\t\t__mmap_complete(\u0026map, vma);\n3040:\t\n3041:\t\tif (!allocated_new) {\n3042:\t\t\t/* Merged, so need to drop refcount. */\n3043:\t\t\tput_map(\u0026map);\n3044:\t\t} else if (have_mmap_prepare) {\n3045:\t\t\terror = mmap_action_complete(vma, \u0026desc.action,\n3046:\t\t\t\t\t\t /*is_compat=*/false);\n3047:\t\t\tif (error)\n3048:\t\t\t\treturn error;\n3049:\t\t}\n"
} |
| 51/3 | 2026/09/14 19:13 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 60144 output: 31 thoughts: 2798 Thoughts: **Deep Dive into Kernel Memory Mapping and Page Handling** Okay, so I'm tracing the flow of memory mapping, specifically in the context of kernel drivers. My initial focus is on understanding how `desc` is used and its scope throughout the `__mmap_region` process. I see that `desc` sticks around in scope through the important parts of this function. I'm paying close attention to what's stored in `desc->private_data`. It seems like in `cs_char_mmap_prepare`, `desc->private_data` is used to store a pointer to the page, and the code later dereferences this pointer through `pages = (struct page **)&desc->private_data`. I've verified that `call_mmap_prepare` does not overwrite the contents of `desc->private_data`. This confirms my initial assumption that it safely holds the `struct page *`. The `mmap_action_map_kernel_pages_full` function correctly uses the pointer stored in `desc->private_data`. This makes me feel good about the first part of the code path. I also checked the compat mode and confirmed similar behavior. Now, I'm digging into `sg_mmap_prepare` and the discontiguous memory mapping. I'm focusing on how `sg` handles the page allocation, and if it could have potential issues. Here, `desc->private_data` is being used to store the `Sg_fd` structure. This also seems okay. The interesting part is how `sg_discontig_get` interacts with the allocated pages. The code uses `rsv_schp->pages[nr_pages >> order]`. I'm starting to think about different scenarios based on the allocation order. The question is, if we allocate pages in a compound page (order > 0), does `sg_discontig_get` fetch the correct page in the right iteration. Now I am looking at how `discontig_kernel_map_page` works. This decides if the mapping involves a compound folio or a single page. If `folio_test_large` is true, then the pages are part of a compound page. The code iterates through the individual pages in the compound page, which seems fine. Then I am considering the case when `folio_test_large` is false, i.e. not a compound page. The `nr_pages` incremented correctly. The issue is that the code fetches page by `rsv_schp->pages[nr_pages >> order]` in `sg_discontig_get`. However, if the page allocation was not compound, the array would be indexed incorrectly and this might result in the same physical page being used more than once. This would mean that the logic in `sg.c` might have a bug. Let's see how `sg.c` actually allocates pages. Time to grep and check it! |
| 52/3 | 2026/09/14 19:13 | tool | grepper |
0mArgs:{
"Expression": "alloc_pages",
"PathPrefix": "drivers/scsi/sg.c"
}Results: {
"Output": "drivers/scsi/sg.c=1841=sg_build_indirect(Sg_scatter_hold * schp, Sg_fd * sfp, int buff_size)\n--\ndrivers/scsi/sg.c-1881-\ndrivers/scsi/sg.c:1882:\t\tschp-\u003epages[k] = alloc_pages(gfp_mask, order);\ndrivers/scsi/sg.c-1883-\t\tif (!schp-\u003epages[k])\n"
} |
| 53/3 | 2026/09/14 19:13 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 63729 output: 37 thoughts: 55 Thoughts: Here's my summary, as though I'm thinking through this process: **Diving into the `sg.c` SCSI Driver for Memory Allocation** Okay, I'm digging into a memory allocation issue – probably related to scatter-gather lists, given the context. The crucial function is `alloc_pages(gfp_mask, order)`. The `gfp_mask` is the key – it seems to be using `GFP_ATOMIC | __GFP_COMP | ...` or something comparable. I need to understand the implications of those flags, especially the interaction between atomic context and compression hints. Where's the best place to check that memory allocation code? Let's quickly jump into the `drivers/scsi/sg.c` driver, specifically line 1870. That's where I bet I'll find the call to `alloc_pages` that's bothering me and hopefully its surrounding context. I'm looking for the allocation flags and how they are used, and whether there's anything unusual there. Time to dig in and see if this tells me the whole story or if I need to back up and dig around further in the stack. |
| 54/3 | 2026/09/14 19:13 | tool | read-file |
0mArgs:{
"File": "drivers/scsi/sg.c",
"FirstLine": 1870,
"LineCount": 25
}Results: {
"Contents": "1870:\t\t}\n1871:\t\n1872:\t\torder = get_order(num);\n1873:\tretry:\n1874:\t\tret_sz = 1 \u003c\u003c (PAGE_SHIFT + order);\n1875:\t\n1876:\t\tfor (k = 0, rem_sz = blk_size; rem_sz \u003e 0 \u0026\u0026 k \u003c mx_sc_elems;\n1877:\t\t k++, rem_sz -= ret_sz) {\n1878:\t\n1879:\t\t\tnum = (rem_sz \u003e scatter_elem_sz_prev) ?\n1880:\t\t\t\tscatter_elem_sz_prev : rem_sz;\n1881:\t\n1882:\t\t\tschp-\u003epages[k] = alloc_pages(gfp_mask, order);\n1883:\t\t\tif (!schp-\u003epages[k])\n1884:\t\t\t\tgoto out;\n1885:\t\n1886:\t\t\tif (num == scatter_elem_sz_prev) {\n1887:\t\t\t\tif (unlikely(ret_sz \u003e scatter_elem_sz_prev)) {\n1888:\t\t\t\t\tscatter_elem_sz = ret_sz;\n1889:\t\t\t\t\tscatter_elem_sz_prev = ret_sz;\n1890:\t\t\t\t}\n1891:\t\t\t}\n1892:\t\n1893:\t\t\tSCSI_LOG_TIMEOUT(5, sg_printk(KERN_INFO, sfp-\u003eparentdp,\n1894:\t\t\t\t\t \"sg_build_indirect: k=%d, num=%d, ret_sz=%d\\n\",\n"
} |
| 55/3 | 2026/09/14 19:13 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 64486 output: 37 thoughts: 21 Thoughts: Here's my summary, as if I were thinking these thoughts: **Tracing the Genesis of `gfp_mask` in `sg_build_indirect`** Okay, so I need to understand where the `gfp_mask` variable gets its initial value within the `sg_build_indirect` function. It's a critical parameter, clearly influencing how scatter-gather lists are constructed, and thus, performance or memory management. I need to know the origin of this value to understand its implications within the larger context of the system. I'll need to trace back from that function call to its definition, likely checking header files or looking at preceding code blocks, to determine where `gfp_mask` is set. Understanding that initialization is key to unraveling any issues related to this specific aspect of the implementation. Let's see what the code reveals. |
| 56/3 | 2026/09/14 19:13 | tool | read-file |
0mArgs:{
"File": "drivers/scsi/sg.c",
"FirstLine": 1845,
"LineCount": 25
}Results: {
"Contents": "1845:\t\tint blk_size = buff_size, order;\n1846:\t\tgfp_t gfp_mask = GFP_ATOMIC | __GFP_COMP | __GFP_NOWARN | __GFP_ZERO;\n1847:\t\n1848:\t\tif (blk_size \u003c 0)\n1849:\t\t\treturn -EFAULT;\n1850:\t\tif (0 == blk_size)\n1851:\t\t\t++blk_size;\t/* don't know why */\n1852:\t\t/* round request up to next highest SG_SECTOR_SZ byte boundary */\n1853:\t\tblk_size = ALIGN(blk_size, SG_SECTOR_SZ);\n1854:\t\tSCSI_LOG_TIMEOUT(4, sg_printk(KERN_INFO, sfp-\u003eparentdp,\n1855:\t\t\t\"sg_build_indirect: buff_size=%d, blk_size=%d\\n\",\n1856:\t\t\tbuff_size, blk_size));\n1857:\t\n1858:\t\t/* N.B. ret_sz carried into this block ... */\n1859:\t\tmx_sc_elems = sg_build_sgat(schp, sfp, sg_tablesize);\n1860:\t\tif (mx_sc_elems \u003c 0)\n1861:\t\t\treturn mx_sc_elems;\t/* most likely -ENOMEM */\n1862:\t\n1863:\t\tnum = scatter_elem_sz;\n1864:\t\tif (unlikely(num != scatter_elem_sz_prev)) {\n1865:\t\t\tif (num \u003c PAGE_SIZE) {\n1866:\t\t\t\tscatter_elem_sz = PAGE_SIZE;\n1867:\t\t\t\tscatter_elem_sz_prev = PAGE_SIZE;\n1868:\t\t\t} else\n1869:\t\t\t\tscatter_elem_sz_prev = num;\n"
} |
| 57/3 | 2026/09/14 19:13 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 65210 output: 34 thoughts: 327 Thoughts: **Analyzing Potential Memory Safety Issues in USB Monitor Driver** Okay, so `gfp_mask` includes `__GFP_COMP | __GFP_ZERO`. Great, that means the pages are compound pages and, importantly, zero-filled. No concerns about uninitialized memory there, that's a relief. Now, let's dive into `mon_bin.c`. Specifically, the `mon_bin_discontig_get` function. Hmm, what's going on here? I see `discontig_kernel_map_page` being called with `rp->b_vec[chunk_idx].pg`. But how does `mon_bin` actually work? This is where it gets interesting... or potentially tricky. I'm seeing `CHUNK_SIZE` and I need to understand it to assess what's happening. I need to figure out what that `CHUNK_SIZE` value represents. I'm going to search for its definition within `drivers/usb/mon/mon_bin.c` to understand how the memory is chunked and potentially allocated. That's the key to understanding if there's any potential memory corruption or race conditions. |
| 58/3 | 2026/09/14 19:13 | tool | grepper |
0mArgs:{
"Expression": "CHUNK_SIZE",
"PathPrefix": "drivers/usb/mon/mon_bin.c"
}Results: {
"Output": "drivers/usb/mon/mon_bin.c-63- */\ndrivers/usb/mon/mon_bin.c:64:#define CHUNK_SIZE PAGE_SIZE\ndrivers/usb/mon/mon_bin.c:65:#define CHUNK_ALIGN(x) (((x)+CHUNK_SIZE-1) \u0026 ~(CHUNK_SIZE-1))\ndrivers/usb/mon/mon_bin.c-66-\n--\ndrivers/usb/mon/mon_bin.c=205=static inline struct mon_bin_hdr *MON_OFF2HDR(const struct mon_reader_bin *rp,\n--\ndrivers/usb/mon/mon_bin.c-208-\treturn (struct mon_bin_hdr *)\ndrivers/usb/mon/mon_bin.c:209:\t (rp-\u003eb_vec[offset / CHUNK_SIZE].ptr + offset % CHUNK_SIZE);\ndrivers/usb/mon/mon_bin.c-210-}\n--\ndrivers/usb/mon/mon_bin.c=234=static unsigned int mon_copy_to_buff(const struct mon_reader_bin *this,\n--\ndrivers/usb/mon/mon_bin.c-245-\t\tstep_len = length;\ndrivers/usb/mon/mon_bin.c:246:\t\tin_page = CHUNK_SIZE - (off \u0026 (CHUNK_SIZE-1));\ndrivers/usb/mon/mon_bin.c-247-\t\tif (in_page \u003c step_len)\n--\ndrivers/usb/mon/mon_bin.c-252-\t\t */\ndrivers/usb/mon/mon_bin.c:253:\t\tbuf = this-\u003eb_vec[off / CHUNK_SIZE].ptr + off % CHUNK_SIZE;\ndrivers/usb/mon/mon_bin.c-254-\t\tmemcpy(buf, from, step_len);\n--\ndrivers/usb/mon/mon_bin.c=266=static int copy_from_buf(const struct mon_reader_bin *this, unsigned int off,\n--\ndrivers/usb/mon/mon_bin.c-277-\t\tstep_len = length;\ndrivers/usb/mon/mon_bin.c:278:\t\tin_page = CHUNK_SIZE - (off \u0026 (CHUNK_SIZE-1));\ndrivers/usb/mon/mon_bin.c-279-\t\tif (in_page \u003c step_len)\n--\ndrivers/usb/mon/mon_bin.c-284-\t\t */\ndrivers/usb/mon/mon_bin.c:285:\t\tbuf = this-\u003eb_vec[off / CHUNK_SIZE].ptr + off % CHUNK_SIZE;\ndrivers/usb/mon/mon_bin.c-286-\t\tif (copy_to_user(to, buf, step_len))\n--\ndrivers/usb/mon/mon_bin.c=470=static void mon_bin_get_isodesc(const struct mon_reader_bin *rp,\n--\ndrivers/usb/mon/mon_bin.c-478-\t\tdp = (struct mon_bin_isodesc *)\ndrivers/usb/mon/mon_bin.c:479:\t\t (rp-\u003eb_vec[offset / CHUNK_SIZE].ptr + offset % CHUNK_SIZE);\ndrivers/usb/mon/mon_bin.c-480-\t\tdp-\u003eiso_status = fp-\u003estatus;\n--\ndrivers/usb/mon/mon_bin.c=678=static int mon_bin_open(struct inode *inode, struct file *file)\n--\ndrivers/usb/mon/mon_bin.c-706-\ndrivers/usb/mon/mon_bin.c:707:\tsize = sizeof(struct mon_pgmap) * (rp-\u003eb_size/CHUNK_SIZE);\ndrivers/usb/mon/mon_bin.c-708-\tif ((rp-\u003eb_vec = kzalloc(size, GFP_KERNEL)) == NULL) {\n--\ndrivers/usb/mon/mon_bin.c-712-\ndrivers/usb/mon/mon_bin.c:713:\tif ((rc = mon_alloc_buff(rp-\u003eb_vec, rp-\u003eb_size/CHUNK_SIZE)) \u003c 0)\ndrivers/usb/mon/mon_bin.c-714-\t\tgoto err_allocbuff;\n--\ndrivers/usb/mon/mon_bin.c=783=static int mon_bin_release(struct inode *inode, struct file *file)\n--\ndrivers/usb/mon/mon_bin.c-796-\ndrivers/usb/mon/mon_bin.c:797:\tmon_free_buff(rp-\u003eb_vec, rp-\u003eb_size/CHUNK_SIZE);\ndrivers/usb/mon/mon_bin.c-798-\tkfree(rp-\u003eb_vec);\n--\ndrivers/usb/mon/mon_bin.c=989=static long mon_bin_ioctl(struct file *file, unsigned int cmd, unsigned long arg)\n--\ndrivers/usb/mon/mon_bin.c-1031-\t\tsize = CHUNK_ALIGN(arg);\ndrivers/usb/mon/mon_bin.c:1032:\t\tvec = kzalloc_objs(struct mon_pgmap, size / CHUNK_SIZE);\ndrivers/usb/mon/mon_bin.c-1033-\t\tif (vec == NULL) {\n--\ndrivers/usb/mon/mon_bin.c-1037-\ndrivers/usb/mon/mon_bin.c:1038:\t\tret = mon_alloc_buff(vec, size/CHUNK_SIZE);\ndrivers/usb/mon/mon_bin.c-1039-\t\tif (ret \u003c 0) {\n--\ndrivers/usb/mon/mon_bin.c-1046-\t\tif (rp-\u003emmap_active) {\ndrivers/usb/mon/mon_bin.c:1047:\t\t\tmon_free_buff(vec, size/CHUNK_SIZE);\ndrivers/usb/mon/mon_bin.c-1048-\t\t\tkfree(vec);\n--\ndrivers/usb/mon/mon_bin.c-1050-\t\t} else {\ndrivers/usb/mon/mon_bin.c:1051:\t\t\tmon_free_buff(rp-\u003eb_vec, rp-\u003eb_size/CHUNK_SIZE);\ndrivers/usb/mon/mon_bin.c-1052-\t\t\tkfree(rp-\u003eb_vec);\n--\ndrivers/usb/mon/mon_bin.c=1272=static int mon_bin_discontig_get(struct discontig_kernel_page_state *state)\n--\ndrivers/usb/mon/mon_bin.c-1285-\t}\ndrivers/usb/mon/mon_bin.c:1286:\tchunk_idx = offset / CHUNK_SIZE;\ndrivers/usb/mon/mon_bin.c-1287-\tdiscontig_kernel_map_page(state, rp-\u003eb_vec[chunk_idx].pg);\n"
} |
| 59/3 | 2026/09/14 19:13 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 67541 output: 27 thoughts: 1429 Thoughts: **Analyzing the Memory Mapping Changes** Okay, so I'm diving into this memory mapping rework. The first thing I see is that the code's now handling chunks as pages, where `CHUNK_SIZE` equals `PAGE_SIZE`. The old code had a `mon_bin_vma_fault` function with a spinlock and calculated offsets, page indexes, and used `get_page`. It looks like that has been replaced now by mapping the pages at `mmap_prepare` time using `discontig_kernel_map_page`. Seems cleaner. Next, I need to check `drivers/infiniband/hw/hfi1/file_ops.c`. Aha, the change here involves replacing a `vma_fault` function with `remap_vmalloc_range`. This is likely to have improved things. The old method used `vmalloc_to_page` which has some known issues. Good, no obvious uninitialized memory or race conditions there. The `remap_vmalloc_range` function looks like a solid replacement. Moving onto `mm/memory.c`, specifically `map_discontig_kernel_pages_complete`. This is where the core logic of the new mapping happens. I need to make sure the `discontig_kernel_page_state` struct is initialized correctly. Looking at the struct definition, it seems pretty straightforward: `start`, `end`, `addr`, `pgoff`, `nr_pages_mapped`, `nr_pages_remain`, `vm_private_data`, and `private`. I recall that designated initializers in C99, initialize all fields not listed to zero. Furthermore, inside the loop in that code, the `state.action` is initialized, and helper functions like `discontig_kernel_map_*` are used to set the action. The `apply_discontig_action` function then uses the `action`, `__nr_pages`, and the union member corresponding to the action. It seems like a clean and well-structured approach. I am not seeing anything that stands out, or is concerning. Okay, let's keep going. I'm taking a look at other changes. In `mm/vma.c`, I noticed `call_mmap_prepare` copies `desc` to `prev_desc`. Where is `desc` being initialized? Ah, line 2997. It looks like it is initialized with `.mm`, `.file`, `.action` with `.type = MMAP_NOTHING`, and `.vm_ops`. But, what about the rest? I need to check `__mmap_setup`. I bet it's initialized there. |
| 60/3 | 2026/09/14 19:14 | tool | grepper |
0mArgs:{
"Expression": "__mmap_setup",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/vma.c=2505=static void set_desc_from_map(struct vm_area_desc *desc,\n--\nmm/vma.c-2517-/*\nmm/vma.c:2518: * __mmap_setup() - Prepare to gather any overlapping VMAs that need to be\nmm/vma.c-2519- * unmapped once the map operation is completed, check limits, account mapping\n--\nmm/vma.c-2529- */\nmm/vma.c:2530:static int __mmap_setup(struct mmap_state *map, struct vm_area_desc *desc,\nmm/vma.c-2531-\t\t\tstruct list_head *uf)\n--\nmm/vma.c=2987=static unsigned long __mmap_region(struct file *file, unsigned long addr,\n--\nmm/vma.c-3009-\nmm/vma.c:3010:\terror = __mmap_setup(\u0026map, \u0026desc, uf);\nmm/vma.c-3011-\tif (error)\n--\nmm/vma.c-3052-\nmm/vma.c:3053:\t/* Accounting was done by __mmap_setup(). */\nmm/vma.c-3054-unacct_error:\n"
} |
| 61/3 | 2026/09/14 19:14 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 69919 output: 35 thoughts: 46 Thoughts: Here's my summary, as though I'm thinking through this process: **Diving into the Kernel's Memory Management** Alright, let's get down to brass tacks. I need to understand how the kernel's virtual memory areas (VMAs) are set up, specifically how descriptors are linked to memory mappings. I'm going to jump into the `mm/vma.c` file. I'm particularly interested in the code around lines 2505 through 2560. My goal is to carefully examine the interaction between `set_desc_from_map` and `__mmap_setup`. I'm guessing `set_desc_from_map` is responsible for populating the relevant descriptor with information about the memory mapping, perhaps attributes like access permissions or flags. `__mmap_setup` likely handles the more fundamental allocation and setup of the VMA itself and interacts with the page table structures. By analyzing the interplay of these two functions, I expect to gain a solid grasp of the inner workings of how user-space memory is mapped and managed within the kernel's address space. Let's see if this analysis will give me the insights I'm seeking! |
| 62/3 | 2026/09/14 19:14 | tool | read-file |
0mArgs:{
"File": "mm/vma.c",
"FirstLine": 2505,
"LineCount": 55
}Results: {
"Contents": "2505:\tstatic void set_desc_from_map(struct vm_area_desc *desc,\n2506:\t\t\tconst struct mmap_state *map)\n2507:\t{\n2508:\t\tdesc-\u003estart = map-\u003eaddr;\n2509:\t\tdesc-\u003eend = map-\u003eend;\n2510:\t\n2511:\t\tdesc-\u003epgoff = map-\u003epgoff;\n2512:\t\tdesc-\u003evm_file = map-\u003evm_file;\n2513:\t\tdesc-\u003evma_flags = map-\u003evma_flags;\n2514:\t\tdesc-\u003epage_prot = map-\u003epage_prot;\n2515:\t}\n2516:\t\n2517:\t/*\n2518:\t * __mmap_setup() - Prepare to gather any overlapping VMAs that need to be\n2519:\t * unmapped once the map operation is completed, check limits, account mapping\n2520:\t * and clean up any pre-existing VMAs.\n2521:\t *\n2522:\t * As a result it sets up the @map and @desc objects.\n2523:\t *\n2524:\t * @map: Mapping state.\n2525:\t * @desc: VMA descriptor\n2526:\t * @uf: Userfaultfd context list.\n2527:\t *\n2528:\t * Returns: 0 on success, error code otherwise.\n2529:\t */\n2530:\tstatic int __mmap_setup(struct mmap_state *map, struct vm_area_desc *desc,\n2531:\t\t\t\tstruct list_head *uf)\n2532:\t{\n2533:\t\tint error;\n2534:\t\tstruct vma_iterator *vmi = map-\u003evmi;\n2535:\t\tstruct vma_munmap_struct *vms = \u0026map-\u003evms;\n2536:\t\n2537:\t\t/* Find the first overlapping VMA and initialise unmap state. */\n2538:\t\tvms-\u003evma = vma_find(vmi, map-\u003eend);\n2539:\t\tinit_vma_munmap(vms, vmi, vms-\u003evma, map-\u003eaddr, map-\u003eend, uf,\n2540:\t\t\t\t/* unlock = */ false);\n2541:\t\n2542:\t\t/* OK, we have overlapping VMAs - prepare to unmap them. */\n2543:\t\tif (vms-\u003evma) {\n2544:\t\t\tmt_init_flags(\u0026map-\u003emt_detach,\n2545:\t\t\t\t vmi-\u003emas.tree-\u003ema_flags \u0026 MT_FLAGS_LOCK_MASK);\n2546:\t\t\tmt_on_stack(map-\u003emt_detach);\n2547:\t\t\tmas_init(\u0026map-\u003emas_detach, \u0026map-\u003emt_detach, /* addr = */ 0);\n2548:\t\t\t/* Prepare to unmap any existing mapping in the area */\n2549:\t\t\terror = vms_gather_munmap_vmas(vms, \u0026map-\u003emas_detach);\n2550:\t\t\tif (error) {\n2551:\t\t\t\t/* On error VMAs will already have been reattached. */\n2552:\t\t\t\tvms-\u003enr_pages = 0;\n2553:\t\t\t\treturn error;\n2554:\t\t\t}\n2555:\t\n2556:\t\t\tmap-\u003enext = vms-\u003enext;\n2557:\t\t\tmap-\u003eprev = vms-\u003eprev;\n2558:\t\t} else {\n2559:\t\t\tmap-\u003enext = vma_iter_next_rewind(vmi, \u0026map-\u003eprev);\n"
} |
| 63/3 | 2026/09/14 19:14 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 71191 output: 35 thoughts: 24 Thoughts: Here's my attempt at summarizing the thought process, tailored for an expert audience: **Tracing `set_desc_from_map` within `__mmap_setup`** Okay, so I'm digging into `__mmap_setup` and need to understand the role of `set_desc_from_map`. My initial thought is to pinpoint *where* exactly within the `__mmap_setup` function this particular call is made. Knowing the call context is critical – is it early in the process, during the initial descriptor allocation, or later, perhaps after address space considerations have been taken into account? The goal is to fully understand the flow and logic surrounding its use. This will inform my assessment of the function's overall memory mapping behavior. |
| 64/3 | 2026/09/14 19:14 | tool | read-file |
0mArgs:{
"File": "mm/vma.c",
"FirstLine": 2560,
"LineCount": 35
}Results: {
"Contents": "2560:\t\t}\n2561:\t\n2562:\t\t/* Check against address space limit. */\n2563:\t\tif (!may_expand_vm(map-\u003emm, \u0026map-\u003evma_flags, map-\u003epglen - vms-\u003enr_pages))\n2564:\t\t\treturn -ENOMEM;\n2565:\t\n2566:\t\t/* Private writable mapping: check memory availability. */\n2567:\t\tif (accountable_mapping(map)) {\n2568:\t\t\tmap-\u003echarged = map-\u003epglen;\n2569:\t\t\tmap-\u003echarged -= vms-\u003enr_accounted;\n2570:\t\t\tif (map-\u003echarged) {\n2571:\t\t\t\terror = security_vm_enough_memory_mm(map-\u003emm, map-\u003echarged);\n2572:\t\t\t\tif (error)\n2573:\t\t\t\t\treturn error;\n2574:\t\t\t}\n2575:\t\n2576:\t\t\tvms-\u003enr_accounted = 0;\n2577:\t\t\tvma_flags_set(\u0026map-\u003evma_flags, VMA_ACCOUNT_BIT);\n2578:\t\t}\n2579:\t\n2580:\t\t/*\n2581:\t\t * Clear PTEs while the vma is still in the tree so that rmap\n2582:\t\t * cannot race with the freeing later in the truncate scenario.\n2583:\t\t * This is also needed for mmap_file(), which is why vm_ops\n2584:\t\t * close function is called.\n2585:\t\t */\n2586:\t\tvms_clean_up_area(vms, \u0026map-\u003emas_detach);\n2587:\t\n2588:\t\tset_desc_from_map(desc, map);\n2589:\t\treturn 0;\n2590:\t}\n2591:\t\n2592:\tstatic bool map_same_file(struct mmap_state *map)\n2593:\t{\n2594:\t\treturn map-\u003evm_file == map-\u003efile;\n"
} |
| 65/3 | 2026/09/14 19:14 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 71942 output: 32 thoughts: 343 Thoughts: **Analyzing the Initialization and Validation Process** Okay, so `desc` is fully initialized before we even get to `call_mmap_prepare`. Now, let's dive into `mm/util.c` and specifically look at `compat_set_desc_from_vma` within `__compat_vma_mmap`. The code flow is interesting. First, we're taking a snapshot of the descriptor's state using `compat_set_desc_from_vma` and storing it in `prev_desc`. This seems crucial for later validation, right? We're essentially establishing a baseline. Then, `mmap_action_prepare` is called to potentially do some preparatory work. If that fails, we jump to `err_put`. Next comes the validation step with `mmap_prepare_validate`. This compares our baseline `prev_desc` with the current `desc`. This is where we ensure the caller hasn't messed things up during the preparation phase. If the validation fails, again, we hit `err_put`. Assuming everything's kosher, the VMA is updated from the descriptor using `compat_set_vma_from_desc`. Finally, any requested mmap actions are completed using `mmap_action_complete`. It all seems pretty solid in terms of ensuring consistency and preventing unexpected behavior. That initial `compat_set_desc_from_vma` is the key. It sets the stage for a proper comparison and validation. Now, let's see what `compat_set_desc_from_vma` actually does. |
| 66/3 | 2026/09/14 19:14 | tool | grepper |
0mArgs:{
"Expression": "compat_set_desc_from_vma",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/util.c=1163=EXPORT_SYMBOL(flush_dcache_folio);\n--\nmm/util.c-1166-/**\nmm/util.c:1167: * compat_set_desc_from_vma() - assigns VMA descriptor @desc fields from a VMA.\nmm/util.c-1168- * @desc: A VMA descriptor whose fields need to be set.\n--\nmm/util.c-1179- */\nmm/util.c:1180:void compat_set_desc_from_vma(struct vm_area_desc *desc,\nmm/util.c-1181-\t\t\t const struct file *file,\n--\nmm/util.c-1199-}\nmm/util.c:1200:EXPORT_SYMBOL(compat_set_desc_from_vma);\nmm/util.c-1201-\n--\nmm/util.c-1210- * The function assumes that you have obtained a VMA descriptor @desc from\nmm/util.c:1211: * compat_set_desc_from_vma(), and already executed the mmap_prepare() hook upon\nmm/util.c-1212- * it.\n--\nmm/util.c=1224=int __compat_vma_mmap(struct vm_area_desc *desc,\n--\nmm/util.c-1230-\t/* Derive state prior to mmap_prepare hook. */\nmm/util.c:1231:\tcompat_set_desc_from_vma(\u0026prev_desc, desc-\u003efile, vma);\nmm/util.c-1232-\t/* Perform any preparatory tasks for mmap action. */\n--\nmm/util.c=1278=int compat_vma_mmap(struct file *file, struct vm_area_struct *vma)\n--\nmm/util.c-1283-\nmm/util.c:1284:\tcompat_set_desc_from_vma(\u0026desc, file, vma);\nmm/util.c-1285-\terr = vfs_mmap_prepare(file, \u0026desc);\n"
} |
| 67/3 | 2026/09/14 19:14 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 73198 output: 34 thoughts: 19 Thoughts: Here's my attempt at summarizing the thought process, tailored for an expert audience: **Diving into `mm/util.c` - Line 1180 and Beyond** Okay, let's pull up the source code for `mm/util.c` and zero in on the vicinity of line 1180. I'm probably looking at some core memory management utility function, likely dealing with allocation, deallocation, or perhaps tracking. Knowing this file, it's probably *not* the low-level page allocation routines, but rather, something higher up the stack. It could be some sort of error handling, or perhaps a helper for statistics gathering. I'll need to see the context surrounding that line to get the full picture, but based on the naming of the file, it's not likely to be anything performance-critical, or I would have suggested a different file. I anticipate finding some sort of debug or tracking code. Let's see... |
| 68/3 | 2026/09/14 19:14 | tool | read-file |
0mArgs:{
"File": "mm/util.c",
"FirstLine": 1175,
"LineCount": 30
}Results: {
"Contents": "1175:\t * the purposes of doing this.\n1176:\t *\n1177:\t * Once the conversion of drivers is complete this function will no longer be\n1178:\t * required and will be removed.\n1179:\t */\n1180:\tvoid compat_set_desc_from_vma(struct vm_area_desc *desc,\n1181:\t\t\t\t const struct file *file,\n1182:\t\t\t\t const struct vm_area_struct *vma)\n1183:\t{\n1184:\t\tmemset(desc, 0, sizeof(*desc));\n1185:\t\n1186:\t\tdesc-\u003emm = vma-\u003evm_mm;\n1187:\t\tdesc-\u003efile = (struct file *)file;\n1188:\t\tdesc-\u003estart = vma-\u003evm_start;\n1189:\t\tdesc-\u003eend = vma-\u003evm_end;\n1190:\t\n1191:\t\tdesc-\u003epgoff = vma_start_pgoff(vma);\n1192:\t\tdesc-\u003evm_file = vma-\u003evm_file;\n1193:\t\tdesc-\u003evma_flags = vma-\u003eflags;\n1194:\t\tdesc-\u003epage_prot = vma-\u003evm_page_prot;\n1195:\t\tdesc-\u003evm_ops = vma-\u003evm_ops;\n1196:\t\n1197:\t\t/* Default. */\n1198:\t\tdesc-\u003eaction.type = MMAP_NOTHING;\n1199:\t}\n1200:\tEXPORT_SYMBOL(compat_set_desc_from_vma);\n1201:\t\n1202:\t/**\n1203:\t * __compat_vma_mmap() - Similar to compat_vma_mmap(), only it allows\n1204:\t * flexibility as to how the mmap_prepare callback is invoked, which is useful\n"
} |
| 69/3 | 2026/09/14 19:14 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 73950 output: 35 thoughts: 1562 Thoughts: **Analyzing Potential Uninitialized Memory Issues** Okay, so I'm diving into this code, trying to see if there are any subtle bugs related to uninitialized memory being used. It looks like the `memset` in `compat_set_desc_from_vma` should be good; `prev_desc` is completely zeroed before being passed to `mmap_prepare_validate`. That's a good starting point. Now, let's look at `mmap_file`. I see that `prev_start` and `prev_flags` are copied *before* the call to `vfs_mmap`, which is important for validation later. Then, `mmap_hook_validate` is called with these saved values. I'm checking that the error handling and dummy `vm_ops` assignment make sure that any uninitialized parts of `vma` wouldn't be used, and that the original `vma` values are maintained if there is an error. I'm feeling pretty good so far. Next, I examined `mmap_validate`. Here, the code seems pretty safe. It's comparing the old and new VMA flags and addresses to ensure the driver hasn't done anything crazy like change the address or make a non-writable mapping writable. Everything here is using references to initialized flags/addresses, which is as expected. Then, I went through `mlock.c`. I zeroed in on the `mlock_vma_pages_range` function. It seems that a boolean value, `lock`, is passed as private data and then cast to `void *`. But it's then cast back to `bool` in `mlock_pte_range`, which is a pattern that I often see and that should be perfectly fine, if it's done correctly. And it's used for conditional locking, which should not cause an issue. I see nothing obviously wrong here. Okay, now let's check `fs/coredump.c`. The code now uses `vma_test(vma, VMA_IO_BIT)` to check if a VMA should be excluded from a core dump. The previous code was using `vma->vm_flags & VM_IO`, which looked a lot less safe. The comment said "Do not dump I/O mapped devices or special mappings" but the code was literally checking only `VM_IO`. So, the old code wasn't doing what it said it was, which is worrying but not directly related to uninitialized memory. But what's going on with the new code? Is it doing the right thing, or is there a vulnerability? It's checking `VMA_IO_BIT` now, which is the current implementation. I need to make sure that the old code was correctly excluding `VM_IO`, since the old comment included the phrase "special mappings", which are not addressed by the code. That doesn't seem to be an issue here, since this code seems to be an improvement in both form and intent. All should be well. Let me look at line 1610 just to be sure... |
| 70/3 | 2026/09/14 19:14 | tool | read-file |
0mArgs:{
"File": "fs/coredump.c",
"FirstLine": 1605,
"LineCount": 30
}Results: {
"Contents": "1605:\t\t\tif (!(vma-\u003evm_flags \u0026 VM_SHARED) \u0026\u0026 FILTER(DAX_PRIVATE))\n1606:\t\t\t\tgoto whole;\n1607:\t\t\treturn 0;\n1608:\t\t}\n1609:\t\n1610:\t\t/* Hugetlb memory check */\n1611:\t\tif (vma_is_hugetlb(vma)) {\n1612:\t\t\tif ((vma-\u003evm_flags \u0026 VM_SHARED) \u0026\u0026 FILTER(HUGETLB_SHARED))\n1613:\t\t\t\tgoto whole;\n1614:\t\t\tif (!(vma-\u003evm_flags \u0026 VM_SHARED) \u0026\u0026 FILTER(HUGETLB_PRIVATE))\n1615:\t\t\t\tgoto whole;\n1616:\t\t\treturn 0;\n1617:\t\t}\n1618:\t\n1619:\t\t/* Do not dump memory-mapped I/O, which may have side effects on read. */\n1620:\t\tif (vma_test(vma, VMA_IO_BIT))\n1621:\t\t\treturn 0;\n1622:\t\n1623:\t\t/* By default, dump shared memory if mapped from an anonymous file. */\n1624:\t\tif (vma-\u003evm_flags \u0026 VM_SHARED) {\n1625:\t\t\tif (file_inode(vma-\u003evm_file)-\u003ei_nlink == 0 ?\n1626:\t\t\t FILTER(ANON_SHARED) : FILTER(MAPPED_SHARED))\n1627:\t\t\t\tgoto whole;\n1628:\t\t\treturn 0;\n1629:\t\t}\n1630:\t\n1631:\t\t/* Dump segments that have been written to. */\n1632:\t\tif ((!IS_ENABLED(CONFIG_MMU) || vma-\u003eanon_vma) \u0026\u0026 FILTER(ANON_PRIVATE))\n1633:\t\t\tgoto whole;\n1634:\t\tif (vma-\u003evm_file == NULL)\n"
} |
| 71/3 | 2026/09/14 19:14 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 76707 output: 405 thoughts: 2570 Thoughts: **KMSAN Assessment: No Need for a Session** Okay, let's break this down. The primary focus of this analysis is to determine if this patch series introduces or exposes any potential uninitialized memory reads or information leaks that would necessitate a Kernel Memory Sanitizer (KMSAN) run. My line of reasoning is as follows: First, I examined the code, focusing on the changes and new code introduced. I looked for any instances where kernel structures are being sent to user space directly (e.g., via `copy_to_user`, `put_user`, netlink, ioctl, etc.), or indirectly (through BPF buffers). I also checked for conditional logic or branching that might depend on variables or struct fields that could be uninitialized. Then I looked at the new data structures and their initialization. Finally, I reviewed potential issues that could be found with the new code using KASAN and other common tools. * In `fs/coredump.c`, I noted that the `vma_test` check is equivalent to a bitwise AND operation, meaning the check `vma->vm_flags & VM_IO` is still the most efficient way to achieve the same result. * In `madvise.c`, the patch refactors the logic around `MADV_DODUMP`. The original code had a more complex check using `VM_SPECIAL` and `VM_DROPPABLE`. The refactored code correctly utilizes `vma_is_persistent`, which is a more direct way to determine if a mapping can be dumped, and provides the `-EINVAL` error as expected when the memory is not persistent. * In `bpf/arena.c`, `VM_MIXEDMAP` is now set on the arena VMA, along with `VM_DONTEXPAND` and `VM_DONTCOPY`. This change is specific and does not raise immediate concerns. * In `kernel/events/uprobes.c` and `arch/x86/kernel/uprobes.c`, `VM_MIXEDMAP` replaces `VM_IO` in the VMA flags used for uprobes. Since uprobes involve mapping pages, `VM_MIXEDMAP` is more appropriate than `VM_IO`, which was preventing merging and expansion. This is a semantic improvement. * In `security/selinux/selinuxfs.c`, the patch removes a check on `VM_SHARED` in `sel_mmap_policy`. This removal is safe because the `mmap` path checks `f_mode & FMODE_WRITE` for `MAP_SHARED | PROT_WRITE`, and `sel_open_policy` explicitly denies write access. Also, `mmap_hook_validate` prevents clearing `VM_MAYWRITE` unless it is kernel-owned, etc. I then meticulously examined all newly introduced local variables and data structures, and the usage of existing structs. For each, I verified that they are initialized deterministically. Any new variables or structs either are initialized on declaration, are explicitly set later, or are zeroed via `memset` or a designated initializer. I saw no evidence of uninitialized memory reads. I carefully considered the four criteria that would trigger a KMSAN session. There were no kernel structures sent to user space in a way that would be vulnerable to uninitialized fields or padding. There were no conditional branches depending on uninitialized data, and the data structures were all initialized before use. There were no changes that allow surrounding code to access uninitialized bytes of existing buffers. Finally, I considered what KASAN and other standard kernel tools might catch (such as reference counting bugs or page table inconsistencies). However, these issues would be caught by existing tools, such as KASAN, LOCKDEP, or standard assertions. In conclusion, after a comprehensive review, I am confident that this patch series does NOT introduce any uninitialized memory risks or kernel information leaks that would be detectable only by KMSAN. Therefore, a KMSAN session is not needed. **NeedsKMSAN = false.** |
| 72/3 | 2026/09/14 19:15 | tool | set-results |
0mArgs:{
"KMSANReasoning": "The patch series refactors VMA flag helpers (e.g. replacing is_vm_hugetlb_page() with vma_is_hugetlb(), introducing vma_is_kernel_owned(), vma_can_merge(), vma_is_persistent(), etc.), adds descriptor-based mmap validation (mmap_prepare_validate(), mmap_hook_validate()), introduces a new discontiguous kernel page mapping mechanism (struct discontig_kernel_page_state, struct discontig_kernel_page_ops), and migrates several drivers (cmt_speech, hfi1, sg, usb mon, snd_pcm) to the new mmap / discontig mapping infrastructure.\n\nKASAN vs KMSAN applicability:\n- KMSAN detects reads of uninitialized memory (stack, heap, or page allocations) and info-leaks to user space. In this patch series, no new kernel-to-user copy interfaces (copy_to_user, put_user, netlink, ioctls) are introduced or modified. All newly introduced structures, such as struct discontig_kernel_page_state in map_discontig_kernel_pages_complete() and prev_desc in __compat_vma_mmap(), are fully zero-initialized via designated initializers or memset() prior to use. The mapped pages originate from existing driver buffers or zero-allocated pages (e.g. __GFP_ZERO in sg).\n- Potential risks in these changes relate to VMA lifecycle, reference counting (fput/get_file), page table manipulations, locking, or out-of-bounds page array indexing. These types of bugs are covered by standard KASAN, page table debugging, and LOCKDEP builds.\n\nTherefore, a dedicated KMSAN fuzzing session is not warranted.",
"NeedsKMSAN": false
}Results: {
"KMSANReasoning": "The patch series refactors VMA flag helpers (e.g. replacing is_vm_hugetlb_page() with vma_is_hugetlb(), introducing vma_is_kernel_owned(), vma_can_merge(), vma_is_persistent(), etc.), adds descriptor-based mmap validation (mmap_prepare_validate(), mmap_hook_validate()), introduces a new discontiguous kernel page mapping mechanism (struct discontig_kernel_page_state, struct discontig_kernel_page_ops), and migrates several drivers (cmt_speech, hfi1, sg, usb mon, snd_pcm) to the new mmap / discontig mapping infrastructure.\n\nKASAN vs KMSAN applicability:\n- KMSAN detects reads of uninitialized memory (stack, heap, or page allocations) and info-leaks to user space. In this patch series, no new kernel-to-user copy interfaces (copy_to_user, put_user, netlink, ioctls) are introduced or modified. All newly introduced structures, such as struct discontig_kernel_page_state in map_discontig_kernel_pages_complete() and prev_desc in __compat_vma_mmap(), are fully zero-initialized via designated initializers or memset() prior to use. The mapped pages originate from existing driver buffers or zero-allocated pages (e.g. __GFP_ZERO in sg).\n- Potential risks in these changes relate to VMA lifecycle, reference counting (fput/get_file), page table manipulations, locking, or out-of-bounds page array indexing. These types of bugs are covered by standard KASAN, page table debugging, and LOCKDEP builds.\n\nTherefore, a dedicated KMSAN fuzzing session is not warranted.",
"NeedsKMSAN": false
} |
| Total Calls | Total Tokens | Avg Tokens | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|
| Total Calls | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|