| Seq | Timestamp | Type | Name | Duration |
|---|---|---|---|---|
| 0/0 | 2026/09/04 15:31 | flow | patch-triage |
1mArgs:null Results: {
"EnableConfigs": [
"HUGETLBFS",
"TRANSPARENT_HUGEPAGE",
"USERFAULTFD",
"PROC_PAGE_MONITOR",
"PAGE_TABLE_CHECK",
"BPF_SYSCALL"
],
"FocusSymbols": [
"hugetlb_mfill_atomic_pte",
"copy_hugetlb_page_range",
"move_hugetlb_page_tables",
"huge_pmd_share",
"huge_pmd_unshare",
"collapse_huge_page",
"try_collapse_pte_mapped_thp",
"folio_walk_start",
"__page_table_check_ptes_set",
"do_pagemap_scan"
],
"KMSANReasoning": "The patch series is a purely mechanical type-system refactoring across the memory management subsystem and arch-independent code. It introduces the `hw_pte_t` type (and `__pte_from_hw()` helper) to distinguish between pointers directly referencing hardware page table entries (`hw_pte_t *`) and page table values manipulated in software (`pte_t`).\n\nNo changes are made to data initialization, memory allocation semantics, struct layouts exposed to userspace, or data copying mechanisms (e.g. `copy_to_user`). There are no new branching conditions on uninitialized memory, no modifications to bounds checks, and no risk of uninitialized memory reads or information leaks to userspace. Therefore, dedicated KMSAN fuzzing is not required.",
"NeedsKMSAN": false,
"Reasoning": "The patch introduces a new hw_pte_t type abstraction and converts raw pte_t* pointers across core memory management subsystems (hugetlb, khugepaged, pagewalk, userfaultfd, page_table_check, proc task_mmu, and bpf arena) to hw_pte_t*. Because this modifies core reachable memory management interfaces and data flow, it is worth fuzzing to detect regressions or unintended side effects.",
"WorthFuzzing": true
} |
| 1/1 | 2026/09/04 15:31 | action | read-patch-diff |
0mArgs:null Results: {
"PatchDiff": "commit e8ceacb10b59eb9d38e460ed31a10da5b37fe459\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Fri Sep 4 15:30:59 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/MAINTAINERS b/MAINTAINERS\nindex a9245d827ddb6..b0ce3e6693045 100644\n--- a/MAINTAINERS\n+++ b/MAINTAINERS\n@@ -17140,6 +17140,7 @@ F:\tinclude/linux/mmu_notifier.h\n F:\tinclude/linux/pagewalk.h\n F:\tinclude/linux/pgalloc.h\n F:\tinclude/linux/pgtable.h\n+F:\tinclude/linux/pgtable_types.h\n F:\tinclude/linux/ptdump.h\n F:\tinclude/linux/vmpressure.h\n F:\tinclude/linux/vmstat.h\ndiff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c\nindex d01acfb7d93d0..056faf4a3618b 100644\n--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c\n+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c\n@@ -1690,7 +1690,7 @@ static int igt_mmap_gpu(void *arg)\n \treturn 0;\n }\n \n-static int check_present_pte(pte_t *pte, unsigned long addr, void *data)\n+static int check_present_pte(hw_pte_t *pte, unsigned long addr, void *data)\n {\n \tpte_t ptent = ptep_get(pte);\n \n@@ -1703,7 +1703,7 @@ static int check_present_pte(pte_t *pte, unsigned long addr, void *data)\n \treturn 0;\n }\n \n-static int check_absent_pte(pte_t *pte, unsigned long addr, void *data)\n+static int check_absent_pte(hw_pte_t *pte, unsigned long addr, void *data)\n {\n \tpte_t ptent = ptep_get(pte);\n \ndiff --git a/drivers/gpu/drm/i915/i915_mm.c b/drivers/gpu/drm/i915/i915_mm.c\nindex fd89e7c7d8d6f..aab88e8edf946 100644\n--- a/drivers/gpu/drm/i915/i915_mm.c\n+++ b/drivers/gpu/drm/i915/i915_mm.c\n@@ -48,7 +48,7 @@ static inline unsigned long sgt_pfn(const struct remap_pfn *r)\n \t\treturn r-\u003esgt.pfn + (r-\u003esgt.curr \u003e\u003e PAGE_SHIFT);\n }\n \n-static int remap_sg(pte_t *pte, unsigned long addr, void *data)\n+static int remap_sg(hw_pte_t *pte, unsigned long addr, void *data)\n {\n \tstruct remap_pfn *r = data;\n \n@@ -70,7 +70,7 @@ static int remap_sg(pte_t *pte, unsigned long addr, void *data)\n #define EXPECTED_FLAGS (VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP)\n \n #if IS_ENABLED(CONFIG_X86)\n-static int remap_pfn(pte_t *pte, unsigned long addr, void *data)\n+static int remap_pfn(hw_pte_t *pte, unsigned long addr, void *data)\n {\n \tstruct remap_pfn *r = data;\n \ndiff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c\nindex 1dcc4675580ed..b013bcad99b5b 100644\n--- a/drivers/xen/gntdev.c\n+++ b/drivers/xen/gntdev.c\n@@ -301,7 +301,7 @@ void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)\n \n /* ------------------------------------------------------------------ */\n \n-static int find_grant_ptes(pte_t *pte, unsigned long addr, void *data)\n+static int find_grant_ptes(hw_pte_t *pte, unsigned long addr, void *data)\n {\n \tstruct gntdev_grant_map *map = data;\n \tunsigned int pgnr = (addr - map-\u003epages_vm_start) \u003e\u003e PAGE_SHIFT;\ndiff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c\nindex 7cfc28f1bb86f..e724cc053e264 100644\n--- a/drivers/xen/privcmd.c\n+++ b/drivers/xen/privcmd.c\n@@ -1658,7 +1658,7 @@ static int privcmd_mmap(struct file *file, struct vm_area_struct *vma)\n * on a per pfn/pte basis. Mapping calls that fail with ENOENT\n * can be then retried until success.\n */\n-static int is_mapped_fn(pte_t *pte, unsigned long addr, void *data)\n+static int is_mapped_fn(hw_pte_t *pte, unsigned long addr, void *data)\n {\n \treturn pte_none(ptep_get(pte)) ? 0 : -EBUSY;\n }\ndiff --git a/drivers/xen/xenbus/xenbus_client.c b/drivers/xen/xenbus/xenbus_client.c\nindex 27682cb5e58ab..387f97a555594 100644\n--- a/drivers/xen/xenbus/xenbus_client.c\n+++ b/drivers/xen/xenbus/xenbus_client.c\n@@ -749,7 +749,7 @@ int xenbus_unmap_ring_vfree(struct xenbus_device *dev, void *vaddr)\n EXPORT_SYMBOL_GPL(xenbus_unmap_ring_vfree);\n \n #ifdef CONFIG_XEN_PV\n-static int map_ring_apply(pte_t *pte, unsigned long addr, void *data)\n+static int map_ring_apply(hw_pte_t *pte, unsigned long addr, void *data)\n {\n \tstruct map_ring_valloc *info = data;\n \ndiff --git a/drivers/xen/xlate_mmu.c b/drivers/xen/xlate_mmu.c\nindex 8efd7b55223fb..d7244f9f9a545 100644\n--- a/drivers/xen/xlate_mmu.c\n+++ b/drivers/xen/xlate_mmu.c\n@@ -93,7 +93,7 @@ static void setup_hparams(unsigned long gfn, void *data)\n \tinfo-\u003efgfn++;\n }\n \n-static int remap_pte_fn(pte_t *ptep, unsigned long addr, void *data)\n+static int remap_pte_fn(hw_pte_t *ptep, unsigned long addr, void *data)\n {\n \tstruct remap_data *info = data;\n \tstruct page *page = info-\u003epages[info-\u003eindex++];\n@@ -269,7 +269,7 @@ struct remap_pfn {\n \tunsigned long i;\n };\n \n-static int remap_pfn_fn(pte_t *ptep, unsigned long addr, void *data)\n+static int remap_pfn_fn(hw_pte_t *ptep, unsigned long addr, void *data)\n {\n \tstruct remap_pfn *r = data;\n \tstruct page *page = r-\u003epages[r-\u003ei];\ndiff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c\nindex 7611a8470ea26..ddaab714c3e02 100644\n--- a/fs/hugetlbfs/inode.c\n+++ b/fs/hugetlbfs/inode.c\n@@ -328,7 +328,8 @@ static void hugetlb_delete_from_page_cache(struct folio *folio)\n static bool hugetlb_vma_maps_pfn(struct vm_area_struct *vma,\n \t\t\t\tunsigned long addr, unsigned long pfn)\n {\n-\tpte_t *ptep, pte;\n+\thw_pte_t *ptep;\n+\tpte_t pte;\n \n \tptep = hugetlb_walk(vma, addr, huge_page_size(hstate_vma(vma)));\n \tif (!ptep)\ndiff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c\nindex e671b4fd8dedd..4211c33b06491 100644\n--- a/fs/proc/task_mmu.c\n+++ b/fs/proc/task_mmu.c\n@@ -982,7 +982,7 @@ static void smaps_pte_hole_lookup(unsigned long addr, struct mm_walk *walk)\n #endif\n }\n \n-static void smaps_pte_entry(pte_t *pte, unsigned long addr,\n+static void smaps_pte_entry(hw_pte_t *pte, unsigned long addr,\n \t\tstruct mm_walk *walk)\n {\n \tstruct mem_size_stats *mss = walk-\u003eprivate;\n@@ -1076,7 +1076,7 @@ static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,\n \t\t\t struct mm_walk *walk)\n {\n \tstruct vm_area_struct *vma = walk-\u003evma;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tspinlock_t *ptl;\n \n \tptl = pmd_trans_huge_lock(pmd, vma);\n@@ -1199,7 +1199,7 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma)\n }\n \n #ifdef CONFIG_HUGETLB_PAGE\n-static int smaps_hugetlb_range(pte_t *pte, unsigned long hmask,\n+static int smaps_hugetlb_range(hw_pte_t *pte, unsigned long hmask,\n \t\t\t\t unsigned long addr, unsigned long end,\n \t\t\t\t struct mm_walk *walk)\n {\n@@ -1622,7 +1622,7 @@ static inline bool pte_is_pinned(struct vm_area_struct *vma, unsigned long addr,\n }\n \n static inline void clear_soft_dirty(struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *pte)\n+\t\tunsigned long addr, hw_pte_t *pte)\n {\n \tif (!pgtable_supports_soft_dirty())\n \t\treturn;\n@@ -1690,7 +1690,8 @@ static int clear_refs_pte_range(pmd_t *pmd, unsigned long addr,\n {\n \tstruct clear_refs_private *cp = walk-\u003eprivate;\n \tstruct vm_area_struct *vma = walk-\u003evma;\n-\tpte_t *pte, ptent;\n+\thw_pte_t *pte;\n+\tpte_t ptent;\n \tspinlock_t *ptl;\n \tstruct folio *folio;\n \n@@ -2090,7 +2091,7 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end,\n \tstruct vm_area_struct *vma = walk-\u003evma;\n \tstruct pagemapread *pm = walk-\u003eprivate;\n \tspinlock_t *ptl;\n-\tpte_t *pte, *orig_pte;\n+\thw_pte_t *pte, *orig_pte;\n \tint err = 0;\n \n #ifdef CONFIG_TRANSPARENT_HUGEPAGE\n@@ -2128,7 +2129,7 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end,\n \n #ifdef CONFIG_HUGETLB_PAGE\n /* This function walks within one hugetlb entry in the single call */\n-static int pagemap_hugetlb_range(pte_t *ptep, unsigned long hmask,\n+static int pagemap_hugetlb_range(hw_pte_t *ptep, unsigned long hmask,\n \t\t\t\t unsigned long addr, unsigned long end,\n \t\t\t\t struct mm_walk *walk)\n {\n@@ -2419,7 +2420,7 @@ static unsigned long pagemap_page_category(struct pagemap_scan_private *p,\n }\n \n static void make_uffd_wp_pte(struct vm_area_struct *vma,\n-\t\t\t unsigned long addr, pte_t *pte, pte_t ptent)\n+\t\t\t unsigned long addr, hw_pte_t *pte, pte_t ptent)\n {\n \tif (pte_present(ptent)) {\n \t\tpte_t old_pte;\n@@ -2552,7 +2553,7 @@ static unsigned long pagemap_hugetlb_category(struct vm_area_struct *vma,\n }\n \n static void make_uffd_wp_huge_pte(struct vm_area_struct *vma,\n-\t\t\t\t unsigned long addr, pte_t *ptep,\n+\t\t\t\t unsigned long addr, hw_pte_t *ptep,\n \t\t\t\t pte_t ptent)\n {\n \tconst unsigned long psize = huge_page_size(hstate_vma(vma));\n@@ -2786,7 +2787,7 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start,\n \tstruct pagemap_scan_private *p = walk-\u003eprivate;\n \tstruct vm_area_struct *vma = walk-\u003evma;\n \tunsigned long addr, flush_end = 0;\n-\tpte_t *pte, *start_pte;\n+\thw_pte_t *pte, *start_pte;\n \tspinlock_t *ptl;\n \tint ret;\n \n@@ -2880,7 +2881,7 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start,\n }\n \n #ifdef CONFIG_HUGETLB_PAGE\n-static int pagemap_scan_hugetlb_entry(pte_t *ptep, unsigned long hmask,\n+static int pagemap_scan_hugetlb_entry(hw_pte_t *ptep, unsigned long hmask,\n \t\t\t\t unsigned long start, unsigned long end,\n \t\t\t\t struct mm_walk *walk)\n {\n@@ -2961,7 +2962,7 @@ static int pagemap_scan_hugetlb_hole_wp(struct vm_area_struct *vma,\n \tunsigned long psize = huge_page_size(h);\n \tstruct mm_struct *mm = vma-\u003evm_mm;\n \tspinlock_t *ptl;\n-\tpte_t *ptep;\n+\thw_pte_t *ptep;\n \tpte_t pte;\n \n \tfor (addr = ALIGN_DOWN(addr, psize); addr \u003c end; addr += psize) {\n@@ -3345,8 +3346,8 @@ static int gather_pte_stats(pmd_t *pmd, unsigned long addr,\n \tstruct numa_maps *md = walk-\u003eprivate;\n \tstruct vm_area_struct *vma = walk-\u003evma;\n \tspinlock_t *ptl;\n-\tpte_t *orig_pte;\n-\tpte_t *pte;\n+\thw_pte_t *orig_pte;\n+\thw_pte_t *pte;\n \n #ifdef CONFIG_TRANSPARENT_HUGEPAGE\n \tptl = pmd_trans_huge_lock(pmd, vma);\n@@ -3379,7 +3380,7 @@ static int gather_pte_stats(pmd_t *pmd, unsigned long addr,\n \treturn 0;\n }\n #ifdef CONFIG_HUGETLB_PAGE\n-static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask,\n+static int gather_hugetlb_stats(hw_pte_t *pte, unsigned long hmask,\n \t\tunsigned long addr, unsigned long end, struct mm_walk *walk)\n {\n \tpte_t huge_pte;\n@@ -3402,7 +3403,7 @@ static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask,\n }\n \n #else\n-static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask,\n+static int gather_hugetlb_stats(hw_pte_t *pte, unsigned long hmask,\n \t\tunsigned long addr, unsigned long end, struct mm_walk *walk)\n {\n \treturn 0;\ndiff --git a/include/asm-generic/hugetlb.h b/include/asm-generic/hugetlb.h\nindex 635c41cc34797..3bfad4a99b64d 100644\n--- a/include/asm-generic/hugetlb.h\n+++ b/include/asm-generic/hugetlb.h\n@@ -60,7 +60,7 @@ static inline int huge_pte_uffd(pte_t pte)\n \n #ifndef __HAVE_ARCH_HUGE_PTE_CLEAR\n static inline void huge_pte_clear(struct mm_struct *mm, unsigned long addr,\n-\t\t pte_t *ptep, unsigned long sz)\n+\t\t hw_pte_t *ptep, unsigned long sz)\n {\n \tpte_clear(mm, addr, ptep);\n }\n@@ -68,7 +68,7 @@ static inline void huge_pte_clear(struct mm_struct *mm, unsigned long addr,\n \n #ifndef __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT\n static inline void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,\n-\t\tpte_t *ptep, pte_t pte, unsigned long sz)\n+\t\thw_pte_t *ptep, pte_t pte, unsigned long sz)\n {\n \tset_pte_at(mm, addr, ptep, pte);\n }\n@@ -76,7 +76,7 @@ static inline void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,\n \n #ifndef __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR\n static inline pte_t huge_ptep_get_and_clear(struct mm_struct *mm,\n-\t\tunsigned long addr, pte_t *ptep, unsigned long sz)\n+\t\tunsigned long addr, hw_pte_t *ptep, unsigned long sz)\n {\n \treturn ptep_get_and_clear(mm, addr, ptep);\n }\n@@ -84,7 +84,7 @@ static inline pte_t huge_ptep_get_and_clear(struct mm_struct *mm,\n \n #ifndef __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH\n static inline pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep)\n+\t\tunsigned long addr, hw_pte_t *ptep)\n {\n \treturn ptep_clear_flush(vma, addr, ptep);\n }\n@@ -99,7 +99,7 @@ static inline int huge_pte_none(pte_t pte)\n \n #ifndef __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT\n static inline void huge_ptep_set_wrprotect(struct mm_struct *mm,\n-\t\tunsigned long addr, pte_t *ptep)\n+\t\tunsigned long addr, hw_pte_t *ptep)\n {\n \tptep_set_wrprotect(mm, addr, ptep);\n }\n@@ -107,7 +107,7 @@ static inline void huge_ptep_set_wrprotect(struct mm_struct *mm,\n \n #ifndef __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS\n static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep,\n+\t\tunsigned long addr, hw_pte_t *ptep,\n \t\tpte_t pte, int dirty)\n {\n \treturn ptep_set_access_flags(vma, addr, ptep, pte, dirty);\n@@ -115,7 +115,8 @@ static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma,\n #endif\n \n #ifndef __HAVE_ARCH_HUGE_PTEP_GET\n-static inline pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep)\n+static inline pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr,\n+\t\thw_pte_t *ptep)\n {\n \treturn ptep_get(ptep);\n }\ndiff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h\nindex 051aa1331051c..b35e5a2158ada 100644\n--- a/include/asm-generic/pgalloc.h\n+++ b/include/asm-generic/pgalloc.h\n@@ -16,7 +16,7 @@\n *\n * Return: pointer to the allocated memory or %NULL on error\n */\n-static inline pte_t *__pte_alloc_one_kernel_noprof(struct mm_struct *mm)\n+static inline hw_pte_t *__pte_alloc_one_kernel_noprof(struct mm_struct *mm)\n {\n \tstruct ptdesc *ptdesc = pagetable_alloc_noprof(GFP_PGTABLE_KERNEL, 0);\n \n@@ -40,7 +40,7 @@ static inline pte_t *__pte_alloc_one_kernel_noprof(struct mm_struct *mm)\n *\n * Return: pointer to the allocated memory or %NULL on error\n */\n-static inline pte_t *pte_alloc_one_kernel_noprof(struct mm_struct *mm)\n+static inline hw_pte_t *pte_alloc_one_kernel_noprof(struct mm_struct *mm)\n {\n \treturn __pte_alloc_one_kernel_noprof(mm);\n }\n@@ -52,7 +52,7 @@ static inline pte_t *pte_alloc_one_kernel_noprof(struct mm_struct *mm)\n * @mm: the mm_struct of the current context\n * @pte: pointer to the memory containing the page table\n */\n-static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)\n+static inline void pte_free_kernel(struct mm_struct *mm, hw_pte_t *pte)\n {\n \tpagetable_dtor_free(virt_to_ptdesc(pte));\n }\ndiff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h\nindex bdcc2778ac64f..2c3517800a9a8 100644\n--- a/include/asm-generic/tlb.h\n+++ b/include/asm-generic/tlb.h\n@@ -644,7 +644,8 @@ static inline void tlb_flush_p4d_range(struct mmu_gather *tlb,\n }\n \n #ifndef __tlb_remove_tlb_entry\n-static inline void __tlb_remove_tlb_entry(struct mmu_gather *tlb, pte_t *ptep, unsigned long address)\n+static inline void __tlb_remove_tlb_entry(struct mmu_gather *tlb,\n+\t\thw_pte_t *ptep, unsigned long address)\n {\n }\n #endif\n@@ -670,7 +671,7 @@ static inline void __tlb_remove_tlb_entry(struct mmu_gather *tlb, pte_t *ptep, u\n * consecutive ptes instead of only a single one.\n */\n static inline void tlb_remove_tlb_entries(struct mmu_gather *tlb,\n-\t\tpte_t *ptep, unsigned int nr, unsigned long address)\n+\t\thw_pte_t *ptep, unsigned int nr, unsigned long address)\n {\n \ttlb_flush_pte_range(tlb, address, PAGE_SIZE * nr);\n \tfor (;;) {\ndiff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h\nindex 80a5a03e9cee7..6bbf2449e7d27 100644\n--- a/include/linux/hugetlb.h\n+++ b/include/linux/hugetlb.h\n@@ -141,7 +141,7 @@ unsigned long hugetlb_total_pages(void);\n vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma,\n \t\t\tunsigned long address, unsigned int flags);\n #ifdef CONFIG_USERFAULTFD\n-int hugetlb_mfill_atomic_pte(pte_t *dst_pte,\n+int hugetlb_mfill_atomic_pte(hw_pte_t *dst_pte,\n \t\t\t struct vm_area_struct *dst_vma,\n \t\t\t unsigned long dst_addr,\n \t\t\t unsigned long src_addr,\n@@ -161,7 +161,7 @@ void hugetlb_fix_reserve_counts(struct inode *inode);\n extern struct mutex *hugetlb_fault_mutex_table;\n u32 hugetlb_fault_mutex_hash(struct address_space *mapping, pgoff_t idx);\n \n-pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,\n+hw_pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,\n \t\t unsigned long addr, pud_t *pud);\n bool hugetlbfs_pagecache_present(struct hstate *h,\n \t\t\t\t struct vm_area_struct *vma,\n@@ -185,22 +185,22 @@ void hugetlb_bootmem_set_nodes(void);\n * which may go down to the lowest PTE level in their huge_pte_offset() and\n * huge_pte_alloc(): to avoid reliance on pte_offset_map() without pte_unmap().\n */\n-static inline pte_t *pte_offset_huge(pmd_t *pmd, unsigned long address)\n+static inline hw_pte_t *pte_offset_huge(pmd_t *pmd, unsigned long address)\n {\n \treturn pte_offset_kernel(pmd, address);\n }\n-static inline pte_t *pte_alloc_huge(struct mm_struct *mm, pmd_t *pmd,\n+static inline hw_pte_t *pte_alloc_huge(struct mm_struct *mm, pmd_t *pmd,\n \t\t\t\t unsigned long address)\n {\n \treturn pte_alloc(mm, pmd) ? NULL : pte_offset_huge(pmd, address);\n }\n #endif\n \n-pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,\n+hw_pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,\n \t\t\tunsigned long addr, unsigned long sz);\n /*\n * huge_pte_offset(): Walk the hugetlb pgtable until the last level PTE.\n- * Returns the pte_t* if found, or NULL if the address is not mapped.\n+ * Returns the hw_pte_t* if found, or NULL if the address is not mapped.\n *\n * IMPORTANT: we should normally not directly call this function, instead\n * this is only a common interface to implement arch-specific\n@@ -235,11 +235,11 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,\n * a concurrent pmd unshare, but it makes sure the pgtable page is safe to\n * access.\n */\n-pte_t *huge_pte_offset(struct mm_struct *mm,\n+hw_pte_t *huge_pte_offset(struct mm_struct *mm,\n \t\t unsigned long addr, unsigned long sz);\n unsigned long hugetlb_mask_last_page(struct hstate *h);\n int huge_pmd_unshare(struct mmu_gather *tlb, struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep);\n+\t\tunsigned long addr, hw_pte_t *ptep);\n void huge_pmd_unshare_flush(struct mmu_gather *tlb, struct vm_area_struct *vma);\n void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma,\n \t\t\t\tunsigned long *start, unsigned long *end);\n@@ -301,7 +301,8 @@ static inline struct address_space *hugetlb_folio_mapping_lock_write(\n }\n \n static inline int huge_pmd_unshare(struct mmu_gather *tlb,\n-\t\tstruct vm_area_struct *vma, unsigned long addr, pte_t *ptep)\n+\t\tstruct vm_area_struct *vma, unsigned long addr,\n+\t\thw_pte_t *ptep)\n {\n \treturn 0;\n }\n@@ -393,7 +394,7 @@ static inline int is_hugepage_only_range(struct mm_struct *mm,\n }\n \n #ifdef CONFIG_USERFAULTFD\n-static inline int hugetlb_mfill_atomic_pte(pte_t *dst_pte,\n+static inline int hugetlb_mfill_atomic_pte(hw_pte_t *dst_pte,\n \t\t\t\t\t struct vm_area_struct *dst_vma,\n \t\t\t\t\t unsigned long dst_addr,\n \t\t\t\t\t unsigned long src_addr,\n@@ -405,7 +406,7 @@ static inline int hugetlb_mfill_atomic_pte(pte_t *dst_pte,\n }\n #endif /* CONFIG_USERFAULTFD */\n \n-static inline pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr,\n+static inline hw_pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr,\n \t\t\t\t\tunsigned long sz)\n {\n \treturn NULL;\n@@ -985,7 +986,8 @@ static inline bool htlb_allow_alloc_fallback(enum migrate_reason reason)\n }\n \n static inline spinlock_t *huge_pte_lockptr(struct hstate *h,\n-\t\t\t\t\t struct mm_struct *mm, pte_t *pte)\n+\t\t\t\t\t struct mm_struct *mm,\n+\t\t\t\t\t hw_pte_t *pte)\n {\n \tconst unsigned long size = huge_page_size(h);\n \n@@ -1049,7 +1051,8 @@ static inline void hugetlb_count_sub(long l, struct mm_struct *mm)\n #ifndef huge_ptep_modify_prot_start\n #define huge_ptep_modify_prot_start huge_ptep_modify_prot_start\n static inline pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma,\n-\t\t\t\t\t\tunsigned long addr, pte_t *ptep)\n+\t\t\t\t\t\tunsigned long addr,\n+\t\t\t\t\t\thw_pte_t *ptep)\n {\n \tunsigned long psize = huge_page_size(hstate_vma(vma));\n \n@@ -1060,7 +1063,8 @@ static inline pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma,\n #ifndef huge_ptep_modify_prot_commit\n #define huge_ptep_modify_prot_commit huge_ptep_modify_prot_commit\n static inline void huge_ptep_modify_prot_commit(struct vm_area_struct *vma,\n-\t\t\t\t\t\tunsigned long addr, pte_t *ptep,\n+\t\t\t\t\t\tunsigned long addr,\n+\t\t\t\t\t\thw_pte_t *ptep,\n \t\t\t\t\t\tpte_t old_pte, pte_t pte)\n {\n \tunsigned long psize = huge_page_size(hstate_vma(vma));\n@@ -1248,7 +1252,8 @@ static inline bool htlb_allow_alloc_fallback(enum migrate_reason reason)\n }\n \n static inline spinlock_t *huge_pte_lockptr(struct hstate *h,\n-\t\t\t\t\t struct mm_struct *mm, pte_t *pte)\n+\t\t\t\t\t struct mm_struct *mm,\n+\t\t\t\t\t hw_pte_t *pte)\n {\n \treturn \u0026mm-\u003epage_table_lock;\n }\n@@ -1265,21 +1270,23 @@ static inline void hugetlb_count_sub(long l, struct mm_struct *mm)\n {\n }\n \n-pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep);\n+pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, hw_pte_t *ptep);\n unsigned long huge_pte_dirty(pte_t pte);\n \n static inline pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,\n-\t\t\t\t\t unsigned long addr, pte_t *ptep)\n+\t\t\t\t\t unsigned long addr, hw_pte_t *ptep)\n {\n #ifdef CONFIG_MMU\n \treturn ptep_get(ptep);\n #else\n-\treturn *ptep;\n+\t/* No attached PTE that requires ptep_get(). */\n+\treturn __pte_from_hw(*ptep);\n #endif\n }\n \n static inline void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,\n-\t\t\t\t pte_t *ptep, pte_t pte, unsigned long sz)\n+\t\t\t\t hw_pte_t *ptep, pte_t pte,\n+\t\t\t\t unsigned long sz)\n {\n }\n \n@@ -1307,7 +1314,7 @@ static inline void hugetlb_bootmem_struct_page_init(void)\n #endif\t/* CONFIG_HUGETLB_PAGE */\n \n static inline spinlock_t *huge_pte_lock(struct hstate *h,\n-\t\t\t\t\tstruct mm_struct *mm, pte_t *pte)\n+\t\t\t\t\tstruct mm_struct *mm, hw_pte_t *pte)\n {\n \tspinlock_t *ptl;\n \n@@ -1325,12 +1332,12 @@ static inline __init void hugetlb_cma_reserve(void)\n #endif\n \n #ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING\n-static inline bool hugetlb_pmd_shared(pte_t *pte)\n+static inline bool hugetlb_pmd_shared(hw_pte_t *pte)\n {\n \treturn ptdesc_pmd_is_shared(virt_to_ptdesc(pte));\n }\n #else\n-static inline bool hugetlb_pmd_shared(pte_t *pte)\n+static inline bool hugetlb_pmd_shared(hw_pte_t *pte)\n {\n \treturn false;\n }\n@@ -1357,7 +1364,7 @@ bool __vma_private_lock(struct vm_area_struct *vma);\n * Safe version of huge_pte_offset() to check the locks. See comments\n * above huge_pte_offset().\n */\n-static inline pte_t *\n+static inline hw_pte_t *\n hugetlb_walk(struct vm_area_struct *vma, unsigned long addr, unsigned long sz)\n {\n #if defined(CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING) \u0026\u0026 defined(CONFIG_LOCKDEP)\ndiff --git a/include/linux/kasan.h b/include/linux/kasan.h\nindex bf233bde68c7e..ff41949c0ac08 100644\n--- a/include/linux/kasan.h\n+++ b/include/linux/kasan.h\n@@ -51,7 +51,7 @@ typedef unsigned int __bitwise kasan_vmalloc_flags_t;\n #endif\n \n extern unsigned char kasan_early_shadow_page[PAGE_SIZE];\n-extern pte_t kasan_early_shadow_pte[MAX_PTRS_PER_PTE + PTE_HWTABLE_PTRS];\n+extern hw_pte_t kasan_early_shadow_pte[MAX_PTRS_PER_PTE + PTE_HWTABLE_PTRS];\n extern pmd_t kasan_early_shadow_pmd[MAX_PTRS_PER_PMD];\n extern pud_t kasan_early_shadow_pud[MAX_PTRS_PER_PUD];\n extern p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D];\ndiff --git a/include/linux/mm.h b/include/linux/mm.h\nindex c49ef99b4413b..892b280fe9c0c 100644\n--- a/include/linux/mm.h\n+++ b/include/linux/mm.h\n@@ -769,7 +769,7 @@ struct vm_fault {\n \t\t\t\t\t * VM_FAULT_ERROR).\n \t\t\t\t\t */\n \t/* These three entries are valid only while holding ptl lock */\n-\tpte_t *pte;\t\t\t/* Pointer to pte entry matching\n+\thw_pte_t *pte;\t\t\t/* Pointer to pte entry matching\n \t\t\t\t\t * the 'address'. NULL if the page\n \t\t\t\t\t * table hasn't been allocated.\n \t\t\t\t\t */\n@@ -3231,7 +3231,7 @@ struct follow_pfnmap_args {\n \t * The caller shouldn't touch any of these.\n \t */\n \tspinlock_t *lock;\n-\tpte_t *ptep;\n+\thw_pte_t *ptep;\n \t/**\n \t * Outputs:\n \t *\n@@ -3568,7 +3568,7 @@ static inline pud_t pud_mkspecial(pud_t pud)\n }\n #endif\t/* CONFIG_ARCH_SUPPORTS_PUD_PFNMAP */\n \n-extern pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,\n+extern hw_pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,\n \t\t\t spinlock_t **ptl);\n \n #ifdef __PAGETABLE_P4D_FOLDED\n@@ -3846,7 +3846,7 @@ static inline spinlock_t *pte_lockptr(struct mm_struct *mm, pmd_t *pmd)\n \treturn ptlock_ptr(page_ptdesc(pmd_page(*pmd)));\n }\n \n-static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, pte_t *pte)\n+static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, hw_pte_t *pte)\n {\n \tBUILD_BUG_ON(IS_ENABLED(CONFIG_HIGHPTE));\n \tBUILD_BUG_ON(MAX_PTRS_PER_PTE * sizeof(pte_t) \u003e PAGE_SIZE);\n@@ -3877,7 +3877,7 @@ static inline spinlock_t *pte_lockptr(struct mm_struct *mm, pmd_t *pmd)\n {\n \treturn \u0026mm-\u003epage_table_lock;\n }\n-static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, pte_t *pte)\n+static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, hw_pte_t *pte)\n {\n \treturn \u0026mm-\u003epage_table_lock;\n }\n@@ -3918,19 +3918,19 @@ static inline bool pagetable_pte_ctor(struct mm_struct *mm,\n \treturn true;\n }\n \n-pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp);\n+hw_pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp);\n \n-static inline pte_t *pte_offset_map(pmd_t *pmd, unsigned long addr)\n+static inline hw_pte_t *pte_offset_map(pmd_t *pmd, unsigned long addr)\n {\n \treturn __pte_offset_map(pmd, addr, NULL);\n }\n \n-pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd,\n+hw_pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd,\n \t\t\t unsigned long addr, spinlock_t **ptlp);\n \n-pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,\n+hw_pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,\n \t\t\t\tunsigned long addr, spinlock_t **ptlp);\n-pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,\n+hw_pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,\n \t\t\t\tunsigned long addr, pmd_t *pmdvalp,\n \t\t\t\tspinlock_t **ptlp);\n \n@@ -4873,7 +4873,7 @@ static inline bool gup_can_follow_protnone(const struct vm_area_struct *vma,\n \treturn !vma_is_accessible(vma);\n }\n \n-typedef int (*pte_fn_t)(pte_t *pte, unsigned long addr, void *data);\n+typedef int (*pte_fn_t)(hw_pte_t *pte, unsigned long addr, void *data);\n extern int apply_to_page_range(struct mm_struct *mm, unsigned long address,\n \t\t\t unsigned long size, pte_fn_t fn, void *data);\n extern int apply_to_existing_page_range(struct mm_struct *mm,\n@@ -5122,7 +5122,7 @@ void *vmemmap_alloc_block(unsigned long size, int node);\n struct vmem_altmap;\n void *vmemmap_alloc_block_buf(unsigned long size, int node,\n \t\t\t struct vmem_altmap *altmap);\n-void vmemmap_verify(pte_t *, int, unsigned long, unsigned long);\n+void vmemmap_verify(hw_pte_t *, int, unsigned long, unsigned long);\n void vmemmap_set_pmd(pmd_t *pmd, void *p, int node,\n \t\t unsigned long addr, unsigned long next);\n int vmemmap_check_pmd(pmd_t *pmd, int node,\n@@ -5487,7 +5487,7 @@ static inline bool snapshot_page_is_faithful(const struct page_snapshot *ps)\n \n void snapshot_page(struct page_snapshot *ps, const struct page *page);\n \n-void map_anon_folio_pte_nopf(struct folio *folio, pte_t *pte,\n+void map_anon_folio_pte_nopf(struct folio *folio, hw_pte_t *pte,\n \t\tstruct vm_area_struct *vma, unsigned long addr,\n \t\tbool uffd_wp);\n \ndiff --git a/include/linux/page_table_check.h b/include/linux/page_table_check.h\nindex 12268a32e8be1..933c1028c2232 100644\n--- a/include/linux/page_table_check.h\n+++ b/include/linux/page_table_check.h\n@@ -7,6 +7,8 @@\n #ifndef __LINUX_PAGE_TABLE_CHECK_H\n #define __LINUX_PAGE_TABLE_CHECK_H\n \n+#include \u003clinux/pgtable_types.h\u003e\n+\n #ifdef CONFIG_PAGE_TABLE_CHECK\n #include \u003clinux/jump_label.h\u003e\n \n@@ -21,7 +23,7 @@ void __page_table_check_pmd_clear(struct mm_struct *mm, unsigned long addr,\n void __page_table_check_pud_clear(struct mm_struct *mm, unsigned long addr,\n \t\t\t\t pud_t pud);\n void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr,\n-\t\tpte_t *ptep, pte_t pte, unsigned int nr);\n+\t\thw_pte_t *ptep, pte_t pte, unsigned int nr);\n void __page_table_check_pmds_set(struct mm_struct *mm, unsigned long addr,\n \t\tpmd_t *pmdp, pmd_t pmd, unsigned int nr);\n void __page_table_check_puds_set(struct mm_struct *mm, unsigned long addr,\n@@ -74,7 +76,8 @@ static inline void page_table_check_pud_clear(struct mm_struct *mm,\n }\n \n static inline void page_table_check_ptes_set(struct mm_struct *mm,\n-\t\t\t\t\t unsigned long addr, pte_t *ptep,\n+\t\t\t\t\t unsigned long addr,\n+\t\t\t\t\t hw_pte_t *ptep,\n \t\t\t\t\t pte_t pte, unsigned int nr)\n {\n \tif (static_branch_likely(\u0026page_table_check_disabled))\n@@ -137,7 +140,8 @@ static inline void page_table_check_pud_clear(struct mm_struct *mm,\n }\n \n static inline void page_table_check_ptes_set(struct mm_struct *mm,\n-\t\t\t\t\t unsigned long addr, pte_t *ptep,\n+\t\t\t\t\t unsigned long addr,\n+\t\t\t\t\t hw_pte_t *ptep,\n \t\t\t\t\t pte_t pte, unsigned int nr)\n {\n }\ndiff --git a/include/linux/pagewalk.h b/include/linux/pagewalk.h\nindex b41d7265c01bc..7ab2eb39e02c0 100644\n--- a/include/linux/pagewalk.h\n+++ b/include/linux/pagewalk.h\n@@ -38,7 +38,7 @@ enum page_walk_lock {\n *\t\t\tnot trigger for any populated ranges.\n * @hugetlb_entry:\tif set, called for each hugetlb entry. This hook\n *\t\t\tfunction is called with the vma lock held, in order to\n- *\t\t\tprotect against a concurrent freeing of the pte_t* or\n+ *\t\t\tprotect against a concurrent freeing of the hw_pte_t* or\n *\t\t\tthe ptl. In some cases, the hook function needs to drop\n *\t\t\tand retake the vma lock in order to avoid deadlocks\n *\t\t\twhile calling other functions. In such cases the hook\n@@ -76,11 +76,11 @@ struct mm_walk_ops {\n \t\t\t unsigned long next, struct mm_walk *walk);\n \tint (*pmd_entry)(pmd_t *pmd, unsigned long addr,\n \t\t\t unsigned long next, struct mm_walk *walk);\n-\tint (*pte_entry)(pte_t *pte, unsigned long addr,\n+\tint (*pte_entry)(hw_pte_t *pte, unsigned long addr,\n \t\t\t unsigned long next, struct mm_walk *walk);\n \tint (*pte_hole)(unsigned long addr, unsigned long next,\n \t\t\tint depth, struct mm_walk *walk);\n-\tint (*hugetlb_entry)(pte_t *pte, unsigned long hmask,\n+\tint (*hugetlb_entry)(hw_pte_t *pte, unsigned long hmask,\n \t\t\t unsigned long addr, unsigned long next,\n \t\t\t struct mm_walk *walk);\n \tint (*test_walk)(unsigned long addr, unsigned long next,\n@@ -89,7 +89,7 @@ struct mm_walk_ops {\n \t\t struct mm_walk *walk);\n \tvoid (*post_vma)(struct mm_walk *walk);\n \tint (*install_pte)(unsigned long addr, unsigned long next,\n-\t\t\t pte_t *ptep, struct mm_walk *walk);\n+\t\t\t pte_t *ptentp, struct mm_walk *walk);\n \tenum page_walk_lock walk_lock;\n };\n \n@@ -173,7 +173,7 @@ struct folio_walk {\n \tstruct page *page;\n \tenum folio_walk_level level;\n \tunion {\n-\t\tpte_t *ptep;\n+\t\thw_pte_t *ptep;\n \t\tpud_t *pudp;\n \t\tpmd_t *pmdp;\n \t};\ndiff --git a/include/linux/pgtable.h b/include/linux/pgtable.h\nindex e3c8ab96941c5..41337da0d65aa 100644\n--- a/include/linux/pgtable.h\n+++ b/include/linux/pgtable.h\n@@ -4,6 +4,7 @@\n \n #include \u003clinux/pfn.h\u003e\n #include \u003casm/pgtable.h\u003e\n+#include \u003clinux/pgtable_types.h\u003e\n \n #define PMD_ORDER\t(PMD_SHIFT - PAGE_SHIFT)\n #define PUD_ORDER\t(PUD_SHIFT - PAGE_SHIFT)\n@@ -93,26 +94,26 @@ static inline void pud_init(void *addr)\n #endif\n \n #ifndef pte_offset_kernel\n-static inline pte_t *pte_offset_kernel(pmd_t *pmd, unsigned long address)\n+static inline hw_pte_t *pte_offset_kernel(pmd_t *pmd, unsigned long address)\n {\n-\treturn (pte_t *)pmd_page_vaddr(*pmd) + pte_index(address);\n+\treturn (hw_pte_t *)pmd_page_vaddr(*pmd) + pte_index(address);\n }\n #define pte_offset_kernel pte_offset_kernel\n #endif\n \n #ifdef CONFIG_HIGHPTE\n #define __pte_map(pmd, address) \\\n-\t((pte_t *)kmap_local_page(pmd_page(*(pmd))) + pte_index((address)))\n+\t((hw_pte_t *)kmap_local_page(pmd_page(*(pmd))) + pte_index((address)))\n #define pte_unmap(pte)\tdo {\t\\\n \tkunmap_local((pte));\t\\\n \trcu_read_unlock();\t\\\n } while (0)\n #else\n-static inline pte_t *__pte_map(pmd_t *pmd, unsigned long address)\n+static inline hw_pte_t *__pte_map(pmd_t *pmd, unsigned long address)\n {\n \treturn pte_offset_kernel(pmd, address);\n }\n-static inline void pte_unmap(pte_t *pte)\n+static inline void pte_unmap(hw_pte_t *pte)\n {\n \trcu_read_unlock();\n }\n@@ -172,7 +173,7 @@ static inline pmd_t *pmd_off_k(unsigned long va)\n \treturn pmd_offset(pud_offset(p4d_offset(pgd_offset_k(va), va), va), va);\n }\n \n-static inline pte_t *virt_to_kpte(unsigned long vaddr)\n+static inline hw_pte_t *virt_to_kpte(unsigned long vaddr)\n {\n \tpmd_t *pmd = pmd_off_k(vaddr);\n \n@@ -407,7 +408,7 @@ static inline void lazy_mmu_mode_resume(void) {}\n *\n * May be overridden by the architecture, else pte_batch_hint is always 1.\n */\n-static inline unsigned int pte_batch_hint(pte_t *ptep, pte_t pte)\n+static inline unsigned int pte_batch_hint(hw_pte_t *ptep, pte_t pte)\n {\n \treturn 1;\n }\n@@ -442,7 +443,7 @@ static inline pte_t pte_advance_pfn(pte_t pte, unsigned long nr)\n * to the same folio. The PTEs are all in the same PMD.\n */\n static inline void set_ptes(struct mm_struct *mm, unsigned long addr,\n-\t\tpte_t *ptep, pte_t pte, unsigned int nr)\n+\t\thw_pte_t *ptep, pte_t pte, unsigned int nr)\n {\n \tpage_table_check_ptes_set(mm, addr, ptep, pte, nr);\n \n@@ -459,7 +460,7 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr,\n \n #ifndef __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS\n extern int ptep_set_access_flags(struct vm_area_struct *vma,\n-\t\t\t\t unsigned long address, pte_t *ptep,\n+\t\t\t\t unsigned long address, hw_pte_t *ptep,\n \t\t\t\t pte_t entry, int dirty);\n #endif\n \n@@ -490,9 +491,9 @@ static inline int pudp_set_access_flags(struct vm_area_struct *vma,\n #endif\n \n #ifndef ptep_get\n-static inline pte_t ptep_get(pte_t *ptep)\n+static inline pte_t ptep_get(hw_pte_t *ptep)\n {\n-\treturn READ_ONCE(*ptep);\n+\treturn __pte_from_hw(READ_ONCE(*ptep));\n }\n #endif\n \n@@ -526,7 +527,7 @@ static inline pgd_t pgdp_get(pgd_t *pgdp)\n \n #ifndef __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG\n static inline bool ptep_test_and_clear_young(struct vm_area_struct *vma,\n-\t\tunsigned long address, pte_t *ptep)\n+\t\tunsigned long address, hw_pte_t *ptep)\n {\n \tpte_t pte = ptep_get(ptep);\n \tbool young = true;\n@@ -565,7 +566,7 @@ static inline bool pmdp_test_and_clear_young(struct vm_area_struct *vma,\n \n #ifndef __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH\n bool ptep_clear_flush_young(struct vm_area_struct *vma,\n-\t\tunsigned long address, pte_t *ptep);\n+\t\tunsigned long address, hw_pte_t *ptep);\n #endif\n \n #ifndef __HAVE_ARCH_PMDP_CLEAR_YOUNG_FLUSH\n@@ -644,7 +645,7 @@ static inline void arch_check_zapped_pud(struct vm_area_struct *vma, pud_t pud)\n #ifndef __HAVE_ARCH_PTEP_GET_AND_CLEAR\n static inline pte_t ptep_get_and_clear(struct mm_struct *mm,\n \t\t\t\t unsigned long address,\n-\t\t\t\t pte_t *ptep)\n+\t\t\t\t hw_pte_t *ptep)\n {\n \tpte_t pte = ptep_get(ptep);\n \tpte_clear(mm, address, ptep);\n@@ -673,7 +674,7 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm,\n * pages that belong to the same folio. The PTEs are all in the same PMD.\n */\n static inline void clear_young_dirty_ptes(struct vm_area_struct *vma,\n-\t\t\t\t\t unsigned long addr, pte_t *ptep,\n+\t\t\t\t\t unsigned long addr, hw_pte_t *ptep,\n \t\t\t\t\t unsigned int nr, cydp_t flags)\n {\n \tpte_t pte;\n@@ -698,7 +699,7 @@ static inline void clear_young_dirty_ptes(struct vm_area_struct *vma,\n #endif\n \n static inline void ptep_clear(struct mm_struct *mm, unsigned long addr,\n-\t\t\t pte_t *ptep)\n+\t\t\t hw_pte_t *ptep)\n {\n \tpte_t pte = ptep_get(ptep);\n \n@@ -739,7 +740,7 @@ static inline void ptep_clear(struct mm_struct *mm, unsigned long addr,\n * present bit set *unless* it is 'l'. Because get_user_pages_fast() only\n * operates on present ptes we're safe.\n */\n-static inline pte_t ptep_get_lockless(pte_t *ptep)\n+static inline pte_t ptep_get_lockless(hw_pte_t *ptep)\n {\n \tpte_t pte;\n \n@@ -777,7 +778,7 @@ static inline pmd_t pmdp_get_lockless(pmd_t *pmdp)\n * We require that the PTE can be read atomically.\n */\n #ifndef ptep_get_lockless\n-static inline pte_t ptep_get_lockless(pte_t *ptep)\n+static inline pte_t ptep_get_lockless(hw_pte_t *ptep)\n {\n \treturn ptep_get(ptep);\n }\n@@ -844,7 +845,8 @@ static inline pud_t pudp_huge_get_and_clear_full(struct vm_area_struct *vma,\n \n #ifndef __HAVE_ARCH_PTEP_GET_AND_CLEAR_FULL\n static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,\n-\t\t\t\t\t unsigned long address, pte_t *ptep,\n+\t\t\t\t\t unsigned long address,\n+\t\t\t\t\t hw_pte_t *ptep,\n \t\t\t\t\t int full)\n {\n \treturn ptep_get_and_clear(mm, address, ptep);\n@@ -872,7 +874,7 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,\n * pages that belong to the same folio. The PTEs are all in the same PMD.\n */\n static inline pte_t get_and_clear_full_ptes(struct mm_struct *mm,\n-\t\tunsigned long addr, pte_t *ptep, unsigned int nr, int full)\n+\t\tunsigned long addr, hw_pte_t *ptep, unsigned int nr, int full)\n {\n \tpte_t pte, tmp_pte;\n \n@@ -908,7 +910,7 @@ static inline pte_t get_and_clear_full_ptes(struct mm_struct *mm,\n * pages that belong to the same folio. The PTEs are all in the same PMD.\n */\n static inline pte_t get_and_clear_ptes(struct mm_struct *mm, unsigned long addr,\n-\t\tpte_t *ptep, unsigned int nr)\n+\t\thw_pte_t *ptep, unsigned int nr)\n {\n \treturn get_and_clear_full_ptes(mm, addr, ptep, nr, 0);\n }\n@@ -933,7 +935,7 @@ static inline pte_t get_and_clear_ptes(struct mm_struct *mm, unsigned long addr,\n * pages that belong to the same folio. The PTEs are all in the same PMD.\n */\n static inline void clear_full_ptes(struct mm_struct *mm, unsigned long addr,\n-\t\tpte_t *ptep, unsigned int nr, int full)\n+\t\thw_pte_t *ptep, unsigned int nr, int full)\n {\n \tfor (;;) {\n \t\tptep_get_and_clear_full(mm, addr, ptep, full);\n@@ -962,7 +964,7 @@ static inline void clear_full_ptes(struct mm_struct *mm, unsigned long addr,\n * pages that belong to the same folio. The PTEs are all in the same PMD.\n */\n static inline void clear_ptes(struct mm_struct *mm, unsigned long addr,\n-\t\tpte_t *ptep, unsigned int nr)\n+\t\thw_pte_t *ptep, unsigned int nr)\n {\n \tclear_full_ptes(mm, addr, ptep, nr, 0);\n }\n@@ -977,13 +979,14 @@ static inline void clear_ptes(struct mm_struct *mm, unsigned long addr,\n */\n #ifndef update_mmu_tlb_range\n static inline void update_mmu_tlb_range(struct vm_area_struct *vma,\n-\t\t\t\tunsigned long address, pte_t *ptep, unsigned int nr)\n+\t\t\t\tunsigned long address, hw_pte_t *ptep,\n+\t\t\t\tunsigned int nr)\n {\n }\n #endif\n \n static inline void update_mmu_tlb(struct vm_area_struct *vma,\n-\t\t\t\tunsigned long address, pte_t *ptep)\n+\t\t\t\tunsigned long address, hw_pte_t *ptep)\n {\n \tupdate_mmu_tlb_range(vma, address, ptep, 1);\n }\n@@ -1000,7 +1003,7 @@ static inline void update_mmu_tlb(struct vm_area_struct *vma,\n * The PTEs are all in the same PMD.\n */\n static inline void clear_nonpresent_ptes(struct mm_struct *mm,\n-\t\tunsigned long addr, pte_t *ptep, unsigned int nr)\n+\t\tunsigned long addr, hw_pte_t *ptep, unsigned int nr)\n {\n \t(void)addr;\n \n@@ -1016,7 +1019,7 @@ static inline void clear_nonpresent_ptes(struct mm_struct *mm,\n #ifndef __HAVE_ARCH_PTEP_CLEAR_FLUSH\n extern pte_t ptep_clear_flush(struct vm_area_struct *vma,\n \t\t\t unsigned long address,\n-\t\t\t pte_t *ptep);\n+\t\t\t hw_pte_t *ptep);\n #endif\n \n #ifndef __HAVE_ARCH_PMDP_HUGE_CLEAR_FLUSH\n@@ -1044,7 +1047,8 @@ static inline pmd_t pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma)\n \n #ifndef __HAVE_ARCH_PTEP_SET_WRPROTECT\n struct mm_struct;\n-static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long address, pte_t *ptep)\n+static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long address,\n+\t\t\t\t hw_pte_t *ptep)\n {\n \tpte_t old_pte = ptep_get(ptep);\n \tset_pte_at(mm, address, ptep, pte_wrprotect(old_pte));\n@@ -1070,7 +1074,7 @@ static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long addres\n * ptep_try_set as an identity macro. The generic stub returns false, which is\n * correct for callers that fall through to oops on failure.\n */\n-static inline bool ptep_try_set(pte_t *ptep, pte_t new_pte)\n+static inline bool ptep_try_set(hw_pte_t *ptep, pte_t new_pte)\n {\n \treturn false;\n }\n@@ -1113,7 +1117,7 @@ static inline void flush_tlb_before_set(unsigned long addr)\n * pages that belong to the same folio. The PTEs are all in the same PMD.\n */\n static inline void wrprotect_ptes(struct mm_struct *mm, unsigned long addr,\n-\t\tpte_t *ptep, unsigned int nr)\n+\t\thw_pte_t *ptep, unsigned int nr)\n {\n \tfor (;;) {\n \t\tptep_set_wrprotect(mm, addr, ptep);\n@@ -1144,7 +1148,7 @@ static inline void wrprotect_ptes(struct mm_struct *mm, unsigned long addr,\n * pages that belong to the same folio. The PTEs are all in the same PMD.\n */\n static inline bool clear_flush_young_ptes(struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep, unsigned int nr)\n+\t\tunsigned long addr, hw_pte_t *ptep, unsigned int nr)\n {\n \tbool young = false;\n \n@@ -1181,7 +1185,7 @@ static inline bool clear_flush_young_ptes(struct vm_area_struct *vma,\n * Returns: whether any PTE was young.\n */\n static inline bool test_and_clear_young_ptes(struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep, unsigned int nr)\n+\t\tunsigned long addr, hw_pte_t *ptep, unsigned int nr)\n {\n \tbool young = false;\n \n@@ -1585,7 +1589,7 @@ static inline int pmd_none_or_clear_bad(pmd_t *pmd)\n \n static inline pte_t __ptep_modify_prot_start(struct vm_area_struct *vma,\n \t\t\t\t\t unsigned long addr,\n-\t\t\t\t\t pte_t *ptep)\n+\t\t\t\t\t hw_pte_t *ptep)\n {\n \t/*\n \t * Get the current pte state, but zero it out to make it\n@@ -1597,7 +1601,7 @@ static inline pte_t __ptep_modify_prot_start(struct vm_area_struct *vma,\n \n static inline void __ptep_modify_prot_commit(struct vm_area_struct *vma,\n \t\t\t\t\t unsigned long addr,\n-\t\t\t\t\t pte_t *ptep, pte_t pte)\n+\t\t\t\t\t hw_pte_t *ptep, pte_t pte)\n {\n \t/*\n \t * The pte is non-present, so there's no hardware state to\n@@ -1623,7 +1627,7 @@ static inline void __ptep_modify_prot_commit(struct vm_area_struct *vma,\n */\n static inline pte_t ptep_modify_prot_start(struct vm_area_struct *vma,\n \t\t\t\t\t unsigned long addr,\n-\t\t\t\t\t pte_t *ptep)\n+\t\t\t\t\t hw_pte_t *ptep)\n {\n \treturn __ptep_modify_prot_start(vma, addr, ptep);\n }\n@@ -1636,7 +1640,8 @@ static inline pte_t ptep_modify_prot_start(struct vm_area_struct *vma,\n */\n static inline void ptep_modify_prot_commit(struct vm_area_struct *vma,\n \t\t\t\t\t unsigned long addr,\n-\t\t\t\t\t pte_t *ptep, pte_t old_pte, pte_t pte)\n+\t\t\t\t\t hw_pte_t *ptep, pte_t old_pte,\n+\t\t\t\t\t pte_t pte)\n {\n \t__ptep_modify_prot_commit(vma, addr, ptep, pte);\n }\n@@ -1667,7 +1672,7 @@ static inline void ptep_modify_prot_commit(struct vm_area_struct *vma,\n */\n #ifndef modify_prot_start_ptes\n static inline pte_t modify_prot_start_ptes(struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep, unsigned int nr)\n+\t\tunsigned long addr, hw_pte_t *ptep, unsigned int nr)\n {\n \tpte_t pte, tmp_pte;\n \n@@ -1707,7 +1712,7 @@ static inline pte_t modify_prot_start_ptes(struct vm_area_struct *vma,\n */\n #ifndef modify_prot_commit_ptes\n static inline void modify_prot_commit_ptes(struct vm_area_struct *vma, unsigned long addr,\n-\t\tpte_t *ptep, pte_t old_pte, pte_t pte, unsigned int nr)\n+\t\thw_pte_t *ptep, pte_t old_pte, pte_t pte, unsigned int nr)\n {\n \tint i;\n \ndiff --git a/include/linux/pgtable_types.h b/include/linux/pgtable_types.h\nnew file mode 100644\nindex 0000000000000..d6c5a7548550b\n--- /dev/null\n+++ b/include/linux/pgtable_types.h\n@@ -0,0 +1,19 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+#ifndef _LINUX_PGTABLE_TYPES_H\n+#define _LINUX_PGTABLE_TYPES_H\n+\n+#include \u003casm/page.h\u003e\n+\n+#ifndef __ASSEMBLY__\n+\n+#ifdef CONFIG_ARCH_HAS_HW_PTE_T\n+typedef struct __hw_pte_t { pte_t __pte; } hw_pte_t;\n+#define __pte_from_hw(pte)\t((pte).__pte)\n+#else\n+#define hw_pte_t pte_t\n+#define __pte_from_hw(pte)\t(pte)\n+#endif\n+\n+#endif /* !__ASSEMBLY__ */\n+\n+#endif /* _LINUX_PGTABLE_TYPES_H */\ndiff --git a/include/linux/rmap.h b/include/linux/rmap.h\nindex 0b332770abeed..d28a7fb6f2c0e 100644\n--- a/include/linux/rmap.h\n+++ b/include/linux/rmap.h\n@@ -868,7 +868,7 @@ struct page_vma_mapped_walk {\n \tstruct vm_area_struct *vma;\n \tunsigned long address;\n \tpmd_t *pmd;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tspinlock_t *ptl;\n \tunsigned int flags;\n \tbool pgoff_is_anon : 1;\ndiff --git a/include/linux/swapops.h b/include/linux/swapops.h\nindex e7d0d529f3e0f..40d2fd920558f 100644\n--- a/include/linux/swapops.h\n+++ b/include/linux/swapops.h\n@@ -216,7 +216,8 @@ static inline swp_entry_t make_migration_entry_dirty(swp_entry_t entry)\n \n extern void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd,\n \t\t\t\t\tunsigned long address);\n-extern void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr, pte_t *pte);\n+extern void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr,\n+\t\t\t\t hw_pte_t *pte);\n #else /* CONFIG_MIGRATION */\n static inline swp_entry_t make_readable_migration_entry(pgoff_t offset)\n {\n@@ -236,7 +237,8 @@ static inline swp_entry_t make_writable_migration_entry(pgoff_t offset)\n static inline void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd,\n \t\t\t\t\tunsigned long address) { }\n static inline void migration_entry_wait_huge(struct vm_area_struct *vma,\n-\t\t\t\t\t unsigned long addr, pte_t *pte) { }\n+\t\t\t\t\t unsigned long addr,\n+\t\t\t\t\t hw_pte_t *pte) { }\n \n static inline swp_entry_t make_migration_entry_young(swp_entry_t entry)\n {\ndiff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h\nindex aed121d729b01..666ff2b3741da 100644\n--- a/include/linux/vmalloc.h\n+++ b/include/linux/vmalloc.h\n@@ -8,7 +8,7 @@\n #include \u003clinux/init.h\u003e\n #include \u003clinux/list.h\u003e\n #include \u003clinux/llist.h\u003e\n-#include \u003casm/page.h\u003e\t\t/* pgprot_t */\n+#include \u003clinux/pgtable_types.h\u003e\t/* pgprot_t, hw_pte_t */\n #include \u003clinux/rbtree.h\u003e\n #include \u003clinux/overflow.h\u003e\n \n@@ -120,7 +120,7 @@ static inline unsigned long arch_vmap_pte_range_map_size(unsigned long addr, uns\n \n #ifndef arch_vmap_pte_range_unmap_size\n static inline unsigned long arch_vmap_pte_range_unmap_size(unsigned long addr,\n-\t\t\t\t\t\t\t pte_t *ptep)\n+\t\t\t\t\t\t\t hw_pte_t *ptep)\n {\n \treturn PAGE_SIZE;\n }\ndiff --git a/include/trace/events/xen.h b/include/trace/events/xen.h\nindex ad384969e2cb2..1972d50b043a5 100644\n--- a/include/trace/events/xen.h\n+++ b/include/trace/events/xen.h\n@@ -138,10 +138,10 @@ TRACE_EVENT(xen_mc_extend_args,\n TRACE_DEFINE_SIZEOF(pteval_t);\n \n TRACE_EVENT(xen_mmu_set_pte,\n-\t TP_PROTO(pte_t *ptep, pte_t pteval),\n+\t TP_PROTO(hw_pte_t *ptep, pte_t pteval),\n \t TP_ARGS(ptep, pteval),\n \t TP_STRUCT__entry(\n-\t\t __field(pte_t *, ptep)\n+\t\t __field(hw_pte_t *, ptep)\n \t\t __field(pteval_t, pteval)\n \t\t ),\n \t TP_fast_assign(__entry-\u003eptep = ptep;\n@@ -207,12 +207,12 @@ TRACE_EVENT(xen_mmu_set_p4d,\n \n DECLARE_EVENT_CLASS(xen_mmu_ptep_modify_prot,\n \t TP_PROTO(struct mm_struct *mm, unsigned long addr,\n-\t\t pte_t *ptep, pte_t pteval),\n+\t\t hw_pte_t *ptep, pte_t pteval),\n \t TP_ARGS(mm, addr, ptep, pteval),\n \t TP_STRUCT__entry(\n \t\t __field(struct mm_struct *, mm)\n \t\t __field(unsigned long, addr)\n-\t\t __field(pte_t *, ptep)\n+\t\t __field(hw_pte_t *, ptep)\n \t\t __field(pteval_t, pteval)\n \t\t ),\n \t TP_fast_assign(__entry-\u003emm = mm;\n@@ -227,7 +227,7 @@ DECLARE_EVENT_CLASS(xen_mmu_ptep_modify_prot,\n #define DEFINE_XEN_MMU_PTEP_MODIFY_PROT(name)\t\t\t\t\\\n \tDEFINE_EVENT(xen_mmu_ptep_modify_prot, name,\t\t\t\\\n \t\t TP_PROTO(struct mm_struct *mm, unsigned long addr,\t\\\n-\t\t\t pte_t *ptep, pte_t pteval),\t\t\\\n+\t\t\t hw_pte_t *ptep, pte_t pteval),\t\t\\\n \t\t TP_ARGS(mm, addr, ptep, pteval))\n \n DEFINE_XEN_MMU_PTEP_MODIFY_PROT(xen_mmu_ptep_modify_prot_start);\ndiff --git a/kernel/bpf/arena.c b/kernel/bpf/arena.c\nindex 7b6847200b431..62005229b2e94 100644\n--- a/kernel/bpf/arena.c\n+++ b/kernel/bpf/arena.c\n@@ -155,7 +155,7 @@ struct clear_range_data {\n \tstruct llist_head *free_pages;\n };\n \n-static int apply_range_set_cb(pte_t *pte, unsigned long addr, void *data)\n+static int apply_range_set_cb(hw_pte_t *pte, unsigned long addr, void *data)\n {\n \tstruct apply_range_data *d = data;\n \tstruct page *page;\n@@ -207,7 +207,7 @@ static void flush_vmap_cache(unsigned long start, unsigned long size)\n \tflush_cache_vmap(start, start + size);\n }\n \n-static int apply_range_clear_cb(pte_t *pte, unsigned long addr, void *data)\n+static int apply_range_clear_cb(hw_pte_t *pte, unsigned long addr, void *data)\n {\n \tstruct clear_range_data *d = data;\n \tpte_t old_pte;\n@@ -238,7 +238,8 @@ static int apply_range_clear_cb(pte_t *pte, unsigned long addr, void *data)\n \treturn 0;\n }\n \n-static int apply_range_set_scratch_cb(pte_t *pte, unsigned long addr, void *data)\n+static int apply_range_set_scratch_cb(hw_pte_t *pte, unsigned long addr,\n+\t\t\t\t void *data)\n {\n \tstruct page *scratch_page = data;\n \n@@ -340,7 +341,7 @@ static struct bpf_map *arena_map_alloc(union bpf_attr *attr)\n \treturn ERR_PTR(err);\n }\n \n-static int existing_page_cb(pte_t *ptep, unsigned long addr, void *data)\n+static int existing_page_cb(hw_pte_t *ptep, unsigned long addr, void *data)\n {\n \tstruct bpf_arena *arena = data;\n \tstruct page *page;\ndiff --git a/kernel/events/core.c b/kernel/events/core.c\nindex a6c8e38a31104..0369f6c95ba7d 100644\n--- a/kernel/events/core.c\n+++ b/kernel/events/core.c\n@@ -8517,7 +8517,8 @@ static u64 perf_get_pgtable_size(struct mm_struct *mm, unsigned long addr)\n \tp4d_t *p4dp, p4d;\n \tpud_t *pudp, pud;\n \tpmd_t *pmdp, pmd;\n-\tpte_t *ptep, pte;\n+\thw_pte_t *ptep;\n+\tpte_t pte;\n \n \tpgdp = pgd_offset(mm, addr);\n \tpgd = pgdp_get(pgdp);\ndiff --git a/mm/Kconfig b/mm/Kconfig\nindex c1ddf59c0d71a..5f462ec6fa5e5 100644\n--- a/mm/Kconfig\n+++ b/mm/Kconfig\n@@ -1312,6 +1312,9 @@ comment \"GUP_TEST needs to have DEBUG_FS enabled\"\n config GUP_GET_PXX_LOW_HIGH\n \tbool\n \n+config ARCH_HAS_HW_PTE_T\n+\tbool\n+\n config DMAPOOL_TEST\n \ttristate \"Enable a module to run time tests on dma_pool\"\n \tdepends on HAS_DMA\ndiff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c\nindex 7a2e40bc7baed..f844a44752a81 100644\n--- a/mm/damon/ops-common.c\n+++ b/mm/damon/ops-common.c\n@@ -39,7 +39,7 @@ struct folio *damon_get_folio(unsigned long pfn)\n \treturn folio;\n }\n \n-void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr)\n+void damon_ptep_mkold(hw_pte_t *pte, struct vm_area_struct *vma, unsigned long addr)\n {\n \tpte_t pteval = ptep_get(pte);\n \tstruct folio *folio;\ndiff --git a/mm/damon/ops-common.h b/mm/damon/ops-common.h\nindex 38d295488fa18..34b7e5715fcf9 100644\n--- a/mm/damon/ops-common.h\n+++ b/mm/damon/ops-common.h\n@@ -7,7 +7,7 @@\n \n struct folio *damon_get_folio(unsigned long pfn);\n \n-void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr);\n+void damon_ptep_mkold(hw_pte_t *pte, struct vm_area_struct *vma, unsigned long addr);\n void damon_pmdp_mkold(pmd_t *pmd, struct vm_area_struct *vma, unsigned long addr);\n void damon_folio_mkold(struct folio *folio);\n bool damon_folio_young(struct folio *folio);\ndiff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c\nindex f884d3f78f30a..482e905c973a6 100644\n--- a/mm/damon/vaddr.c\n+++ b/mm/damon/vaddr.c\n@@ -258,7 +258,7 @@ static void damon_va_walk_page_range(struct mm_struct *mm, unsigned long start,\n static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned long addr,\n \t\tunsigned long next, struct mm_walk *walk)\n {\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tspinlock_t *ptl;\n \n \tptl = pmd_trans_huge_lock(pmd, walk-\u003evma);\n@@ -283,7 +283,7 @@ static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned long addr,\n }\n \n #ifdef CONFIG_HUGETLB_PAGE\n-static void damon_hugetlb_mkold(pte_t *pte, struct mm_struct *mm,\n+static void damon_hugetlb_mkold(hw_pte_t *pte, struct mm_struct *mm,\n \t\t\t\tstruct vm_area_struct *vma, unsigned long addr)\n {\n \tbool referenced = false;\n@@ -310,7 +310,7 @@ static void damon_hugetlb_mkold(pte_t *pte, struct mm_struct *mm,\n \tfolio_put(folio);\n }\n \n-static int damon_mkold_hugetlb_entry(pte_t *pte, unsigned long hmask,\n+static int damon_mkold_hugetlb_entry(hw_pte_t *pte, unsigned long hmask,\n \t\t\t\t unsigned long addr, unsigned long end,\n \t\t\t\t struct mm_walk *walk)\n {\n@@ -379,7 +379,7 @@ struct damon_young_walk_private {\n static int damon_young_pmd_entry(pmd_t *pmd, unsigned long addr,\n \t\tunsigned long next, struct mm_walk *walk)\n {\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tpte_t ptent;\n \tspinlock_t *ptl;\n \tstruct folio *folio;\n@@ -423,7 +423,7 @@ static int damon_young_pmd_entry(pmd_t *pmd, unsigned long addr,\n }\n \n #ifdef CONFIG_HUGETLB_PAGE\n-static int damon_young_hugetlb_entry(pte_t *pte, unsigned long hmask,\n+static int damon_young_hugetlb_entry(hw_pte_t *pte, unsigned long hmask,\n \t\t\t\t unsigned long addr, unsigned long end,\n \t\t\t\t struct mm_walk *walk)\n {\n@@ -512,7 +512,7 @@ static unsigned int damon_va_check_accesses(struct damon_ctx *ctx)\n \n static bool damos_va_filter_young_match(struct damos_filter *filter,\n \t\tstruct folio *folio, struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep, pmd_t *pmdp)\n+\t\tunsigned long addr, hw_pte_t *ptep, pmd_t *pmdp)\n {\n \tbool young = false;\n \n@@ -534,7 +534,7 @@ static bool damos_va_filter_young_match(struct damos_filter *filter,\n \n static bool damos_va_filter_out(struct damos *scheme, struct folio *folio,\n \t\tstruct vm_area_struct *vma, unsigned long addr,\n-\t\tpte_t *ptep, pmd_t *pmdp)\n+\t\thw_pte_t *ptep, pmd_t *pmdp)\n {\n \tstruct damos_filter *filter;\n \tbool matched;\n@@ -631,7 +631,8 @@ static int damos_va_migrate_pmd_entry(pmd_t *pmd, unsigned long addr,\n \tstruct damos_migrate_dests *dests = \u0026s-\u003emigrate_dests;\n \tstruct folio *folio;\n \tspinlock_t *ptl;\n-\tpte_t *start_pte, *pte, ptent;\n+\thw_pte_t *start_pte, *pte;\n+\tpte_t ptent;\n \tint nr;\n \n #ifdef CONFIG_TRANSPARENT_HUGEPAGE\n@@ -794,7 +795,8 @@ static int damos_va_stat_pmd_entry(pmd_t *pmd, unsigned long addr,\n \tstruct vm_area_struct *vma = walk-\u003evma;\n \tstruct folio *folio;\n \tspinlock_t *ptl;\n-\tpte_t *start_pte, *pte, ptent;\n+\thw_pte_t *start_pte, *pte;\n+\tpte_t ptent;\n \tint nr;\n \n #ifdef CONFIG_TRANSPARENT_HUGEPAGE\ndiff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c\nindex 2875fd22d7bb0..54e194d2a5854 100644\n--- a/mm/debug_vm_pgtable.c\n+++ b/mm/debug_vm_pgtable.c\n@@ -50,7 +50,7 @@ struct pgtable_debug_args {\n \tp4d_t\t\t\t*p4dp;\n \tpud_t\t\t\t*pudp;\n \tpmd_t\t\t\t*pmdp;\n-\tpte_t\t\t\t*ptep;\n+\thw_pte_t\t\t*ptep;\n \n \tp4d_t\t\t\t*start_p4dp;\n \tpud_t\t\t\t*start_pudp;\ndiff --git a/mm/filemap.c b/mm/filemap.c\nindex 00fd89cf6f550..98f89dcc55603 100644\n--- a/mm/filemap.c\n+++ b/mm/filemap.c\n@@ -3490,7 +3490,7 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf)\n {\n \tstruct vm_area_struct *vma = vmf-\u003evma;\n \tvm_fault_t ret = 0;\n-\tpte_t *ptep;\n+\thw_pte_t *ptep;\n \n \t/*\n \t * We might have COW'ed a pagecache folio and might now have an mlocked\n@@ -3796,7 +3796,7 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf,\n \tvm_fault_t ret = 0;\n \tstruct page *page = folio_page(folio, start);\n \tunsigned int count = 0;\n-\tpte_t *old_ptep = vmf-\u003epte;\n+\thw_pte_t *old_ptep = vmf-\u003epte;\n \tunsigned long addr0;\n \n \t/*\ndiff --git a/mm/gup.c b/mm/gup.c\nindex d1ba59da94e9a..549f462a23b2f 100644\n--- a/mm/gup.c\n+++ b/mm/gup.c\n@@ -761,7 +761,7 @@ static struct page *follow_huge_pmd(struct vm_area_struct *vma,\n #endif\t/* CONFIG_PGTABLE_HAS_HUGE_LEAVES */\n \n static int follow_pfn_pte(struct vm_area_struct *vma, unsigned long address,\n-\t\tpte_t *pte, unsigned int flags)\n+\t\thw_pte_t *pte, unsigned int flags)\n {\n \tif (flags \u0026 FOLL_TOUCH) {\n \t\tpte_t orig_entry = ptep_get(pte);\n@@ -806,7 +806,8 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,\n \tstruct folio *folio;\n \tstruct page *page;\n \tspinlock_t *ptl;\n-\tpte_t *ptep, pte;\n+\thw_pte_t *ptep;\n+\tpte_t pte;\n \tint ret;\n \n \tptep = pte_offset_map_lock(mm, pmd, address, \u0026ptl);\n@@ -1035,7 +1036,7 @@ static int get_gate_page(struct mm_struct *mm, unsigned long address,\n \tp4d_t *p4d;\n \tpud_t *pud;\n \tpmd_t *pmd;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tpte_t entry;\n \tint ret = -EFAULT;\n \n@@ -2838,7 +2839,7 @@ static unsigned long gup_fast_pte_range(pmd_t pmd, pmd_t *pmdp,\n \t\tunsigned int flags, struct page **pages)\n {\n \tunsigned long nr_pages = 0;\n-\tpte_t *ptep, *ptem;\n+\thw_pte_t *ptep, *ptem;\n \n \tptem = ptep = pte_offset_map(\u0026pmd, addr);\n \tif (!ptep)\ndiff --git a/mm/highmem.c b/mm/highmem.c\nindex a33e411839517..87f94cac6106b 100644\n--- a/mm/highmem.c\n+++ b/mm/highmem.c\n@@ -141,7 +141,7 @@ EXPORT_SYMBOL(__totalhigh_pages);\n static int pkmap_count[LAST_PKMAP];\n static __cacheline_aligned_in_smp DEFINE_SPINLOCK(kmap_lock);\n \n-pte_t *pkmap_page_table;\n+hw_pte_t *pkmap_page_table;\n \n /*\n * Most architectures have no use for kmap_high_get(), so let's abstract\n@@ -532,9 +532,9 @@ static inline bool kmap_high_unmap_local(unsigned long vaddr)\n \treturn false;\n }\n \n-static pte_t *__kmap_pte;\n+static hw_pte_t *__kmap_pte;\n \n-static pte_t *kmap_get_pte(unsigned long vaddr, int idx)\n+static hw_pte_t *kmap_get_pte(unsigned long vaddr, int idx)\n {\n \tif (IS_ENABLED(CONFIG_KMAP_LOCAL_NON_LINEAR_PTE_ARRAY))\n \t\t/*\n@@ -549,8 +549,9 @@ static pte_t *kmap_get_pte(unsigned long vaddr, int idx)\n \n void *__kmap_local_pfn_prot(unsigned long pfn, pgprot_t prot)\n {\n-\tpte_t pteval, *kmap_pte;\n \tunsigned long vaddr;\n+\thw_pte_t *kmap_pte;\n+\tpte_t pteval;\n \tint idx;\n \n \t/*\n@@ -597,7 +598,7 @@ EXPORT_SYMBOL(__kmap_local_page_prot);\n void kunmap_local_indexed(const void *vaddr)\n {\n \tunsigned long addr = (unsigned long) vaddr \u0026 PAGE_MASK;\n-\tpte_t *kmap_pte;\n+\thw_pte_t *kmap_pte;\n \tint idx;\n \n \tif (addr \u003c __fix_to_virt(FIX_KMAP_END) ||\n@@ -646,7 +647,7 @@ EXPORT_SYMBOL(kunmap_local_indexed);\n void __kmap_local_sched_out(void)\n {\n \tstruct task_struct *tsk = current;\n-\tpte_t *kmap_pte;\n+\thw_pte_t *kmap_pte;\n \tint i;\n \n \t/* Clear kmaps */\n@@ -683,7 +684,7 @@ void __kmap_local_sched_out(void)\n void __kmap_local_sched_in(void)\n {\n \tstruct task_struct *tsk = current;\n-\tpte_t *kmap_pte;\n+\thw_pte_t *kmap_pte;\n \tint i;\n \n \t/* Restore kmaps */\ndiff --git a/mm/hmm.c b/mm/hmm.c\nindex 2f1e98c6b6440..0c959611eda01 100644\n--- a/mm/hmm.c\n+++ b/mm/hmm.c\n@@ -240,7 +240,7 @@ static inline unsigned long pte_to_hmm_pfn_flags(struct hmm_range *range,\n }\n \n static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr,\n-\t\t\t unsigned long end, pmd_t *pmdp, pte_t *ptep,\n+\t\t\t unsigned long end, pmd_t *pmdp, hw_pte_t *ptep,\n \t\t\t unsigned long *hmm_pfn)\n {\n \tstruct hmm_vma_walk *hmm_vma_walk = walk-\u003eprivate;\n@@ -411,7 +411,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp,\n \t\t\u0026range-\u003ehmm_pfns[(start - range-\u003estart) \u003e\u003e PAGE_SHIFT];\n \tunsigned long npages = (end - start) \u003e\u003e PAGE_SHIFT;\n \tunsigned long addr = start;\n-\tpte_t *ptep;\n+\thw_pte_t *ptep;\n \tpmd_t pmd;\n \n again:\n@@ -547,7 +547,7 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end,\n #endif\n \n #ifdef CONFIG_HUGETLB_PAGE\n-static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask,\n+static int hmm_vma_walk_hugetlb_entry(hw_pte_t *pte, unsigned long hmask,\n \t\t\t\t unsigned long start, unsigned long end,\n \t\t\t\t struct mm_walk *walk)\n {\ndiff --git a/mm/huge_memory.c b/mm/huge_memory.c\nindex dd66c6ad5af13..629c773ed313a 100644\n--- a/mm/huge_memory.c\n+++ b/mm/huge_memory.c\n@@ -3149,7 +3149,7 @@ static void __split_huge_zero_page_pmd(struct vm_area_struct *vma,\n \tpgtable_t pgtable;\n \tpmd_t _pmd, old_pmd;\n \tunsigned long addr;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tint i;\n \n \t/*\n@@ -3199,7 +3199,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,\n \tbool soft_dirty, uffd_wp = false, young = false, write = false;\n \tbool anon_exclusive = false, dirty = false;\n \tunsigned long addr;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tint i;\n \n \tVM_BUG_ON(haddr \u0026 ~HPAGE_PMD_MASK);\ndiff --git a/mm/hugetlb.c b/mm/hugetlb.c\nindex ceb29adc6dc19..687d87801979d 100644\n--- a/mm/hugetlb.c\n+++ b/mm/hugetlb.c\n@@ -129,7 +129,7 @@ static void hugetlb_vma_lock_free(struct vm_area_struct *vma);\n static void hugetlb_vma_lock_alloc(struct vm_area_struct *vma);\n static void __hugetlb_vma_unlock_write_free(struct vm_area_struct *vma);\n static int __huge_pmd_unshare(struct mmu_gather *tlb,\n-\t\tstruct vm_area_struct *vma, unsigned long addr, pte_t *ptep,\n+\t\tstruct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,\n \t\tbool check_locks);\n static void hugetlb_unshare_pmds(struct vm_area_struct *vma,\n \t\tunsigned long start, unsigned long end, bool take_locks);\n@@ -4891,7 +4891,7 @@ static pte_t make_huge_pte(struct vm_area_struct *vma, struct folio *folio,\n }\n \n static void set_huge_ptep_writable(struct vm_area_struct *vma,\n-\t\t\t\t unsigned long address, pte_t *ptep)\n+\t\t\t\t unsigned long address, hw_pte_t *ptep)\n {\n \tpte_t entry;\n \n@@ -4901,14 +4901,14 @@ static void set_huge_ptep_writable(struct vm_area_struct *vma,\n }\n \n static void set_huge_ptep_maybe_writable(struct vm_area_struct *vma,\n-\t\t\t\t\t unsigned long address, pte_t *ptep)\n+\t\t\t\t\t unsigned long address, hw_pte_t *ptep)\n {\n \tif (vma-\u003evm_flags \u0026 VM_WRITE)\n \t\tset_huge_ptep_writable(vma, address, ptep);\n }\n \n static void\n-hugetlb_install_folio(struct vm_area_struct *vma, pte_t *ptep, unsigned long addr,\n+hugetlb_install_folio(struct vm_area_struct *vma, hw_pte_t *ptep, unsigned long addr,\n \t\t struct folio *new_folio, pte_t old, unsigned long sz)\n {\n \tpte_t newpte = make_huge_pte(vma, new_folio, true);\n@@ -4934,7 +4934,8 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,\n \t\t\t struct vm_area_struct *dst_vma,\n \t\t\t struct vm_area_struct *src_vma)\n {\n-\tpte_t *src_pte, *dst_pte, entry;\n+\thw_pte_t *src_pte, *dst_pte;\n+\tpte_t entry;\n \tstruct folio *pte_folio;\n \tunsigned long addr;\n \tbool cow = vma_is_cow_mapping(src_vma);\n@@ -5126,7 +5127,8 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,\n }\n \n static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr,\n-\t\t\t unsigned long new_addr, pte_t *src_pte, pte_t *dst_pte,\n+\t\t\t unsigned long new_addr, hw_pte_t *src_pte,\n+\t\t\t hw_pte_t *dst_pte,\n \t\t\t unsigned long sz)\n {\n \tbool need_clear_uffd_wp = vma_has_uffd_without_event_remap(vma);\n@@ -5188,7 +5190,7 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma,\n \tstruct mm_struct *mm = vma-\u003evm_mm;\n \tunsigned long old_end = old_addr + len;\n \tunsigned long last_addr_mask;\n-\tpte_t *src_pte, *dst_pte;\n+\thw_pte_t *src_pte, *dst_pte;\n \tstruct mmu_notifier_range range;\n \tstruct mmu_gather tlb;\n \n@@ -5249,7 +5251,7 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,\n \tstruct mm_struct *mm = vma-\u003evm_mm;\n \tconst bool folio_provided = !!folio;\n \tunsigned long address;\n-\tpte_t *ptep;\n+\thw_pte_t *ptep;\n \tpte_t pte;\n \tspinlock_t *ptl;\n \tstruct hstate *h = hstate_vma(vma);\n@@ -5783,7 +5785,7 @@ static inline vm_fault_t hugetlb_handle_userfault(struct vm_fault *vmf,\n * false if pte changed or is changing.\n */\n static bool hugetlb_pte_stable(struct hstate *h, struct mm_struct *mm, unsigned long addr,\n-\t\t\t pte_t *ptep, pte_t old_pte)\n+\t\t\t hw_pte_t *ptep, pte_t old_pte)\n {\n \tspinlock_t *ptl;\n \tbool same;\n@@ -6308,7 +6310,7 @@ static struct folio *alloc_hugetlb_folio_vma(struct hstate *h,\n * Used by userfaultfd UFFDIO_* ioctls. Based on userfaultfd's mfill_atomic_pte\n * with modifications for hugetlb pages.\n */\n-int hugetlb_mfill_atomic_pte(pte_t *dst_pte,\n+int hugetlb_mfill_atomic_pte(hw_pte_t *dst_pte,\n \t\t\t struct vm_area_struct *dst_vma,\n \t\t\t unsigned long dst_addr,\n \t\t\t unsigned long src_addr,\n@@ -6367,7 +6369,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,\n \n \t\tfolio = alloc_hugetlb_folio(dst_vma, dst_addr, false);\n \t\tif (IS_ERR(folio)) {\n-\t\t\tpte_t *actual_pte = hugetlb_walk(dst_vma, dst_addr, PMD_SIZE);\n+\t\t\thw_pte_t *actual_pte = hugetlb_walk(dst_vma, dst_addr, PMD_SIZE);\n \t\t\tif (actual_pte) {\n \t\t\t\tret = -EEXIST;\n \t\t\t\tgoto out;\n@@ -6535,7 +6537,7 @@ long hugetlb_change_protection(struct vm_area_struct *vma,\n {\n \tstruct mm_struct *mm = vma-\u003evm_mm;\n \tunsigned long start = address;\n-\tpte_t *ptep;\n+\thw_pte_t *ptep;\n \tpte_t pte;\n \tstruct hstate *h = hstate_vma(vma);\n \tlong pages = 0, psize = huge_page_size(h);\n@@ -7020,15 +7022,15 @@ void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma,\n * racing tasks could either miss the sharing (see huge_pte_offset) or select a\n * bad pmd for sharing.\n */\n-pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,\n+hw_pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,\n \t\t unsigned long addr, pud_t *pud)\n {\n \tstruct address_space *mapping = vma-\u003evm_file-\u003ef_mapping;\n \tconst pgoff_t idx = linear_page_index(vma, addr);\n \tstruct vm_area_struct *svma;\n \tunsigned long saddr;\n-\tpte_t *spte = NULL;\n-\tpte_t *pte;\n+\thw_pte_t *spte = NULL;\n+\thw_pte_t *pte;\n \n \ti_mmap_lock_read(mapping);\n \tmapping_rmap_tree_foreach(svma, mapping, idx, idx) {\n@@ -7059,13 +7061,13 @@ pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,\n \t}\n \tspin_unlock(\u0026mm-\u003epage_table_lock);\n out:\n-\tpte = (pte_t *)pmd_alloc(mm, pud, addr);\n+\tpte = (hw_pte_t *)pmd_alloc(mm, pud, addr);\n \ti_mmap_unlock_read(mapping);\n \treturn pte;\n }\n \n static int __huge_pmd_unshare(struct mmu_gather *tlb,\n-\t\tstruct vm_area_struct *vma, unsigned long addr, pte_t *ptep,\n+\t\tstruct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,\n \t\tbool check_locks)\n {\n \tunsigned long sz = huge_page_size(hstate_vma(vma));\n@@ -7106,7 +7108,7 @@ static int __huge_pmd_unshare(struct mmu_gather *tlb,\n *\t was not a shared PMD table.\n */\n int huge_pmd_unshare(struct mmu_gather *tlb, struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep)\n+\t\tunsigned long addr, hw_pte_t *ptep)\n {\n \treturn __huge_pmd_unshare(tlb, vma, addr, ptep, /*check_locks=*/true);\n }\n@@ -7136,21 +7138,21 @@ void huge_pmd_unshare_flush(struct mmu_gather *tlb, struct vm_area_struct *vma)\n \n #else /* !CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING */\n \n-pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,\n+hw_pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,\n \t\t unsigned long addr, pud_t *pud)\n {\n \treturn NULL;\n }\n \n static int __huge_pmd_unshare(struct mmu_gather *tlb,\n-\t\tstruct vm_area_struct *vma, unsigned long addr, pte_t *ptep,\n+\t\tstruct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,\n \t\tbool check_locks)\n {\n \treturn 0;\n }\n \n int huge_pmd_unshare(struct mmu_gather *tlb, struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep)\n+\t\tunsigned long addr, hw_pte_t *ptep)\n {\n \treturn 0;\n }\n@@ -7171,13 +7173,13 @@ bool want_pmd_share(struct vm_area_struct *vma, unsigned long addr)\n #endif /* CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING */\n \n #ifdef CONFIG_ARCH_WANT_GENERAL_HUGETLB\n-pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,\n+hw_pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,\n \t\t\tunsigned long addr, unsigned long sz)\n {\n \tpgd_t *pgd;\n \tp4d_t *p4d;\n \tpud_t *pud;\n-\tpte_t *pte = NULL;\n+\thw_pte_t *pte = NULL;\n \n \tpgd = pgd_offset(mm, addr);\n \tp4d = p4d_alloc(mm, pgd, addr);\n@@ -7186,13 +7188,13 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,\n \tpud = pud_alloc(mm, p4d, addr);\n \tif (pud) {\n \t\tif (sz == PUD_SIZE) {\n-\t\t\tpte = (pte_t *)pud;\n+\t\t\tpte = (hw_pte_t *)pud;\n \t\t} else {\n \t\t\tBUG_ON(sz != PMD_SIZE);\n \t\t\tif (want_pmd_share(vma, addr) \u0026\u0026 pud_none(*pud))\n \t\t\t\tpte = huge_pmd_share(mm, vma, addr, pud);\n \t\t\telse\n-\t\t\t\tpte = (pte_t *)pmd_alloc(mm, pud, addr);\n+\t\t\t\tpte = (hw_pte_t *)pmd_alloc(mm, pud, addr);\n \t\t}\n \t}\n \n@@ -7214,7 +7216,7 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,\n * size @sz doesn't match the hugepage size at this level of the page\n * table.\n */\n-pte_t *huge_pte_offset(struct mm_struct *mm,\n+hw_pte_t *huge_pte_offset(struct mm_struct *mm,\n \t\t unsigned long addr, unsigned long sz)\n {\n \tpgd_t *pgd;\n@@ -7232,14 +7234,14 @@ pte_t *huge_pte_offset(struct mm_struct *mm,\n \tpud = pud_offset(p4d, addr);\n \tif (sz == PUD_SIZE)\n \t\t/* must be pud huge, non-present or none */\n-\t\treturn (pte_t *)pud;\n+\t\treturn (hw_pte_t *)pud;\n \tif (!pud_present(*pud))\n \t\treturn NULL;\n \t/* must have a valid entry and size to go further */\n \n \tpmd = pmd_offset(pud, addr);\n \t/* must be pmd huge, non-present or none */\n-\treturn (pte_t *)pmd;\n+\treturn (hw_pte_t *)pmd;\n }\n \n /*\n@@ -7418,7 +7420,7 @@ static void hugetlb_unshare_pmds(struct vm_area_struct *vma,\n \tstruct mmu_gather tlb;\n \tunsigned long address;\n \tspinlock_t *ptl;\n-\tpte_t *ptep;\n+\thw_pte_t *ptep;\n \n \tif (!(vma-\u003evm_flags \u0026 VM_MAYSHARE))\n \t\treturn;\ndiff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c\nindex eb339c4a71f40..bf85d82c29b96 100644\n--- a/mm/hugetlb_vmemmap.c\n+++ b/mm/hugetlb_vmemmap.c\n@@ -34,7 +34,7 @@\n *\t\t\toperations.\n */\n struct vmemmap_remap_walk {\n-\tvoid\t\t\t(*remap_pte)(pte_t *pte, unsigned long addr,\n+\tvoid\t\t\t(*remap_pte)(hw_pte_t *pte, unsigned long addr,\n \t\t\t\t\t struct vmemmap_remap_walk *walk);\n \n \tunsigned long\t\tnr_walked;\n@@ -56,7 +56,7 @@ static int vmemmap_split_pmd(pmd_t *pmd, struct page *head, unsigned long start,\n \tpmd_t __pmd;\n \tint i;\n \tunsigned long addr = start;\n-\tpte_t *pgtable;\n+\thw_pte_t *pgtable;\n \n \tpgtable = pte_alloc_one_kernel(\u0026init_mm);\n \tif (!pgtable)\n@@ -65,7 +65,8 @@ static int vmemmap_split_pmd(pmd_t *pmd, struct page *head, unsigned long start,\n \tpmd_populate_kernel(\u0026init_mm, \u0026__pmd, pgtable);\n \n \tfor (i = 0; i \u003c PTRS_PER_PTE; i++, addr += PAGE_SIZE) {\n-\t\tpte_t entry, *pte;\n+\t\tpte_t entry;\n+\t\thw_pte_t *pte;\n \t\tpgprot_t pgprot = PAGE_KERNEL;\n \n \t\tentry = mk_pte(head + i, pgprot);\n@@ -137,7 +138,7 @@ static int vmemmap_pmd_entry(pmd_t *pmd, unsigned long addr,\n \treturn vmemmap_split_pmd(pmd, head, addr \u0026 PMD_MASK, vmemmap_walk);\n }\n \n-static int vmemmap_pte_entry(pte_t *pte, unsigned long addr,\n+static int vmemmap_pte_entry(hw_pte_t *pte, unsigned long addr,\n \t\t\t unsigned long next, struct mm_walk *walk)\n {\n \tstruct vmemmap_remap_walk *vmemmap_walk = walk-\u003eprivate;\n@@ -199,7 +200,7 @@ static void free_vmemmap_page_list(struct list_head *list)\n \t\tfree_vmemmap_page(page);\n }\n \n-static void vmemmap_remap_pte(pte_t *pte, unsigned long addr,\n+static void vmemmap_remap_pte(hw_pte_t *pte, unsigned long addr,\n \t\t\t struct vmemmap_remap_walk *walk)\n {\n \tstruct page *page = pte_page(ptep_get(pte));\n@@ -233,7 +234,7 @@ static void vmemmap_remap_pte(pte_t *pte, unsigned long addr,\n \tset_pte_at(\u0026init_mm, addr, pte, entry);\n }\n \n-static void vmemmap_restore_pte(pte_t *pte, unsigned long addr,\n+static void vmemmap_restore_pte(hw_pte_t *pte, unsigned long addr,\n \t\t\t\tstruct vmemmap_remap_walk *walk)\n {\n \tstruct page *src = pte_page(ptep_get(pte)), *dst;\ndiff --git a/mm/internal.h b/mm/internal.h\nindex da14c56fb24e1..8c14f0cf07878 100644\n--- a/mm/internal.h\n+++ b/mm/internal.h\n@@ -266,7 +266,7 @@ void unmap_vmas(struct mmu_gather *tlb, struct unmap_desc *unmap);\n #ifdef CONFIG_MMU\n \n bool cond_install_uffd_wp_ptes(struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep, pte_t pte,\n+\t\tunsigned long addr, hw_pte_t *ptep, pte_t pte,\n \t\tunsigned long nr_ptes);\n \n static inline void get_anon_vma(struct anon_vma *anon_vma)\n@@ -397,7 +397,7 @@ static inline pte_t __pte_batch_clear_ignored(pte_t pte, fpb_t flags)\n * Return: the number of table entries in the batch.\n */\n static inline unsigned int folio_pte_batch_flags(struct folio *folio,\n-\t\tstruct vm_area_struct *vma, pte_t *ptep, pte_t *ptentp,\n+\t\tstruct vm_area_struct *vma, hw_pte_t *ptep, pte_t *ptentp,\n \t\tunsigned int max_nr, fpb_t flags)\n {\n \tbool any_writable = false, any_young = false, any_dirty = false;\n@@ -451,7 +451,7 @@ static inline unsigned int folio_pte_batch_flags(struct folio *folio,\n \treturn min(nr, max_nr);\n }\n \n-unsigned int folio_pte_batch(struct folio *folio, pte_t *ptep, pte_t pte,\n+unsigned int folio_pte_batch(struct folio *folio, hw_pte_t *ptep, pte_t pte,\n \t\tunsigned int max_nr);\n \n /**\n@@ -508,11 +508,11 @@ static inline pte_t pte_next_swp_offset(pte_t pte)\n *\n * Return: the number of table entries in the batch.\n */\n-static inline int swap_pte_batch(pte_t *start_ptep, int max_nr, pte_t pte)\n+static inline int swap_pte_batch(hw_pte_t *start_ptep, int max_nr, pte_t pte)\n {\n \tpte_t expected_pte = pte_next_swp_offset(pte);\n-\tconst pte_t *end_ptep = start_ptep + max_nr;\n-\tpte_t *ptep = start_ptep + 1;\n+\tconst hw_pte_t *end_ptep = start_ptep + max_nr;\n+\thw_pte_t *ptep = start_ptep + 1;\n \n \tVM_WARN_ON(max_nr \u003c 1);\n \tVM_WARN_ON(!softleaf_is_swap(softleaf_from_pte(pte)));\n@@ -1550,7 +1550,7 @@ static inline void maybe_rmap_unlock_action(struct vm_area_struct *vma,\n \n #ifdef CONFIG_MMU_NOTIFIER\n static inline bool clear_flush_young_ptes_notify(struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep, unsigned int nr)\n+\t\tunsigned long addr, hw_pte_t *ptep, unsigned int nr)\n {\n \tbool young;\n \n@@ -1571,7 +1571,7 @@ static inline bool pmdp_clear_flush_young_notify(struct vm_area_struct *vma,\n }\n \n static inline bool test_and_clear_young_ptes_notify(struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep, unsigned int nr)\n+\t\tunsigned long addr, hw_pte_t *ptep, unsigned int nr)\n {\n \tbool young;\n \ndiff --git a/mm/kasan/init.c b/mm/kasan/init.c\nindex 66a8838879876..9bbc2a41d23f0 100644\n--- a/mm/kasan/init.c\n+++ b/mm/kasan/init.c\n@@ -64,7 +64,7 @@ static inline bool kasan_pmd_table(pud_t pud)\n \treturn false;\n }\n #endif\n-pte_t kasan_early_shadow_pte[MAX_PTRS_PER_PTE + PTE_HWTABLE_PTRS]\n+hw_pte_t kasan_early_shadow_pte[MAX_PTRS_PER_PTE + PTE_HWTABLE_PTRS]\n \t__bss_pgtbl;\n \n static inline bool kasan_pte_table(pmd_t pmd)\n@@ -92,7 +92,7 @@ static __init void *early_alloc(size_t size, int node)\n static void __ref zero_pte_populate(pmd_t *pmd, unsigned long addr,\n \t\t\t\tunsigned long end)\n {\n-\tpte_t *pte = pte_offset_kernel(pmd, addr);\n+\thw_pte_t *pte = pte_offset_kernel(pmd, addr);\n \tpte_t zero_pte;\n \n \tzero_pte = pfn_pte(PFN_DOWN(__pa_symbol(kasan_early_shadow_page)),\n@@ -122,7 +122,7 @@ static int __ref zero_pmd_populate(pud_t *pud, unsigned long addr,\n \t\t}\n \n \t\tif (pmd_none(*pmd)) {\n-\t\t\tpte_t *p;\n+\t\t\thw_pte_t *p;\n \n \t\t\tif (slab_is_available())\n \t\t\t\tp = pte_alloc_one_kernel(\u0026init_mm);\n@@ -281,9 +281,9 @@ int __ref kasan_populate_early_shadow(const void *shadow_start,\n \treturn 0;\n }\n \n-static void kasan_free_pte(pte_t *pte_start, pmd_t *pmd)\n+static void kasan_free_pte(hw_pte_t *pte_start, pmd_t *pmd)\n {\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tint i;\n \n \tfor (i = 0; i \u003c PTRS_PER_PTE; i++) {\n@@ -341,7 +341,7 @@ static void kasan_free_p4d(p4d_t *p4d_start, pgd_t *pgd)\n \tpgd_clear(pgd);\n }\n \n-static void kasan_remove_pte_table(pte_t *pte, unsigned long addr,\n+static void kasan_remove_pte_table(hw_pte_t *pte, unsigned long addr,\n \t\t\t\tunsigned long end)\n {\n \tunsigned long next;\n@@ -369,7 +369,7 @@ static void kasan_remove_pmd_table(pmd_t *pmd, unsigned long addr,\n \tunsigned long next;\n \n \tfor (; addr \u003c end; addr = next, pmd++) {\n-\t\tpte_t *pte;\n+\t\thw_pte_t *pte;\n \n \t\tnext = pmd_addr_end(addr, end);\n \ndiff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c\nindex d286e0a045437..86fc7ed45dcd0 100644\n--- a/mm/kasan/shadow.c\n+++ b/mm/kasan/shadow.c\n@@ -189,7 +189,7 @@ static bool shadow_mapped(unsigned long addr)\n \tp4d_t *p4d;\n \tpud_t *pud;\n \tpmd_t *pmd;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \n \tif (pgd_none(*pgd))\n \t\treturn false;\n@@ -297,7 +297,7 @@ struct vmalloc_populate_data {\n \tstruct page **pages;\n };\n \n-static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,\n+static int kasan_populate_vmalloc_pte(hw_pte_t *ptep, unsigned long addr,\n \t\t\t\t void *_data)\n {\n \tstruct vmalloc_populate_data *data = _data;\n@@ -465,7 +465,7 @@ int __kasan_populate_vmalloc(unsigned long addr, unsigned long size, gfp_t gfp_m\n \treturn 0;\n }\n \n-static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr,\n+static int kasan_depopulate_vmalloc_pte(hw_pte_t *ptep, unsigned long addr,\n \t\t\t\t\tvoid *unused)\n {\n \tpte_t pte;\ndiff --git a/mm/khugepaged.c b/mm/khugepaged.c\nindex f49a6710933b1..4a7074f58b55d 100644\n--- a/mm/khugepaged.c\n+++ b/mm/khugepaged.c\n@@ -639,8 +639,8 @@ static void release_pte_folio(struct folio *folio)\n \tfolio_putback_lru(folio);\n }\n \n-static void release_pte_pages(pte_t *pte, pte_t *_pte,\n-\t\tstruct list_head *compound_pagelist)\n+static void release_pte_pages(hw_pte_t *pte, hw_pte_t *_pte,\n+\t\t\t struct list_head *compound_pagelist)\n {\n \tstruct folio *folio, *tmp;\n \n@@ -685,7 +685,8 @@ static void count_collapse_event(unsigned int order, enum vm_event_item vm_event\n }\n \n static enum scan_result __collapse_huge_page_isolate(struct vm_area_struct *vma,\n-\t\tunsigned long start_addr, pte_t *pte, struct collapse_control *cc,\n+\t\tunsigned long start_addr, hw_pte_t *pte,\n+\t\tstruct collapse_control *cc,\n \t\tunsigned int order, struct list_head *compound_pagelist)\n {\n \tconst unsigned int max_ptes_none = collapse_max_ptes_none(cc, vma, order);\n@@ -694,7 +695,7 @@ static enum scan_result __collapse_huge_page_isolate(struct vm_area_struct *vma,\n \tstruct page *page = NULL;\n \tstruct folio *folio = NULL;\n \tunsigned long addr = start_addr;\n-\tpte_t *_pte;\n+\thw_pte_t *_pte;\n \tint none_or_zero = 0, shared = 0, referenced = 0;\n \tenum scan_result result = SCAN_FAIL;\n \n@@ -840,16 +841,18 @@ static enum scan_result __collapse_huge_page_isolate(struct vm_area_struct *vma,\n \treturn result;\n }\n \n-static void __collapse_huge_page_copy_succeeded(pte_t *pte,\n-\t\tstruct vm_area_struct *vma, unsigned long address,\n-\t\tspinlock_t *ptl, unsigned int order,\n-\t\tstruct list_head *compound_pagelist)\n+static void __collapse_huge_page_copy_succeeded(hw_pte_t *pte,\n+\t\t\t\t\t\tstruct vm_area_struct *vma,\n+\t\t\t\t\t\tunsigned long address,\n+\t\t\t\t\t\tspinlock_t *ptl,\n+\t\t\t\t\t\tunsigned int order,\n+\t\t\t\t\t\tstruct list_head *compound_pagelist)\n {\n \tconst unsigned long nr_pages = 1UL \u003c\u003c order;\n \tunsigned long end = address + (PAGE_SIZE * nr_pages);\n \tstruct folio *src, *tmp;\n \tpte_t pteval;\n-\tpte_t *_pte;\n+\thw_pte_t *_pte;\n \tunsigned int nr_ptes;\n \n \tfor (_pte = pte; _pte \u003c pte + nr_pages; _pte += nr_ptes,\n@@ -904,9 +907,11 @@ static void __collapse_huge_page_copy_succeeded(pte_t *pte,\n \t}\n }\n \n-static void __collapse_huge_page_copy_failed(pte_t *pte,\n-\t\tpmd_t *pmd, pmd_t orig_pmd, struct vm_area_struct *vma,\n-\t\tunsigned int order, struct list_head *compound_pagelist)\n+static void __collapse_huge_page_copy_failed(hw_pte_t *pte,\n+\t\t\t\t\t pmd_t *pmd, pmd_t orig_pmd,\n+\t\t\t\t\t struct vm_area_struct *vma,\n+\t\t\t\t\t unsigned int order,\n+\t\t\t\t\t struct list_head *compound_pagelist)\n {\n \tconst unsigned long nr_pages = 1UL \u003c\u003c order;\n \tspinlock_t *pmd_ptl;\n@@ -942,10 +947,14 @@ static void __collapse_huge_page_copy_failed(pte_t *pte,\n * @ptl: lock on raw pages' PTEs\n * @compound_pagelist: list that stores compound pages\n */\n-static enum scan_result __collapse_huge_page_copy(pte_t *pte, struct folio *folio,\n-\t\tpmd_t *pmd, pmd_t orig_pmd, struct vm_area_struct *vma,\n-\t\tunsigned long address, spinlock_t *ptl, unsigned int order,\n-\t\tstruct list_head *compound_pagelist)\n+static enum scan_result __collapse_huge_page_copy(hw_pte_t *pte,\n+\t\t\t\t\t\t struct folio *folio,\n+\t\t\t\t\t\t pmd_t *pmd, pmd_t orig_pmd,\n+\t\t\t\t\t\t struct vm_area_struct *vma,\n+\t\t\t\t\t\t unsigned long address,\n+\t\t\t\t\t\t spinlock_t *ptl,\n+\t\t\t\t\t\t unsigned int order,\n+\t\t\t\t\t\t struct list_head *compound_pagelist)\n {\n \tconst unsigned long nr_pages = 1UL \u003c\u003c order;\n \tunsigned int i;\n@@ -1164,7 +1173,7 @@ static enum scan_result __collapse_huge_page_swapin(struct mm_struct *mm,\n \tvm_fault_t ret = 0;\n \tunsigned long addr, end = start_addr + (PAGE_SIZE \u003c\u003c order);\n \tenum scan_result result;\n-\tpte_t *pte = NULL;\n+\thw_pte_t *pte = NULL;\n \tspinlock_t *ptl;\n \n \tfor (addr = start_addr; addr \u003c end; addr += PAGE_SIZE) {\n@@ -1292,7 +1301,7 @@ static enum scan_result collapse_huge_page(struct mm_struct *mm, unsigned long s\n \tconst unsigned long end_addr = start_addr + (PAGE_SIZE \u003c\u003c order);\n \tLIST_HEAD(compound_pagelist);\n \tpmd_t *pmd, _pmd;\n-\tpte_t *pte = NULL;\n+\thw_pte_t *pte = NULL;\n \tpgtable_t pgtable;\n \tstruct folio *folio;\n \tspinlock_t *pmd_ptl, *pte_ptl;\n@@ -1606,7 +1615,8 @@ static enum scan_result collapse_scan_pmd(struct mm_struct *mm,\n \tunsigned int max_ptes_none = collapse_max_ptes_none(cc, vma, HPAGE_PMD_ORDER);\n \tenum tva_type tva_flags = cc-\u003eis_khugepaged ? TVA_KHUGEPAGED : TVA_FORCED_COLLAPSE;\n \tpmd_t *pmd;\n-\tpte_t *pte, *_pte, pteval;\n+\thw_pte_t *pte, *_pte;\n+\tpte_t pteval;\n \tint i;\n \tint none_or_zero = 0, shared = 0, referenced = 0;\n \tenum scan_result result = SCAN_FAIL;\n@@ -1871,7 +1881,7 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign\n \tunsigned long end = haddr + HPAGE_PMD_SIZE;\n \tstruct vm_area_struct *vma = vma_lookup(mm, haddr);\n \tstruct folio *folio;\n-\tpte_t *start_pte, *pte;\n+\thw_pte_t *start_pte, *pte;\n \tpmd_t *pmd, pgt_pmd;\n \tspinlock_t *pml = NULL, *ptl;\n \tint i;\ndiff --git a/mm/ksm.c b/mm/ksm.c\nindex 624f37975e129..3882d8916efa4 100644\n--- a/mm/ksm.c\n+++ b/mm/ksm.c\n@@ -618,7 +618,7 @@ static int break_ksm_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned long en\n {\n \tunsigned long *found_addr = (unsigned long *) walk-\u003eprivate;\n \tstruct mm_struct *mm = walk-\u003emm;\n-\tpte_t *start_ptep, *ptep;\n+\thw_pte_t *start_ptep, *ptep;\n \tspinlock_t *ptl;\n \tint found = 0;\n \n@@ -1292,7 +1292,7 @@ static u32 calc_checksum(struct page *page)\n }\n \n static int write_protect_page(struct vm_area_struct *vma, struct folio *folio,\n-\t\t\t pte_t *orig_pte)\n+\t\t\t pte_t *ptentp)\n {\n \tstruct mm_struct *mm = vma-\u003evm_mm;\n \tDEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, 0, 0);\n@@ -1371,7 +1371,7 @@ static int write_protect_page(struct vm_area_struct *vma, struct folio *folio,\n \n \t\tset_pte_at(mm, pvmw.address, pvmw.pte, entry);\n \t}\n-\t*orig_pte = entry;\n+\t*ptentp = entry;\n \terr = 0;\n \n out_unlock:\n@@ -1399,7 +1399,7 @@ static int replace_page(struct vm_area_struct *vma, struct page *page,\n \tstruct folio *folio = page_folio(page);\n \tpmd_t *pmd;\n \tpmd_t pmde;\n-\tpte_t *ptep;\n+\thw_pte_t *ptep;\n \tpte_t newpte;\n \tspinlock_t *ptl;\n \tunsigned long addr;\n@@ -2530,7 +2530,8 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon\n {\n \tstruct ksm_next_page_arg *private = walk-\u003eprivate;\n \tstruct vm_area_struct *vma = walk-\u003evma;\n-\tpte_t *start_ptep = NULL, *ptep, pte;\n+\thw_pte_t *start_ptep = NULL, *ptep;\n+\tpte_t pte;\n \tstruct mm_struct *mm = walk-\u003emm;\n \tstruct folio *folio;\n \tstruct page *page;\ndiff --git a/mm/madvise.c b/mm/madvise.c\nindex 73c2901b9adbf..a08270c9f7610 100644\n--- a/mm/madvise.c\n+++ b/mm/madvise.c\n@@ -198,7 +198,7 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned long start,\n {\n \tstruct vm_area_struct *vma = walk-\u003eprivate;\n \tstruct swap_io_ctx ctx = {};\n-\tpte_t *ptep = NULL;\n+\thw_pte_t *ptep = NULL;\n \tspinlock_t *ptl;\n \tunsigned long addr;\n \n@@ -350,7 +350,7 @@ static inline bool can_do_file_pageout(struct vm_area_struct *vma)\n }\n \n static inline int madvise_folio_pte_batch(unsigned long addr, unsigned long end,\n-\t\t\t\t\t struct folio *folio, pte_t *ptep,\n+\t\t\t\t\t struct folio *folio, hw_pte_t *ptep,\n \t\t\t\t\t pte_t *ptentp)\n {\n \tint max_nr = (end - addr) / PAGE_SIZE;\n@@ -368,7 +368,8 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,\n \tbool pageout = private-\u003epageout;\n \tstruct mm_struct *mm = tlb-\u003emm;\n \tstruct vm_area_struct *vma = walk-\u003evma;\n-\tpte_t *start_pte, *pte, ptent;\n+\thw_pte_t *start_pte, *pte;\n+\tpte_t ptent;\n \tspinlock_t *ptl;\n \tstruct folio *folio = NULL;\n \tLIST_HEAD(folio_list);\n@@ -670,7 +671,8 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,\n \tstruct mm_struct *mm = tlb-\u003emm;\n \tstruct vm_area_struct *vma = walk-\u003evma;\n \tspinlock_t *ptl;\n-\tpte_t *start_pte, *pte, ptent;\n+\thw_pte_t *start_pte, *pte;\n+\tpte_t ptent;\n \tstruct folio *folio;\n \tint nr_swap = 0;\n \tunsigned long next;\n@@ -1095,7 +1097,7 @@ static int guard_install_pmd_entry(pmd_t *pmd, unsigned long addr,\n \treturn pmd_trans_huge(pmdval);\n }\n \n-static int guard_install_pte_entry(pte_t *pte, unsigned long addr,\n+static int guard_install_pte_entry(hw_pte_t *pte, unsigned long addr,\n \t\t\t\t unsigned long next, struct mm_walk *walk)\n {\n \tpte_t pteval = ptep_get(pte);\n@@ -1113,12 +1115,12 @@ static int guard_install_pte_entry(pte_t *pte, unsigned long addr,\n }\n \n static int guard_install_set_pte(unsigned long addr, unsigned long next,\n-\t\t\t\t pte_t *ptep, struct mm_walk *walk)\n+\t\t\t\t pte_t *ptentp, struct mm_walk *walk)\n {\n \tunsigned long *nr_pages = (unsigned long *)walk-\u003eprivate;\n \n \t/* Simply install a PTE marker, this causes segfault on access. */\n-\t*ptep = make_pte_marker(PTE_MARKER_GUARD);\n+\t*ptentp = make_pte_marker(PTE_MARKER_GUARD);\n \t(*nr_pages)++;\n \n \treturn 0;\n@@ -1238,7 +1240,7 @@ static int guard_remove_pmd_entry(pmd_t *pmd, unsigned long addr,\n \treturn 0;\n }\n \n-static int guard_remove_pte_entry(pte_t *pte, unsigned long addr,\n+static int guard_remove_pte_entry(hw_pte_t *pte, unsigned long addr,\n \t\t\t\t unsigned long next, struct mm_walk *walk)\n {\n \tpte_t ptent = ptep_get(pte);\ndiff --git a/mm/mapping_dirty_helpers.c b/mm/mapping_dirty_helpers.c\nindex e0efa36e0a076..dcbd39912f819 100644\n--- a/mm/mapping_dirty_helpers.c\n+++ b/mm/mapping_dirty_helpers.c\n@@ -31,7 +31,7 @@ struct wp_walk {\n * The function write-protects a pte and records the range in\n * virtual address space of touched ptes for efficient range TLB flushes.\n */\n-static int wp_pte(pte_t *pte, unsigned long addr, unsigned long end,\n+static int wp_pte(hw_pte_t *pte, unsigned long addr, unsigned long end,\n \t\t struct mm_walk *walk)\n {\n \tstruct wp_walk *wpwalk = walk-\u003eprivate;\n@@ -86,7 +86,7 @@ struct clean_walk {\n * in the address_space, as well as the first and last of the bits\n * touched.\n */\n-static int clean_record_pte(pte_t *pte, unsigned long addr,\n+static int clean_record_pte(hw_pte_t *pte, unsigned long addr,\n \t\t\t unsigned long end, struct mm_walk *walk)\n {\n \tstruct wp_walk *wpwalk = walk-\u003eprivate;\ndiff --git a/mm/memory-failure.c b/mm/memory-failure.c\nindex a8b03e2920ba8..a89f3fde47a5d 100644\n--- a/mm/memory-failure.c\n+++ b/mm/memory-failure.c\n@@ -342,7 +342,7 @@ static unsigned long dev_pagemap_mapping_shift(struct vm_area_struct *vma,\n \tp4d_t *p4d;\n \tpud_t *pud;\n \tpmd_t *pmd;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tpte_t ptent;\n \n \tVM_BUG_ON_VMA(address == -EFAULT, vma);\n@@ -742,7 +742,7 @@ static int hwpoison_pte_range(pmd_t *pmdp, unsigned long addr,\n {\n \tstruct hwpoison_walk *hwp = walk-\u003eprivate;\n \tint ret = 0;\n-\tpte_t *ptep, *mapped_pte;\n+\thw_pte_t *ptep, *mapped_pte;\n \tspinlock_t *ptl;\n \n \tptl = pmd_trans_huge_lock(pmdp, walk-\u003evma);\n@@ -770,7 +770,7 @@ static int hwpoison_pte_range(pmd_t *pmdp, unsigned long addr,\n }\n \n #ifdef CONFIG_HUGETLB_PAGE\n-static int hwpoison_hugetlb_range(pte_t *ptep, unsigned long hmask,\n+static int hwpoison_hugetlb_range(hw_pte_t *ptep, unsigned long hmask,\n \t\t\t unsigned long addr, unsigned long end,\n \t\t\t struct mm_walk *walk)\n {\ndiff --git a/mm/memory.c b/mm/memory.c\nindex ec63dd6212ac5..7fc8ce10503c3 100644\n--- a/mm/memory.c\n+++ b/mm/memory.c\n@@ -462,7 +462,7 @@ int __pte_alloc(struct mm_struct *mm, pmd_t *pmd)\n \n int __pte_alloc_kernel(pmd_t *pmd)\n {\n-\tpte_t *new = pte_alloc_one_kernel(\u0026init_mm);\n+\thw_pte_t *new = pte_alloc_one_kernel(\u0026init_mm);\n \tif (!new)\n \t\treturn -ENOMEM;\n \n@@ -909,7 +909,7 @@ struct page *vm_normal_page_pud(struct vm_area_struct *vma,\n */\n static void restore_exclusive_pte(struct vm_area_struct *vma,\n \t\tstruct folio *folio, struct page *page, unsigned long address,\n-\t\tpte_t *ptep, pte_t orig_pte)\n+\t\thw_pte_t *ptep, pte_t orig_pte)\n {\n \tpte_t pte;\n \n@@ -946,7 +946,7 @@ static void restore_exclusive_pte(struct vm_area_struct *vma,\n * sleeping.\n */\n static int try_restore_exclusive_pte(struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep, pte_t orig_pte)\n+\t\tunsigned long addr, hw_pte_t *ptep, pte_t orig_pte)\n {\n \tconst softleaf_t entry = softleaf_from_pte(orig_pte);\n \tstruct page *page = softleaf_to_page(entry);\n@@ -969,7 +969,7 @@ static int try_restore_exclusive_pte(struct vm_area_struct *vma,\n \n static unsigned long\n copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm,\n-\t\tpte_t *dst_pte, pte_t *src_pte, struct vm_area_struct *dst_vma,\n+\t\thw_pte_t *dst_pte, hw_pte_t *src_pte, struct vm_area_struct *dst_vma,\n \t\tstruct vm_area_struct *src_vma, unsigned long addr, int *rss)\n {\n \tpte_t orig_pte = ptep_get(src_pte);\n@@ -1083,7 +1083,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm,\n */\n static inline int\n copy_present_page(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,\n-\t\t pte_t *dst_pte, pte_t *src_pte, unsigned long addr, int *rss,\n+\t\t hw_pte_t *dst_pte, hw_pte_t *src_pte, unsigned long addr, int *rss,\n \t\t struct folio **prealloc, struct page *page)\n {\n \tstruct folio *new_folio;\n@@ -1122,7 +1122,7 @@ copy_present_page(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma\n }\n \n static __always_inline void __copy_present_ptes(struct vm_area_struct *dst_vma,\n-\t\tstruct vm_area_struct *src_vma, pte_t *dst_pte, pte_t *src_pte,\n+\t\tstruct vm_area_struct *src_vma, hw_pte_t *dst_pte, hw_pte_t *src_pte,\n \t\tpte_t pte, unsigned long addr, int nr)\n {\n \tstruct mm_struct *src_mm = src_vma-\u003evm_mm;\n@@ -1172,7 +1172,7 @@ static __always_inline void __copy_present_ptes(struct vm_area_struct *dst_vma,\n */\n static inline int\n copy_present_ptes(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,\n-\t\t pte_t *dst_pte, pte_t *src_pte, pte_t pte, unsigned long addr,\n+\t\t hw_pte_t *dst_pte, hw_pte_t *src_pte, pte_t pte, unsigned long addr,\n \t\t int max_nr, int *rss, struct folio **prealloc)\n {\n \tfpb_t flags = FPB_MERGE_WRITE;\n@@ -1272,8 +1272,8 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,\n {\n \tstruct mm_struct *dst_mm = dst_vma-\u003evm_mm;\n \tstruct mm_struct *src_mm = src_vma-\u003evm_mm;\n-\tpte_t *orig_src_pte, *orig_dst_pte;\n-\tpte_t *src_pte, *dst_pte;\n+\thw_pte_t *orig_src_pte, *orig_dst_pte;\n+\thw_pte_t *src_pte, *dst_pte;\n \tpmd_t dummy_pmdval;\n \tpte_t ptent;\n \tspinlock_t *src_ptl, *dst_ptl;\n@@ -1666,7 +1666,7 @@ static inline bool zap_drop_markers(struct zap_details *details)\n * Returns true if uffd-wp PTEs were installed, false otherwise.\n */\n bool cond_install_uffd_wp_ptes(struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep, pte_t pte,\n+\t\tunsigned long addr, hw_pte_t *ptep, pte_t pte,\n \t\tunsigned long nr_ptes)\n {\n \tbool arm_uffd_pte = false;\n@@ -1720,7 +1720,7 @@ bool cond_install_uffd_wp_ptes(struct vm_area_struct *vma,\n */\n static inline bool\n zap_install_uffd_wp_if_needed(struct vm_area_struct *vma,\n-\t\t\t unsigned long addr, pte_t *pte, int nr,\n+\t\t\t unsigned long addr, hw_pte_t *pte, int nr,\n \t\t\t struct zap_details *details, pte_t pteval)\n {\n \tif (zap_drop_markers(details))\n@@ -1731,7 +1731,7 @@ zap_install_uffd_wp_if_needed(struct vm_area_struct *vma,\n \n static __always_inline void zap_present_folio_ptes(struct mmu_gather *tlb,\n \t\tstruct vm_area_struct *vma, struct folio *folio,\n-\t\tstruct page *page, pte_t *pte, pte_t ptent, unsigned int nr,\n+\t\tstruct page *page, hw_pte_t *pte, pte_t ptent, unsigned int nr,\n \t\tunsigned long addr, struct zap_details *details, int *rss,\n \t\tbool *force_flush, bool *force_break, bool *any_skipped)\n {\n@@ -1781,7 +1781,7 @@ static __always_inline void zap_present_folio_ptes(struct mmu_gather *tlb,\n * Returns the number of processed (skipped or zapped) PTEs (at least 1).\n */\n static inline int zap_present_ptes(struct mmu_gather *tlb,\n-\t\tstruct vm_area_struct *vma, pte_t *pte, pte_t ptent,\n+\t\tstruct vm_area_struct *vma, hw_pte_t *pte, pte_t ptent,\n \t\tunsigned int max_nr, unsigned long addr,\n \t\tstruct zap_details *details, int *rss, bool *force_flush,\n \t\tbool *force_break, bool *any_skipped)\n@@ -1827,7 +1827,7 @@ static inline int zap_present_ptes(struct mmu_gather *tlb,\n }\n \n static inline int zap_nonpresent_ptes(struct mmu_gather *tlb,\n-\t\tstruct vm_area_struct *vma, pte_t *pte, pte_t ptent,\n+\t\tstruct vm_area_struct *vma, hw_pte_t *pte, pte_t ptent,\n \t\tunsigned int max_nr, unsigned long addr,\n \t\tstruct zap_details *details, int *rss, bool *any_skipped)\n {\n@@ -1898,7 +1898,7 @@ static inline int zap_nonpresent_ptes(struct mmu_gather *tlb,\n }\n \n static inline int do_zap_pte_range(struct mmu_gather *tlb,\n-\t\t\t\t struct vm_area_struct *vma, pte_t *pte,\n+\t\t\t\t struct vm_area_struct *vma, hw_pte_t *pte,\n \t\t\t\t unsigned long addr, unsigned long end,\n \t\t\t\t struct zap_details *details, int *rss,\n \t\t\t\t bool *force_flush, bool *force_break,\n@@ -1961,7 +1961,7 @@ static bool zap_pte_table_if_empty(struct mm_struct *mm, pmd_t *pmd,\n \t\tunsigned long addr, pmd_t *pmdval)\n {\n \tspinlock_t *pml, *ptl = NULL;\n-\tpte_t *start_pte, *pte;\n+\thw_pte_t *start_pte, *pte;\n \tint i;\n \n \tpml = pmd_lock(mm, pmd);\n@@ -2001,8 +2001,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb,\n \tstruct mm_struct *mm = tlb-\u003emm;\n \tint rss[NR_MM_COUNTERS];\n \tspinlock_t *ptl;\n-\tpte_t *start_pte;\n-\tpte_t *pte;\n+\thw_pte_t *start_pte;\n+\thw_pte_t *pte;\n \tpmd_t pmdval;\n \tunsigned long start = addr;\n \tbool direct_reclaim = true;\n@@ -2384,7 +2384,7 @@ static pmd_t *walk_to_pmd(struct mm_struct *mm, unsigned long addr)\n \treturn pmd;\n }\n \n-pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,\n+hw_pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,\n \t\t spinlock_t **ptl)\n {\n \tpmd_t *pmd = walk_to_pmd(mm, addr);\n@@ -2442,7 +2442,7 @@ static int validate_page_before_insert(struct vm_area_struct *vma,\n \treturn 0;\n }\n \n-static int insert_page_into_pte_locked(struct vm_area_struct *vma, pte_t *pte,\n+static int insert_page_into_pte_locked(struct vm_area_struct *vma, hw_pte_t *pte,\n \t\t\t\tunsigned long addr, struct page *page,\n \t\t\t\tpgprot_t prot, bool mkwrite)\n {\n@@ -2487,7 +2487,7 @@ static int insert_page(struct vm_area_struct *vma, unsigned long addr,\n \t\t\tstruct page *page, pgprot_t prot, bool mkwrite)\n {\n \tint retval;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tspinlock_t *ptl;\n \n \tretval = validate_page_before_insert(vma, page);\n@@ -2504,7 +2504,7 @@ static int insert_page(struct vm_area_struct *vma, unsigned long addr,\n \treturn retval;\n }\n \n-static int insert_page_in_batch_locked(struct vm_area_struct *vma, pte_t *pte,\n+static int insert_page_in_batch_locked(struct vm_area_struct *vma, hw_pte_t *pte,\n \t\t\tunsigned long addr, struct page *page, pgprot_t prot)\n {\n \tint err;\n@@ -2522,7 +2522,7 @@ static int insert_pages(struct vm_area_struct *vma, unsigned long addr,\n \t\t\tstruct page **pages, unsigned long *num, pgprot_t prot)\n {\n \tpmd_t *pmd = NULL;\n-\tpte_t *start_pte, *pte;\n+\thw_pte_t *start_pte, *pte;\n \tspinlock_t *pte_lock;\n \tstruct mm_struct *const mm = vma-\u003evm_mm;\n \tunsigned long curr_page_idx = 0;\n@@ -2765,7 +2765,8 @@ static vm_fault_t insert_pfn(struct vm_area_struct *vma, unsigned long addr,\n \t\t\tunsigned long pfn, pgprot_t prot, bool mkwrite)\n {\n \tstruct mm_struct *mm = vma-\u003evm_mm;\n-\tpte_t *pte, entry;\n+\thw_pte_t *pte;\n+\tpte_t entry;\n \tspinlock_t *ptl;\n \n \tpte = get_locked_pte(mm, addr, \u0026ptl);\n@@ -3006,7 +3007,7 @@ static int remap_pte_range(struct mm_struct *mm, pmd_t *pmd,\n \t\t\tunsigned long addr, unsigned long end,\n \t\t\tunsigned long pfn, pgprot_t prot)\n {\n-\tpte_t *pte, *mapped_pte;\n+\thw_pte_t *pte, *mapped_pte;\n \tspinlock_t *ptl;\n \tint err = 0;\n \n@@ -3407,7 +3408,7 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd,\n \t\t\t\t pte_fn_t fn, void *data, bool create,\n \t\t\t\t pgtbl_mod_mask *mask)\n {\n-\tpte_t *pte, *mapped_pte;\n+\thw_pte_t *pte, *mapped_pte;\n \tint err = 0;\n \tspinlock_t *ptl;\n \n@@ -4710,7 +4711,7 @@ static vm_fault_t handle_pte_marker(struct vm_fault *vmf)\n /*\n * Check if the PTEs within a range are contiguous swap entries.\n */\n-static bool can_swapin_thp(struct vm_fault *vmf, pte_t *ptep, int nr_pages)\n+static bool can_swapin_thp(struct vm_fault *vmf, hw_pte_t *ptep, int nr_pages)\n {\n \tunsigned long addr;\n \tint idx;\n@@ -4762,7 +4763,7 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)\n \tunsigned long addr;\n \tsoftleaf_t entry;\n \tspinlock_t *ptl;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tint order;\n \n \t/*\n@@ -4856,7 +4857,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)\n \tint nr_pages;\n \tunsigned long page_idx;\n \tunsigned long address;\n-\tpte_t *ptep;\n+\thw_pte_t *ptep;\n \n \tif (!pte_unmap_same(vmf))\n \t\tgoto out;\n@@ -5025,7 +5026,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)\n \t\tunsigned long idx = folio_page_idx(folio, page);\n \t\tunsigned long folio_start = address - idx * PAGE_SIZE;\n \t\tunsigned long folio_end = folio_start + nr * PAGE_SIZE;\n-\t\tpte_t *folio_ptep;\n+\t\thw_pte_t *folio_ptep;\n \t\tpte_t folio_pte;\n \n \t\tif (unlikely(folio_start \u003c max(address \u0026 PMD_MASK, vma-\u003evm_start)))\n@@ -5255,7 +5256,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)\n \treturn ret;\n }\n \n-static bool pte_range_none(pte_t *pte, int nr_pages)\n+static bool pte_range_none(hw_pte_t *pte, int nr_pages)\n {\n \tint i;\n \n@@ -5274,7 +5275,7 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf)\n \tunsigned long orders;\n \tstruct folio *folio;\n \tunsigned long addr;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tgfp_t gfp;\n \tint order;\n \n@@ -5356,7 +5357,7 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf)\n \treturn folio_prealloc(vma-\u003evm_mm, vma, vmf-\u003eaddress, true);\n }\n \n-void map_anon_folio_pte_nopf(struct folio *folio, pte_t *pte,\n+void map_anon_folio_pte_nopf(struct folio *folio, hw_pte_t *pte,\n \t\tstruct vm_area_struct *vma, unsigned long addr,\n \t\tbool uffd_wp)\n {\n@@ -5377,7 +5378,7 @@ void map_anon_folio_pte_nopf(struct folio *folio, pte_t *pte,\n \tupdate_mmu_cache_range(NULL, vma, addr, pte, nr_pages);\n }\n \n-static void map_anon_folio_pte_pf(struct folio *folio, pte_t *pte,\n+static void map_anon_folio_pte_pf(struct folio *folio, hw_pte_t *pte,\n \t\tstruct vm_area_struct *vma, unsigned long addr, bool uffd_wp)\n {\n \tconst unsigned int order = folio_order(folio);\n@@ -6162,7 +6163,7 @@ int numa_migrate_check(struct folio *folio, struct vm_fault *vmf,\n }\n \n static void numa_rebuild_single_mapping(struct vm_fault *vmf, struct vm_area_struct *vma,\n-\t\t\t\t\tunsigned long fault_addr, pte_t *fault_pte,\n+\t\t\t\t\tunsigned long fault_addr, hw_pte_t *fault_pte,\n \t\t\t\t\tbool writable)\n {\n \tpte_t pte, old_pte;\n@@ -6184,7 +6185,7 @@ static void numa_rebuild_large_mapping(struct vm_fault *vmf, struct vm_area_stru\n \tunsigned long start, end, addr = vmf-\u003eaddress;\n \tunsigned long addr_start = addr - (nr \u003c\u003c PAGE_SHIFT);\n \tunsigned long pt_start = ALIGN_DOWN(addr, PMD_SIZE);\n-\tpte_t *start_ptep;\n+\thw_pte_t *start_ptep;\n \n \t/* Stay within the VMA and within the page table. */\n \tstart = max3(addr_start, pt_start, vma-\u003evm_start);\n@@ -6944,7 +6945,7 @@ int __pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address)\n #endif /* __PAGETABLE_PMD_FOLDED */\n \n static inline void pfnmap_args_setup(struct follow_pfnmap_args *args,\n-\t\t\t\t spinlock_t *lock, pte_t *ptep,\n+\t\t\t\t spinlock_t *lock, hw_pte_t *ptep,\n \t\t\t\t pgprot_t pgprot, unsigned long pfn_base,\n \t\t\t\t unsigned long addr_mask, bool writable,\n \t\t\t\t bool special)\n@@ -7013,7 +7014,8 @@ int follow_pfnmap_start(struct follow_pfnmap_args *args)\n \tp4d_t *p4dp, p4d;\n \tpud_t *pudp, pud;\n \tpmd_t *pmdp, pmd;\n-\tpte_t *ptep, pte;\n+\thw_pte_t *ptep;\n+\tpte_t pte;\n \n \tpfnmap_lockdep_assert(vma);\n \ndiff --git a/mm/mempolicy.c b/mm/mempolicy.c\nindex 2ad0a5f18280a..51ab12fa4cb61 100644\n--- a/mm/mempolicy.c\n+++ b/mm/mempolicy.c\n@@ -710,7 +710,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr,\n \tstruct folio *folio;\n \tstruct queue_pages *qp = walk-\u003eprivate;\n \tunsigned long flags = qp-\u003eflags;\n-\tpte_t *pte, *mapped_pte;\n+\thw_pte_t *pte, *mapped_pte;\n \tpte_t ptent;\n \tspinlock_t *ptl;\n \tint max_nr, nr;\n@@ -790,7 +790,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr,\n \treturn 0;\n }\n \n-static int queue_folios_hugetlb(pte_t *pte, unsigned long hmask,\n+static int queue_folios_hugetlb(hw_pte_t *pte, unsigned long hmask,\n \t\t\t unsigned long addr, unsigned long end,\n \t\t\t struct mm_walk *walk)\n {\ndiff --git a/mm/migrate.c b/mm/migrate.c\nindex 15b45832bcfa7..1786ab5e941c2 100644\n--- a/mm/migrate.c\n+++ b/mm/migrate.c\n@@ -494,7 +494,7 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd,\n \t\t\t unsigned long address)\n {\n \tspinlock_t *ptl;\n-\tpte_t *ptep;\n+\thw_pte_t *ptep;\n \tpte_t pte;\n \tsoftleaf_t entry;\n \n@@ -525,7 +525,7 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd,\n *\n * This function will release the vma lock before returning.\n */\n-void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep)\n+void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep)\n {\n \tspinlock_t *ptl = huge_pte_lockptr(hstate_vma(vma), vma-\u003evm_mm, ptep);\n \tsoftleaf_t entry;\ndiff --git a/mm/migrate_device.c b/mm/migrate_device.c\nindex 0c437004329d9..10da0b00aaca5 100644\n--- a/mm/migrate_device.c\n+++ b/mm/migrate_device.c\n@@ -254,7 +254,7 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp,\n \tspinlock_t *ptl;\n \tstruct folio *fault_folio = migrate-\u003efault_page ?\n \t\tpage_folio(migrate-\u003efault_page) : NULL;\n-\tpte_t *ptep;\n+\thw_pte_t *ptep;\n \n again:\n \tif (pmd_trans_huge(*pmdp) || !pmd_present(*pmdp)) {\n@@ -989,7 +989,7 @@ static void migrate_vma_insert_page(struct migrate_vma *migrate,\n \tp4d_t *p4dp;\n \tpud_t *pudp;\n \tpmd_t *pmdp;\n-\tpte_t *ptep;\n+\thw_pte_t *ptep;\n \tpte_t orig_pte;\n \n \t/* Only allow populating anonymous memory */\ndiff --git a/mm/mincore.c b/mm/mincore.c\nindex c086836bc4bcc..65cc43c85d03f 100644\n--- a/mm/mincore.c\n+++ b/mm/mincore.c\n@@ -24,7 +24,7 @@\n #include \"swap.h\"\n #include \"internal.h\"\n \n-static int mincore_hugetlb(pte_t *pte, unsigned long hmask, unsigned long addr,\n+static int mincore_hugetlb(hw_pte_t *pte, unsigned long hmask, unsigned long addr,\n \t\t\tunsigned long end, struct mm_walk *walk)\n {\n #ifdef CONFIG_HUGETLB_PAGE\n@@ -164,7 +164,7 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,\n {\n \tspinlock_t *ptl;\n \tstruct vm_area_struct *vma = walk-\u003evma;\n-\tpte_t *ptep;\n+\thw_pte_t *ptep;\n \tunsigned char *vec = walk-\u003eprivate;\n \tint nr = (end - addr) \u003e\u003e PAGE_SHIFT;\n \tint step, i;\ndiff --git a/mm/mlock.c b/mm/mlock.c\nindex 39215a3eab1fb..cc6c24e46348a 100644\n--- a/mm/mlock.c\n+++ b/mm/mlock.c\n@@ -305,7 +305,7 @@ void munlock_folio(struct folio *folio)\n }\n \n static inline unsigned int folio_mlock_step(struct folio *folio,\n-\t\tpte_t *pte, unsigned long addr, unsigned long end)\n+\t\thw_pte_t *pte, unsigned long addr, unsigned long end)\n {\n \tunsigned int count = (end - addr) \u003e\u003e PAGE_SHIFT;\n \tpte_t ptent = ptep_get(pte);\n@@ -353,7 +353,7 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr,\n {\n \tstruct vm_area_struct *vma = walk-\u003evma;\n \tspinlock_t *ptl;\n-\tpte_t *start_pte, *pte;\n+\thw_pte_t *start_pte, *pte;\n \tpte_t ptent;\n \tstruct folio *folio;\n \tunsigned int step = 1;\ndiff --git a/mm/mprotect.c b/mm/mprotect.c\nindex 2888ee638d872..1d23475e0bb76 100644\n--- a/mm/mprotect.c\n+++ b/mm/mprotect.c\n@@ -103,7 +103,7 @@ bool can_change_pte_writable(struct vm_area_struct *vma, unsigned long addr,\n \treturn can_change_shared_pte_writable(vma, pte);\n }\n \n-static int mprotect_folio_pte_batch(struct folio *folio, pte_t *ptep,\n+static int mprotect_folio_pte_batch(struct folio *folio, hw_pte_t *ptep,\n \t\t\t\t pte_t pte, int max_nr_ptes, fpb_t flags)\n {\n \t/* No underlying folio, so cannot batch */\n@@ -118,7 +118,7 @@ static int mprotect_folio_pte_batch(struct folio *folio, pte_t *ptep,\n \n /* Set nr_ptes number of ptes, starting from idx */\n static __always_inline void prot_commit_flush_ptes(struct vm_area_struct *vma,\n-\t\tunsigned long addr, pte_t *ptep, pte_t oldpte, pte_t ptent,\n+\t\tunsigned long addr, hw_pte_t *ptep, pte_t oldpte, pte_t ptent,\n \t\tint nr_ptes, int idx, bool set_write, struct mmu_gather *tlb)\n {\n \t/*\n@@ -170,7 +170,7 @@ static __always_inline int page_anon_exclusive_batch(int start_idx, int max_len,\n * retrieve sub-batches.\n */\n static __always_inline void commit_anon_folio_batch(struct vm_area_struct *vma,\n-\t\tstruct folio *folio, struct page *first_page, unsigned long addr, pte_t *ptep,\n+\t\tstruct folio *folio, struct page *first_page, unsigned long addr, hw_pte_t *ptep,\n \t\tpte_t oldpte, pte_t ptent, int nr_ptes, struct mmu_gather *tlb)\n {\n \tbool expected_anon_exclusive;\n@@ -189,7 +189,7 @@ static __always_inline void commit_anon_folio_batch(struct vm_area_struct *vma,\n }\n \n static __always_inline void set_write_prot_commit_flush_ptes(struct vm_area_struct *vma,\n-\t\tstruct folio *folio, struct page *page, unsigned long addr, pte_t *ptep,\n+\t\tstruct folio *folio, struct page *page, unsigned long addr, hw_pte_t *ptep,\n \t\tpte_t oldpte, pte_t ptent, int nr_ptes, struct mmu_gather *tlb)\n {\n \tbool set_write;\n@@ -212,7 +212,7 @@ static __always_inline void set_write_prot_commit_flush_ptes(struct vm_area_stru\n }\n \n static long change_softleaf_pte(struct vm_area_struct *vma,\n-\tunsigned long addr, pte_t *pte, pte_t oldpte, unsigned long cp_flags)\n+\tunsigned long addr, hw_pte_t *pte, pte_t oldpte, unsigned long cp_flags)\n {\n \tconst bool uffd_prot = cp_flags \u0026 (MM_CP_UFFD_WP | MM_CP_UFFD_RWP);\n \tconst bool uffd_prot_resolve = cp_flags \u0026\n@@ -279,7 +279,7 @@ static long change_softleaf_pte(struct vm_area_struct *vma,\n }\n \n static __always_inline void change_present_ptes(struct mmu_gather *tlb,\n-\t\tstruct vm_area_struct *vma, unsigned long addr, pte_t *ptep,\n+\t\tstruct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,\n \t\tint nr_ptes, unsigned long end, pgprot_t newprot,\n \t\tstruct folio *folio, struct page *page, unsigned long cp_flags)\n {\n@@ -332,7 +332,8 @@ static long change_pte_range(struct mmu_gather *tlb,\n \t\tstruct vm_area_struct *vma, pmd_t *pmd, unsigned long addr,\n \t\tunsigned long end, pgprot_t newprot, unsigned long cp_flags)\n {\n-\tpte_t *pte, oldpte;\n+\thw_pte_t *pte;\n+\tpte_t oldpte;\n \tspinlock_t *ptl;\n \tlong pages = 0;\n \tbool is_private_single_threaded;\n@@ -727,7 +728,7 @@ long change_protection(struct mmu_gather *tlb,\n \treturn pages;\n }\n \n-static int prot_none_pte_entry(pte_t *pte, unsigned long addr,\n+static int prot_none_pte_entry(hw_pte_t *pte, unsigned long addr,\n \t\t\t unsigned long next, struct mm_walk *walk)\n {\n \treturn pfn_modify_allowed(pte_pfn(ptep_get(pte)),\n@@ -736,7 +737,7 @@ static int prot_none_pte_entry(pte_t *pte, unsigned long addr,\n }\n \n #ifdef CONFIG_HUGETLB_PAGE\n-static int prot_none_hugetlb_entry(pte_t *pte, unsigned long hmask,\n+static int prot_none_hugetlb_entry(hw_pte_t *pte, unsigned long hmask,\n \t\t\t\t unsigned long addr, unsigned long next,\n \t\t\t\t struct mm_walk *walk)\n {\ndiff --git a/mm/mremap.c b/mm/mremap.c\nindex 7c368440fafe2..5272cea12f80e 100644\n--- a/mm/mremap.c\n+++ b/mm/mremap.c\n@@ -176,7 +176,7 @@ static pte_t move_soft_dirty_pte(pte_t pte)\n }\n \n static int mremap_folio_pte_batch(struct vm_area_struct *vma, unsigned long addr,\n-\t\tpte_t *ptep, pte_t pte, int max_nr)\n+\t\thw_pte_t *ptep, pte_t pte, int max_nr)\n {\n \tstruct folio *folio;\n \n@@ -200,7 +200,7 @@ static int move_ptes(struct pagetable_move_control *pmc,\n \tstruct vm_area_struct *vma = pmc-\u003eold;\n \tbool need_clear_uffd_wp = vma_has_uffd_without_event_remap(vma);\n \tstruct mm_struct *mm = vma-\u003evm_mm;\n-\tpte_t *old_ptep, *new_ptep;\n+\thw_pte_t *old_ptep, *new_ptep;\n \tpte_t old_pte, pte;\n \tpmd_t dummy_pmdval;\n \tspinlock_t *old_ptl, *new_ptl;\ndiff --git a/mm/page_table_check.c b/mm/page_table_check.c\nindex 143a918c0bde2..705794a817bad 100644\n--- a/mm/page_table_check.c\n+++ b/mm/page_table_check.c\n@@ -208,7 +208,7 @@ static void page_table_check_pte_flags(pte_t pte)\n }\n \n void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr,\n-\t\t\t\t pte_t *ptep, pte_t pte, unsigned int nr)\n+\t\t\t\t hw_pte_t *ptep, pte_t pte, unsigned int nr)\n {\n \tunsigned int i;\n \n@@ -279,7 +279,7 @@ void __page_table_check_pte_clear_range(struct mm_struct *mm,\n \t\treturn;\n \n \tif (!pmd_none(pmd) \u0026\u0026 !pmd_bad(pmd) \u0026\u0026 !pmd_leaf(pmd)) {\n-\t\tpte_t *ptep = pte_offset_map(\u0026pmd, addr);\n+\t\thw_pte_t *ptep = pte_offset_map(\u0026pmd, addr);\n \t\tunsigned long i;\n \n \t\tif (WARN_ON(!ptep))\ndiff --git a/mm/pagewalk.c b/mm/pagewalk.c\nindex 7411702a37f58..b6a29a06a19bc 100644\n--- a/mm/pagewalk.c\n+++ b/mm/pagewalk.c\n@@ -26,7 +26,7 @@ static int real_depth(int depth)\n \treturn depth;\n }\n \n-static int walk_pte_range_inner(pte_t *pte, unsigned long addr,\n+static int walk_pte_range_inner(hw_pte_t *pte, unsigned long addr,\n \t\t\t\tunsigned long end, struct mm_walk *walk)\n {\n \tconst struct mm_walk_ops *ops = walk-\u003eops;\n@@ -61,7 +61,7 @@ static int walk_pte_range_inner(pte_t *pte, unsigned long addr,\n static int walk_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,\n \t\t\t struct mm_walk *walk)\n {\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tint err = 0;\n \tspinlock_t *ptl;\n \n@@ -341,7 +341,7 @@ static int walk_hugetlb_range(unsigned long addr, unsigned long end,\n \tunsigned long next;\n \tunsigned long hmask = huge_page_mask(h);\n \tunsigned long sz = huge_page_size(h);\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tconst struct mm_walk_ops *ops = walk-\u003eops;\n \tint err = 0;\n \n@@ -905,7 +905,8 @@ struct folio *folio_walk_start(struct folio_walk *fw,\n \tstruct page *page;\n \tpud_t *pudp, pud;\n \tpmd_t *pmdp, pmd;\n-\tpte_t *ptep, pte;\n+\thw_pte_t *ptep;\n+\tpte_t pte;\n \tspinlock_t *ptl;\n \tpgd_t *pgdp;\n \tp4d_t *p4dp;\ndiff --git a/mm/percpu.c b/mm/percpu.c\nindex 3eff382e565ad..ec0e58e9f6c09 100644\n--- a/mm/percpu.c\n+++ b/mm/percpu.c\n@@ -3173,7 +3173,7 @@ void __init __weak pcpu_populate_pte(unsigned long addr)\n \n \tpmd = pmd_offset(pud, addr);\n \tif (!pmd_present(*pmd)) {\n-\t\tpte_t *new;\n+\t\thw_pte_t *new;\n \n \t\tnew = memblock_alloc_or_panic(PTE_TABLE_SIZE, PTE_TABLE_SIZE);\n \t\tpmd_populate_kernel(\u0026init_mm, pmd, new);\ndiff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c\nindex 224c444cf45d9..df548ad3d9a54 100644\n--- a/mm/pgtable-generic.c\n+++ b/mm/pgtable-generic.c\n@@ -80,7 +80,7 @@ void pmd_clear_bad(pmd_t *pmd)\n * force that call on sun4c so we changed this macro slightly\n */\n int ptep_set_access_flags(struct vm_area_struct *vma,\n-\t\t\t unsigned long address, pte_t *ptep,\n+\t\t\t unsigned long address, hw_pte_t *ptep,\n \t\t\t pte_t entry, int dirty)\n {\n \tint changed = !pte_same(ptep_get(ptep), entry);\n@@ -94,7 +94,7 @@ int ptep_set_access_flags(struct vm_area_struct *vma,\n \n #ifndef __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH\n bool ptep_clear_flush_young(struct vm_area_struct *vma,\n-\t\tunsigned long address, pte_t *ptep)\n+\t\tunsigned long address, hw_pte_t *ptep)\n {\n \tbool young;\n \n@@ -107,7 +107,7 @@ bool ptep_clear_flush_young(struct vm_area_struct *vma,\n \n #ifndef __HAVE_ARCH_PTEP_CLEAR_FLUSH\n pte_t ptep_clear_flush(struct vm_area_struct *vma, unsigned long address,\n-\t\t pte_t *ptep)\n+\t\t hw_pte_t *ptep)\n {\n \tstruct mm_struct *mm = (vma)-\u003evm_mm;\n \tpte_t pte;\n@@ -294,7 +294,7 @@ static unsigned long pmdp_get_lockless_start(void) { return 0; }\n static void pmdp_get_lockless_end(unsigned long irqflags) { }\n #endif\n \n-pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp)\n+hw_pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp)\n {\n \tunsigned long irqflags;\n \tpmd_t pmdval;\n@@ -320,11 +320,11 @@ pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp)\n \treturn NULL;\n }\n \n-pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,\n+hw_pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,\n \t\t\t\tunsigned long addr, spinlock_t **ptlp)\n {\n \tpmd_t pmdval;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \n \tpte = __pte_offset_map(pmd, addr, \u0026pmdval);\n \tif (likely(pte))\n@@ -332,11 +332,11 @@ pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,\n \treturn pte;\n }\n \n-pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,\n+hw_pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,\n \t\t\t\tunsigned long addr, pmd_t *pmdvalp,\n \t\t\t\tspinlock_t **ptlp)\n {\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \n \tVM_WARN_ON_ONCE(!pmdvalp);\n \tpte = __pte_offset_map(pmd, addr, pmdvalp);\n@@ -402,12 +402,12 @@ pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,\n * table, and may not use RCU at all: \"outsiders\" like khugepaged should avoid\n * pte_offset_map() and co once the vma is detached from mm or mm_users is zero.\n */\n-pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd,\n+hw_pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd,\n \t\t\t unsigned long addr, spinlock_t **ptlp)\n {\n \tspinlock_t *ptl;\n \tpmd_t pmdval;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n again:\n \tpte = __pte_offset_map(pmd, addr, \u0026pmdval);\n \tif (unlikely(!pte))\ndiff --git a/mm/ptdump.c b/mm/ptdump.c\nindex 5851096e6f656..8f19f20be3c44 100644\n--- a/mm/ptdump.c\n+++ b/mm/ptdump.c\n@@ -19,7 +19,7 @@ static inline int note_kasan_page_table(struct mm_walk *walk,\n {\n \tstruct ptdump_state *st = walk-\u003eprivate;\n \n-\tst-\u003enote_page_pte(st, addr, kasan_early_shadow_pte[0]);\n+\tst-\u003enote_page_pte(st, addr, ptep_get(kasan_early_shadow_pte));\n \n \twalk-\u003eaction = ACTION_CONTINUE;\n \n@@ -117,7 +117,7 @@ static int ptdump_pmd_entry(pmd_t *pmd, unsigned long addr,\n \treturn 0;\n }\n \n-static int ptdump_pte_entry(pte_t *pte, unsigned long addr,\n+static int ptdump_pte_entry(hw_pte_t *pte, unsigned long addr,\n \t\t\t unsigned long next, struct mm_walk *walk)\n {\n \tstruct ptdump_state *st = walk-\u003eprivate;\ndiff --git a/mm/rmap.c b/mm/rmap.c\nindex b5cc9273fe5a4..0223c4ab0b361 100644\n--- a/mm/rmap.c\n+++ b/mm/rmap.c\n@@ -1122,7 +1122,7 @@ static int page_vma_mkclean_one(struct page_vma_mapped_walk *pvmw)\n \n \t\taddress = pvmw-\u003eaddress;\n \t\tif (pvmw-\u003epte) {\n-\t\t\tpte_t *pte = pvmw-\u003epte;\n+\t\t\thw_pte_t *pte = pvmw-\u003epte;\n \t\t\tpte_t entry = ptep_get(pte);\n \n \t\t\t/*\n@@ -2139,7 +2139,7 @@ static pte_t swp_pte_prepare(swp_entry_t entry, pte_t old_pte,\n \n static bool ttu_anon_swapbacked_folio(struct vm_area_struct *vma,\n \t\tstruct folio *folio, struct page *page, unsigned long address,\n-\t\tpte_t *ptep, pte_t pteval)\n+\t\thw_pte_t *ptep, pte_t pteval)\n {\n \tconst bool anon_exclusive = folio_test_anon(folio) \u0026\u0026\n \t\t\t\t PageAnonExclusive(page);\n@@ -2174,7 +2174,7 @@ static bool ttu_anon_swapbacked_folio(struct vm_area_struct *vma,\n }\n \n static bool ttu_anon_folio(struct vm_area_struct *vma, struct folio *folio,\n-\t\tstruct page *page, unsigned long address, pte_t *ptep,\n+\t\tstruct page *page, unsigned long address, hw_pte_t *ptep,\n \t\tpte_t pteval, unsigned long nr_pages)\n {\n \t/*\ndiff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c\nindex e62e6aa07f126..77eb1749e123f 100644\n--- a/mm/sparse-vmemmap.c\n+++ b/mm/sparse-vmemmap.c\n@@ -135,7 +135,7 @@ static void * __meminit altmap_alloc_block_buf(unsigned long size,\n \treturn __va(__pfn_to_phys(pfn));\n }\n \n-void __meminit vmemmap_verify(pte_t *pte, int node,\n+void __meminit vmemmap_verify(hw_pte_t *pte, int node,\n \t\t\t\tunsigned long start, unsigned long end)\n {\n \tunsigned long pfn = pte_pfn(ptep_get(pte));\n@@ -236,11 +236,11 @@ static __meminit void *vmemmap_alloc_pte(unsigned long pfn, int node,\n \treturn page_address(page);\n }\n \n-static pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node,\n+static hw_pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node,\n \t\t\t\t struct vmem_altmap *altmap,\n \t\t\t\t unsigned long ptpfn, unsigned long flags)\n {\n-\tpte_t *pte = pte_offset_kernel(pmd, addr);\n+\thw_pte_t *pte = pte_offset_kernel(pmd, addr);\n \tunsigned long pfn = page_to_pfn((struct page *)addr);\n \n \tif (pte_none(ptep_get(pte))) {\n@@ -323,7 +323,7 @@ static pgd_t * __meminit vmemmap_pgd_populate(unsigned long addr, int node)\n \treturn pgd;\n }\n \n-static pte_t * __meminit vmemmap_populate_address(unsigned long addr, int node,\n+static hw_pte_t * __meminit vmemmap_populate_address(unsigned long addr, int node,\n \t\t\t\t\t struct vmem_altmap *altmap,\n \t\t\t\t\t unsigned long ptpfn,\n \t\t\t\t\t unsigned long flags)\n@@ -332,7 +332,7 @@ static pte_t * __meminit vmemmap_populate_address(unsigned long addr, int node,\n \tp4d_t *p4d;\n \tpud_t *pud;\n \tpmd_t *pmd;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \n \tpgd = vmemmap_pgd_populate(addr, node);\n \tif (!pgd)\n@@ -361,7 +361,7 @@ static int __meminit vmemmap_populate_range(unsigned long start,\n \t\t\t\t\t unsigned long flags)\n {\n \tunsigned long addr = start;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \n \tfor (; addr \u003c end; addr += PAGE_SIZE) {\n \t\tpte = vmemmap_populate_address(addr, node, altmap,\n@@ -394,7 +394,7 @@ void vmemmap_wrprotect_hvo(unsigned long addr, unsigned long end,\n \t\t\t\t int node, unsigned long headsize)\n {\n \tunsigned long maddr;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \n \tfor (maddr = addr + headsize; maddr \u003c end; maddr += PAGE_SIZE) {\n \t\tpte = virt_to_kpte(maddr);\n@@ -413,7 +413,7 @@ int __weak __meminit vmemmap_check_pmd(pmd_t *pmd, int node,\n {\n \tif (!pmd_leaf(pmdp_get(pmd)))\n \t\treturn 0;\n-\tvmemmap_verify((pte_t *)pmd, node, addr, next);\n+\tvmemmap_verify((hw_pte_t *)pmd, node, addr, next);\n \n \treturn 1;\n }\n@@ -497,9 +497,9 @@ static bool __meminit reuse_compound_section(unsigned long start_pfn,\n \treturn !IS_ALIGNED(offset, nr_pages) \u0026\u0026 nr_pages \u003e PAGES_PER_SUBSECTION;\n }\n \n-static pte_t * __meminit compound_section_tail_page(unsigned long addr)\n+static hw_pte_t * __meminit compound_section_tail_page(unsigned long addr)\n {\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \n \taddr -= PAGE_SIZE;\n \n@@ -520,7 +520,7 @@ static int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn,\n \t\t\t\t\t\t struct dev_pagemap *pgmap)\n {\n \tunsigned long size, addr;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tint rc;\n \n \tif (reuse_compound_section(start_pfn, pgmap)) {\ndiff --git a/mm/swap_state.c b/mm/swap_state.c\nindex 305877e1f4d7b..d654449f393e8 100644\n--- a/mm/swap_state.c\n+++ b/mm/swap_state.c\n@@ -918,7 +918,8 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,\n \tstruct swap_io_ctx ctx = {};\n \tstruct blk_plug plug;\n \tstruct folio *folio;\n-\tpte_t *pte = NULL, pentry;\n+\thw_pte_t *pte = NULL;\n+\tpte_t pentry;\n \tint win;\n \tunsigned long start, end, addr;\n \tpgoff_t ilx = targ_ilx;\ndiff --git a/mm/swapfile.c b/mm/swapfile.c\nindex 869a918b18e8e..2f6aa8dc2f5bb 100644\n--- a/mm/swapfile.c\n+++ b/mm/swapfile.c\n@@ -2500,7 +2500,8 @@ static int unuse_pte(struct vm_area_struct *vma, pmd_t *pmd,\n \tstruct page *page;\n \tstruct folio *swapcache;\n \tspinlock_t *ptl;\n-\tpte_t *pte, new_pte, old_pte;\n+\thw_pte_t *pte;\n+\tpte_t new_pte, old_pte;\n \tbool hwpoisoned = false;\n \tint ret = 1;\n \n@@ -2611,7 +2612,7 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd,\n \t\t\tunsigned long addr, unsigned long end,\n \t\t\tunsigned int type)\n {\n-\tpte_t *pte = NULL;\n+\thw_pte_t *pte = NULL;\n \n \tdo {\n \t\tstruct folio *folio;\ndiff --git a/mm/userfaultfd.c b/mm/userfaultfd.c\nindex 79cc7b546f130..8281796638c0a 100644\n--- a/mm/userfaultfd.c\n+++ b/mm/userfaultfd.c\n@@ -334,13 +334,13 @@ static int mfill_atomic_install_pte(pmd_t *dst_pmd,\n {\n \tint ret;\n \tstruct mm_struct *dst_mm = dst_vma-\u003evm_mm;\n-\tpte_t _dst_pte, *dst_pte;\n+\thw_pte_t *dst_pte;\n \tbool writable = dst_vma-\u003evm_flags \u0026 VM_WRITE;\n \tbool vm_shared = dst_vma-\u003evm_flags \u0026 VM_SHARED;\n \tspinlock_t *ptl;\n \tstruct folio *folio = page_folio(page);\n \tbool page_in_cache = folio_mapping(folio);\n-\tpte_t dst_ptep;\n+\tpte_t _dst_pte, dst_ptep;\n \n \t_dst_pte = mk_pte(page, dst_vma-\u003evm_page_prot);\n \t_dst_pte = pte_mkdirty(_dst_pte);\n@@ -629,7 +629,8 @@ static int mfill_atomic_pte_zeropage(struct mfill_state *state)\n \tstruct vm_area_struct *dst_vma = state-\u003evma;\n \tunsigned long dst_addr = state-\u003edst_addr;\n \tpmd_t *dst_pmd = state-\u003epmd;\n-\tpte_t _dst_pte, *dst_pte;\n+\thw_pte_t *dst_pte;\n+\tpte_t _dst_pte;\n \tspinlock_t *ptl;\n \tint ret;\n \n@@ -710,7 +711,8 @@ static int mfill_atomic_pte_poison(struct mfill_state *state)\n \tstruct mm_struct *dst_mm = dst_vma-\u003evm_mm;\n \tunsigned long dst_addr = state-\u003edst_addr;\n \tpmd_t *dst_pmd = state-\u003epmd;\n-\tpte_t _dst_pte, *dst_pte;\n+\thw_pte_t *dst_pte;\n+\tpte_t _dst_pte;\n \tspinlock_t *ptl;\n \tint ret;\n \n@@ -757,7 +759,7 @@ static __always_inline ssize_t mfill_atomic_hugetlb(\n {\n \tstruct mm_struct *dst_mm = dst_vma-\u003evm_mm;\n \tssize_t err;\n-\tpte_t *dst_pte;\n+\thw_pte_t *dst_pte;\n \tunsigned long src_addr, dst_addr;\n \tlong copied;\n \tstruct folio *folio;\n@@ -1234,7 +1236,7 @@ void double_pt_unlock(spinlock_t *ptl1,\n \t\t__release(ptl2);\n }\n \n-static inline bool is_pte_pages_stable(pte_t *dst_pte, pte_t *src_pte,\n+static inline bool is_pte_pages_stable(hw_pte_t *dst_pte, hw_pte_t *src_pte,\n \t\t\t\t pte_t orig_dst_pte, pte_t orig_src_pte,\n \t\t\t\t pmd_t *dst_pmd, pmd_t dst_pmdval)\n {\n@@ -1252,7 +1254,8 @@ static inline bool is_pte_pages_stable(pte_t *dst_pte, pte_t *src_pte,\n */\n static struct folio *check_ptes_for_batched_move(struct vm_area_struct *src_vma,\n \t\t\t\t\t\t unsigned long src_addr,\n-\t\t\t\t\t\t pte_t *src_pte, pte_t *dst_pte)\n+\t\t\t\t\t\t hw_pte_t *src_pte,\n+\t\t\t\t\t\t hw_pte_t *dst_pte)\n {\n \tpte_t orig_dst_pte, orig_src_pte;\n \tstruct folio *folio;\n@@ -1283,7 +1286,7 @@ static long move_present_ptes(struct mm_struct *mm,\n \t\t\t struct vm_area_struct *dst_vma,\n \t\t\t struct vm_area_struct *src_vma,\n \t\t\t unsigned long dst_addr, unsigned long src_addr,\n-\t\t\t pte_t *dst_pte, pte_t *src_pte,\n+\t\t\t hw_pte_t *dst_pte, hw_pte_t *src_pte,\n \t\t\t pte_t orig_dst_pte, pte_t orig_src_pte,\n \t\t\t pmd_t *dst_pmd, pmd_t dst_pmdval,\n \t\t\t spinlock_t *dst_ptl, spinlock_t *src_ptl,\n@@ -1370,7 +1373,7 @@ static long move_present_ptes(struct mm_struct *mm,\n \n static int move_swap_pte(struct mm_struct *mm, struct vm_area_struct *dst_vma,\n \t\t\t unsigned long dst_addr, unsigned long src_addr,\n-\t\t\t pte_t *dst_pte, pte_t *src_pte,\n+\t\t\t hw_pte_t *dst_pte, hw_pte_t *src_pte,\n \t\t\t pte_t orig_dst_pte, pte_t orig_src_pte,\n \t\t\t pmd_t *dst_pmd, pmd_t dst_pmdval,\n \t\t\t spinlock_t *dst_ptl, spinlock_t *src_ptl,\n@@ -1435,7 +1438,7 @@ static int move_zeropage_pte(struct mm_struct *mm,\n \t\t\t struct vm_area_struct *dst_vma,\n \t\t\t struct vm_area_struct *src_vma,\n \t\t\t unsigned long dst_addr, unsigned long src_addr,\n-\t\t\t pte_t *dst_pte, pte_t *src_pte,\n+\t\t\t hw_pte_t *dst_pte, hw_pte_t *src_pte,\n \t\t\t pte_t orig_dst_pte, pte_t orig_src_pte,\n \t\t\t pmd_t *dst_pmd, pmd_t dst_pmdval,\n \t\t\t spinlock_t *dst_ptl, spinlock_t *src_ptl)\n@@ -1481,8 +1484,8 @@ static long move_pages_ptes(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd\n \tpte_t orig_src_pte, orig_dst_pte;\n \tpte_t src_folio_pte;\n \tspinlock_t *src_ptl, *dst_ptl;\n-\tpte_t *src_pte = NULL;\n-\tpte_t *dst_pte = NULL;\n+\thw_pte_t *src_pte = NULL;\n+\thw_pte_t *dst_pte = NULL;\n \tpmd_t dummy_pmdval;\n \tpmd_t dst_pmdval;\n \tstruct folio *src_folio = NULL;\n@@ -2597,7 +2600,8 @@ static inline bool userfaultfd_huge_must_wait(struct userfaultfd_ctx *ctx,\n \t\t\t\t\t unsigned long reason)\n {\n \tstruct vm_area_struct *vma = vmf-\u003evma;\n-\tpte_t *ptep, pte;\n+\thw_pte_t *ptep;\n+\tpte_t pte;\n \n \tassert_fault_locked(vmf);\n \n@@ -2670,7 +2674,7 @@ static inline bool userfaultfd_must_wait(struct userfaultfd_ctx *ctx,\n \tp4d_t *p4d;\n \tpud_t *pud;\n \tpmd_t *pmd, _pmd;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tpte_t ptent;\n \tbool ret;\n \ndiff --git a/mm/util.c b/mm/util.c\nindex bf0513d1d3d08..cc253050f6892 100644\n--- a/mm/util.c\n+++ b/mm/util.c\n@@ -1564,7 +1564,7 @@ EXPORT_SYMBOL(mmap_action_complete);\n *\n * Return: the number of table entries in the batch.\n */\n-unsigned int folio_pte_batch(struct folio *folio, pte_t *ptep, pte_t pte,\n+unsigned int folio_pte_batch(struct folio *folio, hw_pte_t *ptep, pte_t pte,\n \t\tunsigned int max_nr)\n {\n \treturn folio_pte_batch_flags(folio, NULL, ptep, \u0026pte, max_nr, 0);\ndiff --git a/mm/vmalloc.c b/mm/vmalloc.c\nindex 6ed6c160abed7..60ba46838bbeb 100644\n--- a/mm/vmalloc.c\n+++ b/mm/vmalloc.c\n@@ -97,7 +97,7 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,\n \t\t\tphys_addr_t phys_addr, pgprot_t prot,\n \t\t\tunsigned int max_page_shift, pgtbl_mod_mask *mask)\n {\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tu64 pfn;\n \tstruct page *page;\n \tunsigned long size = PAGE_SIZE;\n@@ -389,7 +389,7 @@ int ioremap_page_range(unsigned long addr, unsigned long end,\n static void vunmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,\n \t\t\t pgtbl_mod_mask *mask)\n {\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tpte_t ptent;\n \tunsigned long size = PAGE_SIZE;\n \n@@ -550,7 +550,7 @@ static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr,\n \t\tpgtbl_mod_mask *mask)\n {\n \tint err = 0;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \n \t/*\n \t * nr is a running index into the array which helps higher level\n@@ -827,7 +827,8 @@ struct page *vmalloc_to_page(const void *vmalloc_addr)\n \tp4d_t *p4d;\n \tpud_t *pud;\n \tpmd_t *pmd;\n-\tpte_t *ptep, pte;\n+\thw_pte_t *ptep;\n+\tpte_t pte;\n \n \t/*\n \t * XXX we might need to change this if we add VIRTUAL_BUG_ON for\n@@ -3615,7 +3616,7 @@ struct vmap_pfn_data {\n \tunsigned int\tidx;\n };\n \n-static int vmap_pfn_apply(pte_t *pte, unsigned long addr, void *private)\n+static int vmap_pfn_apply(hw_pte_t *pte, unsigned long addr, void *private)\n {\n \tstruct vmap_pfn_data *data = private;\n \tunsigned long pfn = data-\u003epfns[data-\u003eidx];\ndiff --git a/mm/vmscan.c b/mm/vmscan.c\nindex 245f68c75b289..4cadd4f4b00d1 100644\n--- a/mm/vmscan.c\n+++ b/mm/vmscan.c\n@@ -3579,7 +3579,7 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end,\n {\n \tint i;\n \tbool dirty;\n-\tpte_t *pte;\n+\thw_pte_t *pte;\n \tspinlock_t *ptl;\n \tunsigned long addr;\n \tint total = 0;\n@@ -3612,7 +3612,7 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end,\n \tfor (i = pte_index(start), addr = start; addr != end; i += nr, addr += nr * PAGE_SIZE) {\n \t\tunsigned long pfn;\n \t\tstruct folio *folio;\n-\t\tpte_t *cur_pte = pte + i;\n+\t\thw_pte_t *cur_pte = pte + i;\n \t\tpte_t ptent = ptep_get(cur_pte);\n \n \t\tnr = 1;\n@@ -4343,7 +4343,7 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr)\n \tstruct lru_gen_mm_walk *walk;\n \tstruct folio *last = NULL;\n \tint young = nr;\n-\tpte_t *pte = pvmw-\u003epte;\n+\thw_pte_t *pte = pvmw-\u003epte;\n \tunsigned long addr = pvmw-\u003eaddress;\n \tstruct vm_area_struct *vma = pvmw-\u003evma;\n \tstruct folio *folio = pfn_folio(pvmw-\u003epfn);\n"
} |
| 2/1 | 2026/09/04 15:31 | agent | patch-evaluator |
1mModel:core Args: null Results: {
"EnableConfigs": [
"HUGETLBFS",
"TRANSPARENT_HUGEPAGE",
"USERFAULTFD",
"PROC_PAGE_MONITOR",
"PAGE_TABLE_CHECK",
"BPF_SYSCALL"
],
"FocusSymbols": [
"hugetlb_mfill_atomic_pte",
"copy_hugetlb_page_range",
"move_hugetlb_page_tables",
"huge_pmd_share",
"huge_pmd_unshare",
"collapse_huge_page",
"try_collapse_pte_mapped_thp",
"folio_walk_start",
"__page_table_check_ptes_set",
"do_pagemap_scan"
],
"Reasoning": "The patch introduces a new hw_pte_t type abstraction and converts raw pte_t* pointers across core memory management subsystems (hugetlb, khugepaged, pagewalk, userfaultfd, page_table_check, proc task_mmu, and bpf arena) to hw_pte_t*. Because this modifies core reachable memory management interfaces and data flow, it is worth fuzzing to detect regressions or unintended side effects.",
"WorthFuzzing": true
}Instruction: You are an expert Linux kernel maintainer and security engineer.
Your job is to review a provided patch series and evaluate whether it warrants fuzzing with syzkaller.
IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in
your workspace. Do NOT rely on internal assumptions. You must actively use your code access
tools to inspect the actual source code, callers, and surrounding context.
================================================================================
1. CORE TRIAGE PHILOSOPHY
================================================================================
The goal of patch fuzzing is to discover crashes, regressions, exposed latent bugs,
and newly triggered assertions introduced by the patch series.
- REACHABILITY IS THE PRIMARY GATE:
Fuzzing can only discover bugs in code that can actually execute in standard virtualized
environments (GCE or QEMU, utilizing software-emulated devices like USB gadgets, netdev, tun/tap).
If the modified code is structurally unreachable (see Section 2), it MUST NOT be fuzzed,
regardless of whether it adds assertions or complex logic.
- DO NOT BLINDLY TRUST "NO FUNCTIONAL CHANGE" (NFCI) OR "REFACTORING" CLAIMS:
Patch authors routinely label changes as "cleanups", "refactorings", or state
"No functional change intended". Do NOT take these claims at face value.
Code refactorings that rearrange logic, introduce helper functions, or alter state management
in core subsystems frequently introduce subtle semantic shifts or uncover latent kernel bugs.
If reachable executable code is modified or refactored, it MUST be fuzzed.
- NEW OR MODIFIED ASSERTIONS IN REACHABLE CODE MUST BE FUZZED:
When a patch introduces or modifies runtime checks or assertions (e.g., WARN_ON*, VM_WARN_ON*,
BUG_ON*, lockdep_assert*) in reachable code paths, it enforces new or stricter invariants.
Even if the author believes the invariant always holds, fuzzing is essential to verify whether
an unusual sequence of operations can violate it.
================================================================================
2. WHEN TO RETURN WorthFuzzing=false (NEGATIVE CRITERIA)
================================================================================
Return WorthFuzzing=false ONLY IF all modified code falls strictly into one or more of these categories:
- Non-kernel and non-executable changes:
* Modifications to Documentation/, comments, or spelling fixes.
* User-space directories, self-tests, samples, or scripts (e.g., tools/, samples/, scripts/, usr/)
that do not affect the compiled kernel image (vmlinux) or kernel modules.
* Purely decorative logging (e.g., message strings in pr_err, printk, dev_info) or tracepoints
that do not alter control flow or data structures.
* Build system or Kconfig changes that do not alter compiled C logic.
- Structurally unreachable hardware:
* Vendor-specific PCIe switches, SmartNICs, or GPU drivers (e.g., mlxsw, pds_core, qed,
ionic, amdgpu) requiring physical ASIC/PCIe cards not emulated in standard QEMU.
- Unreachable execution paths:
* Driver teardown callbacks (.remove, .shutdown, pci_unregister_driver) executed only during
physical PCI hot-unplug or manual sysfs driver unbinding.
* Code paths exclusive to architectures other than the target architecture.
================================================================================
3. WHEN TO RETURN WorthFuzzing=true (POSITIVE CRITERIA)
================================================================================
Return WorthFuzzing=true whenever the patch touches reachable executable code, including:
- Core Subsystems:
* Any logic modifications in memory management (mm/), synchronization/locking (kernel/locking/),
BPF, scheduler, core networking, VFS, or syscall handling.
- Refactorings and Code Cleanups:
* Any restructuring of reachable data structures, helper abstractions, or algorithm flows.
- Runtime Assertions and Defensive Checks:
* Any introduction or alteration of assertions (WARN_ON*, VM_WARN_ON*, BUG_ON*, etc.) in reachable paths.
- Reachable Drivers and Protocols:
* Drivers accessible via virtual buses (virtio, USB gadget, loopback, netlink, binder, sockets, etc.).
================================================================================
4. EXTRACTING FocusSymbols (PREVENTING DILUTION)
================================================================================
When WorthFuzzing=true, you must extract specific kernel functions into FocusSymbols to guide the fuzzer:
- AVOID UBIQUITOUS LIFECYCLE HOT-PATHS:
Do NOT list generic, ubiquitous functions called by almost every program in the corpus
(including, but not limited to: general memory allocators and deallocators, page fault
and trap handlers, or core synchronization primitives; this is not an exhaustive list).
Listing ubiquitous functions causes the fuzzer to classify thousands of unrelated tests as "focused",
which severely dilutes fuzzing effort away from the actual changes.
- TARGET SPECIFIC FEATURE LOGIC AND ENTRYPOINTS:
List functions that specifically implement the logic being added or altered, or direct API entrypoints
for the subsystem feature under review.
- HANDLING STATIC INLINE FUNCTIONS IN HEADERS (.h):
Compiler-inlined static functions (such as static inlines in mm/*.h or include/linux/*.h) lack
distinct symbol addresses in vmlinux and cannot be targeted directly by symbol coverage filters.
If the changes are primarily in static inline helpers, identify non-static, feature-specific caller
functions in .c files that exercise them (avoiding ubiquitous lifecycle wrappers).
================================================================================
5. IDENTIFYING EnableConfigs
================================================================================
Identify any specific CONFIG_ options required to properly compile and reach the modified code:
- Inspect Kconfig files and #ifdef guards; do not make assumptions.
- Check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed.
- Strip any 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4").
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Target architecture: amd64
For your convenience, here is the diff of the changes:
commit e8ceacb10b59eb9d38e460ed31a10da5b37fe459
Author: syz-cluster <triage@syzkaller.com>
Date: Fri Sep 4 15:30:59 2026 +0000
syz-cluster: applied patch under review
diff --git a/MAINTAINERS b/MAINTAINERS
index a9245d827ddb6..b0ce3e6693045 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -17140,6 +17140,7 @@ F: include/linux/mmu_notifier.h
F: include/linux/pagewalk.h
F: include/linux/pgalloc.h
F: include/linux/pgtable.h
+F: include/linux/pgtable_types.h
F: include/linux/ptdump.h
F: include/linux/vmpressure.h
F: include/linux/vmstat.h
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
index d01acfb7d93d0..056faf4a3618b 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
@@ -1690,7 +1690,7 @@ static int igt_mmap_gpu(void *arg)
return 0;
}
-static int check_present_pte(pte_t *pte, unsigned long addr, void *data)
+static int check_present_pte(hw_pte_t *pte, unsigned long addr, void *data)
{
pte_t ptent = ptep_get(pte);
@@ -1703,7 +1703,7 @@ static int check_present_pte(pte_t *pte, unsigned long addr, void *data)
return 0;
}
-static int check_absent_pte(pte_t *pte, unsigned long addr, void *data)
+static int check_absent_pte(hw_pte_t *pte, unsigned long addr, void *data)
{
pte_t ptent = ptep_get(pte);
diff --git a/drivers/gpu/drm/i915/i915_mm.c b/drivers/gpu/drm/i915/i915_mm.c
index fd89e7c7d8d6f..aab88e8edf946 100644
--- a/drivers/gpu/drm/i915/i915_mm.c
+++ b/drivers/gpu/drm/i915/i915_mm.c
@@ -48,7 +48,7 @@ static inline unsigned long sgt_pfn(const struct remap_pfn *r)
return r->sgt.pfn + (r->sgt.curr >> PAGE_SHIFT);
}
-static int remap_sg(pte_t *pte, unsigned long addr, void *data)
+static int remap_sg(hw_pte_t *pte, unsigned long addr, void *data)
{
struct remap_pfn *r = data;
@@ -70,7 +70,7 @@ static int remap_sg(pte_t *pte, unsigned long addr, void *data)
#define EXPECTED_FLAGS (VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP)
#if IS_ENABLED(CONFIG_X86)
-static int remap_pfn(pte_t *pte, unsigned long addr, void *data)
+static int remap_pfn(hw_pte_t *pte, unsigned long addr, void *data)
{
struct remap_pfn *r = data;
diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
index 1dcc4675580ed..b013bcad99b5b 100644
--- a/drivers/xen/gntdev.c
+++ b/drivers/xen/gntdev.c
@@ -301,7 +301,7 @@ void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
/* ------------------------------------------------------------------ */
-static int find_grant_ptes(pte_t *pte, unsigned long addr, void *data)
+static int find_grant_ptes(hw_pte_t *pte, unsigned long addr, void *data)
{
struct gntdev_grant_map *map = data;
unsigned int pgnr = (addr - map->pages_vm_start) >> PAGE_SHIFT;
diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index 7cfc28f1bb86f..e724cc053e264 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -1658,7 +1658,7 @@ static int privcmd_mmap(struct file *file, struct vm_area_struct *vma)
* on a per pfn/pte basis. Mapping calls that fail with ENOENT
* can be then retried until success.
*/
-static int is_mapped_fn(pte_t *pte, unsigned long addr, void *data)
+static int is_mapped_fn(hw_pte_t *pte, unsigned long addr, void *data)
{
return pte_none(ptep_get(pte)) ? 0 : -EBUSY;
}
diff --git a/drivers/xen/xenbus/xenbus_client.c b/drivers/xen/xenbus/xenbus_client.c
index 27682cb5e58ab..387f97a555594 100644
--- a/drivers/xen/xenbus/xenbus_client.c
+++ b/drivers/xen/xenbus/xenbus_client.c
@@ -749,7 +749,7 @@ int xenbus_unmap_ring_vfree(struct xenbus_device *dev, void *vaddr)
EXPORT_SYMBOL_GPL(xenbus_unmap_ring_vfree);
#ifdef CONFIG_XEN_PV
-static int map_ring_apply(pte_t *pte, unsigned long addr, void *data)
+static int map_ring_apply(hw_pte_t *pte, unsigned long addr, void *data)
{
struct map_ring_valloc *info = data;
diff --git a/drivers/xen/xlate_mmu.c b/drivers/xen/xlate_mmu.c
index 8efd7b55223fb..d7244f9f9a545 100644
--- a/drivers/xen/xlate_mmu.c
+++ b/drivers/xen/xlate_mmu.c
@@ -93,7 +93,7 @@ static void setup_hparams(unsigned long gfn, void *data)
info->fgfn++;
}
-static int remap_pte_fn(pte_t *ptep, unsigned long addr, void *data)
+static int remap_pte_fn(hw_pte_t *ptep, unsigned long addr, void *data)
{
struct remap_data *info = data;
struct page *page = info->pages[info->index++];
@@ -269,7 +269,7 @@ struct remap_pfn {
unsigned long i;
};
-static int remap_pfn_fn(pte_t *ptep, unsigned long addr, void *data)
+static int remap_pfn_fn(hw_pte_t *ptep, unsigned long addr, void *data)
{
struct remap_pfn *r = data;
struct page *page = r->pages[r->i];
diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index 7611a8470ea26..ddaab714c3e02 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -328,7 +328,8 @@ static void hugetlb_delete_from_page_cache(struct folio *folio)
static bool hugetlb_vma_maps_pfn(struct vm_area_struct *vma,
unsigned long addr, unsigned long pfn)
{
- pte_t *ptep, pte;
+ hw_pte_t *ptep;
+ pte_t pte;
ptep = hugetlb_walk(vma, addr, huge_page_size(hstate_vma(vma)));
if (!ptep)
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index e671b4fd8dedd..4211c33b06491 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -982,7 +982,7 @@ static void smaps_pte_hole_lookup(unsigned long addr, struct mm_walk *walk)
#endif
}
-static void smaps_pte_entry(pte_t *pte, unsigned long addr,
+static void smaps_pte_entry(hw_pte_t *pte, unsigned long addr,
struct mm_walk *walk)
{
struct mem_size_stats *mss = walk->private;
@@ -1076,7 +1076,7 @@ static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
struct vm_area_struct *vma = walk->vma;
- pte_t *pte;
+ hw_pte_t *pte;
spinlock_t *ptl;
ptl = pmd_trans_huge_lock(pmd, vma);
@@ -1199,7 +1199,7 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma)
}
#ifdef CONFIG_HUGETLB_PAGE
-static int smaps_hugetlb_range(pte_t *pte, unsigned long hmask,
+static int smaps_hugetlb_range(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
@@ -1622,7 +1622,7 @@ static inline bool pte_is_pinned(struct vm_area_struct *vma, unsigned long addr,
}
static inline void clear_soft_dirty(struct vm_area_struct *vma,
- unsigned long addr, pte_t *pte)
+ unsigned long addr, hw_pte_t *pte)
{
if (!pgtable_supports_soft_dirty())
return;
@@ -1690,7 +1690,8 @@ static int clear_refs_pte_range(pmd_t *pmd, unsigned long addr,
{
struct clear_refs_private *cp = walk->private;
struct vm_area_struct *vma = walk->vma;
- pte_t *pte, ptent;
+ hw_pte_t *pte;
+ pte_t ptent;
spinlock_t *ptl;
struct folio *folio;
@@ -2090,7 +2091,7 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end,
struct vm_area_struct *vma = walk->vma;
struct pagemapread *pm = walk->private;
spinlock_t *ptl;
- pte_t *pte, *orig_pte;
+ hw_pte_t *pte, *orig_pte;
int err = 0;
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
@@ -2128,7 +2129,7 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end,
#ifdef CONFIG_HUGETLB_PAGE
/* This function walks within one hugetlb entry in the single call */
-static int pagemap_hugetlb_range(pte_t *ptep, unsigned long hmask,
+static int pagemap_hugetlb_range(hw_pte_t *ptep, unsigned long hmask,
unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
@@ -2419,7 +2420,7 @@ static unsigned long pagemap_page_category(struct pagemap_scan_private *p,
}
static void make_uffd_wp_pte(struct vm_area_struct *vma,
- unsigned long addr, pte_t *pte, pte_t ptent)
+ unsigned long addr, hw_pte_t *pte, pte_t ptent)
{
if (pte_present(ptent)) {
pte_t old_pte;
@@ -2552,7 +2553,7 @@ static unsigned long pagemap_hugetlb_category(struct vm_area_struct *vma,
}
static void make_uffd_wp_huge_pte(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep,
+ unsigned long addr, hw_pte_t *ptep,
pte_t ptent)
{
const unsigned long psize = huge_page_size(hstate_vma(vma));
@@ -2786,7 +2787,7 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start,
struct pagemap_scan_private *p = walk->private;
struct vm_area_struct *vma = walk->vma;
unsigned long addr, flush_end = 0;
- pte_t *pte, *start_pte;
+ hw_pte_t *pte, *start_pte;
spinlock_t *ptl;
int ret;
@@ -2880,7 +2881,7 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start,
}
#ifdef CONFIG_HUGETLB_PAGE
-static int pagemap_scan_hugetlb_entry(pte_t *ptep, unsigned long hmask,
+static int pagemap_scan_hugetlb_entry(hw_pte_t *ptep, unsigned long hmask,
unsigned long start, unsigned long end,
struct mm_walk *walk)
{
@@ -2961,7 +2962,7 @@ static int pagemap_scan_hugetlb_hole_wp(struct vm_area_struct *vma,
unsigned long psize = huge_page_size(h);
struct mm_struct *mm = vma->vm_mm;
spinlock_t *ptl;
- pte_t *ptep;
+ hw_pte_t *ptep;
pte_t pte;
for (addr = ALIGN_DOWN(addr, psize); addr < end; addr += psize) {
@@ -3345,8 +3346,8 @@ static int gather_pte_stats(pmd_t *pmd, unsigned long addr,
struct numa_maps *md = walk->private;
struct vm_area_struct *vma = walk->vma;
spinlock_t *ptl;
- pte_t *orig_pte;
- pte_t *pte;
+ hw_pte_t *orig_pte;
+ hw_pte_t *pte;
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
ptl = pmd_trans_huge_lock(pmd, vma);
@@ -3379,7 +3380,7 @@ static int gather_pte_stats(pmd_t *pmd, unsigned long addr,
return 0;
}
#ifdef CONFIG_HUGETLB_PAGE
-static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask,
+static int gather_hugetlb_stats(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long end, struct mm_walk *walk)
{
pte_t huge_pte;
@@ -3402,7 +3403,7 @@ static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask,
}
#else
-static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask,
+static int gather_hugetlb_stats(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long end, struct mm_walk *walk)
{
return 0;
diff --git a/include/asm-generic/hugetlb.h b/include/asm-generic/hugetlb.h
index 635c41cc34797..3bfad4a99b64d 100644
--- a/include/asm-generic/hugetlb.h
+++ b/include/asm-generic/hugetlb.h
@@ -60,7 +60,7 @@ static inline int huge_pte_uffd(pte_t pte)
#ifndef __HAVE_ARCH_HUGE_PTE_CLEAR
static inline void huge_pte_clear(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, unsigned long sz)
+ hw_pte_t *ptep, unsigned long sz)
{
pte_clear(mm, addr, ptep);
}
@@ -68,7 +68,7 @@ static inline void huge_pte_clear(struct mm_struct *mm, unsigned long addr,
#ifndef __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT
static inline void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t pte, unsigned long sz)
+ hw_pte_t *ptep, pte_t pte, unsigned long sz)
{
set_pte_at(mm, addr, ptep, pte);
}
@@ -76,7 +76,7 @@ static inline void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
#ifndef __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR
static inline pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep, unsigned long sz)
+ unsigned long addr, hw_pte_t *ptep, unsigned long sz)
{
return ptep_get_and_clear(mm, addr, ptep);
}
@@ -84,7 +84,7 @@ static inline pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
#ifndef __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH
static inline pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep)
+ unsigned long addr, hw_pte_t *ptep)
{
return ptep_clear_flush(vma, addr, ptep);
}
@@ -99,7 +99,7 @@ static inline int huge_pte_none(pte_t pte)
#ifndef __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT
static inline void huge_ptep_set_wrprotect(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep)
+ unsigned long addr, hw_pte_t *ptep)
{
ptep_set_wrprotect(mm, addr, ptep);
}
@@ -107,7 +107,7 @@ static inline void huge_ptep_set_wrprotect(struct mm_struct *mm,
#ifndef __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS
static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep,
+ unsigned long addr, hw_pte_t *ptep,
pte_t pte, int dirty)
{
return ptep_set_access_flags(vma, addr, ptep, pte, dirty);
@@ -115,7 +115,8 @@ static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma,
#endif
#ifndef __HAVE_ARCH_HUGE_PTEP_GET
-static inline pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep)
+static inline pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr,
+ hw_pte_t *ptep)
{
return ptep_get(ptep);
}
diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h
index 051aa1331051c..b35e5a2158ada 100644
--- a/include/asm-generic/pgalloc.h
+++ b/include/asm-generic/pgalloc.h
@@ -16,7 +16,7 @@
*
* Return: pointer to the allocated memory or %NULL on error
*/
-static inline pte_t *__pte_alloc_one_kernel_noprof(struct mm_struct *mm)
+static inline hw_pte_t *__pte_alloc_one_kernel_noprof(struct mm_struct *mm)
{
struct ptdesc *ptdesc = pagetable_alloc_noprof(GFP_PGTABLE_KERNEL, 0);
@@ -40,7 +40,7 @@ static inline pte_t *__pte_alloc_one_kernel_noprof(struct mm_struct *mm)
*
* Return: pointer to the allocated memory or %NULL on error
*/
-static inline pte_t *pte_alloc_one_kernel_noprof(struct mm_struct *mm)
+static inline hw_pte_t *pte_alloc_one_kernel_noprof(struct mm_struct *mm)
{
return __pte_alloc_one_kernel_noprof(mm);
}
@@ -52,7 +52,7 @@ static inline pte_t *pte_alloc_one_kernel_noprof(struct mm_struct *mm)
* @mm: the mm_struct of the current context
* @pte: pointer to the memory containing the page table
*/
-static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
+static inline void pte_free_kernel(struct mm_struct *mm, hw_pte_t *pte)
{
pagetable_dtor_free(virt_to_ptdesc(pte));
}
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index bdcc2778ac64f..2c3517800a9a8 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -644,7 +644,8 @@ static inline void tlb_flush_p4d_range(struct mmu_gather *tlb,
}
#ifndef __tlb_remove_tlb_entry
-static inline void __tlb_remove_tlb_entry(struct mmu_gather *tlb, pte_t *ptep, unsigned long address)
+static inline void __tlb_remove_tlb_entry(struct mmu_gather *tlb,
+ hw_pte_t *ptep, unsigned long address)
{
}
#endif
@@ -670,7 +671,7 @@ static inline void __tlb_remove_tlb_entry(struct mmu_gather *tlb, pte_t *ptep, u
* consecutive ptes instead of only a single one.
*/
static inline void tlb_remove_tlb_entries(struct mmu_gather *tlb,
- pte_t *ptep, unsigned int nr, unsigned long address)
+ hw_pte_t *ptep, unsigned int nr, unsigned long address)
{
tlb_flush_pte_range(tlb, address, PAGE_SIZE * nr);
for (;;) {
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 80a5a03e9cee7..6bbf2449e7d27 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -141,7 +141,7 @@ unsigned long hugetlb_total_pages(void);
vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma,
unsigned long address, unsigned int flags);
#ifdef CONFIG_USERFAULTFD
-int hugetlb_mfill_atomic_pte(pte_t *dst_pte,
+int hugetlb_mfill_atomic_pte(hw_pte_t *dst_pte,
struct vm_area_struct *dst_vma,
unsigned long dst_addr,
unsigned long src_addr,
@@ -161,7 +161,7 @@ void hugetlb_fix_reserve_counts(struct inode *inode);
extern struct mutex *hugetlb_fault_mutex_table;
u32 hugetlb_fault_mutex_hash(struct address_space *mapping, pgoff_t idx);
-pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,
+hw_pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,
unsigned long addr, pud_t *pud);
bool hugetlbfs_pagecache_present(struct hstate *h,
struct vm_area_struct *vma,
@@ -185,22 +185,22 @@ void hugetlb_bootmem_set_nodes(void);
* which may go down to the lowest PTE level in their huge_pte_offset() and
* huge_pte_alloc(): to avoid reliance on pte_offset_map() without pte_unmap().
*/
-static inline pte_t *pte_offset_huge(pmd_t *pmd, unsigned long address)
+static inline hw_pte_t *pte_offset_huge(pmd_t *pmd, unsigned long address)
{
return pte_offset_kernel(pmd, address);
}
-static inline pte_t *pte_alloc_huge(struct mm_struct *mm, pmd_t *pmd,
+static inline hw_pte_t *pte_alloc_huge(struct mm_struct *mm, pmd_t *pmd,
unsigned long address)
{
return pte_alloc(mm, pmd) ? NULL : pte_offset_huge(pmd, address);
}
#endif
-pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
+hw_pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
unsigned long addr, unsigned long sz);
/*
* huge_pte_offset(): Walk the hugetlb pgtable until the last level PTE.
- * Returns the pte_t* if found, or NULL if the address is not mapped.
+ * Returns the hw_pte_t* if found, or NULL if the address is not mapped.
*
* IMPORTANT: we should normally not directly call this function, instead
* this is only a common interface to implement arch-specific
@@ -235,11 +235,11 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
* a concurrent pmd unshare, but it makes sure the pgtable page is safe to
* access.
*/
-pte_t *huge_pte_offset(struct mm_struct *mm,
+hw_pte_t *huge_pte_offset(struct mm_struct *mm,
unsigned long addr, unsigned long sz);
unsigned long hugetlb_mask_last_page(struct hstate *h);
int huge_pmd_unshare(struct mmu_gather *tlb, struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep);
+ unsigned long addr, hw_pte_t *ptep);
void huge_pmd_unshare_flush(struct mmu_gather *tlb, struct vm_area_struct *vma);
void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma,
unsigned long *start, unsigned long *end);
@@ -301,7 +301,8 @@ static inline struct address_space *hugetlb_folio_mapping_lock_write(
}
static inline int huge_pmd_unshare(struct mmu_gather *tlb,
- struct vm_area_struct *vma, unsigned long addr, pte_t *ptep)
+ struct vm_area_struct *vma, unsigned long addr,
+ hw_pte_t *ptep)
{
return 0;
}
@@ -393,7 +394,7 @@ static inline int is_hugepage_only_range(struct mm_struct *mm,
}
#ifdef CONFIG_USERFAULTFD
-static inline int hugetlb_mfill_atomic_pte(pte_t *dst_pte,
+static inline int hugetlb_mfill_atomic_pte(hw_pte_t *dst_pte,
struct vm_area_struct *dst_vma,
unsigned long dst_addr,
unsigned long src_addr,
@@ -405,7 +406,7 @@ static inline int hugetlb_mfill_atomic_pte(pte_t *dst_pte,
}
#endif /* CONFIG_USERFAULTFD */
-static inline pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr,
+static inline hw_pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr,
unsigned long sz)
{
return NULL;
@@ -985,7 +986,8 @@ static inline bool htlb_allow_alloc_fallback(enum migrate_reason reason)
}
static inline spinlock_t *huge_pte_lockptr(struct hstate *h,
- struct mm_struct *mm, pte_t *pte)
+ struct mm_struct *mm,
+ hw_pte_t *pte)
{
const unsigned long size = huge_page_size(h);
@@ -1049,7 +1051,8 @@ static inline void hugetlb_count_sub(long l, struct mm_struct *mm)
#ifndef huge_ptep_modify_prot_start
#define huge_ptep_modify_prot_start huge_ptep_modify_prot_start
static inline pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep)
+ unsigned long addr,
+ hw_pte_t *ptep)
{
unsigned long psize = huge_page_size(hstate_vma(vma));
@@ -1060,7 +1063,8 @@ static inline pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma,
#ifndef huge_ptep_modify_prot_commit
#define huge_ptep_modify_prot_commit huge_ptep_modify_prot_commit
static inline void huge_ptep_modify_prot_commit(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep,
+ unsigned long addr,
+ hw_pte_t *ptep,
pte_t old_pte, pte_t pte)
{
unsigned long psize = huge_page_size(hstate_vma(vma));
@@ -1248,7 +1252,8 @@ static inline bool htlb_allow_alloc_fallback(enum migrate_reason reason)
}
static inline spinlock_t *huge_pte_lockptr(struct hstate *h,
- struct mm_struct *mm, pte_t *pte)
+ struct mm_struct *mm,
+ hw_pte_t *pte)
{
return &mm->page_table_lock;
}
@@ -1265,21 +1270,23 @@ static inline void hugetlb_count_sub(long l, struct mm_struct *mm)
{
}
-pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep);
+pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, hw_pte_t *ptep);
unsigned long huge_pte_dirty(pte_t pte);
static inline pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep)
+ unsigned long addr, hw_pte_t *ptep)
{
#ifdef CONFIG_MMU
return ptep_get(ptep);
#else
- return *ptep;
+ /* No attached PTE that requires ptep_get(). */
+ return __pte_from_hw(*ptep);
#endif
}
static inline void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t pte, unsigned long sz)
+ hw_pte_t *ptep, pte_t pte,
+ unsigned long sz)
{
}
@@ -1307,7 +1314,7 @@ static inline void hugetlb_bootmem_struct_page_init(void)
#endif /* CONFIG_HUGETLB_PAGE */
static inline spinlock_t *huge_pte_lock(struct hstate *h,
- struct mm_struct *mm, pte_t *pte)
+ struct mm_struct *mm, hw_pte_t *pte)
{
spinlock_t *ptl;
@@ -1325,12 +1332,12 @@ static inline __init void hugetlb_cma_reserve(void)
#endif
#ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING
-static inline bool hugetlb_pmd_shared(pte_t *pte)
+static inline bool hugetlb_pmd_shared(hw_pte_t *pte)
{
return ptdesc_pmd_is_shared(virt_to_ptdesc(pte));
}
#else
-static inline bool hugetlb_pmd_shared(pte_t *pte)
+static inline bool hugetlb_pmd_shared(hw_pte_t *pte)
{
return false;
}
@@ -1357,7 +1364,7 @@ bool __vma_private_lock(struct vm_area_struct *vma);
* Safe version of huge_pte_offset() to check the locks. See comments
* above huge_pte_offset().
*/
-static inline pte_t *
+static inline hw_pte_t *
hugetlb_walk(struct vm_area_struct *vma, unsigned long addr, unsigned long sz)
{
#if defined(CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING) && defined(CONFIG_LOCKDEP)
diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index bf233bde68c7e..ff41949c0ac08 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -51,7 +51,7 @@ typedef unsigned int __bitwise kasan_vmalloc_flags_t;
#endif
extern unsigned char kasan_early_shadow_page[PAGE_SIZE];
-extern pte_t kasan_early_shadow_pte[MAX_PTRS_PER_PTE + PTE_HWTABLE_PTRS];
+extern hw_pte_t kasan_early_shadow_pte[MAX_PTRS_PER_PTE + PTE_HWTABLE_PTRS];
extern pmd_t kasan_early_shadow_pmd[MAX_PTRS_PER_PMD];
extern pud_t kasan_early_shadow_pud[MAX_PTRS_PER_PUD];
extern p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D];
diff --git a/include/linux/mm.h b/include/linux/mm.h
index c49ef99b4413b..892b280fe9c0c 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -769,7 +769,7 @@ struct vm_fault {
* VM_FAULT_ERROR).
*/
/* These three entries are valid only while holding ptl lock */
- pte_t *pte; /* Pointer to pte entry matching
+ hw_pte_t *pte; /* Pointer to pte entry matching
* the 'address'. NULL if the page
* table hasn't been allocated.
*/
@@ -3231,7 +3231,7 @@ struct follow_pfnmap_args {
* The caller shouldn't touch any of these.
*/
spinlock_t *lock;
- pte_t *ptep;
+ hw_pte_t *ptep;
/**
* Outputs:
*
@@ -3568,7 +3568,7 @@ static inline pud_t pud_mkspecial(pud_t pud)
}
#endif /* CONFIG_ARCH_SUPPORTS_PUD_PFNMAP */
-extern pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,
+extern hw_pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,
spinlock_t **ptl);
#ifdef __PAGETABLE_P4D_FOLDED
@@ -3846,7 +3846,7 @@ static inline spinlock_t *pte_lockptr(struct mm_struct *mm, pmd_t *pmd)
return ptlock_ptr(page_ptdesc(pmd_page(*pmd)));
}
-static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, pte_t *pte)
+static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, hw_pte_t *pte)
{
BUILD_BUG_ON(IS_ENABLED(CONFIG_HIGHPTE));
BUILD_BUG_ON(MAX_PTRS_PER_PTE * sizeof(pte_t) > PAGE_SIZE);
@@ -3877,7 +3877,7 @@ static inline spinlock_t *pte_lockptr(struct mm_struct *mm, pmd_t *pmd)
{
return &mm->page_table_lock;
}
-static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, pte_t *pte)
+static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, hw_pte_t *pte)
{
return &mm->page_table_lock;
}
@@ -3918,19 +3918,19 @@ static inline bool pagetable_pte_ctor(struct mm_struct *mm,
return true;
}
-pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp);
+hw_pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp);
-static inline pte_t *pte_offset_map(pmd_t *pmd, unsigned long addr)
+static inline hw_pte_t *pte_offset_map(pmd_t *pmd, unsigned long addr)
{
return __pte_offset_map(pmd, addr, NULL);
}
-pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd,
+hw_pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, spinlock_t **ptlp);
-pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,
+hw_pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, spinlock_t **ptlp);
-pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,
+hw_pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, pmd_t *pmdvalp,
spinlock_t **ptlp);
@@ -4873,7 +4873,7 @@ static inline bool gup_can_follow_protnone(const struct vm_area_struct *vma,
return !vma_is_accessible(vma);
}
-typedef int (*pte_fn_t)(pte_t *pte, unsigned long addr, void *data);
+typedef int (*pte_fn_t)(hw_pte_t *pte, unsigned long addr, void *data);
extern int apply_to_page_range(struct mm_struct *mm, unsigned long address,
unsigned long size, pte_fn_t fn, void *data);
extern int apply_to_existing_page_range(struct mm_struct *mm,
@@ -5122,7 +5122,7 @@ void *vmemmap_alloc_block(unsigned long size, int node);
struct vmem_altmap;
void *vmemmap_alloc_block_buf(unsigned long size, int node,
struct vmem_altmap *altmap);
-void vmemmap_verify(pte_t *, int, unsigned long, unsigned long);
+void vmemmap_verify(hw_pte_t *, int, unsigned long, unsigned long);
void vmemmap_set_pmd(pmd_t *pmd, void *p, int node,
unsigned long addr, unsigned long next);
int vmemmap_check_pmd(pmd_t *pmd, int node,
@@ -5487,7 +5487,7 @@ static inline bool snapshot_page_is_faithful(const struct page_snapshot *ps)
void snapshot_page(struct page_snapshot *ps, const struct page *page);
-void map_anon_folio_pte_nopf(struct folio *folio, pte_t *pte,
+void map_anon_folio_pte_nopf(struct folio *folio, hw_pte_t *pte,
struct vm_area_struct *vma, unsigned long addr,
bool uffd_wp);
diff --git a/include/linux/page_table_check.h b/include/linux/page_table_check.h
index 12268a32e8be1..933c1028c2232 100644
--- a/include/linux/page_table_check.h
+++ b/include/linux/page_table_check.h
@@ -7,6 +7,8 @@
#ifndef __LINUX_PAGE_TABLE_CHECK_H
#define __LINUX_PAGE_TABLE_CHECK_H
+#include <linux/pgtable_types.h>
+
#ifdef CONFIG_PAGE_TABLE_CHECK
#include <linux/jump_label.h>
@@ -21,7 +23,7 @@ void __page_table_check_pmd_clear(struct mm_struct *mm, unsigned long addr,
void __page_table_check_pud_clear(struct mm_struct *mm, unsigned long addr,
pud_t pud);
void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t pte, unsigned int nr);
+ hw_pte_t *ptep, pte_t pte, unsigned int nr);
void __page_table_check_pmds_set(struct mm_struct *mm, unsigned long addr,
pmd_t *pmdp, pmd_t pmd, unsigned int nr);
void __page_table_check_puds_set(struct mm_struct *mm, unsigned long addr,
@@ -74,7 +76,8 @@ static inline void page_table_check_pud_clear(struct mm_struct *mm,
}
static inline void page_table_check_ptes_set(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep,
+ unsigned long addr,
+ hw_pte_t *ptep,
pte_t pte, unsigned int nr)
{
if (static_branch_likely(&page_table_check_disabled))
@@ -137,7 +140,8 @@ static inline void page_table_check_pud_clear(struct mm_struct *mm,
}
static inline void page_table_check_ptes_set(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep,
+ unsigned long addr,
+ hw_pte_t *ptep,
pte_t pte, unsigned int nr)
{
}
diff --git a/include/linux/pagewalk.h b/include/linux/pagewalk.h
index b41d7265c01bc..7ab2eb39e02c0 100644
--- a/include/linux/pagewalk.h
+++ b/include/linux/pagewalk.h
@@ -38,7 +38,7 @@ enum page_walk_lock {
* not trigger for any populated ranges.
* @hugetlb_entry: if set, called for each hugetlb entry. This hook
* function is called with the vma lock held, in order to
- * protect against a concurrent freeing of the pte_t* or
+ * protect against a concurrent freeing of the hw_pte_t* or
* the ptl. In some cases, the hook function needs to drop
* and retake the vma lock in order to avoid deadlocks
* while calling other functions. In such cases the hook
@@ -76,11 +76,11 @@ struct mm_walk_ops {
unsigned long next, struct mm_walk *walk);
int (*pmd_entry)(pmd_t *pmd, unsigned long addr,
unsigned long next, struct mm_walk *walk);
- int (*pte_entry)(pte_t *pte, unsigned long addr,
+ int (*pte_entry)(hw_pte_t *pte, unsigned long addr,
unsigned long next, struct mm_walk *walk);
int (*pte_hole)(unsigned long addr, unsigned long next,
int depth, struct mm_walk *walk);
- int (*hugetlb_entry)(pte_t *pte, unsigned long hmask,
+ int (*hugetlb_entry)(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long next,
struct mm_walk *walk);
int (*test_walk)(unsigned long addr, unsigned long next,
@@ -89,7 +89,7 @@ struct mm_walk_ops {
struct mm_walk *walk);
void (*post_vma)(struct mm_walk *walk);
int (*install_pte)(unsigned long addr, unsigned long next,
- pte_t *ptep, struct mm_walk *walk);
+ pte_t *ptentp, struct mm_walk *walk);
enum page_walk_lock walk_lock;
};
@@ -173,7 +173,7 @@ struct folio_walk {
struct page *page;
enum folio_walk_level level;
union {
- pte_t *ptep;
+ hw_pte_t *ptep;
pud_t *pudp;
pmd_t *pmdp;
};
diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
index e3c8ab96941c5..41337da0d65aa 100644
--- a/include/linux/pgtable.h
+++ b/include/linux/pgtable.h
@@ -4,6 +4,7 @@
#include <linux/pfn.h>
#include <asm/pgtable.h>
+#include <linux/pgtable_types.h>
#define PMD_ORDER (PMD_SHIFT - PAGE_SHIFT)
#define PUD_ORDER (PUD_SHIFT - PAGE_SHIFT)
@@ -93,26 +94,26 @@ static inline void pud_init(void *addr)
#endif
#ifndef pte_offset_kernel
-static inline pte_t *pte_offset_kernel(pmd_t *pmd, unsigned long address)
+static inline hw_pte_t *pte_offset_kernel(pmd_t *pmd, unsigned long address)
{
- return (pte_t *)pmd_page_vaddr(*pmd) + pte_index(address);
+ return (hw_pte_t *)pmd_page_vaddr(*pmd) + pte_index(address);
}
#define pte_offset_kernel pte_offset_kernel
#endif
#ifdef CONFIG_HIGHPTE
#define __pte_map(pmd, address) \
- ((pte_t *)kmap_local_page(pmd_page(*(pmd))) + pte_index((address)))
+ ((hw_pte_t *)kmap_local_page(pmd_page(*(pmd))) + pte_index((address)))
#define pte_unmap(pte) do { \
kunmap_local((pte)); \
rcu_read_unlock(); \
} while (0)
#else
-static inline pte_t *__pte_map(pmd_t *pmd, unsigned long address)
+static inline hw_pte_t *__pte_map(pmd_t *pmd, unsigned long address)
{
return pte_offset_kernel(pmd, address);
}
-static inline void pte_unmap(pte_t *pte)
+static inline void pte_unmap(hw_pte_t *pte)
{
rcu_read_unlock();
}
@@ -172,7 +173,7 @@ static inline pmd_t *pmd_off_k(unsigned long va)
return pmd_offset(pud_offset(p4d_offset(pgd_offset_k(va), va), va), va);
}
-static inline pte_t *virt_to_kpte(unsigned long vaddr)
+static inline hw_pte_t *virt_to_kpte(unsigned long vaddr)
{
pmd_t *pmd = pmd_off_k(vaddr);
@@ -407,7 +408,7 @@ static inline void lazy_mmu_mode_resume(void) {}
*
* May be overridden by the architecture, else pte_batch_hint is always 1.
*/
-static inline unsigned int pte_batch_hint(pte_t *ptep, pte_t pte)
+static inline unsigned int pte_batch_hint(hw_pte_t *ptep, pte_t pte)
{
return 1;
}
@@ -442,7 +443,7 @@ static inline pte_t pte_advance_pfn(pte_t pte, unsigned long nr)
* to the same folio. The PTEs are all in the same PMD.
*/
static inline void set_ptes(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t pte, unsigned int nr)
+ hw_pte_t *ptep, pte_t pte, unsigned int nr)
{
page_table_check_ptes_set(mm, addr, ptep, pte, nr);
@@ -459,7 +460,7 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr,
#ifndef __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS
extern int ptep_set_access_flags(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep,
+ unsigned long address, hw_pte_t *ptep,
pte_t entry, int dirty);
#endif
@@ -490,9 +491,9 @@ static inline int pudp_set_access_flags(struct vm_area_struct *vma,
#endif
#ifndef ptep_get
-static inline pte_t ptep_get(pte_t *ptep)
+static inline pte_t ptep_get(hw_pte_t *ptep)
{
- return READ_ONCE(*ptep);
+ return __pte_from_hw(READ_ONCE(*ptep));
}
#endif
@@ -526,7 +527,7 @@ static inline pgd_t pgdp_get(pgd_t *pgdp)
#ifndef __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG
static inline bool ptep_test_and_clear_young(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep)
+ unsigned long address, hw_pte_t *ptep)
{
pte_t pte = ptep_get(ptep);
bool young = true;
@@ -565,7 +566,7 @@ static inline bool pmdp_test_and_clear_young(struct vm_area_struct *vma,
#ifndef __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH
bool ptep_clear_flush_young(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep);
+ unsigned long address, hw_pte_t *ptep);
#endif
#ifndef __HAVE_ARCH_PMDP_CLEAR_YOUNG_FLUSH
@@ -644,7 +645,7 @@ static inline void arch_check_zapped_pud(struct vm_area_struct *vma, pud_t pud)
#ifndef __HAVE_ARCH_PTEP_GET_AND_CLEAR
static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
unsigned long address,
- pte_t *ptep)
+ hw_pte_t *ptep)
{
pte_t pte = ptep_get(ptep);
pte_clear(mm, address, ptep);
@@ -673,7 +674,7 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
* pages that belong to the same folio. The PTEs are all in the same PMD.
*/
static inline void clear_young_dirty_ptes(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep,
+ unsigned long addr, hw_pte_t *ptep,
unsigned int nr, cydp_t flags)
{
pte_t pte;
@@ -698,7 +699,7 @@ static inline void clear_young_dirty_ptes(struct vm_area_struct *vma,
#endif
static inline void ptep_clear(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep)
+ hw_pte_t *ptep)
{
pte_t pte = ptep_get(ptep);
@@ -739,7 +740,7 @@ static inline void ptep_clear(struct mm_struct *mm, unsigned long addr,
* present bit set *unless* it is 'l'. Because get_user_pages_fast() only
* operates on present ptes we're safe.
*/
-static inline pte_t ptep_get_lockless(pte_t *ptep)
+static inline pte_t ptep_get_lockless(hw_pte_t *ptep)
{
pte_t pte;
@@ -777,7 +778,7 @@ static inline pmd_t pmdp_get_lockless(pmd_t *pmdp)
* We require that the PTE can be read atomically.
*/
#ifndef ptep_get_lockless
-static inline pte_t ptep_get_lockless(pte_t *ptep)
+static inline pte_t ptep_get_lockless(hw_pte_t *ptep)
{
return ptep_get(ptep);
}
@@ -844,7 +845,8 @@ static inline pud_t pudp_huge_get_and_clear_full(struct vm_area_struct *vma,
#ifndef __HAVE_ARCH_PTEP_GET_AND_CLEAR_FULL
static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
- unsigned long address, pte_t *ptep,
+ unsigned long address,
+ hw_pte_t *ptep,
int full)
{
return ptep_get_and_clear(mm, address, ptep);
@@ -872,7 +874,7 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
* pages that belong to the same folio. The PTEs are all in the same PMD.
*/
static inline pte_t get_and_clear_full_ptes(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep, unsigned int nr, int full)
+ unsigned long addr, hw_pte_t *ptep, unsigned int nr, int full)
{
pte_t pte, tmp_pte;
@@ -908,7 +910,7 @@ static inline pte_t get_and_clear_full_ptes(struct mm_struct *mm,
* pages that belong to the same folio. The PTEs are all in the same PMD.
*/
static inline pte_t get_and_clear_ptes(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, unsigned int nr)
+ hw_pte_t *ptep, unsigned int nr)
{
return get_and_clear_full_ptes(mm, addr, ptep, nr, 0);
}
@@ -933,7 +935,7 @@ static inline pte_t get_and_clear_ptes(struct mm_struct *mm, unsigned long addr,
* pages that belong to the same folio. The PTEs are all in the same PMD.
*/
static inline void clear_full_ptes(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, unsigned int nr, int full)
+ hw_pte_t *ptep, unsigned int nr, int full)
{
for (;;) {
ptep_get_and_clear_full(mm, addr, ptep, full);
@@ -962,7 +964,7 @@ static inline void clear_full_ptes(struct mm_struct *mm, unsigned long addr,
* pages that belong to the same folio. The PTEs are all in the same PMD.
*/
static inline void clear_ptes(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, unsigned int nr)
+ hw_pte_t *ptep, unsigned int nr)
{
clear_full_ptes(mm, addr, ptep, nr, 0);
}
@@ -977,13 +979,14 @@ static inline void clear_ptes(struct mm_struct *mm, unsigned long addr,
*/
#ifndef update_mmu_tlb_range
static inline void update_mmu_tlb_range(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep, unsigned int nr)
+ unsigned long address, hw_pte_t *ptep,
+ unsigned int nr)
{
}
#endif
static inline void update_mmu_tlb(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep)
+ unsigned long address, hw_pte_t *ptep)
{
update_mmu_tlb_range(vma, address, ptep, 1);
}
@@ -1000,7 +1003,7 @@ static inline void update_mmu_tlb(struct vm_area_struct *vma,
* The PTEs are all in the same PMD.
*/
static inline void clear_nonpresent_ptes(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep, unsigned int nr)
+ unsigned long addr, hw_pte_t *ptep, unsigned int nr)
{
(void)addr;
@@ -1016,7 +1019,7 @@ static inline void clear_nonpresent_ptes(struct mm_struct *mm,
#ifndef __HAVE_ARCH_PTEP_CLEAR_FLUSH
extern pte_t ptep_clear_flush(struct vm_area_struct *vma,
unsigned long address,
- pte_t *ptep);
+ hw_pte_t *ptep);
#endif
#ifndef __HAVE_ARCH_PMDP_HUGE_CLEAR_FLUSH
@@ -1044,7 +1047,8 @@ static inline pmd_t pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma)
#ifndef __HAVE_ARCH_PTEP_SET_WRPROTECT
struct mm_struct;
-static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long address, pte_t *ptep)
+static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long address,
+ hw_pte_t *ptep)
{
pte_t old_pte = ptep_get(ptep);
set_pte_at(mm, address, ptep, pte_wrprotect(old_pte));
@@ -1070,7 +1074,7 @@ static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long addres
* ptep_try_set as an identity macro. The generic stub returns false, which is
* correct for callers that fall through to oops on failure.
*/
-static inline bool ptep_try_set(pte_t *ptep, pte_t new_pte)
+static inline bool ptep_try_set(hw_pte_t *ptep, pte_t new_pte)
{
return false;
}
@@ -1113,7 +1117,7 @@ static inline void flush_tlb_before_set(unsigned long addr)
* pages that belong to the same folio. The PTEs are all in the same PMD.
*/
static inline void wrprotect_ptes(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, unsigned int nr)
+ hw_pte_t *ptep, unsigned int nr)
{
for (;;) {
ptep_set_wrprotect(mm, addr, ptep);
@@ -1144,7 +1148,7 @@ static inline void wrprotect_ptes(struct mm_struct *mm, unsigned long addr,
* pages that belong to the same folio. The PTEs are all in the same PMD.
*/
static inline bool clear_flush_young_ptes(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, unsigned int nr)
+ unsigned long addr, hw_pte_t *ptep, unsigned int nr)
{
bool young = false;
@@ -1181,7 +1185,7 @@ static inline bool clear_flush_young_ptes(struct vm_area_struct *vma,
* Returns: whether any PTE was young.
*/
static inline bool test_and_clear_young_ptes(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, unsigned int nr)
+ unsigned long addr, hw_pte_t *ptep, unsigned int nr)
{
bool young = false;
@@ -1585,7 +1589,7 @@ static inline int pmd_none_or_clear_bad(pmd_t *pmd)
static inline pte_t __ptep_modify_prot_start(struct vm_area_struct *vma,
unsigned long addr,
- pte_t *ptep)
+ hw_pte_t *ptep)
{
/*
* Get the current pte state, but zero it out to make it
@@ -1597,7 +1601,7 @@ static inline pte_t __ptep_modify_prot_start(struct vm_area_struct *vma,
static inline void __ptep_modify_prot_commit(struct vm_area_struct *vma,
unsigned long addr,
- pte_t *ptep, pte_t pte)
+ hw_pte_t *ptep, pte_t pte)
{
/*
* The pte is non-present, so there's no hardware state to
@@ -1623,7 +1627,7 @@ static inline void __ptep_modify_prot_commit(struct vm_area_struct *vma,
*/
static inline pte_t ptep_modify_prot_start(struct vm_area_struct *vma,
unsigned long addr,
- pte_t *ptep)
+ hw_pte_t *ptep)
{
return __ptep_modify_prot_start(vma, addr, ptep);
}
@@ -1636,7 +1640,8 @@ static inline pte_t ptep_modify_prot_start(struct vm_area_struct *vma,
*/
static inline void ptep_modify_prot_commit(struct vm_area_struct *vma,
unsigned long addr,
- pte_t *ptep, pte_t old_pte, pte_t pte)
+ hw_pte_t *ptep, pte_t old_pte,
+ pte_t pte)
{
__ptep_modify_prot_commit(vma, addr, ptep, pte);
}
@@ -1667,7 +1672,7 @@ static inline void ptep_modify_prot_commit(struct vm_area_struct *vma,
*/
#ifndef modify_prot_start_ptes
static inline pte_t modify_prot_start_ptes(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, unsigned int nr)
+ unsigned long addr, hw_pte_t *ptep, unsigned int nr)
{
pte_t pte, tmp_pte;
@@ -1707,7 +1712,7 @@ static inline pte_t modify_prot_start_ptes(struct vm_area_struct *vma,
*/
#ifndef modify_prot_commit_ptes
static inline void modify_prot_commit_ptes(struct vm_area_struct *vma, unsigned long addr,
- pte_t *ptep, pte_t old_pte, pte_t pte, unsigned int nr)
+ hw_pte_t *ptep, pte_t old_pte, pte_t pte, unsigned int nr)
{
int i;
diff --git a/include/linux/pgtable_types.h b/include/linux/pgtable_types.h
new file mode 100644
index 0000000000000..d6c5a7548550b
--- /dev/null
+++ b/include/linux/pgtable_types.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_PGTABLE_TYPES_H
+#define _LINUX_PGTABLE_TYPES_H
+
+#include <asm/page.h>
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_ARCH_HAS_HW_PTE_T
+typedef struct __hw_pte_t { pte_t __pte; } hw_pte_t;
+#define __pte_from_hw(pte) ((pte).__pte)
+#else
+#define hw_pte_t pte_t
+#define __pte_from_hw(pte) (pte)
+#endif
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* _LINUX_PGTABLE_TYPES_H */
diff --git a/include/linux/rmap.h b/include/linux/rmap.h
index 0b332770abeed..d28a7fb6f2c0e 100644
--- a/include/linux/rmap.h
+++ b/include/linux/rmap.h
@@ -868,7 +868,7 @@ struct page_vma_mapped_walk {
struct vm_area_struct *vma;
unsigned long address;
pmd_t *pmd;
- pte_t *pte;
+ hw_pte_t *pte;
spinlock_t *ptl;
unsigned int flags;
bool pgoff_is_anon : 1;
diff --git a/include/linux/swapops.h b/include/linux/swapops.h
index e7d0d529f3e0f..40d2fd920558f 100644
--- a/include/linux/swapops.h
+++ b/include/linux/swapops.h
@@ -216,7 +216,8 @@ static inline swp_entry_t make_migration_entry_dirty(swp_entry_t entry)
extern void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd,
unsigned long address);
-extern void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr, pte_t *pte);
+extern void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr,
+ hw_pte_t *pte);
#else /* CONFIG_MIGRATION */
static inline swp_entry_t make_readable_migration_entry(pgoff_t offset)
{
@@ -236,7 +237,8 @@ static inline swp_entry_t make_writable_migration_entry(pgoff_t offset)
static inline void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd,
unsigned long address) { }
static inline void migration_entry_wait_huge(struct vm_area_struct *vma,
- unsigned long addr, pte_t *pte) { }
+ unsigned long addr,
+ hw_pte_t *pte) { }
static inline swp_entry_t make_migration_entry_young(swp_entry_t entry)
{
diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index aed121d729b01..666ff2b3741da 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -8,7 +8,7 @@
#include <linux/init.h>
#include <linux/list.h>
#include <linux/llist.h>
-#include <asm/page.h> /* pgprot_t */
+#include <linux/pgtable_types.h> /* pgprot_t, hw_pte_t */
#include <linux/rbtree.h>
#include <linux/overflow.h>
@@ -120,7 +120,7 @@ static inline unsigned long arch_vmap_pte_range_map_size(unsigned long addr, uns
#ifndef arch_vmap_pte_range_unmap_size
static inline unsigned long arch_vmap_pte_range_unmap_size(unsigned long addr,
- pte_t *ptep)
+ hw_pte_t *ptep)
{
return PAGE_SIZE;
}
diff --git a/include/trace/events/xen.h b/include/trace/events/xen.h
index ad384969e2cb2..1972d50b043a5 100644
--- a/include/trace/events/xen.h
+++ b/include/trace/events/xen.h
@@ -138,10 +138,10 @@ TRACE_EVENT(xen_mc_extend_args,
TRACE_DEFINE_SIZEOF(pteval_t);
TRACE_EVENT(xen_mmu_set_pte,
- TP_PROTO(pte_t *ptep, pte_t pteval),
+ TP_PROTO(hw_pte_t *ptep, pte_t pteval),
TP_ARGS(ptep, pteval),
TP_STRUCT__entry(
- __field(pte_t *, ptep)
+ __field(hw_pte_t *, ptep)
__field(pteval_t, pteval)
),
TP_fast_assign(__entry->ptep = ptep;
@@ -207,12 +207,12 @@ TRACE_EVENT(xen_mmu_set_p4d,
DECLARE_EVENT_CLASS(xen_mmu_ptep_modify_prot,
TP_PROTO(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t pteval),
+ hw_pte_t *ptep, pte_t pteval),
TP_ARGS(mm, addr, ptep, pteval),
TP_STRUCT__entry(
__field(struct mm_struct *, mm)
__field(unsigned long, addr)
- __field(pte_t *, ptep)
+ __field(hw_pte_t *, ptep)
__field(pteval_t, pteval)
),
TP_fast_assign(__entry->mm = mm;
@@ -227,7 +227,7 @@ DECLARE_EVENT_CLASS(xen_mmu_ptep_modify_prot,
#define DEFINE_XEN_MMU_PTEP_MODIFY_PROT(name) \
DEFINE_EVENT(xen_mmu_ptep_modify_prot, name, \
TP_PROTO(struct mm_struct *mm, unsigned long addr, \
- pte_t *ptep, pte_t pteval), \
+ hw_pte_t *ptep, pte_t pteval), \
TP_ARGS(mm, addr, ptep, pteval))
DEFINE_XEN_MMU_PTEP_MODIFY_PROT(xen_mmu_ptep_modify_prot_start);
diff --git a/kernel/bpf/arena.c b/kernel/bpf/arena.c
index 7b6847200b431..62005229b2e94 100644
--- a/kernel/bpf/arena.c
+++ b/kernel/bpf/arena.c
@@ -155,7 +155,7 @@ struct clear_range_data {
struct llist_head *free_pages;
};
-static int apply_range_set_cb(pte_t *pte, unsigned long addr, void *data)
+static int apply_range_set_cb(hw_pte_t *pte, unsigned long addr, void *data)
{
struct apply_range_data *d = data;
struct page *page;
@@ -207,7 +207,7 @@ static void flush_vmap_cache(unsigned long start, unsigned long size)
flush_cache_vmap(start, start + size);
}
-static int apply_range_clear_cb(pte_t *pte, unsigned long addr, void *data)
+static int apply_range_clear_cb(hw_pte_t *pte, unsigned long addr, void *data)
{
struct clear_range_data *d = data;
pte_t old_pte;
@@ -238,7 +238,8 @@ static int apply_range_clear_cb(pte_t *pte, unsigned long addr, void *data)
return 0;
}
-static int apply_range_set_scratch_cb(pte_t *pte, unsigned long addr, void *data)
+static int apply_range_set_scratch_cb(hw_pte_t *pte, unsigned long addr,
+ void *data)
{
struct page *scratch_page = data;
@@ -340,7 +341,7 @@ static struct bpf_map *arena_map_alloc(union bpf_attr *attr)
return ERR_PTR(err);
}
-static int existing_page_cb(pte_t *ptep, unsigned long addr, void *data)
+static int existing_page_cb(hw_pte_t *ptep, unsigned long addr, void *data)
{
struct bpf_arena *arena = data;
struct page *page;
diff --git a/kernel/events/core.c b/kernel/events/core.c
index a6c8e38a31104..0369f6c95ba7d 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -8517,7 +8517,8 @@ static u64 perf_get_pgtable_size(struct mm_struct *mm, unsigned long addr)
p4d_t *p4dp, p4d;
pud_t *pudp, pud;
pmd_t *pmdp, pmd;
- pte_t *ptep, pte;
+ hw_pte_t *ptep;
+ pte_t pte;
pgdp = pgd_offset(mm, addr);
pgd = pgdp_get(pgdp);
diff --git a/mm/Kconfig b/mm/Kconfig
index c1ddf59c0d71a..5f462ec6fa5e5 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -1312,6 +1312,9 @@ comment "GUP_TEST needs to have DEBUG_FS enabled"
config GUP_GET_PXX_LOW_HIGH
bool
+config ARCH_HAS_HW_PTE_T
+ bool
+
config DMAPOOL_TEST
tristate "Enable a module to run time tests on dma_pool"
depends on HAS_DMA
diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c
index 7a2e40bc7baed..f844a44752a81 100644
--- a/mm/damon/ops-common.c
+++ b/mm/damon/ops-common.c
@@ -39,7 +39,7 @@ struct folio *damon_get_folio(unsigned long pfn)
return folio;
}
-void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr)
+void damon_ptep_mkold(hw_pte_t *pte, struct vm_area_struct *vma, unsigned long addr)
{
pte_t pteval = ptep_get(pte);
struct folio *folio;
diff --git a/mm/damon/ops-common.h b/mm/damon/ops-common.h
index 38d295488fa18..34b7e5715fcf9 100644
--- a/mm/damon/ops-common.h
+++ b/mm/damon/ops-common.h
@@ -7,7 +7,7 @@
struct folio *damon_get_folio(unsigned long pfn);
-void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr);
+void damon_ptep_mkold(hw_pte_t *pte, struct vm_area_struct *vma, unsigned long addr);
void damon_pmdp_mkold(pmd_t *pmd, struct vm_area_struct *vma, unsigned long addr);
void damon_folio_mkold(struct folio *folio);
bool damon_folio_young(struct folio *folio);
diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
index f884d3f78f30a..482e905c973a6 100644
--- a/mm/damon/vaddr.c
+++ b/mm/damon/vaddr.c
@@ -258,7 +258,7 @@ static void damon_va_walk_page_range(struct mm_struct *mm, unsigned long start,
static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned long addr,
unsigned long next, struct mm_walk *walk)
{
- pte_t *pte;
+ hw_pte_t *pte;
spinlock_t *ptl;
ptl = pmd_trans_huge_lock(pmd, walk->vma);
@@ -283,7 +283,7 @@ static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned long addr,
}
#ifdef CONFIG_HUGETLB_PAGE
-static void damon_hugetlb_mkold(pte_t *pte, struct mm_struct *mm,
+static void damon_hugetlb_mkold(hw_pte_t *pte, struct mm_struct *mm,
struct vm_area_struct *vma, unsigned long addr)
{
bool referenced = false;
@@ -310,7 +310,7 @@ static void damon_hugetlb_mkold(pte_t *pte, struct mm_struct *mm,
folio_put(folio);
}
-static int damon_mkold_hugetlb_entry(pte_t *pte, unsigned long hmask,
+static int damon_mkold_hugetlb_entry(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
@@ -379,7 +379,7 @@ struct damon_young_walk_private {
static int damon_young_pmd_entry(pmd_t *pmd, unsigned long addr,
unsigned long next, struct mm_walk *walk)
{
- pte_t *pte;
+ hw_pte_t *pte;
pte_t ptent;
spinlock_t *ptl;
struct folio *folio;
@@ -423,7 +423,7 @@ static int damon_young_pmd_entry(pmd_t *pmd, unsigned long addr,
}
#ifdef CONFIG_HUGETLB_PAGE
-static int damon_young_hugetlb_entry(pte_t *pte, unsigned long hmask,
+static int damon_young_hugetlb_entry(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
@@ -512,7 +512,7 @@ static unsigned int damon_va_check_accesses(struct damon_ctx *ctx)
static bool damos_va_filter_young_match(struct damos_filter *filter,
struct folio *folio, struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, pmd_t *pmdp)
+ unsigned long addr, hw_pte_t *ptep, pmd_t *pmdp)
{
bool young = false;
@@ -534,7 +534,7 @@ static bool damos_va_filter_young_match(struct damos_filter *filter,
static bool damos_va_filter_out(struct damos *scheme, struct folio *folio,
struct vm_area_struct *vma, unsigned long addr,
- pte_t *ptep, pmd_t *pmdp)
+ hw_pte_t *ptep, pmd_t *pmdp)
{
struct damos_filter *filter;
bool matched;
@@ -631,7 +631,8 @@ static int damos_va_migrate_pmd_entry(pmd_t *pmd, unsigned long addr,
struct damos_migrate_dests *dests = &s->migrate_dests;
struct folio *folio;
spinlock_t *ptl;
- pte_t *start_pte, *pte, ptent;
+ hw_pte_t *start_pte, *pte;
+ pte_t ptent;
int nr;
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
@@ -794,7 +795,8 @@ static int damos_va_stat_pmd_entry(pmd_t *pmd, unsigned long addr,
struct vm_area_struct *vma = walk->vma;
struct folio *folio;
spinlock_t *ptl;
- pte_t *start_pte, *pte, ptent;
+ hw_pte_t *start_pte, *pte;
+ pte_t ptent;
int nr;
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
index 2875fd22d7bb0..54e194d2a5854 100644
--- a/mm/debug_vm_pgtable.c
+++ b/mm/debug_vm_pgtable.c
@@ -50,7 +50,7 @@ struct pgtable_debug_args {
p4d_t *p4dp;
pud_t *pudp;
pmd_t *pmdp;
- pte_t *ptep;
+ hw_pte_t *ptep;
p4d_t *start_p4dp;
pud_t *start_pudp;
diff --git a/mm/filemap.c b/mm/filemap.c
index 00fd89cf6f550..98f89dcc55603 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -3490,7 +3490,7 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf)
{
struct vm_area_struct *vma = vmf->vma;
vm_fault_t ret = 0;
- pte_t *ptep;
+ hw_pte_t *ptep;
/*
* We might have COW'ed a pagecache folio and might now have an mlocked
@@ -3796,7 +3796,7 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf,
vm_fault_t ret = 0;
struct page *page = folio_page(folio, start);
unsigned int count = 0;
- pte_t *old_ptep = vmf->pte;
+ hw_pte_t *old_ptep = vmf->pte;
unsigned long addr0;
/*
diff --git a/mm/gup.c b/mm/gup.c
index d1ba59da94e9a..549f462a23b2f 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -761,7 +761,7 @@ static struct page *follow_huge_pmd(struct vm_area_struct *vma,
#endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */
static int follow_pfn_pte(struct vm_area_struct *vma, unsigned long address,
- pte_t *pte, unsigned int flags)
+ hw_pte_t *pte, unsigned int flags)
{
if (flags & FOLL_TOUCH) {
pte_t orig_entry = ptep_get(pte);
@@ -806,7 +806,8 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
struct folio *folio;
struct page *page;
spinlock_t *ptl;
- pte_t *ptep, pte;
+ hw_pte_t *ptep;
+ pte_t pte;
int ret;
ptep = pte_offset_map_lock(mm, pmd, address, &ptl);
@@ -1035,7 +1036,7 @@ static int get_gate_page(struct mm_struct *mm, unsigned long address,
p4d_t *p4d;
pud_t *pud;
pmd_t *pmd;
- pte_t *pte;
+ hw_pte_t *pte;
pte_t entry;
int ret = -EFAULT;
@@ -2838,7 +2839,7 @@ static unsigned long gup_fast_pte_range(pmd_t pmd, pmd_t *pmdp,
unsigned int flags, struct page **pages)
{
unsigned long nr_pages = 0;
- pte_t *ptep, *ptem;
+ hw_pte_t *ptep, *ptem;
ptem = ptep = pte_offset_map(&pmd, addr);
if (!ptep)
diff --git a/mm/highmem.c b/mm/highmem.c
index a33e411839517..87f94cac6106b 100644
--- a/mm/highmem.c
+++ b/mm/highmem.c
@@ -141,7 +141,7 @@ EXPORT_SYMBOL(__totalhigh_pages);
static int pkmap_count[LAST_PKMAP];
static __cacheline_aligned_in_smp DEFINE_SPINLOCK(kmap_lock);
-pte_t *pkmap_page_table;
+hw_pte_t *pkmap_page_table;
/*
* Most architectures have no use for kmap_high_get(), so let's abstract
@@ -532,9 +532,9 @@ static inline bool kmap_high_unmap_local(unsigned long vaddr)
return false;
}
-static pte_t *__kmap_pte;
+static hw_pte_t *__kmap_pte;
-static pte_t *kmap_get_pte(unsigned long vaddr, int idx)
+static hw_pte_t *kmap_get_pte(unsigned long vaddr, int idx)
{
if (IS_ENABLED(CONFIG_KMAP_LOCAL_NON_LINEAR_PTE_ARRAY))
/*
@@ -549,8 +549,9 @@ static pte_t *kmap_get_pte(unsigned long vaddr, int idx)
void *__kmap_local_pfn_prot(unsigned long pfn, pgprot_t prot)
{
- pte_t pteval, *kmap_pte;
unsigned long vaddr;
+ hw_pte_t *kmap_pte;
+ pte_t pteval;
int idx;
/*
@@ -597,7 +598,7 @@ EXPORT_SYMBOL(__kmap_local_page_prot);
void kunmap_local_indexed(const void *vaddr)
{
unsigned long addr = (unsigned long) vaddr & PAGE_MASK;
- pte_t *kmap_pte;
+ hw_pte_t *kmap_pte;
int idx;
if (addr < __fix_to_virt(FIX_KMAP_END) ||
@@ -646,7 +647,7 @@ EXPORT_SYMBOL(kunmap_local_indexed);
void __kmap_local_sched_out(void)
{
struct task_struct *tsk = current;
- pte_t *kmap_pte;
+ hw_pte_t *kmap_pte;
int i;
/* Clear kmaps */
@@ -683,7 +684,7 @@ void __kmap_local_sched_out(void)
void __kmap_local_sched_in(void)
{
struct task_struct *tsk = current;
- pte_t *kmap_pte;
+ hw_pte_t *kmap_pte;
int i;
/* Restore kmaps */
diff --git a/mm/hmm.c b/mm/hmm.c
index 2f1e98c6b6440..0c959611eda01 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -240,7 +240,7 @@ static inline unsigned long pte_to_hmm_pfn_flags(struct hmm_range *range,
}
static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr,
- unsigned long end, pmd_t *pmdp, pte_t *ptep,
+ unsigned long end, pmd_t *pmdp, hw_pte_t *ptep,
unsigned long *hmm_pfn)
{
struct hmm_vma_walk *hmm_vma_walk = walk->private;
@@ -411,7 +411,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp,
&range->hmm_pfns[(start - range->start) >> PAGE_SHIFT];
unsigned long npages = (end - start) >> PAGE_SHIFT;
unsigned long addr = start;
- pte_t *ptep;
+ hw_pte_t *ptep;
pmd_t pmd;
again:
@@ -547,7 +547,7 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end,
#endif
#ifdef CONFIG_HUGETLB_PAGE
-static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask,
+static int hmm_vma_walk_hugetlb_entry(hw_pte_t *pte, unsigned long hmask,
unsigned long start, unsigned long end,
struct mm_walk *walk)
{
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index dd66c6ad5af13..629c773ed313a 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -3149,7 +3149,7 @@ static void __split_huge_zero_page_pmd(struct vm_area_struct *vma,
pgtable_t pgtable;
pmd_t _pmd, old_pmd;
unsigned long addr;
- pte_t *pte;
+ hw_pte_t *pte;
int i;
/*
@@ -3199,7 +3199,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,
bool soft_dirty, uffd_wp = false, young = false, write = false;
bool anon_exclusive = false, dirty = false;
unsigned long addr;
- pte_t *pte;
+ hw_pte_t *pte;
int i;
VM_BUG_ON(haddr & ~HPAGE_PMD_MASK);
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index ceb29adc6dc19..687d87801979d 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -129,7 +129,7 @@ static void hugetlb_vma_lock_free(struct vm_area_struct *vma);
static void hugetlb_vma_lock_alloc(struct vm_area_struct *vma);
static void __hugetlb_vma_unlock_write_free(struct vm_area_struct *vma);
static int __huge_pmd_unshare(struct mmu_gather *tlb,
- struct vm_area_struct *vma, unsigned long addr, pte_t *ptep,
+ struct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,
bool check_locks);
static void hugetlb_unshare_pmds(struct vm_area_struct *vma,
unsigned long start, unsigned long end, bool take_locks);
@@ -4891,7 +4891,7 @@ static pte_t make_huge_pte(struct vm_area_struct *vma, struct folio *folio,
}
static void set_huge_ptep_writable(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep)
+ unsigned long address, hw_pte_t *ptep)
{
pte_t entry;
@@ -4901,14 +4901,14 @@ static void set_huge_ptep_writable(struct vm_area_struct *vma,
}
static void set_huge_ptep_maybe_writable(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep)
+ unsigned long address, hw_pte_t *ptep)
{
if (vma->vm_flags & VM_WRITE)
set_huge_ptep_writable(vma, address, ptep);
}
static void
-hugetlb_install_folio(struct vm_area_struct *vma, pte_t *ptep, unsigned long addr,
+hugetlb_install_folio(struct vm_area_struct *vma, hw_pte_t *ptep, unsigned long addr,
struct folio *new_folio, pte_t old, unsigned long sz)
{
pte_t newpte = make_huge_pte(vma, new_folio, true);
@@ -4934,7 +4934,8 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
struct vm_area_struct *dst_vma,
struct vm_area_struct *src_vma)
{
- pte_t *src_pte, *dst_pte, entry;
+ hw_pte_t *src_pte, *dst_pte;
+ pte_t entry;
struct folio *pte_folio;
unsigned long addr;
bool cow = vma_is_cow_mapping(src_vma);
@@ -5126,7 +5127,8 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
}
static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr,
- unsigned long new_addr, pte_t *src_pte, pte_t *dst_pte,
+ unsigned long new_addr, hw_pte_t *src_pte,
+ hw_pte_t *dst_pte,
unsigned long sz)
{
bool need_clear_uffd_wp = vma_has_uffd_without_event_remap(vma);
@@ -5188,7 +5190,7 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma,
struct mm_struct *mm = vma->vm_mm;
unsigned long old_end = old_addr + len;
unsigned long last_addr_mask;
- pte_t *src_pte, *dst_pte;
+ hw_pte_t *src_pte, *dst_pte;
struct mmu_notifier_range range;
struct mmu_gather tlb;
@@ -5249,7 +5251,7 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
struct mm_struct *mm = vma->vm_mm;
const bool folio_provided = !!folio;
unsigned long address;
- pte_t *ptep;
+ hw_pte_t *ptep;
pte_t pte;
spinlock_t *ptl;
struct hstate *h = hstate_vma(vma);
@@ -5783,7 +5785,7 @@ static inline vm_fault_t hugetlb_handle_userfault(struct vm_fault *vmf,
* false if pte changed or is changing.
*/
static bool hugetlb_pte_stable(struct hstate *h, struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t old_pte)
+ hw_pte_t *ptep, pte_t old_pte)
{
spinlock_t *ptl;
bool same;
@@ -6308,7 +6310,7 @@ static struct folio *alloc_hugetlb_folio_vma(struct hstate *h,
* Used by userfaultfd UFFDIO_* ioctls. Based on userfaultfd's mfill_atomic_pte
* with modifications for hugetlb pages.
*/
-int hugetlb_mfill_atomic_pte(pte_t *dst_pte,
+int hugetlb_mfill_atomic_pte(hw_pte_t *dst_pte,
struct vm_area_struct *dst_vma,
unsigned long dst_addr,
unsigned long src_addr,
@@ -6367,7 +6369,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,
folio = alloc_hugetlb_folio(dst_vma, dst_addr, false);
if (IS_ERR(folio)) {
- pte_t *actual_pte = hugetlb_walk(dst_vma, dst_addr, PMD_SIZE);
+ hw_pte_t *actual_pte = hugetlb_walk(dst_vma, dst_addr, PMD_SIZE);
if (actual_pte) {
ret = -EEXIST;
goto out;
@@ -6535,7 +6537,7 @@ long hugetlb_change_protection(struct vm_area_struct *vma,
{
struct mm_struct *mm = vma->vm_mm;
unsigned long start = address;
- pte_t *ptep;
+ hw_pte_t *ptep;
pte_t pte;
struct hstate *h = hstate_vma(vma);
long pages = 0, psize = huge_page_size(h);
@@ -7020,15 +7022,15 @@ void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma,
* racing tasks could either miss the sharing (see huge_pte_offset) or select a
* bad pmd for sharing.
*/
-pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,
+hw_pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,
unsigned long addr, pud_t *pud)
{
struct address_space *mapping = vma->vm_file->f_mapping;
const pgoff_t idx = linear_page_index(vma, addr);
struct vm_area_struct *svma;
unsigned long saddr;
- pte_t *spte = NULL;
- pte_t *pte;
+ hw_pte_t *spte = NULL;
+ hw_pte_t *pte;
i_mmap_lock_read(mapping);
mapping_rmap_tree_foreach(svma, mapping, idx, idx) {
@@ -7059,13 +7061,13 @@ pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,
}
spin_unlock(&mm->page_table_lock);
out:
- pte = (pte_t *)pmd_alloc(mm, pud, addr);
+ pte = (hw_pte_t *)pmd_alloc(mm, pud, addr);
i_mmap_unlock_read(mapping);
return pte;
}
static int __huge_pmd_unshare(struct mmu_gather *tlb,
- struct vm_area_struct *vma, unsigned long addr, pte_t *ptep,
+ struct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,
bool check_locks)
{
unsigned long sz = huge_page_size(hstate_vma(vma));
@@ -7106,7 +7108,7 @@ static int __huge_pmd_unshare(struct mmu_gather *tlb,
* was not a shared PMD table.
*/
int huge_pmd_unshare(struct mmu_gather *tlb, struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep)
+ unsigned long addr, hw_pte_t *ptep)
{
return __huge_pmd_unshare(tlb, vma, addr, ptep, /*check_locks=*/true);
}
@@ -7136,21 +7138,21 @@ void huge_pmd_unshare_flush(struct mmu_gather *tlb, struct vm_area_struct *vma)
#else /* !CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING */
-pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,
+hw_pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,
unsigned long addr, pud_t *pud)
{
return NULL;
}
static int __huge_pmd_unshare(struct mmu_gather *tlb,
- struct vm_area_struct *vma, unsigned long addr, pte_t *ptep,
+ struct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,
bool check_locks)
{
return 0;
}
int huge_pmd_unshare(struct mmu_gather *tlb, struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep)
+ unsigned long addr, hw_pte_t *ptep)
{
return 0;
}
@@ -7171,13 +7173,13 @@ bool want_pmd_share(struct vm_area_struct *vma, unsigned long addr)
#endif /* CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING */
#ifdef CONFIG_ARCH_WANT_GENERAL_HUGETLB
-pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
+hw_pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
unsigned long addr, unsigned long sz)
{
pgd_t *pgd;
p4d_t *p4d;
pud_t *pud;
- pte_t *pte = NULL;
+ hw_pte_t *pte = NULL;
pgd = pgd_offset(mm, addr);
p4d = p4d_alloc(mm, pgd, addr);
@@ -7186,13 +7188,13 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
pud = pud_alloc(mm, p4d, addr);
if (pud) {
if (sz == PUD_SIZE) {
- pte = (pte_t *)pud;
+ pte = (hw_pte_t *)pud;
} else {
BUG_ON(sz != PMD_SIZE);
if (want_pmd_share(vma, addr) && pud_none(*pud))
pte = huge_pmd_share(mm, vma, addr, pud);
else
- pte = (pte_t *)pmd_alloc(mm, pud, addr);
+ pte = (hw_pte_t *)pmd_alloc(mm, pud, addr);
}
}
@@ -7214,7 +7216,7 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
* size @sz doesn't match the hugepage size at this level of the page
* table.
*/
-pte_t *huge_pte_offset(struct mm_struct *mm,
+hw_pte_t *huge_pte_offset(struct mm_struct *mm,
unsigned long addr, unsigned long sz)
{
pgd_t *pgd;
@@ -7232,14 +7234,14 @@ pte_t *huge_pte_offset(struct mm_struct *mm,
pud = pud_offset(p4d, addr);
if (sz == PUD_SIZE)
/* must be pud huge, non-present or none */
- return (pte_t *)pud;
+ return (hw_pte_t *)pud;
if (!pud_present(*pud))
return NULL;
/* must have a valid entry and size to go further */
pmd = pmd_offset(pud, addr);
/* must be pmd huge, non-present or none */
- return (pte_t *)pmd;
+ return (hw_pte_t *)pmd;
}
/*
@@ -7418,7 +7420,7 @@ static void hugetlb_unshare_pmds(struct vm_area_struct *vma,
struct mmu_gather tlb;
unsigned long address;
spinlock_t *ptl;
- pte_t *ptep;
+ hw_pte_t *ptep;
if (!(vma->vm_flags & VM_MAYSHARE))
return;
diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c
index eb339c4a71f40..bf85d82c29b96 100644
--- a/mm/hugetlb_vmemmap.c
+++ b/mm/hugetlb_vmemmap.c
@@ -34,7 +34,7 @@
* operations.
*/
struct vmemmap_remap_walk {
- void (*remap_pte)(pte_t *pte, unsigned long addr,
+ void (*remap_pte)(hw_pte_t *pte, unsigned long addr,
struct vmemmap_remap_walk *walk);
unsigned long nr_walked;
@@ -56,7 +56,7 @@ static int vmemmap_split_pmd(pmd_t *pmd, struct page *head, unsigned long start,
pmd_t __pmd;
int i;
unsigned long addr = start;
- pte_t *pgtable;
+ hw_pte_t *pgtable;
pgtable = pte_alloc_one_kernel(&init_mm);
if (!pgtable)
@@ -65,7 +65,8 @@ static int vmemmap_split_pmd(pmd_t *pmd, struct page *head, unsigned long start,
pmd_populate_kernel(&init_mm, &__pmd, pgtable);
for (i = 0; i < PTRS_PER_PTE; i++, addr += PAGE_SIZE) {
- pte_t entry, *pte;
+ pte_t entry;
+ hw_pte_t *pte;
pgprot_t pgprot = PAGE_KERNEL;
entry = mk_pte(head + i, pgprot);
@@ -137,7 +138,7 @@ static int vmemmap_pmd_entry(pmd_t *pmd, unsigned long addr,
return vmemmap_split_pmd(pmd, head, addr & PMD_MASK, vmemmap_walk);
}
-static int vmemmap_pte_entry(pte_t *pte, unsigned long addr,
+static int vmemmap_pte_entry(hw_pte_t *pte, unsigned long addr,
unsigned long next, struct mm_walk *walk)
{
struct vmemmap_remap_walk *vmemmap_walk = walk->private;
@@ -199,7 +200,7 @@ static void free_vmemmap_page_list(struct list_head *list)
free_vmemmap_page(page);
}
-static void vmemmap_remap_pte(pte_t *pte, unsigned long addr,
+static void vmemmap_remap_pte(hw_pte_t *pte, unsigned long addr,
struct vmemmap_remap_walk *walk)
{
struct page *page = pte_page(ptep_get(pte));
@@ -233,7 +234,7 @@ static void vmemmap_remap_pte(pte_t *pte, unsigned long addr,
set_pte_at(&init_mm, addr, pte, entry);
}
-static void vmemmap_restore_pte(pte_t *pte, unsigned long addr,
+static void vmemmap_restore_pte(hw_pte_t *pte, unsigned long addr,
struct vmemmap_remap_walk *walk)
{
struct page *src = pte_page(ptep_get(pte)), *dst;
diff --git a/mm/internal.h b/mm/internal.h
index da14c56fb24e1..8c14f0cf07878 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -266,7 +266,7 @@ void unmap_vmas(struct mmu_gather *tlb, struct unmap_desc *unmap);
#ifdef CONFIG_MMU
bool cond_install_uffd_wp_ptes(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, pte_t pte,
+ unsigned long addr, hw_pte_t *ptep, pte_t pte,
unsigned long nr_ptes);
static inline void get_anon_vma(struct anon_vma *anon_vma)
@@ -397,7 +397,7 @@ static inline pte_t __pte_batch_clear_ignored(pte_t pte, fpb_t flags)
* Return: the number of table entries in the batch.
*/
static inline unsigned int folio_pte_batch_flags(struct folio *folio,
- struct vm_area_struct *vma, pte_t *ptep, pte_t *ptentp,
+ struct vm_area_struct *vma, hw_pte_t *ptep, pte_t *ptentp,
unsigned int max_nr, fpb_t flags)
{
bool any_writable = false, any_young = false, any_dirty = false;
@@ -451,7 +451,7 @@ static inline unsigned int folio_pte_batch_flags(struct folio *folio,
return min(nr, max_nr);
}
-unsigned int folio_pte_batch(struct folio *folio, pte_t *ptep, pte_t pte,
+unsigned int folio_pte_batch(struct folio *folio, hw_pte_t *ptep, pte_t pte,
unsigned int max_nr);
/**
@@ -508,11 +508,11 @@ static inline pte_t pte_next_swp_offset(pte_t pte)
*
* Return: the number of table entries in the batch.
*/
-static inline int swap_pte_batch(pte_t *start_ptep, int max_nr, pte_t pte)
+static inline int swap_pte_batch(hw_pte_t *start_ptep, int max_nr, pte_t pte)
{
pte_t expected_pte = pte_next_swp_offset(pte);
- const pte_t *end_ptep = start_ptep + max_nr;
- pte_t *ptep = start_ptep + 1;
+ const hw_pte_t *end_ptep = start_ptep + max_nr;
+ hw_pte_t *ptep = start_ptep + 1;
VM_WARN_ON(max_nr < 1);
VM_WARN_ON(!softleaf_is_swap(softleaf_from_pte(pte)));
@@ -1550,7 +1550,7 @@ static inline void maybe_rmap_unlock_action(struct vm_area_struct *vma,
#ifdef CONFIG_MMU_NOTIFIER
static inline bool clear_flush_young_ptes_notify(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, unsigned int nr)
+ unsigned long addr, hw_pte_t *ptep, unsigned int nr)
{
bool young;
@@ -1571,7 +1571,7 @@ static inline bool pmdp_clear_flush_young_notify(struct vm_area_struct *vma,
}
static inline bool test_and_clear_young_ptes_notify(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, unsigned int nr)
+ unsigned long addr, hw_pte_t *ptep, unsigned int nr)
{
bool young;
diff --git a/mm/kasan/init.c b/mm/kasan/init.c
index 66a8838879876..9bbc2a41d23f0 100644
--- a/mm/kasan/init.c
+++ b/mm/kasan/init.c
@@ -64,7 +64,7 @@ static inline bool kasan_pmd_table(pud_t pud)
return false;
}
#endif
-pte_t kasan_early_shadow_pte[MAX_PTRS_PER_PTE + PTE_HWTABLE_PTRS]
+hw_pte_t kasan_early_shadow_pte[MAX_PTRS_PER_PTE + PTE_HWTABLE_PTRS]
__bss_pgtbl;
static inline bool kasan_pte_table(pmd_t pmd)
@@ -92,7 +92,7 @@ static __init void *early_alloc(size_t size, int node)
static void __ref zero_pte_populate(pmd_t *pmd, unsigned long addr,
unsigned long end)
{
- pte_t *pte = pte_offset_kernel(pmd, addr);
+ hw_pte_t *pte = pte_offset_kernel(pmd, addr);
pte_t zero_pte;
zero_pte = pfn_pte(PFN_DOWN(__pa_symbol(kasan_early_shadow_page)),
@@ -122,7 +122,7 @@ static int __ref zero_pmd_populate(pud_t *pud, unsigned long addr,
}
if (pmd_none(*pmd)) {
- pte_t *p;
+ hw_pte_t *p;
if (slab_is_available())
p = pte_alloc_one_kernel(&init_mm);
@@ -281,9 +281,9 @@ int __ref kasan_populate_early_shadow(const void *shadow_start,
return 0;
}
-static void kasan_free_pte(pte_t *pte_start, pmd_t *pmd)
+static void kasan_free_pte(hw_pte_t *pte_start, pmd_t *pmd)
{
- pte_t *pte;
+ hw_pte_t *pte;
int i;
for (i = 0; i < PTRS_PER_PTE; i++) {
@@ -341,7 +341,7 @@ static void kasan_free_p4d(p4d_t *p4d_start, pgd_t *pgd)
pgd_clear(pgd);
}
-static void kasan_remove_pte_table(pte_t *pte, unsigned long addr,
+static void kasan_remove_pte_table(hw_pte_t *pte, unsigned long addr,
unsigned long end)
{
unsigned long next;
@@ -369,7 +369,7 @@ static void kasan_remove_pmd_table(pmd_t *pmd, unsigned long addr,
unsigned long next;
for (; addr < end; addr = next, pmd++) {
- pte_t *pte;
+ hw_pte_t *pte;
next = pmd_addr_end(addr, end);
diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c
index d286e0a045437..86fc7ed45dcd0 100644
--- a/mm/kasan/shadow.c
+++ b/mm/kasan/shadow.c
@@ -189,7 +189,7 @@ static bool shadow_mapped(unsigned long addr)
p4d_t *p4d;
pud_t *pud;
pmd_t *pmd;
- pte_t *pte;
+ hw_pte_t *pte;
if (pgd_none(*pgd))
return false;
@@ -297,7 +297,7 @@ struct vmalloc_populate_data {
struct page **pages;
};
-static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,
+static int kasan_populate_vmalloc_pte(hw_pte_t *ptep, unsigned long addr,
void *_data)
{
struct vmalloc_populate_data *data = _data;
@@ -465,7 +465,7 @@ int __kasan_populate_vmalloc(unsigned long addr, unsigned long size, gfp_t gfp_m
return 0;
}
-static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr,
+static int kasan_depopulate_vmalloc_pte(hw_pte_t *ptep, unsigned long addr,
void *unused)
{
pte_t pte;
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index f49a6710933b1..4a7074f58b55d 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -639,8 +639,8 @@ static void release_pte_folio(struct folio *folio)
folio_putback_lru(folio);
}
-static void release_pte_pages(pte_t *pte, pte_t *_pte,
- struct list_head *compound_pagelist)
+static void release_pte_pages(hw_pte_t *pte, hw_pte_t *_pte,
+ struct list_head *compound_pagelist)
{
struct folio *folio, *tmp;
@@ -685,7 +685,8 @@ static void count_collapse_event(unsigned int order, enum vm_event_item vm_event
}
static enum scan_result __collapse_huge_page_isolate(struct vm_area_struct *vma,
- unsigned long start_addr, pte_t *pte, struct collapse_control *cc,
+ unsigned long start_addr, hw_pte_t *pte,
+ struct collapse_control *cc,
unsigned int order, struct list_head *compound_pagelist)
{
const unsigned int max_ptes_none = collapse_max_ptes_none(cc, vma, order);
@@ -694,7 +695,7 @@ static enum scan_result __collapse_huge_page_isolate(struct vm_area_struct *vma,
struct page *page = NULL;
struct folio *folio = NULL;
unsigned long addr = start_addr;
- pte_t *_pte;
+ hw_pte_t *_pte;
int none_or_zero = 0, shared = 0, referenced = 0;
enum scan_result result = SCAN_FAIL;
@@ -840,16 +841,18 @@ static enum scan_result __collapse_huge_page_isolate(struct vm_area_struct *vma,
return result;
}
-static void __collapse_huge_page_copy_succeeded(pte_t *pte,
- struct vm_area_struct *vma, unsigned long address,
- spinlock_t *ptl, unsigned int order,
- struct list_head *compound_pagelist)
+static void __collapse_huge_page_copy_succeeded(hw_pte_t *pte,
+ struct vm_area_struct *vma,
+ unsigned long address,
+ spinlock_t *ptl,
+ unsigned int order,
+ struct list_head *compound_pagelist)
{
const unsigned long nr_pages = 1UL << order;
unsigned long end = address + (PAGE_SIZE * nr_pages);
struct folio *src, *tmp;
pte_t pteval;
- pte_t *_pte;
+ hw_pte_t *_pte;
unsigned int nr_ptes;
for (_pte = pte; _pte < pte + nr_pages; _pte += nr_ptes,
@@ -904,9 +907,11 @@ static void __collapse_huge_page_copy_succeeded(pte_t *pte,
}
}
-static void __collapse_huge_page_copy_failed(pte_t *pte,
- pmd_t *pmd, pmd_t orig_pmd, struct vm_area_struct *vma,
- unsigned int order, struct list_head *compound_pagelist)
+static void __collapse_huge_page_copy_failed(hw_pte_t *pte,
+ pmd_t *pmd, pmd_t orig_pmd,
+ struct vm_area_struct *vma,
+ unsigned int order,
+ struct list_head *compound_pagelist)
{
const unsigned long nr_pages = 1UL << order;
spinlock_t *pmd_ptl;
@@ -942,10 +947,14 @@ static void __collapse_huge_page_copy_failed(pte_t *pte,
* @ptl: lock on raw pages' PTEs
* @compound_pagelist: list that stores compound pages
*/
-static enum scan_result __collapse_huge_page_copy(pte_t *pte, struct folio *folio,
- pmd_t *pmd, pmd_t orig_pmd, struct vm_area_struct *vma,
- unsigned long address, spinlock_t *ptl, unsigned int order,
- struct list_head *compound_pagelist)
+static enum scan_result __collapse_huge_page_copy(hw_pte_t *pte,
+ struct folio *folio,
+ pmd_t *pmd, pmd_t orig_pmd,
+ struct vm_area_struct *vma,
+ unsigned long address,
+ spinlock_t *ptl,
+ unsigned int order,
+ struct list_head *compound_pagelist)
{
const unsigned long nr_pages = 1UL << order;
unsigned int i;
@@ -1164,7 +1173,7 @@ static enum scan_result __collapse_huge_page_swapin(struct mm_struct *mm,
vm_fault_t ret = 0;
unsigned long addr, end = start_addr + (PAGE_SIZE << order);
enum scan_result result;
- pte_t *pte = NULL;
+ hw_pte_t *pte = NULL;
spinlock_t *ptl;
for (addr = start_addr; addr < end; addr += PAGE_SIZE) {
@@ -1292,7 +1301,7 @@ static enum scan_result collapse_huge_page(struct mm_struct *mm, unsigned long s
const unsigned long end_addr = start_addr + (PAGE_SIZE << order);
LIST_HEAD(compound_pagelist);
pmd_t *pmd, _pmd;
- pte_t *pte = NULL;
+ hw_pte_t *pte = NULL;
pgtable_t pgtable;
struct folio *folio;
spinlock_t *pmd_ptl, *pte_ptl;
@@ -1606,7 +1615,8 @@ static enum scan_result collapse_scan_pmd(struct mm_struct *mm,
unsigned int max_ptes_none = collapse_max_ptes_none(cc, vma, HPAGE_PMD_ORDER);
enum tva_type tva_flags = cc->is_khugepaged ? TVA_KHUGEPAGED : TVA_FORCED_COLLAPSE;
pmd_t *pmd;
- pte_t *pte, *_pte, pteval;
+ hw_pte_t *pte, *_pte;
+ pte_t pteval;
int i;
int none_or_zero = 0, shared = 0, referenced = 0;
enum scan_result result = SCAN_FAIL;
@@ -1871,7 +1881,7 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
unsigned long end = haddr + HPAGE_PMD_SIZE;
struct vm_area_struct *vma = vma_lookup(mm, haddr);
struct folio *folio;
- pte_t *start_pte, *pte;
+ hw_pte_t *start_pte, *pte;
pmd_t *pmd, pgt_pmd;
spinlock_t *pml = NULL, *ptl;
int i;
diff --git a/mm/ksm.c b/mm/ksm.c
index 624f37975e129..3882d8916efa4 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -618,7 +618,7 @@ static int break_ksm_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned long en
{
unsigned long *found_addr = (unsigned long *) walk->private;
struct mm_struct *mm = walk->mm;
- pte_t *start_ptep, *ptep;
+ hw_pte_t *start_ptep, *ptep;
spinlock_t *ptl;
int found = 0;
@@ -1292,7 +1292,7 @@ static u32 calc_checksum(struct page *page)
}
static int write_protect_page(struct vm_area_struct *vma, struct folio *folio,
- pte_t *orig_pte)
+ pte_t *ptentp)
{
struct mm_struct *mm = vma->vm_mm;
DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, 0, 0);
@@ -1371,7 +1371,7 @@ static int write_protect_page(struct vm_area_struct *vma, struct folio *folio,
set_pte_at(mm, pvmw.address, pvmw.pte, entry);
}
- *orig_pte = entry;
+ *ptentp = entry;
err = 0;
out_unlock:
@@ -1399,7 +1399,7 @@ static int replace_page(struct vm_area_struct *vma, struct page *page,
struct folio *folio = page_folio(page);
pmd_t *pmd;
pmd_t pmde;
- pte_t *ptep;
+ hw_pte_t *ptep;
pte_t newpte;
spinlock_t *ptl;
unsigned long addr;
@@ -2530,7 +2530,8 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon
{
struct ksm_next_page_arg *private = walk->private;
struct vm_area_struct *vma = walk->vma;
- pte_t *start_ptep = NULL, *ptep, pte;
+ hw_pte_t *start_ptep = NULL, *ptep;
+ pte_t pte;
struct mm_struct *mm = walk->mm;
struct folio *folio;
struct page *page;
diff --git a/mm/madvise.c b/mm/madvise.c
index 73c2901b9adbf..a08270c9f7610 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -198,7 +198,7 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned long start,
{
struct vm_area_struct *vma = walk->private;
struct swap_io_ctx ctx = {};
- pte_t *ptep = NULL;
+ hw_pte_t *ptep = NULL;
spinlock_t *ptl;
unsigned long addr;
@@ -350,7 +350,7 @@ static inline bool can_do_file_pageout(struct vm_area_struct *vma)
}
static inline int madvise_folio_pte_batch(unsigned long addr, unsigned long end,
- struct folio *folio, pte_t *ptep,
+ struct folio *folio, hw_pte_t *ptep,
pte_t *ptentp)
{
int max_nr = (end - addr) / PAGE_SIZE;
@@ -368,7 +368,8 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
bool pageout = private->pageout;
struct mm_struct *mm = tlb->mm;
struct vm_area_struct *vma = walk->vma;
- pte_t *start_pte, *pte, ptent;
+ hw_pte_t *start_pte, *pte;
+ pte_t ptent;
spinlock_t *ptl;
struct folio *folio = NULL;
LIST_HEAD(folio_list);
@@ -670,7 +671,8 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
struct mm_struct *mm = tlb->mm;
struct vm_area_struct *vma = walk->vma;
spinlock_t *ptl;
- pte_t *start_pte, *pte, ptent;
+ hw_pte_t *start_pte, *pte;
+ pte_t ptent;
struct folio *folio;
int nr_swap = 0;
unsigned long next;
@@ -1095,7 +1097,7 @@ static int guard_install_pmd_entry(pmd_t *pmd, unsigned long addr,
return pmd_trans_huge(pmdval);
}
-static int guard_install_pte_entry(pte_t *pte, unsigned long addr,
+static int guard_install_pte_entry(hw_pte_t *pte, unsigned long addr,
unsigned long next, struct mm_walk *walk)
{
pte_t pteval = ptep_get(pte);
@@ -1113,12 +1115,12 @@ static int guard_install_pte_entry(pte_t *pte, unsigned long addr,
}
static int guard_install_set_pte(unsigned long addr, unsigned long next,
- pte_t *ptep, struct mm_walk *walk)
+ pte_t *ptentp, struct mm_walk *walk)
{
unsigned long *nr_pages = (unsigned long *)walk->private;
/* Simply install a PTE marker, this causes segfault on access. */
- *ptep = make_pte_marker(PTE_MARKER_GUARD);
+ *ptentp = make_pte_marker(PTE_MARKER_GUARD);
(*nr_pages)++;
return 0;
@@ -1238,7 +1240,7 @@ static int guard_remove_pmd_entry(pmd_t *pmd, unsigned long addr,
return 0;
}
-static int guard_remove_pte_entry(pte_t *pte, unsigned long addr,
+static int guard_remove_pte_entry(hw_pte_t *pte, unsigned long addr,
unsigned long next, struct mm_walk *walk)
{
pte_t ptent = ptep_get(pte);
diff --git a/mm/mapping_dirty_helpers.c b/mm/mapping_dirty_helpers.c
index e0efa36e0a076..dcbd39912f819 100644
--- a/mm/mapping_dirty_helpers.c
+++ b/mm/mapping_dirty_helpers.c
@@ -31,7 +31,7 @@ struct wp_walk {
* The function write-protects a pte and records the range in
* virtual address space of touched ptes for efficient range TLB flushes.
*/
-static int wp_pte(pte_t *pte, unsigned long addr, unsigned long end,
+static int wp_pte(hw_pte_t *pte, unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
struct wp_walk *wpwalk = walk->private;
@@ -86,7 +86,7 @@ struct clean_walk {
* in the address_space, as well as the first and last of the bits
* touched.
*/
-static int clean_record_pte(pte_t *pte, unsigned long addr,
+static int clean_record_pte(hw_pte_t *pte, unsigned long addr,
unsigned long end, struct mm_walk *walk)
{
struct wp_walk *wpwalk = walk->private;
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index a8b03e2920ba8..a89f3fde47a5d 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -342,7 +342,7 @@ static unsigned long dev_pagemap_mapping_shift(struct vm_area_struct *vma,
p4d_t *p4d;
pud_t *pud;
pmd_t *pmd;
- pte_t *pte;
+ hw_pte_t *pte;
pte_t ptent;
VM_BUG_ON_VMA(address == -EFAULT, vma);
@@ -742,7 +742,7 @@ static int hwpoison_pte_range(pmd_t *pmdp, unsigned long addr,
{
struct hwpoison_walk *hwp = walk->private;
int ret = 0;
- pte_t *ptep, *mapped_pte;
+ hw_pte_t *ptep, *mapped_pte;
spinlock_t *ptl;
ptl = pmd_trans_huge_lock(pmdp, walk->vma);
@@ -770,7 +770,7 @@ static int hwpoison_pte_range(pmd_t *pmdp, unsigned long addr,
}
#ifdef CONFIG_HUGETLB_PAGE
-static int hwpoison_hugetlb_range(pte_t *ptep, unsigned long hmask,
+static int hwpoison_hugetlb_range(hw_pte_t *ptep, unsigned long hmask,
unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
diff --git a/mm/memory.c b/mm/memory.c
index ec63dd6212ac5..7fc8ce10503c3 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -462,7 +462,7 @@ int __pte_alloc(struct mm_struct *mm, pmd_t *pmd)
int __pte_alloc_kernel(pmd_t *pmd)
{
- pte_t *new = pte_alloc_one_kernel(&init_mm);
+ hw_pte_t *new = pte_alloc_one_kernel(&init_mm);
if (!new)
return -ENOMEM;
@@ -909,7 +909,7 @@ struct page *vm_normal_page_pud(struct vm_area_struct *vma,
*/
static void restore_exclusive_pte(struct vm_area_struct *vma,
struct folio *folio, struct page *page, unsigned long address,
- pte_t *ptep, pte_t orig_pte)
+ hw_pte_t *ptep, pte_t orig_pte)
{
pte_t pte;
@@ -946,7 +946,7 @@ static void restore_exclusive_pte(struct vm_area_struct *vma,
* sleeping.
*/
static int try_restore_exclusive_pte(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, pte_t orig_pte)
+ unsigned long addr, hw_pte_t *ptep, pte_t orig_pte)
{
const softleaf_t entry = softleaf_from_pte(orig_pte);
struct page *page = softleaf_to_page(entry);
@@ -969,7 +969,7 @@ static int try_restore_exclusive_pte(struct vm_area_struct *vma,
static unsigned long
copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm,
- pte_t *dst_pte, pte_t *src_pte, struct vm_area_struct *dst_vma,
+ hw_pte_t *dst_pte, hw_pte_t *src_pte, struct vm_area_struct *dst_vma,
struct vm_area_struct *src_vma, unsigned long addr, int *rss)
{
pte_t orig_pte = ptep_get(src_pte);
@@ -1083,7 +1083,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm,
*/
static inline int
copy_present_page(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
- pte_t *dst_pte, pte_t *src_pte, unsigned long addr, int *rss,
+ hw_pte_t *dst_pte, hw_pte_t *src_pte, unsigned long addr, int *rss,
struct folio **prealloc, struct page *page)
{
struct folio *new_folio;
@@ -1122,7 +1122,7 @@ copy_present_page(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma
}
static __always_inline void __copy_present_ptes(struct vm_area_struct *dst_vma,
- struct vm_area_struct *src_vma, pte_t *dst_pte, pte_t *src_pte,
+ struct vm_area_struct *src_vma, hw_pte_t *dst_pte, hw_pte_t *src_pte,
pte_t pte, unsigned long addr, int nr)
{
struct mm_struct *src_mm = src_vma->vm_mm;
@@ -1172,7 +1172,7 @@ static __always_inline void __copy_present_ptes(struct vm_area_struct *dst_vma,
*/
static inline int
copy_present_ptes(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
- pte_t *dst_pte, pte_t *src_pte, pte_t pte, unsigned long addr,
+ hw_pte_t *dst_pte, hw_pte_t *src_pte, pte_t pte, unsigned long addr,
int max_nr, int *rss, struct folio **prealloc)
{
fpb_t flags = FPB_MERGE_WRITE;
@@ -1272,8 +1272,8 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
{
struct mm_struct *dst_mm = dst_vma->vm_mm;
struct mm_struct *src_mm = src_vma->vm_mm;
- pte_t *orig_src_pte, *orig_dst_pte;
- pte_t *src_pte, *dst_pte;
+ hw_pte_t *orig_src_pte, *orig_dst_pte;
+ hw_pte_t *src_pte, *dst_pte;
pmd_t dummy_pmdval;
pte_t ptent;
spinlock_t *src_ptl, *dst_ptl;
@@ -1666,7 +1666,7 @@ static inline bool zap_drop_markers(struct zap_details *details)
* Returns true if uffd-wp PTEs were installed, false otherwise.
*/
bool cond_install_uffd_wp_ptes(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, pte_t pte,
+ unsigned long addr, hw_pte_t *ptep, pte_t pte,
unsigned long nr_ptes)
{
bool arm_uffd_pte = false;
@@ -1720,7 +1720,7 @@ bool cond_install_uffd_wp_ptes(struct vm_area_struct *vma,
*/
static inline bool
zap_install_uffd_wp_if_needed(struct vm_area_struct *vma,
- unsigned long addr, pte_t *pte, int nr,
+ unsigned long addr, hw_pte_t *pte, int nr,
struct zap_details *details, pte_t pteval)
{
if (zap_drop_markers(details))
@@ -1731,7 +1731,7 @@ zap_install_uffd_wp_if_needed(struct vm_area_struct *vma,
static __always_inline void zap_present_folio_ptes(struct mmu_gather *tlb,
struct vm_area_struct *vma, struct folio *folio,
- struct page *page, pte_t *pte, pte_t ptent, unsigned int nr,
+ struct page *page, hw_pte_t *pte, pte_t ptent, unsigned int nr,
unsigned long addr, struct zap_details *details, int *rss,
bool *force_flush, bool *force_break, bool *any_skipped)
{
@@ -1781,7 +1781,7 @@ static __always_inline void zap_present_folio_ptes(struct mmu_gather *tlb,
* Returns the number of processed (skipped or zapped) PTEs (at least 1).
*/
static inline int zap_present_ptes(struct mmu_gather *tlb,
- struct vm_area_struct *vma, pte_t *pte, pte_t ptent,
+ struct vm_area_struct *vma, hw_pte_t *pte, pte_t ptent,
unsigned int max_nr, unsigned long addr,
struct zap_details *details, int *rss, bool *force_flush,
bool *force_break, bool *any_skipped)
@@ -1827,7 +1827,7 @@ static inline int zap_present_ptes(struct mmu_gather *tlb,
}
static inline int zap_nonpresent_ptes(struct mmu_gather *tlb,
- struct vm_area_struct *vma, pte_t *pte, pte_t ptent,
+ struct vm_area_struct *vma, hw_pte_t *pte, pte_t ptent,
unsigned int max_nr, unsigned long addr,
struct zap_details *details, int *rss, bool *any_skipped)
{
@@ -1898,7 +1898,7 @@ static inline int zap_nonpresent_ptes(struct mmu_gather *tlb,
}
static inline int do_zap_pte_range(struct mmu_gather *tlb,
- struct vm_area_struct *vma, pte_t *pte,
+ struct vm_area_struct *vma, hw_pte_t *pte,
unsigned long addr, unsigned long end,
struct zap_details *details, int *rss,
bool *force_flush, bool *force_break,
@@ -1961,7 +1961,7 @@ static bool zap_pte_table_if_empty(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, pmd_t *pmdval)
{
spinlock_t *pml, *ptl = NULL;
- pte_t *start_pte, *pte;
+ hw_pte_t *start_pte, *pte;
int i;
pml = pmd_lock(mm, pmd);
@@ -2001,8 +2001,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb,
struct mm_struct *mm = tlb->mm;
int rss[NR_MM_COUNTERS];
spinlock_t *ptl;
- pte_t *start_pte;
- pte_t *pte;
+ hw_pte_t *start_pte;
+ hw_pte_t *pte;
pmd_t pmdval;
unsigned long start = addr;
bool direct_reclaim = true;
@@ -2384,7 +2384,7 @@ static pmd_t *walk_to_pmd(struct mm_struct *mm, unsigned long addr)
return pmd;
}
-pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,
+hw_pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,
spinlock_t **ptl)
{
pmd_t *pmd = walk_to_pmd(mm, addr);
@@ -2442,7 +2442,7 @@ static int validate_page_before_insert(struct vm_area_struct *vma,
return 0;
}
-static int insert_page_into_pte_locked(struct vm_area_struct *vma, pte_t *pte,
+static int insert_page_into_pte_locked(struct vm_area_struct *vma, hw_pte_t *pte,
unsigned long addr, struct page *page,
pgprot_t prot, bool mkwrite)
{
@@ -2487,7 +2487,7 @@ static int insert_page(struct vm_area_struct *vma, unsigned long addr,
struct page *page, pgprot_t prot, bool mkwrite)
{
int retval;
- pte_t *pte;
+ hw_pte_t *pte;
spinlock_t *ptl;
retval = validate_page_before_insert(vma, page);
@@ -2504,7 +2504,7 @@ static int insert_page(struct vm_area_struct *vma, unsigned long addr,
return retval;
}
-static int insert_page_in_batch_locked(struct vm_area_struct *vma, pte_t *pte,
+static int insert_page_in_batch_locked(struct vm_area_struct *vma, hw_pte_t *pte,
unsigned long addr, struct page *page, pgprot_t prot)
{
int err;
@@ -2522,7 +2522,7 @@ static int insert_pages(struct vm_area_struct *vma, unsigned long addr,
struct page **pages, unsigned long *num, pgprot_t prot)
{
pmd_t *pmd = NULL;
- pte_t *start_pte, *pte;
+ hw_pte_t *start_pte, *pte;
spinlock_t *pte_lock;
struct mm_struct *const mm = vma->vm_mm;
unsigned long curr_page_idx = 0;
@@ -2765,7 +2765,8 @@ static vm_fault_t insert_pfn(struct vm_area_struct *vma, unsigned long addr,
unsigned long pfn, pgprot_t prot, bool mkwrite)
{
struct mm_struct *mm = vma->vm_mm;
- pte_t *pte, entry;
+ hw_pte_t *pte;
+ pte_t entry;
spinlock_t *ptl;
pte = get_locked_pte(mm, addr, &ptl);
@@ -3006,7 +3007,7 @@ static int remap_pte_range(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, unsigned long end,
unsigned long pfn, pgprot_t prot)
{
- pte_t *pte, *mapped_pte;
+ hw_pte_t *pte, *mapped_pte;
spinlock_t *ptl;
int err = 0;
@@ -3407,7 +3408,7 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd,
pte_fn_t fn, void *data, bool create,
pgtbl_mod_mask *mask)
{
- pte_t *pte, *mapped_pte;
+ hw_pte_t *pte, *mapped_pte;
int err = 0;
spinlock_t *ptl;
@@ -4710,7 +4711,7 @@ static vm_fault_t handle_pte_marker(struct vm_fault *vmf)
/*
* Check if the PTEs within a range are contiguous swap entries.
*/
-static bool can_swapin_thp(struct vm_fault *vmf, pte_t *ptep, int nr_pages)
+static bool can_swapin_thp(struct vm_fault *vmf, hw_pte_t *ptep, int nr_pages)
{
unsigned long addr;
int idx;
@@ -4762,7 +4763,7 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
unsigned long addr;
softleaf_t entry;
spinlock_t *ptl;
- pte_t *pte;
+ hw_pte_t *pte;
int order;
/*
@@ -4856,7 +4857,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
int nr_pages;
unsigned long page_idx;
unsigned long address;
- pte_t *ptep;
+ hw_pte_t *ptep;
if (!pte_unmap_same(vmf))
goto out;
@@ -5025,7 +5026,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
unsigned long idx = folio_page_idx(folio, page);
unsigned long folio_start = address - idx * PAGE_SIZE;
unsigned long folio_end = folio_start + nr * PAGE_SIZE;
- pte_t *folio_ptep;
+ hw_pte_t *folio_ptep;
pte_t folio_pte;
if (unlikely(folio_start < max(address & PMD_MASK, vma->vm_start)))
@@ -5255,7 +5256,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
return ret;
}
-static bool pte_range_none(pte_t *pte, int nr_pages)
+static bool pte_range_none(hw_pte_t *pte, int nr_pages)
{
int i;
@@ -5274,7 +5275,7 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf)
unsigned long orders;
struct folio *folio;
unsigned long addr;
- pte_t *pte;
+ hw_pte_t *pte;
gfp_t gfp;
int order;
@@ -5356,7 +5357,7 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf)
return folio_prealloc(vma->vm_mm, vma, vmf->address, true);
}
-void map_anon_folio_pte_nopf(struct folio *folio, pte_t *pte,
+void map_anon_folio_pte_nopf(struct folio *folio, hw_pte_t *pte,
struct vm_area_struct *vma, unsigned long addr,
bool uffd_wp)
{
@@ -5377,7 +5378,7 @@ void map_anon_folio_pte_nopf(struct folio *folio, pte_t *pte,
update_mmu_cache_range(NULL, vma, addr, pte, nr_pages);
}
-static void map_anon_folio_pte_pf(struct folio *folio, pte_t *pte,
+static void map_anon_folio_pte_pf(struct folio *folio, hw_pte_t *pte,
struct vm_area_struct *vma, unsigned long addr, bool uffd_wp)
{
const unsigned int order = folio_order(folio);
@@ -6162,7 +6163,7 @@ int numa_migrate_check(struct folio *folio, struct vm_fault *vmf,
}
static void numa_rebuild_single_mapping(struct vm_fault *vmf, struct vm_area_struct *vma,
- unsigned long fault_addr, pte_t *fault_pte,
+ unsigned long fault_addr, hw_pte_t *fault_pte,
bool writable)
{
pte_t pte, old_pte;
@@ -6184,7 +6185,7 @@ static void numa_rebuild_large_mapping(struct vm_fault *vmf, struct vm_area_stru
unsigned long start, end, addr = vmf->address;
unsigned long addr_start = addr - (nr << PAGE_SHIFT);
unsigned long pt_start = ALIGN_DOWN(addr, PMD_SIZE);
- pte_t *start_ptep;
+ hw_pte_t *start_ptep;
/* Stay within the VMA and within the page table. */
start = max3(addr_start, pt_start, vma->vm_start);
@@ -6944,7 +6945,7 @@ int __pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address)
#endif /* __PAGETABLE_PMD_FOLDED */
static inline void pfnmap_args_setup(struct follow_pfnmap_args *args,
- spinlock_t *lock, pte_t *ptep,
+ spinlock_t *lock, hw_pte_t *ptep,
pgprot_t pgprot, unsigned long pfn_base,
unsigned long addr_mask, bool writable,
bool special)
@@ -7013,7 +7014,8 @@ int follow_pfnmap_start(struct follow_pfnmap_args *args)
p4d_t *p4dp, p4d;
pud_t *pudp, pud;
pmd_t *pmdp, pmd;
- pte_t *ptep, pte;
+ hw_pte_t *ptep;
+ pte_t pte;
pfnmap_lockdep_assert(vma);
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 2ad0a5f18280a..51ab12fa4cb61 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -710,7 +710,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr,
struct folio *folio;
struct queue_pages *qp = walk->private;
unsigned long flags = qp->flags;
- pte_t *pte, *mapped_pte;
+ hw_pte_t *pte, *mapped_pte;
pte_t ptent;
spinlock_t *ptl;
int max_nr, nr;
@@ -790,7 +790,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr,
return 0;
}
-static int queue_folios_hugetlb(pte_t *pte, unsigned long hmask,
+static int queue_folios_hugetlb(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
diff --git a/mm/migrate.c b/mm/migrate.c
index 15b45832bcfa7..1786ab5e941c2 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -494,7 +494,7 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd,
unsigned long address)
{
spinlock_t *ptl;
- pte_t *ptep;
+ hw_pte_t *ptep;
pte_t pte;
softleaf_t entry;
@@ -525,7 +525,7 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd,
*
* This function will release the vma lock before returning.
*/
-void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep)
+void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep)
{
spinlock_t *ptl = huge_pte_lockptr(hstate_vma(vma), vma->vm_mm, ptep);
softleaf_t entry;
diff --git a/mm/migrate_device.c b/mm/migrate_device.c
index 0c437004329d9..10da0b00aaca5 100644
--- a/mm/migrate_device.c
+++ b/mm/migrate_device.c
@@ -254,7 +254,7 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp,
spinlock_t *ptl;
struct folio *fault_folio = migrate->fault_page ?
page_folio(migrate->fault_page) : NULL;
- pte_t *ptep;
+ hw_pte_t *ptep;
again:
if (pmd_trans_huge(*pmdp) || !pmd_present(*pmdp)) {
@@ -989,7 +989,7 @@ static void migrate_vma_insert_page(struct migrate_vma *migrate,
p4d_t *p4dp;
pud_t *pudp;
pmd_t *pmdp;
- pte_t *ptep;
+ hw_pte_t *ptep;
pte_t orig_pte;
/* Only allow populating anonymous memory */
diff --git a/mm/mincore.c b/mm/mincore.c
index c086836bc4bcc..65cc43c85d03f 100644
--- a/mm/mincore.c
+++ b/mm/mincore.c
@@ -24,7 +24,7 @@
#include "swap.h"
#include "internal.h"
-static int mincore_hugetlb(pte_t *pte, unsigned long hmask, unsigned long addr,
+static int mincore_hugetlb(hw_pte_t *pte, unsigned long hmask, unsigned long addr,
unsigned long end, struct mm_walk *walk)
{
#ifdef CONFIG_HUGETLB_PAGE
@@ -164,7 +164,7 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
{
spinlock_t *ptl;
struct vm_area_struct *vma = walk->vma;
- pte_t *ptep;
+ hw_pte_t *ptep;
unsigned char *vec = walk->private;
int nr = (end - addr) >> PAGE_SHIFT;
int step, i;
diff --git a/mm/mlock.c b/mm/mlock.c
index 39215a3eab1fb..cc6c24e46348a 100644
--- a/mm/mlock.c
+++ b/mm/mlock.c
@@ -305,7 +305,7 @@ void munlock_folio(struct folio *folio)
}
static inline unsigned int folio_mlock_step(struct folio *folio,
- pte_t *pte, unsigned long addr, unsigned long end)
+ hw_pte_t *pte, unsigned long addr, unsigned long end)
{
unsigned int count = (end - addr) >> PAGE_SHIFT;
pte_t ptent = ptep_get(pte);
@@ -353,7 +353,7 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr,
{
struct vm_area_struct *vma = walk->vma;
spinlock_t *ptl;
- pte_t *start_pte, *pte;
+ hw_pte_t *start_pte, *pte;
pte_t ptent;
struct folio *folio;
unsigned int step = 1;
diff --git a/mm/mprotect.c b/mm/mprotect.c
index 2888ee638d872..1d23475e0bb76 100644
--- a/mm/mprotect.c
+++ b/mm/mprotect.c
@@ -103,7 +103,7 @@ bool can_change_pte_writable(struct vm_area_struct *vma, unsigned long addr,
return can_change_shared_pte_writable(vma, pte);
}
-static int mprotect_folio_pte_batch(struct folio *folio, pte_t *ptep,
+static int mprotect_folio_pte_batch(struct folio *folio, hw_pte_t *ptep,
pte_t pte, int max_nr_ptes, fpb_t flags)
{
/* No underlying folio, so cannot batch */
@@ -118,7 +118,7 @@ static int mprotect_folio_pte_batch(struct folio *folio, pte_t *ptep,
/* Set nr_ptes number of ptes, starting from idx */
static __always_inline void prot_commit_flush_ptes(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, pte_t oldpte, pte_t ptent,
+ unsigned long addr, hw_pte_t *ptep, pte_t oldpte, pte_t ptent,
int nr_ptes, int idx, bool set_write, struct mmu_gather *tlb)
{
/*
@@ -170,7 +170,7 @@ static __always_inline int page_anon_exclusive_batch(int start_idx, int max_len,
* retrieve sub-batches.
*/
static __always_inline void commit_anon_folio_batch(struct vm_area_struct *vma,
- struct folio *folio, struct page *first_page, unsigned long addr, pte_t *ptep,
+ struct folio *folio, struct page *first_page, unsigned long addr, hw_pte_t *ptep,
pte_t oldpte, pte_t ptent, int nr_ptes, struct mmu_gather *tlb)
{
bool expected_anon_exclusive;
@@ -189,7 +189,7 @@ static __always_inline void commit_anon_folio_batch(struct vm_area_struct *vma,
}
static __always_inline void set_write_prot_commit_flush_ptes(struct vm_area_struct *vma,
- struct folio *folio, struct page *page, unsigned long addr, pte_t *ptep,
+ struct folio *folio, struct page *page, unsigned long addr, hw_pte_t *ptep,
pte_t oldpte, pte_t ptent, int nr_ptes, struct mmu_gather *tlb)
{
bool set_write;
@@ -212,7 +212,7 @@ static __always_inline void set_write_prot_commit_flush_ptes(struct vm_area_stru
}
static long change_softleaf_pte(struct vm_area_struct *vma,
- unsigned long addr, pte_t *pte, pte_t oldpte, unsigned long cp_flags)
+ unsigned long addr, hw_pte_t *pte, pte_t oldpte, unsigned long cp_flags)
{
const bool uffd_prot = cp_flags & (MM_CP_UFFD_WP | MM_CP_UFFD_RWP);
const bool uffd_prot_resolve = cp_flags &
@@ -279,7 +279,7 @@ static long change_softleaf_pte(struct vm_area_struct *vma,
}
static __always_inline void change_present_ptes(struct mmu_gather *tlb,
- struct vm_area_struct *vma, unsigned long addr, pte_t *ptep,
+ struct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,
int nr_ptes, unsigned long end, pgprot_t newprot,
struct folio *folio, struct page *page, unsigned long cp_flags)
{
@@ -332,7 +332,8 @@ static long change_pte_range(struct mmu_gather *tlb,
struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr,
unsigned long end, pgprot_t newprot, unsigned long cp_flags)
{
- pte_t *pte, oldpte;
+ hw_pte_t *pte;
+ pte_t oldpte;
spinlock_t *ptl;
long pages = 0;
bool is_private_single_threaded;
@@ -727,7 +728,7 @@ long change_protection(struct mmu_gather *tlb,
return pages;
}
-static int prot_none_pte_entry(pte_t *pte, unsigned long addr,
+static int prot_none_pte_entry(hw_pte_t *pte, unsigned long addr,
unsigned long next, struct mm_walk *walk)
{
return pfn_modify_allowed(pte_pfn(ptep_get(pte)),
@@ -736,7 +737,7 @@ static int prot_none_pte_entry(pte_t *pte, unsigned long addr,
}
#ifdef CONFIG_HUGETLB_PAGE
-static int prot_none_hugetlb_entry(pte_t *pte, unsigned long hmask,
+static int prot_none_hugetlb_entry(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long next,
struct mm_walk *walk)
{
diff --git a/mm/mremap.c b/mm/mremap.c
index 7c368440fafe2..5272cea12f80e 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -176,7 +176,7 @@ static pte_t move_soft_dirty_pte(pte_t pte)
}
static int mremap_folio_pte_batch(struct vm_area_struct *vma, unsigned long addr,
- pte_t *ptep, pte_t pte, int max_nr)
+ hw_pte_t *ptep, pte_t pte, int max_nr)
{
struct folio *folio;
@@ -200,7 +200,7 @@ static int move_ptes(struct pagetable_move_control *pmc,
struct vm_area_struct *vma = pmc->old;
bool need_clear_uffd_wp = vma_has_uffd_without_event_remap(vma);
struct mm_struct *mm = vma->vm_mm;
- pte_t *old_ptep, *new_ptep;
+ hw_pte_t *old_ptep, *new_ptep;
pte_t old_pte, pte;
pmd_t dummy_pmdval;
spinlock_t *old_ptl, *new_ptl;
diff --git a/mm/page_table_check.c b/mm/page_table_check.c
index 143a918c0bde2..705794a817bad 100644
--- a/mm/page_table_check.c
+++ b/mm/page_table_check.c
@@ -208,7 +208,7 @@ static void page_table_check_pte_flags(pte_t pte)
}
void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t pte, unsigned int nr)
+ hw_pte_t *ptep, pte_t pte, unsigned int nr)
{
unsigned int i;
@@ -279,7 +279,7 @@ void __page_table_check_pte_clear_range(struct mm_struct *mm,
return;
if (!pmd_none(pmd) && !pmd_bad(pmd) && !pmd_leaf(pmd)) {
- pte_t *ptep = pte_offset_map(&pmd, addr);
+ hw_pte_t *ptep = pte_offset_map(&pmd, addr);
unsigned long i;
if (WARN_ON(!ptep))
diff --git a/mm/pagewalk.c b/mm/pagewalk.c
index 7411702a37f58..b6a29a06a19bc 100644
--- a/mm/pagewalk.c
+++ b/mm/pagewalk.c
@@ -26,7 +26,7 @@ static int real_depth(int depth)
return depth;
}
-static int walk_pte_range_inner(pte_t *pte, unsigned long addr,
+static int walk_pte_range_inner(hw_pte_t *pte, unsigned long addr,
unsigned long end, struct mm_walk *walk)
{
const struct mm_walk_ops *ops = walk->ops;
@@ -61,7 +61,7 @@ static int walk_pte_range_inner(pte_t *pte, unsigned long addr,
static int walk_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
- pte_t *pte;
+ hw_pte_t *pte;
int err = 0;
spinlock_t *ptl;
@@ -341,7 +341,7 @@ static int walk_hugetlb_range(unsigned long addr, unsigned long end,
unsigned long next;
unsigned long hmask = huge_page_mask(h);
unsigned long sz = huge_page_size(h);
- pte_t *pte;
+ hw_pte_t *pte;
const struct mm_walk_ops *ops = walk->ops;
int err = 0;
@@ -905,7 +905,8 @@ struct folio *folio_walk_start(struct folio_walk *fw,
struct page *page;
pud_t *pudp, pud;
pmd_t *pmdp, pmd;
- pte_t *ptep, pte;
+ hw_pte_t *ptep;
+ pte_t pte;
spinlock_t *ptl;
pgd_t *pgdp;
p4d_t *p4dp;
diff --git a/mm/percpu.c b/mm/percpu.c
index 3eff382e565ad..ec0e58e9f6c09 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -3173,7 +3173,7 @@ void __init __weak pcpu_populate_pte(unsigned long addr)
pmd = pmd_offset(pud, addr);
if (!pmd_present(*pmd)) {
- pte_t *new;
+ hw_pte_t *new;
new = memblock_alloc_or_panic(PTE_TABLE_SIZE, PTE_TABLE_SIZE);
pmd_populate_kernel(&init_mm, pmd, new);
diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c
index 224c444cf45d9..df548ad3d9a54 100644
--- a/mm/pgtable-generic.c
+++ b/mm/pgtable-generic.c
@@ -80,7 +80,7 @@ void pmd_clear_bad(pmd_t *pmd)
* force that call on sun4c so we changed this macro slightly
*/
int ptep_set_access_flags(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep,
+ unsigned long address, hw_pte_t *ptep,
pte_t entry, int dirty)
{
int changed = !pte_same(ptep_get(ptep), entry);
@@ -94,7 +94,7 @@ int ptep_set_access_flags(struct vm_area_struct *vma,
#ifndef __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH
bool ptep_clear_flush_young(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep)
+ unsigned long address, hw_pte_t *ptep)
{
bool young;
@@ -107,7 +107,7 @@ bool ptep_clear_flush_young(struct vm_area_struct *vma,
#ifndef __HAVE_ARCH_PTEP_CLEAR_FLUSH
pte_t ptep_clear_flush(struct vm_area_struct *vma, unsigned long address,
- pte_t *ptep)
+ hw_pte_t *ptep)
{
struct mm_struct *mm = (vma)->vm_mm;
pte_t pte;
@@ -294,7 +294,7 @@ static unsigned long pmdp_get_lockless_start(void) { return 0; }
static void pmdp_get_lockless_end(unsigned long irqflags) { }
#endif
-pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp)
+hw_pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp)
{
unsigned long irqflags;
pmd_t pmdval;
@@ -320,11 +320,11 @@ pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp)
return NULL;
}
-pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,
+hw_pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, spinlock_t **ptlp)
{
pmd_t pmdval;
- pte_t *pte;
+ hw_pte_t *pte;
pte = __pte_offset_map(pmd, addr, &pmdval);
if (likely(pte))
@@ -332,11 +332,11 @@ pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,
return pte;
}
-pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,
+hw_pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, pmd_t *pmdvalp,
spinlock_t **ptlp)
{
- pte_t *pte;
+ hw_pte_t *pte;
VM_WARN_ON_ONCE(!pmdvalp);
pte = __pte_offset_map(pmd, addr, pmdvalp);
@@ -402,12 +402,12 @@ pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,
* table, and may not use RCU at all: "outsiders" like khugepaged should avoid
* pte_offset_map() and co once the vma is detached from mm or mm_users is zero.
*/
-pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd,
+hw_pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, spinlock_t **ptlp)
{
spinlock_t *ptl;
pmd_t pmdval;
- pte_t *pte;
+ hw_pte_t *pte;
again:
pte = __pte_offset_map(pmd, addr, &pmdval);
if (unlikely(!pte))
diff --git a/mm/ptdump.c b/mm/ptdump.c
index 5851096e6f656..8f19f20be3c44 100644
--- a/mm/ptdump.c
+++ b/mm/ptdump.c
@@ -19,7 +19,7 @@ static inline int note_kasan_page_table(struct mm_walk *walk,
{
struct ptdump_state *st = walk->private;
- st->note_page_pte(st, addr, kasan_early_shadow_pte[0]);
+ st->note_page_pte(st, addr, ptep_get(kasan_early_shadow_pte));
walk->action = ACTION_CONTINUE;
@@ -117,7 +117,7 @@ static int ptdump_pmd_entry(pmd_t *pmd, unsigned long addr,
return 0;
}
-static int ptdump_pte_entry(pte_t *pte, unsigned long addr,
+static int ptdump_pte_entry(hw_pte_t *pte, unsigned long addr,
unsigned long next, struct mm_walk *walk)
{
struct ptdump_state *st = walk->private;
diff --git a/mm/rmap.c b/mm/rmap.c
index b5cc9273fe5a4..0223c4ab0b361 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -1122,7 +1122,7 @@ static int page_vma_mkclean_one(struct page_vma_mapped_walk *pvmw)
address = pvmw->address;
if (pvmw->pte) {
- pte_t *pte = pvmw->pte;
+ hw_pte_t *pte = pvmw->pte;
pte_t entry = ptep_get(pte);
/*
@@ -2139,7 +2139,7 @@ static pte_t swp_pte_prepare(swp_entry_t entry, pte_t old_pte,
static bool ttu_anon_swapbacked_folio(struct vm_area_struct *vma,
struct folio *folio, struct page *page, unsigned long address,
- pte_t *ptep, pte_t pteval)
+ hw_pte_t *ptep, pte_t pteval)
{
const bool anon_exclusive = folio_test_anon(folio) &&
PageAnonExclusive(page);
@@ -2174,7 +2174,7 @@ static bool ttu_anon_swapbacked_folio(struct vm_area_struct *vma,
}
static bool ttu_anon_folio(struct vm_area_struct *vma, struct folio *folio,
- struct page *page, unsigned long address, pte_t *ptep,
+ struct page *page, unsigned long address, hw_pte_t *ptep,
pte_t pteval, unsigned long nr_pages)
{
/*
diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
index e62e6aa07f126..77eb1749e123f 100644
--- a/mm/sparse-vmemmap.c
+++ b/mm/sparse-vmemmap.c
@@ -135,7 +135,7 @@ static void * __meminit altmap_alloc_block_buf(unsigned long size,
return __va(__pfn_to_phys(pfn));
}
-void __meminit vmemmap_verify(pte_t *pte, int node,
+void __meminit vmemmap_verify(hw_pte_t *pte, int node,
unsigned long start, unsigned long end)
{
unsigned long pfn = pte_pfn(ptep_get(pte));
@@ -236,11 +236,11 @@ static __meminit void *vmemmap_alloc_pte(unsigned long pfn, int node,
return page_address(page);
}
-static pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node,
+static hw_pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node,
struct vmem_altmap *altmap,
unsigned long ptpfn, unsigned long flags)
{
- pte_t *pte = pte_offset_kernel(pmd, addr);
+ hw_pte_t *pte = pte_offset_kernel(pmd, addr);
unsigned long pfn = page_to_pfn((struct page *)addr);
if (pte_none(ptep_get(pte))) {
@@ -323,7 +323,7 @@ static pgd_t * __meminit vmemmap_pgd_populate(unsigned long addr, int node)
return pgd;
}
-static pte_t * __meminit vmemmap_populate_address(unsigned long addr, int node,
+static hw_pte_t * __meminit vmemmap_populate_address(unsigned long addr, int node,
struct vmem_altmap *altmap,
unsigned long ptpfn,
unsigned long flags)
@@ -332,7 +332,7 @@ static pte_t * __meminit vmemmap_populate_address(unsigned long addr, int node,
p4d_t *p4d;
pud_t *pud;
pmd_t *pmd;
- pte_t *pte;
+ hw_pte_t *pte;
pgd = vmemmap_pgd_populate(addr, node);
if (!pgd)
@@ -361,7 +361,7 @@ static int __meminit vmemmap_populate_range(unsigned long start,
unsigned long flags)
{
unsigned long addr = start;
- pte_t *pte;
+ hw_pte_t *pte;
for (; addr < end; addr += PAGE_SIZE) {
pte = vmemmap_populate_address(addr, node, altmap,
@@ -394,7 +394,7 @@ void vmemmap_wrprotect_hvo(unsigned long addr, unsigned long end,
int node, unsigned long headsize)
{
unsigned long maddr;
- pte_t *pte;
+ hw_pte_t *pte;
for (maddr = addr + headsize; maddr < end; maddr += PAGE_SIZE) {
pte = virt_to_kpte(maddr);
@@ -413,7 +413,7 @@ int __weak __meminit vmemmap_check_pmd(pmd_t *pmd, int node,
{
if (!pmd_leaf(pmdp_get(pmd)))
return 0;
- vmemmap_verify((pte_t *)pmd, node, addr, next);
+ vmemmap_verify((hw_pte_t *)pmd, node, addr, next);
return 1;
}
@@ -497,9 +497,9 @@ static bool __meminit reuse_compound_section(unsigned long start_pfn,
return !IS_ALIGNED(offset, nr_pages) && nr_pages > PAGES_PER_SUBSECTION;
}
-static pte_t * __meminit compound_section_tail_page(unsigned long addr)
+static hw_pte_t * __meminit compound_section_tail_page(unsigned long addr)
{
- pte_t *pte;
+ hw_pte_t *pte;
addr -= PAGE_SIZE;
@@ -520,7 +520,7 @@ static int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn,
struct dev_pagemap *pgmap)
{
unsigned long size, addr;
- pte_t *pte;
+ hw_pte_t *pte;
int rc;
if (reuse_compound_section(start_pfn, pgmap)) {
diff --git a/mm/swap_state.c b/mm/swap_state.c
index 305877e1f4d7b..d654449f393e8 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -918,7 +918,8 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,
struct swap_io_ctx ctx = {};
struct blk_plug plug;
struct folio *folio;
- pte_t *pte = NULL, pentry;
+ hw_pte_t *pte = NULL;
+ pte_t pentry;
int win;
unsigned long start, end, addr;
pgoff_t ilx = targ_ilx;
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 869a918b18e8e..2f6aa8dc2f5bb 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -2500,7 +2500,8 @@ static int unuse_pte(struct vm_area_struct *vma, pmd_t *pmd,
struct page *page;
struct folio *swapcache;
spinlock_t *ptl;
- pte_t *pte, new_pte, old_pte;
+ hw_pte_t *pte;
+ pte_t new_pte, old_pte;
bool hwpoisoned = false;
int ret = 1;
@@ -2611,7 +2612,7 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd,
unsigned long addr, unsigned long end,
unsigned int type)
{
- pte_t *pte = NULL;
+ hw_pte_t *pte = NULL;
do {
struct folio *folio;
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 79cc7b546f130..8281796638c0a 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -334,13 +334,13 @@ static int mfill_atomic_install_pte(pmd_t *dst_pmd,
{
int ret;
struct mm_struct *dst_mm = dst_vma->vm_mm;
- pte_t _dst_pte, *dst_pte;
+ hw_pte_t *dst_pte;
bool writable = dst_vma->vm_flags & VM_WRITE;
bool vm_shared = dst_vma->vm_flags & VM_SHARED;
spinlock_t *ptl;
struct folio *folio = page_folio(page);
bool page_in_cache = folio_mapping(folio);
- pte_t dst_ptep;
+ pte_t _dst_pte, dst_ptep;
_dst_pte = mk_pte(page, dst_vma->vm_page_prot);
_dst_pte = pte_mkdirty(_dst_pte);
@@ -629,7 +629,8 @@ static int mfill_atomic_pte_zeropage(struct mfill_state *state)
struct vm_area_struct *dst_vma = state->vma;
unsigned long dst_addr = state->dst_addr;
pmd_t *dst_pmd = state->pmd;
- pte_t _dst_pte, *dst_pte;
+ hw_pte_t *dst_pte;
+ pte_t _dst_pte;
spinlock_t *ptl;
int ret;
@@ -710,7 +711,8 @@ static int mfill_atomic_pte_poison(struct mfill_state *state)
struct mm_struct *dst_mm = dst_vma->vm_mm;
unsigned long dst_addr = state->dst_addr;
pmd_t *dst_pmd = state->pmd;
- pte_t _dst_pte, *dst_pte;
+ hw_pte_t *dst_pte;
+ pte_t _dst_pte;
spinlock_t *ptl;
int ret;
@@ -757,7 +759,7 @@ static __always_inline ssize_t mfill_atomic_hugetlb(
{
struct mm_struct *dst_mm = dst_vma->vm_mm;
ssize_t err;
- pte_t *dst_pte;
+ hw_pte_t *dst_pte;
unsigned long src_addr, dst_addr;
long copied;
struct folio *folio;
@@ -1234,7 +1236,7 @@ void double_pt_unlock(spinlock_t *ptl1,
__release(ptl2);
}
-static inline bool is_pte_pages_stable(pte_t *dst_pte, pte_t *src_pte,
+static inline bool is_pte_pages_stable(hw_pte_t *dst_pte, hw_pte_t *src_pte,
pte_t orig_dst_pte, pte_t orig_src_pte,
pmd_t *dst_pmd, pmd_t dst_pmdval)
{
@@ -1252,7 +1254,8 @@ static inline bool is_pte_pages_stable(pte_t *dst_pte, pte_t *src_pte,
*/
static struct folio *check_ptes_for_batched_move(struct vm_area_struct *src_vma,
unsigned long src_addr,
- pte_t *src_pte, pte_t *dst_pte)
+ hw_pte_t *src_pte,
+ hw_pte_t *dst_pte)
{
pte_t orig_dst_pte, orig_src_pte;
struct folio *folio;
@@ -1283,7 +1286,7 @@ static long move_present_ptes(struct mm_struct *mm,
struct vm_area_struct *dst_vma,
struct vm_area_struct *src_vma,
unsigned long dst_addr, unsigned long src_addr,
- pte_t *dst_pte, pte_t *src_pte,
+ hw_pte_t *dst_pte, hw_pte_t *src_pte,
pte_t orig_dst_pte, pte_t orig_src_pte,
pmd_t *dst_pmd, pmd_t dst_pmdval,
spinlock_t *dst_ptl, spinlock_t *src_ptl,
@@ -1370,7 +1373,7 @@ static long move_present_ptes(struct mm_struct *mm,
static int move_swap_pte(struct mm_struct *mm, struct vm_area_struct *dst_vma,
unsigned long dst_addr, unsigned long src_addr,
- pte_t *dst_pte, pte_t *src_pte,
+ hw_pte_t *dst_pte, hw_pte_t *src_pte,
pte_t orig_dst_pte, pte_t orig_src_pte,
pmd_t *dst_pmd, pmd_t dst_pmdval,
spinlock_t *dst_ptl, spinlock_t *src_ptl,
@@ -1435,7 +1438,7 @@ static int move_zeropage_pte(struct mm_struct *mm,
struct vm_area_struct *dst_vma,
struct vm_area_struct *src_vma,
unsigned long dst_addr, unsigned long src_addr,
- pte_t *dst_pte, pte_t *src_pte,
+ hw_pte_t *dst_pte, hw_pte_t *src_pte,
pte_t orig_dst_pte, pte_t orig_src_pte,
pmd_t *dst_pmd, pmd_t dst_pmdval,
spinlock_t *dst_ptl, spinlock_t *src_ptl)
@@ -1481,8 +1484,8 @@ static long move_pages_ptes(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd
pte_t orig_src_pte, orig_dst_pte;
pte_t src_folio_pte;
spinlock_t *src_ptl, *dst_ptl;
- pte_t *src_pte = NULL;
- pte_t *dst_pte = NULL;
+ hw_pte_t *src_pte = NULL;
+ hw_pte_t *dst_pte = NULL;
pmd_t dummy_pmdval;
pmd_t dst_pmdval;
struct folio *src_folio = NULL;
@@ -2597,7 +2600,8 @@ static inline bool userfaultfd_huge_must_wait(struct userfaultfd_ctx *ctx,
unsigned long reason)
{
struct vm_area_struct *vma = vmf->vma;
- pte_t *ptep, pte;
+ hw_pte_t *ptep;
+ pte_t pte;
assert_fault_locked(vmf);
@@ -2670,7 +2674,7 @@ static inline bool userfaultfd_must_wait(struct userfaultfd_ctx *ctx,
p4d_t *p4d;
pud_t *pud;
pmd_t *pmd, _pmd;
- pte_t *pte;
+ hw_pte_t *pte;
pte_t ptent;
bool ret;
diff --git a/mm/util.c b/mm/util.c
index bf0513d1d3d08..cc253050f6892 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -1564,7 +1564,7 @@ EXPORT_SYMBOL(mmap_action_complete);
*
* Return: the number of table entries in the batch.
*/
-unsigned int folio_pte_batch(struct folio *folio, pte_t *ptep, pte_t pte,
+unsigned int folio_pte_batch(struct folio *folio, hw_pte_t *ptep, pte_t pte,
unsigned int max_nr)
{
return folio_pte_batch_flags(folio, NULL, ptep, &pte, max_nr, 0);
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 6ed6c160abed7..60ba46838bbeb 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -97,7 +97,7 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
phys_addr_t phys_addr, pgprot_t prot,
unsigned int max_page_shift, pgtbl_mod_mask *mask)
{
- pte_t *pte;
+ hw_pte_t *pte;
u64 pfn;
struct page *page;
unsigned long size = PAGE_SIZE;
@@ -389,7 +389,7 @@ int ioremap_page_range(unsigned long addr, unsigned long end,
static void vunmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
pgtbl_mod_mask *mask)
{
- pte_t *pte;
+ hw_pte_t *pte;
pte_t ptent;
unsigned long size = PAGE_SIZE;
@@ -550,7 +550,7 @@ static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr,
pgtbl_mod_mask *mask)
{
int err = 0;
- pte_t *pte;
+ hw_pte_t *pte;
/*
* nr is a running index into the array which helps higher level
@@ -827,7 +827,8 @@ struct page *vmalloc_to_page(const void *vmalloc_addr)
p4d_t *p4d;
pud_t *pud;
pmd_t *pmd;
- pte_t *ptep, pte;
+ hw_pte_t *ptep;
+ pte_t pte;
/*
* XXX we might need to change this if we add VIRTUAL_BUG_ON for
@@ -3615,7 +3616,7 @@ struct vmap_pfn_data {
unsigned int idx;
};
-static int vmap_pfn_apply(pte_t *pte, unsigned long addr, void *private)
+static int vmap_pfn_apply(hw_pte_t *pte, unsigned long addr, void *private)
{
struct vmap_pfn_data *data = private;
unsigned long pfn = data->pfns[data->idx];
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 245f68c75b289..4cadd4f4b00d1 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3579,7 +3579,7 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end,
{
int i;
bool dirty;
- pte_t *pte;
+ hw_pte_t *pte;
spinlock_t *ptl;
unsigned long addr;
int total = 0;
@@ -3612,7 +3612,7 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end,
for (i = pte_index(start), addr = start; addr != end; i += nr, addr += nr * PAGE_SIZE) {
unsigned long pfn;
struct folio *folio;
- pte_t *cur_pte = pte + i;
+ hw_pte_t *cur_pte = pte + i;
pte_t ptent = ptep_get(cur_pte);
nr = 1;
@@ -4343,7 +4343,7 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr)
struct lru_gen_mm_walk *walk;
struct folio *last = NULL;
int young = nr;
- pte_t *pte = pvmw->pte;
+ hw_pte_t *pte = pvmw->pte;
unsigned long addr = pvmw->address;
struct vm_area_struct *vma = pvmw->vma;
struct folio *folio = pfn_folio(pvmw->pfn);
|
| 3/2 | 2026/09/04 15:31 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 56518 output: 26 thoughts: 1259 Thoughts: **Analyzing a Linux Kernel Patch Series: A Deep Dive** Okay, here's what I'm thinking as I dive into this patch series. The task is to evaluate this patch series applied to the Linux kernel. First, let's establish the scope; this series introduces `hw_pte_t` and `ARCH_HAS_HW_PTE_T`, primarily by replacing `pte_t *` with `hw_pte_t *` across several core subsystems like memory management (`mm`), filesystems (`fs`), various drivers (including `i915`, `xen`), and even `kernel/bpf/arena.c`. Right off the bat, I need to check `include/linux/pgtable_types.h`. This is crucial. It looks like `hw_pte_t` is conditionally defined: If `CONFIG_ARCH_HAS_HW_PTE_T` is *not* set, then `hw_pte_t` simply becomes an alias for `pte_t`. So, is this a real type, or just an alias? This needs more context. I'll also be checking `include/linux/pgtable.h`. The initial assessment suggests it's primarily a type refactoring. Most changes seem confined to parameter type updates and local variable declarations, but this calls for immediate scrutiny. I'm now recalling Section 1 guidance on patch analysis. The mantra is *never* blindly trust claims of "No Functional Change Intended" (NFCI) or "Refactoring" claims. Refactorings can introduce subtle semantic shifts or uncover bugs. If *any* reachable executable code is modified, it must be fuzzed. In Section 3, I'm now reminded of the criteria for when fuzzing is required. Specifically, any logic modification in core memory management (mm/), restructuring of reachable data structures, helper abstractions, or algorithm flows *requires* a "WorthFuzzing=true" verdict. So, is this a pure typedef, or are there underlying logic changes? Let's check `include/linux/pgtable_types.h` again. Specifically, if `CONFIG_ARCH_HAS_HW_PTE_T` is *not* defined, then `hw_pte_t` is just a typedef to `pte_t`. It's a conditional definition. Section 2 provides "WorthFuzzing=false" criteria; let's check. This isn't just documentation changes, user-space, or non-executable changes, so those don't apply. The fundamental question is, does this patch alter compiled C logic, data structures, or type definitions in any way that is reachable? The use of a typedef *could* be a refactoring, but a refactoring that *is* a logic modification. I need to quickly check the surrounding commits with a `git log -n 5` or diff. Let's see if this is an isolated change or part of a larger, more involved modification. I will also be using `grepper` or checking `git log` if more searching is necessary. |
| 4/2 | 2026/09/04 15:31 | tool | grepper |
0mArgs:{
"Expression": "CONFIG_ARCH_HAS_HW_PTE_T"
}Results: {
"Output": "include/linux/pgtable_types.h-8-\ninclude/linux/pgtable_types.h:9:#ifdef CONFIG_ARCH_HAS_HW_PTE_T\ninclude/linux/pgtable_types.h-10-typedef struct __hw_pte_t { pte_t __pte; } hw_pte_t;\n"
} |
| 5/2 | 2026/09/04 15:31 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 58549 output: 24 thoughts: 56 Thoughts: **Investigating the `ARCH_HAS_HW_PTE_T` Macro Usage** Okay, so I need to figure out where this `ARCH_HAS_HW_PTE_T` macro is being utilized. My first thought is to check the commit history, specifically the HEAD commit's message or the `git log` output. That usually gives me a good starting point. If that doesn't immediately reveal the context, I can then turn to tools like `grepper` or delve directly into the patch files associated with the change. That should surface any direct references quickly. However, before I dive into specific commits, let's take a step back. Perhaps the best approach is a broader search. I should first conduct a comprehensive search to see if this macro, `ARCH_HAS_HW_PTE_T`, is used anywhere else in the codebase. This might uncover related functionalities and provide a better overall understanding of its role before I get bogged down in individual commit details. This broader context is crucial for a complete analysis. |
| 6/2 | 2026/09/04 15:31 | tool | grepper |
0mArgs:{
"Expression": "ARCH_HAS_HW_PTE_T"
}Results: {
"Output": "include/linux/pgtable_types.h-8-\ninclude/linux/pgtable_types.h:9:#ifdef CONFIG_ARCH_HAS_HW_PTE_T\ninclude/linux/pgtable_types.h-10-typedef struct __hw_pte_t { pte_t __pte; } hw_pte_t;\n--\nmm/Kconfig=1312=config GUP_GET_PXX_LOW_HIGH\n--\nmm/Kconfig-1314-\nmm/Kconfig:1315:config ARCH_HAS_HW_PTE_T\nmm/Kconfig-1316-\tbool\n"
} |
| 7/2 | 2026/09/04 15:31 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 59009 output: 72 thoughts: 1960 Thoughts: **Patch Analysis: Determining Fuzzing Worthiness and Focus** Okay, here's the breakdown. This patch clearly delves deep into core memory management code. I see modifications across `mm/pagewalk.c`, `mm/memory.c`, `mm/madvise.c`, `mm/mprotect.c`, `mm/hugetlb.c`, and a slew of other critical mm subsystem files. It's touching central functions and data structures. It's also making a big change by introducing `hw_pte_t` and modifying several function signatures. Based on the instructions, this is a clear-cut case. None of the negative criteria apply – no documentation updates, no user-space tools that don't affect the kernel image, no decorative logging, and no unreachable hardware or architecture-specific code. This touches reachable executable code; this is a clear `WorthFuzzing = true` decision. It alters core mm logic and data structures, meeting the positive criteria described. Now, to nail down the `FocusSymbols`. The goal is to identify functions that specifically implement the logic being altered, or represent direct API entrypoints for the subsystem features being reviewed. I see that the patch touches the following key non-static functions in C files: * `kernel/bpf/arena.c`: `arena_map_alloc` * `mm/userfaultfd.c`: `hugetlb_mfill_atomic_pte`, `mfill_atomic_install_pte`, `mfill_atomic_pte_zeropage`, `mfill_atomic_pte_poison`, `mfill_atomic_hugetlb`, `move_present_ptes`, `move_pages_ptes`, `userfaultfd_must_wait`, `userfaultfd_huge_must_wait` * `mm/hugetlb.c`: `copy_hugetlb_page_range`, `move_hugetlb_page_tables`, `__unmap_hugepage_range`, `hugetlb_change_protection`, `huge_pmd_share`, `huge_pmd_unshare`, `huge_pte_alloc`, `huge_pte_offset` * `mm/khugepaged.c`: `collapse_huge_page`, `collapse_scan_pmd`, `try_collapse_pte_mapped_thp`, `__collapse_huge_page_isolate`, `__collapse_huge_page_copy` * `mm/mprotect.c`: `change_protection`, `change_pte_range` * `mm/mremap.c`: `move_ptes` * `mm/madvise.c`: `madvise_cold_or_pageout_pte_range`, `madvise_free_pte_range`, `guard_install_pte_entry`, `guard_remove_pte_entry` * `mm/pagewalk.c`: `walk_pte_range`, `walk_hugetlb_range`, `folio_walk_start` * `mm/ksm.c`: `break_ksm_pmd_entry`, `replace_page`, `ksm_next_page_pmd_entry` * `mm/migrate.c`: `migration_entry_wait`, `migration_entry_wait_huge` * `mm/page_table_check.c`: `__page_table_check_ptes_set`, `__page_table_check_pte_clear_range` * `fs/proc/task_mmu.c`: `smaps_pte_range`, `clear_refs_pte_range`, `pagemap_pmd_range`, `pagemap_scan_pmd_entry`, `gather_pte_stats` * `mm/pgtable-generic.c`: `ptep_set_access_flags`, `ptep_clear_flush_young`, `ptep_clear_flush`, `__pte_offset_map`, `pte_offset_map_ro_nolock`, `pte_offset_map_rw_nolock`, `pte_offset_map_lock` * `mm/sparse-vmemmap.c`: `vmemmap_verify`, `vmemmap_populate_range`, `vmemmap_wrprotect_hvo` * `mm/hugetlb_vmemmap.c`: `vmemmap_split_pmd`, `vmemmap_pmd_entry`, `vmemmap_pte_entry`, `vmemmap_remap_pte`, `vmemmap_restore_pte` I'll focus on |
| 8/2 | 2026/09/04 15:31 | tool | grepper |
0mArgs:{
"Expression": "(hugetlb_mfill_atomic_pte|collapse_huge_page|try_collapse_pte_mapped_thp|folio_walk_start|huge_pmd_share|huge_pmd_unshare)",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/huge_memory.c=4760=static int split_huge_pages_pid(int pid, unsigned long vaddr_start,\n--\nmm/huge_memory.c-4811-\nmm/huge_memory.c:4812:\t\tfolio = folio_walk_start(\u0026fw, vma, addr, 0);\nmm/huge_memory.c-4813-\t\tif (!folio)\n--\nmm/hugetlb.c=130=static void __hugetlb_vma_unlock_write_free(struct vm_area_struct *vma);\nmm/hugetlb.c:131:static int __huge_pmd_unshare(struct mmu_gather *tlb,\nmm/hugetlb.c-132-\t\tstruct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,\n--\nmm/hugetlb.c=4933=int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,\n--\nmm/hugetlb.c-4961-\t\t * returned ptep could go away if part of a shared pmd and\nmm/hugetlb.c:4962:\t\t * another thread calls huge_pmd_unshare.\nmm/hugetlb.c-4963-\t\t */\n--\nmm/hugetlb.c=5182=int move_hugetlb_page_tables(struct vm_area_struct *vma,\n--\nmm/hugetlb.c-5221-\nmm/hugetlb.c:5222:\t\tif (huge_pmd_unshare(\u0026tlb, vma, old_addr, src_pte)) {\nmm/hugetlb.c-5223-\t\t\told_addr |= last_addr_mask;\n--\nmm/hugetlb.c-5236-\ttlb_flush_mmu_tlbonly(\u0026tlb);\nmm/hugetlb.c:5237:\thuge_pmd_unshare_flush(\u0026tlb, vma);\nmm/hugetlb.c-5238-\n--\nmm/hugetlb.c=5247=void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,\n--\nmm/hugetlb.c-5283-\t\tptl = huge_pte_lock(h, mm, ptep);\nmm/hugetlb.c:5284:\t\tif (huge_pmd_unshare(tlb, vma, address, ptep)) {\nmm/hugetlb.c-5285-\t\t\tspin_unlock(ptl);\n--\nmm/hugetlb.c-5403-\nmm/hugetlb.c:5404:\thuge_pmd_unshare_flush(tlb, vma);\nmm/hugetlb.c-5405-}\n--\nmm/hugetlb.c=6050=vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma,\n--\nmm/hugetlb.c-6084-\t * Acquire vma lock before calling huge_pte_alloc and hold\nmm/hugetlb.c:6085:\t * until finished with vmf.pte. This prevents huge_pmd_unshare from\nmm/hugetlb.c-6086-\t * being called elsewhere and making the vmf.pte no longer valid.\n--\nmm/hugetlb.c-6283-/*\nmm/hugetlb.c:6284: * Can probably be eliminated, but still used by hugetlb_mfill_atomic_pte().\nmm/hugetlb.c-6285- */\nmm/hugetlb.c=6286=static struct folio *alloc_hugetlb_folio_vma(struct hstate *h,\n--\nmm/hugetlb.c-6312- */\nmm/hugetlb.c:6313:int hugetlb_mfill_atomic_pte(hw_pte_t *dst_pte,\nmm/hugetlb.c-6314-\t\t\t struct vm_area_struct *dst_vma,\n--\nmm/hugetlb.c=6534=long hugetlb_change_protection(struct vm_area_struct *vma,\n--\nmm/hugetlb.c-6594-\t\tptl = huge_pte_lock(h, mm, ptep);\nmm/hugetlb.c:6595:\t\tif (huge_pmd_unshare(\u0026tlb, vma, address, ptep)) {\nmm/hugetlb.c-6596-\t\t\t/*\n--\nmm/hugetlb.c-6685-\ttlb_flush_mmu_tlbonly(\u0026tlb);\nmm/hugetlb.c:6686:\thuge_pmd_unshare_flush(\u0026tlb, vma);\nmm/hugetlb.c-6687-\t/*\n--\nmm/hugetlb.c=6968=bool want_pmd_share(struct vm_area_struct *vma, unsigned long addr)\n--\nmm/hugetlb.c-6973-#ifdef CONFIG_USERFAULTFD\nmm/hugetlb.c:6974:\tif (uffd_disable_huge_pmd_share(vma))\nmm/hugetlb.c-6975-\t\treturn false;\n--\nmm/hugetlb.c=6994=void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma,\n--\nmm/hugetlb.c-7024- */\nmm/hugetlb.c:7025:hw_pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,\nmm/hugetlb.c-7026-\t\t unsigned long addr, pud_t *pud)\n--\nmm/hugetlb.c-7068-\nmm/hugetlb.c:7069:static int __huge_pmd_unshare(struct mmu_gather *tlb,\nmm/hugetlb.c-7070-\t\tstruct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,\n--\nmm/hugetlb.c-7094-/**\nmm/hugetlb.c:7095: * huge_pmd_unshare - Unmap a pmd table if it is shared by multiple users\nmm/hugetlb.c-7096- * @tlb: the current mmu_gather.\n--\nmm/hugetlb.c-7103- *\nmm/hugetlb.c:7104: * Note: The caller must call huge_pmd_unshare_flush() before dropping the\nmm/hugetlb.c-7105- * i_mmap_rwsem.\n--\nmm/hugetlb.c-7109- */\nmm/hugetlb.c:7110:int huge_pmd_unshare(struct mmu_gather *tlb, struct vm_area_struct *vma,\nmm/hugetlb.c-7111-\t\tunsigned long addr, hw_pte_t *ptep)\nmm/hugetlb.c-7112-{\nmm/hugetlb.c:7113:\treturn __huge_pmd_unshare(tlb, vma, addr, ptep, /*check_locks=*/true);\nmm/hugetlb.c-7114-}\n--\nmm/hugetlb.c-7116-/*\nmm/hugetlb.c:7117: * huge_pmd_unshare_flush - Complete a sequence of huge_pmd_unshare() calls\nmm/hugetlb.c-7118- * @tlb: the current mmu_gather.\n--\nmm/hugetlb.c-7123- *\nmm/hugetlb.c:7124: * This function must be called after a sequence of huge_pmd_unshare()\nmm/hugetlb.c-7125- * calls while still holding the i_mmap_rwsem.\nmm/hugetlb.c-7126- */\nmm/hugetlb.c:7127:void huge_pmd_unshare_flush(struct mmu_gather *tlb, struct vm_area_struct *vma)\nmm/hugetlb.c-7128-{\n--\nmm/hugetlb.c-7140-\nmm/hugetlb.c:7141:hw_pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,\nmm/hugetlb.c-7142-\t\t unsigned long addr, pud_t *pud)\n--\nmm/hugetlb.c-7146-\nmm/hugetlb.c:7147:static int __huge_pmd_unshare(struct mmu_gather *tlb,\nmm/hugetlb.c-7148-\t\tstruct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,\n--\nmm/hugetlb.c-7153-\nmm/hugetlb.c:7154:int huge_pmd_unshare(struct mmu_gather *tlb, struct vm_area_struct *vma,\nmm/hugetlb.c-7155-\t\tunsigned long addr, hw_pte_t *ptep)\n--\nmm/hugetlb.c-7159-\nmm/hugetlb.c:7160:void huge_pmd_unshare_flush(struct mmu_gather *tlb, struct vm_area_struct *vma)\nmm/hugetlb.c-7161-{\n--\nmm/hugetlb.c=7176=hw_pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,\n--\nmm/hugetlb.c-7194-\t\t\tif (want_pmd_share(vma, addr) \u0026\u0026 pud_none(*pud))\nmm/hugetlb.c:7195:\t\t\t\tpte = huge_pmd_share(mm, vma, addr, pud);\nmm/hugetlb.c-7196-\t\t\telse\n--\nmm/hugetlb.c=7411=static void hugetlb_unshare_pmds(struct vm_area_struct *vma,\n--\nmm/hugetlb.c-7451-\t\tptl = huge_pte_lock(h, mm, ptep);\nmm/hugetlb.c:7452:\t\t__huge_pmd_unshare(\u0026tlb, vma, address, ptep, take_locks);\nmm/hugetlb.c-7453-\t\tspin_unlock(ptl);\nmm/hugetlb.c-7454-\t}\nmm/hugetlb.c:7455:\thuge_pmd_unshare_flush(\u0026tlb, vma);\nmm/hugetlb.c-7456-\tif (take_locks) {\n--\nmm/khugepaged.c=679=static void count_collapse_event(unsigned int order, enum vm_event_item vm_event,\n--\nmm/khugepaged.c-686-\nmm/khugepaged.c:687:static enum scan_result __collapse_huge_page_isolate(struct vm_area_struct *vma,\nmm/khugepaged.c-688-\t\tunsigned long start_addr, hw_pte_t *pte,\n--\nmm/khugepaged.c-832-\t\tresult = SCAN_SUCCEED;\nmm/khugepaged.c:833:\t\ttrace_mm_collapse_huge_page_isolate(folio, none_or_zero,\nmm/khugepaged.c-834-\t\t\t\t\t\t referenced, result, order);\n--\nmm/khugepaged.c-838-\trelease_pte_pages(pte, _pte, compound_pagelist);\nmm/khugepaged.c:839:\ttrace_mm_collapse_huge_page_isolate(folio, none_or_zero,\nmm/khugepaged.c-840-\t\t\t\t\t referenced, result, order);\n--\nmm/khugepaged.c-843-\nmm/khugepaged.c:844:static void __collapse_huge_page_copy_succeeded(hw_pte_t *pte,\nmm/khugepaged.c-845-\t\t\t\t\t\tstruct vm_area_struct *vma,\n--\nmm/khugepaged.c-909-\nmm/khugepaged.c:910:static void __collapse_huge_page_copy_failed(hw_pte_t *pte,\nmm/khugepaged.c-911-\t\t\t\t\t pmd_t *pmd, pmd_t orig_pmd,\n--\nmm/khugepaged.c-929-\t * Release both raw and compound pages isolated\nmm/khugepaged.c:930:\t * in __collapse_huge_page_isolate.\nmm/khugepaged.c-931-\t */\n--\nmm/khugepaged.c-935-/*\nmm/khugepaged.c:936: * __collapse_huge_page_copy - attempts to copy memory contents from raw\nmm/khugepaged.c-937- * pages to a hugepage. Cleans up the raw pages if copying succeeds;\n--\nmm/khugepaged.c-949- */\nmm/khugepaged.c:950:static enum scan_result __collapse_huge_page_copy(hw_pte_t *pte,\nmm/khugepaged.c-951-\t\t\t\t\t\t struct folio *folio,\n--\nmm/khugepaged.c-983-\tif (likely(result == SCAN_SUCCEED))\nmm/khugepaged.c:984:\t\t__collapse_huge_page_copy_succeeded(pte, vma, address, ptl,\nmm/khugepaged.c-985-\t\t\t\t\t\t order, compound_pagelist);\nmm/khugepaged.c-986-\telse\nmm/khugepaged.c:987:\t\t__collapse_huge_page_copy_failed(pte, pmd, orig_pmd, vma,\nmm/khugepaged.c-988-\t\t\t\t\t\t order, compound_pagelist);\n--\nmm/khugepaged.c=1144=static enum scan_result check_pmd_still_valid(struct mm_struct *mm,\n--\nmm/khugepaged.c-1167- */\nmm/khugepaged.c:1168:static enum scan_result __collapse_huge_page_swapin(struct mm_struct *mm,\nmm/khugepaged.c-1169-\t\tstruct vm_area_struct *vma, unsigned long start_addr,\n--\nmm/khugepaged.c-1253-out:\nmm/khugepaged.c:1254:\ttrace_mm_collapse_huge_page_swapin(mm, swapped_in, referenced, result,\nmm/khugepaged.c-1255-\t\t\t\t\t order);\n--\nmm/khugepaged.c=1259=static enum scan_result alloc_charge_folio(struct folio **foliop, struct mm_struct *mm,\n--\nmm/khugepaged.c-1290-/*\nmm/khugepaged.c:1291: * collapse_huge_page() expects the mmap_lock to be unlocked before entering and\nmm/khugepaged.c-1292- * will always return with the lock unlocked, to avoid holding the mmap_lock\n--\nmm/khugepaged.c-1295- */\nmm/khugepaged.c:1296:static enum scan_result collapse_huge_page(struct mm_struct *mm, unsigned long start_addr,\nmm/khugepaged.c-1297-\t\tint referenced, int unmapped, struct collapse_control *cc,\n--\nmm/khugepaged.c-1337-\t\t/*\nmm/khugepaged.c:1338:\t\t * __collapse_huge_page_swapin() will return with mmap_lock\nmm/khugepaged.c-1339-\t\t * released when it fails. So we jump out_nolock directly in\n--\nmm/khugepaged.c-1341-\t\t */\nmm/khugepaged.c:1342:\t\tresult = __collapse_huge_page_swapin(mm, vma, start_addr, pmd,\nmm/khugepaged.c-1343-\t\t\t\t\t\t referenced, order);\n--\nmm/khugepaged.c-1397-\tif (pte) {\nmm/khugepaged.c:1398:\t\tresult = __collapse_huge_page_isolate(vma, start_addr, pte, cc,\nmm/khugepaged.c-1399-\t\t\t\t\t\t order, \u0026compound_pagelist);\n--\nmm/khugepaged.c-1429-\nmm/khugepaged.c:1430:\tresult = __collapse_huge_page_copy(pte, folio, pmd, _pmd,\nmm/khugepaged.c-1431-\t\t\t\t\t vma, start_addr, pte_ptl,\n--\nmm/khugepaged.c-1476-\t\tfolio_put(folio);\nmm/khugepaged.c:1477:\ttrace_mm_collapse_huge_page(mm, result == SCAN_SUCCEED, result, order);\nmm/khugepaged.c-1478-\treturn result;\n--\nmm/khugepaged.c=1507=static enum scan_result mthp_collapse(struct mm_struct *mm,\n--\nmm/khugepaged.c-1540-\t\t\tcollapse_address = address + offset * PAGE_SIZE;\nmm/khugepaged.c:1541:\t\t\tret = collapse_huge_page(mm, collapse_address, referenced,\nmm/khugepaged.c-1542-\t\t\t\t\t\t unmapped, cc, order);\n--\nmm/khugepaged.c=1609=static enum scan_result collapse_scan_pmd(struct mm_struct *mm,\n--\nmm/khugepaged.c-1796-\tif (result == SCAN_SUCCEED) {\nmm/khugepaged.c:1797:\t\t/* collapse_huge_page() expects the lock to be dropped before calling */\nmm/khugepaged.c-1798-\t\tmmap_read_unlock(mm);\n--\nmm/khugepaged.c=1836=static enum scan_result set_huge_pmd(struct vm_area_struct *vma, unsigned long addr,\n--\nmm/khugepaged.c-1871-\nmm/khugepaged.c:1872:static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr,\nmm/khugepaged.c-1873-\t\tbool install_pmd)\n--\nmm/khugepaged.c=2117=void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr,\n--\nmm/khugepaged.c-2119-{\nmm/khugepaged.c:2120:\ttry_collapse_pte_mapped_thp(mm, addr, install_pmd);\nmm/khugepaged.c-2121-}\n--\nmm/khugepaged.c=2270=static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,\n--\nmm/khugepaged.c-2651-\t * Remove pte page tables, so we can re-fault the page as huge.\nmm/khugepaged.c:2652:\t * If MADV_COLLAPSE, adjust result to call try_collapse_pte_mapped_thp().\nmm/khugepaged.c-2653-\t */\n--\nmm/khugepaged.c=2823=static enum scan_result collapse_single_pmd(unsigned long addr,\n--\nmm/khugepaged.c-2867-\t\telse\nmm/khugepaged.c:2868:\t\t\tresult = try_collapse_pte_mapped_thp(mm, addr,\nmm/khugepaged.c-2869-\t\t\t\t\t\t\t !cc-\u003eis_khugepaged);\n--\nmm/ksm.c=814=static struct page *get_mergeable_page(struct ksm_rmap_item *rmap_item)\n--\nmm/ksm.c-827-\nmm/ksm.c:828:\tfolio = folio_walk_start(\u0026fw, vma, addr, 0);\nmm/ksm.c-829-\tif (folio) {\n--\nmm/memory-failure.c=1568=int unmap_poisoned_folio(struct folio *folio, unsigned long pfn, bool must_kill)\n--\nmm/memory-failure.c-1598-\t\t * For hugetlb folios in shared mappings, try_to_unmap\nmm/memory-failure.c:1599:\t\t * could potentially call huge_pmd_unshare. Because of\nmm/memory-failure.c-1600-\t\t * this, take semaphore in write mode here and set\n--\nmm/migrate.c=1480=static int unmap_and_move_hugetlb_folio(new_folio_t get_new_folio,\n--\nmm/migrate.c-1533-\t\t\t * In shared mappings, try_to_unmap could potentially\nmm/migrate.c:1534:\t\t\t * call huge_pmd_unshare. Because of this, take\nmm/migrate.c-1535-\t\t\t * semaphore in write mode here and set TTU_RMAP_LOCKED\n--\nmm/migrate.c=2301=static int add_folio_for_migration(struct mm_struct *mm, const void __user *p,\n--\nmm/migrate.c-2314-\tif (vma \u0026\u0026 vma_migratable(vma)) {\nmm/migrate.c:2315:\t\tfolio = folio_walk_start(\u0026fw, vma, addr, FW_ZEROPAGE);\nmm/migrate.c-2316-\t\tif (folio) {\n--\nmm/migrate.c=2457=static void do_pages_stat_array(struct mm_struct *mm, unsigned long nr_pages,\n--\nmm/migrate.c-2474-\nmm/migrate.c:2475:\t\tfolio = folio_walk_start(\u0026fw, vma, addr, FW_ZEROPAGE);\nmm/migrate.c-2476-\t\tif (folio) {\n--\nmm/mmu_gather.c=515=void tlb_finish_mmu(struct mmu_gather *tlb)\n--\nmm/mmu_gather.c-517-\t/*\nmm/mmu_gather.c:518:\t * We expect an earlier huge_pmd_unshare_flush() call to sort this out,\nmm/mmu_gather.c-519-\t * due to complicated locking requirements with page table unsharing.\n--\nmm/pagewalk.c=798=int walk_page_mapping(struct address_space *mapping, pgoff_t first_index,\n--\nmm/pagewalk.c-848-/**\nmm/pagewalk.c:849: * folio_walk_start - walk the page tables to a folio\nmm/pagewalk.c-850- * @fw: filled with information on success.\n--\nmm/pagewalk.c-898- */\nmm/pagewalk.c:899:struct folio *folio_walk_start(struct folio_walk *fw,\nmm/pagewalk.c-900-\t\tstruct vm_area_struct *vma, unsigned long addr,\n--\nmm/rmap.c-26- * folio_lock\nmm/rmap.c:27: * hugetlbfs_i_mmap_rwsem_key (in huge_pmd_share, see hugetlbfs below)\nmm/rmap.c-28- * vma_start_write\n--\nmm/rmap.c=1982=static bool try_to_unmap_poisoned_hugetlb_one(struct folio *folio,\n--\nmm/rmap.c-2016-\t/*\nmm/rmap.c:2017:\t * huge_pmd_unshare may unmap an entire PMD page. There is no way of\nmm/rmap.c-2018-\t * knowing exactly which PMDs may be cached for this mm, so we must\n--\nmm/rmap.c-2024-\t/*\nmm/rmap.c:2025:\t * To call huge_pmd_unshare, i_mmap_rwsem must be held in write mode.\nmm/rmap.c-2026-\t * Caller needs to explicitly do this outside rmap routines.\n--\nmm/rmap.c-2041-\t\ttlb_gather_mmu_vma(\u0026tlb, vma);\nmm/rmap.c:2042:\t\tif (huge_pmd_unshare(\u0026tlb, vma, address, pvmw.pte)) {\nmm/rmap.c-2043-\t\t\thugetlb_vma_unlock_write(vma);\nmm/rmap.c:2044:\t\t\thuge_pmd_unshare_flush(\u0026tlb, vma);\nmm/rmap.c-2045-\t\t\ttlb_finish_mmu(\u0026tlb);\n--\nmm/rmap.c=2460=static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma,\n--\nmm/rmap.c-2574-\t\t\t/*\nmm/rmap.c:2575:\t\t\t * huge_pmd_unshare may unmap an entire PMD page.\nmm/rmap.c-2576-\t\t\t * There is no way of knowing exactly which PMDs may\n--\nmm/rmap.c-2583-\t\t\t/*\nmm/rmap.c:2584:\t\t\t * To call huge_pmd_unshare, i_mmap_rwsem must be\nmm/rmap.c-2585-\t\t\t * held in write mode. Caller needs to explicitly\n--\nmm/rmap.c-2603-\t\t\t\ttlb_gather_mmu_vma(\u0026tlb, vma);\nmm/rmap.c:2604:\t\t\t\tif (huge_pmd_unshare(\u0026tlb, vma, address, pvmw.pte)) {\nmm/rmap.c-2605-\t\t\t\t\thugetlb_vma_unlock_write(vma);\nmm/rmap.c:2606:\t\t\t\t\thuge_pmd_unshare_flush(\u0026tlb, vma);\nmm/rmap.c-2607-\t\t\t\t\ttlb_finish_mmu(\u0026tlb);\n--\nmm/rmap.c=2864=struct page *make_device_exclusive(struct mm_struct *mm, unsigned long addr,\n--\nmm/rmap.c-2921-\t */\nmm/rmap.c:2922:\tfw_folio = folio_walk_start(\u0026fw, vma, addr, 0);\nmm/rmap.c-2923-\tif (fw_folio != folio || fw.page != page ||\n--\nmm/userfaultfd.c=752=static __always_inline ssize_t mfill_atomic_hugetlb(\n--\nmm/userfaultfd.c-860-\nmm/userfaultfd.c:861:\t\terr = hugetlb_mfill_atomic_pte(dst_pte, dst_vma, dst_addr,\nmm/userfaultfd.c-862-\t\t\t\t\t src_addr, flags, \u0026folio);\n--\nmm/userfaultfd.c=2264=static int userfaultfd_register_range(struct userfaultfd_ctx *ctx,\n--\nmm/userfaultfd.c-2324-\nmm/userfaultfd.c:2325:\t\tif (is_vm_hugetlb_page(vma) \u0026\u0026 uffd_disable_huge_pmd_share(vma))\nmm/userfaultfd.c-2326-\t\t\thugetlb_unshare_all_pmds(vma);\n"
} |
| 9/2 | 2026/09/04 15:31 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 69225 output: 33 thoughts: 29 Thoughts: Here's my summary, as though I'm thinking through this process: **Diving into Page Table Validation** Okay, so I need to get a handle on how this page table validation is actually working. My first step is to dig into `mm/page_table_check.c`. Specifically, I'm going to focus on the `__page_table_check_ptes_set` function. That seems like a likely entry point for examining how individual Page Table Entries (PTEs) are being scrutinized. I need to understand the granular mechanics of the checks being performed. I'm hoping to understand what conditions trigger a validation failure and how that is handled. This function likely holds the key to the core logic of verifying PTE consistency. |
| 10/2 | 2026/09/04 15:31 | tool | grepper |
0mArgs:{
"Expression": "__page_table_check_ptes_set",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/page_table_check.c=199=static void page_table_check_pte_flags(pte_t pte)\n--\nmm/page_table_check.c-209-\nmm/page_table_check.c:210:void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr,\nmm/page_table_check.c-211-\t\t\t\t hw_pte_t *ptep, pte_t pte, unsigned int nr)\n--\nmm/page_table_check.c-224-}\nmm/page_table_check.c:225:EXPORT_SYMBOL(__page_table_check_ptes_set);\nmm/page_table_check.c-226-\n"
} |
| 11/2 | 2026/09/04 15:31 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 69644 output: 30 thoughts: 29 Thoughts: Okay, here's my interpretation of that thought process, assuming I'm deep in the weeds of kernel BPF development: **Hunting Down a Memory Allocation Issue in BPF Arena** Alright, I've hit a snag. Something isn't behaving as expected with memory allocation within the BPF arena. My gut tells me the issue is likely tied to either the arena management itself, or the allocation of the BPF maps used within that arena. So, my next step is clear: I need to dive into the core arena implementation. I'm going to jump straight into `kernel/bpf/arena.c`. Specifically, I'm going to zero in on `bpf_arena`, the primary arena structure, or maybe the function called `arena_map_alloc`, which handles allocation of memory within the arena's memory maps. These are the most likely suspects for whatever's gone wrong. Let's see what the code has to say. Time to dig in and understand what's happening under the hood. |
| 12/2 | 2026/09/04 15:32 | tool | grepper |
0mArgs:{
"Expression": "arena_map_alloc",
"PathPrefix": "kernel/bpf/"
}Results: {
"Output": "kernel/bpf/arena.c=260=static int populate_pgtable_except_pte(struct bpf_arena *arena)\n--\nkernel/bpf/arena.c-266-\nkernel/bpf/arena.c:267:static struct bpf_map *arena_map_alloc(union bpf_attr *attr)\nkernel/bpf/arena.c-268-{\n--\nkernel/bpf/arena.c=641=const struct bpf_map_ops arena_map_ops = {\nkernel/bpf/arena.c-642-\t.map_meta_equal = bpf_map_meta_equal,\nkernel/bpf/arena.c:643:\t.map_alloc = arena_map_alloc,\nkernel/bpf/arena.c-644-\t.map_free = arena_map_free,\n"
} |
| 13/2 | 2026/09/04 15:32 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 70142 output: 29 thoughts: 42 Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience: **Memory Management Deep Dive - Initial Thoughts** Alright, I need to get to the bottom of this memory-related issue. My gut tells me it's likely a problem within the kernel's memory management code, specifically related to how we handle page table entries and reference counts within the procfs interface. I should really start by looking into a few functions. My initial instinct is to dig into `pagemap_scan_pmd_entry`. This function's always tricky with its complex logic. Alternatively, `clear_refs_pte_range` could be at play, if we're seeing issues with reference counting or reclaiming pages. Honestly, any function in `fs/proc/task_mmu.c` is fair game here given how they interact with memory. Let's see what these functions can tell us. |
| 14/2 | 2026/09/04 15:32 | tool | grepper |
0mArgs:{
"Expression": "pagemap_scan",
"PathPrefix": "fs/proc/"
}Results: {
"Output": "fs/proc/task_mmu.c=2328=static int pagemap_release(struct inode *inode, struct file *file)\n--\nfs/proc/task_mmu.c-2343-\nfs/proc/task_mmu.c:2344:struct pagemap_scan_private {\nfs/proc/task_mmu.c-2345-\tstruct pm_scan_arg arg;\n--\nfs/proc/task_mmu.c-2351-\nfs/proc/task_mmu.c:2352:static unsigned long pagemap_page_category(struct pagemap_scan_private *p,\nfs/proc/task_mmu.c-2353-\t\t\t\t\t struct vm_area_struct *vma,\n--\nfs/proc/task_mmu.c-2361-\t\t * write-protected. The PAGE_IS_WRITTEN fast path in\nfs/proc/task_mmu.c:2362:\t\t * pagemap_scan_pmd_entry() is now gated on a VM_UFFD_WP VMA;\nfs/proc/task_mmu.c-2363-\t\t * gate the pte_none report here the same way so the two paths\n--\nfs/proc/task_mmu.c=2422=static void make_uffd_wp_pte(struct vm_area_struct *vma,\n--\nfs/proc/task_mmu.c-2440-#ifdef CONFIG_TRANSPARENT_HUGEPAGE\nfs/proc/task_mmu.c:2441:static unsigned long pagemap_thp_category(struct pagemap_scan_private *p,\nfs/proc/task_mmu.c-2442-\t\t\t\t\t struct vm_area_struct *vma,\n--\nfs/proc/task_mmu.c=2555=static void make_uffd_wp_huge_pte(struct vm_area_struct *vma,\n--\nfs/proc/task_mmu.c-2585-#if defined(CONFIG_TRANSPARENT_HUGEPAGE) || defined(CONFIG_HUGETLB_PAGE)\nfs/proc/task_mmu.c:2586:static void pagemap_scan_backout_range(struct pagemap_scan_private *p,\nfs/proc/task_mmu.c-2587-\t\t\t\t unsigned long addr, unsigned long end)\n--\nfs/proc/task_mmu.c-2602-\nfs/proc/task_mmu.c:2603:static bool pagemap_scan_is_interesting_page(unsigned long categories,\nfs/proc/task_mmu.c:2604:\t\t\t\t\t const struct pagemap_scan_private *p)\nfs/proc/task_mmu.c-2605-{\n--\nfs/proc/task_mmu.c-2614-\nfs/proc/task_mmu.c:2615:static bool pagemap_scan_is_interesting_vma(unsigned long categories,\nfs/proc/task_mmu.c:2616:\t\t\t\t\t const struct pagemap_scan_private *p)\nfs/proc/task_mmu.c-2617-{\n--\nfs/proc/task_mmu.c-2626-\nfs/proc/task_mmu.c:2627:static int pagemap_scan_test_walk(unsigned long start, unsigned long end,\nfs/proc/task_mmu.c-2628-\t\t\t\t struct mm_walk *walk)\nfs/proc/task_mmu.c-2629-{\nfs/proc/task_mmu.c:2630:\tstruct pagemap_scan_private *p = walk-\u003eprivate;\nfs/proc/task_mmu.c-2631-\tstruct vm_area_struct *vma = walk-\u003evma;\n--\nfs/proc/task_mmu.c-2660-\nfs/proc/task_mmu.c:2661:\tif (!pagemap_scan_is_interesting_vma(vma_category, p))\nfs/proc/task_mmu.c-2662-\t\treturn 1;\n--\nfs/proc/task_mmu.c-2668-\nfs/proc/task_mmu.c:2669:static bool pagemap_scan_push_range(unsigned long categories,\nfs/proc/task_mmu.c:2670:\t\t\t\t struct pagemap_scan_private *p,\nfs/proc/task_mmu.c-2671-\t\t\t\t unsigned long addr, unsigned long end)\n--\nfs/proc/task_mmu.c-2698-\nfs/proc/task_mmu.c:2699:static int pagemap_scan_output(unsigned long categories,\nfs/proc/task_mmu.c:2700:\t\t\t struct pagemap_scan_private *p,\nfs/proc/task_mmu.c-2701-\t\t\t unsigned long addr, unsigned long *end)\n--\nfs/proc/task_mmu.c-2719-\nfs/proc/task_mmu.c:2720:\tif (!pagemap_scan_push_range(categories, p, addr, *end)) {\nfs/proc/task_mmu.c-2721-\t\t*end = addr;\n--\nfs/proc/task_mmu.c-2732-\nfs/proc/task_mmu.c:2733:static int pagemap_scan_thp_entry(pmd_t *pmd, unsigned long start,\nfs/proc/task_mmu.c-2734-\t\t\t\t unsigned long end, struct mm_walk *walk)\n--\nfs/proc/task_mmu.c-2736-#ifdef CONFIG_TRANSPARENT_HUGEPAGE\nfs/proc/task_mmu.c:2737:\tstruct pagemap_scan_private *p = walk-\u003eprivate;\nfs/proc/task_mmu.c-2738-\tstruct vm_area_struct *vma = walk-\u003evma;\n--\nfs/proc/task_mmu.c-2749-\nfs/proc/task_mmu.c:2750:\tif (!pagemap_scan_is_interesting_page(categories, p))\nfs/proc/task_mmu.c-2751-\t\tgoto out_unlock;\nfs/proc/task_mmu.c-2752-\nfs/proc/task_mmu.c:2753:\tret = pagemap_scan_output(categories, p, start, \u0026end);\nfs/proc/task_mmu.c-2754-\tif (start == end)\n--\nfs/proc/task_mmu.c-2768-\t\tsplit_huge_pmd(vma, pmd, start);\nfs/proc/task_mmu.c:2769:\t\tpagemap_scan_backout_range(p, start, end);\nfs/proc/task_mmu.c-2770-\t\t/* Report as if there was no THP */\n--\nfs/proc/task_mmu.c-2783-\nfs/proc/task_mmu.c:2784:static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start,\nfs/proc/task_mmu.c-2785-\t\t\t\t unsigned long end, struct mm_walk *walk)\nfs/proc/task_mmu.c-2786-{\nfs/proc/task_mmu.c:2787:\tstruct pagemap_scan_private *p = walk-\u003eprivate;\nfs/proc/task_mmu.c-2788-\tstruct vm_area_struct *vma = walk-\u003evma;\n--\nfs/proc/task_mmu.c-2793-\nfs/proc/task_mmu.c:2794:\tret = pagemap_scan_thp_entry(pmd, start, end, walk);\nfs/proc/task_mmu.c-2795-\tif (ret != -ENOENT)\n--\nfs/proc/task_mmu.c-2833-\t\t\t\tcontinue;\nfs/proc/task_mmu.c:2834:\t\t\tret = pagemap_scan_output(p-\u003ecur_vma_category | PAGE_IS_WRITTEN,\nfs/proc/task_mmu.c-2835-\t\t\t\t\t\t p, addr, \u0026next);\n--\nfs/proc/task_mmu.c-2853-\nfs/proc/task_mmu.c:2854:\t\tif (!pagemap_scan_is_interesting_page(categories, p))\nfs/proc/task_mmu.c-2855-\t\t\tcontinue;\nfs/proc/task_mmu.c-2856-\nfs/proc/task_mmu.c:2857:\t\tret = pagemap_scan_output(categories, p, addr, \u0026next);\nfs/proc/task_mmu.c-2858-\t\tif (next == addr)\n--\nfs/proc/task_mmu.c-2883-#ifdef CONFIG_HUGETLB_PAGE\nfs/proc/task_mmu.c:2884:static int pagemap_scan_hugetlb_entry(hw_pte_t *ptep, unsigned long hmask,\nfs/proc/task_mmu.c-2885-\t\t\t\t unsigned long start, unsigned long end,\n--\nfs/proc/task_mmu.c-2887-{\nfs/proc/task_mmu.c:2888:\tstruct pagemap_scan_private *p = walk-\u003eprivate;\nfs/proc/task_mmu.c-2889-\tstruct vm_area_struct *vma = walk-\u003evma;\n--\nfs/proc/task_mmu.c-2901-\nfs/proc/task_mmu.c:2902:\t\tif (!pagemap_scan_is_interesting_page(categories, p))\nfs/proc/task_mmu.c-2903-\t\t\treturn 0;\nfs/proc/task_mmu.c-2904-\nfs/proc/task_mmu.c:2905:\t\treturn pagemap_scan_output(categories, p, start, \u0026end);\nfs/proc/task_mmu.c-2906-\t}\n--\nfs/proc/task_mmu.c-2913-\nfs/proc/task_mmu.c:2914:\tif (!pagemap_scan_is_interesting_page(categories, p))\nfs/proc/task_mmu.c-2915-\t\tgoto out_unlock;\nfs/proc/task_mmu.c-2916-\nfs/proc/task_mmu.c:2917:\tret = pagemap_scan_output(categories, p, start, \u0026end);\nfs/proc/task_mmu.c-2918-\tif (start == end)\n--\nfs/proc/task_mmu.c-2925-\t\t/* Partial HugeTLB page WP isn't possible. */\nfs/proc/task_mmu.c:2926:\t\tpagemap_scan_backout_range(p, start, end);\nfs/proc/task_mmu.c-2927-\t\tp-\u003earg.walk_end = start;\n--\nfs/proc/task_mmu.c-2943- * Write-protect the unpopulated hugetlb entries covering [addr, end) by\nfs/proc/task_mmu.c:2944: * installing uffd-wp markers inline, exactly as pagemap_scan_hugetlb_entry()\nfs/proc/task_mmu.c-2945- * does for populated entries.\n--\nfs/proc/task_mmu.c-2957- */\nfs/proc/task_mmu.c:2958:static int pagemap_scan_hugetlb_hole_wp(struct vm_area_struct *vma,\nfs/proc/task_mmu.c-2959-\t\t\t\t\tunsigned long addr, unsigned long end)\n--\nfs/proc/task_mmu.c-2990-#else\nfs/proc/task_mmu.c:2991:#define pagemap_scan_hugetlb_entry NULL\nfs/proc/task_mmu.c:2992:static int pagemap_scan_hugetlb_hole_wp(struct vm_area_struct *vma,\nfs/proc/task_mmu.c-2993-\t\t\t\t\tunsigned long addr, unsigned long end)\n--\nfs/proc/task_mmu.c-2998-\nfs/proc/task_mmu.c:2999:static int pagemap_scan_pte_hole(unsigned long addr, unsigned long end,\nfs/proc/task_mmu.c-3000-\t\t\t\t int depth, struct mm_walk *walk)\nfs/proc/task_mmu.c-3001-{\nfs/proc/task_mmu.c:3002:\tstruct pagemap_scan_private *p = walk-\u003eprivate;\nfs/proc/task_mmu.c-3003-\tstruct vm_area_struct *vma = walk-\u003evma;\n--\nfs/proc/task_mmu.c-3021-\nfs/proc/task_mmu.c:3022:\tif (!pagemap_scan_is_interesting_page(categories, p))\nfs/proc/task_mmu.c-3023-\t\treturn 0;\nfs/proc/task_mmu.c-3024-\nfs/proc/task_mmu.c:3025:\tret = pagemap_scan_output(categories, p, addr, \u0026end);\nfs/proc/task_mmu.c-3026-\tif (addr == end)\n--\nfs/proc/task_mmu.c-3032-\tif (is_vm_hugetlb_page(vma))\nfs/proc/task_mmu.c:3033:\t\terr = pagemap_scan_hugetlb_hole_wp(vma, addr, end);\nfs/proc/task_mmu.c-3034-\telse\n--\nfs/proc/task_mmu.c-3041-\nfs/proc/task_mmu.c:3042:static const struct mm_walk_ops pagemap_scan_ops = {\nfs/proc/task_mmu.c:3043:\t.test_walk = pagemap_scan_test_walk,\nfs/proc/task_mmu.c:3044:\t.pmd_entry = pagemap_scan_pmd_entry,\nfs/proc/task_mmu.c:3045:\t.pte_hole = pagemap_scan_pte_hole,\nfs/proc/task_mmu.c:3046:\t.hugetlb_entry = pagemap_scan_hugetlb_entry,\nfs/proc/task_mmu.c-3047-};\nfs/proc/task_mmu.c-3048-\nfs/proc/task_mmu.c:3049:static int pagemap_scan_get_args(struct pm_scan_arg *arg,\nfs/proc/task_mmu.c-3050-\t\t\t\t unsigned long uarg)\n--\nfs/proc/task_mmu.c-3090-\nfs/proc/task_mmu.c:3091:static int pagemap_scan_writeback_args(struct pm_scan_arg *arg,\nfs/proc/task_mmu.c-3092-\t\t\t\t unsigned long uargl)\n--\nfs/proc/task_mmu.c-3101-\nfs/proc/task_mmu.c:3102:static int pagemap_scan_init_bounce_buffer(struct pagemap_scan_private *p)\nfs/proc/task_mmu.c-3103-{\n--\nfs/proc/task_mmu.c-3118-\nfs/proc/task_mmu.c:3119:static long pagemap_scan_flush_buffer(struct pagemap_scan_private *p)\nfs/proc/task_mmu.c-3120-{\n--\nfs/proc/task_mmu.c-3145-\nfs/proc/task_mmu.c:3146:static long do_pagemap_scan(struct mm_struct *mm, unsigned long uarg)\nfs/proc/task_mmu.c-3147-{\nfs/proc/task_mmu.c:3148:\tstruct pagemap_scan_private p = {0};\nfs/proc/task_mmu.c-3149-\tunsigned long walk_start;\n--\nfs/proc/task_mmu.c-3152-\nfs/proc/task_mmu.c:3153:\tret = pagemap_scan_get_args(\u0026p.arg, uarg);\nfs/proc/task_mmu.c-3154-\tif (ret)\n--\nfs/proc/task_mmu.c-3158-\t\t\t p.arg.return_mask;\nfs/proc/task_mmu.c:3159:\tret = pagemap_scan_init_bounce_buffer(\u0026p);\nfs/proc/task_mmu.c-3160-\tif (ret)\n--\nfs/proc/task_mmu.c-3184-\t\tret = walk_page_range(mm, walk_start, p.arg.end,\nfs/proc/task_mmu.c:3185:\t\t\t\t \u0026pagemap_scan_ops, \u0026p);\nfs/proc/task_mmu.c-3186-\n--\nfs/proc/task_mmu.c-3191-\nfs/proc/task_mmu.c:3192:\t\tn_out = pagemap_scan_flush_buffer(\u0026p);\nfs/proc/task_mmu.c-3193-\t\tif (n_out \u003c 0)\n--\nfs/proc/task_mmu.c-3211-\t\tp.arg.walk_end = p.arg.end;\nfs/proc/task_mmu.c:3212:\tif (pagemap_scan_writeback_args(\u0026p.arg, uarg))\nfs/proc/task_mmu.c-3213-\t\tret = -EFAULT;\n--\nfs/proc/task_mmu.c=3219=static long do_pagemap_cmd(struct file *file, unsigned int cmd,\n--\nfs/proc/task_mmu.c-3225-\tcase PAGEMAP_SCAN:\nfs/proc/task_mmu.c:3226:\t\treturn do_pagemap_scan(mm, arg);\nfs/proc/task_mmu.c-3227-\n"
} |
| 15/2 | 2026/09/04 15:32 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 75162 output: 42 thoughts: 239 Thoughts: **Kconfig Requirements for My Code Modifications** Okay, let's make sure I've covered all the bases regarding Kconfig dependencies for this new code. First, the introduction of `CONFIG_ARCH_HAS_HW_PTE_T` in `mm/Kconfig` is key, though it's not enabled by default on amd64. I should double-check that it doesn't have any hidden dependencies. It looks like it's just a boolean option, which is good. Then, there's the `hugetlb` aspect. I need to ensure that `CONFIG_HUGETLB_PAGE` and `CONFIG_HUGETLBFS` are appropriately handled. For Transparent Huge Pages, `CONFIG_TRANSPARENT_HUGEPAGE` is definitely a requirement, specifically for the khugepaged and THP features I'm working with. Furthermore, I have to consider `CONFIG_USERFAULTFD` since the patch is touching userfaultfd and pagemap scan uffd wp functionality, as well as `CONFIG_PAGE_TABLE_CHECK` in relation to page table checking. The new arena/bpf interactions mean I need `CONFIG_BPF_SYSCALL` as well. And finally, for the task_mmu.c (/proc/pid/pagemap etc) section, it looks like `CONFIG_PROC_PAGE_MONITOR` is necessary. I'll take a look at the definitions for both `PROC_PAGE_MONITOR` and `PAGE_TABLE_CHECK` for clarity. |
| 16/2 | 2026/09/04 15:32 | tool | grepper |
0mArgs:{
"Expression": "config (PROC_PAGE_MONITOR|PAGE_TABLE_CHECK|HUGETLBFS)",
"PathPrefix": "fs/"
}Results: {
"Output": "fs/Kconfig=247=config ARCH_SUPPORTS_HUGETLBFS\n--\nfs/Kconfig-249-\nfs/Kconfig:250:menuconfig HUGETLBFS\nfs/Kconfig-251-\tbool \"HugeTLB file system support\"\n--\nfs/proc/Kconfig=83=config SYSCTL\n--\nfs/proc/Kconfig-100-\nfs/proc/Kconfig:101:config PROC_PAGE_MONITOR\nfs/proc/Kconfig-102- \tdefault y\n"
} |
| 17/2 | 2026/09/04 15:32 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 75917 output: 28 thoughts: 18 Thoughts: Here's my take on those thoughts: **Navigating the Memory Management Labyrinth** Okay, so I'm thinking, where do I even begin with this memory issue? Right, I need to get granular. The first thing that pops into my head is to dive into the kernel configuration. I should probably start by checking the `PAGE_TABLE_CHECK` option within the `mm/Kconfig` file. That seems like a logical first step to understand the level of page table verification being enabled or disabled. This would immediately tell me if any inherent checks are at play, or if there's an issue with the configuration itself. Getting a handle on how the page tables are being treated at a foundational level will likely lead me down the right path. |
| 18/2 | 2026/09/04 15:32 | tool | grepper |
0mArgs:{
"Expression": "config PAGE_TABLE_CHECK",
"PathPrefix": "mm/"
}Results: {
"Output": "mm/Kconfig.debug=105=config PAGE_OWNER\n--\nmm/Kconfig.debug-121-\nmm/Kconfig.debug:122:config PAGE_TABLE_CHECK\nmm/Kconfig.debug-123-\tbool \"Check for invalid mappings in user page tables\"\n--\nmm/Kconfig.debug-135-\nmm/Kconfig.debug:136:config PAGE_TABLE_CHECK_ENFORCED\nmm/Kconfig.debug-137-\tbool \"Enforce the page table checking by default\"\n"
} |
| 19/2 | 2026/09/04 15:32 | llm | patch-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 76265 output: 280 thoughts: 510 Thoughts: **My Analysis of the Proposed Memory Management Changes** Okay, let's get down to brass tacks. I've been asked to review this proposed patch, and my initial thought is, it's pretty substantial, touching core memory management. Right, first things first, let's nail down the critical focus symbols. I've identified: `hugetlb_mfill_atomic_pte`, `copy_hugetlb_page_range`, `move_hugetlb_page_tables`, `huge_pmd_share`, `huge_pmd_unshare`, `collapse_huge_page`, `try_collapse_pte_mapped_thp`, `folio_walk_start`, `__page_table_check_ptes_set`, and `do_pagemap_scan`. I know these pretty well, but it's always good to double-check their specific locations, just to be sure: `mm/hugetlb.c`, `mm/khugepaged.c`, `mm/pagewalk.c`, `mm/page_table_check.c`, and `fs/proc/task_mmu.c`. Got it. Next, I need to make sure the build environment is set correctly, so I'll check for the necessary configurations: `HUGETLBFS`, `TRANSPARENT_HUGEPAGE`, `USERFAULTFD`, `PROC_PAGE_MONITOR`, `PAGE_TABLE_CHECK`, and `BPF_SYSCALL`. Now, the core of the matter: This patch is introducing `hw_pte_t`, a new type abstraction. The idea is to clearly distinguish software `pte_t` values from the hardware or in-memory page table entries, which are `hw_pte_t *`. This sounds reasonable on the surface, but refactoring function signatures and PTE access across such a broad swath of the kernel – hugetlb, khugepaged, userfaultfd, pagewalk, page_table_check, rmap, proc task_mmu – is where the potential gotchas lie. This is core reachable memory management logic. We're talking about fundamental changes here, which is the perfect recipe for introducing subtle, but potentially devastating, regressions or invariant violations. This is a prime candidate for thorough fuzzing. My conclusion? This needs a deep dive, and a heavy dose of fuzzing. I'll need to prepare a detailed fuzzing plan. Let's call `set-results`. |
| 20/2 | 2026/09/04 15:32 | tool | set-results |
0mArgs:{
"EnableConfigs": [
"HUGETLBFS",
"TRANSPARENT_HUGEPAGE",
"USERFAULTFD",
"PROC_PAGE_MONITOR",
"PAGE_TABLE_CHECK",
"BPF_SYSCALL"
],
"FocusSymbols": [
"hugetlb_mfill_atomic_pte",
"copy_hugetlb_page_range",
"move_hugetlb_page_tables",
"huge_pmd_share",
"huge_pmd_unshare",
"collapse_huge_page",
"try_collapse_pte_mapped_thp",
"folio_walk_start",
"__page_table_check_ptes_set",
"do_pagemap_scan"
],
"Reasoning": "The patch introduces a new hw_pte_t type abstraction and converts raw pte_t* pointers across core memory management subsystems (hugetlb, khugepaged, pagewalk, userfaultfd, page_table_check, proc task_mmu, and bpf arena) to hw_pte_t*. Because this modifies core reachable memory management interfaces and data flow, it is worth fuzzing to detect regressions or unintended side effects.",
"WorthFuzzing": true
}Results: {
"EnableConfigs": [
"HUGETLBFS",
"TRANSPARENT_HUGEPAGE",
"USERFAULTFD",
"PROC_PAGE_MONITOR",
"PAGE_TABLE_CHECK",
"BPF_SYSCALL"
],
"FocusSymbols": [
"hugetlb_mfill_atomic_pte",
"copy_hugetlb_page_range",
"move_hugetlb_page_tables",
"huge_pmd_share",
"huge_pmd_unshare",
"collapse_huge_page",
"try_collapse_pte_mapped_thp",
"folio_walk_start",
"__page_table_check_ptes_set",
"do_pagemap_scan"
],
"Reasoning": "The patch introduces a new hw_pte_t type abstraction and converts raw pte_t* pointers across core memory management subsystems (hugetlb, khugepaged, pagewalk, userfaultfd, page_table_check, proc task_mmu, and bpf arena) to hw_pte_t*. Because this modifies core reachable memory management interfaces and data flow, it is worth fuzzing to detect regressions or unintended side effects.",
"WorthFuzzing": true
} |
| 21/1 | 2026/09/04 15:32 | action | If |
0mArgs:{
"WorthFuzzing": true
}Results: null |
| 22/2 | 2026/09/04 15:32 | agent | kmsan-evaluator |
0mModel:core Args: null Results: {
"KMSANReasoning": "The patch series is a purely mechanical type-system refactoring across the memory management subsystem and arch-independent code. It introduces the `hw_pte_t` type (and `__pte_from_hw()` helper) to distinguish between pointers directly referencing hardware page table entries (`hw_pte_t *`) and page table values manipulated in software (`pte_t`).\n\nNo changes are made to data initialization, memory allocation semantics, struct layouts exposed to userspace, or data copying mechanisms (e.g. `copy_to_user`). There are no new branching conditions on uninitialized memory, no modifications to bounds checks, and no risk of uninitialized memory reads or information leaks to userspace. Therefore, dedicated KMSAN fuzzing is not required.",
"NeedsKMSAN": false
}Instruction: You are an expert Linux kernel security engineer specializing in kernel memory
error detectors (KASAN and KMSAN). Your job is to review the provided patch series and
determine if the code changes justify spawning a dedicated KMSAN (KernelMemorySanitizer)
fuzzing session in addition to standard KASAN fuzzing.
CRITICAL DISTINCTION BETWEEN KASAN AND KMSAN:
- Standard KASAN kernel builds (upstream-apparmor-kasan.config) already enable
a comprehensive suite of debugging tools and sanitizers, including KASAN
(out-of-bounds accesses, use-after-free, double free, invalid free), LOCKDEP
(locking bugs and deadlocks), UB-sanitizers, and memory corruption checks.
- KMSAN (KernelMemorySanitizer) detects reads of UNINITIALIZED memory (stack, heap,
or page allocations) and kernel-to-user memory info-leaks.
Rule: THERE IS NO SENSE IN RUNNING A KMSAN SESSION IF A BUG CAN BE CAUGHT BY KASAN,
LOCKDEP, OR OTHER STANDARD BUG DETECTORS.
A dedicated KMSAN fuzzing session incurs significant resource costs. You must ONLY
set NeedsKMSAN=true if the code changes introduce or expose UNINITIALIZED MEMORY risks
that are detected ONLY by KMSAN.
Look holistically at the patch series and surrounding code. Even if no direct
uninitialized field accesses or new buffer allocations are added in the diff itself,
a patch may alter control flow, bounds checking, or data length calculations in ways
that change how the rest of the code operates on existing buffers (e.g. allowing
uninitialized stack/heap memory to be read, copied to user space, or used in control
flow). Do not hesitate to use your code access tools to inspect the surrounding code,
called functions, and callers.
Set NeedsKMSAN=true ONLY IF the patch introduces or modifies:
1. Kernel structures sent to user space (via copy_to_user, put_user, netlink skb
attributes, ioctl output arguments, socket options, or BPF buffers) where fields
or structure padding might not be fully initialized/zeroed.
2. Conditional logic or branching that depends on potentially uninitialized variables
or struct fields.
3. Allocation or initialization of complex data structures where uninitialized fields
could be read later in reachable code paths.
4. Bounds checks, lengths, or logic in a way that allows surrounding code to access
uninitialized bytes of existing buffers.
Set NeedsKMSAN=false IF:
- The code changes primarily risk out-of-bounds access, array overflows, NULL pointer
dereferences, locking deadlocks, or use-after-free bugs (these are already caught
by KASAN, LOCKDEP, or standard bug detectors).
- All stack/heap structures touched or introduced by the patch are fully zeroed
or initialized (e.g. using = {0}, memset, kzalloc) before being read or copied.
- The patch does not introduce any risk of uninitialized memory usage or info-leaks.
Use your code access tools to inspect the surrounding code if necessary, then provide
detailed KMSANReasoning contrasting KASAN vs KMSAN applicability for this patch.
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Target architecture: amd64
For your convenience, here is the diff of the changes:
commit e8ceacb10b59eb9d38e460ed31a10da5b37fe459
Author: syz-cluster <triage@syzkaller.com>
Date: Fri Sep 4 15:30:59 2026 +0000
syz-cluster: applied patch under review
diff --git a/MAINTAINERS b/MAINTAINERS
index a9245d827ddb6..b0ce3e6693045 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -17140,6 +17140,7 @@ F: include/linux/mmu_notifier.h
F: include/linux/pagewalk.h
F: include/linux/pgalloc.h
F: include/linux/pgtable.h
+F: include/linux/pgtable_types.h
F: include/linux/ptdump.h
F: include/linux/vmpressure.h
F: include/linux/vmstat.h
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
index d01acfb7d93d0..056faf4a3618b 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
@@ -1690,7 +1690,7 @@ static int igt_mmap_gpu(void *arg)
return 0;
}
-static int check_present_pte(pte_t *pte, unsigned long addr, void *data)
+static int check_present_pte(hw_pte_t *pte, unsigned long addr, void *data)
{
pte_t ptent = ptep_get(pte);
@@ -1703,7 +1703,7 @@ static int check_present_pte(pte_t *pte, unsigned long addr, void *data)
return 0;
}
-static int check_absent_pte(pte_t *pte, unsigned long addr, void *data)
+static int check_absent_pte(hw_pte_t *pte, unsigned long addr, void *data)
{
pte_t ptent = ptep_get(pte);
diff --git a/drivers/gpu/drm/i915/i915_mm.c b/drivers/gpu/drm/i915/i915_mm.c
index fd89e7c7d8d6f..aab88e8edf946 100644
--- a/drivers/gpu/drm/i915/i915_mm.c
+++ b/drivers/gpu/drm/i915/i915_mm.c
@@ -48,7 +48,7 @@ static inline unsigned long sgt_pfn(const struct remap_pfn *r)
return r->sgt.pfn + (r->sgt.curr >> PAGE_SHIFT);
}
-static int remap_sg(pte_t *pte, unsigned long addr, void *data)
+static int remap_sg(hw_pte_t *pte, unsigned long addr, void *data)
{
struct remap_pfn *r = data;
@@ -70,7 +70,7 @@ static int remap_sg(pte_t *pte, unsigned long addr, void *data)
#define EXPECTED_FLAGS (VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP)
#if IS_ENABLED(CONFIG_X86)
-static int remap_pfn(pte_t *pte, unsigned long addr, void *data)
+static int remap_pfn(hw_pte_t *pte, unsigned long addr, void *data)
{
struct remap_pfn *r = data;
diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
index 1dcc4675580ed..b013bcad99b5b 100644
--- a/drivers/xen/gntdev.c
+++ b/drivers/xen/gntdev.c
@@ -301,7 +301,7 @@ void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
/* ------------------------------------------------------------------ */
-static int find_grant_ptes(pte_t *pte, unsigned long addr, void *data)
+static int find_grant_ptes(hw_pte_t *pte, unsigned long addr, void *data)
{
struct gntdev_grant_map *map = data;
unsigned int pgnr = (addr - map->pages_vm_start) >> PAGE_SHIFT;
diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index 7cfc28f1bb86f..e724cc053e264 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -1658,7 +1658,7 @@ static int privcmd_mmap(struct file *file, struct vm_area_struct *vma)
* on a per pfn/pte basis. Mapping calls that fail with ENOENT
* can be then retried until success.
*/
-static int is_mapped_fn(pte_t *pte, unsigned long addr, void *data)
+static int is_mapped_fn(hw_pte_t *pte, unsigned long addr, void *data)
{
return pte_none(ptep_get(pte)) ? 0 : -EBUSY;
}
diff --git a/drivers/xen/xenbus/xenbus_client.c b/drivers/xen/xenbus/xenbus_client.c
index 27682cb5e58ab..387f97a555594 100644
--- a/drivers/xen/xenbus/xenbus_client.c
+++ b/drivers/xen/xenbus/xenbus_client.c
@@ -749,7 +749,7 @@ int xenbus_unmap_ring_vfree(struct xenbus_device *dev, void *vaddr)
EXPORT_SYMBOL_GPL(xenbus_unmap_ring_vfree);
#ifdef CONFIG_XEN_PV
-static int map_ring_apply(pte_t *pte, unsigned long addr, void *data)
+static int map_ring_apply(hw_pte_t *pte, unsigned long addr, void *data)
{
struct map_ring_valloc *info = data;
diff --git a/drivers/xen/xlate_mmu.c b/drivers/xen/xlate_mmu.c
index 8efd7b55223fb..d7244f9f9a545 100644
--- a/drivers/xen/xlate_mmu.c
+++ b/drivers/xen/xlate_mmu.c
@@ -93,7 +93,7 @@ static void setup_hparams(unsigned long gfn, void *data)
info->fgfn++;
}
-static int remap_pte_fn(pte_t *ptep, unsigned long addr, void *data)
+static int remap_pte_fn(hw_pte_t *ptep, unsigned long addr, void *data)
{
struct remap_data *info = data;
struct page *page = info->pages[info->index++];
@@ -269,7 +269,7 @@ struct remap_pfn {
unsigned long i;
};
-static int remap_pfn_fn(pte_t *ptep, unsigned long addr, void *data)
+static int remap_pfn_fn(hw_pte_t *ptep, unsigned long addr, void *data)
{
struct remap_pfn *r = data;
struct page *page = r->pages[r->i];
diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index 7611a8470ea26..ddaab714c3e02 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -328,7 +328,8 @@ static void hugetlb_delete_from_page_cache(struct folio *folio)
static bool hugetlb_vma_maps_pfn(struct vm_area_struct *vma,
unsigned long addr, unsigned long pfn)
{
- pte_t *ptep, pte;
+ hw_pte_t *ptep;
+ pte_t pte;
ptep = hugetlb_walk(vma, addr, huge_page_size(hstate_vma(vma)));
if (!ptep)
diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index e671b4fd8dedd..4211c33b06491 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -982,7 +982,7 @@ static void smaps_pte_hole_lookup(unsigned long addr, struct mm_walk *walk)
#endif
}
-static void smaps_pte_entry(pte_t *pte, unsigned long addr,
+static void smaps_pte_entry(hw_pte_t *pte, unsigned long addr,
struct mm_walk *walk)
{
struct mem_size_stats *mss = walk->private;
@@ -1076,7 +1076,7 @@ static int smaps_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
struct vm_area_struct *vma = walk->vma;
- pte_t *pte;
+ hw_pte_t *pte;
spinlock_t *ptl;
ptl = pmd_trans_huge_lock(pmd, vma);
@@ -1199,7 +1199,7 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma)
}
#ifdef CONFIG_HUGETLB_PAGE
-static int smaps_hugetlb_range(pte_t *pte, unsigned long hmask,
+static int smaps_hugetlb_range(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
@@ -1622,7 +1622,7 @@ static inline bool pte_is_pinned(struct vm_area_struct *vma, unsigned long addr,
}
static inline void clear_soft_dirty(struct vm_area_struct *vma,
- unsigned long addr, pte_t *pte)
+ unsigned long addr, hw_pte_t *pte)
{
if (!pgtable_supports_soft_dirty())
return;
@@ -1690,7 +1690,8 @@ static int clear_refs_pte_range(pmd_t *pmd, unsigned long addr,
{
struct clear_refs_private *cp = walk->private;
struct vm_area_struct *vma = walk->vma;
- pte_t *pte, ptent;
+ hw_pte_t *pte;
+ pte_t ptent;
spinlock_t *ptl;
struct folio *folio;
@@ -2090,7 +2091,7 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end,
struct vm_area_struct *vma = walk->vma;
struct pagemapread *pm = walk->private;
spinlock_t *ptl;
- pte_t *pte, *orig_pte;
+ hw_pte_t *pte, *orig_pte;
int err = 0;
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
@@ -2128,7 +2129,7 @@ static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned long end,
#ifdef CONFIG_HUGETLB_PAGE
/* This function walks within one hugetlb entry in the single call */
-static int pagemap_hugetlb_range(pte_t *ptep, unsigned long hmask,
+static int pagemap_hugetlb_range(hw_pte_t *ptep, unsigned long hmask,
unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
@@ -2419,7 +2420,7 @@ static unsigned long pagemap_page_category(struct pagemap_scan_private *p,
}
static void make_uffd_wp_pte(struct vm_area_struct *vma,
- unsigned long addr, pte_t *pte, pte_t ptent)
+ unsigned long addr, hw_pte_t *pte, pte_t ptent)
{
if (pte_present(ptent)) {
pte_t old_pte;
@@ -2552,7 +2553,7 @@ static unsigned long pagemap_hugetlb_category(struct vm_area_struct *vma,
}
static void make_uffd_wp_huge_pte(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep,
+ unsigned long addr, hw_pte_t *ptep,
pte_t ptent)
{
const unsigned long psize = huge_page_size(hstate_vma(vma));
@@ -2786,7 +2787,7 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start,
struct pagemap_scan_private *p = walk->private;
struct vm_area_struct *vma = walk->vma;
unsigned long addr, flush_end = 0;
- pte_t *pte, *start_pte;
+ hw_pte_t *pte, *start_pte;
spinlock_t *ptl;
int ret;
@@ -2880,7 +2881,7 @@ static int pagemap_scan_pmd_entry(pmd_t *pmd, unsigned long start,
}
#ifdef CONFIG_HUGETLB_PAGE
-static int pagemap_scan_hugetlb_entry(pte_t *ptep, unsigned long hmask,
+static int pagemap_scan_hugetlb_entry(hw_pte_t *ptep, unsigned long hmask,
unsigned long start, unsigned long end,
struct mm_walk *walk)
{
@@ -2961,7 +2962,7 @@ static int pagemap_scan_hugetlb_hole_wp(struct vm_area_struct *vma,
unsigned long psize = huge_page_size(h);
struct mm_struct *mm = vma->vm_mm;
spinlock_t *ptl;
- pte_t *ptep;
+ hw_pte_t *ptep;
pte_t pte;
for (addr = ALIGN_DOWN(addr, psize); addr < end; addr += psize) {
@@ -3345,8 +3346,8 @@ static int gather_pte_stats(pmd_t *pmd, unsigned long addr,
struct numa_maps *md = walk->private;
struct vm_area_struct *vma = walk->vma;
spinlock_t *ptl;
- pte_t *orig_pte;
- pte_t *pte;
+ hw_pte_t *orig_pte;
+ hw_pte_t *pte;
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
ptl = pmd_trans_huge_lock(pmd, vma);
@@ -3379,7 +3380,7 @@ static int gather_pte_stats(pmd_t *pmd, unsigned long addr,
return 0;
}
#ifdef CONFIG_HUGETLB_PAGE
-static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask,
+static int gather_hugetlb_stats(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long end, struct mm_walk *walk)
{
pte_t huge_pte;
@@ -3402,7 +3403,7 @@ static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask,
}
#else
-static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask,
+static int gather_hugetlb_stats(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long end, struct mm_walk *walk)
{
return 0;
diff --git a/include/asm-generic/hugetlb.h b/include/asm-generic/hugetlb.h
index 635c41cc34797..3bfad4a99b64d 100644
--- a/include/asm-generic/hugetlb.h
+++ b/include/asm-generic/hugetlb.h
@@ -60,7 +60,7 @@ static inline int huge_pte_uffd(pte_t pte)
#ifndef __HAVE_ARCH_HUGE_PTE_CLEAR
static inline void huge_pte_clear(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, unsigned long sz)
+ hw_pte_t *ptep, unsigned long sz)
{
pte_clear(mm, addr, ptep);
}
@@ -68,7 +68,7 @@ static inline void huge_pte_clear(struct mm_struct *mm, unsigned long addr,
#ifndef __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT
static inline void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t pte, unsigned long sz)
+ hw_pte_t *ptep, pte_t pte, unsigned long sz)
{
set_pte_at(mm, addr, ptep, pte);
}
@@ -76,7 +76,7 @@ static inline void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
#ifndef __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR
static inline pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep, unsigned long sz)
+ unsigned long addr, hw_pte_t *ptep, unsigned long sz)
{
return ptep_get_and_clear(mm, addr, ptep);
}
@@ -84,7 +84,7 @@ static inline pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
#ifndef __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH
static inline pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep)
+ unsigned long addr, hw_pte_t *ptep)
{
return ptep_clear_flush(vma, addr, ptep);
}
@@ -99,7 +99,7 @@ static inline int huge_pte_none(pte_t pte)
#ifndef __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT
static inline void huge_ptep_set_wrprotect(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep)
+ unsigned long addr, hw_pte_t *ptep)
{
ptep_set_wrprotect(mm, addr, ptep);
}
@@ -107,7 +107,7 @@ static inline void huge_ptep_set_wrprotect(struct mm_struct *mm,
#ifndef __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS
static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep,
+ unsigned long addr, hw_pte_t *ptep,
pte_t pte, int dirty)
{
return ptep_set_access_flags(vma, addr, ptep, pte, dirty);
@@ -115,7 +115,8 @@ static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma,
#endif
#ifndef __HAVE_ARCH_HUGE_PTEP_GET
-static inline pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep)
+static inline pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr,
+ hw_pte_t *ptep)
{
return ptep_get(ptep);
}
diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h
index 051aa1331051c..b35e5a2158ada 100644
--- a/include/asm-generic/pgalloc.h
+++ b/include/asm-generic/pgalloc.h
@@ -16,7 +16,7 @@
*
* Return: pointer to the allocated memory or %NULL on error
*/
-static inline pte_t *__pte_alloc_one_kernel_noprof(struct mm_struct *mm)
+static inline hw_pte_t *__pte_alloc_one_kernel_noprof(struct mm_struct *mm)
{
struct ptdesc *ptdesc = pagetable_alloc_noprof(GFP_PGTABLE_KERNEL, 0);
@@ -40,7 +40,7 @@ static inline pte_t *__pte_alloc_one_kernel_noprof(struct mm_struct *mm)
*
* Return: pointer to the allocated memory or %NULL on error
*/
-static inline pte_t *pte_alloc_one_kernel_noprof(struct mm_struct *mm)
+static inline hw_pte_t *pte_alloc_one_kernel_noprof(struct mm_struct *mm)
{
return __pte_alloc_one_kernel_noprof(mm);
}
@@ -52,7 +52,7 @@ static inline pte_t *pte_alloc_one_kernel_noprof(struct mm_struct *mm)
* @mm: the mm_struct of the current context
* @pte: pointer to the memory containing the page table
*/
-static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)
+static inline void pte_free_kernel(struct mm_struct *mm, hw_pte_t *pte)
{
pagetable_dtor_free(virt_to_ptdesc(pte));
}
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h
index bdcc2778ac64f..2c3517800a9a8 100644
--- a/include/asm-generic/tlb.h
+++ b/include/asm-generic/tlb.h
@@ -644,7 +644,8 @@ static inline void tlb_flush_p4d_range(struct mmu_gather *tlb,
}
#ifndef __tlb_remove_tlb_entry
-static inline void __tlb_remove_tlb_entry(struct mmu_gather *tlb, pte_t *ptep, unsigned long address)
+static inline void __tlb_remove_tlb_entry(struct mmu_gather *tlb,
+ hw_pte_t *ptep, unsigned long address)
{
}
#endif
@@ -670,7 +671,7 @@ static inline void __tlb_remove_tlb_entry(struct mmu_gather *tlb, pte_t *ptep, u
* consecutive ptes instead of only a single one.
*/
static inline void tlb_remove_tlb_entries(struct mmu_gather *tlb,
- pte_t *ptep, unsigned int nr, unsigned long address)
+ hw_pte_t *ptep, unsigned int nr, unsigned long address)
{
tlb_flush_pte_range(tlb, address, PAGE_SIZE * nr);
for (;;) {
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index 80a5a03e9cee7..6bbf2449e7d27 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -141,7 +141,7 @@ unsigned long hugetlb_total_pages(void);
vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma,
unsigned long address, unsigned int flags);
#ifdef CONFIG_USERFAULTFD
-int hugetlb_mfill_atomic_pte(pte_t *dst_pte,
+int hugetlb_mfill_atomic_pte(hw_pte_t *dst_pte,
struct vm_area_struct *dst_vma,
unsigned long dst_addr,
unsigned long src_addr,
@@ -161,7 +161,7 @@ void hugetlb_fix_reserve_counts(struct inode *inode);
extern struct mutex *hugetlb_fault_mutex_table;
u32 hugetlb_fault_mutex_hash(struct address_space *mapping, pgoff_t idx);
-pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,
+hw_pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,
unsigned long addr, pud_t *pud);
bool hugetlbfs_pagecache_present(struct hstate *h,
struct vm_area_struct *vma,
@@ -185,22 +185,22 @@ void hugetlb_bootmem_set_nodes(void);
* which may go down to the lowest PTE level in their huge_pte_offset() and
* huge_pte_alloc(): to avoid reliance on pte_offset_map() without pte_unmap().
*/
-static inline pte_t *pte_offset_huge(pmd_t *pmd, unsigned long address)
+static inline hw_pte_t *pte_offset_huge(pmd_t *pmd, unsigned long address)
{
return pte_offset_kernel(pmd, address);
}
-static inline pte_t *pte_alloc_huge(struct mm_struct *mm, pmd_t *pmd,
+static inline hw_pte_t *pte_alloc_huge(struct mm_struct *mm, pmd_t *pmd,
unsigned long address)
{
return pte_alloc(mm, pmd) ? NULL : pte_offset_huge(pmd, address);
}
#endif
-pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
+hw_pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
unsigned long addr, unsigned long sz);
/*
* huge_pte_offset(): Walk the hugetlb pgtable until the last level PTE.
- * Returns the pte_t* if found, or NULL if the address is not mapped.
+ * Returns the hw_pte_t* if found, or NULL if the address is not mapped.
*
* IMPORTANT: we should normally not directly call this function, instead
* this is only a common interface to implement arch-specific
@@ -235,11 +235,11 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
* a concurrent pmd unshare, but it makes sure the pgtable page is safe to
* access.
*/
-pte_t *huge_pte_offset(struct mm_struct *mm,
+hw_pte_t *huge_pte_offset(struct mm_struct *mm,
unsigned long addr, unsigned long sz);
unsigned long hugetlb_mask_last_page(struct hstate *h);
int huge_pmd_unshare(struct mmu_gather *tlb, struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep);
+ unsigned long addr, hw_pte_t *ptep);
void huge_pmd_unshare_flush(struct mmu_gather *tlb, struct vm_area_struct *vma);
void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma,
unsigned long *start, unsigned long *end);
@@ -301,7 +301,8 @@ static inline struct address_space *hugetlb_folio_mapping_lock_write(
}
static inline int huge_pmd_unshare(struct mmu_gather *tlb,
- struct vm_area_struct *vma, unsigned long addr, pte_t *ptep)
+ struct vm_area_struct *vma, unsigned long addr,
+ hw_pte_t *ptep)
{
return 0;
}
@@ -393,7 +394,7 @@ static inline int is_hugepage_only_range(struct mm_struct *mm,
}
#ifdef CONFIG_USERFAULTFD
-static inline int hugetlb_mfill_atomic_pte(pte_t *dst_pte,
+static inline int hugetlb_mfill_atomic_pte(hw_pte_t *dst_pte,
struct vm_area_struct *dst_vma,
unsigned long dst_addr,
unsigned long src_addr,
@@ -405,7 +406,7 @@ static inline int hugetlb_mfill_atomic_pte(pte_t *dst_pte,
}
#endif /* CONFIG_USERFAULTFD */
-static inline pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr,
+static inline hw_pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr,
unsigned long sz)
{
return NULL;
@@ -985,7 +986,8 @@ static inline bool htlb_allow_alloc_fallback(enum migrate_reason reason)
}
static inline spinlock_t *huge_pte_lockptr(struct hstate *h,
- struct mm_struct *mm, pte_t *pte)
+ struct mm_struct *mm,
+ hw_pte_t *pte)
{
const unsigned long size = huge_page_size(h);
@@ -1049,7 +1051,8 @@ static inline void hugetlb_count_sub(long l, struct mm_struct *mm)
#ifndef huge_ptep_modify_prot_start
#define huge_ptep_modify_prot_start huge_ptep_modify_prot_start
static inline pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep)
+ unsigned long addr,
+ hw_pte_t *ptep)
{
unsigned long psize = huge_page_size(hstate_vma(vma));
@@ -1060,7 +1063,8 @@ static inline pte_t huge_ptep_modify_prot_start(struct vm_area_struct *vma,
#ifndef huge_ptep_modify_prot_commit
#define huge_ptep_modify_prot_commit huge_ptep_modify_prot_commit
static inline void huge_ptep_modify_prot_commit(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep,
+ unsigned long addr,
+ hw_pte_t *ptep,
pte_t old_pte, pte_t pte)
{
unsigned long psize = huge_page_size(hstate_vma(vma));
@@ -1248,7 +1252,8 @@ static inline bool htlb_allow_alloc_fallback(enum migrate_reason reason)
}
static inline spinlock_t *huge_pte_lockptr(struct hstate *h,
- struct mm_struct *mm, pte_t *pte)
+ struct mm_struct *mm,
+ hw_pte_t *pte)
{
return &mm->page_table_lock;
}
@@ -1265,21 +1270,23 @@ static inline void hugetlb_count_sub(long l, struct mm_struct *mm)
{
}
-pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep);
+pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, hw_pte_t *ptep);
unsigned long huge_pte_dirty(pte_t pte);
static inline pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep)
+ unsigned long addr, hw_pte_t *ptep)
{
#ifdef CONFIG_MMU
return ptep_get(ptep);
#else
- return *ptep;
+ /* No attached PTE that requires ptep_get(). */
+ return __pte_from_hw(*ptep);
#endif
}
static inline void set_huge_pte_at(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t pte, unsigned long sz)
+ hw_pte_t *ptep, pte_t pte,
+ unsigned long sz)
{
}
@@ -1307,7 +1314,7 @@ static inline void hugetlb_bootmem_struct_page_init(void)
#endif /* CONFIG_HUGETLB_PAGE */
static inline spinlock_t *huge_pte_lock(struct hstate *h,
- struct mm_struct *mm, pte_t *pte)
+ struct mm_struct *mm, hw_pte_t *pte)
{
spinlock_t *ptl;
@@ -1325,12 +1332,12 @@ static inline __init void hugetlb_cma_reserve(void)
#endif
#ifdef CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING
-static inline bool hugetlb_pmd_shared(pte_t *pte)
+static inline bool hugetlb_pmd_shared(hw_pte_t *pte)
{
return ptdesc_pmd_is_shared(virt_to_ptdesc(pte));
}
#else
-static inline bool hugetlb_pmd_shared(pte_t *pte)
+static inline bool hugetlb_pmd_shared(hw_pte_t *pte)
{
return false;
}
@@ -1357,7 +1364,7 @@ bool __vma_private_lock(struct vm_area_struct *vma);
* Safe version of huge_pte_offset() to check the locks. See comments
* above huge_pte_offset().
*/
-static inline pte_t *
+static inline hw_pte_t *
hugetlb_walk(struct vm_area_struct *vma, unsigned long addr, unsigned long sz)
{
#if defined(CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING) && defined(CONFIG_LOCKDEP)
diff --git a/include/linux/kasan.h b/include/linux/kasan.h
index bf233bde68c7e..ff41949c0ac08 100644
--- a/include/linux/kasan.h
+++ b/include/linux/kasan.h
@@ -51,7 +51,7 @@ typedef unsigned int __bitwise kasan_vmalloc_flags_t;
#endif
extern unsigned char kasan_early_shadow_page[PAGE_SIZE];
-extern pte_t kasan_early_shadow_pte[MAX_PTRS_PER_PTE + PTE_HWTABLE_PTRS];
+extern hw_pte_t kasan_early_shadow_pte[MAX_PTRS_PER_PTE + PTE_HWTABLE_PTRS];
extern pmd_t kasan_early_shadow_pmd[MAX_PTRS_PER_PMD];
extern pud_t kasan_early_shadow_pud[MAX_PTRS_PER_PUD];
extern p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D];
diff --git a/include/linux/mm.h b/include/linux/mm.h
index c49ef99b4413b..892b280fe9c0c 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -769,7 +769,7 @@ struct vm_fault {
* VM_FAULT_ERROR).
*/
/* These three entries are valid only while holding ptl lock */
- pte_t *pte; /* Pointer to pte entry matching
+ hw_pte_t *pte; /* Pointer to pte entry matching
* the 'address'. NULL if the page
* table hasn't been allocated.
*/
@@ -3231,7 +3231,7 @@ struct follow_pfnmap_args {
* The caller shouldn't touch any of these.
*/
spinlock_t *lock;
- pte_t *ptep;
+ hw_pte_t *ptep;
/**
* Outputs:
*
@@ -3568,7 +3568,7 @@ static inline pud_t pud_mkspecial(pud_t pud)
}
#endif /* CONFIG_ARCH_SUPPORTS_PUD_PFNMAP */
-extern pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,
+extern hw_pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,
spinlock_t **ptl);
#ifdef __PAGETABLE_P4D_FOLDED
@@ -3846,7 +3846,7 @@ static inline spinlock_t *pte_lockptr(struct mm_struct *mm, pmd_t *pmd)
return ptlock_ptr(page_ptdesc(pmd_page(*pmd)));
}
-static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, pte_t *pte)
+static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, hw_pte_t *pte)
{
BUILD_BUG_ON(IS_ENABLED(CONFIG_HIGHPTE));
BUILD_BUG_ON(MAX_PTRS_PER_PTE * sizeof(pte_t) > PAGE_SIZE);
@@ -3877,7 +3877,7 @@ static inline spinlock_t *pte_lockptr(struct mm_struct *mm, pmd_t *pmd)
{
return &mm->page_table_lock;
}
-static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, pte_t *pte)
+static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, hw_pte_t *pte)
{
return &mm->page_table_lock;
}
@@ -3918,19 +3918,19 @@ static inline bool pagetable_pte_ctor(struct mm_struct *mm,
return true;
}
-pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp);
+hw_pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp);
-static inline pte_t *pte_offset_map(pmd_t *pmd, unsigned long addr)
+static inline hw_pte_t *pte_offset_map(pmd_t *pmd, unsigned long addr)
{
return __pte_offset_map(pmd, addr, NULL);
}
-pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd,
+hw_pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, spinlock_t **ptlp);
-pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,
+hw_pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, spinlock_t **ptlp);
-pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,
+hw_pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, pmd_t *pmdvalp,
spinlock_t **ptlp);
@@ -4873,7 +4873,7 @@ static inline bool gup_can_follow_protnone(const struct vm_area_struct *vma,
return !vma_is_accessible(vma);
}
-typedef int (*pte_fn_t)(pte_t *pte, unsigned long addr, void *data);
+typedef int (*pte_fn_t)(hw_pte_t *pte, unsigned long addr, void *data);
extern int apply_to_page_range(struct mm_struct *mm, unsigned long address,
unsigned long size, pte_fn_t fn, void *data);
extern int apply_to_existing_page_range(struct mm_struct *mm,
@@ -5122,7 +5122,7 @@ void *vmemmap_alloc_block(unsigned long size, int node);
struct vmem_altmap;
void *vmemmap_alloc_block_buf(unsigned long size, int node,
struct vmem_altmap *altmap);
-void vmemmap_verify(pte_t *, int, unsigned long, unsigned long);
+void vmemmap_verify(hw_pte_t *, int, unsigned long, unsigned long);
void vmemmap_set_pmd(pmd_t *pmd, void *p, int node,
unsigned long addr, unsigned long next);
int vmemmap_check_pmd(pmd_t *pmd, int node,
@@ -5487,7 +5487,7 @@ static inline bool snapshot_page_is_faithful(const struct page_snapshot *ps)
void snapshot_page(struct page_snapshot *ps, const struct page *page);
-void map_anon_folio_pte_nopf(struct folio *folio, pte_t *pte,
+void map_anon_folio_pte_nopf(struct folio *folio, hw_pte_t *pte,
struct vm_area_struct *vma, unsigned long addr,
bool uffd_wp);
diff --git a/include/linux/page_table_check.h b/include/linux/page_table_check.h
index 12268a32e8be1..933c1028c2232 100644
--- a/include/linux/page_table_check.h
+++ b/include/linux/page_table_check.h
@@ -7,6 +7,8 @@
#ifndef __LINUX_PAGE_TABLE_CHECK_H
#define __LINUX_PAGE_TABLE_CHECK_H
+#include <linux/pgtable_types.h>
+
#ifdef CONFIG_PAGE_TABLE_CHECK
#include <linux/jump_label.h>
@@ -21,7 +23,7 @@ void __page_table_check_pmd_clear(struct mm_struct *mm, unsigned long addr,
void __page_table_check_pud_clear(struct mm_struct *mm, unsigned long addr,
pud_t pud);
void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t pte, unsigned int nr);
+ hw_pte_t *ptep, pte_t pte, unsigned int nr);
void __page_table_check_pmds_set(struct mm_struct *mm, unsigned long addr,
pmd_t *pmdp, pmd_t pmd, unsigned int nr);
void __page_table_check_puds_set(struct mm_struct *mm, unsigned long addr,
@@ -74,7 +76,8 @@ static inline void page_table_check_pud_clear(struct mm_struct *mm,
}
static inline void page_table_check_ptes_set(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep,
+ unsigned long addr,
+ hw_pte_t *ptep,
pte_t pte, unsigned int nr)
{
if (static_branch_likely(&page_table_check_disabled))
@@ -137,7 +140,8 @@ static inline void page_table_check_pud_clear(struct mm_struct *mm,
}
static inline void page_table_check_ptes_set(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep,
+ unsigned long addr,
+ hw_pte_t *ptep,
pte_t pte, unsigned int nr)
{
}
diff --git a/include/linux/pagewalk.h b/include/linux/pagewalk.h
index b41d7265c01bc..7ab2eb39e02c0 100644
--- a/include/linux/pagewalk.h
+++ b/include/linux/pagewalk.h
@@ -38,7 +38,7 @@ enum page_walk_lock {
* not trigger for any populated ranges.
* @hugetlb_entry: if set, called for each hugetlb entry. This hook
* function is called with the vma lock held, in order to
- * protect against a concurrent freeing of the pte_t* or
+ * protect against a concurrent freeing of the hw_pte_t* or
* the ptl. In some cases, the hook function needs to drop
* and retake the vma lock in order to avoid deadlocks
* while calling other functions. In such cases the hook
@@ -76,11 +76,11 @@ struct mm_walk_ops {
unsigned long next, struct mm_walk *walk);
int (*pmd_entry)(pmd_t *pmd, unsigned long addr,
unsigned long next, struct mm_walk *walk);
- int (*pte_entry)(pte_t *pte, unsigned long addr,
+ int (*pte_entry)(hw_pte_t *pte, unsigned long addr,
unsigned long next, struct mm_walk *walk);
int (*pte_hole)(unsigned long addr, unsigned long next,
int depth, struct mm_walk *walk);
- int (*hugetlb_entry)(pte_t *pte, unsigned long hmask,
+ int (*hugetlb_entry)(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long next,
struct mm_walk *walk);
int (*test_walk)(unsigned long addr, unsigned long next,
@@ -89,7 +89,7 @@ struct mm_walk_ops {
struct mm_walk *walk);
void (*post_vma)(struct mm_walk *walk);
int (*install_pte)(unsigned long addr, unsigned long next,
- pte_t *ptep, struct mm_walk *walk);
+ pte_t *ptentp, struct mm_walk *walk);
enum page_walk_lock walk_lock;
};
@@ -173,7 +173,7 @@ struct folio_walk {
struct page *page;
enum folio_walk_level level;
union {
- pte_t *ptep;
+ hw_pte_t *ptep;
pud_t *pudp;
pmd_t *pmdp;
};
diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
index e3c8ab96941c5..41337da0d65aa 100644
--- a/include/linux/pgtable.h
+++ b/include/linux/pgtable.h
@@ -4,6 +4,7 @@
#include <linux/pfn.h>
#include <asm/pgtable.h>
+#include <linux/pgtable_types.h>
#define PMD_ORDER (PMD_SHIFT - PAGE_SHIFT)
#define PUD_ORDER (PUD_SHIFT - PAGE_SHIFT)
@@ -93,26 +94,26 @@ static inline void pud_init(void *addr)
#endif
#ifndef pte_offset_kernel
-static inline pte_t *pte_offset_kernel(pmd_t *pmd, unsigned long address)
+static inline hw_pte_t *pte_offset_kernel(pmd_t *pmd, unsigned long address)
{
- return (pte_t *)pmd_page_vaddr(*pmd) + pte_index(address);
+ return (hw_pte_t *)pmd_page_vaddr(*pmd) + pte_index(address);
}
#define pte_offset_kernel pte_offset_kernel
#endif
#ifdef CONFIG_HIGHPTE
#define __pte_map(pmd, address) \
- ((pte_t *)kmap_local_page(pmd_page(*(pmd))) + pte_index((address)))
+ ((hw_pte_t *)kmap_local_page(pmd_page(*(pmd))) + pte_index((address)))
#define pte_unmap(pte) do { \
kunmap_local((pte)); \
rcu_read_unlock(); \
} while (0)
#else
-static inline pte_t *__pte_map(pmd_t *pmd, unsigned long address)
+static inline hw_pte_t *__pte_map(pmd_t *pmd, unsigned long address)
{
return pte_offset_kernel(pmd, address);
}
-static inline void pte_unmap(pte_t *pte)
+static inline void pte_unmap(hw_pte_t *pte)
{
rcu_read_unlock();
}
@@ -172,7 +173,7 @@ static inline pmd_t *pmd_off_k(unsigned long va)
return pmd_offset(pud_offset(p4d_offset(pgd_offset_k(va), va), va), va);
}
-static inline pte_t *virt_to_kpte(unsigned long vaddr)
+static inline hw_pte_t *virt_to_kpte(unsigned long vaddr)
{
pmd_t *pmd = pmd_off_k(vaddr);
@@ -407,7 +408,7 @@ static inline void lazy_mmu_mode_resume(void) {}
*
* May be overridden by the architecture, else pte_batch_hint is always 1.
*/
-static inline unsigned int pte_batch_hint(pte_t *ptep, pte_t pte)
+static inline unsigned int pte_batch_hint(hw_pte_t *ptep, pte_t pte)
{
return 1;
}
@@ -442,7 +443,7 @@ static inline pte_t pte_advance_pfn(pte_t pte, unsigned long nr)
* to the same folio. The PTEs are all in the same PMD.
*/
static inline void set_ptes(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t pte, unsigned int nr)
+ hw_pte_t *ptep, pte_t pte, unsigned int nr)
{
page_table_check_ptes_set(mm, addr, ptep, pte, nr);
@@ -459,7 +460,7 @@ static inline void set_ptes(struct mm_struct *mm, unsigned long addr,
#ifndef __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS
extern int ptep_set_access_flags(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep,
+ unsigned long address, hw_pte_t *ptep,
pte_t entry, int dirty);
#endif
@@ -490,9 +491,9 @@ static inline int pudp_set_access_flags(struct vm_area_struct *vma,
#endif
#ifndef ptep_get
-static inline pte_t ptep_get(pte_t *ptep)
+static inline pte_t ptep_get(hw_pte_t *ptep)
{
- return READ_ONCE(*ptep);
+ return __pte_from_hw(READ_ONCE(*ptep));
}
#endif
@@ -526,7 +527,7 @@ static inline pgd_t pgdp_get(pgd_t *pgdp)
#ifndef __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG
static inline bool ptep_test_and_clear_young(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep)
+ unsigned long address, hw_pte_t *ptep)
{
pte_t pte = ptep_get(ptep);
bool young = true;
@@ -565,7 +566,7 @@ static inline bool pmdp_test_and_clear_young(struct vm_area_struct *vma,
#ifndef __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH
bool ptep_clear_flush_young(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep);
+ unsigned long address, hw_pte_t *ptep);
#endif
#ifndef __HAVE_ARCH_PMDP_CLEAR_YOUNG_FLUSH
@@ -644,7 +645,7 @@ static inline void arch_check_zapped_pud(struct vm_area_struct *vma, pud_t pud)
#ifndef __HAVE_ARCH_PTEP_GET_AND_CLEAR
static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
unsigned long address,
- pte_t *ptep)
+ hw_pte_t *ptep)
{
pte_t pte = ptep_get(ptep);
pte_clear(mm, address, ptep);
@@ -673,7 +674,7 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
* pages that belong to the same folio. The PTEs are all in the same PMD.
*/
static inline void clear_young_dirty_ptes(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep,
+ unsigned long addr, hw_pte_t *ptep,
unsigned int nr, cydp_t flags)
{
pte_t pte;
@@ -698,7 +699,7 @@ static inline void clear_young_dirty_ptes(struct vm_area_struct *vma,
#endif
static inline void ptep_clear(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep)
+ hw_pte_t *ptep)
{
pte_t pte = ptep_get(ptep);
@@ -739,7 +740,7 @@ static inline void ptep_clear(struct mm_struct *mm, unsigned long addr,
* present bit set *unless* it is 'l'. Because get_user_pages_fast() only
* operates on present ptes we're safe.
*/
-static inline pte_t ptep_get_lockless(pte_t *ptep)
+static inline pte_t ptep_get_lockless(hw_pte_t *ptep)
{
pte_t pte;
@@ -777,7 +778,7 @@ static inline pmd_t pmdp_get_lockless(pmd_t *pmdp)
* We require that the PTE can be read atomically.
*/
#ifndef ptep_get_lockless
-static inline pte_t ptep_get_lockless(pte_t *ptep)
+static inline pte_t ptep_get_lockless(hw_pte_t *ptep)
{
return ptep_get(ptep);
}
@@ -844,7 +845,8 @@ static inline pud_t pudp_huge_get_and_clear_full(struct vm_area_struct *vma,
#ifndef __HAVE_ARCH_PTEP_GET_AND_CLEAR_FULL
static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
- unsigned long address, pte_t *ptep,
+ unsigned long address,
+ hw_pte_t *ptep,
int full)
{
return ptep_get_and_clear(mm, address, ptep);
@@ -872,7 +874,7 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
* pages that belong to the same folio. The PTEs are all in the same PMD.
*/
static inline pte_t get_and_clear_full_ptes(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep, unsigned int nr, int full)
+ unsigned long addr, hw_pte_t *ptep, unsigned int nr, int full)
{
pte_t pte, tmp_pte;
@@ -908,7 +910,7 @@ static inline pte_t get_and_clear_full_ptes(struct mm_struct *mm,
* pages that belong to the same folio. The PTEs are all in the same PMD.
*/
static inline pte_t get_and_clear_ptes(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, unsigned int nr)
+ hw_pte_t *ptep, unsigned int nr)
{
return get_and_clear_full_ptes(mm, addr, ptep, nr, 0);
}
@@ -933,7 +935,7 @@ static inline pte_t get_and_clear_ptes(struct mm_struct *mm, unsigned long addr,
* pages that belong to the same folio. The PTEs are all in the same PMD.
*/
static inline void clear_full_ptes(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, unsigned int nr, int full)
+ hw_pte_t *ptep, unsigned int nr, int full)
{
for (;;) {
ptep_get_and_clear_full(mm, addr, ptep, full);
@@ -962,7 +964,7 @@ static inline void clear_full_ptes(struct mm_struct *mm, unsigned long addr,
* pages that belong to the same folio. The PTEs are all in the same PMD.
*/
static inline void clear_ptes(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, unsigned int nr)
+ hw_pte_t *ptep, unsigned int nr)
{
clear_full_ptes(mm, addr, ptep, nr, 0);
}
@@ -977,13 +979,14 @@ static inline void clear_ptes(struct mm_struct *mm, unsigned long addr,
*/
#ifndef update_mmu_tlb_range
static inline void update_mmu_tlb_range(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep, unsigned int nr)
+ unsigned long address, hw_pte_t *ptep,
+ unsigned int nr)
{
}
#endif
static inline void update_mmu_tlb(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep)
+ unsigned long address, hw_pte_t *ptep)
{
update_mmu_tlb_range(vma, address, ptep, 1);
}
@@ -1000,7 +1003,7 @@ static inline void update_mmu_tlb(struct vm_area_struct *vma,
* The PTEs are all in the same PMD.
*/
static inline void clear_nonpresent_ptes(struct mm_struct *mm,
- unsigned long addr, pte_t *ptep, unsigned int nr)
+ unsigned long addr, hw_pte_t *ptep, unsigned int nr)
{
(void)addr;
@@ -1016,7 +1019,7 @@ static inline void clear_nonpresent_ptes(struct mm_struct *mm,
#ifndef __HAVE_ARCH_PTEP_CLEAR_FLUSH
extern pte_t ptep_clear_flush(struct vm_area_struct *vma,
unsigned long address,
- pte_t *ptep);
+ hw_pte_t *ptep);
#endif
#ifndef __HAVE_ARCH_PMDP_HUGE_CLEAR_FLUSH
@@ -1044,7 +1047,8 @@ static inline pmd_t pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma)
#ifndef __HAVE_ARCH_PTEP_SET_WRPROTECT
struct mm_struct;
-static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long address, pte_t *ptep)
+static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long address,
+ hw_pte_t *ptep)
{
pte_t old_pte = ptep_get(ptep);
set_pte_at(mm, address, ptep, pte_wrprotect(old_pte));
@@ -1070,7 +1074,7 @@ static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long addres
* ptep_try_set as an identity macro. The generic stub returns false, which is
* correct for callers that fall through to oops on failure.
*/
-static inline bool ptep_try_set(pte_t *ptep, pte_t new_pte)
+static inline bool ptep_try_set(hw_pte_t *ptep, pte_t new_pte)
{
return false;
}
@@ -1113,7 +1117,7 @@ static inline void flush_tlb_before_set(unsigned long addr)
* pages that belong to the same folio. The PTEs are all in the same PMD.
*/
static inline void wrprotect_ptes(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, unsigned int nr)
+ hw_pte_t *ptep, unsigned int nr)
{
for (;;) {
ptep_set_wrprotect(mm, addr, ptep);
@@ -1144,7 +1148,7 @@ static inline void wrprotect_ptes(struct mm_struct *mm, unsigned long addr,
* pages that belong to the same folio. The PTEs are all in the same PMD.
*/
static inline bool clear_flush_young_ptes(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, unsigned int nr)
+ unsigned long addr, hw_pte_t *ptep, unsigned int nr)
{
bool young = false;
@@ -1181,7 +1185,7 @@ static inline bool clear_flush_young_ptes(struct vm_area_struct *vma,
* Returns: whether any PTE was young.
*/
static inline bool test_and_clear_young_ptes(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, unsigned int nr)
+ unsigned long addr, hw_pte_t *ptep, unsigned int nr)
{
bool young = false;
@@ -1585,7 +1589,7 @@ static inline int pmd_none_or_clear_bad(pmd_t *pmd)
static inline pte_t __ptep_modify_prot_start(struct vm_area_struct *vma,
unsigned long addr,
- pte_t *ptep)
+ hw_pte_t *ptep)
{
/*
* Get the current pte state, but zero it out to make it
@@ -1597,7 +1601,7 @@ static inline pte_t __ptep_modify_prot_start(struct vm_area_struct *vma,
static inline void __ptep_modify_prot_commit(struct vm_area_struct *vma,
unsigned long addr,
- pte_t *ptep, pte_t pte)
+ hw_pte_t *ptep, pte_t pte)
{
/*
* The pte is non-present, so there's no hardware state to
@@ -1623,7 +1627,7 @@ static inline void __ptep_modify_prot_commit(struct vm_area_struct *vma,
*/
static inline pte_t ptep_modify_prot_start(struct vm_area_struct *vma,
unsigned long addr,
- pte_t *ptep)
+ hw_pte_t *ptep)
{
return __ptep_modify_prot_start(vma, addr, ptep);
}
@@ -1636,7 +1640,8 @@ static inline pte_t ptep_modify_prot_start(struct vm_area_struct *vma,
*/
static inline void ptep_modify_prot_commit(struct vm_area_struct *vma,
unsigned long addr,
- pte_t *ptep, pte_t old_pte, pte_t pte)
+ hw_pte_t *ptep, pte_t old_pte,
+ pte_t pte)
{
__ptep_modify_prot_commit(vma, addr, ptep, pte);
}
@@ -1667,7 +1672,7 @@ static inline void ptep_modify_prot_commit(struct vm_area_struct *vma,
*/
#ifndef modify_prot_start_ptes
static inline pte_t modify_prot_start_ptes(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, unsigned int nr)
+ unsigned long addr, hw_pte_t *ptep, unsigned int nr)
{
pte_t pte, tmp_pte;
@@ -1707,7 +1712,7 @@ static inline pte_t modify_prot_start_ptes(struct vm_area_struct *vma,
*/
#ifndef modify_prot_commit_ptes
static inline void modify_prot_commit_ptes(struct vm_area_struct *vma, unsigned long addr,
- pte_t *ptep, pte_t old_pte, pte_t pte, unsigned int nr)
+ hw_pte_t *ptep, pte_t old_pte, pte_t pte, unsigned int nr)
{
int i;
diff --git a/include/linux/pgtable_types.h b/include/linux/pgtable_types.h
new file mode 100644
index 0000000000000..d6c5a7548550b
--- /dev/null
+++ b/include/linux/pgtable_types.h
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_PGTABLE_TYPES_H
+#define _LINUX_PGTABLE_TYPES_H
+
+#include <asm/page.h>
+
+#ifndef __ASSEMBLY__
+
+#ifdef CONFIG_ARCH_HAS_HW_PTE_T
+typedef struct __hw_pte_t { pte_t __pte; } hw_pte_t;
+#define __pte_from_hw(pte) ((pte).__pte)
+#else
+#define hw_pte_t pte_t
+#define __pte_from_hw(pte) (pte)
+#endif
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* _LINUX_PGTABLE_TYPES_H */
diff --git a/include/linux/rmap.h b/include/linux/rmap.h
index 0b332770abeed..d28a7fb6f2c0e 100644
--- a/include/linux/rmap.h
+++ b/include/linux/rmap.h
@@ -868,7 +868,7 @@ struct page_vma_mapped_walk {
struct vm_area_struct *vma;
unsigned long address;
pmd_t *pmd;
- pte_t *pte;
+ hw_pte_t *pte;
spinlock_t *ptl;
unsigned int flags;
bool pgoff_is_anon : 1;
diff --git a/include/linux/swapops.h b/include/linux/swapops.h
index e7d0d529f3e0f..40d2fd920558f 100644
--- a/include/linux/swapops.h
+++ b/include/linux/swapops.h
@@ -216,7 +216,8 @@ static inline swp_entry_t make_migration_entry_dirty(swp_entry_t entry)
extern void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd,
unsigned long address);
-extern void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr, pte_t *pte);
+extern void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr,
+ hw_pte_t *pte);
#else /* CONFIG_MIGRATION */
static inline swp_entry_t make_readable_migration_entry(pgoff_t offset)
{
@@ -236,7 +237,8 @@ static inline swp_entry_t make_writable_migration_entry(pgoff_t offset)
static inline void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd,
unsigned long address) { }
static inline void migration_entry_wait_huge(struct vm_area_struct *vma,
- unsigned long addr, pte_t *pte) { }
+ unsigned long addr,
+ hw_pte_t *pte) { }
static inline swp_entry_t make_migration_entry_young(swp_entry_t entry)
{
diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index aed121d729b01..666ff2b3741da 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -8,7 +8,7 @@
#include <linux/init.h>
#include <linux/list.h>
#include <linux/llist.h>
-#include <asm/page.h> /* pgprot_t */
+#include <linux/pgtable_types.h> /* pgprot_t, hw_pte_t */
#include <linux/rbtree.h>
#include <linux/overflow.h>
@@ -120,7 +120,7 @@ static inline unsigned long arch_vmap_pte_range_map_size(unsigned long addr, uns
#ifndef arch_vmap_pte_range_unmap_size
static inline unsigned long arch_vmap_pte_range_unmap_size(unsigned long addr,
- pte_t *ptep)
+ hw_pte_t *ptep)
{
return PAGE_SIZE;
}
diff --git a/include/trace/events/xen.h b/include/trace/events/xen.h
index ad384969e2cb2..1972d50b043a5 100644
--- a/include/trace/events/xen.h
+++ b/include/trace/events/xen.h
@@ -138,10 +138,10 @@ TRACE_EVENT(xen_mc_extend_args,
TRACE_DEFINE_SIZEOF(pteval_t);
TRACE_EVENT(xen_mmu_set_pte,
- TP_PROTO(pte_t *ptep, pte_t pteval),
+ TP_PROTO(hw_pte_t *ptep, pte_t pteval),
TP_ARGS(ptep, pteval),
TP_STRUCT__entry(
- __field(pte_t *, ptep)
+ __field(hw_pte_t *, ptep)
__field(pteval_t, pteval)
),
TP_fast_assign(__entry->ptep = ptep;
@@ -207,12 +207,12 @@ TRACE_EVENT(xen_mmu_set_p4d,
DECLARE_EVENT_CLASS(xen_mmu_ptep_modify_prot,
TP_PROTO(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t pteval),
+ hw_pte_t *ptep, pte_t pteval),
TP_ARGS(mm, addr, ptep, pteval),
TP_STRUCT__entry(
__field(struct mm_struct *, mm)
__field(unsigned long, addr)
- __field(pte_t *, ptep)
+ __field(hw_pte_t *, ptep)
__field(pteval_t, pteval)
),
TP_fast_assign(__entry->mm = mm;
@@ -227,7 +227,7 @@ DECLARE_EVENT_CLASS(xen_mmu_ptep_modify_prot,
#define DEFINE_XEN_MMU_PTEP_MODIFY_PROT(name) \
DEFINE_EVENT(xen_mmu_ptep_modify_prot, name, \
TP_PROTO(struct mm_struct *mm, unsigned long addr, \
- pte_t *ptep, pte_t pteval), \
+ hw_pte_t *ptep, pte_t pteval), \
TP_ARGS(mm, addr, ptep, pteval))
DEFINE_XEN_MMU_PTEP_MODIFY_PROT(xen_mmu_ptep_modify_prot_start);
diff --git a/kernel/bpf/arena.c b/kernel/bpf/arena.c
index 7b6847200b431..62005229b2e94 100644
--- a/kernel/bpf/arena.c
+++ b/kernel/bpf/arena.c
@@ -155,7 +155,7 @@ struct clear_range_data {
struct llist_head *free_pages;
};
-static int apply_range_set_cb(pte_t *pte, unsigned long addr, void *data)
+static int apply_range_set_cb(hw_pte_t *pte, unsigned long addr, void *data)
{
struct apply_range_data *d = data;
struct page *page;
@@ -207,7 +207,7 @@ static void flush_vmap_cache(unsigned long start, unsigned long size)
flush_cache_vmap(start, start + size);
}
-static int apply_range_clear_cb(pte_t *pte, unsigned long addr, void *data)
+static int apply_range_clear_cb(hw_pte_t *pte, unsigned long addr, void *data)
{
struct clear_range_data *d = data;
pte_t old_pte;
@@ -238,7 +238,8 @@ static int apply_range_clear_cb(pte_t *pte, unsigned long addr, void *data)
return 0;
}
-static int apply_range_set_scratch_cb(pte_t *pte, unsigned long addr, void *data)
+static int apply_range_set_scratch_cb(hw_pte_t *pte, unsigned long addr,
+ void *data)
{
struct page *scratch_page = data;
@@ -340,7 +341,7 @@ static struct bpf_map *arena_map_alloc(union bpf_attr *attr)
return ERR_PTR(err);
}
-static int existing_page_cb(pte_t *ptep, unsigned long addr, void *data)
+static int existing_page_cb(hw_pte_t *ptep, unsigned long addr, void *data)
{
struct bpf_arena *arena = data;
struct page *page;
diff --git a/kernel/events/core.c b/kernel/events/core.c
index a6c8e38a31104..0369f6c95ba7d 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -8517,7 +8517,8 @@ static u64 perf_get_pgtable_size(struct mm_struct *mm, unsigned long addr)
p4d_t *p4dp, p4d;
pud_t *pudp, pud;
pmd_t *pmdp, pmd;
- pte_t *ptep, pte;
+ hw_pte_t *ptep;
+ pte_t pte;
pgdp = pgd_offset(mm, addr);
pgd = pgdp_get(pgdp);
diff --git a/mm/Kconfig b/mm/Kconfig
index c1ddf59c0d71a..5f462ec6fa5e5 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -1312,6 +1312,9 @@ comment "GUP_TEST needs to have DEBUG_FS enabled"
config GUP_GET_PXX_LOW_HIGH
bool
+config ARCH_HAS_HW_PTE_T
+ bool
+
config DMAPOOL_TEST
tristate "Enable a module to run time tests on dma_pool"
depends on HAS_DMA
diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c
index 7a2e40bc7baed..f844a44752a81 100644
--- a/mm/damon/ops-common.c
+++ b/mm/damon/ops-common.c
@@ -39,7 +39,7 @@ struct folio *damon_get_folio(unsigned long pfn)
return folio;
}
-void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr)
+void damon_ptep_mkold(hw_pte_t *pte, struct vm_area_struct *vma, unsigned long addr)
{
pte_t pteval = ptep_get(pte);
struct folio *folio;
diff --git a/mm/damon/ops-common.h b/mm/damon/ops-common.h
index 38d295488fa18..34b7e5715fcf9 100644
--- a/mm/damon/ops-common.h
+++ b/mm/damon/ops-common.h
@@ -7,7 +7,7 @@
struct folio *damon_get_folio(unsigned long pfn);
-void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr);
+void damon_ptep_mkold(hw_pte_t *pte, struct vm_area_struct *vma, unsigned long addr);
void damon_pmdp_mkold(pmd_t *pmd, struct vm_area_struct *vma, unsigned long addr);
void damon_folio_mkold(struct folio *folio);
bool damon_folio_young(struct folio *folio);
diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
index f884d3f78f30a..482e905c973a6 100644
--- a/mm/damon/vaddr.c
+++ b/mm/damon/vaddr.c
@@ -258,7 +258,7 @@ static void damon_va_walk_page_range(struct mm_struct *mm, unsigned long start,
static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned long addr,
unsigned long next, struct mm_walk *walk)
{
- pte_t *pte;
+ hw_pte_t *pte;
spinlock_t *ptl;
ptl = pmd_trans_huge_lock(pmd, walk->vma);
@@ -283,7 +283,7 @@ static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned long addr,
}
#ifdef CONFIG_HUGETLB_PAGE
-static void damon_hugetlb_mkold(pte_t *pte, struct mm_struct *mm,
+static void damon_hugetlb_mkold(hw_pte_t *pte, struct mm_struct *mm,
struct vm_area_struct *vma, unsigned long addr)
{
bool referenced = false;
@@ -310,7 +310,7 @@ static void damon_hugetlb_mkold(pte_t *pte, struct mm_struct *mm,
folio_put(folio);
}
-static int damon_mkold_hugetlb_entry(pte_t *pte, unsigned long hmask,
+static int damon_mkold_hugetlb_entry(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
@@ -379,7 +379,7 @@ struct damon_young_walk_private {
static int damon_young_pmd_entry(pmd_t *pmd, unsigned long addr,
unsigned long next, struct mm_walk *walk)
{
- pte_t *pte;
+ hw_pte_t *pte;
pte_t ptent;
spinlock_t *ptl;
struct folio *folio;
@@ -423,7 +423,7 @@ static int damon_young_pmd_entry(pmd_t *pmd, unsigned long addr,
}
#ifdef CONFIG_HUGETLB_PAGE
-static int damon_young_hugetlb_entry(pte_t *pte, unsigned long hmask,
+static int damon_young_hugetlb_entry(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
@@ -512,7 +512,7 @@ static unsigned int damon_va_check_accesses(struct damon_ctx *ctx)
static bool damos_va_filter_young_match(struct damos_filter *filter,
struct folio *folio, struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, pmd_t *pmdp)
+ unsigned long addr, hw_pte_t *ptep, pmd_t *pmdp)
{
bool young = false;
@@ -534,7 +534,7 @@ static bool damos_va_filter_young_match(struct damos_filter *filter,
static bool damos_va_filter_out(struct damos *scheme, struct folio *folio,
struct vm_area_struct *vma, unsigned long addr,
- pte_t *ptep, pmd_t *pmdp)
+ hw_pte_t *ptep, pmd_t *pmdp)
{
struct damos_filter *filter;
bool matched;
@@ -631,7 +631,8 @@ static int damos_va_migrate_pmd_entry(pmd_t *pmd, unsigned long addr,
struct damos_migrate_dests *dests = &s->migrate_dests;
struct folio *folio;
spinlock_t *ptl;
- pte_t *start_pte, *pte, ptent;
+ hw_pte_t *start_pte, *pte;
+ pte_t ptent;
int nr;
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
@@ -794,7 +795,8 @@ static int damos_va_stat_pmd_entry(pmd_t *pmd, unsigned long addr,
struct vm_area_struct *vma = walk->vma;
struct folio *folio;
spinlock_t *ptl;
- pte_t *start_pte, *pte, ptent;
+ hw_pte_t *start_pte, *pte;
+ pte_t ptent;
int nr;
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
index 2875fd22d7bb0..54e194d2a5854 100644
--- a/mm/debug_vm_pgtable.c
+++ b/mm/debug_vm_pgtable.c
@@ -50,7 +50,7 @@ struct pgtable_debug_args {
p4d_t *p4dp;
pud_t *pudp;
pmd_t *pmdp;
- pte_t *ptep;
+ hw_pte_t *ptep;
p4d_t *start_p4dp;
pud_t *start_pudp;
diff --git a/mm/filemap.c b/mm/filemap.c
index 00fd89cf6f550..98f89dcc55603 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -3490,7 +3490,7 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf)
{
struct vm_area_struct *vma = vmf->vma;
vm_fault_t ret = 0;
- pte_t *ptep;
+ hw_pte_t *ptep;
/*
* We might have COW'ed a pagecache folio and might now have an mlocked
@@ -3796,7 +3796,7 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf,
vm_fault_t ret = 0;
struct page *page = folio_page(folio, start);
unsigned int count = 0;
- pte_t *old_ptep = vmf->pte;
+ hw_pte_t *old_ptep = vmf->pte;
unsigned long addr0;
/*
diff --git a/mm/gup.c b/mm/gup.c
index d1ba59da94e9a..549f462a23b2f 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -761,7 +761,7 @@ static struct page *follow_huge_pmd(struct vm_area_struct *vma,
#endif /* CONFIG_PGTABLE_HAS_HUGE_LEAVES */
static int follow_pfn_pte(struct vm_area_struct *vma, unsigned long address,
- pte_t *pte, unsigned int flags)
+ hw_pte_t *pte, unsigned int flags)
{
if (flags & FOLL_TOUCH) {
pte_t orig_entry = ptep_get(pte);
@@ -806,7 +806,8 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
struct folio *folio;
struct page *page;
spinlock_t *ptl;
- pte_t *ptep, pte;
+ hw_pte_t *ptep;
+ pte_t pte;
int ret;
ptep = pte_offset_map_lock(mm, pmd, address, &ptl);
@@ -1035,7 +1036,7 @@ static int get_gate_page(struct mm_struct *mm, unsigned long address,
p4d_t *p4d;
pud_t *pud;
pmd_t *pmd;
- pte_t *pte;
+ hw_pte_t *pte;
pte_t entry;
int ret = -EFAULT;
@@ -2838,7 +2839,7 @@ static unsigned long gup_fast_pte_range(pmd_t pmd, pmd_t *pmdp,
unsigned int flags, struct page **pages)
{
unsigned long nr_pages = 0;
- pte_t *ptep, *ptem;
+ hw_pte_t *ptep, *ptem;
ptem = ptep = pte_offset_map(&pmd, addr);
if (!ptep)
diff --git a/mm/highmem.c b/mm/highmem.c
index a33e411839517..87f94cac6106b 100644
--- a/mm/highmem.c
+++ b/mm/highmem.c
@@ -141,7 +141,7 @@ EXPORT_SYMBOL(__totalhigh_pages);
static int pkmap_count[LAST_PKMAP];
static __cacheline_aligned_in_smp DEFINE_SPINLOCK(kmap_lock);
-pte_t *pkmap_page_table;
+hw_pte_t *pkmap_page_table;
/*
* Most architectures have no use for kmap_high_get(), so let's abstract
@@ -532,9 +532,9 @@ static inline bool kmap_high_unmap_local(unsigned long vaddr)
return false;
}
-static pte_t *__kmap_pte;
+static hw_pte_t *__kmap_pte;
-static pte_t *kmap_get_pte(unsigned long vaddr, int idx)
+static hw_pte_t *kmap_get_pte(unsigned long vaddr, int idx)
{
if (IS_ENABLED(CONFIG_KMAP_LOCAL_NON_LINEAR_PTE_ARRAY))
/*
@@ -549,8 +549,9 @@ static pte_t *kmap_get_pte(unsigned long vaddr, int idx)
void *__kmap_local_pfn_prot(unsigned long pfn, pgprot_t prot)
{
- pte_t pteval, *kmap_pte;
unsigned long vaddr;
+ hw_pte_t *kmap_pte;
+ pte_t pteval;
int idx;
/*
@@ -597,7 +598,7 @@ EXPORT_SYMBOL(__kmap_local_page_prot);
void kunmap_local_indexed(const void *vaddr)
{
unsigned long addr = (unsigned long) vaddr & PAGE_MASK;
- pte_t *kmap_pte;
+ hw_pte_t *kmap_pte;
int idx;
if (addr < __fix_to_virt(FIX_KMAP_END) ||
@@ -646,7 +647,7 @@ EXPORT_SYMBOL(kunmap_local_indexed);
void __kmap_local_sched_out(void)
{
struct task_struct *tsk = current;
- pte_t *kmap_pte;
+ hw_pte_t *kmap_pte;
int i;
/* Clear kmaps */
@@ -683,7 +684,7 @@ void __kmap_local_sched_out(void)
void __kmap_local_sched_in(void)
{
struct task_struct *tsk = current;
- pte_t *kmap_pte;
+ hw_pte_t *kmap_pte;
int i;
/* Restore kmaps */
diff --git a/mm/hmm.c b/mm/hmm.c
index 2f1e98c6b6440..0c959611eda01 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -240,7 +240,7 @@ static inline unsigned long pte_to_hmm_pfn_flags(struct hmm_range *range,
}
static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr,
- unsigned long end, pmd_t *pmdp, pte_t *ptep,
+ unsigned long end, pmd_t *pmdp, hw_pte_t *ptep,
unsigned long *hmm_pfn)
{
struct hmm_vma_walk *hmm_vma_walk = walk->private;
@@ -411,7 +411,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp,
&range->hmm_pfns[(start - range->start) >> PAGE_SHIFT];
unsigned long npages = (end - start) >> PAGE_SHIFT;
unsigned long addr = start;
- pte_t *ptep;
+ hw_pte_t *ptep;
pmd_t pmd;
again:
@@ -547,7 +547,7 @@ static int hmm_vma_walk_pud(pud_t *pudp, unsigned long start, unsigned long end,
#endif
#ifdef CONFIG_HUGETLB_PAGE
-static int hmm_vma_walk_hugetlb_entry(pte_t *pte, unsigned long hmask,
+static int hmm_vma_walk_hugetlb_entry(hw_pte_t *pte, unsigned long hmask,
unsigned long start, unsigned long end,
struct mm_walk *walk)
{
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index dd66c6ad5af13..629c773ed313a 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -3149,7 +3149,7 @@ static void __split_huge_zero_page_pmd(struct vm_area_struct *vma,
pgtable_t pgtable;
pmd_t _pmd, old_pmd;
unsigned long addr;
- pte_t *pte;
+ hw_pte_t *pte;
int i;
/*
@@ -3199,7 +3199,7 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd,
bool soft_dirty, uffd_wp = false, young = false, write = false;
bool anon_exclusive = false, dirty = false;
unsigned long addr;
- pte_t *pte;
+ hw_pte_t *pte;
int i;
VM_BUG_ON(haddr & ~HPAGE_PMD_MASK);
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index ceb29adc6dc19..687d87801979d 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -129,7 +129,7 @@ static void hugetlb_vma_lock_free(struct vm_area_struct *vma);
static void hugetlb_vma_lock_alloc(struct vm_area_struct *vma);
static void __hugetlb_vma_unlock_write_free(struct vm_area_struct *vma);
static int __huge_pmd_unshare(struct mmu_gather *tlb,
- struct vm_area_struct *vma, unsigned long addr, pte_t *ptep,
+ struct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,
bool check_locks);
static void hugetlb_unshare_pmds(struct vm_area_struct *vma,
unsigned long start, unsigned long end, bool take_locks);
@@ -4891,7 +4891,7 @@ static pte_t make_huge_pte(struct vm_area_struct *vma, struct folio *folio,
}
static void set_huge_ptep_writable(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep)
+ unsigned long address, hw_pte_t *ptep)
{
pte_t entry;
@@ -4901,14 +4901,14 @@ static void set_huge_ptep_writable(struct vm_area_struct *vma,
}
static void set_huge_ptep_maybe_writable(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep)
+ unsigned long address, hw_pte_t *ptep)
{
if (vma->vm_flags & VM_WRITE)
set_huge_ptep_writable(vma, address, ptep);
}
static void
-hugetlb_install_folio(struct vm_area_struct *vma, pte_t *ptep, unsigned long addr,
+hugetlb_install_folio(struct vm_area_struct *vma, hw_pte_t *ptep, unsigned long addr,
struct folio *new_folio, pte_t old, unsigned long sz)
{
pte_t newpte = make_huge_pte(vma, new_folio, true);
@@ -4934,7 +4934,8 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
struct vm_area_struct *dst_vma,
struct vm_area_struct *src_vma)
{
- pte_t *src_pte, *dst_pte, entry;
+ hw_pte_t *src_pte, *dst_pte;
+ pte_t entry;
struct folio *pte_folio;
unsigned long addr;
bool cow = vma_is_cow_mapping(src_vma);
@@ -5126,7 +5127,8 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
}
static void move_huge_pte(struct vm_area_struct *vma, unsigned long old_addr,
- unsigned long new_addr, pte_t *src_pte, pte_t *dst_pte,
+ unsigned long new_addr, hw_pte_t *src_pte,
+ hw_pte_t *dst_pte,
unsigned long sz)
{
bool need_clear_uffd_wp = vma_has_uffd_without_event_remap(vma);
@@ -5188,7 +5190,7 @@ int move_hugetlb_page_tables(struct vm_area_struct *vma,
struct mm_struct *mm = vma->vm_mm;
unsigned long old_end = old_addr + len;
unsigned long last_addr_mask;
- pte_t *src_pte, *dst_pte;
+ hw_pte_t *src_pte, *dst_pte;
struct mmu_notifier_range range;
struct mmu_gather tlb;
@@ -5249,7 +5251,7 @@ void __unmap_hugepage_range(struct mmu_gather *tlb, struct vm_area_struct *vma,
struct mm_struct *mm = vma->vm_mm;
const bool folio_provided = !!folio;
unsigned long address;
- pte_t *ptep;
+ hw_pte_t *ptep;
pte_t pte;
spinlock_t *ptl;
struct hstate *h = hstate_vma(vma);
@@ -5783,7 +5785,7 @@ static inline vm_fault_t hugetlb_handle_userfault(struct vm_fault *vmf,
* false if pte changed or is changing.
*/
static bool hugetlb_pte_stable(struct hstate *h, struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t old_pte)
+ hw_pte_t *ptep, pte_t old_pte)
{
spinlock_t *ptl;
bool same;
@@ -6308,7 +6310,7 @@ static struct folio *alloc_hugetlb_folio_vma(struct hstate *h,
* Used by userfaultfd UFFDIO_* ioctls. Based on userfaultfd's mfill_atomic_pte
* with modifications for hugetlb pages.
*/
-int hugetlb_mfill_atomic_pte(pte_t *dst_pte,
+int hugetlb_mfill_atomic_pte(hw_pte_t *dst_pte,
struct vm_area_struct *dst_vma,
unsigned long dst_addr,
unsigned long src_addr,
@@ -6367,7 +6369,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte,
folio = alloc_hugetlb_folio(dst_vma, dst_addr, false);
if (IS_ERR(folio)) {
- pte_t *actual_pte = hugetlb_walk(dst_vma, dst_addr, PMD_SIZE);
+ hw_pte_t *actual_pte = hugetlb_walk(dst_vma, dst_addr, PMD_SIZE);
if (actual_pte) {
ret = -EEXIST;
goto out;
@@ -6535,7 +6537,7 @@ long hugetlb_change_protection(struct vm_area_struct *vma,
{
struct mm_struct *mm = vma->vm_mm;
unsigned long start = address;
- pte_t *ptep;
+ hw_pte_t *ptep;
pte_t pte;
struct hstate *h = hstate_vma(vma);
long pages = 0, psize = huge_page_size(h);
@@ -7020,15 +7022,15 @@ void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma,
* racing tasks could either miss the sharing (see huge_pte_offset) or select a
* bad pmd for sharing.
*/
-pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,
+hw_pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,
unsigned long addr, pud_t *pud)
{
struct address_space *mapping = vma->vm_file->f_mapping;
const pgoff_t idx = linear_page_index(vma, addr);
struct vm_area_struct *svma;
unsigned long saddr;
- pte_t *spte = NULL;
- pte_t *pte;
+ hw_pte_t *spte = NULL;
+ hw_pte_t *pte;
i_mmap_lock_read(mapping);
mapping_rmap_tree_foreach(svma, mapping, idx, idx) {
@@ -7059,13 +7061,13 @@ pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,
}
spin_unlock(&mm->page_table_lock);
out:
- pte = (pte_t *)pmd_alloc(mm, pud, addr);
+ pte = (hw_pte_t *)pmd_alloc(mm, pud, addr);
i_mmap_unlock_read(mapping);
return pte;
}
static int __huge_pmd_unshare(struct mmu_gather *tlb,
- struct vm_area_struct *vma, unsigned long addr, pte_t *ptep,
+ struct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,
bool check_locks)
{
unsigned long sz = huge_page_size(hstate_vma(vma));
@@ -7106,7 +7108,7 @@ static int __huge_pmd_unshare(struct mmu_gather *tlb,
* was not a shared PMD table.
*/
int huge_pmd_unshare(struct mmu_gather *tlb, struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep)
+ unsigned long addr, hw_pte_t *ptep)
{
return __huge_pmd_unshare(tlb, vma, addr, ptep, /*check_locks=*/true);
}
@@ -7136,21 +7138,21 @@ void huge_pmd_unshare_flush(struct mmu_gather *tlb, struct vm_area_struct *vma)
#else /* !CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING */
-pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,
+hw_pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,
unsigned long addr, pud_t *pud)
{
return NULL;
}
static int __huge_pmd_unshare(struct mmu_gather *tlb,
- struct vm_area_struct *vma, unsigned long addr, pte_t *ptep,
+ struct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,
bool check_locks)
{
return 0;
}
int huge_pmd_unshare(struct mmu_gather *tlb, struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep)
+ unsigned long addr, hw_pte_t *ptep)
{
return 0;
}
@@ -7171,13 +7173,13 @@ bool want_pmd_share(struct vm_area_struct *vma, unsigned long addr)
#endif /* CONFIG_HUGETLB_PMD_PAGE_TABLE_SHARING */
#ifdef CONFIG_ARCH_WANT_GENERAL_HUGETLB
-pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
+hw_pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
unsigned long addr, unsigned long sz)
{
pgd_t *pgd;
p4d_t *p4d;
pud_t *pud;
- pte_t *pte = NULL;
+ hw_pte_t *pte = NULL;
pgd = pgd_offset(mm, addr);
p4d = p4d_alloc(mm, pgd, addr);
@@ -7186,13 +7188,13 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
pud = pud_alloc(mm, p4d, addr);
if (pud) {
if (sz == PUD_SIZE) {
- pte = (pte_t *)pud;
+ pte = (hw_pte_t *)pud;
} else {
BUG_ON(sz != PMD_SIZE);
if (want_pmd_share(vma, addr) && pud_none(*pud))
pte = huge_pmd_share(mm, vma, addr, pud);
else
- pte = (pte_t *)pmd_alloc(mm, pud, addr);
+ pte = (hw_pte_t *)pmd_alloc(mm, pud, addr);
}
}
@@ -7214,7 +7216,7 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma,
* size @sz doesn't match the hugepage size at this level of the page
* table.
*/
-pte_t *huge_pte_offset(struct mm_struct *mm,
+hw_pte_t *huge_pte_offset(struct mm_struct *mm,
unsigned long addr, unsigned long sz)
{
pgd_t *pgd;
@@ -7232,14 +7234,14 @@ pte_t *huge_pte_offset(struct mm_struct *mm,
pud = pud_offset(p4d, addr);
if (sz == PUD_SIZE)
/* must be pud huge, non-present or none */
- return (pte_t *)pud;
+ return (hw_pte_t *)pud;
if (!pud_present(*pud))
return NULL;
/* must have a valid entry and size to go further */
pmd = pmd_offset(pud, addr);
/* must be pmd huge, non-present or none */
- return (pte_t *)pmd;
+ return (hw_pte_t *)pmd;
}
/*
@@ -7418,7 +7420,7 @@ static void hugetlb_unshare_pmds(struct vm_area_struct *vma,
struct mmu_gather tlb;
unsigned long address;
spinlock_t *ptl;
- pte_t *ptep;
+ hw_pte_t *ptep;
if (!(vma->vm_flags & VM_MAYSHARE))
return;
diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c
index eb339c4a71f40..bf85d82c29b96 100644
--- a/mm/hugetlb_vmemmap.c
+++ b/mm/hugetlb_vmemmap.c
@@ -34,7 +34,7 @@
* operations.
*/
struct vmemmap_remap_walk {
- void (*remap_pte)(pte_t *pte, unsigned long addr,
+ void (*remap_pte)(hw_pte_t *pte, unsigned long addr,
struct vmemmap_remap_walk *walk);
unsigned long nr_walked;
@@ -56,7 +56,7 @@ static int vmemmap_split_pmd(pmd_t *pmd, struct page *head, unsigned long start,
pmd_t __pmd;
int i;
unsigned long addr = start;
- pte_t *pgtable;
+ hw_pte_t *pgtable;
pgtable = pte_alloc_one_kernel(&init_mm);
if (!pgtable)
@@ -65,7 +65,8 @@ static int vmemmap_split_pmd(pmd_t *pmd, struct page *head, unsigned long start,
pmd_populate_kernel(&init_mm, &__pmd, pgtable);
for (i = 0; i < PTRS_PER_PTE; i++, addr += PAGE_SIZE) {
- pte_t entry, *pte;
+ pte_t entry;
+ hw_pte_t *pte;
pgprot_t pgprot = PAGE_KERNEL;
entry = mk_pte(head + i, pgprot);
@@ -137,7 +138,7 @@ static int vmemmap_pmd_entry(pmd_t *pmd, unsigned long addr,
return vmemmap_split_pmd(pmd, head, addr & PMD_MASK, vmemmap_walk);
}
-static int vmemmap_pte_entry(pte_t *pte, unsigned long addr,
+static int vmemmap_pte_entry(hw_pte_t *pte, unsigned long addr,
unsigned long next, struct mm_walk *walk)
{
struct vmemmap_remap_walk *vmemmap_walk = walk->private;
@@ -199,7 +200,7 @@ static void free_vmemmap_page_list(struct list_head *list)
free_vmemmap_page(page);
}
-static void vmemmap_remap_pte(pte_t *pte, unsigned long addr,
+static void vmemmap_remap_pte(hw_pte_t *pte, unsigned long addr,
struct vmemmap_remap_walk *walk)
{
struct page *page = pte_page(ptep_get(pte));
@@ -233,7 +234,7 @@ static void vmemmap_remap_pte(pte_t *pte, unsigned long addr,
set_pte_at(&init_mm, addr, pte, entry);
}
-static void vmemmap_restore_pte(pte_t *pte, unsigned long addr,
+static void vmemmap_restore_pte(hw_pte_t *pte, unsigned long addr,
struct vmemmap_remap_walk *walk)
{
struct page *src = pte_page(ptep_get(pte)), *dst;
diff --git a/mm/internal.h b/mm/internal.h
index da14c56fb24e1..8c14f0cf07878 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -266,7 +266,7 @@ void unmap_vmas(struct mmu_gather *tlb, struct unmap_desc *unmap);
#ifdef CONFIG_MMU
bool cond_install_uffd_wp_ptes(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, pte_t pte,
+ unsigned long addr, hw_pte_t *ptep, pte_t pte,
unsigned long nr_ptes);
static inline void get_anon_vma(struct anon_vma *anon_vma)
@@ -397,7 +397,7 @@ static inline pte_t __pte_batch_clear_ignored(pte_t pte, fpb_t flags)
* Return: the number of table entries in the batch.
*/
static inline unsigned int folio_pte_batch_flags(struct folio *folio,
- struct vm_area_struct *vma, pte_t *ptep, pte_t *ptentp,
+ struct vm_area_struct *vma, hw_pte_t *ptep, pte_t *ptentp,
unsigned int max_nr, fpb_t flags)
{
bool any_writable = false, any_young = false, any_dirty = false;
@@ -451,7 +451,7 @@ static inline unsigned int folio_pte_batch_flags(struct folio *folio,
return min(nr, max_nr);
}
-unsigned int folio_pte_batch(struct folio *folio, pte_t *ptep, pte_t pte,
+unsigned int folio_pte_batch(struct folio *folio, hw_pte_t *ptep, pte_t pte,
unsigned int max_nr);
/**
@@ -508,11 +508,11 @@ static inline pte_t pte_next_swp_offset(pte_t pte)
*
* Return: the number of table entries in the batch.
*/
-static inline int swap_pte_batch(pte_t *start_ptep, int max_nr, pte_t pte)
+static inline int swap_pte_batch(hw_pte_t *start_ptep, int max_nr, pte_t pte)
{
pte_t expected_pte = pte_next_swp_offset(pte);
- const pte_t *end_ptep = start_ptep + max_nr;
- pte_t *ptep = start_ptep + 1;
+ const hw_pte_t *end_ptep = start_ptep + max_nr;
+ hw_pte_t *ptep = start_ptep + 1;
VM_WARN_ON(max_nr < 1);
VM_WARN_ON(!softleaf_is_swap(softleaf_from_pte(pte)));
@@ -1550,7 +1550,7 @@ static inline void maybe_rmap_unlock_action(struct vm_area_struct *vma,
#ifdef CONFIG_MMU_NOTIFIER
static inline bool clear_flush_young_ptes_notify(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, unsigned int nr)
+ unsigned long addr, hw_pte_t *ptep, unsigned int nr)
{
bool young;
@@ -1571,7 +1571,7 @@ static inline bool pmdp_clear_flush_young_notify(struct vm_area_struct *vma,
}
static inline bool test_and_clear_young_ptes_notify(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, unsigned int nr)
+ unsigned long addr, hw_pte_t *ptep, unsigned int nr)
{
bool young;
diff --git a/mm/kasan/init.c b/mm/kasan/init.c
index 66a8838879876..9bbc2a41d23f0 100644
--- a/mm/kasan/init.c
+++ b/mm/kasan/init.c
@@ -64,7 +64,7 @@ static inline bool kasan_pmd_table(pud_t pud)
return false;
}
#endif
-pte_t kasan_early_shadow_pte[MAX_PTRS_PER_PTE + PTE_HWTABLE_PTRS]
+hw_pte_t kasan_early_shadow_pte[MAX_PTRS_PER_PTE + PTE_HWTABLE_PTRS]
__bss_pgtbl;
static inline bool kasan_pte_table(pmd_t pmd)
@@ -92,7 +92,7 @@ static __init void *early_alloc(size_t size, int node)
static void __ref zero_pte_populate(pmd_t *pmd, unsigned long addr,
unsigned long end)
{
- pte_t *pte = pte_offset_kernel(pmd, addr);
+ hw_pte_t *pte = pte_offset_kernel(pmd, addr);
pte_t zero_pte;
zero_pte = pfn_pte(PFN_DOWN(__pa_symbol(kasan_early_shadow_page)),
@@ -122,7 +122,7 @@ static int __ref zero_pmd_populate(pud_t *pud, unsigned long addr,
}
if (pmd_none(*pmd)) {
- pte_t *p;
+ hw_pte_t *p;
if (slab_is_available())
p = pte_alloc_one_kernel(&init_mm);
@@ -281,9 +281,9 @@ int __ref kasan_populate_early_shadow(const void *shadow_start,
return 0;
}
-static void kasan_free_pte(pte_t *pte_start, pmd_t *pmd)
+static void kasan_free_pte(hw_pte_t *pte_start, pmd_t *pmd)
{
- pte_t *pte;
+ hw_pte_t *pte;
int i;
for (i = 0; i < PTRS_PER_PTE; i++) {
@@ -341,7 +341,7 @@ static void kasan_free_p4d(p4d_t *p4d_start, pgd_t *pgd)
pgd_clear(pgd);
}
-static void kasan_remove_pte_table(pte_t *pte, unsigned long addr,
+static void kasan_remove_pte_table(hw_pte_t *pte, unsigned long addr,
unsigned long end)
{
unsigned long next;
@@ -369,7 +369,7 @@ static void kasan_remove_pmd_table(pmd_t *pmd, unsigned long addr,
unsigned long next;
for (; addr < end; addr = next, pmd++) {
- pte_t *pte;
+ hw_pte_t *pte;
next = pmd_addr_end(addr, end);
diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c
index d286e0a045437..86fc7ed45dcd0 100644
--- a/mm/kasan/shadow.c
+++ b/mm/kasan/shadow.c
@@ -189,7 +189,7 @@ static bool shadow_mapped(unsigned long addr)
p4d_t *p4d;
pud_t *pud;
pmd_t *pmd;
- pte_t *pte;
+ hw_pte_t *pte;
if (pgd_none(*pgd))
return false;
@@ -297,7 +297,7 @@ struct vmalloc_populate_data {
struct page **pages;
};
-static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,
+static int kasan_populate_vmalloc_pte(hw_pte_t *ptep, unsigned long addr,
void *_data)
{
struct vmalloc_populate_data *data = _data;
@@ -465,7 +465,7 @@ int __kasan_populate_vmalloc(unsigned long addr, unsigned long size, gfp_t gfp_m
return 0;
}
-static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr,
+static int kasan_depopulate_vmalloc_pte(hw_pte_t *ptep, unsigned long addr,
void *unused)
{
pte_t pte;
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index f49a6710933b1..4a7074f58b55d 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -639,8 +639,8 @@ static void release_pte_folio(struct folio *folio)
folio_putback_lru(folio);
}
-static void release_pte_pages(pte_t *pte, pte_t *_pte,
- struct list_head *compound_pagelist)
+static void release_pte_pages(hw_pte_t *pte, hw_pte_t *_pte,
+ struct list_head *compound_pagelist)
{
struct folio *folio, *tmp;
@@ -685,7 +685,8 @@ static void count_collapse_event(unsigned int order, enum vm_event_item vm_event
}
static enum scan_result __collapse_huge_page_isolate(struct vm_area_struct *vma,
- unsigned long start_addr, pte_t *pte, struct collapse_control *cc,
+ unsigned long start_addr, hw_pte_t *pte,
+ struct collapse_control *cc,
unsigned int order, struct list_head *compound_pagelist)
{
const unsigned int max_ptes_none = collapse_max_ptes_none(cc, vma, order);
@@ -694,7 +695,7 @@ static enum scan_result __collapse_huge_page_isolate(struct vm_area_struct *vma,
struct page *page = NULL;
struct folio *folio = NULL;
unsigned long addr = start_addr;
- pte_t *_pte;
+ hw_pte_t *_pte;
int none_or_zero = 0, shared = 0, referenced = 0;
enum scan_result result = SCAN_FAIL;
@@ -840,16 +841,18 @@ static enum scan_result __collapse_huge_page_isolate(struct vm_area_struct *vma,
return result;
}
-static void __collapse_huge_page_copy_succeeded(pte_t *pte,
- struct vm_area_struct *vma, unsigned long address,
- spinlock_t *ptl, unsigned int order,
- struct list_head *compound_pagelist)
+static void __collapse_huge_page_copy_succeeded(hw_pte_t *pte,
+ struct vm_area_struct *vma,
+ unsigned long address,
+ spinlock_t *ptl,
+ unsigned int order,
+ struct list_head *compound_pagelist)
{
const unsigned long nr_pages = 1UL << order;
unsigned long end = address + (PAGE_SIZE * nr_pages);
struct folio *src, *tmp;
pte_t pteval;
- pte_t *_pte;
+ hw_pte_t *_pte;
unsigned int nr_ptes;
for (_pte = pte; _pte < pte + nr_pages; _pte += nr_ptes,
@@ -904,9 +907,11 @@ static void __collapse_huge_page_copy_succeeded(pte_t *pte,
}
}
-static void __collapse_huge_page_copy_failed(pte_t *pte,
- pmd_t *pmd, pmd_t orig_pmd, struct vm_area_struct *vma,
- unsigned int order, struct list_head *compound_pagelist)
+static void __collapse_huge_page_copy_failed(hw_pte_t *pte,
+ pmd_t *pmd, pmd_t orig_pmd,
+ struct vm_area_struct *vma,
+ unsigned int order,
+ struct list_head *compound_pagelist)
{
const unsigned long nr_pages = 1UL << order;
spinlock_t *pmd_ptl;
@@ -942,10 +947,14 @@ static void __collapse_huge_page_copy_failed(pte_t *pte,
* @ptl: lock on raw pages' PTEs
* @compound_pagelist: list that stores compound pages
*/
-static enum scan_result __collapse_huge_page_copy(pte_t *pte, struct folio *folio,
- pmd_t *pmd, pmd_t orig_pmd, struct vm_area_struct *vma,
- unsigned long address, spinlock_t *ptl, unsigned int order,
- struct list_head *compound_pagelist)
+static enum scan_result __collapse_huge_page_copy(hw_pte_t *pte,
+ struct folio *folio,
+ pmd_t *pmd, pmd_t orig_pmd,
+ struct vm_area_struct *vma,
+ unsigned long address,
+ spinlock_t *ptl,
+ unsigned int order,
+ struct list_head *compound_pagelist)
{
const unsigned long nr_pages = 1UL << order;
unsigned int i;
@@ -1164,7 +1173,7 @@ static enum scan_result __collapse_huge_page_swapin(struct mm_struct *mm,
vm_fault_t ret = 0;
unsigned long addr, end = start_addr + (PAGE_SIZE << order);
enum scan_result result;
- pte_t *pte = NULL;
+ hw_pte_t *pte = NULL;
spinlock_t *ptl;
for (addr = start_addr; addr < end; addr += PAGE_SIZE) {
@@ -1292,7 +1301,7 @@ static enum scan_result collapse_huge_page(struct mm_struct *mm, unsigned long s
const unsigned long end_addr = start_addr + (PAGE_SIZE << order);
LIST_HEAD(compound_pagelist);
pmd_t *pmd, _pmd;
- pte_t *pte = NULL;
+ hw_pte_t *pte = NULL;
pgtable_t pgtable;
struct folio *folio;
spinlock_t *pmd_ptl, *pte_ptl;
@@ -1606,7 +1615,8 @@ static enum scan_result collapse_scan_pmd(struct mm_struct *mm,
unsigned int max_ptes_none = collapse_max_ptes_none(cc, vma, HPAGE_PMD_ORDER);
enum tva_type tva_flags = cc->is_khugepaged ? TVA_KHUGEPAGED : TVA_FORCED_COLLAPSE;
pmd_t *pmd;
- pte_t *pte, *_pte, pteval;
+ hw_pte_t *pte, *_pte;
+ pte_t pteval;
int i;
int none_or_zero = 0, shared = 0, referenced = 0;
enum scan_result result = SCAN_FAIL;
@@ -1871,7 +1881,7 @@ static enum scan_result try_collapse_pte_mapped_thp(struct mm_struct *mm, unsign
unsigned long end = haddr + HPAGE_PMD_SIZE;
struct vm_area_struct *vma = vma_lookup(mm, haddr);
struct folio *folio;
- pte_t *start_pte, *pte;
+ hw_pte_t *start_pte, *pte;
pmd_t *pmd, pgt_pmd;
spinlock_t *pml = NULL, *ptl;
int i;
diff --git a/mm/ksm.c b/mm/ksm.c
index 624f37975e129..3882d8916efa4 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -618,7 +618,7 @@ static int break_ksm_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned long en
{
unsigned long *found_addr = (unsigned long *) walk->private;
struct mm_struct *mm = walk->mm;
- pte_t *start_ptep, *ptep;
+ hw_pte_t *start_ptep, *ptep;
spinlock_t *ptl;
int found = 0;
@@ -1292,7 +1292,7 @@ static u32 calc_checksum(struct page *page)
}
static int write_protect_page(struct vm_area_struct *vma, struct folio *folio,
- pte_t *orig_pte)
+ pte_t *ptentp)
{
struct mm_struct *mm = vma->vm_mm;
DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, 0, 0);
@@ -1371,7 +1371,7 @@ static int write_protect_page(struct vm_area_struct *vma, struct folio *folio,
set_pte_at(mm, pvmw.address, pvmw.pte, entry);
}
- *orig_pte = entry;
+ *ptentp = entry;
err = 0;
out_unlock:
@@ -1399,7 +1399,7 @@ static int replace_page(struct vm_area_struct *vma, struct page *page,
struct folio *folio = page_folio(page);
pmd_t *pmd;
pmd_t pmde;
- pte_t *ptep;
+ hw_pte_t *ptep;
pte_t newpte;
spinlock_t *ptl;
unsigned long addr;
@@ -2530,7 +2530,8 @@ static int ksm_next_page_pmd_entry(pmd_t *pmdp, unsigned long addr, unsigned lon
{
struct ksm_next_page_arg *private = walk->private;
struct vm_area_struct *vma = walk->vma;
- pte_t *start_ptep = NULL, *ptep, pte;
+ hw_pte_t *start_ptep = NULL, *ptep;
+ pte_t pte;
struct mm_struct *mm = walk->mm;
struct folio *folio;
struct page *page;
diff --git a/mm/madvise.c b/mm/madvise.c
index 73c2901b9adbf..a08270c9f7610 100644
--- a/mm/madvise.c
+++ b/mm/madvise.c
@@ -198,7 +198,7 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned long start,
{
struct vm_area_struct *vma = walk->private;
struct swap_io_ctx ctx = {};
- pte_t *ptep = NULL;
+ hw_pte_t *ptep = NULL;
spinlock_t *ptl;
unsigned long addr;
@@ -350,7 +350,7 @@ static inline bool can_do_file_pageout(struct vm_area_struct *vma)
}
static inline int madvise_folio_pte_batch(unsigned long addr, unsigned long end,
- struct folio *folio, pte_t *ptep,
+ struct folio *folio, hw_pte_t *ptep,
pte_t *ptentp)
{
int max_nr = (end - addr) / PAGE_SIZE;
@@ -368,7 +368,8 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
bool pageout = private->pageout;
struct mm_struct *mm = tlb->mm;
struct vm_area_struct *vma = walk->vma;
- pte_t *start_pte, *pte, ptent;
+ hw_pte_t *start_pte, *pte;
+ pte_t ptent;
spinlock_t *ptl;
struct folio *folio = NULL;
LIST_HEAD(folio_list);
@@ -670,7 +671,8 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
struct mm_struct *mm = tlb->mm;
struct vm_area_struct *vma = walk->vma;
spinlock_t *ptl;
- pte_t *start_pte, *pte, ptent;
+ hw_pte_t *start_pte, *pte;
+ pte_t ptent;
struct folio *folio;
int nr_swap = 0;
unsigned long next;
@@ -1095,7 +1097,7 @@ static int guard_install_pmd_entry(pmd_t *pmd, unsigned long addr,
return pmd_trans_huge(pmdval);
}
-static int guard_install_pte_entry(pte_t *pte, unsigned long addr,
+static int guard_install_pte_entry(hw_pte_t *pte, unsigned long addr,
unsigned long next, struct mm_walk *walk)
{
pte_t pteval = ptep_get(pte);
@@ -1113,12 +1115,12 @@ static int guard_install_pte_entry(pte_t *pte, unsigned long addr,
}
static int guard_install_set_pte(unsigned long addr, unsigned long next,
- pte_t *ptep, struct mm_walk *walk)
+ pte_t *ptentp, struct mm_walk *walk)
{
unsigned long *nr_pages = (unsigned long *)walk->private;
/* Simply install a PTE marker, this causes segfault on access. */
- *ptep = make_pte_marker(PTE_MARKER_GUARD);
+ *ptentp = make_pte_marker(PTE_MARKER_GUARD);
(*nr_pages)++;
return 0;
@@ -1238,7 +1240,7 @@ static int guard_remove_pmd_entry(pmd_t *pmd, unsigned long addr,
return 0;
}
-static int guard_remove_pte_entry(pte_t *pte, unsigned long addr,
+static int guard_remove_pte_entry(hw_pte_t *pte, unsigned long addr,
unsigned long next, struct mm_walk *walk)
{
pte_t ptent = ptep_get(pte);
diff --git a/mm/mapping_dirty_helpers.c b/mm/mapping_dirty_helpers.c
index e0efa36e0a076..dcbd39912f819 100644
--- a/mm/mapping_dirty_helpers.c
+++ b/mm/mapping_dirty_helpers.c
@@ -31,7 +31,7 @@ struct wp_walk {
* The function write-protects a pte and records the range in
* virtual address space of touched ptes for efficient range TLB flushes.
*/
-static int wp_pte(pte_t *pte, unsigned long addr, unsigned long end,
+static int wp_pte(hw_pte_t *pte, unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
struct wp_walk *wpwalk = walk->private;
@@ -86,7 +86,7 @@ struct clean_walk {
* in the address_space, as well as the first and last of the bits
* touched.
*/
-static int clean_record_pte(pte_t *pte, unsigned long addr,
+static int clean_record_pte(hw_pte_t *pte, unsigned long addr,
unsigned long end, struct mm_walk *walk)
{
struct wp_walk *wpwalk = walk->private;
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index a8b03e2920ba8..a89f3fde47a5d 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -342,7 +342,7 @@ static unsigned long dev_pagemap_mapping_shift(struct vm_area_struct *vma,
p4d_t *p4d;
pud_t *pud;
pmd_t *pmd;
- pte_t *pte;
+ hw_pte_t *pte;
pte_t ptent;
VM_BUG_ON_VMA(address == -EFAULT, vma);
@@ -742,7 +742,7 @@ static int hwpoison_pte_range(pmd_t *pmdp, unsigned long addr,
{
struct hwpoison_walk *hwp = walk->private;
int ret = 0;
- pte_t *ptep, *mapped_pte;
+ hw_pte_t *ptep, *mapped_pte;
spinlock_t *ptl;
ptl = pmd_trans_huge_lock(pmdp, walk->vma);
@@ -770,7 +770,7 @@ static int hwpoison_pte_range(pmd_t *pmdp, unsigned long addr,
}
#ifdef CONFIG_HUGETLB_PAGE
-static int hwpoison_hugetlb_range(pte_t *ptep, unsigned long hmask,
+static int hwpoison_hugetlb_range(hw_pte_t *ptep, unsigned long hmask,
unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
diff --git a/mm/memory.c b/mm/memory.c
index ec63dd6212ac5..7fc8ce10503c3 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -462,7 +462,7 @@ int __pte_alloc(struct mm_struct *mm, pmd_t *pmd)
int __pte_alloc_kernel(pmd_t *pmd)
{
- pte_t *new = pte_alloc_one_kernel(&init_mm);
+ hw_pte_t *new = pte_alloc_one_kernel(&init_mm);
if (!new)
return -ENOMEM;
@@ -909,7 +909,7 @@ struct page *vm_normal_page_pud(struct vm_area_struct *vma,
*/
static void restore_exclusive_pte(struct vm_area_struct *vma,
struct folio *folio, struct page *page, unsigned long address,
- pte_t *ptep, pte_t orig_pte)
+ hw_pte_t *ptep, pte_t orig_pte)
{
pte_t pte;
@@ -946,7 +946,7 @@ static void restore_exclusive_pte(struct vm_area_struct *vma,
* sleeping.
*/
static int try_restore_exclusive_pte(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, pte_t orig_pte)
+ unsigned long addr, hw_pte_t *ptep, pte_t orig_pte)
{
const softleaf_t entry = softleaf_from_pte(orig_pte);
struct page *page = softleaf_to_page(entry);
@@ -969,7 +969,7 @@ static int try_restore_exclusive_pte(struct vm_area_struct *vma,
static unsigned long
copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm,
- pte_t *dst_pte, pte_t *src_pte, struct vm_area_struct *dst_vma,
+ hw_pte_t *dst_pte, hw_pte_t *src_pte, struct vm_area_struct *dst_vma,
struct vm_area_struct *src_vma, unsigned long addr, int *rss)
{
pte_t orig_pte = ptep_get(src_pte);
@@ -1083,7 +1083,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm,
*/
static inline int
copy_present_page(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
- pte_t *dst_pte, pte_t *src_pte, unsigned long addr, int *rss,
+ hw_pte_t *dst_pte, hw_pte_t *src_pte, unsigned long addr, int *rss,
struct folio **prealloc, struct page *page)
{
struct folio *new_folio;
@@ -1122,7 +1122,7 @@ copy_present_page(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma
}
static __always_inline void __copy_present_ptes(struct vm_area_struct *dst_vma,
- struct vm_area_struct *src_vma, pte_t *dst_pte, pte_t *src_pte,
+ struct vm_area_struct *src_vma, hw_pte_t *dst_pte, hw_pte_t *src_pte,
pte_t pte, unsigned long addr, int nr)
{
struct mm_struct *src_mm = src_vma->vm_mm;
@@ -1172,7 +1172,7 @@ static __always_inline void __copy_present_ptes(struct vm_area_struct *dst_vma,
*/
static inline int
copy_present_ptes(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
- pte_t *dst_pte, pte_t *src_pte, pte_t pte, unsigned long addr,
+ hw_pte_t *dst_pte, hw_pte_t *src_pte, pte_t pte, unsigned long addr,
int max_nr, int *rss, struct folio **prealloc)
{
fpb_t flags = FPB_MERGE_WRITE;
@@ -1272,8 +1272,8 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma,
{
struct mm_struct *dst_mm = dst_vma->vm_mm;
struct mm_struct *src_mm = src_vma->vm_mm;
- pte_t *orig_src_pte, *orig_dst_pte;
- pte_t *src_pte, *dst_pte;
+ hw_pte_t *orig_src_pte, *orig_dst_pte;
+ hw_pte_t *src_pte, *dst_pte;
pmd_t dummy_pmdval;
pte_t ptent;
spinlock_t *src_ptl, *dst_ptl;
@@ -1666,7 +1666,7 @@ static inline bool zap_drop_markers(struct zap_details *details)
* Returns true if uffd-wp PTEs were installed, false otherwise.
*/
bool cond_install_uffd_wp_ptes(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, pte_t pte,
+ unsigned long addr, hw_pte_t *ptep, pte_t pte,
unsigned long nr_ptes)
{
bool arm_uffd_pte = false;
@@ -1720,7 +1720,7 @@ bool cond_install_uffd_wp_ptes(struct vm_area_struct *vma,
*/
static inline bool
zap_install_uffd_wp_if_needed(struct vm_area_struct *vma,
- unsigned long addr, pte_t *pte, int nr,
+ unsigned long addr, hw_pte_t *pte, int nr,
struct zap_details *details, pte_t pteval)
{
if (zap_drop_markers(details))
@@ -1731,7 +1731,7 @@ zap_install_uffd_wp_if_needed(struct vm_area_struct *vma,
static __always_inline void zap_present_folio_ptes(struct mmu_gather *tlb,
struct vm_area_struct *vma, struct folio *folio,
- struct page *page, pte_t *pte, pte_t ptent, unsigned int nr,
+ struct page *page, hw_pte_t *pte, pte_t ptent, unsigned int nr,
unsigned long addr, struct zap_details *details, int *rss,
bool *force_flush, bool *force_break, bool *any_skipped)
{
@@ -1781,7 +1781,7 @@ static __always_inline void zap_present_folio_ptes(struct mmu_gather *tlb,
* Returns the number of processed (skipped or zapped) PTEs (at least 1).
*/
static inline int zap_present_ptes(struct mmu_gather *tlb,
- struct vm_area_struct *vma, pte_t *pte, pte_t ptent,
+ struct vm_area_struct *vma, hw_pte_t *pte, pte_t ptent,
unsigned int max_nr, unsigned long addr,
struct zap_details *details, int *rss, bool *force_flush,
bool *force_break, bool *any_skipped)
@@ -1827,7 +1827,7 @@ static inline int zap_present_ptes(struct mmu_gather *tlb,
}
static inline int zap_nonpresent_ptes(struct mmu_gather *tlb,
- struct vm_area_struct *vma, pte_t *pte, pte_t ptent,
+ struct vm_area_struct *vma, hw_pte_t *pte, pte_t ptent,
unsigned int max_nr, unsigned long addr,
struct zap_details *details, int *rss, bool *any_skipped)
{
@@ -1898,7 +1898,7 @@ static inline int zap_nonpresent_ptes(struct mmu_gather *tlb,
}
static inline int do_zap_pte_range(struct mmu_gather *tlb,
- struct vm_area_struct *vma, pte_t *pte,
+ struct vm_area_struct *vma, hw_pte_t *pte,
unsigned long addr, unsigned long end,
struct zap_details *details, int *rss,
bool *force_flush, bool *force_break,
@@ -1961,7 +1961,7 @@ static bool zap_pte_table_if_empty(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, pmd_t *pmdval)
{
spinlock_t *pml, *ptl = NULL;
- pte_t *start_pte, *pte;
+ hw_pte_t *start_pte, *pte;
int i;
pml = pmd_lock(mm, pmd);
@@ -2001,8 +2001,8 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb,
struct mm_struct *mm = tlb->mm;
int rss[NR_MM_COUNTERS];
spinlock_t *ptl;
- pte_t *start_pte;
- pte_t *pte;
+ hw_pte_t *start_pte;
+ hw_pte_t *pte;
pmd_t pmdval;
unsigned long start = addr;
bool direct_reclaim = true;
@@ -2384,7 +2384,7 @@ static pmd_t *walk_to_pmd(struct mm_struct *mm, unsigned long addr)
return pmd;
}
-pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,
+hw_pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,
spinlock_t **ptl)
{
pmd_t *pmd = walk_to_pmd(mm, addr);
@@ -2442,7 +2442,7 @@ static int validate_page_before_insert(struct vm_area_struct *vma,
return 0;
}
-static int insert_page_into_pte_locked(struct vm_area_struct *vma, pte_t *pte,
+static int insert_page_into_pte_locked(struct vm_area_struct *vma, hw_pte_t *pte,
unsigned long addr, struct page *page,
pgprot_t prot, bool mkwrite)
{
@@ -2487,7 +2487,7 @@ static int insert_page(struct vm_area_struct *vma, unsigned long addr,
struct page *page, pgprot_t prot, bool mkwrite)
{
int retval;
- pte_t *pte;
+ hw_pte_t *pte;
spinlock_t *ptl;
retval = validate_page_before_insert(vma, page);
@@ -2504,7 +2504,7 @@ static int insert_page(struct vm_area_struct *vma, unsigned long addr,
return retval;
}
-static int insert_page_in_batch_locked(struct vm_area_struct *vma, pte_t *pte,
+static int insert_page_in_batch_locked(struct vm_area_struct *vma, hw_pte_t *pte,
unsigned long addr, struct page *page, pgprot_t prot)
{
int err;
@@ -2522,7 +2522,7 @@ static int insert_pages(struct vm_area_struct *vma, unsigned long addr,
struct page **pages, unsigned long *num, pgprot_t prot)
{
pmd_t *pmd = NULL;
- pte_t *start_pte, *pte;
+ hw_pte_t *start_pte, *pte;
spinlock_t *pte_lock;
struct mm_struct *const mm = vma->vm_mm;
unsigned long curr_page_idx = 0;
@@ -2765,7 +2765,8 @@ static vm_fault_t insert_pfn(struct vm_area_struct *vma, unsigned long addr,
unsigned long pfn, pgprot_t prot, bool mkwrite)
{
struct mm_struct *mm = vma->vm_mm;
- pte_t *pte, entry;
+ hw_pte_t *pte;
+ pte_t entry;
spinlock_t *ptl;
pte = get_locked_pte(mm, addr, &ptl);
@@ -3006,7 +3007,7 @@ static int remap_pte_range(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, unsigned long end,
unsigned long pfn, pgprot_t prot)
{
- pte_t *pte, *mapped_pte;
+ hw_pte_t *pte, *mapped_pte;
spinlock_t *ptl;
int err = 0;
@@ -3407,7 +3408,7 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd,
pte_fn_t fn, void *data, bool create,
pgtbl_mod_mask *mask)
{
- pte_t *pte, *mapped_pte;
+ hw_pte_t *pte, *mapped_pte;
int err = 0;
spinlock_t *ptl;
@@ -4710,7 +4711,7 @@ static vm_fault_t handle_pte_marker(struct vm_fault *vmf)
/*
* Check if the PTEs within a range are contiguous swap entries.
*/
-static bool can_swapin_thp(struct vm_fault *vmf, pte_t *ptep, int nr_pages)
+static bool can_swapin_thp(struct vm_fault *vmf, hw_pte_t *ptep, int nr_pages)
{
unsigned long addr;
int idx;
@@ -4762,7 +4763,7 @@ static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
unsigned long addr;
softleaf_t entry;
spinlock_t *ptl;
- pte_t *pte;
+ hw_pte_t *pte;
int order;
/*
@@ -4856,7 +4857,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
int nr_pages;
unsigned long page_idx;
unsigned long address;
- pte_t *ptep;
+ hw_pte_t *ptep;
if (!pte_unmap_same(vmf))
goto out;
@@ -5025,7 +5026,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
unsigned long idx = folio_page_idx(folio, page);
unsigned long folio_start = address - idx * PAGE_SIZE;
unsigned long folio_end = folio_start + nr * PAGE_SIZE;
- pte_t *folio_ptep;
+ hw_pte_t *folio_ptep;
pte_t folio_pte;
if (unlikely(folio_start < max(address & PMD_MASK, vma->vm_start)))
@@ -5255,7 +5256,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
return ret;
}
-static bool pte_range_none(pte_t *pte, int nr_pages)
+static bool pte_range_none(hw_pte_t *pte, int nr_pages)
{
int i;
@@ -5274,7 +5275,7 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf)
unsigned long orders;
struct folio *folio;
unsigned long addr;
- pte_t *pte;
+ hw_pte_t *pte;
gfp_t gfp;
int order;
@@ -5356,7 +5357,7 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf)
return folio_prealloc(vma->vm_mm, vma, vmf->address, true);
}
-void map_anon_folio_pte_nopf(struct folio *folio, pte_t *pte,
+void map_anon_folio_pte_nopf(struct folio *folio, hw_pte_t *pte,
struct vm_area_struct *vma, unsigned long addr,
bool uffd_wp)
{
@@ -5377,7 +5378,7 @@ void map_anon_folio_pte_nopf(struct folio *folio, pte_t *pte,
update_mmu_cache_range(NULL, vma, addr, pte, nr_pages);
}
-static void map_anon_folio_pte_pf(struct folio *folio, pte_t *pte,
+static void map_anon_folio_pte_pf(struct folio *folio, hw_pte_t *pte,
struct vm_area_struct *vma, unsigned long addr, bool uffd_wp)
{
const unsigned int order = folio_order(folio);
@@ -6162,7 +6163,7 @@ int numa_migrate_check(struct folio *folio, struct vm_fault *vmf,
}
static void numa_rebuild_single_mapping(struct vm_fault *vmf, struct vm_area_struct *vma,
- unsigned long fault_addr, pte_t *fault_pte,
+ unsigned long fault_addr, hw_pte_t *fault_pte,
bool writable)
{
pte_t pte, old_pte;
@@ -6184,7 +6185,7 @@ static void numa_rebuild_large_mapping(struct vm_fault *vmf, struct vm_area_stru
unsigned long start, end, addr = vmf->address;
unsigned long addr_start = addr - (nr << PAGE_SHIFT);
unsigned long pt_start = ALIGN_DOWN(addr, PMD_SIZE);
- pte_t *start_ptep;
+ hw_pte_t *start_ptep;
/* Stay within the VMA and within the page table. */
start = max3(addr_start, pt_start, vma->vm_start);
@@ -6944,7 +6945,7 @@ int __pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address)
#endif /* __PAGETABLE_PMD_FOLDED */
static inline void pfnmap_args_setup(struct follow_pfnmap_args *args,
- spinlock_t *lock, pte_t *ptep,
+ spinlock_t *lock, hw_pte_t *ptep,
pgprot_t pgprot, unsigned long pfn_base,
unsigned long addr_mask, bool writable,
bool special)
@@ -7013,7 +7014,8 @@ int follow_pfnmap_start(struct follow_pfnmap_args *args)
p4d_t *p4dp, p4d;
pud_t *pudp, pud;
pmd_t *pmdp, pmd;
- pte_t *ptep, pte;
+ hw_pte_t *ptep;
+ pte_t pte;
pfnmap_lockdep_assert(vma);
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 2ad0a5f18280a..51ab12fa4cb61 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -710,7 +710,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr,
struct folio *folio;
struct queue_pages *qp = walk->private;
unsigned long flags = qp->flags;
- pte_t *pte, *mapped_pte;
+ hw_pte_t *pte, *mapped_pte;
pte_t ptent;
spinlock_t *ptl;
int max_nr, nr;
@@ -790,7 +790,7 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned long addr,
return 0;
}
-static int queue_folios_hugetlb(pte_t *pte, unsigned long hmask,
+static int queue_folios_hugetlb(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
diff --git a/mm/migrate.c b/mm/migrate.c
index 15b45832bcfa7..1786ab5e941c2 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -494,7 +494,7 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd,
unsigned long address)
{
spinlock_t *ptl;
- pte_t *ptep;
+ hw_pte_t *ptep;
pte_t pte;
softleaf_t entry;
@@ -525,7 +525,7 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd,
*
* This function will release the vma lock before returning.
*/
-void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep)
+void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep)
{
spinlock_t *ptl = huge_pte_lockptr(hstate_vma(vma), vma->vm_mm, ptep);
softleaf_t entry;
diff --git a/mm/migrate_device.c b/mm/migrate_device.c
index 0c437004329d9..10da0b00aaca5 100644
--- a/mm/migrate_device.c
+++ b/mm/migrate_device.c
@@ -254,7 +254,7 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp,
spinlock_t *ptl;
struct folio *fault_folio = migrate->fault_page ?
page_folio(migrate->fault_page) : NULL;
- pte_t *ptep;
+ hw_pte_t *ptep;
again:
if (pmd_trans_huge(*pmdp) || !pmd_present(*pmdp)) {
@@ -989,7 +989,7 @@ static void migrate_vma_insert_page(struct migrate_vma *migrate,
p4d_t *p4dp;
pud_t *pudp;
pmd_t *pmdp;
- pte_t *ptep;
+ hw_pte_t *ptep;
pte_t orig_pte;
/* Only allow populating anonymous memory */
diff --git a/mm/mincore.c b/mm/mincore.c
index c086836bc4bcc..65cc43c85d03f 100644
--- a/mm/mincore.c
+++ b/mm/mincore.c
@@ -24,7 +24,7 @@
#include "swap.h"
#include "internal.h"
-static int mincore_hugetlb(pte_t *pte, unsigned long hmask, unsigned long addr,
+static int mincore_hugetlb(hw_pte_t *pte, unsigned long hmask, unsigned long addr,
unsigned long end, struct mm_walk *walk)
{
#ifdef CONFIG_HUGETLB_PAGE
@@ -164,7 +164,7 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
{
spinlock_t *ptl;
struct vm_area_struct *vma = walk->vma;
- pte_t *ptep;
+ hw_pte_t *ptep;
unsigned char *vec = walk->private;
int nr = (end - addr) >> PAGE_SHIFT;
int step, i;
diff --git a/mm/mlock.c b/mm/mlock.c
index 39215a3eab1fb..cc6c24e46348a 100644
--- a/mm/mlock.c
+++ b/mm/mlock.c
@@ -305,7 +305,7 @@ void munlock_folio(struct folio *folio)
}
static inline unsigned int folio_mlock_step(struct folio *folio,
- pte_t *pte, unsigned long addr, unsigned long end)
+ hw_pte_t *pte, unsigned long addr, unsigned long end)
{
unsigned int count = (end - addr) >> PAGE_SHIFT;
pte_t ptent = ptep_get(pte);
@@ -353,7 +353,7 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr,
{
struct vm_area_struct *vma = walk->vma;
spinlock_t *ptl;
- pte_t *start_pte, *pte;
+ hw_pte_t *start_pte, *pte;
pte_t ptent;
struct folio *folio;
unsigned int step = 1;
diff --git a/mm/mprotect.c b/mm/mprotect.c
index 2888ee638d872..1d23475e0bb76 100644
--- a/mm/mprotect.c
+++ b/mm/mprotect.c
@@ -103,7 +103,7 @@ bool can_change_pte_writable(struct vm_area_struct *vma, unsigned long addr,
return can_change_shared_pte_writable(vma, pte);
}
-static int mprotect_folio_pte_batch(struct folio *folio, pte_t *ptep,
+static int mprotect_folio_pte_batch(struct folio *folio, hw_pte_t *ptep,
pte_t pte, int max_nr_ptes, fpb_t flags)
{
/* No underlying folio, so cannot batch */
@@ -118,7 +118,7 @@ static int mprotect_folio_pte_batch(struct folio *folio, pte_t *ptep,
/* Set nr_ptes number of ptes, starting from idx */
static __always_inline void prot_commit_flush_ptes(struct vm_area_struct *vma,
- unsigned long addr, pte_t *ptep, pte_t oldpte, pte_t ptent,
+ unsigned long addr, hw_pte_t *ptep, pte_t oldpte, pte_t ptent,
int nr_ptes, int idx, bool set_write, struct mmu_gather *tlb)
{
/*
@@ -170,7 +170,7 @@ static __always_inline int page_anon_exclusive_batch(int start_idx, int max_len,
* retrieve sub-batches.
*/
static __always_inline void commit_anon_folio_batch(struct vm_area_struct *vma,
- struct folio *folio, struct page *first_page, unsigned long addr, pte_t *ptep,
+ struct folio *folio, struct page *first_page, unsigned long addr, hw_pte_t *ptep,
pte_t oldpte, pte_t ptent, int nr_ptes, struct mmu_gather *tlb)
{
bool expected_anon_exclusive;
@@ -189,7 +189,7 @@ static __always_inline void commit_anon_folio_batch(struct vm_area_struct *vma,
}
static __always_inline void set_write_prot_commit_flush_ptes(struct vm_area_struct *vma,
- struct folio *folio, struct page *page, unsigned long addr, pte_t *ptep,
+ struct folio *folio, struct page *page, unsigned long addr, hw_pte_t *ptep,
pte_t oldpte, pte_t ptent, int nr_ptes, struct mmu_gather *tlb)
{
bool set_write;
@@ -212,7 +212,7 @@ static __always_inline void set_write_prot_commit_flush_ptes(struct vm_area_stru
}
static long change_softleaf_pte(struct vm_area_struct *vma,
- unsigned long addr, pte_t *pte, pte_t oldpte, unsigned long cp_flags)
+ unsigned long addr, hw_pte_t *pte, pte_t oldpte, unsigned long cp_flags)
{
const bool uffd_prot = cp_flags & (MM_CP_UFFD_WP | MM_CP_UFFD_RWP);
const bool uffd_prot_resolve = cp_flags &
@@ -279,7 +279,7 @@ static long change_softleaf_pte(struct vm_area_struct *vma,
}
static __always_inline void change_present_ptes(struct mmu_gather *tlb,
- struct vm_area_struct *vma, unsigned long addr, pte_t *ptep,
+ struct vm_area_struct *vma, unsigned long addr, hw_pte_t *ptep,
int nr_ptes, unsigned long end, pgprot_t newprot,
struct folio *folio, struct page *page, unsigned long cp_flags)
{
@@ -332,7 +332,8 @@ static long change_pte_range(struct mmu_gather *tlb,
struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr,
unsigned long end, pgprot_t newprot, unsigned long cp_flags)
{
- pte_t *pte, oldpte;
+ hw_pte_t *pte;
+ pte_t oldpte;
spinlock_t *ptl;
long pages = 0;
bool is_private_single_threaded;
@@ -727,7 +728,7 @@ long change_protection(struct mmu_gather *tlb,
return pages;
}
-static int prot_none_pte_entry(pte_t *pte, unsigned long addr,
+static int prot_none_pte_entry(hw_pte_t *pte, unsigned long addr,
unsigned long next, struct mm_walk *walk)
{
return pfn_modify_allowed(pte_pfn(ptep_get(pte)),
@@ -736,7 +737,7 @@ static int prot_none_pte_entry(pte_t *pte, unsigned long addr,
}
#ifdef CONFIG_HUGETLB_PAGE
-static int prot_none_hugetlb_entry(pte_t *pte, unsigned long hmask,
+static int prot_none_hugetlb_entry(hw_pte_t *pte, unsigned long hmask,
unsigned long addr, unsigned long next,
struct mm_walk *walk)
{
diff --git a/mm/mremap.c b/mm/mremap.c
index 7c368440fafe2..5272cea12f80e 100644
--- a/mm/mremap.c
+++ b/mm/mremap.c
@@ -176,7 +176,7 @@ static pte_t move_soft_dirty_pte(pte_t pte)
}
static int mremap_folio_pte_batch(struct vm_area_struct *vma, unsigned long addr,
- pte_t *ptep, pte_t pte, int max_nr)
+ hw_pte_t *ptep, pte_t pte, int max_nr)
{
struct folio *folio;
@@ -200,7 +200,7 @@ static int move_ptes(struct pagetable_move_control *pmc,
struct vm_area_struct *vma = pmc->old;
bool need_clear_uffd_wp = vma_has_uffd_without_event_remap(vma);
struct mm_struct *mm = vma->vm_mm;
- pte_t *old_ptep, *new_ptep;
+ hw_pte_t *old_ptep, *new_ptep;
pte_t old_pte, pte;
pmd_t dummy_pmdval;
spinlock_t *old_ptl, *new_ptl;
diff --git a/mm/page_table_check.c b/mm/page_table_check.c
index 143a918c0bde2..705794a817bad 100644
--- a/mm/page_table_check.c
+++ b/mm/page_table_check.c
@@ -208,7 +208,7 @@ static void page_table_check_pte_flags(pte_t pte)
}
void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr,
- pte_t *ptep, pte_t pte, unsigned int nr)
+ hw_pte_t *ptep, pte_t pte, unsigned int nr)
{
unsigned int i;
@@ -279,7 +279,7 @@ void __page_table_check_pte_clear_range(struct mm_struct *mm,
return;
if (!pmd_none(pmd) && !pmd_bad(pmd) && !pmd_leaf(pmd)) {
- pte_t *ptep = pte_offset_map(&pmd, addr);
+ hw_pte_t *ptep = pte_offset_map(&pmd, addr);
unsigned long i;
if (WARN_ON(!ptep))
diff --git a/mm/pagewalk.c b/mm/pagewalk.c
index 7411702a37f58..b6a29a06a19bc 100644
--- a/mm/pagewalk.c
+++ b/mm/pagewalk.c
@@ -26,7 +26,7 @@ static int real_depth(int depth)
return depth;
}
-static int walk_pte_range_inner(pte_t *pte, unsigned long addr,
+static int walk_pte_range_inner(hw_pte_t *pte, unsigned long addr,
unsigned long end, struct mm_walk *walk)
{
const struct mm_walk_ops *ops = walk->ops;
@@ -61,7 +61,7 @@ static int walk_pte_range_inner(pte_t *pte, unsigned long addr,
static int walk_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
struct mm_walk *walk)
{
- pte_t *pte;
+ hw_pte_t *pte;
int err = 0;
spinlock_t *ptl;
@@ -341,7 +341,7 @@ static int walk_hugetlb_range(unsigned long addr, unsigned long end,
unsigned long next;
unsigned long hmask = huge_page_mask(h);
unsigned long sz = huge_page_size(h);
- pte_t *pte;
+ hw_pte_t *pte;
const struct mm_walk_ops *ops = walk->ops;
int err = 0;
@@ -905,7 +905,8 @@ struct folio *folio_walk_start(struct folio_walk *fw,
struct page *page;
pud_t *pudp, pud;
pmd_t *pmdp, pmd;
- pte_t *ptep, pte;
+ hw_pte_t *ptep;
+ pte_t pte;
spinlock_t *ptl;
pgd_t *pgdp;
p4d_t *p4dp;
diff --git a/mm/percpu.c b/mm/percpu.c
index 3eff382e565ad..ec0e58e9f6c09 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -3173,7 +3173,7 @@ void __init __weak pcpu_populate_pte(unsigned long addr)
pmd = pmd_offset(pud, addr);
if (!pmd_present(*pmd)) {
- pte_t *new;
+ hw_pte_t *new;
new = memblock_alloc_or_panic(PTE_TABLE_SIZE, PTE_TABLE_SIZE);
pmd_populate_kernel(&init_mm, pmd, new);
diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c
index 224c444cf45d9..df548ad3d9a54 100644
--- a/mm/pgtable-generic.c
+++ b/mm/pgtable-generic.c
@@ -80,7 +80,7 @@ void pmd_clear_bad(pmd_t *pmd)
* force that call on sun4c so we changed this macro slightly
*/
int ptep_set_access_flags(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep,
+ unsigned long address, hw_pte_t *ptep,
pte_t entry, int dirty)
{
int changed = !pte_same(ptep_get(ptep), entry);
@@ -94,7 +94,7 @@ int ptep_set_access_flags(struct vm_area_struct *vma,
#ifndef __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH
bool ptep_clear_flush_young(struct vm_area_struct *vma,
- unsigned long address, pte_t *ptep)
+ unsigned long address, hw_pte_t *ptep)
{
bool young;
@@ -107,7 +107,7 @@ bool ptep_clear_flush_young(struct vm_area_struct *vma,
#ifndef __HAVE_ARCH_PTEP_CLEAR_FLUSH
pte_t ptep_clear_flush(struct vm_area_struct *vma, unsigned long address,
- pte_t *ptep)
+ hw_pte_t *ptep)
{
struct mm_struct *mm = (vma)->vm_mm;
pte_t pte;
@@ -294,7 +294,7 @@ static unsigned long pmdp_get_lockless_start(void) { return 0; }
static void pmdp_get_lockless_end(unsigned long irqflags) { }
#endif
-pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp)
+hw_pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp)
{
unsigned long irqflags;
pmd_t pmdval;
@@ -320,11 +320,11 @@ pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp)
return NULL;
}
-pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,
+hw_pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, spinlock_t **ptlp)
{
pmd_t pmdval;
- pte_t *pte;
+ hw_pte_t *pte;
pte = __pte_offset_map(pmd, addr, &pmdval);
if (likely(pte))
@@ -332,11 +332,11 @@ pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd,
return pte;
}
-pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,
+hw_pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, pmd_t *pmdvalp,
spinlock_t **ptlp)
{
- pte_t *pte;
+ hw_pte_t *pte;
VM_WARN_ON_ONCE(!pmdvalp);
pte = __pte_offset_map(pmd, addr, pmdvalp);
@@ -402,12 +402,12 @@ pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd,
* table, and may not use RCU at all: "outsiders" like khugepaged should avoid
* pte_offset_map() and co once the vma is detached from mm or mm_users is zero.
*/
-pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd,
+hw_pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd,
unsigned long addr, spinlock_t **ptlp)
{
spinlock_t *ptl;
pmd_t pmdval;
- pte_t *pte;
+ hw_pte_t *pte;
again:
pte = __pte_offset_map(pmd, addr, &pmdval);
if (unlikely(!pte))
diff --git a/mm/ptdump.c b/mm/ptdump.c
index 5851096e6f656..8f19f20be3c44 100644
--- a/mm/ptdump.c
+++ b/mm/ptdump.c
@@ -19,7 +19,7 @@ static inline int note_kasan_page_table(struct mm_walk *walk,
{
struct ptdump_state *st = walk->private;
- st->note_page_pte(st, addr, kasan_early_shadow_pte[0]);
+ st->note_page_pte(st, addr, ptep_get(kasan_early_shadow_pte));
walk->action = ACTION_CONTINUE;
@@ -117,7 +117,7 @@ static int ptdump_pmd_entry(pmd_t *pmd, unsigned long addr,
return 0;
}
-static int ptdump_pte_entry(pte_t *pte, unsigned long addr,
+static int ptdump_pte_entry(hw_pte_t *pte, unsigned long addr,
unsigned long next, struct mm_walk *walk)
{
struct ptdump_state *st = walk->private;
diff --git a/mm/rmap.c b/mm/rmap.c
index b5cc9273fe5a4..0223c4ab0b361 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -1122,7 +1122,7 @@ static int page_vma_mkclean_one(struct page_vma_mapped_walk *pvmw)
address = pvmw->address;
if (pvmw->pte) {
- pte_t *pte = pvmw->pte;
+ hw_pte_t *pte = pvmw->pte;
pte_t entry = ptep_get(pte);
/*
@@ -2139,7 +2139,7 @@ static pte_t swp_pte_prepare(swp_entry_t entry, pte_t old_pte,
static bool ttu_anon_swapbacked_folio(struct vm_area_struct *vma,
struct folio *folio, struct page *page, unsigned long address,
- pte_t *ptep, pte_t pteval)
+ hw_pte_t *ptep, pte_t pteval)
{
const bool anon_exclusive = folio_test_anon(folio) &&
PageAnonExclusive(page);
@@ -2174,7 +2174,7 @@ static bool ttu_anon_swapbacked_folio(struct vm_area_struct *vma,
}
static bool ttu_anon_folio(struct vm_area_struct *vma, struct folio *folio,
- struct page *page, unsigned long address, pte_t *ptep,
+ struct page *page, unsigned long address, hw_pte_t *ptep,
pte_t pteval, unsigned long nr_pages)
{
/*
diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c
index e62e6aa07f126..77eb1749e123f 100644
--- a/mm/sparse-vmemmap.c
+++ b/mm/sparse-vmemmap.c
@@ -135,7 +135,7 @@ static void * __meminit altmap_alloc_block_buf(unsigned long size,
return __va(__pfn_to_phys(pfn));
}
-void __meminit vmemmap_verify(pte_t *pte, int node,
+void __meminit vmemmap_verify(hw_pte_t *pte, int node,
unsigned long start, unsigned long end)
{
unsigned long pfn = pte_pfn(ptep_get(pte));
@@ -236,11 +236,11 @@ static __meminit void *vmemmap_alloc_pte(unsigned long pfn, int node,
return page_address(page);
}
-static pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node,
+static hw_pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node,
struct vmem_altmap *altmap,
unsigned long ptpfn, unsigned long flags)
{
- pte_t *pte = pte_offset_kernel(pmd, addr);
+ hw_pte_t *pte = pte_offset_kernel(pmd, addr);
unsigned long pfn = page_to_pfn((struct page *)addr);
if (pte_none(ptep_get(pte))) {
@@ -323,7 +323,7 @@ static pgd_t * __meminit vmemmap_pgd_populate(unsigned long addr, int node)
return pgd;
}
-static pte_t * __meminit vmemmap_populate_address(unsigned long addr, int node,
+static hw_pte_t * __meminit vmemmap_populate_address(unsigned long addr, int node,
struct vmem_altmap *altmap,
unsigned long ptpfn,
unsigned long flags)
@@ -332,7 +332,7 @@ static pte_t * __meminit vmemmap_populate_address(unsigned long addr, int node,
p4d_t *p4d;
pud_t *pud;
pmd_t *pmd;
- pte_t *pte;
+ hw_pte_t *pte;
pgd = vmemmap_pgd_populate(addr, node);
if (!pgd)
@@ -361,7 +361,7 @@ static int __meminit vmemmap_populate_range(unsigned long start,
unsigned long flags)
{
unsigned long addr = start;
- pte_t *pte;
+ hw_pte_t *pte;
for (; addr < end; addr += PAGE_SIZE) {
pte = vmemmap_populate_address(addr, node, altmap,
@@ -394,7 +394,7 @@ void vmemmap_wrprotect_hvo(unsigned long addr, unsigned long end,
int node, unsigned long headsize)
{
unsigned long maddr;
- pte_t *pte;
+ hw_pte_t *pte;
for (maddr = addr + headsize; maddr < end; maddr += PAGE_SIZE) {
pte = virt_to_kpte(maddr);
@@ -413,7 +413,7 @@ int __weak __meminit vmemmap_check_pmd(pmd_t *pmd, int node,
{
if (!pmd_leaf(pmdp_get(pmd)))
return 0;
- vmemmap_verify((pte_t *)pmd, node, addr, next);
+ vmemmap_verify((hw_pte_t *)pmd, node, addr, next);
return 1;
}
@@ -497,9 +497,9 @@ static bool __meminit reuse_compound_section(unsigned long start_pfn,
return !IS_ALIGNED(offset, nr_pages) && nr_pages > PAGES_PER_SUBSECTION;
}
-static pte_t * __meminit compound_section_tail_page(unsigned long addr)
+static hw_pte_t * __meminit compound_section_tail_page(unsigned long addr)
{
- pte_t *pte;
+ hw_pte_t *pte;
addr -= PAGE_SIZE;
@@ -520,7 +520,7 @@ static int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn,
struct dev_pagemap *pgmap)
{
unsigned long size, addr;
- pte_t *pte;
+ hw_pte_t *pte;
int rc;
if (reuse_compound_section(start_pfn, pgmap)) {
diff --git a/mm/swap_state.c b/mm/swap_state.c
index 305877e1f4d7b..d654449f393e8 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -918,7 +918,8 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,
struct swap_io_ctx ctx = {};
struct blk_plug plug;
struct folio *folio;
- pte_t *pte = NULL, pentry;
+ hw_pte_t *pte = NULL;
+ pte_t pentry;
int win;
unsigned long start, end, addr;
pgoff_t ilx = targ_ilx;
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 869a918b18e8e..2f6aa8dc2f5bb 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -2500,7 +2500,8 @@ static int unuse_pte(struct vm_area_struct *vma, pmd_t *pmd,
struct page *page;
struct folio *swapcache;
spinlock_t *ptl;
- pte_t *pte, new_pte, old_pte;
+ hw_pte_t *pte;
+ pte_t new_pte, old_pte;
bool hwpoisoned = false;
int ret = 1;
@@ -2611,7 +2612,7 @@ static int unuse_pte_range(struct vm_area_struct *vma, pmd_t *pmd,
unsigned long addr, unsigned long end,
unsigned int type)
{
- pte_t *pte = NULL;
+ hw_pte_t *pte = NULL;
do {
struct folio *folio;
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 79cc7b546f130..8281796638c0a 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -334,13 +334,13 @@ static int mfill_atomic_install_pte(pmd_t *dst_pmd,
{
int ret;
struct mm_struct *dst_mm = dst_vma->vm_mm;
- pte_t _dst_pte, *dst_pte;
+ hw_pte_t *dst_pte;
bool writable = dst_vma->vm_flags & VM_WRITE;
bool vm_shared = dst_vma->vm_flags & VM_SHARED;
spinlock_t *ptl;
struct folio *folio = page_folio(page);
bool page_in_cache = folio_mapping(folio);
- pte_t dst_ptep;
+ pte_t _dst_pte, dst_ptep;
_dst_pte = mk_pte(page, dst_vma->vm_page_prot);
_dst_pte = pte_mkdirty(_dst_pte);
@@ -629,7 +629,8 @@ static int mfill_atomic_pte_zeropage(struct mfill_state *state)
struct vm_area_struct *dst_vma = state->vma;
unsigned long dst_addr = state->dst_addr;
pmd_t *dst_pmd = state->pmd;
- pte_t _dst_pte, *dst_pte;
+ hw_pte_t *dst_pte;
+ pte_t _dst_pte;
spinlock_t *ptl;
int ret;
@@ -710,7 +711,8 @@ static int mfill_atomic_pte_poison(struct mfill_state *state)
struct mm_struct *dst_mm = dst_vma->vm_mm;
unsigned long dst_addr = state->dst_addr;
pmd_t *dst_pmd = state->pmd;
- pte_t _dst_pte, *dst_pte;
+ hw_pte_t *dst_pte;
+ pte_t _dst_pte;
spinlock_t *ptl;
int ret;
@@ -757,7 +759,7 @@ static __always_inline ssize_t mfill_atomic_hugetlb(
{
struct mm_struct *dst_mm = dst_vma->vm_mm;
ssize_t err;
- pte_t *dst_pte;
+ hw_pte_t *dst_pte;
unsigned long src_addr, dst_addr;
long copied;
struct folio *folio;
@@ -1234,7 +1236,7 @@ void double_pt_unlock(spinlock_t *ptl1,
__release(ptl2);
}
-static inline bool is_pte_pages_stable(pte_t *dst_pte, pte_t *src_pte,
+static inline bool is_pte_pages_stable(hw_pte_t *dst_pte, hw_pte_t *src_pte,
pte_t orig_dst_pte, pte_t orig_src_pte,
pmd_t *dst_pmd, pmd_t dst_pmdval)
{
@@ -1252,7 +1254,8 @@ static inline bool is_pte_pages_stable(pte_t *dst_pte, pte_t *src_pte,
*/
static struct folio *check_ptes_for_batched_move(struct vm_area_struct *src_vma,
unsigned long src_addr,
- pte_t *src_pte, pte_t *dst_pte)
+ hw_pte_t *src_pte,
+ hw_pte_t *dst_pte)
{
pte_t orig_dst_pte, orig_src_pte;
struct folio *folio;
@@ -1283,7 +1286,7 @@ static long move_present_ptes(struct mm_struct *mm,
struct vm_area_struct *dst_vma,
struct vm_area_struct *src_vma,
unsigned long dst_addr, unsigned long src_addr,
- pte_t *dst_pte, pte_t *src_pte,
+ hw_pte_t *dst_pte, hw_pte_t *src_pte,
pte_t orig_dst_pte, pte_t orig_src_pte,
pmd_t *dst_pmd, pmd_t dst_pmdval,
spinlock_t *dst_ptl, spinlock_t *src_ptl,
@@ -1370,7 +1373,7 @@ static long move_present_ptes(struct mm_struct *mm,
static int move_swap_pte(struct mm_struct *mm, struct vm_area_struct *dst_vma,
unsigned long dst_addr, unsigned long src_addr,
- pte_t *dst_pte, pte_t *src_pte,
+ hw_pte_t *dst_pte, hw_pte_t *src_pte,
pte_t orig_dst_pte, pte_t orig_src_pte,
pmd_t *dst_pmd, pmd_t dst_pmdval,
spinlock_t *dst_ptl, spinlock_t *src_ptl,
@@ -1435,7 +1438,7 @@ static int move_zeropage_pte(struct mm_struct *mm,
struct vm_area_struct *dst_vma,
struct vm_area_struct *src_vma,
unsigned long dst_addr, unsigned long src_addr,
- pte_t *dst_pte, pte_t *src_pte,
+ hw_pte_t *dst_pte, hw_pte_t *src_pte,
pte_t orig_dst_pte, pte_t orig_src_pte,
pmd_t *dst_pmd, pmd_t dst_pmdval,
spinlock_t *dst_ptl, spinlock_t *src_ptl)
@@ -1481,8 +1484,8 @@ static long move_pages_ptes(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd
pte_t orig_src_pte, orig_dst_pte;
pte_t src_folio_pte;
spinlock_t *src_ptl, *dst_ptl;
- pte_t *src_pte = NULL;
- pte_t *dst_pte = NULL;
+ hw_pte_t *src_pte = NULL;
+ hw_pte_t *dst_pte = NULL;
pmd_t dummy_pmdval;
pmd_t dst_pmdval;
struct folio *src_folio = NULL;
@@ -2597,7 +2600,8 @@ static inline bool userfaultfd_huge_must_wait(struct userfaultfd_ctx *ctx,
unsigned long reason)
{
struct vm_area_struct *vma = vmf->vma;
- pte_t *ptep, pte;
+ hw_pte_t *ptep;
+ pte_t pte;
assert_fault_locked(vmf);
@@ -2670,7 +2674,7 @@ static inline bool userfaultfd_must_wait(struct userfaultfd_ctx *ctx,
p4d_t *p4d;
pud_t *pud;
pmd_t *pmd, _pmd;
- pte_t *pte;
+ hw_pte_t *pte;
pte_t ptent;
bool ret;
diff --git a/mm/util.c b/mm/util.c
index bf0513d1d3d08..cc253050f6892 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -1564,7 +1564,7 @@ EXPORT_SYMBOL(mmap_action_complete);
*
* Return: the number of table entries in the batch.
*/
-unsigned int folio_pte_batch(struct folio *folio, pte_t *ptep, pte_t pte,
+unsigned int folio_pte_batch(struct folio *folio, hw_pte_t *ptep, pte_t pte,
unsigned int max_nr)
{
return folio_pte_batch_flags(folio, NULL, ptep, &pte, max_nr, 0);
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 6ed6c160abed7..60ba46838bbeb 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -97,7 +97,7 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
phys_addr_t phys_addr, pgprot_t prot,
unsigned int max_page_shift, pgtbl_mod_mask *mask)
{
- pte_t *pte;
+ hw_pte_t *pte;
u64 pfn;
struct page *page;
unsigned long size = PAGE_SIZE;
@@ -389,7 +389,7 @@ int ioremap_page_range(unsigned long addr, unsigned long end,
static void vunmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
pgtbl_mod_mask *mask)
{
- pte_t *pte;
+ hw_pte_t *pte;
pte_t ptent;
unsigned long size = PAGE_SIZE;
@@ -550,7 +550,7 @@ static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr,
pgtbl_mod_mask *mask)
{
int err = 0;
- pte_t *pte;
+ hw_pte_t *pte;
/*
* nr is a running index into the array which helps higher level
@@ -827,7 +827,8 @@ struct page *vmalloc_to_page(const void *vmalloc_addr)
p4d_t *p4d;
pud_t *pud;
pmd_t *pmd;
- pte_t *ptep, pte;
+ hw_pte_t *ptep;
+ pte_t pte;
/*
* XXX we might need to change this if we add VIRTUAL_BUG_ON for
@@ -3615,7 +3616,7 @@ struct vmap_pfn_data {
unsigned int idx;
};
-static int vmap_pfn_apply(pte_t *pte, unsigned long addr, void *private)
+static int vmap_pfn_apply(hw_pte_t *pte, unsigned long addr, void *private)
{
struct vmap_pfn_data *data = private;
unsigned long pfn = data->pfns[data->idx];
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 245f68c75b289..4cadd4f4b00d1 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3579,7 +3579,7 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end,
{
int i;
bool dirty;
- pte_t *pte;
+ hw_pte_t *pte;
spinlock_t *ptl;
unsigned long addr;
int total = 0;
@@ -3612,7 +3612,7 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end,
for (i = pte_index(start), addr = start; addr != end; i += nr, addr += nr * PAGE_SIZE) {
unsigned long pfn;
struct folio *folio;
- pte_t *cur_pte = pte + i;
+ hw_pte_t *cur_pte = pte + i;
pte_t ptent = ptep_get(cur_pte);
nr = 1;
@@ -4343,7 +4343,7 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr)
struct lru_gen_mm_walk *walk;
struct folio *last = NULL;
int young = nr;
- pte_t *pte = pvmw->pte;
+ hw_pte_t *pte = pvmw->pte;
unsigned long addr = pvmw->address;
struct vm_area_struct *vma = pvmw->vma;
struct folio *folio = pfn_folio(pvmw->pfn);
|
| 23/3 | 2026/09/04 15:32 | llm | kmsan-evaluator |
0mModel:gemini-3.7-flash Args: null Results: null Tokens: input: 55730 output: 181 thoughts: 1954 Thoughts: **KMSAN Evaluation: PTE Type Refactoring**
Alright, let's break this down. My goal is to determine if this patch warrants a dedicated KMSAN fuzzing session. The patch introduces a new type, `hw_pte_t`, to distinguish pointers to hardware page table entries from `pte_t` values. I need to figure out if this refactoring introduces any risks related to uninitialized memory or information leaks.
First, I examined the patch in detail. It defines `hw_pte_t` as either a `struct __hw_pte_t { pte_t __pte; }` or simply `pte_t` depending on whether the architecture has hardware PTEs. The core of the patch involves replacing instances of `pte_t *` with `hw_pte_t *` in various parts of the kernel's memory management code, including core MM, drivers, and related structures like `pgtable.h` and function signatures across the board, such as `apply_to_page_range`, `walk_pte_range`, etc. In effect, we're specializing the pointer type for clarity and safety, but not changing how the values are used.
Then I checked for any subtle bugs or uninitialized variables introduced by the patch. I meticulously reviewed all the locations where variable declarations were changed. I noted where `pte_t *` was replaced with `hw_pte_t *` or variables introduced, along with the other variables in those scopes: `fs/hugetlbfs/inode.c`, `fs/proc/task_mmu.c`, `kernel/events/core.c`, `mm/damon/vaddr.c`, `mm/gup.c`, etc. I've gone through the whole list.
The key here is that the patch itself doesn't change the underlying memory initialization, control flow, allocations, or how data is passed to userspace. It's strictly a type refactoring. No memory allocation logic is changed. No values that were previously uninitialized will be affected, and no new padding or struct members are sent to userspace.
Remembering the KMSAN rules: I need to consider it if the patch:
1. Modifies kernel structures sent to userspace, where padding could be uninitialized. Nope.
2. Introduces conditional logic based on potentially uninitialized variables. Nope.
3. Changes the allocation or initialization of complex data structures where uninitialized fields could be read. Nope.
4. Modifies bounds checks/lengths/logic to allow access to uninitialized bytes. Nope.
Since this patch is a pure type refactoring, with no changes to the initialization of any memory locations, I'm confident that it does *not* introduce any KMSAN concerns. Therefore, I'm going to set `NeedsKMSAN=false`. The explanation will be straightforward: "This patch is a clean type refactoring, introducing `hw_pte_t` to differentiate pointers to hardware PTE tables from `pte_t` values. No memory initialization logic is altered, and there is no risk of uninitialized memory usage or info-leaks."
|
| 24/3 | 2026/09/04 15:32 | tool | set-results |
0mArgs:{
"KMSANReasoning": "The patch series is a purely mechanical type-system refactoring across the memory management subsystem and arch-independent code. It introduces the `hw_pte_t` type (and `__pte_from_hw()` helper) to distinguish between pointers directly referencing hardware page table entries (`hw_pte_t *`) and page table values manipulated in software (`pte_t`).\n\nNo changes are made to data initialization, memory allocation semantics, struct layouts exposed to userspace, or data copying mechanisms (e.g. `copy_to_user`). There are no new branching conditions on uninitialized memory, no modifications to bounds checks, and no risk of uninitialized memory reads or information leaks to userspace. Therefore, dedicated KMSAN fuzzing is not required.",
"NeedsKMSAN": false
}Results: {
"KMSANReasoning": "The patch series is a purely mechanical type-system refactoring across the memory management subsystem and arch-independent code. It introduces the `hw_pte_t` type (and `__pte_from_hw()` helper) to distinguish between pointers directly referencing hardware page table entries (`hw_pte_t *`) and page table values manipulated in software (`pte_t`).\n\nNo changes are made to data initialization, memory allocation semantics, struct layouts exposed to userspace, or data copying mechanisms (e.g. `copy_to_user`). There are no new branching conditions on uninitialized memory, no modifications to bounds checks, and no risk of uninitialized memory reads or information leaks to userspace. Therefore, dedicated KMSAN fuzzing is not required.",
"NeedsKMSAN": false
} |
| Total Calls | Total Tokens | Avg Tokens | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|
| Total Calls | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|