huge_ptep_get() can be handed a virtual address pointing to the middle of a contpmd/contpte mapped hugetlb folio (examples of callers are pagemap_hugetlb_range, page_mapped_in_vma). The arm64 helper rewalks the pgtables in find_num_contig to answer whether the huge pte we have maps a contpmd or a contpte hugetlb folio, and returns CONT_PMDS or CONT_PTES, so that it can collect a/d bits over the contiguous ptes. We can falsely return CONT_PTES instead of CONT_PMDS if the addr is not aligned. Fix this by aligning the pmdp pointer down to a contpmd base before checking equality with the passed huge pte pointer, to correctly answer whether the huge pte is the base of a contpmd block. Fixes: 29cb80519689 ("arm64: hugetlb: Cleanup huge_pte size discovery mechanisms") Cc: stable@vger.kernel.org Acked-by: David Hildenbrand (Arm) Signed-off-by: Dev Jain --- arch/arm64/mm/hugetlbpage.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 30772a909aea3..8e799c1fe0aa6 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -87,7 +87,7 @@ static int find_num_contig(struct mm_struct *mm, unsigned long addr, p4dp = p4d_offset(pgdp, addr); pudp = pud_offset(p4dp, addr); pmdp = pmd_offset(pudp, addr); - if ((pte_t *)pmdp == ptep) { + if ((pte_t *)PTR_ALIGN_DOWN(pmdp, sizeof(*pmdp) * CONT_PMDS) == ptep) { *pgsize = PMD_SIZE; return CONT_PMDS; } -- 2.43.0 try_to_unmap_one() handles hugetlb folios when memory failure needs to replace a poisoned hugetlb mapping with a hwpoison entry. In that case page_vma_mapped_walk() returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads it with ptep_get(). On arches which provide their own huge_ptep_get() to dereference a huge pte pointer, accessing via ptep_get() would cause pte_pfn(), pte_present() etc to misbehave. It is not clear whether this has a trivially visible effect to userspace. Just use huge_ptep_get() for dereferencing a huge pte pointer. Fixes: c7ab0d2fdc84 ("mm: convert try_to_unmap_one() to use page_vma_mapped_walk()") Cc: stable@vger.kernel.org Reported-by: David Hildenbrand Signed-off-by: Dev Jain --- include/linux/hugetlb.h | 2 ++ mm/rmap.c | 16 ++++++++++------ 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 2abaf99321e90..6996d02acf171 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1261,6 +1261,8 @@ static inline void hugetlb_count_sub(long l, struct mm_struct *mm) { } +pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); + static inline pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) { diff --git a/mm/rmap.c b/mm/rmap.c index 1c77d5dc06e9f..03737daa348c0 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -2095,14 +2095,19 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, /* Unexpected PMD-mapped THP? */ VM_BUG_ON_FOLIO(!pvmw.pte, folio); - /* - * Handle PFN swap PTEs, such as device-exclusive ones, that - * actually map pages. - */ - pteval = ptep_get(pvmw.pte); + address = pvmw.address; + if (folio_test_hugetlb(folio)) { + pteval = huge_ptep_get(mm, address, pvmw.pte); + } else { + pteval = ptep_get(pvmw.pte); + } if (likely(pte_present(pteval))) { pfn = pte_pfn(pteval); } else { + /* + * Handle PFN swap PTEs, such as device-exclusive ones, + * that actually map pages. + */ const softleaf_t entry = softleaf_from_pte(pteval); pfn = softleaf_to_pfn(entry); @@ -2110,7 +2115,6 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, } subpage = folio_page(folio, pfn - folio_pfn(folio)); - address = pvmw.address; anon_exclusive = folio_test_anon(folio) && PageAnonExclusive(subpage); -- 2.43.0 try_to_migrate_one() is used by folio migration to replace a present mapping with a migration entry. For hugetlb folios, page_vma_mapped_walk() returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads the huge pte entry with ptep_get(). On arches which provide their own huge_ptep_get() to dereference a huge pte pointer, accessing via ptep_get() would cause pte_pfn(), pte_present() etc to misbehave. It is not clear whether this has a trivially visible effect to userspace. Use huge_ptep_get() to dereference a huge pte pointer. Commit a98a2f0c8ce1 copied the bug from try_to_unmap_one into try_to_migrate_one. Fixes: a98a2f0c8ce1 ("mm/rmap: split migration into its own function") Cc: stable@vger.kernel.org Acked-by: Muchun Song Signed-off-by: Dev Jain --- mm/rmap.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index 03737daa348c0..ea4ad5e662f7d 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -2505,14 +2505,19 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, /* Unexpected PMD-mapped THP? */ VM_BUG_ON_FOLIO(!pvmw.pte, folio); - /* - * Handle PFN swap PTEs, such as device-exclusive ones, that - * actually map pages. - */ - pteval = ptep_get(pvmw.pte); + address = pvmw.address; + if (folio_test_hugetlb(folio)) { + pteval = huge_ptep_get(mm, address, pvmw.pte); + } else { + pteval = ptep_get(pvmw.pte); + } if (likely(pte_present(pteval))) { pfn = pte_pfn(pteval); } else { + /* + * Handle PFN swap PTEs, such as device-exclusive ones, + * that actually map pages. + */ const softleaf_t entry = softleaf_from_pte(pteval); pfn = softleaf_to_pfn(entry); @@ -2520,7 +2525,6 @@ static bool try_to_migrate_one(struct folio *folio, struct vm_area_struct *vma, } subpage = folio_page(folio, pfn - folio_pfn(folio)); - address = pvmw.address; anon_exclusive = folio_test_anon(folio) && PageAnonExclusive(subpage); -- 2.43.0 remove_migration_pte() converts migration entries back to present PTEs after folio migration completes. For hugetlb folios, page_vma_mapped_walk() returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads it with ptep_get(). On arches which provide their own huge_ptep_get() to dereference a huge pte pointer, accessing via ptep_get() would cause pte_pfn(), pte_present() etc to misbehave. It is not clear whether this has a trivially visible effect to userspace. Use huge_ptep_get() to dereference a huge pte pointer. Fixes: 290408d4a250 ("hugetlb: hugepage migration core") Cc: stable@vger.kernel.org Acked-by: Muchun Song Acked-by: David Hildenbrand (Arm) Signed-off-by: Dev Jain --- mm/migrate.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/mm/migrate.c b/mm/migrate.c index d9b23909d716c..c65f0f43df7eb 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -371,7 +371,11 @@ static bool remove_migration_pte(struct folio *folio, continue; } #endif - old_pte = ptep_get(pvmw.pte); + if (folio_test_hugetlb(folio)) + old_pte = huge_ptep_get(vma->vm_mm, pvmw.address, + pvmw.pte); + else + old_pte = ptep_get(pvmw.pte); if (rmap_walk_arg->map_unused_to_zeropage && try_to_map_unused_to_zeropage(&pvmw, folio, old_pte, idx)) continue; -- 2.43.0 check_pte() is the final validation step in page_vma_mapped_walk(). It reads pvmw->pte with ptep_get() to decide whether the entry maps the PFN range being walked. For hugetlb VMAs, that pointer refers to a hugetlb entry. On arches which provide their own huge_ptep_get() to dereference a huge pte pointer, accessing via ptep_get() would cause pte_pfn(), pte_present() etc to misbehave. It is not clear whether this has a trivially visible effect to userspace. Use huge_ptep_get() to dereference a huge pte pointer. Fixes: ace71a19cec5 ("mm: introduce page_vma_mapped_walk()") Cc: stable@vger.kernel.org Acked-by: David Hildenbrand (Arm) Reviewed-by: Muchun Song Signed-off-by: Dev Jain --- mm/page_vma_mapped.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index 2ccbabfb2cc17..18e1d341f463c 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -107,7 +107,13 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, pmd_t *pmdvalp, static bool check_pte(struct page_vma_mapped_walk *pvmw, unsigned long pte_nr) { unsigned long pfn; - pte_t ptent = ptep_get(pvmw->pte); + pte_t ptent; + + if (is_vm_hugetlb_page(pvmw->vma)) + ptent = huge_ptep_get(pvmw->vma->vm_mm, pvmw->address, + pvmw->pte); + else + ptent = ptep_get(pvmw->pte); if (pvmw->flags & PVMW_MIGRATION) { const softleaf_t entry = softleaf_from_pte(ptent); -- 2.43.0 prot_none_hugetlb_entry() is the hugetlb callback for the early mprotect(PROT_NONE) PFN permission walk on x86. The callback passes the decoded PFN to pfn_modify_allowed(). For a hugetlb callback, the pte pointer refers to a hugetlb entry. On architectures where hugetlb entries need huge_ptep_get(), reading that entry with ptep_get() can make the permission check use the wrong PFN. Use huge_ptep_get() before decoding the hugetlb PFN. Currently there is no path which can trigger a bug: huge_ptep_get() is a simple ptep_get() for x86, and the prot_none walk occurs only for x86. So no need to backport - use the correct helper anyways. Fixes: 42e4089c7890 ("x86/speculation/l1tf: Disallow non privileged high MMIO PROT_NONE mappings") Reviewed-by: Muchun Song Signed-off-by: Dev Jain --- mm/mprotect.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/mm/mprotect.c b/mm/mprotect.c index 9cbf932b028cf..6360d8a378023 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -699,14 +699,20 @@ static int prot_none_pte_entry(pte_t *pte, unsigned long addr, 0 : -EACCES; } +#ifdef CONFIG_HUGETLB_PAGE static int prot_none_hugetlb_entry(pte_t *pte, unsigned long hmask, unsigned long addr, unsigned long next, struct mm_walk *walk) { - return pfn_modify_allowed(pte_pfn(ptep_get(pte)), - *(pgprot_t *)(walk->private)) ? - 0 : -EACCES; + const pte_t entry = huge_ptep_get(walk->mm, addr, pte); + + if (pfn_modify_allowed(pte_pfn(entry), *(pgprot_t *)(walk->private))) + return 0; + return -EACCESS; } +#else +#define prot_none_hugetlb_entry NULL +#endif static int prot_none_test(unsigned long addr, unsigned long next, struct mm_walk *walk) -- 2.43.0