5.10-stable review patch. If anyone has any objections, please let me know. ------------------ From: Lorenzo Stoakes (ARM) [ Upstream commit 26444eb71465c9934d9d418ef69c43f61185329b ] Currently there is a nasty race between ptdump and ioremap when attempting to map a huge P4D, PUD or PMD entry: * ptdump walks kernel page table ranges it doesn't own. * When ioremap maps ranges it tries to promote existing ones to huge page tables in ioremap_try_huge_[p4d,pud,pmd]() at P4D, PUD and PMD level, freeing the lower page table in [p4d,pud,pmd]_free_[pud,pmd,pte]_page() when it succeeds. Both of these things can happen at the same time and as a result ptdump can access a freed page table, resulting in a use-after-free and memory corruption. This is possible because while ptdump_walk_pgd() holds both the mem hotplug lock and the mmap write lock before invoking walk_page_range_novma(), ioremap takes no relevant locks at all. Fix this by holding the mmap read lock in ioremap_try_huge_*() when freeing page tables. The read lock is sufficient: ptdump is the only walker that must be excluded and it holds the mmap write lock. Other holders of the read lock may run concurrently, but each exclusively owns the range it operates on and cannot reach the page tables freed here. We also hold the lock while assigning the huge page table entry, which means page table walkers observe only the huge or non-huge page table entry. We use a trylock to prevent ptdump from blocking ioremap making forward progress. This is fine because it's an optimisation in any case, and thus the ioremap can safely proceed regardless. All other kernel page table walkers that touch vmalloc ranges either exclusively own the memory walked or acquire the mmap lock, so this correctly excludes those walkers. The issue has existed as long as ptdump was available and vmap freed page tables when promoting to a huge leaf entry, that is, since commit b6bdb7517c3d ("mm/vmalloc: add interfaces to free unmapped page table") for huge ioremap, and commit 121e6f3258fe ("mm/vmalloc: hugepage vmalloc mappings") for huge vmalloc. Since the former is the earlier of the two we choose that for our Fixes tag. This patch is based on work by David Carlier (linked), with gratitude! Fixes: b6bdb7517c3d ("mm/vmalloc: add interfaces to free unmapped page table") Signed-off-by: Lorenzo Stoakes (ARM) Reported-by: syzbot+fd95a72470f5a44e464c@syzkaller.appspotmail.com Closes: https://lore.kernel.org/all/6a287988.39669fcc.33b062.00a0.GAE@google.com/T/ Link: https://lore.kernel.org/linux-mm/20260706203128.162335-1-devnexen@gmail.com/ Reviewed-by: Mike Rapoport (Microsoft) Reviewed-by: Dev Jain Acked-by: David Hildenbrand (Arm) Reviewed-by: Kiryl Shutsemau Cc: Signed-off-by: Andrew Morton [ ljs - Applied to ioremap_try_huge_*() in mm/ioremap.c, open-coded missing stuff + other smaller changes. Updated commit msg to reflect. ] Signed-off-by: Lorenzo Stoakes (ARM) Signed-off-by: Sasha Levin --- mm/ioremap.c | 45 +++++++++++++++++++++++++++++++++++++++------ 1 file changed, 39 insertions(+), 6 deletions(-) diff --git a/mm/ioremap.c b/mm/ioremap.c index 5fa1ab41d1526..0f3087e4ea9ef 100644 --- a/mm/ioremap.c +++ b/mm/ioremap.c @@ -85,6 +85,8 @@ static int ioremap_try_huge_pmd(pmd_t *pmd, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { + int ret = 0; + if (!ioremap_pmd_enabled()) return 0; @@ -97,10 +99,23 @@ static int ioremap_try_huge_pmd(pmd_t *pmd, unsigned long addr, if (!IS_ALIGNED(phys_addr, PMD_SIZE)) return 0; - if (pmd_present(*pmd) && !pmd_free_pte_page(pmd, addr)) + if (!pmd_present(*pmd)) + return pmd_set_huge(pmd, phys_addr, prot); + + /* + * Acquire the mmap read lock to exclude ptdump, which walks + * kernel page tables it does not own under the mmap write lock. + * + * Concurrent read lock holders are safe: each exclusively owns + * the range it operates on and cannot reach this page table. + */ + if (!mmap_read_trylock(&init_mm)) return 0; + if (pmd_free_pte_page(pmd, addr)) + ret = pmd_set_huge(pmd, phys_addr, prot); + mmap_read_unlock(&init_mm); - return pmd_set_huge(pmd, phys_addr, prot); + return ret; } static inline int ioremap_pmd_range(pud_t *pud, unsigned long addr, @@ -131,6 +146,8 @@ static int ioremap_try_huge_pud(pud_t *pud, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { + int ret = 0; + if (!ioremap_pud_enabled()) return 0; @@ -143,10 +160,17 @@ static int ioremap_try_huge_pud(pud_t *pud, unsigned long addr, if (!IS_ALIGNED(phys_addr, PUD_SIZE)) return 0; - if (pud_present(*pud) && !pud_free_pmd_page(pud, addr)) + if (!pud_present(*pud)) + return pud_set_huge(pud, phys_addr, prot); + + /* See comment in ioremap_try_huge_pmd(). */ + if (!mmap_read_trylock(&init_mm)) return 0; + if (pud_free_pmd_page(pud, addr)) + ret = pud_set_huge(pud, phys_addr, prot); + mmap_read_unlock(&init_mm); - return pud_set_huge(pud, phys_addr, prot); + return ret; } static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, @@ -177,6 +201,8 @@ static int ioremap_try_huge_p4d(p4d_t *p4d, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { + int ret = 0; + if (!ioremap_p4d_enabled()) return 0; @@ -189,10 +215,17 @@ static int ioremap_try_huge_p4d(p4d_t *p4d, unsigned long addr, if (!IS_ALIGNED(phys_addr, P4D_SIZE)) return 0; - if (p4d_present(*p4d) && !p4d_free_pud_page(p4d, addr)) + if (!p4d_present(*p4d)) + return p4d_set_huge(p4d, phys_addr, prot); + + /* See comment in ioremap_try_huge_pmd(). */ + if (!mmap_read_trylock(&init_mm)) return 0; + if (p4d_free_pud_page(p4d, addr)) + ret = p4d_set_huge(p4d, phys_addr, prot); + mmap_read_unlock(&init_mm); - return p4d_set_huge(p4d, phys_addr, prot); + return ret; } static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, -- 2.53.0