change_protection() applies vma->vm_page_prot, which lacks the write bit for CoW and write-notified mappings, so it write-protects every entry in them. When an entry would simply be made writable on fault without further checks (e.g. dirty PTEs in shared mappings, exclusive anonymous pages), MM_CP_TRY_CHANGE_WRITABLE makes the entry writable again to avoid the future fault. However, writable entries in shared mappings that are not dirty still become read-only. For example, a uffd-wp unprotect makes clean shmem PTEs read-only even if they were never write-protected, e.g. when they were mapped by a read fault before uffd-wp was registered. The next write to each of them causes a fault that only restores the write bit. Now that MM_CP_PERM_UPGRADE indicates whether mprotect() only adds permissions, change_protection() can tell, together with the userfaultfd unprotect flags, when a change revokes no access. Keep writable entries writable in these cases, at the PTE, PMD and PUD levels. This is consistent with the write fault handler, as an entry that is already writable needs no write fault. Signed-off-by: Carlos López --- mm/huge_memory.c | 7 +++++++ mm/internal.h | 7 +++++++ mm/mprotect.c | 4 ++++ 3 files changed, 18 insertions(+) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 506e1e61c0fd..6e23d0e4ae66 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2809,6 +2809,10 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, if (userfaultfd_rwp(vma) && pmd_uffd(entry)) entry = pmd_modify(entry, PAGE_NONE); + /* See change_present_ptes(). */ + if (mm_cp_is_perm_upgrade(cp_flags) && pmd_write(oldpmd)) + entry = pmd_mkwrite(entry, vma); + /* See change_pte_range(). */ if ((cp_flags & MM_CP_TRY_CHANGE_WRITABLE) && !pmd_write(entry) && can_change_pmd_writable(vma, addr, entry)) @@ -2885,6 +2889,9 @@ int change_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma, */ oldpud = pudp_invalidate(vma, addr, pudp); entry = pud_modify(oldpud, newprot); + /* See change_present_ptes(). */ + if (mm_cp_is_perm_upgrade(cp_flags) && pud_write(oldpud)) + entry = pud_mkwrite(entry); /* See change_pte_range(). */ if ((cp_flags & MM_CP_TRY_CHANGE_WRITABLE) && !pud_write(entry) && can_change_pud_writable(vma, addr, entry)) diff --git a/mm/internal.h b/mm/internal.h index 566cbd2dce48..a1a66f3b4b9c 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1423,6 +1423,13 @@ static inline bool pte_needs_soft_dirty_wp(struct vm_area_struct *vma, pte_t pte return vma_soft_dirty_enabled(vma) && !pte_soft_dirty(pte); } +/* Whether a change_protection() only adds permissions. */ +static inline bool mm_cp_is_perm_upgrade(unsigned long cp_flags) +{ + return cp_flags & (MM_CP_PERM_UPGRADE | MM_CP_UFFD_WP_RESOLVE | + MM_CP_UFFD_RWP_RESOLVE); +} + /* shrinker related functions */ unsigned long shrink_slab(gfp_t gfp_mask, int nid, struct mem_cgroup *memcg, int priority); diff --git a/mm/mprotect.c b/mm/mprotect.c index 683131c9c4a0..16b8aef755ab 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -306,6 +306,10 @@ static __always_inline void change_present_ptes(struct mmu_gather *tlb, if (userfaultfd_rwp(vma) && pte_uffd(ptent)) ptent = pte_modify(ptent, PAGE_NONE); + /* A permission upgrade keeps writable entries writable. */ + if (mm_cp_is_perm_upgrade(cp_flags) && pte_write(oldpte)) + ptent = pte_mkwrite(ptent, vma); + /* * In some writable, shared mappings, we might want * to catch actual write access -- see -- 2.51.0