Rather than referring to VMA flags with uncertain meaning, add a new predicate that explicitly describes what possession of the VMA_PFNMAP_BIT or VMA_MIXEDMAP_BIT flags mean, and then refer to that function for determining VMA mergeability. Either flag means the contents of the mapping are owned by the kernel, usually a driver, rather than by the core mm: the memory may be MMIO, kernel-allocated pages or even ordinary pages the driver maps itself, but the core must not populate, reclaim, migrate, copy-on-write or merge the range on its own initiative. We initially also include VMA_IO_BIT here, as by implication, these must be kernel-owned. (mlock() also sets VMA_IO_BIT transiently on ordinary VMAs while locking them, which is addressed later in this series.) However the intent is to in future remove this, as no mapping should be marked as an I/O mapping without also being marked with VMA_PFNMAP_BIT. This forms the basis of further work intended to improve how we express VMA properties such as this. Also update the VMA userland tests to reflect the change. No functional change intended. Signed-off-by: Lorenzo Stoakes (ARM) --- include/linux/mm.h | 56 ++++++++++++++++++++++++++++++++++++++++- tools/testing/vma/include/dup.h | 29 ++++++++++++++++++++- 2 files changed, 83 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 4604cd011ca4..45c59474c853 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1612,6 +1612,44 @@ static inline bool vma_is_shared_maywrite(const struct vm_area_struct *vma) return is_shared_maywrite(&vma->flags); } +/** + * vma_flags_is_kernel_owned() - Do the specified VMA flags indicate that the + * contents of the VMA are owned by the kernel rather than the core mm? + * @flags: The VMA flags to test. + * + * A kernel-owned mapping is one whose contents are established and controlled + * by the kernel, typically a driver, rather than by the core mm's fault and + * rmap machinery. + * + * The mapping may be memory-mapped I/O, kernel-allocated pages or ordinary + * pages the owner has chosen to map itself (shmem via a PFN map, for instance). + * + * But in all cases core mm must not populate, reclaim, migrate, Copy-on-Write + * or merge it of its own accord. + * + * The pages mapped, if any, may or may not be reference counted or map counted. + * + * Returns: true if the flags indicate a kernel-owned mapping. + */ +static inline bool vma_flags_is_kernel_owned(const vma_flags_t *flags) +{ + return vma_flags_test_any(flags, VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT, + VMA_IO_BIT); +} + +/** + * vma_is_kernel_owned() - Are the contents of @vma owned by the kernel? + * @vma: The VMA to test. + * + * See vma_flags_is_kernel_owned() for a description of this property. + * + * Returns: true if the VMA is kernel-owned. + */ +static inline bool vma_is_kernel_owned(const struct vm_area_struct *vma) +{ + return vma_flags_is_kernel_owned(&vma->flags); +} + /** * vma_flags_can_merge() - Do the specified VMA flags permit the VMA to be * merged with another? @@ -1620,7 +1658,23 @@ static inline bool vma_is_shared_maywrite(const struct vm_area_struct *vma) */ static inline bool vma_flags_can_merge(const vma_flags_t *flags) { - return !vma_flags_test_any_mask(flags, VMA_SPECIAL_FLAGS); + /* + * VMA merging assumes that a VMA's flags and fields completely describe + * its state. + * + * However, kernel-owned mappings may have established state upon mapping + * not embodied in any attribute of the VMA. + * + * Additionally, private (CoW) PFN maps encode the source PFN of the + * range in vma->vm_pgoff, which may otherwise cause spurious merges. + */ + if (vma_flags_is_kernel_owned(flags)) + return false; + /* VMA explicitly marked as being unmergeable. */ + if (vma_flags_test(flags, VMA_DONTEXPAND_BIT)) + return false; + + return true; } /** diff --git a/tools/testing/vma/include/dup.h b/tools/testing/vma/include/dup.h index 52eee05e6c32..3fe40e0f4034 100644 --- a/tools/testing/vma/include/dup.h +++ b/tools/testing/vma/include/dup.h @@ -1659,7 +1659,34 @@ static inline bool file_is_dev_zero(const struct file *file) return file && file->f_op == &zero_fops; } +static inline bool vma_flags_is_kernel_owned(const vma_flags_t *flags) +{ + return vma_flags_test_any(flags, VMA_PFNMAP_BIT, VMA_MIXEDMAP_BIT, + VMA_IO_BIT); +} + +static inline bool vma_is_kernel_owned(const struct vm_area_struct *vma) +{ + return vma_flags_is_kernel_owned(&vma->flags); +} + static inline bool vma_flags_can_merge(const vma_flags_t *flags) { - return !vma_flags_test_any_mask(flags, VMA_SPECIAL_FLAGS); + /* + * VMA merging assumes that the properties of a VMA completely describe + * the properties of that VMA. + * + * However, kernel-owned mappings may have established state upon mapping + * not embodied in any attribute of the VMA. + * + * Additionally, PFN maps encode the source PFN of the range in + * vma->vm_pgoff, which may otherwise cause spurious merges. + */ + if (vma_flags_is_kernel_owned(flags)) + return false; + /* VMA explicitly marked as being unmergeable. */ + if (vma_flags_test(flags, VMA_DONTEXPAND_BIT)) + return false; + + return true; } -- 2.55.0