The tmpfs.rst doc references the has_transparent_hugepage() helper, which is an implementation detail in the kernel and not relevant for users wishing to properly configure THP support for tmpfs. Remove it. Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Baolin Wang Reviewed-by: Lance Yang Reviewed-by: Zi Yan Reviewed-by: Dev Jain Signed-off-by: Luiz Capitulino --- Documentation/filesystems/tmpfs.rst | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/Documentation/filesystems/tmpfs.rst b/Documentation/filesystems/tmpfs.rst index d677e0428c3f..46fc986c3388 100644 --- a/Documentation/filesystems/tmpfs.rst +++ b/Documentation/filesystems/tmpfs.rst @@ -109,9 +109,8 @@ noswap Disables swap. Remounts must respect the original settings. ====== =========================================================== tmpfs also supports Transparent Huge Pages which requires a kernel -configured with CONFIG_TRANSPARENT_HUGEPAGE and with huge supported for -your system (has_transparent_hugepage(), which is architecture specific). -The mount options for this are: +configured with CONFIG_TRANSPARENT_HUGEPAGE and with huge pages +supported for your system. The mount options for this are: ================ ============================================================== huge=never Do not allocate huge pages. This is the default. -- 2.55.0 Today, shmem_getattr() sets stat->blksize to PMD size whenever shmem_huge_global_enabled() returns non-zero. While this works fine for the normal THP-enabled case as explained by Baolin in [1], this has two problems: 1. Theoretically, when shmem is configured for within_size, this could set blksize to PMD size even though the allocation may be a smaller mTHP order 2. A future commit will allow shmem THP support to be enabled even when the CPU doesn't support PMD-sized pages. We should not allow blksize to be set to PMD size in this case In order to fix #1 and prepare for #2, this commit sets blksize to the size of the highest supported order returned by shmem_huge_global_enabled(). [1] https://lore.kernel.org/linux-mm/6591a74c-7ef9-4614-9ae9-cb2fbed86ebf@linux.alibaba.com/ Suggested-by: Baolin Wang Acked-by: Zi Yan Signed-off-by: Luiz Capitulino --- mm/shmem.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 5071177059a9..19534e21b61f 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1287,6 +1287,7 @@ static int shmem_getattr(struct mnt_idmap *idmap, { struct inode *inode = path->dentry->d_inode; struct shmem_inode_info *info = SHMEM_I(inode); + unsigned int orders; /* Fast-path hint; recalc under info->lock corrects any stale read. */ if (data_race(info->alloced - info->swapped != inode->i_mapping->nrpages)) @@ -1303,8 +1304,9 @@ static int shmem_getattr(struct mnt_idmap *idmap, STATX_ATTR_NODUMP); generic_fillattr(idmap, request_mask, inode, stat); - if (shmem_huge_global_enabled(inode, 0, 0, false, NULL, 0)) - stat->blksize = HPAGE_PMD_SIZE; + orders = shmem_huge_global_enabled(inode, 0, 0, false, NULL, 0); + if (orders) + stat->blksize = PAGE_SIZE << highest_order(orders); if (request_mask & STATX_BTIME) { stat->result_mask |= STATX_BTIME; -- 2.55.0 Currently, we have two helpers that check for PMD-sized pages but have different names and slightly different semantics: - has_transparent_hugepage(): the name suggests it checks if THP is enabled, but when CONFIG_TRANSPARENT_HUGEPAGE=y and the architecture implements this helper, it actually checks if the CPU supports PMD-sized pages - thp_disabled_by_hw(): the name suggests it checks if THP is disabled by the hardware, but it just returns a cached value acquired with has_transparent_hugepage(). This helper is used in fast paths This commit introduces a new helper called pgtable_has_pmd_leaves() which is intended to replace both has_transparent_hugepage() and thp_disabled_by_hw(). pgtable_has_pmd_leaves() has very clear semantics: it returns true if the CPU supports PMD-sized pages and false otherwise. It always returns a cached value, so it can be used in fast paths. The new helper requires an initialization step which is performed by pgtable_leaf_support_init(). We call pgtable_leaf_support_init() early during boot from mm_core_init(). The next commits will convert users of both has_transparent_hugepage() and thp_disabled_by_hw() to pgtable_has_pmd_leaves(). Signed-off-by: Luiz Capitulino --- include/linux/pgtable.h | 16 ++++++++++++++++ mm/memory.c | 9 +++++++++ mm/mm_init.c | 1 + 3 files changed, 26 insertions(+) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 8c093c119e5a..4c00cf53e183 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -14,6 +14,7 @@ #include #include #include +#include #include #include @@ -2313,6 +2314,21 @@ static inline const char *pgtable_level_to_str(enum pgtable_level level) } } +#ifdef CONFIG_MMU +DECLARE_STATIC_KEY_TRUE(__arch_has_pmd_leaves_key); +static inline bool pgtable_has_pmd_leaves(void) +{ + return static_branch_likely(&__arch_has_pmd_leaves_key); +} +void __init pgtable_leaf_support_init(void); +#else +static inline bool pgtable_has_pmd_leaves(void) +{ + return false; +} +static inline void __init pgtable_leaf_support_init(void) { } +#endif + #endif /* !__ASSEMBLER__ */ #if !defined(MAX_POSSIBLE_PHYSMEM_BITS) && !defined(CONFIG_64BIT) diff --git a/mm/memory.c b/mm/memory.c index 56b244552f13..d6540d6d4ef9 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -164,6 +164,15 @@ __setup("norandmaps", disable_randmaps); unsigned long highest_memmap_pfn __read_mostly; +DEFINE_STATIC_KEY_TRUE(__arch_has_pmd_leaves_key); +EXPORT_SYMBOL(__arch_has_pmd_leaves_key); + +void __init pgtable_leaf_support_init(void) +{ + if (!has_transparent_hugepage()) + static_branch_disable(&__arch_has_pmd_leaves_key); +} + void mm_trace_rss_stat(struct mm_struct *mm, int member) { trace_rss_stat(mm, member); diff --git a/mm/mm_init.c b/mm/mm_init.c index 711f821f7b3c..dcc072db4558 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -2699,6 +2699,7 @@ void __init mm_core_init(void) { arch_mm_preinit(); init_zero_page_pfn(); + pgtable_leaf_support_init(); /* Initializations relying on SMP setup */ BUILD_BUG_ON(MAX_ZONELISTS > 2); -- 2.55.0 dax_align_valid() uses has_transparent_hugepage() to check if PMD-sized pages are supported, use pgtable_has_pmd_leaves() instead. Acked-by: David Hildenbrand (Red Hat) Signed-off-by: Luiz Capitulino --- drivers/dax/dax-private.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/dax/dax-private.h b/drivers/dax/dax-private.h index 81e4af49e39c..35744ff6592a 100644 --- a/drivers/dax/dax-private.h +++ b/drivers/dax/dax-private.h @@ -123,7 +123,7 @@ static inline bool dax_align_valid(unsigned long align) { if (align == PUD_SIZE && IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD)) return true; - if (align == PMD_SIZE && has_transparent_hugepage()) + if (align == PMD_SIZE && pgtable_has_pmd_leaves()) return true; if (align == PAGE_SIZE) return true; -- 2.55.0 nd_pfn_supported_alignments() and nd_pfn_default_alignment() use has_transparent_hugepage() to check if THP is supported with PMD-sized pages. Use pgtable_has_pmd_leaves() instead. Also, check for IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) to preserve the current implementation semantics. Acked-by: Zi Yan Signed-off-by: Luiz Capitulino --- drivers/nvdimm/pfn_devs.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/nvdimm/pfn_devs.c b/drivers/nvdimm/pfn_devs.c index 8fa9c16aba7e..457eb54e7ab6 100644 --- a/drivers/nvdimm/pfn_devs.c +++ b/drivers/nvdimm/pfn_devs.c @@ -94,7 +94,8 @@ static unsigned long *nd_pfn_supported_alignments(unsigned long *alignments) alignments[0] = PAGE_SIZE; - if (has_transparent_hugepage()) { + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && + pgtable_has_pmd_leaves()) { alignments[1] = HPAGE_PMD_SIZE; if (has_transparent_pud_hugepage()) alignments[2] = HPAGE_PUD_SIZE; @@ -109,7 +110,8 @@ static unsigned long *nd_pfn_supported_alignments(unsigned long *alignments) static unsigned long nd_pfn_default_alignment(void) { - if (has_transparent_hugepage()) + if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && + pgtable_has_pmd_leaves()) return HPAGE_PMD_SIZE; return PAGE_SIZE; } -- 2.55.0 debug_vm_pgtable calls has_transparent_hugepage() in multiple places to check if PMD-sized pages are supported, use pgtable_has_pmd_leaves() instead. Reviewed-by: Lance Yang Reviewed-by: Zi Yan Signed-off-by: Luiz Capitulino --- mm/debug_vm_pgtable.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index 2875fd22d7bb..25038b74c7bc 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -177,7 +177,7 @@ static void __init pmd_basic_tests(struct pgtable_debug_args *args, int idx) unsigned long val = idx, *ptr = &val; pmd_t pmd; - if (!has_transparent_hugepage()) + if (!pgtable_has_pmd_leaves()) return; pr_debug("Validating PMD basic (%pGv)\n", ptr); @@ -222,7 +222,7 @@ static void __init pmd_advanced_tests(struct pgtable_debug_args *args) pmd_t pmd; unsigned long vaddr = args->vaddr; - if (!has_transparent_hugepage()) + if (!pgtable_has_pmd_leaves()) return; page = (args->pmd_pfn != ULONG_MAX) ? pfn_to_page(args->pmd_pfn) : NULL; @@ -283,7 +283,7 @@ static void __init pmd_leaf_tests(struct pgtable_debug_args *args) { pmd_t pmd; - if (!has_transparent_hugepage()) + if (!pgtable_has_pmd_leaves()) return; pr_debug("Validating PMD leaf\n"); @@ -688,7 +688,7 @@ static void __init pmd_protnone_tests(struct pgtable_debug_args *args) if (!IS_ENABLED(CONFIG_ARCH_HAS_PTE_PROTNONE)) return; - if (!has_transparent_hugepage()) + if (!pgtable_has_pmd_leaves()) return; pr_debug("Validating PMD protnone\n"); @@ -737,7 +737,7 @@ static void __init pmd_soft_dirty_tests(struct pgtable_debug_args *args) if (!pgtable_supports_soft_dirty()) return; - if (!has_transparent_hugepage()) + if (!pgtable_has_pmd_leaves()) return; pr_debug("Validating PMD soft dirty\n"); @@ -754,7 +754,7 @@ static void __init pmd_leaf_soft_dirty_tests(struct pgtable_debug_args *args) !IS_ENABLED(CONFIG_ARCH_HAS_PMD_SOFTLEAVES)) return; - if (!has_transparent_hugepage()) + if (!pgtable_has_pmd_leaves()) return; pr_debug("Validating PMD swap soft dirty\n"); @@ -825,7 +825,7 @@ static void __init pmd_softleaf_tests(struct pgtable_debug_args *args) swp_entry_t arch_entry; pmd_t pmd1, pmd2; - if (!has_transparent_hugepage()) + if (!pgtable_has_pmd_leaves()) return; pr_debug("Validating PMD swap\n"); @@ -906,7 +906,7 @@ static void __init pmd_thp_tests(struct pgtable_debug_args *args) { pmd_t pmd; - if (!has_transparent_hugepage()) + if (!pgtable_has_pmd_leaves()) return; pr_debug("Validating PMD based THP\n"); @@ -997,7 +997,7 @@ static void __init destroy_args(struct pgtable_debug_args *args) } if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && - has_transparent_hugepage() && + pgtable_has_pmd_leaves() && args->pmd_pfn != ULONG_MAX) { debug_vm_pgtable_free_huge_page(args, args->pmd_pfn, HPAGE_PMD_ORDER); args->pmd_pfn = ULONG_MAX; @@ -1249,7 +1249,7 @@ static int __init init_args(struct pgtable_debug_args *args) } if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && - has_transparent_hugepage()) { + pgtable_has_pmd_leaves()) { page = debug_vm_pgtable_alloc_huge_page(args, HPAGE_PMD_ORDER); if (page) { args->pmd_pfn = page_to_pfn(page); -- 2.55.0 In order to enable THP support in shmem today, besides the user configuration required, the CPU must support PMD-sized pages. This is the case because of the following has_transparent_hugepage() usage: - shmem_parse_one() and shmem_parse_huge(): Check if THP is built-in and if the CPU supports PMD-sized pages - shmem_init(): Since the CONFIG_TRANSPARENT_HUGEPAGE guard is outside the code block calling has_transparent_hugepage(), the has_transparent_hugepage() call is exclusively checking if the CPU supports PMD-sized pages While it's necessary to check if CONFIG_TRANSPARENT_HUGEPAGE is enabled in all cases, shmem can determine THP size support at folio allocation time. Therefore, drop the has_transparent_hugepage() usage listed above while keeping the CONFIG_TRANSPARENT_HUGEPAGE checks. Additionally, we need to check if PMD size order is supported in shmem_getattr(). Use pgtable_has_pmd_leaves() for that. Reviewed-by: Baolin Wang Signed-off-by: Luiz Capitulino --- mm/shmem.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index 19534e21b61f..68ae606b5c4c 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -689,7 +689,7 @@ static int shmem_parse_huge(const char *str) else return -EINVAL; - if (!has_transparent_hugepage() && + if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && huge != SHMEM_HUGE_NEVER && huge != SHMEM_HUGE_DENY) return -EINVAL; @@ -1305,6 +1305,8 @@ static int shmem_getattr(struct mnt_idmap *idmap, generic_fillattr(idmap, request_mask, inode, stat); orders = shmem_huge_global_enabled(inode, 0, 0, false, NULL, 0); + if (!pgtable_has_pmd_leaves()) + orders &= ~BIT(PMD_ORDER); if (orders) stat->blksize = PAGE_SIZE << highest_order(orders); @@ -4589,8 +4591,7 @@ static int shmem_parse_one(struct fs_context *fc, struct fs_parameter *param) case Opt_huge: ctx->huge = result.uint_32; if (ctx->huge != SHMEM_HUGE_NEVER && - !(IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && - has_transparent_hugepage())) + !IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) goto unsupported_parameter; ctx->seen |= SHMEM_SEEN_HUGE; break; @@ -5385,7 +5386,7 @@ void __init shmem_init(void) #endif #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (has_transparent_hugepage() && shmem_huge > SHMEM_HUGE_DENY) + if (shmem_huge > SHMEM_HUGE_DENY) SHMEM_SB(shm_mnt->mnt_sb)->huge = shmem_huge; else shmem_huge = SHMEM_HUGE_NEVER; /* just in case it was patched */ -- 2.55.0 has_transparent_hugepage() definition is guarded by CONFIG_TRANSPARENT_HUGEPAGE, but there's a need to use it when CONFIG_TRANSPARENT_HUGEPAGE=n. Move the definition out of the THP guard. Signed-off-by: Luiz Capitulino --- arch/s390/include/asm/pgtable.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h index e882663a58e7..5aa8f621df04 100644 --- a/arch/s390/include/asm/pgtable.h +++ b/arch/s390/include/asm/pgtable.h @@ -1825,13 +1825,13 @@ static inline int pmd_trans_huge(pmd_t pmd) { return pmd_leaf(pmd); } +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ #define has_transparent_hugepage has_transparent_hugepage static inline int has_transparent_hugepage(void) { return cpu_has_edat1() ? 1 : 0; } -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ #ifdef CONFIG_PAGE_TABLE_CHECK static inline bool pte_user_accessible_page(struct mm_struct *mm, unsigned long addr, pte_t pte) -- 2.55.0 has_transparent_hugepage() definition is guarded by CONFIG_TRANSPARENT_HUGEPAGE, but there's a need to use it when CONFIG_TRANSPARENT_HUGEPAGE=n. Move the definition out of the THP guard. Signed-off-by: Luiz Capitulino --- arch/powerpc/include/asm/book3s/64/hash-4k.h | 2 +- arch/powerpc/include/asm/book3s/64/hash-64k.h | 2 +- arch/powerpc/include/asm/book3s/64/pgtable.h | 18 +++++++++--------- arch/powerpc/include/asm/book3s/64/radix.h | 14 +++++++------- arch/powerpc/mm/book3s64/hash_pgtable.c | 4 ++-- 5 files changed, 20 insertions(+), 20 deletions(-) diff --git a/arch/powerpc/include/asm/book3s/64/hash-4k.h b/arch/powerpc/include/asm/book3s/64/hash-4k.h index 8e5bd9902bed..79511e6abfca 100644 --- a/arch/powerpc/include/asm/book3s/64/hash-4k.h +++ b/arch/powerpc/include/asm/book3s/64/hash-4k.h @@ -165,9 +165,9 @@ extern void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp); -extern int hash__has_transparent_hugepage(void); #endif +extern int hash__has_transparent_hugepage(void); #endif /* !__ASSEMBLER__ */ #endif /* _ASM_POWERPC_BOOK3S_64_HASH_4K_H */ diff --git a/arch/powerpc/include/asm/book3s/64/hash-64k.h b/arch/powerpc/include/asm/book3s/64/hash-64k.h index 7deb3a66890b..a4a44a112ff9 100644 --- a/arch/powerpc/include/asm/book3s/64/hash-64k.h +++ b/arch/powerpc/include/asm/book3s/64/hash-64k.h @@ -278,9 +278,9 @@ extern void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp); extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp); -extern int hash__has_transparent_hugepage(void); #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +extern int hash__has_transparent_hugepage(void); #endif /* __ASSEMBLER__ */ #endif /* _ASM_POWERPC_BOOK3S_64_HASH_64K_H */ diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index f4db7d7fbd5c..2a795f6d9263 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -1121,15 +1121,6 @@ static inline void update_mmu_cache_pud(struct vm_area_struct *vma, { } -extern int hash__has_transparent_hugepage(void); -static inline int has_transparent_hugepage(void) -{ - if (radix_enabled()) - return radix__has_transparent_hugepage(); - return hash__has_transparent_hugepage(); -} -#define has_transparent_hugepage has_transparent_hugepage - static inline int has_transparent_pud_hugepage(void) { if (radix_enabled()) @@ -1441,6 +1432,15 @@ static inline bool arch_needs_pgtable_deposit(void) #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +extern int hash__has_transparent_hugepage(void); +static inline int has_transparent_hugepage(void) +{ + if (radix_enabled()) + return radix__has_transparent_hugepage(); + return hash__has_transparent_hugepage(); +} +#define has_transparent_hugepage has_transparent_hugepage + #define __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION pte_t ptep_modify_prot_start(struct vm_area_struct *, unsigned long, pte_t *); void ptep_modify_prot_commit(struct vm_area_struct *, unsigned long, diff --git a/arch/powerpc/include/asm/book3s/64/radix.h b/arch/powerpc/include/asm/book3s/64/radix.h index da954e779744..50545cf519bd 100644 --- a/arch/powerpc/include/asm/book3s/64/radix.h +++ b/arch/powerpc/include/asm/book3s/64/radix.h @@ -298,22 +298,22 @@ extern pmd_t radix__pmdp_huge_get_and_clear(struct mm_struct *mm, pud_t radix__pudp_huge_get_and_clear(struct mm_struct *mm, unsigned long addr, pud_t *pudp); -static inline int radix__has_transparent_hugepage(void) +static inline int radix__has_transparent_pud_hugepage(void) { - /* For radix 2M at PMD level means thp */ - if (mmu_psize_defs[MMU_PAGE_2M].shift == PMD_SHIFT) + /* For radix 1G at PUD level means pud hugepage support */ + if (mmu_psize_defs[MMU_PAGE_1G].shift == PUD_SHIFT) return 1; return 0; } +#endif -static inline int radix__has_transparent_pud_hugepage(void) +static inline int radix__has_transparent_hugepage(void) { - /* For radix 1G at PUD level means pud hugepage support */ - if (mmu_psize_defs[MMU_PAGE_1G].shift == PUD_SHIFT) + /* For radix 2M at PMD level means thp */ + if (mmu_psize_defs[MMU_PAGE_2M].shift == PMD_SHIFT) return 1; return 0; } -#endif struct vmem_altmap; struct dev_pagemap; diff --git a/arch/powerpc/mm/book3s64/hash_pgtable.c b/arch/powerpc/mm/book3s64/hash_pgtable.c index d9b5b751d7b7..50316f3fc5d3 100644 --- a/arch/powerpc/mm/book3s64/hash_pgtable.c +++ b/arch/powerpc/mm/book3s64/hash_pgtable.c @@ -391,6 +391,8 @@ pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm, return old_pmd; } +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ + int hash__has_transparent_hugepage(void) { @@ -422,8 +424,6 @@ int hash__has_transparent_hugepage(void) } EXPORT_SYMBOL_GPL(hash__has_transparent_hugepage); -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ - #ifdef CONFIG_STRICT_KERNEL_RWX struct change_memory_parms { -- 2.55.0 has_transparent_hugepage() definition is guarded by CONFIG_TRANSPARENT_HUGEPAGE, but there's a need to use it when CONFIG_TRANSPARENT_HUGEPAGE=n. Move the definition out of the THP guard. Signed-off-by: Luiz Capitulino --- arch/mips/include/asm/pgtable.h | 6 +++--- arch/mips/mm/tlb-r4k.c | 4 ---- 2 files changed, 3 insertions(+), 7 deletions(-) diff --git a/arch/mips/include/asm/pgtable.h b/arch/mips/include/asm/pgtable.h index fa7b935f947c..b038da872ec6 100644 --- a/arch/mips/include/asm/pgtable.h +++ b/arch/mips/include/asm/pgtable.h @@ -615,9 +615,6 @@ unsigned long io_remap_pfn_range_pfn(unsigned long pfn, unsigned long size); /* We don't have hardware dirty/accessed bits, generic_pmdp_establish is fine.*/ #define pmdp_establish generic_pmdp_establish -#define has_transparent_hugepage has_transparent_hugepage -extern int has_transparent_hugepage(void); - static inline int pmd_trans_huge(pmd_t pmd) { return !!(pmd_val(pmd) & _PAGE_HUGE); @@ -743,6 +740,9 @@ static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm, #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#define has_transparent_hugepage has_transparent_hugepage +extern int has_transparent_hugepage(void); + #ifdef _PAGE_HUGE #define pmd_leaf(pmd) ((pmd_val(pmd) & _PAGE_HUGE) != 0) #define pud_leaf(pud) ((pud_val(pud) & _PAGE_HUGE) != 0) diff --git a/arch/mips/mm/tlb-r4k.c b/arch/mips/mm/tlb-r4k.c index 24fe85fa169d..f4e369342a56 100644 --- a/arch/mips/mm/tlb-r4k.c +++ b/arch/mips/mm/tlb-r4k.c @@ -432,8 +432,6 @@ void add_wired_entry(unsigned long entrylo0, unsigned long entrylo1, #endif } -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - int has_transparent_hugepage(void) { static unsigned int mask = -1; @@ -452,8 +450,6 @@ int has_transparent_hugepage(void) } EXPORT_SYMBOL(has_transparent_hugepage); -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ - /* * Used for loading TLB entries before trap_init() has started, when we * don't actually want to add a wired entry which remains throughout the -- 2.55.0 has_transparent_hugepage() definition is guarded by CONFIG_TRANSPARENT_HUGEPAGE, but there's a need to use it when CONFIG_TRANSPARENT_HUGEPAGE=n. Move the definition out of the THP guard. Signed-off-by: Luiz Capitulino --- arch/x86/include/asm/pgtable.h | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 8e0018fadd14..3c62b33147fa 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -306,12 +306,6 @@ static inline int pud_trans_huge(pud_t pud) } #endif -#define has_transparent_hugepage has_transparent_hugepage -static inline int has_transparent_hugepage(void) -{ - return boot_cpu_has(X86_FEATURE_PSE); -} - #ifdef CONFIG_ARCH_SUPPORTS_PMD_PFNMAP static inline bool pmd_special(pmd_t pmd) { @@ -337,6 +331,12 @@ static inline pud_t pud_mkspecial(pud_t pud) #endif /* CONFIG_ARCH_SUPPORTS_PUD_PFNMAP */ #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#define has_transparent_hugepage has_transparent_hugepage +static inline int has_transparent_hugepage(void) +{ + return boot_cpu_has(X86_FEATURE_PSE); +} + static inline pte_t pte_set_flags(pte_t pte, pteval_t set) { pteval_t v = native_pte_val(pte); -- 2.55.0 Now that all the has_transparent_hugepage() callers have been converted to pgtable_has_pmd_leaves(), this commit does two things: 1. Rename has_transparent_hugepage() arch implementations to arch_has_pmd_leaves(), since that's what the helper checks for 2. Introduce the default implementation of arch_has_pmd_leaves() as IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE). This means that if the arch doesn't implement arch_has_pmd_leaves() we default to checking CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE as a way to determine if PMD-sized pages are supported Note that arch_has_pmd_leaves() is supposed to be called only by init_arch_has_pmd_leaves(). The remaining exception is hugepage_init() which will be converted in a future commit. Signed-off-by: Luiz Capitulino --- arch/mips/include/asm/pgtable.h | 4 ++-- arch/mips/mm/tlb-r4k.c | 4 ++-- arch/powerpc/include/asm/book3s/64/hash-4k.h | 2 +- arch/powerpc/include/asm/book3s/64/hash-64k.h | 2 +- arch/powerpc/include/asm/book3s/64/pgtable.h | 10 +++++----- arch/powerpc/include/asm/book3s/64/radix.h | 2 +- arch/powerpc/mm/book3s64/hash_pgtable.c | 4 ++-- arch/s390/include/asm/pgtable.h | 4 ++-- arch/x86/include/asm/pgtable.h | 4 ++-- include/linux/pgtable.h | 4 ++-- mm/huge_memory.c | 2 +- mm/memory.c | 2 +- 12 files changed, 22 insertions(+), 22 deletions(-) diff --git a/arch/mips/include/asm/pgtable.h b/arch/mips/include/asm/pgtable.h index b038da872ec6..2a0a7ddf8bee 100644 --- a/arch/mips/include/asm/pgtable.h +++ b/arch/mips/include/asm/pgtable.h @@ -740,8 +740,8 @@ static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm, #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ -#define has_transparent_hugepage has_transparent_hugepage -extern int has_transparent_hugepage(void); +#define arch_has_pmd_leaves arch_has_pmd_leaves +extern int arch_has_pmd_leaves(void); #ifdef _PAGE_HUGE #define pmd_leaf(pmd) ((pmd_val(pmd) & _PAGE_HUGE) != 0) diff --git a/arch/mips/mm/tlb-r4k.c b/arch/mips/mm/tlb-r4k.c index f4e369342a56..9339ebb0815f 100644 --- a/arch/mips/mm/tlb-r4k.c +++ b/arch/mips/mm/tlb-r4k.c @@ -432,7 +432,7 @@ void add_wired_entry(unsigned long entrylo0, unsigned long entrylo1, #endif } -int has_transparent_hugepage(void) +int arch_has_pmd_leaves(void) { static unsigned int mask = -1; @@ -448,7 +448,7 @@ int has_transparent_hugepage(void) } return mask == PM_HUGE_MASK; } -EXPORT_SYMBOL(has_transparent_hugepage); +EXPORT_SYMBOL(arch_has_pmd_leaves); /* * Used for loading TLB entries before trap_init() has started, when we diff --git a/arch/powerpc/include/asm/book3s/64/hash-4k.h b/arch/powerpc/include/asm/book3s/64/hash-4k.h index 79511e6abfca..d532e3f0dfc3 100644 --- a/arch/powerpc/include/asm/book3s/64/hash-4k.h +++ b/arch/powerpc/include/asm/book3s/64/hash-4k.h @@ -167,7 +167,7 @@ extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp); #endif -extern int hash__has_transparent_hugepage(void); +extern int hash__arch_has_pmd_leaves(void); #endif /* !__ASSEMBLER__ */ #endif /* _ASM_POWERPC_BOOK3S_64_HASH_4K_H */ diff --git a/arch/powerpc/include/asm/book3s/64/hash-64k.h b/arch/powerpc/include/asm/book3s/64/hash-64k.h index a4a44a112ff9..d523c80f44f7 100644 --- a/arch/powerpc/include/asm/book3s/64/hash-64k.h +++ b/arch/powerpc/include/asm/book3s/64/hash-64k.h @@ -280,7 +280,7 @@ extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp); #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ -extern int hash__has_transparent_hugepage(void); +extern int hash__arch_has_pmd_leaves(void); #endif /* __ASSEMBLER__ */ #endif /* _ASM_POWERPC_BOOK3S_64_HASH_64K_H */ diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index 2a795f6d9263..41166d44fa75 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -1432,14 +1432,14 @@ static inline bool arch_needs_pgtable_deposit(void) #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ -extern int hash__has_transparent_hugepage(void); -static inline int has_transparent_hugepage(void) +extern int hash__arch_has_pmd_leaves(void); +static inline int arch_has_pmd_leaves(void) { if (radix_enabled()) - return radix__has_transparent_hugepage(); - return hash__has_transparent_hugepage(); + return radix__arch_has_pmd_leaves(); + return hash__arch_has_pmd_leaves(); } -#define has_transparent_hugepage has_transparent_hugepage +#define arch_has_pmd_leaves arch_has_pmd_leaves #define __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION pte_t ptep_modify_prot_start(struct vm_area_struct *, unsigned long, pte_t *); diff --git a/arch/powerpc/include/asm/book3s/64/radix.h b/arch/powerpc/include/asm/book3s/64/radix.h index 50545cf519bd..e8b74528953f 100644 --- a/arch/powerpc/include/asm/book3s/64/radix.h +++ b/arch/powerpc/include/asm/book3s/64/radix.h @@ -307,7 +307,7 @@ static inline int radix__has_transparent_pud_hugepage(void) } #endif -static inline int radix__has_transparent_hugepage(void) +static inline int radix__arch_has_pmd_leaves(void) { /* For radix 2M at PMD level means thp */ if (mmu_psize_defs[MMU_PAGE_2M].shift == PMD_SHIFT) diff --git a/arch/powerpc/mm/book3s64/hash_pgtable.c b/arch/powerpc/mm/book3s64/hash_pgtable.c index 50316f3fc5d3..a67bc1723404 100644 --- a/arch/powerpc/mm/book3s64/hash_pgtable.c +++ b/arch/powerpc/mm/book3s64/hash_pgtable.c @@ -393,7 +393,7 @@ pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm, #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ -int hash__has_transparent_hugepage(void) +int hash__arch_has_pmd_leaves(void) { if (!mmu_has_feature(MMU_FTR_16M_PAGE)) @@ -422,7 +422,7 @@ int hash__has_transparent_hugepage(void) return 1; } -EXPORT_SYMBOL_GPL(hash__has_transparent_hugepage); +EXPORT_SYMBOL_GPL(hash__arch_has_pmd_leaves); #ifdef CONFIG_STRICT_KERNEL_RWX diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h index 5aa8f621df04..24ee90c7e9bf 100644 --- a/arch/s390/include/asm/pgtable.h +++ b/arch/s390/include/asm/pgtable.h @@ -1827,8 +1827,8 @@ static inline int pmd_trans_huge(pmd_t pmd) } #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ -#define has_transparent_hugepage has_transparent_hugepage -static inline int has_transparent_hugepage(void) +#define arch_has_pmd_leaves arch_has_pmd_leaves +static inline int arch_has_pmd_leaves(void) { return cpu_has_edat1() ? 1 : 0; } diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 3c62b33147fa..3c6a641e6c21 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -331,8 +331,8 @@ static inline pud_t pud_mkspecial(pud_t pud) #endif /* CONFIG_ARCH_SUPPORTS_PUD_PFNMAP */ #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ -#define has_transparent_hugepage has_transparent_hugepage -static inline int has_transparent_hugepage(void) +#define arch_has_pmd_leaves arch_has_pmd_leaves +static inline int arch_has_pmd_leaves(void) { return boot_cpu_has(X86_FEATURE_PSE); } diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 4c00cf53e183..d571d161a84b 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -2344,8 +2344,8 @@ static inline void __init pgtable_leaf_support_init(void) { } #endif #endif -#ifndef has_transparent_hugepage -#define has_transparent_hugepage() IS_BUILTIN(CONFIG_TRANSPARENT_HUGEPAGE) +#ifndef arch_has_pmd_leaves +#define arch_has_pmd_leaves() IS_BUILTIN(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE) #endif #ifndef has_transparent_pud_hugepage diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 04e8a6b55343..3f3a929e3069 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1010,7 +1010,7 @@ static int __init hugepage_init(void) int err; struct kobject *hugepage_kobj; - if (!has_transparent_hugepage()) { + if (!arch_has_pmd_leaves()) { transparent_hugepage_flags = 1 << TRANSPARENT_HUGEPAGE_UNSUPPORTED; return -EINVAL; } diff --git a/mm/memory.c b/mm/memory.c index d6540d6d4ef9..b0d3a8a8a25d 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -169,7 +169,7 @@ EXPORT_SYMBOL(__arch_has_pmd_leaves_key); void __init pgtable_leaf_support_init(void) { - if (!has_transparent_hugepage()) + if (!arch_has_pmd_leaves()) static_branch_disable(&__arch_has_pmd_leaves_key); } -- 2.55.0 Despite its name, thp_disabled_by_hw() just checks whether the architecture supports PMD-sized pages. It returns true when TRANSPARENT_HUGEPAGE_UNSUPPORTED is set in transparent_hugepage_flags, this only occurs if the architecture implements arch_has_pmd_leaves() and that function returns false. Since pgtable_has_pmd_leaves() provides the same semantics, use it instead. Reviewed-by: Lance Yang Reviewed-by: Baolin Wang Acked-by: Zi Yan Acked-by: David Hildenbrand (Arm) Signed-off-by: Luiz Capitulino --- include/linux/huge_mm.h | 7 ------- mm/huge_memory.c | 6 ++---- mm/memory.c | 2 +- mm/shmem.c | 2 +- 4 files changed, 4 insertions(+), 13 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index c745f7ad2298..525acea2eb59 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -47,7 +47,6 @@ vm_fault_t vmf_insert_folio_pud(struct vm_fault *vmf, struct folio *folio, bool write); enum transparent_hugepage_flag { - TRANSPARENT_HUGEPAGE_UNSUPPORTED, TRANSPARENT_HUGEPAGE_FLAG, TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG, TRANSPARENT_HUGEPAGE_DEFRAG_DIRECT_FLAG, @@ -383,12 +382,6 @@ static inline bool vma_thp_disabled(struct vm_area_struct *vma, return mm_flags_test(MMF_DISABLE_THP_EXCEPT_ADVISED, vma->vm_mm); } -static inline bool thp_disabled_by_hw(void) -{ - /* If the hardware/firmware marked hugepage support disabled. */ - return transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_UNSUPPORTED); -} - unsigned long thp_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags); unsigned long thp_get_unmapped_area_vmaflags(struct file *filp, unsigned long addr, diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 3f3a929e3069..78dd16513a11 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -136,7 +136,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma, if (!vma->vm_mm) /* vdso */ return 0; - if (thp_disabled_by_hw() || vma_thp_disabled(vma, vm_flags, forced_collapse)) + if (!pgtable_has_pmd_leaves() || vma_thp_disabled(vma, vm_flags, forced_collapse)) return 0; /* khugepaged doesn't collapse DAX vma, but page fault is fine. */ @@ -1010,10 +1010,8 @@ static int __init hugepage_init(void) int err; struct kobject *hugepage_kobj; - if (!arch_has_pmd_leaves()) { - transparent_hugepage_flags = 1 << TRANSPARENT_HUGEPAGE_UNSUPPORTED; + if (!pgtable_has_pmd_leaves()) return -EINVAL; - } /* * hugepages can't be allocated by the buddy allocator diff --git a/mm/memory.c b/mm/memory.c index b0d3a8a8a25d..f39d9383494e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5576,7 +5576,7 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct folio *folio, struct page *pa * PMD mappings if THPs are disabled. As we already have a THP, * behave as if we are forcing a collapse. */ - if (thp_disabled_by_hw() || vma_thp_disabled(vma, vma->vm_flags, + if (!pgtable_has_pmd_leaves() || vma_thp_disabled(vma, vma->vm_flags, /* forced_collapse=*/ true)) return ret; diff --git a/mm/shmem.c b/mm/shmem.c index 68ae606b5c4c..4d2e1b91ec26 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1832,7 +1832,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode, vm_flags_t vm_flags = vma ? vma->vm_flags : 0; unsigned int global_orders; - if (thp_disabled_by_hw() || (vma && vma_thp_disabled(vma, vm_flags, shmem_huge_force))) + if (!pgtable_has_pmd_leaves() || (vma && vma_thp_disabled(vma, vm_flags, shmem_huge_force))) return 0; global_orders = shmem_huge_global_enabled(inode, index, write_end, -- 2.55.0 If PMD-sized pages are not supported on an architecture (ie. the arch implements arch_has_pmd_leaves() and it returns false) then the current code disables all THP, including mTHP. This commit fixes this by allowing mTHP to be always enabled for all archs. When PMD-sized pages are not supported, its sysfs entry won't be created and their mapping will be disallowed at page-fault time. Similarly, this commit implements the following changes for shmem in shmem_allowable_huge_orders(): - Drop the pgtable_has_pmd_leaves() check so that mTHP sizes are considered - Filter out PMD and PUD orders from allowable orders when PMD-sized pages are not supported by the CPU Reviewed-by: Baolin Wang Signed-off-by: Luiz Capitulino --- mm/huge_memory.c | 25 ++++++++++++++++++++----- mm/shmem.c | 14 +++++++++----- 2 files changed, 29 insertions(+), 10 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 78dd16513a11..385a305e7af6 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -129,6 +129,15 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma, else supported_orders = THP_ORDERS_ALL_FILE_DEFAULT; + if (!pgtable_has_pmd_leaves()) { + /* + * If the CPU does not support PMD leaves, assume for + * now that it does not support PUD leaves and disable + * both folio orders. + */ + supported_orders &= ~(BIT(PMD_ORDER) | BIT(PUD_ORDER)); + } + orders &= supported_orders; if (!orders) return 0; @@ -136,7 +145,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma, if (!vma->vm_mm) /* vdso */ return 0; - if (!pgtable_has_pmd_leaves() || vma_thp_disabled(vma, vm_flags, forced_collapse)) + if (vma_thp_disabled(vma, vm_flags, forced_collapse)) return 0; /* khugepaged doesn't collapse DAX vma, but page fault is fine. */ @@ -874,7 +883,7 @@ static int __init hugepage_init_sysfs(struct kobject **hugepage_kobj) * disable all other sizes. powerpc's PMD_ORDER isn't a compile-time * constant so we have to do this here. */ - if (!anon_orders_configured) + if (!anon_orders_configured && pgtable_has_pmd_leaves()) huge_anon_orders_inherit = BIT(PMD_ORDER); *hugepage_kobj = kobject_create_and_add("transparent_hugepage", mm_kobj); @@ -896,6 +905,15 @@ static int __init hugepage_init_sysfs(struct kobject **hugepage_kobj) } orders = THP_ORDERS_ALL_ANON | THP_ORDERS_ALL_FILE_DEFAULT; + if (!pgtable_has_pmd_leaves()) { + /* + * If the CPU does not support PMD leaves, assume for + * now that it does not support PUD leaves and disable + * both folio orders. + */ + orders &= ~(BIT(PMD_ORDER) | BIT(PUD_ORDER)); + } + order = highest_order(orders); while (orders) { thpsize = thpsize_create(order, *hugepage_kobj); @@ -1010,9 +1028,6 @@ static int __init hugepage_init(void) int err; struct kobject *hugepage_kobj; - if (!pgtable_has_pmd_leaves()) - return -EINVAL; - /* * hugepages can't be allocated by the buddy allocator */ diff --git a/mm/shmem.c b/mm/shmem.c index 4d2e1b91ec26..008df558af23 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1830,16 +1830,19 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode, unsigned long mask = READ_ONCE(huge_shmem_orders_always); unsigned long within_size_orders = READ_ONCE(huge_shmem_orders_within_size); vm_flags_t vm_flags = vma ? vma->vm_flags : 0; - unsigned int global_orders; + unsigned int global_orders, disabled_orders = 0; - if (!pgtable_has_pmd_leaves() || (vma && vma_thp_disabled(vma, vm_flags, shmem_huge_force))) + if (vma && vma_thp_disabled(vma, vm_flags, shmem_huge_force)) return 0; + if (!pgtable_has_pmd_leaves()) + disabled_orders = BIT(PMD_ORDER); + global_orders = shmem_huge_global_enabled(inode, index, write_end, shmem_huge_force, vma, vm_flags); /* Tmpfs huge pages allocation */ if (!vma || !vma_is_anon_shmem(vma)) - return global_orders; + return global_orders & ~disabled_orders; /* * Following the 'deny' semantics of the top level, force the huge @@ -1853,7 +1856,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode, * means non-PMD sized THP can not override 'huge' mount option now. */ if (shmem_huge == SHMEM_HUGE_FORCE) - return READ_ONCE(huge_shmem_orders_inherit); + return READ_ONCE(huge_shmem_orders_inherit) & ~disabled_orders; /* Allow mTHP that will be fully within i_size. */ mask |= shmem_get_orders_within_size(inode, within_size_orders, index, 0); @@ -1864,6 +1867,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode, if (global_orders > 0) mask |= READ_ONCE(huge_shmem_orders_inherit); + mask &= ~disabled_orders; return THP_ORDERS_ALL_FILE_DEFAULT & mask; } @@ -5395,7 +5399,7 @@ void __init shmem_init(void) * Default to setting PMD-sized THP to inherit the global setting and * disable all other multi-size THPs. */ - if (!shmem_orders_configured) + if (!shmem_orders_configured && pgtable_has_pmd_leaves()) huge_shmem_orders_inherit = BIT(HPAGE_PMD_ORDER); #endif return; -- 2.55.0