Currently, we have two helpers that check for PMD-sized pages but have different names and slightly different semantics: - has_transparent_hugepage(): the name suggests it checks if THP is enabled, but when CONFIG_TRANSPARENT_HUGEPAGE=y and the architecture implements this helper, it actually checks if the CPU supports PMD-sized pages - thp_disabled_by_hw(): the name suggests it checks if THP is disabled by the hardware, but it just returns a cached value acquired with has_transparent_hugepage(). This helper is used in fast paths This commit introduces a new helper called pgtable_has_pmd_leaves() which is intended to replace both has_transparent_hugepage() and thp_disabled_by_hw(). pgtable_has_pmd_leaves() has very clear semantics: it returns true if the CPU supports PMD-sized pages and false otherwise. It always returns a cached value, so it can be used in fast paths. The new helper requires an initialization step which is performed by init_arch_has_pmd_leaves(). We call init_arch_has_pmd_leaves() early during boot in start_kernel() right after parse_early_param() but before parse_args(). This allows early_param() handlers to change CPU flags if needed (eg. parse_memopt() in x86-32) while also allowing users to use the API from __setup() handlers. The next commits will convert users of both has_transparent_hugepage() and thp_disabled_by_hw() to pgtable_has_pmd_leaves(). Signed-off-by: Luiz Capitulino --- include/linux/pgtable.h | 15 +++++++++++++++ init/main.c | 1 + mm/memory.c | 9 +++++++++ 3 files changed, 25 insertions(+) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index cdd68ed3ae1a..b365be3516bf 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -2243,6 +2243,21 @@ static inline const char *pgtable_level_to_str(enum pgtable_level level) } } +#ifdef CONFIG_MMU +DECLARE_STATIC_KEY_TRUE(__arch_has_pmd_leaves_key); +static inline bool pgtable_has_pmd_leaves(void) +{ + return static_branch_likely(&__arch_has_pmd_leaves_key); +} +void __init init_arch_has_pmd_leaves(void); +#else +static inline bool pgtable_has_pmd_leaves(void) +{ + return false; +} +static inline void __init init_arch_has_pmd_leaves(void) { } +#endif + #endif /* !__ASSEMBLY__ */ #if !defined(MAX_POSSIBLE_PHYSMEM_BITS) && !defined(CONFIG_64BIT) diff --git a/init/main.c b/init/main.c index 96f93bb06c49..eea7c5bdddf7 100644 --- a/init/main.c +++ b/init/main.c @@ -1053,6 +1053,7 @@ void start_kernel(void) print_kernel_cmdline(saved_command_line); /* parameters may set static keys */ parse_early_param(); + init_arch_has_pmd_leaves(); after_dashes = parse_args("Booting kernel", static_command_line, __start___param, __stop___param - __start___param, diff --git a/mm/memory.c b/mm/memory.c index ea6568571131..90b2d9e84320 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -164,6 +164,15 @@ __setup("norandmaps", disable_randmaps); unsigned long highest_memmap_pfn __read_mostly; +DEFINE_STATIC_KEY_TRUE(__arch_has_pmd_leaves_key); +EXPORT_SYMBOL(__arch_has_pmd_leaves_key); + +void __init init_arch_has_pmd_leaves(void) +{ + if (!has_transparent_hugepage()) + static_branch_disable(&__arch_has_pmd_leaves_key); +} + void mm_trace_rss_stat(struct mm_struct *mm, int member) { trace_rss_stat(mm, member); -- 2.53.0