thp_underused() decides whether a large folio on the deferred split list is underused and should be split so that its zero-filled subpages can be reclaimed. Today it only ever checks if the folio is underused on PMD-sized folios. As such, thp_underused() assumes throughout that the folio has HPAGE_PMD_NR pages. A later patch will be queueing anonymous mTHP folios on the deferred split list as well, at which point this assumption will not be correct anymore. Generalize thp_underused() to be folio-size-aware instead of hardcoded to PMD-sized folios. The check for whether a folio can ever be underused is moved into its own function, thp_can_be_underused(), as this will get reused in the next patch when determining if the folio should get added to the deferred split queue. khugepaged_max_ptes_none keeps its meaning as an absolute number of pages, which is also how khugepaged applies it when collapsing to PMD order. It is deliberately not scaled down per folio order (collapse rejected proportional scaling of intermediate values because it either lets the memory footprint creep or ends up confusing to reason about). Read as an absolute count, this knob gains a useful second meaning for mTHP as being the smallest folio size that takes part in underused splitting at all. There is no functional change, as for a PMD-sized folio nr_pages is HPAGE_PMD_NR. Suggested-by: Johannes Weiner Signed-off-by: Joanne Koong --- mm/huge_memory.c | 31 +++++++++++++++++++++++++------ 1 file changed, 25 insertions(+), 6 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1e5d68acf62a..c6ca2a541128 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -4541,6 +4541,23 @@ bool __folio_unqueue_deferred_split(struct folio *folio) return unqueued; /* useful for debug warnings */ } +static bool thp_can_be_underused(unsigned long nr_pages, + unsigned long max_ptes_none) +{ + /* + * The sysctl maximum means the user tolerates any number of zero-filled + * pages, so nothing is ever underused. + */ + if (max_ptes_none == HPAGE_PMD_NR - 1) + return false; + + /* + * A folio no larger than the number of zero-filled pages the user + * tolerates can never exceed it. It can never be underused. + */ + return nr_pages > max_ptes_none; +} + /* partially_mapped=false won't clear PG_partially_mapped folio flag */ void deferred_split_folio(struct folio *folio, bool partially_mapped) { @@ -4602,25 +4619,27 @@ static unsigned long deferred_split_count(struct shrinker *shrink, static bool thp_underused(struct folio *folio) { - int num_zero_pages = 0, num_filled_pages = 0; - int i; + const unsigned long max_ptes_none = khugepaged_max_ptes_none; + const unsigned long nr_pages = folio_nr_pages(folio); + unsigned long num_zero_pages = 0, num_filled_pages = 0; + unsigned long i; - if (khugepaged_max_ptes_none == HPAGE_PMD_NR - 1) + if (!thp_can_be_underused(nr_pages, max_ptes_none)) return false; if (folio_contain_hwpoisoned_page(folio)) return false; - for (i = 0; i < folio_nr_pages(folio); i++) { + for (i = 0; i < nr_pages; i++) { if (pages_identical(folio_page(folio, i), ZERO_PAGE(0))) { - if (++num_zero_pages > khugepaged_max_ptes_none) + if (++num_zero_pages > max_ptes_none) return true; } else { /* * Another path for early exit once the number * of non-zero filled pages exceeds threshold. */ - if (++num_filled_pages >= HPAGE_PMD_NR - khugepaged_max_ptes_none) + if (++num_filled_pages >= nr_pages - max_ptes_none) return false; } } -- 2.52.0