page_ext lays out client payloads consecutively without aligning their offsets or the final entry size. This happened to work while the active payload sizes kept the stride naturally aligned, but IOMMU debug page allocation added a four-byte payload after data requiring eight-byte alignment. When page owner and IOMMU debug page allocation are both enabled on x86-64, struct page_ext, struct page_owner, and the IOMMU payload use 8, 64, and 4 bytes respectively. This produces a 76-byte entry stride, leaving struct page_ext and page_owner data in every other entry only four-byte aligned instead of eight-byte aligned. Derive a suitable alignment for each payload from the lowest set bit of its size, capped at unsigned long alignment. Align each client offset and round the final entry size to the largest active alignment so every entry preserves the alignment required by all active clients. Fixes: 466ae6978a5b ("iommu: Add page_ext for IOMMU_DEBUG_PAGEALLOC") Signed-off-by: Muchun Song --- v2: - New patch to fix the existing page_ext client alignment bug --- mm/page_ext.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/mm/page_ext.c b/mm/page_ext.c index b679a8c1f7d7..dbc1bfa258f1 100644 --- a/mm/page_ext.c +++ b/mm/page_ext.c @@ -116,8 +116,22 @@ static int __init setup_early_page_ext(char *str) } early_param("early_page_ext", setup_early_page_ext); +static inline size_t page_ext_data_alignment(size_t size) +{ + if (!size) + return 1; + + /* + * A type's size is a multiple of its natural alignment. The lowest + * set bit of the size is therefore at least that alignment. Cap it + * at unsigned long alignment to avoid excessive per-page padding. + */ + return min_t(size_t, 1UL << __ffs(size), __alignof__(unsigned long)); +} + static bool __init invoke_need_callbacks(void) { + size_t max_align = 1; int i; int entries = ARRAY_SIZE(page_ext_ops); bool need = false; @@ -126,6 +140,7 @@ static bool __init invoke_need_callbacks(void) if (page_ext_ops[i]->need()) { if (page_ext_ops[i]->need_shared_flags) { page_ext_size = sizeof(struct page_ext); + max_align = __alignof__(struct page_ext); break; } } @@ -133,11 +148,17 @@ static bool __init invoke_need_callbacks(void) for (i = 0; i < entries; i++) { if (page_ext_ops[i]->need()) { + size_t align; + + align = page_ext_data_alignment(page_ext_ops[i]->size); + page_ext_size = ALIGN(page_ext_size, align); page_ext_ops[i]->offset = page_ext_size; page_ext_size += page_ext_ops[i]->size; + max_align = max(max_align, align); need = true; } } + page_ext_size = ALIGN(page_ext_size, max_align); return need; } -- 2.54.0