When vmalloc shadow memory is established the modification of the corresponding page tables is not protected by any locks. Instead, the locking is done per-PTE. This scheme however has defects. kasan_populate_vmalloc_pte() - while ptep_get() read is atomic the sequence pte_none(ptep_get()) is not. Doing that outside of the lock might lead to a concurrent PTE update and what could be seen as a shadow memory corruption as result. kasan_depopulate_vmalloc_pte() - by the time a page whose address was extracted from ptep_get() read and cached in a local variable outside of the lock is attempted to get free, could actually be freed already. To avoid these put ptep_get() itself and the code that manipulates the result of the read under lock. In addition, move freeing of the page out of the atomic context. Fixes: 3c5c3cfb9ef4 ("kasan: support backing vmalloc space with real shadow memory") Signed-off-by: Alexander Gordeev --- mm/kasan/shadow.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index d2c70cd2afb1..4d846d146d02 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -305,9 +305,6 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr, pte_t pte; int index; - if (likely(!pte_none(ptep_get(ptep)))) - return 0; - index = PFN_DOWN(addr - data->start); page = data->pages[index]; __memset(page_to_virt(page), KASAN_VMALLOC_INVALID, PAGE_SIZE); @@ -461,18 +458,19 @@ int kasan_populate_vmalloc(unsigned long addr, unsigned long size) static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, void *unused) { - unsigned long page; - - page = (unsigned long)__va(pte_pfn(ptep_get(ptep)) << PAGE_SHIFT); + pte_t pte; + int none; spin_lock(&init_mm.page_table_lock); - - if (likely(!pte_none(ptep_get(ptep)))) { + pte = ptep_get(ptep); + none = pte_none(pte); + if (likely(!none)) pte_clear(&init_mm, addr, ptep); - free_page(page); - } spin_unlock(&init_mm.page_table_lock); + if (likely(!none)) + __free_page(pfn_to_page(pte_pfn(pte))); + return 0; } -- 2.48.1 Functions __kasan_populate_vmalloc() and __kasan_depopulate_vmalloc() use apply_to_pte_range() function, which enters the lazy MMU mode. In that mode updating PTEs may not be observed until the mode is left. That may lead to a situation that otherwise correct reads and writes to a PTE using ptep_get(), set_pte(), pte_clear() and other access primitives bring wrong results when the vmalloc shadow memory is being (de-)populated. To avoid these hazards leave the lazy MMU mode before and re-enter it after each PTE manipulation. Fixes: 3c5c3cfb9ef4 ("kasan: support backing vmalloc space with real shadow memory") Signed-off-by: Alexander Gordeev --- mm/kasan/shadow.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 4d846d146d02..e2ceebf737ef 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -305,6 +305,8 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr, pte_t pte; int index; + arch_leave_lazy_mmu_mode(); + index = PFN_DOWN(addr - data->start); page = data->pages[index]; __memset(page_to_virt(page), KASAN_VMALLOC_INVALID, PAGE_SIZE); @@ -317,6 +319,8 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr, } spin_unlock(&init_mm.page_table_lock); + arch_enter_lazy_mmu_mode(); + return 0; } @@ -461,6 +465,8 @@ static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, pte_t pte; int none; + arch_leave_lazy_mmu_mode(); + spin_lock(&init_mm.page_table_lock); pte = ptep_get(ptep); none = pte_none(pte); @@ -471,6 +477,8 @@ static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, if (likely(!none)) __free_page(pfn_to_page(pte_pfn(pte))); + arch_enter_lazy_mmu_mode(); + return 0; } -- 2.48.1