The expedited membarrier commands need a full memory barrier after the scheduler stores to rq->curr, before the CPU returns to user-space. membarrier() skips the IPI to a CPU which it sees running another mm. This barrier orders the user-space accesses of the incoming thread after that check. See scenario (A) in kernel/sched/membarrier.c. parisc has no dedicated barrier. It used to get it from smp_store_release() in finish_task() and from spin_unlock() in finish_lock_switch(). The first one issued a SYNC instruction. The second one issued an LDCW instruction, which is strongly ordered. Both are ordered stores since v5.9: - commit 157e9afcc4fa ("Revert "parisc: Revert "Release spinlocks using ordered store"""), - commit e96ebd589deb ("parisc: Implement __smp_store_release and __smp_load_acquire barriers"). Linux runs with the PSW O bit clear. The PA-RISC 2.0 architecture then allows plain loads and stores to be weakly ordered. It orders an ordered store before a following ordered load, not before a following plain load. So an ordered store is not a full barrier. Nothing else is a full barrier on a switch between two user processes. switch_mm_irqs_off() only writes control and space registers. The PA-RISC 2.0 manual lists the strongly ordered accesses: I/O space, pages with the TLB U-bit set, LDCW, LDCD, the TLB purges and SYNC. MTCTL, MTSP and RFI are not among them. This was found by reading the code and the architecture manual. On a PA-RISC 2.0 SMP machine, a thread which is scheduled in would run user-space code without the promised barrier. User-space would see a memory ordering failure: the caller of an expedited membarrier command would miss a recent store of that thread. PA-RISC 1.1 is strongly ordered and not affected. Do as powerpc and riscv: add membarrier_arch_switch_mm() and call it from switch_mm_irqs_off() when the mm changes. It issues smp_mb() only when the next mm registered for an expedited command. Compile-tested only. Fixes: 157e9afcc4fa ("Revert "parisc: Revert "Release spinlocks using ordered store""") Fixes: e96ebd589deb ("parisc: Implement __smp_store_release and __smp_load_acquire barriers") Cc: stable@vger.kernel.org Signed-off-by: Mathieu Desnoyers --- arch/parisc/Kconfig | 1 + arch/parisc/include/asm/membarrier.h | 33 +++++++++++++++++++++++++++ arch/parisc/include/asm/mmu_context.h | 2 ++ 3 files changed, 36 insertions(+) create mode 100644 arch/parisc/include/asm/membarrier.h diff --git a/arch/parisc/Kconfig b/arch/parisc/Kconfig index 77f67028ad89..3d6b4cc8d6a5 100644 --- a/arch/parisc/Kconfig +++ b/arch/parisc/Kconfig @@ -13,6 +13,7 @@ config PARISC select ARCH_HAS_DMA_ALLOC if PA11 select ARCH_HAS_DMA_OPS select ARCH_HAS_ELF_RANDOMIZE + select ARCH_HAS_MEMBARRIER_CALLBACKS select ARCH_HAS_STRICT_KERNEL_RWX select ARCH_HAS_STRICT_MODULE_RWX select ARCH_HAS_UBSAN diff --git a/arch/parisc/include/asm/membarrier.h b/arch/parisc/include/asm/membarrier.h new file mode 100644 index 000000000000..822dcadaf9bb --- /dev/null +++ b/arch/parisc/include/asm/membarrier.h @@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _ASM_PARISC_MEMBARRIER_H +#define _ASM_PARISC_MEMBARRIER_H + +static inline void membarrier_arch_switch_mm(struct mm_struct *prev, + struct mm_struct *next, + struct task_struct *tsk) +{ + /* + * Only need the full barrier when switching between processes. + * Barrier when switching from kernel to userspace is not + * required here, given that it is implied by mmdrop(). Barrier + * when switching from userspace to kernel is not needed after + * store to rq->curr. + */ + if (IS_ENABLED(CONFIG_SMP) && + likely(!(atomic_read(&next->membarrier_state) & + (MEMBARRIER_STATE_PRIVATE_EXPEDITED | + MEMBARRIER_STATE_GLOBAL_EXPEDITED)))) + return; + + /* + * The membarrier system call requires a full memory barrier + * after storing to rq->curr, before going back to user-space. + * + * Nothing else provides it on this path. spin_unlock() and + * smp_store_release() are ordered stores. PA-RISC 2.0 does not + * order an ordered store with a following weakly ordered load. + */ + smp_mb(); +} + +#endif /* _ASM_PARISC_MEMBARRIER_H */ diff --git a/arch/parisc/include/asm/mmu_context.h b/arch/parisc/include/asm/mmu_context.h index c9187fe836a3..42ff1b9444f1 100644 --- a/arch/parisc/include/asm/mmu_context.h +++ b/arch/parisc/include/asm/mmu_context.h @@ -4,6 +4,7 @@ #include #include +#include #include #include #include @@ -51,6 +52,7 @@ static inline void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, struct task_struct *tsk) { if (prev != next) { + membarrier_arch_switch_mm(prev, next, tsk); #ifdef CONFIG_TLB_PTLOCK /* put physical address of page_table_lock in cr28 (tr4) for TLB faults */ -- 2.47.3