IN_GUEST_MODE is advertised too late, when state has already been moved onto CPU registers and cannot be observed by another CPU anymore. Move the IN_GUEST_MODE publication ahead of the state being loaded, and immediately check for pending request. This ensures that there is no window of opportunity for another CPU to observe OUTSIDE_GUEST_MODE and yet for state to be loaded. This also simplifies the "state rollback", which solely amounts to synchronising the timer when no irqchip is present. Signed-off-by: Marc Zyngier Cc: stable@vger.kernel.org --- arch/arm64/kvm/arm.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 8b46fafdd3494..9956049e592ab 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1321,31 +1321,33 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) local_irq_disable(); - kvm_vgic_flush_hwstate(vcpu); - - kvm_pmu_update_vcpu_events(vcpu); - /* * Ensure we set mode to IN_GUEST_MODE after we disable * interrupts and before the final VCPU requests check. * See the comment in kvm_vcpu_exiting_guest_mode() and * Documentation/virt/kvm/vcpu-requests.rst + * + * Requests to self must be posted before this point, or + * won't be taken into account until the next exit. No state + * loading other than what happens at vcpu_load() time can + * be done before we have checked for pending requests. */ smp_store_mb(vcpu->mode, IN_GUEST_MODE); if (ret <= 0 || kvm_vcpu_exit_request(vcpu, &ret)) { - isb(); /* Ensure work in x_flush_hwstate is committed */ - if (kvm_vcpu_has_pmu(vcpu)) - kvm_pmu_sync_hwstate(vcpu); if (unlikely(!irqchip_in_kernel(vcpu->kvm))) kvm_timer_sync_user(vcpu); - kvm_vgic_sync_hwstate(vcpu); smp_store_release(&vcpu->mode, OUTSIDE_GUEST_MODE); local_irq_enable(); preempt_enable(); continue; } + /* From here onwards, we are commited to entering the guest */ + kvm_vgic_flush_hwstate(vcpu); + + kvm_pmu_update_vcpu_events(vcpu); + kvm_arch_vcpu_ctxflush_fp(vcpu); /************************************************************** -- 2.47.3