6.18-stable review patch. If anyone has any objections, please let me know. ------------------ From: Sean Christopherson commit db095727ff5739f4f46ee641ee6ef450032886db upstream. Manually invoke the front half and back half of the "zap all fast" flow when invalidating a memslot so that mmu_lock is acquired at function scope in kvm_arch_flush_shadow_memslot(). This will allow putting more code inside the critical section without having to take mmu_lock twice in quick succession. Opportunistically open code checking whether or not to do the fast zap, to discourage removing the local "zap_all" in a future cleanup, i.e. to ensure the SLOT_ZAP_ALL quirk is queried exactly once. Processing the front half but not the back half of the fast zap (if SLOT_ZAP_ALL were disabled concurrently) would result in KVM unnecessarily keeping invalid TDP MMU roots until the VM is destroyed. No functional change intended. Cc: stable@vger.kernel.org # 6.12.x Reviewed-by: Michael Roth Link: https://patch.msgid.link/20260709204948.1988414-10-seanjc@google.com Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman --- arch/x86/kvm/mmu/mmu.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -7348,12 +7348,6 @@ out_flush: kvm_mmu_remote_flush_or_zap(kvm, &invalid_list, flush); } -static inline bool kvm_memslot_flush_zap_all(struct kvm *kvm) -{ - return kvm->arch.vm_type == KVM_X86_DEFAULT_VM && - kvm_check_has_quirk(kvm, KVM_X86_QUIRK_SLOT_ZAP_ALL); -} - void kvm_arch_flush_shadow_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) { @@ -7364,16 +7358,23 @@ void kvm_arch_flush_shadow_memslot(struc .may_block = true, .attr_filter = KVM_FILTER_PRIVATE | KVM_FILTER_SHARED, }; + bool zap_all = kvm->arch.vm_type == KVM_X86_DEFAULT_VM && + kvm_check_has_quirk(kvm, KVM_X86_QUIRK_SLOT_ZAP_ALL); bool flush; - if (kvm_memslot_flush_zap_all(kvm)) { - kvm_mmu_zap_all_fast(kvm); + write_lock(&kvm->mmu_lock); + + if (zap_all) { + __kvm_mmu_zap_all_fast_front_half(kvm); } else { - write_lock(&kvm->mmu_lock); flush = kvm_unmap_gfn_range(kvm, &range); kvm_mmu_zap_memslot_pages_and_flush(kvm, slot, flush); - write_unlock(&kvm->mmu_lock); } + + write_unlock(&kvm->mmu_lock); + + if (zap_all) + __kvm_mmu_zap_all_fast_back_half(kvm); } void kvm_mmu_invalidate_mmio_sptes(struct kvm *kvm, u64 gen)