From: Wanpeng Li Integrate IPI tracking with directed yield to improve scheduling when vCPUs spin waiting for IPI responses. Implement priority-based candidate selection in kvm_vcpu_on_spin() with three tiers: Priority 1: Use kvm_vcpu_is_ipi_receiver() to identify confirmed IPI targets within the recency window, addressing lock holders spinning on IPI acknowledgment. Priority 2: Leverage existing kvm_arch_dy_has_pending_interrupt() for compatibility with arch-specific fast paths. Priority 3: Fall back to conventional preemption-based logic when yield_to_kernel_mode is requested, providing a safety net for non-IPI scenarios. Add kvm_vcpu_is_good_yield_candidate() helper to consolidate these checks, preventing over-aggressive boosting while enabling targeted optimization when IPI patterns are detected. Performance testing (16 pCPUs host, 16 vCPUs/VM): Dedup (simlarge): 2 VMs: +47.1% throughput 3 VMs: +28.1% throughput 4 VMs: +1.7% throughput VIPS (simlarge): 2 VMs: +26.2% throughput 3 VMs: +12.7% throughput 4 VMs: +6.0% throughput Gains stem from effective directed yield when vCPUs spin on IPI delivery, reducing synchronization overhead. The improvement is most pronounced at moderate overcommit (2-3 VMs) where contention reduction outweighs context switching cost. Signed-off-by: Wanpeng Li --- virt/kvm/kvm_main.c | 46 ++++++++++++++++++++++++++++++++++++--------- 1 file changed, 37 insertions(+), 9 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index ff771a872c6d..45ede950314b 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3970,6 +3970,41 @@ bool __weak kvm_vcpu_is_ipi_receiver(struct kvm_vcpu *sender, return false; } +/* + * IPI-aware candidate selection for directed yield. + * + * Priority order: + * 1) Confirmed IPI receiver of 'me' within recency window (always boost) + * 2) Arch-provided fast pending interrupt hint (user-mode boost) + * 3) Kernel-mode yield: preempted-in-kernel vCPU (traditional boost) + * 4) Otherwise, be conservative and skip + */ +static bool kvm_vcpu_is_good_yield_candidate(struct kvm_vcpu *me, + struct kvm_vcpu *vcpu, + bool yield_to_kernel_mode) +{ + /* Priority 1: recently targeted IPI receiver */ + if (kvm_vcpu_is_ipi_receiver(me, vcpu)) + return true; + + /* Priority 2: fast pending-interrupt hint (arch-specific) */ + if (kvm_arch_dy_has_pending_interrupt(vcpu)) + return true; + + /* + * Minimal preempted gate for remaining cases: + * Require that the target has been preempted, and if yielding to + * kernel mode, additionally require preempted-in-kernel. + */ + if (!READ_ONCE(vcpu->preempted)) + return false; + + if (yield_to_kernel_mode && !kvm_arch_vcpu_preempted_in_kernel(vcpu)) + return false; + + return true; +} + void kvm_vcpu_on_spin(struct kvm_vcpu *me, bool yield_to_kernel_mode) { int nr_vcpus, start, i, idx, yielded; @@ -4017,15 +4052,8 @@ void kvm_vcpu_on_spin(struct kvm_vcpu *me, bool yield_to_kernel_mode) if (kvm_vcpu_is_blocking(vcpu) && !vcpu_dy_runnable(vcpu)) continue; - /* - * Treat the target vCPU as being in-kernel if it has a pending - * interrupt, as the vCPU trying to yield may be spinning - * waiting on IPI delivery, i.e. the target vCPU is in-kernel - * for the purposes of directed yield. - */ - if (READ_ONCE(vcpu->preempted) && yield_to_kernel_mode && - !kvm_arch_dy_has_pending_interrupt(vcpu) && - !kvm_arch_vcpu_preempted_in_kernel(vcpu)) + /* IPI-aware candidate selection */ + if (!kvm_vcpu_is_good_yield_candidate(me, vcpu, yield_to_kernel_mode)) continue; if (!kvm_vcpu_eligible_for_directed_yield(vcpu)) -- 2.43.0