When resetting a vCPU's dirty ring, stop if reset_index has caught up to dirty_index instead of relying solely on the RESET flag of the next entry. The ring is mapped writable into userspace, so userspace can set KVM_DIRTY_GFN_F_RESET on entries that KVM never pushed, in which case KVM advances reset_index past dirty_index. The u32 math in kvm_dirty_ring_used() then underflows, and the next push hits the WARN_ON_ONCE() in kvm_dirty_ring_push(): WARNING: virt/kvm/dirty_ring.c:225 at kvm_dirty_ring_push+0x2b2/0x5e0 Call Trace: make_spte+0x71a/0xc80 kvm_tdp_mmu_map+0x140e/0x1f40 kvm_tdp_page_fault+0x2f7/0x3a0 kvm_mmu_page_fault+0x1f8/0xc00 vcpu_run+0x4da1/0x7890 kvm_arch_vcpu_ioctl_run+0xe83/0x18b0 kvm_vcpu_ioctl+0xa61/0xfd0 Userspace that scribbles on its own dirty ring gets to keep the pieces, but KVM shouldn't WARN, and keeping reset_index from passing dirty_index is trivial. To avoid skipping entries that were legitimately harvested while the vCPU is running, bump dirty_index before publishing the DIRTY flag in kvm_dirty_ring_push(), and check dirty_index after the load-acquire of the RESET flag in kvm_dirty_ring_reset(). If KVM observes the RESET flag for an entry, i.e. userspace observed the DIRTY flag, then KVM is guaranteed to also observe the updated dirty_index. Fixes: fb04a1eddb1a ("KVM: X86: Implement ring-based dirty memory tracking") Reported-by: syzbot+55eaa971f4924b62c9a5@syzkaller.appspotmail.com Closes: https://lore.kernel.org/all/6afbad4d.f3cfa7c9.844a4.0002.GAE@google.com Cc: stable@vger.kernel.org Cc: Peter Xu Signed-off-by: James Houghton --- virt/kvm/dirty_ring.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c index 572b854edf74..8fab75dace76 100644 --- a/virt/kvm/dirty_ring.c +++ b/virt/kvm/dirty_ring.c @@ -140,6 +140,16 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_dirty_ring *ring, if (!kvm_dirty_gfn_harvested(entry)) break; + /* + * Don't reset entries that KVM hasn't pushed, e.g. if userspace + * set the RESET flag on an unused entry. Check the number of + * used entries after checking the flag, so that dirty_index is + * up-to-date for entries that were harvested (pairs with the + * ordering in kvm_dirty_ring_push()). + */ + if (!kvm_dirty_ring_used(ring)) + break; + next_slot = READ_ONCE(entry->slot); next_offset = READ_ONCE(entry->offset); @@ -229,12 +239,14 @@ void kvm_dirty_ring_push(struct kvm_vcpu *vcpu, u32 slot, u64 offset) entry->slot = slot; entry->offset = offset; /* - * Make sure the data is filled in before we publish this to - * the userspace program. There's no paired kernel-side reader. + * Bump dirty_index before publishing the entry, so that + * kvm_dirty_ring_reset() sees the new dirty_index if it sees the entry + * as harvested, and make sure the data is filled in before publishing + * the entry to userspace. */ + WRITE_ONCE(ring->dirty_index, ring->dirty_index + 1); smp_wmb(); kvm_dirty_gfn_set_dirtied(entry); - ring->dirty_index++; trace_kvm_dirty_ring_push(ring, slot, offset); if (kvm_dirty_ring_soft_full(ring)) base-commit: 666539eecdf4db623f5e3c3e55cf9f2dfad24aa4 -- 2.56.1.708.g77570a6a1d-goog