Add the KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION vCPU device attribute to enable a Partitioned PMU on hosts with VHE and PMUv3 support. When partitioning is enabled (KVM_ARCH_FLAG_PARTITION_PMU_ENABLED), userspace must configure guest general-purpose counters via KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS before KVM_ARM_VCPU_PMU_V3_INIT: - Require kvm_arm_pmu_part_hpmn(n) < max_counters so at least one general-purpose counter remains available for host profiling. - Allow n == 0 (cycle counter only) on hosts without FEAT_HPMN0 when max_counters >= 2, reserving physical counter 0 (MDCR_EL2.HPMN = 1) and hiding it from the guest via traps. Because PMU attributes are invoked per-vCPU while arm_pmu, nr_pmu_counters, and partitioning state are VM-wide, VMMs such as QEMU iterate over vCPUs (CPU_FOREACH) and call KVM_ARM_VCPU_PMU_V3_INIT on vCPU 0 before configuring vCPU 1..N. Allow SET_PMU, SET_NR_COUNTERS, and ENABLE_PARTITION to succeed when a vCPU PMU is already initialized as long as the requested value matches the VM's existing configuration, and return -EBUSY on conflicting changes. Signed-off-by: Colton Lewis --- Documentation/virt/kvm/devices/vcpu.rst | 57 +++++++++++++++-- arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/include/uapi/asm/kvm.h | 2 + arch/arm64/kvm/pmu.c | 83 ++++++++++++++++++++++++- arch/arm64/kvm/sys_regs.c | 5 +- include/kvm/arm_pmu.h | 6 ++ 6 files changed, 148 insertions(+), 6 deletions(-) diff --git a/Documentation/virt/kvm/devices/vcpu.rst b/Documentation/virt/kvm/devices/vcpu.rst index 7e23584fd74aa..1140c1d731d63 100644 --- a/Documentation/virt/kvm/devices/vcpu.rst +++ b/Documentation/virt/kvm/devices/vcpu.rst @@ -57,6 +57,9 @@ Returns: hardware PMU, or interrupt number not set (non-GICv5 guests, only) -EBUSY PMUv3 already initialized + -EINVAL Partitioning enabled without explicitly configuring + a valid counter count via + KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS ======= ====================================================== Request the initialization of the PMUv3. If using the PMUv3 with an in-kernel @@ -161,16 +164,62 @@ the cpu field to the processor id. an event filter has already been set -EFAULT Error accessing the value pointed to by addr -ENODEV PMUv3 not supported or GIC not initialized - -EINVAL No PMUv3 explicitly selected, or value of N out of - range + -EINVAL No PMUv3 explicitly selected, or value of N > max_counters, + or N > max_counters - 1 (or N == 0 when max_counters < 2 + without FEAT_HPMN0) when partitioning is enabled ======= ==================================================== Set the number of implemented event counters in the virtual PMU. This mandates that a PMU has explicitly been selected via KVM_ARM_VCPU_PMU_V3_SET_PMU, and will fail when no PMU has been explicitly selected, or the number of counters is out of range for the -selected PMU. Selecting a new PMU cancels the effect of setting this -attribute. +selected PMU. The maximum number of general-purpose counters (max_counters) +implemented by a PMU can be queried from sysfs at +/sys/bus/event_source/devices//caps/num_counters. Selecting a new PMU +cancels the effect of setting this attribute. + +1.6 ATTRIBUTE: KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION +--------------------------------------------------- + +:Parameters: in kvm_device_attr.addr the address to an unsigned int (u32) + boolean value (non-zero to enable PMU partitioning, 0 to disable) + +:Returns: + + ======= ======================================================== + -EBUSY PMUv3 already initialized or a VCPU has already run + -EFAULT Error accessing the value pointed to by addr + -ENODEV KVM_ARM_VCPU_PMU_V3 feature missing from VCPU + -EPERM Host hardware or kernel configuration does not support + PMU partitioning (requires ARM64 VHE mode and PMUv3) + -EINVAL No PMUv3 associated with the VM, or + KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS was already set to + > max_counters - 1 (or 0 when max_counters < 2 without + FEAT_HPMN0) + -ENXIO Returned by KVM_HAS_DEVICE_ATTR when PMU partitioning is + unsupported on host hardware + ======= ======================================================== + +Enable or disable hardware PMU partitioning for the VM. When enabled, physical +PMUv3 hardware counters are partitioned between the guest and host using +MDCR_EL2.HPMN (and FEAT_FGT fine-grained traps when supported by hardware). +This grants the guest direct, untrapped EL0/EL1 hardware access to event +counters 0..HPMN-1 and the cycle counter (PMCCNTR_EL0), and restricts the VM's +supported physical CPUs to those associated with the VM's PMU. + +When PMU partitioning is enabled, userspace must explicitly configure the +number of guest event counters via KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS with a +value strictly less than the maximum number of general-purpose counters +implemented by the PMU (leaving at least one general-purpose counter reserved +for host profiling) prior to calling KVM_ARM_VCPU_PMU_V3_INIT. Configuring 0 +event counters (cycle counter only) on hosts without FEAT_HPMN0 programs +MDCR_EL2.HPMN = 1, reserves physical counter 0 in addition to PMCCNTR_EL0 +while hiding it from the guest via traps, and therefore requires at least two +implemented general-purpose counters (max_counters >= 2) so at least one +remains for host profiling. Note that KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS +configures general-purpose event counters (PMCR_EL0.N / MDCR_EL2.HPMN); the +dedicated cycle counter (PMCCNTR_EL0) is unconditionally assigned to the +guest partition when partitioning is enabled. 2. GROUP: KVM_ARM_VCPU_TIMER_CTRL ================================= diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 1fcb12995c34c..01f26a37c7f9c 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -472,6 +472,7 @@ struct kvm_arch { /* Maximum number of counters for the guest */ u8 nr_pmu_counters; + bool pmu_nr_counters_specified; /* PMMIR_EL1.SLOTS value exposed to the guest. */ u8 pmmir_slots; diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index 65ee09bd14636..80e2504c1d9fb 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -453,6 +453,8 @@ struct kvm_vgic_v5_ist { #define KVM_ARM_VCPU_PMU_V3_FILTER 2 #define KVM_ARM_VCPU_PMU_V3_SET_PMU 3 #define KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS 4 +#define KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION 5 + #define KVM_ARM_VCPU_TIMER_CTRL 1 #define KVM_ARM_VCPU_TIMER_IRQ_VTIMER 0 #define KVM_ARM_VCPU_TIMER_IRQ_PTIMER 1 diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c index 02a1968bd6cb1..e11f54a92ecfe 100644 --- a/arch/arm64/kvm/pmu.c +++ b/arch/arm64/kvm/pmu.c @@ -479,12 +479,22 @@ int kvm_arm_pmu_v3_enable(struct kvm_vcpu *vcpu) return 0; } +bool kvm_arm_pmu_part_nr_valid(struct kvm *kvm, unsigned int n) +{ + return kvm_arm_pmu_part_hpmn(n) < kvm_arm_pmu_get_max_counters(kvm); +} + static int kvm_arm_pmu_v3_init(struct kvm_vcpu *vcpu) { /* Only possible when using KVM_ARM_VCPU_PMU_V3_STRICT */ if (!vcpu->kvm->arch.arm_pmu) return -ENXIO; + if (kvm_pmu_is_partitioned(vcpu->kvm) && + (!vcpu->kvm->arch.pmu_nr_counters_specified || + !kvm_arm_pmu_part_nr_valid(vcpu->kvm, vcpu->kvm->arch.nr_pmu_counters))) + return -EINVAL; + if (irqchip_in_kernel(vcpu->kvm)) { int ret; @@ -596,11 +606,25 @@ static void kvm_arm_set_nr_counters(struct kvm *kvm, unsigned int nr) } } +static bool kvm_arm_pmu_any_vcpu_created(struct kvm *kvm) +{ + struct kvm_vcpu *vcpu; + unsigned long i; + + kvm_for_each_vcpu(i, vcpu, kvm) { + if (vcpu->arch.pmu.created) + return true; + } + + return false; +} + static void kvm_arm_set_pmu(struct kvm *kvm, struct arm_pmu *arm_pmu) { lockdep_assert_held(&kvm->arch.config_lock); kvm->arch.arm_pmu = arm_pmu; + kvm->arch.pmu_nr_counters_specified = false; kvm_arm_set_nr_counters(kvm, kvm_arm_pmu_get_max_counters(kvm)); } @@ -647,6 +671,11 @@ static int kvm_arm_pmu_v3_set_pmu(struct kvm_vcpu *vcpu, int pmu_id) break; } + if (kvm_arm_pmu_any_vcpu_created(kvm)) { + ret = (kvm->arch.arm_pmu == arm_pmu) ? 0 : -EBUSY; + break; + } + kvm_arm_set_pmu(kvm, arm_pmu); cpumask_copy(kvm->arch.supported_cpus, &arm_pmu->supported_cpus); @@ -672,14 +701,31 @@ static int kvm_arm_pmu_v3_set_pmu(struct kvm_vcpu *vcpu, int pmu_id) static int kvm_arm_pmu_v3_set_nr_counters(struct kvm_vcpu *vcpu, unsigned int n) { struct kvm *kvm = vcpu->kvm; + unsigned int max_counters; + + if (kvm_vm_has_ran_once(kvm)) + return -EBUSY; + + if (kvm_arm_pmu_any_vcpu_created(kvm)) { + if (!kvm->arch.pmu_nr_counters_specified || + kvm->arch.nr_pmu_counters != n) + return -EBUSY; + return 0; + } if (!kvm->arch.arm_pmu) return -EINVAL; - if (n > kvm_arm_pmu_get_max_counters(kvm)) + max_counters = kvm_arm_pmu_get_max_counters(kvm); + if (n > max_counters) + return -EINVAL; + + if (kvm_pmu_is_partitioned(kvm) && + !kvm_arm_pmu_part_nr_valid(kvm, n)) return -EINVAL; kvm_arm_set_nr_counters(kvm, n); + kvm->arch.pmu_nr_counters_specified = true; return 0; } @@ -791,6 +837,36 @@ int kvm_arm_pmu_v3_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) return kvm_arm_pmu_v3_set_nr_counters(vcpu, n); } + case KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION: { + unsigned int __user *uaddr = (unsigned int __user *)(long)attr->addr; + u32 val; + + if (get_user(val, uaddr)) + return -EFAULT; + + if (!has_kvm_pmu_partition_support()) + return -EPERM; + + if (kvm_vm_has_ran_once(kvm)) + return -EBUSY; + + if (kvm_arm_pmu_any_vcpu_created(kvm)) + return kvm_pmu_is_partitioned(kvm) == !!val ? 0 : -EBUSY; + + if (!kvm->arch.arm_pmu) + return -EINVAL; + + if (val && kvm->arch.pmu_nr_counters_specified && + !kvm_arm_pmu_part_nr_valid(kvm, kvm->arch.nr_pmu_counters)) + return -EINVAL; + + if (val) + cpumask_copy(kvm->arch.supported_cpus, + &kvm->arch.arm_pmu->supported_cpus); + + kvm_pmu_partition_enable(kvm, val); + return 0; + } case KVM_ARM_VCPU_PMU_V3_INIT: return kvm_arm_pmu_v3_init(vcpu); } @@ -832,6 +908,11 @@ int kvm_arm_pmu_v3_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) case KVM_ARM_VCPU_PMU_V3_SET_NR_COUNTERS: if (kvm_vcpu_has_pmu(vcpu)) return 0; + break; + case KVM_ARM_VCPU_PMU_V3_ENABLE_PARTITION: + if (kvm_vcpu_has_pmu(vcpu) && has_kvm_pmu_partition_support()) + return 0; + break; } return -ENXIO; diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 4967954f42b01..22d78605b8f65 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1591,7 +1591,10 @@ static int set_pmcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r, if (!kvm_vm_has_ran_once(kvm) && !vcpu_has_nv(vcpu) && !kvm_vcpu_has_pmuv3_strict(vcpu) && - new_n <= kvm_arm_pmu_get_max_counters(kvm)) + !kvm->arch.pmu_nr_counters_specified && + new_n <= kvm_arm_pmu_get_max_counters(kvm) && + (!kvm_pmu_is_partitioned(kvm) || + kvm_arm_pmu_part_nr_valid(kvm, new_n))) kvm->arch.nr_pmu_counters = new_n; mutex_unlock(&kvm->arch.config_lock); diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h index bebf981855040..479eabc96b412 100644 --- a/include/kvm/arm_pmu.h +++ b/include/kvm/arm_pmu.h @@ -138,6 +138,7 @@ u64 kvm_pmu_evtyper_mask(struct kvm *kvm); int kvm_arm_set_default_pmu(struct kvm *kvm); u8 kvm_arm_pmu_get_max_counters(struct kvm *kvm); unsigned int kvm_arm_pmu_part_hpmn(unsigned int n); +bool kvm_arm_pmu_part_nr_valid(struct kvm *kvm, unsigned int n); u64 kvm_vcpu_read_pmcr(struct kvm_vcpu *vcpu); bool kvm_pmu_counter_is_hyp(struct kvm_vcpu *vcpu, unsigned int idx); @@ -283,6 +284,11 @@ static inline unsigned int kvm_arm_pmu_part_hpmn(unsigned int n) return n; } +static inline bool kvm_arm_pmu_part_nr_valid(struct kvm *kvm, unsigned int n) +{ + return false; +} + static inline u64 kvm_vcpu_read_pmcr(struct kvm_vcpu *vcpu) { return 0; -- 2.56.1.708.g77570a6a1d-goog