From: "Maciej S. Szmigiero" <maciej.szmigiero@oracle.com>

Since KVM commit 5f76f6f5ff96 ("KVM: nVMX: Do not expose MPX VMX controls when guest MPX disabled")
it is not possible to disable MPX on a "-cpu host" just by adding "-mpx"
there if the host CPU does indeed support MPX.
QEMU will fail to set MSR_IA32_VMX_TRUE_{EXIT,ENTRY}_CTLS MSRs in this case
and so trigger an assertion failure.

Instead, besides "-mpx" one has to explicitly add also
"-vmx-exit-clear-bndcfgs" and "-vmx-entry-load-bndcfgs" to QEMU command
line to make it work, which is a bit convoluted.

Sanitize MPX-related bits in MSR_IA32_VMX_TRUE_{EXIT,ENTRY}_CTLS after
setting the vCPU CPUID instead so such workarounds are no longer necessary.

Signed-off-by: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
---
 target/i386/kvm/kvm.c | 34 ++++++++++++++++++++++++++++------
 1 file changed, 28 insertions(+), 6 deletions(-)

diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index a9ee8eebd7..435cb18753 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -2934,6 +2934,17 @@ static uint64_t make_vmx_msr_value(uint32_t index, uint32_t features)
     return must_be_one | (((uint64_t)can_be_one) << 32);
 }
 
+static void kvm_msr_entry_add_if_supported(X86CPU *cpu, uint32_t msr,
+                                           uint32_t mask, uint32_t features,
+                                           uint64_t value_or)
+{
+    uint32_t supported =
+        kvm_arch_get_supported_msr_feature(kvm_state, msr) >> 32;
+    uint32_t feat_eff = features & (~mask | (mask & supported));
+
+    kvm_msr_entry_add(cpu, msr, make_vmx_msr_value(msr, feat_eff) | value_or);
+}
+
 static void kvm_msr_entry_add_vmx(X86CPU *cpu, FeatureWordArray f)
 {
     uint64_t kvm_vmx_basic =
@@ -2996,12 +3007,23 @@ static void kvm_msr_entry_add_vmx(X86CPU *cpu, FeatureWordArray f)
     kvm_msr_entry_add(cpu, MSR_IA32_VMX_TRUE_PINBASED_CTLS,
                       make_vmx_msr_value(MSR_IA32_VMX_TRUE_PINBASED_CTLS,
                                          f[FEAT_VMX_PINBASED_CTLS]));
-    kvm_msr_entry_add(cpu, MSR_IA32_VMX_TRUE_EXIT_CTLS,
-                      make_vmx_msr_value(MSR_IA32_VMX_TRUE_EXIT_CTLS,
-                                         f[FEAT_VMX_EXIT_CTLS]) | fixed_vmx_exit);
-    kvm_msr_entry_add(cpu, MSR_IA32_VMX_TRUE_ENTRY_CTLS,
-                      make_vmx_msr_value(MSR_IA32_VMX_TRUE_ENTRY_CTLS,
-                                         f[FEAT_VMX_ENTRY_CTLS]));
+
+    /*
+     * When disabling MPX on a host that supports this function it is not
+     * enough to clear the relevant CPUID bit, MPX-related bits in
+     * MSR_IA32_VMX_TRUE_{EXIT,ENTRY}_CTLS have to be cleared, too.
+     *
+     * Otherwise setting these MSRs will fail.
+     */
+    kvm_msr_entry_add_if_supported(cpu, MSR_IA32_VMX_TRUE_EXIT_CTLS,
+                                   VMX_VM_EXIT_CLEAR_BNDCFGS,
+                                   f[FEAT_VMX_EXIT_CTLS],
+                                   fixed_vmx_exit);
+    kvm_msr_entry_add_if_supported(cpu, MSR_IA32_VMX_TRUE_ENTRY_CTLS,
+                                   VMX_VM_ENTRY_LOAD_BNDCFGS,
+                                   f[FEAT_VMX_ENTRY_CTLS],
+                                   0);
+
     kvm_msr_entry_add(cpu, MSR_IA32_VMX_PROCBASED_CTLS2,
                       make_vmx_msr_value(MSR_IA32_VMX_PROCBASED_CTLS2,
                                          f[FEAT_VMX_SECONDARY_CTLS]));

On Fri, May 20, 2022 at 8:33 PM Maciej S. Szmigiero
<mail@maciej.szmigiero.name> wrote:
>
> From: "Maciej S. Szmigiero" <maciej.szmigiero@oracle.com>
>
> Since KVM commit 5f76f6f5ff96 ("KVM: nVMX: Do not expose MPX VMX controls when guest MPX disabled")
> it is not possible to disable MPX on a "-cpu host" just by adding "-mpx"
> there if the host CPU does indeed support MPX.
> QEMU will fail to set MSR_IA32_VMX_TRUE_{EXIT,ENTRY}_CTLS MSRs in this case
> and so trigger an assertion failure.
>
> Instead, besides "-mpx" one has to explicitly add also
> "-vmx-exit-clear-bndcfgs" and "-vmx-entry-load-bndcfgs" to QEMU command
> line to make it work, which is a bit convoluted.
>
> Sanitize MPX-related bits in MSR_IA32_VMX_TRUE_{EXIT,ENTRY}_CTLS after
> setting the vCPU CPUID instead so such workarounds are no longer necessary.

Can you use feature_dependencies instead? See for example

    {
        .from = { FEAT_7_0_EBX,             CPUID_7_0_EBX_RDSEED },
        .to = { FEAT_VMX_SECONDARY_CTLS,    VMX_SECONDARY_EXEC_RDSEED_EXITING },
    },

Paolo

On 21.05.2022 10:54, Paolo Bonzini wrote:
> On Fri, May 20, 2022 at 8:33 PM Maciej S. Szmigiero
> <mail@maciej.szmigiero.name> wrote:
>>
>> From: "Maciej S. Szmigiero" <maciej.szmigiero@oracle.com>
>>
>> Since KVM commit 5f76f6f5ff96 ("KVM: nVMX: Do not expose MPX VMX controls when guest MPX disabled")
>> it is not possible to disable MPX on a "-cpu host" just by adding "-mpx"
>> there if the host CPU does indeed support MPX.
>> QEMU will fail to set MSR_IA32_VMX_TRUE_{EXIT,ENTRY}_CTLS MSRs in this case
>> and so trigger an assertion failure.
>>
>> Instead, besides "-mpx" one has to explicitly add also
>> "-vmx-exit-clear-bndcfgs" and "-vmx-entry-load-bndcfgs" to QEMU command
>> line to make it work, which is a bit convoluted.
>>
>> Sanitize MPX-related bits in MSR_IA32_VMX_TRUE_{EXIT,ENTRY}_CTLS after
>> setting the vCPU CPUID instead so such workarounds are no longer necessary.
> 
> Can you use feature_dependencies instead? See for example
> 
>      {
>          .from = { FEAT_7_0_EBX,             CPUID_7_0_EBX_RDSEED },
>          .to = { FEAT_VMX_SECONDARY_CTLS,    VMX_SECONDARY_EXEC_RDSEED_EXITING },
>      },

The "feature_dependencies" way seems to work fine too,
and it definitely looks neater.

Thanks for the pointer, will post v2 in a moment.

> Paolo
> 

Thanks,
Maciej