Re: [Xen-devel] Xen 4.6.1 crash with altp2m enabledbydefault

[<Kevin.Mayer@xxxxxxxx>]

Hi

I took another look at Xen and a new crashdump.
The last successful __vmwrite should be in 
static void vmx_vcpu_update_vmfunc_ve(struct vcpu *v)
[...]
    __vmwrite(SECONDARY_VM_EXEC_CONTROL,
              v->arch.hvm_vmx.secondary_exec_control);
[...]
After this the altp2m_vcpu_destroy wakes up the vcpu and is then finished.

In nestedhvm_vcpu_destroy (nvmx_vcpu_destroy) the vmcs can overwritten (but is 
not reached in our case as far as I can see):
    if ( nvcpu->nv_n1vmcx )
        v->arch.hvm_vmx.vmcs = nvcpu->nv_n1vmcx;

In conclusion:
When destroying a domain the altp2m_vcpu_destroy(v); path seems to mess up the 
vmcs which ( only ) sometimes leads to a failed __vmwrite in vmx_fpu_leave.
That is as far as I can get with my understanding of the Xen code.

Do you guys have any additional ideas what I could test / analyse?

> -----Ursprüngliche Nachricht-----
> Von: Jan Beulich [mailto:JBeulich@xxxxxxxx]
> Gesendet: Montag, 8. August 2016 12:29
> An: Mayer, Kevin <Kevin.Mayer@xxxxxxxx>
> Cc: andrew.cooper3@xxxxxxxxxx; xen-devel@xxxxxxxxxxxxx
> Betreff: Re: [Xen-devel] Xen 4.6.1 crash with altp2m enabledbydefault
> 
> >>> On 08.08.16 at 11:48, <Kevin.Mayer@xxxxxxxx> wrote:
> > vmx_vmenter_helper is not part of the call stack. The address is
> > simply the location of the ud2 to which the __vmwrite(HOST_CR0,
> > v->arch.hvm_vmx.host_cr0); In static void vmx_fpu_leave(struct vcpu
> > *v) jumps.
> > There are two vmwrites in vmx_vcpu_update_eptp (called by
> > altp2m_vcpu_destroy):
> > __vmwrite(EPT_POINTER, ept_get_eptp(ept)); __vmwrite(EPTP_INDEX,
> > vcpu_altp2m(v).p2midx);
> >
> > And four in vmx_vcpu_update_vmfunc_ve (also called by
> > altp2m_vcpu_destroy) __vmwrite(VM_FUNCTION_CONTROL,
> > VMX_VMFUNC_EPTP_SWITCHING); __vmwrite(EPTP_LIST_ADDR,
> > virt_to_maddr(d->arch.altp2m_eptp));
> > __vmwrite(VIRT_EXCEPTION_INFO, mfn_x(mfn) << PAGE_SHIFT);
> > __vmwrite(SECONDARY_VM_EXEC_CONTROL,
> > v->arch.hvm_vmx.secondary_exec_control);
> >
> > After the altp2m-part hvm_vcpu_destroy also calls
> > nestedhvm_vcpu_destroy(v), but this code path is executed
> > unconditionally so I assume that the error lies somewhere in the
> altp2m_vcpu_destroy(v).
> >
> > What exactly are the vmx_vmcs_enter / exit required for? I often see
> > the vmx_vmcs_enter; __vmwrite; vmx_vmcs_exit combination. Need the
> > __vmwrites be guarded by an enter / exit ( which Is not the case in
> > the static void vmx_fpu_leave(struct vcpu *v) )?
> 
> On code paths where the correct VMCS may not be the current one it is
> necessary to frame vmread / vmwrite accordingly.
> 
> > Is it possible that the
> > altp2m_vcpu_destroy->vmx_vcpu_update_eptp->vmx_vmcs_exit-
> >vmx_clear_vm
> > cs invalidates the vmcs for the current vcpu?
> 
> I certainly can't exclude this possibility.
> 
> Jan
____________
Virus checked by G Data MailSecurity
Version: AVA 25.7943 dated 19.08.2016
Virus news: www.antiviruslab.com

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel