Re: [Xen-devel] "X86_PV_VCPU_MSRS record truncated" during domain restore

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 21/07/2016 00:20, Marek Marczykowski-Górecki wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On Wed, Jul 20, 2016 at 02:33:20PM +0200, Massimo Colombi wrote:
>> I retried (it's not the first time) to regenerate new savefile, but DispVM
>> doesn't work.
>> I attach the results.
> For me it looks like a bug in savefile handling. Moving to xen-devel. Any
> idea?
> Background info: it's about restoring a domain through libvirt->libxl
> (virsh restore equivalent). Xen 4.6.1. Full error:
>
>> 2016-07-20 14:23:01 CEST xc: error: X86_PV_VCPU_MSRS record truncated: 
>> length 8, min 9: Internal error
>> 2016-07-20 14:23:01 CEST xc: error: Restore failed (0 = Success): Internal 
>> error

[EDIT] While writing the explanation below, I have spotted the bug.

The restore side found a record in the stream of type X86_PV_VCPU_MSRS
and 8 bytes long.

The X86_PV_VCPU_MSRS record is an 8 byte header, followed by 1 or more
bytes which are treated as an opaque blob, and handed back to Xen.  As
such, 8 bytes is insufficient to contain a nonzero sized payload.

Is it possible to do an `xl save` equivalent on the domain, and run
tools/python/scripts/verify-stream-v2 against the resulting file?  That
should identify whether it is a malformed X86_PV_VCPU_MSRS record but
otherwise intact stream, or whether the stream becomes corrupted elsewhere?

If not, another line if inquiry would be to instrument
tools/libxc/xc_sr_save_x86_pv.c write_one_vcpu_msrs() to identify what
XEN_DOMCTL_get_vcpu_msrs is returning on the source side (although this
function does deliberately check for a zero-sized payload and omit the
record).

The hypervisor side implementation is in xen/arch/x86/domctl.c at the
XEN_DOMCTL_get_vcpu_msrs case label.

Re-reading the write_one_vcpu_msrs(), there is an error.  We first query
Xen for the maximum number of MSRs it might potentially give to us,
which will return 4 on this hardware.  We then actually ask for the MSR
content, but Xen only writes non-zero MSRs into payload.

A domain which isn't using Debug Extensions will return 4 for the first
query (maximum number of MSRs), then 0 for the second query (actual MSR
content), and write a X86_PV_VCPU_MSRS record with 0 payload into the
stream, which the receiving side is objecting to.

I will make a patch series tomorrow to address this issue.  I think
there are similar issues for other records as well.

~Andrew

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel