Re: staging: unable to restore HVM with Viridian param set

[Igor Druzhinin <igor.druzhinin@xxxxxxxxxx>]

On 02/02/2021 08:35, Paul Durrant wrote:
>> -----Original Message-----
>> From: Igor Druzhinin <igor.druzhinin@xxxxxxxxxx>
>> Sent: 02 February 2021 00:20
>> To: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>; Tamas K Lengyel 
>> <tamas.k.lengyel@xxxxxxxxx>; Xen-devel
>> <xen-devel@xxxxxxxxxxxxxxxxxxxx>; Wei Liu <wl@xxxxxxx>; Ian Jackson 
>> <iwj@xxxxxxxxxxxxxx>; Anthony
>> PERARD <anthony.perard@xxxxxxxxxx>; Paul Durrant <paul@xxxxxxx>
>> Subject: Re: staging: unable to restore HVM with Viridian param set
>>
>> n 01/02/2021 22:57, Andrew Cooper wrote:
>>> On 01/02/2021 22:51, Tamas K Lengyel wrote:
>>>> Hi all,
>>>> trying to restore a Windows VM saved on Xen 4.14 with Xen staging results 
>>>> in:
>>>>
>>>> # xl restore -p /shared/cfg/windows10.save
>>>> Loading new save file /shared/cfg/windows10.save (new xl fmt info 
>>>> 0x3/0x0/1475)
>>>>  Savefile contains xl domain config in JSON format
>>>> Parsing config from <saved>
>>>> xc: info: Found x86 HVM domain from Xen 4.14
>>>> xc: info: Restoring domain
>>>> xc: error: set HVM param 9 = 0x0000000000000065 (17 = File exists):
>>>> Internal error
>>>> xc: error: Restore failed (17 = File exists): Internal error
>>>> libxl: error: libxl_stream_read.c:850:libxl__xc_domain_restore_done:
>>>> restoring domain: File exists
>>>> libxl: error: libxl_create.c:1581:domcreate_rebuild_done: Domain
>>>> 8:cannot (re-)build domain: -3
>>>> libxl: error: libxl_domain.c:1182:libxl__destroy_domid: Domain
>>>> 8:Non-existant domain
>>>> libxl: error: libxl_domain.c:1136:domain_destroy_callback: Domain
>>>> 8:Unable to destroy guest
>>>> libxl: error: libxl_domain.c:1063:domain_destroy_cb: Domain
>>>> 8:Destruction of domain failed
>>>>
>>>> Running on staging 419cd07895891c6642f29085aee07be72413f08c
>>>
>>> CC'ing maintainers and those who've edited the code recently.
>>>
>>> What is happening is xl/libxl is selecting some viridian settings,
>>> applying them to the domain, and then the migrations stream has a
>>> different set of viridian settings.
>>>
>>> For a migrating-in VM, nothing should be set during domain build.
>>> Viridian state has been part of the migrate stream since before mig-v2,
>>> so can be considered to be everywhere relevant now.
>>
>> The fallout is likely from my changes that modified default set of viridian
>> settings. The relevant commits:
>> 983524671031fcfdb24a6c0da988203ebb47aebe
>> 7e5cffcd1e9300cab46a1816b5eb676caaeed2c1
>>
>> The same config from migrated domains now implies different set of viridian
>> extensions then those set at the source side. That creates inconsistency in
>> libxl. I don't really know how to address it properly in libxl other than
>> don't extend the default set ever.
>>
> 
> Surely it should be addressed properly in libxl by not messing with the 
> viridian settings on migrate-in/resume, as Andrew says? TBH I thought this 
> was already the case. There should be no problem with adding to the default 
> set as this is just an xl/libxl concept; the param flags in Xen are always 
> define the *exact* set of enabled enlightenments.

If maintainers agree with this approach I can make a patch.

Igor