Re: [Xen-devel] [PATCH v6 1/2] docs/designs: Add a design document for non-cooperative live migration

[Julien Grall <julien@xxxxxxx>]

Hi Paul,

On 05/03/2020 17:30, pdurrant@xxxxxxxx wrote:
> From: Paul Durrant <pdurrant@xxxxxxxxxx>
>
> It has become apparent to some large cloud providers that the current
> model of cooperative migration of guests under Xen is not usable as it
> relies on software running inside the guest, which is likely beyond the
> provider's control.
> This patch introduces a proposal for non-cooperative live migration,
> designed not to rely on any guest-side software.
>
> Signed-off-by: Paul Durrant <pdurrant@xxxxxxxxxx>
> ---
> Cc: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
> Cc: George Dunlap <George.Dunlap@xxxxxxxxxxxxx>
> Cc: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
> Cc: Jan Beulich <jbeulich@xxxxxxxx>
> Cc: Julien Grall <julien@xxxxxxx>
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
> Cc: Stefano Stabellini <sstabellini@xxxxxxxxxx>
> Cc: Wei Liu <wl@xxxxxxx>
>
> v6:
>   - Addressed comments from Julien
>
> v5:
>   - Note that PV domain are not just expected to co-operate, they are
>     required to
>
> v4:
>   - Fix issues raised by Wei
>
> v2:
>   - Use the term 'non-cooperative' instead of 'transparent'
>   - Replace 'trust in' with 'reliance on' when referring to guest-side
>     software
> ---
>   docs/designs/non-cooperative-migration.md | 280 ++++++++++++++++++++++
>   1 file changed, 280 insertions(+)
>   create mode 100644 docs/designs/non-cooperative-migration.md
>
> diff --git a/docs/designs/non-cooperative-migration.md 
> b/docs/designs/non-cooperative-migration.md
> new file mode 100644
> index 0000000000..e28161562f
> --- /dev/null
> +++ b/docs/designs/non-cooperative-migration.md
> @@ -0,0 +1,280 @@
> +# Non-Cooperative Migration of Guests on Xen
> +
> +## Background
> +
> +The normal model of migration in Xen is driven by the guest because it was
> +originally implemented for PV guests, where the guest must be aware it is
> +running under Xen and is hence expected to co-operate. This model dates from
> +an era when it was assumed that the host administrator had control of at
> +least the privileged software running in the guest (i.e. the guest kernel)
> +which may still be true in an enterprise deployment but is not generally
> +true in a cloud environment. The aim of this design is to provide a model
> +which is purely host driven, requiring no co-operation from the software
> +running in the guest, and is thus suitable for cloud scenarios.
> +
> +PV guests are out of scope for this project because, as is outlined above,
> +they have a symbiotic relationship with the hypervisor and therefore a
> +certain level of co-operation is required.
> +
> +HVM guests can already be migrated on Xen without guest co-operation but
> +only if they don’t have PV drivers installed[1] or are not in power state
> +S0. The reason for not expecting co-operation if the guest is any sort of
Sorry to be picky, but this still not clear this is x86/ACPI specific.

How about "x86 HVM guests can already be migrated on Xen without 
guest-co-operation but only if they don't have PV drivers installed[1] 
or are not in ACPI power state S0".
[...]

> +For a PV guest the `arch_shared_info` sub-structure contains important
> +information about the guest’s P2M, but this information is not relevant for
> +an HVM guest where the P2M is not directly manipulated via the guest. The
> +other state contained in the `shared_info` structure relates the domain
> +wall-clock (the state of which should already be transferred by the `RTC`
> +HVM context information which contained in the `HVM_CONTEXT` save record)
> +and some event channel state (particularly if using the *2l* protocol).
> +Event channel state will need to be fully transferred if we are not going
> +to require the guest co-operation to re-open the channels and so it should
> +be possible to re-build a shared info page for an HVM guest from such other
> +state.
> +
> +Note that the shared info page also contains an array of
> +`XEN_LEGACY_MAX_VCPUS` (32) `vcpu_info` structures. A domain may nominate
I would say (32 for x86)

> +a different guest physical address to use for the vcpu info. Fix x86
Do you mean 'For' instead of 'Fix'.

> +domains this is mandatory if a domain wants to use more than 32 vCPUs
> +and optional otherwise. This mapping is not currently transferred in the
I would make this sentence arch-agnostic. e.g: "This is mandatory if a 
domain wants to use more than `XEN_LEGACY_MAX_VCPUS` and optional 
otherwise".
> +migration state so this will either need to be added into an existing save
> +record, or an additional type of save record will be needed.
> +
Cheers,

--
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel