Re: [Xen-devel] Future support of 5-level paging in Xen:wq

[Stefano Stabellini <stefano@xxxxxxxxxxx>]

On Fri, 9 Dec 2016, Juergen Gross wrote:
> On 09/12/16 00:50, Stefano Stabellini wrote:
> > On Thu, 8 Dec 2016, Andrew Cooper wrote:
> >> On 08/12/2016 19:18, Stefano Stabellini wrote:
> >>> On Thu, 8 Dec 2016, Andrew Cooper wrote:
> >>>> On 08/12/16 16:46, Juergen Gross wrote:
> >>>>> The first round of (very preliminary) patches for supporting the new
> >>>>> 5-level paging of future Intel x86 processors [1] has been posted to
> >>>>> lkml:
> >>>>>
> >>>>> https://lkml.org/lkml/2016/12/8/378
> >>>>>
> >>>>> An explicit note has been added: "CONFIG_XEN is broken." and
> >>>>> "I would appreciate help with the code."
> >>>>>
> >>>>> I think we should start a discussion what we want to do in future:
> >>>>>
> >>>>> - are we going to support 5-level paging for PV guests?
> >>>>> - do we limit 5-level paging to PVH and HVM?
> >>>> The 64bit PV ABI has 16TB of virtual address space just above the upper
> >>>> 48-canonical boundary.
> >>>>
> >>>> Were Xen to support 5-level PV guests, we'd either leave the PV guest
> >>>> kernel with exactly the same amount of higher half space as it currently
> >>>> has, or we'd have to recompile Xen as properly position-independent and
> >>>> use a different virtual range in different paging mode.
> >>>>
> >>>> Another pain point is the quantity of virtual address space handed away
> >>>> in the ABI.  We currently had 97% of the virtual address space away to
> >>>> 64bit PV guests, and frankly this is too much.  This is the wrong way
> >>>> around when Xen has more management to do than the guest.  If we were to
> >>>> go along the 5-level PV guests route, I will insist that there is a
> >>>> rather more even split of virtual address space baked into the ABI.
> >>>>
> >>>> However, a big question is whether any of this effort is worth doing, in
> >>>> the light of PVH.
> >>> With my Aporeto hat on, I'll say that given the overwhelming amount of
> >>> hardware available out there without virtualization support, this work
> >>> is worth doing. I am referring to all the public cloud virtual machines,
> >>> which can support Xen PV guests but cannot support PVH guests.
> >>
> >> Why is Xen supporting 5-level guests useful for running in a PV cloud
> >> VM?  Xen doesn't run PV.
> >>
> >> I am not suggesting that we avoid adding 5-level support to Xen.  We
> >> should absolutely do that.  The question is only whether we extend the
> >> PV ABI to support 5-level PV guests.  Conceptually, its very easy to
> >> have a 5-level Xen but only supporting 4-level PV guests.
> >>
> >> VT-x and SVM date from 2005/2006 and are now 10 years old.  I would be
> >> surprised if you would find much hardware of this age in any cloud; you
> >> can't by anything that old, and support contracts have probably run out
> >> if you have owned that hardware for 10 years.
> > 
> > I am thinking that in a couple of years, we might already find VMs so
> > large that to use all the memory in a nested virt scenario, we need
> > 5-level PV abi support.
> > 
> 
> No, I don't think so. I believe there will be no hardware capable of
> 5-level paging but without VMX/SVM support. Support of PVH/HVM for such
> large guests should be enough. We don't need to extend PV which we want
> to get rid of in Linux anyway, no?

I am talking about nested virtualization when the L1 virtual machine
does not support nested VMX or SVM. No Amazon AWS virtual machines
support nested VMX, but it is possible to run Xen PV virtual machines on
top of any Amazon HVM instance.

When 5-level pagetable hardware will become available on Amazon AWS, it
might be possible to get virtual machines so large that in order to use
all the memory, you need to use 5-level pagetables in L1 Xen. In this
scenario, if we want to create a L2 virtual machine as large as possible
we will need support for 5-level page tables in the PV ABI.

Please correct me if I am wrong.


P.S.
I used the following terminology:

L0 Xen is the one running on the hardware
L1 virtual machine, is a VM created by L0 Xen. In this context this is
   an Amazon AWS HVM instance without nested VMX support.
L1 Xen is the one installed inside an L1 virtual machine
L2 virtual machine, is a VM created by L1 Xen

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel