Re: [Xen-devel] (v2) Design proposal for RMRR fix

["Tian, Kevin" <kevin.tian@xxxxxxxxx>]

> From: Jan Beulich [mailto:JBeulich@xxxxxxxx]
> Sent: Thursday, January 15, 2015 4:38 PM
> 
> >>> On 14.01.15 at 19:29, <george.dunlap@xxxxxxxxxxxxx> wrote:
> > Just to be clear -- what we're talking about here is that at the
> > do_domain_create() level (called by libxl_domain_create_new()), it will
> > take a list of pci devices, and the rmrr list above (including "host"
> > and individual ranges), and generate a list of RMRRs to pass to the
> > lower layer.  The lower layer will simply see the range, and a "force /
> > no force" flag, and behave appropriately.  The determination of which
> > RMRRs to force will be done at the domain_create level.
> >
> > Is that about right?
> 
> That's certainly a sensible model.
> 

It's really a mess for my outlook to sort multiple threads under same
topic... so I'll reply to this one after reading through all previous good
discussions.

First, sorry that I used '0/1' as a bad example for user options, and thanks
for your suggestion on a right way defining them.

I also agree above model. Policy is all decided at domain_create while
lower layer only reacts to specified regions which have individual policy
to indicate 'force' or 'not force'. 

We'll need to make that skeleton ready first. Then regarding to config 
interface, how about making some simplification by only considering
statically-assigned device and hotplug now (leaving migration to future 
based on necessity, and extending that doesn't change low level skeleton)?

pci option can be extended as:
        pci = [ 'bdf, rmrr_check=force/try' ]
domain_create queries Xen hypervisor about RMRRs associated with
assigned devices, and then mark each region with user specified override. 

User can also specify a rmrr_host option as:
        rmrr = [ 'host, check=force/try' ]
when rmrr_host is specified, domain_create queries all RMRRs reported
in this platform, and mark per-region policy accordingly. 

per-device override is always favored if a conflicting setting in rmrr_host.
 
The composed reserved region list is then passed to domain builder,
which tries to detect and avoid conflicts when populating guest RAM.
To avoid breaking lowmem/highmem layout, we can define a 
lowmem_guard so if making hole for a region would make lowmem_top
below lowmem_guard we'll treat this region as a conflict. We may
either just hardcode the value like 2G (or other reasonable value in your
mind), or allow user to config e.g.:
        rmrr = [ 'host, check=force/try', 'lowmem_boundary=2G' ]

after domain builder, libxl will request actual device assignment to
Xen hypervisor. At that point, current assignment hypercall needs
to be extended to include the override value, so Xen will handle
conflict accordingly when setting up identity map.

last step is in hvmloader, which is passed with all the necessary RMRR
information, and then handles potential conflicts accordingly.

In the future when we think necessary to allow user specify random
regions for migration usage, it can be easily extended and we can
argue whether to introduce same override.

If above high level flow can be agreed, then we can move forward to
discuss next level detail e.g. how to pass the rmrr list cross different
components. :-)

Thanks
Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel