Re: [Xen-devel] [DRAFT C] PVH CPU hotplug design document

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

Hello Al,

Thanks for your comments, please see below.

On Mon, Feb 06, 2017 at 04:06:45PM -0700, Al Stone wrote:
> On 01/24/2017 07:20 AM, Boris Ostrovsky wrote:
> > 
> >> Yes, the only remaining risk is some vendor using _SB.XEN0, and AFAICT 
> >> there's
> >> no way to reserve anything in there (mostly because it's assumed that ACPI
> >> tables will be created by a single entity I guess).
> >>
> >> I think that the chance of this happening is 0%, and that there's no single
> >> system out there with a _SB.XEN0 node. I've been wondering whether I 
> >> should try
> >> to post this to the ACPI working group, and try to get some feedback there.
> > 
> > If you end up asking there, I'd suggest including Rafael Wysocki and Len
> > Brown (rafael@xxxxxxxxxx and lenb@xxxxxxxxxx) and maybe 
> > linux-acpi@xxxxxxxxxxxxxxx as well.
> > 
> > -boris
> > 
> 
> My apologies for not leaping into this discussion earlier; real life has been
> somewhat complicated lately.  Hopefully I won't annoy too many people.
> 
> So, I am on the ASWG (ACPI Spec Working Group) as a Red Hat and/or Linaro
> representative.  To clarify something mentioned quite some time ago, the STAO
> and XENV tables are in the ACPI in a special form.  Essentially, there are two
> classes of tables within ACPI: official tables defined in the spec itself that
> are meant to be used anywhere ACPI is used, and, tables whose names are to be
> recognized but whose content is defined elsewhere.  The STAO and XENV belong
> to this second class -- the spec reserved their signatures so that others do
> not use them, but then points to an external source -- Xen, specifically -- 
> for
> the definition.  The practical implication is that Xen can change definitions
> as they wish, without direct oversight of the ASWG.  Just the same, it is
> considered bad form to do so, however, so new revisions should at least be 
> sent
> to the ASWG for discussion (it may make sense to pull the table into the spec
> itself...).  Stefano and I worked together to get the original reservation 
> made
> for the STAO and XENV tables.
> 
> The other thing I've noticed so far in the discussion is that everything
> discussed may work on x86 or ia64, but will not work at all on arm64.  The
> HARDWARE_REDUCED flag in the FADT was mentioned -- this is the crux of the
> problem.  For arm64, that flag is required to be set, so overloading it is 
> most
> definitely an issue.  More problematic, however, is the notion of using GPE
> blocks; when the HARDWARE_REDUCED flag is set, the spec requires GPE block
> definitions are to be ignored.

Yes, this document is specific to x86. I believe that the difference between
x86 and ARM regarding ACPI would make it too complicated to come up with a
solution that's usable on both, mainly because ACPI tables on ARM and x86 are
already too different.

> Then it gets messy :).  The APIC and/or x2APIC subtables of the MADT are not
> likely to exist on arm64; chances are just about zero, actually.  There are
> other similar MADT subtables for arm64, but APIC, x2APIC and many more just
> won't be there.  There is some overlap with ia64, but not entirely.

ia64 is also out of the picture here, the more that Xen doesn't support it, and
it doesn't look like anyone is working on it.

> The other issue is that a separate name space for the added CPUs would have
> to be very carefully done.  If not, then the processor hierarchy information
> in the AML either becomes useless, or at the least inconsistent, and OSPMs
> are just now beginning to use some of that info to make scheduling decisions.
> It would be possible to just assume the hot plug CPUs are outside of any
> existing processor hierarchy, but I would then worry that power management
> decisions made by the OSPM might be wrong; I can imagine a scenario where
> a CPU is inserted and shares a power rail with an existing CPU, but the
> existing CPU is idle so it decides to power off since it's the last in the
> hierarchy, so the power rail isn't needed, and now the power gets turned off
> to the unit just plugged in because the OSPM doesn't realize it shares power.

Well, my suggestion was to add the processor objects of the virtual CPUs inside
an ACPI Module Device that has the _SB.XEN0 namespace. However, AFAIK there's
no way to reserve the _SB.XEN0 namespace, so a vendor could use that for
something else. I think the chances of that happening are very low, but it's
not impossible.

Is there anyway in ACPI to reserve a namespace for a certain usage? (ie: would
it be possible to somehow reserve _SB.XEN0 for Xen usage?)

Or if we want to go more generic, we could reserve _SB.VIRT for generic
hypervisor usage.

> So at a minimum, it sounds like there would need to be a solution for each
> architecture, with maybe some fiddling around on ia64, too.  Unfortunately,
> I believe the ACPI spec provides a way to handle all of the things wanted,
> but an ASL interpreter would be required because it does rely on executing
> methods (e.g., _CRS to determine processor resources on hot plug).  The ACPICA
> code is dual-licensed, GPL and commercial, and there is the OpenBSD code.
> But without an interpreter, it feels like we're trying to push dynamic
> behavior into static tables, and they really weren't designed for that.

Yes, I think an arch-specific solution is needed in this case. Currently Dom0
passes all this information to Xen using hypercalls, but I don't think an AML
parser in Xen is strictly needed in order to implement the solution that I'm
proposing. We can get the ACPI processor object IDs from the MADT, and that
could be used in the STAO to hide them from Dom0 (provided that the STAO is
modified to add a new field, as described in the design document).

I'm also a member of the ACPI working group, and I was planning to send this
design document there for further discussion, just haven't found the time yet
to write a proper mail :(.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel