Re: [Xen-devel] [PATCH 4/4] xen/arm: update the docs about heterogeneous computing

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Fri, 16 Feb 2018, Julien Grall wrote:
> On 16/02/2018 21:15, Stefano Stabellini wrote:
> > On Fri, 16 Feb 2018, Julien Grall wrote:
> > > On 16/02/2018 20:50, Stefano Stabellini wrote:
> > > > On Fri, 16 Feb 2018, Julien Grall wrote:
> > > > > Hi Stefano,
> > > > > 
> > > > > On 15/02/18 23:17, Stefano Stabellini wrote:
> > > > > > Update the documentation of the hmp-unsafe option to explain how to
> > > > > > use
> > > > > > it safely, together with the right cpu affinity setting, on
> > > > > > big.LITTLE
> > > > > > systems.
> > > > > > 
> > > > > > Also update the warning message to point users to the docs.
> > > > > > 
> > > > > > Signed-off-by: Stefano Stabellini <sstabellini@xxxxxxxxxx>
> > > > > > CC: jbeulich@xxxxxxxx
> > > > > > CC: konrad.wilk@xxxxxxxxxx
> > > > > > CC: tim@xxxxxxx
> > > > > > CC: wei.liu2@xxxxxxxxxx
> > > > > > CC: andrew.cooper3@xxxxxxxxxx
> > > > > > CC: George.Dunlap@xxxxxxxxxxxxx
> > > > > > CC: ian.jackson@xxxxxxxxxxxxx
> > > > > > 
> > > > > > ---
> > > > > >     docs/misc/xen-command-line.markdown | 10 +++++++++-
> > > > > >     xen/arch/arm/smpboot.c              |  9 +++++----
> > > > > >     2 files changed, 14 insertions(+), 5 deletions(-)
> > > > > > 
> > > > > > diff --git a/docs/misc/xen-command-line.markdown
> > > > > > b/docs/misc/xen-command-line.markdown
> > > > > > index 2184cb9..a1ebeea 100644
> > > > > > --- a/docs/misc/xen-command-line.markdown
> > > > > > +++ b/docs/misc/xen-command-line.markdown
> > > > > > @@ -1007,7 +1007,15 @@ Control Xens use of the APEI Hardware Error
> > > > > > Source
> > > > > > Table, should one be found.
> > > > > >       Say yes at your own risk if you want to enable heterogenous
> > > > > > computing
> > > > > >     (such as big.LITTLE). This may result to an unstable and
> > > > > > insecure
> > > > > > -platform. When the option is disabled (default), CPUs that are not
> > > > > > +platform, unless you manually specify the cpu affinity of all
> > > > > > domains
> > > > > > so
> > > > > > +that all vcpus are scheduled on the same class of pcpus (big or
> > > > > > LITTLE
> > > > > > +but not both). vcpu migration between big cores and LITTLE cores is
> > > > > > not
> > > > > > +supported. Thus, if the first 4 pcpus are big and the last 4 are
> > > > > > LITTLE,
> > > > > > +all domains need to have either cpus = "0-3" or cpus = "4-7" in
> > > > > > their
> > > > > > VM
> > > > > > +config. Moreover, dom0_vcpus_pin needs to be passed on the Xen
> > > > > > command
> > > > > > +line.
> > > > > 
> > > > > In your example here you suggest to have all the vCPUs of a guest to
> > > > > either on
> > > > > big or LITTLE cores. How about giving an example where the guest can
> > > > > have
> > > > > 2
> > > > > LITTLE vCPUs and one big vCPU?
> > > > 
> > > > I would rather discourage it at the moment, given that it requires more
> > > > complex cpu affinity settings, or vcpu pinning. Also, I am afraid that
> > > > without matching corresponding topology information on the guest device
> > > > tree, guests might not work as expected in such a scenario.
> > > > 
> > > > What do you think?
> > > 
> > > You already know my view on this. I would rather strongly discourage
> > > anyone
> > > pinning all vCPUs of a domain to big cores. We should avoid to provide
> > > shortcuts to use that could have potentially damageable impact on their
> > > platform without telling them.
> > 
> > Do you have a link to a doc somewhere that provides more details about
> > this? We could add a link to it here to inform users. It would be
> > useful.
> 
> This is quite well described in
> https://xenbits.xen.org/docs/unstable/man/xl.cfg.5.html#CPU-Allocation see
> "cpus".

OK, I'll add the link in a new big.LITTLE doc. Also, do you have any
documentation or link about big core being potentially damaging? It
would be good to provide information about that too in the big.LITTLE
doc.


> > > > I see. What about:
> > > > 
> > > >     See docs/misc/xen-command-line.markdown:hmp-unsafe
> > > > 
> > > > Or:
> > > > 
> > > >     See hmp-unsafe under
> > > > https://xenbits.xen.org/docs/unstable/misc/xen-command-line.html
> > > 
> > > I believe that people looking at big.LITTLE (and really want it) are smart
> > > enough to look at the docs or the code themselves. Given how fragile is
> > > your
> > > solution, I would rather avoid to help people doing bad thing.
> > 
> > People might know very well the hardware, and have very detailed
> > information about big.LITTLE and their platform, but might not know that
> > much about Xen.
> 
> We also might have student who wants to try Xen on their platform and have no
> clue how it works. So we want to provide safe information here and avoid to
> induce pinning on big cores is always safe.
> 
> I think with the tools we have, it is not very difficult provide a quick tuto
> how to do big.LITTLE in Xen. Obviously this will not be as easy as the
> solution suggested by Dario in its design document.
> 
> The most important bits is libxl provide an extensive way to set CPUs
> affinity. OS such as Linux will be able to deal with big.LITTLE even without
> DT thought scheduling might not be that good... I have no idea how Android
> detects big.LITTLE and will leave that to person who knows more.

Sure, I'll write a new doc.


> > I would like to introduce a tie between the warning message and the
> > documentation provided. The tie doesn't have to be around hmp-unsafe:
> > the intention wasn't really to provide a shortcut to do more damage, but
> > rather to inform about the situation. In fact, I didn't mean to
> > recommend the usage of hmp-unsafe.
> 
> In that case we should make clear hmp-unsafe is not recommended. When I read
> the suggested paragraph for the command line option, it feels that it is safe
> to use it if you pin your domain on big cores only.

I'll make it clearer that hmp-unsafe is not recommended. I'll completely
separate the big.LITTLE information doc from hmp-unsafe.


> > Maybe we could say:
> > 
> >    See big.LITTLE under
> > https://xenbits.xen.org/docs/unstable/misc/xen-command-line.html
> > 
> > or create docs/mics/arm/big.LITTLE ?
> 
> A doc would make more sense over a long paragraph in the xen-command-line.
> 
> But if we decide to no recommend the usage of hmp-unsafe (as you suggested
> above), then we should avoid to tease the user with that. Anyone caring enough
> about big.LITTLE could find easily a documentation to enable it on Xen.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel