Re: [Xen-devel] [RFC 0/5] xen/arm: support big.little SoC

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Tue, 20 Sep 2016, Dario Faggioli wrote:
> On Mon, 2016-09-19 at 14:03 -0700, Stefano Stabellini wrote:
> > On Mon, 19 Sep 2016, Dario Faggioli wrote:
> > > Setting thing up like this, even automatically, either in
> > hypervisor or
> > > toolstack, is basically already possible (with all the good and bad
> > > aspects of pinning, of course).
> > > 
> > > Then, sure (as I said when replying to George), we may want things
> > to
> > > be more flexible, and we also probably want to be on the safe side
> > --if 
> > > ever some components manages to undo our automatic pinning-- wrt
> > the
> > > scheduler not picking up work for the wrong architecture... But
> > still
> > > I'm a bit surprised this did not came up... Julien, Peng, is that
> > > because you think this is not doable for any reason I'm missing?
> > 
> > Let's suppose that Xen detects big.LITTLE and pins dom0 vcpus to big
> > automatically. How can the user know that she really needs to be
> > careful
> > in the way she pins the vcpus of new VMs? Xen would also need to pin
> > automatically vcpus of new VMs to either big or LITTLE cores, or xl
> > would have to do it.
> > 
> Actually doing things with what we currently have for pinning is only
> something I've brought up as an example, and (potentially) useful for
> proof-of-concept, or very early stage level support.
> 
> In the long run, when thinking to the scheduler based solution, I see
> things happening the other way round: you specify in xl config file
> (and with boot parameters, for dom0) how many big and how many LITTLE
> vcpus you want, and the scheduler will know that it can only schedule
> the big ones on big physical cores, and the LITTLE ones on LITTLE
> physical cores.
> 
> Note that we're saying 'pinning' (yeah, I know, I did it myself in the
> first place :-/), but that would not be an actual 1-to-1 pinning. For
> instance, if domain X has 4 big pcpus, say 0,1,2,3,4, and the host has
> 8 big pcpus, say 8-15, then dXv1, dXv2, dXv3 and dXv4 will only be run
> by the scheduler on pcpus 8-15. Any of them, and with migration and
> load balancing within the set possible. This is what I'm talking about.
> 
> And this would work even if/when there is only one cpupool, or in
> general for domains that are in a pool that has both big and LITTLE
> pcpus. Furthermore, big.LITTLE support and cpupools will be orthogonal,
> just like pinning and cpupools are orthogonal right now. I.e., once we
> will have what I described above, nothing prevents us from implementing
> per-vcpu cpupool membership, and either create the two (or more!) big
> and LITTLE pools, or from mixing things even more, for more complex and
> specific use cases. :-)

I think that everybody agrees that this is the best long term solution.


> Actually, with the cpupool solution, if you want a guest (or dom0) to
> actually have both big and LITTLE vcpus, you necessarily have to
> implement per-vcpu (rather than per-domain, as it is now) cpupool
> membership. I said myself it's not impossible, but certainly it's some
> work... with the scheduler solution you basically get that for free!
> 
> So, basically, if we use cpupools for the basics of big.LITTLE support,
> there's no way out of it (apart from going implementing scheduling
> support afterwords, but that looks backwards to me, especially when
> thinking at it with the code in mind).

The question is: what is the best short-term solution we can ask Peng to
implement that allows Xen to run on big.LITTLE systems today? Possibly
getting us closer to the long term solution, or at least not farther
from it?


> > The whole process would be more explicit and obvious if we used
> > cpupools. It would be easier for users to know what it is going on --
> > they just need to issue an `xl cpupool-list' command and they would
> > see
> > two clearly named pools (something like big-pool and LITTLE-pool). 
> >
> Well, I guess that, as part of big.LITTLE support, there will be a way
> to tell what pcpus are big and which are LITTLE anyway, probably both
> from `xl info' and from `xl cpupool-list -c' (and most likely in other
> ways too).

Sure, but it needs to be very clear. We cannot ask people to spot
architecture specific flags among the output of `xl info' to be able to
appropriately start a guest. Even what I suggested isn't great as `xl
cpupool-list' isn't a common command to run.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel