Re: [Xen-devel] [RFC 0/5] xen/arm: support big.little SoC

[George Dunlap <George.Dunlap@xxxxxxxxxxxxx>]

On Tue, Sep 20, 2016 at 11:03 AM, Peng Fan <van.freenix@xxxxxxxxx> wrote:
> Hi Dario,
> On Tue, Sep 20, 2016 at 02:54:06AM +0200, Dario Faggioli wrote:
>>On Mon, 2016-09-19 at 17:01 -0700, Stefano Stabellini wrote:
>>> On Tue, 20 Sep 2016, Dario Faggioli wrote:
>>> > And this would work even if/when there is only one cpupool, or in
>>> > general for domains that are in a pool that has both big and LITTLE
>>> > pcpus. Furthermore, big.LITTLE support and cpupools will be
>>> > orthogonal,
>>> > just like pinning and cpupools are orthogonal right now. I.e., once
>>> > we
>>> > will have what I described above, nothing prevents us from
>>> > implementing
>>> > per-vcpu cpupool membership, and either create the two (or more!)
>>> > big
>>> > and LITTLE pools, or from mixing things even more, for more complex
>>> > and
>>> > specific use cases. :-)
>>>
>>> I think that everybody agrees that this is the best long term
>>> solution.
>>>
>>Well, no, that wasn't obvious to me. If that's the case, it's already
>>something! :-)
>>
>>> >
>>> > Actually, with the cpupool solution, if you want a guest (or dom0)
>>> > to
>>> > actually have both big and LITTLE vcpus, you necessarily have to
>>> > implement per-vcpu (rather than per-domain, as it is now) cpupool
>>> > membership. I said myself it's not impossible, but certainly it's
>>> > some
>>> > work... with the scheduler solution you basically get that for
>>> > free!
>>> >
>>> > So, basically, if we use cpupools for the basics of big.LITTLE
>>> > support,
>>> > there's no way out of it (apart from going implementing scheduling
>>> > support afterwords, but that looks backwards to me, especially when
>>> > thinking at it with the code in mind).
>>>
>>> The question is: what is the best short-term solution we can ask Peng
>>> to
>>> implement that allows Xen to run on big.LITTLE systems today?
>>> Possibly
>>> getting us closer to the long term solution, or at least not farther
>>> from it?
>>>
>>So, I still have to look closely at the patches in these series. But,
>>with Credit2 in mind, if one:
>>
>>??- take advantage of the knowledge of what arch a pcpu belongs inside??
>
>>?? ??the code that arrange the pcpus in runqueues, which means we'll end??
>>?? ??up with big runqueues and LITTLE runqueues. I re-wrote that code, I
>>?? ??can provide pointers and help, if necessary;
>>??- tweak the one or two instance of for_each_runqueue() [*] that there
>>?? ??are in the code into a for_each_runqueue_of_same_class(), i.e.:
>
> Do you have plan to add this support for big.LITTLE?
>
> I admit that this is the first time I look into the scheduler part.
> If I understand wrongly, please correct me.
>
> There is a runqueue for each physical cpu, and there are several vcpus in the 
> runqueue.
> The scheduler will pick a vcpu in the runqueue to run on the physical cpu.
>
> A vcpu is bind to a physical cpu when alloc_vcpu, but the vcpu can be 
> scheduled
> or migrated to a different physical cpu.
>
> Settings cpu soft affinity and hard affinity to restrict vcpus be scheduled
> on specific cpus. Then is there a need to introuduce more runqueues?

Runqueues is a scheduler-specific thing.  The simplest thing to do
would be, in the toolstack, to limit the hard affinity of a vcpu to
its cpu class (either big or LITTLE).  Then the scheduler will simply
do the right thing.

> This seems more complicated than cpupool (:

It's more complicated than simply making 2 cpupools and having any
given domain be entirely big or entirely LITTLE.

But it's a lot *less* complicated than trying to make a single domain
cross two different kinds of cpupools. :-)

>>As mentioned in previous mail, and as drafted when replying to Peng,
>>the only think that the user should know is how many big and how many
>>LITTLE vcpus she wants (and, potentially, which one would be each). :-)
>
> Yeah. Comes a new question to me.
>
> For big.LITTLE, how to decide the physical cpu is a big CPU or a little cpu?
>
> I'd like to add a computing capability in xen/arm, like this:
>
> struct compute_capatiliby
> {
>    char *core_name;
>    uint32_t rank;
>    uint32_t cpu_partnum;
> };
>
> struct compute_capatiliby cc=
> {
>   {"A72", 4, 0xd08},
>   {"A57", 3, 0xxxx},
>   {"A53", 2, 0xd03},
>   {"A35", 1, ...},
> }
>
> Then when identify cpu, we decide which cpu is big and which cpu is little
> according to the computing rank.
>
> Any comments?

I think we definitely need to have Xen have some kind of idea the
order between processors, so that the user doesn't need to figure out
which class / pool is big and which pool is LITTLE.  Whether this sort
of enumeration is the best way to do that I'll let Julien and Stefano
give their opinion.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel