Re: [Xen-devel] [RFC 0/5] xen/arm: support big.little SoC

[Dario Faggioli <dario.faggioli@xxxxxxxxxx>]

On Wed, 2016-09-21 at 20:28 +0100, Julien Grall wrote:
> On 21/09/2016 16:45, Dario Faggioli wrote:
> > This does not seem to match with what has been said at some point
> > in
> > this thread... And if it's like that, how's that possible, if the
> > pcpus' ISAs are (even only slightly) different?
> 
> Right, at some point I mentioned that the set of errata and features 
> will be different between processor.
> 
Yes, I read that, but wasn't (and still am not) sure about whether or
not that meant a vcpu can move freely between classes or not, in the
way that the scheduler does that.

In fact, you say:

> With a bit of work in Xen, it would be possible to do move the vCPU 
> between big and LITTLE cpus. As mentioned above, we could sanitize
> the 
> features to only enable a common set. 
> You can view the big.LITTLE 
> problem as a local live migration between two kind of CPUs.
> 
Local migration basically --from the vcpu perspective-- means create a
new vcpu, stop the original vcpu, copy the state from original to new,
destroy the original vcpu and start the new one. My point is that this
is not something that can be done within nor initiated by the
scheduler, e.g., during a context switch or a vcpu wakeup!

And I'm saying this because...

> In your suggestion you don't mention what would happen if the guest 
> configuration does not contain the affinity. Does it mean the vCPU
> will 
> be scheduled anywhere? A pCPU/class will be chosen randomly?
> 
...in my example there were vcpus for which no set of classes was
specified, and I said that it meant those vcpus can run on any pcpu of
any class. And this would be what I think we should do even in cases
where no "vcpuclass" parameter is specified at all.

*BUT* that is only possible if moving a vcpu from a pcpu of class A to
a pcpu of class B does *NOT* require the steps described above, similar
to local migration. IOW, this is only possible if moving a vcpu from a
pcpu of class A to a pcpu of class B *ONLY* requires a context switch.

If changing class requires local migration, the scheduler must be told
that he should never move vcpus between classes (or set of classes made
by homogeneous enough vcpus for which a context switch is sufficient).
If changing class is --or can be made to be, with some work in Xen--
just a context switch, then we can have the scheduler moving vcpus
between (set of) classes.

It's probably not too big of a deal, wrt the end result (see below),
but it changes the implementation a lot.

But, yeah, if changing class can be made simple with some work in Xen,
but is not simple/possible **right now**, then this means that,
_for_now_, vcpus for which a class is not specified must be assigned to
a class (or a set of classes within which the scheduler can freely move
vcpus). In future, we can change this, broadening the "default class"
as much as seamless migration within its pcpus allows that.

Hope I made myself clear enough. :-D

> To be honest, I quite like this idea. 
>
:-)

> It could be used as soft/hard 
> affinity for the moment. But can be extended in the future if/when
> the 
> scheduler gain knowledge of power efficiency and vCPU can migrate 
> between big and LITTLE.
> 
Yes, exactly, and this is, I think, true in both of the above outlined
cases. What I meant when I said it is the implementation, rather than
the end result that changes, is that:
 - if complex migration-alike operations are necessary for changing 
   class, migrating between classes (e.g., between big and LITTLE) 
   will have to happen, e.g., in a load and energy management and
   balancing component implemented above the scheduler itself
 - if just plain context switch is enough, the scheduler can do
   everything by itself.

But yes, in any case, the model we're coming up with looks to be a very
good starting point, because it is orthogonal to and independent from
other components and solution (e.g., cpupools) and is pretty simple and
basic, and leaves room for future extensions.

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel