Re: [Xen-devel] [DOC RFC] Heterogeneous Multi Processing Support in Xen

[George Dunlap <George.Dunlap@xxxxxxxxxx>]

> On Dec 9, 2016, at 5:09 PM, Jan Beulich <JBeulich@xxxxxxxx> wrote:
> 
>>>> On 09.12.16 at 09:29, <dario.faggioli@xxxxxxxxxx> wrote:
>> On Fri, 2016-12-09 at 01:13 -0700, Jan Beulich wrote:
>>>>>> On 08.12.16 at 22:54, <dario.faggioli@xxxxxxxxxx> wrote:
>>>> Yeah, that was what was puzzling me too. Keeping them ordered has
>>>> the
>>>> nice property that if a user says the following in a config file:
>>>> 
>>>> vcpuclass=["0-3:class0", "4-7:class1"]
>>>> 
>>>> (assuming that class0 and class1 are the always available Xen
>>>> names) it
>>> 
>>> This, btw, is another aspect I think has a basic problem: class0 and
>>> class1 say nothing about the properties of a class, and hence are
>>> tied to one particular host.
>>> 
>> The other way round, I'd say. I mean, since they say nothing, they're
>> _not_ host specific?
> 
> No, not really. Or perhaps we mean different things. The name
> itself of course can be anything, but what is relevant here is
> what it stands for. And "class0" may mean one thing on host 1
> and a completely different thing on host2. Yet we need a certain
> name to always mean the same thing (or else we'd need
> translation when moving VMs between hosts).
> 
>>> I think class names need to be descriptive
>>> and uniform across hosts. That would allow migration of such VMs as
>>> well as prevent starting them on a host not having suitable hardware.
>>> 
>> ...what George suggested (but please, George, when back, correct me if
>> I'm misrepresenting your ideas :-)) that:
>> - something generic, such as class0, class1 will always exist (well, 
>>   at least class0). They would basically constitute the Xen interface;
>> - toolstack will accept more specific names, such as 'big' and 
>>   'little', and also 'A57' and 'A43' (I'm making up the names), etc.
>> - a VM with vCPUs in class0 and class1 will always be created and run 
>>   on any 2 classes system;
> 
> How can that work, if you don't know what class1 represents?
> 
>> a VM with big and little vCPUs will only 
>>   run on an ARM big.LITTLE incarnation; a VM with A57 and A43 vCPUs 
>>   will only run on an host that has at least one A57 and one A43 
>>   pCPUs.
>> 
>> What's not clear to me is how to establish:
>> - the ordering among classes;
> 
> As said before - there's at best some partial ordering going to be
> possible.
> 
>> - the mapping between Xen's neuter names and the toolstack's (arch) 
>>   specific ones.
> 
> Perhaps it needs re-consideration whether class names make
> sense in the first place? What about, for example, making class
> names something entirely local to the domain config file, and
> besides specifying
> 
> vcpuclass=["0-3:class0", "4-7:class1"]
> 
> requiring for it to also specify the properties of the classes it
> uses:
> 
> class0=["..."]
> class1=["..."]
> 
> The specifiers then would be architecture specific, e.g.
> 
> class0=["arm64"]
> class1=["arm64.big"]
> 
> or on x86
> 
> class0=["x86-64"]
> class1=["x86.avx", "x86.avx2"]
> class2=["x86.XeonPhi"]
> 
> Of course this goes quite a bit in the direction of CPUID handling,
> so Andrew may have a word to say here.

So my goal when I made my suggestion was that:

1. People who knew exactly what they wanted and knew what their hardware would 
be could specify exactly what they wanted to happen.  This would probably 
include embedded chip vendors designing a custom system.

2. People who had a general preference but didn’t know the exact hardware could 
specify vague parameters (such as “large class” or “small class”) and get 
something approximating the vague parameters.  This might include people who 
were writing a generic piece of software to be run on a large class of 
potential devices (automotive, routers, &c).

3. People who didn’t specify anything would get a default behavior which was 
sensible.

From what I remember of the last discussion, there is no “arm64.big”.  You 
might have an A15 core and an A7 core; and in that case the A15 core would be 
“big”.  But you also might have two A15 cores, one with a higher clock speed 
and/or more cache than the other.  So “arm64.big” and “arm64.little" isn't 
actually any more precise than “class0” and “class1”.

So my idea was (to re-iterate):
1. Sort them into classes by power
2. Allow the user to either specify the class number (class0 > class 1 > 
class2), *or* to make more specific requests (“arm64.A15”, &c).

That way, people descibed by #3 can not specify anything, and the toolstack can 
decide whether to give it class 0/1 based on some heuristic and / or policy; 
people described by #2 can just say “class 0” or “class 1”, and people in class 
#1 can specify exactly what they want.

Now I understand that it may not always be clear which of two processors is 
“more powerful” — but to accomplish the above-stated goal, it turns out that’s 
not necessary.  If two processors are about equally powerful but in slightly 
different ways, then it doesn’t matter which one you get when you ask for “the 
bigger one”; so it doesn’t matter which order you put them in (although it 
should probably be repeatable).

 -George



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel