Re: [Xen-devel] What is the target CPU "topology" of an SMP HVM machine?

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 14/08/13 20:23, Eric Shelton wrote:
> In doing some work to run OS X under Xen on my MacBook Air 2012 (Ivy
> Bridge), I ran into some issues in Darwin's probing of what it refers
> to as the CPU topology.  Although the Darwin kernel may make certain
> assumptions about the platforms on which it is being run, it
> nevertheless appears the various values Xen returns via CPUID and MSR
> are not wholly consistent.  For example, when I configured the domain
> to have only 1 vcpu, Darwin was still able to infer that the system
> had multiple processors (maybe even the correct numbers of cores and
> processors).

The extended model name is passed through from the real CPU, so Darwin
could easily be working on logic such as "I have found a CPU which
claims to be this type of IvyBridge - I know it has these details"

>
> Adding the following to the domain config file got things to move past
> a divide by zero resulting from the topology info reported by Xen:
>
> cpuid = [ '4,3:eax=0001xxxxxxxxxx1111xxxxxxxxxxxxxx' ]
>
> The '1111' portion is the key part, and was merely copied from the
> bits natively reported by the CPU outside of Xen - a configuration
> providing 4 logical processors.
>
> So, seeing as this information is being closely interrogated, what is
> the target virtual CPU topology?  How should this be reported via
> CPUID and MSR?  Darwin appears to be trying to determine or take into
> account things such as a number of packages, dies per package, cores
> per pie & package, and threads/logical CPUs per core & package; the
> degrees of sharing of caches by CPUs at various cache levels, and the
> presence of hyperthreading.

Xen by default advertises all VCPUs as separate sockets, to try and
dissuade "clever" schedulers from doing dumb things based on false
information.

>
> For example,  Darwin's osfmk/i386/cpu_threads.c (thankfully open
> source), will report the following - I believe just based on the CPUID
> and MSR values:
>
>     TOPO_DBG("\nCache Topology Parameters:\n");
>     TOPO_DBG("\tLLC Depth:           %d\n", topoParms.LLCDepth);
>     TOPO_DBG("\tCores Sharing LLC:   %d\n", topoParms.nCoresSharingLLC);
>     TOPO_DBG("\tThreads Sharing LLC: %d\n", topoParms.nLCPUsSharingLLC);
>     TOPO_DBG("\tmax Sharing of LLC:  %d\n", topoParms.maxSharingLLC);
>
>     TOPO_DBG("\nLogical Topology Parameters:\n");
>     TOPO_DBG("\tThreads per Core:  %d\n", topoParms.nLThreadsPerCore);
>     TOPO_DBG("\tCores per Die:     %d\n", topoParms.nLCoresPerDie);
>     TOPO_DBG("\tThreads per Die:   %d\n", topoParms.nLThreadsPerDie);
>     TOPO_DBG("\tDies per Package:  %d\n", topoParms.nLDiesPerPackage);
>     TOPO_DBG("\tCores per Package: %d\n", topoParms.nLCoresPerPackage);
>     TOPO_DBG("\tThreads per Package: %d\n", topoParms.nLThreadsPerPackage);
>
>     TOPO_DBG("\nPhysical Topology Parameters:\n");
>     TOPO_DBG("\tThreads per Core: %d\n", topoParms.nPThreadsPerCore);
>     TOPO_DBG("\tCores per Die:     %d\n", topoParms.nPCoresPerDie);
>     TOPO_DBG("\tThreads per Die:   %d\n", topoParms.nPThreadsPerDie);
>     TOPO_DBG("\tDies per Package:  %d\n", topoParms.nPDiesPerPackage);
>     TOPO_DBG("\tCores per Package: %d\n", topoParms.nPCoresPerPackage);
>     TOPO_DBG("\tThreads per Package: %d\n", topoParms.nPThreadsPerPackage);
>
> In addition to CPUID and MSR, does any of this get reflected in the
> ACPI tables?  Also, is there a presumed relationship between the
> number of dies or cores and the number of HPET comparators to be
> concerned with?

There is a distinct lack of consistency between the various mechanisms. 
The ACPI tables are essentially static from build time.

>
> Finally, included in all of this is the use of an undocumented MSR
> 0x35, which appears to be available on at least Nehalem on up, which
> reports the number of cores and processors, and reports this
> information slightly differently between some of the Intel
> architectures.  Would it be OK to trap & emulate this behavior where
> CPUID is reporting a model that implements MSR 0x35?  Would it be
> better to be able to override MSR values in the domain config file,
> much as with CPUID?
>
> Thanks,
> Eric

From a quick glance at the documentation, there are several different
generations of processors which use MSR 0x35 for different purposes,
although it does appear to be somewhat common as performance counters of
one form or another.

What reference are you using to find out that this msr provides topology
information?

It would certainly be a good project to try and make this information
more consistent and easier to configure.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel