[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH] RFC: Linux: disable APERF/MPERF feature in PV kernels

On 05/22/2012 11:00 PM, Konrad Rzeszutek Wilk wrote:
On Tue, May 22, 2012 at 11:02:01PM +0200, Andre Przywara wrote:
On 05/22/2012 07:18 PM, Konrad Rzeszutek Wilk wrote:
On Tue, May 22, 2012 at 06:07:11PM +0200, Andre Przywara wrote:

while testing some APERF/MPERF semantics I discovered that this
feature is enabled in Xen Dom0, but is not reliable.
The Linux kernel's scheduler uses this feature if it sees the CPUID
bit, leading to costly RDMSR traps (a few 100,000s during a kernel
compile) and bogus values due to VCPU migration during the

Can you point me to the Linux scheduler code that does this? Thanks.

arch/x86/kernel/cpu/sched.c contains code to read out and compute
APERF/MPERF registers. I added a Xen debug-key to dump a usage
counter added in traps.c and thus could prove that it is actually
the kernel that accesses these registers.
As far as I understood this the idea is to learn about boosting and
down-clocking (P-states) to get a fairer view on the actual
computing time a process consumed.

Looks like its looking for this:


Perhaps masking that should do it? Something along this in enlighten.c:

          cpuid_leaf1_edx_mask =
                  ~((1<<  X86_FEATURE_MCE)  |  /* disable MCE */
                    (1<<  X86_FEATURE_MCA)  |  /* disable MCA */
                    (1<<  X86_FEATURE_MTRR) |  /* disable MTRR */
                    (1<<  X86_FEATURE_ACC));   /* thermal monitoring

would be more appropiate?

Or is that attribute on a different leaf?

Right, it is bit 0 on level 6. That's why I couldn't use any of the predefined masks and I didn't feel like inventing a new one just for this single bit. We could as well explicitly use clear_cpu_cap somewhere, but I didn't find any code place in the Xen tree already doing this, instead it looks like it belongs to where I put it (we handle leaf 5 in a special way already here)

The attached patch explicitly disables this CPU capability inside
the Linux kernel, I couldn't measure any APERF/MPERF reads anymore
with the patch applied.
I am not sure if the PVOPS code is the right place to fix this, we
could as well do it in the HV's xen/arch/x86/traps.c:pv_cpuid().
Also when the Dom0 VCPUs are pinned, we could allow this, but I am
not sure if it's worth to do so.

Awaiting your comments.


P.S. Of course this doesn't fix pure userland software like
cpupower, but I would consider this in the user's responsibility to

Which would not work anymore as the cpufreq support is disabled
when it boots under Xen.

Do you mean with "anymore" in a future kernel? I tested this on
3.4.0 and cpupower monitor worked fine. Right, cpufreq is not
enabled, but cpupower uses the /dev/cpu/<n>/msr device file to
directly read the MSRs. So I get this output if run on an idle Dom0:

Ahh. Neat. Will have to play with that.

Bad news is we cannot forbid cpupower querying the feature directly using the CPUID instruction in PV guests. Only we could patch it to use /proc/cpuinfo readout instead, as this reflects the kernel view of available features. With my patch aperfmperf is no longer there.


Andre Przywara
AMD-OSRC (Dresden)
Tel: x29712

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.