[Xen-ia64-devel] domU faster than native? Not really....

["Magenheimer, Dan (HP Labs Fort Collins)" <dan.magenheimer@xxxxxx>]

I think I can explain the recent confusion where some
developers are measuring domU performance (on kernel
build) as being faster than native and if I am right
it illustrates a measurement methodology issue that
we should make consistent.

We recently switched the default boot for Xen/ia64
from UP to SMP and I think most of us are running
on 2-way or higher machines.

The default configuration on an SMP box is for dom0
to run on one processor and domU to run on another
processor.  I have observed that building the kernel
native is faster on an SMP box, even when not using
"make -j2".  I suspect some significant portion of
the domU kernel build (e.g. filling I/O buffers) is
taking place in dom0 in parallel when they are on
separate processors.

Although this is an interesting data point, when testing
for domU performance degradation, we need to measure the
total work performed by the entire system, including
the hypervisor and dom0.  Probably the best way to
do that is to always boot Xen with "nosmp" when running
domU performance regression testing.

On a related note, as others have pointed out, it is
not good to compare one system with many daemons running
with another system with fewer or no daemons.  I've
seen that the "telinit 1" command can be run to kill
off nearly all daemons.  If it is executed on dom0 before
running "xend start", it appears that it is still possible
to create a domU (paravirtualized at least).  I suggest
that we standardize on running "telinit 1" on native,
and on BOTH dom0 and domU when testing for domU
performance regression.

We also need to ensure we are consistent about memory
usage.  If a domU is thrashing to (virtual) swap disk,
the measurement can be very different than if the files
are all in buffer cache.  Alex has a method to avoid
measuring disk I/O as part of the kernel build
essentially reading/writing all build files to ramdisk.
Perhaps we should use this approach.  (Note that
domU requires 1GB of memory minimum and the first
build measurement should be ignored.)  Unless of course
we are trying to measure for driver performance.

Last, kernel build time is not consistent from one
kernel version to another and from one version of gcc
to another.  In order to track domU performance over time,
I suggest we agree on one kernel version to use and
one build environment (I build 2.6.9 on gcc 3.4.4 on
RH 4.2.)  Not everybody needs to run the identical
regression test, but it would be good if one developer
runs the same performance regression test regularly
(daily?) and publishes the results.

P.S. I am getting kernel build times of about 12-13
minutes on my 1.6Ghz Mckinley.  This is far speedier
than one might estimate from a ratio of clock rate vs
Alex's Mad6M and Anthony's Montecito.  I wonder if
maybe gcc 3.4.4 is faster than what others are using?

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ia64-devel