This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-devel] Xen Benchmarking guidelines

To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] Xen Benchmarking guidelines
From: "Nick L. Petroni Jr." <npetroni@xxxxxxxxxx>
Date: Tue, 14 Aug 2007 06:37:14 -0400 (EDT)
Cc: Michael Hicks <mwh@xxxxxxxxxx>
Delivery-date: Tue, 14 Aug 2007 03:37:37 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx

I'm trying to perform some simple benchmarks for an HVM domain and I'm wondering if there is a set of guidelines available for configuration tweaking etc. VMware gives a set of guidelines along the lines of "pre-allocate disks, don't use page sharing, etc." and I'm curious if there is an analogous document/resource for Xen.

The main problem I am having is high variance in my benchmark runs, specifically SPECCPU2006 workloads in an HVM domain. Sometimes I get fast runs of a given workload once or twice, but invariably at least one of three consecutive runs is much worse than the best overall time.

Below are some details of my specific problem. I'm most interested in test repeatability, but maximizing performance would be great too.

Thanks and sorry if I missed this info on this or other forums.



When running SPECCPU2006 in HVM domain, experience high variance in test times. Here are some specific results to consider (times given for three consecutive runs of each workload, in seconds).

Xen First test:
  403.gcc:         2290, 2290, 2290   <-- low variance, poor performance
  456.hmmer:       3530, 3530, 3530   <-- low variance, poor performance
  458.sjeng:       1340, 2640, 2630   <-- High variance
  462.libquantum:  3340, 3820, 3810   <-- High variance, poor perofrmance

Xen Second test:
  403.gcc:         1540, 2780, 2770   <-- High variance
  456.hmmer:       1770, 3580, 3590   <-- High variance
  458.sjeng:       1360, 1360, 2680   <-- High variance
  462.libquantum:  2300, 3130, 3900   <-- High variance

Bare HW test for comparison:
  403.gcc:         767, 775, 784
  456.hmmer:       1660, 1670, 1640
  458.sjeng:       1190, 1220, 1220
  462.libquantum:  2130, 2080, 2030

 - IBM/Lenovo T60
 - Core Duo T2500 2.00GHz, VT-x and multicore enabled in BIOS
 - 1.5GB physical memory

 - Similar results for both 3.0.2-2 and 3.1.0 stable releases

 - Debian Lenny, default 2.6.18-xen kernel build

HVM domain:
 - 1200MB RAM
 - 1 vcpu
 - file disk image
 - Red Hat 7.3 with default 2.4.18-3 (non-smp) kernel
   (yes this is old, but I'm stuck with this distro for other reasons)
 - SPECCPU2006, base run

In general, I'm using standard benchmarking best practices, i.e., disabling unnecessary services in host and guest, etc. Perhaps this is related to VCPU scheduling? Should I try pinning a vcpu? Please let me know if I can provide more details. Any help or insight would be much appreciated!

Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>