RE: [Xen-devel] A Performance Comparison of Hypervisors

["Petersson, Mats" <Mats.Petersson@xxxxxxx>]

> -----Original Message-----
> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx 
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of 
> Anthony Liguori
> Sent: 03 February 2007 16:15
> To: Nicholas Lee
> Cc: Xen development list
> Subject: Re: [Xen-devel] A Performance Comparison of Hypervisors
> 
> Nicholas Lee wrote:
> > Obviously not a very fair comparison [1]. I can't see how this was 
> > done well at all.
> 
> I wonder why you say this.  I thought the benchmark was done 
> very well.  
> What we need is more benchmarking, not less.  Unfortunately, VMware 
> makes publishing benchmarks difficult as you have to get 
> their approval.
> 
> This benchmark tells us something, the question is what does it tell 
> us.  Let's take a look at the benchmarks they choose.  
> SPECcpu2000 and 
> SPECjbb2005 are two favorite benchmarks of virtualization 
> vendors.  They 
> are favorites because everyone does well under them :-)  Both aren't 
> sensitive to PTE update or context switch latency and don't 
> involve IO 
> very much.  Even QEMU wouldn't look so bad against these :-)
> 
> I'm not familiar with Passmark, but it looks like it's mostly CPU 
> bound.  For all of these virtualization friendly workloads, Xen does 
> pretty well compared to VMware.   For some of the Passmark bits, Xen 
> actually inches out VMware.  Considering we're Open Source, 
> they really 
> have no excuse to ever be slower than we are :-)
> 
> The compile workload was, IMHO, the most serious of the benchmarks.  
> VMware walloped us on that one.  I suspect that's a some 
> shadow paging 
> overhead and perhaps some disk IO overhead.
> 
> The Netperf results are a tad silly.  They choose Win2k3 for 
> the guest 
> OS.  They installed a paravirtual network driver in their guest 
> (vmxnet).  However, since no PV network driver is available 
> for Windows 
> for Xen 3.0.3, they used emulated IO[1].  Of course 
> performance is going 
> to suck.
> 
> I would have rather seen the benchmarks done with a Linux guest using 
> the PV drivers that are in the tree.
> 
> The only embarrassing part is that they weren't able to boot a Win2k3 
> guest with SMP support.  I suspect we need either more QA for 
> HVM or a 
> better statement of supported guest confirmations.

I believe official support for SMP HVM guest wasn't in there until
3.0.4, so not really surprising that it doesn't work right in 3.0.3 ;-)
[It was, I think, possible to make SMP HVM guests work, but it involved
recompiling the BIOS code, which of course is a bit beyond what you'd
expect the average reviewer to do...]

--
Mats
> 
> Regards,
> 
> Anthony Liguori
> 
> [1] The PV drivers that come in XenEnterprise are, AFAIK, only for 
> XenEnterprise.
> 
> > VMWare are a bit silly to release stuff like this, just lowers the 
> > whole game.
> >
> >
> > [1] http://www.vmware.com/vmtn/resources/711
> > 
> --------------------------------------------------------------
> ----------
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-devel
> >   
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel
> 
> 
> 



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel