WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] MPI benchmark performance gap between native linux anddo

To: Ian Pratt <m+Ian.Pratt@xxxxxxxxxxxx>
Subject: Re: [Xen-devel] MPI benchmark performance gap between native linux anddomU
From: xuehai zhang <hai@xxxxxxxxxxxxxxx>
Date: Mon, 04 Apr 2005 18:43:08 -0500
Cc: Xen-devel@xxxxxxxxxxxxxxxxxxx
Delivery-date: Mon, 04 Apr 2005 23:55:45 +0000
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <A95E2296287EAD4EB592B5DEEFCE0E9D1E3A1D@xxxxxxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <A95E2296287EAD4EB592B5DEEFCE0E9D1E3A1D@xxxxxxxxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla Thunderbird 1.0.2 (Windows/20050317)
Ian,

Thanks for the quick response! Explainations to your comments are inline below.

I did the following experiments to explore the MPI application execution performance on both native linux machines and inside of unpriviledged Xen user domains. I use 8 machines with identical HW configurations (498.756 MHz dual CPU, 512MB memory, on a 10MB/sec LAN) and I use Pallas MPI Benchmarks (PMB).


The expreiment results show, running a same MPI benchmark in user domains usually results in a worse (sometimes very bad) performance comparing with on native linux machines. The following are the results for PMB SendRecv benchmark for both experiments (table1 and table2 report throughput and latency respectively). As you may notice, SendRecv can achieve a 14.9MB/sec throughput on native linux machines but can get a maximum 7.07 MB/sec throughput if running inside of user domains. The latency results also have big gap.


I will appreciate your help if you had the similar experience and wanna share your insights.


Xen (or any kind of virtualization) is not particularly well suited to
MPI type applications, at least unless you're using Inifiniband or some
other smart NIC that avoids having to use dom0 to do the IO
virtualization.

However, the results you are seeing are lower than I'd expect.

Are you running dom0 and the domU on the same CPU or different CPUs. How
does changing this effect the results?

I did not specify "cpu" option in Xen's configuration file, so I think both dom0 and domU run on the same CPU (1st CPU). I will try to assign them to different CPUs later.

Also, are you sure the MTU is the same in all cases?

The outputs of "ifconfig" show MTU is 1500 in all cases.

Further, please can you repeat the experiements with just a dom0 running
on each node.

I will do it and update you later.

Thanks again for the help.

Xuehai

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel