[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] MPI benchmark performance gap between native linux anddomU



Ian,

Thanks for the quick response! Explainations to your comments are inline below.

I did the following experiments to explore the MPI application execution performance on both native linux machines and inside of unpriviledged Xen user domains. I use 8 machines with identical HW configurations (498.756 MHz dual CPU, 512MB memory, on a 10MB/sec LAN) and I use Pallas MPI Benchmarks (PMB).


The expreiment results show, running a same MPI benchmark in user domains usually results in a worse (sometimes very bad) performance comparing with on native linux machines. The following are the results for PMB SendRecv benchmark for both experiments (table1 and table2 report throughput and latency respectively). As you may notice, SendRecv can achieve a 14.9MB/sec throughput on native linux machines but can get a maximum 7.07 MB/sec throughput if running inside of user domains. The latency results also have big gap.


I will appreciate your help if you had the similar experience and wanna share your insights.


Xen (or any kind of virtualization) is not particularly well suited to
MPI type applications, at least unless you're using Inifiniband or some
other smart NIC that avoids having to use dom0 to do the IO
virtualization.

However, the results you are seeing are lower than I'd expect.

Are you running dom0 and the domU on the same CPU or different CPUs. How
does changing this effect the results?

I did not specify "cpu" option in Xen's configuration file, so I think both dom0 and domU run on the same CPU (1st CPU). I will try to assign them to different CPUs later.

Also, are you sure the MTU is the same in all cases?

The outputs of "ifconfig" show MTU is 1500 in all cases.

Further, please can you repeat the experiements with just a dom0 running
on each node.

I will do it and update you later.

Thanks again for the help.

Xuehai

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.