> NetApp GX with two heads and 10GigE's.
> Measured at over 2Gigabytes/sec!
> Should easily handle 200MBytes/sec
> Network path:
> VirtHostA -GigE-> 4948-10G (port 1 )-10gigE-> 6509 -> -10GigE-> NetApp
> VirtHostB -GigE-> 4948-10G (port 17)-10gigE-> 6509 -> -10GigE-> NetApp
Ok, I see you're preferring *real* equipment ;)
I don't think the network stack of the domU's has a problem, as one
dd works. As I'm pretty sure, you always checked each domU is connected
to its own bridge, I would take a look at the domU config.
Have you tried to pin the VCPU's to dedicated cores? For a quick test,
I would reduce the domU's to three cores and keep 2 in dom0.
cpus = '1-3'
vcpus = '3'
vif [ 'bridge=xenbr0,mac= ...' ]
cpus = '5-7'
vcpus = '3'
vif [ 'bridge=xenbr1,mac= ...' ]
or, temporarily, use xm vcpu-pin / xm vcpu-set
I found similar (MP dualcore and MP quadcore Xeon) systems performing
much better if the domU's are using only cores located at the same cpu.
Without deeper knowlegde about this, I assume this has to do with a
better use of caches.
> On Dec 28, 2007 7:30 PM, Stephan Seitz <s.seitz@xxxxxxxxxxxx> wrote:
>> Don't get me wrong,
>> but my first thought was: What is the maximum expected throughput of the
>> nfs server? It should at least be connected with 2 GBit/s to the switch,
>> to serve two dd's with each ~100MB/s.
>> Well, I assume both domU's are using the same nfs server.
>> tmac schrieb:
>>> I have a beefy machine
>>> (Intel dual-quad core, 16GB memory 2 x GigE)
>>> I have loaded RHEL5.1-xen on the hardware and have created two logical
>>> 4 cpus, 7.5 GB memory 1 x Gige
>>> Following RHEL guidelines, I have it set up so that eth0->xenbr0 and
>>> Each of the two RHEL5.1 guests uses one of the interfaces and this is
>>> verified at the
>>> switch by seeing the unique MAC addresses.
>>> If I do a crude test from one guest over nfs,
>>> dd if=/dev/zero of=/nfs/test bs=32768 count=32768
>>> This yields almost always 95-100MB/sec
>>> When I run two simultaneously, I cannot seem to get above 25MB/sec from
>>> It starts off with a large burst like each can do 100MB/sec, but then
>>> in a couple
>>> of seconds, tapers off to the 15-40MB/sec until the dd finishes.
>>> Things I have tried (installed on the host and the guests)
>>> net.core.rmem_max = 16777216
>>> net.core.wmem_max = 16777216
>>> net.ipv4.tcp_rmem = 4096 87380 16777216
>>> net.ipv4.tcp_wmem = 4096 65536 16777216
>>> net.ipv4.tcp_no_metrics_save = 1
>>> net.ipv4.tcp_moderate_rcvbuf = 1
>>> # recommended to increase this for 1000 BT or higher
>>> net.core.netdev_max_backlog = 2500
>>> sysctl -w net.ipv4.tcp_congestion_control=cubic
>>> Any ideas?
>> Stephan Seitz
>> Senior System Administrator
>> *netz-haut* e.K.
>> multimediale kommunikation
>> zweierweg 22
>> 97074 würzburg
>> fon: +49 931 2876247
>> fax: +49 931 2876248
>> web: www.netz-haut.de <http://www.netz-haut.de/>
>> registriergericht: amtsgericht würzburg, hra 5054
Senior System Administrator
fon: +49 931 2876247
fax: +49 931 2876248
web: www.netz-haut.de <http://www.netz-haut.de/>
registriergericht: amtsgericht würzburg, hra 5054
Description: OpenPGP digital signature
Xen-users mailing list