WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] Network performance - sending from VM to VM using TCP

To: Cherie Cheung <ccyxen@xxxxxxxxx>
Subject: Re: [Xen-devel] Network performance - sending from VM to VM using TCP
From: Nivedita Singhvi <niv@xxxxxxxxxx>
Date: Wed, 25 May 2005 16:24:13 -0700
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx, xen-users@xxxxxxxxxxxxxxxxxxx
Delivery-date: Wed, 25 May 2005 23:23:46 +0000
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4713f859050525152448a0f609@xxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <4713f859050525152448a0f609@xxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla Thunderbird 0.8 (X11/20041020)
Cherie Cheung wrote:
Hi,

I have been simulating a network using dummynet and evaluating it

I haven't played with dummynet and don't know if there are
additional issues inherent in using dummynet itself...

using netperf. Xen3.0-unstable is used and the VMs are
vmlinuz-2.6.11-xenU. The simulated link is 300Mbps with 80ms RTT.
Using netperf, I sent data using TCP from domain-0 of machine 1 to
domain-0 of machine 2. Then I repeat the experiment, but this time
from VM-1 of machine 1 to VM-1 of machine 2.

However, the performance across the two VMs is substantially worse
than that across domain-0. Here's the result:

FROM VM to VM:
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to dw10.ucsd.edu
(172.19.222.210) port 0 AF_INET
Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 87380 65536 65536 80.28 24.83

Your send message size is exactly your socket size. It is also
the size of the default write buffer. The kernel uses half the
buffer (very roughly) for data

Were you testing with 65536 bytes exactly for some reason?
This is stop and go traffic and normally the kernel doesn't
use the entire buffer to store data - it's roughly half...

Could you test with different send sizes?

FROM domain-0 to domain-0:
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to damp.ucsd.edu
(137.110.222.236) port 0 AF_INET
Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6bits/sec 87380 65536 65536 80.11 280.62
Here's the setting of the network buffer:

net.core.wmem_max = 8388608
net.core.rmem_max = 8388608
net.ipv4.tcp_bic = 1
net.ipv4.tcp_rmem = 4096        87380   8388608
net.ipv4.tcp_wmem = 4096        65536   8388608

Does anyone know why the performance across two VMs is so bad? Any fix
to it? Thank you.

If you just want to improve your peformance, increase your
buffer sizes!

For example:
tcp_rmem = 4096 1398080 8388608
tcp_wmem = 4096 1398080 8388608

Were you seeing losses, queue overflows?

More importantly, how much memory do you have in the system and
how were you allocating it?


thanks,
Nivedita

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel