WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] iSCSI initiator on Dom0, exported to DomU via xvd, Disk

To: Ross Walker <rswwalker@xxxxxxxxx>
Subject: Re: [Xen-users] iSCSI initiator on Dom0, exported to DomU via xvd, Disk IO Drops in Half...
From: Ross Walker <rswwalker@xxxxxxxxx>
Date: Tue, 13 Jan 2009 18:53:11 -0500
Cc: Christopher Chen <muffaleta@xxxxxxxxx>, Xen Users Mailing List <xen-users@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Tue, 13 Jan 2009 15:54:24 -0800
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:references:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:x-mailer :mime-version:subject:date:cc; bh=UuHZHvbjpZKe+1jmK2ZmxVdupPpi0EmwAdgKGe9CFDQ=; b=vu6UKkUjIDKR3R1OHbCG//Z6d+KkRjAlsmp6/VXXJ60w8aIHVrHpYKG2qMMkeqzZZD mlaF8F42URcHMEAOq4rN/NrJIFeeHg4OVhbXTvPNsdInwGlEULI6eO1pUt3bggzxCWHU r757E5pwrvdT5kGs7nuWGfrBbQj/GNjfb/nA4=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=references:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:x-mailer:mime-version:subject:date:cc; b=j2jg+chdI99era6nFKwD/RuZHuJS5cB/f8XJKwC3ik7DoR8FZsrUQf4U/oRoSneMz5 BBwIrL5ZXgL8KFe0cqUzIhgRI7g0sbRBMRelJ3CROGx71820+AnzlXaa8a4Z7NMNoWWn c/Mq1I/VHIfZiO9QLlCZ7/qPFRVm3F2pC23pU=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <E7A7EB56-F845-46BE-9AB7-9B07D46317F2@xxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <7bc80d500901131448o45857054ia681a1c00da3d96@xxxxxxxxxxxxxx> <E7A7EB56-F845-46BE-9AB7-9B07D46317F2@xxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx

On Jan 13, 2009, at 6:37 PM, Ross Walker <rswwalker@xxxxxxxxx> wrote:

On Jan 13, 2009, at 5:48 PM, "Christopher Chen" <muffaleta@xxxxxxxxx> wrote:

Hi there!

I've been wrestling with an issue for a little bit now--

In my test environment, I have tgtd running on a Centos 5.2 box, with
a raid 10 array backing it.

The initiators are also Centos 5.2 boxes running Xen 3.0.3 userland
with a Xen 3.1.2/Linux 2.6.18 kernel (as from repos).

Bonnie++ on the Dom0 shows about 110MB/sec writes, and 45MB/sec reads.

That's kind of lopsided I'd expect it the  other way around.

Is this hardware RAID on the backend with write-back cache?


I've attached the iSCSI LUN to the DomU as a virtual block device, and
I'm seeing 47MB/sec writes, and 39MB/sec reads.

How did you attach it, what Xen driver did you use phy: or file:?

Sorry, missed the virtual block device bit...

I've tried a few things, like running against a local disk, and
suprisingly, writes on the DomU are faster than the Dom0--can I assume
the writes are buffered by the Dom0.

I'm confused.

I thought you said above you got 110MB/s on dom0 and 45MB/s on the domU?

Never mind my comment, writes are only buffered using file: io, but they are buffered in the domU's page cache which is where you might be seeing the performance difference.

I'm going to give a shot doing the initialization from the DomU (just
for kicks...)...and wow! 129MB/sec writes, 49MB/sec reads.

You've completely lost me now, what do you mean initialization? Do you mean boot domU off of iSCSI directly?

After re-reading I guessed you meant you attached to the iSCSI lun after booting into the VM not as the OS disk.

Again you are most likely seeing all cache affect and not the real io.

This is all with bonnie++ -d /mnt -f -u root:root

Anyone seen this, or have any ideas?

Is any additional latency provided by the xen virtual block device
causing a degradation in TCP performance (i.e. a window size or
delayed ACK problem) or is the buffering also causing pain? I'm going
to keep looking, but I thought I'd ask all of you.

Any layer you add is going to create latency.

If you can be a little more clearer I'm sure an accurate explanation can be made.

Try increasing the size of the bonnie test file to defeat the cache, say 2x the memory of the dom0 or domU or target which ever is largest.

-Ross


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users