WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] Disk IO trouble in Xen

To: "xen-users@xxxxxxxxxxxxxxxxxxx" <xen-users@xxxxxxxxxxxxxxxxxxx>
Subject: Re: [Xen-users] Disk IO trouble in Xen
From: Greg Hellings <greg.hellings@xxxxxxxxxxxx>
Date: Tue, 06 May 2008 09:38:05 -0700
Delivery-date: Tue, 06 May 2008 09:39:34 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <E2BB8074E5500C42984D980D4BD78EF9022A7190@xxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: Aciu+W7brUb+EBrsEd2umQAX8t0tvQAAWDiwACcvwbE=
Thread-topic: [Xen-users] Disk IO trouble in Xen
User-agent: Microsoft-Entourage/11.3.3.061214
So I tracked down my network freezing issues.  I have a few xen servers all
connected to the same network I'm working with here. All of them I used the
virt-manager tool to install.  One of the other hypervisors had a domu
running on it that virt-manager had chosen the same mac address as the domu
on the hypervisor I was having problems with.

I still see the same performance difference between the dom0 and domu when
using hdparm -tT, but maybe that isn't a good test.  I do notice that if I
run hdparm over and over on the domu's the numbers improve, suggesting that
caching is effecting the out come.  Under iozone, the number between domu
and the dom0 are almost exactly the same.

--
Greg


On 5/5/08 2:58 PM, "Ross S. W. Walker" <rwalker@xxxxxxxxxxxxx> wrote:

> Greg Hellings wrote:
>> 
>> I started this thinking I was having an IO problem.  Now, I'm
>> not so sure.
>> I am using a pre-allocated disk, not a sparse file.
>> 
>> Write performance seems about the same for both
>> 
>> Dom0
>> [root@localhost ~]# dd if=/dev/zero of=/tmp/dummy count=2097152
>> 2097152+0 records in
>> 2097152+0 records out
>> 1073741824 bytes (1.1 GB) copied, 29.8368 seconds, 36.0 MB/s
>> 
>> DomU
>> [root@localhost ~]# dd if=/dev/zero of=/tmp/dummy count=2097152
>> 2097152+0 records in
>> 2097152+0 records out
>> 1073741824 bytes (1.1 GB) copied, 30.4716 seconds, 35.2 MB/s
>> 
>> Iozone shows very similar numbers on both the Dom0 and DomU. The DomU seems
>> to freeze periodically and drop network connections.  I assumed this was IO
>> related because everything else seems fine and those hdparm numbers are so
>> far off.  Also heavy IO related tasks seem slow.  I'm fairly baffled now.
>> 
> 
> Run a 'vmstat 1' and take a look at the interrupts and io waits.
> 
> Maybe the network is generating a high load?
> 
> Performance, and performance under load are two different things.
> 
> 
>> On 5/2/08 4:04 AM, "Sadique Puthen" <sputhenp@xxxxxxxxxx> wrote:
>> 
>>> Are you using a pre-allocated image or a sparse file? If
>> the latter it's
>>> likely to have performance problems like you found?  What
>> if you test
>>> with dd or iozone or bonnie++?
>>> 
>>> --Sadique
>>> 
>>> Greg Hellings wrote:
>>>> I'm having terrible IO trouble with an image based domu
>>>> 
>>>> 
>>>> This is the performance on the Dom0
>>>> [root@localhost /]# hdparm -tT /dev/sda
>>>> 
>>>> /dev/sda:
>>>>  Timing cached reads:   3228 MB in  2.00 seconds = 1617.22 MB/sec
>>>>  Timing buffered disk reads:  174 MB in  3.02 seconds =
>> 57.60 MB/sec
>>>> 
>>>> 
>>>> And this is the performance on the DomU
>>>> [root@localhost ~]# hdparm -tT /dev/xvda
>>>> 
>>>> /dev/xvda:
>>>>  Timing cached reads:   3336 MB in  2.00 seconds = 1669.98 MB/sec
>>>>  Timing buffered disk reads:   26 MB in  3.13 seconds =
>> 8.30 MB/sec
>>>> 
>>>> 
>>>> Is it normal for the performance difference to be so
>> great?  I'm running
>>>> CentOS 5 and the DomU image is stored on a LVM ReiserFS
>> formatted partition.
>>>> 
>>>> Here is the config for the DomU
>>>> name = "dddevdb"
>>>> uuid = "927f1e915813c2dec60c8f76c2716783"
>>>> maxmem = 3072
>>>> memory = 2042
>>>> vcpus = 2
>>>> bootloader = "/usr/bin/pygrub"
>>>> on_poweroff = "destroy"
>>>> on_reboot = "restart"
>>>> on_crash = "restart"
>>>> vfb = [ "type=vnc,vncunused=1,keymap=en-us" ]
>>>> disk = [ "tap:aio:/var/lib/xen/images/dddevdb.img,xvda,w"]
>>>> vif = [ "mac=00:16:3e:62:7a:d1,bridge=xenbr0" ]
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Xen-users mailing list
>>>> Xen-users@xxxxxxxxxxxxxxxxxxx
>>>> http://lists.xensource.com/xen-users
>>>> 
>>>>   
>>> 
>> 
>> 
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-users
>> 
> 
> ______________________________________________________________________
> This e-mail, and any attachments thereto, is intended only for use by
> the addressee(s) named herein and may contain legally privileged
> and/or confidential information. If you are not the intended recipient
> of this e-mail, you are hereby notified that any dissemination,
> distribution or copying of this e-mail, and any attachments thereto,
> is strictly prohibited. If you have received this e-mail in error,
> please immediately notify the sender and permanently delete the
> original and any copy or printout thereof.
> 


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users