WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] Xen & I/O in clusters - problems! New Information

To: Ian Pratt <Ian.Pratt@xxxxxxxxxxxx>
Subject: Re: [Xen-devel] Xen & I/O in clusters - problems! New Information
From: Håvard Bjerke <Havard.Bjerke@xxxxxxxxxxx>
Date: Wed, 27 Oct 2004 19:11:49 +0200
Cc: Rune Johan Andresen <Rune.Johan.Andresen@xxxxxxxxxxx>, Xen Virtual Machine Monitor <xen-devel@xxxxxxxxxxxxxxxxxxxxx>, =?unknown-8bit?q?H=E5vard?= Bjerke <Havard.Bjerke@xxxxxxxxxxx>
Delivery-date: Thu, 28 Oct 2004 09:53:09 +0100
Envelope-to: steven.hand@xxxxxxxxxxxx
In-reply-to: <E1CL3Vo-00067O-00@xxxxxxxxxxxxxxxxx>
List-archive: <http://sourceforge.net/mailarchive/forum.php?forum=xen-devel>
List-help: <mailto:xen-devel-request@lists.sourceforge.net?subject=help>
List-id: List for Xen developers <xen-devel.lists.sourceforge.net>
List-post: <mailto:xen-devel@lists.sourceforge.net>
List-subscribe: <https://lists.sourceforge.net/lists/listinfo/xen-devel>, <mailto:xen-devel-request@lists.sourceforge.net?subject=subscribe>
List-unsubscribe: <https://lists.sourceforge.net/lists/listinfo/xen-devel>, <mailto:xen-devel-request@lists.sourceforge.net?subject=unsubscribe>
References: <95384844-2448-11D9-8725-000A95B44940@xxxxxxxxxxx> <E1CL3Vo-00067O-00@xxxxxxxxxxxxxxxxx>
Sender: xen-devel-admin@xxxxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.4i
On Fri, Oct 22, 2004 at 06:47:35PM +0100, Ian Pratt wrote:
> > 
> > We tried to compile xen0 with CONFIG_E1000_NAPI = y and got the samre 
> > results between
> > two xen dom0 nodes. I am not sure if these interrupts tells anything:
> 
> It's the different rate at which the eth0 interrupt counts go up
> during your bandwidth tests that is interesting. e.g. poll it
> once a second during the test.
> 

We tried sending 1 MB and measured:
non-SMP native Linux:
        ~ 130k interrupts
        114 kB/s
native Linux with compiled-in SMP support, single CPU:
        ~ 140k interrupts
        114 kB/s
Xen0:
        ~ 180k interrupts
        80 kB/s

> It looks like your native Linux is not using legacy PIC mode
> rather than using the ioapic. Have you tried an SMP native
> kernel to see if it gets the same interrupt layout as Xen?
>  

Layout in native linux w/o SMP:

           CPU0       
  0:  130710161          XT-PIC  timer
  1:          2          XT-PIC  keyboard
  2:          0          XT-PIC  cascade
  3:   12872557          XT-PIC  eth0
  7:          0          XT-PIC  ehci-hcd
  8:          1          XT-PIC  rtc
 10:          3          XT-PIC  usb-uhci
 11:     445263          XT-PIC  aic7xxx, aic7xxx, usb-uhci
 14:          1          XT-PIC  ide0
 15:          0          XT-PIC  libata
NMI:          0 
ERR:          0


Layout in native linux compiled with SMP:

           CPU0       
  0:     121362    IO-APIC-edge  timer
  1:          2    IO-APIC-edge  keyboard
  2:          0          XT-PIC  cascade
 14:          5    IO-APIC-edge  ide0
 16:          0   IO-APIC-level  usb-uhci
 18:     177583   IO-APIC-level  eth0
 19:          0   IO-APIC-level  usb-uhci
 24:       7587   IO-APIC-level  aic7xxx
 25:         30   IO-APIC-level  aic7xxx
NMI:          0 
LOC:     121307 
ERR:          0
MIS:          0


Layout in Xen0:

           CPU0       
  1:          2        Phys-irq  keyboard
 14:          3        Phys-irq  ide0
 18:     267274        Phys-irq  eth0
 24:      15910        Phys-irq  aic7xxx
 25:         30        Phys-irq  aic7xxx
128:          1     Dynamic-irq  misdirect
129:          0     Dynamic-irq  ctrl-if
130:     771917     Dynamic-irq  timer
131:          0     Dynamic-irq  timer_dbg, net-be-dbg
132:          0     Dynamic-irq  console
NMI:          0 
ERR:          0


Questions:
Do you think interrupt sharing is the problem?
Or the use of IO-APIC?
Is this an inherent problem with Xen?
Is it possible to change the interrupt scheme in Xen in order to achieve the 
same performance as in native linux?


Cheers,
Håvard


-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_idU88&alloc_id065&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/xen-devel

<Prev in Thread] Current Thread [Next in Thread>