This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-devel] swiotlb=force in Konrad's xen-pcifront-0.8.2 pvops domU

To: Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>
Subject: Re: [Xen-devel] swiotlb=force in Konrad's xen-pcifront-0.8.2 pvops domU kernel with PCI passthrough
From: Chris Mason <chris.mason@xxxxxxxxxx>
Date: Thu, 18 Nov 2010 12:28:34 -0500
Cc: Jeremy Fitzhardinge <jeremy@xxxxxxxx>, Xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>, "mathieu.desnoyers" <mathieu.desnoyers@xxxxxxxxxx>, "andrew.thomas" <andrew.thomas@xxxxxxxxxx>, "keir.fraser" <keir.fraser@xxxxxxxxxxxxx>, "mukesh.rathor" <mukesh.rathor@xxxxxxxxxx>, Dante Cinco <dantecinco@xxxxxxxxx>
Delivery-date: Thu, 18 Nov 2010 09:32:00 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <20101118171936.GA29275@xxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <20101112165541.GA10339@xxxxxxxxxxxx> <EB4C61A1A2501842A04B573FE42B14D601374FBFD2@xxxxxxxxxxxxxxxxx> <20101112223333.GD26189@xxxxxxxxxxxx> <AANLkTi=H6r2=-zJE+6eCtP4VXacYhd_e47+KRW5vdwjS@xxxxxxxxxxxxxx> <20101116185748.GA11549@xxxxxxxxxxxx> <AANLkTikw8reKXwd9CcXc3qqHuXKjbMEatAVfn19uwzs3@xxxxxxxxxxxxxx> <20101116201349.GA18315@xxxxxxxxxxxx> <AANLkTin7SRKuT5qQQ_1NSis1asOG3eJ1SmmC3fppsGnv@xxxxxxxxxxxxxx> <20101118171936.GA29275@xxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Sup/git
Excerpts from Konrad Rzeszutek Wilk's message of 2010-11-18 12:19:36 -0500:
> Keir, Dan, Mathieu, Chris, Mukesh,
> This fellow is passing in a PCI device to his Xen PV guest and trying
> to get high IOPS. The kernel he is using is a 2.6.36 with tglx's
> sparse_irq rework.
> > I wanted to confirm that bounce buffering was indeed occurring so I
> > modified swiotlb.c in the kernel and added printks in the following
> > functions:
> > swiotlb_bounce
> > swiotlb_tbl_map_single
> > swiotlb_tbl_unmap_single
> > Sure enough we were calling all 3 five times per I/O. We took your
> > suggestion and replaced pci_map_single with pci_pool_alloc. The
> > swiotlb calls were gone but the I/O performance only improved 6% (29k
> > IOPS to 31k IOPS) which is still abysmal.
> Hey! 6% that is nothing to sneeze at.

How fast does it go on bare metal?

I usually do four things:

1) perf record -g -a -f 'sleep 15'
(use perf report to look at the biggest CPU hogs)

2) mpstat -P ALL 1 to find the CPU doing all the softirq processing

3) perf record -g -C N -f 'sleep 15' where N was the CPU in mpstat -P
ALL that was doing all the softirq processing

4) Turn off the intel iommu.  This isn't an option of for virtualized
though, but I'd try it on/off on bare metal.


Xen-devel mailing list

<Prev in Thread] Current Thread [Next in Thread>