This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


RE: [Xen-devel] [PATCH 0/6] xen,xend,tools: NUMA support for Xen

To: "Ryan Harper" <ryanh@xxxxxxxxxx>, "Keir Fraser" <Keir.Fraser@xxxxxxxxxxxx>
Subject: RE: [Xen-devel] [PATCH 0/6] xen,xend,tools: NUMA support for Xen
From: "Ian Pratt" <m+Ian.Pratt@xxxxxxxxxxxx>
Date: Tue, 11 Jul 2006 22:28:18 +0100
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Delivery-date: Tue, 11 Jul 2006 14:28:39 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcalEjhS7PRJNyEZRWG5TDiwNYTyAgAG0y+g
Thread-topic: [Xen-devel] [PATCH 0/6] xen,xend,tools: NUMA support for Xen
> > What sort of box are these numbers taken from? If it's not a NUMA
> > system then the slowdowns are rather poor. We're particularly
> > interested in not slowing down non-NUMA and small-NUMA (e.g., AMD
> > x86 systems. They are what we really want to see measurements from.
> The measurements are taken from a two-way Operton 248 (2.1Ghz) ,
> small-NUMA.  I agree that there is significant overhead, however, we
> aren't talking about fast path here; correct me if I'm wrong.   We
> are only adding overhead to during domain startup.  The end result
> being we pay for local memory allocation at creation time while
> benefiting from local memory access for the lifetime of the domain.
> I'm going to gather some oprofile data to see if I missed something
> obvious, but in general I think that having local memory is of greater
> benefit for the lifetime of a domain than the cost we incur during its
> creation.

What do the numbers look like on a 1 node system?

The shadow mode code potentially churns the page allocator a fair bit.
It'll be disappointing if we have to add complexity of quicklists etc. 

It does kind of surprise me that the overhead is as high as you've
measured. In the case where there's memory available in the favoured
node I'd expect allocation performance to be very similar. 4 times
slower and worsening for large allocations seems odd -- 0.3 microseconds
a page is a bit more than I'd expect during back-to-back allocations.
It's certainly worth trying to understand the overhead a bit more.


Xen-devel mailing list