WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] numa=on broken

To: Keir Fraser <Keir.Fraser@xxxxxxxxxxxx>
Subject: Re: [Xen-devel] numa=on broken
From: Ryan Harper <ryanh@xxxxxxxxxx>
Date: Sun, 1 Apr 2007 00:20:18 -0500
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Delivery-date: Sun, 01 Apr 2007 06:20:30 +0100
Envelope-to: Keir.Fraser@xxxxxxxxxxxx
In-reply-to: <C233E334.533E%Keir.Fraser@xxxxxxxxxxxx>
References: <20070330193958.GZ28736@xxxxxxxxxx> <C233E334.533E%Keir.Fraser@xxxxxxxxxxxx>
User-agent: Mutt/1.5.6+20040907i
* Keir Fraser <Keir.Fraser@xxxxxxxxxxxx> [2007-03-31 04:09]:
> On 30/3/07 20:39, "Ryan Harper" <ryanh@xxxxxxxxxx> wrote:
> 
> >> Turning on by default is pointless because guests are not restricted to
> >> running on specific nodes by default. Since manual intervention is required
> >> to achieve that (right now at least) requiring numa=on is not much of a
> >> hardship.
> > 
> > I'm getting ready to re-submit patches to export the topology information
> > so the userspace tools can use that info to make intelligent selections.
> > This was available back in October, but was never picked up, or even
> > commented upon.
> 
> But can tools make sane automatic decisions on domain creation? And if tools

I don't think the tools would do any worse than what an admin would do:
keep the domains within a node.

> decide not to use NUMA-ness of the system, should the Xen allocator still
> hoover up all the memory of the node that vcpu0 happens to start on?

I'm not sure I understand what you mean by decide to not use the
NUMA-ness.

> 
> My primary concern is simply whether enabling NUMA by default can hurt
> performance, or cause problems by hitting certain memory nodes or memory
> zones harder than others, for the great majority of users who will not use
> NUMA (even if they have a small NUMA system like AMD K8).

Folks without NUMA hardware see the same path through the allocator
today whether they pass numa=on or not.  Last summer, I did [1]overhead
testing specifically on small NUMA systems to address this concern.  I
assumed that those numbers were satisfactory or the patches would not
have been picked up in the first place.

On systems with NUMA, when the domains are kept within a NUMA node, we
see significant performance [2]boost.  I don't have any data to to say
how well a NUMA system would perform with a mixed load of on and off
node access, but when presented with the option of running a domain
entirely within a node on a NUMA system, we should.

What amount of testing is enough for you to consider enabling numa=on by
default post 3.0.5?  I think we ought to cook numa=on by default through
another development cycle so we have time to address any performance
issues that may arise.

1. http://lists.xensource.com/archives/html/xen-devel/2006-07/msg01057.html
2. http://lists.xensource.com/archives/html/xen-devel/2006-09/msg00958.html

-- 
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
ryanh@xxxxxxxxxx


<Prev in Thread] Current Thread [Next in Thread>