Re: [Xen-devel] [PATCH v2 08/16] xen: derive NUMA node affinity from hard and soft CPU affinity

[George Dunlap <george.dunlap@xxxxxxxxxxxxx>]

On 14/11/13 16:30, Dario Faggioli wrote:
> On gio, 2013-11-14 at 15:21 +0000, George Dunlap wrote:
> > > Signed-off-by: Dario Faggioli <dario.faggioli@xxxxxxxxxx>
> > > diff --git a/xen/common/domain.c b/xen/common/domain.c
> > > @@ -373,8 +379,12 @@ void domain_update_node_affinity(struct domain *d)
> > >    
> > >        for_each_vcpu ( d, v )
> > >        {
> > > +        /* Build up the mask of online pcpus we have hard affinity with */
> > >            cpumask_and(online_affinity, v->cpu_hard_affinity, online);
> > >            cpumask_or(cpumask, cpumask, online_affinity);
> > > +
> > > +        /* As well as the mask of all pcpus we have soft affinity with */
> > > +        cpumask_or(cpumask_soft, cpumask_soft, v->cpu_soft_affinity);
> > Is this really the most efficient way to do this?  I would have thought
> > or'ing cpumask and cpumask_soft, and then if it's not empty, then use
> > it; maybe use a pointer so you don't have to copy one into the other one?
> I'm not sure I fully get what you mean... I cannot afford neglecting
> online_affinity, independently from how cpumask and cpumask_soft look
> like, because that's what's needed to account for cpupools. Anyway, I'll
> think more about it and see if I can make it better.
So what you have here is (in pseudocode):

 online_affinity = hard_affinity & online;
 cpumask |= online_affinity;
 cpumask_soft |= soft_affinity;

if ( intersects(cpumask, cpumask_soft) )
  cpumask &= cpumask_soft;

So at least four full bitwise operations, plus "intersects" which will 
be the equivalent of a full bitwise operation if it turns out to be false.
How about something like the following:

cpumask_hard = hard_affinity & online;
cpumask_soft = cpumask_hard & soft_affinity;

cpumask_p = is_empty(cpumask_soft) ? &cpumask_hard : &cpumask_soft;

So only two full bitwise operations, plus "is_empty", which I think 
should be faster than "intersects" even if it turns out to be true, 
since we can (I think) check full words at a time.
Then use cpumask_p in the subsequent node_affinity update.  (Or just do 
a full copy, since doing the nodewise check for intersection is going to 
be fairly slow anyway.)
 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel