Re: [Xen-devel] [PATCH 11 of 11] Some automatic NUMA placement documentation

[Dario Faggioli <raistlin@xxxxxxxx>]

On Thu, 2012-05-31 at 16:08 +0100, Ian Jackson wrote:
> Dario Faggioli writes ("[PATCH 11 of 11] Some automatic NUMA placement 
> documentation"):
> > About rationale, usage and API.
> ...
> > +## Guest Placement in libxl ##
> 
> Oh here's the API documentation!
> 
Yep. :-)

> In general I would prefer to see docs come in the same patch but I
> guess I can read it here:
> 
I can put it there. Would you recommend _moving_ it in the source file o
also leaving a copy, o maybe just some pointers, here?

> > +xl achieves automatic NUMA placement by means of the following API
> > +calls, provided by libxl.
> 
> Can you try to write some more general comment about what order these
> functions should be called in ?
> 
Yes I can, good point.

<snip>

> > +        int libxl_numa_candidate_add_cpus(libxl_ctx *ctx,
> > +                                          int min_cpus, int max_nodes,
> > +                                          libxl_numa_candidate *candidate);
> > +
> > +This is what should be used to ensure a placement candidate has at least
> > +min_cpus CPUs. In case it does not, the function also take care of
> > +adding more nodes to the candidate itself (up to when the value specified
> > +in max_nodes is reached). When adding new nodes, the one that has the
> > +smallest "distance" from the current node map is selected at each step.
> 
> `add_cpus' doesn't seem the same as `ensure a candidate has at least
> min_cpus CPUs'.  In what sense are the CPUs added ?
> 
In the sense that, if you have a candidate placement which comprises,
say, 2 nodes, summing up to 12 CPUs, but you want your VM to have access
to a minimum of 16 CPUs the this checks that and add more nodes to the
candidate placement until it contains enough of them for giving you the
number of CPUs you asked for.

> And, as before, why might I want to call this ?  And when would I call
> it ?  Why does the interface to libxl expose this rather than just
> offering a single function
> 
>  libxl_do_automatic_numa_placement_according_to_heuristics
> 
> ?
> 
Right, that is of couse an option, and I thought about going that way
for quite a while. However, what I think is best for libxl is to provide
a set of building blocks for its user to be able to implement the actual
heuristics.

That's why I tried to offer a 'generate the placement candidates' and
'manipulate the placement candidate' kind of API, to make it possible
for lower levels to use it when putting their own placing algorithm
together. It is also kind of extendable, I mean, we can always add
another 'manipulate the candidates in some other way' function if needed
or wanted by us or whatever other libxl user.

Doing it the other way, i.e., one big function doing everything, would
mean that as soon as we want to change or improve the placement
heuristics, we need to modify the behaviour of that API call, which I
think it is suboptimal. Also, if some other user of libxl does not like
the heuristics I came out with, they have to reimplement the whole
placement.

So, like it is right now, the actual heuristics is implemented in xl, on
top of these placement candidate generation and manipulation facilities,
which I finally decided it was the way I liked this whole thing
most. :-)

Again, you're right in asking this reasoning to be part of the
documentation, and to be put in the proper place, and I will do that.
However, now that I've put it here, do you think it makes some sense?

> > +        libxl_numa_candidate_count_domains(libxl_ctx *ctx,
> > +                                           libxl_numa_candidate 
> > *candidate);
> > +
> > +This is what counts the number of domains that are currently pinned
> > +to the CPUs of the nodes of a given candidate.
> 
> Why is that useful ?  It is used by some of your other code so I guess
> this is a facility which is useful to the implementors of other
> placement algorithms ?
> 
As I tried to explain above, yes, exactly like that.

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel