Xen project Mailing List

Hi David,

On Mon, Aug 19, 2013 at 8:58 PM, David Vrabel <david.vrabel@xxxxxxxxxx> wrote:

The physical NUMA topology must not be exposed to guests that have a

virtual NUMA topology -- only the toolstack and Xen should know the
mapping between the two.

A guest cannot make sensible use of a machine topology as it may be
migrated to a host with a different topology.

Most of you have the same option that the interface should be in Xen, not in Guest

balloon. I'm agree with it. In next version I'll think of how to implement this interface

between xen and balloon.

It is not clear to me that this is the optimal decision. What
tools/information will be available that the user can use to make
sensible decisions here? e.g., is the current layout available to tools?

Remember that the "user" in this example is most often some automated
process and not a human.

We'd like to have a libxl tool for user, or automated process to change the node affinity

of a Guest. The decision is made by who call this libxl tool.

> +So, right now, the user has no way to specify that he wants to balloon down
> +_G_ in such a way that he will get as much as possible free pages from pnode
> +#2, rather than from pnode #0. He can ask _G_ to balloon down, but there is
> +no guarantee on from what pnode the memory will be freed.
> +
> +The same applies to the ballooning up case, when the user, for some specific
> +reasons, wants to be sure that it is the memory of some (other) specific pnode
> +that will be used.

I would like to see some real world examples of cases where this is
sensible.

In general, I'm not keen on adding ABIs or interfaces that don't solve
real world problems, particularly if they're easy to misuse and end up
with something that is very suboptimal.

Dario, could the test examples that you sent to me several month be represented

as a real-word example?

The example shows that, after several guests create and shut-up, the node

affinity is a mess

The Xenstore interface should be the primary interface being documented.
The libxl interface is secondary and (probably) a consequence of the
xenstore interface.

> +* _pnid_ -- which is the pnode id of which node the user wants to try get some
> + free memory on
> +* _nodeexact_ -- which is a bool specifying whether or not, in case it is not
> + possible to reach the new ballooning target only with memory from pnode
> + _pnid_, the user is fine with using memory from other pnodes.
> + If _nodeexact_ is true, it is possible that the new target is not reached; if
> + it is false, the new target will (probably) be reached, but it is possible
> + that some memory is freed on pnodes other than _pnid_.
> +
> +To let the ballooning driver know about these new parameters, a new xenstore
> +key exists in ~/memory/target\_nid. So, for a proper NUMA aware ballooning
> +operation to occur, the user should write the proper values in both the keys:
> +~/memory/target\_nid and ~/memory/target.

If we decide we do need such control, I think the xenstore interface
should look more like:

memory/target

as before

memory/target-by-nid/0

target for virtual node 0

...

memory/target-by-nid/N

target for virtual node N

I think this better reflects the goal which is an adjusted NUMA layout
for the guest rather than the steps required to reach it (release P
pages from node N).

The balloon driver attempts to reach target, whist simultaneously trying
to reach the individual node targets. It should prefer to balloon
up/down on the node that is furthest from its node target.

In cases where target and the sum of target-by-nid/N don't agree (or are
not present) the balloon driver should use target, and balloon up/down
evenly across all NUMA nodes.

Thew libxl interface does not necessarily have to match the xenstore
interface if that's the initial tools would prefer.

Finally a style comment, please avoid the use of a single gender
specific pronouns in documentation/comments (i.e., don't always use
he/his etc.). I prefer to use a singular "they" but you could consider
"he or she" or using "he" for some examples and "she" in others.

David

Oh, I think this is a better interface!

I'd very appreciate this than what I have now. However, the code I show here

_does_not_really_work_. It just pass some small tests. And I'm afraid that

there may be some bugs in my code.

Could I set up David's interface as a secondary goal, waiting until this code

is full tested? I'm really not that confident :)

Yechen Li

Team of System Virtualization and Cloud Computing
School of Electronic Engineering and Computer Science,
Peking University, China

Nothing is impossible because impossible itself says: " I'm possible "

lccycc From PKU

Re: [Xen-devel] [RFC v2][PATCH 1/3] docs: design and intended usage for NUMA-aware ballooning