[Xen-devel] [PATCH RFC 0/7] vnuma topology support

[Elena Ufimtseva <ufimtseva@xxxxxxxxx>]

This series of patches introduces vNUMA topology implementation and
provides interfaces and data structures, exposing to PV guest virtual topology
and enabling guest OS to use its own NUMA placement mechanisms.

vNUMA topology support for Linux PV guest comes in a separate patch.

Please review and send your comments.

Introduction
-------------

vNUMA topology is exposed to the PV guest to improve performance when running
workloads on NUMA machines.
XEN vNUMA implementation provides a way to create vNUMA-enabled guests on 
NUMA/UMA
and map vNUMA topology to physical NUMA in a optimal way.

XEN vNUMA support

Current set of patches introduces subop hypercall that is available for 
enlightened
PV guests with vNUMA patches applied.

Domain structure was modified to reflect per-domain vNUMA topology for use in 
other
vNUMA-aware subsystems (e.g. ballooning).

libxc

libxc provides interfaces to build PV guests with vNUMA support and in case of 
NUMA
machines provides initial memory allocation on physical NUMA nodes. This 
implemented by
 trying to allocate all vnodes on one NUMA node. In case of insufficient 
memory, vnodes
are allocated on other physical NODE with enough memory. If none of physical 
nodes
has enough memory, the memory allocation is done using default mechanism and 
vnodes
may have pages allocated on different nodes.

libxl

libxl provides a way to predefine in VM config vNUMA topology - number of 
vnodes,
memory arrangement, vcpus to vnodes assignment, distance map.

PV guest

As of now, only PV guest can take advantage of vNUMA functionality. vNUMA Linux 
patches
should be applied and NUMA support should be compiled in kernel.


Patchset v. 0.1
---------------

[PATCH RFC 1/7] xen/vNUMA: hypercall and vnuma topology structures
Defines XENMEM subop hypercall for PV vNUMA enabled guests and provides
vNUMA topology information from per-domain vnuma topology build info.

[PATCH RFC 2/7] xen/vnuma: domctl subop for vnuma setup
Defines domctl subop hypercall for per-domain vNUMA topology construct

[PATCH RFC 3/7] libxc/vnuma: domain per-domain vnuma structures.
Makes use of domctl vnuma subop and initializes per-domain vnuma topology.

[PATCH RFC 4/7] libxl/vnuma: vnuma domain config and pre-build
Defines VM config options for vNUMA PV domain creation as follows:

[PATCH RFC 5/7] libxl/vnuma: vnuma enabler
Enables libxl vnuma ABI by LIBXL_HAVE_BUILDINFO_VNUMA

[PATCH RFC 6/7] libxc/vnuma: vnuma per phys NUMA allocation.
Allows for vNUMA enabled domains to allocate vnodes on physical NUMA nodes.
Tries to allocate all vnodes on one NUMA node, or on next one if not all
vnodes fit. If no physical numa node found, will let xen decide.

[PATCH RFC 7/7] xen/vnuam: basic vnuma debug info
Prints basic vnuma info per domain on 'debug-keys u'.

TODO:
---------------
- initial boot mem allocation alghoritm for vNUMA nodes on physical nodes;
- linux vnuma memblocks and e820 holes needs testing;
- move XENMEM subop hypercall in xen to sysctl subop;
- some kind of statistics for vnuma enabled guests, xl info/list;
- take into account cpu pinning if defined in VM config;
- take into account automatic NUMA placement mechanism;
- arch dependend pieces tests;
- help files;

Elena Ufimtseva (7):
  xen/vnuma: subop hypercall and vnuma topology structures.
  xen/vnuma: domctl subop for vnuma setup.
  libxc/vnuma: per-domain vnuma structures.
  libxl/vnuma: vnuma domain config
  libxl/vnuma: vnuma enabler.
  libxc/vnuma: vnuma per phys NUMA allocation.
  xen/vnuma: basic vnuma debug info

 tools/libxc/xc_dom.h         |   10 +++
 tools/libxc/xc_dom_x86.c     |   79 +++++++++++++++--
 tools/libxc/xc_domain.c      |   63 ++++++++++++++
 tools/libxc/xenctrl.h        |   17 ++++
 tools/libxc/xg_private.h     |    4 +
 tools/libxl/libxl.c          |   28 ++++++
 tools/libxl/libxl.h          |   23 +++++
 tools/libxl/libxl_arch.h     |    6 ++
 tools/libxl/libxl_dom.c      |  115 ++++++++++++++++++++++--
 tools/libxl/libxl_internal.h |    3 +
 tools/libxl/libxl_types.idl  |    6 +-
 tools/libxl/libxl_x86.c      |   91 +++++++++++++++++++
 tools/libxl/xl_cmdimpl.c     |  197 +++++++++++++++++++++++++++++++++++++++++-
 xen/arch/x86/numa.c          |   16 +++-
 xen/common/domain.c          |    6 ++
 xen/common/domctl.c          |   72 ++++++++++++++-
 xen/common/memory.c          |   90 ++++++++++++++++++-
 xen/include/public/domctl.h  |   15 +++-
 xen/include/public/memory.h  |    1 +
 xen/include/public/vnuma.h   |   12 +++
 xen/include/xen/domain.h     |    9 ++
 xen/include/xen/sched.h      |    1 +
 xen/include/xen/vnuma.h      |   27 ++++++
 23 files changed, 871 insertions(+), 20 deletions(-)
 create mode 100644 xen/include/public/vnuma.h
 create mode 100644 xen/include/xen/vnuma.h

-- 
1.7.10.4


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel