Re: [Xen-devel] [PATCH 6 of 8 [RFC]] libxc: introduce xc_domain_move_memory

[Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>]

> From: Dario Faggioli [mailto:dario.faggioli@xxxxxxxxxx]
> Subject: [PATCH 6 of 8 [RFC]] libxc: introduce xc_domain_move_memory

(NUMA discussion...)

> 
> XXX Open issues are:
>      - TMEM: how to "move" it?

(Konrad added to cc list.)

Tmem memory is, by definition, the lowest priority memory
for the domain and the hypervisor may already be storing it as
efficiently as possible (i.e. the page may be deduplicated).
When it is accessed by the domain (it is never directly
addressable by a domain, and a hypercall is required
to access it), an entire page is sequentially copied from
a physical page in the hypervisor to the domain.  Juergen may
know otherwise, but I'd guess this inter-node copy would be
very efficiently pipelined, cache-line by cache-line
possibly even with hardware pre-fetching.

So the best answer to "how to move it?" may be "don't
move it at all!".  In fact, a good design for a NUMA-aware
implementation of tmem might intentionally store the data on
"any node other than the node making this tmem-put hypercall".

Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel