WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] RE: tmem - really default to on?

To: Jan Beulich <JBeulich@xxxxxxxxxx>
Subject: [Xen-devel] RE: tmem - really default to on?
From: Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>
Date: Thu, 4 Feb 2010 13:01:06 -0800 (PST)
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx, Keir Fraser <keir.fraser@xxxxxxxxxxxxx>
Delivery-date: Thu, 04 Feb 2010 13:03:16 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4B669BDB020000780002CEFC@xxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Sorry for the delay... I was away from the office.

> >Rather than turn it off in Xen, maybe best solution is to turn it
> >off in non-pvops dom0, at least by default.  Or really there's
> >no reason it needs to be on for dom0 at all, so Xen code could
> >check for dom0 and disable it if dom0.
> 
> I don't know that much about tmem, but why would Dom0 not benefit
> from it being enabled if there's memory available in Xen?

Oops, yes, I misspoke.  There are uses for tmem from dom0,
especially when other guests are using "file:" virtual disks
(i.e. *not* O_DIRECT) and those other guests are not tmem-aware.

> >And the tmem code that limits to single page allocation should IIRC
> >only be invoked once all system memory has been absorbed (which
> >I suppose could be the case for dom0 when dom0_mem is not specified?)
> 
> Or if enough VMs had been created to consume all freely available
> (outside of tmem) memory.
>
> As said in the original mail - we have seen systems with more than 2G
> "freeable" memory still causing domain creation to fail with 
> "not enough memory".

I can guess at a few ways this could happen, but would probably
need more info about the specific environment to be sure:

A) There is some order>0 memory allocation in Xen domain creation
that doesn't fall back to order=0, that I've not seen in my testing
but shows up in your systems (or has very recently been added).
If true, this is a problem not only for tmem but also for all other
memory optimization work and we need to identify and (if possible)
fix it.

B) The code in domain creation that frees freeable memory (see
tools/python/xen/xend/balloon.py, search for "tmem") is computing
a too-small value, maybe due to recent changes reserving "hidden
extra" memory for domains).

C) The above code is not getting executed at all during domain
creation, perhaps because you have different/old domain creation
tools that aren't aware of tmem (or use some API that
I didn't know about and consequently didn't test).

Any thoughts?

Thanks,
Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel