Re: eliminating 166G limit (was Re: [Xen-devel] Problem withnr_n

To:	"Krysan, Susan" <KRYSANS@xxxxxxxxxx>
Subject:	Re: eliminating 166G limit (was Re: [Xen-devel] Problem withnr_nodes on large memory NUMA machine)
From:	"Yinghai Lu" <yhlu.kernel@xxxxxxxxx>
Date:	Sat, 10 May 2008 15:38:37 -0700
Cc:	Keir Fraser <Keir.Fraser@xxxxxxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxx, eak@xxxxxxxxxx
Delivery-date:	Sat, 10 May 2008 15:38:59 -0700
Dkim-signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=6HSabhYhOBxLtfmXnNzm5citfhk8wsRsxSlk9b8HIco=; b=amBufrTIfVCLQI0k69CnOxDya6nF7sLkom0teq2StjneyOO2U+QrPIkreIhriwJf9BpwQ1mjnlEr3jDM05jsB3zJ2hf4+i2GOtqChihqobkSyd38E1Ouhmy8v95Zkk9IO0dWpd/bKqQD5ubcnZNODu0QGEFb7RFqdqvZXkP/Hd4=
Domainkey-signature:	a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=OQJM4XZ5NuQwMtfV3uvkluK8x7vFnm+PsxMJvSKTvzNR76qSkpYrt6xZR0JFXF4YXPzp+9ow+oiG4avtMk/nMpFLI7e5ANxl38xfYNhmXgdsN1KHfdk8p9pr1YsfclrepHyti5eiy1mI1aI+ihBzIe8isVi9GOwFZ55vzIaei1Y=
Envelope-to:	www-data@xxxxxxxxxxxxxxxxxx
In-reply-to:	<CE091CB077F4A74A9CBA33A4E4A8A3BF0C8286DF@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>
List-help:	<mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id:	Xen developer discussion <xen-devel.lists.xensource.com>
List-post:	<mailto:xen-devel@lists.xensource.com>
List-subscribe:	<http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe:	<http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References:	<47545DE2.2080509@xxxxxxxxxx> <C37DAC49.197EB%Keir.Fraser@xxxxxxxxxxxx> <CE091CB077F4A74A9CBA33A4E4A8A3BF0C8286DF@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>
Sender:	xen-devel-bounces@xxxxxxxxxxxxxxxxxxx

On Fri, Dec 7, 2007 at 6:20 AM, Krysan, Susan <KRYSANS@xxxxxxxxxx> wrote:
> I tested this changeset on Unisys ES7000 with 256G ram and 64 processors
> and it works:
>
> xentop - 06:30:59   Xen 3.2-unstable
> 1 domains: 1 running, 0 blocked, 0 paused, 0 crashed, 0 dying, 0
> shutdown
> Mem: 268172340k total, 7669456k used, 260502884k free    CPUs: 64 @
> 3400MHz
>
> I will be running our full test suite on this configuration today.
>
> Thanks,
> Sue Krysan
> Linux Systems Group
> Unisys Corporation
>
>
> -----Original Message-----
> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Keir Fraser
> Sent: Thursday, December 06, 2007 8:40 AM
> To: eak@xxxxxxxxxx
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: eliminating 166G limit (was Re: [Xen-devel] Problem
> withnr_nodes on large memory NUMA machine)
>
> Try xen-unstable changeset 16548.

Susan,

how about TOM? i mean MMIO size...

my system with 256g/32cores, and 0xe0000000 - 0x100000000 is mmio
mtrr
reg00: base=0x00000000 (   0MB), size=2048MB: write-back, count=1
reg01: base=0x80000000 (2048MB), size=1024MB: write-back, count=1
reg02: base=0xc0000000 (3072MB), size= 512MB: write-back, count=1
TOM: 00000000e0000000 aka 3584M
TOM2: 0000004020000000 aka 262656M

only can work with mem=200g

mem=224g, xen kernel will say alloc_bootmem_low can not get range..

xen kernel is from

http://people.redhat.com/dzickus/el5/92.el5/x86_64/

YH

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

WARNING - OLD ARCHIVES

xen-devel

Re: eliminating 166G limit (was Re: [Xen-devel] Problem withnr_nodes on