This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-users] xen-4.0.1 from make world - hypervisor alone consumes around

To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-users] xen-4.0.1 from make world - hypervisor alone consumes around 800 MB RAM , OOMs when try to baloon to lower amount, no domUs running yet
From: Petr Vacek <tachyon.eagle@xxxxxxxxx>
Date: Thu, 20 Jan 2011 00:32:40 +0100
Delivery-date: Wed, 19 Jan 2011 15:34:30 -0800
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:from:date:message-id:subject:to :content-type; bh=qbd1A7zAFMlqxrC805ZE0uaGKp9GS3qQN5UtRfGWFdU=; b=TA7EPMCzbiZ1Ba9JrIdkHUE0rkP3/nq4ArC0CL3GutAc0wTXV+t/SocLw/7nD4yScb kvPmLfiNgQdQ07dBTZqRymEmc8WkK8R69j6cicnp31Fnzb775I4H1dUzmO7ux91AM3Ee z/I202ohWJwd5MV1p5muRsS7r/W8U/r9CU8AQ=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:from:date:message-id:subject:to:content-type; b=Kvhx02d2M571+V0rpbrnoJrh/fH+nCD1HXuVr4C2kH542vMiKndQwPfRvgoa3nocWo jN9gurt5/gDas3v5EGRa2E42BKcoXH0iXG1cuegTU4eudWWHecJJOE/H9VwlcMksKPru gaAGuO240qFhI3BpUzPHD8WyVhFAz1PNaRIdI=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
I've just setuped a new AMD opteron 6168 12-core server with Xen-4.0.1 - downloading sources , running 'make world' - adding raid+ata modules into kernel and running ...
Everything works fine (filesystem,raid,domUs crunching numbers) , save for one troubling memory issue with dom0 :

When running with hypervisor enabled , I see used memory as 860 MB (in top) right after linux start, even without Xend service and any domU running,
when I strech domain 0 under 800 MB , I get OOM killer and restart.  This way I know that I cannot possibly use this memory and it's eaten up -
I cannot give it to domU certainly .

I can see this amount in meminfo in directMap4k , but it does not add up with other parts of meminfo to me - it does not belong to any visible process.
When I use same booting configuration without Xen hypervisor, this memory usage goes away - linux eats about 60 MB , only service running is SSH and very basic Ubuntu 10.10.
Kernel is from xen sources : - kernel config as here ( http://pavouk.mediawork.cz/xen/2011/kernel-config.txt )
I've placed dmesg, xm dmesg, dmidecode machine information here: http://pavouk.mediawork.cz/xen/2011/  

I've tried building 4.0.2 - memory issue was still present + kernel bugs so I reverted to older 4.0.1 runnning otherwise stable for my setup.

Could this be caused by some kernel misconfiguration or xen hypervisor misconfiguration ?

I've noticed some warning during initialization , but I think it's something different :
Performance Events: AMD PMU driver.                                                                                                                             
[    0.006998] ------------[ cut here ]------------                                                                                                                            
[    0.006998] WARNING: at /opt/xen/xen-4.0.1/linux-2.6-pvops.git/arch/x86/xen/enlighten.c:729 xen_apic_write+0x15/0x17()                                                      
[    0.006998] Hardware name: H8SGL                                                                                                                                            
[    0.006998] Modules linked in:                                                                                                                                              
[    0.006998] Pid: 0, comm: swapper Not tainted #9                                                                                                                  
[    0.006998] Call Trace:                                                                                                                                                     
[    0.006998]  [<ffffffff8107970b>] warn_slowpath_common+0x77/0x8f                                                                                                            
[    0.006998]  [<ffffffff81079732>] warn_slowpath_null+0xf/0x11                                                                                                               
[    0.006998]  [<ffffffff810355c2>] xen_apic_write+0x15/0x17                                                                                                                  
[    0.006998]  [<ffffffff8104872a>] perf_events_lapic_init+0x2e/0x30                                                                                                          
[    0.006998]  [<ffffffff81a976a2>] init_hw_perf_events+0x33e/0x3db                                                                                                           
[    0.006998]  [<ffffffff81a971c0>] identify_boot_cpu+0x3c/0x3e                                                                                                               
[    0.006998]  [<ffffffff81a97340>] check_bugs+0x9/0x2d                                                                                                                       
[    0.006998]  [<ffffffff81a8ddbf>] start_kernel+0x432/0x447                                                                                                                  
[    0.006998]  [<ffffffff81a8d2c1>] x86_64_start_reservations+0xac/0xb0                                                                                                       
[    0.006998]  [<ffffffff81a91ad0>] xen_start_kernel+0x5b7/0x5be                                                                                                              
[    0.006998] ---[ end trace 4eaa2a86a8e2da22 ]---

The troubling meminfo dump:

cat /proc/meminfo :
MemTotal:        1229376 kB
MemFree:          345440 kB
Buffers:             412 kB
Cached:             8572 kB
SwapCached:            0 kB
Active:            45304 kB
Inactive:          14112 kB
Active(anon):      38464 kB
Inactive(anon):    12304 kB
Active(file):       6840 kB
Inactive(file):     1808 kB
Unevictable:          20 kB
Mlocked:              20 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:               948 kB
Writeback:             0 kB
AnonPages:         50508 kB
Mapped:             6600 kB
Shmem:               320 kB
Slab:              15904 kB
SReclaimable:       5236 kB
SUnreclaim:        10668 kB
KernelStack:        1016 kB
PageTables:         1664 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      614688 kB
Committed_AS:     153440 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      264292 kB
VmallocChunk:   34359473836 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:     8647280 kB
DirectMap2M:           0 kB

Thanks in advance for any hints how to tamper this memory burden
Petr Vacek
Xen-users mailing list
<Prev in Thread] Current Thread [Next in Thread>
  • [Xen-users] xen-4.0.1 from make world - hypervisor alone consumes around 800 MB RAM , OOMs when try to baloon to lower amount, no domUs running yet, Petr Vacek <=