WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

[Xen-users] dom0 - oom-killer - memory leak somewhere ?

To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-users] dom0 - oom-killer - memory leak somewhere ?
From: Adrien Urban <adrien.urban@xxxxxxxxxxxxxx>
Date: Thu, 10 Nov 2011 13:56:03 +0100
Delivery-date: Mon, 14 Nov 2011 09:57:44 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.16) Gecko/20111004 Icedove/3.0.11
Hello,

I work in a hosting company, we have tens of Xen dom0 running just fine, but unfortunately we do have a few that get out of control.

Reported behaviour :
- dom0 uses more and more memory
- no process can be found using that memory
- at some point, oom killer kicks in, and kills everything, until even ssh the box becomes hard - when there is really no more process to kill, it crashes even more, and we are forced to reboot

Configuration summary :
- dom0 with debian/stable, xen 4.0.1
- 512MB, or up to 2GB after some crash


I have tryed to find something that differs between a working dom0 and a buggy one, but didn't manage to find anything. Install from the same template, same packages, same hardware (but serials and mac addresses).


I didn't manage to find anything about leak in dom0 ending up with oom killer without doubt.

I tried to gather as much log as i thought could be helpful in attachments.
Host bk - about to get a reboot, as xend already got killed
Host sw - 800MB/2GB used for nothing,

Attachments contains :

- memory graph (by munin) - it might help to see the pattern of memory usage

cat from :
- grub.cfg
- /proc/meminfo
- /proc/slabinfo
- /proc/vmstat
- /var/log/kern.log
- /var/log/xen/xend.log

Result from :
- dmesg
- dpkg -l
- free
- lsmod
- top
- vmstat
- xm info
- xm info -c


I'd appreciate any feedback about such behaviour, and would be happy to provide additional information. Those are productions servers, the only thing i'd really like to avoid as much as possible is rebooting them for tests.


Regards,

--
Adrien URBAN

Attachment: bk.tar.bz2
Description: Binary data

Attachment: sw.tar.bz2
Description: Binary data

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
<Prev in Thread] Current Thread [Next in Thread>