WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] Re: Please help: domU becomes unresponsive

To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] Re: Please help: domU becomes unresponsive
From: "Jerry Amundson" <jamundso@xxxxxxxxx>
Date: Thu, 17 Jul 2008 13:02:11 -0500
Delivery-date: Thu, 17 Jul 2008 11:02:40 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=KhqoSwihwQarIgoZHFfJLYzdI4pTItlQtAqn0//TjJ4=; b=czXWEjlHb6xAkvpFLK7lqy+WdNesHNXcuiLlRWDVDXIqUwWguSAOVan7Zj4hgXh/p0 zzIx0pMzk1Cq/uBZdGZI4PlzF1Kg0OTtSwvDZjqPYvMPwo40gjOPyCwNz86y2bESNJ8P 8BcOdFfgQJ1d+vimuVulBX9Yrixj1qTlGYBQo=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=mhz5SME8DVT6ziqpE0PbnjFk8ofIIxzzkZQAIgXr07B/iIIBrUXq+VnHy1d/Epq2KH ozs9gDEBAWJ3Nz0jIQz4X7CESNVz+g3CykWm5+Eozn0DJUAcGcYyzwTkfc4IJtkhK+sk r5ZeMb8Tzg+I/YMmo386TThMYCYrUZqp6z2A8=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <6d06ce20807160750l6bef0809o18d12c4a8860fb3c@xxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <6d06ce20807160750l6bef0809o18d12c4a8860fb3c@xxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
More info from crash... Can someone enlighten me as to what [swapper]
is attempting here?

   PID    PPID  CPU       TASK        ST  %MEM     VSZ    RSS  COMM
      0      0   0  ffffffff80322b40  RU   0.0       0      0  [swapper]
      0      1   1  ffffff800e7b1030  RU   0.0       0      0  [swapper]
      0      1   2  ffffff80006727f0  RU   0.0       0      0  [swapper]
      0      1   3  ffffff8000672030  RU   0.0       0      0  [swapper]
      0      1   4  ffffff80000057f0  RU   0.0       0      0  [swapper]
      0      1   5  ffffff8000005030  RU   0.0       0      0  [swapper]
      1      0   2  ffffff800e7b17f0  IN   0.0    4756    552  init
      2      1   0  ffffff80000037f0  IN   0.0       0      0  [migration/0]
      3      1   0  ffffff8000003030  IN   0.0       0      0  [ksoftirqd/0]
      4      1   0  ffffff80000a67f0  IN   0.0       0      0  [events/0]
      5      1   1  ffffff80000a6030  IN   0.0       0      0  [khelper]
      6      1   4  ffffff82800037f0  IN   0.0       0      0  [kthread]
      7      6   0  ffffff8280003030  IN   0.0       0      0  [xenwatch]
      8      6   0  ffffff82800127f0  IN   0.0       0      0  [xenbus]
     16      6   1  ffffff8280012030  IN   0.0       0      0  [migration/1]
     17      6   1  ffffff82800677f0  IN   0.0       0      0  [ksoftirqd/1]
     18      6   1  ffffff8280067030  RU   0.0       0      0  [events/1]
     20      6   2  ffffff82800697f0  IN   0.0       0      0  [migration/2]
     21      6   2  ffffff8280069030  IN   0.0       0      0  [ksoftirqd/2]
     22      6   2  ffffff82800b67f0  IN   0.0       0      0  [events/2]
     24      6   3  ffffff82800b6030  IN   0.0       0      0  [migration/3]
     25      6   3  ffffff80007047f0  IN   0.0       0      0  [ksoftirqd/3]

On Wed, Jul 16, 2008 at 9:50 AM, Jerry Amundson <jamundso@xxxxxxxxx> wrote:
> Hi all, sorry to intrude on xen-devel, but I think I need direction
> from the expertise here. I've admin'd Xen servers of various flavors
> for a couple years, but never seen this before. After a period ranging
> from several hours to several days, my  primary database and
> development DomU completely locks up. Net disconnects, but CPU(sec)
> continues to tick in xentop. No errors, and nothing logged. All dom's
> are CentOS, so I'm pasting below what I've already posted to
> centos-devel and centos-virt.
>
> On Mon, Jul 14, 2008 at 3:49 PM, Jerry Amundson <jamundso@xxxxxxxxx> wrote:
>> Two Dell 6950 (now called R905, 4 Dual-Core AMD Opteron 8200 series)
>> heartbeat/drbd nodes running the stock CentOS 5.2 Dom0. The domU's are
>> the only resources in heartbeat.
>> Dom1 is a perfectly running, updated, CentOS 5.2 Apache/MySQL/Samba
>> Dom2 is a CentOS 4.6 software development and database server
>
> So crash tells me that Dom2 gets to this point:
>  SYSTEM MAP: System.map-2.6.9-67.0.20.ELxenU
> DEBUG KERNEL: /usr/lib/debug/lib/modules/2.6.9-67.0.20.ELxenU/vmlinux
> (2.6.9-67.0.20.ELxenU)
>   DUMPFILE: /public/IntSys/tmp/m1.dmp
>       CPUS: 6
>       DATE: Mon Jul 14 11:53:59 2008
>     UPTIME: 6 days, 11:39:33
> LOAD AVERAGE: 548.07, 542.95, 434.99
>      TASKS: 2721
>   NODENAME: monolith
>    RELEASE: 2.6.9-67.0.20.ELxenU
>    VERSION: #1 SMP Thu Jun 26 08:36:44 EDT 2008
>    MACHINE: x86_64  (2194 Mhz)
>     MEMORY: 10 GB
>      PANIC: ""
>        PID: 0
>    COMMAND: "swapper"
>       TASK: ffffffff80322b40  (1 of 6)  [THREAD_INFO: ffffffff80426000]
>        CPU: 0
>      STATE: TASK_RUNNING
>    WARNING: panic task not found
>
> crash> bt
> PID: 0      TASK: ffffffff80322b40  CPU: 0   COMMAND: "swapper"
>  #0 [ffffffff80427ec0] schedule at ffffffff80294d9a
>  #1 [ffffffff80427f98] cpu_idle at ffffffff8010b85d
> crash> kmem -i
>             PAGES        TOTAL      PERCENTAGE
>  TOTAL MEM  2621696        10 GB         ----
>     FREE     8884      34.7 MB    0% of TOTAL MEM
>     USED  2612812        10 GB   99% of TOTAL MEM
>   SHARED        0            0    0% of TOTAL MEM
>  BUFFERS    59585     232.8 MB    2% of TOTAL MEM
>   CACHED  1325825       5.1 GB   50% of TOTAL MEM
>     SLAB   358565       1.4 GB   13% of TOTAL MEM
>
> TOTAL HIGH        0            0    0% of TOTAL MEM
>  FREE HIGH        0            0    0% of TOTAL HIGH
>  TOTAL LOW  2621696        10 GB  100% of TOTAL MEM
>  FREE LOW     8884      34.7 MB    0% of TOTAL LOW
>
> kmem: swap_info[0].swap_map at ffffff00001ea000 is unaccessible
>
> So I see where the DomU is, but how did it get there? Can I find out
> from crash, or do I need something "real-time" within the DomU? Of
> course, searching has given me nothing to go on, hence this post, but
> I'll continue...
>
> Tia,
> jerry
>
> --
> "Your life is trite and jaded, boring and confiscated." - Twisted Sister
>



-- 
"Your life is trite and jaded, boring and confiscated." - Twisted Sister

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel