WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] Random oopsen with Xen on Ubuntu Gutsy

To: "Matt Mackall" <mpm@xxxxxxxxxxx>
Subject: Re: [Xen-users] Random oopsen with Xen on Ubuntu Gutsy
From: "Todd Deshane" <deshantm@xxxxxxxxx>
Date: Thu, 7 Feb 2008 12:18:57 -0500
Cc: xen-users@xxxxxxxxxxxxxxxxxxx, Dave Rolsky <autarch@xxxxxxxx>
Delivery-date: Thu, 07 Feb 2008 09:19:32 -0800
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=RVaFBM5a2kVN19xMx5R02TXo0IOOfgnxdzcoxYnosaw=; b=AhBRvRN9FPStwjlG8+O7jrVt+qDVEmuTuYqQeI69PihS92T1+KvVqfK6dwsGMzVhmDCvOsWcfCcq60q2HfTH6AKL04WnuJZZgU9uU0cX2EdR9r3Xte5k7HSEso8lQXb6IeolKB88NG9S5ZUqwHHKmduNbQPXuu/koGp4gywRMjg=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=CGSV1tZF6o/NZ1H53P1LnQA4OUQtw1xSrQDodXPg2vYL1Qu39amYLNbSchfWuTc2j/g07xzpQ+WfT/DTs3aa+iFamFLTMdlVhpe9ZXwdwKXrJYEdVHaaFNK7tH0ej+9CmvbE2B4ij+sDei5eKwz2+pw9WtGEmp4UsPScbekzmeQ=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <1202246684.17934.228.camel@xxxxxxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <1202246684.17934.228.camel@xxxxxxxxxxxxxxxx>
Reply-to: deshantm@xxxxxxxxx
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
Can you post the below to https://launchpad.net/ubuntu

I was running into something similar wasn't able to save the stack trace.

My error was when doing simple I/O, but the strace looks similar.

The gutsy kernel was working fine during alpha until they moved to 2.622+

I fought was it for awhile and got past some bugs. not all yet.

I am also using AMD64.

Todd


On Feb 5, 2008 4:24 PM, Matt Mackall <mpm@xxxxxxxxxxx> wrote:
> I've got an AMD64 machine that's exhibiting random oopses and database
> corruption using 2.6.22-14 kernel from Ubuntu Gutsy and Xen
> 3.1.0-0ubuntu18 also from Gutsy,
>
> Running the same database without Xen under heavy load appears to work
> fine. Machine has 3G of ECC and memtest86 works fine as well. Also,
> machine ran the database just fine for a year before (unfortunately
> simultaneously) switching to Gutsy and Xen.
>
> Any suggestions?
>
> MySQL said:
>
> The relevant bit of the logs is this:
>
>   Jan 10 23:46:26 vegguide mysqld[22376]: InnoDB: Database page corruption on 
> disk or a failed
>   Jan 10 23:46:26 vegguide mysqld[22376]: InnoDB: file read of page 1559.
>   Jan 10 23:46:26 vegguide mysqld[22376]: InnoDB: You may have to recover 
> from a backup.
>
> A couple of the oopsen pasted below:
>
> [297993.758358] BUG: unable to handle kernel NULL pointer dereference at 
> virtual address 00000001
> [297993.758381]  printing eip:
> [297993.758392] 006c6000 -> *pde = 00000000:6d67f001
> [297993.758398] 27f46000 -> *pme = 00000000:00000000
> [297993.758405] Oops: 0000 [#1]
> [297993.758408] SMP
> [297993.758415] Modules linked in: af_packet xt_multiport iptable_filter 
> ip_tables x_tables ipv6 evdev ext3 jbd mbcach    e dm_mirror dm_snapshot 
> dm_mod fuse apparmor commoncap
> [297993.758448] CPU:    0
> [297993.758449] EIP:    0061:[<c0176eec>]    Not tainted VLI
> [297993.758452] EFLAGS: 00010002   (2.6.22-14-xen #1)
> [297993.758468] EIP is at kmem_cache_alloc+0x5c/0xe0
> [297993.758474] eax: 00000000   ebx: 00000001   ecx: 00000000   edx: c1bfe8a0
> [297993.758480] esi: 00000000   edi: 00000000   ebp: 00000020   esp: e0003c00
> [297993.758487] ds: 007b   es: 007b   fs: 00d8  gs: 0000  ss: 0069
> [297993.758493] Process pdflush (pid: 4538, ti=e0002000 task=c210ea60 
> task.ti=e0002000)
> [297993.758498] Stack: 00000000 c1c34f60 c028b35d 00000000 c20b4b40 00000003 
> c20b4b40 c028b35d
> [297993.758515]        c12a7480 c20b4b40 00000003 c029fd40 c029f25c c013bdba 
> c03d26a0 c2048000
> [297993.758532]        c03ec4a8 c20b4b40 00000000 00000400 c0290b67 c2048000 
> 00000000 003d0900
> [297993.758549] Call Trace:
> [297993.758555]  [<c028b35d>] skb_clone+0x2d/0x250
> [297993.758567]  [<c028b35d>] skb_clone+0x2d/0x250
> [297993.758575]  [<c029fd40>] snap_rcv+0x0/0xa0
> [297993.758582]  [<c029f25c>] llc_rcv+0xdc/0x2b0
> [297993.758589]  [<c013bdba>] clocksource_get_next+0x3a/0x40
> [297993.758602]  [<c0290b67>] netif_receive_skb+0x237/0x420
> [297993.758613]  [<c027119d>] netif_poll+0x4fd/0xbf0
> [297993.758621]  [<c010891b>] sched_clock+0x3b/0x80
> [297993.758630]  [<c011e5f3>] scheduler_tick+0xf3/0x100
> [297993.758643]  [<c02930de>] net_rx_action+0xde/0x260
> [297993.758652]  [<c0127302>] __do_softirq+0x92/0x130
> [297993.758662]  [<c012742c>] do_softirq+0x8c/0x90
> [297993.758670]  [<c0106e20>] do_IRQ+0x40/0x70
> [297993.758677]  [<ee08b799>] journal_add_journal_head+0x69/0x160 [jbd]
> [297993.758699]  [<c0259946>] evtchn_do_upcall+0xb6/0xf0
> [297993.758709]  [<c01057a6>] hypervisor_callback+0x46/0x4e
> [297993.758718]  [<ee0c18cc>] walk_page_buffers+0x1c/0x70 [ext3]
> [297993.758740]  [<ee0c4ad1>] ext3_ordered_writepage+0xe1/0x1a0 [ext3]
> [297993.758759]  [<ee0c1920>] bget_one+0x0/0x10 [ext3]
> [297993.758774]  [<c0157a38>] __writepage+0x8/0x30
> [297993.758782]  [<c0157ef4>] write_cache_pages+0x214/0x310
> [297993.758791]  [<c0157a30>] __writepage+0x0/0x30
> [297993.758801]  [<c0158010>] generic_writepages+0x20/0x30
> [297993.758810]  [<c0158069>] do_writepages+0x49/0x50
> [297993.758817]  [<c01980c3>] __writeback_single_inode+0x93/0x3c0
> [297993.758828]  [<c012bb9e>] del_timer_sync+0xe/0x20
> [297993.758838]  [<c02ff246>] schedule+0x356/0x900
> [297993.758847]  [<c01f5d9d>] _atomic_dec_and_lock+0x3d/0x70
> [297993.758857]  [<c019877e>] sync_sb_inodes+0x17e/0x240
> [297993.758866]  [<c0198c49>] writeback_inodes+0x99/0xd0
> [297993.758876]  [<c0158715>] wb_kupdate+0x85/0xf0
> [297993.758885]  [<c0158ab0>] pdflush+0x0/0x260
> [297993.758892]  [<c0158bf8>] pdflush+0x148/0x260
> [297993.758900]  [<c0158690>] wb_kupdate+0x0/0xf0
> [297993.758908]  [<c0136312>] kthread+0x42/0x70
> [297993.758915]  [<c01362d0>] kthread+0x0/0x70
> [297993.758922]  [<c0105927>] kernel_thread_helper+0x7/0x10
> [297993.758930]  =======================
> [297993.758934] Code: 02 01 c6 44 02 01 01 89 f9 0f b6 f1 64 a1 08 00 42 c0 
> 8b 94 83 90 00 00 00 85 d2 74 72 8b 42 0c     85 c0 74 6b 8b 5a 0c 0f b7 42 
> 0a <8b> 04 83 89 42 0c 89 fa 84 d2 74 2e 64 a1 08 00 42 c0 c1 e0 06
> [297993.759012] EIP: [<c0176eec>] kmem_cache_alloc+0x5c/0xe0 SS:ESP 
> 0069:e0003c00
> [297993.759030] Kernel panic - not syncing: Fatal exception in interrupt
>
>
> [16785.730498] BUG: unable to handle kernel                    paging request 
> at virtual address 00100104
> [16785.730517]  printing eip:
> [16785.730528] 2d463000 -> *pde = 00000000:45af1001
> [16785.730533] 2d506000 -> *pme = 00000000:00000000
> [16785.730540] Oops: 0000 [#1]
> [16785.730543] SMP
> [16785.730550] Modules linked in: xt_multiport iptable_filter ip_tables 
> x_tables                    ipv6 evdev ext3 jbd mbcache dm_mirror dm_snapshot 
> dm_mod fuse apparmor commonca                   p
> [16785.730581] CPU:    0
> [16785.730582] EIP:    0061:[<ee0c18c9>]    Not tainted VLI
> [16785.730584] EFLAGS: 00010286   (2.6.22-14-xen #1)
> [16785.730607] EIP is at walk_page_buffers+0x19/0x70 [ext3]
> [16785.730613] eax: 00000000   ebx: fffffffe   ecx: ffffffff   edx: 00100100
> [16785.730619] esi: 00100100   edi: c18404b4   ebp: ffffffff   esp: c2149e04
> [16785.730625] ds: 007b   es: 007b   fs: 00d8  gs: 0000  ss: 0069
> [16785.730631] Process pdflush (pid: 60, ti=c2148000 task=c20fcf90 
> task.ti=c2148                   000)
> [16785.730635] Stack: 00000000 c1143210 c1830b20 c18404b4 c1143210 c1143210 
> ee0c                   4ad1 00001000
> [16785.730652]        00000000 ee0c1920 00000000 c1830b20 c2149f70 c28b46f4 
> 0000                   0000 c2149f70
> [16785.730666]        00000000 c0157a38 c1830b20 c0157ef4 00000000 0000000e 
> c015                   7a30 c28b46f4
> [16785.730681] Call Trace:
> [16785.730687]  [<ee0c4ad1>] ext3_ordered_writepage+0xe1/0x1a0 [ext3]
> [16785.730703]  [<ee0c1920>] bget_one+0x0/0x10 [ext3]
> [16785.730717]  [<c0157a38>] __writepage+0x8/0x30
> [16785.730727]  [<c0157ef4>] write_cache_pages+0x214/0x310
> [16785.730736]  [<c0157a30>] __writepage+0x0/0x30
> [16785.730745]  [<c0158010>] generic_writepages+0x20/0x30
>
> [16785.730753]  [<c0158069>] do_writepages+0x49/0x50
> [16785.730760]  [<c01980c3>] __writeback_single_inode+0x93/0x3c0
> [16785.730770]  [<c012bb9e>] del_timer_sync+0xe/0x20
> [16785.730780]  [<c02ff246>] schedule+0x356/0x900
> [16785.730789]  [<c019877e>] sync_sb_inodes+0x17e/0x240
> [16785.730799]  [<c0198c49>] writeback_inodes+0x99/0xd0
> [16785.730807]  [<c0158715>] wb_kupdate+0x85/0xf0
> [16785.730816]  [<c0158ab0>] pdflush+0x0/0x260
> [16785.730823]  [<c0158bf8>] pdflush+0x148/0x260
> [16785.730830]  [<c0158690>] wb_kupdate+0x0/0xf0
> [16785.730838]  [<c0136312>] kthread+0x42/0x70
> [16785.730846]  [<c01362d0>] kthread+0x0/0x70
> [16785.730852]  [<c0105927>] kernel_thread_helper+0x7/0x10
> [16785.730861]  =======================
> [16785.730864] Code: 74 f0 31 c0 39 cb 5b 0f 92 c0 c3 8d b4 26 00 00 00 00 55 
> 57                    89 d7 56 53 83 ec 08 89 0c 24 31 c9 89 44 24 04 8b 6a 
> 14 8d 5c 0d 00 <8b> 72 04                    3b 1c 24 76 2f 89 d8 29 e8 3b 44 
> 24 1c 73 25 8b 44 24
> [16785.730938] EIP: [<ee0c18c9>] walk_page_buffers+0x19/0x70 [ext3] SS:ESP 
> 0069:                   c2149e04
>
> --
> Mathematics is the supreme nostalgia of our time.
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

<Prev in Thread] Current Thread [Next in Thread>