[Xen-devel] merlot0 and merlot1 (was Re: [xen-4.4-testing test] 103756: trouble: blocked/broken/fail/pass)

[Ian Jackson <ian.jackson@xxxxxxxxxxxxx>]

Jan Beulich writes ("[Xen-devel] merlot0 and merlot1 (was Re: [xen-4.4-testing 
test] 103756: trouble: blocked/broken/fail/pass)"):
> I consider this difficult to imagine, but it's also not entirely
> impossible. Since this appears to recur, are there perhaps logs
> (ideally more than one instance, to allow finding a possible
> pattern) from right before the issue occurred?

If it was software induced it was caused by

  
http://logs.test-lab.xenproject.org/osstest/logs/103741/test-xtf-amd64-amd64-3/info.html
  
http://logs.test-lab.xenproject.org/osstest/logs/103737/test-amd64-amd64-xl-xsm/info.html

(merlot0 and merlot1 respectively)

Both are Xen boot failures in xen-unstable.

The former failed with a Xen panic (see below).  The latter also with
a similar-looking panic.

I have noticed a tendency for boot order to be corrupted in situations
where Xen on the hard disk crashes on boot.  Or rather, when I find
that I am resetting the boot order, I disproportionately often find
that the host in question is in a crash/reboot loop.

Ian.


Dec 19 05:19:50.177998 (XEN) ----[ Xen-4.9-unstable  x86_64  debug=y   Not 
tainted ]----
Dec 19 05:19:50.185993 (XEN) CPU:    0
Dec 19 05:19:50.186022 (XEN) RIP:    e008:[<ffff82d080100660>] 
__bitmap_and+0x24/0x33
Dec 19 05:19:50.194012 (XEN) RFLAGS: 0000000000010202   CONTEXT: hypervisor
Dec 19 05:19:50.202015 (XEN) rax: 0000000000000000   rbx: ffff82d08030fd58   
rcx: 0000000000000001
Dec 19 05:19:50.209997 (XEN) rdx: ffff82d0802edda0   rsi: ffff83023ffd1ef0   
rdi: 0000000000000000
Dec 19 05:19:50.217999 (XEN) rbp: ffff82d08030fd18   rsp: ffff82d08030fd18   
r8:  0000000000000004
Dec 19 05:19:50.226030 (XEN) r9:  ffff82d080323a48   r10: 0000000000000000   
r11: 00000000ffffffff
Dec 19 05:19:50.234016 (XEN) r12: 0000000000000028   r13: ffff83023ffd1ef0   
r14: 0000000000000000
Dec 19 05:19:50.242002 (XEN) r15: ffff82d080343aa0   cr0: 000000008005003b   
cr4: 00000000000406e0
Dec 19 05:19:50.242044 (XEN) cr3: 00000000bdaff000   cr2: 0000000000000000
Dec 19 05:19:50.250003 (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 
0000   cs: e008
Dec 19 05:19:50.258000 (XEN) Xen code around <ffff82d080100660> 
(__bitmap_and+0x24/0x33):
Dec 19 05:19:50.266001 (XEN)  48 8b 0c c2 48 23 0c c6 <48> 89 0c c7 48 83 c0 01 
41 39 c0 7f eb 5d c3 55
Dec 19 05:19:50.274021 (XEN) Xen stack trace from rsp=ffff82d08030fd18:
Dec 19 05:19:50.274059 (XEN)    ffff82d08030fd48 ffff82d0801719cc 
ffff83023e203800 ffff83023e384030
Dec 19 05:19:50.282033 (XEN)    ffff82d08030fd58 0000000000000000 
ffff82d08030fd88 ffff82d080171bbc
Dec 19 05:19:50.290024 (XEN)    0000000000000000 0000000000000000 
ffff82d08030fd88 ffff83023e384010
Dec 19 05:19:50.298002 (XEN)    ffff82d080329b70 0000000000000038 
ffff82d08030fdc8 ffff82d0802afb8b
Dec 19 05:19:50.305991 (XEN)    00000000c0000000 0000000000000000 
00000000ffffffed ffff82d08030ffff
Dec 19 05:19:50.313991 (XEN)    0000000000000000 ffff82d080343aa0 
ffff82d08030fdd8 ffff82d0802b00d7
Dec 19 05:19:50.321999 (XEN)    ffff82d08030fdf8 ffff82d0802ac9cb 
ffff82d080343aa0 ffff83023fff5fe0
Dec 19 05:19:50.329995 (XEN)    ffff82d08030ff08 ffff82d0802c0ff2 
0000000000000000 0000000000100000
Dec 19 05:19:50.337992 (XEN)    00000000000001ff 0000000000000001 
0000000000000001 0000000000000001
Dec 19 05:19:50.345995 (XEN)    0000000000000001 0000000000000001 
0000000000000001 0000000000000001
Dec 19 05:19:50.354022 (XEN)    0000000000000000 00000000011f3000 
000000000043efff 0000000000800163
Dec 19 05:19:50.362005 (XEN)    fffffffffffff001 000000043ee00000 
0000000000000002 ffff83000008dd80
Dec 19 05:19:50.369994 (XEN)    ffff83000008def0 ffff83000008dfb0 
0000000000000000 0000000000000000
Dec 19 05:19:50.378000 (XEN)    0000000800000000 000000010000006e 
0000000000000003 00000000000002f8
Dec 19 05:19:50.386046 (XEN)    0000000000000000 0000000000000000 
0000000000000000 0000000000000000
Dec 19 05:19:50.393995 (XEN)    0000000000000000 0000000000000000 
0000000000000000 ffff82d080100073
Dec 19 05:19:50.401991 (XEN)    0000000000000000 0000000000000000 
0000000000000000 0000000000000000
Dec 19 05:19:50.410017 (XEN)    0000000000000000 0000000000000000 
0000000000000000 0000000000000000
Dec 19 05:19:50.417998 (XEN)    0000000000000000 0000000000000000 
0000000000000000 0000000000000000
Dec 19 05:19:50.426028 (XEN)    0000000000000000 0000000000000000 
0000000000000000 0000000000000000
Dec 19 05:19:50.434004 (XEN) Xen call trace:
Dec 19 05:19:50.434037 (XEN)    [<ffff82d080100660>] __bitmap_and+0x24/0x33
Dec 19 05:19:50.442054 (XEN)    [<ffff82d0801719cc>] msi_compose_msg+0x7e/0xf5
Dec 19 05:19:50.442092 (XEN)    [<ffff82d080171bbc>] __setup_msi_irq+0x2f/0x5c
Dec 19 05:19:50.450046 (XEN)    [<ffff82d0802afb8b>] amd_iommu_init+0x460/0x6b7
Dec 19 05:19:50.457996 (XEN)    [<ffff82d0802b00d7>] amd_iov_detect+0x69/0xad
Dec 19 05:19:50.465982 (XEN)    [<ffff82d0802ac9cb>] iommu_setup+0x61/0x1c1
Dec 19 05:19:50.466019 (XEN)    [<ffff82d0802c0ff2>] __start_xen+0x23a8/0x2878
Dec 19 05:19:50.473994 (XEN)    [<ffff82d080100073>] __high_start+0x53/0x55
Dec 19 05:19:50.481998 (XEN) 
Dec 19 05:19:50.482025 (XEN) Pagetable walk from 0000000000000000:
Dec 19 05:19:50.482055 (XEN)  L4[0x000] = 000000023fff3063 ffffffffffffffff
Dec 19 05:19:50.489984 (XEN)  L3[0x000] = 000000023fff2063 ffffffffffffffff
Dec 19 05:19:50.497989 (XEN)  L2[0x000] = 000000023fff1063 ffffffffffffffff 
Dec 19 05:19:50.498025 (XEN)  L1[0x000] = 0000000000000000 ffffffffffffffff
Dec 19 05:19:50.506007 (XEN) 
Dec 19 05:19:50.506035 (XEN) ****************************************
Dec 19 05:19:50.514001 (XEN) Panic on CPU 0:
Dec 19 05:19:50.514032 (XEN) FATAL PAGE FAULT
Dec 19 05:19:50.522011 (XEN) [error_code=0002]
Dec 19 05:19:50.522042 (XEN) Faulting linear address: 0000000000000000
Dec 19 05:19:50.529978 (XEN) ****************************************
Dec 19 05:19:50.530011 (XEN) 
Dec 19 05:19:50.537956 (XEN) Reboot in five seconds...


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel