Xen project Mailing List

[Xen-devel] [SPAM] dom0 pvops boot crash on very ordinary Dell R310

To: Jeremy Fitzhardinge <Jeremy.Fitzhardinge@xxxxxxxxxx>

From: Ian Jackson <Ian.Jackson@xxxxxxxxxxxxx>

Date: Tue, 26 Oct 2010 16:57:24 +0100

Delivery-date: Tue, 26 Oct 2010 08:58:47 -0700

Importance: Low

List-id: Xen developer discussion <xen-devel.lists.xensource.com>

The crash below happens every time I boot 32-bit dom0 pvops on 64-bit xen-unstable, but only on some machines. Circumstantial evidence, to be confirmed, suggests that 32-bit PAE pvops dom0 on 32-bit PAE Xen doesn't work either, and that the XCP kernel works fine on this hardware. Crash dump below is from: linux 2472a9ccf02b04a36300cf1d3515f3f14d40aa4b qemu 706bc8c588cbda284a9579cdb7e494ad5c9d6383 xen 22267:00b92112b055 Massaging the oops file and feeding it and the dom0 kernel Symbols file to ksymoops produces the output you find below. Ian. Oct 26 15:26:46 (XEN) trace.c:90:d32767 calc_tinfo_first_offset: NR_CPUs 128, offset_in_bytes 25 Oct 26 15:26:46 (XEN) Xen trace buffers: disabled Oct 26 15:26:46 (XEN) Std. Loglevel: All Oct 26 15:26:46 (XEN) Guest Loglevel: All Oct 26 15:26:46 (XEN) Xen is relinquishing VGA console. Oct 26 15:26:46 (XEN) *** Serial input -> DOM0 (type 'CTRL-x' three times to switch input to Xe) Oct 26 15:26:46 (XEN) Freed 188kB init memory. Oct 26 15:26:46 mapping kernel into physical memory Oct 26 15:26:46 Xen: setup ISA identity maps Oct 26 15:26:46 about to get started... Oct 26 15:26:46 (XEN) mm.c:889:d0 Error getting mfn 3409e (pfn 5555555555555555) from L1 entry 0 Oct 26 15:26:46 (XEN) mm.c:4953:d0 ptwr_emulate: fixing up invalid PAE PTE 000000003409e023 Oct 26 15:26:46 (XEN) mm.c:889:d0 Error getting mfn 3409f (pfn 5555555555555555) from L1 entry 0 Oct 26 15:26:46 (XEN) mm.c:4953:d0 ptwr_emulate: fixing up invalid PAE PTE 000000003409f023 Oct 26 15:26:46 (XEN) d0:v0: unhandled page fault (ec=0000) Oct 26 15:26:46 (XEN) Pagetable walk from 00000000c009fc00: Oct 26 15:26:46 (XEN) L4[0x000] = 0000000235747027 0000000000001747 Oct 26 15:26:46 (XEN) L3[0x003] = 00000002358fd027 00000000000018fd Oct 26 15:26:46 (XEN) L2[0x000] = 000000023620c067 000000000000220c Oct 26 15:26:46 (XEN) L1[0x09f] = 0000000000000000 ffffffffffffffff Oct 26 15:26:46 (XEN) domain_crash_sync called from entry.S Oct 26 15:26:46 (XEN) Domain 0 (vcpu#0) crashed on cpu#0: Oct 26 15:26:46 (XEN) ----[ Xen-4.1-unstable x86_64 debug=y Not tainted ]---- Oct 26 15:26:46 (XEN) CPU: 0 Oct 26 15:26:46 (XEN) RIP: e019:[<00000000c17d500b>] Oct 26 15:26:46 (XEN) RFLAGS: 0000000000000206 EM: 1 CONTEXT: pv guest Oct 26 15:26:46 (XEN) rax: 000000000009fc00 rbx: 00000000c009fc00 rcx: 0000000000000001 Oct 26 15:26:46 (XEN) rdx: 0000000000000400 rsi: 0000000036dfe000 rdi: 0000000000000400 Oct 26 15:26:46 (XEN) rbp: 00000000c1745f00 rsp: 00000000c1745ee0 r8: 0000000000000000 Oct 26 15:26:46 (XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 Oct 26 15:26:46 (XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 Oct 26 15:26:46 (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000026f0 Oct 26 15:26:46 (XEN) cr3: 000000023a3a9000 cr2: 00000000c009fc00 Oct 26 15:26:46 (XEN) ds: e021 es: e021 fs: 00d8 gs: 0000 ss: e021 cs: e019 Oct 26 15:26:46 (XEN) Guest stack trace from esp=c1745ee0: Oct 26 15:26:46 (XEN) 00000000 c17d500b 0001e019 00010006 00000001 00000001 36dfe000 00000000 Oct 26 15:26:46 (XEN) c1745f0c c17d50fe 00000000 c1745fb0 c17cdd06 c1063e9a 00000000 c1745fc0 Oct 26 15:26:46 (XEN) 0218a600 00000000 01a68000 00000000 00722600 01a68000 c106439f 00000000 Oct 26 15:26:46 (XEN) 00000035 c1745f71 00000005 00000000 00000000 00000000 c1745f78 0000009e Oct 26 15:26:46 (XEN) 205b0000 30202020 3030302e 5d303030 c1800020 00000000 00000000 c174f350 Oct 26 15:26:46 (XEN) c1745f8c c102d295 00000000 c1745fa0 c14e5178 00000000 c1800670 c174f350 Oct 26 15:26:46 (XEN) c1745fb0 00000000 c1800670 c174f350 c1745fc8 c17c75d9 c1659dee c14eb010 Oct 26 15:26:47 (XEN) c18019d0 00000000 c1745fd4 c17c70a2 c220b490 c1745ffc c17cae57 1fc9dbf5 Oct 26 15:26:47 (XEN) 80980281 00100800 000106e5 00000001 00000000 c220b000 00000000 00000000 Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 Oct 26 15:26:47 (XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 Oct 26 15:26:47 (XEN) Domain 0 crashed: rebooting machine in 5 seconds. Oct 26 15:26:52 <Modem lines changed: -CTS -CD -DSR> Oct 26 15:27:10 <Modem lines changed: +CTS +CD +DSR> mariner:~> ksymoops -d -K -m t -L -V <u | grep -v 'Before first symbol' DEBUG (parse): 'K' '(null)' DEBUG (parse): 'm' 't' DEBUG (parse): 'L' '(null)' DEBUG (parse): 'V' '(null)' DEBUG (convert_uname): /lib/modules/*r/ in DEBUG (convert_uname): /lib/modules/2.6.26-2-amd64/ out ksymoops 2.4.11 on x86_64 2.6.26-2-amd64. Options used -V (specified) -K (specified) -L (specified) -o /lib/modules/2.6.26-2-amd64/ (default) -m t (specified) DEBUG (main): level 1 DEBUG (read_env): default KSYMOOPS_NM=/usr/bin/nm DEBUG (read_env): default KSYMOOPS_FIND=/usr/bin/find DEBUG (read_env): default KSYMOOPS_OBJDUMP=/usr/bin/objdump DEBUG (re_compile): '^([0-9a-fA-F]{4,}) +([^ ]) +([^ ]+)( +\[([^ ]+)\])?$' 5 sub expression(s) DEBUG (re_compile): '^ *\[*<([0-9a-fA-F]{4,})>\]* *' 1 sub expression(s) DEBUG (re_compile): '^ *<\[([0-9a-fA-F]{4,})\]> *' 1 sub expression(s) DEBUG (re_compile): '^ *([0-9a-fA-F]{4,}) *' 1 sub expression(s) No modules in ksyms, skipping objects DEBUG (read_system_map): t DEBUG (re_compile): '^(.*)_R.*[0-9a-fA-F]{8,}$' 1 sub expression(s) DEBUG (ss_sort_na): System.map DEBUG (merge_maps): DEBUG (ss_sort_atn): merged DEBUG (ss_sort_atn): merged DEBUG (ss_sort_atn): merged DEBUG (re_compile): '^( +|[^ ]{3} [ 0-9][0-9] [0-9]{2}:[0-9]{2}:[0-9]{2} [^ ]+( kernel:)? +|<[0-9]+>|[0-9]+\|[^|]+\|[^|]+\|[^|]+\||[0-9][AC] [0-9]{3} [0-9]{3}[cir][0-9]{2}:\+)' 2 sub expression(s) DEBUG (re_compile): '^((Stack: )|(Stack from )|([0-9a-fA-F]{4,})|(Call Trace: *)|(\[*<([0-9a-fA-F]{4,})>\]* *)|(Version_[0-9]+)|(Trace:)|(Call backtrace:)|(bh:)|<\[([0-9a-fA-F]{4,})\]> *|($[^ ]+$ *\[*<([0-9a-fA-F]{4,})>\]* *)|([0-9]+ +base=)|(Kernel BackChain)|EBP *EIP|0x([0-9a-fA-F]{4,}) *0x([0-9a-fA-F]{4,}) *|Process .*stackpage=|Code *: |Kernel panic|In swapper task|kmem_free|swapper|Pid:|r[0-9]{1,2} *[:=]|Corrupted stack page|invalid operand: |Oops: |Cpu:* +[0-9]|current->tss|\*pde +=|EIP: |EFLAGS: |eax: |esi: |ds: |CR0: |wait_on_|irq: |Stack dumps:|RAX: |RSP: |RIP: |RDX: |RBP: |FS: |CS: |CR2: |PML4|pc[:=]|68060 access|Exception at |d[04]: |Frame format=|wb [0-9] stat|push data: |baddr=|Arithmetic fault|Instruction fault|Bad unaligned kernel|Forwarding unaligned exception|: unhandled unaligned exception|pc *=|[rvtsa][0-9]+ *=|gp *=|spinlock stuck|tsk->|PSR: |[goli]0: |Instruction DUMP: |spin_lock|TSTATE: |[goli]4: |Caller\[|CPU\[|MSR: |TASK = |last math |GPR[0-9]+: |.* in .*\.[ch]:.*, line [0-9]+:|\$[0-9 ]+:|Hi *:|Lo *:|epc *:|badvaddr *:|Status *:|Cause *:|Backtrace:|Function entered at|\*pgd =|Internal error|pc :|sp :|Flags:|Control:|WARNING:|this_stack:|i:|PSW|cr[0-9]+:|machine check|Exception in |Program Check |System restart |IUCV |unexpected external |Kernel stack |User process fault:|failing address|User PSW|User GPRS|User ACRS|Kernel PSW|Kernel GPRS|Kernel ACRS|illegal operation|task:|Entering kdb|eax *=|esi *=|ebp *=|ds *=|psr |unat |rnat |ldrs |xip |iip |ipsr |ifa |pr |itc |ifs |bsp |[bfr][0-9]+ |irr[0-9] |General Exception|MCA|SAL|Processor State|Bank [0-9]+|Register Stack|Processor Error Info|proc err map|proc state param|proc lid|processor structure|check info|target identifier|IRP *: )' 18 sub expression(s) DEBUG (re_compile): 'Unable to handle kernel|Aiee|die_if_kernel|NMI |BUG |$L-TLB$|$NOTLB$|$[0-9]$: Oops |: memory violation|: Exception at|: Arithmetic fault|: Instruction fault|: arithmetic trap|: unaligned trap|$[0-9]+$: (Whee|Oops|Kernel|.*Penguin|BOGUS)|kernel pc |trap at PC: |bad area pc |NIP: | ra *==' 1 sub expression(s) DEBUG (re_compile): '^(i[04]: |Instruction DUMP: |Caller\[)' 1 sub expression(s) CPU: 0 DEBUG (re_compile): '^PSR: [0-9a-fA-F]+ PC: ([0-9a-fA-F]{4,}) *' 1 sub expression(s) DEBUG (re_compile): '^TSTATE: [0-9a-fA-F]{16} TPC: ([0-9a-fA-F]{4,}) *' 1 sub expression(s) DEBUG (re_compile): '(kernel pc |trap at PC: |bad area pc |NIP: )([0-9a-fA-F]{4,}) *' 2 sub expression(s) DEBUG (re_compile): '^epc *:+ *([0-9a-fA-F]{4,}) *' 1 sub expression(s) DEBUG (re_compile): ' ip *:+ *\[*<([0-9a-fA-F]{4,})>\]* *' 1 sub expression(s) DEBUG (re_compile): '[xi]ip *$[^)]*$ *: *0x([0-9a-fA-F]{4,}) *' 1 sub expression(s) DEBUG (re_compile): '(^0x([0-9a-fA-F]{4,}) *0x([0-9a-fA-F]{4,}) *.*+0x)|(^Entering kdb on processor.*0x([0-9a-fA-F]{4,}) *)' 5 sub expression(s) DEBUG (re_compile): '^ *IRP *: *([0-9a-fA-F]{4,}) *' 1 sub expression(s) DEBUG (re_compile): '^(EIP: +.*|RIP: +.*|PC *= *|pc *: *)\[*<([0-9a-fA-F]{4,})>\]* *' 2 sub expression(s) DEBUG (re_compile): '^spinlock stuck at ([0-9a-fA-F]{4,}) *.*owner.*at ([0-9a-fA-F]{4,}) *' 2 sub expression(s) DEBUG (re_compile): '^spin_lock[^ ]*$([0-9a-fA-F]{4,}) *.*stuck at *([0-9a-fA-F]{4,}) *.*PC\(([0-9a-fA-F]{4,}) *' 3 sub expression(s) DEBUG (re_compile): 'ra *=+ *([0-9a-fA-F]{4,}) *' 1 sub expression(s) DEBUG (re_compile): '^(\$[0-9]{1,2}) *: *([0-9a-fA-F]{4,}) *' 2 sub expression(s) DEBUG (re_compile): '^(GPR[0-9]{1,2}) *: *([0-9a-fA-F]{4,}) *' 2 sub expression(s) DEBUG (re_compile): '^PSW *flags: *([0-9a-fA-F]{4,}) * *PSW addr: *([0-9a-fA-F]{4,}) *' 2 sub expression(s) DEBUG (re_compile): '^Kernel PSW: *([0-9a-fA-F]{4,}) *([0-9a-fA-F]{4,}) *' 2 sub expression(s) DEBUG (re_compile): '^((Call Trace: *)|(Trace:)|(\[*<([0-9a-fA-F]{4,})>\]* *)|(Call backtrace:)|([0-9a-fA-F]{4,}) *|Function entered at (\[*<([0-9a-fA-F]{4,})>\]* *)|Caller\[([0-9a-fA-F]{4,}) *\]|(<\[([0-9a-fA-F]{4,})\]> *)|(\([0-9]+$ *(\[*<([0-9a-fA-F]{4,})>\]* *))|([0-9]+ +base=0x([0-9a-fA-F]{4,}) *)|(Kernel BackChain.*))' 18 sub expression(s) DEBUG (re_compile): '^((GP|o)?r[0-9]{1,2}|[goli][0-9]{1,2}|[eR][ABCD]X|[eR][DS]I|RBP|e[bs]p|[fsi]p|IRP|SRP|D?CCR|USP|MOF|ret_pc) *[:=] *([0-9a-fA-F]{4,}) * *' 3 sub expression(s) DEBUG (re_compile): '^(Kernel GPRS.*)' 1 sub expression(s) DEBUG (re_compile): '^(IRP|SRP|D?CCR|USP|MOF): *([0-9a-fA-F]{4,}) *' 2 sub expression(s) DEBUG (re_compile): '^b[0-7] *($[^)]*$ *)?: *(0x)?([0-9a-fA-F]{4,}) *' 3 sub expression(s) DEBUG (re_compile): '^([^ ]+) +[^ ] *([^ ]+).*$(L-TLB|NOTLB)$' 3 sub expression(s) DEBUG (re_compile): '^((Instruction DUMP)|(Code:? *)): +((general protection.*)|(Bad E?IP value.*)|(<[0-9]+>)|(([<(]?[0-9a-fA-F]+[>)]? +)+[<(]?[0-9a-fA-F]+[>)]?))(.*)$' 10 sub expression(s) RIP: e019:[<00000000c17d500b>] DEBUG (re_compile): 'pc *: *\[*<([0-9a-fA-F]{4,})>\]* * *lr *: *\[*<([0-9a-fA-F]{4,})>\]* *' 2 sub expression(s) DEBUG (re_compile): 'pc *= *\[*<([0-9a-fA-F]{4,})>\]* * *ra *= *\[*<([0-9a-fA-F]{4,})>\]* *' 2 sub expression(s) Using defaults from ksymoops -t elf32-i386 -a i386 rax: 000000000009fc00 rbx: 00000000c009fc00 rcx: 0000000000000001 rdx: 0000000000000400 rsi: 0000000036dfe000 rdi: 0000000000000400 rbp: 00000000c1745f00 rsp: 00000000c1745ee0 r8: 0000000000000000 Warning (Oops_set_regs): garbage 'rsp: 00000000c1745ee0 r8: 0000000000000000' at end of register line ignored r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000 r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000 r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000026f0 Warning (Oops_set_regs): garbage 'cr0: 000000008005003b cr4: 00000000000026f0' at end of register line ignored cr3: 000000023a3a9000 cr2: 00000000c009fc00 ds: e021 es: e021 fs: 00d8 gs: 0000 ss: e021 cs: e019 Trace: <00000000> <c17d500b> <0001e019> <00010006> <00000001> <00000001> <36dfe000> <00000000> DEBUG (re_compile): '^($[0-9]+$ *)' 1 sub expression(s) Trace: <c1745f0c> <c17d50fe> <00000000> <c1745fb0> <c17cdd06> <c1063e9a> <00000000> <c1745fc0> Trace: <0218a600> <00000000> <01a68000> <00000000> <00722600> <01a68000> <c106439f> <00000000> Trace: <00000035> <c1745f71> <00000005> <00000000> <00000000> <00000000> <c1745f78> <0000009e> Trace: <205b0000> <30202020> <3030302e> <5d303030> <c1800020> <00000000> <00000000> <c174f350> Trace: <c1745f8c> <c102d295> <00000000> <c1745fa0> <c14e5178> <00000000> <c1800670> <c174f350> Trace: <c1745fb0> <00000000> <c1800670> <c174f350> <c1745fc8> <c17c75d9> <c1659dee> <c14eb010> Trace: <c18019d0> <00000000> <c1745fd4> <c17c70a2> <c220b490> <c1745ffc> <c17cae57> <1fc9dbf5> Trace: <80980281> <00100800> <000106e5> <00000001> <00000000> <c220b000> <00000000> <00000000> Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> Trace: <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> <00000000> Warning (Oops_read): Code line not seen, dumping what data is available DEBUG (Oops_format): >>EIP; c17d500b <smp_scan_config+35/ff> <===== >>rbx; c009fc00 <phys_startup_32+bf09fc00/c0000000> >>rsi; 36dfe000 <phys_startup_32+35dfe000/c0000000> >>rbp; c1745f00 <init_thread_union+1f00/2000> Trace; c17d500b <smp_scan_config+35/ff> Trace; 36dfe000 <phys_startup_32+35dfe000/c0000000> Trace; c1745f0c <init_thread_union+1f0c/2000> Trace; c17d50fe <default_find_smp_config+29/5d> Trace; c1745fb0 <init_thread_union+1fb0/2000> Trace; c17cdd06 <setup_arch+84b/a4d> Trace; c1063e9a <release_console_sem+1b0/1dd> Trace; c1745fc0 <init_thread_union+1fc0/2000> Trace; 0218a600 <phys_startup_32+118a600/c0000000> Trace; 01a68000 <phys_startup_32+a68000/c0000000> Trace; 01a68000 <phys_startup_32+a68000/c0000000> Trace; c106439f <vprintk+30f/332> Trace; c1745f71 <init_thread_union+1f71/2000> Trace; c1745f78 <init_thread_union+1f78/2000> Trace; 205b0000 <phys_startup_32+1f5b0000/c0000000> Trace; 30202020 <phys_startup_32+2f202020/c0000000> Trace; 3030302e <phys_startup_32+2f30302e/c0000000> Trace; 5d303030 <phys_startup_32+5c303030/c0000000> Trace; c1800020 <md_setup_args+f30/1400> Trace; c174f350 <reboot_cpu+0/4> Trace; c1745f8c <init_thread_union+1f8c/2000> Trace; c102d295 <__raw_callee_save_xen_restore_fl+9/c> Trace; c1745fa0 <init_thread_union+1fa0/2000> Trace; c14e5178 <_spin_unlock_irqrestore+43/48> Trace; c1800670 <map.24465+0/a00> Trace; c174f350 <reboot_cpu+0/4> Trace; c1745fb0 <init_thread_union+1fb0/2000> Trace; c1800670 <map.24465+0/a00> Trace; c174f350 <reboot_cpu+0/4> Trace; c1745fc8 <init_thread_union+1fc8/2000> Trace; c17c75d9 <start_kernel+86/31a> Trace; c1659dee <kallsyms_token_index+4d6/c9a6> Trace; c14eb010 <linux_banner+0/8c> Trace; c18019d0 <command_line+0/800> Trace; c1745fd4 <init_thread_union+1fd4/2000> Trace; c17c70a2 <i386_start_kernel+91/96> Trace; c220b490 <END_OF_CODE+7a3490/????> Trace; c1745ffc <init_thread_union+1ffc/2000> Trace; c17cae57 <xen_start_kernel+535/53d> Trace; 1fc9dbf5 <phys_startup_32+1ec9dbf5/c0000000> Trace; 80980281 <phys_startup_32+7f980281/c0000000> Trace; c220b000 <END_OF_CODE+7a3000/????> 3 warnings issued. Results may not be reliable. mariner:~>

_______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxxxxxxxx http://lists.xensource.com/xen-devel

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.