Xen project Mailing List

Re: [Xen-devel] Xen dom0 crash: "d0:v0: unhandled page fault (ec=0000)"

From: "Alan J. Wylie" <NDA5OWUy@xxxxxxxxxxx>

Date: Wed, 13 Oct 2010 14:52:59 +0100

Cc: Jeremy Fitzhardinge <jeremy@xxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxx, Andreas Kinzler <ml-xen-devel@xxxxxx>

Delivery-date: Wed, 13 Oct 2010 06:54:12 -0700

List-id: Xen developer discussion <xen-devel.lists.xensource.com>

at 12:04 on Wed 13-Oct-2010 Alan J. Wylie (NDA5OWUy@xxxxxxxxxxx) wrote: > I can confirm that changing the debug settings makes the difference > between booting and crashing. > > However, trying to narrow it down to a particular config option is > proving tricky. > > At first I thought that just turning CONFIG_DEBUG_SPINLOCK on stopped > it crashing, however further checking was inconclusive. My suspicion > is that it's things being moved around in memory as a result of code > being enabled that's the issue. > > I'll continue investigating. Here's one set of differences between .configs that makes the difference between booting and crashing: 8<------8<------8<------8<------8<------8<------8<------8<------8<------8<------ xenity jeremy-git-xen-test # diff myconfig.009.boots myconfig.010.crashes 4c4 < # Wed Oct 13 12:17:38 2010 --- > # Wed Oct 13 14:29:49 2010 108c108 < CONFIG_KALLSYMS_ALL=y --- > # CONFIG_KALLSYMS_ALL is not set 8<------8<------8<------8<------8<------8<------8<------8<------8<------8<------ 531 diff myconfig.009.boots myconfig.010.crashes 534 rm -r _dom0/ 535 mkdir _dom0/ 536 cp -p myconfig.009.boots _dom0/.config 537 make0 oldconfig 538 make0 539 scp -p _dom0/arch/x86/boot/bzImage BB:/boot/vmlinuz-test 540 rm -r _dom0/ && mkdir _dom0/ && cp -p myconfig.010.crashes _dom0/.config 541 make0 oldconfig && make0 542 scp -p _dom0/arch/x86/boot/bzImage BB:/boot/vmlinuz-test 8<------8<------8<------8<------8<------8<------8<------8<------8<------8<------ (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x1734000 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000238000000->000000023a000000 (253952 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff81000000->ffffffff81734000 (XEN) Init. ramdisk: ffffffff81734000->ffffffff81734000 (XEN) Phys-Mach map: ffffffff81734000->ffffffff81934000 (XEN) Start info: ffffffff81934000->ffffffff819344b4 (XEN) Page tables: ffffffff81935000->ffffffff81946000 (XEN) Boot stack: ffffffff81946000->ffffffff81947000 (XEN) TOTAL: ffffffff80000000->ffffffff81c00000 (XEN) ENTRY ADDRESS: ffffffff8154d200 (XEN) Dom0 has maximum 1 VCPUs (XEN) Scrubbing Free RAM: ......................................................................done. (XEN) trace.c:89:d32767 calc_tinfo_first_offset: NR_CPUs 128, offset_in_bytes 258, t_info_first_offset 65 (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) ********************************************** (XEN) ******* WARNING: CONSOLE OUTPUT IS SYNCHRONOUS (XEN) ******* This option is intended to aid debugging of Xen by ensuring (XEN) ******* that all output is synchronously delivered on the serial line. (XEN) ******* However it can introduce SIGNIFICANT latencies and affect (XEN) ******* timekeeping. It is NOT recommended for production use! (XEN) ********************************************** (XEN) 3... 2... 1... (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen) (XEN) Freed 172kB init memory. mapping kernel into physical memory Xen: setup ISA identity maps <7>ALANW get_phys_to_machine pfn 000018C3<7> topidx 00000000<7> mididx 0000000C<7> idx 000000C3<7> returns 002398C3 about to get started... Linux version 2.6.32.24-xen0 (root@xenity) (gcc version 4.4.3 (Gentoo 4.4.3-r2 p1.2) ) #1 SMP Wed Oct 13 14:41:42 BST 2010 Command line: root=/dev/sda2 console=hvc0 earlyprintk=xen loglevel=10 initcall_debug debug KERNEL supported cpus: Intel GenuineIntel AMD AuthenticAMD Centaur CentaurHauls released 0 pages of unused memory BIOS-provided physical RAM map: Xen: 0000000000000000 - 0000000000098c00 (usable) Xen: 0000000000098c00 - 0000000000100000 (reserved) Xen: 0000000000100000 - 0000000040000000 (usable) Xen: 00000000bf78e000 - 00000000bf790000 type 9 Xen: 00000000bf790000 - 00000000bf79e000 (ACPI data) Xen: 00000000bf79e000 - 00000000bf7d0000 (ACPI NVS) Xen: 00000000bf7d0000 - 00000000bf7e0000 (reserved) Xen: 00000000bf7ec000 - 00000000c0000000 (reserved) Xen: 00000000e0000000 - 00000000f0000000 (reserved) Xen: 00000000fec00000 - 00000000fec01000 (reserved) Xen: 00000000fec8a000 - 00000000fec8b000 (reserved) Xen: 00000000fee00000 - 00000000fee01000 (reserved) Xen: 00000000ffc00000 - 0000000100000000 (reserved) Xen: 0000000240000000 - 00000002bf780000 (usable) bootconsole [xenboot0] enabled DMI present. AMI BIOS detected: BIOS may corrupt low RAM, working around it. e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved) last_pfn = 0x2bf780 max_arch_pfn = 0x400000000 x86 PAT enabled: cpu 0, old 0x50100070406, new 0x7010600070106 last_pfn = 0x40000 max_arch_pfn = 0x400000000 initial memory mapped : 0 - 20000000 init_memory_mapping: 0000000000000000-0000000040000000 0000000000 - 0040000000 page 4k kernel direct mapping tables up to 40000000 @ 100000-302000 init_memory_mapping: 0000000100000000-00000002bf780000 0100000000 - 02bf780000 page 4k kernel direct mapping tables up to 2bf780000 @ 1946000-2f4e000 ACPI: RSDP 00000000000fac10 00024 (v02 ACPIAM) ACPI: XSDT 00000000bf790100 0007C (v01 SMCI 20100528 MSFT 00000097) ACPI: FACP 00000000bf790290 000F4 (v03 052810 FACP1443 20100528 MSFT 00000097) ACPI: DSDT 00000000bf7906a0 0650A (v01 10006 10006000 00000000 INTL 20051117) ACPI: FACS 00000000bf79e000 00040 ACPI: APIC 00000000bf790390 0011E (v01 052810 APIC1443 20100528 MSFT 00000097) ACPI: MCFG 00000000bf7904b0 0003C (v01 052810 OEMMCFG 20100528 MSFT 00000097) ACPI: SLIT 00000000bf7904f0 00030 (v01 052810 OEMSLIT 20100528 MSFT 00000097) ACPI: OEMB 00000000bf79e040 0007D (v01 052810 OEMB1443 20100528 MSFT 00000097) ACPI: HPET 00000000bf79a6a0 00038 (v01 052810 OEMHPET 20100528 MSFT 00000097) ACPI: SSDT 00000000bf79ee90 00363 (v01 DpgPmm CpuPm 00000012 INTL 20051117) ACPI: EINJ 00000000bf79a6e0 00130 (v01 AMIER AMI_EINJ 20100528 MSFT 00000097) ACPI: BERT 00000000bf79a870 00030 (v01 AMIER AMI_BERT 20100528 MSFT 00000097) ACPI: ERST 00000000bf79a8a0 001B0 (v01 AMIER AMI_ERST 20100528 MSFT 00000097) ACPI: HEST 00000000bf79aa50 000A8 (v01 AMIER ABC_HEST 20100528 MSFT 00000097) ACPI: Local APIC address 0xfee00000 (9 early reservations) ==> bootmem [0000000000 - 02bf780000] #0 [0000000000 - 0000001000] BIOS data page ==> [0000000000 - 0000001000] #1 [0001935000 - 0001946000] XEN PAGETABLES ==> [0001935000 - 0001946000] #2 [0000006000 - 0000008000] TRAMPOLINE ==> [0000006000 - 0000008000] #3 [0001000000 - 000160a280] TEXT DATA BSS ==> [0001000000 - 000160a280] #4 [0001734000 - 0001935000] XEN START INFO ==> [0001734000 - 0001935000] #5 [0240000000 - 02bf780000] XEN EXTRA ==> [0240000000 - 02bf780000] #6 [000160b000 - 000161521d] BRK ==> [000160b000 - 000161521d] #7 [0000100000 - 00002ee000] PGTABLE ==> [0000100000 - 00002ee000] #8 [0001946000 - 0001d44000] PGTABLE ==> [0001946000 - 0001d44000] found SMP MP-table at [ffff8800000ff780] ff780 Zone PFN ranges: DMA 0x00000010 -> 0x00001000 DMA32 0x00001000 -> 0x00100000 Normal 0x00100000 -> 0x002bf780 Movable zone start PFN for each node early_node_map[3] active PFN ranges 0: 0x00000010 -> 0x00000098 0: 0x00000100 -> 0x00040000 0: 0x00240000 -> 0x002bf780 On node 0 totalpages: 784136 DMA zone: 64 pages used for memmap DMA zone: 497 pages reserved DMA zone: 3415 pages, LIFO batch:0 DMA32 zone: 16320 pages used for memmap DMA32 zone: 241728 pages, LIFO batch:31 Normal zone: 28638 pages used for memmap Normal zone: 493474 pages, LIFO batch:31 8<------8<------8<------8<------8<------8<------8<------8<------8<------8<------ (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 64-bit, lsb, compat32 (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x16e4000 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 0000000238000000->000000023a000000 (253952 pages to be allocated) (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: ffffffff81000000->ffffffff816e4000 (XEN) Init. ramdisk: ffffffff816e4000->ffffffff816e4000 (XEN) Phys-Mach map: ffffffff816e4000->ffffffff818e4000 (XEN) Start info: ffffffff818e4000->ffffffff818e44b4 (XEN) Page tables: ffffffff818e5000->ffffffff818f6000 (XEN) Boot stack: ffffffff818f6000->ffffffff818f7000 (XEN) TOTAL: ffffffff80000000->ffffffff81c00000 (XEN) ENTRY ADDRESS: ffffffff814fd200 (XEN) Dom0 has maximum 1 VCPUs (XEN) Scrubbing Free RAM: ......................................................................done. (XEN) trace.c:89:d32767 calc_tinfo_first_offset: NR_CPUs 128, offset_in_bytes 258, t_info_first_offset 65 (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) ********************************************** (XEN) ******* WARNING: CONSOLE OUTPUT IS SYNCHRONOUS (XEN) ******* This option is intended to aid debugging of Xen by ensuring (XEN) ******* that all output is synchronously delivered on the serial line. (XEN) ******* However it can introduce SIGNIFICANT latencies and affect (XEN) ******* timekeeping. It is NOT recommended for production use! (XEN) ********************************************** (XEN) 3... 2... 1... (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen) (XEN) Freed 172kB init memory. mapping kernel into physical memory Xen: setup ISA identity maps <7>ALANW get_phys_to_machine pfn 000018C3<7> topidx 00000000<7> mididx 0000000C<7> idx 000000C3<7> returns 002398C3 about to get started... Linux version 2.6.32.24-xen0 (root@xenity) (gcc version 4.4.3 (Gentoo 4.4.3-r2 p1.2) ) #1 SMP Wed Oct 13 14:48:16 BST 2010 Command line: root=/dev/sda2 console=hvc0 earlyprintk=xen loglevel=10 initcall_debug debug KERNEL supported cpus: Intel GenuineIntel AMD AuthenticAMD Centaur CentaurHauls released 0 pages of unused memory BIOS-provided physical RAM map: Xen: 0000000000000000 - 0000000000098c00 (usable) Xen: 0000000000098c00 - 0000000000100000 (reserved) Xen: 0000000000100000 - 0000000040000000 (usable) Xen: 00000000bf78e000 - 00000000bf790000 type 9 Xen: 00000000bf790000 - 00000000bf79e000 (ACPI data) Xen: 00000000bf79e000 - 00000000bf7d0000 (ACPI NVS) Xen: 00000000bf7d0000 - 00000000bf7e0000 (reserved) Xen: 00000000bf7ec000 - 00000000c0000000 (reserved) Xen: 00000000e0000000 - 00000000f0000000 (reserved) Xen: 00000000fec00000 - 00000000fec01000 (reserved) Xen: 00000000fec8a000 - 00000000fec8b000 (reserved) Xen: 00000000fee00000 - 00000000fee01000 (reserved) Xen: 00000000ffc00000 - 0000000100000000 (reserved) Xen: 0000000240000000 - 00000002bf780000 (usable) bootconsole [xenboot0] enabled DMI present. AMI BIOS detected: BIOS may corrupt low RAM, working around it. e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved) last_pfn = 0x2bf780 max_arch_pfn = 0x400000000 x86 PAT enabled: cpu 0, old 0x50100070406, new 0x7010600070106 last_pfn = 0x40000 max_arch_pfn = 0x400000000 initial memory mapped : 0 - 20000000 init_memory_mapping: 0000000000000000-0000000040000000 0000000000 - 0040000000 page 4k kernel direct mapping tables up to 40000000 @ 100000-302000 init_memory_mapping: 0000000100000000-00000002bf780000 0100000000 - 02bf780000 page 4k kernel direct mapping tables up to 2bf780000 @ 18f6000-2efe000 (XEN) d0:v0: unhandled page fault (ec=0000) (XEN) Pagetable walk from ffffffff816f07b0: (XEN) L4[0x1ff] = 0000000239003067 0000000000001003 (XEN) L3[0x1fe] = 0000000239007067 0000000000001007 (XEN) L2[0x00b] = 0000000000000000 ffffffffffffffff (XEN) domain_crash_sync called from entry.S (XEN) Domain 0 (vcpu#0) crashed on cpu#0: (XEN) ----[ Xen-4.0.2-rc1-pre x86_64 debug=n Tainted: C ]---- (XEN) CPU: 0 (XEN) RIP: e033:[<ffffffff8100c679>] (XEN) RFLAGS: 0000000000000206 EM: 1 CONTEXT: pv guest (XEN) rax: ffffffff816f0000 rbx: 00000000000018f6 rcx: 0000000000000118 (XEN) rdx: ffffffff81566000 rsi: ffffffffff400000 rdi: 00000000000018f6 (XEN) rbp: ffffffff81463bb8 rsp: ffffffff81463b50 r8: 8000000000000163 (XEN) r9: 000000000000000f r10: ffffffff81586130 r11: ffffffff81585d30 (XEN) r12: 00000000000000f6 r13: 000000000000000c r14: 0000000000000000 (XEN) r15: 0000000000000001 cr0: 000000008005003b cr4: 00000000000026f0 (XEN) cr3: 0000000239001000 cr2: ffffffff816f07b0 (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e02b cs: e033 (XEN) Guest stack trace from rsp=ffffffff81463b50: (XEN) 0000000000000118 ffffffff81585d30 0000000000000000 ffffffff8100c679 (XEN) 000000010000e030 0000000000010006 ffffffff81463b98 000000000000e02b (XEN) 00003ffffffff032 80000000018f6063 00000000018f6000 00000000018f6000 (XEN) 8000000000000163 ffffffff81463bd8 ffffffff8100c954 ffffffff81463c08 (XEN) 80000000018f6063 ffffffff81463bf8 ffffffff8100ca31 0000000000000009 (XEN) 0000000000000000 ffffffff81463c48 ffffffff8100b8ab ffffffff81585d30 (XEN) ffffffff81586130 000000000000000f 8000000000000163 80000000018f6063 (XEN) ffffffffff400000 ffffffff81566000 0000000000000118 ffffffff81463ca8 (XEN) ffffffff8150f706 ffffffff814f40a0 0000000000000000 000009ff00000001 (XEN) ffffffffff400000 ffffffff81585d30 ffffffff81463d50 00000000018f6000 (XEN) 0000000240000000 00000002bf780000 00000002bf780000 ffffffff81463cb8 (XEN) ffffffff8150f7e2 ffffffff81463ce8 ffffffff81323321 0000000000000000 (XEN) ffff880001002048 00000002bf780000 0000000240000000 ffffffff81463d88 (XEN) ffffffff8152a0db ffffffff81463d98 ffffffff8103dc9e ffffffff81463d58 (XEN) 8000000000000163 ffff880001002000 0000000000000000 0000000000000000 (XEN) 0000000000000004 0000000581463d88 0000000000000000 ffffffff81585d30 (XEN) ffffffff81586130 0000000000000010 ffffffff81001880 00000002bf780000 (XEN) ffff8802bf780000 ffffffff813e37f4 ffff8802bf780000 ffffffff81463df8 (XEN) ffffffff8152a287 ffffffff81463df8 ffffffff81334a3c ffff880100000000 (XEN) 0000000000000000 ffffffff81463dc8 0000000040000000 ffffffff81463df8 (XEN) Domain 0 crashed: 'noreboot' set - not rebooting. -- Alan J. Wylie http://www.wylie.me.uk/ _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxxxxxxxx http://lists.xensource.com/xen-devel

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.