[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] (XEN) Domain 2 (vcpu#0) crashed on cpu#3:


  • To: xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>
  • From: David F Barrera <dfbp@xxxxxxxxxx>
  • Date: Thu, 29 Sep 2005 09:03:09 -0500
  • Delivery-date: Thu, 29 Sep 2005 16:24:45 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xensource.com>

Trace produced while running xm-test on IBM xSeries 335, 4GB RAM, PAE
enabled SLES 9 SP2 box.

http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=267


changeset:   7124:805ee053e61f

Previously, same workload was causing the Ouch! messages. 
Also, time has virtually stopped on the machine--it is moving forward
but very slowly, like 1 second for every 2 or 3 real minutes.


x335b:~ # xm dmesg
 __  __            _____  ___         _                _
 \ \/ /___ _ __   |___ / / _ \     __| | _____   _____| |
  \  // _ \ '_ \    |_ \| | | |__ / _` |/ _ \ \ / / _ \ |
  /  \  __/ | | |  ___) | |_| |__| (_| |  __/\ V /  __/ |
 /_/\_\___|_| |_| |____(_)___/    \__,_|\___| \_/ \___|_|

 http://www.cl.cam.ac.uk/netos/xen
 University of Cambridge Computer Laboratory

 Xen version 3.0-devel (root@xxxxxxxxxxxxxxxxxx) (gcc version 3.3.3
(SuSE Linux) ) Thu Sep 29 08:17:15 CDT 2005
 Latest ChangeSet: Thu Sep 29 07:05:43 2005 +0100 805ee053e61f

(XEN) Physical RAM map:
(XEN)  0000000000000000 - 000000000009d400 (usable)
(XEN)  000000000009d400 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000f7fec140 (usable)
(XEN)  00000000f7fec140 - 00000000f7ff0000 (ACPI data)
(XEN)  00000000f7ff0000 - 00000000f8000000 (reserved)
(XEN)  00000000fec00000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000000108000000 (usable)
(XEN) System RAM: 4095MB (4193828kB)
(XEN) Xen heap: 10MB (10496kB)
(XEN) PAE enabled, limit: 16 GB
(XEN) found SMP MP-table at 0009d540
(XEN) DMI 2.3 present.
(XEN) Using APIC driver default
(XEN) ACPI: RSDP (v000 IBM                                   ) @
0x000fdfc0
(XEN) ACPI: RSDT (v001 IBM    SERONYXP 0x00001000 IBM  0x45444f43) @
0xf7feff80
(XEN) ACPI: FADT (v001 IBM    SERONYXP 0x00001000 IBM  0x45444f43) @
0xf7feff00
(XEN) ACPI: MADT (v001 IBM    SERONYXP 0x00001000 IBM  0x45444f43) @
0xf7fefe40
(XEN) ACPI: ASF! (v016 IBM    SERONYXP 0x00000001 IBM  0x45444f43) @
0xf7fefd80
(XEN) ACPI: DSDT (v001 IBM    SERTURQU 0x00001000 MSFT 0x0100000b) @
0x00000000
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 15:2 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x06] enabled)
(XEN) Processor #6 15:2 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
(XEN) Processor #1 15:2 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x07] enabled)
(XEN) Processor #7 15:2 APIC version 20
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] dfl dfl lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x06] dfl dfl lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] dfl dfl lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x07] dfl dfl lint[0x1])
(XEN) ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
(XEN) ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
(XEN) IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
(XEN) ACPI: IOAPIC (id[0x0c] address[0xfec02000] gsi_base[32])
(XEN) IOAPIC[2]: apic_id 12, version 17, address 0xfec02000, GSI 32-47
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 3 I/O APICs
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Initializing CPU#0
(XEN) Detected 3189.565 MHz processor.
(XEN) Using scheduler: Simple EDF Scheduler (sedf)
(XEN) CPU: Trace cache: 12K uops, L1 D cache: 8K
(XEN) CPU: L2 cache: 512K
(XEN) CPU: L3 cache: 1024K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU0: Intel(R) Xeon(TM) CPU 3.20GHz stepping 05
(XEN) Booting processor 1/1 eip 90000
(XEN) Initializing CPU#1
(XEN) CPU: Trace cache: 12K uops, L1 D cache: 8K
(XEN) CPU: L2 cache: 512K
(XEN) CPU: L3 cache: 1024K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU1: Intel(R) Xeon(TM) CPU 3.20GHz stepping 05
(XEN) Booting processor 2/6 eip 90000
(XEN) Initializing CPU#2
(XEN) CPU: Trace cache: 12K uops, L1 D cache: 8K
(XEN) CPU: L2 cache: 512K
(XEN) CPU: L3 cache: 1024K
(XEN) CPU: Physical Processor ID: 3
(XEN) CPU2: Intel(R) Xeon(TM) CPU 3.20GHz stepping 05
(XEN) Booting processor 3/7 eip 90000
(XEN) Initializing CPU#3
(XEN) CPU: Trace cache: 12K uops, L1 D cache: 8K
(XEN) CPU: L2 cache: 512K
(XEN) CPU: L3 cache: 1024K
(XEN) CPU: Physical Processor ID: 3
(XEN) CPU3: Intel(R) Xeon(TM) CPU 3.20GHz stepping 05
(XEN) Total of 4 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) ..TIMER: vector=0x31 pin1=2 pin2=-1
(XEN) checking TSC synchronization across 4 CPUs: passed.
(XEN) Platform timer is 1.193MHz PIT
(XEN) Brought up 4 CPUs
(XEN) mtrr: v2.0 (20020519)
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen-ELF header found:
'GUEST_OS=linux,GUEST_VER=2.6,XEN_VER=3.0,VIRT_BASE=
0xC0000000,PAE=yes,LOADER=generic'
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000003800000->0000000004000000 (125952 pages
to be all ocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: c0100000->c06396e4
(XEN)  Init. ramdisk: c063a000->c063a000
(XEN)  Phys-Mach map: c063a000->c06b7000
(XEN)  Start info:    c06b7000->c06b8000
(XEN)  Page tables:   c06b8000->c06c1000
(XEN)  Boot stack:    c06c1000->c06c2000
(XEN)  TOTAL:         c0000000->c0800000
(XEN)  ENTRY ADDRESS: c0100000
(XEN) Scrubbing Free
RAM: ...........................................done.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch
input to Xen ).
(XEN) microcode: CPU1 updated from revision 0x11 to 0x29, date =
08112004
(XEN) microcode: CPU2 updated from revision 0x11 to 0x29, date =
08112004
(XEN) microcode: CPU3 updated from revision 0x11 to 0x29, date =
08112004
(XEN) microcode: CPU0 updated from revision 0x11 to 0x29, date =
08112004
(XEN) mtrr: type mismatch for fd000000,800000 old: write-back new:
write-combini ng
(XEN) mtrr: type mismatch for fd000000,800000 old: write-back new:
write-combini ng
(XEN) Domain 2 (vcpu#0) crashed on cpu#3:
(XEN) CPU:    3
(XEN) EIP:    e019:[<c0115a26>]
(XEN) EFLAGS: 00000246   CONTEXT: guest
(XEN) eax: 00000e90   ebx: 00000000   ecx: 00000000   edx: 00000000
(XEN) esi: 00000000   edi: 00000000   ebp: 00000e90   esp: c0355e8c
(XEN) cr0: 8005003b   cr3: f7f0d000
(XEN) ds: e021   es: e021   fs: 0000   gs: e021   ss: e021   cs: e019
(XEN) Guest stack trace from esp=c0355e8c:
(XEN)    c0000e94 00000003 c0115a26 0001e019 00010246 00000000 00000000
00000000 
(XEN)    c0000000 c0361885 c27d2000 00001000 00000000 00000000 c022b798
c0384580 
(XEN)    00000000 c0800000 c0745020 00000800 c0361bff c0745020 00000400
c02eff74 
(XEN)    c0355f60 00004000 00000000 00000000 00000000 00000000 00004000
00000000 
(XEN)    00000004 00000003 c0744018 00000000 c0744000 00000000 00000000
c0362035 
(XEN)    c0744000 00000000 c0364890 c038b850 c0353060 c02c71f7 00000000
c0362308 
(XEN)    0064b7ff c02eff74 c0353060 c035d1e5 c02eff74 c0100000 c035d83b
c0355ff4 
(XEN)    c036cc00 00000080 00000000 0000006c 00000000 00000000 00000000
00000000 
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 
(XEN)    00000000 00000000 00000000 00000000 00000000 ffffe000 c0741000
00000000 
(XEN)    ffffe000 c0741000 00000000 00000000 c0356867 c0355ff4 00000000
00000000 
(XEN)    00000000 00000000 c03838c0 0702080b c010007a
(XEN) (file=memory.c, line=57) Could not allocate order=0 extent: id=5
flags=0 ( 36782 of 62338)
(XEN) Domain 6 (vcpu#0) crashed on cpu#3:
(XEN) CPU:    3
(XEN) EIP:    e019:[<c0115a26>]
(XEN) EFLAGS: 00000246   CONTEXT: guest
(XEN) eax: 00000898   ebx: 00000000   ecx: 00000000   edx: 00000000
(XEN) esi: 00000000   edi: 00000000   ebp: 00000898   esp: c0355e8c
(XEN) cr0: 8005003b   cr3: f6a9c000
(XEN) ds: e021   es: e021   fs: 0000   gs: e021   ss: e021   cs: e019
(XEN) Guest stack trace from esp=c0355e8c:
(XEN)    c000089c 00000003 c0115a26 0001e019 00010246 00000000 00000000
00000000 
(XEN)    c0000000 c0361885 c3513000 00001000 00000000 00000000 c022b798
c0384580 
(XEN)    00000000 c0800000 c0745020 00000800 c0361bff c0745020 00000400
c02eff74 
(XEN)    c0355f60 00004000 00000000 00000000 00000000 00000000 00004000
00000000 
(XEN)    00000004 00000003 c0744018 00000000 c0744000 00000000 00000000
c0362035 
(XEN)    c0744000 00000000 c0364890 c038b850 c0353060 c02c71f7 00000000
c0362308 
(XEN)    0064b7ff c02eff74 c0353060 c035d1e5 c02eff74 c0100000 c035d83b
c0355ff4 
(XEN)    c036cc00 00000080 00000000 0000006c 00000000 00000000 00000000
00000000 
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 
(XEN)    00000000 00000000 00000000 00000000 00000000 ffffe000 c0741000
00000000 
(XEN)    ffffe000 c0741000 00000000 00000000 c0356867 c0355ff4 00000000
00000000 
(XEN)    00000000 00000000 c03838c0 0702080b c010007a
(XEN) Domain 15 (vcpu#0) crashed on cpu#3:
(XEN) CPU:    3
(XEN) EIP:    e019:[<c0115a26>]
(XEN) EFLAGS: 00000246   CONTEXT: guest
(XEN) eax: 00000480   ebx: 00000000   ecx: 00000000   edx: 00000000
(XEN) esi: 00000000   edi: 00000000   ebp: 00000480   esp: c0355e8c
(XEN) cr0: 8005003b   cr3: f6ac8000
(XEN) ds: e021   es: e021   fs: 0000   gs: e021   ss: e021   cs: e019
(XEN) Guest stack trace from esp=c0355e8c:
(XEN)    c0000484 00000003 c0115a26 0001e019 00010246 00000000 00000000
00000000 
(XEN)    c0000000 c0361885 c3490000 00001000 00000000 00000000 c022b798
c0384580 
(XEN)    00000000 c0800000 c0745020 00000800 c0361bff c0745020 00000400
c02eff74 
(XEN)    c0355f60 00004000 00000000 00000000 00000000 00000000 00004000
00000000 
(XEN)    00000004 00000003 c0744018 00000000 c0744000 00000000 00000000
c0362035 
(XEN)    c0744000 00000000 c0364890 c038b850 c0353060 c02c71f7 00000000
c0362308 
(XEN)    0064b7ff c02eff74 c0353060 c035d1e5 c02eff74 c0100000 c035d83b
c0355ff4 
(XEN)    c036cc00 00000080 00000000 0000006c 00000000 00000000 00000000
00000000 
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 
(XEN)    00000000 00000000 00000000 00000000 00000000 ffffe000 c0741000
00000000 
(XEN)    ffffe000 c0741000 00000000 00000000 c0356867 c0355ff4 00000000
00000000 
(XEN)    00000000 00000000 c03838c0 0702080b c010007a
(XEN) Domain 25 (vcpu#0) crashed on cpu#3:
(XEN) CPU:    3
(XEN) EIP:    e019:[<c0115a26>]
(XEN) EFLAGS: 00000246   CONTEXT: guest
(XEN) eax: 00000d28   ebx: 00000000   ecx: 00000000   edx: 00000000
(XEN) esi: 00000000   edi: 00000000   ebp: 00000d28   esp: c0355e8c
(XEN) cr0: 8005003b   cr3: f6af3000
(XEN) ds: e021   es: e021   fs: 0000   gs: e021   ss: e021   cs: e019
(XEN) Guest stack trace from esp=c0355e8c:
(XEN)    c0000d2c 00000003 c0115a26 0001e019 00010246 00000000 00000000
00000000 
(XEN)    c0000000 c0361885 c1fa5000 00001000 00000000 00000000 c022b798
c0384580 
(XEN)    00000000 c0800000 c0745020 00000800 c0361bff c0745020 00000400
c02eff74 
(XEN)    c0355f60 00004000 00000000 00000000 00000000 00000000 00004000
00000000 
(XEN)    00000004 00000003 c0744018 00000000 c0744000 00000000 00000000
c0362035 
(XEN)    c0744000 00000000 c0364890 c038b850 c0353060 c02c71f7 00000000
c0362308 
(XEN)    0064b7ff c02eff74 c0353060 c035d1e5 c02eff74 c0100000 c035d83b
c0355ff4 
(XEN)    c036cc00 00000080 00000000 0000006c 00000000 00000000 00000000
00000000 
(XEN)    00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 
(XEN)    00000000 00000000 00000000 00000000 00000000 ffffe000 c0741000
00000000 
(XEN)    ffffe000 c0741000 00000000 00000000 c0356867 c0355ff4 00000000
00000000 
(XEN)    00000000 00000000 c03838c0 0702080b c010007a


-- 
Regards,

David F Barrera
Linux Technology Center
Systems and Technology Group, IBM

"The wisest men follow their own direction. "
                                                        Euripides

Attachment: xend-debug.log
Description: Text Data

Attachment: xend.log
Description: Text Data

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.