WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] xen/master pvops kernel boot consistently hangs on i7 with x

To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] xen/master pvops kernel boot consistently hangs on i7 with xen 3.4/3.5 where 2.6.30.3 git kernel works.
From: Bruce Edge <bruce.edge@xxxxxxxxx>
Date: Sun, 11 Oct 2009 07:22:43 -0700
Delivery-date: Sun, 11 Oct 2009 07:23:29 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=V4X4Q69CeWhPY/UhVdl+YBl3IXJWEMX28AIXnUDj/Jw=; b=mmtzh+L987LGq+IM+ro9Kcg4a3vLqBlry6XLa4SjSMQuR+F3Sp1175lvXhnkU21/NA RxA/uo/IZIMFZvhWU5zLrBPMbzGmt7Rnz0kwTByl5ItyVPbyPkLJUB8+1Afgk4zsm4nh iUbukaONrpIgaUBXwamS2DT4QOno67AnXNpdg=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=DeMjbxWVR3Q88NW10HlvzGka+15mmFwV3csnUQDU6nAqcywv16b4bk1In5A72gW8ph 7gD2ApZHY6BczuCGreKFB8KT1CLewpLX1h1ASFSuYdYamIXroOPVF+fpwXQbp/vpeJVQ j6LyHEb4BDsmQ0VEbIjDfFc+PEFEfAav5phsA=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
I'm running out of things to try. I would really appreciate a clue as
to what to look at next. It's not a full lockup as the CTRL-A*3 xen
debug console still works, but the xen/master dom0 shows no signs of
life.

The current 2.6.31.1 pvops  xen/master kernel hangs when running under
Xen on a proliant the a supermicro, both X58 i7 boxes.

I've tried booting with these xen versions (both of which work with a
2.6.30.3 dom0)
   http://xenbits.xensource.com/xen-unstable.hg
   http://xenbits.xensource.com/xen-3.4-testing.hg

And with the 2.6.31.1 source from both (is there a difference?)
   git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git linux-2.6-xen
   git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git xen/master

I've attached my .config, but here are the xen related settings:
CONFIG_XEN=y
CONFIG_XEN_TIME_VSYSCALL=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=32
CONFIG_XEN_SAVE_RESTORE=y
CONFIG_XEN_DEBUG_FS=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_DOM0_PCI=y
CONFIG_MICROCODE_XEN=y
CONFIG_PCI_XEN=y
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_XEN_KBDDEV_FRONTEND=m
CONFIG_HVC_XEN=y
CONFIG_XEN_FBDEV_FRONTEND=m
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_DEV_EVTCHN=y
CONFIG_XEN_BACKEND=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_XENFS=y
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_XEN_S3=y
CONFIG_XEN_GNTDEV=y

I'm building the kernels with make deb-pkg, installing the resultant 2
packages, firmware and kernel, and building the initrd. I assume that
that's no different from running make; make modules_install, etc... on
the target box?

The boot options I'm using are:
title           Xen 3.5-unstable / Ubuntu 9.04, kernel 2.6.31.1 clean
uuid            915b8d5f-dc27-4234-adac-26382dfef7c2
kernel          /xen-3.5-unstable.gz com1=115200,8n1 console=com1
module          /vmlinuz-2.6.31.1
root=UUID=e0a8abaf-ebb4-4a46-a968-36764653b87f ro console=xvc0
module          /initrd.img-2.6.31.1

as well as:

title           Xen 3.5-unstable / Ubuntu 9.04, kernel 2.6.31.1
uuid            915b8d5f-dc27-4234-adac-26382dfef7c2
kernel          /xen-3.5-unstable.gz com1=115200,8n1 console=com1
acpi=force apic=on dom0_max_vcpus=1 dom0_vcpus_pin=true
iommu=1,no-intremap,passthrough loglvl=all loglvl_guest=all
module          /vmlinuz-2.6.31.1
root=UUID=e0a8abaf-ebb4-4a46-a968-36764653b87f ro console=xvc0
pciback.hide=(07:00.0)(07:00.1)(07:00.2)(07:00.3)(14:00.0)(14:00.1)(17:00.0)(17:00.1)
acpi=force
module          /initrd.img-2.6.31.1

and everything in between.

I've tried booting the xen/master 2.6.31.1 kernel on bare metal
instead of under xen and there's zero console output.

Should this work, or does this kernel _only_ run under Xen?

I can boot both xen versions, 3.5-unstable and 3.4-testing with a
2.6.30.3 dom0. This _does_ work, but it uses the older kernel:

title           Xen 3.5-unstable / Ubuntu 9.04, kernel 2.6.30.3
uuid            915b8d5f-dc27-4234-adac-26382dfef7c2
kernel          /xen-3.5-unstable.gz com1=115200,8n1 console=com1
acpi=force apic=on dom0_max_vcpus=1 dom0_vcpus_pin=true
iommu=1,no-intremap,passthrough loglvl=all loglvl_guest=all
module          /vmlinuz-2.6.30.3
root=UUID=e0a8abaf-ebb4-4a46-a968-36764653b87f ro console=xvc0
pciback.hide=(07:00.0)(07:00.1)(07:00.2)(07:00.3)(14:00.0)(14:00.1)(17:00.0)(17:00.1)
acpi=force
module          /initrd.img-2.6.30.3

I've tried toggling these settings in the BIOS with no effect:
- hyperthreading
- exec memory protection
- disabling all unneeded PCI cards
- APIC

Any other BIOS items I should tweak to test?

This is the tail of the console output before the hang

(XEN) PCI add device 17:00.0
(XEN) PCI add device 17:00.1
(XEN) PCI add device 07:00.0
(XEN) PCI add device 07:00.1
(XEN) PCI add device 07:00.2
(XEN) PCI add device 07:00.3
(XEN) PCI add device 02:00.0
(XEN) PCI add device 02:00.1
(XEN) PCI add device 03:00.0
(XEN) PCI add device 03:00.1
(XEN) PCI add device 01:03.0
(XEN) PCI add device 01:04.0
(XEN) PCI add device 01:04.2
(XEN) PCI add device 01:04.4
(XEN) PCI add device 01:04.6
(XEN) io_apic.c:2208:
(XEN) ioapic_guest_write: apic=0, pin=20, irq=20
(XEN) ioapic_guest_write: new_entry=0001a014
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) io_apic.c:2208:
(XEN) ioapic_guest_write: apic=0, pin=23, irq=23
(XEN) ioapic_guest_write: new_entry=0001a017
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) io_apic.c:2208:
(XEN) ioapic_guest_write: apic=0, pin=22, irq=22
(XEN) ioapic_guest_write: new_entry=0001a016
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) *** Serial input -> Xen (type 'CTRL-a' three times to switch
input to DOM0)

The CTRL-a does still work. What output from that can I post to help debug this?

I've already posted the full boot output so I'll resist the temptation
to include it again.

-Bruce

Attachment: xen.config
Description: Binary data

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
<Prev in Thread] Current Thread [Next in Thread>