WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] xen/master pvops kernel boot consistently hangs on i7 wi

To: Bruce Edge <bruce.edge@xxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: RE: [Xen-devel] xen/master pvops kernel boot consistently hangs on i7 with xen 3.4/3.5 where 2.6.30.3 git kernel works.
From: "Yu, Ke" <ke.yu@xxxxxxxxx>
Date: Mon, 12 Oct 2009 14:10:30 +0800
Accept-language: en-US
Acceptlanguage: en-US
Cc:
Delivery-date: Sun, 11 Oct 2009 23:11:21 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <45c43ef50910110722m2598f485ibff531b8fb892953@xxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <45c43ef50910110722m2598f485ibff531b8fb892953@xxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcpKfmwderD7OjziSjWfswa1KkbykAAXYdFw
Thread-topic: [Xen-devel] xen/master pvops kernel boot consistently hangs on i7 with xen 3.4/3.5 where 2.6.30.3 git kernel works.
Could you try to add " hpet=disable" option to dom0 command line. If it works, 
then we meet the same issue. this issue is caused by the dom0 misusing of HPET, 
we are cooking a patch to fix this issue.

Best Regards
Ke

-----Original Message-----
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx 
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Bruce Edge
Sent: Sunday, October 11, 2009 10:23 PM
To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] xen/master pvops kernel boot consistently hangs on i7 with 
xen 3.4/3.5 where 2.6.30.3 git kernel works.

I'm running out of things to try. I would really appreciate a clue as
to what to look at next. It's not a full lockup as the CTRL-A*3 xen
debug console still works, but the xen/master dom0 shows no signs of
life.

The current 2.6.31.1 pvops  xen/master kernel hangs when running under
Xen on a proliant the a supermicro, both X58 i7 boxes.

I've tried booting with these xen versions (both of which work with a
2.6.30.3 dom0)
   http://xenbits.xensource.com/xen-unstable.hg
   http://xenbits.xensource.com/xen-3.4-testing.hg

And with the 2.6.31.1 source from both (is there a difference?)
   git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git linux-2.6-xen
   git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git xen/master

I've attached my .config, but here are the xen related settings:
CONFIG_XEN=y
CONFIG_XEN_TIME_VSYSCALL=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=32
CONFIG_XEN_SAVE_RESTORE=y
CONFIG_XEN_DEBUG_FS=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_DOM0_PCI=y
CONFIG_MICROCODE_XEN=y
CONFIG_PCI_XEN=y
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_XEN_KBDDEV_FRONTEND=m
CONFIG_HVC_XEN=y
CONFIG_XEN_FBDEV_FRONTEND=m
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_DEV_EVTCHN=y
CONFIG_XEN_BACKEND=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_XENFS=y
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_XEN_S3=y
CONFIG_XEN_GNTDEV=y

I'm building the kernels with make deb-pkg, installing the resultant 2
packages, firmware and kernel, and building the initrd. I assume that
that's no different from running make; make modules_install, etc... on
the target box?

The boot options I'm using are:
title           Xen 3.5-unstable / Ubuntu 9.04, kernel 2.6.31.1 clean
uuid            915b8d5f-dc27-4234-adac-26382dfef7c2
kernel          /xen-3.5-unstable.gz com1=115200,8n1 console=com1
module          /vmlinuz-2.6.31.1
root=UUID=e0a8abaf-ebb4-4a46-a968-36764653b87f ro console=xvc0
module          /initrd.img-2.6.31.1

as well as:

title           Xen 3.5-unstable / Ubuntu 9.04, kernel 2.6.31.1
uuid            915b8d5f-dc27-4234-adac-26382dfef7c2
kernel          /xen-3.5-unstable.gz com1=115200,8n1 console=com1
acpi=force apic=on dom0_max_vcpus=1 dom0_vcpus_pin=true
iommu=1,no-intremap,passthrough loglvl=all loglvl_guest=all
module          /vmlinuz-2.6.31.1
root=UUID=e0a8abaf-ebb4-4a46-a968-36764653b87f ro console=xvc0
pciback.hide=(07:00.0)(07:00.1)(07:00.2)(07:00.3)(14:00.0)(14:00.1)(17:00.0)(17:00.1)
acpi=force
module          /initrd.img-2.6.31.1

and everything in between.

I've tried booting the xen/master 2.6.31.1 kernel on bare metal
instead of under xen and there's zero console output.

Should this work, or does this kernel _only_ run under Xen?

I can boot both xen versions, 3.5-unstable and 3.4-testing with a
2.6.30.3 dom0. This _does_ work, but it uses the older kernel:

title           Xen 3.5-unstable / Ubuntu 9.04, kernel 2.6.30.3
uuid            915b8d5f-dc27-4234-adac-26382dfef7c2
kernel          /xen-3.5-unstable.gz com1=115200,8n1 console=com1
acpi=force apic=on dom0_max_vcpus=1 dom0_vcpus_pin=true
iommu=1,no-intremap,passthrough loglvl=all loglvl_guest=all
module          /vmlinuz-2.6.30.3
root=UUID=e0a8abaf-ebb4-4a46-a968-36764653b87f ro console=xvc0
pciback.hide=(07:00.0)(07:00.1)(07:00.2)(07:00.3)(14:00.0)(14:00.1)(17:00.0)(17:00.1)
acpi=force
module          /initrd.img-2.6.30.3

I've tried toggling these settings in the BIOS with no effect:
- hyperthreading
- exec memory protection
- disabling all unneeded PCI cards
- APIC

Any other BIOS items I should tweak to test?

This is the tail of the console output before the hang

(XEN) PCI add device 17:00.0
(XEN) PCI add device 17:00.1
(XEN) PCI add device 07:00.0
(XEN) PCI add device 07:00.1
(XEN) PCI add device 07:00.2
(XEN) PCI add device 07:00.3
(XEN) PCI add device 02:00.0
(XEN) PCI add device 02:00.1
(XEN) PCI add device 03:00.0
(XEN) PCI add device 03:00.1
(XEN) PCI add device 01:03.0
(XEN) PCI add device 01:04.0
(XEN) PCI add device 01:04.2
(XEN) PCI add device 01:04.4
(XEN) PCI add device 01:04.6
(XEN) io_apic.c:2208:
(XEN) ioapic_guest_write: apic=0, pin=20, irq=20
(XEN) ioapic_guest_write: new_entry=0001a014
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) io_apic.c:2208:
(XEN) ioapic_guest_write: apic=0, pin=23, irq=23
(XEN) ioapic_guest_write: new_entry=0001a017
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) io_apic.c:2208:
(XEN) ioapic_guest_write: apic=0, pin=22, irq=22
(XEN) ioapic_guest_write: new_entry=0001a016
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) *** Serial input -> Xen (type 'CTRL-a' three times to switch
input to DOM0)

The CTRL-a does still work. What output from that can I post to help debug this?

I've already posted the full boot output so I'll resist the temptation
to include it again.

-Bruce

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>