Sorry, something went wrong with copy and pasting, here is the complete output:
(gdb) tar rem :9999
Remote debugging using :9999
[New Thread 0]
[Switching to Thread 0]
smp_call_function_many (mask=0xffffffff81ce8e70, func=0xffffffff81007c4d
<stop_self>, info=0x0, wait=true) at kernel/smp.c:104
104 while (data->flags & CSD_FLAG_LOCK)
(gdb) thread apply all bt
[New Thread 1]
[New Thread 2]
[New Thread 3]
Thread 4 (Thread 3):
#0 0xffffffff8100130a in hypercall_page ()
#1 0xffffffffffff02dd in ?? ()
#2 0x0000000000000080 in irq_stack_union ()
#3 0xffffffff81007c9a in stop_self (v=<value optimized out>) at
/usr/src/new/linux-2.6-usb-msi/arch/x86/include/asm/xen/hypercall.h:413
#4 0xffffffff81077adf in generic_smp_call_function_interrupt () at
kernel/smp.c:200
#5 0xffffffff81007eb2 in xen_call_function_interrupt (irq=<value optimized
out>, dev_id=<value optimized out>) at arch/x86/xen/smp.c:472
#6 0xffffffff8109907b in handle_IRQ_event (irq=286, action=0xffff88001e8257e0)
at kernel/irq/handle.c:68
#7 0xffffffff8109ae9c in handle_percpu_irq (irq=286, desc=0xffff88001ea05500)
at kernel/irq/chip.c:684
#8 0xffffffff8127a3b0 in __xen_evtchn_do_upcall () at
include/linux/irqdesc.h:119
#9 0xffffffff8127ab80 in xen_evtchn_do_upcall (regs=<value optimized out>) at
drivers/xen/events.c:1143
#10 0xffffffff8100bc2e in xen_do_hypervisor_callback () at
arch/x86/kernel/entry_64.S:1233
#11 0xffff88001e9e3e28 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 3 (Thread 2):
#0 0xffffffff8100130a in hypercall_page ()
#1 0xffffffff81006fbd in xen_force_evtchn_callback () at
/usr/src/new/linux-2.6-usb-msi/arch/x86/include/asm/xen/hypercall.h:362
#2 0xffffffff81077adf in generic_smp_call_function_interrupt () at
kernel/smp.c:200
#3 0xffffffff81007eb2 in xen_call_function_interrupt (irq=<value optimized
out>, dev_id=<value optimized out>) at arch/x86/xen/smp.c:472
#4 0xffffffff8109907b in handle_IRQ_event (irq=291, action=0xffff88001e825600)
at kernel/irq/handle.c:68
#5 0xffffffff8109ae9c in handle_percpu_irq (irq=291, desc=0xffff88001ea05000)
at kernel/irq/chip.c:684
#6 0xffffffff8127a3b0 in __xen_evtchn_do_upcall () at
include/linux/irqdesc.h:119
#7 0xffffffff8127ab80 in xen_evtchn_do_upcall (regs=<value optimized out>) at
drivers/xen/events.c:1143
#8 0xffffffff8100bc2e in xen_do_hypervisor_callback () at
arch/x86/kernel/entry_64.S:1233
#9 0xffff88001e9e1e28 in ?? ()
#10 0x0000000000000000 in ?? ()
Current language: auto; currently asm
Thread 2 (Thread 1):
#0 0xffffffff8100130a in hypercall_page ()
#1 0xffffffff81ce8e70 in cpu_possible_bits ()
#2 0x0000000000000080 in irq_stack_union ()
#3 0xffffffff81007c9a in stop_self (v=<value optimized out>) at
/usr/src/new/linux-2.6-usb-msi/arch/x86/include/asm/xen/hypercall.h:413
#4 0xffffffff81077adf in generic_smp_call_function_interrupt () at
kernel/smp.c:200
#5 0xffffffff81007eb2 in xen_call_function_interrupt (irq=<value optimized
out>, dev_id=<value optimized out>) at arch/x86/xen/smp.c:472
#6 0xffffffff8109907b in handle_IRQ_event (irq=296, action=0xffff88001e825420)
at kernel/irq/handle.c:68
#7 0xffffffff8109ae9c in handle_percpu_irq (irq=296, desc=0xffff88001e824b00)
at kernel/irq/chip.c:684
#8 0xffffffff8127a3b0 in __xen_evtchn_do_upcall () at
include/linux/irqdesc.h:119
#9 0xffffffff8127ab80 in xen_evtchn_do_upcall (regs=<value optimized out>) at
drivers/xen/events.c:1143
#10 0xffffffff8100bc2e in xen_do_hypervisor_callback () at
arch/x86/kernel/entry_64.S:1233
#11 0xffff88001e9d7e28 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 1 (Thread 0):
#0 smp_call_function_many (mask=0xffffffff81ce8e70, func=0xffffffff81007c4d
<stop_self>, info=0x0, wait=true) at kernel/smp.c:104
#1 0xffffffff81077a5a in smp_call_function (func=0x40 <irq_stack_union+64>,
info=<value optimized out>, wait=<value optimized out>) at kernel/smp.c:506
#2 0xffffffff81007eda in xen_stop_other_cpus (wait=<value optimized out>) at
arch/x86/xen/smp.c:431
#3 0xffffffff81003172 in xen_reboot () at
/usr/src/new/linux-2.6-usb-msi/arch/x86/include/asm/smp.h:81
#4 0xffffffff810031b5 in xen_machine_halt () at arch/x86/xen/enlighten.c:1039
#5 0xffffffff81023795 in machine_halt () at arch/x86/kernel/reboot.c:726
#6 0xffffffff8105e9b3 in kernel_halt () at kernel/sys.c:336
#7 0xffffffff8105eb03 in sys_reboot (magic1=-18751827, magic2=672274793,
cmd=3454992675, arg=0x0) at kernel/sys.c:407
#8 0xffffffff8100acc2 in system_call_fastpath () at
arch/x86/kernel/entry_64.S:479
#9 0x0000000000000202 in irq_stack_union ()
#10 0x0000000000000000 in ?? ()
Current language: auto; currently c
Friday, November 19, 2010, 6:07:43 PM, you wrote:
> On 11/18/2010 11:57 AM, Sander Eikelenboom wrote:
>> Hi Jeremy/Keir,
>>
>> I have tried to add the following lines:
>> if (!strcmp(type, "console"))
>> return 0;
>>
>> But it seems to be a red herring .. although the "the xenbus_dev_shutdown:
>> device/console/0: Initialising != Connected, skipping" warning is gone.
>> The PV domU guest still doesn't get completely halted:
>> - It stays in a running state
>> - It uses 100% cpu according to xentop
>> - I can still connect to it's console
>>
>> So probably something is stuck in a waiting loop, which doesn't sleep in
>> between and consumes 100% cpu.
> Hm, I generally see clean shutdowns of my multi-VCPU domains.
> Can you do
> # gdbsx -a <domid> <arch-size> 9999 (arch-size is 32 or 64)
> then
> $ gdb vmlinux
> (gdb) tar rem :9999
> (gdb) thread apply all bt
> to see what's going on? (You may need to enable CONFIG_DEBUG_INFO to
> get useful info if you haven't already.)
> J
>> Last lines of domU's console(with some additional printk's in
>> xenbus_dev_shutdown):
>>
>> Cannot access the Hardware Clock via any known method.
>> Use the --debug option to see the details of our search for an access method.
>> Stopping enhanced syslogd: rsyslogd.
>> Asking all remaining processes to terminate...done.
>> All processes ended within 2 seconds....done.
>> Deconfiguring network interfaces...done.
>> Cleaning up ifupdown....
>> Deactivating swap...done.
>> Unmounting local filesystems...done.
>> Will now halt.
>> [ 46.643035] md: stopping all md devices.
>> [ 47.643320] xenbus_dev_shutdown: trying shutdown of device/vif/0:
>> Connected
>> [ 47.716448] xenbus_dev_shutdown: result of shutdown of device/vif/0:
>> Closed
>> [ 47.716461] xenbus_dev_shutdown: trying shutdown of device/vbd/51714:
>> Connected
>> [ 47.772293] xenbus_dev_shutdown: result of shutdown of device/vbd/51714:
>> Closed
>> [ 47.772306] xenbus_dev_shutdown: trying shutdown of device/vbd/51713:
>> Connected
>> [ 47.829384] xenbus_dev_shutdown: result of shutdown of device/vbd/51713:
>> Closed
>> [ 47.829415] System halted.
>>
>>
>>
>> it's a domU using a 2.6.37-rc2 kernel, file: based disk access, nothing very
>> special.
>>
>> the only difference that triggers it is:
>> vcpus=1 works fine ..
>> vcpus>1 symptoms above ..
>>
>> I have also added xend.log, with first a boot and shutdown with vpcus=1 and
>> then a boot and shutdown with vpcus=4
>> I left the domain(with 4vcpus) running with 100% cpu for 10 minutes, but
>> doesn't seem to hit any timeout, and keeps running.
>> On 20:40 it ends up stuck, on 20:51 i'm shooting the domain off with xm
>> destroy.
>>
>>
>> Any pointers for adding extra debug info ?
>>
>> --
>> Sander
>>
>>
>>
>>
>> Wednesday, November 17, 2010, 10:57:47 PM, you wrote:
>>
>>> On 11/17/2010 12:22 PM, Sander Eikelenboom wrote:
>>>> Ah yes, 2.6.18 does contain the following line in xenbus_probe.c and
>>>> 2.6.37-rc2 not:
>>>>
>>>> - if (!strcmp(type, "console"))
>>>> - return 0;
>>>> It seems to be changed,
>>>>
>>>>
>>>> There seems to have been a patch
>>>> http://lists.xensource.com/archives/html/xen-devel/2009-11/msg01072.html
>>>> but it seems it's not applied to xenbus in linux upstream.
>>> And does this patch fix things for you?
>>> But I have to say that's pretty gross. Is it really the right thing to do?
>>> J
>>>
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>>> http://lists.xensource.com/xen-devel
--
Best regards,
Sander mailto:linux@xxxxxxxxxxxxxx
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
Home