|
xen-devel
[Xen-devel] Re: [RFC PATCH linux-2.6.18-xen] pciback: clean up (MSI-X ve
(Perhaps it's best to write a separate mail about this.)
On 06/01/11 12:05, Laszlo Ersek wrote:
> Hi,
>
> this is more of an RFC than a patch now for linux-2.6.18-xen. Describing the
> situation captured in RHBZ#688673, in a nutshell:
>
> - let's say we have an Intel 82576 card (igb),
> - the card exports virtual functions (igbvf),
> - one virtual function is passed through to a PV guest,
> - the igbvf driver, co-operating with pcifront, pciback, and the hypervisor,
> sets up three MSI-X vectors for the virtual function PCI device,
> - when the domU is shut down, the MSI-X vectors are "leaked" in dom0, because
> nobody ever reaches dom0's pci_disable_msix() / msi_remove_pci_irq_vectors()
> during shutdown,
> - therefore configuration of the VF during the next boot of such a guest
> doesn't succeed (dom0 thinks, based on its stale list, that MSI-X vectors are
> already allocated/mapped for the device)
I tried to install a Fedora 15 PV guest as well (the host/hv was RHEL-5
kernel-xen, -264 build). Surprisingly, when the igbvf driver called
pci_enable_msix() in the PV guest at initialization time, it bugged out:
$ addr2line -p -f -i -e
/usr/lib/debug/lib/modules/2.6.38.6-26.rc1.fc15.x86_64/vmlinux
<<<ffffffff8125658d
readl at
/usr/src/debug/kernel-2.6.38.fc15/linux-2.6.38.x86_64/arch/x86/include/asm/io.h:58
(inlined by) msix_program_entries at
/usr/src/debug/kernel-2.6.38.fc15/linux-2.6.38.x86_64/drivers/pci/msi.c:527
(inlined by) msix_capability_init at
/usr/src/debug/kernel-2.6.38.fc15/linux-2.6.38.x86_64/drivers/pci/msi.c:578
(inlined by) pci_enable_msix at
/usr/src/debug/kernel-2.6.38.fc15/linux-2.6.38.x86_64/drivers/pci/msi.c:806
(inlined by) pci_enable_msix at
/usr/src/debug/kernel-2.6.38.fc15/linux-2.6.38.x86_64/drivers/pci/msi.c:773
Thanks,
lacos
[168711.841672] BUG: unable to handle kernel paging request at ffffc90001a9600c
[168711.841984] IP: [<ffffffff8125658d>] pci_enable_msix+0x270/0x354
[168711.842343] PGD 9c838067 PUD 9c839067 PMD 3216067 PTE 0
[168711.842675] Oops: 0000 [#1] SMP [168711.843016] last sysfs file:
/sys/devices/virtual/block/dm-0/dev
[168711.843486] CPU 0 [168711.843507] Modules linked in: igbvf(+) joydev
xen_pcifront xen_netfront ipv6 xen_blkfront
[168711.844504] [168711.845030] Pid: 449, comm: modprobe Not tainted
2.6.38.6-26.rc1.fc15.x86_64 #1 [168711.845722] RIP: e030:[<ffffffff8125658d>]
[<ffffffff8125658d>] pci_enable_msix+0x270/0x354
[168711.846440] RSP: e02b:ffff880098b4bc58 EFLAGS: 00010286
[168711.847172] RAX: ffffc90001a9600c RBX: ffff880003304000 RCX:
0000000000000006
[168711.847960] RDX: 0000000000000006 RSI: ffff88009c800200 RDI:
ffff88009c800200
[168711.848775] RBP: ffff880098b4bcc8 R08: ffff88009c80c000 R09:
0000000000000002
[168711.849634] R10: ffff880098361400 R11: 0000000000000003 R12:
ffff880002ceac80
[168711.850531] R13: ffff880096278280 R14: 0000000000000000 R15:
ffff8800033048f0
[168711.851452] FS: 00007f2ee5360720(0000) GS:ffff88009fdf5000(0000)
knlGS:0000000000000000
[168711.852434] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[168711.853454] CR2: ffffc90001a9600c CR3: 00000000033ce000 CR4:
0000000000002620
[168711.854529] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[168711.855638] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000000
[168711.856776] Process modprobe (pid: 449, threadinfo ffff880098b4a000, task
ffff880003acae40)
[168711.857954] Stack:
[168711.859139] ffff880096278280 0000000000000002 0000000000000000
ffff8800033048f0
[168711.860445] 0000007298b4bcc8 ffffc9000000000c ffff880000000008
c002ffff00000000
[168711.861777] ffff880003304000 ffff880098350000 ffff880003304000
ffff880003304090
[168711.863149] Call Trace:
[168711.864527] [<ffffffffa0083400>] igbvf_probe+0x308/0x85b [igbvf]
[168711.865978] [<ffffffff81006c3f>] ? xen_restore_fl_direct_end+0x0/0x1
[168711.867469] [<ffffffff81080b0f>] ? arch_local_irq_restore+0xb/0xd
[168711.868984] [<ffffffff812463ce>] local_pci_probe+0x44/0x75
[168711.870531] [<ffffffff81246e60>] pci_device_probe+0xd0/0xff
[168711.872170] [<ffffffff812e6af9>] driver_probe_device+0x124/0x1ff
[168711.873804] [<ffffffff812e6c2e>] __driver_attach+0x5a/0x7e
[168711.875436] [<ffffffff812e6bd4>] ? __driver_attach+0x0/0x7e
[168711.875441] [<ffffffff812e5b9a>] bus_for_each_dev+0x53/0x89
[168711.875446] [<ffffffff812e6706>] driver_attach+0x1e/0x20
[168711.875455] [<ffffffff812e6318>] bus_add_driver+0xe2/0x235
[168711.875462] [<ffffffffa0089000>] ? igbvf_init_module+0x0/0x51 [igbvf]
[168711.875468] [<ffffffff812e6e17>] driver_register+0x98/0x105
[168711.875474] [<ffffffffa0089000>] ? igbvf_init_module+0x0/0x51 [igbvf]
[168711.875480] [<ffffffff81247723>] __pci_register_driver+0x56/0xc3
[168711.875486] [<ffffffffa0089000>] ? igbvf_init_module+0x0/0x51 [igbvf]
[168711.875493] [<ffffffffa008904f>] igbvf_init_module+0x4f/0x51 [igbvf]
[168711.875500] [<ffffffff810020a4>] do_one_initcall+0x7f/0x137
[168711.875506] [<ffffffff810852dd>] sys_init_module+0xa6/0x1e4
[168711.875511] [<ffffffff81009bc2>] system_call_fastpath+0x16/0x1b
[168711.875514] Code: 00 0f b7 42 04 c1 e0 04 83 c0 0c 89 45 b8 41 8b 44 24 0c
89 02 41 8b 7c 24 0c 89 4d a0 e8 d0 80 e5 ff 48 63 45 b8 49 03 44 24 20 <8b> 00
be 01 00 00 00 41 89 44 24 08 4c 89 e7 e8 c1 f2 ff ff 4d [168711.875552] RIP
[<ffffffff8125658d>] pci_enable_msix+0x270/0x354
[168711.875558] RSP <ffff880098b4bc58>
[168711.875560] CR2: ffffc90001a9600c
[168711.875564] ---[ end trace c82c0d02bae884a2 ]---
[168711.875570] BUG: sleeping function called from invalid context at
kernel/rwsem.c:21
[168711.875573] in_atomic(): 0, irqs_disabled(): 1, pid: 449, name: modprobe
[168711.875577] Pid: 449, comm: modprobe Tainted: G D
2.6.38.6-26.rc1.fc15.x86_64 #1
[168711.875580] Call Trace:
[168711.875586] [<ffffffff81047d33>] __might_sleep+0xeb/0xf0
[168711.875593] [<ffffffff81474eee>] down_read+0x21/0x38
[168711.875598] [<ffffffff8108be35>] acct_collect+0x4a/0x182
[168711.875603] [<ffffffff81058627>] do_exit+0x216/0x732
[168711.875608] [<ffffffff8147588c>] ? _raw_spin_unlock_irqrestore+0x17/0x19
[168711.875613] [<ffffffff81006c52>] ? check_events+0x12/0x20
[168711.875618] [<ffffffff81476b8e>] oops_end+0xbc/0xc5
[168711.875623] [<ffffffff8146c0b4>] no_context+0x203/0x212
[168711.875628] [<ffffffff81004415>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
[168711.875633] [<ffffffff8146c257>] __bad_area_nosemaphore+0x194/0x1b7
[168711.875637] [<ffffffff8146b8e4>] ? pmd_val+0x10/0x12
[168711.875642] [<ffffffff8146b945>] ? pte_offset_kernel+0x19/0x3f
[168711.875647] [<ffffffff8146c28d>] bad_area_nosemaphore+0x13/0x15
[168711.875651] [<ffffffff81478c5d>] do_page_fault+0x1c5/0x37a
[168711.875657] [<ffffffff81476095>] page_fault+0x25/0x30
[168711.875662] [<ffffffff8125658d>] ? pci_enable_msix+0x270/0x354
[168711.875667] [<ffffffff81256584>] ? pci_enable_msix+0x267/0x354
[168711.875675] [<ffffffffa0083400>] igbvf_probe+0x308/0x85b [igbvf]
[168711.875680] [<ffffffff81006c3f>] ? xen_restore_fl_direct_end+0x0/0x1
[168711.875684] [<ffffffff81080b0f>] ? arch_local_irq_restore+0xb/0xd
[168711.875689] [<ffffffff812463ce>] local_pci_probe+0x44/0x75
[168711.875693] [<ffffffff81246e60>] pci_device_probe+0xd0/0xff
[168711.875698] [<ffffffff812e6af9>] driver_probe_device+0x124/0x1ff
[168711.875703] [<ffffffff812e6c2e>] __driver_attach+0x5a/0x7e
[168711.875708] [<ffffffff812e6bd4>] ? __driver_attach+0x0/0x7e
[168711.875713] [<ffffffff812e5b9a>] bus_for_each_dev+0x53/0x89
[168711.875718] [<ffffffff812e6706>] driver_attach+0x1e/0x20
[168711.875722] [<ffffffff812e6318>] bus_add_driver+0xe2/0x235
[168711.875729] [<ffffffffa0089000>] ? igbvf_init_module+0x0/0x51 [igbvf]
[168711.875734] [<ffffffff812e6e17>] driver_register+0x98/0x105
[168711.875741] [<ffffffffa0089000>] ? igbvf_init_module+0x0/0x51 [igbvf]
[168711.875746] [<ffffffff81247723>] __pci_register_driver+0x56/0xc3
[168711.875752] [<ffffffffa0089000>] ? igbvf_init_module+0x0/0x51 [igbvf]
[168711.875759] [<ffffffffa008904f>] igbvf_init_module+0x4f/0x51 [igbvf]
[168711.875764] [<ffffffff810020a4>] do_one_initcall+0x7f/0x137
[168711.875769] [<ffffffff810852dd>] sys_init_module+0xa6/0x1e4
[168711.875773] [<ffffffff81009bc2>] system_call_fastpath+0x16/0x1b
[168711.879089] udevd-work[367]: '/sbin/modprobe -bv
pci:v00008086d000010CAsv000015D9sd000010C9bc02sc00i00' unexpected exit with
status 0x0009
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
|
|
Home