[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] Xen-unstable: pci-passthrough of device using MSI-X interrupts not working after commit x86/MSI: track host and guest masking separately



Hi Jan / Andrew,

I'm having some trouble with a xhci controller passed through with 
pci-passthrough to one of my HVM guests.
It uses MSI-X for interrupts, a bisection turned up the following commit:


ad28e42bd1d28d746988ed71654e8aa670629753 is the first bad commit
commit ad28e42bd1d28d746988ed71654e8aa670629753
Author: Jan Beulich <jbeulich@xxxxxxxx>
Date:   Fri Jun 19 10:59:53 2015 +0200

    x86/MSI: track host and guest masking separately

    In particular we want to avoid losing track of our own intention to
    have an entry masked. Physical unmasking now happens only when both
    host and guest requested so.

    Signed-off-by: Jan Beulich <jbeulich@xxxxxxxx>
    Reviewed-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>


Although from a first glance it looks as if the controller is correctly 
initialize during the boot of the HVM guest (no worrying messages in dmesg yet).
It utterly fails a simple "lsusb" this results in the hang pasted below.

Other devices  i passthrough which use legacy or MSI interrupts seem to be 
unaffected.

Please say so if you need any specific output from Xen debug keys or anything 
else !

--
Sander

The lspci from witin the guest (doesn't differ between good or bad):
00:05.0 USB controller [0c03]: NEC Corporation uPD720200 USB 3.0 Host 
Controller [1033:0194] (rev 03) (prog-if 30 [XHCI])
        Subsystem: ASUSTeK Computer Inc. P8P67 Deluxe Motherboard [1043:8413]
        Physical Slot: 5
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- 
<MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 36
        Region 0: Memory at f3270000 (64-bit, non-prefetchable) [size=8K]
        Capabilities: [50] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA 
PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [70] MSI: Enable- Count=1/1 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [90] MSI-X: Enable+ Count=8 Masked-
                Vector table: BAR=0 offset=00001000
                PBA: BAR=0 offset=00001080
        Capabilities: [a0] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s 
unlimited, L1 unlimited
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- 
Unsupported-
                        RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- 
TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit 
Latency L0s <4us, L1 unlimited
                        ClockPM+ Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- 
BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR+, 
OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, 
OBFF Disabled
                LnkSta2: Current De-emphasis Level: -6dB, 
EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, 
LinkEqualizationRequest-
        Kernel driver in use: xhci_hcd

lspci from dom0:
08:00.0 USB controller [0c03]: NEC Corporation uPD720200 USB 3.0 Host 
Controller [1033:0194] (rev 03) (prog-if 30 [XHCI])
        Subsystem: ASUSTeK Computer Inc. P8P67 Deluxe Motherboard [1043:8413]
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- 
<MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 37
        Region 0: Memory at fe0fe000 (64-bit, non-prefetchable) [size=8K]
        Capabilities: [50] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA 
PME(D0+,D1-,D2-,D3hot+,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [70] MSI: Enable- Count=1/8 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [90] MSI-X: Enable+ Count=8 Masked-
                Vector table: BAR=0 offset=00001000
                PBA: BAR=0 offset=00001080
        Capabilities: [a0] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s 
unlimited, L1 unlimited
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- 
Unsupported-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- 
TransPend-
                LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit 
Latency L0s <4us, L1 unlimited
                        ClockPM+ Surprise- LLActRep- BwNot-
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
                        ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- 
BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR+, 
OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, 
OBFF Disabled
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, 
EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, 
EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, 
LinkEqualizationRequest-
        Capabilities: [100 v1] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- 
RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- 
RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- 
RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 14, GenCap- CGenEn- ChkCap- ChkEn-
        Capabilities: [140 v1] Device Serial Number ff-ff-ff-ff-ff-ff-ff-ff
        Capabilities: [150 v1] Latency Tolerance Reporting
                Max snoop latency: 0ns
                Max no snoop latency: 0ns
        Kernel driver in use: pciback





[  240.783386] INFO: task kworker/3:1:853 blocked for more than 120 seconds.
[  240.788883]       Not tainted 4.1.0-rc7-20150609-security+ #1
[  240.793566] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this 
message.
[  240.805612] kworker/3:1     D ffff88003cdbbb38     0   853      2 0x00000000
[  240.817493] Workqueue: usb_hub_wq hub_event
[  240.824566]  ffff88003cdbbb38 00000000000040d0 ffff88003dafe8d0 
0000000000030010
[  240.846212]  ffff88003cdbc000 7fffffffffffffff ffff88003cedc150 
ffff88003dafe8d0
[  240.861651]  ffff88003dafe8d0 ffff88003cdbbb58 ffffffff81af0812 
0000000000000002
[  240.875103] Call Trace:
[  240.879301]  [<ffffffff81af0812>] schedule+0x32/0x80
[  240.885835]  [<ffffffff81af2fb9>] schedule_timeout+0x139/0x190
[  240.895976]  [<ffffffff8176b2cb>] ? usb_alloc_dev+0x2b/0x310
[  240.905071]  [<ffffffff81af11ee>] wait_for_common+0xce/0x1e0
[  240.929943]  [<ffffffff810d3800>] ? wake_up_process+0x50/0x50
[  240.938833]  [<ffffffff81af1318>] wait_for_completion+0x18/0x20
[  240.946345]  [<ffffffff817a064f>] xhci_alloc_dev+0x9f/0x270
[  240.954751]  [<ffffffff8176b309>] usb_alloc_dev+0x69/0x310
[  240.962074]  [<ffffffff81771719>] hub_event+0x7c9/0x12f0
[  240.968884]  [<ffffffff810c538d>] process_one_work+0x13d/0x380
[  240.978075]  [<ffffffff810c563b>] worker_thread+0x6b/0x550
[  240.985503]  [<ffffffff810c55d0>] ? process_one_work+0x380/0x380
[  240.992991]  [<ffffffff810caa66>] kthread+0xd6/0xf0
[  240.999994]  [<ffffffff810ca990>] ? kthread_create_on_node+0x180/0x180
[  241.008045]  [<ffffffff81af44a2>] ret_from_fork+0x42/0x70
[  241.015867]  [<ffffffff810ca990>] ? kthread_create_on_node+0x180/0x180
[  241.027458] INFO: task udevd:1600 blocked for more than 120 seconds.
[  241.035950]       Not tainted 4.1.0-rc7-20150609-security+ #1
[  241.043372] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this 
message.
[  241.055299] udevd           D ffff88003b4ffc38     0  1600   1562 0x00000004
[  241.066287]  ffff88003b4ffc38 ffff88003b4ffc48 ffff88003d328000 
ffff88003b4ffc28
[  241.080800]  ffff88003b500000 ffff88003db978f4 ffff88003d328000 
00000000ffffffff
[  241.095124]  ffff88003db978f8 ffff88003b4ffc58 ffffffff81af0812 
ffff88003db978f0
[  241.108802] Call Trace:
[  241.113716]  [<ffffffff81af0812>] schedule+0x32/0x80
[  241.120665]  [<ffffffff81af0b19>] schedule_preempt_disabled+0x9/0x10
[  241.128191]  [<ffffffff81af1ed5>] __mutex_lock_slowpath+0x95/0x110
[  241.137652]  [<ffffffff81af1f6e>] mutex_lock+0x1e/0x40
[  241.144911]  [<ffffffff8177ea3f>] product_show+0x1f/0x50
[  241.153177]  [<ffffffff816888fb>] dev_attr_show+0x1b/0x50
[  241.161507]  [<ffffffff81af1f61>] ? mutex_lock+0x11/0x40
[  241.168354]  [<ffffffff8120067b>] sysfs_kf_seq_show+0xbb/0x150
[  241.180683]  [<ffffffff811fef2b>] kernfs_seq_show+0x1b/0x20
[  241.190340]  [<ffffffff811b0c6d>] seq_read+0xcd/0x3b0
[  241.200103]  [<ffffffff811ffa55>] kernfs_fop_read+0x105/0x170
[  241.209954]  [<ffffffff8119e9a5>] ? do_filp_open+0x35/0xb0
[  241.220031]  [<ffffffff8118e5c3>] __vfs_read+0x23/0xd0
[  241.230325]  [<ffffffff811ab008>] ? __fd_install+0x48/0x60
[  241.238748]  [<ffffffff8118edc5>] vfs_read+0x85/0x140
[  241.245479]  [<ffffffff8118fb65>] SyS_read+0x45/0xc0
[  241.253530]  [<ffffffff81af40ae>] system_call_fastpath+0x12/0x71


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.