Xen project Mailing List

On Fri, Jul 15, 2016 at 11:45 PM, Andrey Grodzovsky <andrey2805@xxxxxxxxx> wrote:

On Fri, Jul 15, 2016 at 6:04 AM, Andrew Cooper <andrew.cooper3@xxxxxxxxxx> wrote:

On 12/07/16 04:59, Andrey Grodzovsky wrote:

Hello

Some background -

We are trying to run Qualcomm Atheros AR928X Wireless Network Adapter and have a crash right on driver load, following are our observations and questions.

Jurgen's observation -

" The Atheros card "Qualcomm Atheros AR928X Wireless Network Adapter (PCI-Express) (rev 01)" is plugged into the host system (datatron).
When I attach it to the DomU - the module "ath9k" is automatically loaded, but it gives an exception "iowrite32+0x2b/0x30".

No idea what the issue is (tried also with another Atheros Card (ath10k) - similar problem). When I try an Intel card, it works.
(the card also works on the Dom0 - so the Linux driver and HW is OK)."

Debugging -

After some investigation with kgdb and iommu trace on DomU it seems the iomap of PCI BAR for the device returns a a mapping f which first 0x1000 bytes are read only and that causes access violation when trying to write registers mapped to this area (all the regs with offset < 0x1000) - why this happens i still don't know. Register writes with offsets > 0x1000 are fine.

Running same driver on Dom0 is totally fine

Bellow the sigsev backtrace and xen dmesg from DomU

As can be seen there is ioremap_ of size 0x10000 starting at ffffc900402c0000 but as i said, i noticed that anything bellow ffffc900402c1000 is not writable (from gdb using set addr = val) and only readable while anything above this address is writeable.

Question -

Please give any advise on this issue and especially how to approach debugging this both on Domu and Dom0 and where in xen code to look for possible issues.

First of all, is this a PV or HVM domU ?

Is this BAR the same BAR which has the MSI-X table in? For safety, Xen has to trap and emulate updates to the MSI/MSI-X configuration. It is possible that that logic has gone wrong.

~Andrew

As much as I understand from looking at lspci -vvv for this device on the guest (after attaching and SIGSEV) It is on the same (and and only) BAR but MSI/MS-X is not enabled ? (the "-" next to the property)
Please take a look at the dump.

00:00.0 Network controller: Qualcomm Atheros AR928X Wireless Network Adapter (PCI-Express) (rev 01)
Subsystem: Qualcomm Atheros AR928X Wireless Network Adapter (PCI-Express)
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 25
Region 0: Memory at f7b00000 (64-bit, non-prefetchable) [size=64K]
Capabilities: [40] Power Management version 2
Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit-
Address: 00000000 Data: 0000
Capabilities: [60] Express (v1) Legacy Endpoint, MSI 00
DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L1, Exit Latency L0s <512ns, L1 <64us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 128 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
Capabilities: [90] MSI-X: Enable- Count=1 Masked-
Vector table: BAR=0 offset=00000000
PBA: BAR=0 offset=00000000
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Capabilities: [140 v1] Virtual Channel
Caps: LPEVC=0 RefClk=100ns PATEntryBits=1
Arb: Fixed- WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=01
Status: NegoPending- InProgress-
Capabilities: [160 v1] Device Serial Number 00-00-00-00-00-00-00-00
Kernel driver in use: ath9k
Kernel modules: ath9k
00: 8c 16 2a 00 07 01 10 00 01 00 80 02 08 00 00 00
10: 04 00 b0 f7 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 8c 16 99 30
30: 00 00 00 00 40 00 00 00 00 00 00 00 10 01 00 00
40: 01 50 c2 03 00 00 00 00 00 00 00 00 00 00 00 00
50: 05 60 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 10 90 11 00 c0 0c 90 05 00 20 01 00 11 38 03 00
70: 48 00 11 10 00 00 00 00 c0 03 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 11 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

I created a full xenalyze trace from the time i run the command to attach the device to after the kernel oops.

I hope it can give clues to the root cause.

Re: [Xen-devel] Atheros WiFi - memory paging failure on driver load