|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: Xen data from meta-virtualization layer
Hello Stefano,
> On 17 Nov 2020, at 11:53 pm, Stefano Stabellini
> <stefano.stabellini@xxxxxxxxxx> wrote:
>
> Adding Bertrand, Oleksandr, Julien, and others -- they have a more
> recent experience with GICv3 ITS than me and might be able to help.
> I am attaching the device tree Leo sent a few days ago for reference.
>
>
> Typically when you can set the ethernet link up and no packets are
> exchanged it is because of a missing interrupt. In this case a missing
> MSI.
>
> Bertrand, I believe you tried the GIC ITS driver with PCI devices
> recently. It is expected to work correctly with MSIs in Dom0, right?
>
Yes we are using the XEN GIC ITS driver and MSI interrupts is working fine in
DOM0.
20: 112 0 0 0 ITS-MSI 1572864 Edge eth0
21: 441 0 0 0 ITS-MSI 3670016 Edge
eth1
22: 4286 0 0 0 ITS-MSI 4194304 Edge
xhci_hcd
Regards,
Rahul
>
> On Tue, 17 Nov 2020, Leo Krueger wrote:
>> Hi,
>>
>> I enabled CONFIG_HAS_ITS (what a stupid mistake by me to not set it
>> before...) but then had to add the following node to my device tree
>>
>> gic_lpi_base: syscon@0x80000000 {
>> compatible = "gic-lpi-base";
>> reg = <0x0 0x80000000 0x0 0x100000>;
>> max-gic-redistributors = <2>;
>> };
>>
>> to somehow change something in regard to the ITS and MSI/MSI-X
>>
>> (XEN) GICv3 initialization:
>> (XEN) gic_dist_addr=0x00000006000000
>> (XEN) gic_maintenance_irq=25
>> (XEN) gic_rdist_stride=0
>> (XEN) gic_rdist_regions=1
>> (XEN) redistributor regions:
>> (XEN) - region 0: 0x00000006040000 - 0x00000006080000
>> (XEN) GICv3: using at most 57344 LPIs on the host.
>> (XEN) GICv3: 288 lines, (IID 0001143b).
>> (XEN) GICv3: Found ITS @0x6020000
>> (XEN) using non-cacheable ITS command queue
>> (XEN) GICv3: CPU0: Found redistributor in region 0 @000000004001c000
>>
>> [ 0.000000] GICv3: Distributor has no Range Selector support
>> [ 0.000000] GICv3: no VLPI support, no direct LPI support
>> [ 0.000000] ITS [mem 0x06020000-0x0603ffff]
>> [ 0.000000] ITS@0x0000000006020000: allocated 65536 Devices @dc880000
>> (flat, esz 8, psz 64K, shr 1)
>> [ 0.000000] ITS@0x0000000006020000: allocated 32768 Interrupt Collections
>> @dc820000 (flat, esz 2, psz 64K, shr 1)
>> [ 0.000000] GIC: using LPI property table @0x00000000dc830000
>> [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x0000000006040000
>> [ 0.000000] CPU0: using LPI pending table @0x00000000dc840000
>> ...
>> [ 0.040080] Platform MSI: gic-its domain created
>> [ 0.040136] PCI/MSI: /interrupt-controller/gic-its domain created
>> [ 0.040181] fsl-mc MSI: /interrupt-controller/gic-its domain created
>>
>>
>> Still I am ending up with the " Failed to add - passthrough or MSI/MSI-X
>> might fail!" log messages for some PCI devices, but at least the on-board
>> ethernet ports (fsl_enetc ) are initialized.
>> I can set the link up and a link is successfully established.
>>
>> But (!) I cannot receive or transmit anything (no error message...) and
>> there seem to be no interrupts:
>>
>> 29: 0 ITS-MSI 1 Edge gbe0-rxtx0
>> 32: 0 ITS-MSI 8192 Edge ptp_qoriq
>>
>> (from /proc/interrupts).
>>
>> Any idea on this one? I keep digging and checking whether my device tree
>> needs some additional fixes.
>>
>> Kind regards,
>> Leo
>>
>> --
>> Leo Krüger, M.Sc.
>> Senior Systems Engineer Distributed Systems
>> Intelligent Digital Cabin
>>
>> ZAL Zentrum für Angewandte Luftfahrtforschung GmbH
>> Hein-Saß-Weg 22
>> 21129 Hamburg
>>
>> +49 (0) 40 248 595-154
>>
>> zal.aero | twitter.com/ZALTechCenter | facebook.com/ZALTechCenter
>>
>> ZAL Zentrum für Angewandte Luftfahrtforschung GmbH
>> Sitz der Gesellschaft / Legal Domicile: Hamburg
>> Registergericht / Registration Court: Amtsgericht Hamburg HRB 110232
>> Vorsitzender des Aufsichtsrates / Chairman of the Supervisory Board: StR
>> Andreas Rieckhof
>> Geschäftsführung / Board of Management: Roland Gerhards
>>
>> Disclaimer:
>> This e-mail may contain confidential and/or privileged information. If you
>> are not the intended recipient (or have
>> received this mail in error), please notify the sender immediately and
>> destroy this e-mail. Any unauthorised copying,
>> disclosure or distribution of the material in this e-mail is strictly
>> forbidden.
>>
>>> -----Ursprüngliche Nachricht-----
>>> Von: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>
>>> Gesendet: Dienstag, 17. November 2020 01:59
>>> An: Leo Krueger <leo.krueger@xxxxxxxx>
>>> Cc: Peng Fan <peng.fan@xxxxxxx>; Stefano Stabellini
>>> <stefano.stabellini@xxxxxxxxxx>; brucea@xxxxxxxxxx; Cornelia Bruelhart
>>> <cornelia.bruelhart@xxxxxxxx>
>>> Betreff: Re: AW: AW: AW: Xen data from meta-virtualization layer
>>>
>>> Replies inline below
>>>
>>>
>>> On Sun, 15 Nov 2020, Leo Krueger wrote:
>>>> Hi Peng, hi Stefano,
>>>>
>>>>
>>>>
>>>> sorry for the long silence…
>>>>
>>>>
>>>>
>>>> I tried the change suggested (and hope I didn’t do anything wrong
>>>> while doing so…) on top of XEN 4.13.2 (before, I always tried with
>>>> 4.12 but wanted to give 4.13.2 a try as well) but I do not see any
>>>> difference,
>>> still the same “unhandled context fault” log entries pop up and I cannot
>>> access my sdcard.
>>>>
>>>>
>>>>
>>>> As it seems to work without respectively disabled iommu, that would be
>>>> fine for me for now. What I am worried about a bit more is PCIe or
>>> MSI/MSIX to be exact.
>>>>
>>>>
>>>>
>>>> Here is the gic-v3 and its node from my device tree:
>>>>
>>>>
>>>>
>>>> interrupt-controller@6000000 {
>>>>
>>>> compatible = "arm,gic-v3";
>>>>
>>>> #address-cells = <0x2>;
>>>>
>>>> #size-cells = <0x2>;
>>>>
>>>> ranges;
>>>>
>>>> reg = <0x0 0x6000000 0x0 0x10000 0x0 0x6040000 0x0 0x40000>;
>>>>
>>>> #interrupt-cells = <0x3>;
>>>>
>>>> interrupt-controller;
>>>>
>>>> interrupts = <0x1 0x9 0xf08>;
>>>>
>>>> phandle = <0x1>;
>>>>
>>>>
>>>>
>>>> gic-its@6020000 {
>>>>
>>>> compatible = "arm,gic-v3-its";
>>>>
>>>> msi-controller;
>>>>
>>>> reg = <0x0 0x6020000 0x0 0x20000>;
>>>>
>>>> phandle = <0xd>;
>>>>
>>>> };
>>>>
>>>> };
>>>>
>>>>
>>>>
>>>> And here are some kernel log excerpts related to GIC when booting
>>> without (!) XEN:
>>>>
>>>>
>>>>
>>>> [ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode
>>>>
>>>> [ 0.000000] GICv3: Distributor has no Range Selector support
>>>>
>>>> [ 0.000000] GICv3: no VLPI support, no direct LPI support
>>>>
>>>> [ 0.000000] ITS [mem 0x06020000-0x0603ffff]
>>>>
>>>> [ 0.000000] ITS@0x0000000006020000: allocated 65536 Devices
>>>> @20fb880000 (flat, esz 8, psz 64K, shr 0)
>>>>
>>>> [ 0.000000] ITS: using cache flushing for cmd queue
>>>>
>>>> [ 0.000000] GIC: using LPI property table @0x00000020fb830000
>>>>
>>>> [ 0.000000] GICv3: CPU0: found redistributor 0 region
>>>> 0:0x0000000006040000
>>>>
>>>> [ 0.000000] CPU0: using LPI pending table @0x00000020fb840000
>>>>
>>>> [ 0.000000] GIC: using cache flushing for LPI property table
>>>>
>>>>
>>>>
>>>> However, when booting with XEN, only the following three lines are
>>> contained in the kernel log:
>>>>
>>>>
>>>>
>>>> [ 0.000000] GICv3: Distributor has no Range Selector support
>>>>
>>>> [ 0.000000] GICv3: no VLPI support, no direct LPI support
>>>>
>>>> [ 0.000000] GICv3: CPU0: found redistributor 0 region
>>>> 0:0x0000000006040000
>>>
>>> "no VLPI support" is very suspicious, it looks like Dom0 doesn't find any
>>> ITS
>>> support.
>>>
>>> Can you double check that you have the ITS driver in Xen built-in? That
>>> would
>>> be CONFIG_HAS_ITS. If you do "make menuconfig" and enable "Configure
>>> standard Xen features (expert users)" you should get a new option "GICv3
>>> ITS MSI controller support" under "Architecture Features".
>>> Make sure to enable it.
>>>
>>> Let me know if that works!
> <devicetree.dts>
Regards,
Rahul
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |