Re: Question on PCIe Device Tree bindings, Was: [PATCH] xen/arm: domain_build: Ignore device nodes with invalid addresses

[Rob Herring <robh@xxxxxxxxxx>]

On Thu, Feb 4, 2021 at 9:51 PM Elliott Mitchell <ehem+undef@xxxxxxx> wrote:
>
> On Thu, Feb 04, 2021 at 03:52:26PM -0600, Rob Herring wrote:
> > On Thu, Feb 4, 2021 at 3:33 PM Stefano Stabellini
> > <sstabellini@xxxxxxxxxx> wrote:
> > >
> > > On Thu, 4 Feb 2021, Rob Herring wrote:
> > > > On Thu, Feb 4, 2021 at 2:36 PM Stefano Stabellini
> > > > <sstabellini@xxxxxxxxxx> wrote:
> > > > >
> > > > > On Thu, 4 Feb 2021, Rob Herring wrote:
> > > > > > On Thu, Feb 4, 2021 at 11:56 AM Stefano Stabellini
> > > > > > <sstabellini@xxxxxxxxxx> wrote:
> > > > > > >
> > > > > > > Hi Rob,
> > > > > > >
> > > > > > > We have a question on the PCIe device tree bindings. In summary, 
> > > > > > > we have
> > > > > > > come across the Raspberry Pi 4 PCIe description below:
> > > > > > >
> > > > > > >
> > > > > > > pcie0: pcie@7d500000 {
> > > > > > >    compatible = "brcm,bcm2711-pcie";
> > > > > > >    reg = <0x0 0x7d500000  0x0 0x9310>;
> > > > > > >    device_type = "pci";
> > > > > > >    #address-cells = <3>;
> > > > > > >    #interrupt-cells = <1>;
> > > > > > >    #size-cells = <2>;
> > > > > > >    interrupts = <GIC_SPI 148 IRQ_TYPE_LEVEL_HIGH>,
> > > > > > >                 <GIC_SPI 148 IRQ_TYPE_LEVEL_HIGH>;
> > > > > > >    interrupt-names = "pcie", "msi";
> > > > > > >    interrupt-map-mask = <0x0 0x0 0x0 0x7>;
> > > > > > >    interrupt-map = <0 0 0 1 &gicv2 GIC_SPI 143
> > > > > > >                                                      
> > > > > > > IRQ_TYPE_LEVEL_HIGH>;
> > > > > > >    msi-controller;
> > > > > > >    msi-parent = <&pcie0>;
> > > > > > >
> > > > > > >    ranges = <0x02000000 0x0 0xc0000000 0x6 0x00000000
> > > > > > >              0x0 0x40000000>;
> > > > > > >    /*
> > > > > > >     * The wrapper around the PCIe block has a bug
> > > > > > >     * preventing it from accessing beyond the first 3GB of
> > > > > > >     * memory.
> > > > > > >     */
> > > > > > >    dma-ranges = <0x02000000 0x0 0x00000000 0x0 0x00000000
> > > > > > >                  0x0 0xc0000000>;
> > > > > > >    brcm,enable-ssc;
> > > > > > >
> > > > > > >    pci@1,0 {
> > > > > > >            #address-cells = <3>;
> > > > > > >            #size-cells = <2>;
> > > > > > >            ranges;
> > > > > > >
> > > > > > >            reg = <0 0 0 0 0>;
> > > > > > >
> > > > > > >            usb@1,0 {
> > > > > > >                    reg = <0x10000 0 0 0 0>;
> > > > > > >                    resets = <&reset 
> > > > > > > RASPBERRYPI_FIRMWARE_RESET_ID_USB>;
> > > > > > >            };
> > > > > > >    };
> > > > > > > };
> > > > > > >
> > > > > > >
> > > > > > > Xen fails to parse it with an error because it tries to remap reg 
> > > > > > > =
> > > > > > > <0x10000 0 0 0 0> as if it was a CPU address and of course it 
> > > > > > > fails.
> > > > > > >
> > > > > > > Reading the device tree description in details, I cannot tell if 
> > > > > > > Xen has
> > > > > > > a bug: the ranges property under pci@1,0 means that pci@1,0 is 
> > > > > > > treated
> > > > > > > like a default bus (not a PCI bus), hence, the children regs are
> > > > > > > translated using the ranges property of the parent 
> > > > > > > (pcie@7d500000).
> > > > > > >
> > > > > > > Is it possible that the device tree is missing device_type =
> > > > > > > "pci" under pci@1,0? Or is it just implied because pci@1,0 is a 
> > > > > > > child of
> > > > > > > pcie@7d500000?
> > > > > >
> > > > > > Indeed, it should have device_type. Linux (only recently due to
> > > > > > another missing device_type case) will also look at node name, but
> > > > > > only 'pcie'.
> > > > > >
> > > > > > We should be able to create (or extend pci-bus.yaml) a schema to 
> > > > > > catch
> > > > > > this case.
> > > > >
> > > > > Ah, that is what I needed to know, thank you!  Is Linux considering a
> > > > > node named "pcie" as if it has device_type = "pci"?
> > > >
> > > > Yes, it was added for Rockchip RK3399 to avoid a DT update and 
> > > > regression.
> > > >
> > > > > In Xen, also to cover the RPi4 case, maybe I could add a check for the
> > > > > node name to be "pci" or "pcie" and if so Xen could assume 
> > > > > device_type =
> > > > > "pci".
> > > >
> > > > I assume this never worked for RPi4 (and Linux will have the same
> > > > issue), so can't we just update the DT in this case?
> > >
> > > I am not sure where the DT is coming from, probably from the RPi4 kernel
> > > trees or firmware. I think it would be good if somebody got in touch to
> > > tell them they have an issue.
> >
> > So you just take whatever downstream RPi invents? Good luck.
> >
> > > Elliot, where was that device tree coming from originally?
>
> Please excuse my very weak device-tree fu...
>
> I'm unsure which section is the problem, but looking at `git blame` what
> shows is commt d5c8dc0d4c880fbde5293cc186b1ab23466254c4.
>
> This commit is present in the Linux master branch and the linux-5.10.y
> branch.
>
> You were saying?

That commit looks perfectly fine. The problem is the PCI bridge node
shown above which doesn't exist upstream.

Rob