Re: Troubles running Xen on Raspberry Pi 4 with 5.6.1 DomU

[Julien Grall <julien@xxxxxxx>]

On 13/05/2020 19:19, Julien Grall wrote:
> Hi,
>
> On 13/05/2020 16:11, Stefano Stabellini wrote:
> > On Wed, 13 May 2020, Julien Grall wrote:
> > > Hi,
> > >
> > > On 13/05/2020 01:33, Stefano Stabellini wrote:
> > > > I worked with Roman to do several more tests and here is an update on
> > > > the situation. We don't know why my patch didn't work when Boris' patch
> > > > [1] worked.  Both of them should have worked the same way.
> > > >
> > > > Anyway, we continued with Boris patch to debug the new mmc error which
> > > > looks like this:
> > > >
> > > > [    3.084464] mmc0: unrecognised SCR structure version 15
> > > > [    3.089176] mmc0: error -22 whilst initialising SD card
> > > >
> > > > I asked to add a lot of trancing, see attached diff.
> > > Please avoid attachment on mailing list and use pastebin for diff.
> > >
> > > > We discovered a bug
> > > > in xen_swiotlb_init: if io_tlb_start != 0 at the beginning of
> > > > xen_swiotlb_init, start_dma_addr is not set correctly. This oneline
> > > > patch fixes it:
> > > >
> > > > diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
> > > > index 0a40ac332a4c..b75c43356eba 100644
> > > > --- a/drivers/xen/swiotlb-xen.c
> > > > +++ b/drivers/xen/swiotlb-xen.c
> > > > @@ -191,6 +191,7 @@ int __ref xen_swiotlb_init(int verbose, bool early)
> > > >            * IO TLB memory already allocated. Just use it.
> > > >            */
> > > >           if (io_tlb_start != 0) {
> > > > +               start_dma_addr = io_tlb_start;
> > > >                   xen_io_tlb_start = phys_to_virt(io_tlb_start);
> > > >                   goto end;
> > > >           }
> > > >
> > > > Unfortunately it doesn't solve the mmc0 error.
> > > >
> > > >
> > > > As you might notice from the logs, none of the other interesting 
> > > > printks
> > > > printed anything, which seems to mean that the memory allocated by
> > > > xen_swiotlb_alloc_coherent and mapped by xen_swiotlb_map_page should be
> > > > just fine.
> > > >
> > > > I am starting to be out of ideas. Do you guys have any suggestions on
> > > > what could be the issue or what could be done to discover it?
> > > Sorry if my suggestions are going to be obvious, but I can't confirm 
> > > whether
> > > this was already done:
> > >      1) Does the kernel boot on baremetal (i.e without Xen)? This 
> > > should help
> > > to confirm whether the bug is Xen is related.
> > Yes it boots
> >
> > >      2) Swiotlb should not be necessary for basic dom0 boot on Arm. 
> > > Did you try
> > > to disable it? This should help to confirm whether swiotlb is the 
> > > problem or
> > > not.
> > It boots disabling swiotlb-xen
> Thank you for the answer! swiotlb-xen should basically be a NOP for 
> dom0. So this suggests swiotlb is doing some transformation on the DMA 
> address.
> I have an idea what may have gone wrong. AFAICT, xen-swiotlb seems to 
> assume the DMA address space and CPU address space is the same. Is RPI 
> using the same address space?
Another question, is the DMA request bounced? If so, are we sure the 
bounce buffer is in the first GB?
Cheers,

--
Julien Grall