Re: Troubles running Xen on Raspberry Pi 4 with 5.6.1 DomU

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Wed, 3 Jun 2020, Corey Minyard wrote:
> On Tue, Jun 02, 2020 at 12:24:05PM -0700, Stefano Stabellini wrote:
> > On Tue, 2 Jun 2020, Corey Minyard wrote:
> > > Snip
> > > 
> > > > > > > > whether
> > > > > > > > this was already done:
> > > > > > > >      1) Does the kernel boot on baremetal (i.e without Xen)? 
> > > > > > > > This should
> > > > > > > > help
> > > > > > > > to confirm whether the bug is Xen is related.
> > > > > > > 
> > > > > > > Yes it boots
> > > > > > > 
> > > > > > > >      2) Swiotlb should not be necessary for basic dom0 boot on 
> > > > > > > > Arm. Did
> > > > > > > > you try
> > > > > > > > to disable it? This should help to confirm whether swiotlb is 
> > > > > > > > the
> > > > > > > > problem or
> > > > > > > > not.
> > > > > > > 
> > > > > > > It boots disabling swiotlb-xen
> > > > > > 
> > > > > > Thank you for the answer! swiotlb-xen should basically be a NOP for 
> > > > > > dom0. So
> > > > > > this suggests swiotlb is doing some transformation on the DMA 
> > > > > > address.
> > > > > > 
> > > > > > I have an idea what may have gone wrong. AFAICT, xen-swiotlb seems 
> > > > > > to assume
> > > > > > the DMA address space and CPU address space is the same. Is RPI 
> > > > > > using the
> > > > > > same address space?
> > > > > 
> > > > > Another question, is the DMA request bounced? If so, are we sure the 
> > > > > bounce
> > > > > buffer is in the first GB?
> > > > 
> > > > Yes, it is. This is actually where we spent the last few days, and I
> > > > found another little related bug in the initialization of the
> > > > swiotlb-xen but now I am sure the memory is under 1GB 
> > > > (0x34000000-0x38000000)
> > > 
> > > Was anything ever resolved on this issue?  It just kind of ended for me,
> > > and I looked in the main kernel and didn't find anything that looked
> > > related.
> > 
> > Yes, we have a patch series on the list for the Linux kernel to fix this
> > issue but it hasn't been merged yet:
> > 
> > https://marc.info/?l=linux-kernel&m=159001831406263&w=2
> 
> Just FYI, I pulled the changes on top of
>   https://github.com/raspberrypi/linux.git rpi-5.4.y
> Along with change
>   56e35f9c5b87ec dma-mapping: drop the dev argument to arch_sync_dma_for_*
> before the series so it applies on 5.4, and I was able to boot and
> create a domU.  So:
> 
> Tested-by: Corey Minyard <cminyard@xxxxxxxxxx>
> 
> At least on 5.4.  If you think it would be valuable, I can test on
> rpi-5.7.y.

I'd feel better adding your Tested-by to my next upstream submission of
the series if you could also test on 5.7. Thank you in advance!


> I'll be integrating this in with our Pi Xen yocto layer at
> https://github.com/MontaVista-OpenSourceTechnology/meta-raspberrypi-xen

That's great!