Re: Keystone Issue

[Julien Grall <julien@xxxxxxx>]

Hi,

On 04/06/2020 13:07, CodeWiz2280 wrote:
> On Thu, Jun 4, 2020 at 6:16 AM Julien Grall <julien@xxxxxxx> wrote:
> > Hi,
> >
> > On 04/06/2020 10:08, Bertrand Marquis wrote:
> > > I would have thought that linux would have need some memory, even small in the 
> > > 32bit space in order to boot.
> > Yes it needs some, but then they are switching to use the high memory
> > alias after the MMU has been switch on.
> >
> >   From my understanding, the only difference is the page-tables will
> > point to the high memory alias address rather than the low memory one.
> > Linux will still be located at the same place but now accessed from the
> > high memory alias rather than the low one.
> >
> > Note that AFAICT the secondary CPUs will still be brought-up using the
> > low memory alias.
> >
> > > I could understand that some memory in the low address space needs to be 
> > > reserved by Linux as DMA area for peripherals not supporting 36-bit addresses, 
> > > but the whole low memory sounds like a big restriction.
> > Many platforms have devices only supporting 32-bit DMA, but none of them
> > require such aliasing. So this doesn't look to be the issue here.
> >
> > TBH, this code is only used by Keystone and switching address space is
> > expensive (you have to turn off the MMU, updates page-tables, flush the
> > cache...). I find hard to believe a developper would have come up with
> > this complexity if it were possible to always use the low memory address
> > range. It is even harder to believe Linux community would have accepted it.
> >
> > > Would it be possible to have a bit more information on the “problem with 
> > > peripherals” here ?
> > I am curious as well, so I looked in more depth :). Going through the
> > Linux history, one of the commit message [1] suggests they are switching
> > to a coherent address space. The datasheet [2] (page 75) also confirm
> > that the low region is not IO coherent.
> >
> > So I think you would not be able to do DMA without flush the cache which
> > can be pretty expensive. For a PoC, it might be possible to force Linux
> > flushing the area before and after each DMA request. This should be
> > possible by marking the devices as not coherent.
> >
> > Although, I am not entirely sure if there is any fallout.
> >
> > @Dave, do you think it is possible for you to have a try? I can provide
> > the patch for Linux to disable DMA coherency if possible.
> I attempted to do that, where I removed the "dma-coherent" flags from
> the device tree.  There are likely other issues, but the most glaring
> problem that I ran into is that the ethernet does not work.  Eth0
> shows up in ifconfig but there is no activity on it after a small
> handful of message exchanges, whereas booting without Xen it seems to
> work fine even if left in 32-bit mode (with the dma-coherent
> disabled).  I don't know what implications behind the scenes there are
> trying to stay in the lower 0x8000_0000 alias range either though. 
Thank you for the answer. As wrote, Linux is working fine in 32-bit mode 
when dma-coherent is left in 32-bit mode. So this suggest a different 
issue on the platform.
Given that you receive an handful of packet and then nothing, this would 
lead to maybe an interrupt problem. Can you check whether the number of 
interrupts increments the same way on baremetal and on Xen?
Dumping /proc/interrupts should be sufficient.

> I
> would rather run it as intended by switching to the upper
> 0x8_0000_0000 alias region.
I agree this would be ideal :).

Cheers,

--
Julien Grall