Re: [PATCH for-4.14 0/3] Remove the 1GB limitation on Rasberry Pi 4

[Roman Shaposhnik <roman@xxxxxxxxxx>]

On Tue, May 19, 2020, 7:15 PM Tamas K Lengyel <tamas.k.lengyel@xxxxxxxxx> wrote:

On Tue, May 19, 2020 at 5:50 PM Roman Shaposhnik <roman@xxxxxxxxxx> wrote:
>
> On Tue, May 19, 2020 at 4:44 PM Tamas K Lengyel
> <tamas.k.lengyel@xxxxxxxxx> wrote:
> >
> > On Tue, May 19, 2020 at 11:23 AM Julien Grall <julien@xxxxxxx> wrote:
> > >
> > >
> > >
> > > On 19/05/2020 04:08, Tamas K Lengyel wrote:
> > > > On Mon, May 18, 2020 at 5:32 AM Julien Grall <julien@xxxxxxx> wrote:
> > > >>
> > > >> From: Julien Grall <jgrall@xxxxxxxxxx>
> > > >>
> > > >> Hi all,
> > > >>
> > > >> At the moment, a user who wants to boot Xen on the Raspberry Pi 4 can
> > > >> only use the first GB of memory.
> > > >>
> > > >> This is because several devices cannot DMA above 1GB but Xen doesn't
> > > >> necessarily allocate memory for Dom0 below 1GB.
> > > >>
> > > >> This small series is trying to address the problem by allowing a
> > > >> platform to restrict where Dom0 banks are allocated.
> > > >>
> > > >> This is also a candidate for Xen 4.14. Without it, a user will not be
> > > >> able to use all the RAM on the Raspberry Pi 4.
> > > >>
> > > >> This series has only be slighlty tested. I would appreciate more test on
> > > >> the Rasbperry Pi 4 to confirm this removing the restriction.
> > > >
> > > > Hi Julien,
> > >
> > > Hi,
> > >
> > > > could you post a git branch somewhere? I can try this on my rpi4 that
> > > > already runs 4.13.
> > >
> > > I have pushed a branch based on unstable and the v2 of the series:
> > >
> > > git://xenbits.xen.org/people/julieng/xen-unstable.git
> > >
> > > branch arm-dma/v2
> > >
> >
> > I've updated my image I built with
> > https://github.com/tklengyel/xen-rpi4-builder a while ago and I've
> > defined 2048m as total_mem and Xen seems to be booting fine and passes
> > execution to dom0. With 512m being set as the Xen cmdline for dom0_mem
> > it was working. When I increased the mem for dom0 the boot is now
> > stuck at:
> >
> > [    1.427788] of_cfs_init
> > [    1.429667] of_cfs_init: OK
> > [    1.432561] clk: Not disabling unused clocks
> > [    1.437239] Waiting for root device /dev/mmcblk0p2...
> > [    1.451599] mmc1: queuing unknown CIS tuple 0x80 (2 bytes)
> > [    1.458156] mmc1: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.464729] mmc1: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.472804] mmc1: queuing unknown CIS tuple 0x80 (7 bytes)
> > [    1.479370] mmc1: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.546902] random: fast init done
> > [    1.564590] mmc1: new high speed SDIO card at address 0001
> >
> > Could this be because the DTB I compiled from a fresh checkout of
> > https://github.com/raspberrypi/linux.git branch rpi-4.19.y whereas the
> > kernel itself is from a checkout ~5 months ago? I guess that must be
> > the cause because even if I decrease the dom0_mem to 512m it still
> > gets stuck at the same spot whereas it was booting fine before.
>
> Stefano and I are testing the fix right now -- for now just set your
> Dom0 mem to less than 512m.

Actually seems to work after I recompiled the kernel and reinstalled
all kernel modules. Xen boots with 4gb RAM and dom0 boots with 2g:

xl info:
...
total_memory           : 3956
free_memory            : 1842

cat /proc/meminfo
MemTotal:        1963844 kB

I get an emergency shell during boot on the console complaining about
xenbr0 not coming up but if I just hit continue it boots fine and the
network is up. So AFAICT things are good.

What exact version of the kernel are you using and what did you build it from?

FWIW: 5.6.x clearly has an issue with DMA.

Thanks,

Roman.