Re: Xen 4.14.1 on RPI4: device tree generation failed

[Roman Shaposhnik <roman@xxxxxxxxxx>]

On Mon, Feb 1, 2021 at 5:40 PM Stefano Stabellini
<sstabellini@xxxxxxxxxx> wrote:
>
> On Mon, 1 Feb 2021, Tamas K Lengyel wrote:
> > On Mon, Feb 1, 2021 at 10:23 AM Tamas K Lengyel
> > <tamas.k.lengyel@xxxxxxxxx> wrote:
> > >
> > > On Mon, Feb 1, 2021 at 12:54 AM Elliott Mitchell <ehem+xen@xxxxxxx> wrote:
> > > >
> > > > On Sun, Jan 31, 2021 at 10:06:21PM -0500, Tamas K Lengyel wrote:
> > > > > With rpi-4.19.y kernel and dtbs
> > > > > (cc39f1c9f82f6fe5a437836811d906c709e0661c) Xen boots fine and the
> > > > > previous error is not present. I get the boot log on the serial with
> > > > > just console=hvc0 from dom0 but the kernel ends up in a panic down the
> > > > > line:
> > > >
> > > > > This seems to have been caused by a monitor being attached to the HDMI
> > > > > port, with HDMI unplugged dom0 boots OK.
> > > >
> > > > The balance of reports seem to suggest 5.10 is the way to go if you want
> > > > graphics on a RP4 with Xen.  Even without Xen 4.19 is looking rickety on
> > > > RP4.
> > > >
> > > >
> > > > On Sun, Jan 31, 2021 at 09:43:13PM -0500, Tamas K Lengyel wrote:
> > > > > On Sun, Jan 31, 2021 at 8:59 PM Elliott Mitchell <ehem+xen@xxxxxxx> 
> > > > > wrote:
> > > > > >
> > > > > > On Sun, Jan 31, 2021 at 06:50:36PM -0500, Tamas K Lengyel wrote:
> > > > > > > On Sun, Jan 31, 2021 at 6:33 PM Elliott Mitchell 
> > > > > > > <ehem+undef@xxxxxxx> wrote:
> > > > > > > > Presently the rpixen script is grabbing the RPF's 4.19 branch, 
> > > > > > > > dates
> > > > > > > > point to that last being touched last year.  Their tree is at
> > > > > > > > cc39f1c9f82f6fe5a437836811d906c709e0661c.
> > > > > > >
> > > > > > > I've moved the Linux branch up to 5.10 because there had been a 
> > > > > > > fair
> > > > > > > amount of work that went into fixing Xen on RPI4, which got merged
> > > > > > > into 5.9 and I would like to be able to build upstream everything
> > > > > > > without the custom patches coming with the rpixen script repo.
> > > > > >
> > > > > > Please keep track of where your kernel source is checked out at 
> > > > > > since
> > > > > > there was a desire to figure out what was going on with the 
> > > > > > device-trees.
> > > > > >
> > > > > >
> > > > > > Including "console=hvc0 console=AMA0 console=ttyS0 console=tty0" in 
> > > > > > the
> > > > > > kernel command-line should ensure you get output from the kernel if 
> > > > > > it
> > > > > > manages to start (yes, Linux does support having multiple consoles 
> > > > > > at the
> > > > > > same time).
> > > > >
> > > > > No output from dom0 received even with the added console options
> > > > > (+earlyprintk=xen). The kernel build was from rpi-5.10.y
> > > > > c9226080e513181ffb3909a905e9c23b8a6e8f62. I'll check if it still boots
> > > > > with 4.19 next.
> > > >
> > > > So, their current HEAD.  This reads like you've got a problematic kernel
> > > > configuration.  What procedure are you following to generate the
> > > > configuration you use?
> > > >
> > > > Using their upstream as a base and then adding the configuration options
> > > > for Xen has worked fairly well for me (`make bcm2711_defconfig`,
> > > > `make menuconfig`, `make zImage`).
> > > >
> > > > Notably the options:
> > > > CONFIG_PARAVIRT
> > > > CONFIG_XEN_DOM0
> > > > CONFIG_XEN
> > > > CONFIG_XEN_BLKDEV_BACKEND
> > > > CONFIG_XEN_NETDEV_BACKEND
> > > > CONFIG_HVC_XEN
> > > > CONFIG_HVC_XEN_FRONTEND
> > > >
> > > > Should be set to "y".
> > >
> > > Yes, these configs are all set the same way for all Linux builds by the 
> > > script:
> > >         make O=.build-arm64 ARCH=arm64
> > > CROSS_COMPILE=aarch64-none-linux-gnu- bcm2711_defconfig xen.config
> > >
> > > I tried with both the rpi-5.10.y and rpi-5.9.y, neither boot up as
> > > dom0. So far only 4.19 boots.
> >
> > rpi-5.4.y boots but ends up in yet another different kernel panic:
>
> That's an interesting error. However, I can tell you that I can boot
> rpi-5.9.y just fine (without a monitor attached) with:
>
> cd linux
> KERNEL=kernel7l
> make bcm2711_defconfig
>
> As mentioned here:
>
> https://www.raspberrypi.org/documentation/linux/kernel/building.md
>
> and also taking the device tree from arch/arm64/boot/dts/broadcom/.

FWIW: I see the same results with stock upstream 5.10.7 effectively
following the steps you're doing.

However, as I keep saying -- the combination of firmware and u-boot
(in my case) is a very sensitive combination -- hence we're relying
on a very particular set of bits for there in EVE and will refuse to work
with anything else.

It may be helpful to take that combination outside of EVE's context and
try it out in your experiments Tamas.

Thanks,
Roman.

P.S. I'm running into a DomU issue in certain places when running with 5.10.7
but that's a subject for a different thread.