[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] REGRESSION: Xen 4.13 RC5 fails to bootstrap Dom0 on ARM


On 18/12/2019 17:09, Roman Shaposhnik wrote:

On Wed, Dec 18, 2019 at 4:56 AM Julien Grall <julien@xxxxxxx> wrote:
So that is, in fact, my first question -- why is Xen not showing
available memory in xl info?

I am not entirely sure what exact information you want.

The output you dumped above contain the available memory for the memory
(see "free_memory").

Are you looking from something different?

Just to be clear: I was giving 2G via devicetrees (the same device
trees that would
make Linux detect 2G of RAM) hence I was expecting xl info to show that. Instead
I only got 1120M shown by xl info.

On 18/12/2019 00:04, Roman Shaposhnik wrote:
          memory {
                  device_type = "memory";
                  reg = <0x0 0x0 0x0 0x5e00000 0x0 0x5f00000 0x0 0x1000
0x0 0x5f02000 0x0 0xefd000 0x0 0x6e00000 0x0 0x60f000 0x0 0x7410000
0x0 0x1aaf0000 0x0 0x21f00000 0x0 0x100000 0x0 0x22000000 0x0

          reserved-memory {
                  #size-cells = <0x2>;
                  #address-cells = <0x2>;

                  ramoops@21f00000 {
                          ftrace-size = <0x20000>;
                          console-size = <0x20000>;
                          reg = <0x0 0x21f00000 0x0 0x100000>;
                          record-size = <0x20000>;
                          compatible = "ramoops";

                  linux,cma {
                          size = <0x0 0x8000000>;
                          compatible = "shared-dma-pool";

If you look at the REG -- it does now add up to 2Gb, but booting Xen
with it has exactly the
same effect as booting it with: reg = <0x0 0x0 0x0 0x80000000>;\

If you boot Xen using EFI, the memory information wil come from EFI and
the DT node will be ignored. So unless UEFI is able to pick up the
modification of the DT memory node, modifying the DT is not going to
affect anything.

That's a good point, but given that I always go through GRUB, I was
expecting devicetree command to completely overshadow whatever
information UEFI may have. Am I wrong?

GRUB will load Xen/Linux as an EFI application. Both of them will ignore the memory nodes when booting using EFI. For more details, see the answer I wrote separately.

I am attaching a full log, and I see the following in the logs:

(XEN) Allocating 1:1 mappings totalling 720MB for dom0:
(XEN) BANK[0] 0x00000008000000-0x0000001c000000 (320MB)
(XEN) BANK[1] 0x00000040000000-0x00000058000000 (384MB)
(XEN) BANK[2] 0x0000007b000000-0x0000007c000000 (16MB)

Which sort of makes sense, I guess -- but I still don't understand
where all these ranges
are coming from and how come Xen doesn't see the full 2Gb even with various
devicetrees I tried.

The range aboves describe the memory range given to Dom0. For all the
memory given to Xen,m you want to look at the top of your log:

(XEN) Checking for initrd in /chosen
(XEN) RAM: 0000000000000000 - 0000000005dfffff
(XEN) RAM: 0000000005f00000 - 0000000006dfefff
(XEN) RAM: 0000000006e00000 - 000000000740efff
(XEN) RAM: 0000000007410000 - 000000001db8dfff
(XEN) RAM: 00000000350f0000 - 000000003dbd2fff
(XEN) RAM: 000000003dbd3000 - 000000003dffffff
(XEN) RAM: 0000000040000000 - 000000005a653fff
(XEN) RAM: 000000007ada0000 - 000000007ada3fff
(XEN) RAM: 000000007aea8000 - 000000007afa9fff
(XEN) RAM: 000000007afaa000 - 000000007ec73fff
(XEN) RAM: 000000007ec74000 - 000000007fdddfff
(XEN) RAM: 000000007fdde000 - 000000007fea5fff
(XEN) RAM: 000000007fea6000 - 000000007ff6dfff
(XEN) RAM: 000000007ffff000 - 000000007fffffff

Looking at the differences with the Linux logs, there is indeed some
memory not detected by Xen.

On Xen, we only consider usuable memory any EFI description with
EfiConventionalMemory, EfiBootServicesCode and EfiBootServicesData.

Linux include more type here, so this may explain why we see a difference.

While Looking at it, I have also noticed that we don't seem to care
about the memory attribute. I suspect this could be another latent issue
in Xen if the attribute does not match.

Anything I can do to help debug this? I can run any kind of debug builds, etc.
if needed.

Thank you for the offer, I think I have a good understanding of the problem now. So debug should not be necessary.

However, I would appreciate if anyone could help to write a patch for it.

I mean -- at this point it would be really great to get HiKey back to the status
of Xen-on-ARM developer board.

Any ideas here would be greatly apprecaited!


P.S. Any guess at what these mean?

(XEN) traps.c:1973:d0v0 HSR=0x93880006 pc=0x00ffff87355558
gva=0xffff872f2000 gpa=0x000000000f0000
(XEN) traps.c:1973:d0v0 HSR=0x93880006 pc=0x00ffffb734e558
gva=0xffffb72eb000 gpa=0x000000000f0000
(XEN) traps.c:1973:d0v0 HSR=0x93880006 pc=0x00ffff8f9d2558
gva=0xffff8f96f000 gpa=0x000000000f0000

It means that Linux has tried to access something that has not been
mapped in stage-2. As Dom0 is mapped 1:1, the GPA also give you the host
physical address. In this case, it is trying to access 0xf0000.

This seems to belong to the RAM, but this part has not been allocated to

Got it! Thank you! Am I correct in guessing that this can only come from
a driver of some sorts trying to tickle the hardware? IOW, I should be
looking for some abnormalities in my linux kernel messages to try and
see what this could be.

Xen will inject a data abort to the guest. So you should get a stack trace from Linux.


Julien Grall

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.