|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] REGRESSION: Xen 4.13 RC5 fails to bootstrap Dom0 on ARM
On Tue, Dec 17, 2019 at 5:51 PM Stefano Stabellini <sstabellini@xxxxxxxxxx> wrote: > > On Tue, 17 Dec 2019, Roman Shaposhnik wrote: > > On Tue, Dec 17, 2019 at 11:26 AM Stefano Stabellini > > <sstabellini@xxxxxxxxxx> wrote: > > > > > > On Tue, 17 Dec 2019, Roman Shaposhnik wrote: > > > > On Tue, Dec 17, 2019 at 10:30 AM Stefano Stabellini > > > > <sstabellini@xxxxxxxxxx> wrote: > > > > > > > > > > On Tue, 17 Dec 2019, Julien Grall wrote: > > > > > > Hi, > > > > > > > > > > > > On 17/12/2019 04:39, Roman Shaposhnik wrote: > > > > > > > On Mon, Dec 16, 2019 at 6:55 PM Stefano Stabellini > > > > > > > <sstabellini@xxxxxxxxxx> wrote: > > > > > > > > On Mon, 16 Dec 2019, Roman Shaposhnik wrote: > > > > > > > > If I sum all the memory sizes together I get 0x3ddfd000 which > > > > > > > > is 990M. > > > > > > > > If so, I wonder how you could boot succesfully with > > > > > > > > dom0_mem=1024M even > > > > > > > > on Xen 4.12... :-? > > > > > > > > > > > > > > That is a very interesting observation indeed! I actually don't > > > > > > > remember where that device tree came from, but I think it was > > > > > > > from one > > > > > > > of the Linaro sites. > > > > > > > > > > > > This is mostly likely because of: > > > > > > > > > > > > commit 6341a674573f1834f083f0ab0f5b36b075f9e02e > > > > > > Author: Julien Grall <julien.grall@xxxxxxx> > > > > > > Date: Wed Aug 21 22:42:31 2019 +0100 > > > > > > > > > > > > xen/arm: domain_build: Don't continue if unable to allocate all > > > > > > dom0 banks > > > > > > > > > > > > Xen will only print a warning if there are memory unallocated > > > > > > when using > > > > > > 1:1 mapping (only used by dom0). This also includes the case > > > > > > where no > > > > > > memory has been allocated. > > > > > > > > > > > > It will bring to all sort of issues that can be hard to > > > > > > diagnostic for > > > > > > users (the warning can be difficult to spot or disregard). > > > > > > > > > > > > If the users request 1GB of memory, then most likely they want > > > > > > the exact > > > > > > amount and not 512MB. So panic if all the memory has not been > > > > > > allocated. > > > > > > > > > > > > After this change, the behavior is the same as for non-1:1 > > > > > > memory > > > > > > allocation (used by domU). > > > > > > > > > > > > At the same time, reflow the message to have the format on a > > > > > > single > > > > > > line. > > > > > > > > > > > > Signed-off-by: Julien Grall <julien.grall@xxxxxxx> > > > > > > Acked-by: Stefano Stabellini <sstabellini@xxxxxxxxxx> > > > > > > > > > > Ah! Roman, could you please post the full boot log of a successful > > > > > 4.12 > > > > > boot? > > > > > > > > > > If it has a "Failed to allocate requested dom0 memory" message, then > > > > > we > > > > > know what the issue is. > > > > > > > > Aha! Our messages seems to have crossed ;-) Full log is attached and > > > > yes -- that's > > > > the problem indeed. > > > > > > > > So at least that mystery is solved. But I'm still not able to get to a > > > > full 1G of memory > > > > even with your update to the device tree file. Any chance you can send > > > > me the > > > > device tree file that works for you? > > > > > > I didn't try on real hardware, I only tried on QEMU with a similar > > > configuration. I went back and check the HiKey device tree I used and it > > > is the same as yours (including the ramoops reserved-memory error). > > > > > > Apparently there are 1G and 2G variants of the HiKey, obviously both > > > yours and my device tree are for the 1G variant. I try to dig through > > > the docs but couldn't find the details of the 2G variant. I cannot find > > > anywhere the memory range for the top 1G of memory not even on the > > > LeMaker docs! :-/ > > > > Yup. That's exactly the issue on my end as well - can't seem to find an > > authoritative source for that devicetree. > > > > I did find this, though: > > https://releases.linaro.org/96boards/hikey/linaro/debian/15.11/ > > which looks like it has the latest (at least file timestamp-wise) > > devicetree. > > > > If you look at the memory and reserved memory nodes there, they > > are actually much simpler than what we've got: > > > > memory { > > device_type = "memory"; > > reg = <0x0 0x0 0x0 0x40000000>; > > }; > > Which is still 1G, but it is surprisingly simpler. > > > > reserved-memory { > > #address-cells = <0x2>; > > #size-cells = <0x2>; > > ranges; > > > > mcu-buf@05e00000 { > > no-map; > > reg = <0x0 0x5e00000 0x0 0x100000 0x0 > > 0x740f000 0x0 0x1000>; > > }; > > > > mbox-buf@06dff000 { > > no-map; > > reg = <0x0 0x6dff000 0x0 0x1000>; > > }; > > }; > > > > So -- just on a whim -- I changed it to: > > reg = <0x0 0x0 0x0 0x80000000>; > > I would have tried that too :-) > > > > Interestingly enough, Xen booted, and complained about only 192MB > > unallocated this time. > > So, I dropped the size of Dom0 to 640M and I got it boot and here's > > what I'm seeing as > > an output of xl info: > > total_memory : 1120 > > free_memory : 390 > > It still nowhere close to 2G. > > > > Then I booted the Linux kernel without Xen and it correctly identified > > all 2G worth of RAM, and in fact, > > Good! We can work with that. So that is, in fact, my first question -- why is Xen not showing available memory in xl info? > > when I converted /sys/firmware/devicetree/base back into dts, here's > > what I've got: > > > > memory { > > device_type = "memory"; > > reg = <0x0 0x0 0x0 0x5e00000 0x0 0x5f00000 0x0 0x1000 > > 0x0 0x5f02000 0x0 0xefd000 0x0 0x6e00000 0x0 0x60f000 0x0 0x7410000 > > 0x0 0x1aaf0000 0x0 0x21f00000 0x0 0x100000 0x0 0x22000000 0x0 > > 0x1c000000>; > > }; > > > > reserved-memory { > > ranges; > > #size-cells = <0x2>; > > #address-cells = <0x2>; > > > > ramoops@21f00000 { > > ftrace-size = <0x20000>; > > console-size = <0x20000>; > > reg = <0x0 0x21f00000 0x0 0x100000>; > > record-size = <0x20000>; > > compatible = "ramoops"; > > }; > > > > linux,cma { > > linux,cma-default; > > reusable; > > size = <0x0 0x8000000>; > > compatible = "shared-dma-pool"; > > }; > > }; > > > > If you look at the REG -- it does now add up to 2Gb, > > I am a bit confused by this. I did the calculation twice and it is still > only 990MB. In fact, what you pasted here really looks like the old > device tree. Is it possible that you run the test with the old device > tree? There's something weird going on when it comes to Linux on this box. Basically, it seems that regardless of what devicetree I pass, Linux kernel seems to successfully discover all of 2G of memory. For example, I'm attaching a Linux boot log that clearly shows: Memory: 1877336K/2062392K available Even though I double checked that devicetree is only advertising 1G. But see below for more on this: > > but booting Xen > > with it has exactly theo > > same effect as booting it with: reg = <0x0 0x0 0x0 0x80000000>; > > > > I am attaching a full log, and I see the following in the logs: > > > > (XEN) Allocating 1:1 mappings totalling 720MB for dom0: > > (XEN) BANK[0] 0x00000008000000-0x0000001c000000 (320MB) > > (XEN) BANK[1] 0x00000040000000-0x00000058000000 (384MB) > > (XEN) BANK[2] 0x0000007b000000-0x0000007c000000 (16MB) > > > > Which sort of makes sense, I guess -- but I still don't understand > > where all these ranges > > are coming from and how come Xen doesn't see the full 2Gb even with various > > devicetrees I tried. > > > > Any ideas here would be greatly apprecaited! > > I think you might have run the test with the old device tree by mistake? > If you are sure that Linux can boot OK with memory as: > > reg = <0x0 0x0 0x0 0x80000000>; > > and correctly sees 2GB, then it should work with Xen too. Well, that's the issue -- it seems that Linux somehow doesn't depend *at all* on what I put in devicetrees -- it always detects full 2G. > In fact, looking at the logs you pasted, the choice of memory for dom0: > > 0x40000000-0x58000000 > 0x7b000000-0x7c000000 > > means that Xen was succesfully able to see the RAM above 0x40000000! > So, it looked like it already worked some extent! Exactly! That's the other surprising bit -- I noticed that too -- its not like Xen doesn't see any of the memory above 1G -- it just doesn't see enough of it. So the question is -- what is Linux doing that Xen doesn't? Thanks, Roman. Attachment:
linux.log _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxxxxxxxxx https://lists.xenproject.org/mailman/listinfo/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |