Re: [Xen-devel] Is: MTRR issue + Xen + memory clipping? Was:Re: Xen not working with stock Debian Wheezy 3.2 kernel on a Core 2 Duo box

[Wei Liu <wei.liu2@xxxxxxxxxx>]

On Tue, Jul 09, 2013 at 11:46:51AM +0200, Stefan Bader wrote:
> On 08.07.2013 21:47, Konrad Rzeszutek Wilk wrote:
> > On Mon, Jul 08, 2013 at 02:16:07PM +0100, Jan Beulich wrote:
> >>>>> On 08.07.13 at 14:50, Wei Liu <wei.liu2@xxxxxxxxxx> wrote:
> >>> On Mon, Jul 08, 2013 at 12:29:02PM +0100, Jan Beulich wrote:
> >>>> One question of course is where this pretty unusual "unusable"
> >>>> memory block comes from on that system. Is this block visible the
> >>>> same way when booting a native kernel, or is this being forced to
> >>>> "unusable" by Xen?
> >>>
> >>> Vanilla 3.10 + Xen unstable:
> >>> [    0.000000] Xen: [mem 0x0000000000000000-0x000000000008efff] usable
> >>> [    0.000000] Xen: [mem 0x000000000008f000-0x00000000000fffff] reserved
> >>> [    0.000000] Xen: [mem 0x0000000000100000-0x00000000ce699fff] usable
> >>> [    0.000000] Xen: [mem 0x00000000ce69a000-0x00000000ce6f0fff] ACPI NVS
> >>> [    0.000000] Xen: [mem 0x00000000ce6f1000-0x00000000cf5fafff] usable
> >>> [    0.000000] Xen: [mem 0x00000000cf5fb000-0x00000000cf607fff] reserved
> >>> [    0.000000] Xen: [mem 0x00000000cf608000-0x00000000cf6a4fff] usable
> >>> [    0.000000] Xen: [mem 0x00000000cf6a5000-0x00000000cf6a9fff] ACPI data
> >>> [    0.000000] Xen: [mem 0x00000000cf6aa000-0x00000000cf6aafff] usable
> >>> [    0.000000] Xen: [mem 0x00000000cf6ab000-0x00000000cf6f1fff] ACPI NVS
> >>> [    0.000000] Xen: [mem 0x00000000cf6f2000-0x00000000cf6fefff] ACPI data
> >>> [    0.000000] Xen: [mem 0x00000000cf6ff000-0x00000000cf6fffff] usable
> >>> [    0.000000] Xen: [mem 0x00000000cf700000-0x00000000cfffffff] reserved
> >>> [    0.000000] Xen: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
> >>> [    0.000000] Xen: [mem 0x00000000fff00000-0x00000000ffffffff] reserved
> >>> [    0.000000] Xen: [mem 0x0000000100000000-0x0000000227ffffff] usable
> >>> [    0.000000] Xen: [mem 0x0000000228000000-0x000000022bffffff] unusable
> >>> [    0.000000] ERROR: earlyprintk= xenboot already used
> >>> [    0.000000] NX (Execute Disable) protection: active
> >>> [    0.000000] SMBIOS 2.4 present.
> >>> [    0.000000] No AGP bridge found
> >>> [    0.000000] e820: last_pfn = 0x228000 max_arch_pfn = 0x400000000
> >>> [    0.000000] e820: last_pfn = 0xcf700 max_arch_pfn = 0x400000000
> >>> [    0.000000] Scanning 1 areas for low memory corruption
> >>> [    0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
> >>> [    0.000000] init_memory_mapping: [mem 0x219e00000-0x219ffffff]
> >>> [    0.000000] init_memory_mapping: [mem 0x218000000-0x219dfffff]
> >>> [    0.000000] init_memory_mapping: [mem 0x200000000-0x217ffffff]
> >>> [    0.000000] init_memory_mapping: [mem 0x00100000-0xce699fff]
> >>> [    0.000000] init_memory_mapping: [mem 0xce6f1000-0xcf5fafff]
> >>> [    0.000000] init_memory_mapping: [mem 0xcf608000-0xcf6a4fff]
> >>> [    0.000000] init_memory_mapping: [mem 0xcf6aa000-0xcf6aafff]
> >>> [    0.000000] init_memory_mapping: [mem 0xcf6ff000-0xcf6fffff]
> >>> [    0.000000] init_memory_mapping: [mem 0x100000000-0x1ffffffff]
> >>> [    0.000000] init_memory_mapping: [mem 0x21a000000-0x227ffffff]
> >>>
> >>> We also have the similar unusable block, however the kernel doesn't map 
> >>> it.
> >>
> >> Right, iirc a fix for this was done not too long ago. Konrad may
> >> recall further details...
> > 
> > Stefan hit this I think. With the MTRR clipping blowing things. CC-ing him
> > here.
> 
> Not personally but we had a bug report about this. Unfortunately it was a bit
> confusing as in the beginning it was not really obvious whether the problem 
> was
> or was not fixed in later kernels or whether the different installations used
> different dom0_mem arguments.
> 
> Reading the old bug report (which never completed as it seemed the reporters
> apparently lost interest) I think the problem was that the hypervisor detects
> the MTRR problem and set the remainder of memory as unusable.
> Then dom0 kernel (and if I parse that report right, it may have been fixed
> between 3.2 and 3.3 but hard to say when all you get is them saying yes or no)
> would somehow still try to put mappings in there.
> 
> The link below is that bug report. Not sure of how much help it is.
> 
> 
> [1] https://bugs.launchpad.net/ubuntu/+source/xen/+bug/1111470

Thanks Stefan.

At least I now know that a fix went in between 3.2 and 3.3, which is
much better range than 3.2..3.10.

I will try to bisect that if I have time, however it is low priority on
my list...


Wei.


> >>> 3.2.0-4-amd64 (just found out that there's actually backtrace in dmesg):
> >>> [    0.000000] BIOS-provided physical RAM map:
> >>> [    0.000000]  BIOS-e820: 0000000000000000 - 000000000008f000 (usable)
> >>> [    0.000000]  BIOS-e820: 000000000008f000 - 00000000000a0000 (reserved)
> >>> [    0.000000]  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
> >>> [    0.000000]  BIOS-e820: 0000000000100000 - 00000000ce69a000 (usable)
> >>> [    0.000000]  BIOS-e820: 00000000ce69a000 - 00000000ce6f1000 (ACPI NVS)
> >>> [    0.000000]  BIOS-e820: 00000000ce6f1000 - 00000000cf5fb000 (usable)
> >>> [    0.000000]  BIOS-e820: 00000000cf5fb000 - 00000000cf608000 (reserved)
> >>> [    0.000000]  BIOS-e820: 00000000cf608000 - 00000000cf6a5000 (usable)
> >>> [    0.000000]  BIOS-e820: 00000000cf6a5000 - 00000000cf6aa000 (ACPI data)
> >>> [    0.000000]  BIOS-e820: 00000000cf6aa000 - 00000000cf6ab000 (usable)
> >>> [    0.000000]  BIOS-e820: 00000000cf6ab000 - 00000000cf6f2000 (ACPI NVS)
> >>> [    0.000000]  BIOS-e820: 00000000cf6f2000 - 00000000cf6ff000 (ACPI data)
> >>> [    0.000000]  BIOS-e820: 00000000cf6ff000 - 00000000cf700000 (usable)
> >>> [    0.000000]  BIOS-e820: 00000000cf700000 - 00000000d0000000 (reserved)
> >>> [    0.000000]  BIOS-e820: 00000000fff00000 - 0000000100000000 (reserved)
> >>> [    0.000000]  BIOS-e820: 0000000100000000 - 000000022c000000 (usable)
> >>
> >> So no such block right off the BIOS. You're not using Xen TXT code
> >> by chance? Off the top of my head I don't recall any other place
> >> where multiple RAM pages might get turned into "unusable".
> >>
> >> Jan
> >>
> >> _______________________________________________
> >> Xen-devel mailing list
> >> Xen-devel@xxxxxxxxxxxxx
> >> http://lists.xen.org/xen-devel
> 
> 



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel