RE: [Xen-devel] 2.6.32.27 dom0 + latest xen staging boot failure

["Kay, Allen M" <allen.m.kay@xxxxxxxxx>]

Konrad/Stefano,

Getting back to the xen/dom0 boot failure on my Sandybridge SDP I reported a 
few weeks ago.

I finally got around to narrow down the problem the call to xen_add_extra_mem() 
in arch/x86/xen/setup.c/xen_memory_setup().  This call increase the top of E820 
memory in dom0 beyond what is actually available.

Before xen_add_extra_mem() is called, the last entry of dom0 e820 table is:

    0000000100000000 - 000000016b45a000 (usable)

After xen_add_extra_mem() is called, the last entry of dom0 e820 table becomes:

    0000000100000000 - 000000023a6f4000 (usable)

This pushes the top of RAM beyond what was reported by Xen's e820 table, which 
is:

(XEN)  0000000100000000 - 00000001de600000 (usable)

AFAICT, the failure is caused by dom0 accessing non-existent physical memory.  
The failure went away after I removed the call to xen_add_extra_mem().

Another potential problem I noticed with e820 processing is that there is a 
discrepancy between how Xen processes e820 and how dom0 does it.  In Xen 
(arch/x86/setup.c/start_xen()), e820 entries are aligned on L2_PAGETABLE_SHIFT 
boundary while dom0 e820 code does not.  As a result, one of my e820 entry that 
is 1 page in size got dropped by Xen but got picked up in dom0.  This does not 
cause problem in my case but the inconsistency on how memory is used by xen and 
dom0 can potentially be a problem.

Allen

-----Original Message-----
From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@xxxxxxxxxx] 
Sent: Friday, January 28, 2011 7:48 AM
To: Kay, Allen M
Cc: xen-devel; Stefano Stabellini
Subject: Re: [Xen-devel] 2.6.32.27 dom0 + latest xen staging boot failure

On Fri, Jan 28, 2011 at 10:28:43AM -0500, Konrad Rzeszutek Wilk wrote:
> On Thu, Jan 27, 2011 at 10:51:42AM -0800, Kay, Allen M wrote:
> > Following are the brief error messages from the serial console log.  I have 
> > also attached the full serial console log and dom0 system map.
> > 
> > (XEN) mm.c:802:d0 Bad L1 flags 400000
> 
> On a second look, this is a different issue than I had encountered.
> 
> The 400000 translates to Xen thinking you had PAGE_GNTTAB set, but that
> is not right. Googling for this shows that I had fixed this with a
> Xorg server at some point, but I can't remember the details so that is not
> that useful :-(
> 
> You said it works if you give the domain 1024MB, but I wonder if
> it also works if you disable the IOMMU? What happens then?

Can you also patch your Xen hypervisor with this patch? It will print out the
other 89 entries so we can see what type of values they have.. You might need to
move it a bit as this is for xen-unstable.

diff -r 003acf02d416 xen/arch/x86/mm.c
--- a/xen/arch/x86/mm.c Thu Jan 20 17:04:06 2011 +0000
+++ b/xen/arch/x86/mm.c Fri Jan 28 10:46:13 2011 -0500
@@ -1201,11 +1201,12 @@
     return 0;
 
  fail:
-    MEM_LOG("Failure in alloc_l1_table: entry %d", i);
+    MEM_LOG("Failure in alloc_l1_table: entry %d of L1 (mfn: %lx). Other L1 
values:", i, pfn);
     while ( i-- > 0 )
-        if ( is_guest_l1_slot(i) )
+        if ( is_guest_l1_slot(i) ) {
+            MEM_LOG("L1[%d] = %lx", i, (unsigned long)l1e_get_intpte(pl1e[i]));
             put_page_from_l1e(pl1e[i], d);
-
+       }
     unmap_domain_page(pl1e);
     return -EINVAL;
 }

> 
> > (XEN) mm.c:1204:d0 Failure in alloc_l1_table: entry 90
> > (XEN) mm.c:2142:d0 Error while validating mfn 1d7e97 (pfn 3d69) for type 
> > 1000000
> > 000000000: caf=8000000000000003 taf=1000000000000001
> > (XEN) mm.c:2965:d0 Error while pinning mfn 1d7e97
> > (XEN) traps.c:451:d0 Unhandled invalid opcode fault/trap [#6] on VCPU 0 
> > [ec=0000
> > ]
> > (XEN) domain_crash_sync called from entry.S
> > (XEN) Domain 0 (vcpu#0) crashed on cpu#0:
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

Attachment: failure.log
Description: failure.log

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel