WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-ia64-devel

Re: [Xen-ia64-devel] RE: VTI will crash with memory=3073M

To: Akio Takebe <takebe_akio@xxxxxxxxxxxxxx>
Subject: Re: [Xen-ia64-devel] RE: VTI will crash with memory=3073M
From: Isaku Yamahata <yamahata@xxxxxxxxxxxxx>
Date: Wed, 24 Sep 2008 19:03:12 +0900
Cc: xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
Delivery-date: Wed, 24 Sep 2008 03:03:17 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <20080924050735.GC4150%yamahata@xxxxxxxxxxxxx>
List-help: <mailto:xen-ia64-devel-request@lists.xensource.com?subject=help>
List-id: Discussion of the ia64 port of Xen <xen-ia64-devel.lists.xensource.com>
List-post: <mailto:xen-ia64-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-ia64-devel>, <mailto:xen-ia64-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-ia64-devel>, <mailto:xen-ia64-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-ia64-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.5.6i
On Wed, Sep 24, 2008 at 02:07:35PM +0900, Isaku Yamahata wrote:
> On Tue, Sep 02, 2008 at 03:23:15PM +0900, Akio Takebe wrote:
> > Zhang, Jingke wrote:
> > >Akio Takebe wrote:
> > >>Zhang, Jingke wrote:
> > >>>Tristan Gingold wrote:
> > >>>>On Tue, Sep 02, 2008 at 11:43:58AM +0800, Zhang, Jingke wrote:
> > >>>>>Hi Tristan,
> > >>>>>    We found VTI guest with 3073M (memory is a little more than 3G)
> > >>>>>    can not be booted up. After some investigation, our engineer
> > >>>>>found some pages are ruined by unknow reason.  A EFI driver was
> > >>>>>loaded to 4G+, and its initilization code used the ruined pages, so
> > >>>>>bug out. I used latest openGFW binary (xenia64-gfw-126.bin). This
> > >>>>>issue is very easy to reproduce. Could you please help to look at
> > >>>>>it? Thank you very much!
> > >>>>Hi,
> > >>>>
> > >>>>Can you explain what do you mean by 'ruined' ?
> > >>>>
> > >>>>Tristan.
> > >>>Hi Tristan,
> > >>>    We found some code pages are polluted by an EFI driver
> > >>>(currently, we did not locate which driver), and leads to the issue.
> > >>>Thanks!
> > >>I guess the same problem.
> > >>http://lists.xensource.com/archives/html/xen-ia64-devel/2008-07/msg00200.html
> > >
> > >Yes, it should be the issue. Have any idea to the root cause? Thank you 
> > >very much!
> > >
> > I don't know the detail.
> > Kuwamura checked the memory area that we cannot boot hvm a guest.
> > 
> > He tested from 3073MB to 3299MB, the results are below:
> >   FAIL: 3073-3077, 3079-3094, 3098MB
> >   PASS: 3078, 3095-3097, 3099-3299MB
> > 
> > He said we would avoid this problem if we don't specify 3073-3098MB.
> 
> Hi.
> 
> I looked into the issue a bit.
> EFI_PEI_SERVICE funciton pointers are located on the stack for PEI.
> Its service should be available until DxeIpl done and it is surely
> used.
> However if the stack page happens to be reused during DxeIpl, the 
> function pointers are overwritten. Then guest panic occurs the next
> time the pei service is called via this function pointers which
> don't contain correct value.
> 
> Pei page allocate seems to depend on initial memory layout so that
> which cases fail/pass depends on the allocator.
> 
> Hmm, the right solution looks like that
> - prevent the stack page from being allocated until DxeIPL done.
> - (Optionally) free the page after DxeIpl done.
>   Probably at the beginning of Dxe.

I think I tracked down the cause and created a patch.
Please comment or give it a try.
However gfw seems very unstable now. So this patch isn't for
commit for the moment.

The cause lived in the PEI page allocator in
edk2/EdkModulePkg/Core/Pei/Memory/MemoryServices.c
The available memory is registered by PeiInstallPeiMemory()
in EdkXenPkg/Pei/XenMemoryScan/XenMemoryScan.c.
Then, the PEI page allocator, PeiAllocatePages(), allocates pages
from the area which is lastly registered.
The stack is allocated at the beginning (lowest address) of the lastly
registered area.

On the other hand, PeiAllocatePages() allocates pages from the end
(highest address) of the area and it doesn't check overlapping
with stack. So if the last registered area isn't large enough,
it happens to use the stack. Ouch! This looks like PEI memory allocator
limitation.

On the other hand, Xen HVM domain memory is allocated as
[0, 0xA000) [VGA hole) [0xC000, 3GB) [hole for GFW) [4GB, end)
and those areas are registered in lower to higher order.
So if the memory assigned to the HVM domain is 3GB + small amount,
the last area, [4GB, end), happens to be very small and so the stack
is used.

thanks,

install biggest memory range last to avoid out of memory.

Signed-off-by: Isaku Yamahata <yamahata@xxxxxxxxxxxxx>

diff -r dd0ca9fcc64a edk2-sparse/EdkXenPkg/Pei/XenMemoryScan/XenMemoryScan.c
--- a/edk2-sparse/EdkXenPkg/Pei/XenMemoryScan/XenMemoryScan.c   Wed Sep 24 
18:13:43 2008 +0900
+++ b/edk2-sparse/EdkXenPkg/Pei/XenMemoryScan/XenMemoryScan.c   Wed Sep 24 
18:46:12 2008 +0900
@@ -89,6 +89,8 @@
 
   UINT8 *hob = (UINT8 *)GFW_HOB_START;
   struct XenHobHeader *hdr;
+  UINT64 BaseOfMaxLen;
+  UINT64 MaxLen = 0UL;
 
   do
     {
@@ -101,6 +103,9 @@
       data = (UINT64 *)(hdr + 1);
       data_len = hdr->length - sizeof (struct XenHobHeader);
       if (hdr->type == HOB_TYPE_MEM && data_len == 16) {
+       UINT64 mem_start;
+       UINT64 mem_len = 0;
+
         Base = data[0];
         Len = data[1];
 
@@ -109,8 +114,6 @@
         // VGA RAM (0xA0000 ~ 0xBFFFF) + VGA ROM (0xC0000 ~ 0xDFFFF)
         //
         if (Base < VGA_END && (Base + Len) >= VGA_BASE) {
-          UINT64 mem_start;
-          UINT64 mem_len;
 
           if (Base < VGA_BASE) {
             mem_start = Base;
@@ -119,25 +122,45 @@
             RegisterMemoryArea(PeiServices, mem_start, mem_len);
             DEBUG((EFI_D_ERROR, "Build memory hob at 0x%lx, len is 0x%lx\n",
                    mem_start, mem_len));
+
+           mem_len = 0;
           }
 
           if (Base + Len > VGA_END) {
             mem_start = VGA_END;
             mem_len = (Base + Len) - VGA_END;
-
-            RegisterMemoryArea(PeiServices, mem_start, mem_len);
-            DEBUG((EFI_D_ERROR, "Build memory hob at 0x%lx, len is 0x%lx\n",
-                   mem_start, mem_len));
           }
         } else {
-          RegisterMemoryArea(PeiServices, Base, Len);
-          DEBUG((EFI_D_ERROR, "Build memory hob at 0x%lx, len is 0x%lx\n",
-                 Base, Len));
+         mem_start = Base;
+         mem_len = Len;
         }
+
+
+       if (mem_len > MaxLen) {
+         UINT64 tmp_mem_start = mem_start;
+         UINT64 tmp_mem_len = mem_len;
+
+         mem_start = BaseOfMaxLen;
+         mem_len = MaxLen;
+         
+         BaseOfMaxLen = tmp_mem_start;
+         MaxLen = tmp_mem_len;
+       }
+       if (mem_len > 0) {
+         RegisterMemoryArea(PeiServices, mem_start, mem_len);
+         DEBUG((EFI_D_ERROR, "Build memory hob at 0x%lx, len is 0x%lx\n",
+                mem_start, mem_len));
+       }
       }
       hob += hdr->length;
     }
   while (hdr->type != HOB_TYPE_TERMINAL);
+
+  if (MaxLen > 0) {
+    RegisterMemoryArea(PeiServices, BaseOfMaxLen, MaxLen);
+    DEBUG((EFI_D_ERROR, "Build memory hob at 0x%lx, len is 0x%lx\n",
+          BaseOfMaxLen, MaxLen));
+  }
 
   //
   // Build the CPU hob with 44-bit addressing and 16-bits of IO space.

-- 
yamahata

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ia64-devel