WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] PAE guest address maps?

To: Steven Hand <Steven.Hand@xxxxxxxxxxxx>, xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: Re: [Xen-devel] PAE guest address maps?
From: Randy Thelen <rthelen@xxxxxxxxxx>
Date: Mon, 30 Oct 2006 12:46:07 -0800
Cc: Craig Everhart <everhart@xxxxxxxxxx>
Delivery-date: Thu, 02 Nov 2006 13:50:23 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <E1GcvIQ-0005qA-00@xxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <E1GcvIQ-0005qA-00@xxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Steven Hand wrote:

Take a look at xen/include/asm-x86/config.h ; this defines the
overall memory layout for x86_32, PAE & x86_64 systems, as well
as including some comments that should help you.

Thanks, Steven.  I hadn't seen that file before.

However, I'm still confused (or frustrated, I'm not entirely sure which!). Here's the memory layout table from that file:

/*
* Memory layout (high to low):                          SIZE   PAE-SIZE
*                                                       ------ ------
*  I/O remapping area                                   ( 4MB)
*  Direct-map (1:1) area [Xen code/data/heap]           (12MB)
*  Per-domain mappings (inc. 4MB map_domain_page cache) ( 8MB)
*  Shadow linear pagetable                              ( 4MB) ( 8MB)
*  Guest linear pagetable                               ( 4MB) ( 8MB)
*  Machine-to-physical translation table [writable]     ( 4MB) (16MB)
*  Frame-info table                                     (24MB) (96MB)
*   * Start of guest inaccessible area
*  Machine-to-physical translation table [read-only]    ( 4MB) (16MB)
*   * Start of guest unmodifiable area
*/

When my guest domain starts (FreeBSD in PAE mode), I begin looking at the page tables. I note that I've modified Xend (the libxc code) to build 4 pages of Page Directories because FreeBSD uses the model of assuming that there are four Page Directories linearly mapped. This way it can just forget about the 4 PTD pointers in the L3 page. It means that 16KB are consumed for every address space, but it greatly simplifies the code.

(gdb) x/4x xen_start_info->pt_base
0xc03ae000:     0x00000000fb152001      0x00000000fb151001
0xc03ae010:     0x00000000fb150001      0x00000000fb14f001

And, my kernel starts at virtual address 0xC000.0000, so only page 3 (in 0..3) of the 4 pages needs to have anything. And sure enough, that's what I find:

# ./xenctrl 4 -l3 0xfb153 dump64 0xc03af000 0x4000 # I'm dumping 64 bit values
0xc03af000:      -- page of zeros --
0xc03b0000:      -- page of zeros --
0xc03b1000:      -- page of zeros --
0xc03b2000: 0x00fb14e067 0x00fb14d067 0x00fb14c067 0x00fb14b067
        ...
0xc03b2d60: 0x00fbc000e1 0x00fba000e1 0x0000000000 0x0000000000
        ...
0xc03b2da0: 0x00018001e3 0x0001a001e3 0x0001c001e3 0x0001e001e3 0xc03b2dc0: 0x00020001e3 0x00022001e3 0x00024001e3 0x00026001e3 0xc03b2de0: 0x00028001e3 0x0002a001e3 0x0002c001e3 0x0002e001e3
        ...
0xc03b2f20: 0x00fbc001e3 0x00fba001e3 0x0000000000 0x0000000000
        ...
0xc03b2f60: 0x00fb152063 0x00fb151063 0x00fb150063 0x00fb14f063
        ...
0xc03b2fa0: 0x0000224063 0x0000225063 0x0000226063 0x0000227063 0xc03b2fc0: 0x0000211063 0x0000212063 0x0000213063 0x0000214063 0xc03b2fe0: 0x0000215063 0x0000216063 0x0000217063 0x0000bfe063
#

The four page table entries at 0xc03b2000 map my kernel (base address 0xC000.0000 means the first entries in page 3 of 0..3). And, my kernel is 8MB, so all is well.

The following entries are broken-out as follows:

0xC03B2D60 has two entries which map to base address 0xF580.0000 for 4MB. This is 168 MB below 4GB. So, this should be the machine-to- physical translation table. However, I can't map it from a tool in Dom0, nor can I see it in GDB:

(gdb) x/32w 0xf5800000
0xf5800000:     Cannot access memory at address 0xf5800000

Nor can I map the physical pages (of the L1 page table!):

# ./xenctrl 4 -l3 0xfb153 dump64-mfn 0xfbc00000 0x1000
Error: Couldn't map mfn fbc00000

This tells me I can't determine the L0 pages used.  That's too bad.

So, I'm not sure how I'm suppose to use this. My guest domain -can- see the F58 address region. And, so I can bcopy() the data into a visible space for gdb. In the following snippet, I am using two tools: GDB and a homebrew tool to view domain 0 pages, xenctrl:

(gdb) call bzero(0xc0000000, 0x1000) # Just to certify that I can affect 0xC0000000
$22 = 0

# ./xenctrl 4 -l3 0xfb153 dump 0xc0000000 0x1000 # Verify that I can view it
0xc0000000:      -- page of zeros --

(gdb) call bcopy(0xf5800000, 0xc0000000, 0x1000) # See what's in the machine-to-phys table
$23 = -897581056

# ./xenctrl 4 -l3 0xfb153 dump 0xc0000000 0x100
0xc0000000: 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xc0000020: 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xc0000040: 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xc0000060: 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xc0000080: 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xc00000a0: 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xc00000c0: 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xc00000e0: 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0xffffffff

I'm not sure what I'm suppose to make of those numbers. If I view the whole page (all 4K) I get long sequences of 0xffff's and 0x5555's.

Is there any documentation on the machine-to-physical region of memory? Any sample code on its usage? I tried looking in Linux, but I wasn't able to identify the machine-to-physical conversion macros or functions for x86-32 on PAE.


Moving on to the next address space, I find Page Directory Entries for the address space 0xFD800000. I have learned that this address space is the recursive page table address space. I.e., in theory my guest domain could perform virtual to machine page translations here with the algorithm I stated last Wednesday: ((virtual-address & 0xFFFF.F000) >> 9) +0xFD80.0000 would generate the address of the PTE for the current virtual page (assuming the virtual address is valid in the first place!). But, the comment in the config.h file says this is beyond the "Start of guest inaccessible area." Which is a damn shame:

(gdb) call bcopy(0xfde00000, 0xc0000000, 8)
<hang>

Because I can get access to the data from Dom0 (I actually executed the following steps -before- the steps above, which caused a hang):

(gdb) p/x ((0xC0000000 & 0xFFFFF000) >> 9) +0xFD800000
$26 = 0xfde00000
(gdb) x/1g $26
0xfde00000:     0x00000000d1620063
(gdb)

It would be -very- useful to my program if I could use the existing recursive page table.

This all comes to pass because I'm trying to build the recursive address table for FreeBSD and I need information I can't find; i.e., the machine to physical table.

-- Randy



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>