RE: [Xen-ia64-devel] [PATCH][RFC]discontig memory support

["Tian, Kevin" <kevin.tian@xxxxxxxxx>]

>From: Kouya SHIMURA
>Sent: 2006年4月3日 20:32
>Hi xen/ia64 developers.
>
>The attached patch supports discontiguous memory.
>It also makes over 4GB memory available.
>Please comment and review.
>
>Signed-off-by: Kouya Shimura <kouya@xxxxxxxxxxxxxx>

Good work, Kouya.

>
>I have no idea which has good performance. Attached patch implements
>the same way as VIRTUAL_MEM_MAP because of ease. But I think
>SPARSEMEM
>should be implemented in future because it would be a mainstream in
>Linux. In this case, common code might be modified.

It's OK to start from easy way first, and then evaluate which one is better. 
>From what I recalled, there's no obvious preference on ia64/linux 
community upon these two approaches(simplicity Vs complex, TLB Vs 
cache) and that's why two approaches still co-exist waiting for more data 
to distinguish. However things may become a bit different for xen/ia64, 
since currently there's no VHPT table used for Xen hypervisor. A rough 
thought based on that background is, people may have to face many TLB 
misses caused by access to vmem_map table and thus the benefit of 
VIRTUAL_MEM_MAP is pulled down...

>
>[DESIGN]
>* Location of the virtual address of frame_table
>  I set the frame_table start at 0xf300000000000000 which region
>  number is 7. Region number 7 is used for Xen hypervisor. The page
>  size is 16KB. VHPT is disabled in this region and 16KB/16MB/64MB
>  page size are mixed. The virtual aliasing might occur but the
>  performance wouldn't be affected.

Let's make it clearer. Region 7 is shared among guest and xen hypervisor, 
where 0xe800.... - 0xf7ff.... is reserved for the later. In this context, 
region 
ID is shared.

Another point is, VHPT walker is enabled in region 7 at dom0 boot phase 
and then on, though that VHPT table only caches guest translation 
content. (set_one_rr)

For virtual aliasing, IA64 processor ensures cache coherency and data 
dependencies in the presence of an alias. You need be more careful 
about the virtual attribute aliasing which is a disaster to the whole system. 
But normally it should be OK, since frame table itself just comes from 
normal memory. :-)

>  - Location of PGD.
>    A page(swapper_pg_dir) pointed from init_mm.pgd seems to be
>never
>    used. So I use this page as PGD for virtual frame_table. If
>    someone uses this page, please tell me.

Though I'm not sure whether some code path with reference to this page 
will be actually executed, it never hurt to allocate a new page/name 
specific for your purpose which is also clearer.

>
>I appended a code that checks the fault address is inside of
>frame_table and traverses the address translation table and inserts a
>translation cache. To avoid nested data TLB fault, data address
>translation is disabled at the time of table walking. On fail of table
>walking, ia64_fault() is called.

If you can measure the TLB miss ratio of accessing vmemmap, that could 
help a lot to make right choice.

>
>[STATUS]
>  * Xen and domU and domVTI (cset:9484:ddc279c91502) work well on
>    Tiger4 with 8MB physical memory.
>
>[TODO]
>  * mpt_table (defined in xen/arch/ia64/xen/xenmem.c) implies the same
>    problem. We have to fix it.
>  * redesign of address translation table
>  * performance evaluation
>  * cleanup #ifdefs of CONFIG_VIRTUAL_MEM_MAP
>--------------------------------------------------------------------

So last comment is, you may need to keep 
CONFIG_VIRTUAL_MEM_MAP option for a while, instead of merging 
code immediately. That can help add or remove a feature as an integrity, 
especially when direction is not clear. Also like Linux, it's better to add a 
check upon holes even when CONFIG_VIRTUAL_MEM_MAP is enabled, 
since overhead can be avoided on boxes without memory holes by using 
physical memmap.

Thanks,
Kevin

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ia64-devel