[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] Re: [PATCH 00 of 36] x86/paravirt: groundwork for 64-bit Xen support

Ingo Molnar wrote:
"paravirt/x86_64: move __PAGE_OFFSET to leave a space for hypervisor"

This moves __PAGE_OFFSET up by 16 GDT slots, from 0xffff810000000000 to 0xffff880000000000. I have no general justification for this: the specific reason is that Xen claims the first 16 kernel GDT slots for itself, and we must move up the mapping to make room. In the process I parameterised the compile-time construction of the initial pagetables in head_64.S to cope with it.

This reduces native kernel max memory support from around 127 TB to around 120 TB. We also limit the Xen hypervisor to ~7 TB of physical memory - is that wise in the long run? Sure, current CPUs support 40 physical bits [1 TB] for now so it's all theoretical at this moment.

my guess is that CPU makers will first extend the physical lines all the way up to 46-47 bits before they are willing to touch the logical model and extend the virtual space beyond 48 bits (47 bits of that available to kernel-space in practice - i.e. 128 TB).

So eventually, in a few years, we'll feel some sort of crunch when the # of physical lines approaches the # of logical bits - just like when 32-bit felt a crunch when physical lines went to 31 and beyond.

There's no inherent reason why Xen itself needs to be able to have all memory mapped at once. 32-bit Xen doesn't and can survive quite happily. It's certainly nice to be able to access anything directly, but it's just a performance optimisation. In practice, the guest generally has almost everything interesting mapped anyway, and Xen maintains a recursive mapping of the pagetable to make its access to the pagetable very efficient, so it's only when a hypercall is doing something to an unmapped page that there's an issue.

The main limitation the hole-size imposes is the max size of the machine to physical map. That uses 8bytes/page, and reserves 256GB of space for it, meaning that the current limit is 2^47 bytes - but there's another 256GB of reserved and unused space next to it, so that could be easily extended to 2^48 if that really becomes an issue.

That should be fine too - and probably useful for 64-bit kmemcheck support as well.

To further increase the symmetry between 64-bit and 32-bit, could you please also activate the mem=nopentium switch on 64-bit to allow the forcing of a non-PSE native 64-bit bootup? (Obviously not a good idea normally, as it wastes 0.1% of RAM and increases PTE related CPU cache footprint and TLB overhead, but it is useful for debugging.)

OK. Though it might be an idea to add "nopse" and start deprecating nopentium.

a few other risk areas:

- the vmalloc-sync changes. Are you absolutely sure that it does not
  matter for performance?

Oh, I didn't mean to include that one. I think it's probably safe (from both the performance and correctness stands), but it's not necessary for 64-bit Xen.

- "The 32-bit early_ioremap will work equally well for 64-bit, so just
   use it." Famous last words ;-)

Anyway, that's all theory - i'll try out your patchset in -tip to see what breaks in practice ;-)

Yep, thanks,


Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.