[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Bug: Limitation of <=2GB RAM in domU persists with 4.3.0



On 31/07/13 20:35, Gordan Bobic wrote:
On 07/31/2013 06:53 PM, George Dunlap wrote:
On Fri, Jul 26, 2013 at 2:11 PM, Gordan Bobic <gordan@xxxxxxxxxx> wrote:
Now that is intereting - if this makes the memory holes the same between
the guest and the host, does it also implicitly vBAR=pBAR?


Another thing that occurred to me might be useful to check - it is
pretty easy to modify the BAR size on Nvidia cards. The defaults are
64MB and 128MB for the two BARs. They can be made much, much larger,
and there is often advantage to enlarging them to at least be equal to
VRAM size. Soooooo... If I boost the BAR from 128MB to 2GB, being a
64-bit BAR, it might make the BIOS do the sane thing and map it above
4GB. With the other BAR also suitably enlarged and it being done on
the second GPU as well, there is no obvious option but to map them
above 4GB (unless the BIOS is broken, which it may well be, in
which case all bets are off).

Which may just alleviate the memory issue if not completely fix
the problem.

Will try this and see what happens.

I believe XenServer has a patch that allows the toolstack (in this
case xapi) to set the default size of the MMIO hole.  Andrew, did that
ever make it upstream?

Unfortunately, it is unlikely to work with upstream qemu until we fix
the memory relocation issue...

Interesting you should mention something like this. I've been pondering whether it might be easier (even if it is a bodge) to simply always set the domU E820 map to have 0x80000000 - 0xFFFFFFFF (2GB->4GB) reserved. I have not yet seen a motherboard that maps 32-bit BARs below 2GB.

I'm pretty sure we've seen a memory hole larger than 2GiB, in a box loaded up with a boatload of GPUs.

The main problem with doing this unconditionally is that the relocated memory isn't available to non-PAE 32-bit guests. I think we should have a work-around in place for 4.4 that will avoid a collision between the host MMIO and guest memory addresses; but it will need to be off by default, at least for guests that don't have a passed-through device.

 -George


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.