[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] GPU passthrough issue when VM is configured with 4G memory

On 05/22/2013 07:11 PM, Andrew Bobulsky wrote:
On Wed, May 22, 2013 at 11:17 AM, Gordan Bobic <gordan@xxxxxxxxxx
<mailto:gordan@xxxxxxxxxx>> wrote:

    On Tue, 21 May 2013 04:09:25 +0000, Hanweidong
    <hanweidong@xxxxxxxxxx <mailto:hanweidong@xxxxxxxxxx>> wrote:

            -----Original Message-----
            From: Gordan Bobic [mailto:gordan@xxxxxxxxxx
            Sent: 2013å5æ21æ 2:43
            To: George Dunlap
            Cc: Hanweidong; Stefano Stabellini; xudong.hao@xxxxxxxxx
            <mailto:xudong.hao@xxxxxxxxx>; Yanqiangjun;
            Luonengjun; Wangzhenguo; Yangxiaowei; Gonglei (Arei);
            Anthony Perard;
            xen-devel@xxxxxxxxxxxxx <mailto:xen-devel@xxxxxxxxxxxxx>;
            xiantao.zhang@xxxxxxxxx <mailto:xiantao.zhang@xxxxxxxxx>
            Subject: Re: [Xen-devel] GPU passthrough issue when VM is
            with 4G memory

            I'd also like to stress this is not only an issue for > 4GB
            of RAM in
            domU - I am seeing the issue with > 2GB of RAM in domU.

        What's the total mmio size of you domU?

    How can I find that out?

        When RAM of domU overlaps
        with pci_mem_start, hvmloader will relocate RAM, and it will cause
        problem due to hvmloader and QEMU don't setup the memory layout
        consistently. I suspect the mmio size of your domU is close to 2GB.
        When you configured RAM > 2G, then RAM of your domU overlapped with
        pci_mem_start, and resulted in failure.

    Is there a way to force a hole between, say, 2GB and 4GB explicitly
    to avoid the PCI memory being clobbered?

I'll second that question!
Gordan, one thing I didn't think of until now: the card I was having
trouble with, a Radeon 6990, is a 4GB card, laid out as two 2GB GPUs.
The card with which I don't have issues (aside from the occasional need
to reboot the whole system to get it to work right), a Radeon 5850, is a
1GB card.  If as this thread has suggested, along with what I've read
about the PCI Hole, the memory on the GPU determines the "clobber size"
(yes I made that term up :P),

I'm pretty sure the GPU RAM size is not relevant. I've seen this with the 1GB Nvidia Quadro 2000, 1GB ATI 7450 and the 6GB 7970. The GPU memory size does not affect the aperture size or location. I for one would love it if all of the GPU RAM was in fact mapped as a BAR - that would open up some very funky approaches to processing large amounts of data, e.g. by just mapping the data straight to the GPU. It would also make it trivially easy to use all that RAM as a RAM disk when the GPU isn't being used (e.g. you can use the RAM for fast swap for normal desktop tasks, and free it up when gaming). but that's a whole different topic.


Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.