This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-devel] RE: Intel GPU pass-through with > 3G

To: Jean Guyader <jean.guyader@xxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: [Xen-devel] RE: Intel GPU pass-through with > 3G
From: "Kay, Allen M" <allen.m.kay@xxxxxxxxx>
Date: Wed, 10 Nov 2010 16:04:18 -0800
Accept-language: en-US
Acceptlanguage: en-US
Delivery-date: Wed, 10 Nov 2010 16:05:19 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <AANLkTimNJZXkHDhwQDhYr6oiQE_uXarktA9-AL4Hp9xn@xxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <AANLkTimNJZXkHDhwQDhYr6oiQE_uXarktA9-AL4Hp9xn@xxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcuBIU4EwMns3lXjRpuUmN1pmBunrQAES/2A
Thread-topic: Intel GPU pass-through with > 3G

Do you see any boot time difference between passing through integrated graphics 
for the very first time and the subsequent times?  Which platform are you using?


-----Original Message-----
From: Jean Guyader [mailto:jean.guyader@xxxxxxxxx] 
Sent: Wednesday, November 10, 2010 1:50 PM
To: xen-devel@xxxxxxxxxxxxxxxxxxx
Cc: Kay, Allen M
Subject: Intel GPU pass-through with > 3G


I'm passing through a graphic card to a guest that has more than 3G of
RAM (4G to be precise in my case).

What happen is that the VM creation is stuck in the process, so I put
some tracing in the Xen code to see what
was taking the time. I discovered that the guest was stuck in
hvmloader inside this loop:

   while ( (pci_mem_start >> PAGE_SHIFT) < hvm_info->low_mem_pgend )
        struct xen_add_to_physmap xatp;
        if ( hvm_info->high_mem_pgend == 0 )
            hvm_info->high_mem_pgend = 1ull << (32 - PAGE_SHIFT);
        xatp.domid = DOMID_SELF;
        xatp.space = XENMAPSPACE_gmfn;
        xatp.idx   = --hvm_info->low_mem_pgend;
        xatp.gpfn  = hvm_info->high_mem_pgend++;
        if ( hypercall_memory_op(XENMEM_add_to_physmap, &xatp) != 0 )

This loop relocate the RAM on the top to leave so space for the PCI BARs.
It's loop on each page so in my case it's quite a big loop because the
GPU has a BAR of 256M.

So the interesting is that the function add_to_physmap takes most of
the time. I believe
that what takes most part of it is the iommu iotlb flush that come
with the iommu_map_pages
or the iommu_unmap_page which are called when we manipulate the p2m table.

In my case the iommu flush take a very long time (because of the intel
gpu ?), about 10
milliseconds. So if I'm patient enough my domain will start, about 10 minutes.

A way to go will be to create a range interface to iommu_map_page
since iommu_flush are so expensive. Then some work need to be done to
add a range interface
to all the function between add_to_physmap and the p2m_set_entry which
would be a big
patch. I hope there is another way out of this problem.


Xen-devel mailing list