WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] Intel GPU pass-through with > 3G

To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] Intel GPU pass-through with > 3G
From: Jean Guyader <jean.guyader@xxxxxxxxx>
Date: Wed, 10 Nov 2010 21:50:21 +0000
Cc: "Kay, Allen M" <allen.m.kay@xxxxxxxxx>
Delivery-date: Wed, 10 Nov 2010 13:51:17 -0800
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:cc:content-type; bh=xt1E+G3ZLZJz6N6xA1603Qy7xGdoWDzrkj5TmDUP8Xs=; b=KSzpCo7y8auBWn4CtvqJLC4gf8W1cpw6hI8Edtn+VAOzJ+LHkthfuwZ1BmwfNrLf/t BZzdVUgp6eqJX925NQmjVOt65GMCtyda5r8WDecYRO1vq0yfy2VR1ZS/h646ml08gukO WwlmeAZ0Xh7709segkMZGjtAJcaHbq27ufiiQ=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=Y5lA4JqILCqOXE/TTZsyuYan1Xg/9WjN+ZoIJ0T+8UqGS127Jh3prItX+hWnQxRtfk Dy0UpfUFumSi5YiDZRVA2Xe+PUUSzlA7hypt86dY7p51H4azg/99HmyMC3L+9k2vW6fs DOft/yxDQkPBIrN4c07rAruS0vIDjsQBDfs/0=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Hello,

I'm passing through a graphic card to a guest that has more than 3G of
RAM (4G to be precise in my case).

What happen is that the VM creation is stuck in the process, so I put
some tracing in the Xen code to see what
was taking the time. I discovered that the guest was stuck in
hvmloader inside this loop:

   while ( (pci_mem_start >> PAGE_SHIFT) < hvm_info->low_mem_pgend )
    {
        struct xen_add_to_physmap xatp;
        if ( hvm_info->high_mem_pgend == 0 )
            hvm_info->high_mem_pgend = 1ull << (32 - PAGE_SHIFT);
        xatp.domid = DOMID_SELF;
        xatp.space = XENMAPSPACE_gmfn;
        xatp.idx   = --hvm_info->low_mem_pgend;
        xatp.gpfn  = hvm_info->high_mem_pgend++;
        if ( hypercall_memory_op(XENMEM_add_to_physmap, &xatp) != 0 )
            BUG();
    }

This loop relocate the RAM on the top to leave so space for the PCI BARs.
It's loop on each page so in my case it's quite a big loop because the
GPU has a BAR of 256M.

So the interesting is that the function add_to_physmap takes most of
the time. I believe
that what takes most part of it is the iommu iotlb flush that come
with the iommu_map_pages
or the iommu_unmap_page which are called when we manipulate the p2m table.

In my case the iommu flush take a very long time (because of the intel
gpu ?), about 10
milliseconds. So if I'm patient enough my domain will start, about 10 minutes.

A way to go will be to create a range interface to iommu_map_page
iommu_unmap_page
since iommu_flush are so expensive. Then some work need to be done to
add a range interface
to all the function between add_to_physmap and the p2m_set_entry which
would be a big
patch. I hope there is another way out of this problem.

Thanks,
Jean

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel