[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Question about xen and Rasp 4B

Glad to hear it works! IIRC, the swiotlb may become necessary when running
guest if the guest memory ends up to be used for DMA transaction.

It is necessary if you are using PV network or PV disk: memory shared by
another domU could end up being used in a DMA transaction. For that, you
need swiotlb-xen.

Sounds bad :).

Now that swiotlb is disabled what does it mean?

I can see two reasons:
   1) You have limited memory below the 30 bits mark. So Swiotlb and CMA may
try to fight for the low memory.
   2) We found a few conversion bugs in the swiotlb on RPI4 last year (IIRC the
DMA and physical address may be different). I looked at the Linux branch you
are using and they seem to all be there. So there might be another bug.

I am not sure how to figure out where is the problem. Stefano, do you have a
suggestion where to start?

Both 1) and 2) are possible. It is also possible that another driver,
probably something related to CMA or DRM, has some special dma_ops
handling that doesn't work well together with swiotlb-xen.

Given that the original error seemed to be related to vc4_bo_create,
which calls dma_alloc_wc, I would add a couple of printks to
xen_swiotlb_alloc_coherent to help us figure it out:

diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 2b385c1b4a99..cac8b09af603 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -295,6 +295,7 @@ xen_swiotlb_alloc_coherent(struct device *hwdev, size_t 
        /* Convert the size to actually allocated. */
        size = 1UL << (order + XEN_PAGE_SHIFT);
+ printk("DEBUG %s %d size=%lu flags=%lx attr=%lx\n",__func__,__LINE__,size,flags,attrs);
        /* On ARM this function returns an ioremap'ped virtual address for
         * which virt_to_phys doesn't return the corresponding physical
         * address. In fact on ARM virt_to_phys only works for kernel direct
@@ -315,16 +316,20 @@ xen_swiotlb_alloc_coherent(struct device *hwdev, size_t 
        phys = dma_to_phys(hwdev, *dma_handle);
        dev_addr = xen_phys_to_dma(hwdev, phys);
        if (((dev_addr + size - 1 <= dma_mask)) &&
-           !range_straddles_page_boundary(phys, size))
+           !range_straddles_page_boundary(phys, size)) {
                *dma_handle = dev_addr;
-       else {
+               printk("DEBUG %s %d phys=%lx 
+       } else {
                if (xen_create_contiguous_region(phys, order,
                                                 fls64(dma_mask), dma_handle) 
!= 0) {
+                       printk("DEBUG %s %d\n",__func__,__LINE__);
                        xen_free_coherent_pages(hwdev, size, ret, 
(dma_addr_t)phys, attrs);
                        return NULL;
                *dma_handle = phys_to_dma(hwdev, *dma_handle);
+               printk("DEBUG %s %d dma_mask=%d page_boundary=%d phys=%lx 
+                       ((dev_addr + size - 1 <= 
dma_mask)),range_straddles_page_boundary(phys, size),phys,*dma_handle);
        memset(ret, 0, size);
        return ret;

Thanks I will try this.

And also can I pass the GPU to domU? Raspberry Pi 4 is limited HW and
doesn't have IOMMU. I'm trying to create similar OS like QubesOS where GPU,
Network, keyboard/mouse, ... are isolated to their own VMs.

Without an IOMMU or any other HW mechamisns (e.g. MPU), it would not be safe
to assign a DMA-capable device to a non-trusted VM.

If you trust the VM where you assigned a device, then a possible approach
would be to have the VM direct mapped (e.g. guest physical address == host
physical address). Although, I can foreese some issues if you have multiple
VMs requires memory below 30 bits (there seem to be limited amount)>

If you don't trust the VM where you assigned a device, then your best option
will be to expose a PV interface of the device and have your backend
sanitizing the request and issuing it on behalf of the guest.

FYI you could do that with the existing PVFB drivers that only support 2D
I'll keep that in my mind, thanks.



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.