Re: Troubles running Xen on Raspberry Pi 4 with 5.6.1 DomU

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Wed, 6 May 2020, Nataliya Korovkina wrote:
> On Wed, May 6, 2020 at 9:43 AM Boris Ostrovsky
> <boris.ostrovsky@xxxxxxxxxx> wrote:
> >
> >
> > On 5/6/20 9:08 AM, Nataliya Korovkina wrote:
> > > Hello,
> > >
> > > What I found out: rpi_firmware_property_list() allocates memory from
> > > dma_atomic_pool which was mapped to VMALLOC region, so virt_to_page()
> > > is not eligible in this case.
> >
> >
> > So then it seems it didn't go through xen_swiotlb_alloc_coherent(). In
> > which case it has no business calling xen_swiotlb_free_coherent().
> >
> >
> > -boris
> >
> >
> >
> >
> 
> It does go.
> dma_alloc_coherent() indirectly calls xen_swiotlb_alloc_coherent(),
> then  xen_alloc_coherent_pages() eventually calls arch_dma_alloc() in
> remap.c which successfully allocates pages from atomic pool.
> 
> The patch Julien offered for domian_build.c moved Dom0 banks in the
> first G of RAM.
> So it covered the previous symptom (a crash during allocation) because
> now we avoid pathway  when we mark a page "XenMapped".
> 
> But the symptom still remains in xen_swiotlb_free_coherent() because
> "TestPage..." is called unconditionally. virt_to_page() is not
> applicable to such allocations.

Ah! So this is the crux of the issue. I saw this kind of problem before
on ARM32 (in fact there are several comments warning not to use
virt_to_phys on ARM in drivers/xen/swiotlb-xen.c).


So, to recap we have 2 issues as far as I can tell:

- virt_to_page not working in some cases on ARM, leading to a crash
- WARN_ON for range_straddles_page_boundary which is normal on ARM

The appended patch addresses them by:

- using pfn_to_page instead virt_to_page
- moving the WARN_ON under a #ifdef (Juergen might have a better
  suggestion on how to rework the WARN_ON)

Please let me know if this patch works!

Cheers,

Stefano


diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index b6d27762c6f8..0a40ac332a4c 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -322,7 +322,7 @@ xen_swiotlb_alloc_coherent(struct device *hwdev, size_t 
size,
                        xen_free_coherent_pages(hwdev, size, ret, 
(dma_addr_t)phys, attrs);
                        return NULL;
                }
-               SetPageXenRemapped(virt_to_page(ret));
+               SetPageXenRemapped(pfn_to_page(PFN_DOWN(phys)));
        }
        memset(ret, 0, size);
        return ret;
@@ -346,9 +346,14 @@ xen_swiotlb_free_coherent(struct device *hwdev, size_t 
size, void *vaddr,
        /* Convert the size to actually allocated. */
        size = 1UL << (order + XEN_PAGE_SHIFT);
 
-       if (!WARN_ON((dev_addr + size - 1 > dma_mask) ||
-                    range_straddles_page_boundary(phys, size)) &&
-           TestClearPageXenRemapped(virt_to_page(vaddr)))
+#ifdef CONFIG_X86
+       if (WARN_ON(dev_addr + size - 1 > dma_mask) ||
+                   range_straddles_page_boundary(phys, size)) {
+           return;
+       }
+#endif
+
+       if (TestClearPageXenRemapped(pfn_to_page(PFN_DOWN(phys))))
                xen_destroy_contiguous_region(phys, order);
 
        xen_free_coherent_pages(hwdev, size, vaddr, (dma_addr_t)phys, attrs);