Re: Xen on RP4

[Elliott Mitchell <ehem+xen@xxxxxxx>]

On Fri, Oct 23, 2020 at 04:59:30PM -0700, Stefano Stabellini wrote:
> This is what is going on. kernel/dma/swiotlb.c:swiotlb_init gets called
> and tries to allocate a buffer for the swiotlb. It does so by calling
> 
>   memblock_alloc_low(PAGE_ALIGN(bytes), PAGE_SIZE);
> 
> In your case, the allocation must fail, no_iotlb_memory is set, and I
> expect you see this warning among the boot messages:
> 
>   Cannot allocate buffer
> 
> Later during initialization swiotlb-xen comes in
> (drivers/xen/swiotlb-xen.c:xen_swiotlb_init) and given that io_tlb_start
> is != 0 it thinks the memory is ready to use when actually it is not.
> 
> When the swiotlb is actually needed, swiotlb_tbl_map_single gets called
> and since no_iotlb_memory is set the kernel panics.
> 
> 
> The reason why you are only seeing it with a 2G dom0 is because
> swiotlb_init is only called when:
> 
>   max_pfn > PFN_DOWN(arm64_dma_phys_limit ? : arm64_dma32_phys_limit))
> 
> see arch/arm64/mm/init.c:mem_init. So when dom0 is 512MB swiotlb_init is
> not called at all. swiotlb-xen does the allocation itself with
> memblock_alloc and it succeeds.
> 
> Note that I tried to repro the issue here at my end but it works for me
> with device tree. So the swiotlb_init memory allocation failure probably
> only shows on ACPI, maybe because ACPI is reserving too much low memory.
> 
> In any case, I think the issue might be "fixed" by this patch:
> 
> 
> 
> diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
> index c19379fabd20..84e15e7d3929 100644
> --- a/kernel/dma/swiotlb.c
> +++ b/kernel/dma/swiotlb.c
> @@ -231,6 +231,7 @@ int __init swiotlb_init_with_tbl(char *tlb, unsigned long 
> nslabs, int verbose)
>               io_tlb_orig_addr[i] = INVALID_PHYS_ADDR;
>       }
>       io_tlb_index = 0;
> +     no_iotlb_memory = false;
>  
>       if (verbose)
>               swiotlb_print_info();
> @@ -263,8 +264,11 @@ swiotlb_init(int verbose)
>               return;
>  
>       if (io_tlb_start)
> +     {
>               memblock_free_early(io_tlb_start,
>                                   PAGE_ALIGN(io_tlb_nslabs << IO_TLB_SHIFT));
> +             io_tlb_start = 0;
> +     }
>       pr_warn("Cannot allocate buffer");
>       no_iotlb_memory = true;
>  }
> @@ -362,6 +366,7 @@ swiotlb_late_init_with_tbl(char *tlb, unsigned long 
> nslabs)
>               io_tlb_orig_addr[i] = INVALID_PHYS_ADDR;
>       }
>       io_tlb_index = 0;
> +     no_iotlb_memory = false;
>  
>       swiotlb_print_info();

This does indeed appear to take care of the domain 0 panic.

Last issue is the framebuffer and this project has reached usability.
My impression was framebuffer was an issue for device-tree as well.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         ehem+sigmsg@xxxxxxx  PGP 87145445         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445