Re: [PATCH 09/15] swiotlb: make the swiotlb_init interface more useful

[Nathan Chancellor <nathan@xxxxxxxxxx>]

On Wed, Jun 01, 2022 at 08:21:41PM +0200, Christoph Hellwig wrote:
> On Wed, Jun 01, 2022 at 11:11:57AM -0700, Nathan Chancellor wrote:
> > On Wed, Jun 01, 2022 at 07:57:43PM +0200, Christoph Hellwig wrote:
> > > On Wed, Jun 01, 2022 at 10:46:54AM -0700, Nathan Chancellor wrote:
> > > > On Wed, Jun 01, 2022 at 07:34:41PM +0200, Christoph Hellwig wrote:
> > > > > Can you send me the full dmesg and the content of
> > > > > /sys/kernel/debug/swiotlb/io_tlb_nslabs for a good and a bad boot?
> > > > 
> > > > Sure thing, they are attached! If there is anything else I can provide
> > > > or test, I am more than happy to do so.
> > > 
> > > Nothing interesting.  But the performance numbers almost look like
> > > swiotlb=force got ignored before (even if I can't explain why).
> > 
> > I was able to get my performance back with this diff but I don't know if
> > this is a hack or a proper fix in the context of the series.
> 
> This looks good, but needs a little tweak.  I'd go for this variant of
> it:

Tested-by: Nathan Chancellor <nathan@xxxxxxxxxx>

Thanks a lot for the quick fix!

> diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
> index dfa1de89dc944..cb50f8d383606 100644
> --- a/kernel/dma/swiotlb.c
> +++ b/kernel/dma/swiotlb.c
> @@ -192,7 +192,7 @@ void __init swiotlb_update_mem_attributes(void)
>  }
>  
>  static void swiotlb_init_io_tlb_mem(struct io_tlb_mem *mem, phys_addr_t 
> start,
> -                                 unsigned long nslabs, bool late_alloc)
> +             unsigned long nslabs, unsigned int flags, bool late_alloc)
>  {
>       void *vaddr = phys_to_virt(start);
>       unsigned long bytes = nslabs << IO_TLB_SHIFT, i;
> @@ -203,8 +203,7 @@ static void swiotlb_init_io_tlb_mem(struct io_tlb_mem 
> *mem, phys_addr_t start,
>       mem->index = 0;
>       mem->late_alloc = late_alloc;
>  
> -     if (swiotlb_force_bounce)
> -             mem->force_bounce = true;
> +     mem->force_bounce = swiotlb_force_bounce || (flags & SWIOTLB_FORCE);
>  
>       spin_lock_init(&mem->lock);
>       for (i = 0; i < mem->nslabs; i++) {
> @@ -275,8 +274,7 @@ void __init swiotlb_init_remap(bool addressing_limit, 
> unsigned int flags,
>               panic("%s: Failed to allocate %zu bytes align=0x%lx\n",
>                     __func__, alloc_size, PAGE_SIZE);
>  
> -     swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, false);
> -     mem->force_bounce = flags & SWIOTLB_FORCE;
> +     swiotlb_init_io_tlb_mem(mem, __pa(tlb), nslabs, flags, false);
>  
>       if (flags & SWIOTLB_VERBOSE)
>               swiotlb_print_info();
> @@ -348,7 +346,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask,
>  
>       set_memory_decrypted((unsigned long)vstart,
>                            (nslabs << IO_TLB_SHIFT) >> PAGE_SHIFT);
> -     swiotlb_init_io_tlb_mem(mem, virt_to_phys(vstart), nslabs, true);
> +     swiotlb_init_io_tlb_mem(mem, virt_to_phys(vstart), nslabs, 0, true);
>  
>       swiotlb_print_info();
>       return 0;
> @@ -835,8 +833,8 @@ static int rmem_swiotlb_device_init(struct reserved_mem 
> *rmem,
>  
>               set_memory_decrypted((unsigned long)phys_to_virt(rmem->base),
>                                    rmem->size >> PAGE_SHIFT);
> -             swiotlb_init_io_tlb_mem(mem, rmem->base, nslabs, false);
> -             mem->force_bounce = true;
> +             swiotlb_init_io_tlb_mem(mem, rmem->base, nslabs, SWIOTLB_FORCE,
> +                             false);
>               mem->for_alloc = true;
>  
>               rmem->priv = mem;
> 

Cheers,
Nathan