BUG kernel 6.12.19 defautl_swiotlb_limit() returns wrong value for CONFIG_SWIOTLB_DYNAMIC=y effects atm only under XEN dom0

[Andreas Greve <andreas.greve@xxxxxxxxxx>]

Hello to all,

please excuse my bad English and I hope this is the right List.

In Xen 4.20 kernel 6.12.19  Xen with CONFIG_SWIOTLB_DYNAMIC enabled I could not load the xhci driver for  my ASMEDIA ASM1042 usb3 controller in dom0.

it always failes with -EOI (-5) in dma_set_mask (struct device *dev, u64 mask) from kernel/dma/mapping.c when it is called with mask = DMA_BIT_MASK(32). dma_set_mask is called by xhci_gen_setup(struct usb_hcd *hcd, xhci_get_quirks_t get_quirks) from drivers/usb/host/xhci.c

The reason is that the function dma_supported(dev, mask) from mapping.c calling ops->dma_supported(dev, mask) which is xen_swiotlb_dma_supported(struct device *hwdev, u64 mask) in drivers/xen/swiotlb-xen.c failed.

The function xen_swiotlb_dma_supported fails at return xen_phys_to_dma(hwdev, default_swiotlb_limit()) <= mask ;  because
xen_phys_to_dma(hwdev, default_swiotlb_limit()) returns INVALID_P2M_ENTRY which is greater then mask (DMA_BIT_MASK(32)).

The reason is that the function default_swiottlb_limit() defined in ./kernel/dma/swiotlb.c as

/**
 * default_swiotlb_limit() - get the address limit of the default SWIOTLB
 *
 * Get the highest physical address used by the default software IO TLB pool.
 */
phys_addr_t default_swiotlb_limit(void)
{
#ifdef CONFIG_SWIOTLB_DYNAMIC
    return io_tlb_default_mem.phys_limit;
#else
    return io_tlb_default_mem.defpool.end - 1;
#endif
}

returns in case of CONFIG_SWIOTLB_DYNAMIC=y   io_tlb_default_mem.phys_limit.

io_tlb_default_mem.phys_limit points to the end (0x83effffff) of the highest Xen usable memory region which is on my system 83effffff

BIOS-provided physical RAM map:
[....]
Xen: [mem 0x0000000100001000-0x000000083effffff] usable
[....]

In the boot phase this address is always mapped in p2m to  INVALID_P2M_ENTRY. I have never seen that it is mapped to a valid machine address.

When  CONFIG_SWIOTLB_DYNAMIC  is not set the function default_swiotlb_limit() returns io_tlb_default_mem.defpool.end - 1 which maps to a value less than DAM_BIT_MASK(32).

In a non -Xen environment, the error for my system does not occur because the ASM1042 is there handled via an IOMMU path. IMO the problem described above also exists there.

Also my radeon (Radeon HD 4550 RV710) driver compiled in  coming from  radeon_device_init  over dma_set_mask fails with -EOI (-5) .

 I see 2 possible solutions:

Atm the function default_swiotlb_limit() is only used at one place in swiotlb-xen.c (fgrep over  kernel source today master branch)

1) if I take the function comment 

* Get the highest physical address used by the default software IO TLB pool.

of default_swiotlb_limit() literally I would replace return io_tlb_default_mem.phys_limit; with return io_tlb_default_mem.defpool.end - 1;

or writing a new function default_swiotlb_current_limit() with that content.

2) I would have to iterate over the list io_tlb_default_mem.pools to find the highest mapped address or writing a new function default_swiotlb_current_limit() with that content.

I would prefer 1) I because less work, less errors susceptible and more efficient. Since I do not have the necessary overview and there could be side effects, I need your help with the decision.

Thanks in advance

Andreas