Re: [Xen-devel] PCI DMA Limitations

["Stephen Donnelly" <sfdonnelly@xxxxxxxxx>]

On 3/26/07, Jan Beulich <jbeulich@xxxxxxxxxx> wrote:
>>> "Stephen Donnelly" <
sfdonnelly@xxxxxxxxx> 26.03.07 05:35 >>>

>Coherent mappings via dma_alloc_coherent exchange VM pages for contiguous
>low hypervisor pages. The allocation size is limited by MAX_CONTIG_ORDER = 9
>in xen_create_contiguous_region to 2^9 * 4k = 2MB?

Yes, though for very special cases (AGP aperture) extending this limit is
being considered, though not likely by just bumping MAX_CONTIG_ORDER
(due to the effect this would have on static variables' sizes).

>Is it possible to increase MAX_CONTIG_ORDER in a guest OS unilaterally, or
>is there a matching limitation in the hypervisor? I didn't see any options
>to Xen to configure the amount of memory reserved for coherent DMA mappings.

Again, Xen doesn't limit the order significantly, and statically bumping
MAX_CONTIG_ORDER doesn't seem like too good an idea.
Xen can reserve memory for DMA purposes via the dma_emergency_pool
command line option.

Would it be possible to implement something less expensive, but with larger than page granularity, perhaps similar to HugeTLB? This could keep the static state requirements down while allowing larger regions to be used.

>Is there a simpler/more direct way to provide DMA access to large buffers in
>guest VMs? I was curious about how RDMA cards (e.g. Infiniband) are
>supported, are they required to use DAC and scatter-gather in some way?

Yes, s/g is certainly very much preferable here, due to the possibly huge
amounts of data otherwise needing copying. Also, RDMA is (hopefully) not
restricted to 32-bit machine addresses, as that would be another reason
to force it though the swiotlb.

My device really wants large (.5-2 GB) contiguous bus address ranges with a 32-bit DMA mask limitation, and zero copy access from VM user spaces. This doesn't seem to be possible with the current architecture.

If the 32-bit DMA mask limitation was lifted (allowing full 64-bit addressing), does this help? The VMs would still need to be able to allocate and map large physically contiguous address regions somehow?

Regards,
Stephen.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel