Re: [Xen-devel] Xen ARM Dom0less passthrough without IOMMU

[Julien Grall <julien@xxxxxxx>]

On 16/12/2019 18:02, Andrei Cherechesu wrote:
> Hello,
Hello,

> My name is Andrei Cherechesu and I'm a Software Engineer at NXP
>
> Semiconductors in the Automotive department, Linux BSP Team.
>
> I would like to tell you have done a great job so far with Xen.
Thank you for your interest in Xen on Arm.

> Thus, we have ported and integrated Xen ARM in the Linux BSP for our
>
> boards.
>
> Currently, we are trying to passthrough devices in order to statically
>
> assign them to different DomUs at boot time (Dom0less configuration).
>
> Our boards do not have an IOMMU and so our work is only possible
>
> with the help of the patches submitted by Stefano Stabellini that
>
> implemented "xen,force-assign-without-iommu" property in the
>
> partial device trees used for passthrough.
>
> I made a configuration in order to setup the dtb /chosen
>
> node to boot a Dom0less VM in parallel with Dom0, with a
>
> peripherical device statically assigned to it.
>
> However, only if I advertise the passthrough dtb node in u-boot,
>
> when I boot Xen, I am given the following error:
>
> (XEN) *** LOADING DOMAIN 0 ***
>
> (XEN) Loading d0 kernel from boot module @ 00000000be800000
>
> (XEN) Loading ramdisk from boot module @ 00000000b1000000
>
> (XEN) Allocating 1:1 mappings totalling 512MB for dom0:
>
> (XEN) BANK[0] 0x000000d0000000-0x000000f0000000 (512MB)
>
> (XEN) Grant table range: 0x00000080200000-0x00000080240000
>
> (XEN) Allocating PPI 16 for event channel interrupt
>
> (XEN) Loading zImage from 00000000be800000 to 
> 00000000d0080000-00000000d0930008
> (XEN) Loading dom0 initrd from 00000000b1000000 to 
> 0x00000000d8200000-0x00000000e01fffff
> (XEN) Loading dom0 DTB to 0x00000000d8000000-0x00000000d800596a
>
> (XEN) Initial low memory virq threshold set at 0x4000 pages.
>
> (XEN) Scrubbing Free RAM in background
>
> (XEN) Std. Loglevel: All
>
> (XEN) Guest Loglevel: All
>
> (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
>
> (XEN) domain.c:322: IOMMU is not enabled
>
> (XEN)
>
> (XEN) ****************************************
>
> (XEN) Panic on CPU 0:
>
> (XEN) Error creating domain domU1
>
> (XEN) ****************************************
>
> My Xen is taken from Stefano’s tree 
> (http://xenbits.xenproject.org/git-http/people/sstabellini/xen-unstable.git)
> that he specified in this 
> <https://lists.xenproject.org/archives/html/xen-devel/2019-10/msg00414.html> 
> thread index, and only has the additional commits
> that add support for our boards applied by myself.
This series as been merged in the upcoming release of Xen. I would 
therefore recommend to use the upstream staging/master in case bugs may 
have been fixed there.
>  From what I've read on the mailing lists, the Dom0less Device
>
> Passthrough Without IOMMU feature is supported, at least,
>
> on boards that do have a physical IOMMU but have some devices that
>
> are not behind it.
>
> My question is: In the way that it is currently implemented,
>
> is the Dom0less Device Passthrough feature currently supported
>
> on boards that do not have an IOMMU at all?
Looking at the code, there are a few issues in current Xen that would 
prevent passthrough without IOMMU to work.
For a first, Xen is requesting to create instiate an IOMMU context for 
the domain if a partial DT is present (see create_domUs()). The 
following path should get rid of the panic above (/!\ it is a gross hack 
and not for upstream/production):
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index dd9c3b73ba..5700b992df 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -2474,9 +2474,6 @@ void __init create_domUs(void)
             panic("Missing property 'cpus' for domain %s\n",
                   dt_node_name(node));

-        if ( dt_find_compatible_node(node, NULL, "multiboot,device-tree") )
-            d_cfg.flags |= XEN_DOMCTL_CDF_iommu;
-
         if ( !dt_property_read_u32(node, "nr_spis", &d_cfg.arch.nr_spis) )
         {
             d_cfg.arch.nr_spis = gic_number_lines() - 32;

But even with this patch, RAM in DomU is not direct mapped (i.e Guest 
Physical Address == Host Physical Address). This means that DMA-capable 
device would not work properly in DomU.
We could theoritically map DomU direct mapped, but this would break the 
isolation provided by the hypervisor.
What sort of device do you plan to assign?

Best regards,

--
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel