Re: [Xen-devel] [PATCH V1] xen/arm: domain_build: introduce dom0_lowmem bootargs

[Peng Fan <van.freenix@xxxxxxxxx>]

Hi Edgar,
On Thu, Sep 15, 2016 at 10:26:46AM +0200, Edgar E. Iglesias wrote:
>On Thu, Sep 15, 2016 at 08:20:33AM +0800, Peng Fan wrote:
>> Hi Edgar,
>> On Wed, Sep 14, 2016 at 04:16:58PM +0200, Edgar E. Iglesias wrote:
>> >On Wed, Sep 14, 2016 at 08:40:09PM +0800, Peng Fan wrote:
>> >> On Wed, Sep 14, 2016 at 01:34:10PM +0100, Julien Grall wrote:
>> >> >
>> >> >
>> >> >On 14/09/16 13:18, Peng Fan wrote:
>> >> >>Hello Julien,
>> >> >>
>> >> >>On Wed, Sep 14, 2016 at 01:06:01PM +0100, Julien Grall wrote:
>> >> >>>
>> >> >>>
>> >> >>>On 14/09/16 13:03, Peng Fan wrote:
>> >> >>>>Hello Julien,
>> >> >>>
>> >> >>>Hello Peng,
>> >> >>>
>> >> >>>>On Wed, Sep 14, 2016 at 11:47:10AM +0100, Julien Grall wrote:
>> >> >>>>>Hello,
>> >> >>>>>
>> >> >>>>>On 14/09/16 08:41, Peng Fan wrote:
>> >> >>>>>>On Wed, Sep 14, 2016 at 08:23:24AM +0100, Julien Grall wrote:
>> >> >>>>>>diff --git a/xen/arch/arm/domain_build.c 
>> >> >>>>>>b/xen/arch/arm/domain_build.c
>> >> >>>>>>index 35ab08d..cc71e6f 100644
>> >> >>>>>>--- a/xen/arch/arm/domain_build.c
>> >> >>>>>>+++ b/xen/arch/arm/domain_build.c
>> >> >>>>>>@@ -28,6 +28,8 @@
>> >> >>>>>>
>> >> >>>>>>static unsigned int __initdata opt_dom0_max_vcpus;
>> >> >>>>>>integer_param("dom0_max_vcpus", opt_dom0_max_vcpus);
>> >> >>>>>>+static bool_t __initdata opt_dom0_use_lowmem;
>> >> >>>>>>+boolean_param("dom0_use_lowmem", opt_dom0_use_lowmem);
>> >> >>>>>>
>> >> >>>>>>int dom0_11_mapping = 1;
>> >> >>>>>>
>> >> >>>>>>@@ -244,7 +246,7 @@ static void allocate_memory(struct domain *d, 
>> >> >>>>>>struct
>> >> >>>>>>kernel_info *kinfo)
>> >> >>>>>>   unsigned int order = 
>> >> >>>>>> get_11_allocation_size(kinfo->unassigned_mem);
>> >> >>>>>>   int i;
>> >> >>>>>>    
>> >> >>>>>>-    bool_t lowmem = is_32bit_domain(d);
>> >> >>>>>>+    bool_t lowmem = is_32bit_domain(d) || opt_dom0_use_lowmem;
>> >> >>>>>>   unsigned int bits;
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>Pass "dom0_use_lowmem=1" to xen to allocate lowmem as much as 
>> >> >>>>>>possible.
>> >> >>>>>
>> >> >>>>>Again, what is the benefit to have a command line option for that?
>> >> >>>>
>> >> >>>>Then you prefer directly change "bool_t lowmem = is_32bit_domain(d);" 
>> >> >>>>to "bool_t lowmem = true" ?
>> >> >>>>I just want to give user a choice.
>> >> >>>
>> >> >>>We don't add new command line parameter just because they look cool to 
>> >> >>>have.
>> >> >>>So far, you did not explain why it would be good to let the choice to 
>> >> >>>the
>> >> >>>user and how it could be used.
>> >> >>
>> >> >>I have not try, if there is no lowmem.
>> >> >>
>> >> >>I have not look into alloc_domheap_pages.
>> >> >>I am not sure whether there is such a platform or not,
>> >> >>just thinking if there is soc that dram memory starts from 4GB, and 
>> >> >>there is no dram
>> >> >>below 4GB. If we still can get memory when lowmem is true, I am ok to 
>> >> >>change directly assign
>> >> >>lowmem with value true. Anyway I have not look into the internals of 
>> >> >>domheap and
>> >> >>not sure whether there is such a platform that no lowmem (:-
>> >> >
>> >> >We cannot exclude this possibility. However, the only reason that Xen is
>> >> >requiring to allocate a bank below 4GB for 32-bit domain is to handle
>> >> >non-LPAE kernel.
>> >> 
>> >> Now also need to handle device that have DMA limitation -:)
>> >
>> >Hi Peng,
>> >
>> >Doesn't your platform have an IOMMU/SMMU?
>> 
>> We have SMMU. This is not related to SMMU. Dom0 use 1:1 mapping and no SMMU 
>> involved,
>> the physical memory assigned to Dom0 maybe higher than 4GB, but Some IPs only
>> supports 32bits DMA in Dom0. Then assign a 64bits dma address to a device 
>> only supports 32
>> bits device in Linux will hang the device or else.
>
>Well, I think it is somewhat related to the IOMMU.
>
>If your SMMU supports S1 + S2 translations, allthough not supported
>by Xen/ARM today, we could support nested SMMU's so that Dom0 could
>get it's own private S1 portion of the SMMU.

For DomU or hardware domain, I think supporting SMMU with Linux S1 + XEN S2 is 
a good feature to have.
For Dom0, I think no need to use S1 + S2 for SMMU, It is control domain.

>
>Another option is to perhaps join into the efforts of PV-IOMMU
>and try to see if it would work for Xen on ARM:
>https://lists.xen.org/archives/html/xen-devel/2016-02/msg01428.html

This is new to me -:) Will get back to you when I know more about this.

>
>For platforms that have an IOMMU, I think both of these options may
>solve the use-case of dom0 using 32bit DMA devs without any lowmem.
>In addition to that, these features would be very nice as they also
>enable DMA API isolation and VFIO user-space drivers in dom0.
>Spending time on these kind of options seems worthwhile to me.

If SMMU S1 is supported, we could use VFIO. This's good feature to have.
If you have any progress or new update, please kindly CC me. I am happy
to see this.

>
>With 32bit DMA devs, without an IOMMU, lowmem becomes critical but
>such systems are not really secure as has already been mentioned.
>I'm not sure it's worth introducing workarounds/hax for such
>systems.

The proposed two options may cost big efforts I think -:)
I prefer Julien's suggestion to fix the issue I met, since easy to implement.
Anyway from the long run, your two options are good feature to have.

Thanks,
Peng.

>
>Cheers,
>Edgar

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel