[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH 2/2] arm/efi: Use dom0less configuration when using EFI boot


  • To: Stefano Stabellini <sstabellini@xxxxxxxxxx>
  • From: Luca Fancellu <luca.fancellu@xxxxxxx>
  • Date: Thu, 16 Sep 2021 13:03:53 +0100
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=SDoawRWnyDQaw9PR09u3yHVtj17NII4KBcSYmjun2sA=; b=d6shWSm9PFoDg1bZIrLLdFj1TnClaPGWsZG5GO0VnTzpLCvQyfdbtxAKRZnhF8Szw5XQnP65Fm4C47qHjVlI19J13rNb7t+YGWJw5G2iLZQMeDVOiixaM6gKvLe6wUR3ZGPMlzNPUGnpli7UL1OO7atSNvsmxEj09kj1m9QCj0kgcQO74po+1vFgO/1BfM83QTo8pb6am6bM10MLpCx+mKtQkvOPBuswrGFDZwloYH4ewB+Abx8BsTWSlN8/vEmF2ycK/TxxEQZ7R0npghad++DK/bEwx/ojNF3X9t0+/OnfQWLWpwE4ehQ/++tg0yhe6hmGomCx3wBngt7KH1VW0w==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eRobXTIn7yD6XEHX2b7O2zqfQeGOgsDGNEdL+TqBACALwQ6GEJpP4ptzenDoWETPGD+V/rdN6zgVN8rzMYz5TQ96lZrmMoCYTZM/u+UiUV9dsKe5LjeQJdvOvEQKRXt6aGvahUnvbZRrz+Cwu9I5HquX2wSoalvwWGD/rDW6UdJ1slN7aXt7TvgrSqkokCTvDeal2DTAyC+DroJdxSl+sL05obhwk3INdmHsCB8v8lY3vSSwfynpsQW9XkwZLcjs2jwLuBJ0B/Nm8HOTF2Ed+cKLRrDVGUoF3GBWNKyxKjrboT3fVDr7LmzNbckbsYZu5h1r88akEnnY8N8BL43CyQ==
  • Authentication-results-original: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;
  • Cc: xen-devel@xxxxxxxxxxxxxxxxxxxx, Bertrand Marquis <bertrand.marquis@xxxxxxx>, wei.chen@xxxxxxx, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, George Dunlap <george.dunlap@xxxxxxxxxx>, Ian Jackson <iwj@xxxxxxxxxxxxxx>, Jan Beulich <jbeulich@xxxxxxxx>, Julien Grall <julien@xxxxxxx>, Wei Liu <wl@xxxxxxx>, Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>
  • Delivery-date: Thu, 16 Sep 2021 12:04:20 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Nodisclaimer: true
  • Original-authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=arm.com;


> On 16 Sep 2021, at 02:16, Stefano Stabellini <sstabellini@xxxxxxxxxx> wrote:
> 
> On Wed, 15 Sep 2021, Luca Fancellu wrote:
>> This patch introduces the support for dom0less configuration
>> when using UEFI boot on ARM, it permits the EFI boot to
>> continue if no dom0 kernel is specified but at least one domU
>> is found.
>> 
>> Introduce the new property "uefi,binary" for device tree boot
>> module nodes that are subnode of "xen,domain" compatible nodes.
>> The property holds a string containing the file name of the
>> binary that shall be loaded by the uefi loader from the filesystem.
>> 
>> Update efi documentation about how to start a dom0less
>> setup using UEFI
>> 
>> Signed-off-by: Luca Fancellu <luca.fancellu@xxxxxxx>
>> ---
>> docs/misc/efi.pandoc        |  37 ++++++
>> xen/arch/arm/efi/efi-boot.h | 244 +++++++++++++++++++++++++++++++++++-
>> xen/common/efi/boot.c       |  20 ++-
>> 3 files changed, 294 insertions(+), 7 deletions(-)
>> 
>> diff --git a/docs/misc/efi.pandoc b/docs/misc/efi.pandoc
>> index ac3cd58cae..db9b3273f8 100644
>> --- a/docs/misc/efi.pandoc
>> +++ b/docs/misc/efi.pandoc
>> @@ -165,3 +165,40 @@ sbsign \
>>      --output xen.signed.efi \
>>      xen.unified.efi
>> ```
>> +
>> +## UEFI boot and dom0less on ARM
>> +
>> +Dom0less feature is supported by ARM and it is possible to use it when Xen 
>> is
>> +started as an EFI application.
>> +The way to specify the domU domains is by Device Tree as specified in the
>> +[dom0less](dom0less.html) documentation page under the "Device Tree
>> +configuration" section, but instead of declaring the reg property in the 
>> boot
>> +module, the user must specify the "uefi,binary" property containing the name
>> +of the binary file that has to be loaded in memory.
>> +The UEFI stub will load the binary in memory and it will add the reg 
>> property
>> +accordingly.
>> +
>> +An example here:
>> +
>> +    domU1 {
>> +        #address-cells = <1>;
>> +        #size-cells = <1>;
>> +        compatible = "xen,domain";
>> +        memory = <0 0x20000>;
>> +        cpus = <1>;
>> +        vpl011;
>> +
>> +        module@1 {
>> +            compatible = "multiboot,kernel", "multiboot,module";
>> +            uefi,binary = "vmlinuz-3.0.31-0.4-xen";
>> +            bootargs = "console=ttyAMA0";
>> +        };
>> +        module@2 {
>> +            compatible = "multiboot,ramdisk", "multiboot,module";
>> +            uefi,binary = "initrd-3.0.31-0.4-xen";
>> +        };
>> +        module@3 {
>> +            compatible = "multiboot,ramdisk", "multiboot,module";
>> +            uefi,binary = "passthrough.dtb";
>> +        };
>> +    };
> 
> Can you please also update docs/misc/arm/device-tree/booting.txt ?
> Either a link to docs/misc/efi.pandoc or a definition of the uefi,binary
> property (mentioning that it is EFI-only.)

Yes I will update it.

> 
> 
>> diff --git a/xen/arch/arm/efi/efi-boot.h b/xen/arch/arm/efi/efi-boot.h
>> index 5ff626c6a0..8d7ced70f2 100644
>> --- a/xen/arch/arm/efi/efi-boot.h
>> +++ b/xen/arch/arm/efi/efi-boot.h
>> @@ -8,9 +8,39 @@
>> #include <asm/setup.h>
>> #include <asm/smp.h>
>> 
>> +typedef struct {
>> +    char* name;
>> +    int name_len;
>> +} dom0less_module_name;
>> +
>> +/*
>> + * Binaries will be translated into bootmodules, the maximum number for 
>> them is
>> + * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
>> + */
>> +#define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
>> +static struct file __initdata dom0less_files[MAX_DOM0LESS_MODULES];
>> +static dom0less_module_name __initdata 
>> dom0less_bin_names[MAX_DOM0LESS_MODULES];
> 
> I suggest a slightly different model where we don't call AllocatePool to
> allocate dom0less_module_name.name and instead we just set the pointer
> directly to the fdt string. There is no risk of the fdt going away at
> this point so it should be safe to use.

Yes I thought about this approach but since I was not sure how the DTB behaves 
when we modify
It to add the reg property or to modify the module name, then I used this other 
approach.
Are you sure that the pointed memory will stay the same after we modify the 
DTB? My main concern
was that the DTB structure was going to be modified and the string I was 
pointing in the DTB memory
can be relocated elsewhere. 

> 
> Also, I don't think we need a global array of struct file, we only
> really need 1 struct file which would be freed immediately after loading
> to memory. We do need to remember the address and size in memory though.
> So I would do something like:
> 
> typedef struct {
>    const char* name;
>    int name_len;
>    EFI_PHYSICAL_ADDRESS addr;
>    UINTN size;
> } dom0less_module_name;
> 
> /*
> * Binaries will be translated into bootmodules, the maximum number for them is
> * MAX_MODULES where we should remove a unit for Xen and one for Xen DTB
> */
> #define MAX_DOM0LESS_MODULES (MAX_MODULES - 2)
> static dom0less_module_name __initdata 
> dom0less_bin_names[MAX_DOM0LESS_MODULES];
> 
> 
> The purpose is to reduce memory allocations and memory consumption.

Yes I can do that.

> 
> 
>> +static uint32_t __initdata dom0less_modules_available = 
>> MAX_DOM0LESS_MODULES;
>> +static uint32_t __initdata dom0less_modules_idx = 0;
>> +
>> +#define ERROR_DOM0LESS_FILE_NOT_FOUND -1
>> +
>> void noreturn efi_xen_start(void *fdt_ptr, uint32_t fdt_size);
>> void __flush_dcache_area(const void *vaddr, unsigned long size);
>> 
>> +static int __init get_dom0less_file_index(const char* name, int name_len);
>> +static uint32_t __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
>> +                                              const char* name, int 
>> name_len);
>> +static void __init handle_dom0less_module_node(EFI_FILE_HANDLE dir_handle,
>> +                                               int module_node_offset,
>> +                                               int reg_addr_cells,
>> +                                               int reg_size_cells);
>> +static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>> +                                               int domain_node,
>> +                                               int addr_cells,
>> +                                               int size_cells);
>> +static bool __init check_dom0less_efi_boot(EFI_FILE_HANDLE dir_handle);
>> +
>> #define DEVICE_TREE_GUID \
>> {0xb1b621d5, 0xf19c, 0x41a5, {0x83, 0x0b, 0xd9, 0x15, 0x2c, 0x69, 0xaa, 
>> 0xe0}}
>> 
>> @@ -552,8 +582,209 @@ static void __init efi_arch_handle_module(const struct 
>> file *file,
>>                          kernel.size) < 0 )
>>             blexit(L"Unable to set reg property.");
>>     }
>> -    else
>> +    else if ( !((file >= &dom0less_files[0]) &&
>> +               (file <= &dom0less_files[MAX_DOM0LESS_MODULES-1])) )
>> +        /*
>> +         * If file is not a dom0 module file and it's not any domU modules,
>> +         * stop here.
>> +         */
>>         blexit(L"Unknown module type");
> 
> Without &dom0less_files we would have to do without this sanity check.

Sure, it will simplify to 
+ else if ( file != &dom0less_file )         

> 
> 
>> +    /*
>> +     * dom0less_modules_available is decremented here because for each dom0
>> +     * file added, there will be an additional bootmodule, so the number
>> +     * of dom0less module files will be decremented because there is
>> +     * a maximum amount of bootmodules that can be loaded.
>> +     */
>> +    dom0less_modules_available--;
>> +}
>> +
>> +/*
>> + * This function checks for a binary previously loaded with a give name, it
>> + * returns the index of the file in the dom0less_files array or a negative
>> + * number if no file with that name is found.
>> + */
>> +static int __init get_dom0less_file_index(const char* name, int name_len)
>> +{
>> +    int ret = ERROR_DOM0LESS_FILE_NOT_FOUND;
>> +
>> +    for (uint32_t i = 0; i < dom0less_modules_idx; i++)
> 
> uint32_t i;
> 
> for ( i = 0; i < dom0less_modules_idx; i++ )

Will fix that.

> 
> 
>> +    {
>> +        dom0less_module_name* mod = &dom0less_bin_names[i];
>> +        if ( (mod->name_len == name_len) &&
>> +             (strncmp(mod->name, name, name_len) == 0) )
>> +        {
>> +            ret = i;
>> +            break;
>> +        }
>> +    }
>> +    return ret;
>> +}
>> +
>> +/*
>> + * This function allocates a binary and keeps track of its name, it
>> + * returns the index of the file in the dom0less_files array.
>> + */
>> +static uint32_t __init allocate_dom0less_file(EFI_FILE_HANDLE dir_handle,
>> +                                              const char* name, int 
>> name_len)
>> +{
>> +    dom0less_module_name* file_name;
>> +    union string module_name;
>> +    struct file* file;
>> +    uint32_t ret_idx;
>> +
>> +    /*
>> +     * Check if there is any space left for a domU module, the variable
>> +     * dom0less_modules_available is updated each time we use read_file(...)
>> +     * successfully.
>> +     */
>> +    if ( !dom0less_modules_available )
>> +        blexit(L"No space left for domU modules");
>> +    module_name.s = (char*) name;
>> +    ret_idx = dom0less_modules_idx;
>> +    file = &dom0less_files[ret_idx];
>> +
>> +    /* Save at this index the name of this binary */
>> +    file_name = &dom0less_bin_names[ret_idx];
>> +
>> +    if ( efi_bs->AllocatePool(EfiLoaderData, (name_len + 1) * sizeof(char),
>> +                              (void**)&file_name->name) != EFI_SUCCESS )
>> +        blexit(L"Error allocating memory for dom0less binary name");
> 
> As far as I can tell we could just set file_name = name;

If you are sure I will modify that, I will wait your confirmation.

> 
> 
>> +    /* Save name and length of the binary in the data structure */
>> +    strlcpy(file_name->name, name, name_len);
>> +    file_name->name_len = name_len;
>> +
>> +    /* Load the binary in memory */
>> +    read_file(dir_handle, s2w(&module_name), file, NULL);
>> +
>> +    /* s2w(...) allocates some memory, free it */
>> +    efi_bs->FreePool(module_name.w);
>> +
>> +    dom0less_modules_idx++;
>> +
>> +    return ret_idx;
>> +}
>> +
>> +/*
>> + * This function checks for the presence of the uefi,binary property in the
>> + * module, if found it loads the binary as dom0less module and sets the 
>> right
>> + * address for the reg property into the module DT node.
>> + */
>> +static void __init handle_dom0less_module_node(EFI_FILE_HANDLE dir_handle,
>> +                                          int module_node_offset,
>> +                                          int reg_addr_cells,
>> +                                          int reg_size_cells)
>> +{
>> +    const void* uefi_name_prop;
>> +    char mod_string[24]; /* Placeholder for module@ + a 64-bit number + \0 
>> */
>> +    int uefi_name_len, file_idx;
>> +    struct file* file;
>> +
>> +    /* Read uefi,binary property to get the file name. */
>> +    uefi_name_prop = fdt_getprop(fdt, module_node_offset, "uefi,binary",
>> +                                 &uefi_name_len);
>> +
>> +    if ( NULL == uefi_name_prop )
>> +        /* Property not found */
>> +        return;
>> +
>> +    file_idx = get_dom0less_file_index(uefi_name_prop, uefi_name_len);
>> +    if (file_idx < 0)
>> +        file_idx = allocate_dom0less_file(dir_handle, uefi_name_prop,
>> +                                          uefi_name_len);
>> +
>> +    file = &dom0less_files[file_idx];
>> +
>> +    snprintf(mod_string, sizeof(mod_string), "module@%"PRIx64, file->addr);
>> +
>> +    /* Rename the module to be module@{address} */
>> +    if ( fdt_set_name(fdt, module_node_offset, mod_string) < 0 )
>> +        blexit(L"Unable to add domU ramdisk FDT node.");
>> +
>> +    if ( fdt_set_reg(fdt, module_node_offset, reg_addr_cells, 
>> reg_size_cells,
>> +                     file->addr, file->size) < 0 )
>> +        blexit(L"Unable to set reg property.");
>> +}
>> +
>> +/*
>> + * This function checks for boot modules under the domU guest domain node
>> + * in the DT.
>> + */
>> +static void __init handle_dom0less_domain_node(EFI_FILE_HANDLE dir_handle,
>> +                                               int domain_node,
>> +                                               int addr_cells,
>> +                                               int size_cells)
>> +{
>> +    /*
>> +     * Check for nodes compatible with 
>> multiboot,{kernel,ramdisk,device-tree}
>> +     * inside this node
>> +     */
>> +    for ( int module_node = fdt_first_subnode(fdt, domain_node);
> 
> int module_node;
> 
> for ( module_node = fdt_first_subnode(fdt, domain_node);
> 

Will fix that.

> 
>> +          module_node > 0;
>> +          module_node = fdt_next_subnode(fdt, module_node) )
>> +    {
>> +        if ( (fdt_node_check_compatible(fdt, module_node,
>> +                                        "multiboot,kernel") == 0) ||
>> +             (fdt_node_check_compatible(fdt, module_node,
>> +                                        "multiboot,ramdisk") == 0) ||
>> +             (fdt_node_check_compatible(fdt, module_node,
>> +                                        "multiboot,device-tree") == 0) )
>> +        {
>> +            /* The compatible is one of the strings above, check the module 
>> */
>> +            handle_dom0less_module_node(dir_handle, module_node, addr_cells,
>> +                                        size_cells);
>> +        }
>> +    }
>> +}
>> +
>> +/*
>> + * This function checks for xen domain nodes under the /chosen node for 
>> possible
>> + * domU guests to be loaded.
>> + */
>> +static bool __init check_dom0less_efi_boot(EFI_FILE_HANDLE dir_handle)
>> +{
>> +    int chosen;
>> +    int addr_len, size_len;
>> +
>> +    /* Check for the chosen node in the current DTB */
>> +    chosen = setup_chosen_node(fdt, &addr_len, &size_len);
>> +    if ( chosen < 0 )
>> +        blexit(L"Unable to setup chosen node");
>> +
>> +    /* Check for nodes compatible with xen,domain under the chosen node */
>> +    for ( int node = fdt_first_subnode(fdt, chosen);
>> +          node > 0;
>> +          node = fdt_next_subnode(fdt, node) )
>> +    {
>> +        int addr_cells, size_cells, len;
>> +        const struct fdt_property *prop;
>> +
>> +        if ( fdt_node_check_compatible(fdt, node, "xen,domain") != 0 )
>> +            continue;
>> +
>> +        /* Get or set #address-cells and #size-cells */
>> +        prop = fdt_get_property(fdt, node, "#address-cells", &len);
>> +        if ( !prop )
>> +            blexit(L"#address-cells not found in domain node.");
>> +
>> +        addr_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
>> +
>> +        prop = fdt_get_property(fdt, node, "#size-cells", &len);
>> +        if ( !prop )
>> +            blexit(L"#size-cells not found in domain node.");
>> +
>> +        size_cells = fdt32_to_cpu(*((uint32_t *)prop->data));
>> +
>> +        /* Found a node with compatible xen,domain; handle this node. */
>> +        handle_dom0less_domain_node(dir_handle, node, addr_cells, 
>> size_cells);
>> +    }
>> +
>> +    if ( dom0less_modules_idx > 0 )
>> +        return true;
>> +
>> +    return false;
>> }
>> 
>> static void __init efi_arch_cpu(void)
>> @@ -562,8 +793,19 @@ static void __init efi_arch_cpu(void)
>> 
>> static void __init efi_arch_blexit(void)
>> {
>> +    uint32_t i = 0;
>>     if ( dtbfile.need_to_free )
>>         efi_bs->FreePages(dtbfile.addr, PFN_UP(dtbfile.size));
>> +    /* Free dom0less files if any */
>> +    for ( ; i < dom0less_modules_idx; i++ )
>> +    {
>> +        /* Free dom0less binary names */
>> +        efi_bs->FreePool(dom0less_bin_names[i].name);
>> +        /* Free dom0less binaries */
>> +        if ( dom0less_files[i].need_to_free )
>> +            efi_bs->FreePages(dom0less_files[i].addr,
>> +                              PFN_UP(dom0less_files[i].size));
>> +    }
>>     if ( memmap )
>>         efi_bs->FreePool(memmap);
>> }
>> diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c
>> index 758f9d74d2..65493c4b46 100644
>> --- a/xen/common/efi/boot.c
>> +++ b/xen/common/efi/boot.c
>> @@ -1134,8 +1134,9 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE 
>> *SystemTable)
>>     EFI_GRAPHICS_OUTPUT_PROTOCOL *gop = NULL;
>>     union string section = { NULL }, name;
>>     bool base_video = false;
>> -    const char *option_str;
>> +    const char *option_str = NULL;
>>     bool use_cfg_file;
>> +    bool dom0less_found = false;
>> 
>>     __set_bit(EFI_BOOT, &efi_flags);
>>     __set_bit(EFI_LOADER, &efi_flags);
>> @@ -1285,14 +1286,21 @@ efi_start(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE 
>> *SystemTable)
>>             efi_bs->FreePool(name.w);
>>         }
>> 
>> -        if ( !name.s )
>> -            blexit(L"No Dom0 kernel image specified.");
>> -
>>         efi_arch_cfg_file_early(loaded_image, dir_handle, section.s);
>> 
>> -        option_str = split_string(name.s);
>> +#ifdef CONFIG_ARM
>> +        /* dom0less feature is supported only on ARM */
>> +        dom0less_found = check_dom0less_efi_boot(dir_handle);
>> +#endif
> 
> Rather than an #ifdef here you can simply implement
> check_dom0less_efi_boot on x86 as a static inline returning always
> false.

Sure I will create that on x86 code and I will update the code here.

> 
> Also, we are under the if ( use_cfg_file ) code path. So maybe it is
> reasonable that dom0 is required if we are booting with use_cfg_file
> = true. After all, it is specified as a required property today of
> xen.cfg.
> 
> If you follow my suggestion with an explicit enabled/disabled of xen.cfg
> from device tree, a true dom0less configuration could be fully specified
> without xen.cfg.
> 
> If we do that, then here probable we don't need to change this code path.
> 

Please check my reply on the previous patch.

Cheers,

Luca

> 
> 
>> +        if ( !name.s && !dom0less_found )
>> +            blexit(L"No Dom0 kernel image specified.");
>> +
>> +        if ( name.s != NULL )
>> +            option_str = split_string(name.s);
>> 
>> -        if ( !read_section(loaded_image, L"kernel", &kernel, option_str) )
>> +        if ( (!read_section(loaded_image, L"kernel", &kernel, option_str)) 
>> &&
>> +             (name.s != NULL) )
>>         {
>>             read_file(dir_handle, s2w(&name), &kernel, option_str);
>>             efi_bs->FreePool(name.w);




 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.