Re: [XEN PATCH v3 1/1] build: replace get-fields.sh by a python script

[Luca Fancellu <Luca.Fancellu@xxxxxxx>]

> On 16 Jan 2023, at 18:10, Anthony PERARD <anthony.perard@xxxxxxxxxx> wrote:
> 
> The get-fields.sh which generate all the include/compat/.xlat/*.h
> headers is quite slow. It takes for example nearly 3 seconds to
> generate platform.h on a recent machine, or 2.3 seconds for memory.h.
> 
> Rewriting the mix of shell/sed/python into a single python script make
> the generation of those file a lot faster.
> 
> No functional change, the headers generated are identical.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@xxxxxxxxxx>
> ---
> 
> Notes:
>    To test the header generation, I've submit a branch to gitlab ci,
>    where all the headers where generated, and for each one both the shell
>    script and the python script where run and the result of both
>    compared.
> 
>    v3:
>        convert to python script instead of perl
>        - this should allow more developper do be able to review/edit it.
>        - it avoid adding a dependency on perl for the hypervisor build.
> 
>        It can be twice as slow than the perl version, but overall, when doing
>        a build with make, there isn't much difference between the perl and
>        python script.
>        We might be able to speed the python script up by precompiling the
>        many regex, but it's probably not worth it. (python already have a
>        cache of compiled regex, but I think it's small, maybe 10 or so)
> 
>    v2:
>    - Add .pl extension to the perl script
>    - remove "-w" from the shebang as it is duplicate of "use warning;"
>    - Add a note in the commit message that the "headers generated are 
> identical".
> 
> xen/include/Makefile            |   6 +-
> xen/tools/compat-xlat-header.py | 468 ++++++++++++++++++++++++++++
> xen/tools/get-fields.sh         | 528 --------------------------------
> 3 files changed, 470 insertions(+), 532 deletions(-)
> create mode 100644 xen/tools/compat-xlat-header.py
> delete mode 100644 xen/tools/get-fields.sh
> 
> diff --git a/xen/include/Makefile b/xen/include/Makefile
> index 65be310eca..b950423efe 100644
> --- a/xen/include/Makefile
> +++ b/xen/include/Makefile
> @@ -60,9 +60,7 @@ cmd_compat_c = \
> 
> quiet_cmd_xlat_headers = GEN     $@
> cmd_xlat_headers = \
> -    while read what name; do \
> -        $(SHELL) $(srctree)/tools/get-fields.sh "$$what" compat_$$name $< || 
> exit $$?; \
> -    done <$(patsubst $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename 
> $<)).lst >$@.new; \
> +    $(PYTHON) $(srctree)/tools/compat-xlat-header.py $< $(patsubst 
> $(obj)/compat/%,$(obj)/compat/.xlat/%,$(basename $<)).lst > $@.new; \
>     mv -f $@.new $@
> 
> targets += $(headers-y)
> @@ -80,7 +78,7 @@ $(obj)/compat/%.c: $(src)/public/%.h $(srcdir)/xlat.lst 
> $(srctree)/tools/compat-
> $(call if_changed,compat_c)
> 
> targets += $(patsubst compat/%, compat/.xlat/%, $(headers-y))
> -$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst 
> $(srctree)/tools/get-fields.sh FORCE
> +$(obj)/compat/.xlat/%.h: $(obj)/compat/%.h $(obj)/compat/.xlat/%.lst 
> $(srctree)/tools/compat-xlat-header.py FORCE
> $(call if_changed,xlat_headers)
> 
> quiet_cmd_xlat_lst = GEN     $@
> diff --git a/xen/tools/compat-xlat-header.py b/xen/tools/compat-xlat-header.py
> new file mode 100644
> index 0000000000..c1b361ac56
> --- /dev/null
> +++ b/xen/tools/compat-xlat-header.py
> @@ -0,0 +1,468 @@
> +#!/usr/bin/env python

Would it make sense to start with python3 since it is a new script?