[RFC PATCH 0/5] Fun with git

[Elliott Mitchell <ehem+xen@xxxxxxx>]

I ended up looking at the use of git in the build system.  There is some
inconsistency in use/behavior and I really dislike how certain pieces of
it have evolved.  Most of what I'm sending is RFC since I've got too many
questions, but the first two are appropriate right now.

I find it unsurprising the script predates Xen's conversion to git.
Likely the script made it rather more obvious git was winning, then it
finally came time to officially convert Xen to using git.

Of the four uses of the `git-checkout.sh` script 1 gives a full path and
3 give only the subdirectory.  I'm unsure which is appropriate, but
inconsistency is a Bad Thing(tm).

What is with the naming of the subproject repository directories?  Why
the "-dir" for "qemu-xen-dir", "ovmf-dir" and "seabios-dir"?  Whereas
for mini-os it is "extras/mini-os".  I imagine there was some good
reason at some point, but now it simply seems odd.

Then there is the issue of all the duplicated project subdirectory
variants.  If the `git-checkout.sh` script is used it creates
<repo>-remote, than a symbolic link <repo> -> <repo>-remote.  Then for
most (all?) of these the directory <repo>-build is created where the
actual build is done.


Then there are the problems with factors related to the existence of the
script and the way it works.

One problem with the script is is breaks cleaning the repository.
"Hmm, `git status --ignored` still shows directories, okay let me
`rm -rf tools/firmware/*-dir tools/qemu-xen-dir extras/mini-os`"   Then
the next build you end up redownloading those repositories from scratch.
Perhaps the Xen Project's git server can handle the load, but it
certainly adds some.

Then there is a major slowdown in the build process.  Clone the git
repository, then move to your fast build machine with lots of processors.
Suddenly you're waiting on your Internet link.  Then it gets worse if
that is the moment when the git server is down.  Just to make it worse
this also functions as a tracker for finding people building Xen.

Yet another issue is, is it still possible to build Xen 4.0?  Nominally
most/all of the repository histories reach back that far, but you'll have
no idea which versions were in use.  The combination of outside
repository versions needed for a given Xen version build might be quite
hard to figure out in the future.


I'm including some patches I created while looking into this situation.
I think the first is worthwhile no matter what choices are made.  The
rest are simply resolving issues I found.  The last is a proposed
adjustment to the script.  The script adjustment is unfinished, it is
somehow breaking Qemu's build process.  It also only successfully
addresses the issue of needing to redownload full repositories.

A better option might be using `git subtree`.  This would address all my
concerns in regards to the handling of external project repositories.
Does seem a largish step though.


Elliott Mitchell (5):
  make: add use of $(realpath ) in top and main Rules.mk files
  Config.mk: record hash of SeaBIOS tag
  scripts/git-checkout.sh: add standard scripting safety steps
  tools/build: use full path when checking out git repositories
  scripts/git-checkout.sh: store repositories with Xen, use worktrees

 Config.mk               |  3 ++-
 Makefile                | 12 +++++++----
 scripts/git-checkout.sh | 44 ++++++++++++++++++++++++-----------------
 tools/Makefile          | 12 ++++-------
 tools/Rules.mk          |  2 ++
 tools/firmware/Makefile | 14 ++++++-------
 6 files changed, 48 insertions(+), 39 deletions(-)

-- 
2.47.3