[Xen-devel] [OSSTEST PATCH 4/5] cs-bisection-step: Handle build job chains

[Ian Jackson <ian.jackson@xxxxxxxxxxxxx>]

cs-bisection-step assumes that every job it will need to create has a
unique name.  Indeed, in general, it is probably not reasonable to
expect it to work if that is not the case.

build-amd64-freebsd needs a previous build-amd64-freebsd.

Currently cs-bisection-step confuses itself into writing a flight
where build-amd64-freebsd uses itself as its freebsdbuildjob.
This naturally does not work very well.

I think the right approach is for cs-bisection-step to spot when it
its recursion through the jobs, via *job runvars, would descend into a
job name which it was already encoutered earlier in the dependency
chain, and, when that occurs, to simply skip regenerating that deeper
copy of the job.

That is achieved here by filtering the subjob out of the list, before
we go looking for jobs to reuse.  As a result, not only will no new
job be created, but the original deeper job will be reused because the
runvar value will not be updated.

FTR, the circular dependency produces this error from sg-execute-flight:

  wait for process failed: no children
      while executing
  "wait -nohang"
      (procedure "main_iteration" line 14)
      invoked from within
  "main_iteration"
      (procedure "main" line 8)
      invoked from within
  "main"
      (file "./sg-execute-flight" line 238)

This is because sg-execute-flight's algorithm assumes that if there
are no jobs running there must be some job whose dependencies are done.
That is true if the job graph has no cycles.

Improving the error message from sg-execute-flight is left as an
exercise for the future.

Signed-off-by: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
---
 cs-bisection-step | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/cs-bisection-step b/cs-bisection-step
index 01ee6e5..a7e0336 100755
--- a/cs-bisection-step
+++ b/cs-bisection-step
@@ -1133,6 +1133,7 @@ END
 
 our %jobs_created;
 our %builds_investigated; # $builds_investigated{$popjob} = 0, or {..row..}
+our %recursion_track;
 
 sub preparejob ($$$);
 sub preparejob ($$$) {
@@ -1144,6 +1145,11 @@ sub preparejob ($$$) {
        return $jobs_created{$popjob};
     }
 
+    local $recursion_track{$popjob} = {
+        Depth => (1 + keys %recursion_track),
+        Spec => "$copyflight.$popjob"
+    };
+
     print STDERR "Need $popflight.$popjob (like $copyflight)\n";
 
     # Create a temporary table containing the runvars we want
@@ -1208,6 +1214,21 @@ END
         }
     }
 
+    @$subjobs = grep {
+        if ($recursion_track{ $_->{job} }) {
+            print STDERR "Not recursively creating another $_->{job} (".
+                (
+                 join " -> ",
+                 map { $_->{Spec} }
+                 sort { $a->{Depth} <=> $b->{Depth} }
+                 values %recursion_track
+                ). " -> $_->{orgflight}.$_->{job}\n";
+            0;
+        } else {
+            1;
+        }
+    } @$subjobs;
+
     # See if there's a build we can reuse
 
     my ($recipe) = $dbh_tests->selectrow_array(<<END,{}, $copyflight,$popjob);
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel