Re: [Xen-devel] [linux-4.1 test] 60785: tolerable FAIL - PUSHED

[Ian Jackson <Ian.Jackson@xxxxxxxxxxxxx>]

Ian Campbell writes ("Re: [linux-4.1 test] 60785: tolerable FAIL - PUSHED"):
> 60030 is the "bad" push which uses "fail like NNNNNN-bisect" to justify
> ignoring all three.

For example, you mean the line in 60030's report saying

  test-amd64-i386-xl   13 guest-saverestore   fail like 60094-bisect

Well, 60094's report is contained within

  Date: Tue, 28 Jul 2015 21:44:28 GMT
  Subject: "[linux-4.1 bisection] complete test-amd64-i386-xl

and says

  test-amd64-i386-xl   13 guest-saverestore   fail baseline untested

There was no build job in 60094; for Linux (build-i386-pvops) it used
59826 (ie 5cf9896d) and for Xen (build-i386) 59934 (xen.git#3a9ace01).

> IIRC there was a bug in the osstest machinery (now fixed by Ian) which lead
> to this, but I mention it in case I'm recalling incorrectly.

I think that actually this is another problem: sg-report-flight when
testing X' (with a baseline of X) can justify a failure of T(X',Y,Z)
with a bisection failure of T(X,Y'',Z).

If Y'' breaks T then this makes it look to sg-report-flight like T was
already broken in X.

The simple solution is for sg-report-flight for cr-daily-branch to to
look at bisections.

The more complicated one would be for sg-report-flight to compare
versions of Y when looking for justifications, but I'm not sure this
is desirable because it would reset the justification search each time
any other tree changed.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel