Re: [Xen-devel] [xen-unstable test] 4607: regressions - FAIL

["Jan Beulich" <JBeulich@xxxxxxxxxx>]

>>> On 07.01.11 at 08:16, xen.org <ian.jackson@xxxxxxxxxxxxx> wrote:
Comparing this

> flight 4607 xen-unstable real [real]
> http://www.chiark.greenend.org.uk/~xensrcts/logs/4607/ 
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking:
>  test-amd64-i386-xl           18 leak-check/check           fail REGR. vs. 
> 4586
>  test-amd64-xcpkern-i386-xl-multivcpu  5 xen-boot           fail REGR. vs. 
> 4586

with

> flight 4597 xen-unstable real [real]
> http://www.chiark.greenend.org.uk/~xensrcts/logs/4597/ 
>
> Regressions :-(
>
> Tests which did not succeed and are blocking:
>  test-i386-xcpkern-i386-pair  16 guest-start                fail REGR. vs. 
> 4586
>  test-i386-xcpkern-i386-xl    14 guest-localmigrate/x10     fail REGR. vs. 
> 4586

which both state

> version targeted for testing:
>  xen                  cb756381087c
> baseline version:
>  xen                  7b4c82f07281

hardens my suspicion that some of the tests (the leak-check/check
ones in particular based on past observation) fail and succeed
randomly. Shouldn't either tests that cause regular random false
negatives be removed from the set (or at least made non-blocking),
or regular random false positives imply that more than one run
must succeed before an automatic push?

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel