[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] How does the regression testing stuff work?

Alex Bligh writes ("[Xen-devel] How does the regression testing stuff work?"):
> So this came in on 5th April (latest example of a regression
> message for qemu-upstream-4.2-testing). I would take this to
> mean the changesets applied (the two listed below) caused
> a regression in the test guest-localmigrate/x10 (whatever
> that is).

Yes.  Although the apparent regression appeared once in this one test,
so if the tree is unreliable (sadly it seems to be) it could be a

> However, the two commits did get pushed from staging to the
> main repos, with two subsequent 'tolerable FAIL - pushed' messages.


> I'm not the author of the two commits in question, but if I
> was I'd want to know what test actually failed and why. The logs
> weren't vastly illuminating without knowing what the test does.
> I'd also be keen to know whether I should be investigating this
> or whether waiting a couple of days might result in a retest
> that appears to pass the code.

If you visit this url:

> flight 17485 qemu-upstream-4.2-testing real [real]
> http://www.chiark.greenend.org.uk/~xensrcts/logs/17485/

You'll see a big grid.

> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  test-amd64-amd64-xl-qemuu-winxpsp3 12 guest-localmigrate/x10 fail REGR.
> vs. 17143

Look at the column for the failure.  You can see it because of the
red/pink REGR entry in the grid, or you can look at the column

Click on the column heading.

This will show you a lot of logs.  The one which is most relevant is
the test script log, which is linked to from "fail" in the Steps table


So you can see there that the tester ran the 9th repetition of

  ssh ... root@xxxxxxxxxxxx xl migrate win.guest.osstest localhost

and that appeared to work in the sense that it produced sane-looking
output, but it didn't exit.  So 400s later the tester says this:

 ssh -o StrictHostKeyChecking=no -o BatchMode=yes -o
 ConnectTimeout=100 -o ServerAliveInterval=100 -o
 PasswordAuthentication=no -o ChallengeResponseAuthentication=no -o
 root@xxxxxxxxxxxx xl migrate win.guest.osstest localhost
 status (timed out) at Osstest/TestSupport.pm line 375.

I don't know why this didn't work, but if it passed later it must be a
race of some kind.  It therefore may be the case that the changes
being tested were not in fact responsible.


Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.