[Xen-devel] How does the regression testing stuff work?

[Alex Bligh <alex@xxxxxxxxxxx>]

I've been trying to figure out how the automated regression testing
stuff works, partly because I'm interested in whether the patches
I submitted broke anything and what I should be looking for.

So this came in on 5th April (latest example of a regression
message for qemu-upstream-4.2-testing). I would take this to
mean the changesets applied (the two listed below) caused
a regression in the test guest-localmigrate/x10 (whatever
that is).

However, the two commits did get pushed from staging to the
main repos, with two subsequent 'tolerable FAIL - pushed' messages.

I'm not the author of the two commits in question, but if I
was I'd want to know what test actually failed and why. The logs
weren't vastly illuminating without knowing what the test does.
I'd also be keen to know whether I should be investigating this
or whether waiting a couple of days might result in a retest
that appears to pass the code.

Is there a newbie guide for this?

Alex


---------- Forwarded Message ----------
Date: 5 April 2013 23:03:10 +0100
From: "xen.org" <ian.jackson@xxxxxxxxxxxxx>
To: xen-devel@xxxxxxxxxxxxxxxxxxx
CC: ian.jackson@xxxxxxxxxxxxx
Subject: [Xen-devel] [qemu-upstream-4.2-testing test] 17485: regressions - 
	FAIL
flight 17485 qemu-upstream-4.2-testing real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/17485/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
test-amd64-amd64-xl-qemuu-winxpsp3 12 guest-localmigrate/x10 fail REGR.
vs. 17143

Tests which did not succeed, but are not blocking:
build-armhf                   4 xen-build                    fail   never
pass  test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 13 guest-stop         fail
never pass  test-amd64-amd64-xl-qemuu-win7-amd64 13 guest-stop
fail never pass  test-amd64-i386-xl-qemuu-win7-amd64 13 guest-stop
fail never pass  test-i386-i386-xl-qemuu-winxpsp3 13 guest-stop
fail never pass  test-amd64-i386-xend-qemuu-winxpsp3 16 leak-check/check
fail never pass

version targeted for testing:
qemuu                bf8e2e756e3e98b8388068173e24d99d1e09d2dd
baseline version:
qemuu                351f94ff4bf3a7795ca5b282305aa610e598eec0

------------------------------------------------------------
People who touched revisions under test:
 Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>
 Weidong Han <hanweidong@xxxxxxxxxx>
 Zhenguo Wang <wangzhenguo@xxxxxxxxxx>
------------------------------------------------------------

jobs:
build-amd64                                                  pass
build-armhf                                                  fail
build-i386                                                   pass
build-amd64-oldkern                                          pass
build-i386-oldkern                                           pass
build-amd64-pvops                                            pass
build-i386-pvops                                             pass
test-amd64-i386-qemuu-rhel6hvm-amd                           pass
test-amd64-amd64-xl-qemuu-win7-amd64                         fail
test-amd64-i386-xl-qemuu-win7-amd64                          fail
test-amd64-i386-qemuu-rhel6hvm-intel                         pass
test-amd64-i386-xl-qemuu-winxpsp3-vcpus1                     fail
test-amd64-i386-xend-qemuu-winxpsp3                          fail
test-amd64-amd64-xl-qemuu-winxpsp3                           fail
test-i386-i386-xl-qemuu-winxpsp3                             fail


------------------------------------------------------------
sg-report-flight on woking.cam.xci-test.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images

Logs, config files, etc. are available at
   http://www.chiark.greenend.org.uk/~xensrcts/logs

Test harness code can be found at
   http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit bf8e2e756e3e98b8388068173e24d99d1e09d2dd
Author: Hanweidong <hanweidong@xxxxxxxxxx>
Date:   Tue Apr 2 13:22:41 2013 +0000

   xen-mapcache: pass the right size argument to test_bits

   Compute the correct size for test_bits().
   qemu_get_ram_ptr() and qemu_safe_ram_ptr() will call xen_map_cache()
   with size is 0 if the requested address is in the RAM.  Then
   xen_map_cache() will pass the size 0 to test_bits() for checking if the
   corresponding pfn was mapped in cache. But test_bits() will always
   return 1 when size is 0 without any bit testing. Actually, for this
   case, test_bits should check one bit. So this patch introduced a
   __test_bit_size which is greater than 0 and a multiple of XC_PAGE_SIZE,
   then test_bits can work correctly with __test_bit_size
   >> XC_PAGE_SHIFT as its size.

   upstream-commit-id: 044d4e1aae539bd4214175bd9591b3de7986cf18

   Signed-off-by: Zhenguo Wang <wangzhenguo@xxxxxxxxxx>
   Signed-off-by: Weidong Han <hanweidong@xxxxxxxxxx>
   Signed-off-by: Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>

commit 0718150131d1d68436ba92b209fa50430198e6f4
Author: Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>
Date:   Tue Apr 2 13:23:40 2013 +0000

   xen-mapcache: replace last_address_index with a last_entry pointer

   Replace last_address_index and last_address_vaddr with a single pointer
   to the last MapCacheEntry used.

   upstream-commit-id: e2deee3ea6136b6189e8cfd26379420b9a398d96

   Signed-off-by: Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel



---------- End Forwarded Message ----------



--
Alex Bligh

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel