[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [xen-unstable test] 10481: regressions - FAIL



On Tue, 2011-12-13 at 05:22 +0000, xen.org wrote:
> flight 10481 xen-unstable real [real]
> http://www.chiark.greenend.org.uk/~xensrcts/logs/10481/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking:
>  test-amd64-amd64-win          7 windows-install           fail REGR. vs. 
> 10474

Failed the network connection after install
        011-12-13 01:38:35 Z guest win.guest.osstest 5a:36:0e:f1:00:0c 8936 
link/ip/tcp: waiting 7000s...
        2011-12-13 01:38:35 Z guest win.guest.osstest 5a:36:0e:f1:00:0c 8936 
link/ip/tcp: no active lease (waiting) ...
        2011-12-13 01:57:28 Z guest win.guest.osstest 5a:36:0e:f1:00:0c 8936 
link/ip/tcp: nc: 256 (UNKNOWN) [10.80.251.132] 8936 (?) : Connection timed out 
|  (waiting) ...

VNC screenshot suggests the guest has booted fine and is sitting at the
screensaver.

brctl and ifconfig show that the devices are set up and in the right
places. PV NIC is not initialised (as expected) so we would expect to be
using the tap interface which has seemingly passed some traffic:
        tap5.0    Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff  
                  inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
                  UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
                  RX packets:442 errors:0 dropped:0 overruns:0 frame:0
                  TX packets:378027 errors:0 dropped:0 overruns:0 carrier:0
                  collisions:0 txqueuelen:500 
                  RX bytes:33714 (32.9 KiB)  TX bytes:24976607 (23.8 MiB)

This being a Windows guest there isn't much debug from within the guest.
Could this be an infrastructure thing? Other similar tests passed just
fine although two similar winxp sp3 tests also failed in a similar way
(this one was sp2, some other sp3 tests did pass though). This was a
xend test while others which failed similarly were xl based so there
appears to be no correlation there.

> Tests which are failing intermittently (not blocking):
>  test-amd64-amd64-pv           9 guest-start                 fail pass in 
> 10480

This PV guest appears to have failed for similar reasons to the above
Windows guests:

        2011-12-13 00:58:20 Z FAILURE: guest debian.guest.osstest 
5a:36:0e:f1:00:01 22 link/ip/tcp: wait timed out: no active lease.
        failure: guest debian.guest.osstest 5a:36:0e:f1:00:01 22 link/ip/tcp: 
wait timed out: no active lease.

The guest log shows it did get an IP address at least:

        Setting kernel variables ...done.
        Configuring network interfaces...Internet Systems Consortium DHCP 
Client 4.1.1-P1
        Copyright 2004-2010 Internet Systems Consortium.
        All rights reserved.
        For info, please visit https://www.isc.org/software/dhcp/
        
        Listening on LPF/eth0/5a:36:0e:f1:00:01
        Sending on   LPF/eth0/5a:36:0e:f1:00:01
        Sending on   Socket/fallback
        DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 7
        DHCPOFFER from 10.80.2.2
        DHCPREQUEST on eth0 to 255.255.255.255 port 67
        DHCPACK from 10.80.2.3
        bound to 10.80.3.117 -- renewal in 18880 seconds.

>  test-amd64-amd64-xl-sedf-pin 11 guest-localmigrate          fail pass in 
> 10480
>  test-amd64-amd64-xl-sedf     11 guest-localmigrate fail in 10480 pass in 
> 10481
> 
> Tests which did not succeed, but are not blocking,
> including regressions (tests previously passed) regarded as allowable:

(reordered to group similar)

> test-amd64-i386-xl-winxpsp3-vcpus1 13 guest-stop               fail never 
> pass 
> test-amd64-i386-xl-win7-amd64 13 guest-stop                   fail  never pass
> test-amd64-amd64-xl-win7-amd64 13 guest-stop                   fail never pass
> test-amd64-amd64-xl-win      13 guest-stop                   fail   never pass
> test-amd64-i386-xl-win-vcpus1 13 guest-stop                   fail  never pass
> test-i386-i386-xl-win        13 guest-stop                   fail   never 
> pass 

These are all failing because "xl shutdown" cannot be used with a guest
without PV drivers (and prints a warning to that effect).
        2011-12-13 01:33:56 Z executing ssh ... root@xxxxxxxxxxxx xl shutdown 
-w win.guest.osstest
        libxl: error: libxl.c:581:libxl_domain_shutdown: HVM domain without PV 
drivers: graceful shutdown not possible, use destroy
        shutdown failed (rc=-3)
        
I think "xl trigger <dom> power" would be what is wanted here -- e.g.
send an ACPI power event. It could be argued that xl shutdown could do
this automatically?

>  test-amd64-amd64-xl-pcipt-intel  9 guest-start                 fail never 
> pass

No active link message again but this time the guest says:
        For info, please visit https://www.isc.org/software/dhcp/
        
        SIOCSIFADDR: No such device
        eth0: ERROR while getting interface flags: No such device
        eth0: ERROR while getting interface flags: No such device
        Bind socket to interface: No such device
        Failed to bring up eth0.
        done.
        Cleaning up temporary files....

If we could preserve a guest in that state and login it might prove
informative. My guess would either be a missing/faulty VF driver or udev
renaming things.

>  test-amd64-i386-rhel6hvm-intel  9 guest-start.2                fail never 
> pass
>  test-amd64-i386-rhel6hvm-amd  9 guest-start.2                fail   never 
> pass

Both of these appear to be similar harness issues:
        2011-12-13 02:11:21 Z toolstack xl
        Use of uninitialized value in concatenation (.) or string at 
./ts-guest-start line 13.
        2011-12-13 02:11:21 Z executing ssh ... root@xxxxxxxxxxxxx xl create 
        Config file not specified and none in save file

>  test-amd64-i386-xend-winxpsp3 16 leak-check/check             fail  never 
> pass
>  test-i386-i386-win           16 leak-check/check             fail   never 
> pass
>  test-amd64-i386-win-vcpus1   16 leak-check/check             fail   never 
> pass
>  test-amd64-i386-win          16 leak-check/check             fail   never 
> pass

These all leaked a load of /var/lib/xen/qemu-resume.N. This should be
quick & easy to fix, I'll have a look.

>  test-amd64-amd64-xl-winxpsp3  7 windows-install              fail   never 
> pass
>  test-i386-i386-xl-winxpsp3    7 windows-install              fail   never 
> pass

These are failing in the same way as the one discussed right at the top.

>  test-amd64-i386-xl-winxpsp3-vcpus1  7 windows-install fail in 10480 like 
> 10474
>  test-amd64-i386-xend-winxpsp3  7 windows-install      fail in 10480 like 
> 10474
>  test-amd64-i386-xl-win7-amd64  7 windows-install      fail in 10480 like 
> 10474
>  test-amd64-amd64-xl-win7-amd64  7 windows-install     fail in 10480 like 
> 10474
>  test-amd64-i386-win           7 windows-install       fail in 10480 like 
> 10473

These don't appear to have failed per the grid at
http://www.chiark.greenend.org.uk/~xensrcts/logs/10481/ ?

e.g. test-amd64-i386-xl-winxpsp3-vcpus1 appears to have failed at
guest-stop instead (and indeed is also listed above in that capacity)

This appears to be reporting a failure in a previous run, part of the
heisenbug detector? It might be nice to put those in a separate section
or to include some indication as the the criteria being evaluated (e.g.
are we waiting for a 3rd test to tiebreak?)

> 
> version targeted for testing:
>  xen                  7ca56cca09ad
> baseline version:
>  xen                  1c58bb664d8d
> 
> ------------------------------------------------------------
> People who touched revisions under test:
>   Andre Przywara <andre.przywara@xxxxxxx>
>   Ian Campbell <ian.campbell@xxxxxxxxxx>
>   Ian Jackson <ian.jackson@xxxxxxxxxxxxx>
>   Jan Beulich <jbeulich@xxxxxxxx>
>   Roger Pau Monne <roger.pau@xxxxxxxxxxxxx>
> ------------------------------------------------------------
> 
> jobs:
>  build-amd64                                                  pass    
>  build-i386                                                   pass    
>  build-amd64-oldkern                                          pass    
>  build-i386-oldkern                                           pass    
>  build-amd64-pvops                                            pass    
>  build-i386-pvops                                             pass    
>  test-amd64-amd64-xl                                          pass    
>  test-amd64-i386-xl                                           pass    
>  test-i386-i386-xl                                            pass    
>  test-amd64-i386-rhel6hvm-amd                                 fail    
>  test-amd64-amd64-xl-win7-amd64                               fail    
>  test-amd64-i386-xl-win7-amd64                                fail    
>  test-amd64-i386-xl-credit2                                   pass    
>  test-amd64-amd64-xl-pcipt-intel                              fail    
>  test-amd64-i386-rhel6hvm-intel                               fail    
>  test-amd64-i386-xl-multivcpu                                 pass    
>  test-amd64-amd64-pair                                        pass    
>  test-amd64-i386-pair                                         pass    
>  test-i386-i386-pair                                          pass    
>  test-amd64-amd64-xl-sedf-pin                                 fail    
>  test-amd64-amd64-pv                                          fail    
>  test-amd64-i386-pv                                           pass    
>  test-i386-i386-pv                                            pass    
>  test-amd64-amd64-xl-sedf                                     pass    
>  test-amd64-i386-win-vcpus1                                   fail    
>  test-amd64-i386-xl-win-vcpus1                                fail    
>  test-amd64-i386-xl-winxpsp3-vcpus1                           fail    
>  test-amd64-amd64-win                                         fail    
>  test-amd64-i386-win                                          fail    
>  test-i386-i386-win                                           fail    
>  test-amd64-amd64-xl-win                                      fail    
>  test-i386-i386-xl-win                                        fail    
>  test-amd64-i386-xend-winxpsp3                                fail    
>  test-amd64-amd64-xl-winxpsp3                                 fail    
>  test-i386-i386-xl-winxpsp3                                   fail    
> 
> 
> ------------------------------------------------------------
> sg-report-flight on woking.cam.xci-test.com
> logs: /home/xc_osstest/logs
> images: /home/xc_osstest/images
> 
> Logs, config files, etc. are available at
>     http://www.chiark.greenend.org.uk/~xensrcts/logs
> 
> Test harness code can be found at
>     http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary
> 
> 
> Not pushing.
> 
> (No revision log; it would be 336 lines long.)
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.