[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [xen-unstable test] 169819: regressions - FAIL



On 29.04.22 13:00, Roger Pau Monné wrote:
On Fri, Apr 29, 2022 at 12:37:13PM +0200, Jan Beulich wrote:
On 29.04.2022 12:30, Roger Pau Monné wrote:
On Fri, Apr 29, 2022 at 07:46:47AM +0000, osstest service owner wrote:
flight 169819 xen-unstable real [real]
flight 169843 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/169819/
http://logs.test-lab.xenproject.org/osstest/logs/169843/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
  test-arm64-arm64-examine      8 reboot                   fail REGR. vs. 169775
  test-arm64-arm64-libvirt-xsm  8 xen-boot                 fail REGR. vs. 169775
  test-arm64-arm64-libvirt-raw  8 xen-boot                 fail REGR. vs. 169775
  test-arm64-arm64-xl-credit1   8 xen-boot                 fail REGR. vs. 169775
  test-arm64-arm64-xl-thunderx  8 xen-boot                 fail REGR. vs. 169775
  test-arm64-arm64-xl           8 xen-boot                 fail REGR. vs. 169775

Tests which are failing intermittently (not blocking):
  test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass 
in 169843-retest

Looked into this one, and it's slightly concerning, guest seems to be
stuck at installation:

Select and install software  [  481.093857] watchdog: BUG: soft lockup - CPU#1 
stuck for 23s! [ksoftirqd/1:17]
[  509.093865] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[  545.093820] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[  573.093809] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[  609.093855] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[  637.093836] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[  673.093957] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[  701.093854] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[  733.093805] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[  761.093817] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[  797.093898] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[  825.093863] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[  861.093865] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[  889.093945] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[  925.093974] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[  953.093925] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[  985.093832] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1013.093855] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1049.094031] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 1077.093860] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 1113.093938] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1141.093803] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1177.094051] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 1205.093805] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 1237.093955] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 1265.094004] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 1301.093835] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1329.094039] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1365.093883] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 1393.094167] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1429.093857] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1457.093900] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 1489.094026] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1517.093997] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1553.093996] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1581.094064] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1617.094076] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 1645.093882] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 1681.093896] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1709.094022] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1741.093870] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1769.093854] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1805.094017] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 1833.093837] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 1869.094043] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1897.094101] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1933.093879] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 1961.093933] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 1997.093952] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2025.093925] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 2057.093895] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2085.094172] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2121.094018] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2149.094021] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2185.093931] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 2213.093864] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 2249.093951] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2277.093899] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2309.094054] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2337.093956] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2373.094111] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 2401.094132] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 2437.094064] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2465.094003] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2501.093959] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 2529.094050] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2561.094027] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 2589.094061] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 2625.093999] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2653.093956] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2689.094024] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2717.094093] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2753.093913] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 2781.094095] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 2813.093959] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2841.094117] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2877.094041] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2905.094009] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 2941.093893] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 2969.094085] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! 
[ksoftirqd/1:17]
[ 3005.094061] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 3033.093917] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]
[ 3065.094060] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[ksoftirqd/1:17]

debina1 is one of the boxes I've put into production recently.

ISTR this issue having surfaced randomly before. With the randomness making
it difficult to investigate.

Indeed, it seems to happen on other boxes also:

http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-xl-qemut-debianhvm-i386-xsm/xen-unstable.html

At least doesn't seem to be explicitly related to debinas.

How valuable is it to test systems using a kernel from 2019?

Chances are high that bugs which might trigger are corrected already.
IMO trying to find probably known bugs is a waste of time.


Juergen

Attachment: OpenPGP_0xB0DE9DD628BF132F.asc
Description: OpenPGP public key

Attachment: OpenPGP_signature
Description: OpenPGP digital signature


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.