Xen project Mailing List

Re: [xen-unstable test] 169819: regressions - FAIL

From: Roger Pau Monné <roger.pau@xxxxxxxxxx>

Date: Mon, 2 May 2022 12:49:56 +0200

Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none

Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pLm4VKES2xR4AoS7eswrhG3rPzPMjBBZN3lUm7Ngte8=; b=neBSSdEL82kFrqJQLAqgz0w/DjM0LVEKe4hhljgESX+QIuBZ7I0NokNTyi5L9INutFxf7lzGEdI5uoJQ7lBWoEotKsGaZjJra6z+8nGJaIEwjxrCB/g+g7Gd1TpGK+Qiq8UK5dLgScwYg0e0g+zP0XmcnzdWQ3vY/2uelkBRllq6WSol099jP5Az78Ioy4hLa4CuRNXkZrExCC/1Ibhe7RhasXxWFAO9zczHhBfwII6x+qfOp/k2UpsoZ8L1Rk3Kqxtn1/pV5M/IPnAtLwjwi9+caC/GxQRFN29NAr0uheK91vZ0Um4p5ebmet9Kd0bLomy+gsN7n/pXt7WozAREHQ==

Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OHlFBrgn/FFDALMXg1DEjUkPnMDWtxrbDm+BeAzAduBoi7f3VsDkC2ggS41XzhjLpgV4d2Cfm8TtqaXEnuG1BYjPzobwB6OeJ1LCiDJAIm3VC8nYjMuH+AyEDRbbwEQY4yZRflrM2q5wXMHMCD5tnCG1mQrQQxDhVP2+4oh/P7Kv88Yc5lHUZ8o4a46auzpnjKnIoFmayfBmco+d+cwR7B4U7+iCNXw5T0x2bmNQPBEwMt2b237uH53FSg73r9AH2HQOH8OaoJFwOzZnqeSiWHhwBmKGapgFNxhE+QIairdgsYQxB/fb2hLlLaLGR9ISnQRulfOBN3ZjKX3YISO9lQ==

Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=citrix.com;

Cc: Jan Beulich <jbeulich@xxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxxx, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Boris Ostrovsky <boris.ostrovsky@xxxxxxxxxx>, osstest service owner <osstest-admin@xxxxxxxxxxxxxx>

Delivery-date: Mon, 02 May 2022 10:50:32 +0000

Ironport-data: A9a23:wD+G8qNknVo0HXbvrR2QlsFynXyQoLVcMsEvi/4bfWQNrUp0gWQAy TZJWm2DbqyIM2T3fNggOY21/R9Su5Hcx9dgGgto+SlhQUwRpJueD7x1DKtR0wB+jCHnZBg6h ynLQoCYdKjYdleF+lH1dOKJQUBUjclkfJKlYAL/En03FFYMpBsJ00o5wbZk2tcw27BVPivW0 T/Mi5yHULOa82Yc3lI8s8pvfzs24ZweEBtB1rAPTagjUG32zhH5P7pGTU2FFFPqQ5E8IwKPb 72rIIdVXI/u10xF5tuNyt4Xe6CRK1LYFVDmZnF+A8BOjvXez8CbP2lS2Pc0MC9qZzu1c99Z9 MpVj5XueV0Qb4KWmNVCT0N7IQxSBPgTkFPHCSDXXc276WTjKiGp79AwSUY8MMsf5/p9BnxI+ boAMjcRYxufhuWwhrWmVu1rgcdlJ87uVG8dkig4kXeFUrB7ENaaHPuiCdxwhV/cguhUGvnTf YwBYCdHZxXceRxffFwQDfrSmc/33iOjLGQB+Tp5o4IRwXbvkAVz0YL0IcX7YvePZMQEgliX8 zeuE2PRR0ty2Mak4TiK6HW3ncfUgDj2HokVEdWQ9PRnnVmSzWw7EwANWB2wpvzRol6zXZdTJ lIZ/gIqrLMu7wq7Q9/lRRq6rXWY+BkGVLJ4Geog6RrLzbHI+QGHHWsVZjlbYdciuYk9QjlC/ liShM/kHiAqubGQSHS15rqStSm1OyUeMSkFfyBscOcey9zqoYV2gheRSN9mSfSxloesRm+2x C2Wpi8jgblVldQMy6iw4VHAhXSru4TNSQk2oA7QWwpJ8z9EWWJsXKTwgXCz0BqKBN/xooWp1 JTcp/Wj0Q==

Ironport-hdrordr: A9a23:UoacVaNZ6JnMTsBcT0j155DYdb4zR+YMi2TDiHoddfUFSKalfp 6V98jztSWatN/eYgBEpTmlAtj5fZq6z+8P3WBxB8baYOCCggeVxe5ZjbcKrweQeBEWs9Qtr5 uIEJIOd+EYb2IK6voSiTPQe7hA/DDEytHPuQ639QYQcegAUdAF0+4WMHf4LqUgLzM2eKbRWa DsrvZvln6FQzA6f867Dn4KU6zqoMDKrovvZVojCwQ84AeDoDu04PqieiLolCs2Yndq+/MP4G LFmwv26uGKtOy68AbV0yv2445NkNXs59NfDIini9QTKB/rlgG0Db4REIGqjXQQmqWC+VwqmN 7Dr1MJONly0WrYeiWPrR7ky2DboUETA9OL8y7qvVLT5ejCAB4qActIgoxUNjHD7VA7gd162K VXm0qEqpt+F3r77WrAzumNcysvulu/oHIkn+JWpWdYS5EiZLhYqpFa1F9JEa0HADnx5OkcYa ZT5fnnlbZrmG6hHjPkVjEF+q3vYp1zJGbLfqE6gL3V79AM90oJinfxx6Qk7wM9HdwGOt15Dt //Q9VVfYF1P7ErhJ1GdZc8qOuMexrwqEH3QSuvyWqOLtB0B1v977jK3Z4S2MaGPLQ18bpaou W1bLofjx9+R37T

List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On Mon, May 02, 2022 at 08:51:40AM +0200, Juergen Gross wrote: > On 29.04.22 13:00, Roger Pau Monné wrote: > > On Fri, Apr 29, 2022 at 12:37:13PM +0200, Jan Beulich wrote: > > > On 29.04.2022 12:30, Roger Pau Monné wrote: > > > > On Fri, Apr 29, 2022 at 07:46:47AM +0000, osstest service owner wrote: > > > > > flight 169819 xen-unstable real [real] > > > > > flight 169843 xen-unstable real-retest [real] > > > > > http://logs.test-lab.xenproject.org/osstest/logs/169819/ > > > > > http://logs.test-lab.xenproject.org/osstest/logs/169843/ > > > > > > > > > > Regressions :-( > > > > > > > > > > Tests which did not succeed and are blocking, > > > > > including tests which could not be run: > > > > > test-arm64-arm64-examine 8 reboot fail REGR. > > > > > vs. 169775 > > > > > test-arm64-arm64-libvirt-xsm 8 xen-boot fail REGR. > > > > > vs. 169775 > > > > > test-arm64-arm64-libvirt-raw 8 xen-boot fail REGR. > > > > > vs. 169775 > > > > > test-arm64-arm64-xl-credit1 8 xen-boot fail REGR. > > > > > vs. 169775 > > > > > test-arm64-arm64-xl-thunderx 8 xen-boot fail REGR. > > > > > vs. 169775 > > > > > test-arm64-arm64-xl 8 xen-boot fail REGR. > > > > > vs. 169775 > > > > > > > > > > Tests which are failing intermittently (not blocking): > > > > > test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install > > > > > fail pass in 169843-retest > > > > > > > > Looked into this one, and it's slightly concerning, guest seems to be > > > > stuck at installation: > > > > > > > > Select and install software [ 481.093857] watchdog: BUG: soft lockup > > > > - CPU#1 stuck for 23s! [ksoftirqd/1:17] > > > > [ 509.093865] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 545.093820] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 573.093809] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 609.093855] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 637.093836] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 673.093957] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 701.093854] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 733.093805] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 761.093817] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 797.093898] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 825.093863] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 861.093865] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 889.093945] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 925.093974] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 953.093925] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 985.093832] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1013.093855] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1049.094031] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 1077.093860] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 1113.093938] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1141.093803] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1177.094051] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 1205.093805] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 1237.093955] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 1265.094004] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 1301.093835] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1329.094039] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1365.093883] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 1393.094167] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1429.093857] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1457.093900] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 1489.094026] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1517.093997] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1553.093996] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1581.094064] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1617.094076] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 1645.093882] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 1681.093896] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1709.094022] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1741.093870] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1769.093854] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1805.094017] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 1833.093837] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 1869.094043] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1897.094101] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1933.093879] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 1961.093933] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 1997.093952] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2025.093925] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 2057.093895] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2085.094172] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2121.094018] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2149.094021] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2185.093931] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 2213.093864] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 2249.093951] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2277.093899] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2309.094054] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2337.093956] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2373.094111] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 2401.094132] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 2437.094064] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2465.094003] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2501.093959] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 2529.094050] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2561.094027] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 2589.094061] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 2625.093999] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2653.093956] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2689.094024] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2717.094093] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2753.093913] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 2781.094095] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 2813.093959] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2841.094117] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2877.094041] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2905.094009] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 2941.093893] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 2969.094085] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! > > > > [ksoftirqd/1:17] > > > > [ 3005.094061] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 3033.093917] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > [ 3065.094060] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! > > > > [ksoftirqd/1:17] > > > > > > > > debina1 is one of the boxes I've put into production recently. > > > > > > ISTR this issue having surfaced randomly before. With the randomness > > > making > > > it difficult to investigate. > > > > Indeed, it seems to happen on other boxes also: > > > > http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-xl-qemut-debianhvm-i386-xsm/xen-unstable.html > > > > At least doesn't seem to be explicitly related to debinas. > > How valuable is it to test systems using a kernel from 2019? > > Chances are high that bugs which might trigger are corrected already. > IMO trying to find probably known bugs is a waste of time. This is the kernel from oldstable Debian distribution, which is still supported (until June 2024 AFAICT). I think it's not unlikely that people are running those kernels in guests. I wouldn't mind testing newer guest kernels, but someone would have to update osstest to use Debian 11 which ships with kernel 5.10. Thanks, Roger.

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.