[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: NetBSD dom0 PVH: hardware interrupts stalls


  • To: Manuel Bouyer <bouyer@xxxxxxxxxxxxxxx>
  • From: Roger Pau Monné <roger.pau@xxxxxxxxxx>
  • Date: Wed, 18 Nov 2020 09:57:38 +0100
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zIKlsQmdtLHf5IqZKe8dJqMJBLrKcdtfuwQUc3PNPI4=; b=PtYBk+0yTdwHGZNgkhEZlDwFKMGSxzVHhm1P7PP9Ku+C2rIg3F/7L7N2WNAeiix1sJgF1PmSXrltufbpf92SlLXWhbqXV2vUtmiTKu/Vsg4uaWIpWvaeUw3oQIMmjkB2Al7elzOCKiOLYrQcBK2jwvf4eBlC4NApoutPdGa8FDA5+INNxh7yVnW88l2JRvlPjszomZTqGUyJmT2wTWPc6V1l/E4dL9QVR63Zbv6EkqYmK/zBxsTD8s+X3K/atYs3dMc/RWaFExGOKMlRkFgxRhXm4lCGddEq5b7bqyZB1+uPx1ZEeVABriZ+VoDi0SoAUQNYu5BUeBwfrV0BUd6dEA==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DRjcTJFbVnlFs/oF0VEbwF9Bnpd071Dj0N+PkunByx/HHPzdYHIt+QOoO/xDfbzQ/YHuTdLhf/D56nb8wk2hOa0TUI6NCPLbrpaBPFAyfZHB2xM5ey0eJK5j6x29MvjPWnb6Dm8WVgDE1iDxsWV9taXcdWWCCUPmAbTYMyaJoWt7vZBmJD9ZQFt7lwr+DP0Gw9vbIhktolNoio4sx42mlzJVmvdthZAMbC+5m2Dap4PmzNLKbBaeExkExlcQLKFMkYnuHzM/CSxE30liifDlPKzKOL0wfHv/49mVCNXRELI+hp7BqNMPWpMJUgp2DdRREJhtvnFA3w1NtZ+sPzhE+g==
  • Authentication-results: esa3.hc3370-68.iphmx.com; dkim=pass (signature verified) header.i=@citrix.onmicrosoft.com
  • Cc: <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Wed, 18 Nov 2020 08:57:54 +0000
  • Ironport-sdr: a2oQwtXcury+NJZQnQ7pVs1ZV5oxcEWUcD96KIWa5dxDZWOi1qk79yiAK+aoAqmdmlVGauPDQt FXkgargbkQmgm5JgFrYvHD83jz0AAuzA3Jz01ByAXHlQFo5iuFtgspZSkq1e8f3xJwdfVDc45Y 7K1oOvGa/eh4+5POydf0POk5whYWAQRaVjDhJvX2iDrS+aFHJa2o4feMzOFLFm1deBAc7hJoqO 7AIcaZgpfvj0cDdvJefRel5BiQFNUauKPo8M96tq2gS2xA/RXRoDefrmdRfetNwEEPE7dxrcjT XUc=
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On Tue, Nov 17, 2020 at 05:40:33PM +0100, Manuel Bouyer wrote:
> On Tue, Nov 17, 2020 at 04:58:07PM +0100, Roger Pau Monné wrote:
> > [...]
> > 
> > I have attached a patch below that will dump the vIO-APIC info as part
> > of the 'i' debug key output, can you paste the whole output of the 'i'
> > debug key when the system stalls?
> 
> see attached file. Note that the kernel did unstall while 'i' output was
> being printed, so it is mixed with some NetBSD kernel output.
> The idt entry of the 'ioapic2 pin2' interrupt is 103 on CPU 0.
> 
> I also put the whole sequence at
> http://www-soc.lip6.fr/~bouyer/xen-log3.txt

On one of the instances the pin shows up as masked, but I'm not sure
if that's relevant since later it shows up as unmasked. Might just be
part of how NetBSD handles such interrupts.

So taking a look at the information dumped:

(XEN)     IRQ 34 Vec 81:
(XEN)       Apic 0x02, Pin  2: 5vec=51 delivery=LoPri dest=L status=1 
polarity=1 irr=1 trig=L mask=0 dest_id:00000001
(XEN) ioapic 2 pin 2 gsi 34 vector 0x67
(XEN)   delivery mode 0 dest mode 0 delivery status 0
(XEN)   polarity 1 IRR 0 trig mode 1 mask 1 dest id 0
[...]
(XEN)     IRQ 34 Vec 81:
(XEN)       Apic 0x02, Pin  2: 0vec=51 delivery=LoPri dest=L status=0 
polarity=1 irr=0 trig=L mask=0 dest_id:00000001
(XEN)   delivery mode 0 dest mode 0 delivery status 0
(XEN)   polarity 1 IRR 0 trig mode 1 mask 0 dest id 0

The state between the physical IO-APIC and the emulated one seems
consistent at least.

On a maybe unrelated question, how do you setup the event channel
callback, is it using HVM_PARAM_CALLBACK_IRQ and
HVM_PARAM_CALLBACK_TYPE_VECTOR?

Are you EOI'ing such vector on the local APIC when servicing the
interrupt?

> You'll see that I did hit 'i' 2 times to get the NetBSD kernel to boot
> multiuser.
> 
> > 
> > Can you assert that you properly EOI the vectors on the local APIC? (I
> > don't have a patch to dump the emulated lapic ISR right now, but could
> > provide one if needed).
> 
> Reading the code, I think it's OK (assuming I properly understood what
> you mean too). Wouldn't it cause problems on real hardware too
> if the vectors were not EOI'd ?

Yes it should, just wasn't sure whether NetBSD would use the same
handler code when running under Xen.

I was mostly asking because Xen has a weird semantic with the vector
callback when using HVM_PARAM_CALLBACK_IRQ, see above.

Roger.



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.