Re: [PATCH] PCI/MSI: Fix masking MSI/MSI-X on Xen PV

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Mon, Oct 25, 2021 at 12:53:31PM +0100, David Woodhouse wrote:
> On Mon, 2021-10-25 at 13:43 +0200, Roger Pau Monné wrote:
> > It's kind of optional for HVM guests, as it depends on
> > XENFEAT_hvm_pirqs, which sadly gets unconditionally set for HVM
> > guests, thus dropping any benefits from having hardware assisted APIC
> > virtualization or posted interrupts support.
> 
> Indeed. After implementing PIRQ support for Xen guests running under
> KVM, I spent a "happy" couple of days swearing at it because it
> actually *worked* if something would just *unmask* the f***ing MSI, but
> the guest inexplicably (to me) didn't do that.
> 
> Took me a while to work out that Xen itself is *snooping* on the MSI
> table writes even while they are *masked*, to capture the magic MSI
> message (with vector==0) which means it's actually a PIRQ# in the
> destination ID bits, and then magically unmask the MSI when the guest
> binds that PIRQ to an event channel.
> 
> I did not enjoy implementing that part.

I can see that. It's even better because none of this is actually
documented.

> FWIW the *guest* could potentlaly be smarter here and elect not to use
> PIRQs when hardware assisted vAPIC is present. Aren't there some bits
> in the CPUID that Xen advertises, which indicate that? 

Yes, it's in leaf 0x40000x04. FWIW, I would also be fine with removing
XENFEAT_hvm_pirqs, as I don't think diverging from the hardware
specifications gives us much benefit. We avoid a couple of vm exits
for sure, but the cost of having all those modifications in guest
OSes is not worth it.

Roger.