Re: [Xen-devel] Xen-unstable: pci-passthrough of device using MSI-X interrupts not working after commit x86/MSI: track host and guest masking separately

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>]

On Fri, 26 Jun 2015, Jan Beulich wrote:
> >>> On 26.06.15 at 17:04, <JBeulich@xxxxxxxx> wrote:
> >>>> On 26.06.15 at 15:38, <linux@xxxxxxxxxxxxxx> wrote:
> >> On 2015-06-26 14:41, Jan Beulich wrote:
> >>>>>> On 26.06.15 at 13:02, <linux@xxxxxxxxxxxxxx> wrote:
> >>>> Strange, i don't see *any* of your printk's being hit ... (xl dmesg
> >>>> attached).
> >>> 
> >>> So does the guest (in the working case) use MSI-X at all for the
> >>> device? I.e. it might be worth comparing the guest's /proc/interrupts
> >>> from both cases, as the lack of any of the debug messages clearly
> >>> suggests that such interrupts aren#t being set up.
> >> 
> >> In the good case it uses one of them.
> >> (probably one per port and it has only one usb device connected at 
> >> present)
> >> 
> >> --
> >> Sander
> >> 
> >>           CPU0       CPU1       CPU2       CPU3
> >>    0:         42          0          0          0   IO-APIC   2-edge      
> > timer
> >>[...]
> >>   83:          8          0          0          0   xen-dyn-event     
> >> eth0-q3-rx
> >>   84:       2101          0          0          0  xen-pirq-msi-x    
> >> xhci_hcd
> > 
> > I think this explains it - you're running in PVHVM mode, which I
> > never tried with those patches. I'd even have to go dig to see how
> > they drive MSI-X in the first place in that case.
> 
> I have an idea: In
> 
> static unsigned int startup_msi_irq(struct irq_desc *desc)
> {
>     bool_t guest_masked = (desc->status & IRQ_GUEST) &&
>                           is_hvm_domain(desc->msi_desc->dev->domain);
> 
>     if ( unlikely(!msi_set_mask_bit(desc, 0, guest_masked)) )
>         WARN();
>     return 0;
> }
> 
> I think we need to also exclude the emuirq case (which is what I
> understand backs the pvhvm interrupt in the guest - Stefano,
> please confirm). For testing purposes, could you try simply passing
> zero instead of guest_masked here?

emuirq is an irq (or msi) that is generated by an emulated device.

Obviously a guest cannot tell whether a device is emulated or
passthrough, as a consequence when the guest remaps irqs (and msis) into
event channels (arch/x86/pci/xen.c:acpi_register_gsi_xen_hvm and
arch/x86/pci/xen.c:xen_hvm_setup_msi_irqs), some of the irqs (and msis)
might actually be emulated.

Xen uses the concept of "emuirq" to distinguish irqs (and msis) of a
passthrough device from those belonging to an emulated device. However
you should know that in a default configuration, without virtio, there
are no emulated msis. Maybe emuirqs are not really the problem here.

In the case of passthrough devices that generate MSI-X, the guest remaps
MSIs into event channels by writing a magic number and the desired pirq
in the MSI-X table (xen_hvm_setup_msi_irqs), then QEMU retrieves the
pirq and calls xc_physdev_map_pirq_msi & friends on its behalf.
Afterwards Linux uses the regular xen_pirq_chip functions to
enable/disable the pirq. Specifically masking the MSI, is done by
masking the event channel.

In the quoted function above, if the issue is finding out whether the
msi is masked from the guest point of view, then you also need to check
if it has been remapped into an event channel (you can use
domain_irq_to_pirq for that), then check its status.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel