Re: [Xen-devel] Xen-unstable: pci-passthrough regression bisected to: x86/smp: use APIC ALLBUT destination shorthand when possible

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Mon, Feb 03, 2020 at 01:44:06PM +0100, Sander Eikelenboom wrote:
> On 03/02/2020 13:41, Roger Pau Monné wrote:
> > On Mon, Feb 03, 2020 at 01:30:55PM +0100, Sander Eikelenboom wrote:
> >> On 03/02/2020 13:23, Roger Pau Monné wrote:
> >>> On Mon, Feb 03, 2020 at 09:33:51AM +0100, Sander Eikelenboom wrote:
> >>>> Hi Roger,
> >>>>
> >>>> Last week I encountered an issue with the PCI-passthrough of a USB 
> >>>> controller. 
> >>>> In the guest I get:
> >>>>     [ 1143.313756] xhci_hcd 0000:00:05.0: xHCI host not responding to 
> >>>> stop endpoint command.
> >>>>     [ 1143.334825] xhci_hcd 0000:00:05.0: xHCI host controller not 
> >>>> responding, assume dead
> >>>>     [ 1143.347364] xhci_hcd 0000:00:05.0: HC died; cleaning up
> >>>>     [ 1143.356407] usb 1-2: USB disconnect, device number 2
> >>>>
> >>>> Bisection turned up as the culprit: 
> >>>>    commit 5500d265a2a8fa63d60c08beb549de8ec82ff7a5
> >>>>    x86/smp: use APIC ALLBUT destination shorthand when possible
> >>>
> >>> Sorry to hear that, let see if we can figure out what's wrong.
> >>
> >> No problem, that is why I test stuff :)
> >>
> >>>> I verified by reverting that commit and now it works fine again.
> >>>
> >>> Does the same controller work fine when used in dom0?
> >>
> >> Will test that, but as all other pci devices in dom0 work fine,
> >> I assume this controller would also work fine in dom0 (as it has also
> >> worked fine for ages with PCI-passthrough to that guest and still works
> >> fine when reverting the referenced commit).
> > 
> > Is this the only device that fails to work when doing pci-passthrough,
> > or other devices also don't work with the mentioned change applied?
> > 
> > Have you tested on other boxes?
> > 
> >> I don't know if your change can somehow have a side effect
> >> on latency around the processing of pci-passthrough ?
> > 
> > Hm, the mentioned commit should speed up broadcast IPIs, but I don't
> > see how it could slow down other interrupts. Also I would think the
> > domain is not receiving interrupts from the device, rather than
> > interrupts being slow.
> > 
> > Can you also paste the output of lspci -v for that xHCI device from
> > dom0?
> > 
> > Thanks, Roger.
> 
> Will do this evening including the testing in dom0 etc.
> Will also see if there is any pattern when observing /proc/interrupts in
> the guest.

Thanks! I also have some trivial patch that I would like you to try,
just to discard send_IPI_mask clearing the scratch_cpumask under
another function feet.

Roger.
---
diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c
index 65eb7cbda8..aeeb506155 100644
--- a/xen/arch/x86/smp.c
+++ b/xen/arch/x86/smp.c
@@ -66,7 +66,8 @@ static void send_IPI_shortcut(unsigned int shortcut, int 
vector,
 void send_IPI_mask(const cpumask_t *mask, int vector)
 {
     bool cpus_locked = false;
-    cpumask_t *scratch = this_cpu(scratch_cpumask);
+    static DEFINE_PER_CPU(cpumask_t, send_ipi_cpumask);
+    cpumask_t *scratch = &this_cpu(send_ipi_cpumask);
 
     /*
      * This can only be safely used when no CPU hotplug or unplug operations


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel