Re: [Xen-devel] [PATCH v2] x86/passthrough: fix migration of MSI when using posted interrupts

["Tian, Kevin" <kevin.tian@xxxxxxxxx>]

> From: Roger Pau Monné [mailto:roger.pau@xxxxxxxxxx]
> Sent: Thursday, October 31, 2019 11:23 PM
> 
> On Thu, Oct 31, 2019 at 07:52:23AM -0700, Joe Jin wrote:
> > On 10/31/19 1:01 AM, Jan Beulich wrote:
> > > On 30.10.2019 19:01, Joe Jin wrote:
> > >> On 10/30/19 10:23 AM, Roger Pau Monné wrote:
> > >>> On Wed, Oct 30, 2019 at 09:38:16AM -0700, Joe Jin wrote:
> > >>>> On 10/30/19 1:24 AM, Roger Pau Monné wrote:
> > >>>>> Can you try to add the following debug patch on top of the existing
> > >>>>> one and report the output that you get on the Xen console?
> > >>>>
> > >>>> Applied debug patch and run the test again, not of any log printed,
> > >>>> attached Xen log on serial console, seems pi_update_irte() not been
> > >>>> called for iommu_intpost was false.
> > >>>
> > >>> I have to admit I'm lost at this point. Does it mean the original
> > >>> issue had nothing to do with posted interrupts?
> > >>
> > >> Looks when inject irq by vlapic_set_irq(), it checked by
> > >> hvm_funcs.deliver_posted_intr rather than iommu_intpost:
> > >>
> > >>  176     if ( hvm_funcs.deliver_posted_intr )
> > >>  177         hvm_funcs.deliver_posted_intr(target, vec);
> > >>
> > >> And deliver_posted_intr() would be there, when vmx enabled:
> > >>
> > >> (XEN) HVM: VMX enabled
> > >> (XEN) HVM: Hardware Assisted Paging (HAP) detected
> > >> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
> > >
> > > I can't see the connection. start_vmx() has
> > >
> > >     if ( cpu_has_vmx_posted_intr_processing )
> > >     {
> > >         alloc_direct_apic_vector(&posted_intr_vector,
> pi_notification_interrupt);
> > >         if ( iommu_intpost )
> > >             alloc_direct_apic_vector(&pi_wakeup_vector,
> pi_wakeup_interrupt);
> > >
> > >         vmx_function_table.deliver_posted_intr = vmx_deliver_posted_intr;
> > >         vmx_function_table.sync_pir_to_irr     = vmx_sync_pir_to_irr;
> > >         vmx_function_table.test_pir            = vmx_test_pir;
> > >     }
> > >
> > > i.e. the hook is present only when posted interrupts are
> > > available in general. I.e. also with just CPU-side posted
> > > interrupts, yes, which gets confirmed by your "apicv=0"
> > > test. Yet with just CPU-side posted interrupts I'm
> > > struggling again to understand your original problem
> > > description, and the need to fiddle with IOMMU side code.
> >
> > Yes, on my test env, cpu_has_vmx_posted_intr_processing == true &&
> iommu_intpost == false,
> > with this, posted interrupts been enabled.
> 
> I'm still quite lost. My reading of the Intel VT-d spec is that the
> posted interrupt descriptor (which contains the PIRR) is used in
> conjunction with a posted interrupt remapping entry in the iommu, so
> that interrupts get recorded in the PIRR and later synced by the
> hypervisor into the vlapic IRR when resuming the virtual CPU.

there are two parts. Intel first implements CPU posted interrupt,
which allows one CPU to post IPI into non-root context in another
CPU through posted interrupt descriptor. Later VT-d posted 
interrupt comes, which use interrupt remapping entry and the
same posted interrupt descriptor (using more fields) to convert
a device interrupt into a posted interrupt. The posting process is
same on the dest CPU, regardless of whether it's from another CPU
or a device. 

> 
> How is the PIRR filled if there's no interrupt remapping entry
> pointing to it?
> 
> I have to admit I'm not super-familiar with the implementation in Xen,
> so it's likely I'm missing something.
> 
> Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel