[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v2] xen/pci: Refactor PCI MSI interrupts related code


  • To: Julien Grall <julien@xxxxxxx>
  • From: Rahul Singh <Rahul.Singh@xxxxxxx>
  • Date: Mon, 19 Apr 2021 07:16:52 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YoKdU2VMFlNwXP19o9K8VMJuJDahd7Bp3b3fiXkrdPg=; b=CL1scc8ppTwpL6am3B53DiuCxVWI6rq5lmMd22oaDL7UIjsgojmE6xaeXOOe09f4gH20EpsKQi3OAI0QYmIvAWdzQKdL0QtPrKxKOBD3ERP8X+syVNSSDdIChIa2DtJ6Jk2nIodNjt+u88XHLYZlhuE9uPI2q3YMw6rfYc+lsHEgxHReYftN8voWvGQ1twKh1EAXI7qhJDxXFhQOomg3MYu4neDlQdX9Dpeg2zWxV1G/CBD1qksuFgf5wdo8rNEhGUiSPQcm6iA7PLpg41YJQT+upZBqaMrOTufNcpI16+Xix6mY6f3L/trYMH2/blWLtcwZ6DpFiZ6QNMvc/4mnKg==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=llY8Rqt1ECH1/8CePR/QpOrjJNk1pqz13m5ystYEXvZpjZC0dUfDIm2YJTMIgqoOkSgYNPkHJFv8sMVJm8Ir73Ua1Pm4Ta7A64NHf79bw66vFuhM6hgGF4+3d0YUX2gmaE/1E+yrmXbzrhieSghNq8pArx5KweacCTdE9ZXMMyQuwBgDXS23DY9MW0Ql1NqzDgBNFdiP4xcLviv/2r2tw2aXg29Uowh4y2MxOhtfgW0k7R9DruTXs01wmNgGQ6Ds1SPzZxSGvNW48ieU5/ZldbiGeKG27wGz5w7E6EX7vitczO5zvgwxZ31tnN3NDND28BAUldEZjCn1vioTMsyd8w==
  • Authentication-results-original: xen.org; dkim=none (message not signed) header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
  • Cc: Roger Pau Monné <roger.pau@xxxxxxxxxx>, xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Bertrand Marquis <Bertrand.Marquis@xxxxxxx>, Jan Beulich <jbeulich@xxxxxxxx>, Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Wei Liu <wl@xxxxxxx>, George Dunlap <george.dunlap@xxxxxxxxxx>, Ian Jackson <iwj@xxxxxxxxxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Paul Durrant <paul@xxxxxxx>, Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>, Daniel De Graaf <dgdegra@xxxxxxxxxxxxx>
  • Delivery-date: Mon, 19 Apr 2021 07:17:23 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Nodisclaimer: true
  • Original-authentication-results: xen.org; dkim=none (message not signed) header.d=none;xen.org; dmarc=none action=none header.from=arm.com;
  • Thread-index: AQHXLVmJnBz/2T+xZEqfD4R39q2wV6qwuDwAgAH9VwCAAPmtgIAADD+AgAHfkACAAAF1gIAF4LkA
  • Thread-topic: [PATCH v2] xen/pci: Refactor PCI MSI interrupts related code

Hi All,

> On 15 Apr 2021, at 2:31 pm, Julien Grall <julien@xxxxxxx> wrote:
> 
> Hi Roger,
> 
> On 15/04/2021 14:26, Roger Pau Monné wrote:
>> On Wed, Apr 14, 2021 at 09:49:37AM +0100, Julien Grall wrote:
>>> Hi Roger,
>>> 
>>> On 14/04/2021 09:05, Roger Pau Monné wrote:
>>>> On Tue, Apr 13, 2021 at 06:12:10PM +0100, Julien Grall wrote:
>>>>> Hi Roger,
>>>>> 
>>>>> On 12/04/2021 11:49, Roger Pau Monné wrote:
>>>>>> On Fri, Apr 09, 2021 at 05:00:41PM +0100, Rahul Singh wrote:
>>>>>>> diff --git a/xen/drivers/passthrough/pci.c 
>>>>>>> b/xen/drivers/passthrough/pci.c
>>>>>>> index 705137f8be..1b473502a1 100644
>>>>>>> --- a/xen/drivers/passthrough/pci.c
>>>>>>> +++ b/xen/drivers/passthrough/pci.c
>>>>>>> @@ -1303,12 +1279,15 @@ static int __init setup_dump_pcidevs(void)
>>>>>>>    }
>>>>>>>    __initcall(setup_dump_pcidevs);
>>>>>>> +
>>>>>>> +#ifdef CONFIG_PCI_MSI_INTERCEPT
>>>>>>>    int iommu_update_ire_from_msi(
>>>>>>>        struct msi_desc *msi_desc, struct msi_msg *msg)
>>>>>>>    {
>>>>>>>        return iommu_intremap
>>>>>>>               ? iommu_call(&iommu_ops, update_ire_from_msi, msi_desc, 
>>>>>>> msg) : 0;
>>>>>>>    }
>>>>>>> +#endif
>>>>>> 
>>>>>> This is not exactly related to MSI intercepts, the IOMMU interrupt
>>>>>> remapping table will also be used for interrupt entries for devices
>>>>>> being used by Xen directly, where no intercept is required.
>>>>> 
>>>>> On Arm, this is not tie to the IOMMU. Instead, this handled is a separate
>>>>> MSI controller (such as the ITS).
>>>>> 
>>>>>> 
>>>>>> And then you also want to gate the hook from iommu_ops itself with
>>>>>> CONFIG_PCI_MSI_INTERCEPT, if we want to got this route.
>>>>> 
>>>>> 
>>>>> All the callers are in the x86 code. So how about moving the function in 
>>>>> an
>>>>> x86 specific file?
>>>> 
>>>> Yes, that seems fine. Just place it in asm-x86/iommu.h. I wonder why
>>>> update_ire_from_msi wasn't moved when the rest of the x86 specific
>>>> functions where moved there.
>>> 
>>> I am guessing it is because the function was protected by CONFIG_HAS_PCI
>>> rather than CONFIG_X86. So it was deferred until another arch enables
>>> CONFIG_HAS_PCI (it is easier to know what code should be moved).
>>> 
>>>> Was there an aim to use that in other
>>>> arches?
>>> 
>>> In the future we may need to use MSIs in Xen (IIRC some SMMUs only support
>>> MSI interrupt).
>> Then I think some of the bits that are moved from
>> xen/drivers/passthrough/pci.c (alloc_pci_msi, free_pci_msi and
>> dump_pci_msi) need to be protected by a Kconfig option different than
>> CONFIG_PCI_MSI_INTERCEPT, as those are not related to MSI interception,
>> but to MSI handling in general (ie: Xen devices using MSI need
>> those).
> 
> Well... x86-specific devices yes. We don't know in what form MSI will be 
> added on for other arch-specific devices.
> 
> The same with the msi_list and msix fields of pci_dev, those
>> are not only used for MSI interception.
>> Or at least might be worth mentioning that the functions will be
>> needed for Xen to be able to use MSI interrupts itself, even if not
>> for interception purposes.
> 
> My preference would be to comment in the commit message (although, there is 
> no promise they will ever get re-used). We can then modify the #ifdef once we 
> introduce any use.
> 

Thanks you everyone for reviewing the code. I will summarise what I have 
understood from all the comments 
and what I will be doing for the next version of the patch.  Please let me know 
your view on this.

1. Create a separate non-arch specific file "msi-intercept.c"  for the below 
newly introduced function and 
    compile that file if CONFIG_PCI_MSI_INTERCEPT is 
enabled.CONFIG_PCI_MSI_INTERCEPT  will  be 
    enabled for x86 by default. Also Mention in the commit message that these 
function will be needed for Xen to 
    support MSI interrupt within XEN.

        pdev_msi_initi(..)
        pdev_msi_deiniti(..)
        pdev_dump_msi(..),
        pdev_msix_assign(..)

2. Create separate patch for iommu_update_ire_from_msi() related code. There 
are two suggestion please help me which one to choose. 
 
     - Move the iommu_update_ire_from_msi() function to asm-x86/iommu.h and 
also move the hook from iommu_ops under CONFIG_X86.

      - Implement a more generic function "arch_register_msi()"). This could 
call iommu_update_ire_from_msi() on x86 and the 
        ITS related code once implemented on Arm. Introduce the new Kconfig 
CONFIG_HAS_IOMMU_INTERRUPT_REMAP for this option.

3. As per the discussion I will gate the struct vpci_msi {..} and struct 
vpci_msix{..} using CONFIG_PCI_MSI_INTERCEPT so that 
    they will not will be accessible on ARM.

Regards,
Rahul
> Cheers,
> 
> -- 
> Julien Grall


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.