Re: [RESEND PATCH v12 3/3] vpci/msix: Implement cleanup function for MSI-X

["Chen, Jiqian" <Jiqian.Chen@xxxxxxx>]

On 2026/1/21 17:25, Roger Pau Monné wrote:
> On Mon, Dec 08, 2025 at 04:18:15PM +0800, Jiqian Chen wrote:
>> When MSI-X initialization fails, vPCI hides the capability, but
>> removing handlers and datas won't be performed until the device is
>> deassigned. So, implement MSI-X cleanup hook that will be called
>> to cleanup MSI-X related handlers and free it's associated data when
>> initialization fails.
>>
>> Since cleanup function of MSI-X is implemented, delete the open-code
>> in vpci_deassign_device().
>>
>> Signed-off-by: Jiqian Chen <Jiqian.Chen@xxxxxxx>
>> ---
>> cc: "Roger Pau Monné" <roger.pau@xxxxxxxxxx>
>> ---
>> v11->v12 changes:
>> * In cleanup_msix(), move check "if ( !hide )" above vpci_remove_registers().
>> * Remove the check "!pdev->msix_pos" since current callers already do that.
>>
>> v10->v11 changes:
>> * Move calling all cleanup hook in vpci_deassign_device() out of this patch.
>> * Add hide parameter to cleanup_msix().
>> * Check hide, if it is false, return directly instead of letting ctrl RO.
>>
>> v9->v10 changes:
>> * Call all cleanup hook in vpci_deassign_device() instead of cleanup_msix().
>>
>> v8->v9 changes:
>> * Modify commit message.
>> * Call cleanup_msix() in vpci_deassign_device() to remove the open-code to 
>> cleanup msix datas.
>> * In cleanup_msix(), move "list_del(&vpci->msix->next);" above for loop of 
>> iounmap msix tables.
>>
>> v7->v8 changes:
>> * Given the code in vpci_remove_registers() an error in the removal of
>>   registers would likely imply memory corruption, at which point it's
>>   best to fully disable the device. So, Rollback the last two modifications 
>> of v7.
>>
>> v6->v7 changes:
>> * Change the pointer parameter of cleanup_msix() to be const.
>> * When vpci_remove_registers() in cleanup_msix() fails, not to return
>>   directly, instead try to free msix and re-add ctrl handler.
>> * Pass pdev->vpci into vpci_add_register() instead of pdev->vpci->msix in
>>   init_msix() since we need that every handler realize that msix is NULL
>>   when msix is freed but handlers are still in there.
>>
>> v5->v6 changes:
>> * Change the logic to add dummy handler when !vpci->msix in cleanup_msix().
>>
>> v4->v5 changes:
>> * Change definition "static void cleanup_msix" to "static int cf_check 
>> cleanup_msix"
>>   since cleanup hook is changed to be int.
>> * Add a read-only register for MSIX Control Register in the end of 
>> cleanup_msix().
>>
>> v3->v4 changes:
>> * Change function name from fini_msix() to cleanup_msix().
>> * Change to use XFREE to free vpci->msix.
>> * In cleanup function, change the sequence of check and remove action 
>> according to
>>   init_msix().
>>
>> v2->v3 changes:
>> * Remove unnecessary clean operations in fini_msix().
>>
>> v1->v2 changes:
>> new patch.
>>
>> Best regards,
>> Jiqian Chen.
>> ---
>>  xen/drivers/vpci/msix.c | 44 ++++++++++++++++++++++++++++++++++++++++-
>>  xen/drivers/vpci/vpci.c |  8 --------
>>  2 files changed, 43 insertions(+), 9 deletions(-)
>>
>> diff --git a/xen/drivers/vpci/msix.c b/xen/drivers/vpci/msix.c
>> index 032e471bb1c0..8dcf2cf9d598 100644
>> --- a/xen/drivers/vpci/msix.c
>> +++ b/xen/drivers/vpci/msix.c
>> @@ -656,6 +656,48 @@ int vpci_make_msix_hole(const struct pci_dev *pdev)
>>      return 0;
>>  }
>>  
>> +static int cf_check cleanup_msix(const struct pci_dev *pdev, bool hide)
>> +{
>> +    int rc;
>> +    struct vpci *vpci = pdev->vpci;
>> +    const unsigned int msix_pos = pdev->msix_pos;
>> +
>> +    if ( vpci->msix )
>> +    {
>> +        list_del(&vpci->msix->next);
>> +        for ( unsigned int i = 0; i < ARRAY_SIZE(vpci->msix->table); i++ )
>> +            if ( vpci->msix->table[i] )
>> +                iounmap(vpci->msix->table[i]);
>> +
>> +        XFREE(vpci->msix);
>> +    }
>> +
>> +    if ( !hide )
>> +        return 0;
>> +
>> +    rc = vpci_remove_registers(vpci, msix_control_reg(msix_pos), 2);
>> +    if ( rc )
>> +    {
>> +        printk(XENLOG_ERR "%pd %pp: fail to remove MSIX handlers rc=%d\n",
>> +               pdev->domain, &pdev->sbdf, rc);
>> +        ASSERT_UNREACHABLE();
>> +        return rc;
>> +    }
>> +
>> +    /*
>> +     * The driver may not traverse the capability list and think device
>> +     * supports MSIX by default. So here let the control register of MSIX
>> +     * be Read-Only is to ensure MSIX disabled.
>> +     */
>> +    rc = vpci_add_register(vpci, vpci_hw_read16, NULL,
>> +                           msix_control_reg(msix_pos), 2, NULL);
>> +    if ( rc )
>> +        printk(XENLOG_ERR "%pd %pp: fail to add MSIX ctrl handler rc=%d\n",
>> +               pdev->domain, &pdev->sbdf, rc);
> 
> Like the previous patch, I don't think this last bit is relevant for
> domUs?  Only the hardware domain needs to have the control register
> explicitly handled.
> 
> I don't mind adjusting at commit if we agree.
I agree with you.
Thank you for help to make changes of this and previous patch when you submit.

> 
> Reviewed-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>
> 
> Thanks, Roger.

-- 
Best regards,
Jiqian Chen.