Re: [PATCH 02/10] arm/pci: Maintain PCI assignable list

[Oleksandr Andrushchenko <Oleksandr_Andrushchenko@xxxxxxxx>]

On 11/11/20 5:03 PM, Roger Pau Monné wrote:
> On Wed, Nov 11, 2020 at 02:38:47PM +0000, Oleksandr Andrushchenko wrote:
>> On 11/11/20 3:53 PM, Roger Pau Monné wrote:
>>> On Mon, Nov 09, 2020 at 02:50:23PM +0200, Oleksandr Andrushchenko wrote:
>>>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@xxxxxxxx>
>>>>
>>>> The original code depends on pciback to manage assignable device list.
>>>> The functionality which is implemented by the pciback and the toolstack
>>>> and which is relevant/missing/needed for ARM:
>>>>
>>>> 1. pciback is used as a database for assignable PCI devices, e.g. xl
>>>>      pci-assignable-{add|remove|list} manipulates that list. So, whenever 
>>>> the
>>>>      toolstack needs to know which PCI devices can be passed through it 
>>>> reads
>>>>      that from the relevant sysfs entries of the pciback.
>>>>
>>>> 2. pciback is used to hold the unbound PCI devices, e.g. when passing 
>>>> through
>>>>      a PCI device it needs to be unbound from the relevant device driver 
>>>> and bound
>>>>      to pciback (strictly speaking it is not required that the device is 
>>>> bound to
>>>>      pciback, but pciback is again used as a database of the passed 
>>>> through PCI
>>>>      devices, so we can re-bind the devices back to their original drivers 
>>>> when
>>>>      guest domain shuts down)
>>>>
>>>> 1. As ARM doesn't use pciback implement the above with additional sysctls:
>>>>    - XEN_SYSCTL_pci_device_set_assigned
>>> I don't see the point in having this sysfs, Xen already knows when a
>>> device is assigned because the XEN_DOMCTL_assign_device hypercall is
>>> used.
>> But how does the toolstack know about that? When the toolstack needs to
>>
>> list/know all assigned devices it queries pciback's sysfs entries. So, with
>>
>> XEN_DOMCTL_assign_device we make that knowledge available to Xen,
>>
>> but there are no means for the toolstack to get it back.
> But the toolstack will figure out whether a device is assigned or
> not by using
> XEN_SYSCTL_pci_device_get_assigned/XEN_SYSCTL_pci_device_enum_assigned?
>
> AFAICT XEN_SYSCTL_pci_device_set_assigned tells Xen a device has been
> assigned, but Xen should already know it because
> XEN_DOMCTL_assign_device would have been used to assign the device?

Ah, I misunderstood you then. So, we only want to drop XEN_DOMCTL_assign_device

and keep the rest.

>
>>>>    - XEN_SYSCTL_pci_device_get_assigned
>>>>    - XEN_SYSCTL_pci_device_enum_assigned
>>>> 2. Extend struct pci_dev to hold assignment state.
>>> I'm not really found of this, the hypervisor is no place to store a
>>> database like this, unless it's strictly needed.
>> I do agree and it was previously discussed a bit
>>> IMO the right implementation here would be to split Linux pciback into
>>> two different drivers:
>>>
>>>    - The pv-pci backend for doing passthrough to classic PV guests.
>> Ok
>>>    - The rest of pciback: device reset, hand-holding driver for devices
>>>      to be assigned and database.
>> These and assigned devices list seem to be the complete set which
>>
>> is needed by the toolstack on ARM. All other functionality provided by
>>
>> pciback is not needed for ARM.
>>
>> Jan was saying [1] that we might still use pciback as is, but simply use only
>>
>> the functionality we need.
>>
>>> I think there must be something similar in KVM that performs the tasks
>>> of my last point, maybe we could piggyback on it?
>> I promised to look at it. I owe this
>>> If we want to go the route proposed by this patch, ie: Xen performing
>>> the functions of pciback you would also have to move the PCI reset
>>> code to Xen, so that you can fully manage the PCI devices from Xen.
>> In case of dom0less this would be the case: no pciback, no Domain-0
> But for dom0less there's no need for any database of assignable
> devices, nor the need to perform pci device resets, as it's all
> assigned at boot time and then never modified?
You are right
>
> Roger.

Thank you,

Oleksandr