Re: [PATCH v12.2 01/15] vpci: use per-domain PCI lock to protect vpci structure

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Thu, Jan 25, 2024 at 08:43:05AM +0100, Jan Beulich wrote:
> On 24.01.2024 18:51, Roger Pau Monné wrote:
> > On Wed, Jan 24, 2024 at 12:34:10PM +0100, Jan Beulich wrote:
> >> On 24.01.2024 10:24, Roger Pau Monné wrote:
> >>> On Wed, Jan 24, 2024 at 09:48:35AM +0100, Jan Beulich wrote:
> >>>> On 23.01.2024 16:07, Roger Pau Monné wrote:
> >>>>> On Tue, Jan 23, 2024 at 03:32:12PM +0100, Jan Beulich wrote:
> >>>>>> On 15.01.2024 20:43, Stewart Hildebrand wrote:
> >>>>>>> @@ -2888,6 +2888,8 @@ int allocate_and_map_msi_pirq(struct domain *d, 
> >>>>>>> int index, int *pirq_p,
> >>>>>>>  {
> >>>>>>>      int irq, pirq, ret;
> >>>>>>>  
> >>>>>>> +    ASSERT(pcidevs_locked() || rw_is_locked(&d->pci_lock));
> >>>>>>
> >>>>>> If either lock is sufficient to hold here, ...
> >>>>>>
> >>>>>>> --- a/xen/arch/x86/physdev.c
> >>>>>>> +++ b/xen/arch/x86/physdev.c
> >>>>>>> @@ -123,7 +123,9 @@ int physdev_map_pirq(domid_t domid, int type, int 
> >>>>>>> *index, int *pirq_p,
> >>>>>>>  
> >>>>>>>      case MAP_PIRQ_TYPE_MSI:
> >>>>>>>      case MAP_PIRQ_TYPE_MULTI_MSI:
> >>>>>>> +        pcidevs_lock();
> >>>>>>>          ret = allocate_and_map_msi_pirq(d, *index, pirq_p, type, 
> >>>>>>> msi);
> >>>>>>> +        pcidevs_unlock();
> >>>>>>>          break;
> >>>>>>
> >>>>>
> >>>>> IIRC (Stewart can further comment) this is done holding the pcidevs
> >>>>> lock to keep the path unmodified, as there's no need to hold the
> >>>>> per-domain rwlock.
> >>>>
> >>>> Yet why would we prefer to acquire a global lock when a per-domain one
> >>>> suffices?
> >>>
> >>> I was hoping to introduce less changes, specially if they are not
> >>> strictly required, as it's less risk.  I'm always quite worry of
> >>> locking changes.
> >>
> >> In which case more description / code commenting is needed. The pattern
> >> of the assertions looks dangerous.
> > 
> > Is such dangerousness perception because you fear some of the pcidevs
> > lock usage might be there not just for preventing the pdev from going
> > away, but also to guarantee exclusive access to certain state?
> 
> Indeed. In my view the main purpose of locks is to guard state. Their
> use here to guard against devices here is imo rather an abuse; as
> mentioned before this should instead be achieved e.g via refcounting.
> And it's bad enough already that pcidevs_lock() alone has been abused
> this way, without proper marking (leaving us to guess in many places).
> It gets worse when a second lock can now also serve this same
> purpose.

The new lock is taken in read mode in most contexts, and hence can't
be used to indirectly gain exclusive access to domain related
structures in a safe way.

Not saying this makes it any better, but so far this is the best
solution we could come up with that didn't involve a full evaluation
and possible re-write of all usage of the pcidevs lock.

I would also prefer a solution that fully replaces the pcidevs lock
with something else, but for once I don't have a clear picture of how
that would look like because the analysis is a huge amount of work,
likely more than the implementation itself.

Hence the proposed compromise solution that should allow the vPCI work
to make progress.

Regards, Roger.