Re: [Xen-devel] [RFC PATCH v2 07/26] ARM: GICv3 ITS: introduce host LPI array

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Fri, 20 Jan 2017, Julien Grall wrote:
> Hi Stefano,
> 
> Sorry for the late answer, still going through my e-mail backlog.
> 
> On 06/01/2017 21:20, Stefano Stabellini wrote:
> > On Fri, 6 Jan 2017, Andre Przywara wrote:
> > > > It is also possible to end up calling mapti with an inexistent eventid
> > > > for host_devid. Could that be a problem?
> > > 
> > > Not at all. Actually there is no such thing as a "nonexistent event ID",
> > > because the event ID will be written by the device as the payload to the
> > > MSI doorbell address, probably because it learned about it by the
> > > driver. So if we provision an ITTE with an event ID which the device
> > > will never send, that LPI will just never fire.
> > > Since Xen (in contrast to the driver in the domain) has no idea how many
> > > and which MSIs the device will use, we just allocate a bunch of them.
> > > The upper limit (32 atm) is something we probably need to still think
> > > about, though.
> > > I tried to learn a limit from Linux ("nvecs" in its_create_device()
> > > seems to be the source), but couldn't find anything useful other than 32.
> > > We will learn about exceeding this as soon as a domain tries to map a
> > > virtual LPI with an event ID higher than 31, however it's too late to
> > > fix this then. We can bark when this happens to learn if any device ever
> > > does this during our testing to get some heuristic data.
> > > 
> > > Eventually all boils down to Xen getting more information from Dom0
> > > about the required number of MSIs. We could then even limit the
> > > allocation to less than 32, if that helps.
> > 
> > Originally Julien and I thought that Xen should map events up to the
> > theoretically maximum for each device, but we realized that they were
> > too many: an MSI-X capable device can generate up to 2048 different
> > events.
> > 
> > Xen needs to find out the exact number of events for each device. The
> > information can either be provided by the guest, or the hypervisor needs
> > to figure it out on its own.
> > 
> > With Julien's PCI Passthrough work, Xen will become able to read the
> > amount of events a device is capable of generating, so in the long term
> > this problem should be easy to solve. But Julien's work might land one
> > or two Xen releases after ITS.
> > 
> > In the meantime, we can extend an existing PHYSDEVOP hypercall or add a
> > new one. Julien, do you agree?
> 
> PHYSDEVOP are stable ABI and some are already being in used by Linux even on
> ARM.
> 
> I am not in favor of adding a PHYSDEVOP which may not be necessary in a couple
> of the release. Furthermore, there are already some issues with how device are
> being added in the ITS. Indeed, the PHYSDEVOP operations will provide the RID
> (can be deduced from the BDF), but the deviceID may not be equal to the RID on
> some platform. This is where IORT (on ACPI) and msi-map (on DT) comes from.
> The plumbing will be added during the PCI work.
> 
> So I would be more in favor to hardcode the device info (DeviceID, number of
> max number of MSIs) per platform until we get PCI passthrough working.

I think that would be OK for a first ITS implementation in Xen. We can
defer the physdevop to later, after you complete PCI passthrough. I also
agree that we'll probably end up with a physdevop anyway, but at least
we'll have a better idea about what we need.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel