[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: SR-IOV: do we need to virtualize in Xen or rely on Dom0?

  • To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@xxxxxxxx>
  • From: Jan Beulich <jbeulich@xxxxxxxx>
  • Date: Fri, 11 Jun 2021 08:45:35 +0200
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0CnMwqsHXT/LLouNSySeFUh/lzId/eX52WltUzhQgoQ=; b=QfqNGBUriTdHbIzUDJD1hcdWLutKNebNSZeTtIf/Z9C5lmyxxHTogBXghr6+vBI8sFIf2M8v9WBa9D5FIotiOvbPrFw0cEhUuTfhWSVBNN2GB8V/RDut77m2ohHXRyNtNfnbrYukCGAm4pU/0IpPeNPY3WZ/+fVvr9sXqKNLiE4GDy7rZ5Vy9V33O7f2vHfSdBCfCRLdIBLqUL2gis3eIVeXPMNpQYH6KZDu/C+h09eoR+DEZ9dMHMkbq/WMuKBRusRzicGvyQURrVg79b54g3i3WEsSbgxbSjtuQbWkgRGz3NoWEg0jYOiQS+DcxD5u85TjS3f/U373bMr9v1Minw==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IagxxOC8NCjbggPbZV7snHpulWJUdLCvqLZtVJS8sWdOPnUZK3Hw48JU/MKNMuz+3V69BMm73gKZtM6qCaOgVE+6uIgCwCU3lETMJaWvEkYHrKNRAPxjIan1lETvO7tCmX8rBE+3wHhPLwGdv1J8DJNNA+97kLOa8L2nOtPYnF9v4Fz15D+IIRyGzQIl+k8eUz8J/6+PsP50fUu6KdW0V9W1QywKhQLIR0SaAvg115Ieo37k+O5z6varZzOWz6JHjvWJGFa04Rzm10XpvRvdzi4ZLYS3OlXGQLsD+DKgrkwlDkJ5Ebmih34n65CXKLMAzpRXKuzMQfzJ7l/zlReO2Q==
  • Authentication-results: citrix.com; dkim=none (message not signed) header.d=none;citrix.com; dmarc=none action=none header.from=suse.com;
  • Cc: "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Julien Grall <julien@xxxxxxx>, Roger Pau Monné <roger.pau@xxxxxxxxxx>
  • Delivery-date: Fri, 11 Jun 2021 06:45:47 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On 10.06.2021 17:33, Oleksandr Andrushchenko wrote:
> On 10.06.21 17:10, Roger Pau Monné wrote:
>> On Thu, Jun 10, 2021 at 10:01:16AM +0000, Oleksandr Andrushchenko wrote:
>>> On 10.06.21 10:54, Roger Pau Monné wrote:
>>>> On Fri, Jun 04, 2021 at 06:37:27AM +0000, Oleksandr Andrushchenko wrote:
>>>>> Hi, all!
>>>>> While working on PCI SR-IOV support for ARM I started porting [1] on top
>>>>> of current PCI on ARM support [2]. The question I have for this series
>>>>> is if we really need emulating SR-IOV code in Xen?
>>>>> I have implemented a PoC for SR-IOV on ARM [3] (please see the top 2
>>>>> patches)
>>>>> and it "works for me": MSI support is still WIP, but I was able to see 
>>>>> that
>>>>> VFs are properly seen in the guest and BARs are properly programmed in 
>>>>> p2m.
>>>>> What I can't fully understand is if we can live with this approach or 
>>>>> there
>>>>> are use-cases I can't see.
>>>>> Previously I've been told that this approach might not work on FreeBSD
>>>>> running
>>>>> as Domain-0, but is seems that "PCI Passthrough is not supported
>>>>> (Xen/FreeBSD)"
>>>>> anyways [4].
>>>> PCI passthorgh is not supported on FreeBSD dom0 because PCI
>>>> passthrough is not supported by Xen itself when using a PVH dom0, and
>>>> that's the only mode FreeBSD dom0 can use.
>>> So, it is still not clear to me: how and if PCI passthrough is supported
>>> on FreeBSD, what are the scenarios and requirements for that?
>>>> PHYSDEVOP_pci_device_add can be added to FreeBSD, so it could be made
>>>> to work. I however think this is not the proper way to implement
>>>> SR-IOV support.
>>> I was not able to find any support for PHYSDEVOP_XXX in FreeBSD code,
>>> could you please point me to where are these used?
>> Those are not used on FreeBSD, because x86 PVHv2 dom0 doesn't
>> implement them anymore. They are implemented on Linux for x86 PV dom0,
>> AFAIK Arm doesn't use them either.
> Well, ARM didn't until we started implementing PCI passthrough [1].
> It was previously discussed [2], "# Discovering PCI devices:" and proposed
> to use PHYSDEVOP_pci_device_add.
> Long story short, it is not easy for ARM to enumerate PCI devices in Xen as 
> there is
> no unified way of doing so: different platforms implement different PCI host 
> bridges
> which require complex initialization including clocks, power domains etc.

Just for my own understanding: If this isn't done by firmware, doesn't
this mean you can't boot an Arm system from e.g. a disk connected through
a PCI-based controller? Host bridge setup is definitely firmware's job on
x86 ...




Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.