[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH 06/10] vpci: Make every domain handle its own BARs


  • To: Jan Beulich <jbeulich@xxxxxxxx>
  • From: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@xxxxxxxx>
  • Date: Mon, 7 Dec 2020 09:11:03 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HMZ/N49DrrxhTR+8FCsgdlxamziIgd/xJSAtrvvk0nE=; b=iZ7f/IoY1pL7ict6ZsOaLUzYyg5DbKEZIG3qehUAAHeKYhYI8PZBL4i0NW7k9k6JpaY+9O7sYp2Q9agFO4ecLJ2aLwgl9pjSis74Ca/k/U2+nPVKTxlIPjdasanL9YT9rIP1mHcauj5Lc3Fro4UA6w2ay1TC+72tb+tPI0gqpQvkgLyiNYTH18LLjRlXhbcxmqPfhWDqkjtEuug7/U3VREmOZLyywWxPVs9cuwHEeQwh3aTHHEEAn1Bh/ZwZcePNHicOhjdezpRhShgAXqpfJCxO7ZjPouVJINIX0a2r17Vv56NiUkJTKY18w4An8JA/voW9BFZ7ZTK6hg8r/X1MFw==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=B8bHkUR1Z6chhQi/mAE/17Y9INbtRa/Xq4V4QqEujtAYz9BqvryYFZsHuz7dnZMIVM/k0q6WjPOFnUqyE6Q7ZQOJ48vqg9hoo7aisIjc3I4ddhNdOHU6vPJS3eIXciSjdYBWC1laqS9wfTEVg70LA7bHMbEV1XYXNuNo7TGahBHxwdVij4e++q6z0xu41bIyazo0Ml6Lag+NM4cm6sG8Qi7/kiTwKiSRI6Pcne/pH4P3HPm570f3BVHxAgQq8B2KmHzUhKN64D/PzEFj17uvTVOTgvggpVEfaJVk/VT968SlyG1pGaO5uoIvpsxdh72xNeTRIKsflF8cysnl0IWl7g==
  • Authentication-results: suse.com; dkim=none (message not signed) header.d=none;suse.com; dmarc=none action=none header.from=epam.com;
  • Cc: Oleksandr Andrushchenko <andr2000@xxxxxxxxx>, "Rahul.Singh@xxxxxxx" <Rahul.Singh@xxxxxxx>, "Bertrand.Marquis@xxxxxxx" <Bertrand.Marquis@xxxxxxx>, "julien.grall@xxxxxxx" <julien.grall@xxxxxxx>, "sstabellini@xxxxxxxxxx" <sstabellini@xxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>, "iwj@xxxxxxxxxxxxxx" <iwj@xxxxxxxxxxxxxx>, "wl@xxxxxxx" <wl@xxxxxxx>, Roger Pau Monné <roger.pau@xxxxxxxxxx>
  • Delivery-date: Mon, 07 Dec 2020 09:11:26 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Thread-index: AQHWtpbzyJg77SbpvkSRWLKjWy/3PanEQmgAgAA8YoCAABlOgIABCD8AgABBOQCAAAXIAIAAAS0AgAADNgCAAAlOAIAAEnYAgAAcfoCAAAKRAIAAAYQAgAABxICAAAHUgIAg/YsAgARU+oCAAAZpgA==
  • Thread-topic: [PATCH 06/10] vpci: Make every domain handle its own BARs

On 12/7/20 10:48 AM, Jan Beulich wrote:
> On 04.12.2020 15:38, Oleksandr Andrushchenko wrote:
>> On 11/13/20 4:51 PM, Jan Beulich wrote:
>>> On 13.11.2020 15:44, Oleksandr Andrushchenko wrote:
>>>> On 11/13/20 4:38 PM, Jan Beulich wrote:
>>>>> On 13.11.2020 15:32, Oleksandr Andrushchenko wrote:
>>>>>> On 11/13/20 4:23 PM, Jan Beulich wrote:
>>>>>>>      Earlier on I didn't say you should get this to work, only
>>>>>>> that I think the general logic around what you add shouldn't make
>>>>>>> things more arch specific than they really should be. That said,
>>>>>>> something similar to the above should still be doable on x86,
>>>>>>> utilizing struct pci_seg's bus2bridge[]. There ought to be
>>>>>>> DEV_TYPE_PCI_HOST_BRIDGE entries there, albeit a number of them
>>>>>>> (provided by the CPUs themselves rather than the chipset) aren't
>>>>>>> really host bridges for the purposes you're after.
>>>>>> You mean I can still use DEV_TYPE_PCI_HOST_BRIDGE as a marker
>>>>>>
>>>>>> while trying to detect what I need?
>>>>> I'm afraid I don't understand what marker you're thinking about
>>>>> here.
>>>> I mean that when I go over bus2bridge entries, should I only work with
>>>>
>>>> those who have DEV_TYPE_PCI_HOST_BRIDGE?
>>> Well, if you're after host bridges - yes.
>>>
>>> Jan
>> So, I started looking at the bus2bridge and if it can be used for both x86 
>> (and possible ARM) and I have an
>>
>> impression that the original purpose of this was to identify the devices 
>> which x86 IOMMU should
>>
>> cover: e.g. I am looking at the find_upstream_bridge users and they are x86 
>> IOMMU and VGA driver.
>>
>> I tried to play with this on ARM, and for the HW platform I have and QEMU I 
>> got 0 entries in bus2bridge...
>>
>> This is because of how xen/drivers/passthrough/pci.c:alloc_pdev is 
>> implemented (which lives in the
>>
>> common code BTW, but seems to be x86 specific): so, it skips setting up 
>> bus2bridge entries for the bridges I have.
> I'm curious to learn what's x86-specific here. I also can't deduce
> why bus2bridge setup would be skipped.

So, for example:

commit 0af438757d455f8eb6b5a6ae9a990ae245f230fd
Author: Suravee Suthikulpanit <suravee.suthikulpanit@xxxxxxx>
Date:   Fri Sep 27 10:11:49 2013 +0200

     AMD IOMMU: fix Dom0 device setup failure for host bridges

     The host bridge device (i.e. 0x18 for AMD) does not require IOMMU, and
     therefore is not included in the IVRS. The current logic tries to map
     all PCI devices to an IOMMU. In this case, "xl dmesg" shows the
     following message on AMD sytem.

     (XEN) setup 0000:00:18.0 for d0 failed (-19)
     (XEN) setup 0000:00:18.1 for d0 failed (-19)
     (XEN) setup 0000:00:18.2 for d0 failed (-19)
     (XEN) setup 0000:00:18.3 for d0 failed (-19)
     (XEN) setup 0000:00:18.4 for d0 failed (-19)
     (XEN) setup 0000:00:18.5 for d0 failed (-19)

     This patch adds a new device type (i.e. DEV_TYPE_PCI_HOST_BRIDGE) which
     corresponds to PCI class code 0x06 and sub-class 0x00. Then, it uses
     this new type to filter when trying to map device to IOMMU.

One of my test systems has DEV_TYPE_PCI_HOST_BRIDGE, so bus2brdige setup is 
ignored

>
>> These are DEV_TYPE_PCIe_BRIDGE and DEV_TYPE_PCI_HOST_BRIDGE. So, the 
>> assumption I've made before
>>
>> that I can go over bus2bridge and filter out the "owner" or parent bridge 
>> for a given seg:bus doesn't
>>
>> seem to be possible now.
>>
>> Even if I find the parent bridge with 
>> xen/drivers/passthrough/pci.c:find_upstream_bridge I am not sure
>>
>> I can get any further in detecting which x86 domain owns this bridge: the 
>> whole idea in the x86 code is
>>
>> that Domain-0 is the only possible one here (you mentioned that).
> Right - your attempt to find the owner should _right now_ result in
> getting back Dom0, on x86. But there shouldn't be any such assumption
> in the new consumer of this data that you mean to introduce. I.e. I
> only did suggest this to avoid ...
>
>> So, I doubt if we can still live with
>>
>> is_hardware_domain for now for x86?
> ... hard-coding information which can be properly established /
> retrieved.
>
> Jan

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.