[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v3 5/7] vpci: add SR-IOV support for PVH Dom0


  • To: Roger Pau Monné <roger.pau@xxxxxxxxxx>
  • From: Jan Beulich <jbeulich@xxxxxxxx>
  • Date: Tue, 12 May 2026 12:44:48 +0200
  • Authentication-results: eu.smtp.expurgate.cloud; dkim=pass header.s=google header.d=suse.com header.i="@suse.com" header.h="Content-Transfer-Encoding:In-Reply-To:Autocrypt:From:Content-Language:References:Cc:To:Subject:User-Agent:MIME-Version:Date:Message-ID"
  • Autocrypt: addr=jbeulich@xxxxxxxx; keydata= xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A nAuWpQkjM1ASeQwSHEeAWPgskBQL
  • Cc: Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>, "Daniel P. Smith" <dpsmith@xxxxxxxxxxxxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxxx>, Stewart Hildebrand <stewart.hildebrand@xxxxxxx>, Mykyta Poturai <Mykyta_Poturai@xxxxxxxx>
  • Delivery-date: Tue, 12 May 2026 10:44:57 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On 12.05.2026 10:58, Roger Pau Monné wrote:
> On Tue, May 12, 2026 at 07:32:20AM +0000, Mykyta Poturai wrote:
>> On 5/12/26 09:20, Jan Beulich wrote:
>>> On 11.05.2026 16:10, Volodymyr Babchuk wrote:
>>>> Okay, so let's clear this. If I remember correct, you discussed this
>>>> with Mykyta in the previous version and suggested to put the vCPU to
>>>> sleep for 100ms.
>>>
>>> I don't think I did (except perhaps from a very abstract perspective),
>>> precisely because of ...
>>>
>>>> I don't think that this is a good idea, because guest
>>>> kernel will not be happy about that.
>>>
>>> ... this. Instead iirc I suggested to refuse (short-circuit) handling
>>> VF register accesses for the next 100ms.
>>
>> Do you have any suggestions on how to ensure that we accurately catch 
>> the window where 100ms have already passed, but guests haven’t tried to 
>> read anything yet, to flip this back? As I mentioned in the previous 
>> version, Linux, for example, doesn’t attempt to re-read anything if the 
>> first read failed after 100ms. So it appears to me that this approach 
>> would be prone to racing with the guest for getting to the VF first.

When we do the write to the control register in Xen, our timer will start
ticking before the guest's. Hence our 100ms will be over (slightly)
earlier, and a well-behaved guest (having waited for the full 100ms
according to its own tracking) will be handled fine.

>> One 
>> approach I can think of is to somehow swap the register handlers back 
>> in-flight during the first read by the guest if 100ms have already 
>> passed. However, this would still depend on Dom0 for registering VFs, 
>> but in a more convoluted way. We also can’t add the VFs before 100ms 
>> have passed and add timing checks to all register handlers, because 
>> pci_add_device and everything below it expects the device to be 
>> functional at the moment of addition.

I fear I'm not following this.

> We could maybe do some middle ground here, kind of similar to what
> Linux does.  The overall idea would be to put on hold any accesses to
> the device(s) PCI config space for 100ms, that would include the PF
> and any VFs.

For the PF, at most parts of the SR-IOV capability should be thus
constrained, I think.

>  At the point when VF enable is set Xen already knows the
> position of the VFs in the PCI config space.
> 
> Any PCI config space access attempts to the PF or VFs during that
> 100ms window would cause the guest vCPU to be put on hold, and the
> access would only be retried once the 100ms window has passed and Xen
> has registered the VFs with vPCI.  This approach needs extra logic to
> put vPCI accesses on hold, similar to what Xen does when mapping a BAR
> into the p2m, and a timer to defer the adding of the Vfs and the
> unlocking of the affected PCI config space region.

I was meaning to have this done in even simpler a way: Simply record
when the VFs were configured, and within the next 100ms terminate all
accesses (read all ones, discard writes).

Jan



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.