Re: [Xen-devel] [PATCH v3 7/7] x86/tlb: use Xen L0 assisted TLB flush when available

[Jan Beulich <jbeulich@xxxxxxxx>]

On 06.02.2020 15:09, Roger Pau Monné wrote:
> On Thu, Feb 06, 2020 at 01:49:35PM +0000, Wei Liu wrote:
>> On Mon, Jan 27, 2020 at 07:11:15PM +0100, Roger Pau Monne wrote:
>>> Use Xen's L0 HVMOP_flush_tlbs hypercall in order to perform flushes.
>>> This greatly increases the performance of TLB flushes when running
>>> with a high amount of vCPUs as a Xen guest, and is specially important
>>> when running in shim mode.
>>>
>>> The following figures are from a PV guest running `make -j32 xen` in
>>> shim mode with 32 vCPUs and HAP.
>>>
>>> Using x2APIC and ALLBUT shorthand:
>>> real        4m35.973s
>>> user        4m35.110s
>>> sys 36m24.117s
>>>
>>> Using L0 assisted flush:
>>> real    1m2.596s
>>> user    4m34.818s
>>> sys     5m16.374s
>>>
>>> The implementation adds a new hook to hypervisor_ops so other
>>> enlightenments can also implement such assisted flush just by filling
>>> the hook. Note that the Xen implementation completely ignores the
>>> dirty CPU mask and the linear address passed in, and always performs a
>>> global TLB flush on all vCPUs.
>>>
>>> Signed-off-by: Roger Pau Monné <roger.pau@xxxxxxxxxx>
>>> ---
>>> Changes since v1:
>>>  - Add a L0 assisted hook to hypervisor ops.
>>> ---
>>>  xen/arch/x86/guest/hypervisor.c        | 10 ++++++++++
>>>  xen/arch/x86/guest/xen/xen.c           |  6 ++++++
>>>  xen/arch/x86/smp.c                     | 11 +++++++++++
>>>  xen/include/asm-x86/guest/hypervisor.h | 17 +++++++++++++++++
>>>  4 files changed, 44 insertions(+)
>>>
>>> diff --git a/xen/arch/x86/guest/hypervisor.c 
>>> b/xen/arch/x86/guest/hypervisor.c
>>> index 4f27b98740..4085b19734 100644
>>> --- a/xen/arch/x86/guest/hypervisor.c
>>> +++ b/xen/arch/x86/guest/hypervisor.c
>>> @@ -18,6 +18,7 @@
>>>   *
>>>   * Copyright (c) 2019 Microsoft.
>>>   */
>>> +#include <xen/cpumask.h>
>>>  #include <xen/init.h>
>>>  #include <xen/types.h>
>>>  
>>> @@ -64,6 +65,15 @@ void hypervisor_resume(void)
>>>          ops->resume();
>>>  }
>>>  
>>> +int hypervisor_flush_tlb(const cpumask_t *mask, const void *va,
>>> +                         unsigned int order)
>>> +{
>>> +    if ( ops && ops->flush_tlb )
>>> +        return ops->flush_tlb(mask, va, order);
>>> +
>>
>> Is there a way to make this an alternative call? I consider tlb flush a
>> frequent operation which can use some optimisation.
>>
>> This can be done as a later improvement if it is too difficult though.
>> This patch already has some substantial improvement.
> 
> I can look into making this an alternative call, if it turn out to be
> too complex I will leave it out for a separate patch.

It'll be two steps - make a global struct hypervisor_ops instance
which the per-hypervisor instances get _copied_ into upon boot
(at that point all of those can go into .init.* sections), and
then switch the call(s) of interest. I.e. while the 2nd step can
of course be done right here, the first will want to be in a
prereq patch.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel