Re: [Xen-devel] Question about VPID during MOV-TO-CR3

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 21.09.16 at 17:16, <tamas.lengyel@xxxxxxxxxxxx> wrote:
> On Wed, Sep 21, 2016 at 9:09 AM, Tamas K Lengyel
> <tamas.lengyel@xxxxxxxxxxxx> wrote:
>> On Wed, Sep 21, 2016 at 8:44 AM, Jan Beulich <JBeulich@xxxxxxxx> wrote:
>>>>>> On 21.09.16 at 16:18, <tamas.lengyel@xxxxxxxxxxxx> wrote:
>>>> On Wed, Sep 21, 2016 at 4:23 AM, Jan Beulich <JBeulich@xxxxxxxx> wrote:
>>>>>>>> On 20.09.16 at 19:29, <tamas.lengyel@xxxxxxxxxxxx> wrote:
>>>>>> I'm trying to figure out the design decision regarding the handling of
>>>>>> guest MOV-TO-CR3 operations and TLB flushes. AFAICT since support for
>>>>>> VPID has been added to Xen, every guest MOV-TO-CR3 flushes the TLB
>>>>>> (vmx_cr_access -> hvm_mov_to_cr -> hvm_set_cr3 -> paging_update_cr3 ->
>>>>>> hap_update_cr3 -> vmx_update_guest_cr -> hvm_asid_flush_vcpu). From a
>>>>>> TLB utilization point-of-view this seems to be rather wasteful.
>>>>>> Furthermore, it even breaks the guests' ability to take advantage of
>>>>>> PCID, as the TLB just guts flushed when a new process is scheduled.
>>>>>> Does anyone have an insight into what was the rationale behind this?
>>>>>
>>>>> Since you don't quote the specific commit(s), I would guess that
>>>>> this was mainly an attempt by the author(s) to keep things simple
>>>>> for themselves, i.e. not having to properly think through under
>>>>> which conditions less than a full TLB flush would suffice.
>>>>
>>>> The commit that added VPID and the TLB flush is
>>>> e2cf9bd6e055ea678da129b776f4521f6a0b50fe x86, vmx: Enable VPID
>>>> (Virtual Processor Identification). So this has been there as long as
>>>> Xen supported VPID. The only case where flushing the TLB on a guest
>>>> MOV-TO-CR3 that possibly would make sense to me is if we are running a
>>>> PV guest. But this is hvm/vmx, so why would we care about what the
>>>> guest does to its cr3 from a TLB standpoint?
>>>
>>> Are you forgetting that a move to CR3 needs to flush all non-global
>>> TLB entries? Or else, why do you think no flushing needs to happen
>>> at all?
>>>
>>
>> The guest can mark entries as global or non-global but it will have no
>> affect on VPID, every translation is still going to be tagged by VPID
>> when the translation was triggered in guest-context. So why does Xen
>> need to jump in flush the TLB when the guest OS likely already done
>> so?

Likely? We can't base anything on likelihood (the more that no matter
what flushing may have been done before the CR3 write, further
flushing may be necessary and mustn't be skipped). We need to
provide architecturally correct behavior, and that includes the flushing
of non-global entries. This doesn't mean we need to flush anything
ourselves, but we have to make previously created non-global TLB
entries unavailable.

>> It will render the guest OS's use of PCID optimization useless.
>> But even if the guest OS didn't flush - for whatever strange reason -
>> it would have no effect on anything else outside the guest context, so
>> Xen jumping in and doing this flush is unwarranted AFAICT.
> 
> Also, Xen flushing on every MOV-TO-CR3 effectively disables the use of
> global TLB entries in the guest as well. So both global TLB entries
> and TLB entries tagged with PCID are disabled with this flush in
> place. That seems to be a bad idea from a performance perspective..

I didn't say what gets done right now looks to be optimal.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel