Re: [Xen-devel] Question about VPID during MOV-TO-CR3

[Tamas K Lengyel <tamas.lengyel@xxxxxxxxxxxx>]

On Wed, Sep 21, 2016 at 9:23 AM, Jan Beulich <JBeulich@xxxxxxxx> wrote:
>>>> On 21.09.16 at 17:16, <tamas.lengyel@xxxxxxxxxxxx> wrote:
>> On Wed, Sep 21, 2016 at 9:09 AM, Tamas K Lengyel
>> <tamas.lengyel@xxxxxxxxxxxx> wrote:
>>> On Wed, Sep 21, 2016 at 8:44 AM, Jan Beulich <JBeulich@xxxxxxxx> wrote:
>>>>>>> On 21.09.16 at 16:18, <tamas.lengyel@xxxxxxxxxxxx> wrote:
>>>>> On Wed, Sep 21, 2016 at 4:23 AM, Jan Beulich <JBeulich@xxxxxxxx> wrote:
>>>>>>>>> On 20.09.16 at 19:29, <tamas.lengyel@xxxxxxxxxxxx> wrote:
>>>>>>> I'm trying to figure out the design decision regarding the handling of
>>>>>>> guest MOV-TO-CR3 operations and TLB flushes. AFAICT since support for
>>>>>>> VPID has been added to Xen, every guest MOV-TO-CR3 flushes the TLB
>>>>>>> (vmx_cr_access -> hvm_mov_to_cr -> hvm_set_cr3 -> paging_update_cr3 ->
>>>>>>> hap_update_cr3 -> vmx_update_guest_cr -> hvm_asid_flush_vcpu). From a
>>>>>>> TLB utilization point-of-view this seems to be rather wasteful.
>>>>>>> Furthermore, it even breaks the guests' ability to take advantage of
>>>>>>> PCID, as the TLB just guts flushed when a new process is scheduled.
>>>>>>> Does anyone have an insight into what was the rationale behind this?
>>>>>>
>>>>>> Since you don't quote the specific commit(s), I would guess that
>>>>>> this was mainly an attempt by the author(s) to keep things simple
>>>>>> for themselves, i.e. not having to properly think through under
>>>>>> which conditions less than a full TLB flush would suffice.
>>>>>
>>>>> The commit that added VPID and the TLB flush is
>>>>> e2cf9bd6e055ea678da129b776f4521f6a0b50fe x86, vmx: Enable VPID
>>>>> (Virtual Processor Identification). So this has been there as long as
>>>>> Xen supported VPID. The only case where flushing the TLB on a guest
>>>>> MOV-TO-CR3 that possibly would make sense to me is if we are running a
>>>>> PV guest. But this is hvm/vmx, so why would we care about what the
>>>>> guest does to its cr3 from a TLB standpoint?
>>>>
>>>> Are you forgetting that a move to CR3 needs to flush all non-global
>>>> TLB entries? Or else, why do you think no flushing needs to happen
>>>> at all?
>>>>
>>>
>>> The guest can mark entries as global or non-global but it will have no
>>> affect on VPID, every translation is still going to be tagged by VPID
>>> when the translation was triggered in guest-context. So why does Xen
>>> need to jump in flush the TLB when the guest OS likely already done
>>> so?
>
> Likely? We can't base anything on likelihood (the more that no matter
> what flushing may have been done before the CR3 write, further
> flushing may be necessary and mustn't be skipped). We need to
> provide architecturally correct behavior, and that includes the flushing
> of non-global entries. This doesn't mean we need to flush anything
> ourselves, but we have to make previously created non-global TLB
> entries unavailable.

What I'm saying is that the guest OS should be in charge of managing
its own TLB when VPID is in use. Whether it does flush the TLB or not
is not of our concern. If it's a sane OS it will likely flush when it
needs to, but we should not be jumping in and doing it as we do right
now. We are actually breaking the architectural behavior by forcing a
flush, MOV-TO-CR3 doesn't by itself flush the TLB on real hardware.
Also, there are no non-global TLB entries we need to flush as long as
we are using VPID. Any translation used by Xen or by any other domain
will have a different VPID, so there is no chance of stale TLB entries
being an issue.

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel