Re: [Xen-devel] stale TLB contents?

To:	Tim Deegan <Tim.Deegan@xxxxxxxxxx>
Subject:	Re: [Xen-devel] stale TLB contents?
From:	Juergen Gross <juergen.gross@xxxxxxxxxxxxxx>
Date:	Mon, 24 Jan 2011 14:29:42 +0100
Cc:	"xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Delivery-date:	Mon, 24 Jan 2011 05:30:31 -0800
Dkim-signature:	v=1; a=rsa-sha256; c=simple/simple; d=ts.fujitsu.com; i=juergen.gross@xxxxxxxxxxxxxx; q=dns/txt; s=s1536b; t=1295875784; x=1327411784; h=message-id:date:from:mime-version:to:cc:subject: references:in-reply-to:content-transfer-encoding; z=Message-ID:=20<4D3D7EC6.3060000@xxxxxxxxxxxxxx>\|Date:=20 Mon,=2024=20Jan=202011=2014:29:42=20+0100\|From:=20Juergen =20Gross=20<juergen.gross@xxxxxxxxxxxxxx>\|MIME-Version: =201.0\|To:=20Tim=20Deegan=20<Tim.Deegan@xxxxxxxxxx>\|CC: =20"xen-devel@xxxxxxxxxxxxxxxxxxx"=20<xen-devel@xxxxxxxxx source.com>\|Subject:=20Re:=20[Xen-devel]=20stale=20TLB=20 contents?\|References:=20<4D3D780A.4030001@xxxxxxxxxxxxxx> =20<20110124131340.GN8286@xxxxxxxxxxxxxxxxxxxxxxx> \|In-Reply-To:=20<20110124131340.GN8286@xxxxxxxxxxxxxxxxxx e.com>\|Content-Transfer-Encoding:=207bit; bh=6+uklFtse7/2qZ1EWx2w938vx2uWtA6LaTSDR4ow4MI=; b=Rw+3JUsJGp06Q8ctdvXrpNAZQ3SeYBhbx99MCcOQ+NAjaT3ZTE/zlWa/ jDn0Wsnp6Jpz111WbeXmdwwecI9lJ0/AOEVp8Mzlp10Rm+e4SX9s/1iQX 2yDN6aSx9cyzV9jdJqqtjuWHPusJ2lq3U8mmfsMRSt1vUYL7NGF+P9aK/ mmOhZxU8e+47QmE4urxXmS68Bh/Gkrtu6Efi2wFMfqTSVzTvwZxik1QNi K1o8b8+iKRowiDO90JkNZ5tAfVxjC;
Domainkey-signature:	s=s1536a; d=ts.fujitsu.com; c=nofws; q=dns; h=X-SBRSScore:X-IronPort-AV:Received:X-IronPort-AV: Received:Received:Message-ID:Date:From:Organization: User-Agent:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=tINbXvWEGkcPzg3hRLfQVizjcDpofHXcp879zsrR2G4Y4iyFZwCr/Yu/ 4sEFvfpeXQtQ1klhG6O1WHVaQVVSXvxMwRVZvRCxfOUuuAhaeaDc2gg3K g69EMIFa3Ao9LiBdUYyYWOg/4n5t1WD74jDfPd1SBNVJTUNgAfLsday/e Tsmo9Xk4aasfCaf1qoKAIj8XS2D0WAdEDLRc1AgWxxFX5OoEpMFRso5W6 iT2l/vi0sFl65cQzHKAiDT7RbGry2;
Envelope-to:	www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to:	<20110124131340.GN8286@xxxxxxxxxxxxxxxxxxxxxxx>
List-help:	<mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id:	Xen developer discussion <xen-devel.lists.xensource.com>
List-post:	<mailto:xen-devel@lists.xensource.com>
List-subscribe:	<http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe:	<http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Organization:	Fujitsu Technology Solutions
References:	<4D3D780A.4030001@xxxxxxxxxxxxxx> <20110124131340.GN8286@xxxxxxxxxxxxxxxxxxxxxxx>
Sender:	xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent:	Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.16) Gecko/20101226 Iceowl/1.0b1 Icedove/3.0.11

On 01/24/11 14:13, Tim Deegan wrote:

At 13:00 +0000 on 24 Jan (1295874058), Juergen Gross wrote:

Hi,

in our BS2000 guest running as HVM with EPT on x86_64 I have a problem which
seems to be related to stale TLB entries. I'm pretty sure I have invalidated
the TLB correctly after a change of the page tables, so I've searched for
possible problems in the hypervisor.

Xen is version 4.0 from SLES 11 SP1.

If I have read the sources correctly, neither INVLPG nor reload of CR3 are
handled by the hypervisor. And I didn't find an explicit clearing of the TLB
when a vcpu is switching physical cpus. So I think the following scenario is
possible:

- a vcpu is running on physical cpu A creating a TLB entry
- the vcpu is scheduled on physical cpu B, while physical cpu A is left idle
- on physical cpu B the TLB entry is cleared by INVLPG or load CR3
- the vcpu is scheduled on physical cpu A again (no other vcpu was active
    there in between), CR3 is same as when vcpu left cpu A
- the old TLB entry from the vcpu is still valid there!

Do I miss something?


vmx_do_resume() calls hvm_asid_flush_vcpu() if the VCPU is migrating
onto this CPU, so the VCPU should get a fresh ASID when it comes back to
CPU A.  Processors with no ASID support flush their TLBs on every
VMENTER and VMEXIT, so I don't see where we could leak TLB entries.


Ah, this was the missing information I needed!
Thanks, I'll keep on searching...


Juergen

--
Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@xxxxxxxxxxxxxx
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

WARNING - OLD ARCHIVES

xen-devel

Re: [Xen-devel] stale TLB contents?