WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] Re: [RFC, PATCH 3/24] i386 Vmi interface definition

To: Chris Wright <chrisw@xxxxxxxxxxxx>
Subject: [Xen-devel] Re: [RFC, PATCH 3/24] i386 Vmi interface definition
From: Zachary Amsden <zach@xxxxxxxxxx>
Date: Mon, 13 Mar 2006 19:16:38 -0800
Cc: Andrew Morton <akpm@xxxxxxxx>, Joshua LeVasseur <jtl@xxxxxxxxxx>, Xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxx>, Pratap Subrahmanyam <pratap@xxxxxxxxxx>, Wim Coekaerts <wim.coekaerts@xxxxxxxxxx>, Jack Lo <jlo@xxxxxxxxxx>, Dan Hecht <dhecht@xxxxxxxxxx>, Linux Kernel Mailing List <linux-kernel@xxxxxxxxxxxxxxx>, Jan Beulich <jbeulich@xxxxxxxxxx>, Christopher Li <chrisl@xxxxxxxxxx>, Virtualization Mailing List <virtualization@xxxxxxxxxxxxxx>, Linus Torvalds <torvalds@xxxxxxxx>, Anne Holler <anne@xxxxxxxxxx>, Jyothy Reddy <jreddy@xxxxxxxxxx>, Kip Macy <kmacy@xxxxxxxxxxx>, Ky Srinivasan <ksrinivasan@xxxxxxxxxx>, Leendert van Doorn <leendert@xxxxxxxxxxxxxx>, Dan Arai <arai@xxxxxxxxxx>
Delivery-date: Tue, 14 Mar 2006 10:06:40 +0000
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <20060314003956.GE12807@xxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <200603131801.k2DI1EAe005650@xxxxxxxxxxxxxxxxxxx> <20060314003956.GE12807@xxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Thunderbird 1.5 (X11/20051201)
Chris Wright wrote:
* Zachary Amsden (zach@xxxxxxxxxx) wrote:
Master definition of VMI interface, including calls, constants, and
interface version.

+/* VROM call table definitions */
+#define VROM_CALL_LEN             32
+
+typedef struct VROMCallEntry {
+   char f[VROM_CALL_LEN];
+} VROMCallEntry;

And the call entry is meant to be handled in whatever mechanism hypervisor
prefers for its entry points (ABI constraints notwithstanding) as in,
arbitrary software interrupt, or call gate, etc?  I guess for transparent
it has to, since those would be local calls.   Quite similar to the
hypercall entry point that Xen places on the hypercall_page, so easily
compatible.

See below.

The document is slightly more descriptive.  The above reserved slots
are shown as:

        char            reserved[32];
        char            elfHeader[64];

But that's only 3 (0-2).  I think I'm missing some small bit of magic.

+typedef struct VROMCallTable {
+   VROMCallEntry    vromCall[128];           // @ 0x80: ROM calls 4-127
+} VROMCallTable;

That comment eludes me.  Are 0-3 special somehow (IOW, I thought it was
just 0-2 as per above), and is it suggesting int 0x80?

Yeah, most of this is rather crufty - it is in transition. We had a full blown ROM image with 32-byte aligned stub points at one point for all of the VMI calls. In fact, it still is in that form, and it was required to overlap exactly with a native ROM that was built into Linux. See patch 6, VMI magic fixes for the details.

The machinery is already in place to do this, and it is a very nice thing that Xen has decided to adopt a similar approach (to the ROM) of publishing hypervisor code. I think they even use 32-byte alignment as well. The power of an indirection at this layer is just too attractive, and once you decide on a single binary image, I think it is inevitable that everyone will converge on the same idea. The same concept surfaces over and over - vsyscall being another example. You're basically dynamically linking in code at runtime, which is a pretty common thing to do, and gives you a very powerful redirection interface.

But we discovered that we couldn't achieve native performance, even using directly linked calls into the ROM. Calling out for sti / cli / popf / pushf on fast paths is just too expensive. We had to inline the native code to match native performance. And this leaves an opportunity for flexibility and performance in the hypervisor implementation.

You don't want direct calls to 32-byte stubs; in fact, as Joshua found, you can't get optimal performance in a hypervisor that way. What you need is the ability to preferentially inline certain hot calls into the VMI layer by using NOP padding. The non-hot calls can call out to the hypercall page.

Publishing code is the first step - inlining is the second, and it gets you back the hit you took when indirecting your fast paths.

Zach

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel