WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] Switching to user mode from domU kernel

To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-devel] Switching to user mode from domU kernel
From: Trammell Hudson <hudson@xxxxxxxxxxxxxx>
Date: Mon, 1 Oct 2007 14:43:06 -0400
Delivery-date: Mon, 01 Oct 2007 11:43:53 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <C3264307.E410%Keir.Fraser@xxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <20070930201143.GC31619@xxxxxxxxxxxxxx> <C3264307.E410%Keir.Fraser@xxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.5.10i
On Mon, Oct 01, 2007 at 06:24:07AM +0100, Keir Fraser wrote:
> On 30/9/07 21:11, "Trammell Hudson" <hudson@xxxxxxxxxxxxxx> wrote:
> > Am I not jumping into user space correctly?  Is there something
> > else that my code should do to make the transition?
> 
> It sounds like event delivery is masked before the iret, and then you have
> IF set in the RFLAGS value in the iret frame, which causes event delivery to
> be unmasked during iret.

That makes sense, although the event channel pending array is
all zero and the shared_info->vcpu_info[0].evtchn_upcall_pending
value is also zero.

If I do not set IF in RFLAGS, Xen crashes when it tries to handle
a GPF:

(XEN) traps.c:1587: GPF (001c): ffff830000165555 -> ffff83000016556b
(XEN) ----[ Xen-3.0.4-1  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e010:[<ffff83000016364b>] restore_all_xen+0x1b/0x20
(XEN) RFLAGS: 0000000000010282   CONTEXT: hypervisor
(XEN) rax: 000000400001c480   rbx: 000000600053dde8   rcx: 0000000000000017
(XEN) rdx: 00000000decafbad   rsi: 0000000000012345   rdi: 000000000badbabe
(XEN) rbp: 00000040005fffc0   rsp: ffff83000074ffa8   r8:  000000000000001f
(XEN) r9:  0000000000000017   r10: 00000040005fffc0   r11: 0000000000000000
(XEN) r12: 0000006000017000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: ffff830000163726   cr0: 000000008005003b   cr4: 00000000000006f0
(XEN) cr3: 000000002734d000   cr2: ffff820000001000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e018   cs: e010
(XEN) Xen stack trace from rsp=ffff83000074ffa8:
(XEN)    000001000001c480 0000006000101000 000000000000e033 000000400058bec0
(XEN)    0000006000102fa8 000000000000e02b 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 ffff830000744080
(XEN) Xen call trace:
(XEN)    [<ffff83000016364b>] restore_all_xen+0x1b/0x20
(XEN) 
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) GENERAL PROTECTION FAULT
(XEN) [error_code=1000]
(XEN) ****************************************
(XEN) 
(XEN) Reboot in five seconds...

> > I am passing flags==0 and only push the values for flags, CS:RIP,
> > RFLAGS, and SS:RSP.  If I push values for RAX, R11, and RCX on the
> > stack the kernel ends up in all sorts of weird places rather than
> > my intended RIP and RSP.
> 
> That doesn't make sense. The iret implementation (for an x86/64 guest)
> always expects RAX/R11/RCX on the stack.

That was my expectation as well, but if I push the three extra values
onto the stack then it jumps to the wrong place:

        context.rax     = 0x12345;
        context.r11     = 0xdecafbad;
        context.rcx     = 0xbadbabe;
        context.rflags  = 0x200;
        context.rip     = start_addr; // 01f:000000400001c480
        context.cs      = USER_CS;
        context.flags   = 0;
        context.rsp     = (uintptr_t) stack; // 017:00000040005fffc0
        context.ss      = USER_DS;

        __asm__ __volatile__ (
                "push   %0\n"
                "push   %1\n"
                "push   %2\n"
                "push   %3\n"
                "push   %4\n"
                "push   %5\n"
                "push   %6\n"
                "push   %7\n"
                "push   %8\n"
                "jmp hypercall_page + __HYPERVISOR_iret*32\n"
                :
                : "r" (context.ss),
                "r" (context.rsp),
                "r" (context.rflags),
                "r" (context.cs),
                "r" (context.rip),
                "r" (context.flags),
                "r" (context.rcx),
                "r" (context.r11),
                "r" (context.rax),
                "r" (USER_DS)
        );

Ends up with RIP e033:00000000decafbad and RSP e02b:000000400001c480:

(XEN) domain_crash_sync called from entry.S
(XEN) Domain 147 (vcpu#0) crashed on cpu#0:
(XEN) ----[ Xen-3.0.4-1  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e033:[<00000000decafbad>]
(XEN) RFLAGS: 0000000000010202   CONTEXT: guest
(XEN) rax: 0000000000000017   rbx: 0000000000000200   rcx: 00000000decafbad
(XEN) rdx: 0000000000012345   rsi: 000000000badbabe   rdi: 0000000000000000
(XEN) rbp: 000000600053dde8   rsp: 000000400001c480   r8:  000000400001c480
(XEN) r9:  000000000000001f   r10: 0000000000000017   r11: 0000000000000200
(XEN) r12: 00000040005fffc0   r13: 0000006000017000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000006f0
(XEN) cr3: 0000000026ae5000   cr2: 00000000decafbad
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033

Looking in hypercall_page_initialise_ring3_kernel() it appears that
the hypervisor call does the rcx, r11 and rax pushing, not the
domU kernel calling the hypercall:

    /*
     * HYPERVISOR_iret is special because it doesn't return and expects a
     * special stack frame. Guests jump at this transfer point instead of
     * calling it.
     */
    p = (char *)(hypercall_page + (__HYPERVISOR_iret * 32));
    *(u8  *)(p+ 0) = 0x51;    /* push %rcx */
    *(u16 *)(p+ 1) = 0x5341;  /* push %r11 */
    *(u8  *)(p+ 3) = 0x50;    /* push %rax */
    *(u8  *)(p+ 4) = 0xb8;    /* mov  $__HYPERVISOR_iret,%eax */
    *(u32 *)(p+ 5) = __HYPERVISOR_iret;
    *(u16 *)(p+ 9) = 0x050f;  /* syscall */

-- Trammell

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>