Re: [Xen-devel] Crash with paravirt-ops 2.6.31.6 kernel

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Tue, 2009-11-24 at 00:39 +0000, Jeremy Fitzhardinge wrote:
> On 11/23/09 07:25, Ian Campbell wrote:
> > On Sun, 2009-11-22 at 09:54 +0000, Bastian Blank wrote:
> >   
> >> On Tue, Nov 17, 2009 at 10:04:36PM +0300, William Pitcock wrote:
> >>     
> >>> [    1.254927] init[1] general protection ip:f779042f sp:ff9b0340 error:0
> >>>       
> >> Hmm, this looks like the old Debian bug 544145[1]. For some reason the
> >> hypervisor jumps back into 64bit mode after a syscall instruction.
> >> Workaround: vdso32=0 or deinstall libc6-i686,
> >>     
> > I just noticed that one of my test boxes has a AMD processor so I took a
> > bit of a look into this.
> >
> > The issue seems to be with this bit of code in the hypervisor
> > (xen/arch/x86/x86_64/entry.S):
> >
> >         restore_all_guest:
> >                 ASSERT_INTERRUPTS_DISABLED
> >                 RESTORE_ALL
> >                 testw $TRAP_syscall,4(%rsp)
> >                 jz    iret_exit_to_guest
> >         
> >                 addq  $8,%rsp
> >                 popq  %rcx                    # RIP
> >                 popq  %r11                    # CS
> >                 cmpw  $FLAT_USER_CS32,%r11
> >                 popq  %r11                    # RFLAGS
> >                 popq  %rsp                    # RSP
> >                 je    1f
> >                 sysretq
> >         1:      sysretl
> >
> > We are attempting to return to the Linux defined __USER_CS32 (0x23)
> > which does not match the test for the Xen defined FLAT_USER_CS32
> > (0xe023) and therefore we hit the sysretq instead of the sysretl which
> > causes us to return with CS 0xe033 (FLAT_USER_CS64) instead of CS
> > 0xe023.
> >   
> 
> Ah, good detective work.
> 
> > This patch to the kernel fixes things but doesn't seem that
> > satisfactory:
> >   
> 
> It is a bit ugly.  I guess you could just assert that FLAT_USER_CS32 is
> part of the iret ABI so the guest has to use it, which appears to be the
> defacto definition.

Yes, I suppose that is reasonable.

> > I'm not sure
> > what can realistically do since doing the Right Thing would seem to
> > involve looking up the descriptor in the GDT to determine if the
> > selector is 32 or 64 bit and/or context switching IA32_STAR in some
> > fashion to allow guests to specify their own userspace CS for sysret 32
> > and 64.
> >   
> 
> That would be a bit awkward to do in the iret fast path.

Agreed, hence "realistically".

> 
> > Perhaps simply not returning guest userspace with sysret (as above)
> > makes most sense, a syscall already takes a trap through the hypervisor
> > on both entry and exit so I'm not sure the difference between sysret and
> > iret is going to be noticeable.
> >
> > Another option might be to define VGCF_compat_mode as a new flag to
> > HYPERVISOR_iret and select sysretq/sysretl based on that. This would
> > still expose Xen descriptors to guests which didn't ask for one but at
> > least it would (probably) be a compatible descriptor.
> >   
> 
> I don't think that's much of an improvement over using the Xen selector
> for cs.  Of course, it requires that the Xen selectors are actually part
> of the ABI and won't change at some later point.

I think the guest accessible-but-Xen-defined descriptors are part of the
ABI, so that's OK.

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel