Re: xl/xm save -c fails - set_vcpucontext EOPNOTSUPP (was Re: [Xen-devel] xl save -c issues with Windows 7 Ultimate)

[Shriram Rajagopalan <rshriram@xxxxxxxxx>]

On Tue, May 10, 2011 at 11:03 AM, Jan Beulich <JBeulich@xxxxxxxxxx> wrote:

>>> On 10.05.11 at 17:50, Shriram Rajagopalan <rshriram@xxxxxxxxx> wrote:
> On Tue, May 10, 2011 at 10:02 AM, Jan Beulich <JBeulich@xxxxxxxxxx> wrote:
>
>> >>> On 10.05.11 at 16:52, Shriram Rajagopalan <rshriram@xxxxxxxxx> wrote:
>> > On Tue, May 10, 2011 at 3:41 AM, Ian Campbell <Ian.Campbell@xxxxxxxxxx
>> >wrote:
>> >> The most plausible looking EOPNOTSUPP from that code is in
>> >> xen/arch/x86/domain.c:arch_set_info_guest() but that is on a PV only
>> >> path.
>> >>
>> >> And that rings with the pv guests I am using. It makes perfect sense,
>> > looking
>> > at that function and especially at the code that returns EOPNOTSUPP (the
>> > only
>> > place in the entire file).
>> >    else
>> >     {
>> >         bool_t fail = v->arch.pv_vcpu.ctrlreg[3] != c(ctrlreg[3]);
>> >
>> > #ifdef CONFIG_X86_64
>> >         fail |= v->arch.pv_vcpu.ctrlreg[1] != c(ctrlreg[1]);
>> > #endif
>> >
>> >         for ( i = 0; i < ARRAY_SIZE(v->arch.pv_vcpu.gdt_frames); ++i )
>> >             fail |= v->arch.pv_vcpu.gdt_frames[i] != c(gdt_frames[i]);
>> >         fail |= v->arch.pv_vcpu.gdt_ents != c(gdt_ents);
>> >
>> >         fail |= v->arch.pv_vcpu.ldt_base != c(ldt_base);
>> >         fail |= v->arch.pv_vcpu.ldt_ents != c(ldt_ents);
>> >
>> >         if ( fail )
>> >            return -EOPNOTSUPP;
>> >     }
>> >
>> > This change was introduced by c/s
>> > changeset:   23142:f5e8d152a565
>> > user:        Jan Beulich <jbeulich@xxxxxxxxxx>
>> > date:        Tue Apr 05 13:01:25 2011 +0100
>> > x86: split struct vcpu
>> >
>> > I think I am missing something really obvious in this piece of code. The
>> > xc_domain_resume code tries to modify the return value of shutdown
>> hypercall
>> > (i.e eax register is set to 1) and this code doesnt seem to check those
>> > registers.
>>
>> Correct - the code here checks only for values where the logic
>> needed to support changing the on an already initialized vCPU isn't
>> implemented. Previously, actual vCPU state and what was tracked
>> in struct vcpu could get out of sync in this case, potentially
>> confusing things further down (including possible security issues).
>>
>> You'll want to figure out which part(s) actually differ, and why.
>> Only then we'll be able to tell whether mentioned c/s introduced
>> false positives.
>>
>> Bit confused. If I understand correctly, this piece of code checks new
> values
> of certain registers against old ones, for an already initialized VCPU. And
> AFAIT,
> it is checking the gdts, ldts & control registers. The xc_domain_resume code
> just
> changes one general purpose register eax. basically,
>
>  get_vcpucontext()
>  set_field(eax, 1) //to indicate SUSPEND_CANCEL
>  set_vcpucontext()
>
> I dont understand what you mean by "which parts actually differ & why".

Quite obviously there are differences in one or more of the now
checked fields, and we need to find out where they are (and
why). This is regardless of the tools apparently only modifying
eax.

> And just a trivial question:
>  is the hypervisor binary always compiled to a 32-bit elf? somehow, the
> symbols file xen-syms-* is getting compiled to 64 bit ELF binary while
> the xen binary is getting compiled to 32-bit binary.

Yes, that's because it wants to boot from 32-bit GrUB (and the
multiboot protocol also is 32-bit only afaik).

Jan

I tried out a simple program that just gets and sets the VCPU 0's context (no change
whatsoever to anything). There is no intermediate code involved (except for the hypercall
bounce buffer stuff). If all is well, then this should work. But it doesnt!! even for a PV guest.

 I get the same Operation Not supported error when I try to "set" the vcpu context with the 
same struct obtained via the get_vcpucontext hypercall!

Setup: 32bit 2.6.18 32-bit domU, 2.6.32.39 64-bit dom0, 64-bit xen-unstable c/s 23300,

I suspend the domain via usual xenstore suspend.
The code for get/set vcpu context was taken verbatim from 
tools/libxc/xc_resume.c:modify_returncode
 (just commented out the SET_FIELD line, that changes eax)

static int pv_guest_width(xc_interface *xch, uint32_t domid)
{
    DECLARE_DOMCTL;
    domctl.domain = domid;
    domctl.cmd = XEN_DOMCTL_get_address_size;
    if ( xc_domctl(xch, &domctl) != 0 )

    {
        perror("Could not get guest address size");
        return -1;
    }
    return domctl.u.address_size.size / 8;
}

static int get_set_vcpu_ctx(xc_interface *xch, unsigned int domid)

{
    vcpu_guest_context_any_t ctxt;
    xc_dominfo_t info;
    xen_capabilities_info_t caps;
    struct domain_info_context _dinfo = {};
    struct domain_info_context *dinfo = &_dinfo;
    int rc;

    if ( xc_domain_getinfo(xch, domid, 1, &info) != 1 )
    {
        perror("Could not get domain info");
        return -1;
    }

    /* Probe PV guest address width. */
    dinfo->guest_width = pv_guest_width(xch, domid);

    if ( dinfo->guest_width < 0 )
          return -1;
 
    if ( (rc = xc_vcpu_getcontext(xch, domid, 0, &ctxt)) != 0 ) {
        perror("getcontext failed");
        return rc;
    }

    //    SET_FIELD(&ctxt, user_regs.eax, 1);

    if ( (rc = xc_vcpu_setcontext(xch, domid, 0, &ctxt)) != 0 ) {
        perror("setcontext failed");
        return rc;
    }

    fprintf(stderr, "get/set vcpu context succeeded\n");

    return 0;
}


and I get - setcontext: operation not supported!

now for the weirdness:
 Since the the setcontext failed I thought I should be able 
to run the above sample code again and again with no side effect 

(please correct my assumption if I am wrong).

But when I run the above code for the second time, I get a XEN panic!

(XEN) Xen BUG at domctl.c:1724
(XEN) ----[ Xen-4.2-unstable  x86_64  debug=y  Not tainted ]----

(XEN) CPU:    2
(XEN) RIP:    e008:[<ffff82c48014dd57>] arch_get_info_guest+0x5f7/0x7b0
(XEN) RFLAGS: 0000000000010202   CONTEXT: hypervisor
(XEN) rax: 0000000000000001   rbx: ffff8300228c4000   rcx: ffff8300228c4040

(XEN) rdx: 0000000000000000   rsi: 0000000000000000   rdi: ffff830450652210
(XEN) rbp: ffff83082a357da8   rsp: ffff83082a357d68   r8:  0000000000000002
(XEN) r9:  0000000000000002   r10: 0000000000000040   r11: 0000000000000000

(XEN) r12: ffff830450652010   r13: 0000000000000001   r14: ffff830829db9000
(XEN) r15: ffff830450652010   cr0: 0000000080050033   cr4: 00000000000026f0
(XEN) cr3: 000000047beef000   cr2: 0000000000d44048
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008

(XEN) Xen stack trace from rsp=ffff83082a357d68:
(XEN)    ffff830829db9000 ffff8300228c4000 ffff83082a357d98 fffffffffffffff4
(XEN)    0000000000d40004 ffff8300228c4000 ffff830829db9000 ffff830450652010
(XEN)    ffff83082a357ef8 ffff82c48010351f ffff83082a357e48 ffff82c48016af84

(XEN)    0000000000000000 0000000000000070 ffff83082a357e28 000000000047beea
(XEN)    0000000000000000 ffff83082a30b000 ffff830450652010 ffff830450652010
(XEN)    ffff83082a357e48 0000000080164c7d aaaaaaaaaaaaaaaa ffff83082a30b000

(XEN)    ffff83082a357ef8 ffff82c480113d73 000000070000000d 0000000000000001
(XEN)    0000000000000000 0000000000d42004 0000000000000000 00007fef43c4a791
(XEN)    0000000000000001 0000000000000000 00007fff27dc7db0 00007fef43a1bd58

(XEN)    0000000000000024 0000000000000001 00007fff27dc9710 0000000000000001
(XEN)    0000000000d3f050 00007fef43c51325 0000000000000011 00007fff27dc7dd0
(XEN)    ffff83082a357ed8 ffff8300bf656000 0000000000000003 00007fff27dc7c60

(XEN)    00007fff27dc7c60 0000000000000000 00007cf7d5ca80c7 ffff82c48020e1e8
(XEN)    ffffffff8100948a 0000000000000024 0000000000000000 00007fff27dc7c60
(XEN)    00007fff27dc7c60 0000000000000003 ffff8807a0f2fe68 ffffffff8148d700

(XEN)    0000000000000282 0000000000000024 0000000000d3f050 0000000000d40004
(XEN)    0000000000000024 ffffffff8100948a 0000000100000000 00007fff27dc7ce0
(XEN)    0000000000d40004 0000010000000000 ffffffff8100948a 000000000000e033

(XEN)    0000000000000282 ffff8807a0f2fe20 000000000000e02b 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000002
(XEN) Xen call trace:
(XEN)    [<ffff82c48014dd57>] arch_get_info_guest+0x5f7/0x7b0

(XEN)    [<ffff82c48010351f>] do_domctl+0x10ad/0x195e
(XEN)    [<ffff82c48020e1e8>] syscall_enter+0xc8/0x122

I would appreciate any pointers on how to go about this. 

shriram
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel