Re: [Xen-devel] fxsave, fnsave, ltr hang for guest OS.

[Keir Fraser <keir@xxxxxxx>]

Since the guest is allegedly continually running, you might try the 'd'
debug key (hit d on Xen serial console) and get execution backtrace for
every physical CPU, and see what the hypervisor is spending its time doing
for your guest on that instruction. Or add tracing to Xen to see where time
is going. Eg. Could be endlessly trying to emulate fxsave in
arch/x86/x86_emulate/x86_emulate.c. Or could be taking endless faults via
arch/x86/hvm/vmx/vmx.c:vmx_fpu_dirty_intercept().

In short, you need to do some more spadework.

 -- Keir

On 02/11/2010 18:13, "alarson@xxxxxxxx" <alarson@xxxxxxxx> wrote:

> Someone suggested that the following query would have been more properly
> addressed to xen-devel than xen-users.
> 
> The short version is that as soon as I turn on paging (CR0|=PG|PE), I
> loose the ability to do an fxsave, ltr, and several other things in my HVM
> client.
> 
> It would appear that Xen is making an assumption about the paging
> environment that my OS is not satisfying.
> 
> ----- Forwarded by Aaron Larson/ddci on 11/02/2010 12:55 PM -----
> 
> Aaron Larson/ddci wrote on 10/28/2010 03:57:59 PM:
> 
>> I am attempting to get a 32-bit protected mode OS running under Xen as
>> an HVM guest, but it is hanging on the execution of fxsave (and
>> fnsave) and ltr (load task register) instructions.  The OS is known to
>> run on a variety of X86 machines from 386 through Core2 DOU, and also
>> VMware workstation and userland qemu.  I saw some discussion of Xen
>> fxsave issue, most notably
>> http://lists.xensource.com/archives/html/xen-devel/2008-07/msg00874.html
>> but my I'm not trying to step, just execute so my interpretation is my
>> problem is different (although I certainly could be wrong).  I didn't
>> see any issues with fnsave or ltr.
>> 
>> I'm using Xen under OpenSuse 11.1 (Xen 3.3.1), although I've also
>> tried OpenSuse 11.3 (Xen 4.0.0) with identical results.
>> 
>> Any assistance would be greatly appreciated.
>> 
>> The code snippet for the fxsave case:
>> 
>>     diagMsg << "\nSaving FP context @" << (DWORD)
>> (&initialFPcontext) << "...";
>>     // Ensure buffer is writeable.
>>     ((BYTE*)(&initialFPcontext))[0]=0;
>>     ((BYTE*)(&initialFPcontext))[511]=0;
>>     // Initialze the FPU.
>>     diagMsg << "fninit...";
>>     asm volatile ("fninit");
>> 
>>     if (simdContextSavingSupported())
>>     {
>>       diagMsg << "fxSave...";
>>       asm volatile ("fxsave (%0)"::"r"(&initialFPcontext));
>>       diagMsg << "done\n";
>>     ...
>> 
>> The results are:
>> 
>>    Saving FP context @C082D470...fninit...fxSave...
>> 
>> An objdump -d of the binary yields:
>> 
>>    c0811ffb:    0f ae 00                fxsave (%eax)
>> 
>> "xm list" shows the VM continuing to get CPU cycles:
>> 
>>   # xm list
>>   Name                    ID   Mem VCPUs      State   Time(s)
>>   Domain-0                 0  3855     2     r-----    367.4
>>   myos                    59    16     1     r-----     17.7
>> 
>> GDB attached to the running VM shows that the processor is executing
>> the fxsave instruction and eax is as expected:
>> 
>>    (gdb) p/x $eip
>>    $1 = 0xc0811ffb
>>    (gdb) p/x $eax
>>    $2 = 0xc082d470
>> 
>> The /var/log/xen/qemu-dm-myos.log:
>>   domid: 59
>>   qemu: the number of cpus is 1
>>   Watching /local/domain/0/device-model/59/logdirty/next-active
>>   Watching /local/domain/0/device-model/59/command
>>   xs_read(): vncpasswd get error. /vm/9ec94cd1-20c5-07da-a1d4-
>> fdd500b8aef6/vncpasswd.
>>   qemu_map_cache_init nr_buckets = 4000 size 196608
>>   /usr/src/packages/BUILD/xen-3.3.1-testing/tools/ioemu-
>> dir/hw/xen_blktap.c:714: Init blktap pipes
>>   shared page at pfn ffe
>>   buffered io page at pfn ffc
>>   Time offset set 0
>>   Register xen platform.
>>   Done register platform.
>>   medium change watch on `hdc' (index: 0): /home/alarson/pci-scanner.iso
>>   I/O request not ready: 0, ptr: 0, port: 0, data: 0, count: 0, size: 0
>> 
>> The fxsave instruction lists the following faults, and I've checked
>> them all.
>> 
>>   Protected Mode faults:
>>     #GP(0) illegal memory operand effective address in segment
>>     register. 
>>     #SS(0) illegal address in SS segment
>>     #PF(fault-code)
>> 
>>      The instruction does not have a segment prefix, and the
>>      preceding code ensures the buffer is writable.
>> 
>>     #NM if CR0[ts] or CR0[em]=1 (bits 3 or 2).
>> 
>>      The CR0 is C0000033
>> 
>>     #UD If CPUID.01H:EDX.FXSR[bit 24] = 0.
>> 
>>       CPUID.01H is EAX=0001067A EBX=00020800 ECX=80082201 EDX=0789FBBF
>> 
>>       EDX[24] is 1
>> 
>>     #UD If the LOCK prefix is used
>> 
>>       Instruction has no prefixes.
>> 
>>     #AC  for alignment
>> 
>>       FP context address is 0xC082D470 (16 byte aligned)
>> 
>> cat /proc/cpuinfo
>> processor       : 0
>> vendor_id       : GenuineIntel
>> cpu family      : 6
>> model           : 23
>> model name      : Intel(R) Core(TM)2 Duo CPU     E8400  @ 3.00GHz
>> stepping        : 10
>> cpu MHz         : 2992.582
>> cache size      : 6144 KB
>> fdiv_bug        : no
>> hlt_bug         : no
>> f00f_bug        : no
>> coma_bug        : no
>> fpu             : yes
>> fpu_exception   : yes
>> cpuid level     : 13
>> wp              : yes
>> flags           : fpu de tsc msr pae cx8 apic mtrr cmov pat clflush
>> acpi mmx fxsr sse sse2 ss ht nx constant_tsc pni est ssse3 sse4_1
>> bogomips        : 5987.06
>> clflush size    : 64
>> power management:
>> 
>> processor       : 1
>> [same as for processor 0]
>> 
>> An xm list -l yields the following for the domain:
>> (domain
>>     (domid 58)
>>     (bootloader )
>>     (on_crash preserve)
>>     (uuid ca61601b-4728-d39c-2a4c-4fe0ced6954e)
>>     (bootloader_args )
>>     (vcpus 1)
>>     (name myos)
>>     (cpus (()))
>>     (on_reboot restart)
>>     (on_poweroff destroy)
>>     (maxmem 16)
>>     (memory 16)
>>     (shadow_memory 2)
>>     (features )
>>     (on_xend_start ignore)
>>     (on_xend_stop ignore)
>>     (start_time 1288284595.39)
>>     (cpu_time 319.894000271)
>>     (online_vcpus 1)
>>     (image
>>         (hvm
>>             (kernel )
>>             (args 'VNC_VIEWER=127.0.0.2:5514 ')
>>             (timer_mode 0)
>>             (vnc 1)
>>             (nographic 0)
>>             (hpet 0)
>>             (rtc_timeoffset 0)
>>             (loader /usr/lib/xen/boot/hvmloader)
>>             (pci ())
>>             (guest_os_type default)
>>             (apic 1)
>>             (sdl 0)
>>             (extid 0)
>>             (hap 0)
>>             (monitor 0)
>>             (device_model /usr/lib/xen/bin/qemu-dm)
>>             (boot d)
>>             (display localhost:10.0)
>>             (pae 0)
>>             (stdvga 0)
>>             (usb 0)
>>             (xauthority /root/.Xauthority)
>>             (isa 0)
>>             (acpi 0)
>>             (vncunused 1)
>>             (localtime 0)
>>             (notes (SUSPEND_CANCEL 1))
>>         )
>>     )
>>     (status 1)
>>     (state --p---)
>>     (store_mfn 4093)
>>     (device
>>         (console
>>             (protocol vt100)
>>             (location 3)
>>             (uuid 9b2b7ffe-8520-8a0b-6a0a-8864bcc3c8c4)
>>         )
>>     )
>>     (device
>>         (vbd
>>             (uuid 65beb1fb-b726-a262-8837-6a99d24a45e3)
>>             (bootable 1)
>>             (dev hdc:cdrom)
>>             (uname file:/home/alarson/pci-scanner.iso)
>>             (mode r)
>>             (backend 0)
>>             (bootable 1)
>>             (VDI )
>>         )
>>     )
>>     (device
>>         (vfb
>>             (vncunused 1)
>>             (type vnc)
>>             (uuid 7cb2e9d4-8376-8f6b-7851-6bf7c4058914)
>>             (location 127.0.0.1:5900)
>>         )
>>     )
>> )
> 
> 
> This message is intended only for the use of the individual or entity to which
> it is addressed. If the reader of this message is not the intended recipient,
> or the employee or agent responsible for delivering the message to the
> intended recipient, you are hereby notified that any dissemination,
> distribution or copying of this message is strictly prohibited. If you have
> received this communication in error, please notify us immediately by replying
> to the sender of this E-Mail by return E-Mail or by telephone.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel