Re: [Xen-devel] Restoring FPU exception state

["Tian, Kevin" <kevin.tian@xxxxxxxxx>]

> From: Jan Beulich [mailto:JBeulich@xxxxxxxx]
> Sent: Wednesday, February 17, 2016 9:46 PM
> 
> >>> On 17.02.16 at 14:08, <david.vrabel@xxxxxxxxxx> wrote:
> > The FPU exception state includes 4 registers:
> >
> > - 64-bit FIP
> > - 16-bit FCS
> > - 64-bit FDP
> > - 16-bit FDS
> >
> > When a CPU takes an FPU exception in long mode, all 4 registers are
> > fully updated.  This state can be saved with a combination of REX.W
> > prefixed XSAVE and FNSTENV.  This state cannot be restored with any
> > combination of instructions as those that restore the 64-bit FIP/FDP
> > clear FCS and FDS; and those that restore FCS and FDS clear the upper
> > 32-bits of FIP and FDP [1].
> >
> > This causes problems when running Microsoft's Driver Verifier in a
> > 64-bit Windows guest (seen with Windows 7 SP1, but other versions may
> > also be affected).
> >
> > The Driver Verifier prior to calling a driver's interrupt handler will
> > save the FPU state, after the handler is called it will save the state
> > again and do a byte-by-byte compare to verify the state has not changed.
> >  A 0x3D (INTERRUPT_EXCEPTION_NOT_HANDLED) BugCheck is raised if the
> > state does not match.
> >
> > Windows uses XSAVE to save the FPU state, but it does not use a REX.W
> > prefixed XSAVE, and saves only the lower 32-bits of FIP/FDP and FCS/FDS.
> 
> Oh, you say that's the case even in 64-bit Windows? That's rather
> unexpected.
> 
> > If the VCPU is descheduled between these two checks, the contents of
> > FCS/FDS is lost, Windows will notice and BugCheck.
> >
> > When saving a VCPUs FPU state, Xen first uses a REX.W prefixed XSAVE and
> > notices that FIP/FDP[64:32] is non-zero and assumes are REX.W prefixed
> > XRSTOR is required to restore the full 64-bits of FIP/FDP.  This clears
> > FCS/FDS.
> >
> > On processors with FPCSDS[2] (bit 13) set in CPUID leaf 0x7, sub-leaf
> > 0x0, do not save FCS/FDS (they always write zeros) and this problem does
> > not occur, because FCS/FDS never needs to be restored.
> >
> > Does anyone any thoughts of a solution for processors without the FPCSDS
> > feature?
> 
> One would assume they have a solution to this problem on Hyper-V,
> but then again their solution may simply be that they don't use REX
> prefixes anywhere (i.e. namely also not in context switch code). In
> which case, though, they'd corrupt state of their non-Windows guests.
> 
> But to answer your question: I have no idea. The handling of these
> FPU code/data pointers has been an unloved child from the very
> beginning of the x86-64 architecture. All I could see us doing would
> be to add a per-domain control to override the auto-detection in the
> XSAVE code path.
> 

Interesting. Let me also have a check internally whether there is other
architectural alternative. BTW, not quite related. Could I think finally
Xen may allow user to specify OS type as a general per-domain control,
and then Xen can do free optimizations underlying based on that type?
I don't expect we want to expose raw FPU related per-domain control
since it's difficult for users to correctly set it up...

Thanks
Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel