Re: [Xen-devel] Future x86 emulator direction

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 13/12/2016 21:55, Razvan Cojocaru wrote:
> On 12/13/2016 11:03 PM, Andrew Cooper wrote:
>> On 13/12/2016 20:51, Razvan Cojocaru wrote:
>>> On 12/13/2016 07:10 PM, Andrew Cooper wrote:
>>>> On 13/12/16 15:58, Razvan Cojocaru wrote:
>>>>> Hello, and first of all thanks for the discussion!
>>>>>
>>>>>> Think of it a bit more like introducing a new action emulator (name
>>>>>> definitely subject to improvement), which implements things such as
>>>>>> wrmsr, cpuid, pagewalk, task_switch, etc.
>>>>>>
>>>>>> The vmexit helpers, given decode assistance from hardware, can directly
>>>>>> call action->task_switch().  If insufficient information is available
>>>>>> (e.g. LMSW on AMD), the helpers invoke the instruction emulator to work
>>>>>> out what to do, and the instruction emulator would invoke the action
>>>>>> emulator as part of its execute phase.
>>>>>>
>>>>>> Wherever possible, the action emulator should be guest-neutral, and
>>>>>> ideally the single point of implementation of non-architectural actions
>>>>>> such as "the vm_event subsystem is interested in this."
>>>>>>
>>>>>>> And to be honest, on the road towards
>>>>>>> completion of the emulator I think the SVM/VMX insns are pretty
>>>>>>> close to the end of the priority list.
>>>>>> I'd expect them to show up frequently during introspection, although
>>>>>> maybe I am wrong.  Razvan: Any thoughts?
>>>>> I definitely think this is a good idea as far as introspection goes -
>>>>> having a single contact surface with the underlying logic would be a
>>>>> significant improvement.
>>>>>
>>>>> As for SVM/VMX instructions, we're interested in anything that is able
>>>>> to trigger an EPT fault (and hence a mem_access event) - we've had our
>>>>> share of adventures with VMX-specific instructions, so they're not low
>>>>> priority for us.
>>>> In reality, this is any instruction if you set EPT NX on a page, I presume?
>>>>
>>>> Do you have stats on which instructions you most frequently have to
>>>> singlestep because of lack of emulator support, or is the spread
>>>> essentially random?
>>> We do set some pages NX, so those too, but there are also a lot of
>>> events coming from instructions that simply try to write to a page
>>> marked RX - so we'll have an EPT fault even for an instruction running
>>> from a legitimate page, but which has a destination address in a
>>> read-only page.
>>>
>>> We've unfortunately not kept track of them beyond the discussions and
>>> patches that occured on xen-devel, but it's been important enough to
>>> warrant writing patches that basically set the MTF and "single-step"
>>> intstructions that fail emulation.
>> The eventual plan is to have a fully complete emulator (but I'd be lying
>> if I suggested that that was just around the corner).
>>
>> Given the previous use of the instruction emulator, I'd be surprised if
>> there were many instructions with memory operands which we were still
>> missing, although I suppose our support of non-mov SSE instructions is
>> about 0.
> We've had some issues specifically with SSE issues in the past, although
> I unfortunately cannot recall exactly which ones now (this happened back
> in 2013 IIRC).
>
>> It would certainly be interesting to see a list of "most frequent
>> instruction unsupported by the emulator" if you were to happen upon
>> one.  While there is no shortage of identifiable holes in the emulator,
>> there is also no priority order to fix them in.
> I'll try to run some tests tomorrow and see if anything pops up with
> basic guest usage. It's unfortunately hard to predict what instruction
> will fail emulation, since we're also protecting user applications, so
> any instruction that, say, Chrome or Firefox uses and can trigger an EPT
> fault is fair game for emulation. Combining user application with
> "regular" guest behaviour yields such a number of potentially failed
> emulation attempts that we thought it would be important to have a way
> to quickly single step un-emulatable instructions.
>
> On a somewhat related note, it's important to also figure out how best
> to avoid emulation races such as the LOCK CMPXCHG issue we've discussed
> in the past. Maybe that's also worth taking into consideration at this
> early stage.

Funny you should ask that.

The only possible way to do this safely is to have the emulator map the
target frame(s) and execute a locked stub instruction with a memory
operand pointing at the mapping.  We have no other way of interacting
with the cache coherency fabric.

I think the shadow cmpxchg() hook is nearly there, although the
implementation does need to up into the body of the emulator along with
a map()/unmap() pair of primitives.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel