Re: [PATCH 62/65] x86/entry: Make IDT entrypoints CET-IBT compatible

[Jan Beulich <jbeulich@xxxxxxxx>]

On 03.12.2021 16:30, Andrew Cooper wrote:
> On 03/12/2021 13:32, Jan Beulich wrote:
>> On 26.11.2021 13:34, Andrew Cooper wrote:
>>> Each IDT vector needs to land on an endbr64 instruction.  This is especially
>>> important for the #CP handler, which will escalate to #DF if the endbr64 is
>>> missing.
>> One question here: How does this work?
> 
> Honestly, I'm not sure.
> 
>>  I don't recall there being any "CET
>> shadow" along the lines of "STI shadow" and "SS shadow", yet there's
>> clearly an insn boundary here that gets "skipped" if the 2nd #CP gets
>> converted to #DF. And fetching of the first handler insn also isn't part
>> of exception delivery (and could cause other exceptions first, like #PF).
> 
> I can't make my observations of real hardware behaviour match the
> description in the spec.

I haven't been able to find a description at all of exception behavior
when the exception occurs in wait-for-endbr state. There is text saying
that #BP and #DB can occur this way, but I couldn't find anything about
the tracker state changes in such cases. While I could see the state to
remain engaged (requiring an ENDBR at the handler's entry point), I
cannot see how the state would get re-engaged upon IRET from the
exception handler, unless the return is back to CPL3.

> Given what a mess it all is, I wouldn't be surprised if the exception
> delivery microcode has a special case to escalate this to #DF.

I am meanwhile wondering whether any exception in wait-for-endbr state
at CPL < 3 would promote to #DF, for loss of state. Albeit there must
still be a distinction between CALL/JMP induced state and that
resulting from interrupt or exception delivery. Yet there's no
architectural (or shadow) state expressing "first insn of an exception
handler".

I'm not even convinced the aforementioned statements about #DB and #BP
are actually meant to cover more than just CPL3, or at best ENDBR at
normal CALL/JMP destinations.

While for Xen's own use we may get away without knowing how all of this
actually works (perhaps accepting the fact that one can't set breakpoints
at exception handler entry points, depending on whether their delivery
would promote to #DF), as soon as we were to support CET-IBT for guests
we'd definitely need to know.

Jan

> If it didn't escalate to #DF, then you'd end up with an infinite stream
> of #CP's, which will most likely cause a stack overflow because #CP
> needs to be not-IST for shadow stack reasons.
> 
> ~Andrew
>