UBSan bug in real mode fpu emulation

[Fabian Specht <f.specht@xxxxxx>]

Dear Xen-Devel team,

we discovered a bug regarding undefined behaviour in the FPU emulation
unit.

if ( !s->rex_prefix )
{
        /* Convert 32-bit real/vm86 to 32-bit prot format. */
        unsigned int fip = fpstate.env.mode.real.fip_lo +
                                           (fpstate.env.mode.real.fip_hi << 16);
        unsigned int fdp = fpstate.env.mode.real.fdp_lo +
                                           (fpstate.env.mode.real.fdp_hi << 16);
        unsigned int fop = fpstate.env.mode.real.fop;

        fpstate.env.mode.prot.fip = fip & 0xf;
        fpstate.env.mode.prot.fcs = fip >> 4;
        fpstate.env.mode.prot.fop = fop;
        fpstate.env.mode.prot.fdp = fdp & 0xf;
        fpstate.env.mode.prot.fds = fdp >> 4;
}

It occurs at arch/x86/arch/x86/x86_emulate/blk.c:85 of the v4.20.0
release during the bit shift and can be triggered using the attached xtf
test. We are not aware of any security consequences.
Simply shifting by 4 should do the trick in my opinion.

Similar code resides in the same file in lines 87, 125 and 127.
The attached xtf test is run for hvm32.

~Fabian

Attachment: main.c
Description: Text document