Re: VIRIDIAN CRASH: 3b c0000096 75b12c5 9e7f1580 0

[Jan Beulich <jbeulich@xxxxxxxx>]

On 04.02.2021 09:46, James Dingwall wrote:
> On Wed, Feb 03, 2021 at 03:55:07PM +0100, Jan Beulich wrote:
>> On 01.02.2021 16:26, James Dingwall wrote:
>>> I am building the xen 4.11 branch at 
>>> 310ab79875cb705cc2c7daddff412b5a4899f8c9 which includes commit 
>>> 3b5de119f0399cbe745502cb6ebd5e6633cc139c "86/msr: fix handling of 
>>> MSR_IA32_PERF_{STATUS/CTL}".  I think this should address this error 
>>> recorded in xen's dmesg:
>>>
>>> (XEN) d11v0 VIRIDIAN CRASH: 3b c0000096 75b12c5 9e7f1580 0
>>
>> It seems to me that you imply some information here which might
>> better be spelled out. As it stands I do not see the immediate
>> connection between the cited commit and the crash. C0000096 is
>> STATUS_PRIVILEGED_INSTRUCTION, which to me ought to be impossible
>> for code running in ring 0. Of course I may simply not know enough
>> about modern Windows' internals to understand the connection.
> 
> Searching for "VIRIDIAN CRASH: 3b" led me to this thread and then to the 
> commit based on the commit log message.
> 
> https://patchwork.kernel.org/project/xen-devel/patch/20201007102032.98565-1-roger.pau@xxxxxxxxxx/
> 
> I have naively assumed that the RCX register indicated MSR_IA32_PERF_CTL 
> based on:
> 
> #define MSR_IA32_PERF_CTL             0x00000199
> 
> I've added this patch:
> 
> diff --git a/xen/arch/x86/msr.c b/xen/arch/x86/msr.c
> index 99c848ff41..7a764907d5 100644
> --- a/xen/arch/x86/msr.c
> +++ b/xen/arch/x86/msr.c
> @@ -232,12 +232,16 @@ int guest_rdmsr(const struct vcpu *v, uint32_t msr, 
> uint64_t *val)
>           */
>      case MSR_IA32_PERF_STATUS:
>      case MSR_IA32_PERF_CTL:
> -        if ( !(cp->x86_vendor & (X86_VENDOR_INTEL | X86_VENDOR_CENTAUR)) )
> +        if ( !(cp->x86_vendor & (X86_VENDOR_INTEL | X86_VENDOR_CENTAUR)) ) {
> +            printk(KERN_DEBUG "JKD: MSR %#x FAULT1: %#x & %#x\n", msr, 
> cp->x86_vendor, (X86_VENDOR_INTEL | X86_VENDOR_CENTAUR));
> +
>              goto gp_fault;
> +        }
>  
>          *val = 0;
>          if ( likely(!is_cpufreq_controller(d)) || rdmsr_safe(msr, *val) == 0 
> )
>              break;
> +        printk(KERN_DEBUG "JKD: MSR FAULT2\n");
>          goto gp_fault;
>  
>          /*
> 
> and now in the hypervisor log when the domain crashes:
> 
> (XEN) JKD: MSR 0x199 FAULT1: 0 & 0x2
> (XEN) d11v0 VIRIDIAN CRASH: 3b c0000096 1146d2c5 6346d580 0
> (XEN) avc:  denied  { reset } for domid=11 scontext=system_u:system_r:domU_t 
> tcontext=system_u:system_r:domU_t_self tclass=event
> 
> I'm not sure what is expected in cp->x86_vendor but this is running on an 
> Intel CPU so I would have thought 0x1 based on
> 
> #define X86_VENDOR_INTEL (1 << 0)

This is the problem - a bad backport. Therefore ...

>> The hypervisor log (at maximum log levels) accompanying this might
>> help some. And of course, if possible, trying on a newer Xen (ideally
>> current master).
> 
> We have a separate upgrade to 4.14.1 in progress and I will test on that too.

I'm sure you'll find this work there. I'll make a patch for the affected
older tree(s).

Jan