Re: regression in recent pvops kernels, dom0 crashes early

[Jan Beulich <jbeulich@xxxxxxxx>]

On 13.05.2021 12:24, Olaf Hering wrote:
> Recent pvops dom0 kernels fail to boot on this particular ProLiant BL465c G5 
> box.
> It happens to work with every Xen and a 4.4 based sle12sp3 kernel, but fails 
> with every Xen and a 4.12 based sle12sp4 (and every newer) kernel.
> 
> Any idea what is going on?
> 
> ....
> (XEN) Freed 256kB init memory.
> (XEN) mm.c:1758:d0 Bad L1 flags 800000
> (XEN) traps.c:458:d0 Unhandled invalid opcode fault/trap [#6] on VCPU 0 
> [ec=0000]
> (XEN) domain_crash_sync called from entry.S: fault at ffff82d08022a2a0 
> create_bounce_frame+0x133/0x143
> (XEN) Domain 0 (vcpu#0) crashed on cpu#0:
> (XEN) ----[ Xen-4.4.20170405T152638.6bf0560e12-9.xen44  x86_64  debug=y  Not 
> tainted ]----
> ....
> 
> ....
> (XEN) Freed 656kB init memory
> (XEN) mm.c:2165:d0v0 Bad L1 flags 800000
> (XEN) d0v0 Unhandled invalid opcode fault/trap [#6, ec=ffffffff]
> (XEN) domain_crash_sync called from entry.S: fault at ffff82d04031a016 
> x86_64/entry.S#create_bounce_frame+0x15d/0x177
> (XEN) Domain 0 (vcpu#0) crashed on cpu#5:
> (XEN) ----[ Xen-4.15.20210504T145803.280d472f4f-6.xen415  x86_64  debug=y  
> Not tainted ]----
> ....
> 
> I can probably cycle through all kernels between 4.4 and 4.12 to see where it 
> broke.

I didn't try to figure out where exactly it broke, but could you give the
patch below a try, perhaps on top of my almost-one-year-old submission at
https://lkml.org/lkml/2020/5/27/1035?

Jan

x86/Xen: swap NX determination and GDT setup on BSP

xen_setup_gdt(), via xen_load_gdt_boot(), wants to adjust page tables.
For this to work when NX is not available, x86_configure_nx() needs to
be called first.

Reported-by: Olaf Hering <olaf@xxxxxxxxx>
Signed-off-by: Jan Beulich <jbeulich@xxxxxxxx>

--- a/arch/x86/xen/enlighten_pv.c
+++ b/arch/x86/xen/enlighten_pv.c
@@ -1262,16 +1262,16 @@ asmlinkage __visible void __init xen_sta
        /* Get mfn list */
        xen_build_dynamic_phys_to_machine();
 
+       /* Work out if we support NX */
+       get_cpu_cap(&boot_cpu_data);
+       x86_configure_nx();
+
        /*
         * Set up kernel GDT and segment registers, mainly so that
         * -fstack-protector code can be executed.
         */
        xen_setup_gdt(0);
 
-       /* Work out if we support NX */
-       get_cpu_cap(&boot_cpu_data);
-       x86_configure_nx();
-
        /* Determine virtual and physical address sizes */
        get_cpu_address_sizes(&boot_cpu_data);