Re: [BUG] Unable to boot Xen 4.11 (shipped with Ubuntu) on Intel 10i3 CPU

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 28/12/2020 18:08, Ondrej Balaz wrote:
> Hi,
> I recently updated my home server running Ubuntu 20.04 (Focal) with
> Xen hypervisor 4.11 (installed using Ubuntu packages). Before the
> upgrade all was running fine and both dom0 and all domUs were booting
> fine. Upgrade was literally taking harddrive from 6th gen Intel CPU
> system to 10th gen Intel CPU one and redoing EFI entries from Ubuntu
> live USB.
>
> After doing so standalone Ubuntu (without Xen multiboot) boots just
> fine but Ubuntu as dom0 with Xen fails pretty early on with following
> error (hand-copied from phone snaps I took with loglvl=all as this is
> barebone system without serial port and I don't know how to dump full
> logs in case of panic):
>
> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[01])
> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> (XEN) ACPI: IRQ0 used by override.
> (XEN) ACPI: IRQ2 used by override 
> (XEN) ACPI: IRQ9 used by override
> (XEN) Enabling APIC mode: Flat.  Using 1 I/O APICs
> (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
> (XEN) ERST table was not found
> (XEN) ACPI: BGRT: invalidating v1 image at 0x7d7c1018
> (XEN) Using ACPI (MADT) for SMP configuration information
> ...
> (XEN) Switched to APIC driver x2apic_cluster
> ...  
> (XEN) Initing memory sharing.
> (XEN) alt table ffff82d08042b840 -> ffff82d08042d7ce
> ...
> (XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) Intel VT-d Snoop Control not enabled 
> (XEN) Intel VT-d Dom0 DMA Passthrough not enabled 
> (XEN) Intel VT-d Queued Invalidation enabled 
> (XEN) Intel VT-d Interrupt Remapping enabled
> (XEN) Intel VT-d Posted Interrupt not enabled  
> (XEN) Intel VT-d Shared EPT tables enabled
> (XEN) I/O virtualisation enabled
> (XEN)  - Dom0 mode: Relaxed
> (XEN) Interrupt remapping enabled
> (XEN) nr_sockets: 1
> (XEN) Enabled directed EOI with ioapic_ack_old on!
> (XEN) ENABLING IO-APIC IRQs
> (XEN)  -> Using old ACK method
> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
> (XEN) ..MP-BIOS bug: 8254 timer not connected to IO-APIC
> (XEN) ...trying to set up timer (IRQ0) through the 8259A ... failed.
> (XEN) ...trying to set up timer as Virtual Wire IRQ... failed.
> (XEN) ...trying to set up timer as ExtINT IRQ...spurious 8259A
> interrupt IRQ7.
> (XEN) CPU0: No irq handler for vector e7 (IRQ -8)
> (XEN) IRQ7 a=0001[0001,0000] v=60[ffffffff] t=IO-APIC-edge s=00000002
> (XEN)  failed :(.
> (XEN)
> (XEN) *******************************
> (XEN) Panic on CPU 0:
> (XEN) IO-APIC + timer doesn't work!  Boot with apic_verbosity=debug
> and send report.  Then try booting with the `noapic` option
> (XEN) *******************************
>
> I suspected that migration of drive could cause problem so I took an
> empty SSD and installed fresh Ubuntu and added Xen hypervisor, after
> reboot I ended up with same panic. I tried booting with noapic (gave
> general page fault) and iommu=0 (said it needs iommu=required/force).
> Trying to boot this exact fresh install on older (6th gen) Intel CPU
> succeeded. I happen to have access to one more system with 10th gen
> Intel CPUs (Lenovo laptop) and no luck booting Xen there too and same
> panic in the end.
>
> Back to my barebone I tried to match BIOS settings between working and
> non-working but it didn't help. Virtualization is enabled, both
> systems are from the same maker (Intel NUC barebones), both systems
> are EFI enabled/secure boot disabled (the later one doesn't seem to
> have an option to disable EFI boot and boot using MBR).
>
> Is this something known? Are there any boot options that can
> potentially fix this?
>
> Any help (including how to dump full Xen boot logs without serial)
> appreciated.

Yes we're aware of it.  It is because modern Intel systems no longer
have a legacy PIT configured by default, and Xen depends on this.  (The
error message is misleading.  It's not checking for a timer, so much as
checking that interrupts works, and depends on the legacy PIT "working"
as the source of interrupts.)

I'm working on a fix.

~Andrew