ISPENDR implementation (WAS Re: [linux-linus test] 159463: regressions - FAIL)

[Julien Grall <julien.grall.oss@xxxxxxxxx>]

Hi all,

On Fri, 19 Feb 2021 at 22:19, osstest service owner
<osstest-admin@xxxxxxxxxxxxxx> wrote:
>
> flight 159463 linux-linus real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/159463/

[...]

>  test-arm64-arm64-xl-seattle                                  fail

[...]

>  test-arm64-arm64-xl-thunderx                                 pass

While looking at the log to check whether we fixed the Arm bug, I
noticed that Linux will boot on Thunder-X but not Seattle.

>From the log:

(XEN) d0v3: vGICD: unhandled word write 0x00000020000000 to ISPENDR44
Feb 18 17:01:19.426532 (XEN) traps.c:2013:d0v3 HSR=0x93820047
pc=0xffff8000104aec2c gva=0xffff80001000522c gpa=0x000000e111022c

[...]

Feb 18 17:01:19.618568 [   27.097702] Call trace:

Feb 18 17:01:19.618612 [   27.100215]  gic_retrigger+0x2c/0x38

Feb 18 17:01:19.630516 [   27.103861]  irq_startup+0x78/0x138

Feb 18 17:01:19.630575 [   27.107419]  __enable_irq+0x70/0x80

Feb 18 17:01:19.630622 [   27.110978]  enable_irq+0x50/0xa0

Feb 18 17:01:19.642499 [   27.114363]  xgbe_one_poll+0xc8/0xd8

Feb 18 17:01:19.642558 [   27.118009]  net_rx_action+0x110/0x3a8

Feb 18 17:01:19.642605 [   27.121828]  __do_softirq+0x124/0x288

Feb 18 17:01:19.654496 [   27.125560]  irq_exit+0xe0/0xf0

Feb 18 17:01:19.654555 [   27.128772]  __handle_domain_irq+0x68/0xc0

Feb 18 17:01:19.654603 [   27.132939]  gic_handle_irq+0xa8/0xe0

Feb 18 17:01:19.654647 [   27.136671]  el1_irq+0xb0/0x180

Feb 18 17:01:19.666482 [   27.139883]  arch_cpu_idle+0x18/0x28

Feb 18 17:01:19.666540 [   27.143528]  default_idle_call+0x24/0x5c

Feb 18 17:01:19.666587 [   27.147524]  do_idle+0x204/0x278

Feb 18 17:01:19.678517 [   27.150819]  cpu_startup_entry+0x24/0x68

Feb 18 17:01:19.678577 [   27.154812]  secondary_start_kernel+0x174/0x188

Feb 18 17:01:19.678625 [   27.159415] Code: f9409063 d37e6821 91080021
8b010061 (b9000022)

Feb 18 17:01:19.690480 [   27.165582] ---[ end trace a7aadb3ae629b57f ]---

It looks like that Linux will now try to set the interrupt pending by
writing ISPENDR when the interrupt is re-enabled.

I think the ISPENDR write emulation is easier to implement compare to
the other missing IS{PENDR, ACTIVER).

It should be possible to emulate as follows:
  1) For virtual interrupts, just call vgic_inject_irq()
  2) For physical interrupts, set pending at the HW level. This will
raise an interrupt that will call vgic_inject_irq().

The vGIC in KVM will directly set the physical interrupt active to
avoid the round trip. But I am not sure we can do it safely in our
current vGIC to avoid the guest de-activating the interrupt too early
(the virtual interrupt may already be pending/active).

Any thoughts?

Cheers,