Re: Keystone Issue

[Marc Zyngier <maz@xxxxxxxxxx>]

On 2020-06-10 09:06, Bertrand Marquis wrote:
> Hi,
>
> > On 9 Jun 2020, at 18:45, Marc Zyngier <maz@xxxxxxxxxx> wrote:
> >
> > Hi Julien,
> >
> > On 2020-06-09 18:32, Julien Grall wrote:
> > > (+ Marc)
> > > On 09/06/2020 18:03, Bertrand Marquis wrote:
> > > > Hi
> > > > > On 9 Jun 2020, at 16:47, Julien Grall <julien@xxxxxxx> wrote:
> > > > > On 09/06/2020 16:28, Bertrand Marquis wrote:
> > > > > > Hi,
> > > > > > > On 9 Jun 2020, at 15:33, CodeWiz2280 <codewiz2280@xxxxxxxxx> 
> > > > > > > wrote:
> > > > > > > There does appear to be a secondary (CIC) controller that can 
> > > > > > > forward
> > > > > > > events to the GIC-400 and EDMA controllers for the keystone 2 
> > > > > > > family.
> > > > > > > Admittedly, i'm not sure how it is being used with regards to the
> > > > > > > peripherals.  I only see mention of the GIC-400 parent for the 
> > > > > > > devices
> > > > > > > in the device tree.  Maybe Bertrand has a better idea on whether 
> > > > > > > any
> > > > > > > peripherals go through the CIC first?  I see that gic_interrupt 
> > > > > > > ()
> > > > > > > fires once in Xen, which calls doIRQ to push out the virtual 
> > > > > > > interrupt
> > > > > > > to the dom0 kernel.  The dom0 kernel then handles the interrupt 
> > > > > > > and
> > > > > > > returns, but gic_interrupt() never fires again in Xen.
> > > > > > I do not remember of any CIC but the behaviour definitely look 
> > > > > > like an interrupt acknowledge problem.
> > > > > > Could you try the following:
> > > > > > --- a/xen/arch/arm/gic-v2.c
> > > > > > +++ b/xen/arch/arm/gic-v2.c
> > > > > > @@ -667,6 +667,9 @@ static void gicv2_guest_irq_end(struct 
> > > > > > irq_desc *desc)
> > > > > >      /* Lower the priority of the IRQ */
> > > > > >      gicv2_eoi_irq(desc);
> > > > > >      /* Deactivation happens in maintenance interrupt / via GICV 
> > > > > > */
> > > > > > +
> > > > > > +    /* Test for Keystone2 */
> > > > > > +    gicv2_dir_irq(desc);
> > > > > >  }
> > > > > > I think the problem I had was related to the vgic not deactivating 
> > > > > > properly the interrupt.
> > > > > Are you suggesting the guest EOI is not properly forwarded to the 
> > > > > hardware when LR.HW is set? If so, this could possibly be 
> > > > > workaround in Xen by raising a maintenance interrupt every time a 
> > > > > guest EOI an interrupt.
> > > > Agree the maintenance interrupt would definitely be the right 
> > > > solution
> > > I would like to make sure we aren't missing anything in Xen first.
> > > From what you said, you have encountered this issue in the past with 
> > > a
> > > different hypervisor. So it doesn't look like to be Xen related.
> > > Was there any official statement from TI? If not, can we try to get
> > > some input from them first?
> > > @Marc, I know you dropped 32-bit support in KVM recently :). 
> > > Although,
> > Yes! Victory is mine! Freedom from the shackles of 32bit, at last! :D
> >
> > > I was wondering if you heard about any potential issue with guest EOI
> > > not forwarded to the host. This is on TI Keystone (Cortex A-15).
> > Not that I know of. A-15 definitely works (TC2, Tegra-K1, Calxeda 
> > Midway all run just fine with guest EOI), and GIC-400 is a pretty 
> > solid piece of kit (it is just sloooooow...).
> > Thinking of it, you would see something like that if the GIC was 
> > seeing the writes coming from the guest as secure instead of NS (cue 
> > the early firmware on XGene that exposed the wrong side of GIC-400).
> > Is there some kind of funky bridge between the CPU and the GIC?
> Yes the behaviour I had was coherent with the GIC seeing the processor
> in secure mode and not in non secure hence making the VGIC ack non
> functional.
Can you please check this with the TI folks? It may be fixable if
the bridge is SW configurable.

> So the only way to solve this is actually to do the interrupt
> deactivate inside Xen (using a maintenance interrupt).
That's a terrible hack, and one that would encourage badly integrated 
HW.
I appreciate the need to "make things work", but I'd be wary of putting
this in released SW. Broken HW must die. I have written more than my 
share
of these terrible hacks (see TX1 support), and I deeply regret it, as
it has only given Si vendors an excuse not to fix things.

> I remember that I also had to do something specific for the
> configuration of edge/level and priorities to have an almost proper
> behaviour.
Well, the moment the GIC observes secure accesses when they should be
non-secure, all bets are off and you have to resort to the above hacks.
The fun part is that if you have secure SW running on this platform,
you can probably DoS it from non-secure. It's good, isn't it?

> Sadly I have no access to the code anymore, so I would need to guess
> back what that was..
I'd say this *is* a good thing.

        M.
--
Jazz is not dead. It just smells funny...