Re: sched=null vwfi=native and call_rcu()

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Thu, 6 Jan 2022, Julien Grall wrote:
> On 06/01/2022 00:40, Stefano Stabellini wrote:
> > As you might remember, we have an outstanding issue with call_rcu() when
> > sched=null vwfi=native are used. That is because in that configuration
> > the CPU never goes idle so rcu_idle_enter() never gets called.
> > 
> > The issue was caught on the field and I managed to repro the problem
> > doing the following:
> > 
> > xl destroy test
> > xl create ./test.cfg
> > 
> > Resulting in the following error:
> > 
> > # Parsing config from ./test.cfg
> > # (XEN) IRQ 54 is already used by domain 1
> > 
> > The test domain has 3 interrupts remapped to it and they don't get
> > released before the new domain creation is requested.
> > 
> > Just FYI, the below hacky patch seems to reliably work-around the
> > problem in my environment.
> > 
> > Do you have any suggestions on what would be the right way to solve
> > this issue?
> 
> This issue and solution were discussed numerous time on the ML. In short, we
> want to tell the RCU that CPU running in guest context are always quiesced.
> For more details, you can read the previous thread (which also contains a link
> to the one before):
> 
> https://lore.kernel.org/xen-devel/fe3dd9f0-b035-01fe-3e01-ddf065f182ab@xxxxxxxxx/

Thanks Julien for the pointer!

Dario, I forward-ported your three patches to staging:
https://gitlab.com/xen-project/people/sstabellini/xen/-/tree/rcu-quiet

I can confirm that they fix the bug. Note that I had to add a small
change on top to remove the ASSERT at the beginning of rcu_quiet_enter:
https://gitlab.com/xen-project/people/sstabellini/xen/-/commit/6fc02b90814d3fe630715e353d16f397a5b280f9

Would you be up for submitting them for upstreaming? I would prefer if
you send out the patches because I cannot claim to understand them
completely (except for the one doing renaming :-P )

I am also attaching the four patches for your convenience.

Attachment: 0001-xen-RCU-bootparam-to-force-quiescence-at-every-call.patch
Description: Text Data

Attachment: 0002-xen-rename-RCU-idle-timer-and-cpumask.patch
Description: Text Data

Attachment: 0003-xen-deal-with-vCPUs-that-do-not-yield-when-idle.patch
Description: Text Data

Attachment: 0004-xen-remove-ASSERT-in-rcu_quiet_enter.patch
Description: Text Data