Re: [PATCH v1 02/17] xen/riscv: add basic VGEIN management for AIA guests

[Jan Beulich <jbeulich@xxxxxxxx>]

On 31.07.2026 16:59, Oleksii Kurochko wrote:
> 
> 
> On 7/30/26 6:03 PM, Jan Beulich wrote:
>> On 30.07.2026 17:46, Oleksii Kurochko wrote:
>>> On 7/30/26 9:42 AM, Jan Beulich wrote:
>>>> On 29.07.2026 16:55, Oleksii Kurochko wrote:
>>>>> On 7/27/26 5:41 PM, Jan Beulich wrote:
>>>>>> On 20.07.2026 18:02, Oleksii Kurochko wrote:
>>>>>>> It was decided to add support for IMSIC from the start instead of 
>>>>>>> having APLIC
>>>>>>> operate in direct delivery mode, as it requires a trap-and-emulation 
>>>>>>> approach,
>>>>>>> which is not optimal from a performance standpoint.
>>>>>>>
>>>>>>> AIA provides a hardware-accelerated mechanism for delivering external
>>>>>>> interrupts to domains via "guest interrupt files" located in IMSIC.
>>>>>>> A single physical hart can implement multiple such files (up to GEILEN),
>>>>>>> allowing several virtual harts to receive interrupts directly from 
>>>>>>> hardware.
>>>>>>>
>>>>>>> Introduce per-CPU tracking of guest interrupt file identifiers (VGEIN)
>>>>>>> for systems implementing AIA specification. Each CPU maintains
>>>>>>> a bitmap describing which guest interrupt files are currently in use.
>>>>>>>
>>>>>>> Add helpers to initialize the bitmap based on the number of available
>>>>>>> guest interrupt files (GEILEN), assign a VGEIN to a vCPU, and release it
>>>>>>> when no longer needed. When assigning a VGEIN, the corresponding value
>>>>>>> is written to the VGEIN field of the guest hstatus register so that
>>>>>>> VS-level external interrupts are delivered from the selected interrupt
>>>>>>> file.
>>>>>>
>>>>>> And when exactly is this "assignment" intended to occur? vgein_assign() 
>>>>>> and
>>>>>> vgein_release() have no callers here, so this remains entirely unclear.
>>>>>
>>>>> [A] Agreed, I should have added that information to the commit message:
>>>>>
>>>>> VGEIN is assigned (via vgein_assign()) before jumping to the new vCPU
>>>>> execution context (in continue_new_vcpu()) and is re-assigned during
>>>>> vCPU migration from one pCPU to another.
>>>>>
>>>>> VGEIN is released (via vgein_release()) on the old pCPU during migration.
>>>>
>>>> That is, state of that vCPU is held in hardware for perhaps an extended
>>>> period of time after the vCPU was last de-scheduled. That's a fair
>>>> optimization (we do something similar on x86, albeit that has been
>>>> increasingly under question lately). However, doesn't this then require
>>>> sync_local_execstate() to become non-empty?
>>>
>>> IIUC, sync_local_execstate() is needed for the lazy context switch case
>>> when switching from vCPUA to the idle vCPU.
>>
>> Or when full state is to be obtained for a vCPU, for example.
> 
> I assume you're referring to XEN_DOMCTL_getvcpucontext, right?

Yes.

> In general, it seems that sync_local_execstate() is primarily an 
> optimization. If lazy switching isn't supported, then every time a vCPU 
> is de-scheduled, its state must be fully saved to memory. My 
> understanding is that everything will still work correctly, just less 
> efficiently.

The lazy switching is an optimization, yes. If any state is kept in
hardware, sync_local_execstate() has to be used when full state of a
vCPU is to be obtained. Supplying back stale state of "guest interrupt
files" can't be correct. (Of course you can also arrange to obtain
up-to-date state by custom means, but imo that's likely less desirable.)

> I'm curious how much this optimization actually helps. How often does it 
> happen that a vCPU is de-scheduled from a pCPU and then immediately 
> scheduled back onto the same pCPU without any other vCPU being scheduled 
> in between?

That heavily depends on overall load of the system. When pCPU-s aren't
over-subscribed, a HVM vCPU getting de-scheduled to wait for qemu to
handle a certain operation may very well be able to resume on the same
pCPU after completion of the ioreq. The less overhead there, the better.
(Just to give an example.)

Jan