Re: Mapping memory into a domain

[Alejandro Vallejo <agarciav@xxxxxxx>]

On Tue May 6, 2025 at 3:02 AM CEST, Demi Marie Obenour wrote:
> On 5/5/25 7:32 AM, Alejandro Vallejo wrote:
>> I suppose this is still about multiplexing the GPU driver the way we
>> last discussed at Xen Summit?
>> 
>> On Mon May 5, 2025 at 12:51 AM CEST, Demi Marie Obenour wrote:
>>> What are the appropriate Xen internal functions for:
>>>
>>> 1. Turning a PFN into an MFN?
>>> 2. Mapping an MFN into a guest?
>>> 3. Unmapping that MFN from a guest?
>> 
>> The p2m is the single source of truth about such mappings.
>> 
>> This is all racy business. You want to keep the p2m lock for the full
>> duration of whatever operation you wish do, or you risk another CPU
>> taking it and pulling the rug under your feet at the most inconvenient
>> time.
>> 
>> In general all this faff is hidden under way too many layers beneath
>> copy_{to,from}_guest(). Other p2m manipulation high-level constructs
>> that might do interesting things worth looking at may be 
>> {map,unmap}_mmio_region()
>> 
>> Note that not every pfn has an associated mfn. Not even every valid pfn
>> has necessarily an associated mfn (there's pod). And all of this is
>> volatile business in the presence of a baloon driver or vPCI placing
>> mmio windows over guest memory.
>
> Can I check that POD is not in use?  

Maybe, but now you're reaching exponential complexity considering each
individual knob of the p2m into account.

>
>> In general anything up this alley would need a cohesive pair for
>> map/unmap and a credible plan for concurrency and how it's all handled
>> in conjunction with other bits that touch the p2m.
>
> Is taking the p2m lock for the entire operation a reasonable approach
> for concurrency?  Will this cause too much lock contention?

Maybe. It'd be fine for a page. Likely not so for several GiB if they
aren't already superpages.

>
>>> The first patch I am going to send with this information is a documentation
>>> patch so that others do not need to figure this out for themselves.
>>> I remember being unsure even after looking through the source code, which
>>> is why I am asking here.
>> 
>> That's not surprising. There's per-arch stuff, per-p2mtype stuff,
>> per-guesttype stuff. Plus madness like on-demand memory. It's no wonder
>> such helpers don't exist and the general manipulations are hard to
>> explain.
>
> Is this a task that is only suitable for someone who has several years
> experience working on Xen, or is it something that would make sense for
> someone who is less experienced?

The p2m is a very complex beast that integrates more features than I
care to count. It requires a lot of prior knowledge. Whoever does it
must know Xen fairly well in many configurations.

The real problem is finding the right primitives that do what you want
without overcomplicating everything else, preserving system security
invariants and have benign (and ideally clear) edge cases.

This was the last email you sent (I think?). Has any of the requirements
changed in any direction?

  https://lore.kernel.org/xen-devel/Z5794ysNE4KDkFuT@itl-email/

Something I'm missing there is how everything works without Xen. That
might help (me, at least) guage what could prove enough to support the
usecase. Are there sequence diagrams anywhere about how this whole thing
works without Xen? I vaguely remember you showing something last year in
Xen Summit in the design session, but my memory isn't that good :)

Cheers,
Alejandro