Re: Enabling hypervisor agnosticism for VirtIO backends

[Oleksandr Tyshchenko <olekstysh@xxxxxxxxx>]

Hi Akashi,

I am sorry for the possible format issues.

> > >
> > > It's a RFC discussion. We have tested this RFC patch internally.
> > > https://lists.xenproject.org/archives/html/xen-devel/2021-
> > 07/msg01532.html
> >
> > I'm afraid that I miss something here, but I don't know
> > why this proposed API will lead to eliminating 'mmap' in accessing
> > the queued payload at every request?
> >
>
> This API give Xen device model (QEMU or kvmtool) the ability to map
> whole guest RAM in device model's address space. In this case, device
> model doesn't need dynamic hypercall to map/unmap payload memory.
> It can use a flat offset to access payload memory in its address
> space directly. Just Like KVM device model does now.

Yes!

 

Thank you. Quickly, let me make sure one thing:
This API itself doesn't do any mapping operations, right?

 

Right. The only purpose of that "API" is to guery hypervisor to find unallocated address space ranges to map the foreign pages into (instead of stealing real RAM pages),

In a nutshell, if you try to map the whole guest memory in the backend address space on Arm (or even cache some mappings) you might end up with memory exhaustion in the backend domain (XSA-300),

and the possibility to hit XSA-300 is higher if your backend needs to serve several Guests. Of course, this depends on the memory assigned to the backend domain and to the Guest(s) it serves...

We believe that with the proposed solution the backend will be able to handle Guest(s) without wasting it's real RAM. However, please note that proposed Xen + Linux changes which are on review now [1] are far from the final solution and require rework and some prereq work to operate in a proper and safe way.

 

So I suppose that virtio BE guest is responsible to
1) fetch the information about all the memory regions in FE,
2) call this API to allocate a big chunk of unused space in BE,
3) create grant/foreign mappings for FE onto this region(S)
in the initialization/configuration of emulated virtio devices.

Is this the way this API is expected to be used?

 

No really, the userspace backend doesn't need to call this API at all, all what backend calls is still xenforeignmemory_map()/xenforeignmemory_unmap(), so let's say "magic" is done by Linux and Xen internally.

You can take a look at the virtio-disk PoC [2] (last 4 patches) to better understand what Wei and I are talking about. There we map the Guest memory at the beginning and just calculate a pointer at runtime. Again, the code

is not in good shape, but it is enough to demonstrate the feasibility of the improvement.

 

Does Xen already has an interface for (1)?

 

I am not aware of anything existing. For the PoC I guessed the Guest memory layout in a really hackish way (I got total Guest memory size, so having GUEST_RAMX_BASE/GUEST_RAMX_SIZE in hand just performed calculation). Definitely, it is a no-go, so 1) deserves additional discussion/design.

[1]
https://lore.kernel.org/xen-devel/1627489110-25633-1-git-send-email-olekstysh@xxxxxxxxx/
https://lore.kernel.org/lkml/1627490656-1267-1-git-send-email-olekstysh@xxxxxxxxx/
https://lore.kernel.org/lkml/1627490656-1267-2-git-send-email-olekstysh@xxxxxxxxx/

[2] 

https://github.com/otyshchenko1/virtio-disk/commits/map_opt_next

--

Regards,

Oleksandr Tyshchenko