Re: [PATCH] xen/x86: allow Dom0 PVH to call XENMEM_exchange

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Mon, 5 May 2025, Roger Pau Monné wrote:
> On Mon, May 05, 2025 at 12:40:18PM +0200, Marek Marczykowski-Górecki wrote:
> > On Mon, Apr 28, 2025 at 01:00:01PM -0700, Stefano Stabellini wrote:
> > > On Mon, 28 Apr 2025, Jan Beulich wrote:
> > > > On 25.04.2025 22:19, Stefano Stabellini wrote:
> > > > > From: Xenia Ragiadakou <Xenia.Ragiadakou@xxxxxxx>
> > > > > 
> > > > > Dom0 PVH might need XENMEM_exchange when passing contiguous memory
> > > > > addresses to firmware or co-processors not behind an IOMMU.
> > > > 
> > > > I definitely don't understand the firmware part: It's subject to the
> > > > same transparent P2M translations as the rest of the VM; it's just
> > > > another piece of software running there.
> > > > 
> > > > "Co-processors not behind an IOMMU" is also interesting; a more
> > > > concrete scenario might be nice, yet I realize you may be limited in
> > > > what you're allowed to say.
> > > 
> > > Sure. On AMD x86 platforms there is a co-processor called PSP running
> > > TEE firmware. The PSP is not behind an IOMMU. Dom0 needs occasionally to
> > > pass addresses to it.  See drivers/tee/amdtee/ and
> > > include/linux/psp-tee.h in Linux.
> > 
> > We had (have?) similar issue with amdgpu (for integrated graphics) - it
> > uses PSP for loading its firmware. With PV dom0 there is a workaround as
> > dom0 kinda knows MFN. I haven't tried PVH dom0 on such system yet, but I
> > expect troubles (BTW, hw1 aka zen2 gitlab runner has amdgpu, and it's
> > the one I used for debugging this issue).
> 
> That's ugly, and problematic when used in conjunction with AMD-SEV.
> 
> I wonder if Xen could emulate/mediate some parts of the PSP for dom0
> to use, while allowing Xen to be the sole owner of the device.  Having
> both Xen and dom0 use it (for different purposes) seems like asking
> for trouble.  But I also have no idea how complex the PSP interface
> is, neither whether it would be feasible to emulate the
> interfaces/registers needed for firmware loading.

Let me take a step back from the PSP for a moment. I am not opposed to a
PSP mediator in Xen, but I want to emphasize that the issue is more
general and extends well beyond the PSP.

In my years working in embedded systems, I have consistently seen cases
where Dom0 needs to communicate with something that does not go through
the IOMMU. This could be due to special firmware on a co-processor, a
hardware erratum that prevents proper IOMMU usage, or a high-bandwidth
device that technically supports the IOMMU but performs poorly unless
the IOMMU is disabled. All of these are real-world examples that I have
seen personally.

In my opinion, we definitely need a solution like this patch for Dom0
PVH to function correctly in all scenarios.

Additionally, we could add a PSP mediator in Xen to provide best PSP
support, and also for cases where both Xen and Dom0 need access, but I
cannot fully assess the complexity involved, as I am not very familiar
with the API. Probably it is not going to be simple.

Even with the PSP mediator in place, I would still recommend this patch.