Re: [Xen-devel] [PATCH v3 33/35] OvmfPkg: Introduce XenIoPvhDxe to initialize Grant Tables

[Anthony PERARD <anthony.perard@xxxxxxxxxx>]

On Wed, Jul 10, 2019 at 04:05:02PM +0200, Laszlo Ersek wrote:
> On 07/04/19 16:42, Anthony PERARD wrote:
> > +
> > +STATIC
> > +VOID
> > +EFIAPI
> > +XenIoPvhDxeNotifyExitBoot (
> > +  IN EFI_EVENT  Event,
> > +  IN VOID       *Context
> > +  )
> > +{
> > +  XEN_IO_PVH_STATE *State;
> > +  EFI_STATUS Status;
> > +
> > +  State = Context;
> > +
> > +  gBS->CloseEvent(&State->ExitBootEvent);
> > +  Status = XenIoMmioUninstall(State->XenIoHandle);
> > +  if (Status == EFI_SUCCESS) {
> > +    //
> > +    // Only free the reserved space for grant table if no driver is using 
> > it.
> > +    //
> > +    FreePages (State->Allocation, XEN_GRANT_FRAMES);
> > +  }
> > +  FreePool (State);
> > +}
> 
> In my v2 review at
> 
> http://mid.mail-archive.com/2c386393-b886-a7e7-e8b9-c5e339c94727@xxxxxxxxxx
> 
> I asked, in point (6), whether we needed the memory allocation to be
> "reserved". I guess this change (in v3) is supposed to address that.
> (And, indeed, version 3 of this patch correctly addresses all other
> points from my v2 review.)
> 
> However: an ExitBootServices notification function *must not* perform
> actions that could change the UEFI memory map. Thus, you can not release
> memory there -- you can't call CloseEvent(),
> UninstallMultipleProtocolInterfaces(), FreePool(), FreePages(), and so on.
> 
> In my earlier review, I wrote,
> 
> > [...] then we could allocate EfiBootServicesData type memory here --
> > and perhaps add an ExitBootServices() callback to disconnect from Xen
> > [...]
> 
> By "disconnect", I meant actions that do *not* include memory allocation
> / release, but only make the hypervisor *forget* its guest RAM
> references that it currently holds.
> 
> ExitBootServices() handlers (notification functions) are different from
> normal "teardown" functions. A "teardown" function generally mirrors a
> "construct" function, where you remove all references, and release all
> memory allocations, in precise reverse order of construction.
> 
> ExitBootServices() handlers perform a *subsequence* of that (usually
> preserving the same relative order between the steps): you only erase
> external references to RAM -- that is, RAM references that are held by
> external entities, such as PCI devices (in-flight DMA), the hypervisor,
> and so on. The referred-to RAM, originally allocated as "boot services
> data", will be released later, whole-sale, by the OS.
> 
> In that sense, ExitBootServices() callbacks implement a kind of
> ownership transfer. You no longer own the UEFI memory map, you just have
> to kick out such external references into it that the OS would not know
> about.
> 
> To summarize:
> 
> - If this memory *needs* to be reserved past ExitBootServices(), then
> stick with AllocateReservedPages(). Otherwise, use AllocatePages(); then
> the pages will be freed automatically by the OS, soon after it is started.
> 
> - In the latter case, if necessary, you can make the hypervisor forget
> about the area, as part of ExitBootServices(), in a notification
> function. But that can only happen without allocating or releasing
> memory in the notification function.
> 
> 
> ... I guess you could make a XENMEM_remove_from_physmap hypercall, for
> example. But, it's *entirely* possible that said hypercall doesn't
> belong in *this* driver. After all, this is not the driver that calls
> XENMEM_add_to_physmap either!
> 
> ... In fact, I think XenBusDxe has a bug. Right now, you have the
> following call chain, when the OS calls ExitBootServices():
> 
> NotifyExitBoot()
>   gBS->DisconnectController()
>     XenBusDxeDriverBindingStop()
>       XenGrantTableDeinit()
>         XenHypercallMemoryOp (XENMEM_remove_from_physmap)
> 
> Unfortunately, this call tree releases memory left and right, and so it
> is not valid on the stack of gBS->ExitBootServices().
> 
> Instead, NotifyExitBoot() should collect just the subsequence of
> XenBusDxeDriverBindingStop() -- interpreting that function as a flatened
> sequence of operations -- that removes external references to RAM.
> Making hypercalls (using parameters constructed on the stack) is fine.
> Using dynamically *pre*allocated memory (for constructing hypercall
> arguments) is also fine. Dynamically allocating or releasing memory *on
> the spot* is not fine.
> 
> (2) So, for now, I'd recommend simply removing
> XenIoPvhDxeNotifyExitBoot(), in this patch.

Ok, I'll remove it.

> You might want to fix XenBusDxe's NotifyExitBoot(), separately.

I'll put that on my TODO list.

> > +    return EFI_UNSUPPORTED;
> > +  }
> > +
> > +  State = AllocatePool (sizeof (*State));
> 
> (4) What do we gain by allocating "State" dynamically? IMO, it could be
> a local variable (= "automatic storage duration").

Without the exit boot notification, State isn't needed anymore, so I'll
remove it.

> > +  if (State == NULL) {
> > +    Status = EFI_OUT_OF_RESOURCES;
> > +    goto Error;
> > +  }
> > +
> > +  Allocation = AllocateReservedPages (XEN_GRANT_FRAMES);
> 
> (5) So, again, please evaluate if this could simply be AllocatePages().

I would prefer to let know the operating system that those pages are
potentially used, unless I'm sure OVMF has let know Xen that those
pages don't contain the grant table anymore.

I'll probably try to make some changes in OVMF to allow to give back
those pages, but for now I think keeping those pages as reserved will
be good enough.

Thanks,

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel