Re: [Xen-devel] [PATCH] xen: Avoid calling device suspend/resume callbacks

[Marek Marczykowski-Górecki <marmarek@xxxxxxxxxxxxxxxxxxxxxx>]

On Tue, Jul 30, 2019 at 07:55:02AM +0000, Jan Beulich wrote:
> On 29.07.2019 19:06, Andrew Cooper wrote:
> > On 29/07/2019 16:57, Jan Beulich wrote:
> >> On 29.07.2019 17:41, Ross Lagerwall wrote:
> >>> When suspending/resuming or migrating under Xen, there isn't much need
> >>> for suspending and resuming all the attached devices since the Xen/QEMU
> >>> should correctly maintain the hardware state. Drop these calls and
> >>> replace with more specific calls to ensure Xen frontend devices are
> >>> properly reconnected.
> >> Is this true for the general pass-through case as well? While migration
> >> may not be (fully) compatible with pass-through, iirc save/restore is.
> > 
> > What gives you this impression?
> > 
> > Migration and save/restore are *literally* the same thing, except that
> > in one case you're piping the data to/from disk, and in the other you're
> > piping it to the destination and restoring it immediately.
> > 
> > If you look at the toolstack code, it is all in terms of reading/writing
> > an fd (including libxl's API) which is either a network socket or a
> > file, as chosen by xl.
> 
> Sure. The main difference is where the restore happens (by default):
> For live migration I expect this to be on a different host, whereas
> for a non-live restore I'd expect this to be the same host. And it
> is only the "same host" case where one can assume the same physical
> piece of hardware to be available again for passing through to this
> guest. In the "different host" case restore _may_ be possible, using
> identical hardware. (And yes, in the "same host" case restore may
> also be impossible, if the hardware meanwhile has been assigned to
> another guest. But as said, I'm talking about the default case here.)
> 
> >> Would qemu restore state of physical PCI devices?
> > 
> > What state would Qemu be in a position to know about, which isn't
> > already present in Qemu's datablob?
> 
> That's a valid (rhetorical) question, but not helping to answer mine.
> 
> > What we do with graphics cards is to merge Xens logdirty bitmap, with a
> > dirty list provided by the card itself.  This needs a device-specific
> > knowledge.  In addition, there is an opaque blob of data produced by the
> > source card, which is handed to the destination card.  That also lives
> > in the stream.
> > 
> > Intel's Scalable IOV spec is attempting to rationalise this by having a
> > standard ways of getting logdirty and "internal state" information out
> > of a device, but for the moment, it requires custom device-driver
> > specific code to do anything migration related with real hardware.
> 
> Which isn't very nice, since it doesn't scale well as a model.
> 
> > As for why its safe to do like this, the best argument is that this is
> > how all other vendors do migration, including KVM.  Xen is the
> > odd-one-out using the full S3 path.
> 
> So how do "all other vendors" deal with device specific state? So
> far I was under the impression that to deal with this is precisely
> why we use the S3 logic in the kernel.

FWIW in Qubes we specifically S3 domUs with PCI devices assigned just
before host S3 - to let the driver save the state and later restore it.
AFAIR in same cases (but it might be PV, not HVM then) not doing so was
breaking host S3 altogether.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

Attachment: signature.asc
Description: PGP signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel