Re: [Xen-devel] [PATCH v2 00/20] VM forking

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Wed, Jan 08, 2020 at 11:14:46AM -0700, Tamas K Lengyel wrote:
> On Wed, Jan 8, 2020 at 11:01 AM Roger Pau Monné <roger.pau@xxxxxxxxxx> wrote:
> >
> > On Wed, Jan 08, 2020 at 08:32:22AM -0700, Tamas K Lengyel wrote:
> > > On Wed, Jan 8, 2020 at 8:08 AM Roger Pau Monné <roger.pau@xxxxxxxxxx> 
> > > wrote:
> > > >
> > > > On Tue, Dec 31, 2019 at 09:36:01AM -0700, Tamas K Lengyel wrote:
> > > > > On Tue, Dec 31, 2019 at 9:08 AM Tamas K Lengyel <tamas@xxxxxxxxxxxxx> 
> > > > > wrote:
> > > > > >
> > > > > > On Tue, Dec 31, 2019 at 8:11 AM Roger Pau Monné 
> > > > > > <roger.pau@xxxxxxxxxx> wrote:
> > > > > > >
> > > > > > > On Tue, Dec 31, 2019 at 08:00:17AM -0700, Tamas K Lengyel wrote:
> > > > > > > > On Tue, Dec 31, 2019 at 3:40 AM Roger Pau Monné 
> > > > > > > > <roger.pau@xxxxxxxxxx> wrote:
> > > > > > > > >
> > > > > > > > > On Mon, Dec 30, 2019 at 05:37:38PM -0700, Tamas K Lengyel 
> > > > > > > > > wrote:
> > > > > > > > > > On Mon, Dec 30, 2019 at 5:20 PM Julien Grall 
> > > > > > > > > > <julien.grall@xxxxxxxxx> wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hi,
> > > > > > > > > > >
> > > > > > > > > > > On Mon, 30 Dec 2019, 20:49 Tamas K Lengyel, 
> > > > > > > > > > > <tamas@xxxxxxxxxxxxx> wrote:
> > > > > > > > > > >>
> > > > > > > > > > >> On Mon, Dec 30, 2019 at 11:43 AM Julien Grall 
> > > > > > > > > > >> <julien@xxxxxxx> wrote:
> > > > > > > > > > >> But keep in mind that the "fork-vm" command even with 
> > > > > > > > > > >> this update
> > > > > > > > > > >> would still not produce for you a "fully functional" VM 
> > > > > > > > > > >> on its own.
> > > > > > > > > > >> The user still has to produce a new VM config file, 
> > > > > > > > > > >> create the new
> > > > > > > > > > >> disk, save the QEMU state, etc.
> > > > > > > > >
> > > > > > > > > IMO the default behavior of the fork command should be to 
> > > > > > > > > leave the
> > > > > > > > > original VM paused, so that you can continue using the same 
> > > > > > > > > disk and
> > > > > > > > > network config in the fork and you won't need to pass a new 
> > > > > > > > > config
> > > > > > > > > file.
> > > > > > > > >
> > > > > > > > > As Julien already said, maybe I wasn't clear in my previous 
> > > > > > > > > replies:
> > > > > > > > > I'm not asking you to implement all this, it's fine if the
> > > > > > > > > implementation of the fork-vm xl command requires you to pass 
> > > > > > > > > certain
> > > > > > > > > options, and that the default behavior is not implemented.
> > > > > > > > >
> > > > > > > > > We need an interface that's sane, and that's designed to be 
> > > > > > > > > easy and
> > > > > > > > > comprehensive to use, not an interface built around what's 
> > > > > > > > > currently
> > > > > > > > > implemented.
> > > > > > > >
> > > > > > > > OK, so I think that would look like "xl fork-vm <parent_domid>" 
> > > > > > > > with
> > > > > > > > additional options for things like name, disk, vlan, or a 
> > > > > > > > completely
> > > > > > > > new config, all of which are currently not implemented, + an
> > > > > > > > additional option to not launch QEMU at all, which would be the 
> > > > > > > > only
> > > > > > > > one currently working. Also keeping the separate "xl 
> > > > > > > > fork-launch-dm"
> > > > > > > > as is. Is that what we are talking about?
> > > > > > >
> > > > > > > I think fork-launch-vm should just be an option of fork-vm (ie:
> > > > > > > --launch-dm-only or some such). I don't think there's a reason to 
> > > > > > > have
> > > > > > > a separate top-level command to just launch the device model.
> > > > > >
> > > > > > It's just that the fork-launch-dm needs the domid of the fork, while
> > > > > > the fork-vm needs the parent's domid. But I guess we can interpret 
> > > > > > the
> > > > > > "domid" required input differently depending on which sub-option is
> > > > > > specified for the command. Let's see how it pans out.
> > > > >
> > > > > How does the following look for the interface?
> > > > >
> > > > >     { "fork-vm",
> > > > >       &main_fork_vm, 0, 1,
> > > > >       "Fork a domain from the running parent domid",
> > > > >       "[options] <Domid>",
> > > > >       "-h                           Print this help.\n"
> > > > >       "-N <name>                    Assign name to VM fork.\n"
> > > > >       "-D <disk>                    Assign disk to VM fork.\n"
> > > > >       "-B <bridge                   Assign bridge to VM fork.\n"
> > > > >       "-V <vlan>                    Assign vlan to VM fork.\n"
> > > >
> > > > IMO I think the name of fork is the only useful option. Being able to
> > > > assign disks or bridges from the command line seems quite complicated.
> > > > What about VMs with multiple disks? Or VMs with multiple nics on
> > > > different bridges?
> > > >
> > > > I think it's easier for both the implementation and the user to just
> > > > use a config file in that case.
> > >
> > > I agree but it sounded to me you guys wanted to have a "complete"
> > > interface even if it's unimplemented. This is what a complete
> > > interface would look to me.
> >
> > I would add those options afterwards if there's a need for them. I was
> > mainly concerned about introducing a top level command (ie: fork-vm)
> > that would require calling other commands in order to get a functional
> > fork. I'm not so concerned about having all the possible options
> > listed now, as long as the default behavior of fork-vm is something
> > sane that produces a working fork, even if not fully implemented at
> > this stage.
> 
> OK
> 
> > > > Why do you need a config file for launching the Qemu device model?
> > > > Doesn't the save-file contain all the information?
> > >
> > > The config is used to populate xenstore, not just for QEMU. The QEMU
> > > save file doesn't contain the xl config. This is not a full VM save
> > > file, it is only the QEMU state that gets dumped with
> > > xen-save-devices-state.
> >
> > TBH I think it would be easier to have something like my proposal
> > below, where you tell xl the parent and the forked VM names and xl
> > does the rest. Even better would be to not have to tell xl the parent
> > VM name (since I guess this is already tracked internally somewhere?).
> 
> The forked VM has no "name" when it's created. For performance reasons
> when the VM fork is created with "--launch-dm no" we explicitly want
> to avoid touching Xenstore. Even parsing the config file would be
> unneeded overhead at that stage.

I think you need another option to tell xl to not name the forked VM,
abusing of "--launch-dm no" to not set a name is not expected I think.

> >
> > Anyway, I'm not going to insist on this but the workflow of the Qemu
> > forking seems to not be very user friendly unless you know exactly how
> > to use it.
> >
> > >
> > > >
> > > > I think you also need something like:
> > > >
> > > > # xl fork-vm --launch-dm late <parent_domid> <fork_domid>
> > > >
> > > > So that a user doesn't need to pass a qemu-save-file?
> > >
> > > This doesn't make much sense to me. To launch QEMU you need the config
> > > file to wire things up correctly. Like in order to launch QEMU you
> > > need to tell it the name of the VM, disk path, etc. that are all
> > > contained in the config.
> >
> > You could get all this information from the parent VM, IIRC libxl has
> > a json version of the config. For example for migration there's no
> > need to pass any config file, since the incoming VM can be recreated
> > from the data in the source VM.
> >
> 
> But again, creating a fork with the exact config of the parent is not
> possible. Even if the tool would rename the fork on-the-fly as it does
> during the migration, the fork would end up thrashing the parent VM's
> disk and making it impossible to create any additional forks. It would
> also mean that at no point can the original VM be unpaused after the
> forks are gone. I don't see any usecase in which that would make any
> sense at all.

You could have the disk(s) as read-only and the VM running completely
from RAM. Alpine-linux has (or had) a mode where it was completely
stateless and running from RAM. I think it's fine to require passing a
config file for the time being, we can look at other options
afterwards.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel