Xen project Mailing List

Hello,

I think it would be beneficial to add local disk migration feature for ‘blkback' backend since it is one of the mostly used backends. I would like to start a discussion about the design of the machinery needed to achieve this feature.

===========================

Objective

Add a feature to migrate VMs that have local storage and use the blkback iface.

===========================

User Interface

Add a cmd line option in “xl migrate” command to specify if local disks need to be copied to the destination node.

===========================

Design

As part of the libxl_domain_suspend, the “disk mirroring machinery” starts an asynchronous job that copies the disks blocks from source to the destination.
The protocol to copy the disks should resemble the one used for memory copy:

Do first initial copy of the disk.
Check of sectors that have been written since copy started. For this, the blkback driver should be aware that migration of disk is happening and in this case forward the write request to the “migration machinery” so that a record of dirty blocks are logged.
Migration machinery copies “dirty” blocks until convergence.
Duplicate all the disk writes/reads to both disks in source and destinations node while VM is being suspended.

Block Diagram

   +—------+
   |  VM   |
   +-------+
      |
      | I/O Write
      |
      V
+----------+       +-----------+                   +-------------+
|  blkback | ----> |  Source   |  sectors Stream   | Destination |
+----------+       |  mirror   |------------------>|   mirror    |
      |            | machinery |   I/O Writes      |  machinery  |
      |            +-----------+                   +-------------+
      |                                                  |
      |                                                  |
      | To I/O block layer                               |
      |                                                  |
      V                                                  V
+----------+                                       +-------------+
|   disk   |                                       |   Mirrored  |
+----------+                                       |     Disk    |
                                                   +-------------+

======================

Initial Questions

Is it possible to leverage the current design of QEMU for drive mirroring for Xen?
What is the best place to implement this protocol? As part of Xen or the kernel?
Is it possible to use the same stream currently used for migrating the memory to also migrate the disk blocks?

Any guidance/feedback for a more specific design is greatly appreciated.

Thanks,

Bruno

On Wed, Feb 22, 2017 at 5:00 AM, Wei Liu <wei.liu2@xxxxxxxxxx> wrote:

Hi Bruno

Thanks for your interest.

On Tue, Feb 21, 2017 at 10:34:45AM -0800, Bruno Alvisio wrote:
> Hello,
>
> I have been to doing some research and as far as I know XEN supports
> Live Migration
> of VMs that only have shared storage. (i.e. iSCSI) If the VM has been
> booted with local storage it cannot be live migrated.
> QEMU seems to support live migration with local storage (I have tested using
> 'virsh migrate with the '--storage-copy-all' option)
>
> I am wondering if this still true in the latest XEN release. Are there plans
> to add this functionality in future releases? I would be interested in
> contributing to the Xen Project by adding this functionality.
>

No plan at the moment.

Xen supports a wide variety of disk backends. QEMU is one of them. The
others are blktap (not upstreamed yet) and in-kernel blkback. The latter
two don't have the capability to copy local storage to the remote end.

That said, I think it would be valuable to have such capability for QEMU
backed disks. We also need to design the machinery so that other
backends can be made to do the same thing in the future.

If you want to undertake this project, I suggest you setup a Xen system,
read xl / libxl source code under tools directory and understand how
everything is put together. Reading source code could be daunting at
times, so don't hesitate to ask for pointers. After you have the big
picture in mind, we can then discuss how to implement the functionality
on xen-devel.

Does this sound good to you?

Wei.

> Thanks,
>
> Bruno

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> https://lists.xen.org/xen-devel

[Xen-devel] Fwd: VM Live Migration with Local Storage