Xen project Mailing List

[Xen-devel] [PATCH v4 3/4] libxl: call hotplug scripts from libxl for vbd

From: Roger Pau Monne <roger.pau@xxxxxxxxxx>

Date: Thu, 10 May 2012 12:28:31 +0100

Cc: Ian Jackson <ian.jackson@xxxxxxxxxxxxx>, Roger Pau Monne <roger.pau@xxxxxxxxxx>

Delivery-date: Thu, 10 May 2012 11:28:56 +0000

List-id: Xen developer discussion <xen-devel.lists.xen.org>

This is a rather big change, that adds the necessary machinery to perform hotplug script calling from libxl for Linux. This patch launches the necessary scripts to attach and detach PHY and TAP backend types (Qemu doesn't use hotplug scripts). Here's a list of the major changes introduced: * libxl_device_disk_add makes use of the new event library and takes the optional parameter libxl_asyncop_how, to choose how the operation has to be issued (sync or async). * Disk addition waits for backend to switch to state 2 (XenbusInitWait) and then launches the hotplug script. * Created and internal function (libxl__device_disk_add), that can be called within an already running AO, and that will plug the requested disk and execute the callback afterward. libxl_device_disk_add is a user of this newly created function. * libxl__devices_destroy no longer calls libxl__device_destroy, and instead calls libxl__initiate_device_remove, so we can disconnect the device and execute the necessary hotplug scripts instead of just deleting the backend entries. So libxl__devices_destroy now uses the event library, and so does libxl_domain_destroy, that has been converted to an AO operation function * The internal API for hotplug scripts has been added, which consists of one function; libxl__device_hotplug that takes the device and a struct with the necessary info about the action to execute. * Linux hotplug scripts are called by setting the necessary env variables to emulate udev rules, so there's no need to change them (although a rework to pass this as parameters insted of env variables would be suitable, so both NetBSD and Linux hotplug scripts take the same parameters). * Added a check in xen-hotplug-common.sh, so scripts are only executed from udev when using xend. xl adds a disable_udev=1 to xenstore libxl local directory and with the modification of the udev rules every call from udev gets UDEV_CALL passed, that points to disable_udev. If UDEV_CALL is passed and points to a value, the hotplug script is not executed. Changes since v3 (changes based on IanJ review): * Store disable_udev in libxl/disable_udev, which now is not device dependant, but backend domain dependant. * Added a global callback to device addition/destruction, that is now called when we have finished all the oprtations and handles the call to ao_complete if no other operations are pending. * Unified both device creation/destruction flows, so now all of them use the same data structure and callbacks. * Used the event interface in a proper way, and merged disk addition with the new async flow of the domain creation. * Make use of the new functions LOG, LOGE, GCNEW_ARRAY... * Passed NULL as ao_how in Python bindings. Changes since v1: * Replaced the hotplug snippet that prevents execution from udev when necessary, and now we set the env var from udev rules instead of libxl. * Replaced the libxl_device_disk_add "if" with a "switch". * Removed libxl__xs_path_cleanup (replaced with xs_rm). * Fixed several spelling mistakes. Cc: Ian Jackson <ian.jackson@xxxxxxxxxxxxx> Signed-off-by: Roger Pau Monne <roger.pau@xxxxxxxxxx> --- docs/man/xl.conf.pod.5 | 8 + tools/examples/xl.conf | 5 + tools/hotplug/Linux/xen-backend.rules | 6 +- tools/hotplug/Linux/xen-hotplug-common.sh | 6 + tools/libxl/Makefile | 3 +- tools/libxl/libxl.c | 351 +++++++-------------- tools/libxl/libxl.h | 7 +- tools/libxl/libxl_create.c | 102 ++++++- tools/libxl/libxl_device.c | 497 +++++++++++++++++++++++++---- tools/libxl/libxl_dm.c | 136 +++++---- tools/libxl/libxl_dom.c | 170 ++++++++++ tools/libxl/libxl_hotplug.c | 84 +++++ tools/libxl/libxl_internal.h | 170 ++++++++++- tools/libxl/libxl_linux.c | 126 ++++++++ tools/libxl/libxl_types.idl | 1 + tools/libxl/xl.c | 4 + tools/libxl/xl.h | 1 + tools/libxl/xl_cmdimpl.c | 15 +- tools/python/xen/lowlevel/xl/xl.c | 2 +- 19 files changed, 1291 insertions(+), 403 deletions(-) create mode 100644 tools/libxl/libxl_hotplug.c diff --git a/docs/man/xl.conf.pod.5 b/docs/man/xl.conf.pod.5 index 8bd45ea..72825a0 100644 --- a/docs/man/xl.conf.pod.5 +++ b/docs/man/xl.conf.pod.5 @@ -55,6 +55,14 @@ default. Default: C<1> +=item B<run_hotplug_scripts=BOOLEAN> + +If disabled hotplug scripts will be called from udev, as it used to +be in the previous releases. With the default option, hotplug scripts +will be launched by xl directly. + +Default: C<1> + =item B<lockfile="PATH"> Sets the path to the lock file used by xl to serialise certain diff --git a/tools/examples/xl.conf b/tools/examples/xl.conf index 56d3b3b..75b00e0 100644 --- a/tools/examples/xl.conf +++ b/tools/examples/xl.conf @@ -12,3 +12,8 @@ # default output format used by "xl list -l" #output_format="json" + +# default option to run hotplug scripts from xl +# if disabled the old behaviour will be used, and hotplug scripts will be +# launched by udev. +#run_hotplug_scripts=1 diff --git a/tools/hotplug/Linux/xen-backend.rules b/tools/hotplug/Linux/xen-backend.rules index 405387f..d55ff11 100644 --- a/tools/hotplug/Linux/xen-backend.rules +++ b/tools/hotplug/Linux/xen-backend.rules @@ -1,11 +1,11 @@ -SUBSYSTEM=="xen-backend", KERNEL=="tap*", RUN+="/etc/xen/scripts/blktap $env{ACTION}" -SUBSYSTEM=="xen-backend", KERNEL=="vbd*", RUN+="/etc/xen/scripts/block $env{ACTION}" +SUBSYSTEM=="xen-backend", KERNEL=="tap*", ENV{UDEV_CALL}="1", RUN+="/etc/xen/scripts/blktap $env{ACTION}" +SUBSYSTEM=="xen-backend", KERNEL=="vbd*", ENV{UDEV_CALL}="1", RUN+="/etc/xen/scripts/block $env{ACTION}" SUBSYSTEM=="xen-backend", KERNEL=="vtpm*", RUN+="/etc/xen/scripts/vtpm $env{ACTION}" SUBSYSTEM=="xen-backend", KERNEL=="vif2-*", RUN+="/etc/xen/scripts/vif2 $env{ACTION}" SUBSYSTEM=="xen-backend", KERNEL=="vif-*", ACTION=="online", RUN+="/etc/xen/scripts/vif-setup online type_if=vif" SUBSYSTEM=="xen-backend", KERNEL=="vif-*", ACTION=="offline", RUN+="/etc/xen/scripts/vif-setup offline type_if=vif" SUBSYSTEM=="xen-backend", KERNEL=="vscsi*", RUN+="/etc/xen/scripts/vscsi $env{ACTION}" -SUBSYSTEM=="xen-backend", ACTION=="remove", RUN+="/etc/xen/scripts/xen-hotplug-cleanup" +SUBSYSTEM=="xen-backend", ACTION=="remove", ENV{UDEV_CALL}="1", RUN+="/etc/xen/scripts/xen-hotplug-cleanup" KERNEL=="evtchn", NAME="xen/%k" SUBSYSTEM=="xen", KERNEL=="blktap[0-9]*", NAME="xen/%k", MODE="0600" SUBSYSTEM=="blktap2", KERNEL=="blktap[0-9]*", NAME="xen/blktap-2/%k", MODE="0600" diff --git a/tools/hotplug/Linux/xen-hotplug-common.sh b/tools/hotplug/Linux/xen-hotplug-common.sh index 8f6557d..6443a63 100644 --- a/tools/hotplug/Linux/xen-hotplug-common.sh +++ b/tools/hotplug/Linux/xen-hotplug-common.sh @@ -15,6 +15,12 @@ # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # +# Hack to prevent the execution of hotplug scripts from udev if the domain +# has been launched from libxl +if [ -n "${UDEV_CALL}" ] && \ + `xenstore-read "libxl/disable_udev" >/dev/null 2>&1`; then + exit 0 +fi dir=$(dirname "$0") . "$dir/hotplugpath.sh" diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile index 5d9227e..9abadff 100644 --- a/tools/libxl/Makefile +++ b/tools/libxl/Makefile @@ -66,7 +66,8 @@ LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o libxl_pci.o \ libxl_dom.o libxl_exec.o libxl_xshelp.o libxl_device.o \ libxl_internal.o libxl_utils.o libxl_uuid.o \ libxl_json.o libxl_aoutils.o \ - libxl_qmp.o libxl_event.o libxl_fork.o $(LIBXL_OBJS-y) + libxl_qmp.o libxl_event.o libxl_fork.o libxl_hotplug.o \ + $(LIBXL_OBJS-y) LIBXL_OBJS += _libxl_types.o libxl_flask.o _libxl_types_internal.o $(LIBXL_OBJS): CFLAGS += $(CFLAGS_libxenctrl) $(CFLAGS_libxenguest) $(CFLAGS_libxenstore) $(CFLAGS_libblktapctl) -include $(XEN_ROOT)/tools/config.h diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 79d7708..b438fbf 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -1058,82 +1058,36 @@ void libxl_evdisable_disk_eject(libxl_ctx *ctx, libxl_evgen_disk_eject *evg) { GC_INIT(ctx); libxl__evdisable_disk_eject(gc, evg); GC_FREE; -} +} -int libxl_domain_destroy(libxl_ctx *ctx, uint32_t domid) -{ - GC_INIT(ctx); - char *dom_path; - char *vm_path; - char *pid; - int rc, dm_present; +/* Callback for domain destruction */ - rc = libxl_domain_info(ctx, NULL, domid); - switch(rc) { - case 0: - break; - case ERROR_INVAL: - LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "non-existant domain %d", domid); - default: - return rc; - } +static void domain_destroy_cb(libxl__egc *, libxl__domain_destroy_state *, int); - switch (libxl__domain_type(gc, domid)) { - case LIBXL_DOMAIN_TYPE_HVM: - dm_present = 1; - break; - case LIBXL_DOMAIN_TYPE_PV: - pid = libxl__xs_read(gc, XBT_NULL, libxl__sprintf(gc, "/local/domain/%d/image/device-model-pid", domid)); - dm_present = (pid != NULL); - break; - default: - abort(); - } - - dom_path = libxl__xs_get_dompath(gc, domid); - if (!dom_path) { - rc = ERROR_FAIL; - goto out; - } - - if (libxl__device_pci_destroy_all(gc, domid) < 0) - LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "pci shutdown failed for domid %d", domid); - rc = xc_domain_pause(ctx->xch, domid); - if (rc < 0) { - LIBXL__LOG_ERRNOVAL(ctx, LIBXL__LOG_ERROR, rc, "xc_domain_pause failed for %d", domid); - } - if (dm_present) { - if (libxl__destroy_device_model(gc, domid) < 0) - LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "libxl__destroy_device_model failed for %d", domid); - - libxl__qmp_cleanup(gc, domid); - } - if (libxl__devices_destroy(gc, domid) < 0) - LIBXL__LOG(ctx, LIBXL__LOG_ERROR, - "libxl__devices_destroy failed for %d", domid); +int libxl_domain_destroy(libxl_ctx *ctx, uint32_t domid, + const libxl_asyncop_how *ao_how) +{ + AO_CREATE(ctx, domid, ao_how); + libxl__domain_destroy_state *dds; - vm_path = libxl__xs_read(gc, XBT_NULL, libxl__sprintf(gc, "%s/vm", dom_path)); - if (vm_path) - if (!xs_rm(ctx->xsh, XBT_NULL, vm_path)) - LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "xs_rm failed for %s", vm_path); + GCNEW(dds); + dds->ao = ao; + dds->domid = domid; + dds->callback = domain_destroy_cb; + libxl__domain_destroy(egc, dds); - if (!xs_rm(ctx->xsh, XBT_NULL, dom_path)) - LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "xs_rm failed for %s", dom_path); + return AO_INPROGRESS; +} - xs_rm(ctx->xsh, XBT_NULL, libxl__xs_libxl_path(gc, domid)); +static void domain_destroy_cb(libxl__egc *egc, libxl__domain_destroy_state *dds, + int rc) +{ + STATE_AO_GC(dds->ao); - libxl__userdata_destroyall(gc, domid); + if (rc) + LOGE(ERROR, "destruction of domain %u failed", dds->domid); - rc = xc_domain_destroy(ctx->xch, domid); - if (rc < 0) { - LIBXL__LOG_ERRNOVAL(ctx, LIBXL__LOG_ERROR, rc, "xc_domain_destroy failed for %d", domid); - rc = ERROR_FAIL; - goto out; - } - rc = 0; -out: - GC_FREE; - return rc; + libxl__ao_complete(egc, ao, rc); } int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num, libxl_console_type type) @@ -1261,171 +1215,56 @@ int libxl_vncviewer_exec(libxl_ctx *ctx, uint32_t domid, int autopass) /******************************************************************************/ -int libxl__device_disk_setdefault(libxl__gc *gc, libxl_device_disk *disk) +/* generic callback for devices that only need to set ao_complete */ +static void libxl__device_cb(libxl__egc *egc, libxl__ao_device *aorm) { - int rc; + STATE_AO_GC(aorm->ao); - rc = libxl__device_disk_set_backend(gc, disk); - if (rc) return rc; + if (aorm->rc) { + LOGE(ERROR, "unable to %s %s with id %u", + aorm->action == DEVICE_CONNECT ? "add" : "remove", + libxl__device_kind_to_string(aorm->dev->kind), + aorm->dev->devid); + goto out; + } - return rc; +out: + libxl__ao_complete(egc, ao, aorm->rc); + return; } -static int libxl__device_from_disk(libxl__gc *gc, uint32_t domid, - libxl_device_disk *disk, - libxl__device *device) -{ - libxl_ctx *ctx = libxl__gc_owner(gc); - int devid; - - devid = libxl__device_disk_dev_number(disk->vdev, NULL, NULL); - if (devid==-1) { - LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "Invalid or unsupported" - " virtual disk identifier %s", disk->vdev); - return ERROR_INVAL; - } - - device->backend_domid = disk->backend_domid; - device->backend_devid = devid; +/******************************************************************************/ - switch (disk->backend) { - case LIBXL_DISK_BACKEND_PHY: - device->backend_kind = LIBXL__DEVICE_KIND_VBD; - break; - case LIBXL_DISK_BACKEND_TAP: - device->backend_kind = LIBXL__DEVICE_KIND_VBD; - break; - case LIBXL_DISK_BACKEND_QDISK: - device->backend_kind = LIBXL__DEVICE_KIND_QDISK; - break; - default: - LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "unrecognized disk backend type: %d\n", - disk->backend); - return ERROR_INVAL; - } +int libxl__device_disk_setdefault(libxl__gc *gc, libxl_device_disk *disk) +{ + int rc; - device->domid = domid; - device->devid = devid; - device->kind = LIBXL__DEVICE_KIND_VBD; + rc = libxl__device_disk_set_backend(gc, disk); + if (rc) return rc; - return 0; + return rc; } -int libxl_device_disk_add(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk) +int libxl_device_disk_add(libxl_ctx *ctx, uint32_t domid, + libxl_device_disk *disk, + const libxl_asyncop_how *ao_how) { - GC_INIT(ctx); - flexarray_t *front; - flexarray_t *back; - char *dev; - libxl__device device; - int major, minor, rc; - - rc = libxl__device_disk_setdefault(gc, disk); - if (rc) goto out; + AO_CREATE(ctx, domid, ao_how); + libxl__ao_device *device; + int rc; - front = flexarray_make(16, 1); - if (!front) { - rc = ERROR_NOMEM; + GCNEW(device); + libxl__init_ao_device(device, ao, NULL); + device->callback = libxl__device_cb; + rc = libxl__device_disk_add(egc, domid, disk, device); + if (rc) { + LOGE(ERROR, "unable to add disk %s", disk->pdev_path); goto out; } - back = flexarray_make(16, 1); - if (!back) { - rc = ERROR_NOMEM; - goto out_free; - } - - if (disk->script) { - LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "External block scripts" - " not yet supported, sorry"); - rc = ERROR_INVAL; - goto out_free; - } - - rc = libxl__device_from_disk(gc, domid, disk, &device); - if (rc != 0) { - LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "Invalid or unsupported" - " virtual disk identifier %s", disk->vdev); - goto out_free; - } - - switch (disk->backend) { - case LIBXL_DISK_BACKEND_PHY: - dev = disk->pdev_path; - do_backend_phy: - libxl__device_physdisk_major_minor(dev, &major, &minor); - flexarray_append(back, "physical-device"); - flexarray_append(back, libxl__sprintf(gc, "%x:%x", major, minor)); - - flexarray_append(back, "params"); - flexarray_append(back, dev); - - assert(device.backend_kind == LIBXL__DEVICE_KIND_VBD); - break; - case LIBXL_DISK_BACKEND_TAP: - dev = libxl__blktap_devpath(gc, disk->pdev_path, disk->format); - if (!dev) { - rc = ERROR_FAIL; - goto out_free; - } - flexarray_append(back, "tapdisk-params"); - flexarray_append(back, libxl__sprintf(gc, "%s:%s", - libxl__device_disk_string_of_format(disk->format), - disk->pdev_path)); - - /* now create a phy device to export the device to the guest */ - goto do_backend_phy; - case LIBXL_DISK_BACKEND_QDISK: - flexarray_append(back, "params"); - flexarray_append(back, libxl__sprintf(gc, "%s:%s", - libxl__device_disk_string_of_format(disk->format), disk->pdev_path)); - assert(device.backend_kind == LIBXL__DEVICE_KIND_QDISK); - break; - default: - LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "unrecognized disk backend type: %d\n", disk->backend); - rc = ERROR_INVAL; - goto out_free; - } - - flexarray_append(back, "frontend-id"); - flexarray_append(back, libxl__sprintf(gc, "%d", domid)); - flexarray_append(back, "online"); - flexarray_append(back, "1"); - flexarray_append(back, "removable"); - flexarray_append(back, libxl__sprintf(gc, "%d", (disk->removable) ? 1 : 0)); - flexarray_append(back, "bootable"); - flexarray_append(back, libxl__sprintf(gc, "%d", 1)); - flexarray_append(back, "state"); - flexarray_append(back, libxl__sprintf(gc, "%d", 1)); - flexarray_append(back, "dev"); - flexarray_append(back, disk->vdev); - flexarray_append(back, "type"); - flexarray_append(back, libxl__device_disk_string_of_backend(disk->backend)); - flexarray_append(back, "mode"); - flexarray_append(back, disk->readwrite ? "w" : "r"); - flexarray_append(back, "device-type"); - flexarray_append(back, disk->is_cdrom ? "cdrom" : "disk"); - - flexarray_append(front, "backend-id"); - flexarray_append(front, libxl__sprintf(gc, "%d", disk->backend_domid)); - flexarray_append(front, "state"); - flexarray_append(front, libxl__sprintf(gc, "%d", 1)); - flexarray_append(front, "virtual-device"); - flexarray_append(front, libxl__sprintf(gc, "%d", device.devid)); - flexarray_append(front, "device-type"); - flexarray_append(front, disk->is_cdrom ? "cdrom" : "disk"); - - libxl__device_generic_add(gc, &device, - libxl__xs_kvs_of_flexarray(gc, back, back->count), - libxl__xs_kvs_of_flexarray(gc, front, front->count)); - rc = 0; - -out_free: - flexarray_free(back); - flexarray_free(front); out: - GC_FREE; - return rc; + if (rc) return AO_ABORT(rc); + return AO_INPROGRESS; } int libxl_device_disk_remove(libxl_ctx *ctx, uint32_t domid, @@ -1433,19 +1272,25 @@ int libxl_device_disk_remove(libxl_ctx *ctx, uint32_t domid, const libxl_asyncop_how *ao_how) { AO_CREATE(ctx, domid, ao_how); - libxl__device device; + libxl__device *device; + libxl__ao_device *aorm = 0; int rc; - rc = libxl__device_from_disk(gc, domid, disk, &device); + GCNEW(device); + rc = libxl__device_from_disk(gc, domid, disk, device); if (rc != 0) goto out; - rc = libxl__initiate_device_remove(egc, ao, &device); - if (rc) goto out; + GCNEW(aorm); + libxl__init_ao_device(aorm, ao, NULL); + aorm->action = DEVICE_DISCONNECT; + aorm->dev = device; + aorm->callback = libxl__device_cb; - return AO_INPROGRESS; + libxl__initiate_device_remove(egc, aorm); out: - return AO_ABORT(rc); + if (rc) return AO_ABORT(rc); + return AO_INPROGRESS; } int libxl_device_disk_destroy(libxl_ctx *ctx, uint32_t domid, @@ -1663,12 +1508,14 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk) ret = 0; + /* fixme-ao */ libxl_device_disk_remove(ctx, domid, disks + i, 0); - libxl_device_disk_add(ctx, domid, disk); + libxl_device_disk_add(ctx, domid, disk, 0); stubdomid = libxl_get_stubdom_id(ctx, domid); if (stubdomid) { + /* fixme-ao */ libxl_device_disk_remove(ctx, stubdomid, disks + i, 0); - libxl_device_disk_add(ctx, stubdomid, disk); + libxl_device_disk_add(ctx, stubdomid, disk, 0); } out: for (i = 0; i < num; i++) @@ -1911,19 +1758,25 @@ int libxl_device_nic_remove(libxl_ctx *ctx, uint32_t domid, const libxl_asyncop_how *ao_how) { AO_CREATE(ctx, domid, ao_how); - libxl__device device; + libxl__device *device; + libxl__ao_device *aorm = 0; int rc; - rc = libxl__device_from_nic(gc, domid, nic, &device); + GCNEW(device); + rc = libxl__device_from_nic(gc, domid, nic, device); if (rc != 0) goto out; - rc = libxl__initiate_device_remove(egc, ao, &device); - if (rc) goto out; + GCNEW(aorm); + libxl__init_ao_device(aorm, ao, NULL); + aorm->action = DEVICE_DISCONNECT; + aorm->dev = device; + aorm->callback = libxl__device_cb; - return AO_INPROGRESS; + libxl__initiate_device_remove(egc, aorm); out: - return AO_ABORT(rc); + if (rc) return AO_ABORT(rc); + return AO_INPROGRESS; } int libxl_device_nic_destroy(libxl_ctx *ctx, uint32_t domid, @@ -2273,19 +2126,25 @@ int libxl_device_vkb_remove(libxl_ctx *ctx, uint32_t domid, const libxl_asyncop_how *ao_how) { AO_CREATE(ctx, domid, ao_how); - libxl__device device; + libxl__device *device; + libxl__ao_device *aorm = 0; int rc; - rc = libxl__device_from_vkb(gc, domid, vkb, &device); + GCNEW(device); + rc = libxl__device_from_vkb(gc, domid, vkb, device); if (rc != 0) goto out; - rc = libxl__initiate_device_remove(egc, ao, &device); - if (rc) goto out; + GCNEW(aorm); + libxl__init_ao_device(aorm, ao, NULL); + aorm->action = DEVICE_DISCONNECT; + aorm->dev = device; + aorm->callback = libxl__device_cb; - return AO_INPROGRESS; + libxl__initiate_device_remove(egc, aorm); out: - return AO_ABORT(rc); + if (rc) return AO_ABORT(rc); + return AO_INPROGRESS; } int libxl_device_vkb_destroy(libxl_ctx *ctx, uint32_t domid, @@ -2406,19 +2265,25 @@ int libxl_device_vfb_remove(libxl_ctx *ctx, uint32_t domid, const libxl_asyncop_how *ao_how) { AO_CREATE(ctx, domid, ao_how); - libxl__device device; + libxl__device *device; + libxl__ao_device *aorm = 0; int rc; - rc = libxl__device_from_vfb(gc, domid, vfb, &device); + GCNEW(device); + rc = libxl__device_from_vfb(gc, domid, vfb, device); if (rc != 0) goto out; - rc = libxl__initiate_device_remove(egc, ao, &device); - if (rc) goto out; + GCNEW(aorm); + libxl__init_ao_device(aorm, ao, NULL); + aorm->action = DEVICE_DISCONNECT; + aorm->dev = device; + aorm->callback = libxl__device_cb; - return AO_INPROGRESS; + libxl__initiate_device_remove(egc, aorm); out: - return AO_ABORT(rc); + if (rc) return AO_ABORT(rc); + return AO_INPROGRESS; } int libxl_device_vfb_destroy(libxl_ctx *ctx, uint32_t domid, diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index 0ff4a83..0ab1531 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -530,7 +530,8 @@ int libxl_domain_suspend(libxl_ctx *ctx, libxl_domain_suspend_info *info, int libxl_domain_resume(libxl_ctx *ctx, uint32_t domid); int libxl_domain_shutdown(libxl_ctx *ctx, uint32_t domid); int libxl_domain_reboot(libxl_ctx *ctx, uint32_t domid); -int libxl_domain_destroy(libxl_ctx *ctx, uint32_t domid); +int libxl_domain_destroy(libxl_ctx *ctx, uint32_t domid, + const libxl_asyncop_how *ao_how); int libxl_domain_preserve(libxl_ctx *ctx, uint32_t domid, libxl_domain_create_info *info, const char *name_suffix, libxl_uuid new_uuid); /* get max. number of cpus supported by hypervisor */ @@ -660,7 +661,9 @@ void libxl_vminfo_list_free(libxl_vminfo *list, int nr); */ /* Disks */ -int libxl_device_disk_add(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk); +int libxl_device_disk_add(libxl_ctx *ctx, uint32_t domid, + libxl_device_disk *disk, + const libxl_asyncop_how *ao_how); int libxl_device_disk_remove(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk, const libxl_asyncop_how *ao_how); diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c index 788e553..db58fd0 100644 --- a/tools/libxl/libxl_create.c +++ b/tools/libxl/libxl_create.c @@ -400,7 +400,7 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_create_info *info, * on exit (even error exit), domid may be valid and refer to a domain */ { libxl_ctx *ctx = libxl__gc_owner(gc); - int flags, ret, rc; + int flags, ret, rc, nb_vm; char *uuid_string; char *dom_path, *vm_path, *libxl_path; struct xs_permissions roperm[2]; @@ -521,6 +521,28 @@ retry_transaction: libxl__sprintf(gc, "%s/hvmloader/generation-id-address", dom_path), rwperm, ARRAY_SIZE(rwperm)); + if (libxl_list_vm(ctx, &nb_vm) < 0) { + LOGE(ERROR, "cannot get number of running guests"); + rc = ERROR_FAIL; + goto out; + } + if (libxl_defbool_val(info->run_hotplug_scripts)) { + if (!libxl__xs_read(gc, t, DISABLE_UDEV_PATH) && (nb_vm - 1)) { + LOGE(ERROR, "cannot change hotplug execution option once set, " + "please shutdown all guests before changing it"); + rc = ERROR_FAIL; + goto out; + } + libxl__xs_write(gc, t, DISABLE_UDEV_PATH, "1"); + } else { + if (libxl__xs_read(gc, t, DISABLE_UDEV_PATH) && (nb_vm - 1)) { + LOGE(ERROR, "cannot change hotplug execution option once set, " + "please shutdown all guests before changing it"); + rc = ERROR_FAIL; + goto out; + } + xs_rm(ctx->xsh, t, DISABLE_UDEV_PATH); + } xs_write(ctx->xsh, t, libxl__sprintf(gc, "%s/uuid", vm_path), uuid_string, strlen(uuid_string)); xs_write(ctx->xsh, t, libxl__sprintf(gc, "%s/name", vm_path), info->name, strlen(info->name)); @@ -577,15 +599,25 @@ static void domcreate_bootloader_done(libxl__egc *egc, libxl__bootloader_state *bl, int rc); +static void domcreate_disk_connected(libxl__egc *egc, + libxl__ao_device *aorm); + static void domcreate_console_available(libxl__egc *egc, libxl__domain_create_state *dcs); /* Our own function to clean up and call the user's callback. - * The final call in the sequence. */ + * The final call in the sequence if domain creation is successful. */ static void domcreate_complete(libxl__egc *egc, libxl__domain_create_state *dcs, int rc); +/* If creation is not successful, this callback will be executed + * when domain destruction is finished */ + +static void domcreate_destruction_cb(libxl__egc *egc, + libxl__domain_destroy_state *dds, + int rc); + static void initiate_domain_create(libxl__egc *egc, libxl__domain_create_state *dcs) { @@ -700,8 +732,13 @@ static void domcreate_bootloader_done(libxl__egc *egc, store_libxl_entry(gc, domid, &d_config->b_info); + GCNEW_ARRAY(dcs->devices, d_config->num_disks); + dcs->num_devices = d_config->num_disks; for (i = 0; i < d_config->num_disks; i++) { - ret = libxl_device_disk_add(ctx, domid, &d_config->disks[i]); + libxl__init_ao_device(&dcs->devices[i], ao, &dcs->devices); + dcs->devices[i].callback = domcreate_disk_connected; + ret = libxl__device_disk_add(egc, domid, &d_config->disks[i], + &dcs->devices[i]); if (ret) { LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "cannot add disk %d to domain: %d", i, ret); @@ -718,6 +755,42 @@ static void domcreate_bootloader_done(libxl__egc *egc, goto error_out; } } + return; + + error_out: + assert(ret); + domcreate_complete(egc, dcs, ret); +} + +static void domcreate_disk_connected(libxl__egc *egc, libxl__ao_device *aorm) +{ + STATE_AO_GC(aorm->ao); + libxl__domain_create_state *dcs = CONTAINER_OF(aorm->base, *dcs, devices); + int i, last, ret = 0; + + /* convenience aliases */ + const uint32_t domid = dcs->guest_domid; + libxl_domain_config *const d_config = dcs->guest_config; + libxl__domain_build_state *const state = &dcs->build_state; + libxl_ctx *const ctx = CTX; + + ret = libxl__ao_device_check_last(gc, aorm, dcs->devices, + dcs->num_devices, &last); + if (!last) return; + if (last && ret) { + LOGE(ERROR, "error connecting disk devices"); + goto error_out; + } + + /* We might be going to call libxl__spawn_local_dm, or _spawn_stub_dm. + * Fill in any field required by either, including both relevant + * callbacks (_spawn_stub_dm will overwrite our trespass if needed). */ + dcs->dmss.dm.spawn.ao = ao; + dcs->dmss.dm.guest_config = dcs->guest_config; + dcs->dmss.dm.build_state = &dcs->build_state; + dcs->dmss.dm.callback = domcreate_devmodel_started; + dcs->dmss.callback = domcreate_devmodel_started; + switch (d_config->c_info.type) { case LIBXL_DOMAIN_TYPE_HVM: { @@ -853,16 +926,31 @@ static void domcreate_complete(libxl__egc *egc, if (rc) { if (dcs->guest_domid) { - int rc2 = libxl_domain_destroy(CTX, dcs->guest_domid); - if (rc2) - LOG(ERROR, "unable to destroy domain %d following" - " failed creation", dcs->guest_domid); + dcs->dds.ao = ao; + dcs->dds.domid = dcs->guest_domid; + dcs->dds.callback = domcreate_destruction_cb; + libxl__domain_destroy(egc, &dcs->dds); + return; } dcs->guest_domid = -1; } dcs->callback(egc, dcs, rc, dcs->guest_domid); } +static void domcreate_destruction_cb(libxl__egc *egc, + libxl__domain_destroy_state *dds, + int rc) +{ + STATE_AO_GC(dds->ao); + libxl__domain_create_state *dcs = CONTAINER_OF(dds, *dcs, dds); + + if (rc) + LOG(ERROR, "unable to destroy domain %u following failed creation", + dds->domid); + + dcs->callback(egc, dcs, rc, dcs->guest_domid); +} + /*----- application-facing domain creation interface -----*/ typedef struct { diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c index c7e057d..275ab43 100644 --- a/tools/libxl/libxl_device.c +++ b/tools/libxl/libxl_device.c @@ -58,6 +58,48 @@ int libxl__parse_backend_path(libxl__gc *gc, return libxl__device_kind_from_string(strkind, &dev->backend_kind); } +static int libxl__num_devices(libxl__gc *gc, uint32_t domid) +{ + char *path; + unsigned int num_kinds, num_devs; + char **kinds = NULL, **devs = NULL; + int i, j, rc = 0; + libxl__device dev; + libxl__device_kind kind; + int numdevs = 0; + + path = GCSPRINTF("/local/domain/%d/device", domid); + kinds = libxl__xs_directory(gc, XBT_NULL, path, &num_kinds); + if (!kinds) { + if (errno != ENOENT) { + LOGE(ERROR, "unable to get xenstore device listing %s", path); + rc = ERROR_FAIL; + goto out; + } + num_kinds = 0; + } + for (i = 0; i < num_kinds; i++) { + if (libxl__device_kind_from_string(kinds[i], &kind)) + continue; + + path = GCSPRINTF("/local/domain/%d/device/%s", domid, kinds[i]); + devs = libxl__xs_directory(gc, XBT_NULL, path, &num_devs); + if (!devs) + continue; + for (j = 0; j < num_devs; j++) { + path = GCSPRINTF("/local/domain/%d/device/%s/%s/backend", + domid, kinds[i], devs[j]); + path = libxl__xs_read(gc, XBT_NULL, path); + if (path && libxl__parse_backend_path(gc, path, &dev) == 0) { + numdevs++; + } + } + } +out: + if (rc) return rc; + return numdevs; +} + int libxl__device_generic_add(libxl__gc *gc, libxl__device *device, char **bents, char **fents) { @@ -110,6 +152,178 @@ retry_transaction: return 0; } +int libxl__device_from_disk(libxl__gc *gc, uint32_t domid, + libxl_device_disk *disk, + libxl__device *device) +{ + libxl_ctx *ctx = libxl__gc_owner(gc); + int devid; + + devid = libxl__device_disk_dev_number(disk->vdev, NULL, NULL); + if (devid == -1) { + LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "Invalid or unsupported" + " virtual disk identifier %s", disk->vdev); + return ERROR_INVAL; + } + + device->backend_domid = disk->backend_domid; + device->backend_devid = devid; + + switch (disk->backend) { + case LIBXL_DISK_BACKEND_PHY: + device->backend_kind = LIBXL__DEVICE_KIND_VBD; + break; + case LIBXL_DISK_BACKEND_TAP: + device->backend_kind = LIBXL__DEVICE_KIND_VBD; + break; + case LIBXL_DISK_BACKEND_QDISK: + device->backend_kind = LIBXL__DEVICE_KIND_QDISK; + break; + default: + LOGE(ERROR, "unrecognized disk backend type: %d", disk->backend); + return ERROR_INVAL; + } + + device->domid = domid; + device->devid = devid; + device->kind = LIBXL__DEVICE_KIND_VBD; + + return 0; +} + +int libxl__device_disk_add(libxl__egc *egc, uint32_t domid, + libxl_device_disk *disk, + libxl__ao_device *aorm) +{ + STATE_AO_GC(aorm->ao); + flexarray_t *front = NULL; + flexarray_t *back = NULL; + char *dev; + libxl__device *device; + int major, minor, rc; + + rc = libxl__device_disk_setdefault(gc, disk); + if (rc) goto out; + + front = flexarray_make(16, 1); + if (!front) { + rc = ERROR_NOMEM; + goto out; + } + back = flexarray_make(20, 1); + if (!back) { + rc = ERROR_NOMEM; + goto out_free; + } + + if (disk->script) { + LOGE(ERROR, "External block scripts not yet supported, sorry"); + rc = ERROR_INVAL; + goto out_free; + } + + GCNEW(device); + rc = libxl__device_from_disk(gc, domid, disk, device); + if (rc != 0) { + LOGE(ERROR, "Invalid or unsupported virtual disk identifier %s", + disk->vdev); + goto out; + } + + switch (disk->backend) { + case LIBXL_DISK_BACKEND_PHY: + dev = disk->pdev_path; + do_backend_phy: + libxl__device_physdisk_major_minor(dev, &major, &minor); + flexarray_append(back, "physical-device"); + flexarray_append(back, GCSPRINTF("%x:%x", major, minor)); + + flexarray_append(back, "params"); + flexarray_append(back, dev); + + flexarray_append(back, "script"); + flexarray_append(back, GCSPRINTF("%s/%s", + libxl__xen_script_dir_path(), + "block")); + + assert(device->backend_kind == LIBXL__DEVICE_KIND_VBD); + break; + case LIBXL_DISK_BACKEND_TAP: + dev = libxl__blktap_devpath(gc, disk->pdev_path, disk->format); + if (!dev) { + rc = ERROR_FAIL; + goto out_free; + } + flexarray_append(back, "tapdisk-params"); + flexarray_append(back, GCSPRINTF("%s:%s", + libxl__device_disk_string_of_format(disk->format), + disk->pdev_path)); + + flexarray_append(back, "script"); + flexarray_append(back, GCSPRINTF("%s/%s", + libxl__xen_script_dir_path(), + "blktap")); + + /* now create a phy device to export the device to the guest */ + goto do_backend_phy; + case LIBXL_DISK_BACKEND_QDISK: + flexarray_append(back, "params"); + flexarray_append(back, GCSPRINTF("%s:%s", + libxl__device_disk_string_of_format(disk->format), + disk->pdev_path)); + assert(device->backend_kind == LIBXL__DEVICE_KIND_QDISK); + break; + default: + LOGE(ERROR, "unrecognized disk backend type: %d", disk->backend); + rc = ERROR_INVAL; + goto out_free; + } + + flexarray_append(back, "frontend-id"); + flexarray_append(back, GCSPRINTF("%d", domid)); + flexarray_append(back, "online"); + flexarray_append(back, "1"); + flexarray_append(back, "removable"); + flexarray_append(back, GCSPRINTF("%d", (disk->removable) ? 1 : 0)); + flexarray_append(back, "bootable"); + flexarray_append(back, GCSPRINTF("%d", 1)); + flexarray_append(back, "state"); + flexarray_append(back, GCSPRINTF("%d", 1)); + flexarray_append(back, "dev"); + flexarray_append(back, disk->vdev); + flexarray_append(back, "type"); + flexarray_append(back, libxl__device_disk_string_of_backend(disk->backend)); + flexarray_append(back, "mode"); + flexarray_append(back, disk->readwrite ? "w" : "r"); + flexarray_append(back, "device-type"); + flexarray_append(back, disk->is_cdrom ? "cdrom" : "disk"); + + flexarray_append(front, "backend-id"); + flexarray_append(front, GCSPRINTF("%d", disk->backend_domid)); + flexarray_append(front, "state"); + flexarray_append(front, GCSPRINTF("%d", 1)); + flexarray_append(front, "virtual-device"); + flexarray_append(front, GCSPRINTF("%d", device->devid)); + flexarray_append(front, "device-type"); + flexarray_append(front, disk->is_cdrom ? "cdrom" : "disk"); + + libxl__device_generic_add(gc, device, + libxl__xs_kvs_of_flexarray(gc, back, back->count), + libxl__xs_kvs_of_flexarray(gc, front, front->count)); + + aorm->dev = device; + aorm->action = DEVICE_CONNECT; + libxl__initiate_device_add(egc, aorm); + + rc = 0; + +out_free: + flexarray_free(back); + flexarray_free(front); +out: + return rc; +} + typedef struct { libxl__gc *gc; libxl_device_disk *disk; @@ -356,49 +570,116 @@ int libxl__device_disk_dev_number(const char *virtpath, int *pdisk, return -1; } - -typedef struct { - libxl__ao *ao; - libxl__ev_devstate ds; -} libxl__ao_device_remove; - -static void device_remove_cleanup(libxl__gc *gc, - libxl__ao_device_remove *aorm) { +static void device_backend_cleanup(libxl__gc *gc, libxl__ao_device *aorm) +{ if (!aorm) return; libxl__ev_devstate_cancel(gc, &aorm->ds); } -static void device_remove_callback(libxl__egc *egc, libxl__ev_devstate *ds, - int rc) { - libxl__ao_device_remove *aorm = CONTAINER_OF(ds, *aorm, ds); - libxl__gc *gc = &aorm->ao->gc; - libxl__ao_complete(egc, aorm->ao, rc); - device_remove_cleanup(gc, aorm); +void libxl__init_ao_device(libxl__ao_device *aorm, libxl__ao *ao, + libxl__ao_device **base) +{ + aorm->ao = ao; + aorm->state = DEVICE_ACTIVE; + aorm->rc = 0; + aorm->base = base; } -int libxl__initiate_device_remove(libxl__egc *egc, libxl__ao *ao, - libxl__device *dev) +int libxl__ao_device_check_last(libxl__gc *gc, libxl__ao_device *device, + libxl__ao_device *list, int num, int *last) { - AO_GC; - libxl_ctx *ctx = libxl__gc_owner(gc); - xs_transaction_t t; - char *be_path = libxl__device_backend_path(gc, dev); + int i, ret = 0; + + device->state = DEVICE_FINISHED; + *last = 1; + for (i = 0; i < num; i++) { + if (list[i].state != DEVICE_FINISHED && *last) { + *last = 0; + if (!device->rc) break; + } + if (device->rc && device->action == DEVICE_CONNECT) { + /* Cancel pending events */ + if (list[i].pid) { + libxl__ev_time_deregister(gc, &list[i].ev); + kill(list[i].pid, SIGKILL); + } else { + libxl__ev_devstate_cancel(gc, &list[i].ds); + } + ret = list[i].rc; + } + } + return ret; +} + +/* Common callbacks for device add/remove */ + +static void device_backend_callback(libxl__egc *egc, libxl__ev_devstate *ds, + int rc); + +void libxl__initiate_device_add(libxl__egc *egc, libxl__ao_device *aorm) +{ + STATE_AO_GC(aorm->ao); + char *be_path = libxl__device_backend_path(gc, aorm->dev); char *state_path = libxl__sprintf(gc, "%s/state", be_path); char *state = libxl__xs_read(gc, XBT_NULL, state_path); int rc = 0; - libxl__ao_device_remove *aorm = 0; + if (aorm->dev->backend_kind == LIBXL__DEVICE_KIND_QDISK) { + aorm->callback(egc, aorm); + return; + } + + if (atoi(state) == XenbusStateInitWait) + goto out_ok; + + libxl__ev_devstate_init(&aorm->ds); + rc = libxl__ev_devstate_wait(gc, &aorm->ds, device_backend_callback, + state_path, XenbusStateInitWait, + LIBXL_INIT_TIMEOUT * 1000); + if (rc) { + LOGE(ERROR, "unable to initialize device %s", be_path); + goto out_fail; + } + + return; + +out_fail: + assert(rc); + aorm->rc = rc; + aorm->callback(egc, aorm); + return; + +out_ok: + libxl__device_hotplug(egc, aorm); + return; +} + +void libxl__initiate_device_remove(libxl__egc *egc, libxl__ao_device *aorm) +{ + STATE_AO_GC(aorm->ao); + libxl_ctx *ctx = libxl__gc_owner(gc); + xs_transaction_t t = 0; + char *be_path = libxl__device_backend_path(gc, aorm->dev); + char *state_path = libxl__sprintf(gc, "%s/state", be_path); + char *state; + int rc = 0; + + if (aorm->force) + libxl__xs_path_cleanup(gc, libxl__device_frontend_path(gc, aorm->dev)); + /* Remove frontend xs paths to force backend disconnection */ + +retry_transaction: + t = xs_transaction_start(ctx->xsh); + state = libxl__xs_read(gc, t, state_path); if (!state) goto out_ok; - if (atoi(state) != 4) { + if (atoi(state) != XenbusStateConnected && + atoi(state) != XenbusStateClosing) { libxl__device_destroy_tapdisk(gc, be_path); - xs_rm(ctx->xsh, XBT_NULL, be_path); goto out_ok; } -retry_transaction: - t = xs_transaction_start(ctx->xsh); - xs_write(ctx->xsh, t, libxl__sprintf(gc, "%s/online", be_path), "0", strlen("0")); + xs_write(ctx->xsh, t, GCSPRINTF("%s/online", be_path), "0", strlen("0")); xs_write(ctx->xsh, t, state_path, "5", strlen("5")); if (!xs_transaction_end(ctx->xsh, t, 0)) { if (errno == EAGAIN) @@ -408,60 +689,67 @@ retry_transaction: goto out_fail; } } + /* mark transaction as ended, to prevent double closing it on out_ok */ + t = 0; libxl__device_destroy_tapdisk(gc, be_path); - aorm = libxl__zalloc(gc, sizeof(*aorm)); - aorm->ao = ao; libxl__ev_devstate_init(&aorm->ds); - - rc = libxl__ev_devstate_wait(gc, &aorm->ds, device_remove_callback, + rc = libxl__ev_devstate_wait(gc, &aorm->ds, device_backend_callback, state_path, XenbusStateClosed, LIBXL_DESTROY_TIMEOUT * 1000); - if (rc) goto out_fail; + if (rc) { + LOGE(ERROR, "unable to remove device %s", be_path); + goto out_fail; + } - return 0; + return; out_fail: assert(rc); - device_remove_cleanup(gc, aorm); - return rc; + aorm->rc = rc; + aorm->callback(egc, aorm); + return; out_ok: - libxl__ao_complete(egc, ao, 0); - return 0; + if (t) xs_transaction_end(ctx->xsh, t, 0); + libxl__device_hotplug(egc, aorm); + return; } -int libxl__device_destroy(libxl__gc *gc, libxl__device *dev) -{ - libxl_ctx *ctx = libxl__gc_owner(gc); - char *be_path = libxl__device_backend_path(gc, dev); - char *fe_path = libxl__device_frontend_path(gc, dev); - - xs_rm(ctx->xsh, XBT_NULL, be_path); - xs_rm(ctx->xsh, XBT_NULL, fe_path); - - libxl__device_destroy_tapdisk(gc, be_path); +/* Callback for device destruction */ - return 0; -} +static void device_remove_callback(libxl__egc *egc, libxl__ao_device *aorm); -int libxl__devices_destroy(libxl__gc *gc, uint32_t domid) +void libxl__devices_destroy(libxl__egc *egc, libxl__devices_remove_state *drs) { - libxl_ctx *ctx = libxl__gc_owner(gc); + STATE_AO_GC(drs->ao); char *path; unsigned int num_kinds, num_devs; char **kinds = NULL, **devs = NULL; - int i, j; - libxl__device dev; + int i, j, rc = 0, numdev = 0; + libxl__device *dev; libxl__device_kind kind; - path = libxl__sprintf(gc, "/local/domain/%d/device", domid); + drs->num_devices = libxl__num_devices(gc, drs->domid); + if (drs->num_devices < 0) { + LOGE(ERROR, "unable to get number of devices for domain %u", + drs->domid); + rc = ERROR_FAIL; + goto out; + } + + GCNEW_ARRAY(drs->aorm, drs->num_devices); + for (i = 0; i < drs->num_devices; i++) { + libxl__init_ao_device(&drs->aorm[i], drs->ao, &drs->aorm); + } + + path = GCSPRINTF("/local/domain/%d/device", drs->domid); kinds = libxl__xs_directory(gc, XBT_NULL, path, &num_kinds); if (!kinds) { if (errno != ENOENT) { - LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "unable to get xenstore" - " device listing %s", path); + LOGE(ERROR, "unable to get xenstore device listing %s", path); + rc = ERROR_FAIL; goto out; } num_kinds = 0; @@ -470,37 +758,106 @@ int libxl__devices_destroy(libxl__gc *gc, uint32_t domid) if (libxl__device_kind_from_string(kinds[i], &kind)) continue; - path = libxl__sprintf(gc, "/local/domain/%d/device/%s", domid, kinds[i]); + path = GCSPRINTF("/local/domain/%d/device/%s", drs->domid, kinds[i]); devs = libxl__xs_directory(gc, XBT_NULL, path, &num_devs); if (!devs) continue; for (j = 0; j < num_devs; j++) { - path = libxl__sprintf(gc, "/local/domain/%d/device/%s/%s/backend", - domid, kinds[i], devs[j]); + path = GCSPRINTF("/local/domain/%d/device/%s/%s/backend", + drs->domid, kinds[i], devs[j]); path = libxl__xs_read(gc, XBT_NULL, path); - if (path && libxl__parse_backend_path(gc, path, &dev) == 0) { - dev.domid = domid; - dev.kind = kind; - dev.devid = atoi(devs[j]); - - libxl__device_destroy(gc, &dev); + GCNEW(dev); + if (path && libxl__parse_backend_path(gc, path, dev) == 0) { + dev->domid = drs->domid; + dev->kind = kind; + dev->devid = atoi(devs[j]); + drs->aorm[numdev].action = DEVICE_DISCONNECT; + drs->aorm[numdev].dev = dev; + drs->aorm[numdev].callback = device_remove_callback; + libxl__initiate_device_remove(egc, &drs->aorm[numdev]); + numdev++; } } } /* console 0 frontend directory is not under /local/domain/<domid>/device */ - path = libxl__sprintf(gc, "/local/domain/%d/console/backend", domid); + path = GCSPRINTF("/local/domain/%d/console/backend", drs->domid); path = libxl__xs_read(gc, XBT_NULL, path); + GCNEW(dev); if (path && strcmp(path, "") && - libxl__parse_backend_path(gc, path, &dev) == 0) { - dev.domid = domid; - dev.kind = LIBXL__DEVICE_KIND_CONSOLE; - dev.devid = 0; + libxl__parse_backend_path(gc, path, dev) == 0) { + dev->domid = drs->domid; + dev->kind = LIBXL__DEVICE_KIND_CONSOLE; + dev->devid = 0; - libxl__device_destroy(gc, &dev); + libxl__device_destroy(gc, dev); } out: + if (!numdev) drs->callback(egc, drs, rc); + return; +} + +static void device_backend_callback(libxl__egc *egc, libxl__ev_devstate *ds, + int rc) +{ + libxl__ao_device *aorm = CONTAINER_OF(ds, *aorm, ds); + STATE_AO_GC(aorm->ao); + + device_backend_cleanup(gc, aorm); + + if (rc == ERROR_TIMEDOUT && aorm->action == DEVICE_DISCONNECT && + !aorm->force) { + aorm->force = 1; + libxl__initiate_device_remove(egc, aorm); + return; + } + + if (rc) { + LOGE(ERROR, "unable to %s device with path %s", + aorm->action == DEVICE_DISCONNECT ? "disconnect" : + "connect", + libxl__device_backend_path(gc, aorm->dev)); + goto error; + } + + libxl__device_hotplug(egc, aorm); + return; + +error: + aorm->rc = rc; + aorm->callback(egc, aorm); + return; +} + +static void device_remove_callback(libxl__egc *egc, libxl__ao_device *aorm) +{ + STATE_AO_GC(aorm->ao); + libxl__devices_remove_state *drs = CONTAINER_OF(aorm->base, *drs, aorm); + int rc = 0, last; + + if (aorm->action == DEVICE_DISCONNECT) { + libxl__xs_path_cleanup(gc, libxl__device_frontend_path(gc, aorm->dev)); + libxl__xs_path_cleanup(gc, libxl__device_backend_path(gc, aorm->dev)); + } + + rc = libxl__ao_device_check_last(gc, aorm, drs->aorm, + drs->num_devices, &last); + + if (last) + drs->callback(egc, drs, rc); + return; +} + +int libxl__device_destroy(libxl__gc *gc, libxl__device *dev) +{ + char *be_path = libxl__device_backend_path(gc, dev); + char *fe_path = libxl__device_frontend_path(gc, dev); + + libxl__xs_path_cleanup(gc, be_path); + libxl__xs_path_cleanup(gc, fe_path); + + libxl__device_destroy_tapdisk(gc, be_path); return 0; } diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c index 5de3aef..b2622ee 100644 --- a/tools/libxl/libxl_dm.c +++ b/tools/libxl/libxl_dm.c @@ -660,6 +660,8 @@ retry_transaction: return 0; } +static void spawn_stub_disk_connected(libxl__egc *egc, libxl__ao_device *aorm); + static void spawn_stubdom_pvqemu_cb(libxl__egc *egc, libxl__dm_spawn_state *stubdom_dmss, int rc); @@ -668,8 +670,7 @@ void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state *sdss) { STATE_AO_GC(sdss->dm.spawn.ao); libxl_ctx *ctx = libxl__gc_owner(gc); - int i, num_console = STUBDOM_SPECIAL_CONSOLES, ret; - libxl__device_console *console; + int i, ret; libxl_device_vfb vfb; libxl_device_vkb vkb; char **args; @@ -787,22 +788,66 @@ retry_transaction: if (errno == EAGAIN) goto retry_transaction; + GCNEW_ARRAY(sdss->devices, dm_config->num_disks); + sdss->num_devices = dm_config->num_disks; for (i = 0; i < dm_config->num_disks; i++) { - ret = libxl_device_disk_add(ctx, dm_domid, &dm_config->disks[i]); - if (ret) + libxl__init_ao_device(&sdss->devices[i], ao, &sdss->devices); + sdss->devices[i].callback = spawn_stub_disk_connected; + ret = libxl__device_disk_add(egc, dm_domid, &dm_config->disks[i], + &sdss->devices[i]); + if (ret) { + LIBXL__LOG(ctx, LIBXL__LOG_ERROR, + "cannot add disk %d to domain: %d", i, ret); + ret = ERROR_FAIL; goto out_free; + } + } + + free(args); + return; + +out_free: + free(args); +out: + assert(ret); + spawn_stubdom_pvqemu_cb(egc, &sdss->pvqemu, ret); +} + +static void spawn_stub_disk_connected(libxl__egc *egc, libxl__ao_device *aorm) +{ + STATE_AO_GC(aorm->ao); + libxl_ctx *ctx = libxl__gc_owner(gc); + libxl__stub_dm_spawn_state *sdss = CONTAINER_OF(aorm->base, *sdss, devices); + int i, num_console = STUBDOM_SPECIAL_CONSOLES, ret = 0, last; + libxl__device_console *console; + + /* convenience aliases */ + libxl_domain_config *const dm_config = &sdss->dm_config; + libxl_domain_config *const guest_config = sdss->dm.guest_config; + const int guest_domid = sdss->dm.guest_domid; + libxl__domain_build_state *const d_state = sdss->dm.build_state; + libxl__domain_build_state *const stubdom_state = &sdss->dm_state; + uint32_t dm_domid = sdss->pvqemu.guest_domid; + + ret = libxl__ao_device_check_last(gc, aorm, sdss->devices, + sdss->num_devices, &last); + if (!last) return; + if (last && ret) { + LOGE(ERROR, "error connecting disk devices"); + goto out; } + for (i = 0; i < dm_config->num_vifs; i++) { ret = libxl_device_nic_add(ctx, dm_domid, &dm_config->vifs[i]); if (ret) - goto out_free; + goto out; } ret = libxl_device_vfb_add(ctx, dm_domid, &dm_config->vfbs[0]); if (ret) - goto out_free; + goto out; ret = libxl_device_vkb_add(ctx, dm_domid, &dm_config->vkbs[0]); if (ret) - goto out_free; + goto out; if (guest_config->b_info.u.hvm.serial) num_console++; @@ -810,7 +855,7 @@ retry_transaction: console = libxl__calloc(gc, num_console, sizeof(libxl__device_console)); if (!console) { ret = ERROR_NOMEM; - goto out_free; + goto out; } for (i = 0; i < num_console; i++) { @@ -846,7 +891,7 @@ retry_transaction: ret = libxl__device_console_add(gc, dm_domid, &console[i], i == STUBDOM_CONSOLE_LOGGING ? stubdom_state : NULL); if (ret) - goto out_free; + goto out; } sdss->pvqemu.guest_domid = dm_domid; @@ -856,13 +901,9 @@ retry_transaction: libxl__spawn_local_dm(egc, &sdss->pvqemu); - free(args); return; -out_free: - free(args); out: - assert(ret); spawn_stubdom_pvqemu_cb(egc, &sdss->pvqemu, ret); } @@ -883,7 +924,7 @@ static void spawn_stubdom_pvqemu_cb(libxl__egc *egc, out: if (rc) { if (dm_domid) - libxl_domain_destroy(CTX, dm_domid); + libxl_domain_destroy(CTX, dm_domid, 0); } sdss->callback(egc, &sdss->dm, rc); } @@ -1075,62 +1116,35 @@ static void device_model_spawn_outcome(libxl__egc *egc, int libxl__destroy_device_model(libxl__gc *gc, uint32_t domid) { - libxl_ctx *ctx = libxl__gc_owner(gc); char *pid; int ret; - pid = libxl__xs_read(gc, XBT_NULL, libxl__sprintf(gc, "/local/domain/%d/image/device-model-pid", domid)); - if (!pid) { - LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "Couldn't find device model's pid"); - ret = ERROR_INVAL; - goto out; - } - if (!atoi(pid)) { - int stubdomid = libxl_get_stubdom_id(ctx, domid); - const char *savefile; - - if (!stubdomid) { - LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "Couldn't find device model's pid"); + pid = libxl__xs_read(gc, XBT_NULL, + GCSPRINTF("/local/domain/%d/image/device-model-pid", + domid)); + if (!pid || !atoi(pid)) { + LOGE(ERROR, "Couldn't find device model's pid"); ret = ERROR_INVAL; goto out; - } - LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "Device model is a stubdom, domid=%d", stubdomid); - ret = libxl_domain_destroy(ctx, stubdomid); - if (ret) - goto out; - - savefile = libxl__device_model_savefile(gc, domid); - ret = unlink(savefile); - /* - * On suspend libxl__domain_save_device_model will have already - * unlinked the save file. - */ - if (ret && errno == ENOENT) ret = 0; - if (ret) { - LIBXL__LOG_ERRNO(ctx, XTL_ERROR, - "failed to remove device-model savefile %s\n", - savefile); - goto out; - } + } + ret = kill(atoi(pid), SIGHUP); + if (ret < 0 && errno == ESRCH) { + LOG(DEBUG, "Device Model already exited"); + ret = 0; + } else if (ret == 0) { + LOG(DEBUG, "Device Model signaled"); + ret = 0; } else { - ret = kill(atoi(pid), SIGHUP); - if (ret < 0 && errno == ESRCH) { - LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "Device Model already exited"); - ret = 0; - } else if (ret == 0) { - LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "Device Model signaled"); - ret = 0; - } else { - LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "failed to kill Device Model [%d]", - atoi(pid)); - ret = ERROR_FAIL; - goto out; - } + LOGEV(ERROR, errno, "failed to kill Device Model [%d]", atoi(pid)); + ret = ERROR_FAIL; + goto out; } out: - xs_rm(ctx->xsh, XBT_NULL, libxl__sprintf(gc, "/local/domain/0/device-model/%d", domid)); - xs_rm(ctx->xsh, XBT_NULL, libxl__sprintf(gc, "/local/domain/%d/hvmloader", domid)); + xs_rm(CTX->xsh, XBT_NULL, GCSPRINTF("/local/domain/0/device-model/%d", + domid)); + xs_rm(CTX->xsh, XBT_NULL, GCSPRINTF("/local/domain/%d/hvmloader", + domid)); return ret; } diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c index c246211..9b016e8 100644 --- a/tools/libxl/libxl_dom.c +++ b/tools/libxl/libxl_dom.c @@ -973,6 +973,176 @@ out: return rc; } +/* Callbacks for libxl__domain_destroy */ +static void stubdom_callback(libxl__egc *egc, libxl__destroy_domid_state *dis, + int rc); +static void domain_callback(libxl__egc *egc, libxl__destroy_domid_state *dis, + int rc); + +void libxl__domain_destroy(libxl__egc *egc, libxl__domain_destroy_state *dds) +{ + STATE_AO_GC(dds->ao); + uint32_t stubdomid = libxl_get_stubdom_id(CTX, dds->domid); + + if (stubdomid) { + dds->stubdom.ao = ao; + dds->stubdom.domid = stubdomid; + dds->stubdom.callback = stubdom_callback; + libxl__destroy_domid(egc, &dds->stubdom); + } else { + dds->stubdom_finished = 1; + } + + dds->domain.ao = ao; + dds->domain.domid = dds->domid; + dds->domain.callback = domain_callback; + libxl__destroy_domid(egc, &dds->domain); +} + +static void stubdom_callback(libxl__egc *egc, libxl__destroy_domid_state *dis, + int rc) +{ + STATE_AO_GC(dis->ao); + libxl__domain_destroy_state *dds = CONTAINER_OF(dis, *dds, stubdom); + const char *savefile; + + if (rc) + LOGE(ERROR, "unable to destroy stubdom with domid %u", dis->domid); + + dds->stubdom_finished = 1; + savefile = libxl__device_model_savefile(gc, dis->domid); + rc = unlink(savefile); + /* + * On suspend libxl__domain_save_device_model will have already + * unlinked the save file. + */ + if (rc && errno == ENOENT) rc = 0; + if (rc) { + LOGEV(ERROR, errno, "failed to remove device-model savefile %s", + savefile); + } + + if (dds->domain_finished) + dds->callback(egc, dds, rc); +} + +static void domain_callback(libxl__egc *egc, libxl__destroy_domid_state *dis, + int rc) +{ + STATE_AO_GC(dis->ao); + libxl__domain_destroy_state *dds = CONTAINER_OF(dis, *dds, domain); + + if (rc) + LOGE(ERROR, "unable to destroy guest with domid %u", dis->domid); + + dds->domain_finished = 1; + if (dds->stubdom_finished) + dds->callback(egc, dds, rc); +} + +/* Callbacks for libxl__domain_clean */ + +/* Device destruction callback */ +static void devices_destroy_cb(libxl__egc *egc, + libxl__devices_remove_state *drs, + int rc); + +void libxl__destroy_domid(libxl__egc *egc, libxl__destroy_domid_state *dis) +{ + STATE_AO_GC(dis->ao); + int rc; + + rc = libxl_domain_info(CTX, NULL, dis->domid); + switch (rc) { + case 0: + break; + case ERROR_INVAL: + LOGE(ERROR, "non-existant domain %d", dis->domid); + default: + goto out; + } + + switch (libxl__domain_type(gc, dis->domid)) { + case LIBXL_DOMAIN_TYPE_HVM: + dis->dm_present = 1; + break; + case LIBXL_DOMAIN_TYPE_PV: + dis->pid = libxl__xs_read(gc, XBT_NULL, + GCSPRINTF("/local/domain/%d/image/device-model-pid", + dis->domid)); + dis->dm_present = (dis->pid != NULL); + break; + default: + abort(); + } + + dis->dom_path = libxl__xs_get_dompath(gc, dis->domid); + if (!dis->dom_path) { + rc = ERROR_FAIL; + goto out; + } + + if (libxl__device_pci_destroy_all(gc, dis->domid) < 0) + LOGE(ERROR, "pci shutdown failed for domid %d", dis->domid); + rc = xc_domain_pause(CTX->xch, dis->domid); + if (rc < 0) { + LOGEV(ERROR, rc, "xc_domain_pause failed for %d", dis->domid); + } + if (dis->dm_present) { + rc = libxl__destroy_device_model(gc, dis->domid); + if (rc < 0) + LOGE(ERROR, "libxl__destroy_device_model failed for %d", + dis->domid); + libxl__qmp_cleanup(gc, dis->domid); + } + dis->drs.ao = ao; + dis->drs.domid = dis->domid; + dis->drs.callback = devices_destroy_cb; + libxl__devices_destroy(egc, &dis->drs); + return; + +out: + assert(rc); + dis->callback(egc, dis, rc); + return; +} + +static void devices_destroy_cb(libxl__egc *egc, + libxl__devices_remove_state *drs, + int rc) +{ + STATE_AO_GC(drs->ao); + libxl__destroy_domid_state *dis = CONTAINER_OF(drs, *dis, drs); + + if (rc < 0) + LOGE(ERROR, "libxl__devices_destroy failed for %d", dis->domid); + + dis->vm_path = libxl__xs_read(gc, XBT_NULL, GCSPRINTF("%s/vm", + dis->dom_path)); + if (dis->vm_path) + if (!xs_rm(CTX->xsh, XBT_NULL, dis->vm_path)) + LOGE(ERROR, "xs_rm failed for %s", dis->vm_path); + + if (!xs_rm(CTX->xsh, XBT_NULL, dis->dom_path)) + LOGE(ERROR, "xs_rm failed for %s", dis->dom_path); + + xs_rm(CTX->xsh, XBT_NULL, libxl__xs_libxl_path(gc, dis->domid)); + + libxl__userdata_destroyall(gc, dis->domid); + + rc = xc_domain_destroy(CTX->xch, dis->domid); + if (rc < 0) { + LOGEV(ERROR, rc, "xc_domain_destroy failed for %d", dis->domid); + rc = ERROR_FAIL; + goto out; + } + rc = 0; + +out: + dis->callback(egc, dis, rc); + return; +} + /* * Local variables: * mode: C diff --git a/tools/libxl/libxl_hotplug.c b/tools/libxl/libxl_hotplug.c new file mode 100644 index 0000000..686a38d --- /dev/null +++ b/tools/libxl/libxl_hotplug.c @@ -0,0 +1,84 @@ +/* + * Copyright (C) 2012 + * Author Roger Pau Monne <roger.pau@xxxxxxxxxx> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published + * by the Free Software Foundation; version 2.1 only. with the special + * exception on linking described in file LICENSE. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU Lesser General Public License for more details. + */ + +#include "libxl_osdeps.h" /* must come before any other headers */ + +#include "libxl_internal.h" + +/* + * Generic hotplug helpers + * + * This helpers are both used by Linux and NetBSD hotplug script + * dispatcher functions. + */ + +static void device_hotplug_timeout(libxl__egc *egc, libxl__ev_time *ev, + const struct timeval *requested_abs) +{ + libxl__ao_device *aorm = CONTAINER_OF(ev, *aorm, ev); + STATE_AO_GC(aorm->ao); + + if (!aorm) return; + if (libxl__ev_child_inuse(&aorm->child)) { + if (kill(aorm->pid, SIGKILL)) { + LOGEV(ERROR, errno, "unable to kill hotplug script %s [%ld]", + aorm->what, (unsigned long)aorm->pid); + goto out; + } + } + +out: + libxl__ev_time_deregister(gc, &aorm->ev); + return; +} + +int libxl__hotplug_launch(libxl__gc *gc, libxl__ao_device *aorm, + const char *arg0, char *const args[], + char *const env[], libxl__ev_child_callback *death) +{ + int rc = 0; + pid_t pid = -1; + + libxl__ev_time_init(&aorm->ev); + libxl__ev_child_init(&aorm->child); + + rc = libxl__ev_time_register_rel(gc, &aorm->ev, device_hotplug_timeout, + LIBXL_HOTPLUG_TIMEOUT * 1000); + if (rc) { + LOGE(ERROR, "unable to register timeout for hotplug script %s", arg0); + goto out; + } + + pid = libxl__ev_child_fork(gc, &aorm->child, death); + if (pid == -1) { + rc = ERROR_FAIL; + goto out; + } + if (!pid) { + /* child */ + libxl__exec(gc, -1, -1, -1, arg0, args, env); + LOGE(ERROR, "unable execute hotplug scripts for device %"PRIu32 " on " + "dom %"PRIu32, aorm->dev->devid, aorm->dev->backend_domid); + abort(); + } +out: + if (libxl__ev_child_inuse(&aorm->child)) { + aorm->pid = pid; + /* we will get a callback when the child dies */ + return 0; + } + libxl__ev_time_deregister(gc, &aorm->ev); + return rc; +} diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index fc2024d..faeb965 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -71,6 +71,8 @@ #include "_libxl_types_internal_json.h" #define LIBXL_DESTROY_TIMEOUT 10 +#define LIBXL_INIT_TIMEOUT 10 +#define LIBXL_HOTPLUG_TIMEOUT 10 #define LIBXL_DEVICE_MODEL_START_TIMEOUT 10 #define LIBXL_XENCONSOLE_LIMIT 1048576 #define LIBXL_XENCONSOLE_PROTOCOL "vt100" @@ -85,6 +87,7 @@ #define STUBDOM_CONSOLE_SERIAL 3 #define STUBDOM_SPECIAL_CONSOLES 3 #define TAP_DEVICE_SUFFIX "-emu" +#define DISABLE_UDEV_PATH "libxl/disable_udev" #define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0])) @@ -779,6 +782,9 @@ _hidden int libxl__domain_pvcontrol_write(libxl__gc *gc, xs_transaction_t t, _hidden char *libxl__device_disk_string_of_backend(libxl_disk_backend backend); _hidden char *libxl__device_disk_string_of_format(libxl_disk_format format); _hidden int libxl__device_disk_set_backend(libxl__gc*, libxl_device_disk*); +_hidden int libxl__device_from_disk(libxl__gc *gc, uint32_t domid, + libxl_device_disk *disk, + libxl__device *device); _hidden int libxl__device_physdisk_major_minor(const char *physpath, int *major, int *minor); _hidden int libxl__device_disk_dev_number(const char *virtpath, @@ -795,7 +801,6 @@ _hidden char *libxl__device_frontend_path(libxl__gc *gc, libxl__device *device); _hidden int libxl__parse_backend_path(libxl__gc *gc, const char *path, libxl__device *dev); _hidden int libxl__device_destroy(libxl__gc *gc, libxl__device *dev); -_hidden int libxl__devices_destroy(libxl__gc *gc, uint32_t domid); _hidden int libxl__wait_for_backend(libxl__gc *gc, char *be_path, char *state); /* @@ -826,13 +831,6 @@ _hidden const char *libxl__device_nic_devname(libxl__gc *gc, uint32_t devid, libxl_nic_type type); -/* Arranges that dev will be removed from its guest. When - * this is done, the ao will be completed. An error - * return from libxl__initiate_device_remove means that the ao - * will _not_ be completed and the caller must do so. */ -_hidden int libxl__initiate_device_remove(libxl__egc*, libxl__ao*, - libxl__device *dev); - /* * libxl__ev_devstate - waits a given time for a device to * reach a given state. Follows the libxl_ev_* conventions. @@ -918,6 +916,65 @@ _hidden int libxl__create_pci_backend(libxl__gc *gc, uint32_t domid, libxl_device_pci *pcidev, int num); _hidden int libxl__device_pci_destroy_all(libxl__gc *gc, uint32_t domid); +/*----- device addition/removal -----*/ + +/* During the init/destruction process, the device can be in several states: + * + * DEVICE_UNKNOWN: device in unknown state (default value when struct is + * initialized). + * + * DEVICE_WAIT_BACKEND: waiting for the backend to switch to XenbusStateInit or + * XenbusStateClosed. + * + * DEVICE_WAIT_HOTPLUG: waiting for hotplug script to finish execution. + * + * DEVICE_FINISHED: device is connected/disconnected. + */ +typedef enum { + DEVICE_ACTIVE, + DEVICE_FINISHED +} libxl__device_state; + +/* Action to perform (either connect or disconnect) */ +typedef enum { + DEVICE_CONNECT, + DEVICE_DISCONNECT +} libxl__device_action; + +typedef struct libxl__ao_device libxl__ao_device; +typedef void libxl__device_callback(libxl__egc*, libxl__ao_device*); + +_hidden void libxl__init_ao_device(libxl__ao_device *aorm, libxl__ao *ao, + libxl__ao_device **base); +_hidden int libxl__ao_device_check_last(libxl__gc *gc, libxl__ao_device *device, + libxl__ao_device *list, + int num, int *last); + +struct libxl__ao_device { + libxl__ao *ao; + /* State in which the device is */ + libxl__device_state state; + /* action being performed */ + libxl__device_action action; + /* device teardown (back for xenstore backend to connect/disconnect) */ + libxl__device *dev; + int force; + /* device hotplug execution */ + pid_t pid; + char *what; + libxl__device_callback *callback; + /* private for implementation */ + int rc; + libxl__ev_time ev; + libxl__ev_child child; + libxl__ev_devstate ds; + void *base; +}; + +_hidden int libxl__device_disk_add(libxl__egc *egc, uint32_t domid, + libxl_device_disk *disk, + libxl__ao_device *aorm); + /* *----- spawn ----- * @@ -1106,6 +1163,8 @@ typedef struct { libxl_domain_config dm_config; libxl__domain_build_state dm_state; libxl__dm_spawn_state pvqemu; + libxl__ao_device *devices; + int num_devices; } libxl__stub_dm_spawn_state; _hidden void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state*); @@ -1797,6 +1856,98 @@ _hidden void libxl__bootloader_init(libxl__bootloader_state *bl); * If callback is passed rc==0, will have updated st->info appropriately */ _hidden void libxl__bootloader_run(libxl__egc*, libxl__bootloader_state *st); +/* + * libxl__hotplug_launch is an internal function that should not be used + * directly, all hotplug script calls have to implement and use + * libxl__device_hotplug. + */ +_hidden void libxl__device_hotplug(libxl__egc *egc, libxl__ao_device *aorm); +_hidden int libxl__hotplug_launch(libxl__gc *gc, libxl__ao_device *aorm, + const char *arg0, char *const args[], + char *const env[], + libxl__ev_child_callback *death); + +/* Arranges that dev will be added to the guest, and the + * hotplug scripts will be executed (if necessary). When + * this is done (or an error happens), the callback in + * aorm->callback will be called. + */ +_hidden void libxl__initiate_device_add(libxl__egc*, libxl__ao_device *aorm); + +/* Arranges that dev will be removed to the guest, and the + * hotplug scripts will be executed (if necessary). When + * this is done (or an error happens), the callback in + * aorm->callback will be called. + */ +_hidden void libxl__initiate_device_remove(libxl__egc *egc, + libxl__ao_device *aorm); + +/*----- Domain destruction -----*/ + +typedef struct libxl__domain_destroy_state libxl__domain_destroy_state; +typedef struct libxl__destroy_domid_state libxl__destroy_domid_state; +typedef struct libxl__devices_remove_state libxl__devices_remove_state; + +typedef void libxl__domain_destroy_cb(libxl__egc *egc, + libxl__domain_destroy_state *dds, + int rc); + +typedef void libxl__domain_clean_cb(libxl__egc *egc, + libxl__destroy_domid_state *dis, + int rc); + +typedef void libxl__devices_remove_callback(libxl__egc *egc, + libxl__devices_remove_state *drs, + int rc); + +struct libxl__devices_remove_state { + /* set by user */ + libxl__ao *ao; + uint32_t domid; + libxl__devices_remove_callback *callback; + /* private */ + libxl__ao_device *aorm; + int num_devices; +}; + +struct libxl__destroy_domid_state { + /* filled in by user */ + libxl__ao *ao; + uint32_t domid; + libxl__domain_clean_cb *callback; + /* private to implementation */ + libxl__devices_remove_state drs; + char *dom_path; + char *vm_path; + char *pid; + int dm_present; +}; + +struct libxl__domain_destroy_state { + /* filled by the user */ + libxl__ao *ao; + uint32_t domid; + libxl__domain_destroy_cb *callback; + /* Private */ + uint32_t stubdomid; + libxl__destroy_domid_state stubdom; + int stubdom_finished; + libxl__destroy_domid_state domain; + int domain_finished; +}; + +/* Entry point for domain destruction */ +_hidden void libxl__domain_destroy(libxl__egc *egc, + libxl__domain_destroy_state *dds); + +/* Used to destroy a domain with the passed id (it doesn't check for stubs) */ +_hidden void libxl__destroy_domid(libxl__egc *egc, + libxl__destroy_domid_state *dis); + +/* Entry point for devices destruction */ +_hidden void libxl__devices_destroy(libxl__egc *egc, + libxl__devices_remove_state *drs); + /*----- Domain creation -----*/ typedef struct libxl__domain_create_state libxl__domain_create_state; @@ -1816,6 +1967,9 @@ struct libxl__domain_create_state { int guest_domid; libxl__domain_build_state build_state; libxl__bootloader_state bl; + libxl__ao_device *devices; + int num_devices; + libxl__domain_destroy_state dds; libxl__stub_dm_spawn_state dmss; /* If we're not doing stubdom, we use only dmss.dm, * for the non-stubdom device model. */ diff --git a/tools/libxl/libxl_linux.c b/tools/libxl/libxl_linux.c index 925248b..315ce15 100644 --- a/tools/libxl/libxl_linux.c +++ b/tools/libxl/libxl_linux.c @@ -25,3 +25,129 @@ int libxl__try_phy_backend(mode_t st_mode) return 1; } + +/* Hotplug scripts helpers */ + +static void cleanup(libxl__gc *gc, libxl__ao_device *aorm) +{ + if (!aorm) return; + libxl__ev_time_deregister(gc, &aorm->ev); +} + +static void callback(libxl__egc *egc, libxl__ev_child *child, + pid_t pid, int status) +{ + libxl__ao_device *aorm = CONTAINER_OF(child, *aorm, child); + STATE_AO_GC(aorm->ao); + + cleanup(gc, aorm); + + if (status) { + libxl_report_child_exitstatus(CTX, aorm->rc ? LIBXL__LOG_ERROR + : LIBXL__LOG_WARNING, + aorm->what, pid, status); + aorm->rc = ERROR_FAIL; + } + aorm->callback(egc, aorm); +} + +static char **get_hotplug_env(libxl__gc *gc, libxl__device *dev) +{ + char *be_path = libxl__device_backend_path(gc, dev); + char *script; + const char *type = libxl__device_kind_to_string(dev->backend_kind); + char **env; + int nr = 0; + + script = libxl__xs_read(gc, XBT_NULL, + GCSPRINTF("%s/%s", be_path, "script")); + if (!script) { + LOGEV(ERROR, errno, "unable to read script from %s", be_path); + return NULL; + } + + GCNEW_ARRAY(env, 9); + env[nr++] = "script"; + env[nr++] = script; + env[nr++] = "XENBUS_TYPE"; + env[nr++] = libxl__strdup(gc, type); + env[nr++] = "XENBUS_PATH"; + env[nr++] = GCSPRINTF("backend/%s/%u/%d", type, dev->domid, dev->devid); + env[nr++] = "XENBUS_BASE_PATH"; + env[nr++] = "backend"; + env[nr++] = NULL; + + return env; +} + +/* Hotplug scripts caller functions */ + +static int libxl__hotplug_disk(libxl__gc *gc, libxl__ao_device *aorm) +{ + char *be_path = libxl__device_backend_path(gc, aorm->dev); + char *script; + char **args, **env; + int nr = 0, rc = 0; + + script = libxl__xs_read(gc, XBT_NULL, + GCSPRINTF("%s/%s", be_path, "script")); + if (!script) { + LOGEV(ERROR, errno, "unable to read script from %s", be_path); + return ERROR_FAIL; + } + + env = get_hotplug_env(gc, aorm->dev); + if (!env) + return ERROR_FAIL; + + GCNEW_ARRAY(args, 3); + + args[nr++] = script; + args[nr++] = aorm->action == DEVICE_CONNECT ? "add" : "remove"; + args[nr++] = NULL; + + aorm->what = GCSPRINTF("%s %s", args[0], args[1]); + LOG(DEBUG, "calling hotplug script: %s %s", args[0], args[1]); + rc = libxl__hotplug_launch(gc, aorm, args[0], args, env, callback); + if (rc) { + goto out_free; + } + + rc = 0; + +out_free: + return rc; +} + +void libxl__device_hotplug(libxl__egc *egc, libxl__ao_device *aorm) +{ + STATE_AO_GC(aorm->ao); + char *disable_udev = libxl__xs_read(gc, XBT_NULL, DISABLE_UDEV_PATH); + + /* Check if we have to run hotplug scripts */ + if (!disable_udev) goto out; + + switch (aorm->dev->backend_kind) { + case LIBXL__DEVICE_KIND_VBD: + aorm->rc = libxl__hotplug_disk(gc, aorm); + if (aorm->rc) goto error; + break; + default: + /* If no need to execute any hotplug scripts, + * call the callback manually + */ + aorm->rc = 0; + aorm->callback(egc, aorm); + break; + } + + return; + +error: + assert(aorm->rc); + LOGE(ERROR, "unable to queue execution of hotplug script for device " + "with path %s", libxl__device_backend_path(gc, aorm->dev)); +out: + aorm->callback(egc, aorm); + return; +} diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index 551e367..b1351de 100644 --- a/tools/libxl/libxl_types.idl +++ b/tools/libxl/libxl_types.idl @@ -220,6 +220,7 @@ libxl_domain_create_info = Struct("domain_create_info",[ ("xsdata", libxl_key_value_list), ("platformdata", libxl_key_value_list), ("poolid", uint32), + ("run_hotplug_scripts",libxl_defbool), ], dir=DIR_IN) MemKB = UInt(64, init_val = "LIBXL_MEMKB_DEFAULT") diff --git a/tools/libxl/xl.c b/tools/libxl/xl.c index d4db1f8..d992c32 100644 --- a/tools/libxl/xl.c +++ b/tools/libxl/xl.c @@ -35,6 +35,7 @@ xentoollog_logger_stdiostream *logger; int dryrun_only; int autoballoon = 1; +libxl_defbool run_hotplug_scripts; char *lockfile; char *default_vifscript = NULL; char *default_bridge = NULL; @@ -66,6 +67,9 @@ static void parse_global_config(const char *configfile, if (!xlu_cfg_get_long (config, "autoballoon", &l, 0)) autoballoon = l; + libxl_defbool_setdefault(&run_hotplug_scripts, true); + xlu_cfg_get_defbool(config, "run_hotplug_scripts", &run_hotplug_scripts, 0); + if (!xlu_cfg_get_string (config, "lockfile", &buf, 0)) lockfile = strdup(buf); else { diff --git a/tools/libxl/xl.h b/tools/libxl/xl.h index 2b6714a..43d727a 100644 --- a/tools/libxl/xl.h +++ b/tools/libxl/xl.h @@ -109,6 +109,7 @@ void postfork(void); /* global options */ extern int autoballoon; +extern libxl_defbool run_hotplug_scripts; extern int dryrun_only; extern char *lockfile; extern char *default_vifscript; diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c index 8908581..3f1bbf6 100644 --- a/tools/libxl/xl_cmdimpl.c +++ b/tools/libxl/xl_cmdimpl.c @@ -562,6 +562,7 @@ static void parse_config_data(const char *configfile_filename_report, } } + c_info->run_hotplug_scripts = run_hotplug_scripts; c_info->type = LIBXL_DOMAIN_TYPE_PV; if (!xlu_cfg_get_string (config, "builder", &buf, 0) && !strncmp(buf, "hvm", strlen(buf))) @@ -1333,7 +1334,7 @@ static int handle_domain_death(libxl_ctx *ctx, uint32_t domid, /* fall-through */ case LIBXL_ACTION_ON_SHUTDOWN_DESTROY: LOG("Domain %d needs to be cleaned up: destroying the domain", domid); - libxl_domain_destroy(ctx, domid); + libxl_domain_destroy(ctx, domid, 0); break; case LIBXL_ACTION_ON_SHUTDOWN_COREDUMP_DESTROY: @@ -1899,7 +1900,7 @@ start: error_out: release_lock(); if (libxl_domid_valid_guest(domid)) - libxl_domain_destroy(ctx, domid); + libxl_domain_destroy(ctx, domid, 0); out: if (logfile != 2) @@ -2386,7 +2387,7 @@ static void destroy_domain(const char *p) fprintf(stderr, "Cannot destroy privileged domain 0.\n\n"); exit(-1); } - rc = libxl_domain_destroy(ctx, domid); + rc = libxl_domain_destroy(ctx, domid, 0); if (rc) { fprintf(stderr,"destroy failed (rc=%d)\n",rc); exit(-1); } } @@ -2660,7 +2661,7 @@ static int save_domain(const char *p, const char *filename, int checkpoint, if (checkpoint) libxl_domain_unpause(ctx, domid); else - libxl_domain_destroy(ctx, domid); + libxl_domain_destroy(ctx, domid, 0); exit(0); } @@ -2892,7 +2893,7 @@ static void migrate_domain(const char *domain_spec, const char *rune, } fprintf(stderr, "migration sender: Target reports successful startup.\n"); - libxl_domain_destroy(ctx, domid); /* bang! */ + libxl_domain_destroy(ctx, domid, 0); /* bang! */ fprintf(stderr, "Migration successful.\n"); exit(0); @@ -3010,7 +3011,7 @@ static void migrate_receive(int debug, int daemonize, int monitor) if (rc) { fprintf(stderr, "migration target: Failure, destroying our copy.\n"); - rc2 = libxl_domain_destroy(ctx, domid); + rc2 = libxl_domain_destroy(ctx, domid, 0); if (rc2) { fprintf(stderr, "migration target: Failed to destroy our copy" " (code %d).\n", rc2); @@ -5077,7 +5078,7 @@ int main_blockattach(int argc, char **argv) return 0; } - if (libxl_device_disk_add(ctx, fe_domid, &disk)) { + if (libxl_device_disk_add(ctx, fe_domid, &disk, 0)) { fprintf(stderr, "libxl_device_disk_add failed.\n"); } return 0; diff --git a/tools/python/xen/lowlevel/xl/xl.c b/tools/python/xen/lowlevel/xl/xl.c index c4f7c52..ce7a2b2 100644 --- a/tools/python/xen/lowlevel/xl/xl.c +++ b/tools/python/xen/lowlevel/xl/xl.c @@ -447,7 +447,7 @@ static PyObject *pyxl_domain_destroy(XlObject *self, PyObject *args) int domid; if ( !PyArg_ParseTuple(args, "i", &domid) ) return NULL; - if ( libxl_domain_destroy(self->ctx, domid) ) { + if ( libxl_domain_destroy(self->ctx, domid, NULL) ) { PyErr_SetString(xl_error_obj, "cannot destroy domain"); return NULL; } -- 1.7.7.5 (Apple Git-26) _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.