Re: [Xen-devel] xen_emul_unplug on xen 4.1, HVM guest 2.6.38

[Alex Bligh <alex@xxxxxxxxxxx>]

Ian,

> No, this will disable PV drivers.
I can confirm that our testing illustrates this :-(

> The decision to unplug is a kernel side decision and in PVHVM Linux
> kernels is not currently possible to have both types of devices by
> default due to the risk of dataloss if the guest is not correctly
> configured (i.e. the kernel can't tell if it is mounting the same
> filesystem via two paths). The xen_emul_unplug option is the current way
> you can override this once you have confirmed that your guest
> configuration is not dangerous. I'm afraid this necessarily involves
> guest config and guest admin interaction.
>
> In principal we might be able to extend the unplug protocol (which would
> involve patches to qemu, the kernel(s) and the toolstack) to allow
> devices to be marked as being not necessary to unplug. Someone would
> have to send patches though and it would be opening up a way for people
> to lose data so we'd need to be careful.
>
> I'm sure that the unplug protocol is documented somewhere in the source
> tree but I can't for the life of me find it :-(
So, the issue is this. We have thousands (literally) of disks in use
by third parties on xen 3.3. Some are Windows, some are ancient linux,
some are modern linux, etc. The hypervisor has no way of whether the
images are going to use /dev/sda or /dev/xvda (i.e. PV or emulated)
drivers. Indeed the most common linux case is that grub uses the
emulated devices to load the kernel, then uses /dev/xvda as a root
device, i.e. both are used (but not simultaneously).

We need to have the xen pci stuff on, so PV drivers operate (in both
new and old kernels). But as modern linux kernels detect the unplug 
functionality, they will unplug the emulated devices and then fail to
boot because (for instance) under Xen3.3 using /dev/sdaX to access
(say) your /boot partition worked perfectly well. What we need is a
switch to revert to the old Xen3.3 (pre-unplug) behaviour, so any
Linux kernel will see the same set of devices. I cannot believe this
is a unique requirement for people attempting to do a Xen3.3 to
Xen4.1 migration.

I think this is in xen_unplug_emulated_devices() in
arch/x86/xen/platform-pci-unplug.c

This uses check_platform_magic(), which I have appended. In order
to avoid unplugging (without relying on the boot line), I need
this to return a non-zero value (XEN_PLATFORM_ERR_MAGIC is
irrelevant as xen_emul_unplug is 0 by assumption).

I can achieve that by either (a) returning a bad magic number,
(b) making the host 'blacklist' the product (how does that work?)
or (c) using a protocol value of (say) 0. I take it Xen 3.3 simply
returns a bad magic number as I don't think XEN_IOPORT_MAGIC existed
in 3.3. As far as I can tell, XEN_IOPORT_MAGIC is only use for
PCI unplug.

So, is the correct approach to disable XEN_IOPORT_MAGIC (or rather
make it return a different value) depending on a configuration option?
If so, I am happy to submit a patch to do that. Or can I do this
without a patch by "blacklisting" everything? (not sure how that is done).

Out of interest, with a default guest Ubuntu Natty install CD, using the
default Xen 4.1 settings, we are seeing the guest (a) unplugging the
emulated devices (fine), then (b) failing to find the emulated devices,
and (c) the install failing. Is that to be expected?

--
Alex Bligh


static int __init check_platform_magic(void)
{
       short magic;
       char protocol;

       magic = inw(XEN_IOPORT_MAGIC);
       if (magic != XEN_IOPORT_MAGIC_VAL) {
               printk(KERN_ERR "Xen Platform PCI: unrecognised magic 
value\n");
               return XEN_PLATFORM_ERR_MAGIC;
       }

       protocol = inb(XEN_IOPORT_PROTOVER);

       printk(KERN_DEBUG "Xen Platform PCI: I/O protocol version %d\n",
                       protocol);

       switch (protocol) {
       case 1:
               outw(XEN_IOPORT_LINUX_PRODNUM, XEN_IOPORT_PRODNUM);
               outl(XEN_IOPORT_LINUX_DRVVER, XEN_IOPORT_DRVVER);
               if (inw(XEN_IOPORT_MAGIC) != XEN_IOPORT_MAGIC_VAL) {
                       printk(KERN_ERR "Xen Platform: blacklisted by 
host\n");
                       return XEN_PLATFORM_ERR_BLACKLIST;
               }
               break;
       default:
               printk(KERN_WARNING "Xen Platform PCI: unknown I/O protocol 
version");
               return XEN_PLATFORM_ERR_PROTOCOL;
       }

       return 0;
}


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel