|
xen-devel
Re: [Xen-devel] xen_emul_unplug on xen 4.1, HVM guest 2.6.38
Ian,
--On 27 October 2011 10:27:08 +0100 Ian Campbell <Ian.Campbell@xxxxxxxxxx>
wrote:
I am not avoiding answering your question (I will get you this) but
what is /meant/ to happen in the following scenario:
(now answered in another email) :-)
Lets call this scenario A:
* Install on recent kernel (e.g. 2.6.37) running on Xen 3.3
* No fancy boot options, xen_emul_unplug not set
* No XEN_IOPORT_MAGIC implemented, so check_platform_magic()
returns an error
Correct, specifically it returns XEN_PLATFORM_ERR_MAGIC.
* Therefore xen_platform_pci_unplug=0
Correct. There is some special casing around XEN_PLATFORM_ERR_MAGIC but
lets assume xen_emul_unplug=unnecessary has not been passed to the
kernel so it does not take effect.
* Therefore blkfront etc. don't init (probe returns
-ENODEV)
* Therefore OS boots with root=/dev/sda
sda must be an emulated IDE device since the 2.6.37 kernel does not
support PV devices with names other than xvd*.
I think this is as expected.
Yes, correct. So an in image created for xen 3.3 will have root=/dev/sda
Now Xen 3.3 is upgraded to Xen 4
Lets call this scenario B:
* Kernel boots, and XEN_IOPORT_MAGIC now exists
* Therefore unplug occurs, and xen_platform_pci_unplug is non zero
* Therefore blkfront etc. inits, and PV drivers start
* OS still boots with root=/dev/sda
Are the PV devices meant to appear as /dev/sdX rather than /dev/xvdX?
No, there is no support for this in upstream kernels (in general all the
old behaviours of Xen kernels where they would hijack other drivers
device names are not upstreamble)
I thought so (from reading the code)
If so, how in the code does this happen? If not, won't the boot fail?
You need to be using root=/dev/xvda here for it to work. Or even better
use root=UUID=thing or root=LABEL=thong (many distros do one of these by
default these days).
(I'd forgotten about the UUID=/LABEL= option til just now -- that might
be the bit of magic which was missing to make this work)
Right, so the same image that worked fine on Xen 3 will not work if
Xen 3 is upgraded to Xen 4 - it will require changes to the image.
It's not only root= on the command line, but also /etc/fstab (we have
partitioned disks in our images).
Sure, using UUIDs works, but unfortunately we have thousands of VMs run by
users who, not possessing crystal balls, did not know the names of their
block devices were to change :-) For completeness, there is a also a
problem with certain versions of grub (pre grub2) where UUID config doesn't
work;
whilst grub of course loads before the kernel and before the unplug, the
grub config file still references a device which has been unplugged, so
the next time grub is reconfigured, it can't find the disk to write to.
> Hmm, yes I think the special treatment of XEN_IOPORT_MAGIC mismatch on
> the kernel side is what I was missing.
>
> It might make sense to have a guest level config option which disables
> these magic ports, i.e. makes them return 0xffff like they would have
> done in 3.3 (I think 0xffff is what you'll get from an invalid port in
> general).
Actually I don't think this will work. If we do this,
xen_plaftofm_pci_unplug will still be zero (as it's only set on exit
of the function after a successful unplug), and that's enough to
prevent blkfront and xenbus_probe_frontend from doing anything useful,
so will effectively disable PV drivers even where they should be enabled.
Correct, this will take you back to scenario A, however if that is how
the guest is configured (to use emulated devices) then this is what you
wanted (or at least it is what the guest configuration is expecting).
If the guest were configured to use PV devices then it would be using
root=/dev/xvda. Such a configuration would have needed
xen_emul_unplug=unnecessary in order to have worked on 3.3 before the
upgrade and this option would be harmless but unnecessary on 4.1.
Yes.
The trouble is some guests are running Windows, some old xenlinux
kernels, and some new kernels that talk to XEN_IOPORT_MAGIC. I need
one config that works for all of them, as we have no way of telling
them apart.
If I disable XEN_IOPORT_MAGIC then new installs of newer operating
systems will not run with PV drivers. I agree that these won't
run (without tweaked kernel boot lines) on Xen 3.3 with PV drivers
either, but I don't want to be stuck this way for ever.
I /think/ what you are saying is that the on new kernels that talk
to XEN_IOPORT_MAGIC, they will use /dev/sdX (emulated disks). If
we upgrade to Xen4, these will all break, because we can't distinguish
them from the behaviour of a new kernel doing a new install, where
we want the unplug to occur. The fix is either to fix the root
device, or to pass xen_emul_unplug=unnecessary in which case it should
continue to use to the emulated devices.
It would have been really nice to have been able to pass something
through XEN_IOPORT_MAGIC to effectively cause the same effect as
xen_emul_unplug=unnecessary on the command line, but I don't see how
that can be done with code already in the wild - unless I'm wrong?
If you were using UUID=/LABEL= then I think things would have worked in
both cases (emulated on 3.3 and pv on 4.1) without additional kernel
parameters.
One detail worth mentioning is that if the guest is using PV drivers and
expecting xvd* named devices then prior to the unplug the devices
xvd[a-d] are also exposed as the IDE devices hd[a-d]. This is how the
bootloader is still able to access the emulated device.
Sure
--
Alex Bligh
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
|
|
Home