This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-users] Cannot boot with LVM inside Xen DomU

To: Prakhar Srivastava <prakhar.apj@xxxxxxxxx>
Subject: Re: [Xen-users] Cannot boot with LVM inside Xen DomU
From: Oliver Wilcock <oliver@xxxxxxx>
Date: Sun, 10 Jul 2011 09:29:13 -0400
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Delivery-date: Sun, 10 Jul 2011 06:30:58 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <CAF+Dr+tZpfhRsVRMfKHY5p+4x0WTicURUmnLHOVbJFfF7o_xSw@xxxxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <20110709190032.D488140E96A3@xxxxxxxxxxxxxx> <4E18E997.5050903@xxxxxxx> <CAF+Dr+tZpfhRsVRMfKHY5p+4x0WTicURUmnLHOVbJFfF7o_xSw@xxxxxxxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv: Gecko/20110616 Thunderbird/3.1.11
If you used a file based image for the guest, then your VM config will need to refer to that image.

The following is not a file based image:
> disk = [ 'phy:/dev/Box1/debian-lvm-image,xvda,w' ].

I don't use file based images, but I think you do something like this:
disk = [ 'file:/home/Box1/image.raw,xvda,w' ].

If you are already doing LVM then I don't see the value of file based images.

Where you are really on the wrong track is with the idea that the LVM is copied during rsync of a file system.

You could copy an LVM partitioned disk but you would use a block/bit based tool like dd.

If you did that, though, you would would create a nested LVM configuration. I think that will cause you grief. Two volume groups with the same name are going to be in conflict.

The following config lines are in conflict:
> and root='/dev/Box1/root
> Withing the virtual disk image, the /etc/fstab contains the entries
> /dev/xvda       /

The following is more likely:

This tells the guest kernel were to find the root image before it reads the fstab.

The idea of using the Dom0 kernel is possible, but harder to maintain in the long run. If you are making hundreds of light-weight VMs you might think the extra effort is worth it, but then you wouldn't copy the entire root file system, would you...

So, based on the snippets of your configuration file it appears that you are trying to use a LVM-backed guest image. Good choice; but poor description.

Mounting the image to copy in the root file system should work. You picked the same file system, right? If the Dom0 was using ext3 then the guest should be the same (cause the Dom0 initrd that you are recycling contains the driver for ext3). Using /dev/xvda will work but I have found, when using SUSE, that this creates challenges when trying to upgrade the guest from one release to another. The install programs expect a disk with a primary partition (or at least, they did the first time I tried this). As a consequence, I typically partition the guest image and use /dev/xvda1 for the root file system.

I hope the above fills in some of the the blanks in your understanding. With Xen you get lots of flexibility, but not all of the choices should necessarily be used together.

To better communicate your challenges it would help if you identified the "error".

So you want an VM inside your guest? And you want to avoid installing the guest? You will need to mount the guest image in the Dom0 and create an LVM with in it.

Here is a hint:
xm block-attach 0 phy:/dev/Dom0guest-space/debian-lvm-guest-image xvda w
pvcreate /dev/xvda
vgcreate Box1 /dev/xvda
lvcreate -n box1-boot -L20GB /dev/Box1

On 2011-07-10 04:24, Prakhar Srivastava wrote:
Thanx a lot for the response.
"Populated Xen guest image with Dom0)" - Take a look at
I have used a file based image and then formatted and mounted it at
/mnt. I copied the entire root disk as described using rsync to the file
based virtual disk image. Once I have the virtual disk image which i
believe should contain the LVM partitions (because the Dom0 system was
LVM partitioned), I copy it to a new logical volume
(say /dev/Box1/debian-lvm-image) which then I use to create a xen domU.

I am using the kernel and initrd of th dom0.

disk = [ 'phy:/dev/Box1/debian-lvm-image,xvda,w' ].
and root='/dev/Box1/root

Withing the virtual disk image, the /etc/fstab contains the entries
/dev/xvda       /
/dev/xvdb       swap
(I use a file based image for swap).

The setup described above does gives an error while booting DomU.
Please let me know if I should clarify further. My overall objective is
to run LVM wihin Xen guest VM.


On Sun, Jul 10, 2011 at 5:21 AM, Oliver Wilcock <oliver@xxxxxxx
<mailto:oliver@xxxxxxx>> wrote:

    I have used LVM inside a guest VM.

    I don't understand your description though.
    In particular I don't know what the following means: "Populated Xen
    guest image with DomO image."  You copied the bits from one LV to
    the new LV?

    If you are trying to avoid installing the guest OS by copying the
    bits, or the file system from your Dom0, then you are going to need
    to explain to it were to find root.

    If the LV that you created for your guest is /dev/Box1/root then the
    guest will need to use this name.

    i.e. /dev/Box1/root

    You will need to pass the entire disk into the guest.  Perhaps:

    disk = [ 'phy:/dev/sdb,xvda,w' ]

    To override the fstab and initrd expectations of the name of the
    root file system you can set the kernel parameter in your guest VM
    configuration.  Perhaps:

    Oh, and if the guest is managing the volume group then the Dom0
    can't. So before you mount the physical disk in the DomU you need to
    deactivate it in Dom0.

    vgchange -a n /dev/Box1

    If I've misunderstood and you are trying to create a nested LVM
    configuration I think you are going to run into trouble having two
    volume groups with the same name.

     > volume.
       You copied the bits from the

    On 2011-07-09 15:00, xen-users-request@xxxxxxxxxxxxxxxxxxxxx
    <mailto:xen-users-request@xxxxxxxxxxxxxxxxxxx> wrote:

        Date: Sat, 9 Jul 2011 22:27:41 +0530
        From: Prakhar Srivastava<prakhar.apj@xxxxxxxxxxx
        Subject: [Xen-users] Cannot boot with LVM inside Xen DomU
        Content-Type: text/plain; charset="iso-8859-1"

        I have Xen 4.0 installation on Debian Squeeze. I am able to boot
        Xen guest
        based on physical partition but failing to boot Xen guest with
        LVM within
        DomU. To use LVM inside DomU, I took the following way:
        1. Installed a basic debian squeeze system on a AMD64 box. Used
        the entire
        disk disk and configure LVM option to set root on LVM. The root
        is at
        /dev/Box1/root and swap is at /dev/Box1/swap_1
        2. Installed Xen and Dom0 from repository.
        3. Populated Xen guest image with DomO image.

        When I try to run the image, I get the error that the volume
        logical volume
        is not found. I have set root to /dev/mapper/root in Xen
        Is there anything I am missing. Has anyone used LVM inside Xen


    Xen-users mailing list
    Xen-users@xxxxxxxxxxxxxxxxxxx <mailto:Xen-users@xxxxxxxxxxxxxxxxxxx>

Xen-users mailing list