WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

[Xen-users] BLKTAP limit in Dom0, can't boot with pygrub

To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-users] BLKTAP limit in Dom0, can't boot with pygrub
From: Jim Cipar <jcipar@xxxxxxx>
Date: Sat, 30 Aug 2008 11:25:24 -0400
Delivery-date: Sat, 30 Aug 2008 08:26:08 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Thunderbird 2.0.0.16 (X11/20080724)
I'm trying to start a large number of domains (hopefully around 70) each backed by a separate qcow disk. They are booting via pygrub. I've included an example config file below. I can start the first 16 DomUs just fine, but any after that fail to start. It looks like I'm hitting a limit to the number of blktap devices the Dom0 can have, so pygrub is failing to load the new disks. I've posted relevant sections of /var/log/xen/xend.log for both a successful and failed domain creation.

I'm nearly certain that the problem is with mounting the image in Dom0 so that pygrub can find a kernel on it; if I modify the config file for a failed domain to boot directly from Dom0's kernel, and then retry the domain creation, it works. Furthermore, if I do an "xm shutdown" on one of the successfully created pygrub domains, I can start one of the failed ones.

Could it be that there is a limit of 16 blktap devices in Dom0 and they're not being cleaned up properly after pygrub uses them? It looks like they are. In the log file, after pygrub returns it says "Unmounting /dev/xvdp from /dev/xvdp".

Thanks,
Jim Cipar





LOG FOR FAILED DOMAIN CREATION
---------------------------------------------------------------
[2008-08-30 10:06:11 5237] INFO (XendDomainInfo:2066) Mounting /home/jcipar/Disks/VM-17-0.qcow on /dev/xvdp. [2008-08-30 10:06:11 5237] DEBUG (DevController:117) DevController: writing {'backend-id': '0', 'virtual-device': '51952', 'device-type': 'disk', 'state': '1', 'backend': '/loca
l/domain/0/backend/tap/0/51952'} to /local/domain/0/device/vbd/51952.
[2008-08-30 10:06:11 5237] DEBUG (DevController:119) DevController: writing {'domain': 'Domain-0', 'frontend': '/local/domain/0/device/vbd/51952', 'uuid': '714fe8ee-0376-7cad-ab 0a-a3a8085166b6', 'dev': '/dev/xvdp', 'state': '1', 'params': 'qcow:/home/jcipar/Disks/VM-17-0.qcow', 'mode': 'w', 'online': '1', 'frontend-id': '0', 'type': 'tap'} to /local/domain/0/backend/tap/0/51952.
[2008-08-30 10:06:11 5237] DEBUG (DevController:155) Waiting for 51952.
[2008-08-30 10:06:11 5237] DEBUG (DevController:594) hotplugStatusCallback /local/domain/0/backend/tap/0/51952/hotplug-status. [2008-08-30 10:06:12 5237] DEBUG (DevController:594) hotplugStatusCallback /local/domain/0/backend/tap/0/51952/hotplug-status. [2008-08-30 10:06:12 5237] DEBUG (DevController:608) hotplugStatusCallback 1.
[2008-08-30 10:06:12 5237] DEBUG (DevController:155) Waiting for 51952.
[2008-08-30 10:06:12 5237] DEBUG (DevController:594) hotplugStatusCallback /local/domain/0/backend/tap/0/51952/hotplug-status. [2008-08-30 10:06:12 5237] DEBUG (DevController:608) hotplugStatusCallback 1.
[2008-08-30 10:06:12 5237] ERROR (XendBootloader:43) Disk isn't accessible
[2008-08-30 10:06:12 5237] INFO (XendDomainInfo:2084) Unmounting /dev/xvdp from /dev/xvdp. [2008-08-30 10:06:12 5237] DEBUG (XendDomainInfo:590) XendDomainInfo.destroyDevice: deviceClass = tap, device = /dev/xvdp
[2008-08-30 10:06:12 5237] ERROR (XendDomainInfo:440) VM start failed



LOG FOR SUCCESSFUL DOMAIN CREATION
-------------------------------------------------------------------------
[2008-08-30 10:06:06 5237] DEBUG (DevController:119) DevController: writing {'domain': 'Domain-0', 'frontend': '/local/domain/0/device/vbd/51952', 'uuid': '3fbc58db-daa3-b555-21 4e-14745062eb03', 'dev': '/dev/xvdp', 'state': '1', 'params': 'qcow:/home/jcipar/Disks/VM-16-0.qcow', 'mode': 'w', 'online': '1', 'frontend-id': '0', '
type': 'tap'} to /local/domain/0/backend/tap/0/51952.
[2008-08-30 10:06:06 5237] DEBUG (DevController:155) Waiting for 51952.
[2008-08-30 10:06:06 5237] DEBUG (DevController:594) hotplugStatusCallback /local/domain/0/backend/tap/0/51952/hotplug-status. [2008-08-30 10:06:06 5237] DEBUG (DevController:594) hotplugStatusCallback /local/domain/0/backend/tap/0/51952/hotplug-status. [2008-08-30 10:06:06 5237] DEBUG (DevController:608) hotplugStatusCallback 1.
[2008-08-30 10:06:06 5237] DEBUG (DevController:155) Waiting for 51952.
[2008-08-30 10:06:06 5237] DEBUG (DevController:594) hotplugStatusCallback /local/domain/0/backend/tap/0/51952/hotplug-status. [2008-08-30 10:06:06 5237] DEBUG (DevController:608) hotplugStatusCallback 1. [2008-08-30 10:06:06 12692] DEBUG (XendBootloader:100) Launching bootloader as ['/usr/bin/pygrub', '--args=root=/dev/xvda1 xencons=tty', '--output=/var/run/xend/boot/xenbl.15861
', '-q', '/dev/xvdp'].
[2008-08-30 10:06:07 5237] INFO (XendDomainInfo:2084) Unmounting /dev/xvdp from /dev/xvdp. [2008-08-30 10:06:07 5237] DEBUG (XendDomainInfo:590) XendDomainInfo.destroyDevice: deviceClass = tap, device = /dev/xvdp



EXAMPLE CONFIG FILE
------------------------------------------
bootloader="/usr/bin/pygrub"
disk=['tap:qcow:/home/jcipar/Disks/VM-1-0.qcow,xvda1,w']
vif = ['ip=172.19.158.1']
memory=100
root="/dev/xvda1"
extra='xencons=tty'


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

<Prev in Thread] Current Thread [Next in Thread>
  • [Xen-users] BLKTAP limit in Dom0, can't boot with pygrub, Jim Cipar <=