WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] debugging VM startup

To: "xen-users@xxxxxxxxxxxxxxxxxxx" <xen-users@xxxxxxxxxxxxxxxxxxx>
Subject: Re: [Xen-users] debugging VM startup
From: Miles Fidelman <mfidelman@xxxxxxxxxxxxxxxx>
Date: Thu, 28 Oct 2010 09:53:22 -0400
Delivery-date: Thu, 28 Oct 2010 06:54:45 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <4CC96E72.5040505@xxxxxxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <4CC96E72.5040505@xxxxxxxxxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.14) Gecko/20100930 SeaMonkey/2.0.9
A little more data:

----background-------
I have a Debian domU that is refusing to boot. Attaching to the console (xm console) it seems to be in the middle of mounting a drive, and then hanging.

....
[    2.387986] EXT3-fs: mounted filesystem with ordered data mode.
Begin: Running /scripts/local-bottom ... done.
done.
Begin: Running /scripts/init-bottom ... done.
INIT: version 2.86 booting
Starting the hotplug events dispatcher: udevd[ 5.467529] udevd version 125 started
.
Synthesizing the initial hotplug events...done.
Waiting for /dev to be fully populated...done.
Starting boot logger: bootlogd[ 6.190409] Adding 3145588k swap on /dev/xvda2. Priority:-1 extents:1 across:3145588k
[   89.796051] EXT3 FS on xvda1, internal journal
<hang>

I know how I might proceed if I were booting a physical machine - try to come up in single user mode, failing that, boot with a live CD and start examining things. But... this is a VM, with more limited "hooks" into the boot process. Any suggestions on how to proceed?

Note, I'm running what's supposed to be a 2-node high-availability configuration (DRBD-mirrored drives, pacemaker). The underlying RAID10 and DRBD arrays are reporting good status, and other VMs are running just fine. This one just refuses to come up on either node in the cluster.
------------

1. looking at another domU that's working fine, I see that its console log reports:

.....
 [  31.088678] EXT3 FS on xvda1, internal journal
[   42.316654] NET: Registered protocol family 10
----

while my broken machine is hanging between these two steps - suggesting either a disk problem or a networking problem

2. I mounted and fsck'd the volume, and it seems to be fine, so it looks like something is either broken in the network startup, or interfering with it.

Which leads to three questions:

1. any suggestions on what might spontaneously go wrong with a previously-functioning network startup?
2. any debugging suggestions?
3. specifically, any suggestions on how to insert myself into a boot process that is normally configured to be driven by pacemaker and xen's startup hooks - so I can do some step by step debugging of the boot process?

Thanks again,

Miles Fidelman


Miles Fidelman wrote:
Hi,

I have a Debian domU that is refusing to boot. Attaching to the console (xm console) it seems to be in the middle of mounting a drive, and then hanging.

....
[    2.387986] EXT3-fs: mounted filesystem with ordered data mode.
Begin: Running /scripts/local-bottom ... done.
done.
Begin: Running /scripts/init-bottom ... done.
INIT: version 2.86 booting
Starting the hotplug events dispatcher: udevd[ 5.467529] udevd version 125 started
.
Synthesizing the initial hotplug events...done.
Waiting for /dev to be fully populated...done.
Starting boot logger: bootlogd[ 6.190409] Adding 3145588k swap on /dev/xvda2. Priority:-1 extents:1 across:3145588k
[   89.796051] EXT3 FS on xvda1, internal journal
<hang>

I know how I might proceed if I were booting a physical machine - try to come up in single user mode, failing that, boot with a live CD and start examining things. But... this is a VM, with more limited "hooks" into the boot process. Any suggestions on how to proceed?

Note, I'm running what's supposed to be a 2-node high-availability configuration (DRBD-mirrored drives, pacemaker). The underlying RAID10 and DRBD arrays are reporting good status, and other VMs are running just fine. This one just refuses to come up on either node in the cluster.

Thanks,

Miles Fidelman





--
In theory, there is no difference between theory and practice.
In<fnord>  practice, there is.   .... Yogi Berra



_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

<Prev in Thread] Current Thread [Next in Thread>