WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

[Xen-users] xen xm list: DomUs blocked

To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-users] xen xm list: DomUs blocked
From: Mark Furner <mark.furner@xxxxxxx>
Date: Tue, 30 Oct 2007 12:41:36 +0100
Delivery-date: Tue, 30 Oct 2007 04:42:31 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
Reply-to: mark.furner@xxxxxxx
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: KMail/1.9.5
Hi

I'm using Xen on my  AMD64 / Ubuntu 7.10 server which I recently upgraded, and 
my 2 VMs are decidedly flaky. They seem to be blocked for most of the time, 
and networking doesn't work properly. How can I debug this and get them 
online reliably? Any tips gratefully received!

* I can log in via console or ssh, can ping the server's NICs but can't ping 
the other VM or beyond the machine. From the server I can ping the VMs and 
outside.
* Firewall settings on the server are very relaxed at this point.
* The exceptions below about hda1 and hda2 don't make sense to me, since the 
devices are mounted via /etc/fstab as root and swap.
* /var/log/xen/xend-debug.log reports a broken pipe

Thanks!

M

uname string: 
Linux server.home.xx 2.6.19-4-generic-amd64 #2 SMP Sat Oct 6 01:35:53 UTC 2007 
x86_64 GNU/Linux

[11:57:03] $ sudo xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      782     1 r-----    487.2
slimserv                                   5       56     1 ------    183.3
work                                       7      128     1 ------      2.8
[11:57:06] $ sudo xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      782     1 r-----    488.3
slimserv                                   5       56     1 -b----    185.6
work                                       7      128     1 -b----      7.2
[11:57:20] $ sudo xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      782     1 r-----    521.0
slimserv                                   5       56     1 ------    267.3
work                                       7      128     1 -b----      7.3
[12:19:09] $  

Here is an excerpt from the xend-debug.log

Traceback (most recent call last):
  File "SocketServer.py", line 464, in process_request_thread
    self.finish_request(request, client_address)
  File "SocketServer.py", line 254, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "SocketServer.py", line 522, in __init__
    self.handle()
  File "BaseHTTPServer.py", line 316, in handle
    self.handle_one_request()
  File "BaseHTTPServer.py", line 310, in handle_one_request
    method()
  File "/usr/lib/python2.5/site-packages/xen/util/xmlrpclib2.py", line 66, in 
do_POST
    self.send_response(200)
  File "BaseHTTPServer.py", line 368, in send_response
    (self.protocol_version, code, message))
  File "socket.py", line 262, in write
    self.flush()
  File "socket.py", line 249, in flush
    self._sock.sendall(buffer)
error: (32, 'Broken pipe')


Here is a tail excerpt from /var/log/xen/xend.log:

[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:190) 
XendDomainInfo.create(['vm', ['name', 'work'], ['memory', '128'], 
['on_poweroff', 'destroy'], ['on_reboot', 'restart'], 
['on_crash', 'restart'], ['vcpus', 1], ['image', ['linux', 
['kernel', '/boot/vmlinuz-2.6.19-4-generic-amd64'], 
['ramdisk', '/boot/initrd.img-2.6.19-4-generic-amd64'], ['root', '/dev/hda1 
ro']]], ['device', ['vbd', ['uname', 'file:/var/xen/domains/work/disk.img'], 
['dev', 'hda1'], ['mode', 'w']]], ['device', ['vbd', 
['uname', 'file:/var/xen/domains/work/swap.img'], ['dev', 'hda2'], 
['mode', 'w']]], ['device', ['vif', ['ip', '192.168.2.6'], 
['mac', '00:16:3e:b1:d7:91'], ['vifname', 'vif2.0']]]])
[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:296) 
parseConfig: config is ['vm', ['name', 'work'], ['memory', '128'], 
['on_poweroff', 'destroy'], ['on_reboot', 'restart'], 
['on_crash', 'restart'], ['vcpus', 1], ['image', ['linux', 
['kernel', '/boot/vmlinuz-2.6.19-4-generic-amd64'], 
['ramdisk', '/boot/initrd.img-2.6.19-4-generic-amd64'], ['root', '/dev/hda1 
ro']]], ['device', ['vbd', ['uname', 'file:/var/xen/domains/work/disk.img'], 
['dev', 'hda1'], ['mode', 'w']]], ['device', ['vbd', 
['uname', 'file:/var/xen/domains/work/swap.img'], ['dev', 'hda2'], 
['mode', 'w']]], ['device', ['vif', ['ip', '192.168.2.6'], 
['mac', '00:16:3e:b1:d7:91'], ['vifname', 'vif2.0']]]]
[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:395) 
parseConfig: result is {'shadow_memory': None, 'uuid': 
None, 'on_crash': 'restart', 'on_reboot': 'restart', 'localtime': 
None, 'image': ['linux', ['kernel', '/boot/vmlinuz-2.6.19-4-generic-amd64'], 
['ramdisk', '/boot/initrd.img-2.6.19-4-generic-amd64'], ['root', '/dev/hda1 
ro']], 'on_poweroff': 'destroy', 'bootloader_args': None, 'cpus': 
None, 'name': 'work', 'backend': [], 'vcpus': 1, 'cpu_weight': 
None, 'features': None, 'vcpu_avail': None, 'memory': 128, 'device': [('vbd', 
['vbd', ['uname', 'file:/var/xen/domains/work/disk.img'], ['dev', 'hda1'], 
['mode', 'w']]), ('vbd', ['vbd', 
['uname', 'file:/var/xen/domains/work/swap.img'], ['dev', 'hda2'], 
['mode', 'w']]), ('vif', ['vif', ['ip', '192.168.2.6'], 
['mac', '00:16:3e:b1:d7:91'], ['vifname', 'vif2.0']])], 'bootloader': 
None, 'cpu': None, 'maxmem': None}
[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:1262) 
XendDomainInfo.construct: None
[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:1294) 
XendDomainInfo.initDomain: 7 1.0
[2007-10-30 11:57:01 xend 16192] DEBUG (balloon:127) Balloon: 131680 KiB free; 
need 131072; done.
[2007-10-30 11:57:01 xend 16192] INFO (image:136) buildDomain os=linux dom=7 
vcpus=1
[2007-10-30 11:57:01 xend 16192] DEBUG (image:199) dom            = 7
[2007-10-30 11:57:01 xend 16192] DEBUG (image:200) image          
= /boot/vmlinuz-2.6.19-4-generic-amd64
[2007-10-30 11:57:01 xend 16192] DEBUG (image:201) store_evtchn   = 1
[2007-10-30 11:57:01 xend 16192] DEBUG (image:202) console_evtchn = 2
[2007-10-30 11:57:01 xend 16192] DEBUG (image:203) cmdline        =  
root=/dev/hda1 ro
[2007-10-30 11:57:01 xend 16192] DEBUG (image:204) ramdisk        
= /boot/initrd.img-2.6.19-4-generic-amd64
[2007-10-30 11:57:01 xend 16192] DEBUG (image:205) vcpus          = 1
[2007-10-30 11:57:01 xend 16192] DEBUG (image:206) features       =
[2007-10-30 11:57:02 xend 16192] DEBUG (blkif:24) exception looking up device 
number for hda1: [Errno 2] No such file or directory: '/dev/hda1'
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:110) DevController: 
writing 
{'backend-id': '0', 'virtual-device': '769', 'device-type': 'disk', 'state': 
'1', 'backend': '/local/domain/0/backend/vbd/7/769'} 
to /local/domain/7/device/vbd/769.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:112) DevController: 
writing 
{'domain': 'work', 'frontend': '/local/domain/7/device/vbd/769', 'dev': 'hda1', 
'state': '1', 'params': '/var/xen/domains/work/disk.img', 'mode': 'w', 
'online': '1', 'frontend-id': '7', 'type': 'file'} 
to /local/domain/0/backend/vbd/7/769.
[2007-10-30 11:57:02 xend 16192] DEBUG (blkif:24) exception looking up device 
number for hda2: [Errno 2] No such file or directory: '/dev/hda2'
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:110) DevController: 
writing 
{'backend-id': '0', 'virtual-device': '770', 'device-type': 'disk', 'state': 
'1', 'backend': '/local/domain/0/backend/vbd/7/770'} 
to /local/domain/7/device/vbd/770.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:112) DevController: 
writing 
{'domain': 'work', 'frontend': '/local/domain/7/device/vbd/770', 'dev': 'hda2', 
'state': '1', 'params': '/var/xen/domains/work/swap.img', 'mode': 'w', 
'online': '1', 'frontend-id': '7', 'type': 'file'} 
to /local/domain/0/backend/vbd/7/770.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:110) DevController: 
writing 
{'backend-id': '0', 'mac': '00:16:3e:b1:d7:91', 'handle': '0', 'state': '1', 
'backend': '/local/domain/0/backend/vif/7/0'} 
to /local/domain/7/device/vif/0.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:112) DevController: 
writing 
{'domain': 'work', 'handle': '0', 'script': '/etc/xen/scripts/vif-route', 'ip': 
'192.168.2.6', 'state': '1', 'frontend': '/local/domain/7/device/vif/0', 'mac': 
'00:16:3e:b1:d7:91', 'vifname': 'vif2.0', 'online': '1', 'frontend-id': '7'} 
to /local/domain/0/backend/vif/7/0.
[2007-10-30 11:57:02 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:713) 
Storing VM details: 
{'shadow_memory': '0', 'uuid': '89708af1-f491-4aed-55e9-1be3acd352b7', 
'on_reboot': 'restart', 'start_time': '1193741822.28', 'on_poweroff': 
'destroy', 'name': 'work', 'xend/restart_count': '0', 'vcpus': '1', 
'vcpu_avail': '1', 'memory': '128', 'on_crash': 'restart', 'image': "(linux 
(kernel /boot/vmlinuz-2.6.19-4-generic-amd64) 
(ramdisk /boot/initrd.img-2.6.19-4-generic-amd64) (root '/dev/hda1 
ro'))", 'maxmem': '128'}
[2007-10-30 11:57:02 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:748) 
Storing domain details: 
{'console/ring-ref': '9890', 'console/port': '2', 'name': 'work', 
'console/limit': '1048576', 'vm': '/vm/89708af1-f491-4aed-55e9-1be3acd352b7', 
'domid': '7', 'cpu/0/availability': 'online', 'memory/target': '131072', 
'store/ring-ref': '9891', 'store/port': '1'}
[2007-10-30 11:57:02 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:938) 
XendDomainInfo.handleShutdownWatch
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:143) Waiting for devices 
vif.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:149) Waiting for 0.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:464) 
hotplugStatusCallback /local/domain/0/backend/vif/7/0/hotplug-status.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:464) 
hotplugStatusCallback /local/domain/0/backend/vif/7/0/hotplug-status.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:478) 
hotplugStatusCallback 1.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:143) Waiting for devices 
usb.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:143) Waiting for devices 
vbd.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:149) Waiting for 769.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:464) 
hotplugStatusCallback /local/domain/0/backend/vbd/7/769/hotplug-status.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:464) 
hotplugStatusCallback /local/domain/0/backend/vbd/7/769/hotplug-status.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:478) 
hotplugStatusCallback 1.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:149) Waiting for 770.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:464) 
hotplugStatusCallback /local/domain/0/backend/vbd/7/770/hotplug-status.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:478) 
hotplugStatusCallback 1.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices 
irq.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices 
pci.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices 
ioports.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices 
tap.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices 
vtpm.
[2007-10-30 11:57:03 xend 16192] INFO (XendDomain:370) Domain work (7) 
unpaused.
[12:26:51] $

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

<Prev in Thread] Current Thread [Next in Thread>