Hi
I'm using Xen on my AMD64 / Ubuntu 7.10 server which I recently upgraded, and
my 2 VMs are decidedly flaky. They seem to be blocked for most of the time,
and networking doesn't work properly. How can I debug this and get them
online reliably? Any tips gratefully received!
* I can log in via console or ssh, can ping the server's NICs but can't ping
the other VM or beyond the machine. From the server I can ping the VMs and
outside.
* Firewall settings on the server are very relaxed at this point.
* The exceptions below about hda1 and hda2 don't make sense to me, since the
devices are mounted via /etc/fstab as root and swap.
* /var/log/xen/xend-debug.log reports a broken pipe
Thanks!
M
uname string:
Linux server.home.xx 2.6.19-4-generic-amd64 #2 SMP Sat Oct 6 01:35:53 UTC 2007
x86_64 GNU/Linux
[11:57:03] $ sudo xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 782 1 r----- 487.2
slimserv 5 56 1 ------ 183.3
work 7 128 1 ------ 2.8
[11:57:06] $ sudo xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 782 1 r----- 488.3
slimserv 5 56 1 -b---- 185.6
work 7 128 1 -b---- 7.2
[11:57:20] $ sudo xm list
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 782 1 r----- 521.0
slimserv 5 56 1 ------ 267.3
work 7 128 1 -b---- 7.3
[12:19:09] $
Here is an excerpt from the xend-debug.log
Traceback (most recent call last):
File "SocketServer.py", line 464, in process_request_thread
self.finish_request(request, client_address)
File "SocketServer.py", line 254, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "SocketServer.py", line 522, in __init__
self.handle()
File "BaseHTTPServer.py", line 316, in handle
self.handle_one_request()
File "BaseHTTPServer.py", line 310, in handle_one_request
method()
File "/usr/lib/python2.5/site-packages/xen/util/xmlrpclib2.py", line 66, in
do_POST
self.send_response(200)
File "BaseHTTPServer.py", line 368, in send_response
(self.protocol_version, code, message))
File "socket.py", line 262, in write
self.flush()
File "socket.py", line 249, in flush
self._sock.sendall(buffer)
error: (32, 'Broken pipe')
Here is a tail excerpt from /var/log/xen/xend.log:
[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:190)
XendDomainInfo.create(['vm', ['name', 'work'], ['memory', '128'],
['on_poweroff', 'destroy'], ['on_reboot', 'restart'],
['on_crash', 'restart'], ['vcpus', 1], ['image', ['linux',
['kernel', '/boot/vmlinuz-2.6.19-4-generic-amd64'],
['ramdisk', '/boot/initrd.img-2.6.19-4-generic-amd64'], ['root', '/dev/hda1
ro']]], ['device', ['vbd', ['uname', 'file:/var/xen/domains/work/disk.img'],
['dev', 'hda1'], ['mode', 'w']]], ['device', ['vbd',
['uname', 'file:/var/xen/domains/work/swap.img'], ['dev', 'hda2'],
['mode', 'w']]], ['device', ['vif', ['ip', '192.168.2.6'],
['mac', '00:16:3e:b1:d7:91'], ['vifname', 'vif2.0']]]])
[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:296)
parseConfig: config is ['vm', ['name', 'work'], ['memory', '128'],
['on_poweroff', 'destroy'], ['on_reboot', 'restart'],
['on_crash', 'restart'], ['vcpus', 1], ['image', ['linux',
['kernel', '/boot/vmlinuz-2.6.19-4-generic-amd64'],
['ramdisk', '/boot/initrd.img-2.6.19-4-generic-amd64'], ['root', '/dev/hda1
ro']]], ['device', ['vbd', ['uname', 'file:/var/xen/domains/work/disk.img'],
['dev', 'hda1'], ['mode', 'w']]], ['device', ['vbd',
['uname', 'file:/var/xen/domains/work/swap.img'], ['dev', 'hda2'],
['mode', 'w']]], ['device', ['vif', ['ip', '192.168.2.6'],
['mac', '00:16:3e:b1:d7:91'], ['vifname', 'vif2.0']]]]
[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:395)
parseConfig: result is {'shadow_memory': None, 'uuid':
None, 'on_crash': 'restart', 'on_reboot': 'restart', 'localtime':
None, 'image': ['linux', ['kernel', '/boot/vmlinuz-2.6.19-4-generic-amd64'],
['ramdisk', '/boot/initrd.img-2.6.19-4-generic-amd64'], ['root', '/dev/hda1
ro']], 'on_poweroff': 'destroy', 'bootloader_args': None, 'cpus':
None, 'name': 'work', 'backend': [], 'vcpus': 1, 'cpu_weight':
None, 'features': None, 'vcpu_avail': None, 'memory': 128, 'device': [('vbd',
['vbd', ['uname', 'file:/var/xen/domains/work/disk.img'], ['dev', 'hda1'],
['mode', 'w']]), ('vbd', ['vbd',
['uname', 'file:/var/xen/domains/work/swap.img'], ['dev', 'hda2'],
['mode', 'w']]), ('vif', ['vif', ['ip', '192.168.2.6'],
['mac', '00:16:3e:b1:d7:91'], ['vifname', 'vif2.0']])], 'bootloader':
None, 'cpu': None, 'maxmem': None}
[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:1262)
XendDomainInfo.construct: None
[2007-10-30 11:57:01 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:1294)
XendDomainInfo.initDomain: 7 1.0
[2007-10-30 11:57:01 xend 16192] DEBUG (balloon:127) Balloon: 131680 KiB free;
need 131072; done.
[2007-10-30 11:57:01 xend 16192] INFO (image:136) buildDomain os=linux dom=7
vcpus=1
[2007-10-30 11:57:01 xend 16192] DEBUG (image:199) dom = 7
[2007-10-30 11:57:01 xend 16192] DEBUG (image:200) image
= /boot/vmlinuz-2.6.19-4-generic-amd64
[2007-10-30 11:57:01 xend 16192] DEBUG (image:201) store_evtchn = 1
[2007-10-30 11:57:01 xend 16192] DEBUG (image:202) console_evtchn = 2
[2007-10-30 11:57:01 xend 16192] DEBUG (image:203) cmdline =
root=/dev/hda1 ro
[2007-10-30 11:57:01 xend 16192] DEBUG (image:204) ramdisk
= /boot/initrd.img-2.6.19-4-generic-amd64
[2007-10-30 11:57:01 xend 16192] DEBUG (image:205) vcpus = 1
[2007-10-30 11:57:01 xend 16192] DEBUG (image:206) features =
[2007-10-30 11:57:02 xend 16192] DEBUG (blkif:24) exception looking up device
number for hda1: [Errno 2] No such file or directory: '/dev/hda1'
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:110) DevController:
writing
{'backend-id': '0', 'virtual-device': '769', 'device-type': 'disk', 'state':
'1', 'backend': '/local/domain/0/backend/vbd/7/769'}
to /local/domain/7/device/vbd/769.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:112) DevController:
writing
{'domain': 'work', 'frontend': '/local/domain/7/device/vbd/769', 'dev': 'hda1',
'state': '1', 'params': '/var/xen/domains/work/disk.img', 'mode': 'w',
'online': '1', 'frontend-id': '7', 'type': 'file'}
to /local/domain/0/backend/vbd/7/769.
[2007-10-30 11:57:02 xend 16192] DEBUG (blkif:24) exception looking up device
number for hda2: [Errno 2] No such file or directory: '/dev/hda2'
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:110) DevController:
writing
{'backend-id': '0', 'virtual-device': '770', 'device-type': 'disk', 'state':
'1', 'backend': '/local/domain/0/backend/vbd/7/770'}
to /local/domain/7/device/vbd/770.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:112) DevController:
writing
{'domain': 'work', 'frontend': '/local/domain/7/device/vbd/770', 'dev': 'hda2',
'state': '1', 'params': '/var/xen/domains/work/swap.img', 'mode': 'w',
'online': '1', 'frontend-id': '7', 'type': 'file'}
to /local/domain/0/backend/vbd/7/770.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:110) DevController:
writing
{'backend-id': '0', 'mac': '00:16:3e:b1:d7:91', 'handle': '0', 'state': '1',
'backend': '/local/domain/0/backend/vif/7/0'}
to /local/domain/7/device/vif/0.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:112) DevController:
writing
{'domain': 'work', 'handle': '0', 'script': '/etc/xen/scripts/vif-route', 'ip':
'192.168.2.6', 'state': '1', 'frontend': '/local/domain/7/device/vif/0', 'mac':
'00:16:3e:b1:d7:91', 'vifname': 'vif2.0', 'online': '1', 'frontend-id': '7'}
to /local/domain/0/backend/vif/7/0.
[2007-10-30 11:57:02 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:713)
Storing VM details:
{'shadow_memory': '0', 'uuid': '89708af1-f491-4aed-55e9-1be3acd352b7',
'on_reboot': 'restart', 'start_time': '1193741822.28', 'on_poweroff':
'destroy', 'name': 'work', 'xend/restart_count': '0', 'vcpus': '1',
'vcpu_avail': '1', 'memory': '128', 'on_crash': 'restart', 'image': "(linux
(kernel /boot/vmlinuz-2.6.19-4-generic-amd64)
(ramdisk /boot/initrd.img-2.6.19-4-generic-amd64) (root '/dev/hda1
ro'))", 'maxmem': '128'}
[2007-10-30 11:57:02 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:748)
Storing domain details:
{'console/ring-ref': '9890', 'console/port': '2', 'name': 'work',
'console/limit': '1048576', 'vm': '/vm/89708af1-f491-4aed-55e9-1be3acd352b7',
'domid': '7', 'cpu/0/availability': 'online', 'memory/target': '131072',
'store/ring-ref': '9891', 'store/port': '1'}
[2007-10-30 11:57:02 xend.XendDomainInfo 16192] DEBUG (XendDomainInfo:938)
XendDomainInfo.handleShutdownWatch
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:143) Waiting for devices
vif.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:149) Waiting for 0.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:464)
hotplugStatusCallback /local/domain/0/backend/vif/7/0/hotplug-status.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:464)
hotplugStatusCallback /local/domain/0/backend/vif/7/0/hotplug-status.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:478)
hotplugStatusCallback 1.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:143) Waiting for devices
usb.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:143) Waiting for devices
vbd.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:149) Waiting for 769.
[2007-10-30 11:57:02 xend 16192] DEBUG (DevController:464)
hotplugStatusCallback /local/domain/0/backend/vbd/7/769/hotplug-status.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:464)
hotplugStatusCallback /local/domain/0/backend/vbd/7/769/hotplug-status.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:478)
hotplugStatusCallback 1.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:149) Waiting for 770.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:464)
hotplugStatusCallback /local/domain/0/backend/vbd/7/770/hotplug-status.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:478)
hotplugStatusCallback 1.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices
irq.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices
pci.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices
ioports.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices
tap.
[2007-10-30 11:57:03 xend 16192] DEBUG (DevController:143) Waiting for devices
vtpm.
[2007-10-30 11:57:03 xend 16192] INFO (XendDomain:370) Domain work (7)
unpaused.
[12:26:51] $
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home