Hello,
The last few days i did some further research.
It seems i have run into some limit for the Xen grant tables (although,
that was what people told me).
I tried to replicate the error so i set up a new server. Since the
machine wich is having the troubles is a pretty heavy machine (64GB ram)
i could not exactly replicate it.
So i set up a new server with 6GB ram, but with exactly the same
versions of Xen and i was NOT able to reproduce the error.
I created 115 domU's (with 48MB of memory) and they all had network
connectivity.
So i started reading more about the Grant tables and found out that also
vbd's are mapped in these tables.
For the test i used this config for the domU:
*****************************************************************
kernel = '/boot/vmlinuz-2.6.24-22-xen'
ramdisk = '/boot/initrd.img-2.6.24-22-xen'
memory = 512
vcpus = 1
root = '/dev/xvda1 ro'
disk = [
'phy:/dev/xen-domains-root/vps78-root,xvda1,w'
'phy:/dev/xen-domains-swap/vps78-swap,xvda2,w'
]
name = 'vps78'
vif = [ 'mac=00:16:3e:12:de:e4,vifname=vps78,bridge=vlanbr710' ]
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
extra='xencons=tty1 rootflags=quota'
*****************************************************************
This did NOT work, it resulted in the following message:
vif vif-279-0: 28 mapping shared-frames 768/769 port 8
After searching in drivers/xen/interface.c i found out that it was bad,
it is a error with code 28.
As i ran out of options i remove the xvda2 line from my domU's config
and i'll be damned, the network connectivity was back!
I created a new domU (#80) and this had the same troubles, no network.
But when i removed the xvda2 line (for the swap) the network was OK.
So for the last test i added the swap after it was created:
root@vps-pool-01:~# xm block-attach
vps78 /dev/xen-domains-swap/vps78-swap xvda2 w
root@vps-pool-01:~# dmesg
[2540234.810635] blkback: ring-ref 299, event-channel 8, protocol 1
(x86_64-abi)
[2540234.810813] vbd vbd-280-51714: 28 mapping ring-ref 299 port 8
root@vps-pool-01:~#
As you can see i have the same kind of error, mapping the vbd fails.
Today i ordered a new server wich is exactly the same as the machine
wich is running into this limit. So i will be able to reproduce a exact
setup and hopefully reproduce the errors as well.
I hope to receive this server within one week.
But in the meantime i hope their is somebody who has any clues of what
this could be.
root@vps-pool-01:~# xm info
host : vps-pool-01.xen.pcextreme.nl
release : 2.6.24-22-xen
version : #1 SMP Mon Nov 24 21:35:54 UTC 2008
machine : x86_64
nr_cpus : 8
nr_nodes : 1
cores_per_socket : 4
threads_per_core : 1
cpu_mhz : 2500
hw_caps :
bfebfbff:20100800:00000000:00000140:040ce3bd:00000000:00000001
total_memory : 64766
free_memory : 26797
max_free_memory : 26797
max_para_memory : 26793
max_hvm_memory : 26722
node_to_cpu : node0:0-7
xen_major : 3
xen_minor : 2
xen_extra : .1-rc1-pre
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
cc_compiler : gcc version 4.2.3 (Ubuntu 4.2.3-2ubuntu7)
cc_compile_by : buildd
cc_compile_domain : buildd
cc_compile_date : Fri Apr 11 01:13:51 UTC 2008
xend_config_format : 4
root@vps-pool-01:~#
Thank you in advance!
-
Met vriendelijke groet,
Wido den Hollander
Hoofd Systeembeheer / CSO
Telefoon Support Nederland: 0900 9633 (45 cpm)
Telefoon Support België: 0900 70312 (45 cpm)
Telefoon Direct: (+31) (0)20 50 60 104
Fax: +31 (0)20 50 60 111
E-mail: support@xxxxxxxxxxxx
Website: http://www.pcextreme.nl
Kennisbank: http://support.pcextreme.nl/
Netwerkstatus: http://nmc.pcextreme.nl
On Thu, 2009-03-12 at 18:47 +0100, PCextreme B.V. - Wido den Hollander
wrote:
> Hello,
>
> Today i ran into a problem with new domU's on my Xen 3.2 system.
>
> On this server i have 78 domU's running, but when creating domU #79 the
> network for this domU fails.
>
> I allready submitted a bugreport with Ubuntu and please forgive me for
> just posting a link:
> https://bugs.launchpad.net/ubuntu/+source/xen-meta/+bug/341846
>
> Since this machine is already in production i can't simply reboot or
> build a new kernel.
>
> Could somebody give me a clue where to find the solution?
>
> -
> Met vriendelijke groet,
>
> Wido den Hollander
> Hoofd Systeembeheer / CSO
> Telefoon Support Nederland: 0900 9633 (45 cpm)
> Telefoon Support België: 0900 70312 (45 cpm)
> Telefoon Direct: (+31) (0)20 50 60 104
> Fax: +31 (0)20 50 60 111
> E-mail: support@xxxxxxxxxxxx
> Website: http://www.pcextreme.nl
> Kennisbank: http://support.pcextreme.nl/
> Netwerkstatus: http://nmc.pcextreme.nl
>
signature.asc
Description: This is a digitally signed message part
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home