Hi,
* Olivier Hanesse <olivier.hanesse@xxxxxxxxx> [20101111 06:25]:
> Hello,
>
> Did you still got some delays for the domUs network to come up though ?
> I got some, around 90s ... After that delay, everything is working fine.
The migration itself takes less than 60s im my tests so far and I can
connect to the console immediately (or over ssh).
I noticed however that sometimes after the migration is done the domUs
network just hang, ie from the xend.log I see the migration is
completed but the DomU network hangs or doesn't start, taking
sometimes very very long times to emerges --hours sometimes-- if it
does. I know for a fact that it's up as the uptime on the domU when it
finally resurfaces shows it. As you have remarked before , it seems to
sort of depend on the uptime of the dom0s, but I haven't any
conclusive evidence of that yet.
I have done tcpdumps of the network traffic between the dom0s while
migrating but nothing jumps out of the ordinary: I thought that a
gratuitous arp request was sent by the domU? Is this still the case? I
haven't seen any...
Right now both dom0s have 5+ days of uptime and things work
flawlessly. Tomorrow, I'll reboot one of them and see if I can
reproduce my earlier problems.
Thank you for your time!
jf
>
> any ideas ?
>
> Regards
>
> 2010/11/3 Jean-Francois Malouin <Jean-Francois.Malouin@xxxxxxxxxxxxxxxxx>
>
> > * Jean-Francois Malouin <Jean-Francois.Malouin@xxxxxxxxxxxxxxxxx>
> > [20101102 16:41]:
> > > Replying to myself,
> >
> > Continuing my soliloque,
> >
> > Updated Dom0 kernel to debian/squeeze to 2.6.32-27 (from -26)
> > Looks like this resolves the save/migration issue.
> > Still experiencing some delays for the domUs network to come up though...
> >
> > jf
> >
> > >
> > > * Jean-Francois Malouin <Jean-Francois.Malouin@xxxxxxxxxxxxxxxxx>
> > [20101102 14:53]:
> > > >
> > > > Hi,
> > > >
> > > > In view of the problems I was having with DomU network timeout after a
> > > > live migration (I posted that problems here a while ago but never got
> > > > anything except from private emails) I finally updated my
> > > > Debian/Squeeze Dom0s last night to a new kernel, from 2.6.32-23 to
> > > > 2.6.32-26.
> > > >
> > > > Now live migration just hangs...Any ideas?
> > >
> > > Forgot to add the kernel log from the DomU (same kernel as dom0)
> > >
> > > [10765.336002] BUG: soft lockup - CPU#0 stuck for 61s! [xenwatch:16]
> > > [10765.336002] Modules linked in: xt_tcpudp iptable_filter ip_tables
> > x_tables snd_pcm snd_timer snd soundcore snd_page_alloc evdev pcspkr ext3
> > jbd mbcache dm_mod raid1 md_mod xen_netfront xen_blkfront
> > > [10765.336002] CPU 0:
> > > [10765.336002] Modules linked in: xt_tcpudp iptable_filter ip_tables
> > x_tables snd_pcm snd_timer snd soundcore snd_page_alloc evdev pcspkr ext3
> > jbd mbcache dm_mod raid1 md_mod xen_netfront
> > > [10765.336002] Pid: 16, comm: xenwatch Not tainted 2.6.32-5-xen-amd64 #1
> > > [10765.336002] RIP: e030:[<ffffffff81068705>] [<ffffffff81068705>]
> > lock_hrtimer_base+0x3a/0x3c
> > > [10765.336002] RSP: e02b:ffff8800ffaadd70 EFLAGS: 00000246
> > > [10765.336002] RAX: ffff880002a28680 RBX: 0000000000000000 RCX:
> > 0000000000000006
> > > [10765.336002] RDX: ffff8800fcd70850 RSI: ffff8800ffaadda0 RDI:
> > ffff880002a2f820
> > > [10765.336002] RBP: ffff880002a2f820 R08: 0000000000000000 R09:
> > 0000000000000000
> > > [10765.336002] R10: ffff8800fcd70c50 R11: ffffffff8122b649 R12:
> > ffff8800ffaadda0
> > > [10765.336002] R13: 0000000000000002 R14: ffff8800fcd708a0 R15:
> > ffff8800ffaaddf0
> > > [10765.336002] FS: 00007fedc5e08710(0000) GS:ffff8800039bf000(0000)
> > knlGS:0000000000000000
> > > [10765.336002] CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> > > [10765.336002] CR2: 00007fae1857c120 CR3: 0000000002fe3000 CR4:
> > 0000000000002660
> > > [10765.336002] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> > 0000000000000000
> > > [10765.336002] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> > 0000000000000400
> > > [10765.336002] Call Trace:
> > > [10765.336002] [<ffffffff8106875b>] ? hrtimer_try_to_cancel+0x16/0x43
> > > [10765.336002] [<ffffffff8122b649>] ? serial8250_suspend+0x0/0x48
> > > [10765.336002] [<ffffffff81068794>] ? hrtimer_cancel+0xc/0x16
> > > [10765.336002] [<ffffffffa0009147>] ? netfront_suspend+0x19/0x1d
> > [xen_netfront]
> > > [10765.336002] [<ffffffff811f569b>] ? xenbus_dev_suspend+0x1f/0x3b
> > > [10765.336002] [<ffffffff81233872>] ? dpm_suspend_start+0x359/0x45b
> > > [10765.336002] [<ffffffff811f2ca0>] ? shutdown_handler+0x15f/0x25c
> > > [10765.336002] [<ffffffff8130b475>] ? mutex_lock+0xd/0x31
> > > [10765.336002] [<ffffffff811f47ad>] ? xenwatch_thread+0x117/0x14a
> > > [10765.336002] [<ffffffff81065afe>] ? autoremove_wake_function+0x0/0x2e
> > > [10765.336002] [<ffffffff811f4696>] ? xenwatch_thread+0x0/0x14a
> > > [10765.336002] [<ffffffff81065831>] ? kthread+0x79/0x81
> > > [10765.336002] [<ffffffff81012baa>] ? child_rip+0xa/0x20
> > > [10765.336002] [<ffffffff81011d61>] ? int_ret_from_sys_call+0x7/0x1b
> > > [10765.336002] [<ffffffff8101251d>] ? retint_restore_args+0x5/0x6
> > > [10765.336002] [<ffffffff8102ddac>] ?
> > pvclock_clocksource_read+0x3a/0x8b
> > > [10765.336002] [<ffffffff81012ba0>] ? child_rip+0x0/0x20
> > >
> > > googling digged this:
> > >
> > >
> > http://www.linux-archive.org/debian-kernel/442963-bug-600992-further-logs.html
> > >
> > > which pretty much what I'm seeing on my systems.
> > > jf
> > >
> > >
> > > >
> > > > Xen-related Debian packages (all from Debian repositary, except drbd
> > > > which is from linbit):
> > > >
> > > > ~# dpkg -l \*xen\* | grep ^i
> > > > drbd8-module-2.6.32-5-xen-amd64 2:8.3.8-0+2.6.32-26 RAID 1 over
> > tcp/ip for Linux kernel module
> > > > libxenstore3.0 4.0.1-1
> > Xenstore communications library for Xen
> > > > linux-headers-2.6.32-5-common-xen 2.6.32-26
> > Common header files for Linux 2.6.32-5-xen
> > > > linux-headers-2.6.32-5-xen-amd64 2.6.32-26
> > Header files for Linux 2.6.32-5-xen-amd64
> > > > linux-image-2.6.32-5-xen-amd64 2.6.32-26 Linux
> > 2.6.32 for 64-bit PCs, Xen dom0 support
> > > > xen-hypervisor-4.0-amd64 4.0.1-1 The Xen
> > Hypervisor on AMD64
> > > > xen-qemu-dm-4.0 4.0.1-1 Xen
> > Qemu Device Model virtual machine hardware emulator
> > > > xen-tools 4.2-1 Tools to
> > manage Xen virtual servers
> > > > xen-utils-4.0 4.0.1-1 XEN
> > administrative tools
> > > > xen-utils-common 4.0.0-1 XEN
> > administrative tools - common files
> > > > xenstore-utils 4.0.1-1
> > Xenstore utilities for Xen
> > > > xenwatch 0.5.4-2
> > Virtualization utilities, mostly for Xen
> > > >
> > > > ~# xm info
> > > > host : dom0
> > > > release : 2.6.32-5-xen-amd64
> > > > version : #1 SMP Wed Oct 20 02:22:18 UTC 2010
> > > > machine : x86_64
> > > > nr_cpus : 8
> > > > nr_nodes : 2
> > > > cores_per_socket : 4
> > > > threads_per_core : 1
> > > > cpu_mhz : 2000
> > > > hw_caps :
> > bfebfbff:28100800:00000000:00001b40:009ce3bd:00000000:00000001:00000000
> > > > virt_caps : hvm hvm_directio
> > > > total_memory : 12279
> > > > free_memory : 3934
> > > > node_to_cpu : node0:0-3
> > > > node1:4-7
> > > > node_to_memory : node0:1996
> > > > node1:1938
> > > > node_to_dma32_mem : node0:1995
> > > > node1:0
> > > > max_node_id : 1
> > > > xen_major : 4
> > > > xen_minor : 0
> > > > xen_extra : .1
> > > > xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
> > hvm-3.0-x 86_32p hvm-3.0-x86_64
> > > > xen_scheduler : credit
> > > > xen_pagesize : 4096
> > > > platform_params : virt_start=0xffff800000000000
> > > > xen_changeset : unavailable
> > > > xen_commandline : dom0_mem=2048M dom0_max_vcpus=2 loglvl=all
> > guest_loglvl=all console=tty0
> > > > cc_compiler : gcc version 4.4.5 20100824 (prerelease)
> > (Debian 4.4.4-11)
> > > > cc_compile_by : waldi
> > > > cc_compile_domain : debian.org
> > > > cc_compile_date : Fri Sep 3 15:38:12 UTC 2010
> > > > xend_config_format : 4
> > > >
> > > > thanks
> > > > jf
> > > > --
> > > > <° >< Jean-François Malouin McConnell Brain Imaging Centre
> > > > Systems/Network Administrator Montréal Neurological Institute
> > > > 3801 Rue University, Suite WB219 Montréal, Québec, H3A 2B4
> > > > Phone: 514-398-8924 Fax: 514-398-8948
> > > >
> > > > _______________________________________________
> > > > Xen-users mailing list
> > > > Xen-users@xxxxxxxxxxxxxxxxxxx
> > > > http://lists.xensource.com/xen-users
> > >
> > > _______________________________________________
> > > Xen-users mailing list
> > > Xen-users@xxxxxxxxxxxxxxxxxxx
> > > http://lists.xensource.com/xen-users
> >
> > --
> > <° >< Jean-François Malouin McConnell Brain Imaging Centre
> > Systems/Network Administrator Montréal Neurological Institute
> > 3801 Rue University, Suite WB219 Montréal, Québec, H3A 2B4
> > Phone: 514-398-8924 Fax: 514-398-8948
> >
> > _______________________________________________
> > Xen-users mailing list
> > Xen-users@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-users
> >
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home