Re: [Xen-devel] interrupted live migration kills Xen

[Tim Deegan <Tim.Deegan@xxxxxxxxxx>]

At 22:20 +0000 on 14 Dec (1292365230), Olaf Hering wrote:
> 
> Should an interrupted live migration of a HVM guest crash Xen?
> I'm debugging live migration and made a mistake, so xc_save crashed in the
> middle of the operation. During guest shutdown I got the backtrace below. The
> same ASSERT is still in paging_free_log_dirty_bitmap() in xen-unstable.
> Is the ASSERT correct, or should it be removed?

The ASSERT is morally correct but clearly not actually correct. :) 
The bug, I think, is that this function doesn't hold the log-dirty
lock, so it can race with another CPU calling mark_dirty. 

I'll send a patch in a few minutes to fix that.

Tim.

> [  863.693653] br0: port 2(vif1.0) entering disabled state
> [  863.725132] br0: port 2(vif1.0) entering disabled state
> (XEN) Assertion 'd->arch.paging.log_dirty.allocs == 0' failed at paging.c:201
> (XEN) Debugging connection not set up.
> (XEN) ----[ Xen-4.0.1_01-20101214.214555  x86_64  debug=y  Tainted:    C ]----
> (XEN) CPU:    3
> (XEN) RIP:    e008:[<ffff82c4801d078b>] 
> paging_free_log_dirty_bitmap+0x237/0x260
> (XEN) RFLAGS: 0000000000010206   CONTEXT: hypervisor
> (XEN) rax: 0000000000000000   rbx: ffff83000f11c000   rcx: ffff82c4804a8fe0
> (XEN) rdx: ffff82f6001e1380   rsi: 0000000000000000   rdi: ffff82c4802520b8
> (XEN) rbp: ffff83013fed7d28   rsp: ffff83013fed7cb8   r8:  ffffffffffffffff
> (XEN) r9:  0000000000000020   r10: 0000000000000001   r11: 0000000000000001
> (XEN) r12: ffff8301385e0000   r13: ffff83000f11c000   r14: ffff8301385e0000
> (XEN) r15: ffff83000f11d000   cr0: 0000000080050033   cr4: 00000000000026f0
> (XEN) cr3: 0000000132b29000   cr2: 0000000000c2e030
> (XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e010   cs: e008
> (XEN) Xen stack trace from rsp=ffff83013fed7cb8:
> (XEN)    ffff83000f11e000 ffff830000001000 ffff830000000000 000004ffffffffff
> (XEN)    ffff830000001000 ffff83000f11d000 ffff83000f11dff8 ffff83000f11cff8
> (XEN)    ffff83000f11e000 ffff83013fed7f28 ffff8301385e0000 ffff8301385e0f30
> (XEN)    0000000000000000 00007f7c0f715830 ffff83013fed7d58 ffff82c4801d0897
> (XEN)    0000000000000000 ffff8301385e0000 00000000ffffffea ffff8301385e0000
> (XEN)    ffff83013fed7d98 ffff82c48015719c ffff83013fed7d98 ffff8301385e0000
> (XEN)    00000000ffffffea ffff8800ec795e58 0000000000000000 00007f7c0f715830
> (XEN)    ffff83013fed7db8 ffff82c4801070cf fffffffffffffff3 0000000001268000
> (XEN)    ffff83013fed7f08 ffff82c4801058ed ffff83013fed7de8 ffff82c48015233c
> (XEN)    ffff83013fe60000 0000000000000004 ffff83013fed7df8 ffff82c480152375
> (XEN)    ffff83013fed7e28 ffff82c4801083ea 0000000000000292 ffff83013fe60000
> (XEN)    0000000000000004 ffff83013fe60190 ffff83013fed7e58 ffff82c4801088b1
> (XEN)    ffff83013fed7e68 ffff82c48012421f 0000000700000002 00000000011f0001
> (XEN)    0000000000cdaff9 0000000000d853a0 00007f7c1a6a4608 00007f7c1a405457
> (XEN)    0000003000000018 00007f7c0f7158b0 00007f7c0f7157f0 000000000120ab40
> (XEN)    00007f7c1a6a4608 0000000000000000 00007f7c0f7158c4 0000000000000000
> (XEN)    0000000000f2a468 00007f7c1a690380 0000000000000000 0000000000e55a10
> (XEN)    0000000000000000 ffff8300bf4ea000 ffff8800f1531c00 ffff8800ec795e58
> (XEN)    0000000000000000 00007f7c0f715830 00007cfec01280b7 ffff82c4802091bf
> (XEN)    ffffffff8000348a 0000000000000024 00007f7c0f715830 0000000000000000
> (XEN) Xen call trace:
> (XEN)    [<ffff82c4801d078b>] paging_free_log_dirty_bitmap+0x237/0x260
> (XEN)    [<ffff82c4801d0897>] paging_teardown+0xe3/0x13f
> (XEN)    [<ffff82c48015719c>] domain_relinquish_resources+0x6b/0x23f
> (XEN)    [<ffff82c4801070cf>] domain_kill+0x93/0xe7
> (XEN)    [<ffff82c4801058ed>] do_domctl+0xa97/0x12aa
> (XEN)    [<ffff82c4802091bf>] syscall_enter+0xef/0x149
> (XEN)    
> (XEN) 
> (XEN) ****************************************
> (XEN) Panic on CPU 3:
> (XEN) Assertion 'd->arch.paging.log_dirty.allocs == 0' failed at paging.c:201
> (XEN) ****************************************
> (XEN) 
> (XEN) Reboot in five seconds...
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

-- 
Tim Deegan <Tim.Deegan@xxxxxxxxxx>
Principal Software Engineer, Xen Platform Team
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel