Re: [Xen-devel] xend segfaults when starting

[Ian Campbell <Ian.Campbell@xxxxxxxxxxxxx>]

Thanks for the analysis. I'm a bit confused though.

On Wed, 2010-08-18 at 11:44 +0100, Christoph Egger wrote:

> I tracked down where the error happens. In safe_munlock(),
> the munlock() fails.
> 
> The trace is:
> 
> xc_interface_close -> _xc_clean_hcall_buf -> unlock_pages -> safe_munlock -> 
> munlock
> 
> hcall_buf->buf has the address 0x7f7ffdfe7040

Mustn't this be page aligned, due to 
        hcall_buf->buf = xc_memalign(PAGE_SIZE, PAGE_SIZE);
?

This appears to turn into valloc on NetBSD which (at least according to
the Linux manpages) returns a page aligned result.

> In unlock_pages, the address and length passed to munlock() is:
> 
>  laddr 0x7f7ffdfe7000, llen 0x2000

> The reason why munlock() fails is that mlock() hasn't been called before.
> The hcall_buf_prep() is not called at all before the first call to 
> _xc_clean_hcall_buf().

If hcall_buf_prep() has never been called then
"pthread_getspecific(hcall_buf_pkey)" should return NULL and
_xc_clean_hcall_buf will never be called from xc_clean_hcall_buf.
_xc_clean_hcall_buf also ignores NULL values itself.

However you say that hcall_buf_pkey is not NULL, but rather contains a
valid hcall_buf containing 0x7f7ffdfe7040. The only call to
"pthread_setspecific(hcall_buf_pkey, ...)" with a non-NULL value is in
hcall_buf_prep(), so it must have been called at some point.

Please can you confirm if _xc_init_hcall_buf() is ever called and what
the behaviour of "pthread_getspecific(hcall_buf_pkey)" is if
_xc_init_hcall_buf() has never been called. I think it is supposed to
return NULL in this case and we certainly rely on that.

pthread_getspecific(hcall_buf_pkey) is supposed to return NULL on error,
however hcall_buf_pkey is uninitialised until _xc_init_hcall_buf,
perhaps on NetBSD the uninitialised value somehow looks valid? It's not
clear what the correct value to initialise a pthread_key_t to in order
for it to appear invalid until it is properly setup is, but I suppose we
should be initialising it before use. Please can you try this patch:

diff -r 7e4d798e8726 tools/libxc/xc_private.c
--- a/tools/libxc/xc_private.c  Wed Aug 18 11:29:35 2010 +0100
+++ b/tools/libxc/xc_private.c  Wed Aug 18 13:12:41 2010 +0100
@@ -232,17 +236,19 @@ static void _xc_clean_hcall_buf(void *m)
     pthread_setspecific(hcall_buf_pkey, NULL);
 }
 
+static void _xc_init_hcall_buf(void)
+{
+    pthread_key_create(&hcall_buf_pkey, _xc_clean_hcall_buf);
+}
+
 static void xc_clean_hcall_buf(void)
 {
     void *hcall_buf = pthread_getspecific(hcall_buf_pkey);
 
+    pthread_once(&hcall_buf_pkey_once, _xc_init_hcall_buf);
+
     if (hcall_buf)
         _xc_clean_hcall_buf(hcall_buf);
-}
-
-static void _xc_init_hcall_buf(void)
-{
-    pthread_key_create(&hcall_buf_pkey, _xc_clean_hcall_buf);
 }
 
 int hcall_buf_prep(void **addr, size_t len)


If that doesn't work perhaps you can reduce the issue to a simple test
case like the attached? (which doesn't reproduce the issue for me on
Linux) If you can do that then please run it with the attached libxc
patch and post the output.

Ian.

Attachment: libxc-mlock-debug.patch
Description: Text Data

Attachment: mlock.c
Description: Text Data

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel