This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-devel] [PATCH] Ensure FD_CLOEXEC is set on all XenD file handles

To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] [PATCH] Ensure FD_CLOEXEC is set on all XenD file handles
From: "Daniel P. Berrange" <berrange@xxxxxxxxxx>
Date: Tue, 15 Aug 2006 02:23:57 +0100
Delivery-date: Mon, 14 Aug 2006 18:24:34 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Reply-to: "Daniel P. Berrange" <berrange@xxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.4.1i
While debugging an issue with libvirt I discovered a serious problem
with XenD's management of file handles. Basically it does not set the
close-on-exec flag on any[1] of the file handles it has open. This means
that all XenD's file handles propagated to programs it spawns - eg, the
network device setup scripts, qemu-dm and others those in turn spawn.

The particular case that lead me to discover the problem was when I disabled
HTTP port on XenD and wondered why port 8000 was still in LISTEN state - it
turned out qemu-dm had a handle to the server socket. Aside from this it had
access to about 10 open handles on /proc/xen/privcmd, the HTTP *client*
connection which requested the domain creation[2], and any of the other
server sockets XenD happened to have open.

I'm attaching three prototype patches - I say prototype because there may
be other (better) places to set the FD_CLOEXEC flag - I've just picked
the place closest to the original socket()/accept() call.

 * xen-xend2-cloexec.patch - set the flag on the XMLRPC & HTTP server
   ports, both TCP & UNIX domain socket versions. Also ensure incoming
   clients have it set
 * xen-xc-cloexec.patch - set the flag on all handles to /proc/xen/privcmd
 * xen-xs2-cloexec.patch - set the flag on all connections to the
   xenstore daemon.

The only file handle I've not yet set the flag on is the one to the log
file /var/log/xend.log which (in my system) ends up on FD #9.

To test which file handles were being left open I created a dummy script:

  $ cat > /root/fake-qemu-dm.sh <<EOF

  lsof -P -n -p $$ >> /tmp/qemu-fds.log

  exec /usr/lib64/xen/bin/qemu-dm "$@"

And then in the domain's config file set 'device_model' to point to the
file '/root/fake-qemu-dm.sh'

BTW, the patches were prepared against the latest Xen userspace code in
Fedora Core 6, test2 - this is trailing xen-unstable by a couple of weeks
but I think they should still apply. If people agree with the approach
taken in the patch I'll re-diff against xen-unstable before posting again.


[1] Actually it turns out the relocation port does have the flag set
[2] This explains an issue discovered with libvirt where it would hang 
     forever after creating an HVM domain waiting for the server to close
    its end of the socket
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

Attachment: xen-xc-cloexec.patch
Description: Text document

Attachment: xen-xend2-cloexec.patch
Description: Text document

Attachment: xen-xs2-cloexec.patch
Description: Text document

Xen-devel mailing list
<Prev in Thread] Current Thread [Next in Thread>