WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] xend leaks/bugs/etc

On Sun, 2005-04-17 at 16:42 +0100, Ian Pratt wrote: 

> Allen, I think we've come to the conclusion that Twisted was rather
> overkill for our needs, and led to some rather confusing code that has
> proved hard to maintain.

With all due respect, I work on 5 projects that use Twisted and,
overall, they're the easiest codebases to extend that I've dealt with.
xend's code is by far some of the worst Python code I've worked on.

>  I've no doubt that someone more experienced
> with using Twisted could have done a better job, but do you really think
> it's the best route forward?  Xend is a 'control plane' daemon and
> doesn't have to handle a high rate of invocations.

This is a point in favor of Twisted, I'd think; if you needed very high
performance in that area, Python might not be appropriate.

> It needs some ability to handle asynchronous or out-of-order events, but 
> this could be handled by simple language-level threads (we don't need 
> concurrency). 

Given the current architecture (a daemon that accepts connections from a
commandline tool or from a web interface), it would seem that you do
need concurrency; personally, I'd find it inconvenient if this was
handled differently. Plus, the languages that I'm familiar with that
provide language-level threads require at least as much
infrastructure/resource usage as Python.

> 
> The other downside of using Twisted is that its not available in some
> distros, and we've had a few issues with version mismatches.

Other projects (such as Chandler) have dealt with this by shipping
Twisted in their release tarballs. I believe this strategy would be
reasonable for Xen, especially now that Twisted has split some of its
less-used subprojects into separate packages. 


>  It also has quite an impact on the RSS memory footprint, which is not ideal.

I believe that the memory footprint can be significantly reduced; the
current codebase seems to have a good deal of unnecessary complexity.

> What do you think?

I think that there are probably some Xen deployments that would benefit
from a minimal-functionality, minimal-resource-usage control daemon, but
that they are not the only use case. The project that led to my interest
in Xen is a good example: I want to do dynamic auction-based resource
allocation to domains, a la Miller and Drexler's "Incentive Engineering
for Computational Resource Management". 
(http://www.agorics.com/Library/agoricpapers.html ) This would be most
easily achieved by putting my auction/resource-allocation code in the
same process as xend. Unfortunately its current implementation makes
that prohibitively difficult -- my current prototype uses the HTTP
interface, with some difficulty. 

Given these concerns -- greater flexibility, lower memory usage, more
comprehensible code -- I believe my best choice is to reimplement xend,
using the existing lowlevel xc and xu modules. I need it for the things
I want to write anyway, but hopefully enough configuration/UI
compatibility can be maintained for it to be useful to the community.

Allen

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel