Re: [Xen-devel] myrinet dma

[Tim Freeman <tfreeman@xxxxxxxxxxx>]

Thankyou for your comments, they are very helpful.  Your reply makes it
sound like some of these options could be feasible which means I will
put everyone's comments to the folks with the cluster in question.

Thankyou for everyone's help!

On Fri, 27 Aug 2004 16:14:58 +0100
Mark Williamson <Mark.Williamson@xxxxxxxxxxxx> wrote:

> IMO, this is roughly what would need to be done:
> 
> Direct data path:
> The OS component would have to be modified so that in dom0 it would perform 
> the usual tasks of pinning memory AND talking to the hardware but in 
> unprivileged domains it would pin memory itself and then request that dom0 
> set 
> up the hardware.  This is control path, not data path so the indirection 
> shouldn't hurt performance - guest applications can talk to the hardware 
> directly.
> 
> It may be possible to use an existing library as-is, I'm not sure.
> 
> Writing the code to do this should be quite tractable for someone with the 
> appropriate experience.  I'd imagine that user applications would receive 
> similar performance to in non-virtualised configurations, with the 
> qualification that if you run lots of domains on one CPU, they will obviously 
> tend to experience less CPU time and higher latency anyway.
> 
> This approach limits you to no more clients than you have channels.
> 
> Multiplexed data path:
> Multiplexing multiple guests onto single a channel seems a bit more 
> difficult. 
>  Perhaps it could be done with modifications to allow dom0 to control the 
> channel, with other domains requesting data path as well as control path 
> operations from it.  This could still give zero copy into guest applications 
> but there might be some performance hit in latency due to the extra level of 
> indirection, although suitable pipelining may provide good bandwith (as for 
> the existing net and block drivers).
> 
> This  would be more work to implement than direct data path.  I guess there's 
> also the possibility that your next interface might have lots of channels, 
> making such multiplexing less important...
> 
> > Do your plans for infiniband allow 100s of guests to each have high speed
> > networking?  How much might the performance degrade?
> 
> Simply having plenty of channels on the host interface card would be more 
> straightforward than sharing them, see the above comment for the direct data 
> path.
> 
> I don't personally know what is planned regarding infiniband support, though.
> 
> > If I'm thinking about this correctly, it sounds like all of these domains'
> > traffic could be put onto one Myrinet channel and five special domains
> > could truly take advantage of Myrinet?
> 
> As for the issue of multiplexing some domains onto an ethernet-type interface 
> and having some privileged domains also accessing the card directly, yes this 
> sounds plausible in the first scenario described above (control-path 
> multiplexing with direct data-path).
> 
> Just my $0.02
> Mark
> 


-- 


-------------------------------------------------------
This SF.Net email is sponsored by BEA Weblogic Workshop
FREE Java Enterprise J2EE developer tools!
Get your free copy of BEA WebLogic Workshop 8.1 today.
http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/xen-devel