This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


[Xen-users] RE: [ofa-general] Infiniband performance

To: "Jan Ruffing" <ruffing@xxxxxxxxxx>
Subject: [Xen-users] RE: [ofa-general] Infiniband performance
From: "Robert Dunkley" <Robert@xxxxxxxxx>
Date: Fri, 12 Dec 2008 08:23:14 -0000
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Delivery-date: Fri, 12 Dec 2008 00:27:07 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <9FA59C95FFCBB34EA5E42C1A8573784F017A4199@xxxxxxxxxxxxxxxxx> <C1EAC9C5E752D24C968FF091D446D8232DEBBE@ALTERNATEREALIT> <49413D20.1040805@xxxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: Aclbq4+0qE7vBMBNQ1SwL8pzxY7GvgAhPG7w
Thread-topic: [ofa-general] Infiniband performance
Hi Jan,

I asked almost the exact same question as you about 6 months ago and someone 
provided some Gen4 results for me (But I can't seem to find them in email), 
they were a fair bit better than Gen3. 

With IPOIB, you want connected mode and a large 32Kb+ MTU for max bandwidth, 
1Gbyte/sec or more should be possible with Gen 4.

Here are some of my original test results on my Opteron System with Gen3 cards:

#qperf tcp_bw tcp_lat
     bw  =  866 MB/sec
     latency  =  30.8 us

# qperf rc_bi_bw
    bw  =  1.71 GB/sec

The second result is bi directional, Gen4 on PCI-E V2 should get quite a lot 


-----Original Message-----
From: Jan Ruffing [mailto:ruffing@xxxxxxxxxx] 
Sent: 11 December 2008 16:18
To: Robert Dunkley
Cc: Gilad Shainer; general@xxxxxxxxxxxxxxxxxxxxx
Subject: Re: [ofa-general] Infiniband performance

Robert Dunkley wrote:
> The newer gen4 20Gb cards get about 14-15Gbit/sec because of their better 
> lower latency processing and support for higher bandwidth PCI-E 2.0 (PCI-E is 
> not 100% efficient so a 20Gbit PCIE connect will hold back a 20Gbit IC).
Thank you.
What TCP bandwidth did you get with the gen4 cards?


Jan Ruffing
Software Developer

Motama GmbH
Lortzingstraße 10 · 66111 Saarbrücken · Germany
tel +49 681 940 85 50 · fax +49 681 940 85 49
ruffing@xxxxxxxxxx · www.motama.com

Companies register · district council Saarbrücken · HRB 15249
CEOs · Dr.-Ing. Marco Lohse, Michael Repplinger

This e-mail may contain confidential and/or privileged information. 
If you are not the intended recipient (or have received this e-mail 
in error) please notify the sender immediately and destroy this 
e-mail. Any unauthorized copying, disclosure or distribution of the 
material in this e-mail is strictly forbidden.

The SAQ Group

Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ
SAQ is the trading name of SEMTEC Limited. Registered in England & Wales
Company Number: 06481952

http://www.saqnet.co.uk AS29219

SAQ Group Delivers high quality, honestly priced communication and I.T. 
services to UK Business.

Broadband : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : 
Backups : Managed Networks : Remote Support.

ISPA Member

Find us in http://www.thebestof.co.uk/petersfield

Xen-users mailing list

<Prev in Thread] Current Thread [Next in Thread>
  • [Xen-users] RE: [ofa-general] Infiniband performance, Robert Dunkley <=