single server 40 gbit/s disk to network streaming...ciena ome 6500 ciena ome 6500 sara ronald van...

SARA Computing and Networking Services P. O. Box 94613 1090 GP Amsterdam The Netherlands www.sara.nl [email protected] Single Server 40 Gbit/s Disk to Network Streaming End-to-End Performance A pre-rendered interactive visualization is streamed from 32 solid state disks on a single server in Amsterdam to a 61 Megapixel Tiled Panel Display in Geneva. By streamlining the application around system design of the I/O busses of the sending server, a disk-to-network throughput of over 34 Gbit/s (back-to-back) was achieved. Remote Scientific Visualization The visual content being streamed shows data from CosmoGrid, an astrophysical N-body simulation using 8,589,934,592 interacting particles. The simulation was performed on multiple supercomputers, including SARA's Huygens. Displaying on a tiled panel display allows researchers to see very small details, as well as a bird's eye view using a single display setup. IN COLLABORATION WITH: Sending Server Brand Supermicro X8DAH+-F CPU 2x Intel Xeon X5677 Memory 24 GiB DDR3 Disk XFS on 32x 160 GB Intel X25-M SSDs Controller 4x 3Ware 9750-8i Network 2x Myricom 10G-PCIE2-8B2-2S OS Linux 2.6.32 (64 bit) Tiled Panel Display Monitor 15x Dell 3008 WFP 30” LCD Frame 5x3 monitors Pixels 12800 x 4800 (61.44 Mpixel) PC Dell Precision T5500 Video card NVidia GeForce GTX 460 OS Linux 2.6.31 SARA has built a server with off-the-shelf components that is capable of transmitting well over 30 Gbit/s from disk to the network. Previously, clusters and grid setups were needed to reach these application speeds. This creates new possibilities for demanding applications. A systematic approach was used to tune the end-to-end performance, with studies on file systems, I/O system design, processor and interrupt affinity, link bonding and 40 Gbit/s interworking. The result is a remote visualization demonstration that shows how applications can benefit from an integrated approach towards end-to-end performance optimization. http://nrg.sara.nl/ Science Park Amsterdam Brussels 40 Gbit/s 1650 km Paris CERN, Geneva Ciena OME 6500 Ciena OME 6500 SARA Ronald van der Pol Freek Dijkstra Sander Boele Pieter de Boer Igor Idziejczak Walter de Jong Tijs de Kler Mark Meijerink Paul Melis Hanno Pet Mark van de Sanden Dennis Stam Peter Tavenier Paul Wielinga Extreme X650 40 Gbit/s Bliksem Supermicro X8DAH+-F NIC Intel 5520 DDR3 1333MHz Xeon 5677 Xeon 5677 Intel 5520 6.4 GT/s QPI PCI-E 2.0 16 X-25M SSD 2.4 TiByte 16 X-25M SSD 2.4 TiByte NIC Extreme X650 8x 10 Gbit/s 40 Gbit/s 4x 10 Gbit/s SURFnet / GigaPort 3, Cinegrid.nl, European Fund for Regional Development, Pieken in de Delta, Province of North-Holland, Municipality of Amsterdam Hier wordt geïnvesteerd in uw toekomst. Dit project wordt mede mogelijk gemaakt door het Europees Fonds voor Regionale Ontwikkeling Sponsored by: In collaboration with: CERN, Ciena, Extreme Networks, Leiden University, University of Amsterdam nrg.sara.nl

Upload: others

Post on 18-Jan-2021

3 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Page 1: Single Server 40 Gbit/s Disk to Network Streaming...Ciena OME 6500 Ciena OME 6500 SARA Ronald van der Pol Freek Dijkstra Sander Boele Pieter de Boer Igor Idziejczak Walter de Jong

SARA Computing and Networking Services

P. O. Box 946131090 GP AmsterdamThe Netherlands

www.sara.nl [email protected]

Single Server 40 Gbit/s Disk to Network Streaming

End-to-End PerformanceA pre-rendered interactive visualization is streamed from 32 solid state disks on a single server in Amsterdam to a 61 Megapixel Tiled Panel Display in Geneva. By streamlining the application around system design of the I/O busses of the sending server, a disk-to-network throughput of over 34 Gbit/s (back-to-back) was achieved.

Remote Scientifi c VisualizationThe visual content being streamed shows data from CosmoGrid, an astrophysical N-body simulation using 8,589,934,592 interacting particles. The simulation was performed on multiple supercomputers, including SARA's Huygens. Displaying on a tiled panel display allows researchers to see very small details, as well as a bird's eye view using a single display setup.

SPONSORED BY: IN COLLABORATION WITH:European Regional Development FundPieken in de DeltaProvincial Government of North-HollandMunicipality of Amsterdam

GigaPort3

Sending ServerBrand Supermicro X8DAH+-FCPU 2x Intel Xeon X5677Memory 24 GiB DDR3Disk XFS on 32x 160 GB Intel X25-M SSDsController 4x 3Ware 9750-8iNetwork 2x Myricom 10G-PCIE2-8B2-2SOS Linux 2.6.32 (64 bit)

Tiled Panel DisplayMonitor 15x Dell 3008 WFP 30” LCDFrame 5x3 monitorsPixels 12800 x 4800 (61.44 Mpixel)PC Dell Precision T5500Video card NVidia GeForce GTX 460OS Linux 2.6.31

SARA has built a server with off-the-shelf components that is capable of transmitting well over 30 Gbit/s from disk to the network. Previously, clusters and grid setups were needed to reach these application speeds. This creates new possibilities for demanding applications. A systematic approach was used to tune the end-to-end performance, with studies on fi le systems, I/O system design, processor and interrupt affi nity, link bonding and 40 Gbit/s interworking. The result is a remote visualization demonstration that shows how applications can benefi t from an integrated approach towards end-to-end performance optimization.

http://nrg.sara.nl/

Science Park Amsterdam Brussels 40 Gbit/s 1650 km Paris CERN, Geneva

Ciena OME6500

Science Park Amsterdam

SARARonald van der PolFreek DijkstraSander BoelePieter de BoerIgor IdziejczakWalter de JongTijs de KlerMark MeijerinkPaul MelisHanno PetMark van de SandenDennis StamPeter TavenierPaul Wielinga

Extreme X65040 Gbit/s

BliksemSupermicro

X8DAH+-F

NIC

Intel 5520

DDR3 1333MHz

Xeon 5677

Intel 5520

6.4 GT/s QPI

PCI-E 2.0

16 X-25M SSD

2.4 TiByte16 X-25M