o ak r idge n ational l aboratory u.s. d epartment of e nergy probe plans and status scidac kickoff...

OAK RIDGE NATIONAL LABORATORYU.S. DEPARTMENT OF ENERGY

Probe Plans and StatusSciDAC Kickoff

July, 2001

Dan Million

Randy Burris

ORNL, Center for Computational Sciences


Overview Proposal’s plans for Probe involvement HPSS overview Probe as a “place to be” SciDAC applications with Probe connections Technology on hand


Plans as stated in the proposal Five research thrusts

High-bandwidth transfers of very large datasets• Optimize blocksizes and configurations

• TCP-friendly UDP service

• Test AIX 5L for improvements in TCP/IP stack

• Evaluate ATM’s effect on TCP Performance

• Web100

Enhancements to HSI (Hierarchical Storage Interface) Evaluate Gigabyte System Network and trunked Ethernet Investigate new protocols (ST, Storage over IP) Adapt and utilize simulation model of HPSS

Provide “place to be” – testbed for other SDM ETC activities


HPSS overview

High Performance Storage System Targets petabyte capacities and gigabyte/second

transfer rates Core servers manage metadata and control

movers Movers communicate over data network and move

data from network to storage and vice versa Distributed architecture – core servers and movers

can be replicated and run on hundreds of nodes


HPSS overview (continued) ORNL is a development partner Can establish HPSS systems without limit (4 now)

Production HPSS/AIX, most current version HPSS/Sun, next version HPSS/AIX, next version

Can modify HPSS itself


Probe – “Place to be”Overview of ORNL Probe Cell, June 2001

Stingray

RS/6000 S80

MarlinRS/6000

H70

STK Silo

300 GB SCSIRAID Disks

SunE250

CompaqDS20

360 GB SunFibreChannel

Disks

360 GB STKFibreChannel

Disks

FibreChannel Switch

GSN Switch

Origin 2000Reality Monster

RS/6000B80

ExternalEsnet Router

To NERSC Probe

SunUltra 10

STK Silo

IBM SP Compaq Sierra

3494 Library

GSN Bridge

RS/600044P-170

ProbeShared with Production

IntelDual Xeon

Linux

SunE450

IBMF50

SGIOrigin

200

SN6000

Gigabit Ethernet


ORNL Probe Cell, June 2001 – equipment for agent infrastructure

Stingray

RS/6000 S80

MarlinRS/6000

H70


RS/6000B80


To NERSC Probe



IntelDual Xeon

Linux

SunE450

Gigabit EthernetJumbo

Agents might run on any machine; variety of platforms shown.

CPU intensive agents might run on Stingray or some supercomputer.

SunE250

CompaqDS20

IBMF50

SGIOrigin

200


ORNL Probe Cell, June 2001 – equipment for file-transfer work

Stingray

RS/6000 S80

MarlinRS/6000

H70

STK Silo



Disks

GSN Switch


RS/6000B80


To NERSC Probe


STK Silo


3494 Library

GSN Bridge


IntelDual Xeon

Linux

SunE450

SN6000

Variety of hostsVariety of storage equipmentVariety of links

SunE250

CompaqDS20


Disks

FibreChannel Switch

IBMF50

SGIOrigin

200


ORNL Probe Cell, June 2001 – equipment for data research

Stingray – S806-processor RS/6000

2GBExcellent CPU and

I/O capability

Marlin – H704-processor

RS/60002 GB

Oracle, DB2

STK Silo



Disks


Disks

FibreChannel Switch


To NERSC Probe


STK Silo


3494 Library


IntelDual Xeon

Linux

SunE450

Oracle, DB2

HPSS

CPU intensive work might run on Stingray, Marlin or some supercomputer.

Available: HPSS/IBM, HPSS/Solaris, Oracle 8i and DB2

HPSS


Application involvements with Probe Climate

Goal: production use of WAN bulk transfers Probe provides:

• Some storage space (source of data)• Access to new transfer mechanism and associated equipment• Development and testing to expedite their work

Astrophysics Goal: high-bandwidth transfers between heterogeneous platforms Probe expects to provide:

• ~300 GB storage (disk and tape)• Access to S80 or perhaps other machines• Access to GSN equipment• Assistance using the above


Climate: Wide-Area bulk transfers WAN bulk transfers are a vital element of SDM ETC Wide-area bulk data transfer requires:

High sustained rate Asynchronous transfer (so user isn’t tied up) Suitability for many source/sink pairs

“Remote Mover” architecture works Designed for Climate – one direction, one source/sink pair Not suitable for SciDAC needs

Considering implementing a front-end for HPSS Currently testing ESnet III

ORNL/NERSC performance poor (bandwidth quadrupled but latency doubled; other problems too)


ORNL GSN Configuration

ODS/EssentialGSN Switch

6 ports

GenrocoBridge

StingrayRS/6000

Model S80AIX 4.3.3

CapsicumOrigin 2000

Reality Monster

IRIX 6.5.10

Compaq Alpha DS20

Tru64 4.0f

2 Gigabit Ethernet blades

2 FibreChannel blades

CaveVisualizationEquipment

Sun, Clariion or Compaq

disks

IBM, Compaq useGenroco 6466 NICs(64 bit 66 MHz)

MarlinRS/6000

Model H70(50 MHz slots)

StorageEquipment

EarlRS/6000

Model B80(50 MHz slots)

Alternative nodes


ORNL Probe Cell, June 2001 – Possible equipment for supernova visualization

Stingray

RS/6000 S80

STK Silo


CompaqDS20


Disks


Disks

FibreChannel Switch

GSN Switch


RS/6000B80


To NERSC Probe


STK Silo


3494 Library

GSN Bridge

ORNL BackboneProbeShared with Production

SunE450

AIX 5L, GSN

SGIOrigin

200

GSN

RAIT

HPSS

SN6000

MarlinRS/6000

H70HPSS, GSN

CPU intensive work might run on Stingray, Marlin or some supercomputer.


Technology on hand and available On hand

HPSS (unlimited instantiations) and HPSS development license ESnet III OC12 Oracle 8i and DB2 (current developer’s editions) Inter-HPSS hsi application TCP-friendly UDP service OPNET modeling product Variety of current storage and networking hardware and software RAIT (scheduled for trial in August)

In plan: ORNL GRID node (MICS funded) Web100 participation (MICS funded) MPI/IO software


Questions?

o ak r idge n ational l aboratory u.s. d epartment of e nergy probe plans and status scidac kickoff...

Documents