national computational science alliance the alliance distributed supercomputing facilities opening...

National Computational Science Alliance

The Alliance Distributed Supercomputing Facilities

• Opening Talk to the Alliance User Advisory Council

• Held at Supercomputing ‘98 in Orlanda, Florida, December 5,1998


The National PACI Program -Partners and Supercomputer Users

850 Projects in 280 Universities60 Partner Universities


PACI - The NSF Partnerships for Advanced Computational Infrastructure

• The Two Partnerships (NPACI & Alliance) Each Have:– Leading-Edge Site

– The Site With the Very Large Scale Computing Systems

– Mid-Level Resource Sites– Partners With Alternative or Experimental Computer Architectures,

Data Stores, Visualization Capabilities, Etc.

– Applications Technologies– Computational Science Partners Involved in Development, Testing

and Evaluation of Infrastructure

– Enabling Technologies– Computer Science Partners, Developing Tools and Software

Infrastructure Driven by Application Partners

– Education, Outreach, and Training Partners

• Network Infrastructure Is Critical.

www.npaci.edu www.ncsa.uiuc.edu


NCSA is Combining Shared Memory Programming with Massive Parallelism

1

10

100

1000

10000

Jan

-94

Jan

-95

Jan

-96

Jan

-97

Jan

-98

Jan

-99

Jan

-00

Jan

-01

SG

I Pro

cess

ors

Doubling Every Nine Months!

Challenge

Power Challenge

Origin

SN1


NCSA Users by System

0

100

200

300

400

500

600S

ep

94

Nov9

4

Jan

95

Mar9

5

May9

5

Jul9

5

Sep

95

Nov9

5

Jan

96

Mar9

6

May9

6

Jul9

6

Sep

96

Nov9

6

Jan

97

Mar9

7

May9

7

Jul9

7

Sep

97

Nov9

7

Jan

98

Mar9

8

May9

8

Jul9

8

Sep

98

Nu

mb

er

of

Users

SGI Power Challenge ArrayCM5

Convex C3880Convex ExemplarCray Y-MPOriginSPP-2000

C3880 (retired 10/95)

SPP-1200

Y-MP

(retired 12/94)

Origin

SPP-2000

CM-5(retired 1/97)

PCA(retired 7/98)

(retired 5/98)


Millions of NUs Used at NCSA FY93 to FY98

0

1

2

3

4

5

FY93 FY94 FY95 FY96 FY97 FY98

NU

s U

se

d A

nn

ua

lly (

Mill

ion

s)


10

100

1000

10000

100000

1000000

1

34

67

10

0

13

3

16

6

19

9

23

2

26

5

29

8

33

1

Projects in Ranked Order

An

nu

al N

Us

Pe

r P

roje

ct

Solomon (82)

NCSA Supplies Cycles to a Logarithmic Demand Function of Projects

Super

Large

Medium

Small

Tiny

Sugar (2)

Knight (67)

FY98 Usage < 10 NUs : Evans, Ghoniem, Jacobs, Long, York

Karniadakis (31)

Hawley (6)Suen (4)

Goddard (41)

Go

od

ric

h (

59

)

Kollman (10)

Chen (125)

Dro

eg

em

ier

(24

)


Evolution of the NCSA Project Distribution Function FY93 to FY98

10

100

1000

10000

100000

1000000

1 31 61 91 121

151

181

211

241

271

301

331

361

391

Ranked Order

An

nu

al N

Us

Per

Pro

ject

fy93

fy94

fy95

fy96

fy97

fy98


Rapid Increase in Large projects at NCSA FY93-98

10000

100000

1000000

1 7

13

19

25

31

37

43

49

55

61

67

73

79

Project Ranked Order

An

nu

al

NU

s P

er

Pro

jec

t

fy93

fy94

fy95

fy96

fy97

fy98


Breakout in Supporting Super Projects at NCSA in the Last Year

0

2

4

6

8

10


Nu

mb

er o

f Su

per

Pro

ject

s


Migration of NCSA User Distribution Toward the High End

0

20

40

60

80

100

120

140

160

180

200

Super Large Medium Small Tiny

Nu

mb

er

of

An

nu

al P

roje

cts

FY94

FY98

+400%

+350%

+114%

-27%

-79%

Number of Projects


Alliance LES Chose 27 Large PSC Projects to Track Out of 100 Targeted Projects

Includes:Droegemeir, Freeman, Karplus, Kollman, Schulten, Sugar

0

100,000

200,000

300,000

400,000

500,000

600,000

700,000

800,000

1QFY97 2QFY97 3QFY97 4QFY97 1QFY98 2QFY98 3QFY98

NU

s P

er Q

uar

ter

8 7 12 1516

24

26

Number of the 27 Projects Computing at NCSABar Shows NUs at NCSA Used Per Quarter


Disciplines Represented in the Large Academic Projects at the Alliance LES

Over 5,000 NUs Annually Per Project

>100 Projects

Over 3.2 Million NUs

Note Mapping to AT Teams:

NanomaterialsCosmology

Chemical EngineeringMolecular Biology

Environmental HydrologyAstro and Bio Instruments

6/1/97 to 5/31/98 NCSA

MAT

ASTR

CHEM

BIO

ENG

PHY

MATHCS

ATM


Application Performance Scaling on 128-Processor Origin

024

68

101214

161820

0

20 40 60 80

100

120

Processors

GFL

OP

S

QMCMatrix-VectorZEUS-MP BlastRIEMANN-HPFLaplace-DSM(250Mhz)

Laplace-MPI(250 Mhz)Woodward-PPMCactus (250Mhz)Freeman-Elec. Struc.

Conclusion -- 128-Processor Origin is a 15 GF Machine(20-25% of Peak)


Origin Brings Shared Memory to MPP Scalability

0

5

10

15

20

25

30

CA

CT

US

QM

C

QC

D

La

pla

ce

ZE

US

-MP

PIQ

MC

RIE

MA

NN

GF

LO

PS

Origin-128

T3E-256


Let’s Blow This Up!Let’s Blow This Up!

The Growth Rate of the National Capacity is Slowing Down Again

10,000

100,000

1,000,000

10,000,000

100,000,000

1,000,000,000

1986

1988

1990

1992

1994

1996

1998

2000

2002

Fiscal Year

No

rmal

ized

CP

U H

ou

rs

Total NU

70% Annual Growth This Year

Source: Quantum Research


The Drop in High End Capacity Available to National Academic Researchers

Monthly Normalized Usage (QRC data)

0

500,000

1,000,000

1,500,000

2,000,000O

ct-9

5

Dec

-95

Feb-

96

Apr

-96

Jun-

96

Aug

-96

Oct

-96

Dec

-96

Feb-

97

Apr

-97

Jun-

97

Aug

-97

Oct

-97

Dec

-97

Feb-

98

Apr

-98

Jun-

98

Aug

-98

No

rmal

ized

CP

U H

ou

rs

Cornell

PSC

SDSC

NCSA

Quantum Research FY96-98


Major Gap Has Developed in National Usage at NSF Supercomputer Centers

-

500,000

1,000,000

1,500,000

2,000,000

2,500,000

3,000,000

3,500,000

Oct

-95

Ap

r-96

Oct

-96

Ap

r-97

Oct

-97

Ap

r-98

Oct

-98

Ap

r-99

Oct

-99

NU

s U

sed

Per

Yea

r

Grand Total

70% Growth Rate

Projection


Allocated Capacity for Meritorious NSF Large National Projects

0%

20%

40%

60%

80%

100%

120%

140%

FY97/FY96 FY98/FY97

Incr

ease

d A

llo

cate

d C

apac

ity

Doubled

Data from NSF Metacenter and NRAC Reviews


Clustered Shared Memory Computers are Today’s High End

NCSA has 6 x 128 Origin ProcessorsASC has 4 x 128ARL has 3 x 128CEWES has 1 x 128NAVO has 1 x 128

Los Alamos ASCI Blue Will Have 48 x 128!

Livermore ASCI Blue has 1536x4 IBM SP


High-End Computing Enables High Resolution of Flow Details

1024x1024x1024-A Billion Zone Computation of Compressible

Turbulence

This Simulation Run on Los Alamos SGI

Origin Array

U. Minn.SGI Visual Supercomputer Renders Images

Vorticity

LCSE, Univ of Minnesota www.lcse.umn.edu/research/lanlrun/


Cycles Used by NSF Community at the NSF Supercomputer Centers by Vendor

SGI/Cray

IBM

HP

DEC

SGI SN1 is the Natural Upgrade for 84% of Cycles!

June 1, 1997 through May 31, 1998CTC, NCSA, PSC, SDSC

1019 Projects Using 100% of the Cycles

T3D/EOrigin/PC

C/T90


Peak Teraflops in Aggressive Upgrade Plan

0

10

20

30

40

50


Pe

ak

TF

LO

PS

SN2

SN1

Origin


Deputy Director Bordogna on NSF Leadership in Information Technologies

• Three Important Priorities for NSF in the Area of IT for the Future:

– The First Area Is Fundamental and High-Risk IT Research Advanced Computation Research. .

– The Second Priority Area for NSF Is Competitive Access and Use of High-end Computing and Networking.

– The Third Priority Is Investing in IT Education at All Levels.


President’s Information Technology Advisory Committee Interim Report

• More Long Term IT Research Needed– Fundamental Research in Software Development– R & D and Testbeds for Scalable Infrastructure– Increase Support for High End Computing

• Socio-Economic and Workforce Impacts– Address the Shortage of High-Tech Workers– Study Social and Economic Impacts of IT Adoption

• Modes and Management of Federal IT Research– Fund Projects of Broader Scope and Longer Duration– Virtual Centers for Expeditions into the 21st Century– NSF as Lead Agency for Coordinating IT Research

Congressional Testimony 10/6/98


PITAC Draft Refinement of High-End Acquisition Recommendation

• Fund the Acquisition of the Most Powerful High-End Computing Systems to Support Long Term Basic Research in Science and Engineering

• Access for (Highest Priority):

– ALL Academic Researchers– ALL Disciplines– ALL Universities

• Access for (Second Priority):– Government Researchers– Industrial Researchers


Harnessing the Unused Cycles of Networks of Workstations

Condor Cycles

University of Kansas is Installing Condor

Alliance Nanotechnologies TeamUsed Univ. of Wisconsin Condor Cluster -

Burned 1 CPU-Year in Two Weeks!


NT Workstation Shipments Rapidly Surpassing UNIX

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1995 1996 1997

Wo

rkst

atio

ns

Sh

ipp

ed (

Mill

ion

s)

UNIX

NT

Source: IDC, Wall Street Journal, 3/6/98


128 Hewlett Packard 300 MHz

64 Compaq 333 MHz

• Andrew Chien, CS UIUC-->UCSD• • Rob Pennington, NCSA• Reagan Moore, SDSC

• Plan to Link UCSD & UIUC Clusters

“Supercomputer performance at mail-order prices”-- Jim Gray, Microsoft

PACI Fostering Commodity Computing

Various Applications Sustain 7 GF on 128 Processors


Performance Analysis is Key Computer Science Research Enabling Computational Science

Mflops/Proc Flops/Byte Flops/NetworkRTCray T3E 1200 ~2 ~2,500

SGI Origin2000 500 ~0.5 ~1,000

HPVM NT Supercluster 300 ~3.2 ~6,000

IBM SP2 550 ~3.7 ~38,000

Berkeley NOW II 320 ~8.0 ~6,400

Beowulf(100Mbit) 300 ~25 ~500,000


0

1

2

3

4

5

6

7

0

10

20

30

40

50

60

Processors

Gig

afl

op

s

Origin-DSM

Origin-MPI

NT-MPI

SP2-MPI

T3E-MPI

SPP2000-DSM

Performance of Scalable SystemsShows the Promise of Local Clustered PCs

Danesh Tafti, Rob Pennington, NCSA; Andrew Chien (UIUC, UCSD)

Solving 2D Navier-Stokes Kernel


Near Perfect Scaling of Cactus - 3D Dynamic Solver for the Einstein GR Equations

0

20

40

60

80

100

1200

20

40

60

80

10

0

12

0Processors

Sc

alin

g

Origin

NT SC

Ratio of GFLOPsOrigin = 2.5x NT SC

Paul Walker, John Shalf, Rob Pennington, Andrew Chien NCSA

Cactus was Developed by Paul Walker, MPI-PotsdamUIUC, NCSA


QCD Performance on Various Machines

Doug Toussaint and Kostas Orginos, University of Arizona


The Road to Intel’s MercedThe Convergence of Scientific and Commercial Computing

http://developer.intel.com/solutions/archive/issue5/focus.htm#FOUR

IA-64 Co-Developed by Intel and Hewlett-Packard


User Web BrowserOutput to User

User Input

Format Translator, Query Engine and Program Driver

Workbench Server

Results to User

User Instructions and queries

Application Programs

(May have varyinginterfaces and be written in different

languages)Results

Instructions

Information Sources(May be of

varying formats)

Information

Queries

NCSA Computational Biology Group

The NCSA Information Workbench - An Architecture for Web-Based Computing


Structure & Function

Pathways & Physiology

Populations& Evolution

Ecosystems

Genomes

Gene Products

Using a Web Browser to Run Programs and Analyze Data Worldwide

NCSA Biology WorkbenchHas Over 6,000 Users From Over 20 Countries

national computational science alliance the alliance distributed supercomputing facilities opening...

Documents