1

A H

isto

ry o

f Sup

erco

mpu

ting

Bur

ton

Smith

Mic

roso

ft

2

Out

line

!D

efin

ition

s!

Early

day

s: C

olos

sus a

nd E

niac

!M

ainf

ram

es: U

niva

c LA

RC

to C

DC

760

0!

SIM

D sy

stem

s"

Proc

esso

r arr

ays:

Illia

c IV

, PEP

E, a

nd B

SP"

One

-bit

arra

ys: S

tara

n, D

AP,

and

the

Goo

dyea

r MPP

"V

ecto

r pip

elin

ing:

TI-

ASC

, CD

C S

tar-

100,

and

Cra

y-1

"W

ide

inst

ruct

ions

: FPS

, Mul

tiflo

w, a

nd C

ydro

me

!M

IMD

syst

ems

"Sh

ared

mem

ory:

Cra

y X

-MP

to S

GI A

ltix

"D

istri

bute

d m

emor

y: C

osm

ic C

ube

to B

lue

Gen

e!

Japa

n-U

S co

mpe

titio

n!

Wha

t nex

t?

3

Def

initi

ons

!Su

perc

ompu

ter:

the

wor

ld�s

fast

est c

ompu

ter a

t the

tim

e!

SIM

D: S

ingl

e In

stru

ctio

n St

ream

, Mul

tiple

Dat

a St

ream

"O

ne in

stru

ctio

n at

a ti

me,

eac

h op

erat

ing

on a

n ar

ray

of d

ata

!M

IMD

: Mul

tiple

Inst

ruct

ion

Stre

am, M

ultip

le D

ata

Stre

am"

Mul

tiple

pro

cess

ors a

sync

hron

ousl

y is

suin

g in

stru

ctio

ns!

Shar

ed M

emor

y: M

IMD

com

pute

r in

whi

ch a

com

mon

m

emor

y is

acc

essi

ble

by a

ll pr

oces

sors

"U

MA

: Uni

form

mem

ory

acce

ss b

y al

l pro

cess

ors

"N

UM

A: N

on-u

nifo

rm m

emor

y ac

cess

, bas

ed o

n pl

acem

ent

!D

istri

bute

d M

emor

y: M

IMD

com

pute

r in

whi

ch th

e m

emor

y is

par

titio

ned

into

pro

cess

or-p

rivat

e re

gion

s!

RIS

C: R

educ

ed In

stru

ctio

n Se

t Com

pute

r"

Use

s fas

t and

sim

ple

inst

ruct

ions

to d

o co

mpl

ex th

ings

!Pi

pelin

ing:

Pro

cess

ing

in a

ssem

bly-

line

styl

e

4

Nam

es a

nd P

lace

s!

BM

D/A

TC: B

allis

tic M

issi

le D

efen

se A

dvan

ced

Tech

nolo

gy C

ente

r, H

unts

ville

, AL

!B

RL:

Bal

listic

s (no

w A

rmy}

Res

earc

h La

bora

tory

, Abe

rdee

n, M

D!

DA

RPA

: Def

ense

Adv

ance

d R

esea

rch

Proj

ects

Age

ncy,

VA

!N

ASA

-Am

es: A

mes

Res

earc

h C

ente

r, M

ount

ain

Vie

w, C

A!

NA

SA-G

odda

rd: G

odda

rd S

pace

Flig

ht C

ente

r, G

reen

belt,

MD

!Li

verm

ore:

Law

renc

e Li

verm

ore

Nat

iona

l Lab

orat

ory,

Liv

erm

ore,

CA

!Lo

s Ala

mos

: Los

Ala

mos

Nat

iona

l Lab

orat

ory,

Los

Ala

mos

, NM

!N

SA: N

atio

nal S

ecur

ity A

genc

y, F

ort M

eade

, MD

!Sa

ndia

: San

dia

Nat

iona

l Lab

orat

orie

s, A

lbuq

uerq

ue, N

M

5

Ear

ly d

ays:

Col

ossu

s!

Use

d at

Ble

tchl

ey P

ark,

Eng

land

dur

ing

WW

II"

for c

rypt

anal

ysis

of L

oren

z SZ

40/4

2 ro

tor s

yste

ms (

�Fis

h�)

"O

nly

rece

ntly

dec

lass

ified

by

Her

Maj

esty

�s G

over

nmen

t!

Con

cept

s by

Max

New

man

et a

l.!

Des

igne

d an

d bu

ilt b

y To

mm

y Fl

ower

s"

at th

e Po

st O

ffic

e R

esea

rch

Stat

ion,

Dol

lis H

ill!

Feat

ures

:"

Pape

r tap

e lo

op d

ata

inpu

t at 3

0 m

ph =

500

0 ch

arac

ters

/sec

"15

00 v

acuu

m tu

bes (

�val

ves�

) in

the

Mar

k I,

2400

in M

ark

II"

Prog

ram

med

by

switc

hes

"N

ot a

gen

eral

-pur

pose

com

pute

r

6

Col

ossu

s

7

Ear

ly d

ays:

Eni

ac!

Bui

lt at

Pen

n�s M

oore

Sch

ool f

or B

RL,

Abe

rdee

n, M

D"

The

mot

ivat

ing

appl

icat

ion

was

arti

llery

firin

g ta

bles

"Its

firs

t use

was

eva

luat

ing

H-b

omb

feas

ibili

ty!

Des

igne

d an

d bu

ilt b

y J.

Pres

per E

cker

t and

John

Mau

chly

!U

nvei

led

at th

e M

oore

Sch

ool o

n Fe

brua

ry 1

5, 1

946

!It

had

17,4

68 v

acuu

m tu

bes a

nd c

onsu

med

174

KW

"R

elia

bilit

y w

as a

con

cern

, so

it w

as le

ft on

con

tinuo

usly

!It

was

80

feet

long

and

wei

ghed

30

tons

!Pr

ogra

mm

ing

was

via

switc

hes a

nd p

atch

cab

les

!N

ot a

gen

eral

-pur

pose

com

pute

r

8

Eni

ac

9

Oth

er e

arly

supe

rcom

pute

rs

Zuse

Z3

(194

1)

The

IAS

mac

hine

s (19

52)

Man

ches

ter/F

erra

nti M

ark

I (19

51)

Uni

vac

1 (1

951)

10

Mai

nfra

mes

!Th

e 50

�s a

nd 6

0�s s

aw fa

st p

rogr

ess i

n co

mpu

ter t

echn

olog

y!

Mos

t of i

t was

firs

t use

d in

�m

ainf

ram

e� su

perc

ompu

ters

"Th

ese

syst

ems w

ere

used

for b

oth

busi

ness

and

scie

nce

"La

ter,

supe

rcom

pute

rs b

ecam

e m

uch

mor

e sc

ienc

e-or

ient

ed!

Exam

ples

of n

ew te

chno

logy

incl

uded

:"

Mag

netic

cor

e m

emor

y"

Tran

sist

or lo

gic

circ

uits

"Fl

oatin

g po

int h

ardw

are

"Pi

pelin

ing

"H

igh

leve

l lan

guag

es a

nd c

ompi

lers

11

Mai

nfra

mes

: LA

RC

!B

egun

in 1

955

for L

iver

mor

e an

d de

liver

ed in

196

0!

Had

dua

l pro

cess

ors a

nd d

ecim

al a

rithm

etic

!Em

ploy

ed su

rfac

e-ba

rrie

r tra

nsis

tors

and

cor

e m

emor

y

12

Mai

nfra

mes

: Str

etch

and

Har

vest

�IB

M 7

030

(STR

ETC

H)

�Del

iver

ed to

Los

Ala

mos

4/6

1�P

ione

ered

in b

oth

arch

itect

ure

and

impl

emen

tatio

n at

IBM

�IB

M 7

950

(HA

RV

EST)

�Del

iver

ed to

NSA

2/6

2�W

as S

TRET

CH

+ 4

boxe

s�I

BM

795

1 St

ream

uni

t�I

BM

795

2 C

ore

stor

age

�IB

M 7

955

Tape

uni

t�I

BM

795

9 I/O

Exc

hang

e

13

Mai

nfra

mes

: CD

C 6

600

!Se

ymou

r Cra

y an

d B

ill N

orris

left

Uni

vac

to st

art C

DC

!C

ray�

s CD

C 1

604

was

the

first

succ

essf

ul tr

ansi

stor

syst

em!

He

wan

ted

to b

uild

a sc

ient

ific

supe

rcom

pute

r!

The

6600

, bui

lt w

ith si

licon

tran

sist

ors,

ship

ped

Sept

. 196

4"

The

chie

f arc

hite

cts w

ere

Seym

our C

ray

and

Jim

Tho

rnto

n!

The

6600

had

seve

ral n

otab

le fe

atur

es"

A v

ery

sim

ple

inst

ruct

ion

set,

lend

ing

itsel

f to

mor

e sp

eed

#N

o de

cim

al a

rithm

etic

nee

ded

or w

ante

d"

10 a

rithm

etic

func

tion

units

abl

e to

pro

cess

in p

aral

lel

#Th

is o

verla

pped

exe

cutio

n st

yle

is n

ow c

alle

d su

pers

cala

r#

It w

as c

oord

inat

ed b

y a

fam

ous c

ircui

t cal

led

the

scor

eboa

rd"

Para

llel I

/O p

roce

ssor

s usi

ng a

new

idea

cal

led

mul

tithr

eadi

ng#

Som

e sa

y R

ISC

mea

ns �

Rea

lly In

vent

ed b

y Se

ymou

r Cra

y�"

A fu

turis

tic o

pera

tor�

s con

sole

was

pro

vide

d at

no

extra

cha

rge

14

CD

C 6

600

Con

sole

15

Mai

nfra

mes

: IB

M 3

60/9

1!

IBM

�s T

om W

atso

n w

as a

ngry

:"L

ast w

eek,

Con

trol D

ata

...an

noun

ced

the

6600

syst

em.

I und

erst

and

that

in th

e la

bora

tory

...th

ere

are

only

34

peop

le in

clud

ing

the

jani

tor.

Of

thes

e, 1

4 ar

e en

gine

ers a

nd 4

are

pro

gram

mer

s. C

ontra

stin

g th

ism

odes

t ef

fort

with

our

vas

t dev

elop

men

t act

iviti

es, I

fail

to u

nder

stan

d w

hy w

e ha

ve lo

st o

ur in

dust

ry le

ader

ship

pos

ition

by

letti

ng so

meo

ne e

lse

offe

r th

e w

orld

's m

ost p

ower

ful c

ompu

ter."

!Th

e 36

0/91

(AK

A 3

70/1

95) w

as IB

M�s

ans

wer

!It

was

del

iver

ed to

NA

SA-G

odda

rd in

Oct

ober

196

7!

Kill

er A

pp: P

asse

nger

Airl

ine

Res

erva

tion

Syst

ems (

1964

)"

Join

tly d

evel

oped

by

IBM

, Am

eric

an, a

nd U

nite

d A

irlin

es"

Orig

inal

ly w

ritte

n in

IBM

ass

embl

y (i.

e. m

achi

ne) l

angu

age

!It

was

ver

y cl

ose

to th

e C

DC

760

0 in

per

form

ance

!Th

e 36

0/91

als

o ha

d a

pret

ty im

pres

sive

con

sole

�

16

IBM

360

/91

Con

sole

17

Mai

nfra

mes

: CD

C 7

600

!Th

e 76

00 w

as d

eliv

ered

to L

iver

mor

e in

ear

ly 1

969

!It

was

hig

hly

com

patib

le w

ith th

e 66

00!

Bes

ides

bei

ng fa

ster

, its

arit

hmet

ic w

as p

ipel

ined

"Th

is m

eant

the

arith

met

ic u

nits

wer

e ne

ver b

usy

!It

was

so c

ompa

ct it

had

to b

e co

oled

with

freo

n!

It w

as a

lso

one

of th

e w

orld

s mos

t bea

utifu

l com

pute

rs

Seym

our C

ray

18

Two

CD

C 7

600s

19

SIM

D a

rray

s: Il

liac

IV!

By

the

late

60�

s, it

was

cle

ar m

ainf

ram

es w

eren

�t en

ough

!To

impr

ove

perf

orm

ance

, SIM

D a

rray

mac

hine

s wer

e bu

ilt

or p

ropo

sed

with

man

y ar

ithm

etic

pro

cess

ing

units

"So

lom

on w

as a

n ea

rly W

estin

ghou

se S

IMD

arr

ay p

roto

type

!Th

e Ill

iac

IV w

as a

Uni

vers

ity o

f Illi

nois

/Bur

roug

hs p

roje

ct"

Fund

ed b

y D

AR

PA fr

om 1

964

onw

ard,

it w

as u

sabl

e in

197

5"

The

chie

f arc

hite

ct, D

an S

lotn

ick,

cam

e fr

om W

estin

ghou

se!

It w

as to

hav

e 25

6 ar

ithm

etic

uni

ts, l

ater

cut

bac

k to

64

!Th

e th

in-f

ilm m

emor

y sy

stem

was

a m

ajor

hea

dach

e!

Afte

r stu

dent

dem

onst

ratio

ns a

t Illi

nois

in M

ay 1

970,

the

proj

ect w

as m

oved

to N

ASA

-Am

es!

Lang

uage

s lik

e IV

tran

(not

pron

ounc

ed fo

ur-tr

an) a

imed

to

use

para

llel l

oops

to e

xpre

ss th

e ne

cess

ary

para

llelis

m"

Muc

h co

mpi

ler e

xper

tise

was

dev

elop

ed w

orki

ng o

n Ill

iac

IV

20

ILLI

AC

IV

21

SIM

D A

rray

s: P

EPE

and

BSP

!B

oth

wer

e B

urro

ughs

des

igns

der

ived

from

Illia

c IV

!PE

PE h

ad 2

88 p

roce

ssin

g el

emen

ts a

nd w

as d

esig

ned

to

track

bal

listic

mis

sile

s"

It w

as d

eliv

ered

to B

MD

/ATC

in H

unts

ville

in 1

976

!B

SP h

ad 1

6 pr

oces

sing

ele

men

ts a

nd w

as a

imed

at

com

mer

cial

use

"R

icha

rd S

toke

s was

chi

ef a

rchi

tect

"Th

e fir

st c

usto

mer

was

Mar

atho

n O

il C

o. in

Den

ver,

CO

"Th

e pr

ojec

t was

can

celle

d in

198

0, b

efor

e its

firs

t del

iver

y

22

One

-bit

Arr

ays:

STA

RA

N, D

AP,

MPP

!Th

ese

wer

e ar

rays

of p

roce

ssor

s tha

t wer

e on

ly o

ne b

it w

ide

"Th

ey w

ere

inex

pens

ive

to im

plem

ent,

espe

cial

ly o

n on

e ch

ip"

Arit

hmet

ic w

as se

rial (

like

grad

e-sc

hool

, onl

y bi

nary

)#

It co

uld

use

wha

teve

r pre

cisi

on m

ade

sens

e fo

r the

pro

blem

!G

oody

ear A

eros

pace

dev

elop

ed S

TAR

AN

in 1

972

"It

was

a 2

56 b

y 25

6 ar

ray

with

lots

of d

ata

perm

utin

g po

wer

"K

en B

atch

er, a

n Ill

inoi

s gra

duat

e, w

as c

hief

arc

hite

ct"

It w

as g

reat

at i

mag

e pr

oces

sing

, tho

ugh

not i

nten

ded

for i

t "

The

ASP

RO

ver

sion

was

use

d on

Nav

y A

WA

CS

airc

raft

!Th

e G

oody

ear M

PP w

as a

sing

le-c

hip

STA

RA

N su

cces

sor

"Th

e ar

ray

size

was

redu

ced

to 1

28 b

y 12

8"

Del

iver

ed to

NA

SA-G

odda

rd in

198

4!

ICL

(UK

) dev

elop

ed th

e D

istri

bute

d A

rray

Pro

cess

or"

Stua

rt R

edda

way

was

the

arch

itect

23

One

-bit

Arr

ays:

Con

nect

ion

Mac

hine

!D

anny

Hill

is a

nd h

is c

olle

ague

s ini

tially

bas

ed th

e de

sign

on

a �

conn

ectio

nist

� ap

proa

ch to

arti

ficia

l int

ellig

ence

"Th

e de

sign

cam

e ou

t of D

anny

�s P

hD th

esis

at M

IT!

The

CM

-1 w

as a

hyp

ercu

be-c

onne

cted

one

-bit

arra

y"

Hig

h-le

vel l

angu

age

supp

ort w

as e

xcel

lent

"Th

e m

achi

ne p

rove

d us

eful

for s

cien

tific

com

puta

tion

!Th

e C

M-2

/CM

-200

add

ed so

me

32-b

it flo

atin

g po

int h

ardw

are

"M

any

HPC

use

rs li

ke N

SA a

ndLo

s Ala

mos

bou

ght t

hem

!Th

e C

M-2

had

an

arra

y of

ligh

t-em

ittin

g di

odes

(rig

ht)

"N

ot o

ne p

er p

roce

ssor

, sad

ly!

The

CM

-5 w

as M

IMD

, not

SIM

DC

M-2

24

Vec

tor p

ipel

inin

g: T

I ASC

!

The

ASC

was

des

igne

d to

exp

loit

Fortr

an lo

ops

"Th

is m

otiv

ated

the

vect

or p

ipel

inin

g de

cisi

on"

The

ASC

�s F

ortra

n co

mpi

ler w

as st

ate-

of-th

e-ar

t!

Seve

ral s

yste

ms s

hipp

ed, b

egin

ning

in 1

974

"G

eoph

ysic

al F

luid

Dyn

amic

s Lab

orat

ory,

Prin

ceto

n, N

J"

Bal

listic

Mis

sile

Def

ense

, Hun

tsvi

lle"

Nav

al R

esea

rch

Labo

rato

ry, A

naco

stia

, MD

!Th

e G

FDL

mac

hine

had

an

inte

rest

ing

perf

orm

ance

tool

: a

strip

-cha

rt re

cord

er p

erch

ed o

n to

p th

at p

lotte

d ve

ctor

leng

th"

The

reco

rdin

g w

as re

turn

ed to

the

user

with

the

job

outp

ut

25

Vec

tor P

ipel

inin

g: C

DC

Sta

r 100

!C

DC

use

d A

PL ra

ther

than

For

tran

loop

s as a

vec

tor m

odel

"A

PL is

a la

ngua

ge in

vent

ed b

y K

en Iv

erso

n of

IBM

Res

earc

h"

Nei

l Lin

coln

and

Jim

Tho

rnto

n w

ere

the

arch

itect

s"

Sidn

ey F

ernb

ach

at L

iver

mor

e w

as th

e ta

rget

cus

tom

er!

The

Star

-100

had

poo

r sca

lar (

non-

vect

or) p

erfo

rman

ce"

The

resu

lt w

as h

ighl

y lim

ited

appl

icab

ility

"Th

is e

ffec

t is c

alle

d Am

dahl

�s la

w!

Bot

h Li

verm

ore

and

Los A

lam

osw

ere

sche

dule

d to

get

one

"Lo

s Ala

mos

bac

ked

out

in fa

vor o

f the

Cra

y-1

"Li

verm

ore

got b

oth

Star

-100

syst

ems a

s a re

sult

Star

-100

26

Am

dahl

�s la

w!

If w

1w

ork

is d

one

at sp

eed

s 1an

d w

2at

spee

d s 2

,th

e av

erag

e sp

eed

sis (

w1+

w2)

/(w1/s

1+

w2/s

2)"

This

is ju

st th

e to

tal w

ork

divi

ded

by th

e to

tal t

ime

!Fo

r exa

mpl

e, if

w1=

9, w

2= 1

, s1=

100

, and

s 2=

1th

en s

= 10

/1.0

9 ≅

9"

This

is o

bvio

usly

not

the

aver

age

of s 1

and

s 2

Am

dahl

, Gen

e M

, �V

alid

ity o

f the

sing

lepr

oces

sor a

ppro

ach

to a

chie

ving

larg

esc

ale

com

putin

g ca

pabi

litie

s�,

Proc

. SJC

C, A

FIPS

Pre

ss, 1

967

27

Vec

tor P

ipel

inin

g: C

ray-

1!

Unl

ike

the

CD

C S

tar-

100,

ther

e w

as n

o de

velo

pmen

t co

ntra

ct fo

r the

Cra

y-1

"M

r. C

ray

disl

iked

gov

ernm

ent�s

look

ing

over

his

shou

lder

!In

stea

d, C

ray

gave

Los

Ala

mos

a o

ne-y

ear f

ree

trial

!A

lmos

t no

softw

are

was

pro

vide

d by

Cra

y R

esea

rch

"Lo

s Ala

mos

dev

elop

ed o

r ada

pted

exi

stin

g so

ftwar

e!

Afte

r the

yea

r was

up,

Los

Ala

mos

leas

ed th

e sy

stem

"Th

e le

ase

was

fina

nced

by

a N

ew M

exic

o pe

trole

um p

erso

n!

The

Cra

y-1

defin

itely

did

not

suff

er fr

om A

mda

hl�s

law

"Its

scal

ar p

erfo

rman

ce w

as tw

ice

that

of t

he 7

600

"O

nce

vect

or so

ftwar

e m

atur

ed, 2

x be

cam

e 8x

or m

ore

!W

hen

peop

le sa

y �s

uper

com

pute

r�, t

hey

thin

k C

ray-

1

28

Cra

y-1

29

VLI

W: F

PS, M

ultif

low

, and

Cyd

rom

e!

VLI

W st

ands

for V

ery

Long

Inst

ruct

ion

Wor

d"

It is

a k

ind

of S

IMD

syst

em in

whi

ch e

ach

arith

met

ic u

nit c

an

be d

oing

som

ethi

ng d

iffer

ent (

thus

the

need

for w

idth

)!

Floa

ting

Poin

t Sys

tem

s led

in 1

976

with

the

AP-

120B

"G

lenn

Cul

ler w

as th

e ch

ief a

rchi

tect

"Th

ese

syst

ems w

ere

embe

dded

in G

E C

AT

scan

ners

"Th

ey w

ere

also

use

d w

idel

y fo

r sei

smic

dat

a pr

oces

sing

!M

ultif

low

was

foun

ded

in 1

984

by Jo

sh F

ishe

r to

expl

oit

som

e co

mpi

ler t

echn

olog

y he

had

bee

n pu

rsui

ng a

t Yal

e"

The

first

ship

men

ts o

f the

Tra

ce 7

/200

wer

e in

198

7!

Bob

Rau

�s c

ompa

ny, C

ydro

me,

was

als

o fo

unde

d in

198

4 an

d de

mon

stra

ted

the

Cyd

ra-5

mac

hine

in 1

987

!B

oth

syst

ems h

ad in

stru

ctio

ns w

ith se

ven

field

s and

256

bits

!Te

chni

cally

, non

e of

thes

e m

achi

nes w

as a

supe

rcom

pute

r

30

Shar

ed M

emor

y: C

ray

Vec

tor S

yste

ms

!C

ray

Res

earc

h, b

y Se

ymou

r Cra

y"

Cra

y-1

(197

6): 1

pro

cess

or"

Cra

y-2

(198

5): u

p to

4 p

roce

ssor

s*!

Cra

y R

esea

rch,

not

by

Seym

our C

ray

"C

ray

X-M

P (1

982)

: up

to 4

pro

cs"

Cra

y Y

-MP

(198

8): u

p to

8 p

rocs

"C

ray

C90

: (19

91?)

: up

to 1

6 pr

ocs

"C

ray

T90:

(199

4): u

p to

32

proc

s"

Cra

y X

1: (2

003)

: up

to 8

192

proc

s!

Cra

y C

ompu

ter,

by S

eym

our C

ray

"C

ray-

3 (1

993)

: up

to 1

6 pr

ocs

"C

ray-

4 (u

nfin

ishe

d): u

p to

64

proc

s!

All

are

UM

A sy

stem

s exc

ept t

he X

1, w

hich

is N

UM

A

*One

8-p

roce

ssor

Cra

y-2

was

bui

lt

Cra

y-2

31

Shar

ed M

emor

y: M

ultit

hrea

ding

!Th

ese

syst

ems p

ipel

ine

MIM

D a

rchi

tect

ure

in th

e sa

me

way

ve

ctor

syst

ems p

ipel

ine

SIM

D a

rchi

tect

ure

"Li

ke v

ecto

r mul

tipro

cess

ors,

they

can

be

UM

A o

r NU

MA

!Ex

ampl

es in

clud

e"

Den

elco

r HEP

(198

0): u

p to

16

proc

s"

Tera

MTA

-1 (1

998)

: up

to 1

6 pr

ocs

"C

ray

MTA

-2 (2

002)

: up

to 2

56 p

rocs

!A

var

iant

form

of m

ultit

hrea

ding

isSi

mul

tane

ous M

ultit

hrea

ding

(SM

T)"

Inte

l®ca

lls S

MT

�Hyp

er-T

hrea

ding

�C

ray

MTA

-2

32

Shar

ed M

emor

y: C

C-N

UM

A!

Cac

he-c

oher

ent N

UM

A sy

stem

s wer

e re

sear

ched

at

Stan

ford

and

dev

elop

ed a

s pro

duct

s by

Silic

on G

raph

ics

"K

enda

ll Sq

uare

Res

earc

h us

ed a

var

iant

cal

led

CO

MA

, w

hich

stan

ds fo

r Cac

he-O

nly

Mem

ory

Arc

hite

ctur

e!

Thes

e sy

stem

s aut

omat

ical

ly m

ove

data

am

ong

cach

esat

ea

ch p

roce

ssor

to m

inim

ize

proc

esso

r dat

a ac

cess

tim

es!

SGI C

C-N

UM

A sy

stem

s:"

Orig

in 2

000

(199

7): u

p to

128

pro

cs"

Orig

in 3

000

(200

3): u

p to

512

pro

cs"

Alti

x (2

003)

: up

to 5

12 p

rocs

*!

Ken

dall

Squa

re C

OM

A sy

stem

s:"

KSR

1: (1

991)

: up

to 1

088

proc

s"

KSR

2: (1

994)

: up

to 5

000

proc

s

* �C

olum

bia�

at N

ASA

-Am

es c

lust

ers 2

0 of

thes

eSGI A

ltix

33

Dis

trib

uted

Mem

ory:

Cos

mic

Cub

e!

Chu

ck S

eitz

and

Geo

ffre

y Fo

x pi

onee

red

the

idea

at C

alte

ch"

The

first

syst

em (1

982)

was

cal

led

the

Cos

mic

Cub

e"

It ha

d 64

Inte

l®80

86 p

roce

ssor

s, ea

ch w

ith a

n 80

87 a

ttach

ed"

The

proc

esso

rs w

ere

conn

ecte

d in

a 6

-dim

ensi

onal

hyp

ercu

be"

They

com

mun

icat

ed b

y se

ndin

g m

essa

ges t

o ea

ch o

ther

"Th

is sy

stem

arc

hite

ctur

e no

w d

omin

ates

supe

rcom

putin

g!

Early

com

mer

cial

syst

ems i

nclu

ded:

"A

met

ek/S

14 (1

986)

: 680

20, o

ver 1

000

proc

s"

Inte

l®IP

SC/1

(198

5): 8

0286

, up

to 6

4 pr

ocs

"nC

UB

E/10

(198

5): c

usto

m, u

p to

102

4 pr

ocs

!V

aria

tions

sinc

e th

en a

re le

gion

nCU

BE/

10

34

Dis

trib

uted

Mem

ory:

Cla

ssic

MPP

s!

A c

lass

ic M

PP is

a d

istri

bute

d m

emor

y sy

stem

who

se

proc

essi

ng e

lem

ents

are

indi

vidu

al (m

icro

)pro

cess

ors

!Ex

ampl

es in

clud

e:"

Cra

y T3

D (1

994)

, T3E

(199

6), X

T3"

Inte

l Tou

chst

one

serie

s, Pa

rago

n (1

992)

"IB

M S

P-1

(199

2), S

P-2,

Dee

p B

lue,

Blu

e G

ene

"A

nd m

any

othe

rs

IBM

Blu

e G

ene/

LC

ray

XT3

35

Dis

trib

uted

Mem

ory:

Clu

ster

s!

A c

lust

er M

PP is

a d

istri

bute

d m

emor

y sy

stem

who

se

proc

essi

ng e

lem

ents

are

shar

ed m

emor

y m

ultip

roce

ssor

s!

Exam

ples

incl

ude:

"IB

M e

Serv

er p

575

"N

EC E

arth

Sim

ulat

or (2

002)

, SX

-8"

A v

ast m

ultit

ude

of h

ome-

mad

e �B

eow

ulf�

clu

ster

s#

Usu

ally

bas

ed o

n In

tel o

r AM

D p

roce

ssor

s#

Even

Gov

ernm

ent L

abs h

ave

done

this

Earth

Sim

ulat

or (a

bove

)Ea

rth S

imul

ator

(bel

ow)

36

The

Top

500

List

!Th

is is

a li

st o

f the

fast

est c

ompu

ters

in th

e w

orld

"w

ww

.top5

00.o

rg"

By

defin

ition

, it�s

a g

reat

pla

ce fo

r sup

erco

mpu

ter s

potti

ng!

Be

very

car

eful

abo

ut th

e te

rm �

fast

est�

, tho

ugh

"Sp

eed

is m

easu

red

by o

ne b

ench

mar

k pr

ogra

m, L

INPA

CK

"LI

NPA

CK

doe

sn�t

care

how

wel

l the

syst

em is

con

nect

ed"

Shar

ed m

emor

y is

n�t u

sefu

l for

goo

d LI

NPA

CK

resu

lts!

With

that

cav

eat,

it�s w

ell w

orth

a lo

ok

37

Japa

nese

Sys

tem

s: F

ujits

u, N

EC

, Hita

chi

!Ja

pane

se su

perc

ompu

ters

mos

tly u

se v

ecto

r pro

cess

ors w

ith

shar

ed o

r (m

ost r

ecen

tly) d

istri

bute

d m

emor

y ar

chite

ctur

e!

Fujit

su sh

ippe

d its

firs

t vec

tor s

yste

m (F

acom

230

) in

1976

:"

This

was

follo

wed

by

the

VP2

00 (1

982)

, VP4

00 (1

985)

,V

P200

0 (1

989)

, VP2

600(

1990

), an

d di

strib

uted

mem

ory

VPP

500(

1992

), V

PP70

0(19

96),

and

VPP

5000

(199

9)"

Rec

ently

, Fuj

itsu

has b

een

selli

ng sc

alar

NU

MA

syst

ems

!N

EC g

ot a

late

star

t on

vect

ors,

but c

augh

t up

very

qui

ckly

:"

SX-1

(198

3), S

X-2

(198

5), S

X-3

(199

0), S

X-4

(199

4),

SX-5

(200

0), a

nd d

istri

bute

d m

emor

y SX

-6 a

nd S

X-7

(200

2),

Earth

Sim

ulat

or (2

002)

, and

SX

-8 (2

006)

"Ta

dash

i Wat

anab

e is

the

chie

f arc

hite

ct o

f the

se sy

stem

s!

Hita

chi h

as so

ld fe

w su

perc

ompu

ters

out

side

Japa

n"

An

exce

ptio

n is

the

dist

ribut

ed m

emor

y SR

8000

(200

0)

38

Japa

nese

Nat

iona

l Pro

ject

s!

The

Nat

iona

l Sup

ersp

eed

Com

pute

r Pro

ject

(198

1)"

Abo

ut ¥

23 b

illio

n ov

er 9

yea

rs"

Inte

nded

to p

rodu

ce a

supe

rcom

pute

r"

Two

proj

ects

: arc

hite

ctur

e an

d te

chno

logy

"C

ompa

nies

: Fuj

itsu,

NEC

, Hita

chi,

Mits

ubis

hi, O

ki, T

oshi

ba!

The

Fifth

Gen

erat

ion

Proj

ect (

1982

)"

Abo

ut ¥

55 b

illio

n ov

er 1

0 ye

ars

"C

ente

red

at a

new

labo

rato

ry: I

CO

T"

Focu

sed

on a

rtific

ial i

ntel

ligen

ce: i

nfer

ence

, vis

ion,

spee

ch"

MC

C in

Aus

tin, T

X w

as a

US

indu

stria

l res

pons

e

39

DA

RPA

Str

ateg

ic C

ompu

ting

Prog

ram

!Th

is p

rogr

am w

as in

tend

ed to

acc

eler

ate

the

deve

lopm

ent

and

appl

icat

ion

of p

aral

lel c

ompu

ting

!It

fund

ed se

vera

l ind

ustri

al d

evel

opm

ent p

roje

cts o

ver t

he

perio

d 19

83-1

993:

"Th

inki

ng M

achi

nes:

Con

nect

ion

Mac

hine

(1-b

it SI

MD

arr

ay)

"B

olt,

Ber

anek

, and

New

man

: But

terf

ly (s

hare

d m

emor

y)"

Inte

l: To

uchs

tone

serie

s (di

strib

uted

mem

ory)

"Te

ra: M

TA (s

hare

d m

emor

y)"

Cra

y: T

3D (d

istri

bute

d m

emor

y)!

Gov

ernm

ent a

genc

ies w

ere

enco

urag

ed to

try

thes

e sy

stem

s

40

The

DO

E A

SCI P

rogr

am!

The

Dep

artm

ent o

f Ene

rgy

was

dire

cted

to u

se c

ompu

ter

sim

ulat

ion

to re

plac

e nu

clea

r wea

pons

test

ing

!It

has p

rocu

red

a se

ries o

f dis

tribu

ted

mem

ory

syst

ems:

Syst

em N

ame

Lab

orat

ory

Man

ufac

ture

rD

ate

Red

Sand

iaIn

tel

1995

Blu

e Pa

cific

Live

rmor

eIB

M19

96B

lue

Mou

ntai

nLo

s Ala

mos

SGI

1998

Whi

teLi

verm

ore

IBM

2000

QLo

s Ala

mos

Com

paq

2002

Red

Sto

rmSa

ndia

Cra

y20

04Pu

rple

Live

rmor

eIB

M20

06

!Th

e pe

rfor

man

ce ra

nge

is a

bout

100

-fol

d

41

Wha

t Nex

t?!

Para

llelis

m is

bec

omin

g m

ains

tream

"In

tel a

nd A

MD

are

put

ting

mul

tiple

pro

cess

ors o

n a

chip

"It�

s onl

y a

few

now

, but

just

wai

t!!

This

will

cha

nge

the

supe

rcom

pute

r pla

ying

fiel

d, a

gain

"Pa

ralle

l app

licat

ion

softw

are

will

em

erge

"Pa

ralle

l lan

guag

es a

nd c

ompi

lers

will

ena

ble

it"

Para

llel a

rchi

tect

ure

will

evo

lve

to m

atch

!Th

e w

orld

�s c

ompu

ters

bec

ome

one

gian

t clu

ster

"Th

e cl

oud

links

them

"W

e m

ust m

ake

them

safe

!W

hat a

n ex

citin

g fie

ld it

is w

ith su

ch c

halle

nges

!

42

Apo

logi

es: C

ompa

nies

Not

Men

tione

d!

Alli

ant

!C

onve

x!

HP

!M

asPa

r!

Mat

sush

ita!

Mei

ko!

Pars

ys!

Pars

ytec

!Q

uadr

ics

!Si

emen

s-N

ixdo

rf!

Sun