starplane take-off cees de laat surfnetdelaat/starplane/docs/cdl-2006-02-2… · starplane take-off...

StarPlane take-offCees de Laat

SURFnetBSIK

EUUniversity of Amsterdam

SARATI

TNONCFNCF

History - 1DAS = Distributed ASCI Supercomputer• Project DAS-1 started in 1997 by Andrew Tanenbaum• To prove distributed clusters were as effective as super...• 4-5 clusters connected via high speed links

– DAS-1 -> 6 Mbit/s full mesh ATM– DAS-2 -> Gbit/s L3– DAS-3 -> StarPlane

• DAS-1 ran BSD, changed to Linux (Andrew... :-)• DAS-1 and 2 uniform architecture, not so in DAS-3• http://www.cs.vu.nl/das/

32 compute nodes

Fast interconnectLocal interconnect

Ethernet

head node

To localUniversity

and wide area interconnect

DAS 1 - 2 Cluster

History - 2SURFnet6 Architecture discussions 2001-2002• photonic backbone• (L2 and) L3 services• NORTEL• Static• Summer 2004 K&C• NWO-GLANCE• StarPlane• PHD-PD-SP• Start 1-feb-06, Li Xu, Jan Philip Velders, Jason Maasen

– Henri Bal, Paola Grosso, Herbert Bos, CdL, SN-folks.

R

R

Rref: cdl-2002-01-18-UCL-opt.ppt

2 ms3 ms

CommonPhotonic

Layer(CPL) in

SURFnet6

>5300 km

Dordrecht1

Breda1

Tilburg1

DenHaag

NLR

BT

BT NLR

BT

Zutphen1

Lelystad1

Subnetwork 4:Blue Azur

Subnetwork 3:Red

Subnetwork 1:Green

Subnetwork 2:Dark blue

Subnetwork 5:Grey

Emmeloord

Zwolle1

Venlo1

Enschede1

Groningen1

LeeuwardenHarlingen

Den Helder

Alkmaar1

Haarlem1

Leiden1

Assen1

Beilen1

Meppel1

Emmen1

Arnhem

Apeldoorn1

Bergen-op-

ZoomZierikzee

Middelburg

Vlissingen Krabbendijke

Breukelen1

Ede

Heerlen2Geleen1

DLO

Schiphol-Rijk

Wageningen1 Nijmegen1

Hilversum1

Hoogeveen1

Lelystad2

Amsterdam1

Dwingeloo1

Amsterdam2

Den Bosch1

Utrecht1

Beilen1

Nieuwegein1Rotterdam1

Delft1

Heerlen1

Heerlen1

Maastricht1

Eindhoven1

Maasbracht1

Rotterdam4

3XLSOP

IBG1 & IBG2Middenmeer1

StarPlaneDWDMbackplane

R

CPU’s

R

CPU’sR

CPU’s

CPU’

s

R

CPU’s

R

NOC

CdL

CPU’s

switch

university SURFnet

WS+AAANOCWS+AAA

SURFnet

Module Operation

> this schematic shows• several input fibres and one output fibre• light is focused and diffracted such that each

channel lands on a different MEMS mirror• the MEMS mirror is electronically controlled to tilt

the reflecting surface• the angle of tilt directs the light to the correct port

> in this example:• channel 1 is coming in on port 1 (shown in red)• when it hits the MEMS mirror the mirror is tilted to

direct this channel from port 1 to the common• only port 1 satisfies this angle, therefore all other

ports are blocked

diffraction grating

input and output fibres

collimating

lens

MEMS mirror array

(1 pixel per channel)

!1

!n

port 1common

ref Eric Bernier, NORTEL

32 compute nodes

Fast interconnectLocal interconnect

10 Gbit/sEthernet lanphy

1 Gbit/sEthernet

To SURFnet

head node

To localUniversity

DAS-3 Cluster

Hoogwaardig internet voor hoger onderwijs en onderzoek

Existing ring 1 – to be upgradedExisting Ring 1

GMD

G4

GMD

G4

G3G3

GM

D

GM

DGMD

G6

GMD

G6

G1G1

GM

D

GM

D

GMD

G9

GMD G9

GM

D

GM

D

UtrechtDelft

Den Haag

Leiden Hilversum

CMDCMDCMDCMDCMDCMD


Upgrade Day 1 –creating the ‘static’ mesh

Delft

Amsterdam 1Amsterdam 2

Amsterdam 3

G5 G5

G5 G5

WSS <-

split

select

select

split

WSS ->

WSS <-

WSS ->

WSS <-

split

select

select

split

WSS ->

WSS <-

WSS ->

G3

Delft

G5G3

G3G3

GMD

GMD

G5

Utrecht

G1

Delft

G1

G1G1

GMD

GMD

G9

G9

G9

G9

GMD

GMD

Hilversum

G6

Del f

t

G6

G6

G6

GMD

GMD

G5

G5

Leiden

G4

G4

G4

G4

GMD

GMD

Den Haag

G5 G5

G5G5 G5

G5

G5

G5

• Band 5 (up to 8 channels)added at all participatingnodes

• Wavelength SelectiveSwitches (WSS) added forreconfigurability support

• “Spur” to connect VU

• Full photonic mesh nowpossible between DAS-3sites

• This architectureis supported ‘now’

“spur” –hardwiredconnections

WSS


Day 2 set-up: branching out…

Delft

Amsterdam 1Amsterdam 2

Amsterdam 3

G5 G5

G5 G5

WSS <-

split

select

select

split

WSS ->

WSS <-

WSS ->

s e lec t

spli t

WSS

->

WSS

<-

WSS <-

split

select

select

split

WSS ->

WSS <-

WSS ->

s el ect

spl i t

WSS

->

WSS

<-

G3

Delft

G5G3

G3G3

GMD

GMD

G5

Utrecht

G1

Delft

G1

G1G1

GMD

GMD

G9

G9

G9

G9

GMD

GMD

Hilversum

G6

Delft

G6

G6

G6

GMD

GMD

G5

G5

Leiden

G4

G4

G4

G4

GMD

GMD

Den Haag

• Add WSSes atAmsterdam sites

• Is NOT supported inMarch 2006

• Full reconfigurabilityachieved

• Only limits are– Presence of card– Wavelength

blocking

• No changes to basic‘static’ mesh

“branch” insteadof “spur”

Adding WSSesincreasesreconfigurability


Day 2 detail

Delft

Leiden

Amsterdam 1

WSS <-

split

select

select

split

WSS ->

WSS <-

WSS ->

select

spl it

WS

S ->

WS

S <

-

Does this require CMDs

on all internal patches ??

A-D1, A -L2,

A -L3, A -V5,

A -V6, A -D8

Amsterdam 2

Amsterdam 3 - VU

WSS <-

split

select

select

split

WSS ->

WSS <-

WSS ->

TO DELFT

G5

G5

G5

G5

G5

G5

se lect

s pl it

WS

S ->

WS

S <

-

Does this require CMDs

on all internal patches ??

L-V1, L-A2, L-

A3, L-D4

L-V1, L-A2,

L-A3, L-D4

L-V1, A -V5,

A -V6, D-V7

V-L1, V-A5,

V-A6, VD 7

V-L1, A-L2,

A -L3, D-L4

L-A2, L-A3,

L-D4, V -A5,

V-A6, V-D7

V-L1, V-A5,

V-A6, VD7

L-V1, L-A2,

L-A3, L-D4

D-A1, D-L4,

D-V7, D-A8

A-L2, A -L3,

D-L4, A-V5,

A-V6, D-V7

A-L2, A -L3, D-L4,

A -V5, A-V6, D-V71 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8

L-A2, L-A3, L-D4,

V-A5, V -A6, V -D7

G5 G5

A-D1, L-D4,

V-D7, A-D8

L-A2, L-A3,

L-D4, V -A5,

V-A6, V-D7

A-D1, A-L2,

A -L3, A-D4,

A -V5, A -V6,

A-D1, A -L2,

A -L3, A -V5,

A -V6,A -D8

D-A1, D-L4,

D-V7, D-A8

D-A1, L-A2,

L-A3, V-A5,

V-A6, D-A8

• Wavelengthassignment remains– no externalchanges

• Adding WSSes allowsredirectingwavelengths from/toVU and AMS


Day 2 – black box reconfigurability

Amsterdam 1

Delft Leiden

Amsterdam VU

1

8

5

3

2

4

7 1

6

Hilversum

1

8

4

7

2

3

4

1

1657

581 6 3 2

AMS_VU + 1AMS-LE + 1AMS_DE + 0VU-LE + 0VU-DE + 1LE-DE + 1

DAS-3 Switch

DAS-3 Switch

DAS-3 Switch

DAS-3 Switch

DAS-3 Switch

• Compared today 1 now fourinstead of onepossibleredirection

• Redirectiononly limited bypresence ofcards andinternalwavelengthblocking


Day 2 – increased reconfigurability -adding cards

• Adding two cardsallows to createmore connectivitybetween ALLsites!

• Some sites canconnectivitythreefold (from10 Gb/s to 30Gb/s)

Amsterdam 1

Delft Leiden

Amsterdam VU

1

8

5

3

2

4

7 1

6

Hilversum

1

8

4

7 1

2

3

4

1

1657

581 6 3 2

4

4

4

4

44

1

1

1

AMS_VU + 2AMS-LE + 2AMS_DE + 1VU-LE + 1VU-DE + 2LE-DE + 1

StarPlane Goals1. fast, application-driven allocation of the photonic network resources2. application-specific composition of the protocol stack that controls

the resources3. low-level resource partitioning (and, hence, no interference)4. high-level requests (whereby policies and inference are used to

assist the user)..

To achieve and validate these goals the project will deliver:1. the implementation of the StarPlane management infrastructure2. the implementation of an intelligent broker service to handle high-

level requests3. the modification of a set of real applications to exploit the

functionality of such a management plane4. a library of standard components (protocols, middleware) to support

and build new applications

GRID-Colocation problem space

CPU DATA

Lambda’s

Extensivelyunderresearch

New!

Key issue #1:how to describe such networks?

UvA/SARA LightHouseA joint network research lab of the University of Amsterdam and SARA.

Connects end resources to NetherLight.Proof of concept e.g. tier 0/1, webservices, GSP

Semantic web

“a universal medium for the exchange of data where datacan be shared and processed by automated tools as wellas by people”

The Resource Description Framework (RDF) uses XML asan interchange syntax.

Data is described by triplets:

ObjectSubject Predicate

NDL - Network Description Language

A way to describe network resources using RDF.Parser can use the data to:- generate network maps- provide information to schedulers

<ndl:Device rdf:about="#Vangogh3"> <ndl:name>Vangogh3</ndl:name> <rb:isOfType>ComputingElement</rb:isOfType> <ndl:locatedAt rdf:resource="#Lighthouse"/> <ndl:hasInterface rdf:resource="#Vangogh3:eth2"/></ndl:Device>

NetherLight in RDF<?xml version="1.0" encoding="UTF-8"?><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"

xmlns:ndl="http://www.science.uva.nl/research/air/ndl#"><ndl:Location rdf:about="#Netherlight">

<ndl:name>Netherlight Optical Exchange</ndl:name></ndl:Location><ndl:Device rdf:about="#tdm3.amsterdam1.netherlight.net">

<ndl:name>tdm3.amsterdam1.netherlight.net</ndl:name><ndl:locatedAt rdf:resource="#amsterdam1.netherlight.net"/><ndl:hasInterface rdf:resource="#tdm3.amsterdam1.netherlight.net:501/1"/><ndl:hasInterface rdf:resource="#tdm3.amsterdam1.netherlight.net:501/3"/><ndl:hasInterface rdf:resource="#tdm3.amsterdam1.netherlight.net:501/4"/><ndl:hasInterface rdf:resource="#tdm3.amsterdam1.netherlight.net:503/1"/><ndl:hasInterface rdf:resource="#tdm3.amsterdam1.netherlight.net:503/2"/><ndl:hasInterface rdf:resource="#tdm3.amsterdam1.netherlight.net:503/3"/><ndl:hasInterface rdf:resource="#tdm3.amsterdam1.netherlight.net:503/4"/><ndl:hasInterface rdf:resource="#tdm3.amsterdam1.netherlight.net:504/1"/><ndl:hasInterface rdf:resource="#tdm3.amsterdam1.netherlight.net:504/2"/><ndl:hasInterface rdf:resource="#tdm3.amsterdam1.netherlight.net:504/3"/><ndl:hasInterface rdf:resource="#tdm3.amsterdam1.netherlight.net:504/4"/><ndl:hasInterface rdf:resource="#tdm3.amsterdam1.netherlight.net:501/2"/>



<ndl:Interface rdf:about="#tdm3.amsterdam1.netherlight.net:501/1"><ndl:name>tdm3.amsterdam1.netherlight.net:POS501/1</ndl:name><ndl:connectedTo rdf:resource="#tdm4.amsterdam1.netherlight.net:5/1"/>

</ndl:Interface><ndl:Interface rdf:about="#tdm3.amsterdam1.netherlight.net:501/2">

<ndl:name>tdm3.amsterdam1.netherlight.net:POS501/2</ndl:name><ndl:connectedTo rdf:resource="#tdm1.amsterdam1.netherlight.net:12/1"/>

</ndl:Interface>

RDF -> Picture

Key issue #2:How to book resources on such

networks?

Web services

Web services interfaces provide the API for thereservation framework:

<wsdl:operation name="getResourceInformation"> <wsdl:operation name="getResourceList"> <wsdl:operation name="getTypeList"> <wsdl:operation name="getResourcesOfType"> <wsdl:operation name="reservePath"> <wsdl:operation name="getPossiblePaths"> <wsdl:operation name="isPathAvailable"> <wsdl:operation name="confirmPathReservation"> <wsdl:operation name="cancelPathReservation">

Questions ?

Credits:– Leon Gommans, Paola Grosso, Marten Hoekstra, Arie Taal, Freek Dijkstra, Bert Andree, Jeroen van der Ham, Hans

Blom, Yuri Demchenko, Fred Wan, Karst Koymans, Martijn Steenbakkers Jaap van Ginkel, Li Xu– SURFnet / GigaPort, Kees Neggers, Erik-Jan Bos, et al!– NORTEL: Franco Travostino, Kim Roberts, Rod Wilson– SARA: Anwar Osseryan, Paul Wielinga, Pieter de Boer, Ronald van der Pol, teams– Joe Mambretti, Bill stArnaud, GLIF community– Tom & Maxine & Larry, Laurin, OptIPuter, OnVector team !!!!

starplane take-off cees de laat surfnetdelaat/starplane/docs/cdl-2006-02-2… · starplane take-off...

Documents