Cees de Laat!GLIF.is founding member!

OFC 2009 – Grid vs Cloud Computing and Why This Should Concern the Optical Networking Community

Challenges in Enabling Grid Computing

over Optical Networks

100000 flops/byte!

Telescopes

Input nodes

Correlator nodes

Output node

..... To equal the hardware

correlator we need:

16 streams of 1Gbps

16 * 1Gbps of data

2 Tflops CPU power

2 TFlop / 16 Gbps =

1000 flops/byte

THIS IS A DATA FLOW

PROBLEM !!!

SCARIe: a research project to create a Software Correlator for e-VLBI. VLBI Correlation: signal processing technique to get high precision image from

spatially distributed radio-telescope.

The SCARIe project

The “Dead Cat” demo" SC2004 & iGrid2005!

SC2004,

Pittsburgh,

Nov. 6 to 12, 2004

iGrid2005,

San Diego,

sept. 2005

Produced by:

Michael Scarpa

Robert Belleman

Peter Sloot

Many thanks to:

AMC

SARA

GigaPort

UvA/AIR

Silicon Graphics,

Inc.

Zoölogisch Museum

US and International OptIPortal Sites!

NCMIR SIO

UIC

USGS EDC

TAMU UCI

SARA KISTI

NCSA &

TRECC

CALIT2

AIST RINCON & Nortel

IJKDIJK!

Sensor grid: instrument the dikes!

30000 sensors (microphones) to cover all Dutch dikes!

First controlled breach occurred on sept 27th ‘08:!

e-F

oo

d!

e-B

ioS

cie

nce!

e-B

iob

an

kin

g!

e-C

OA

ST!

e-E

co

log

y!

e-D

ata

-

inte

nsiv

e

scie

nc

es!

.....!

Virtual Laboratory!

generic e-Science services!

High Performance & Distributed Computing!

Web & Grid services!

GLIF 2008! Visualization courtesy of Bob Patterson, NCSA Data collection by Maxine Brown.

Management

Visualisation

Mining

Web2.0

Media

Backup

Security

NetherLight!Meta

CineGrid

Medical

TV

Gaming

Conference

RemoteControl

Clouds

Distributed Simulations

Predictions

EventProcessing StreamProcessing

Workflow

DataExploration

Who (just to name a few) working Interfaces - similar

solutions addressing the previous challenges

•! CANARIE - UCLP

•! ESnet = OSCARS

•! G-LAMBDA - GNS-WSI

•! HPDM - VLAN based lightpaths

•! NORTEL - DRAC

•! UvA - Token based service, NDL, etc

•! Phosphorous - G2MPLS, UCLP2, NRPS - > NSP

•! GN2 - JRA3 - AutoBahn - IDM

•! I2 - DRAGON/ HOPI - DCN

•! FermiLab - Lambda Station

•! DoE - LambdaStation, TeraPaths

Slide: Gigi Karmous Edwards!

•! Harmony architecture (I) !

•!15!

Grid-aware GMPLS (G2MPLS) for Grid Network Services

!! Two models for the layering between Grid and Network resources

G2MPLS overlay model

different scope with respect to the IETF GMPLS Overlay & Peer

G2MPLS integrated model

My view!

GLIF!

GN2/3!

Federica !

Phosphorus!

Onelab!

Planetlab!

FIRE!

etc.!

OSI!

7!

6!

5!

4!

3!

2!

1!

0!

•! needs

repeatable

experiments!

•! needs QoS &

lightpaths!

•! needs capacity

and capability!

•! needs

infrastructure

descriptions!

TeraThinking!

•! What constitutes a Tb/s network?!

•! CALIT2 has 8000 Gigabit drops ?->? Terabit Lan?!

•! look at 80 core Intel processor!

–! cut it in two, left and right communicate 8 TB/s!

•! think back to teraflop computing!!

–! MPI turns a room full of pc’s in a teraflop machine!

•! massive parallel channels in hosts, NIC’s!

•! TeraApps programming model supported by!

–! TFlops ! !-> !MPI / Globus!

–! TBytes ! !-> !OGSA/DAIS!

–! TPixels ! !-> !SAGE!

–! TSensors !-> !LOFAR, LHC, LOOKING, CineGrid, ...!

–! Tbit/s ! !-> !?!

ref Larry Smarr & CdL!

The Problem!

A

G

F E

D

C

B

H

I want HC and AB!

Success depends on the order!

Wouldn’t it be nice if I could request [HC, AB, ...]!

?!

Network Description Language

Object Subject Predicate

•! From semantic Web / Resource Description Framework.

•! The RDF uses XML as an interchange syntax.

•! Data is described by triplets:

Location! Device! Interface! Link!

name! description! locatedAt! hasInterface!

connectedTo! capacity! encodingType! encodingLabel!

Object

Subject Subject

Object

Subject

Object

Subject

Object

Subject

Network Description Language

Choice of RDF instead of XML syntax

Grounded modeling based on G0805 description:

Article: F. Dijkstra, B. Andree, K. Koymans, J. van der Ham, P. Grosso, C. de Laat, "A Multi-Layer Network

Model Based on ITU-T G.805"

NDL + PROLOG!

•!Reason about

graphs!

•!Find sub-graphs

that comply with

rules !

Research Questions:!

•!order of requests!

•!complex requests!

•!Usable leftovers!

139.63.145.0

139.63.145.1

139.63.145.15

139.63.145.16139.63.145.17

139.63.145.18

139.63.145.2

139.63.145.3

139.63.145.31

139.63.145.32

139.63.145.33

139.63.145.34

139.63.145.38

139.63.145.40

139.63.145.41

139.63.145.42139.63.145.43

139.63.145.44

139.63.145.45

139.63.145.46

139.63.145.49

139.63.145.50

139.63.145.51

139.63.145.52

139.63.145.63

139.63.145.64

139.63.145.65

139.63.145.66

139.63.145.68

139.63.145.69139.63.145.70139.63.145.71

139.63.145.72

139.63.145.73139.63.145.74

139.63.145.75

139.63.145.79

139.63.145.81

139.63.145.82

139.63.145.83

139.63.145.84

139.63.145.85

139.63.145.86

139.63.145.87

139.63.145.88

139.63.145.94 192.168.0.1 192.168.0.2

192.168.0.3

192.168.0.4

192.168.0.5

192.168.0.6

192.168.1.1

192.168.1.2

192.168.1.3

192.168.1.4

192.168.2.1

192.168.2.2

192.168.2.3

192.168.2.4

Visualisation

Initialization and BFS discovery of NEs

95.9

94.5

95.8

96.

95.9

99.9

99.

100.

100.

94.599.9

99.9

100.

95.899.

96.

99.9

100.

100.

99.8

100.

100.

98.9

99.8

100.

100.

100.

100.

100.

98.9

101.

100.

101.100.

100.

100.

139.63.145.94192.168.0.1192.168.0.2

192.168.0.3

192.168.0.4

192.168.0.5

192.168.0.6

192.168.1.1

192.168.1.2

192.168.1.3

192.168.1.4

192.168.2.1

192.168.2.2

192.168.2.3

192.168.2.4

Network flows using real-time bandwidth measurements

nodePath = ConvertIndicesToNodes[

ShortestPath[ g,

Node2Index[nids,"192.168.3.4"],

Node2Index[nids,"139.63.77.49"]],

nids];

Print["Path: ", nodePath];

If[NetworkTokenTransaction[nodePath, "green"]==True,

Print["Committed"], Print["Transaction failed"]];

Path:

{192.168.3.4,192.168.3.1,139.63.77.30,139.63.77.49}

Committed

Transaction on shortest path with tokens

Topology matters can be dealt with algorithmically

Results can be persisted using a transaction service built in UPVN

Needs["WebServices`"]

<<DiscreteMath`Combinatorica` <<DiscreteMath`GraphPlot`

InitNetworkTopologyService["edge.ict.tno.nl"]

Available methods:

{DiscoverNetworkElements,GetLinkBandwidth,GetAllIpLinks,Remote, NetworkTokenTransaction}

Global`upvnverbose = True;

AbsoluteTiming[nes = BFSDiscover["139.63.145.94"];][[1]]

AbsoluteTiming[result = BFSDiscoverLinks["139.63.145.94", nes];][[1]]

Getting neigbours of: 139.63.145.94

Internal links: {192.168.0.1, 139.63.145.94} (...)

Getting neigbours of:192.168.2.3

Internal links: {192.168.2.3}

Mathematica enables advanced graph queries, visualizations and real-

time network manipulations on UPVNs"

ref: Robert J. Meijer, Rudolf J. Strijkers, Leon Gommans, Cees de Laat, User Programmable Virtualiized Networks, accepted for publication to the IEEE e-Science 2006 conference Amsterdam.

Interactive programmable networks

Multi Layer Service Architecture!

Network

layers

Application

layers Use Interface

Control Interface

(protocols API’s)

Network

Service

Application

5b of 6!

n.a.v. interview met Kees Neggers (SURFnet) & Cees de Laat (UvA) "

•!BSIK projects

GigaPort &!

•!VL-e / e-

Science!

cookreport.com!

Questions ?!

Thanks: Paola Grosso & Jeroen vd Ham & Freek

Dijkstra & team for several of the slides.!

A Declarative Approach to Multi-Layer Path Finding Based on Semantic Network Descriptions.!

http://delaat.net:/~delaat/papers/declarative_path_finding.pdf!