International Research Computing “The convergence of next generation computing

and next generation networking”

Bill St. Arnaud

CANARIE Inc – www.canarie.ca

Bill.st.arnaud@canarie.ca

Who is CANARIE?

> Mandate to develop Canada’s next generation Internet> Private, not for profit corporation funded by government of

Canada> Partners include major telephone companies, Cisco, Nortel,

Alcatel, Ericsson, IBM, universities, research centers etc> We build and operate Canada’s national research and

education network – CA*net 4

Bringing CANARIE South to California

New 72 channel x 40 Gbps ROADM Networks

Boston

San Diego

Amsterdam

10 Gbps Wave from CENIC

CA*net 4 Network details

> CA*net 4 is NOT a single homogenous network> CA*net 4 is made up of many virtual networks (APNs) dedicated

to different communities and applications on a common substrate

– High energy physics network– Network for government research labs– Network for distributed computer backplane– Virtual networks for network research

> One of the parallel networks is a general purpose IP network– This is the only network that carries IPv6– All other networks only carry IPv4

> New ROADM will allow us offer 10G wavelengths for $25k per year

Carleton University

School of Architecture

PhysicsDepartment

Engineering Telecom

Firewall

Main campusNetwork

BorderRouter

University

Internet

Eucalyptus*Design Network

GlobalPhysicsNetwork

ResearchTestbed

NREN

Campus CWDM

10G

1G

1G

1G

10G

*Will connect to UCLA Architecture and Optiputer

University of British Columbia

Research Hospital

TRIUMF

Engineering Telecom

Firewall

Main campusNetwork

BorderRouter

University

Internet

Health Network

GlobalPhysicsNetwork

3D HDTV toMcGill

BCnet

Campus DWDM

5G

1G

1G

1G

3G

1GTier 1

Tier 2

CERN

Convergence of Next Generation Computing and Next Generation Networks

> Moving from channels to platforms– The old Internet and telecom was focused on setting up

communication channels to distribute data – routers, switches, paths, etc

– But data should not have locality

> Convergence of cyber-infrastructure and next generation Internet

> Integration of Web 2.0, SOA, P2P, Enterprise 2.0 and NGI

> E.g. Van Jacobson content centric networking> Grids

Enterprise 2.0

> “… a defining moment in business history. We are on the threshold of a dramatic shift in the way that firms are organized, innovate and create value. Information technology and new networked business structures are removing the sources of friction in our economy.

> .. A new breed of open, networked organization-the Enterprise 2.0-is emerging. ... Collaboration is the new foundation [of this new paradigm]...

> Normally the term collaboration conjures up images of office workers interacting effectively together. But the concept is changing. By "collaboration" we mean the increasing richness of means by which objects (things, people and firms) can work together enhanced by the medium of the Internet. We have described this as the fundamental transition of the Internet from being a communications platform to a computation platform.”

Modern HPC System Architecture

Computations Simulations

64-bit or 32-bitSMP or Cluster

Data Storage & Management

DB server, SAN, NAS, Tape Lib.

SAN

Lab or Field Data collection

PC, Laptop, PDA, .... etc.

Interpretation- visualization

Graphic WkstnRender Engine

Interpretation - data mining

Content Manager,AI Tools, ... etc.

Collaboration

GroupwareVirtualization

Web Portal

Remote Viz.

Computations Simulations

64-bit or 32-bitSMP or Cluster

Typical Large system today

Sensor Sensor Instrument Instrument Sensor

Layer 2 switch

Layer 3 switch/router

SONET/DWDM

ProcessProcess

Process Process Process

SONET/DWDM

HPC

Firewall

Internet

VPN

USER

Instrument Pod

Service Oriented Architectures

Sensor Sensor Instrument Instrument Sensor

Layer 2/3 switch

LAN

LAN

Data Management System

CA*net 4

VPN

USER

Instrument Pod

WS*

WS*WS

CA*net 4Lightpath

Process

ProcessWS**

WS*

Process

ProcessWS**

WS*

Process

ProcessWS

WS

Web serviceInterface

*CANARIE UCLP

**New web services

HPC

Workflow Screen Shot

Integrated Grid-Lightpath Workflow

Future Internet Research and Experimentation

ICT unit F4 New Infrastructure Paradigms & Experimental FacilitiesDG Information Society and Media, European Commission, Brussels

ICT unit F4 New Infrastructure Paradigms & Experimental FacilitiesDG Information Society and Media, European Commission, Brussels

The European FIRE Initiative

Per Blixt, Max Lemke, Fabrizio Sestini

http://cordis.europa.eu/fp7/ict/fire

The European FIRE Initiative

Per Blixt, Max Lemke, Fabrizio Sestini

http://cordis.europa.eu/fp7/ict/fire

New Architectures

Situated Services

BIONETSBIONETSAutonomic service evolution

HAGGLEHAGGLEOpportunistic networking (cross-layer)

Common research issues:Security, resilience, self-* (organisation,

evolution,healing, …) interaction of new

paradigms with society

ANAANA Beyond IP

self-org.

Autonomic communication elements

CASCADASCASCADAS

FIRE-baselineFIRE-baseline

GENI Facility Conceptual Design

Slicing, Virtualization, Programmability

Mobile Wireless Network

Sensor Network

Edge Site

Source: Peter Freeman NSF

A New Science ParadigmComputers – Networks - International

> Thousand years ago: Experimental Science - description of natural phenomena

> Last few hundred years: Theoretical Science - Newton’s Laws, Maxwell’s Equations …

> Last few decades: Computational Science - simulation of complex phenomena

> Today: e-Science or Data-centric Science - unify theory, experiment, and simulation - using data exploration and data mining• Data captured by instruments • Data generated by simulations• Data generated by sensor networks– Scientist analyzes databases/files

(With thanks to Jim Gray)

2

22.

3

4

a

cG

a

a

2

22.

3

4

a

cG

a

a

Source: Tony Hey Microsoft

Bill Gate’s vision of Science in 2020: Project NeptunePresented at SC’06

Source: Tony Hey Microsoft

Science user perspective

Sensor/InstrumentWS**

LANWS*Science Pod

LANWS*

Log Archive Process 1WS**

Log Archive Process 2WS**

ONS15454WS* NLR or CA*net 4

WS* CANARIE UCLP

WS* New Web service

DMAS

WS** New development

LightpathWS*

WS AAA process

WS**

User defined WSFL bindings

WS HPC Process

WS**

USER with WSFL binding software

UDDI orWSIL service registry

Undersea Sensor

Network

Connected & Controllable

Over the Internet

Sensors and instruments controlled over the Internet using cyber-infrastructure workflow

Source: Tony Hey Microsoft

The Means: NEPTUNE -CANARIE Project

> Embrace open standards to further innovation, collaboration and knowledge sharing (Open Source, Linux, OASIS)

> Migration from proprietary or legacy instrument control schemes (RS-232, RS-485, LECIS, SCADA)

> Abstract and service-enable system end-points (instruments, PC control stations, compute clusters, data management or analytical applications)

> Introduce a loosely-coupled integration fabric which places the burden on the “service bus” rather than the end-points (SOA, ESB, SOAP/XML)

> Exploit reliable asynchronous messaging or synchronous messaging as a means for instrument control or data interchange

> Research driven process and service choreography (BPEL, BPEL4WS)

> Adopt self describing data packets that can be inspected or transformed in transit (XML)

> Provide dynamic and ever-evolving experimental design or conditional handling support via content based routing for alerting, workflow, and event management

To adopt and further refine CANARIE to realize a generic web-service enabled telemetry and control system for VENUS and NEPTUNE

The Reality: Canada and California delivering Bill Gate’s vision today

Integrate Instruments & Sensors into a LambdaGrid Computing Environment With Web Services Interfaces and workflow

Two projects funded by CANARIE: McGill underwater HDTV

Neptune Canada Cyber-InfrastructureClose collaboration with SDSC/Scripps “LOOKING” and NSF ORION

Several Canadian Industry Partners especially SMEs

Eucalyptus Participatory Design Studio Grid – using UCLP and SOA

> Carleton Immersive Media Studio (CIMS), Carleton University, Canada

The Participatory Design Studio will allow architects and industrial designers at multiple locations to collaborate in real time by sharing computational resources, geometry datasets, and multimedia content. The expected result is the development and field testing of a Service Oriented Architecture utilizing User Controlled Light Paths (UCLPv2) on CA*net 4 that provides university architecture staff and students in Ottawa and Montreal with on-demand simultaneous shared access to visualization, modeling, and visual communication tools. The project is innovative because commercially available architectural tools not originally intended for long-distance use will become easy-to-use powerful enablers of long-distance design participation.

Source: Maxine Brown

Winter Simulation Conference 2006 • Monterey, California • December 2006

CIMS-Ottawa - home

CIMS-La Jolla - remote

Eucalyptus: Collaborative Architecture Design between California and Canada

Source: Gabriel Wainer

Eucalyptus Network (APN)

San Diego State this spring

Source: Gabriel Wainer

Eucalyptus SOA

Source: Gabriel Wainer

Eucalyptus APN

Source: Gabriel Wainer

The OptIPuter Project – Creating High Resolution Portals Over Dedicated Optical Channels to Global Science Data

> Establishing an OptIPuter Node at CRC will Enable the BADLABTM to Develop Collaborative Visualization Environments Using Lightpath Services across CAnet 4 to Calit2

Source: Larry Smarr CAL-It2

CineGrid™ Initiative

CineGrid Initiative

> UCSD/Calit2, USA> Keio/DMC, Japan> USC/CNTV, USA> UIC/EVL, USA> UW/Research Channel, USA> UCI/Calit2, USA> UIUC/NCSA, USA> Tohoku University, Japan> TUT/CTL, Japan> SFSU/INGI, USA> Ryerson University, Canada> NTT Labs, Japan> Cisco, USA> City University of Hong Kong, School

Creative Media, Hong Kong> Italian National Cinema School, Italy> USC/Entertainment Technology Center,

USA> LSU/Center for Computation and

Technology, USA> Digital Cinema Technology Forum, Japan> Connecticut State Library, USA> Canadian Film Center, Canada> University of Amsterdam, Netherlands> De Waag (Netherlands), Netherlands

CineGrid is an initiative to provide media professionals access to global cyberinfrastructure capable of carrying ultra-high-performance digital media using photonic networks, middleware, transport protocols and collaboration tools originally developed for scientific research, visualization, and Grid computing.

Photo: Harry Ammons

Tokyo

Keio/DMC

Seattle Chicago

San Diego

UCSD/Calit2

Abilene

JGN II

PNWGP

Pacific WaveCENIC

CAVEwave

StarLight

Otemachi

GEMnet2/NTT

CineGridTM International Real-Time Streaming 4K Digital Cinema

TorontoRyerson

Source: Maxine Brown

UCLP for international science

> Hyugen’s Cassinni probe landing on Titan

> First time VBLI data ever transferred over networks

> AARNet’s was connected to a CANARIE (CA) switch and a User Controlled LightPath (UCLP) set up at 1GbE to the Joint Institution for VLBI in Europe (JIVE, NL)

Data Reservoir Project

> Goal to create a global grid infrastructure to enable distributed data sharing and high-speed computing for data analysis and numerical simulations

> Online 2-PFLOPS system (part of the GRAPE-DR project), to be operational in 2008

• University of Tokyo, WIDE Project, JGN2 network, APAN, Fujitsu Computer Technologies, NTT Communications, Japan

• Chelsio Communications• StarLight, PNWGP, IEEAF,

USA• CANARIE, Canada• SURFnet, SARA and

University of Amsterdam, The Netherlands

Won April 26, 2006 Internet2 Land Speed Records (I2-LSR) in theIPv4 and IPv6 single and multi-stream categories. For IPv4, created a network path over 30,000 kilometers crossing eight international networks and exchange points, and transferred data at a rate of 8.80Gbps, or 264,147 terabit-meters per second(Tb-mps). For IPv6: created a path over 30,000 kilometers, crossing five international networks, and transferred data at a rate of 6.96 Gbps, or 208,800 Tb-mps.

http://data-reservoir.adm.s.u-tokyo.ac.jp

Source: Maxine Brown

GridJam: A Networked 3D Immersive Performance

Gridjam is an art and research project to study real-time, interactive, low-latency, partly improvised, 3D visualized, musical performances. The Virtual Color Organ (VCO) is a 3D immersive environment in which music is visually realized in colored and image-textured shapes as it is heard. The VCO visually illustrates information in a music’s score, the composer’s instructions to the musicians, and the musicians’ contributions to the score as they improvise in reaction to one another’s performances and to the immersive visual experiences. The VCO displays the emergent properties within the meaning of music, both as information and as art.

> Fine Arts Department, ARTSLab and Center for High Performance Computing, University of New Mexico

> Mills College, CA> Calit2, UCSD, CA> University of Alberta, Canada> De Waag, NL (tentative)> V2_, Institute for the Unstable Media, NL (tentative)

http://jackox.net/pages/gridjampages/Gridjam1.htmlSource: Maxine Brown

Phosphorus: Lambda User Controlled Infrastructure For European Research

> European Union (EU) Research Networking Testbeds IST program

– 30-month project, to begin October 2006

> An alliance of European and Global partners to develop advanced solutions of application-level middleware and underlying management and control plane technologies

> Project Vision and Mission

– To address key technical challenges in enabling on-demand end-to-end network services across multiple domains

– To treat the underlying network as a first-class Grid resource– To demonstrate solutions and functionalities across a testbed involving European

NRENs, GÉANT2, Cross Border Dark Fibre and GLIF connectivity infrastructuresSource: Maxine Brown

Global Lambda Visualization Facility (GLVF)

www.evl.uic.edu/cavern/glvf

– Problem: Optical networks and LambdaGrids enable large-scale global science collaborations − but interoperable visualization and collaboration tools are missing!

– Solution: Launched in September 2005, a group of iGrid 2005 Workshop participants who were designing and developing complementary, distributed visualization and collaboration technologies decided to pool expertise, build on each other’s successes, and integrate their work into a coherent whole, providing a unique model for international partnerships.

− Jason Leigh, Electronic Visualization Laboratory, University of Illinois at Chicago (organizer)

Source: Maxine Brown

NCMIRSIO

UIC

USGS EDC

SARA

NCSA & TRECC

Calit2

AIST

SFU KISTI

U MichiganNortel

www.optiputer.netwww.evl.uic.edu/cavern/glvf

GLVF Visualization Technologies

Source: Maxine Brown

Next: San Diego Interactive Imaging of High Resolution Brain Slices from McGill University

Source: Mark Ellisman, UCSD, Calit2

There are 7407 Slices at 20 µmEach Image has 8513 x 12,472 pixels

CA*net4

Capacity Capability Vector Major Data StorageSMP

Canada’s National Platform for HPC

Future? - Interconnection of Compute Canada and NSF Petascale and TeraGrid

Net Neutrality – CALEA- etc

> Universities and Telephone companies face the same problem of small number of heavy users consuming expensive Internet bandwidth

> University solution is to cap bandwidth from dormitories and/or block types of traffic

> Telecom solution is to build a two tiered Internet or doing volume capping– A high speed un-congested channel for the telco traffic particularly

aimed at carrying video> Universities can play a leadership role in piloting new last mile

(hundred feet) architectures that address problems of dormitories – May serve as possible model for telcos

> University students are ideal early adopters and were instrumental in diffusion of the Internet throughout larger community

One possible solution

> Following is example of one possible solution– There may be others- this is not intended to be definitive or exclusive

> Work with a few universities on a small number of pilots where interested students can lease or control dedicated fiber/copper to university colo point

> They can directly peer with other students in the dormitory across a “white light” switch or user controlled VLAN switch; and/or

> Connect to service providers of their and/or setup point to point user controlled VLANs to other students across NLR, CA*net 4, GLORIAD, GLIF, SURFnet, i2Cat, KREOnet, etc

> Primary application would be collaborative video such as Inuk, YouTube and/or CineGrid

Advantages for student

> One time very small cost for UNLIMITED bandwidth forever to university colo

> Cross connect to service provider of their choice or research network(s)

> NO or very low monthly Internet service fees for connection to content providers or connection across research networks

> Participate in new global collaborative models such as Inuk, YouTube.com, MySpace or CineGrid

> Direct connection to content and application providers> Student installs transceiver or simple media hub at their

computer– Media hub with CWDM for about $200 which includes laser, Gbe

transceiver etc

Inuk Networks www.inuknetworks.com

> Offering free triple play to university dormitories in the UK

> Free cable TV, telephony> Over the air channels from around the world> Also deliver university content to cable systems and

other institutions around the world> Make money by selling eyeballs> Joost has same strategy

Ottawa backbone fiber

Background material

> http://www.multichannel.com/article/CA6332098.html> it only takes about 10 BitTorrent users bartering files

on a node (of around 500) to double the delays experienced by everybody else.

> http://www.news.utoronto.ca/bin6/060222-2074.asp > University students played critical role in diffusion of

the Internet to the global community