canarie, ca*net 4 and intelligent-infrastructure university of victoria november 28, 2005 andrew k....

40
CANARIE, CA*net 4 and Intelligent-Infrastructure University of Victoria November 28, 2005 Andrew K. Bjerring President and CEO

Post on 19-Dec-2015

216 views

Category:

Documents


0 download

TRANSCRIPT

CANARIE, CA*net 4 and Intelligent-Infrastructure

University of Victoria November 28, 2005

Andrew K. BjerringPresident and CEO

Outline

Background CA*net 4, Lightpaths and UCLP Grids and Intelligent Infrastructure

Background

The Ultimate Driver...

Moore’s Law: The Cost of Computing Power, Memory and Bandwidth Continues to Decrease…

US

$ -

Log

arith

mic

Sca

le

1

10

100

1,000

10,000

100, 000

1,000,000

0. 1

Computing (3)

Bandwidth (1)

70 80 90 00 10

(1) Cost of sending 1 tri llion bi ts(2) Cost of 1 megabyte storage(3) Cost of 1 MHz processor s peed

Storage (2)

Source : Federal Reserve Bank of Dallas, 1999 Annual Report : The New Paradigm

Cost of Communication Applications

...and the effect on networks

“When the network is as fast as the computer’s internal links, the machine disintegrates across the net into a set of special purpose appliances” -George Gilder

The Evolution of the Internet

First wave Application: Transfer of text and data Network: An add-on to the computer Organizations: Real and local

Second wave Application: Widespread access to images, sound and video. Network: A partner with the computer Organizations: Increasing “virtual” elements

Third wave * Application: Linking services; real time transactions Network: Integrated into a

“cyber infrastructure” or “intelligent infrastructure” Organizations: Emphasis shifting to “communities of interest”

*Forbes.com: “...the next phase of [the networking} revolution comes from adding new intelligence and services to the electronic "plumbing" that now envelopes the very way we conduct business.  From the Web to wireless, a new level of intelligent infrastructure is fast-changing the business landscape”.

(April, 2005)  

CANARIE’s First 12 Years (1993-2005)

• Advanced Networks: 4 generations of CA*net and associated regional nets

• Next-Generation Applications: Over 200 R&D projects focusing on e-learning, e-health, e-business, e-content and underlying technology

• Domestic and International Outreach: Broad range of more general awareness raising, community development and branding activities.

Next Generation Applications

Phase 1: 1993-1995 $26 m Stand-alone projects

Phase 2: 1995-1999 $80 m Collaborative projects

Phase 3: 1999-2004 $78 m Strategic sectoral development

Phase 4: ?? Intelligent infrastructure and e-solutions

Example of Sectoral Development: ECCnet

• Industry-wide data synchronization solution for supply chain management in grocery retail, foodservice, and pharmacy sectors

• Large enterprises, MNEs and SMEs all represented• For grocery retail sector: 2,500 companies, 300,000 products,

85% of transactions• $4.5M CANARIE funding; over $2.0M paid back to date

General Lesson Learned

Next-Generation applications involve applying new technologies to develop and re-design processes and relationships across

organizational boundaries

Technological Innovation

Web services and grids

Convergence of networks and IT

Leading to shared “intelligent infrastructure”

Process and Relational Innovation

Opportunities for productivity gains

Inter-organizational processes

Leading to a “community of interest” (COI) based planning

Status of CANARIE Strategy(2005-2010)

CA*net 4 →Current funding ends March 31, 2007→Extension to 2010 approved in principle by federal cabinet →Funding for post March 31, 2007 pending →Change from 3-5 year funding model part of proposal

“E-solutions” Proposal→Element of Industry Canada’s broader ICT Sector Strategy→Election call has introduced further delays and uncertainties

CA*net 4, Lightpaths and UCLP

Years Core

Technology Initial

Capacity (Mb/sec)

Applications

CA*net 1990-1997

Leased Lines

0.056 Basic email and file transfer

CA*net 2 1995-2000

ATM 155 Web applications plus large file sharing

CA*net 3 1998-2002

Optical 5,000 Full audiovisual sharing plus collaborative computing and data environments

CA*net 4 2002-7 Lightpaths 20,000 Grids and other third-wave applications

CA*net Generations

CA*net 4 Overview

• National Backbone: Connects regional R&E networks (ORANs) in every province, with links to global counterparts

• Very High Bandwidth: Currently based on five 10 Gbps wavelengths

• Hybrid Design: Fraction of one lambda is run as layer 3 network; balance of capacity is allocated as separate “lightpaths” to specific research projects and services (a.k.a. “lambda network”)

• Enables e-science: For example, supports dedicated connections to: beamlines at Canadian Light Source; remote HPC and visualization clusters; telescopes; distributed sensor arrays; data repositories; etc.

Global Lambda Integrated Facility

Research & Education Network bandwidth for scheduled application and middleware research experiments

LightPath Examples

1. KREONet2 to StarLight   Network link 10G  2. UoAlberta to CERN ATLAS High Level Trigger 1G 3. Montreal to Emily Carr Institute Art Net

155M4. McGill to Fermilab Data Transfer 1G 5. Ireland to Chicago Network link 1G 6. Victoria to Ottawa to Montréal Linking NRC Institutes 1G 7. Nortel (Ottawa) to Chicago Research 1G8. Alberta (WestGrid) to Fermilab Data transfer 1G9. TRIUMPH to CERN Data transfer 10G 10. Chicago to NewYork GLORIAD 1G11. Montreal to Hull RISQ backup 1G12. Kingston to Chicago Data Mining 1G 13. Ottawa (CRC) to Barcelona UCLP Demo 1G14. California to Europe Caltech performance testing1G 15. Seattle to Ireland UCLP demo 1G

CA*net 4 Architecture

Calgary ReginaWinnipeg

Montreal

Toronto

Halifax

St. John’s

Charlottetown

ChicagoSeattle

New York

Saskatoon

Victoria

Vancouver

Edmonton

Ottawa

Fredericton

Optical Switches

User Controlled LightPaths (UCLP)

Status

Software to partition and distribute control of optical switches Allows users to create and manage end-to-end lightpaths from the edge Uses web-services based Service-Oriented-Architecture (SOA) Three designs for initial version (UCLP v1) funded Diffusion is underway (open source)

→ Deployed across CA*net 4, KREOnet, I2Cat→ Used by ORANO, Netera, BCnet, HEAnet, RISQ→ Being tested by Internet2, SURFnet, AARnet, Nortel, Cisco, Meriton and others

• UCLP v2 development underway with three additional contracts

New CA*net 4 Programs

ORAN Support and Outreach: • $15 million• to support lightpath capability • to ensure universities, colleges and remote campuses can access CA*net 4• to undertake promotion and awareness activities ($1 m) • provincial allocations confirmedSTATUS: Several Statements of Work approved and work started

CANARIE Connections Program: • $10 million • to assist federal labs and private sector to connect sites and facilities to CA*net 4• Submissions received from NRC, NRCan, Environment, DFO, Agriculture Canada, Stats Canada

and othersSTATUS: First connections approved; first departmental MOU signed; other discussions continuing

Intelligent Infrastructure: • $15 million• to extend Service Oriented Architecture (SOA) to instruments, sensors and controllers and other

next-generation applications STATUS: First seven projects chosen; other discussions re “gaps and opportunities” underway

Grids and Intelligent Infrastructure

ATLAS Grid

• TRIUMF has been selected as Tier 1 site for CERN data→One of only ten Tier 1 sites in the world→Requires 10 Gbps connection to CERN by Q1-2006 and 5 Gbps

connection to backup site→Will also require dedicated 2 Gbe lightpaths to Tier 2 sites →Circuit may also be used to support a routed network for HEPnet

(Canada’s High Energy Physics network)

Astronomical Data Storage

• NRC is major participant in two Hawaii-based telescopes →CFHT – Canada France Hawaii Telescope→Gemini – Hawaii and Chile

• NRC Herzberg will be main repository for telescope data →Initial data volumes in 2005 will be over 7 Terabytes per month →Increasing to 100 Terabytes per month by 2006

• Collaborative project involving NRC, CANARIE, the Australian research network AARnet, National Lambda Rail in U.S. to swap lightpaths in order to make all the connections from the telescopes to Canada

Neptune Grid

Joint US-Canadian project to build large undersea fiber network off west coast of US and Canada

Undersea network will connect instrumentation devices, robotic submarines, sensors, under sea cameras, etc

Distributed computing and data storage devices on CA*net 4 and Internet 2 will be used to analyze and store data

U.S. “Cyberinfrastructure”

Information technology-enabled systems, tools and services have accelerated science and engineering research and education. Integrated to create a national cyberinfrastructure, these systems, tools and services are enabling individuals, groups and organizations to advance science and engineering in ways that revolutionize what they can do, how they do it, and who can participate.

www.nsf.gov

U.S. “Cyberinfrastructure”

CI research itself includes building new networking protocols, determining how to discover and access remote resources, providing programming abstractions of distributed resources to make them appear local, and understanding how science can take advantage of the networked world. I want to call out explicitly that a frontier component of CI is on the "edges," where researchers are extending the physical infrastructure to sensors and sensor networks wirelessly.

Peter, Arzberger, Chair of the PRAGMA steering committee in GRIDtoday

U.S. “Cyberinfrastructure”

To realize the enormous potential [of CI], new collaborations and partnerships are needed. Tighter bonds must be forged among the science and engineering community to ensure the broadest possible participation among researchers. New collaborations with the private sector are critical, as they will create the equipment that underpins tomorrow’s cyberinfrastructure. Finally, new partnerships will be needed across the globe, to glean insight, ensure openness, interoperability and a high level of trust within the international community.

Arden L. Bement, Jr., Director, NSF

E.U.: e-Infrastructures

The term e-Infrastructure is used to indicate the integrated ICT-based Research Infrastructure in Europe [building on] networks, supercomputers and storage... The initiative is intended to support the creation of a framework (political, technological and administrative) for the easy and cost-effective shared use of distributed electronic resources across Europe - particularly for grid computing, storage and networking.

http://www.cordis.lu/ist/rn/ri-cnd/einfrastructures.htm

e-Infrastructures Roadmap

• In 2004 the EC proposed to have a strategic roadmap for Research Infrastructures for next 10-20 yrs. • Four roadmaps: Physical Sci/Engineering, Life Sci., Social Sci/Humanities and e-Infrastructures • Key components off the current e-infrastructures roadmap (version 9) are:

networking infrastructures: The network, including lambda-networks, is at the heart of everything

middleware and organisation: The middleware and virtual organizations connect the distributed resources, data and storage facilities in a seamless way. Middleware plays the intermediary role to facilitate a deep integration of individual components. New processes and procedures have to be devised to alter the way organisations work.

resources: The European Science Grid should be populated with a number of resources, covering literally everything that is of interest to science from computers, large storage facilities, telescopes, satellites, special physics equipment, weather balloons, lasers, spectrometers, visualization means, large sensor networks, large data collections, artificial intelligence agents and even people.

http://www.e-irg.org/roadmap/eIRG-roadmap.pdf

E.U.: Adaptive Services Grid

• Project under Sixth Framework Programme• “Over the last few years, the service-oriented-architecture (SoA)

paradigm has gained massive interest from industry and academia...it will have massive impact on how we build software systems in the future”, Mathias Weske, ASG Scientific Coordinator

• The use of Grid technology in the underlying infrastructure ensures efficient resource usage...

• The ASG consortium has 22 partners from six EU countries and Australia

www.asg-platform.org

SoA

• A set of self contained functional components (of higher level applications) encapsulated as services

• Interaction between services through well-defined interfaces• Interaction details that are hidden behind the service interface • Enables new modes of user interaction• Fundamentally alters the meaning of an application and the means

by which data can be retrieved, edited, manipulated and re-inserted • An application is now the sum of functions (services) addressable

through APIs (interfaces) • “Mashups” are combinations of disparate services through a

common interface

Peter Brantley; University of California, Digital Library

In brief....

SoA is about putting down the paving, the yellow median stripe, and the shoulders for a fast road. It's about letting folks drive it without steering for them. Basically, drivers need minimal guidelines: stay on the road, don't go in the wrong direction. By providing simple and excellent infrastructure, you don't have to design the next generation car - the users will do it for you. You just need to get out of the way.

Peter Brantley; University of California

Kepler

• Scientists typically carry out tasks involving the design and execution of a series of steps, or workflow

• Kepler helps organize and automate scientific tasks, a sort of "scientific robot" • Easy for scientists to create both low-level 'plumbing workflows' to move data

around, start jobs on remote computers and create high-level data analysis pipelines chaining together standard or custom algorithms

• Researcher begins by identifying and accessing initial data sets, and proceeds through additional steps using software tools such as modeling and simulation programs, image processing programs and visualization software

• Systematic approach to scientific workflows also intended to promote communication and collaboration through publishing analyses, models, data transformation programs and derived data sets, giving scientists a way to track provenance

Paul Tooby, Bertram Ludäscher, UCSD (http://kepler-project.org/ )

Typical Hierarchical System

Sensor Sensor Instrument Instrument Sensor

Layer 2 switch

Layer 3 switch/router

LAN

ProcessProcess

Process Process Process

LAN

Data Management System

Security

VPN

USER

Instrument Pod

SoA Approach

Sensor/InstrumentWS**

LANWS*Instrument Pod

LANWS*

Log Archive Process 1WS**

Log Archive Process 2WS**

ONS15454WS* CA*net 4

WS* CANARIE UCLP

WS* New Web service

Data ManagementSystem

WS** New development

LightpathWS*

WS Process

WS**WS HPC Process

WS**

USER with Workflow software

SoA Registry

Australia’s e-Research Coordinating Committee Interim

Report

• Advances in ICT make it easier to access, diffuse, communicate and manipulate large quantities of data, information and knowledge. This is a worldwide phenomenon ...

• The ability of researchers to access data, research facilities, computational capacity and capability and to work together online, through advances in ICT, is a significant enabler.

• Australian researchers are already increasingly undertaking research in collaboration with other researchers, in response to the growing scale and complexity of research ...This is occurring on a national, international, public and private sector, and inter-disciplinary basis.

Context

Australian Report: Issues

1. Human capabilities • success of the strategic e-Research framework will be dependent on people • researchers need easy and structured ways of acquiring basic e-Research skills. • researchers need day-to-day support and high level ICT and information management support

2. Access to data • Digital data in all its manifestations is now the core of modern research and knowledge generation • Stakeholders need to adopt best practice in data management, standards and security

3. Need for linkage of e-Research resources • researchers need to seamlessly link networks, repositories, instruments and distributed computers 4. Structural and cultural change • The increasing adoption of e-Research will require change to organisational structures and cultures

5. Lack of awareness and support • take-up of ICTs and infrastructure has not been equally experienced in all disciplines.

6. More collaboration in research • Australian researchers need to increasingly collaborate in national and international research.

Australian Report

Recommendations

1. The Committee’s final report will include an implementation plan 2. As a matter of priority, the Committee will report back to Ministers on

strategies and actions aimed at addressing the skills issues 3. The e-Research strategic framework developed will address the need for an

integrated approach in undertaking further research into e-Research enabling technologies and applications

4. The Committee’s final report will specifically address the delivery of e-Research outreach, support and research activities on a national basis.

http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_issues/ncris/strategic_roadmap.htm

Thank you !Are there any questions?