Next Generation R&E InfrastructureNext Generation R&E Infrastructure

Steve Corbató, Internet2/Univ. of Utah

CIO ForumInternet2 Member Meeting

Washington, D.C.May 4, 2005

Steve Corbató, Internet2/Univ. of Utah

CIO ForumInternet2 Member Meeting

Washington, D.C.May 4, 2005

TopicsTopics

• Landscape• Lightpaths & GLIF

• Overview• Abilene• NLR• Regionals• International

• Next generation architecture• HOPI testbed• Design timeline and process

What’s different now?What’s different now?

•Over the last 5 years, the U.S. research universities have become wholesale customers of telecoms for raw assets & ancillary services•Upwards of 30,000 route-miles of inter-city dark fiber are held by this community• > 60% in the regionals

•More than 20 Regional Optical Networks (RONs) have emerged • mostly state based• many with strong gubernatorial support• often concurrent economic development objectives

• All at a time when carriers have not made major CapEx in new optical infrastructure

What’s different now?What’s different now?

• Increasing end-to-end network transparency• A high-end server is capable of transmitting aggregate flows of O(10) Gbps on the global scale• A well-tuned IP network can sustain this flow

•Grid computing wants to view the network as a schedulable resource•On par with CPU, memory, storage, and visualization capabilities

•Active examination of new network service models is underway•Moving beyond the common bearer service view of best-effort IP

A preview of the U.S.’s coming Internet ‘nuclear winter’ ?A preview of the U.S.’s coming Internet ‘nuclear winter’ ?

NORDUNet 2005 – Svalbard, Norway – April 2005

LightpathsLightpaths

• Essentially a ‘new age’ circuit• Can be ‘customer’ (ie, non-telco) provisioned• SONET or Ethernet framing

• Various forms of provisioning end-to-end, dedicated capabilities• MPLS tunnels (through IP networks)• SONET circuits• Ethernet channels• Dedicated wavelengths (λ’s)

• Potential motivations• Guaranteed bandwidth, latency/jitter• Security/privacy

Global Lambda Integration Facility (GLIF)Global Lambda Integration Facility (GLIF)

Abilene BackboneAbilene Backbone

Abilene in a nutshellAbilene in a nutshell

• 10-Gbps ‘best effort’, over-provisoned IP network• Current normal load < 2 Gbps

• Carrier provisioned backbone λ’s • ~3.4 9’s availability

• SONET backhaul available to connectors• IPv4/IPv6, native multicast, MPLS• Research facilitation (data + colo)• Abilene Observatory project

• Extensive domestic and int’l R&E peering• Cost recovery model incents utilization and

bandwidth upgrade

Generic NLR Node Layout –Generic NLR Node Layout Layer 1, 2, and 3 Capabilities

–Layer 1, 2, and 3 Capabilities

CRS-1

Colo

15808

6509

15454West East

NLR demarc

DWDM

1G wave, link or port10G wave, link or port

State and Regional Optical NetworksState and Regional Optical Networks

• Alabama• Arkansas• California (CALREN)• Colorado (FRGP/BRAN)• Connecticut (Conn. Education

Network)• Florida (Florida LambdaRail)• Georgia (Southern Light Rail)• Indiana (I-LIGHT)• Illinois (I-WIRE)• Louisiana (LONI)• Maryland, D.C. & northern

Virginia (MAX)• Michigan (MiLR)• Minnesota

• New England region (NEREN)• New York (NYSERNet, Cornell)• North Carolina (NC LambdaRail)• Ohio (Third Frontier Network)• Oklahoma (OneNet)• Oregon• Pacific Northwest (Lariat – NIH

BRIN, PNNL)• Rhode Island (OSHEAN)• SRON (southeastern U.S.)• Tennessee (OneTN)• Texas (LEARN)• Virginia (MATP)• Wyoming

U.S. International Exchange PointsU.S. International Exchange Points

•Critical peering junctions and advanced service experimental platforms for R&E nets• Pacific Wave – West Coast

• Distributed: Seattle, Los Angeles, and soon San Francisco Bay Area

• Star Light - Chicago•MAN LAN - New York City• AMPATH - Miami• Atlantic Wave – East Coast

• Distributed: NYC, D.C., Atlanta, and Miami – soon

• Increasingly co-located at major ‘carrier hotels’•Significant recognition of regional/international leadership through recent NSF IRNC awards

HOPI General ProblemHOPI General Problem

HOPI - ResourcesHOPI - Resources

• Design Team and Corporate Advisory Team• Abilene Network • MPLS tunnels and the packet switched network

• Internet2’s 10-Gbps λ on the NLR national footprint

• MAN LAN exchange point in New York City• International 10-Gbps λ ‘s (SURFnet & GEANT2)

• NYC-Amsterdam (IEEAF) and NYC-London • Layer 1 and 2 switching

• Collaborations with U.S. Regional Optical Networks (RONs) and other related efforts• E.g., GLIF, UltraLight, DRAGON

HOPI NodeHOPI Node

HOPI TopologyHOPI Topology

Abilene Network futuresAbilene Network futures

• October 2007 – End of recent 1-year extension to Abilene transport MoU with Qwest• Provides 3rd-generation network timeline

• Buildable architecture: end 4Q05• Transport arrangement(s): end 1Q06• Router and other equipment selection: end 2Q06• Backbone deployed: end 4Q06• Connector transition: 2007

• Concurrently, review service, business, and management model

• Coincidentally…• Late 2007 – expiration of ESnet transport agreement with

Qwest

Next Generation National Network DesignNext Generation National Network Design

•Critical factors•RON and international integration •Advanced service support• (Multicast, v6), High Performance Throughput, Measurement

•Enhanced network research facilitation•Network and end-user security•Options for increased reliability

Next Generation Network Design ProcessNext Generation Network Design Process

•Hybrid architecture evaluation (HOPI)• IP core using 40?/10 Gbps λ transport•Dedicated capabilities (λ’s, MPLS tunnels)

•Evaluation of optical transport capabilities

•NLR, commercial providers & RONs

•Design & planning collaboration with U.S. & international partners

•ESnet, TeraGrid, NLR, SURFnet, GEANT2

Evolving optical transport requirementsEvolving optical transport requirements

•Core wavelengths• 10 and hopefully 40 Gbps•Optimized availability• Applications: IP network backbone and backhaul, weather forecasting grids, radiological image transfer

•Flexible wavelengths• 10, 2.5, and 1 Gbps• Less stringent reliability requirements, lower cost• Potentially short duration (rapid provisioning and tear-down)• Applications: Grid clusters, IP network overflow, network research projects, highly redundant IP networks

Service (re)differentiationService (re)differentiation•Potential spectrum of services and capabilities•Dark fiber, wireless spectrum•Wavelengths• Subchannels (GigE channels, SONET, MPLS)• IPv4/v6•Overlay network support

•Period of market evaluation and winnowing?•Need for new model of customer support and end-to-end connectivity delivery assurance•Working across campus, regional & national scales• Effective campus penetration of new services is a critical issue

Implications for campus networksImplications for campus networks

• We’ve spent a lot of time the last 4 years on national and regional networks

• But what about the campus?• Just ask Bill Decker and the CETF! (talk #3)• Emerging suite of differentiated services has

significant implications for the campus• Provisioning and support models• Campus colocation (‘co-generation’) facilities

• Fertile ground for hard work• Planning for workshop this fall

Consultation (current and near term)Consultation (current and near term)

• Internet2 governance• Board• Advisory Councils - NPPAC, ASC, ISC, NRLC• Campus Expectations Task Force

• Internet2 technical advisory groups• Abilene TAC (Core IP service)• HOPI Design Team (Hybrid network)• SALSA (Security)• E2E TAG (Performance)• MACE (Middleware)

• National LambdaRail (NLR)• Board

• Advanced Infrastructure Community• The Quilt and its membership• Abilene Community - campuses, connectors, partners• Emerging State and Regional Optical Networks

• Network Researchers (via NRLC and NLR NRC)

Currently planned engagement opportunitiesCurrently planned engagement opportunities

• Quilt Fiber Workshop, Providence: 1-3 June• Joint Techs Workshop, Vancouver BC: 17-21

July• Side workshop for Abilene technical community

• SAC, Snowmass: 7-10 August• Fall Member Meeting, Philadelphia: 19 – 22

September• Campus-focused workshop (in planning):

early fall • Other ideas?

ConclusionConclusion

•Thanks to the relentless acquisition and innovative infrastructure development efforts of the last few years, we have the opportunity to build a new form of service-rich network from the ground up•Not just reinventing the telco way• The real work has just begun…•Critical: a commitment to open standards and interoperability as new services develop from the crucible of testbeds and advance capabilities