Internet2 UpdateR/D and Infrastructure

Guy AlmesInternet2 Project

<almes@internet2.edu>

NANOG MeetingDearborn — 9 June 1998

Outline of the TalkTechnical Working GroupsThe Challenge of Delay-

Bandwidth ProductsAbilene Project Update

Applications and Engineering

Applications

Engineering

Motivate Enables

Internet2 Working Groups - Presentation to JET

Comments on Apps and Plumbing

Advanced applications transform high-speed plumbing into value

Advanced plumbing enables advanced applications

Profligate use of bandwidth, per se, does not make an application ‘advanced’

Megalomaniac plumbing, per se, does not make the plumbing ‘advanced’

Technical Working Groups IPv6 Measurement Multicast Network

Management Network

Storage

Quality of Service

Routing Security Topology

IPv6 Chair: Dale Finkelson, Univ Nebraska

<dmf@unl.edu> Membership: Total 12; 9 .edu, 3 .com,

1 .gov Focus:

Explore the rôle that IPv6 might play in the Internet2 project

Work with those interested in IPv6 to build IPv6 testbeds across the Internet2 structure, including vBNS and Abilene

MeasurementChair: David Wasley, Univ California

<david.wasley@ucop.edu>Focus:

Places to measure: at campuses, at gigaPoPs, within interconnect(s)

Things to measure traffic utilization performance: delay and packet loss traffic characterization

One example measurement technology

IETF IPPM WG defining one-way delay

Take all delay to be due to: Propagation Transmission Queuing

Variation in delay suggests congestion

MulticastChair: vacant [Dave Meyer, Univ

Oregon still serving] Nearing completion of naming a

successorMembership: Total 3; 3 .eduFocus: Make native IP multicast

scalable and operationally effective

Network Management

Chair: Mark Johnson, MCNC <mj@ncren.net>

Membership: Total 4; 3 .edu, 1 .comFocus:

Common trouble ticket system How can all our interconnects and

gigaPoPs and universities appear to be a seamless whole?

Network Storage

Chair: Micah Beck, Univ Tennessee <mbeck@utk.edu>

Membership: Total 13; 9 .edu, 4 .comFocus: Distributed Storage

Infrastructure for Internet2 Replication Physical proximity Transparency

Quality of ServiceChair: Ben Teitelbaum, Internet2 staff

<ben@internet2.edu>Membership: Total 36; 17 .edu,

19 .comFocus: Multi-network IP based QoS

Relevant to advanced applications Interoperability: carriers and kit Scalable Administratable and Measurable Hosts, campus/gigaPoP/Interconnect

routers/switches

Quality of Service Sketch

• Does the approach support advanced applications?• Are there implementations that work? Only one?• If cloud ‘A’ and cloud ‘B’ both implement QoS, does the combined A+B catenation implement QoS?

A B

QoS, continued

Results to date: Requirements document Series of technical recommendations

First Internet2 Joint Applications/ Engineering QoS WorkshopSanta Clara, CaliforniaMay 21-22, 1998Hosted by Bay Networks

Routing Chair: Steve Corbato, Univ Washington

<corbato@cac.washington.edu> Membership: Total 48; 32 .edu, 16 .com Focus: Internal and External routing

Critical issues gigaPoP internal routing design

Explicit routing requirement (the “fish problem”) Met at UCSD in January (21 attendees)

gigaPoP external routing recommendations Subscribers (Internet2 campuses) National interconnects (vBNS, Abilene, and NGI networks)

SecurityChair: Peter Berger, Carniege

Mellon Univ <peterb@hoopoe.psc.edu>

Membership: Total 13; 13 .eduFocus:

Authentication Application to QoS Application to Digital Libraries

TopologyChair: Paul Love, Internet2 staff

<epl@internet2.edu>Membership:

Total 16; 13 .edu, 2 .com, 1 .govFocus: Topology of Internet2

Internal Internet2 Connections Internet2 with other Advanced

Research Networks

Summary Internet2’s WGs focus on

project’s needsComplement IETF WGsMembership by invitation -

welcome participation by Internet2 corporate members

Large Delay-Bandwidth Products

As the product of delay and bandwidth grows: The number of unacknowledged packets grows It becomes more difficult to sustain a steady

stream of data from end to endSeveral consequences:

Need for direct physical paths Tradeoff between buffering and

variation in delay

A pessimistic result from Mathis et al.

Mathis, Semke, Mahdavi, and Ott, "The Macroscopic Behavior of the TCP Congestion Avoidance Algorithm", Computer Communication Review, July 1997.

www.psc.edu/networking/papers/model_abstract.html

BW C * packet-size / (delay * packet-loss)

Consider the implications for the international

high-performance InternetBW packet-sizeBW 1 / delayBW 1 / packet-loss

Example: Delay

BW C / delay

delay due to distance

original raw bandwidth

Example: Delay with fatter pipe

BW C / delay

delay due to distance

more raw bandwidth

Example: Packet Losssimilar phenomenon, but … to double bandwidth, you mustcut packet loss by four

Abilene Update

UCAID Project Addresses infrastructure needs of

Internet2

Goals and Objectives

Provide high-quality, widely available Interconnect among participating gigaPoPs/universities

Connect to Internet2 members via the vBNS and to other key research/ education sites via Internet2/NGI-class federal and non-US nets

Goals and Objectives, continued

Support QoS architecture as it evolves

Support other advanced functionality as it evolves

Maximize Robustness Minimize Latency Provide Capacity to Avoid Congestion

Evolution of Abilene with Time

Phase 1: use of operational Qwest Sonet Phase 2: use of separate wavelengths Phase 3: use of separate fibers

Allows capacity to grow with Internet2 needs

Key Attributes

IP over Sonet Benefit from Qwest OC-48 Sonet

capacity and collocation sites Benefit from Nortel OC-192 Sonet kit and

Lucent fiber Benefit from Cisco GSR 12000 routers

Architecture: Core

About 11 (up to 30) core nodes Each located at a Qwest PoP Each with a Cisco 12008 router Rack also contains measurements/ management computers

Interior lines connect core nodes OC-12 and (eventually) OC-48 Sonet IP-over-Sonet interfaces

sttl

eugn

scrm

anhm

phnx

albq

hstn nwor tlhs

atln

rlgh

wash

phil

nycm

bstnsyrc

clev

pitb

dtrtmilw chcg

mpls

kscydnvrslkc

ipls

lsvl

nsvlrcmd

Subset of Route Map of Interest to Abilene

tpka

alby

elpa

Attitude toward interior lines

Robustness: mesh plus Sonet

Latency: direct physical paths

Capacity: avoid congestion

Architecture: Access Access node at many Qwest PoPs

Qwest Sonet switches needed equipment

Access lines connect from core node to gigaPoP Local part: gigaPoP to access node Long distance part: access node to core node IP-over-Sonet or IP-over-ATM possible OC-3 and OC-12 typical

One cost-sharing implication

Long-distance part of access line is considered part of the ‘backbone’

Thus, number/location of core nodes does not affect costs borne by gigaPoP

One robustness implication

Each access line is Sonet Long-distance part (at least) will be

configured from protected Sonet ring

Thus, single access line can tolerate a break in the long-distance part of the access line

OK, so where’s the map?

Self-selection is key Each gigaPoP will determine where,

when, at what speed it connects

Detailed topology will be based on engineering considerations