The Front Range GigaPop (FRGP)

Jeff Custard

Network Engineer for the Network Engineering and Technology Section at

NCAR

CHECO Presentation September 27, 2000

The GigaPop Concept

What Is A GigaPop?

• Multiple sites agree to aggregate to a central location and share high-speed access from there, instead of each maintaining direct links to multiple networks

• Share costs through sharing infrastructure• Share Commodity Internet expenses• Essentially statistical multiplexing of

expensive high-speed resources– at any given time much more bandwidth is available to

each institution than each could afford without sharing

• Share engineering and management expertise• More clout with vendors

Front Range GigaPop (FRGP)

http://www.frgp.net

FRGP History:

• Initial concept — October 1999 time-frame (including decision to house at UCD)

• Formed because high performance network funding was going away (vBNS)

• Many discussions prior to that time regarding a “big pipe consortium” of some kind

• Abilene made the GigaPOP concept lucrative

• Equipment in place at UCD ~April 1999 (CSU and UW purchased routers)

• First Abilene connections active ~April 1999

Why NCAR as the FRGP Operator?

• NCAR already had GigaPop operational experience

• NCAR was already serving the FRGP members– Abilene connection was an incremental addition to existing

gigapop

– Didn’t require a completely new effort from scratch

• NCAR has a 24 x 365 NOC

• NCAR is “university-neutral”

FRGP: Current Services• Abilene (Internet2) access

• Shared Commodity Internet access to Qwest

• Intra-GigaPop access

• 24 x 365 NOC (Network Operation Center)

• Engineering and management (including monitoring and statistics)

• IPv6 router

• ANS’ Surveyor monitor

FRGP: Future Services

• Shared Commodity Internet access to the two state contract providers: AT&T and C&W

• Web cache servers– Akamai cache service

– Client-side cache server

• Netnews server

• Video server

• NLANR AMP machine

FRGP Advantages• Ability to aggregate and provide economies

of scale

• High-quality physical environment

• High-quality support

• Ease of passing lots of traffic among members using existing connections

• Members control the infrastructure

• Aggregate news and web feeds

• Easy, low-cost migration path to I2 connectivity

Existing FRGP connections4/24/00 PJS

FRGPCisco 1010ATM switch

UCB

UCBrouter(s)

UCBNetwork(s)

CSU

CSUrouter(s)

CSUNetwork(s)

NOAA

NOAARouter(s)

NOAANetwork(s)

NOAACisco 1010

ATM switch

NCAR

NCARRouter(s)

NCARCisco 1010

ATM switch NCARNetwork(s)

UW

UWRouter(s)

UW Network(s)

UCD

Qwestcommodity

FRGP

UCDrouter(s)

UCDNetwork(s)

4-campusCisco 1010

ATM switch

CSUCisco 1010

ATM switch

UWCisco 1010

ATM switch

UCDCisco 1010

ATM switchUCB

Cisco 1010ATM switch

FRGPcommodity

router

Abilene

AbileneIPV6

router

OC-3

DS-3

Planned FRGP connections4/24/00 PJS

FRGPCisco 1010ATM switch

UCB

UCBrouter(s)

UCBNetwork(s)

CSU

CSUrouter(s)

CSUNetwork(s)

NOAA

NOAARouter(s)

NOAANetwork(s)

NOAACisco 1010

ATM switch

NCAR

NCARRouter(s)

NCARCisco 1010

ATM switch NCARNetwork(s)

UW

UWRouter(s)

UW Network(s)

UCD

FRGP

UCDrouter(s)

UCDNetwork(s)

4-campusCisco 1010

ATM switch

CSUCisco 1010

ATM switch

UWCisco 1010

ATM switch

UCDCisco 1010

ATM switchUCB

Cisco 1010ATM switch

Abilene

AbileneIPV6

router

OC-3

DS-3

Packet Over Sonet

FRGPcommodity

routerISP2

ISP1

FRGP: Membership Types

• Primary Member– Primary I2 FRGP member

– Primary Commodity FRGP member

• Secondary Users

FRGP: Membership Types Cont’d

• Primary I2 FRGP member definition:– Must contract with the FRGP I2 Service operator (UCAR)

– Must pay the FRGP I2 Service operator the costs associated with FRGP I2 transit service

– Receives a single PVC/BGP peering session via FRGP equipment that can advertise approved Internet2 routes and can transit I2 traffic

– Has voting membership on the FRGP Management Committee (FMC)

– Has direct access to the FRGP NOC and FRGP engineers

FRGP: Membership Types Cont’d

• Secondary Users– A "secondary" FRGP user isn't a contractual member of FRGP,

but is any entity receiving FRGP services in a passthrough fashion from a primary FGRP member.

– A secondary FRGP user is not a member of FRGP and the only financial and technical ties that a secondary user has are with one or more primary members

» Service charges, if any, to secondary users are levied, and collected, solely by individual primary members

– A secondary user does not have a PVC/BGP peering session with the FRGP for either the Commodity or I2 networks

– A secondary user does not have membership on the FMC

– A secondary user does not have access to the FRGP NOC or FRGP engineers

FRGP: Initial Primary Members

• University of Colorado at Boulder

• Colorado State University

• University of Colorado at Denver

• NCAR/UCAR

• University of Wyoming

FRGP: New Primary Members

• Colorado School of Mines

• State of Colorado

• University of Colorado Health Sciences Center

• University of Denver

• University of Colorado at Colorado Springs

FRGP: Secondary Members

• Check the “Routes” list on FRGP web page for current information on FRGP secondaries

FRGP Contractual Foundation

Contractual Foundation

• UCAR is FRGP operator

• UCAR is the only legal FRGP entity

• Each Primary member has independent MOU with UCAR

– Cost agreement outlines principles of cost allocation, but amounts float from month to month

– MOU delegates some authority to FMC, such as authorization to purchase equipment and to change bandwidth amounts to service providers

– MOU terminable by either party without cause with 60-day notice

Contractual Foundation (cont.)• Primary members pay all expenses up to the

FRGP demarc (FRGP ATM switch port)• All payments for FRGP expenses are made by

primary members to UCAR• UCAR as FRGP operator handles all

subcontracts for FRGP services and disperses all FRGP funds

• UCAR bills primary members on a monthly basis, though UCAR fronts certain annual expenses such as the Abilene connect fee, and generally buffers floating expenses and floating membership numbers

Governance

• FMC: FRGP Management Committee– One voting member per MOU

– Majority decisions

• FTC: FRGP Technical Committee– Delegated by FMC to operate FTC on a daily basis and

make operational decisions

– Shared expertise among the primary members

FRGP: Documents

• Sample FRGP Agreement, DRAFT Service Level Agreement, DRAFT Operational Guidelines, etc., can be found at:

– http://www.frgp.net/docs

FRGP Cost Sharing Policies

Cost Allocation

• Circuit and service fees allocated proportional to rate-limited PVCs to primary members:

– Abilene

– Commodity

• Operational costs divided equally among primary members – Engineer salary

– Equipment maintenance

• One-time costs divided equally among primary members– Equipment

– Capital improvements

Example spreadsheet

Most recent spreadsheet is available if questions—please see me after the presentation

Useful URLs• http://www.scd.ucar.edu/nets/intro/staff/jcustard

• http://www.frgp.net

• http://www.ucaid.edu

• http://www.internet2.edu

• http://www.qwest.com

• http://www.att.com

• http://www.cwusa.com

• http://www.akamai.com

• http://www.nlanr.net

• http://www.caida.org

• http://www.advanced.org

Almost lunch time!

But then it’s the end of Fall 2000 CHECO . . .

and it’s back to work . . .

Questions?