CANARIE – CA*net 3

“The Customer Empowered Networking Revolution”

http://www.canarie.ca

http://www.canet3.net

Background Papers on Gigabit toThe Home and Optical InternetArchitecture Design AvailableOptical Internet News list:Send e-mail to Bill@Canarie.ca

Bill.St.Arnaud@canarie.cahttp://www.canarie.ca/~bstarnTel: +1.613.785.0426

The Message In mid 1990s the prevailing wisdom was that commercial sector would drive

design of Internet infrastructure R&E networks would focus on applications or specialized services

As a result in North America R&E networks were commercialized or discontinued e.g NSFnet & CA*net

However new network technologies and most importantly dark fiber is allowing R&E networks to once again redefine telecommunications not only for themselves but also for businesses and most importantly the last mile to the home

R&E networks may become the cornerstone of municipal fiber to the home networks LAN architectures, technologies and most importantly LAN economics are

invading the WAN Control and management of the optics and wavelengths will increasingly be under

the domain of the LAN customer at the edge, as opposed to the traditional carrier in the center

Over time the current hierarchical “connection oriented” telecom environment will look more like the Internet which is made up of autonomous peering networks.

These new concepts in customer empowered networking are starting in the same place as the Internet started – the university and research community.

Customer Empowered Networks Universities in Quebec are building their own 2000km fiber network Universities in Alberta are deploying their own 400 km 4xGbe dark

fiber network School boards and municipalities throughout North America are

deploying their own open access, dark fiber networks Carrier are selling “dim wavelengths” managed by customer to

interconnect dark fiber networks Williams, Level 3, Hermes

Typical cost is one time $20K US per school for a 20 year IRU In Ottawa we are deploying a 60km- 144 strand network

connecting 26 institutions – cost $1m US

Dark Fiber Builds in Quebec

Ottawa Fiber Build

Consortium consists of 16 members from various sectors including businesses, hospitals, schools, universities, research institutes

26 sites Point-to-point topology 144 fibre pairs Route diversity requirement for one member 85 km run $11k - $50K per site Total project cost $CDN 1.25 million Cost per strand less than $.50 per strand per meter 80% aerial Due to overwhelming response to first build – planning for second

build under way

Why Customer Owned Dark Fiber First - low cost

Up to 1000% reduction over current telecom prices. 6-12 month payback Second - LAN invades the WAN – no complex SONET or ATM required in network

Network Restoral & Protection can be done by customer using a variety of techniques such as wireless backup, or relocating servers to a multi-homed site, etc

Third - Enables new applications and services not possible with traditional telecom service providers Relocation of servers and extending LAN to central site Out sourcing LAN and web servers to a 3rd party because no performance impact IP telephony in the wide area (Spokane) HDTV video

Fourth – Allows access to new competitive low cost telecom and IT companies at carrier neutral meet me points Much easier to out source servers, e-commerce etc to a 3rd party at a carrier neutral collocation

facility Customers will start with dark fiber but will eventually extend further outwards with

customer owned wavelengths Extending the Internet model of autonomous peering networks to the telecom world

CA*net 4??

Concept: Industry/Government partnership to build network with scalable growth in

number of wavelengths Many wavelengths using Canadian WDM gear designed for carrying IP

only Network infrastructure to be funded up to 20 years Where possible, use RAN fiber infrastructure to connect between provinces

Three models: International wavelengths and national network: $50 - $150m National network only: $20-50m Minimal network (not coast to coast): $5 - 10m

Connect together existing dark fiber projects in provinces RANs without dark fiber may not be able to participate

Optical BGP - OBGP

Proposed new protocol where RANs ( and eventually universities) control routing of wavelengths across the network

Marriage of CA*net 2 and CA*net 3 concepts

RANs would have direct peering with each other and international peers CANARIE would offer optional aggregation and international peering where

applicable May significantly reduce cost of commodity Internet by allowing direct

peerings with many other networks (commercial and non-commercial)

O-BGP (Optical BGP) Control of optical routing and switches across an optical cloud is by the customer – not the carrier

A radical new approach to the challenge of scaling of large networks Use establishment of BGP neighbors or peers at network configuration stage for process to establish

light path cross connects Edge routers have large number of direct adjacencies to other routers Customers control of portions of OXC which becomes part of their AS Optical cross connects look like BGP speaking peers BGP peering sessions are setup with separate TCP channel outside of optical path or with a Lightpath

Route Arbiter All customer requires from carrier is dark fiber, dim wavelengths, dark spaces and dumb switches Traditional BGP gives no indication of route congestion or QoS, but with DWDM wave lengths

edge router will have a simple QoS path of guaranteed bandwidth Wavelengths will become new instrument for settlement and exchange eventually leading to

futures market in wavelengths May allow smaller ISPs and R&E networks to route around large ISPs that dominate the Internet

by massive direct peerings with like minded networks

Current View of Optical Internets

Big Carrier Optical Cloud using MPLS and IGP for management of wavelengths for provisioning, restoral and protection

Customers buy managed service at the edge

Optical VLAN

Customer

ISP

AS 1

AS 2

AS 3

AS 1AS 4

BGP Peering is done at the

edge

OBGP Optical Internets

Big Carrier Optical Cloud disappears other than provisioning of electrical

power to switches

Customer is now responsible for wavelength

configuration, restoral and protection

BGP

Customer

ISP

BGP Peering is done inside the optical

switch

BGP Routing + OXC = OBGP

Router ARouter C

AS 300190.10.10.0

AS 200180.10.10.0

AS 100170.10.10.0

3.3.3.1

3.3.3.2

1.1.1.1

1.1.1.2

BGP N

eighbor

BGP Neighbor

4.4.4.12.2.2.1

2.2.2.24.4.4.2

Router B

Metric 200Metric 200

Metric 100Metric 100

Figure 2.0

Virtual BGP Router

Router A

Router B

Router C

AS 300AS 200AS 100

2.2.2.1

2.2.2.21.1.1.1

1.1.1.2

BGP NeighborBGP Neighbor

Figure 4.0

L0 172.16.90.1255.255.255.255

170.10.10.0

180.10.10.0

L0 172.16.2.254255.255.255.255

190.10.10.0

L0 172.16.40.1255.255.255.255

3.3.3.24.4.4.2

3.3.3.1

4.4.4.1

BGP Neighbor BGP Neighbor

L0 172.16.1.254255.255.255.255

CA*net 4 – Distributed OIX

AS 549ONetAS 271

BCnet AS 376RISQ

Figure 12.0

OBGP

OBGPOBGP

New York

Chicago

Seattle

ITN first step to global matrix of OBGP wavelengths?

With ITN to support international transit? North American transit through Abilene & CA*net 3 European transit through DANTE, Nordunet,

SURFnet, etc? Asian transit through APAN?

Eventually layer 3 transit to be replaced by wavelength transit? But use OBGP for wavelength management

Discussions are underway with international carriers to acquire initial wavelengths