CANARIE Inc

September 1998

http://www.canet2.net

turcotte@canarie.cahttp://www.canarie.caTel: +1.450.671.8539

““Canada’s National Optical Internet”Canada’s National Optical Internet”

CA*net 3 World’s first national optical Internet First Internet network built from the ground up to support Internet first, voice

second All existing Internet networks are built on technology originally designed for voice -

e.g. SDH/SONET & ATM

Consortium members include Nortel, Newbridge, Cambrian, CISCO, Bell, etc

Key features: use of individual DWDM wavelengths directly coupled to routers Use intrinsic self healing capabilities of Internet and eliminate SDH/SONET and

ATM layers MPLS for layer 3 restoral, protection and traffic engineering

GigaPOP

National Optical Network

Vancouver

Calgary ReginaWinnipeg

Ottawa

Montreal

Toronto

Halifax

St. John’s

FrederictonCharlottetown

RAN

BCnet

WURCnet SRnet MRnet

ONetRISQ

ACORNOC3

OC3 DS3 OC12

OC48

ChicagoSTAR TAP

CA*net 3

OC12 Teleglobe

What is an Optical Internet? WDM fibers where individual wavelengths are the link layer

interconnect directly connected to routers via Optical ADM (Add Drop Mux) or WDM coupler

High Performance Router acts as the main switching routing device Bypass or cut-thru connections via dedicated wavelengths SONET or Gigabit Ethernet framing (also 10xGbE) Use intrinsic self healing nature of Internet for redundancy and

protection (don’t require SONET/SDH layer) Traffic engineering and network management done via MPLS Network design optimized for unique characteristics of Internet traffic

Why build an Optical Internet? Dramatic growth in IP traffic

ISPs are already starting to deploy OC-48 IP networks Customers are starting to order OC-12 IP local loops How soon before we need OC-192 or OC-768 IP??

Future trends indicate IP growth will continue IP telephony could be very, very big New Internet 2 and CA*net 2 applications

Internet characteristics significantly different than traditional telecommunications traffic

If IP is the dominant traffic then optimize network design for IP CA*net 3 will be world’s first network designed from the ground up to carry first

and foremost, Internet traffic

Traffic Growth

0

50

100

150200

250

300

350

1990 1995 2000 2005

Year

Rel

ativ

e L

oad

Data

Voice

Data is 23xVoice Traffic

Data is 5xVoice Traffic

Source:Lightwave April 1998

The real driver for Optical Internet Traditional OC-48 SDH/SONET network costs about $US 4000 - $5000 km per

year before overhead, engineering and maintenance 20 year amortization on fiber and installation 5 year amortization on optical amps, regen, SONET Mux, etc

Optical Internet with today’s technology costs about $US 500-$750 per kilometer per year

With low cost regen (e.g.10xGbE), low dispersion fiber, and long range optical amplifiers optical Internet will cost $US 100 - $200 per km per year

Optical Internet also has significantly less overhead, engineering and maintenance costs. see Engineering paper http://www.canet2.net for financial analysis

Opportunity for Canada World leader in SONET/optical networking - JDS Fitel, Nortel,

Cambrian, Positron Fiber Systems, CISCO Canada, PMC Sierra, QNX Over 75% of the world’s Internet traffic is carried on equipment made

in Canada CISCO GSR12000 SONET I/F made in Ontario -95% market Nortel Optical Transport made in Montreal - 75% market Newbridge ATM switches made in Ottawa - 50% market JDS Fitel optical components made in Ottawa -85% market

Possibility of leveraging our technology and leadership to increase export opportunities and job growth in this area

A network for basic research unparalleled anywhere in the world

Acceptable Use Policy Same AUP as CA*net 2

Any Canadian organization that is doing high performance meritorious research or applications development that cannot be carried out on the commercial Internet

CA*net 3 will only interconnect GigaPOPs One GigaPOP per province plus Ottawa - others may be added GigaPOPs interconnect to regional high speed networks

Same Tier A/B/C policy as C2 Allows CA*net 3 to peer with similar international research networks like

Abilene, vBNS, etc

All institutions must maintain separate commercial Internet connection

National IP/WDM Network

Ottawa

Montreal

Toronto

WinnipegReginaCalgary

Edmonton

Vancouver

Saskatoon

- CANARIE Drop Site Halifax

Fredericton

Chicago

Additional OC-192 WDM Routes for future use

4/BLSR

4/BLSR

4/BLSR

CANARIE OC-192 Route

CANARIE OC-48 Route

Charlettown

St. John’s

Teleglobe

8 Wavelengths per route4 reserved for traditional SONET 4/BLSR by carrier

Optical Internet Architecture

High PriorityTrafficCannot exceed50% of bandwidthin case of fiber cut

3 0C-48 Tx2 OC-48 Rx

Both sides of 4/BLSR 1:1 span ring used for IP traffic

Low priority trafficthat can be bufferedor have packet lossin case of fiber cut

AsymmetricTx/Rx lambdasthat can bedynamicallyaltered

WDM WDM

Traditional SONET Restoral

Traditional SONETMux or DCSTraditional SONET

Mux or DCS

Layer 3 Restoral IP network is intrinsically self healing via routing protocols By cranking down timers on interface cards and keep alive message time-out we

can achieve same restoral speed as SONET Biggest delay is re-calculation and announcement of changes in routing tables

across the network MPLS promises to simply the problem

maintain a set of attributes for restoral and optimization may provide a consistent management interface over all transport services -WDM,

SONET/SDH, ATM, Frame Relay, etc

Layer 3 restoral allows for more intelligent restoral can use a hybrid mix of restoral and protection circuits Can use QoS to prioritize customers and services Only UDP packets (e.g telephony) require fast restoral allows simultaneous use of both working and protection circuits

Regional Optical Network

University BATM

GigaPOPRouter

To CA*net 3To Commercial Internet

Packet overSONET

GigaBit Ethernet

University ARouter

OADMOXC?

EthernetLocal WDM Fiber Ring Provided by Cable Company or Telco

ATM

OADM

OADMOADM

Dual RedundantPaths - can be switchprotected or dual path

Central Office

AnalogVideo

Reuse of samewavelength

Example Physical Layer

CSI RoutePolicy Server

CA*net 2

Distributed Municipal GigaPOP

ResearchInstitute B

Research Institute A Telco

CommunityCollege

DWDM

DWDMATM

ATM

ATM

CA*net 3

University A

University B

Wireless

Wireless RAN

Example IP Layer

CSI RoutePolicy Server

CA*net 2

Distributed GigaPOP

RegionalGigaPOPInstitutional

GigaPOPTelco

CommunityCollege

OSPFOSPF

iBGP

OSPF

CA*net 3

RAN

University A

University B

Wireless

Wireless

iBGP

PNNI

AS ##2AS ##1

AS ##3

Daughter Cachex.x.x.x/

X.x.x.x/

BGP Confederation ###

eBGP

Intermediate Cache

High Schoolor CAP site

X.x.x.x/

X.x.x.x/

X.x.x.x/

X.x.x.x/

X.x.x.x/

Optical Internet Exchange

ISP C

Common Internet Exchange Router

Packet overSONET

ISP AEthernetATM

OADM

OADMOADM

OADM

ISP B

Web Server

Small ISPs

Optical Internet ExchangeLogical Diagram

ISP C

Common Internet Exchange Router

ISP A

ISP B

Web Farm

Small ISPs

Gigabit Ethernet Framing Gigabit Ethernet Framing advantages

frame size = packet size therefore packet switching and SAR more efficient and easier to implement

data format consistent with LAN format with no translation low cost tributary service - do not need to terminate link on a router or SONET DCS

equipment new 10xGigabit Ethernet will equal OC-192 standard SNMP MIBs, but not accessible by out of band interoperable standard from many vendors No scrambling sync or packet loss

Gigabit Ethernet Framing disadvantages not very efficient with 8B/10B block coding

new 10xGigabit Ethernet may use more efficient coding

No standard out of band management or monitoring But some WDM suppliers provide this

SONET Framing SONET framing advantages

well established jitter specifications out of band management systems can be used in SONET networks for fast restoral and protection very high efficiency - over 98%

SONET framing disadvantages no interoperable standard SAR processing more complex as there can be multiple packets per frame, or

packets can cross frame boundaries tributary services require SONET mux services no well established carrier network management protocols for fault detection

and location, especially on long haul when SONET used in independent links

Future Optical InternetIntegrated Transport Services

Different Protocol Stacks Integratedto provide different size bandwidthpipes and CoS

HDWDMOC-3084

OXC

OXC OXC

ADMADM

ATM/IP Network

IP SONET Network

IP Optical Network

OADM

ADM

IP/ATM Network

OADM

ADM

IP SONET Network

IP over ATM IP OpticalIP Sonet

QoS & VPNs up to OC3 OC3, OC12, OC48 Greater than OC-48

Future Optical InternetMPLS as common management layer

OC3, OC12

IP over ATM IP OpticalIP Sonet

QoS & VPNs up to OC3 OC-48, OC-192

DWDM LSP

ATM VCs

SONET LSP

SONET LSP

ATM VCs