CANARIE

“Community Condo Fiber Networks”The Customer Empowered Networking Revolution

http://www.canarie.ca

http://www.canet3.net

Bill.St.Arnaud@canarie.caTel: +1.613.785.0426

David.Macneil@canarie.caTel: +1.613.943.5377

Mission: To facilitate the development of Canada’s communications infrastructure and stimulate next generation products, applications and services

Canadian equivalent to Internet 2 and NGI private-sector led, not-for-profit consortium consortium formed 1993 federal funding of $300m (1993-99) total project costs estimated well over $600 M currently over 140 members; 21 Board members

CANARIE Inc

Canada & the Optical Age World leader in optical networking - JDS Fitel, Nortel, Cambrian, Positron

Fiber Systems, CISCO Canada, PMC Sierra, QNX, Tropic, Edgeflow, Accelight, Lumenon

Over 75% of the world’s Internet traffic is carried on equipment made in Canada

Nortel Optical Transport made in Montreal Newbridge ATM switches made in Ottawa JDS Fitel optical components made in Ottawa CISCO GSR12000 made in Ottawa, Toronto and Vancouver

CA*net 3 - A network for basic research unparalleled anywhere in the world Canada could be poised to be a world leader in the “optical age”

Silicon Valley was the capital of the “silicon age” Britain was the capital of the “industrial age”

Canada has the critical industrial base for the optical age

GigaPOP

CA*net 3 National Optical Internet

Vancouver

Calgary ReginaWinnipeg

Ottawa

Montreal

Toronto

Halifax

St. John’s

FrederictonCharlottetown

ORAN

BCnet

Netera SRnet MRnet

ONet RISQ

ACORN

ChicagoSTAR TAP

CA*net 3 Primary Route

Seattle

New York

CA*net 3 Diverse Route

Deploying a 4 channel CWDM Gigabit Ethernet

network – 400 km

Deploying a 4 channel Gigabit

Ethernet transparent optical DWDM–

1500 km

Multiple Customer Owned Dark Fiber

Networks connecting

universities and schools

16 channel DWDM-8 wavelengths @OC-192 reserved for CANARIE-8 wavelengths for carrier and other customers

Consortium Partners:Bell Nexxia

NortelCisco

JDS UniphaseNewbridge

Condo Dark Fiber Networks

connecting universities and

schools

Condo Fiber Network linking all

universities and hospital

CA*net 3 & Community Networks

E-research grids Researchers will want to use computing resources of

schools and homes SETI@Home New grid projects in bio-informatics, pharmaceutical

research, particle physics need access to millions of computers

Democratization of research Next big discovery in cancer or particle physics could be

made at your local high school

Customer Empowered Networks School boards and municipalities throughout North America are building

condominium dark fiber networks in partnership with next generation carrier

Individual institutions – the customers – own and control their own strands of fiber

Fiber are configured in point to point private networks; or Connect to local ISP or carrier hotel Private sector maintains the fiber

Low cost LAN architectures and optics are used to light the fiber These new concepts in customer empowered networking are starting in the

same place as the Internet started – the university and research community. Customers will start with dark fiber but will eventually extend further

outwards with customer control and ownership of wavelengths Extending the Internet model of autonomous peering networks to the

telecom world

Examples of CENCustomer Empowered Networks

Universities in Quebec are building their own 3500km “condominium” fiber network in partnership with 6 next gen carriers- $US 2million

Will deploy and manage their own optics and long haul transmission gear Universities in Alberta are deploying their own 400 km 4xGbe dark fiber network

- $US 200K Deploy and manage their own optics and long haul transmission gear

Chicago is building a fiber networking linking all public sector institutions - $US 250m

City of Montreal is second most fibered city in the world because of municipal owned open access conduit

In Ottawa is deploying a 85km- 144 strand “condominium” network connecting 26 institutions – cost $1m US

Peel County – Missassuaga & Brampton has built a 200km public sector fiber network - $US 5m

Many other cities including Ashland OR, Halifax, Fredericton, Toronto are looking at similar initiatives

Market Drivers First - low cost

Up to 1000% reduction over current telecom prices. 6-12 month payback Second - LAN invades the WAN – no complex SONET or ATM required in

network Network Restoral & Protection can be done by customer using a variety of

techniques such as wireless backup, or relocating servers to a multi-homed site, etc

Third - Enables new applications and services not possible with traditional telecom service providers Relocation of servers and extending LAN to central site Out sourcing LAN and web servers to a 3rd party because no performance impact IP telephony in the wide area (Spokane) HDTV video

Fourth – Allows access to new competitive low cost telecom and IT companies at carrier neutral meet me points Much easier to out source servers, e-commerce etc to a 3rd party at a carrier

neutral collocation facility

What is condominium fiber? A number of organizations such as schools, hospitals, businesses and universities get together

to fund and build a fiber network Carrier partners are also invited to be part of condominium project

Several next generation carriers and fiber brokers are now arranging condominium fiber builds IMS, QuebecTel, Videotron, Cogeco, Dixon Cable, GT Telecom, etc etc

Fiber is installed, owned and maintained by 3rd party professional fiber contractors – usually the same contractors used by the carriers for their fiber builds

Each institution gets its own set of fibers, at cost, on a 20 year IRU (Indefeasible Right of Use) One time up front cost, plus annual maintenance and right of way cost approx 5% of the

capital cost Institution lights up their own strands with whatever technology they want – Gigabit Ethernet,

ATM, PBX, etc New long range laser will reach 120 km

Ideal solution for point to point links for large fixed institutions Payback is usually less than 18 months

• Province wide network of condominium fiber to 420 communities in Alberta

• Guaranteed cost of bandwidth to all public sector institutions• $500/mo for 10 Mbps, $700/mo for 100 Mbps

• Network a mix of fibre builds and existing supplier infrastructure (swap/buy/lease)

• Condominium approach: All suppliers can• Buy (or swap) a share of the fibre (during build or after)• Lease bandwidth at competitive rates

• GOA has perpetual right to use (IRU) • Ownership will be held at arms length• GOA/stakeholder rates are costs to run divided over users• Because of fibre capacity, bandwidth can be made available to

businesses at urban competitive rate• Total cost $193m• Bell Intrigna prime contractor

Alberta SUPERnet

Extended Area• 372 communities• GOA/stakeholder needs• Proceeds from

businesses (urban benchmarked rates) to GOA to further network

Base Area• 48 communities• GOA/stakeholder needs• Business proceeds to Bell

(urban benchmarked rates)

- $143 Million GOA

- 100% GOA IRU

- $50 Million

GOA

- 33%GOA IRU

- $102 Million

Bell

- 67% Bell IRU

Alberta SUPERnet IRUs

Combination:

• Fibre build

• Use of Existing Infrastructure

Communities with one or more school

Bell Legend

Extended Fibre Network

Base Fibre Network

Backhaul From Other Vendor

BUFFALO TRAILS SCHOOL DIVISION

Clandonald

Tulliby Lake

Lloydminster Public School District # 1753

Innisfree

VermilionKitscoty

Mannville

Paradise Valley

Dewberry

Chauvin

Edgerton

Hardisty

Amisk

Wainwright

Irma

St. Paul Education Regional Division # 1

Elk Island Public Schools Regional Division # 14

Golden Hills Regional Division # 15

Aspen View Regional Division # 19

Clearview School Division # 71

Prairie Land Regional Division # 25

Battle River Regional Division # 31

Northland School Division # 61

Coronation

Elk Point

Killam

Provost

Sedgewick

Two Hills

Vegreville

Viking

Alliance

Consort

Czar

Forestburg

Galahad

Hairy Hill

Heisler

Holden

Lavoy

Marwayne

Myrnam

Strome

Veteran

Willingdon

Alberta SUPERnet Example

Current (Typical)Residences

• 56 Kbps dial Internet ($85/Month)

• No high speed Internet

Businesses

• Some T1 Facilities ($2000/Month average - rates distance sensitive)

• Some high speed business service on special setup arrangement

Future (Everywhere)Residences• High speed DSL residential

Internet at urban rates ($40/month)

Businesses• High speed business services

available at competitive urban rates (eg $820/month - T1)

• Higher speeds at comparable rates

RURAL COMMUNITIES

Alberta SUPERnet Impact

•CivicNet - A City-Wide Condominium Fiber Project •connecting up 1600 public sector institutions•Oriented to Development of Backbone Infrastructure•With Gateways to Tributary Systems•More Fiber in More Places Faster•Ubiquitous, Pervasive: 1,600 Locations•E-Z High-Performance Low-Cost Internet Connectivity•Foundation = Existing City Fiber Builds

Chicago CivicNet

Fredericton, N.B.

“At this morning's quarterly Mayor's Business Breakfast, City of Fredericton Mayor A.M. (Sandy) DiGiacinto released details about the high-speed fibre optic cable which will be used to connected the new NRC E-Commerce Institute (UNB campus) to the Greater Fredericton Knowledge Park. Staff have been instructed by City Council to prepare a business case that would leverage this 3 km stretch of cable into an Ultra High-Speed Community Network managed by the municipality.”

Nov. 2000

Fredericton

Started as Economic Development tool MUSH, Govt., Research - ISP, carriers

invited to participate Build partners emerged quickly, $50,000

“donated” by three firms Contracting now for 8 km phase 1,

$110,000, complete Sept 2001 48 fiber min.

À venir

Bande passante louée

Projet démarré

Construit

Observatoire Mont-Mégantic

Val d’Or/Rouyn

MAN de Montréal

MAN de Québec

MAN de Sherbrooke

MAN d’Ottawa/Hull

Quebec University Condo Network

St-Laurent/Vanier

Lanaudière

Maisonneuve

Marie-Victorin

Champlain

Rosemont

Sorel-Tracy

Montmorency

Édouard-Montpetit

Vieux-Montréal

Bois-de-Boulogne

Ahuntsic

Lionel-Groulx

Vers Québec

Gérald-Godin

John-Abbott

André-Laurendeau

Dawson

À venir

Bande passante louée

Projet démarré

Construit

Montreal Public Sector Condominium Networks

School Board

Stud

y

Eng

inee

ring

Con

stru

ctio

n

PROJECT

Capitale x 140 km of fibre optics 80 schoolsRégion-de-Sherbrooke x 180 km of fibre optics 66 schoolsRivière-du-Nord x 175 km of fibre optics 52 schoolsSeigneurie-des-Mille-Iles x x x 200 km of fibre optics 80 schools 4 partnersAmiante x x x 12 km of fibre optics 9 schoolsLaval x x x 170 km of fibre optics 111 schools 3 partnersSaint-Hyacinthe x 250 km of fibre optics 51 schoolsAffluents x x x 170 km of fibre optics 70 schools 4 partnersBois-Francs x x x 60 km of fibre optics 12 schools 4 partnersDraveurs x x 90 km of fibre optics 40 schoolsGrandes-Seigneuries x 210 km of fibre optics 58 schoolsHautes-Rivières x 250 km of fibre optics 54 schoolsLaurentides x 200 km of fibre optics 35 schoolsPatriotes x 2 km of fibre optics 3 schoolsPremières-Seigneurie x 190 km of fibre optics 73 schoolsSamares x 460 km of fibre optics 72 schoolsTrois-Lacs x x 45 km of fibre optics 15 schoolsChemin-du-Roy x 29 km of fibre optics 11 sitesMarie-Victorin x 6 km of fibre optics 5 schoolsSir-Wilfrid-Laurier x x x 92 km of fibre optics 20 schools

List of School Board Fiber Builds

South Dundas IROQUOIS

MORRISBURG

South Dundas Results

Morrisburg , Iroquios Have Fibre Hung Electronics In and Fibre Lit ISP’s , ASP’s all Want In the Fibre Major Employers Inquiring Very Positive Attitude in Community Digital Desert to Digital Oasis

Peel County Municipal Fiber Network

Mississauga, Brampton, Pell 200 km of Fibre 96 strand backbone

“Enough for small country”

12-60 strands elsewhere 12,000 strand-kilometers

Laid end-to-end = Victoria to St. John’s …...and back again

Ottawa Fiber Condominium

Consortium consists of 16 members from various sectors including businesses, hospitals, schools, universities, research institutes

26 sites Point-to-point topology 144 fibre pairs Route diversity requirement for one member 85 km run $11k - $50K per site Total project cost $CDN 1.25 million Cost per strand less than $.50 per strand per meter 80% aerial Due to overwhelming response to first build – planning for second

build under way

Typical Fiber Capital Costs

Average total cost between $7 and $15 per meter as follows: Engineering and Design:

$1 - $3 per meter for engineering, design, supervision, splicing Plus Installation:

$7 to $10 per meter for install in existing conduit; or $3 to $6 per meter for install on existing poles

Plus Premise termination: Average $5k each

Plus cost of fiber: 15¢ per strand per meter for 36 strands or less 12¢ per strand per meter for 96 strands or less 10¢ per strand per meter 192 strands or less 5¢ per strand per meter over 192 strands

Condo Fiber Costs - Examples Des affluents: Total cost $1,500,000 ($750,000 for schools)

70 schools 12 municipal buildings 204 km fiber $1,500,000 total cost average cost per building - $18,000 per building

Mille-Isles: Total cost $2,100,000 ($1,500,000 for schools) 80 schools 18 municipal buildings 223km $21,428 per building

Laval: Total cost $1,800,000 ($1,000,000 for schools) 111 schools 45 municipal buildings 165 km $11,500 per building

Peel county: Total cost $5m – 100 buildings Cost per building $50,000

Typical Payback for school(Real example – des affluents – north of Montreal)

Over 3 years total expenditure of $1,440,000 for DSL service Total cost of dark fiber network for 75 schools $1,350,000 Additional condominium participants were brought in to

lower cost to school board to $750,000 School board can now centralize routers and network servers

at each school Estimated savings in travel and software upgrades

$800,000 Payback typically 8 –16 months Independent Study by Group Secor available upon request

Before

After fiber

fiberAntennas 780Novell Servers 82 1SQL Servers 13 3Lotus Notes Servers 21Tape Backup Servers 12 4Ethernet switches/hubs 10 98Routers 1083Cache/proxy (Linux) 120Fire walls (Linux) 11

Reduction in the number of servers

Condo fiber for Business

Significant reduction in price for local loop costs No increase in local loop costs as bandwidth demands increase Ability to outsource LAN and web servers to distant location as LAN

speeds and performance can be maintained over dark fiber Access to lower cost competitive service providers at carrier neutral

hotels New entrants cannot afford high cost of building out their own

fiber networks Even small businesses with less than 20 employees can realize

significant savings and benefits Examples:

Colgate-Palmolive build in Cincinnati Nortel, Cisco, Gov’t depts in Ottawa

Advantage of Condo Fiber

CentralOffice

CentralOffice

Today: Customer pays 2 telcos for SONET connections

Carrier managed SONET ring

Customer Owned Dark FiberLong reach lasers

SONET Mux and ADM

ISP ISP

ISPISP

Tomorrow: Multiple Customer owned dark fiber links to ISPs

$50K one timeUnlimited Bandwidth

$50K one timeUnlimited bandwidth

Monthly costFixed Bandwidth

Condo fiber for cities In downtown core minimizes digging up streets

If N carriers are trying to deploy service then number of times roads has to be torn up is N squared

However with condominium fiber road only has to be torn up once

Produces a competitive market place and level playing field New competitive carrier can meet customers at carrier neutral collocation facilities Eliminates market advantage of incumbents

In suburban areas eliminates duopoly of cable and TV companies The first company to install fiber into suburban neighborhoods will likely have a

natural monopoly In Stockholm home owners have a choice of 4 cable companies

Makes cities a much more attractive place for new high tech businesses and service

Facilities based competition in the residential neighborhood?

Facilities based competition is alive and well in downtown core The biggest challenge for governments is manage and coordinate the

digging up of streets Outside of downtown in big cities

Usually only a monopoly telecom provider At best a duopoly

How do we introduce facilities based competition into this market (or at least come as close as possible to true facilities based competition)?

As well how can we assure scalable high speed Internet services to the home that eventually will support Gigabit speeds or higher?

Community Fiber Architecture A community consortia would put together a plan to fiber up all public sector

buildings in their community A community can be a province, a municipality, village, etc

A fiber splice box that terminates the fiber at the street side nearby each public sector building such as school, hospital, library is called a “Node”

Community should must insure that potential facilities exist near the for private sector equipment to connect up future home owners – colo facility

Colo facility allows private sector to extend wireless, VDSL or HFC services to the neighbourhood around the school

Public sector buildings will have dedicated fiber strands that connect to a “Supernode” which is a fiber splice box on the street beside outside of major public sector central facility such as school board office, city hall, university, etc

Community should insure that facilities exist nearby the Supernode for the private sector to install equipment to service home owners and businesses – colo facility

Additional fibers are made available from the Supernode to all Nodes such that competitive service providers can purchase fiber to the node at some future date

Possible architecture for large town

School

School board office

School

Telco Central Office

Central OfficeFor Wireless

Company

VDSL, HFC or FiberProvisioned by service provider

Condominium Fiber with separate strands owned by school and by service providers

Carrier Owned Fiber

Cable head end

Average Fiber Penetration to 250-500

homes

ColoFacility

Benefits to Industry For cablecos and telcos it help them accelerate the deployment of high speed

internet services into the community Currently deployment of DSL and cable modem deployment is hampered

by high cost of deploying fiber into the neighbourhoods Cable companies need fiber to every 250 homes for cable modem service,

but currently only have fiber on average to every 5000 homes Telephone companies need to get fiber to every 250 homes to support

VDSL or FSAN technologies Wireless companies need to get fiber to every 250 homes for new high

bandwidth wireless services and mobile Internet It will provide opportunities for small innovative service providers to offer

service to public institutions as well as homes For e-commerce and web hosting companies it will generate new business in out

sourcing and web hosting For Canadian optical manufacturing companies it will provide new opportunities

for sales of optical technology and components

There is a clear trend in all formerly monopoly services to move to unbundled competitive services

Roads and highway systems vs railways: infrastructure was largely “public”, but the services (e.g. trucking) were private and competitive

Electrical distribution systems: regulated monopolies (unbundling is on horizon)

Gas distribution systems: regulated monopolies (unbundling is well underway)

Legacy telecommunications systems: moving to unbundled fiber and facilities based competition

Historical Reference Points

Carriers are not the only decision maker in the last mile

Governments and consumers are becoming more active voice in determining the future of broadband to home

Do not assume that carrier best technical solution is the only approach Open access is becoming a critical political issue

Consumers want more than duopoly of cable and telco Facilities based competition the best

Municipalities object to their streets being torn up Dig once – bury lots of fiber

Residents object to street furniture and antennae

Governments promote the framework for GITH networks by funding schools, universities, libraries, hospitals and municipal buildings as first customers and early adopters of dark fiber and optical networks

Private sector leverages that investment by government to promote high speed Internet access to schools and universities to extend the fiber to the home

Electric utility companies, municipal governments, CLECs, SMEs, entrepreneurs, as well as traditional telcos and cablecos can participate as providers, provided they subscribe to the architecture of open access, facilities based competition through dark fiber (or wavelengths)

Emphasize the development and use of technology that specifically addresses the new architecture and the last mile, which must therefore be open, cheap and Internet-only

An important Role for Government

Conclusion

Many governments have recognized the importance of access to low cost dark fiber as fundamental economic enabler

It will be the 21st century equivalent to the roads and railways that were built in the 20th century