11/3/2003 1

Network Planning Task Force

                                                               

“Operational Briefing”

11/3/2003 2

Active Task Force Members http://www.upenn.edu/computing/group/nptf/

Mary Alice Annecharico / Rod MacNeil, SOM

Mark Aseltine* / Mike Lazenka, ISC Robin Beck, ISC Doug Berger / Manuel Pena, Housing

& Conference Services Chris Bradie / *Dave Carroll,

Business Services Chris Field, GPSA (student) Cathy DiBonaventura, School of

Design* Geoff Filinuk, ISC Bonnie Gibson, Office of Provost Roy Heinz / John Keane, Library Robert Helfman, Budget Mgmt.

Analysis John Irwin, GSE Marilyn Jost, ISC Carol Katzman, Vet School

Deke Kassabian / Melissa Muth, ISC James Kaylor / CCEB* Dan Margolis, SEAS* (student) Chris Massicotte, Audit & Compliance Kayann McDonnell, Law Donna Milici, Nursing Dave Millar, ISC Michael Palladino, ISC (Chair) Dominic A. Pasqualino / Audit &

Compliance* David Seidell, Wharton* Dan Shapiro, Dental Mary Spada, VPUL Marilyn Spicer, College Houses* Steve Stines / Jeff Linso, Div. of

Finance Ira Winston / Helen Anderson, SEAS,

SAS, School of Design

*New FY ‘04

9/15/2003 3

NPTF FY 2004 Agenda

Summer9/159/2910/811/311/1712/112/15

Focus group sessionsSetting the stageSecurity discussions (Part I)Security discussions (Part II)Operational briefingStrategic discussions“Voting”/preliminary rate settingState of the Union

11/3/2003 4

Operational Briefing

■ Major progress■ Telecommunications services■ Central service bundle■ Internet/Internet II ■ Connection maintenance■ Port rental■ Next Generation PennNet■ Wireless

11/3/2003 5

Major Progress Last 12 Months■ Redundancy, Reliability

■ More proactive network management tools and procedures.

■ Huntsman NAP operational■ Southern NAP construction underway■ NGP building redundancy in design stage.

■ Many buildings could still have 1-2 week outage with major catastrophe

11/3/2003 6

Major Progress (Continued)

■ Network Upgrades/Enhancements■ Gig routing core■ Fast Ethernet (100 Mbps) to buildings 90%

completed■ Gig (1000 Mbps) backbones in buildings 50%

completed■ 60% of closet electronics 10/100 Mbps

11/3/2003 7

Major Progress (Continued)■ Services

■ PoBox email service back to high availability with added functionality of SPAM filtering and virus scanning.

■ Wireless connectivity expanding. Service in 11 schools and centers, 4 public spaces, and study lounges in 3 College Houses. Combined service can provide connectivity for 3825 users at one time.

■ Wireless authentication progress made.■ Cellular program nearly ready to be announced.■ VoIP pilots underway within N&T.■ Automatic Speech Recognition System (6-4242)

being refined.

11/3/2003 8

Major Progress (Continued)

■ Operational efficiencies■ Fiber ring replacing MAN services will lock in lower

local loop costs even as bandwidth demands increase for Internet/Internet2.

■ Bandwidth management techniques in College Houses solidified with SLA.

■ Elimination of legacy network infrastructure (PAS)■ 99% switched ports (3 buildings with coax)

11/3/2003 9

Projected N&T 5-Year BudgetDIRECT CHARGES FY'04 Budget FY'05 Budget FY'06 Budget FY'07 Budget FY'08 Budget TELECOMMUNICATIONS 10,200,000$ 10,000,000$ 9,900,000$ 9,800,000$ 9,700,000$ CENTRAL SERVICE FEES 4,680,000$ 5,040,000$ 5,430,000$ 5,830,000$ 6,250,000$ NETWORK INSTALLATIONS/PROJECTS 1,200,000$ 1,200,000$ 1,200,000$ 1,200,000$ 1,200,000$ ISC MAINTAINED WALLPLATE CONNECTIONS 2,625,000$ 2,850,000$ 3,115,000$ 3,295,000$ 3,480,000$ EMAIL, WEB HOSTING, VIDEO (NON-PVN) ETC. 550,000$ 590,000$ 640,000$ 690,000$ 750,000$ MAGPI SERVICES 900,000$ 1,000,000$ 1,100,000$ 1,100,000$ 1,100,000$ SUBTOTAL DIRECT CHARGES 20,155,000$ 20,680,000$ 21,385,000$ 21,915,000$ 22,480,000$

ALLOCATED COSTS NEXT GENERATION PENNNET 337,000$ -$ -$ -$ -$ NETWORK ENGINEERING AND SERVICES (NES) 1,054,000$ 928,000$ 465,000$ -$ -$ SUBTOTAL ALLOCATED COSTS 1,391,000$ 928,000$ 465,000$ -$ -$

GENERAL FEE PENN VIDEO NETWORK 578,000$ 578,000$ 578,000$ 578,000$ 578,000$ SUBTOTAL GENERAL FEE 578,000$ 578,000$ 578,000$ 578,000$ 578,000$

TOTAL INCOME 22,124,000$ 22,186,000$ 22,428,000$ 22,493,000$ 23,058,000$ TOTAL EXPENSE 21,947,000$ 22,078,000$ 22,368,000$ 22,623,000$ 23,078,000$

Cumulative (Surplus) / Deficit 168,000$ 60,000$ -$ 130,000$ 150,000$ *Includes FY2003 deficit of $345k

11/3/2003 10

Projected Telecommunications Revenue FY2004 (Summer, 2003 Data)

Service Admin Student TotalEquipment/lines 3,760,000$ 615,000$ 4,375,000$ Voicemail 795,000$ 25,000$ 820,000$ Allocation 1,345,000$ 1,345,000$ Installation 800,000$ 132,000$ 932,000$ Usage (LD/local) 1,810,000$ 720,000$ 2,530,000$ Miscellaneous 163,000$ 35,000$ 198,000$ Total 8,673,000$ 1,527,000$ 10,200,000$

11/3/2003 11

FY2004 Projected CSF Expenses(Fall, 2002 Data)

External Computer Software N&T Staff CSF BUNDLE OF SERVICES HW & OS & Lic/Maint (non-allocated) Subtotal % Total CAMPUS BACKBONE INFRASTRUCTURE -$ 878,000$ 99,300$ 505,053$ 1,482,353$ 32.0%INTERNET & BANDWIDTH MANAGEMENT 1,000,000$ 100,000$ 41,500$ 259,673$ 1,401,173$ 30.2%INTERNET2 197,568$ 50,000$ 15,000$ 182,055$ 444,623$ 9.6%NOC/NETWORK MANAGEMENT/EXT HOURS -$ 78,250$ 8,645$ 388,526$ 475,421$ 10.2%FIBER AND CABLE MANAGEMENT -$ -$ 100,000$ 135,334$ 235,334$ 5.1%WWW -$ 40,000$ 27,000$ -$ 67,000$ 1.4%INFRASTRUCTURE SOFTWARE SERVICES -$ 110,000$ 11,000$ -$ 121,000$ 2.6%R&D -$ 90,000$ 9,000$ -$ 99,000$ 2.1%NETNEWS -$ 40,000$ 18,500$ -$ 58,500$ 1.3%MAIL RELAY, LISTSERV, DIRECTORY -$ 25,000$ 4,500$ -$ 29,500$ 0.6%NETWORK SECURITY INCIDENT RESPONSE -$ -$ -$ 36,094$ 36,094$ 0.8%CENTRALIZED WIRELESS AUTHENTICATION -$ 25,000$ 3,750$ 20,000$ 48,750$ 1.1%PENN COMMUNITY/SECURITY TOOLS 90,000$ -$ -$ 90,000$ 1.9%DIFFERENTIAL BILLING DEVELOPMENT 50,000$ 50,000$ 1.1%TOTAL 1,287,568$ 1,436,250$ 338,195$ 1,576,735$ 4,638,748$ 100%FY2004 PROJECTED AVG IP ADDRESSES 38,210 FY2004 CSF Rate 10.12

11/3/2003 12

Internet/Internet2

■ Current status/utilization■ Strategy■ Costs■ Bandwidth management■ Next steps

11/3/2003 13

External Connectivity

11/3/2003 14

Growth in Internet BandwidthGrowth in Internet Bandwith

0.0

100.0

200.0

300.0

400.0

500.0

600.0

Time

Mb

ps

'86 '90 '92 '96 '97 '99 '00 '01 '02 '03 '04

11/3/2003 15

Internet Usage October, 2003

11/3/2003 16

Internet2 Usage October 2003

11/3/2003 17

Internet Strategy■ Have multiple Internet Service Providers with diverse

paths and national backbones. (Currently have 3 ISPs)■ Have a presence at 401 N. Broad Street in the Telecom.

Hotel to rapidly switch ISPs, get additional bandwidth and lower local loop costs. (Currently lease 100 SF)

■ Have reliable and redundant lines from 401 N. Broad to main campus. (Currently have links from both YIPES and PECO Hyperion, migrating to leased fiber ring.)

■ Have sufficient Internet capacity to meet current and future needs. (Current infrastructure/ISPs are capable of over 2000 Mbps.)

11/3/2003 18

Internet Strategy (Continued)■ Maintain peering links with ISPs. (Currently have 100

Mbps link to DCAnet, talking to both Comcast & Verizon.)■ Continue to provide cost-effective service for Penn.

(Current monthly Internet cost is $3.05 per user/IP address.)

■ Continue experimentation with low-cost providers. (This has been very successful with cost per meg of Internet service dropping from $750 to $40 per meg per month).

■ Continue limits for aggregate outbound residential traffic as well as individual residential IP address outbound limits. (There are no limits on non-residential usage.)

11/3/2003 19

Internet External Expenses

$0

$200,000

$400,000

$600,000

$800,000

$1,000,000

$1,200,000

'86 '90 '92 '96 '97 '99 '00 '01 '02 '03 '04

11/3/2003 20

Cost Per Mbps Per Month

$0

$100

$200

$300

$400

$500

$600

$700

$800

'01'99'97 '00 '02 '03 '04

11/3/2003 21

Internet Bandwidth Management - Background

■ A Central Service Fee is assessed for each IP address used.

■ Internet costs are part of the Central Service Fee bundle of services that N&T provides.

■ Internet costs are distributed equally across Penn’s 38,000 users.

■ The cost per IP address per month is $3.05 in FY ’04.

11/3/2003 22

Internet Bandwidth Management (Continued)

■ Internet bandwidth usage and costs increased exponentially during the late 90’s.

■ Periodic “snapshots” of usage demonstrated that College House residents had a different profile of usage than PennNet’s non-residential customers.

■ 24 X 7 rather than 7AM to 8PM■ Demands for bandwidth greatly exceeding almost all

other users.

■ Residential limits on outbound bandwidth have helped slow the increase in Internet expenses.

11/3/2003 23

Creation of a Residential PennNet Service

■ Differential charging is not yet technically practical.

■ Charging all users more to accommodate growing cost due to residential use was not acceptable.

■ Creation of a custom service level designed for residential users, with differences but at a price equal to pre-existing services.

11/3/2003 24

Bandwidth Management - Current Status

■ Bandwidth management techniques in the College Houses are being successfully implemented.

■ Upper limits on aggregate outbound usage (255Mbps)■ Maximum outbound bandwidth limits per IP address

(400Kbps with a 400 KB burst)

■ The limits on residential Internet traffic plays a major role in controlling costs.

11/3/2003 25

Bandwidth Management – Next Steps

■ Improve our ability to identify traffic patterns, heavily used applications, most demanding users and quick Information Security incident response.

■ Use this information to help in the evaluation of service.

■ To business and research/education users■ To residential users

11/3/2003 26

FY2004 Projected CSF Expenses(Fall, 2002 Data)

External Computer Software N&T Staff CSF BUNDLE OF SERVICES HW & OS & Lic/Maint (non-allocated) Subtotal % Total CAMPUS BACKBONE INFRASTRUCTURE -$ 878,000$ 99,300$ 505,053$ 1,482,353$ 32.0%INTERNET & BANDWIDTH MANAGEMENT 1,000,000$ 100,000$ 41,500$ 259,673$ 1,401,173$ 30.2%INTERNET2 197,568$ 50,000$ 15,000$ 182,055$ 444,623$ 9.6%NOC/NETWORK MANAGEMENT/EXT HOURS -$ 78,250$ 8,645$ 388,526$ 475,421$ 10.2%FIBER AND CABLE MANAGEMENT -$ -$ 100,000$ 135,334$ 235,334$ 5.1%WWW -$ 40,000$ 27,000$ -$ 67,000$ 1.4%INFRASTRUCTURE SOFTWARE SERVICES -$ 110,000$ 11,000$ -$ 121,000$ 2.6%R&D -$ 90,000$ 9,000$ -$ 99,000$ 2.1%NETNEWS -$ 40,000$ 18,500$ -$ 58,500$ 1.3%MAIL RELAY, LISTSERV, DIRECTORY -$ 25,000$ 4,500$ -$ 29,500$ 0.6%NETWORK SECURITY INCIDENT RESPONSE -$ -$ -$ 36,094$ 36,094$ 0.8%CENTRALIZED WIRELESS AUTHENTICATION -$ 25,000$ 3,750$ 20,000$ 48,750$ 1.1%PENN COMMUNITY/SECURITY TOOLS 90,000$ -$ -$ 90,000$ 1.9%DIFFERENTIAL BILLING DEVELOPMENT 50,000$ 50,000$ 1.1%TOTAL 1,287,568$ 1,436,250$ 338,195$ 1,576,735$ 4,638,748$ 100%FY2004 PROJECTED AVG IP ADDRESSES 38,210 FY2004 CSF Rate 10.12

11/3/2003 27

FY2004 Connection Maintenance Expense Breakdown (10baseT)

Elements Costs (Annual)

Staff(NOC, Net Management, second shift, OT, beeper pay, overhead)

$633,000$343,000 (54.2% overhead) $976,000

Hardware/Software/Tools/Maintenance Contracts

$259,000

TOTAL $1,235,000

Number Ports 32,878 ÷ 12 month

Monthly Rate $3.13 per port

11/3/2003 28

FY2004 Port Rental Expense Breakdown (10baseT)

Elements Costs (Annual)

Staff including overhead $155,000

Equipment $1,000,000

TOTAL $1,155,000

Number Ports 32,878 ÷ 12 month

Monthly Rate $2.93 per port

11/3/2003 29

Next Generation PennNet (NGP)

■ Goals■ Current status■ Strategy■ Future plans

11/3/2003 30

NGP Infrastructure Goals■ Distribute routing core across campus to minimize

single point of catastrophic network failure.■ Build redundant network links between the Network

Aggregation Points (NAPs) and to critical buildings.■ Upgrade 20 year-old multi-mode fiber and install

single-mode fiber to prepare for multi-Gigabit network speeds.

■ Build Next Generation PennNet infrastructure to prepare for future technologies and convergence.

■ To continue to provide secure, “cutting-edge” network connectivity to support Penn’s research, academic and administrative needs.

11/3/2003 31

NGP Current Status■ Vagelos and Huntsman NAPs and associated

single-mode fiber links completed.■ Strategic conduit installed partnering with

Facilities. (Locust Walk, Spruce Street, Levine, Hillel, Huntsman, etc.)

■ Distributed and redundant routers, servers and systems in Vagelos, Huntsman, College Hall and 3401 Walnut.

■ Southern NAP construction underway.■ Western and Northern NAPs site selections in

progress.

11/3/2003 32

NGP Current Status (Continued)■ Redundant connectivity (dual fiber and

pathway feeds) for 3401 Walnut, FB, VPL, Vagelos and Huntsman.

■ Most Area 1 buildings linked to Vagelos NAP.■ Some area 2 buildings linked to Huntsman

NAP.■ Catastrophic failure reduced from 2 weeks to

1-2 days for these buildings.

11/3/2003 33

NAP4WESTERN

TIER

NAP3SOUTHERN TIER

MOD5

NAP1EASTERN TIER

VAGELOS

NAP5NORTHERN

TIER

NAP2CENTRAL TIER

HUNTSMAN

Next Generation PennNetCurrent Status/Plan

Penn Building (typ.)

Completed

FY’05

FY’04

ProposedConstruction Phases

Building Recover Time

2 Week Recovery

2 Day Recovery

2 Hour Revovery

VPL

FB

NAP-CHCOLLEGE HALL

NODE ROOM3401

FY’06

Symbol Legend

NAP Connectivity

Building Connectivity

11/3/2003 34

NGP Future Plans■ Plan FY2004

■ Southern NAP fully operational (Jan. ’04)

■ Northern NAP site selected. (March ’04)

■ Western NAP under construction (June ’04)

■ Southern NAP building redundancy links underway.

■ Plan FY2005■ Western and Northern

NAPs fully operational.■ Most NAP to NAP links

completed.■ Western and Northern

building redundancy links underway.

■ Potential funding issues if NGP funds (Telecom surplus) shrink or if there is not enough existing fiber.

11/3/2003 35

Wireless

■ Brief Overview■ Goals■ Strategic Objectives■ Current Research■ Short – Long Term Strategy■ Challenges■ Wireless Network Topologies

11/3/2003 36

Wireless Terminology■ wLAN: Wireless Local Area Network■ AP (Access Point): Point of Entry from

Wireless to a Wired Network■ Authentication Gateway: Point of entry from

unsecured network into PennNet.■ Association: an 802.11 mapping of a

wireless device to an AP.■ Coverage Area: Radio frequency range of an

AP or group of AP’s, which comprises the wireless network.

11/3/2003 37

Overview of Wireless PennNet■ Wireless Networks at Penn are based on 802.11b

standard, which is 11mb shared access to the network.■ Bandwidth per user depends on several factors:

■ Number of devices associated with a single AP■ Number of devices transmitting or receiving data■ Distance of device from AP■ Interference

■ Other APs■ Other Devices using same frequency■ Interface cards and antennas■ Physical position of the device relative to the AP

■ Each wLAN offers seamless roaming from one area of a building to another.

■ Wireless service provides convenience rather than high performance.

11/3/2003 38

Goals■ To meet the wireless needs of the Penn

community for schools and centers and public spaces.

■ To have a common authentication approach across campus (strong security).

■ To have seamless roaming from one wLAN to another.

■ To determine a sustainable funding model for one-time and all ongoing operational expenses.

11/3/2003 39

Strategic Objectives■ Using Enterprise Class Technologies

■ Designed to serve large organizations■ Long-term customer support included■ Provides more robust set of features, including:

■ Adjustable Signal Strength■ Stability■ Monitoring & Statistics■ Tri-Band Capabilities

■ Aiming to Provide Scalable & Secure Solutions■ Staged Approach to Deployment

■ Working with Standards-Based Products■ Avoid being locked into relationship with single vendor■ Cards that comply with wireless standards

11/3/2003 40

Current Research■ Authentication Gateway Tests

■ Testing with New Vendor Going Well■ Short Term Plans

■ Work with Both Vendors (support exiting base)■ Local Testing Ends 10/27■ Pilot Deployment (Small Area) early November■ Expand to Larger Pilot Mid December■ Resume replacement of MAC Authentication

■ Hit Target Dates for FY04

11/3/2003 41

Short – Medium Term Strategy■ Security

■ Continue to Pursue Authentication Gateway Testing

■ Maintain Vendor Relationships for Current Install Base■ Test Another Vendor’s Product■ Explore Vendor’s Product Road Map with LT Strategy

■ Network Management■ Work on Better Network Monitoring and Reporting

■ Resume Replacement of MAC Authentication■ Staged Approach

■ Begin Test and Research high-speed wireless

11/3/2003 42

Long Term Strategy

■ Security■ Authenticated & Authorized Access Control

(802.1X)■ Extensible authentication methods

■ Secure, encrypted channel (802.11i)■ Successor to Wired Equivalent Privacy

■ VPN Gateways & Clients■ Wireless Authentication Gateway could be VPN

Gateway for wireless clients■ May require VPN software on client

11/3/2003 43

Long Term Strategy (continued)

■ Network Management■ SMNP v3

■ Secure means of managing the network■ Automated Reporting

■ Usage Statistics

■ Quality of Service (802.11e)■ Support for delay-sensitive applications■ Wireless Voice over IP

11/3/2003 44

Challenges■ Funding

■ No Central Funding■ Slower Roll Out in Some Areas■ IP Address/CSF fee issue for public spaces

■ Security■ Authenticated Access (Still being worked out)■ Data Encryption Lacking■ Authorization

■ Support■ Central and Local Support

■ Not a static situation in a wLAN environment

11/3/2003 45

Current Status Public WirelessLocation Fundin

gIndoor/Outdoor

Components

Capacity Auth Public/Private

U Square Facilities

Outdoor 2 AP 50 users PennKey

Public

Perelman VPUL Indoor & Outdoor

4 AP 100 users PennKey

Public

Hill House ISC/CHC Indoor 4 AP 100 users PennKey

Public

Harnwell ISC/CHC Indoor 1 AP 25 users PennKey

Public

Hamilton CHC Indoor 5 AP 125 users PennKey

Public

Grad Ctr. VPUL Indoor 1 AP 25 users PennKey

Public

3401 Walnut ISC N&T Indoor 5 AP 125 users PennKey

Public

Sansom West ISC Indoor 3 AP 75 Users PennKey

Public

VAN, SDH, HNT

Wharton

Indoor & Outdoor

57 AP 1425 users

MAC Public

Van Pelt Library Indoor 19 AP 475 users MAC Public

Bio Pond SAS Outdoor 1 AP 25 users MAC Public

Bio Med Library

Library Indoor 3 AP 75 users MAC Public

11/3/2003 46

Current Status Private Wireless

Location Funding

Indoor/Outdoor

Components

Capacity Auth Public/Private

Law School Law Indoor & Outdoor

34 AP 850 users MAC School Only

Dental Dental Indoor 5 AP 125 users MAC School Only

Furness Design Indoor 2 AP 2 Bridges

50 users MAC School Only

4200 Pine VPUL Indoor 2 AP 50 users MAC Department Only

Colonial Penn VPUL Indoor 2 AP 50 users MAC Department Only

Meyerson Design Indoor 1 AP 25 users MAC School Only

Fels Center SAS Indoor 1 AP 25 users MAC School Only

DRL SAS Indoor 1 AP 25 users MAC School Only