best practice design for campus networks

© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF

March 25 – 27th, 2014 І Orlando, FL

#AvayaATF© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya.

Best Practice Design for Campus NetworksSteve Emert, Avaya

Keith Nuehring, City of Cedar Rapids

#AvayaATF

© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF#AvayaATF

Best Practice Design for Campus Networks

City of Cedar Rapids Fabric Connect Case Study

© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 3

Agenda

Campus Best Practices Design with Fabric Connect Deploying Fabric Connect in the Campus Design Options with Compact Form Factor Switches Capabilities to Ease Integration with Conventional Networks Models to Extend the Fabric to the Wiring Closet Edge

City of Cedar Rapids, Iowa case studyKeith Nuehring, IT Operations Manager, City of Cedar Rapids Cedar Rapids network before SPB and Fabric Connect Considerations and the decision to move to Fabric Connect Network Redesign Goals and Objectives Planning and Staging the Network Cutover Cutover Weekend Observations and Lessons Learned


Best Practice Network Designs for the Campus

First…. The sales pitch… not that you haven’t heard it before! Use Fabric Connect! Whether a small/medium enterprise, large campus network, or a campus

network that is distributed across a city or even a larger geography Why?

It will make your life easier It will free up your time and your staff’s time to work on more strategic

projects It will allow you to support new initiatives

improving your business – improve time to service

It will help simplify your efforts to maintain PCI DSS or other security compliance requirements or regulations

It will save you time and money in operational expenditures

It will make your network run more smoothly


Deploying Fabric Connect in the Campus

What SPB Network services to use, and where … Layer 2 VSNs

Stretching VLANs across the campus Rack to Rack, Row to Row, Data Center to Data Center for VM Moves Special purpose L2 networks

Totally constrained networks if no IP interface created on BEBs Routable if IP interface configured

STP BPDUs not transported across ISID STP becomes a “local construct” only for edge protection

BEB BEBBCBBCB

VLAN 200

I-SID200

VLAN 200

I-SID200



What SPB Network services to use, and where … IP Shortcuts (GRT Route Redistribution to ISIS)

One-hop IP routing across the fabric Eliminates “transit” IP Subnets, simplifying the routing table Enhances security – end users cannot determine “core” routers by

workstation Traceroute Typical Uses

Simplest migration from conventional IP routed network to Fabric Connect

Simply enable route redistribution to SPB/ISIS

BEB BEBBCBBCBVLAN 100

VLAN 300

VLAN 400

VLAN 200

Route redistributionacross ISIS



What SPB Network services to use, and where … Layer 3 VSNs

Multiple isolated/segregated IP routed networks within a single fabric infrastructure More efficient than conventional VRFs with a single instance of the routing

protocol (ISIS for SPB), single LSDB with info for all VRFs Typical Uses

Fully routed private networks for security segregation/isolation PCI DSS, IP Video surveillance networks, SCADA or HVAC Mergers and acquisitions, partnerships – duplicated IP addresses are allowed

when in different VRFs

BEB BEBBCBBCB

VLAN 100

VLAN 200

VLAN 300

I-SID500

I-SID500

VRF VRF


Deploying Fabric Connect in the Campus What SPB Network services to use, and where

ERS 4800Unicast only

ERS 4800Multicast

Clients

ERS 4500or

4500/4800

ERS 5000

VSP 4450 /4850VSP 8284

L2 VSN

IP Shortcuts

NNI

NNI

VSP 7024

IP Shortcuts

L2 VSN

Q Tagged

NNI

Q Tagged

NNI

VSP 7024

IP Shortcuts

IP Shortcuts

L2 VSN

IP Shortcuts

Q Tagged


Design Options with Compact Form Factor Switches

VSP 8284

VSP 7024

VSP 4850

VSP 4450

ERS 4800

Port Capacity Fabric Services

80 10Gig, 4 40Gig BCB, L2 VSN, IP Shortcuts,L3 VSN in 2015

24 10Gig (SFP+, 10GBase-T), 8 port MDA

BCB, L2 VSN

48 UTP 10/100/1000, 2 shared SFP, 2 10Gig SFP

BCB, L2 VSN, IP Shortcuts, L3 VSN

12 UTP, 36 SFP, 2 10Gig SFP

BCB, L2 VSN, IP Shortcuts,L3 VSN

48 UTP 10/100/1000 PoE+ or non PoEStackable

L2 VSN only


Fabric Connect Deployment Best Practices

Establish a consistent, understandable and useful naming/numbering system Backbone VLAN IDs 4051 and 4052 – per informational RFC System IDs – 00xx.xxxx.xx00

Stay away from first two hex digits and last two hex digits

Building or network . Subnet or other identifier . Switch number in location Example: 0001.07a1.0100

Nick-Names (ISIS Source Address) – x.xx.xx Base it on the System ID for ease of correlation

Example: 1.a1.01 System names (CLI Prompt names)

Meaningful, avoid special characters – make it easy to type!


Fabric Connect Deployment Best Practices

Ensure STP/MSTP/RSTP is disabled on NNI ports Default behavior on most switches Faster convergence when STP does not interfere with NNI

Avoid putting CVLANs on NNI ports Except where needed during conventional network to Fabric

migration Network Edge connectivity and protection

Spanning Tree Fast Learning or MSTP Edge-Port SLPP and SLPP Guard

Can use SLPP even in some non-traditional locations VLACP

Use is optional, but can improve NNI failover times Can specify Ethertype to be unique for the specific link


Capabilities to Ease Integration with Conventional Networks

Current capabilities Split BEB for SMLT compatibility 802.1Q tagged interfaces to non-SPB switches Transparent UNI Route redistribution between ISIS/SPB and other IGPs

Upcoming capabilities Virtual IST Fabric Attach Fabric Connect over IP SFP Adapter


Route Redistribution between SPB/ISIS and Other IGPs

Avaya SPB Domain

IP RoutedDomain

Avaya SPB Domain

Layer 3 Switch

Layer 2 Switch

OSPF/RIP/BGP enabled on external facing interfaces.

Redistribute ISIS Routes into other IGPs:ip <rip|ospf|bgp> redistribute isis <create|enable|apply>

Redistribute direct/static/other IGP routes into ISIS:ip isis redistribute <direct|static> <create|enable|apply>ip isis redistribute <rip|ospf|bgp> <create|enable|apply>

Layer 2 Switch

Edge VLANs/IP Subnets are present on VLAN interfaces within the SPB BEB

switch, associated with ISIDs

Redistribute Direct to ISIS

Layer 3 Switch

RIP or OSPF used to exchange routes with external routing switch

Redistribute RIP or OSPF to ISIS


Virtual IST for SMLT (vIST) Virtual IST Concepts

Delivers SMLT/RSMLT w/ virtual IST capability. Ability to run IST over SPB fabric. Removing the

need for direct links between IST peers. Can be deployed as conventional SMLT/RSMLT solution with direct IST links.

First phase: configure ISIS and SPB & IST protocol between a cluster pair

Virtual IST Benefits Further increased resiliency & more flexible

routing for IST connectivity. (no need for direct IST trunk)

Allows mixing of IST node types (VSP8k, VSP4k, …)

Future benefits for FA and distributed LAG Platform Implementation Timelines

VSP8k Release 4.0 1H2014 VSP4k Release 4.1 2H2014

Seamless & Painless Scalability

Virtual IST

Single, Unified, Logical Core


Models to Extend the Fabric to the Wiring Closet Edge

ERS 4800 SPB L2 VSN

Fabric Attach

L2 VSN

NNI

NNI

ERS 5600

FA Client

FA Hosts


Fabric Attach (FA)

Concepts Automatic attachment of non-fabric switches (e.g.

ERS 56xx) and hosts/devices (Servers, Cameras, APs) to Fabric Connect networks.

Introduces FA Server, FA Switch & FA Devices Uses signaling protocol to signal VLAN/ISID

memberships between attached hosts and non-fabric switches/devices to FA Server switches (BEBs).

First step to Zero-Config-Edge: Establishes node connectivity for FA hosts/devices out of the box and auto attaches it to fabric

Timelines (solution more widely available in 2015) FA Server: VSP7k, ERS4k demo now, VSP8k,

VSP9k, VSP4k 2015 FA Switch: ERS4k, ERS5k demo now FA Devices: TBD


Fabric Attach Extends the Benefits of Fabric to the Wiring Closet & Network Attached Devices

Customer Value Automated identification and provisioning of

end points (e.g., wireless AP’s and cameras) Simplified network provisioning for devices

outside the Fabric Reduction in network configuration errors Simplifies adds, moves, and changes

Technology Builds on top of Fabric Connect architecture Extends Fabric benefits to non-Fabric

Connect platforms AND endpoints / users Client/Device identification, authentication

and authorization via Identity Engines

Fabric ConnectSwitch Stack of ERS 4800

Fabric AttachSwitch ERS 4800

FabricAttach

FabricAttach

Employee Zone

Contractor Zone

Management Zone

Guest Zone

Authentication &Authorization

Fabric AttachSwitch ERS 5600

Authentication &Authorization

Fabric AttachServer ERS 4800

DHCP

Fabric AttachAccess Point


Fabric Connect SPB over IP

Concepts Solution allows extension of Avaya’s Fabric Connect fabrics

over IP networks. Introduces 100Mbps/1Gbps SFP compatible adapter which

tunnels Ethernet VLANs over IP. The SFP “sleeve” can be inserted into an SFP/SFP+ NNI port

and is used in conjunction with any supported SFP (sorry, no 10Gig SFP+).

SFP adapter provides IP tunneling capabilities for SPB NNI connections by adding additional IP tunnel header to the SPB MAC-in-MAC packets.

IP connection MTU requirements: 1582 to 2000 bytes – work with the carrier to ensure compatibility!

Benefits Allows extending SPB/Fabric Connect over an IP network. IP

network can be campus backbone or MAN/WAN IP MPLS network.

Full fabric capabilities remain intact over IP (except MTU considerations).

Support for Hub and Spoke topologies (with up to 64 tunnels per adapter)

Allows extending IST over WAN solution with vIST

Timelines Proof Of Concept occurring now Production anticipated 2H2014, early 2015


Fabric Connect over IP Deployment Scenario – Hub and Spoke

Adapter establish multiple tunnels per device (POC restricted to 2) For POC hub site requires translation bridge (ERS4800) to convert

NNI interfaces into 1 uplink port

Main Site

Site 1

VSP4000

VSP4000

WAN

SPB Fabric

Site 2

Site 3

Site 4

VSP4000

VSP4000

VSP4000

4 interfaces

ERS4800


March 25 – 27th, 2014 І Orlando, FL

#AvayaATF© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya.

City of Cedar Rapids, Iowa Fabric Connect/SPBm Implementation Case Study

Keith Nuehring – IT Operations Manager


Cedar Rapids Network before SPB and Fabric Connect

Many device types – ERS 8300, ERS 5530, ERS 4548, ERS 2526, SR 1004, SR 3120

With three primary locations, SMLT (two-switch cluster) didn’t really fit well ERS 5530’s required at least annual reboot to maintain stability Maintained a Spanning Tree ring to support smaller venues within the City


Considerations and the Decision to move to Fabric Connect

Should we stay with Avaya or move to another vendor? Nortel’s Chapter 11 Uncertainty over whether Avaya would continue to support and expand

the networking business acquired with Nortel Enterprise Solutions Concerns with technical support after Avaya’s NES acquisition Concerns with stability and product life of existing equipment

ERS 8300 ERS 5530

Attended ATF Orlando February 2012 Came away from first ATF with decision to implement SPB


Network Redesign Goals and Objectives

New network in conjunction with new City Services Center Upgrade from 1Gigabit to 10Gigabit City-wide network Increase network resilience – target of milliseconds, not seconds Consolidate Layer 3 routing into three main sites Reduce number of devices types to support

From … ERS 8300 and ERS 5500 both performing IP routing ERS 4000 series and ERS 2500 series edge switching

Different platforms, different capabilities, different CLIs To …

VSP 7000 SPB Fabric Switches and IP Routing ERS 4000 series and ERS 3500 series edge switching

Single CLI common across entire network


Planning and Staging the Network Cutover

Considerations in selecting the network products to use Large chassis switches not practical for City of Cedar Rapids VSP 7000 – 10 Gigabit switching, SPB Fabric, but no

simultaneous SPB Fabric and IP Routing VSP 4000 – Limited number of 10 Gigabit interfaces, semi-

external USB flash, does have IP Shortcut Routing Selected VSP 7000 both for SPB Switching Fabric and for IP

Routing external to the SPB Fabric Network design

Three-site mesh topology with all links active NetApp storage and servers sync between sites across L2 VSN

Training and Staging Used to-be live equipment for on-site hands-on training for staff Pre-built and tested entire network in City Services Center lab


New City of Cedar Rapids SPB Network Design

Involta Data CenterVSP 7000

L2 Fabric Core and in-building IDF Distribution

VSP 7000L3 Fabric Services

ERS 48001Gig Server Switching

VSP 7000L2 Fabric Core and in-

building IDF Distribution



Central FireVSP 7000

L2 Fabric Core and in-building IDF Distribution



Public Works

City Hall

Police Department

Water

Shown:• Main site network core switching

and routing only

Not shown:• Small, seasonal venues (parks,

rinks, etc.)• In-building wiring closets

City Services Center


Cutover Weekend

Moved pre-configured equipment from City Services Center lab to final location

Plan and document all steps, label all cables Walk through migration steps in a dry run to test procedures With single-strand BX optics, could bring up basics of new network

simultaneously with old network for easing the migration Proactively open up Avaya Support case, have support engineer

review plan and configurations before cutover Problems? … a few …

PVID set wrong on a switch Missed moving a cable, caused a routing loop VSP 7000 in Involta Data Center random rebooting

Updated boot loader to fix


Observations and Lessons Learned

SPB resiliency works One optic failed some time after cutover and the NNI was

bouncing No visible effect to users – even while link was bouncing for 2-

1/2 hours Simply disabled link over weekend until optic could be

changed the next Monday Proved during cutover that even with a VSP 7000 failing, the

alternate switch and path would work to maintain the network VSP 7000 that was rebooting was one of two switches

connecting the NetApp storage! Simplified IP routing by consolidating into VSP 7000’s with VRRP Standardized on Direct and Static routing and VRRP Single CLI syntax on VSP 7000, ERS 4000, ERS 3500 is nice Was it worth making the change?

© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF#AvayaATF

Thank You!


BE SURE TO TWEET YOUR FEEDBACK ON THIS PRESENTATION

#AvayaATF

29

BEST OF ATF SPEAKER AND TEAM AWARD

Winners will be announced at closing of event

#AvayaATF

best practice design for campus networks

Documents

avaya fabric

fabric connectdeploying

fabric connectconsiderations

large campus network

campus best practice

ip routing

avayaatfbest practice

staffs time