core design 2
TRANSCRIPT
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 1
CampusHighAvailabilityJRES2009
Jean‐[email protected]
February2009
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 2
CiscoSwitchingPorIolio
Catalyst29xx
Catalyst3750Catalyst3750‐ECatalyst3560
Catalyst3560‐E
Catalyst4500E‐Series
Catalyst6500
Catalyst6500
Small Medium-sized Large
Number of Employees/Density
Features,Scalability,Lon
gevity
WiringCloset
DatacenterAccess
DistribuDon/Core
BladeSwitches
Catalyst6500
Catalyst4500E‐Series
Catalyst4900Series
Nexus7000
Nexus5000
Catalyst29xxLite
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 3
NextGeneraBonCampusDesignUnifiedCommunicaBonsEvoluBon
• IPTelephonyisnowamainstreamtechnology
• OngoingevoluBontothefullspectrumofVideoandCollaboraBontechnologies
• HighDefiniBonExecuBveCommunicaBonApplicaBonsrequirestringentService‐LevelAgreement(SLA)– ReliableService–HighlyAvailableInfrastructure– ApplicaDonServiceManagement‐QoS
© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID 3
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 4
Minimal Impact to Voice
User Hangs Up
Minimal Impact to Video,
none to Voice
Phone Resets*
Seco
nds
of D
ata
Loss
* Phone to reset time depends on the signaling protocol, SCCP or SIP, and call state; active, ringing, …
NextGeneraBonCampusDesignUnifiedCommunicaBonsEvoluBon
• AvailabilityRequirementsforUCaremorethanjustfive9’s
• AlsoneedtoconsiderthesubjecBveimpacttorealBmecommunicaBons
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 5
Cisco’sCampusArchitectureHierarchical,ModularandResilient
• Offershierarchyforscalability
• Modularbuildingblocks—Easytogrow,understand,andtroubleshoot
• Predictabletrafficpa`ernsundernormalandfailurecondiBons
• Smallfaultdomainstoisolateproblems
• Promotesloadbalancingandfastfailover
• Canbeappliedtoallcampusdesigns;MulB‐LayerL2/L3andRoutedAccessdesigns
DataCenterWAN Internet
RedundantSwitches
RedundantSupervisor
Layer3EqualCostLink’s
RedundantL3Links
Layer2orLayer3
Access
Distribution
Core
Distribution
Access
BuildingBlocks
BuildingBlocks
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 6 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID
DataCenterServicesBlock
DistribuDonBlocks
AGENDA
• SystemsLevelResiliency
• NetworkLevelResiliency–RouBng
• CampusCoreandFoundaBonServices
• EmergingCampusDesign
– RoutedAccess– VirtualSwitchCampusDesign
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 7
• Nexus7000,Catalyst6500and4500highlyredundantModularsystems
RedundanthotswappableSupervisors
RedundanthotswappablePowerSupplies
N+1redundantfanswithhotswappablefantraysHotswappablelinecards
Passivedatabackplane
Redundantsystemclockmodules
• Catalyst3750/3750EStackwisePlus*technology
1:NMasterredundancy
Hotswappablestackmembers
HotswappablePowerSupplies*
SystemLevelResiliencyComprehensivePhysicalRedundancy
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 8
GracefulRestartNon‐StopForwarding/StatefulSwitch‐Over
• NSF/SSOisasupervisorredundancymechanismforintra‐chassissupervisorfailover
• SSOsynchronizeslayer2protocolstate,hardwareL2/L3tables(MAC,FIB,adjacencytable),ACLandQoStables
– SSOsynchronizesstatefor:trunks,interfaces,EtherChannels,portsecurity,SPAN/RSPAN,STP,UDLD,VTP
• Non‐StopForwarding(NSF)providesthecapabilityfortherouBngprotocolstogracefullyrestartaneranSSOfail‐over
– ThenewlyacBveredundantsupervisorconBnuesforwardingtrafficusingthesynchronizedHWforwardingtables
– TheNSFcapableRouBngProtocolrequestsagracefulneighborstart
– RouBngneighborsreformwithnolossoftraffic
• AggressiveRPBmersmaynotworkinNSF/SSOenvironment
NSF‐Aware NSF‐Aware
NSF‐Capable
NoRouteFlapsDuringRecovery
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 9
• StatefulRestartwithPSS– CheckpointsstatestoPSS– RecoverstatesfromPSS
uponrestart
• StatefulRestartwithGR– Freshstartwithouttracesfromformer
instanBaBon.
– GracefulRestart(NSF)forL3Protocols• SupervisorSwitchover• Non‐disrupBveInServiceSonwareUpgrade
Nexus7000ServiceRestart
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 10
Kernel
BG
P
HS
RP
PIM
TCP
/UD
P
IPv6
STP
OS
PF
LAC
P
etc
HA Manager
Restart process!
PSS
The Traffic keeps being forwarded by the Linecard Forwarding
Engine
N7K Data Plane Dataplanestreams
Nexus7000StatefulFaultRecoveryUsingPSS
Ifafaultoccursinaprocess:• “Sysmgr”determinesbestrecoveryacBon(restartprocess,switchovertoredundantsupervisor)• ProcessrestartswithnoimpactondataplaneStatecheckpoinBng(PSS)allowsinstant,statefulprocessrecovery
• MulBpleServiceInstances
• Independentmemory‐protectedre‐startableprocesses
• ServicescheckpointtheirrunBmestatetothePSSforrecoveryintheeventofafailure
– Layer2Services
– Layer3Services
• Neighborsneverseeeventoccur
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 11
Linux Kernel
BG
P
EIG
RP
PIM
TCP
/UD
P
IPv6
STP
HS
RP
LAC
P
etc
HA Manager
Table Update
NX7K Data Plane
Data plane streams
Nexus7000StatefulFaultRecoveryUsingGracefulRestart[1]
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 12
Linux Kernel
BG
P
EIG
RP
PIM
TCP
/UD
P
IPv6
STP
HS
RP
LAC
P
etc
HA Manager
Restart process!
Table Update
NX7K Data Plane
Data plane streams
Nexus7000StatefulFaultRecoveryUsingGracefulRestart[2]
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 13
Linux Kernel
BG
P
EIG
RP
PIM
TCP
/UD
P
IPv6
STP
HS
RP
LAC
P
etc
HA Manager
Restart process!
Graceful restart Graceful restart
Routing updates Routing updates Table Update
NX7K Data Plane
Data plane streams
Nexus7000StatefulFaultRecoveryUsingGracefulRestart[3]
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 14
DesignConsideraBonsforNSF/SSOSinglePointsofConnecBvity=SSOandNSF
• AccessswitchisthesinglepointoffailureeveninHAcampusdesign
• Businessrequirementdrivingnewrequirements
– UnifiedCommunicaBonsintegraBonrequireshighupBme
– CriBcallocaBonsrequireconBnuousconnecBvity(eg.Hospital,CallCenter)
– Mustprotectforbothplannedandunplannedoutages
• SupervisordisrupBonismostcommoncauseofaccessswitchoutages
• NetworkoutageunBlphysicalreplacementorreloadvs.1to3seconds
L2 = SSO L3 = NSF/SSO
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 15
InServiceSonwareUpgradeprocessCatalyst4500and6500
• ISSUupgradeisa4stepprocess• Possibletorollback(abort)upunBlyoucompletethe4thstep(committofinal
state)• LeveragesNSF/SSOtoimplementsupervisortransiBon• RequiresthatthetwoimagesarecompaBbleforupgrade/downgradeprocessing*The[issuacceptversion]isanopBonalstepduringtheISSUprocedure
12.2(xy)SG
12.2(xw)SG
loadversion
12.2(31)SGA
12.2(31)SGA
12.2(31)SGA
12.2(31)SGA1
12.2(31)SGA
12.2(31)SGA1
12.2(31)SGA
12.2(31)SGA1
12.2(31)SGA1
12.2(31)SGA1
runversion *acceptversion commitversion
abortversion Initial state
Final state
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 16
standby
Standby
Active ISSU loadversion Old
2
3 ISSU runversion
ISSU commitversion
ISSU abortversion
5
Old
Old
New
New
4 ISSU acceptversion
Old New New
ISSU abortversion
Switch-1 Switch-2 Switch-1 Switch-2
Switch-1 Switch-2
Switch-2 Switch-2 Switch-1 Switch-1
ISSU abortversion
Standby Active Old
Switch-1 Standby
New Standby
Active New
Active Active Active New
1 Copy the new software image to Active and Standby supervisor flash memory
Oldsonwareimageisrepresentedwithgreencolorandnewsonwareimageis
representedwithpeachcolor
InServiceSonwareUpgradeprocessCatalyst6500VSSSystem
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 17
Release4.0
Release4.1
LinuxKernel
OSPF
BGP
PIM
etc.
HAManager
LinuxKernel
HAManager
AcBve
I/OModuleImages
Upgradeandreboot
Release4.0
Release4.1
OSPF
BGP
PIM
etc.
Standby
IniBatestatefulfailover
Upgradeandreboot
UpgradeandrebootI/Omodules
n7k# install all kickstart bootdisk:4.1-kickstart system bootdisk:4.1-system n7k#
Release4.0
Release4.1
TheTraffickeepsbeingforwardedbytheLinecardForwarding
Engine
N7KDataPlane
Dataplanestreams
InServiceSonwareUpgrade(ISSU)Nexus7000
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 18 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID
DataCenterServicesBlock
DistribuDonBlocks
AGENDA
• SystemsLevelResiliency
• NetworkLevelResiliency–RouBng
• CampusCoreandFoundaBonServices
• EmergingCampusDesign
– RoutedAccess– VirtualSwitchCampusDesign
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 19
NetworkLevelResiliencyRouBngConvergenceImprovements
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 20
RedundancyandProtocolInteracBonLinkNeighborFailureDetecBon
• IndirectlinkfailurestakeBmetodetect
• WithnodirectHWnoBficaBonoflinkornodeloss,convergenceBmesaredependentonSpanningTreeBPDUsorRouBngProtocolHellos
• HardwaredetecBonandrecoveryisbothfasterandmoredeterminisBc
• Usepoint‐to‐pointroutedlinksinCampusCore!
Hellos
HWdetect&recovery
SWiniBatedrecovery
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 21
ImprovingIndirectLayer3NeighbourFailureDetecBon
• EIGRP,OSPF,IS‐IS,mBGPallhavenaBvehello/deadmechanisms
• BidirecBonalForwardingDetecBon(BFD)*providesalightweightprotocolindependentmechanism
• Withpoint‐to‐pointroutedlinks,thecostsofsub‐secondBmers(processingload,complexity,..)mayoutweighthebenefits.
*Verify Cisco IOS Release Availability, ESE does not yet have specific configuration guidance
!Send BFD Hellos every 100ms interface gigabitethernet 4/1 dampening ip address 10.122.0.26 255.255.255.254 bfd interval 100 min_rx 100 multiplier 3 bfd neighbor 10.122.0.27
router eigrp 100 bfd interface gigabitethernet4/1
BFD
RouBngProtocolHello
Metro GigE
Metro GigE
!Send OSPF Hellos every 250ms (1sec/4) interface gigabitethernet 4/1 dampening ip address 10.122.0.26 255.255.255.254 ip ospf dead-interval minimal hello-multiplier 4 ip ospf network point-to-point
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 22
OSPFDesignRulesforHACampusLSA/SPFExponenBalBack‐OffThro`leMechanism
• Sub‐secondBmerswithoutrisk1. spf‐startoriniBalholdBmercontrolshowlongtowaitpriortostarBngtheSPF
calculaBon
2. Ifanewtopologychangeeventisreceivedduringtheholdinterval,theSPFcalculaBonisdelayedunBltheholdintervalexpiresandtheholdintervalistemporarilydoubled
3. TheholdintervalcangrowunBlthemaximumperiodconfiguredisreached
4. AnertheexpiraBonofanyholdinterval,theBmerisreset
Time [ms]
200 1600 msec 100 400 800 msec
Topology Change Events
SPF Calculations
router ospf 100 timers throttle spf <spf-start> <spf-hold> <spf-max-wait> timers throttle lsa all <lsa-start> <lsa-hold> <lsa-max-wait> timers lsa arrival <lsa-hold>
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 23
StableConvergenceIPEventDampening
• PreventsrouBngprotocolchurncausedbyconstantinterfacestatechanges• Dampeningisappliedonasystem:nothingisexchangedbetweenrouBng
protocolsStaBcrouBng,RIP,EIGRP,OSPF,IS‐IS,BGP
InaddiBon,itsupportsHSRPandCLNSrouBng
Appliesonphysicalinterfacesandcan’tbeappliedonsubinterfacesindividually
interface GigabitEthernet1/1 dampening ip address 10.120.0.205 255.255.255.254
Up
Interface State Perceived by OSPF/EIGRP/HSRP, ….
Interface State
Down
Up
Down
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 24
MulBcastDeployment–BestPracBces
• UseIGMPSnoopingcapablehardwareintheaccess
• MulBcastSubsecondConvergence– Join/pruneaggregaBon– PIMHELLOopBon– TriggeredRPF
• UsePIMsparsemode– EnablePIMonALLinterfaces
– EnablePIMsparsemodeonrouBngnodes(Core,DistribuBon,andpossiblyAccess)
– UseAnycastRP&MSDPforRPredundancyandfastconvergence
– ThereareothercombinaBonsofRPredundancy,RPassignmentandothersopBons.
• UsePIM‐SSMEliminatesneedforRPEliminateneedforMSDPHelpspreventunknownsources WAN Internet
RP-Right 10.122.100.1
RP-Left 10.122.100.1
End-to-End Multicast
IPmc Sources
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 25
MulBcastRPEngineeringAnycastRPConfiguraBon
MSDP
Interface loopback 0 ip address 10.0.0.3 255.255.255.255
Interface loopback 1 ip address 10.1.1.1 255.255.255.255 ! ip msdp peer 10.0.0.2 connect-source loopback0 ip msdp originator-id loopback 0
interface loopback 0 ip address 10.0.0.2 255.255.255.255
interface loopback 1 ip address 10.1.1.1 255.255.255.255 ! ip msdp peer 10.0.0.3 connect-source loopback0 ip msdp originator-id loopback 0
ip pim rp-address 10.1.1.1 ip pim rp-address 10.1.1.1
10.1.1.1
RP2
10.1.1.1
RP1
ForYourReference
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 26
MovingtoPIMSourceSpecificMode
Receiverlearnsofsource,group/port
B A C D
F E IGMPv3(S,G)Join
ReceiversendsIGMPv3(S,G)Join
(S,G)Join
First‐hopsendsPIM(S,G)JoindirectlytowardSource
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 27
MovingtoPIMSourceSpecificMode
Result:Shortestpathtreerootedatthesource,withnosharedtree.
B A C D
F E
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 28
IGMPv2 join
Set Top Box (STB)
DNS response:
Group G -> Source S
PIM (S,G) join
PIM (S,G) join
SSMMapping
• UseanexternalorinternaldatabaseforSourcetoGroupmapping
• AllowsonlyforonesourceperGroup
• Routermapsgrouptoasinglesource
• UseseitherDNSorstaBcinternaldatabase
• DNSmethodallowscontentproviderstoprovidethemapping
independentfromnetworkoperators
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 29
SSMDNSMapping–ConfiguraBon
ip igmp ssm-map enable ip igmp ssm-map query dns ! ip pim ssm range SSM-GROUP ! ip access-list standard SSM-GROUP permit 239.0.0.0 0.255.255.255 ! ip name-server 10.151.1.103 ip domain multicast ssm.cisco.fr ip domain-name cisco.fr
Enabling SSM Mapping
Enabling DNS Mapping
cat-3#sh ip igmp ssm-mapping SSM Mapping : Enabled DNS Lookup : Enabled Mcast domain : ssm.cisco.fr Name servers : 10.151.1.103 cat-3#
Specific DNS Server zone for SSM
cat-3#sh ip igmp ssm-mapping 239.1.1.2 Group address: 239.1.1.2 Database : DNS DNS name : 2.1.1.239.ssm.cisco.fr Expire time : 604623026 Source list : 10.151.1.102 cat-3#
1.1.1.239 IN A 10.151.1.104 2.1.1.239 IN A 10.151.1.102 DNS zone for SSM
ForYourReference
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 30 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID
DataCenterServicesBlock
DistribuDonBlocks
AGENDA
• SystemsLevelResiliency
• NetworkLevelResiliency–RouBng
• CampusCoreandFoundaBonServices
• EmergingCampusDesign
– RoutedAccess– VirtualSwitchCampusDesign
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 31
BestPracBces—CampusCore
• AppliestobothMulB‐LayerandRoutedAccesscampusdesigns
• KeeptheCoresimple‐higherthroughputandfastestrecoveryaroundfailures.
• UseL3redundantlinksbetweentheDistribuBonandCore
– Fastre‐routearoundfailures– OpBmalloadbalancing– Noblack‐holesduringrecovery
• Usepoint‐to‐pointroutedinterfaces– NoLayer2switchesorVLANs(SVIs)
• SummarizeroutesintotheCore– FaultisolaBonandfasterfailover
Data Center WAN Internet
Point‐to‐PointInterfaces
RouteSummarizaDon
intoCore
Layer3EqualCostLink’s
interface TenGigabitEthernet3/1 description 10GigE to Distribution dampening ip address 10.122.0.20 255.255.255.254 ip ospf network point-to-point mls qos trust dscp
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 32
• RedundantLayer3equalcostlinksprovidefastconvergence• Hardwarebased—fastrecoverytoremainingpath• Convergenceisextremelyfast(dualequal‐costpaths:noneedforOSPFor
EIGRPtorecalculateanewpath)
Triangles:Link/BoxFailureDoesNotRequireRouDngProtocolConvergence
ModelA
Squares:Link/BoxFailureRequiresRouDngProtocolConvergence
ModelB
BestPracBce—BuildTrianglesNotSquaresDeterminisBcvs.Non‐DeterminisBc
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 33
CEFEqualCostPathRecoveryRedundancyandProtocolInteracBon
• TherecoveryfrommostcomponentfailuresisbasedonL3CEFequalcostpathrecovery
• Timetorestoretrafficflowsisbasedon– Timetodetectlinkfailure
– ProcesstheremovalofthelostroutesfromtheSWFIB
– UpdatetheHWFIB
• Nodependenceonexternalevents(norouBngprotocolconvergencerequired)
• BehaviorisdeterminisBcEqual Cost Links: Link/Box Failure Does Not Require Multi-Box Interaction
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 34
…
…
CEFEqualCostPathRecoveryload‐balancingHashingMechanism
Switch#show mls cef exact-route 10.77.17.8 10.100.20.199
Interface: Gi1/1, Next Hop: 10.10.1.2, Vlan: 1019, Destination Mac: 0030.f272.31fe
Switch#show mls cef exact-route 10.44.91.111 10.100.20.199
Interface: Gi2/2, Next Hop: 10.40.1.2, Vlan: 1018, Destination Mac: 000d.6550.a8ea
Prefix Entries
Adjacency Entry #1
Adjacency Entry #15 Adjacency Entry #16
New MAC and VLAN New MAC and VLAN New MAC and VLAN Adj Idx 15: Rewrite info
New MAC and VLAN New MAC and VLAN New MAC and VLAN New MAC and VLAN New MAC and VLAN New MAC and VLAN
Adj Idx 15+2: Rewrite info Adj Idx 15+1: Rewrite info
Source IP Dest IP Optional L4 Ports
Load-Balancing Hash
IPv4 Lookup—10.100.20.199
Prefix Entries / FIB
172.20.45.1 10.100.20.100
MASK (/32) …
10.100.3.0 10.100.2.0
… 10.100.0.0 172.16.0.0
MASK (/24)
MASK (/16)
Result Memory
Adjacency Entry #25 Adj Idx 15 - Path Count 3
Adjacency Entry #2
Adj Offset: 0 Adj Offset: 1 Adj Offset: 2 Adjacency Table
Hash Result
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 35
Layer2LoopsandSpanningTreeSpanningTreeShouldBehavetheWayYouExpect
• Therootbridgeshouldstaywhereyouputit
LoopguardandrootguardUDLD
• OnlyendstaBontrafficshouldbeseenonanedgeport
BPDUguardPort‐Security
• ThereisareasonablelimittoB‐CastandM‐Casttrafficvolumes
ConfigurestormcontrolonbackuplinkstoaggressivelyratelimitB‐CastandM‐Cast
UBlizeSup720ratelimitersorSupIV/VwithHWqueuingstructure
BPDUGuardorRootguard
PortFastPortSecurity
Rootguard
Loopguard
STPRoot
Loopguard
StormControl
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 36
L2
MulBlayerCampusDesignLayer2AccesswithLayer3DistribuBon
• EachaccessswitchhasuniqueVLAN’s
• Nolayer2loops
• Layer3linkbetweendistribuBon
• Noblockedlinks
• AtleastsomeVLAN’sspanmulBpleaccessswitches
• Layer2loops
• Layer2and3runningoverlinkbetweendistribuBon
• Blockedlinks
Vlan10 Vlan20 Vlan30 Vlan30 Vlan30 Vlan30
L3
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 37
MulBlayerNetworkDesignGoodSolidDesignOpBon,but….
• UBlizesmulBpleControlProtocols– SpanningTree(802.1w,…),FHRP
(HSRP,…),RouBngProtocol(EIGRP,…)
• ConvergenceisdependentonmulBplefactors
– FHRP‐900msecto9seconds
– SpanningTree‐Upto50seconds
– Poorloadbalancing–singleuplink,asymmetricrouBngetc
• STP,ifitbreaksbadly,noinherentmechanismtostoptheloop
MulD‐LayerConvergence
3/2 3/2
3/1 3/1Switch1 Switch2
DSTMAC0000.0000.4444
DSTMAC0000.0000.4444
Seco
nds
of V
oIP
pack
et lo
ss
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 38 © 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential Presentation_ID
DataCenterServicesBlock
DistribuDonBlocks
AGENDA
• SystemsLevelResiliency
• NetworkLevelResiliency–RouBng
• CampusCoreandFoundaBonServices
• EmergingCampusDesign
– RoutedAccess– VirtualSwitchCampusDesign
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 39
RoutedAccessLayer3DistribuBonwithLayer3Access
• MovetheLayer2/3demarcaBontothenetworkedge
• UpstreamconvergenceBmestriggeredbyhardwaredetecBonoflightlostfromupstreamneighbor
• Beneficialfortherightenvironment
10.1.20.010.1.120.0
VLAN20DataVLAN120Voice
VLAN40DataVLAN140Voice
10.1.40.010.1.140.0
EIGRP/OSPF EIGRP/OSPF
GLBP Model
Layer3
Layer2
Layer3
Layer2EIGRP/OSPF EIGRP/OSPF
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 40
RoutedAccessDesignConsideraBonsDesignMoBvaBons
• SimplifiedControlPlane– NoSTPfeatureplacement(rootbridge,loopguard,…)
– Nodefaultgatewayredundancysetup/tuning
– NomatchingofSTP/HSRPpriority
– NoL2/L3mulBcasttopologyinconsistencies
• EaseofTroubleshooBng(leveragewellknowtoolset)– Showiproute– Traceroute– Pingandextendedpings– Extensiveprotocoldebugs– ConsistenttroubleshooBng:access,dist,core
• Failuredifferences– Routedtopologiesfailclosed—i.e.neighborloss– Layer2topologiesfailopen—i.e.broadcastandunknowns
flooded
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 41
RoutedAccessSimplifiedNetworkRecovery
• RoutedAccessnetworkrecoveryisdependentonL3re‐route
• TimetorestoredownstreamflowsisbasedonafullrouBngprotocolre‐route
– Timetodetectlinkfailure
– Timetodeterminenewroute
– ProcesstheupdateoftheSWRIB&FIB
– UpdatetheHWFIB
• TimetorestoreupstreamtrafficflowsisbasedonECMPre‐route
– Timetodetectlinkfailure
– ProcesstheremovalofthelostroutesfromtheSWFIB
– UpdatetheHWFIBUpstream:ECMPRecoveryDownstream:RouDngProtocolRecovery
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 42
RoutedAccessDesignConsideraBonsDesignRequirements
• VLANsarelocalizedtoasinglewiringclosetswitch
• IPaddressing—doyouhaveanaddressallocaBonplantosupportaroutedaccessdesign?
• PlaIormrequirements;– RequiresaCiscoCatalyst3560orabove
– CiscoCatalystIOSFeatureSetconsideraBons
• IPBaseforEIGRP‐StubandPIM*IPServicesforOSPFandPIM
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 43
RoutedAccessDesignAdvantages,YesintheRightEnvironment
• EaseofimplementaBon,lesstogetright
– NomatchingofSTP/FHRPpriority– NoL2/L3mulBcasttopology
inconsistencies– NoSTPconfiguraBoninDist
• Singlecontrolplaneandwellknowntoolset
– traceroute,showiproute,showipeigrpneighbor,etc.
• MostCiscoCatalystssupportL3switchingtoday
• EIGRPconvergesin<200msec
• OSPFconvergesin<200msecwithtuning
• RPVST+convergenceBmesdependentonGLBP/HSRPtuning
BothL2andL3CanProvideSub‐SecondConvergence
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 44
CurrentNetworkScalingChallengesCampusandDataCenter
TradiBonalDataCenterdesignsarerequiringeverincreasingLayer2adjacenciesbetweenserversduetoapplicaBons,VirtualizaBontechnologyandservergrowth.ThesizeofLayer2networksisstretched,placingmoreburdenonloop‐avoidanceprotocols(SpanningTree)
L3Core
L2/L3 Aggregation
L2Access
Dual‐HomedServers,SingleacBveuplinkperVLAN(PVST),L2reconvergence
FHRP,SingleacBveuplinkperVLAN,L2reconvergence,excessiveBPDUs
BGP,IGP,ECMPPolicyManagement
DC Pod: L2 Domain
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 45
VirtualSwitchCatalyst6500VirtualSwitchingSystem(VSS)
• VirtualSwitchingSystemconsistsoftwoCatalyst6500’sdefinedasmembersofthesamevirtualswitchdomainrunningaVSL(VirtualSwitchLink)betweenthem
• SingleControlPlanewithDualAcBveForwardingPlanes
• ExtendsNSF/SSOinfrastructuretoTwoSwitches
VSSSwitch1+Switch2 =
Virtual Switch Domain
VirtualSwitchLink(VSL)
Catalyst6500Series
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 46
VirtualSwitch–VSSTwotoOne
TwoswitcheslooklikeoneTwophysicalswitchesOnevirtualswitch
VirtualSwitch:AllportsappeartobeonthesamephysicalswitchSinglepointofmanagementSingleconfiguraBonSingleIP/MACSinglecontrolplaneprotocolinstance
BenefitsSimplifyinfrastructuremanagementL2DCInterconnectHighAvailability
VirtualSwitchDomain
STP HSRP
OSPF SNMP
STP HSRP
OSPF SNMP STP HSRP
OSPF SNMP
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 47
ImpactofVSSontheCampusDesignControlPlaneSimplificaBon
• VirtualSwitchDesignsimplifiesthetopology
• RedundantsupervisorsprovideresiliencyviaSSO
• NoneedforHSRP,GLBPorVRRP
• AsinglemulBcastrouterontheaccesssubnetssimplifiesthemulBcasttopology
• NoL2loopsinthetopologysoneedforspanningtreetoprovideforlinkredundancy
• DoNOTdisablespanningtreeasitissBllpossibletocreateanexternalloop
• Catalyst6500Load‐balancingschememodifiedtokeeptrafficforwardinglocal
RootBridge
CISF,BPDUGuard
LoopGuard
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 48
VirtualSwitchSystemCampusandDataCenter
AVirtualSwitch‐enabledCampus/Datacenterallowsformaximumscalabilitysobandwidthcanbeaddedwhenrequired,butsBllprovidingalargerLayer2hierarchicalarchitecturefreeofrelianceonSpanningTree…
L3 Core
L2/L3 Aggregation
L2 Access
Dual‐HomedServers,SingleacBveuplinkperVLAN(PVST),FastL2convergence
DualAcBveUplinks,FastL2convergence,minimizedL2ControlPlane,Scalable
Singlerouternode,FastL2convergence,Scalablearchitecture
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 49
VirtualPortchannelsNexus7000vPC
AG1 AG2
2 2 4
TwoPhysicaltoasinglelogicalDevicesconnecttoasingle“logical”switchConnecBonsaretreatedasportchannel
VirtualPortChannel:Portstovirtualswitchcouldformacross‐chassisportchannelvirtualPortchannelbehaveslikearegularEtherchannel
BenefitsProvidenon‐blockingL2pathsLessenRelianceonSTP
Nexus7000
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 50
VirtualPortchannelsNexus7000vPC
WhatIsVirtualPortChannel EliminatesSTPblockedports Leveragesallavailableuplinkbandwidth EliminatesacBve‐standbymodeondual‐homedservers Providesfast,transparentconvergenceuponlink/device
failure
Worksseamlesslywithcurrentnetworkdesign/topology
Downstreamend:– Standardlinkloadbalancingprotocolsavailable
(dependsondownstreamdevice;src/dst‐mac,round‐robin,etc.)
– WorkswithLACPandmanuallyconfiguredlinks
vPCend:– Sameasabove
– Load‐balancingschememodifiedtokeeptrafficforwardinglocal(i.e.,packetheadedintothelinkaggregaBongroupwilluseoneofthelocallinksratherthanacrossthevPCpeer‐link)
Standard Port Channel on Downstream Switches
vPConvPCpeerswithlocalforwarding
Standard Port Channel on Downstream Switches
StandardPortChannelonDownstreamSwitches
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 51
NextGeneraBonCampusDesignEvolvingtheCampusFoundaBonArchitecture
• TradiBonalLayer2designsremainvalid
• Evolvingarchitecturesprovide– SimplifiedControlPlane:
RemovedependenceonSTP
– IncreasedCapacity:Provideflow‐basedloadbalancing
– HighAvailability:200msecorbe`errecovery
• FlexibilitytoprovidefortherightimplementaBonforeachnetworkrequirement
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 52
hbp://www.cisco.com/go/srnd&hbp://www.cisco.com/go/cvd
CampusDesignGuidanceWheretogoformoreinformaBon
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Confidential JMB 53