gigabit campus network design 1

8/7/2019 GIGABIT CAMPUS NETWORK DESIGN 1

1/26

Public

Copyright 2000 Cisco Systems, Inc. All Rights Reserved.

Page 1 of 26

WHITE PAPER

Gigabit Campus DesignConfiguration and Recovery Analysis

Introduction

Some of the most useful variations of the large-scale multilayer campus internetwork design were tested for failure recovery. Please refer to

the companion document Gigabit Campus DesignPrinciples and Architecture for an explanation of several implementations of the

multilayer design. In all cases, Gigabit EtherChannel links were used between distribution-layer switches and core-layer switches.

Redundant Gigabit Ethernet uplinks were used to connect the access-layer switches to the distribution-layer switches. The Catalyst 4000,

5500, and 6000 family switches were used in the wiring closet configurations. The Catalyst 6500 with Multilayer Switch Feature Card

(MSFC) and Catalyst 8540 were both used as the Layer 3 switch. The Catalyst 6500 is used as the Layer 2 switch in the core.

This paper documents configuration and recovery with three fundamental designs:

1. Layer 3 distribution with dual-path Layer 2 core (good)

a) Catalyst 6500 standard building block

2. Layer 3 distribution with Layer 3 core (better)


3. Layer 3 distribution with dual-path Layer 3 core (best)


b) Catalyst 6500 virtual LAN(VLAN) building block

The following two design variations are described in the appendix:

A1. Layer 3 distribution with Layer 3 core


A2. Layer 3 distribution with dual-path Layer 3 core


Layer 2 spanning-tree loops were eliminated in the core and in the standard building block. Avoidance of spanning-tree loops in the core is

important for high availability. Within the core and the standard building block all Layer 3 switches were configured with native routed

interfaces. VLAN trunks were only used within the VLAN building block. The VLAN building block is typically used in a server farm to

provide for redundant server connection with dual Network Interface Cards (NICs).The Enhanced Interior Gateway Routing Protocol (EIGRP) was tuned for fast convergence with a one-second-hello timer and a

three-second-hold timer. The Hot Standby Routing Protocol (HSRP) was tuned for fast convergence with a one-second-hello timer and a

three-second-holdtimer.When using protocolhello timers of onesecond, the limit of accuracyfor recoverymeasurement is about onesecond


2/26

Public

Copyright 2000 Cisco Systems, Inc. All Rights Reserved.Page 2 of 26

Thus, all results are rounded to units of whole seconds, representing the worst case. Network recovery times were very predictable, with some

variation due to the state of things such as the Address Resolution Protocol (ARP) cache. OSPF was configured with the same timer values

as EIGRP, with similar results.

Every router and switch in the network was provided an out-of-band Ethernet management interface on VLAN 99. One Ethernet port on

each device was configured in VLAN 99 and wired outside the test network to a separate switched management network. In addition to the

management IP address on VLAN 99, we also configured Cisco 2511 terminal servers with RJ-45 serial cables to access the console port of

every router and switch in the layout. With this arrangement, we have two out-of-band paths to every switch and router, as well as a number

of in-band paths.

Wherever possible Gigabit EtherChannel was configured with ports on two different cards to increase availability. For example,

port-channel 1 on CT85 (core top 8540) used interfaces gigabit 0/0/0 and gigabit 1/0/0, which are the first physical ports on two different

cards. EtherChannel recovery was not tested, as it is much faster than one second and falls below the measurement threshold of this testing.

Configuration of the Standard Building Block

The standard building block appears on the left side (west block) of Figures 3a, 3b, and 3c. Configuration of the standard building block is

very simplebecause all spanning-tree loopsareeliminated. Hence,there is no Layer 2 tuning, such as selecting the best root switch fora given

VLAN. No VLAN trunks are used and uplinks are connected to native routed interfaces on the Layer 3 switches in the distribution layer.

Catalyst 6000 Access Switch: Standard Building BlockSet prompt aw6 (access layer, west block, Catalyst 6000)

Set vtp domain west

Set vtp mode transparent

(no VLAN trunks, use transparent mode)

Set vlan 99 (used for out of band management)

Set vlan 99 3/48 (last physical port on switch used for oob management)

Set int sc0 99 172.29.196.51 255.255.254.0

(logical console port for oob management)

Set ip route default 172.29.196.1

(gateway router in oob management network)

Set port channel 1/1-2 mode off

(turn off channel negotiation on uplinks)

(not using EtherChannel in this configuration)

set trunk 1/1-2 off(turn off VLAN trunking on uplinks)

set vlan 10(VLAN 10 corresponds to subnet 10 ie 10.10.0.0)

set vlan 10 1/1-2(all other ports are part of VLAN 10)

set vlan 10 3/1-47

We disable channel negotiation on the uplinks to make connection faster after a failure has been restored. This is appropriate because the

uplinks are routed connections to Layer 3 switches in the distribution layer, not Layer 2 connections that require spanning tree. (Nothing in

the configuration indicates that all clients and uplinks attached to the switch are in subnet 10 (10.10.0.0) with mask 255.255.0.0. VLAN 10

is configured everywhere to match subnet 10 [10.10.0.0]).

Configuration of the Standard Building Block with Load Balancing

To achieve load balancing with thestandardbuilding block,a couple of changes are required. Configuretwo VLANs (two subnets)on a wiring

closet switch and use VLAN trunks for uplinks. For example, on switch aw6, configure VLAN 10 and VLAN 11. All outbound VLAN 10

traffic will take one uplink to the HSRP primary gateway router for subnet 10.10.0.0. All outbound VLAN 11 traffic will take the other uplink

to the HSRP primary gateway router for subnet 10.11.0.0. This load balancing configuration maintains all the advantages of the standard

building block and is only marginally more complex.


3/26

Public


Catalyst 6000 Access Switch: Standard Building Block with Load Balancing

Set prompt aw6 (access layer, west block, Catalyst 6000)

Set vtp domain west


(use transparent mode, configure VLANs explicitly)



Set int sc0 99 172.29.196.51 255.255.254.0(logical console port for oob management)





(not using EtherChannel on this switch)

set trunk 1/1-2 on 10,11 dot1q

(VLAN trunking on uplinks, for VLAN 10 and VLAN 11)

set vlan 10 3/1-48(VLAN 10 corresponds to subnet 10 ie 10.10.0.0)


Configuration of the VLAN Building Block

The VLAN building block appears on the right side (east block) of Figures 3a, 3b, and 3c. If dual-attached servers are used, then a VLAN

trunking configuration is required within the server distribution block. The two distribution-layer switches are the root bridges of the even andodd numbered VLANs. UplinkFast is configured on the access-layer switches. BackboneFast is configured on the access and distribution

switchesfor faster spanningtree recovery. Formaximum determinism, VTPtransparent modeis usedand allVLANs are configured explicitly.

HSRP at Layer 3 is configured to match the Layer 2 spanning tree configuration. This way, the HSRP primary gateway router for

even-numbered subnets is also the spanning tree root for even-numbered VLANs. The HSRP primary gateway router for odd-numbered

subnets is also the spanning tree root for odd-numbered VLANs.

Configuration of Catalyst 6000 Access Switch:

Set prompt ae6 (access layer, east block, Catalyst 6000)

Set vtp domain east


(use transparent mode, configure all VLANs explicitly)


Set vlan 99 4/48 (last physical port on switch used for oob management)Set int sc0 99 172.29.196.51 255.255.254.0




set spantree uplinkfast enable

(set uplinkfast on the access switch only)

set spantree backbonefast enable

(enable backbonefast on all switches in block)



(not using EtherChannel on this switch)

set trunk 1/1-2 on 50,51,52,53 dot1q

(dot1q VLAN trunking on uplinks, state VLANs explicitly)

(these four VLANs are used in this building block)




4/26

Public


Configuration of Catalyst 6500 Distribution Switch:

The ARP cache timeout on the MSFC is four hours. However, the Layer 2 CAM table times out in 300 seconds by default. This may result

in some IP unicast traffic being flooded. In the following configuration the CAM timeout agingtime is set to 4hours*60min/hour*60sec/

min. = 14400 seconds to match the ARP cache timeout.

Set prompt det65 (distribution layer, east block, top, Catalyst 6500)

Set vtp domain east

Set vtp mode transparent(use transparent mode, configure all VLANs explicitly)



Set int sc0 99 172.29.196.51 255.255.254.0




set cam agingtime 1-1000 14400

(set CAM timeout to 4 hours to match ARP timeout)

set spantree backbonefast enable

(enable backbonefast on all switches in block)


(turn off channel negotiation on non-Etherchannel links)

set port channel 3/7-8 mode off

set port channel 4/7-8 mode offset port channel 3/1-2 mode on

(EtherChannel used for routed links to the core)

set port channel 4/1-2 mode on

(EtherChannel used for routed links to the core)

set vlan 40 3/1,4/1

(routed etherchannel link VLAN40=10.40.0.0 spans two cards)

set vlan 42 3/2,4/2

(routed etherchannel link VLAN42=10.42.0.0 spans two cards)

set trunk 3/7 on 50,51,52,53 dot1q

(set VLANs and dot1q trunking explicitly)

(3/7-8 and 4/7-8 are uplinks to wiring closet switches)





(this is the backup trunk to other distribution switch)

set spantree root 50,52

(make this root bridge for even VLANs)

set spantree root secondary 51,53

(make this backup root bridge odd VLANs)

set trunk 3/1-2 off

(no VLAN trunking on routed links to core)

set trunk 4/1-2 off

(no VLAN trunking on routed links to core)

As shown in Figures 3b and 3c, VLANs 50, 51, 52, and 53 correspond to the wiring closet subnets. VLANs 40 and 42 correspond to routed

links to the core switches. Refer to section three of the test results for the corresponding MSFC (router) configuration.


5/26

Public


Notes on Test Procedures

Test results are measured by recovery of multiple two-way PING traffic flows. In all cases, test traffic flows are between clients attached to

the network, not to and from the switches within the network. We measure the outage when the failure is created in terms of seconds of lost

traffic. Then we measure the outage when the failure is restored in terms of seconds of lost traffic. Results are rounded up to the next second,

which is the limit of accuracy with one-second-hello protocols. If recovery is given as three seconds in the table, that means we measured two

seconds and three seconds over several tests. The small routing table just reflects the subnets within the test bed. To stress routing protocol

convergence, the large routing table includes 3000 static host routes injected into the test bed from the six Layer 3 switches.

Summary of IP Routing with Small Routing Table

Summary of IP Routing with Large Routing Table

To stress the control plane (routing software running on the CPU) static routes are added to each of the routers (Layer 3 switches) in the test

bed by Trivial File Transfer Protocol (TFTP). TFTP configuration is accomplished with the configure network command. The files on theTFTP server consist of a series of commands of the form:

ip route 10.0.1.001 255.255.255.255 null 0 1

ip route 10.0.1.002 255.255.255.255 null 0 1

ip route 10.0.1.003 255.255.255.255 null 0 1

ip route 10.0.1.004 255.255.255.255 null 0 1

ip route 10.0.1.254 255.255.255.255 null 0 1

end

ctmsfc#sho ip route summary

Route Source Networks Subnets Overhead Memory (bytes)

connected 1 5 336 864

static 1 0 56 144

eigrp 1 0 11 616 1584

internal 2 2328

Total 4 16 1008 4920

ctmsfc#sho ip ro sum

Route Source Networks Subnets Overhead Memory (bytes)

connected 1 5 336 864

static 1 500 28056 72144

eigrp 1 0 2511 140616 361584

internal 2 2328

Total 4 3016 169008 436920


6/26

Public


1. Layer 3 Distribution with Dual-Path Layer 2 CoreCatalyst 6500

Please refer to Figures 1a, 1b, and 1c for this section. Clients test1 through test8 were attached to the wiring closet switches in order to

test network recovery times as different failures were induced. This campus design features redundant Layer 3 switches in each

distribution layer building block and redundant Layer 2 switches in the core. The dual-path Layer 2 core consists of two separate switched

VLANs with no loops andno VLAN trunks. The links into thecore VLANs arenative routedinterfaces on thedistribution-layer switches,

and VLAN trunks are not used.


7/26

Public


Configuration of MSFC: Client-Side Interface on HSRP Prim ary Distribution Switch

interface Vlan10

ip address 10.10.0.81 255.255.0.0

no ip redirects

no ip directed-broadcast

ip hello-interval eigrp 1 1

(set eigrp hello timer 1 second)

ip hold-time eigrp 1 3(set eigrp hold timer 3 seconds)

standby 10 timers 1 3

(set HSRP hello timer 1 hold timer 3 seconds)

(convention - HSRP group number 10 matches VLAN number)

standby 10 priority 200 preempt delay 60

(this is the primary gateway router for subnet 10)

(preempt delay 60 seconds allows EIGRP to stabilize before

HSRP switches back upon power recovery)

standby 10 ip 10.10.0.200

(10.10.0.200 is the HSRP gateway router address)

standby 10 track Vlan31 75


(if you lose both links to the backbone, drop priority

by 150 to initiate HSRP recovery)

Configuration of MSFC: Client-Side Interface on HSRP Secondary Distribution Switch

interface Vlan10

ip address 10.10.0.82 255.255.0.0

no ip redirects



ip hold-time eigrp 1 3



(HSRP secondary or backup gateway router for subnet 10)

standby 10 ip 10.10.0.200



Additional MSFC ConfigurationInterface to Backbone VLAN 31

interface Vlan31

ip address 10.31.0.81 255.255.0.0




Additional MSFC ConfigurationInterface on Management VLAN

interface Vlan99

ip address 172.26.196.81 255.255.254.0


Additional MSFC ConfigurationEIGRP with Passive Interfaces to Wiring Closets

router eigrp 1

passive-interface Vlan10

passive-interface Vlan11passive-interface Vlan12



network 10.0.0.0


8/26

Public


2. Layer 3 Distribution with Layer 3 CoreCatalyst 6500


test network recovery times as different failures were induced. Each distribution-layer switch has a single connection into the core, but

each building block as a whole has redundant connectivity into the core.

This campus design features redundant Layer 3 switches in the distribution layer. For building block west the redundant switches are

dwt65 (distribution west top 6500) and dwb65 (distribution west bottom 6500). Each distribution switch has a two-port GigabitEtherChannel connection to a Layer 3 switch in the core. The HSRP track is configured on dwt65 and dwb65 so that fast HSRP recovery

will take place if the routed link to the core is broken.

There is one important caveat with this design: because each distribution-layer switch has a single path into the core, a redundant routed

path must be provided in case this fails. If the physical link breaks, then HSRP will handle the recovery. However, in the event of some

logical failure of the MSFC within the core switch, HSRP will not be triggered. An example of this is when the MSFC in the core router

is reloaded. Eliminate the passive interface command on two of the wiring closet VLAN interfaces to provide two routed backup paths

between the distribution-layer switches.

Table 3 Results with Small Routing Table

Note: When power is restored to the core switch, the links out to the distribution layer come up before EIGRP has stabilized. Therefore,

HSRP on the distribution switch preempts the primary gateway router function a few seconds before EIGRP on the distribution switch has a

routing table built; hence the 14-second interruption as noted. The preempt delay command-line option refers to delay upon powerup of the

HSRP switch itself, and does not affect this value.

Test FailureTime of

InterruptionRecovery

Mechanism

Wiring Closet Uplink

Fail 4s HSRP

Restore 1s HSRP

Distribution Switch

Fail 4s HSRP

Restore 1s HSRP

Core Switch

Fail 4s HSRP

Restore 14s (See note below)


9/26

Public


Table 4 Results with Large Routing Table

Note: When power is restored to the core switch, the links out to the distribution layer come up before EIGRP has stabilized. Therefore,

HSRP on the distribution switch preempts the primary gateway router function a few seconds before EIGRP on the distribution switch has a

routing table built; hence the 30-second interruption as noted. The preempt delay command-line option refers to delay upon powerup of the

switch running HSRP in the distribution layer, and is not effective when the core switch is powered up.

Configuration of MSFC: Client-Side Interface on HSRP Prim ary Distribution Switch

interface Vlan10

ip address 10.10.0.81 255.255.0.0

no ip redirects





(set eigrp hold timer 3 seconds)








standby 10 ip 10.10.0.200



(if you lose the link to the backbone, drop priority


Test FailureTime of


Mechanism


Fail 4s HSRP

Restore 1s HSRP

Distribution Switch

Fail 4s HSRP

Restore 1s HSRP

Core Switch

Fail 4s HSRP



10/26

Public


Configuration of MSFC: Client-Side Interface on HSRP Secondary Distribution Switch

interface Vlan10

ip address 10.10.0.82 255.255.0.0

no ip redirects




standby 10 timers 1 3standby 10 priority 100 preempt delay 60


standby 10 ip 10.10.0.200


Additional MSFC ConfigurationAny Routed Interface to Ba ckbone

interface Vlan21

ip address 10.21.0.81 255.255.0.0





interface Vlan99

ip address 172.26.196.81 255.255.254.0no ip directed-broadcast


router eigrp 1



(no passive interface on VLAN 12 or VLAN 13)

(two VLANs are kept as redundant routed paths)


network 10.0.0.0



testnetwork recoverytimes as different failureswereinduced. Thisdesign has thehighest level of redundancyand thehighest linkcapacity

into the core.

This campus design features redundant Layer 3 switches in the distribution layer. For building block west the redundant switches are

dwt65 (distribution west top 6500) and dwb65 (distribution west bottom 6500). Each distribution switch has redundant two-port Gigabit

EtherChannel connections to both Layer 3 switches in the core. The HSRP track is configured on dwt65 and dwb65 so that fast HSRP

recovery will take place if both routed links to the core are broken.


11/26

Public


Table 5 Testing with the Standard Building BlockSmall Routing Table

Table 6 Testing with the Standard Building BlockLarge Routing Table

Test FailureTime of


Mechanism


Fail 3s HSRP

Restore 0s HSRP

Distribution Switch

Fail 3s HSRP

Restore 0s HSRP

Core Switch

Fail 0s Dual-path redundancy

Restore 6s EIGRP

Test FailureTime of


Mechanism


Fail 3s HSRP

Restore 0s HSRP

Distribution Switch

Fail 3s HSRP

Restore 0s HSRP

Core Switch


Restore 20s EIGRP


12/26

Public


Table 7 Testing with the Standard Building BlockSmall Routing Table

Note: Restoration of the distribution-layer switch causes a 15-second outage. This is a result of the fact that the Layer 2 and Layer 3 path

determination functions on the switch recover at slightly different moments. When the routing table is increased to 3000 routes in the next

test, this anomaly is eliminated.

Table 8 Testing with the VLAN Building BlockLarge Routing Table

Test FailureTime of


Mechanism


Fail 2s UplinkFast

Restore 0s UplinkFast

Distribution Switch

Fail 3s UplinkFast/HSRP


Core Switch


Restore 0s Dual-path redundancy

VLAN Backup Trunk

Fail 31s Spanning tree

Restore 28s Spanning tree

Test FailureTime of


Mechanism


Fail 2s UplinkFast

Restore 0s UplinkFast

Distribution Switch

Fail 3s UplinkFast/HSRP


Core Switch


Restore 0s Dual-path redundancy

VLAN Backup Trunk

Fail 31s Spanning tree

Restore 28s Spanning tree


13/26

Public


Note: When the distribution switch is restored, a five-second outage results because the Layer 2 topology stabilizes at a slightly different

moment than the Layer 3 topology. With the large routing table this difference is reduced from the previous test.

Configuration of MSFC: Client-Side Interface on Distribution SwitchHSRP Primary

interface Vlan10

ip address 10.10.0.81 255.255.0.0

no ip redirects





(set eigrp hold timer 3 seconds)








standby 10 ip 10.10.0.200


standby 10 track Vlan20 75standby 10 track Vlan21 75

(if you lose both links to the backbone, drop priority


Configuration of MSFC: Client-Side Interface on Distribution SwitchHSRP Secondary

interface Vlan10

ip address 10.10.0.82 255.255.0.0

no ip redirects







standby 10 ip 10.10.0.200



(track both links to the backbone)

Additional MSFC ConfigurationAny Routed Interface to Backbone

interface Vlan21

ip address 10.21.0.81 255.255.0.0





interface Vlan99

ip address 172.26.196.81 255.255.254.0



router eigrp 1






network 10.0.0.0


14/26

Public


4. Layer 3 Distribution with Layer 3 CoreCatalyst 8540

Please refer to Figures A1a, A1b, and A1c for this section. Clients target1 through target6 were attached to the wiring closet switches in

order to test network recovery times as different failures were induced. Each distribution-layer switch has a single connection into the

core, but each building block as a whole has redundant connectivity into the core.

This campus design features redundant Layer 3 switches in the distribution layer. For building block left the redundant switches are

dlt85 (distribution left top 8540) and dlb85 (distribution left bottom 8540). Each distribution switch has a two-port Gigabit EtherChannelconnection to a Layer 3 switch in the core. The HSRP track is configured on dlt85 and dlb85 so that fast HSRP recovery will take place

if the routed link to the core is broken.



Test FailureTime of


Mechanism


Fail 4s HSRP

Restore 1s HSRP

Distribution Switch

Fail 3s HSRP

Restore 1s HSRP

Core Switch

Fail 6s EIGRP

Restore 3s HSRP

Test FailureTime of


Mechanism


Fail 4s HSRP

Restore 1s HSRP

Distribution Switch

Fail 3s HSRP

Restore 2s HSRP

Core Switch

Fail 7s EIGRP

Restore 3s HSRP


15/26

Public


Configuration Details of HSRP Primary Distribution Router dlt85

interface Port-channel1

ip address 10.70.0.53 255.255.0.0




hold-queue 300 in

!interface Port-channel2

ip address 10.71.0.53 255.255.0.0




hold-queue 300 in

!

interface GigabitEthernet0/0/1

no ip address


channel-group 1

!


ip address 10.60.0.53 255.255.0.0

no ip redirectsno ip directed-broadcast



standby timers 1 3

standby priority 200

standby preempt

standby ip 10.60.0.200

standby track Port-channel1 150

!


no ip address


channel-group 1

!


ip address 10.61.0.53 255.255.0.0

no ip redirects




standby timers 1 3


standby preempt



!


no ip address

no ip directed-broadcastchannel-group 2

!


ip address 10.62.0.53 255.255.0.0

no ip redirects





16/26

Public


standby timers 1 3


standby preempt



!


no ip address


channel-group 2

!

interface Ethernet0 (management interface)

ip address 172.26.196.53 255.255.254.0


!

!

router eigrp 1

passive-interface GigabitEthernet0/0/0




network 10.0.0.0!

end


Please refer to Figures A2a, A2b, and A2c for this section. Clients target1 through target6 were attached to the wiring closet switches in

order to test network recovery times as different failures were induced. The dual-path design has the highest level of redundancy and the

highest link capacity into the core.

This campus design features redundant Layer 3 switches in the distribution layer. For building block left the redundant switches are

dlt85 (distribution left top 8540) and dlb85 (distribution left bottom 8540). Each distribution switch has redundant two-port Gigabit

EtherChannel connections to both Layer 3 switches in the core. The HSRP track is configured on dlt85 and dlb85 so that fast HSRP

recovery will take place if both routed links to the core are broken.


Test FailureTime of


Mechanism


Fail 3s HSRP

Restore 0s HSRP

Distribution Switch

Fail 3s HSRP

Restore 0s HSRP

Core Switch

Fail 3s EIGRP

Restore 1s EIGRP


17/26

Public



Note: Restoring the distribution switch results in an outage between the time that HSRP switches and the EIGRP routing table is built. This

can be corrected by adding a preempt delay as in the following command:


Configuration details of HSRP primary distribution router dlt85


ip address 10.70.0.53 255.255.0.0




hold-queue 300 in

!


ip address 10.71.0.53 255.255.0.0




hold-queue 300 in

!


no ip address


channel-group 1

!


ip address 10.60.0.53 255.255.0.0

no ip redirectsno ip directed-broadcast



standby timers 1 3


standby preempt




!

Test FailureTime of


Mechanism


Fail 3s HSRP

Restore 0s HSRP

Distribution Switch

Fail 3s HSRP


Core Switch

Fail 10s EIGRP

Restore 1s EIGRP


18/26

Public



no ip address


channel-group 1

!


ip address 10.61.0.53 255.255.0.0

no ip redirects




standby timers 1 3


standby preempt




!


no ip address


channel-group 2

!interface GigabitEthernet3/0/0

ip address 10.62.0.53 255.255.0.0

no ip redirects




standby timers 1 3


standby preempt




!


no ip address


channel-group 2

!

interface Ethernet0

ip address 172.26.196.53 255.255.254.0


!

!

router eigrp 1




passive-interface GigabitEthernet3/0/0network 10.0.0.0

!

end


19/26

Public


1A: Dual-Path Layer 2 CoreLayout, Catalyst 6500

1B: Dual-Path Layer 2 CoreLogical, Catalyst 6500

aw3

aw4

aw5

aw6

ae3

ae4

ae5

ae6

Test1

Test2

Test3

Test4

Test5

Test6

West Block

Standard

East Block

VLAN

dwt65

dwb65

ct65

cb65

det65

deb65

Access Distribution Core Distribution Access

Gigabit VLAN Trunk

Gigabit Ethernet

Gigabit EtherChannelDual

aw3

aw4

aw5

aw6

ae3

ae4

ae5

ae6

10.10.0.101

10.11.0.101

10.12.0.101

10.13.0.101

10.50.0.101

10.51.0.101

10.52.0.101

10.53.0.101

Test4

Test5

Test6

West Block

Standard

East Block

VLAN

dwt65

dwb65

ct65

cb65

det65

deb65


Gigabit VLAN Trunk

Gigabit Ethernet


10.10.0.0

172.26.196.11

10.x.0.25

10.x.0.200 Pri

172.26.196.25

10.x.0.29

10.x.0.200 Pri

172.26.196.29

10.x.0.26

10.x.0.200 Sec

172.26.196.26

10.x.0.30

10.x.0.200 Sec

172.26.196.30

10.31.0.0

172.26.196.27

10.32.0.0

172.26.196.28

10.11.0.0172.26.196.12

10.12.0.0172.26.196.13

10.13.0.0172.26.196.14

10.5x.0.0

172.26.196.21

10.5x.0.0172.26.196.22

10.5x.0.0

172.26.196.23

10.5x.0.0172.26.196.24

Test1

Test2

Test3


20/26

Public


1C: Dual-Path Layer 2 CoreBackbone, Catalyst 6500

2A: Layer 3 CoreLayout, Catalyst 6500

West Block East Block

10.31.0.0

10.32.0.0

VLAN 31

VLAN 31 VLAN 31

VLAN 31

VLAN 32 VLAN 32

VLAN 32 VLAN 32

ct65

cb65

dwt65 det65

dwb65 deb65

Gigabit VLAN TrunkGigabit Ethernet


aw3

aw4

aw5

aw6

ae3

ae4

ae5

ae6

Test1

Test2

Test3

Test4

Test5

Test6

West Block

Standard

East Block

VLAN

dwt65

dwb65

ct65

cb65

det65

deb65


Gigabit VLAN Trunk

Gigabit Ethernet



21/26

Public


2B: Layer 3 CoreLogical, Catalyst 6500

2C: Layer 3 CoreBackbone, Catalyst 6500

aw3

aw4

aw5

aw6

ae3

ae4

ae5

ae6

10.10.0.101

10.11.0.101

10.12.0.101

10.13.0.101

10.50.0.101

10.51.0.101

10.52.0.101

10.53.0.101

Test4

Test5

Test6

West Block

Standard

East Block

VLAN

dwt65

dwb65

ct65

cb65

det65

deb65


Gigabit VLAN Trunk

Gigabit Ethernet


10.10.0.0

172.26.196.11

10.x.0.25

10.x.0.200 Pri

172.26.196.25

10.x.0.29

10.x.0.200 Pri

172.26.196.29

10.x.0.26

10.x.0.200 Sec

172.26.196.26

10.x.0.30

10.x.0.200 Sec

172.26.196.30

10.x.0.27

172.26.196.27

10.x.0.28

172.26.196.28

10.11.0.0

172.26.196.12

10.12.0.0

172.26.196.13

10.13.0.0

172.26.196.14

10.5x.0.0

172.26.196.21

10.5x.0.0

172.26.196.22

10.5x.0.0

172.26.196.23

10.5x.0.0

172.26.196.24

Test1

Test2

Test3


VLAN 20 VLAN 20

VLAN 30

VLAN 30

VLAN 40VLAN 40

VLAN 23 VLAN 23 VLAN 43VLAN 43

ct65

cb65

dwt65 det65

dwb65 deb65

Gigabit VLAN Trunk

Gigabit Ethernet


10.20.0.0 10.40.0.0

10.23.0.0 10.43.0.0

10.30.0.0


22/26

Public


3A: Dual-Path Layer 3 CoreLayout, Catalyst 6500

3B: Dual-Path Layer 3 CoreLogical, Catalyst 6500

aw3

aw4

aw5

aw6

ae3

ae4

ae5

ae6

Test1

Test2

Test3

Test4

Test5

Test6

West Block

Standard

East Block

VLAN

dwt65

dwb65

ct65

cb65

det65

deb65


Gigabit VLAN Trunk

Gigabit Ethernet



23/26

Public


3C: Dual-Path Layer 3 CoreBackbone, Catalyst 6500

A1A: Layer 3 CoreLayout, Catalyst 8540


VLAN 22

VLAN 22

VLAN 20 VLAN 20

VLAN 30

VLAN 30

VLAN 41

VLAN 42VLAN 21

VLAN 21

VLAN 40VLAN 40

VLAN 23 VLAN 23 VLAN 43VLAN 43

VLAN 42

VLAN 41

ct65

cb65

dwt65 det65

dwb65 deb65

Gigabit VLAN Trunk

Gigabit Ethernet


10.20.0.0 10.40.0.0

10.23.0.0 10.43.0.0

10.42.0.0

10.41.0.0

10.30.0.0

10.21.0.0

10.22.0.0

al4

al5

al6

ar4

ar5

ar6

Target1

Target2

Target3

Target4

Left Block Right Block

dlt85

dlb85

ct85

cb85

drt85

drb85


Gigabit VLAN Trunk

Gigabit Ethernet



24/26

Public


A1B: Layer 3 CoreLogical, Catalyst 8540

A1C: Layer 3 CoreTopology, Catalyst 8540

al4

al5

al6

ae4

ae5

ae6

10.60.0.101

10.61.0.101

10.62.0.101

10.100.0.101

10.101.0.101

10.102.0.101

Target3

Target4


dlt85

dlb85

ct65

cb85

drt85

drb85


Gigabit VLAN Trunk

Gigabit Ethernet


10.x.0.53

10.x.0.200 Pri

172.26.196.53

10.x.0.59

10.x.0.200 Pri

172.26.196.59

10.x.0.54

10.x.0.200 Sec

172.26.196.54

10.x.0.60

10.x.0.200 Sec

172.26.196.60

10.x.0.57

172.26.196.57

10.x.0.58

172.26.196.58

10.60.0.0

172.26.196.50

10.61.0.0

172.26.196.51

10.62.0.0

172.26.196.52

10.100.0.0

172.26.196.61

10.101.0.0

172.26.196.62

10.102.0.0

172.26.196.63

Target1

Target2


dlt85 ct85 drt85

10.90.0.0

10.80.0.0

10.70.0.0

Port-chan 1

int gig 0/0/1

int gig 1/0/1

Port-chan 1

int gig 0/0/0

int gig 1/0/0

Port-chan 3

int gig 0/0/1

int gig 1/0/1

Port-chan 1

int gig 0/0/0

int gig 1/0/0

dlb85 cb85 drb85

10.93.0.010.73.0.0

Port-chan 2

int gig 2/0/1

int gig 3/0/1

Port-chan 2

int gig 2/0/0

int gig 3/0/0

Port-chan 4

int gig 2/0/1

int gig 3/0/1

Port-chan 2

int gig 2/0/0

int gig 3/0/0

Port-chan 5

int gig 9/0/0

int gig 9/0/1

Port-chan 5

int gig 9/0/0

int gig 9/0/1

Gigabit VLAN Trunk

Gigabit Ethernet



25/26

Public


A2A: Dual-Path Layer 3 CoreLayout, Catalyst 8540

A2B: Dual-Path Layer 3 CoreLogical, Catalyst 8540


al4

al5

al6

ar4

ar5

ae6

Target1

Target2

Target3

Target4

dlt85 ct85

dlb85 cb85

drt85

drb85


Gigabit VLAN Trunk

Gigabit Ethernet


al4

al5

al6

ar4

ar5

ar6

10.60.0.101

10.61.0.101

10.62.0.101

10.100.0.101

10.101.0.101

10.102.0.101

Target3

Target4


dlt85

dlb85

ct85

cb85

drt85

drb85


Gigabit VLAN Trunk

Gigabit Ethernet


10.x.0.5310.x.0.200 Pri

172.26.196.53

10.x.0.5910.x.0.200 Pri

172.26.196.59

10.x.0.54

10.x.0.200 Sec

172.26.196.54

10.x.0.60

10.x.0.200 Sec

172.26.196.60

10.x.0.57

172.26.196.57

10.x.0.58

172.26.196.58

10.60.0.0

172.26.196.50

10.61.0.0

172.26.196.51

10.62.0.0

172.26.196.52

10.100.0.0

172.26.196.61

10.101.0.0

172.26.196.62

10.102.0.0

172.26.196.63

Target1

Target2


26/26

Cisco Systems has more than 200 offices in the following countries. Addresses, phone numbers, and fax numbers are listed on the

C i s c o C o n n e c t i o n O n l i n e W e b s i t e a t h t t p : / / w w w . c i s c o . c o m / o f f i c e s .

Argentina Australia Austria Belgium Brazil Canada Chile China Colombia Costa Rica Croatia Czech Republic Denmark Dubai, UAE Finland France

Germany Greece Hong Kong Hungary India Indonesia Ireland Israel Italy Japan Korea Luxembourg Malaysia Mexico The Netherlands New

Corporate Headquarters

Cisco Systems, Inc.170 West Tasman DriveSan Jose, CA 95134-1706USAhttp://www.cisco.comTel: 408 526-4000

800 553-NETS (6387)Fax: 408 526-4100

European Headquarters

Cisco Systems Europe s.a.r.l.Parc Evolic, Batiment L1/L216 Avenue du QuebecVillebon, BP 70691961 Courtaboeuf CedexFrancehttp://www-europe.cisco.comTel: 33 1 69 18 61 00Fax: 33 1 69 28 83 26

Americas

HeadquartersCisco Systems, Inc.170 West Tasman DriveSan Jose, CA 95134-1706USAhttp://www.cisco.comTel: 408 526-7660Fax: 408 527-0883

Asia Headquarters

Nihon Cisco Systems K.K.Fuji Building, 9th Floor3-2-3 MarunouchiChiyoda-ku, Tokyo 100Japanhttp://www.cisco.comTel: 81 3 5219 6250Fax: 81 3 5219 6001

A2C: Dual-Path Layer 3 CoreBackbone, Catalyst 8540


Port 2

Port 1

Port 1 Port 1

Port 5

Port 5

Port 4

Port 3Port 1

Port 2

VLAN 40Port 3

Port 2 Port 2 Port 2Port 4

Port 2

Port 1

ct85

cb85

dlt85 drt85

dlb85 drb85

Gigabit VLAN Trunk

Gigabit Ethernet


10.70.0.0 10.90.0.0

10.73.0.0 10.93.0.0

10.92.0.0

10.91.0.0

10.80.0.0

10.71.0.0

10.72.0.0

Conclusion

Choose a deterministic, structured design model to achieve high

availability in your enterprise network. Apply redundancy in the

mission-critical parts of the network. Scale the enterprise

network to the size required by choosing the appropriate

building block model and combining with the right backbone

model. For a better understanding of the different choices

referred to in this paper refer to the paper Gigabit Campus

Network DesignPrinciples and Architecture.

Geoff Haviland ([email protected])Network Design

Engineer.

gigabit campus network design 1

Documents