cisco catalyst 6500 switch architecture

1© 2006 Cisco Systems, Inc. All rights reserved. Cisco PublicRST-3465 12523_04_2006_c2

Cisco Catalyst 6500 Switch Architecture

RST-3465

2© 2006 Cisco Systems, Inc. All rights reserved.RST-3465 12523_04_2006_c1 Cisco Public

Session Goal

To provide you with a thorough understanding of the Catalyst® 6500 switching architecture, packet flow, and key forwarding engine functions

222


Agenda

• Chassis Architecture

• Supervisor Engine and Switch Fabric Architecture

• Switching Module Architecture

• Layer 2 Forwarding

• IPv4 Forwarding

• IPv4 Multicast Forwarding

• Security and Feature ACLs

• QoS

• NetFlow


Chassis Architecture


Catalyst 6500 Chassis Architecture

• Modular chassis in variety of form factors3, 4, 6, 9, and 13- slot versions

• Enhanced (“E”) chassis offer higher system power capacity and better signal integrity

3, 4, 6, and 9- slot versions

• Classic switching bus traces/connectors• Crossbar fabric traces/connectors• Redundant power supplies• Fan tray for system cooling

6509- NEB- A chassis offers redundant fan trays and air filtration

• Redundant voltage termination (VTT)/clock modules

• Redundant MAC addressEEPROMs


Catalyst 6503/6503E and 6504E

• Slots 1 and 2—Supervisor engine, or switching module

• Other slots—Any switching module

• 2 fabric channels per slot

• Power supplies in rear6503/6503E—Power entry modules (PEMs) in front of chassis provides power connection

• 950W AC/DC and 1400W AC power supplies for 6503/6503E

• 2700W AC/DC power supplies for 6504E

Slot 1Slot 2

Slot 3

VTT/Clock Modules EEPROMs

Shared BusCrossbar

Dual ChannelsDual Channels

Dual ChannelsDual ChannelsDual ChannelsDual ChannelsFan Tray

PowerSupply

PowerSupply

Note: CEF720 modules not supported in

Catalyst 6503 (non-E) chassis

Dual ChannelsDual Channels Slot 4

5 RU

4 RU


EEPROMs

Slot 1

Slot 2

Slot 3

Shared BusCrossbar










Slot 4

Slot 5

Slot 6

Slot 7

Slot 8

Slot 9

Fan Tray

PowerSupply

PowerSupply

Catalyst 6506/6509 and 6506E/6509E• Slots 1 and 2—Supervisor Engine 2, or

switching module

• Slots 5 and 6—Supervisor Engine 32/720, or switching module

• Other slots—Any switching module

• 2 fabric channels per slot

• Wide variety of power supplies, from legacy 1000W to new 6000W—E chassis requires at least 2500W PS

• NEB-A chassis has vertical slot alignment, dual fan trays, front-to-back air flow, air filtration system

VTT/Clock Modules

12 RU

15 RU

21 RU


VTT/Clock Modules

Slot 1

Slot 2

Slot 3

EEPROMs

Shared BusCrossbar

Single ChannelSingle Channel









Slot 4

Slot 5

Slot 6

Slot 7

Slot 8

Slot 9Dual ChannelsDual Channels



Dual ChannelsDual ChannelsSlot 10

Slot 11

Slot 12

Slot 13

Fan Tray

PowerSupply

PowerSupply

Catalyst 6513

• Slots 1 and 2—Supervisor Engine 2, or switching module

• Slots 7 and 8—Supervisor Engine 32/720, or switching module

• Wide variety of power supplies, from 2500W to new 6000W

• 1 fabric channel slots 1–8

Dual-fabric modules not supported in slots 1–8!

• 2 fabric channels slots 9–13

Any switching module

19 RU


Agenda





• IPv4 Forwarding



• QoS

• NetFlow


Supervisor Engine and Switch Fabric Architecture


Supervisor 2

• PFC2 forwarding engine daughter card

• Switch Processor CPU (300MHz R7000)

• Optional MSFC2 daughter card with Route Processor CPU (300MHz R7000)

• 256MB/256MB (Sup2) or 256MB/512MB (Sup2U) DRAM

• Internal RP and SP bootflash (32MB each)

• External PCMCIA flash slot

• Supports optional Switch Fabric Module (SFM)/SFM2

• 2 x 1GE GBIC uplink ports

Supported from Cisco IOS 12.1(5c)EX and Catalyst OS 6.1(1)/12.1(3a)E1


Supervisor 2 / PFC2 Architecture

Supervisor 2 Baseboard

PFC2Daughter

Card

ACLTCAM

FIBTCAM

L2CAM

DBUSRBUS16 Gbps

BusEOBC

GbEUplinksMSFC2 Daughter Card

1 Gbps

DRAM

MET

ReplicationEngineFabric Interface

8 Gbps

To SFM/SFM2

QoSTCAM

ADJ

NetFlow

L2/L4Engine

Layer 3Engine

BusInterface

LCDBUS

LCRBUS

Port ASICSP (NMP)

CPU 1 Gbps

DRAM RP (MSFC2)CPU

SP CPU runs L2 protocols and

manages hardware

RP CPU runs L3 protocols and

maintains control plane state

Interface to fabric and bus

Replication engine for multicast/

SPAN

ADJ contains rewrite info

NetFlow table for stats and features

FIB contains IPv4 prefix entries

QoS TCAM contains QoS ACL

entries

L2 CAM contains

MAC entries

ACL TCAM contains security

and feature ACL entries


Supervisor 720

• 720Gbps crossbar fabric

• PFC3 forwarding engine daughter card

• Integrated RP/SP CPUs on MSFC3 daughter card (600MHz MIPS)

• 512/512MB (3A/B) or 1/1GB (3BXL) DRAM

• Internal RP and SP bootflash (64MB each)

• Optional 512MB CF bootflash upgrade for SP (WS-CF-UPG=)

• Dual external compact flash slots

• 2 x GbE uplink ports—2 x SFP <or>1 x SFP and 1 x 10/100/1000

Supported from Cisco IOS 12.2(14)SX and Catalyst OS 8.1(1)/12.2(14)SX2


Supervisor 720 / PFC3 Architecture

Supervisor 720 Baseboard

PFC3Daughter

Card

Integrated720 Gbps

Switch Fabric

L3/4Engine

NetFlowACLTCAM

QoSTCAM

FIBTCAM ADJ

L2CAM

…

20 Gbps

17 x 20 GbpsFabric

Channels

DBUSRBUS

16 GbpsBus

FabricInterface/

ReplicationEngine

1 Gbps

1 Gbps

CPU Daughter CardGbE Uplinks

MET

CounterFPGA

(B/BXL Only)

DRAM

DRAM

EOBC

Port ASIC

L2 Engine

RP (MSFC3)CPU

SP (NMP)CPU

RP and SP both sit on MSFC3 CPU daughter

card

Crossbar switch fabric integrated

on supervisor baseboard

Fabric interface and replication

engine combinedACL and QoS

classification move to L3/4 engine

L2 CAM moved on-chip for

higher performance

Addition of ACL TCAM counters


Supervisor 32• Classic supervisor—no fabric, uses

16Gig bus only

• PFC3B forwarding engine daughter card

• SP CPU (400MHz Sibyte)

• MSFC2a routing engine

• 256MB/256MB DRAM (512MB/512MB with non-$0 feature set)

• Internal CF bootdisk (256MB) and MSFC2A bootflash (64MB)

• External CF slot

• Uplink options:8 SFP + 1 10/100/1000

2 10GE + 1 10/100/1000

Supported from Cisco IOS 12.2(18)SXF and Catalyst OS 8.4(1)/12.2(17)SXB7

8 1GE SFP +1 10/100/1000 RJ-45

uplink ports

2 10GE Xenpak +1 10/100/1000 RJ-45 uplink ports


Supervisor 32-GE / PFC3 Architecture

Supervisor Engine 32 Baseboard

DBUSRBUS

16 GbpsBus

EOBC

GbE Uplinks

MSFC2a Daughter Card

1 Gbps DRAM

DRAMSP CPU

MET

1 GbpsPort ASIC

ReplicationEngine

PFC3Daughter

Card

L3/4Engine

NetFlowACLTCAM

QoSTCAM

FIBTCAM ADJ

L2CAM

CounterFPGA

L2 Engine

RP CPU

WS-SUP32-GE-3B

Bus attached only; no fabric support

PFC3 exactly the same as on

Supervisor 720


Supervisor Engine 32 Baseboard

DBUSRBUS

16 GbpsBus

EOBC

10GE Uplinks

MSFC2a Daughter Card

1 Gbps DRAM

DRAMSP CPU

MET

1 Gbps

ReplicationEngine

PFC3Daughter

Card

L3/4Engine

NetFlowACLTCAM

QoSTCAM

FIBTCAM ADJ

L2CAM

CounterFPGA

L2 Engine

RP CPU

WS-SUP32-10GE-3B

PortASIC

FPGAMUX

PortASIC

Supervisor 32-10GE / PFC3 Architecture

Dual port ASICs to support two 10GE

interfaces


Supervisor Chassis Requirements

Supervisor 720 and Supervisor 32 require:

• Catalyst 6500 or 6500-E chassis

• High speed fan tray (FAN2/E-FAN)

• 2500W power supply (AC or DC) or greater3000W supply recommended for new deployments

• Specific chassis slots:Slot 1 or 2 in 3/4 slot

Slot 5 or 6 in 6/9 slot

Slot 7 or 8 in 13 slot


Crossbar Switch Fabric

• Provides multiple conflict-free paths between switching modulesDedicated bandwidth per slot

Compare to system bus which is shared by all bus-attached modules

• 18 fabric channels in total

• Two fabric channels per slot in 6503/6504/6506/6509

• In 6513:One fabric channel slots 1–8

Two fabric channels slots 9–13

“Dual-fabric channel” modules not supported in slots 1–8 of 6513


Switch Fabric Module and SFM2

• 256 Gbps crossbar switch fabric

• Works with Supervisor 2 and CEF256/dCEF256 modules

• Fabric channels run at 8 Gbps full duplex8 Gbps in/8 Gbps out per channel

• Fabric module occupies a full slot6506/6509—Slots 5 and 6

6513—Slots 7 and 8

• SFM—Supports 6506 and 6509 (and E-versions)

• SFM2—Supports 6506, 6509, and 6513 (and E-versions)

• Not supported in 6503/6504


• 720 Gbps crossbar switch fabric • Integrated on Supervisor 720 baseboard• Fabric channels run at 20 Gbps

full duplex20 Gbps in/20 Gbps out per channel

• Works with all fabric-capable modulesFabric channels auto-sync speed onper-slot basis (8 Gbps or 20Gbps)

Supervisor 720 Switch Fabric


Monitoring Fabric Status and Utilization

• Cisco IOS: show fabric [active | channel-counters | errors | fpoe | medusa | status | switching-mode | utilization]

• Catalyst OS: show fabric {channel {counters | switchmode | utilization} | status}

6506#show fabric utilization

slot channel speed Ingress % Egress %

1 0 8G 22 23

2 0 8G 4 9

3 0 20G 0 1

3 1 20G 11 12

4 0 20G 0 1

4 1 20G 10 13

6 0 20G 0 1

6506#


Monitoring System Bus Utilization

• Monitor the traditional Catalyst 6500 bus when using:Classic modules Centralized forwarding with a fabric

• Cisco IOS: show catalyst6000 traffic-meter• Catalyst OS: show traffic

6506#show catalyst6000 traffic-meter

traffic meter = 7% Never cleared

peak = 46% reached at 08:07:50 PST Fri Dec 30 2005

6506#


Policy Feature Cards

• Mandatory daughter card for supervisor engine

• Provides the key components enabling high-performance hardware packet processing

• Supervisor 2 supports PFC2

• Supervisor 32 supports PFC3B

• Supervisor 720 supports:PFC3A

PFC3B

PFC3BXL


Policy Feature Cards (Cont.)

• Layer 2 switching

• IPv4 unicast forwarding

• IPv4 multicast forwarding

• Security ACLs

• QoS/policing

• NetFlow accounting

PFC3 also supports:

• IPv6, MPLS*/VRF-lite, Bidir PIM, NAT/PAT, GRE/v6 tunnels, CoPP

Key hardware-enabled features:

* MPLS on 3B/3BXL only


Ingress Forwarding

Engine

High-Level Forwarding Engine Logic

FIB lookup

Input ACL lookup

NetFlow lookup

FIB TCAM

ACL TCAM

NetFlow Table

Yes

Output Layer 2 lookup

Layer 2 Table

Output QoS lookup*

Output ACL lookup

ACL TCAMQoS TCAM

Input QoS lookup

QoS TCAM

Bridged NetFlow

NetFlow Table

No

Input ACL lookup

QoS TCAM

Transmit frame

ACL TCAMQoS TCAM

Output QoS lookup*

Output ACL lookup

Input QoS lookup

ACL TCAM

Router MAC?

Frame received

Layer 2 Table

Input Layer 2 lookup

*PFC3 only


PFC TCAM Technology• TCAM—Ternary Content

Addressable Memory

• Leveraged heavily in Catalyst 6500

FIB, ACL, QoS, NetFlow all utilize TCAM memory

• All entries accessed in parallel—fixed performance independent of number of entries

• Memory consists of groups of values and associated masks

8:1 ratio of values to masks• Masks are used to “wildcard”

some portion of values

Masks

Mask 1

Mask 2

Values

Value 1Value 2

Value 4Value 5Value 6Value 7Value 8

Value 3

Value 1Value 2

Value 4Value 5Value 6Value 7Value 8

Value 3


123456

78

123456

78

Compare

Result

11111100

Masks Values

110110xx000111xx

100111xx000000xx010010xx111111xx001100xx

101101xx

11110000

0111xxxx1011xxxx

0110xxxx1110xxxx0011xxxx0000xxxx1000xxxx

1101xxxx

Generic TCAM Lookup Logic

1. Relevant fields read from contents of packet

2. Lookup key created

3. As lookup key compared to value entries, associated mask applied

4. Longest match returns resultResult format varies depending on lookup type

Lookup Key

1=“Compare”0=“Mask”

Generate Lookup

KeyFieldsPacket1

2

3

0110xxxx011010xx01101010

HIT!4


Agenda





• IPv4 Forwarding



• QoS

• NetFlow


Switching Module Architecture


Classic Module

Classic Module

DBUSRBUS

PortASIC

PortASIC

PortASIC

PortASIC

Classic Module

DBUSRBUS

PortASIC

Port ASICs for physical connectivity, buffering,

and queueing

4xGE 4xGE 4xGE 4xGE

Example: WS-X6416-GBIC

48x10/100Example: WS-X6148A-RJ-45


DBUSRBUS

8Gbps FabricChannel

CEF256Module

LCDBUSLCRBUS

ReplicationEngine Port

ASICPortASIC

PortASIC

PortASIC

MET

4xGE 4xGE 4xGE 4xGE

FabricInterface

CEF256 ModuleExample: WS-X6516-GBIC

Fabric interface to interface with fabric and bus

Replication engine for local SPAN/multicast

replicationLocal linecardbus for ASIC

interconnection


CEF256 Module with DFC

CEF256Module

with DFC

8Gbps FabricChannel

LCDBUSLCRBUS

ReplicationEngine Port

ASICPortASIC

PortASIC

PortASIC

MET

4xGE 4xGE 4xGE 4xGE

Example: WS-X6516-GBIC with WS-F6K-DFC

L3Engine

DFC

Layer 2/4EngineFabric

Interface

Layer 3 Engine for

FIB/Adj and NetFlow lookups

Layer 2/4 Engine for L2 and ACL/QoS

lookups


CEF720 Module

CEF720Module

CFC

Complex BComplex A

PortASIC

FabricInterface &Replication

Engine

20Gbps FabricChannel


METMET


Engine

PortASIC

PortASIC

PortASIC

12xGE 12xGE 12xGE 12xGE

DBUSRBUS

Example: WS-X6748-SFP

BusInterface

BusInterface

Combined fabric interface and

replication engine Transparent bus interface

Bus interface for control data only!!


L3/4EngineDFC3

Layer 2Engine

Layer 2Engine

CEF720Module

with DFC3

Complex BComplex A

PortASIC


Engine



METMET


Engine

PortASIC

PortASIC

PortASIC

12xGE 12xGE 12xGE 12xGE

CEF720 Module with DFC3Example: WS-X6748-SFP with WS-F6700-DFC3B

Layer3/4 Engine for

FIB/Adj, ACL, QoS and NetFlow lookups

Layer 2 Engine for L2

lookups


Distributed Forwarding

• One or more modules have local forwarding engine (DFC—Distributed Forwarding Card)

• Central engine and distributed engines perform different lookupsindependently and simultaneously

• Implementation is fully distributedAll hardware from PFC is present on the DFC

Full Layer 2, Layer 3, ACL/QoS information downloaded from Supervisor

Ingress DFC performs all lookups locally

• Deterministic, highly scalable—Not flow-based

• NOT just for local switching—destination interface irrelevant

• DFCs always require Cisco IOS software and a switch fabric


Distributed Forwarding Cards• DFCs work in conjunction with specific

supervisorDFC works with PFC2 on Supervisor 2DFC3A/3B/3BXL works with PFC3 on Supervisor 720

• PFC/DFC “major” module version must be identicalPFC/DFC “minor” module version mismatch supported in lowest common denominator modeExample: System with PFC3B and DFC3As runs in PFC3A mode

• DFC is optional daughter card for CEF256 modules• DFC3 is optional daughter card for CEF256/CEF720 modules

Several flavors and form factors available

• WS-X6816-GBIC module REQUIRES either DFC or DFC3• Local CPU for managing hardware tables• Use remote login module command to access DFC console

Commands available on DFC console for troubleshooting use, under direction from Cisco TAC/escalation


SupervisorEngine 32

PFC3

L3/4Engine

DBUSRBUS

ClassicModule A

ClassicModule BL2 Engine

PortASIC

SBlue

D

PortASIC

Red

PortASIC

Centralized Forwarding

2

4

Source

Destination

Blue VLAN

Red VLAN

Entire Packet

Packet Header

DS

PortASIC

1

3


SupervisorEngine 720

PFC3

L3/4Engine

DBUSRBUS

CEF256Module A

8Gbps

LCDBUSLCRBUS

PortASIC

PortASIC

LCRBUSLCDBUS

CEF256Module B

FabricInterface

8GbpsL2 Engine

PortASIC

FabricInterface

720Gbps SwitchFabric

SBlue

D

PortASIC

Red

Centralized Forwarding with Fabric

5Source

Destination

Blue VLAN

Red VLAN

Entire Packet

Packet Header

DS

2

3

1

4

6


DFC3 L3/4Engine

CEF720Module B

w/DFC3

PortASIC

Supervisor Engine 720

PFC3

CEF720Module A

w/DFC3

L3/4EngineDFC3

Layer 2Engine

Layer 2Engine

Fabric Interface/Replication

Engine

720Gbps SwitchFabric

20Gbps

20G

bps

S

DRed

Blue

Fabric Interface/Replication

Engine

PortASIC

Distributed Forwarding

1

23

4

5

PortASIC

PortASIC

Source

Destination

Blue VLAN

Red VLAN

Entire Packet

Packet Header

DS


Agenda





• IPv4 Forwarding



• QoS

• NetFlow


Layer 2 Forwarding


Ingress Forwarding

Engine

Layer 2 Lookups

FIB lookup

Input ACL lookup

NetFlow lookup

FIB TCAM

ACL TCAM

NetFlow Table

Yes


Layer 2 Table

Output QoS lookup

Output ACL lookup

ACL TCAMQoS TCAM

Input QoS lookup

QoS TCAM

Bridged NetFlow

NetFlow Table

No

Input ACL lookup

QoS TCAM

Transmit frame

ACL TCAMQoS TCAM

Output QoS lookup

Output ACL lookup

Input QoS lookup

ACL TCAM

Router MAC?

Frame received

Layer 2 Table



Layer 2 Forwarding

• Layer 2 forwarding based on {VLAN, MAC} pairsSame MAC can be learned in multiple VLANs

• MAC learning fully hardware basedCPU not involved in learning

• PFC and DFCs have copies of MAC tableRefreshing of entries based on “seeing” traffic—forwarding engines age entries independently

New learns on one forwarding engine communicated to other engines

• MAC table size:128K entries on PFC2 (32K effective)

64K entries on PFC3 (32K effective)


L2 flooding

No

L2 forwardingYes

Known MAC?

No

L3 forwardingYes

Update entry

No

Layer 2 Table

LearnYes

Layer 2 Table

Layer 2 Forwarding Logic

Router MAC?New MAC?

Frame received

SMAC lookup DMAC lookupLayer 2 Table Layer 2 Table


Layer 2 Forwarding Table Design

MAC Table

16 pages 4096 rows

MAC Table8 pages

16384 rows

PFC2

PFC3

16K*8=128K entries 4K*16=64K entries


Destination interface(s)

DMAC lookup

UpdateEntry

SMAC lookup

2101 | 4334.5445.6556

111 | 9000.8000.7000

444 | 6666.6666.6666

44 | 2468.ace0.2468

3999 | 9090.9090.9090

4000 | 3233.1111.3333

44 | 0100.5e01.0101

100 | 0000.1111.1111

40 | 0000.1111.2222

10 | 0000.aaaa.aaaa

30 | 0000.dddd.dddd

10 | 0000.bbbb.bbbb

20 | 0000.cccc.cccc

PFC2 Layer 2 Lookup

MAC Table8 pages

16384 rows

Hash Function

Starting Pageand Row

Compare

10 | 0000.aaaa.aaaa

HIT!

VLAN MAC Address

Lookup Key

Frame1

2

3

4

5

6


Destination interface(s)

DMAC lookup

UpdateEntry

SMAC lookup

6

MAC Table16 pages

4096 rows20 | 0000.cccc.cccc

10 | 0000.bbbb.bbbb

30 | 0000.dddd.dddd

10 | 0000.aaaa.aaaa

PFC3 Layer 2 Lookup

Compare

10 | 0000.aaaa.aaaaVLAN MAC Address

Lookup Key

Frame

Hash Function

MAC TableRow

HIT!

1

2

3

4

5


Displaying the Layer 2 Table

• Cisco IOS: show mac-address-table• Catalyst OS: show cam

6509#show mac-address-table dynamic vlan 30

Codes: * - primary entry

vlan mac address type learn qos ports

------+----------------+--------+-----+---+-----------------------* 30 0003.a088.c408 dynamic Yes -- Fa3/18

* 30 0012.d949.04d2 dynamic Yes -- Gi5/1

* 30 0003.a08a.15f3 dynamic Yes -- Fa3/24

* 30 0090.a400.1850 dynamic Yes -- Fa3/14

* 30 0003.a08a.15f9 dynamic Yes -- Fa3/25

<…>

6509#


Agenda





• IPv4 Forwarding



• QoS

• NetFlow


IPv4 Forwarding


Ingress Forwarding

Engine

IPv4 Lookups

FIB lookup

Input ACL lookup

NetFlow lookup

FIB TCAM

ACL TCAM

NetFlow Table

Yes


Layer 2 Table

Output QoS lookup

Output ACL lookup

ACL TCAMQoS TCAM

Input QoS lookup

QoS TCAM

Bridged NetFlow

NetFlow Table

No

Input ACL lookup

QoS TCAM

Transmit frame

ACL TCAMQoS TCAM

Output QoS lookup

Output ACL lookup

Input QoS lookup

ACL TCAM

Router MAC?

Frame received

Layer 2 Table



Hardware-Based CEF

• Catalyst 6500 leverages existing software Cisco Express Forwarding (CEF) model

• Supervisor 2, Supervisor 32, Supervisor 720 extend CEF to hardware

• What is CEF, in a nutshell?Boil down the routing table = FIB table

Boil down the ARP table = adjacency table

• FIB table contains IP prefixes

• Adjacency table contains next-hop information


Hardware-Based CEF (Cont.)

• Decouples control plane and data planeForwarding tables built on control plane

Tables downloaded to hardware for data plane forwarding

• Hardware CEF process:FIB lookup based on destination prefix (longest-match)

FIB “hit” returns adjacency, adjacency contains rewrite information (next-hop)

ACL, QoS, and NetFlow lookups occur in parallel and affect finalresult


FIB TCAM and Adjacency EntriesFIB:• IPv4 entries logically arranged from

most to least specific• 0/0 default entry terminates unicast

FIB entries• Overall FIB hardware shared by

IPv4 unicastIPv4 multicastIPv6 unicastIPv6 multicastMPLS

Adjacency table:• Hardware adjacency table also

shared among protocols• Actual adjacency table entries are

NOT shared

10.1.0.0172.16.0.0

…

172.20.45.110.1.1.100

…10.1.3.010.1.2.0

…

0.0.0.0

MASK (/24)

MASK (/16)

MASK (/32)

MASK (/0)

FIB TCAM

IF, MACs, MTU

IF, MACs, MTU

IF, MACs, MTU

IF, MACs, MTU

Adjacency Table


123456

78

123456

78

Adj Index

Result

IF, MACs, MTU

IF, MACs, MTU

IF, MACs, MTU

IF, MACs, MTU

Compare

FIB TCAMMasks Values

FFFFFFFF

10.1.1.210.1.1.3

10.10.0.1010.10.0.10010.10.0.3310.100.1.110.100.1.2

10.1.1.4

FFFFFF00

10.1.2.xx10.1.3.xx

10.1.1.xx10.100.1.xx10.10.0.xx

10.100.1.xx

10.10.100.xx

Lookup Key

IPv4 FIB TCAM LookupGenerate Lookup

Key

DIP10.1.1.10

Packet

/32 entries (compare all

bits)

/24 entries (mask last

octet)

10.1.1.xx10.1.1.1010.1.1.10

HIT!

Load-SharingHash

Flow Data

Adjacency Table

Offset

1

2

3

4

56


Displaying IPv4 Forwarding Summary Information

• Cisco IOS:show mls cef summary

show mls cef statistics

show mls statistics

show mls cef hardware

• Catalyst OS:show mls cef

show mls

6509-neb#show mls cef summary

Total routes: 8309

IPv4 unicast routes: 5948

IPv4 Multicast routes: 2359

MPLS routes: 0

IPv6 unicast routes: 0

IPv6 multicast routes: 0

EoM routes: 0

6509-neb#


Displaying Hardware IPv4 Prefix Entries6509-neb#show mls cef

Codes: decap - Decapsulation, + - Push Label

Index Prefix Adjacency

64 127.0.0.51/32 receive

65 127.0.0.0/32 receive

66 127.255.255.255/32 receive

67 0.0.0.0/32 receive

68 255.255.255.255/32 receive

75 10.10.1.1/32 receive

76 10.10.1.0/32 receive

77 10.10.1.255/32 receive

78 10.10.1.2/32 Gi1/1, 0030.f272.31fe

3200 224.0.0.0/24 receive

3201 10.10.1.0/24 glean

3202 10.100.0.0/24 Gi1/1, 0030.f272.31fe

3203 10.100.1.0/24 Gi1/1, 0030.f272.31fe

3204 10.100.2.0/24 Gi1/1, 0030.f272.31fe

3205 10.100.3.0/24 Gi1/1, 0030.f272.31fe

<…>

• Cisco IOS: show mls cef

• Catalyst OS: show mls entry cef ip


Displaying Detailed Hardware Entries

• Cisco IOS: show mls cef <prefix> [detail]

show mls cef adjacency [entry <entry> [detail]]

• Catalyst OS:show mls entry cef ip <prefix/mask> [adjacency]

6509-neb#show mls cef 10.100.20.0 detail

<…>

M(3222 ): E | 1 FFF 0 0 0 0 255.255.255.0

V(3222 ): 8 | 1 0 0 0 0 0 10.100.20.0 (A:98304 ,P:1,D:0,m:0 ,B:0 )

6509-neb#show mls cef adjacency entry 98304

Index: 98304 smac: 000f.2340.5dc0, dmac: 0030.f272.31fe

mtu: 1518, vlan: 1019, dindex: 0x0, l3rw_vld: 1

packets: 4203, bytes: 268992

6509-neb#


Finding the Longest-Match Prefix Entry

• Cisco IOS: show mls cef lookup <ip_address> [detail]

6509-neb#show mls cef 10.101.1.0



6509-neb#show mls cef lookup 10.101.1.0



3203 10.101.0.0/16 Gi2/12, 0007.b30a.8bfc

6509-neb#


IPv4 CEF Load Sharing• Up to 8 hardware load-sharing paths per

prefix• Use maximum-paths command in routing

protocols to control number of load-sharing paths

• IPv4 CEF load-sharing is per-IP flow• Per-packet load-balancing NOT supported• Load-sharing based on Source and

Destination IP addresses by default“Unique ID” in PFC3 prevents polarization

• Configuration option supports inclusion of L4 ports in the hashmls ip cef load-sharing full

• Unique ID not included in hash in “full” mode

10.10.0.0/16

A B

10.10.0.0/16via Rtr-Avia Rtr-B


Load-Sharing Prefix Entry Example

• show mls cef • show mls cef lookup

6509-neb#show mls cef lookup 10.100.20.1



3222 10.100.20.0/24 Gi1/1, 0030.f272.31fe

Gi1/2, 0008.7ca8.484c

Gi2/1, 000e.382d.0b90

Gi2/2, 000d.6550.a8ea

6509-neb#


Identifying the Load-Sharing Path

show mls cef exact-route

6509-neb#show mls cef exact-route 10.77.17.8 10.100.20.199

Interface: Gi1/1, Next Hop: 10.10.1.2, Vlan: 1019, Destination Mac: 0030.f272.31fe

6509-neb#show mls cef exact-route 10.44.91.111 10.100.20.199

Interface: Gi2/2, Next Hop: 10.40.1.2, Vlan: 1018, Destination Mac: 000d.6550.a8ea

6509-neb#


IPv4 Unicast RPF Check

Supervisor 2:• One RPF interface per prefix in hardware• Enabling uRPF check halves available FIB

TCAM (128K entries)

Supervisor 720/Supervisor 32:• Up to 6 RPF interfaces per prefix in hardware• Enabling does not affect available FIB entries• Two reverse-path interfaces for all prefixes• Four user-configurable “multipath interface

groups” to define additional interfaces for uRPF

10.255.0.0/16 10.20.0.0/16gig 6/3

g1/2

g2/1

g2/2

6500 Routing TablePrefix Next Hop Interface

10.255.0.0/16 10.10.1.1 gig 1/110.20.1.1 gig 1/210.30.1.1 gig 2/110.40.1.1 gig 2/2

10.20.0.0/16 10.20.1.1 gig 6/3

Gotcha: System supports only a global uRPF mode—strict or loose—last configured mode overridesGotcha: uRPF with exception ACL not recommended due to software processing

g1/1


Verifying uRPF Check Configuration

• show mls cef ip rpf [<prefix>](PFC3 only)

6509#show mls cef ip rpf

RPF global mode: strict

RPF mpath mode: punt

Index Interfaces

-------+----------------------------------------

0

1

2

3

6509#show mls cef ip rpf 192.168.1.0

RPF information for prefix 192.168.1.0

uRPF check performed in the hardware for interfaces:

Vlan776

Vlan777

uRPF check punted to software for interfaces:

uRPF check disabled for interfaces:

6509#

Global uRPF check mode

Global uRPF multipath mode

uRPF interface groups (not configured)

uRPF details for specific IP prefix


Agenda





• IPv4 Forwarding



• QoS

• NetFlow


IPv4 Multicast Forwarding


Ingress Forwarding

Engine

IPv4 Multicast Lookups

FIB lookup

Input ACL lookup

NetFlow lookup

FIB TCAM

ACL TCAM

NetFlow Table

Yes


Layer 2 Table

Output QoS lookup

Output ACL lookup

ACL TCAMQoS TCAM

Input QoS lookup

QoS TCAM

Bridged NetFlow

NetFlow Table

No

Input ACL lookup

QoS TCAM

Transmit frame

ACL TCAMQoS TCAM

Output QoS lookup

Output ACL lookup

Input QoS lookup

ACL TCAM

Router MAC?

Frame received

Layer 2 Table



IPv4 Multicast Forwarding

• Central and distributed IPv4 multicast hardware forwarding

• Distributed multicast replication with appropriate switching modules†

• PIM-SSM and PIM-SM forwarding in hardware

• BiDir-PIM forwarding in hardware‡

• Off-loads majority of forwarding tasks from RP CPU

† Supervisor 2/SFM and Supervisor 720 only, with fabric-enabled modules‡ Supervisor 32 and Supervisor 720 only


Multicast Forwarding Tables

• RP CPU derives 3 key data structures from multicast routing table

Multicast FIB—Consists of (S,G) and (*,G) entries, and RPF VLAN

Adjacency table—Contains rewrite MAC and MET index

Multicast Expansion Table (MET)—Contains output interface lists (OILs), i.e., lists of interfaces requiring replication

• RP CPU downloads tables to SP CPU

• SP CPU installs tables in the appropriate hardware

Multicast FIB and adjacency tables installed in PFC/DFC hardware

MET installed in replication engines

• SP CPU also maintains L2 table for IGMP snooping


Multicast Hardware Entries

• FIBIPv4 multicast entries arranged logically from most to least specific

• Adjacency tableDifferent format than unicast

Key piece of data is MET index

• METContains OILs for multicast routes

Memory resident on replication engines (not PFC/DFC)

MAC, MET Index

MAC, MET Index

MAC, MET Index

MAC, MET Index

…10.1.1.0, 224.0.0.0

…

172.21.4.19, 225.3.3.310.1.44.199, 240.9.8.1

…

…

*, 229.0.1.1

*, 234.0.1.1

MASK IF 224/4 Entries

10.1.1.1, 239.1.1.1

MASK (S,G) /32

MASK BiDir Entries

MASK PIM-SM (*,G) /32

FIB TCAM

MET

OIL #1OIL #2OIL #3OIL #4

Adjacency Table


123456

78

Adj IndexRPF VLAN

Result

Replication Engine(s)

Compare

FIB TCAMMasks Values

FFFFFFFF FFFFFFFF

10.1.1.10, 239.1.1.110.1.1.10, 239.1.1.1

10.1.1.10, 239.1.1.110.1.1.10, 239.1.1.110.1.1.10, 239.1.1.110.1.1.10, 239.1.1.110.1.1.10, 239.1.1.1

10.1.1.10, 239.1.1.1

Lookup Key

Multicast FIB TCAM LookupGenerate Lookup

Key

S,G10.1.1.10, 239.1.1.1

Multicast Packet

MAC, MET Index

MAC, MET Index

MAC, MET Index

MAC, MET Index

Adjacency Table

1

2

3

4

S,G compares all bits in SIP

and GIP

MET

OIL #1OIL #2OIL #3OIL #4

5

10.1.1.10, 239.1.1.1

HIT!

6


Displaying Summary Hardware Multicast Information

• Cisco IOS: show mls ip multicast summary

• show mls ip multicast statistics

• Catalyst OS: show mlsmulticast

6506#show mls ip multicast summary

21210 MMLS entries using 3394656 bytes of memory

Number of partial hardware-switched flows: 0

Number of complete hardware-switched flows: 21210

Directly connected subnet entry install is enabled

Hardware shortcuts for mvpn mroutes supported

Current mode of replication is Ingress

Auto-detection of replication mode is enabled

Consistency checker is enabled

Bidir gm-scan-interval: 10

6506#


Displaying Hardware Multicast Forwarding Entries

• Cisco IOS: show mls ip multicast

• Catalyst OS: show mls multicast entry

6506#show mls ip multicast

Multicast hardware switched flows:

(10.3.1.100, 239.1.1.100) Incoming interface: Gi3/1, Packets switched: 720396460

Hardware switched outgoing interfaces:

Gi3/2 Vlan100 Vlan150 Gi4/1 Gi4/2 Vlan200

RPF-MFD installed

(10.3.1.103, 230.100.1.1) Incoming interface: Gi3/1, Packets switched: 443201

Hardware switched outgoing interfaces:

Gi3/2 Gi4/1

RPF-MFD installed

<…>

For more details, attend:“RST-3262: Catalyst 6500 IP Multicast Architecture and Troubleshooting”


Agenda





• IPv4 Forwarding



• QoS

• NetFlow


Security and Feature ACLs


Ingress Forwarding

Engine

ACL Lookups

FIB lookup

Input ACL lookup

NetFlow lookup

FIB TCAM

ACL TCAM

NetFlow Table

Yes


Layer 2 Table

Output QoS lookup

Output ACL lookup

ACL TCAMQoS TCAM

Input QoS lookup

QoS TCAM

Bridged NetFlow

NetFlow Table

No

Input ACL lookup

QoS TCAM

Transmit frame

ACL TCAMQoS TCAM

Output QoS lookup

Output ACL lookup

Input QoS lookup

ACL TCAM

Router MAC?

Frame received

Layer 2 Table



Security ACLs• Enforce security policies based on Layer 2,

Layer 3, and Layer 4 information

• Dedicated ACL TCAM ensures security ACLs do not affect system performance

• Router ACL (RACL)—Enforced for all traffic crossing a Layer 3 interface in a specified direction

IPv4, IPX†, IPv6‡ RACLs supported

• VLAN ACLs (VACLs)—Enforced for all traffic in the VLAN

IPv4, IPX†, MAC VACLs supported

• Port ACLs (PACLs)††—Enforced for all traffic input on a Layer 2 interface

IPv4, MAC PACLs supported

† IPX ACLs in Supervisor 2 only‡ IPv6 ACLs on Supervisor 720 and Supervisor 32 only†† PACLs in Supervisor 720 and Supervisor 32 in CatOS only


Feature ACLs

• Classify traffic that requires additional or special handling

Policy-Based Routing (PBR)

Reflexive ACLs

Network Address Translation (NAT/PAT)

WCCP redirection

• Programmed in ACL TCAM to preserve performance

• Override FIB forwarding decision to allow alternative processing

• Typically paired with NetFlow table and/or Adjacency table


ACL Merge

• Sophisticated feature merge algorithm allows multiple security and feature ACLs to be applied to a single interface/VLAN

• What is merging?PFC/DFC hardware supports limited number of ACL lookups on a single packet

May need two or more ACL features on a single interface (e.g., RACL and PBR)

Merge produces ACEs that return correct result in a single lookup

• Downside: Can cause TCAM blowupACE intersection/interrelations can require lots of TCAM entries

• Two algorithms: ODM and BDD (Supervisor 2 only)• If using Supervisor 2, USE ODM! (mls aclmerge algorithm odm)

• PFC3 dual-bank TCAM architecture can avoid merge entirely

White Paper on ACL Merge Algorithms and ACL Hardware Resources:http://www.cisco.com/warp/public/cc/pd/si/casi/ca6000/tech/65acl_wp.pdf

http://www.cisco.com/warp/public/cc/pd/si/casi/ca6000/tech/65acl_wp.pdf


12345678

12345678

Permit

PermitDenyDeny

PermitDenyDeny

Permit

00000000 FFFFFFFF 00 0000 0000

Masks Values

xxxxxxxx 10.1.2.100 xx xxxx xxxx



ip access-list extended example

permit ip any host 10.1.2.100

deny ip any host 10.1.68.101


permit tcp any any eq 22

deny tcp any any eq 23

deny udp any any eq 514


permit udp any any eq 161

ACL TCAM Entry Population

00000000 00000000 FF 0000 FFFF

xxxxxxxx xxxxxxxx 06 xxxx 0016



xxxxxxxx xxxxxxxx 11 xxxx 00A1


Dest IP

Protocol

Source IP

Source PortDest Port

1=“Compare”0=“Mask”


12345678

12345678

Permit

Result

Compare

00000000 FFFFFFFF 00 0000 0000




00000000 00000000 FF 0000 FFFF




xxxxxxxx xxxxxxxx 11 xxxx 00A1


ACL TCAM Lookup

ip access-list extended example

permit ip any host 10.1.2.100




deny tcp any any eq 23

deny udp any any eq 514


permit udp any any eq 161

Generate Lookup

Key

SIP=10.1.1.10DIP=10.1.2.11Protocol=TCP (6)SPORT=33992DPORT=80

Packet

Entries matching only destination IP

Entries matching only protocol and destination port

Lookup Key

Masks Values

1

2

3

4

xxxxxxxx xxxxxxxx 06 xxxx 0050xxxxxxxx 10.1.2.11 xx xxxx xxxx10.1.1.10 | 10.1.2.11 | 06 | 84C8 | 0050

HIT!


Monitoring ACL TCAM Utilization

6509- neb#show tcam counts

Used Free Percent Used Reserved

- --- - --- - ----------- - - ------

Labels: 23 4073 0

ACL_TCAM

--------

Masks: 2902 1194 70 72

Entries: 15261 17507 46 576

QOS_TCAM

--------

Masks: 7 4089 0 18

Entries: 32 32736 0 144

LOU: 47 81 36

ANDOR: 1 15 6

ORAND: 0 16 0

ADJ: 0 2048 0

6509- neb#

• Cisco IOS: show tcam counts

• Catalyst OS: show security acl resource-usage


Verifying Hardware ACL Enforcement• show fm summary

6509-neb#show fm summary

Interface: Vlan199 is up

TCAM screening for features: ACTIVE inbound



TCAM screening for features: ACTIVE outbound



TCAM screening for features: ACTIVE outbound


TCAM screening for features: INACTIVE inbound



6509-neb#

fm = “Feature Manager”ACTIVE = ACL policy is installed in hardwareINACTIVE = ACL policy is NOT installed in hardware


Displaying Hardware ACL “Hit Counters”Cisco IOS: show tcam interface <interface> acl {in | out} ip

6509-neb#show tcam interface vlan199 acl in ip

<…>

permit udp any 10.89.210.0 0.0.0.255 (234265 matches)











deny ip any any (448691555 matches)

6509-neb#

ACL Hit Counters Supported on PFC3B/BXL Only!Global or per-ACL entry

(use [no] mls acl tcamshare-global to toggle)


Agenda





• IPv4 Forwarding



• QoS

• NetFlow


QoS


ClassifyClassify

Catalyst 6500 QoS Model

IngressIngressPolicePolice

ReceiveInterface

EgressEgressPolicePolice

InputInputQueueQueue

ScheduleSchedule

CongestionAvoidance

TransmitInterface

OutputQueue

Schedule

QoS Actions at Ingress Port ASIC

QoS Actions at PFC/DFC

QoS Actions at Egress Port ASIC

MarkMark


Ingress Forwarding

Engine

QoS Lookups

FIB lookup

Input ACL lookup

NetFlow lookup

FIB TCAM

ACL TCAM

NetFlow Table

Yes


Layer 2 Table

Output QoS lookup*

Output ACL lookup

ACL TCAMQoS TCAM

Input QoS lookup

QoS TCAM

Bridged NetFlow

NetFlow Table

No

Input ACL lookup

QoS TCAM

Transmit frame

ACL TCAMQoS TCAM

Output QoSlookup*

Output ACL lookup

Input QoS lookup

ACL TCAM

Router MAC?

Frame received

Layer 2 Table


*PFC3 only


Classification

•Selects traffic for further QoS processingMarking

Policing

•Based on—Port trust

QoS ACLs


QoS ACLs

• Support standard and extended IPv4, IPv6,† and MAC ACLs for classification

• Use QoS TCAM to classify traffic for marking and policing

• Leverage dedicated QoS TCAM32K entries/4K masks

• Share other resources (LOUs and labels) with security ACLs

† PFC3 only


QoS ACL Lookup Results

• QoS TCAM lookups behave exactly the same as ACL TCAM lookups

• But, returned result differs:Index into Aggregate table (identifies aggregate policer to use)

Index into Microflow table (identifies microflow policer to use)

Remarked DSCP/IP precedence value


Marking

• Untrusted port—Set a default QoS value

• Trusted port—Use the marking (COS, precedence, DSCP) provided by upstream device

• QoS ACLs / service-policies—Set QoS values based on standard or extended ACL match


Policing

• Enforces a policy on a port or VLAN for traffic matching classification policy

Markdown

Police (drop)

• Two types of policers:Aggregate

Microflow

• Based on a classic token bucket schemeAdd tokens to bucket at constant rate (equivalent to policed rate)

Packets are “in profile” if enough tokens exist in the bucket totransmit the packet

Packets without adequate tokens are dropped or marked down

• Note! PFC2 uses Layer 3 packet size, PFC3 uses Layer 2 frame size, when determining rate


Aggregate Policing

• Bandwidth limit applied cumulatively to all flows that match theassociated class

Example—All FTP flows in a VLAN limited in aggregate to configured rate

• Ingress policing performed on per-switchport, per-Layer 3 interface, or per-VLAN basis

PFC2 and PFC3 both support ingress policing

• Egress policing on a performed on per-Layer 3 interface or per-VLAN basis

NOT possible on a per-switchport basisPFC3 support only

• Dual-rate policers allow for combined markdown and drop policiesNormal rate and excess rate are configurablePFC3 support only


Microflow Policing

• Bandwidth limit applied separately to each individual flow that matches the associated class

Every individual FTP flow limited to configured rate

• User-based rate limiting using source-only and destination-only flow masks

All FTP from a given source IP limited to configured rate

PFC3 only

• Leverages NetFlow table

• Microflow policing performed on ingress only


Remarking Traffic with Policers

• Policing action may remark certain trafficFor example, transmit with marked-down DSCP

• Dual-rate aggregate policer can mark-down traffic exceeding the normal rate and drop traffic exceeding the excess rate

• Use markdown maps to configure marked-down DSCP valuesmls qos map policed-dscp (Cisco IOS) or set qos policed-dscp-map (CatOS)


Monitoring Service Policies (Marking and Policing)

6506#show policy- map interface vlan 100

Service- policy input: VLAN- 100

class- map: NET- 44- TCP (match- all)

Match: access- group name POL- 44- TCP

police :

100000000 bps 100000 limit 100000 extended limit

Earl in slot 6 :

2940073472 bytes

5 minute offered rate 358172704 bps

aggregate- forwarded 608631808 bytes action: transmit

exceeded 2331441664 bytes action: drop

aggregate- forward 100352000 bps exceed 384495616 bps

class- map: NET- 55 (match- all)

Match: access- group name MARK- 55

set precedence 5:

Earl in slot 6 :

2940069888 bytes

5 minute offered rate 358172616 bps

aggregate- forwarded 2940069888 bytes

6506#

• Cisco IOS: show policy-map interface*

• Catalyst OS: show qos statistics {aggregate-policer | l3stats}

PolicedClass

MarkedClass

* Shows aggregate policer stats only; use NetFlow table to monitor microflow policing


Agenda





• IPv4 Forwarding



• QoS

• NetFlow


NetFlow


Ingress Forwarding

Engine

NetFlow Lookups

FIB lookup

Input ACL lookup

NetFlow lookup

FIB TCAM

ACL TCAM

NetFlow Table

Yes


Layer 2 Table

Output QoS lookup

Output ACL lookup

ACL TCAMQoS TCAM

Input QoS lookup

QoS TCAM

Bridged NetFlow

NetFlow Table

No

Input ACL lookup

QoS TCAM

Transmit frame

ACL TCAMQoS TCAM

Output QoS lookup

Output ACL lookup

Input QoS lookup

ACL TCAM

Router MAC?

Frame received

Layer 2 Table



IPv4 NetFlow• Tracks statistics for traffic flows through the system• Entries created in NetFlow table when new flows

startFlow mask determines format of entries

• Entries removed when flows expireTimer and session based expiration

• Full collection by default when NetFlow enabledAlso support time- and packet-based NetFlow sampling

• Flow statistics can be exported using NetFlow Data Export (NDE)

Supported export formats include NetFlow v5 and v7NetFlow v9 export format supported in Supervisor 720 and Supervisor 32


Displaying NetFlow Statistics Entries

• Cisco IOS: show mls netflow ip

• Catalyst OS: show mls statistics entry

6506#show mls netflow ip

Displaying Netflow entries in Supervisor Earl

DstIP SrcIP Prot:SrcPort:DstPort Src i/f :AdjPtr

---------------------------------------------------------------------------

Pkts Bytes Age LastSeen Attributes

---------------------------------------------------

10.102.130.213 10.214.39.79 tcp :46528 :www Vl39 :0x0

7 3766 17 15:47:37 L3 - Dynamic

10.230.215.148 10.155.22.221 tcp :51813 :45912 Vl22 :0x0

25 21329 47 15:47:39 L3 - Dynamic

10.97.36.200 10.17.64.177 tcp :65211 :www Vl144 :0x0

9 7664 17 15:47:38 L3 - Dynamic

10.90.33.185 10.46.13.211 tcp :27077 :60425 Vl13 :0x0

2569654 1269409076 17 15:47:38 L3 - Dynamic

<…>

Which fields are populated depends on

the configured flow mask


NetFlow Table Utilization• PFC2

NetFlow table contains 128K entriesHash ~25% efficient (32K entries)Probability of collision increases after 32K entries

• PFC3NetFlow table size varies

• PFC3A/B—128K entries• PFC3BXL—256K entries

Hash ~50–90% efficient (64/96/230K entries for PFC3A/B/BXL)Probability of collision increases after 64K/96K/230K entriesAlias CAM handles hash collision cases


UpdateStatistics

10.1.1.2 | 10.1.1.1 | 6 | 80 | 1030

10.1.1.1 | 10.1.1.2 | 6 | 1030 | 80

10.1.1.2 | 10.1.1.1 | 6 | 80 | 1030

10.1.1.1 | 10.1.1.2 | 6 | 1030 | 80

10.1.1.2 | 10.1.1.1 | 6 | 80 | 1030

10.1.1.1 | 10.1.1.2 | 6 | 1030 | 80

10.1.1.2 | 10.1.1.1 | 6 | 80 | 1030

10.1.1.1 | 10.1.1.2 | 6 | 1030 | 80

10.1.1.2 | 10.1.1.1 | 6 | 80 | 1030

10.1.1.1 | 10.1.1.2 | 6 | 1030 | 80

PFC2 NetFlow Table Architecture

NetFlow Table8 pages

16K rows

10.1.1.2 | 10.1.1.1 | 6 | 80 | 1030

10.1.1.1 | 10.1.1.2 | 6 | 1030 | 80

10.1.1.2 | 10.1.1.1 | 6 | 22 | 3245

172.16.1.1 | 172.16.2.2 | 17 | 2334 | 23

10.1.1.1 | 239.1.1.1 | 17 | 5000 | 5000

192.168.1.1 | 10.1.1.2 | 1 | 0 | 0

10.99.1.1 | 10.99.100.1 | 6 | 4444 | 25

10.99.100.1 | 10.4.5.6 | 6 | 25 | 1080

10.10.10.1 | 10.20.1.1 | 6 | 2334 | 80

172.16.8.2 | 192.168.1.2 | 6 | 1025 | 80

10.10.20.1 | 10.20.2.2 | 6 | 1044 | 80

10.4.4.4 | 172.16.8.8 | 17 | 1025 | 514

Hash Function

Starting Pageand Row

10.10.20.1 | 10.20.2.2 | 6 | 1044 | 80

HIT!

SIP DIP Proto SPort DPort

Flow Key

Packet

Compare

1

2

3

4

5

6

106© 2006 Cisco Systems, Inc. All rights reserved.RST-3465 12523_04_2006_c1 Cisco PublicAlias CAM

NetFlowTable Index

Result

128K/256Kentries

128K/256Krows

StatisticsMask

KeyKey

KeyKeyKeyKeyKey

Key

Mask

KeyKeyKey

Flow DataFlow Data

Flow DataFlow DataFlow DataFlow DataFlow Data

Flow Data

Flow DataFlow Data

Flow DataFlow Data

Key

PFC3 NetFlow Lookups

Netflow TCAM Netflow Table

Compare

Flow Key

Hash KeyHash Key

HIT!

HIT!

128 entries

Compare

Hash Function

Hash Key

Flow Key

Packet1

2

3

4

5

6

7


Monitoring NetFlow Table Usage• Cisco IOS: show mls netflow table-contention

• Catalyst OS: show mls debug

6506#show mls netflow table- contention detailed

Earl in Module 6

Detailed Netflow CAM (TCAM and ICAM) Utilization

================================================

TCAM Utilization : 100%

ICAM Utilization : 82%

Netflow TCAM count : 131072

Netflow ICAM count : 105

Netflow Creation Failures : 3432605

Netflow CAM aliases : 8

6506#show mls netflow table- contention aggregate

Earl in Module 6

Aggregate Netflow CAM Contention Information

=============================================

Netflow Creation Failures : 222917949

Netflow Hash Aliases : 834

6506#

Current utilization

Clear on read

Cumulative


NetFlow Aging

• Process of removing stale NetFlow entries

• Types of agingNormal—Fixed idle time for flows

Fast—Threshold-based aging of flows

Long—Maximum lifetime for flows

Session-based—Based on TCP FIN/RST flags

• Default timers are conservativeTuning is recommended!

Start with more aggressive normal aging timer—Reduce until no creation failures seen or CPU is at threshold

Enable fast aging to remove short-lived flows—Adjust until creation failures cease or CPU is at threshold


Changing and Viewing the NetFlow Aging Configuration

• Cisco IOS: mls aging {normal | fast | long}

show mls netflow aging

• Catalyst OS: set mls agingtime [fast | long-duration]

show mls

6506#show mls netflow aging

enable timeout packet threshold

------ ------- ----------------

normal aging true 300 N/A

fast aging false 32 100

long aging true 1920 N/A

6506#


Conclusion

• You should now have a thorough understanding of the Catalyst 6500 switching architecture, packet flow, and key forwarding engine functions…

ANY QUESTIONS?


Related Networkers Sessions

• RST-3262: IP Multicast Architecture and Troubleshooting for the Cisco Catalyst 6500 Series

• RST-3143: Troubleshooting Catalyst 6500 Series Switches

• RST-2031: Multilayer Campus Architectures and Design Principles

• RST-3466: Cisco IOS Software Modularity—Architecture and Deployment

• TECRST-3101: Troubleshooting Cisco Catalyst Switches

• TECRST-2001: Enterprise High Availability

• BoF-06: Enterprise Switching


Q and A


Recommended Reading

• Continue your Cisco Networkers learning experience with further reading from Cisco Press

• Check the Recommended Reading flyer for suggested books

Available Onsite at the Cisco Company Store


Complete Your Online Session Evaluation• Win fabulous prizes; Give us your feedback

• Receive ten Passport Points for each session evaluation you complete

• Go to the Internet stations located throughout the Convention Center to complete your session evaluation

• Drawings will be held in theWorld of Solutions

Tuesday, June 20 at 12:15 p.m.

Wednesday, June 21 at 12:15 p.m.

Thursday, June 22 at 12:15 p.m. and 2:00 p.m.

cisco catalyst 6500 switch architecture

Documents