advanced cisco catalyst 6500/6800 series...
TRANSCRIPT
Advanced Cisco Catalyst 6500/6800 Series Troubleshooting
BRKCRS-3148
Ravi U. Krishna ([email protected])
Customer Support Engineer, Cisco Services
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Goal of the Session
Educate you commonly used commands and methods to troubleshoot Cisco Catalyst 6500 Virtual Switch (VSS) based on Supervisor 2T.
In the latter part of this session we shall introduce Catalyst Instant Access Solution and teach how to troubleshoot this device through some common troubleshooting commands.
For more in depth understanding and detailed troubleshooting of Cisco Catalyst 6500 VSS Switch, its recommended to have attended BRKCRS-3143 session prior to this.
3
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Agenda
• Introduction
• Troubleshooting Virtual Switching System
• Introduction to Instant Access(IA) Solution
• Troubleshooting 6800ia Solution
• Conclusion
• Q and A
4
Introduction
5
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
What do I care about as an Network Admin..
• Is my VSS working fine ? Health check
• How can I check the path of a packet that is ingressing into my VSS
• What happens when one of my VSS supervisor engine crashes and how quickly can I recover from this downtime
6
Troubleshooting Virtual Switch System
7
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Troubleshooting Virtual Switching System
• Architecture of 2T VSS
• Control plane health check
• L2/L3 Packet flow troubleshooting
• Dual Active detection
• VSS Supervisor Engine failover and recovery
Agenda
8
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Virtual Switching System (VSS) Architecture
9
VSS Physical View VSS Logical View
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Virtual Switching System (VSS) Architecture
Virtual Switch
Domain
Virtual Switch Link
Active Control Plane
Active Data Plane
Hot Standby Control Plane
Active Data Plane
Virtual Switch Primary Virtual Switch Secondary
10
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Terminologies
• PFC: Policy Feature Card
• DFC: Distributed Forwarding Card
• FE: Forwarding Engine
• TCAM: Ternary Content Addressable Memory
• ACE: Access Control Entry
• ADJ: Adjacency information
• LIF: Logical Interface
• MSFC: Multilayer Switch Fabric Card
• FIRE: Fabric Interface and Replication Engine
• FIB: Forwarding Information Base
11
11
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Architecture 2T VSS Internal Architecture
DBUS
PFC4
Traces # 1 to 26 RBUS EOBC
MSFC 5
Flash DRAM
CL1 TCAM
NET FLOW TCAM
L3/4 Engine
L2 Engine
ACE Counter
CL2 TCAM
40
Gbps
Fabric Interface
AND
Replication
Engine
Switch Fabric
26 x 40G Traces
2 x 1
Gbps
Central
Management
Processor
CPU
Port ASIC
FIB TCAM
ADJ TCAM
LIF Table
LIF Stats
RPF Table
LIF MAP
MSFC5 Complex contains single (combined SP/RP) CMP CPU FIRE ASIC that interfaces with fabric and bus and for multicast/SPAN replication
1GE / 10GE
Uplinks
Switch Fabric
LIF Table and Statistics
TCAMs for FIB/ADJ Layer 3/4 Forwarding Engine
Layer 2 Forwarding Engine
ACE Hardware Counters
Port ASIC to host external interface
12
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Troubleshooting Virtual Switching System
• Architecture of 2T VSS
• Control plane health check
• L2/L3 Packet flow troubleshooting
• Dual Active detection
• VSS Supervisor Engine failover and recovery
Agenda
13
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Control Plane health check VSS Specific Protocols (VSLP)
• Link Management Protocol (LMP)
– Used to track and reject Unidirectional Links, Exchange Chassis ID and other information between the two switches
• Role Resolution Protocol (RRP)
– Used to determine the compatible hardware and software versions to form the VSL. Also used to determine which Switch becomes Active and Hot Standby from control plane perspective
After the roles have been resolved through RRP, a Configuration Consistency Check is performed across VSL switches to ensure proper VSL operation. A failure in this check will result in reloading the hot-standby to operate in RPR mode.
14
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Control Plane health check VSS Configuration and operation check
sup2T# show switch virtual
Switch mode : Virtual Switch
Virtual switch domain number : 100
Local switch number : 1
Local switch operational role: Virtual Switch Active
Peer switch number : 2
Peer switch operational role : Virtual Switch Standby
sup2T#
Unique number in your network between 1 and 255
Switch numbers to identify primary and secondary
Switch roles (active or standby)
15
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Control Plane health check VSS Configuration and operation check
sup2T# show switch virtual link
VSL Status : UP
VSL Uptime : 22 hours, 43 minutes
VSL SCP Ping : Pass
VSL ICC Ping : Pass
VSL Control Link : Te1/2/4
VSL Encryption : Configured Mode - Off, Operational Mode – Off
sup2T#
Control Link carries EOBC and IBC control
Messages (SCP and ICC/IPC)
16
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Control Plane health check VSLP Ping Check
Check if we can ping the two switches using VSLP protocol
sup2T# ping vslp output interface tenGigabitEthernet 1/2/4
Type escape sequence to abort.
Sending 5, 100-byte VSLP ping to peer-sup via output port 1/2/4, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 12/12/12 ms
sup2T#
Use – ‘show switch virtual link port-channel’
to determine the physical interfaces which are part of VSL
17
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Control Plane health check VSS Configuration and operation check sup2T# show switch virtual slot-map Virtual Slot to Remote Switch/Physical Slot Mapping Table: Virtual Remote Physical Module Slot No Switch No Slot No Uptime ---------+-----------+----------+---------- 17 1 1 - 18 1 2 3d18h 19 1 3 3d18h 20 1 4 - 21 1 5 - <snip> 30 1 14 - 31 1 15 - 32 1 16 - 33 2 1 22:41:23 34 2 2 - 35 2 3 - 36 2 4 22:37:31 37 2 5 - <snip>
Gives mapping between the Switch number, Physical Slot number to its Virtual Slot number. It also shows how long the module is online in this configuration
Module in Switch 1 Slot 2 is up since 3 days and 18 hours
Module in Switch 2 Slot 1 is up since 22 hours, 41 mins and 23 secs
18
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Control Plane health check
show switch virtual dual-active summary
show switch virtual link detail
show switch virtual redundancy
show switch virtual role detail
show switch virtual link port-channel
show vslp [lmp|rrp] [counters|fsm|summary]
debug vslp [lmp|rrp] packet*
debug switch virtual vsl-drop packets*
Other helpful commands Displays Dual Active capability
Displays info on LMP viz., counters, neighbors, hello timer, FSM info, etc
Displays peer switch redundancy info
Displays switch numbers and redundancy role
VSS Port-Channel info
Debug LMP/RRP hello’s between VSS
Debug dropped packets seen on VSL links
Displays RRP/LMP information
19
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Troubleshooting Virtual Switching System
• Architecture of 2T VSS
• Control plane health check
• L2/L3 Packet flow troubleshooting
• Dual Active Detection
• VSS Supervisor Engine failover and recovery
Agenda
20
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Layer2 Packet flow troubleshooting Topology
DUT is the Device Under Test we are troubleshooting
DUT is a VSS Chassis with Supervisor 2T
Four TenGigabitEthernet L2 Etherchannel (Switch A DUT)
Four TenGigabitEthernet L2 Etherchannel (DUT Switch B)
SwitchA and SwitchB are 6500 standalone switch with Supervisor 2T
Po11
Ten1/4 Ten 1/1/1
Ten 1/1/2 Ten1/8
Ten 2/1/1 Ten1/5
Ten 2/1/2 Ten1/7
Po11
Switch A DUT Switch B
Po12
Ten 1/1/5 Ten1/3
Ten1/4 Ten 1/1/6
Ten1/7 Ten 2/1/5
Ten1/8 Ten 2/1/6
Po12
21
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
L2 Unicast Traffic
sup2T# show ip arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.10.1 - b414.8961.3780 ARPA Vlan10
Internet 192.168.10.2 31 0006.5bbc.81a2 ARPA Vlan10
Internet 192.168.10.3 32 0006.5bbc.7acb ARPA Vlan10
Traffic Configuration
22
Po11 Ten1/4 Ten 1/1/1
Ten 1/1/2 Ten1/8
Ten 2/1/1 Ten1/5
Ten 2/1/2 Ten1/7
Po11
Switch A sup2T Switch B
Po12 Ten 1/1/5 Ten1/3
Ten1/4 Ten 1/1/6
Ten1/7 Ten 2/1/5
Ten1/8 Ten 2/1/6
Po12
PC 1
192.168.10.2 PC 2
192.168.10.3
Vlan 10
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
L2 Unicast Traffic
sup2T# show mac address-table address 0006.5bbc.7acb
Legend: * - primary entry
age - seconds since last seen
<snip>
Displaying entries from Switch 1 DFC linecard [1]:
vlan mac address type learn age ports
----+----+---------------+-------+-----+---------+--------
* 10 0006.5bbc.7acb dynamic Yes 0 Po12
Displaying entries from Switch 2 DFC linecard [1]:
vlan mac address type learn age ports
----+----+---------------+-------+-----+----------+-------
10 0006.5bbc.7acb dynamic Yes 110 Po12
Where are the MAC Addresses Learned? sup2T# show mac address-table address 0006.5bbc.81a2
Legend: * - primary entry
age - seconds since last seen
<snip>
Displaying entries from Switch 1 DFC linecard [1]:
vlan mac address type learn age ports
----+----+---------------+-------+-----+----------+-----
10 0006.5bbc.81a2 dynamic Yes 5 Po11
Displaying entries from Switch 2 DFC linecard [1]:
vlan mac address type learn age ports
----+----+---------------+-------+-----+----------+-----
* 10 0006.5bbc.81a2 dynamic Yes 90 Po11
23
Host PC 1
Host PC 2
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
L2 Unicast Traffic Which Link in the EtherChannel Is Being Used?
24
SwitchA# show etherchannel load-balance module 1
EtherChannel Load-Balancing Configuration:
src-dst-ip vlan included
mpls label-ip
EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
IPv4: Source XOR Destination IP address
IPv6: Source XOR Destination IP address
MPLS: Label or IP
SwitchA# show etherhannel load-balance interface po11 ip 192.168.10.2 192.168.10.3
Computed RBH: 0x3
Would select Te1/8 of Po11
Mode is “src-dst-ip”. Only use src and dest IP as argument.
Link selected is Ten1/8 in Po11 of SwitchA for traffic sent to 192.168.10.3
Check load balancing configuration
Use ingress Module number in command in case per-module load-balancing is configured (SXH images and later)
Po11
Ten1/4 Ten 1/1/1
Ten 1/1/2 Ten1/8
Ten 2/1/1 Ten1/5
Ten 2/1/2 Ten1/7
Po11
Switch A DUT Switch B
Po12
Ten 1/1/5 Ten1/3
Ten1/4 Ten 1/1/6
Ten1/7 Ten 2/1/5
Ten1/8 Ten 2/1/6
Po12
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
L2 Unicast Traffic Network Path Verification: Result
Each packet flow can use different links in the bundles !
Po11
Ten1/4 Ten1/1/1
Ten1/1/2 Ten1/8
Ten2/1/1 Ten1/5
Ten2/1/2 Ten1/7
Po11
Switch A DUT Switch B
Po12 Ten1/1/5 Ten1/3
Ten1/4 Ten1/1/6
Ten1/7 Ten2/1/5
Ten1/8 Ten2/1/6
Po12
Po11
Ten1/4 Ten1/1/1
Ten1/1/2 Ten1/8
Ten2/1/1 Ten1/5
Ten2/1/2 Ten1/7
Po11
Switch A DUT Switch B
Po12 Ten1/1/5 Ten1/3
Ten1/4 Ten1/1/6
Ten1/7 Ten2/1/5
Ten1/8 Ten2/1/6
Po12
25
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Layer3 Packet flow troubleshooting Topology
26
DUT is the Device Under Test we are troubleshooting (6500 VSS with Sup2T)
R1 and R2 are two 6500 standalone switches with supervisor 720 in them.
Four TenGigabitEthernet L2 Etherchannel Trunk (R1 DUT)
- Vlan 10, 20, 30 and 40 are assigned with 192.168.10.0 /24,
192.168.20.0 /24, 192.168.30.0 /24 and 192.168.40.0 /24 subnets respectively.
Four L3 Links (DUT R2)
- Four links are assigned with 172.16.10.0 /24, 172.16.20.0 /24,
172.16.30.0 /24 and 172.16.40.0 /24 subnets respectively.
Po11 Ten1/4 Ten 1/1/1
Ten 1/1/2 Ten1/8
Ten 2/1/1 Ten1/5
Ten 2/1/2 Ten1/7
Po11
R1 DUT R2
Ten1/3
Ten1/4
Ten1/7
Ten1/8
Ten 1/1/5
Ten 1/1/6
Ten 2/1/5
Ten 2/1/6 Host 1
100.100.100.1
Host 2
200.200.200.1
VLANS 10,20,30 and 40 L3 Links
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
L3 Unicast Traffic Different Switching Paths for L3 Traffic in Catalyst 6500
27
27
Process Switching Path
Software-based CEF Switching Path
Hardware-based CEF switching Path
DUT
Gives logical representation of different switching paths in Catalyst 6500.
Host1 Host2
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Layer3 Packet flow troubleshooting Host 1 Host 2: Which L3 Next Hop / L2 Link from R1?
28
SW R1# show ip route 200.200.200.1
Routing entry for 200.200.200.1/32
Known via "ospf 100", distance 110, metric 3, type intra area
Last update from 192.168.40.1 on Vlan40, 00:10:12 ago
Routing Descriptor Blocks:
192.168.40.1, from 192.168.0.2, 00:10:12 ago, via Vlan40
Route metric is 3, traffic share count is 1
192.168.30.1, from 192.168.0.2, 00:10:12 ago, via Vlan30
Route metric is 3, traffic share count is 1
* 192.168.20.1, from 192.168.0.2, 00:10:12 ago, via Vlan20
Route metric is 3, traffic share count is 1
192.168.10.1, from 192.168.0.2, 00:10:12 ago, via Vlan10
Route metric is 3, traffic share count is 1
R1# show ip cef exact-route 100.100.100.1 200.200.200.1
100.100.100.1 -> 200.200.200.1 => IP adj out of Vlan40, addr 192.168.40.1
R1# show mls cef exact-route 100.100.100.1 0 200.200.200.1 0
Interface: Vl10, Next Hop: 192.168.20.1, Vlan: 10, Destination Mac:
b414.8961.3780
R1# show etherchannel load-bal int port-ch 11 ip 100.100.100.1 200.200.200.1
Computed RBH: 0x7
Would select Te1/8 of Po11
Next hop used for HW based CEF
(HW forwarding path). Note: “0” is used for both src and dest L4 port numbers as test flow was ICMP echo
Check which link between R1 and DUT is chosen.
HW
Equal Cost Routes to the destination prefix
Next hop used for SW based CEF (SW forwarding data path)
Note: R1 is a Cat6500 with Sup720.
HW HW
HW HW
HW 28
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Layer3 Packet flow troubleshooting Host 1 Host 2: Which L3 Next Hop / L2 Link from DUT?
Next hop used for HW based CEF
(HW forwarding path). Note: “0” is used for both src and dest L4 port numbers as test flow was ICMP echo
29
sup2T# show ip route 200.200.200.1 Routing entry for 200.200.200.1/32
Known via "ospf 100", distance 110, metric 2, type intra area
Last update from 172.16.20.2 on TenGigabitEthernet1/1/6, 00:36:01 ago
Routing Descriptor Blocks:
172.16.40.2, from 192.168.0.2, 00:36:01 ago, via TenGigabitEthernet2/1/6
Route metric is 2, traffic share count is 1
172.16.30.2, from 192.168.0.2, 00:36:01 ago, via TenGigabitEthernet2/1/5
Route metric is 2, traffic share count is 1
172.16.20.2, from 192.168.0.2, 00:36:01 ago, via TenGigabitEthernet1/1/6
Route metric is 2, traffic share count is 1
* 172.16.10.2, from 192.168.0.2, 00:36:01 ago, via TenGigabitEthernet1/1/5
Route metric is 2, traffic share count is 1
sup2T# show ip cef exact-route 100.100.100.1 200.200.200.1 100.100.100.1 -> 200.200.200.1 => IP adj out of TenGigabitEthernet1/1/6, addr 172.16.20.2
sup2T# show platform hardware cef exact-route 100.100.100.1 0 200.200.200.1 0 Interface: Te2/1/6, Next Hop: 172.16.40.2, ifnum: 0x12, Destination Mac: f866.f2d2.fa80
LIF: 0x20004013
SW
HW
Next hop used for SW based CEF (SW forwarding data path)
Equal Cost Routes to the destination prefix
29
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Troubleshooting Virtual Switching System
• Architecture of 2T VSS
• Control plane health check
• L2/L3 Packet flow troubleshooting
• Dual Active Detection
• VSS Supervisor Engine failover and recovery
Agenda
30
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Dual Active Detection Possibility of VSL breaking
• In a VSS domain, one Switch is elected VSS Active and the other is elected VSS Standby after RRP negotiation.
• Since VSS is configured on a PortChannel interface with more than one physical interface bundled, the possibility of PortChannel going down is remote.
• However it’s a possibility… Such a situation is called dual-active scenario. VSS must detect this dual-active condition and take recovery action.
• There are three methods to detect such a condtion
– Using enhanced PAgP (ePAgP)
– Using Dual-Active Fast Hello Packets
– Using Instant Access
VSL
Active Standby
Active Active
All these are by default enabled. However we need to configure the physical interface(s) and Port-Channels to be part of these individual methods
31
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Dual Active Detection Dual Active Detection Fast Hellos Troubleshooting
sup2T(config)# interface range Gig1/7/1, Gig2/7/1
sup2T(config-if-range)# dual-active fast-hello
%VSDA-SW1_SP-5-LINK_UP: Interface Gi1/7/1 is now dual-active detection capable
%VSDA-SW2_SPSTBY-5-LINK_UP: Interface Gi2/7/1 is now dual-active detection capable
sup2T# show switch virtual dual-active fast-hello Fast-hello dual-active detection enabled: Yes Fast-hello dual-active interfaces: Port Local State Peer Port Remote State -------------------------------------------------
Gi1/7/1 Link up Gi2/7/1 Link up
VSL
Gi1/7/1 Gi2/7/1
Fast Hello
Dual Active Trusted
ePAgP MEC
SW1 SW2
Fast Hello configuration requires an extra physical connection between both virtual switch nodes
Po101
All dual active detection protocols and methods can be implemented in parallel
R1
32
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Dual Active Detection
sup2T(config)# interface Port-Channel 101
sup2T(config)# shutdown
sup2T(config)# switch virtual domain 100
sup2T(config)# dual-active detection pagp trust channel-group 101
sup2T(config)# interface Port-Channel 101
sup2T(config)# no shutdown
Dual Active Detection ePAgP Troubleshooting
sup2T# show switch virtual dual-active pagp PAgP dual-active detection enabled: Yes
PAgP dual-active version: 1.1
Channel group 101 dual-active detect capability
Dual-Active trusted group: Yes
Dual-Active Partner Partner Partner
Port Detective Capable Name Port Version
Gi1/2/3 Yes R1 Gi2/4 1.1
Gi1/6/2 Yes R1 Gi5/2 1.1
Gi2/9/36 Yes R1 Gi4/16 1.1
Ensure neighbor runs a SW version that supports ePAgP
Choose interfaces on each Switch and different modules for redundancy
Prior to implementing trust settings, network administrators should plan for a downtime to provision ePAgP based dual active configuration
Enabling or disabling dual-active trusted mode on Layer 2/Layer 3 MEC requires MEC to be in an administrative shutdown state.
33
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Dual Active Detection
• Managing the VSS system during a dual-active condition becomes challenging when two individual systems in the same network tier contain a common network configuration.
• To recover from this condition the Old-Active Virtual Switch disables _all_ physical and logical interfaces.
• To minimize network instability, it is highly recommended to exclude only network management ports from both virtual-switch chassis.
Dual Active Recovery sup2T(config)# switch virtual domain 100
sup2T(config-vs-domain)# dual-active exclude interface Gig 1/5/2
sup2T(config-vs-domain)# dual-active exclude interface Gig 2/5/2
sup2T# show switch virtual dual-active summary
Pagp dual-active detection enabled: Yes
Bfd dual-active detection enabled: Yes
Interfaces excluded from shutdown in recovery mode:
Gi1/5/2 , Gi2/5/2
In dual-active recovery mode: No
Do NOT change the configuration when the Old Active goes to Recovery Mode.
Configuration must be saved MANUALLY and reloaded to bring the switch back up into the VSS
34
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Troubleshooting Virtual Switching System
• Architecture of 2T VSS
• Control plane health check
• L2/L3 Packet flow troubleshooting
• Dual Active detection
• VSS Supervisor Engine failover and recovery
Agenda
35
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
VSS Supervisor Engine failover and recovery
• Standby Supervisor Crash
– Standby Switch and Chassis goes for a reload
– All modules in Standby Chassis are power-cycled
– Recovers by joining back the Active in SSO mode
• Active Supervisor Crash
– Standby Switch(SW2) takes over as New Active
– SW2 now handles all Control Plane traffic
– SW1 chassis reboots as new Standby and joins the VSS pair in SSO mode
Single supervisor in each chassis
VSL
SW1(Active) SW2(Standby)
VSL
SW1(Active) SW2(Standby) SW1(Standby) SW2(Active)
36
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
VSS Supervisor Engine failover and recovery QuadSup in SSO – sup2T running 15.1(1)SY1
ICA
ICS
ICA
ICS
VSL Link
Fast Hello
SW1 SW2
SW2 ICA / SSO Standby
SW2 ICS
SW1 ICA / SSO Active
SW1 ICS
What Happens
when SSO Active
Crashes ?
Which is the
switchover
target ?
ICA : In-Chassis Active
ICS : In-Chassis Standby
(Standby HOT)
37
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
VSS Supervisor Engine failover and recovery QuadSup in SSO – sup2T running 15.1(1)SY1
ICS
ICA
ICA
ICS
VSL Link
Fast Hello
SW1 SW2
SW2 ICA / SSO Active
SW2 ICS SW1 ICA / SSO Standby
SW1 ICS
ICA : In-Chassis Active
ICS : In-Chassis Standby
(RPR Warm /
ChassisSSO)
38
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
VSS Supervisor Engine failover and recovery
sup2T-qsup# show switch virtual redundancy | inc Proc|Software state
Switch 1 Slot 1 Processor Information :
Current Software state = ACTIVE
Switch 1 Slot 2 Processor Information :
Current Software state = STANDBY HOT (CHASSIS)
Switch 2 Slot 1 Processor Information :
Current Software state = STANDBY HOT (switchover target)
Switch 2 Slot 2 Processor Information :
Current Software state = STANDBY HOT (CHASSIS)
Sup2T-qsup#
QuadSup in SSO – sup2T running 15.1(1)SY1
39
Switchover Target
(SSO Standby)
Switch 1 Slot 1 is the
Current SSO Active
InChassis Standby
InChassis Standby
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
VSS Supervisor Engine failover and recovery
sup2T-qsup# show module switch all | inc Switch|---|Supervisor
Switch Number: 1 Role: Virtual Switch Active
--- ----- -------------------------------------- ------------------ -----------
1 5 Supervisor Engine 2T 10GE w/ CTS (Acti VS-SUP2T-10G SAL1746G471
2 5 Supervisor Engine 2T 10GE w/ CTS (CSSO VS-SUP2T-10G SAL1731APWE
--- ---------------------------------- ------ ------------ ------------ -------
Switch Number: 2 Role: Virtual Switch Standby
--- ----- -------------------------------------- ------------------ -----------
1 5 Supervisor Engine 2T 10GE w/ CTS (Hot) VS-SUP2T-10G SAL1746G476
2 5 Supervisor Engine 2T 10GE w/ CTS (CSSO VS-SUP2T-10G SAL1731APX3
--- ---------------------------------- ------ ------------ ------------ -------
Sup2T-qsup#
QuadSup in SSO – sup2T running 15.1(1)SY1
40
Hot : Switchover target
Acti : Current Active
CSSO: InChassis Standby in RPR Warm/Chassis SSO
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
VSS Supervisor Engine failover and recovery
show redundancy switchover
show redundancy state domain in-chassis
show switch virtual redundancy
show switch virtual role detail
show switch virtual redundancy mismatch
show switch virtual slot-map
Other helpful commands Displays Uptime since last switchover
Displays Inchassis redundancy information
Displays overall redundancy state along with image name, uptime etc
Displays Local and remote switch nos
Config-sync mismatch details
Gives mapping between the Switch number, Physical Slot number and its Virtual Slot number. Uptime gives us information on which slot experienced switchover
41
Introduction to Instant Access(IA) Solution
43
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
IT Spends Most of Their Time in Repetitive Operational Actions for Access Switches
44
INTRODUCING 28% Monitoring,
troubleshooting
19%
Security
configurations
18%
Initial install,
configs, testing
14% Upgrading equipment
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Catalyst Instant Access Evolution
45
STANDALONE
ACCESS
SWITCH
LACP /
PAGP
ACCESS
SWITCH
VSS
SiSi SiSi
ACCESS
SWITCH ACCESS
SWITCH
LACP /
PAGP
VSL
INSTANT ACCESS
INSTANT
ACCESS
CLIENT
INSTANT
ACCESS
CLIENT
VSL
SiSi SiSi
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Traditional Campus with stacking at access
46
SiSi SiSi
SiSi SiSi
Building 1 Building 2 Building 3 Building 4
Core
SiSi SiSi SiSi SiSi SiSi SiSi
34 Total Devices Management (image and configuration)
48 Access Trunks/Port-Channels
4032 User Ports
46
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
VSS Campus with Stacking
47
Building 1 Building 2 Building 3 Building 4
Core
29 Total Devices for Image and Configuration Management
48 Access Trunks/Port-Channels
4032 User Ports
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Catalyst Instant Access
48
Core
5 Total Devices for Image and Configuration Management
Automated Trunk Configuration
4032 User Ports
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Catalyst Instant Access
49
Core
NO Routing Protocols or Spanning-Tree configuration
between Access and Distribution
NO Trunks to Configure from Access to Distribution
NO Configuration or Image Management at Access
Troubleshooting 6800ia Solution
50
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Terminologies
• FEX – Fabric Extender
• IA Parent – Instant Access Parent / Controller Switch
• IA Client – Instant Access Client / Cat6k Remote Line Card
• SDP – Switch Discovery Protocol
• SRP – Switch Role Protocol
• SCP – Switch Configuration Protocol
• RSL – Remote Satellite Link (fabric link interconnecting IA Parent with IA Client)
• VIF – Virtual Interface (logical representation of FEX physical ports)
• RPF – Route Path Forwarding
• VNTAG – Virtual Native Tagging
51
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Troubleshooting 6800ia solution Instant Access Components
IA Parent
52
FEX-Fabric IA Client
IA Parent
Supervisor 2T
X6904 -16x10G
Supervisor 2T
X6904 -16x10G
Virtual Switching System (VSS)
52
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Troubleshooting 6800ia solution Instant Access Components
IA Parent
53
Supervisor 2T
X6904 -16x10G
Supervisor 2T
X6904 -16x10G
Catalyst 6880-X
Catalyst 6807-XL Catalyst 6500E
FEX-Fabric IA Client
53
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Troubleshooting 6800ia solution
• Configuring Instant Access (FEX) and Discovery
– Topology
– Configuration CLI’s
– Troubleshooting CLI’s when FEX does not come online
• Unicast Packet Forwarding through FEX’es
– Topology
– Layer 2 unicast traffic CLI’s
– Layer 3 unicast traffic CLI’s
• Multicast Packet Forwarding through FEX
– Topology
– IGMP / MCAST forwarding CLI’s
Agenda
54
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Configuring Instant Access (FEX) and Discovery FEX Layer2 Topology
55
• VSS is Supervisor 2T based system operating in Virtual
Switching mode
• DUT1 and DUT2 are the Device Under Test we are
troubleshooting
• DUT1 and DUT2 are two 6800IA’s dual homed to VSS
Modules
• 2 x WS-C6504-E
• 2 x VS-SUP2T-10G
• 2 x WS-X6904-40G
• 2 x C6800IA-48TD
DUT-1 DUT-2
VSS
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Configuring Instant Access (FEX) and Discovery
sup-2t# config t
sup-2t(config)# interface port-channel 100
sup-2t(config-if)# switchport
sup-2t(config-if)# switchport mode fex-fabric
sup-2t(config-if)# fex associate 110
sup-2t(config-if)# no shut
sup-2t(config-if)# exit
sup-2t(config)# interface range TenGig 1/3/8, TenGig 2/4/5
sup-2t(config-if)# switchport
sup-2t(config-if)# channel-group 100 mode on
sup-2t(config-if)# no shut
FEX Configuration commands
3. Associate it with a
fex-id 110
1. Create a Layer2
Port-channel 100
2. Configure its
mode as fex-fabric
4. Select the Fabric physical links
5. Bundle them into
Port-Channel in mode on
Repeat the configuration for configuring second FEX Client (with FEX ID 120)
56
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Configuring Instant Access (FEX) and Discovery
sup2T# show fex
FEX FEX FEX FEX
Number Description State Model Serial
---------------------------------------------------------------------------
110 FEX0110 online C6800IA-48TD FOC1736W1A8
120 FEX0120 online C6800IA-48FPD FOC1736W197
sup2T#
Verify FEX is online
FEX id’s
110 and 120
FEX states:- Init, Connected, Registration, Image Download, Registered, Online, Offline
FEX model FEX Serial
Numbers
57
New IA Client releasing this week at CiscoLive 2014.
C6800IA-48FPDR
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Configuring Instant Access (FEX) and Discovery
sup2T# show fex detail
FEX: 110 Description: FEX0110 state: online
FEX version: 15.0(2)EX4
Extender Model: C6800IA-48TD, Extender Serial: FOC1736W1A8
FCP ready: yes
Image Version Check: enforced
Fabric Portchannel Ports: 2
Fabric port for control traffic: Te2/4/5
Fabric interface state:
Po100 - Interface Up.
Te1/3/8 - Interface Up. state: bound
Te2/4/5 - Interface Up. state: bound
Contd..
Verify Individual FEX states in detail
Contd..
FEX: 120 Description: FEX0120 state: online
FEX version: 15.0(2)EX4
Extender Model: C6800IA-48FPD, Extender Serial: FOC1736W197
FCP ready: yes
Image Version Check: enforced
Fabric Portchannel Ports: 2
Fabric port for control traffic: Te1/3/5
Fabric interface state:
Po200 - Interface Up.
Te1/3/5 - Interface Up. state: bound
Te2/4/8 - Interface Up. state: bound
sup2T#
58
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Configuring Instant Access (FEX) and Discovery
show fex <fex-id> detail
show switch virtual slot-map
test scp ping <vslot id of fex>
show module fex <fex-id>
show platform fex-debug fex <fex-id> sdp
show monitor event-trace fex clock <hh:mm>
show fex image bundle version
debug fex [error | sdp | srp | init] <fex-id> *
debug switch virtual fexmgr [error | event | packet] <fex-id> *
Other helpful commands Check in which state FEX is stuck
Get vslot id of fex and
Check if FEX Internal Network is Up
Check FEX diagnostics have passed
Check if SDP handshake was successful
Event trace utility to store all fex related events at different stages
Turn on common FEX debugs and restart FEX
Check the version and the stack member details
59
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Configuring Instant Access (FEX) and Discovery
sup2T# show fex system platform usage
FEX id usage details
Fex-ids inuse: 110, 120
Fex-ids online: 110, 120
Total Used Free
----- ---- ----
12 2 10
FEX slot usage details
FEX-id Switch-id Vslot Pslot Status
------ -------- ----- ----- ------
110 3 50 1 In-use
120 4 51 2 In-use
120 4 52 1 In-use
Total Used Reserved Free
----- ---- -------- ----
21 3 0 18
FEX ports usage details
Contd..
Stacked FEX and Usage details FEX-id Switch-id Ports
------ -------- -----
110 3 48
120 4 96
Total Used Free
----- ---- ----
1008 144 864
Stack members usage details
FEX-id Switch-id Used Free
------ -------- ---- ----
110 3 1 4
120 4 2 3
VNTAG MGR Usage
-----------------------
Max unicast VIFs available 2048
Total unicast VIFs used 144
Max non-mdest VIFs available 1019
Total non-mdest VIFs used 2
Max mdest VIFs available 13309
Total mdest VIFs used 12
sup2T#
Total FEX ports supported is 1008 in this release (15.1(2)SY1)
Total FEX modules that can be stacked is 5
Total FEX ID’s configurable is 12 in this release (15.1(2)SY1)
Each FEX module will consume one vslot id
FEX 120 is a stacked fex with 2 switches bundled into one FEX id
60
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Troubleshooting 6800ia solution
• Configuring Instant Access (FEX) and Discovery
– Topology
– Configuration CLI’s
– Troubleshooting CLI’s when FEX does not come online
• Unicast Packet Forwarding through FEX’es
– Topology
– Layer 2 unicast traffic CLI’s
– Layer 3 unicast traffic CLI’s
• Multicast Packet Forwarding through FEX
– Topology
– IGMP / MCAST forwarding CLI’s
Agenda
61
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Unicast Packet Forwarding through FEX’es Topology for Layer2 traffic
62
• VSS is Supervisor 2T based
system operating in Virtual
Switching mode
• FEX110 is dual homed to VSS
through fabric PortChannel 100
• FEX12 0 is dual homed to VSS
through fabric PortChannel 200
• One Host PC is directly
connected to sup2T uplink port
• Other two Host PCs are directly
connected to each of FEX’es
• All the hosts are on VLAN 10 with
subnet ip’s 10.10.10.0/24
10.10.10.2
10.10.10.3
10.10.10.4
VLA
N 1
0
Po100 Po200
FEX110 FEX120
VSS
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
FEX Interface Naming
interface GigabitEthernet 110/1/0/1
<Interface-type>/<fex-id>/<module>/<submode>/<port>
FEX-ID
101-199 Stack
Sub
Module FEX
Port
63
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Unicast Packet Forwarding through FEX’es
sup2T# show ip arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.10.10.1 - 0007.7d75.7d80 ARPA Vlan10
Internet 10.10.10.2 0 0050.568b.6012 ARPA Vlan10
Internet 10.10.10.3 91 c8f9.f90c.2941 ARPA Vlan10
Internet 10.10.10.4 90 0024.c4a5.bcc1 ARPA Vlan10
sup2T#
Layer2 unicast traffic troubleshooting
64
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Unicast Packet Forwarding through FEX’es
sup2T# show mac address-table address 0024.c4a5.bcc1
Legend: * - primary entry
<snip>
Displaying entries from active supervisor:
vlan mac address type learn age ports
----+----+---------------+-------+-----+----------+-----------------------------
10 0024.c4a5.bcc1 dynamic Yes 230 Gi110/1/0/1
Displaying entries from DFC switch [1] linecard [3]:
vlan mac address type learn age ports
----+----+---------------+-------+-----+----------+-----------------------------
* 10 0024.c4a5.bcc1 dynamic Yes 0 Gi110/1/0/1
Contd..
Layer2 unicast traffic troubleshooting
Contd..
Displaying entries from standby supervisor:
vlan mac address type learn age ports
----+----+---------------+-------+-----+----------+-----------------------------
10 0024.c4a5.bcc1 dynamic Yes 225 Gi110/1/0/1
Displaying entries from DFC switch [2] linecard [4]:
vlan mac address type learn age ports
----+----+---------------+-------+-----+----------+-----------------------------
* 10 0024.c4a5.bcc1 dynamic Yes 230 Gi110/1/0/1
sup2T#
* Denotes the primary forwarding entry. This entry indicates the ingress DFC linecard on which this mac was first learned.
65
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Unicast Packet Forwarding through FEX’es
sup2T# show mac address-table address c8f9.f90c.2941
Legend: * - primary entry
<snip>
Displaying entries from active supervisor:
vlan mac address type learn age ports
----+----+---------------+-------+-----+----------+-----------------------------
10 c8f9.f90c.2941 dynamic Yes 45 Gi120/1/0/1
Displaying entries from DFC switch [1] linecard [3]:
vlan mac address type learn age ports
----+----+---------------+-------+-----+----------+-----------------------------
* 10 c8f9.f90c.2941 dynamic Yes 45 Gi120/1/0/1
Contd..
Layer2 unicast traffic troubleshooting
Contd..
Displaying entries from standby supervisor:
vlan mac address type learn age ports
----+----+---------------+-------+-----+----------+-----------------------------
10 c8f9.f90c.2941 dynamic Yes 45 Gi120/1/0/1
1 c8f9.f90c.2941 dynamic Yes 190 Gi120/1/0/1
Displaying entries from DFC switch [2] linecard [4]:
vlan mac address type learn age ports
----+----+---------------+-------+-----+----------+-----------------------------
* 10 c8f9.f90c.2941 dynamic Yes 45 Gi120/1/0/1
sup2T#
66
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Unicast Packet Forwarding through FEX’es
sup2T# show counters interface Gig110/1/0/1
Time since last clear
---------------------
Never
64 bit counters:
0. rxHCTotalPkts = 80764
1. txHCTotalPkts = 448759
2. rxHCUnicastPkts = 60545
3. txHCUnicastPkts = 0
4. rxHCMulticastPkts = 20219
5. txHCMulticastPkts = 448249
6. rxHCBroadcastPkts = 0
7. txHCBroadcastPkts = 510
8. rxHCOctets = 8626030
9. txHCOctets = 38547300
10. rxTxHCPkts64Octets = 372976
<snip>
Layer2 unicast traffic troubleshooting
sup2T# clear counters
sup2T# show counters interface Gig110/1/0/1 delta
Time since last clear
---------------------
00:00:07
64 bit counters:
0. rxHCTotalPkts = 1
1. txHCTotalPkts = 4
2. rxHCUnicastPkts = 1
3. txHCUnicastPkts = 0
4. rxHCMulticastPkts = 0
5. txHCMulticastPkts = 4
<snip>
67
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Unicast Packet Forwarding through FEX’es Topology for Layer3 traffic
68
10.10.10.2
20.20.20.3 10.10.10.4
Po100
FEX110 FEX120
VSS
Gig120/1/0/1 Gig110/1/0/1
Gig1/2/1
Po200
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Unicast Packet Forwarding through FEX’es Layer3 unicast traffic troubleshooting
sup2T# show ip arp 20.20.20.3
Protocol Address Age (min) Hardware Addr Type Interface
Internet 20.20.20.3 22 c8f9.f90c.2944 ARPA Vlan20
sup2T# show mac address-table address c8f9.f90c.2944
<snip>
Displaying entries from active supervisor:
vlan mac address type learn age ports
---+----+-------------+----------+-------+-----+----------
20 c8f9.f90c.2944 dynamic Yes 360 Gi120/1/0/1
Displaying entries from DFC switch [1] linecard [3]:
vlan mac address type learn age ports
----+----+---------------+-------+-------+------+-------------
* 20 c8f9.f90c.2944 dynamic Yes 355 Gi120/1/0/1
sup2T# show ip arp 10.10.10.4
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.10.10.4 26 588d.093c.09c1 ARPA Vlan10
sup2T# show mac address-table address 588d.093c.09c1
<snip>
Displaying entries from active supervisor:
vlan mac address type learn age ports
----+----+---------------+-------+-----+----------
10 588d.093c.09c1 dynamic Yes 0 Gi110/1/0/1
Displaying entries from DFC switch [1] linecard [3]:
vlan mac address type learn age ports
----+----+---------------+-------+-----+--------+-----------
* 10 588d.093c.09c1 dynamic Yes 5 Gi110/1/0/1
69
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Unicast Packet Forwarding through FEX’es Layer3 unicast traffic troubleshooting
sup2T# show ip route 10.10.10.4
Routing entry for 10.10.10.0/24
Known via "connected", distance 0, metric 0 (connected, via interface)
Routing Descriptor Blocks:
* directly connected, via Vlan10
Route metric is 0, traffic share count is 1
sup2T#show ip route 20.20.20.3
Routing entry for 20.20.20.0/24
Known via "connected", distance 0, metric 0 (connected, via interface)
Routing Descriptor Blocks:
* directly connected, via Vlan20
Route metric is 0, traffic share count is 1
sup2T#
sup2T# show ip cef 10.10.10.4
10.10.10.4/32
attached to Vlan10
sup2T#show ip cef 20.20.20.3
20.20.20.3/32
attached to Vlan20
sup2T#show ip cef exact-route 20.20.20.3 10.10.10.4
20.20.20.3 -> 10.10.10.4 => IP adj out of Vlan10, addr 10.10.10.4
sup2T#show ip cef exact-route 10.10.10.4 20.20.20.3
10.10.10.4 -> 20.20.20.3 => IP adj out of Vlan20, addr 20.20.20.3
sup2T#
Use ‘exact-route’ command to get next hop adjacency of the packet between source and destination ip during CEF switching
70
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Unicast Packet Forwarding through FEX’es Layer3 unicast traffic troubleshooting
sup2T# show platform hardware cef lookup 10.10.10.4
Codes: decap - Decapsulation, + - Push Label
Index Prefix Adjacency
50 10.10.10.4/32 Vl10 ,0024.c4a5.bcc1
sup2T#
sup2T# show platform hardware cef exact-route 20.20.20.3 10.10.10.4
Interface: Vl10, Next Hop: 10.10.10.4, ifnum: 0x9E, Destination Mac: 588d.093c.09c1 LIF: 0x2000000A
sup2T#
sup2T# show adjacency 10.10.10.4 detail
Protocol Interface Address
IP Vlan10 10.10.10.4(8)
0 packets, 0 bytes
epoch 0
sourced in sev-epoch 0
Encap length 14
0024C4A5BCC100077D757D800800
L2 destination address byte offset 0
L2 destination address byte length 6
Link-type after encap: ip
ARP
sup2T#
Rewrite(next hop)info. Read as
DMAC|SMAC|0800
71
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Unicast Packet Forwarding through FEX’es Layer3 unicast traffic troubleshooting
sup2T# show platform hardware cef ip 10.10.10.4 detail
<snip>
Format:IPV4 (valid class vpn prefix)
M(50 ): 1 F 2FFF 255.255.255.255
V(50 ): 1 0 0 10.10.10.4
(A:163841, LS:0, NR:0, RI:0, DF:0 CP:0 DGTv:1, DGT:0)
sup2T# show platform hardware cef entry 50
Codes: decap - Decapsulation, + - Push Label
Index Prefix Adjacency
50 10.10.10.4/32 Vl10 ,0024.c4a5.bcc1
sup2T#
72
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Unicast Packet Forwarding through FEX’es Layer3 unicast traffic troubleshooting
sup2T# show platform hardware cef adjacencies entry 163841 detail
Index: 163841 -- Valid entry (valid = 1) --
Adjacency fields:
___________________________________________________
|adj_stats = EN | fwd_stats = EN | trig = 0
|_________________|__________________|______________
|l3_enable = ON (classify as Layer3) | age = 3
|_________________|__________________|______________
|format = IP | rdt = OFF | ignr_emut = 0
|_________________|__________________|______________
|vpn = 0x3FFF | elif = 0xA | ri = 3
|_________________|__________________|______________
|top_sel = 0 | zone_enf = OFF | fltr_en = OFF
|_________________|__________________|______________
Contd..
RIT fields: The entry has a Layer2 Format
|add_shim_hdr= NO | rec_findex = N/A | rec_shim_op = N/A
|_________________|__________________|____________________
|rec_dti_type = N/A | rec_data = N/A
|____________________________________|____________________
|modify_smac = YES| modify_dmac = YES| egress_mcast = NO
|____________________________________|____________________
|ip_to_mac = NO
|_________________________________________________________
|dest_mac = 0024.c4a5.bcc1 | src_mac = 0007.7d75.7d80 |
|____________________________|_____________________________
Statistics: Packets = 0
Bytes = 0
73
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Troubleshooting 6800ia solution
• Configuring Instant Access (FEX) and Discovery
– Topology
– Configuration CLI’s
– Troubleshooting CLI’s when FEX does not come online
• Unicast Packet Forwarding through FEX’es
– Topology
– Layer 2 unicast traffic CLI’s
– Layer 3 unicast traffic CLI’s
• Multicast Packet Forwarding through FEX
– Topology
– IGMP / MCAST forwarding CLI’s
Agenda
74
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Multicast packet forwarding through FEX Topology
75
10.10.10.2
FEX110 FEX120
VSS
Gig120/1/0/1 Gig110/1/0/1
Gig1/2/1 L3 Cloud
Vlan 20
Vlan 20 Vlan 10
Receiver 20.20.20.3 Receiver
172.16.25.2 Source
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Multicast Packet Forwarding through FEX’es Layer3 multicast traffic troubleshooting
sup2T# show ip mroute 239.1.1.1
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
<snip>
(*, 239.1.1.1), 1w0d/00:02:51, RP 10.10.10.1, flags: SJC
Incoming interface: Null, RPF nbr 0.0.0.0
Outgoing interface list:
Vlan10, Forward/Sparse, 03:05:38/00:02:32
Vlan20, Forward/Sparse, 03:22:02/00:02:51
(172.16.25.2, 239.1.1.1), 03:28:03/00:02:46, flags: T
Incoming interface: Vlan20, RPF nbr 20.20.20.22
Outgoing interface list:
Vlan10, Forward/Sparse, 03:05:39/00:02:31
sup2T#
*,G entry
S,G entry
OIL
RPF Neighbor
Incoming interface
76
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Multicast Packet Forwarding through FEX’es Layer3 multicast traffic troubleshooting
sup2T# show ip igmp groups 239.1.1.1
IGMP Connected Group Membership
Group Address Interface Uptime Expires Last Reporter Group Accounted
239.1.1.1 Vlan10 02:59:40 00:02:33 10.10.10.2
239.1.1.1 Vlan20 6d02h 00:02:23 20.20.20.3
sup2T# show mac address-table multicast igmp-snooping 239.1.1.1
vlan mac/ip address LTL ports
+----+-----------------------------------------+------+-----------------------
10 ( *,239.1.1.1) 0x6112 Router Po100 Gi110/1/0/1
20 ( *,239.1.1.1) 0x6116 Router Gi1/2/1 Po20 Po200 Gi120/1/0/1
sup2T#
Both vlan’s show as part of this igmp group (239.1.1.1)
Local Target Logic is used to determine the physical port that will forward this packet
77
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Multicast Packet Forwarding through FEX’es Layer3 multicast traffic troubleshooting
sup2T# test platform software switch virtual vntag_mgr vif-map ltl 0x6112 detail
VIF INFO:
VIF# 3072
Type MULTI-DESTINATION VIF
LTL# 6112
OperStatus# 2
Number of Ports:1
List of Ports: Gi110/1/0/1,
sup2T#
Check which FEX ports will carry
this multicast traffic
Provides a mapping between
VIF and LTL index
78
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Multicast Packet Forwarding through FEX’es Layer3 multicast traffic troubleshooting
sup2T# test platform software switch virtual vntag_mgr vif-map ltl 0x6116 detail
VIF INFO:
VIF# 3074
Type MULTI-DESTINATION VIF
LTL# 6116
OperStatus# 2
Number of Ports:1
List of Ports: Gi120/1/0/1,
sup2T#
79
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Multicast Packet Forwarding through FEX’es
sup2T# show platform hardware multicast routing ip source 172.16.25.2 group 239.1.1.1 detail
IPv4 Multicast CEF Entries for VPN#0
(172.16.25.2, 239.1.1.1/32)
FIBAddr: 0x40 IOSVPN: 0 RpfType: SglRpfChk SrcRpf: Vl20
CPx: 0 s_star_pri: 1 non-rpf drop: 0
PIAdjPtr: 0x28002 Format: IP rdt: off elif: 0xC5409
fltr_en: off idx_sel/bndl_en: 0 dec_ttl: on mtu_idx: 2(1518)
PV: 1 rwtype: MCAST_L2_EXPS
met3: 0x0 met2: 0x8022
Packets: 13961 Bytes: 14351908
Contd..
Layer3 multicast traffic troubleshooting
Check for hardware programming.
Ensure packets/bytes are incrementing
PIAdjPtr provides statistics for ingress
LC carrying mcast traffic on same VLAN
80
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Multicast Packet Forwarding through FEX’es Layer3 multicast traffic troubleshooting
sup2T-sw1-dfc3# show platform hardware multicast routing ip source 172.16.25.2 group 239.1.1.1 detail
IPv4 Multicast CEF Entries for VPN#0
<snip>
NPIAdjPtr: 0x18003 Format: IP rdt: on elif: 0xC5409
fltr_en: off idx_sel/bndl_en: 0 dec_ttl: off
PV: 0 rwtype: MCAST_L3_REWRITE
met3: 0x58 met2: 0x0 DestNdx: 0x7FF3
Packets: 13325 Bytes: 13698100
MET offset: 0x58
OIF AdjPtr Elif CR (Sw1)
+-------------+-----------+-----------+------------+
Vl10 0x800A 0xA 3/T2
sup2T-sw1-dfc3#
Check for correct hardware programming.
Ensure packets/bytes are incrementing
Check this on the egress module/dfc
NPIAdjPtr provides statistics for egress
LC carrying mcast traffic
81
Questions ?
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Complete Your Online Session Evaluation
• Give us your feedback and you could win fabulous prizes. Winners announced daily.
• Complete your session evaluation through the Cisco Live mobile app or visit one of the interactive kiosks located throughout the convention center.
Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online
83
© 2014 Cisco and/or its affiliates. All rights reserved. BRKCRS-3148 Cisco Public
Continue Your Education
• Demos in the Cisco Campus
• Walk-in Self-Paced Labs
• Table Topics
• Meet the Engineer 1:1 meetings
84