© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
NX-OS Multicast Design and Recommended
Practices BRKIPM-3062
Ron Fuller– CCIE #5851 (R&S/Storage) Technical Marketing Engineer, Nexus 7000 [email protected]
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Housekeeping
We value your feedback- don't forget to complete your online session
evaluations after each session & the Overall Conference Evaluation which
will be available online from Thursday
Visit the World of Solutions and Meet the Engineer
Visit the Cisco Store to purchase your recommended readings
After the event don’t forget to visit Cisco Live Virtual:
www.ciscolivevirtual.com
Please switch off your mobile phones
Follow us on Twitter for real time updates of the event:
@ciscolive
3
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Course Objective
What you will learn…..
NX-OS Multicast Features – Platform Independent
Platform Specific Information
Recommended Practices with NX-OS
Summary
4
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Technical Breakout Sessions
Session-ID Session Name
BRKARC-3470 Cisco Nexus 7000 Switch Architecture
BRKARC-3452 Cisco Nexus 5000/5500 and 2000 Switch Architecture
BRKARC-3471 Cisco NX-OS Software Architecture
BRKARC-3472 Cisco NX-OS Routing and Layer 3 Switching
BRKDCT-2121 Virtual Device Context (VDC) Design and Implementation Considerations with
Nexus 7000
TECRST-3190 Advanced IP Routing Fast Convergence
BRKIPM-3062 Nexus Multicast Design Best Practices
TECDCT-3297 Operating and Deploying NX-OS Nexus Devices in the Network Infrastructure
TECVIR-2003 Enterprise Network Virtualization
Related Cisco Live 2012 Events
5
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Agenda
NX-OS Multicast Features – Platform Independent
Nexus 7000 Platform Specifics
Nexus 5000 Platform Specifics
Nexus 3000 Platform Specifics
Recommended Practices with NX-OS
Summary
Q&A
6
NX-OS Multicast Features – Platform
Independent
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
NX-OS Multicast Architecture
NX-OS is a modular operating system
‒ Modularity helps with high availability, resource allocation and scale
Some software components are always loaded
‒ Others may be conditional
Modularity includes multicast components
Multicast in NX-OS is ―VRF-Aware‖
NX-OS has unique features which can change traditional multicast models
NX-OS does not support PIM Dense Mode
8
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
NX-OS IGMP Snooping
IGMP Snooping on by default and supports v1/v2/v3
224.0.0.X are reserved for protocol use. All switches should flood the frame with destination IP address 224.0.0.X
All frames with destination MAC 0100.5E00.00XX will be flooded. Avoid using IP multicast groups that map to this MAC address range
Packets destined to unknown IGMP groups are dropped (except 224.0.0.x)
Detect mrouter ports via IGMP query and PIM hello
Can be configured as IGMP V3 querier. Support hosts running all IGMP version with backward compatibility
Fast leave is disabled by default
IGMP v3 explicit tracking is on. Track joins from individual host
9
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
PIM Sparse Mode
General purpose multicast routing protocol
Data-driven multicast state
Automatic source discovery
Efficient on-demand packet delivery
Uses both shared and source-based trees
‒ Distribution trees are unidirectional
Can support arbitrary source and receiver distribution
Group membership tracked via IGMPv1, v2, or v3
PIM-SM
RP
Shared Tree a.k.a RPT, rooted at the rendezvous point
Source Tree a.k.a SPT, rooted at the source
Source
Receivers
10
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Source2
PIM Source-Specific Multicast
Simplifies one-to-many multicast delivery— uses source trees only
Control-plane multicast state
Assumes one-to-many model
‒ Internet/inter-domain multicast
‒ Video distribution
Hosts responsible for source discovery—
‒ Typically via some out-of-band mechanism (web page, content server, etc.)
‒ Eliminates need for RP and shared trees
‒ Eliminates need for MSDP
Group membership tracked via IGMPv3
‒ SSM mapping also supported
PIM-SSM Source Tree Rooted at Source1
Source1
S1,G1
Source Tree Rooted at Source2
S2,G1 S1,G1 S1,G1
Receivers
S2,G1
11
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Bidirectional PIM
Massively scalable—ideal for many-to-many applications
Data-flow independent—no registers, asserts, non-RPF issues
Drastically reduces network mroute state
‒ Eliminates ALL (S,G) state in the network for Bidir groups
‒ Shortest path trees from sources to RP eliminated
‒ Source traffic flows both up and down shared RP tree
‒ Permits virtually unlimited sources
Bidir-PIM
RP
Shared Tree Bidirectional tree rooted at the rendezvous point
Source
Sources/Receivers
12
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 13
PIM
Conditional service
‒ Parser available after you enable via feature pim command
‒ Process runs after you configure an interface for PIM
Single PIM process runs any/all PIM flavors for VDC
PIM process responsibilities:
‒ Form PIM neighborships
‒ Processes inbound and outbound PIM protocol packets
‒ Encapsulate and transmit PIM registers, process PIM Register Stop messages (first-hop router)
‒ Decapsulate and process PIM registers, send Register Stop messages (PIM RP)
‒ Interface with MRIB to provide/learn multicast routes
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 14
IGMP
IGMP process always running
Single IGMP process provides both Layer 3 IGMP processing, and Layer 2 IGMP snooping functions
Layer 3 IGMP functions include:
‒ Send IGMP queries on PIM-enabled interfaces
‒ Process IGMP reports (joins) and leaves received from multicast receivers
‒ Interface with MRIB to provide/learn multicast routes
Layer 2 IGMP snooping functions include:
‒ Process snooped multicast router packets (PIM Hellos, IGMP queries)
‒ Process IGMP reports and leaves sent by receivers
‒ Interface with MFDM to provide snooping entries
‒ Send special IGMP Leave messages to mrouters on TCN if switch is STP root
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 15
MSDP
Conditional service
‒ Parser available after you enable via feature pim command
‒ Process runs after you configure at least one MSDP peer
Functions of MSDP process include:
‒ Establish MSDP peering relationships
‒ Transmit source-active messages to configured peers
‒ Receive source-active messages from peers
‒ Interface with MRIB to provide/learn multicast routes
MSDP SA cache enabled by default (non configurable)
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 16
MRIB
Multicast routing information base (MRIB) process always running
Functions include:
‒ Interface with various client processes to provide/learn multicast routes
‒ Combine information from various sources into single multicast routing table (the
MRIB)
‒ Interface with MFDM to provide routes for hardware programming IPv4 (M4RIB)
and IPv6 (M6RIB)
‒ RPF services via U4RIB / U6RIB
‒ Client-driven and designed for easy insertion of new clients
‒ Detailed traffic statistics
16
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 17
MFDM
Multicast Forwarding Distribution Manager (MFDM) process always running
Acts as interface between platform-independent Supervisor Engine processes, and platform-specific I/O module processes
Translates MRIB data into data structures required by hardware
Distributes that platform-specific information to I/O modules
17
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 18
IPFIB and L2MCAST (Nexus 7000)
Processes run on each I/O module
‒ Single instance of each handles requests from all configured VDCs
Both processes
‒ Interface with MFDM to receive platform-specific data structures required to program multicast forwarding entries into hardware
‒ Interact with hardware drivers to program ASIC forwarding tables
IPFIB – Responsible for programming (*,G) and (S,G) entries in FIB/ADJ, OILs in MET
L2MCAST – Responsible for programming IGMP snooping entries in MAC table
18
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 19
PIXM/PIXMC (Nexus 7000 Only)
Port Index Manager (PIXM) process handles index table management for all VDCs
‒ Interfaces with MFDM in each configured VDC
Serves critical function of allocating and managing index tables (LTL and FPOE) for system
Pushes table information to PIXM Client (PIXMC) process running on each I/O module
PIXMC interfaces with hardware drivers to program hardware
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 20
Multicast State Creation
How is state created?
Depends on:
‒ Where router sits relative to sources, receivers, and RP (if applicable)
‒ What flavor of PIM used
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 21
PIM-SM State Creation
(*,G) state driven by control plane protocols
‒ On last-hop router, IGMP joins create (*,G) state
‒ On upstream routers, PIM joins create (*,G) state all the way to the RP
(S,G) state frequently driven by data packets
‒ On first-hop router, packets punted to Supervisor Engine CPU to create (S,G) state, trigger PIM registers
‒ On last-hop router, packets received on shared tree punted to Supervisor Engine CPU to create (S,G) state, trigger SPT switchover
‒ On RP, PIM registers sent to Supervisor Engine CPU to create (S,G) state, trigger register stops and SPT switchover
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 22
PIM-SSM and PIM-Bidir State Creation
PIM-SSM (S,G) state driven by control plane protocols
‒ On last-hop router, IGMPv3 joins create (S,G) state
‒ On upstream routers, PIM-SSM joins create (S,G) state all the way to the first-hop
router
PIM-Bidir (*,G) state driven by control plane protocols
‒ On last-hop router, IGMP joins create (*,G) state
‒ On upstream routers, PIM joins create (*,G) state all the way to the RP
‒ On source-only branches, control plane installs (*,G/m) entries to enable data
forwarding toward bidir RP
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 23
Factors Influencing Control-Plane State
Creation Data-driven state creation consumes inband bandwidth and requires CPU cycles
Nexus 7000 implements hardware rate limiters and CoPP by default to protect these resources
Default values may not be appropriate/optimized for all environments
Examples:
‒ Multicast data packets (i.e., outside 224.0.0.0/24 range) match CoPP class-default
‒ copp-system-class-important lumps PIM registers with other traffic (FHRPs etc.)
‒ Directly-connected and local-groups rate limiters set to 3000 pps each
Default rate for mcast-snooping rate limiter (10000 pps) should be plenty
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Multicast Packet Flow with VPC (1) Source in L3, Remote RP
1. Receiver sends IGMP join, EtherChannel hash on access switch happens to select link to vpc-peer2
Creates snooping, IGMP, and (*,G) mroute state with VPC VLAN as OIF
2. vpc-peer2 sends IGMP packet encapsulated in CFS to vpc-peer1
Creates identical state to vpc-peer2
3. Both VPC peers send PIM (*,G) joins to the RP to join the RPT
If ECMP to RP, hash selects the RPF interface
Source
Receiver
vpc-peer1 VPC Pri PIM-DR
vpc-peer2 VPC Sec Proxy-DR
IGMP Join
IGMP in CFS
1
2
(*,G) (*,G) PIM Join
3
4
core1 Anycast-RP
core2 Anycast-RP
24
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 25
State After IGMP Join from Receiver (1)
Source
Receiver
vpc-peer1 VPC Pri PIM-DR
vpc-peer2 VPC Sec Proxy-DR
(*,G) (*,G)
vpc-peer1# sh ip igmp snooping groups vlan 101
Type: S - Static, D - Dynamic, R - Router port, F - Fabricpath core port
Vlan Group Address Ver Type Port list
101 */* - R Vlan101 Po1
101 239.1.1.1 v2 D Po100
vpc-peer1# sh ip igmp groups vlan 101
IGMP Connected Group Membership for Interface "Vlan101" - 1 total entries
Type: S - Static, D - Dynamic, L - Local, T - SSM Translated
Group Address Type Interface Uptime Expires Last Reporter
239.1.1.1 D Vlan101 00:00:33 00:04:11 10.100.101.100
vpc-peer1# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:00:39, igmp ip pim
Incoming interface: port-channel51, RPF nbr: 10.1.1.1
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:00:39, igmp
vpc-peer1#
vpc-peer2# sh ip igmp snooping groups vlan 101
Type: S - Static, D - Dynamic, R - Router port, F - Fabricpath core port
Vlan Group Address Ver Type Port list
101 */* - R Vlan101 Po1
101 239.1.1.1 v2 D Po100
vpc-peer2# sh ip igmp groups vlan 101
IGMP Connected Group Membership for Interface "Vlan101" - 1 total entries
Type: S - Static, D - Dynamic, L - Local, T - SSM Translated
Group Address Type Interface Uptime Expires Last Reporter
239.1.1.1 D Vlan101 00:01:19 00:03:26 10.100.101.100
vpc-peer2# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:01:24, igmp ip pim
Incoming interface: port-channel52, RPF nbr: 10.1.1.13
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:01:24, igmp
vpc-peer2#
vlan101
po100 po100
po1 po51 po52
po1 po2
core1 Anycast-RP
core2 Anycast-RP
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 26
State After IGMP Join from Receiver (2)
Source
Receiver
vpc-peer1 VPC Pri PIM-DR
vpc-peer2 VPC Sec Proxy-DR
(*,G) (*,G)
core1# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:01:48, pim ip
Incoming interface: loopback2, RPF nbr: 200.200.200.200
Outgoing interface list: (count: 1)
port-channel1, uptime: 00:01:48, pim
core1#
core2# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:02:02, pim ip
Incoming interface: loopback2, RPF nbr: 200.200.200.200
Outgoing interface list: (count: 1)
port-channel2, uptime: 00:02:02, pim
core2# vlan101
po100 po100
po1
po51 po52
po1 po2
core1 Anycast-RP
core2 Anycast-RP
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
(*,G)
Multicast Packet Flow with VPC (2) Source in L3, Remote RP
5. Source begins transmitting
FHR registers source to RPs, etc.
6. One or both VPC peers receive (S,G) traffic on shared tree
Depends on upstream state
7. VPC peer switches negotiate for forwarder role
CFS messages exchanged to determine forwarder
Best routing metric, with VPC role as tie-breaker
8. Elected forwarder for (S,G), sends PIM (S,G) joins toward source
Joins SPT, prunes RPT
Adds VPC VLAN as L3 OIF
9. Data traffic flows down source tree to forwarding peer
Traffic also forwarded on peer link, dropped by other peer
Source
Receiver
vpc-peer1 VPC Pri PIM-DR
vpc-peer2 VPC Sec Proxy-DR
CFS
6 + (S,G) (*,G) PIM Join
9
Data
7
8 PIM Prune
5
+ (S,G)
core1 Anycast-RP
core2 Anycast-RP
27
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 28
State After Peer Joins SPT
Source
Receiver
vpc-peer1 VPC Pri PIM-DR
vpc-peer2 VPC Sec Proxy-DR
(*,G) + (S,G)
vpc-peer1# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:18:13, igmp ip pim
Incoming interface: port-channel51, RPF nbr: 10.1.1.1
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:18:13, igmp
(10.200.0.100/32, 239.1.1.1/32), uptime: 00:14:49, ip pim mrib
Incoming interface: port-channel51, RPF nbr: 10.1.1.1
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:14:48, mrib
vpc-peer1#
vpc-peer2# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:18:29, igmp ip pim
Incoming interface: port-channel52, RPF nbr: 10.1.1.13
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:18:29, igmp
(10.200.0.100/32, 239.1.1.1/32), uptime: 00:05:01, ip pim
Incoming interface: port-channel51, RPF nbr: 10.1.1.5
Outgoing interface list: (count: 0)
vpc-peer2#
vlan101
po100 po100
po1 po51 po52
po1 po2
(*,G) + (S,G)
core1 Anycast-RP
core2 Anycast-RP
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Multicast Packet Flow with VPC (1) Source in L3, Local RP
1. Receiver sends IGMP join, EtherChannel hash on access switch happens to select link to vpc-peer2
Creates snooping, IGMP, and (*,G) mroute state with VPC VLAN as OIF
2. vpc-peer2 sends IGMP packet encapsulated in CFS to vpc-peer1
Creates identical state to vpc-peer2
3. VPC peers are Anycast-RPs so no further PIM
activity
Source
Receiver
vpc-peer1 VPC Pri PIM-DR Anycast-RP
vpc-peer2 VPC Sec Proxy-DR Anycast-RP
IGMP Join IGMP in CFS 1
2
(*,G) (*,G)
core1 core2
29
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 30
State After IGMP Join from Receiver
Source
Receiver
(*,G) (*,G)
vpc-peer1# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:00:15, igmp pim ip
Incoming interface: loopback2, RPF nbr: 100.100.100.100
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:00:15, igmp
vpc-peer1#
vpc-peer2# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:00:06, igmp pim ip
Incoming interface: loopback2, RPF nbr: 100.100.100.100
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:00:06, igmp
vpc-peer2# vlan101
po100 po100
po1 po51 po52
po1 po2
vpc-peer1 VPC Pri PIM-DR Anycast-RP
vpc-peer2 VPC Sec Proxy-DR Anycast-RP
core1 core2
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
(*,G)
Multicast Packet Flow with VPC (2) Source in L3, Local RP
4. Source begins transmitting
FHR registers source to the RP
One VPC peer receives PIM registers
5. VPC peer switches negotiate for forwarder role
CFS messages exchanged to determine forwarder
Best routing metric, with VPC role as tie-breaker
6. Elected forwarder for (S,G) joins SPT, sends register
stops
7. Data traffic flows down source tree to forwarding peer
Traffic forwarded on peer link, dropped by other peer
Source
Receiver
CFS
5 + (S,G) (*,G)
PIM Join + Register Stop
6
vpc-peer1 VPC Pri PIM-DR Anycast-RP
vpc-peer2 VPC Sec Proxy-DR Anycast-RP
Data
4
core1 core2
+ (S,G)
7
31
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 32
State After Peer Joins SPT (1)
Source
Receiver
(*,G) + (S,G)
vpc-peer1# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:00:39, igmp pim ip
Incoming interface: loopback2, RPF nbr: 100.100.100.100
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:00:39, igmp
(10.200.0.100/32, 239.1.1.1/32), uptime: 00:00:22, ip msdp pim mrib
Incoming interface: port-channel51, RPF nbr: 10.1.1.1
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:00:22, mrib
vpc-peer1#
vpc-peer2# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:00:57, igmp pim ip
Incoming interface: loopback2, RPF nbr: 100.100.100.100
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:00:57, igmp
(10.200.0.100/32, 239.1.1.1/32), uptime: 00:00:40, pim ip
Incoming interface: port-channel51, RPF nbr: 10.1.1.5, internal
Outgoing interface list: (count: 0)
vpc-peer2#
vlan101
po100 po100
po1 po51
po51
po1 po2
(*,G) + (S,G) vpc-peer1 VPC Pri PIM-DR Anycast-RP
vpc-peer2 VPC Sec Proxy-DR Anycast-RP
core1 core2
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 33
State After Peer Joins SPT (2)
core1# sh ip mroute IP Multicast Routing Table for VRF "default"
(10.200.0.100/32, 239.1.1.1/32), uptime: 00:00:55, ip pim
Incoming interface: Vlan200, RPF nbr: 10.200.0.100
Outgoing interface list: (count: 1)
port-channel1, uptime: 00:00:49, pim
core1#
core2# sh ip mroute
IP Multicast Routing Table for VRF "default"
(10.200.0.100/32, 239.1.1.1/32), uptime: 00:01:12, ip pim
Incoming interface: Vlan200, RPF nbr: 10.200.0.100
Outgoing interface list: (count: 0)
core2#
Source
Receiver
(*,G) + (S,G)
vlan101
po100 po100
po1 po51 po52
po1 po2
(*,G) + (S,G) vpc-peer1 VPC Pri PIM-DR Anycast-RP
vpc-peer2 VPC Sec Proxy-DR Anycast-RP
core1 core2
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
(*,G) (*,G)
Pre-Build SPT Option
ip pim pre-build-spt
1. Both forwarder and non-forwarder peer switches join SPT for new sources
2. Data traffic flows down source tree to both Peer-1 and Peer-2
3. On failure of forwarder (Peer-1), new forwarder (Peer-2) already has (S,G) state, is receiving traffic, and only needs to add OIFs
Pre-build SPT considerations:
Creates Live/Live data stream
Consumes bandwidth and replication capacity on primary and secondary data path in steady state
Decreases reconvergence time on failure (no need to create upstream state)
Source
Receiver
vpc-peer1 VPC Pri PIM-DR Anycast-RP
vpc-peer1 VPC Sec Proxy-DR Anycast-RP
2 + (S,G) + (S,G)
PIM Join
3 Data
1 PIM Join 1
2
X 4
Add OIFs
core1 core2
34
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 35
State After Both Peers Join SPT (pre-build-
spt option) Source
Receiver
(*,G) + (S,G)
core1# sh ip mroute
IP Multicast Routing Table for VRF "default"
(10.200.0.100/32, 239.1.1.1/32), uptime: 00:00:11, ip pim
Incoming interface: Vlan200, RPF nbr: 10.200.0.100
Outgoing interface list: (count: 2)
port-channel1, uptime: 00:00:10, pim
port-channel2, uptime: 00:00:10, pim
core1#
core2# sh ip mroute
IP Multicast Routing Table for VRF "default"
(10.200.0.100/32, 239.1.1.1/32), uptime: 00:00:23, ip pim
Incoming interface: Vlan200, RPF nbr: 10.200.0.100
Outgoing interface list: (count: 0)
core2#
vlan101
po100 po100
po1 po51 po52
po2
(*,G) + (S,G) vpc-peer1 VPC Pri PIM-DR Anycast-RP
vpc-peer2 VPC Sec Proxy-DR Anycast-RP
po1
core1 core2
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
(*,G)
Source Receiver
+ (S,G) (*,G)
Multicast Packet Flow with VPC Source in L2, Remote RP
1. Source traffic arrives on vpc-peer1, creates
(S,G) state, initiates PIM regsiters etc.
2. Data traffic flows down VPC to receiver
3. vpc-peer1 also forwards data traffic over peer
link to vpc-peer2, which creates (S,G) state
Traffic dropped by vpc-peer2
core1 Anycast-RP
core2 Anycast-RP
Data
+ (S,G)
1 2
3
vpc-peer1 VPC Pri PIM-DR
vpc-peer2 VPC Sec Proxy-DR
36
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 37
State After Source Starts Sending (1)
vpc-peer1# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:01:54, igmp ip pim
Incoming interface: port-channel51, RPF nbr: 10.1.1.1
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:01:54, igmp
(10.100.112.100/32, 239.1.1.1/32), uptime: 00:00:54, ip pim mrib
Incoming interface: Vlan112, RPF nbr: 10.100.112.100
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:00:53, mrib
vpc-peer1#
vpc-peer2# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:02:06, igmp ip pim
Incoming interface: port-channel52, RPF nbr: 10.1.1.13
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:02:06, igmp
(10.100.112.100/32, 239.1.1.1/32), uptime: 00:01:05, ip pim mrib
Incoming interface: Vlan112, RPF nbr: 10.100.112.100
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:01:05, mrib
vpc-peer2#
(*,G)
Source Receiver
+ (S,G) (*,G)
core1 Anycast-RP
core2 Anycast-RP
+ (S,G)
vlan101 vlan112
po51 po52
po2 po1
vpc-peer1 VPC Pri PIM-DR
vpc-peer2 VPC Sec Proxy-DR
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 38
State After Source Starts Sending (2)
core1# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:02:29, pim ip
Incoming interface: loopback2, RPF nbr: 200.200.200.200
Outgoing interface list: (count: 1)
port-channel1, uptime: 00:02:29, pim
(10.100.112.100/32, 239.1.1.1/32), uptime: 00:01:29, pim mrib ip
Incoming interface: port-channel2, RPF nbr: 10.1.1.6, internal
Outgoing interface list: (count: 0)
core1#
core2# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:02:43, pim ip
Incoming interface: loopback2, RPF nbr: 200.200.200.200
Outgoing interface list: (count: 1)
port-channel2, uptime: 00:02:43, pim
(10.100.112.100/32, 239.1.1.1/32), uptime: 00:01:43, pim mrib ip
Incoming interface: port-channel2, RPF nbr: 10.1.1.14, internal
Outgoing interface list: (count: 0)
core2#
(*,G)
Source Receiver
+ (S,G) (*,G)
core1 Anycast-RP
core2 Anycast-RP
+ (S,G)
vlan101 vlan112
po51 po52
po2 po1
vpc-peer1 VPC Pri PIM-DR
vpc-peer2 VPC Sec Proxy-DR
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
(*,G)
Multicast Packet Flow with VPC Source in L2, Local RP
1. Source traffic arrives on vpc-peer1, creates
(S,G) state
VPC peers are Anycast-RPs so no further PIM activity
2. Data traffic flows down VPC to receiver
3. vpc-peer1 also forwards data traffic over peer
link to vpc-peer2, which creates (S,G) state
Traffic dropped by vpc-peer2
Source Receiver
+ (S,G) (*,G)
Data
1
vpc-peer1 VPC Pri PIM-DR Anycast-RP
vpc-peer2 VPC Sec Proxy-DR Anycast-RP
+ (S,G)
2
3
39
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 40
State After Source Starts Sending
vpc-peer1# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:00:12, igmp pim ip
Incoming interface: loopback2, RPF nbr: 100.100.100.100
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:00:12, igmp
(10.100.112.100/32, 239.1.1.1/32), uptime: 00:00:03, ip pim mrib
Incoming interface: Vlan112, RPF nbr: 10.100.112.100, internal
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:00:03, mrib
vpc-peer1#
vpc-peer2# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 00:00:30, igmp pim ip
Incoming interface: loopback2, RPF nbr: 100.100.100.100
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:00:30, igmp
(10.100.112.100/32, 239.1.1.1/32), uptime: 00:00:21, ip msdp pim mrib
Incoming interface: Vlan112, RPF nbr: 10.100.112.100
Outgoing interface list: (count: 1)
Vlan101, uptime: 00:00:20, mrib
vpc-peer2#
(*,G)
Source Receiver
+ (S,G) (*,G)
vpc-peer1 VPC Pri PIM-DR Anycast-RP
vpc-peer2 VPC Sec Proxy-DR Anycast-RP
+ (S,G)
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Source
(*,G)
Source Receiver
+ (S,G) (*,G)
Multicast Packet Flow with VPC Source in L2 and L3, Remote RP
State is combination of previous examples
(*,G) and (S,G) state on both VPC peers
For source in L2, either VPC peer can forward
For source in L3, one peer chosen to forward
+ (S,G)
Data
Data
vpc-peer1 VPC Pri PIM-DR
vpc-peer2 VPC Sec Proxy-DR
core1 Anycast-RP
core2 Anycast-RP
41
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 42
State After Sources Start Sending (1)
vpc-peer1# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 02:13:51, igmp ip pim
Incoming interface: port-channel51, RPF nbr: 10.1.1.1
Outgoing interface list: (count: 1)
Vlan101, uptime: 02:13:51, igmp
(10.100.112.100/32, 239.1.1.1/32), uptime: 02:12:50, ip pim mrib
Incoming interface: Vlan112, RPF nbr: 10.100.112.100
Outgoing interface list: (count: 1)
Vlan101, uptime: 02:12:50, mrib
(10.200.0.100/32, 239.1.1.1/32), uptime: 02:10:35, ip pim mrib
Incoming interface: port-channel51, RPF nbr: 10.1.1.1
Outgoing interface list: (count: 1)
Vlan101, uptime: 02:10:35, mrib
vpc-peer1#
vpc-peer2# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 02:14:07, igmp ip pim
Incoming interface: port-channel52, RPF nbr: 10.1.1.13
Outgoing interface list: (count: 1)
Vlan101, uptime: 02:14:07, igmp
(10.100.112.100/32, 239.1.1.1/32), uptime: 02:13:07, ip pim mrib
Incoming interface: Vlan112, RPF nbr: 10.100.112.100
Outgoing interface list: (count: 1)
Vlan101, uptime: 02:13:07, mrib
(10.200.0.100/32, 239.1.1.1/32), uptime: 00:03:08, ip pim
Incoming interface: port-channel51, RPF nbr: 10.1.1.5
Outgoing interface list: (count: 0)
vpc-peer2#
Source
(*,G)
Source Receiver
+ (S,G) (*,G) + (S,G) vpc-peer1 VPC Pri PIM-DR
vpc-peer2 VPC Sec Proxy-DR
core1 Anycast-RP
core2 Anycast-RP
vlan101 vlan112
po52
po2 po1
po51 po51
vlan200
po2
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 43
State After Sources Start Sending (2)
core1# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 02:14:29, pim ip
Incoming interface: loopback2, RPF nbr: 200.200.200.200
Outgoing interface list: (count: 1)
port-channel1, uptime: 02:14:29, pim
(10.100.112.100/32, 239.1.1.1/32), uptime: 02:13:29, pim mrib ip
Incoming interface: port-channel2, RPF nbr: 10.1.1.6, internal
Outgoing interface list: (count: 0)
(10.200.0.100/32, 239.1.1.1/32), uptime: 02:11:13, ip mrib pim
Incoming interface: Vlan200, RPF nbr: 10.200.0.100, internal
Outgoing interface list: (count: 1)
port-channel1, uptime: 02:11:13, mrib, pim
core1#
core2# sh ip mroute
IP Multicast Routing Table for VRF "default"
(*, 239.1.1.1/32), uptime: 02:14:50, pim ip
Incoming interface: loopback2, RPF nbr: 200.200.200.200
Outgoing interface list: (count: 1)
port-channel2, uptime: 02:14:50, pim
(10.100.112.100/32, 239.1.1.1/32), uptime: 02:13:50, pim mrib ip
Incoming interface: port-channel2, RPF nbr: 10.1.1.14, internal
Outgoing interface list: (count: 0)
(10.200.0.100/32, 239.1.1.1/32), uptime: 02:11:34, ip mrib pim
Incoming interface: Vlan200, RPF nbr: 10.200.0.100, internal
Outgoing interface list: (count: 0)
core2#
Source
(*,G)
Source Receiver
+ (S,G) (*,G) + (S,G) vpc-peer1 VPC Pri PIM-DR
vpc-peer2 VPC Sec Proxy-DR
core1 Anycast-RP
core2 Anycast-RP
vlan101 vlan112
po52
po2 po1
po51 po51
vlan200
po2
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 44
Scoping Multicast Group Ranges
You can specify multiple RPs
‒ Static RP configuration always overrides dynamically learned RP information
Two options for specifying groups to map to each RP
‒ ―Inline‖ group-lists
‒ Specify route-map containing groups
RP used for particular group based on longest-match mask length
Highest RP IP address used for tie-breaks
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 45
“Inline” RP Group-Lists
ip pim rp-address <ip> group-list <ip>/<mask>
Useful if group ranges are contiguous and relatively simple
You can specify multiple lines for the same RP ‒ ip pim rp-address 100.100.100.100 group-list 239.1.2.1/32
‒ ip pim rp-address 100.100.100.100 group-list 239.1.3.1/32
‒ ip pim rp-address 100.100.100.100 group-list 239.1.5.1/32
‒ ip pim rp-address 100.100.100.100 group-list 239.1.6.1/32
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 46
Route-Maps for RP Scoping
ip pim rp-address <ip> route-map <name>
Useful for more complex scoping configurations
NOTE: As of 4.2(6)/5.0(3), ―deny‖ semantics are ignored in RP-scoping route-maps
‒ Can make non-contiguous scoping difficult
Interim solution: use static route to Null0 and define a ―blackhole RP‖ for the unneeded groups
‒ ip pim rp-address 100.100.100.100 route-map real-rp
‒ ip pim rp-address 255.255.255.254 route-map blackhole-rp
‒ route-map blackhole-rp permit 10
‒ match ip multicast group 239.1.4.1/32
‒ route-map blackhole-rp permit 20
‒ match ip multicast group 239.1.1.1/32
‒ route-map real-rp permit 10
‒ match ip multicast group 239.1.0.0/16
‒ ip route 255.255.255.254/32 Null0
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 47
RP Scoping and Source State on First-Hop
Router RP scoping does not prevent state creation on the first-hop router
To completely prevent state creation, use a RACL denying the groups in
question
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 48
Controlling (S,G) Expiration Time
Default (S,G) expiration time is 3 minutes
―Intermittent sources‖ in PIM-SM can cause latency/loss issues
Change expiration time with: ‒ ip pim sg-expiry-timer <sec> [sg-list <route-map>]
Route-map defines list of (S,G) entries to which the timer applies
Technically only needed on last-hop routers ‒ PIM joins keep state alive on upstream routers
7010-1# sh run pim | in sg
ip pim sg-expiry-timer 36000 sg-list sg-expiry
7010-1# sh route-map sg-expiry
route-map sg-expiry, permit, sequence 10
Match clauses:
ip multicast: group 239.1.2.0/23
Set clauses:
7010-1# sh ip pim route | eg -v \* | in expires
(10.200.200.3/32, 239.1.2.1/32), expires 09:59:25
(10.200.200.4/32, 239.1.3.1/32), expires 09:59:25
(10.200.200.5/32, 239.1.4.1/32), expires 00:02:25
(10.200.200.6/32, 239.1.5.1/32), expires 00:02:25
7010-1#
36000 seconds = 10 hours
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 49
PIM Join/Prune Policies
Define route-map to control PIM join/prune policy
‒ ―Permit‖ or ―deny‖ stanzas define for which groups PIM join/prunes are processed
‒ Implicit deny for unmatched groups
Use ―ip pim jp-policy‖ to apply to interface
‒ Can apply inbound, outbound, or both
Example: ‒ route-map pim-policy deny 10
‒ match ip multicast group 239.1.18.1/32
‒ route-map pim-policy permit 20
‒ match ip multicast group 224.0.0.0/4
‒ interface port-channel300
‒ ip pim jp-policy pim-policy in
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 50
“Multicast Boundary” – Combining Control-Plane Policies and
ACLs
NX-OS does not provide ―ip multicast-boundary‖ command
Each control-plane protocol controlled independently using policy configuration
Data-plane traffic controlled using IP ACLs
Example: prevent PIM, Auto-RP/BSR, and data-plane traffic from entering an interface
7010-1# sh run int po300
interface port-channel300
ip access-group no-mcast-data in
ip address 10.18.0.2/30
ip ospf network point-to-point
ip router ospf 10 area 0.0.0.0
ip pim sparse-mode
ip pim border
ip pim jp-policy pim-policy in
7010-1# sh route-map pim-policy
route-map pim-policy, deny, sequence 10
Match clauses:
ip multicast: group 224.0.0.0/4
Set clauses:
Limits data-plane traffic
Limits PIM joins
Limits BSR/Auto-RP
7010-1# sh ip access no-mcast-data
IP access list no-mcast-data
10 permit ip any 224.0.0.0/24
20 deny ip any 224.0.0.0/4
7010-1#
Nexus 7000 Platform Specifics
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
I/O Module
Supervisor
NX-OS and Nexus 7000 Multicast Routing
Architecture
• Push routes to platform
• Route download
• Translate routes to hardware format
• Program hardware forwarding and
replication engines
RPF updates
Multicast Routing Information Base (mRIB)
m4RIB m6RIB
FIB Manager
Forwarding Hardware
mFDM
uRIB
PIM MSDP IGMP PIM6 ICMPv6 / MLD
Add (*,G) & (S,G) from reports Add (S,G) from SAs Add (*,G) & (S,G) from Join/Prune
& Register/Assert Add (*,G) & (S,G) from reports
• Add routes, OIFs
• Update when RPF changes
52
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Where Are Multicast Routes Stored?
sh ip pim route sh ip igmp route [sh ip igmp groups] sh ip igmp snooping groups sh ip msdp route [sh ip msdp sa-cache] etc.
sh routing ip multicast [sh ip mroute]
sh forwarding distribution ip multicast route sh forwarding distribution ip igmp snooping
sh forwarding ip multicast route
sh system internal forwarding ip multicast route sh system internal ip igmp snooping
ADJ Table MET
MAC Table FIB TCAM
I/O Module
Supervisor Engine
PIM MSDP
MRIB
State Database (PSS)
IGMP
URIB
STP
MFDM
PIXMC
PIXM
IP FIB
Other HW
Hardware Drivers
L2MCAST
53
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 54
Nexus 7000 Hardware IP Multicast
Fully distributed Layer 2 multicast hardware switching
Fully distributed Layer 3 IPv4 multicast hardware switching
(S,G), (*,G), and (*,G/m) mroute forwarding in hardware
Distributed Layer 2 and Layer 3 multicast packet replication using egress
replication
Up to 8 Bidir RPs per VRF
IGMPv2/IGMPv3 snooping with IP-based traffic constraint
VRF-lite for multicast
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Multicast with vPC
vPC supports PIM-SM only
vPC uses CFS to sync IGMP state
For sources in vPC domain, both vPC peers are
forwarders
‒ Duplicates avoided via vPC loop-avoidance logic
For sources in Layer 3 cloud, unicast best metric
determines active forwarder (vPC operational primary in
case of tie)
‒ CFS used to negotiate active forwarder role on per-source
basis
Source
Receivers Source
55
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 56
Multicast Software Architecture with VDCs
Core multicast services always present in each VDC
‒ MRIB, MFDM, IGMP
PIM/MSDP spawned as configured on per-VDC basis
Other lower-level processes run in global space on Supervisor Engine and I/O modules
‒ PIXM on Supervisor Engine, IP FIB, ASIC drivers on I/O modules
Be aware of other global services that affect multicast, such as hardware rate limiters and CoPP
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
VDC1 VDC2 VDC3
Multicast Software Architecture with VDCs
I/O Module I/O Module I/O Module
Hardware
IP FIB
Hardware Hardware
IP FIB IP FIB
Supervisor Engine
PIM etc.
MRIB
PIM etc.
MRIB
PIM etc.
MRIB
MFDM MFDM MFDM
57
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 58
BFD for PIM
NX-OS version 5.0(2a) introduces BFD support
PIM process can be BFD client
Enable globally for PIM, disable per interface if desired
‒ ip pim bfd (global)
‒ ip pim bfd-instance disable (interface)
7010-1# sh run pim | in bfd
ip pim bfd
7010-1# sh ip pim neighbor vlan 102
PIM Neighbor Status for VRF "default"
Neighbor Interface Uptime Expires DR Bidir- BFD
Priority Capable State
10.100.102.3 Vlan102 09:42:38 00:01:19 1 yes Up
7010-1# sh bfd neighbor int vlan 102 detail | in pim
Registered protocols: hsrp_engine pim
7010-1#
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 59
Clearing Mroutes
What happens when you ―clear ip mroute‖?
NOT what happens in traditional Cisco IOS router
‒ Cisco IOS stores multicast routing table in monolithic data structure
In NX-OS, clear ip mroute == clear routing ip multicast
In other words, command removes routes from the MRIB
MRIB notifies MFDM, MFDM removes route, notifies MFIB (IPFIB) on I/O modules, MFIB removes route from hardware
MRIB immediately requests client processes (PIM, IGMP, MSDP) to repopulate the MRIB
MRIB adds routes back, notifies MFDM, MFDM notifies MFIB, MFIB reprograms hardware
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 60
Clearing a Route from the MRIB 7010-1# clear ip mroute 239.1.18.1
7010-1# 2010 Jul 29 15:53:59.332105 mrib: [4600] (default-base) Schedule route removal from mrib
2010 Jul 29 15:53:59.332191 mrib: [4600] (default-base) Route removed from mrib
2010 Jul 29 15:53:59.332235 mrib: [4600] (default-base) Schedule route removal from mrib
2010 Jul 29 15:53:59.332270 mrib: [4600] (default-base) Route removed from mrib
2010 Jul 29 15:53:59.332973 igmp: Received repopulate route notification for VRF default(1)
2010 Jul 29 15:53:59.333058 pim: Received repopulate mroute notification from MRIB for VRF default
2010 Jul 29 15:53:59.334172 igmp: Processing repopulate route request for igmp mpib, for VRF default (*, 239.1.18.1/32)
2010 Jul 29 15:53:59.334291 pim: repopulate (*, 239.1.18.1/32)
<etc.>
debug ip pim internal debug ip igmp internal debug ip mrouting summary
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 61
Clearing PIM Routes
Use clear ip pim route to remove route entries from PIM
Only PIM routes created by periodic PIM join messages removed
PIM routes created to trigger upstream joins not removed
‒ Example: IGMP join from directly connected receiver causes MRIB to notify PIM to create a
PIM route
If PIM route is ―mixed‖ (e.g., mroute has both PIM and IGMP OIFs), PIM OIFs
removed but route remains in PIM database
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 62
Clearing Data-Created Mroutes
Use clear ip mroute data-driven to remove route entries created by netstack
‒ E.g., (S,G) entries created on FHR
Supported in NX-OS 4.2(6) and 5.1(1) and later
Nexus 5000 Platform Specifics
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Multicast Enhancement in Nexus 5500
Increased IGMP groups to 4K groups
Lower latency ~2.1-2.2us
Better throughput and latency with complex traffic
pattern with more multicast VOQ
Improved multicast load sharing over PortChannel
Drop multicast traffic for congested egress ports
Supports PIM BiDir in vPC
64
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Multicast with vPC
vPC supports PIM-SM, PIM-SSM and PIM BiDir
vPC uses CFS to sync IGMP state
For sources in vPC domain, both vPC peers are
forwarders
‒ Duplicates avoided via vPC loop-avoidance logic
For sources in Layer 3 cloud, unicast best metric
determines active forwarder (vPC operational primary in
case of tie)
‒ CFS used to negotiate active forwarder role on per-source
basis
Source
Receivers Source
65
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Multicast Superframing
Multicast frame sharing same fanout are superframed
automatically
When the switch fabric access grant is received the
multicast packets in VOQ that has same fanout as the
first packet in the queue will be packed and sent to
egress ports within one scheduling cycle
Superframing improves throughput and reduce latency
Up to 10KB per superframe.
No waiting period. Only the complete frames in the queue can
be packed.
Superframing in on by default for both unicast and
multicast. Hardware always packs the frames
whenever it is possible
66
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Multicast Optimization Configuration Multicast optimization is turned on by default for ―class-default‖. It means all multi-
destination traffic will be assigned to multicast VOQ according to their fanout
Multi-destination traffic include
‒IP multicast
‒Unknown unicast flooding
‒Broadcast traffic
‒L2 multicast traffic
Multicast optimization can only be turned on for one system class.
8 multicast VOQ reserved for QoS queuing. The rest of 120 queues for multicast
optimization
N5k(config-cmap-qos)# policy-map type qos Mcast_optimize
N5k(config-pmap-qos)# class type qos class-ip-multicast
N5k(config-pmap-c-qos)# set qos-group 2
N5k(config-pmap-c-qos)# exit
N5k(config-pmap-qos)# class type network-qos IP_mcast
N5k(config-cmap-nq)# match qos-group 2
N5k(config-cmap-nq)# policy-map type network-qos Mcast_optimize
N5k(config-pmap-nq)# class type network-qos IP_mcast
N5k(config-pmap-nq-c)# multicast-optimize
N5k(config-pmap-nq-c)# queue-limit 170000
67
Nexus 3000 Platform Specifics
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Line rate multicast forwarding and replication for all ports and
all frame size with features on
Ultra low latency even with mesh traffic pattern
4K IGMP snooping entries
4K IP multicast routes
Multicast Performance/Scalability
69
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 70
Reduce Burstiness of IGMP with MRT
IGMP packets are rate limited to 400pps at hardware as of 5.0(3)U1
Host intended to join the group has to response within the time specified
by the MRT(Maximum Response Time) in query message.
Reduce burstiness of IGMP message with larger MRT value. Hosts will
delay the IGMP for a random amount of time which is less than MRT
Recommend to increase MRT value with large number of IGMP join
Default MRT value 10s. Configure at IGMP querier or IGMP snooping
querier
N3k-1(config)#interface vlan 101
N3k-1(config-vlan)# ip igmmp query-max-response-time 25
N3k-1(config-vlan)# ip igmp last-member-query-response-time 25
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 71
PIM Features
Support PIM-SM and PIM-SSM
RP selection: static RP, BSR, Auto-RP
Anycast RP with PIM. Anycast RP with MSDP
VRF-aware
PIM policies: Neighbor policy, Join/prune policy, registration policy
No Support for PIM-BiDir
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 72
L3 Multicast Table Size Host Table and LPM(Longest Prefix Match) in Nexus 3064 routing engine
Host Table stores ARP entries, host routes (/32) and multicast routes-(*,G) and (S,G). One
multicast route((*,G) or (S,G)) consume two HW entries
LPM table stores summarized routes, ARP and host routes
CLI to increase IP multicast region to support up to 4000 mroutes
Default Table Partition Increased multicast Table size
IP Multicast
(2k routes
4K HW entries)
Host Table
(8K entries)
ARP /32 routes 4k entries ARP and
routes
LPM Table
(8K entries)
IP Multicast
(4k routes
8K HW entries)
ARP and
routes
Host Table
(8K entries)
LPM Table
(8K entries)
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 73
L3 Multicast Table Size By default Nexus 3064 supports 2K multicast routes. The following CLI increases that to 4K at the
expense of ARP and routing table size. The CLI will fail if there are pre-existing ARP entries or /32
host route. Recommend to configure the CLI at the beginning.
N3k-1(config)# hardware profile multicast max-limit 4000
N3k-1(config)#
• Hardware space allocated for IP multicast are reserved for IP multicast and can’t be shared with ARP
• Check the hardware resource utilization
N3k-1# sh hardware profile status
Reserved LPM Entries = 1024.
Reserved Host Entries = 96.
Reserved Mcast Entries = 4000.
Used LPM Entries = 3.
Used Host Entries in LPM = 0.
Used Mcast Entries = 3500.
Used Host Entries in Host = 15.
N3k-1#
Recommended Practices with NX-OS
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
This presentation will use
the topology at the right to
illustrate designs and use
cases for multicast.
Typical 3-layer hierarchical
network design
Classical Ethernet
Topologies
Multicast and Places in the Network
Layer 3
Layer 2
75
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Control Plane Policing (CoPP) enabled by default
May wish to tune for multicast
Multi-step process
‒ Monitor existing CoPP policy
‒ Adjust specific attributes (PIM, IGMP, MSDP, etc)
‒ Monitor and tune as needed
As much science as an art – network requirements change!
Control Plane Policing - CoPP
76
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Show policy-map interface control-plane
<snip>
set cos 7
police cir 39600 kbps , bc 250 ms
module 4 :
conformed 1187892 bytes; action: transmit
violated 0 bytes; action: drop
*NOTE* CoPP is enforced per module on Nexus 7000
Tune policy
N7K-1# copp copy profile strict prefix tuned
N7K-1(config)# policy-map type control-plane tuned-copp-policy-strict
N7K-1(config-pmap)# class tuned-copp-class-critical
N7K-1(config-pmap-c)# police cir 64000 kbps bc 250 ms conform transmit violate drop
N7K-1(config-pmap-c)# end
N7K-1# show run copp
N7K-1# show policy-map int control-plane
<snip>
set cos 7
police cir 64000 kbps , bc 250 ms
module 4 :
conformed 172 bytes; action: transmit
violated 0 bytes; action: drop
CoPP - Example
77
Look for increments
here
Monitor for more
increments
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Lightweight hello protocol over different data protocols
‒IPv4, IPv6, MPLS
Used for fast (often sub-second) communication failure detection
Single, common & standardized mechanism
Independent of specific routing, FHRP and other client protocols using BFD
Any ―interested application‖ (OSPF, BGP, EIGRP, PIM, etc.) registers with BFD and
is notified as soon as BFD recognizes a neighbor loss
UDP port 3784 / 3785 (for echo)
RFC 5880
Available on Nexus 7000 and 3000
Bidirectional Forwarding Detection (BFD)
78
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Enable BFD
Enable BFD for PIM
Verify BFD Configuration
BFD - Example
79
Config
N7K-1-Core1# (config) feature bfd
N7K-1-Core1# (config) pim bfd
N7K-1-Core1# (config) end
N7K-1-Core1# show bfd neighbors application pim
OurAddr NeighAddr LD/RD RH/RS Holdown(mult) State Int Vrf
10.1.0.29 10.1.0.30 1124073473/0 Down N/A(3) Down Eth4/7 default
10.1.0.1 10.1.0.2 1124073474/1090519042 Up 4954(3) Up Eth4/1 default
10.1.0.5 10.1.0.6 1124073475/1107296259 Up 4520(3) Up Eth4/2 default
N7K-1-Core1#
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 80
Recommendations for RL/CoPP for
Multicast On first-hop router:
‒ Add CoPP class(es) for multicast data groups – match all data groups, or define multiple classes for different group ranges (critical groups, important groups, best-effort groups)
‒ Tweak directly-connected rate limiter – may want to increase rate, understanding potential implications
On last-hop router:
‒ Add CoPP class(es) for multicast data groups
‒ Tweak local-groups rate limiter
On RP:
‒ Consider creating separate PIM register class
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Enable PIM BFD
‒ Don’t forget to disable ip redirects
‒ For non-BFD devices, timers may be tuned – test impact!
Tune Control Plane Policing Policy
Typical RP protocols can be used
Consider anycast-rp commands (RFC 4610)
Multicast in L3 Core
81
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Enable PIM BFD
‒ Don’t forget to disable ip redirects
‒ For non-BFD devices, timers may be tuned – test impact!
Tune Control Plane Policing Policy
Typical RP protocols can be used
Consider anycast-rp commands (RFC 4610)
Multicast at L2/L3 Boundary
82
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 83
NX-OS versus Cisco IOS Multicast
Configuration No need to enable multicast globally
Must enable each feature (PIM, MSDP) explicitly before configuring
No support for PIM-DM
No support for PIM sparse-dense mode
‒ Use Auto-RP listener/forwarder configuration instead
MSDP SA cache enabled by default (non configurable)
Static RP configuration based on longest-match prefix length
Support for PIM-based Anycast-RP (RFC 4610)
Uses multicast multipath RPF by default for ECMP prefixes (non configurable)
Supports PIM neighbor authentication
No direct support for multicast boundary
‒ Use protocol-specific filtering policies and/or data-plane RACLs
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public 84
NX-OS/Nexus 7000 versus Cisco
IOS/Catalyst 6500 Multicast Operation Supports egress local Layer 3 multicast replication only
Clearing mroute state behaves differently
‒ ―clear ip mroute‖ clears state from MRIB down to hardware – does not clear protocol state – use clear ip pim route, clear ip igmp route, etc.
Software replication disabled by default:
‒ Controls whether software routes initial leaked multicast packets – ip routing multicast software-replicate
Use show ip mroute summary count instead of show ip mroute count
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Migration from IOS PIM Auto-RP
IOS
interface Loopback10 ip address 172.16.1.1
255.255.255.255
ip pim sparse-mode
ip pim send-rp-announce Loopback10 scope 32
ip pim send-rp-discovery Loopback10 scope 32
ip pim autorp listener
NX-OS
interface loopback10 ip address 172.16.1.1/32
ip pim sparse-mode
ip pim auto-rp rp-candidate loopback10 group-
list 224.0.0.0/4
ip pim auto-rp mapping-agent loopback10
ip pim auto-rp forward listen
or
ip pim send-rp-announce loopback10 group-list
224.0.0.0/4
ip pim send-rp-discovery loopback10
ip pim auto-rp forward listen
85
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Migration from IOS PIM Anycast RP – RFC 4610
Cisco IOS Software does not have the
ability to enable the PIM Anycast RP
feature.
NX-OS
interface loopback0 ip address 192.168.10.1/32
ip pim sparse-mode
interface loopback10
description Anycast-RP-Address
ip address 172.16.1.1/32
ip pim sparse-mode
ip pim anycast-rp 172.16.1.1 192.168.10.1
ip pim anycast-rp 172.16.1.1 192.168.10.2
86
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Migration from IOS Configuring PIM in a non-default VRF Instance
IOS
ip vrf production ip multicast-routing vrf
production
interface Loopback10
ip vrf forwarding production
ip address 172.16.1.1 255.255.255.255
ip pim sparse-mode
interface TenGigabitEthernet1/1
ip vrf forwarding production
ip address 192.168.10.1 255.255.255.0
ip pim sparse-mode
ip pim vrf production rp-address 172.16.1.1
NX-OS
vrf context production ip pim rp-address
172.16.1.1 group-list 224.0.0.0/4
interface loopback10
vrf member production
ip address 172.16.1.1/32
interface Ethernet1/1
vrf member production
ip address 192.168.10.1/24
ip pim sparse-mode
87
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Migration from IOS Configuring MSDP with Anycast-RP
IOS
interface Loopback0 description MSDP Peer
Address
ip address 192.168.1.1 255.255.255.255
interface Loopback10
description PIM RP Address
ip address 1.1.1.1 255.255.255.255
ip pim rp-address 1.1.1.1
ip msdp peer 192.168.2.1 connect-source
Loopback0
ip msdp cache-sa-state
NX-OS
feature msdp
interface loopback0 description MSDP Peer
Address
ip address 192.168.1.1/32
interface loopback10
description PIM RP Address
ip address 1.1.1.1/32
ip pim rp-address 1.1.1.1 group-list
224.0.0.0/4
ip msdp peer 192.168.2.1 connect-source
loopback0
88
Summary
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Summary
NX-OS supports a wide variety of multicast technologies across the
different family members
Common architecture with small platform specific components
Flexible options to address different needs in the network
90
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Complete Your Online
Session Evaluation Give us your feedback and you could
win fabulous prizes.
Winners announced daily.
Receive 20 Passport points for each
session evaluation you complete.
Complete your session evaluation
online now (open a browser through
our wireless network to access our
portal) or visit one of the Internet
stations throughout the Convention
Center.
Don’t forget to activate your
Cisco Live Virtual account for access to
all session material, communities, and
on-demand and live activities throughout
the year. Activate your account at the
Cisco booth in the World of Solutions or visit
www.ciscolive.com.
91
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Final Thoughts
Get hands-on experience with the Walk-in Labs located in World of
Solutions, booth 1042
Come see demos of many key solutions and products in the main Cisco
booth 2924
Visit www.ciscoLive365.com after the event for updated PDFs, on-
demand session videos, networking, and more!
Follow Cisco Live! using social media:
‒ Facebook: https://www.facebook.com/ciscoliveus
‒ Twitter: https://twitter.com/#!/CiscoLive
‒ LinkedIn Group: http://linkd.in/CiscoLI
92
© 2012 Cisco and/or its affiliates. All rights reserved. BRKIPM-3062 Cisco Public
Reference Slides
94