unified fabric - ciscosimplified management with scale architectural flexibility active-active...

Post on 27-Jun-2020

4 Views

Category:

Documents

0 Downloads

Preview:

Click to see full reader

TRANSCRIPT

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 1

Unified Fabric Maurizio Portolani

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2

• Introduction

• Spanning Tree and vPC

• Fabric Extender

• vPC and FEX design best practice

• FabricPath

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3

Scale-up and Scale-out a Key Requirement

TOP CAREABOUTS IMPACT FABRIC

General Purpose

High Frequency Trading

and High Performance

Computing

Cloud and Service Provider

Web 2.0 and Big Data

• Large Scale L2/L3 Fabric

• Multi-tenancy

• Simplified Management

• Security

• Convergence

• L2/L3 Fabric Scale

• Operational Continuity

• Simplified Management

• Server Virtualization

• Convergence

• Multicast at Scale

• Low Latency

• East/West traffic

• L3 Fabric at Scale (10,000+)

• Open API

• Auto Provisioning and Monitoring

Need For Architectural Flexibility

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4

CONVERGENCE

SCALE

INTELLIGENCE

Foundational Technologies

DCB/FCoE

VDC

FEX

Architecture

FabricPath

vPC

OTV

LISP

Consolidated I/O

Virtualizes the Switch

Simplified Management

with Scale

Architectural Flexibility

Active-Active Uplinks

Workload Mobility

Scalability and Mobility

Deployment Flexibility Unified Ports

IO Accelerator Replication and Back up

SME/DMM Compliance and Workload Mobility

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5

• Wire once and walk away

• Lower complexity

• Does not disrupt existing Infrastructure

• Eliminate parallel networks – no stranded assets, less space, less heat, less cooling

• Simpler cabling – easier installation, better airflow

• Interoperates with existing LAN and SAN

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6

Virtualising the Network Port

LAN LAN

Switch port extended over

Fabric Extender

Lo

gic

al S

wit

ch

Switch

Switch

Legacy multi-tier architecture FEX architecture

Switch

FEX

Collapse network tiers, fewer network management points

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7

Where Is the Edge?

NIC

PCI-E Bus

Operating System and

Device Drivers

FC 3/11

HBA

Edge of the Network and Fabric

Eth 2/12

pNIC

PCI-E Bus

Hypervisor provides virtualization of PCI-E

resources

FC 3/11

HBA

Eth 2/12

Edge of the

Fabric

VMF

S

SCSI

VNIC

VETH

Still 2 PCI Addresses

on the BUS

PCI-E Bus

Hypervisor provides virtualization of PCI-

E resources

Edge of the

Fabric

VMFS

SCSI

PCIe

Eth

ern

et

Fib

re C

han

nel

10G

bE

10G

bE

Link

Eth 2/12

vFC 3

Converged Network Adapter provides

virtualization of the

physical Media

VNIC

VETH

PCI-E Bus

Hypervisor provides

virtualization of PCI-E resources

Edge of the Fabric

V

M

FS

S

C

SI

veth 1

vFC 4

802.1BR

Eth

1

FC

2

Eth

3

FC

4

Eth

126

vFC 2

vFC 3

vFC 126

P

as

s

T

hr

u

VNIC

VETH

SR-IOV adapter

provides multiple

PCIe resources

Compute and Fabric Edge are Merging

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8

Some classic Layer 2 Limitations

• Local problems have network-wide impact, troubleshooting is difficult

• Tree topology provides limited bandwidth

• Tree topology introduces sub-optimal paths

• Each Network device is another management point

A B

STP convergence is disruptive

MAC address tables don’t scale

Host flooding impacts the whole network

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9

Fabric simplification

With FabricPath:

• Traffic is routed inside the fabric local problems have local impact

• L2 can be extended without risk

FabricPath

VLAN X

VLAN Y

VLAN Z

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10

Larger Distributed Topologies

Server, Storage, Application and Facilities are driving Layer 2 Scalability requirements

Server Virtualization and Clustering driving the need for every / any VLAN everywhere based design

Facilities requirements defining the network topology

• VM requirements along with Data Storage growth mandating a need for more efficient and pervasive network based storage

Technology changes will impact any cabling plant design

Migration to 10GE as the default LoM technology

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 11

L2 Domain Elasticity: vPC, FabricPath/TRILL

OTV LAN extensions

OTV

Device Virtualization: VDCs,

VRF enhancements

MPLS VPN

Location of compute resources is transparent to the user

VN-link

notifications

VM-awareness: VN-link

Port Profiles

OTV

OTV

OTV

IP Mobility: LISP

Multi-tenancy/segmentation: Segment-ID, VXLAN, FabricPath and OTV

Virtualized Workload Mobility Connecting Virtualized Data Centers

OTV

OTV

OTV

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12

Evolution of Enterprise Environments

L3

L2

L3

L2

L3 Cloud

From STP ----------------------------------------------- to FabricPath

From 2-way redundancy ----------------------------- to N-way redundancy

From Limited L2 paths ------------------------------- to L2 ECMP

From Classic learning --------------------------------- to conversational learning

From rigid topologies --------------------------------- to highly flexible topologies

From hard pod boundaries -------------------------- to segment overlay extensions

From disparate fabrics ------------------------------- to converged fabric

From virtualization unaware fabric ----------------- to virtualization aware fabric

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13

Evolution of SPDC Environments

From Restricted VLAN/L2 Scale ----------- to highly scalable VLAN/MAC Space

From Classic L4-7 Scale/Mobility ---------- Physical/Virtual Scale and Service Mobility

From VLAN-Based Multi-tenancy ---------- Tenant ID Based environments

From GE nodes ------------------------------ to 10G nodes pushing 40G uplink adoption

From limited Hypervisors offerings ------- “open” primarily and other as needed

From Stitch-together Orchestration ------ Integrated Process Flow Driven Orchestration

L3 Cloud

L3

L3

L2

L3 L3 Cloud

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14

• Introduction

• Spanning Tree and vPC

• Fabric Extender

• vPC and FEX design best practice

• FabricPath

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 15

! Enable vpc on the switch

dc11-5020-1(config)# feature vpc

! Check the feature status

dc11-5020-1(config)# show feature | include vpc

vpc 1 enabled

• vPC allows a single device to use a port channel across two neighbour switches (vPC peers)

• Eliminate STP blocked ports

• Layer 2 port channel only

• Provide fast convergence upon link/device failure

• Available on Nexus 3000, 5000/5500 and 7000*

Virtual Port Channel - vPC Multi-chassis Etherchannel (MCEC)

MCEC

vPC Peers

MCEC

vPC Peers

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 16

L

2

L

3

Core

Aggregati

on

Access

WAN

Core FC

Co

re

Edg

e

LAN and SAN utilize different High Availability Models

SAN is dual fabric, LAN is fully meshed fabric

vPC enables ‘both’ architectures at the edge (single device models not acceptable to SAN customers)

Why vPC and not VSS or Stackwise Co-existence of LAN and SAN

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 17

• Do not disable STP !!

• Configure aggregation vPC peers as root and secondary root

If vPC peer-switch is implemented, both vPC peers will behave a single STP root

• Align STP primary root, HSRP active router and PIM DR with vPC primary peer

• BA is enabled by default on vPC peer-link

• Do not enable Loopguard and BA on vPC (disabled by default)

• Enable STP port type “edge” and port type “edge trunk” on host ports

• Enable STP BPDU-guard globally

• Disable STP channel-misconfig guard if supported by access switches

Spanning Tree Recommendations STP Best Practices

BPDU-guard BPDU-guard

Port Type

Edge / Edge Trunk

routing peer agg1b agg1a

Acc1 Acc2

Disable STP channel-

misconfig guard

vPC primary

VLAN 1-4094 root

MST 0 – 3 root

HSRP Active

PIM DR

vPC primary STP

VLAN 1-4094 sec root

MST 0 – 3 sec root

HSRP Standby

vPC_PL

vPC_PKL

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 18

vPC maintains layer 2 synchronization between peers ‘not’ Layer 3

Don’t use L2 port channel to attach routers to a vPC domain

If both routed and bridged traffic is required:

Individual L3 links for routed traffic

L2 port-channel (vPC) for bridged traffic

vPC Interaction with Layer 3 Do not create an L3 peer over a vPC interface

ECMP LInks

L3

L2

ECMP LInks

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 19

• HSRP active process communicates the active MAC to its neighbour

• Only the HSRP active process responds to ARP requests

• HSRP active MAC is populated into the L3 hardware forwarding tables, creating a local forwarding capability on the HSRP standby device

• No need to configure aggressive FHRP hello timers as both switches are active

‘peer-gateway’ command allows a vPC peer to respond both the the HSRP virtual and the real MAC address of both itself and it’s peer

dca-n7k1-vdc2(config-vpc-domain)# peer-gateway

Note:

------:: Disable IP redirects on all interface-vlans of this vPC domain

for correct operation of this feature ::------

vPC Interaction with Layer 3 FHRP

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 20

N5k(config-vpc-domain)# delay restore ?

<1-3600> Delay in bringing up vPC links (in seconds)

N5k(config-vpc-domain)# delay restore 360 Delay activating vPC interfaces

vPC interaction with Routing convergence on system restart

After a vPC device reloads and comes back up routing protocol needs time to reconverge

vPCs may blackhole routed traffic from access to core until layer 3 connectivity is reestablished

vPC Delay Restore

On system restart delays bringing up the vPC interfaces for ‘n’ seconds

Like HSRP preempt delay it needs to be tuned for your network

vPC and Layer 3 InteractionsvPC Delay Restore

vPC

Primary

vPC

Secondary

L3

L2

OSPF

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 21

HSRP Active

HSRP Standby

L2

L3

RMAC-A RMAC-B

N5k-1 N5k-2

Some devices, such as NetApp filer, could send packets to interface MAC rather than the virtual MAC of FHRP protocol

Such packets will be forwarded over peer-link

The “peer-gateway” will address this issue

L3-N5548-1(config-vpc-domain)# peer-gateway

vPC and Layer 3 InteractionsPeer-Gateway

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 22

Parent Switch vPC Peer-link (10 GE Only) VPC Interfaces

N7K-M132XP-12

✓ ✓

N7K-M132XP-12L

✓ ✓

N7K-M108X2-12L

✓ ✓

N7K-F132XP-15

✓ ✓

F2 – 48 x 10G

✓ ✓

M2 40G & 100G

Future Future

For Your Reference

vPC Supported Hardware Nexus 7000

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23

vPC Supported Hardware VDC Combinations with F1/F2/M1 Module

F2

Only

VDC

M1-F1

Mix VDC

F2

Only

VDC

M1-F1-F2

Mix VDC

vPC Peer Link

F1 Only VDC – F1 ports used for the peer-link

F2 Only VDC – F2 ports used for the peer-link

M1/F1 Mixed VDC – F1 ports (16K MAC addresses)

M1/F1 Mixed VDC – M1 ports (128K MAC addresses)

F1

Only

VDC

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 24

Switch vPC Peer Link vPC Interfaces

Nexus 5020

✓ ✓

Nexus 5010

✓ ✓

Nexus

5548P/UP

✓ ✓

Nexus

5596UP

✓ ✓

For Your Reference

vPC Supported Hardware Nexus 5000/5500

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 25

vPC – Virtual Port Channel Nexus 3000

vPC is supported on the Nexus 3000 as of

5.0(3)U2(1)

The maximum number of vPC configurable on

the Cisco Nexus 3000 Series Switches is 64

For Your Reference

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 26

• Introduction

• Spanning Tree and vPC

• Fabric Extender

• vPC and FEX design best practice

• FabricPath

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 27

Distributed High

Density Edge

Switching System

(up to 4096 virtual

Ethernet interfaces)

+

Cisco Nexus® 2000 FEX

Cisco Nexus® 5500

Cisco Nexus® 2000 FEX

Cisco Nexus® 7000

+

Cisco FEX-link: Virtualized Access Switch Nexus 2000 Fabric Extender

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 28

• De-Coupling of the Layer 1 and Layer 2 Topologies

• Simplified Management Model, plug and play provisioning, centralized configuration

• Line Card Portability (N2K supported with Multiple Parent Switches – N5K, 6100, N7K)

• Unified access for any server (100M1GE10GE FCoE): Scalable Ethernet, HPC, unified fabric or virtualization deployment

Virtualized

Switch

. . .

Nexus 2000: Virtualized Access Switch Changing the device paradigm

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 29

Parent Switch FEX Supported

# FEX FEX HIF Capabilities

Nexus 5020

N2K-C2148T, N2K-C2248TP

N2K-C2248TP-E, N2K-C2224TP,

N2K-C2232PP, N2K-C2232TM,

B22-HP

12 STP Edge Ports

FCoE VF ports

Nexus 5010

N2K-C2148T, N2K-C2248TP

N2K-C2248TP-E, N2K-C2224TP,

N2K-C2232PP, N2K-C2232TM,

B22-HP

12 STP Edge Ports

FCoE VF ports

Nexus 5548P/UP

N2K-C2148T, N2K-C2248TP

N2K-C2248TP-E, N2K-C2224TP,

N2K-C2232PP, N2K-C2232TM,

B22-HP

24 – L2

8 – L3

STP Edge Ports

FCoE VF ports

FabricPath Edge Ports

CTS Edge Ports

Nexus 5596UP

N2K-C2148T, N2K-C2248TP

N2K-C2248TP-E, N2K-C2224TP,

N2K-C2232PP, N2K-C2232TM,

B22-HP

24 – L2

8 – L3

STP Edge Ports

FCoE VF ports

FabricPath Edge Ports

CTS Edge Ports

For Your Reference

Cisco Nexus 5000 Parent Switch Supported Nexus 2000

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 30

Parent Switch FEX Supported # FEX FEX HIF Capabilities

N7K-M132XP-12

N2K-C2248TP, N2K-C2224TP,

N2K-C2232PP, N2K-C2232TM* 32 - L2

32 - L3

STP Edge Ports

CTS Edge Ports

N7K-M132XP-12L

N2K-C2248TP, N2K-C2224TP,

N2K-C2232PP, N2K-C2232TM* 32 - L2

32 - L3

STP Edge Ports

CTS Edge Ports

N7K-M108X2-12L

N.A. (FEX not Supported)

N.A. N.A. (FEX not Supported)

N7K-F132XP-15

N.A. (FEX not Supported) N.A. N.A. (FEX not Supported)

F2 – 48 x 10G

N2K-C2248TP, N2K-C2224TP,

N2K-C2232PP, N2K-C2232TM* 32 - L2

32 - L3

STP Edge Ports, FabricPath Edge Ports

CTS Edge Ports, FCoE VF ports (CY12)

M2 40G & 100G

Future N.A. Future

For Your Reference

Cisco Nexus 7000 Parent Switch Supported Nexus 2000

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 31

N2224TP 24 Port 100/1000M Host

Interfaces

2 x 10G Uplinks

N2232TM 32 Port 1/10GBASE-T Host

Interfaces

8 x 10G Uplinks (Module)

N2148T 48 Port 1000M Host

Interfaces

4 x 10G Uplinks

N2248TP 48 Port 100/1000M Host

Interfaces

4 x 10G Uplinks

N2232PP 32 Port 1/10G FCoE Host

Interfaces

8 x 10G Uplinks

FET-10G Cost Effective Fabric

Extender Transceiver

N2248TP-E 48 Port 100/1000M Host

Interfaces

4 x 10G Uplinks

32MB Shared Buffer

B22HP 16 x 1/10G Host Interfaces

8 x 10G Uplinks

Cisco Nexus 2000Series Platform Overview

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 32

Virtualized Access Switch Fabric Extender Transceiver (FET)

• Cost-effective transceiver to interconnect Nexus 2000 and Nexus 5000 and 7000 parent switch (only supported on FEX Fabric interfaces)

• SFP+ form-factor

• Multimode fiber (MMF)

• FET with OM3 MMF can operate up to 100m

• FET with OM2 MMF can operate up to 20m

• FET with 62.5/125um MMF can operate up to 10m

• Approximately 1 watt (W) per transceiver

• Incompatible with SR optics

FET supported

only on Fabric

Interfaces

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 33

Bridges that support

Interface

Virtualization ports

must support VNTag

and the VIC protocol

NIV uplink ports

must connect to

an NIV capable

bridge or an NIV

Downlink

Hypervisor

NIV downlink

ports may be

connected to an

NIV uplink port,

bridge or NIC

NIV may be

cascaded

extending the

port extension

one additional

level

NIV downlink ports

are assigned a virtual

identifier (VIF) that

corresponds to a

virtual interface on

the bridge and is

used to forward

frames through NIV’s

LIF

VIF NIV capable

adapters may

extending the

port extension

VIF

The FEXLink Architecture provides the ability to extend the bridge (switch) interface to downstream devices

FEXLink associates the Logical Interface (LIF) to a Virtual Interface (VIF)

LIF

Nexus 2000 Fabric Extender FEX Link Architecture

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 34

2248TP-E

N5596-L3-2(config-fex)# hardware N2248TPE queue-limit 4000000 rx

N5596-L3-2(config-fex)# hardware N2248TPE queue-limit 4000000 tx

N5596-L3-2(config)#interface e110/1/1

N5596-L3-2(config-if)# hardware N2348TP queue-limit 4096000 tx

Optimized for BigData, distributed storage, distributed computing, market data, video editing (bursty applications)

Each HIF has 128KB dedicated buffer and each NIF has 64KB dedicated buffer. The remaining 26MB is all shared

Interface allocates buffer from its dedicated pool first and then get more buffer if needed from shared pool until queue limit is reached

Queue limit specify how many buffer can be allocated for each queue.

Tune ingress queue limit when the FEX uplink experience temporary congestion.

Tune egress queue limit for better burst absorption or scenario with many to one traffic pattern

Per FEX buffer tuning

Nexus 2248TP-E Better buffering and counters

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 35

Extends FEX connectivity into the partner blade chassis

Cisco Nexus 5000 Switch is a single management point for all the blade chassis I/O modules

End-to-end FCoE support

66% decrease in blade management points*

Blade & Rack networking consistency

Interoperable with Nexus 2000 Fabric Extenders in the same Nexus parent switch

HP provides tier 1-2 support. Additional support via Nexus 5000

Cisco Nexus

B22 Series

Blade FEX

* Assuming 10 HP Blade

Chassis, Includes OA Modules

10G Nexus Fabric Extender

[16x 1GE/10GE KR server facing ports]

[8x10GE SFP+ network facing ports]

Over subscription 2:1 to 16:1 depending on number of uplinks

Cisco Nexus B22 Series Fabric Extender FEX Connectivity for the Blade Server Ecosystem

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 36

3+3 C19 cords

Eight

interconnect

bays usable

for B22 FEX

– 10/1G &

FCoE

Hot-plug

redundant

fans

Network and serial

connectors

Enclosure links

Redundant

Onboard

Administrators

Cisco Nexus B22 Series Fabric Extender FEX Connectivity for HPC C7000 & C3000 Chassis

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 37

• Introduction

• Spanning Tree and vPC

• Fabric Extender

• vPC and FEX design best practice

• FabricPath

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 38

Redundancy model – Dual Switch with redundant fabric

Provides isolation for Storage topologies (SAN ‘A’ and ‘B’)

Port Channel and Pinning supported for Fabric Link

Redundancy model – Single switch with dual ‘supervisor’ for fabric, data control & management planes

No SAN ‘A’ and ‘B’ isolation

FEX-link & vPC Virtualized Access Switch Nexus 5000/5500 Topologies prior to 5.1(3)N1

vPC Supported with up to 2 x 8

links

Local Etherchannel with up to 8

links

FCoE Adapters supported on 10G

N2K interfaces

Straight Through Dual Homed

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 39

Redundancy model – Dual Switch (each switch supports redundant supervisors)

vPC Supported with NX-OS 5.2

Fabric links supported on N7K-M132XP-12, N7K-M132XP-12L & N7K-F248XP-25

Port Channel only supported for Fabric Links

FEX-link & vPC Virtualized Access Switch Nexus 7000 Topologies supported as of 5.2

vPC Supported with up to 2 x 8

links

Local Etherchannel with up to 8

links

Nexus 7000 – vPC Nexus 7000 – vPC

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 40

• How is this achieved?

• Configuration associates FCoE traffic to a specific fabric link

switchA(config)# fex 101

switchA(config-fex)# fcoe

switchB(config)# fex 101

CNA

Nexus 2000

Fabric

Extender (FEX)

FCoE

Nexus 5000

(San B) Nexus 5000

(San A)

FCoE

Storage

SAN B SAN A

Enhanced vPC Isolating SAN A and SAN B

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 41

• Introduction

• Spanning Tree and vPC

• Fabric Extender

• vPC and FEX design best practice

• FabricPath

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 42

“FabricPath brings Layer 3

routing benefits to flexible Layer 2

bridged Ethernet networks”

Easy

Configuration

Plug & Play

Provisioning

Flexibility

Multi-pathing

(ECMP)

Fast

Convergence

Highly Scalable

Switching Routing

FabricPath

Cisco FabricPath NX-OS Innovation Enhancing L2 with L3

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 43

CE Edge

Ports

FP Core

Ports

Spine

Switch

Leaf

Switch

Interface connected to traditional network device

Sends/receives traffic in standard 802.3 Ethernet frame format

Participates in STP domain

Forwarding based on MAC table

Classical Ethernet (CE)

S

10

S

20

S

30

S

40

S10

0

S20

0

S300

1/

1 1/

2

FabricPath (FP)

A B

Interface connected to another FabricPath device

Sends/receives traffic with FabricPath header

Does not run spanning tree

Does not perform MAC learning!

Exchanges topology info through L2 ISIS adjacency

Forwarding based on ‘Switch ID Table’

Cisco FabricPath Terminology

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 44

Classical Ethernet

(CE)

S

10

S

20

S

30

S

40

S10

0

S20

0 S30

0

1/

1

• The association MAC address/Switch ID is maintained at the edge

• Traffic is encapsulated across the Fabric S300: CE MAC

Address Table

MAC IF

B 1/2

… …

MAC IF

B 1/2

A S100

1/

2

S300:

FabricPath

Routing Table

Switc

h

IF

… …

S100 L1, L2, L3,

L4

FabricPath (FP)

Switch ID

space:

Routing

decisions are

made based

on the

FabricPath

routing table

MAC adress

space:

Switching

based on MAC

address tables

S100 S300 A

B

A B

Cisco FabricPath A New Data Plane

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 45

S100: CE MAC

Address Table

A

S

10

S

20

S

30

S

40

S10

0

S20

0 S300

FabricPath

B

1/

1

Classical

Ethernet

S300: CE MAC

Address Table

MAC IF

B 1/2

… …

S200: CE MAC

Address Table

MAC IF

… …

… …

S100 M A

B

Lookup B: Miss

Don’t learn

Lookup B: Miss

Flood

Lookup B: Hit

Learn source A

MAC IF

B 1/2

A S100

MAC IF

… …

… …

MAC IF

A 1/1

… …

1/

2

FabricPath Key Concept #1 Conversational MAC Learning

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 46

Classical

Ethernet Conversational

Learning

S100: CE MAC

Address Table

A

S

10

S

20

S

30

S

40

S10

0

S20

0

S300

FabricPath

B

1/

1 S300: CE MAC

Address Table

MAC IF

B 1/2

… …

S200: CE MAC

Address Table

MAC IF

… …

… …

MAC IF

B 1/2

A S100

MAC IF

… …

… …

MAC IF

A 1/1

… …

1/

2

S300:

FabricPath

Routing Table

Switc

h

IF

… …

S100 L1, L2, L3,

L4

S300 S100 B A

Lookup A: Hit

Send to S100 Lookup A: Hit

Learn source B

MAC IF

A 1/1

B S300

FabricPath Key Concept #1 Conversational MAC Learning

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 47

S1

00

S

10

S

20

S

30

S

40

S2

00

S3

00

FabricPath

• Describes shortest (best) paths to each Switch ID based on link metrics

• Equal-cost paths supported between FabricPath switches

FabricPath

Routing Table on

S100 Switc

h

IF

S10 L1

S20 L2

S30 L3

S40 L4

S200 L1, L2, L3,

L4

… …

S300 L1, L2, L3,

L4

One ‘best’ path

to S10 (via L1)

Four equal-cost

paths to S300

FabricPath Key Concept #2 It’s a Routed Network

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 48

S

10

S

20

S

30

S

40

• Multi-destination traffic constrained to loop-free trees touching all FabricPath switches

• Root switch elected for each multi-destination tree in the FabricPath domain

• Loop-free tree built from each Root assigned a network-wide identifier (Ftag)

• Support for multiple multi-destination trees provides multipathing for multi-destination traffic

Two multi-destination trees supported in NX-OS release 5.1

Root for

Tree 1

S100 S

20

Root for

Tree 2

S

10 S200

S300

S

30

S

40 Logical

Tree 1

Root

S

40

S100

S200

S300

S

10

S

20

S

30 Logical

Tree 2

Root

S1

00 S2

00

S3

00

FabricPath

FabricPath Key Concept #3 Multicast

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 49

FabricPath

MAC Table on

S100 MAC IF/SID MAC IF/SID

A e1/13 (local)

e2/29

S

10

S

20

S

30

S

40

Root for

Tree 1

Root for

Tree 2

S1

00 S2

00

S3

00

MAC A MAC B

Multidestinatio

n

Trees on

Switch 100 Tre

e

IF

1 po10

2 po10,po20,po30

,po40

Broadcast →

DMAC→FF

SMAC→A

Payload

Multidestinatio

n

Trees on

Switch 10 Tre

e

IF

1 po100,po200,po

300

2 po100

po10 po20

po40

po30

Ftag →

Ftag →

DMAC→FF

SMAC→A

Payload

DA→FF Ftag→1

SA→100.0.12

DMAC→FF

SMAC→A

Payload

po100

po300

po200

e1/13

po10

po20 po30 po40

1

3

2

4

6

DMAC→FF

SMAC→A

Payload

DA→FF Ftag→1

SA→100.0.12

Learn MACs of directly-connected

devices unconditionally

Don’t learn MACs from

flood frames

FabricPath

MAC Table on

S200 MAC IF/SID

Multidestinatio

n

Trees on

Switch 300 Tre

e

IF

1 po10,po20,po30

,po40

2 po40

5

Putting it all together – Host A to Host B (1) Broadcast ARP Request

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 50

e2/29

S

10

S

20

S

30

S

40

Root for

Tree 1

Root for

Tree 2

S2

00

S3

00

MAC A MAC B

po10 po20

po40

po30

po100

po300

po200

e1/13

po10

po20 po30 po40

Multidestinatio

n

Trees on

Switch 100 Tre

e

IF

1 po10

2 po10,po20,po30

,po40

Ftag →

Multidestinatio

n

Trees on

Switch 10 Tre

e

IF

1 po100,po200,po

300

2 po100

Ftag →

11

10

DMAC→A

SMAC→B

Payload

DA→MC1 Ftag→1

SA→300.0.64

DMAC→A

SMAC→B

Payload

7 Unknown →

A →

DMAC→A

SMAC→B

Payload

DA→MC1 Ftag→1

SA→300.0.64

FabricPath

MAC Table on

S300 MAC IF/SID MAC IF/SID

B e2/29 (local)

8

MISS

If DMAC is known, then

learn remote MAC

Multidestinatio

n

Trees on

Switch 300 Tre

e

IF

1 po10,po20,po30

,po40

2 po40

9 FabricPath

MAC Table on

S100 MAC IF/SID

A e1/13 (local)

MAC IF/SID

A e1/13 (local)

B 300.0.64

(remote)

12 DMAC→A

SMAC→B

Payload

*MC1 = 01:0f:ff:c1:01:c0

Putting it all together – Host A to Host B (2) Broadcast ARP Reply

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 51

e2/29

S

10

S

20

S

30

S

40

S2

00

S3

00

MAC A MAC B

po10 po20

po40

po30

e1/13

po10

po20 po30 po40

S1

00

DMAC→B

SMAC→A

Payload

FabricPath

MAC Table on

S100

DMAC→B

SMAC→A

Payload

13 MA

C IF/SID

A e1/13 (local)

B 300.0.64

(remote)

B →

14

S300 →

FabricPath

Routing Table

on S100 Switc

h

IF

S10 po10

S20 po20

S30 po30

S40 po40

S200

po10,

po20,

po30,

po40

S300

po10,

po20,

po30,

po40

DMAC→B

SMAC→A

Payload

DA→300.0.64 Ftag→1

SA→100.0.12

15

S300 →

FabricPath

Routing Table

on S30 Switc

h

IF

… …

S300 po300

16

S300 →

Hash

DMAC→B

SMAC→A

Payload

DA→300.0.64 Ftag→1

SA→100.0.12

FabricPath

MAC Table on

S300 MA

C IF/SID

B e2/29 (local)

MA

C IF/SID

A S100.0.12

(remote)

B e2/29 (local)

18

FabricPath

Routing Table

on S300 Switc

h

IF

… …

S300 Use LID

(64)

17

If DMAC is known, then

learn remote MAC

po300

Putting it all together – Host A to Host B Unicast Data - Routed

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 52

Switch CE edge port FP core port Routing for

FP VLAN

Nexus 5020

✗ ✗ ✗

Nexus 5010

✗ ✗ ✗

Nexus 5548P/UP

*

✓ ✓ ✓

Nexus 5596UP *

✓ ✓ ✓

For Your Reference

* L3 daughter card is ‘not’ needed in 5550 to run

FabricPath, only L3 routing for FabricPath VLANs

FabricPath Hardware & Software Nexus 5500

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 53

I/O Module CE edge port FP core port Routing for FP

VLAN

N7K-M132XP-12

✗ ✗ ✓

N7K-M148GT-11

✗ ✗ ✓

N7K-M148GS-11

N7K-M148GS-11L ✗ ✗ ✓

N7K-M108X2-12L

✗ ✗ ✓

N7K-F132XP-15 ✓ ✓ ✗

N7K-F248XP-25 * ✓ ✓ ✓

* F2 module needs to be in its own VDC or system. It is not possible to mix F1 and

F2 LC in the same VDC

For Your Reference

FabricPath Hardware & Software Nexus 7000

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 54

The Nexus 7000 features two kinds of IO Modules:

M series and F series.

M I/O Modules cannot switch FabricPath traffic

When running FabricPath,

FP Core and CE Edge ports must be on an F module or 5500

New FabricPath/CE locally significant VLAN mode:

FabricPath VLANs can only be enabled on F modules (FEX+F2 with NX-OS 6.0

release as well) or 5500 (5500 + FEX as well)

S100(config)# vlan 10

S100(config-vlan)# mode ?

ce Classical Ethernet VLAN mode

fabricpath Fabricpath VLAN mode

S100(config-vlan)# mode fabricpath

S100(config-vlan)#

FabricPath

E

F FabricPath Core

Port

Classical

Ethernet Edge

Port

FabricPath Design – VLAN Mode F1/F2/M1 Interaction

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 55

SVI SVI

F1 M1

VLAN 10 = CE VLAN

switchport VLAN 10

switchport VLAN 10

F1 M1

VLAN 10 = CE VLAN VLAN 20 = FP VLAN

switchport VLAN 10

switchport VLAN 20

M1

VLAN 10 = FP VLAN

switchport VLAN 10

F1

VLAN 10 = CE VLAN VLAN 20 = FP VLAN

switchport trunk VLAN 10,20 CE FP

M1

VLAN 10 = CE VLAN VLAN 20 = FP VLAN

switchport trunk VLAN 10,20

CE

F1

VLAN 10 = CE VLAN VLAN 20 = FP VLAN

switchport trunk VLAN 10,20

FP

Bridging in CE VLAN from M1 port to F1 port in CE mode

Routing via SVIs from M1 port in CE VLAN to F1 port in FP VLAN

F1 trunk port in CE mode carrying CE VLAN and FP VLAN

F1 port in FP mode carrying CE VLAN and FP VLAN

M1 trunk port carrying CE VLAN and FP VLAN

M1 port in FP VLAN

Note : F2 needs to be on

its own VDC so the FP

and CE VLAN

compatibility mode does

not apply

FabricPath Design F1/F2/M1 Interaction

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 56

• Introduction

• Spanning Tree and vPC

• Fabric Extender

• vPC and FEX design best practice

• FabricPath

• Summary

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 57

Leading with Innovation

NX-OS

Nexus 7K

• Industry leading scale and density

• Enabling scalable fabrics: FabricPath

• High-Availability with hitless ISSU

• Workload Mobility: OTV & LISP

• DC Consolidation: VDCs, FCoE

Nexus 2K

• Remote linecard for N5K & N7K

• 1GE & 10GE optimized options with FCoE

• Consistent architecture for blade and rack servers

Nexus 5K

• Low latency Non-blocking

• Unified ports: 1/10GE, FCoE, 2/4/8G FC

• Enabling Scalable Fabrics: FabricPath

• Virtualization: Adapter FEX, VM-FEX

Blade Offerings

• Nexus 4K : 10GE FCoE blade switch for IBM

• B22 HP: Fabric extender for HP blade server chassis

Nexus 1K

• Visibility and security for virtual machines

• Support for multiple hypervisors

• Services integration

Nexus 3K

• Ultra low latency 1/10/40GE Switching

• Optimized for high-performance workloads

• Rich L2/L3 features

Thank you.

top related