bgp-evpn and sr dc fabricd2zmdbbm9feqrf.cloudfront.net/2017/eur/pdf/brkspg-2509.pdf · bgp-evpn and...

BGP-EVPN and SR DC FabricAddressing the evolving Data Center requirements

Ahmad Bilal, Technical Marketing Engineer

Samir Thoria, Distinguished Engineer

BRKSPG-2509

• EVPN-SR DC Fabric Introduction

• DC Fabric Building Blocks

• Segment Routing in data center

• EVPN in data center

• Conclusion

Agenda

© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public

Session Non-Objectives

• Following topics are not covered in details in this session

• NCS5500 Platform

• Segment Routing deep dive

Related Sessions:

• BRKSPG-2900: Cloud Scale Networking: NCS 5500 and NCS 5000 Series Deepdive

• BRKRST-3122: Segment Routing: Technology deep-dive and advanced use cases

BRKSPG-2509


Evolving DC Requirements

• Provide high performance any to any connectivity

• Flexible service/workload placement

• Workload mobility

• Scale

• Traffic engineering

• Efficient bandwidth utilization

• Multi-tenancy with L2 and L3 VPN

BRKSPG-2509

EVPN-SR DC Fabric Introduction


BGP

EVPN

EVPN-SR DC Fabric

EVPN-SR Data Center Fabric

Segment

Routing

IOS-XR NCS 5500

PlatformBRKSPG-2509


Underlay vs. Overlay

Underlay

Physical Network

IP/Label Transport

Full bandwidth utilization

Overlay

Virtual Network

Provides VPN services

Policy driven

BRKSPG-2509


Data Center Underlay – SR/MPLS

Leaf

Spine

ECMP Links - Layer 3

• Underlay is routing protocol + end to end topology

• Leaf – Spine Topology

• Uniform Reachability, Deterministic Latency

• High Redundancy: Node/Link Failure

BRKSPG-2509


Data Center Overlay – BGP-EVPN/MPLS

Leaf

SpineRR RR

RR Clients

RR Route Reflector

iBGP Adjacency

EVPN advertisement

BGP EVPN Overlay

•Distribute tenant routes and external network reachability

•Route-Reflectors deployed for scaling purposes

•Optionally use eBGP for overlay peering

•EVPN next hops are reachable via SR path(s)BRKSPG-2509


End-to-end Unified Control-Plane & Transport

A1 WAN/CoreAccess

PE1

PE2 DCI

DCI

Spin

eS

pin

e

Leaf

Leaf

Leaf

C3

C2

C1

BGP-EVPN base L2/L3 VPN using MPLS/SR • Single SDN-enabling forwarding

• Common control-plane for

L2/L3 VPN

• Simplified Traffic Engineering

• Consistent Data Model & APIs

across network

L2/L3VPN - VPLS, EoMPLS, RSVP-TEVPLS, Other

Overlays

L2, STP, IP

DC Fabric

Legacy Protocol Stack

Simplified Protocol Stack

A1: Access

C1-3: compute

NCS 5500 in Data Center


NCS 5500 Product Family• NCS 5501

Fixed 1 RU

800 Gbps @ 243 W

• NCS 5502Fixed 2 RU

4.8 Tbps @ 1450 W

• NCS 5508Modular 8 slots

13 RU (1/3 rack)

28.8 Tbps @ 7000 W

• NCS 5516Modular 16 slots

21 RU (1/2 rack)

57.6 Tbps @ ~18000 W

NCS 5502NCS 5501 NCS 5508 NCS 5516

BRKSPG-2509


SR Enabled DC Architectures at the TOR / Spine / Super Spine

• High 100G Density

• Low power / Low per port cost

• Medium / High Scale FIB

• Deep Buffers

• SR / MPLS Transport

• EVPN Control Plane

• Traffic Engineering

• L3 Data Center Interconnect (DCI)

Leaf

POD 1 POD 2

Super

Spine

Spine

Leaf

NCS 5501 / NCS 5502

Super Spine

NCS 5508 / NCS 5516

Spine

NCS 5502 / NCS 5508

NCS 5500 in Data Center

BRKSPG-2509


IOS-XR in Data Center

• Most dominant & well-known OS in the core/backbone space

• Strong MPLS & SR feature support

• Comprehensive L2/L3 VPN services

• Streaming telemetry support

• Evolved programmability with model driven operations

• Ability to run (host) 3rd party apps

• Common APIs for Unified manageability across the SP portfolio

Cloud Scale networking operations with XR

BRKSPG-2509

Segment Routing in Data Center


Segment Routing Overview

• Source Routing

• the source chooses a path and encodes it in the packet header as an ordered list of segments

• the rest of the network executes the encoded instructions

• Segment: an identifier for any type of instruction

• Forwarding Plane:

• MPLS: an ordered list of segments is represented as a stack of labels

• IPv6: an ordered list of segments is encoded in a routing extension header

• Multi-Vendor solution

This presentation:

MPLS Data plane

BRKSPG-2509


Segment Routing: IGP segmentsIGP Prefix Segments

• Shortest-path to the IGP prefix

• Equal Cost MultiPath (ECMP)-aware

• Global Segment

• Label = 16000 + Index

• Distributed by ISIS/OSPF

IGP Adjacency Segment

• Forward on the IGP adjacency

• Local Segment

• Advertised as label value

• Distributed by ISIS/OSPF

All nodes use default SRGB

16,000 – 23,999

BRKSPG-2509


Segment Routing Underlay

NCS5508

NCS5501

Segment Routing

Underlay

Each device in the fabric

is assigned a prefix-SID,

visible to all other devices

Underlay adjacency between

the nodes exchanging Prefix-

SID (labels)

16001 16002 16003 16004

16005 16006

Leaf

Spine

BRKSPG-2509


MPLS-SR Data Plane Operations

• Leaf-4 advertises its loopback ipv4 prefix 4.4.4.4/32with attached prefix-SID 16004

• Spine performs the PHP functionality

Segment 16004

Leaf-1 Spine

Push

Payload

Leaf-4

Pop

Payload

16004

Payload

4.4.4.4/32

Prefix-SID 16004

1.1.1.1/32

Prefix-SID 16001

BRKSPG-2509


EVPN-SR DC Fabric

DCI

WAN

VM

16004

24501Packet to C

24501

BA

16002

16004

C

Packet to C

24501Packet to C

Packet to C

• Policy driven path selection at the Leaf

• Steer traffic on any path through the network

• No path is signaled

• No LDP and RSVP required

Segment Routing in Data Center• Simplified Traffic Engineering

BRKSPG-2509

Introduction to BGP EVPN


What is EVPN

• EVPN family introduces next generation solutions for Ethernet services

• BGP control-plane for Ethernet Segment and MAC distribution learning over MPLS and VXLAN data-plane

• Same principles and operational experience as in IP VPNs

• No use of Pseudo wires

• Uses MP2P tunnels for unicast

• Multi-destination frame delivery via ingress replication (via MP2P tunnels) or LSM

• Multi-vendor solutions

EVPN-VPWS

P2P Multipoint

EVPN PBB-EVPN

EVPN

RFC 7432

RFC 7432

RFC 7623draft-ietf-bess-evpn-vpws

BRKSPG-2509


EVPN – Control and Data plane

Control-

Plane

EVPN

(MP-BGP)RFC7432

Data-

Plane

Multi-Protocol Label Switching

(MPLS)RFC7432

Network Virtualization Overlay

(VXLAN, NVGRE,

MPLSoGRE)draft-ietf-bess-evpn-overlay

LDP, SR or any

MPLS transport

Provider Backbone Bridges

(PBB+MPLS)RFC7623

BRKSPG-2509


BGP EVPN - Ethernet VPN

• Leafs run Multi-Protocol BGP to advertise & learn MAC/IP addresses over the DC Fabric

• MAC/IP addresses are advertised along with an MPLS label to rest of Leafs

Leaf

Spine

NCS5508

NCS5501

MAC/IP advertisement &

learning via BGP EVPN NLRI

Data Plane learning

from the hosts All Active multi-homing

Ethernet SegmentVMVM VMVM

RR RR

BRKSPG-2509

BGP EVPN Constructs


BGP EVPN – EVI

NCS5508

NCS5501

VMVM VMVMVM

EVI 20

EVI 10

EVI extended over BGP-EVPN

Fabric to all the Leafs

belonging to the EVI

Leafs that don’t belong to a specific

EVI will not have MAC-VRF for that

EVI, providing efficient scalability

EVI: An EVPN instance extends Layer 2 between the Leafs

Leaf

Spine

BRKSPG-2509


BGP EVPN – Ethernet-Segment for Multi-Homing

VMVMVMVM

ESI-1

Unique 10-byte global identifier

per Ethernet Segment

ESI-2

The bundle on the Leafs connecting to a node should have Identical ES identifier (ESI)

Ethernet Segment represents a node

connected multiple Leafs

Leaf

Spine

BRKSPG-2509


BGP EVPN – Host Connectivity Options

• Ethernet Segment Identifier (ESI) ‘0’

• No DF election

Single Home Device (SHD)Multi-home (MHD) All-Active

(Per-Flow) LB

VM VM

ESI-0 ESI-0 ESI-1 ESI-1

• Identical ESI on Leafs

• Identical ESI MAC Address

• Per VLAN DF election

VMSingle homed hostMulti-homing with Link Bundling

Leaf

Spine

BRKSPG-2509


EVPN IRB in Data Center

VM

BVI-1

GW MACBVI-2

GW MAC

VM

BVI-1

GW MAC

BVI-2

GW MAC

BD-1 BD-1 BD-2 BD-2

VM VM

Intra-subnet -

Bridged

Inter-subnet -

Routed

BRKSPG-2509


Distributed Any cast Gateway with BGP-EVPN

VM VM

BVI

GW MAC

BVI

GW MACBVI

GW MAC

BVI

GW MAC

Identical Anycast Gateway Virtual IP

and MAC address are configured on

all the Leafs

Distributed Anycast Gateway serves

as the gateway for connected hosts

All the BVIs perform active forwarding

in contrast to active/standby like FHRP

Optimal intra and inter-subnet connectivity with seamless workload mobility

Leaf

Spine

BRKSPG-2509

BGP EVPN in Data Center


Centralized vs. Distributed Routing

Distributed Routing

Fabric

Leaf

Boarder

Leaf

• Optimized forwarding of east-west traffic

• ARP/MAC state localized to Leafs

• Helps with horizontal scaling of DC

Centralized Routing

• All east<->west routed traffic traverses to centralized gateways

• Centralized gateways have full ARP/MAC state in the DC

• Scale challenge

Fabric

Leaf

Centralized GW

Subnet 1 Subnet 2 VLAN 1 VLAN 2

L2

L3

L2

L3

BRKSPG-2509


Symmetric IRB Asymmetric IRB

• Flexible workload placement – any subnet

anywhere

• ARP/MAC state localized to Leafs

• Helps with horizontal scaling of DC

• Egress subnet must be local

• Ingress Leaf needs ARP/MAC state for every egress

leaf

• Limits scale

Fabric

Leaf

Boarder

Leaf

Fabric

Leaf

Boarder

Leaf

Integrated Routing and Bridging


BGP EVPN All Active per-flow Load balancing

Leaf

Spine

VM

• No dedicated cross link between leafs

required

• EVPN based service carving for load

balancing of BUM traffic forwarding

• Mass withdraw for faster convergence

BRKSPG-2509


BGP EVPN Split Horizon

Leaf

Spine

VMVM

ESI-1

Echo !

Challenge:

How to prevent flooded traffic from echoing back to a multi-homed Ethernet Segment?

BUM Label

SH Label

BRKSPG-2509


BGP EVPN Designated Forwarder (DF)Challenge:

How to prevent duplicate copies of flooded traffic from being delivered to a multi-homed

Ethernet Segment?

BRKSPG-2509


BGP EVPN AliasingChallenge:

How to load-balance traffic towards a multi-homed device across multiple Leafs when

MAC addresses are learnt by only a single Leaf?

BRKSPG-2509


BGP EVPN MAC Mass-WithdrawChallenge:

How to inform other Leafs of a failure affecting many MAC addresses quickly while the

control-plane re-converges?

BRKSPG-2509


BGP EVPN MAC MobilityChallenge:

How to detect the correct location of MAC after the movement of host from one Ethernet

Segment to another also called “MAC move”?

40

VMVM

IP-1 MAC-1

Leaf-3Leaf-1

MAC IP ESI Seq. Next-Hop

MAC-1 IP-1 0 0 Leaf-1

Host move

Leaf-4Leaf-2

Sequence number and Next-Hop value

will be changed after the host move

BRKSPG-2509


VMVM

IP-1 MAC-1

Leaf-3Leaf-1

MAC IP ESI Seq. Next-Hop

MAC-1 IP-1 0 1 Leaf-3

Leaf-4

ESI-1

Leaf-2Sequence number is incremented and

Next-hop is changed to Leaf-3

BGP EVPN MAC Mobility, continued

BRKSPG-2509


Data center interconnect

MPLS Core

DCI

DCI

EVPN - MPLS VPNv4/EVPN - MPLS EVPN - MPLS

SP Access/Aggregation

Data Center 2

DCI

Data Center 1


MPLS Data center with EVPN/SR

SR- Explicit Path Control

- Full path programmability

- TE based on application needs

EVPN- MPBGP for MAC/IP Distribution

- L2 and L3 VPN services

IOS-XR- Modular & extensible software

- Automation @ scale

- Visibility & Telemetry

- Strong MPLS & SR support

• Multi-tenant, scalable, high performance data center

• Provides common operation models across DC & WAN with IOS-XR

• Seamless transport with SR & efficient control plane with EVPN


Complete Your Online Session Evaluation

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online

• Please complete your Online Session Evaluations after each session

• Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt

• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations

BRKSPG-2509

CiscoLive.com/Online

http://ciscolive.com/Online


Continue Your Education

• Walk-in Self-Paced Labs

• LABSPG-2014: Configuring and Implementing EVPN-SR based Data Center

• Meet the Engineer 1:1 meetings

• Demos in the Cisco campus

• Related sessions

• BRKSPG-2900: Cloud Scale Networking: NCS 5500 and NCS 5000 Series Deepdive

• BRKSPG-1001: Designing High Density SP & DC Networks with NCS5500

• BRKSPG-2404: IOS-XR Platforms: System and Hardware Architectures

• BRKRST-3122: Segment Routing: Technology deep-dive and advanced use cases

• BRKSPG-2210: Designing Programmable Access Networks

BRKSPG-2509

Thank You

bgp-evpn and sr dc fabricd2zmdbbm9feqrf.cloudfront.net/2017/eur/pdf/brkspg-2509.pdf · bgp-evpn and...

Documents