net1416be nsx logical routing or distribution for ... · •this overview of new technology...

Yves HertoghsPooja Patel

NET1416BE

#VMworld #NET1416BE

NSX Logical Routing

VMworld 2017 Content: Not fo

r publication or distri

bution

• This presentation may contain product features that are currently under development.

• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.

• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.

• Technical feasibility and market demand will affect final delivery.

• Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Disclaimer

2



bution

Objectives

NSX for vSphere

• Understand the different logical routing components and interaction in NSX

• Find out how high availability routing is performed in NSX

• Learn how to deploy logical routing

NSX-T for heterogeneous hypervisors and new Apps

• Discover logical routing in NSX-T through a demo

3



bution

Agenda

4

1 NSX Introduction

2 NSX for vSphere Logical Routing

3 NSX for vSphere deployment topologies

4 NSX-T Logical Routing

5 Summary and Q&AVMworld 2017 Content: Not fo


bution

Provides

5

A faithful reproduction of network and security services in software

Management APIs, UI

Switching Routing/NAT FirewallingLoadbalancing

VPN Connectivity to physical networksPolicies,

groups, tags

DHCP

DHCP

Endpointmonitoring



bution

Agenda

6

1 NSX Introduction






bution

NSX Logical Routing Component – Distributed Logical Router

▪ Optimized for E-W.

▪ Instantiated on ESX hosts

▪ LIFs are defined on the

Distributed Router to handle

VM default gateway traffic

▪ Multiple LIFs per DLR instance

▪ Multiple DLR instances to

isolate separate tenant

domains

▪ DLR Control VM peers with

the Edge Service Gateway

and exchanges routing

information

7

DLR Control VM

DLR Instance

ESXi

Hypervisor Kernel Modules

(VIBs)

LIF1 LIF2 LIF3

Distributed logical router

Anim

ate

d S

lide



bution

NSX Logical Routing Component – Edge Services Gateway

8

VPN

▪ On/Off-Ramp connectivity between logical and physical.

- Optimized for N-S RoutingStatic, OSPF, BGP

- Network ServicesFirewallNATLoad BalancingVPNDHCPDNS

8

NSX Edge Services

GatewayVMworld 2017 Content: N

ot for publicatio

n or distribution

NSX Logical Routing – Topology view

9

Logical view

VPN

External

VXLAN 5003

VLAN

VXLAN 5001 VXLAN 5002


External Network

Physical view

VXLAN 5001

VXLAN 5002

VXLAN 5003

ESX Host A

LIF1 LIF2 LIF3

ESX Host B

LIF1 LIF2 LIF3

ESX Host C

LIF1 LIF2 LIF3

NSX Edge VM DLR Control VM

Peering

VLAN based network

VPN

Distributed logical router Distributed logical router Distributed logical router

VM1

VM2

VM2VM1

Anim

ate

d S

lide



bution

NSX Logical Routing : Components Interaction

10

NSX Edge

(Acting as next hop router)

Web App

192.168.2.1

192.168.2.2

Forwarding Address

192.168.2.11

DLR Control VM

Data

Path

Control

Controller Cluster

Control

NSX Mgr

Distributed Logical Router created using

NSX Manager UI or Rest API.1

OSPF/BGP peering between the NSX

Edge and logical router control VM3

Learnt routes from the NSX Edge are

pushed to the Controller for distribution4

Controller sends the route updates to all

ESXi hosts5

Routing kernel modules on the hosts

handle the data path traffic6

1

34

5

6

Controller pushes logical router LIF

configuration to ESXi hosts2

2

OSPF, BGP

Db

VXLAN

VLAN

VPN

Pe

erin

g

External Network


Anim

ate

d S

lide



bution

Distributed Routing Traffic Flow

11

Same Host

vSphere Host

vSphere Distributed Switch

vSphere Host

VXLAN 5001

VXLAN 5002

Host 1 Host 2

1

2

LIF1 : 172.16.1.1

LIF2 : 172.16.2.1LIF2 – ARP Table

VM IP VM MAC

172.16.2.10 MAC2

DA: vMAC

SA: MAC1

PayloadL2 IP

DA: 172.16.2.10

SA: 172.16.1.10

MAC1

MAC2

LIF1

LIF2 vMAC Internal LIFs

Destination

InterfaceMask Gateway Connect

172.16.1.0 255.255.255.0 0.0.0.0 Direct

172.16.2.0 255.255.255.0 0.0.0.0 Direct

Routing Table

3

4

10.10.10.10/24 20.20.20.20/24

Transport Network

172.16.1.10

172.16.2.10

VM1

VM2

DLR DLR

Anim

ate

d S

lide



bution

High Availability



bution

VPN

I am ACTIVE ☺

Active/Standby HA Model

13

HA Interface

How does Active/Standby HA work?

▪ Edge High-availability – Configurable on Edge Services Gateways & DLR Control VMs.

▪ Keepalives + State Sync Information - Exchanged between Active & Standby Edges on a designated HA interface.

▪ Declare Dead Timer - Configurable

▪ Non-preemptive HA

▪ Stateful failover for services:

• FW - connection tracking LB - Sticky table

• Routing - Graceful restart extensions to OSPF/BGP plus NSF via FIB sync

Active StandbyStandby

Hypervisor 1 Hypervisor 2

VPN

Declare

Dead Timer

Expiry

Let me send

probes on my

interfaces…

No response on

any of the

interfaces :(

Sending

GARPs.Waiting......

I am not receiving

keep-alives from

my peer

Active

VPNVPNX

Anim

ate

d S

lide



bution

Active/Standby HA Model

14

Physical Router

VXLAN

VLAN

Active Standby

.2

.1

.2

.1

E1-1E1-0

192.168.100.0/24

192.168.2.0/24

Routing peering

Active

VPN

Standby

External Network

▪ All N-S traffic handled by the Active NSX Edge.

Only active NSX Edge establishes routing adjacencies to the DLR Control VM and the physical router.

▪ Anti-affinity & Graceful Restart enabled by default.

▪ Stateful services are supported on the NSX Edge pair

▪ HA Recommendations

Dynamic Routing Timers - OSPF 30/120 BGP 60/180

Dedicate Logical Switch as the HA Interface for DLR Control VMs/ESGs.

Declare Dead Timer is configurable and can be tuned down to 6 seconds

Web172.16.10.0/24

App172.16.20.0/24

DB172.16.30.0/24


VPN

Anim

ate

d S

lide



bution

ECMP HA Model (Up to 8 NSX Edges)

15

E3 E8E1 …

Routing peerings

VXLAN

VLAN

Routing peerings

E2

Web DBApp

Physical Routers

External Network

▪North-South traffic is handled by all Active NSX Edges

• Multiple equal cost paths in the DLR FIB

• Traffic is hashed based on Src/Dst IP address values

▪HA Recommendations

• No need to enable Edge HA for each Active Edge.

• Aggressive Routing Timers for fast failover

• Asymmetric routing paths – Stateful services not supported

(Stateful Firewall, NAT, LB, VPN)

• DFW is supported

• URPF setting: loose

.4 .5 .6


Active Standby

X

Anim

ate

d S

lide



bution

Comparison of Edge HA ModelsActive/Standby HA Model

BandwidthSingle Path

(~10 Gbps/Tenant)

Stateful Services Supported - NAT, LB, FW, DHCP

AvailabilityConvergence with stateful services

enabled

ECMP Model

BandwidthUp to 8 Paths

(~80 Gbps/Tenant)

Stateful Services Not Supported *DFW is supported

AvailabilityHigh

~ 3-4 sec with (1,3 sec) timers tuning

E1

Physical Router

Active Standby

E2

Routing peering

Web DBApp

Active Standby

DLRControl VM

…E8E3E1

Physical Router

E2

Routing peerings

Web DBApp

Active Standby

DLRControl VM

VPN

1

2

VPN



16



bution

Agenda

18

1 NSX Introduction






bution

VLAN 20

Edge Uplink

Physical Routers

NSX ECMP Edges

VXLAN 5020

Transit Link

Enterprise Routing Topology

19

…

Reference Design for SDDC

with NSX & vSphereNET1535BE

…E1 E2 E3 E8

DLR Control VMsRouting peerings

FIB update

Routing peerings

VXLAN

VLAN

Web1 App1DB1

WebN AppN DBN

External Network

VM VM VM VM VMVM VMVM VM VM VM


VM



bution

High Scale Multi Tenant Topology – 2-tier

20

Tenant 1

…

Tenant NSX Edge with

HA NAT/LB features

ECMP NSX Edge

(Route Aggregation Layer)

ECMP Tenant

NSX Edge

VXLAN Uplinks (or

VXLAN Trunk) VXLAN Uplinks (or

VXLAN Trunk)

VXLAN 5100

Transit

…E1 E8

Web1 App1DB1

VM VM VM VMVM VM

DLR Instance Tenant Y

Web1 App1 DB1

VM VM VM VMVM VM

External Network

DLR Instance Tenant XDistributed logical router


VPN

VPN



bution

Cross-VC Multi-site topology

21

ULS App1

ULS Web1

Site A Site B

vCenterServer A

vCenterServer B

Universal

Controller Cluster

NET1192BEMulti-Site Networking and Security

with Cross-VC NSX

Universal Transport Zone

External Network

Control VM

w/ Local EgressControl VM

w/ Local Egress

ULS Transit A ULS Transit B


VM VM

VM VM VM

VM



bution

Agenda

22

1 NSX Introduction






bution

Introducing NSX-T



bution

NSX Vision: Driving NSX everywhere

24

End users

Branch offices/Edge computing/IOT

Cloud

New app frameworks

On-premise

BARE METAL

Automation

IT at the Speed of Business

Security

Inherently Secure Infrastructure

Application Continuity

Data Center Anywhere



bution

Introducing NSX-T

NSX common capabilities

• Software based network virtualization

• Distributed routing

• Connectivity to the physical

• Edge services

• Distributed firewalling

• API-driven automation

NSX-T

Multiple Hypervisors - ESX, KVM

Multiple Endpoints - Containers, VMs, AWS Instances

Multiple Clouds - On-premise, Hosted or AWS

NOW available across

NET1863BE NSX-T Advanced Architecture Concepts

25



bution

NSX-T Feature DemoDistributed Routing



bution

NSX-T Distributed Routing

27

vSphere Host

NSX vSwitch

KVM Host

ESX Host

TEP A TEP B

Transport Network

Tenant1 Logical Router

Tenant1-Web

10.114.215.80/29

Tenant1-App

172.16.20.0/24

Tenant1-DB

172.16.30.0/24

Logical Topology

KVM Host

app

VM1web

VM1

db

VM1

web

VM1

app

VM1

db

VM1

• Distributed Routing can also be enabled between containers

Anim

ate

d S

lide



bution

28

DEMO 1: NSX-T



bution

NSX-T Feature DemoN/S Routing using BGP



bution

Terminology: Two-Tier Routing

30

Admin

Tenants/CMP

To physical

Designed for multi-tenancy and scale

• Provider Logical Router – Tier0 LR

– Role – Attach to the physical routing infrastructure

– Manual management

• Tenant Logical Router – Tier1 LR

– Role – Per tenant first hop router

– Cloud Management Platform (CMP) driven management



bution

Terminology: Edge Nodes

31

• Edge Nodes are appliances with pools of capacity for handling stateful services that are

not distributed.

- Peering with physical infrastructure- Services like NAT, DHCP Server, Firewall etc.

• Edge Nodes are available in 2 form factors – Bare Metal & VM

- Leverages Linux Foundation Project DPDK for high performance



bution

NSX-T N/S Configuration

32

Tenant1Logical Router

VM

Tenant1-Web

10.114.215.80/29

Tenant1-App

172.16.20.0/24

Tenant1-DB

172.16.30.0/24

Tier0 Logical Router

1

eBGP

AS 64520

AS 64530

Arista-1 Arista-2

VLAN 81

Edge

BM1

Edge

BM2

VLAN 86

VM VM VM

standby



bution

NSX-T N/S Configuration – Configure BGP

33


VM

Tenant1-Web

10.114.215.80/29

Tenant1-App

172.16.20.0/24

Tenant1-DB

172.16.30.0/24


2

eBGP

AS 64520

AS 64530

Arista-1 Arista-2

Edge

BM1

Edge

BM2

VM VM VM

10.14.215.237/30

10.114.215.238/30

10.114.215.225/30

10.114.215.226/30

standby



bution

34

DEMO 2: BGP



bution

NSX-T N/S Configuration – Redistribution

35


VM

Tenant1-Web

10.114.215.80/29

Tenant1-App

172.16.20.0/24

Tenant1-DB

172.16.30.0/24


3

eBGP

AS 64520

AS 64530

Arista-1 Arista-2

Edge

BM1

Edge

BM2

VM VM VM

Route Redistribution: Redistribute NSX connected, NSX static

standby



bution

36

DEMO 3: BGP Cont…



bution

NSX-T N/S Configuration – BFD

37


VM

Tenant1-Web

10.114.215.80/29

Tenant1-App

172.16.20.0/24

Tenant1-DB

172.16.30.0/24


4

eBGP

AS 64520

AS 64530

Arista-1 Arista-2

Edge

BM1

Edge

BM2

VM VM VM

10.14.215.237/30

10.114.215.238/30

10.114.215.225/30

10.114.215.226/30

standby

BFD Configuration



bution

38

DEMO 4: BFDVMworld 2017 Content: Not fo


bution

NSX-T Feature DemoFast convergence



bution

NSX-T N/S Configuration – Convergence

40


VM

Tenant1-Web

10.114.215.80/29

Tenant1-App

172.16.20.0/24

Tenant1-DB

172.16.30.0/24


eBGP

AS 64520

AS 64530

Arista-1 Arista-2

Edge

BM1

Edge

BM2

VM VM VM

10.14.215.237/30

10.114.215.238/30

10.114.215.225/30

10.114.215.226/30Xstandby

Anim

ate

d S

lide



bution

41

DEMO 5:Fast Convergence



bution

NSX-T Routing feature-set

BGP

• eBGP multihop

• Aggregate

• IP Prefix-list

• Route-map

• Set: AS path prepending, weight, MED, community

Performance

• DPDK based Edge node

• Fast convergence: BFD northbound, sub-second BFD timers on BM

42



bution

Want to try out NSX-T?

43

SPL182601U VMware NSX-T – Getting StartedSPL182602U VMware NSX-T - NSX-T with Kubernetes



bution

Agenda

44

1 NSX Introduction






bution

Key Takeaways

• NSX Logical Routing enables communication between workloads belonging to different subnets.

– Distributed Routing optimizes traffic flows for E-W communication.

– Edges handle N-S communication to the physical network & provide network services.

• Two models for High Availability - Active-Standby and ECMP model

• These building blocks are now available on NSX-T across multiple hypervisors, VMs, containers and public cloud.

45



bution

Relevant Sessions and References

▪ Sessions

▪ References

NSX for vSphere Network Virtualization Design Guide (Ver 3.0)

https://communities.vmware.com/docs/DOC-27683

46

NET1535BE

NET1536BE

Reference Design for SDDC with NSX and vSphere: Part 1 & 2

NET2542BE Deep Dive into Operationalizing NSX for vSphere

NET1192BE Multisite Networking and Security with Cross-VC NSX

NET1863BE NSX-T Advanced Architecture Concepts



bution

https://communities.vmware.com/docs/DOC-27683



bution

Questions?



bution

Join VMUG for exclusive access to NSX

vmug.com/VMUG-Join/VMUG-Advantage

Connect with your peers

communities.vmware.com

Find NSX Resources

vmware.com/products/nsx

Network Virtualization Blog

blogs.vmware.com/networkvirtualization

Where to get started

Dozens of Unique NSX Sessions

Spotlights, breakouts, quick talks & group discussions

Visit the VMware Booth

Product overview, use-case demos

Visit Technical Partner Booths

Integration demos – Infrastructure, security, operations,

visibility, and more

Meet the Experts

Join our Experts in an intimate roundtable discussion

Free Hands-on Labs

Test drive NSX yourself with expert-led or self-paces

hands-on labs

labs.hol.vmware.com

Training and Certification

Several paths to professional certifications. Learn

more at the Education & Certification Lounge.

vmware.com/go/nsxtraining

Engage and Learn Experience

Try Take

50



bution

https://www.vmug.com/VMUG-Join/VMUG-Advantage

https://communities.vmware.com/community/vmtn/nsx

http://vmware.com/products/nsx

http://blogs.vmware.com/networkvirtualization

http://labs.hol.vmware.com/

http://www.vmware.com/go/nsxtraining

net1416be nsx logical routing or distribution for ... · •this overview of new technology...

Documents