vxlan deep dive session rev0.5 final

Virtual eXtensible Local Area Net-work (VXLAN)RFC 7348 - A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks

CCIEx2 Security, Data Center2014-10-25 KwonSun Bae.

Agenda• What is VXLAN?• Why use VXLAN?• Before the learn VXLAN.

Acronyms and Definitions.

• VXLAN Overview. VXLAN’s History.

• VXLAN Deep Dive. VXLAN Packet Flow VTEP VXLAN Frame Format

• VXLAN Demo Cisco VXLAN Configuration VXLAN on vEOS Packet Captures

• VXLAN Overlay Comparisons(Options)

What is VXLAN?

VXLAN is ...• VXLAN

Virtual eXtensible Local Area Network

• VXLAN’s goal is allowing dynamic large scale isolated virtual L2 networks to be created for virtualized and multi-tenant environments.

• VXLAN is one protocol of Network overlay.

• https://sites.google.com/site/amitsciscozone/home/data-center/vxlan

Why use VXLAN?

Why use VXLAN?• Traditionally, all data centers use VLANs to enforce Layer2 isolation. As data centers

grow and needs arise for extending Layer2 networks across data center or may be be-yond a data center, the shortcomings of VLANs are evident. These shortcomings are –

In a data center, there are requirements of thousands of VLANs to partition traffic in a multi-tenant environment sharing the same L2/L3 infrastructure for a Cloud Service Provider. The current limit of 4096 VLANs (some are reserved) is not enough.

Due to Server virtualization, each Virtual Machine (VM) requires a unique MAC address and an IP address. So, there are thousands of MAC table entries on upstream switches. This places much larger demand on table capacity of the switches.

VLANs are too restrictive in terms of distance and deployment. VTP can be used to deploy VLANs across the L2 switches but most people prefer to disable VTP due to its destructive nature.

Using STP to provide L2 loop free topology disables most redundant links. Hence, Equal-Cost Multi-Path (ECMP) is hard to achieve. However, ECMP is easy to achieve in IP network.

Why use VXLAN?• Data Center Grows (Server Side)

https://www.arista.com/en/products/eos/cloud-scale-architecture/articletabs/0

Why use VXLAN?• Types of Overlay Edge Devices

VXLAN – VTEP Deployment Designs

* Cisco Live 365 - LTRDCT-1223 - Implementing VXLAN in DataCenter

Before the learn VXLAN.Acronyms and Definitions

Acronyms and Definitions• PIM

Protocol Independent Multicast

• SPB Shortest Path Bridging

• STP Spanning Tree Protocol

• ToR Top of Rack

• TRILL Transparent Interconnection of Lots

of Links

• VLAN Virtual Local Area Network

• VM Virtual Machine

• VNI VXLAN Network Identifier (or VXLAN Segment ID)

• VTEP VXLAN Tunnel End Point. An entity that origi-

nates and/or terminates VXLAN tunnels

• VXLAN Virtual eXtensible Local Area Network

• VXLAN Segment VXLAN Layer 2 overlay network over which VMs

communicate

• VXLAN Gateway an entity that forwards traffic between VXLANs

VXLAN Overview.

VXLAN Operation.

• http://www.definethecloud.net/vxlan-deep-divepart-2/

VXLAN History• https://datatracker.ietf.org/doc/rfc7348/history/

https://datatracker.ietf.org/doc/rfc7348/history/

https://datatracker.ietf.org/doc/rfc7348/history/

Important Diff from Previous• http://

www.ietf.org/rfcdiff?url1=draft-mahalingam-dutt-dcops-vxlan-02&url2=draft-mahalingam-dutt-dcops-vxlan-03 UDP Protocol NO fixed to 17 for IPv4 VXLAN Frame Format with IPv6 Outer Header added.

• http://www.ietf.org/rfcdiff?url1=draft-mahalingam-dutt-dcops-vxlan-03&url2=draft-mahalingam-dutt-dcops-vxlan-04 A well-known UDP port (4789) has been assigned by IANA for VXLAN.

• http://www.ietf.org/rfcdiff?url1=draft-mahalingam-dutt-dcops-vxlan-07&url2=draft-mahalingam-dutt-dcops-vxlan-08 VTEPs MUST not fragment VXLAN packets.

http://www.ietf.org/rfcdiff?url1=draft-mahalingam-dutt-dcops-vxlan-02&url2=draft-mahalingam-dutt-dcops-vxlan-03









VXLAN Deep Dive.

VXLAN BUM Traffic over Transport Multicast• VXLAN BUM (Broadcast, Unknown Unicast and Multicast) traffic is trans-

ported over the VXLAN segment control multicast group.


VXLAN VTEP Peer Discovery & Address Learning


VXLAN Packet Forwarding Flow


VXLAN Interface (VTEP)

*http://www.definethecloud.net/vxlan-deep-dive/

VXLAN Frame Format

* BRKDCT-2404 - VXLAN Deployment Models

VXLAN Demo.

Cisco VTEP ConfigurationCisco NX-OS N9K Cisco NX-OS N1Kv

+ So Many Manual Tasks!! http://www.cisco.com/c/en/us/products/collateral/switches/nexus-7000-series-switches/guide_c07-728863.html

Layer 3 Network

VLAN 101 VLAN 100

VLAN 100

10.183.100.130 10.183.100.131 10.183.100.132

10.183.100.1/24

VTEP VTEP

VXLAN VNI 20100

VTEP

External Network

VXLAN on vEOSvEOS-C#-----------------------------------

vlan 100

interface Ethernet1 mtu 9000 no switchport ip address 1.1.12.2/24 ip pim sparse-mode


interface Ethernet3 mtu 9000 switchport access vlan 100

interface Loopback0 ip address 1.1.1.3/32

interface Vxlan1 vxlan multicast-group 239.1.1.1 vxlan source-interface Loopback0 vxlan udp-port 4789 vxlan vlan 101 vni 100

All Devices for multicast-----------------------------------

ip pim rp-address 1.1.1.3ip multicast-routing

router ospf 1 router-id 1.1.1.x passive-interface default no passive-interface EthernetX network 0.0.0.0/0 area 0.0.0.0

Layer 3 Network

VLAN 101 VLAN 100

VLAN 100

10.183.100.130 10.183.100.131 10.183.100.132

10.183.100.1/24

VTEP VTEP

VXLAN VNI 20100

VTEP

External Network

VXLAN on vEOS

vEOS-A#-----------------------------------

vlan 101


interface Ethernet2 - 3 mtu 9000 switchport access vlan 101



vEOS-B#-----------------------------------

vlan 100


interface Ethernet2 mtu 9000 switchport access vlan 100



VXLAN on vEOS

Layer 3 Network

VLAN 101 VLAN 100

VLAN 100

10.183.100.130 10.183.100.131 10.183.100.132

10.183.100.1/24

VTEP VTEP

VXLAN VNI 20100

VTEP

External Network

Packet Capture - I

Packet Capture - II

Packet Capture - III

VXLAN Overlay Compar-isons

*Cisco Live 365 - BRKVIR-2014 - Architecting Scalable Clouds using VXLAN and N1kv

VXLAN / STTStateless Transport Tunneling Protocol

Similarities• IP Transport• IP Multicast

For broadcast and multicast frames

• Port Channel Load Distribution 5 Tuple Hashing (UDP vs TCP)

Differences• IETF Draft Authors

VXLAN: Cisco, VMware, Citrix, Red Hat, Broadcom, Arista

STT: Nicira

• Encapsulation VXLAN: UDP with 50 bytes STT: “TCP-like” with 72 to 54 bytes (not uniform) *

• Segment ID Size VXLAN: 24 bit STT: 64 bit

• Firewall ACL can act on VXLAN UDP port Firewalls will likely block STT since it has no TCP state

machine handshake

• Forwarding Logic VXLAN: Flooding/Learning STT: Not specified

VXLAN / NVGRENetwork Virtualization using Generic Routing Encapsulation

Similarities• IP Transport• IP Multicast

For broadcast and multicast frames

• 24 Bit Segment ID

Differences• IETF Draft Authors

VXLAN: Cisco, VMware, Citrix, Red Hat, Broadcom, Arista

STT: Microsoft, Intel, Dell, HP, Broadcom, Emulex, Arista

• Encapsulation VXLAN: UDP with 50 bytes NVGRE: GRE with 42 bytes

• Port Channel Load Distribution VXLAN: UDP 5-tuple hashing Most (if not all) current switches do not hash on the

GRE header

• Firewall ACL can act on VXLAN UDP port Difficult for firewall to act on the GRE Protocol Type field

• Forwarding Logic VXLAN: Flooding/Learning NVGRE: Not specified

VXLAN / OTVOverlay Transport Virtualization

Similarities• Same UDP based encapsulation

header VXLAN does not use the OTV Overlay

ID field

• IP Multicast For broadcast and multicast frames

(optional for OTV)

• 24 Bit Segment ID

Differences• Forwarding Logic

VXLAN: Flooding/Learning OTV: Uses the IS-IS protocol to advertise the

MAC address to IP bindings

• OTV can locally terminate ARP and doesn’t flood unknown MACs

• OTV can use an adjacency server to elim-inate the need for IP multicast

• OTV is optimized for Data Center Inter-connect to extend VLANs between or across data centers

• VXLAN is optimized for intra-DC and multi-tenancy

VXLAN / LISPLocator / ID Separation Protocol

Similarities• Same UDP based encapsulation

header VXLAN does not control flag bits or

Nonce/MapVersion field 24 Bit Segment ID

Differences• LISP carries IP packets, while VXLAN carries

Ethernet frames• Forwarding Logic

VXLAN: Flooding/Learning LISP: Uses a mapping system to register/resolve

inner IP to outer IP mappings

• IP Multicast is only required to carry host IP multicast traffic

• LISP is designed to give IP address (Identi-fier) mobility / multi-homing and IP core route scalability

• LISP can provide optimal traffic routing when Identifier IP addresses move to a different location

vxlan deep dive session rev0.5 final

Internet