optimizing ingress routing with lisp across …...the biggest use case of lisp in a data center...

© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. of 18

White Paper

Optimizing Ingress Routing with LISP across Multiple

VXLAN/EVPN Sites


Contents

What You Will Learn ................................................................................................................................................ 3

LISP Overview .......................................................................................................................................................... 3

Why Use LISP in the Data Center ........................................................................................................................... 4 Mobility across Multiple Data Centers with Ingress Route Optimization ............................................................... 4

VXLAN Overview ...................................................................................................................................................... 5

VXLAN EVPN Overview ........................................................................................................................................... 5

VXLAN EVPN Integration with LISP ....................................................................................................................... 6 Host Move Detection in a VXLAN EVPN Fabric .................................................................................................... 6 Host Mobility across VXLAN EVPN Fabrics .......................................................................................................... 8

Summary ................................................................................................................................................................ 11

Functional Roles and Configuration .................................................................................................................... 11 Hardware and Software Details .......................................................................................................................... 12 Border Spine Configuration in Data Center 1 (BGP AS 65001) .......................................................................... 12 Border Leaf Configuration in Data Center 2 (BGP AS 65002) ............................................................................ 14 LISP Map-System Database Configuration......................................................................................................... 14 Branch Site Configuration ................................................................................................................................... 15 Verification .......................................................................................................................................................... 15

Conclusion ............................................................................................................................................................. 16

Appendix: Other Benefits of LISP in the Data Center ......................................................................................... 16 IPv6 Enablement ................................................................................................................................................. 16 Multitenancy and Large-Scale VPNs .................................................................................................................. 17 Efficient Multihoming at the WAN Edge .............................................................................................................. 17

For More Information ............................................................................................................................................. 18


What You Will Learn

Locator/Identity Separation Protocol (LISP) is a data center interconnect (DCI) solution that provides a simplified

way of handling multitenant connectivity in the fabric and mobility semantics across fabrics. This document

describes how to integrate Virtual Extensible LAN (VXLAN) Ethernet Virtual Private Network (EVPN) fabric with

LISP, using a configuration example. LISP, when integrated with VXLAN EVPN fabric, can help solve route

optimization problems that result from workload mobility across data center fabrics.

This document assumes that you have a basic knowledge of VXLAN, EVPN, and LISP technologies.

LISP Overview

Locator/Identity Separation Protocol is a new routing architecture that creates a model by separating the device

identity, known as the endpoint identifier (EID), and the routing locator (RLOC). The EIDs are assigned to the end

hosts, and the RLOCs are assigned to the devices (primarily routers) that make up the global routing system. This

separation adds flexibility to the network in a single protocol, helping enable mobility, scalability, and security. LISP

uses a dynamic tunneling approach rather than preconfigured tunnel endpoints. It’s designed to work in a

multihomed environment and supports communication between LISP and non-LISP sites for internetworking.

The main benefits of LISP include simplified WAN edge multihoming with ingress traffic engineering capabilities,

multitenancy over the Internet, simplified IPv6 transition support, and IP mobility for geographically dispersed data

centers.

In the traditional approach, an IPv4/IPv6 address represents both a device’s identity and location, as shown in

Figure 1.

Figure 1. Traditional IP Address

In LISP, an IPv4/IPv6 address represents a device’s identity only, and the RLOC identifies the location, as shown

in Figure 2.


Figure 2. IP Address in LISP

Why Use LISP in the Data Center

The Biggest use case of LISP in a data center environment is ingress route optimization due to workload mobility.

Mobility across Multiple Data Centers with Ingress Route Optimization

In today’s enterprise data center deployments, server virtualization and high availability requires workloads to move

from one data center to another across geographically dispersed locations. This mobility brings the challenge of

route optimization when virtual servers move: how best to route traffic to the virtual server’s current location? It also

brings the challenge of maintaining the server’s identity (IP address) when the server moves: how to retain the

IP address across moves so that clients can continue to send traffic to it regardless of the server’s current location.

With LISP, when virtual servers move, the IP address and EIDs don’t change; and only the RLOC identifiers

change. As endpoints move, traffic is routed to these endpoints in their correct location following the best possible

path (Figure 3).

Figure 3. LISP IP Address Mobility between Data Centers

There are other use cases of LISP in the data center, which are discussed in the Appendix section.


VXLAN Overview

Virtual Extensible LAN is a MAC address-in-User Datagram Protocol (UDP) tunneling mechanism. It identifies

the Layer 2 segment through a 24-bit segment identifier called the VXLAN network identifier (VNI). The large VNI

range allows the fabric to scale to 16 million segments, whereas a traditional Layer 2 network can scale to only

4096 VLANs. The original Layer 2 frame has a VXLAN header added and is then placed in a UDP-IP packet, thus

enabling VXLAN to tunnel a layer packet over a Layer 3 network. Figure 4 shows the VXLAN packet format.

Figure 4. VXLAN Packet Format

VXLAN is an overlay technology that provides Layer 2 connectivity for workloads residing at noncontiguous points

in the data center network. VXLAN provides flexibility by allowing workloads to be placed anywhere, and it offers

the traffic separation required in a multitenant environment. Unlike in traditional Layer 2 technologies, VXLAN

packets are transported through the underlay using IP information (Layer 3 header) and can take advantage of

Equal-Cost Multipath (ECMP) Layer 3 routing.

VXLAN EVPN Overview

VXLAN Ethernet Virtual Private Network is a standards-based overlay solution that deploys VXLAN fabric with

a Border Gateway Protocol (BGP)-based control plane that specifies the BGP EVPN control plane for overlays.

The Cisco® BGP control-plane solution for VXLAN uses the proven features of BGP to provide a more scalable,

flexible, and policy-based alternative. It uses Multiprotocol BGP (MP-BGP) to distribute the required overlay

reachability information. MP-BGP introduced new network layer reachability information (NLRI) called EVPN NLRI.

This information carries both Layer 2 MAC address and Layer 3 IP address information at the same time

(Figure 5).

VXLAN EVPN provides significant advantages in the overlay network by getting the Layer 3 routing as close

to the end host as possible. The BGP control plane is used to reduce flooding behavior and proactively distribute

end-host information to participating VXLAN tunnel endpoints (VTEPs).

The BGP control plane is used to:

● Discover VTEPs dynamically

● Distribute attached host MAC and IP addresses and avoid the need for the flood-and-learn mechanism

for unknown unicast traffic

● Terminate Address Resolution Protocol (ARP) requests early to avoid flooding

Many data centers today deploy a two-tier spine-and-leaf architecture for better scalability and flexibility. The

traditional Layer 2 networks are contained in the leaf (top of rack) switches. VXLAN EVPN is used to extend these

Layer 2 domains over the Layer 3 network for connectivity between the leaf switches. The leaf switches (which are

also VTEP devices) run Multiprotocol Interior BGP (MP-iBGP) and peer with route reflectors that run on the spine

switches. The function of the route reflectors is to reflect BGP updates between iBGP peers so that they don’t need

to form a fully meshed iBGP peering topology.


Figure 5. BGP EVPN Control Plane for VXLAN

VXLAN EVPN Integration with LISP Host Move Detection in a VXLAN EVPN Fabric

In the VXLAN EVPN fabric, the host routes and MAC address information are distributed in the MP-BGP EVPN

control plane, which means that the fabric itself performs the host detection. The LISP site gateways use these

host routes for triggering the LISP mobility encapsulation and decapsulation. LISP, when integrated with VXLAN

fabric, provides ingress route optimization for traffic from the clients to the data center (Figure 6).

Figure 6. LISP Functional Roles in A VXLAN Fabric

For detailed configuration of VXLAN using the EVPN control plane, please see the following white paper:

http://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/guide-c07-734107.html -

_Toc414541701.

http://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/guide-c07-734107.html#_Toc414541701

http://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/guide-c07-734107.html#_Toc414541701


When a virtual machine or host attaches to a leaf or top-of-rack (ToR) switch, the Layer 2 information is transported

to its peers in the fabric using MP-BGP. This approach helps ensure connectivity between hosts within a data

center fabric (Figure 7).

Figure 7. Host 1 in VLAN 1000 Attaches to Leaf or ToR Switch 1 and Is Associated with VNI 5000

When the virtual machine or host moves from one leaf switch to another, the new leaf switch detects that a virtual

machine has moved behind it by snooping on Domain Host Configuration Protocol (DHCP) or ARP packets. It

populates the reachability information in MP-BGP and advertises the updated MAC address route to its peers with

an updated sequence number (Figure 8).

Figure 8. Host 1 Moves from Leaf or ToR Switch 1 to Leaf or ToR Switch 3


When the original leaf or ToR switch receives the route update with the modified sequence number, it sends a

withdraw message for the stale reachability information (Figure 9).

Figure 9. BGP Control Plane: Old Route Withdrawn from Leaf or ToR Switch 1

Host Mobility across VXLAN EVPN Fabrics

When the leaf or ToR switch detects a host movement across data centers, it injects that host route into the

MP-BGP EVPN control plane with an updated sequence number. The sequence number is a mobility community

attribute that represents the state of mobility. It increments every time the server moves from one location to

another. This sequence number attribute has to be carried to the original leaf or ToR switch from which the host

moved, because it needs to withdraw that particular host route from BGP. The host route withdrawal happens only

when the leaf or ToR switch receives a route with an updated sequence number. LISP currently cannot carry the

mobility community attribute across the data center through the WAN.

To help LISP achieve mobility semantics across VXLAN EVPN fabrics, you need to establish an Exterior BGP

(eGBP) relationship between the data centers. This eBGP relationship is used to carry the mobility community

attribute in BGP EVPN across the data center sites for the stale reachability information (Figure 10 and figure 13).


Figure 10. HOST Mobility across Data Centers with LISP

In Figure 10:

1. The end system or server, after moving to a new location, sends a DHCP and ARP packet to join the new

network.

2. The leaf or ToR switch detects the new host and redistributes the IP address and MAC reachability information

in the MP-BGP EVPN control plane with an updated sequence number. This sequence number attribute is

carried across the data centers using an eBGP relationship between AS 65001 and 65002. When the original

leaf or ToR switch receives the route information with an updated sequence number, it withdraws its original

route from BGP.

When the host first comes online (before moving across data centers), the sequence number attribute will be

0. This value indicates that this was the first time that the host is coming online in any data center (Figure 11).

Figure 11. Host Mobility with Sequence Number “0”


After the host moves from one location to another, the sequence number is updated to 1, which triggers the route

update through the eBGP connection and the route withdrawal from the original leaf or ToR switch (Figure 12).

Figure 12. Host Mobility with Sequence Number “1”

3. When the LISP site gateway (also running MP-BGP EVPN in the fabric) detects this new host, it sends a

map-register message to the map-system database to register the new IP address in its own data center

(BGP AS 65002).

4. When the map system receives the map-register message from BGP, AS 65002 sends a map-notify message

to the old LISP site gateways, notifying them that the host has moved from their data center. This message

helps ensure that the LISP site gateways install a Null 0 route for that prefix in their routing tables. This Null 0

prefix indicates that the host is in a location remote to that data center.

Figure 13. LISP Map System Updates

5. When the clients in the remote branch sites try to send traffic to the LISP site gateways at which the host was

present (BGP AS 65001) before the mobility event, the site gateways see that the host is reachable through a

Null 0 route. This event triggers a solicit-map request (SMR) from the site gateways to the LISP-enabled router

in the branch site asking it to update its database.

6. The branch router then sends a map request to the mapping system asking for the new location of the host.

This request is relayed to the LISP site gateways to which the host has moved (BGP AS 65002).


7. The LISP site gateways in BGP AS 65002 unicast a map reply to the LISP-enabled branch router asking it to

update its database with the new location.

Now data traffic starts to flow to the correct data center (BGP AS 65002).

Summary

LISP as a solution is very easy to configure (with just a few commands, as shown in the configurations that follow),

and it provides an optimal way to resolve ingress route optimization challenges that result from workload mobility

across data centers. The Cisco Nexus 7000 Series and 7700 platform are switches with comprehensive feature

sets that can be used to implement the VXLAN-to-LISP solution discussed in this document using the F3 line

cards.

F3 line cards provide multiple-data-plane encapsulation in hardware and control-plane protocols. VXLAN

encapsulation is implemented in hardware on the southbound side, and LISP is implanted in hardware on the

northbound side on the F3 cards, making the Cisco Nexus 7000 Series and 7700 platform with F3 line cards an

excellent solution.

Functional Roles and Configuration

Figure 14 shows the topology of the LISP solution.

Figure 14. Topology

* In this topology the EBGP EVPN relationship between the two data centers is through an Layer 3 Data-center Interconnect (DCI). The Layer 3 connection between the data centers is highlighted using green dotted lines in the above topology


Hardware and Software Details

Table 1 summarizes the hardware and software versions used in the configuration example.

Table 1. Hardware and Software Used in Configuration Example

Functional Role Hardware Platform Software Version

Border spine and border leaf Cisco Nexus 7000 Series and 7700 platform with F3 line card Cisco NX-OS Software Release 7.2

Map server and map resolver Cisco ASR 1000 Series Aggregation Services Routers Cisco IOS®

XE Software Release 3.13.2

Border Spine Configuration in Data Center 1 (BGP AS 65001)

This section summarizes the steps for configuring LISP for hand-off from VXLAN on the border spine or border

leaf switch.

Step 1. Enable the LISP control plane.

Step 2. Configure the LISP map-server and map-resolver reachability.

Step 3. Configure the LISP hand-off for the tenant VRF instances.

The following example shows a configuration for a two-tenant VRF instance.


* If you need to configure additional EID (IP address) subnets to map to the VRF instance, then you will have to create another dynamic EID subnet name.

Example:

The LISP instance ID provides a means of maintaining unique address spaces in the control and data plane.

Instance IDs are numerical tags defined in the LISP canonical address format (LCAF). The instance ID has been

added to LISP to support virtualization.

When multiple organizations within a LISP site are using private addresses as EID prefixes, their address spaces

must remain segregated to prevent address duplication. An instance ID in the address encoding can be used to

create multiple segmented VPNs within a LISP site at which you want to keep using EID-prefix-based subnets. The

LISP instance ID is currently supported in LISP ingress tunnel routers and egress tunnel routers (ITRs and ETRs),

map server (MS), and map resolver (MR).

The LISP locator VRF is used to associate a VRF table through which the routing locator address space is

reachable with a router LISP instantiation.


Border Leaf Configuration in Data Center 2 (BGP AS 65002)

Configuration of border leaf is the same as the border spine we discussed above

For the other Border Spine in Data center 1(BGP AS 65001) and Border Leaf in Data center 2 (BGP AS 65002) the

above configuration can be replicated.

LISP Map-System Database Configuration

Step 1. Configure the map server and map resolver on the switch.

The map server and map resolver can be on either the same device or multiple devices.

The scenario here uses an ASR 1000 Series router as the map server and map resolver.


Branch Site Configuration

Verification

To check for the EID (host IP address) learned on the LISP site gateway on a Cisco Nexus 7000 Series or

7700 platform switch, use the configuration shown here.

To check for LISP map-cache entries on the map server, use the configuration shown here.


Conclusion

This document provided a brief overview of VXLAN, VXLAN EVPN, and LISP before delving into how to integrate

VXLAN EVPN with LISP.

Appendix: Other Benefits of LISP in the Data Center

LISP also supports these additional capabilities in your data center environment:

● IPv6 enablement

● Multitenancy and large-scale VPNs

● Efficient multihoming at the WAN edge

IPv6 Enablement

Enterprises wanting to use IPv6 often have problems because their current WAN supports only IPv4 traffic.

LISP can help resolve this problem because you can transition to IPv6 in phases while still having other sites and

the underlay network on IPv4. This technique is an efficient way to create and operate IPv6 islands within the

current network deployment. You can do this using the existing IPv4 underlay by encapsulating IPv6 host packets

within IPv4 headers. LISP provides support for both IPv4 and IPv6 EIDs and RLOCs (Figure 15).

Figure 15. IPv6 Enablement with LISP


Multitenancy and Large-Scale VPNs

LISP implements location and ID separation, which creates two namespaces: one for RLOCs (locations) and one

for EIDs (IP addresses). These namespaces provide tenant separation using the LISP mapping system because

LISP binds virtual routing and forwarding (VRF) to instance IDs. The LISP instance ID is a 24-bit value, which is

included in the LISP header to provide control- and data-plane traffic separation.

The LISP multitenancy solution also supports VPNs across enterprise networks to extend the network

segmentation beyond local network boundaries. This extension is accomplished with multiple VRF instances using

the LISP mapping system. Each VRF instance is tied to instance IDs for the address space (EID) in the VRF

instance. This use case enables all the new VRF instances to be transported over one WAN network separated

logically using VPNs (Figure 16).

Figure 16. Multitenancy and Large-Scale VPNs

Efficient Multihoming at the WAN Edge

The built-in multihoming and traffic engineering features are one of the primary benefits of LISP. Multihoming with

LISP is the capability to efficiently adjust the load on each WAN link without having to use advanced BGP traffic

engineering. This is accomplished very simply by setting the RLOC weight. This approach enables you to manage

and balance the utilization of the ingress bandwidth by setting the priorities. This design offers preference for

egress tunnel routers (ETRs) over others, allowing some systems to act as primary ETRs and others to act as

backups, thus inherently providing multihoming. This feature is implemented using the priority field, with lower-

priority systems being preferable over higher-priority systems (Figure 17).

Figure 17. Multihoming at the WAN Edge


For More Information

For a detailed understanding of VXLAN and LISP, see:

● http://www.cisco.com/c/en/us/products/collateral/switches/nexus-5000-series-switches/white-paper-c11-

733618.html

● http://www.cisco.com/c/en/us/products/collateral/switches/nexus-7000-series-switches/white_paper_c11-

693627.html

Printed in USA C11-734843-00 06/15

http://www.cisco.com/c/en/us/products/collateral/switches/nexus-5000-series-switches/white-paper-c11-733618.html

http://www.cisco.com/c/en/us/products/collateral/switches/nexus-5000-series-switches/white-paper-c11-733618.html

http://www.cisco.com/c/en/us/products/collateral/switches/nexus-7000-series-switches/white_paper_c11-693627.html

http://www.cisco.com/c/en/us/products/collateral/switches/nexus-7000-series-switches/white_paper_c11-693627.html

optimizing ingress routing with lisp across …...the biggest use case of lisp in a data center...

Documents