hpe flexfabric drni vrd - drni tors in an ospf leaf/spine fabric · 2019-01-29 · the same remote...

HPE FlexFabric DRNI VRD - DRNI ToRs in an OSPF Leaf/Spine Fabric

Published: Jan 2019

Rev: 1

© Copyright 2018 Hewlett Packard Enterprise Development LP Notices

The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.

Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use, or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license.

Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett Packard Enterprise has no control over and is not responsible for information outside the Hewlett Packard Enterprise website.

0

CONTENTS Introduction .............................................................................................................................................................................................................. 1 DRNI Overview .......................................................................................................................................................................................................... 1 DRNI Components ................................................................................................................................................................................................... 1 DRNI in OSPF Fabric Deployment Example: ...................................................................................................................................................... 2 DRNI Example Product List .................................................................................................................................................................................... 3 DRNI Configuration Steps ...................................................................................................................................................................................... 4

5940-6 – DRNI Device A .................................................................................................................................................................................... 4

5940-3 – DRNI Device B .................................................................................................................................................................................... 6

Configure the attached ‘host’ 5940-7 ............................................................................................................................................................. 8

Verifying DRNI and VRRP ................................................................................................................................................................................... 9

Verifying OSPF fabric configuration .............................................................................................................................................................. 11 DRNI Failure Tests .................................................................................................................................................................................................. 13

Test #1: Downlink failure simulation to host facing interfaces .............................................................................................................. 13

Test #2: IPL link failure simulation ................................................................................................................................................................ 14

Test #3: Switch failure simulation ................................................................................................................................................................. 14 Appendix: Complete switch configurations ..................................................................................................................................................... 15

DRNI Switch 5940-6 ......................................................................................................................................................................................... 15

DRNI Switch 5940-3 ......................................................................................................................................................................................... 16 For more information ........................................................................................................................................................................................... 19

1

HPE FlexFabric Based DRNI Solution in OSPF Fabric

Introduction This document provides details on how to deploy a Distributed Resilient Network Interconnect (DRNI) Top-of-Rack (ToR) solution in a standard OSPF Spine and Leaf Fabric.

DRNI Overview DRNI is a IEEE standards based solution which allows two physical switches to be virtualized into a single distributed-relay (DR) system which is able to connect to remote devices through a multi-chassis aggregate link. To the remote device the DR system appears and behaves as a single device.

While standards based, the pair of physical devices to be deployed into a DR system will only work with FlexFabric switches. Devices attaching into the DR System can be any device as normal.

DRNI Components The DR member devices are DR peers to each other. DRNI defines the following interface roles for each DR member device:

• DR System – Up to two physical devices which connect to form one single DR system from providing MLAG solution to attached devices.

• DR Device – Refers to s single device within a DR System. • DR interface - Layer 2 aggregate interface connected to the remote aggregation system. DR interfaces connected to

the same remote aggregation system belong to one DR group. DR interfaces in a DR group form a multi-chassis aggregate link.

• Intra-portal link (IPL) – Physical link/s used between two DR systems to provide internal peer control. A DR system has only one IPL.

• Intra-portal port (IPP) - Interface connected to the DR peer for internal control. Each DR member device has only one IPP. The IPPs of the DR member devices transmit DRNI protocol packets through the IPL established between them.

• Keep alive link – Layer 3 link between DR devices to monitor the state of the primary device. The DR member devices periodically send keepalive packets over the keepalive link. The IPL is down the secondary DR device takes over.

2

DRNI in OSPF Fabric Deployment Example: The below image shows the basic OSPF Spine and Leaf Fabric topology used in this example.

The focus on the document will be the configuration of the DRNI pair attached at the ToR. In this example:

• DR system connects to southbound switch to simulate host using a layer 2 link aggregation group (LAG). • DR Device 5940-6 will be the Master for traffic in VLAN10 • DR Device 5940-3 will be the Master for traffic in VLAN 20 • DR System connects to OSPF fabric via /24 routed interfaces with ECMP. (Configuration of the Spine switches and

Rack 2 switch are out of scope) o Each DRNI pair connects to both spine switches

As shown later, Ixia was hooked up to the system to simulate traffic while various failure scenarios were tested on the DRNI system.

3

DRNI Example Product List • Rack 1 (DRNI Rack)

o Qty x 2 HPE FlexFabric 5940 2-slot Switch (JH397A) Version 7.1.070, Release 2612P02

o Qty x 2 HPE 5930 24p Conv Port and 2p QSFP+ Module (JH184A) o Qty x 2 HPE 5930 24p 10GBASE-T/2p MSec QSFP+ Module (JH182A) o Qty x 4 HPE X711 Front (port) – Back (power) HV Fan Tray (JG552A) o Qty x 4 HPE 58x0AF 650W AC Power Supply (JC680A)

• Spine o Qty x 2 HPE FlexFabric 12904E Switch Chassis (JH262A)

Version 7.1.070, Release 2712 o Qty x 2 HPE FlexFabric 48p 10G/2p 100G HB Module (JH360A) o Qty x 4 HPE 12904E Main Processing Unit (JH263A) o Qty x 12 HPE FF 12904E 7.2Tbps Type H Fabric Module (JH364A) o Qty x 4 HPE 12904E Fan Tray Assy (JH265A) o Qty x 4 HPE 12900E 2400W AC PSU (JH108A)

• Rack 2 o Qty x 1 HPE FlexFabric 5940 48SFP+ 6QSFP+ Switch (JH395A)

Version 7.1.070, Release 2612P02 o Qty x 2 HPE X711 Front (port) – Back (power) HV Fan Tray (JG552A) o Qty x 2 HPE 58x0AF 650W AC Power Supply (JC680A)

4

Topology

DRNI Configuration Steps 5940-6 – DRNI Device A

# Create the VLANs that will need to be supported by the host, This example uses two VLANs (10, 20). system-view # vlan 10 # vlan 20

# Configure DR system settings (system-mac/system-number/system-priority) drni system-mac 0001-0001-0001 # drni system-number 1 # drni system-priority 123

# Specify the source and destination IP addresses of the keepalive packets sent by the local DR member device. drni keepalive ip destination 1.1.1.1 source 1.1.1.2

# Configure the keepalive interface as needed interface Ten-GigabitEthernet1/1/24 port link-mode route ip address 1.1.1.2 255.255.255.0

5

# Exclude the DR keepalive interface from the shutdown action by DRNI MAD drni mad exclude interface Ten-GigabitEthernet1/1/24

# Create Layer 2 dynamic aggregate interface Bridge-Aggregation to be used for the IPL link between the two DRNI devices interface Bridge-Aggregation3 description IPP link link-aggregation mode dynamic

# Assign the appropriate interface/s to the Layer 2 BAGG 3 interface FortyGigE 1/2/25 port link-aggregation group 3 quit

# Specify Bridge-Aggregation 3 will be used as the IPP link interface Bridge-Aggregation3 port drni intra-portal-port 1

# Make sure you enable the IPP BAGG with any VLANs that need to be supported by the attached hosts. This example trunks all VLANs, whereas in production a customer should probably only enable the VLANs they want. Bridge-Aggregation 3 will be used as the IPP link.

interface Bridge-Aggregation3 port link-type trunk port trunk permit vlan all quit

# Create a Layer 2 dynamic aggregate interface Bridge-Aggregation to be used to connect to hosts interface Bridge-Aggregation4 description to 5940-7 ‘host’ link-aggregation mode dynamic

# Assign the appropriate interface/s to the Layer 2 BAGG 4 interface ten-gigabitethernet 5/1/1 port link-aggregation group 4 quit

# Specify that Bridge-Aggregation 4 will be used as the IPP link facing a host interface Bridge-Aggregation4 port drni group 1

# Ensure the Bridge-Aggregation 4 facing the host is enabled with any VLANs that host requires interface Bridge-Aggregation4 port link-type trunk port trunk permit vlan all

# Create VLAN-interface 10 and VLAN-interface 20, and assign IP addresses to them. interface Vlan-interface10 ip address 1.10.1.254 255.255.255.0 quit #

6

interface Vlan-interface20 ip address 1.20.1.254 255.255.255.0 quit

# Create a VRRP group 1 on VLAN-interface 10 and set its virtual IP address to 1.10.1.100. interface Vlan-interface10 vrrp vrid 1 virtual-ip 1.10.1.100

# If this will be the master for a VLAN set a higher priority on appropriate VLAN interface Vlan-interface10 vrrp vrid 1 priority 200

# Set a track reference to reduce the priority of the device in case of link failure interface Vlan-interface10 vrrp vrid 1 track 1 priority reduced 110 quit


# Configure OSPF as needed per the uplinks. Make sure you choose a method to advertise the networks the connected hosts utilize. This example has the DRNI Leafs configured an GR Restarters and the Spines are configured as GR helpers. ospf 1 router-id 30.30.30.30 import-route direct enable link-local-signaling enable out-of-band-resynchronization graceful-restart nonstandard area 0.0.0.0 network 1.10.1.0 0.0.0.255 network 1.20.1.0 0.0.0.255 network 10.50.3.0 0.0.0.255 network 10.50.4.0 0.0.0.255

5940-3 – DRNI Device B

# Create the VLANs that will need to be supported by the host, This example uses two VLANs (10, 20). system-view # vlan 10 # vlan 20

# Configure DR system settings (system-mac/system-number/system-priority) drni system-mac 0001-0001-0001 # drni system-number 2 # drni system-priority 123

7

# Specify the source and destination IP addresses of the keepalive packets sent by the local DR member device. drni keepalive ip destination 1.1.1.2 source 1.1.1.1

# Configure the keepalive interface as needed interface Ten-GigabitEthernet5/2/24 port link-mode route ip address 1.1.1.1 255.255.255.0

# Exclude the DR keepalive interface from the shutdown action by DRNI MAD drni mad exclude interface Ten-GigabitEthernet5/2/24

# Create Layer 2 dynamic aggregate interface Bridge-Aggregation to be used for the IPL link between the two DRNI devices interface Bridge-Aggregation3 description IPP link link-aggregation mode dynamic

# Assign the appropriate interface/s to the Layer 2 BAGG 3 interface FortyGigE 5/2/25 port link-aggregation group 3 quit

# Specify Bridge-Aggregation 3 will be used as the IPP link interface Bridge-Aggregation3 port drni intra-portal-port 1

# Make sure you enable the IPP BAGG with any VLANs that need to be supported by the attached hosts. This example trunks all VLANs, whereas in production a customer should probably only enable the VLANs they want. Bridge-Aggregation 3 will be used as the IPP link. interface Bridge-Aggregation3 port link-type trunk port trunk permit vlan all quit

# Create a Layer 2 dynamic aggregate interface Bridge-Aggregation to be used to connect to hosts interface Bridge-Aggregation4 description to 5940-7 ‘host’ link-aggregation mode dynamic

# Assign the appropriate interface/s to the Layer 2 BAGG 4 interface ten-gigabitethernet 1/2/1 port link-aggregation group 4 quit

# Specify that Bridge-Aggregation 4 will be used as the IPP link interface Bridge-Aggregation4 port drni group 1 quit

8

# Ensure the Bridge-Aggregation 4 facing the host is enabled with any VLANs that host requires interface Bridge-Aggregation4 port link-type trunk port trunk permit vlan all

# Create VLAN-interface 10 and VLAN-interface 20, and assign IP addresses to them. interface Vlan-interface10 ip address 1.10.1.253 255.255.255.0 quit # interface Vlan-interface20 ip address 1.20.1.253 255.255.255.0 quit



# If this will be the master for a VLAN set a higher priority on appropriate VLAN interface Vlan-interface20 vrrp vrid 1 virtual-ip 1.20.1.100 vrrp vrid 1 priority 200

# Set a track reference to reduce the priority of the device in case of link failure interface Vlan-interface20 vrrp vrid 1 track 1 priority reduced 110

# Configure OSPF as needed per the uplinks. Make sure you choose a method to advertise the networks the connected hosts utilize. This example has the DRNI Leafs configured an GR Restarters and the Spines are configured as GR helpers. ospf 1 router-id 31.31.31.31 import-route direct enable link-local-signaling enable out-of-band-resynchronization graceful-restart nonstandard area 0.0.0.0 network 1.10.1.0 0.0.0.255 network 1.20.1.0 0.0.0.255 network 10.50.1.0 0.0.0.255 network 10.50.2.0 0.0.0.255

Configure the attached ‘host’ 5940-7

# Create Layer 2 dynamic aggregate interface Bridge-Aggregation 4. interface Bridge-Aggregation4 description link to DRNI devices link-aggregation mode dynamic

# Assign the appropriate interface/s to the Layer 2 BAGG 4

9

interface ten-gigabitethernet 1/0/1 port link-aggregation group 4 quit interface ten-gigabitethernet 1/0/2 port link-aggregation group 4 quit

# Ensure the Bridge-Aggregation 4 facing the DRNI pair is enabled with any VLANs that host requires interface Bridge-Aggregation4 port link-type trunk port trunk permit vlan all

Verifying DRNI and VRRP

# Verify that the keepalive link is working correctly on Device A. system-view # display drni keepalive

10

# Verify that the IPP and the DR interface are working correctly on 5940-6

# Verify that all member ports of aggregation group 4 are in Selected state on both devices

11

# Verify the VRRP status on both devices

Verifying OSPF fabric configuration

# Ensure both DRNI Devices show both spines as peers, as well as the other DRNI device for attached VLANs. system-view display ospf peer

12

# Ensure all uplink fabric facing interfaces are in an UP state. The below displays show the uplink interfaces, VLAN interfaces, keepalive interfaces, IPL interfaces, as well as host facing BAGGs.

system-view display interface brief | include UP

13

DRNI Failure Tests In this test scenario, Ixia was hooked up to the switch in Rack2 as well as the switch in Rack1 acting as a server. This ‘server’ provides connectivity up to the DRNI pair of switches.

Ixia was configured to send 100 x streams with 50% configured as VLAN 10 and the other 50% as VLAN 20.

VLAN10/20 will send BI-Di traffic to Host in Rack 2 while links and switch failures will be simulated.

Test #1: Downlink failure simulation to host facing interfaces # After traffic has been started, shut down uplink and downlink interfaces on each DRNI switch. system-view interface Ten-GigabitEthernet1/2/1 shutdown

14

# After traffic disruption has been recorded and traffic is flowing normally again un shut the interface and repeat for each interface.

Interface Total Disruption Results (ms)

Ten-GigabitEthernet 1/2/1 64.724






Test #2: IPL link failure simulation # After traffic has been started, shut down IPL interfaces both DRNI devices # After traffic disruption has been recorded and traffic is flowing normally again un shut the interface, record the disruption and repeat for each device.


FortyGigE 5/2/25 - going down 112.939

FortyGigE 5/2/25 - going up 2,909.043

FortyGigE 1/2/25- going down 141.993

FortyGigE 1/2/25 - going up 1,460.271

Test #3: Switch failure simulation # After traffic has been started, shut down DRNI-A and record the disruption. After the switch is back up and traffic is flowing normally again repeat for the other DRNI device. and repeat for each device.


DRNI Device 5940-6 going down 78.515

DRNI Device 5940-6 rejoins DRNI 1,333.972

DRNI Device 5940-3 going down 132.724

DRNI Device 5940-3 rejoins DRNI 2,866.420

15

Appendix: Complete switch configurations

DRNI Switch 5940-6 # version 7.1.070, Release 2612P02 # sysname 5940-6 # ospf 1 router-id 30.30.30.30 import-route direct enable link-local-signaling enable out-of-band-resynchronization graceful-restart nonstandard area 0.0.0.0 network 1.10.1.0 0.0.0.255 network 1.20.1.0 0.0.0.255 network 10.50.3.0 0.0.0.255 network 10.50.4.0 0.0.0.255 # lldp global enable # vlan 1 # vlan 10 # vlan 20 # stp global enable # interface Bridge-Aggregation3 description IPP link port link-type trunk port trunk permit vlan all link-aggregation mode dynamic port drni intra-portal-port 1 # interface Bridge-Aggregation4 description to 5940-7 port link-type trunk port trunk permit vlan all link-aggregation mode dynamic port drni group 1 #

16

interface Vlan-interface10 ip address 1.10.1.254 255.255.255.0 vrrp vrid 1 virtual-ip 1.10.1.100 vrrp vrid 1 priority 200 vrrp vrid 1 track 1 priority reduced 110 # interface Vlan-interface20 ip address 1.20.1.254 255.255.255.0 vrrp vrid 1 virtual-ip 1.20.1.100 # description ISL link to 5940-3 port link-type trunk port trunk permit vlan all flow-interval 5 port link-aggregation group 3 # interface M-GigabitEthernet0/0/0 ip address 99.99.94.2 255.255.255.0 # interface Ten-GigabitEthernet1/1/24 port link-mode route ip address 1.1.1.2 255.255.255.0 # interface Ten-GigabitEthernet1/2/17 port link-mode route description to OSPF Spine 2 flow-interval 5 ip address 10.50.3.1 255.255.255.0 # interface Ten-GigabitEthernet1/2/18 port link-mode route description to OSPF Spine 1 flow-interval 5 ip address 10.50.4.1 255.255.255.0 # interface Ten-GigabitEthernet1/2/1 port link-mode bridge description BAGG 4 to 5940-7 port link-type trunk port trunk permit vlan all flow-interval 5 port link-aggregation group 4 # drni system-mac 0001-0001-0001 drni system-number 1 drni system-priority 123 drni keepalive ip destination 1.1.1.1 source 1.1.1.2 drni mad exclude interface Ten-GigabitEthernet1/1/24 # mad exclude interface Ten-GigabitEthernet1/1/24 # <5940-6>

DRNI Switch 5940-3 #

17

version 7.1.070, Release 2612P02 # sysname 5940-3 # ospf 1 router-id 31.31.31.31 import-route direct enable link-local-signaling enable out-of-band-resynchronization graceful-restart nonstandard area 0.0.0.0 network 1.10.1.0 0.0.0.255 network 1.20.1.0 0.0.0.255 network 10.50.1.0 0.0.0.255 network 10.50.2.0 0.0.0.255 # vlan 1 # vlan 10 # vlan 20 # stp global enable # interface Bridge-Aggregation3 port link-type trunk port trunk permit vlan all link-aggregation mode dynamic port drni intra-portal-port 1 # interface Bridge-Aggregation4 description To 5940-7 port link-type trunk port trunk permit vlan all link-aggregation mode dynamic port drni group 1 # interface Vlan-interface10 ip address 1.10.1.253 255.255.255.0 vrrp vrid 1 virtual-ip 1.10.1.100 # interface Vlan-interface20 ip address 1.20.1.253 255.255.255.0 vrrp vrid 1 virtual-ip 1.20.1.100 vrrp vrid 1 priority 200 # interface FortyGigE5/2/25 port link-mode bridge description IPL link port link-type trunk port trunk permit vlan all flow-interval 5 port link-aggregation group 3 # interface M-GigabitEthernet0/0/0 ip address 99.99.94.1 255.255.255.0 #

18

interface Ten-GigabitEthernet5/1/17 port link-mode route description to Spine 2 flow-interval 5 ip address 10.50.2.1 255.255.255.0 # interface Ten-GigabitEthernet5/1/18 port link-mode route description to Spine 1 flow-interval 5 ip address 10.50.1.1 255.255.255.0 # interface Ten-GigabitEthernet5/2/24 port link-mode route flow-interval 5 ip address 1.1.1.1 255.255.255.0 # interface Ten-GigabitEthernet5/1/1 port link-mode bridge description To 5940-7 port link-type trunk port trunk permit vlan all flow-interval 5 port link-aggregation group 4 # drni system-mac 0001-0001-0001 drni system-number 2 drni system-priority 123 drni keepalive ip destination 1.1.1.2 source 1.1.1.1 drni mad exclude interface Ten-GigabitEthernet5/2/24 # <5940-3>

19

For more information Aruba networks

HPE Networking

HPE information library

https://www.arubanetworks.com/

https://www.hpe.com/us/en/networking.html

http://h17007.www1.hpe.com/us/en/networking/library/index.aspx#.XE-Vb1xKiUl

hpe flexfabric drni vrd - drni tors in an ospf leaf/spine fabric · 2019-01-29 · the same remote...

Documents