Aruba OS-CX VSX Cluster

Interconnect with HPE Synergy

Published: Feb 2019

Rev: 2

© Copyright 2018 Hewlett Packard Enterprise Development LP

Notices

The information contained herein is subject to change without notice. The only warranties for Hewlett Packard

Enterprise products and services are set forth in the express warranty statements accompanying such products and

services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise

shall not be liable for technical or editorial errors or omissions contained herein.

Confidential computer software. Valid license from Hewlett Packard Enterprise required for possession, use, or

copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software

Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's

standard commercial license.

Links to third-party websites take you outside the Hewlett Packard Enterprise website. Hewlett Packard Enterprise has no control over and is not responsible for information outside the Hewlett Packard Enterprise website.

CONTENTS Introduction .................................................................................................................................................................................. 1

Synergy Virtual Connect in MLAG & ArubaOS-CX Switches in VSX Cluster ............................................................................... 1

Topology .................................................................................................................................................................................. 2

Bill of Materials ........................................................................................................................................................................ 2

HPE Synergy Oneview MLAG configuration ................................................................................................................................ 2

Create Networks ...................................................................................................................................................................... 3

Create a Network Sets ............................................................................................................................................................. 4

Create a Logical Interconnect Uplink Set ................................................................................................................................. 6

Update Server Profile Network Connections ............................................................................................................................ 7

Connectivity verification from Synergy Virtual Connects .......................................................................................................... 8

ArubaCX VSX Cluster Configurations ...................................................................................................................................... 9

Connectivity Verifications from the CX Switch ....................................................................................................................... 11

ESXi/VM Network Configuration ............................................................................................................................................ 13

Simple Network Connectivity Test ......................................................................................................................................... 14

Conclusion ............................................................................................................................................................................. 15

Tables of Figures........................................................................................................................................................................ 16

References ................................................................................................................................................................................. 16

1

Aruba OS-CX VSX Cluster Interconnect with HPE Synergy

Introduction

This document provides details on how to integrate HPE Synergy Virtual Connect with ArubaOS-CX VSX Cluster solution..

Synergy Virtual Connect in MLAG & ArubaOS-CX Switches in VSX Cluster

MC-LAG/MLAG is a type of link aggregation group (LAG) with constituent ports that terminate on separate chassis (chassis in

this context refers to a physical switch) peers. The primary purpose of MC-LAG is to provide redundancy in the event one of

the peers fail, but MC-LAG also offers better bandwidth utilization and support for active/active topologies. Its implementation

varies by vendor which means that the protocol existing between the peers is often proprietary. The MC-LAG peers use an

inter chassis control protocol to communicate control information and coordinate with each other to ensure that data traffic is

forwarded.

VSX is a high availability technology solution purpose built for the campus core. Designed using the best features of existing

HA technologies such as Multi-chassis Link Aggregation (MC-LAG) and Virtual Switching Framework (VSF), VSX enables a

distributed and redundant architecture that is highly available during upgrades inherently by architecture design. High

availability is delivered through redundancy gained by deploying two chassis with each chassis maintaining its independent

control yet staying synchronizing information via the ArubaOS-CX unique database architecture

VSX’s benefits include the flexibility to support network designs offered by other virtualization approaches. Supported designs

include:

Dual control plane architecture: Allows for better redundancy and independently upgradable firmware. With the

enhanced configuration synchronization features and unified troubleshooting capabilities, VSX management is highly simplified.

Active-Active L2: There is no need for a spanning-tree protocol, there are no blocked links and network quickly re-

convergences in the event of link or device failures.

Active-Active L3: VSX switches can run OSPF, BGP and PIM over MCLAG links for communication between

aggregation and core. While the control plane is split, the data path is unified. The first switch that gets the packet will forward the packet to the downstream neighbor and traffic traverse between VSX peers prior to forwarding for downstream neighbor.

DHCP Relay redundancy: Both aggregation switches can be configured as DHCP forwarders but only one of the devices

plays an active role in relaying DHCP requests between the clients and the DHCP server.

No First Hop Redundancy Protocol (FHRP) configuration required: Superior to many vendor MLAG solutions, Aruba

VSX does not require the VRRP protocol and if one of the devices fail, the other device will simply take over and forward all traffic.

Note:

Networking vendors frequently use different nomenclature when referring to MC-LAG. MC-LAG is used generically to refer

to the concept of aggregating ports across physically separate switches

For example, HPE FlexFabric switches use the term “Intelligent Resilient Framework” or “IRF,” while Arista switches use the term “MLAG” and Cisco switches often use “virtual port channels” or “vPC.” In this paper,. For HPE ArubaOS-CX switches, in this paper uses the term VSX. For Synergy Virtual Connect Interconnect Modules, this paper uses the term “MLAG.”

2

Topology

Synergy Virtual Connects are connected to VSX Cluster as shown in topology below, this is a high-level, logical depiction of

the network design

Figure 1: Synergy Virtual Connect in MLAG & ArubaOS-CX Switches in VSX Cluster

Figure 1 shows two ports from each of the VC modules (labeled as VC1 and VC2) have been designated for use in their own

MLAG (note that HPE typically uses the term “MLAG” for Synergy VC modules). All four of these VC uplink ports are

aggregated Ethernet connections and participate in an MLAG. The Synergy VC’s implementation of MLAG allows the modules

to easily interoperate with ArubaOS-CX Switches or any third-party ToR or End of Row (EoR) switch. In this design all four

uplinks are active and are able to forward traffic.

Bill of Materials

1. 1 Qty - HPE Synergy with 2 Virtual Connect Modules and 2 Compute Blade & One Storage Blade

2. 2 Qty - ArubaOS-CX Switches

3. 1 Qty - VMware vCenter

HPE Synergy Oneview MLAG configuration

To achieve redundancy from each server to the VSX enabled upstream network switches, two connections required: one is

from the server’s VC1-Q1 & VC2-Q1 to ArubaOS-CX Switch-1 and the other is from the server’s VC1-Q2 & VC2-Q2

The following steps are necessary to create and configure these connections:

1. Create Networks

3

2. Create Network Sets

3. Create a Logical Interconnect Uplink Set

4. Create Server Profile Network Connections

Create Networks

Navigate as shown here OneView > Networking

To create/add VLANs, Navigate as shown here OneView > Networking > Networks

Define VLAN as below

Similarly create other vlans, and then you can see the vlans/Networks as shown below

4

Create a Network Sets

Navigate as shown here OneView > Networking > Network Sets

5

Select Networks & add

Here is the network-set after the networks(VLANs) added to the Network Set,

6

Create a Logical Interconnect Uplink Set

Navigate as shown here OneView > Networking > Logical InterConnect

Update/add Uplinks as below and update/add which Networks (vlans) supposed to use these Uplinks

7

Once the uplink-set added/updated, here is the view of logical interconnects

Update Server Profile Network Connections

Navigate as shown here OneView > Servers > Server Profiles, Select the specific Server, click on Actions on the top right

corner > Edit , scroll down for Connections, you can add connection and select individual networks or a Network set that

created in previous steps

8

Connectivity verification from Synergy Virtual Connects

Navigate as shown here OneView > Networking > Logical InterConnect,

As shown below, LACP Link aggregation farmed using Four uplinks, and LAG id “6” assigned dynamically to this uplink-set,

also you can notice the Inter Stacking Links between the Virtual Connects

9

ArubaCX VSX Cluster Configurations

8320-SW01 Configuration

vrf KeepAlive

interface 1/1/16

no shutdown

vrf attach KeepAlive

ip address 192.168.10.1/29

!

vlan 20

name MGMT

vsx-sync

vlan 21

name DATA

vsx-sync

vlan 22

name VOICE

vsx-sync

!

interface lag 1

description ISL LAG

no shutdown

10

no routing

vlan trunk native 1

vlan trunk allowed 20,21,22

lacp mode active

!

vsx

inter-switch-link lag 1

role primary

keepalive peer 192.168.10.2 source 192.168.10.1 vrf KeepAlive vsx-sync mclag-interfaces

!

interface lag 11 multi-chassis

description VSX LAG–Synergy Srv1

no shutdown

no routing

vlan trunk native 1

vlan trunk allowed 20,21,22

lacp mode active

loop-protect

loop-protect vlan 20-22

interface 1/1/49

description Synergy Srv1-VC1

no shutdown

mtu 9000

lag 11

exit

!

interface 1/1/50

description Synergy Srv1-VC2

no shutdown

mtu 9000

lag 11

exit

interface 1/1/15,1/1/48

no shutdown

mtu 9000

lag 1

exit

!

8320-SW02 Configuration

vrf KeepAlive

interface 1/1/16

no shutdown

vrf attach KeepAlive

ip address 192.168.10.2/29

!

vlan 20

name MGMT

vsx-sync

vlan 21

name DATA

vsx-sync

vlan 22

name VOICE

vsx-sync

!

interface lag 1

description ISL LAG

no shutdown

11

no routing

vlan trunk native 1

vlan trunk allowed 20,21,22

lacp mode active

!

vsx

inter-switch-link lag 1

role primary

keepalive peer 192.168.10.1 source 192.168.10.2 vrf KeepAlive vsx-sync mclag-interfaces

!

interface lag 11 multi-chassis

description VSX LAG–Synergy Srv1

no shutdown

no routing

vlan trunk native 1

vlan trunk allowed 20,21,22

lacp mode active

loop-protect

interface 1/1/49

description Synergy Srv1-VC1

no shutdown

mtu 9000

lag 11

exit

interface 1/1/50

description Synergy Srv1-VC2

no shutdown

mtu 9000

lag 11

exit

interface 1/1/15,1/1/48

no shutdown

mtu 9000

lag 1

exit

!

Connectivity Verifications from the CX Switch

SW02- Verification

8320-SW2# show vsx status

VSX Operational State

---------------------

ISL channel : In-Sync

ISL mgmt channel : operational

Config Sync Status : in-sync

NAE : peer_reachable

HTTPS Server : peer_reachable

Attribute Local Peer

------------ -------- --------

ISL link lag1 lag1

ISL version 2 2

System MAC d0:67:26:49:6b:fa d0:67:26:49:cc:f2

Platform 8320 8320

Software Version TL.10.02.0001 TL.10.02.0001

Device Role secondary primary

12

8320-SW2# sh run vsx-sync

Current vsx-sync configuration:

!Version ArubaOS-CX TL.10.02.0001

interface lag 1

description ISL LAG

no shutdown

no routing

vlan trunk native 1 tag

vlan trunk allowed 20-22

lacp mode active

interface lag 11 multi-chassis

description VSX Synergy Srv1LAG

no shutdown

no routing

vlan trunk native 1

vlan trunk allowed 20-22

lacp mode active

loop-protect

loop-protect vlan 20-22

vlan 20

name MGMT

vsx-sync

vlan 21

name DATA

vsx-sync

vlan 22

name VOICE

vsx-sync

8320-SW2# show run vsx-sync peer-diff

No difference in configs.

8320-SW2# show lacp aggregates

Aggregate name : lag1 <<ISL link>>

Interfaces : 1/1/48 1/1/15

Heartbeat rate : Slow

Hash : l3-src-dst

Aggregate mode : Active

Aggregate name : lag11 (multi-chassis)

Interfaces : 1/1/50 1/1/49

Peer interfaces : 1/1/50 1/1/49

Heartbeat rate : Slow

Hash : l3-src-dst

Aggregate mode : Active

8320-SW2# show interface lag11

Aggregate-name lag11

Aggregate lag11 is up

Admin state is up

Description : VSX Synergy Srv1LAG

MAC Address : d0:67:26:49:cc:f2

Aggregated-interfaces : 1/1/49 1/1/50

Aggregation-key : 11

Aggregate mode : active

Speed 80000 Mb/s

L3 Counters: Rx Disabled, Tx Disabled

qos trust none

VLAN Mode: native-untagged

13

Native VLAN: 1

Allowed VLAN List: 20-22

Rx

13639 input packets 1000487 bytes

0 input error 0 dropped

0 CRC/FCS

Tx

11302 output packets 1320476 bytes

0 input error 0 dropped

0 collision

8320-SW1# show interface brief

1/1/48 1 trunk SFP+DA3 yes up 10000

1/1/49 1 trunk QSFP+DA5 yes up 40000

1/1/50 1 trunk QSFP+DA5 yes up 40000

lag1 1 trunk -- yes up -- 20000

lag11 1 trunk -- yes up -- 80000

8320-SW01# show vlan

-------------------------------------------------------------------------------------------

VLAN Name Status Reason Type Interfaces

-------------------------------------------------------------------------------------------

20 MGMT up ok static lag1,lag11

21 DATA up ok static lag1,lag11

22 VOICE up ok static lag1,lag11

ESXi/VM Network Configuration

For testing purposes, An ESXi host was deployed as the virtualized infrastructure platform in the solution. The ESXi host was

added to vCenter and configured with a distributed virtual switch (DVS) containing dual uplinks as shown below

14

The network connection was assigned to a port group called “PG-v20” and is setup to use VLAN 20.

Simple Network Connectivity Test

A VM (Window 2012R2 – 192.168.0.104 with MAC address of 00-50-56-A2-CE-4E) connected to port-group “PG-v20” was

used to verify basic network connectivity.

1. Start the VM network information and internet access test, by logging in to the operating system of the VM

2. From a command prompt, verify the connectivity test by pinging the Gateway address 192.168.0.254

PS C:\Users\Administrator> ipconfig

Windows IP Configuration

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . :

Link-local IPv6 Address . . . . . : fe80::c810:5b55:e669:3043%11

IPv4 Address. . . . . . . . . . . : 192.168.0.104

Subnet Mask . . . . . . . . . . . : 255.255.255.0

Default Gateway . . . . . . . . . : 192.168.0.254

PS C:\Users\Administrator> ping 192.168.0.254

Pinging 192.168.0.254 with 32 bytes of data:

Reply from 192.168.0.254: bytes=32 time<1ms TTL=255

Reply from 192.168.0.254: bytes=32 time<1ms TTL=255

Reply from 192.168.0.254: bytes=32 time<1ms TTL=255

Reply from 192.168.0.254: bytes=32 time<1ms TTL=255

Ping statistics for 192.168.0.254:

15

Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds:

Minimum = 0ms, Maximum = 0ms, Average = 0ms

Conclusion

By utilizing VSX on Aruba CX-OS Switches and MLAG on Synergy Servers, customers can achieve a robust, redundant

network environment that can survive multiple points (N+2) of failure within the environment and still keep data traffic moving

form system to system.

Utilizing HPE’s Virtual Connect version of MLAG (or MC-LAG) as implemented in the HPE Synergy solution in conjunction with

Aruba 8320 series switches, this design goal can be achieved.

Customer may also use Synergy MLAG or VSX independently or in combination of both in different topologies as shown below

depending on the requirements. Where customer may use Virtual Connects in MLAG and connects to TOR Switches where

MLAG/VSX is not supported/enabled

Figure 2: Synergy Virtual Connects in MLAG & TOR Switches not in VSX/MLAG

16

Tables of Figures

Figure 1: Synergy Virtual Connect in MLAG & ArubaOS-CX Switches in VSX Cluster .........................................2

Figure 2: Synergy Virtual Connects in MLAG & TOR Switches not in VSX/MLAG ............................................. 15

References 1. ArubaOS-CX Virtual Switching eXtension (VSX) Guide for ArubaOS-CX 10.02 2. Implementing MLAG for HPE Synergy and FlexFabric 5900 Series