third-party switch link-aggregation configuration

Table of Contents

Introduction 2Terminology 2General Rules/Notes 3Oversubscribed Ports 43Com 6

Superstack 4924 6Allied Telesyn 7

AT9410-GB 7Cisco 4006 8

Cisco IOS 8Cisco CatOS 10

Cisco 6500 11Dell 12

PowerConnect 5224 12Enterasys (Cabletron) 14

Matrix E1 Series 14Matrix E5 Series 15

Extreme 16Summit 1i/5i/7i 16Black Diamond 16Alpine 16

Force 10 17Force 10 E-Series 17

Foundry 18FastIron/BigIron Series 18EdgeIron 24G 19FastIronEdge FES12GC 19

HP 20ProCurve Series 20

Nortel Networks 21Passport 8300 21

APPLICATION NOTE:

Third-Party Switch Link-Aggregation ConfigurationAbstract: The implementation of Link-Aggregation on the Panasas Gigabit Ethernet Switch is described, and examples of configuration commands for several switches that might be encountered in a customer’s network are given.

Third-Party Switch Link-Aggregation Configuration APPLICATION NOTE

Introduction

The four Network Ports on the back panel of the Panasas Gigabit Ethernet Switch are fixed in a link-aggregated mode, and are treated as a single connection to the customer's network. The Panasas Switch does not support Link-Aggregation Control Protocol (LACP), where ports automatically enter into an aggregated configuration, therefore, the external third-party switch ports that connect to the Panasas Switch ports must be forced to link-aggregated mode as well. The following pages show how to configure switches from several different vendors for this "forced-on" aggregated mode.

Terminology

It is important to note the different terminology between vendors for aggregated ports:

3Com: “port trunks” Allied Telesyn: “port trunking” Cisco: “EtherChannel” or “port-channel” Dell: “channel-group” Extreme: “load sharing” (static) Enterasys/Cabletron: “port trunking” or “SmartTrunking” Foundry: “trunk group” HP: “port trunking” Nortel: “multilink trunking”

PANASAS APPLICATION NOTE: Third-Party Switch Link-Aggregation Configuration 2


General Rules/Notes

Sometimes the terms “link-aggregation” and “trunking” are used interchangeably, but it should be noted that what Cisco calls a “trunk” is actually a single port or port-channel that supports multiple VLANs.

The ports on both ends of the aggregated link must be configured for the same speed, duplex mode, and flow control settings, as well as VLAN tagging, QoS priority, and CoS parameters, if any.

When only two of the four available aggregated ports on the Panasas Gigabit Ethernet Switch are used, either Ports 1 & 3 or Ports 2 & 4 should be used, for even traffic distribution over the two links.

Three-port aggregated links are not supported, only 2 and 4-port configurations are valid. A single-port connection is valid as well, to any one of the four ports, or to the Downlink port on the Panasas Switch.

The aggregated link configuration should be completed on the Third-Party Switch before connecting the cables to the Panasas ports, to prevent error conditions and reports being generated on the customer’s network.

In the following examples, the commands to be entered by the user are denoted in bold type.



Oversubscribed Ports

In switches and routers available today, most high-density Gigabit Ethernet modules (24 to 48 ports per module) are designed to be over-subscribed – that is, some number of ports (2 – 8) form a group that shares a connection to the switch fabric that is less than the combined bandwidth of the ports in this group (see Figure 1). This is a cost-effective method used to give users the capability of bursting to Gigabit Ethernet data rates through these ports, without providing the full bandwidth of a GE link.

When a customer is interested in maximizing the performance of the Panasas ActiveScale system, the Link-Aggregated ports from the shelf should not be connected to ports within the same over-subscribed group on a module (as shown at the top of Figure 1).

Some vendors allow Link-Aggregated ports to span over-subscribed groups, but still the potential exists for limited available bandwidth in this configuration, due to the traffic from the other connections within the group (as shown in the center of Figure 1).

The preferred connection from the Panasas shelf to a third-party switch is to ports that are non-blocking, each with the full Gigabit Ethernet link bandwidth to the Third-Party Switch fabric (as shown at the bottom of Figure 1).



Figure 1 Panasas Shelf Connections to Third-Party Switch Modules



3Com

Superstack 4924

Notes: These commands have not been tested.

Setting Up Port Trunks

Navigate to the Port Trunk Setup page:

1. Click the Configuration icon on the side-bar.

2. Click the Port Trunks hotlink.

Display the Ports that Belong to Each Port Trunk

1. From the Port Trunks Available listbox, select a port trunk.

2. Click the Select button. The Available Ports listbox displays the ports that are available to be placed in the port trunk. The Trunk Members listbox displays the ports that belong to the port trunk.

Placing Ports in a Port Trunk

1. From the Port Trunks Available listbox, select a port trunk.

2. Click the Select button.

3. Click a port in the Available Ports listbox.

4. Click the Add >> button. The port is assigned to the port trunk, and the port is displayed in the Trunk Members listbox.

To verify the trunk configuration, use the commands listed above in the “Display the Ports that Belong to Each Port Trunk” section.



Allied Telesyn

Notes: AT9410-GB is non-blocking, no restrictions. These commands have not been tested.

AT9410-GB

1. From the Main Menu, type A to select Advanced Switch Configuration.2. From the Advanced Switch Configuration Menu, type T to select Trunk

Configuration.3. Type A to select Add Trunk Member.4. Enter the trunk group number to configure and press Enter.5. Enter the ports that will constitute the port trunk and press Enter. Up

to a maximum of 4 ports can be specified, either individually (e.g., 1, 2, 3, 4) or as a range (e.g., 7-10).

6. To set trunk status, type S to display the status of the trunk.7. Enter the trunk group number to be set and press Enter.8. Type E to enable the new port trunk.9. Repeat steps 3 - 8 (using unique trunk-group numbers in step 7) to set

up trunks for additional shelves.



Cisco 4006

Notes: 6 GE links per slot to SupervisorIII’s K2 ASIC over the backplane WS-X4424 24-Port 10/100/1000BaseT card is 4:1 over-subscribed. It has six

ASICs, each with 1 Gbps of backplane bandwidth – use every fourth port for a trunk for maximum bandwidth to the backplane

WS-X4448 48-Port 10/100/1000BaseT card is 8:1 over-subscribed. It has six ASICs, each with 1 Gbps of backplane bandwidth – use every eighth port for a trunk for maximum bandwidth to the backplane

4500 Series advertises 32 Non-Blocking GigE Ports, 64 Gbps Switching Capacity

Cisco IOS

Notes: The following commands create two groups of aggregated ports, group 1 with

ports 1-4 (for Shelf 1) and group 2 with ports 5-8 (for Shelf 2), on a 24-port module in Slot 2 of the Cisco. Ports 9-12 of this module are added (for clients) with the aggregated groups to vlan 101.

login:password:

>enable>password:

Create vlan 101:

# vlan database(vlan)# vlan 101(vlan)# exit

Setup the aggregated-port load-balancing algorithm:

# config(config)# port-channel load-balance src-dst-port

Setup parameters for vlan 101:

(config)# interface vlan 101(config-if)# ip address 10.129.101.1 255.255.255.0



(config-if)# no shutdown

Setup the parameters for the four ports in the first port-channel:

(config)# interface range gig 2/1 - 4where 2/1-4 is module_number/port_number(s)

(config-if-range)# flowcontrol receive on(config-if-range)# flowcontrol send on(config-if-range)# switchport mode access(config-if-range)# switchport access vlan 101(config-if-range)# channel-group 1 mode on(config-if-range)# exit

(From the Cisco GUI, the equivalent of the above “channel-group xx mode on” command is selecting “Manual” trunking mode for the selected ports in the group.)

Setup the aggregated-port group (the “port-channel”) in the vlan:

(config)# interface port-channel 1(config-if)# switchport access vlan 101(config-if)# flowcontrol receive on(config-if)# exit

Setup the parameters for the four ports in the second port-channel:

(config)# interface range gig 2/5 - 8(config-if-range)# flowcontrol receive on(config-if-range)# flowcontrol send on(config-if-range)# switchport mode access(config-if-range)# switchport access vlan 101(config-if-range)# channel-group 2 mode on(config-if-range)# exit

Setup the aggregated-port group in the vlan:

(config)# interface port-channel 2(config-if)# switchport access vlan 101(config-if)# flowcontrol receive on(config-if)# exit

Setup the parameters for the client ports in the vlan:

(config)# interface range gig 2/9 - 12(config-if-range)# flowcontrol receive on(config-if-range)# flowcontrol send on



(config-if-range)# switchport mode access(config-if-range)# switchport access vlan 101(config-if-range)# exit

(config)# exit

Display parameters to check configuration:# show running-config interface port-channel 1# show etherchannel 1 port-channel # show etherchannel 1 detail # show running-config interface gig 2/1..# show running-config interface gig 2/4# show interfaces gig 2/1 etherchannel..# show interfaces gig 2/4 etherchannel# show vlan id 101# show ip interfaces# clear counters

# show interface counters module 2

Save this configuration:# copy running-config startup-config

Cisco CatOS

Notes: The following commands create two groups of aggregated ports, group 1 with

ports 1-4 (for Shelf 1) and group 2 with ports 5-8 (for Shelf 2), on a 24-port module in Slot 2 of the Cisco. Ports 9-12 of this module are added (for clients) with the aggregated groups to vlan 101.

login:password:

>enable>password:

Displaying configuration information:

switch_name> (enable) show version



switch_name> (enable) show moduleswitch_name> (enable) show portswitch_name> (enable) show port capabilities 2/1

where 2/1 is module_number/port_numberswitch_name> (enable) show port channelswitch_name> (enable) show trunk

Note that all ports to be added to a port-channel (or aggregated) must be identically configured, including the speed, duplex, vlan, and trunk mode.

Create vlan 101:

switch_name> (enable) set vtp domain testDomainswitch_name> (enable) set vlan 101 name vlan101

Add ports to vlan 101:switch_name> (enable) set vlan 101 2/1switch_name> (enable) set vlan 101 2/2..switch_name> (enable) set vlan 101 2/12

Add ports to port-channel:switch_name> (enable) set port channel 2/1-4 onswitch_name> (enable) set port channel 2/5-8 on

Display parameters to check configuration:switch_name> (enable) show portswitch_name> (enable) show port channel

Save this configuration:

Cisco 6500

Notes: 6500 Series advertises 142 Non-Blocking GigE Ports, 256 Gbps Switching

Capacity



Dell

PowerConnect 5224

Notes: Dell 5224 has two Broadcom 5632 Switch devices, each supporting 12 ports,

with 10G link between 5632s (internal) Keep trunk ports within the two groups of 12 ports (i.e. 1-12 or 13-24) for

maximum bandwidth The following commands create two groups of aggregated ports, group 1 with

ports 1-4 (for Shelf 1) and group 2 with ports 5-8 (for Shelf 2). Ports 9-12 are added (for clients) with the aggregated groups to vlan 101.

console# config

Create vlan 101:

console(config)#vlan databaseconsole(config-vlan)#vlan 101 media ethernetconsole(config-vlan)#exit

Add ports to channel groups and vlan:

console(config)#interface ethernet 1/1console(config-if)#flowcontrolconsole(config-if)#switchport allowed vlan add 101console(config-if)#channel-group 1console(config-if)#exit...console(config)#interface ethernet 1/4console(config-if)#flowcontrolconsole(config-if)#switchport allowed vlan add 101console(config-if)#channel-group 1console(config-if)#exit

console(config)#interface ethernet 1/5console(config-if)#flowcontrolconsole(config-if)#switchport allowed vlan add 101console(config-if)#channel-group 2console(config-if)#exit.



..console(config)#interface ethernet 1/8console(config-if)#flowcontrolconsole(config-if)#switchport allowed vlan add 101console(config-if)#channel-group 2console(config-if)#exit

Add client ports to vlan:

console(config)#interface ethernet 1/9console(config-if)#flowcontrolconsole(config-if)#switchport allowed vlan add 101console(config-if)#exit...console(config)#interface ethernet 1/12console(config-if)#flowcontrolconsole(config-if)#switchport allowed vlan add 101console(config-if)#exit

console(config)#exit



Enterasys (Cabletron)

Notes: These commands have not been tested. The following commands create two groups of aggregated ports, group 1 with

ports 1-4 (for Shelf 1) and group 2 with ports 5-8 (for Shelf 2). Ports 9-12 are added (for clients) with the aggregated groups to vlan 101.

Matrix E1 Series(Firmware Version 2.05.xx)Notes:

Ports can only be assigned to one trunk; ports in a trunk must belong to the same port group, and must be of the same port type.

Only one trunk can be configured per port group. None of the ports in a trunk can be configured as a mirror source port or mirror

target port. All the ports in a trunk have to be treated as a whole when moved from/to, added

or deleted from a VLAN. Before removing a port trunk via CLI commands, all network cables must be

removed, to prevent loops. To disable a single link within a port trunk, first remove the network cable, and

then disable both ends of the link. This allows the traffic passing across that link to be automatically distributed to the other links in that trunk, without losing any significant amount of traffic.

Matrix> set trunk algorithm machashing

Create channel groups and add ports:

Matrix> set trunk trunk1 createMatrix> set trunk trunk1 enableMatrix> set trunk trunk2 createMatrix> set trunk trunk2 enableMatrix> set trunk port trunk1 ge.1.1-4Matrix> set trunk port trunk2 ge.1.5-8

Create vlan 101 and add ports:

Matrix> set vlan create 101Matrix> set port vlan ge.1.1-12 101



Matrix E5 Series

1. Select the Device Control Menu from the Main Menu.2. Select SmartTrunking Configuration.3. Enter a SmartTrunk Group ID number from 1 to 6 to identify the trunk.4. Select from two to eight ports to configure as one trunk. The ports used for each

trunk must all be on the same internal switch chip, which is synonymous with the SmartTrunk Group ID.



Extreme

Summit 1i/5i/7i

Notes: Summit Series switches are non-blocking – no bandwidth limitations between

ports. Link-Aggregated ports should be grouped on same side (left or right) of the 7i Tested on the 7i, the following commands create two groups of aggregated ports,

group 1 with ports 1-4 (for Shelf 1) and group 2 with ports 5-8 (for Shelf 2). Ports 9-12 are added (for clients) with the aggregated groups to vlan 101.

config sharing address-based L2_L3

enable sharing 1 grouping 1,2,3,4 algorithm addrenable sharing 5 grouping 5,6,7,8 algorithm addr

Create vlan 101, and add ports:

config vlan 101 add ports 1,5,9,10,11,12

To clear the settings for a group:

dIsable sharing 1

Black Diamond

Notes: 8 GE backplane links per slot for all chassis sizes (6804, 6808, 6816) G8Ti 8-port blade is non-blocking, no restrictions on port groups. G24T3 24-port blade is 4:1 oversubscribed. Groups of 4 ports share a GE link to

local switch module fabric. LinkAgg should spread across groups (e.g. 1,5,9,13).

Alpine

Notes: 4 GE backplane links per slot for all chassis sizes (3802, 3804, 3808) GM4Ti 4-port blade is non-blocking, no restrictions. Trunks can span modules,

no restrictions. GM16T3 16-port blade is 4:1 oversubscribed. Groups of 4 ports share 1 GE link

to backplane; LinkAgg should span groups.



Force 10

Notes: These commands have not been tested. The following commands create two groups of aggregated ports, group 1 with

ports 1-4 (for Shelf 1) and group 2 with ports 5-8 (for Shelf 2).

Force 10 E-Series

# configure

(conf)# interface range 2/1-4(conf-if-range)# no switchport(conf-if-range)# exit(conf)# interface range 2/5-8(conf-if-range)# no switchport(conf-if-range)# exit

(conf)# interface port-channel 1(conf-if-portch)# channel-member gigabitethernet 2/1-4(conf-if-portch)# switchport(conf-if-portch)# no shutdown(conf-if-portch)# exit(conf)# interface port-channel 2(conf-if-portch)# channel-member gigabitethernet 2/5-8(conf-if-portch)# switchport(conf-if-portch)# no shutdown(conf-if-portch)# exit

(conf)# exit

#



Foundry

FastIron/BigIron Series

Notes: These commands have not been tested. 8.4 Gbps per slot over the backplane Jumbo frame support on JetCore ASIC-based modules A port cannot be configured as a member of a trunk group if 802.3ad Link

Aggregation (LACP) is enabled on the port. A trunk group must start with a primary port – the first in every 4 ports: 1, 5, 9, 13,

17, 21, etc. Port assignment must be contiguous: ports 1-4, for example. Any changes to port parameters must be done on the primary port, and will be

automatically applied to the other ports in the trunk group. 8-port Mini-GBIC module: non-blocking, supports copper mini-GBIC 16-port Copper (RJ45) 10/100/1000 module: non-blocking on-board, 2:1

oversubscribed to backplane (8.4Gbps is dynamically allocated to active ports) In the “trunk” command, the Load-Balancing method of “server” distributes traffic

based on source and destination IP addresses. (vs. “switch” which is based only on destination IP addresses).

FastIron# configFastIron(config)# trunk server ethernet 1/1 to 1/4FastIron(config)# trunk server ethernet 1/5 to 1/8

Create vlan 101, and add ports:

FastIron(config)# vlan 101 by portFastIron(config-vlan-101)# untag ethernet 1/1FastIron(config-vlan-101)# untag ethernet 1/5FastIron(config-vlan-101)# untag ethernet 1/9-12FastIron(config-vlan-101)# exit

Save config and reboot switch for changes to take effect:

FastIron(config)# write memoryFastIron(config)# exitFastIron# reload



EdgeIron 24G

Notes: 24-port Copper (RJ45) 10/100/1000 non-blocking Layer 2 switch Supports up to 4 links in a trunk, with LACP or static trunking

FastIronEdge FES12GC

Notes: 12-port Copper or Fiber 10/100/1000 non-blocking Layer 3 switch Same trunking rules as chassis modules



HP

ProCurve Series

Notes: These commands have not been tested. The 4108GL Chassis advertises a 36.6Gbps central switching fabric, but each

module has only two full-duplex Gigabit Ethernet connections to this fabric.

1. From the Switch Console Main Menu, select:3. Switch Configuration5. Advanced Features2. Load Balancing (Meshing, FEC, Trunks)Press E (for Edit) to access the load balancing parameters.

2. In the Group column, move the cursor to the port to be configured.3. Use the Spacebar (or type the trunk name, such as trk5) to choose a trunk

assignment for the selected port.4. Move the cursor to the Type column for the selected port and use the Spacebar to

select the trunk type:– Trunk (Source Address/Destination Address trunk; the default

5. When finished assigning ports to trunks, press Enter, then S (for Save) and return to the Main Menu. (It is not necessary to reboot the switch.)



Nortel Networks

Passport 8300

Notes: Nortel calls groups of aggregated ports “multilink trunks” (mlt). These are

identified by multilink IDs, or MIDs. For 8348TX 10/100 modules, allowable MIDs are 1-7. For 8324GT 10/100/1000 modules, allowable MIDs are 1-31. With dual 8393SF fabric modules, each module will have a 20gig (or 40gig full

duplex) dedicated connection to the switch fabrics. The following commands create two multilink trunks, MID 1 with ports 1-4 (for

Shelf 1) and MID 2 with ports 5-8 (for Shelf 2), on a 24-port module in Slot 5. Ports 9-12 of this module are added (for clients) with the multilink trunks to (port-based) vlan 101.

config vlan 101 create byport 1 (where 1 is the default Spanning-Tree Group ID)

config vlan 101 ports add 5/9-5/12

config mlt 1 create

config mlt 1 add ports 5/1-5/4

config mlt 1 add vlan 101

config mlt 2 create

config mlt 2 add ports 5/5-5/8

config mlt 2 add vlan 101

To view the settings, statistics, and errors for the mlts:

show mlt info

show mlt stats 1

show mlt stats 2

show mlt error main 1



show mlt error main 2

To clear the settings for a multilink trunk:

config mlt 1 delete


third-party switch link-aggregation configuration

Documents