brocade storage

8/15/2019 Brocade Storage

1/44

53-1003534-01

October 2014

rocade VCS Storage

Deployment Guide


2/44

2014, Brocade Communications Systems, Inc. All Rights Reserved.

Brocade, the B-wing symbol, Brocade Assurance, ADX, AnyIO, DCX, Fabric OS, FastIron, HyperEdge, ICX, MLX, MyBrocade, NetIron,

OpenScript, VCS, VDX, and Vyatta are registered trademarks, and The Effortless Network and the On-Demand Data Center are trademarks

of Brocade Communications Systems, Inc., in the United States and in other countries. Other brands and product names mentioned may be

trademarks of others.

Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning anyequipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this document

at any time, without notice, and assumes no responsibility for its use. This informational document describes features that may not be

currently available. Contact a Brocade sales office for information on feature and product availability. Export of technical data contained in

this document may require an export license from the United States government.

The authors and Brocade Communications Systems, Inc. assume no liability or responsibility to any person or entity with respect to the

accuracy of this document or any loss, cost, liability, or damages arising from the information contained herein or the computer programs that

accompany it.

The product described by this document may contain open source software covered by the GNU General Public License or other open

source license agreements. To find out which open source software is included in Brocade products, view the licensing terms applicable to

the open source software, and obtain a copy of the programming source code, please visit http://www.brocade.com/support/oscd.

http://www.brocade.com/support/oscd


3/44

Contents

Brocade VCS Storage Deployment Guide...................................................................................5

Preface..............................................................................................................5

Storage Technology Overview.......................................................................... 5

Direct-Attached Storage........................................................................6

Network-attached storage.....................................................................6

Storage Area Network...........................................................................7

Deployment options........................................................................................ 13

Rack server and ToR connectivity to the FC SAN using FC routing...14

Rack server and ToR connectivity to an FC SAN using an access

gateway......................................................................................... 33

FC/FCoE storage connectivity from ToR, rack server, and ToR

connectivity to storage...................................................................37

Brocade VCS Storage Deployment Guide 3

53-1003534-01


4/44

4 Brocade VCS Storage Deployment Guide

53-1003534-01


5/44

Brocade VCS Storage Deployment Guide

● Preface..............................................................................................................................5

● Storage Technology Overview..........................................................................................5

● Deployment options........................................................................................................ 13

Preface

This document is a deployment guide for integrating a storage area network (SAN) into a Brocade VCS

fabric. It is written for technology decision-makers, architects, systems engineers, NOC engineers and

other experts responsible for network upgrades and migration.

This document explains the purpose and operation of a SAN, provides the configuration procedures,and presents three common use cases for deploying a SAN in a VCS fabric.

This document provides step-by-step examples to prepare, perform, and verify the deployment of

storage in a VCS fabric. It is assumed that the reader is familiar with establishing console access and

entering commands using the Brocade CLI. For information about the Brocade CLI, refer to the Network

OS Administration Guide.

Document History

Date Version Description

October 2014 1.0 Initial version

Storage Technology Overview

Data storage on hard disks, tape drives, and so forth, evolved quickly after the introduction of personal

computers. The Small Computer System Interface (SCSI) protocol was developed to provide a

standardized method for connecting the CPUs in PCs and servers to disk storage. Over time, three

methods of connecting storage devices and arrays have been developed to CPU hardware.

• Direct-Attached Storage (DAS)

• Network Attached Storage (NAS)

• Storage Area Network (SAN)


53-1003534-01


6/44

Direct-Attached Storage

FIGURE 1 Direct-Attached Storage

Figure 1 illustrates DAS, where the CPU is attached directly to the disk drives over the system bus,

using the SCSI protocol to write and read blocks of data to and from disk storage devices

Network-attached storage

FIGURE 2 Network-attached storage

Direct-Attached Storage


53-1003534-01


7/44

Figure 2 illustrates NAS, where servers and other storage consumers are connected to a storage array

through an Ethernet LAN. Servers communicate with the storage arrays using the following protocols:

• Network File System (NFS): NFS reads and write files to a storage device, which has a controller

that converts the file to blocks of data (block I/O) that are written and read to the local disks on the

device.

• Internet Small computer system interface (iSCSI): iSCSI sends and receives blocks of data from

and to the disk on the storage device over an IP network.

SCSI reads and writes data in blocks directly from and to the storage device. Block I/O is faster

because it directly accesses the storage device.

Storage Area Network

FIGURE 3 Storage Area Network with LAN

Figure 3 illustrates a SAN, which provides access to consolidated, block level data storage over a FC

(FC) network. A SAN typically has its own network of storage devices that are generally not accessible

through the LAN by other devices. The SAN network typically has FC switches to interconnect thestorage devices and servers.

Storage Area Network


53-1003534-01


8/44

FIGURE 4 Data center with LAN and SAN

Figure 4 shows a data-center where servers have both LAN and SAN connectivity. Servers are

connected to the storage arrays on the FC network using host bus adapters (HBAs). On the LAN,

servers use Ethernet ports on a network interface card (NIC) to connect to applications and clients.

Converged Networks

FC over Ethernet (FCoE) allows the transmission of FC traffic over an Ethernet network so servers do

not need separate FC HBAs to connect to the FC network. Servers in a converged network use

converged network adapters (CNAs) which provide both Ethernet and FC ports. FcoE allows a

converged network to carry both IP and FC traffic, combining the functionality of a LAN and a SAN.

Many modern data-centers use converged LAN and SAN networks working together as shown in

Figure 5 .

Converged Networks


53-1003534-01


9/44

FIGURE 5 Converged networks

Fibre Channel

Fibre Channel (FC) is the technology used to build SANS. FC standards are defined by T11 of the

International Committee for Information Technology Standards.

FC was developed to address the limitations of traditional Ethernet networks, which in the past hadlower speed, and had lower reliability because it is based on collision detection which causes some loss

of frames even under normal operation. FC was developed to be lossless and to provide higher speed

and lower latency, starting with a speed of 1 Gb/s, which has been increased over time to 2, 4, 8, and

16 Gb/s.

FC Protocol (FCP) is a transport protocol similar to TCP that transports SCSI commands over FC

networks.

FCoE

FC over Ethernet (FCoE) is an encapsulation protocol that enables the transport of FC storage traffic

over a version of Ethernet called Data Center Bridging (DCB). FCoE encapsulates the entire FC frame,

keeping it intact and unchanged, into DCB frames.

DCB is an enhanced Ethernet version that enables the convergence of various applications in data

centers (LAN, SAN) into a single interconnected network.

FCoE provides a method of encapsulating FC traffic over a physical Ethernet link. FCoE frames use an

EtherType of 0x8906 and FCoE Initialization Protocol (FIP) uses an Ethertype of 0x8914. FIP enables

FCoE SAN traffic and legacy LAN Ethernet traffic to be carried on the same link.

FC frames are encapsulated in an Ethernet frame and sent from one FCoE-aware device across an

Ethernet network to a second FCoE-aware device. The FCoE-aware devices may be FCoE end nodes

(E-nodes) such as servers, storage arrays, or tape drives on one end and FCoE Forwarders on the

Fibre Channel


53-1003534-01


10/44

other end. FCoE Forwarders are switches that provide SAN fabric services and may also provide

FCoE-to-FC bridging.

FC Ports

Figure 6 illustrates the FC port types that are used in storage networks.

FIGURE 6 FC Port Types

The following summarizes the function of each FC port types:

• N_port is a port on the node (such as a host or storage device); also known as a node port.

• F_port is a port on the switch that connects to connects to an N_port on a node; also known as

fabric port

• E_port is the connection between two FC switches; also known as an expansion port. When

E_ports between two switches form a link, that link is referred to as an inter-switch link (ISL)

• EX_port provides the connection between a FC router and a FC switch. On the side of the switch it

looks like a normal E_port, but on the side of the router it is an EX_port.

When connecting FC ports between two fabrics, FC traffic needs to be routed. In Figure 6 , FC ports

on VCS Fabric are connected to FC ports on the Fabric OS (FOS) Fabric. This requires FC routing on

the FOS fabric that does the FC routing.

On an VCS fabric, FC ports come up as E_ports and on an FC router, FC ports come up as EX_ports.

Worldwide names

A worldwide name (WWN) is a unique identifier used in FC storage networks. The WWN is similar to a

MAC address in an Ethernet LAN except that it is 8 bytes in length. There are two types of WWNs,

worldwide node names (WWNs) and worldwide port names (WWPNs). The WWNN is used to identify

the switch or the host, while the WWPN is used to identify individual ports on the switch or the host.

FC Ports


53-1003534-01


11/44

FC network address

The FC network address, called the FCID, is a 3 byte address. When a node is attached to the fabric, a

unique 24-bit address is assigned to the node. The FCID, as illustrated in Figure 7 , contains Domain

ID, Area ID and Node Address fields. The FCID is used to route traffic through the fabric.

FIGURE 7 FCID

Domain ID

In a VCS fabric, rbridge IDs are used to discover fabric creation problems, such as duplicate IDs. The

rbridge ID of a cluster unit is equal to the Domain ID of a FC switch.

Rbridge ID assignment uses the Domain ID assignment protocols used in the FC SAN. The Request for

Domain ID (RDI) and Domain ID Assignment (DIA) protocols ensure that a single switch, the principal

switch, centrally allocates the Domain IDs for every rbridge in the fabric and detects any Domain ID

conflicts. In case of a conflict, the node with a duplicate Domain ID is segmented from the fabric and the

user must resolve the conflict.

Device login

A FC device can be a storage server, host, or switch. When new devices are introduced into the fabric,

powered on, and connected, the following logins occur:

1. FLOGI/FCoE login: Fabric Login (FLOGI) establishes a 24-bit FCID for the device logging in, and

establishes buffer-to-buffer credits and the class of service supported.

2. PLOGI: Port Login (PLOGI) logs the device into the name server to register and query for devices

that share the same zone. During the PLOGI process, information is exchanged between the new

device and the fabric. The following types of information exchanges occur:

• SCR: State Change Registration (SCR) registers the device. If a change in the fabric occurs, such

as a zoning change or a change in the state of a device to which this device has access, the

device receives a Registered State message.

• Change Notification Registration (RSCN): The device exchanges registration information with the

name server.

• Query: The devices queries the name server for information about devices it can access.

Zoning

Zoning is a fabric-based partition that creates a logical group of devices. Zoning provides security

because devices outside the zone cannot access any device inside the zone and vice versa. Zoning

also reduces SCR broadcasts from devices because it restricts all the broadcast traffic to a specific

zone.

You can use zoning to partition the FCS fabric for a number of reasons, including the following:

FC network address


53-1003534-01


12/44

• Partition the network into zones, such as winzone and unixzone, to keep Windows servers and

storage devices separated from UNIX servers and storage.

• Logically consolidate equipment for efficiency or to facilitate time-sensitive functions. For example,

you can create a temporary zone to back up non-member devices.

A device in a zone can communicate only with other devices connected to the fabric within the same

zone. A device not included in the zone is not available to zone members. When zoning is enabled,

devices that are not included in any zone configuration are inaccessible to all other devices in the

fabric.

Zones can be configured dynamically and can vary in size, depending on the number of fabric

connected devices, and devices can belong to more than one zone.

FIGURE 8 FC Zones

Logical S N zones

A Logical SAN (LSAN) consists of zones in two or more edge or backbone fabrics that contain the

same devices. LSANs essentially provide selective device connectivity between fabrics without

merging the fabrics.

Logical SAN zones


53-1003534-01


13/44

FIGURE 9 Logical SAN zoning with FC router

FC routers provide mechanisms to manage inter-fabric device connectivity through extensions to

existing switch management interfaces.

Deployment options

The use cases that can be used for deploying zones in a VCS fabric, and which are described in this

document, include the following:

• Rack server and Top of Rack (ToR) connectivity to the FC SAN using FC routing from the ToR

• Rack server and ToR connectivity to the FC SAN using an access gateway

• FC/FCoE storage connectivity from ToR server and ToR connectivity to storage

Deployment options


53-1003534-01


14/44

Rack server and ToR connectivity to the FC SAN using FC routing

FIGURE 10 FC routing from the ToR

After a rack server connects to a storage device in the SAN, a Logical Unit Number (LUN) on the

storage device can be mapped to the server. This allows the server to store and read data on the

remote LUN as required. The configuration required includes the following:

• Configure the server with redundant links to redundant ToR switches in the VCS fabric

• Configure the VCS fabric to access the storage network

• Configure a LUN on the storage device

• Connect FC redundant links to storage switches

You can also configure zoning to provide access control for specific servers or LUNs.

The following summarizes the steps to configure a LUN on the storage device and map it to a server:

• Configure the server network

• Configure the storage network

• Configure a LUN on the storage device

• Map a LUN on the server

Configuring the server network

Server network is nothing but bringing up the VCS fabric using brocade 6730s and 6720s/8770s.

Following are the steps involved in bringing up server network.

Step 1: Configure the VCS fabric.

Step 2: Configure FCoE on the edge interfaces.

Step 3: Configure FC connectivity to the storage network.

Step 4: Configure the LSAN_Zone.

Step 5: Configure FCoE connectivity.

Step 6: Verify the FC interfaces and ISLs.

Step 7: Verify that the ISDL ports come up as E_ports.

Step 8: Configure the LSAN zoning.

Rack server and ToR connectivity to the FC SAN using FC routing


53-1003534-01


15/44

Procedure

Step 1: Configure the VCS fabric.

A VCS Fabric combines Ethernet, FCoE and FC technology to create virtualized, flexible, and

converged data center networks. When two or more Brocade VDX fabric switches are connected, they

form an Ethernet fabric and exchange information among each other using distributed intelligence. Tothe rest of the network, the Ethernet fabric appears as a single logical chassis.

VCS fabric formation has the following characteristics:

• All switches in the VCS fabric have the same VCS ID.

• Each switch has a unique rbridge ID in the range 1 to 239.

• The VCS fabric uses a link state routing protocol, First Shortest Path First (FSPF) to create the

network topology.

• A broadcast tree is established to distribute fabric broadcast and multicast traffic.

Before connecting the VDX switches, configure the following:

• Set the VCS Fabric ID and enable VCS Fabric mode.

• Set the rbridge ID.

The following is the configuration to bring up the VCS fabric on TOR3 and TOR4 switches. A similar

configuration should be applied on the EOR1, EOR2, TOR5 and TOR6 devices. Rbridge IDs must be

unique; in this example they are configured as follows:

• EOR1 Rbridge ID 1

• EOR2 Rbridge ID 2

• TOR1 Rbridge ID 3

• TOR2 Rbridge ID 4

TOR1TOR2TOR1# vcs vcsid 1 enableThis operation will change the configuration to default and reboot the switch. Do youwant to continue? [y/n]:y….TOR1#TOR1# conf tEntering configuration mode terminalTOR1(config)# rbridge-id 3TOR1(config-rbridge-id-3)#

TOR2# vcs vcsid 1This operation will change the configuration to default and reboot the switch. Do youwant to continue? [y/n]:y…..TOR2#TOR2# conf tEntering configuration mode terminalTOR2(config)# rbridge-id 4rbridge-id 4TOR2(config-rbridge-id-4)#

VCS fabrics form adjacencies, called ISLs, with directly connected VCS-enabled neighboring switches.

ISLs can be separate links or Brocade-proprietary VCS fabric trunks. By default, all interfaces are

configured to detect ISLs. In this use case all ISLs are trunks. Verify that all the ISLs are up and have

formed adjacencies with neighboring devices.

Step 2: Verify the ISL status on all the devices.

EOR1EOR1# show fab isl

Rbridge-id: 1 #ISLs: 2

Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BW TrunkNbr-Name



53-1003534-01


16/44

----------------------------------------------------------------------------------------------13 Te 1/0/14 17 Te 3/0/2 10:00:00:05:33:B8:00:18 20GYes"TOR1"14 Te 1/0/15 18 Te 4/0/3 10:00:00:05:33:EC:2A:D2 10GYes

"TOR2"EOR1#EOR2EOR2# show fabric isl


Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BWTrunk Nbr-Name----------------------------------------------------------------------------------------------13 Te 2/0/14 17 Te 4/0/2 10:00:00:05:33:EC:2A:D2 20GYes"TOR2"14 Te 2/0/15 18 Te 3/0/3 10:00:00:05:33:B8:00:18 20GYes

"TOR1"EOR2#TOR1TOR1# show fabric isl


Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BWTrunk Nbr-Name----------------------------------------------------------------------------------------------17 Te 3/0/2 13 Te 1/0/14 10:00:00:05:33:A2:EB:B7 20GYes"EOR1"18 Te 3/0/3 14 Te 2/0/15 10:00:00:05:33:86:D2:B6 20GYes"EOR2"

TOR1#TOR2TOR2# show fabric isl


Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BWTrunk Nbr-Name----------------------------------------------------------------------------------------------17 Te 4/0/2 13 Te 2/0/14 10:00:00:05:33:86:D2:B6 20GYes"EOR2"18 Te 4/0/3 14 Te 1/0/15 10:00:00:05:33:A2:EB:B7 10GYes"EOR1"

TOR2#

Step 3: Verify that the VCS fabric is formed on all the devices.

EOR1EOR1# show vcsConfig Mode : Local-OnlyVCS Mode : Fabric Cluster

VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabric



53-1003534-01


17/44

Status HostName--------------------------------------------------------------------------------------------------------------1 10:00:00:05:33:A2:EB:B7* 10.17.0.74 OnlineOnlineEOR1

2 >10:00:00:05:33:86:D2:B6 10.17.0.75 OnlineOnlineEOR23 10:00:00:05:33:B8:00:18 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2 10.17.0.112 OnlineOnlineTOR2EOR1#

Note: “>10:00:00:05:33:86:D2:B6” listed above is the principal switch.EOR2EOR2# show vcsConfig Mode : Local-OnlyVCS Mode : Fabric Cluster

VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabric

Status HostName--------------------------------------------------------------------------------------------------------------1 10:00:00:05:33:A2:EB:B7 10.17.0.74 OnlineOnlineEOR12 >10:00:00:05:33:86:D2:B6* 10.17.0.75 OnlineOnlineEOR23 10:00:00:05:33:B8:00:18 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2 10.17.0.112 OnlineOnlineTOR2EOR2#TOR1TOR1# show vcsConfig Mode : Local-OnlyVCS Mode : Fabric Cluster

VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabricStatus HostName--------------------------------------------------------------------------------------------------------------1 10:00:00:05:33:A2:EB:B7 10.17.0.74 OnlineOnlineEOR12 >10:00:00:05:33:86:D2:B6 10.17.0.75 OnlineOnlineEOR23 10:00:00:05:33:B8:00:18* 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2 10.17.0.112 OnlineOnlineTOR2TOR1#TOR2TOR2# show vcsConfig Mode : Local-OnlyVCS Mode : Fabric Cluster

VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabricStatus HostName--------------------------------------------------------------------------------------



53-1003534-01


18/44

------------------------1 10:00:00:05:33:A2:EB:B7 10.17.0.74 OnlineOnlineEOR12 >10:00:00:05:33:86:D2:B6 10.17.0.75 OnlineOnline

EOR23 10:00:00:05:33:B8:00:18 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2* 10.17.0.112 OnlineOnlineTOR2TOR2#

Step 4: Configure FCoE on the edge interfaces.

TOR1. FCoE configTOR1# show running-config interface TenGigabitEthernet 3/0/10interface TenGigabitEthernet 3/0/10 fabric isl enable fabric trunk enable edge-loop-detection vlan 0 fcoeport default no shutdown!TOR1#

TOR2. CoE configTOR2# show running-config interface TenGigabitEthernet 4/0/10interface TenGigabitEthernet 4/0/10 fabric isl enable fabric trunk enable edge-loop-detection vlan 0 fcoeport default no shutdown!TOR2#

Currently, FCoE VLANs can be only 802.1Q VLANs. They cannot be classified or used as C-TAGs for

other VLAN classification. All tenant FCoE traffic uses the same default FCoE VLAN (1002) as in

previous Brocade NOS releases.

TOR1. FCoE login

TOR1# show fcoe fabric-map default ==================================================================================================Fabric-Map VLAN VFID Pri FCMAP FKA Timeout Total-FCoE-Enodes==================================================================================================default 1002[D] 128[D] 3[D] 0xefc00[D] 8000[D] Enabled[D]64[D]

Total number of Fabric Maps = 1TOR1#TOR1#TOR1#TOR1# show fcoe login =====================================================================================

========FCOE-Port Eth-port Device WWN Device MAC SessionMAC=============================================================================================Fcoe 1/3/1 Te 3/0/10 10:00:00:05:33:90:a7:00 00:05:33:90:a7:00 0e:fc:00:03:04:00

Total number of Logins = 1TOR1#

TOR1# show name-server detail PID: 030400


19/44

Node Name: 20:00:00:05:33:90:A7:00 SCR: 3 FC4s: FCPPortSymb: [91] "Brocade-1020 | 3.2.1.0 | esxi51-0-99.englab.brocade.com |

VMware_ESXi_5.1.0_build-799733 | " NodeSymb: [58] "Brocade-1020 | 3.2.1.0 | esxi51-0-99.englab.brocade.com | " Fabric Port Name: 50:00:53:3B:80:05:30:4C

Permanent Port Name: 10:00:00:05:33:90:A7:00 Device type: Physical Initiator Interface: Fcoe 1/3/1 Physical Interface: Te 3/0/10 Share Area: Yes Redirect: NoPartial: No

TOR2. FCoE loginTOR2# show fcoe fabric-map default ==================================================================================================Fabric-Map VLAN VFID Pri FCMAP FKA Timeout Total-FCoE-Enodes==================================================================================================default 1002[D] 128[D] 3[D] 0xefc00[D] 8000[D] Enabled[D]64[D]

Total number of Fabric Maps = 1TOR2#TOR2# show name-server detailPID: 040400


20/44

trunk-enable no shutdown!TOR2#

Step 6: Verify that the FC interfaces are UP and have formed an ISL with 8510.

NOTE

Refer to the configuration on the 8510 in the section "Configuring the storage network" later in this

document and make sure the configuration for the 8510 is also applied.

TOR1.TOR1# show interface Fibrechannel 3/0/1fibrechannel 3/0/1 is up (In_Sync). Protocol state is up (connected).Pluggable media presentLineSpeed Actual: 8G AutoPortSpeed: N8GbpsportDisableReason: NonePortId: 0303c0PortIfId: 43011407PortWwn: 20:00:00:05:33:b8:00:18Distance: normal

Last clearing of show interface counters: 00:00:00Interrupts: 0 Link_failure: 0 Frjt: 0Unknown: 0 Loss_of_sync: 3 Fbsy: 0Lli: 130 Loss_of_sig: 5Proc_rqrd: 522359 Protocol_err: 0Timed_out: 0 Invalid_word: 107Rx_flushed: 0 Invalid_crc: 0Tx_unavail: 0 Delim_err: 0Free_buffer: 0 Address_err: 0Overrun: 0 Lr_in: 5Suspended: 0 Lr_out: 0Parity_err: 0 Ols_in: 02_parity_err: 0 Ols_out: 2CMI_bus_err: 0

Rate info: Bandwidth: 8.00G Tx performance: 35.0KB/sec Rx performance:275.1MB/sec

TOR1#TOR1# show fabric isl


Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BWTrunk Nbr-Name----------------------------------------------------------------------------------------------0 Fi 3/0/1 217 Fi 60/-/- 50:00:53:3C:01:20:0E:65 8GYes"fcr_fd_60"17 Te 3/0/2 13 Te 1/0/14 10:00:00:05:33:A2:EB:B7 20GYes"EOR1"18 Te 3/0/3 14 Te 2/0/15 10:00:00:05:33:86:D2:B6 20G

Yes"EOR2"

TOR1#

Login to TOR devices as root to verify that ISL ports have come up as E(Expansion)ports.

TOR1. E port VerificationTOR1:root>TOR1:root> switchshowName: sw0Type: 107.6State: Online



53-1003534-01


21/44

switchMode: NativeRole: Fabric SubordinateRbridge-id: 3switchId: fffc03WWN: 10:00:00:05:33:b8:00:18zoning: ON (cfg_vcs4)switchBeacon: OFF

FC Router: OFFFC Router BB Fabric ID: 128Address Mode: 0

Index Port Address Media Speed State Proto============================================== 0 0 0303c0 id N8 Online FC E-Port 50:00:53:3c:01:20:0e:65"fcr_fd_60"(downstream)(Trunk Primary)

TOR2.TOR2# show interface Fibrechannel 4/0/2fibrechannel 4/0/2 is up (In_Sync). Protocol state is up (connected).Pluggable media presentLineSpeed Actual: 8G AutoPortSpeed: N8GbpsportDisableReason: NonePortId: 040340PortIfId: 43011405PortWwn: 20:01:00:05:33:ec:2a:d2Distance: normal

Last clearing of show interface counters: 00:00:00Interrupts: 0 Link_failure: 0 Frjt: 0Unknown: 0 Loss_of_sync: 3 Fbsy: 0Lli: 90 Loss_of_sig: 5Proc_rqrd: 522692 Protocol_err: 0Timed_out: 0 Invalid_word: 9Rx_flushed: 0 Invalid_crc: 0Tx_unavail: 0 Delim_err: 0Free_buffer: 0 Address_err: 0Overrun: 0 Lr_in: 4Suspended: 0 Lr_out: 1Parity_err: 0 Ols_in: 12_parity_err: 0 Ols_out: 1CMI_bus_err: 0

Rate info: Bandwidth: 8.00G

Tx performance: 40.4KB/sec Rx performance:288.2MB/secTOR2#TOR2# show fabric isl


Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BW TrunkNbr-Name----------------------------------------------------------------------------------------------1 Fi 4/0/2 2 Fi 61/-/- 50:00:53:3D:E0:5A:5E:65 8G Yes"fcr_fd_61"17 Te 4/0/2 13 Te 2/0/14 10:00:00:05:33:86:D2:B6 20G Yes"EOR2"18 Te 4/0/3 14 Te 1/0/15 10:00:00:05:33:A2:EB:B7 10G Yes

"EOR1"

TOR2#

Step 7: Login to the ToR devices as root to verify that the ISL ports have come up as E_ports.

TOR2. E port VerificationTOR2:root> switchshowName: sw0Type: 107.6State: OnlineswitchMode: NativeRole: Fabric SubordinateRbridge-id: 4



53-1003534-01


22/44

switchId: fffc04WWN: 10:00:00:05:33:ec:2a:d2zoning: ON (cfg_vcs4)switchBeacon: OFFFC Router: OFFFC Router BB Fabric ID: 128Address Mode: 0

Index Port Address Media Speed State Proto============================================== 0 0 0403c0 id N8 No_Light FC1 1 040340 id N8 Online FC E-Port 50:00:53:3d:e0:5a:5e:65

"fcr_fd_61"(downstream)(Trunk Primary)

Step 8: Configure LSAN zoning in the VCS fabric.

LSAN Zone is the logical SAN zone which will be used to have zoning with multiple fabrics. In this

case, there are 2 fabrics "Server Network" and "Storage network". LSAN Zoning will be used.

Zoning config is a global config in the fabric, so this config can be applied on any one switch.

TOR1.TOR1# show running-config zoningzoning defined-configuration cfg cfg_vcs member-zone LSAN_T1

member-zone LSAN_T2 !zoning defined-configuration zone LSAN_T1 member-entry 50:06:01:6b:3e:a0:41:62 member-entry 10:00:00:05:33:90:a7:00!zoning defined-configuration zone LSAN_T2 member-entry 50:06:01:6a:3e:a0:41:62 member-entry 10:00:00:05:33:90:a7:01!zoning enabled-configuration cfg-name cfg_vcszoning enabled-configuration default-zone-access allaccesszoning enabled-configuration cfg-action cfg-saveTOR1#

TOR1# show zoning enabled-configuration zoning enabled-configuration cfg-name cfg_vcs

zoning enabled-configuration enabled-zone LSAN_T1

member-entry 50:06:01:6b:3e:a0:41:62 member-entry 10:00:00:05:33:90:a7:00

zoning enabled-configuration enabled-zone LSAN_T2 member-entry 50:06:01:6a:3e:a0:41:62 member-entry 10:00:00:05:33:90:a7:01TOR1#

Configuring the storage network

The following summarizes the steps involved in bringing up the storage network.


Step 2: Configure FC routing.

Step 3: Configure the EX_ports on the FC router.Step 4: Configure and verify other FC ports in the fabric.

To configure the storage network, complete the following steps.

Procedure


Configuring the storage network


53-1003534-01


23/44

Configure LSAN Zoning when there are multiple fabrics that require zoning (see the section Logical

SAN). Below is the running-config for the LSAN configuration in the current example.

DCX1.DCX1# zonecreate LSAN_T1 “50:06:01:6b:3e:a0:41:62; 10:00:00:05:33:90:a7:00”DCX1#cfgcreate cfg_DCX1,"LSAN_T1"DCX1#cfgenable cfg_DCX1

DCX:root> cfgshowDefined configuration: cfg: cfg_DCX1 LSAN_T1zone: LSAN_T1 50:06:01:6b:3e:a0:41:62; 10:00:00:05:33:90:a7:00Effective configuration: cfg: cfg_DCX1 zone: LSAN_T1 50:06:01:6b:3e:a0:41:62 10:00:00:05:33:90:a7:00

DCX2.DCX1# zonecreate LSAN_T2 “50:06:01:6a:3e:a0:41:62:62; 10:00:00:05:33:90:a7:01”DCX1#cfgcreate cfg_DCX2,"LSAN_T1"DCX1#cfgenable cfg_DCX2DCX:root> cfgshowDefined configuration: cfg: cfg_DCX2 LSAN_T2zone: LSAN_T2 50:06:01:6a:3e:a0:41:62; 10:00:00:05:33:90:a7:01

Effective configuration: cfg: cfg_DCX2 zone: LSAN_T2 50:06:01:6a:3e:a0:41:62 10:00:00:05:33:90:a7:01

Step 2: Configure FC routing.

Configure FC routing, which interconnects FC fabrics, whenever there are multiple fabrics in the SAN.

In this example, the 8510 has FC rouer blades and acts as the FC router.

DCX1. Enable FCRDCX1:root> fosconfig –enable fcrDCX:root> fosconfig --showFC Routing service: enabled iSCSI service: Service not supported on this PlatformiSNS client service: Service not supported on this PlatformVirtual Fabric: disabledEthernet Switch Service: disabledDCX:root>

DCX2. Enable FCRDCX2:root> fosconfig –enable fcrDCX2:root> fosconfig --showFC Routing service: enabled iSCSI service: Service not supported on this PlatformiSNS client service: Service not supported on this PlatformVirtual Fabric: disabledEthernet Switch Service: Service not supported on this PlatformDCX2:root>

Step 3: Configure EX_ports between both fabrics on the 8510.

DCX1. Enable FCRDCX1:root> portdisable 8/25DCX1:root>portcfgexport 8/25 -a 1 -f 101 -m 5 -d 60DCX1:root> portenable 8/25DCX:root> portcfgexport 8/25

Port 8/25 info Admin: enabled State: OKPid format: Not ApplicableOperate mode: Brocade NOSEdge Fabric ID: 101Front Domain ID: 60Front WWN: 50:00:53:3c:01:20:0e:65Principal Switch: 2Principal WWN: 10:00:00:05:33:86:d2:b6Fabric Parameters: Auto NegotiateR_A_TOV: 10000(N)E_D_TOV: 2000(N)Authentication Type: None



53-1003534-01


24/44

DH Group: N/A

Hash Algorithm: N/AEdge fabric's primary wwn: N/AEdge fabric's version stamp: N/A

DCX:root>

DCX:root> switchshowswitchName: DCXswitchType: 121.3switchState: OnlineswitchMode: NativeswitchRole: PrincipalswitchDomain: 1switchId: fffc01switchWwn: 10:00:00:05:33:c0:12:00zoning: ON (cfg_DCX1)switchBeacon: OFFFC Router: ONFC Router BB Fabric ID: 128Address Mode: 0

Index Slot Port Address Media Speed State Proto217 8 25 01d900 id N8 Online FC EX-Port10:00:00:05:33:b8:00:18

"TOR1" (fabric id = 101 )(Trunk master)E-Port 50:00:53:3c:01:2e:0f:0c "fcr_xd_2_101" 218 8 26 01da00 id N8 In_Sync FC Disabled 219 8 27 01db00 id N8 No_Light FC

DCX2. Enable FCRDCX2:root> portdisable 2DCX2:root>portcfgexport 2 -a 1 -f 101 -m 5 -d 61DCX2:root> portenable 2DCX2:root> portcfgexport 2CAT5100:root> portcfgexport 2 Port 2 infoAdmin: enabledState: OKPid format: Not ApplicableOperate mode: Brocade NOSEdge Fabric ID: 101Front Domain ID: 61Front WWN: 50:00:53:3d:e0:5a:5e:65Principal Switch: 2Principal WWN: 10:00:00:05:33:86:d2:b6Fabric Parameters: Auto NegotiateR_A_TOV: 10000(N)E_D_TOV: 2000(N)Authentication Type: NoneDH Group: N/AHash Algorithm: N/AEdge fabric's primary wwn: N/AEdge fabric's version stamp: N/A

DCX2:root>DCX2:root>switchshowswitchName: CAT5100switchType: 66.1switchState: OnlineswitchMode: NativeswitchRole: PrincipalswitchDomain: 10switchId: fffc0aswitchWwn: 10:00:00:05:33:de:05:a5zoning: ON (cfg_51001)switchBeacon: OFFFC Router: ONFC Router BB Fabric ID: 120Address Mode: 0

Index Port Address Media Speed State Proto============================================== 0 0 0a0000 id N8 Online FC F-Port 50:0a:09:85:8d:ae:5c:9c

1 1 0a0100 id N8 No_Light FC2 2 0a0200 id N8 Online FC EX-Port 10:00:00:05:33:ec:2a:d2



53-1003534-01


25/44

"TOR2" (fabric id = 101 )(Trunk master)E-Port 50:00:53:3d:e0:5c:5f:08 "fcr_xd_1_101"

3 3 0a0300 id N8 No_Light FC

Step 4: Configure and verify all other FC ports in the fabric.

The following shows how to configure and verify the other FC ports in the fabric.

DCX1. Enable FCRDCX1:root> portenable 8/25DCX:root> DCX:root> switchshowswitchName: DCXswitchType: 121.3switchState: OnlineswitchMode: NativeswitchRole: PrincipalswitchDomain: 1switchId: fffc01switchWwn: 10:00:00:05:33:c0:12:00zoning: ON (cfg_DCX1)switchBeacon: OFFFC Router: ONFC Router BB Fabric ID: 128Address Mode: 0

Index Slot Port Address Media Speed State Proto

238 8 46 01ee00 id 8G Online FC F-Port 50:06:01:6b:3e:a0:41:62DCX2. Enable FCRDCX2:root> portenable 11DCX2:root> switchshowswitchName: CAT5100switchType: 66.1switchState: OnlineswitchMode: NativeswitchRole: PrincipalswitchDomain: 10switchId: fffc0aswitchWwn: 10:00:00:05:33:de:05:a5zoning: ON (cfg_51001)switchBeacon: OFFFC Router: ONFC Router BB Fabric ID: 120Address Mode: 0

Index Port Address Media Speed State Proto============================================== 0 0 0a0000 id N8 Online FC F-Port 50:0a:09:85:8d:ae:5c:9c

1 1 0a0100 id N8 No_Light FC2 2 0a0200 id N8 Online FC EX-Port 10:00:00:05:33:ec:2a:d2

"TOR2"(fabric id = 101 )(Trunk master)

E-Port 50:00:53:3d:e0:5c:5f:08 "fcr_xd_1_101"3 3 0a0300 id N8 No_Light FC4 4 0a0400 id N8 No_Light FC5 5 0a0500 id N8 No_Light FC6 6 0a0600 id N8 No_Light FC7 7 0a0700 id N8 No_Light FC8 8 0a0800 id N8 No_Light FC9 9 0a0900 id N8 No_Light FC

10 10 0a0a00 id N8 No_Light FC Disabled 11 11 0a0b00 id N8 Online FC F-Port 50:06:01:6a:3e:a0:41:62

Configuring LUNs on the storage device

This example demonstrates storage configuration with the EMC VNX5300 storage array with 15.5 TB

total hard disk space, and total usable storage of 15.4 TB.

The VNX5300 supports both file and block data services, optimized for virtual applications with VMware

vSphere or Microsoft Hyper-V.

To configure the LUNs on the VNX5300, complete the following steps

Configuring LUNs on the storage device


53-1003534-01


26/44


27/44

FIGURE 12 Creating the LUN

The system launches a wizard to guide you through the process of creating the LUN.

FIGURE 13 Creating the LUN Wizard

Step 3: Click Next and enter the size of the LUN in the page that is displayed.



53-1003534-01


28/44

FIGURE 14 Sizing the LUN

Step 4: Choose an existing folder name for the LUN or create and assign a new folder.

FIGURE 15 Assigning a Folder to the LUN

Step 5: Click Finish to create the LUN.



53-1003534-01


29/44

FIGURE 16 Creating the LUN

Step 6: Confirm the LUN configuration details and note down unique LUN ID.

FIGURE 17 Confirming LUN Configuration

Note that unique LUN ID.



53-1003534-01


30/44


31/44

FIGURE 19 Login to the Server and Verify LUN Detection

Step 2: Verify that LUN ID matches with the device ID.

Step 3: Now assign this device to the host 10.17.0.95.

Step 4: To select the host, choose Edit Settings > Add > Harddisk > Raw device mappings.

FIGURE 20 Selecting the Host

Step 5: Choose the device to assign to this host.



53-1003534-01


32/44


33/44

Rack server and ToR connectivity to an FC SAN using an access

gateway

FIGURE 23 Rack server TOR connectivity to an FC SAN using an access gateway

NOTE

Figure 23 is the same as Figure 10 on page 14, but it has been repeated here to illustrate a different

use case.

FIGURE 24 Access gateway

Brocade Access Gateway (AG) is a feature provided by both NOS and FOS that lets you configure thefabric to handle additional N_Ports instead of domains. Switches in AG mode are logically transparent

to the host and the fabric. It increases the number of hosts that have access to the fabric without

increasing the number of switches in the fabric. This improves fabric scalability and simplifies

configuration and management in a large fabric by limiting the number of Domain IDs and ports.

In Figure 23 , ToR devices are configured as AGs. This causes the ToR device to appear like the

N_Port on an end node device in the SAN, so traffic is transmitted to and received only from the ToR

device, which then provides a gateway service to the hosts on the VCS NOS fabric to which it is

connected.

Rack server and ToR connectivity to an FC SAN using an access gateway


53-1003534-01


34/44

The AG feature is supported only on the VDX 6730 or 6740 switches with FC ports from NOS Release

4.1 and later.

Enable the AG feature on VDX 6730/40 platforms to configure FC ports as N_Ports and map specific

virtual fabric (VF) ports to the N_Ports. This allows direct connection of hosts attached to VF_Ports on

the VDX switch with F_Ports on a FC fabric edge switch instead of through ISL connections from a

VDX 6730 to the FC router.

AG configuration changes are required on both the VCS fabric and the SAN. The rest of the

configuration is the same as the first example. The following is a summary of the special configuration

required.

Step 1: Configure the VCS server network.

Step 2: Configure the FC SAN.

Step 3: Configure LUNs on the storage devices.

Refer to the section Configuring LUNs on the storage device on page 25 for details.

Step 4: Map the LUNs on the server.

Refer to the section Mapping LUNs on the server on page 30 for details.

To configure the AG feature, complete the following steps.

Procedure


On the VCS fabric, configure the ToR VDX switch as an access gateway.

TOR1# ag enable ?Possible completions: | TOR1# ag enable

WARNING: Access Gateway mode changes the standard behavior of the switch.Please check Access Gateway Administrator's Guide before proceeding.

Enabling agmode will remove all the configuration data on the switchincluding zoning configuration and security database. Please backup yourconfiguration.

Enabling Access Gateway mode on the switch will reboot the switch. Do you want tocontinue?[y/n]:y

Repeat the configuration on the redundant ToR switch.

TOR2. Access gateway ConfigurationTOR2# ag enable

WARNING: Access Gateway mode changes the standard behavior of the switch.Please check Access Gateway Administrator's Guide before proceeding.Enabling agmode will remove all the configuration data on the switchincluding zoning configuration and security database. Please backup your

configuration.

Enabling Access Gateway mode on the switch will reboot the switch. Do you want tocontinue?[y/n]:y

The devices reboot and come up as an access gateway.

Verify that both ToR devices are running as access gateways.

TOR1. Access Gateway verificationTOR1# show agRbridge-ID 3:



53-1003534-01


35/44

----------------------------------------------------------------------------------------Name : TOR1NodeName : 10:00:00:05:33:b8:00:18Number of Ports : 76IP Address(es) : 10.17.0.111Firmware Version : v4.1.0_bld27

Number of N_Ports(Fi) : 1Number of VF_Ports : 1Policies Enabled : pgPersistent ALPA : DisabledPort Group information : PG_ID PG_Name PG_Mode PG_Members ---------------------------------------------------------- 0 pg0 lb 3/0/1, 3/0/2, 3/0/3, 3/0/4,

3/0/5, 3/0/6, 3/0/7, 3/0/8,3/0/9, 3/0/10, 3/0/11, 3/0/12,3/0/13, 3/0/14, 3/0/15, 3/0/16

----------------------------------------------------------Fabric Information : Attached Fabric Name N_Ports(Fi) ---------------------------------------------------------- 10:00:00:05:33:c0:12:00 3/0/1 ----------------------------------------------------------

N_Port(Fi) information :

Port PortID Attached PWWN IP_Addr VF_Ports ------------------------------------------------------------------------------------ Fi 3/0/1 0x01d900 20:d9:00:05:33:c0:12:00 10.17.0.131 1/3/1 ------------------------------------------------------------------------------------

VF_Port information : VF_Port Eth_Port PortID Attached PWWN N_Port(Fi) ------------------------------------------------------------------------------------ 1/3/1 Te 3/0/10 0x01d901 10:00:00:05:33:90:a7:00 3/0/1

------------------------------------------------------------------------------------TOR1#

Repeat this command on the second ToR switch.

TOR2.TOR2# show agRbridge-ID 4:----------------------------------------------------------------------------------------Name : TOR2NodeName : 10:00:00:05:33:ec:2a:d2Number of Ports : 76IP Address(es) : 10.17.0.112Firmware Version : v4.1.0_bld27Number of N_Ports(Fi) : 1Number of VF_Ports : 1Policies Enabled : pgPersistent ALPA : DisabledPort Group information : PG_ID PG_Name PG_Mode PG_Members ---------------------------------------------------------- 0 pg0 lb 4/0/1, 4/0/2, 4/0/3, 4/0/4,

4/0/5, 4/0/6, 4/0/7, 4/0/8,4/0/9, 4/0/10, 4/0/11, 4/0/12,

4/0/13, 4/0/14, 4/0/15, 4/0/16

----------------------------------------------------------Fabric Information : Attached Fabric Name N_Ports(Fi) ---------------------------------------------------------- 10:00:00:05:33:de:05:a5 4/0/2 ----------------------------------------------------------

N_Port(Fi) information : Port PortID Attached PWWN IP_Addr VF_Ports ------------------------------------------------------------------------------------ Fi 4/0/2 0x0a0200 20:02:00:05:33:de:05:a5 10.17.0.134 1/4/1



53-1003534-01


36/44

------------------------------------------------------------------------------------

VF_Port information : VF_Port Eth_Port PortID Attached PWWN N_Port(Fi) ------------------------------------------------------------------------------------ 1/4/1 Te 4/0/10 0x0a0201 10:00:00:05:33:90:a7:01 4/0/2

------------------------------------------------------------------------------------TOR2#

NOTE

N_port and VF_Port information does not appear until SAN configuration is completed, as described

later.

Step 2: Configure the SAN.

This example uses a SAN an 8510 device facing the VCS, which must be configured as a regular

F_port as shown below.

DCX1. Enable FCRDCX1:root>portdisable 8/25DCX1:root> portcfgexport 8/25 -a 2DCX1:root>portenable 8/25DCX:root> switchshowswitchName: DCXswitchType: 121.3switchState: OnlineswitchMode: NativeswitchRole: PrincipalswitchDomain: 1switchId: fffc01switchWwn: 10:00:00:05:33:c0:12:00zoning: ON (cfg_DCX1)switchBeacon: OFFFC Router: ONFC Router BB Fabric ID: 128Address Mode: 0

Index Slot Port Address Media Speed State Proto217 8 25 01d900 id N8 Online FC F-Port 1 N Port + 1 NPIV publicDCX2. Enable FCRDCX2:root> portdisable 2DCX2:root> portcfgexport 2 -a 2DCX2:root>portenable 2DCX2:root> switchshowswitchName: CAT5100switchType: 66.1switchState: OnlineswitchMode: NativeswitchRole: PrincipalswitchDomain: 10switchId: fffc0aswitchWwn: 10:00:00:05:33:de:05:a5zoning: ON (cfg_51001)switchBeacon: OFFFC Router: ONFC Router BB Fabric ID: 120Address Mode: 0

Index Port Address Media Speed State Proto============================================== 0 0 0a0000 id N8 Online FC

1 1 0a0100 id N8 No_Light FC2 2 0a0200 id N8 Online FC F-Port 1 N Port + 1 NPIV public







53-1003534-01


37/44

FC/FCoE storage connectivity from ToR rack server and ToR

connectivity to storage

FIGURE 25 FC/FCoE storage connectivity from ToR, Rack server and ToR connectivity to storage

With NOS release 4.1 and later, storage devices can be directly connected to FC interfaces of 6730.

With NOS release 5.0 and later, you can also use the VDX 6740.

In this use case there is no need for a separate SAN. VCS interconnects servers and using FCoE

interfaces and storage devices using FC interfaces. Because there is only one fabric, zoning

configuration is not required, although zoning is supported for access control.

The following are the steps required to implement this use case.


• Verify the ISL status on all the devices.

• Verify that VCS fabric is formed on all devices.

• Configure FCoE on the edge interfaces.

• Configure and verify FC interfaces on the TOR1 and TOR2 devices.

Step 2: Configure LUNs on the storage device.




Procedure

To configure this example of converged network configuration, complete the following steps.


Bring up the VCS between TOR1, TOR2, EOR1, and EOR2 as shown below.

TOR1TOR2

FC/FCoE storage connectivity from ToR, rack server, and ToR connectivity to storage


53-1003534-01


38/44

TOR1# vcs vcsid 1 enableThis operation will change the configuration to default and reboot the switch. Doyou want to continue? [y/n]:y….TOR1#TOR1# conf tEntering configuration mode terminal

TOR1(config)# rbridge-id 3TOR1(config-rbridge-id-3)#

TOR2# vcs vcsid 1This operation will change the configuration to default and reboot the switch. Doyou wantto continue? [y/n]:y…..TOR2#TOR2# conf tEntering configuration mode terminalTOR2(config)# rbridge-id 4TOR2(config-rbridge-id-4)#

EOR1EOR2EOR1# vcs vcsid 1 enableThis operation will change the configuration to default and reboot the switch. Doyou want to continue? [y/n]:y….EOR1#EOR1# conf tEntering configuration mode terminalEOR1(config)# rbridge-id 1EOR1(config-rbridge-id-3)#

EOR2# vcs vcsid 1This operation will change the configuration to default and reboot the switch. Doyou want to continue? [y/n]:y…..EOR2#EOR2# conf tEntering configuration mode terminalEOR2(config)# rbridge-id 4EOR2(config-rbridge-id-4)#

Brocade VCS fabric forms adjacencies with directly connected Brocade VCS enabled neighboring

switches. These adjacencies are called ISLs. These ISLs can be separate links or Brocade proprietary

VCS Fabric Trunks. By default all interfaces are configured to detect ISL(Inter Switch Links) links/Trunks. In this use case all the ISLs are trunks. Verify if all the ISLs have come up and formed

adjacencies with neighboring devices.

Verify the ISL status on all the devices by entering the following commands:EOR1EOR1# show fab isl


Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BWTrunk Nbr-Name----------------------------------------------------------------------------------------------13 Te 1/0/14 17 Te 3/0/2 10:00:00:05:33:B8:00:18 20G

Yes"TOR1"14 Te 1/0/15 18 Te 4/0/3 10:00:00:05:33:EC:2A:D2 10GYes"TOR2"

EOR1#EOR2EOR2# show fabric isl


Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BW



53-1003534-01


39/44

Trunk Nbr-Name----------------------------------------------------------------------------------------------13 Te 2/0/14 17 Te 4/0/2 10:00:00:05:33:EC:2A:D2 20G Yes"TOR2"

14 Te 2/0/15 18 Te 3/0/3 10:00:00:05:33:B8:00:18 20G Yes"TOR1"

EOR2#TOR1TOR1# show fabric isl


Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BW TrunkNbr-Name----------------------------------------------------------------------------------------------17 Te 3/0/2 13 Te 1/0/14 10:00:00:05:33:A2:EB:B7 20G Yes"EOR1"18 Te 3/0/3 14 Te 2/0/15 10:00:00:05:33:86:D2:B6 20G Yes"EOR2"

TOR1#TOR2TOR2# show fabric isl


Src Src Nbr NbrIndex Interface Index Interface Nbr-WWN BW TrunkNbr-Name----------------------------------------------------------------------------------------------17 Te 4/0/2 13 Te 2/0/14 10:00:00:05:33:86:D2:B6 20G Yes"EOR2"18 Te 4/0/3 14 Te 1/0/15 10:00:00:05:33:A2:EB:B7 10G Yes"EOR1"

TOR2#

Verify that VCS fabric is formed on all the devices by entering the following commands:

EOR1EOR1# show vcsConfig Mode : Local-OnlyVCS Mode : Fabric Cluster

VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabricStatus HostName--------------------------------------------------------------------------------------------------------------1 10:00:00:05:33:A2:EB:B7* 10.17.0.74 OnlineOnlineEOR12 >10:00:00:05:33:86:D2:B6 10.17.0.75 Online

OnlineEOR23 10:00:00:05:33:B8:00:18 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2 10.17.0.112 OnlineOnlineTOR2EOR1#

Note: “>10:00:00:05:33:86:D2:B6” listed above is the principal switch.EOR2EOR2# show vcsConfig Mode : Local-Only



53-1003534-01


40/44

VCS Mode : Fabric Cluster VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabricStatus HostName-------------------------------------------------------------------------------------

-------------------------1 10:00:00:05:33:A2:EB:B7 10.17.0.74 OnlineOnlineEOR12 >10:00:00:05:33:86:D2:B6* 10.17.0.75 OnlineOnlineEOR23 10:00:00:05:33:B8:00:18 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2 10.17.0.112 OnlineOnlineTOR2EOR2#TOR1TOR1# show vcsConfig Mode : Local-OnlyVCS Mode : Fabric Cluster

VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabricStatus HostName--------------------------------------------------------------------------------------------------------------1 10:00:00:05:33:A2:EB:B7 10.17.0.74 OnlineOnlineEOR12 >10:00:00:05:33:86:D2:B6 10.17.0.75 OnlineOnlineEOR23 10:00:00:05:33:B8:00:18* 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2 10.17.0.112 OnlineOnlineOR2TOR1#TOR2TOR2# show vcsConfig Mode : Local-OnlyVCS Mode : Fabric Cluster

VCS ID : 1Total Number of Nodes : 4Rbridge-Id WWN Management IP VCS StatusFabricStatus HostName--------------------------------------------------------------------------------------------------------------1 10:00:00:05:33:A2:EB:B7 10.17.0.74 OnlineOnlineEOR12 >10:00:00:05:33:86:D2:B6 10.17.0.75 OnlineOnlineEOR23 10:00:00:05:33:B8:00:18 10.17.0.111 OnlineOnlineTOR14 10:00:00:05:33:EC:2A:D2* 10.17.0.112 OnlineOnlineTOR2TOR2#

Configure FCoE on the edge interfaces by entering the following commands:

TOR1. FCoE configTOR1# show running-config interface TenGigabitEthernet 3/0/10interface TenGigabitEthernet 3/0/10 fabric isl enable



53-1003534-01


41/44

fabric trunk enable edge-loop-detection vlan 0 fcoeport default no shutdown!TOR1#TOR2. CoE config

TOR2# show running-config interface TenGigabitEthernet 4/0/10interface TenGigabitEthernet 4/0/10 fabric isl enable fabric trunk enable edge-loop-detection vlan 0 fcoeport default no shutdown!TOR2#

Currently, FCoE VLANs can be only 802.1Q VLANs. They cannot be classified or used as C-TAGs for

other VLAN classification. All tenant FCoE traffic is carried on the same default FCoE VLAN (1002) as

in previous NOS releases.

TOR1. FCoE loginTOR1# show fcoe fabric-map default==================================================================================================

Fabric-Map VLAN VFID Pri FCMAP FKA Timeout Total-FCoE-Enodes==================================================================================================default 1002[D] 128[D] 3[D] 0xefc00[D] 8000[D] Enabled[D]64[D]

Total number of Fabric Maps = 1TOR1#TOR1#TOR1#TOR1# show fcoe login =============================================================================================FCOE-Port Eth-port Device WWN Device MAC SessionMAC

=============================================================================================Fcoe 1/3/1 Te 3/0/10 10:00:00:05:33:90:a7:00 00:05:33:90:a7:00 0e:fc:00:03:04:00

Total number of Logins = 1TOR1#

TOR1# show name-server detailPID: 030400


42/44

Enodes==================================================================================================default 1002[D] 128[D] 3[D] 0xefc00[D] 8000[D] Enabled[D]64[D]

Total number of Fabric Maps = 1TOR2#TOR2# show name-server detailPID: 040400 switchshowName: sw0Type: 107.6State: OnlineswitchMode: NativeRole: Fabric Subordinate



53-1003534-01


43/44

Rbridge-id: 3switchId: fffc03WWN: 10:00:00:05:33:b8:00:18zoning: OFFswitchBeacon: OFFFC Router: OFFFC Router BB Fabric ID: 128

Address Mode: 0Index Port Address Media Speed State Proto============================================== 0 0 0303c0 id N8 Online FC E-Port 50:00:53:3c:01:20:0e:65"fcr_fd_60"(downstream)(Trunk Primary)1 1 030340 id N8 No_Sync FC Disabled (Persistent)2 2 0302c0 -- N8 No_Module FC3 3 030240 id N8 Online FC F-Port 50:06:01:69:3e:a0:41:62 4 4 030380 -- N8 No_Module FC5 5 030300 -- N8 No_Module FC6 6 030280 -- N8 No_Module FC7 7 030200 -- N8 No_Module FC8 8 0301c0 -- N8 No_Module FC9 9 030140 -- N8 No_Module FC10 10 0300c0 -- N8 No_Module FC11 11 030040 -- N8 No_Module FC

TOR2. FCoE loginTOR2# show running-config interface FibreChannel 4/0/4interface FibreChannel 4/0/4 desire-distance 0 no isl-r_rdy trunk-enable no shutdown!TOR2# show interface Fibrechannel 4/0/4fibrechannel 4/0/4 is up (In_Sync). Protocol state is up (connected).Pluggable media presentLineSpeed Actual: 8G AutoPortSpeed: N8GbpsportDisableReason: NonePortId: 041400PortIfId: 43011401PortWwn: 20:03:00:05:33:ec:2a:d2Distance: normal

Last clearing of show interface counters: 00:00:00

Interrupts: 0 Link_failure: 2 Frjt: 0Unknown: 0 Loss_of_sync: 7 Fbsy: 0Lli: 551 Loss_of_sig: 13Proc_rqrd: 44 Protocol_err: 0Timed_out: 0 Invalid_word: 30Rx_flushed: 0 Invalid_crc: 0Tx_unavail: 0 Delim_err: 0Free_buffer: 0 Address_err: 0Overrun: 0 Lr_in: 3Suspended: 0 Lr_out: 3Parity_err: 0 Ols_in: 32_parity_err: 0 Ols_out: 3CMI_bus_err: 0

Rate info: Bandwidth: 8.00G Tx performance: 0 B/sec Rx performance: 0 B/secTOR2#

Also login as root and verify that FC interface has detected the peers wwn.

TOR2:root> switchshowName: sw0Type: 107.6State: OnlineswitchMode: NativeRole: Fabric SubordinateRbridge-id: 4switchId: fffc04WWN: 10:00:00:05:33:ec:2a:d2zoning: OFFswitchBeacon: OFF



53-1003534-01


44/44

FC Router: OFFFC Router BB Fabric ID: 128Address Mode: 0

Index Port Address Media Speed State Proto============================================== 0 0 0403c0 id N8 Online FC E-Port 50:00:53:3d:e0:5a:5e:65

"fcr_fd_61"(downstream)(Trunk Primary)1 1 040340 id N8 Online FC E-Port (Trunk port, Primary is Port

0 ) 2 2 0402c0 -- N8 No_Module FC Disabled (Persistent)3 3 041400 id N8 Online FC F-Port 50:06:01:6f:3e:a0:41:62






brocade storage

Documents