brocade fabric technology with the pure storage fa-420 ...€¦ · 4 brocade fabric technology with...

53-1003803-01May 2015

Brocade Fabric Technologywith the Pure StorageFA-420Validation Test Report

Supporting Fabric OS v7.3.1

© 2015, Brocade Communications Systems, Inc. All Rights Reserved.

ADX, Brocade, Brocade Assurance, the B-wing symbol, DCX, Fabric OS, HyperEdge, ICX, MLX, MyBrocade, OpenScript, The EffortlessNetwork, VCS, VDX, Vplane, and Vyatta are registered trademarks, and Fabric Vision and vADX are trademarks of BrocadeCommunications Systems, Inc., in the United States and/or in other countries. Other brands, products, or service names mentioned may betrademarks of others.

Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning anyequipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this documentat any time, without notice, and assumes no responsibility for its use. This informational document describes features that may not becurrently available. Contact a Brocade sales office for information on feature and product availability. Export of technical data contained inthis document may require an export license from the United States government.

The authors and Brocade Communications Systems, Inc. assume no liability or responsibility to any person or entity with respect to theaccuracy of this document or any loss, cost, liability, or damages arising from the information contained herein or the computer programs thataccompany it.

The product described by this document may contain open source software covered by the GNU General Public License or other opensource license agreements. To find out which open source software is included in Brocade products, view the licensing terms applicable tothe open source software, and obtain a copy of the programming source code, please visit http://www.brocade.com/support/oscd.

http://www.brocade.com/support/oscd

Contents

Preface..................................................................................................................................... 5Overview........................................................................................................... 5Purpose of This Document................................................................................5Audience........................................................................................................... 5Objectives......................................................................................................... 6Test Conclusions...............................................................................................6Related Documents...........................................................................................6Document History..............................................................................................6About Pure Storage...........................................................................................7About Brocade.................................................................................................. 7

Test Plan...................................................................................................................................9Scope................................................................................................................ 9Test Configuration...........................................................................................10DUT Descriptions............................................................................................ 10DUT Specifications..........................................................................................11Test Equipment............................................................................................... 12Configure Equipment...................................................................................... 12

Step 1: Brocade FC Fabric Configuration........................................... 12Step 2: Pure Storage Array Configuration...........................................14Step 3: Host Setup.............................................................................. 16

Test Cases.............................................................................................................................. 21Fabric Initialization – Base Functionality......................................................... 22

Storage Device – Physical and Logical Login with SpeedNegotiation.................................................................................... 22

Zoning and LUN Mapping................................................................... 23Storage Device Fabric IO Integrity...................................................... 23Storage Device Multipath Configuration – Path integrity.....................24

Fabric – Advanced Functionality..................................................................... 24Storage Device Bottleneck Detection using MAPS FPI – w/

Congested Host.............................................................................25Storage Device Bottleneck Detection using MAPS FPI – w/

Congested Fabric.......................................................................... 26Storage Device – QoS Integrity...........................................................26Storage Device – FC Protocol Jammer Test Suite............................. 27

Stress and Error Recovery with Device Multi-Path......................................... 28Storage Device Fabric IO integrity – Congested Fabric......................28Storage Device Nameserver Integrity – Device Recovery with

Port Toggle.................................................................................... 28Storage Device Nameserver Integrity – Device Recovery with

Device Relocation..........................................................................29Storage Device – Device Recovery with Device Port Toggle –

Extended Run................................................................................30Storage Device Recovery – ISL Port Toggle ..................................... 31Storage Device Recovery – ISL Port Toggle (entire switch)............... 32Storage Device Recovery – Director Blade Maintenance................... 33Storage Device Recovery – Switch Offline......................................... 34

Brocade Fabric Technology with the Pure Storage FA-420 Validation Test Report 353-1003803-01

Storage Device Recovery – Switch Firmware Download..................35Storage Device – Fibre Channel Routing (FCR) InterNetworking Tests.......36

Storage Device InterNetworking Validation with FC host................. 36Storage Device InterNetworking Validation w/FCoE Test using

VDX FlexPort...............................................................................37Storage Device Edge Recovery after FCR Disruptions.................... 38Storage Device Backbone Recovery after FCR Disruptions.............39

Optional/Additional Tests.............................................................................. 40Storage Device firmware update.......................................................40Performance I/O testing with varying workload dimensions............. 40VMWare IOAnalyzer workload test................................................... 41

4 Brocade Fabric Technology with the Pure Storage FA-420 Validation Test Report53-1003803-01

Preface

● Overview........................................................................................................................... 5● Purpose of This Document................................................................................................5● Audience........................................................................................................................... 5● Objectives......................................................................................................................... 6● Test Conclusions...............................................................................................................6● Related Documents...........................................................................................................6● Document History..............................................................................................................6● About Pure Storage...........................................................................................................7● About Brocade.................................................................................................................. 7

OverviewThe Solid State Ready (SSR) program is a comprehensive testing and configuration initiative to provideFibre Channel SAN and IP interoperability with flash storage. This program provides testing of multiplefabrics, heterogeneous servers, NICs, and HBAs in large port-count Brocade environments. Testingcovers future Brocade OS versions and vendor’s software releases.

The SSR qualification program will help verify seamless interoperability and optimum performance withsolid state storage in Brocade SAN fabrics.

Purpose of This DocumentThis document provides the validation of Brocade fabric technology with the Pure Storage FA-420 all-flash storage array, using multiple switch platforms, HBAs, and server operating systems. Thisvalidation shows that the FA-420 interoperates properly within a Brocade Fibre Channel fabric, whilesupporting the performance and low latency associated with solid state storage.

AudienceThe content in this document is written for a technical audience, including solution architects, systemengineers, and technical development representatives.


Objectives1. Test the Pure FA-420 array with Brocade FC fabrics, in single and routed configurations for different

stress and error recovery scenarios, to validate the interoperability and integration of the array withBrocade FC fabrics.

2. Validate the performance of FC fabric in a solid state storage environment for high throughput andlow latency applications.

Test Conclusions1. Achieved 100% pass rate on all the test cases in the SSR qualification test plan. The network and

the storage were able handle the various stress and error recovery scenarios without any issues.2. Different I/O workload scenarios were simulated using Medusa and VMware IOAnalyzer tools and

sustained performance levels were demonstrated across all workload types.3. The results confirm that the Pure Storage FA-420 array interoperates seamlessly with Brocade

Fibre Channel fabrics, and together demonstrate high availability, performance, and low latency.4. It is recommended to enable the Monitoring and Alerting Policy Suite (MAPS) heath monitor on all

switches in the FC fabric to report fabric-wide events and traffic performance metrics. The additionalMAPS feature of Fabric Performance Impact monitoring should also be enabled to detectbottlenecks in the form of timeouts and latency. Using MAPS is recommended to maximize thebenefit of high performance low-latency storage.

5. For optimal availability and performance, consideration should be given to multipath configurationon the host side. While Windows 2008 and 2012 will provide Round-Robin behavior by default,Linux systems will benefit from adding a custom entry to /etc/multipath.conf, and VMWare hostssystems should be changed from the default ‘Most Recently Used (VMWare)’ setting to ‘Round-Robin (VMWare)’. Actively using all available paths provides a significant improvement inperformance throughput.

6. The Brocade Gen5 16Gb FC switches were able to handle the sustained throughput and latencyperformance requirements efficiently with fewer ISL trunks. Multiple ISL’s to different switches in thefabric should be setup for providing path redundancy through the fabric.

Related Documents• Fabric OS Administrator’s Guide v7.3.0• Brocade SAN Design and Best Practices• Brocade SAN Fabric Administration Best Practices Guide• Monitoring and Alerting Policy Suite Administrator’s Guide, v7.3.0• Network OS Layer 2 Switching Configuration Guide v5.0

Document History

Date Version Description

Objectives


http://www.brocade.com/downloads/documents/product_manuals/B_SAN/FOS_AdminGd_v730.pdf

http://www.brocade.com/downloads/documents/best_practice_guides/san-design-best-practices.pdf

http://www.brocade.com/downloads/documents/best_practice_guides/san-admin-best-practices-bp.pdf

http://www.brocade.com/forms/getFile?p=documents/product_manuals/B_SAN/FOS_MAPS_AdminGd_v730.pdf

http://www.brocade.com/downloads/documents/product_manuals/B_VDX/NOS_L2Guide_v501.pdf

May 2015 1.0 Initial version

About Pure StoragePure Storage has a simple mission: to enable the widespread adoption of flash in the enterprise datacenter. We're a team of some of the brightest minds in storage, flash memory and related technicalindustries. Founded on principles of teamwork and knowledge sharing, we focus on producing anexceptional result while we transform the landscape of the enterprise storage market (and have somefun along the way).

About BrocadeBrocade®networking solutions empower the world's leading organizations to transition smoothly to aworld where applications and information reside anywhere. By delivering agility and innovation forcloud-based environments, Brocade helps organizations modernize their networks and accelerate theirjourney to the New IP.

In particular, Brocade solutions for storage networking, data center routing, Software-DefinedNetworking (SDN), and Network Functions Virtualization (NFV) give organizations the power tocapitalize on the unique business opportunities driven by virtualization and the cloud.

To deliver a best-in-class solution, Brocade partners with world-class IT companies around the globe.www.brocade.com.

About Pure Storage


HTTP://WWW.BROCADE.COM

HTTP://WWW.BROCADE.COM

About Brocade


Test Plan

● Scope................................................................................................................................ 9● Test Configuration...........................................................................................................10● DUT Descriptions............................................................................................................ 10● DUT Specifications..........................................................................................................11● Test Equipment............................................................................................................... 12● Configure Equipment...................................................................................................... 12

The storage array is connected to two SAN fabrics and multiple server hosts to drive IO in a multipathconfiguration. Error injection is introduced, and failover & recovery behaviors are observed. IOperformance is observed across different workload configurations.

ScopeTesting will be performed with a mix of GA and development versions of Brocade’s Fabric OperatingSystem (FOS) in a heterogeneous environment. Test beds will include Brocade directors and switchesconfigured in routed and non-routed fabric configurations.

Testing is centered on interoperability and optimal configuration. Performance is observed within thecontext of best practice fabric configuration; however absolute maximum benchmark reporting ofstorage performance is beyond the scope of this test.

Details of the test steps are covered under “Test Case Descriptions” section. Standard test bed setupincludes IBM/HP/Dell chassis server hosts with Brocade/QLogic/Emulex HBAs with two uplinks fromevery host to a Brocade FC fabric. IO generators included Medusa Labs Test Tools and VMWareIOAnalyzer.


Test ConfigurationFIGURE 1 Test Configuration

DUT DescriptionsThe following tables provide details about the devices under test (DUT)

Storage ArrayTABLE 1

DUT ID Model Vendor Description

Pure StorageFA-420

FA-420 Pure Storage Pure Storage FA-420

The Pure Storage FA-420 flash storagearray is an all-flash array that supports11-35 TB raw capacity. Each controllersupports 4x 8Gb Fibre Channelconnections, 2x 10Gb iSCSI connections,and 2x Infiniband connections.

SwitchTABLE 2

DUT ID Model Vendor Description

Test Configuration


Switch (Continued)TABLE 2

6510-1..9 BR-6510 Brocade 48 port 16Gb FC switch

DCX-1 DCX 8510-8 Brocade 8 slot 16Gb FC chassis

DCX-2 DCX 8510-4 Brocade 4 slot 16Gb FC chassis

VDX-1,2 VDX 6740 Brocade 48 port 10Gb switch (48x10Gb/4x40Gb)

DUT Specifications

Storage Version

Pure Storage FA-420 solid state flash array Purity version 3.4.8

Brocade switches Version

DCX 8510-8 FOS 7.3.1 development and release

DCX 8510-4 FOS 7.3.1 development and release

6510 + Integrated Routing, Fabric Vision Licenses FOS 7.3.1 development and release

VDX 6740 NOS 5.0.1

Adapters Version

Brocade 1860 2-port 16Gb FC HBA Driver and firmware version 3.2.5.1

QLogic QLE2672 2-port 16GB FC HBA Driver 8.07.00.16, firmware 7.04.01

QLogic QLE2562 2-port 8GB FC HBA Driver 8.07.00.16, firmware 7.01.00

Emulex LPe 16202-X 2-port 16Gb FC HBA Driver 10.2.370.8, firmware 1.1.55.0

Brocade 1020 2-port CNA adapter Driver and firmware version 3.2.4.0

DUT ID Servers RAM Processor OS

SRV-1 HP Proliant DL380P G8 160GB Intel Xeon E5-2640 VMWare 5.5 [cluster]

SRV-2 HP Proliant DL380P G8 160GB Intel Xeon E5-2640 VMWare 5.5 [cluster]

SRV-3 IBM System x3630 M4 24GB Intel Xeon E5-2420 Windows Server 2012 R2

DUT Specifications


SRV-4 HP Proliant DL380P G8 64GB Intel Xeon E5-2640 SLES 12.0 x86_64

SRV-5 Dell Poweredge R720 160GB Intel Xeon E5-2640 RHEL 7.0 x86_64

SRV-6 HP Proliant DL385p G8 16GB AMD Opteron 6212 Windows Server 2012

SRV-7 Dell Poweredge R720 16GB Intel Xeon E5-2620 SLES 11.3 x86_64

SRV-8 IBM System x3630 M4 16GB Intel Xeon E5-2620 RHEL 6.5 x86_64

Test Equipment

Device/Software Tools Version

Finisar 16Gb Analyzer/Jammer XGIG5K2001153

Medusa Labs Test Tools 6.0.1.156759

VMWare IOAnalyzer 1.6.0

Configure EquipmentThe steps for configuration of the test bed are covered here.

Step 1: Brocade FC Fabric ConfigurationConfiguration settings of the Brocade switches in the test bed are covered here.

1. Create zones for each host initiator groupExample zone:< ========== >> zoneshow hb067168_pure zone: hb067168_pure10:00:8c:7c:ff:24:a0:00; 10:00:8c:7c:ff:24:a0:01;52:4a:93:7d:f3:5f:61:00; 52:4a:93:7d:f3:5f:61:01;52:4a:93:7d:f3:5f:61:10; 52:4a:93:7d:f3:5f:61:11;< ========== >

2. Configure MAPS on switches, which requires a Fabric Vision License. This will enable reporting oflatency and congestion alerts on each switch, as well as a number of switch and fabric healthmetrics. For more information on configuring and using MAPS, seeMonitoring and Alerting PolicySuite Administrator’s Guide, v7.3.0.

Enable the desired MAPS policy using any of the available default policies or create a custompolicy. For this test the default ‘Aggressive Policy’ is used. This provides the most sensitivethreshold levels for detection of latency and congestion.< ========== > > mapsconfig --enablemaps -policy dflt_aggressive_policy< ========== >Define the reporting actions MAPS will take. In this test, errors discovered by

Test Equipment




MAPS will generate a RASLog entry and send email to the configured recipients.< ========== >> mapsconfig --actions raslog,email< ========== >The Fabric Performance Impact (FPI) category within MAPS monitors the current condition of thelatency seen on F_Ports over different time windows and uses that to determine the performanceimpact to the fabric and network.

In order to use FPI monitoring, legacy 'bottelneckmon' must be disabled:< ========== >> bottleneckmon –disable< ========== >Enable FPI monitoring:< ========== >> mapsConfig –enableFPImon< ========== >To examine the effective MAPS configuration, use this command:< ========== >> mapsconfig –showConfigured Notifications: RASLOG,EMAILMail Recipient: [email protected],[email protected] Monitoring: EnabledPaused members :===============PORT :CIRCUIT :SFP :< ========== >Use this command to view the summary of the events or rules triggered, and the objects on which therules were triggered over a specified period of time.< ========== >> mapsdb –show1 Dashboard Information:=======================DB start time: Wed Dec 17 20:59:09 2014Active policy: dflt_aggressive_policyConfigured Notifications: RASLOG,EMAILFenced Ports : NoneDecommissioned Ports : None< ========== >

3. Configure FlexPorts on VDX switches for Fibre Channel support (FCoE Base License is required).

These ports are used as uplinks from VCS fabric to the Fibre Channel fabric in this test. For detailedinformation on configuring VDX6740 switches for Fibre Channel support, see Network OS Layer 2Switching Configuration Guide v5.0.

Example of configuring a Fibre Channel port on a Brocade VDX6740 switch:< ========== ># conf t(config)# hardware(config-hardware)# flexport 112/0/38(config-flexport-112/0/38)# type fibre-channel< ========== >FlexPort speeds are configured in connector-groups. In this test bed example, the speed ‘HighMixed’is chosen. This will allow the Fibre Channel FlexPort to operate at 16Gb speed, which is optimalbecause the uplink is attaching to a 16Gb Fibre Chanel SAN. Also, not all of the ports in thisconnector group are configured as Fibre Channel ports; some are still configured as Ethernet ports,and ‘HighMixed’ setting will support the Ethernet speeds on these ports as well. For detailedinformation on configuring VDX6740 connector group speeds, see Network OS Layer 2 SwitchingConfiguration Guide.

Test Plan






• LowMixed: 2/4/8G Fibre Channel, and Ethernet speeds (default)• HighMixed: 16G Fibre Channel, and Ethernet speeds• FibreChannel: 2/4/8/16G Fibre Channel

Use this command to set the connector-group speed to HighMixed:< ========== >(config-hardware)# connector-group 111/0/3(config-connector-group-111/0/3)# speed HighMixed< ========== >

4. Configure zones for FCoE initiators on VDX switches.

Example zone on Brocade VDX6740 switch:< ========== ># show zoning enabled-configurationzoning enabled-configuration cfg-name NOS_SSRzoning enabled-configuration enabled-zone lsan_hb067166_pure member-entry 10:00:8c:7c:ff:1f:7b:00 member-entry 10:00:8c:7c:ff:1f:7b:01 member-entry 52:4a:93:7d:f3:5f:61:00 member-entry 52:4a:93:7d:f3:5f:61:01 member-entry 52:4a:93:7d:f3:5f:61:10 member-entry 52:4a:93:7d:f3:5f:61:11< ========== >

5. Configure Fibre Channel Routing. (An Integrated Routing license is required). Detailed informationon FCR setup can be found in the Fabric OS Administrator's Guide v7.3.0.

Example FCR configuration is shown below:< ========== >> fcrconfigure –bbfid 100> fosconfig --enable fcr> portcfgexport [port#] -a1 –m[0/5] -f 10 {0=Brocade FC fabric; 5=Brocade NOS fabric}< ========== >The prefix 'lsan' is used when configuring zones for use in Fibre Channel Routing.> zoneshow lsan_hb067166_pure zone: lsan_hb067166_pure 10:00:8c:7c:ff:23:b7:00; 10:00:8c:7c:ff:23:b7:01; 52:4a:93:7d:f3:5f:61:00; 52:4a:93:7d:f3:5f:61:01 52:4a:93:7d:f3:5f:61:10; 52:4a:93:7d:f3:5f:61:11Example of zone prefixed with 'lsan':> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PIDin Fabric in Fabric---------------------------------------------------------------------------- 10 21:00:00:24:ff:48:b9:6a 02f001 20 551a00 Imported 10 21:00:00:24:ff:48:b9:6b 02f101 20 541e00 Imported 10 52:4a:93:7d:f3:5f:61:00 02f201 20 550e00 Imported 10 52:4a:93:7d:f3:5f:61:01 02f401 20 540400 Imported< ========== >

Step 2: Pure Storage Array ConfigurationConfiguration steps for the Pure Storage array are covered here.

1. Create a host initiator group for each host:

Step 2: Pure Storage Array Configuration


HTTP://WWW.BROCADE.COM/DOWNLOADS/DOCUMENTS/PRODUCT_MANUALS/B_SAN/FOS_ADMINGD_V730.PDF

FIGURE 2 New host creation on Pure Storage array

2. Add WWPNs to the host:

FIGURE 3 Adding host ports to host configuration on Pure Storage array

3. Create new volumes. For this test configuration, 8 x 10GB volumes are created.

Test Plan


FIGURE 4 Creating new volumes on Pure Storage array

FIGURE 5 Finished LUN configuration and presentation on Pure Storage array

Step 3: Host SetupConfiguration settings of the servers in the test bed are covered here.

1. Provision a minimum of two uplinks from the host to the FC fabric for redundancy, and use nativemultipath tools to manage the available paths and load-balance across them.

2. Multipath configuration on Linux hosts

This configuration allows all paths to be used in a round-robin fashion. This provides superiorperformance to the default Linux settings which would only use a single active path per LUN.

Step 3: Host Setup


Recommended /etc/multipath.conf entry on Linux systems:

< =========== > device { vendor "PURE" path_selector "round-robin 0" path_grouping_policy multibus rr_min_io 1 path_checker tur fast_io_fail_tmo 10 dev_loss_tmo 60 } < =========== >Example multipath configuration on Linux:

< =========== > # multipath -llmpathp (3624a9370a15a66e949f7d1440001009a) dm-19 PURE ,FlashArray size=5.0G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=1 status=active |- 1:0:1:13 sdm 8:192 active ready running |- 1:0:2:13 sdu 65:64 active ready running |- 1:0:3:13 sdac 65:192 active ready running |- 1:0:4:13 sdak 66:64 active ready running |- 10:0:2:13 sddm 71:64 active ready running |- 10:0:4:13 sdec 128:64 active ready running |- 10:0:3:13 sddu 71:192 active ready running `- 10:0:5:13 sdek 128:192 active ready runningmpatho (3624a9370a15a66e949f7d1440001009c) dm-20 PURE ,FlashArray size=5.0G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=1 status=active |- 1:0:1:15 sdo 8:224 active ready running |- 1:0:2:15 sdw 65:96 active ready running |- 1:0:3:15 sdae 65:224 active ready running |- 1:0:4:15 sdam 66:96 active ready running |- 10:0:2:15 sddo 71:96 active ready running |- 10:0:3:15 sddw 71:224 active ready running |- 10:0:4:15 sdee 128:96 active ready running `- 10:0:5:15 sdem 128:224 active ready runningmpathn (3624a9370a15a66e949f7d14400010099) dm-16 PURE ,FlashArray size=5.0G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=1 status=active |- 1:0:1:12 sdl 8:176 active ready running |- 1:0:2:12 sdt 65:48 active ready running |- 1:0:3:12 sdab 65:176 active ready running |- 1:0:4:12 sdaj 66:48 active ready running |- 10:0:2:12 sddl 71:48 active ready running |- 10:0:3:12 sddt 71:176 active ready running |- 10:0:4:12 sdeb 128:48 active ready running `- 10:0:5:12 sdej 128:176 active ready runningmpathm (3624a9370a15a66e949f7d1440001009d) dm-6 PURE ,FlashArray size=5.0G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=1 status=active |- 1:0:1:16 sdp 8:240 active ready running |- 1:0:2:16 sdx 65:112 active ready running |- 1:0:3:16 sdaf 65:240 active ready running |- 1:0:4:16 sdan 66:112 active ready running |- 10:0:2:16 sddp 71:112 active ready running |- 10:0:3:16 sddx 71:240 active ready running |- 10:0:4:16 sdef 128:112 active ready running `- 10:0:5:16 sden 128:240 active ready runningmpathl (3624a9370a15a66e949f7d1440001009b) dm-18 PURE ,FlashArray size=5.0G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=1 status=active |- 1:0:1:14 sdn 8:208 active ready running |- 1:0:2:14 sdv 65:80 active ready running |- 1:0:3:14 sdad 65:208 active ready running |- 1:0:4:14 sdal 66:80 active ready running |- 10:0:3:14 sddv 71:208 active ready running |- 10:0:2:14 sddn 71:80 active ready running |- 10:0:4:14 sded 128:80 active ready running `- 10:0:5:14 sdel 128:208 active ready runningmpathk (3624a9370a15a66e949f7d1440001009e) dm-7 PURE ,FlashArray size=5.0G features='0' hwhandler='0' wp=rw

Test Plan


`-+- policy='round-robin 0' prio=1 status=active |- 1:0:1:17 sdq 65:0 active ready running |- 1:0:2:17 sdy 65:128 active ready running |- 1:0:4:17 sdao 66:128 active ready running |- 1:0:3:17 sdag 66:0 active ready running |- 10:0:3:17 sddy 128:0 active ready running |- 10:0:2:17 sddq 71:128 active ready running |- 10:0:4:17 sdeg 128:128 active ready running `- 10:0:5:17 sdeo 129:0 active ready runningmpathj (3624a9370a15a66e949f7d14400010097) dm-3 PURE ,FlashArray size=5.0G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=1 status=active |- 1:0:1:10 sdj 8:144 active ready running |- 1:0:2:10 sdr 65:16 active ready running |- 1:0:3:10 sdz 65:144 active ready running |- 1:0:4:10 sdah 66:16 active ready running |- 10:0:2:10 sddj 71:16 active ready running |- 10:0:3:10 sddr 71:144 active ready running |- 10:0:4:10 sddz 128:16 active ready running `- 10:0:5:10 sdeh 128:144 active ready runningmpathi (3624a9370a15a66e949f7d14400010098) dm-8 PURE ,FlashArray size=5.0G features='0' hwhandler='0' wp=rw`-+- policy='round-robin 0' prio=1 status=active |- 1:0:1:11 sdk 8:160 active ready running |- 1:0:2:11 sds 65:32 active ready running |- 1:0:3:11 sdaa 65:160 active ready running |- 1:0:4:11 sdai 66:32 active ready running |- 10:0:2:11 sddk 71:32 active ready running |- 10:0:3:11 sdds 71:160 active ready running |- 10:0:5:11 sdei 128:160 active ready running `- 10:0:4:11 sdea 128:32 active ready running< =========== >

3. Multipath configuration on Windows hosts

Use Windows MPIO dialog to discover and manage multipath entries for Pure Storage.

FIGURE 6 Windows MPIO multipathing with Pure Storage array

Test Plan


4. Multipath configuration on VMWare hosts

Change path selection to "Round-Robin". This provides superior performance to the 'Most RecentlyUsed' setting that woulduse only a single active path per LUN.

FIGURE 7 Round robin multipath configuration on VMWare

5. Apply any additional host tuning (for Linux systems). Settings shown below help to support highIOPS operation of the Pure Storage flash array with Linux hosts.

Create a file named "/etc/udev/rules.d/99-pure-storage.rules", and add the following lines:

< ========== ># Recommended settings for Pure Storage FlashArray.# Use noop scheduler for high-performance solid-state storageACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{queue/scheduler}="noop"# Reduce CPU overhead due to entropy collectionACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{queue/add_random}="0"# Spread CPU load by redirecting completions to originating CPUACTION=="add|change", KERNEL=="sd*[!0-9]", SUBSYSTEM=="block", ENV{ID_VENDOR}=="PURE", ATTR{queue/rq_affinity}="2"< ========== >

6. Setup workload generators

On Windows and Linux system, Medusa Labs Test Tools is installed. On VMWare systems,VMWare's IOAnalyzer is installed.

Test Plan


Step 3: Host Setup


Test Cases

● Fabric Initialization – Base Functionality......................................................................... 22● Fabric – Advanced Functionality..................................................................................... 24● Stress and Error Recovery with Device Multi-Path......................................................... 28● Storage Device – Fibre Channel Routing (FCR) InterNetworking Tests.........................36● Optional/Additional Tests................................................................................................ 40

1.0 FABRIC INITIALIZATION – BASE FUNCTIONALITY

Confirm basic Fibre Channel functionality of the storage array

1.0.1 Storage Device – Physical and Logical Login with Speed Negotiation

1.0.2 Zoning and LUN Mapping

1.0.3 Storage Device Fabric IO Integrity

1.0.4 Storage Device Multipath Configuration – Path integrity

1.1 FABRIC – ADVANCED FUNCTIONALTY

Examine the storage behavior related to more advanced fabric features such asQoS, Bottleneck Detection, and advanced frame recovery

1.1.1 Storage Device Bottleneck Detection – w/Congested Host

1.1.2 Storage Device Bottleneck Detection – w/Congested Fabric

1.1.3 Storage Device – QOS Integrity

1.1.4 Storage Device – FC Protocol Jammer Test Suite

1.2 STRESS and ERROR RECOVERY WITH DEVICE MULTI-PATH

Confirm proper HA/failover behavior of storage in a multipath environment

1.2.1 Storage Device Fabric IO integrity – Congested Fabric

1.2.2 Storage Device Nameserver Integrity – Device Recovery with Port Toggle

1.2.3 Storage Device Nameserver Integrity – Device Recovery with Device Relocation

1.2.4 Storage Device Nameserver Stress – Device Recovery with Device Port Toggle

1.2.5 Storage Device Recovery – ISL Port Toggle

1.2.6 Storage Device Recovery – ISL Port Toggle (entire switch)

1.2.7 Storage Device Recovery – Director Blade Maintenance


1.2.8 Storage Device Recovery – Switch Offline

1.2.9 Storage Device Recovery – Switch Firmware Download

1.3 STORAGE DEVICE – FIBRE CHANNEL ROUTING (FCR) INTERNETWORKINGTESTS

Confirm proper storage functioning within routed fabrics

1.3.1 Storage Device InterNetworking Validation w/FC host

1.3.2 Storage Device InterNetworking Validation w/FCoE using VDX FlexPort

1.3.3 Storage Device Edge Recovery after FCR Disruptions

1.3.4 Storage Device BackBone Recovery after FCR Disruptions

1.4 OPTIONAL/ADDITIONAL TESTS

1.4.1 Storage device non-disruptive firmware update

1.4.2 Performance I/O testing with varying workload dimensions

1.4.3 Performance testing with VMware IOAnalyzer application workload suite

Fabric Initialization – Base Functionality

Storage Device – Physical and Logical Login with Speed Negotiation

Test ObjectiveVerify device login to switch and nameserver with all supported speed settings.

ProcedureTest Execution:

Set switch ports to 2/4/8/Auto_Negotiate speed settings.

< ========== >portcfgspeed <port> [2/4/8/0]< ========== >Result Validation:

1. Validate link states on the array and verify speed negotiation and device login at different speeds.Use command ‘portshow’ on the switch to check the link state and speed, and the command‘nscamshow’ to verify device login to the fabric.

2. Check switch port status and verify the “actual” and “configured” link speed. Check name server fordevice login<==========># nscamshow

Fabric Initialization – Base Functionality


# portshow Xroot> portshow 19portIndex: 19portName: port19portHealth: HEALTHYportState: 1 OnlineProtocol: FCportWwn of device(s) connected: 10:00:8c:7c:ff:22:f7:81Distance: normalportSpeed: N16Gbps <==========>

Result

PASS. Storage logs into fabric and is link up at 2Gb/4Gb/8Gb. Run IO to verify.

Zoning and LUN Mapping

Test ObjectiveVerify host to LUN access exists with valid zoning.


1. Create FC zone on the fabric with the initiator and target WWNs.2. Create Host Groups and LUNs on the array with access to initiator WWN.

Result Validation:

1. Verify LUNs are discovered on the hosts with host-specific tools.<==========>- Linux: Check output of ‘lsscsi’- Windows: Check output of ‘Computer Management’ -> ‘Storage” -> ‘Disk Management’- VMware: Check output at ‘Configuration’ -> ‘Storage’ -> ‘Devices’<==========>

Result

PASS. For each host, create a zone containing four storage ports and two host ports. Verified LUNs arepresented to host; verify with IO.

Storage Device Fabric IO Integrity

Test ObjectiveValidate single path host-to-LUN IO with write/read/verify testing.


1. Setup read/write I/O to LUN using Medusa2. Perform link disruptions with port-toggles, cable pulls.

Zoning and LUN Mapping


Result Validation:

1. Check Medusa I/O logs and verify I/O resumes after a short downtime. Medusa I/O may pause, butshould recover without errors.

Result

PASS. IO resumes without errors.

Storage Device Multipath Configuration – Path integrity

Test ObjectiveVerify multi-path configures successfully and verify integrity of each path.


1. Setup host with at least 2 initiator ports zoned with 2 target ports on array.2. Setup multipath on host.3. Start IO.4. Isolate paths individually and run IO on the separate paths.

Result Validation:

1. Check host multipath properties to verify the toggled path recovers.<==========>- Windows: mpclaim –s –d - Linux: multipath –ll- VMware: Check the paths at ‘Configuration’ -> ‘Storage’ -> ‘Devices’ -> ‘Manage Paths’ <==========>

2. Check the host and storage logs for any failures.3. Check the switch error logs and switch port status after toggle.

<==========># errdumpall# portstatsshow X# portshow Xroot> portshow 19portHealth: HEALTHY …………portState: 1 Online …………..portWwn of device(s) connected: 10:00:8c:7c:ff:22:f7:81 Distance: normalportSpeed: N16Gbps<==========>

4. Check I/O logs and verify I/O continues without any errors.

Result

PASS. Each individual path is valid and the paths combine successfully in a multipath configuration.

Fabric – Advanced Functionality

Storage Device Multipath Configuration – Path integrity


Storage Device Bottleneck Detection using MAPS FPI – w/CongestedHost

Test ObjectiveVerify congestion on host ports is detected. Verify storage device and fabric behavior during congestion.


1. Configure MAPS and FPI on all switches. (More info in the Testbed Configuration section.)2. Start I/O from single host initiator to multiple targets.3. Monitor switch logs for IO_PERF_IMPACT/ IO_FRAME_LOSS warnings.

Result Validation:

Check switch error logs and MAPS dashboard for bottleneck warnings.# errdumpall | grep IO_root> errdumpall | grep IO_2014/12/17-11:56:00:672622, [MAPS-1003], 117148/115018, FID 128, WARNING, B6510_066_088, Port 16, Condition=ALL_F_PORTS(DEV_LATENCY_IMPACT==IO_PERF_IMPACT), Current Value:[DEV_LATENCY_IMPACT,IO_PERF_IMPACT, 30.0% in 10 secs], RuleName=defALL_F_PORTS_IO_PERF_IMPACT, Dashboard Category=Fabric Performance Impact., actionHndlr.c, line: 755, comp:md, ltime:2014/12/17-11:56:00:671909# mapsdb --show allroot> mapsdb --show1 Dashboard Information:=======================Active policy: dflt_aggressive_policy…………………2 Switch Health Report:=======================Current Switch Policy Status: HEALTHY3.1 Summary Report:===================Category |Today |Last 7 days |--------------------------------------------------------------------------------Port Health |No Errors |Out of operating range |Fru Health |In operating range |In operating range |Security Violations |No Errors |In operating range |Fabric State Changes |No Errors |In operating range |Switch Resource |In operating range |In operating range |Traffic Performance |In operating range |In operating range |FCIP Health |Not applicable |Not applicable |Fabric Performance Impact|In operating range |In operating range |<==========>Result

PASS. With MAPS reporting configured, performance warnings are reported as expected. Thebottlenecked ports are displayed on the MAPS dashboard and also a RASLog warning is created.

Storage Device Bottleneck Detection using MAPS FPI – w/Congested Host


Storage Device Bottleneck Detection using MAPS FPI – w/CongestedFabric

Test ObjectiveValidate bottleneck detection with congested fabric.


1. Configure MAPS on all switches. (More info in the Testbed Configuration section.)2. Isolate single ISL in the fabric.3. Start I/O from multiple host initiators to multiple targets.4. Monitor switch logs for Traffic Performance warnings.

Result Validation:

1. Check switch error logs and MAPS dashboard for bottleneck warnings.# errdumpall | grep Traffic2014/12/18-08:39:03:497512, [MAPS-1003], 9637/5496, FID 128, WARNING, B6510_066_082, Port 33, Condition=ALL_TARGET_PORTS(RX/hour>60.00), Current Value:[RX,65.16 %], RuleName=defALL_TARGET_PORTSRX_60, Dashboard Category=Traffic Performance., actionHndlr.c, line: 755, comp:md, ltime:2014/12/18-08:39:03:496754# mapsdb --show allroot> mapsdb --show1 Dashboard Information:=======================Active policy: dflt_aggressive_policy…………………2 Switch Health Report:=======================Current Switch Policy Status: HEALTHY3.1 Summary Report:===================Category |Today |Last 7 days |--------------------------------------------------------------------------------Port Health |No Errors |Out of operating range |Fru Health |In operating range |In operating range |Security Violations |No Errors |In operating range |Fabric State Changes |No Errors |In operating range |Switch Resource |In operating range |In operating range |Traffic Performance |In operating range |In operating range |FCIP Health |Not applicable |Not applicable |Fabric Performance Impact|In operating range |In operating range |<==========>

Result

PASS. With MAPS reporting configured, performance warnings are reported as expected.

Storage Device – QoS Integrity

Test ObjectiveValidate QoS functionality

Storage Device Bottleneck Detection using MAPS FPI – w/Congested Fabric



1. Setup initiator-target pairs with Low/Medium/High QoS zones in the fabric.2. Start I/O across all pairs and verify no errors in logs

Result Validation:

1. Check I/O logs and verify I/O continues without any errors.2. Check switch error logs and switch port status for any errors.

<==========># porterrshowroot> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<==========>

Result

PASS. IO completes without errors on low, medium, and high hosts.

Storage Device – FC Protocol Jammer Test Suite

Test ObjectivePerform FC Jammer Tests including areas such as CRC corruption, packet corruption, missing frame,host error recovery, and target error recovery.


1. Insert Jammer device in the I/O path on the storage link.2. Execute the following Jammer scenarios:

• Delete one frame• Delete R_RDY• Replace CRC of data frame• Replace EOF of data frame• Replace “good status” with “check condition”• Replace IDLE with LR• Truncate frame• Create S_ID/D_ID error of data frame

3. Verify Jammer operations and recovery with Analyzer.

Result Validation:

Check Medusa log and verify IO recovers and completes without errors.

Result

Storage Device – FC Protocol Jammer Test Suite


PASS. Introduce packet anomalies and verify proper recovery.

Stress and Error Recovery with Device Multi-Path

Storage Device Fabric IO integrity – Congested Fabric

Test ObjectiveValidate I/O integrity in a congested fabric environment.


From all initiators, start a mixture of READ, READ/WRITE, and WRITE traffic continuously to all thetargets for a 24-hour period.

Result Validation1. Check the host and storage logs for any errors.2. Verify the link congestion and check the switch logs for any errors.

<==========># errdumpall# portperfshow# porterrshowroot> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<==========>

3. Check I/O generator tool logs to verify I/O runs without errors.

Result

PASS. In a congested fabric, IO runs successfully without errors or failover.

Storage Device Nameserver Integrity – Device Recovery with PortToggle

Test ObjectiveValidate path recovery behavior on storage and host ports.

Stress and Error Recovery with Device Multi-Path



1. Setup multipath on host and start I/O.2. Perform multiple iterations of sequential port toggles across initiator and target switch ports.

Result Validation1. Check switch port status after toggle and for any errors in the switch error logs.

<==========># errdumpall# portstatsshow X# portshow Xroot> portshow 19portHealth: HEALTHY …………portState: 1 Online …………..portWwn of device(s) connected: 10:00:8c:7c:ff:22:f7:81 Distance: normalportSpeed: N16Gbps<==========>

2. Check host multipath status on hosts to verify the toggled path recovers.<==========>- Windows: mpclaim –s –d - Linux: multipath –ll- VMware: Check the paths at ‘Configuration’ -> ‘Storage’ -> ‘Devices’ -> ‘Manage Paths’ <==========>

3. Check host and storage error logs and verify I/O continues without errors.

Result

PASS. Failover between 8 logical paths (2 host x 4 storage) tested successfully. Paths recover, and IOcompletes without errors.

Storage Device Nameserver Integrity – Device Recovery with DeviceRelocation

Test ObjectiveValidate storage device path recovery after physical port relocation.


1. Sequentially performed for each Storage Device port.2. Disconnect and reconnect port to different switch port in same fabric.

Result Validation1. Check for any errors in the switch error logs and the switch port status at the new switch port.

<==========># errdumpall# portstatsshow X

Storage Device Nameserver Integrity – Device Recovery with Device Relocation


# portshow Xroot> portshow 8portHealth: HEALTHY …………portState: 1 Online …………..portWwn of device(s) connected: 52:4a:93:7d:f3:5f:61:11 Distance: normalportSpeed: N8Gbps<==========>

2. Check host multipath status on hosts to verify the toggled path recovers.<==========>- Windows: mpclaim –s –d - Linux: multipath –ll- VMware: Check the paths at ‘Configuration’ -> ‘Storage’ -> ‘Devices’ -> ‘Manage Paths’ <==========>

3. Confirm there are no errors in the Medusa log.

Result

PASS. Physical move of storage port shows successful recovery. Paths recover, and IO completeswithout errors.

Storage Device – Device Recovery with Device Port Toggle – ExtendedRun

Test ObjectiveValidate path recovery behavior on storage and host ports for an extended duration run.


1. Setup multipath on host and start I/O.2. Sequentially toggle each initiator and target port in fabric (multiple iterations).3. Run for an extended period.

Result Validation:

1. Check switch port status after toggle and for any errors in the switch error logs.<==========># errdumpall# portstatsshow X# portshow Xroot> portshow 19portHealth: HEALTHY …………portState: 1 Online …………..portWwn of device(s) connected: 10:00:8c:7c:ff:22:f7:81 Distance: normalportSpeed: N16Gbps<==========>

2. Check host multipath properties for iSCSI hosts to verify the toggled path recovers.<==========>On Windows: mpclaim –s –d On Linux: multipath –llOn VMware: Check the paths at ‘Configuration’ -> ‘Storage’ -> ‘Devices’ ->

Storage Device – Device Recovery with Device Port Toggle – Extended Run


‘Manage Paths’ <==========>


Result

PASS. 48-hr run; paths recover and IO completes without errors.

Storage Device Recovery – ISL Port Toggle

Test ObjectiveValidate path recovery and IO integrity when ISL links are disabled.


1. Setup multipath on host and start I/O.2. Sequentially toggle each ISL path (one at a time) on all switches.

Result Validation:

1. Check FC fabric status after ISL toggle. Verify all nodes are online<==========># fabricshowroot> fabricshowSwitch ID Worldwide Name Enet IP Addr FC IP Addr Name------------------------------------------------------------------------- 1: fffc01 50:00:53:35:b1:d3:df:1b 0.0.0.0 0.0.0.0 "fcr_xd_1_40" 3: fffc03 10:00:00:05:33:13:95:9a 10.38.66.73 0.0.0.0 "B6510_066_073" 19: fffc13 10:00:00:05:33:a5:bf:86 10.38.66.74 0.0.0.0 >"B6510_066_074" 82: fffc52 10:00:00:05:33:13:96:5a 10.38.66.82 0.0.0.0 "B6510_066_082" 83: fffc53 10:00:00:05:33:5b:1d:1d 10.38.66.83 0.0.0.0 "B6510_066_083"The Fabric has 5 switches Fabric Name: SSR<==========>

2. Check the switch logs for any errors and verify I/O failed over to alternate ISL path in the fabric.<==========># errdumpall# portperfshow# porterrshowroot> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<==========>


Result

Storage Device Recovery – ISL Port Toggle


PASS. Paths recover, and IO completes without errors.

Storage Device Recovery – ISL Port Toggle (entire switch)

Test ObjectiveValidate path recovery and IO integrity when all ISL links on a switch are disabled.


1. Ensure ISL redundancy by provisioning multiple ISL’s connected to different switches to providemultiple paths through the fabric.

2. Setup multipath on host and start I/O.3. On each switch, disable all ISL links at once.

Result Validation:

1. Check FC fabric status after ISL toggle. Verify all nodes are online<==========># fabricshowroot> fabricshowSwitch ID Worldwide Name Enet IP Addr FC IP Addr Name------------------------------------------------------------------------- 1: fffc01 50:00:53:35:b1:d3:df:1b 0.0.0.0 0.0.0.0 "fcr_xd_1_40" 3: fffc03 10:00:00:05:33:13:95:9a 10.38.66.73 0.0.0.0 "B6510_066_073" 19: fffc13 10:00:00:05:33:a5:bf:86 10.38.66.74 0.0.0.0 >"B6510_066_074" 82: fffc52 10:00:00:05:33:13:96:5a 10.38.66.82 0.0.0.0 "B6510_066_082" 83: fffc53 10:00:00:05:33:5b:1d:1d 10.38.66.83 0.0.0.0 "B6510_066_083"The Fabric has 5 switches Fabric Name: SSR<==========>



Result

Storage Device Recovery – ISL Port Toggle (entire switch)



Storage Device Recovery – Director Blade Maintenance

Test ObjectiveValidate path recovery and IO integrity during Director blade maintenance.


1. Uplink edge switch ISLs to different blades on the directors.2. Sequentially toggle each Director blade.3. Include blade disable/enable and blade power on/off .

Syntax:< =========== >slotpoweroff/slotpoweron, bladedisable/bladeenable, slotshow< =========== >Result Validation:

1. Check FC fabric status after the blade toggle. Verify all nodes are present in the fabric.< ========== ># fabricshowroot> fabricshowSwitch ID Worldwide Name Enet IP Addr FC IP Addr Name------------------------------------------------------------------------- 1: fffc01 50:00:53:39:47:bd:6e:0a 0.0.0.0 0.0.0.0 "fcr_fd_1" 2: fffc02 10:00:00:05:1e:09:15:3e 10.38.51.5 0.0.0.0 "sw5300sus1005" 3: fffc03 10:00:c4:f5:7c:07:c2:0c 10.38.51.3 0.0.0.0 "sw300sus1003" 4: fffc04 10:00:c4:f5:7c:7b:86:1f 10.38.51.4 0.0.0.0 "sw6510sus1004" 5: fffc05 10:00:00:05:33:13:80:ef 10.38.51.24 0.0.0.0 "swBESsus1024" 13: fffc0d 10:00:00:05:1e:4c:ff:00 10.38.51.13 0.0.0.0 "sw8510sus1013" 16: fffc10 10:00:00:05:1e:44:02:00 10.38.51.16 0.0.0.0 >"sw8518sus1016" 22: fffc16 10:00:00:05:1e:07:7d:99 10.38.51.22 0.0.0.0 "sw5100sus1022" 23: fffc17 10:00:00:27:f8:2b:9f:aa 10.38.51.23 0.0.0.0 "sw6520sus1023" 25: fffc19 10:00:00:05:1e:54:8b:d4 10.38.51.25 0.0.0.0 "swBESsus1025" 26: fffc1a 10:00:00:05:33:8f:b6:77 10.38.51.26 0.0.0.0 "sw6510sus1026" 29: fffc1d 10:00:00:05:33:83:a4:00 10.38.51.29 0.0.0.0 "swDCX4sus1029" 32: fffc20 10:00:00:05:1e:b8:c1:00 10.38.51.32 0.0.0.0 "swDCXsus1032"160: fffca0 10:00:00:05:1e:a8:35:9d 10.38.51.21 0.0.0.0 "swVA40FCsus1021"171: fffcab 10:00:00:05:1e:d7:1a:04 10.38.51.171 172.172.172.26 "sw5480sus1171" 2620:100:4:fa00:205:1eff:fed7:1a04183: fffcb7 10:00:00:05:33:00:b2:e2 10.38.51.183 0.0.0.0 "swM5424sus1183"195: fffcc3 10:00:00:05:33:8d:df:d1 10.38.51.195 0.0.0.0 "sw5480sus1195"199: fffcc7 10:00:00:27:f8:44:33:61 10.38.51.199 0.0.0.0

Storage Device Recovery – Director Blade Maintenance


"sw6547sus1199" :100:4:fa00:227:f8ff:fe44:3362220: fffcdc 10:00:00:05:1e:86:1b:1f 10.38.51.220 0.0.0.0 "sw5450sus1220"The Fabric has 19 switches



Result


Storage Device Recovery – Switch Offline

Test ObjectiveValidate path recovery and IO integrity during switch offline events.


1. Toggle each switch in sequential order.2. Include switch enable/disable, power on/off, and reboot testing.

Result Validation:

1. Check FC fabric status after the switch toggle. Verify all nodes are present in the fabric.<==========># fabricshowroot> fabricshowSwitch ID Worldwide Name Enet IP Addr FC IP Addr Name------------------------------------------------------------------------- 1: fffc01 50:00:53:35:b1:d3:df:1b 0.0.0.0 0.0.0.0 "fcr_xd_1_40" 3: fffc03 10:00:00:05:33:13:95:9a 10.38.66.73 0.0.0.0 "B6510_066_073" 19: fffc13 10:00:00:05:33:a5:bf:86 10.38.66.74 0.0.0.0 >"B6510_066_074" 82: fffc52 10:00:00:05:33:13:96:5a 10.38.66.82 0.0.0.0 "B6510_066_082" 83: fffc53 10:00:00:05:33:5b:1d:1d 10.38.66.83 0.0.0.0 "B6510_066_083"The Fabric has 5 switches

Storage Device Recovery – Switch Offline


Fabric Name: SSR<==========>

2. Check the switch logs for any errors and verify the toggled switch has recovered.<==========># errdumpall# switchshowroot> switchshowswitchName: B6510_066_088switchType: 109.1switchState: Online switchMode: NativeswitchRole: SubordinateswitchDomain: 88switchId: fffc58switchWwn: 10:00:00:27:f8:06:23:28zoning: ON (SSR)switchBeacon: OFFFC Router: ONFC Router BB Fabric ID: 100Address Mode: 0Fabric Name: SSR_2<==========>


Result

PASS. Paths fail over and recover, and IO completes without errors.

Storage Device Recovery – Switch Firmware Download

Test ObjectiveVerify I/O will continue with minimal disruption throughout the switch firmware upgrade process.


1. Setup host multipath with links on different switches in the FC fabric and start I/O.2. Sequentially perform firmware upgrades on all switches in the fabric.

Result Validation:

1. 1. Verify firmware upgrade completes successfully on each switch node and they merge back in theFC fabric.<==========># versionroot> versionKernel: 2.6.14.2Fabric OS: v7.3.1 Made on: Thu Dec 11 14:30:38 2014Flash: Wed Dec 17 09:53:59 2014BootProm: 1.0.11# fabricshowroot> fabricshowSwitch ID Worldwide Name Enet IP Addr FC IP Addr Name------------------------------------------------------------------------- 1: fffc01 50:00:53:35:b1:d3:df:1b 0.0.0.0 0.0.0.0 "fcr_xd_1_40" 3: fffc03 10:00:00:05:33:13:95:9a 10.38.66.73 0.0.0.0 "B6510_066_073" 19: fffc13 10:00:00:05:33:a5:bf:86 10.38.66.74 0.0.0.0 >"B6510_066_074" 82: fffc52 10:00:00:05:33:13:96:5a 10.38.66.82 0.0.0.0

Storage Device Recovery – Switch Firmware Download


"B6510_066_082" 83: fffc53 10:00:00:05:33:5b:1d:1d 10.38.66.83 0.0.0.0 "B6510_066_083"The Fabric has 5 switches Fabric Name: SSR<==========>

2. Check I/O generator tool logs to verify I/O runs without errors throughout the firmware upgrade.3. Check the switch logs for any errors and verify I/O resumes on the node after the firmware upgrade

is complete.

Result

PASS. I/O failover and recovery is successful during firmware download. Paths recover, and IOcompletes without errors.

Storage Device – Fibre Channel Routing (FCR) InterNetworking Tests

Storage Device InterNetworking Validation with FC host

Test ObjectiveValidate storage targets are imported successfully in a routed Fibre Channel environment.


1. Setup FCR in an Edge-Backbone-Edge configuration.2. Setup LSAN zoning and verify host access to target LUNs and start I/O.

Result Validation:

1. Verify name server and FCR fabric state.<==========># fcrfabricshowroot> fcrfabricshowFC Router WWN: 10:00:00:05:33:13:96:5a, Dom ID: 82,Info: 10.38.66.82, "B6510_066_082" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 40 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088" 41 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92"FC Router WWN: 10:00:00:05:33:5b:1d:1d, Dom ID: 83,Info: 10.38.66.83, "B6510_066_083" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 40 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92" 41 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088"# fcrproxydevshowroot> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PID

Storage Device – Fibre Channel Routing (FCR) InterNetworking Tests


in Fabric in Fabric---------------------------------------------------------------------------- 40 10:00:00:05:1e:60:b4:6b 02ff02 100 132400 Imported 40 10:00:00:05:1e:60:b4:6c 02fe02 100 032400 Imported………………. 100 50:05:07:60:5e:80:76:52 01fe02 40 612800 Imported 100 50:05:07:60:5e:80:76:71 01ff02 40 642300 ImportedTotal devices displayed: 20<==========>

2. Verify I/O runs successfully without any errors.

Result

PASS. All devices are available via routed fabric. Running IO confirms successful routing.

Storage Device InterNetworking Validation w/FCoE Test using VDXFlexPort

Test ObjectiveVerify storage device availability from FCoE initiators routed through VCS fabric.


1. Setup FCoE host initiator ports on VCS fabric using VDX6740 switches.2. Enable Fibre Channel support on VDX switch ports (more detail in testbed configuration section)3. Connect VCS fabric to FC fabric via FCR ports.4. Configure LSAN zones in VCS and Fibre Channel fabrics.

Result Validation:

1. 1. Verify name server and FCR fabric state.<==========># fcrfabricshowroot> fcrfabricshowFC Router WWN: 10:00:00:05:33:5b:7d:06, Dom ID: 97,Info: 10.38.66.97, "B6510_066_097" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 18 110 10.38.66.120 10:00:50:eb:1a:62:8c:33 "VDX6740_066_120"FC Router WWN: 10:00:00:05:33:5b:10:e8, Dom ID: 100,Info: 10.38.66.100, "B6510_066_100" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 8 110 10.38.66.119 10:00:50:eb:1a:62:83:7b "VDX6740_066_119"# fcrproxydevshowroot> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PIDin Fabric in Fabric---------------------------------------------------------------------------- 110 50:05:07:60:5e:80:76:52 02f001 120 612800 Imported 110 50:05:07:60:5e:80:76:71 02f101 120 642300 Imported 120 10:00:8c:7c:ff:00:48:00 03f101 110 781000 Imported 120 10:00:8c:7c:ff:00:48:01 03f001 110 771000 ImportedTotal devices displayed: 4<==========>

2. Verify I/O runs successfully without any errors.

Storage Device InterNetworking Validation w/FCoE Test using VDX FlexPort


Result

PASS. Storage targets are available through FCoE/FC routed configuration; IO completes withouterrors.

Storage Device Edge Recovery after FCR Disruptions

Test ObjectiveValidate storage target paths recover successfully from disruptions in a routed Fibre Channelenvironment. Edge-Backbone-Edge configuration.


1. Setup FCR in an Edge-Backbone-Edge configuration.2. Setup LSAN zoning.3. With IO running, perform sequential reboots, switch disables and ISL port toggles on the switches in

the backbone fabric.

Result Validation:

1. 1. Verify FCR fabric state throughout the disruptions.<==========># fcrfabricshowroot> fcrfabricshowFC Router WWN: 10:00:00:05:33:13:96:5a, Dom ID: 82,Info: 10.38.66.82, "B6510_066_082" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 40 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088" 41 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92"FC Router WWN: 10:00:00:05:33:5b:1d:1d, Dom ID: 83,Info: 10.38.66.83, "B6510_066_083" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 40 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92" 41 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088"# fcrproxydevshowroot> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PIDin Fabric in Fabric---------------------------------------------------------------------------- 40 10:00:00:05:1e:60:b4:6b 02ff02 100 132400 Imported 40 10:00:00:05:1e:60:b4:6c 02fe02 100 032400 Imported………………. 100 50:05:07:60:5e:80:76:52 01fe02 40 612800 Imported 100 50:05:07:60:5e:80:76:71 01ff02 40 642300 ImportedTotal devices displayed: 20<==========>

2. Check the switch logs for any errors.<==========># errdumpall# portperfshow<==========>

3. Check host and storage logs, and verify I/O runs without any errors.

Storage Device Edge Recovery after FCR Disruptions


Result

PASS. Paths failover and recover with FCR disruptions, and IO completes without errors.

Storage Device Backbone Recovery after FCR Disruptions

Test ObjectiveValidate storage target paths recover successfully from disruptions in a routed Fibre Channelenvironment. Edge-Backbone configuration.


1. Setup FCR in an Edge-Backbone configuration.2. Setup LSAN zoning.3. With IO running, perform sequential reboots, switch disables and ISL port toggles on the switches in

the backbone fabric.

Result Validation:

1. 1. Verify FCR fabric state throughout the disruptions.<==========># fcrfabricshowFC Router WWN: 10:00:00:05:33:13:96:5a, Dom ID: 82,Info: 10.38.66.82, "B6510_066_082" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 40 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088" 41 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92"FC Router WWN: 10:00:00:05:33:5b:1d:1d, Dom ID: 83,Info: 10.38.66.83, "B6510_066_083" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 40 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92" 41 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088"# fcrproxydevshowroot> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PIDin Fabric in Fabric---------------------------------------------------------------------------- 40 10:00:00:05:1e:60:b4:6b 02ff02 100 132400 Imported 40 10:00:00:05:1e:60:b4:6c 02fe02 100 032400 Imported………………. 100 50:05:07:60:5e:80:76:52 01fe02 40 612800 Imported 100 50:05:07:60:5e:80:76:71 01ff02 40 642300 ImportedTotal devices displayed: 20<==========>

2. Check the switch logs for any errors.<==========># errdumpall# portperfshow<==========>

3. Check host and storage logs, and verify I/O runs without any errors.

Result

PASS. Paths fail over successfully and IO completes without errors.

Storage Device Backbone Recovery after FCR Disruptions


Optional/Additional Tests

Storage Device firmware update

Test ObjectiveExecute a non-disruptive firmware update on the array while running IO, and confirm that there are noIO errors.


1. Run continuous IO to the array.2. Execute code update procedure as described in vendor documentation.

Result Validation:

1. Confirm the updated version on all array components.2. Check the I/O generator tools logs to verify I/O completes without any errors.3. Check the host and storage logs for any errors throughout the I/O operations.4. Check the switch error logs and port stats for any errors or I/O drops.<==========># errdumpall# porterrshowroot> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<==========>Result

PASS. Update successful (from 3.4.0 to 3.4.8) with proper failover and no IO errors.

Performance I/O testing with varying workload dimensions

Test ObjectiveRun I/O workload generation with varying dimensions (block size, read/write/mix), and verify that theperformance characteristics are as expected.


Optional/Additional Tests


1. Run IO loop at block transfer sizes of 512, 1k, 2k, 4k, 8k, 16k, 32k, 64k, 128k, 256k, 512k, and 1m.2. Include a nested loop of 100% read, 100% write, and 50% read/write.

Repeat the test for the following configurations:

• 1 host port to 1 target port (single path)• 2 host ports to multiple target ports (multipath)• multiple hosts to multiple target ports (multi-host multipath)

Result Validation:

1. Check the I/O generator tools log to verify that the I/O completes without any errors.2. Check the host and storage logs for any errors throughout the I/O operations.3. Check the switch error logs and port stats for any errors or I/O drops.<==========># errdumpall# porterrshowroot> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<==========>Result

PASS. All workload runs are monitored at the host, storage and fabric and verify they complete withoutany I/O errors or faults. Performance behavior is as expected.

VMWare IOAnalyzer workload test

Test ObjectiveRun workload test suite including varying application workloads generated from multiple VMs.


1. Configure a 2-host VMWare cluster with multipath on 2 initiator ports per host, and 4 target ports2. Configure workload generation from 8 worker VMs. using VMWare IOAnalyzer3. Run a variety of application IO workload patterns from the IOAnalyzer suite. For this round of testing,

the following workloads were used:

• Workstation• Webserver• Video on Demand• Max Write Throughput• Max Write IOPS• Max IOPS• Max Throughput

VMWare IOAnalyzer workload test


• SQL Server 16k• 4k Read• Exchange 2007• OLTP 4k

Result Validation:

1. Check the I/O generator tools logs to verify I/O completes without any errors.2. Check the host and storage logs for any errors throughout the I/O operations.3. Check the switch error logs and port stats for any errors or I/O drops.<==========># errdumpall# porterrshowroot> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<==========>Result

PASS. All workload runs are monitored at the host, storage and fabric and verify they completewithout any I/O errors.

Test Cases
