svc zoning 101 best practices_q1_2014-v1.2

8/21/2019 SVC Zoning 101 Best Practices_Q1_2014-V1.2

http://slidepdf.com/reader/full/svc-zoning-101-best-practicesq12014-v12 1/50

© 2013 IBM Corporation IBM Systems

I nnovation that matters1st Quarter 2014 Update V1.0

SVC – Zoning Best Practices 101

Chuck Laing - Senior Technical Staff Member



IBM System StorageTM

© 2013 IBM Corporation2

Change History 1/22/2013 - Added ‘verbiage and defined the term “pseudo-hosts”

1/30/2013 – Showing correct and incorrect backend to SVC and host port sharing

4/3/2013 – Added tower ownership to Multipathing test scenarios’ on pages 43 & 44

4/4/2013 - Reorganized the deck to flow better and removed many redundant example pages

10/1/2013 – Added CG8 – 8 port GM strategy to the deck

10/1/2013 – Added 8 port zoning strategy for host and storage

3/26/2014 – Added new zoning strategies for Dual Core Fabric to SVC Cluster zoning

Slide provided by Chuck Laing





Contributing Input/Resources Chuck Laing – Author – STSM GTS/SO

Bill Marshall – SME / Technical Webmaster (TWM) - Samba & Linux GTS/SO Server Tower

Bill Wiegand – Consulting IT Specialist - Storage – Virtualization STG/ATS

Brad Tidd - Distributed Systems Storage Architect GTS/SO

Brad Worthen - Subject Matter Expert - Solaris, Technical WebmasterGTS/SO – Server Tower

Brian Sherman – Distinguished Engineer - Americas Storage STG/ATS

Byron Grossnickel – Consulting IT Specialist Storage – Virtualization STG/ATS

Carlos Fuente – STSM STG/Hursley Development Support

Jason Moras - ITD SSO Web Delivery SME / Technical Webmaster GTS/SO – Server Tower

Jim Olson – Distinguished Engineer GTS/SSA

John Locke - SAN Storage Architect GTS/SO

Kamesh Bhaskarabhatla – Epert Level IT Specialist GTS/SO

Keith Williams – Storage Architect GTS/SO

Kirby Dahman - STSM GTS/SO

Mark Chitti – STSM GTS/SO

Ramesh Palakodeti - Technical Web Master, AIX, Global SMD Architect/SMEGTS/SO – Server Tower

Ronda McCain – Senior Architect GTS/SO

Global Design Authority (DA) team approval –GTS/SO & SSA

Global ACB team approval - GTS/SO & SSA




IBM Systems4

Agenda

SVC zoning, Volume Pathing and Multipathing testing

Most common questions

Understanding the physical and logical

General SVC Best Practice Zoning Concepts

SVC Zone Concepts

► SVC Cluster Zone (Just do it!)

► Storage to SVC Zoning

► Host HBA to SVC zoning (preferred/non preferred pathing)

● Dual Host HBAs

● Multiple Host HBAs

● Host HBA multipathing settings and testing

Zone Types for 8 port Nodes► MM/GM and heartbeat zones for 8 port nodes

SVC Cluster zoning for Dual Core Fabrics examples

► Cluster Zone 4 port vs. 8 port nodes






Most common zoning questions - BP recommendations

What is the max number of WWNN/WWPNs devices?

– Always check the Max Limit configuration URL for the most current updates – http://www-01.ibm.com/support/docview.wss?uid=ssg1S1004368

– Currently 1024 WWNN per cluster and 1024 WWPNs per cluster

– Recommendation - The more wwpns per wwnn, the more throughput up to 16

How many IO connections/zones per storage device to the SVC should Izone?

– 16 from any “one” storage device unit zoned with all SVC node ports

Should I mask host connections through zoning or SVC

configurations?

– Best practice is to use the default value of 1111 (all ports enabled) and controlmasking through zoning

– Use host type Target Port Group (TPGS) for Solaris host, HP/UX for HP and

Generic for everything else other than OpenVMS

– Separate disk and tape IO on host HBAs

Input provided Kamesh Bhaskarabhatla


http://www-01.ibm.com/support/docview.wss?uid=ssg1S1004368








What works better, 4 or 8 paths per Vdisk?

– Recommendation is 4 paths per Vdisk

How many Iogrps should I map to a host? 4?

– Recommendation is to size per throughput and number of hosts per cluster – May need to isolate/sement host activity/chatter from one Iogrp to another

– For more horsepower – spread IO across more LUNs from more IOgrps

Can a host have only one connection to the SVC ?

– Dual host HBA connections

Most common zoning questions - BP recommendations






Back panel of SVC node:

Fabric A Fabric B

IBM_2145:admin> svc in fo lsnode 1

port_id 5005076801100135 (port 3)

port_status active port_id 5005076801200135 (port 4)

port_status active

port_id 5005076801300135 (port 2)

port_status active

port_id 5005076801400135 (port 1)

port_id 5005076801500135 (port 5)

port_status active

port_id 5005076801600135 (port 6) port_status active

port_id 5005076801700135 (port 7

port_status active

port_id 5005076801800135 (port 8)

port_status active

1

3 4

2

Ports PhysicallyNumbered 5-8 Left

to Right

Numbers in yellow used

to make WWPN unique

5 6 7 8

Do no t add po rts 5-8 into

the clus ter zone

Use these ports for

MM/GM only w ith version

6.4.1 and 7.1.x fam ily

Zoning BP – understanding the physical and logical

Slide provided by Bill Wiegand

Modified to include 8 ports by Chuck Laing





Physical to Logical Port numbering

Logical port #’s reflecting the embedded WWPN

Blue ports go to one Fabric

Red Ports go to the other Fabric

Logical port with wwpn # embedded

Physical port number






General SVC Best Practice Zoning Concepts Create two cluster zones (only consisting of node ports not in MM/GM and node to node traffic)

Each Backend Storage device should be separated into its own zone with SVC

Zone Backend Storage ports and SVC ports together

−Never span zones to include more than one Backend storage device!

Never put Host OS ports, SVC ports and Backend Storage ports together in the same zone

− Instead - Create zones with Host ports and SVC ports

- Create zones with Backend and SVC ports

− Never use the same DS8K ports or any native back-end port for connectivity to SVC and an attached host

− If SVC is attached to the DS8K or other native back-end devices and the DS8K or other back-end device is using

native GM (not SVC GM) then dedicate appropriate back-end ports specifically for GM, not to be used for attaching

any other device, whether Host Server, SVC or other connectivity relationships.

Dual Core Fabrics require a breakout in cluster zoning to separate SVC node ports on core1 from

core2 to prevent IO from spanning/routing across ISLs and Edge switches in a Core/Edge design

−SVC node ports plugged into dir1 core1 should reside in a separate zone from SVC node ports plugged into dir1

core2.

− for MM/GM zones, take advantage of spreading the IO across both cores through careful zoning

−Ensure the Intra-cluster/heartbeat zone is also isolated properly between each fabrics dual cores

Note - Never make zoning changes on redundant Fabrics at the same time

−Make changes on one fabric and wait 30 min in-between

−Please see the following link for zoning BP’s

• Cisco:

http://g25aciwas09.con.can.ibm.com:9080/ram/assetDetail/generalDetails.faces?guid=5EB2DDF9-7568-368A-7F99-1BCFFF85E472

• Brocade:

http://g25aciwas09.con.can.ibm.com:9080/ram/assetDetail/generalDetails.faces?guid=57A9649D-A04B-5931-05DA-F389F697D4D3



























Zoning Best Practices - continued

Too many paths to a Vdisk If the recommended number of paths to a vdisk are exceeded a path failure may

not be recovered in the required amount of time

− Causes excessive I/O waits, resulting in application failures

− Under certain circumstances, it can reduce performance

• Note: 8 paths are supported but 4 are optimum for SDD/SDDDSM/SDDPCM

SVC host zones

There must be a single zone for each host port. This zone must contain the host

port, and one port from each SVC node that the host will need to access. While

there are two ports from each node per SAN fabric in a usual dual-fabric

configuration, make sure that the host only accesses one of them.

− Check with the Host System Administrators to receive output from the SDD or PCM pathquery device commands on each host to verify proper pathing between the SVC nodes

and host systems

• Note: It is a supported configuration to have eight paths to each VDisk, but this design provides no

performance benefit, and it does not improve reliability or availability by any significant degree







SVC Preferred Node Scheme

Hosts with four (or more) Host Bus Adapters (HBAs) Four or more HBA’s:

− Takes a little more planning.

− Because eight paths are not an optimum number, you must instead configure your SVC

Host Definitions (and zoning) as though the single host is two or more separate hosts

− During Vdisk assignment, alternate which Vdisk is assigned to one of the “pseudo-hosts”,

in a round robin fashion (a pseudo-host is nothing more than another regular host definitionin the SVC host config. Each pseudo-host will contain 2 unique host WWPNs, 1 WWPN

mapped to each fabric )

• Note: A pseudo-host, is not a defined function or feature of the SVC. If you need to define a pseudo-

host, you are simply adding another host id to the SVC host config. Instead of creating one host id

with 4 WWPNs, you would define 2 hosts with 2 WWPNs. This is now the reference for the term

pseudo-host.

• Note: Be careful not to share the Vdisk to more than two adapters per host, so as to not

oversubscribe the number of datapaths per vdisk per host

SVC Path Reduction Guidance

− To mitigate I/O interruption risks - removing paths and ports (wwpns) require an STQRP

and should be done within an application outage window- ensuring applications are

stopped prior to changes






SVC Preferred Node Scheme - Continued

The SAN Volume Controller provides Logical Unit Number (LUN)masking capability.

This allows you to associate or dissociate a storage volume with a fibre-based host

initiator through the worldwide port names (WWPNs).

Best practice is to use the default value of 1111 (all ports enabled) and control

masking through zoning.

The port mask must contain 4 binary characters.

-iogrp iogrp_list (Optional) Specifies a set of one or more I/O groups that the host

can access the VDisks from. The port mask is four binary bits and is made up of a

combination of 0's and 1's, where 0 indicates that the corresponding target port

cannot be used and 1 indicates that it can be used.. The right-most bit in the mask

corresponds to the lowest numbered target port (1 not 4) on a node. Valid mask

values range from 0000 (no ports enabled) to 1111 (all ports enabled). For

example, a mask of 0011 enables port 1 and port 2.






SVC Cluster zone for a single core fabric with a 4 port nodeContinued

Cisco Example

CLI View of one Fabric /4 node SVC− zone name SVC_ALL_PORTS vsan 10

− fcalias name RUBSTLSVC01_N1_P3 vsan 10

− pwwn 50:05:07:68:01:10:b3:88


− pwwn 50:05:07:68:01:10:b3:5e


− pwwn 50:05:07:68:01:10:b3:65


− pwwn 50:05:07:68:01:10:b3:7a


− pwwn 50:05:07:68:01:40:b3:7a


−

pwwn 50:05:07:68:01:40:b3:65− fcalias name RUBSTLSVC01_N1_P1 vsan 10

− pwwn 50:05:07:68:01:40:b3:88


− pwwn 50:05:07:68:01:40:b3:5°

GUI View of one Fabric/4 node SVC

SVC Node Ports






DS8K Right I/O EnclosuresBay 1

1

3

0

1

3

1

1

3

2

1

3

3

C1R4

1

0

0

1

0

1

1

0

2

1

0

3

C0R2

Bay 3

3

3

0

3

3

1

3

3

2

3

3

3

C1R7

3

0

0

3

0

1

3

0

2

3

0

3

C0R3

Bay 5

5

3

0

5

3

1

5

3

2

5

3

3

C1R8

5

0

0

5

0

1

5

0

2

5

0

3

C0R4

Bay 7

7

3

0

7

3

1

7

3

2

7

3

3

C1R77

0

0

7

0

1

7

0

2

7

0

3

C0R3

DS8K Left I/O EnclosuresBay 0

0

0

0

0

0

1

0

0

2

0

0

3

C1L4

0

3

0

0

3

1

0

3

2

0

3

3

C0L2

Bay 2

2

0

0

2

0

1

2

0

2

2

0

3

C1L7

2

3

0

2

3

1

2

3

2

2

3

3

C0L3

Bay 4

4

0

0

4

0

1

4

0

2

4

0

3

C1L8

4

3

0

4

3

1

4

3

2

4

3

3

C0L4

Bay 6

6

0

0

6

0

1

6

0

2

6

0

3

C1L7

6

3

0

6

3

1

6

3

2

6

3

3

C0L3

iogrp 02048 LUNs max

Node 1

HBA 1

P1 P2 P3 P4

HBA 2

Node 2

HBA 1

P1 P2 P3 P4

HBA 2

Node 3

HBA 1

P1 P2 P3 P4

HBA 2

Node 4

HBA 1

P1 P2 P3 P4

HBA 2


4 Node SVC MAX Vdisk 4096 wwpn5005076801

DIR1 SAN Fabric DIR2 SAN Fabric

Supported DS8K to SVC ZoningEither ports 1&3 and 2&4 should be

zoned to a fabric or ports as shown on the

next page, both configs are supported

Newer SVC nodes may contain

one HBA card with 4 ports

port1=11052ca port1=11052b7 port1=110529e port1=110528b




Example Back-end Storage to SVC ZoningStorage to SVC Zoning





© 2013 IBM Corporation

SAN Fabric 1

15

Correct Storage to SVC zoning with 8 ports in an existing environment

SAN Fabric 2






SAN Fabric 1

16

Correct Storage to SVC zoning with 8 ports in a new environment

SAN Fabric 2






Zoning Best Practices - Continued

DS8K Right I/O Enclosures

Bay 1

13

0

13

1

13

2

13

3

C1R4

10

0

10

1

10

2

10

3

C0R2

Bay 3

33

0

33

1

33

2

33

3

C1R7

30

0

30

1

30

2

30

3

C0R3

Bay 5

53

0

53

1

53

2

53

3

C1R8

50

0

50

1

50

2

50

3

C0R4

Bay 7

73

0

73

1

73

2

73

3

C1R7

70

0

70

1

70

2

70

3

C0R3


00

0

00

1

00

2

00

3

C1L4

03

0

03

1

03

2

03

3

C0L2

Bay 2

20

0

20

1

20

2

20

3

C1L7

23

0

23

1

23

2

23

3

C0L3

Bay 4

40

0

40

1

40

2

40

3

C1L8

43

0

43

1

43

2

43

3

C0L4

Bay 6

60

0

60

1

60

2

60

3

C1L7

63

0

63

1

63

2

63

3

C0L3


Node 1

HBA 1

P1 P2 P3 P4

HBA 2

Node 2

HBA 1

P1 P2 P3 P4

HBA 2

Node 3

HBA 1

P1 P2 P3 P4

HBA 2

Node 4

HBA 1

P1 P2 P3 P4

HBA 2








Application Host Server

A2 B2

The same native

backend ports should not be

shared for both direct host connectivity

and SVC connectivity as shown here.

The correct way to bypass SVC is to use

other backend ports not connected to the

SVC

DS8K Right I/O Enclosures

Bay 1

13

0

13

1

13

2

13

3

C1R4

10

0

10

1

10

2

10

3

C0R2

Bay 3

33

0

33

1

33

2

33

3

C1R7

30

0

30

1

30

2

30

3

C0R3

Bay 5

53

0

53

1

53

2

53

3

C1R8

50

0

50

1

50

2

50

3

C0R4

Bay 7

73

0

73

1

73

2

73

3

C1R7

70

0

70

1

70

2

70

3

C0R3


00

0

00

1

00

2

00

3

C1L4

03

0

03

1

03

2

03

3

C0L2

Bay 2

20

0

20

1

20

2

20

3

C1L7

23

0

23

1

23

2

23

3

C0L3

Bay 4

40

0

40

1

40

2

40

3

C1L8

43

0

43

1

43

2

43

3

C0L4

Bay 6

60

0

60

1

60

2

60

3

C1L7

63

0

63

1

63

2

63

3

C0L3


Node 1

HBA 1

P1 P2 P3 P4

HBA 2

Node 2

HBA 1

P1 P2 P3 P4

HBA 2

Node 3

HBA 1

P1 P2 P3 P4

HBA 2

Node 4

HBA 1

P1 P2 P3 P4

HBA 2








Application Host Server

A2 B2

The same native

backend ports should not be

shared for both direct host connectivity

and SVC connectivity as shown here.

The correct way to bypass SVC is to use

other backend ports not connected to the

SVC

The same port on the back-end is being used for host and SVC

SVC Correct Example SVC Incorrect Example






Zoning Best Practices - XIV





vdisk1vdisk2

Preferred path for vdisk1 is SVC

N1P2 & N1P3

Non Preferred path for vdisk1

is SVC N2P2 &N2P3

Preferred path for vdisk2 is SVC

N2P2 & N2P3

Non Preferred path for vdisk2

is SVC N1P2 &N1P5

DEV#: 5 DEVICE NAME: hdisk5 TYPE: 2145 ALGORITHM: Load Balance

SERIAL: 600507680181059C4000000000000007

==============================================================

Path# Adapter/Path Name State Mode Select Errors

0 fscsi0/path0 OPEN NORMAL 1996022 0

1* fscsi0/path1 OPEN NORMAL 29 0



Server/Host view of the datapaths

Examples of correct Host to SVC Cluster zoning prior to 8 ports


IBM S t St





SAN Fabric 1

20

vdisk1 vdisk2

Preferred path fo r vdis k1 is SVC

N1P1 & N1P5Non Preferred path for v disk1 is

SVC N2P1 &N2P5

Preferred path for vdis k2 is SVC

N2P1 & N2P5Non Preferred path for v disk2 is

SVC N1P1 &N1P5


SERIAL: 600507680181059C4000000000000007

==============================================================







Examples of correct Host to SVC Cluster zoning with 8 ports

Host

B1 A1

SAN Fabric 2


IBM S t St





Examples of incorrect Host to SVC Cluster zoning

vdisk1

Zone for p770_1_vio1a_b1

10000000c9779a4b500507680120B374

500507680140B374

500507680120B363

500507680140B363

2 Node

Zone for p770_1_vio1a_d1

10000000c9779a4a500507680110B374

500507680130B374

500507680110B363

500507680130B363

DEV#: 3 DEVICE NAME: hdisk3 TYPE: 2145 ALGORITHM: LoadBalance

SERIAL: 600507680181059BA000000000000005

==========================================================








6* fscsi2/path6 OPEN NORMAL 197 0 7 fscsi2/path7 OPEN NORMAL 493451 0









SVC Host Definitionsid:29

name:rocviof5san1

fsc5=10000000c9779a4b

fcs7=10000000C9973C5b

fcs2=10000000C9779a4a

fcs0=10000000C9973A5a

In

Both the zoning and SVC host definitions make this incorrect

Extra Paths

Should be either

1’s or 3’s

Not both

Extra Paths

Should be either

2’s or 4’s

Not both


IBM S t St

I t Z i E l






Node 1

HBA 1

P1 P2 P3 P4

HBA 2

Node 2

HBA 1

P1 P2 P3 P4

HBA 2

Node 3

HBA 1

P1 P2 P3 P4

HBA 2

Node 4

HBA 1

P1 P2 P3 P4

HBA 2


4 Node SVC MAX Vdisk 4096 CF8wwpn

5005076801


b03vio101Host

fcs0 fsc5

SVC Host Definitions

id:29

name:rocviof5san1

fsc5=10000000C9973A0B

fcs7=10000000C9973CDB

fcs2=10000000C9973A06

fcs0=10000000C9973A26

fcs2 fcs7


SERIAL: 60050768018E822BB8000000000000CE

============================================================






port1=104577 port1=104B04 port1=104585 port1=10457C




vdisk1 vdisk1vdisk1 vdisk1


- Vdisk1 has a path on all 4 Host adapters

- Vdisk1 has a path on 4 SVC ports in iogrp 0

Fabric A: Zone Definitions

zone: rocviof5san1_fcs0_cokrocsvc01_p2

CKROCSVC01_n1p2

CKROCSVC01_n3p2

CKROCSVC01_n5p2

CKROCSVC01_n7p2

rocviof5san1_fcs0zone: rocviof5san1_fcs2_cokrocsvc01_p2

CKROCSVC01_n2p2

CKROCSVC01_n4p2

CKROCSVC01_n6p2

CKROCSVC01_n8p2

rocviof5san1_fcs2

Fabric B: Zone Definitions


CKROCSVC01_n1p3

CKROCSVC01_n3p3

CKROCSVC01_n5p3

CKROCSVC01_n7p3rocviof5san1_fcs5


CKROCSVC01_n2p3

CKROCSVC01_n4p3

CKROCSVC01_n6p3

CKROCSVC01_n8p3

rocviof5san1_fcs7

5

7

Inco rrect Zon ing Examp le

Note: Each Iogrp wi l l on ly see either 2 preferred or two non preferred or 1 preferred and 1 non preferred

Also this m ethod wi l l l imi t the max VDisks per host id to 512 f rom a possible 1024 in this scenar io

Input provided by Keith Williams for content


IBM S t St TM





Incorrect SVC zoning definitions to multiple Host HBAs


Node 1

HBA 1

P1 P2 P 3 P4

HBA 2

Node 2

HBA 1

P1 P2 P 3 P4

HBA 2

Node 3

HBA 1

P1 P2 P 3 P4

HBA 2

Node 4

HBA 1

P1 P2 P 3 P4

HBA 2




b03vio101b03vio101

A1 A2 A3 B1


name:b03vio100_apvg

10000000C97549BD

10000000C97549BC

id:3

name:b03vio100_dbvg

10000000C9771AAF

10000000C9771AAE

Port to Fabrics

not evenly distributed

In this example we show

2 SVC Host definitions, for 1 host,

with uneven port to fabric distribution

and 4 zones As you can see...A1 will failover to A2,

on the same fabric..this is a Single point

of Falure (SPoF)

See the next page for correct zoning

where there are multiple Host HBAs,

zoned to the SVC





Fabric_A Zone Definitions

Zone 101_a Zone_101_d A1=10000000c97549bc B2=10000000c9771aae

P1=50050768011052ca P1=50050768011052ca

P1=50050768011052b7 P1=50050768011052b7

P1=500507680110529e P1=500507680110529e

P1=500507680110528b P1=500507680110528b

Zone_101_b A2=10000000c97549bd

P1=50050768013052ca

P1=50050768013052b7

P1=500507680130529e

P1=500507680130528b

Fabric_B Zone Definitions

Zone 101_c

B1=10000000c9771aaf P1=50050768014052ca

P1=50050768014052b7

P1=500507680140529e

P1=500507680140528b








Node 1

HBA 1

P1 P2 P3 P4

HBA 2

Node 2

HBA 1

P1 P2 P3 P4

HBA 2

Node 3

HBA 1

P1 P2 P 3 P4

HBA 2

Node 4

HBA 1

P1 P2 P 3 P4

HBA 2




b03vio101_a

A1 B1


name:b03vio101_a

10000000C97549BC10000000C9771AAF

id:3

name:b03vio101_b

10000000C97549BD

10000000C9771AAE


Zone 101_a A1=10000000c97549bc

P1=50050768011052ca

P1=50050768011052b7

P1=500507680110529eP1=500507680110528b

Zone_101_b

A2=10000000c97549bd

P1=50050768013052ca

P1=50050768013052b7

P1=500507680130529e

P1=500507680130528b

The host definitions in the SVC

should be defined as 2 hosts

and 4 zones in the Fabrics


2 seperate SVC Host definitions and

2 Fabric zones, per host per Fabric

Totaling 4 zones

This allows proper failover from Primary

to Alternate nodes within the SVC iogrp.

b03vio101_b

A2 B2






Zone 101_aB1=10000000c9771aaf

P1=50050768014052ca

P1=50050768014052b7

P1=500507680140529e

P1=500507680140528b

Zone_101_bB2=10000000c9771aae

P1=50050768012052ca

P1=50050768012052b7

P1=500507680120529e

P1=500507680120528b

Correct SVC zoning definitions to multiple Host HBAs







b03vio210

A1A2A3A4 B1B2B3B4


Node 1

HBA 1

P1 P2 P3 P4

HBA 2

Node 2

HBA 1

P1 P2 P3 P4

HBA 2

Node 3

HBA 1

P1 P2 P3 P4

HBA 2

Node 4

HBA 1

P1 P2 P3 P4

HBA 2





name:b03vio210_allvg10000000C942B618

10000000C942B44B

10000000C9428016

10000000C94289AC

10000000C93FF72F

10000000C93F7416

10000000C93F8054

10000000C93F7537


A1=10:00:00:00:c9:3f:75:37

A2=10:00:00:00:c9:3f:80:54

A3=10:00:00:00:c9:42:89:ac

A4=10:00:00:00:c9:42:80:16


B1=10:00:00:00:c9:3f:74:16

B2=10:00:00:00:c9:3f:f7:2f

B3=10:00:00:00:c9:42:b4:4b

B4=10:00:00:00:c9:42:b6:18


1 SVC Host definitions and

2 Fabric zones, per host per Fabric.

This does not allow proper failover from

Primary to Alternate nodes within the

SVC iogrp.

See the next page for proper zoning

Incorrect SVC zoning definitions to multiple Host HBAs







b03vio210_2

A2 B2


Node 1

HBA 1

P1 P2 P3 P4

HBA 2

Node 2

HBA 1

P1 P2 P3 P4

HBA 2

Node 3

HBA 1

P1 P2 P3 P4

HBA 2

Node 4

HBA 1

P1 P2 P3 P4

HBA 2




SVC Host Definitionsid:1 name:b03vio210_1

10000000C93F7537

10000000C9428016

id:2 name:b03vio210_2

10000000C93F8054

10000000C93FF72F


10000000C94289AC

10000000C942B44B


10000000C93F7416

10000000C942B618


4 seperate SVC Host definitions and

2 seperate Fabric zones, per host

per Fabric

This does allow proper failover from

Primary to Alternate nodes within the

SVC iogrp.

b03vio210_3

A3 B3

b03vio210_4

A4 B4

b03vio210_1

A1 B1


A1=10:00:00:00:c9:3f:75:37Fabric_B Zone Definitions

B4=10:00:00:00:c9:42:b6:18


A4=10:00:00:00:c9:42:80:16Fabric_A Zone Definitions

A3=10:00:00:00:c9:42:89:ac


A2=10:00:00:00:c9:3f:80:54


B3=10:00:00:00:c9:42:b4:4b

Fabric_B Zone DefinitionsB2=10:00:00:00:c9:3f:f7:2f


B1=10:00:00:00:c9:3f:74:16

Correct SVC zoning definitions to multiple Host HBAs








Node 1

HBA 1

P1 P2 P3 P4

HBA 2

Node 2

HBA 1

P1 P2 P3 P4

HBA 2

Node 3

HBA 1

P1 P2 P3 P4

HBA 2

Node 4

HBA 1

P1 P2 P3 P4

HBA 2


4 Node SVC MAX Vdisk 4096 CF8wwpn

5005076801


b03vio101NRPOKVIO1A

d1 d3


name:P770_1_vio1A

10000000C9C0B3DB

10000000C9C0DC7F

10000000C9C0E0E0

10000000C9C0A984

fscsi0=10000000C9C0A984

fscsi2=10000000C9C0E0E0

fscsi5=10000000C9C0DC7F

fscsi7=10000000C9C0B3DB

port1=10B374 port1=10B363 port1=10B371 port1=10B335





10000000c9779a4a

500507680110B374500507680130B374

500507680110B363

500507680130B363

500507680110B371

500507680130B371

500507680110B335

500507680130B335

Zone for p770_1_vio1a_d3_SVC

10000000C9C0DC7F

500507680120B374

500507680140B374

500507680120B363

500507680140B363

500507680120B371

500507680140B371

500507680120B335

500507680140B335

d2 d4

Zone for p770_1_vio1a_d4_SVC

10000000C9C0B3DB

500507680120B374

500507680140B374500507680120B363

500507680140B363

500507680120B371

500507680140B371

500507680120B335

500507680140B335


10000000C9C0E0E0

500507680110B374500507680130B374

500507680110B363

500507680130B363

500507680110B371

500507680130B371

500507680110B335

500507680130B335

Over subscribed SVC to HostHBA Zoning causing to manydatapathsDEV#: 3 DEVICE NAME: hdisk3 TYPE: 2145 ALGORITHM: LoadBalance

SERIAL: 600507680181059BA000000000000005

==========================================================




2* fscsi0/path2 OPEN NORMAL 197 0 3 fscsi0/path3 OPEN NORMAL 493559 0













Server/Host Datapath - Oversubscribed







SDD Driver Testing for Proper HBA Failover

On an AIX vio server, check the AIX system to verify IO activity still continues on thealternate ports by testing with SDD and /or SDDPCM commands

The Server Admin - Create a mount point for logical volume that can be manipulated to generate IO traffic for

the purpose of this test

The Server Admin - Verify and record selected (targets yet to be determined) datapaths for preferred and

alternate status (active and inactive) by using the SDD "pcmpath query device" or "datapath query device"

command on the AIX vio server

Note the path selection counts on the multiple paths. There should only be two paths under the "Select"column, above zero (0). These are the two open paths on the preferred node. (if paths 0 and 2 show numbers

under the "Select " column, other than zer0, then do the following:

1. Take one path off-line by issuing the command (pcmpath set device 0 path 0 offline) or (datapath set device 0 path 0 offline) -

Path 0 should now be in a dead state .

2. Go to the mount point of lv and edit a f ile to create traffic. After creating the traffic, reissue the pcmpath or datapath query

command "pcmpath query device" or "datapath query device" and look at the path selection numbers. Notice only path selection

count for Path 2 increased for the other preferred path

3. Close Path 2 by issuing the command "pcmpath set device 0 path 2 offline" or "datapath set device 0 path 2 offline"

4. Return to the mount point and add or edit files to create IO.

5. Execute the "pcmpath query device" or "datapath query device“ command, to look at the path selection count. Disk access

should now be via the other paths. (This is now load balancing to the non-prefered SVC node for this Vdisk)

Reestablish both preferred paths by executing the following commands: "pcmpath set device 0 path 0 online" and "pcmpath set

device 0 path 2 online" or "datapath set device 0 path 0 online" and "datapath set device 0 path 2 online”







Non SDD Driver Testing for Proper HBA Failover

Further "proper" testing should be done during a maintenance window

Testing the redundancy between the Fabric and the host

1. Open a change record to reflect the change (Make sure all necessary approvers are notified)

2. The Server Admin - Identify and verify which host HBA's are active for I/O activity by performing a test read and write to theSAN disk from the host

3. The Server Admin - Stop I/O between the host and the Disk Storage

4. The SAN Admin - On the San Fabric, disable the Switch port on the "even" fabric zoned between the host and the storagedevice.

5. The Server Admin - Perform another read/write test to the same LUN

6. The Server Admin - Identify and verify which host HBA is active for I/O activity

7. The SAN Admin - On the even SAN fabric enable the Switch port

8. The SAN Admin - On the San Fabric, disable the Switch port on the "odd" fabric zoned between the host and the storagedevice.

9. The Server Admin - Perform another read/write test to the same LUN

10. The Server Admin - Identify and verify which host HBA is active for I/O activity

11. The SAN Admin - the odd SAN fabric enable the Switch port

If the I/O activity toggles between the two HBA's then the test is successful

When a new Host server or Storage device is added to the environment testing is strongly recommended

Note: Ideally this type of test is best done during the initial implementation of newequipment, before it is turned over to the customer or placed in production





IBM System Storage


Host Multipath Configuration Best Practices examples-Running a script can show current status

For AIX vio - driver used - sddpcm

– Note: Script to be posted when available

– Multipath installed = Yes, Set to: 4 paths/ hdisk, fcsi_settings:2145: fast_fail,

– Multipath Policy =load_balance

For Linux/ESX/VMWare - – https://w3-connections.ibm.com/files/app#/file/1ba027ec-5d20-4c60-a281-f18f16192f7a

– Device –mapper – multipath HBA elements=4,

– For Windows – driver used = MPIO=SDDDSM – https://w3-connections.ibm.com/files/app#/file/3e52f54c-a445-4b17-aa5d-a5da43d4bedb

– Multipath installed = Yes, HBA elements = 4, MPIO Policy = Optimized

For Solaris – driver MPxIO Https://w3-connections.ibm.com/files/app#/file/66ea3228-4b26-48bd-a8fd-55751a02fc42

− Multipath installed = MPxIO, Path Subscription= 4, MPIO Policy = round-robin

Content input from : Bill Marshall, Jason Moras, Brad Worthen, Ramesh Palakodeti



https://w3-connections.ibm.com/files/app#/file/1ba027ec-5d20-4c60-a281-f18f16192f7a

https://w3-connections.ibm.com/files/app#/file/66ea3228-4b26-48bd-a8fd-55751a02fc42




























IBM System Storage


SUN Solaris Configuration Best PracticesOn the SVC

SVC (V4.2 and later), Storwize V7000 supports load balancing for MPxIO hosts, if the host type called Target Port Group Support ('TPGS') is used.

The following steps are required to enable load balancing on Solaris MPxIO hosts

Change the SVC or Storwize V7000 host configuration so that the host type is tpgs

On the Host Server

Install the following patches from SUN

Solaris 9 - 113039-23 or higher

Solaris 10 sparc - 138888-01 or higher

Solaris 10 x86 - 138889-01 or higher

Check/Set the following parameters in /kernel/drv/scsi_vhci.conf:

load-balance="round-robin";auto-failback="enable";

Remove or comment out any lines which match the following:

device-type-scsi-options-list = "IBM 2145", "symmetric-option";symmetric-option = 0x1000000;

Activate the new configuration Solaris 9: reboot the Solaris host

Solaris 10: run "stmsboot -u"

Verify that the new configuration is active by following these steps:

Run 'luxadm display /dev/rdsk/cXtYdZs2'where cXtYdZs2 is a SVC or Storwize V7000 device

Check that half of the paths (the ones to the preferred node) are shown as PRIMARY/ONLINE and the other half (theones to the non-preferred node) are shown as SECONDARY/ONLINE.





IBM System Storage


Host to SVC Zoning Best Practices


SERIAL: 600507680181059C4000000000000007==============================================================






Example of correct/incorrect number of datapaths with Host to SVC zoning

Correct datapathing has 3 factors

− Proper zoning

− Proper SVC Host definitions (SVC logical config of the host def)

− Proper redundancy for the SVC preferred /non preferred pathing


SERIAL: 600507680181059BA000000000000005============================================================










8 fscsi5/path8 OPEN NORMAL 492225 09* fscsi5/path9 OPEN NORMAL 197 0







Correct Incorrect





IBM System Storage


Correct way to make MM/GM zone, new implementation of CG8, code 7.1.x

With 7.1 you can use port masking to tell the cluster what ports to use for heartbeat/MM/GM

Ports 7-8 cannot be used for storage connections. Ports 7-8 can only be used for SVC to

host connections, (heartbeat) and / or GM/MM traffic.

– Exclude the targeted GM/MM and heartbeat ports from your normal host to SVC andstorage to SVC traffic)

For heartbeat on existing environments – put/dedicate all port 7’s (:70:) in one zone on one

fabric and all port 8’s (:80:) in one zone on the other fabric

– In scenarios where replication (GM/MM) is not part of the environment, ports 7 and 8

would be used for heartbeat and 5 and 6 would be used for disk and host connections /traffic

For heartbeat on new builds, take care to use physical port 2(:30:) on HBA1 and port 8(:80:)

on HBA2 for more resiliency across HBA cards or physical ports 3(:10:) and 7(:70:)

respectively to avoid a total failure in the event of one card failing

Input for port assignments from Keith Williams





IBM System Storage


New zoning BP with 7.1.0.x and 8 port nodes, on existing

Think of 4 types of zones per fabric (Use case - existing environment, adding 2 FA

HBA)

FA-ST1

Storage Zone

N1P1, N1P3, N2P1, N2P3

SP1, SP3

FA-Host 1

Host Zone

N1P1&N2P1or N1P3&N2P3,

Host HBA 1

FA-SVC Zone

Node to Node/Heartbeat

N1P8, N2P8

FA-MM/GM

Mirror Zone

N1P6, N2P6,

RN1P6, RN2P6

FB-ST1

Storage Zone

N1P4, N1P2, N2P4, N2P2

SP2, SP4

FB-Host 1

Host ZoneN1P2&N2P2 or N1P4&N2P4

Host HBA 2

FB-SVC Zone


N1P7, N2P7

FB-MM/GM

Mirror Zone

N1P5, N2P5,

RN1P5, RN2P5

Input for deck provided by Carlos Fuente, Byron Grossnickle, Bill Weigand, Brian Sherman, Mark Chitti, Keith Williams , Jim Olson

and Chuck Laing (GTS/ITD) Slide provided by Chuck Laing




y g


New zoning BP with 7.1.0.x and 8 port nodes on “ new ” build

Think of 4 types of zones per fabric-Use case – new build, with resiliency across FA

HBAs

FA-ST1

Storage Zone

N1P1, N1P6, N2P1, N2P6

SP1, SP3

FA-Host 1

Host Zone

N1P1&N2P1 or N1P6&N2P6

Host HBA 1

FA-SVC Zone


N1P3, N2P3…

FA-MM/GM

Mirror Zone

N1P8, N2P8,

RN1P8, RN2P8

FB-ST1

Storage Zone

N1P4, N1P5, N2P4, N2P5

SP2, SP4

FB-Host 1

Host Zone

N1P5&N2P5 or N1P4&N2P4

Host HBA 2

FB-SVC Zone


N1P7, N2P7…

FB-MM/GM

Mirror Zone

N1P2, N2P2,

RN1P2, RN2P2






y g


WAN direct connection to the

FCiP connection to the SVC ports

GM/MM zone A



GM/MM zone B

Adding 2 nd HBA - MM/GM zone, existing environment - 7.1.0.x

Make the local_fc_port_mask (Node to Node/heartbeat) = 11000000

Make the partner_fc_port_mask (MM/GM) = 00110000






y g




GM/MM zone A



GM/MM zone B



Make the partner_fc_port_mask (MM/GM = 10000010






y g


The Correct way to make the SVC Global Mirror Zone with 4 portnodes prior to 8 ports

One SVC port from each node on each fabric should be zoned for GM traffic, taking care not to include both ports that a server might use. This means ports thatwould also be used for replication would be either ports 1 & 2, or ports 3 & 4

For each node in a cluster, exactly two fibre channel ports should be zoned to exactly two fibre channel ports from each node in the partner cluster.

If dual-redundant ISLs are available, then the two ports from each node should be split evenly between the two ISLs, i.e. exactly one port from each node should bezoned across each ISL.

Local cluster zoning should continue to follow the standard requirement for all ports on all nodes in a cluster to be zoned to one another. This is discussed more verbosely on the Flash published for this issue on the IBM website:

http://www-01.ibm.com/support/docview.wss?uid=ssg1S1003634 and https://www-304.ibm.com/support/docview.wss?uid=ssg1S1003634

Input for deck provided by Keith Williams and Chuck Laing (GTS/ITD)








IBM Systems39

Zoning for Dual Core Fabrics

Planning design for correct “Dual Core” fabric

Problem examples

► Dual Core zoning problem - Backend to SVC zoning

► Dual Core zoning problem - Host to SVC zoning

Added Dual Core Fabric zoning rule

SVC Single vs. Dual core Fabric “Zone” example

Incorrect Dual Core zone/connectivity example

Correct Dual Core zone/connectivity example

Correct port zone/connectivity breakout example

Examples - right/wrong Host to SVC zoning

Dual Core MM/GM example

Question and Answer




IBM Systems40

Planning design for correct “Dual Core” fabric zoning example

Input for deck provided by John Locke, Kirby Dahman and Chuck Laing (GTS North America SO)

Slide provided by John Locke

Example: SVC Back-end Dual-Core Storage Zoning Fix



Problem: A single back-end zone had ports p1 and p2 from SVC 7 and port a3 from XIV (example). SVC 7 will form

paths from p1 to a3 (purple) and from p3 to a3 (green). Purple paths traverse 2 cores and 1 edge switch – interfering

with host edge-to-core traffic. This issue is pervasive. Also, eliminate (unnecessary) separate heartbeat zone.

Fix: Split zones so that only common-core switch (green paths) can occur. Reduce paths, contention, latency. Also

group XIVn array ports attaching common core switch in common zone.

p g g

SVC7 node1 SVC7 node8…

p1 p1p3 p3

XIV1 m1 XIV1 m3…

a1a3 a3a1

Original zone:(XIV1m3a3, svc7n1p1, svc7n1p3,

…)

New zones: (XIV1m1a1,

XIV1m2a1, XIV1m3a1, svc7n1p1,

… svc7n8p1), (XIV1m1a3,

XIV1m2a3, XIV1m3a3, svc7n1p3,

… svc7n8p3)

Slide provided by Kirby Dahman

Example: SVC Front-end Dual-Core Zoning Fix



Problem: Many host HBA ports are in a single zone with SVC ports cabled to different core switches. This does not hurt

the host directly, but as a side-effect it permits SVC inter-node cache updates (after each host write) from preferred to

non-preferred node in an IOgroup to cross from core-switch-to-edge switch-to-core switch (see purple path).

Fix: Split zones so that only common-core switch (green paths) can occur for inter-node cache updates.

p g

SVC7 node1 SVC7 node2

p1 p1p3 p3

… SVC7 node 7 SVC7 node 8

p1 p1 p3p3

Iogroup 0 Io rou 3

Host Xh1

Original zone: (HostX-h1,svc7n1p1, svc7n1p3, svc7n2p1,

svc7n2p3, …)

New zones: (HostX-h1,

svc7n1p1, svc7n2p1, …),

(HostX-h1, svc7n1p3, svc7n2p3)

Slide provided by Kirby Dahman





A Single core cluster “zone” BrocadeExample

CLI View of one Fabric /8 node SVC− zone: lsan_allsvc_zone_a

− 50:05:07:68:01:10:54:28

− 50:05:07:68:01:40:54:28

− 50:05:07:68:01:10:54:62

− 50:05:07:68:01:40:54:62

− 50:05:07:68:01:10:51:07

− 50:05:07:68:01:40:51:07

− 50:05:07:68:01:10:54:73

− 50:05:07:68:01:40:54:73

− 50:05:07:68:01:10:4e:c3

− 50:05:07:68:01:40:4e:c3

− 50:05:07:68:01:10:45:64

− 50:05:07:68:01:40:45:64

− 50:05:07:68:01:10:4f:67

− 50:05:07:68:01:40:4f:67

−

50:05:07:68:01:10:4f:66− 50:05:07:68:01:40:4f:66

A Dual Core cluster “zone” BrocadeExample

CLI View of one Fabric /8 node SVC

“Fabric 1 Core1” − zone: lsan_allsvc_zone_a

− 50:05:07:68:01:10:54:28

− 50:05:07:68:01:10:54:62

− 50:05:07:68:01:10:51:07

−

50:05:07:68:01:10:54:73− 50:05:07:68:01:10:4e:c3

− 50:05:07:68:01:10:45:64

− 50:05:07:68:01:10:4f:67

− 50:05:07:68:01:10:4f:66

CLI View of one Fabric /8 node SVC

“Fabric Core2”

− 50:05:07:68:01:40:54:28

− 50:05:07:68:01:40:54:28

− 50:05:07:68:01:40:54:62

− 50:05:07:68:01:40:51:07

− 50:05:07:68:01:40:54:73

− 50:05:07:68:01:40:4e:c3

− 50:05:07:68:01:40:45:64

− 50:05:07:68:01:40:4f:67

− 50:05:07:68:01:40:4f:66

43

SVC Cluster zone for a “Single vs. Dual Core” per fabric






Incorrect “Dual Core” fabric zoning example for a 2 node SVC Cluster







Correct “Dual Core” fabric zoning example for a 2 node SVC Cluster







Correct “Dual Core” fabric zoning example for a 4 node SVC Cluster







Incorrect vs. correct Dual core fabric / Host to SVC port zoning

Host

B1 A1

Fabric1 Core1 Fabric 1 Core2

4 Node CG8 SVC

Node 1

12 34

HBA 1

1

0

2

0

4

0

3

0

56 78

HBA 2

7

0

8

0

5

0

6

0


Node 2

12 34

HBA 1

1

0

2

0

4

0

3

0

56 78

HBA 2

7

0

8

0

5

0

6

0

Node 1

12 34

HBA 1

1

0

2

0

4

0

3

0

56 78

HBA 2

7

0

8

0

5

0

6

0


Node 2

12 34

HBA 1

1

0

2

0

4

0

3

0

56 78

HBA 2

7

0

8

0

5

0

6

0


Host

B1 A1


4 Node CG8 SVC

Node 1

12 34

HBA 1

1

0

2

0

4

0

3

0

56 78

HBA 2

7

0

8

0

5

0

6

0


Node 2

12 34

HBA 1

1

0

2

0

4

0

3

0

56 78

HBA 2

7

0

8

0

5

0

6

0

Node 1

12 34

HBA 1

1

0

2

0

4

0

3

0

56 78

HBA 2

7

0

8

0

5

0

6

0


Node 2

12 34

HBA 1

1

0

2

0

4

0

3

0

56 78

HBA 2

7

0

8

0

5

0

6

0


Incorrect Correct






Incorrect vs. correct Dual core fabric / Host to SVC port zoning

Host1

B1 A1


4 Node CG8 SVC

Node 1

12 34

HBA 1

1

0

2

0

4

0

3

0

56 78

HBA 2

7

0

8

0

5

0

6

0


Node 2

12 34

HBA 1

1

0

2

0

4

0

3

0

56 78

HBA 2

7

0

8

0

5

0

6

0

Node 1

12 34

HBA 1

1

0

2

0

4

0

3

0

56 78

HBA 2

7

0

8

0

5

0

6

0


Node 2

12 34

HBA 1

1

0

2

0

4

0

3

0

56 78

HBA 2

7

0

8

0

5

0

6

0


Host2

B1 A1

Correct Correct








Make the partner_fc_port_mask (MM/GM = 01000100

Blue ports go to one Fabric

Red Ports go to the other Fabric

Logical port with wwpn # embedded

Physical port number

SVC has to be connected to an F-port. If the FC-IP router device provides the F-port thenconnect to the FCiP

GTS/SSA/SO BP is to by pass the SAN and directly co nnect into the

FCIP Rou terSome routers (a popular example comes from Brocade) are a combination switch and

FCiP device.

The SVC might in some cases be s connected to a switch port. But that switch might also

provide local fabric connectivity - not just long distance connectivity

Some routers are implemented as blades in a director switch. In this case you have to

connect SVC to the core





innovation that matters

Question and Answer Time

svc zoning 101 best practices_q1_2014-v1.2

Documents