architecting fibre channel ha solutions

Architecting Fibre Channel HA Solutions

Rick Jooss

richard.jooss@netapp.com

2NetApp Confidential -- Do Not Distribute

Agenda

CFModes

Single System Imagine

Multipathing

Host Clustering

Storage System Backend HA

Agenda

CFModes

Single System Image

Multipathing

Host Clustering

CFMODE – Cluster Failover Mode

What is CFMODE?– FCP Setting– Determines behavior of FC Target Ports, particularly

during a CFO event

Why is there more than one CFMODE?– Original CFMODE (standby) did not work for all host

types (HP-UX, AIX)– Original CFMODE did not work with the 270C

because it only has a single FC port

Available Paths- Standby Mode

Switch/Fabric 1 Switch/Fabric 2

LUNs LUNs

0c 0d 0c 0d0a 0b 0a 0bHA Configuration

Controller 1 Controller 2

Solid Blue are paths to the LUNs being served by Controller 1

Dashed Purple are paths to the LUNs being served by Controller 2

Path Access (Switch Failure) – Standby Mode

LUNs LUNs

Solid and Blue are paths to the LUNs being served by Controller 1

Dashed and Purple are paths to the LUNs being served by Controller 2

Switch/Fabric 1 will experience a failureMP layer works around the failure

Controller 2 Takes over all operations

Path Access (CFO event)- Standby Mode

LUNs LUNs

Conntroller 1 will experience a failure

Filer Head 2 Takes over all operationsMP layer is not involved in switchover

Path Access (CFO event)- Standby Mode

LUNs LUNs

Controller 1 will experience a failure

WWN1 WWN2 WWN3 WWN4 WWN5 WWN6 WWN7 WWN8

Available Paths - Partner Mode

LUNs LUNs

Available Paths - Partner Mode – FAS3000 Default Configuration

LUNs LUNs

0c 0d 0c 0dHA Configuration

Available Paths - Dual Fabric

HA Configuration

LUNsLUNs

0c_00c_2 0c_2

Agenda

CFModes

Multipathing

Host Clustering

What is the single system image cfmode?

Universal cfmode– Works on all HA storage systems– Works on all switches

Presents the HA configuration as a single target

All LUNs are visible on all controller ports

All hosts require multipathing software

0c 0c 0d0dHA Configuration

Available Paths - Single System Image – Single Card

LUNs LUNs

Path Access (Switch Failure) - Single System Image – Single Card

LUNs LUNs

Solid and Blue are paths to the LUNs being served by Head 1

Dashed and Purple are paths to the LUNs being served by Head 2

MP layer works around the failureSwitch/Fabric 1 will experience a failure

Path Access (CFO event) - Single System Image – Single Card

LUNs LUNs

Controller 2 takes over all operations MP layer works around the failure

Controller 1 will experience a failure

Available Paths - Single System Image – Single Port

LUNs LUNs

0d 0dHA Configuration

Available Paths - Single System Image – Single Port

LUNs LUNs

0d 0dHA Configuration

Loop ModeLoop Mode

Why SSI mode?

Works in all configurations

Makes us look more like other SAN vendors

Reduces port burn without using FC Loop– Fully redundant config requires only 1 “wire” per

controller, instead of 2.

Simpler wiring, no a/b port distinctions and no requirement to run the same cables from each controller to the same switch.

Management changes

Unified LUN mapping address space across the HA configuration.– Controller prevents these conflicts by checking with

the partner controller.

If the controller interconnect is down, some operations are disabled by default– Igroup add, lun map, lun online, igroup set ostype

SSI Roadmap

Introduced in ONTAP 7.1

Refer to FCP host compatibility matrix http://now.netapp.com/NOW/knowledge/docs/san/fcp_iscsi_config/index.shtml for specific host support

Agenda

CFModes

Multipathing

Host Clustering

Multipathing

Multipathing provides multiple paths from the host to the external storage device

Provides High-Availability – Protects against path failures– Ensures high availability of applications and data by

eliminating single points of failure

Provides Improved Performance – Increases potential performance by utilizing multiple

Multipathing

LUNs LUNs

A/P (active passive) policy – Single LUN

LUNs LUNs

A/P (active passive) policy – No Round Robining

LUN1 LUN3LUN2 LUN4

A/P (active passive) policy - Round Robining

LUN1 LUN3LUN2 LUN4

A/P (active/passive)

Active/Passive Configuration– 1 active path to a single LUN

• Performance to a LUN is limited by that paths capability (HBA, switch, target port)

– Possible to round robin multiple LUNs across multiple paths

– All other paths to the LUN are passive– On failover

• Primary paths are tried first• Secondary paths are used if no primary paths are

available

A/A (Active active) policy (cfmode = standby)

LUNs LUNs

A/A (active/active)

Host accessing data from a single LUN across multiple paths simultaneously– Typically used for load balancing

• Round Robin• Least Queue Depth• Weighted

– On failure I/Os are sent down remaining available paths

A/A/A (asymmetric active active)

LUNs LUNs

A/A/A (asymmetric active active)

Distinguishes between primary and secondary paths

Does active/active across primary paths only

Only uses secondary paths when no primary are available

NetApp’s Multipathing Strategy

2 pronged strategy– Support for “native” solutions

• What most customers rightly feel best about– Support for host and storage independent

solution• VERITAS• Allows common solution across various

server as well as storage variants

Multipathing For Windows

Windows MPIO– Uses the Microsoft standard infrastructure– A/P Policy– Automatically chooses primary paths for failover

before trying proxy ones– In standby the LUNS are automatically round

robined across all paths MPIO

Partner/SSI cfmode A/P

Standby cfmode A/P

Dual Fabric cfmode A/P

MultiPathing For Solaris

DMP 4.0 MPxIO

Partner/SSI cfmode A/A/A A/P

Standby cfmode A/A N/A

Dual Fabric cfmode A/P A/P

MultiPathing For Solaris

VERITAS DMP 4.0– NetApp ASL 4.0– Supports A/P, A/A, & A/A/A (Active Passive

Concurrent)

SUN Native MPxIO– Not supported with standby cfmode– Supports A/P – Can be A/A but required manual failback– Manual configuration required– Round Robining of the LUNs possible– Sometimes called

• Traffic Manager • Leadville Stack

MultiPathing For Linux

Qlogic– A/P Policy– Manually configured– Round Robining of LUNs is possible

DCM– Linux native solution

Qlogic DM

Partner/SSI cfmode A/P A/A/A

Standby cfmode A/P A/A

MultiPathing For AIX

DMP 4.0 SANpath MPIO

Partner/SSI cfmode A/A/A A/A/A A/A/A

Standby cfmode N/A N/A NA

Dual Fabric cfmode A/P A/P A/P

MultiPathing For AIX

SANpath– A/A/A– Automatically chooses primary paths for failover

before trying proxy ones– Special policy for SCSI-2 reservation

– Required for host clustering HACMP– Can only use A/P

VERITAS DMP 4.0– Only supports A/A/A

IBM MPIO– IBM native solution with NetApp PCM

Multipathing for HP-UX

Partner/SSI cfmode A/P A/P

Standby cfmode N/A N/A

PVLinks DMP 3.5

Multipathing for HP-UX

PVlinks/LVM– A/P policy– Single active path per LUN, user controlled– Ordering for remaining paths for failover– ntap_config_paths

• NETAPP script to define path ordering based on filer path types: primary, proxy

• automatically round robin primary paths among all LUNS

– Supports both FCP and iSCSI paths

VERITAS DMP 3.5– A/P Policy

Multipathing for VMware

VMware– A/P Policy– Manually configured– Round Robining of LUNs possible

VMware

Standby cfmode A/P

Multipathing for Netware

Novell– A/P Policy– Manually configured– Round Robining of LUNs possible

Novell

Standby cfmode A/P

Fibre Channel SAN Host Support

Partner/SSI cfmode

Windows“NTAP DSM”

Standby cfmode

Solaris “DMP”

Dual Fabric cfmode

Linux: Qlogic“Failover Mode”

A/P A/P

VMwareMultipathing

Solaris“MPxIO”

A/A/A A/A A/P

A/P A/P A/P

A/P N/A A/P

AIX“SANpath”

A/A/A N/A A/P

HP-UX“PVLinks”

Novell

A/P N/A A/P

A/P A/P A/P

Agenda

CFModes

Multipathing

Host Clustering

Host Clustering & Storage

LUNs need to be made visible to host simultaneously

Some Host Clustering solutions require SCSI reservations to avoid to split brain

Switch/Fabric 1

Host 1

Controller 10b 0d

Controller 2

Controller 1 Active Shelf(s)

Controller 2 Active Shelf(s)

Switch/Fabric 2

0d0b 0a0c

Host 2

Host Clustering for Microsoft

Microsoft Cluster– SnapDrive is integrated to help configuration– WIN2K3 allows single HBA for both boot device &

shared storage– Cannot grow LUN online in cluster

• SnapDrive ability to very quickly grow a LUN minimizes the pain caused by this

Host Clustering for VERITAS

VCS– By default does not us I/O fencing to protect against

split brain– I/O fencing requires SCSI-3 reservations– 7.0.3 will have SCSI-3 reservations that are

compatible with VERITAS– Does not do failover on FC links

Host Clustering for HP-UX

ServiceGuard– 1 to 3 node clusters using SCSI-2 locks as arbitrator

to avoid split brain– Does not do failover in dead FC links

Host Clustering for AIX

HACMP– Uses SCSI-2 locks as arbitrator to avoid split brain

• “setsp –b2” to enable locks with SANpath• SCSI-2 locks to active/active are mutually

exclusive

Fibre Channel SAN Host Support

Host ClusterOS Vendor Multipath File SystemHBA

Native SANpath HACMP

Volume Mgr

QLogic QLogic Novell Clusters NSS

Emulex MPIO MSCS MMC NTFS

Emulex Veritas DMP Veritas VCS Veritas VxVM Veritas VxFS

JFS/2Raw

NativeHP PVLInksVeritas DMP

MC ServiceGuardVeritas VCS

LVMVeritas VxVM

JFS/ HFSRaw

Veritas VxFS

QLogic QLogic Oracle 9i, 10g RACext3ext2

Reiser

QLogic QLogic Oracle 9i, 10g RACext3ext2

Reiser

EmulexQLogic

VMWare MSCSVirtualCenter (VMotion)

VMware VMFS 2.xRaw

Shared Storage

Agenda

CFModes

Multipathing

Host Clustering

Enables Dual Path HA

Key Benefits Full storage hardware

redundancy in HA systems

Prevent cluster failover events due to many storage issues.

Complements CFO for improved HA and resiliency

Key Benefits Full storage hardware

redundancy in HA systems

Prevent cluster failover events due to many storage issues.

Complements CFO for improved HA and resiliency

Loop 1 Loop 2 Loop 3 Loop 4

Protect Against Cable Pulls Or

Breaks

X Protect Against Single HBA Failure

Protect Against Storage Controller

(eg. ESH2) Hot Swap

Switched Back-End

Dual Active Paths for HA Environments– Reduces the number of HA failovers– Improve overall HA performance– Data ONTAP tries to balance load across paths

SyncMirror– SyncMirror requires 100% disk overhead– Proper configuration survives all single failures

Agenda

CFModes

Multipathing

Host Clustering

architecting fibre channel ha solutions

controller 2lunsluns0c

dashed purple

failurenetapp confidential

single targetall luns

hostlunslunssolid blue

single system image

original cfmode standby

aixoriginal cfmode

Documents

architecting estonia

architecting for failure

module 3 architecting & transformation...module 3...

architecting web services

architecting speed

architecting rias

architecting cloud solutions

architecting e learning

architecting for a resilient, cost infrastructure a...

architecting fibre channel ha solutions rick jooss...

architecting windows azure

architecting a robust manufacturing network for the...

sappress architecting edi

end user architecting

architecting javascript code

vmware vcloud architecture toolkit architecting a vmware...

04 architecting navigation

architecting embedded microsystems

architecting happiness preview

architecting the cloud