san design considerations for blade environments bjørn r ...case 1: no ivr. 239 domain ids per vsan...

© 2006 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 1Cisco Confidential Information

SAN design considerations for Blade environments

Bjørn R. Martinussen


Typical Customer Dynamics

Server Admin

LAN Admin

SAN Admin Drivers of Pressure

• Domain ID scalability• Cost• Interoperability

Drivers of Pressure• Simpler Provisioning• Autonomy - Replacement of Servers•High Availability

Drivers of Pressure•Security•Time to provision•Cost•Consistent policies & configurations

Cisco Solution (SAN)•NPV mode for switch

•NPIV for hosts

•Cisco SAN-OS & DCOS

•Uplink trunking (future)

Cisco Solution (LAN)•ACLs at access layer

•Cisco IOS & DCOS

•QoS config at access layer

Cisco Solution (Server)•FlexAttach


VC Value Proposition

Simplicity of Management/Operations/NetworkingNo switches to manage – no STP or Domain Id proliferationServer Admin can provision and manage – similar look and feel as other Proliant mgmt tools More efficient server bring upCable consolidation

Server Admin AutonomyServer adds, moves, changes with no impact to the SAN or LAN

Maintain current operational boundariesNetwork boundary moved outside the blade chassis Blade chassis and associated I/O devices owned and managed by Server Admin


VSAN Support – Up to 16Port-ChannelsFSPF load balancingNon-disruptive firmware upgradeSupported Port Types – F, FL, E, TE, SD & Auto

Up to 16 Buffer Credits per portSecurity

TACACS+, RADIUS, SNMP v3, SSH, SFTP, VSAN RBACAdvance Diagnostic Tools

Integrated Call HomeSPAN, FC Ping, FC Trace Route

Management via Fabric Manager, Device Manager, CLIDynamic load-balancing of FDSC logins.

SANOS Features

Virtual Connect acts as a mux to pass through HBA features, no “switch” intelligence


Blade roadmap to an Intelligent HBA aggregator

MDS blade

Solve thedomain_IDproliferation

Server IdentityVirtualization(for server mgmt)

Intelligentswitch

Intelligent HBA(HBA model+switching benefits)

3.1shipping

3.2(1)Q4CY07

3.3(1)Q1CY08

4.x 2HCY08

Brocade AG

HP VC

MDS blade w/ NPV

MDS blade w/ Flex Attach

HP VC


Differences Between NPIV and NPVNPIV (N-Port ID Virtualization)

– Functionality geared towards server’s host bus adapters (HBA)

– NPIV provides a means to assign multiple Server Logins to a single interface

– The use of different virtual pWWN (port World Wide Name) allows access control (zoning) and port security to be implemented at the application level

– Usage applies to applications such as VMWare, MS Virtual Server and Linux Xen

NPV (N-Port Virtualizer)– Functionality geared towards certain types Fibre Channel (FC) switches

– NPV provides the FC switch’s connections (uplink) to act as server connections – instead of acting like an Inter-Switch Link (ISL)

– Utilizes NPIV type functionality to allow multiple server logins from other switch ports (non-uplink) to use uplink ports

– Usage applies to certain FC switches (ex: Blade Switches)


N-Port Virtualizer (NPV)SANOS 3.2(1)

F-Port

MDS Switch orNon-Cisco Switch

NPIV enabled

Server 1

Server 2

Server N

FC BladeSwitch 1…

…

FC BladeSwitch 2…

N-Ports

Blade Chassis

F-Ports

…

…

NP ports

SAN Fabric

F-Ports

F-Port

Eliminates Domain ID for MDS FC switch in Blade chassis – HBA aggregator.Need to configure the same VSAN between NP ports and core F-ports (unlike other HBA-like switches). In SANOS 3.2 one VSAN per uplink. When F-trunking will be available the limitation will go away.

Server FLOGI/FDISC to first available NP link and all data flow for that server goes through the same NP (uplink). FLOG/FDISC load-balanced

VC roadmap is unlikely to support VSAN allocation (per blade / per bay)


Servers

MDS 9124

MDS 9500

NPV Enabled Environment

NPIV enable

NPV Edge Switches– Need to enable switch in “NPV” mode

– Changing To/From NPV mode

• Disruptive: switch reboots

• Configuration is not kept

– Supports only F, SD (SPAN Destination) and NP modes

– Supports 16 VSANs

– Local switching is NOT Supported

• Switching will be done at the Core

NPV Core Switches– Must enable NPIV feature

– Supports up to 100 NPV Edge Switches


NPV Supported SwitchesNPV Edge Switches

– MDS 9124 & MDS 9134

– IBM and HP Blade Switches

NPV Core Switches– MDS 9500 Family of Directors

– MDS 9216A, MDS 9216i & MDS 9222i

– 3rd Party Switches

• Needs to support NPIV

• Needs Testing/Qualification


NPV Features Support

Supported Features in NPV mode– No needed license for NPV

– In Service Software Upgrade

– Consistent with management and troubleshooting tools of the MDS switches (e.g. Fabric Manager, L2 ping, L2 traceroute etc)

–Role Base Access Control to maintain a clean demarcation between SAN and Server admin.

–VSAN support

–Supports nested NPIV (in virtual server applications)


Cisco Domain ID scalability in perspectiveCase 1: no IVR

239 domain IDs per VSAN

Case 2: IVR version 2 (Used by customers mostly)239 per VSAN – number of VSANs (support NAT: all domain IDs of a VSAN represented by one domain ID in another VSAN)

Case 3: IVR version 1 239 no NAT support, hence numbering space is shared.(If a VSAN is not routing we can use all 239 domains in that VSANs)

Other vendors are limited to 239 Domain IDs or less depending on practical limitations of their fabric.

Customers with small SAN & blade deployments are unlikely to be affected by th DiD l i L t i k t i i i th # f DiD’


NPV Traffic Engineering + Dynamic Load BalancingSANOS 3.3(1)

Traffic Engineering– Ability to associate a server to a particular uplink

Dynamic Load Balancing–Dynamic rebalancing of server to NP ports (useful in failback situations)

FC1/2

PWW

N 1

Server 1

FC1/12 FC1/14

FC1/1 FC1/13

PWW

N 1

Server 11

PWW

N 1

Server 12

PWW

N 1

Server 22

FC1/24

Server 1 Server 2Server 3 Server 4

Server 21 Server 22

VC requires “pinning” of server to uplink and limits load balancing / failover


Nested NPIV Load-Balancing

PWW

N 1

PWW

N 2

PWW

N 3

FC1/2

VMWare 1

PWW

N 1

PWW

N 2

PWW

N 3

PWW

N 1

PWW

N 2

PWW

N 3

FC1/10 FC1/20

FC1/1 FC1/13

VMWare 2 VMWare 3

VMWare1: PWWN1 VMWare2: PWWN1

VMWare1: PWWN2 VMWare2: PWWN2VMWare2: PWWN3VMWare1: PWWN3

VMWare3: PWWN1

VMWare3: PWWN3VMWare3: PWWN2

• NPIV Server FLOGI/FDISC to first available NP link and all data flow goes through the same NP (uplink)

• FLOGI/FDISC on all available NP links is load-balance via Round-Robin

• All Nested NPIV logins will flow through the same NP port as the end device is using


NPV with SDV• SDV enabled at the Core

• Creation of virtual PWWN (vPWWN) linking to PWWN of NPV end device

• In case of failed HBA, manual re-link vPWWN to new PWWN

• No zoning or storage masking necessary

FC1/1

PW

WN

1

Server 1

vPWWN1 PWWN1

FC1/1

PW

WN

2

Server 1

vPWWN1 PWWN2

Before After


Flex Attach for Server ReplacementSANOS 3.3(1)FlexAttach Port (virtual PWWN)

– Creation of virtual PWWN on NPV switch F-port

– Zone vPWWN to storage

– LUN masking is done on vPWWN

– Can swap Blade Server or replace physical HBA (equivalent to HP VC)

• No need for zoning modification

• No LUN masking change required

– vPWWNs are unique in the fabric when we choose the automatic vPWWN assignment based on the fabric. No need to keep track of vPWWNs.

FC1/1

PWW

N 1

Server 1

vPWWN1 FC1/1

PWW

N 2

Server 1

vPWWN1

Before After

Delivers server mobility and blade HA failover capability.


NPV: FlexAttach Port Details

port WWN of D1 = pwwn1fc1/1

End device D1 (server or target)

NPV

N

F

pwwn rewrite rules

NPV-Core Switch (MDS or 3rd party switch with NPIV support)

port WWN of D1= vpwwn1

End device D1 is known by vpwwn1 in the SAN

NP

F P1

P2

P3

P4

Frames needing pWWN rewrite_______________Non-ELS FramesFLOGI FLOGI ACC

ELS FramesPLOGIPLOGI ACCRNID RNID ACCADISCADISC ACC

LOGO


NPV: Server MigrationFlexAttach Port

– Allows to migrate server from one NPV switch port to another port on same NPV switch or another NPV switch

– vPWWN migrates to new port

– Existing interface will receive new vPWWN

– Advantage over HP VC because can span across Blade Chassis

FC1/1

PW

WN

1

Server 1

vPWWN1 FC1/7

PW

WN

1

Server 1

vPWWN1


Positioning of Different Models

Switch Model NPV (HBA Model)

Deploy when

Mid-Size blade switch deployments (< 50 switches )

Domain ID is not an issue

No multi-vendor switch interoperability issues

Deploy when ….

Large scale deployment… > 50 switches

Domain ID is an issue

Multi-vendor environment with switch interoperability issues

Now Now – req SanOS 3.2

san design considerations for blade environments bjørn r ...case 1: no ivr. 239 domain ids per vsan...

Documents