san overview

7/31/2019 San Overview

http://slidepdf.com/reader/full/san-overview 1/61

SAN TECHNOLOGYSAN TECHNOLOGY



WHERE TO GO FOR MORE INFORMATIONWHERE TO GO FOR MORE INFORMATION

•SearchStorage.com

•InfoStore.com•SNIA.org

•Fibrechannel.com•IEEE.org

•All Vendor Web Sites



FIBER CHANNEL INTERCONNECTFIBER CHANNEL INTERCONNECT

FC Optical Cabling

50 Micron50 Micron 9 Micron 62.5

Micron

Type Short wave Long wave ExistingMultimode Single Mode Cabling

Distance 500m 10km 200m

Bandwidth 100MB/sec 100MB/sec 100MB/sec

Short Wave/Multi-mode

9 Micron

Long Wave/Single-mode

62.5 Micron Two Protocol Types:FC-SW (Switched)FC-AL (Loop)Existing Cabling



FC HUBS AND SWITCHES

• Hubs– 7 port and 12 port models

– Much lower cost than switches

– Used in smaller SANs

• Switches– 8 and 16 port models

– Better isolation than hubs– Switches are more $ than hubs.

– Scalable to large SANS

– Cascading, Zoning, Quickloop

– Higher MTBF, easier repair

• GBICs (GigaBaud InterfaceCards)

– Short and long wave variants– LW to 100Km

– SW to 500m



SILKWORM 2800 FRONT PANEL

LED for GBI C s ta t usGBI C

Et h e rne t connec t o r

Pow er connec t o r

On / Of f sw i t chPow er i nd i ca t o r

D isp lay and con t r o l s



QLOGIC/ANCORQLOGIC/ANCOR 6464--PORT DIRECTORPORT DIRECTOR



FC IMPLEMENTATION FC FC IMPLEMENTATION IMPLEMENTATION

Hub/SwitchHub/Switch

Hub/Switch may look the same!• Electrical Isolation (hot connect)

• Radial Topology

• Hot replacement of GBICs

Hubs implement FC-AL• Logically equivalent to a

single path loop

• Bandwidth is 100 MB/Sec

• Nodes see all data

Switches implement FC Fabric

• Switches act like network routers• Multiple communication paths

• Bandwidth is aggregate of paths

• Switches provide logical isolationFC Switches providenon-blocking

Dynamic Paths

LOOP



• FIBRE CHANNEL TOPOLOGIESPoint-to-Point

Public

Loop

PublicLoop

PublicLoop

ServerSCSI

to FCBridge

JBODServer RAID

Arbitrated Loop

Switched Fabric

WEB TOOLS



WEB TOOLS:

SWITCH MANAGEMENT GUIFabricView

SwitchView

TelnetInterface

PortalDetail

View

Brocade ® Web Tools

FabricTopologyView

PerformanceView

HELPInstructionspop-up whenneeded

AdministrativeInterface

• Platformindependent

management• JAVA Based

• Scalable acrossthe entire fabric

• In band data fromswitch to switch

• Detailed data &

management• Statistics• Zoning• Name Services



SIMPLE NAME SERVER

• Standards based network

address assignment

–Allows non-intrusive discovery of devices when logging into thefabric vs. disruptive LIP protocol

on loop based systems

• Registers new devices in the

fabric

• Automatic legacy device

support



SCALABILITY - CASCADING

– Any port to any port, nospecific assignment

needed – Dynamic recovery

– Dynamically configured

routes – Multiple Inter-Switch

Links

– Support up to 239switches in addressingscheme



QUICKLOOP MODE

When a switch operates inQuickLoop mode, theswitch serves as a

concentrator, similar to ahub, offering throughputperformance on each

looplet of 100 MB/s.

QuickLoop



• SWITCH PORT TYPES – Node Port (N_Port)

– Fabric Port (F_Port)• Supports N_Port devices (public, full

address capability)

– Fabric Loop Port (FL_Port)• Supports NL_Port arbitrated loop

devices (public, full addresscapability)Note:only 1 FL per loopallowed.

– Switch-to-Switch Port (E_Port)• A port that connects to other vendor

switches (switch-to-switch)

– Trunk Port (T_Port)• A port that is used to interconnect

two chassis in a multi stagetopology (cascade, mesh, or multi-

stage)

FL-Port

T-Port, E-Port

F-Port

C a s c a d e

Note: N_Port and NL_Port are basic fibrechannel node types



•WORLD-WIDE NAME – A globally unique 64-bit identifier for eachdevice assigned by the manufacturer that

is registered with IEEE – Used for Device ID in soft zoning and LUN

masking using Security

– Same concept as Network Card MACaddress



• Protocol Overview

– Standards-based networkingarchitecture that is defined as multi-layered functional levels

FC-PH

IPIIPI SCSISCSI HIPPIHIPPI 802.2802.2 IPIP ATMATMSBCCSSBCCS

Channels Networks

Common ServicesCommon Services

Data Delivery (Frames/Sequences/Exchanges)Data Delivery (Frames/Sequences/Exchanges)

Ordered Sets / 8B/10B Byte EncodingOrdered Sets / 8B/10B Byte Encoding

266 Mbit/s266 Mbit/s 541 Mbit/s541 Mbit/s 1.062 Gbit/s1.062 Gbit/s 2.125 Gbit/s2.125 Gbit/s Higher RatesHigher Rates

(Future)

Media: Optical - Laser, LEDCopper - Coax, Twisted Pair

FC-4

FC-3

FC-2

FC-1

FC-0

ULP

Intermediate LevelIntermediate LevelFC-AL



• ANSI STANDARDS – Fibre channel protocol

FC-PH Rev 4.3

FC-PH-2

FC-PH-3

FC-AL Rev 4.5

FC-AL-2 Rev 7.0 (draft)

FC-FLAFC-GS-2

See www.fibrechannel.com for detailed descriptions

FC-GS-3

FC-FG

FC-PLDA

FC-Tape

FC-VI

FC-SW-2 (when

complete)Fibre Channel

Element MIB



• CLASSES OF SERVICE

• Defines how frames are routed within a SAN – Class 1 - Dedicated connection

• Must explicitly setup and break down connections (setup at sequenceboundaries)

• Circuit created between initiator and target (route allocated between twoN-ports)

• Relies on acknowledgements (ACK frame response)• Error detection and handling (F_RJT, F_BSY)• Guarantees maximum bandwidth• Deterministic latency• End-to-End and Buffer-to-Buffer (BB) flow control (ACK, R_RDY)• Ideal for high-bandwidth, real-time applications (streaming tape or

video)

– Class 2 - Connectionless with acknowledgements

• Relies on acknowledgements (ACK frame response)• No guaranteed bandwidth (frames routed if buffers available)• Non-deterministic latency (available bandwidth and variable path

dependencies)• Fabric routing• End-to-End and Buffer-to-Buffer (BB) flow control (ACK, R_RDY)• Ideal for data transfers to and from a shared mass-storage system

(bursty traffic, less timing critical)



FIBRE CHANNEL FRAMES

SOF Frame header Payload/Data CRC EOF Idles

4 24 4 4 240-2112 (Bytes)

Parameter

R_CTL

reserved

TYPE

SEQ_ID

OX_ID

D_ID

S_ID

F_CTL

DF_CTL SEQ_CNT

RX_ID

Frame Header Fields (bits):

R_CTL: Routing Control (8)D_ID: Destination Identifier (24)

S_ID: Source Identifier (24)

TYPE: Data Structure Type (8)

F_CTL: Frame Control (24)

SEQ_ID: Sequence Identifier (8)

DF_CTL: Optional headers in data field (8)SEQ_CNT: Sequence Count (16)

OX_ID: Originator Exchange ID(16)

RX_ID: Responder Exchange ID(16)

Parameter: Frame Type Dependent (32)

NOTE: The maximum size frame size is 2148 bytes, with 2112 bytes of payload



• FIBRE CHANNEL IDENTIFIER

– Fibre channel uses 24 bit addresses to routeframes

• Located in Frame Header (Destination ID (DID)

Source ID (SID))• Public devices use all 24 bits

• Private devices use only the lower 8 bits, upper bits

are zero

X X X X X X X X X X X X X X X X X X X X X X X XFabric ID Chassis # Port # AL_PA (Loop Address)

F_Port ID

FL_Port ID

Domain Area Port



• FIBRE CHANNEL COMMUNICATIONS – Frame

• A data unit comprising a Start-Of-Frame (SOF) delimiter,frame header, data payload, CRC and End-Of-Frame (EOF)

delimiter – Sequence

• A unidirectional series of one or more frames flowing fromthe source to destination

– Exchange

• A series of one or more non-concurrent sequences toaccomplish a specific purpose (data transfer or fabric

login)



• Frame Organization

SID=AAADID=BBB OXID=XX

SEQ_ID=YY

SEQ_CNT=ZZ


SEQ_ID=01SEQ_CNT=03

SID=AAADID=BBB OXID=XX SEQ_ID=01SEQ_CNT=06



SEQ_ID=01SEQ_CNT=02



SEQ_ID=01SEQ_CNT=01



Frame

Sequence

Exchange



FRAME CREDIT

Server(N-port)

JBODArbitrated

Loop

RAID(N-Port)

End-to-End Credit

(N-port-to-N-port)

Frame out, (-) credit

ACK back, (+) credit

Frame Out, (-) credit

R_RDY, (+) credit

Buffer-to-Buffer (BB) Credit (link-to-link)

SAN ARCHITECTURE



SAN ARCHITECTURE

Storage Nodes

Server Clusters

FibreChannel

RAID

Tape Drives

JBOD

LAN

End Users

SAN

SAN



SAN: STORAGE AREA NETWORK DEFINITION

A SAN is a collection of hosts,A SAN is a collection of hosts,

plumbing, and storage…simplistic, butplumbing, and storage…simplistic, buttruetrue

But, there is much more to aBut, there is much more to a

SAN….much much more by way ofSAN….much much more by way of

benefitsbenefits



NASNAS SANSAN

NFS protocolNFS protocol SCSI protocolSCSI protocol

Globally over a netGlobally over a net Over a storageOver a storage

interconnect that looks likeinterconnect that looks likea networka network

Slower NFS speed Slower NFS speed Faster SAN SCSI speed Faster SAN SCSI speed

NFS overhead greater NFS overhead greater Lower SAN SCSI overhead Lower SAN SCSI overhead

Inexpensive, easyInexpensive, easy Prices dropping,Prices dropping,

complicatedcomplicatedNAS established todayNAS established today SAN is making headwaySAN is making headway

Shared storageShared storage Shared storageShared storage

Shared dataShared data Promises shared dataPromises shared data

STORAGE AREA NETWORK BENEFITS



STORAGE AREA NETWORK BENEFITS

•Backup and Restore•Storage and Server Consolidation

•Easy Storage Management/Analysis•Fewer Operators for More Storage•Automation/Dynamic Scalability•Flexible Distances/Centralized Control•High Availability Clustering

•Disaster Tolerance/Replication•Cross Platform Virtualization



PER PORT PRICES ARE DROPPING

SANS OF TODAY



SANS OF TODAY

Utility

Integrated

Fabric

SAN100 nodes

Loop Switches/

Fabrics

Large

SANs1,000 nodes

Fabrics/Backbones

Virtualization

Pool

Very Large

SANs

10,000+ nodes

Cluster

External

Arrays

10 nodesHubs/Loop Switches/

Small Fabrics

TodayHost embedded

Storage- Hubs

SANS OF TODAY



SANS OF TODAY

• Plumbing

– Fibre Channel Based - 1Gb

• FCP SCSI for storage access

• FC-IP for in-band management

– Ethernet for most management (out of band)

– Hubs for ‘low-end’ SANs

• Typically a few hosts and storage on a‘shared’ bus of FC-AL.

– Bounded performance

– Out of band management

– 500M separation

SANS OF TODAY



SANS OF TODAY:

• Islands of Storage with Fibre Channelplumbing to hosts

– Mostly Homogenous Storage & Plumbing• Fault Tolerant and/or non-stop with meshed plumbing

(mult paths) or dual plumbing

– Scale is 50-100 ports per Island – Most Businesses have not been able to embrace

the full power of a SAN

• Partly because of business practice of separate sub-businesses within business and separate accountability

• Partly because SANs have not scaled or beenmanageable to the degree necessary for large SANs

SANs of Today -



yFC-AL High Availability

Storage

Array

Storage

Array

Host Host Host

Loop 1

FC- AL FC- AL

Loop 1

SANS OF TODAY



SANS OF TODAY

• SAN distance – Loops can go up to about 10KM but with

performance degradation

– Switches can extend upwards to 120KM withadequate buffering before performance wanes

– LW GBICs now support up to 70KM

– FC extenders can go up to 120KM – DWDM- Dense Wave Division Multiplexing for

Fibre bundling

– FC-ATM tunneling for WAN connectivity andgeographic separation of SANplumbing..extended fabrics

SANS OF TODAY: SAN SERVICES



SANS OF TODAY: SAN SERVICES

• SANs provide for Fabric attachstorage

– Replicated Storage (across the SAN anddistance)

• For point in time copies

• For Disaster Tolerance

• For Copy/Update Services

• For remote Backup – Arrays and/or raw devices. Arrays

dominate

SANS OF TOMORROW



SANS OF TOMORROW…

• No Limits to scale

• SWAN Fabrics will dominate

– Keys to success will be effective management ofe-ports between switches for simple scaling,service, and performance monitoring

• Fabrics will become Heterogenous withE-port standardization (FC-SW-2)

– Simplifies business growth and competition

between vendors, but interoperates

SANS OF TOMORROW



SANS OF TOMORROW

• 2Gb, full duplex Fibre channel physicalconnections emerge. Fully

interoperable with existingSANs…(switches do speed matching!)

– Speed negotiation for mixedsource/destination

• 10Gb connections emerge in 2002/2003

for trunking and potential interoperationwith telco 10Gb links

SANS OF TOMORROW



SANS OF TOMORROW

• FC over IP and for WAN tunnelingemerges for full complement of ATM

and IP telco line usage forgeographic SANs.

• Smaller form factor GBICs enablinglower cost, higher density plumbing.Multi-Color (phase-shifting)Laserslower costs.

SANS OF TOMORROW



SANS OF TOMORROW

• Larger switches will appear

• Scaling of switches will become a

priority for businesses as benefits ofrecent SAN management server work

within SNIA solidifies into products thatfacilitate larger SANs.

• Small Islands will disappear with integrated

capacity management for businesses to tracktheir per business/group storage costs.

CONNECTING ISLANDS TOGETHERCONNECTING ISLANDS TOGETHER



CONNECTING ISLANDS TOGETHERCONNECTING ISLANDS TOGETHER

SAN IMPLEMENTATION EXAMPLES



SAN IMPLEMENTATION EXAMPLES

SAN IMPLEMENTATION EXAMPLE



BACKUP- THE KILLER APP FOR SAN



SAN MANAGEMENT CONSIDERATIONS



PERFORMANCE CONSIDERATIONS



TOPOLOGY CONSIDERATIONS

san overview

Documents