ethernet san - tvar solutions · pdf fileredefining storage economics ethernet san comparison...

Redefining Storage Economics

Ethernet SANComparison of FC (and FCoE), iSCSI, and AoE

Robert Przykucki, Director of Product Management

November 2010


Unique Market Window: Ethernet SAN

40%IT Budget

Unsustainable storage costsin uncertain economy

Legacy SAN is a poor fit withCloud and Virtualization

Storage growth is unstoppableEthernet always wins


Storage Challenges

Legacy SAN 1:1 Mainframe to storage

Today’s SAN Many:1 Servers to storageMany:1 Virtual to physical servers


Ethernet SAN Bare metal performance No controller bottleneck Eliminates complex topologies No more complex multipathing Ethernet simplicity Lightweight protocol wins!

Scale-out Ethernet SAN


Evolution of SAN

Protocol Comparison

Management Comparison

Q&A

Agenda


Complexity

Price-Performance

FICON/ESCON

Fibre Channel

FCoE

Ethernet

iSCSI

The drive to Ethernet


Protocol Stack ComparisonH

eavy

Wei

ght

Pro

toco

l

Block

Session

Transport

Network

Data Link

FibreChannel iSCSI

TCP

FCoE

FCP iSCSI FCP

AoE

SCSI SCSI SCSI ATA

IP FCoE AoE

FC Ethernet DCB Ethernet


Protocol Stack Comparison

FibreChannel iSCSI FCoE AoE

Block SCSI SCSI SCSI ATA

Hea

vy W

eigh

tP

roto

col

Session

Transport

Network

Data Link

FCP

FC

iSCSI

TCP

IP

Ethernet

FCP

FCoE

DCB

AoE

Ethernet

MPIO Multi-Path DriverRequired for FC, iSCSI and FCoE


iSCSI data path

iSCSI: Connection-based, serial delivery

iSCSI 64K IO Transfer CRCSegment Segment Segment Segment Segment Segment Segment Segment Segment

64K IO Transfer

TCP Data (IO Segment) CS

IP Data

Ethernet Data (Jumbo) CRC

Frame #1


IP Data


Frame #2


IP Data


Frame #3


IP Data


Frame #4


IP Data


Frame #5


IP Data


Frame #6


IP Data


Frame #7


IP Data


Frame #8

Initiator(IO is bound to

iSCSI connections)

Port 0

Port 1

Port 2

Port 3

Port 0

Port 1

Port 2

Port 3

Array

Path0

Path1

Path2

Path3

Verify CRC, Remove Ethernet Layer

Remove IP Header

TCP: Re-assemble Byte Segments

iSCSI: Verify IO Integrity

64K IO: Send to disk


IP Data


Frame #9

Frame #1 Frame #2 Frame #3 Frame #4 Frame #5 Frame #6 Frame #7 Frame #8 Frame #9

iSCSI Verify

64K IO Transfer

Disk

ACK

ACKACK

ACK


Port 0

Port 1

Port 2

Port 3

AoE data path

AoE: Connectionless, parallel delivery

AoE Initiator Array

AoE Mapping

64K IO Transfer1 2 3 4 5 6 7 8

Ethernet AoE Data Segment CRC

Port 0

Port 1

Port 2

Port 3

Path0

Path1

Path2

Path3

Write 1A

Write 1B

Write 1C

Write 1D

Write 2A

Write 2B

Write 2C

Write 2D

Write ACK

Write ACK

Disk

ACK IOAoE 8K block (data) x4


Protocol Summary

iSCSI/FC/FCoE AoE

Topology Connection Connectionless

Transport IO Session/Serial delivery

IO TCP scatter/gather

Not required

Disk IO, Ethernet Frames

Multi-Path Multipath Driver

Manual Setup

Not required

Automatic

Dropped Data

Recovery

iSCSI: TCP with Retransmit

FC: Link Flow Control, SCSI timeout

AoE Correlation Timeout

AoE Retransmit

Out of Order

Data

iSCSI: TCP scatter/gather

FC: Prevented via Link Level Flow

Control

In order arrival not

mandated

Overhead Fibre Channel provides 2-3x higher performance than iSCSI

AoE is the most direct and fastest SAN protocol


Management Comparison


Provisioning Fibre Channel

60 Minutes

60 Minutes

60 Minutes

30 Minutes

210 Minutes

FC Stack


Provisioning Fibre Channel

FC Stack


Provisioning AoE

AoE Stack

5 Minutes

<1 Minute

21 Minutes

15 Minutes


“ESG Lab has tested nearly every major storage system in the industry, and we found the ease of implementation and management of Coraid EtherDrive storage shockingly simple compared to most Fibre Channel and iSCSI systems. The simplicity of the system is brilliant, and the price-performance is staggering. It's fair to say we were enormously impressed.”

- Tony Palmer, Senior Engineer at ESG Lab

ESG Lab Validation

• Higher performance than Fibre Channel at 20% the cost

• Performance: 1200+ MB/sec per shelf• ESG was able to provision EtherDrive in

less than 2 minutes from power on


Attendee Questions


What is FCoE?

Host• Requires CNA• Additional Configuration• Reduces cabling

Network• Requires TOR• Requires DCB, not Ethernet• Extra network protocols

Storage• Does not replace FC• Complexities of FC remain• Still a separate network• FCoE is a bridge technology


Cabling from your server is the biggest problem in your datacenter

You have lots invested in FC

FCoE or AoE?

Use FCoE if?

Things to Consider FCoE requires a CNA adapter Requires ToR switch to bridge FC

from FCoE to FC storage island Still requires FC to deliver data to

storage Requires lossless datacenter

Ethernet

Price/Performance and simplicity are high priority

Moving to virtualization and cloud architectures in datacenter

Standardizing on Ethernet

Use AoE if?

Things to Consider No multi-path software Requires jumbo frames (Up to 9K) Use standard Ethernet Works seamlessly with VMware


Q&A

Thank youSend questions/comments to:

[email protected]

Join us for our next webinar:Coraid/VMware: The Secret to Virtualizing Mission Critical

ApplicationsSign up at www.coraid.com

mailto:[email protected]


Extra Slides


ESG Review of Ethernet SAN


How does FCoE compare to AoE?

FCoE StackMulti-Path

Software


Where in the stack is AoE relative to iSCSI?

Protocol Use Case

iSCSI Maps the SCSI protocol onto TCP

Assumes that initiators are talking over the “Internet”

Assumes a lossless link to transport data

Must append a CRC to detect bit errors at destination

TCP First conceived by Ham Radio Operators to transfer data

Goal: Maximize throughput on high latency links

“Creeps up” onto the right throughput

Many standards RFCs describe different TCP implementations

Not designed for low latency environments (storage)

Does not detect bit errors

Guarantees in order byte delivery of data

IP Designed to interconnect Ethernet Networks… “Internetwork”

Very sophisticated automated topology optimization

Hundreds of RFC’s

AoE Maps ATA disk commands onto Ethernet

Assumes multiple paths between initiator and target

Designed to operate on a datagram best effort protocol

Ethernet Very efficient connectionless datagram delivery service

Detects bit errors from source to destination


Fibre Channel iSCSI AoE

Wedge Driver

(Multi-Path)

Not Required

Host Bus Adapter

Switch Ethernet Ethernet

Storage

Why is Coraid easier to manage?

MPIOPower

PathHDLM

SAN

Path

Veritas

DMP


FCoE Summary

Host• Requires CNA• Additional Configuration• Reduces cabling

Network• Requires TOR• Requires DCB, not Ethernet• Extra network protocols

Storage• Does not replace FC• Complexities of FC remain• Still a separate network• FCoE is a bridge technology


FC Ports

FC traffic

LAN

FC traffic

IP traffic

DCB Ports

CNAs

Access Layer Servers

DCB

FC SAN

FCoE Summary

RequiresThese

…AndThese

To SustainThis?


View 1000-user Block on EtherDrive

SRX3500 x457.6TB

10k RPM SATA

View / ESX

Cluster

x2

VMware

Management

View Connection

Manager x2

vCenter x1

Users

512

vDTs512

vDTs

SRX420072TB

User Data Storage


Is iSCSI really slower than FC?

0

50,000

100,000

150,000

200,000

250,000

300,000

350,000

400,000

450,000

Brocade 1020 Emulex OCe10102-F Qlogic QLE8142

FCoE

iSCSI

0

100,000

200,000

300,000

400,000

500,000

600,000

700,000

Brocade 1020 Emulex OCe10102-F Qlogic QLE8142

FCoE

iSCSI

FC is up to 3x faster than iSCSI TCP offload does not help Storage arrays don’t use TCP offload engines. How do they

cope with the additional iSCSI/TCP overhead?

See ITBrand Pulse Test Report: Server I/O with Real World ApplicationsCompares vendor CNA IO performance with a RAM storage device, thus exposing CNA bottlenecks

Oracle 8K Block Exchange 4K Block

74%

drop

ethernet san - tvar solutions · pdf fileredefining storage economics ethernet san comparison...

Documents