©2008 DigiTar, All Rights Reserved

Ditching Fibre Channel & SCSI:

Saying hasta la vista to your vendors and “ooh la la” to open storage.

Pho

to C

redi

t: ht

tp://

ww

w.n

ea.c

om/U

ploa

dedI

mag

es/Im

ageG

alle

ry/o

ld%

20co

mpu

ter%

20im

age.

jpg

©2008 DigiTar, All Rights Reserved

Who am I?• overview of JJWW

• background

• responsibilities

• biases

©2008 DigiTar, All Rights Reserved

• overview of DigiTar

• services

• implications of our services

• tech/storage needs

DigiTar?

©2008 DigiTar, All Rights Reserved

• what is fibre channel?

• is ethernet + TCP/IP different than fibre channel?

• storage networking: just a mechanism for SCSI access over a network

A long time ago in a galaxy far, far away...

Pho

to C

redi

t: ht

tp://

theb

adpl

us.ty

pepa

d.co

m/p

hoto

s/un

cate

goriz

ed/d

eath

star

_2.jp

g

Fibre

Channel

©2008 DigiTar, All Rights Reserved

More flexible than Gumby on a hot plate...

• why use a commodity storage stack?

• block or file access when it suits you

• same commodity troubleshooting tools

• inspection at any OSI layer

• SAN access on a whim

Pho

to C

redi

t: ht

tp://

wal

lpap

ers.

jurk

o.ne

t/wid

escr

een/

uplo

ads/

wal

lpap

er_1

439.

jpg

©2008 DigiTar, All Rights Reserved

Ethernet Everywhere?• performance: FC vs. Ethernet

• latency

• maximum segment sizes

• lossless transmission

• loop elimination

ethernet is rarely the wrong choice

latency (μs)

©2008 DigiTar, All Rights Reserved

We don’t need no stinkin’ revolutions...

• high performance = high RPMs!

• IOPs not RPMs

• 37% more IOPs with SATA

nope.

Pho

to C

redi

t: h

ttp://

en.w

ikip

edia

.org

/wik

i/Im

age:

Fran

cisc

o_Vi

lla.g

if

©2008 DigiTar, All Rights Reserved

“Lies, damn lies, and enterprise drives.”

• greater reliability costs more right?

Pho

to C

redi

t: ht

tp://

ww

w.p

cmag

.com

/imag

e_po

pup/

0,18

71,ii

d=12

4480

,00.

asp

• New data on “enterprise” drive reliability

• “Failure Trends in a Large Disk Drive Population”Google - http://labs.google.com/papers/disk_failures.pdf

• “Disk Failures in the Real World: What Does an MTTF of 1,000,000 Hours Mean to You?”Carnegie Mellon University http://www.usenix.org/events/fast07/tech/schroeder/schroeder.pdf

• wrong.you don’t necessarily get what you pay for:

©2008 DigiTar, All Rights Reserved

Sharing the (commodity) love...

• vendors love commodity hardware

NetApp 6070

4x 2.6GHz Opteron 852

32GB RAM

Disk trays sold separately

Cost: $151,464

SunFire X4500

2x 2.8GHz Dual-Core Opteron 290

16GB RAM

48x 500GB 7200 RPM Disks(24TB)

Cost: $34,995

• consider cheaper, modular arrays for NVRAM

©2008 DigiTar, All Rights Reserved

Redundant controllers, hitless upgrades and other HA myths...

• high availability truths

• redundant controllers fail

• hitless upgrades are miracles...and happen just about as often

• rock-solid availability

• redundant arrays (not controllers)

• application level failover

Pho

to C

redi

t: ht

tp://

imag

es.b

uyco

stum

es.c

om/m

gen/

mer

chan

dise

r/110

25.jp

g

©2008 DigiTar, All Rights Reserved

The missing link.• cheap disk, commodity hardware, gig ethernet...

we’re missing something...

• a storage OS: Linux or OpenSolaris

• storage on Linux can be hazardous to your health

• OpenSolaris

• rock-solid storage subsystem

• ZFS

• iSCSI, NFSv3/v4, CIFS (FC target soon!)

• NTFS ACLs (dual-directory support)

©2008 DigiTar, All Rights Reserved

ZFS-in-a-Slide• copy-on-write

• block checksums

• transactional...no repairs!

• volume manager and filesystem as one brain

• write bundling!

• dynamic striping

©2008 DigiTar, All Rights Reserved

Open Storage: The DigiTar Way

• how does DigiTar use ZFS, Solaris and commodity hardware today?

• how are the benefits realized in DigiTar’s infrastructure?

• cost analysis/savings DigiTar has experienced as a result

+

©2008 DigiTar, All Rights Reserved

Making all the pieces fit...

• enhance your infrastructure uniquely

• favor iSCSI & NAS for new deployments

• utilize SATA

• OpenSolaris: vendor licensing judo

Pho

to C

redi

t: ht

tp://

ww

w.e

xhib

itoro

nlin

e.co

m/e

xhib

itorm

agaz

ine/

imag

es/o

ctob

er_2

006/

ever

ythi

ngm

atte

rs/je

nga_

med

ium

.jpg

©2008 DigiTar, All Rights Reserved

Where next?

• fibre channel & the dodo

• multi-core economics

• open source storage

• distributed storage

Pho

to C

redi

t: ht

tp://

ivor

ybill

.com

/ca/

imag

es/2

d.jp

g

©2008 DigiTar, All Rights Reserved

Questions?

‣ This talk: http://blogs.digitar.com/jjww/?itemid=63

©2008 DigiTar, All Rights Reserved

Thank you!

‣ This talk: http://blogs.digitar.com/jjww/?itemid=63