implementation of dense storage utilizing hdds with ssds and pcie flash accelerator cards

Rick Stehno, Sr. Database and Ceph Storage Performance Architect

Applications and Ecosystem Solutions Development

Implementation of Dense Storage Utilizing HDDs with SSDs and PCIe Flash Accelerator Cards

NY, USAJanuary 2016 1

Seagate Confidential 2

Big Data is Bigger than MORE Data

Demand GROWING at Exponential RATE

Bigger Different $-ConstrainedCreated

Location SHIFTING to the CLOUD

Stored

Capacity

Data Center System Spend

Stable but SLOW growth


Seagate is Evolving to Address Trends

SYSTEMS

DEVICES

ARRAYS

Converged InfrastructureHigh Performance Storage

Hybrid Data Systems

Software-defined Storage

Flash Arrays

NAND PCIeTraditional Enterprise

NAND SASHyperscaleDesktop

Notebook / TabletBranded NAND SATAHybrid

Kinetic

Still Growing

Expanded Offerings

Started with OurCore Products Flash

Controllers


Seagate Enterprise Device PortfolioH

ighe

st P

erfo

rman

ce/$

Highest GB/$Supported by end-to-end silicon capabilities across Flash and HDDs; >1 Billion ARM CPUs shipped

Silicon technology/IP spans Flash and HDD Portfolio across PCIe, SAS and SATA

Archival HDDsCapacity

HDDs

PerformanceHDDs

with TurboBoost™Flash Caching

Nytro® PCIeFlash

Acceleratorswith SandForce®

Technology

SASSSDs

Latency

Density

Thermal

PCIe M.2, 2.5

SAS 2.5

15K/10K 2.5

NL 2.5NL/Value 3.5

Kinetic 3.5Archive 3.5

Perf. 3.5

Nytro PCIeSSDs

Mar

kets

Use-CasesPlatforms

Hyperscale

Traditional-IT

HPC

Mission-Critical/ComputeBusiness-Critical/Near-Line

Archival

JBODsStorage Arrays

Servers

http://www.seagate.com/products/solid-state-flash-storage/flash-controllers/client-flash-controller/


DISK

FS

DISK DISK

OSD

DISK DISK

OSD OSD OSD OSD

FS FS FSFS btrfsxfsext4

MMM

Advanced Storage:• Shingled Magnetic Recording (SMR) Drives

• Excellent density, cost, read performance, and sequential write streaming

• Poor random write performance

• PCIe Flash• Excellent performance, in-server integration

• Poor cost per GB

Object Storage Daemons (OSDs):• 10s to 10000s of OSDs in a cluster

• One per disk or SSD

• Serves stored objects to clients

• Intelligent peers perform replication and recovery tasks

• Support for erasure coding of objects

• Support for strong consistency leveraging write journals

Ceph Deployment


Featuring:• Nytro XP6302 flash accelerator card

• Nytro XD - tuned Write Back caching SW

• 12x 8TB SMR HDDs

Benefits• Accelerate Ceph journals and Metadata

with flash

• By moving this data onto flash, cost-effective SMR drives are well-suited for extremely dense object storage systems reducing storage costs over 40%

• Experienced 11x Ceph Object Storage Performance Gains

Storage Component Deployment in Ceph


Featuring:• Seagate 8TB SMR HDD

Benefits• High density

• Very low cost per/GB

• Read performance

• Sequential write streaming

Weakness• Random write performance

Storage Component Deployment in Ceph

CMR SMR0

50100150200250300350400

RADOS Bench Write Test - 4M

MB

/sec

0200400600RADOS Bench Write Test - 4M

MB

/sec

CMR SMR + Nytro-XD WB Caching

0

5

10

15

20

25

RADOS Bench Write Test - 4K

MB

/s


Ceph Benchmarks – 4k writes

SMR only SMR with SSD SMR Cache Tier

SMR with Nytro XD

0

5

10

15

20

25

30

4K writes - 16 threads

MB/

s

SMR only SMR with SSD SMR Cache Tier

SMR with Nytro XD

0

0.005

0.01

0.015

0.02

0.025

0.03

4K writes - 16 threads

Seco

nds

11x Gain in MB/s 11x drop in Latency


Ceph Benchmarks – 4M writes

SMR only SMR with SSD SMR Cache Tier SMR with Nytro XD

0

100

200

300

400

500

600

700

800

900

4M writes - 16 threads

MB

/s

SMR only SMR with SSD SMR Cache Tier SMR with Nytro XD

00.05

0.10.15

0.20.25

0.30.35

0.40.45

0.5

4M writes - 16 threads

Sec

onds

7x gain in MB/s 6x decrease in Latency


Ceph Storage Costs

SMR + Nytro XD Price per MB/s: Cost of ((Total SMR HDD cost + Nytro PCIe Card) / MB/s for each test)

HDD Price per GB Price MB/s 4k writes Price MB/s 4M writes

CMR 4TB HDD $0.08250 CMR 4TB HDD $589.90 CMR 4TB HDD $11.89

SMR 8TB HDD $0.03250 SMR 8TB HDD + Nytro XD $315.46 SMR 8TB HDD + Nytro XD $11.98

These prices do not include savings from electrical/cooling costs, reducing datacenter floor space, from the reduction of HDD


Scaled-Up Hardware Configuration

Example Ceph configuration using fully Populated Open Rack:

• 18 Winterfell Servers• 33.5 TBs of Flash• 2.2 PBs of SMR Storage

Seagate Confidential 12Seagate Confidential

Thank You!Questions?

Learn how Seagate accelerates storage with one of the broadest SSD and Flash portfolios in the market