walter graf ceph appliance: eternus cd10000 s2 · pdf fileceph appliance: eternus cd10000 s2...

0 © 2015 Fujitsu INTERNAL USE ONLY

Ceph Appliance: ETERNUS CD10000 S2

Walter Graf


Summary Upfront

New Storage Application Areas

How to address with Ceph

Fully integrated and Easy To Use ETERNUS CD10000 S2


New Storage Application Areas

Cloud Infrastructures with OpenStack

Unstructured Data Repositories using Amazon S3 / OpenStack

Swift


The Infrastructure Challenge

Agile Development Lengthy Deployment


The “DevOps” Way

Deployment Automation

Infrastructure as Code

Speeds up Deployment

Makes Q/A possible


ETERNUS CD10000 OpenStack use case

„DBCE“ Cloud Marketplace

„DARZ“ OpenStack Cloud

ETERNUS CD10000 Cloud Storage

Time To Market Quicker Booking & Billing

Agility Automation Flexibility

Cost Reduction

Cinder

Nova

Neutron

Competitive Advantage


Customer / Storage Profile

Service Provider

Large Private Cloud Operators

Own Software Development

Web Frontend

Open Source Minded

Following Cloud Programming Paradigms

Self Managing & Self Healing Storage

Immortal System


The scale of unstructured data is mind-boggling

Data growth 18% p.a. structured data

41% p.a. unstructured data

About 85% is unstructured data


The “Unstructured Data” Opportunity

Big Data repositories Internet of Things

Car Sensor Data

…

Sync & Share „Enterprise Dropbox“

ownCloud, Seafile, ctera, Data Space

…

Create competitive advantages


Object Storage Architecture

Ceph Object Store (ETERNUS CD10000)

RESTful API [S3,Swift]

Ceph Object API

iRODS, Seafile, …

Rados Gateway

Ceph Object API

Owncloud, iRODS, …


University of Mainz

Johannes Gutenberg University of Mainz

~ 36,500 students; > 500 professors

> 150+ departments; > 5,500 employees; multiple institutes and clinics

Central IT department - ZDV

IT services and HPC center


Customer / Storage Profile

Large repositories > 100 TB

Seamless scalability up to multi PB range

Cost efficient storage for less frequently accessed data

Easy accessible (no tape)

Immortal system

No vendor lockin

Viewing object storage as superior over file system storage


Ceph

Scalable

Disaster Resilient

Self-managing

Self-Healing


OpenStack.org User Survey May 2015


Key Components

Calculate instead of lookup

Use objects instead of blocks or files

Automatically place, replicate, balance, and migrate data


True Scalability, Disaster Resilience, Seamless Lifecycle

Access network

Controller 1 Controller 2

Limited Scalability

DR only with multiple systems and

additional software

Major migration efforts at EOSL

Unlimited Scalability

Built in DR

Seamless migration to new nodes, no EOSL

Node 1 Node 2 Node 3 Node 4

…

RAID Ceph


Ceph Software Architecture

Cluster Monitors <10

cluster membership

authentication

cluster state

cluster map

Topology

Authentication

Meta Data Server* (MDS) 10s

for POSIX only

Namespace mgmt.

Metadata ops (open, stat, rename, …)

POSIX

meta data only

Object Storage Daemons (OSD) 10,000s

stores all data / metadata

organises all data in flexibly sized containers

Clients

bulk data traffic

block file object

*) still experimental


From Object to OSD

Object

Placement Group #b

OSD#i

(primary copy)

CRUSH

ruleset

OSD#k

(2nd copy)

OSD#l

(3rd copy)

func(pool, obj_name)

Placement Group #a Placement Group #c


Crush Ruleset Example

1. In fire section 1

i. Using CRUSH select host

ii. Using CRUSH select & remember 1 osd on this host


i. Using CRUSH select 1 host



i. Using CRUSH select 1 host


3 replicas in 3 fire sections


Recover


Why ETERNUS CD10000?

Enterprise Class Architecture – ETERNUS CD10000 S2

End to End Software plus Hardware Integration & Service

Lifecycle Management

Overcome Downsides of SDS


ETERNUS CD10000 > 50 PB

> 200 nodes


ETERNUS CD10000 Architecture

CD10k Backend Network

Mgmt Node

Customer Network

Data

Mgmt

Replica, Recovery, Balancing, Migration

Gateway Node

Gateway Node

…

CD10k Mgmt Network

raw capacity > 50 PB

Storage Node

… Storage

Node Storage

Node Storage

Node

Monitor Node

optional component


Storage Node

44 TB

ETERNUS CD10000 – Flex Nodes

PRIMERGY

2540-M1

(44 TB)

PCIe

SSD

optional

Storage Node

92 TB

PRIMERGY

2540-M1

(44 TB)

ETERNUS JX40 S2

(48 TB)

PCIe

SSD

Storage Node

140 TB

PRIMERGY

2540-M1

(44 TB)

ETERNUS JX40 S2

(48 TB)

ETERNUS JX40 S2

(48 TB)

PCIe

SSD

Storage Node

188 TB

PRIMERGY

2540-M1

(44 TB)

ETERNUS JX40 S2

(48 TB)

ETERNUS JX40 S2

(48 TB)

ETERNUS JX40 S2

(48 TB)

PCIe

SSD


Density Node

Advanced 1U integrated storage node

Based on Intel® Atom™ Processor, which reduces total power

consumption by 50% with no reduction in performance

Equipped with 12 drives, one node provides 60TB capacity

Local SSD Cache option

Innovative drive mounting design eliminates rotational vibration

Ideally suited for use in Erasure Coded pools


ETERNUS CD10000 S2 – Node optimizations

PRIMERGY 2540-M1 (44 TB)

ETERNUS JX40 S2 (48 TB)



PCIe SSD

2.5“ 900 GB 3.5“ 4 TB disks

Modular Capacity Extensions

PCIe SSD becoming optional

PRIMERGY 2530-M1

PRIMERGY 2530-M1

PRIMERGY 2530-M1

1U Mgmt Node

Optional Monitor Node

Optional RADOS GW


Efficiency gains with ETERNUS CD10000 S2

Up to factor 6 less cost per GB with Flex Nodes

Up to 75% reduction in capacity investment for data redundancy with Erasure Coding

Up to factor 10 less costs per GB with Density Nodes

Up to factor 10 higher storage density with Density Nodes

Compared to ETERNUS CD10000 S1

Up to 50% less power consumption with Density Nodes


How To Connect Storage Nodes

10 GbE Switch

10 GbE Switch

10 GbE Switch

8 x 10 GbE

Storage Node

Storage Node

1 x 10 GbE

1 x 10 GbE

2 x 10 GbE Storage Node connection

in a non-redundant configuration

Storage Node connection in a redundant configuration


Switch Cascading

8 x 10 GbE 8 x 10 GbE

2 x 40 GbE

2 x 40 GbE

2 x 40 GbE

2 x 40 GbE

2 x 40 GbE

2 x 40 GbE

10 GbE Switch

10 GbE Switch

10 GbE Switch

10 GbE Switch

10 GbE Switch 10 GbE Switch …

…

…

Uplink connection in a non-redundant configuration

Uplink connection in a redundant configuration


How To Connect Zones (1)

8 x 10 GbE (for redundancy) 8 x 10 GbE

10 GbE Switch

10 GbE Switch

10 GbE Switch

10 GbE Switch

QSFP+

QSFP+

QSFP+

QSFP+

QSFP+

QSFP+

QSFP+

QSFP+

long distance optical cable*

*) depending on cable length MMF or SMF cables / SFPs / QSFPs are needed required cables, SFPs, QSFPs not part of ETERNUS CD10000 portfolio


How To Connect Zones (2)

8 x 10 GbE (for redundancy) 8 x 10 GbE

10 GbE Switch

10 GbE Switch

10 GbE Switch

10 GbE Switch

QSFP+

QSFP+

QSFP+

QSFP+

QSFP+

QSFP+

QSFP+

QSFP+

optical cable**

**) required cables, SFPs, QSFPs not part of ETERNUS CD10000 portfolio

optical cable** Customer Network*

Customer Network* optical cable** optical cable**

*) required components are project specific and are not part of ETERNUS CD10000 portfolio


Ceph is Open Source …

Stay compatible

Contribute instead of fork

Based on Ceph Enterprise Releases

Offer solution maintenance instead of separate contracts for

hardware and software

All backed by Red Hat


Immortal System Concept

Normal Storage

ETERNUS CD10000

storage nodes

other components

initial configuration

initial purchase time normal EOSL

storage upgrade

storage upgrade

new storage nodes

new storage nodes

replacement of old storage nodes and other components

prolonged EOSL prolonged EOSL prolonged EOSL


Ceph Challenge: Installation

Manual Ceph Installation (Plain Ceph)

Setting-up a Ceph cluster with 240+ OSDs

• 1.5 – 2 admin days

Adding an additional node

• from 3 admin hours up to half a day

Automated installation (CD10000)

Setting-up a Ceph cluster with 240+ OSDs

• 1 hour

Adding an additional node

• 0.5 hour


Ceph Admin Challenge: Replacing an HDD

Plain Ceph

taking the failed disk offline in Ceph

taking the failed disk offline on OS /

Controller Level

identify (right) hard drive in server

exchanging hard drive

partitioning hard drive on OS level

Make and mount file system

bring the disk up in Ceph again

On ETERNUS CD10000

vsm_cli <cluster> replace-disk-out

<node> <dev>

exchange hard drive

vsm_cli <cluster> replace-disk-in

<node> <dev>


Ceph Admin Challenge: Adding a Node

Plain Ceph

Install hardware

Install OS

Configure OS

Configure network

Configure S/W repositories

Configure ntp

Configure sudo

Configure ssh

Partition disks (OSDs, Journals)

Make filesystems

Deploy monitor (optional)

Deploy OSDs

On ETERNUS CD10000

Install hardware

• hardware will automatically PXE boot

and install the current cluster

environment including current

configuration

Make node available to GUI

Add node to cluster with mouse click

on GUI


ETERNUS CD10000 = Ceph++

+ Hardware / Software Integration

E2E Solution Contract

Automated Deployment

Cloud Storage

+

+

+ Immortal System

Automated Management +

http://en.wikipedia.org/wiki/File:Ceph_logo.png


ETERNUS CD10000 …

Addresses new storage application areas

Avoids Vendor Lockin

Addresses Key Storage Requirements

scalability, DR, automation, immortal system, seamless migration

Offers optimised price capacity ratio

Stay tuned for further information: http://partners.ts.fujitsu.com

Corporate Partner Portal > Portfolio catalog > Products > Storage solutions > Hyperscale Storage

WP „RAID: End of Era?“, Presentations, Product Insights, Sizing Tool

On Youtube: Search for „ETERNUS CD10000“

http://partners.ts.fujitsu.com/


ETERNUS CD10000 Backup

Rapid Infrastructure Provisioning with OpenStack


The backup principle - still relevant?

Does the 3-2-1 backup principle still rule?

Have at least 3 copies of your data

Keep these backups on 2 different media

Store 1 backup offsite

But, what about, amount of data, bandwith, backup window, cost?


Data protection strategies for petabyte storage

The traditional way

Dominated by explicit requirements

Legacy apps processes

Insists on 3-2-1 principle

New concepts are demanded!

Leverage Ceph functions

Backup-in-the-box

New cloud way

Backup only what you need

Move data as less as possible

Ceph provides

• HW redundancy across sites

• Logical level protection based on Snapshots

OpenStack / Sync & Share make use of the Ceph functions

What we see: Two camps of customers/ mindsets


New: backup-in-the-box

Backup-in-the-box

Pool based backup

Full, incremental, synthetic full

Pool or object-based recovery

To the original pool or to another

CD10000 storage nodes

Ceph Cluster

Pool A Backup

Pool Pool B

walter graf ceph appliance: eternus cd10000 s2 · pdf fileceph appliance: eternus cd10000 s2...

Documents