closing the storage gap - presentation from openstack summit in vancouver 2015

it-novum.com openattic.org openattic.com 1

Closing the storage gap How to administrate Ceph storage resources without deep-diving into the technical details

Michael Kienle, CEO

it-novum GmbH [email protected]

Phone: +49 (661) 103-333 Germany: Edelzeller Str. 44 in 36043 Fulda Siemensdamm 50 in 13629 Berlin Austria: Am Europlatz 2 in 1120 Vienna


Agenda

…it is not an OpenStack deep-dive (pls. look at other tracks) …it is not about OpenStack Storage in general (pls. have a look at the OpenStack Paris video on how to build open source storage with support for OpenStack Cinder / block storage) …it is about how to use a modular and extensible open source framework to face current challenges What are the challenges of SDS – SoftwareDefinedStorage? Is Ceph the answer? How to tackle it without the command line…


Datacenter “100%” open source

Monitoring

Infrastructure Configuration Management

Configuration Management

System (CMS)

Software-Defined

Networking Virtualization

Service Transition Service Operation Service Design

Event Correlation

User

Oth

er M

onito

rs

Capacity Management

Risk Management

Cloud Self Service

Sys

tem

App

licat

ion

End

-2-E

nd

Pro

cess

Change Management

Problem Management

SA

P

Applications Service Desk Network Operation Center

Release Management

Cloud Management

Incident Management

Software-Defined Storage

Discovery

Rep

ortin

g S

ervi

ce L

evel

Man

agem

ent

CMDB

Application Mgmt. Security


Datacenter “100%” open source

Monitoring

Infrastructure Configuration Management

Configuration Management

System (CMS)

Software-Defined

Networking Virtualization

Service Transition Service Operation Service Design

Event Correlation

User

Oth

er M

onito

rs

Capacity Management

Risk Management

Cloud Self Service

Sys

tem

App

licat

ion

End

-2-E

nd

Pro

cess

Change Management

Problem Management

SA

P

Applications Service Desk Network Operation Center

Release Management

Cloud Management

Incident Management

Software-Defined Storage

Discovery

Rep

ortin

g S

ervi

ce L

evel

Man

agem

ent

CMDB

Application Mgmt. Security

OpenStack

OpenStack

Software-Defined Storage (SDS) based on Open Source for building a unified storage system on vanilla hardware

Today: configuring and administrating Ceph for OpenStack


The challenges of Storage for OpenStack

Data growth needs to be managed automatically and way cheaper (and even with OpenStack there is a lot of work to do….)

Huge cost savings by using commodity hardware and Open Source software the SDS-way…

Reducing vendor lock-in through open standards The incumbent players are perfect in securing their huge margins by bundling

Standardization enables automation & abstraction of software from hardware Open Source gained strength through open APIs and commodity hardware, for the whole stack incl. Storage


The challenges of SDS

What do you need? Open standards High scalability High reliability No vendor lock-in Lower costs Easy to manage Performance & functions Integration with OpenStack

What you may get! An own developed Open Source tool stack High scalability and reliability…if your stack works Lower investments but higher operational efforts Lock-in into the own stack, without external support


There are many options. Ceph is one of the renowned – it can be all you need! Nearly :)

Ceph provides object AND block storage in a single system that is compatible with the Swift and Cinder APIs and is self-healing without operator intervention.


What does nearly mean?

Ceph is extremely complex because of too many configuration switches, interdependencies and command-line options Ceph might have performance issues in some huge landscapes Ok for pure OpenStack usage but no native support for UnifiedStorage e.g. iSCSI, FC, FCoE, ZFS, XFS, BTFRS, EXT4, etc. Consistent snapshots on application level are not available (legacy applications still depend on it) No guided (template & wizard based) use case based setup for a well-matched configuration “Real” management GUI is missing, Calamari is “view only”


Calamari

Colorful, but view only No management GUI, just monitoring Scale up to 64 nodes only Complicated deployment and installation


Conclusion

Ceph (and Calamari) will not solve all problems What about Unified Storage (FC, iSCSI, CIFS, NFS) needs? Needed versatility in data center environments (if you not only using OpenStack) Intuitive and scalable management GUI Consistent snapshots on application level

New problems come in addition

Complexity Appropriate commodity hardware configuration Performance in some huge landscapes

Now what?


ITSM i-doit & OTRS

Umbrella Monitoring openITCOCKPIT

Cloud Management OpenStack Cinder

Reporting Pentaho

Billing, Backup,... SAP, SEP sesam

Configuration Deployment

Services & Tools

OPEN

API

Web

CLI

Apps

Status Checking

Application Management

Access Samba, NFS, CIFS,

iSCSI, FC, FCoE

Availability Ceph,

DRBD, Pacemaker, Corosync

Efficiency Ceph

ZFS, LVM, BTFRS, XFS

Backup Snapshotting, Tape-Backup, ...

Analyzing openITCOCKPIT, Nagios, ...

Backend

Core

Frontend

Interfaces

Use a modular, Open Source Framework


Mirroring Block- & File-based Synchronous & asynchronous Object-based

Clustering Automated Failover Multi Node Support

Support & Services

Training & Optimization Integration & Development Guaranteed Response Time Guaranteed Restore Time Hardened Environment Sizing Services

Protection Integration Tape-Systems Snapshotting Consistent Snapshotting

Efficiency Deduplication Compression Performance Config-Set Config-Set for Virtualization

Unified and versatile SAN / Block Storage

(FC, iSCSI, FCoE) NAS / File-based (CIFS, NFS) Object Storage

Central Configuration Web Frontend Command Line Interface

Dynamic Monitoring Availability Performance Capacity

Cloud Support OpenStack/Cinder openQRM Ceph

Umbrella Management

Master/Slave Architecture

Features


Manage your Ceph nodes with openATTICs GUI

Packaged installation for rapid deployment through puppet/foreman (vanilla hardware, bare-metal) Dashboards for cluster status overview Management of clustered server hardware (add/remove node, replace devices) Cluster management (resources, cluster and pool creation) Monitoring the cluster (overall capacity, pool utilization, status of OSDs, Monitor and MDS processes, Placement group status, RBD status) Visualized CRUSH Map Wizard-based best practices for many use cases Hardened appliances based on Supermicro with guaranteed response and restoration times Unified Storage: CIFS, NFS, iSCSI, FC, FCoE, ZFS, XFS, BTFRS, open API, SnapApps & much more


openATTIC at a glance

openATTIC wizards for easy deployment



Volume Monitoring



Replication Rules


Visualization



openATTIC Features

Open API

Object Store

Cloud Storage

Application Snapshotting

HA & Monitoring

Smart SDS

HDFS, CephFS, btrfs, zfs, xfs

FC, iSCSI NFS, CIFS

Unified Storage

Trust in Open Source Increase agility, simplify management and radically reduce costs.

Without compromises on availability, reliability or functionality.


Thank you for your attention

Trust in Open Source Increase agility, simplify management and radically reduce costs.

Without compromises on availability, reliability or functionality.

We are looking for partners! Join the community openattic.org & feel free to contribute

You need enterprise support: openattic.com