January 2013

CDMI:An Introduction

Big Data

Complexity

VolumeSpeed

“Big Data” refers to datasets whose size is beyond the ability of

typical tools to capture, store, manage and analyze

2

3

Trends Shaping the Big Data Market

Scale– 10’s PB of data, millions of users

– “Life time retention” of data

Object Access– Object = Data + Metadata

– APIs/Protocols to provide object access

Automatic Data Management– Object granular automated data mobility and management

Distributed Access– Multi-location repositories, data mobility, local view/access

Commonly used term to describe a data store deployment

Apt to describe a Big Data store, increasingly characterized by:– Object based access and APIs

– Multi-protocol support (NAS: NFS/CIFS/Object, SAN: FC/iSCSI)

– Automatic policy-driven management based on object metadata

– Provides services like billing, audit, security

– Globally federated clouds

Formal term: Data Storage as a Service (DaaS)

Cloud Storage

55NetApp Confidential - Internal Use Only

What is CDMI?

Cloud Data Management Interface CDMI is a open standard for cloud storage

– Defines client-to-cloud and cloud-to-cloud interactions– Defines object semantics for content stored in the cloud

Filesystem Clients

CIFS/NFS

Cloud API Clients

CDMI APIClients

ExistingCloudAPIs

CDMIAPI

Notification, Workflow,

Audit,Billing,

AuthorizationCloud Federation and

Peering

CDMIAPI

Existing Cloud APIs

CDMI Notification Feeds, Accounting, Membership, etc...

SNIA CloudInteractions

6NetApp Confidential - Internal Use Only

What does CDMI let you do?

CDMI allows clients to store and retrieve from the cloud– Named Objects (Accessed by name, similar to files)– Nameless Objects (Accessed by ID)– Queues (first-in-first-out structured storage)

CDMI allows clients to organize stored content– Containers (Groups of named objects, similar to directories)

CDMI allows clients to manage stored content– User Metadata (Structured metadata attached to objects)– Data System Metadata (Allows clients to request storage

characteristics)– Versions of objects

CDMI allows clients to discover content– Query and Notifications

7NetApp Confidential - Internal Use Only

Why a standard?

End customers want it:– Customers want cross-cloud interoperability– Customers want to be able to avoid vendor lock-in, switch

cloud vendors and second-source cloud solutions

Application developers want it:– Developers don’t want to develop custom code to talk with

each vendor’s cloud– Developers want to leverage common libraries,

documentation, test tools, expertise, etc.

Cloud storage vendors want it:– Tired of inventing yet another new API that needs to be

documented, tested and maintained– Don’t want their API to be controlled and lead by vendors

Cloud service providers don’t want it (yet):– They like API lock-in, and feel they can adopt CDMI later.

88NetApp Confidential - Internal Use Only

Who created CDMI?

And many more…

A Technical Working Group in SNIA comprising several major vendors in the Storage and Networking industry

Each of the major contributors have provided at least 20 pages of spec text

9NetApp Confidential - Internal Use Only

What does CDMI look like?

CDMI is a wire protocol, like CIFS and NFS, that provides access to objects

CDMI is based on RESTful HTTP, and is designed to be a very simple protocol

CDMI uses JSON, the Web 2.0 data format, and can be directly accessed by any web browser

Basic CDMI functionality includes Create, Read, Update and Delete of objects

Advanced CDMI functionality includes notifications, query, snapshots, serialization, and compliance

10

NetApp Confidential - Internal Use Only

What does CDMI look like?

CDMI

GET /mycontainer/photos/eqd.jpg HTTP/1.1Host: cdmi.example.comDate: Mon, 26 Mar 2007 19:37:58 +0000Authorization: Basic cm9vdDpyb291

<Image Data>

11

NetApp Confidential - Internal Use Only

Opportunities for Partners!

Develop CDMI clients to make legacy applications CDMI aware

Develop CDMI protocol conformance and performance testing tools

Develop tools to migrate data from non-CDMI compliant clouds to CDMI compliant clouds

Develop utilities/services for cloud provides using CDMI data system metadata

Design applications using object metadata

12NetApp Confidential - Internal Use Only

How to learn more?

CDMI specification:– http://snia.org/sites/default/files/CDMI%20v1.0.2.pdf

Participate in CDMI PlugFests:– http://www.snia.org/cloud/cloudplugfest

Use the opportunities to develop