ibm storage for analytics, cognitive and cloud

© 2017 IBM Corporation

IBM StorageSoftware Defined, Analytics, Cognitive, and Cloud Storage solutions

Tony Pearson – IBM Master Inventor and Senior IT Architect

March 2017


IBM Storage Presentation

Agenda

Perform quickly

Scale Dramatically

Support Elasticity

Cloud, Analytics, Mobil and Social

Support operations

Manage cost

Grow systematically

Traditional workloads

• Software Defined

Storage solutions

• Analytics and Cognitive

Storage solutions

• IBM Cloud Storage

solutions

2



How is Software Defined Environment different?

Applications…� Business Requirements

Infrastructure…� Servers, Storage, Networks

Optimize Your ResourcesOptimize Your Resources

Automate Your Workloads Automate Your Workloads

Sim

plif

ied

Managem

ent

Sim

plif

ied

Managem

ent

ComputeCompute StorageStorage NetworkNetwork

� APIs� Orchestration� Service Levels

� Standard Interfaces� Provisioning� Virtualization

Control

Plane

Data

Plane

Facilities

ServerAdmin

StorageAdmin

Application Owner

NetworkAdmin

3



Hardware independence

Integrated view

Ease of management

Scalability

Cost savings

Increased utilization

Speed

Flexible

Remove bottlenecks

Open approach to infrastructure

SDS Benefits Mentioned (unaided)

Since results are based on qualitative findings specific number of mentions intentionally removed from chart

SDS expected value – Top 7 Reasons

4



Control Plane vs. Data Plane

Data Plane

Control Plane

Resource Pools and Quotas Quality of Service (QoS)

Independent Scalability

5



OpenStack IBM Bluemix Proprietary

IBM is a platinum sponsor of

OpenStack Foundation

OpenStack open source codecan manage IBM compute,

network and storage resources

OpenStack supports x86, POWER

Systems and z System mainframe

IBM offers OpenStack Cinder

interfaces on most of its major

storage products and OpenStack

Swift interfaces for object access

IBM offerings are based on

OpenStack open source code with

value-added proprietary features from IBM

IBM Cloud Orchestratorsupports a variety of server

hypervisors. IBM BlueBox and

IBM PowerVC provide additional

features

IBM Spectrum Control™ provide

reporting, provisioning, trouble

shooting and chargeback

capabilities

VMware and Microsoft are

entirely proprietary, but have a large market share for x86-based

server infrastructure

IBM was VMware’s first OEM and

joint development partner (since

1998) IBM Global Services is one

of VMware’s largest customers

IBM and Microsoft agreement to

offer SQL Server and .NET on

IBM Cloud and IBM software on

Microsoft Azure

“Some assembly required” Enterprise-ready out of the box

x86, POWER systems and z System mainframe Concerns about vendor lock-in

IBM is Focused on these Software Defined Environments

6



Software Defined Control and Deployment

Administrator-Controlled Software-Controlled

How is it controlled?

• ProtecTIER appliance and gateway

• IBM SpectrumStorage™ family

• XIV Storage System• IBM Cloud Object

Storage System

Ho

w is i

td

ep

loyed

?

Industry-Standard Hardware

• Software• Pre-built systems• Cloud services

• Enterprise Tape drivesand libraries

• FlashSystem• Storwize with

Intel QuickAssist• DS8000

Specialized Hardware• Co-processors• ASICs, FPGAs• AdaptersI II

IVIII

Storage for Software Defined Environments

Software Defined Storage

� Runs on industry-standard hardware

� Full suite of storage services

� Embraces multiple storage options

7



Different Clients have Different Focus

Cloud andManaged Service

Providers

• Focus on industry-standard x86 and OpenPOWER servers

that can be quickly re-purposed

• Offer services based on advanced software

that can be deployed as needed on existing

industry-standard equipment

• Predictable, consistent performance

Focus:Industry-standard HW

Focus:SoftwareControls

• Focus on Software APIs to simplify,

automate and control existing investments

• Use specialized hardware to provide optimal

performance for mission-critical applicationsData Centers

8



IBM Storage Portfolio – Family Portrait

IBM Spectrum Control

IBM Spectrum ScaleElastic Storage Server

IBM Spectrum Archive

DeepFlash 150

IBM Spectrum VirtualizeSVC, Storwize

IBM Spectrum Protect

IBM CloudObject

Storage System

Virtual Tape

Systems

FlashSystem

IBM Spectrum Accelerate

LibrariesIBM Spectrum

Copy Data Management

XIV Drives

DS8000

All-Flash

Flash/Disk

Hybrid

Data

Management

Object

Store Disk/Tape

Physical

Tape

9



IBM Storage Portfolio – Software Defined Block Storage

Spectrum Scale

Elastic Storage Server

IBM Spectrum Virtualize

IBM Spectrum Accelerate

FlashSystem XIV

DS8000

All-FlashFlash/Disk

Hybrid

• Storage virtualization of almost 400 different flash

and disk systems

• Thin provisioning, Real-time Compression (RtC),

FlashCopy and Remote Mirroring capability

• Based on technology from SAN Volume Controller

and Storwize family of products

• Offers hot-spot free block-level access across

internal flash and disk resources

• Thin provisioning, Space-efficient Snapshots and

Remote Mirroring capability

• Based on technology from XIV Storage System

10



IBM Storage Portfolio – Software Defined File and Object Storage

11

Spectrum ScaleElastic Storage Server

Spectrum Archive

All-FlashFlash/Disk

HybridData

ManagementObject Store

Physical Tape

• Offers a global name space of file and object

access storage

• Space-efficient snapshots, Information Lifecycle

management (ILM), Active File Management

(AFM) and remote mirroring

• Based on technology from IBM General Parallel

File System (GPFS)

• Extends IBM Spectrum Scale™ to IBM

tape libraries, with LTO or Enterprise

tape drives

• Drastically lowers the cost for

long-term data retention

• Based on technology from IBM Linear

Tape File System (LTFS)



IBM Storage Portfolio – Data Management Solutions

12

Spectrum

Control

Spectrum Protect

Spectrum Copy Data

Management

Data Management

• Capacity Planning and

Provisioning

• Performance Monitoring and

Alerts

• Storage Analytics

• Policy-based Automation

• Service level provisioning

• Reclaim space

• Optimize data placement

• Based on technology from

IBM SmartCloud Virtual

Storage Center, IBM Tivoli

Storage Productivity Center

• Offers Backup, Archive and HSM

capabilities

• Extended to support Virtual and

Cloud environments

• Based on technology from IBM

Tivoli Storage Manager Unified

Backup and Recovery Suite

• In place copy management for

storage reduction, on-premises

and cloud copy creation

automation for increased agility



IBM Spectrum Storage™ Three-pronged StrategySoftware, Systems or Services

Software

• Install software on your

own choice of qualified

industry-standard x86,

POWER and z System

servers

Pre-built Systems

• Deliver hardware pre-

installed with software

• Partner with third party

providers that provided

added value and services

Cloud Services

• Provide offerings on

IBM Bluemix Cloud

• Partner with other Cloud

Service Providers

Scale

13



Agenda

Perform quickly

Scale Dramatically

Support Elasticity


Support operations

Manage cost

Grow systematically



Storage solutions


Storage solutions


solutions

14



A new way at Looking at Data

Transaction and Application data

Database Administrators

System of Record

Gather data into databases

1

ReportsBatchProcessing

Business Analysts

Analyze reports2Business Executives

Make decisions3

statisticalmodels Dashboard

Analyze data2

DataScientists

Empowered Employees

Make Decisions and Take Action

3

System of Insight

Gather and Identify sources of data1

StorageAdministrators

15



What does a Data Scientist do?

� “It’s no longer hard to find the answer to a given question; the hard part is finding the right question. And as questions evolve, we gain better insight into our ecosystem and our business.”

-- Kevin Weil, Lead Analyst at Twitter

� A data scientist must have…

– Strong business acumen

– Modeling, statistics, analytics and math skills

– Ability to communicate findings, tell a story from the data, to both business and IT leaders

� Inquisitive: exploring, doing “what if?” analyses, questioning existing assumptions and processes to spot trends, patterns and hidden insight.

Computers are useless.

They can only give you

answers.

– Pablo Picasso

Source: http://www-01.ibm.com/software/data/infosphere/data-scientist/

http://blog.cloudera.com/blog/2010/09/twitter-analytics-lead-kevin-weil-and-a-presenter-at-hadoop-world-interviewed/

16



� IBM InfoSphere BigInsights is a 100% standard Hadoop distribution� By default, open source components are always deployed� Elect to use proprietary capabilities depending on your needs� In some cases, proprietary capabilities offer significant benefits

Open standards first, but with freedom of choice

HDFS

YARN

HIVE

MapReduce

PIG

SpectrumScale

SpectrumSymphony

BigSQL

AdaptiveMapReduce

BigSheets

Share data with non-Hadoop applications and simplify data management

Re-use existing tools and expertise, Avoid additional development costs

Boost performance, support time-critical workloads, do more with less

True multi-tenancy to boost service levels and avoid duplication on infrastructure

Simplify access for end-users, minimize software development

17



Spectrum Scale – Flexible File and Object Storage

FS1 FS256. . .

Exabyte-Scale, Global Namespace, one big file system, or up to 256 file

systems

Each file system can

be further divided into

fileset containers

Metadata can be separated to

its own Pool or intermixed with

data

Files and objects

can be migrated to

Tape, Object store,

or Cloud

ROBO

Other

Datacenters

18



Elastic Storage Server (ESS) – Spectrum Scale Pre-built Systems

5146-GLx modelsGL2, GL4, GL660-drive 4U drawers• SSD and Nearline HDD

5146-GSx modelsGS1, GS2, GS4, GS624-drive 2U drawers• All SSD• SSD and 10K HDD

Spectrum Scale on

IBM POWER8

servers

5147-GFx modelsDeepFlash ESS GF1, GF264-drive 3U drawers• Pre-loaded with 32 drives• All SSD (8 TB)

19



Hadoop Analytics – HDFS vs IBM Spectrum Scale™

HDFS SaveResults

DiscardR

es

t

IBM HDFS Transparency

Connector allows

HDFS-based programs to process data without application changes (100% compatible)

IBM Spectrum Scale

Application data stored on IBM Spectrum Scale is readily available for analytics

SaveResults

JFS2

NTFS

EXT4

Data Sources mashup of structured and unstructured data from a variety of sources

Actionable Insights Provides answers to the

Who, What, Where, When, Why and How

Business Intelligence & Predictive Analytics> Competitive Advantages> New Threats and Fraud

> Changing Needs and Forecasting

> And More!

20



Agenda

Perform quickly

Scale Dramatically

Support Elasticity


Support operations

Manage cost

Grow systematically



Storage solutions


Storage solutions


solutions

21



Clients are facing explosive growth in Unstructured Data, which is exactly why Object Storage is so critical

* E

xabyte

s

0

20

40

60

80

100

120

2009 2010 2011 2012 2013 2014 2015 2016 2017

Unstructured Data

Structured Data

Source: IDC

Unstructured data growth of

60–80%per yearcreates

Web-scale storage needs

*1 exabyte = 1,000 petabytes =1 million terabytes = 1 billion gigabytes

Problem - Traditional and Legacy Storage Designed for

Transactional, Not Unstructured Data

22



Object Store for Unstructured data

Hot DataHigh-IOPS and Low-Latency

All-Flash and Hybrid Flash/DiskInformation Lifecycle Management (ILM)

Structured data / Random-Access

Virtual Machines and VDI

Single-Tenancy

Static and Stable dataBackups, Files, Archives

Seismic, Research, Telemetry, HPCVideo, Animation, Body Cams

Photos, Images, CAD/CAM, GIS

Music, AudioGenomic, Medical Images

Multi-tenancy

Object Store provides a • Secure

• Reliable • Scalable

• Cost Effective Platform For Unstructured data

Object Storeis not designed for• High IOPS workflows• Transaction

Processing• Inherent ILM

23



How is Object Storage Different?

Block and File Storage

– Decide where to put it

• For block, which array/volume/LUN

• For file, which filer/subdirectory

– Remember where it is to get it back

– Read/Write records, append data

– Limits on LUN size, number of files

Object Storage

– Provide data over to the Object storage

• Get “claim stub” reference locator

– Use or share “claim stub” to access

data HTTP, Openstack Swift, S3

– Get/Put/Delete object in its entirety

– Effectively “unlimited” scalability

24



Data Protection Schemes

Tolerate 1 drive failure Tolerate 2 drive failures Tolerate “M” failures

RAID-1 / RAID-10K pieces � 2 x K slices

RAID-5K pieces � K + 1 slices

2.0X

1.2X

3.0X

1.5X

1.7XTriplicationK pieces � 3 x K slices

RAID-6K pieces � K + 2 slices

Erasure CodingK pieces � K+M =

N slices

25



Data Growth at Petabyte (PB) Scale

PB of data

3 to 5x

Data ProtectionRAID, Mirrors,

Replication, Tape

Data ProtectionHigh Availability & Disaster RecoveryGeo-Distribution & Erasure Coding

InfrastructureProprietary, specialized

hardware, multiple systems

OperationsMore than 1 FTE per PB

Maintenance outages

InfrastructureSoftware Defined,

Commodity Hardware,Single System

OperationsLess than 1 FTE per 6 PB

Single system, SecureSelf-healing

1.7 x 60% LessHardware &Rack space

TraditionalApproach

IBM CloudObject StorageApproach

70%Lower

TCO



CONTENT TRANSFORMATION

IBM Cloud software

encrypts, slices and

applies Information

Dispersal Algorithms

otherwise known as

erasure coding

policies to the data.

Data Ingest

Accesser Software

Storage Nodes

Site 1 Site 2 Site 3

Physical Distribution

Slices are distributed to separate disks and industry standard x86 hardware across geographic locations.

Data Retrieval

Storage Nodes

Reliable Retrieval

An operator defined subset of slices is needed to retrieve data bit perfectly in real time.


BENEFITS

The level of

resiliency is fully

customizable

resulting in a

massively reliable

and efficient way to

store data at scale

as opposed to RAID

and replication

techniques.

AccesserSoftware

Slicestor Software

How the IBM Cloud Object Storage System Works

27



IBM Cloud Object Storage System – Single vs. Multi-Site

28

Single SiteBetter performance, when site-tolerance not a factor, better than traditional RAID-5 / RAID-6

Two Site – Vault MirroringAllows customers to leverage existing two-site infrastructure. Provides concurrent reads and writes despite communication disruption between locations. Local data better than traditional RAID-5 / RAID-6

Three to Nine Sites – Geographically DispersedConsider adding IBM Bluemix Cloud or leverage existing datacenter locations to provide a broader distribution of data for higher availability and scalability



Scalability


S3 Compatible API Openstack Swift Compatible API

Simple Object API

Slicestor Software

Accesser Software

Scalability – Scale performance and/or capacity at any time with no downtime to operations

Need more Performance? Add more Accesser nodes

Need more Capacity? Add more disks to existing Slicestor nodes, or add more storage pools

– All pools must have the same number of nodes

– Difference storage pools can have different amounts of storage

– All nodes in each storage pool must have same amount of storage

StoragePool 1

StoragePool 2

StoragePool 3

29



IBM Cloud Object Storage System – Access Methods

Data



Simple Object API

Slicestor Software

Accesser Software

DIRECT API ACCESS

The Accesser Software exposes three REST

APIs for ingest and retrieval. Applications with

knowledge of these APIs can leverage IBM

Cloud Object Storage directly.



Simple Object API

Slicestor Software

Accesser Software

NAS (NFS/SMB) Backup/Archive General Applications

PARTNER BASED

A variety of Certified technology partners can leverage IBM’s

multi tenancy support to satisfy concurrent use cases on a

single IBM Cloud Object Storage instance.

Hadoop

30



IBM Spectrum Protect provides protection and recovery of:

• Workstations, Files and Systems

• Databases and Applications

• Application-aware hardware-based snapshots

• Virtual Environments

• Replicated critical data

• On premises or Cloud

From remote/branch offices to data centers to disaster recovery sites –

We have you covered with our unified solution.

UNIFIED PROTECTION AND RECOVERY

IBM Spectrum Protect family

31



IBM Spectrum Protect – Data Footprint Reduction techniques

Client compression•Files compressed by client before

transmission•Conserves network bandwidth and server storage

Compress

Client-side Deduplication•Reduces network traffic by deduplication data before transfer

•Sends to server’s dedupe pool

Progressive “Incremental Forever”

•After initial backup, file is not backed up again unless it changes

•Conserves network bandwidth and server

storage

Server-side Deduplication and Compression•Duplicate data only stored once•In-line dedupe to directory or cloud pools

•Files compressed by server after in-line dedupe for containers

•Conserves network bandwidth when writing to cloud

Storage Pool Deduplication and Compression

•Deduplication performed by storage device (VTL or NAS)

•Compression performed by storage device (Disk or Tape)

•Conserves storage capacity

Exclusion Rules•Nodes can configure include/exclude lists

•Eliminates the need to process data for backup, archive and space management

32



IBM Spectrum Protect – Device Classes and Containers

Physical Tape Drives

and Libraries

Virtual Tape

Libraries (VTL)

--- disk pretending

to be tape

Device Class:

TAPE

Device Class:

FILE

Disk pools written

in sequential

manner, allows

multiple reads

Device Class:

DISK

Pre-allocated space

Random access

Sequential Access

DeviceClass

Container Pools

Container:

DIRECTORY

Container:

CLOUD

Slo

we

r, L

ow

er

co

st

Fa

ste

r, m

ore

exp

en

siv

e

• More intelligent space

utilization and I/O balancing

across directories

• Dynamic allocation and

removal of files like

Sequential FILE volumes

• Re-usable space within a

container like Random DISK volumes

On-premises or Off-premises

33



Container Pools can be Directory or Cloud-based

� Containers are managed by the server

– Very little interaction from administrators

Server1

Directory pool has directories with one or more Containers

Cloud Pool has just Containers, represented as Object Store

Dir1

Dir2

Dir3

Dir4

Container 1 Container 5








SAN NAS

34



IBM Spectrum Protect supports Cleversafe!

Client nodes

• IBM Cloud Object Storage System

(using S3-compatible API)

Off-premises:

• IBM SoftLayer

• Amazon S3

On-premises

IBM Spectrum

Protect Server

35



IBM’s approach to storage addresses today’s challenges, and tomorrow’s opportunities…

� Simplify Storage for Hybrid Cloud

� Self-service, Software-defined

�Deliver IT as a Service, increase access to information and improve ROI

� Accelerate Data Delivery and Insight

�Optimal performance with Flash

�Real-time Insights from a variety of data

� Reduce Costs with Storage Efficiency

�Data footprint reduction

� Tape and Object storage systems

IBM Storage – Setting the Data Agenda

36

Analytics &Cognitive

Software Defined Storage

All-Flash Arrays

Object Storage

* Source: IDC, Synergy, Gartner. #1 Flash based on capacity shipped

#1

#1

#1

#1



37



38

IBM Tucson Executive Briefing Center

� Tucson, Arizona is home for storage hardware and software design and development

� IBM Tucson Executive Briefing Center offers:

–Technology briefings

–Product demonstrations

–Solution workshops

� Take a video tour!

–http://youtu.be/CXrpoCZAazg



About the Speaker

Tony Pearson is a Master Inventor and Senior IT Architect for the IBM Storage product line. Tony joined IBM Corporation in

1986 in Tucson, Arizona, USA, and has lived there ever since. In his current role, Tony presents briefings on storage topics

covering the entire IBM Storage product line, IBM Spectrum Storage software products, and topics related to Cloud Computing,

Analytics and Cognitive Solutions. He interacts with clients, speaks at conferences and events, and leads client workshops to

help clients with strategic planning for IBM’s integrated set of storage management software, hardware, and virtualization

solutions.

Tony writes the “Inside System Storage” blog, which is read by thousands of clients, IBM sales reps and IBM Business Partners

every week. This blog was rated one of the top 10 blogs for the IT storage industry by “Networking World” magazine, and #1

most read IBM blog on IBM’s developerWorks. The blog has been published in series of books, Inside System Storage: Volume

I through V.

Over the past years, Tony has worked in development, marketing and consulting for various storage hardware and software

products. Tony has a Bachelor of Science degree in Software Engineering, and a Master of Science degree in Electrical

Engineering, both from the University of Arizona. Tony holds 19 patents for inventions on storage hardware and software

products.

9000 S. Rita Road

Bldg 9032 Floor 1

Tucson, AZ 85744

+1 520-799-4309 (Office)

[email protected]

Tony Pearson

Master Inventor

Senior IT Architect

IBM Storage

39



Email:[email protected]

Twitter:twitter.com/az990tony

Blog: ibm.co/Pearson

Books:www.lulu.com/spotlight/990_tony

IBM Expert Network on Slideshare:www.slideshare.net/az990tony

Facebook:www.facebook.com/tony.pearson.16121

Linkedin:https://www.linkedin.com/in/az990tony

Additional Resources from Tony Pearson

40



Trademarks and Disclaimers

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries. IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency which is now part of the Office of Government Commerce. Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. UNIX is a registered trademark of The Open Group in the United States and other countries. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries.

Other product and service names might be trademarks of IBM or other companies. Information is provided "AS IS" without warranty of any kind.

The customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.

Information concerning non-IBM products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-IBM list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-IBM products. Questions on the capability of non-IBM products should be addressed to the supplier of those products.

All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.

Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here.

Prices are suggested U.S. list prices and are subject to change without notice. Starting price may not include a hard drive, operating system or other features. Contact your IBM representative or Business Partner for the most current pricing in your geography.

Photographs shown may be engineering prototypes. Changes may be incorporated in production models.

© IBM Corporation 2016. All rights reserved.References in this document to IBM products or services do not imply that IBM intends to make them available in every country.

Trademarks of International Business Machines Corporation in the United States, other countries, or both can be found on the World Wide Web at http://www.ibm.com/legal/copytrade.shtml.

ZSP03490-USEN-00

41