big data management: a unified approach to drive business results

Big Data Management: A Unified Approach to Drive Business Results

David Hodgson

MFT04S @dmgh7 #CAWorld

CA Technologies SVP, Business Unit Executive

Mainframe

Bala Venkatrao Cloudera Director of Products

2 © 2014 CA. ALL RIGHTS RESERVED.

Abstract

Traditional data management is changing rapidly attributed to significant changes brought on by evolving big data environments. IT complexity is on the rise as businesses choose the technologies they need to support their big data strategies and targeted business outcomes. Now, more than ever, we need IT management tools that can accommodate and effectively manage these evolving, complex environments to ensure that enterprises can move forward with their preferred technology and vendor choices.

David Hodgson

CA Technologies

SVP, Business Unit Executive

Mainframe Business Unit

3 © 2014 CA. ALL RIGHTS RESERVED.

Agenda

MARKET CHANGES AFFECTING ENTERPRISE IT INVESTMENTS

GAPS/COMPLEXITIES MANAGING THESE ENVIRONMENTS

RECOMMENDED SESSIONS / RELATED ACTIVITIES

360 DEGREE BIG DATA INFRASTRUCTURE MANAGEMENT APPROACH

CLOUDERA’S BIG DATA PLATFORM

SUMMARY

1

2

3

4

5

6

4 © 2014 CA. ALL RIGHTS RESERVED.

Market Changes Affecting Enterprise IT Investments Big Data is Forcing Change

1

Hadoop is a distributed processing technology used for Big Data analysis.

Hadoop market is expanding at a significant rate

Global Hadoop market was valued at $1.5 billion in 2012, and is expected to grow at a CAGR of 58.2% during 2013 to 2020 to reach $50.2 billion by 2020.

Traditional data management has evolved to Big Data Management

Traditional database technologies are evolving from structured data to unstructured and semi-structured data.

Gartner for the first time accepted non-relational technologies–including those based on Hadoop and NoSQL–in its annual Magic Quadrant for Data Warehouses report.

Source: Allied Market Research, March 2014, Lita Person, Link to article

Source: datanami, March 13, 2014, Alex Woodie, Link to article

5 © 2014 CA. ALL RIGHTS RESERVED.

Enterprises Ahead in Big Data Initiatives 1

Survey Category: Is your company currently implementing, planning or considering Big Data projects (i.e. developing strategies and projects to generate more value from existing data)?

Big Data Survey Shows Significant Momentum in Big Data Adoption

<1,000 1,000+ SMB Enterprise

Survey Question SMB % Enterprise%

Have already implemented big data projects 9 16

Have started implementation of big data projects 12 20

Plans are in place to start projects over the next 12 months 10 10

Are considering to begin big data projects next year 9 12

No plans right now but likely to start projects in future 15 12

No plans as of now 44 30

Total % of Big Data projects implemented or planning phase 55 % 70 %

6 © 2014 CA. ALL RIGHTS RESERVED.

Gaps/Complexities Managing these Environments

To keep costs down, you might start with the basic Hadoop distribution from Apache and maybe a free tool or two

You choose several products from multiple vendors for their unique value props.

You eventually evolve to a commercial big data platform and additional database technologies to support your evolving business needs.

Getting started with your Big Data projects may seem fairly easy…BUT…

…by the time you are ready to put this to productive use, you have a huge number of moving parts, tools from many vendors and a ton of complexity

2

7 © 2014 CA. ALL RIGHTS RESERVED.

Gaps/Complexities Managing these Environments 2

Management Challenges – Shortage of Hadoop/Big Data

experts and other skill sets

– Capacity requirements for rapidly changing business needs

– Manual processes

– System problems hard to isolate across the mixed environment

– Unique tools /shortcomings

8 © 2014 CA. ALL RIGHTS RESERVED.

Our Vision - Big Data Infrastructure Management Extending our IT Management Leadership

3

9 © 2014 CA. ALL RIGHTS RESERVED.

Our Vision - Big Data Infrastructure Management Extending our IT Management Leadership

3

“It’s extremely difficult for data scientists, Chief Marketing Officers (CMOs) and other stakeholders to get access to their raw System z data in tandem with machine logs and other types of transactional information,” said Mike Madden, general manager, Mainframe, CA Technologies. “Customers around the world are looking for greater insight to gain competitive advantage and much of the world’s most important transactional data resides on System z. Veristorm provides next-generation data movement technology that makes it easier to move System z data into Hadoop, lowering overall total cost of ownership.”

LAS VEGAS, November 10, 2014 — CA WORLD ’14 — CA Technologies (NASDAQ:CA) today announced a new global distribution agreement with Veristorm, a software company focused on Big Data management. The agreement strengthens CA’s ability to help customers leverage key business data on the mainframe for Big Data and analytics projects.

10 © 2014 CA. ALL RIGHTS RESERVED.

360 Degree Big Data Infrastructure Management Approach 3

Storage Hadoop Distributed File System

(Unstructured/Structured)

Big Data Infrastructure Management Use Cases

Job Mgmt / Monitoring

Alert Management

Multi-Vendor Management System Management

Resource Mgmt / Reporting

Process Mgmt / Automation

Co

nfi

gura

tio

n

Mo

bili

ty

Secu

rity

CA Big Data Infrastructure Management

Big Data Platform Vendors

Hadoop & Hybrid

NAS

Hadoop Distributed File System

Data Movement (ETL)

Data Management

System Health Monitoring

Had

oo

p B

ig D

ata

Pla

tfo

rm V

end

or

A

Had

oo

p B

ig D

ata

Pla

tfo

rm V

end

or

B

Had

oo

p B

ig D

ata

Pla

tfo

rm V

end

or

C

Had

oo

p B

ig D

ata

Pla

tfo

rm V

end

or

D

Hyb

rid

Big

Dat

a P

latf

orm

Ven

do

r A

Hyb

rid

Big

Dat

a P

latf

orm

Ven

do

r B

Hyb

rid

Big

Dat

a P

latf

orm

Ven

do

r C

11 © 2014 CA. ALL RIGHTS RESERVED.

360 Degree Big Data Infrastructure Management Approach 3

SINGLE, CONSISTENT MANAGEMENT UI EXPERIENCE

Linux / x86

SINGLE ACCESS POINT INTO HETEROGENEOUS

ENVIRONMENT

OPERATIONALIZE , MANAGE MULTI-VENDOR HADOOP MANAGEMENT DOMAINS

Big Data Infrastructure

Management Server

CA Big Data Infrastructure Management (In Development) Big Data (Hadoop) Infrastructure

(In Development)

12 © 2014 CA. ALL RIGHTS RESERVED.

CA Big Data Infrastructure Management 3

Demo Scenario: A global financial institution has been using Big Data technologies to bring new investment

products to the market. They are now expanding their Big Data environment to support 6 other business units and an

ever growing number of business initiatives. They also discovered that some of the business units had already started their own Big Data

projects using different big data platforms.

Demonstration

(Under development )

Challenges: Revised budget remains flat and requires a 30% to 50% increase in Big Data environment utilization. Significant complexity associated to hosting multiple Hadoop distributions & an increasing number of business- critical Hadoop clusters to support their business apps.

13 © 2014 CA. ALL RIGHTS RESERVED.

14 © 2014 CA. ALL RIGHTS RESERVED.

15 © 2014 CA. ALL RIGHTS RESERVED.

16 © 2014 CA. ALL RIGHTS RESERVED.

17 © 2014 CA. ALL RIGHTS RESERVED.

18 © 2014 CA. ALL RIGHTS RESERVED.

19 © 2014 CA. ALL RIGHTS RESERVED.

20 © 2014 CA. ALL RIGHTS RESERVED.

Becoming Information-Driven

CA World 2014

Bala Venkatrao | Director of Products | Cloudera

22 © 2014 CA. ALL RIGHTS RESERVED.

Expanding Data Requires A New Approach

23 © 2014 CA. ALL RIGHTS RESERVED.

24 © 2014 CA. ALL RIGHTS RESERVED.

From Hadoop to an Enterprise Data Hub

Hundreds of companies

An energy analytics provider, Opower has saved consumers $500M to date

28 © 2014 CA. ALL RIGHTS RESERVED.

Summary A Few Words to Review

Key Thoughts… The Big Data market is forcing

significant changes to IT.

Most Big Data infrastructures will grow in complexity as business needs evolve.

Think ahead - you will need to effectively manage mixed, heterogeneous Big Data environments.

Next Steps Align a Big Data roadmap to meet

your changing business needs.

Choose the right Big Data technology and vendor for your business needs.

Choose a Big Data Infrastructure management solution that accommodates the range of IT Big Data technologies your business requires now and in the future.

5

29 © 2014 CA. ALL RIGHTS RESERVED.

For More Information

To learn more about Mainframe solutions from

CA Technologies, please visit:

http://bit.ly/1wbiPkl

Insert appropriate screenshot and text overlay from following “More Info Graphics” slide here;

ensure it links to correct page Mainframe

30 © 2014 CA. ALL RIGHTS RESERVED.

For Informational Purposes Only

© 2014 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.

This presentation provided at CA World 2014 is intended for information purposes only and does not form any type of warranty. Some of the specific slides with customer references relate to customer's specific use and experience of CA products and solutions so actual results may vary.

Terms of this Presentation