achieving business value through big data...

Philip Russom

TDWI Research Director for Data Management

October 3, 2012

Achieving Business Value through

Big Data Analytics

2

Sponsor

Speakers

Philip Russom Research Director,

Data Management,

TDWI

Brian Ng

Director, Enterprise Services,

HP

Today’s Agenda

• The Need for Business Value

from Big Data

• Definitions of Big Data Analytics

• Use Cases for Big Data Analytics

that deliver Business Value

• The Future & How to Prepare for It

Background • The quantity and diversity of Big Data has been exploding for years

– Traditional applications grow larger & more numerous every day

– Older big data sources: RFID, call detail record, machine/robotic data

– New big data sources: sensors, social media, new Web apps

• User organizations are starting to achieve business value from big data

– The consensus today is that Advanced Analytics yields valuable business insights

– As long as big data is managed well and treated to the right forms of analytics

• Today we’ll look at how Big Data Analytics can deliver business value

In your organization is big data considered mostly a problem or mostly an opportunity?

Source TDWI. Survey of 325 respondents, June 2011

Opportunity – because it yields detailed

analytics for business advantage

Problem – because it's hard to manage

from a technical viewpoint

70%

30%

Definition of Big Data Analytics

• It’s where advanced analytic techniques operate on big data sets.

• It’s about two things: big data AND advanced analytics. – The two have teamed up to leverage big data.

– The combo turns big data into an opportunity.

• Big Data isn’t new. Advanced Analytics isn’t new.

– Their successful combination is new.

– Both users and technologies are now more capable of success.

• The combo is new & technical.

– But hasn’t yet aligned with business.

Big

Data

Ad

van

ce

d

An

alytic

s

Big Data

Analytics

Big Data

Analytics

The “3 Vs” of Big Data summarize

technical properties

Business Value should be the 4th V,

since this is what IT must deliver.

VOLUME

VARIETY VELOCITY VARIETY VELOCITY

VOLUME

VARIETY VELOCITY

BUSINESS

VALUE

Defining Advanced Analytics – Online Analytic Processing (OLAP)

– It’s somewhat rudimentary, but required.

– Demands multidimensional data modeling, but works well with most EDWs.

– There are multiple approaches to OLAP.

– Extreme SQL – Uses well-known SQL-based tools & techniques.

– Relies on long, complex SQL statements.

– Predictive Analytics – Uses data mining and/or statistics

to anticipate future events.

– Multi-Structured Data Analytics – Natural language processing (NLP)

– Search, text analytics, sentiment & social analytic apps

– Other Analytic Methods – Visualization, artificial intelligence

– Analytic database functions: in-database analytics, in-memory databases, columnar data stores, appliances, etc.

Advanced Analytics

• Discovery oriented

• Excels with Big Data

• Experiencing strong

adoption by users

OLAP & its Variants

• Users have this

• They’ll keep & grow it

• OLAP won’t go away

TDWI SURVEY SAYS:

Opportunities for Big Data Analytics • Anything involving customers benefits from big data analytics

– better-targeted social-influencer marketing (61%)

– customer-base segmentation (41%)

– recognition of sales/market opportunities (38%)

• BI, in general, benefits from big data analytics

– more numerous and accurate business insights (45%)

– understanding business change (30%)

– better planning and forecasting (29%)

– identification of root causes of cost (29%)

• Specific analytics applications are likely beneficiaries

– detection of fraud (33%), quantification of risks (30%)

– market sentiment trending (30%)

Source TDWI. Survey of 325 respondents, June 2011

USE CASE

Exploratory Analytics with Big Data • Big Data enables exploratory analytics.

• Discover patterns and new facts the business didn’t know

– Customer base segments

– Customer behaviors and their meaning

– Forms of churn and their root causes

– Relationships among customers and products

– Root causes for bottom line costs

– State of biz today; predict future events

USE CASE

Analyze Big Data You’ve Hoarded • Yes, it’s true:

– Many firms have “squirreled away” large datasets, because they sensed business value, yet didn’t know how to get value out of big data.

• Finally understand: – Web site visitor behavior

– Products of affinity based on eCommerce shopping carts

– Product and supply quality based on robotic & QA data from manufacturing

– Product movement via RFID in retail

USE CASE

Big Data Analytics per Industry • The type and content of big data can

vary by industry, thus have different

value propositions per industry:

– Call detail records (CDRs) in

telecommunications

– RFID in retail, manufacturing, and

other product-oriented industries

– Sensor data from robots in

manufacturing, especially automotive

and consumer electronics

USE CASE

Analytics for Unstructured Big Data • Tools based on natural language

processing, search, and text analytics (plus new platforms like Hadoop) provide visibility into text-laden business processes:

– Claims process in insurance

– Medical records in healthcare

– Call center and help desk applications in any industry

– Sentiment analysis in customer-oriented businesses, with both enterprise and social media big data

USE CASE

Customer Analytics with

Social Media Data • Customers can influence each other by commenting on brands, reviewing

products, reacting to marketing campaigns, and revealing shared interests

• Predictive analytics to discover patterns, anticipate product/service issues

• Measuring share of voice and brand reputation

• Broader input for customer satisfaction

• Understanding sentiment drivers

• Voice of the customer analytics

• Determining marketing effectiveness

• Identifying new customer segments

“I love/hate

your product!”

USE CASE

Big Data for

Complete Customer Views • Big data can add more granular

detail to analytic datasets:

– Data from all customer touch points

– Broaden 360-degree views of customers and other entities, from hundreds of attributes to thousands

– For more detailed and accurate customer base segmentation, direct marketing, and other customer analytics

USE CASE

Big Data Can Improve Older Analytics • Big data enlarges and improves data samples for

older analytic applications:

– Any analytic technologies that depend on large samples, such as statistics or data mining

– Fraud detection

– Risk management

– Actuarial calculations

USE CASE

Analytics with Streaming Big Data • Monitoring & Analysis in True Real Time

– Energy utility, communication network; any grid, service, facility

– Surveillance, cyber security, situational awareness

– Fraud detection, risk calc

– Logistics, truck/rail freight, mobile asset mgt

• Near Time – Review of loan applications

submitted online

$$$$$

$$$$$

$$$$$

$$$$$

$$$$$

$$$$$

A Look

Into the Future of

Big Data

Analytics

• Big data analytics is here to stay

– It will spread into more apps in more

industries, becoming mainstream

• Big data will be less of a problem

– Due to advances in storage, clouds, CPUs,

memory, databases, analytic tools, etc.

• Analytics will draw biz value from big data

– That’s why the two have come together

• New types of analytic apps will appear

– Old ones will be revamped

• Big Data Analytics is mostly batch today

– Will go real time as users/techs mature

• Analytics is new competency for many shops

– They will hire & train, plus acquire tools and

seek professional services

110100110

101110100

100101011

Recom

mendations

• Insist on business value from big data

– Don’t merely hoard it in a cost center that wastes valuable storage & other resources

– The path to business value is through analytics

• Go beyond reporting and OLAP into advanced analytics

– You need “discovery” analytics, but reporting and OLAP won’t go away

• Embrace the brave new world of big data

– Data from Web, machine, and social sources

• Upgrade, extend or distribute your BI/DW tech stack and other software portfolios with technologies for big data and analytics

– Change is needed to accommodate analytics with big data

• Give the business what it needs

– Discovery analytics to understand change, find opportunities

– Broader, more complete views of customers & other business entities

– Analytics tailored to your industry and your organization’s unique requirements

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Achieving business value through big data analytics HP Enterprise Services, Information Management and

Analytics,

Brian Ng / Oct 2012

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 21

Thriving in the age of big data

Our point of view

We are at a fundamental inflection point in the evolution

of information and intelligence.

Traditional approaches, architectures and organizations

models were not designed for today’s complexity.

Leadership will be defined by those who excel in information

sciences, via innovative solutions, advanced technologies & new

talent models.

Social Call

Records

Risk

Sensors Sentimen

t

RFID

Claims

Fraud

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Use cases and architecture


Service management

Event processing

Data acquisition Repository

Analysis and reporting

Governance

Structured data

Semi-structured

data

Unstructured data

Data

ca

ptu

re

Data transformation

Master data

Derive metadata

and index

Match and combine

Populate repositories

Relational DBMS data warehouse

Non-relational DBMS

(e.g. HDFS, Hbase,...)

Non-relational DBMS

(content mgt systems)

Data mart (e.g. OLAP

cube)

Real-time analytical RDBMS

Complex event

processing Rules engine

NoSQL (e.g.

MapReduce) engine

Search engine

SQL analytics engine

Data mining engine

Visualization

Static and OLAP reports

Dashboards and alerts

Statistical analysis

Portfolio management

SOA services Applications

Data governance

Rules

generator

Data audit, balance and control

Operations management

Predictive analysis

Data Virtualization

Logical architecture

1. Unstructured and structured analysis

External data

Internal data

Data marts

Olap cubes

Reporting Dashboards

Olap Statistical analysis

Data mining Visualization

Human Language

Capture Visualization

Analysis Raw Data Repository

Analytical Data Mart

Applications

Rich Media

Relational DBMS Staging

Integration Enterprise DW

Sentiment, Mark-up & Integrate

Data quality Master data mgt

Integration Aggregation


Service management

Event processing



Governance

Structured data

Semi-structured

data

Unstructured data

Data

ca

ptu

re

Data transformation

Master data

Derive metadata

and index

Match and combine



Non-relational DBMS


Non-relational DBMS



cube)


Complex event


NoSQL (e.g.

MapReduce) engine

Search engine


Data mining engine

Visualization






Data governance

Rules

generator



Predictive analysis

Data Virtualization


2. Machine generated data streams

External data

Internal data





Data marts

Olap cubes




Capture Markup, stream, integrate

Model and Rule development.

Raw Data Repository


Sensor Network

Applications

Real time visualization and analysis

Complex Event Processing

Rules Engine


Fraud detection

Use case: Insurance claim fraud

Business issue • Insurance claim fraud continues to be a major cost

Big Data sources • Claims form (human language)

• Contact records (call center logs, audio, email,

instant message, video calls)

Process • Sentiment analysis and meaning-based scoring

• Input structured result-set into fraud analysis

• Machine learning for key patterns

Business benefit • Avoid cost

• Improve margins

• Competitive pricing


Service management

Event processing



Governance

Structured data

Semi-structured

data

Unstructured data

Data

ca

ptu

re

Data transformation

Master data

Derive metadata

and index

Match and combine



Non-relational DBMS


Non-relational DBMS



cube)


Complex event


NoSQL (e.g.

MapReduce) engine

Search engine


Data mining engine

Visualization






Data governance

Rules

generator



Predictive analysis

Data Virtualization

Human language data and analysis

Use case: Insurance claim fraud

Claims form contact data

Capture Sentiment analyses & integrate

Claims application

Raw data repository

Analytical data mart

External data

Internal data





Data marts

Olap

cubes





Use case: Operations Optimization

Business Issue • Under utilized facilities

• Less effective Supply and Delivery Chains

• Less accurate R&D

Big Data sources • Sensors in supply/delivery chains

• Network sensors (communication, smart grid)

• Physical sensors (seismic, health, equipment)

Process • Statistical, Segmentation and Pattern analysis

• Real time advanced visualization

Business Benefit • Optimized supply and delivery chain operations

• Better utilization of facilities

• Improved R&D results

Supply Chain


Service management

Event processing



Governance

Structured data

Semi-structured

data

Unstructured data

Data

ca

ptu

re

Data transformation

Master data

Derive metadata

and index

Match and combine



Non-relational DBMS


Non-relational DBMS



cube)


Complex event


NoSQL (e.g.

MapReduce) engine

Search engine


Data mining engine

Visualization






Data governance

Rules

generator



Predictive analysis

Data Virtualization


Machine generated data streams

External data

Internal data





Data marts

Olap cubes




Capture Markup, stream, integrate

Model and Rule development.

Raw Data Repository


Sensor Network

Process Applications

Real time visualization and analysis

Complex Event Processing


Advanced Analytics using Vertica, Autonomy, and Hadoop

HP Changing the Analytics Paradigm

Information Insight by

business analyst

SQL

Business Objects, Cognos, OBIEE, Microstrategy

Business

Users

Structured

Transaction

Data

Unstructured

Consumer Data

Device Data

NoSQL

Search Engine, Trends (Market, Consumers, etc) Data Exploration

Advanced

Analytics

SAS, R Predictive, Performance, Operations

Taxonomy

Aggregation

(IDOL)

Analytic Data Store

Vertica

Operational Data

Store

Teradata, Oracle,

DB2

Hadoop

Unstructured

Data Store

Info

rma

tio

n T

axo

no

my

Meaning Based Computing

Ta

xo

no

my A

gg

reg

atio

n

Information Transformation

Seamless Data Exploration

and Analytics

Ability explore unstructured information

to uncover important attributes, time

periods, groups, or areas of

information using Non-Sql techniques

1. Conduct Information research

using data visualization, trends,

and Google like search tools by

accessing the Hadoop information

repository

2. Leverages a common information

taxonomy (ontology) that creates

business views across all

information from all sources

3. Automatically move this data from

research to analytics environment

4. Conduct Business Analytics using

metrics and KPI’s

5. All from real-time information

initiated from End User request

2

1

3 5

Automated Information Integration

4


Next steps

Strategy Roadmap Design Implement Consume

• Big Data Experience Transformation Workshop

• Social Intelligence Workshop

• EDW OnTrack Workshop

• BI Implementation

• Advanced Information

• Services for HP, SAP

• and Microsoft

• On Premise

• Managed Service

• Cloud Service

• Information Strategy

and Organization

Services

• Business Solutions

• Social Intelligence

• Advanced Analytics

• MasterPlan

Services

• Business

Value

Assessment

32

Questions?

Contacting Speakers

• If you have further questions or comments:

Philip Russom, TDWI

[email protected]

Brian Ng, HP

[email protected]

achieving business value through big data...

Documents