driving big data · • better processing performance • extend existing edw capacity • meet...

© 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 1

Davy Nys, VP EMEA & APAC [email protected]

December 2013

Driving

Big Data


The New Reality Simplified Analysis for all Users

ANY Analytics

• Reports

• Dashboards

• Visualizations

• Discovery

• Predictive

Analytics

ANY Environment

• Data warehouses

• Data marts

• Stack vendors

• Cloud

• Embedded

Existing & New Data

Infrastructure &

Processes

ANY Data

• Relational

• Operational

• Big Data

• Data sources not yet

anticipated…

Billing

Location

Social

Media

Customer

Web

Network

© 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 3 © 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 3

Emerging Funded Big Data Use Cases

ENHANCED 360 VIEW OF CUSTOMER • What makes them tick, why they buy, preferences

BIG DATA EXPLORATION • Find, visualize & understand all the data stored across silos

DATA WAREHOUSE AUGMENTATION • Optimize data warehouse – offload appropriate data

MACHINE & OPERATIONAL DATA ANALYSIS • Machine & ops data from sensors, meters, GPS devices…

SECURITY/INTELLIGENCE • Lower risk, detect fraud & monitor cyber security


Evolving Big Data Architectures

P D I

Existing

ETL Tool

or PDI EDW Data Marts

Analytics

Existing

ETL Tool

or PDI

Customer

Provisioning

Billing BI Tools

Location

Web

Social

Media

Network

Existing

Process

or PDI Hadoop

Cluster

NoSQL

P D I

Analytic DB

On-Demand Integration & Blending


Why Blending at the Source Matters Customer Experience Analytics for Loyalty and Revenue

Analytics

Analyze quality of service: • Network outages

• Dropped calls

• Poor quality

• Calls to support center

For profiles of customers: • Up for renewal

• Profitable

• Multiple agreements/services

• In competitive area

Determine best action to take: • Billing Credit

• Customer Coupon

• No Action

EDW

Existing ETL Tool

or PDI Customer

Billing

Provisioning

Call Detail Records from:

• Billing

• Payment

• Usage

NoSQL Network

Location

PDI

Call Detail Records from Network:

• Outages

• Drops

• Service Quality

PDI

Blend revenue-related and

quality-of-service data

together to find customers at

risk


Entry

Tra

nsfo

rm

Advanced

Op

tim

ize

A Spectrum of Big Data Use Cases What the Market is Deploying Today and Planning for Tomorrow

Data

Warehouse

Optimization

Streamlined

Data Hub

Big Data

Exploration

Customer

360 Degree

View Harnessing

Machine &

Sensor Data

Next

Generation

Applications

Internal Big

Data as a

Service

On-Demand

Big Data

Blending

Big Data

Predictive

Analytics

Use Case Complexity

Bu

sin

ess

Imp

act


Data Warehouse Optimization Shrink Data Costs & Boost Analytics Performance for Business Users

PDI

CRM & ERP

Systems

Other Data

Sources

Hadoop

Cluster

Data

Warehouse Analytical

Data Mart

Relational

Layer

PDI

PDI PDI

Why Do It? • Save data capacity & management

costs

• Empower business users to meet

their operational goals on time

Benefits • Lower data management costs

• Better processing performance

• Extend existing EDW capacity

• Meet batch window SLAs to

deliver fresh data to users

• Retain more data for analysis

Challenges • May require new coding skillsets

that are hard to find

• Reporting off ‘active archive’

requires a relational layer on top of

the big data store (such as Impala

or Stinger)


Streamlined Data Hub Drive a Sustainable Analytics Strategy with Big Data ETL at Scale

Transactions –

Batch & Real-time

PDI Enrollments &

Redemptions

Location,

Email, Other

Data

Hadoop

Cluster

PDI Analytical

Database

Analyzer

Reports

Benefits

• Establish usable analytics on diverse

sources at high volume (terabytes+)

• Speed queries substantially with

rapid ingestion & powerful

processing

• Reduce costs of ETL

Challenges

• Expansive integration project

• May require new coding skillsets that

are hard to find

• May call for swapping from a DW to

an Analytical DB, depending on

requirements

Why Do It?

• Give business users insight into all

data

• Scale ETL and data management

cost savings

• Next step after DW optimization


Big Data Exploration & Discovery Tap the Latent Value in Massive Data from Diverse Sources

PDI Social Media

Web/Mobile

Tracking

Hadoop

Cluster

Email

Tracking

Data Mining

& Discovery Analytical

Database BI Tools

PDI

Benefits • Discover new useful information and

understand its value

• First step toward identifying trends

and drivers that can affect business

outcomes

• A low-risk place to start turning Big

Data into business value

Challenges • May require new coding skillsets that

are hard to find

• Must properly scope/contain the

costs of an exploratory project

• Data mining component may require

expensive skillsets – data scientists

and PhDs

Why Do It? • Understand the data you have

• Identify crucial patterns in your

business & operations

• Motivate high-impact projects


Internal Big Data as a Service Cost Effectively Scale Database Service Across Teams

PDI

Hadoop

Cluster

-or-

NoSQL

Transactional

data

PDI

Log data

Relational

Data

Other

sources

IT User

Access to

Data &

Analytics

Benefits • Scale productivity through

centralized data infrastructure

• Provide reliable service and

enterprise-grade SLAs across IT

organization

• Repurpose your high-value tech

experts to service a broader

stakeholder base – share expertise

Challenges • Hard to find skillsets to migrate data

into NoSQL

• Need to scope out reporting strategy

in addition to operational use of Big

Data for shared IT service

• Oriented primarily to IT and

developers – must still address

business user analytics approach

Why Do It? • Save costs by standardizing data

service across all IT teams

• Promote operational efficiency


Customer 360 Degree View A Blended View to Drive Revenue Growth and Service Improvements

NoSQL

CRM

System

Documents &

Images

Admin.

Info

Claims

Online

Interactions

Call Center

View

Research

Analysts

Predictive

Analytics

PDI PDI

Benefits • All customer touch point data in a

single repository for fast queries, &

all key metrics in a single location for

business users

• Blend previously isolated data and

avoid point-to-point integrations

• Boost customer service & revenue

Challenges • Transformative effort in both

technology implementation &

business planning/definition

• Complex data structures and ETL

tools for collecting & enriching data;

complex data schemas

• May require new coding skillsets that

are hard to find

Why Do It? • Learn how your customers perceive

your brand

• Boost revenue

• Lower churn

• Increase cross-sell & upsell

effectiveness


Harnessing Machine & Sensor Data Operational Intelligence to Spur Service Automation & Product Innovation

Device Network

High Velocity

Data Storage

Nodes

Message Queue

(Kafka)

Web Portal –

Dashboard,

Visualization, Admin

Message

Processing

(Storm)

NoSQL

Hadoop

Cluster PDI

PDI

Benefits • Can enhance revenue & cut costs

• Reduce cost of customer support &

increase customer satisfaction

• Optimize service offering according

to consumption patterns

• Ability to retain customers through

understanding their experience

Challenges • Having right skillsets for design,

implementation, & operation

• Project needs to be properly scoped

& defined to ensure scalability

• Often combines several different

emerging technologies

Why Do It? • Understand how your products are

used in the field

• Reduce service costs & churn

• Enable value-added product

innovation


On-Demand Big Data Blending Accelerate Analysis to Support Just-in-time Decisions

PDI

NoSQL PDI

DW Existing

ETL or

PDI Business

Analytics Integration

& Blending

Just-in-time

Customer

Provisioning

Billing

Network

Location Benefits

• Unlock value of near real-time data:

Act on it today, not next week

• Quickly react to customer behavior

• Improve operational effectiveness as

issues arise

• Connect to new data sources without

increasing database cost

Challenges • May require new coding skillsets that

are hard to find

• Proper scoping so that only time-

sensitive data that needs to be

analyzed on demand is streamed

directly ‘from the source'

Why Do It? • To analyze Big Data right away

• Support on-demand info needs

without sacrificing accuracy or

governance


Next Generation Apps Deliver Value via Architecture Innovation & Embedded Analytics

Business

Analytics

Server

Hadoop

Cluster

PDI

RDBMS

Data

Mart

Metadata

Analyzer

Dashboards

Reporting

Embedded Analytics

Content in Web App UI Benefits • Faster data processing for better

application performance

• Optimize service by data mining &

benchmarking on very large data

sets (i.e. customer base) on the fly

• Tap into high volume unstructured

data sources (i.e. Social) to make

apps more intelligent and flexible

Challenges • A 'bet the farm' strategy, a major

change in your app infrastructure

• Open ended - Requires original

thinking to create value proposition

• Heavy investment in skillsets

• Relatively long term project adds to

uncertainty

Why Do It? • To create a unique value

proposition

• Build competitive advantage

• Drive sales & win markets


Big Data Predictive Analytics Supercharge Predictions by Refining Models in Hadoop

PDI Social Media

Hadoop

Cluster

Email

Tracking

Data Mining

& Predictive

Analytics

PDI

Predictive

Analytics

in-cluster

Web

Behavior

Data

Benefits • Data processing power, speed, and

scalability of Big Data stores can

facilitate increased accuracy for

outcome prediction

• Revenue enhancement and risk

reduction potential, depending on

specific use case

Challenges • Requires data scientists and PhDs -

expensive resources

• Usually a second or third use case,

after experience with respect to Big

Data has been developed

Why Do It? • Improve prediction of business

risks, like fraud or security

breaches

• Improve predictions of customer

behavior, like buying decisions


Entry

Tra

nsfo

rm

Advanced

Op

tim

ize

A Spectrum of Big Data Use Cases What the Market is Deploying Today and Planning for Tomorrow

Data

Warehouse

Optimization

Streamlined

Data Hub

Big Data

Exploration

Customer

360 Degree

View Harnessing

Machine &

Sensor Data

Next

Generation

Applications

Internal Big

Data as a

Service

On-Demand

Big Data

Blending

Big Data

Predictive

Analytics

Use Case Complexity

Bu

sin

ess

Imp

act

© 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 17 © 2013, Pentaho. All Rights Reserved. pentaho.com. Worldwide +1 (866) 660-7555 17

Final Parting Thoughts

EXPECT ADOPTION TO DRIVE MORE USE CASES • Have a future proof architecture

KEEP DATA GOVERNANCE IN MIND • User flexibility is key, complex data blending should be architected

LEVERAGE EXISTING INFRASTRUCTURE • Optimize/augment data warehouse – offload appropriate data

AVOID WRITING LEGACY CODE • Flexibility, Time to value & Cost savings

AVOID DATA VENDOR/TYPE LOCK IN • New use cases, new data sources, new data types, ….


Thank You

blog.pentaho.com

@Pentaho @davynys

Facebook.com/Pentaho

Pentaho Business Analytics

JOIN THE CONVERSATION. YOU CAN FIND US ON: