achieving business value by fusing hadoop and corporate data

Grab some

coffee and

enjoy the

pre-show

banter before

the top of the

hour!

The Briefing Room

Achieving Business Value by Fusing Hadoop and Corporate Data

Twitter Tag: #briefr The Briefing Room

Welcome

Host: Eric Kavanagh

[email protected] @eric_kavanagh


  Reveal the essential characteristics of enterprise software, good and bad

  Provide a forum for detailed analysis of today’s innovative technologies

 Give vendors a chance to explain their product to savvy analysts

  Allow audience members to pose serious questions... and get answers!

Mission


Topics

March: BI/ANALYTICS

April: BIG DATA

May: CLOUD


Data Gravity

Ø  Moving data is problematic Ø  Data likes to stay where it is Ø  Access methods are improving Ø  We live in an increasingly multipolar world!


Teradata

  Teradata is known for its analytics data solutions with a focus on integrated data warehousing, big data analytics and business applications

  It offers a broad suite of technology platforms and solutions and a wide range of data management applications

  Last year, Teradata announced QueryGrid, a data access layer that can perform analytics across multiple databases and Hadoop


Guests

Dan Graham Technical Marketing Director Teradata

Dr. Robin Bloor Chief Analyst The Bloor Group

Dr. Richard Hackathorn Industry Analyst Bolder Technologies

9

 Teradata QueryGrid Use Cases  Dr. Richard Hackathorn – Bolder Technology

 Dan Graham, Teradata  March 25, 2015

10

•  Based on technical interviews

•  QueryGrid concepts

•  Customer success stories –  Teradata to Hadoop

–  Aster to Hadoop

–  Aster to both

•  Themes across multiple customers

QueryGrid Successes Agenda

11

 Three-Peer Platform Ecosystem

•  IDW = structured system of reference data widely shared

•  Discovery = research and exploration

•  Data = curate new, changing data sources

Data Scientists

INTEGRATED DATA WAREHOUSE

TERADATA DATABASE

ASTER DATABASE

DISCOVERY PLATFORM

DATA PLATFORM

HADOOP

Business Analysts

Casual Users

Programmers

12 © 2015 Teradata

Teradata and Aster: QueryGrid

IDW

TERADATA DATABASE

Discovery

ASTER DATABASE

Business users Data scientists

TERADATA ASTER

SQL, SQL-MR, SQL-GR

OTHER DATABASES

Oracle, MongoDB

Teradata Systems

TERADATA DATABASE HADOOP

Push-down to Hadoop

SAS, Perl, Python, Ruby, R

LANGUAGES

13

QueryGrid

Data Lake/DataHub Data Warehouse SELECT TDP.Prod_ID ,TDP.Prod_Name ,HW.Sensor_Mfr ,HW.Sensor_ID FROM Sensor_data@HW_Hadoop ,TD_Products TDP WHERE TDP.Sensor_ID = HW.Sensor_ID;

Sensor_data@HW_Hadoop

14

QueryGrid Use Cases Company Use Case Maturity

Vehicle Manufacturer Identifying maintenance targets Daily use

Detecting unnecessary maintenance Prototype

Communications Improving customer retention Daily use

Financial Services Monitoring brokerage compliance Production

Processing weblog sessionization Prototype

Travel Services

Conversion funnel using website logs Daily use

Conversion funnel using IVR call center logs Prototype

Improving website design with A/B testing Daily use

Computer Manufacturer

Generating leads from customer journey Daily use

Telecommunications Reducing customer churn Daily use

Customer satisfaction call center dashboard Prototype

eCommerce Improving website searching Production

Financial Systems Mfg.

Reducing travel costs Production

Electronic Manufacturer

Monitoring process quality control Prototype

15

 Vehicle Manufacturer

 Problem

•  Predicting machine repairs using sensors

•  Business and Engineers disconnected

 Challenge

•  Get more value from the sensor data

•  How to get data to SQL tools and users?

 Results

•  QueryGrid provides the bridge between cultures – now they collaborate!

DATA PLATFORM

HADOOP


TERADATA DATABASE

Business Analysts

Engineers & Programmers

Query Grid

16

 Financial Services

 Problem

•  Brokers overstating financial returns

•  Government audits and $$$ penalties

 Challenge

•  Text analysis of Outlook emails

•  Investigate possible violations

•  Sessionize interactions

 Results

•  50% reduction in false positives

•  Reduced labor reading emails

ASTER DATABASE

DISCOVERY PLATFORM

DATA PLATFORM

HADOOP

Business Analysts

Query Grid


TERADATA DATABASE

Business Analysts

17

 Travel Services

 Problem

•  Conversion funnel of lookers to bookers

•  How best to spend Marketing funds?

 Challenge

•  Correlate weblogs and call center IVRs

 Results

•  Data placement on Hadoop –  Sessionize consumer interactions

•  Calculating A/B testing to increase sales

DATA PLATFORM

HADOOP


TERADATA DATABASE

Business Analysts

Programmers

Query Grid

18

 Computer Manufacturer

 Problem

•  Track customer website journey

 Challenge

•  Hadoop security limitations

•  “Everybody has BI tools” –  No Java or Map reduce skills

 Results

•  Identifies 66% of visitors, up from 33% –  Enhances propensity models

–  Improved marketing campaigns

•  Batch scoring every 2-3 months à near real time

ASTER DATABASE

DISCOVERY PLATFORM

DATA PLATFORM

HADOOP

Business Analysts

Data Scientists

Query Grid

19

 Telecommunications

 Problem

•  What are the steps to canceling service? How many steps until they cancel?

•  What step is the customer in right now?

 Challenge

•  Analyzing 20+ data event streams

 Results

•  Call center 3600 customer dashboard –  Less churn and termination fees

•  3-6 month projects now done in 3 days

ASTER DATABASE

DISCOVERY PLATFORM

DATA PLATFORM

HADOOP

Business Analysts


TERADATA DATABASE

Business Analysts

Qu

ery

G

rid

Query Grid

20

 eCommerce

 Problem

•  Improving website search for online customers

 Challenge

•  Using Hadoop for text mining of EDW data

•  Bring analytic results back into EDW

 Results

•  Allow analysts to do the things they are best at on their preferred platform

ASTER DATABASE

DISCOVERY PLATFORM

DATA PLATFORM

HADOOP

Online Customers


TERADATA DATABASE

Business Analysts

Qu

ery

G

rid

Query Grid

21

 Financial Systems Provider

 Problem

•  Containment of employee travel costs

 Challenge

•  How to correlate phone calls, travel, and Webex use by employee?

 Results

•  Training classes for high travel low Webex use employees drops travel costs

•  Also condition based maintenance analysis of installed machines

ASTER DATABASE

DISCOVERY PLATFORM

DATA PLATFORM

HADOOP

Business Analysts


TERADATA DATABASE

Data Scientist

QueryGrid

Query Grid

22

 Electronics Manufacturer

 Problem

•  Yield management, manufacturing equipment fault detection

•  Slow identification of root cause and fix

 Challenge

•  Huge sensor data volume à Hadoop

•  Difficult to get Hadoop reports to users

 Results

•  Protoype à production in 2015 –  Unit level traceability

–  Quality root cause analysis

DATA PLATFORM

HADOOP


TERADATA DATABASE

Engineers, Plant workers

Finance, & Analysts

Programmers

Query Grid

23

•  QueryGrid provides bridges between cultures and systems

•  QueryGrid expands choices on data placement

•  Marrying new data with the data warehouse generates biz value

•  Understanding event sequences leads to killer apps

•  High speed parallel data exchange enables business innovations

•  Conclusion: Enterprise analytics at scale requires an integrated information ecosystem.

Recurring Themes

24 24

The End

25

QueryGrid Connectivity Usage

Industry Teradata Aster Hadoop Usage

Vehicle Manufacturer X X Bridging cultures

Communications Provider X X Analytic Workflow

Financial Services X X Compliance/Security

Travel Services X X Parallel Streams

Computer Manufacturer X X Precision Views

Telecommunications X X X Massive Discovery Lab

eCommerce Provider X X Website Search

Financial Systems Provider X X X Travel versus WebEx

Electronics Manufacturer X X Process Control

26

Teradata Unified Data Architecture: QueryGrid Layer

TERADATA OR ASTER DATABASE

Marketing Executives

Operational Systems

Customers & Partners

Frontline Workers

Business Analysts

Data Scientists

Engineers & Programmers

TERADATA QUERYGRID ACCESS LAYER

SQL-H SQL SQL, NOSQL

Teradata Unified Data Architecture

DATA PLATFORM

HADOOP OR TERADATA


TERADATA DATABASE

ASTER DATABASE

DISCOVERY PLATFORM

COMPUTE CLUSTERS

SAS, PYTHON, R, PERL, RUBY…

OTHER DATABASES

ORACLE, MONGODB, ETC

SQL VARIOUS

27

QueryGrid Use: BI to Hadoop

•  Business users –  No Hadoop skills

–  No programmers to help

–  Want self service query and reporting

–  And two kinds of data -  Raw and integrated

•  Solution: BI Toolsà QueryGrid –  Sometimes easy is all we need DATA

PLATFORM

HADOOP

DATA WAREHOUSE

TERADATA DATABASE

28

 Communication Provider

 Problem

•  Churn analysis of consumer journey

•  Enormous data volume

 Challenge

•  SQL not for event sequence analysis

•  Analysts use SQL, can’t write Java

 Results

•  Aster direct Hadoop access + nPath simplifies customer journey analysis

ASTER DATABASE

DISCOVERY PLATFORM

DATA PLATFORM

HADOOP

Business Analysts


TERADATA DATABASE

Business Analysts

Query Grid

Robin Bloor, PhD

§  Inexpensive (?) §  Any data §  May have metadata §  Poor performance §  Weak scheduling §  Weak data mgmt §  Security? §  Data lake

§  Expensive §  Prepared data §  Will have metadata §  Optimized performance §  Optimized scheduling §  Good data mgmt §  Secure §  Data workhorse

Hadoop vs. Data Mgmt Engine

Hadoop DBMS/EDW

Big Data Architecture - 1

Think Logical, Implement Physical

Data Access

In our view Hadoop has become strategic

As the Enterprise Staging Area for Data

And as an Enterprise File System


Upcoming Topics

www.insideanalysis.com

March: BI/ANALYTICS

April: BIG DATA

May: CLOUD


THANK YOU for your

ATTENTION!

Some images provided courtesy of Wikimedia Commons