unity: because the sum is greater than the parts

The Briefing Room

Twitter Tag: #briefr

The Briefing Room

Welcome

Host: Eric Kavanagh

[email protected]


The Briefing Room

!   Reveal the essential characteristics of enterprise software, good and bad

!   Provide a forum for detailed analysis of today’s innovative technologies

!   Give vendors a chance to explain their product to savvy analysts

!   Allow audience members to pose serious questions... and get answers!

Mission


The Briefing Room

December: Innovators

January: Big Data

February: Analytics

March: Data in Motion


The Briefing Room

Innovators

!   Necessity is the mother of invention, and the need for unified access is driving innovation

!   Big Data, Cloud and Social Media are the horses in front of the carriage

!   Users want easier, faster and better system management Ph

oto

by G

eorg

e P.

Lan

dow

(w

ww

.vic

tori

anw

eb.o

rg)

This image may be used without prior permission for any scholarly or educational purpose


The Briefing Room

Analyst: Krish Krishnan

Mr. Krishnan is a recognized expert in the strategy, architecture and implementation of high performance data warehousing solutions and unstructured data. He co-authored “Building the Unstructured Data Warehouse” along with Bill Inmon. He also has written eBooks, over 150 articles, viewpoints and case studies in Big Data, Business Intelligence, Data Warehousing, Data Warehouse Appliances and High Performance Architectures. Mr. Krishnan currently is promoting the next generation of data warehousing, focusing on Big Data, Semantic Technologies, Crowdsourcing, Analytics, and Platform Engineering. He leads the Data Warehouse Appliance and Architectures Expert Channel at BEyeNetwork.com and publishes with www.beyenetwork.com\kkrishnan.


The Briefing Room

!   Teradata is known for its analytic data solutions with a focus on integrated data warehousing, big data analytics and business applications.

!   It offers a broad suite of technology platforms and solutions, and a wide range of data management applications and data mining capabilities.

!   Teradata is now offering an integrated portfolio that turns a multi-system environment into an Analytical Ecosystem.

Teradata


The Briefing Room

Imad Birouty is the Program Marketing Manager for the Teradata Analytical Ecosystem, which includes features such as Teradata Unity and Teradata’s Dual Active Solution. Imad is also responsible for Teradata’s Data Mart Consolidation program, the Value of Integrated Data, and Total Cost of Ownership. Prior to this, Imad led the Product Management team responsible for the NCR Platform including the processing nodes, BYNET Interconnect, Server Management, and Operating Systems. He set product strategy and direction and was responsible for seven major platform releases spanning 8 years.

Imad Birouty

Imad Birouty Program Marketing Manager

TERADATA UNITY AND THE UNIFIED DATA ARCHITECTURE

TERADATA UNITY

11 12/4/12

Teradata Unified Data Architecture

AUDIO & VIDEO IMAGES TEXT WEB & SOCIAL MACHINE LOGS CRM SCM ERP

INTEGRATED DATA WAREHOUSE

Teradata Analytical Ecosystem

12 12/4/12




DATA LAB


13 12/4/12



DATA MARTS

TEST/ DEV

ANALYTICAL ARCHIVE


DATA LAB


14 12/4/12



DATA MARTS

TEST/ DEV

ANALYTICAL ARCHIVE


DATA LAB

INDEPENDENT DATA MART


15 12/4/12



DUAL SYSTEMS

DATA MARTS

TEST/ DEV

ANALYTICAL ARCHIVE


DATA LAB



16 12/4/12



DUAL SYSTEMS

DATA MARTS

TEST/ DEV

ANALYTICAL ARCHIVE


VIEWPOINT SUPPORT


UNITY


DATA LAB

17 12/4/12



DUAL SYSTEMS

DATA MARTS

TEST/ DEV

ANALYTICAL ARCHIVE


DATA MINING BUSINESS INTELLIGENCE APPLICATIONS VIEWPOINT SUPPORT



DATA LAB

Business Analysts Marketing Front-Line Workers

Operational Systems Customers / Partners Executives

18 12/4/12

• Meet service level goals >  The right data, in the right place, at the right time, for the

right users >  Track application readiness

• Resource optimization > Match application profiles to the right Teradata platform >  Availability, performance, cost

Teradata’s Analytical Ecosystem Benefits

•  Investment protection >  Seamless application portability between

Teradata platforms >  Platform repurposing

• One source support for Teradata products and services

19 12/4/12

Teradata Unity

Teradata Unity

Unity Director

Query routing and database synchronization between Teradata Systems

Unity Loader

High volume, intelligent data loading to multiple Teradata systems

Unity Ecosystem Manager End-to-end monitoring across data warehouse components

Unity Data Mover Data movement between Teradata systems

Integrated portfolio that turns a multi-system environment into an orchestrated Analytical Ecosystem

20 12/4/12

Unity Director Provides Query Management Reads

•  Two methods for Read queries > User session routing >  Individual query routing

21 12/4/12



Controlled Routing (Named, Preferred)

Great for session and workload control across systems, especially when high availability failover is required

22 12/4/12



Controlled Routing (Named, Preferred)

Great for session and workload control across systems, especially when high availability failover is required

Automatic Routing (Automatic)

Great when systems contain different data and/or for fast setup where session control isn’t required

23 12/4/12

Unity Director Provides Query Management Writes

•  Three methods for Write operations >  Default, Create Preferred, Create Balanced >  Can be used in combination

• Writes to all systems where objects reside

• Great for keeping systems in sync

Default (writes)

24 12/4/12





Default (writes)

Create Preferred (writes)

• Creates to only one system • Subsequent writes to created objects on just one system, such as temporary tables

• Prevents wasted resources on other systems

25 12/4/12





Default (writes)

Create Preferred (writes)

Create Balanced (writes)

• Creates to only one system • Subsequent writes to created objects on just one system, such as temporary tables

• Prevents wasted resources on other systems

• Creates to only one system • Designated system for creates alternates among systems for different sessions based on a round-robin algorithm

26 12/4/12

Unity Loader

•  Bulk loads sent to all participating systems; updates are asynchronous •  More than two sites are supported with additional Unity Loader and

Director servers

Unity Loader

Site 1

27 12/4/12

Unity Loader

Unity Loader

Unity Loader


Director servers

Unity Loader

Site 1

28 12/4/12

Unity Loader

Unity Loader

Unity Loader


Director servers

Unity Loader

Site 1 ü

ü

ü

29 12/4/12

Unity Loader

Unity Loader

Unity Loader


Director servers

Unity Loader

Unity Loader

Site 1

Site 2

ü

ü

ü

30 12/4/12

Unity Loader

Unity Loader

Unity Loader

Unity Loader

Unity Loader


Director servers

Unity Loader

Unity Loader

Site 1

Site 2

ü

ü

ü

31 12/4/12

Unity Loader

Unity Loader

Unity Loader

Unity Loader

Unity Loader


Director servers

Unity Loader

Unity Loader

Site 1

Site 2

ü

ü

ü

TERADATA UNIFIED DATA ARCHITECTURE

Copyright © 2011 Teradata Corporation. 33


DISCOVERY PLATFORM

CAPTURE | STORE | REFINE


UNIFIED DATA ARCHITECTURE

Big Data Analytics

Big Data Management


Business Analyst

Discovery Platform

Discovery

Discovery Platform Data Sources

Structured Data

Multi-Structured Data

Non relational Data

OLTP DBMS’s

Users

Data Scientist

SQL

MapReduce

Statistical Functions

Discovery Tools

•  Structured and multi-structured data

•  Doesn’t require extensive data modeling

•  Doesn’t balance the books

•  Data completeness can be good enough

•  No stringent SLA’s

•  Fraud patterns •  Customer behavior •  Digital marketing

optimization •  Machine data/process

optimization


Hadoop Use Cases Validated with Customers

•  Land/source operational data -  Only one extract from source system

•  History or long term storage -  Low cost storage

•  Preprocess data -  Sessionize data, remove XML tags

•  Transformations -  Structured and semi-structured

•  Exploration -  Investigate value of new data sources

•  Batch scoring •  Single subject reporting

Many define as Analytics

36 12/4/12



DISCOVERY PLATFORM



LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS

Engineers

Data Scientists Business Analysts Marketing Front-Line Workers


37 12/4/12





DISCOVERY PLATFORM

Engineers



38 12/4/12





DISCOVERY PLATFORM

Engineers



39 12/4/12

INTRODUCING TERADATA ASTER BIG ANALYTICS APPLIANCE 3H Industry’s First Big Analytics Appliance with Aster and Hadoop in the same cabinet

Unified System Management and Analytics

Unified Solution for Big Data Storage, Management, and Analytics

Available NOW


Aster SQL-H™ A Business User’s Bridge to Analyze Hadoop Data

Aster SQL-H Gives Analysts and Data Scientists a Better Way to Analyze Data Stored in Hadoop •  Allow standard ANSI SQL access to

Hadoop data

•  Leverage existing BI tool and enable self service

•  Enable 50+ prebuilt SQL-MapReduce Apps and IDE

Hadoop Layer: HDFS

Pig

Hive

Hadoop MR

Aster: SQL-H

HCatalog

Dat

a

Dat

a Fi

ltering

NEW

41 12/4/12



DISCOVERY PLATFORM



LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS VIEWPOINT SUPPORT

Engineers



42 12/4/12

Common Management Console for Aster, Teradata and Apache

Hadoop Aster Specific Portlets •  Aster Node

Monitoring •  Aster Completed

Processes

Trend/ Visualization Portlets •  Capacity

Heat Map •  Metrics Graph •  Metrics Analysis

Query Portlets •  Query Monitor

Admin Portlets •  Teradata System •  Roles Manager

Other Portlets •  System Health •  Canary queries •  Aster Data

Alerting

Teradata Viewpoint Integration Easier, Faster, and Better System Management

Available now with Teradata and Aster. Hadoop availability early 2013

43 12/4/12

•  Customers that want to improve supportability ensure that Teradata Vital Infrastructure is activated in their production environment

•  Teradata Vital Infrastructure is embedded proactive support software available on each Teradata platform

Improve Supportability with Teradata Vital Infrastructure

Regularly Collects Asset Data

Constantly Monitors Events

Identifies Risk Data Patterns/Trends

Detects Fault Alerts

Records Fault Events

Automatic Incident Reports Are Created

Alert Notifications Are Sent and Tracked

62–70% of Incidents Discovered Through TVI

44 12/4/12



DUAL SYSTEMS

DATA MARTS

TEST/ DEV

ANALYTICAL ARCHIVE


DATA MINING BUSINESS INTELLIGENCE APPLICATIONS VIEWPOINT SUPPORT



DATA LAB

Business Analysts Marketing Front-Line Workers


45 12/4/12



DUAL SYSTEMS

DATA MARTS

TEST/ DEV

ANALYTICAL ARCHIVE

LANGUAGES MATH & STATS DATA MINING BUSINESS INTELLIGENCE APPLICATIONS VIEWPOINT SUPPORT


DISCOVERY PLATFORM


DATA LAB


Engineers



46 12/4/12

Questions and Answers

Thank You!


The Briefing Room

Analyst: Krish Krishnan

Perceptions & Questions

S

Next Generation Data Warehouse

Designing & Building Scalable Data Warehouse Infrastructures BriefR

December 4th 2012

Corporate Decision Support Data


Semi-‐Structured

Data

Data Distribution Today


Data Workloads & Processing


Transac1onal Systems ODS

Enterprise Datawarehouse

Datamarts & Analy1cal Databases





Reports

Dashboards

Analy1c Models

Other Applica1ons

Data Transforma1on

Workload Challenge

S  Unknowns S  Do not know what data we are

geCng S  Do not know formats or

volumes of data


S  Requirements S  Assigning the appropriate systems

and processes to manage workloads S  Creates an interchangeable

infrastructure S  Support Homogenous /

Heterogeneous plaIorms S  Need to process data in a

reasonable 1me S  Need a fault tolerant architecture as

large chunks of data cannot be reprocessed if and when a failure occurs

S  Need automa1c resource management

S  Issues S  Long running jobs S  Missed SLA’s S  Performance & Response Issues S  Scalability S  Fault Tolerance

Workload Isolation


Semi-‐Structured

Data

BeneGits

S  Requirements S  Assigning the appropriate systems and processes to manage

workloads S  Creates an interchangeable infrastructure S  Support Homogenous / Heterogeneous plaIorms S  Need to process data in a reasonable 1me S  Need a fault tolerant architecture as large chunks of data cannot

be reprocessed if and when a failure occurs S  Need automa1c resource management


Questions


S  What are the key drivers behind the “unity” architecture?

S  What is the biggest benefit of this applica1on?

S  Will Teradata evolve this fabric in the next genera1ons?

S  What releases of the ecosystem are supported?

S  Will this include integra1on of other Teradata management tools in the future?

S  How can Systems Administrators learn this architecture? What addi1onal skills are needed?

Thank You


Krish Krishnan [email protected] Twi]er Handle: @datagenius


The Briefing Room


The Briefing Room

Upcoming Topics

This month: Innovators

January: Big Data

February: Analytics

2013 Editorial Calendar www.insideanalysis.com


The Briefing Room

Thank You for Your

Attention

unity: because the sum is greater than the parts

Technology

integrated data warehousing

unstructured data warehouse

data warehouse appliances

generation of data warehousing

value of integrated

analytic data solutions

data mining capabilities

briefrthe briefing room