a hybrid technology platform for increasing the speed of operational analytics

46
© 2009 IBM Corporation A Hybrid Technology Platform for Increasing the Speed of Operational Analytics Ed Lynch – Executive Client Technical Professional Data Warehousing and Business Analytics for System z 10 October 2012

Upload: ibmgovernmentca

Post on 18-Nov-2014

706 views

Category:

Technology


0 download

DESCRIPTION

Track 1 d a hybrid technology platform for increasing the speed of operational analytics - ed lynch

TRANSCRIPT

Page 1: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

Ed Lynch – Executive Client Technical Professional

Data Warehousing and Business Analytics for System z10 October 2012

Page 2: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Speaker Biography

Ed Lynch is an Executive Client Technical Professional specializing in IBM’s System z Data Warehousing, Business Analytics, and Information Integration software products. Ed’s twenty-eight year career with IBM has spanned many areas of IBM, and has always involved IBM's Information Management (IM) products. His previous roles have included DB2 for z/OS Development and Management, DB2 Technical Marketing, Development and Delivery of IM Product Education, a Principal of Information Integration Design and Implementation Consulting Services, and DB2 Tools & Information Integration Technical Sales.

Currently, Ed is the lead Technical Specialist for North America’s System z Data Warehousing and Business Analytics, and Information Integration Software Technical Sales team.

Ed has worked extensively with DB2 across the various operating system platforms, InfoSphere Data Replication, InfoSphere Classic Replication Server, DB2 Analytics Accelerator, DB2 DataPropagator, InfoSphere Federation Server, InfoSphere Classic Federation Server, DB2 Connect, IBM Information Server, and IBM’s Business Analytics solutions. In his current role, he frequently works with product development in identifying and prioritizing product requirements, and developing product strategy. He also provides software technical sales support and works extensively with customers to create architectures using these products.

[email protected]

1-972-561-9975

Page 3: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Abstract

With the wealth of data available today, organizations are no longer willing to relegate information to the back office. Modern organizations are demanding access to information. However, it is not enough to capture information, users must be quickly able to sift through these massive amounts of data, extract information and transform it into actionable knowledge. Systems today are enabling organizations to anticipate risk, identify threats, assess readiness, and match the risk assessment to the resources required to address them; all at the time of decision. They use a platform that provides the ability to react to changes decisively, based upon the facts of the situation, not in hours or days- but at the moment of opportunity. They optimize decisions based upon current weather conditions, past threats and behaviors and current resource availability to assure a successful operation.This session will review the architecture and benefits of a hybrid system of MPP and SMP technologies enabling the merging of fit for purpose and mixed workload capabilities into a single system. See how this hybrid system facilitates both transaction-oriented applications and analytics into a single platform for operational analytics. Find out why these enhancements are the next logical steps in creating a highly optimized environment, both in price and performance, that is designed to meet the wide range of analytic workloads that today's organizations need to accommodate.

Page 4: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation44

DB2 Analytics Accelerator V3Further extending the features

Blending System z and Netezza

technologies to deliver unparalleled,

mixed workload performance for

complex analytic business needs.

More insight from your data

• Unprecedented response times for

“right-time” analysis

• Complex queries in seconds rather

than hours

• Transparent to the application

• Inherits all System z DB2 attributes

• No need to create or maintain indices

• Eliminate query tuning

• Fast deployment and time-to-value

Page 5: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation5

DB2 Analytics AcceleratorDB2 Analytics AcceleratorTrainTrain--ofof--thought Analyticsthought Analytics

FASTComplex queries run

up to 2000x faster

while retaining single

record lookup speed

ApplianceNo applications to

change, just plug it

in, load the data,

and gain the value

Cost SavingEliminate costly

query tuning while

offloading complex

query processing

Page 6: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation6

Introducing Introducing

DB2 Analytics Accelerator V3DB2 Analytics Accelerator V3Reducing the Cost of High Speed AnalyticsReducing the Cost of High Speed Analytics

Improve Productivity

Eliminate query tuning

Eliminate table indexing

Minimize storage admin

Consolidate

Reduced complexity

Reduced software costs

Reduced hardware costs

Lower Host Costs

Reduce storage costs

Offload query processing

Defer system upgrades

Page 7: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Fast Time to Value

� IBM DB2 Analytics Accelerator

� Production ready - 1 person, 2 days

� Table Acceleration Setup 7 2 Hours

– DB2 “Add Accelerator”

– Choose a Table for “Acceleration”

– Load the Table (DB2 copy to Netezza)

– Knowledge Transfer

– Query Comparisons

� Initial Load Performance 7

�400 GB “Loaded” in 29 Min

570 million rows (Loads of 800GB to 1.3TB/Hr)

� Actual Query Acceleration 7 1908x faster

�2 Hours 39 Minutes to 5 Seconds

� CPU Utilization Reduction

�35% to ~0%

Actual customer results, October 2011

Page 8: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

8 12 October 2012

Performance & Savings

Accelerating decisions to the speed of business

Queries run faster

• Save CPU resources

• People time

• New Business

opportunities

Actual customer results, October 2011

Times

Faster

Query

Total

Rows

Reviewed

Total

Rows

Returned Hours Sec(s) Hours Sec(s)

Query 1 2,813,571 853,320 2:39 9,540 0.0 5 1,908

Query 2 2,813,571 585,780 2:16 8,220 0.0 5 1,644

Query 3 8,260,214 274 1:16 4,560 0.0 6 760

Query 4 2,813,571 601,197 1:08 4,080 0.0 5 816

Query 5 3,422,765 508 0:57 4,080 0.0 70 58Query 6 4,290,648 165 0:53 3,180 0.0 6 530

Query 7 361,521 58,236 0:51 3,120 0.0 4 780

Query 8 3,425.29 724 0:44 2,640 0.0 2 1,320

Query 9 4,130,107 137 0:42 2,520 0.1 193 13

DB2 Only

DB2 with

IDAA

DB2 Analytics Accelerator: “we had this up and

running in days with queries that ran over 1000

times faster”

DB2 Analytics Accelerator: “we expect ROI in

less than 4 months”

Advance to 32 minute mark for DB2

Analytics Accelerator section of keynote

Page 9: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

IBM DB2 Analytics Accelerator V3 Product Components

9

10Gb

OSA-Express3

10 GbE

Primary

Backup

CLIENT

Data Studio Foundation

DB2 Analytics Accelerator

Admin Plug-in

zEnterprise

Data Warehouse applicationDB2 for z/OS enabled for IBM

DB2 Analytics Accelerator

IBM DB2 Analytics Acelerator

NetezzaTechnology

Users/

Applications

Network

Page 10: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation10

Deep DB2 Integration within zEnterprise

Data

Manager

Buffer

ManagerIRLM

Log

Manager

IBM

DB2

Analytics

Accelerator

Applications DBA Tools, z/OS Console, ...

. . .

Operational Interfaces(e.g. DB2 Commands)

Application Interfaces(standard SQL dialects)

z/OS onSystem z

Netezza

DB2 for z/OS

Superior availability

reliability, security,

Workload management

Superior

performance on

analytic queries

Page 11: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

SMP Hosts

Snippet BladesTM

(S-Blades, SPUs)

Disk Enclosures

Accelerator ServerSQL Compiler, Query Plan, Optimize,Administration2 front/end hosts, IBM 3650M3 or 3850X5clustered active-passive

2 Nehalem-EP Quad-core 2.4GHz per host

Processor & streaming DB logic

High-performance database engine streaming joins, aggregations, sorts, etc.

Slice of User Data

Swap and Mirror partitionsHigh speed data streamingHigh compression rate EXP3000 JBOD Enclosures12 x 3.5” 1TB, 7200RPM, SAS (3Gb/s)max 116MB/s (200-500MB/s compressed data)

e.g. 1000-12: 8 enclosures → 96 HDDs(32/128 TB)

Accelerator powered by Netezza 1000TM

Appliance

Page 12: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

IBM BladeCenter Server

Netezza DB Accelerator

S-Blade™ Components

Intel Quad-Core

Dual-Core FPGA

8 Cores/Blade

8 FPGA Processors/Blade

Page 13: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Eliminating the I/O Bottleneck

Move the SQL to the hardwareE to where the data lives

“Just send

the Answer,

not Raw

Data”

Page 14: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

The Key to the Speed

FPGA

CoreCPU Core

UncompressProjectRestrict,

Visibility

Complex ∑∑∑∑Joins, Aggs, etc.

select DISTRICT,PRODUCTGRP, sum(NRX)from MTHLY_RX_TERR_DATAwhere MONTH = '20091201'and MARKET = 509123and SPECIALTY = 'GASTRO'

Slice of tableMTHLY_RX_TERR_DATA

(compressed)

Slice of tableMTHLY_RX_TERR_DATA

(compressed)

where MONTH = '20091201'and MARKET = 509123and SPECIALTY = 'GASTRO'

where MONTH = '20091201'and MARKET = 509123and SPECIALTY = 'GASTRO'

sum(NRX)sum(NRX) select DISTRICT,PRODUCTGRP,sum(NRX)

select DISTRICT,PRODUCTGRP,sum(NRX)

Zone MapZone Map

Page 15: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Bringing Netezza AMPPTM Architecture to DB2 for z/OS

Advanced Analytics

DBA

Legacy Reporting

BI

FPGA

Memory

CPU

FPGA

Memory

CPU

FPGA

Memory

CPU

SMPHost

Disk

EnclosuresS-Blades™Network

Fabric

IBM DB2 Analytics Accelerator

DB2 for z/OS

AMPP = Asymmetric MassivelyParallel Processing

Page 16: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Query Execution Process Flow

DB2 for z/OS

Optimizer

Accele

rato

r DR

DA

Requesto

r

DB2 Analytics Accelerator

Application

Application

Interface

Queries executed with DB2 Analytics Accelerator

Queries executed without DB2 Analytics Accelerator

Query execution run-time for

queries that cannot be or should

not be off-loaded to Accelerator

SPU

CPU FPGA

Memory

SPU

CPU FPGA

Memory

SPU

CPU FPGA

Memory

SPU

CPU FPGA

Memory

SM

P H

ost

Page 17: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Workload-Optimized Query Execution

DB2 for z/OS andIBM DB2 Analytics Accelerator

OLTP-like queryOLTP-like query

Light ODS-query

Light ODS-query

Heavy BI QueryHeavy BI Query

Light BI QueryLight BI Query

DB2 Native Processing

DB2 Native Processing

User c

ontro

l and D

B2 h

eu

ristic

• Single and unique system

for mixed query workloads

• Dynamic decision for most

efficient execution platform

• New special register

QUERY ACCELERATION

– NONE

– ENABLE

– ENABLE WITH FAILBACK

• New heuristic in DB2

optimizer

Optimized processing for BI Workload

Page 18: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

DB2 for z/OS Accelerator

Accelerator Data Load

Accelerator

Studio

Ac

ce

lera

tor A

dm

inis

trativ

e S

tore

d

Pro

ce

du

res .

.

.

.

.

.

.

.

.

Table A

Part 1

Part 2

Part m

Table C

Table B

Table D

Part 1

Part 2

Part 3

Unload USS Pipe

Unload

Unload

USS Pipe

USS Pipe

CPU FPGA

Memory

CPU FPGA

Memory

CPU FPGA

Memory

CPU FPGA

Memory

Co

ord

inato

r

• 1 TB / h – can vary, depending on CPU resources, table partitioning, 7• Update on table partition level, concurrent queries allowed during load• V2.1 & V3 unload in DB2 internal format, single translation by accelerator

Page 19: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation19

DB2 Analytics Accelerator V3Lowering the Costs of Trusted Analytics

• zEnterprise EC12 Support

Version 3 will support the zEnterprise

EC12, z196 and z114 System z

platforms

•Query Prioritization

Brings System z workload

management down to the individual

query being routed to the Accelerator

•High Capacity

Support has been extended to include

the entire Netezza 1000 line (1.28 PB)

•UNLOAD Lite

Reduces z/OS MIPS consumption, by

moving the preparation off System z.

What’s New?

•High Performance Storage

Saver

Store a DB2 table or partition of data

solely on the Accelerator. Removes

the requirement for the data to be

replicated on both DB2 and the

Accelerator

• Incremental Update

Enables tables within the Accelerator

to be continually updated throughout

the day.

Page 20: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Build a System z Trusted Analytic SystemBuild a System z Trusted Analytic SystemReduce the cost of host storage for historical data by 95%!Reduce the cost of host storage for historical data by 95%!

HistoricalMost data in an analytic

system is historical and not

subject to change. Most

data can be in a Storage

Saver and maintain trusted

performance and security

Low Latency DataTables and partitions that

require updating will be

able to be updated by

incremental update, table

load or partition load

High PerformanceAll aggregate queries run

at the same high speed as

any accelerator supported

query

Page 21: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

21

High Performance Storage SaverReducing the cost of high speed storage

DB2Table A

AcceleratorTable A

AcceleratorTable A

Tables can be resident on:1. DB2 Only

2. DB2 and Accelerator

3. Accelerator Only

Special Registers control behavior

� CURRENT QUERY ACCELERATION

� CURRENT GET_ACCEL_ARCHIVE

Managed by zParms

DB2Table A

SQL

Store historic data on the Accelerator only

When data no longer

requires updating, reclaim

the DB2 storage

High speed High speed

indexed indexed

lookups, best lookups, best

for OLTP for OLTP

type queriestype queries

High speed High speed

aggregate aggregate

lookups, best for lookups, best for

complex DSS type complex DSS type

queriesqueries

Mixed workload type queriesMixed workload type queries

Applications

Page 22: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Save Over 95% of Host Disk Space for Historical Data

1Q

2Q

3Q

4Q

1Q

2Q

3Q

4Q

1Q

2Q

3Q

4Q

1Q

2Q

3Q

4Q

1Q

2Q

3Q

4Q

1Q

2Q

3Q

4Q

1Q

2Q

3Q

4Q

Year Year -7Year -2 Year -3 Year -4 Year -5Year -1

Historical Data

Current Data

One Quarter = 3.57% of 7 years of data

One Month = 1.12% of 7 years of data

One month = 2.78% of 3 years of data

Page 23: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation23

High Performance Storage SaverReducing the cost of high speed storage

� Time-partitioned tables where:

– only the recent partitions are used in a transactional context (frequent data

changes, short running queries)

– the entire table is used for analytics (data intensive, complex queries).

� DB2 partitions are deleted after the High Performance Storage Saver are created

on the accelerator

DB2

#1

Accelerator

#1

Query from

Application

Or

Accelerator

#2

Accelerator

#3

Accelerator

#4

Accelerator

#5

Accelerator

#6

Accelerator

#7

No longer present on DB2 Storage

Page 24: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

24

The Evolution of a High Performance Storage SaverHigh Speed Access to Historical Data

DB2Table A

AcceleratorTable A

AcceleratorTable A

DB2Table A

Backup Backup

Table / Data

Creation

Accelerator

Load / Update /IU

Accelerator

Only

Archive

Only

Page 25: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

25

• Stored on both DB2 and Accelerator

storage

• Mixed query workload with transactions,

single record queries and record updates

with deep analytics, multifaceted,

reporting and complex queries.

• Full table, full partition update, Incremental

update from DB2 data

• Same high speed query access

transparently through DB2

Storage options to match data needsOptimized in both price and performance for differing workloads

• Only stored on Accelerator storage (Less

Cost)

• Optimized performance for

deep analytics, multifaceted, reporting

and complex queries

• Only full table update or full partition

update from backup

• Same high speed query access

transparently through DB2

Dual Disk StoreSingle Disk Store

Database Resident PartitionsHigh Performance Storage Saver

FunctionalityCost The right mix of cost and functionality

Page 26: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation26

The zEnterprise Hybrid SolutionMixed Workloads for Next Generation Business Analytics

Operational

ApplicationsAnalytic

Applications

Mixed Workload

Applications

Transaction Processing Data warehousing Operational BI

Shared Everything DB Shared Nothing DB Hybrid DB

High volume business transactions and batch reporting running concurrently

Low volume complex queries context switching

High volume business transactions and batch reporting running

concurrently with complex queries

Page 27: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation27

Incremental Update

ELT or ETLELT or ETLTable or Table or

Partition UpdatePartition Update

Data Data

ReplicationReplicationIncremental Incremental

UpdateUpdate

OLTP OLTP

ApplicationApplicationData Data

WarehouseWarehouseDB2 Analytics DB2 Analytics

AcceleratorAccelerator

Synchronizing data to lower data latency

from days to minutes/seconds

Page 28: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Option 1: Full Table Refresh

� Changes in data warehouse tables typically

driven by scheduled (nightly or more

frequently) ETL process

� Data used for complex reporting based on

consistent and validated content (e.g., weekly

transaction reporting to the central bank)

� Multiple sources or complex transformations

prevent propagation of incremental changes

� Full table refresh triggered through DB2 stored

procedure (scheduled, integrated into ETL

process or through GUI)DB2 z/OS Query OptimizerDB2 z/OS Query Optimizer

DB2 for z/OS databaseDB2 for z/OS database

DB2 native

processing

DB2 native

processingAccelerator

processing

Accelerator

processing

Operational Analytics, Reports, OLAP, 7Operational Analytics, Reports, OLAP, 7

Continuous

Query

Processing

Continuous

Query

Processing

Full table refresh

Changes / Replacement

ET

L P

roce

ss

ET

L P

roce

ss� Queries may continue

during full table refresh

for accelerator

Page 29: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Option 2: Table Partition Refresh

� Changes in data warehouse table typically driven by “delta” ETL process (considering only changes in source tables compared to previous runs) or by more frequent changes to most recent data

� Optimization of Option 1 when target data warehouse table is partitioned and most recent updates are only applied to the latest partition

DB2 z/OS Query OptimizerDB2 z/OS Query Optimizer

ChangesDB2 for z/OS databaseDB2 for z/OS database

ET

L P

rocess

ET

L P

rocess

DB2 native

processing

DB2 native

processingAccelerator

processing

Accelerator

processing

Operational Analytics, Reports, OLAP, 7Operational Analytics, Reports, OLAP, 7

Continuous

Query

Processing

Continuous

Query

Processing

May

April

March

February

January

Partition refresh

Replic

ation

Replic

ation

� Maintains snapshot semantics for consistent reports

� Queries may continue during table partition refresh for accelerator

� Table partition refresh triggered through DB2 stored procedure (scheduled, integrated into ETL process or through GUI)

Page 30: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Option 3: Incremental Update

� Changes in data warehouse tables typically driven by replication or manual updates

– Corrections after a bulk-ETL-load of a data warehouse table

– Continuously changing data (e.g. trickle-feed updates from a transactional system to an ODS)

� Reporting and analysis based on most recent data

� May be combined with Option 1 & 2 (first table refresh and then continue with incremental updates)

DB2 z/OS Query OptimizerDB2 z/OS Query Optimizer

ChangesDB2 for z/OS databaseDB2 for z/OS database

DB2 native

processing

DB2 native

processingAccelerator

processing

Accelerator

processing

Operational Analytics, Reports, OLAP, 7Operational Analytics, Reports, OLAP, 7

Continuous

Query

Processing

Continuous

Query

Processing

Incremental Update

Rep

licati

on

Rep

licati

on

Ap

plicati

on

Ap

plicati

on

� Incremental update can be configured per database table

Page 31: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Now expandable to 960 cores and 1.28 petabytes

002 005 010 015 020 030 040 060 060 100

Cabinets 1/4 1/2 1 1 1/2 2 3 4 6 8 10

Processing Units 24 48 96 144 192 288 384 576 768 960

Capacity (TB) 8 16 32 48 64 96 128 192 256 320

Effective Capacity (TB)*

32 64 128 192 256 384 512 768 1024 1280

.......

1 10

Capacity = User Data spaceEffective Capacity = User Data Space with compression *: 4X compression assumed

Predictable, Linear Scalability throughout entire family

Low Latency, High Capacity Update

PureData System for Analytics

Page 32: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation32

Connectivity Options

Multiple DB2 systems can connect to a single Accelerator

A single DB2 system can connect to multiple Accelerators

• residing in the same LPAR

• residing in different LPARs

• residing in different CECs

• being independent (non-data sharing)

• belonging to the same data sharing group

• belonging to different data sharing groups

Multiple DB2 systems can connect to multiple Accelerators

Full flexibility for DB2 systems:

Better utilization of Accelerator resourcesBetter utilization of Accelerator resources

ScalabilityScalability

High availabilityHigh availability

DB2 DB2

DB2

DB2 DB2

The same table can be stored in the multiple Accelerators(except High Performance Storage Saver tables)

Page 33: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Analytics Accelerator Table Definition and Deployment

33

DB2 for z/OS DB2 Analytics Accelerator

IBM Data Studio Client

AcceleratorAccelerator

StudioStudio

DB2 CatalogDB2 Catalog

Accelerator Accelerator

Administrative Administrative

Stored ProceduresStored ProceduresNetezza CatalogNetezza Catalog

� The tables need to be defined and deployed to the Accelerator before data is loaded and queries sent to it for processing.

� Definition: identifying tables for which queries need to be accelerated

� Deployment: making tables known to DB2, i.e. storing table meta data in the DB2 and Netezza catalog.

� IBM DB2 Analytics Accelerator Studio guides you through the process of defining and deploying tables, as well as invoking other administrative tasks.

� IBM DB2 Analytics Accelerator Stored Procedures implement and execute various administrative operations such as table deployment, load and update, and serve as the primary administrative interface to the Accelerator from the outside world including Accelerator Studio.

Page 34: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

• All user data and temp space mirrored

• Disk failures transparent to queries and transactions

• Failed drives automatically regenerated

• Bad sectors automatically rewritten or relocated

Shielding Against Disk Failures

Primary

Mirror

Temp

Page 35: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Shielding Against S-BladeTM Failures

• S-Blade failure is automatically detected

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

S-Blades

Page 36: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Shielding Against S-BladeTM Failures

• Drives automatically reassigned to active S-Blades within a chassis

• Read-only queries (that have not returned data yet) automatically restarted

• Transactions and loads interrupted

• Loads automatically restarted from last successful checkpoint

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

S-Blades

Page 37: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

SYSPLEX

Tables of App 1

Tables of App 2

Tables of App 3

Tables of App 4

Tables of App 5

DSG Member 1 DSG Member 2App 1

App 2

App 3

App 4

App 5

Switch Switch

Short Range

Accelerator Instance 1

Disaster Recovery Option 1 – Table Loaded in One Accelerator (1 of 2)

Short Range Short Range

Short Range

Long

Range

Tables of App 1

Tables of App 2

Tables of App 3

Accelerator Instance 2

Tables of App 4

Tables of App 5

Tables of App 1

Tables of App 2

Tables of App 3

Tables Created

but Not Loaded

Page 38: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Accelerator Instance 1

SYSPLEX

Tables of App 1

Tables of App 2

Tables of App 3

Tables of App 4

Tables of App 5

DSG Member 1 DSG Member 2App 1

App 2

App 3

App 4

App 5Switch Switch

Short Range

Short Range Short Range

Short Range

Long

Range

Tables of App 4

Tables of App 5

Tables of App 1

Tables of App 2

Tables of App 3

Already CreatedMust LOAD

Tables of App 1

Tables of App 2

Tables of App 3

App 1

App 2

App 3

Disaster Recovery Option 1 – Table Loaded in One Accelerator (2 of 2)

Accelerator Instance 2

Page 39: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

SYSPLEX

Tables of App 1

Tables of App 2

Tables of App 3

Tables of App 4

Tables of App 5

DSG Member 1 DSG Member 2App 1

App 2

App 3

App 4

App 5

Switch Switch

Short Range

Disaster Recovery Option 2– Table Loaded in Two Accelerators (1 of 2)

Short Range Short Range

Short Range

Long

Range

Tables of App 4

Tables of App 5

Tables of App 4

Tables of App 5

Tables of App 1

Tables of App 2

Tables of App 3

Tables of App 1

Tables of App 2

Tables of App 3

Accelerator Instance 1 Accelerator Instance 2

Page 40: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

SYSPLEX

Tables of App 1

Tables of App 2

Tables of App 3

Tables of App 4

Tables of App 5

DSG Member 1 DSG Member 2App 1

App 2

App 3

App 4

App 5Switch Switch

Short Range

Short Range Short Range

Short Range

Long

Range

Tables of App 4

Tables of App 5

Tables of App 1

Tables of App 2

Tables of App 3

App 1

App 2

App 3

Disaster Recovery Option 2 – Table Loaded in Two Accelerators (2 of 2)

Tables of App 4

Tables of App 5

Tables of App 1

Tables of App 2

Tables of App 3

Data Already

Available

Accelerator Instance 1 Accelerator Instance 2

Page 41: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Why Both?Marrying the best of both worlds

IBM

System z

IBM

PureData N1001

Very diverse workloadVery focused workload

Capitalizing on the strengths of both platforms while driving to the most

cost effective, centralized solution - destroying the myth that transaction

and decision systems had to be on separate platforms

Mixed Workload SystemFocused Appliance

Page 42: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Flexible Integrated SystemTrue Appliance Custom Solution

• Mixed workload system z with operational transaction systems, data warehouse, operational data store, and consolidated

data marts.• Unmatched availability, security and

recoverability• Natural extension to System z to enable

pervasive analytics across the organization.

• Speed and ease of deployment and administration

Tailored to your needsA Hybrid Solution

• Appliance with a streamlined

database and HW acceleration for

performance critical functionality

• Price/performance leader

• Speed and ease of deployment and

administration

• Optimized performance for

deep analytics, multifaceted, reporting

and complex queries

Mixed Workload SystemFocused Appliance

IBM System z with

IBM DB2 Analytics Accelerator

IBM

Netezza

FlexibilitySimplicity The right mix of simplicity and flexibility

Page 43: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

43

Next StepsOpportunity

Data

Consolidation

(ISAS +

Accelerator)

Operational BI

(ISAS +

Accelerator)

New Workload

or Winback

(ISAS +

Accelerator)

Operational Reporting

(Accelerator)OR

Existing z Warehouse

Workload

Assessment

Warehouse

Collaboration

Workshop

Optional POC

Page 44: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation44

The Ultimate Consolidation Platform

Transaction Systems

(OLTP)

Data Warehousing

Business Intelligence

Predictive Analytics

z/OS: Recognized leader in mixed workloads with security,

availabilityand recoverability

Netezza: Recognized leader in cost-effective high speed deep analytics

Data Mart Data Mart Data Mart Data Mart

Data Mart Consolidation

Together:Destroying the myth that transactional and decision support

workloads have to be on separate platforms

Bringing it all together

• Better Business Response

• Reduced Costs

• More Available

• More Secure

• Reduced Data Movement

• Better Governance

• Reduced Data Latency

• Reduced Complexity

• Reduced Resources

System z PR/SMRecognized leader in mixed

virtualization and workload isolation

Page 45: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation

Learn More7

Visit the Data Warehousing &

Business Analytics Webpage

http://www.ibm.com/software/data/businessintelligence/systemz/

Page 46: A Hybrid Technology Platform for Increasing the Speed of Operational Analytics

© 2009 IBM Corporation46

Ed LynchSystem z Data Warehousing & Business Analytics

[email protected]