traditional data warehousing, & business analytics market segmentation

© 2014 IBM Corporation 1

IBM DB2 Data Analytics Accelerator Technology Exploration - User Group Road Show - Detroit Columbus Cincinnati - April 08 09 10

© 2014 IBM Corporation

Traditional Data Warehousing, & Business Analytics Market Segmentation

Customers(e.g. external, Web)

Customer Service & Support(e.g. call centers, sales personnel)

CompanyManagement

Analysts(e.g. Mktg, Research)

C LevelMgt

User CommunityTrans.

Volume

Small

Very Large

Trans.Type

Complex

Simple

AvailabilityLess

Important

Critical

Numberof Users

Few

Many

FewTraditional Distributed

Market

Trans.Latency

LessImportant

Critical

© 2014 IBM Corporation3

Typical Current State of Affairs for Analytics: Execution by Department

Marketing

Sales

Finance

Operations

Customer CareIT

Executive

Management

Finance

Research & Dev.


Multiply that Across each DepartmentDevelopment

Quality Assurance

Production

Disaster Recovery

Development

Quality Assurance

Production

Disaster Recovery

Development

Quality Assurance

Production

Disaster Recovery

Development

Quality Assurance

Production

Disaster Recovery

R&DMarketing

Sales Finance


What this Looks Like in the Bigger Picture

Report

MIS SystemBudgeting

Campaign managementFinancial AnalysisSelling Platforms

Customer Profit AnalysisCRM

Scoring Rules

AnalyticalForesight

Optimized Business ProcessesOptimized Business ProcessesCustomer SupportCustomer Support

Claims ProcessingClaims Processing

UnderwritingUnderwriting

Fraud ManagementFraud Management

Sales EffectivenessSales Effectiveness

MarketingMarketing

Staging Area

Transformation Server

x/p serverData Mover

Staging Area

ODS (RDBMS)

x/p/z server

CleanseTransform

Warehouse

Enterprise Data Warehouse

(RDBMS)

x/p/z server

Batch Process

Continuous feed

Multi-Dimensional

Analysis

x/p server

Departmantal Data Marts

x/p/z server

Hourly/dailyBatch

Process

Analyze

AnalyticsServer

x/p server

Bulk

Data Mining Segmentation

PredictionStatistical Analysis

OLTPOperational

Systems

Online Queries & ReportingBA Tooling


World’s Best Companies Run DB2 for z/OS and System zUndisputed leader in total system availability, scalability, security and reliability

Availability Zero downtime - maintenance and upgrades without service disruption. Hyperswap for storage availability and GDPS for unmatched disaster recovery

Security Highest protection of business critical data and applications on Evaluation Assurance Level 5 (Common Criteria Security Certification)

Scalability Unmatched Near-linear scalability through hardware-based System zCoupling Facility for DB2 data sharing

Performance CPU reductions out-of-the-box

Integration Management integration, multi-platform integration and stack integration

Efficiency Reduced infrastructure complexity through consolidation, automation and virtualization. Savings on software and environmental costs, reduce labor, energy and developmental costs


OLTP vs. Analytics – Examples

OLTP - “Transactional”Transactional Analytics:

(Operational BA)Deep Analytics

Withdrawal from a bank account using an ATM

Approve request to increase credit line based on credit history and customer profile

Regular reporting to central bank – sum of transactions by account

Buying a book at Amazon.com Propose additional books based on similar purchases by other customers

Which books were best-sellers in Europe over the last 2 months?

Check-In for a flight at the airport

Offer an upgrade based on frequent flyer history of all passengers and available seats

Marketing campaign to sell more tickets in off-peak times

Hand-over manufactured printers to an oversea-carrier

Optimize shipping by selecting cheapest and most reliable carrier on demand

Trend of printers sold in emerging countries versus established markets.


Creating the Hybrid Data Server – PureData System for Analytics (Netezza) and System z

Transaction Processing Systems (OLTP)

Deep Analytics

DB2 z/OS: Recognized leader in

transactional workloads with security, availabilityand recoverability

PureData System for Analytics

Recognized leader in cost-effective high speed deep

analytics

Data Mart Data Mart Data Mart

Data Mart Consolidation

Best in Deep Analytics

Proven appliance leader in high speed analytic systems

Best in Consolidation

Unprecedented mixed workload flexibility and virtualization providing the most options for cost effective consolidation

Best in OLTP and Transactional Analytics

Industry recognized leader in mission critical transaction systems

Together:Destroying the myth that transactional and decision support workloads have to be on separate platforms

Transactional Analytics


IBM DB2 Analytics Accelerator for z/OSBlending zEnterprise and Netezza technologies

FastComplex queries run up

to 2000x faster while retaining single record

lookup speed

Cost SavingEliminate costly query tuning while offloading

complex query processing

ApplianceNo applications to

change, just plug it in, load the data, and gain

the value

A high performance analytics

accelerator appliance for IBM

zEnterprise, delivering dramatically

faster complex business analysis

transparently to all users.


IBM DB2 Analytics Accelerator for z/OS What is it? What is the value?

What is it? The IBM DB2 Analytics Accelerator is a dedicated workload optimized, appliance add-on to a DB2 for z/OS environment that services long-running, complex queries.

IDAADeep Analytics

z Enterprise z196, z114, EC12, BC12OLTP and Transactional Analytics

What is the value?

Breakthrough technology enabling new opportunities

Extreme performance for complex analytics (aka Train of Thought Analysis)

Improve price/performance for analytic workloads

Minimize the need to create data marts for performance

Transparent to DB2 applications and users

Highly secure environment for sensitive data analysis

DB2 for z/OS

Versions Supported

IDAA V2 - DB2 for z/OS V9 & V10

IDAA V3 - DB2 for z/OS V10 & V11

IDAA V4 (GA 11/29) - DB2 for z/OS V10 & V11


DB2 Analytics Accelerator V3 Features

High Performance Storage Saver• Significantly reduces the cost for storage resources• Option to store data only once in the accelerator

Incremental Update • Data changes are propagated to the accelerator as they happen• Uses change data capture technology• Extends the accelerator use to reporting on operational data

New optimization• Tables or partitions refresh much faster and less resources intensive• Optimized unloading data from DB2

High Capacity• Capacity has been extended to 1.28 PB for a single Accelerator

New functions• More queries eligible for acceleration

PureData System for Analytics N200111

Better decisions through lower latency of data

Dramatic improvement in scale and growth opportunities

Lowering the cost of analytic computing

Lowering the cost of historical data


More Query Acceleration

E n a b l i n g n e w u s e c a s e s

Static SQL

DB2 Version 11 Support added

Multi-row fetch from local applications

EBCDIC & Unicode in same DB2 system & accelerator

Incremental Update• Greatly improved scalability• Better performance

Improved performance for large result sets

HPSS• Archive to multiple accelerators• Better access control for

archived partitions

Extend WLM support to local applications

Richer system scope monitoring

Report prospective CPU cost & Elapsed time savings

Separation of duties for accelerator system administration operations

EnhancedCapabilities

Automatic workload balancing over multiple accelerators

New RTS ‘last-changed-at’ timestamp

Automated NZKit installation

HPSS

• Built-in restore

• Protection for image copies of archived partitions

Profile controlled special registers

Improved continuous operations for Incremental Update

ImprovedTransparency

DB2 Analytics Accelerator V4 FeaturesNEW


Deep DB2 Integration within zEnterprise

DataManager

BufferManager

IRLMLog

Manager

IBMDB2

AnalyticsAccelerator

Applications DBA Tools, z/OS Console, ...

. .

.

Operational Interfaces(e.g. DB2 Commands)

Application Interfaces(standard SQL dialects)

z/OS on System z


DB2 for z/OS

Superior availabilityreliability, security,

Workload management

Superior performance on analytic queries


Large Insurance Company Adding value by Accelerating the Delivery of Business Reporting

Customer Quote: “we had this up and running in days with queries that ran over 1000 times faster”

Initial Load Performance400 GB Loaded in 29 Minutes570 Million Rows (Actual: Loaded 800 GB to 1.3 TB per hour)

Extreme Query Acceleration - 1908x faster2 Hours 39 minutes to 5 Seconds

CPU Utilization Reduction35% to ~0%

IBM DB2 Analytics Accelerator (PureData System for Analytics 1000-12)

Production ready - 1 person, 2 days

Table Acceleration Setup in 2 Hours - DB2 “Add Accelerator” - Choose a Table for “Acceleration” - Load the Table (DB2 Loads Data to the Accelerator) - Knowledge Transfer - Query Comparisons

With Accelerated Time to Value

Times Faster

QueryTotal Rows Reviewed

Total Qualifying

Rows

Total Rows

Returned Hours Sec(s) Hours Sec(s)

Query 1 591,941,065 2,813,571 853,320 2:39 9,540 0.0 5 1,908

Query 2 591,941,065 2,813,571 585,780 2:16 8,220 0.0 5 1,644

Query 3 813,343,052 8,260,214 274 1:16 4,560 0.0 6 760

Query 4 283,105,125 2,813,571 601,197 1:08 4,080 0.0 5 816

Query 5 591,941,089 3,422,765 508 0:57 4,080 0.0 70 58

Query 6 813,343,052 4,290,648 165 0:53 3,180 0.0 6 530

Query 7 591,941,065 361,521 58,236 0:51 3,120 0.0 4 780

Query 8 813,343,052 3,425,292 724 0:44 2,640 0.0 2 1,320Query 9 813,343,052 4,130,107 137 0:42 2,520 0.1 193 13

DB2 Only DB2 with

IDAA

Load times may vary based other workload running on z/OS


IBM DB2 Analytics Accelerator Product Components

10Gb

Primary

Backup

CLIENT

Data Studio Foundation

DB2 Analytics Accelerator

Admin Plug-in

zEnterprise

Data Warehouse applicationDB2 for z/OS enabled for IBM


IBM DB2 Analytics Acelerator


Technology

Users/Applications

Network

OSA-Express3/4/5

10 GbE


Query routing analysis Values for CURRENT QUERY ACCELERATION

DB2 for z/OS and

IBM DB2 Analytics Accelerator OLTP-like query

OLTP-like query

Light ODS-query

Light ODS-query

Heavy BI Query

Heavy BI Query

Light BI Query

Light BI Query

DB2 Native ProcessingDB2 Native Processing

Use

r con

trol a

nd

DB

2 h

eu

ristic Optimized processing

for BI Workload

Value Description

NONE No query is routed to the accelerator

ENABLE A query is routed to the accelerator if it satisfies the acceleration criteria including the cost and heuristics criteria. Otherwise it is executed in DB2.

If there is an accelerator failure while running the query, or the accelerator returns an error, DB2 will return a negative SQL Code to the application

ENABLE WITH FAILBACK

A query is routed to the accelerator if it satisfies the acceleration criteria including the cost and heuristics criteria. Otherwise it is executed in DB2.

Under certain conditions the query will run on DB2 after it fails in the accelerator. In particular any negative SQL code will cause failback to DB2 during PREPARE or first OPEN. No failback is possible after a successful OPEN of a query

ELIGIBLE A query is routed to the accelerator if it satisfies the acceleration criteria irrespective of the cost and heuristics criteria. Otherwise it is executed in DB2

ALL A query is routed to the accelerator, if it cannot execute the query fails and a negative return code is passed back to the application

Single and unique system for mixed query workloads

Dynamic decision for most efficient execution platform

New special register QUERY ACCELERATION

New heuristic in DB2 optimizer


Routing Criteria

The entire query can be accelerated, i.e. the unit of acceleration is a whole query

– Individual query blocks are no longer units of acceleration– The whole query will either run in DB2 or in the accelerator

The associated cursor is not defined as a scrollable or a rowset cursor

The query is defined as read-only

The query is dynamic (V4 will support static)

The query is a SELECT statement.

The private protocol is not in effect.

Routing to Accelerator is considered more efficient for performance than to execute the query in DB2 mainline

– The decision is based on some heuristic rules

A query can be routed to Accelerator if is one of these types:


Heuristic Routing Criteria – not just based on “elapsed time” … DB2 Optimizer uses a set of rules to determine whether a given query is better off being

executed in DB2 core engine or routed to the accelerator, such as:– In general, typical OLTP access path patterns are not routed to the accelerator

e.g. Equal unique access, One fetch access

– If none of these: WHERE, GROUP BY, ORDER BY, aggregate functions is specified (i.e. all rows are to be returned), the query is not routed

– Threshold specified by the DB2 Profile (1) mechanism:

• If all the tables referred in the query are “small”, the query is not routed

ACCEL_TABLE_THRESHOLD determines total table cardinality for a query

The default value is 1,000,000

• If a “large” result set is expected, the query is not routed

ACCEL_RESULTSIZE_THRESHOLD (number of rows) determines what is a “large” result set.

The default value is -1, which means that this check is ignored

• If estimated total cost for a query is treated as “short running”, the query is not routed

ACCEL_TOTALCOST_THRESHOLD determines estimated total cost for a query

The default value is 5,000 (value is in milliseconds = 5 seconds)

• Recommendation: Use default values. Change only after rigorous testing!

(1) http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/index.jsp?topic=/com.ibm.db2z10.doc.perf/src/tpc/db2z_profiles.htm


Query Off-load Applicability Accelerator is based on Netezza which supports rich set of SQL and data types

– BI tools such as Cognos has run on Netezza for years and will run on Accelerator as well

Due to very large number of query types and SQL functions not all of them could be processed in V3. Key restrictions include:

– No static SQL (Lifted in V4)– Not all DB2 functions,

No Mathematical functions such as SIN, COS, TAN.

No advanced string functions such as HEX, POSITION, LOCATE, LEFT, OVERLAY

No advanced OLAP functions such as RANK, ROLLUP, CUBE– No User Defined Functions– No correlated table expressions or recursive correlated table expressions– No correlated subquery in the SELECT list – Not UTF-16 and MIXED/DBCS EBCDIC– No multiple encoding schemes in the same statement (fixed in V4)– Not all DB2 special registers: CURRENT PATH, SERVER, SQLID, SCHEMA,

APPLICATION ENCODING SCHEME– Not all DB2 data types: LOBs, ROWID, XML, DECFLOAT, BINARY

None of these restrictions is a design problem, IBM plans to lift them in future releases based on customer feedback and needs


Query Execution Process Flow

DB2 for z/OS

Optimizer

IDA

A D

RD

A R

equestor


Application

Application

Interface

Queries executed with DB2 Analytics Accelerator

Queries executed without DB2 Analytics Accelerator

Query execution run-time for queries that cannot be or should

not be off-loaded to IDAA

SPU

CPU FPGA

Memory

SPU

CPU FPGA

Memory

SPU

CPU FPGA

Memory

SPU

CPU FPGA

Memory

SM

P H

ost

Queries executed with value of “ALL” may receive a SQL Error Code if the query cannot run on the accelerator

Heartbeat (DB2 Analytics Accelerator availability and performance indicators)


Accelerator Administrative Stored Procedures ACCEL_ADD_ACCELERATOR ACCEL_TEST_CONNECTION ACCEL_REMOVE_ACCELERATOR ACCEL_UPDATE_CREDENTIALS ACCEL_ADD_TABLES ACCEL_ALTER_TABLES ACCEL_REMOVE_TABLES ACCEL_GET_TABLES_INFO ACCEL_GET_TABLES_DETAILS ACCEL_LOAD_TABLES ACCEL_SET_TABLES_ACCELERATION ACCEL_SET_TABLES_REPLICATION ACCEL_CONTROL_ACCELERATOR ACCEL_UPDATE_SOFTWARE ACCEL_ARCHIVE_TABLES ACCEL_GET_QUERIES ACCEL_GET_QUERY_DETAILS ACCEL_GET_QUERY_EXPLAIN

* For a list of the description of each stored procedure refer to the IDAA Stored Procedure Reference Manual


The Key to the Speed

FPGA Core CPU Core

Uncompress

FROM

Project

SELECT

Restrict,Visibility

WHERE

Complex ∑Joins, Aggs, etc.

GROUP BY

select DISTRICT, PRODUCTGRP, sum(NRX)from MTHLY_RX_TERR_DATAwhere MONTH = '20091201'and MARKET = 509123and SPECIALTY = 'GASTRO'

select DISTRICT, PRODUCTGRP, sum(NRX)from MTHLY_RX_TERR_DATAwhere MONTH = '20091201'and MARKET = 509123and SPECIALTY = 'GASTRO'

Slice of tableMTHLY_RX_TERR_DATA(compressed)

Slice of tableMTHLY_RX_TERR_DATA(compressed)

where MONTH = '20091201'and MARKET = 509123and SPECIALTY = 'GASTRO'

where MONTH = '20091201'and MARKET = 509123and SPECIALTY = 'GASTRO'

sum(NRX)sum(NRX)

select DISTRICT, PRODUCTGRP, sum(NRX)

select DISTRICT, PRODUCTGRP, sum(NRX)


IBM DB2 Analytics Accelerator

Table Definition and Deployment

IDAA StudioIDAA AdministrativeStored Procedures

PureData Catalog

IBM Data Studio Client DB2 for z/OS IDAA

The tables need to be defined and deployed to IDAA before data is loaded and queries sent to it for processing Definition: identifying tables for which queries need to be accelerated Deployment: making tables known to DB2, i.e. storing table metadata in the DB2 and PureData catalog

IDAA Studio guides you through the process of defining and deploying tables, as wall as invoking other administrative tasks

IDAA Stored Procedures implement and execute various administrative operations such as table deployment, load and update, and serve as the primary administrative interface to IDAA from the outside world including IDAA Studio

DB2 Catalog


Synchronization Options with IBM DB2 Analytics Accelerator

Synchronization options Use cases, characteristics and requirements

Full Table Refresh

The entire content of a database table is refreshed for accelerator processing

Existing ETL process replaces entire table

Multiple sources or complex transformations

Smaller, un-partitioned tables

Reporting based on consistent snapshot

Table Partition Refresh

For a partitioned database table, selected partitions can be refreshed for accelerator processing

Optimization for partitioned warehouse tables, typically appending changes “at the end”

More efficient than full table refresh for larger tables

Reporting based on consistent snapshot

Changed partitions only through real-time statistics (starting with IDAA V3 for DB2 V10)

Incremental Update (starting w/IDAA V3 for DB2 V10)

Log-based capturing of changes and propagation to IBM DB2 Analytics Accelerator with low latency (typically few minutes)

Scattered updates after “bulk” load

Reporting on continuously updated data (e.g., an ODS), considering most recent changes

More efficient for smaller updates than full table refresh


DB2 for z/OS Accelerator

Accelerator Data Load

Accelerator Studio

Ac

ce

lera

tor A

dm

inis

trativ

e

Sto

red

Pro

ce

du

res

.

.

.

.

.

.

.

.

.

Table A

Part 1

Part 2

Part m

Table C

Table B

Table D

Part 1

Part 2

Part 3

Unload USS Pipe

Unload

Unload

USS Pipe

USS Pipe

CPU FPGA

Memory

CPU FPGA

Memory

CPU FPGA

Memory

CPU FPGA

Memory

Co

ord

inato

r

• 1 TB / h – can vary, depending on CPU resources, table partitioning, …• Update on table partition level, concurrent queries allowed during load• Unload in DB2 internal format, single translation by accelerator


Incremental Update – IDAA V3 for DB2 V10 Changes in data warehouse tables typically

driven by replication or an other update process

– Corrections after a bulk-ETL-load of a data warehouse table

– Continuously changing data (e.g. trickle-feed updates from a transactional system to the warehouse or ODS)

Reporting and analysis based on most recent data

May be combined with a full/partition table refresh DB2 z/OS Query OptimizerDB2 z/OS Query Optimizer

ChangesDB2 for z/OS databaseDB2 for z/OS database

DB2 native processingDB2 native processing

AcceleratorprocessingAcceleratorprocessing

Operational Analytics, Reports, OLAP, …Operational Analytics, Reports, OLAP, …

ContinuousQuery

Processing

ContinuousQuery

Processing

Incremental Update

Rep

licat

ion

Rep

licat

ion

Ap

plic

atio

nA

pp

licat

ion Incremental update

can be configured per table

Initial full table load is required


DB2 Analytics Accelerator (Host Node)DB2 Analytics Accelerator (Host Node)

Incremental Update Details

DB2 for z/OSDB2 for z/OS

insert

delete

update

CaptureAgent

CaptureAgent

AcceleratorDatabase

AcceleratorDatabase

Apply Agenton NPS host

(Receives log events)

Apply Agenton NPS host

(Receives log events)

Accelerator SERVERAccelerator SERVER

(private network)

Cataloginformation

Cataloginformation

Accelerator Stored Procedures

ACCEL_SET_TABLES_REPLICATIONACCEL_CONTROL_ACCELERATOR

...

Accelerator Stored Procedures

ACCEL_SET_TABLES_REPLICATIONACCEL_CONTROL_ACCELERATOR

...

JCLJCL

Automation code(creates data sources,

subscriptions, etc.)

Automation code(creates data sources,

subscriptions, etc.)

ControllerController

AcceleratorStudio

AcceleratorStudio

IFI LogReads


User InterfaceIncremental update UI elements only visible if it has been enabled on the DB2 subsystem via IBM DB2 Analytics Accelerator configuration console

Start / stop replication process (per subsystem-accelerator pair)

Enable / disable replication (per table)

Trace collection

Information on replication latency and events


Save Over 95% of Host Disk Space for Historical Data

1Q1Q

2Q2Q

3Q3Q

4Q4Q

1Q1Q

2Q2Q

3Q3Q

4Q4Q

1Q1Q

2Q2Q

3Q3Q

4Q4Q

1Q1Q

2Q2Q

3Q3Q

4Q4Q

1Q1Q

2Q2Q

3Q3Q

4Q4Q

1Q1Q

2Q2Q

3Q3Q

4Q4Q

1Q1Q

2Q2Q

3Q3Q

4Q4Q

YearYear Year -7Year -7Year -2Year -2 Year -3Year -3 Year -4Year -4 Year -5Year -5Year -1Year -1

Historical DataHistorical Data

Current DataCurrent Data

One Quarter = 3.57% of 7 years of data

One Month = 1.19% of 7 years of data

One month = 2.78% of 3 years of data

One Quarter = 3.57% of 7 years of data

One Month = 1.19% of 7 years of data

One month = 2.78% of 3 years of data


Accelerator

High Performance Storage Saver Reducing the cost of high speed storage

Time-partitioned tables where:– only the recent partitions are used in a transactional context (frequent

data changes, short running queries) – the entire table is used for analytics (data intensive, complex queries).

High Performance Storage Saver’s “Archive” Process:– Data is loaded into Accelerator if not already loaded– Automatically takes Image Copy of Each Partition to be Archived– Automatically Remove data from DB2 archived tablespace partitions– DBA starts archived partitions as read-only

Part #1

Query from Application

Or

No longer present on DB2 Storage

Part #1

Part #2

Part #3

Part #4

Part #5

Part #6

Part #7

DB2

Active Archive


High Performance Storage SaverReducing the cost of high speed storage

DB2Table A

AcceleratorTable A

Tables can be resident on:1. DB2 Only2. DB2 and Accelerator3. Archive to Accelerator

Applications

Managed by zPARMs

Controlled by Special Registers:

CURRENT QUERY ACCELERATION

CURRENT GET_ACCEL_ARCHIVE

DB2Table A

SQL

Store historic data on the Accelerator only

When data no longer requires updating, reclaim

the DB2 storage

Best for OLTPBest for OLTP

High Speed High Speed Indexed queriesIndexed queries

Active OnlyActive Only Archive OnlyArchive Only Active & ArchiveActive & Archive Mixed WorkloadMixed Workload

Mixed WorkloadMixed Workload

AcceleratorTable AActive & Archive

DB2 Table A

Active


Key Elements of the Implementation Approach

• Older partitions are moved to IDAA and their data no longer exists in DB2• DB2 is still solely responsible for the recovery and maintains all backups (copy images)

• The most recent partitions exist in both DB2 and IDAA

• They are synchronized by existing means• Partition refresh or incremental update (replication-based propagation of changes)

• Having the most recent partition in both, DB2 and IDAA, provides IDAA-driven performance acceleration for analytical queries that access most recent partitions only

• The data move process is encapsulated in a stored procedure• The stored procedure can be invoked directly or via IDAA Studio

• The SQL statements do not change• The fact that some partitions have been moved to IDAA is transparent

• By default, queries access only the data from the most recent partitions• The queries can be executed in DB2 or IDAA based on the standard routing criteria

• If all the data need to be accessed, one of the following mechanisms is used:• Setting a zparm which activates the 'all data' scope for the DB2 subsystem/data sharing group.

This way, none of the applications need to be changed (but this setting has global impact).

• Setting a special register “CURRENT GET_ACCEL_ARCHIVE”, which allows switching between the 'all data' scope and the 'most recent data' scope at any time. This way the application can use both scopes within the same execution at choose scope at SQL statement level.


Work Load Management - Usage scenarios

• Workload Isolation: Ensure that the workload of one DB2 subsystem doesn’t monopolize the resources of a shared accelerator. A development subsystem, attached to the same accelerator as a production subsystem, should not be able to drain all accelerator resources.

• Query Prioritization:More important queries should be executed before and faster than less important queries that are sent from the same DB2 subsystem against the accelerator.


IBM DB2 Analytics Accelerator

Instrumentation

34


IBM DB2 Analytics Accelerator Studio


Moving Partitions with HPSS via IDAA Studio


DB2 Command – DISPLAY ACCEL DETAIL


Product Feature Description Benefit

OM/PE Reporting Utilization of Accelerator (under/over) ROI/Performance

Real time and short term monitoring

Detailed perf reports (batch and online) for Accelerator and apps routed to the Accelerator – takes advantage of Extended Insight

ROI/Performance

Assessment Determine if workload can be accelerated ROI

Query Monitor Cost comparison Measure perf of query with & w/out Accelerator ROI/Performance

Capture workload Capture SQL workload (or queries) for designated time – can be used as input to OQWT for tuning

ROI/Performance

Separate Queries Isolate non-accelerated queries from workload ROI/Performance

OQWT Best fit object Determine best set of objects for query acceleration (both Static and dynamic SQL) ROI/Performance

Best fit queries Which queries are eligible, non-eligible or can be rewritten ROI/Performance

What-if analysis Expert recommendations with estimated cost savings (no actual execution) ROI/Performance

Loader Load non DB2 data Ability to load non-DB2 data directly to Accelerator Hybrid envROI for System z

P.i.T. load Load to a specific point-in-time to fit business needs ROI/reporting

Operational data Ability to load to Accelerator w/out impact to applications Avail/Perf/ROI

Admin/OC AcceleratorAdmin/Mgt

Complete mgt of Accelerator and accelerated objects via familiar tool w/ ISPF Usability

Refresh data? Ability to use RUNSTATS to help determine if refresh of Accelerator data is recommended

ROI/Performance

Chg Mgt Automated reload of data to Accelerator after chg Avail/Usability


Product Documentation Links Information Center for IBM DB2 Analytics Accelerator for z/OS V3.1

– http://publib.boulder.ibm.com/infocenter/dzichelp/v2r2/topic/com.ibm.datatools.aqt.doc/welcome/topics/idaa_start.html

Quick Start Guide– http://publibfp.dhe.ibm.com/epubs/pdf/h1269820.pdf

Installation Guide– http://publibfp.dhe.ibm.com/epubs/pdf/h1269830.pdf

Stored Procedure Reference– http://publibfp.dhe.ibm.com/epubs/pdf/h1269840.pdf

Analytics Accelerator Studio Users Guide– http://publibfp.dhe.ibm.com/epubs/pdf/h1269850.pdf

Getting Started Guide– http://publibfp.dhe.ibm.com/epubs/pdf/h1269860.pdf

Program Directory– http://publibfp.dhe.ibm.com/epubs/pdf/i1950060.pdf

Redbook – Optimizing DB2 Queries with IBM DB2 Analytics Accelerator for z/OS– http://www.redbooks.ibm.com/redpieces/abstracts/sg248005.html?Open

Redbook – Hybrid Analytics Solution using IBM DB2 Analytics Accelerator for z/OS V3.1– http://www.redbooks.ibm.com/redpieces/abstracts/sg248151.html?Open

traditional data warehousing, & business analytics market segmentation

Documents