best practices using db2 performance structures … optimally...db2 for z/os best practices using...

© 2013 IBM Corporation

DB2 for z/OS

Best Practices

Using DB2 Performance

Structures Optimally

Sheryl M. Larsen

IBM WW DB2 for z/OS Evangelist

[email protected]

mailto:[email protected]


DB2 for z/OS Best Practices IBM®

2

Sheryl M. Larsen [email protected]

Sheryl Larsen is an internationally

recognized researcher, consultant and

lecturer, specializing in DB2 and is

known for her extensive expertise in

SQL. She co-authored a book, DB2

Answers, Osborne-McGraw-Hill, 1999.

She now works for IBM, but this material

was developed before joining IBM and is

the culmination of 25 years as an

external consultant specializing in

superhuman efforts to tune customer

workloads.

Sheryl has over 25 years experience

in DB2, has published articles, white

papers, webcasts: bmc.com, ca.com,

softbase.com,

ibm.developerworks.com

Her new title is IBM’s World Wide DB2

for z/OS Evangelist. Other titles include:

President of the Midwest Database

Users Group,

IDUG Hall of Fame Speaker,

IDUG Hall of Fame Volunteer, and

Northern Illinois University Computer

Science Alumni Council Board Member



3

Disclaimer/Trademarks

© Copyright IBM Corporation 2013. All rights reserved.

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM

Corp. THE INFORMATION CONTAINED IN THIS DOCUMENT HAS NOT BEEN SUBMITTED TO ANY FORMAL IBM TEST AND IS DISTRIBUTED AS IS.

THE USE OF THIS INFORMATION OR THE IMPLEMENTATION OF ANY OF THESE TECHNIQUES IS A CUSTOMER RESPONSIBILITY AND

DEPENDS ON THE CUSTOMER’S ABILITY TO EVALUATE AND INTEGRATE THEM INTO THE CUSTOMER’S OPERATIONAL ENVIRONMENT.

WHILE IBM MAY HAVE REVIEWED EACH ITEM FOR ACCURACY IN A SPECIFIC SITUATION, THERE IS NO GUARANTEE THAT THE SAME OR

SIMILAR RESULTS WILL BE OBTAINED ELSEWHERE. ANYONE ATTEMPTING TO ADAPT THESE TECHNIQUES TO THEIR OWN

ENVIRONMENTS DO SO AT THEIR OWN RISK.

ANY PERFORMANCE DATA CONTAINED IN THIS DOCUMENT WERE DETERMINED IN VARIOUS CONTROLLED LABORATORY ENVIRONMENTS

AND ARE FOR REFERENCE PURPOSES ONLY. CUSTOMERS SHOULD NOT ADAPT THESE PERFORMANCE NUMBERS TO THEIR OWN

ENVIRONMENTS AS SYSTEM PERFORMANCE STANDARDS. THE RESULTS THAT MAY BE OBTAINED IN OTHER OPERATING ENVIRONMENTS

MAY VARY SIGNIFICANTLY. USERS OF THIS DOCUMENT SHOULD VERIFY THE APPLICABLE DATA FOR THEIR SPECIFIC ENVIRONMENT.

Trademarks

IBM, the IBM logo, ibm.com, and DB2 are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other

product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at “Copyright and

trademark information” at www.ibm.com/legal/copytrade.shtml.

http://www.ibm.com/legal/copytrade.shtml





4

All the Performance Decisions

Use the Right Stuff for the Job!

Schema Design

Partitioning

Clustering

Base Table Indexes

Index on Expression

Materialize Query Tables (MQTs)

MQT Indexes

zIIPs

zAAPs

Accelerated Query Tables (AQTs)

Appliances

Performance

Structures



5

Optimal Schema & Index & MQT Design

Schema

Indexes MQT

MQT

MQT

MQT

MQT Schema

Indexes

M

Q

T

AQT

Index

On

Expression



6

Designing Optimal Performance

Structures

for One Query or a Workload



Type 2 Index

Leaf Page

o o o o o o

Leaf Page

o o o o o o

Leaf Page

o o o o o o

Non-Leaf Page

O O O

Leaf Page

o o o o o o

Leaf Page

o o o o o o

Non-Leaf Page

O O

Leaf Page

o o o o o o

Leaf Page

o o o o o o

Leaf Page

o o o o o o

Leaf Page

o o o o o o

Non-Leaf Page

O O O O

Root Page

O O O

7

1. Equal predicates first

2. Sequencing columns next

3. Predicates most restrictive to least restrictive

4. All remaining SELECT Columns

One Query at a Time SELECT C7, C8, C12, C2, C3

WHERE C4 = ‘L’

AND C5 = 99

AND C6 = :hv

AND C1 IN (:a,:b,:c)

AND C2 NOT LIKE :hv

AND C3 BETWEEN :hvstart

AND :hvend

AND C9 = (CASE expression)

ORDER BY C1.C2.C3

C4.C5.C6.C1.C2.C3.C9.C7.C8.C12



8

Materialized Query Tables

Can be almost ANY valid SELECT statement

CREATE TABLE MQT1 AS SELECT… use your imagination .. ;

– System-maintained

– SQL REFRESH TABLE option

– Deletes all rows, refills table, updates catalog

– User-maintained (via triggers, replication, batch updates, etc.)

– REFRESH DEFERRED

– Via LOAD, INSERT, UPDATE, DELETE

– REFRESH IMMEDIATE – LUW only

Can be fast!



9

MQT…. To Build or not to Build

Generic MQT strategies

– HUGE MQT with many indexes

– Many small MQTs with few indexes

Optimal MQT strategy

– Most popular queries as MQTs

– Common tables

– Common predicate sets

– Common data translations

– Indexes on the popular ones to allow access to subset of MQT



10

Recursive SQL Syntax

WITH COMMON1 (PART, SUBPART, N) AS

(SELECT PART, SUBPART, 1

FROM LEGO

WHERE PART = ‘L1’

UNION ALL

SELECT L.PART, L.SUBPART, N+1

FROM COMMON1 T1, LEGO L

WHERE T1.SUBPART = L.PART

AND N < 10 )

SELECT PART, SUBPART, N

FROM COMMON1

Starting

Rows

Join to get

Children’s Children

Part Subpart N L1 S1 1

L1 S2 1

S1 S7 2

S2 S5 2

S2 S6 2

S5 S9 3

Starting

Rows

Available on

DB2 LUW

since V5,

z/OS V8



11

Recursive Processing Code

Temp 1

Temp 2

Part Subpart L1 S1

L1 S2

S1 S7

S2 S5

S2 S6

S5 S9

Starting

Rows

New Rows

are appended

Base

Table

New Rows

replace

Temp1

INSERT INTO TEMP1

SELECT PART, SUBPART

FROM LEGO

WHERE PART = ‘L1’

INSERT INTO TEMP2

SELECT L.PART, L.SUBPART

FROM TEMP1 T1, LEGO L

WHERE T1.SUBPART = L.PART

INSERT INTO RESULT


FROM TEMP1



12

Emulating Recursive SQL

Starting

Rows

INSERT INTO TEMP1


FROM LEGO

WHERE PART = ‘L1’;

INSERT INTO RESULT


FROM TEMP2;

Add Rows

to

Result

INSERT INTO RESULT


FROM TEMP1;

Add Rows

to

Result

INSERT INTO TEMP2

SELECT L.PART, L.SUBPART

FROM TEMP1 T1, LEGO L

WHERE T1.SUBPART = L.PART;

Get

New

Rows

IF No New

Rows, END Else

Clear

Temp1 DELETE

FROM TEMP1;

New Rows

replace

Temp1

INSERT INTO TEMP1

SELECT *

FROM TEMP2;

Clear

Temp2

DELETE

FROM TEMP2;



13

Expensive Recursive Syntax

WITH ORG_CHART(BOSS, MANAGES, N) AS

(SELECT BOSS, MANAGES, 1

FROM ORG_TABLE

WHERE BOSS= ‘GINNI’

UNION ALL

SELECT T1.BOSS, T2.MANAGES, N+1

FROM ORG_CHART T2, ORG_TABLE T1

WHERE T2.MANAGES = T2.BOSS

AND N < 500,000)

SELECT BOSS, MANAGES, N

FROM ORG_CHART

Generated

at Execution

Ginni



14

MQT

MQT INDEX:

BOSS

CREATE TABLE ORG_LIST AS ( WITH ORG_CHART(BOSS, MANAGES, N) AS


FROM ORG_TABLE

WHERE BOSS= ‘GINNI’

UNION ALL

SELECT T1.BOSS, T2. MANAGES, N+1


WHERE T2. MANAGES = T2.BOSS )


FROM ORG_CHART)

DATA INITIALLY DEFERRED

REFRESH DEFERRED

MAINTAINED BY SYSTEM

ENABLE QUERY OPTMIZATION;



15

Query Speed Up

BOSS EMP N

JOHN ADAM 501

BETH JIM 634

BETH SUE 634

BETH MARK 635

BETH SAM 635

WITH ORG_CHART (BOSS, MANAGES N) AS


FROM ORG_TABLE

WHERE BOSS= ‘BETH’

UNION ALL

SELECT T1.BOSS, T2.MANAGES, N+1


WHERE T2.MANAGES = T2.BOSS

AND N < 500,000)


FROM ORG_CHART

Accessed at

Execution

SELECT * FROM ORG_LIST

WHERE BOSS = ‘BETH’

ORG_LIST



16

Designing Indexes

For a Workload



17

Table/Column Usage

Tran c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11

tran1

q1 O1

G1

=T4.C2

O2

G2

=:hv

O3

G3

>800

S

=T2.C1 S

S

S

BTW

2and 7

SUM by

C1.C2.C

3

AVG by

C1.C2.C3

MAX by

C1.C2.C3

q2 s s w =22 o1 =:hv o2

q3 =T3.C1 s s s s s o1 s =T3.C11 =T3.C1

tran2

q1 O1 O2

O3 =:hv >:hv

q2 s s =:hv s s s s s O1 IN(list) LIKE :hv

O1 = Order By First Position

G1 = Group By First Position

S = SELECT list

W = Local or Join WHERE/ON or

Specify actual local/join filter Specify

Group by Aggregates as: Aggr BY

Column(s)



18

Final Ranking

tran1 100

4200 q1 O1

G1

W

O2

G2

w

O3

G3

w

S

w S

S

S

w

10 q2 s s w w o1 w o2

9900 q3 w o1 s w w

Weighting Based on

Importance to the

Business

Frequency * Weight

4200

4200

4200

4200 4200

4200

10 10 10 10 10 10 10

9900

4200

4200

4200

4200

4200

4200

4200 4200 4200

9900 9900 9900 9900

Weight

tran2 3

0

0



19

Determining Weight

Copy the frequency report

Sort by “business priorities”

– Use Work Load Manager Goals for guidance

– Use Business Unit Owner Priorities

Remove the top 30%

Sort by Frequency

Pull of the top number

Multiply by 10

Now you have the frequency number to beat

Let’s say 5,500 is top frequent

Then 55,000 is the

goal

Pulls out most critical



20

Grand Total

By Table, by Operation

Total

Activity

c1 c2 c3 c4 c5 c6 c7 c8

O1 55980

G1 4790

O2 2220

O6 70

W 3920

O2 55980

G2 4790

W 400

O3 55980

G3 4790

W 3

S 660

O1 8740

S 8740

w 22090

O2 3940

S 480

S 4000

W 7640

S 8390

W 2020

Use this information to justify making changes to

current index design



21

Where does an Accelerated Query Table Live?

IBM DB2 Analytics Accelerator

A high performance appliance that integrates Netezza

technology with zEnterprise technology, to deliver

dramatically faster business analysis

Speeds up complex queries up to 2000x

No indexes or performance rules to follow

Lowers the cost of long term storage

Minimizes data latency

Improves security and reduces risk

Complements existing investments

AQT



22 22

IBM DB2 Analytics Accelerator

Blending zEnterprise and Netezza

technologies to deliver unparalleled,

combined workload performance for

complex analytic business needs.

• Transparent to the application

• Inherits all zEnterprise DB2 attributes

• No need to create or maintain indices

• Eliminate query tuning

• Fast deployment and time-to-value



OLTP Transactions

Operational analytics

Real time data ingestion

High concurrency DB2 Native

Processing

Standard reports

Complex queries

Historical queries

OLAP

IBM

DB

2 A

na

lytic

s

Acce

lera

tor

Advanced analytics*

DB2/Accelerator Sweet Spots Do Not Overlap

DB2 11 and Big Insights

provide this extended

capability



24

AQTs live in the IBM DB2 Analytics Accelerator S-BladeTM

Intel Quad-Core

IBM BladeCenter Blade

Dual-Core FPGA*

*FPGA - Field Programmable Gate Array



25

The Key to the Speed

FPGA Core CPU Core

Uncompress Project Restrict,

Visibility

Complex ∑

Joins, Aggs, etc.

select DISTRICT,

PRODUCTGRP,

sum(NRX)

from MTHLY_RX_TERR_DATA

where MONTH = '20091201'

and MARKET = 509123

and SPECIALTY = 'GASTRO‘

group by DISTRICT, PRODUCTGRP

Slice of table

MTHLY_RX_TERR_DATA

(compressed)

where MONTH = '20091201'

and MARKET = 509123

and SPECIALTY = 'GASTRO'

sum(NRX)

group by DISTRICT,

PRODUCTGRP

select DISTRICT,

PRODUCTGRP

Field Programmable Gate Array



26

Accelerated Query Tables for Unlimited Searches



27

Current Schema & Index

Schema

Index Other Relational

DBMS

RI discards



28

Index Design Improvements

Every index except for primary and clustering was not servicing the web requests

Limiting factor is number of new indexes due to expensive Q-rep

Minimal index environment for B-Page searches would need 50 indexes!

Optimal B-Page is unlimited search



29

B-Page Web Inquiry Search

29

Attribute 1

________ Attribute 2

_________ Attribute 3

________ Attribute 4

________

Attribute 5

________ Date 1

___________ Date 2

___________



30

Accelerated Query Table Advantage

© Sheryl M. Larsen, Inc. 2000-2013

30

Schema

Indexes

Indexxxxxx

AQT

Index

On

Expression



DB2 for z/OS and

IBM DB2 Analytics Accelerator OLTP Transactions

Real time data ingestion

High concurrency

Standard reports

OLAP

Complex queries

31

DB2 11 The Foundation for Business Critical Analytics

• DB2 11 CPU savings

benefit query workloads

with or without IDAA

Operational analytics

DB2 Native

Processing



32

Optimal ODS Search - RI Index Only & AQT

© Sheryl M. Larsen, Inc. 2000-2013

32

Schema

Index Other Relational

DBMS

RI discards

AQT



33

Optimal Schema & Index & MQT Design

Schema

Indexes MQT

MQT

MQT

MQT

MQT

M

Q

T

AQT

Index

On

Expression



34

Best Practice Sites Have tools and staff to assist in query development and index

design

– IBM’s Data Studio is FREE!

– Index design tools are extra $$,$$$

Justify IBM DB2 Analytic Accelerator for speeding up complex

queries and allowing unlimited long searches

Have production environments emulated in test 100% (this

may not be 100% physical)



35

Thanks for listening!

Use the Right Stuff for the Job!

Base Table Indexes

Index on Expression

Materialize Query Tables (MQTs)

MQT Indexes

Accelerated Query Tables (AQTs)

Appliance

Performance

Structures



IBM Big Data Platform

Stream Computing

Hadoop Systems

Data Warehouse

*

Enterprise Data Sources *

A Single Solution to:

• ACCESS, COMBINE &

MANAGE a relevant

mix of information

• TIMELY & SECURE

ACCESS for more

accurate answers

Big Data & Analytics on IBM zEnterprise Improves Experience Increases Adoption Drives greater insight

Business System / OLTP

Business Critical Analytics

Data Warehousing

Query Acceleration

best practices using db2 performance structures … optimally...db2 for z/os best practices using...

Documents