ibm optim query tuning offerings optimize … optim query tuning offerings optimize performance and...

© 2009 IBM CorporationJune 9, 2010

IBM Optim Query Tuning OfferingsOptimize Performance and Cut Costs

Saghi AmirsoleymaniSolution Architect Integrated Data [email protected]

Information Management

© 2009 IBM Corporation2

© Copyright IBM Corporation [current year]. All rights reserved.U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE.

IBM, the IBM logo, ibm.com, DB2 and IBM Optim Query Tuner are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml\

Disclaimer

http://www.ibm.com/legal/copytrade.shtml



Query Performance Causes Application Outage

Customer example – Application service drops from 3 seconds to 5 minutes – Limited performance management expertise – Took 3 days to identify the problem query– 5 day application outage– Loss of confidence in IT team– Time and money in analysts

Expert analysis revealed – Relevant statistics were not being collected – Wasted CPU resources due to default statistics collection– Index design was very poor – Rapid development practices drove untuned queries into production

Poor data

Wasted resourcesPoor design

Poor process



Why is Query Tuning Such a Pain?

Development– Lack of query tuning skills– Use of frameworks that generate SQL– Inadequate testing environments to drive the workload and data scale– Disconnect between application and data groups

Production– Performance problems appear without warning– Aggregate information from multiple places– Complexity of analysis requires specialized skills– Need to consider entire workloads, not just single queries



IBM Optim Query Tuner Offerings

Optimize performance– Improve customer satisfaction– Improve organizational productivity– Satisfy service level objectives– Fit workloads into maintenance

windows

Reduce costs – Increase overall system capacity– Reduce chargebacks – Simplify analysis for DBAs,

developers, designers, and others – Improve statistics collection efficiency– Reduce time to respond

Optimize performance and reduce costs



Single Query and Workload Offerings

Optim Query Tuner

Tune SQL pre-production while costs and impact are low

Optim Query Workload Tuner

Optimize workload for peak performance

Development Environment Production Environment



Expert-Enabled Query Tuning Out of the Box

Identify query candidates– DB2 catalog– Dynamic statement cache– Development Studio hot spots– Query or performance monitors

Facilitate analysis– Query formatting– Query annotation– Access path visualization and

annotation

Get expert tuning advice– Improve query design– Improve statistics quality– Improve database design



Define or select workload Execute advisors

Streamlined Analysis

Drill into advice Validate improvement



Gather High Cost Queries and Workloads

DB2

Plan table

Statement table

Catalog plan or package

Statement cache

Optim Development Studio

QMF and QMF HPO

DB2 Query Monitor

OMEGAMON XE for DB2

File, text, or exported workload

User defined category

Define or select workload



Visualize Queries and Costs to Speed Analysis

Accelerate analysis, reduce downtime– Spot human errors – Identify where filtering should occur

Easily see tables, sections, join

predicates, etc.Examine table statistics and

additional information



View Optimizer Transformations

Accelerate analysis, reduce downtime– Spot transformations which will occur



Execute Advisors

Statistics – Get recommendations on the

best statistics to capture to influence access path selection

Query– Get recommendations

regarding how to rewrite the query for better efficiency

Index – Get recommendations on

additional indexes that can reduce database scans



Visualize access path – See flow of query processing– See indexes and operations– See optimizer rationale

Assess access path stability to reduce risk of performance regression– Is the optimizer able to apply the filtering early?– Are there indexes that support an efficient path?– Do statistics allow distinction between the choices?

Analyze Access Plans



Analyze Access Plans (cont.)

Interrogate the plan table to discover access plan flags



Improve Statistics Quality and Collection

Provides advice on – Missing statistics– Conflicting statistics– Out-of-date statistics

Results – Accurate estimated costs – Better query performance– Less CPU consumption – Improved maintenance

window throughput

Generates RUNSTATS

control statements

Indicates conflicting and

missing statistics

Conflicting statistics

explanation“Half of access path PMRs could be resolved by statistics advisor before calling IBM support.”

– IBM Support



Improve Query Design

View analysis summary Highlights relevant

components of the query

Recommendation and

rationaleFilter recommendations

by severity

Guard against errors and oversights:Further constrain query, increase index utilization, and reduce data reads

Query Advisor checks for– Missing join predicate for referential

constraint– Predicates that can be rewritten as

indexable– Stage 2 predicates that can be

rewritten as stage 1 predicates



Indexing Advice to Improve Query Efficiency

Improve query efficiency– Indexing foreign keys in queries

that do not have indexes defined– Identifying index filtering and

screening– Support for index only access – Indexing to avoid sorts

Simplify use– Consolidate indexes and provide

a single recommendation – Enables what-if analysis– Provides DDL to create indexes– Run immediately or save

Test before deployment– Utilize virtual index capabilities

built into the DB2 engine



Validate Improvement Capture workloads repeatedly to see tuning

effects

Visualize changes in access plan

“I was able to test a small workload in QWT. 213 dynamic statements were run and captured from cache. Before tuning the workload was 2.49 cpu seconds and 50.97 elapsed seconds. After the advisors the workload was 0.42 cpu seconds and 6.33 elapsed seconds.”

Renee Walton, UPS



Version to Version Migration Saved

Challenge– Performance regression discovered across range of QMF workloads– Client had been using a “trick” to affect optimizer choices that was not

available in the new version– Expected 6-8 months delay plus additional staff costs – Delay in promised value of new database version

Solution – Optim Query Workload Tuner– Handled complexity of queries and workloads

• 30 table joins nested behind several layers of views• Column names changed in views• 100s of queries in multiple workloads

– Saved hours of time formatting, annotating, and analyzing queries – Recommended additional statistics and predicates for better performance – Reduced migration from months to weeks

Order of magnitude reduction in query workload analysis. All workloads as good as or better than prior version.



Workload Tuning vs. Query Tuning

Optimization decisions are based on tradeoffs– Statistics – CPU costs vs. query savings– Indexing – query speed vs resource and transaction

Workload tuning speeds up analysis– Analyzes multiple queries at once

Workload tuning consolidates and optimizes recommendation for overall workload– Statistics recommendations– Index recommendations

Speed up analysis, optimize design, and balance resource usage



Environment Capture Facilitates Collaboration

Enable environment reproduction Speed up service process



Query Tuner Architecture

Windows XP or Vista

Optim Query Tuner Client

Res

trict

ed u

se D

B2

Con

nect

DB2 for z/OS, V8 or laterwith DDF enabled

DB2 stored procedures for use with Optim Query Tuner products

Catalog tablesQT tablesProfile tables

Query Tools• Formatting• Annotating• Access plan graph• Visual plan hints• Reporting• Environment captureQuery Advice• Statistics• Query• Index• Access PathWorkload Advice• Statistics• Index• Workload control

center



Streamline Monitor and Respond Scenarios

Optim Query Tuner

DB2 Query Monitor

OMEGAMON XE



Foster cross-role collaboration

Challenge – Disconnect between application and

data group

Solution- verbose HTML file – Overview– Recommended actions– Input sql statement– Access Plan Summary

• Transformed/optimized query• Predicates• Table access• Join operation

– Table and index catalog information: table, column and index

Released in December!



Reduce Risk of Performance Regression After Migration

Run Access Plan Cost comparison report

Comparing query costs via an HTML report• Version to Version migrations of DB2

– Requires the packages are bound with EXPLAIN in the old version, and then the binds with EXPLAIN are done on the new version into new collections

• Cases where a mass-rebind is needed

Obtain Access Plan information

Filter Criteria



View analysis summary based on TOTAL_COST

Identify Problematic Package Tables

SQL statement with the highest

% cost

Prevent Problems Before They Impact the Business

Tune a single query or at a

workload level

Identify Problematic SQL statement



Validation Against a Production System

Accelerate development– Develop, test, and run queries,

procedures, and functions. Enhance productivity

– Analyze impact of changes– Enhance problem isolation

Be proactive– Leverage best practices for new

development– Test before deployment– Adapt ahead of impact – Minimize production slowdowns

Change database connection

Clone project

Perform analysis



IBM Optim Query Tuner Offerings

Optimize performance– Improve customer satisfaction– Improve organizational productivity– Satisfy service level objectives– Fit workloads into maintenance

windows

Reduce costs – Increase overall system capacity– Reduce chargebacks – Simplify analysis for DBAs,

developers, designers, and others – Improve statistics collection efficiency– Reduce time to respond

Optimize performance and reduce costs



Tuning SQL with Optim Query Tuner, Part 1: Understanding access paths

– Learn how to monitor and tune queries and workloads to improve application performance

If you are a developer, DBA, or query tuning specialist, it is critical that you understand the basics of access paths so that you can precisely tune queries and query workloads before they cause problems in your production environment. This basic understanding, coupled with the visualization and tuning advice provided by IBM® Optim® query tuning solutions, can help make you more efficient at this task. This article provides conceptual background on access paths, shows you how to read an access path graph, and walks through the access path graph to demonstrate critical information regarding access path selection. The article concludes with a sample scenario that demonstrates how to use query annotation, a feature in Optim Query Tuner that helps you with query analysis by providing vital statistical information directly in the SQL statement.

http://www.ibm.com/developerworks/data/library/techarticle/dm1006optimquerytuner1/index.html

New! Article

http://ow.ly/2078C

http://ow.ly/2078C





Resources

Integrated Data Management Community– http://www.ibm.com/developerworks/spaces/optim

IBM Optim Query Workload Tuner web page– https://www-01.ibm.com/software/data/db2imstools/db2tools/opti-

expert-zos/

SQL Tuning, Not just for hardcore DBAs anymore– http://www.ibmdmmagazinedigital.com/dmmagazine/

Redbooks at www.redbooks.ibm.com– DB2 9 for z/OS Performance Topics SG24-7473– IBM DB2 9 for z/OS: New Tools for Query Optimization SG24-7421

http://www.ibm.com/developerworks/spaces/optim

https://www-01.ibm.com/software/data/db2imstools/db2tools/opti-expert-zos



http://www.ibmdmmagazinedigital.com/dmmagazine/



http://www.redbooks.ibm.com/



UGF Usage of Query Workload Tuner

Their new Process to Tune Queries1.Every request of a new object (table or index) from the application

developer

2. From daily/weekly/monthly reports on costs

3. From stress test after existing applications upgrade

4. From alert coming from the Scheduling office What they liked

– Statistics Advisor – Easiness in tracing dynamic statements – What-if analysis– Query Environment Capture



Daily performance reports

After query tuner activities:

AVG CPU: from 0.08 to 0.05

50000*0.03=1500 sec



Achievements:SYSSH200

0

0,02

0,04

0,06

0,08

0,1

0,12

0,14

10-fe

b

11-fe

b

12-fe

b

15-fe

b

16-fe

b

17-fe

b

18-fe

b

19-fe

b

22-fe

b

23-fe

b

24-fe

b

25-fe

b

26-fe

b

1-mar

2-mar

3-mar

4-mar

5-mar

8-mar

9-mar

10-m

ar

11-m

ar

12-m

ar

15-m

ar

16-m

ar

17-m

ar

18-m

ar

19-m

ar

22-m

ar

23-m

ar

24-m

ar

25-m

ar

26-m

ar

29-m

ar

30-m

ar

31-m

ar

1-ap

r

6-ap

r

7-ap

r

8-ap

r

9-ap

r

EL

A

0

0,0002

0,0004

0,0006

0,0008

0,001

0,0012

0,0014

0,0016

0,0018

0,002

CP

U

ELAmedCPUmed

SQLLF000

0

0,02

0,04

0,06

0,08

0,1

0,12

11-fe

b

12-fe

b

15-fe

b

16-fe

b

17-fe

b

18-fe

b

19-fe

b

22-fe

b

23-fe

b

24-fe

b

25-fe

b

26-fe

b

1-mar

2-mar

3-mar

4-mar

5-mar

8-mar

9-mar

10-m

ar

11-m

ar

12-m

ar

15-m

ar

16-m

ar

17-m

ar

18-m

ar

19-m

ar

22-m

ar

23-m

ar

24-m

ar

25-m

ar

26-m

ar

29-m

ar

30-m

ar

31-m

ar

1-ap

r

6-ap

r

7-ap

r

8-ap

r

9-ap

r

EL

A

0

0,0002

0,0004

0,0006

0,0008

0,001

0,0012

0,0014

0,0016

0,0018

CP

U ELAmedCPUmed



Examples at XXX of SQL Query Performance Tuning using Query Workload Tuner PeopleSoft SQL Query Example #1 - continued

– This is the report after applying the changes recommended by Query Tuner’s Tuning Advisors. STMT_ID before (5481805) and after tuning (368), Shows the average CPU per execution improvement and considering this statement is executed by PeopleSoft every 30 seconds, the CPU savings will add up

ibm optim query tuning offerings optimize … optim query tuning offerings optimize performance and...

Documents