understanding/managing the access plan changes and tuning … · 2016. 9. 11. · to hone my sql...

© 2010 IBM Corporation

Information Management

Understanding/Managing the Access Plan

Changes and Tuning Data Studio & Query Workload Tuner

Saghi Amirsoleymani [email protected]

mailto:[email protected]



2 2

Performance tuning challenges across the organization

“We can’t adequately test for

peak workload since we don’t have

enough human or IT resources.”

“I need to get my business results

fast and accurate. What’s going on ?”

”I don’t have time to hone my SQL skills. I need to

focus on developing core

application functionality.”

Application Developer

“I don’t understand why our developers aren’t focused on creating

better performing SQL.”

“Performance problems seem to appear without

warning and deep technical skills are hard

to find.”

QA Manager

DBA

“It is very challenging to

aggregate performance data

across our complex data environment.”

IT ManagerLOB Manager



3

Agenda

Subsystem Configuration Options



4

What is IBM Data Studio? Comprehensive tool

– An integrated environment for managing databases and developing database applications

Built on the popular Eclipse framework

Runs on Red Hat Linux, SUSE Linux, Windows

Web and Eclipse components

Data Studio Web console: health and availability monitoring

NO-charge to download!

http://www.ibm.com/developerworks/downloads/im/data/



5

WebConsole

EclipseClient

Common Tool across Roles Improve Collaboration and Productivity

Database Administration

Catalog navigation Object management Privilege management

Develop

SQL development Routine development Deployment management Query tuning Java data access

Monitor

Alerts Dashboards

Schedule

JDBC SQL Scripts



6

Agenda




7

Launch key tasks

Select lifecycle

Access tutorials and advice

Basic Navigation



8

Perspectives

What are they?–Eclipse term

• Visual container for a set of views and editors–Data Studio has many

• Data• Database Administration• IBM Query Tuner• And more

–The first time you start Data Studio it will open the Database Administration perspective

8



9

Opening a Perspective

9

1 2

3

•In the upper corner click on the highlighted icon and select Other…

•From the menu bar select Window -> Open Perspective - > Other

Both methods will then bring up the displayed dialog box

•Select <IBM Query Tuning> Click OK



10

Tip: Reset Perspective gets you back to normal state

Perspective Management- Resetting Perspective

Task Navigator changes

perspective



11

Web browser-like navigation bar

Object Properties / Results view

Administration Explorer• Connect to databases• Browse database objects• Organized in folders by their type

Editors area• Listing objects• Editing scripts• Running tasks

Basic Navigation



12

Basic Navigation : Filter and Sort

Advance • Filter • Sort

Quick FilterQuick Sort



13

Basic navigation : Create/Alter/Drop

Right-click Alter/DropRight-click Creates

Create a new object



14

Database Object Management :: Editing Data

Browse and edit data in a table– Almost identical UI for browse and edit, main difference is read-only for

browsing data– Edited table cells are highlighted with yellow

Click or press Ctrl + S to commit data changes• Results are displayed in the SQL Results view

Edit data

Filter



15

Agenda




16

Identify query candidates from numerous sources– DB2 catalog– Dynamic statement cache– Data Studio hot spots– Query or performance monitors

Facilitate analysis – Query formatting– Query annotation– Access path visualization and

annotation

Get expert tuning advice– Improve statistics quality– Improve query design– Improve database design

Expert-Enabled Query Tuning Out of the Box -Avoid Common Mistakes



17

Getting Started Query Tuning: Option 1

17

In the Data Source Explorer, right-click a connection and select Connect, expand the connection, right-click the data server, and select Analyze and Tune > Start Tuning.



1818

Creating a Query Tuner Project



19

Streamlined Analysis with workflow assistant

Validate improvement

Define or select a workload Execute Advisors

Drill Down into advice



20

Query Tuner Workflow Assistant Intended to provide a logical workflow for tuning activities

– 1. Status• Provides configuration status information

– 2. Capture• Input or capture of SQL statements

– 3. Manage• Not available in the “No-Charge” offering

– 4. Invoke• Verify or modify Explain Options and Select Tuning Activities

– 5. Review• Review results of the tuning process

– 6. Compare• Not available in the “No-Charge offering

20



21

Gather High Cost Queries and Workloads

DB2

Plan table

Statement table

Catalog plan or package

Statement cache

Optim Performance Manager

QMF and QMF HPO

DB2 Query Monitor

OMEGAMON XE for DB2

File, text, or exported workload

User defined SQL Repository

Define or select workload

new



22

Customize Capture

22

Filtering Options

• Accelerate analysis, reduce downtime

TIP: Query Tuner issues

dynamic SQL if you check

the box we will not

captured that information



23

Capture from DB2 Catalog

TIP: Filter by package

collection and name,

and some potential

“bad” access plans

Source

Filter Rows



24

User Defined SQL RepositoryDefine, edit, delete filter

Define Filter

Column Mapping

Limit number of returned statements

Supply table qualifier and table/view name



25

Review & Invoke Advisors

Capture

Define Filter Criteria

Tune a single query or at a workload

level



26© 2013 IBM Corporation26

Export Captured Statements and meta data

Choose different formats



27

Visualize Queries and Costs to Speed Analysis

Accelerate analysis, reduce downtime– Spot human errors – Identify where filtering should occur

Easily see tables, sections, join

predicates, etc.Examine table statistics

and additional information

Tune



28

View Optimizer Transformations

Accelerate analysis, reduce downtime– Spot transformations which will occur



29

Review and Visualize access path – See flow of query processing– See indexes and operations– See optimizer rationale

Assess access path stability to reduce risk of performance regression– Is the optimizer able to apply the filtering early?– Are there indexes that support an efficient path?– Do statistics allow distinction between the choices?

Analyze Access Plans :Avoid Common Mistakes



30

Improve Statistics Quality and Collection

Results – Accurate estimated costs – Better query performance– Less CPU consumption – Improved maintenance

window throughputConflicting statistics

explanation

Indicates conflicting and

missing statistics

Generates RUNSTATS

control statements

Provides advice on – Missing statistics– Conflicting statistics– Out-of-date statistics

“70 % of access path PMRs could be resolved by statistics advisor before calling IBM support.” – IBM Support



31

What to do next

How are Statistics currently collected?

Why are Histogram statistics not collected?

Distribution Statistics – Are you over collecting statistics?

simple answer– Some collect no or insufficient statistics

• Prime reason for poor performing access paths– Do you want to collect statistics on every column and permutations of combination of

columns? • No way!

Requires similar analysis of SQL as for index design– Have to include columns which you may not benefit from adding to an index– Analysis of queries labor intensive without QWT– Iterative process analyzing explain data (as always)



3232

Tune : Working with SQL Statements reports Summary report of the recommendations

– Foster cross-role collaboration

Challenge – Disconnect between application and

data group– Disconnect between remote

employees and locations

Solution-– verbose HTML file review, email, or

print– Overview– Recommended actions– Statistics Advisor Summary– Access Plan Summary

• Predicates• Table access• Join operation

– Table and index catalog information: – Access Path Advisor Summary– Indexes– Query Tuner Report Log



33

Environment Capture Facilitates Collaboration

Enable environment reproduction

– DDL– Catalog statistics– Zparms (if DSNWZP

stored procedure available)

– Explain tables information (PLAN_TABLE, DSN_DETCOST_TABLE, DSN_PREDICAT_TABLE) (if SQL used as input)

Speed up service process



34

Key Query Tuning Features for DB2 for z/OS in Data StudioKey Query Tuning Features for DB2 for z/OS in Data Studio

Workload Control CenterWorkload Control Center

ReportReport

FormatterFormatter

Query Based Tools and AdvisorsWorkload Advisors and Tools

Query AdvisorsQuery Tools

Service SQLService SQL

Access Path GraphAccess Path Graph

Statistics AdvisorStatistics Advisor

Data Studio Client User Interface Data Studio Client User Interface

Workload Test Candidate IndexWorkload Test Candidate Index

Workload Access Path Advisor

Workload Access Path Advisor Service

WorkloadService Workload

ReportReportWorkload Access Plan Comparison

Workload Access Plan Comparison

Workload Query Advisor

Workload Query Advisor

Workload Statistics Advisor

Workload Statistics Advisor

Workload Index Advisor

Workload Index Advisor

Access Plan ExplorerAccess Plan Explorer

AnnotationAnnotation

Visual Plan HintVisual Plan Hint

Test Candidate Index

Test Candidate Index

Query AdvisorQuery Advisor

Access Path AdvisorAccess Path Advisor

Index AdvisorIndex Advisor

Workload AnalyticsAccelerationAdvisor

Workload AnalyticsAccelerationAdvisor

Statistics Collection Advisor

Statistics Collection Advisor

Note: Features in green boxes are features in Data Studio no-charge offering others requiring license activation



35 35

Statistics – Get recommendations on

the best statistics to capture to influence access path selection

Query– Get recommendations

regarding how to rewrite the query for better efficiency

Index – Get recommendations on

indexes changes that can reduce database scans

Analytics Accelerator – Get recommendations on

optimizing and managing accelerated analytic queries and applications

Execute Advisors



36

Improve Statistics Quality and Collection Integration with the DB2 z/OS 11 SYSSTATFEEDBACK t o interprets information and updates statistics

profile

Result display on the result dialogAccurate estimated costs Better query performanceLess CPU consumption Improved maintenance window throughput

New Demo Video !!! http://youtu.be/M9wV0oAFgpw

Generates RUNSTATS control statements

Include or not

include

New

Update Statistics Profile



37 37

How IBM DB2 tools can maximize Accelerator value

Customers want to learn more about their investment in the Accelerator and maximize its use in their environment

Three different areas where DB2 tools can provide value– Assessment

• Do I have a workload that would benefit from the Accelerator?– Optimization

• Can I optimize the workload to take advantage of the Accelerator?– Administration

• Can I manage the Accelerator more effectively?



38

Optimizing the Selection, Tuning and Access Plan Analysis of Accelerated Workloads

Workload Analytics Accelerator Advisor

Identify candidate queries and tables to be routed to the accelerator

Identify candidate tables to be routed to the accelerator

Implement advisor-based tuning recommendations for mixed workloads of accelerated and un-accelerated queries

Diagram accelerated queries in Access Plan Graphs

Integrates with Query Monitor and OMPE for capturing query workloads for complete analysis

Enable “what if” analysis

Benefits

Shorten the process of selecting tables to be accelerated

Visualize access paths of accelerated queries

Increase productivity by working with accelerated queries through a unified interface Increase overall system capacity

Demo Video http://youtu.be/pQYMRHJW7NU



39 39

Gather Queries and Workloads

DB2

Plan table

Statement table

Catalog plan or package

Statement cache

QMF and QMF HPO

DB2 Query Monitor

OMEGAMON XE for DB2

File, text, or exported workload

User defined SQL Repository

Define or select workload1



40 40

3 Drill down to advice for accelerated workload

Determine tables to be routed the Accelerator

Simplify use– Consolidate tables, queries to provide a single recommendation – Enable what-if analysis– Run immediately

or save Estimated CPU savings

Filter recommendations



41

Drill down to advice for accelerated workload

41

Show Tables referenced by statement

Rationale

Recommendation and rationale

3



42 42

Test Candidate Analytics Acceleration

Test before deployment• Utilize virtual capabilities built into the DB2

engine run subset of candidate tables

4



43 43 43

Validate Analytics Acceleration Result

View analysis summary

Less CPU savings

Fewer Eligible Statements

4



44

Hybrid Workloads

44



45

Removing Tables from the Analytics Accelerator

45

•Analyze tables with existing accelerator as consideration

•Recommendation table to remove from Accelerator



46

Indexing Advice to Improve Database Design

• Indexes are decided at design stage • Lot of effort is spent making SQL to use the provided indexes• But what if the SQL is "right" and it's the indexes that are "wrong“• Cost resources to maintain

– How do you simply test your hypotheses without impacting production?

• Removing Obsolete Indexes Simplify Use – Consolidate indexes and provide a single recommendation – Enables what-if analysis– Provides DDL to create indexes– Run immediately or save

46Indexes are used to increase performance

Obsolete Indexes display “Last Used Time



47 47

Summary: Optim Query Workload Tuner provides expert tuning

Reduces specialized skill requirements for tuning queries

– Provides actionable recommendations– Builds skill with rationale

Integrates with developer and DBA tools

– Lifecycle integration with developer and DBA tools

Improves application performance– Improve query design– Improve statistics quality– Improve index value– Improve query acceleration

Balances costs across workload– Considers entire workload

Solve problems for all types and size of workloads

Analyze single queries or entire workloads

“IBM InfoSphere Optim Query Workload Tuner cuts the DBA’s workload in testing, making them at least 40 to 50 percent more productive.” — He Yu, Senior Database Administrator



48



49

DB2 z/OS Install/Configuration tips



50

Query Tuner Architecture

Windows XP or Vista or 7

InfoSphere Optim Query Workload Tuner Client

Res

tric

ted

use

DB

2 C

onne

ct

DB2 for z/OS, V8 or laterwith DDF enabled

DB2 stored procedures for use

with InfoSphere Optim Query

Workload Tuner products

Catalog tablesQT tablesProfile tables

Query Tools• Formatting• Annotating• Access plan graph• Visual plan hints• Reporting• Environment capture

Query Advice• Statistics• Query• Index• Access Path

Workload Advice• Statistics• Index• Workload control

center



51

Detailed Status Report

Display status of required and optional objects in html format.

It tells how to fix a configuration problem.

It allows user to save the report to a html file or save only the recommended DDLs to a text file.

Consolidated installation

Preferences simplification and deployment

Database Configuration Deployment Made Easy



52

Change Default SQLID and Schema

52



53

Preferences

Accessing the preferences

Activating/De-Activating traces

Explain Options

Statistics Advisor

Export/Import

53



54

Query Tuner Trace

54



55

Preferences – Explain Query Preferences

55



56

Statistic Advisor Preferences

56



57

Statistics Advisor – Cont

57



58

Query Tuner Preferences Page Cont

58



59

Exporting Preferences

59



60

Importing Preferences

60



61

Agenda

61



62

Data Studio

Download and extract files from shopZ or developerworks

IBM Installation Manager–Used when installing most GUI-based IBM products–Triggered when kicking off the setup.exe–Select both Data Studio and OQWT License Activation Kit



63

Query Workload Tuner License Activation

Analyze and Tune- Activate Full Set of Tuning Features

New!



64

Silent Installation option

In a cmd prompt– Navigate to the Installation Manager directory– Run IBMIM.exe with these options:

– If you haven’t installed Data Studio: remove the “skipInstall” parameter

Once Installation manager launches you will navigate to the repository preferences and add the Data Studio and OQWT LAK “diskTag.inf” file

– Root_path_of_installation_media\disk1\diskTag.inf– Root_path_of_installation_media\oqwt\diskTag.inf

Complete the installation and transfer the response file to the workstation(s)



65

Connecting/Defining to DB2 Subsystem Connection

Location = subsystem name

Host = FQDN

Port number = DDF port

User name = TSO id

Password = TSO password

Default Schema = unqualified SQL objects

From SDSF – DB2 Started Task

LOCATION STLEC1 DOMAIN DTEC514.vmec.svl.ibm.com TCPPORT 446



66

Subsystem Configuration Process Bind the packages

Create EXPLAIN and Query Tuner tables

Grant privileges and user access

Create aliases



67

Detailed Status Report

Display status of required and optional objects in html format.

It tells how to fix a configuration problem.

It allows user to save the report to a html file or save only the recommended DDLs to a text file.

Consolidated installation

Preferences simplification and deployment

** PDF & Video will be placed on Wikki

New! Database Configuration for Query Tuning

Deployment Made Easy



68

Configure DB2 subsystem via JCL job

Located in the Data Studio installation directory:–All of the required DBRMs and JCL job



69

Configure DB2 subsystem via JCL job (continued)

JCL job–What does it do?

• AOCDDL10 • (DB2 10 JCL job)



70

Test configuration

Attempt to tune a query–You can create and tune the query in Data Project Explorer module



71

Agenda



72

Authorities and Privileges

Authorizations for configuring subsystem

http://pic.dhe.ibm.com/infocenter/dstudio/v4r1/index.jsp?topic=%2Fcom.ibm.datatools.qrytune.installconfig.doc%2Ftopics%2Fchklst_db2zos.html



73

Resources

Integrated Data Management Community– http://www.ibm.com/developerworks/spaces/optim

IBM Optim Query Workload Tuner web page– https://www-01.ibm.com/software/data/db2imstools/db2tools/opti-expert-zos/

Tuning SQL with IBM Optim Query Workload Tuner

Part 1 Understanding Access Paths

http://www.ibm.com/developerworks/data/library/techarticle/dm-1006optimquerytuner1/

– Part 2 Tuning Individual Queries

http://www.ibm.com/developerworks/data/library/techarticle/dm-1105optimquerytuner2/?cmp=dw&cpb=dwinf&ct=dwnew&cr=dwnen&ccy=zz&csr=052611

– Part 3 Workload Capturing and Tuning

http://www.ibm.com/developerworks/data/library/techarticle/dm-1207optimquerytuner3/index.html?ca=drs

SQL Tuning, Not just for hardcore DBAs anymore– http://www.ibmdmmagazinedigital.com/dmmagazine/

https://www-01.ibm.com/software/data/db2imstools/db2tools/opti-expert-zos

understanding/managing the access plan changes and tuning … · 2016. 9. 11. · to hone my sql...

Documents