© ibm corporation 2007 1 patrick bossman ibm silicon valley lab tuning with optimization service...

© IBM Corporation 2007

1

Patrick Bossman

IBM Silicon Valley Lab

Tuning with Optimization Service Center

Part I

®

Columbia, MDSeptember 12, 2007


2

Agenda

• Overview of Optimization Service Center

• Connect and configure OSC

• Workload (application) Tuning

• Query tuning (Part II)


3

Overview

• Optimization Service Center (OSC)– New no charge product supported for connections

to DB2 9. • Support for DB2 for z/OS V8 connections in open beta

– Workstation tool for monitoring and tuning of queries

– Facilitates the identification and tuning of workloads (sets of queries) as well as individual queries

– New powerful query diagnostic tools enable faster deep analysis of queries


4

Overview

• OSC feature list– Support for workload tuning

• Stand-alone workload creation

• Push-out model (monitoring)

– Visual Explain query graphing capabilities

– Query formatting and annotation

– Query report

– Visual Plan Hint

– Statistics Advisor - query and workload


5

Connect and configure

• Launch OSC– Initial screen is connection and configuration

– Review look and feel

– Review layout of screen


6

Launch OSC


7

Project navigator area


8

Project navigator

• Navigate through open projects


9

Top navigation tabs


10

Top navigator tabs close-up


11

Connection / configuration


12

Connection close-up

• Connection– Subsystems cataloged in DB2 Connect listed– Use subsystem menu option to add / remove subsystems


13

Add Subsystem

• Add DB2 subsystem connection– Select subsystem button

– Choose Add


14

Connection information

• Add DB2 subsystem connection– Subsystem alias is a meaningful name to you

– Location, hostname, port• Can be found in <ssid>MSTR address space DDF startup message DSNL004I


15

DSNL004I

• DSNL004I message– Location = location– Domain = hostname– TCPPORT = port


16

Jeez Pat, Connect already!

• Connect to subsystem– Choose Connection, connect– You can connect to more than one subsystem and perform

activities on more than one subsystem at a time.


17

Login information…


18

Bind packages


19

Create explain tables


20



21



22

Create alias to explain tables

• Create aliases– You can use OSC to create ALIASes to the explain

tables also.


23

Create alias to explain tables


24

Subsystem status


25

Subsystem status


26

Explain-enabled Authid’s


27

EXPLAIN enabled authid’s


28

Connection complete.Now what?

(Welcome menu…)


29

Go to welcome page…


30

Welcome page


31

Welcome page close-ups


32



33



34

Application tuning

• Application tuning– Creating workloads

– Workload options

– Workload tuning features


35

Application tuning process

• What is application tuning?– Identify what the application workload is

• Individual SQL statements

– Get an understanding of the applications behavior• How often are individual SQL statements executing?• What is the performance of individual SQL statements?

– Determine statistics for workload • Workload statistics advisor• More workload analysis features coming… (?)

– Remeasure workload• Identify top tuning candidates• Use query based tools to further analyze (next session)


36

Creating workloads

• Creating a workload– Tune a workload

– View workloads

– Monitoring functions


37

Tune a workload


38

Workload sources• Workload sources

– Snap statement cache– Catalog (static SQL)– QMF / QMF HPO– File– Categories– Other workloads


39

Workload sources


40

Catalog source…


41

Catalog source


42

Package filter options• Package filter options

– Collection id, name, owner, …

– Can use equals, like, in, etc.


43

Package filter options


44

Plan filter options• Plan filter options

– Plan name, plan creator, etc.

– Can use equals, like, in, etc.


45

Plan filter options


46

Cost & Object filter options• Cost and Object filters

– Filters SQL within package / plan filter– Requires the static SQL be bound with explain yes.– DSN_STATEMNT_TABLE must already be populated.– Show only SQL with PROCSU > …– Show SQL which uses table…– Show SQL which uses index…

• Choice to return rows which qualify for ANY or ALL of the cost / object filter conditions


47

Cost & Object filter options


48

Access path filters• Access path filters

– Filters SQL within package / plan filter

– Requires the static SQL be bound with explain yes.

– PLAN_TABLE must already be populated.

• Show SQL which performs…– Tablespace scan

– Sort

– Non-matching index scan

– List prefetch

– Outer join

– ….


49

Cost & Object filter options


50

Capture the workload

• After selecting packages / plans / filters…– Click finish to capture workload

– OSC goes about the business of collecting the statements…

• Queue Jeopardy music, get some coffee.


51

Workload captured…


52

Review workload screen

• Workload header

• Workload options

• Project Navigator


53

Workload header


54

Workload header


55

Workload options


56

Workload options


57

Workload option close-up (1)


58

Workload options close-up (2)


59

Workload statements…


60

Workload statements

• Workload statements– Show the SQL within the workload

– Refine execution runtime information• For catalog capture, runtime information would have to be

manually obtained.

• You don’t really need to change it for statistics advisor…

– Let’s walk through it…


61

Workload statements view


62

Capture more statements…


63

Description


64

Schedule workload activity


65

Remove statements from workload


66

Execute query tools on selected SQL


67

Select “edit runtime information…”


68

Edit runtime information


69

Run workload advisors (SA)


70

Why Statistics Advisor?

• Importance of statistics• Difficulty of determining what statistics to

collect

• Solution: Workload Statistics Advisor!!!


71

Estimating query cost

• Accurately estimating the number of rows is important for to accurately cost a query– Index matching

• Accurately estimate index cost

– Total index filtering• Estimate table access cost via index(es)• Choose how to use index (prefetch?)

– Total table level filtering• Efficient join order• Efficient join method• Appropriate sorts


72Symptoms of inaccurate estimation

• Sometimes - great performance!– Few competitive choices

– Good performance possible, but not a "winning" strategy

• Inefficient access path– Query never performs at optimal level

• Unstable access path– Slight changes in input result in dramatic swings in actual

performance

– Efficient and inefficient access paths can end up with close cost estimates

– More accurate selectivity estimates can improve optimizer "differentiation" of efficient and inefficient access


73

Some observations

• Several candidate indexes?– Need sufficient statistics to differentiate the candidates

• Local filtering spread across many tables?– The more "potentially" desirable outer tables there are, the more

important it is to accurately estimate the true qualified rows

• Lot's of predicates which don't filter much?– Important to make sure DB2 knows that

– When many rows are processed, a single bad choice can be magnified

• The necessity for accurate estimation is related to the number of competitive choices.


74

Why statistics advisor?

• Statistics are collected to help optimizer in costing available access paths

• Determination of statistics required can be daunting

• Automate the determination of statistics required


75

What are the right statistics?

• Identify interesting objects– Tables– Indexes

• Identify interesting columns, column groups– Predicate analysis

• Which column (groups) have predicates?

• Which column (groups) are indexed?

• Which statistics should I collect?– Column statistics– Correlation statistics– Frequencies– Histograms

– Statistic needed depends on predicate types (range, equal)


76

Determining the "right" statistics

• Manual analysis very time consuming, requires significant skill– Format the query to make it readable

– Recognize predicate transformations • Predicate transitive closure

• Between to equal opens new possibilities

• Or to IN creates new boolean term predicates

• Has DB2 rewritten the query (predicate transitive closure)

– Some columns used as predicates hidden in views

– Recognize which statistics are appropriate in which circumstances

• Significant effort even for expert

• Error prone


77

Run workload advisors (WSA)


78

Complete recollection, or repair?


79

RUNSTATS command


80

Table portion of commandRUNSTATS TABLESPACE DB2OSC.WSATS0 TABLE(DB2OSC.DSN_WSA_DATABASES) COLUMN(NAME,SESSIONID) TABLE(DB2OSC.DSN_WSA_CGFREQS) COLUMN(TBCREATOR,TBNAME,COLGROUPCOLNO,SESSIONID) TABLE(DB2OSC.DSN_WSA_KTGFREQS) COLUMN(IXCREATOR,IXNAME,SESSIONID,KEYGROUPKEYNO) TABLE(DB2OSC.DSN_WSA_SESSIONS) COLUMN(SESSIONID) TABLE(DB2OSC.DSN_WSA_COLGROUPS) COLUMN(TBCREATOR,TBNAME,COLGROUPCOLNO,SESSIONID) COLGROUP(COLGROUPCOLNO,SESSIONID,TBCREATOR,TBNAME) TABLE(DB2OSC.DSN_WSA_TSS) COLUMN(NAME,DBNAME,SESSIONID) COLGROUP(DBNAME,SESSIONID) COLGROUP(DBNAME,NAME,SESSIONID) TABLE(DB2OSC.DSN_WSA_KTGS) COLUMN(IXCREATOR,IXNAME,SESSIONID) TABLE(DB2OSC.DSN_WSA_KEYS) COLUMN(IXCREATOR,IXNAME,SESSIONID) TABLE(DB2OSC.DSN_WSA_CGHISTS) COLUMN(TBCREATOR,TBNAME,COLGROUPCOLNO,SESSIONID) TABLE(DB2OSC.DSN_WSA_KTGHISTS) COLUMN(IXCREATOR,IXNAME,SESSIONID,KEYGROUPKEYNO) TABLE(DB2OSC.DSN_WSA_INDEXES) COLUMN(TBCREATOR,TBNAME,SESSIONID) TABLE(DB2OSC.DSN_WSA_TABLES) COLUMN(CREATOR,NAME,TSNAME,DBNAME,SESSIONID) COLGROUP(CREATOR,NAME,SESSIONID) TABLE(DB2OSC.DSN_WSA_LITERALS) COLUMN(TBCREATOR,TBNAME,SESSIONID,COLNO,VALUE) COLGROUP(COLNO,SESSIONID,TBCREATOR,TBNAME,VALUE) TABLE(DB2OSC.DSN_WSA_KEYTARGETS) COLUMN(IXCREATOR,IXNAME,SESSIONID) TABLE(DB2OSC.DSN_WSA_COLUMNS) COLUMN(TBCREATOR,TBNAME,SESSIONID) SORTDEVT SYSDA SHRLEVEL CHANGE REPORT YES


81

Index portion of suggestionRUNSTATS TABLESPACE DB2OSC.WSATS0 …INDEX(DB2OSC.DSN_WSA_IDX_DBS, DB2OSC.DSN_WSA_IDX_CGFS, DB2OSC.DSN_WSA_IDX_KTGFS, DB2OSC.DSN_WSA_IDX_SSNS, DB2OSC.DSN_WSA_IDX_CGS, DB2OSC.DSN_WSA_IDX_TSS, DB2OSC.DSN_WSA_IDX_KTGS, DB2OSC.DSN_WSA_IDX_KEYS KEYCARD, DB2OSC.DSN_WSA_IDX_CGHS KEYCARD, DB2OSC.DSN_WSA_IDX_KTGHS KEYCARD, DB2OSC.DSN_WSA_IDX_IDXS, DB2OSC.DSN_WSA_IDX_TABLES, DB2OSC.DSN_WSA_IDX_LTRS KEYCARD, DB2OSC.DSN_WSA_IDX_KTS KEYCARD, DB2OSC.DSN_WSA_IDX_COLS KEYCARD) SHRLEVEL CHANGE REPORT YES


82

Output looks complex…

• RUNSTATS commands are more targeted– Only collect on tables used in queries

– Only collect column statistics on columns used as predicates

– Collect frequency and histogram statistics based on predicates which would benefit


83

Pause for questions on WSA?


84

Back to our regularly scheduled programming…


85

More OSC GUI review


86

Open projects as tabs on top…


87

Project Navigator


88

Project navigator

• Navigate across projects– As multiple projects opened, you can switch between

projects you’re working on

• Navigate within a project– As you perform different tasks within a project, you

can navigate through the different tuning sections


89

Project Navigator


90

Project tabs


91

Project tabs


92

Project tabs

• Navigate within project– As multiple projects opened, you can switch between

projects you’re working on

• Close sections of project you’re done with– Click x on the tab to close selected tab


93

More OSC GUI review


94

Workload sources - cache


95

Cache statement view

• Cache statement view– Number of executions

– Accumulated & average CPU time

– Accumulated & average elapsed time

– SQL text

• Cache as input provides more visibility into how often SQL executing, cost of statements


96

Workload – cache statements view


97

Monitor workload

• Normal monitor workload– All SQL based on certain criteria

– Useful to profile an application• Which SQL running most frequently?

• Which queries consuming most elapsed / execution time?

• Exception monitor– Workload generated based on exception criteria

• All SQL for package > x seconds elapsed

• All SQL for package > x seconds elapsed


98Application tuning process(revisit)







99

Create normal monitor


100

Add monitor sources


101

Set monitor options


102

Monitor statement view


103

Exception monitor options


104

Application tuning

• Normal monitor allows for capture of all SQL– Specify list of packages

• Prevents workload “pollution” which can occur with statement cache

– Capture valuable execution information

– Ideally initially used in QA environment to profile / tune application before production implementation

• “How often does this execute?”

• “How long does this run?”

• … you can have answers to these questions…

– Selectively use in production to develop application profile and tune application


105Application tuning process(review)







106

Which workload source to use?

• Which workload source provides…– Identification of all SQL statements executed

– Accurate number of execution counts

– Accurate accumulated and average execution performance

– Provides runtime information for any source

– Can re-measure and expect to see the same SQL again, compare results

– Is not polluted with dynamic SQL which are not relevant to the application?

• Workload monitoring (normal)


107

Tuning with OSC I

• End of OSC tuning Part I– Show some demo if time permits

• Next up, Query tuning with OSC– Statistics Advisor

– Query annotation

– Query report

– Visual Explain

– Visual plan hint


108

Patrick Bossman

[email protected]

Tuning with Optimization Service Center I

®

© ibm corporation 2007 1 patrick bossman ibm silicon valley lab tuning with optimization service...

Documents

ibm corporation

workload slide

port slide

subsystem status slide

connection configuration

launch osc slide

welcome page slide

welcome menu slide