adaptive query optimization

2014 © Trivadis

BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA

2014 © Trivadis

Adaptive Query OptimizationOracle OpenWorld, 28 September 2014

Christian Antognini

28 September 2014Adaptive Query Optimization

1

2014 © TrivadisAdaptive Query Optimization28 September 2014

2

@ChrisAntognini

Senior principal consultant, trainer and partnerat Trivadis in Zurich (CH) [email protected] http://antognini.ch

Focus: get the most out of Oracle Database Logical and physical database design Query optimizer Application performance management

Author of Troubleshooting Oracle Performance (Apress, 2008/2014) OakTable Network, Oracle ACE Director

2014 © TrivadisAdaptive Query Optimization

Over the years Oracle has extremely improved the capabilities of the query optimizer

Most of the improvements fall into one of the following areas Enhance the quality of the inputs (e.g. objects statistics) Make the gathering and management of object statistics easier and more efficient Use new data structures Implement or enhance query transformations Improve plan stability Cope with poor estimations that leads to poor execution plans

12c is no exception, every one of these areas were improved

28 September 2014

3

Introduction

2014 © Trivadis

Adaptive plans (Enterprise Edition only) Join methods Parallel distribution methods Star transformation

Adaptive statistics SQL plan directives Automatic reoptimization

- Statistics feedback (evolution of cardinality feedback)- Performance feedback

Dynamic statistics (evolution of dynamic sampling)


4

Adaptive Query Optimization Features


Enables or disables adaptive query optimization features Adaptive plans SQL plan directives Automatic reoptimization

- It isn’t the case in 12.1.0.1 (bug 16824474)

The default value is TRUE

OPTIMIZER_DYNAMIC_SAMPLING controls dynamic statistics

28 September 2014

5

OPTIMIZER_ADAPTIVE_FEATURES


Enables or disables reporting mode for adaptive query optimization features

Useful to assess how an execution plan would change

Use DBMS_XPLAN to get detail information about the analysis Might fail with an ORA-1001 (bug 17270605)

The default value is FALSE

28 September 2014

6

OPTIMIZER_ADAPTIVE_REPORTING_ONLY

SELECT * FROM table(dbms_xplan.display_cursor(format=>'report'))


Object statistics don’t always provide sufficient information to find an optimal execution plan

To get additional insights, the query optimizer can take advantage of features like dynamic sampling and cardinality feedback Don’t solve all issues, though

28 September 2014

7

Adaptive Plans – Challenge


As of 12c, the query optimizer can postpone some decisions until the execution phase

The idea is to leverage information collected while executing part of an execution plan to determine how another part should be carried out

The query optimizer uses adaptive plans in three situations: To switch the join method from a nested loops join to a hash join and vice versa To switch the PX distribution method from hash to broadcast/round-robin To disable the access to a dimension for execution plans using the star transformation

28 September 2014

8

Adaptive Plans – Concept


The query optimizer adds subplans to execution plans One represents the nested loops join, the other the hash join One of the alternatives is the default plan

One of the subplans is chosen during the first execution The choice is based on the number of rows actually processed The query optimizer computes an inflection point

A new row source operation is used to partially buffer and count the rows STATISTICS COLLECTOR Limitation: if a too large buffer is required, no adaptive plan is used

The execution plan that is actually executed is called the final plan

28 September 2014

9

Adaptive Plans – Join Method Switch


1028 September 2014

Adaptive Plans – Join Method Switch Example

SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n = 666

-----------------------------------------------| Id | Operation | Name |-----------------------------------------------| 0 | SELECT STATEMENT | || 1 | HASH JOIN | || 2 | NESTED LOOPS | || 3 | NESTED LOOPS | || 4 | STATISTICS COLLECTOR | || 5 | TABLE ACCESS FULL | T1 || 6 | INDEX UNIQUE SCAN | T2_PK || 7 | TABLE ACCESS BY INDEX ROWID| T2 || 8 | TABLE ACCESS FULL | T2 |-----------------------------------------------

adaptive_plan.sql


1128 September 2014




adaptive_plan.sql


1228 September 2014




adaptive_plan.sql

2014 © Trivadis

For both join methods, the cost associated to different cardinalities is estimated The cardinality of the outer table varies The cardinality of the inner table remains fixed

The query optimizer uses a binary search

The search takes place between a minimum and maximum cardinality


13

Adaptive Plans – Join Method Switch Inflection Point

2014 © Trivadis


14

Adaptive Plans – Join Method Switch Inflection Point Example

1

19239.93

9620.46

4810.73

2405.87

1203.43

602.22

301.61

151.3

76.15

38.58

57.36

47.97

52.67

50.32

49.14

48.56

20

200

2,000

20,000

Nested Loops Join

Hash Join

Cardinality(In Order of Execution)

Co

st

2014 © Trivadis

New distribution method HYBRID HASH It helps avoiding some data skewing problems

The actual distribution method is chosen at execution time STATISTICS COLLECTOR The decision takes place during each execution

Number of rows < 2 * DOP Left input: BROADCAST / Right input: ROUND-ROBIN

Number of rows ≥ 2 * DOP Left input: HASH / Right input: HASH


15

Adaptive Plans – Parallel Distribution Method

2014 © Trivadis


16

Adaptive Plans – Parallel Distribution Method Example

------------------------------------------------------------------------| Operation | Name | TQ |IN-OUT| PQ Distrib |------------------------------------------------------------------------| SELECT STATEMENT | | | | || SORT AGGREGATE | | | | || PX COORDINATOR | | | | || PX SEND QC (RANDOM) | :TQ10002 | Q1,02 | P->S | QC (RAND) || SORT AGGREGATE | | Q1,02 | PCWP | || HASH JOIN | | Q1,02 | PCWP | || PX RECEIVE | | Q1,02 | PCWP | || PX SEND HYBRID HASH | :TQ10000 | Q1,00 | P->P | HYBRID HASH|| STATISTICS COLLECTOR | | Q1,00 | PCWC | || PX BLOCK ITERATOR | | Q1,00 | PCWC | || TABLE ACCESS FULL | T1 | Q1,00 | PCWP | || PX RECEIVE | | Q1,02 | PCWP | || PX SEND HYBRID HASH | :TQ10001 | Q1,01 | P->P | HYBRID HASH|| PX BLOCK ITERATOR | | Q1,01 | PCWC | || TABLE ACCESS FULL | T2 | Q1,01 | PCWP | |------------------------------------------------------------------------

hybrid_hash.sql

2014 © Trivadis

With the star transformation, the data of each dimension that has a restriction applied to it might be “joined” to the corresponding bitmap index of the fact

If the number of rowids returned by such a “join” is underestimated, applying the filter can be detrimental to the performance

With an adaptive plan the access to some dimensions can be disabled Decision takes place during the first execution only


17

Adaptive Plans – Star Transformation

2014 © Trivadis


18

Adaptive Plans – Star Transformation Example

-----------------------------------------------------| Operation | Name |-----------------------------------------------------| … | || VIEW | VW_ST_5497B905 || NESTED LOOPS | || BITMAP CONVERSION TO ROWIDS | || BITMAP AND | || BITMAP MERGE | || BITMAP KEY ITERATION | || TABLE ACCESS FULL | COLORS || BITMAP INDEX RANGE SCAN | CAR_COLOR_IDX || STATISTICS COLLECTOR | || BITMAP MERGE | || BITMAP KEY ITERATION | || TABLE ACCESS FULL | MODELS || BITMAP INDEX RANGE SCAN| CAR_MODEL_IDX || … | || TABLE ACCESS BY USER ROWID | CARS |-----------------------------------------------------

2014 © Trivadis

NULL means that the execution plan associated to the cursor isn’t adaptive

N means that the final execution plan hasn’t been determined

Y means that the final execution plan was determined Also set if reporting mode is enabled

Set for join method switches and star transformation only


19

Adaptive Plans – V$SQL.IS_RESOLVED_ADAPTIVE_PLAN


SQL plan directives are automatically created when misestimates occur They are temporarily stored in the SGA and flushed to disk every 15 min by a

background process

They aren’t tied to a specific SQL statement, but to specific tables/columns Several of them can be used for a single SQL statement

They instruct the database engine to automatically create extended statistics Only column groups are considered The limit of 20 extensions applies to them as well If creating extended statistics isn’t supported/possible/sensible, they instruct the query

optimizer to use dynamic sampling

28 September 2014

20

Adaptive Statistics – SQL Plan Directives

2014 © Trivadis

The database engine automatically maintains SQL plan directives They are automatically created They are automatically purged if not used (by default after 53 weeks)

They can be manually managed through DBMS_SPD Flush to disk SQL plan directives temporarily stored in the SGA Change retention period Immediate purge Alter attributes (ENABLED, AUTO_DROP) Pack/Unpack Requirement: ADMINISTER SQL MANAGEMENT OBJECT privilege


21

Adaptive Statistics – Management of SQL Plan Directives

2014 © Trivadis

DBA_SQL_PLAN_DIRECTIVES

DBA_SQL_PLAN_DIR_OBJECTS


22

Adaptive Statistics – SQL Plan Directives in the Data Dictionary

SQL> SELECT type, reason, count(*) 2 FROM dba_sql_plan_directives 3 GROUP BY type, reason;

TYPE REASON COUNT(*)---------------- ------------------------------------ ----------DYNAMIC_SAMPLING SINGLE TABLE CARDINALITY MISESTIMATE 81DYNAMIC_SAMPLING JOIN CARDINALITY MISESTIMATE 180DYNAMIC_SAMPLING GROUP BY CARDINALITY MISESTIMATE 6


Evolution of cardinality feedback

Used for single-table cardinalities as well as join cardinalities

Information about misestimates might be persisted through SQL plan directives For misestimates due to table functions no information is stored

V$SQL.IS_REOPTIMIZABLE Y means that the next execution will trigger a reoptimization N means that no reoptimization is necessary R means that reoptimization information is available, but reporting mode was enabled

28 September 2014

23

Adaptive Statistics – Statistics Feedbackstatistics_feedback1.sqlstatistics_feedback2.sql


Enabled when PARALLEL_DEGREE_POLICY = ADAPTIVE

Assessment of the auto DOP after the first execution

If auto DOP is suboptimal, the next execution will trigger a reoptimization

28 September 2014

24

Adaptive Statistics – Performance Feedback


Evolution of dynamic sampling

Used for single-table cardinalities as well as join and group-by cardinalities

The query optimizer decides when and how to use dynamic statistics

The data collected through dynamic statistics is shared through the result cache

The sampling can’t exceed a given amount of time The maximum is determined based on past executions (cursor cache, AWR)

To use the new features OPTIMIZER_DYNAMIC_SAMPLING has to be set to 11

28 September 2014

25

Adaptive Statistics – Dynamic Statisticsdynamic_statistics.sql

2014 © Trivadis

Text Text Adaptive query optimization introduced a number of very interesting features

The query optimizer is getting more and more dynamic


26

Summary

2014 © Trivadis

Questions and answers ...

2013 © Trivadis

BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA

Christian Antognini

Senior Principal Consultant

[email protected]


27

adaptive query optimization

Technology