safe harbor statement · the oracle optimizer in oracle database 12c 12c optimizer column usage...
TRANSCRIPT
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
2
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Oracle Database 18cAnd other unrelated topics
Nigel BaylissOptimizer PMhttp://blogs.oracle.com/optimizer
@vldbb
September 2018
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Oracle Database 18c/ADWC
Navigating the Adaptive Optimizer
Oracle Database 12c
Fourth topic
Fifth topic
1
2
3
4
5
4
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 5
A Subjective Guide
New Optimizer Features and Enhancements
Oracle Database 11g
Oracle Database 12cRelease 1
Oracle Database 12cRelease 2
Oracle Database 18c
[12.2.0.2]
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 6
A Subjective Guide
New Optimizer Features and Enhancements
Oracle Database 11g
Oracle Database 12cRelease 1
Oracle Database 12cRelease 2
Oracle Database 18c
A focus on Oracle Autonomous Data Warehouse Cloud(ADWC)
https://cloud.oracle.com/en_US/datawarehouse
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Oracle Database 18c
7
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Oracle Database 18c
Top-N Approximate Queries
Plan Stability in Standard Edition
ADW…
1
2
3
8
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Top-N Approximate Query
9
• Approximate query processing background
• Approximate results for common top n queries
– How many approximate page views did the top five blog posts get last week?
–What were the top 50 customers in each region that spend most?
• Using much less resource yet yielding high accuracy
–Order of magnitudes smaller memory usage
–Max error < 0.7% * total number of rows
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Top-N Approximate Query
10
• New approximate functions APPROX_COUNT(), APPROX_SUM(), APPROX_RANK()
SELECT deptno, job, APPROX_SUM(sal), APPROX_COUNT(*)FROM empGROUP BY deptno, jobHAVING APPROX_RANK(partition by deptno ORDER BY APPROX_SUM(sal) desc) <= 2AND APPROX_RANK(partition by deptno ORDER BY APPROX_COUNT(*) desc) <= 3;
https://github.com/oracle/oracle-db-examples/tree/master/optimizer/topn
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Oracle Database 18c
Top-N Approximate Queries
Plan Stability in Standard Edition
ADW…
1
2
3
11
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Plan Stability in Standard EditionStored outlines an plan stability in SE
• Stored outlines were deprecated in Oracle Database 11g Release 1
• Stored outlines are still often used in Oracle Standard Edition to control SQL execution plans
• Stored outlines still function in Oracle Database 18c
12
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Subset of SQL Plan Management in Standard EditionPlan stability for Standard Edition
• Prior to Oracle Database 18c, SPM was an EE-only feature
• Oracle Database 18 Release 1 Standard Edition includes a subset of EE SQL plan management features
– Similar capabilities to stored outlines
• Easy migration from stored outlines to SQL plan baselines– DBMS_SPM.MIGRATE_STORED_OUTLINE()
• Control plans using SPM rather than stored outlines...
13
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 14
Stored outlines are no longer needed
Example Use-Case with SPM in Standard Edition
FULL INDEX
HintedTest
Query
INDEX
SQL Plan Baseline
SELECT SUM(value)FROM salesWHERE region_id = :1
Better plan has been hinted
Plan is nowfully constrained
https://github.com/oracle/oracle-db-examples/tree/master/optimizer/spm_plan_control/SE
BUG 27500184 - CURSOR WITH DIFFERENT PLAN TO SQL PLAN BASELINE IS INVALIDATED REPEATEDLY IN SE
SELECT /*+ INDEX(..) */ SUM(value)
FROM salesWHERE region_id = :1
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 15
Capability is similar to stored outlines
Subset of SQL Plan Management in Standard Edition
Standard Edition Enterprise Edition
Multiple SQL plan baselines per SQL statement
Plan Evolution
Auto Capture
Capture from SQL Tuning Set
Capture from cursor cache
Capture from AWR
Auto purge unused SQL plan baselines
Export/Import (pack/unpack)
Convert stored outlines to SPBs
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
ADWC
16
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Extended Online Statistics GatheringADWC Focus
17
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Online Statistics Gathering: Pre-18c Behavior
• Gather statistics on load for empty tables only
• Non-partitioned table or extended syntax INSERT
• Direct path load only – single table or partition (with extended syntax)
– INSERT /*+ APPEND */
– Create Table As Select (CTAS)
• Histograms are not created
– Use GATHER AUTO after load to create them
• Creates synopses automatically if incremental statistics are enabled
18
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Extended Online Statistics Gathering
• Gathers statistics on load for non-empty tables
• Direct path load (DPL) only
– INSERT /*+ APPEND */
• Existing histograms are maintained automatically
19
https://github.com/oracle/oracle-db-examples/tree/master/optimizer/autonomous/stats_on_load
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Statistics-Based Query TransformationADWC Focus
20
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Statistics-Based Query Transformation
• For aggregate queries such as MIN/MAX/COUNT, the Oracle Database may need to scan all rows to calculate the value (if there are no WHERE-clause predicates)
• Table scans can be time-consuming
• When statistics are gathered, the database retains information on how may rows there are as well as details of column data such minimum and maximum values and the number of distinct values (NDV)
• If the database is sure that statistics are up-to-date, then it is in a position to answer some queries using the information it has already gathered
21
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 22
ADWC Example
Statistics-Based Query Transformation
select max(num0),min(num1) from fact1;
---------------------------------------------------------------| Id | Operation | Name |---------------------------------------------------------------| 0 | SELECT STATEMENT | || 1 | RESULT CACHE | cpuu29wy174jfbu3a7zcxzkw12 || 2 | VIEW | VW_SQT_65BBF4BE || 3 | SORT AGGREGATE | || 4 | PX COORDINATOR | || 5 | PX SEND QC (RANDOM) | :TQ10000 || 6 | SORT AGGREGATE | || 7 | PX BLOCK ITERATOR | ||* 8 | TABLE ACCESS FULL| FACT1 |---------------------------------------------------------------
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Supported
• Tables and views
• Create table as select
• MIN/MAX [Number/Date]
• approx_count_distinct
• COUNT()
23
https://github.com/oracle/oracle-db-examples/tree/master/optimizer/autonomous/stats_answering
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Restrictions• Queries must not have:
– WHERE clause
– Group By
– Join
– Non-merged view
– Connect By
• Not supported:
– Fixed tables
– GTTs
– WITH (cursor-specific temp table)
– Query TABLE function
– Table partition (via extended table syntax) – not in ADWC yet anyway
– SYS objects
24
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Restrictions
• Transformation is not used inside transaction
– YES: DML, commit, Q1
– NO: DML, Q1
• Stats branch is serial in PX queries
25
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Hint Parameters
26
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Hint Parameters Are No Longer Hidden
• optimizer_ignore_hints
• optimizer_ignore_parallel_hints
27
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Navigating the Adaptive Optimizer
28
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 29
Oracle Database 11g11g Optimizer
Column Usage Statistics
Answers the question,"How are columnsused in queries?"
This metadata is persisted to the data dictionary G a t h e r S t a t i s t i c s
F O R A L L C O L U M N SS I Z E A U T O( t h e d e f a u l t )
HistogramsBetter statistics forskewed datasets
Learn frompreviousexecutions (on aquery-by-querybasis)
Information isheld in memoryand lost when query is aged out
Cardinality Feedback
The DataDictionary
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 30
Dynamic Sampling In Oracle Database 11g11g Optimizer
Column Usage Statistics
Histograms
DynamicSampling
Sample data toaccount for missingor inadequatestatistics/* OPT_DYN_SAMP */
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 31
The Adaptive Optimizer
New in Oracle Database 12c Release 1
12c Optimizer
AdaptiveStatistics
Optimizer Adaptive Features
Change SQL execution plans at runtimeLearn from previous executions and choose better SQL execution plans
AdaptivePlans
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 32
From Oracle Database 12cRelease 1
The Oracle Optimizer in Oracle Database 12c12c Optimizer
Column Usage Statistics
SQL Plan Directives
The lessons learnedfrom previous executions are SHAREDbetween queries
Statistics feedbackincludes joins
DynamicStatistics"DS_SVC"
AdaptivePlans
Adapt at runtime
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 33
From Oracle Database 12cRelease 1
Oracle Optimizer Adaptive Features12c Optimizer
Column Usage Statistics
SQL Plan Directives
DynamicStatistics"DS_SVC"
G a t h e r S t a t i s t i c s
Column groupstatistics createdautomatically
G a t h e r S t a t i s t i c sF O R A L L C O L U M N SS I Z E A U T O
Histograms
Statistics feedbackincludes joins
AdaptivePlans
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Viewing Optimizer Metadata
• DBA_SQL_PLAN_DIRECTIVES
• HISTOGRAM column in DBA_TAB_COLUMNS
• DBA_STAT_EXTENSIONS
• SELECT DBMS_STATS.REPORT_COL_USAGE('SH,'EMP')FROM DUAL;–SYS.COL_USAGE$
34
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 35
Example Parameter SettingsFYI - Not recommended
Oracle Database 12c Release 1 12c Optimizer
SQL Plan Directives
Statistics feedbackincludes joins
DynamicStatistics"DS_SVC"_optimizer_dsdir_usage_control = 0
_optimizer_ads_use_result_cache = FALSE
Removes... /*+ RESULT_CACHE ... */
"_fix_control"='20243268: OFF'
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 36
FYI - Not recommendedX-Instance and PX
Oracle Database 12c Release 1 12c Optimizer
DynamicStatistics"DS_SVC"
Adaptive DynamicStats for PX and X-Instance Queries
"_fix_control"='12914055:OFF'
12.2…optimizer_adaptive_statistics = FALSE
(_optimizer_ads_for_pq)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Oracle Database 12c Release 2
37
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 38
Controlling adaptive features
Oracle Database 12c Release 1
12c Optimizer
AdaptiveStatistics
Optimizer Adaptive Features
Change plans at runtime Learn from previous executions
OPTIMIZER_ADAPTIVE_FEATURES
AdaptivePlans
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 39
Finer control of adaptive features – new database parameters
From Oracle Database 12c Release 2
12c Optimizer
AdaptiveStatistics
Optimizer Adaptive Features
Change plans at runtime Learn from previous executions
OPTIMIZER_ADAPTIVE_FEATURES Obsolete
OPTIMIZER_ADAPTIVE_PLANS OPTIMIZER_ADAPTIVE_STATISTICS
AdaptivePlans
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
OPTIMIZER_ADAPTIVE_STATISTICS (FALSE)
40
New default behavior
From Oracle Database 12c Release 2
12c Optimizer
AdaptiveStatistics
Optimizer Adaptive Features
Change plans at runtime Learn from previous executions
OPTIMIZER_ADAPTIVE_PLANS (TRUE)
AdaptivePlans
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 41
Oracle Database 12cRelease 2
Oracle Optimizer Adaptive Features – Fully Adaptive12c Optimizer
Column Usage Statistics
SQL Plan Directives
DynamicStatistics"DS_SVC"
G a t h e r S t a t i s t i c s
Column groupstatistics createdautomatically
G a t h e r S t a t i s t i c sF O R A L L C O L U M N SS I Z E A U T O
Histograms
Statistics feedbackincludes joins
AdaptivePlans
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 42
Oracle Database 12cRelease 2
Oracle Optimizer Adaptive Features – Fully Adaptive12c Optimizer
Column Usage Statistics
SQL Plan Directives
DynamicStatistics"DS_SVC"
G a t h e r S t a t i s t i c sG a t h e r S t a t i s t i c sF O R A L L C O L U M N SS I Z E A U T O
Histograms
Statistics feedbackincludes joins
AdaptivePlans
No.../*+ RESULT_CACHE */
DBA_SQL_PLAN_DIRECTIVES
instead
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 43
Oracle Database 12cRelease 2
Oracle Optimizer Adaptive Features – New Defaults12c Optimizer
Column Usage Statistics
SQL Plan Directives
Statistics feedbacknot including JOINS
G a t h e r S t a t i s t i c sG a t h e r S t a t i s t i c sF O R A L L C O L U M N SS I Z E A U T O
Histograms
DynamicStatistics
AdaptivePlans
/* OPT_DYN_SAMP */
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 44
Oracle Database 12cRelease 2
Oracle Optimizer Adaptive Features – New Defaults12c Optimizer
Column Usage Statistics
SQL Plan Directives
Statistics feedbacknot including JOINS
G a t h e r S t a t i s t i c sG a t h e r S t a t i s t i c sF O R A L L C O L U M N SS I Z E A U T O
Histograms
DynamicStatistics
AdaptivePlans
PX and X-Instance also use /* OPT_DYN_SAMP */
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 45
Oracle Database 12cRelease 2
Oracle Optimizer Adaptive Features – New Defaults12c Optimizer
Column Usage Statistics
SQL Plan Directives
Statistics feedbacknot including JOINS
G a t h e r S t a t i s t i c sG a t h e r S t a t i s t i c sF O R A L L C O L U M N SS I Z E A U T O
Histograms
DynamicStatistics
AdaptivePlans
DBMS_STATS preference:AUTO_STAT_EXTENSIONSDefault is OFF
Defaults:
OPTIMIZER_ADAPTIVE_PLANS=TRUE
OPTIMIZER_ADAPTIVE_STATISTICS=FALSE
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 46
Oracle Database 12cRelease 2
Oracle Optimizer Adaptive Features12c Optimizer
Column Usage Statistics
SQL Plan Directives
Statistics feedbacknot including JOINS
G a t h e r S t a t i s t i c sG a t h e r S t a t i s t i c sF O R A L L C O L U M N SS I Z E A U T O
Histograms
DynamicStatistics
OPTIMIZER_ADAPTIVE_PLANS=FALSE
OPTIMIZER_ADAPTIVE_STATISTICS=FALSE
= Always Enabled
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 47
Oracle Database 12cRelease 2
Oracle Optimizer Adaptive Features12c Optimizer
Column Usage Statistics
SQL Plan Directives
Statistics feedbacknot including JOINS
G a t h e r S t a t i s t i c sG a t h e r S t a t i s t i c sF O R A L L C O L U M N SS I Z E A U T O
Histograms
DynamicStatistics
OPTIMIZER_ADAPTIVE_PLANS=FALSE
OPTIMIZER_ADAPTIVE_STATISTICS=FALSE
Bug 25058954 : [BUGDB] HIGH FOREGROUND CPU IN 12.2 UNDER SQL PLAN DIRECTIVE
Fixed in 12.2.0.1
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Oracle Database 12c Release 1Options and Recommendations
48
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Oracle Database 12c Release 1
• Set optimizer_adaptive_features=FALSE–Prioritizes stability and new features are available
–optimizer_dynamic_sampling = 0 is not usually necessary but it's an option
–Upgrading to 12.2 defaults: optimizer_adaptive_plans=TRUE and statistics feedback will kick in (without joins)
• Set optimizer_features_enable='11.2.0.4'–"Quick fix"
–More testing required for upgrade 12.1->12.2
49
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 50
Adaptive Features = FALSE
Oracle Optimizer Adaptive Features – Release 112c Optimizer
Column Usage Statistics
G a t h e r S t a t i s t i c sG a t h e r S t a t i s t i c sF O R A L L C O L U M N SS I Z E A U T O
Histograms
DynamicStatistics
OPTIMIZER_ADAPTIVE_FEATURES=FALSE
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Oracle Database 12c Release 1 - Recommendation• Install latest Proactive Bundle Patch (>=October 2017)
– e.g. January 2018, Patch 27010930 - Database Proactive Bundle Patch 12.1.0.2.180116
• Recommendations for Adaptive Features in Oracle Database 12c Release 1 (Adaptive Features, Adaptive Statistics and 12c SQL Performance) (Doc ID 2312911.1)
– _fix_control='26664361:7','16732417:1','20243268:1'
• Remove optimizer_adaptive_features parameter from pfile/spfile
• Set the following optimizer parameters (these are the defaults in 12cR2 and 18c):
– optimizer_adaptive_plans=TRUE
– optimizer_adaptive_statistics=FALSE
– Because: BUG 27626925 - OPTIMIZER ADAPTIVE STATS DEFAULT FALSE NOT HONORED WHEN ENABLED IN OCT OR JAN BP
• On the database, execute the following ('OFF' is the default)
– EXEC DBMS_STATS.SET_GLOBAL_PREFS('AUTO_STAT_EXTENSIONS','OFF')
• Testing will be required if patches are enabled because plans may change
51
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 52
MT-Related Issues – pre Oracle Database 12c Release 2
BUG 24321547 - MODIFIED UNDERSCORE PARAMETER VALUES NOT VISIBLE IN PDB AFTER RESTART Available in 12.1.0.2 as a one-off, and included in Proactive Bundle Patch 180717 onwards.
BUG 20700587 - PDB DOESN'T TAKE UNDERSCORE PARAMETERS DURING DATABASE START
BUG 20020430 - PROBLEM WITH UMBRELLA PARAMETER CASCADING
BUG 28519141 - ALTER SESSION SET CONTAINER DOES NOT SET PARAMETERS DEPENDING ON PDB PARAMETERS
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 53
Using 12.2-style parameters on selected PDBs
Recipe - pre Oracle Database 12c Release 2
Clean
REM will report errors if the parameters are not currently included in the parameter filealter system reset "_fix_control";alter system reset optimizer_adaptive_features;alter system reset optimizer_adaptive_plans;alter system reset optimizer_adaptive_statistics;alter system reset "_optimizer_dsdir_usage_control";alter system reset "_optimizer_use_feedback_for_join";alter system reset "_optimizer_ads_for_pq";CDB
REM set defaults for fix control and dependent hidden parametersalter system set "_fix_control" = '26664361:0','16732417:0','20243268:0' scope=spfile;alter system set "_optimizer_dsdir_usage_control" = 126 scope = spfile;alter system set "_optimizer_use_feedback_for_join" = true scope = spfile;alter system set "_optimizer_ads_for_pq" = true scope = spfile;PDB
REM set recommended values for fix control and parameters, plus dependent hidden parametersalter system set "_fix_control" = '26664361:7','16732417:1','20243268:1' scope=spfile;alter system set optimizer_adaptive_plans = true scope = spfile;alter system set optimizer_adaptive_statistics = false scope = spfile;alter system set "_optimizer_dsdir_usage_control" = 0 scope = spfile;alter system set "_optimizer_use_feedback_for_join" = false scope = spfile;alter system set "_optimizer_ads_for_pq" = false scope = spfile;
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 54
Default Adaptive Features
Oracle Optimizer Adaptive Features – Release 1 + BP12c Optimizer
Column Usage Statistics
SQL Plan Directives
Statistics feedbackno join card f/b
DynamicStatistics
AdaptivePlans
G a t h e r S t a t i s t i c sG a t h e r S t a t i s t i c sF O R A L L C O L U M N SS I Z E A U T O
Histograms
Defaults:
OPTIMIZER_ADAPTIVE_PLANS=TRUE
OPTIMIZER_ADAPTIVE_STATISTICS=FALSE
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Oracle Database 12c and 18c - Recommendation
• Use the defaults–OPTIMIZER_ADAPTIVE_PLANS=TRUE
–OPTIMIZER_ADAPTIVE_STATISTICS=FALSE
55
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Adaptive Statistics
• Queries are longer-running
• Queries are complex
• Schema is complex
• Data distribution is complex
• Ad-hoc query environments–SQL plan directives allow the optimizer to learn from
queries and share this information with others
56
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
System Statistics
• The default settings are appropriate for most systems
• Oracle recommends you don't gather them–EXADATA option can be useful in pure
DW workloads on engineered systems
• If you do not intend to gather system statistics, delete them and restart your initial database
• Always test any changes you make
57
https://blogs.oracle.com/optimizer/should-you-gather-system-statistics
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Oracle Database 12c Release 2
58
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
4
Oracle Database 18c
Cost-based OR Expansion Transformation
Incremental Statistics
SQL Plan Management
Statistics Advisor
1
2
3
59
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Cost-based OR Expansion Transformation
• Example – Convert query containing disjunctions (ORs) to UNION-ALLs
60
SELECT * FROM prods T1, shops T2 WHERE (T1.vendid1 = 10 OR T2.vendid2 = 20) AND T1.delivery_batch = T2.delivery_batch;
SELECT * FROM prods T1, shops T2 WHERE T1.vendid1 = 10AND T1.delivery_batch = T2.delivery_batch UNION ALLSELECT * FROM prods T1, shops T2 WHERE T2.vendid2 = 20 AND T1.delivery_batch = T2.delivery_batchAND LNNVL(T1.vendid1=10);
Note: The LNNVL() function avoids duplicates
Opportunity to find newaccess paths
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Cost-Based OR Expansion Transformation
• Limitations of pre 12.2 OR expansion– Cannot leverage other query transformations on top of OR expansion
– Cannot parallelize CONCATENATION branches
• New in 12.2:– Uses UNION ALL instead of CONCATENATION
–Opportunity for other transformations to be applied on top of UNION ALL branches
– Can execute UNION ALL branches in parallel
–OR expansion using Disjunctive Normal Form (DNF) and UNION ALL
61
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 62
Comparing 11g OR Expansion with 12c Cost-Based OR Expansion
---------------------------------------------------| Id | Operation | Name |---------------------------------------------------| 0 | SELECT STATEMENT | || 1 | SORT AGGREGATE | || 2 | CONCATENATION | ||* 3 | HASH JOIN | || 4 | TABLE ACCESS FULL | T_4K || 5 | MERGE JOIN CARTESIAN | ||* 6 | INDEX FAST FULL SCAN | T_4K_CONCAT4 || 7 | BUFFER SORT | || 8 | INDEX FAST FULL SCAN| T_10K_HUNDRED ||* 9 | HASH JOIN | || 10 | NESTED LOOPS | ||* 11 | TABLE ACCESS FULL | T_4K ||* 12 | INDEX RANGE SCAN | T_10K_HUNDRED ||* 13 | INDEX FAST FULL SCAN | T_4K_CONCAT4 |---------------------------------------------------
---------------------------------------------------------| Id | Operation | Name |---------------------------------------------------------| 0 | SELECT STATEMENT | || 1 | SORT AGGREGATE | ||* 2 | HASH JOIN | || 3 | VIEW | VW_JF_SET$9CE2290B || 4 | UNION-ALL | ||* 5 | HASH JOIN | ||* 7 | INDEX FAST FULL SCAN | T_4K_CONCAT4 ||* 9 | TABLE ACCESS FULL | T_4K ||* 10 | HASH JOIN | ||* 11 | TABLE ACCESS FULL | T_4K || 12 | INDEX FAST FULL SCAN | T_4K_CONCAT3 || 13 | INDEX FAST FULL SCAN | T_10K_HUNDRED |---------------------------------------------------------
SELECT COUNT(*)FROM t_4k T1, t_4k T2, t_10k T3WHERE T1.ten = T2.ten AND
T1.hundred = T3.hundred AND(T1.unique1 = 10 OR T2.unique2 = 20);
Pre-12.2 12.2
Note that T_10K_HUNDRED is scanned only once in the 12.2 case.
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
4
Oracle Database 18c
Cost-based OR Expansion Transformation
Incremental Statistics
SQL Plan Management
Statistics Advisor
1
2
3
63
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 64
Incremental Maintenance for Partitioned Tables
Global table statisticsmust be maintained…
• Total #rows• Min/Max• Number of distinct values
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 65
Incremental Maintenance for Partitioned Tables
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Incremental Maintenance Overview for Partitioned Tables • Incremental maintenance (introduced in 11.1) significantly expedites statistics
gathering on partitioned tables– Only partitions with stale statistics have statistics re-gathered
– Global statistics derived from partition level statistics
• Typical usage– Tables range partitioned on dates and only the last few partitions have data changes
• exec dbms_stats.set_table_prefs (user,'apptab','INCREMENTAL','TRUE')
• exec dbms_stats.set_global_prefs('INCREMENTAL_STALENESS', 'USE_STALE_PERCENT')
66
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Synopses
• Each partition maintains synopses in incremental mode
• Synopses are detailed information about Number of Distinct Values (NDV)
• Global level NDV is derived by aggregating synopses
• Synopses stored on disk in SYSAUX tablespace
• However: synopses can be large when …
– Large number of partitions
–Many columns in the table
– NDV in each partition is high
67
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Controlling the Synopsis Generation Algorithm in 12.2
• Previous algorithm is still supported!
• DBMS_STATS preference provided to control which algorithm to use
– Preference name: APPROXIMATE_NDV_ALGORITHM
– Preference value:
• “REPEAT OR HYPERLOGLOG” (default):– If synopses of old format exists, continue using the pre-12.2 algorithm
– If no synopses yet or synopses are in new format, use the new 12.2 algorithm
• “ADAPTIVE SAMPLING”– Force to use pre-12.2 adaptive sampling algorithm
– Any synopses in new 12.2 format are deleted and the stats are regathered
– Preserves behavior prior to 12.2
• “HYPERLOGLOG”– Force use of new algorithm
68
DBMS_STATS.SET_TABLE_PREFS…
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Preference: HYPERLOGLOGReplacing old format with new
69
SALESSYSAUX Tablespace
P1
P2
P1 has a synopsisin pre-12.2 format
Re-gatherStatistics P1
New synopsisfor P1 in compact12.2 format
PartitionedTable
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Preference: REPEAT OR HYPERLOGLOGCoexisting Old and New Formats
70
SALESSYSAUX Tablespace
P1
P2
P1 has a synopsisin pre-12.2 format
Synopsis forP2 in compact12.2 format
GatherStatistics
PartitionedTable
https://github.com/oracle/oracle-db-examples/tree/master/optimizer/new_synopses
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
4
Oracle Database 18c
Cost-based OR Expansion Transformation
Incremental Statistics
SQL Plan Management
Statistics Advisor
1
2
3
71
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
SQL Plan Management AWR Capture
72
• Manually capture SQL plans from AWR:
– dbms_spm.load_plans_from_awr(begin_snap => 5, end_snap => 8);
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
SQL Plan Baseline Auto Capture
73
• In 12.1, automatic capture applies to all repeatable queries
– Includes many non-critical queries, wasting storage space in SYSAUX
• In 12.2, capture only what you need
– Specific applications and schemas
– Filter the SQL statements you need• SQL Text
• Parsing Schema
• Module
• Action
ExecutionPlan
Execute
SQL PlanBaselines
12.2
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
SQL Plan Management Auto Capture
74
• Set filters for automatic capture:
– dbms_spm.configure('AUTO_CAPTURE_PARSING_SCHEMA_NAME',
'SCOTT', TRUE);
– Current filter settings stored in DBA_SQL_MANAGEMENT_CONFIG
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
4
Oracle Database 18c
Cost-based OR Expansion Transformation
Incremental Statistics
SQL Plan Management
Statistics Advisor
1
2
3
75
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Inferior Statistics Cause Problems
• Inferior statistics cause query performance problems
• For example:
– Stale, out of date statistics
–Missing statistics
– Poor quality – ignoring recommended approaches
– Inconsistencies between tables and indexes
– Inconsistencies between primary-key/foreign-key relationships
• Problems can be difficult to diagnose– Resulting in time-consuming analysis and MOS Service Requests
76
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Why Might You Have Inferior Statistics?• Oracle continuously enhances statistics gathering features but
enhancements can be overlooked post-upgrade
– For example: Auto Sample Size
• DBAs may use legacy scripts to gather statistics manually
– Reluctance to change “proven” procedures
• Statistics gathering can be overlooked because of inexperience
• Statistics might not be maintained during batch processing– Developer inexperience or perceived lack of time in batch window
• Not all DBAs read best practice white papers
• “Inherited” systems where nobody understands the scripts
77
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
The Optimizer Statistics Advisor
• Rules that have not been followed will generate findings
• Findings can lead to specific recommendations
• Recommendations can be implemented using actions
78
Rules Findings Recommendations Actions
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
A Specific Example
79
Use default parameters in DBMS_STATS.GATHER...STATS
ESTIMATE_PERCENT=>0.01 is used in maintenance scripts
Use default ESTIMATE_PERCENT (AUTO_SAMPLE_SIZE)
GATHER_DATABASE_STATS(obj_filter_list)
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Optimizer Statistics Advisor Task
• Advisor task gathers and stores data
– Low performance overhead
– DBA can choose which rules are applicable using filters
– Filters can also constrain the scope of the task (tables, schemas and so on)
80
DataDictionary
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Optimizer Statistics Advisor Reports and Action Script
• Reports are generated from retained task data
• Generate the HTML or text-based report
• Generate the action SQL script
81
TaskData
HTML
SQL
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Flow Summary
• Create task:– dbms_stats.create_advisor_task('q1_advisor_run');
• Optionally, configure a filter:– dbms_stats.configure_advisor_filter ('q1_advisor_run' ...);
• Manually execute the advisor:– dbms_stats.execute_advisor_task('q1_advisor_run');
• Generate the report:– dbms_stats.report_advisor_task('q1_advisor_run', ftype=>'html' ...);
– Use AUTO_STATS_ADVISOR_TASK as task name for automatic task
• Generate the action script:– dbms_stats.script_advisor_task('q1_advisor_run');
82
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
83
Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | 84