real-world performance training · rwp training extreme dw hands-on exercise: star transformation...
TRANSCRIPT
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Real-World Performance TrainingHands-On Exercise
Real-World Performance Team
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Solution Progress, Step 1Initial Observations
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Baselineq.low.sql
Runs in 5 seconds
Nested Loops joins and B*Tree index access on fact
table
Almost all the time on random I/O against fact table
index and table
Cardinality estimate problem on PART
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Baselineq.med.sql
Runs in 183 seconds
Nested Loops joins and B*Tree index access on fact
table
Almost all the time on random I/O against fact table
index and table
Cardinality estimate problem on PART
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Baselineq.high.sql
Runs in 474 seconds
Nested Loops joins and B*Tree index access on fact
table
Almost all the time on random I/O against fact table
index and table
Cardinality estimate problem on PART
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Progress
q.low.sql q.med.sql q.high.sql Total
Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Solution Progress, Step 2Gather Stats on PART
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Stats on PARTq.low.sql
Runs in 3 seconds
Nested Loops joins and B*Tree index access on fact
table
Now performing hash join to DATE_DIM
Better cardinality estimates on PART, but still off
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Stats on PARTq.med.sql
Runs in 3 seconds
Now using Bloom Filter for PART and HASH JOIN
Hash Joins to other dimensions
Better cardinality estimates on PART, but still off
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Stats on PARTq.high.sql
Runs in 6 seconds
Filtering on SUPPLIER Bloom Filter with Hash Join
Hash Joins to other dimensions
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Progress
q.low.sql q.med.sql q.high.sql Total
Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds
Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Solution Progress, Step 3Extended Stats on PART
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Extended Stats on PARTq.low.sql
Runs in 3 seconds
Now doing NL join between DATE_DIM and LINEORDER
Hash joining to PART and SUPPLIER
Accurate cardinality estimates on PART changed
join orderReference to CUSTOMER
indicates lack of Join Elimination (and constraints)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Extended Stats on PARTq.med.sql
Runs in 3 seconds
Accurate cardinality estimate on PART yield a slightly
different right-deep plan
Accurate cardinality estimates on PART changed
join orderReference to CUSTOMER
indicates lack of Join Elimination (and constraints)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Extended Stats on PARTq.high.sql
Runs in 6 seconds
No plan change from previous step
Reference to CUSTOMER indicates lack of Join
Elimination (and constraints)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Progress
q.low.sql q.med.sql q.high.sql Total
Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds
Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Solution Progress, Step 4Constraints (PK, FK, NOT NULL)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Constraintsq.low.sql
Runs in 3 seconds
No reference to CUSTOMER; Join Elimination takes place
Still doing NL join & index access for
DATE_DIM/LINEORER join
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Constraintsq.med.sql
Runs in 4 seconds
No reference to CUSTOMER; Join Elimination takes place
Now doing Hash Joins with Bloom Filters for both DATE_DIM and PART
dimensions
Scanning all partitions; not performing Bloom pruning
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Constraintsq.high.sql
Runs in 4 seconds
No reference to CUSTOMER; Join Elimination takes place
Now doing Hash Joins with Bloom Filters for both
DATE_DIM and SUPPLIER dimensions
Scanning all partitions; not performing Bloom pruning
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Progress
q.low.sql q.med.sql q.high.sql Total
Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds
Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Constraints 3 seconds 4 seconds 4 seconds 11 seconds
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Solution Progress, Step 5Partition LINEORDER on LO_ORDERDATE
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Partitioningq.low.sql
Runs in 2 seconds
Using BF and Hash Join with PART dimension
Now iterating over fact table partitions for DATE_DIM NL
join
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Partitioningq.med.sql
Runs in 31 seconds
No Bloom Filters used
Execution time partition iteration due to data type
mismatch between DATE_DIM and LINEORDER
No Bloom Pruning
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Partitioningq.high.sql
Runs in 2 seconds
Bloom Filtering, Bloom Pruning, and Hash Joins
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Progress
q.low.sql q.med.sql q.high.sql Total
Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds
Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Constraints 3 seconds 4 seconds 4 seconds 11 seconds
Partitioning 2 seconds 31 seconds 2 seconds 35 seconds
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Solution Progress, Step 6Fix DATE_DIM Data Type
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Data Typesq.low.sql
Runs in 3 seconds
With proper data types, reverted to index access path
and NL joins
Performing partition range scan after NL join between
PART/LINEORDER
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Data Typesq.med.sql
Runs in 0.2 seconds
Right-deep, BF plan with hash joins
Bloom pruning possible with proper data types
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Data Typesq.high.sql
Runs in 1 seconds
Bloom Filtering, Bloom Pruning, and Hash Joins –
same as previous test
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Progress
q.low.sql q.med.sql q.high.sql Total
Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds
Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Constraints 3 seconds 4 seconds 4 seconds 11 seconds
Partitioning 2 seconds 31 seconds 2 seconds 35 seconds
Data Types 3 seconds 0.2 seconds 1 second 4.2 seconds
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Solution Progress, Step 7Bitmap Indexes and Star Transformation
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Star Transformationq.low.sql Runs in 0.2 seconds
Star transformation yielded decent response time
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Star Transformationq.med.sql Runs in 5 seconds
Here, the transformation isn’t engaged but we’re still accessing the fact table via a bitmap index
Spending most of our time in random I/O
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Star Transformationq.high.sql Runs in 45 seconds
Star transformation
Spending most of our time in random I/O. Clearly, star transformation & bitmap indexes isn’t the right choice for this workload. This is
the risk with small bitmap indexes; Oracle often will cost these types of plans in when full scans
are faster
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Progress
q.low.sql q.med.sql q.high.sql Total
Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds
Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Constraints 3 seconds 4 seconds 4 seconds 11 seconds
Partitioning 2 seconds 31 seconds 2 seconds 35 seconds
Data Types 3 seconds 0.2 seconds 1 second 4.2 seconds
Bitmap Index Star Transform 0.2 seconds 5 seconds 45 seconds 50.2 seconds
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Solution Progress, Step 8Exadata, No Star Transformation, No Bitmap Indexes, DoP=4 on LINEORDER
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Exadata DoP 4q.low.sql Runs in 1 second
In parallel, query runs quite fast
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Exadata DoP 4q.med.sql Runs in 0.55 seconds
In parallel, query runs quite fast
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: Exadata DoP 4q.high.sql Runs in 1 second
In parallel, query runs quite fast
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Progress
q.low.sql q.med.sql q.high.sql Total
Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds
Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Constraints 3 seconds 4 seconds 4 seconds 11 seconds
Partitioning 2 seconds 31 seconds 2 seconds 35 seconds
Data Types 3 seconds 0.2 seconds 1 second 4.2 seconds
Bitmap Index Star Transform 0.2 seconds 5 seconds 45 seconds 50.2 seconds
Exadata with DoP=4 1 second 0.55 seconds 1 second 2.55 seconds
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Solution Progress, Step 9Database In-Memory, DoP=4 on LINEORDER
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: DBIM DoP 4q.low.sql Runs in 0.03 seconds
TABLE ACCESS STORAGE FULL becomes TABLE ACCESS INMEMORY
FULL
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: DBIM DoP 4q.med.sql Runs in 1 second
TABLE ACCESS STORAGE FULL becomes TABLE ACCESS INMEMORY
FULL
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: DBIM DoP 4q.high.sql Runs in 0.52 seconds
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Progress
q.low.sql q.med.sql q.high.sql Total
Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds
Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Constraints 3 seconds 4 seconds 4 seconds 11 seconds
Partitioning 2 seconds 31 seconds 2 seconds 35 seconds
Data Types 3 seconds 0.2 seconds 1 second 4.2 seconds
Bitmap Index Star Transform 0.2 seconds 5 seconds 45 seconds 50.2 seconds
Exadata with DoP=4 1 second 0.55 seconds 1 second 2.55 seconds
DBIM with DoP=4 0.03 seconds 1 second 0.52 seconds 1.55 seconds
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Solution Progress, Step 10In-Memory Aggregation (DoP = 1)
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: IMAq.low.sql Runs in 0.03 seconds
The IMA transformation
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: IMAq.med.sql Runs in 0.09 seconds
The IMA transformation
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: IMAq.high.sql Runs in 1 second
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Progressq.low.sql q.med.sql q.high.sql Total
Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds
Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Constraints 3 seconds 4 seconds 4 seconds 11 seconds
Partitioning 2 seconds 31 seconds 2 seconds 35 seconds
Data Types 3 seconds 0.2 seconds 1 second 4.2 seconds
Bitmap Index Star Transform 0.2 seconds 5 seconds 45 seconds 50.2 seconds
Exadata with DoP=4 1 second 0.55 seconds 1 second 2.55 seconds
DBIM with DoP=4 0.03 seconds 1 second 0.52 seconds 1.55 seconds
IMA with DoP=1 0.03 seconds 0.09 seconds 1 seconds 1.12 seconds
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Solution Progress, Step 11Materialized Views In-Memory, Clustered
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: MJV DBIMq.low.sql Runs in 0.017 seconds
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: MJV DBIMq.med.sql Runs in 0.026 seconds
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise: MJV DBIMq.high.sql Runs in 0.127 seconds
You can make things really fast if you take it this far. A total of 0.17
seconds for each of the 3 “typical” star queries!
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Progressq.low.sql q.med.sql q.high.sql Total
Initial Configuration 5 seconds 183 seconds 474 seconds 662 seconds
Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Extended Stats on PART 3 seconds 3 seconds 6 seconds 12 seconds
Constraints 3 seconds 4 seconds 4 seconds 11 seconds
Partitioning 2 seconds 31 seconds 2 seconds 35 seconds
Data Types 3 seconds 0.2 seconds 1 second 4.2 seconds
Bitmap Index Star Transform 0.2 seconds 5 seconds 45 seconds 50.2 seconds
Exadata with DoP=4 1 second 0.55 seconds 1 second 2.55 seconds
DBIM with DoP=4 0.03 seconds 1 second 0.52 seconds 1.55 seconds
IMA with DoP=1 0.03 seconds 0.09 seconds 1 seconds 1.12 seconds
MJV with DBIM 0.017 seconds 0.026 seconds 0.127 seconds 0.17 seconds
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
RWP Training Extreme DW Hands-On Exercise
Objective: Demonstrate multiple steps to achieve query optimization
1. Download SQL*Monitor reports from https://stbeehive.oracle.com/content/dav/st/Real%20World%20Training/Documents/3.5%20Day%20Course/New%20PPTX%20October%202014%20Week%202/handson_sqlmon.zip
2. Show reports in “baseline” and “constraints” section – just open a browser to show them. The demo application will take too long to run them
3. Go to the demo system and launch Star Query Demo. Use Windows for display/VGA purposes and Chrome
4. Click through each of the numbered queries on the list. There 8 steps with 3 queries each; the steps are “Constraints and Stats”, “Extended Stats”, “Partitioning on the right column”, “Star transform w/ bitmap indexes”, “Exadata DoP=4”, “DBIM DoP=4”, “IMA with DoP=1”, and “MV In-Memory DoP=4”
5. You can maximize any window and click on the Description to generate the SQL*Monitor report
If things go wrong:
1. Login to scao08adm01 as oracle
2. Go to /home/oracle/users/jclarke/training/hands_on/solution
3. SQL*plus as DW_JC/DW_JC
4. Run solution.sql
Instructor Notes
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
ConstraintsNOT NULL, Primary Key, and Foreign Key Constraints
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
StatisticsUnlocking, Gathering Stats & Extended Stats
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Star TransformationBitmap Indexes and Star Transformation
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Star TransformationPartitioning LINEORDER on correct column
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Star TransformationMark Tables for In-Memory Column Store and Populate
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Star TransformationCheck In-Memory Population Status