breitling - histograms, myths and facts oracle

8/14/2019 Breitling - Histograms, Myths and Facts Oracle

1/42

Histograms-

Myths and Facts

Wolfgang Breitlingwww.centrexcc.com


2/42

Hotsos Symposium, March 6-9, 2005 Wolfgang Breitling, Centrex Consulting Corporation2

Who am I

Independent consultant since 1996

specializing in Oracle and Peoplesoft setup,administration, and performance tuning

Member of the Oaktable Network

25+ years in database managementDL/1, IMS, ADABAS, SQL/DS, DB2, Oracle

OCP certified DBA - 7, 8, 8i

, 9i

Oracle since 1993 (7.0.12)

Mathematics major at University of Stuttgart


3/42


4/42


Myth or Fact ?

Histograms allow the CBO tomake a better choice between

index access and table scanand are therefore only useful for

indexed columns


5/42


Histogram on non-indexed Column

create table t1 (pk1 number

, pk2 number

, fk1 number, fk2 number

, d1 date

, d2 number

, d3 number, d4 varchar2(2000)

);

create unique index t1p on t1(pk1, pk2);

create index t1x on t1(d2);

select sum(t1.d2*t2.d3*t3.d3)

from t1, t2, t3

where t1.fk1 = t2.pk1

and t3.pk1 = t2.fk1

and t3.d2 = 35

and t1.d3 = 0;


6/42


7/42


Without Histogram

call count cpu elapsed disk query current rows

Parse 1 0.01 0.00 0 0 0 0

Execute 1 0.00 0.00 0 0 0 0

Fetch 2 1.18 6.61 32526 34767 0 1

total 4 1.20 6.61 32526 34767 0 1

Rows Row Source Operation

1 SORT AGGREGATE (cr=34767 r=32526 w=0 time=6615582 us)479 HASH JOIN (cr=34767 r=32526 w=0 time=6615065 us)

2 TABLE ACCESS FULL T1 (cr=23391 r=22253 w=0 time=3317822 us)

61428 HASH JOIN (cr=11376 r=10273 w=0 time=3224089 us)

492 TABLE ACCESS BY INDEX ROWID T3 (cr=486 r=0 w=0 time=2985 us)

492 INDEX RANGE SCAN T3X (cr=4 r=0 w=0 time=471 us)(object id 226985)62500 TABLE ACCESS FULL T2 (cr=10890 r=10273 w=0 time=3105971 us)


8/42


9/42


NAME 1 2

LATCH.cache buffers chains 102258 102486 102166 70308 70305 70329

LATCH.cache buffers lru chain 32533 32607 32467 22210 22223 22233

LATCH.enqueue hash chains 20 39 16 8 6 8

LATCH.enqueues 21 41 16 8 6 8

LATCH.library cache 248 189 286 73 71 69LATCH.library cache pin 90 75 95 33 29 29

LATCH.library cache pin allocation 70 74 70 18 18 18

LATCH.multiblock read objects 32494 32494 32428 22182 22192 22202

LATCH.shared pool 170 134 179 67 58 62

STAT..CPU used by this session 134 107 133 73 95 78

STAT..buffer is not pinned count 33099 33095 33095 22824 22824 22824

STAT..buffer is pinned count 501 501 501 1414 1414 1414STAT..calls to get snapshot scn: kcmgss 1672 1671 1671 1140 1140 1140

STAT..consistent gets 34776 34774 34774 23978 23978 23978

STAT..db block gets 49 31 33 41 33 31

STAT..free buffer requested 32536 32531 32467 22216 22222 22231

STAT..index scans kdiixs1 1 1 1 3 3 3

STAT..no work - consistent read gets 33098 33096 33096 22826 22826 22826

STAT..physical reads 32526 32526 32461 22205 22216 22226

STAT..prefetched blocks 16245 16245 16212 11091 11096 11101

STAT..session logical reads 34825 34805 34807 24019 24011 24009

STAT..table fetch by rowid 494 492 492 992 992 992

STAT..table scan blocks gotten 32612 32612 32612 22253 22253 22253

STAT..table scan rows gotten 125000 125000 125000 62500 62500 62500

STAT..table scans (long tables) 2 2 2 1 1 1

Histogram on non-indexed Column


10/42


Without Histogram

Card Plan

1 SELECT STATEMENT (choose)(Cost 5532)

1 SORT (aggregate)

7,704,878 HASH JOIN (Cost 5532)

31,25031,250 TABLE ACCESS T1 (full)(Cost 3379)

61,639 HASH JOIN (Cost 2061)

492 TABLE ACCESS T3 (by index rowid)(Cost 485)

492 INDEX NON-UNIQUE T3X (range scan)(Columns 1)(Cost 3)

62,500 TABLE ACCESS T2 (full)(Cost 1574)

Rows

1

479

2

61,428

492

492

62,500


11/42


With Histogram

Card Plan

1 SELECT STATEMENT (choose)(Cost 3955)

1 SORT (aggregate)

493 HASH JOIN (Cost 3955)


250 INDEX NON-UNIQUE T3X (range scan)(Columns 1)(Cost 3)


500 NESTED LOOPS (Cost 3469)

2 TABLE ACCESS T1 (full)(Cost 3379)

250 INDEX UNIQUE T2P (range scan)(Columns 1)(Cost 3)

Rows

1

479

492

492

500

503

2

500


12/42


13/42


SELECT B1.SETID, B3.EMPLID , B3.EMPL_RCD, B3.EFFSEQ, B3.EFFDT b3_effdt, B3.CURRENCY_CD,...From PS_GRP_DTL B1, PS_JOB B3, PS_JOBCODE_TBL B4

WHERE B1.SETID = rtrim(:b1,' ')AND B3.EMPLID = rtrim(:b2, ' ')AND B1.SETID = B4.SETIDAND B1.BUSINESS_UNIT IN (B3.BUSINESS_UNIT, ' ')AND B3.BUSINESS_UNIT IN ('BU001', 'BU007', 'BU017', 'BU018', 'BU502', 'BU101', ' ')AND B1.DEPTID IN (B3.DEPTID, ' ')AND B1.JOB_FAMILY IN (B4.JOB_FAMILY, ' ')AND B1.LOCATION IN (B3.LOCATION, ' ')AND B3.JOBCODE = B4.JOBCODE

AND B4.EFF_STATUS = 'A'AND B1.EFFDT = (SELECT MAX(A1.EFFDT) FROM PS_GRP_DTL A1WHERE B1.SETID = A1.SETID AND B1.JOB_FAMILY = A1.JOB_FAMILYAND B1.LOCATION = A1.LOCATION AND B1.DEPTID = A1.DEPTIDAND A1.EFFDT


14/42



Parse 1 0.01 0.00 0 0 0 0Execute 1 0.00 0.00 0 0 0 0

Fetch 2 0.01 0.00 0 68 0 2

total 4 0.02 0.01 0 68 0 2

without histograms


Parse 1 0.01 0.00 0 0 0 0

Execute 1 0.00 0.00 0 0 0 0

Fetch 2 80.11 78.23 0 770842 0 2

total 4 80.12 78.24 0 770842 0 2

with histograms on all indexed columns

Histograms on all Indexed Columns?


15/42



2 SORT ORDER BY

2 FILTER

4 NESTED LOOPS1 NESTED LOOPS

1 TABLE ACCESS BY INDEX ROWID PS_JOB

1 INDEX RANGE SCAN PSAJOB

1 SORT AGGREGATE

1 FIRST ROW

1 INDEX RANGE SCAN (MIN/MAX) PSAJOB1 SORT AGGREGATE

1 FIRST ROW

1 INDEX RANGE SCAN (MIN/MAX) PSAJOB

1 TABLE ACCESS BY INDEX ROWID PS_JOBCODE_TBL

1 INDEX RANGE SCAN PS_JOBCODE_TBL

1 SORT AGGREGATE1 INDEX FAST FULL SCAN PS1JOBCODE_TBL

4 TABLE ACCESS FULL PS_GRP_DTL

2 SORT AGGREGATE

4 INDEX RANGE SCAN PS_GRP_DTL

Without Histograms


16/42



2 SORT ORDER BY

2 TABLE ACCESS BY INDEX ROWID PS_JOB

49225 NESTED LOOPS49033 NESTED LOOPS

237 TABLE ACCESS BY INDEX ROWID PS_JOBCODE_TBL

1313 INDEX RANGE SCAN PS_JOBCODE_TBL

1321 SORT AGGREGATE

3635 INDEX FAST FULL SCAN PS1JOBCODE_TBL

49033 TABLE ACCESS BY INDEX ROWID PS_GRP_DTL136275 INDEX RANGE SCAN PS_GRP_DTL

156161 SORT AGGREGATE

194112 INDEX RANGE SCAN PS_GRP_DTL

191 INDEX RANGE SCAN PS1JOB

1 SORT AGGREGATE

1 FIRST ROW1 INDEX RANGE SCAN (MIN/MAX) PSAJOB

1 SORT AGGREGATE

1 FIRST ROW

1 INDEX RANGE SCAN (MIN/MAX) PSAJOB

With Histograms


17/42


Misplaced Histogram

If that scenario is correct, i.e. histogram on

xxx, then Id test what happens if you get ridof the histogram. How many distinct valuesof xxx are in the table?

thx; good idea; checkin into it...

I dropped the histogram ...did helpthx a lot ....(hope the rest will still work ..)


18/42


Myth or Fact ?

When collecting a histogram,

the greater the number of bucketsthe higher the accuracy


19/42


20/42


Height Balanced Histogram

select min(minbkt), maxbkt

, min(val) minval, max(val) maxval

, sum(rep)sum(rep) sumrepsumrep, sum(sum(repsqrepsq)) sumrepsqsumrepsq

, max(rep) maxrep, count(*) bktndv

from ( select val

, min(bkt) minbkt

, max(bkt) maxbkt

, count(val) rep

, count(val)*count(val) repsq

from ( select val

, ntile() over (order by ) bkt

from . [partition ()] t

where is not null )group by val )

group by maxbkt

order by maxbkt


21/42



val minbkt maxbkt rep repsq1 1 1 7 492 1 1 7 493 1 1 20 4004 1 1 7 495 1 1 7 496 1 1 7 497 1 2 7 498 2 2 7 499 2 2 30 900

10 2 2 7 4911 2 2 7 49

12 2 2 7 49120 1790 density = 1790/120/ 120 = 1.2431e-1

Buckets = 2


22/42



val minbkt maxbkt rep repsq1 1 1 7 492 1 1 7 493 1 2 20 4004 2 2 7 495 2 2 7 496 3 3 7 497 3 3 7 498 3 3 7 49

99 33 55 3030 90090010 5 5 7 4911 5 5 7 49

12 5 5 7 4990 890 density = 890/90/ 120 = 8.2407e-2

Buckets = 5


23/42




10 5 6 7 4911 6 6 7 49

12 6 6 7 49120 1790 density = 1790/120/ 120 = 1.2431e-1

Buckets = 6


24/42



val minbkt maxbkt rep repsq1 1 1 7 492 1 1 7 49

33 11 33 2020 4004004 3 3 7 495 3 4 7 496 4 4 7 497 4 5 7 498 5 5 7 49

99 55 77 3030 90090010 7 8 7 4911 8 8 7 49

12 8 8 7 4970 490 density = 490/70/ 120 = 5.8333e-2

Buckets = 8


25/42



val minbkt maxbkt rep repsq1 1 1 7 492 1 2 7 49

33 22 44 2020 4004004 4 5 7 495 5 5 7 496 5 6 7 497 6 7 7 498 7 7 7 49

99 77 1010 3030 90090010 10 11 7 4911 11 12 7 49

12 12 12 7 4970 490 density = 490/70/ 120 = 5.8333e-2

Buckets = 12


26/42




10 15 15 7 4911 16 16 7 49

12 17 17 7 49

70 490 density = 1 / (2 *120) = 4.1667e-3

Buckets = 17


27/42


28/42


Size SKEWONLY ?

So how are you supposed to gather

histograms and set # of buckets (size)

SKEWONLY ?I highly recommend that all Oracle DBA's use the method_opt=skewonly

option to automatically identify skewed column values and generate

histograms.*

* http://www.dba-oracle.com/oracle_tips_skewonly.htm


29/42


30/42


create table test (n1 number not null);

insert into test (n1) select mod(rownum,5)+1 from dba_objects where

rownum 'FOR ALL COLUMNS SIZE 1');

@colstats test

table column NDV density nulls lo hi

TEST N1 5 2.0000E-01 0 1 5

@histogram test n1

table column EP value

TEST N1 0 1

TEST N1 1 5

Size SKEWONLY ?


31/42


exec DBMS_STATS.DELETE_TABLE_STATS (null, 'test);

exec DBMS_STATS.GATHER_TABLE_STATS (null, 'test', method_opt => 'FOR COLUMNS N1 SIZE SKEWONLY');

@colstats test


TEST N1 5 5.0000E-03 0 1 5

@histogram test n1


TEST N1 20 1

TEST N1 40 2TEST N1 60 3

TEST N1 80 4

TEST N1 100 5

Size SKEWONLY??


32/42


create table test (n1 number not null);

insert into test (n1) select 1 from dba_objects where rownum 'FOR ALL COLUMNS SIZE 1');

@colstats test


TEST N1 1 1.0000E+00 0 1 1

@histogram test n1


TEST N1 0 1

TEST N1 1 1

Size SKEWONLY ?


33/42


exec DBMS_STATS.DELETE_TABLE_STATS (null, 'test);

exec DBMS_STATS.GATHER_TABLE_STATS (null, 'test', method_opt => 'FOR COLUMNS N1 SIZE SKEWONLY');

@colstats test


TEST N1 1 5.0000E-03 0 1 1

@histogram test n1


TEST N1 100 1

Size SKEWONLY???


34/42


Size REPEAT ?

create table test( n1 number not null, n2 number not null, n3 number not null, filler varchar2(4000));

exec dbms_random.seed(134217727);

insert into testselect 100+trunc(20*dbms_random.normal),100+trunc(20*dbms_random.normal),

decode(mod(trunc(10000*dbms_random.normal),16383),0,0,1),dbms_random.string('a',2000)from dba_objectswhere rownum


35/42


Size REPEAT ?

dbms_stats.gather_table_stats (null, 'test', estimate_percent => dbms_stats.auto_sample_size);

table column NDV density lo hi bkts sampleTEST N1 63 1.5873E-02 66 133 1 5.5K

N2 21 4.7619E-02 90 110 1N3 2 5.0000E-01 0 1 1FILLER 5000 1.9976E-04 1

dbms_stats.gather_table_stats (null, 'test', estimate_percent => null, method_opt => 'for columns size skewonly n1,n2,n3');

table column NDV density lo hi bkts sample

TEST N1 64 1.2500E-05 66 133 63 40.0KN2 21 1.2500E-05 90 110 20N3 2 1.2500E-05 0 1 1

FILLER 5000 1.9976E-04 1 5.5K


36/42


Size REPEAT ?

dbms_stats.gather_table_stats (null, 'test', estimate_percent => dbms_stats.auto_sample_size, method_opt => 'for all columns size repeat');

table column NDV density lo hi bkts sampleTEST N1 61 1.2701E-05 66 130 60 5.0K

N2 21 1.2701E-05 90 110 20

N3 2 1.2701E-05 0 1 1FILLER 5000 2.0016E-04 1


TEST N3 88 0TEST N3 4000040000 1


TEST N3 22 0TEST N3 49734973 1


37/42


Size REPEAT ?

S i i G h i


38/42


Statistics Gathering

Use individual statistic gathering commands for more

control

Gather statistics on tables with a 5% sample

Gather statistics on indexes with compute

Add histograms where column data is known to be

skewed

Statistics Gathering: Frequency and Strategy Guidelines Metalink Note 44961.1


39/42

R f


40/42


References

http://www.ixora.com.au/newsletter/2001_04.htm

44961.1 Statistics Gathering: Frequency andStrategy Guidelines

72539.1 Interpreting Histogram Information

100229.1 Indexes - Selectivity

175258.1 How to Compute Statistics on PartitionedTables and Indexes

1031826.6 Histograms: An Overview

M f i b i


41/42


My favorite websites

asktom.oracle.com (Thomas Kyte)

www.evdbt.com (Tim Gorman)

www.ixora.com.au (Steve Adams)

www.jlcomp.demon.co.uk (Jonathan Lewis)

www.hotsos.com (Cary Millsap)

www.miracleas.dk (Mogens Nrgaard) www.oracledba.co.uk (Connor McDonald)

www.oraperf.com (Anjo Kolk)

www.orapub.com (Craig Shallahamer)


42/42

Wolfgang Breitling

[email protected]

Centrex Consulting Corp.

www.centrexcc.com

breitling - histograms, myths and facts oracle

Documents