statistics and the query optimizer
DESCRIPTION
Statistics are the driving force behind all the decisions made by the query optimizer. This session explores how statistics works in order to deliver a better understanding of why you get the performance you do in your systems. Automatica and manual creation as well as automatic and manual maintenance are covered in detail.TRANSCRIPT
![Page 1: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/1.jpg)
Grant Fritchey | www.ScaryDBA.com
www.ScaryDBA.com
Statistics and
the Query Optimization
Grant FritcheyProduct EvangelistRed Gate Software
![Page 2: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/2.jpg)
Grant Fritchey | www.ScaryDBA.com
Goals
Learn how SQL Server creates, stores and maintains statistics
Understand how the optimizer consumes statistics to arrive at an execution plan
Learn various methods for controlling statistics to take more direct control of your queries
![Page 3: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/3.jpg)
Grant Fritchey | www.ScaryDBA.com
[email protected] www.scarydba.com
@gfritchey
www.linkedin.com/in/s
carydba
Grant Fritchey
Product Evangelist, Red Gate Software
![Page 4: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/4.jpg)
Grant Fritchey | www.ScaryDBA.com
STATISTICS IN ACTION
4
![Page 5: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/5.jpg)
Grant Fritchey | www.ScaryDBA.com
CREATE PROC dbo.spAddressByCity @City NVARCHAR(30)AS
SELECT a.AddressID,a.AddressLine1,a.AddressLine2,a.City,sp.[Name] AS StateProvinceName,a.PostalCode
FROM Person.Address AS aJOIN Person.StateProvince AS spON a.StateProvinceID = sp.StateProvinceID
WHERE a.City = @City;
![Page 6: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/6.jpg)
Grant Fritchey | www.ScaryDBA.com 6
![Page 7: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/7.jpg)
Grant Fritchey | www.ScaryDBA.com 7
![Page 8: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/8.jpg)
Grant Fritchey | www.ScaryDBA.com 8
![Page 9: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/9.jpg)
Grant Fritchey | www.ScaryDBA.com
WHAT ARE STATISTICS
9
![Page 10: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/10.jpg)
Grant Fritchey | www.ScaryDBA.com 10
DBCCSHOW_STATISTICS('Person.Address',_WA_Sys_00000004_164452B1);
![Page 11: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/11.jpg)
Grant Fritchey | www.ScaryDBA.com
Key Statistic Terms
Rows Sampled
Steps
Density
Range_hi_key
Range_rows
Eq_rows
Avg_range_rows
Cardinality
Cardinality
Cardinality
11
![Page 12: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/12.jpg)
Grant Fritchey | www.ScaryDBA.com
Cardinality
Selectivity
» Returned values / Total Values
Histogram
Density
» 1/distinct values
Compound Columns
» Selectivity * Selectivity
» Density * Density
+ Trace Flags which we’ll talk about
12
![Page 13: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/13.jpg)
Grant Fritchey | www.ScaryDBA.com
Estimates vs. Cardinality
Comparing variables
Using != and NOT
Predicates comparing columns within a table
Functions w/o constant value
Joins using arithmetic or string operations
No statistics!
Skewed distribution (eh)
13
![Page 14: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/14.jpg)
Grant Fritchey | www.ScaryDBA.com
Statistics are used to…
Determine cardinality which is used to…
» Determine number of rows processed which is used to…— Determine cost of the operation in the plan which is
used to…– Pick the operations used in the plan which is used to
» Return your data in an efficient manner which is used for whatever the heck the business wants
14
![Page 15: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/15.jpg)
Grant Fritchey | www.ScaryDBA.com
CAPTURING BEHAVIOR
15
![Page 16: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/16.jpg)
Grant Fritchey | www.ScaryDBA.com
Capture Mechanisms
Static
» DBCC SHOW_STATISTICS
» Execution Plans (sort of)
Dynamic
» Trace Events
» Extended Events
16
![Page 17: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/17.jpg)
Grant Fritchey | www.ScaryDBA.com
Auto_stats
17
![Page 18: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/18.jpg)
Grant Fritchey | www.ScaryDBA.com
Missing_column_statistics
18
![Page 19: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/19.jpg)
Grant Fritchey | www.ScaryDBA.com
Query_optimizer_estimate
_cardinality
19
![Page 20: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/20.jpg)
Grant Fritchey | www.ScaryDBA.com
STATS ARE CREATED
20
![Page 21: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/21.jpg)
Grant Fritchey | www.ScaryDBA.com 21
SELECT s.name,s.auto_created,s.user_created,s.filter_definition,sc.column_id,c.name AS ColumnName
FROM sys.stats AS sJOIN sys.stats_columns AS sc ON sc.stats_id = s.stats_id
AND sc.object_id = s.object_idJOIN sys.columns AS c ON c.column_id = sc.column_id
AND c.object_id = s.object_idWHERE s.object_id = OBJECT_ID('dbo.Address2')
![Page 22: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/22.jpg)
Grant Fritchey | www.ScaryDBA.com 22
![Page 23: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/23.jpg)
Grant Fritchey | www.ScaryDBA.com 23
![Page 24: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/24.jpg)
Grant Fritchey | www.ScaryDBA.com 24
![Page 25: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/25.jpg)
Grant Fritchey | www.ScaryDBA.com
What?
25
_WA_Sys_00000001_0A1E72EE
![Page 26: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/26.jpg)
Grant Fritchey | www.ScaryDBA.com
What?
26
_WA_Sys_00000001_0A1E72EE
Hexidecimal Object ID
![Page 27: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/27.jpg)
Grant Fritchey | www.ScaryDBA.com
What?
27
_WA_Sys_00000001_0A1E72EE
Hexidecimal Object ID
Table Column Number
![Page 28: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/28.jpg)
Grant Fritchey | www.ScaryDBA.com
What?
28
_WA_Sys_00000001_0A1E72EE
Hexidecimal Object ID
Table Column Number
System
![Page 29: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/29.jpg)
Grant Fritchey | www.ScaryDBA.com
What?
29
_WA_Sys_00000001_0A1E72EE
Hexidecimal Object ID
Table Column Number
System
The US State of Washington… yes I’m serious
![Page 30: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/30.jpg)
Grant Fritchey | www.ScaryDBA.com
YOU CAN CREATE STATS
30
![Page 31: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/31.jpg)
Grant Fritchey | www.ScaryDBA.com 31
CREATE STATISTICS MyStatsON Person.Person (Suffix)WITH FULLSCAN;
![Page 32: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/32.jpg)
Grant Fritchey | www.ScaryDBA.com 32
CREATE STATISTICS MyJrStatsON Person.Person (Suffix)WHERE Suffix = 'Jr.'WITH FULLSCAN;
![Page 33: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/33.jpg)
Grant Fritchey | www.ScaryDBA.com 33
CREATE STATISTICS MyComboStatsON Person.Person (Title,Suffix)WHERE Suffix = 'PhD'WITH FULLSCAN;
![Page 34: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/34.jpg)
Grant Fritchey | www.ScaryDBA.com
…Or Drop Them
34
DROP STATISTICS Person.Person.MyStats;DROP STATISTICS Person.Person.MyJrStats;DROP STATISTICS Person.Person.MyComboStats;
![Page 35: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/35.jpg)
Grant Fritchey | www.ScaryDBA.com
STATS ARE MAINTAINED
35
![Page 36: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/36.jpg)
Grant Fritchey | www.ScaryDBA.com
Dungeons and Dragons
Add 1 Row when 0
Add > 500 Rows when < 500
Add 20% + 500 Rows when > 500
36
![Page 37: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/37.jpg)
Grant Fritchey | www.ScaryDBA.com 37
CREATE INDEX AddressCityON dbo.Address2 (City);
DBCC SHOW_STATISTICS(Address2,AddressCity)
![Page 38: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/38.jpg)
Grant Fritchey | www.ScaryDBA.com 38
SELECT * FROM dbo.Address2 AS aWHERE City = 'Springfield';
![Page 39: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/39.jpg)
Grant Fritchey | www.ScaryDBA.com 39
![Page 40: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/40.jpg)
Grant Fritchey | www.ScaryDBA.com
Advanced D&D
Trace Flag 2371» SQL Sever 2008 R2 Sp1 and above» After 25,000 rows
— Percentage changed based on number of rows— Reducing as the number grows
Trace Flag 4137» Minimum selectivity instead of multiplication on
AND predicates
Trace Flag 9471 (SQL Server 2014)» Minimum selectivity on AND and OR predicates
Trace Flag 9472 (SQL Server 2014)» Independence (pre-2014 behavior)
40
![Page 41: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/41.jpg)
Grant Fritchey | www.ScaryDBA.com
YOU CAN SHOULD
MAINTAIN STATS
MANUALLY
41
![Page 42: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/42.jpg)
Grant Fritchey | www.ScaryDBA.com
Automatic Maintenance
AUTO_CREATE_STATISTICS
AUTO_UPDATE_STATISTICS
» Uses rules in previous section
AUTO_UPDATE_STATISTICS_ASYNC
» Test, test, test
42
![Page 43: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/43.jpg)
Grant Fritchey | www.ScaryDBA.com
sp_updatestats
43
ALTER procedure [sys].[sp_updatestats]@resample char(8)='NO'As…if ((@ind_rowmodctr <> 0) or ((@is_ver_current is notnull) and (@is_ver_current = 0)))…
![Page 44: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/44.jpg)
Grant Fritchey | www.ScaryDBA.com
sp_updatestats
44
ALTER procedure [sys].[sp_updatestats]@resample char(8)='NO'As…
if ((@ind_rowmodctr <> 0)
or ((@is_ver_current is not null) and (@is_ver_current =0)))…
![Page 45: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/45.jpg)
Grant Fritchey | www.ScaryDBA.com
UPDATE STATISTICS
45
UPDATE STATISTICS Person.Address;
![Page 46: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/46.jpg)
Grant Fritchey | www.ScaryDBA.com
UPDATE STATISTICS
46
UPDATE STATISTICS Person.AddressWITH FULLSCAN;
![Page 47: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/47.jpg)
Grant Fritchey | www.ScaryDBA.com
UPDATE STATISTICS
47
UPDATE STATISTICS dbo.Address2 AddressCity;
![Page 48: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/48.jpg)
Grant Fritchey | www.ScaryDBA.com
UPDATE STATISTICS
48
UPDATE STATISTICSdbo.Address2 AddressCityWITH FULLSCAN,NORECOMPUTE;
![Page 49: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/49.jpg)
Grant Fritchey | www.ScaryDBA.com
STATISTICS AND
OPTIMIZER AT WORK
49
![Page 50: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/50.jpg)
Grant Fritchey | www.ScaryDBA.com
Statistics Matter
50
![Page 51: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/51.jpg)
Grant Fritchey | www.ScaryDBA.com
Compatibility Levels
51
ALTER DATABASEAdventureWorks2012 SETCOMPATIBILITY_LEVEL = 120;
![Page 52: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/52.jpg)
Grant Fritchey | www.ScaryDBA.com
Optimizer Switches
52
OPTION (QUERYTRACEON 2312);
DBCC TRACEON(4199);DBCC FREEPROCCACHE();
![Page 53: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/53.jpg)
Grant Fritchey | www.ScaryDBA.com
Recommendations
Compatibility setting
Automatic creation
Automatic update
Update asynchronous where necessary
Use appropriate sample rate
53
![Page 54: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/54.jpg)
Grant Fritchey | www.ScaryDBA.com
Goals
Learn how SQL Server creates, stores and maintains statistics
Understand how the optimizer consumes statistics to arrive at an execution plan
Learn various methods for controlling statistics to take more direct control of your queries
![Page 55: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/55.jpg)
Grant Fritchey | www.ScaryDBA.com
Resources
Scarydba.com/resources
Understanding SQL Server Cardinality Estimations
Fixing Cardinality Estimation Errors
Statistics Used by the Optimizer
First look at the query_optimizer_estimate_cardinality XE Event
Changes to automatic update statistics inSQLServer – traceflag 2371
Cardinality Estimation for Multiple Predicates
55
![Page 56: Statistics and the Query Optimizer](https://reader033.vdocument.in/reader033/viewer/2022052906/558b265fd8b42a310a8b45be/html5/thumbnails/56.jpg)
Grant Fritchey | www.ScaryDBA.com
Questions?
56