seas2009 query optimization final

65
SQL Server Enterprise Architect Summit  June 09-12, 2009 Shanghai, China SEAS09 SEAS09

Upload: serkan-kiraci

Post on 10-Apr-2018

218 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 1/65

SQL Server

Enterprise ArchitectSummit

 June 09-12, 2009Shanghai, China

SEAS09SEAS09

Page 2: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 2/65

Query Optimization and

Query Tuning in SQLServer 2008

Howard YinPrincipal Program Manager

SQL Server Customer Advisory Team

Microsoft Corp.

Page 3: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 3/65

3

SEAS09SEAS09

| SQL Server Customer Advisory Team

Agenda

• SQLCAT Introduction• Query Optimizer Architecture

• Query Plan Analysis

• Influencing Plan Selection• Special Considerations

 – Parameterized Queries

 – Temporary Tables and Table Variables

 – Parallel Queries

• References

Page 4: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 4/65

4

SEAS09SEAS09

| SQL Server Customer Advisory Team

Agenda

• SQLCAT Introduction• Query Optimizer Architecture

• Statistics Used by Optimizer

• Query Plan Analysis

• Influencing Plan Selection

• Parameterized Queries

• References

Page 5: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 5/65

5

SEAS09SEAS09

| SQL Server Customer Advisory Team

SQLCAT Introduction

• SQL Server Customer Advisory Team – Part of SQL Server development group

• SQLCAT Mission – Making customer successful

• Engage with the largest and most challenging SQLdeployments in the world, provide architecturalguidance and reviews, with direct access to productdevelopment engineers

 – Improve the product•  Through lessons learned, drive new features into the

product, share field experience with product team,

and participate in product planning – Improve the community

• Presentation at SQL PASS, TechEd, TechReady andmany internal and external events

• Whitepapers, Technical white papers and Blogs basedon lesson learned are published at SQLCAT website:www.sqlcat.com

• Lesson learned based training event:SEAS – SQL Server Enterprise Architect

 

Page 6: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 6/65

6

SEAS09SEAS09

| SQL Server Customer Advisory Team

SQLCAT Profile

• SQLCAT structure – Customer Advisory Team including

international and technologyexcellence team

 – ISV team – Best practice team

• Small team with worldwide coverage – Senior database architects with many

years of SQL Server and other RDBMSexperience – Extensive customer, project management

experience – Mixed consulting services, product

development background

CustomerTeamQL

IS V

SQLCAT

 QL BP

Page 7: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 7/65

7

SEAS09SEAS09

| SQL Server Customer Advisory Team

SQL Server Design-winProgram• Identify the most challenging, unique, interesting and

innovative applications running on the Microsoftdatabase platform

• Invest in Large Scale, referenceable SQL Serverprojects across the world

 – SQLCAT technical & project experience

 – Design & architecture review covering performance,operation, scalability, availability and many aspects

 – Priority use of customer lab environment

• Use customer experiences to make a better product• Share the knowledge learned from these challenging

implementations to help the world wide SQLcommunity.

• Reference the customer and make them a case story

Page 8: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 8/65

8

SEAS09SEAS09

| SQL Server Customer Advisory Team

Engaging SQLCAT

• Engagement through Design-win program• Coverage/Contacts in your region

 – US & worldwide:• Lubor Kollar – [email protected]

 – Latin America

• Stuart Ozer – [email protected] – Asia

• Howard Yin (GCR & Japan) – [email protected]

• Peter Carlin (Taiwan & Japan) – [email protected] 

• Prem Mehra (India & US) – [email protected] 

• Nicholas Dritsas (APAC) – [email protected]

 – Europe•  Thomas Kejser – [email protected]

• Alexei Khalyako – [email protected]

 – US• Michael Thomassy – [email protected] 

• Kevin Cox – [email protected] 

Page 9: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 9/65

9

SEAS09SEAS09

| SQL Server Customer Advisory Team

Agenda

• SQLCAT Introduction

• Query Optimizer Architecture

• Statistics Used by Optimizer• Query Plan Analysis

• Influencing Plan Selection

• Parameterized Queries• References

Page 10: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 10/65

Microsoft Confidential 10

SEAS09SEAS09

| SQL Server Customer Advisory Team

SQL Server Relational ServerHigh-Level Architecture

SQL Relat ional Server 

Storage Engine (Access Methods, Database PageCache, Locking, Transactions, …)

SQL-OS (Schedulers, Buffer Pool, MemoryManagement, Synchronization Primitives, …)

Query Optimization(Plan Generation, IndexSelection, Join Ordering,

Statistics, Costing) Query Execution(Query Operators,Memory Grants,Parallelism)

Language Processing (Parse/Bind/View Expansion)Utilities( D

B

CC,

Backup

/Restore

,BCP,…)

M

etadat a

,Types ys

tem,

E

xpressionServ

ice

s

Statement/BatchExecution

Plan Cache Management

Page 11: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 11/65

Microsoft Confidential 11

SEAS09SEAS09

| SQL Server Customer Advisory Team

SQL Server Relational ServerHigh-Level Architecture

SQL Relat ional Server 

Storage Engine (Access Methods, Database PageCache, Locking, Transactions, …)

SQL-OS (Schedulers, Buffer Pool, MemoryManagement, Synchronization Primitives, …)

Query Execution(Query Operators,Memory Grants,Parallelism)

Language Processing (Parse/Bind/View Expansion)Utilities( D

B

CC,

Backup

/Re

store

,BCP,…)

M

etadat a

,Types ys

tem,

E

xpressionServ

ice

s

Statement/BatchExecution

Plan Cache Management

Query Optimization(Plan Generation, IndexSelection, Join Ordering,Statistics, Costing)

Page 12: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 12/65

Microsoft Confidential 12

SEAS09SEAS09

| SQL Server Customer Advisory Team

Storage Engine (Access Methods, Database PageCache, Locking, Transactions, …)

SQL-OS (Schedulers, Buffer Pool, MemoryManagement, Synchronization Primitives, …)

Query Optimization(Plan Generation, IndexSelection, Join Ordering,

Statistics, Costing) Query Execution(Query Operators,Memory Grants,Parallelism)

Language Processing (Parse/Bind/View Expansion)Utilities( D

B

CC,

Backup

/Re

store

,BCP,…)

M

etadat a

,Types y

stem,

E

xpressionServ

ice

s

Statement/BatchExecution

Plan Cache Management

SQL Server Relational Server This Talk – Query Optimization

SQL Relat ional Server 

Query Opt im izat ion(Plan Generat ion,Index Select ion, JoinOrdering, St at ist ics,

Costing)

Page 13: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 13/65

SEAS09SEAS09

| SQL Server Customer Advisory Team

Query Optimization

• Query Optimization is cost based inSQL Server

• Input to Query Optimization is

Operator Tree produced by – Parsing SQL statement (Syntax)

 – Semantic analysis and checking (akaAlgebrizing – this includes type

derivation, type checking, bindingvariables, table and column names,etc.)

• Output is Query Plan

Page 14: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 14/65

| SQL Server Customer Advisory Team

SEAS09SEAS09

Query Optimization

parse/bindparse/bindQueryQuery

optimizer optimizer 

Select *Select *From …From …

JoinJoin

UnionUnion……

…… …… ……

MetadataMetadatalIndicesIndiceslStatisticsStatisticslConstraintsConstraintslUser User 

Hash joinHash join

ConcatConcat……

OutputOutput

…… …… ……

lGenerateGeneratealternativesalternatives

l

lEstimate costEstimate cost

InputInputPhysicalPhysical

PropertiesPropertieslMemoryMemorylNumber of Number of 

corescores

Page 15: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 15/65

SEAS09SEAS09

| SQL Server Customer Advisory Team

Relational Expressions

• SQL is the standard front-endlanguage

• Query processing is based on

relational operator trees»

• Query blockGroupBy T.c, sum(T.a)GroupBy T.c, sum(T.a)

Filter (T.b=R.b and R.c = 5)Filter (T.b=R.b and R.c = 5)

RRTT

Cross productCross product

SELECT SUM(T.a)SELECT SUM(T.a)

FROM T, RFROM T, R

WHERE T.b = R.bWHERE T.b = R.bAND R.c = 5AND R.c = 5

GROUP BY T.cGROUP BY T.c

GroupBy T.c, sum(T.a)GroupBy T.c, sum(T.a)

Filter (R.c = 5)Filter (R.c = 5)

RR

TT

Join (T.b=R.b)Join (T.b=R.b)

Page 16: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 16/65

| SQL Server Customer Advisory Team

SEAS09SEAS09

 Tree Transformations

Filter (A.x = 5)Filter (A.x = 5)

JoinJoin

BBAA

Filter (A.x = 5)Filter (A.x = 5)

JoinJoin

BB

AA

GroupBy A.x, sum(A.y)GroupBy A.x, sum(A.y)

JoinJoin

BBAA

JoinJoin

BB

AA

GroupBy A.x, sum(A.y)GroupBy A.x, sum(A.y)

JoinJoin

BBAA

Hash-JoinHash-Join

BBAA

SimplificationSimplification ImplementationImplementationExplorationExploration

Page 17: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 17/65

| SQL Server Customer Advisory Team

SEAS09SEAS09

Sets Of Transformations

•  Join reordering• Outerjoins

• Subqueries

• Aggregation• Union

• Index plans

• Large IN lists

• Empty tablesimplification

• Bitmap filters

• Star and Snowflakequeries

•  Join elimination

• Indexed views

• Update plans

• Constraint checking

• Parallel plans

• Remote queries• Partitioned tables

• …

Page 18: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 18/65

SEAS09SEAS09

| SQL Server Customer Advisory Team

Costing

• Based on – Estimated intermediate and final result

size

 – Detailed model of execution algorithm(I/O, number of CPUs, memory)

• Estimation of result size – Histograms on distribution of values

for base table columns (tablestatistics)

 – Derived on each operation, e.g. filter,

 join

Page 19: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 19/65

SEAS09SEAS09

| SQL Server Customer Advisory Team

Multi-Stage Optimization

• Stages – No-choice queries (trivial plan)

 – Transaction processing queries (Phase0)

 – Complex query I (Phase 1)

 – Complex query II (Phase 2)

• Parameters of a stage

 – Entry condition – Transformations allowed

 – Termination condition

Page 20: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 20/65

| SQL Server Customer Advisory Team

SEAS09SEAS09

SQL Server Cost-based QueryOptimization Process

T0T0simplifysimplify

T1T1 Pool of alternat ivesPool of alternat ives

search(0)search(0) search(1)search(1) search(2)search(2)

T2T2

(input)(input)

(output)(output)

Cost-based optimizationCost-based optimization

use simplificationuse simplification

transformationstransformations

use explorat ion anduse exp loration and

implementationimplementationt ransformations, costt ransformations, cost

alternativesalternatives

SEAS09

Page 21: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 21/65

SEAS09SEAS09

| SQL Server Customer Advisory Team

Optimization Phases

• Goal: don’t overspend on optimization – E.g. don’t spend additional 1 second of optimization

time to shave off 0.5 second of run time• Phase 1 is not entered if Phase 0 finds a plan with

cost<0.2

• Phase 2 is not entered if Phase 1 finds a plan with

cost<1.0• Each additional phase has richer transformations and

considers additional plan types – In Phase 0 only NL and Hash joins

 – In Phase 1 initially only non-parallel plans areconsidered; if the best plan so far found has cost>=5, Phase 1 is repeated to find a parallel plan andthe “cheaper” of the parallel and non-parallelversions will advance to Phase 2

 – Phase 2 is performing for example Outer Joinreordering, Index View matching and Star Queryoptimization

• Optimization may end up in “time-out” and the best sofar if an ! lan is sent for execution

SEAS09SEAS09

Page 22: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 22/65

| SQL Server Customer Advisory Team

SEAS09SEAS09

IBM DB2 On NT And UNIX

T0T0simplifysimplify

T1T1 Pool of alternat ivesPool of alternat ives

search(0)search(0) search(1)search(1) search(2)search(2)

T2T2

(input(input

))

(output)(output)

Cost-based optimizationCost-based optimizationQuery rewrit eQuery rewriteheuristically change blocksheuristically change blocks

bott om-up join enumerationbott om-up join enumeration

knobs to lim it searchknobs to lim it search

lNon-uniform query representationNon-uniform query representationlCost-based optimization focused on joinCost-based optimization focused on join

ØDoes not cross “ query block boundary”Does not cross “ query block boundary ”

SEAS09SEAS09

Page 23: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 23/65

SEAS09SEAS09

| SQL Server Customer Advisory Team

Oracle Architecture

• Old “rule-based optimizer” – Use heuristics, and follow SQL syntax

 – No cost-estimation considered

• New cost-based optimizer – Follow IBM’s (old, System-R kind of 

approach)

SEAS09SEAS09

Page 24: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 24/65

SEAS09SEAS09

| SQL Server Customer Advisory Team

SQL Server - OptimizerExtensibility

Goal: Support high paced innovation (typicalfor MS products)

• Easy way to add new operators – Logical operators, or execution algorithms

 – Cardinality and cost estimation functions

• Adding new tree transformations – Handle new operators

 – Add new reordering alternatives

• Apply cost-based optimization uniformly onall operations, beyond joins/SQL queryblock

SEAS09SEAS09

Page 25: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 25/65

25

SEAS09SEAS09

| SQL Server Customer Advisory Team

 Tools to monitor optimization

• SQL Server 2008 has tools to see “inside”the optimization

• Showplan contains for example(will be covered in depth next)

 – Operator cost and subtree cost – Cardinalities of all operators (estimated

number of rows returned by the operator) – Optimizer warnings (e.g. missing join

predicate) – Values of variables used for optimization

• SQL Trace• DMV sys.dm_exec_query_optimizer_info• Perfmon counters in SQL Server: SQL

Statistics

SEAS09SEAS09

Page 26: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 26/65

26

SEAS09SEAS09

| SQL Server Customer Advisory Team

Agenda

• SQLCAT Introduction• Query Optimizer Architecture

• Statistics Used by Optimizer

• Query Plan Analysis

• Influencing Plan Selection

• Parameterized Queries

• References

SEAS09SEAS09

Page 27: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 27/65

27

SEAS09SEAS09

| SQL Server Customer Advisory Team

What affects optimizerchoices?

• Optimizer Statistics

• Indexes and constraints

•Memory• Logical number of processors

SEAS09SEAS09

Page 28: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 28/65

Microsoft Confidential 28

SEAS09SEAS09

| SQL Server Customer Advisory Team

Optimizer Statistics

• Used for costing – Histogram• Captures distribution of values for a column• Max number of steps = 200• Used for estimating cardinality of search predicates• Single-column only

 – Density

• Number of occurrences of each unique value in acolumn• Used for estimating cardinality of search predicates,

 joins, agg – String index (statistics) – new in SQL Server 2005

• Viewing statistics – DBCC SHOW_STATISTICS

• CREATE / UPDATE STATISTICS – Can specify FULL SCAN or sample size or sample % – Default sample size increases logarithmically with the

number of rows – Sp_createstats, sp_updatestats

• By default SQL Server creates and maintains statistics

automatically – the Best Practice is to use thedefault

SEAS09SEAS09

Page 29: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 29/65

Microsoft Confidential 29

SEAS09SEAS09

| SQL Server Customer Advisory Team

Auto Statistics

• Database options – AUTO_CREATE_STATISTICS• Stats on columns created when a query needs

them – AUTO_UPDATE_STATISTICS – when?

• Permanent tables: 500; 500 + 20% * rows in tablewhen compiled

•  Temp tables: 6; 500; 500 + 20% * rows in tablewhen compiled• See

http://www.microsoft.com/technet/prodtechnol/sql/2for more details

 – Beware AUTO UPDATE uses default samplingeven if created as FULLSCAN => consequenceCREATE INDEX creates statistics with FULLSCANbut subsequent auto update for the same indexwill use sampling

 – No auto-create or auto-update stats for tablevariables (and you cannot create statisticseither)

• sp_autostats – Display, change auto-stats setting (ON/OFF)

 – Can disable auto-stats per index, table

SEAS09SEAS09

Page 30: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 30/65

Microsoft Confidential 30

SEAS09SEAS09

| SQL Server Customer Advisory Team

Statistics Issues 

… and Best Practices

• Statistics are used to derive Cardinality Estimate (CE) on eachquery operation

 – Filter, Join, GroupBy, … – Warning column in showplan if missing statistics

• Limitations – CE gets progressively less reliable across multiple joins and group-

by’s

 – Hidden correlations tough to handle• Data-level correlation: Two columns that are correlated• Predicate-level correlation: Multiple predicates that are not

independent – Data skew impacts estimates, especially with sampled statistics

• Hard to correctly guess density from sample

• Recompilation if number of changes to a column exceedsthresholds

• (Re)Compilations requiring a statistic stall if auto_update stats isexecuting – Consider AUTO_UPDATE_STATISTICS_ASYNC Database option

• Best practices – Reasons to turn off AUTO_UPDATE on a statistic

• If it is causing unnecessary recompilation - lots of updates butthey don’t change nature/distribution of data

• For increased predictability during work-day if can be scheduledduring off time

 – Sometimes, FULLSCAN stats necessary – schedule periodic updatestats

SEAS09SEAS09

Page 31: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 31/65

Microsoft Confidential 31

SEAS09SEAS09

| SQL Server Customer Advisory Team

Upgrade Considerations

• Upgrade from SS 2000 – All SS 2000 stats are considered

automatically “stale” => autostats willtake care of the “upgrade”

 – Best Practice is to update statistics

manually as a part of the upgrade step – If stats are maintained manually they allshould be manually updated duringupgrade

• sp_updatestats @resample ='resample‘

 – Details of changes are described inhttp://www.microsoft.com/technet/prodtechnol• Upgrade from SS 2005

 – No need to update statistics

SEAS09SEAS09

Page 32: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 32/65

32

SEAS09SEAS09

| SQL Server Customer Advisory Team

Agenda

• SQLCAT Introduction• Query Optimizer Architecture

• Statistics Used by Optimizer

• Query Plan Analysis

• Influencing Plan Selection

• Parameterized Queries

• References

SEAS09SEAS09

Page 33: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 33/65

33

SEAS09SEAS09

| SQL Server Customer Advisory Team

Query Plan aka Showplan

• 3 forms: textual, XML , graphical• By contents: operators only, with cost and

cardinality estimates and with runtimeinformation

••

• SQL Trace has 8 different Showplan events in

Performance event category

Text XML Graphical

Operators SET SHOWPLAN_TEXTON

 N/A N/A

Operators andestimated costsand card.

SET SHOWPLAN_ALLON

SETSHOWPLAN_XMLON

Display EstimatedExecution Plan inManagement Studio

Run time info SET STATISTICSPROFILE ON

SET STATISTICSXML ON

Include ActualExecution Plan in

Management Studio

SEAS09SEAS09

Page 34: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 34/65

34

SEAS09SEAS09

| SQL Server Customer Advisory Team

Demo – Showplan in SSMS

• Estimated Query Plan – Properties of the Index Seek

 – Estimated # of rows, # of executions

• Actual Query Plan – Watch for “Actual” number of rows,

estimated number of rows, numberof executions

• SHOWPLAN_TEXT, ALL, XML

• Saving and sharing the query plans

 –

SEAS09SEAS09

Page 35: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 35/65

35

SEAS09SEAS09

| SQL Server Customer Advisory Team

Actual Query Plan

SEAS09SEAS09

Page 36: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 36/65

36

SEAS09S S09

| SQL Server Customer Advisory Team

Demo – Trivial Plan in SSMS

•  Trivial plan – By investigating the syntax and

metadata optimizer knows “there areno choices”

 – Optimization Level = Trivial

• Non-trivial plan – Optimization Level = FULL

 – if early termination happens• Reason For Early Termination has a value

 – Good Enough Plan Found OR

 –  Timeout OR

 – Memory Limit Exceeded

 –

SEAS09SEAS09

Page 37: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 37/65

37

S S09

| SQL Server Customer Advisory Team

Using DMV

• sys.dm_exec_query_optimizer_infocaptures aggregate counters of various compile time events, for

example – How many compilations happened so

far?

 – How many finished in which phase(0,1,2,3) of optimization?

 – How many Inserts, Deletes, Updates?

 – Etc.

SEAS09SEAS09

Page 38: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 38/65

38| SQL Server Customer Advisory Team

Demo - Optimizer Warnings

• SHOWPLAN shows warnings• Missing Column Statistics

• Missing Join Predicate

• SQL Trace events – Errors and Warnings Event Category

• Missing Column Statistics

• Missing Join Predicate

SEAS09SEAS09

Page 39: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 39/65

39| SQL Server Customer Advisory Team

Demo – Database TuningAdvisor (DTA)

• Input batch file or SQL trace file or table

• GUI interface surfaces only a subset of capabilities of the DTA engine

 – Command line version is using XMLdocument conforming toDTASchema.xsd

 – SQL Server schemas are published at

http://schemas.microsoft.com/sqlserver/

• Recommended use for smaller sets of 

queries (as was the demo)

SEAS09SEAS09

Page 40: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 40/65

40| SQL Server Customer Advisory Team

Missing Indexes

• In some cases the Optimizer will recordinformation about “missing index” inShowplan and in several DMVs

 – sys.dm_db_missing_index_groups – sys.dm_db_missing_index_group_stats

 – sys.dm_db_missing_index_details

 – sys.dm_db_missing_index_columns

•  The information is opportunistic and notexhaustive

SEAS09SEAS09

Page 41: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 41/65

41| SQL Server Customer Advisory Team

Query Plan Debugging – bestpractices• Collect “Actual Execution Plan”• Compare “Estimated” and “Actual” row counts

 – Large discrepancies indicate potential problemspots

• Watch for Execution Warnings – BOL describesremedies for each case – Sort Warning

• sort operations do not fit into memory

 – Exchange Spill

• buffers in a parallel query plan have beentemporarily written to the tempdb database

 – Hash Warning• hash recursion or cessation of hashing (hash

bailout) has occurred during a hashingoperation

 – Bitmap Warning• -

SEAS09SEAS09

Page 42: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 42/65

42| SQL Server Customer Advisory Team

Agenda

• SQLCAT Introduction• Query Optimizer Architecture

• Statistics Used by Optimizer

• Query Plan Analysis

• Influencing Plan Selection

• Parameterized Queries

• References

SEAS09SEAS09

Page 43: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 43/65

Microsoft Confidential 43| SQL Server Customer Advisory Team

Hints

•  Types of Query hints – Locking (PAGLOCK, ROWLOCK, NOLOCK,

HOLDLOCK…) – Isolation Level (REPEATABLEREAD,

SERIALIZABLE ,…) – Other (KEEPIDENTITY, KEEPDEFAULTS,

NOWAIT, NOEXPAND,…)

 –Optimizer

• Table access• Join• Query

SEAS09SEAS09

Page 44: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 44/65

44| SQL Server Customer Advisory Team

Optimizer Hint Classification

SELECT ProductName, Products.ProductID

FROMdbo.[Order Details] WITH (INDEX(0))

INNER LOOP JOINdbo.Products

ON [Order Details].ProductID = Products.ProductID

WHERE Products.UnitPrice > 100

OPTION (RECOMPILE, LOOP JOIN)

TABLEaccessHINT

JOIN HINT

QUERYHINT

SEAS09SEAS09

Page 45: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 45/65

Microsoft Confidential 45| SQL Server Customer Advisory Team

Optimizer Hint Examples

•  Table access hint: INDEX (n) | (name) – Cannot force seek versus scan until 2008

•  Join order, method – FORCE ORDER query hint

• But cannot specify order across sub-queries, orpush aggregations below joins – always QUERYlevel

 – LOOP | MERGE | HASH JOIN – both at the querylevel and individual join

• Aggregation – HASH GROUP => Hash Aggregation – ORDER GROUP => Stream Aggregation

• For more details see BOL –  TABLE HINT section for Table access hints –  Join Hints section for Join hints – OPTION clause section for Query hints

SEAS09SEAS09

Page 46: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 46/65

Microsoft Confidential 46| SQL Server Customer Advisory Team

Hints - New in 2005 and 2008

• RECOMPILE hint for individual queries•  The following hints provide more control to

“fix” the plan – OPTIMIZE FOR hint to force particular values

of parameters to be used by the optimizer – USE PLAN to enforce particular query plan – PARAMETERIZATION hint

• Database level• Inside a Plan Guide

 – Plan Guides• FORCESEEK – new table access hint in SQL

Server 2008

SEAS09SEAS09

Page 47: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 47/65

Microsoft Confidential 47| SQL Server Customer Advisory Team

Recompile Hint

• In SS 2000: CREATE PROCEDURE WITHRECOMPILE

• In SS 2005 and 2008, an individual

statement may have OPTION(RECOMPILE) – Better alternative to dynamic SQL when

forcing recompilation for each new set

of variables was suggested in Shiloh – Easier syntax

 – No need to grant access to allreferenced objects; execute on the spsuffices

SEAS09SEAS09

Page 48: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 48/65

Microsoft Confidential 48| SQL Server Customer Advisory Team

Optimize For Hint

OPTION ( OPTIMIZE FOR( @variable_name = literal_constant  [ ,…n ] ) )

• Values are used only for theoptimization, not for execution

• Counteracts parameter detection – mayachieve more predictable behavior

• Cannot be used without dataknowledge

SEAS09SEAS09

Page 49: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 49/65

Microsoft Confidential 49| SQL Server Customer Advisory Team

Use Plan Hint

• Plan Stability Feature

• Idea: Use a captured XML Showplanin a USE PLAN query hint to guide

the optimizer to choose the sameplan

SEAS09SEAS09

Page 50: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 50/65

Microsoft Confidential 50| SQL Server Customer Advisory Team

Plan Forcing

Usage:

SELECT * FROM t OPTION ( USE PLAN N‘<ShowPlanXML> …

</ShowPlanXML>’ )

Query

Query Hint

String literal

of XML Showplan

SEAS0SEAS0

99

Page 51: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 51/65

Microsoft Confidential 51

99

| SQL Server Customer Advisory

Team

What’s Forced by Plan Hint

• Plan topology and order of evaluation

• Execution algorithms

• Index solutions

• Objects referenced in the query

Plan Forcing is used to force costbased decisions.

SEAS0SEAS0

99

Page 52: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 52/65

Microsoft Confidential 52

99

| SQL Server Customer Advisory

Team

Plan Hint Limitations

• No INSERT/UPDATE/DELETE in 2005, YES in 2008

• No Distributed Query

• No Full-text queries

• Static, Fast_Forward cursors only

SEAS0SEAS0

99

Page 53: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 53/65

Microsoft Confidential 53

99

| SQL Server Customer Advisory

Team

Plan Guides - Idea

Original Query Hinted Version

SELECT … SELECT … OPTION (…)

UPDATE … UPDATE … OPTION (…)

Out-of-band query hinting

Scenario: query is known but cannot bemodified directly in the source code yetsome hinting is desirable

Replace the Original Query by the hintedversion using “Plan Guide” system table

SEAS0SEAS0

99Plan Guides Feature

Page 54: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 54/65

Microsoft Confidential 54

99

| SQL Server Customer Advisory

Team

Plan Guides FeatureOverview• Create: sp_create_plan_guide

• Drop/Enable/Disable:sp_control_plan_guide

• Matching: – During module (SP, DML trigger, multi-

statement TVF, scalar UDF)compilation

 – During batch compilation – Targets

• original user query

• Simple- or Force-parameterized query

SEAS0SEAS0

99

S i ( ll t i t

Page 55: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 55/65

Microsoft Confidential 55

99

| SQL Server Customer Advisory

Team

Scenarios (all pertain to an appyou can’t change)

• Force use of a MERGE JOIN operation for aspecific query in a user-submitted batch

• Force optimizer to always use “worst case”parameter value

• Force compilation sharing for queries with aformat that is the same except constantliterals are different – Specify FORCED PARAMETERIZATION in plan

guide• For use of a specific plan for a manually

parameterized query

SEAS09SEAS09

New around plan guides in

Page 56: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 56/65

Microsoft Confidential 56| SQL Server Customer Advisory Team

New around plan guides in2008• Plan Freezing - ability to create a plan from the

plan cache directly –  The plan guide is automatically created by

invoking sp_create_plan_guide_from_cache• See documentation for the sp for boilerplate

code• sys.fn_validate_plan_guide function validates if 

the plan guide is still valid (after e.g. somemetadata changes)

• SQL Profiler trace events

"Plan_guide_successful" and"Plan_guide_unsuccessful" are located underthe "Performance" category

• Perfmon counters "Guided Plan Executions/sec"and "Misguided Plan Executions/sec“ provide

similar function as the trace events above

SEAS09SEAS09

Page 57: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 57/65

57| SQL Server Customer Advisory Team

Agenda

• SQLCAT Introduction• Query Optimizer Architecture

• Query Plan Analysis

• Influencing Plan Selection• Parameterized Queries

• References

SEAS09SEAS09

Page 58: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 58/65

Microsoft Confidential 58| SQL Server Customer Advisory Team

Parameterization

• In SQL Server 2000 “auto parameterization” isused only for very simple queries – Referred to as SIMPLE parameterization in SS

2005

•Discoverable by investigating the plan;“parameter” is suddenly there. For example

Query: * =select from T where c 5Plan: -- ( :([ ].[ ].[ ]), :| Table Scan OBJECT tst1 dbo T WHERE

([ ].[ ]=T C Convert([ ])@1 ))

• Advantage: no need to recompile for thesame query with different value instead of “5”

SEAS09SEAS09

Page 59: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 59/65

Microsoft Confidential 59| SQL Server Customer Advisory Team

Parameterization (contd.)

• In SQL Server 2005 “parameterization”may be forced on a database level forALL queries – Set PARAMETERIZATION option to

FORCED in ALTER DATABASE stmt• Use only when you are sure compilation

is the bottleneck• Some queries may get worse plans if 

forcefully parameterized• Works only against databases in 90 and

higher compatibility level

SEAS09SEAS09

Page 60: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 60/65

Microsoft Confidential 60| SQL Server Customer Advisory Team

Parameterization (Example)

Query: * = =select from T where c 1 or c 2Plan: -- ( :([ ].[ ].[ ]), :([ ].[ ].[ ].| Table Scan OBJECT tt dbo T WHERE tt dbo T

[ ]=c ( )1 [ ].[ ].[ ].[ ]=OR tt dbo T c ( )2 ))

ALTER DATABASE <dbname> SETPARAMETERIZATION FORCED

Plan: -- ( :([ ].[ ].[ ]), :([ ].[ ].| Table Scan OBJECT tt dbo T WHERE tt dbo[ ].[ ]=T c [ ]@0 [ ].[ ].[ ].[ ]=OR tt dbo T c [ ]@1 ))

ALTER DATABASE <dbname> SETPARAMETERIZATION SIMPLE

SEAS09SEAS09

Page 61: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 61/65

61| SQL Server Customer Advisory Team

Agenda

• SQLCAT Introduction• Query Optimizer Architecture

• Query Plan Analysis

• Influencing Plan Selection• Parameterized Queries

• References

SEAS09SEAS09

Page 62: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 62/65

62| SQL Server Customer Advisory Team

Optimizer References

•  Tips, Tricks, and Advice from the SQLServer Query Processing Teamhttp://blogs.msdn.com/sqlqueryprocessin 

• Query Tuning Recommendationshttp://technet.microsoft.com/en-us/librar 

• Data Warehouse Query Performancehttp://technet.microsoft.com/en-us/maga 

Page 63: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 63/65

Microsoft Confidential 63

Some showplan references

• Find Top N costly query plans in adhocbatches ormodules...http://blogs.msdn.com/sqltips/archive/2005/10/05/T 

•  Troubleshooting Performance Problems in SQLServer 2005http://www.microsoft.com/technet/prodtechnol/sql/2

• Processing XML Showplans Using SQLCLR in SQL

Server 2005http://msdn.microsoft.com/library/default.asp?url=/l 

SEAS09SEAS09

Page 64: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 64/65

64| SQL Server Customer Advisory Team

QQ AA&&

Page 65: SEAS2009 Query Optimization Final

8/8/2019 SEAS2009 Query Optimization Final

http://slidepdf.com/reader/full/seas2009-query-optimization-final 65/65

© 2008 Microsoft Corporation All rights reserved Microsoft Windows Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U S and/or other countries The