Microsoft SQL ServerMicrosoft SQL ServerQuery TuningQuery TuningQuery Tuning Query Tuning 

Speaker: Dean RichardsSenior DBA, Confio Software

Silicon Valley SQL Server User GroupNovember 2010

Mark Ginnebaugh, User Group Leader, mark@designmind.com

Query TuningGet it Right the First Time

Dean Richards Senior DBA, Confio Software

2

Who Am I?

Dean Richards, of course!, 20+ Years in SQL Server & Oracle

• DBA and Developerp

Senior DBA for Confio Software• DeanRichards@confio.com• Makers of Ignite8 Response Time Analysis Tools• http://www.ignitefree.com – only free RTA Tool

Specialize in Performance Tuning Presented at 24 Hours of PASS

3

13 SQL Saturdays and counting…

Agenda

Introduction Which Query Should I Tune Query PlansQuery Plans SQL Diagramming

• Who registered yesterday for Tuning Classg y y g• Check order status

4

Why Focus on Queries

Most Applications• Read and Write data to/from database• Read and Write data to/from database• Simple manipulation and smaller amounts of data• Inefficiencies would not be noticed• Inefficiencies would not be noticed

Most Queries• Examine larger amounts of data, return a littleExamine larger amounts of data, return a little• Inefficiencies quickly become bottleneck

Why Tune Queries?y Q• “Gives the most bang for your buck”• Changes to SQL are usually safer

5

• ~85% of performance issues are SQL related

Who Should Tune

Developers?p• Developing applications is very difficult• Typically focused on functionality• Not much time left to tune SQL• Do not get enough practice

SQL diff tl i P d ti th D /T t• SQL runs differently in Production than Dev/Test

DBA?D t k th d lik d l d• Do not know the code like developers do

• Focus on “Keep the Lights On”• Very complex environment

6

• Very complex environment

Need a team approach

Which SQL

User / Batch Job Complaints Queries Performing Most I/O (LIO PIO) Queries Performing Most I/O (LIO, PIO) Queries Consuming CPU Queries Doing Table or Index ScansQueries Doing Table or Index Scans Known Poorly Performing SQL Server Side Tracing Highest Response Times (Ignite8)SELECT sql_handle, statement_start_offset,

statement end offset, plan handle, execution count,statement_end_offset, plan_handle, execution_count, total_logical_reads, total_physical_reads, total_elapsed_time, st.text

FROM sys.dm_exec_query_stats AS qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) AS st

7

ORDER BY total_elapsed_time DESC

Why is SQL Slow – Wait States

Focus on Response Time

Understand the total time a Query spends in Database Measure time while Query executes

8

SQL Server helps by providing Wait Types

Wait Time Tables (SQL 2005/8)

http://msdn.microsoft.com/en-us/library/ms188754.aspx

dm_exec_query_statsexecution_counttotal_logical_writestotal_physical_readstotal logical reads

dm_exec_requestsstart_timestatussql_handleplan handle total_logical_reads

total_elapsed_timeplan_handlestart/stop offsetdatabase_iduser_idblocking_sessionwait type

dm_exec_query_planquery plan_ yp

wait_time

dm_exec_text_query_planquery plan

dm_exec_sql_textt t

dm_exec_sessionslogin_timelogin_namehost_nameprogram name

9

textprogram_namesession_id

Base Monitoring Query

INSERT INTO SessionWaitInfoSELECT i id l h dl t t t t t ff tSELECT r.session_id, r.sql_handle, r.statement_start_offset,

r.statement_end_offset, r.plan_handle, r.database_id,r.blocking_session_id, r.wait_type, r.query_hash,s.host_name, s.program_name, s.host_process_id,s.login_name, CURRENT_TIMESTAMP cdt

FROM sys dm exec requests rFROM sys.dm_exec_requests rINNER JOIN sys.dm_exec_sessions s ON s.session_id = r.session_idWHERE r.status <> 'background'AND d 'AWAITING COMMAND'AND r.command <> 'AWAITING COMMAND'AND s.session_id > 50AND s.session_id <> @@SPID

10

RTA - Proactive

11

RTA - Firefighting

12

RTA - Correlation

13

Sample Wait Types

WRITELOG• Waiting for a log flush to complete

LCK_M_S, LCK_M_U, LCK_M_X…• Waiting to acquire locks

NETWORKIO, ASYNC_NETWORK_IO• Waiting on the network

PAGEIOLATCH_SH, PAGEIOLATCH_EX…• Physical disk reads

WAITFOR (idle event)W i i d i WAITFOR d

14

• Waiting during a WAITFOR command

Tracing

Tracing with waits gathers very good dataC b Hi h O h d i P fil Can be High Overhead via Profiler

Use Server-Side Tracing• sp_trace_create – create the trace definitionp_ _• sp_trace_setevent – add events to trace• sp_trace_setfilter – apply filters to trace• sp trace setstatus – start/stop the tracesp_t ace_setstatus sta t/stop t e t ace

Use Profiler to Create Initial Trace• Use File > Script Trace to Get Script

Cumbersome to review data Cumbersome to review data Set trace file sizes appropriately

15

Summary of Response Time

Using Response Time Analysis (RTA) Ensures you Work on the Correct Problem Shows Exactly Why Performance is Suffering

l bl Helps Prioritize Problems Do Not Rely Exclusively on Health Stats (CPU

Utilization Disk IO Cache Hit Ratio)Utilization, Disk IO, Cache Hit Ratio) Data Collection

• DMVs – build it yourselfDMVs build it yourself• Tracing – know how to process trace data• Tools – Ensure they use Wait Time and Health

16

Why is SQL Slow - Plans

SQL Server Management Studio• Estimated Execution Plan - can be wrong• Actual Execution Plan – must execute query, can be

dangerous in production and also wrong in testdangerous in production and also wrong in test

SQL Server Profiler Tracing• Event to collect: Performance : Showplan All• Event to collect: Performance : Showplan All• Works when you know a problem will occur

DM EXEC QUERY PLANDM_EXEC_QUERY_PLAN, DM_EXEC_TEXT_QUERY_PLAN(@handle,@s,@e)• Real execution plan of executed query

17

p q y

DM_EXEC_QUERY_PLAN

18

Case Studies

SQL DiagrammingQ g g• Who registered yesterday for Tuning Class• Check order status

19

SQL Statement 1

Who registered yesterday for SQL Tuning

SELECT s.fname, s.lname, r.signup_dateFROM student s INNER JOIN i t ti ON t d t id t d t idINNER JOIN registration r ON s.student_id = r.student_idINNER JOIN class c ON r.class_id = c.class_idWHERE c.name = 'SQL TUNING'AND r.signup_date BETWEEN @BeginDate AND @EndDateAND r.cancelled = 'N'

Execution Stats – 9,634 Logical Reads

20

Database Diagram

21

Execution Plan

R d ti f SSMSRecommendation from SSMSCREATE NONCLUSTERED INDEX [<Name of Missing Index>]ON [dbo].[registration] ([cancelled],[signup_date])INCLUDE ([student id] [class id])

22

INCLUDE ([student_id],[class_id])

SQL Diagramming

Great Book “SQL Tuning” by Dan Tow• Great book that teaches SQL Diagramming

registration .03

• http://www.singingsql.com

g37

1

1293

1

student class .001

select count(1) from registration where cancelled = 'N'and signup_date between '2010-04-23 00:00' and '2010-04-24 00:00'

54,554 / 1,639,186 = 0.03

select count(1) from class where name = 'SQL TUNING'

23

select count(1) from class where name = SQL TUNING

2 / 1,267 = .001

New Execution Plan

CREATE INDEX cl_name ON class(name)

Execution Stats – 9,139 Logical Reads Why would an Index Scan still occur on REGISTRATION?

24

Why would an Index Scan still occur on REGISTRATION?

Database Diagram

25

New Execution Plan

CREATE INDEX reg_alt ON registration(class_id)

Execution Stats – 621 Logical Reads

26

Execution Stats 621 Logical Reads

Better Execution Plan

CREATE INDEX reg_alt ON registration(class_id) INCLUDE (signup date cancelled)INCLUDE (signup_date, cancelled)

l d

27

Execution Stats – 20 Logical Reads

Alternative from SSMS

CREATE INDEX reg_can ON registration(cancelled, signup_date)INCLUDE (class_id, student_id)

Execution Stats – 595 Logical Reads

CREATE NONCLUSTERED INDEX [<Name of Missing Index>]

28

CREATE NONCLUSTERED INDEX [<Name of Missing Index>]ON [dbo].[registration] ([class_id],[cancelled],[signup_date])INCLUDE ([student_id])

SQL Statement 2

Paychecks for specific employees

SELECT e.first_name, e.last_name, l.descriptionFROM emp eINNER JOIN loc l ON e.loc id = l.loc id_ _WHERE (e.first_name = @fname OR e.last_name = @lname)AND EXISTS (

SELECT 1FROM tFROM wage_pmt wWHERE w.emp_id = e.emp_idAND w.pay_date>= DATEADD(day,-31,CURRENT_TIMESTAMP)))

Execution Stats – 64,206 Logical Reads

29

Database Diagram

30

SQL Diagramming

wage_pmt .02

emp

90

1

.0005 .0009

loc

1000

1

loc

select count(1) from wage_pmtwhere pay_date >= DATEADD(day,-31,CURRENT_TIMESTAMP)

40,760 / 1,915,088 = .02

select top 5 first_name, count(1) from emp group by first name order by 2 desc

31

_

12 / 23,798 = .0005 – first_name22 / 23,789 = .0009 – last_name

Execution Plan

32

New Execution Plan

CREATE INDEX ix2_fname ON emp(first_name)

33

Which Index?

SSMS RecommendationCREATE INDEX wp pay date ON wage pmt(pay date)CREATE INDEX wp_pay_date ON wage_pmt(pay_date)INCLUDE (emp_id)

50 000 L i l R d50,000 Logical Reads

or…

Better OptionCREATE INDEX wp emp pd ON wage pmt(emp id, pay date)CREATE INDEX wp_emp_pd ON wage_pmt(emp_id, pay_date)

46 Logical Reads

34

New Execution Plan

CREATE INDEX wp_emp_pd ON wage_pmt(emp_id, pay_date)

35

SQL Statement 2

Lookup order status for caller

SELECT o.OrderID, c.LastName, p.ProductID, p.Description, sd.ActualShipDate, sd.ShipStatus, sd.ExpectedShipDate

FROM [Order] oINNER JOIN It i ON i O d ID O d IDINNER JOIN Item i ON i.OrderID = o.OrderIDINNER JOIN Customer c ON c.CustomerID = o.CustomerIDINNER JOIN ShipmentDetails sd ON sd.ShipmentID = i.ShipmentIDLEFT OUTER JOIN Product p ON p.ProductID = i.ProductIDLEFT OUTER JOIN Product p ON p.ProductID i.ProductIDLEFT OUTER JOIN Address a ON a.AddressID = sd.AddressIDWHERE c.LastName LIKE ISNULL(@LastName,'') + '%'--AND c.FirstName LIKE ISNULL(@FirstName,'') + '%' AND o.OrderDate >= DATEADD(day, -30, CURRENT_TIMESTAMP)AND sd.ShipStatus <> 'C'

Execution Stats 10 159 Logical Reads

36

Execution Stats – 10,159 Logical Reads

Database Diagram

37

Execution Plan

38

SQL Diagramming

o .08 .005

.03i c

psd psd

aSELECT COUNT(1)*1.0/(SELECT COUNT(1) FROM Customer) FROM Customer WHERE LastName LIKE 'SMI%'.03SELECT COUNT(1)*1.0/(SELECT COUNT(1) FROM [Order]) FROM [Order] WHERE OrderDate >= DATEADD(day, -30, CURRENT_TIMESTAMP).08SELECT COUNT(1)*1.0/(SELECT COUNT(1) FROM [Order]) FROM [Order] WHERE OrderStatus <> 'C'.005

39

.005-- Combined.005

Data Skew Problems

SELECT OrderStatus, COUNT(1)FROM [Order]FROM [Order]GROUP BY OrderStatus

Only 0.5% of rows are <> ‘C’ How about changing the query?g g q y

• AND o.OrderStatus = 'I'

Add an Index on ShipStatus

40

New Execution Plan

CREATE INDEX IX2_OrderStatus ON [Order] (OrderStatus)INCLUDE (OrderID,CustomerID) ( , )

41Execution Stats – 3,052 Logical Reads

Takeaway Points

Tuning Queries gives more “bang for the buck”M k t i th t Make sure you are tuning the correct query Use Wait Types and Response Time Analysis

L ki bl b Q T i i• Locking problems may not be a Query Tuning issue• Wait types tell you where to start

Use “Real Execution Plans” Use “Real Execution Plans”• Get plan_handle from DM_EXEC_REQUESTS• Pass plan handle to DM EXEC QUERY PLAN()• Pass plan_handle to DM_EXEC_QUERY_PLAN()

SQL Diagramming - “Get it right the First Time”• Query Tuner Tools can mislead you• Query Tuner Tools can mislead you

Confio Software

Wait-Based Performance Tools Igniter Suite

• Ignite for SQL Server, Oracle, DB2, Sybaseg Q , , , y• Ignite for Databases on VMWare (Beta)

Helps show which SQL to tune Provides visibility into entire stack Based in Colorado, worldwide customers, Free trial at www.confio.com

43

To learn more or inquire about speaking opportunities, please contact:

Mark Ginnebaugh, User Group Leadermark@designmind.com

http://www.meetup.com/The‐SiliconValley‐SQL‐Server‐User‐Group/