Monitoring and Optimizing SQL Server 2005 Performance

Anil Desai

Speaker Information

• Anil Desai– Independent consultant (Austin, TX)– Author of several SQL Server books– Instructor, “Implementing and Managing SQL

Server 2005” (Keystone Learning)– Info: http://AnilDesai.net or Anil@AnilDesai.net

Agenda and Overview

• Performance Monitoring Overview• Using SQL Profiler

– Integrating with Windows System Monitor

• Getting Performance Information– Dynamic Management Views (DMVs)– SQL Server Management Studio Reports

• Using the Database Engine Tuning Advisor• Managing Processes, Locking, and Deadlocks• Partitioning and Physical Storage Design

Performance Monitoring Approaches

• Performance optimization process:1. Establish a baseline2. Identify bottlenecks3. Make one change at a time4. Measure performance5. Repeat (if desired)

• Recommendations– Optimize for real-world workloads – Monitor/review performance regularly – Focus on specific issues

Overview of Performance Monitoring

• System– Windows System Monitor– Alerts (Performance-Based)

• SQL Server– SQL Profiler / SQL Trace– Activity Monitor– Dynamic Management Views (DMVs)– SQL Server Agent Alerts

• Query-Level– Database Engine Tuning Advisor– Query Execution Plans

Understanding SQL Profiler

• Purpose / Features: – GUI for managing SQL Trace– Monitor important events– Capture performance data / resource usage– Replaying of workloads / transactions– Identifying performance bottlenecks– Correlation of data with System Monitor– Workloads for Database Tuning Advisor

• Examples:– Generate a list of the 100 slowest queries– Monitor all failed logins (Security)

SQL Server Profiler Architecture

• SQL Profiler Terminology– Trace Definitions– Events– Columns– Filters

• Creating and Managing SQL Traces– SQL Profiler (GUI)– System Stored Procedures (Transact-SQL)

• Trace Templates (Built-In)– Standard (Default), SP_Counts– TSQL, TSQL_Duration,

TSQL_Grouped,TSQL_Replay, TSQL_SPs– Tuning

Configuring Trace Events

• Groupings:– Event Categories – Event Classes– Events

• Examples:– TSQL– Stored Procedures– Performance– Errors and Warnings– Security auditing

Configuring Trace Columns

• Specify the details to be recorded• Columns can be ordered and grouped• Values can be filtered• Examples of Columns:

– StartTime / EndTime– TextData– Duration– Resource Usage (CPU, Reads, Writes)– Information: User, Database, App. Names

Trace Output Options

• Interactive– Good for “live” monitoring of small sets of data

• Trace Files (*.trc)– Can enable file rollover based on size– “Server processes trace data” option

• Trace table– Will automatically create the table– Can set maximum number of rows

• Scheduling of traces (stop time)

Creating Profiler Traces

• Launching SQL Profiler

• Connecting to a database instance

• Configuring output options

• Create a trace definition

• Specifying events, columns, and filters

Other SQL Profiler Options

• Creating new templates

• Scripting trace definitions

• Extracting SQL Server Events– Transact-SQL Events– ShowPlan Events– Deadlock Events

Using System Monitor with SQL Profiler

• Purpose / Goal:– Correlate server performance with database

performance

• Process:– Define and start a counter log– Define and start a SQL Profiler trace– Import Performance Data in SQL Profiler

• Required Trace properties– StartTime– EndTime

Windows Performance Monitor

• Can monitor local or remote computers• Performance Statistics:

– Objects– Counters– Instances

• Modes:– System Monitor– Performance Logs and Alerts

• Counter Logs• Trace Logs• Alerts

Using Dynamic Management Views (DMVs)

• Purpose:– Monitoring and troubleshooting– View server state and performance details– Returns relational result sets

• Scopes:– Server level– Database level

DMV Examples

• Examples:– Database Engine

• Sys.DM_DB_File_Space_Usage

– Indexes• Sys.DM_DB_Index_Operational_Stats• Sys.DM_DB_Index_Physical_Stats

– I/O Related• Sys.DM_IO_Pending_IO_Requests• Sys.DM_IO_Virtual_File_Stats

– Common Language Runtime– Database Mirroring– Transactions

SQL Server Management Studio Reports

• Overviews of SQL Server usage– Can export to Excel or PDF

• Server-Level Report Examples:– Server Dashboard– Memory Consumption– Activity – All Block Transactions– Activity – Top Sessions– Performance – Batch Execution Statistics– Performance – Top Queries by Average CPU– Object Execution Statistics

Database-Level Reports

• Examples:– Disk Usage– All Transactions– All Blocking Transactions– Index Usage Statistics– Top Transactions by Age– Schema Changes History

Monitoring SQL Server Logs

• Windows Event Logs / Event Viewer – Application and System Event Logs

• SQL Server Management Studio– SQL Server Logs

• Can configure max. # of log files

– SQL Server Agent Error logs• Can configure logging levels (Errors, Warnings, Information)

• Using the Log File Viewer– Can Export / Load log information– Can search for specific errors/messages

Using SQLDiag

• Data Collected:– System Information (MSINFO)– Windows Event Logs– SQL Server configuration

• Command-Line Utility (SQLDiag.exe)– Stores output to files– Configuration file: SQLDiag.xml– Can run as a service (/R)– Can run in continuous mode

Database Engine Tuning Advisor

• Can make performance-related recommendations

• Replaces the “Index Tuning Wizard”• Evaluates Physical Design Structures (PDS)

– Indexes (clustered, non-clustered)– Indexed Views– Partitions

• Numerous analysis options• Output

– Generates modification scripts– Generates Reports for later analysis

Workloads

• Files– Transact-SQL Files– XML Files– Should represent commonly-used queries

• SQL Profiler Trace Files / Tables– Use Tuning built-in trace template– Events:

• Transact-SQL Batch• Remote Procedure Call (RPC)

– Columns: Event Class and Text Data

DTA Options

• Limit tuning time

• Tuning Options– Allowed Physical Design Structures (PDS)– Keep all/specific existing objects– Maximum storage space– Online or offline recommendations– Partitioning

DTA Reports

• Can export to XML files• Examples:

– Workload analysis– Column access– Table access– View-Table Relations– Statement cost– Event frequency– Index Usage (current / recommended)

Using the Database Engine Tuning Advisor

• Process:– Generate a workload (file or table)– Select tuning options– Run the analysis– View reports– Save and/or apply recommendations

• Running the DTA:– Database Engine Tuning Advisor Application (GUI)– Dta.exe command-line utility

Understanding Processes

• Processes– Interactive users

• SQL Server Management Studio

– Applications (Connection Pooling)• SQL Profiler• Database Engine Tuning Advisor• Replication• Service Broker

• Process IDs < 50 are system-related

Monitoring Processes

• SQL Server Activity Monitor– Processes (connected users)– Locks (by Process / by Object)– Filtering options– Auto-refresh option

• System Stored Procedures / Views– Sys.DM_Exec_Sessions – Sys.DM_Exec_Requests– Sys.SysProcesses– sp_who / sp_who2

Managing Processes

• Process Information– Current Process ID: @@SPID– Session Options: DBCC USEROPTIONS

• Killing Processes– KILL ProcessID [WITH STATUSONLY]

• Viewing Last Activity– DBCC INPUTBUFFER(ProcessID)– DBCC OUTPUTBUFFER(ProcessID)

Understanding Locking

• Coordinates multiple accesses to the same data• Ensures ACID Properties for transactions

(Atomic, Consistent, Independent, Durable)• Contention can reduce performance• Locking granularity:

– Row-Level, Page-Level, Table-Level, etc.

• Lock Modes: – Shared, Exclusive, etc.

• Lock escalation

Understanding Blocking

• Blocking– When transaction(s) must wait for a lock on a

resource– LOCK_TIMEOUT setting (default = wait

forever)

• Locking Models:– Pessimistic– Optimistic

Transaction Isolation Levels

• Balance of concurrency (performance) vs. consistency– Affects SELECT queries– SET TRANSACTION ISOLATION LEVEL

• Transaction Isolation Levels– READ UNCOMMITTED– READ COMMITTED (default)– REPEATABLE READ– SERIALIZABLE

• Row-Versioning:– ALLOW_SNAPSHOT_ISOLATION – READ_COMMITTED_SNAPSHOT

Monitoring Locking Activity

• Activity Monitor• SQL Profiler

– Locks Event Category

• System Monitor: – SQL Server Locks Object

• System Views– Sys.DM_Tran_Locks– Sys.DM_Exec_Requests

• System Stored Procedures– sp_Lock

Understanding the Deadlock Process

• Deadlocks:– Two or more tasks permanently block each other

based on resource locks– Default resolution is within 5 seconds

• Deadlock victim – Transaction is rolled-back– Process receives a 1205 error

• Example:– Process 1 locks the Customers table and requires

access to the Orders Table– Process 2 locks the Orders table and requires access

to the Customers Table

Avoiding Deadlocks

• Minimize transaction times– Commit / Rollback transactions as quickly as

possibly– Avoid user-related time within a transaction

• Access objects in a consistent order

• Change the transaction isolation level– Use a lower level isolation level, if appropriate– Use snapshot-based isolation levels

Deadlock Victims

• Deadlock priorities:– SET DEADLOCK_PRIORITY (LOW, NORMAL, HIGH, integer)

• Deadlock resolution:– Lower priority is killed first– If equal priorities, least expensive transaction

becomes the victim– Application or user should attempt to re-run

the transaction

Monitoring Deadlocks

• SQL Server Error Log• SQL Profiler

– Locks Event Category• Lock:Deadlock Chain• Lock:Deadlock• Deadlock Graph

– Events Extraction Trace Property– Export deadlock XML (.xdl) file

• Viewing Deadlock Files– SQL Server Management Studio (File Open

SQL Deadlock Files (*.xdl)

Deadlock Graph

Understanding Partitions

• Physical distribution of data– Partitions can be on the same or different filegroups

• Partitioning Methods– Horizontal Partitioning– Vertical Partitioning

• Example:– Partition 1: Current Sales Data (Current Year)– Partition 2: Archived Sales Data

Benefits of Partitioning

• No special requirements for users or applications

• Can partition tables and indexes

• Fast transfer of data between partitions

• Supports “Sliding” windows

• Can improve query performance

• Simplifies management of large tables

• Increases maintenance performance (indexes)

Designing Partitions

• Reasons to implement partitioning– Large tables– Performance problems– Data usage (read-only historical data)– Partial backups

• Supported in Enterprise and Developer Editions– Can have up to 1,000 partitions

Implementing Partitions

• Process:– Create a partition function– Create a partition scheme– Create a table using the scheme

• Other options:– Managing partitioned data– Creating CHECK constraints

Creating a Partition Function

• Defines sets of data based on ranges within the partitioning column– Examples: Dates, Row IDs, alpha values– RANGE LEFT (default) / RIGHT– Computed columns must be persisted

• Example:CREATE PARTITION FUNCTION fn_part1 (int) AS RANGE LEFT FOR VALUES (1000, 2000, 3000)

• $PARTITION Function shows on which partition data would reside

Creating a Partition Scheme

• Defines storage options for each partition• Based on a single partition function• Can specify files or filegroups for storage

– A filegroup can be used for multiple partitions

• Example:CREATE PARTITION SCHEME Partition1AS PARTITION PartitionFunctionTO (FileGroup1, FileGroup2, FileGroup3)

Creating Partitioned Tables

• Specify the partition scheme when creating a table– CREATE TABLE … ON PartitionScheme– Data storage will be based on the partition

scheme

Creating Partitioned Indexes

• “Aligned” partitioning– Index partitions are based on table partitions– Managed automatically by SQL Server– Optimizes switching performance

• Manual partition management– Can use CREATE INDEX … ON clause

• Partition column is automatically included in clustered and non-clustered indexes

Managing Partitioned Data

• Physical location of data may not be moved (only pointers are updated)

• CHECK constraints can be used to manage data• “Sliding Windows”• Partition functions can be modified

– Splitting: Adding a new partition– Merging: Combining two partitions

• Partition switching– ALTER TABLE … SWITCH– Moves data between partitions

Working with Partitioned Data

• SQL Server Management Studio – Database Reports Disk Space Used by

Partitions ObjectName

• System Views:– Sys.Tables / Sys.Indexes– Sys.Partitions– Sys.Partition_Schemes– Sys.Partition_Parameters– Sys.Partition_Range_Values

For More Information

• Resources from Anil Desai– Web Site (http://AnilDesai.net)– E-Mail: Anil@AnilDesai.net – Keystone Learning Course: “Microsoft

SQL Server 2005: Implementation and Maintenance (Exam 70-431)”

– The Rational Guide to Managing Microsoft Virtual Server 2005

– The Rational Guide to Scripting Microsoft Virtual Server 2005