Outline

SQL Server OptimizerEnumeration architectureSearch space: flexibility/extensibilityCost and statistics

Automatic Physical TuningDatabase Tuning AdvisorNew Directions

Running Example (TPC-H Database)

Primary Index

Secondary Index

Running Example (Query)

SELECT l_orderkey, l_linenumber, o_orderstatusFROM lineitem JOIN orders ON l_orderkey = o_orderkeyWHERE l_suppkey < 2000 AND l_partkey < 2000

lineitem

l_suppkey<2000, l_partkey<2000

l_orderkey=o_orderkey

orders

l_orderkey, l_linenumber, o_orderstatus

Sample query: Obtain information about certain ordered line-items that are filtered by suppliers and parts.

SQL Server Query Optimizer Based on Cascades Framework

Transformation-based, top-down approach Optimization = Tasks + Memo

( Programs = Algorithms + Data Structures )

Fully cost-based Flexible and Extensible

Search space easy to change New operators and rules easy to add

SQLQuery

Parsing,Validation

Normalization(predicate unfolding, join collapsing, view

substitution, etc.)

Cost-based Optimization

Execution Plan

The Memo Search Space Memory

Compactly stores all explored alternatives (AND-OR graph) Groups together equivalent operator trees and their plans Provides memoization, duplicate detection, property and cost

management, etc.

1) SELECT (b>20, )

b>20

S1) SELECT (a<10, )

a<10

R

2) JOIN (x=y, , )1) SELECT (a<10, )a<10 b>20

R S

1) GET(S)S1) GET(R)R

b>20R

S

1) JOIN (x=y, , )2) ...

3) ...

GroupsExpressions

Initialize MemoOptimize Root Group

Optimize GroupExplore GroupObtain enforcers and implementation rules for all expressionsApply Rules by Promise

Optimize InputsObtain optimization contextOptimize children groupsCalculate costs, pick best

Apply RulePruning checksGenerate bindings, substitutesRestrict rulesetoptimizing?Opt Inputs:Explore

Explore ExpressionApply-always rulesExplore InputsRest of Apply-always RulesOther Rules by Promise

Explore GroupFor each expression-Get Guidance -Explore Expression.

Optimization Tasks

SQL Server Optimizer: Summary

Transformation-based, top-down approachNo need for bottom-up interesting orders

Fully Cost-basedNo separation into phases (heuristic+cost)

Flexible and ExtensibleNew operators, rules, and strategies are simple to add

Adaptive Automatic statistics create and refresh Automatic optimization levels Physical Tuning

Problem Statement

Workload Database

Configuration

Physical Design Tool

Set of physical

structures (i.e., indexes and views) that make

similar workloads execute as

fast as possible

Challenges Recommend a variety of physical structures.

Indexes, indexed views, partitions, XML indexes, etc.

Support space constraints, update queries. Exceptionally large search space, especially for

materialized views and partitions. Strong interaction among access paths. Merging needed: Optimal solution with suboptimal

parts. Cannot implement and test alternatives!

Industrial-strength quality (not trivial!)

Design Principles

Workload-driven: Take into account database usage.

What-if API: Determine impact without actually materializing physical design.

Don’t second-guess the query optimizer! An index is useful only if the query optimizer decides

to use it. Don’t use external estimator of goodness, but the

optimizer-estimated cost.

Keep tool reasonably separated from today’s optimizer (extensibility).

Database Tuning Advisor in YukonWorkload

Compress Workload

Candidate Selection(per query)

Enumeration

Time?Yes

No

Merging

Recommendation

TuningClient

…

Query Optimizer Database

ServerWhat-if

APIMetadata

- Create Hypothetical Index/View.- Optimize Query with respect to hypothetical configurations.

New Architecture

…

Query Optimizer

DatabaseServer

What-ifAPI

Metadata

Relaxation

Time?Yes

No

Recommendation

TuningClient

Workload

Request Identification

Get Optimal Configuration(per query)

Instrumenting the query optimizer. Search strategy based on relaxations.

RequestsAPI

Related Bibliography Surajit Chaudhuri

An Overview of Query Optimization in Relational Systems. PODS 1998: 34-43

Goetz GraefeThe Cascades Framework for Query Optimization. IEEE Data Eng. Bull. 18(3): 19-29 (1995)

Surajit Chaudhuri, Vivek R. NarasayyaAn Efficient Cost-Driven Index Selection Tool for Microsoft SQL Server. VLDB 1997: 146-155

Sanjay Agrawal, Surajit Chaudhuri, Vivek R. NarasayyaAutomated Selection of Materialized Views and Indexes in SQL Databases. VLDB 2000: 496-505

Nicolas Bruno, Surajit ChaudhuriAutomatic Physical Database Tuning: A Relaxation-based Approach. SIGMOD 2005