T-SQL Enhancements in T-SQL Enhancements in SQL Server “Yukon”SQL Server “Yukon”

Syed Sajid WasimSyed Sajid WasimOnirbanOnirban

CRAWiz TeamCRAWiz Team

AgendaAgenda Query language: do more in a single Query language: do more in a single

statement statement Recursive queriesRecursive queries Ranking functionsRanking functions Relational operatorsRelational operators DML with outputsDML with outputs Fulltext searchFulltext search

Richer data typesRicher data types Procedural T-SQLProcedural T-SQL

Common table expressions Common table expressions (CTEs) and Recursive Queries(CTEs) and Recursive Queries

Ability to traverse recursive hierarchies Ability to traverse recursive hierarchies in a single queryin a single query

ScenariosScenarios Hierarchy in a table (MGRID-EMPID, Hierarchy in a table (MGRID-EMPID,

Part-Subpart)Part-Subpart) Find all employees reporting to a manager orFind all employees reporting to a manager or Find all parts required to assemble a productFind all parts required to assemble a product

Common Table ExpressionsCommon Table Expressions As per SQL-99As per SQL-99 Common table expressionsCommon table expressions

WITH WITH <<CTENameCTEName>> ( < ( <column-listcolumn-list> )> )ASAS( <( <CTECTE>)>)<<SELECT using CTESELECT using CTE>>

Both recursive and non-recursive formsBoth recursive and non-recursive forms Non-recursive:Non-recursive:

Rewrite queries with derived tables to be more readable.Rewrite queries with derived tables to be more readable.

Recursive CTEsRecursive CTEs Recursive, when <CTE> references itselfRecursive, when <CTE> references itself Recursive form of CTERecursive form of CTE

<non-recursive SELECT><non-recursive SELECT>UNION ALLUNION ALL<SELECT referencing CTE><SELECT referencing CTE>

Recursion stops when 2Recursion stops when 2ndnd SELECT SELECT produces empty resultsproduces empty results

Initialize

Accumulate

Without Recursive QueriesWithout Recursive QueriesDECLARE @RowsAdded int DECLARE @RowsAdded int

-- table variable to hold accumulated results -- table variable to hold accumulated results DECLARE @reports TABLE (empid nchar(5) primary key, empname nvarchar(50) NOT NULL, mgrid DECLARE @reports TABLE (empid nchar(5) primary key, empname nvarchar(50) NOT NULL, mgrid

nchar(5), title nvarchar(30), processed tinyint default 0) nchar(5), title nvarchar(30), processed tinyint default 0)

-- initialize @Reports with direct reports of the given employee -- initialize @Reports with direct reports of the given employee INSERT @reports INSERT @reports SELECT empid, empname, mgrid, title, 0 SELECT empid, empname, mgrid, title, 0 FROM employees FROM employees WHERE empid = ‘12345’WHERE empid = ‘12345’

SET @RowsAdded = @@rowcount SET @RowsAdded = @@rowcount

-- While new employees were added in the previous iteration -- While new employees were added in the previous iteration WHILE @RowsAdded > 0 WHILE @RowsAdded > 0 BEGIN /*Mark all employee records whose direct reports are going to be found in this BEGIN /*Mark all employee records whose direct reports are going to be found in this

iteration with processed=1.*/ iteration with processed=1.*/ UPDATE @reports UPDATE @reports SET processed = 1 SET processed = 1 WHERE processed = 0 WHERE processed = 0

-- Insert employees who report to employees marked 1. -- Insert employees who report to employees marked 1. INSERT @reports INSERT @reports SELECT e.empid, e.empname, e.mgrid, e.title, 0 SELECT e.empid, e.empname, e.mgrid, e.title, 0 FROM employees e, @reports r FROM employees e, @reports r WHERE e.mgrid=r.empid and e.mgrid <> e.empid and r.processed = 1 WHERE e.mgrid=r.empid and e.mgrid <> e.empid and r.processed = 1

SET @RowsAdded = @@rowcount SET @RowsAdded = @@rowcount /*Mark all employee records whose direct reports have been found in this iteration.*/ /*Mark all employee records whose direct reports have been found in this iteration.*/

UPDATE @reports SET processed = 2 WHERE processed = 1 UPDATE @reports SET processed = 2 WHERE processed = 1 END END

With Recursive QueriesWith Recursive QueriesWITH WITH EmpCTEEmpCTE(empid, empname, mgrid)(empid, empname, mgrid)ASAS( ( SELECT empid, empname, mgridSELECT empid, empname, mgrid FROM EmployeesFROM Employees WHERE empid = ‘12345’WHERE empid = ‘12345’ UNION ALLUNION ALL SELECT E.empid, E.empname, E.mgridSELECT E.empid, E.empname, E.mgrid FROM Employees AS E JOIN FROM Employees AS E JOIN EmpCTEEmpCTE AS M AS M ON E.mgrid = M.empidON E.mgrid = M.empid))SELECT * FROM EmpCTESELECT * FROM EmpCTE

Ranking FunctionsRanking Functions Ranking functions in Yukon Ranking functions in Yukon

RANK()RANK() DENSE_RANK()DENSE_RANK() NTILE(<expression>) NTILE(<expression>) ROW_NUMBER() ROW_NUMBER()

Syntax based on SQL-99 OLAP ExtensionsSyntax based on SQL-99 OLAP Extensions<<ranking_functionranking_function> > OVER( [PARTITION BY <OVER( [PARTITION BY <columncolumn>] >]

ORDER BY <ORDER BY <column>column>))

Ranking Functions: ScenariosRanking Functions: Scenarios Data analysis (RANK, DENSE_RANK, Data analysis (RANK, DENSE_RANK,

NTILE)NTILE) Ability to generate ranks based on different Ability to generate ranks based on different

criteria in same querycriteria in same query Ability to separate presentation order from ranksAbility to separate presentation order from ranks

Paging using ROW_NUMBERPaging using ROW_NUMBER Common scenario for walking through result setsCommon scenario for walking through result sets

Ranking Function Scenarios:Ranking Function Scenarios:Data analysisData analysis

SELECTSELECTRANKRANK() OVER(ORDER BY UnitSales), () OVER(ORDER BY UnitSales), RANKRANK() OVER(ORDER BY Revenue), () OVER(ORDER BY Revenue), RANKRANK() OVER(ORDER BY UnitProfit) () OVER(ORDER BY UnitProfit)

FROM ProductSales FROM ProductSales ORDER BY NameORDER BY Name

Find ranking of products based on Unit sales, Revenue

Ranking functions, Ranking functions, paging using paging using row_numberrow_number

New Relational Operators New Relational Operators PIVOT/UNPIVOT/APPLYPIVOT/UNPIVOT/APPLY

PIVOTPIVOT Transforms a set of rows to columnsTransforms a set of rows to columns Similar to Access TRANSFORMSimilar to Access TRANSFORM Useful for open schemas/OLAP scenariosUseful for open schemas/OLAP scenarios

UNPIVOTUNPIVOT Reverse operation of PIVOTReverse operation of PIVOT

APPLYAPPLY Allows evaluating a table-valued function for every Allows evaluating a table-valued function for every

row of outer-tablerow of outer-table

New Relational OperatorsNew Relational OperatorsPIVOTing Name-Value pairsPIVOTing Name-Value pairs

SELECT *SELECT *FROM table FROM table PIVOT(MIN(PIVOT(MIN(PropValPropVal) FOR ) FOR PropNamePropName IN ([Name], IN ([Name],[Author])) t[Author])) t

ObjIDObjID PropNamePropName PropValPropVal11 NameName x.docx.doc11 CrDateCrDate 12/3/200112/3/200122 NameName Sales.xlsSales.xls22 AuthorAuthor Mary Mary

HigginsHiggins

ObjIDObjID NameName AuthorAuthor11 Spec.docSpec.doc NULLNULL22 Sales.xlsSales.xls Mary Mary

HigginsHiggins

DML with OUTPUTDML with OUTPUTOUTPUT clause for DMLOUTPUT clause for DML Ability to return rows as part of DML operationsAbility to return rows as part of DML operations Use “Inserted” and “Deleted” columns available to get Use “Inserted” and “Deleted” columns available to get

pre- and post-update valuespre- and post-update values Option to store returned rowsOption to store returned rows

OUTPUT… INTO…OUTPUT… INTO…

UPDATE OrdersUPDATE OrdersSET status=’processed’SET status=’processed’OUTPUT DELETED.*, INSERTED.* OUTPUT DELETED.*, INSERTED.* WHERE status=‘unprocessed’WHERE status=‘unprocessed’

Full-Text SearchFull-Text Search Fast and powerful index and query Fast and powerful index and query

of textual dataof textual dataSELECT ProductName SELECT ProductName FROM Products FROM Products WHERE CONTAINS(ProductName, 'spread WHERE CONTAINS(ProductName, 'spread NEAR Boysenberry')NEAR Boysenberry')

Scalability, Performance enhancements:Scalability, Performance enhancements: Index building is order of magnitude fasterIndex building is order of magnitude faster Query Performance is 30-50% fasterQuery Performance is 30-50% faster

Full-Text Search: New FeaturesFull-Text Search: New Features Thesaurus supportThesaurus support Multi-column specification on Multi-column specification on

Full-Text queriesFull-Text queriesCONTAINS((col1,col2), ‘Yukon’)CONTAINS((col1,col2), ‘Yukon’)

Improved language support, configurable Improved language support, configurable accent sensitivity/insensitivity accent sensitivity/insensitivity

Enhanced support for distributed queries:Enhanced support for distributed queries: Fulltext queries against linked serversFulltext queries against linked servers

Support for indexed viewsSupport for indexed views

Fulltext Search: Better Fulltext Search: Better ManageabilityManageability

Integrated Backup/Restore: Recovery Integrated Backup/Restore: Recovery process fully includes Full-Text catalogsprocess fully includes Full-Text catalogs

Enhanced transportability of Full-Text Enhanced transportability of Full-Text catalogs through database attach/detachcatalogs through database attach/detach

Instance-level resources (no more shared Instance-level resources (no more shared components) components)

MS Search runs in same security context as MS Search runs in same security context as SQL ServerSQL Server

AgendaAgenda Expressive power – do more in a Expressive power – do more in a

single T-SQL statementsingle T-SQL statement Richer data typesRicher data types

XMLXML Varchar(max), nvarchar(max), Varchar(max), nvarchar(max),

varbinary(max)varbinary(max) Procedural T-SQLProcedural T-SQL

XML SupportXML SupportScenariosScenarios

Platform independent data/message format that Platform independent data/message format that enables data and application Integrationenables data and application Integration

Query and update semi-structured dataQuery and update semi-structured data Documents, emailDocuments, email

Now supported deeply in the SQL Server Now supported deeply in the SQL Server databasedatabase

Learn more about XML data type tomorrow: Learn more about XML data type tomorrow: DAT 402 (10/29/03 10:00AM)DAT 402 (10/29/03 10:00AM)

XML data typeXML data type First class data type in T-SQLFirst class data type in T-SQL

Columns, Variables, ParametersColumns, Variables, Parameters Optionally constrained by XML SchemaOptionally constrained by XML Schema Data ManipulationData Manipulation

XQuery (W3C Standard) + DMLXQuery (W3C Standard) + DML Ability to reference columns variables in XQueryAbility to reference columns variables in XQuery

XML IndexesXML IndexesCREATE TABLE Candidates (Id int, CREATE TABLE Candidates (Id int,

Resume XML)Resume XML)SELECT Id, Resume::Query(‘//Education’) SELECT Id, Resume::Query(‘//Education’) FROM CandidatesFROM Candidates

Varchar(max) TypeVarchar(max) Type Extension to varchar, nvarchar, varbinary up Extension to varchar, nvarchar, varbinary up

to 2GBto 2GB Uses MAX size specifierUses MAX size specifier

CREATE TABLE myTableCREATE TABLE myTable(Id int, (Id int,

Picture varbinary(max))Picture varbinary(max))

Alternative to text/ntext/imageAlternative to text/ntext/image No text pointer supportNo text pointer support

Varchar(max) TypeVarchar(max) Type

ComparisonsComparisons TriggersTriggers ConcatenationConcatenation

AggregatesAggregates ParametersParameters VariablesVariables

Brings together programming model of small and large values

Facilitates smooth transition when small string/binary data outgrows 8k limit

Varchar(max) TypeVarchar(max) Type All string functions operate on varchar(max)All string functions operate on varchar(max)

SUBSTRING used to read chunksSUBSTRING used to read chunks UPDATE statement enhanced to support UPDATE statement enhanced to support

update of CHUNKSupdate of CHUNKS

UPDATE myTableUPDATE myTableSET Picture::Write(@newchunk, SET Picture::Write(@newchunk, @offset, @remove)@offset, @remove)

AgendaAgenda Expressive power – do more in a single T-SQL Expressive power – do more in a single T-SQL

statement statement Richer data typesRicher data types Procedural T-SQLProcedural T-SQL

DDL TriggersDDL Triggers Transaction abort handlingTransaction abort handling Performance: Statement-level recompilePerformance: Statement-level recompile

DDL TriggersDDL Triggers Extension of traditional triggers for DDL eventsExtension of traditional triggers for DDL events Triggering events include all DDL statementsTriggering events include all DDL statements

CREATE_TABLE, ALTER_PROCEDURE, CREATE_TABLE, ALTER_PROCEDURE, DROP_LOGIN, etc.DROP_LOGIN, etc.

Scoping at Database and Server levelsScoping at Database and Server levels Event data available inside trigger through Event data available inside trigger through

eventdata() functioneventdata() function

DDL Triggers: ScenariosDDL Triggers: Scenarios Enforce development rules/standards for Enforce development rules/standards for

objects in a DBobjects in a DB Fail CREATE/ALTER if rule is violatedFail CREATE/ALTER if rule is violated

Protect from accidental dropsProtect from accidental drops Object checkin/checkoutObject checkin/checkout Source versioningSource versioning Log management activityLog management activity

DDL TriggersDDL Triggers

Transaction Abort Handling: Transaction Abort Handling: ScenariosScenarios

Log state when errors fatal to the transaction Log state when errors fatal to the transaction occuroccur

Eliminate “if @@error” codeEliminate “if @@error” code If you can run w/ XACT_ABORT ONIf you can run w/ XACT_ABORT ON

Hold on to transaction after severe errorHold on to transaction after severe error Explicitly doom transactions using Explicitly doom transactions using

RAISERRORRAISERROR

Tran Abort Handling: DetailsTran Abort Handling: Details TRY/CATCH blocksTRY/CATCH blocks

Control flow shifts to CATCH on errors fatal to Control flow shifts to CATCH on errors fatal to the transactionthe transaction

Transaction remains in “doomed” state until Transaction remains in “doomed” state until explicitly rolled backexplicitly rolled back

No actions which result in log writes may be performed No actions which result in log writes may be performed in a doomed transactionin a doomed transaction

@@error may be queried as first statement in @@error may be queried as first statement in CATCH blockCATCH block

RAISERROR … WITH TRAN_ABORT to RAISERROR … WITH TRAN_ABORT to explicitly doom transactionexplicitly doom transaction

Transaction Abort Handling: Transaction Abort Handling: ExampleExample

CREATE PROCEDURE add_to_foo @a int, @b nvarchar(1000)CREATE PROCEDURE add_to_foo @a int, @b nvarchar(1000)ASASBEGIN TRYBEGIN TRY

BEGIN TRANBEGIN TRAN--Constraint violations cause txn/batch-abort; control to client--Constraint violations cause txn/batch-abort; control to clientINSERT foo VALUES (@a, @b)INSERT foo VALUES (@a, @b)

COMMIT TRANCOMMIT TRANEND TRYEND TRYBEGIN CATCH TRAN_ABORTBEGIN CATCH TRAN_ABORT

ROLLBACKROLLBACKINSERT bad_foo VALUES (@a, @b, GETDATE())INSERT bad_foo VALUES (@a, @b, GETDATE())RAISERROR (‘Logged bad insert’, 17, 1) WITH TRAN_ABORTRAISERROR (‘Logged bad insert’, 17, 1) WITH TRAN_ABORT

END CATCHEND CATCH

Statement Level RecompileStatement Level Recompile SQL Server 2000: Module-level compilation and SQL Server 2000: Module-level compilation and

recompilationrecompilation Module: sp, trigger, function, batch, etc.Module: sp, trigger, function, batch, etc.

Yukon: Module-level compilation, statement level Yukon: Module-level compilation, statement level recompilationrecompilation

Recompilation due to various conditions: Recompilation due to various conditions: E.g., statistics change, changes in SET options, etc. E.g., statistics change, changes in SET options, etc.

Statement Level RecompileStatement Level Recompile Recompilations cost less; fewer Recompilations cost less; fewer

recompilationsrecompilations Statement level plans cached along with Statement level plans cached along with

module level plansmodule level plans Scenarios that benefit most:Scenarios that benefit most:

Recompiles caused by statistics changesRecompiles caused by statistics changes Batch is large and only small portion needs recompile Batch is large and only small portion needs recompile

(single statement in the case of statistics change)(single statement in the case of statistics change) New table is created inside the batchNew table is created inside the batch

T-SQL DebuggingT-SQL Debugging No more configuration stepsNo more configuration steps Works out of the box with:Works out of the box with:

SQL Workbench (beta 2)SQL Workbench (beta 2) Visual Studio “Whidbey” (now)Visual Studio “Whidbey” (now)

Debugging SQL client → SQL Server using Debugging SQL client → SQL Server using Visual Studio “Whidbey”Visual Studio “Whidbey”

Seamless cross-language debugging between T-Seamless cross-language debugging between T-SQL and managedSQL and managed