business intelligence and...

SQL_2008 / Microsoft SQL Server 2012: ABG / Petkovic / 176160-8 / Chapter 23

Chapter 23

In This Chapter

c Window Constructc Extensions of GROUP BYc OLAP Query Functionsc Standard and Nonstandard

Analytic Functions

Business Intelligence and Transact-SQL

Ch23.indd 627 1/25/12 12:18:26 PM

6 2 8 M i c r o s o f t S Q L S e r v e r 2 0 1 2 : A B e g i n n e r ’s G u i d e


Have you ever tried to write a Transact-SQL query that computes the percentage change in values between the last two quarters? Or one that implements cumulative sums or sliding aggregations? If you have ever tried,

you know how difficult these tasks are. Today, you don’t have to implement them anymore. The SQL:1999 standard has adopted a set of online analytical processing (OLAP) functions that enable you to easily perform these calculations as well as many others that used to be very complex for implementation. This part of the SQL standard is called SQL/OLAP. Therefore, SQL/OLAP comprises all functions and operators that are used for data analysis.

Using OLAP functions has several advantages for users:

Users with standard knowledge of the SQL language can easily specify the Cc

calculations they need.Database systems, such as the Database Engine, can perform these calculations Cc

much more efficiently.Because there is a standard specification of these functions, they’re now much Cc

more economical for tool and application vendors to exploit.Almost all the analytic functions proposed by the SQL:1999 standard are Cc

implemented in enterprise database systems in the same way. For this reason, you can port queries in relation to SQL/OLAP from one system to another, without any code changes.

The Database Engine offers many extensions to the SELECT statement that can be used primarily for analytic operations. Some of these extensions are defined according to the SQL:1999 standard and some are not. The following sections describe both standard and nonstandard SQL/OLAP functions and operators.

The most important extension of Transact-SQL concerning data analysis is the window construct, which will be described next.

Window ConstructA window (in relation to SQL/OLAP) defines a partitioned set of rows to which a function is applied. The number of rows that belong to a window is dynamically determined in relation to the user’s specifications. The window construct is specified using the OVER clause.

Ch23.indd 628 1/25/12 12:18:26 PM


C h a p t e r 2 3 : B u s i n e s s I n t e l l i g e n c e a n d Tr a n s a c t - S Q L 6 2 9

The standardized window construct has three main parts:

PartitioningCc

OrderingCc

Aggregation groupingCc

NoteThe Database Engine doesn’t support aggregation grouping yet, so this feature is not discussed here.

Before you delve into the window construct and its parts, take a look at the table that will be used for the examples. Example 23.1 creates the project_dept table, shown in Table 23-1, which is used in this chapter to demonstrate Transact-SQL extensions concerning SQL/OLAP.

dept_name emp_cnt budget date_monthResearch 5 50000 01.01.2007

Research 10 70000 02.01.2007

Research 5 65000 07.01.2007

Accounting 5 10000 07.01.2007

Accounting 10 40000 02.01.2007

Accounting 6 30000 01.01.2007

Accounting 6 40000 02.01.2008

Marketing 6 100000 01.01.2008

Marketing 10 180000 02.01.2008

Marketing 3 100000 07.01.2008

Marketing NULL 120000 01.01.2008

Table 23-1 Content of the project_dept Table

Ch23.indd 629 1/25/12 12:18:26 PM



ExAmPLE 23.1

USE sample;

CREATE TABLE project_dept

( dept_name CHAR( 20 ) NOT NULL,

emp_cnt INT,

budget FLOAT,

date_month DATE );

The project_dept table contains several departments and their employee counts as well as budgets of projects that are controlled by each department. Example 23.2 shows the INSERT statements that are used to insert the rows shown in Table 23-1.

ExAmPLE 23.2

USE sample;

INSERT INTO project_dept VALUES

('Research', 5, 50000, '01.01.2007');


('Research', 10, 70000, '02.01.2007');


('Research', 5, 65000, '07.01.2007');


('Accounting', 5, 10000, '07.01.2007');


('Accounting', 10, 40000, '02.01.2007');


('Accounting', 6, 30000, '01.01.2007');


('Accounting', 6, 40000, '02.01.2008');


('Marketing', 6, 100000, '01.01.2008');


('Marketing', 10, 180000, '02.01.2008');


('Marketing', 3, 100000, '07.01.2008');


('Marketing', NULL, 120000, '01.01.2008');

PartitioningPartitioning allows you to divide the result set of a query into groups, so that each row from a partition will be displayed separately. If no partitioning is specified, the entire set of rows comprises a single partition. Although the partitioning looks like a grouping

Ch23.indd 630 1/25/12 12:18:26 PM



using the GROUP BY clause, it is not the same thing. The GROUP BY clause collapses the rows in a partition into a single row, whereas the partitioning within the window construct simply organizes the rows into groups without collapsing them.

The following two examples show the difference between partitioning using the window construct and grouping using the GROUP BY clause. Suppose that you want to calculate several different aggregates concerning employees in each department. Example 23.3 shows how the OVER clause with the PARTITION BY clause can be used to build partitions.

ExAmPLE 23.3

Using the window construct, build partitions according to the values in the dept_name column and calculate the sum and the average for the Accounting and Research departments:USE sample; SELECT dept_name, budget, SUM( emp_cnt ) OVER( PARTITION BY dept_name ) AS emp_cnt_sum, AVG( budget ) OVER( PARTITION BY dept_name ) AS budget_avg FROM project_dept WHERE dept_name IN ('Accounting', 'Research');

The result is

dept_name budget emp_cnt_sum budget_avgAccounting 10000 27 30000

Accounting 40000 27 30000



Research 50000 20 61666.6666666667

Research 70000 20 61666.6666666667

Research 65000 20 61666.6666666667

Example 23.3 uses the OVER clause to define the corresponding window construct. Inside it, the PARTITION BY option is used to specify partitions. (Both partitions in Example 23.3 are grouped using the values in the dept_name column.) Finally, an aggregate function is applied to the partitions. (Example 23.3 calculates two aggregates, the sum of the values in the emp_cnt column and the average value of budgets.) Again, as you can see from the result of the example, the partitioning organizes the rows into groups without collapsing them.

Example 23.4 shows a similar query that uses the GROUP BY clause.

Ch23.indd 631 1/25/12 12:18:26 PM



ExAmPLE 23.4

Group the values in the dept_name column for the Accounting and Research departments and calculate the sum and the average for these two groups:

USE sample;

SELECT dept_name, SUM(emp_cnt) AS cnt, AVG( budget ) AS budget_avg

FROM project_dept

WHERE dept_name IN ('Accounting', 'Research')

GROUP BY dept_name;

The result is

dept_name cnt budget_avgAccounting 27 30000

Research 20 61666.6666666667

As already stated, when you use the GROUP BY clause, each group collapses in one row.

NoteThere is another significant difference between the OVER clause and the GROUP BY clause. As can be seen from Example 23.3, when you use the OVER clause, the corresponding SELECT list can contain any column name from the table. This is obvious, because partitioning organizes the rows into groups without collapsing them. (If you add the budget column in the SELECT list of Example 23.4, you will get an error.)

Ordering and FramingThe ordering within the window construct is like the ordering in a query. First, you use the ORDER BY clause to specify the particular order of the rows in the result set. Second, it includes a list of sort keys and indicates whether they should be sorted in ascending or descending order. The most important difference is that ordering inside a window is applied only within each partition.

In contrast to SQL Server 2008, SQL Server 2012 supports ordering inside a window construct for aggregate functions. In other words, the OVER clause for aggregate functions can contain now the ORDER BY clause, too. Example 23.5 shows this.

ExAmPLE 23.5

Using the window construct, partition the rows of the project_dept table using the values in the dept_name column and sort the rows in each partition using the values in the budget column:

Ch23.indd 632 1/25/12 12:18:27 PM



USE sample;

SELECT dept_name, budget, emp_cnt,

SUM(budget) OVER(PARTITION BY dept_name ORDER BY budget) AS sum_dept

FROM project_dept;

The query in Example 23.5, which is generally called “cumulative aggregations,” or in this case “cumulative sums,” uses the ORDER BY clause to specify ordering within the particular partition. This functionality can be extended using framing. Framing means that the result can be further narrowed using two boundary points that restrict the set of rows to a subset (see the following example).

ExAmPLE 23.6

USE sample;

SELECT dept_name, budget, emp_cnt,

SUM(budget) OVER(PARTITION BY dept_name ORDER BY budget

ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)

AS sum_dept

FROM project_dept;

Example 23.6 uses two clauses: UNBOUNDED PRECEDING and CURRENT ROW to specify the boundary points of the selected rows. For the query in the example this means that based on the order of the budget values the displayed subset of rows is with no low boundary point and until the current row. (The result set contains all together 11 rows.)

The frame bounds used in Example 23.6 are not the only ones you can use. The UNBOUNDED FOLLOWING clause means that the specified frame does not have an upper boundary point. Also, both boundary points can be specified using an offset from the current row. In other words, you can use the n PRECEDING or n FOLLOWING clauses to specify n rows before or n rows after the current one, respectively. Therefore, the following frame specifies all together three rows: the current row, the previous one, and the next one:

ROWS BETWEEN 1 PRECEDING and 1 FOLLOWING

SQL Server 2012 also introduces two new functions related to framing: LEAD and LAG. LEAD has the ability to compute an expression on the next rows (rows that are going to come after the current row). In other words, the LEAD function returns the next nth row value in an order. The function has three parameters: The first one specifies the name of the column to compute the leading row, the second one is the index of the leading row relative to the current row, and the last one is the value to return if the offset points to a row outside of the partition range. (The semantics of the LAG function is similar: it returns the previous nth row value in an order.)

Ch23.indd 633 1/25/12 12:18:27 PM



Example 23.6 uses the ROWS clause to limit the rows within a partition by physically specifying the number of rows preceding or following the current row. Alternatively, SQL Server 2012 supports the RANGE clause, which logically limits the rows within a partition. In other words, when you use the ROWS clause, the exact number of rows will be specified based on the defined frame. On the other hand, the RANGE clause does not define the exact number of rows, because the specified frame can contain duplicates, too.

You can use several columns from a table to build different partitioning schemas in a query, as shown in Example 23.7.

ExAmPLE 23.7

Using the window construct, build two partitions for the Accounting and Research departments: one using the values of the budget column and the other using the values of the dept_name column. Calculate the sums for the former partition and the averages for the latter partition.

USE sample;

SELECT dept_name, CAST( budget AS INT ) AS budget,

SUM( emp_cnt ) OVER( PARTITION BY budget ) AS emp_cnt_sum,

AVG( budget ) OVER( PARTITION BY dept_name ) AS budget_avg

FROM project_dept

WHERE dept_name IN ('Accounting', 'Research');

The result is

dept_name budget emp_cnt_sum budget_avgAccounting 10000 5 30000




Research 50000 5 61666.6666666667

Research 65000 5 61666.6666666667

Research 70000 10 61666.6666666667

The query in Example 23.7 has two different partitioning schemas: one over the values of the budget column and one over the values of the dept_name column. The former is used to calculate the number of employees in relation to the departments with the same budget. The latter is used to calculate the average value of budgets of departments grouped by their names.

Ch23.indd 634 1/25/12 12:18:27 PM



Example 23.8 shows how you can use the NEXT VALUE FOR expression of the CREATE SEQUENCE statement to control the order in which the values are generated using the OVER clause. (For the description of the CREATE SEQUENCE statement, see Chapter 6.)

ExAmPLE 23.8

USE sample;

CREATE SEQUENCE Seq START WITH 1 INCREMENT BY 1;

GO

CREATE TABLE T1 (col1 CHAR(10), col2 CHAR(10));

GO

INSERT INTO dbo.T1(col1, col2)

SELECT NEXT VALUE FOR Seq OVER(ORDER BY dept_name ASC), budget

FROM (SELECT dept_name, budget

FROM project_dept

ORDER BY budget, dept_name DESC

OFFSET 0 ROWS FETCH FIRST 5 ROWS ONLY) AS D;

The content of the T1 table is as follows:

col1 col21 10000

2 30000

3 40000

4 40000

5 50000

The first two statements create the Seq sequence and the auxiliary table T1. The following INSERT statement uses a subquery to filter the five departments with the highest budget, and generates sequence values for them. This is done using OFFSET/FETCH, which is described in Chapter 6. (You can find several other examples concerning OFFSET/FETCH in the subsection with the same name later in this chapter.)

Extensions of GROUP BYTransact-SQL extends the GROUP BY clause with the following operators and functions:

CUBECc

ROLLUPCc

Ch23.indd 635 1/25/12 12:18:27 PM



Grouping functionsCc

Grouping setsCc

The following sections describe these operators and functions.

CUBE OperatorThis section looks at the differences between grouping using the GROUP BY clause alone and grouping using GROUP BY in combination with the CUBE and ROLLUP operators. The main difference is that the GROUP BY clause defines one or more columns as a group such that all rows within any group have the same values for those columns. CUBE and ROLLUP provide additional summary rows for grouped data. These summary rows are also called multidimensional summaries.

The following two examples demonstrate these differences. Example 23.9 applies the GROUP BY clause to group the rows of the project_dept table using two criteria: dept_name and emp_cnt.

ExAmPLE 23.9

Using GROUP BY, group the rows of the project_dept table that belong to the Accounting and Research departments using the dept_name and emp_cnt columns:

USE sample;

SELECT dept_name, emp_cnt, SUM(budget) sum_of_budgets

FROM project_dept


GROUP BY dept_name, emp_cnt;

The result is

dept_name emp_cnt sum_of_budgetsAccounting 5 10000

Research 5 115000

Accounting 6 70000

Accounting 10 40000

Research 10 70000

Example 23.10 and its result set shows the difference when you additionally use the CUBE operator.

Ch23.indd 636 1/25/12 12:18:27 PM



ExAmPLE 23.10

Group the rows of the project_dept table that belong to the Accounting and Research departments using the dept_name and emp_cnt columns and additionally display all possible summary rows:

USE sample;


FROM project_dept


GROUP BY CUBE (dept_name, emp_cnt);

The result is


Research 5 115000

NULL 5 125000

Accounting 6 70000

NULL 6 70000

Accounting 10 40000

Research 10 70000

NULL 10 110000

Accounting NULL 120000

Research NULL 185000

NULL NULL 305000

The main difference between the last two examples is that the result set of Example 23.9 displays only the values in relation to the grouping, while the result set of Example 23.10 contains, additionally, all possible summary rows. (Because the CUBE operator displays every possible combination of groups and summary rows, the number of rows is the same, regardless of the order of columns in the GROUP BY clause.) The placeholder for the values in the unneeded columns of summary rows is displayed as NULL. For example, the following row from the result set

NULL NULL 305000

Ch23.indd 637 1/25/12 12:18:27 PM



shows the grand total (that is, the sum of all budgets of all existing projects in the table), while the row

NULL 5 125000

shows the sum of all budgets for all projects that employ exactly five employees.

NoteThe syntax of the CUBE operator in Example 23.10 corresponds to the standardized syntax of that operator. Because of its backward compatibility, the Database Engine also supports the old-style syntax:

USE sample;


FROM project_dept


GROUP BY dept_name, emp_cnt

WITH CUBE;

ROLLUP OperatorIn contrast to CUBE, which returns every possible combination of groups and summary rows, the group hierarchy using ROLLUP is determined by the order in which the grouping columns are specified. Example 23.11 shows the use of the ROLLUP operator.

ExAmPLE 23.11

Group the rows of the project_dept table that belong to the Accounting and Research departments using the dept_name and emp_cnt columns and additionally display summary rows for the dept_name column:

USE sample;


FROM project_dept


GROUP BY ROLLUP (dept_name, emp_cnt);

The result is


Accounting 6 70000

Ch23.indd 638 1/25/12 12:18:27 PM





Research 5 115000

Research 10 70000


NULL NULL 305000

As you can see from the result of Example 23.11, the number of retrieved rows in this example is smaller than the number of displayed rows in the example with the CUBE operator. The reason is that the summary rows are displayed only for the first column in the GROUP BY ROLLUP clause.

NoteThe syntax used in Example 23.11 is the standardized syntax. The old-style syntax for ROLLUP is similar to the syntax for CUBE, which is shown in the second part of Example 23.10.

Grouping FunctionsAs you already know, NULL is used in combination with CUBE and ROLLUP to specify the placeholder for the values in the unneeded columns. In such a case, it isn’t possible to distinguish NULL in relation to CUBE and ROLLUP from the NULL value. Transact-SQL supports the following two standardized grouping functions that allow you to resolve the problem with the ambiguity of NULL:

GROUPINGCc

GROUPING_IDCc

The following subsections describe in detail these two functions.

GROUPING FunctionThe GROUPING function returns 1 if the NULL in the result set is in relation to CUBE or ROLLUP, and 0 if it represents the group of NULL values.

Example 23.12 shows the use of the GROUPING function.

Ch23.indd 639 1/25/12 12:18:27 PM



ExAmPLE 23.12

Using the GROUPING function, clarify which NULL values in the result of the following SELECT statement display summary rows:

USE sample;

SELECT dept_name, emp_cnt, SUM(budget) sum_b, GROUPING(emp_cnt) gr

FROM project_dept

WHERE dept_name IN ('Accounting', 'Marketing')

GROUP BY ROLLUP (dept_name, emp_cnt);

The result is

dept_name emp_cnt sum_b grAccounting 5 10000 0



Accounting NULL 120000 1

Marketing NULL 120000 0

Marketing 3 100000 0




NULL NULL 620000 1

If you take a look at the grouping column (gr), you will see that some values are 0 and some are 1. The value 1 indicates that the corresponding NULL in the emp_cnt column specifies a summary value, while the value 0 indicates that NULL stands for itself, i.e., it is the NULL value.

GROUPING_ID FunctionThe GROUPING_ID function computes the level of grouping. GROUPING_ID can be used only in the SELECT list, HAVING clause, or ORDER BY clause when GROUP BY is specified.

Example 23.13 shows the use of the GROUPING_ID function.

ExAmPLE 23.13USE sample; SELECT dept_name, YEAR(date_month), SUM(budget), GROUPING_ID (dept_name, YEAR(date_month)) AS gr_dept FROM project_dept GROUP BY ROLLUP (dept_name, YEAR(date_month));

Ch23.indd 640 1/25/12 12:18:27 PM



The result is

dept_name date_month budget gr_deptAccounting 2007 80000 0


Accounting NULL 120000 1

Marketing 2008 500000 0


Research 2007 185000 0

Research NULL 185000 1

NULL NULL 805000 3

The GROUPING_ID function is similar to the GROUPING function, but becomes very useful for determining the summarization of multiple columns, as is the case in Example 23.13.The function returns an integer that, when converted to binary, is a concatenation of the 1s and 0s representing the summarization of each column passed as the parameter of the function. (For example, the value 3 of the gr_dept column in the last row of the result means that summarization is done over both the dept_name and date_month columns. The binary value (11)2 is equivalent to the value 3 in the decimal system.)

Grouping SetsGrouping sets are an extension to the GROUP BY clause that lets users define several groups in the same query. You use the GROUPING SETS operator to implement grouping sets. Example 23.14 shows the use of this operator.

ExAmPLE 23.14

Calculate the sum of budgets for the Accounting and Research departments using the combination of values of the dept_name and emp_cnt columns first, and after that using the values of the single column dept_name:

USE sample;

SELECT dept_name, emp_cnt, SUM(budget) sum_budgets

FROM project_dept


GROUP BY GROUPING SETS ((dept_name, emp_cnt),(dept_name));

Ch23.indd 641 1/25/12 12:18:27 PM



The result is

dept_name emp_cnt sum_budgetsAccounting 5 10000

Accounting 6 70000

Accounting 10 40000


Research 5 115000

Research 10 70000


As you can see from the result set of Example 23.14, the query uses two different groupings to calculate the sum of budgets: first using the combination of values of the dept_name and emp_cnt columns, and second using the values of the single column dept_name. The first three rows of the result set display the sum of budgets for three different groupings of the first two columns (Accounting, 5; Accounting, 6; and Accounting, 10). The fourth row displays the sum of budgets for all Accounting departments. The last three rows displays the similar results for the Research department.

You can use the series of grouping sets to replace the ROLLUP and CUBE operators. For instance, the following series of grouping sets

GROUP BY GROUPING SETS ((dept_name, emp_cnt), (dept_name), ())

is equivalent to the following ROLLUP clause:

GROUP BY ROLLUP (dept_name, emp_cnt)

Also,

GROUP BY GROUPING SETS ((dept_name, emp_cnt), (emp_cnt, dept_name),

(dept_name), ())

is equivalent to the following CUBE clause:

GROUP BY CUBE (dept_name, emp_cnt)

OLAP Query FunctionsTransact-SQL supports two groups of functions that are categorized as OLAP query functions:

Ch23.indd 642 1/25/12 12:18:27 PM



Ranking functionsCc

Statistical aggregate functionsCc

The following subsections describe these functions.

NoteThe GROUPING function, discussed previously, also belongs to the OLAP functions.

Ranking FunctionsRanking functions return a ranking value for each row in a partition group. Transact-SQL supports the following ranking functions:

RANKCc

DENSE_RANKCc

ROW_NUMBERCc

Example 23.15 shows the use of the RANK function.

ExAmPLE 23.15

Find all departments with a budget not greater than 30000, and display the result set in descending order:

USE sample;

SELECT RANK() OVER(ORDER BY budget DESC) AS rank_budget,

dept_name, emp_cnt, budget

FROM project_dept

WHERE budget <= 30000;

The result is

rank_budget dept_name emp_cnt budget1 Accounting 6 30000

2 Accounting 5 10000

Example 23.15 uses the RANK function to return a number (in the first column of the result set) that specifies the rank of the row among all rows. The example uses the

Ch23.indd 643 1/25/12 12:18:27 PM



OVER clause to sort the result set by the budget column in the descending order. (In this example, the PARTITION BY clause is omitted. For this reason, the whole result set will belong to only one partition.)

NoteThe RANK function uses logical aggregation. In other words, if two or more rows in a result set are tied (have a same value in the ordering column), they will have the same rank. The row with the subsequent ordering will have a rank that is one plus the number of ranks that precede the row. For this reason, the RANK function displays “gaps” if two or more rows have the same ranking.

Example 23.16 shows the use of the two other ranking functions, DENSE_RANK and ROW_NUMBER.

ExAmPLE 23.16

Find all departments with a budget not greater than 40000, and display the dense rank and the sequential number of each row in the result set:

USE sample;

SELECT DENSE_RANK() OVER( ORDER BY budget DESC ) AS dense_rank,

ROW_NUMBER() OVER( ORDER BY budget DESC ) AS row_number,

dept_name, emp_cnt, budget

FROM project_dept

WHERE budget <= 40000;

The result is

dense_rank row_number dept_name emp_cnt budget1 1 Accounting 10 40000

1 2 Accounting 6 40000



The first two columns in the result set of Example 23.16 show the values for the DENSE_RANK and ROW_NUMBER functions, respectively. The output of the DENSE_RANK function is similar to the output of the RANK function (see Example 23.15). The only difference is that the DENSE_RANK function returns no “gaps” if two or more ranking values are equal and thus belong to the same ranking.

The use of the ROW_NUMBER function is obvious: it returns the sequential number of a row within a result set, starting at 1 for the first row.

Ch23.indd 644 1/25/12 12:18:27 PM



In the last two examples, the OVER clause is used to determine the ordering of the result set. As you already know, this clause can also be used to divide the result set produced by the FROM clause into groups (partitions), and then to apply an aggregate or ranking function to each partition separately.

Example 23.17 shows how the RANK function can be applied to partitions.

ExAmPLE 23.17

Using the window construct, partition the rows of the project_dept table according to the values in the date_month column. Sort the rows in each partition and display them in ascending order.

USE sample;

SELECT date_month, dept_name, emp_cnt, budget,

RANK() OVER( PARTITION BY date_month ORDER BY emp_cnt desc ) AS rank

FROM project_dept;

The result is

date_month dept_name emp_cnt budget rank2007-01-01 Accounting 6 30000 1

2007-01-01 Research 5 50000 2

2007-02-01 Research 10 70000 1

2007-02-01 Accounting 10 40000 1

2007-07-01 Research 5 65000 1

2007-07-01 Accounting 5 10000 1

2008-01-01 Marketing 6 100000 1

2008-01-01 Marketing NULL 120000 2

2008-02-01 Marketing 10 180000 1

2008-02-01 Accounting 6 40000 2

2008-07-01 Marketing 3 100000 1

The result set of Example 23.17 is divided (partitioned) into eight groups according to the values in the date_month column. After that the RANK function is applied to each partition. If you take a closer look and compare the previous example with Example 23.5, you will see that the last example works, while Example 23.5 displays an error, although both examples use the same window construct. As previously stated, Transact-SQL currently supports the OVER clause for aggregate functions only with the PARTITION BY clause, but in the case of ranking functions, the system supports general SQL standard syntax with the PARTITION BY and ORDER BY clauses.

Ch23.indd 645 1/25/12 12:18:28 PM



Statistical Aggregate FunctionsChapter 6 introduced statistical aggregate functions. There are four of them:

VARCc Computes the variance of all the values listed in a column or expression.VARPCc Computes the variance for the population of all the values listed in a column or expression.STDEVCc Computes the standard deviation of all the values listed in a column or expression. (The standard deviation is computed as the square root of the corresponding variance.)STDEVPCc Computes the standard deviation for the population of all the values listed in a column or expression.

You can use statistical aggregate functions with or without the window construct. Example 23.18 shows how the functions VAR and STDEV can be used with the window construct.

ExAmPLE 23.18

Using the window construct, calculate the variance and standard deviation of budgets in relation to partitions formed using the values of the dept_name column:

USE sample; SELECT dept_name, budget, VAR(budget) OVER(PARTITION BY dept_name) AS budget_var, STDEV(budget) OVER(PARTITION BY dept_name) AS budget_stdev FROM project_dept WHERE dept_name in ('Accounting', 'Research');

The result is

dept_name budget budget_var budget_stdevAccounting 10000 200000000 14142.135623731

Accounting 40000 200000000 14142.135623731

Accounting 30000 200000000 14142.135623731

Accounting 40000 200000000 14142.135623731

Research 50000 108333333,333333 10408.3299973306

Research 70000 108333333,333333 10408.3299973306

Research 65000 108333333,333333 10408.3299973306

Ch23.indd 646 1/25/12 12:18:28 PM



Example 23.18 uses the statistical aggregate functions VAR and STDEV to calculate the variance and standard deviation of budgets in relation to partitions formed using the values of the dept_name column.

Standard and Nonstandard Analytic FunctionsThe Database Engine contains the following standard and nonstandard OLAP functions:

TOPCc

OFFSET/FETCHCc

NTILECc

PIVOT and UNPIVOTCc

The second function, OFFSET/FETCH, is specified in the SQL standard, while the three others are Transact-SQL extensions. The following sections describe these analytic functions and operators.

TOP ClauseThe TOP clause specifies the first n rows of the query result that are to be retrieved. This clause should always be used with the ORDER BY clause, because the result of such a query is always well defined and can be used in table expressions. (A table expression specifies a sample of a grouped result table.) A query with TOP but without the ORDER BY clause is nondeterministic, meaning that multiple executions of the query with the same data must not always display the same result set.

Example 23.19 shows the use of this clause.

ExAmPLE 23.19

Retrieve the four projects with the highest budgets:

USE sample;

SELECT TOP (4) dept_name, budget

FROM project_dept

ORDER BY budget DESC;

Ch23.indd 647 1/25/12 12:18:28 PM



The result is

dept_name budgetMarketing 180000

Marketing 120000

Marketing 100000

Marketing 100000

As you can see from Example 23.19, the TOP clause is part of the SELECT list and is written in front of all column names in the list.

NoteYou should write the input value of TOP inside parentheses, because the system supports any self-contained expression as input.

The TOP clause is a nonstandard Transact-SQL implementation used to display the ranking of the top n rows from a table. A query equivalent to Example 23.19 that uses the window construct and the RANK function is shown in Example 23.20.

ExAmPLE 23.20

Retrieve the four projects with the highest budgets:

USE sample;

SELECT dept_name, budget

FROM (SELECT dept_name, budget,

RANK() OVER (ORDER BY budget DESC) AS rank_budget

FROM project_dept) part_dept

WHERE rank_budget <= 4;

The TOP clause can also be used with the additional PERCENT option. In that case, the first n percent of rows are retrieved from the result set. The additional option WITH TIES specifies that additional rows will be retrieved from the query result if they have the same value in the ORDER BY column(s) as the last row that belongs to the displayed set. Example 23.21 shows the use of the PERCENT and WITH TIES options.

Ch23.indd 648 1/25/12 12:18:28 PM



ExAmPLE 23.21

Retrieve the top 25 percent of rows with the smallest number of employees:

USE sample;

SELECT TOP (25) PERCENT WITH TIES emp_cnt, budget

FROM project_dept

ORDER BY emp_cnt ASC;

The result is

emp_cnt budgetNULL 120000

3 100000

5 50000

5 65000

5 10000

The result of Example 23.21 contains five rows, because there are three projects with five employees.

You can also use the TOP clause with UPDATE, DELETE, and INSERT statements. Example 23.22 shows the use of this clause with the UPDATE statement.

ExAmPLE 23.22

Find the three projects with the highest budget amounts and reduce them by 10 percent:

USE sample;

UPDATE TOP (3) project_dept

SET budget = budget * 0.9

WHERE budget in (SELECT TOP (3) budget

FROM project_dept

ORDER BY budget desc);

Example 23.23 shows the use of the TOP clause with the DELETE statement.

Ch23.indd 649 1/25/12 12:18:28 PM



ExAmPLE 23.23

Delete the four projects with the smallest budget amounts:

USE sample;

DELETE TOP (4)

FROM project_dept

WHERE budget IN

(SELECT TOP (4) budget FROM project_dept

ORDER BY budget ASC);

In Example 23.23, the TOP clause is used first in the subquery, to find the four projects with the smallest budget amounts, and then in the DELETE statement, to delete these projects.

OFFSET/FETCHChapter 6 showed how OFFSET/FETCH can be used for server-side paging. This application of OFFSET/FETCH is only one of many. Generally, OFFSET/FETCH allows you to filter several rows according to the given order. Additionally, you can specify how many rows of the result set should be skipped and how many of them should be returned. For this reason, OFFSET/FETCH is similar to the TOP clause. However, there are certain differences:

OFFSET/FETCH is a standardized way to filter data, while the TOP clause is an Cc

extension of Transact-SQL. For this reason, it is possible that OFFSET/FETCH will replace the TOP clause in the future.OFFSET/FETCH is more flexible than TOP insofar as it allows skipping of Cc

rows using the OFFSET clause. (The Database Engine doesn’t allow you to use the FETCH clause without OFFSET. In other words, even when no rows are skipped, you have to set OFFSET to 0.)The TOP clause is more flexible than OFFSET/FETCH insofar as it can be used Cc

in DML statements INSERT, UPDATE, and DELETE (see Examples 23.22 and 23.23).

Examples 23.24 and 23.25 show how you can use OFFSET/FETCH with the ROW_NUMBER() ranking function. (Before you execute the following examples, repopulate the project_dept table. First delete all the rows, and then execute the INSERT statements from Example 23.2.)

Ch23.indd 650 1/25/12 12:18:28 PM



ExAmPLE 23.24

USE sample;

SELECT date_month, budget, ROW_NUMBER()

OVER (ORDER BY date_month DESC, budget DESC) as row_no

FROM project_dept

ORDER BY date_month DESC, budget DESC

OFFSET 5 ROWS FETCH NEXT 4 ROWS ONLY;

The result is

date_month Budget row_no2007-07-01 65000 6

2007-07-01 10000 7

2007-02-01 70000 8

2007-02-01 40000 9

Example 23.24 displays the rows of the project_dept table in relation to the date_month and budget columns. (The first five rows of the result set are skipped and the next four are displayed.) Additionally, the row number of these rows is returned. The row number of the first row in the result set starts with 6 because row numbers are assigned to the result set before the filtering. (OFFSET/FETCH is part of the ORDER BY clause and therefore is executed after the SELECT list, which includes the ROW_NUMBER function. In other words, the values of ROW_NUMBER are determined before OFFSET/FETCH is applied.)

If you want to get the row numbers starting with 1, you need to modify the SELECT statement. Example 23.25 shows the necessary modification.

ExAmPLE 23.25

USE sample;

SELECT *, ROW_NUMBER()

OVER (ORDER BY date_month DESC, budget DESC) as row_no

FROM (SELECT date_month, budget

FROM project_dept

ORDER BY date_month DESC, budget DESC

OFFSET 5 ROWS FETCH NEXT 4 ROWS ONLY) c;

The result of Example 23.25 is identical to the result of Example 23.24 except that row numbers start with 1. The reason is that in Example 23.25 the query with OFFSET/FETCH is written as a table expression inside the outer query with the ROW_NUMBER() function in the SELECT list. That way, the values of ROW_NUMBER() are determined before OFFSET/FETCH is executed.

Ch23.indd 651 1/25/12 12:18:28 PM



NTILE FunctionThe NTILE function belongs to the ranking functions. It distributes the rows in a partition into a specified number of groups. For each row, the NTILE function returns the number of the group to which the row belongs. For this reason, this function is usually used to arrange rows into groups.

NoteThe NTILE function breaks down the data based only on the count of values.

Example 23.26 shows the use of the NTILE function.

ExAmPLE 23.26

USE sample;

SELECT dept_name, budget,

CASE NTILE(3) OVER (ORDER BY budget ASC)

WHEN 1 THEN 'Low'

WHEN 2 THEN 'Medium'

WHEN 3 THEN 'High'

END AS groups

FROM project_dept;

The result is

dept_name budget groupsAccounting 10000 Low

Accounting 30000 Low



Research 50000 Medium



Marketing 100000 Medium

Marketing 100000 High



Ch23.indd 652 1/25/12 12:18:28 PM



Pivoting DataPivoting data is a method that is used to transform data from a state of rows to a state of columns. Additionally, some values from the source table can be aggregated before the target table is created.

There are two operators for pivoting data:

PIVOTCc

UNPIVOTCc

The following subsections describe these operators in detail.

PIVOT OperatorPIVOT is a nonstandard relational operator that is supported by Transact-SQL. You can use it to manipulate a table-valued expression into another table. PIVOT transforms such an expression by turning the unique values from one column in the expression into multiple columns in the output, and it performs aggregations on any remaining column values that are desired in the final output.

To demonstrate how the PIVOT operator works, let us use a table called project_dept_pivot, which is derived from the project_dept table specified at the beginning of this chapter. The new table contains the budget column from the source table and two additional columns: month and year. The year column of the project_dept_pivot table contains the years 2007 and 2008, which appear in the date_month column of the project_dept table. Also, the month columns of the project_dept_pivot table (january, february, and july) contain the summaries of budgets corresponding to these months in the project_dept table.

Example 23.27 creates the project_dept_pivot table.

ExAmPLE 23.27

USE sample;

SELECT budget, month(date_month) as month, year(date_month) as year

INTO project_dept_pivot

FROM project_dept;

The content of the new table is given in Table 23-2.Suppose that you get a task to return a row for each year, a column for each month,

and the budget value for each year and month intersection. Table 23-3 shows the desired result.

Ch23.indd 653 1/25/12 12:18:28 PM



Example 23.28 demonstrates how you can solve this problem using the standard SQL language.

ExAmPLE 23.28

USE sample;

SELECT year,

SUM(CASE WHEN month = 1 THEN budget END ) AS January,

SUM(CASE WHEN month = 2 THEN budget END ) AS February,

SUM(CASE WHEN month = 7 THEN budget END ) AS July

FROM project_dept_pivot

GROUP BY year;

budget Month year50000 1 2007

70000 2 2007

65000 7 2007

10000 7 2007

40000 2 2007

30000 1 2007

40000 2 2008

100000 1 2008

180000 2 2008

100000 7 2008

120000 1 2008

Table 23-2 Content of the project_dept_pivot Table

Year January February July2007 80000 110000 75000

2008 220000 220000 100000

Table 23-3 Budgets for each year and month

Ch23.indd 654 1/25/12 12:18:28 PM



The process of pivoting data can be divided into three steps:

Group the dataCc Generate one row in the result set for each distinct “on rows” element. In Example 23.28, the “on rows” element is the year column and it appears in the GROUP BY clause of the SELECT statement.Manipulate the dataCc Spread the values that will be aggregated to the columns of the target table. In Example 23.28, the columns of the target table are all distinct values of the month column. To implement this step, you have to apply a CASE expression for each of the different values of the month column: 1 ( January), 2 (February), and 7 ( July).Aggregate the dataCc Aggregate the data values in each column of the target table. Example 23.28 uses the SUM function for this step.

Example 23.29 solves the same problem as Example 23.28 using the PIVOT operator.

ExAmPLE 23.29

USE sample;

SELECT year, [1] as January, [2] as February, [7] July FROM

(SELECT budget, year, month from project_dept_pivot) p2

PIVOT (SUM(budget) FOR month IN ([1],[2],[7])) AS P;

The SELECT statement in Example 23.29 contains an inner query, which is embedded in the FROM clause of the outer query. The PIVOT clause is part of the inner query. It starts with the specification of the aggregation function: SUM (of budgets). The second part specifies the pivot column (month) and the values from that column to be used as column headings—the first, second, and seventh months of the year. The value for a particular column in a row is calculated using the specified aggregate function over the rows that match the column heading.

The most important advantage of using the PIVOT operator in relation to the standard solution is its simplicity in the case in which the target table has many columns. In this case, the standard solution is verbose because you have to write one CASE expression for each column in the target table.

UNPIVOT OperatorThe UNPIVOT operator performs the reverse operation of PIVOT, by rotating columns into rows. Example 23.30 shows the use of this operator.

Ch23.indd 655 1/25/12 12:18:28 PM



ExAmPLE 23.30

USE sample;

CREATE TABLE project_dept_pvt (year int, January float, February float,

July float);

INSERT INTO project_dept_pvt VALUES (2007, 80000, 110000, 75000);

INSERT INTO project_dept_pvt VALUES (2008, 50000, 80000, 30000);

--UNPIVOT the table

SELECT year, month, budget

FROM

(SELECT year, January, February, July

FROM project_dept_pvt) p

UNPIVOT (budget FOR month IN (January, February, July)

)AS unpvt;

The result is

year month budget2007 January 80000

2007 February 110000

2007 July 75000

2008 January 50000

2008 February 80000

2008 July 30000

Example 23.30 uses the project_dept_pvt table to demonstrate the UNPIVOT relational operator. UNPIVOT’s first input is the column name (budget), which holds the normalized values. After that, the FOR option is used to determine the target column name (month). Finally, as part of the IN option, the selected values of the target column name are specified.

NoteUNPIVOT is not the exact reverse of PIVOT, because any NULL values in the table being transformed cannot be used as column values in the output.

Ch23.indd 656 1/25/12 12:18:28 PM



SummarySQL/OLAP extensions in Transact-SQL support data analysis facilities. There are four main parts of SQL/OLAP that are supported by the Database Engine:

Window constructCc

Extensions of the GROUP BY clauseCc

OLAP query functionsCc

Standard and nonstandard analytic functionsCc

The window construct is the most important extension. In combination with ranking and aggregate functions, it allows you to easily calculate analytic functions, such as cumulative and sliding aggregates, as well as rankings. There are several extensions to the GROUP BY clause that are described in the SQL standard and supported by the Database Engine: the CUBE, ROLLUP, and GROUPING SETS operators as well as the grouping functions GROUPING and GROUPING_ID.

The most important analytic query functions are ranking functions: RANK, DENSE_RANK, and ROW_NUMBER. Transact-SQL supports several nonstandard analytic functions and operators, TOP, NTILE, PIVOT, and UNPIVOT, as well as a standard one, OFFSET/FETCH.

The next chapter describes Reporting Services, a business intelligence component of SQL Server.

Exercises E.23.1

Find the average number of the employees in the Accounting department. Solve this problem:

a. using the window constructb. using the GROUP BY clause

E.23.2

Using the window construct, find the department with the highest budget for the years 2007 and 2008.

Ch23.indd 657 1/25/12 12:18:28 PM



E.23.3

Find the sum of employees according to the combination of values in the departments and budget amounts. All possible summary rows should be displayed, too.

E.23.4

Solve E.23.3 using the ROLLUP operator. What is the difference between this result set and the result set of E.23.3?

E.23.5

Using the RANK function, find the three departments with the highest number of employees.

E.23.6

Solve E.23.5 using the TOP clause.

E.23.7

Calculate the ranking of all departments for the year 2008 according to the number of employees. Display the values for the DENSE_RANK and ROW_NUMBER functions, too.

Ch23.indd 658 1/25/12 12:18:28 PM

business intelligence and...

Documents