real time operational analytics in sql 2016 · types of analytics run the business drive the...

Real Time

Operational

Analytics in

SQL 2016MICHAEL JOHNSON

Types of analytics

Run the business Drive the business

Operational BI Tactical BI Strategic BI

What products should

we be developing for

the future

How many units of

product X did we sell last

year

Do I have enough stock

in the warehouse to fill

an order.

Traditional BI

OLTP

Application

Data warehouseETL

Challenges of traditional BI to

Operational Analytics

Cost

Complexity

Latency

Can we use a column store index

Good

Fast (Up to 100x)

Small (Huge compression for typical DW environments)

Reduced IO

Bad

In 2012 they were read only

In 2014 they were updatable but had to be the clustered index.

DML operation are Slow

Enter the non-clustered updatable

column store indexC1 C2 C3 C5C4

…

Column Store

Delta Store

… Delete bitmap

Enter the non-clustered updatable

column store indexC1 C2 C3 C5C4

Column Store

… Delete bitmap

Tuple mover

Enter the non-clustered updatable

column store indexC1 C2 C3 C5C4

Column Store

REORGANIZE \ REBUILD

Creating the basic index

CREATE TABLE dbo.sales

(

[SaleID] INT PRIMARY KEY,

[SaleDate] DATETIME2 NOT NULL,

[CustomerKey] VARCHAR(20) NOT NULL,

[SaleStatus] VARCHAR(20) NOT NULL,

[SalesAmount] MONEY NOT NULL,

[SomeOtherData] VARCHAR(MAX)

)

CREATE NONCLUSTERED COLUMNSTORE INDEX Sales_NCCI

ON dbo.sales ([SaleDate], [CustomerKey], [SaleStatus],[SalesAmount]);

Non clustered column store index

challenges

We are only interested in the Hot portion of the data

DML operations on column store indexes are slow

How NCCI overcomes these

challenges

NCCI can be filtered

CREATE NONCLUSTERED COLUMNSTORE INDEX Sales_NCCI

ON dbo.sales ([SaleDate], [CustomerKey], [SaleStatus],[SalesAmount])

Where [SaleStatus] = ‘Open’;

The tuple moved can be delayed

CREATE NONCLUSTERED COLUMNSTORE INDEX Sales_NCCI

ON dbo.sales ([SaleDate], [CustomerKey], [SaleStatus],[SalesAmount])

WITH (DATA_COMPRESSION= COLUMNSTORE, COMPRESSION_DELAY = 100)

Where [SaleStatus] = ‘Open’;

References

https://msdn.microsoft.com/en-us/library/dn817827.aspx

https://blogs.technet.microsoft.com/dataplatforminsider/2015/12/0

9/real-time-operational-analytics-using-in-memory-technology/

https://channel9.msdn.com/Events/DataDriven/SQLServer2016/Real

-Time-Operational-analytics