sql_server_2016_history_tables

28
SQL Server 2016 Temporal Tables Arthur Luz | SQL Server MCSA & MCT [email protected] http://arthurluz.wordpress.com

Upload: arthurjosemberg

Post on 11-Apr-2017

41 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: sql_server_2016_history_tables

SQL Server 2016 Temporal

TablesArthur Luz | SQL Server MCSA & [email protected]://arthurluz.wordpress.com

Page 2: sql_server_2016_history_tables

Who am I?

Data Insights Consultant at Microsoft

Technical Writer at Data’s Light Blog

MCSA e MCT em SQL Server

Official Instructor at Hepta Novintec

Page 3: sql_server_2016_history_tables

Overview – SQL Server 2016 Temporal Tables

New Clauses – Data Definition New Clauses – Data Manipulation Considerations for resource use Main Differences - CDC and Temporal

Tables Usage Scenarios Feature Improvement

Schedule for today

Page 4: sql_server_2016_history_tables

Overview – History Tables

Page 5: sql_server_2016_history_tables

Overview – History Tables

Page 6: sql_server_2016_history_tables

Overview – History Tables

Auditing all data changes and performing data forensics when necessary

Reconstructing state of the data as of any time in the past

Calculating trends over time – Anomaly Detection

Maintaining a slowly changing dimension for decision support applications

Recovering from accidental data changes and application errors

Page 7: sql_server_2016_history_tables

New clauses – Data DefinitionCREATE TABLE Department (

DeptID int NOT NULL PRIMARY KEY CLUSTERED , DeptName varchar(50) NOT NULL , ManagerID INT NULL , ParentDeptID int NULL , SysStartTime datetime2 GENERATED ALWAYS AS ROW START NOT NULL , SysEndTime datetime2 GENERATED ALWAYS AS ROW END NOT NULL , PERIOD FOR SYSTEM_TIME (SysStartTime, SysEndTime)

) WITH ( SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.DepartmentHistory) ) ;

GENERATED ALWAYS AS ROW START – Determina que essa coluna recebá do sistema a data inicial de validade do registro

GENERATED ALWAYS AS ROW END – Determina que essa colina receberá do sistema a data final de validade do registro.

PERIOD FOR SYSTEM_TIME – Realiza a organização histórica do registro de acordo com as datas de inicio e fim de validade da tupla.

SYSTEM_VERSIONING = ON – Habilita a feature para a tabela que está sendo criada ou alterada.

Page 8: sql_server_2016_history_tables

BEGIN TRANSACTION

-- Add required period columns and designationALTER TABLE dbo.Person ADD DateTimeStart DATETIME2(0) GENERATED ALWAYS AS ROW START NOT NULL CONSTRAINT DFT_person_datetimeStart DEFAULT('19000101 00:00:00'), DateTimeEnd DATETIME2(0) GENERATED ALWAYS AS ROW END NOT NULL CONSTRAINT DFT_person_datetimeEnd DEFAULT('99991231 23:59:59'), PERIOD FOR SYSTEM_TIME (DateTimeStart, DateTimeEnd);

-- Remove temporary DEFAULT constraintsALTER TABLE dbo.Person DROP CONSTRAINT DFT_person_datetimeStart, DFT_person_datetimeEnd;

-- Turn system versioning onALTER TABLE dbo.Person SET ( SYSTEM_VERSIONING = ON ( HISTORY_TABLE = dbo.PersonHistory ) );

COMMIT TRANSACTION;

Page 9: sql_server_2016_history_tables

Demonstration 1Creating Temporal Table

in SQL Server 2016

Page 10: sql_server_2016_history_tables

New clauses – Data ManipulationDECLARE @datetime AS DATETIME2 = '2016-11-08 20:00:00'

SELECT *FROM dbo.Person FOR SYSTEM_TIME

AS OF @datetimeORDER BY DateTimeStart ASC,

DateTimeEnd ASC

Clauses SYSTEM_TIME allow to realize temporal queries transparently for user and applications.

Page 11: sql_server_2016_history_tables

FOR SYSTEM_TIME AS OF @datetime

Reference: http://sqlmag.com/sql-server/first-look-system-versioned-temporal-tables-part-2-querying-data-and-optimization-conside (Itzik Ben Gan)

sysEnd > @datetime

sysStart <= @datetime

New clauses – Data Manipulation

Returns a table with a rows containing the values that were actual (current) at the specified point in time in the past.

Page 12: sql_server_2016_history_tables

FOR SYSTEM_TIME FROM @Start TO @End

Reference: http://sqlmag.com/sql-server/first-look-system-versioned-temporal-tables-part-2-querying-data-and-optimization-conside (Itzik Ben Gan)

sysStart < @End

sysEnd > @Start

New clauses – Data Manipulation

Returns a table with the values for all row versions that were active within the specified time range, regardless of whether they started being active before the <start_date_time> parameter value for the FROM argument or ceased being active after the <end_date_time> parameter value for the TO argument.

Page 13: sql_server_2016_history_tables

FOR SYSTEM_TIME BETWEEN @Start AND @End

Reference: http://sqlmag.com/sql-server/first-look-system-versioned-temporal-tables-part-2-querying-data-and-optimization-conside (Itzik Ben Gan)

sysStart <= @End

sysEnd > @Start

New clauses – Data Manipulation

Same as above in the FOR SYSTEM_TIME FROM <start_date_time> TO <end_date_time> description, except the table of rows returned includes rows that became active on the upper boundary defined by the <end_date_time> endpoint.

Page 14: sql_server_2016_history_tables

FOR SYSTEM_TIME CONTAINED IN (@Start,@End)

Reference: http://sqlmag.com/sql-server/first-look-system-versioned-temporal-tables-part-2-querying-data-and-optimization-conside (Itzik Ben Gan)

sysStart >= @Start

sysEnd <= @End

New clauses – Data Manipulation

Returns a table with the values for all row versions that were opened and closed within the specified time range defined by the two datetime values for the CONTAINED IN argument.

Page 15: sql_server_2016_history_tables

Demonstration 2Manipulation historical data in SQL Server Temporal Tables

Page 16: sql_server_2016_history_tables

Considerations for resource use A temporal table must have a primary key defined in order to correlate records between

the current table and the history table, and the history table cannot have a primary key defined.

The SYSTEM_TIME period columns used to record the SysStartTime and SysEndTime values must be defined with a datatype of datetime2.

If the name of a history table is specified during history table creation, you must specify the schema and table name.

By default, the history table is PAGE compressed. If current table is partitioned, the history table is created on default file group because

partitioning configuration is not replicated automatically from the current table to the history table.

Temporal and history tables cannot be FILETABLE and can contain columns of any supported datatype other than FILESTREAM since FILETABLE and FILESTREAM allow data manipulation outside of SQL Server and thus system versioning cannot be guaranteed.

Page 17: sql_server_2016_history_tables

Considerations for resource use While temporal tables support blob data types, such as (n)varchar(max), varbinary(max),

(n)text, and image, their will incur significant storage costs and have performance implications due to their size. As such, when designing your system, care should be taken when using these data types.

History table must be created in the same database as the current table. Temporal querying over Linked Server is not supported.

History table cannot have constraints (primary key, foreign key, table or column constraints).

Indexed views are not supported on top of temporal queries (queries that use FOR SYSTEM_TIME clause)

Online option (WITH (ONLINE = ON) has no effect on ALTER TABLE ALTER COLUMN in case of system-versioned temporal table. ALTER column is not performed as online regardless of which value was specified for ONLINE option.

INSERT and UPDATE statements cannot reference the SYSTEM_TIME period columns. Attempts to insert values directly into these columns will be blocked.

Page 18: sql_server_2016_history_tables

Considerations for resource use TRUNCATE TABLE is not supported while SYSTEM_VERSIONING is ON Direct modification of the data in a history table is not permitted. ON DELETE CASCADE and ON UPDATE CASCADE are not permitted on the current table.

In other words, when temporal table is referencing table in the foreign key relationship (corresponding to parent_object_id in sys.foreign_keys) CASCADE options are not allowed. To work around this limitation, use application logic or after triggers to maintain consistency on delete in primary key table (corresponding to referenced_object_id in sys.foreign_keys). If primary key table is temporal and referencing table is non-temporal, there’s no such limitation.

INSTEAD OF triggers are not permitted on either the current or the history table and AFTER triggers are permitted only on the current table.

Regular queries only affect data in the current table. To query data in the history table, you must use temporal queries.

Page 19: sql_server_2016_history_tables

Considerations for resource use

An optimal indexing strategy will include a clustered columns store index and / or a B-tree rowstore index on the current table and a clustered columnstore index on the history table for optimal storage size and performance. If you create / use your own history table, we strongly recommend that you create this type of index consisting of period columns starting with the end of period column to speed up temporal querying as well as speeding up the queries that are part of the data consistency check. The default history table has a clustered rowstore index created for you based on the period columns (end, start). At a minimum, a non-clustered rowstore index is recommended.

Page 20: sql_server_2016_history_tables

Table 1

Main Differences – CDC x Temporal Tables

Log

_CDC 1

Uses all table columns.

Possible to select specific columns.

Works synchronously.

Works asynchronously.

Allows change in table when feature is active.

It does not allow change in the table when the feature is active.

Used to maintain history in the transactional system.

Used only for momentary capture of history.

Simple queries that are transparent to users and applications.

Complex queries that require in-depth knowledge.

Version INSERTEs, UPDATEs and DELETEs.

Version only UPDATEs and DELETEs.

Page 21: sql_server_2016_history_tables

Demonstration 3CDC x History Tables

Main Differences

Page 22: sql_server_2016_history_tables

Usage Scenarios

Data Audit

Point in Time Analysis

(Time Travel)

Calculating trends

over time(Anomaly Detection)

Slowly-Changing

Dimensions

Repairing Row-Level

Data Corruption

Page 23: sql_server_2016_history_tables

Demonstration 4Getting rollback of committed transactions

Page 24: sql_server_2016_history_tables

Columnstore Index

Table Partitioning

Stretch Database

In-Memory Tables

Cleanup

Feature Improvement and Retention

Page 25: sql_server_2016_history_tables

Feature Improvement – In-Memory Only durable memory-optimized tables can be system-versioned

(DURABILITY = SCHEMA_AND_DATA).

Queries that affect only the current table (in-memory) can be used in natively compiled T-SQL modules. Temporal queries using the FOR SYSTEM TIME clause are not supported in natively compiled modules. Use of the FOR SYSTEM TIME clause with memory-optimized tables in ad hoc queries and non-native modules is supported.

When SYSTEM_VERSIONING = ON, an internal memory-optimized staging table is automatically created to accept the most recent system-versioned changes that are results of update and delete operations on memory-optimized current table.

Data from the internal memory-optimized staging table is regularly moved to the disk-based history table by the asynchronous data flush task. This data flush mechanism has a goal to keep the internal memory buffers at less than 10% of the memory consumption of their parent objects.

Hybrid Transactional/Analytical Processing (HTAP)

Page 26: sql_server_2016_history_tables

Demonstration 5Creating Temporal Table

in SQL Server 2016 In-Memory Table

Page 27: sql_server_2016_history_tables

Thank you so much!!

Questions???

Page 28: sql_server_2016_history_tables

Email - [email protected]

Linkedin – Arthur Luz

Twitter - @arthurjosemberg

Skype - arthurjosemberg

Blog – arthurluz.wordpress.com

Contacts