sqlintersection putting the "squeeze" on large tables improve performance and save space...

SQLintersectionPutting the "Squeeze" on Large

TablesImprove Performance and Save Space with

Data CompressionJustin Randall

[email protected]

Tuesday, 2:15-3:30

© SQLintersection. All rights reserved.http://www.SQLintersection.com

Overview

Why compress data in SQL Server? Data compression basics Deciding what to compress Planning your compression efforts Resource requirements, options, and side effects Summary


Why Compress Data?

Save Space on Disk & in Memory

Improve Performance


Enterprise Edition & Azure SQL DB Only!

Reference: http://msdn.microsoft.com/en-us/library/cc645993(v=sql.120).aspx#Scalability

Scalability and Performance

http://msdn.microsoft.com/en-us/library/cc645993(v=sql.120).aspx#Scalability


Data Compression Basics

Compression does not change syntax or semantics of data Application changes are not required Compression is applied to a table, index, or partition

A whole table stored as a heap A whole table stored as a clustered index A whole non-clustered index A whole indexed view Partition-specific compression on partitioned tables


Data Compression Basics

In-row data only Compression is applied at the data page level Compressed pages remain compressed in memory Fill factor is not affected Special cases: Unicode and Columnstore More rules and conditions in Books Online

http://msdn.microsoft.com/en-us/library/cc280449.aspx

http://msdn.microsoft.com/en-us/library/cc280449.aspx


Data Compression Types: Row

A storage optimization technique Reduces metadata overhead associated with the row Stores fixed-length data types in variable-length storage

format Applies to leaf-level & non-leaf-level pages of indexes Removes blank characters NULL and 0 values require no additional space

Effect on Storage by data type

https://msdn.microsoft.com/en-US/library/cc280576%28v=sql.120%29.aspx

https://msdn.microsoft.com/en-US/library/cc280576(v=sql.120).aspx



Data Compression Types: Row

Highest gains: data pages containing rows with a lot of unused space NULL values fixed-length character and numeric values not needing max

defined space Examples

char(100) column with most values < 50 – 75 characters integer values substantially smaller than max size allowed


Data Compression Types: Page

A superset of row compression Seeks to minimize data redundancy Applies equally to all data types Applies to leaf-level pages of tables and indexes only Pages must be full or nearly full before page level compression is

attempted Two techniques

Prefix compression Dictionary compression



Prefix Compression Per column, identify a "common" prefix Row of all prefix values for a column is stored in CI structure of

page Repeated values in column replaced by reference to prefix in CI

Example from SQL Server 2014 Books Online: https://msdn.microsoft.com/en-us/library/cc280464%28v=sql.120%29.aspx

Prefix compression

applied



Dictionary Compression Follows prefix compression Finds repeated values anywhere on the page Repeated values replaced by reference to value in CI

Example from SQL Server 2014 Books Online: https://msdn.microsoft.com/en-us/library/cc280464%28v=sql.120%29.aspx

Dictionary compression

applied

Demo

Examining data compression's effecton data pages and data types


Deciding What to CompressQuestions to Ask

Why?What?

Impact?


Evaluate Potential TargetsCompute Resources, SQL Server Instance, and Databases

SQL Server Edition Database portability Need for space savings

Database Files on Disk Backup Files

Available CPU headroom Maintenance window and available resources for initial compression

and index rebuilds


Evaluate Potential TargetsData Suitability

Is data likely to achieve high compression ratios? Compressible data types Empty space in columns NULLS Repeated values

EXEC sp_estimate_data_compression_savings @Schema_Name = 'dbo', @Object_Name = 'Contact', @Index_ID = 1, @Partition_Number = NULL, @Data_Compression = 'ROW';

Demo

Estimating compression ratios


Evaluate Row vs. Page CompressionWorkload Suitability

Use row compression at a minimum when: Compression results in (significant) space savings, and System can accommodate < 10% increase in CPU utilization

Consider page compression when Low % of update operations vs. total operations High % of scans vs. total operations Total operations = scans + DMLs + lookups


Evaluate Workload

SELECT

ObjectName = object_schema_name(idx.object_id) + '.' + object_name(idx.object_id)

,IndexName = idx.name

,IndexType = CASE WHEN is_unique = 1 THEN 'Unique ' ELSE '' END + idx.type_desc

,User_Scans = us.user_scans

,User_Lookups = us.user_lookups

,User_Updates = us.user_updates

FROM sys.indexes idx

LEFT JOIN sys.dm_db_index_usage_stats us ON idx.object_id = us.object_id

AND idx.index_id = us.index_id

AND us.database_id = db_id()

WHERE OBJECT_NAME (us.[object_id]) = 'customer'

ORDER BY us.user_scans + us.user_updates DESC


Decision Matrix

Table.Index

Savings

ROW %

Savings PAGE

%S U

Decision

Notes

customer. customer_pk

12% 26% 77.7%

22.2%

PAGE Large, High S, Low UBig Savings

customer. ix_customer_nationkey

29% 34% 34.1%

65.9%

ROW Large, Low S, High USimilar Savings

customer. ix_customer_mktsegment_name

16% 66% 23.3%

76.7%

ROW Medium, Low S, High USimilar Savings

customer. ix_customer_name_phone_acctbal

16% 42% 85% 15% PAGE Medium, High S, Low U

Demo

Evaluating workload suitability


Resource Requirements

Compression requires workspace, CPU, and I/O The mechanism is the same as rebuilding an index

Resource requirements depend on What is being compressed (heap, CI, NCI) Sort_In_Tempdb on or off Online option on or off Recovery Model


Resource RequirementsResources Affected During Compression

Workspace Free workspace needed in user db, transaction log, tempdb

I/O Proportional to workspace used

CPU Row compression uses about 1.5 times the CPU used for an

index rebuild Page compression 2 – 5 times the CPU used for an index

rebuild


Compression Operations Options

Online vs. offline Offline is faster, requires less resources, but locks table for

duration Sequential vs. concurrent

Concurrent requires greater resources (workspace, I/O, CPU) Order of operations

Smallest to largest so more free disk space is available for larger objects

Sort_In_Tempdb On/Off On is recommended, as requires less workspace in user

databases


Compression Side Effects

Compression rebuilds a partition or index so fragmentation is removed Compressing a heap rebuilds any nonclustered indexes

When ONLINE is OFF, indexes are rebuilt sequentially When ONLINE is ON, indexes are rebuilt simultaneously Space for an uncompressed heap is not released until index

rebuild is complete Compression frees space in the data file

Managing free space considerations apply as normal


Other Impacts

Workload performance Index rebuilds are slower, require more CPU Bulk inserts are slower Partition manipulation TDE not affected


References

SQL Server Books Onlinehttps://msdn.microsoft.com/en-US/library/cc280449(v=sql.120).aspx

White Paper: Data Compression: Strategy, Capacity Planning and Best Practiceshttp://technet.microsoft.com/en-us/library/dd894051(v=sql.100).aspx

A Look Inside SQL Server Row and Page Compression – Jes Borlandhttp://blogs.lessthandot.com/index.php/datamgmt/dbprogramming/how-sql-server-data-compression/

Applying Compression to the SQL Sentry Database – Melissa Connorshttp://blogs.sqlsentry.com/melissaconnors/sql-sentry-data-compression-1/


http://technet.microsoft.com/en-us/library/dd894051(v=sql.100).aspx

http://blogs.lessthandot.com/index.php/datamgmt/dbprogramming/how-sql-server-data-compression/

http://blogs.lessthandot.com/index.php/datamgmt/dbprogramming/how-sql-server-data-compression/

http://blogs.sqlsentry.com/melissaconnors/sql-sentry-data-compression-1/


References

Data Compression and Backup Compression – Chad Boydhttps://www.mssqltips.com/sqlservertip/2253/sql-server-2008-data-compression-and-backup-compression/

Estimating Data Compression Ratios for All Partitionshttps://www.mssqltips.com/sqlservertip/2219/estimating-data-compression-ratios-for-all/

Estimating Data Compression Savings in SQL Server 2012 – Glenn Berryhttp://www.sqlskills.com/blogs/glenn/estimating-data-compression-savings-in-sql-server-2012/

https://www.mssqltips.com/sqlservertip/2253/sql-server-2008-data-compression-and-backup-compression/

https://www.mssqltips.com/sqlservertip/2253/sql-server-2008-data-compression-and-backup-compression/

https://www.mssqltips.com/sqlservertip/2219/estimating-data-compression-ratios-for-all/

https://www.mssqltips.com/sqlservertip/2219/estimating-data-compression-ratios-for-all/

http://sqlserverperformance.wordpress.com/category/diagnostic-queries/

http://sqlserverperformance.wordpress.com/category/diagnostic-queries/


Review

Why compress data in SQL Server? Data compression basics Deciding what to compress Planning your compression efforts Resource requirements, options, and side effects Summary

Don’t forget to complete an online evaluation on EventBoard!

Your evaluation helps organizers build better conferences and helps speakers improve their sessions.

Questions?

Thank you!

Putting the "Squeeze" on Large TablesImprove Performance and Save Space with Data Compression

sqlintersection putting the "squeeze" on large tables improve performance and save space...

Documents