heat map and automatic data optimization with oracle database 12c

Heat Map and Automatic Data Optimization with Oracle Database 12c

Jean-Philippe Breysse


Program Agenda

Heat Map

Automatic Data Optimization

Benefits of Heat Map and ADO


Data Compression Reduce storage footprint, read compressed data faster

Hot Data

111010101010101001101010101011010001011011000110100101000001001110001010101101001011010010110001010010011111001001000010001010101101000

10101010111010100110101

11000010100010110111010

10100101001001000010001

01010110100101101001110

00010100100101000010010

00010001010101110011010

Warm Data

101010101110101001101011100001010001011011101010100101001001000010001010101101001011010011100001010010010100001001000010001010101101001

10101010111010100110101110000101000101

10111010101001010010010000100010101011

01001011010011100001010010010100001001

00001000101010111001101110011000111010

Archive Data

101010101110101001101011100001010001011011101010100101001001000010001010101101001011010011100001010010010100001001000010001010101101001

10101010111010100110101110000101000101101110101

01001010010010000100010101011010010110100111000

01010010010100001001000010001010101110011011100

3X Advanced Row Compression

10X

Columnar Query Compression

15X

Columnar Archive Compression


Oracle Advanced Compression Transparent, Smaller, Faster

100% Application Transparent

End-to-end Cost/Performance Benefits across CPU, DRAM, Flash,

Disk & Network

Runs Faster: OLTP Apps (Transactional & Analytics) & DW

Reduces Database Footprint

– CapEx & OpEx savings

– Increases Cloud ROI through Database Footprint reduction in

DRAM Memory


Details: Oracle Advanced Compression + New Features, New Feature Names

Ora

cle

A

dva

nc

ed

Co

mp

re

ssio

n

Oracle Database 11g Oracle Database 12c

OLTP Compression Advanced Row Compression

Secure Files Compression Advanced LOB Compression

Secure Files De-duplication Advanced LOB Deduplication

NEW Heat Map (Object and Row Level)

NEW Automatic Data Optimization

NEW Temporal Optimization

+

Hybrid Columnar Compression Hybrid Columnar Compression


Understanding Data Usage Patterns Database ‘heat map’

1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1

0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0

0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1

0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1

0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1

0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1

0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 1

1 1

1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0

1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1

0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1

0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1

0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0

0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1

0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1

0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0

0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1

0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 0 0 0 0 1 1 1 0 0 1 1

0 1 0 1 0 1 1 0 0 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 1 1 1 1 0 0 1

0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 1 1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 0 0 1 1 1 0 0 1 1 1 1 0 0 0

1 0 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 1 1 1 1 0 0 0


Heat Map What it tracks “Heat Map” tracking

– Access and modification times tracked for tables and

partitions

– Modification times tracked for database blocks

Comprehensive

– Segment level shows both reads and writes

– Distinguishes index lookups from full table scans

– Automatically excludes stats gathering, DDLs, table

redefinitions, etc

High Performance – Object level at no cost

– Block level < 5% cost

Active

Frequent

Access

Occasional

Access

Dormant

HOT

COLD

WARM


Heat Map Enterprise Manager


0101101110101010010100100100001000

1010101101001011010011100001010010

Archive Data

011100001010001011011

101010100101001001000

010001010101101001011

010101001010010010001

Automatic Data Optimization Usage Based Data Compression

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Restricted 12

Hot Data

3X

Advanced Row Compression

Warm Data

1010101011101010011010111000010100

0101101110101010010100100100001000

1010101101001011010011100001010010

0101000010010000100010101011010010

10X

Columnar Query Compression

1000010100100101001010110111000010

101010101110101001101011100001010001011011

101010100101001001000010001010101101001011

010011100001010010010100001001000010001010

101010101110101001101011100001010001011011

15X

Columnar Archive Compression

01110101010010

10000100010101

01011100001010

10101010111010100110101

11000010100010110111010

10100101001001000010001

01010110100101101001110

00010100100101000010010

00010001010101110011010

10100101001001000010001

1110010100100101001010110111011010

101010101110101001101011100001011101011001


Automatic Data Optimization Simple Declarative SQL extension

ALTER TABLE sales ILM add

Active

Frequent

Access

Occasional

Access

Dormant

Advanced Row Compression (2-4x)

Affects ONLY candidate rows

Cached in DRAM & FLASH

row store compress advanced row

after 2 days of no update

Warehouse Compression(10x)

High Performance Storage

column store compress for query low

after 1 week of no update

Warehouse Compression(10x)

Low Cost Storage

tier to SATA Tablespace

Archive Compression(15-50X)

Archival Storage

column store compress for archive high

after 6 months of no access



This Quarter This Year Prior Years

Row Store

for fast OLTP

Compressed Column Store for fast analytics

10x compressed 15x compressed As data cools

down, Automatic

Data Optimization

automatically

converts data to

columnar

compressed

Online

Archive Compressed Column Store for max compression

Reporting Compliance & Reporting OLTP


Up to 15x Smaller Footprint & Faster Queries

Both Columnar & Archive Compression now complement Advanced

Row Compression

Best Practice:

– Step 1: Use Advanced Row Compression for entire DB and then

– Step 2: ADO automatically converts into columnar compressed once

the updates cool down, and is used mainly for reporting

=> Query speed of Columnar & 10x smaller footprint

– Step 3: ADO automatically converts into archive compressed once

data cools down further and is no longer frequently queried

=> 15-50x smaller footprint

Automatic Data Optimization for OLTP


Optimizes Data Based on Heat Map Automatic Data Optimization for DW

Data generally comes in via Bulk Loading

Workload dominated by queries, even during loading

Step 1: Bulk Load directly into Columnar Compressed

– 10x smaller footprint, Query speed of Columnar

Step 2: ADO automatically converts to Archive Compressed

and moves to Lower Cost Storage once its queried infrequently

– Data remains online, with 15-50x smaller footprint, & lower

storage cost


Fast, Flexible Loads & Queries on Columnar Fastest Load with uncompressed & Fastest Queries with columnar

– Mixed workloads often use Java app or 3rd party tools to insert and update data

that does not use Bulk Loads, so cannot use Columnar

Step 1: Load into uncompressed, conventional inserts & updates

– Fast loading, & flexibility of using a regular OLTP app for loading

Step 2: ADO moves to Row Compressed or Columnar

Compressed or Low Cost Storage once updates cool down

– Faster Queries, 3-10x smaller footprint

Automatic Data Optimization – Mixed Use


Backup & Recovery Automatic Data Optimization

Read / Write

Tablespace Read-mostly data is moved to a READONLY Tablespace

10x compressed 15x compressed

READONLY

TBS

As data cools

down, Automatic

Data Optimization

automatically

moves it to a

READONLY TBS,

it’s backed up only

once after that

Reporting Compliance & Reporting OLTP


Summary Heat Map & Automatic Data Optimization

Heat Map

– Automatically tracks access

– Database-aware: maintenance jobs, backups, etc. don’t affect Heat Map


– Declarative easy-to-use syntax to define data compression & movement

policies

– Extensible with business-specific logic

– Automates and simplifies ILM for Oracle Database data

heat map and automatic data optimization with oracle database 12c

Technology

oracle andor

data compression

goracle database

database footprint reduction

row level

storage footprint

new features

dram memory5copyright