oracle 12cr2 database in-memory: adventures with ... · 12-10-2017 · dynamically resizable...

Oracle 12cR2Database In-Memory:

Adventures withSwingbench TPC-DS

Jim Czuprynski

ViON Corporation

October 12, 2017

April 2-6, 2017 in Las Vegas, NV USA #C17LV

My Credentials

• 35+ years of database-centric IT experience

• Oracle DBA since 2001

• Oracle 9i, 10g, 11g, 12c OCP

• Oracle ACE Director

• 100+ articles on databasejournal.com and IOUG SELECT

• Co-author of 2+ Oracle books (new book: Fall 2017)

• Oracle-centric blog (Generally, It Depends)

• Regular speaker at Oracle OpenWorld, IOUG COLLABORATE, Hotsos Symposium, and Regional OUGs


My Latest Books

Use our extensive examples of precisely how to leverage the

myriad tools already built intoOracle Database 12cR2 for

better performance

Learn the intricacies of Oracle Database 12cR2 multitenancy

and how it’s really designed to get you to the Cloud right now*

*Beer not included. Always code responsibly.


Our Agenda

• DBIM: 12cR1 Refresher

• Key 12cR2 DBIM Features• Dynamic IMCS Resizing

• In-Memory Expressions and the Expressions Statistics Store (ESS)

• In-Memory Filtering Improvements

• In-Memory Joins and Join Groups

• Heat Maps, ILM ADO Policies, and Intelligent Evictions

• Other New 12cR2 DBIM Features• In-Memory Fast Start

• IMCS in Active Data Guard (ADG)

• IMCS Improvements for RAC

• Conclusions


Testing Environment: TPC-DS Schema

24 tables consisting of 17 dimensions and 7 fact tables:• INVENTORY• CATALOG_SALES• CATALOG_RETURNS• STORE_SALES• STORE_RETURNS• WEB_SALES• WEB_RETURNS

STORE_SALES Entity-Relationship DiagramSwingbench DataLoadWizard leveraged for:• Schema creation• Initial data loading• Index and constraint

creation• Statistics gathering


Testing Environment: Introducing Swingbench 2.6.1

For the first time ever, Swingbench includes TPC-DS functionality!• TPC-DS schema

generation and initial data load

• TPC-DS query workload generator

• TPC-DS DML workload generator

A plethora of new enhancements:• Improved SQL query

editor for building workloads

• Secure Oracle Cloud connectivity

• Improved charting and statistics collection

DBIM: A Refresher• How It Worked in 12cR1

• Its New Features in 12cR2


IMCS: Turning Sideways For Better Performance

ord# part# suppl# line# qty extdamt rtn sts shipdt comtdt rcptdt

12389 103 19 1 19 190.00 N A 2014-01-01 2014-01-02 2014-01-05

12389 987 22 2 48 960.00 N A 2014-01-01 2014-01-02 2014-01-04

12389 623 23 3 10 200.00 N A 2014-01-01 2014-01-02 2014-01-05

12389 103 19 4 5 100.00 N A 2014-01-02 2014-01-04 2014-01-05

12389 103 19 5 17 51.00 N A 2014-01-02 2014-01-04 2014-01-05

12389 623 23 6 5 190.00 Y I 2014-01-02 2014-01-03 2014-01-05

12389 623 23 7 1 190.00 N A 2014-01-05 2014-01-05 2014-01-14

12389 109 22 8 34 68.50 Y P 2014-01-05 2014-01-05 2014-01-08

ord# 12389 12389 12389 12389 12389 12389 12389 12389

part# 103 987 623 103 103 623 623 109

suppl# 19 22 23 19 19 23 23 22

line# 1 2 3 4 5 6 7 8

qty 19 48 10 5 17 5 1 34

extdamt 190.00 960.00 200.00 100.00 51.00 190.00 190.00 68.50

rtn N N N N N Y N Y

sts A A A A A I A P

shipdt 2014-01-01 2014-01-01 2014-01-01 2014-01-02 2014-01-02 2014-01-02 2014-01-05 2014-01-05

comtdt 2014-01-02 2014-01-02 2014-01-02 2014-01-04 2014-01-04 2014-01-03 2014-01-05 2014-01-05

Rcptdt 2014-01-05 2014-01-04 2014-01-05 2014-01-05 2014-01-05 2014-01-05 2014-01-14 2014-01-08

This columnar representation results in far fewer I/Os than with row major format. Reading only needed columns and ignoring the rest is called columnar projection.

Row-major storage works great for single-row access, especially DML …

… but can seriously reduce query performance when just a few column

values must be accessed

By simply turning the table structure table sideways to the left … now the same query

only has to scan two columns.

SELECT COUNT(*)

FROM tpch.h_lineitem

WHERE rtn <> ‘N’

AND sts <> ‘A’;

INSERT INTO tpch.h_lineitem

VALUES (12389, 109, 22 ...);

Pre-12.1.0.2: Row-major storage (ORGANIZATION HEAP)

UPDATE tpch.h_lineitem

SET rtn = ‘Y’

WHERE ord# = 12389

AND ...);

12.1.0.2: Columnar Storage


IMCS: A Simple Example of How It Works

+WARM

ADO_WARM_DATA

ADO_WARM_IDX

SGA

IMCO SMCOWnnn

User

Physical Storage

Transaction Journal

In-Memory Column Store

User issues

query against

IMCS

candidate

table

1

If there’s enough

space in IMCS, rows

are populated in

columnar format and

compressed

2

2

4

The next query retrieves

directly from IMCS …

3

… else, unpopulated data

is retrieved from disk

4

3

1


What’s New in 12cR2: Covered In Depth

• In-Memory Area Dynamic Resizing

• In-Memory Expressions• SMUs and IMEUs

• In-Memory Virtual Columns

• Improved In-Memory Joins and Filtering • HASH JOINs vs. Bloom Filters

• Join Groups

• Intelligent IMCS Population (via ILM ADO policies)

10


What’s New in 12cR2: Covered Briefly

• In-Memory Fast Start• A blazingly-fast method to repopulate IMCS on instance restart

• Active Data Guard (ADG) support• Now possible to populate different database objects in primary database vs. ADG

database

• Improved support for Real Application Clusters (RAC)• Improved scaling across multiple nodes in a cluster

• IMCS storage within Exadata Smart Flash Cache• Now possible to retain data in columnar format within Smart Flash Cache, saving

repopulation cycles

11

DBIM: Growing Dynamically


Dynamically Resizable In-Memory Area

• In 12.1.0.2, the In-Memory Area was in essence a static cache• Resizing the cache required bouncing the database instance

• In 12.2.0.1, the In-Memory Area is no longer static

• Some restrictions apply:• New size must be at least 128MB greater than its current size

• There must be free memory within the instance’s SGA to accommodate the new size

• It can never be made smaller dynamically

13


Dynamic IMCS Resizing: Examples

SQL> CONN / as sysdba

Connected.

SQL> ALTER SYSTEM SET INMEMORY_SIZE = 1024M SCOPE=SPFILE;

System altered.

SQL> show sga

Total System Global Area 2.5770E+10 bytes

Fixed Size 8802296 bytes

Variable Size 1.2012E+10 bytes

Database Buffers 1.3690E+10 bytes

Redo Buffers 58306560 bytes

SQL> show parameter inmemory;

NAME TYPE VALUE

-------------------------------------------- ----------- ----------

inmemory_clause_default string

inmemory_expressions_usage string ENABLE

inmemory_force string DEFAULT

inmemory_max_populate_servers integer 0

inmemory_query string ENABLE

inmemory_size big integer 0

inmemory_trickle_repopulate_servers_percent integer 1

inmemory_virtual_columns string MANUAL

optimizer_inmemory_aware boolean TRUE

Set In-Memory Area size to 1GB:1Bounce database instance to allocate memory to In-Memory Area:

2

SQL> shutdown immediate;

Database closed.

Database dismounted.

ORACLE instance shut down.

SQL> startup;

ORACLE instance started.




Database Buffers 1.0536E+10 bytes


In-Memory Area 1073741824 bytes

Database mounted.

Database opened.

Dynamic In-Memory Area resizing attempt fails if:(a) it’s made smaller than its current size; or (b) it doesn’t grow at least 128MB larger!

3

SQL> ALTER SYSTEM SET INMEMORY_SIZE = 0;

ALTER SYSTEM SET INMEMORY_SIZE = 0

*

ERROR at line 1:

ORA-02097: parameter cannot be modified because specified value is invalid

ORA-02095: specified initialization parameter cannot be modified

SQL> ALTER SYSTEM SET INMEMORY_SIZE = 1100M;

ALTER SYSTEM SET INMEMORY_SIZE = 1100M

*

ERROR at line 1:

ORA-02097: parameter cannot be modified because specified value is invalid

ORA-02095: specified initialization parameter cannot be modified

Otherwise, dynamic resizing of the In-Memory works as long as there’s sufficient room in the SGA!

4

SQL> ALTER SYSTEM SET INMEMORY_SIZE = 2048M;

System altered.

SQL> show sga




Database Buffers 9797894144 bytes


In-Memory Area 2147483648 bytes

DBIM: In-Memory Expressions


IMCS in 12cR2: Better. Stronger. Faster.

In-Mmeory Area

IMEUs (1MB)

IMCUs (1MB)

SMUs (64KB)In

-Me

mo

ry A

rea

D

A

T

AIn-Memory

Expression Units contain

Virtual Columns and the Expressions Statistics Store

In-Memory Compression Units contain data from tables and

partitions projected in columnar format

Snapshot Metadata Units store metadata for IMCUs

and IMEUs


In-Memory Expressions (IME)

IMEs are pre-computed, frequently evaluated expressions

• Created for virtual columns and automatic capture • Frequently evaluated expressions

• Expressions needed for filtering, joins, etc.

• Virtual columns (defined within a table)

• Auto-detected ”hot” (i.e. dynamic) expressions

• JSON functions also supported• Automatically retained in Expression Statistics Store (ESS)

• Since they need no disk storage, IMEs may replace the need for Materialized Views (MVs)

• No need to refresh MV table when underlying base tables change


In-Memory Expressions: Advantages

Instead of calculating an

expression over and over again …

… the expression itselfcan be retained within a corresponding IMEU …

… and the 12.2 optimizer knows to retrieve it from the IMEU instead, thus

offering potential reduction in intense CPU

cycles


Controlling In-Memory Expressions

19

INMEMORY_VIRTUAL_COLUMNS*

Value Description

MANUAL (Default) No virtual columns will be stored in-memory unless they are explicitly marked for INMEMORY, or they have been specified with a different compression level than their base object’s compression level

ENABLE All virtual columns will be stored in-memory unless they are explicitly marked for NO INMEMORY, or they have been specified with a different compression level than their base object’s compression level

DISABLE No virtual columns will ever be stored in-memory

INMEMORY_EXPRESSIONS_USAGE*

Value Description

ENABLE (Default) Both static and dynamic IM expressions are populated into IMCS

STATIC_ONLY Only IM expressions containing JSON and Oracle NUMBERs will be populated into IMCS

DYNAMIC_ONLY IM expressions are automatically created and populated within the IMCS, if used in conjunction with procedure DBMS_INMEMORY.IME_CAPTURE_EXPRESSIONS

DISABLE No IM expressions at all will be populated into IMCS

* ALTER SYSTEM scope only


Controlling IME Population: DBMS_INMEMORY

DBMS_INMEMORY controls IME capture, population and removal at a

granular (table, partition, or sub-partition) level:

20

DBMS_INMEMORY Procedures and Functions

Procedure / Function Description

POPULATE Populates a selected table, partition, or sub-partition within the IMCS

REPOPULATE Forces repopulation of the specified table, partition, or sub-partition within the IMCS

IME_DROP_EXPRESSIONS Drops one or more virtual columns from a table

SEGMENT_DEALLOCATE_VERSIONS Deallocates any non-current IMCUs for the specified table, partition, or sub-partition within the IMCS (which automatically happens every 2 minutes)


Controlling IME Population: DBMS_INMEMORY_ADMIN

DBMS_INMEMORY_ADMIN controls IME capture, population and removal at

the database level:

21

DBMS_INMEMORY_ADMIN Procedures and Functions

Procedure / Function Description

IME_CAPTURE_EXPRESSIONS Captures and populates the top 20 most frequently accessed expressions for a specified time frame

IME_POPULATE_EXPRESSIONS Forces immediate population of the most recently gathered set of expressions captured via last invocation of IME_CAPTURE_EXPRESSIONS

IME_DROP_ALL_EXPRESSIONS Drops all current SYS_IME expression virtual columns from the database


Demonstrating IME Value

Prepare for In-Memory Expressions1

-- Deactivate usage of IMEs and clear any existing ones:

SQL> ALTER SYSTEM SET inmemory_virtual_columns = DISABLE;

ALTER SYSTEM SET inmemory_expressions_usage = DISABLE;

BEGIN

DBMS_INMEMORY_ADMIN.IME_DROP_ALL_EXPRESSIONS();

END;

/

-- Reactivate usage of IMEs:

SQL> ALTER SYSTEM SET inmemory_virtual_columns = ENABLE;

ALTER SYSTEM SET inmemory_expressions_usage = ENABLE;

Add virtual columns to STORE_SALES containing expressions2

SQL> ALTER TABLE tpcds.store_sales

ADD (ss_gross_revenue NUMBER(12,2)

VISIBLE

GENERATED ALWAYS

AS ((ss_sales_price - ss_wholesale_cost) * ss_ext_discount_amt)

VIRTUAL);


ADD (ss_net_revenue NUMBER(12,2)

VISIBLE

GENERATED ALWAYS

AS ((ss_ext_sales_price - ss_ext_wholesale_cost) * ss_ext_discount_amt)

VIRTUAL);


ADD (ss_extd_gpm NUMBER(12,2)

VISIBLE

GENERATED ALWAYS

AS ((ss_ext_sales_price - ss_ext_list_price) * ss_coupon_amt)

VIRTUAL);

Generate a query workload3

SQL> SELECT

d_year

,d_qoy

,hd_income_band_sk

,cd_gender

,cd_marital_status

,SUM(ss_gross_revenue)

,SUM(ss_net_revenue)

,SUM(ss_extd_gpm)

FROM

tpcds.store_sales SS

,tpcds.customer_demographics CD

,tpcds.household_demographics HD

,tpcds.date_dim DD

WHERE SS.ss_cdemo_sk = CD.cd_demo_sk

AND SS.ss_hdemo_sk = HD.hd_demo_sk

AND SS.ss_sold_date_sk = DD.d_date_sk

GROUP BY

d_year

,d_qoy

,hd_income_band_sk

,cd_gender

,cd_marital_status

ORDER BY

d_year

,d_qoy

,hd_income_band_sk

,cd_gender

,cd_marital_status;

Capture In-Memory Expressions4

SQL> BEGIN

-- Capture all IMEs since the database was created

DBMS_INMEMORY_ADMIN.IME_CAPTURE_EXPRESSIONS(

snapshot => 'CUMULATIVE');

-- Force population of all captured IMEs

DBMS_INMEMORY_ADMIN.IME_POPULATE_EXPRESSIONS();

END;

/

Show captured and populated In-Memory Expressions5

SQL>

SET LINESIZE 150

COL owner FORMAT A12 HEADING "Owner"

COL table_name FORMAT A20 HEADING "Table|Name"

COL column_name FORMAT A24 HEADING "Column|Name"

COL sql_expression HEADING "In-Memory Expression"

TTITLE "Current In-Memory Expressions|(from DBA_IM_EXPRESSIONS)"

SELECT

table_name

,column_name

,sql_expression

FROM dba_im_expressions

ORDER BY 1

;

TTITLE OFF

Current In-Memory Expressions

(from DBA_IM_EXPRESSIONS)

Table Column Name +

Name SQL Expression

-------------------- --------------------------------------------------------------------------------

CATALOG_SALES SYS_IME00010000000CBE23

("CS_EXT_LIST_PRICE"-"CS_EXT_WHOLESALE_COST"-"CS_EXT_DISCOUNT_AMT"+"CS_EXT_SALES

ITEM SYS_IME00010000000CBE46

SUBSTR("I_ITEM_DESC",1,30)

STORE_SALES SYS_IME00010000000CBE34

COALESCE("SS_SALES_PRICE"*"SS_QUANTITY",0)


"SS_EXT_LIST_PRICE"-"SS_EXT_DISCOUNT_AMT"


("SS_EXT_LIST_PRICE"-"SS_EXT_WHOLESALE_COST"-"SS_EXT_DISCOUNT_AMT"+"SS_EXT_SALES


"SS_QUANTITY"*"SS_SALES_PRICE"

WEB_SALES SYS_IME00010000000CBE2C

("WS_EXT_LIST_PRICE"-"WS_EXT_WHOLESALE_COST"-"WS_EXT_DISCOUNT_AMT"+"WS_EXT_SALES

WEB_SALES SYS_IME00010000000CBE29

"WS_EXT_LIST_PRICE"-"WS_EXT_DISCOUNT_AMT"


Demonstrating IME Value No IME Enabled:1

Elapsed: 00:00:07.31

Plan hash value: 4044706610

----------------------------------------------------------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |

----------------------------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 1910 | 158K| 6756 (2)| 00:00:01 |

| 1 | PX COORDINATOR | | | | | |

| 2 | PX SEND QC (ORDER) | :TQ10002 | 1910 | 158K| 6756 (2)| 00:00:01 |

|* 3 | FILTER | | | | | |

| 4 | SORT GROUP BY | | 1910 | 158K| 6756 (2)| 00:00:01 |

| 5 | PX RECEIVE | | 1910 | 158K| 6756 (2)| 00:00:01 |

| 6 | PX SEND RANGE | :TQ10001 | 1910 | 158K| 6756 (2)| 00:00:01 |

| 7 | HASH GROUP BY | | 1910 | 158K| 6756 (2)| 00:00:01 |

|* 8 | HASH JOIN | | 5773K| 467M| 6712 (1)| 00:00:01 |

| 9 | PX RECEIVE | | 1920K| 18M| 51 (10)| 00:00:01 |

| 10 | PX SEND BROADCAST | :TQ10000 | 1920K| 18M| 51 (10)| 00:00:01 |

| 11 | PX BLOCK ITERATOR | | 1920K| 18M| 51 (10)| 00:00:01 |

| 12 | TABLE ACCESS INMEMORY FULL| CUSTOMER_DEMOGRAPHICS | 1920K| 18M| 51 (10)| 00:00:01 |

|* 13 | HASH JOIN | | 5760K| 412M| 6654 (1)| 00:00:01 |

| 14 | TABLE ACCESS INMEMORY FULL | DATE_DIM | 73049 | 927K| 5 (20)| 00:00:01 |

|* 15 | HASH JOIN | | 5760K| 340M| 6646 (1)| 00:00:01 |

| 16 | TABLE ACCESS INMEMORY FULL | HOUSEHOLD_DEMOGRAPHICS | 7200 | 50400 | 2 (0)| 00:00:01 |

| 17 | PX BLOCK ITERATOR | | 5760K| 302M| 6639 (1)| 00:00:01 |

| 18 | TABLE ACCESS FULL | STORE_SALES | 5760K| 302M| 6639 (1)| 00:00:01 |

----------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):

---------------------------------------------------

3 - filter(MAX()-MIN()<=SUM()*1.2)

8 - access("SS"."SS_CDEMO_SK"="CD"."CD_DEMO_SK")

13 - access("SS"."SS_SOLD_DATE_SK"="DD"."D_DATE_SK")

15 - access("SS"."SS_HDEMO_SK"="HD"."HD_DEMO_SK")

Note

-----

- Degree of Parallelism is 4 because of session

In-Memory Session-Level Statistics

(from V$MYSTAT)

Statistic

Statistic Name Value

----------------------------------------- ----------

IM scan CUs columns accessed 24

IM scan CUs columns theoretical max 168

IM scan CUs current 12

IM scan CUs memcompress for query low 12

IM scan CUs no cleanout 12

IM scan EU bytes in-memory 0

IM scan EU bytes uncompressed 0

IM scan EU rows 0

IM scan EUs columns accessed 0

IM scan EUs columns decompressed 0

IM scan EUs columns theoretical max 0

IM scan EUs memcompress for capacity high 0

IM scan EUs memcompress for capacity low 0

IM scan EUs memcompress for dml 0

IM scan EUs memcompress for query high 0

IM scan EUs memcompress for query low 0

IM scan EUs no memcompress 0

IM scan EUs split pieces 0

After IME Activation:2

Elapsed: 00:00:02.90

Execution Plan

----------------------------------------------------------


-----------------------------------------------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)|

-----------------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 1910 | 158K| 426 (30)|

| 1 | PX COORDINATOR | | | | |

| 2 | PX SEND QC (ORDER) | :TQ10002 | 1910 | 158K| 426 (30)|

|* 3 | FILTER | | | | |

| 4 | SORT GROUP BY | | 1910 | 158K| 426 (30)|

| 5 | PX RECEIVE | | 1910 | 158K| 426 (30)|

| 6 | PX SEND RANGE | :TQ10001 | 1910 | 158K| 426 (30)|

| 7 | HASH GROUP BY | | 1910 | 158K| 426 (30)|

|* 8 | HASH JOIN | | 5773K| 467M| 382 (22)|

| 9 | PX RECEIVE | | 1920K| 18M| 51 (10)|

| 10 | PX SEND BROADCAST | :TQ10000 | 1920K| 18M| 51 (10)|

| 11 | PX BLOCK ITERATOR | | 1920K| 18M| 51 (10)|

| 12 | TABLE ACCESS INMEMORY FULL| CUSTOMER_DEMOGRAPHICS | 1920K| 18M| 51 (10)|

|* 13 | HASH JOIN | | 5760K| 412M| 324 (22)|

| 14 | TABLE ACCESS INMEMORY FULL | DATE_DIM | 73049 | 927K| 5 (20)|

|* 15 | HASH JOIN | | 5760K| 340M| 315 (22)|

| 16 | TABLE ACCESS INMEMORY FULL | HOUSEHOLD_DEMOGRAPHICS | 7200 | 50400 | 2 (0)|

| 17 | PX BLOCK ITERATOR | | 5760K| 302M| 309 (21)|

| 18 | TABLE ACCESS INMEMORY FULL| STORE_SALES | 5760K| 302M| 309 (21)|

-----------------------------------------------------------------------------------------------------


---------------------------------------------------

3 - filter(MAX()-MIN()<=SUM()*1.2)

8 - access("SS"."SS_CDEMO_SK"="CD"."CD_DEMO_SK")

13 - access("SS"."SS_SOLD_DATE_SK"="DD"."D_DATE_SK")

15 - access("SS"."SS_HDEMO_SK"="HD"."HD_DEMO_SK")

Note

-----

- Degree of Parallelism is 4 because of session

In-Memory Session-Level Statistics

(from V$MYSTAT)

Statistic

Statistic Name Value

----------------------------------------------- ------------

IM scan CUs columns accessed 72

IM scan CUs columns theoretical max 720

IM scan CUs current 36

IM scan CUs memcompress for query low 36

IM scan CUs no cleanout 36

IM scan CUs readlist creation accumulated time 136

IM scan CUs readlist creation number 36

IM scan CUs split pieces 67

IM scan EU bytes in-memory 262,752,350

IM scan EU bytes uncompressed 385,750,275

IM scan EU rows 8,819,545

IM scan EUs columns accessed 24

IM scan EUs columns theoretical max 744

IM scan EUs memcompress for query low 24

IM scan EUs split pieces 7


Expression Statistics Store (ESS)

The Expression Statistics Store (ESS):• Is maintained by the optimizer

• Identifies expression during hard parse into a canonical representation

• Keeps expression evaluation statistics that are periodically populated back to storage

• ESS metadata is stored within IMEUs

• About once every hour, any automatically captured expressions will be re-evaluated for recent usage

• View DBA_EXPRESSION_STATISTICS captures statistics

24


DBA_EXPRESSION_STATISTICS

Remember, only the “top” 50 expressions are retained within the ESS for each table - regardless of whether the IME element is enabled or disabled for INMEMORY storage!

COL table_name FORMAT A20 HEADING "Table Name"

COL expression_id HEADING "Expression ID"

COL snapshot FORMAT A10 HEADING "Snapshot"

COL evaluation_count FORMAT 99999999999 HEADING "Eval|Count"

COL expression_text FORMAT A40 HEADING "Expression Text" WRAP

TTITLE "In-Memory Expression Statistics|(From DBA_EXPRESSION_STATISTICS)"

SELECT

table_name

,expression_id

,snapshot

,evaluation_count

,expression_text

FROM dba_expression_statistics

WHERE owner = 'TPCDS'

AND snapshot = 'LATEST'

AND table_name LIKE '%_SALES'

ORDER BY table_name;

TTITLE OFF

In-Memory Expression Statistics

(From DBA_EXPRESSION_STATISTICS)

Eval

Table Name Expression ID Snapshot Count Expression Text

-------------------- ------------- ---------- ------------ -----------------------------------------------

. . .

WEB_SALES 1.8063E+19 LATEST 99976995 ("WS_EXT_LIST_PRICE"-"WS_EXT_WHOLESALE_C

OST"-"WS_EXT_DISCOUNT_AMT"+"WS_EXT_SALES

_PRICE")/2

. . .

WEB_SALES 1.9968E+18 LATEST 94600 SYS_OP_BLOOM_FILTER(:BF0000,"WS_SHIP_DATE_SK")

WEB_SALES 1.5474E+19 LATEST 64 "WS_SHIP_DATE_SK"-"WS_SOLD_DATE_SK"

WEB_SALES 6.4902E+18 LATEST 105765202 "WS_EXT_LIST_PRICE"-"WS_EXT_DISCOUNT_AMT"

WEB_SALES 4.4269E+18 LATEST 34164285 SYS_OP_BLOOM_FILTER(:BF0002,"WS_SOLD_DATE_SK")

. . .

WEB_SALES 7.4796E+18 LATEST 222144 "WS_QUANTITY"*"WS_LIST_PRICE"

. . .

DBIM: Filtering Improvements


DBIM: Improved Filtering Operations

• Explicit and implicit predicates can be pushed to In-Memory Scans

• IMCU storage indexes leveraged for faster elimination• Requires that MEMCOMPRESS is set to any other level than NONE (so that IMCU

dictionary entries are created)

• Query optimizer is now fully aware of advantages of using In-Memory Filtering

• Complex joins can often be transformed into scan and filter operations

• A plethora of new IM session-level statistics help explain what’s happening during operations


In-Memory Filtering: Making Joins Faster

IMCS excels at scanning and filtering data … and that’s good news because many joins can be converted into a scan + filter operation

• Bloom Filters (BFs) can now convert certain joins into scan and filteroperations

• BFs can also leverage In-Memory storage indexes and MIN/MAX pruning

• BFs are extremely effective for multi-set joins between multiple dimensions to a fact table

• Some LEFT DEEP TREE joins can be transformed to RIGHT DEEP TREE joins via SWAP JOIN INPUT optimization


Left Deep Tree vs. Right Deep Tree Joins

LEFT DEEP TREE Hash JoinsHASH JOIN

HASH JOIN

HASH JOIN

ITEMS STORE_SALES

HOUSEHOLD_DEMOGRAPHICS

DATE_DIM

A Bloom Filter is created for ITEMS :

STORE SALES …

… and it’s then applied against STORE_SALES …

… but since this is a LEFT DEEP TREE join, BFs can’t be

used for DATE_DIM and HOUSEHOLD_DEMOGRAPHICS

tables

RIGHT DEEP TREE Hash Joins

HASH JOIN

DATE_DIM HASH_JOIN

HOUSEHOLD_DEMOGRAPHICS

HASH_JOIN

ITEMS STORE_SALES

But for a RIGHT DEEP TREE join, a Bloom

Filter is created for allthree dimensions …

… and the BFs can be used at all

levels, essentially applied as a single

filter just once


Advantages of LEFT-TREE vs. RIGHT-TREE Conversion

A simple, four-table join that can be transformed via SWAP_JOIN_INPUTS:

SQL> EXPLAIN PLAN FOR

SELECT /*+ SJI */

d_year

,d_moy

,hd_dep_count

,i_category

,SUM(cs_ship_date_sk - cs_sold_date_sk) AS "Elapsed Days"

FROM

catalog_sales

,date_dim

,item

,household_demographics

WHERE cs_ship_date_sk = d_date_sk

AND cs_item_sk = i_item_sk

AND cs_bill_hdemo_sk = hd_demo_sk

AND d_date BETWEEN TO_DATE('1999-11-01','yyyy-mm-dd')

AND TO_DATE('2000-02-29','yyyy-mm-dd')

AND i_category IN ('Books','Electronics')

AND hd_dep_count BETWEEN 0 AND 2

GROUP BY

d_year

,d_moy

,i_category

,hd_dep_count

ORDER BY

d_year

,d_moy

,hd_dep_count

,i_category;


------------------------------------------------------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |

------------------------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 3106 | 303K| 803 (13)| 00:00:01 |

| 1 | SORT GROUP BY | | 3106 | 303K| 803 (13)| 00:00:01 |

|* 2 | HASH JOIN | | 14120 | 1378K| 801 (13)| 00:00:01 |

|* 3 | TABLE ACCESS INMEMORY FULL | HOUSEHOLD_DEMOGRAPHICS | 2600 | 18200 | 2 (0)| 00:00:01 |

|* 4 | HASH JOIN | | 37120 | 3371K| 799 (13)| 00:00:01 |

| 5 | JOIN FILTER CREATE | :BF0000 | 3960 | 216K| 14 (8)| 00:00:01 |

|* 6 | TABLE ACCESS INMEMORY FULL | ITEM | 3960 | 216K| 14 (8)| 00:00:01 |

|* 7 | HASH JOIN | | 181K| 6574K| 785 (13)| 00:00:01 |

| 8 | JOIN FILTER CREATE | :BF0001 | 122 | 1952 | 17 (6)| 00:00:01 |

| 9 | VIEW | VW_GBF_17 | 122 | 1952 | 17 (6)| 00:00:01 |

|* 10 | TABLE ACCESS INMEMORY FULL | DATE_DIM | 122 | 2562 | 17 (6)| 00:00:01 |

| 11 | JOIN FILTER USE | :BF0000 | 2883K| 57M| 760 (12)| 00:00:01 |

| 12 | JOIN FILTER USE | :BF0001 | 2883K| 57M| 760 (12)| 00:00:01 |

| 13 | PARTITION RANGE ALL | | 2883K| 57M| 760 (12)| 00:00:01 |

|* 14 | TABLE ACCESS INMEMORY FULL| CATALOG_SALES | 2883K| 57M| 760 (12)| 00:00:01 |


---------------------------------------------------

2 - access("CS_BILL_HDEMO_SK"="HD_DEMO_SK")

3 - inmemory("HD_DEP_COUNT"<=2 AND "HD_DEP_COUNT">=0)

filter("HD_DEP_COUNT"<=2 AND "HD_DEP_COUNT">=0)

4 - access("CS_ITEM_SK"="I_ITEM_SK")

6 - inmemory("I_CATEGORY"='Books' OR "I_CATEGORY"='Electronics')

filter("I_CATEGORY"='Books' OR "I_CATEGORY"='Electronics')

7 - access("CS_SHIP_DATE_SK"="ITEM_1")

10 - inmemory("D_DATE"<=TO_DATE(' 2000-02-29 00:00:00', 'syyyy-mm-dd hh24:mi:ss')

AND "D_DATE">=TO_DATE(' 1999-11-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))

filter("D_DATE"<=TO_DATE(' 2000-02-29 00:00:00', 'syyyy-mm-dd hh24:mi:ss')

AND "D_DATE">=TO_DATE(' 1999-11-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))

14 - inmemory(SYS_OP_BLOOM_FILTER_LIST(SYS_OP_BLOOM_FILTER(:BF0001,"CS_SHIP_DATE_SK")

,SYS_OP_BLOOM_FILTER(:BF0000,"CS_ITEM_SK")))

filter(SYS_OP_BLOOM_FILTER_LIST(SYS_OP_BLOOM_FILTER(:BF0001,"CS_SHIP_DATE_SK")

,SYS_OP_BLOOM_FILTER(:BF0000,"CS_ITEM_SK")))

Outline Data

-------------

/*+

BEGIN_OUTLINE_DATA

FULL(@"SEL$7502D05D" "DATE_DIM"@"SEL$1")

SWAP_JOIN_INPUTS(@"SEL$F6D5D92C" "HOUSEHOLD_DEMOGRAPHICS"@"SEL$1")

SWAP_JOIN_INPUTS(@"SEL$F6D5D92C" "ITEM"@"SEL$1")

PX_JOIN_FILTER(@"SEL$F6D5D92C" "ITEM"@"SEL$1")

PX_JOIN_FILTER(@"SEL$F6D5D92C" "CATALOG_SALES"@"SEL$1")

USE_HASH(@"SEL$F6D5D92C" "HOUSEHOLD_DEMOGRAPHICS"@"SEL$1")

USE_HASH(@"SEL$F6D5D92C" "ITEM"@"SEL$1")

USE_HASH(@"SEL$F6D5D92C" "CATALOG_SALES"@"SEL$1")

LEADING(@"SEL$F6D5D92C" "VW_GBF_17"@"SEL$22C70CCB" "CATALOG_SALES"@"SEL$1" "ITEM"@"SEL$1"

"HOUSEHOLD_DEMOGRAPHICS"@"SEL$1")

FULL(@"SEL$F6D5D92C" "HOUSEHOLD_DEMOGRAPHICS"@"SEL$1")

FULL(@"SEL$F6D5D92C" "ITEM"@"SEL$1")

FULL(@"SEL$F6D5D92C" "CATALOG_SALES"@"SEL$1")

NO_ACCESS(@"SEL$F6D5D92C" "VW_GBF_17"@"SEL$22C70CCB")

. . .

ALL_ROWS

DB_VERSION('12.2.0.1')

OPTIMIZER_FEATURES_ENABLE('12.2.0.1')

IGNORE_OPTIM_EMBEDDED_HINTS

END_OUTLINE_DATA

*/

DBIM: In-Memory Joins


Improved In-Memory Joins (IMJ)

What if there are no filter predicates on joined tables?

• Only choice in 12cR1: HASH JOIN in PGA memory

• Each IMCU as a local dictionary – so subscripts for same values may be different

• Data has to be decompressed from IMCS (expensive) and then hashed values are

probed in PGA

• In 12cR2, In-Memory Joins can leverage Join Groups

• Useful when query offers no filter predicates on joined tables

• Leverages new IMCU global dictionary to find matching data

• Since subscripts are identical for same values in all tables, no decompression is required!

• Join Groups are identified with CREATE INMEMORY JOIN GROUP command

• Viewable in V$IM_SEGDICT


IMCUs and Column Units: A Closer Look

CUgeo_id

CUpart_id

CUqty

CUsold_on

SALES_FACT With a local dictionary, each table’s IMCUs stores the column value within the CU

These minimum and maximum values can be leveraged for MIN/MAX pruning via In-Memory

Storage Indexes


Join Groups: A Common Dictionary for IMCUs

SALES_FACT

CommonDictionary

Column Value

ID

Austria 0

Canada 2

… …

Sweden 12

USA 13

Zimbabwe 14

IMCU2150

113

1311

IMCU13

1112131

144

IMCU103491463

IMCU5

123

112

121

14

GEO_DIM

IMCU789

1011121314

Holland

India

Netherlands

Poland

Russia

Sweden

USA

Zimbabwe

1.3

1350

0.9

24.1

183.2

4.1

326

1.6

IMCUgeo_id

0123456

name

Austria

Belgium

Canada

Denmark

Ethiopia

France

Germany

pop

1.5

0.7

15.2

2.8

13.1

42.8

52.1

With a common dictionary, each

table’s IMCUs have the same value

stored in every CU …

… for the samecolumn defined in the Join Group …

… instead of the actual column

value itself


Hash Joins Without Join Groups

GEO_DIM

IMCU789

1011121314

Holland

India

Netherlands

Poland

Russia

Sweden

USA

Zimbabwe

1.3

1350

0.9

24.1

183.2

4.1

326

1.6

IMCUgeo_id

0123456

name

Austria

Belgium

Canada

Denmark

Ethiopia

France

Germany

pop

1.5

0.7

15.2

2.8

13.1

42.8

52.1

SALES_FACTIMCU

2150

113

1311

IMCU

13

1112131

144

IMCU

103491463

IMCU

5123

112

121

14

PGA

SELECT SF.*

FROM sales_fact SF, geo_dim GD

WHERE SF.geo_id = GD.geo_id

AND GD.pop >= 50;

HASH TABLE

Scan GEO_DIM, decompress matching rows, and send them to Hash Join

1 Use uncompressed GEO_DIM values to build hash join table in PGA

2 Scan SALES_FACT and filter rows based on predicate

3 Decompress just the matching SALES_FACT rows, then send them to hash join

4 Probe hash join table for matching GEO_ID value, then return matches to query’s session

5

1

2

3

4

5

? = ?


Hash Joins With Join Groups

GEO_DIM

IMCU789

1011121314

Holland

India

Netherlands

Poland

Russia

Sweden

USA

Zimbabwe

1.3

1350

0.9

24.1

183.2

4.1

326

1.6

IMCUgeo_id

0123456

name

Austria

Belgium

Canada

Denmark

Ethiopia

France

Germany

pop

1.5

0.7

15.2

2.8

13.1

42.8

52.1

SALES_FACTIMCU

2150

113

1311

IMCU

13

1112131

144

IMCU

103491463

IMCU

5123

112

121

14

PGA

SELECT SF.*

FROM sales_fact SF, geo_dim GD

WHERE SF.geo_id = GD.geo_id

AND GD.pop >= 50;

1

2

3

4

5

Common Dictionary Array

0 0 0 0 0 0 1

0 1 0 0 1 0 1

Scan GEO_DIM and send common dictionary codes for matching GEO_IDs in compressed format to Hash Join

1 Using compressed common dictionary values, build an array of distinct values

2 Scan SALES_FACT and filter rows based on predicate

3 Send just matching SALES_FACT rows in compressed format to hash join

4 Complete hash join operation by looking up compressed values in array

5

? = ?


Creating Join Groups

Add Join Groups for TPCDS.STORE_SALES:1

SQL> CREATE INMEMORY JOIN GROUP tpcds.jg_ss_date (

tpcds.store_sales (ss_sold_date_sk)

,tpcds.date_dim (d_date_sk)

);

CREATE INMEMORY JOIN GROUP tpcds.jg_ss_cdemo (

tpcds.store_sales (ss_cdemo_sk)

,tpcds.customer_demographics (cd_demo_sk)

);

CREATE INMEMORY JOIN GROUP tpcds.jg_ss_hdemo (

tpcds.store_sales (ss_hdemo_sk)

,tpcds.household_demographics (hd_demo_sk)

);

Note: Only one Join Group per column permitted!2

SQL> CREATE INMEMORY JOIN GROUP tpcds.jg_cs_date (

tpcds.catalog_sales (cs_sold_date_sk)

,tpcds.date_dim (d_date_sk)

);

SQL> 2 3 4 ,tpcds.date_dim (d_date_sk)

*

ERROR at line 3:

ORA-00957: duplicate column name

Join Group common directory entries (after repopulation in IMCS)3

SELECT

joingroup_name

,table_name

,column_name

,gd_address

FROM dba_joingroups

ORDER BY 1,2;

Join Groups

(from DBA_JOINGROUPS)

Common

Join Group Directory

Name Table Name Column Name Address

------------ ------------------------ ------------------- ----------------

JG_SS_CDEMO CUSTOMER_DEMOGRAPHICS CD_DEMO_SK 0000000655005E60

JG_SS_CDEMO STORE_SALES SS_CDEMO_SK 0000000655005E60

JG_SS_DATE DATE_DIM D_DATE_SK 00000006550E5E60

JG_SS_DATE STORE_SALES SS_SOLD_DATE_SK 00000006550E5E60

JG_SS_HDEMO HOUSEHOLD_DEMOGRAPHICS HD_DEMO_SK 0000000655135E60

JG_SS_HDEMO STORE_SALES SS_HDEMO_SK 0000000655135E60


Detecting Join Group Usage

Query against TPCDS.STORE_SALES and other dimensions that can benefit from Join Groups:

1

SQL> SELECT /*query100*/

d_year

,d_qoy

,hd_income_band_sk

,cd_gender

,cd_marital_status

,SUM(ss_gross_revenue)

,SUM(ss_net_revenue)

,SUM(ss_extd_gpm)

FROM

tpcds.store_sales SS

,tpcds.customer_demographics CD

,tpcds.household_demographics HD

,tpcds.date_dim DD

WHERE SS.ss_cdemo_sk = CD.cd_demo_sk

AND SS.ss_hdemo_sk = HD.hd_demo_sk

AND SS.ss_sold_date_sk = DD.d_date_sk

GROUP BY

d_year, d_qoy

,hd_income_band_sk, cd_gender, cd_marital_status

ORDER BY

d_year, d_qoy

,hd_income_band_sk, cd_gender, cd_marital_status;

Did a Join Group get used? Proof via PL/SQL …2

SET SERVEROUTPUT ON

DECLARE

b_sqlid VARCHAR2(32);

vc_jgusage VARCHAR2(50);

BEGIN

SELECT prev_sql_id

INTO :b_sqlid

FROM v$session

WHERE SID = USERENV('SID');

vc_jgusage :=

DBMS_SQLTUNE.REPORT_SQL_MONITOR_XML(

sql_id => :b_sqlid).EXTRACT(q'#//operation[@name='HASH

JOIN']/rwsstats/stat[@id='9']#').GETCLOBVAL(2,2) join_group_usage;

DBMS_OUTPUT.PUT_LINE(‘Join Group Usage: ‘ || vc_jgusage);

END;

/

… or via SQL:3

SET LONGCHUNKSIZE 10000000 LONG 10000000

COL join_group_usage FORMAT A50

SELECT

DBMS_SQLTUNE.REPORT_SQL_MONITOR_XML(sql_id=>'9mtgnc8fxmg1f').EXTRACT(q'#//opera

tion[@name='HASH JOIN']/rwsstats/stat[@id='9']#').GETCLOBVAL(2,2)

join_group_usage

FROM DUAL;JOIN_GROUP_USAGE

---------------------

<stat id="9">1</stat>

<stat id="9">1</stat>


Demonstrating Join Group Value

PARALLEL(4), but INMEMORY_QUERY = DISABLE:1

Elapsed: 00:00:07.62Execution Plan

----------------------------------------------------------


--------------------------------------------------------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |

--------------------------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 38190 | 3170K| | 18506 (1)| 00:00:01 |

| 1 | PX COORDINATOR | | | | | | |

| 2 | PX SEND QC (ORDER) | :TQ10002 | 38190 | 3170K| | 18506 (1)| 00:00:01 |

| 3 | SORT GROUP BY | | 38190 | 3170K| 530M| 18506 (1)| 00:00:01 |

| 4 | PX RECEIVE | | 38190 | 3170K| | 18506 (1)| 00:00:01 |

| 5 | PX SEND RANGE | :TQ10001 | 38190 | 3170K| | 18506 (1)| 00:00:01 |

| 6 | HASH GROUP BY | | 38190 | 3170K| 530M| 18506 (1)| 00:00:01 |

|* 7 | HASH JOIN | | 5773K| 467M| | 8012 (1)| 00:00:01 |

| 8 | PX RECEIVE | | 1920K| 18M| | 1234 (1)| 00:00:01 |

| 9 | PX SEND BROADCAST | :TQ10000 | 1920K| 18M| | 1234 (1)| 00:00:01 |

| 10 | PX BLOCK ITERATOR | | 1920K| 18M| | 1234 (1)| 00:00:01 |

| 11 | TABLE ACCESS FULL| CUSTOMER_DEMOGRAPHICS | 1920K| 18M| | 1234 (1)| 00:00:01 |

|* 12 | HASH JOIN | | 5760K| 412M| | 6772 (1)| 00:00:01 |

| 13 | TABLE ACCESS FULL | DATE_DIM | 73049 | 927K| | 114 (0)| 00:00:01 |

|* 14 | HASH JOIN | | 5760K| 340M| | 6653 (1)| 00:00:01 |

| 15 | TABLE ACCESS FULL | HOUSEHOLD_DEMOGRAPHICS | 7200 | 50400 | | 10 (0)| 00:00:01 |


| 17 | TABLE ACCESS FULL| STORE_SALES | 5760K| 302M| | 6639 (1)| 00:00:01 |

--------------------------------------------------------------------------------------------------------------

PARALLEL(4), but INMEMORY_QUERY = ENABLE:2

Elapsed: 00:00:03.43

Execution Plan

----------------------------------------------------------


-----------------------------------------------------------------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |

-----------------------------------------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 38190 | 3170K| | 10876 (2)| 00:00:01 |

| 1 | PX COORDINATOR | | | | | | |

| 2 | PX SEND QC (ORDER) | :TQ10002 | 38190 | 3170K| | 10876 (2)| 00:00:01 |

| 3 | SORT GROUP BY | | 38190 | 3170K| 530M| 10876 (2)| 00:00:01 |

| 4 | PX RECEIVE | | 38190 | 3170K| | 10876 (2)| 00:00:01 |

| 5 | PX SEND RANGE | :TQ10001 | 38190 | 3170K| | 10876 (2)| 00:00:01 |

| 6 | HASH GROUP BY | | 38190 | 3170K| 530M| 10876 (2)| 00:00:01 |

|* 7 | HASH JOIN | | 5773K| 467M| | 382 (22)| 00:00:01 |

| 8 | PX RECEIVE | | 1920K| 18M| | 51 (10)| 00:00:01 |

| 9 | PX SEND BROADCAST | :TQ10000 | 1920K| 18M| | 51 (10)| 00:00:01 |


| 11 | TABLE ACCESS INMEMORY FULL| CUSTOMER_DEMOGRAPHICS | 1920K| 18M| | 51 (10)| 00:00:01 |

|* 12 | HASH JOIN | | 5760K| 412M| | 324 (22)| 00:00:01 |

| 13 | TABLE ACCESS INMEMORY FULL | DATE_DIM | 73049 | 927K| | 5 (20)| 00:00:01 |

|* 14 | HASH JOIN | | 5760K| 340M| | 315 (22)| 00:00:01 |

| 15 | TABLE ACCESS INMEMORY FULL | HOUSEHOLD_DEMOGRAPHICS | 7200 | 50400 | | 2 (0)| 00:00:01 |


| 17 | TABLE ACCESS INMEMORY FULL| STORE_SALES | 5760K| 302M| | 309 (21)| 00:00:01 |

-----------------------------------------------------------------------------------------------------------------------

DBIM: ILM and ADO


In-Memory Area: At Last, A True Storage Tier 0!

• Heat Maps can now used to decide which segments should remain in IMCS or be evicted based on memory pressure

• In 12cR2, Information Lifecycle Management (ILM) now supports Automatic Data Optimization (ADO) policies for managing objects within the In-Memory Area

41

This effectively enables the In-Memory Area to function as a true Tier 0 storage tier above flash storage!


Heat Maps and ILM ADO Policies

• Heat Map data is flushed to disk once per hour to facilitate effective utilization of In-Memory Area

• New view _SYS_HEAT_MAP_SEG_HISTOGRAM shows actual counts of heat map accesses

• ILM ADO policies can now leverage NO INMEMORY criteria – i.e. to evict an object from the IMCS

• ADO is included in the price of 12cR2 In-Memory option

• Advanced Compression Option (ACO) doesn’t need to be licensed if it’s going to be used to manage the IMA

42


Creating ILM ADO Policies for In-Memory Area Objects

SQL>

-- Enable Heat Maps

ALTER SYSTEM SET heat_map = ON;

-- Build ILM Policy for a table that places it into IMCS

-- two (2) days after it was created:

ALTER TABLE tpcds.catalog_sales

ILM ADD POLICY

SET INMEMORY MEMCOMPRESS FOR QUERY LOW

PRIORITY HIGH SEGMENT

AFTER 2 DAYS OF CREATION;

-- Build ILM Policy for fact table with multiple partitions,

-- favoring most recent partitions and aging out oldest partitions

ALTER TABLE tpcds.store_sales

ILM ADD POLICY

MODIFY INMEMORY MEMCOMPRESS FOR QUERY HIGH

PRIORITY MEDIUM SEGMENT

AFTER 15 DAYS OF CREATION;

-- Build ILM Policy for a fact table with multiple partitions,

-- favoring most recently-accessed partitions and aging out

-- oldest partitions

ALTER TABLE tpcds.inventory

ILM ADD POLICY

NO INMEMORY AFTER 180 DAYS OF NO ACCESS;

ALTER TABLE tpcds.catalog_returns

ILM ADD POLICY


ALTER TABLE tpcds.store_returns

ILM ADD POLICY


ALTER TABLE tpcds.web_returns

ILM ADD POLICY


ALTER TABLE tpcds.web_sales

ILM ADD POLICY


SELECT

IO.policy_name

,IO.object_name

,IP.action_type

,IP.condition_type

,IP.condition_days

,TO_CHAR(SUBSTR(IP.actionc_clob,1,120)) ilm_action

FROM

dba_ilmobjects IO

,dba_ilmdatamovementpolicies IP

WHERE IO.policy_name = IP.policy_name

AND IO.object_type = 'TABLE'

ORDER BY IO.policy_name

,IO.object_name

,IO.subobject_name

,IO.object_type;

ILM Data Movement Policies

(from DBA_ILMOBJECTS and DBA_ILMDATAMOVEMENTPOLICIES)

ILM Action Cndtn

Policy Table Name Type Condition Type Days Custom Function

------ ------------------------ ----------- -------------------- ------ ---------------------

P28 CATALOG_SALES ANNOTATE CREATION TIME 2 INMEMORY MEMCOMPRESS

FOR QUERY LOW

PRIORITY HIGH

P29 STORE_SALES COMPRESSION CREATION TIME 15 INMEMORY MEMCOMPRESS

FOR QUERY HIGH

PRIORITY MEDIUM

P30 INVENTORY EVICT LAST ACCESS TIME 180

P31 CATALOG_RETURNS EVICT LAST ACCESS TIME 180

P32 STORE_RETURNS EVICT LAST ACCESS TIME 180

P33 WEB_RETURNS EVICT LAST ACCESS TIME 180

P34 WEB_SALES EVICT LAST ACCESS TIME 180


Example: ILM ADO Policy Evaluation

Speed experimentation by changing evaluation period to SECONDS instead of DAYS1

BEGIN

DBMS_ILM_ADMIN.CUSTOMIZE_ILM(

parameter => DBMS_ILM_ADMIN.POLICY_TIME

,value => DBMS_ILM_ADMIN.ILM_POLICY_IN_SECONDS

);

END;

/

Next, generate a SwingbenchTPCDS Query Workload …

2

Enable ILM, then force execution of ILM ADO policies for specific tables. (This overrides the normal ILM evaluation period that occurs during the next scheduled maintenance window.)

2

SQL> EXEC DBMS_ILM_ADMIN.ENABLE_ILM;

DECLARE

tid NUMBER;

BEGIN

DBMS_ILM.EXECUTE_ILM(owner => 'TPCDS'

,object_name => 'CATALOG_SALES’ ,task_id => tid);

DBMS_ILM.EXECUTE_ILM(owner => 'TPCDS‘

,object_name => 'STORE_SALES’ ,task_id => tid);


,object_name => 'WEB_SALES’ ,task_id => tid);


,object_name => 'INVENTORY‘ ,task_id => tid);


,object_name => 'CATALOG_RETURNS‘ ,task_id => tid);


,object_name => 'STORE_RETURNS‘ ,task_id => tid);


,object_name => 'WEB_RETURNS‘ ,task_id => tid);

END;

/

… and via In-Memory Central, watch the ILM ADO Policies fire to modify compression levels, as well as evict any “colder” database objects!

4


DBIM: Other Enhancements

• IMCS Fast-Start

• Active Data Guard (ADG)

• RAC Improvements

• Extending IMCS to Exadata Smart Flash Cache


In-Memory Fast Start

• Fast way to populate IMCS from disk using a special FASTSTART tablespace

• Oldest 15% of segments in IM FastStart are dropped

• Checkpoints continue until IMFS tablespace space is exhausted

46


IMCS and Active Data Guard (ADG)

IMCS has now been extended to an Active Data Guard environment

• Enabled on standby only via INMEMORY_ADG_ENABLED (default: TRUE)

• This actually required significant enhancements in how redo generation is

now handled:

• Redo generated by DML activity has been extended to maintain info on whether

changes need to be applied to an in-memory object

• BEGIN … END transaction redo now has info on which objects were touched

• DDL affecting an object in IMCS needed to be tracked, too

• IMCS is populated on ADG either during first access, or based on priority of

object in IMCS

• Redo shipped to ADG will capture ROWIDs of changed rows and will be used

to refresh those objects in ADG’s IMCS

47


IMCS and Active Data Guard (ADG)

Three scenarios:

48

Primary

TPCDS_DATA

IMCS

TPCDS_DATA

IMCS

PhysicalStandby

Same objects in IMCS on Primary and ADG Standby1

CATALOG_SALESCATALOG_RETURNSDATE_DIM. . .


No objects in IMCS on Primary, only in IMCS on ADG Standby (useful for offloading analytic workloads to

secondary environment)

2


“Hottest” objects in IMCS of Primary, “colder” objects in ADG IMCS (Note: Different tables / partitions in primary vs. ADG must use DISTRIBUTE FOR SERVICE)

3

CATALOG_SALES[Y17Q1]CATALOG_RETURNS[Y17Q1]DATE_DIM. . .

CATALOG_SALES[Y16]CATALOG_RETURNS[Y16]DATE_DIM. . .


IMCS and RAC: Improvements

• IMCS can be scaled out across all instances in a RAC database

• DISTRIBUTE clause determines how IM data is distributed across instances• AUTO (default)

• Also BY PARTITION, BY SUBPARTITION, and BY ROWID RANGE

• IMCUs are never shipped across the interconnect

49


IMCS and RAC

IMCS for RAC has been upgraded:

• DISTRIBUTE BY {partition | subpartition | row range} enhanced with FOR

SERVICE <mode> <service_name> subclause

• DEFAULT

• PRIORITY populates to all instances with a column store enabled

• ON DEMAND populates to all instances mapped by issuing query’s service

• 12cR2: bytes_not_populated column in GV$IM_SEGMENTS has been

improved

• If a specified service is stopped, and FOR SERVICE NONE is specified, then

even though the service is stopped all the other IMCS attributes are retained

in the instance of the stopped service

50

Database In-Memory Features: Conclusions


Conclusions

12cR2 DBIM architecture enhancements bring significant opportunities for

performance improvement:

• Dynamically-Resizable IMCS means hardly ever having to bounce an instance

to change DBIM size

• SWAP JOIN INPUT operations make LEFT DEEP TREE into RIGHT DEEP TREE

joins so efficient Bloom Filtering can be applied

• Join Groups enable joins for queries without selection criteria to quickly find

matching data between huge tables

• ILM ADO Policies enable IMCS as a true Tier 0 storage level

• DBIM now supports Active Data Guard – great for analytical processing

offload


If you have any questions or comments, feel free to:

E-mail me at [email protected]

Follow my blog (Generally, It Depends): http://jimczuprynski.wordpress.com

Follow me on Twitter (@JimTheWhyGuy)

Connect with me on LinkedIn (Jim Czuprynski)

Please reach out if you have questionsWe appreciate your feedback and insights!

mailto:[email protected]

http://jimczuprynski.wordpress.com/

oracle 12cr2 database in-memory: adventures with ... · 12-10-2017 · dynamically resizable...

Documents