working with partitioned tables the unpleasant details brian hitchcock ocp 8, 8i, 9i dba sun...

Working with Partitioned Tables The Unpleasant Details

Brian HitchcockOCP 8, 8i, 9i DBA

Sun Microsystems

[email protected]

[email protected]

Brian Hitchcock November 13, 2003

mailto:[email protected]

mailto:[email protected]

NoCOUG


The Application – History

Website click-stream data Large amounts of data generated daily Users want to access data immediately

– Next day was acceptable Users want 14 months of data on-line Performance was poor

– Huge tables, millions of rows– Inserting new data very slow– Retrieving data very slow– User generate SQL

No controls on the quality of the SQL

NoCOUG


Application – How it Works

Website click-stream log files sent to db machine daily Database tables setup with weekly partitioning

– 3 main tables– Range partitioning on date

New data added to db once per day– Load must finish before users can access each day– Failure of load process, users can’t work, must load 2 days

data to catch up

NoCOUG


The Application – Act I

Consultants setup weekly partitioning– Vendor scripts all setup for daily partitioning– We had to setup our own maintenance scripts

Once per week Create new week partition oldest weekly partition

Analyze new partition– Vendor tech support won’t support weekly partitioning

14 months of data on-line, 60 weekly partitions

NoCOUG


The Application – Act II

The Consultant Is Gone (call Brian…) Vendor wants us to upgrade

– New app version is much faster (“trust us”)– Need to move to daily partitioning

Users agree to 7 months data online– Improve performance overall (less total data)

We can use vendor’s maintenance scripts– Reduced headcount makes our scripts hard to maintain– Easier to get support from vendor tech support

NoCOUG


Project Phases

As I inherited the project– 14 months weekly partitions

Vendor Upgrade I– Move to daily partitioning– Drop 7 months of data

Vendor Upgrade II– Required renaming of index partitions

Must match table partition names

NoCOUG


What Are Partitioned Tables? Single Big Table

– Many sub-tables (partitions) Each partition

– Acts like a separate table– Index (partition) for each partition

Performance– Load to separate partition(s)– Analyze each partition separately– SQL can be satisfied with small subset of entire table

Partition pruning Answer to a Jeopardy question

– Alternate term for DBA job security…

NoCOUG


Why Use Them?

Performance Faster SQL

– Partition Pruning

Faster Analyze– Only analyze single partition

Logical– Application data is ‘partitioned’ (weekly for this case)– Easy to add new partition, drop oldest partition

Faster than deleting some rows from larger table Faster than inserting new rows into larger table

NoCOUG


Partitioned Table

Data Data 10/01/2003
























Non-PartitionedPartitioned

Column A Column B Date




Table_1Table_1 Partition_1

Table_1 Partition_2

Table_1 Partition_3

NoCOUG


Inserting Data

Oracle examines– Inserted value of column used for range

partitioning– High_value of existing partitions

Rows with values greater than or equal to the highest high_value

– If MAXVALUE used, inserted– If MAXVALUE not used, rejected

NoCOUG


Partitioned Table













Partitioned




Partition_1 high_value 10/02/2003



Values > or = 10/02/2003 rejected

Rows with range values less than 02/02/2003Will be inserted into Partition_3

Rows with range values less than 06/02/2003And > or = 02/02/2003Will be inserted into Partition_2

Rows with range values less than 10/02/2003And > or = to 06/02/2003Will be inserted into Partition_1

NoCOUG


Partitioned Table MAXVALUE













Partitioned – Using MAXVALUE




Partition 1 high_value MAXVALUE

Partition 2 high_value 06/02/2003

Partition 3 high_value 02/02/2003

Rows with range values less than 02/02/2003Will be inserted into Partition 3

Rows with range values less than 06/02/2003Will be inserted into Partition 2

Rows with range values greater than or = to 06/02/2003Will be inserted into Partition 1

NoCOUG


Partitioning -- Example

CREATE TABLE stock_xactions

(stock_symbol CHAR(5), stock_series CHAR(1), num_shares NUMBER(10), price NUMBER(5,2),

trade_date DATE)

STORAGE (INITIAL 100K NEXT 50K) LOGGING

PARTITION BY RANGE (trade_date)

(PARTITION sx1992 VALUES LESS THAN (TO_DATE('01-JAN-1993','DD-MON-YYYY')),

PARTITION sx1993 VALUES LESS THAN (TO_DATE('01-JAN-1994','DD-MON-YYYY')),

PARTITION sx1994 VALUES LESS THAN (MAXVALUE));

select * from user_tab_partitions;

TABLE_NAME COM PARTITION_NAME SUBPARTITI HIGH_VALUE

----------------------------- --- ------------------------------ ---------- ---------------------------------------------------------------------

STOCK_XACTIONS N0 SX1992 0 TO_DATE(' 1993-01-01 00:00:00', 'SYYYY-MM-DD HH24:MI:SS',

STOCK_XACTIONS N0 SX1993 0 TO_DATE(' 1994-01-01 00:00:00', 'SYYYY-MM-DD HH24:MI:SS',

STOCK_XACTIONS N0 SX1994 0 MAXVALUE

3 rows selected.

NoCOUG


Weekly Partition Maintenance Once per week

– Compute high_value of next weekly partition– Create partition for next week– Drop oldest weekly partition– Rebuild indexes

Or drop and recreate indexes– Load data into new partition

Analyze new partition

Once loaded, analyzed– Maximum read performance– Partition no longer has inserts or analyze

NoCOUG


Weekly Maintenance New

Latest

Oldest

Add New Partition – Load Data, Analyze

Drop Oldest Partition

Existing Partitions

Existing Partitions

14 months, 60 weekly partitions

NoCOUG


Partitions and Tablespaces For performance, want to control placement of

partitions– New, data being loaded– Existing, data being retrieved

Use separate tablespace for each partition Spread across 8 filesystems (8 sets of disks)

– Tablespaces of sequential table partitions– Same for index partitions but offset– Corresponding table/index partitions always on

different filesystems

NoCOUG


Partitions and Tablespaces

Design Goal– Partition being loaded, analyzed

One separate disks from partitions being read Assumes most queries read data from most recent

partitions If partition_1 being loaded

Partitions 2, 3, 4 can be read without conflicting with the load, analyze

– Partitions can be used for fine-grained placement of data on disk

NoCOUG


A Plug for Certification

I didn’t know anything about Partitioned Tables

Wasn’t planning to use them Memorized what I needed to pass the OCP

exam Within a month…

– Phone rings – “Can you help with partitioned tables?”

NoCOUG


Types of Partitioning

Range– Partition based on value of column(s) of table

Date Part Number SSN

Hash– Rows spread evenly across all partitions

List (9i)– Partitions based on user-specified lists of values

NoCOUG


Partitioning Details

Create partitions– You need to name each partition– If you don’t

Oracle names them for you The names are not easy to use

Many other partitioning possibilities– This is not a training course on partitioned tables– Only cover what I actually used– More options with 9i, this was for 8i

NoCOUG


What About Indexes?

Indexes of Partitioned Tables– Local

Index partitions always align with table partitions

– Global Different partitioning from table partitioning

– Become invalid when Add, drop, split existing partitions

– SQL to manually rebuild invalid or unusable index partitions

NoCOUG


Partition Maintenance

Depends on your application May need to handle

– Adding new– Dropping old– Splitting– Analyzing

NoCOUG


Partitioned Table Info

System Tables– dba_tables– dba_tab_partitions– dba_indexes– dba_ind_partitions– dba_part_key_columns

NoCOUG


Partitioned Table Info -- SQL Number of partitioned tables

– select count(*) from dba_tables where partitioned='YES';

Name of each partitioned table– select table_name, partitioned from dba_tables where partitioned='YES'

order by table_name;

Number of partitions in each partitioned table– select table_name, count(*) from dba_tab_partitions group by table_name;

Same info for indexes– select count(*) from dba_indexes where partitioned='YES';– select index_name, partitioned from dba_indexes where

partitioned='YES' order by index_name;– select index_name, count(*) from dba_ind_partitions group by

index_name;

NoCOUG


Partitioned Table Info -- SQL

Number of columns used, range partitioning– select name, object_type, count(*) from

dba_part_key_columns group by name, object_type;

Partitioning key columns– select * from dba_part_key_columns;

Indexes on each partitioned table– select table_name, index_name from dba_indexes where

tablespace_name is NULL order by table_name, index_name;

NoCOUG


Partitioned Table Info -- SQL

High values for table partitions– set long 30– select table_name, tablespace_name, partition_name,

high_value from dba_tab_partitions order by table_name, tablespace_name, partition_name;

High values for index partitions– set long 30– select index_name, tablespace_name, partition_name,

high_value from dba_ind_partitions order by index_name, tablespace_name, partition_name;

NoCOUG


Partition Range How to find ‘range’ for each partition?

– Only have high_value for each partition– Have to examine ordered list of high_value– Ranges not stored in system tables

Range of values is dynamic– Range of highest partition changes as higher values

inserted (assumes MAXVALUES used)– Same for lowest partition

Examine ordered list of high_value– Can’t “order by” LONG– high_value is a LONG

NoCOUG


Partition Range

Need ordered list of high_values Use DBA_TAB_PARTITIONS

– partition_position– High_value

Use partition_position?– Oracle docs only say

position of the partition within the table What exactly does that mean?

NoCOUG


Partition Range

Partition_Position– Numbers the partitions in order of high_value– Numbers change with each add, split, drop of any

partition– Is dynamic

What to do?– I decided to use high_value because it doesn’t

change as partitions are manipulated– But, can’t “order by”

More on this later…

NoCOUG


Vendor Upgrade I

Existing 14 months data– Weekly partitions

Reduce to 7 months– Hard to find the correct partitions to drop

Split each weekly partition– 7 daily partitions

NoCOUG


Do the Math

7 Months (36 weeks)– 252 daily partitions– Times 3 tables– Times 11 indexes– Total of 3528 partitions = (756 table + 2772 index)

Vendor upgrade– Requires specific, rigid partition naming scheme– I don’t get to pick the partition names

NoCOUG


Oracle Utilities? There aren’t any that

– Split tables based on your requirements– Take a set of existing partitions, tablespaces– Move to new partitions, tablespaces

No utility to map from one partitioning to another As the number of partitions grows, so does the work

involved to shift from one partitioning scheme to another

The same things that make partitioned tables so good for performance make them a lot more work to maintain

– Lots of small pieces to maintain

NoCOUG


Partition Split Issues

When splitting– Need new tablespaces– Have existing tablespaces– Split SQL sends both table and index partitions to same

new tablespace– New tablespace needs to be 2x final size– Separate tablespace for new index partitions– Reclaim disk space from new table partition tablespace

Overall need 3x disk space during splits– Need to reclaim this disk space after move to daily

partitions

NoCOUG


Disk Space Needed

Existing Table Partition TP1 Existing Index Partition IP1

Post-split Table Partition TP1a, Index Partition IP1a

Tablespace Ind1Tablespace Tab1

Tablespace Tab2

Post-split Table Partition TP1b, Index Partition IP1b

Tablespace Ind2

Assuming index partition is same size as table partitionAssumes indexes not dropped before split

Table Partition TP1a Index Partition IP1a Table Partition TP1b Index Partition IP1b

Tablespace Tab3 Tablespace Ind3 Tablespace Tab4 Tablespace Ind4

Need 3x original tablespace (disk space)

Original Partitions

Split Partitions

Separate Table/Index Partitions

NoCOUG


Disk Space Reclaimed

After split, need to reclaim 2/3 of disk space

Existing Table Partition TP1 Existing Index Partition IP1

Post-split Table Partition TP1a, Index Partition IP1a

Tablespace Ind1Tablespace Tab1

Tablespace Tab2

Post-split Table Partition TP1b, Index Partition IP1b

Tablespace Ind2

Table Partition TP1a Index Partition IP1a Table Partition TP1b Index Partition IP1b

Tablespace Tab3 Tablespace Ind3 Tablespace Tab4 Tablespace Ind4

EMPTY TBLSPC

NoCOUG


Tasks for Daily Partitioning

Create new tablespaces– Single tablespace for partitions of each table– Twice as big as final size– Single tablespace for partitions of each index

Drop existing indexes (weekly partitions)– Total disk space needed reduced from 3x to 2x

Split weekly partitions into daily partitions Re-create indexes

– Daily partitions created automatically Reclaim disk space

NoCOUG


Move to Daily Partitioning

Reduce to 7 months week partitions– SQL to drop existing weekly partitions simple to generate– Hard to see which partitions to drop since you can’t order

by high_value

Split Existing Weekly Partitions– Generate SQL for one week– Cut/paste/edit for other weekly partitions

Tedious, error prone SQL isn’t consistent enough to make automation

straight-forward 37 weekly partitions need splitting

NoCOUG


Weekly Partition Split-- existing weekly partition BRH_TAB_PART_5

partition BRH_TAB_PART_5_nexta

partition BRH_TAB_PART_5_nextb

partition BRH_TAB_PART_5_nextc

partition BRH_TAB_PART_5_nextd

BRH_TAB_PART_5_nexte

BRH_TAB_ PART_5_7

BRH_TAB_ PART_5_7

BRH_TAB_ PART_5_6

BRH_TAB_ PART_5_5

BRH_TAB_ PART_5_4

BRH_TAB_ PART_5_3

BRH_TAB_ PART_5_2

BRH_TAB_ PART_5_1

BRH_TAB_ PART_5_3

BRH_TAB_ PART_5_4

BRH_TAB_ PART_5_6

BRH_TAB_ PART_5_5

BRH_TAB_ PART_5_2

BRH_TAB_ PART_5_1

a

High_value = 2003-05-10

High_value = 2003-05-09

High_value = 2003-05-08

High_value = 2003-05-07

High_value = 2003-05-06

High_value = 2003-05-05

High_value = 2003-05-04

2003-05-10

2003-05-09

2003-05-08

2003-05-07

2003-05-06

2003-05-05

NoCOUG


Weekly Partition Split SQL-- existing weekly partition BRH_TAB_PART_5 TO_DATE('2003-05-10 00:00:00'alter table brhuser.BRH_TABLE split partition BRH_TAB_PART_5 at ( TO_DATE('2003-05-09 00:00:00', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')) into (partition BRH_TAB_PART_5_nexta tablespace BRH_TAB_TBLSPC, partition BRH_TAB_PART_5_7 tablespace BRH_TAB_TBLSPC);

alter table brhuser.BRH_TABLE split partition BRH_TAB_PART_5_nexta at ( TO_DATE('2003-05-08 00:00:00', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')) into (partition BRH_TAB_PART_5_nextb tablespace BRH_TAB_TBLSPC, partition BRH_TAB_PART_5_6 tablespace BRH_TAB_TBLSPC);

alter table brhuser.BRH_TABLE split partition BRH_TAB_PART_5_nextb at ( TO_DATE('2003-05-07 00:00:00', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')) into (partition BRH_TAB_PART_5_nextc tablespace BRH_TAB_TBLSPC, partition BRH_TAB_PART_5_5 tablespace BRH_TAB_TBLSPC);

alter table brhuser.BRH_TABLE split partition BRH_TAB_PART_5_nextc at ( TO_DATE('2003-05-06 00:00:00', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')) into (partition BRH_TAB_PART_5_nextd tablespace BRH_TAB_TBLSPC, partition BRH_TAB_PART_5_4 tablespace BRH_TAB_TBLSPC);

alter table brhuser.BRH_TABLE split partition BRH_TAB_PART_5_nextd at ( TO_DATE('2003-05-05 00:00:00', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')) into (partition BRH_TAB_PART_5_nexte tablespace BRH_TAB_TBLSPC, partition BRH_TAB_PART_5_3 tablespace BRH_TAB_TBLSPC);

alter table brhuser.BRH_TABLE split partition BRH_TAB_PART_5_nexte at ( TO_DATE('2003-05-04 00:00:00', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')) into (partition BRH_TAB_PART_5_1 tablespace BRH_TAB_TBLSPC, partition BRH_TAB_PART_5_2 tablespace BRH_TAB_TBLSPC);

NoCOUG


Weekly Partition Split SQL Split partition SQL

– Requires two new partition names– Splits existing partition by specifying new high_value

Existing data moves to two new partitions based on above or below new high_value

If you don’t specify partition names, Oracle names them for you

Oracle partition names are not intuitive…– New partitions move to specified tablespaces– New index partitions also moved to same tablespaces

Need to move new index partitions to separate tablespace(s) as separate step

Or, drop indexes, recreate after all splits done– Indexes split automatically as table splits

NoCOUG


Vendor Upgrade II

Now that daily partitioning is setup– Vendor upgrade fails– Specific index partition naming requirements– Our home-grown weekly partitioning didn’t meet these

We need this done by Monday…– Labor Day Weekend

I didn’t want to do this manually again Need to rename existing index partition names

– Number in partition name must agree with number in table partition name

Lots of SQL that must be perfect

NoCOUG


But Wait, There’s More

Vendor had executed a script that partially renamed some partitions

I had to fix this Existing names of these partitions conflicted

with the names I had to generate Vendor upgrade process requires specific

format for daily partition name and number– Where is the Vendor DBA?

NoCOUG


How to Automate?

What needs to be done– Document existing names of table and index

partitions– Rename index partitions using vendor

requirements– Deal with vendor’s mistakes – some partitions

already renamed– Develop process to deal with this and future

needs

NoCOUG


SQL

Document existing table and index partition names

Create new names for index partitions– Index partitions use same numbering as

corresponding table partition names

Generate all SQL statements to alter existing index partition names

Document all intermediate steps

NoCOUG


Details

High Value of each partition stored in LONG column

LONG datatype is special– Can’t be used in any SQL function– Can’t order by etc.– In SQL*Plus, set LONG 100 to see all of

high_value

NoCOUG


SQL Script

Need to ‘order by’ high value to show existing weekly partitions

– How to get around LONG limitations? Export

LONG is converted to a string Import

Import string into VARCHAR2 column Can ‘order by’ on the VARCHAR2 column

NoCOUG


Table of Index Partition Names

Store data about existing table and index partitions

Use SQL to create new index partition names within the table

– Flexible– Self-documenting

Select from table to generate SQL for– Changing index partition names– Fix vendor partition naming mistakes

NoCOUG


For Each Partitioned Index Create table

– Table name– Table tablespace name– Table partition name– Table partition high value (text version)– Index name– Index tablespace name– Index partition name (Existing)– Index partition high value (text version)– Temporary index partition name– Final index partition name (used for renaming)

NoCOUG


Index Partition Renaming

The process– Spool existing table and index info

Including high_value for each partition– Use SQL*Loader to insert info into table– Use SQL to insert temporary and final index

partition names into table Temporary index partition name deals with any

name conflicts…– Use SQL to generate all SQL statements to

actually change index partition names

NoCOUG


SQL Used--> generate file of table partitions for BRH_TABLE...

sqlplus -s brhuser@BRH_DB1

spool BRH_TABLE_partition_data_05242003.txt

set long 30

set pagesize 1500

set linesize 150

set echo off

set feedback off

set heading off

select SUBSTR(table_name, 1, 25), ',', SUBSTR(tablespace_name,1,20), ',', SUBSTR(partition_name,1,20),',"', high_value, '"' from dba_tab_partitions where table_name='BRH_TABLE' order by partition_name;

spool off

exit

example data:

BRH_TABLE , BRH_TAB_TBLSPC , BRH_TAB_PART_1 ," TO_DATE(' 2003-05-04 00:00:00' "





NoCOUG


SQL Used create table to load this data into...

This table holds the existing table partition data ordered by partition name

create table BRH_TABLE_part_data_t

(table_name VARCHAR(30), tab_tblspc_name VARCHAR2(25), tab_part_name VARCHAR2(25), tab_high_value_text VARCHAR2(35));

--> sqlldr control file to load this data...

sqlldr_load_BRH_TABLE_partition_data.ctl

load data

infile BRH_TABLE_partition_data_05242003.txt'

into table BRH_TABLE_part_data_t

fields terminated by ',' optionally enclosed by '"'

(table_name, tab_tblspc_name, tab_part_name, tab_high_value_text)

--> sqlldr command...

sqlldr userid=psycho/psycho control=sqlldr_load_BRH_TABLE_partition_data.ctl log=sqlldr_load_BRH_TABLE_partition_data.log

NoCOUG


SQL Used--> trim the high_value_text

update BRH_TABLE_part_data_t set tab_high_value_text=SUBSTR(tab_high_value_text,12,19);

commit;

Example data:

BRH_TABLE BRH_TAB_TBLSPC BRH_TAB_PART_1 2003-05-04 00:00:00





NoCOUG


SQL Used--> create table to hold same data but in high_value order...

create table BRH_TABLE_part_data

(table_name VARCHAR(30),

tab_tblspc_name VARCHAR2(25),

tab_part_name VARCHAR2(25),

tab_high_value_text VARCHAR2(35));

--> populate the new table...

insert into BRH_TABLE_part_data

(select * from BRH_TABLE_part_data_t) order by tab_high_value_text;

select * from BRH_TABLE_part_data;

commit;

repeat for the other 2 partitioned tables...

NoCOUG


SQL UsedNow, for the index partitions, for each of 11 indexes, generate file of index partitions for BRH_INDEX...

spool BRH_INDEX_partition_data_05252003.txt

set long 30

set pagesize 1500

set linesize 150

set echo off

set feedback off

set heading off

select SUBSTR(index_name, 1, 25), ',', SUBSTR(tablespace_name,1,20), ',', SUBSTR(partition_name,1,20),',"', high_value, '"' from dba_ind_partitions where index_name='X_BRH_INDEX' order by partition_name;

spool off

example data:

X_BRH_INDEX, BRH_IND_TBLSPC , BRH_IND_PART_246 ," TO_DATE(' 2003-05-04 00:00:00' "





note how these partition numbers do not line up with the table partition numbers, and, note that these 5 rows were spread all over the spool file making it very hard to see which table/index partition names didn't have the same partition numbers...

NoCOUG


SQL Used--> create table to load this data into...

create table BRH_INDEX_part_data_t

(index_name VARCHAR(30), ind_tblspc_name VARCHAR2(25), ind_part_name VARCHAR2(25), ind_high_value_text VARCHAR2(35));

--> sqlldr control file to load this data...

sqlldr_load_BRH_INDEX_partition_data.ctl

load data

infile BRH_INDEX _partition_data_05252003.txt'

into table BRH_INDEX _part_data_t

fields terminated by ',' optionally enclosed by '"'

(index_name, ind_tblspc_name, ind_part_name, ind_high_value_text)

--> sqlldr command...

sqlldr userid=psycho/psycho control=sqlldr_load_ BRH_INDEX _partition_data.ctl log=sqlldr_load_ BRH_INDEX _partition_data.log

NoCOUG


SQL Used--> trim the high_value_text

update BRH_INDEX_part_data_t set ind_high_value_text=SUBSTR(ind_high_value_text,12,19);

commit;

--> create table to hold same data but in high_value order...

create table BRH_INDEX_part_data

(index_name VARCHAR(30),

ind_tblspc_name VARCHAR2(25),

ind_part_name VARCHAR2(25),

ind_high_value_text VARCHAR2(35));

NoCOUG


SQL Used--> populate the new table...

insert into BRH_INDEX_part_data

(select * from BRH_INDEX_part_data_t) order by ind_high_value_text;

select * from BRH_INDEX_part_data;

example data:

X_BRH_INDEX BRH_IND_TBLSPC BRH_IND _PART_15 2002-09-15 00:00:00

X_BRH_INDEX BRH_IND _TBLSPC BRH_IND _PART_16 2002-09-16 00:00:00




commit;

repeat for the other 10 indexes

NoCOUG


SQL Usedwe now have partition info for all 3 tables, 11 indexes in separate tables, all ordered by high_value -- still very difficult to see which partition names don't line up between tables and indexes since they are all in separate tables

INDEX_NAME IND_TBLSPC_NAME IND_PART_NAME IND_HIGH_VALUE_TEXT

------------------------------ ------------------------- ------------------------- -----------------------------------

X_BRH_INDEX BRH_IND_TBLSPC BRH_IND_PART_15 2002-09-15 00:00:00



...



X_BRH_INDEX BRH_IND_TBLSPC BRH_IND_PART _252 2003-05-10 00:00:00








X_BRH_INDEX BRH_IND_TBLSPC BRH_TAB_PART_15 2003-05-18 00:00:00







NoCOUG


SQL Used For this index

– Partition names progress <index name>_15 to 252 <index name>_8 to 14 <table name>_15 to 21

– When ordered by high_value– the 15 to 21 are the partition names messed up

by the vendor script... other indexes are <index name>_1 to 252

– table for these indexes has partition names running 22-252, 1-21

NoCOUG


SQL Usedfor each index, create table that will contain the table partition names and the index partition names

create table BRH_INDEX_TI_PART_NAMES

(table_name 2 VARCHAR2(30),

tab_tblspc_name VARCHAR2(25),

tab_part_name VARCHAR2(25),

tab_high_value_text VARCHAR2(35),

index_name VARCHAR2(30),

ind_tblspc_name VARCHAR2(25),

ind_part_name VARCHAR2(25),

ind_high_value_text VARCHAR2(35),

temp_ind_part_name VARCHAR2(25),

final_ind_part_name VARCHAR2(25));

--> insert table partition name info

insert into BRH_INDEX_TI_PART_NAMES (table_name, tab_tblspc_name, tab_part_name, tab_high_value_text)

(select * from BRH_TABLE_part_data) order by tab_high_value_text;

--> insert index partition name info

insert into BRH_INDEX_TI_PART_NAMES (index_name, ind_tblspc_name, ind_part_name, ind_high_value_text)

(select * from BRH_INDEX_part_data) order by ind_high_value_text;

--> DUH!!! this creates 252 rows of table partition data followed by 252 rows of index partition data...

NoCOUG


SQL Usedtry again

--> insert table partition name info

insert into BRH_INDEX_TI_PART_NAMES (table_name, tab_tblspc_name, tab_part_name, tab_high_value_text)

(select * from BRH_TABLE_part_data) order by tab_high_value_text;

update BRH_INDEX_TI_PART_NAMES a

set (index_name, ind_tblspc_name, ind_part_name, ind_high_value_text) =

(select index_name , ind_tblspc_name, ind_part_name, ind_high_value_text

from BRH_INDEX_part_data b

where a.tab_high_value_text=b.ind_high_value_text);

NoCOUG


SQL UsedNow we can look at table partition names and index partition names at the same time

selected from BRH_INDEX_TI_PART_NAMES;

TAB_PART_NAME TAB_HIGH_VALUE_TEXT IND_PART_NAME IND_HIGH_VALUE_TEXT

------------------------------ ------------------------- ------------------------- -----------------------------------

BRH_TAB_PART_22 2002-09-15 00:00:00 BRH_IND_PART_15 2002-09-15 00:00:00





...






BRH_TAB_PART_15 2003-05-18 00:00:00 BRH_TAB_PART_15 2003-05-18 00:00:00


...







NoCOUG


SQL UsedNote the additional columns TEMP_IND_PART_NAME, FINAL_IND_PART_NAME these are going to hold the two new names for each partition

some of the index partitions already have the vendor specified naming convention, if we simply changed all existing index partition names to the vendor convention, we would

get errors since, for some partitions, the partition name is already in use...

update BRH_INDEX_TI_PART_NAMES set temp_ind_part_name=TRIM(ind_part_name)||'_temp';

be careful to order by high_value...

SQL> select * from BRH_INDEX_TI_PART_NAMES order by tab_high_value_text;

TABLE_NAME TAB_TBLSPC_NAME TAB_PART_NAME TAB_HIGH_VALUE_TEXT INDEX_NAME IND_TBLSPC_NAME IND_PART_NAME

IND_HIGH_VALUE_TEXT TEMP_IND_PART_NAME FINAL_IND_PART_NAME

------------------------------ ------------------------- ------------------------- ----------------------------------- ------------------------------ ------------------------- ------------------------- ----------------------------------- ------------------------- -------------------------

BRH_TABLE BRH_TAB_TBLSPC BRH_TAB_PART_22 2002-09-15 00:00:00 X_BRH_INDEX BRH_IND_TBLSPC BRH_IND_PART_15 2002-09-15 00:00:00

BRH_IND_PART_15_temp









NoCOUG


SQL Usedupdate BRH_INDEX_TI_PART_NAMES set temp_ind_part_name=TRIM(ind_part_name)||'_temp';

fix index partition names that were messed up by vendor script

update BRH_INDEX_TI_PART_NAMES set temp_ind_part_name='BRH_INtmp'||SUBSTR(temp_ind_part_name,12,25)

where temp_ind_part_name like 'BRH_TAB_PART%';

example

BRH_TAB_PART_15_temp becomes BRH_INtmp_15_temp

--> start building final index partition name, start with the table partition name...

this gives us the number for the final index partition name, i.e. the same as the table partition name number

update BRH_INDEX_TI_PART_NAMES set final_ind_part_name=tab_part_name;

this replaces the table name with the vendor required spec for the index partition name

update BRH_INDEX_TI_PART_NAMES set final_ind_part_name='BRH_IND_PART'||SUBSTR(final_ind_part_name,12,25);

NoCOUG


SQL UsedFinally, we have all the data we need to construct the SQL statements to alter all the index partition names

First, rename all the index partition names to the temporary index partition names to deal with duplicate conflicts

SQL> select 'alter index analysis.'||TRIM(index_name)||' rename partition '||TRIM(ind_part_name)||' TO '||TRIM(temp_ind_part_name)||';‘ from BRH_INDEX_TI_PART_NAMES order by tab_high_value_text;

alter index analysis.X_BRH_INDEX rename partition BRH_IND_PART_15 TO BRH_IND_PART_15_temp;









...



alter index analysis.X_BRH_INDEX rename partition BRH_TAB_PART_15 TO BRH_INtmp_15_temp;







NoCOUG


SQL UsedSecond, rename index partition names to their final names using the same number(s) as the table partition numbers

select 'alter index analysis.'||TRIM(index_name)||' rename partition '||TRIM(temp_ind_part_name)||' TO '||TRIM(final_ind_part_name)||';‘ from BRH_INDEX_TI_PART_NAMES order by tab_high_value_text;

alter index analysis.X_BRH_INDEX rename partition BRH_IND_PART_15_temp TO BRH_IND_PART_22;









...



alter index analysis.X_BRH_INDEX rename partition BRH_INtmp_15_temp TO BRH_IND_PART_15;







NoCOUG


SQL Usedthe SQL is spooled to separate files and executed against the db

spool BRH_INDEX_TI_PART_NAMES_tmp_SQL.txt

select 'alter index analysis.'||TRIM(index_name)||' rename partition '||TRIM(ind_part_name)||' TO '||TRIM(temp_ind_part_name)||';'

from BRH_INDEX_TI_PART_NAMES order by tab_high_value_text;

spool off

spool BRH_INDEX_TI_PART_NAMES_final_SQL.txt

select 'alter index analysis.'||TRIM(index_name)||' rename partition '||TRIM(temp_ind_part_name)||' TO '||TRIM(final_ind_part_name)||';'

from BRH_INDEX_TI_PART_NAMES order by tab_high_value_text;

spool off

--> repeat for the other 10 indexes (11 indexes total)

NoCOUG


Script Testing

Use development environment Subset of production data

– Same number of daily partitions as production– Rename existing index partitions

NoCOUG


Result?

Scripts ran without error Everyone is happy Prepared to do it all again if needed No need for cut/paste Demonstrated method to handle high_value

– LONG datatype work-around

NoCOUG


Performance Issues

After all this– Are you sure you are getting the benefits?

Explain plan shows partitions accessed Partition Pruning

– Optimizer can tell which partitions aren’t involved in completing the query

Need to check– Partition pruning is happening– Where clause can prevent this

User generated SQL is problematic

NoCOUG


Explain Plan -- Goodexplain plan set Statement_Id = 'TEST' for

CREATE TABLE BRHPSYCHO1 NOLOGGING

AS SELECT t161.brh_col1 AS col0, t168.brh_col2 AS col1, t172.brh_col2 AS col2,

t176.brh_col2 AS col3,

COUNT(DISTINCT t161.brh_col3) AS countcol

FROM brh_table1 t161, brh_table2 t168, brh_table2 t172, brh_table2 t176, brh_table2 t185

WHERE t172.brh_col4 = t176.brh_col3 AND t168.brh_col4 = t172.brh_col3 AND

t161.brh_col5 = t185.brh_col3 AND t185.brh_col4 = t168.brh_col3 AND

t161.brh_col6 >= TO_DATE('2001/08/08 00:00:00','YYYY/MM/DD HH24:MI:SS') AND

t161.brh_col6 <= TO_DATE('2001/08/0823:59:59','YYYY/MM/DDHH24:MI:SS') AND


t185.brh_col7 <= TO_DATE('2001/08/08 23:59:59','YYYY/MM/DDHH24:MI:SS') AND


t168.brh_col7 <= TO_DATE('2001/08/08 23:59:59','YYYY/MM/DD HH24:MI:SS') AND




t176.brh_col7 <= TO_DATE('2001/08/08 23:59:59','YYYY/MM/DDHH24:MI:SS')

GROUP BY t161.brh_col1, t168.brh_col2, t172.brh_col2, t176.brh_col2;

NoCOUG


Explain Plan -- Good--------------------------------------------------------------------------------

| Operation | Name | Rows | Bytes| Cost | Pstart| Pstop

--------------------------------------------------------------------------------

| CREATE TABLE STATEMENT | | 4 | 448 | 67 | | |

| LOAD AS SELECT | | | | | | |

| SORT GROUP BY | | 4 | 448 | 67 | | |

| NESTED LOOPS | | 4 | 448 | 55 | | |

| NESTED LOOPS | | 4 | 372 | 43 | | |

| NESTED LOOPS | | 4 | 272 | 31 | | |

| NESTED LOOPS | | 4 | 172 | 19 | | |

| TABLE ACCESS BY LOC|brh_table1| 4 | 92 | 7 | 1 | 1 |

| INDEX RANGE SCAN |brh_ind1 | 4 | | 3 | 1 | 1 |

--------------------------------------------------------------------------------

| TABLE ACCESS BY LOC|brh_table2| 28 | 560 | 3 | 2 | 2 |


| TABLE ACCESS BY LOCA|brh_table2| 28 | 700 | 3 | 2 | 2 |


| TABLE ACCESS BY LOC |brh_table2| 28 | 700 | 3 | 2 | 2 |


| TABLE ACCESS BY LOCAL |brh_table2| 28 | 532 | 3 | 2 | 2 |


--------------------------------------------------------------------------------

$ORACLE_HOME/rdbms/admin/utlxpls (use correct version!)

NoCOUG


Explain Plan -- Badexplain plan set Statement_Id = 'TEST' for

CREATE TABLE BRHPSYCHO1 NOLOGGING

AS SELECT t161.brh_col1 AS col0, t168.brh_col2 AS col1, t172.brh_col2 AS col2,

t176.brh_col2 AS col3,

COUNT(DISTINCT t161.brh_col3) AS countcol

FROM brh_table1 t161, brh_table2 t168, brh_table2 t172, brh_table2 t176,

brh_table2 t185

WHERE t172.brh_col4 = t176.brh_col3 AND t168.brh_col4 = t172.brh_col3 AND

t161.brh_col5 = t185.brh_col3 AND t185.brh_col4 = t168.brh_col3 AND



t185.brh_col7 >= TO_DATE('2001/08/08 00:00:00','YYYY/MM/DD HH24:MI:SS') OR t185.brh_col3=0 AND

t185.brh_col7 <= TO_DATE('2001/08/08 23:59:59','YYYY/MM/DD HH24:MI:SS') OR t185.brh_col3=0 AND






t176.brh_col7 <= TO_DATE('2001/08/08 23:59:59','YYYY/MM/DDHH24:MI:SS')

GROUP BY t161.brh_col1, t168.brh_col2, t172.brh_col2, t176.brh_col2;

NoCOUG


Explain Plan -- BadPlan Table --------------------------------------------------------------------------------

| Operation | Name | Rows | Bytes| Cost | Pstart| Pstop |

--------------------------------------------------------------------------------

| CREATE TABLE STATEMENT | |4254322|443760|70970810| | |

| LOAD AS SELECT | | | | | | |

| SORT GROUP BY | |4254322|443760| | | |

| SORT GROUP BY | |4254322|443760| | | |

| SORT GROUP BY | |4254322|443760| | | |

| CONCATENATION | | | | | | |

| MERGE JOIN CARTESIAN| | 4 | 172 | 83 | | |

| MERGE JOIN CARTESIA| | 4 | 272 | 167 | | |

| MERGE JOIN CARTESI| | 4 | 92 | | | |

Plan Table --------------------------------------------------------------------------------

| MERGE JOIN CARTES| | 4 | 448 | 335 | | |

| PARTITION RANGE | | | | | 1 | 10 |

| TABLE ACCESS BY|brh_table2| 4M| 115M| 21 | 1 | 10 |

| INDEX RANGE SC|brh_ind2 | 4M| | 20 | 1 | 10 |

| SORT JOIN | | 4M| 87M| | | |



| SORT JOIN| | 4 | 92 | | | |

| PARTITION RANGE | | | | | 1 | 9 |

| TABLE ACCESS FU|brh_table1| 4 | 92 | 7 | 1 | 9 |

| SORT JOIN | | 4M| 115M| | | |

NoCOUG



| PARTITION RANGE A| | | | | 1 | 10 |

| TABLE ACCESS FUL|brh_table2| 4M| 115M| 21 | 1 | 10 |

| SORT JOIN | | 4M| 92M| | | |

| PARTITION RANGE AL| | | | | 1 | 10 |

| TABLE ACCESS FULL|brh_table2| 4M| 92M| 19 | 1 | 10 |



| NESTED LOOPS| | 4 | 448 | 335 | | |

| MERGE JOIN CARTES| | 4 | 92 | | | |



Plan Table --------------------------------------------------------------------------------


| SORT JOIN| | 4 | 92 | | | |

| PARTITION RANGE| | | | | 1 | 9 |

| TABLE ACCESS F|brh_table1| 4 | 92 | 7 | 1 | 9 |



| SORT JOIN | | 4M| 115M| | | |



| SORT JOIN | | 4M| 92M| | | |


NoCOUG




| NESTED LOOPS| | 4 | 372 | 251 | | |


| MERGE JOIN CARTESI| | 4 | 448 | 335 | | |


| PARTITION RANGE | | | | | 1 | 10 |



| SORT JOIN| | 4 | 92 | | | |



Plan Table --------------------------------------------------------------------------------

| SORT JOIN| | 4M| 87M| | | |

| PARTITION RANGE | | | | | 1 | 10 |

| TABLE ACCESS FU|brh_table2| 4M| 87M| 21 | 1 | 10 |

| SORT JOIN | | 4M| 92M| | | |



| PARTITION RANGE ITE| | | | | 2 | 10 |

| TABLE ACCESS FULL |brh_table2| 4M| 115M| 21 | 2 | 10 |


| NESTED LOOPS| | 4 | 448 | 335 | | |

| NESTED LOOPS| | 4 | 372 | 251 | | |

NoCOUG







| SORT JOIN| | 4 | 92 | | | |







Plan Table --------------------------------------------------------------------------------

| SORT JOIN | | 4M| 92M| | | |



| NESTED LOOPS| | 4 | 272 | 167 | | |

| NESTED LOOP| | 4 | 448 | 335 | | |

| MERGE JOIN CARTESI| | 4 | 372 | 251 | | |


| PARTITION RANGE | | | | | 1 | 10 |



| SORT JOIN| | 4 | 92 | | | |

NoCOUG





| SORT JOIN| | 4M| 115M| | | |

| PARTITION RANGE | | | | | 1 | 10 |

| TABLE ACCESS FU|brh_table2| 4M| 115M| 21 | 1 | 10 |



| PARTITION RANGE ITE| | | | | 2 | 10 |


| NESTED LOOPS| | 4 | 448 | 335 | | |

| NESTED LOOP| | 4 | 372 | 251 | | |

Plan Table --------------------------------------------------------------------------------

| NESTED LOOPS| | 4 | 272 | 167 | | |





| SORT JOIN| | 4 | 92 | | | |






NoCOUG




| PARTITION RANGE ALL| | | | | 1 | 10 |


| NESTED LOOPS| | 4 | 448 | 335 | | |

| NESTED LOOPS| | 4 | 372 | 251 | | |

| NESTED LOOPS| | 4 | 272 | 167 | | |

| NESTED LOOPS | | 4 | 172 | 83 | | |

| TABLE ACCESS BY |brh_table1| 4 | 92 | 7 | 1 | 1 |

| INDEX RANGE SCA|brh_ind1 | 4 | | 3 | 1 | 1 |

| PARTITION RANGE | | | | | 2 | 10 |


Plan Table --------------------------------------------------------------------------------



| TABLE ACCESS BY |brh_table2| 4M| 115M| 21 | 1 | 10 |

| INDEX RANGE SCA|brh_ind2 | 4M| | 20 | 1 | 10 |


| TABLE ACCESS BY L|brh_table2| 4M| 115M| 21 | 1 | 10 |

| INDEX RANGE SCAN|brh_ind2 | 4M| | 20 | 1 | 10 |

| PARTITION RANGE ALL| | | | | 1 | 10 |

| TABLE ACCESS BY LO|brh_table2| 4M| 87M| 21 | 1 | 10 |

| INDEX RANGE SCAN |brh_ind2 | 4M| | 20 | 1 | 10 |

--------------------------------------------------------------------------------

132 rows selected.

NoCOUG


Why Bad?

Optimizer can’t predict the future– Bind variables

Don’t know range of values needed until run time

– Join order Can’t see which partitions needed until too late

Other reasons– Old statistics

Distribution of rows among partitions

SQL may address all partitions

NoCOUG


How To Choose Partition Size?

For queries– partition size may not matter– SQL must be written specifically to take advantage of

partitions (pruning)– Do most queries look at a continuous range of days,

weeks? For Loading data

– How often do you load data?– Good to analyze small amounts of data

For backups– Once loaded, partitions in dedicated tablespace can be

read-only, reduce size of backups

NoCOUG


Partitioned Tables

Good– Improve performance– Improve management

Load, analyze

Bad– Increased administration– Confusing (3528 partitions to deal with)– Only specific SQL will perform well– Which tables to partition?

working with partitioned tables the unpleasant details brian hitchcock ocp 8, 8i, 9i dba sun...

Documents