exadata resource management teile und herrsche! · exadata resource management –teile und...

2014 © Trivadis

BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN

2014 © Trivadis

Exadata Resource Management

– teile und herrsche!

Konrad HÄFELI

Senior Solution Manager

Infrastructure Engineering

14.05.2014

DOAG Exaday: Exadata Resource Management - teile und herrsche!

1

2014 © Trivadis

Trivadis ist führend bei der IT-Beratung, der Systemintegration,

dem Solution-Engineering und der Erbringung von IT-Services

mit Fokussierung auf und Technologien

im D-A-CH-Raum.

Unsere Leistungen erbringen wir aus den strategischen Geschäftsfeldern:

Trivadis Services übernimmt den korrespondierenden Betrieb

Ihrer IT Systeme.

Unser Unternehmen

14.05.2014


B E T R I E B

2

2014 © Trivadis

Mit über 600 IT- und Fachexperten bei Ihnen vor Ort

3

12 Trivadis Niederlassungen mit

über 600 Mitarbeitenden

200 Service Level Agreements

Mehr als 4'000 Trainingsteilnehmer

Forschungs- und Entwicklungs-

budget: CHF 5.0 Mio. / EUR 4.0

Mio.

Finanziell unabhängig und

nachhaltig profitabel

Erfahrung aus mehr als 1'900

Projekten pro Jahr bei über 800

Kunden

Stand 12/2013

Hamburg

Düsseldorf

Frankfurt

FreiburgMünchen

Wien

Basel

ZürichBern

Lausanne

3

Stuttgart

14.05.2014


Brugg

3

2014 © Trivadis

AGENDA

1. Introduction

2. Database Resource Manager

3. Exadata IO Resource Manager

4. Tests

5. Tooling

6. Conclusion


14.05.2014

4

2014 © Trivadis

14.05.2014


Introduction

5

2014 © Trivadis

Introduction

Exadata is known for its enormous performance

This is optimal for consolidation

Concurrency forces bottlenecks

… but the resource cake is not endless

How can I guarantee everybody's piece?

Resource Management

14.05.2014


6

2014 © Trivadis

Database Resource Management

The goal of resource management is to optimize resources allocation

among different users / groups or programs

Example: Online users get more power than batch users

Example: „batch“ gets many resources during the night and little during the

day

Sounds familiar!?!

DBRM was introduced in Oracle 8i

It’s not just about CPU (more introduced over the coming versions)

Also helps to manage runaway queries, parallel queries, idle sessions, undo

and temp space

Instance caging

Limits, and takes automatic actions if a limit is exceeded

14.05.2014


7

2014 © Trivadis

Exadata Resource Management

Intra-Database

via DBRM

Inter-Database is introduced

with IORM on Exadata

14.05.2014


8

2014 © Trivadis

14.05.2014


Database Resource Manager

9

2014 © Trivadis

Overview DBRM

Oracle Resource Manager is organized with

Resource Plans

Directives

Consumer Groups

14.05.2014


Ref: Oracle 11gR2 Database Administrator‘s guide

10

2014 © Trivadis

DBRM – Resource Plans

A resource plan regulates, how specific resources are shared among the

consumer groups. It also regulates how unused resources are

redistributed

Resource Plans get activated on Oversubscription

Only one resource plan is active at a specific time

It can be enabled manually by parameter

Or it can be enabled by a scheduler window

14.05.2014


11

2014 © Trivadis

DBRM – Directives

A resource plan directive assigns values/rules to the resource plan.

There are many parameters available, specifying values for such as

cpu consumption up to 8 levels

number of concurrently active sessions

parallel degree limit

amount of undo information in KB for a session

Maximum idle time for a blocking session

Amount of MB transferred (Read/Write) by a session before action specified

in SWITCH_GROUP ist taken

14.05.2014


12

2014 © Trivadis

DBRM – Consumer Groups

A consumer group is a collection of sessions that have similar

requirements

Consumer groups can also be created to represent certain kinds of

workload

There exist predefined consumer groups in the oracle server, for

example SYS_GROUP and the default group for sessions

OTHER_GROUPS

A consumer group mapping rule defines, how sessions are mapped to

consumer groups, for ex:

service_name

oracle_user

client_program

client_machine

14.05.2014


13

2014 © Trivadis

DBRM – Manipulation (1)

The Resource Manager can be switched on- or off online

SQL> alter system set resource_manager_plan = ‘DEFAULT_PLAN‘

sid=‘*‘;

SQL> select plan from dba_rsrc_plans order by 1;

PLAN

------------------------------

APPQOS_PLAN

DAYTIME_PLAN

DEFAULT_MAINTENANCE_PLAN

DEFAULT_PLAN

DSS_PLAN

ETL_CRITICAL_PLAN

INTERNAL_PLAN

INTERNAL_QUIESCE

MIXED_WORKLOAD_PLAN

ORA$AUTOTASK_HIGH_SUB_PLAN

ORA$AUTOTASK_SUB_PLAN

14.05.2014


14

2014 © Trivadis

15

DBRM – Manipulation (2)

BEGIN

DBMS_RESOURCE_MANAGER.CREATE_PENDING_AREA();

DBMS_RESOURCE_MANAGER.CREATE_PLAN('DAYTIME_PLAN', 'Resource plan for managing

all applications between 9 am and 5 pm');

DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP('CRITICAL_APPS', 'Critically

important Apps');

DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP('ONLINE_APPS', 'Online Apps');

DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP('DEVELOPMENT', 'Developpers');

DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE('DAYTIME_PLAN', 'CRITICAL_APPS',

'Allocation for CRITICAL_APPS', MGMT_P1 => 60);

DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE('DAYTIME_PLAN', 'ONLINE_APPS',

'Allocation for ONLINE_APPS', MGMT_P1 => 25);

DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE('DAYTIME_PLAN', 'DEVELOPMENT',

'Allocation for DEVELOPMENT', MGMT_P1 => 10);

DBMS_RESOURCE_MANAGER.CREATE_PLAN_DIRECTIVE('DAYTIME_PLAN', 'OTHER_GROUPS',

'Allocation for default group', MGMT_P1 => 5);

DBMS_RESOURCE_MANAGER.VALIDATE_PENDING_AREA();

DBMS_RESOURCE_MANAGER.SUBMIT_PENDING_AREA();

END;

/


14.05.2014

15

2014 © Trivadis

DBRM – Components

Ratio Based

Reports wants 40% -> Okay?

- Yes, if not oversubscribed

14.05.2014


Priority Based

16

2014 © Trivadis

DBRM – Instance Caging

Small but useful new feature of Oracle 11gR2

Limits (only) the amount of CPU’s a Instance can use

One Instance will not be able to allocate all the CPU resources of a

server

Instance Caging has no impact to the CPU’s that has to be licensed

Two different approaches:

Over Provisioning

Partitioning

SQL> alter system set resource_manager_plan='some plan';

SQL> alter system set cpu_count=5;

14.05.2014


17

2014 © Trivadis

14.05.2014


Exadata IO Resource Manager

18

2014 © Trivadis

Exadata I/O Resource Management (1)

Consumer Groups / Categories

Resource Manager

I/O Metadata: DB Name, Request Type, Consumer Group


14.05.2014

19

2014 © Trivadis

Overview Exadata IORM (1)

Adds the possibility to prioritize I/O’s

Can be used on cell only (dbPlan, Inter Database IORM) or in combination with Resource Manager within the DB (catPlan, Intra Database Plan)

Hard limits (limit) or soft limits (allocation) possible

Limits the I/O requests only if there is I/O pressure on the cell (soft limits)

Without pressure, every database or consumer group is able to use 100% of the I/O throughput of a cell (soft limits)

IORM Plans must be implemented and activated on every cell. Use dclifor that.

14.05.2014


20

2014 © Trivadis

Overview Exadata IORM (2)

IORM Plans can have multiple levels

Sublevels get what is left over from the upper levels

DB Plans and Category Plans always need a “OTHER” group

Database Role specific settings possible

IORM can be activated, deactivated or altered online

oracle@exadb01:bin/ [rdbms1120] ./dcli -g mycells cellcli -e 'alter

iormplan active'

exacel01.trivadis.com: IORMPLAN successfully altered



14.05.2014


21

2014 © Trivadis

Traditional I/O schedulers are black boxes

No possibility to influence

I/O requests are processed in FIFO order

IORM – I/O Scheduling Traditional Way

H H LLLLLRDBMS

Traditional

Storage

Server

I/O

Request

Disk Queue

High Priority

Workload

Low Priority

Workload


14.05.2014

22

2014 © Trivadis

I/O Requests are executed based on the priorisation

Low priority Requests can be queued if needed

IORM – I/O Scheduling Exadata Way

Disk Queue

H L HHLHHRDBMS

I/O

Request

Exadata

HH

L LLLL

I/O

Resource

Manager


14.05.2014

23

2014 © Trivadis

Intra DB RM controls the resources within one database

Tools to use:

- dbms_resource_manager

- Consumer Groups and Plan Directives: CPU and I/O resources

- (Consumer groups represent collections of users within a database)

Inter DB RM controls the resources between multiple databases

Tools to use:

- (Instance Caging: CPU Resources)

- IORM: I/O Resources (dbPlan and/or catPlan on cell)

- dbms_resource_manager Categories: I/O resources

- The category’s must be present in the DB and on each cell

- (categories represent collections of consumer groups across all

databases)

Inter- / Intra Database Resource Management


14.05.2014

24

2014 © Trivadis

Categories in Databases

DBMS_RESOURCE_MANAGER.CREATE_CATEGORY(

CATEGORY => 'prio1',

COMMENT => 'Priority 1 consumer group');







DBMS_RESOURCE_MANAGER.CREATE_CONSUMER_GROUP(

consumer_group => 'CRITICAL_APPS',

comment => 'Critically important Apps',

category => 'PRIO1');


consumer_group => 'ONLINE_APPS',

comment => 'Online Apps',



consumer_group => 'DEVELOPMENT',

comment => 'Developpers',



14.05.2014

25

2014 © Trivadis

IORMPLAN on Cells

dcli -g mycells cellcli -e \

"ALTER IORMPLAN catPlan=$ \

\(name=administrative, level=1, allocation=80$, \

$name=prio1, level=2, allocation=60$, \



$name=other, level=3, allocation=100$ \

\), \

dbPlan=$ \

\(name=rmtest1, level=1, allocation=60$, \

$name=rmtest11, level=1, allocation=10$, \

$name=other, level=2, allocation=100$ \

\)"


14.05.2014

26

2014 © Trivadis

IORM – Priorities

Cell 1

Database 1

60%

42%

Database 2

40%

28%

Database 1

60%

18%

Database 2

40%

12%

CG 1

50%

21%

CG2

50%

21%

CG 3

50%

14%

CG 4

50%

14%

CG 5

75%

13.5%

CG 6

25%

4.5%

CG 7

80%

9.6%

CG 8

20%

2.4%

High

70%

Low

30%Categories

Intra Database

Inter Database


14.05.2014

27

2014 © Trivadis

New feature (1)

11.2.3.2.1 - I/O Resource Management Support for More Than 32

Databases

supports share-based plans which allocate resources based on shares

instead of percentages

which can support up to 1024 databases, and up to 1024 directives for

interdatabase plans

11.2.3.3 - Network Resource Management

Network Resource Management automatically and transparently prioritizes

critical database network messages fabric ensuring fast response times for

latency critical operations.

Prioritization is implemented in the database and the different InfiniBand

adapters,

This feature is enabled by default, and requires no configuration or

management

14.05.2014


28

2014 © Trivadis

New feature (2)

12.1.0.1 - IORM Support for Container Databases and Pluggable

Databases

By using CDB plans and PDB plans, I/O Resource Management (IORM)

provides the ability to manage I/O resource utilization among different PDBs

as well as manage the workloads within each PDB

14.05.2014


...

DBMS_RESOURCE_MANAGER.CREATE_CDB_PLAN(

plan => ''NEWCDB_PLAN ',

comment => 'CDB resource plan for newcdb');

DBMS_RESOURCE_MANAGER.CREATE_CDB_PLAN_DIRECTIVE(

plan => 'NEWCDB_PLAN',

pluggable_database => 'SALESPDB',

shares => 3,

utilization_limit => 100);

...

29

2014 © Trivadis

Kundenbeispiel: Datenbankkonsolidierung auf Exadata

14.05.2014


Exadata Database Machine X4-2

Half Rack

• 3 aktivierte DB Servern (anstatt 4)

• 3 Storage Server Zellen = 36 Disks

(anstatt 7*12=84)

Exadata Database Machine X4-2

Quarter Rack

• 2 DB Servern

• 3 Storage Server Zellen = 36 Disks

Server 1

PROD

•Prod DB‘s

Server 1

•DB1-Prod

•….

•DBn-Prod

•Failover DB‘s

von Server 2

•DB(a)-Prod

•…

•DB(z)-Prod

Server 2

PROD

•Failover DB‘s

von Server 1

•DB1-Prod

•…

•DBn-Prod

• Prod DB‘s

Server 2

•DB(a)-Prod

•….

•DB(z)-Prod

Server 3

•Test

•Entwicklung

•Integration

Server 1

PROD

•Prod DB‘s

Server 1

•DB1-Prod

•….

•DBn-Prod

•Failover DB‘s

von Server 2

•DB(a)-Prod

•…

•DB(z)-Prod

Server 2

PROD

•Failover DB‘s

von Server 1

•DB1-Prod

•…

•DBn-Prod

• Prod DB‘s

Server 2

•DB(a)-Prod

•….

•DB(z)-Prod

Standby-Dataguard 1Standby-Dataguard 2

30

2014 © Trivadis

14.05.2014


Tests

31

2014 © Trivadis

Test runs

80 parallel sessions running a

create table as select on a 3GB

table

DBRM and IORM Tests – Test Environment

Exadata

I1_RMTEST1

exadb1

Oracle GI

I1_RMTEST11

Oracle ASM1

exadb2

Oracle GI

Oracle ASM2

FRA‘FRA

DATA’DATA

I2_RMTEST1

I2_RMTEST11

14.05.2014


32

2014 © Trivadis

33

2 Databases, 1 Instance each, no IORM

Rivalry is visible nicely

DBRM and IORM Tests – Results Inter-Database IORM

(1)


14.05.2014

0

100

200

300

400

500

600

1 3 5 7 9 11 13 15 17

MB

/s

min

2 DBs, 1 Instance each without IORM

RMTEST1 Instance 1

RMTEST11 Instance 2

33

2014 © Trivadis

34

2 Databases, 1 Instance each,

IORM Hard limit (limit=50 and limit=20, level 1 of max possible throughput)

Hard limit: RMTEST11 does not get the full throughput even if RMTEST1 is

idle


(2)


14.05.2014

0

100

200

300

400

500

600

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

MB

/s

min

2 DBs, RMTEST1 Limit 50, RMTEST11 Limit 20, both Level 1

RMTEST1, Instance 1

RMTEST11, Instance 2

34

2014 © Trivadis

35

2 Databases, 1 Instance each, IORM Soft limit

Soft limit: RMTEST1 has 3x more resources (alloc=30 and alloc=10)

As soon as RMTEST1 is finish, RMTEST11 gets all the resources


(3)


14.05.2014

2 DBs, no Limits RMTEST1 alloc 30 RMTEST11 alloc 10, both

Level 1

0

100

200

300

400

500

600

700

800

1 3 5 7 9 11 13 15 17 19

min

MB

/s RMTEST1, Instance 1


35

2014 © Trivadis

36

1 Database, 2 Instances

consumer group “CRITICAL_APPS” (category “prio 1”): 60% on DB

consumer group “DEVELOPMENT” (category “prio 3”): 10% on DB

Category “prio 1”: alloc 30

Category “prio 3”: alloc 5

DBRM and IORM Tests – Results Intra-Database IORM

(1)


14.05.2014

1 DB, 2 Instances, Schema 1 60% (CRITICAL_APPS), Schema 2

10% (DEVELOPMENT) (like Test 9) with IORMPLAN, Schema 1

alloc 30 (Prio 1), Schema 2 alloc 5 (Prio 3)

0

100

200

300

400

500

600

700

1 3 5 7 9 11 13 15 17 19 21

RMTEST1, Instance 1

RMTEST1, Instance 2

36

2014 © Trivadis

37

2 Databases, 4 Instances consumer group “CRITICAL_APPS”

(category “prio 1”): 60% on DB

consumer group “DEVELOPMENT”

(category “prio 3”): 10% on DB

Category “prio 1” Level 2: alloc 60 on cell

Category “prio 3” Level 2: alloc 10 on cell

RMTEST1: alloc 60% on cell

RMTEST11: alloc 10% on cell

CATEGORY is evaluated

first

RMTEST1, Instance 1

gets more resources

because of the priorisation

DBRM and IORM Tests – Results Intra-Database IORM

(2)


14.05.2014

2 DBs, 4 Instances, Schema 1 60% (CRITICAL_APPS),

Schema 2 10% (DEVELOPMENT) with IORM Level 2,

Schema 1 alloc 60, Schema 2 alloc 10, Instance 1 alloc 60,

Instance 2 alloc 10

0

100

200

300

400

500

600

700

800

1 4 7 10 13 16 19 22 25 28 31 34 37 40

min

MB

/s

RMTEST1, Instance 1

RMTEST1, Instance 2



37

2014 © Trivadis

Kind of “complicated”

Has to be changed according the systems needs New Database

New Applications

New User

Flexible and easy to use

Has to be reproducible

Monitorable

Resilient

Write a tool with a configuration file

39

Resource Management Challenges


14.05.2014

39

2014 © Trivadis

40

Resource Management Tool


14.05.2014

Errorcheck, mail, exit

SID list

done

Read conf

Start Script

Generate SID

code

Apply code

to SID-list

Apply code

to CELL-list

40

2014 © Trivadis

Intra Database Category's

Intra Database ConsumerGroups

Intra Database IORM catPlan

Intra Database Resource Plan

Intra Database Resource Plan Directives

Intra Database Consumer Group Mapping

Intra Database Consumer Group Grants

Inter Database IORM dbPlan

41

Resource Management Tool – Configuration Sections


14.05.2014

# Primary databases

dbplan name=RMTEST1, level=1, allocation=60, role=primary

dbplan name=RMTEST11, level=1, allocation=10, role=primary

dbplan name=ECAZT, level=1, allocation=10, role=primary

# Standby databases

dbplan name=RMTEST1, level=2, allocation=60, role=standby

41

2014 © Trivadis

42

Resource Management Tool – Generate script (1)


14.05.2014

# *************************************************************** *

# * M A I N *

# *************************************************************** *

# *-------------------------------------------------------------- *

clear

# ---------------------------------

# Exadata IORM

# ---------------------------------

getCatPlans

getDBPlans

genCellCommand

getRMPlans

getRMCategorys

getRMConsGroups

getRMPlanDirectives

getRMCGMappings

getRMCGGrants

42

2014 © Trivadis

43

Resource Management Tool – Generate script (2)


14.05.2014

.

.

else

genRMScript

applyRMScript

fi

done

fi

checklog

# *************************************************************** *

# * E N D *

# *************************************************************** *

sendMail

43

2014 © Trivadis

Resource Management in Consolidation Environments is a MUST

DBRM and IORM work very well

KISS! Keep it … simple!

First implement a good CATEGORY Plan CRITICAL_APPS, BATCH_LOADING, DEVELOPMENT, REPORTS etc.

Second implement a Database IORM Plan

Only limit the Consumer Groups within a database if needed, it’s easier to use category’s within a database only

A generator Script helps to manage the needed PL/SQL Code

45

Oracle Exadata Resource Management – Conclusion

(1)


14.05.2014

45

2014 © Trivadis

Oracle Exadata Resource Management – Conclusion

(2)

How can you keep the smile on your Database?

With Resource Management!

14.05.2014


46

2014 © Trivadis

BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN

Fragen und Antworten...

2014 © Trivadis

Konrad Häfeli

Senior Solution Manager

Infrastructure Engineering

[email protected]

14.05.2014


exadata resource management teile und herrsche! · exadata resource management –teile und...

Documents