boost your oracle rac manageability with policy-managed databases

2013 © Trivadis

BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN

2013 © Trivadis

Boost your Oracle RAC manageability

with Policy-Managed Databases

Ludovico CALDARA

Oracle ACE, Senior Consultant

Trivadis AG

Datum

Ansicht > Kopf und Fusszeile

1

2013 © Trivadis

AGENDA

1. Admin vs Policy Management

2. Server Pools

3. Databases and services

4. Practical tips

5. Conclusion

Datum


2

2013 © Trivadis

About Ludovico Caldara

Datum


3

■ 14 Years DBA (started with 7.3.4)

▪ I do it everywhere (also on Windows)

■ RAC ATTACK Ninja & writer

■ IOUG RAC SIG Board member

■ OCP (11g, 12c, MySQL) & OCE

■ Moved recently from Italy to Switzerland

■ http://www.ludovicocaldara.net

■ @ludodba

■ ludovicocaldara

http://www.ludovicocaldara.net/

2013 © Trivadis

Trivadis is a market leader in IT consulting, system integration,

solution engineering and the provision of IT services focusing

on and technologies in Switzerland,

Germany and Austria.

We offer our services in the following strategic business fields:

Trivadis Services takes over the interacting operation of your IT systems.

Our company

O P E R A T I O N

Date

Trivadis – the company

2013 © Trivadis

With over 600 specialists and IT experts in your region

5

11 Trivadis branches and more than

600 employees

200 Service Level Agreements

Over 4,000 training participants

Research and development budget:

CHF 5.0 / EUR 4 million

Financially self-supporting and

sustainably profitable

Experience from more than 1,900

projects per year at over 800

customers

Hamburg

Düsseldorf

Frankfurt

Freiburg Munich

Vienna

Basle

Zurich Bern

Lausanne

5

Stuttgart

Trivadis – the company

Date

2013 © Trivadis

Datum


Admin vs Policy Management

6

2013 © Trivadis

Admin-managed databases

Datum


7

Database instances are statically assigned to servers

RAC1 RAC2

HR1 HR2

2013 © Trivadis


Datum


8


To add nodes, several things are configured manually

(add undo, redo, spfile, etc)

RAC1 RAC2

HR1 HR2

RAC3 RAC4

2013 © Trivadis


Datum


9


To add nodes, several things are configured manually

(add undo, redo, spfile, etc)

Then instances are added to the cluster and started $ srvctl add instance –db hr –instance hr3 –node rac3 (same for node 4)

RAC1 RAC2

HR1 HR2

RAC3 RAC4

HR3 HR4

2013 © Trivadis


Datum


10

What if different databases are distributed amongst different servers?

RAC3 RAC4 RAC1 RAC2

HR1 HR2 CRM1 CRM2

2013 © Trivadis


Datum


11


After a node crash you may have to reconfigure instances

RAC3 RAC4 RAC1 RAC2

HR1 HR2 CRM1 CRM2

2013 © Trivadis


Datum


12



This can be error-prone and time consuming

RAC3 RAC4 RAC1 RAC2

HR1 HR2 CRM1 CRM2

2013 © Trivadis


Datum


13



This can be error-prone and time consuming

Hint: Configure an instance per node and run as many as you want

RAC3 RAC4 RAC1 RAC2

HR1 HR2 CRM1 CRM2

2013 © Trivadis

Policy-managed databases

Datum


14

Databases (not instances) are assigned to a server pool $ srvctl add database –db HR –serverpool HR –dbtype

RAC

RAC3 RAC4 RAC1 RAC2

HR_1 HR_2

Pool “HR” MIN_SIZE=2

2013 © Trivadis


Datum


15


RAC

Instead of adding instances, increase server pool size

RAC3 RAC4 RAC1 RAC2

HR_1 HR_2


2013 © Trivadis


Datum


16


RAC


New instances are configured and started dynamically!

RAC3 RAC4 RAC1 RAC2

HR_1 HR_2


HR_3 HR_4

2013 © Trivadis


Datum


17


RAC


New instances are configured and started dynamically!

Have you noticed the underscore?

RAC3 RAC4 RAC1 RAC2

HR_1 HR_2


HR_3 HR_4

2013 © Trivadis


Datum


18

With PMDs, you can assign priorities (Importance)

RAC3 RAC4 RAC1 RAC2

HR_1 CRM_1 CRM_2 HR_2

Pool “HR” Pool “Customers”

2013 © Trivadis


Datum


19


In case of node failure, servers are reassigned automatically, based on

priority

RAC3 RAC4 RAC1 RAC2

HR_1 CRM_1 CRM_2 HR_2


2013 © Trivadis


Datum


20



priority

Critical services will have a guaranteed number of servers

RAC3 RAC4 RAC1 RAC2

HR_1 CRM_1 CRM_2


CRM_1

2013 © Trivadis


Datum


21



priority

Critical services will have a guaranteed number of servers

RAC3 RAC4 RAC1 RAC2

HR_1 CRM_1 CRM_2


CRM_3

2013 © Trivadis

Datum


Server Pools

22

2013 © Trivadis

Generic and Free

Datum


23

RAC3 RAC4 RAC1 RAC2

$ srvctl status serverpool

Server pool name: Free

Active servers count: 4

Server pool name: Generic


Pool “Free”

2013 © Trivadis

Generic and Free

Datum


24

RAC3 RAC4 RAC1 RAC2

$ srvctl status serverpool





HR1 HR2

Pool “Generic” Pool “Free”

2013 © Trivadis

Adding a server pool

Datum


25

RAC3 RAC4 RAC1 RAC2

HR1 HR2

Pool “Generic” Pool “HR”

$ srvctl add srvpool -serverpool hr -min 1 -max 2 -importance 10





Server pool name: hr


2013 © Trivadis

Converting DB from Admin to Policy

Datum


26

RAC3 RAC4 RAC1 RAC2

HR_1 HR_2

Pool “Generic” Pool “HR”

$ srvctl modify database -db hr -serverpool hr

DB is stopped and restarted as policy-managed

Need to reconfigure DB Control in 11g

2013 © Trivadis

Server assignment: The Trinity

Datum


27

MIN_SIZE MAX_SIZE

IMPORTANCE

2013 © Trivadis

Server assignment: explanation

Datum


28

List Server Pools

in order of IMPORTANCE

Fill until MIN_SIZE

reached

Cluster START

GENERIC

Pool used?

Fill until MAX_SIZE

reached

Assign to FREE

Server Pool

Fill GENERIC Pool

User Server

Pools?

Assign to FREE

Server Pool

Steps performed by OCR Master node

Y N

Y

N

2013 © Trivadis


Datum


29

RAC3 RAC4 RAC1 RAC2 RAC7 RAC8 RAC5 RAC6

Min 2; Max 3; Imp 5 Min 1; Max 3; Imp 2 Min 2; Max 3; Imp 8

2013 © Trivadis


Datum


30

RAC3 RAC4 RAC2 RAC7 RAC8 RAC5 RAC6

Min 2; Max 3; Imp 5

RAC1

Min 1; Max 3; Imp 2 Min 2; Max 3; Imp 8

2013 © Trivadis


Datum


31

RAC3 RAC4 RAC7 RAC8 RAC5 RAC6

Min 2; Max 3; Imp 5

RAC1 RAC2


2013 © Trivadis


Datum


32

RAC4 RAC7 RAC8 RAC5 RAC6

Min 2; Max 3; Imp 5

RAC3 RAC1 RAC2


2013 © Trivadis


Datum


33

RAC7 RAC8 RAC5 RAC6

Min 2; Max 3; Imp 5

RAC3 RAC4 RAC1 RAC2


2013 © Trivadis


Datum


34

RAC7 RAC8 RAC6

Min 2; Max 3; Imp 5

RAC3 RAC4 RAC1 RAC2


RAC5

2013 © Trivadis


Datum


35

RAC7 RAC8

Min 2; Max 3; Imp 5

RAC3 RAC4 RAC1 RAC2


RAC5

RAC6

2013 © Trivadis


Datum


36

RAC8

Min 2; Max 3; Imp 5

RAC3 RAC4 RAC1 RAC2


RAC5

RAC6 RAC7

2013 © Trivadis


Datum


37

Min 2; Max 3; Imp 5

RAC3 RAC4 RAC1 RAC2


RAC5

RAC6 RAC7

RAC8

2013 © Trivadis


Datum


38

Min 2; Max 3; Imp 5

RAC3 RAC4 RAC1 RAC2


RAC5

RAC6 RAC7

RAC8

2013 © Trivadis


Datum


39

Min 2; Max 3; Imp 5

RAC3 RAC4 RAC1

RAC2


RAC5

RAC6

RAC7

RAC8

2013 © Trivadis


Datum


40

Min 2; Max 3; Imp 5

RAC3 RAC4 RAC1

RAC2


RAC5

RAC6

RAC7

RAC8

2013 © Trivadis


Datum


41

Min 2; Max 3; Imp 5

RAC3 RAC4 RAC1

RAC2


RAC6

RAC7 RAC8

RAC5

2013 © Trivadis


Datum


42

Min 2; Max 3; Imp 5

RAC3 RAC4 RAC1

RAC2


RAC6

RAC7 RAC8

RAC5

2013 © Trivadis


Datum


43

Min 2; Max 3; Imp 5

RAC3 RAC4 RAC1

RAC2


RAC5 RAC6

RAC8

RAC7

2013 © Trivadis


Generic and Free pools have importance 0 by default

Servers cannot be claimed from Generic if Admin-managed databases

exist

After moving to PMD make sure that old admin-managed instances are

removed from the cluster

Datum


44

2013 © Trivadis

What-IF

CRSCTL: $ crsctl eval modify serverpool ora.crm -attr

"MIN_SIZE=4,MAX_SIZE=4" –f

[…]

Resource 'ora.hr.payroll.svc' (1/1) will be in

state [OFFLINE]

Server 'rac4' will be moved from pools [ora.hr]

to pools [ora.crm]

Resource 'ora.hr.db' (2/1) will be in state [OFFLINE]

Datum


45

2013 © Trivadis

What-IF

SRVCTL:

$ srvctl add serverpool -serverpool bi -min 5 -max 5 -

importance 10 –eval

Service payroll will be stopped on node rac4

Database hr will be stopped on node rac4

Server rac4 will be moved from pool hr to pool bi

Datum


46

2013 © Trivadis

Policy Sets

For monthly payroll, HR needs more resources

Datum


47

POLICY

NAME=monthly_payroll

SERVERPOOL

NAME=ora.hr

IMPORTANCE=10

MIN_SIZE=3

MAX_SIZE=3

SERVERPOOL

NAME=ora.crm

IMPORTANCE=5

MIN_SIZE=1

MAX_SIZE=1

2013 © Trivadis

Policy Sets

For nightly billing, CRM needs more resources

Datum


48

POLICY

NAME=nightly_billing

SERVERPOOL

NAME=ora.hr

IMPORTANCE=5

MIN_SIZE=1

MAX_SIZE=1

SERVERPOOL

NAME=ora.crm

IMPORTANCE=5

MIN_SIZE=3

MAX_SIZE=4

2013 © Trivadis

Policy Sets

Create the policy set file from running values $ crsctl create policyset -file cluster_policies.txt

Modify the policy set file with an external editor

Reload the policy set file into the cluster $ crsctl modify policyset -file cluster_policies.txt

Activate a policy $ crsctl modify policyset –attr \ "LAST_ACTIVATED_POLICY=monthly_payroll"

Datum


49

2013 © Trivadis

Policy Sets

“crsctl eval modify policyset” not implemented yet

Cluster managed policy sets are mutually exclusive with QOS policies

No integrated way to switch policy: use CRON instead

(or QOS!)

Datum


50

2013 © Trivadis

Child Server Pools

Divide Server pools in sub groups (child pools)

Cannot be created with srvctl, only crsctl

They do not work with policy-managed Databases!

You may try them and see funny errors!

Datum


51

2013 © Trivadis

Database behavior: admin managed

Datum


53

Instances assigned statically: less agility (better control?)

RAC3 RAC4 RAC1 RAC2

HR3 CRM4 CRM2

Pool “Generic”

CRM1

WEB3 HR4 WEB2 HR1

2013 © Trivadis

Database behavior: policy managed

Datum


54

RAC Databases always span the entire Serverpool

RAC3 RAC4 RAC1 RAC2

CRM_3 CRM_4 CRM_2

Pool “Mixed”

CRM_1

HR_3 HR_4 HR_2 HR_1

2013 © Trivadis

Database behavior

Datum


55

RAC One Node can be deployed without problems

RAC3 RAC4 RAC1 RAC2

RCV_1 WEB_1 BI_1

Pool “Mixed”

CRM_1

ABU_1 BCK_1 OE_1 HR_1

2013 © Trivadis

Database behavior

Datum


56

RAC/RAC One node in the same or different pools

RAC3 RAC4 RAC1 RAC2

RCV_1 WEB_1

Pool “multi_node”

ABU_1 BCK_1

CRM_2 CRM_1

HR_2 HR_1

Pool “one_node”

2013 © Trivadis

Database behavior

If you consolidate many databases, use Instance Caging to manage

CPU contention (CPU_COUNT)

If DB are concurring also for other resources, implement QOS

QOS needs Policy-management and it’s the Oracle way to manage

inter-database resources

For intra-database resources, Resource Mgr is the way

Multitenant consolidation + RM to manage PDB resources

Datum


57

2013 © Trivadis

Services

Datum


58

SINGLETON: assigned dynamically to a single node

UNIFORM: assigned to every instance in the server pool

(not compatible with QOS)

srvctl add service -db CRM -service SALES \

-cardinality UNIFORM

RAC3 RAC4 RAC1 RAC2

CRM_3 CRM_4 CRM_2

Pool “Mixed”

CRM_1

SALES

REPORTING FIN_READ

2013 © Trivadis

Services

Databases can be configured to span several pools

$ srvctl modify database -db hr -serverpool "hr,crm"

But services can run only in ONE database and ONE pool!

$ srvctl modify service -service payroll -db hr \

-serverpool "hr,crm“

PRKO-2003 : Invalid command line option value: hr,crm

Use multiple services in order to reach all instances if the DB spans

multiple pools

Multiple services: use Resource Manager!

Datum


59

2013 © Trivadis

Crsctl output may be misleading!

Instances are volatile with PMD

Instances may be reported OFFLINE by crsctl or EM12c, that’s expected

(it doesn’t mean I like it!)

This happens after pool shrink or RAC One relocations

Count the ONLINE instances instead, and expect to be greater or equal

than serverpool’s min_size

Datum


62

2013 © Trivadis

Centralize the diagnostic_dest!

With local FS, you may not find the right instance, at the right time, on

the right host

Use a CFS, e.g. ACFS if it’s available for your platform

ACFS? Then take care of resource dependencies

Datum


63

2013 © Trivadis

We love it!!

Server failures do not affect critical services

Very fast instance provisioning

Proven agility and scaling

Easy administration

Requirement for QOS and GDS (as of 12.1.0.1)

Oracle recommends it!

Datum


65

vistaICO.com

2013 © Trivadis

well… not always!

Some combinations of features don’t work

Can’t choose different cardinalities for services or DBs

Stopped instances reported OFFLINE when pools shrink!

EM12c is not always reactive when pools resize

Other features need attention when pools expand (GG, DG)

Datum


66

vistaICO.com

2013 © Trivadis

BASEL BERN BRUGG LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN

2013 © Trivadis

Thank you!

Ludovico Caldara

Oracle ACE, Senior Consultant

Trivadis AG

[email protected]

Datum


boost your oracle rac manageability with policy-managed databases

Technology