provisioning oracle rac in a virtualized environment, using oracle

Provisioning Oracle RAC in a

Virtualized Environment, Using Oracle

Enterprise Manager

Oracle OpenWorld 2009, Session #S312109 Kai Yu Rajat Nigam Sr. System Engineer Consultant Senior Product Manager Dell Global Solutions Engineering Oracle Corporation

Introduction to Oracle Enterprise Manager and Datacenter Automation

Dell Oracle Grid Infrastructure Project

Oracle Virtualization Infrastructure Implementation

Provisioning Oracle 11g RAC using Oracle Enterprise Manager

Extending Oracle 11g RAC using Oracle Enterprise Manager

Oracle Provisioning & Virtualization Products Road Map

QA

AGENDA

2

Kai Yu

Senior System Engineer, Dell Oracle Solutions Lab

– 14 years Oracle DBA and Solutions Engineering

– Specialized in Oracle RAC, Oracle EBS and OVM

– Oracle Technology article author and frequent presenter at OOW 06/07/08/09 and Collaborate 08/09

– IOUG Oracle RAC SIG President

Rajat Nigam

Senior Product Manager, Oracle Corporation

– Focus areas

– Enterprise Manager’s Datacenter Automation solution–Provisioning, Patching, Upgrade, Migration

– End user experience of Enterprise Manager install and upgrade

ABOUT AUTHORS

3

INTRODUCTION

4

Oracle Enterprise Manager

– Overview

– Datacenter Automation (Provisioning and Patching) Overview

– Oracle Database and RAC Provisioning and Patching

– Oracle VM Provisioning and Patching

5

ORACLE ENTERPRISE MANAGERTOP-DOWN, INTEGRATED APPLICATION MANAGEMENT

6

COMPLETE, OPEN AND INTEGRATED MANAGEMENT

7

SOFTWARE DEPLOYMENT LIFECYCLE MANAGEMENT

8

DATABASE PROVISIONING

9

RAC PROVISIONING

10

CRITICAL PATCH ADVISORY

11

DATABASE PATCHINGMASS DEPLOY UPDATES ACROSS PLATFORMS AND VERSIONS

12

RAC ROLLING PATCHINGZERO DOWNTIME PATCHING

ORACLE VM SERVER PROVISIONINGAUTOMATED PROVISIONING ON BARE METAL MACHINES

13

ORACLE VM TEMPLATES PRE-CONFIGURED SOFTWARE DEPLOYMENT IN MINUTES

14

PATCHING GUEST VIRTUAL MACHINES

15

16

EXTENSIBILITY

17

ORACLE DEEP DIVE SESSIONS

1) S308168 – That was Easy! Push Button Database

Deployment and Patching

Tuesday 13th Oct, 11:30 am – 12:30 pm

Moscone South, Room 305

2) S308071 – Integrated Physical and Virtual System

Management with Oracle Enterprise Manager

Wednesday 14th Oct, 10:15 am – 11:15 am


3) S307465 – Manage Your Virtualized Environment with

Oracle VM Management Pack

Thursday 15th Oct, 10:30 am – 11:30 am


ORACLE GRID INFRASTRUCTURE PROJECT

18

Grid Infrastructure Project

– POC Project Goal:

a. Expand the Grid to include virtualization

b. Implement the Grid on Commodity hardware and SAN storage

c. Grid control as the unified management for the grid

Oracle Grid Architecture

– Grid : Physical Grid and Virtual Grid

– Grid based on Commodity Hardware Dell Blade Servers

– Use EqualLogic iSCSI SAN for shared storage

– Refer to #S308185 for Grid infrastructure and implementation

– Focus of this presentation:

a. Establish the VM infrastructure for running 11g RAC on VM

b. Provision 11g RAC on VMs using Enterprise Manager Grid Control

VIRTUAL GRID DESIGN

19

Virtual Grid is based on Oracle VM Technology

– Virtual Server: Xen Hypervisor, management domain dom0 Linux kernel with support of devices, IO, networking, etc.

– Virtual Server Pool: an autonomous region of VM servers

– VM: guest OS with applications running on domU

VIRTUAL GRID DESIGN

20

Virtual Grid Components:

– Virtual Servers: Dell blade servers

– Shared storage: Dell EqualLogic iSCSI SAN

– Unified Management: Grid Control with VM Management

ORACLE VIRTUALIZATIONINFRASTRUCTURE IMPLEMENTATION

21

Dell Blade servers as VM server platform

– Blade servers offer the great advantages for data center

– Blade Modular and enclosure

– Blade servers: the latest 11g blades M610, M710

Dell EqualLogic iSCSI SAN as the shared storage– PS6000 XV


22

Oracle VM server Installation– Enable virtualization for CPUs on BIOS

– Install Oracle VM server 2.1.2

– Configure dom0 memory by editing /boot/grub/menu.lst

– Secure OVM agent is running: service ovs-agent status

Oracle VM server network configuration

– NICs in Dom0 Xen bridges virtual NICs in DomU


23

– Customize the default Xen bridge configuration

a. Disable the default Xen bridge settings.

b. Create ifcfg-ethn, incfg-bond0, ifcfg-xenbr0, ifcfg-xenbr1 scripts

c. Restart the network services

Oracle VM server shared storage configuration

– Shared storage partitions added into OVM repositories

/dev/mapper/ovs_data4p1; /dev/mapper/ovs_data6p1

– Additional disk partitions for guest VMs through physical disks mappings for RAC shared disks

2 X OCRs, 3 X Votingdisks, ASM diskgroup

– Detailed implementation referred to #S308185


24

Grid Control Management Infrastructure – Install EM Grid control 10.2.0.3,Upgraded to 10.2.0.5

– Enabled Virtual Management Pack

– Detailed implementation referred to #S308185

Connect the VM servers to Grid Control– Pre-configure steps on VM servers

– Create VM server pool with the VM server


25

Create Guest VMs using the VM template– Download/Register OVM_EL5U2_X86_64_11GRAC_PVM

– Create VMs from the template

– Present xenbr0 and xenbr1 as eth0 (public) and eth1(private)

vm.cfg: vif = ['bridge=xenbr0,mac=00:16:3E:11:8E:CE,type=netfront',

'bridge=xenbr1,mac=00:16:3E:50:63:25,type=netfront', ]

– Add the additional disk and shared storages to the VMsa. Attached the shared disks from VM repositories


26

vm.cfg: disk = [''file:/OVS/sharedDisk/racdb.img,xvdc,w!',

b. Attached the physical partitions to the guest VM

vm.cfg: disk = ['‘phy: /dev/mapper/vmracdbp1, xvdc,w!',

PROVISION ORACLE 11G RAC USING ORACLE ENTERPRISE MANAGER

27

Pre-requisite steps

– Install/configure Grid control Agent on RAC node VMsa. Grid control agent 10.2.5 installed on two guest VMs

b. Disable firewall services

c. Check host and cluster preferred credentials of Oracle users on

the guest VMs


28

– Setup software library: DeploymentProvisioning Administration:

Software Library configuration

– Load the Oracle RAC shipped software to the software libraryDownload 11g clusterware/11g RAC software the guest VM host

bvmrac1(155.16.9.101); DeploymentComponents


29

select Oracle Database shiphome and click edit button, pick source to

upload it. Follow the same step to load Oracle clusterware

shiphome


30

Prepare guest VM nodes for RAC provisioning

– Install/configure Grid control Agent on RAC node VMs

– Prepare shared devices: 2 X OCR and 3 X votingdisk, ASM disk

– /udev/rules.d/65-raw.rules and 89-raw.rules: map raw devices

– Change kernel parameters for RAC install

– Extend OS partition by adding another 20 GB disks using LVM

– Prepare /etc/hosts to include public, private, VIPs: hostnames/IPs


31

Provision Oracle RAC on guest VM

– Start the provisioning procedure: DeploymentProvisioning

Deployment procedure ManagerRAC Provisioning Procedure

Click Schedule deployment and specify the source of software


32

Select

Target

Host:


33

Target: clusterware/ASM RAC Database


34

Target: Storage for RAC

Submit the provision job and check the job status


35


36

Verify the RAC database configuration

– Single node RAC database vmracdb

EXTEND ORACLE 11G RAC USING ORACLE ENTERPRISE MANAGER

37

– Prepare the guest VM for the new node

a. Create a new guest VM

b. Same network, disks, kernel settings, rpms. disable firewall

c. Install Grid control agent 10.2.0.5

d. Add host and cluster preferred credentials of Oracle users

– Schedule the provisioning procedure to extend the RAC

DeploymentProvisioning Deployment Procedure Manager

RAC Provisioning Procedure

Select “One Click Extend Cluster Database”, click “Schedule Deployment” select the RAC database to extend


38


39

Specify the new node to add and the user credentials

Submit the job


40

Check the job status: procedure completed successfully


41


42

Verify the RAC extension results:– Check clusterware status:

[root@bvmrac2 bin]# ./crsctl check crs

Cluster Synchronization Services appears healthy

Cluster Ready Services appears healthy

Event Manager appears healthy

– Check database service:[oracle@bvmrac2 bin]$ srvctl status database -d vmracdb

Instance vmracdb1 is running on node bvmrac1

Instance vmracdb2 is running on node bvmrac2

SUMMARY

43

• Dell Grid POC Project: Expand Grid to include virtualization

• Grid control as the unified management solution for the Grid

• Use Grid control to manage the virtual servers

• Use Grid Control Provisioning pack to provision 11g RAC on VMs

and extent the RAC to additional VM node.

• Acknowledge the support of Oracle engineers: Akanksha Sheoran, Daniel Dibbets, Kurt Hackel, Channabasappa Nekar, Premjith Rayaroth, and Dell Engineer: Roger Lopez

• Related OpenWorld Presentations:

– ID#: S308185, Building an Oracle Grid with Oracle VM on Blade Servers and iSCSI Storage, Wed. 10/14/2009 17:00 - 18:00, Kai Yu & David Mar

– ID#: S310132, Oracle E-Business Suite on Oracle RAC and Oracle VM: Architecture and Implementation, Wed.10/14/09 10:15 -11:15 Kai Yu and John Tao

• Dell Oracle Whitepapers: www.dell.com/oracle

provisioning oracle rac in a virtualized environment, using oracle

Documents