rac checklist 11gr2 grid infrastructure for clusters
DESCRIPTION
AC-checklist-11gR2-grid-infrastructure-for-clustersTRANSCRIPT
11gR2 Grid Infrastructure for Clusters
Please note that this cheat sheet is not meant to substitute the official Oracle 11gR2 documentation. If you need more information about 11gR2 you should consult the Oracle 11gR2 documentation.
Table of Contents1)Setup system related tasks............................................................................................................ 2
Create GI group and user etc.......................................................................................................2Setup swap and system limits etc................................................................................................2Setup /etc/hosts for Single Client Access Name (SCAN)...........................................................2Create Grid Infrastructure directories etc....................................................................................3Setup SSH................................................................................................................................... 3Create ASM candidate disks etc..................................................................................................5
2)Prepare for a Grid Infrastructure for Clusters installation............................................................7Install ORCLudlm (SPARC only)...............................................................................................7Install RAC framework resources...............................................................................................7Setup VNC.................................................................................................................................. 7
3)Install Grid Infrastructure for Clusters via a VNC client............................................................. 8Install and configure Grid Infrastructure for Clusters................................................................. 8Display the Grid Infrastructure resources................................................................................. 17Create another ASM disk group................................................................................................18
4)Install and create an 11gR1 database via a VNC client .............................................................19Install the 11gR1 database software..........................................................................................19Create the 11gR1 database using dbca...................................................................................... 21Create the 11gR1 database listener using netca........................................................................ 22Copy some 11gR1 database files to the another node............................................................... 24Verify the 11gR1 database installation......................................................................................26Check the Grid Infrastructure resources................................................................................... 28
5)Install and create an 11gR2 database via a VNC client .............................................................29Install the 11gR2 database software..........................................................................................29Create the 11gR2 database using dbca...................................................................................... 30Copy some 11gR2 database files to the another node............................................................... 31Verify the 11gR2 database installation......................................................................................33Check the Grid Infrastructure resources................................................................................... 34
6)Using SCAN for the 11gR2 single instance database................................................................ 357)Place everything under Oracle Solaris Cluster ......................................................................... 36
Register the resource types........................................................................................................36Register and enable the clustered ASM instance resource........................................................37Register and enable the clustered ASM disk group resource....................................................38Register and enable the 11gR1 database and listener resources .............................................. 38Switch the 11gR1 Resource Group........................................................................................... 40Register and enable the 11gR2 database resource ....................................................................41
Switch the 11gR2 Resource Group...................................................................................... 44Final picture ............................................................................................................................. 47
1
1) Setup system related tasks
Create GI group and user etc.
bash-3.00# vi /etc/nsswitch.conf (remove nis from passwd: and group:)
bash-3.00# groupadd -g 300 oinstallbash-3.00# groupadd -g 301 dba bash-3.00# useradd -g 300 -G 301 -u 300 -d /export/home/oragrid -s /usr/bin/bash oragrid
bash-3.00# mkdir -p /export/home/oragridbash-3.00# chown oragrid:oinstall /export/home/oragrid
bash-3.00# passwd oragridNew Password: Re-enter new Password: passwd: password successfully changed for oragridbash-3.00#
Setup swap and system limits etc.
bash-3.00# mkfile 8g /swapfilebash-3.00# swap -a /swapfile
Add to /etc/vfstab
bash-3.00# cat /etc/vfstab | grep swapfile/swapfile - - swap - no -bash-3.00#
Copy the following entries into /etc/project
bash-3.00# cat /etc/projectsystem:0::::process.max-sem-nsems=(privileged,2048,deny);project.max-shm-memory=(privileged,2199023255552,deny)user.root:1::::noproject:2::::default:3::::process.max-file-descriptor=(priv,65536,deny);project.max-shm-memory=(priv,4294967296,deny);project.max-user-process=(basic,16384,deny)group.staff:10::::bash-3.00#
Setup /etc/hosts for Single Client Access Name (SCAN)
Here, freak-scan-lh will be used as the SCAN name and freak-1 and freak-2 are VIPs for each cluster node that SCAN will use. You will need to determine which IP addresses are free to use for your cluster. bash-3.00# cat /etc/hosts## Internet host table
2
#::1 localhost127.0.0.1 localhost10.11.188.56 pfreak1 loghost10.11.188.57 pfreak2 # Cluster Node10.11.188.60 vzfreak1a freak-scan-lh10.11.188.58 freak-1 10.11.188.59 freak-2bash-3.00#
Create Grid Infrastructure directories etc.
bash-3.00# mkdir -p /u01/app/oragridbash-3.00# mkdir -p /u01/app/oraInventorybash-3.00# mkdir -p /u01/app/11.2.0/grid
bash-3.00# chown -R oragrid:oinstall /u01bash-3.00# chmod -R 775 /u01
Setup SSH.
On pfreak1:
bash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005-bash-3.00$ mkdir .ssh-bash-3.00$ chmod 700 .ssh-bash-3.00$ cd .ssh
On pfreak2:
bash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005-bash-3.00$ mkdir .ssh-bash-3.00$ chmod 700 .ssh-bash-3.00$ cd .ssh
On pfreak1:
Note: we will also NOT specify a passphrase.
-bash-3.00$ ssh-keygen -t rsaGenerating public/private rsa key pair.Enter file in which to save the key (/export/home/oragrid/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /export/home/oragrid/.ssh/id_rsa.Your public key has been saved in /export/home/oragrid/.ssh/id_rsa.pub.The key fingerprint is:e6:63:c9:71:fe:d1:8f:71:77:70:97:25:2a:ee:a9:33 [email protected]$-bash-3.00$ pwd/export/home/oragrid/.ssh
3
-bash-3.00$ cat id_rsa.pub >> authorized_keys-bash-3.00$ chmod 600 authorized_keys-bash-3.00$ scp authorized_keys pfreak2:/export/home/oragrid/.sshThe authenticity of host 'pfreak2 (10.11.188.57)' can't be established.RSA key fingerprint is 4d:d3:84:72:44:fe:0c:48:a7:76:fb:7d:65:c4:da:a9.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'pfreak2,10.11.188.57' (RSA) to the list of known hosts.Password: authorized_keys 100% |*****************************************************************************************| 223 00:00 -bash-3.00$
On pfreak2:
Note: we will also NOT specify a passphrase.
-bash-3.00$ ssh-keygen -t rsaGenerating public/private rsa key pair.Enter file in which to save the key (/export/home/oragrid/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /export/home/oragrid/.ssh/id_rsa.Your public key has been saved in /export/home/oragrid/.ssh/id_rsa.pub.The key fingerprint is:be:8f:e6:14:ff:ff:06:b7:c7:64:1b:44:3b:d3:5a:e8 [email protected]$-bash-3.00$ pwd/export/home/oragrid/.ssh-bash-3.00$ cat id_rsa.pub >> authorized_keys-bash-3.00$ chmod 600 authorized_keys-bash-3.00$ scp authorized_keys pfreak1:/export/home/oragrid/.sshThe authenticity of host 'pfreak1 (10.11.188.56)' can't be established.RSA key fingerprint is b8:46:db:d0:88:ed:5c:33:b5:b0:d0:f1:d6:be:a0:38.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'pfreak1,10.11.188.56' (RSA) to the list of known hosts.Password: authorized_keys 100% |*****************************************************************************************| 446 00:00 -bash-3.00$
Test ssh has been setup correctly.
On pfreak1:
-bash-3.00$ ssh pfreak1 dateThe authenticity of host 'pfreak1 (10.11.188.56)' can't be established.RSA key fingerprint is b8:46:db:d0:88:ed:5c:33:b5:b0:d0:f1:d6:be:a0:38.
4
Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'pfreak1,10.11.188.56' (RSA) to the list of known hosts.Fri May 14 07:30:55 PDT 2010-bash-3.00$ -bash-3.00$ ssh pfreak1 dateFri May 14 07:30:57 PDT 2010-bash-3.00$ ssh pfreak2 dateFri May 14 07:31:02 PDT 2010-bash-3.00$
On pfreak2:
-bash-3.00$ ssh pfreak2 dateThe authenticity of host 'pfreak2 (10.11.188.56)' can't be established.RSA key fingerprint is dc:6e:af:36:f8:c7:74:b1:c6:b1:1c:96:e3:da:11:51.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'pfreak2,10.11.188.56' (RSA) to the list of known hosts.Fri May 14 07:33:02 PDT 2010-bash-3.00$ -bash-3.00$ ssh pfreak2 dateFri May 14 07:33:04 PDT 2010-bash-3.00$ ssh pfreak1 dateFri May 14 07:33:08 PDT 2010-bash-3.00$
Create ASM candidate disks etc.
bash-3.00# cldev list -vDID Device Full Device Path---------- ----------------d1 pfreak1:/dev/rdsk/c3t500000E0183E1140d0d2 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Fd0d2 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Fd0d3 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Ed0d3 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Ed0d4 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Dd0d4 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Dd0d5 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Cd0d5 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Cd0d6 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Bd0d6 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Bd0d7 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Ad0d7 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E60Ad0d8 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E609d0d8 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E609d0d9 pfreak1:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E608d0d9 pfreak2:/dev/rdsk/c3t600C0FF0000000000928FF1E6623E608d0d10 pfreak2:/dev/rdsk/c3t500000E016AE6B70d0bash-3.00# bash-3.00# for i in 2 3 4 5 6 7 8 9
5
> do> dd if=/dev/zero of=/dev/did/rdsk/d${i}s0 bs=1025k count=200> chown oragrid:oinstall /dev/did/rdsk/d${i}s0> chmod 660 /dev/did/rdsk/d${i}s0> ls -lHL /dev/did/rdsk/d${i}s0> done200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 64 May 17 03:23 /dev/did/rdsk/d2s0200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 96 May 17 03:23 /dev/did/rdsk/d3s0200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 128 May 17 03:23 /dev/did/rdsk/d4s0200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 160 May 17 03:23 /dev/did/rdsk/d5s0200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 192 May 17 03:23 /dev/did/rdsk/d6s0200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 224 May 17 03:23 /dev/did/rdsk/d7s0200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 256 May 17 03:23 /dev/did/rdsk/d8s0200+0 records in200+0 records outcrw-rw---- 1 oragrid oinstall 239, 288 May 17 03:23 /dev/did/rdsk/d9s0bash-3.00#
On pfreak2.
bash-3.00# for i in 2 3 4 5 6 7 8 9> do> chown oragrid:oinstall /dev/did/rdsk/d${i}s0> chmod 660 /dev/did/rdsk/d${i}s0> ls -lHL /dev/did/rdsk/d${i}s0> donecrw-rw---- 1 oragrid oinstall 239, 64 May 17 03:24 /dev/did/rdsk/d2s0crw-rw---- 1 oragrid oinstall 239, 96 May 17 03:24 /dev/did/rdsk/d3s0crw-rw---- 1 oragrid oinstall 239, 128 May 17 03:24 /dev/did/rdsk/d4s0crw-rw---- 1 oragrid oinstall 239, 160 May 17 03:24 /dev/did/rdsk/d5s0crw-rw---- 1 oragrid oinstall 239, 192 May 17 03:24 /dev/did/rdsk/d6s0crw-rw---- 1 oragrid oinstall 239, 224 May 17 03:24 /dev/did/rdsk/d7s0crw-rw---- 1 oragrid oinstall 239, 256 May 17 03:24 /dev/did/rdsk/d8s0crw-rw---- 1 oragrid oinstall 239, 288 May 17 03:24 /dev/did/rdsk/d9s0bash-3.00#
6
2) Prepare for a Grid Infrastructure for Clusters installation
Install ORCLudlm (SPARC only)
bash-3.00# pwd/net/dv5/swlib/oracle/released/db/11gR2/11.2.0.1/sparc-sunos/64bit-installable/grid/udlmbash-3.00# ls -ltotal 1816-rw-r--r-- 1 root root 899375 Nov 6 2009 ORCLudlm.tar.Z-rw-r--r-- 1 root root 7493 Nov 6 2009 README.udlm-rw-r--r-- 1 root root 770 Nov 6 2009 release_notes.33410bash-3.00# cp ORCLudlm.tar.Z /var/tmpbash-3.00# cd /var/tmpbash-3.00# uncompress ORCLudlm.tar.Zbash-3.00# tar -xf ORCLudlm.tarbash-3.00# pkgadd -d . ORCLudlm
Install RAC framework resources
bash-3.00# clrt listSUNW.LogicalHostname:3SUNW.SharedAddress:2bash-3.00# bash-3.00# clrt register SUNW.rac_frameworkbash-3.00# clrt register SUNW.rac_udlm bash-3.00# clrg create -S rac-fmwk-rgbash-3.00# clrs create -g rac-fmwk-rg -t SUNW.rac_framework rac-fmwk-rsbash-3.00# clrs create -g rac-fmwk-rg -t SUNW.rac_udlm -p resource_dependencies=rac-fmwk-rs rac-udlm-rsbash-3.00# clrg online -eM rac-fmwk-rgbash-3.00# clrs status
=== Cluster Resources ===
Resource Name Node Name State Status Message------------- --------- ----- --------------rac-fmwk-rs pfreak1 Online Online pfreak2 Online Online
rac-udlm-rs pfreak1 Online Online pfreak2 Online Online
bash-3.00#
Setup VNC
bash-3.00# cd; vi .profilePATH=${PATH}:/usr/cluster/bin:/usr/X11/bin:/usr/openwin/bin; export PATH
MANPATH=/usr/man:/usr/cluster/man; export MANPATHbash-3.00# . .profile
7
bash-3.00# vncserver :10
You will require a password to access your desktops.
Password:Verify:xauth: creating new authority file //.Xauthority
New 'pfreak1:10 ()' desktop is pfreak1:10
Creating default startup script //.vnc/xstartupStarting applications specified in //.vnc/xstartupLog file is //.vnc/pfreak1:10.log
bash-3.00#
3) Install Grid Infrastructure for Clusters via a VNC client
Install and configure Grid Infrastructure for Clusters
bash-3.00# xhost +bash-3.00# su - oragrid-bash-3.00$ export DISPLAY=:10.0-bash-3.00$ cd /net/dv5/swlib/oracle/released/db/11gR2/11.2.0.1/sparc*/64*/grid-bash-3.00$ ./runInstaller
Select Installation Option • Select Install and Configure Grid Infrastructure for a Cluster Server• Click Next
Select Installation Type• Select Advanced Installation• Click Next
Select Product Languages Type• Click Next
Grid Plug and Play Information• Cluster Name – freak-cluster• SCAN Name – freak-scan-lh• SCAN Port - 1521• Click Next
Cluster Node Information• Edit Virtual IP Name• pfreak1-vip to freak-1• pfreak2-vip to freak-2• Click Next
Specify Network Interface Usage• Ensure main IPMP NIC is public• Ensure clprivnet0 is private• Click Next
Storage Option Information• Click ASM
8
• Click NextCreate ASM Disk Group
• Disk group Name – your choice (CRS)• Select Redundancy – External• Select Candidate Disks • Click Change Discovery Path• Enter /dev/did/rdsk/d* - OK• May take a few minutes to discover all the candidate disks• Select two disks • Click Next
Specify ASM Password• Use same password – manager (click yes to ignore warning)
• Click NextSpecify Operating System Groups
• ASM OSDBA, OSOPER and OSASM are oinstall• Click Next• Click Yes (OSDBA, OSOPER and OSASM are in the same OS Group)
Specify Installation Location• Oracle Base - /u01/app/oragrid• Software Location - /u01/app/11.2.0/grid• Click Next
Create Inventory• Inventory Directory - /u01/app/oraInventory• Click Next
Perform Prerequisite Checks• Ignore max open file descriptors• Ignore Compiler patch 124861-15• Ignore users with same UID• Resolve other issues• Click Ignore all• Click Next
Summary• Click Finish
Execute Configuration Scripts• Click OK once done
Note, please see below, i.e. after the root scripts output for a continuation of the installation.
pfreak1 scripts
bash-3.00# pwd/u01/app/oraInventorybash-3.00#bash-3.00# ./orainstRoot.shChanging permissions of /u01/app/oraInventory.Adding read,write permissions for group.Removing read,write,execute permissions for world.
Changing groupname of /u01/app/oraInventory to oinstall.The execution of the script is complete.
9
bash-3.00# pwd/u01/app/oraInventorybash-3.00#
Repeat above for pfreak2. Then on pfreak1,
bash-3.00# ./root.shRunning Oracle 11g root.sh script...
The following environment variables are set as: ORACLE_OWNER= oragrid ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]: Creating /usr/local/bin directory... Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ...
Creating /var/opt/oracle/oratab file...Entries will be added to the /var/opt/oracle/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root.sh script.Now product-specific root actions will be performed.2010-05-17 05:09:57: Parsing the host name2010-05-17 05:09:57: Checking for super user privileges2010-05-17 05:09:57: User has super user privilegesUsing configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_paramsCreating trace directoryLOCAL ADD MODE Creating OCR keys for user 'root', privgrp 'root'..Operation successful. root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP
10
profile reader pa cert TP profile reader peer cert TP peer user cert pa user certAdding daemon to inittabCRS-4123: Oracle High Availability Services has been started.ohasd is startingCRS-2672: Attempting to start 'ora.gipcd' on 'pfreak1'CRS-2672: Attempting to start 'ora.mdnsd' on 'pfreak1'CRS-2676: Start of 'ora.gipcd' on 'pfreak1' succeededCRS-2676: Start of 'ora.mdnsd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.gpnpd' on 'pfreak1'CRS-2676: Start of 'ora.gpnpd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.cssdmonitor' on 'pfreak1'CRS-2676: Start of 'ora.cssdmonitor' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.cssd' on 'pfreak1'CRS-2672: Attempting to start 'ora.diskmon' on 'pfreak1'CRS-2676: Start of 'ora.diskmon' on 'pfreak1' succeededCRS-2676: Start of 'ora.cssd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.ctssd' on 'pfreak1'CRS-2676: Start of 'ora.ctssd' on 'pfreak1' succeeded
ASM created and started successfully.
DiskGroup CRS created successfully.
clscfg: -install mode specifiedSuccessfully accumulated necessary OCR keys.Creating OCR keys for user 'root', privgrp 'root'..Operation successful.CRS-2672: Attempting to start 'ora.crsd' on 'pfreak1'CRS-2676: Start of 'ora.crsd' on 'pfreak1' succeededCRS-4256: Updating the profileSuccessful addition of voting disk 6f0f679d5b804f04bf7433abb9133ccf.Successfully replaced voting disk group with +CRS.CRS-4256: Updating the profileCRS-4266: Voting file(s) successfully replaced## STATE File Universal Id File Name Disk group-- ----- ----------------- --------- --------- 1. ONLINE 6f0f679d5b804f04bf7433abb9133ccf (/dev/did/rdsk/d2s0) [CRS]Located 1 voting disk(s).CRS-2673: Attempting to stop 'ora.crsd' on 'pfreak1'CRS-2677: Stop of 'ora.crsd' on 'pfreak1' succeededCRS-2673: Attempting to stop 'ora.asm' on 'pfreak1'CRS-2677: Stop of 'ora.asm' on 'pfreak1' succeededCRS-2673: Attempting to stop 'ora.ctssd' on 'pfreak1'CRS-2677: Stop of 'ora.ctssd' on 'pfreak1' succeededCRS-2673: Attempting to stop 'ora.cssdmonitor' on 'pfreak1'CRS-2677: Stop of 'ora.cssdmonitor' on 'pfreak1' succeededCRS-2673: Attempting to stop 'ora.cssd' on 'pfreak1'CRS-2677: Stop of 'ora.cssd' on 'pfreak1' succeeded
11
CRS-2673: Attempting to stop 'ora.gpnpd' on 'pfreak1'CRS-2677: Stop of 'ora.gpnpd' on 'pfreak1' succeededCRS-2679: Attempting to clean 'ora.gpnpd' on 'pfreak1'CRS-2681: Clean of 'ora.gpnpd' on 'pfreak1' succeededCRS-2673: Attempting to stop 'ora.gipcd' on 'pfreak1'CRS-2677: Stop of 'ora.gipcd' on 'pfreak1' succeededCRS-2673: Attempting to stop 'ora.mdnsd' on 'pfreak1'CRS-2677: Stop of 'ora.mdnsd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.mdnsd' on 'pfreak1'CRS-2676: Start of 'ora.mdnsd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.gipcd' on 'pfreak1'CRS-2676: Start of 'ora.gipcd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.gpnpd' on 'pfreak1'CRS-2676: Start of 'ora.gpnpd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.cssdmonitor' on 'pfreak1'CRS-2676: Start of 'ora.cssdmonitor' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.cssd' on 'pfreak1'CRS-2672: Attempting to start 'ora.diskmon' on 'pfreak1'CRS-2676: Start of 'ora.diskmon' on 'pfreak1' succeededCRS-2676: Start of 'ora.cssd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.ctssd' on 'pfreak1'CRS-2676: Start of 'ora.ctssd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.asm' on 'pfreak1'CRS-2676: Start of 'ora.asm' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.crsd' on 'pfreak1'CRS-2676: Start of 'ora.crsd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.evmd' on 'pfreak1'CRS-2676: Start of 'ora.evmd' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.asm' on 'pfreak1'CRS-2676: Start of 'ora.asm' on 'pfreak1' succeededCRS-2672: Attempting to start 'ora.CRS.dg' on 'pfreak1'CRS-2676: Start of 'ora.CRS.dg' on 'pfreak1' succeeded
pfreak1 2010/05/17 05:18:24 /u01/app/11.2.0/grid/cdata/pfreak1/backup_20100517_051824.olrConfigure Oracle Grid Infrastructure for a Cluster ... succeededUpdating inventory properties for clusterwareStarting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB. Actual 19161 MB PassedThe inventory pointer is located at /var/opt/oracle/oraInst.locThe inventory is located at /u01/app/oraInventory'UpdateNodeList' was successful.Bash-3.00#
Finally, on pfreak2.
bash-3.00# ./root.shRunning Oracle 11g root.sh script...
The following environment variables are set as:
12
ORACLE_OWNER= oragrid ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]: Creating /usr/local/bin directory... Copying dbhome to /usr/local/bin ... Copying oraenv to /usr/local/bin ... Copying coraenv to /usr/local/bin ...
Creating /var/opt/oracle/oratab file...Entries will be added to the /var/opt/oracle/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root.sh script.Now product-specific root actions will be performed.2010-05-17 05:34:28: Parsing the host name2010-05-17 05:34:28: Checking for super user privileges2010-05-17 05:34:28: User has super user privilegesUsing configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_paramsCreating trace directoryLOCAL ADD MODE Creating OCR keys for user 'root', privgrp 'root'..Operation successful.Adding daemon to inittabCRS-4123: Oracle High Availability Services has been started.ohasd is startingCRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node pfreak1, number 1, and is terminatingAn active cluster was found during exclusive startup, restarting to join the clusterCRS-2672: Attempting to start 'ora.mdnsd' on 'pfreak2'CRS-2676: Start of 'ora.mdnsd' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.gipcd' on 'pfreak2'CRS-2676: Start of 'ora.gipcd' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.gpnpd' on 'pfreak2'CRS-2676: Start of 'ora.gpnpd' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.cssdmonitor' on 'pfreak2'CRS-2676: Start of 'ora.cssdmonitor' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.cssd' on 'pfreak2'CRS-2672: Attempting to start 'ora.diskmon' on 'pfreak2'CRS-2676: Start of 'ora.diskmon' on 'pfreak2' succeededCRS-2676: Start of 'ora.cssd' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.ctssd' on 'pfreak2'CRS-2676: Start of 'ora.ctssd' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.asm' on 'pfreak2'CRS-2676: Start of 'ora.asm' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.crsd' on 'pfreak2'CRS-2676: Start of 'ora.crsd' on 'pfreak2' succeededCRS-2672: Attempting to start 'ora.evmd' on 'pfreak2'CRS-2676: Start of 'ora.evmd' on 'pfreak2' succeeded
13
pfreak2 2010/05/17 05:38:36 /u01/app/11.2.0/grid/cdata/pfreak2/backup_20100517_053836.olrConfigure Oracle Grid Infrastructure for a Cluster ... succeededUpdating inventory properties for clusterwareStarting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB. Actual 19148 MB PassedThe inventory pointer is located at /var/opt/oracle/oraInst.locThe inventory is located at /u01/app/oraInventory'UpdateNodeList' was successful.bash-3.00#
After running the root scripts on all node Click OK.
The following tasks then failed:
– Configure Oracle Grid Infrastructure for a Cluster.– Oracle Cluster Verification Utility.
Looking at the installation log, the following errors are seen:
bash-3.00# pwd/u01/app/oraInventory/logsbash-3.00# bash-3.00# tail -200f installActions2010-05-17_04-12-47AM.log
INFO: Checking name resolution setup for "freak-scan-lh"...INFO: ERROR: INFO: PRVF-4664 : Found inconsistent name resolution entries for SCAN name "freak-scan-lh"INFO: ERROR: INFO: PRVF-4657 : Name resolution setup check for "freak-scan-lh" (IP address: 10.11.188.60) failedINFO: ERROR: INFO: PRVF-4663 : Found configuration issue with the 'hosts' entry in the /etc/nsswitch.conf file
Searching metalink for the above, resulting in Metalink note 887471.1, the solution is to verify that “ping freak-scan-lh” works and if it does then ignore the above error.
bash-3.00# ping freak-scan-lhfreak-scan-lh is alivebash-3.00#
INFO: NTP Configuration file check started...INFO: ERROR: INFO: PRVF-5402 : Warning: Could not find NTP configuration file "/etc/inet/ntp.conf" on node "pfreak2"INFO: ERROR:
14
INFO: PRVF-5402 : Warning: Could not find NTP configuration file "/etc/inet/ntp.conf" on node "pfreak1"INFO: PRVF-5405 : The NTP configuration file "/etc/inet/ntp.conf" does not exist on all nodesINFO: pfreak2,pfreak1INFO: PRVF-5414 : Check of NTP Config file failed on all nodes. Cannot proceed further for the NTP testsINFO: Clock synchronization check using Network Time Protocol(NTP) failedINFO: PRVF-9652 : Cluster Time Synchronization Services check failed
Looking at /etc/inet we can see ntp.conf.cluster
bash-3.00# ls -l ntp*-rw-r--r-- 1 root sys 291 Jan 21 2005 ntp.client-rw-r--r-- 1 root sys 4710 Dec 5 2006 ntp.cluster-rw-r--r-- 1 root root 4379 May 17 02:48 ntp.conf.cluster-rw-r--r-- 1 root sys 2809 Jan 21 2005 ntp.server-rw-r--r-- 1 root sys 3632 Jul 2 2009 ntp4.client-rw-r--r-- 1 root sys 4061 Jul 2 2009 ntp4.serverbash-3.00# pwd/etc/inetbash-3.00# ls -l ntp*-rw-r--r-- 1 root sys 291 Jan 21 2005 ntp.client-rw-r--r-- 1 root sys 4710 Dec 5 2006 ntp.cluster-rw-r--r-- 1 root root 4379 May 17 02:48 ntp.conf.cluster-rw-r--r-- 1 root sys 2809 Jan 21 2005 ntp.server-rw-r--r-- 1 root sys 3632 Jul 2 2009 ntp4.client-rw-r--r-- 1 root sys 4061 Jul 2 2009 ntp4.serverbash-3.00#
To overcome this error I created a symlink.
bash-3.00# cd /etc/inetbash-3.00# ls -l ntp*-rw-r--r-- 1 root sys 291 Jan 21 2005 ntp.client-rw-r--r-- 1 root sys 4710 Dec 5 2006 ntp.cluster-rw-r--r-- 1 root root 4379 May 17 02:48 ntp.conf.cluster-rw-r--r-- 1 root sys 2809 Jan 21 2005 ntp.server-rw-r--r-- 1 root sys 3632 Jul 2 2009 ntp4.client-rw-r--r-- 1 root sys 4061 Jul 2 2009 ntp4.serverbash-3.00# ln -s ntp.conf.cluster ntp.conf bash-3.00# ls -l ntp*-rw-r--r-- 1 root sys 291 Jan 21 2005 ntp.client-rw-r--r-- 1 root sys 4710 Dec 5 2006 ntp.clusterlrwxrwxrwx 1 root root 16 May 17 06:09 ntp.conf -> ntp.conf.cluster-rw-r--r-- 1 root root 4379 May 17 02:48 ntp.conf.cluster-rw-r--r-- 1 root sys 2809 Jan 21 2005 ntp.server-rw-r--r-- 1 root sys 3632 Jul 2 2009 ntp4.client-rw-r--r-- 1 root sys 4061 Jul 2 2009 ntp4.serverbash-3.00#
15
After the above click retry, after which the following failed:
INFO: Checking name resolution setup for "freak-scan-lh"...INFO: ERROR: INFO: PRVF-4664 : Found inconsistent name resolution entries for SCAN name "freak-scan-lh"INFO: ERROR: INFO: PRVF-4657 : Name resolution setup check for "freak-scan-lh" (IP address: 10.11.188.60) failedINFO: ERROR: INFO: PRVF-4663 : Found configuration issue with the 'hosts' entry in the /etc/nsswitch.conf fileINFO: Verification of SCAN VIP and Listener setup failed
Which, so long as you can ping freak-scan-lh is OK to ignore.
INFO: NTP daemon slewing option check failed on some nodesINFO: Check failed on nodes: INFO: pfreak2,pfreak1INFO: PRVF-5436 : The NTP daemon running on one or more nodes lacks the slewing option "slewalways yes"INFO: Clock synchronization check using Network Time Protocol(NTP) failedINFO: PRVF-9652 : Cluster Time Synchronization Services check failedINFO: Post-check for cluster services setup was unsuccessful on all the nodes.
For this error and after looking at metalink for
The NTP daemon running on one or more nodes lacks the slewing option
Resulting in Metalink note 1056693.1
I added the following to /etc/inet/ntp.conf and restarted NTP:
slewalways yesdisable pll
bash-3.00# ps -ef | grep ntp root 1021 1 0 02:51:14 ? 0:01 /usr/lib/inet/xntpd -c /etc/inet/ntp.conf.clusterbash-3.00# kill -9 1021bash-3.00# /usr/lib/inet/xntpd -c /etc/inet/ntp.conf.clusterbash-3.00# ps -ef | grep ntp root 19214 1 0 06:27:34 ? 0:00 /usr/lib/inet/xntpd -c /etc/inet/ntp.conf.clusterbash-3.00#
Then Click Retry.
Afterwards the NTP daemon verification passed as did the rest of the Cluster
16
Verification Utility.
INFO: NTP Configuration file check started...INFO: NTP Configuration file check passedINFO: Checking daemon liveness...INFO: Liveness check passed for "xntpd"INFO: NTP daemon slewing option check passedINFO: NTP daemon's boot time configuration check for slewing option passedINFO: NTP common Time Server Check started...INFO: Check of common NTP Time Server passedINFO: Clock time offset check from NTP Time Server started...INFO: Clock time offset check passedINFO: Clock synchronization check using Network Time Protocol(NTP) passedINFO: Oracle Cluster Time Synchronization Services check passedINFO: Post-check for cluster services setup was unsuccessful on all the nodes. Then Click Next/Close.
Display the Grid Infrastructure resources
bash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ cat .profileORACLE_HOME=/u01/app/11.2.0/gridPATH=${PATH}:${ORACLE_HOME}/bin
MANPATH=/usr/man:/usr/cluster/man; export MANPATH-bash-3.00$-bash-3.00$ crsctl stat res -t--------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS --------------------------------------------------------------------------------Local Resources--------------------------------------------------------------------------------ora.CRS.dg ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.LISTENER.lsnr ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.asm ONLINE ONLINE pfreak1 Started ONLINE ONLINE pfreak2 Started ora.eons ONLINE ONLINE pfreak1
17
ONLINE ONLINE pfreak2 ora.gsd OFFLINE OFFLINE pfreak1 OFFLINE OFFLINE pfreak2 ora.net1.network ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.ons ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 --------------------------------------------------------------------------------Cluster Resources--------------------------------------------------------------------------------ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE pfreak1 ora.oc4j 1 OFFLINE OFFLINE ora.pfreak1.vip 1 ONLINE ONLINE pfreak1 ora.pfreak2.vip 1 ONLINE ONLINE pfreak2 ora.scan1.vip 1 ONLINE ONLINE pfreak1 -bash-3.00$
Create another ASM disk group
bash-3.00# xhost +bash-3.00# su - oragrid-bash-3.00$ export DISPLAY=:10.0-bash-3.00$ /u01/app/11.2.0/grid/bin/asmca
ASM Configuration Assistant: Configure ASM: ASM Instances• Select Disk Groups tab• Click Create• Disk Group Name – DATA• Redundancy – External• Select remaining disks• Click OK• Note thaat DATA is MOUNTED (2 of 2)• Click Exit/Yes
Note that a Grid Infrastructure ASM Disk Group resource has been created.-bash-3.00$ crsctl stat res -t--------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS --------------------------------------------------------------------------------
18
Local Resources--------------------------------------------------------------------------------ora.CRS.dg ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.DATA.dg ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.LISTENER.lsnr ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.asm ONLINE ONLINE pfreak1 Started ONLINE ONLINE pfreak2 Started ora.eons ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.gsd OFFLINE OFFLINE pfreak1 OFFLINE OFFLINE pfreak2 ora.net1.network ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.ons ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 --------------------------------------------------------------------------------Cluster Resources--------------------------------------------------------------------------------ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE pfreak1 ora.oc4j 1 OFFLINE OFFLINE ora.pfreak1.vip 1 ONLINE ONLINE pfreak1 ora.pfreak2.vip 1 ONLINE ONLINE pfreak2 ora.scan1.vip 1 ONLINE ONLINE pfreak1 -bash-3.00$
4) Install and create an 11gR1 database via a VNC client
Install the 11gR1 database software
Before installing an 11gR1 database we need to pin the cluster nodes. Please refer to Metalink note 949073.1
19
bash-3.00# cd /u01/app/11.2.0/grid/binbash-3.00# ./olsnodes -t -npfreak1 1 Unpinnedpfreak2 2 Unpinnedbash-3.00#bash-3.00# ./crsctl pin css -n pfreak1 pfreak2CRS-4664: Node pfreak1 successfully pinned.CRS-4664: Node pfreak2 successfully pinned.bash-3.00# bash-3.00# ./olsnodes -t -npfreak1 1 Pinnedpfreak2 2 Pinnedbash-3.00#
bash-3.00# mkdir -p /export/home/ora11gbash-3.00# useradd -g oinstall -G dba -u 302 -d /export/home/ora11g -s /usr/bin/bash ora11gbash-3.00# chown ora11g:oinstall /export/home/ora11gbash-3.00#bash-3.00# mkdir /u01/app/ora11gbash-3.00# chown ora11g:oinstall /u01/app/ora11gbash-3.00# chmod 775 /u01/app/ora11gbash-3.00# bash-3.00# passwd ora11gNew Password: Re-enter new Password: passwd: password successfully changed for ora11gbash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ cat > .profile <<-EOF> ORACLE_BASE=/u01/app/ora11g> EOF-bash-3.00$
From a VNC client on pfreak1.
bash-3.00# xhost +bash-3.00# su – ora11g-bash-3.00$ export DISPLAY=:10.0-bash-3.00$ cd /net/dv5/swlib/oracle/released/db/11gR1/11.1.0.6/sparc*/64*/database-bash-3.00$ ./runInstaller
Welcome• Click Next
Select Installation Type• Select Custom
20
• Click NextSpecify installation Location
• Oracle Base - /u01/app/ora11g• Software Location • Name: OraDb11g_home1• Path: /u01/app/ora11g/product/11.1.0/db_1• Click Next
Specify Hardware Cluster Installation Mode• Select Local Installation• Click Next
Product-Specific Pre-requisite Checks • Click Next
Available Product Components• Click Next
Privileged Operating System groups• OSDBA – dba• OSOPER – oinstall• OSASM - oinstall• Click Next
Create Database• Select Install database Software only• Click Next
Summary• Click Install
Execute Configuration scripts• Execute root.sh and Click OK/Close
Repeat the above to install the database software on pfreak2, however ensure that the Path is the same within “Specify installation Location”, as this install is on local file system.
Create the 11gR1 database using dbca
Create 11gR1 database via dbca on pfreak1 via a VNC client.
bash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ export DISPLAY=:10.0-bash-3.00$ cd /u01/app/ora11g/product/11.1.0/db_1/bin-bash-3.00$ ./dbca
Welcome to dbca• Click Next• Step 1/15 – Create a Database – Click Next• Step 2/15 – General Purpose – Click Next• Step 3/15 – DB1 – Click Next• Step 4/15 – Click Next• Step 5/15 – Use same password (manager) – Click Next
21
• Step 6/15 – Select ASM Click Next • Specify ASMSNMP password to ASM (manager) – Click OK• Step 7/16 – Select DATA - Click Next• Step 8/16 – Click Next• Step 9/16 – Click Next• Step 10/16 – Click Next• Step 11/16 – memory your choice (1024) – Click Next• Step 12/15 – Click Next• Step 13/15 – Click Next• Step 14/15 – Click Next• Step 15/15 – Create Database - Click Finish/OK• Summary – Click Exit
Create the 11gR1 database listener using netca
Create 11gR1 database listener via netca on pfreak1 via a VNC client.
bash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ export DISPLAY=:10.0-bash-3.00$ cd /u01/app/ora11g/product/11.1.0/db_1/bin-bash-3.00$ ./netca
Select the type of Oracle Net Services configuration• Select Single node configuration• Click Next
Welcome• Select Listener configuration• Click Next
Listener_name• Enter LISTENER_DB1• Click Next
Listener proptocol• Click Next
Listener port number• Use another port number 1522• Click Next
More listeners• Select no• Click Next/Next/Finish
bash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$-bash-3.00$ cd /u01/app/ora11g/product/11.1.0/db_1/network/admin
22
Change listener.ora and tnsnames.ora to use a logical host.
-bash-3.00$ cat listener.ora # listener.ora Network Configuration File: /u01/app/ora11g/product/11.1.0/db_1/network/admin/listener.ora# Generated by Oracle configuration tools.
LISTENER_DB1 = (DESCRIPTION_LIST = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = vzfreak2a)(PORT = 1522)) (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1522)) ) )
-bash-3.00$ -bash-3.00$ cat tnsnames.oraLISTENER_DB1 = (ADDRESS = (PROTOCOL = TCP)(HOST = vzfreak2a)(PORT = 1522))-bash-3.00$
-bash-3.00$ export ORACLE_HOME=/u01/app/ora11g/product/11.1.0/db_1-bash-3.00$ export ORACLE_SID=DB1-bash-3.00$ $ORACLE_HOME/bin/sqlplus "/as sysdba"
SQL*Plus: Release 11.1.0.6.0 - Production on Wed May 19 03:43:40 2010
Copyright (c) 1982, 2007, Oracle. All rights reserved.
Connected to:Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - 64bit ProductionWith the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL>SQL> show parameters local_listener
NAME TYPE VALUE------------------------------------ ----------- ------------------------------local_listener stringSQL>SQL> alter system set local_listener='LISTENER_DB1' scope=both;
System altered.
SQL>SQL> show parameters local_listener
NAME TYPE VALUE
23
------------------------------------ ----------- ------------------------------local_listener string LISTENER_DB1SQL> SQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - 64bit ProductionWith the Partitioning, OLAP, Data Mining and Real Application Testing options-bash-3.00$
Copy some 11gR1 database files to the another node
After dbca and netca are finished we need to copy some files from pfreak1 to pfreak2. Note that the ../../.. represents the ora11g home directories, i.e. /export/home/ora11g.On pfreak2.
bash-3.00# uname -npfreak2bash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ cd /u01/app/ora11g/diag/rdbms-bash-3.00$ ls -ltotal 0-bash-3.00$ /usr/sfw/bin/wget -r -nH --cut-dirs=8 ftp://ora11g:ora11g@pfreak1:/../../../u01/app/ora11g/diag/rdbms/db1<...>-bash-3.00$ ls -ltotal 2drwxr-xr-x 3 ora11g oinstall 512 May 18 08:11 db1-bash-3.00$ cd ../..-bash-3.00$-bash-3.00$ pwd/u01/app/ora11g-bash-3.00$ ls -ltotal 4drwxrwxr-x 11 ora11g oinstall 512 May 18 06:49 diagdrwxr-xr-x 3 ora11g oinstall 512 May 18 06:35 product-bash-3.00$-bash-3.00$ /usr/sfw/bin/wget -r -nH --cut-dirs=6 ftp://ora11g:ora11g@pfreak1:/../../../u01/app/ora11g/admin<...>-bash-3.00$ ls -ltotal 6drwxr-xr-x 3 ora11g oinstall 512 May 18 08:12 admindrwxrwxr-x 11 ora11g oinstall 512 May 18 06:49 diagdrwxr-xr-x 3 ora11g oinstall 512 May 18 06:35 product-bash-3.00$
24
-bash-3.00$ cd /u01/app/ora11g/product/11.1.0/db_1/dbs-bash-3.00$-bash-3.00$ ls -ltotal 44-rw-r--r-- 1 ora11g oinstall 8385 Sep 11 1998 init.ora-rw-r--r-- 1 ora11g oinstall 12920 May 3 2001 initdw.ora-bash-3.00$-bash-3.00$ ftp pfreak1Connected to pfreak1.220 pfreak1 FTP server ready.Name (pfreak1:root): ora11g331 Password required for ora11g.Password:230 User ora11g logged in.Remote system type is UNIX.Using binary mode to transfer files.ftp> cd /u01/app/ora11g/product/11.1.0/db_1/dbs250 CWD command successful.ftp> lcd /u01/app/ora11g/product/11.1.0/db_1/dbsLocal directory now /u01/app/ora11g/product/11.1.0/db_1/dbsftp> prompt Interactive mode off.ftp> mget *DB1200 PORT command successful.150 Opening BINARY mode data connection for lkDB1 (24 bytes).226 Transfer complete.local: lkDB1 remote: lkDB124 bytes received in 0.014 seconds (1.68 Kbytes/s)200 PORT command successful.150 Opening BINARY mode data connection for orapwDB1 (1536 bytes).226 Transfer complete.local: orapwDB1 remote: orapwDB11536 bytes received in 0.013 seconds (111.88 Kbytes/s)ftp> ftp> cd /u01/app/ora11g/product/11.1.0/db_1/network/admin250 CWD command successful.ftp> lcd /u01/app/ora11g/product/11.1.0/db_1/network/adminLocal directory now /u01/app/ora11g/product/11.1.0/db_1/network/adminftp> mget *ora*200 PORT command successful.150 Opening BINARY mode data connection for listener.ora (337 bytes).226 Transfer complete.local: listener.ora remote: listener.ora337 bytes received in 0.012 seconds (26.59 Kbytes/s)200 PORT command successful.150 Opening BINARY mode data connection for tnsnames.ora (77 bytes).226 Transfer complete.local: tnsnames.ora remote: tnsnames.ora77 bytes received in 0.011 seconds (6.80 Kbytes/s)ftp> bye221-You have transferred 1974 bytes in 4 files.
25
221-Total traffic for this session was 3428 bytes in 6 transfers.221-Thank you for using the FTP service on pfreak1.221 Goodbye.-bash-3.00$
Verify the 11gR1 database installation
bash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ export ORACLE_HOME=/u01/app/ora11g/product/11.1.0/db_1-bash-3.00$ export ORACLE_SID=DB1-bash-3.00$ $ORACLE_HOME/bin/sqlplus "/as sysdba"
SQL*Plus: Release 11.1.0.6.0 - Production on Tue May 18 08:16:41 2010
Copyright (c) 1982, 2007, Oracle. All rights reserved.
Connected to:Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - 64bit ProductionWith the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL> show sga
Total System Global Area 801939456 bytesFixed Size 2092920 bytesVariable Size 260051080 bytesDatabase Buffers 536870912 bytesRedo Buffers 2924544 bytesSQL> SQL> create user homer identified by simpson;
User created.
SQL> alter user homer default tablespace system quota 1m on system;
User altered.
SQL> grant select on v_$sysstat to homer;
Grant succeeded.
SQL> grant select on v_$archive_dest to homer;
Grant succeeded.
SQL> grant create session to homer;
Grant succeeded.
26
SQL> grant create table to homer;
Grant succeeded.
SQL> grant select on v_$database to homer;
Grant succeeded.
SQL> shutdown immediateDatabase closed.Database dismounted.ORACLE instance shut down.SQL>SQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options-bash-3.00$
And on pfreak2.
bash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ export ORACLE_HOME=/u01/app/ora11g/product/11.1.0/db_1-bash-3.00$ export ORACLE_SID=DB1-bash-3.00$ $ORACLE_HOME/bin/sqlplus "/as sysdba"
SQL*Plus: Release 11.1.0.6.0 - Production on Wed May 19 01:53:13 2010
Copyright (c) 1982, 2007, Oracle. All rights reserved.
Connected to an idle instance.
SQL> startupORACLE instance started.
Total System Global Area 801939456 bytesFixed Size 2092920 bytesVariable Size 268439688 bytesDatabase Buffers 528482304 bytesRedo Buffers 2924544 bytesDatabase mounted.Database opened.SQL>SQL> select username from all_users where username='HOMER';
27
USERNAME------------------------------HOMER
SQL>SQL> shutdown immediateDatabase closed.Database dismounted.ORACLE instance shut down.SQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - 64bit ProductionWith the Partitioning, OLAP, Data Mining and Real Application Testing options-bash-3.00$
Check the Grid Infrastructure resources
Note that Grid Infrastructure does not know about DB1 as 11gR1 is not 11gR2 Grid Infrastructure aware. As such there is no Grid Infrastructure resource ora.db1.db.
bash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ -bash-3.00$ /u01/app/11.2.0/grid/bin/crsctl stat res -t--------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS --------------------------------------------------------------------------------Local Resources--------------------------------------------------------------------------------ora.CRS.dg ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.DATA.dg ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.LISTENER.lsnr ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.asm ONLINE ONLINE pfreak1 Started ONLINE ONLINE pfreak2 Started ora.eons ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2
28
ora.gsd OFFLINE OFFLINE pfreak1 OFFLINE OFFLINE pfreak2 ora.net1.network ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.ons ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 --------------------------------------------------------------------------------Cluster Resources--------------------------------------------------------------------------------ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE pfreak1 ora.oc4j 1 OFFLINE OFFLINE ora.pfreak1.vip 1 ONLINE ONLINE pfreak1 ora.pfreak2.vip 1 ONLINE ONLINE pfreak2 ora.scan1.vip 1 ONLINE ONLINE pfreak1 -bash-3.00$
5) Install and create an 11gR2 database via a VNC client
Install the 11gR2 database software
Note that here I will install the database software on each node and afterwards create the database.
From a VNC client on pfreak1.
bash-3.00# xhost +bash-3.00# su – ora11g -bash-3.00$ export DISPLAY=:10.0-bash-3.00$ cd /net/dv5/swlib/oracle/released/db/11gR2/11.2.0.1/sparc*/64*/database-bash-3.00$ ./runInstaller
Configure Security Updates• Deselect I wish to receive security updates...• Click Next/Yes• Select Install database software • Click Next
Node Selection• Select single instance database installation• Click Next
29
Select Product Languages• Click Next
Select Database Edition• Select Enterprise Edition• Click Next
Specify installation Location• Oracle Base - /u01/app/ora11g• Software Location - /u01/app/ora11g/product/11.2.0/dbhome_1• Click Next
Privileged Operating System groups• OSDBA – dba• OSOPER – oinstall• Click Next
Perform Prerequisite Checks• Ignore Compiler patch 124861-15• Ignore users with same UID• Select Ignore All• Click Next
Summary• Click Finish
Execute Configuration scripts• Execute root.sh and Click OK/Close
Repeat the above to install the database software on pfreak2.
Create 11gR2 database via dbca on pfreak1 via a VNC client.
As per Metalink note 983715.1, if creating 11.2.0.1 (fixed with 11.2.0.2) amend the following on each node.
/etc/init.d/ohasd
start(){ $ECHO -n $"Starting $PROG: " ulimit -n 65536 ulimit -l 50000000
Create the 11gR2 database using dbca
bash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ export DISPLAY=:10.0-bash-3.00$ cd /u01/app/ora11g/product/11.2.0/dbhome_1/bin-bash-3.00$ ./dbca
Welcome to dbca• Click Next
30
• Step 1/12 – Create a Database – Click Next• Step 2/12 – General Purpose – Click Next• Step 3/12 – DB2 – Click Next• Step 4/12 – Click Next• Step 5/12 – Use same password (manager) – Click Next/yes• Step 6/12 – Storage Type (ASM) Database Area (+DATA)• Specify ASMSNMP password to ASM (manager) – Click OK• Step 7/11 – Click Next• Step 8/11 – Click Next• Step 9/11 – memory your choice (1024) – Click Next• Step 10/11 – Click Next• Step 11/11 – Create Database - Click Finish/OK• Summary – Click Exit
Copy some 11gR2 database files to the another node
After dbca is finished we need to copy some files from pfreak1 to pfreak2. Note that the ../../.. represents the ora11g home directories, i.e. /export/home/ora11g.
On pfreak2.
bash-3.00# uname -npfreak2bash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ cd /u01/app/ora11g/diag/rdbms-bash-3.00$ ls -ltotal 2drwxr-xr-x 3 ora11g oinstall 512 May 18 08:11 db1-bash-3.00$-bash-3.00$ /usr/sfw/bin/wget -r -nH --cut-dirs=8 ftp://ora11g:ora11g@pfreak1:/../../../u01/app/ora11g/diag/rdbms/db2<...>-bash-3.00$ ls -ltotal 4drwxr-xr-x 3 ora11g oinstall 512 May 18 08:11 db1drwxr-xr-x 3 ora11g oinstall 512 May 18 11:41 db2-bash-3.00$ cd ../../admin-bash-3.00$ ls -ltotal 2drwxr-xr-x 5 ora11g oinstall 512 May 18 08:12 DB1-bash-3.00$-bash-3.00$ /usr/sfw/bin/wget -r -nH --cut-dirs=7 ftp://ora11g:ora11g@pfreak1:/../../../u01/app/ora11g/admin/DB2<...>-bash-3.00$ pwd/u01/app/ora11g/admin
31
-bash-3.00$ ls -ltotal 4drwxr-xr-x 5 ora11g oinstall 512 May 18 08:12 DB1drwxr-xr-x 5 ora11g oinstall 512 May 18 11:43 DB2-bash-3.00$ -bash-3.00$ cd /u01/app/ora11g/product/11.2.0/dbhome_1/dbs-bash-3.00$-bash-3.00$ ftp pfreak1Connected to pfreak1.220 pfreak1 FTP server ready.Name (pfreak1:root): ora11g331 Password required for ora11g.Password:
230 User ora11g logged in.Remote system type is UNIX.Using binary mode to transfer files.ftp> cd /u01/app/ora11g/product/11.2.0/dbhome_1/dbs250 CWD command successful.ftp> promptInteractive mode off.ftp> mget *DB2*200 PORT command successful.150 Opening BINARY mode data connection for hc_DB2.dat (1544 bytes).226 Transfer complete.local: hc_DB2.dat remote: hc_DB2.dat1544 bytes received in 0.02 seconds (76.25 Kbytes/s)200 PORT command successful.150 Opening BINARY mode data connection for initDB2.ora (33 bytes).226 Transfer complete.local: initDB2.ora remote: initDB2.ora33 bytes received in 0.011 seconds (3.05 Kbytes/s)200 PORT command successful.150 Opening BINARY mode data connection for lkDB2 (24 bytes).226 Transfer complete.local: lkDB2 remote: lkDB224 bytes received in 0.013 seconds (1.83 Kbytes/s)200 PORT command successful.150 Opening BINARY mode data connection for orapwDB2 (1536 bytes).226 Transfer complete.local: orapwDB2 remote: orapwDB21536 bytes received in 0.012 seconds (125.02 Kbytes/s)ftp> bye221-You have transferred 3137 bytes in 4 files.221-Total traffic for this session was 4307 bytes in 5 transfers.221-Thank you for using the FTP service on pfreak1.221 Goodbye.-bash-3.00$
32
Verify the 11gR2 database installation
Note that for an 11gR2 database we do not need to create a user that would normally be used by the HA-Oracle (SUNW.oracle_server) probe, as probing is mainly provided by a Grid Infrastructure database resource, which has been automatically created when the database was created using dbca.
Also note that at this point we cannot verify that the database DB2 can start on the other node as the database is configured, via dbca, to only run on one node. That node is the node where the database was created.
When HA-Oracle (SUNW.oracle_server) is deployed later on, the server pool associated with the database is modified (by SUNW.oracle_server) so that the database can run on another node. This is transparent to the administrator and is managed by SUNW.oracle_server.
bash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ export ORACLE_HOME=/u01/app/ora11g/product/11.2.0/dbhome_1-bash-3.00$ export ORACLE_SID=DB2-bash-3.00$ $ORACLE_HOME/bin/sqlplus "/as sysdba"
SQL*Plus: Release 11.2.0.1.0 Production on Wed May 19 02:12:42 2010
Copyright (c) 1982, 2009, Oracle. All rights reserved.
Connected to:Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options
SQL> show sga
Total System Global Area 1069252608 bytesFixed Size 2154936 bytesVariable Size 717233736 bytesDatabase Buffers 343932928 bytesRedo Buffers 5931008 bytesSQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options-bash-3.00$
33
Check the Grid Infrastructure resources
Note that for an 11gR2 database Grid Infrastructure does know about the database. As such there is Grid Infrastructure resource ora.db2.db.
-bash-3.00$ exitlogoutbash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ srvctl status db -d db2Instance DB2 is running on node pfreak1-bash-3.00$ -bash-3.00$ crsctl stat res -t--------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS --------------------------------------------------------------------------------Local Resources--------------------------------------------------------------------------------ora.CRS.dg ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.DATA.dg ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.LISTENER.lsnr ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.asm ONLINE ONLINE pfreak1 Started ONLINE ONLINE pfreak2 Started ora.eons ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.gsd OFFLINE OFFLINE pfreak1 OFFLINE OFFLINE pfreak2 ora.net1.network ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 ora.ons ONLINE ONLINE pfreak1 ONLINE ONLINE pfreak2 --------------------------------------------------------------------------------Cluster Resources-----------------------------------------------------------------------------
34
---ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE pfreak2 ora.db2.db 1 ONLINE ONLINE pfreak1 Open ora.oc4j 1 OFFLINE OFFLINE ora.pfreak1.vip 1 ONLINE ONLINE pfreak1 ora.pfreak2.vip 1 ONLINE ONLINE pfreak2 ora.scan1.vip 1 ONLINE ONLINE pfreak2 -bash-3.00$
6) Using SCAN for the 11gR2 single instance database
In order for the single instance database to use SCAN we need to amend the remote_listener parameter.
-bash-3.00$ srvctl config scanSCAN name: freak-scan-lh, Network: 1/10.11.188.0/255.255.255.0/bge0SCAN VIP name: scan1, IP: /vzfreak1a/10.11.188.60-bash-3.00$ -bash-3.00$ srvctl config scan_listenerSCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521-bash-3.00$
SQL> show parameters listener
NAME TYPE VALUE------------------------------------ ----------- ------------------------------listener_networks stringlocal_listener string (DESCRIPTION=(ADDRESS_LIST=(AD
DRESS=(PROTOCOL=TCP)(HOST=frea k-1)(PORT=1521))))
remote_listener stringSQL>SQL> alter system set remote_listener = 'freak-scan-lh:1521' scope=both;
System altered.
SQL> show parameters listener
NAME TYPE VALUE------------------------------------ ----------- ------------------------------listener_networks stringlocal_listener string (DESCRIPTION=(ADDRESS_LIST=(AD
DRESS=(PROTOCOL=TCP)(HOST=frea
35
k-1)(PORT=1521))))remote_listener string freak-scan-lh:1521SQL>
On the client, as we are using /etc/hosts for the SCAN address, you will need an entry within /etc/hosts
10.11.188.60 freak-scan-lh
also the client $ORACLE_HOME/network/admin/tnsnames.ora
DB2 = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = freak-scan-lh)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = DB2) ) )
7) Place everything under Oracle Solaris Cluster
Register the resource types
bash-3.00# clrt listbash-3.00# clrt listSUNW.LogicalHostname:3SUNW.SharedAddress:2SUNW.rac_framework:5SUNW.rac_udlm:3bash-3.00#bash-3.00# clrt register SUNW.crs_frameworkbash-3.00# clrt register SUNW.scalable_asm_instance_proxybash-3.00# clrt register SUNW.scalable_asm_diskgroup_proxybash-3.00# clrt register SUNW.oracle_serverbash-3.00# clrt register SUNW.oracle_listenerbash-3.00# clrt listSUNW.LogicalHostname:3SUNW.SharedAddress:2SUNW.rac_framework:5SUNW.rac_udlm:3SUNW.crs_framework:3SUNW.scalable_asm_instance_proxySUNW.scalable_asm_diskgroup_proxySUNW.oracle_server:6SUNW.oracle_listener:5bash-3.00#
36
bash-3.00# clrg status
=== Cluster Resource Groups ===
Group Name Node Name Suspended Status---------- --------- --------- ------rac-fmwk-rg pfreak1 No Online pfreak2 No Online
bash-3.00# clrs create -g rac-fmwk-rg -t SUNW.crs_framework -p resource_dependencies=rac-fmwk-rs crs-fmwk-rsbash-3.00# clrs status
=== Cluster Resources ===
Resource Name Node Name State Status Message------------- --------- ----- --------------rac-fmwk-rs pfreak1 Online Online pfreak2 Online Online
rac-udlm-rs pfreak1 Online Online pfreak2 Online Online
crs-fmwk-rs pfreak1 Online Online pfreak2 Online Online
bash-3.00#
Register and enable the clustered ASM instance resourcebash-3.00# clrg create -S asm-inst-rgbash-3.00# clrg set -p rg_affinities=++rac-fmwk-rg asm-inst-rgbash-3.00# clrs create -g asm-inst-rg -t SUNW.scalable_asm_instance_proxy -p oracle_home=/u01/app/11.2.0/grid -p 'oracle_sid{pfreak1}'=+ASM1 -p 'oracle_sid{pfreak2}'=+ASM2 -p crs_home=/u01/app/11.2.0/grid -p resource_dependencies_offline_restart=crs-fmwk-rs -d asm-inst-rsbash-3.00# clrg online -eM asm-inst-rgbash-3.00# clrs status
=== Cluster Resources ===
Resource Name Node Name State Status Message------------- --------- ----- --------------rac-fmwk-rs pfreak1 Online Online pfreak2 Online Online
rac-udlm-rs pfreak1 Online Online pfreak2 Online Online
crs-fmwk-rs pfreak1 Online Online pfreak2 Online Online
37
asm-inst-rs pfreak1 Online Online - +ASM1 is UP and ENABLED pfreak2 Online Online - +ASM2 is UP and ENABLED
bash-3.00#
Register and enable the clustered ASM disk group resource
bash-3.00# clrg create -S asm-dg-rgbash-3.00# clrg set -p rg_affinities=++asm-inst-rg asm-dg-rgbash-3.00# clrs create -g asm-dg-rg -t SUNW.scalable_asm_diskgroup_proxy -p asm_diskgroups=DATA -p resource_dependencies_offline_restart=asm-inst-rs -d asm-dg-rsbash-3.00# clrg online -eM asm-dg-rgbash-3.00# clrs status
=== Cluster Resources ===
Resource Name Node Name State Status Message------------- --------- ----- --------------rac-fmwk-rs pfreak1 Online Online pfreak2 Online Online
rac-udlm-rs pfreak1 Online Online pfreak2 Online Online
crs-fmwk-rs pfreak1 Online Online pfreak2 Online Online
asm-inst-rs pfreak1 Online Online - +ASM1 is UP and ENABLED pfreak2 Online Online - +ASM2 is UP and ENABLED
asm-dg-rs pfreak1 Online Online - Mounted: DATA pfreak2 Online Online - Mounted: DATA
bash-3.00#
Register and enable the 11gR1 database and listener resources
bash-3.00# clrg create ora-11gr1-rgbash-3.00# clrg set -p rg_affinities=++asm-dg-rg ora-11gr1-rgbash-3.00# clrslh create -g ora-11gr1-rg -h vzfreak2a ora-db1-lhbash-3.00# clrs create -g ora-11gr1-rg -t SUNW.oracle_server -p oracle_home=/u01/app/ora11g/product/11.1.0/db_1 -p oracle_sid=DB1 -p alert_log_file=/u01/app/ora11g/diag/rdbms/db1/DB1/trace/alert_DB1.log -p connect_string=homer/simpson -p resource_dependencies_offline_restart=asm-dg-rs -d ora-db1-rs
38
bash-3.00#bash-3.00# clrg online -eM ora-11gr1-rgbash-3.00# clrs status -g ora-11gr1-rg
=== Cluster Resources ===
Resource Name Node Name State Status Message------------- --------- ----- --------------ora-db1-rs pfreak1 Online Online pfreak2 Offline Offline
ora-db1-lh pfreak1 Online Online - LogicalHostname online. pfreak2 Offline Offline
bash-3.00#
bash-3.00# su - ora11gSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$-bash-3.00$ export ORACLE_HOME=/u01/app/ora11g/product/11.1.0/db_1-bash-3.00$ /u01/app/ora11g/product/11.1.0/db_1/bin/lsnrctl status LISTENER_DB1
LSNRCTL for Solaris: Version 11.1.0.6.0 - Production on 19-MAY-2010 03:51:57
Copyright (c) 1991, 2007, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=vzfreak2a)(PORT=1522)))STATUS of the LISTENER------------------------Alias LISTENER_DB1Version TNSLSNR for Solaris: Version 11.1.0.6.0 - ProductionStart Date 19-MAY-2010 03:37:02Uptime 0 days 0 hr. 14 min. 55 secTrace Level offSecurity ON: Local OS AuthenticationSNMP OFFListener Parameter File /u01/app/ora11g/product/11.1.0/db_1/network/admin/listener.oraListener Log File /u01/app/ora11g/diag/tnslsnr/pfreak1/listener_db1/alert/log.xmlListening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=pfreak1)(PORT=1522))) (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1522)))Services Summary...
39
Service "DB1" has 1 instance(s). Instance "DB1", status READY, has 1 handler(s) for this service...Service "DB1XDB" has 1 instance(s). Instance "DB1", status READY, has 1 handler(s) for this service...Service "DB1_XPT" has 1 instance(s). Instance "DB1", status READY, has 1 handler(s) for this service...The command completed successfully-bash-3.00$ exitlogoutbash-3.00#bash-3.00# clrs create -g ora-11gr1-rg -t SUNW.oracle_listener -p listener_name=LISTENER_DB1 -p oracle_home=/u01/app/ora11g/product/11.1.0/db_1 -p resource_dependencies=ora-db1-lh -d ora-db1-lsrbash-3.00# clrs enable ora-db1-lsrbash-3.00# clrs status -g ora-11gr1-rg=== Cluster Resources ===
Resource Name Node Name State Status Message------------- --------- ----- --------------ora-db1-rs pfreak1 Online Online pfreak2 Offline Offline
ora-db1-lh pfreak1 Online Online - LogicalHostname online. pfreak2 Offline Offline
ora-db1-lsr pfreak1 Online Online pfreak2 Offline Offline
bash-3.00#
Switch the 11gR1 Resource Group
bash-3.00# clrg status
=== Cluster Resource Groups ===
Group Name Node Name Suspended Status---------- --------- --------- ------rac-fmwk-rg pfreak1 No Online pfreak2 No Online
asm-inst-rg pfreak1 No Online pfreak2 No Online
asm-dg-rg pfreak1 No Online pfreak2 No Online
ora-11gr1-rg pfreak1 No Online pfreak2 No Offline
40
bash-3.00# clrg switch -n pfreak2 ora-11gr1-rgbash-3.00# clrg status
=== Cluster Resource Groups ===
Group Name Node Name Suspended Status---------- --------- --------- ------rac-fmwk-rg pfreak1 No Online pfreak2 No Online
asm-inst-rg pfreak1 No Online pfreak2 No Online
asm-dg-rg pfreak1 No Online pfreak2 No Online
ora-11gr1-rg pfreak1 No Offline pfreak2 No Online
bash-3.00# clrs status -g ora-11gr1-rg
=== Cluster Resources ===
Resource Name Node Name State Status Message------------- --------- ----- --------------ora-db1-rs pfreak1 Offline Offline pfreak2 Online Online
ora-db1-lh pfreak1 Offline Offline - LogicalHostname offline. pfreak2 Online Online - LogicalHostname online.
ora-db1-lsr pfreak1 Offline Offline pfreak2 Online Online
bash-3.00#
Register and enable the 11gR2 database resource
Note that for a single insatnce 11gR2 database we do not need to specify the ALERT_LOG_FILE and CONNECT_STRING extension properties. However, it is still okay to specify those extension properties, they would just be ignored for a single instance 11gR2 database.
bash-3.00# clrg create ora-11gr2-rgbash-3.00# clrg set -p rg_affinities=++asm-dg-rg ora-11gr2-rgbash-3.00# clrs create -g ora-11gr2-rg -t SUNW.oracle_server -p oracle_home=/u01/app/ora11g/product/11.2.0/dbhome_1 -p oracle_sid=DB2 -p db_unique_name=DB2 -p resource_dependencies_offline_restart=asm-dg-rs -d ora-db2-rsbash-3.00#
41
bash-3.00# clrg online -eM ora-11gr2-rgbash-3.00# clrs status -g ora-11gr2-rg
=== Cluster Resources ===
Resource Name Node Name State Status Message------------- --------- ----- --------------ora-db2-rs pfreak1 Online Online pfreak2 Offline Offline
bash-3.00#
Also note that the preferred listener for a single instance 11gR2 database is to use the Grid Infrastructure for Clusters SCAN Listener. Briefly, the SIDB registers itself to a local Oracle listener. Note that local Oracle listeners are each running on all cluster nodes, however the SIDB only registers itself to one local listener at any one time. As the SIDB is switched or failed over between nodes it will unregister itself from the local listener on the node where it was running and register itself with the local listener on the node where it is now running. SCAN provides the ability to present the client with a single name of where the SIDB is running, regardless of where the SIDB is located within the cluster. As such when the client tries to connect to the SIDB the SCAN GI resource determines which local Oracle listener is listening for the SIDB service and returns that address to the client. The client then continues to connect with the SID using the address provided by SCAN.
If the administrator does not want to deploy or leverage SCAN, then the existing SUNW.oracle_listener RT can be deployed and does not require development changes. Nevertheless, the 11gR2 SIDB needs to have it's local_listener system parameter changed to reflect the local listener that is then managed by the SUNW.oracle_listener RT. Appropriate tnsnames.ora and listener.ora files need to be defined to reflect the local_listener definition as described within the SC/OSC HA-Oracle documentation.
By default, when creating the 11gR2 database with dbca, the following local_listener entry is defined. Note that earlier we also modified the remote_listener to use SCAN.
bash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ export ORACLE_HOME=/u01/app/ora11g/product/11.2.0/dbhome_1-bash-3.00$ export ORACLE_SID=DB2-bash-3.00$ $ORACLE_HOME/bin/sqlplus "/as sysdba"
SQL*Plus: Release 11.2.0.1.0 Production on Wed May 19 08:24:10 2010
Copyright (c) 1982, 2009, Oracle. All rights reserved.
42
Connected to:Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options
SQL> show parameters listener
NAME TYPE VALUE------------------------------------ ----------- ------------------------------listener_networks stringlocal_listener string (DESCRIPTION=(ADDRESS_LIST=(AD
DRESS=(PROTOCOL=TCP)(HOST=frea k-1)(PORT=1521))))
remote_listener string freak-scan-lh:1521SQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options-bash-3.00$ exitlogoutbash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ lsnrctl status
LSNRCTL for Solaris: Version 11.2.0.1.0 - Production on 19-MAY-2010 08:24:53
Copyright (c) 1991, 2009, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))STATUS of the LISTENER------------------------Alias LISTENERVersion TNSLSNR for Solaris: Version 11.2.0.1.0 - ProductionStart Date 19-MAY-2010 08:04:37Uptime 0 days 0 hr. 20 min. 16 secTrace Level offSecurity ON: Local OS AuthenticationSNMP OFFListener Parameter File /u01/app/11.2.0/grid/network/admin/listener.oraListener Log File /u01/app/oragrid/diag/tnslsnr/pfreak1/listener/alert/log.xmlListening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.11.188.56)(PORT=1521)))
43
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.11.188.58)(PORT=1521)))Services Summary...Service "+ASM" has 1 instance(s). Instance "+ASM1", status READY, has 1 handler(s) for this service...Service "DB2" has 1 instance(s). Instance "DB2", status READY, has 1 handler(s) for this service...Service "DB2XDB" has 1 instance(s). Instance "DB2", status READY, has 1 handler(s) for this service...The command completed successfully-bash-3.00$
-bash-3.00$ $ORACLE_HOME/bin/lsnrctl status LISTENER_SCAN1
LSNRCTL for Solaris: Version 11.2.0.1.0 - Production on 03-JUN-2010 07:46:01
Copyright (c) 1991, 2009, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1)))STATUS of the LISTENER------------------------Alias LISTENER_SCAN1Version TNSLSNR for Solaris: Version 11.2.0.1.0 - ProductionStart Date 03-JUN-2010 06:11:13Uptime 0 days 1 hr. 34 min. 47 secTrace Level offSecurity ON: Local OS AuthenticationSNMP OFFListener Parameter File /u01/app/11.2.0/grid/network/admin/listener.oraListener Log File /u01/app/11.2.0/grid/log/diag/tnslsnr/pover2/listener_scan1/alert/log.xmlListening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN1))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.11.150.149)(PORT=1521)))Services Summary...Service "DB2" has 1 instance(s). Instance "DB2", status READY, has 1 handler(s) for this service...Service "DB2XDB" has 1 instance(s). Instance "DB2", status READY, has 1 handler(s) for this service...The command completed successfully-bash-3.00$
Switch the 11gR2 Resource Group
Also note that after a switch over the local_listener is updated by Grid Infrastructure to ensure that the database connects to the local listener on the new target node, e.g.
bash-3.00# clrs status -g ora-11gr2-rg
=== Cluster Resources ===
44
Resource Name Node Name State Status Message------------- --------- ----- --------------ora-db2-rs pfreak1 Online Online pfreak2 Offline Offline
bash-3.00# clrg switch -n pfreak2 ora-11gr2-rg bash-3.00# clrs status -g ora-11gr2-rg
=== Cluster Resources ===
Resource Name Node Name State Status Message------------- --------- ----- --------------ora-db2-rs pfreak1 Offline Offline pfreak2 Online Online
bash-3.00#bash-3.00# uname -n pfreak2bash-3.00#bash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ export ORACLE_HOME=/u01/app/ora11g/product/11.2.0/dbhome_1-bash-3.00$ export ORACLE_SID=DB2-bash-3.00$ $ORACLE_HOME/bin/sqlplus "/as sysdba"
SQL*Plus: Release 11.2.0.1.0 Production on Wed May 19 09:30:09 2010
Copyright (c) 1982, 2009, Oracle. All rights reserved.
Connected to:Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit ProductionWith the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options
SQL> show parameters listener
NAME TYPE VALUE------------------------------------ ----------- ------------------------------listener_networks stringlocal_listener string (DESCRIPTION=(ADDRESS_LIST=(AD
DRESS=(PROTOCOL=TCP)(HOST=frea k-2)(PORT=1521))))
remote_listener string freak-scan-lh:1521SQL> exitDisconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
45
With the Partitioning, Automatic Storage Management, OLAP, Data Miningand Real Application Testing options-bash-3.00$
bash-3.00# uname -n pfreak2bash-3.00# su - oragridSun Microsystems Inc. SunOS 5.10 Generic January 2005
SC Lab Install: COMPLETE - install logs are in /var/sadm/system/logs
-bash-3.00$ lsnrctl status
LSNRCTL for Solaris: Version 11.2.0.1.0 - Production on 19-MAY-2010 09:32:38
Copyright (c) 1991, 2009, Oracle. All rights reserved.
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))STATUS of the LISTENER------------------------Alias LISTENERVersion TNSLSNR for Solaris: Version 11.2.0.1.0 - ProductionStart Date 19-MAY-2010 09:05:45Uptime 0 days 0 hr. 26 min. 53 secTrace Level offSecurity ON: Local OS AuthenticationSNMP OFFListener Parameter File /u01/app/11.2.0/grid/network/admin/listener.oraListener Log File /u01/app/oragrid/diag/tnslsnr/pfreak2/listener/alert/log.xmlListening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.11.188.57)(PORT=1521))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.11.188.59)(PORT=1521)))Services Summary...Service "+ASM" has 1 instance(s). Instance "+ASM2", status READY, has 1 handler(s) for this service...Service "DB2" has 1 instance(s). Instance "DB2", status READY, has 1 handler(s) for this service...Service "DB2XDB" has 1 instance(s). Instance "DB2", status READY, has 1 handler(s) for this service...The command completed successfully-bash-3.00$ -bash-3.00$ crsctl stat res ora.db2.db -t--------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS --------------------------------------------------------------------------------Cluster Resources
46
--------------------------------------------------------------------------------ora.db2.db 1 ONLINE ONLINE pfreak2 Open -bash-3.00$
Final picture
bash-3.00# clrg status
=== Cluster Resource Groups ===
Group Name Node Name Suspended Status---------- --------- --------- ------rac-fmwk-rg pfreak1 No Online pfreak2 No Online
asm-inst-rg pfreak1 No Online pfreak2 No Online
asm-dg-rg pfreak1 No Online pfreak2 No Online
ora-11gr1-rg pfreak1 No Online pfreak2 No Offline
ora-11gr2-rg pfreak1 No Offline pfreak2 No Online
bash-3.00# clrs list -g ora-11gr1-rgora-db1-lhora-db1-lsrora-db1-rsbash-3.00# clrs list -g ora-11gr2-rgora-db2-rsbash-3.00# clrs status
=== Cluster Resources ===
Resource Name Node Name State Status Message------------- --------- ----- --------------rac-fmwk-rs pfreak1 Online Online pfreak2 Online Online
rac-udlm-rs pfreak1 Online Online pfreak2 Online Online
crs-fmwk-rs pfreak1 Online Online pfreak2 Online Online
asm-inst-rs pfreak1 Online Online - +ASM1 is UP and
47
ENABLED pfreak2 Online Online - +ASM2 is UP and ENABLED
asm-dg-rs pfreak1 Online Online - Mounted: DATA pfreak2 Online Online - Mounted: DATA
ora-db1-lh pfreak1 Online Online - LogicalHostname online. pfreak2 Offline Offline - LogicalHostname offline.
ora-db1-lsr pfreak1 Online Online pfreak2 Offline Offline
ora-db1-rs pfreak1 Online Online pfreak2 Offline Offline
ora-db2-rs pfreak1 Offline Offline pfreak2 Online Online
bash-3.00#
48