12c_rac_on_oel7 - · pdf file5/12/2016 · the oracle database 12c r1 for linux...
TRANSCRIPT
12cR1 RAC | OEL 7.2 Page 1
This RAC installation article is based on RAC Attack hands-on learning lab
https://en.wikibooks.org/wiki/RAC_Attack_-_Oracle_Cluster_Database_at_Home/RAC_Attack_12c
The article performs the following task:-
1. Deploying Oracle Grid Infrastructure 12cR1 (12.1.0.2.0)
2. Deploying Oracle database software 12cR1 (12.1.0.2.0)
3. Using Oracle ASM with udev rules
Below the hardware details
Windows 7 64 bit
8 CPUs
16Gb memory, 8GB is used for the RAC
50Gb for VDI (Dynamic Allocation)
30Gb for ASM disk (Fixed Size)
Below are the software components
The VirtualBox for Windows: https://www.virtualbox.org/wiki/Downloads
The Oracle Linux (OEL 7.2 OS -- V100082-01.iso) : https://edelivery.oracle.com/linux
The Oracle Database 12c R1 for Linux x86-64:
http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html
1. Complete the Installation of Oracle VM VirtualBox
2. Run VirtualBox Manager.
3. From the main screen, choose File > Preferences > Network.
4. Double click on VirtualBox Host-Only Ethernet Adapter. VirtualBox will suggest a default
address of 192.168.56.1. Please update the settings as shown and click Ok twice.
12cR1 RAC | OEL 7.2 Page 2
IPv4 Address: 192.168.78.1
IPv4 Network Mask: 255.255.255.0
5. In VirtualBox, from the main screen, click the New icon in the upper left hand corner.
12cR1 RAC | OEL 7.2 Page 3
6. Set the memory size. I have set to 4G.
7. Create virtual hard drive.
12cR1 RAC | OEL 7.2 Page 4
8. Leave VDI selected and click next.
9. Leave dynamically allocated selected and click Next.
12cR1 RAC | OEL 7.2 Page 5
10. Type in the full path or choose through the browse icon the Location for the disk file to be
created. Have named the file “racnode1.vdi” and all the vdi files created are in location
“D:\rac12c\”.
12cR1 RAC | OEL 7.2 Page 6
11. From the main screen, select the virtual machine racnode1 and click the Settings icon in the
upper left hand corner.
12cR1 RAC | OEL 7.2 Page 7
12. Open the Storage sub-menu. Delete the Empty disk under the IDE Controller.
13. Click on the Add CD/DVD/Optical Device.
12cR1 RAC | OEL 7.2 Page 8
14. Locate and open the Oracle_Linux_7_64.iso (V100082-01.iso) file.
12cR1 RAC | OEL 7.2 Page 9
15. Open the Network sub-menu. Under the Adapter 1 tab, change the Attached to: dropdown to
Host-only Adapter.
16. Choose the Adapter 2 tab. Check the box for Enable Network Adapter. Change the Attached to:
dropdown to Internal Network and type the name “rac-priv” in the Name field.
12cR1 RAC | OEL 7.2 Page 10
17. Choose the Adapter 3 tab. Check the box for Enable Network Adapter. Change the
Attached to: dropdown to NAT.
18. Select the Shared Folders tab. Click Add Shared Folder. Enter the path where you have
downloaded the Oracle installation media and enter a name for your folder.
12cR1 RAC | OEL 7.2 Page 11
19. Open the USB sub-menu. Uncheck the Enable USB Controller check box.
20. Click on Start to boot the virtual machine.
12cR1 RAC | OEL 7.2 Page 12
21. The Oracle Linux install screen will automatically come up.
22. Select the language and click on “Continue” .
12cR1 RAC | OEL 7.2 Page 13
23. Next is the “Installation Summary” screen.
24. Select the “Installation Destination”. If you are happy with automatic partitioning of
the whole disk, click “Done” button to return to previous screen.
12cR1 RAC | OEL 7.2 Page 14
25. On “Software Selection” screen select “Server with GUI” for your “Base Environment”.
Then select extra software you want to install by selecting it in “Add-Ons for Selected
Environment”. For Oracle software installation and general development on Oracle
Linux 7 please choose the below option and click “Done” button.
12cR1 RAC | OEL 7.2 Page 15
26. Select the “Network & Host Name” section from “Installation Summary”.
12cR1 RAC | OEL 7.2 Page 16
27. Select Ethernet (enp0s3), the “Public IP” and click on Configure. In earlier releases of
OEL we had eth0, eth1, eth2, which is now enp0s3, enp0s8, enp0s9
a. Click the “Connect Automatically” checkbox in General Tab
b. Click on IPv4 Settings and select “Manual” method.
c. Click “Add” and type “192.168.78.51” for the address and “255.255.255.0” for
Netmask
d. Type 192.168.78.51, 192.168.78.52 in DNS servers and “racattack” in search domain.
e. In Hostname type “racnode1.racattack”.
f. Click on Save
12cR1 RAC | OEL 7.2 Page 17
28. Select Ethernet (enp0s8) and click on “Configure”
a. Click on “Connect Automatically” checkbox in General Tab
b. Move to “IPv4 Settings” , change to “Manual” method and click on “Add”
c. Type “172.16.100.51” in Address and “255.255.255.0” in Netmask
d. Save the settings
12cR1 RAC | OEL 7.2 Page 18
29. Select Ethernet (enp0s9) and click on “Configure”
a. Click on “Connect Automatically” checkbox in General Tab
b. Move to “IPv4 Settings” , change to “Automatic (DHCP) addresses only”
method
12cR1 RAC | OEL 7.2 Page 19
12cR1 RAC | OEL 7.2 Page 20
30. Click on “Begin Installation” and the screen changes to “CONFIGURATION” window.
31. Set the “ROOT PASSWORD” and “USER CREATION” while the installation is going on.
12cR1 RAC | OEL 7.2 Page 21
32. Completion Screen: - After the Oracle Linux is successfully installed, go ahead and
reboot it. After reboot you would need to complete “License Information”.
12cR1 RAC | OEL 7.2 Page 22
12cR1 RAC | OEL 7.2 Page 23
33. Turn off and disable the Firewall. “systemctl” and “systemd” have replaced chkconfig
and init under OEL7.
34. Change the setting of SELinux to “disabled” by editing “/etc/selinux/config”.
12cR1 RAC | OEL 7.2 Page 24
35. Check the “ntpd” status. Either configure NTP, or make sure it is not configured so that
CTSSd can synchronize the times of RAC node. In my installation I de-configured NTP.
# systemctl disable ntpd
# mv /etc/ntp.conf /etc/ntp.conf.orig
36. Verify that all the network interfaces are up.
12cR1 RAC | OEL 7.2 Page 25
37. Configure DNS.
12cR1 RAC | OEL 7.2 Page 26
Original “/etc/named.conf” file before any changes -
Add “192.168.78.51” to “listen-on port 53” and “192.168.78.0/24” to “allow-query” and “allow-
transfer”.
12cR1 RAC | OEL 7.2 Page 27
At the end of file add the “zone” and “reverse zone” as shown below.
Create the zone file for the racattack domain on racnode1.
12cR1 RAC | OEL 7.2 Page 28
Create the reverse zone file
Verify the changes made to “/etc/named.conf” file as shown below. In case of any errors, rectify it
and re-check.
12cR1 RAC | OEL 7.2 Page 29
Enabled the named service and started it
Check the forward and reverse lookup using “nslookup” command.
12cR1 RAC | OEL 7.2 Page 30
12cR1 RAC | OEL 7.2 Page 31
38. Install the database preinstall package “oracle-rdbms-server-12cR1-preinstall”.
Select the “Devices > Optical Drives > oracle_linux_7_64.iso”
12cR1 RAC | OEL 7.2 Page 32
Run the following commands.
Make a backup copy of file, delete the content and add the below lines to /etc/yum.repos.d/public-
yum-ol7.repo.
Install the “oracle-rdbms-server-12cR1-preinstall” package. It performs all prerequisite setup like
installing required rpms, creating oracle user, required groups, setting parameters in
/etc/sysctl.conf etc.
12cR1 RAC | OEL 7.2 Page 33
One thing to note is, Oracle user limit does not reflect values in /etc/security/limit.conf if oracle-
rdbms-server-12cR1-preinstall RPM is used. The file to be checked is “/etc/security/limits.d/oracle-
rdbms-server-12cR1-preinstall.conf”.
12cR1 RAC | OEL 7.2 Page 34
39. Unmount/Eject the device (oracle_linux_7_64.iso) and Make the VirtualBox guest
additions available to the OS by clicking Devices->Install Guest Additions.
12cR1 RAC | OEL 7.2 Page 35
40. After the guest addition, the Shared Folders added in step 18 should be visible. The folder is visible as /media/sf_12cR1.
41. Create the directories in which the Oracle Software will be installed and add oracle to
“vboxsf” group.
12cR1 RAC | OEL 7.2 Page 36
42. Create Virtualbox Shared Storage (ASM Disks)
a. Shutdown the server.
b. In the VirtualBox Manager, select the machine racnode1, click Settings -> Storage.
c. Select Controller SATA and click on the Add Hard Disk button. Click “Create new disk”.
12cR1 RAC | OEL 7.2 Page 37
12cR1 RAC | OEL 7.2 Page 38
43. Repeat the same steps to create three more disks:
44. Make all the asm devices sharable: from VirtualBox Manager, click File -> Virtual Media
Manager.
12cR1 RAC | OEL 7.2 Page 39
Repeat the same above steps for all the asm*.vdi disk.
12cR1 RAC | OEL 7.2 Page 40
45. Start the racnode1 and you must see 4 disks named /dev/sd*.
46. Once the new disks are visible in the server, add primary partition on each of them using
“fdisk”.
12cR1 RAC | OEL 7.2 Page 41
47. Add the following to the "/etc/scsi_id.config" file to configure SCSI devices as trusted.
Create the file if it doesn't already exist.
# echo “options=-g” > /etc/scsi_id.config
Check if scsi_id returns data.
# /usr/lib/udev/scsi_id –g –u –d /dev/sdb
Prepare the “/etc/udev/rules.d/99-oracle-asmdevices.rules” by running the below script.
#!/bin/bash
#file ~/createOracleAsmUdevRules.sh
i=1
# ol7 / rhel7
cmd="/usr/lib/udev/scsi_id -g -u -d"
for disk in sdb sdc sdd sde ; do
cat <<EOF >> /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="$cmd /dev/\$parent", \
RESULT=="`$cmd /dev/$disk`", SYMLINK+="asm-disk$i", OWNER="oracle", GROUP="dba", \
MODE="0660"
EOF
i=$(($i+1))
done
12cR1 RAC | OEL 7.2 Page 42
Load updated block device partition tables.
Test the rules are working as expected.
Restart UDEV Service.
12cR1 RAC | OEL 7.2 Page 43
48. Clone VirtualBox VM
a. Shutdown the VM.
# shutdown –h now
b. Clone the disk racnod1.vdi: from VirtualBox Manager, click File -> Virtual Media
Manager.
12cR1 RAC | OEL 7.2 Page 44
12cR1 RAC | OEL 7.2 Page 45
Once the copy is complete, click Close to exit from the Virtual Media Manager.
12cR1 RAC | OEL 7.2 Page 46
49. In VirtualBox, click New icon in the upper left corner to create the new VM.
12cR1 RAC | OEL 7.2 Page 47
12cR1 RAC | OEL 7.2 Page 48
50. Open the Network sub-menu. Under the Adapter 1 tab, change the Attached to:
dropdown to Host-only Adapter.
51. Choose the Adapter 2 tab. Check the box for Enable Network Adapter. Change the
Attached to: dropdown to Internal Network and type the name “rac-priv” in the Name
field.
52. Choose the Adapter 3 tab. Check the box for Enable Network Adapter. Change the
Attached to: dropdown to NAT.
12cR1 RAC | OEL 7.2 Page 49
53. Open the USB sub-menu. Uncheck the Enable USB Controller check box.
54. Select the Shared Folders tab. Click Add Shared Folder. Enter the path where you have downloaded the Oracle installation media and enter a name for your folder.
12cR1 RAC | OEL 7.2 Page 50
55. Click on Controller SATA and “Choose existing disk” to add the asm disk vdi file.
56. Repeat the operation to add all asm disks and then click OK to save the modifications.
12cR1 RAC | OEL 7.2 Page 51
57. Review the summary of new virtual machine.
58. Start the “racnode2” and login as root.
12cR1 RAC | OEL 7.2 Page 52
59. On the top right had side end you would see “Ethernet” setting. Choose one and click
on “Wired Settings”.
60. Below is the screen-shot for enp0s3 (Public IP). Click on the right end bottom mark.
12cR1 RAC | OEL 7.2 Page 53
61. Change the Address to “192.168.78.52” for enp0s3.
62. In the similar way change the Address to “172.16.100.52” (Private-IP) for enp0s8.
12cR1 RAC | OEL 7.2 Page 54
63. For enp0s9 uncheck “Connect Automatically”
64. On terminal as root type “nmtui” (NetworkManager Text User Interface)
65. Change the Hostname
12cR1 RAC | OEL 7.2 Page 55
66. Delete the UUID entry for ifcfg-enp0s[389].
# sed –i –e ‘/UUID/d’ ifcfg-enp0s[389]
12cR1 RAC | OEL 7.2 Page 56
67. Reboot the server and then verify all the network interfaces are up and ASM devices are
visible.
68. Check the firewall status (should be inactive) and stop the DNS.
12cR1 RAC | OEL 7.2 Page 57
69. Delete the “racattack.zone” and “racattack.revzone” from /var/named.
70. Edit the “/etc/named.conf” and change .51 to .52 and the end of the file as below.
12cR1 RAC | OEL 7.2 Page 58
71. Review the “/etc/resolv.conf”.
72. Start DNS and check the forward and reverse lookup using “nslookup” command.
12cR1 RAC | OEL 7.2 Page 59
73. Check that both the master on racnode1 and slave on racnode2 DNS servers are
working. Run all the below statements to verify correct configuration of both DNS
servers :-
[root@racnode2 ~]# dig @racnode1 racnode1.racattack
[root@racnode2 ~]# dig @racnode1 racnode2.racattack
[root@racnode2 ~]# dig @racnode1 racnode1-vip.racattack
[root@racnode2 ~]# dig @racnode1 racnode2-vip.racattack
[root@racnode2 ~]# dig @racnode1 racnode1-priv.racattack
[root@racnode2 ~]# dig @racnode1 racnode2-priv.racattack
[root@racnode2 ~]# dig @racnode1 rac-cluster-scan.racattack
[root@racnode2 ~]# dig @racnode2 racnode1.racattack
[root@racnode2 ~]# dig @racnode2 racnode2.racattack
[root@racnode2 ~]# dig @racnode2 racnode1-vip.racattack
[root@racnode2 ~]# dig @racnode2 racnode2-vip.racattack
[root@racnode2 ~]# dig @racnode2 racnode1-priv.racattack
[root@racnode2 ~]# dig @racnode2 racnode2-priv.racattack
[root@racnode2 ~]# dig @racnode2 rac-cluster-scan.racattack
12cR1 RAC | OEL 7.2 Page 60
74. When both nodes have started, check they can both ping all the public and private IP
addresses using the following commands.
12cR1 RAC | OEL 7.2 Page 61
75. Install the Grid Infrastructure
a. Install the following package for the Oracle Grid media as root user on both the nodes.
b. Setup the passwordless ssh for as Oracle user.
./sshUserSetup.sh –user oracle –hosts “racnode1 racnode2” –noPromptPassphrase –advanced –
exverify –logfile /home/oracle/sshsetup.log
c. At this point you can run the “runcluvfy.sh” (optional)
12cR1 RAC | OEL 7.2 Page 62
d. Run the Grid Infrastructure runInstaller as Oracle
[oracle@racnode1 grid] ./runInstaller
e. Below are the screen-shots for the configuration.
12cR1 RAC | OEL 7.2 Page 63
12cR1 RAC | OEL 7.2 Page 64
f. In the SCAN Name box, enter “rac-cluster-scan.racattack”. In the central box, only the first
node is displayed. Click on Add and enter the network names of the second node.
Public Hostname: - “racnode2.racttack”
Virtual Hostname: - “racnode2-vip.racattack”
Since the SCAN Name chosen is more than 15 character, Oracle proposes different name as Cluster
Name. Click Yes.
12cR1 RAC | OEL 7.2 Page 65
g. Check the correct addresses are set as below.
h. Make sure the “Software Location” is correct and change “Cluster Registry Storage Type”
to Oracle Automatic Storage Management. Enter SYSASM password and confirm. Click Yes.
12cR1 RAC | OEL 7.2 Page 66
i. Set the redundancy to “external”, click the “Change Discovery Path” button and set the
path to “/dev/asm-disk*”. I have changed the disk group name to DATA01.Once completed
click on “Next”.
12cR1 RAC | OEL 7.2 Page 67
j. If you want the root script to run automatically, enter the relevant credentials. I prefer to
run them manually.
12cR1 RAC | OEL 7.2 Page 68
k. Wait while the prerequisite checks complete.
l. The below checks can be ignored for this type of installation.
12cR1 RAC | OEL 7.2 Page 69
m. Save the response file and review the summary. Click the “Install” button.
n. Installation progress
12cR1 RAC | OEL 7.2 Page 70
o. When prompted, run the configuration scripts in each node. Once both the scripts are run
on all the nodes, click “OK”.
12cR1 RAC | OEL 7.2 Page 71
p. The installation continues and may take longtime to complete. “MGMTDB” creation may
take some time.
12cR1 RAC | OEL 7.2 Page 72
q. Optional step: - You can increase the timeout of CRS before it causes fencing of the node.
12cR1 RAC | OEL 7.2 Page 73
r. You can check the crs patch details.
76. As oracle user add new diskgroup for FRA
The asmca opens on the Disk Groups tab, showing the DATA01 Diskgroup created by the
installer. Click on create and enter “FRA” in Disk Group Name. Choose “External (node)”
redundancy. Check one disk to create 10GB diskgroup. Click “OK”
12cR1 RAC | OEL 7.2 Page 74
77. After the diskgroup is created
78. Install Database Software
a. You can use “cluvfy” to perform prerequisite checks (Optional)
cluvfy stage –pre dbinst –n “racnode1,racnode2” –d /u01/app/oracle/product/12.1.0.2/db_1 -verbose
12cR1 RAC | OEL 7.2 Page 75
b. Make sure all nodes are started and then login to racnode1 as oracle user and
start the Oracle Installer.
12cR1 RAC | OEL 7.2 Page 76
c. Make sure both nodes are selected, then click “Next” button
12cR1 RAC | OEL 7.2 Page 77
12cR1 RAC | OEL 7.2 Page 78
d. Make sure the “Oracle Base” and “Software Location” are as needed
12cR1 RAC | OEL 7.2 Page 79
e. For my installation I ignored both the warnings.
12cR1 RAC | OEL 7.2 Page 80
f. Review the summary information and click the “Install” button.
g. Run the configuration script on each node. When the script has been run on each
node, click “OK” button.
12cR1 RAC | OEL 7.2 Page 81
79. Create Database
a. Run the Database Configuration Assistance (DBCA)
12cR1 RAC | OEL 7.2 Page 82
b. Global Database Name:- cdbrac, Pluggable Database Name:- PDB
12cR1 RAC | OEL 7.2 Page 83
c. Review the “Create Database” Summary and Click “Finish”
12cR1 RAC | OEL 7.2 Page 84
12cR1 RAC | OEL 7.2 Page 85