vmwareandoracleebsdeploymentguidefinal

Contents

Deployment of Oracle E-Business Suite on VMware Infrastructure

Desmond Chan Senior ISV Solution Engineer

Product Solutions and Marketing VMware Inc.

Contents

Acknowledgements ......................................................................................................1

1. Introduction ..........................................................................................................2

2. Oracle E-Business Suite .......................................................................................3

2.1. Oracle E-Business Suite Architecture...................................................................................................................3

3. VMware Infrastructure ........................................................................................5

3.1. VMware® ESX Server.......................................................................................................................................................6 3.2. VMware® VirtualCenter.................................................................................................................................................7

4. Benefits of Using VMware Infrastructure.........................................................8

4.1. Server Containment.......................................................................................................................................................8 4.2. Rapid Provisioning..........................................................................................................................................................9 4.3. Change Management ...............................................................................................................................................10 4.4. High Availability: Business Continuity and Disaster Recovery...........................................................11

5. Oracle EBS Deployment on VMware Infrastructure.................................... 13

5.1. Deployment Environment......................................................................................................................................14 5.2. Software Summary ......................................................................................................................................................15 5.3. Hardware Summary ....................................................................................................................................................15 5.4. Deployment Approach Summary......................................................................................................................16

6. VMware Infrastructure Installation................................................................ 18

6.1. ESX Server Host Configuration.............................................................................................................................18 6.2. VMware Virtual Machine Installation ................................................................................................................21 6.3. Guest Operating System Installation................................................................................................................22 6.4. VMware® Tools Installation .....................................................................................................................................24 6.5. Virtual Machine Time Synchronization ...........................................................................................................25 6.6. Add a Virtual Disk..........................................................................................................................................................26

7. Oracle E-Business Suite Installation............................................................... 28

7.1. Rapid Install ......................................................................................................................................................................28 7.2. Oracle Installation Environment..........................................................................................................................32

8. Gold Image for Application Tier Deployment.............................................. 45

8.1. Oracle Rapid Clone......................................................................................................................................................45 8.2. Gold Image Creation ..................................................................................................................................................45

9. VMware High Availability ................................................................................ 55

Contents

9.1. Network Card Redundancy ....................................................................................................................................55 9.2. Configure HA Cluster .................................................................................................................................................56

Contents

Contents

10. VMware VMotion ......................................................................................... 58

10.1. Configure VMKernel Port .........................................................................................................................................58 10.2. Live Migration.................................................................................................................................................................59 10.3. Cold Migration ...............................................................................................................................................................59

11. Best Practices ................................................................................................ 61

12. Conclusion..................................................................................................... 63

13. Resources....................................................................................................... 64

13.1. VMware...............................................................................................................................................................................64 13.2. Oracle...................................................................................................................................................................................64

14. Appendix 1 – Useful Commands............................................................... 65

14.1. Application Tier Commands..................................................................................................................................65

VMware White Paper Deployment of Oracle E-Business Suite on VMware Infrastructure

Acknowledgements Sun Microsystems, in particular the following individuals, has provided extraordinary support to realize this effort: Ronald Graham, x64 Product Specialist, Sun Microsystems Victor Pantaleon, Systems Administrator, Sun Microsystems The following VMware team members have offered valuable support in this effort: Rajagopal Ramanujam, Solution Architect, Partner Solutions and Marketing Christopher Rimer, Director, Oracle Alliances, Product Marketing Ronald Kwan, Senior Systems Engineer, Sun Alliance Tushar Patel, Staff Systems Engineer, Systems Engineers Johanna Holopainen, Senior Manager of ISV Alliances, Product Marketing Tamao Nakahara, Alliances Marketing Specialist, Alliances Christine Holland, ISV Alliance Marketing Communications Manager, Alliances

Acknowledgements 1


1. Introduction Oracle E-Business Suite (EBS) is a fully integrated, comprehensive suite of business applications for the enterprise. It provides information for effective decision-making and enables an adaptive enterprise for optimal responsiveness. Oracle EBS provides a collection of business applications including:

• ERP (Enterprise Resource Planning) • CRM (Customer Relationship Management) • SCM (Supply Chain Management)

VMware is the global leader in virtual infrastructure software for industry-standard systems. The world's largest companies use VMware solutions to simplify their IT, fully leverage their existing computing investments and respond faster to changing business demands. VMware provides a robust, well-tested and high performance software suite for customers to reap benefits ranging from server containment, rapid provisioning, and high availability to change management.

Figure 1. VMware Infrastructure The multi-tier architecture of Oracle E-Business Suite makes it a good candidate for virtualization. Through careful design, you can choose to virtualize certain tiers or all tiers of the software suite. These virtualization efforts will provide lower total cost of ownership (TCO) and higher return on investment (ROI). This paper addresses the deployment of Oracle EBS on VMware Infrastructure 3, providing details on creating virtual machines, installing Oracle EBS components, creating a gold image to streamline provisioning of new EBS environments. It also discusses the details on how to leverage advanced features such as VMware VMotionTM and High Availability (HA).

Introduction 2


2. Oracle E-Business Suite Oracle E-Business Suite (EBS) is an integrated software package containing ERP (Enterprise Resource Planning), CRM (Customer Relationship Management) and SCM (Supply Chain Management) applications developed by Oracle Corporation. It utilizes Oracle’s core Relational Database Management System (RDBMS) technology. Oracle EBS is also referred to as Oracle Applications. Examples of these applications include Oracle Financials, Oracle Logistics, Oracle HR and Oracle Sales.

2.1. Oracle E-Business Suite Architecture Oracle E-Business Suite architecture is a framework for multi-tiered distributed computing. In the Oracle E-Business Suite model, services are distributed among multiple levels or tiers. A service is a process or a group of processes that exercise some business logic and provide a particular functionality. A tier is a grouping of services that can potentially span physical machines. In other words, a tier is a logical grouping of services that is not limited by physical nodes or machines. Each tier can consist of one or more nodes, and each node can accommodate more than one tier. For example, a single machine can contain database, application, and desktop tiers; in the same way, a database can reside on one of many application servers or on a separate machine by itself.

Figure 2. Oracle E-Business Suite Architecture (3 Tiers)

2.1.1. Desktop Tier

The desktop tier is normally the client web browser. The user interface for the desktop tier is provided through HTML for Self-Service applications and a JavaTM applet for forms. The desktop client is installed on demand upon first use and then stored locally for future use. The forms client applet is packaged as JAR files and mainly represents the presentation layer of Oracle forms. The JAR files are also downloaded on the desktop during first use. The forms client must run within a Java Virtual Machine (JVM) and is supplied by Oracle JInitiator.

Oracle E-Business Suite 3


Oracle E-Business Suite 4

2.1.2. Application Tier

The application tier hosts one or more servers that process the business logic and it communicates between the desktop tier and database. The application tier, also commonly known as the middle tier, includes following major components: Web Server, Forms Server, Concurrent Processing Server, Business Intelligence System, Reports Server, Admin Server, and Discover Server. These components usually run as services on the applications tier and may run on one or more nodes. For more details about the components, refer to Oracle Applications Concepts, Part No: B10642-01.

2.1.3. Database Tier

The database tier contains the RDBMS Oracle home, along with the Applications database, which stores all the data maintained by Oracle Applications. It also contains Oracle Applications processing code stored inside the Applications database to optimize performance. In essence, the database tier stores the Oracle database files, the Applications database, and executables. The database does not directly communicate with desktop clients but rather works with applications tier services that mediate the communication between the desktop and database tier.


3. VMware Infrastructure VMware Infrastructure simplifies IT environments so that you can leverage your storage, network, and computing resources to control costs and respond quickly to changing business needs. The VMware Infrastructure approach to IT management creates virtual services out of the physical infrastructure, enabling administrators to allocate these virtual resources quickly to the business units that need them most.

Figure 3. VMware Infrastructure

VMware Infrastructure 5


VMware Infrastructure 3 is the next generation of industry-leading infrastructure virtualization software that virtualizes servers, storage, and networking, allowing multiple unmodified operating systems and their applications to run independently in virtual machines while sharing physical resources. This deployment guide explains the installation and configuration of two VMware Infrastructure 3 software components:

3.1. VMware® ESX Server VMware® ESX server abstracts processor, memory, storage, and networking resources into multiple virtual machines, giving IT greater hardware utilization and flexibility.

Figure 4. VMware Infrastructure 3 - ESX Server



3.2. VMware® VirtualCenter VMware® VirtualCenter enables rapid provisioning of virtual machines and performance monitoring of physical servers and virtual machines. VirtualCenter intelligently optimizes resources, ensures high availability to all applications in virtual machines, and makes IT environments more responsive with virtualization-based distributed services such as VMware® High Availability (HA) and VMware VMotion.

Figure 5. VMware VirtualCenter Management Server



4. Benefits of Using VMware Infrastructure VMware software delivers capabilities that streamline the deployment and maintenance of Oracle E-Business Suite in a virtualized environment. Some of these key capabilities include:

• Server containment • Rapid provisioning • Change management • High availability: business continuity and disaster recovery

4.1. Server Containment Traditional Oracle EBS deployments generate significant server sprawl due to the need to provision separate systems for development, test and production environments. Each developer requires a dedicated environment and each test cycle requires dedicated servers assigned for the duration of the tests. Along with the development requirements, the necessity of dedicated systems for each layer of the Oracle environment results in over-provisioning, manageability, and resource challenges all leading to higher operational and ownership costs. VMware virtualization technology contains server sprawl and increases server utilization by running multiple Oracle EBS components in virtual machines consolidated onto fewer, highly scalable, reliable enterprise-class systems. Customers using VMware Infrastructure have been able to consolidate ten or more servers per physical processor, and have seen many other benefits as outlined below.

Figure 6. Server Containment

Benefits of Using VMware Infrastructure 8


Key Benefits: 1. Provide dedicated and isolated environments for all developers, consolidated onto one physical

system.

2. Run multiple Oracle EBS components on the same physical system, providing consolidation benefits such as less hardware, datacenter space, power consumption, and so forth, while lowering total cost of ownership (TCO).

3. Increase the average utilization of physical servers leading to efficient capacity utilization.

4. Eliminate the need for dedicated hardware and provide interoperability with multiple operating systems and Oracle application versions residing on the same system.

5. Eliminate the need for dedicated test systems with multiple test environments sharing the same physical server.

4.2. Rapid Provisioning VMware virtualization solutions significantly reduce the time required to provision new instances (development, test, or production) of Oracle EBS environments. In a non-virtualized environment, a new Oracle EBS deployment typically requires the procurement of new hardware, followed by installation and configuration of the operating system and the Oracle application software. This process takes significant time and IT resources. Users of VMware Infrastructure can take advantage of Virtual Machine Libraries and templates to provision new pre-configured Oracle EBS environments in minutes. These features enable rapid Oracle EBS deployment with sophisticated automation capabilities, centralized administration and management of hardware resources while giving business units and Oracle application owners complete control over Oracle application resource utilization.

Figure 7. Rapid Provisioning



Key Benefits: 1. Rapidly provision new Oracle component instances from virtual machine templates.

2. Pass Oracle development and test images back and forth between test and development users.

3. Recreate distributed Oracle component instances from a production environment on a single physical system for test purposes.

4. Move test/QA instances to production in minutes.

5. Reset test images (after test completion) from templates and virtual machine libraries, cutting down on test setup and reset time.

6. Store different Oracle component instances and versions in virtual machine libraries that can be provisioned instantly.

7. With the VMware® Snapshot feature, roll back development and test images during problem resolution.

8. Rapidly provision additional Oracle application server instances during peak loads.

4.3. Change Management Testing patches and upgrades are high on the list of IT challenges facing Oracle customers. IT departments face two key change management challenges: • Testing patches and upgrades for compatibility with standard corporate hardware, operating system,

and Oracle configurations. • Efficiently deploying critical patches and upgrades throughout the enterprise.

Traditionally, IT organizations procure hardware that mirrors production and create test beds that mirror the operating system and Oracle configurations of the production environment. With VMware Infrastructure, users can clone a set of production virtual machines, or create a set of virtual machine libraries that mirror production. They can use these images to provision the test environment and test the latest patches and upgrades against Oracle enterprise applications, eliminating the need for dedicated hardware to perform these tests. The patches can then be rolled into production with minimal interruption to end users. In case of Oracle application problems, the virtual machines can be instantly rolled back to their pre-patched state using Snapshots. Key Benefits: 1. Accelerate change management with fewer system resource requirements.

2. Test patches on multiple configurations (guest operating system, Oracle EBS versions, etc.) concurrently all hosted on the same physical system.

3. Instantly roll back Oracle virtual machines (during problem resolution) using VMware Snapshots.

4. Create a library of standard production configurations to perform change management testing and deployment.

5. Dynamically migrate Oracle server instances on virtual machines to other systems to perform hardware maintenance or system changes on the current physical system, without disruption to end users.



4.4. High Availability: Business Continuity and Disaster Recovery VMware virtualization solutions work alongside Oracle application capabilities to deliver enhanced infrastructure and high availability for critical business functions. Using VMware Infrastructure, you can implement a unified disaster recovery (DR) platform that allows production Oracle instances on virtual machines to be recovered in the event of hardware failure, without investing in costly one-to-one mapping of production and DR hardware. A typical Oracle EBS solution consists of multiple layers and instances running simultaneously and working together to provide failover and load balancing. VMware VMotion enhances this capability by enabling the live migration of running virtual machines from one physical server to another with zero downtime, continuous service availability, and complete transaction integrity. Live migration of virtual machines allows companies to perform hardware maintenance without scheduling downtime and disrupting business operations. VMware High Availability provides easy to use, cost-effective high availability for Oracle applications running in virtual machines. In the event of physical server failure, affected virtual machines are automatically restarted on other physical servers that have spare capacity. VMware HA minimizes downtime and IT service disruption while eliminating the need for dedicated stand-by hardware and installation of additional software. VMware HA provides uniform high availability across the entire virtualized IT environment without the cost and complexity of failover solutions tied to either operating systems or specific Oracle applications. VMware® Consolidated Backup provides an easy to use, centralized facility for LAN-free backup of virtual machines. Consolidated Backup simplifies backup administration and reduces the load for ESX Server host machines.

Figure 8. VMware High Availability

Key Benefits: 1. Save layers of images for regression (that is, keep exact versions of operating system, Oracle

applications, patches, state etc.) using Snapshots and Consolidated Backup.




2. Use Snapshots to enable point-in-time restores and rollbacks during test and development problem resolution.

3. Automatically detect failure of physical servers running Oracle application instances in production, and automatically restart failed Oracle application virtual machines with VMware HA.

4. Ensure capacity availability to support Oracle application virtual machine failovers.

5. Perform full and incremental file backup of virtual machines with Consolidated Backup.

6. Migrate Oracle application virtual machines from failing server hardware without disruption to end users with VMotion live migration.

7. Fail over during disaster recovery by using SAN replication and restarting Oracle application virtual machines in DR sites.


5. Oracle EBS Deployment on VMware Infrastructure Oracle E-Business Suite is a good candidate for deployment on VMware Infrastructure. One major benefit is the ability to leverage the cloning features provided by VMware Infrastructure, along with Oracle Rapid Clone, to achieve rapid provisioning. NOTE: The approach illustrated in this section serves as a suggestion and is by no means the recommended approach for Oracle EBS deployment. Refer to Oracle documentation for more information.

Database Tier

Application Tier

Desktop Tier

Web Clients

Internet

Ethernet

Ethernet

Database Servers

Admin ServersConcurrent Managers and Reports Servers

Forms ServersWeb Servers

Figure 9. Oracle E-Business Suite Deployment Sample

Oracle EBS Deployment on VMware Infrastructure 13


5.1. Deployment Environment Figure 10 shows the deployment environment that was used in the Sun lab located in Menlo Park. Two ESX Server hosts were installed on Sun Fire X4600 M2 Servers and both servers were connected to a SAN. A VirtualCenter Management Server was installed on a Sun Fire X4100 Server. A VirtualCenter Client was installed on a laptop.

10.6.160.16310.6.160.161

Ethernet

Ethernet

ESX Server hosts on Sun Fire X4600 M2 Servers

VirtualCenter Management Server on Sun Fire X4100 Server

VirtualCenter Client on a laptop

SAN

Figure 10. Sample VMware Infrastructure Environment for Oracle EBS deployment



5.2. Software Summary The table below contains the software used in the deployment outlined in this guide.

Category

Software

Infrastructure

VMware Infrastructure 3 (VI3)

Applications

Oracle E-Business Suite Release 12

Guest Operating System

Red Hat Enterprise Linux AS 4.0

5.3. Hardware Summary The table below outlines the hardware used to support the virtualized Oracle EBS installation.

Category

Hardware

ESX Server hosts

Sun Fire X4600 M2 servers

VirtualCenter Management Server

Sun Fire X4100 servers

VirtualCenter Client

IBM X60 Lenovo laptop computer




5.4. Deployment Approach Summary The approach outlined here showcases ways to streamline the deployment of Oracle E-Business Suite, by using a gold image that has pre-installed and pre-configured Oracle software. By cloning from the gold image, you can efficiently deploy an Oracle EBS environment with respect to your architecture plans. At the end of this exercise, you will have an environment of two virtual machines, one serving as an application server and a database server while the other serves solely as an application server.1 Deployment steps include: 1. Prepare a VMware Infrastructure environment

a. Procure hardware for two ESX Server hosts, one VirtualCenter Management Server and one VirtualCenter Client.

b. Install ESX Server on the host machines. c. Install VirtualCenter Management Server. d. Install VirtualCenter Client.

2. Create a virtual machine a. Create a virtual machine on one ESX Server host. b. Add the necessary virtual disks to the virtual machine: one for application tier, the

other for database tier. c. Install guest operating system (RHEL AS 4.0) on the virtual machine.

3. Install Oracle EBS environment a. Stage Oracle EBS software on the original virtual disk /dev/sda. b. Prepare Oracle installation environment (for example, create user ‘oracle’). c. Launch Oracle Rapid Install.

4. Create gold image (virtual machine template) a. Clone the virtual machine created without the virtual disk where the database tier is

installed.2 b. Launch the cloned virtual machine. c. Run Oracle AutoConfig on the application tier. d. Run Oracle adpreclone.pl on the application tier. e. Clone the second virtual machine to a template.

5. Configure the cloned virtual machine a. Configure the network interfaces. b. Run Oracle adcfgclone.pl to configure the cloned virtual machine. c. Run Oracle adadd.

6. Configure VMware High Availability cluster a. Configure NIC teaming. b. Configure redundant service console port.

7. Configure VMware VMotion a. Configure VMkernel Port. b. Perform Live Migration. c. Perform Cold Migration.

1 The second virtual machine that hosts an application server only is created from a virtual machine template. You can easily use the virtual machine template to clone other virtual machines for the application tier. The following sections show the corresponding procedures. 2 The virtual disk where the database tier is installed is not needed for application tier cloning. As a result, the virtual disk is not included in the gold image.


Virtualize Oracle EBS

Prepare a VMware Infrastructure environment

Create a virtual machine

Install Oracle EBS environment

Create gold image

Clone virtual machine

Configure VMware High Availability

Configure VMware VMotionTM

Enhance Availability

Use Gold Image for Rapid Provisioning

Figure 11. Oracle E-Business Suite Deployment Approach



6. VMware Infrastructure Installation VMware Infrastructure consists of a number of components: • VMware ESX Server • VirtualCenter Management Server • VirtualCenter Client • License Server

The installation procedures for each component are discussed in detail in the guide located here: http://www.vmware.com/pdf/vi3_installation_guide.pdf.

6.1. ESX Server Host Configuration

• Configure at least one datastore on a shared storage. In the Sun lab, a datastore called OracleVMFS of size 1TB was created on the shared storage.

• Synchronize the clocks on both ESX Server hosts.

6.1.1. Add New Physical Shared Storage

In order to add a shared storage system, such as a new LUN, after ESX Server has been installed, follow the procedure below to rescan and add the shared storage system to the infrastructure using VirtualCenter Client. • Select the desired ESX Server host in Inventory View. • Click on the Configuration tab. • Click on Add Storage… on the top right corner. • In the Add Storage Wizard, enter the following: Disk/LUN Select Disk/LUN Device Location

A list of available LUNs should appear in the window to the right. Select the device to add.

Current Disk Layout

(Indication that the disk is blank should be shown) Click on Next. Datastore Name Properties OracleVMFS

Formatting Keep the defaults. Make sure that the checkbox for Maximize Capacity has been checked.

Ready to Complete

Click on Finish.

6.1.2. ESX Server Host Time Synchronization

It is in general a good practice to have the clock of an ESX Server host synchronized with a trusted source. You can then use the synchronized clock as the source to synchronize the clocks on virtual machines. NTP is installed by default on ESX Server 3.0.1. To enable it, follow the procedure below: • Log in to the ESX Server service console. • Edit the configuration files (/etc/ntp.conf, /etc/ntp/step-tickers, /etc/hosts) on the service

console.


http://www.vmware.com/pdf/vi3_installation_guide.pdf


• Allow NTP client to go through the firewall. • Restart NTP daemon.

Configuration File Modification

Edit the configuration files according to the table listed below:

File Name Suggested Content Reference

/etc/ntp.conf restrict 127.0.0.1 restrict default kod nomodify notrap server 0.pool.ntp.org server 1.pool.ntp.org server 2.pool.ntp.org driftfile /var/lib/ntp/drift

• http://www.pool.ntp.org/use.html for using NTP server pools

• http://www.eecis.udel.edu/~mills/ntp/html/accopt.html for a full description of the access control commands.

• http://www.eecis.udel.edu/~mills/ntp/html/notes.html for a more complete definition of the drift file.

/etc/ntp/step-tickers 0.pool.ntp.org

1.pool.ntp.org 2.pool.ntp.org pool.ntp.org

/etc/hosts

IP_ADDR 0.pool.ntp.org IP_ADDR 1.pool.ntp.org IP_ADDR 2.pool.ntp.org IP_ADDR pool.ntp.org

Adding these entries to /etc/hosts saves unnecessary DNS queries. It is an optimization step.


http://www.pool.ntp.org/use.html

http://www.pool.ntp.org/use.html

http://www.eecis.udel.edu/%7Emills/ntp/html/accopt.html



http://www.eecis.udel.edu/%7Emills/ntp/html/notes.html





Open NTP Client Port

There are two alternative ways to open the NTP client port: 1. By running the ESX Server configuration command on ESX Server service console.

o Run “esxcfg-firewall --enableService ntpClient”

2. By using VirtualCenter Client. o Select the ESX Server host in Inventory View. o Go to Configuration Tab. o Click on Security Profile. o Click on Properties in the upper right corner of the screen. o Check NTP Client.

Figure 12. Firewall Properties Sheet for enabling NTP Client


Restart and Monitor NTP Service

To restart the NTP daemon for time synchronization to take effect, follow the procedure below: • Log in as root to the service console. • Execute the following commands:

Commands Effect # service ntpd restart

Restart the service.

# chkconfig --level 345 ntpd on

Enable the NTP daemon to autostart when the server is rebooted.

# hwclock --systohc

Set the local hardware clock to the NTP synchronized local system time.

6.2. VMware Virtual Machine Installation Follow the procedure below to add a virtual machine to an ESX Server host, using the VirtualCenter Client: • Host > New Virtual Machine • In the New Virtual Machine Wizard, enter the following:

Virtual Machine Configuration Wizard Type Typical Virtual Machine Name: Name and Folder VOEBS

Datastore Select a datastore that is visible to both ESX Server hosts. In this scenario, [OracleVMFS] is chosen. Guest Operating System: Linux Version:

Guest Operating System

Red Hat Enterprise Linux 4 Number of Virtual Processors: CPUs 2

Memory 2 GB Number of NICs: 1 NIC 1:

Network

VM Network/ Connect at Power On Disk Size: Virtual Disk

Capacity 64 GB Ready to Complete

Click on Finish




6.3. Guest Operating System Installation Follow the procedure below to install a guest operating system, in this case Red Hat Enterprise Linux (RHEL 4 AS), on a newly added virtual machine, using the VirtualCenter Client: • Power on the virtual machine VOEBS. • At the prompt for boot, enter the desired Linux version to install: rhel4u2as. • Connect the Virtual CD ROM to an ISO Image or the local CD ROM drive.

o Click on the button Virtual CDROM (ide0:0) and follow the instructions. • Select the language/keyboard type according to your settings. • When prompted for installation method, select “Local CDROM.” • In the installation wizard, enter the following:

Welcome Page: Click on Next Disk Partition Setup: Manually partition with Disk Druid. Configure a mount point

/u01 with at least 40 GB for Oracle software staging. Boot Loader Configuration

Leave as default.

For the HA scenario, only one Ethernet card is needed. Etho0: 10.6.160.165 /255.255.252.0 Host Name: voebs

Network Configuration

DNS settings: Enter proper DNS servers and default gateway. Firewall Configuration:

Leave as default.

Additional Language Support

Leave as default.

Time Zone Selection Select desired time zone. Set Root Password Select desired root password. Package Installation Selection

Customize software packages to be installed. • Select X Window System. • Select GNOME Desktop Environment. • Select Editors.

o Click on Details and select preferred text editor. • Select Graphical Internet. • Select Text-based Internet. • Select Office/Productivity. • Select Sound and Video. • Select Graphics. • Select Server Configuration Tools. • Select FTP Server. • Select Legacy Network Server.

o Click on Details. o Select rsh-server. o Select telnet-server.

• Select Development Tools. • Select Legacy Software Development. • Select Administration Tools. • Select System Tools.

o Click on Details.


o Select sysstat. • Select Printing Support.

About to Install Click on Next.



6.4. VMware® Tools Installation VMware® Tools provides extra drivers to improve the performance of graphics and networking. In addition, it includes software packages providing services such as time synchronization. To install VMware Tools, follow the procedure below: • Select the virtual machine on which you want to install VMware tools. • Right-click on the virtual machine in Inventory view and select Install VMware Tools. • Mount CD-ROM (e.g. mount /dev/cdrom /mnt/cdrom). • In /mnt/cdrom, there is an rpm file and a tar.gz file::

o VMwareTools-3.0.1-32039.i386.rpm o VMwareTools-3.0.1-32039.tar.gz

• Use the command “rpm –Uvh VMwareTools-3.0.1-32039.i386.rpm” to install the package. • After the package has been installed, run /usr/bin/vmware-config-tools.pl.

o Follow the prompted instructions to install the network drivers as necessary.

NOTE: For more information about VMware tools, refer to http://www.vmware.com/pdf/server_vm_manual.pdf.


http://www.vmware.com/pdf/server_vm_manual.pdf


6.5. Virtual Machine Time Synchronization VMware recommends that you perform time synchronization on the virtual machines. The following procedure shows how to use VMware Tools to achieve time synchronization: • Launch VMware Tools in the guest operating system.

o Log in as root. o Launch x-windows (e.g. startx). o Start a terminal. o In the terminal, run the command “/usr/bin/vmware-toolbox&”

• Go to the options tab of vmware-toolbox and check the box labeled “Time Synchronization between the virtual machine and the console operating system.” See Figure 13.

NOTE: This synchronization mechanism works only if the clock on the guest operating system is set to an earlier time.

Figure 13. Time Synchronization Using VMware Tools



6.6. Add a Virtual Disk To add a virtual disk to a virtual machine, follow the procedure below: NOTE: Two virtual disks need to be added to the virtual machine in this deployment. One is of size 40 GB and the other is of size 200 GB. Use the VirtualCenter Client: • Select the virtual machine to add the virtual disk. • Right click on the virtual machine in Inventory View. • Select “Edit Settings.” • Click “Add.” • In “Add Hardware Wizard”, enter the following: Device Type Hard Disk Select a Disk Create a new virtual disk

Disk Capacity 40 GB Location

Disk Capacity

Select [OracleVMFS] since it is connected to both ESX Server hosts, or select store with virtual machine if virtual machine is stored in OracleVMFS. Virtual Device Node SCSI (0:1) Mode

Advanced Options

Leave as default. Ready to Complete

Click on Finish.

Figure 14. Step to select a disk during virtual disk addition



Figure 15. Step to specify disk capacity and location during virtual disk addition

Figure 16. Step to specify advanced options during virtual disk addition



7. Oracle E-Business Suite Installation An installation procedure of Oracle E-Business Suite on a virtual machine, leveraging Oracle Rapid Install utility and VMware Infrastructure functionalities, is outlined in this section,

7.1. Rapid Install You can expedite the installation of Oracle E-Business Suite by pre-staging the software onto a local disk. The Media Pack for Oracle EBS release 12 consists of DVDs, divided into the following groups: • Start Here • APPL_TOP • RDBMS • Tools • Databases You can run a Perl script adautostg.pl located on disk 1 of the Start Here group to stage the content of the DVDs on a local disk.

7.1.1. Disk Space Requirements

Running adautostg.pl: Application Tier and Database Tier prestages two major tiers of the Oracle E-Business Suite software. The stage area needs to have at least 33 GB to accommodate the file system and the database files.

In a standard installation, the application tier requires at least 28 GB of free space to accommodate the components below: • AS 10.1.2 ORACLE_HOME • AS 10.1.3 ORACLE_HOME • COMMON_TOP • APPL_TOP • INST_TOP

The database tier requires: • At least 45 GB for the Database node file system (Fresh Install); or • At least 133 GB for the Database node file system (Vision Demon database) VMware recommends leaving extra space for the database tier in particular.

7.1.2. Mount DVD

The x86 Sun Servers have built-in DVD ROM drives on which you can directly mount the Oracle EBS installation disks. To do so, click on the button labeled “Virtual CDROM (ide0:0)” on VirtualCenter Client and follow the prompts.

7.1.3. Mount ISO Images

An alternative to mounting the disks directly on a DVD ROM drive is to convert them into ISO images and store them in a NFS mount, which can be mounted onto an ESX Server host machine. The ISO images can then be mounted onto the virtual machines as if they were loaded directly into a local DVD ROM drive.



Follow the procedure below to mount the ISO Images on ESX Server service console: • Create a new directory ISOs in /Vmimages • Mount NFS share on the newly created directory.

o Run command in the form of “mount –t nfs server_ip:/mnt/ISOs /VMimages/ISOs” • Change the DVD/CD-ROM drive configuration to mount the ISOs:

o Right-click on the virtual machine in Inventory View. o Select “Edit Settings.” o In the Connection section, check Use ISO image.

Browse to the desired ISO image in the /VMimages/ISOs directory. Click on OK.

7.1.4. Run adautostg.pl

On the Linux Guest OS, run the adautostg.pl to stage the Oracle EBS software for rapid install:

$ perl /mnt/cdrom/Disk1/rapidwiz/adautostg.pl

The stage script prompts for questions below: • The name of the system top-level directory where the software is staged. For example, you

may enter “/u01” as the response to this question. The script creates a subdirectory “stage12” in “/u01” as the parent directory of the staged disk contents.

• The components to stage. You can choose the components to stage:

1. Oracle Applications

2. Oracle Database technology stack (RDBMS)

3. Oracle Applications database (Databases)

4. Oracle Applications technology stack (Tools)

5. APPL_TOP



7.1.5. Stage Area Directory Structure

The Perl script creates subdirectories named startCD, oraApps, oraDB, oraAS, and oraAppDB under the stage12 directory.

Stage12

startCD

Disk 1

oraApps oraDB oraAS oraAppDB

Disk 1Disk 1 Disk n Disk n Disk 1 Disk n Disk 1 Disk n

Figure 17. Stage Area Directory Structure



Figure 18. Oracle E-Business Suite Software Staging



7.2. Oracle Installation Environment Red Hat Enterprise Linux AS 4.0 requires the following packages: • glibc-2.3.4-2.25 • glibc-common-2.3.4-2.25 • binutils-2.15.92.0.2-21 • compat-libstdc++-296-2.96-132.7.2 • gcc-3.4.6-3 • gcc-c++-3.4.6-3 • libgcc-3.4.6-3 • libstdc++-3.4.6-3 • libstdc++-devel-3.4.6-3 • openmotif21-2.1.30-11.RHEL4.6** • pdksh-5.2.14-30.3 • setarch-1.6-1 • make-3.80-6.EL4 • gnome-libs-1.4.1.2.90-44.1 • sysstat-5.0.5-11.rhel4 • compat-db-4.1.25-9 • control-center-2.8.0-12.rhel4.5 • xscreensaver-4.18-5.rhel4.11 • libaio-0.3.105-2 • libaio-devel-0.3.105-2

7.2.1. Software Requirements

The following maintenance tools must be installed and their locations specified both in the PATH of the account that runs Rapid Install and in the PATH of the accounts that will own the database tier and application tier file systems. • ar • gcc • g++ • ld • ksh • make • X Display Server



7.2.2. Kernel Settings

Edit the /etc/sysctl.conf file to configure kernel settings. After editing the file, use the "sysctl -p" command or restart the system to invoke the new settings. NOTE: If the current value for any parameter is higher than the value listed in the following table, then do not change the value of that parameter. For RHEL AS 4.0, use the following kernel settings: kernel.semmsl 256 kernel.semmns 32000 kernel.semopm 100 kernel.semmni 142 kernel.shmall 2097152 kernel.shmmax Half the size of the physical memory (in bytes), and at least 2147483648 kernel.shmmni 4096 kernel.msgmax 8192 kernel.msgmnb 65535 kernel.msgmni 2878 fs.file-max 65536 net.ipv4.ip_local_port_range 1024 65000 net.core.rmem_default 262144 net.core.rmem_max 262144 net.core.wmem_default 262144 net.core.wmem_max 262144

7.2.3. Verify Host Names

1. Verify that the /etc/hosts file is formatted as follows:

127.0.0.1 localhost.localdomain localhost <ip_address> <node_name>.<domain_name> <node_name> For the virtual machine VOEBS, /etc/hosts file reads: 127.0.0.1 localhost.localdomain localhost 10.6.160.165 voebs.voebs.com voebs

2. Verify that the /etc/sysconfig/network file is formatted as follows: HOSTNAME=<node_name>.<domain_name> For the virtual machine VOEBS, /etc/sysconfig/network file reads: voebs=voebs.voebs.com

3. Remove /etc/sysconfig/networking/profiles/default/network file if it exists. 4. Restart the virtual machine if any of the files above have changed.



7.2.4. Setup Oracle Account

Execute the following commands as root in the guest operating system: • groupadd oinstall • groupadd dba • mkdir –p /export/home/oracle • useradd –d /export/home/oracle –g oinstall –G dba –s /bin/ksh oracle • chown oracle:dba /export/home/oracle /u01 • passwd oracle

7.2.5. Virtual Disks Installation

The Oracle EBS installation installs software for both the application tier and the database tier. The space requirements for these tiers are as below:

Node

Space Required

Applications node file system (includes AS 10.1.2 ORACLE_HOME, AS 10.1.3 ORACLE_HOME, COMMON_TOP, APPL_TOP and INST_TOP)

28 GB

Database node file system (Fresh install)

45 GB

Database node file system (Vision Demo database)

133 GB

In this deployment, the application node file system and the database node file system with the Vision Demo database are installed. Each file system is installed on an individual virtual disk. As a result, two individual virtual disks are installed, one of size 40 GB for the application node file system and another of size 200 GB for the database node file system. To install a virtual disk, follow the procedure in section Add a Virtual Disk.



7.2.6. Disk Format in Guest Operating System

The virtual disks have to be formatted in the guest operating system before the Oracle EBS installation. These two virtual disks show as /dev/sdb and /dev/sdc in the Linux guest operating system.

Figure 19. Virtual Disks in Linux Guest Operating System To format the disks, follow the procedure below: • log in to guest operating system as root. • run “fdisk /dev/sdb.” • enter ‘n’ to create a new partition. • enter ‘1’ as the partition number. • use the default values for both the first cylinder and the last cylinder. • enter ‘w’ to write the partition table. • repeat the procedure for /dev/sdc.

The two devices will have /dev/sdb1 and /dev/sdc1 as partitions.




7.2.7. Create file systems

To create ext3 file systems on the partitions of the newly formatted disks, follow the procedure below: • log in to guest operating system as root. • run “mke2fs –j /dev/sdb1.” • run “mke2fs –j /dev/sdc1.”

Figure20. Create an ext3 filesystem on /dev/sdb1 partition


7.2.8. Mount the File Systems

To mount the newly created file systems, follow the procedure below: • log in to guest operating system as root. • run “mkdir –p “/d01/oracle/VIS/apps.” • run “chmod –R 777 /d01.” • edit “/etc/fstab.”

o add the following lines: /dev/sdb1 /d01/oracle/VIS/apps defaults 1 2 /dev/sdc1 /d01/oracle/VIS defaults 1 2

Figure 21. Content of /etc/fstab

• run “mount /dev/sdb1.” • run “mount /dev/sdc1.”



Figure 22. Partitions on virtual disks formatted and mounted as ext3 filesystem

7.2.9. Xhost

To ensure that the Oracle Rapid Wizard GUI can properly run, follow the procedure below: • log in to guest operating system as root. • xhost +



7.2.10. Oracle E-Business Suite Express Installation

An Express Installation sets up a fully configured single-user/single-node system with either a fresh database or a Vision Demo database. You can specify basic configuration parameters, such as database type and name, top-level installation directory, and port increments. Rapid Install supplies default values for the remaining directories and mount points.

You can start the Oracle E-Business Suite Express Installation after you have prepared the environments as described in the previous sections.

Follow the procedure below to perform the Oracle E-Business Suite Express Installation:

1. Run the Rapid Wizard as user ‘oracle’. o Run “/u01/StageR12/startCD/Disk1/rapidwiz”

2. Select Express Installation in the Select Wizard Operation screen: o Click on Install Oracle Applications Release 12. o Check Use Express Configuration. o Click on Next to continue.

3. Choose Oracle Configuration Manager options: NOTE: Oracle Connection Manager (OCM) is an optional component designed to facilitate support for Oracle products. It provides continuous tracking of key Oracle and system statistics of the machine it is running on. Data collected by the configuration manager is sent via secured HTTPS back to Oracle Support, who can thereby maintain an updated view of the customers’ Oracle instance.

NOTE: Oracle recommends the use of Oracle Configuration Manager, but it is optional. You can choose Decline and continue with the Rapid Install run.

o If you choose Accept, another screen is presented to gather the following information:

• Customer Support Identifier (CSI) • OracleMetaLink account details • Optionally a proxy server for use with OCM

• Proxy server name • Proxy server port

o Click on Next to continue. 4. Enter configuration values. A sample configuration is shown below. See Figure :

o Database Type: Vision Demo Database o Database SID: VIS o Domain: voebs.com o Base directory: /d01/oracle/VIS o Instance directory: /d01/oracle/VIS/inst o Port Pool: leave as default o Click on Next to continue.



Figure 23. Express Configuration Information 5. Start installation process:

o Rapid Install displays the pre-install tests as it performs them. When it is complete, the Pre-Install Checks screen appears. See Figure and Figure .

o Click on Next to continue.



Figure 24. System Check Status



Figure 25. Install Oracle Applications – Pre-Install Checks



6. Install:

o Continue by clicking next. See the progress bars shown in Figure .

Figure 26. Oracle Installation In Progress




7. Log in to the Oracle Applications Server: o Launch a web browser. o Go to the link: http://voebs.voebs.com:8000/OA_HTML/AppsLogin. o Use user “sysadmin” and password “sysadmin” to login.

Figure 27. Oracle EBS login page

http://voebs.voebs.com:8000/OA_HTML/AppsLogin


8. Gold Image for Application Tier Deployment VMware Infrastructure enables you to install the software suite on one virtual machine and create a template based on that virtual machine. This template serves as a gold image for rapid provisioning of the rest of the virtual machines. Oracle E-Business Suite users can leverage VMware Infrastructure to achieve the same rapid provisioning of their application tier.3 This section explains the process of creating a gold image.

8.1. Oracle Rapid Clone Oracle Rapid Clone provides a procedure to create a copy of an existing Oracle Applications system. Producing an exact copy of the patch level and data is very important as the cloned system must be able to provide the same output to the end user as the source system. Oracle Rapid Clone is leveraged in the process of creating the gold image, as well as the virtual machine template.

8.2. Gold Image Creation To create a gold image for application tier cloning, follow the procedure below:

8.2.1. Oracle Rapid Clone – Source System Preparation

To prepare the source system VOEBS, follow the procedure below:

1. Run Oracle AutoConfig NOTE: AutoConfig is a tool that automates the configuration of an Oracle Applications system. The information required for configuring an Applications system is collected into an XML repository, called the Applications context. There is one Applications context for each application tier, and one for each database tier. When AutoConfig runs, it uses information from the Applications context file to generate all configuration files, and update database profiles.

ATTENTION: The database server and the database listener must remain available during the AutoConfig run.

o Log in VOEBS as user ‘oracle’. o Run AutoConfig for the application tier, using the command

/d01/oracle/VIS/inst/apps/VIS_voebs/admin/scripts/adautocfg.sh. o Enter the password for user APPS: APPS. o See Appendix for a sample output.

2. Prepare the application tier for cloning by running:

o cd /d01/oracle/VIS/inst/apps/VIS_voebs/admin/scripts/ o perl adpreclone.pl appsTier

3 You can use the virtual machine cloning feature to achieve cloning of the database tier as well. The procedure is similar to the cloning of the application tier. VMware highly recommends that you evaluate the virtualization of the database tier in your deployments carefully.

Gold Image for Application Tier Deployment 45


8.2.2. Virtual Machine Start Up and Shut Down Process

Shutdown both the database tier and the application tier before proceeding with the virtual machine shutdown. To gracefully shutdown database, run the command: • /d01/oracle/VIS/db/tech_st/10.2.0/appsutil/scripts/VIS_voebs/addbctl.sh stop To gracefully shutdown applications, run the command: • /d01/oracle/VIS/inst/apps/VIS_voebs/admin/scripts/adstpall.sh Launch both the database tier and the application tier after launching the virtual machine To start database, run the commands: • /d01/oracle/VIS/db/tech_st/10.2.0/appsutil/scripts/VIS_voebs/addbctl.sh start • /d01/oracle/VIS/db/tech_st/10.2.0/appsutil/scripts/VIS_voebs/addlnctl.sh start VIS To start applications, run the command: • /d01/oracle/VIS/inst/apps/VIS_voebs/admin/scripts/adstrtal.sh

8.2.3. Virtual Machine Cloning

1. Clone the virtual machine without the 200GB virtual disk. o The 200GB virtual disk pertains to the database installation. It is not needed for this

gold image for application tier cloning. o Before cloning, edit the settings of the virtual machine offline to remove the 200GB

virtual disk. Follow the procedure below: 1. In VirtualCenter Client, select the virtual machine to change (VOEBS) in

Inventory View. 2. Shut down VOEBS by gracefully shutting down the Oracle EBS components

(i.e. database tier and application tier), and the operating system. 3. Right click on VOBES and select “edit settings.” 4. Go to the hardware tab and select the virtual disk of size 200GB. 5. Click on Remove from virtual machine for Removal Options.4. 6. Click on OK.

4 It is important not to delete the virtual disk from the physical hard disk because it will need to be re-added to the original virtual machine once the template is created.



Figure 28. Remove virtual disk of size 200GB on VOEBS



2. Perform virtual machine cloning: o Clone the virtual machine without the 200GB virtual disk using the following

procedure: 1. Make sure that VOEBS is shut down. 2. In VirtualCenter Client, select the virtual machine to change (VOEBS) in

Inventory View. 3. Right click on VOBES and select “Clone…” See Figure .

Figure 29. Clone a virtual machine

4. Enter the following parameters in the wizard:

Virtual Machine Name Name and Folder APPSVR2

Host / Cluster Select the host desired for the cloned virtual machine to reside in. 10.6.160.161 is used in this scenario.

Datastore Pick a datastore that is preferably visible to both ESX hosts: OracleVMFS

Guest Customization Select Do not customize Ready to Complete Click “Finish”



8.2.4. Prepare Virtual Machine Template The cloned virtual machine serves as the base to convert to a template later. Before it can be converted to a template, you will need to prepare it following the Oracle Rapid Clone procedure. To prepare the virtual machine, follow the procedure below: 1. Turn on the virtual machine APPSVR2. 2. Customize the virtual machine.

o edit /etc/fstab. 1. remove the line to mount /dev/sdc1.

o Configure the network interface:

As root, execute: # system-config-network

1. Make sure that the IP address and subnet mask are configured properly.

2. Make sure that the MAC address reflects the new hardware. Press the probe button in the GUI to perform a hardware probe.

3. Make sure that the host name does not collide with any existing hosts.

As root, execute: # /etc/init.d/network restart

Make sure that the new settings take effect.

Modify /etc/hosts Make sure that the new host name is entered in this file. An example is shown below: 127.0.0.1 localhost 10.6.160.166 appsvr2 appsvr2.voebs.com



Figure 30. Network Configuration



8.2.5. Clone to Template

Follow the procedure below to clone to a template: 1. Make sure that APPSVR2 is shut down. 2. In VirtualCenter Client, select the virtual machine to change (APPSVR2) in Inventory View.

o Right click on APPSVR2 and select “Clone to template…” o In the wizard, enter the following parameters:

Template Name

Name and Folder

APPSVR_TEMP

Host / Cluster

Select the host where you want the template to reside. 10.6.160.163 is used in this scenario.

Datastore

[OracleVMFS] NOTE: this datastore is visible to both ESX Server hosts.

Disk Format

Select Compact. This saves space and will help when transferring to a different host is necessary.

Ready to Complete

Click on Finish

8.2.6. Configure APPSVR2

Having created a virtual machine template, you can continue configuring APPSVR2 to be the second application tier server. NOTE: The virtual machine template APPSVR_TEMP is used for cloning other virtual machines. After cloning a virtual machine using APPSVR_TEMP, you can follow the same procedure to configure your new virtual application tier server. To configure a newly cloned virtual machine, follow this procedure: 1. Start the virtual machines VOEBS and APPSVR2.

o Start the database. Run the commands in /d01/oracle/VIS/db/tech_st/10.2.0/appsutil/scripts/VIS_voebs

1. addbctl.sh start 2. addlnctl.sh start VIS

2. Make sure that the network configuration has been properly configured on APPSVR2.



3. Run adcfgclone.pl in /d01/oracle/VIS/apps/apps_st/comn/clone/bin. o perl adcfgclone.pl appsTier o enter the APPS password: apps. o Target host name (virtual or normal): appsvr2 o Target system database SID: VIS o Target system database server node: voebs o Target system base directory for source homes: /d01/oracle/VIS/ o Target system base directory for instance homes: /d01/oracle/VIS/inst o Do you want to preserve the Display set to voebs:0.0 (y/n): n o Do you want the target system to have the same port values as the source system

(y/n): y o Choose a value that will be set as APPLTMP value on the target node: /usr/tmp.

4. Run perl adaddnode.pl to add the node to the database. 5. Other settings. Refer to Oracle Metalink document 406982.1, Section Finishing Tasks.



8.2.7. Verification

Verify using Oracle Applications manager: 1. Login to Oracle Applications Server using a web browser. 2. Navigate to Oracle Applications Manager (System Administrators/Dashboard). 3. Go to the Applications Dashboard. 4. Go to the Concurrent Processing Tab. 5. Verify both of the host names (VOEBS, APPSVR2) show on the page. See Figure .

Figure 31. List of Concurrent Processing Servers




6. Go to Overview Tab. 7. Verify that the hosts have their corresponding types:

o VOEBS: Admin, Database, Concurrent Processing, Forms, Web o APPSVR: Admin, Concurrent Processing, Forms, Web o See Figure .

Figure 32. List of Hosts showing corresponding types


9. VMware High Availability

VMware High Availability (HA) is a feature that automatically restarts virtual machines on a failed ESX Server host on another available ESX Server host. The restart of the virtual machines finishes within minutes, without human intervention. VMware HA enhances availability of the Oracle Applications Servers.

9.1. Network Card Redundancy Sun Fire X4600 M2 servers have 4 NIC cards. You can configure network card redundancy in two ways using VMware Infrastructure: • NIC Teaming • Redundant Service Console Port

9.1.1. NIC Teaming

NIC teaming groups NIC cards together to serve one virtual switch. To add a new NIC card to a virtual switch, follow the procedure below: 1. Select desired host. 2. Go to Configuration Tab. 3. Go to Networking. 4. Select Properties for vSwitch0. 5. Go to Network Adaptors Tab. 6. Select Add.

In the Add Adaptor Wizard, enter the following:

Adaptor Select the network adaptor. NIC order Have the new NIC card moved down to be a Standby adapter. Summary Click on Finish.

9.1.2. Redundant Service Console Port

To build a redundant service console port that is connected to a separate NIC card in case the other one fails, follow the procedure below:

Use VirtualCenter Client: • Select desired host. • Go to Configuration Tab. • Go to Networking. • Select Add Networking. • In the Add Network Wizard, enter the following:

Connection Type Select Service Console. Network Access Create a virtual switch using a NIC that has not been connected

to any virtual switch, e.g. vmnic1. Port Group Properties Network Label: Service Console 2 IP Settings

Connection Settings

IP Address: 10.6.160.169/Subnet Mask: 255.255.252.0 Summary Click on Finish.

VMware High Availability 55


9.2. Configure HA Cluster To configure a VMware HA cluster, follow the procedure below:

1. In VirtualCenter Client: o Right-click on Data Center and select New Cluster. o In the New Cluster Wizard, enter the following:

Name ORACLE CLUSTER Cluster Features

Cluster Features

VMware HA Host Failures Number of host failures allowed: 1 Admission Control

VMware HA

Select Do not start virtual machines if they violate availability constraints.

Ready to Complete Click on Finish.

Figure 33. Configure HA Cluster – Cluster Features




Figure 34. Configure HA Cluster – VMware HA – Host Failures + Admission Control 2. Drag-and-drop both of the ESX Server hosts to ORACLE CLUSTER. See Figure .

Figure 35. Configure VMware HA Cluster


10. VMware VMotion VMware VMotion migrates the virtual machine(s) from one ESX Server host to another without disruption of the services running on the virtual machine(s). This feature is called Live Migration. You can leverage Live Migration for scenarios such as scheduled maintenance of a particular ESX Server host. By doing so, you can move the virtual machines without causing any interruption to end users. Before configuring VMware VMotion, ensure the following: • The hardware of the two ESX Server hosts, in particular the CPU, are compatible. • All the virtual switches have identical configurations for both ESX Server hosts • A VMkernel Port is configured on both ESX Server hosts.

10.1. Configure VMkernel Port To configure a VMkernel Port, follow the procedure below: • In VirtualCenter Client, select the desired ESX Server host. • Go to Configuration Tab/Networking. • Select Add Networking. • In the Add Network Wizard, enter the following:

Connection Type Select VMkernel. Network Access Use vSwitch0.

Port Group Properties Network Label: VMkernel Check the box labeled “Use this port group for VMotion.” IP Settings

Connection Settings

IP Address: 10.6.160.169/Subnet Mask: 255.255.255.0 Summary Click on Finish.

VMware VMotion 58


Figure 36. VMkernel Port Properties

10.2. Live Migration To perform Live Migration on a virtual machine, follow the procedure below: • In VirtualCenter Client, right click on the desired virtual machine. • Select Migrate. • In the Migrate Virtual Machine Wizard, enter the following:

Select Destination Select the redundant ESX Server host. 10.6.160.163 is

chosen in this scenario. Select Resource Pool Leave as default. Migration Priority Select High Priority. Ready to Complete Click on Finish.

Once all the virtual machines on the original ESX Server host have been successfully migrated to the other host, it is safe to put the original host in maintenance mode and perform maintenance work on it.

10.3. Cold Migration To perform a cold migration on a virtual machine, follow the procedure below: • Shut down the application and database processes in the virtual machine. • Power off the virtual machine. • Right-click on the virtual machine in Inventory View. • Select “migrate.” • In the Migrate Virtual Machine Wizard, enter the following:

VMware VMotion 59


VMware VMotion 60

Select Destination Select the redundant ESX Server host. In this scenario, it is 10.6.160.163.

Select Resource Pool

Leave as default.

Select Datastore Select Keep virtual machine configuration files and virtual disks in their current locations.

Ready to Complete Click on Finish.


11. Best Practices VMware highly recommends that you refer to the best practices below for deployment planning and designing. Careful Planning Scrutinize the service level agreement with users and carefully lay out the peak and average load to help planning. Understand the hardware needed to sustain the workload requirements and procure sufficient hardware resources for the planned workload. Carefully plan the deployment of virtual machines that will run on the ESX Server hosts. Avoid Memory Over-Commitment Avoid memory over-commitment in virtual machine configuration. Memory over-commitment can result in paging-to-disk activity, which is detrimental to the Oracle EBS workload performance. Allocate sufficient memory for the virtual machine to avoid paging-to-disk. Start with 1 or 2 vCPU Virtual Machines Start with 1 or 2 vCPU virtual machines. Stay at 1 or 2 vCPU unless careful analysis using system tools shows value for higher-vCPU virtual machines. VMware recommends that you use a combination of system OS tools and VMware monitoring tools (such as VirtualCenter resource monitoring). NOTE: vmkusage and esxtop are monitoring tools to monitor CPU, memory and IO characteristics but need to run on the ESX Server service console. VMware does not recommend running any program on the service console unless directed to do so by the administrator or VMware support Use Latest Processor Generation VMware highly recommends using servers built with the latest processor generation (e.g. Intel Xeon 51xx/53xx, rev.E/F AMD at the time of this writing). Leverage VMware High Availability Features Use VMware High Availability and VMotion to provide enhanced high availability and fault tolerance. VMware High Availability can ensure that a virtual machine on a failed ESX Server host can be restarted in a timely manner on another ESX Server host in the same HA cluster. Use VMotion to move a virtual machine from one ESX Server host to another one without having to shut it down, eliminating downtime to the service. NOTE: VMware HA does not provide process level fail-over but rather a virtual machine level restart. Leverage VMware Virtual Machine Templates To deploy additional virtual machines efficiently, VMware highly recommends that you create templates from virtual machines with pre-installed and pre-configured OS and/or Oracle software. You can archive a library of templates for later use, and can benefit tremendously from leveraging the template cloning feature.

Best Practices 61


Best Practices 62

Leverage VMware Virtual Machine Snapshots VMware Snapshots are ideal for journaling virtual machine states at different times. You can generate a snapshot of your virtual machines before making any changes, such as upgrading and patching. In case the changes cause undesirable results, you can easily roll back to a previous snapshot with all of the original states saved. VMware Tools Install the latest VMware Tools on the guest operating systems of all virtual machines to ensure high performance in memory, network bandwidth and graphics. Avoid Running Programs in the ESX Server Service Console Running programs in the ESX Server service console can adversely affect the performance of virtual machines and the ESX Server host if the programs consume excessive amounts of CPU or memory. VMware recommends that you not use the ESX Server service console directly for any administrative tasks. Use the VMware® VirtualCenter management interface, as well as SDK interfaces if required, for the administration of virtual machines running the Oracle EBS environment. VMware NIC teaming Purchase more than one NIC card for an ESX Server host machine, and group the cards as a NIC team to provide redundancy and increased performance resulting from traffic distribution across the NIC cards.


12. Conclusion Oracle E-Business Suite (EBS) is a fully integrated, comprehensive suite of business applications for the enterprise. It provides better information for effective decision-making and enables an adaptive enterprise for optimal responsiveness. Deployment and maintenance of Oracle applications in a physical environment is not a trivial task. The capabilities of VMware Infrastructure enable you to streamline Oracle deployment and maintenance by providing the following capabilities: • Server containment • Rapid provisioning • Change management • High availability: business continuity and disaster recovery

To improve the Oracle EBS virtualization process, this paper provides procedures for streamlining deployment using a gold image and enhancing availability by the way of VMware High Availability and VMware VMotion. VMware recommends that you refer to this paper for deployment planning and design before embarking on your Oracle E-Business Suite virtualization project.

Conclusion 63


13. Resources Customers can find more information about VMware and Oracle E-Business Suite products via the links listed below: VMware Official Website: www.vmware.com Oracle Official Website: www.oracle.com

13.1. VMware VMware Infrastructure 3 product website: http://www.vmware.com/products/data_center.html VMware Infrastructure 3 download website: http://www.vmware.com/download/vi/eval.html VMware support website http://www.vmware.com/vmtn/ VMware Best Practices Papers http://www.vmware.com/pdf/vsmp_best_practices.pdf http://www.vmware.com/pdf/vi_performance_tuning.pdf VMware Products http://www.vmware.com/products/

13.2. Oracle Oracle E-Business Suite Website: http://www.oracle.com/applications/e-business-suite.html Oracle support website: http://www.oracle.com/technology/about/index.html Oracle E-Business Suite Release 12 Documentation Library http://download.oracle.com/docs/cd/B40089_05/current/html/homeset.html Oracle Applications Installation Guide: Using Rapid Install Release 12 http://download-uk.oracle.com/docs/cd/B34956_01/current/acrobat/120oaig.pdf

Metalink Note Title 394692.1 Oracle Applications Documentation Resources, Release 12 406982.1 Cloning Oracle Applications Release 12 with Rapid Clone 387859.1 Using AutoConfig to Manage System Configurations in Oracle E-

Business Suite Release 12

Resources 64

http://www.vmware.com/

http://www.oracle.com/

http://www.vmware.com/products/data_center.html

http://www.vmware.com/download/vi/eval.html

http://www.vmware.com/vmtn/

http://www.vmware.com/pdf/vsmp_best_practices.pdf

http://www.vmware.com/pdf/vi_performance_tuning.pdf

http://www.vmware.com/products/

http://www.oracle.com/applications/e-business-suite.html

http://www.oracle.com/technology/about/index.html


14. Appendix 1 – Useful Commands

14.1. Application Tier Commands

14.1.1. System configuration commands on the application tier

All scripts listed below are located in <INST_TOP>/admin/scripts.

Functionality Command Start Applications services adstrtal.sh Stop Applications services adstpall.sh Run AutoConfig adautocfg.sh

14.1.2. Rapid Clone Commands

Functionality Location Command Sample Usage

Prepare source system application tier for cloning

<INST_TOP>/admin/scripts adpreclone.pl perl adpreclone.pl appsTier

Configure target system application tier server nodes

<COMMON_TOP>/clone/bin adcfgclone.pl perl adcfgclone.pl appsTier

Add a new node to an existing system

<COMMON_TOP>/clone/bin adaddnode.pl perl adaddnode.pl

14.1.3. Run AutoConfig on the Application Tiers

Follow the steps under section "Run AutoConfig on the Application Tiers" in OracleMetaLink Note 387859.1 to Run AutoConfig on all application tier nodes.

AutoConfig Output Listing

[oracle@voebs scripts]$ ./adautocfg.sh Enter the APPS user password: AutoConfig is configuring the Applications environment... AutoConfig will consider the custom templates if present. Using CONFIG_HOME location : /d01/oracle/VIS/inst/apps/VIS_voebs Classpath : /d01/oracle/VIS/apps/apps_st/comn/java/lib/appsborg2.zip:/d01/oracle/VIS/apps/apps_st/comn/java/classes Using Context file : /d01/oracle/VIS/inst/apps/VIS_voebs/appl/admin/VIS_voebs.xml Context Value Management will now update the Context file Updating Context file...COMPLETED Attempting upload of Context file and templates to database...COMPLETED Configuring templates from all of the product tops... Configuring AD_TOP........COMPLETED Configuring FND_TOP.......COMPLETED Configuring ICX_TOP.......COMPLETED Configuring IEO_TOP.......COMPLETED Configuring BIS_TOP.......COMPLETED Configuring AMS_TOP.......COMPLETED

Appendix 1 65

http://metalink.oracle.com/metalink/plsql/ml2_documents.showNOT?p_id=387859.1


Configuring CCT_TOP.......COMPLETED Configuring WSH_TOP.......COMPLETED Configuring CLN_TOP.......COMPLETED Configuring OKE_TOP.......COMPLETED Configuring OKL_TOP.......COMPLETED Configuring OKS_TOP.......COMPLETED Configuring CSF_TOP.......COMPLETED Configuring IGS_TOP.......COMPLETED Configuring IBY_TOP.......COMPLETED Configuring JTF_TOP.......COMPLETED Configuring MWA_TOP.......COMPLETED Configuring CN_TOP........COMPLETED Configuring CSI_TOP.......COMPLETED Configuring WIP_TOP.......COMPLETED Configuring CSE_TOP.......COMPLETED Configuring EAM_TOP.......COMPLETED Configuring FTE_TOP.......COMPLETED Configuring ONT_TOP.......COMPLETED Configuring AR_TOP........COMPLETED Configuring AHL_TOP.......COMPLETED Configuring OZF_TOP.......COMPLETED Configuring IES_TOP.......COMPLETED Configuring CSD_TOP.......COMPLETED AutoConfig completed successfully. The log file for this session is located at: /d01/oracle/VIS/inst/apps/VIS_voebs/admin/log/xxxxxx/adconfig.log

Appendix 1 66


Appendix 1 67

VMware, Inc. 3401 Hillview Ave. Palo Alto CA 94304 USA Tel 1-877-486-9273 Fax 650-427-5001 www.vmware.com © 2007 VMware, Inc. All rights reserved. Protected by one or more of U.S. Patent Nos. 6,397,242, 6,496,847, 6,704,925, 6,711,672, 6,725,289, 6,735,601, 6,785,886, 6,789,156, 6,795,966, 6,880,022, 6,961,941, 6,961,806, 6,944,699, 7,069,413; 7,082,598 and 7,089,377; patents pending.

vmwareandoracleebsdeploymentguidefinal

Documents

customer relationship management

supply chain management

vmware white paper

guest operating system

vision demo database

esx server hosts

virtual machine libraries

esx server host