EMC WHITE PAPER

ORACLE VM 3.3 STORAGE CONNECT

FOR VMAX WITH EMC STORAGE INTEGRATOR V2.0

EMC® VMAX® Engineering White Paper

ABSTRACT

Oracle VM and Storage Connect framework of API, together with EMC Storage

Integrator for Oracle VM x86, enhance IT operations for managing virtualized

infrastructure by allowing EMC storage discovery, provisioning, and replication

operations to be performed with ease directly from Oracle VM Manager.

December, 2014 Version 1.1

2

To learn more about how EMC products, services, and solutions can help solve your business and IT challenges, contact your local

representative or authorized reseller, visit www.emc.com, or explore and compare products in the EMC Store.

Copyright © 2014 EMC Corporation. All Rights Reserved.

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without

notice.

The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with

respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a

particular purpose.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.

Part Number H13603 Version 1.1

3

TABLE OF CONTENTS

EXECUTIVE SUMMARY .............................................................................. 4

AUDIENCE ................................................................................................ 5

PRODUCT OVERVIEW ................................................................................ 5

Oracle VM overview ............................................................................................ 5

Oracle VM Server ............................................................................................... 5

Oracle VM Manager ............................................................................................ 5

Oracle VM Storage Connect ................................................................................. 6

ESI for Oracle VM overview ................................................................................. 6

VMAX3 overview ................................................................................................. 6

INSTALLATION AND CONFIGURATION ..................................................... 8

Test environment and Installation Steps ............................................................... 8

Installation of Oracle VM Server for x86 ................................................................ 9

Installation of Oracle VM Server ......................................................................... 10

Installation of EMC VMAX SMI-S Provider ............................................................ 12

Installation of Oracle VM Manager ...................................................................... 14

Installation of ESI for Oracle VM for x86 ............................................................. 16

View VMAX Storage Information ........................................................................ 20

USE CASES .............................................................................................. 23

Use Case 1 – Basic Storage Management Operations ............................................ 23

Use Case 2 – Provision Storage to Oracle VM Servers ........................................... 27

Use Case 3 – Create a Server Pool ..................................................................... 29

Use Case 4 – Create a Repositry for Server Pool .................................................. 31

Use Case 5 – Create Virtual Machines from a Server Pool ...................................... 33

Use Case 6 – Clone Virtual Machines .................................................................. 36

CONCLUSION .......................................................................................... 38

REFERENCES ........................................................................................... 38

4

EXECUTIVE SUMMARY

Many IT organizations embrace virtualization and ‘as-a-Service’ Cloud deployments as a better way of achieving business agility and

efficiency. While traditional IT operations require elongated processes between different groups and departments (such as Storage,

Network, Systems, Virtualization, Database, and Applications) it is much more desirable to create an infrastructure that is suited to

support such processes in an automated, secure, and optimized way via a set of standard processes and APIs. Such an infrastructure

is provided by Oracle VM and Storage Connect APIs.

Oracle VM is the server virtualization offering from Oracle Corporation. Oracle VM Server for x86 is application-driven built for rapid

deployment of enterprise applications. Oracle VM is a Xen-based hypervisor that can support Linux, Oracle Solaris and Windows

guests. Oracle VM includes an integrated Web based management console, Oracle VM Manager.

EMC Storage Integrator (ESI) for Oracle VM version 3.3 is a plug-in that enables Oracle VM to discover and provision EMC storage

arrays such as VMAX and VNX. The ESI plug-in is built to work in conjunction with Oracle Storage Connect framework. The

framework provides a set of storage discovery and provisioning APIs that enhances the ability to manage and provision storage

devices in an Oracle VM environment.

ESI for Oracle VM uses EMC SMI-S Provider to do the actual storage management work. SMI-S Provider supports the Storage

Networking Industry Association (SNIA) Storage Management Initiative (SMI), an American National Standards Institute (ANSI)

standard for storage management.

Oracle VM and Storage Connect APIs, together with EMC Storage Integrator, enhance IT operations for managing virtualized

infrastructure. As can be seen in Figure 1, the Oracle VM administrator can add storage devices from a pre-assigned storage pools,

create snapshots of these devices, or clone complete VMs with all their attached storage. All this is done with ease from Oracle VM

Manager, utilizing EMC ESI and SMI-S APIs, and take place without need to call a storage admin.

Figure 1 Oracle VM storage tasks

5

In summary, EMC Storage Integrator for Oracle VM for x86 enables the following:

Agile – Discover and provision EMC storage for virtual machines without the need to involve a storage administrator.

Simple – EMC Storage operations are a simple to execute with a few clicks in the Oracle VM Manager.

Integrated – enables flexibility, scalability, and protection of Oracle VM environments with EMC storage.

AUDIENCE

This white paper is intended for database and system administrators, storage administrators, and system architects who are

responsible for implementing, managing, and supporting Oracle VM virtual infrastructures, and Oracle databases and storage

systems. It is assumed that readers have some familiarity with Oracle VM and EMC VMAX and VMAX3 family storage arrays, and are

interested in achieving server and database virtualization, as well as higher database availability and protection.

PRODUCT OVERVIEW

ORACLE VM OVERVIEW

Oracle VM is an integrated application-driven server virtualization solution designed for rapid deployment and simplified management

of enterprise applications. Oracle VM supports a variety of guests such as Linux, Solaris and Windows. Oracle VM is designed to

support mission-critical workloads such as Oracle Database, business applications as well as third-party enterprise applications.

ORACLE VM SERVER

Oracle VM Server provides the host on which virtual machines can run. It is lightweight and secure and while a single Oracle VM

server is sufficient.

Oracle VM Server includes a light-weight Linux kernel (known as dom0), and can manage many virtual machines (known as domU),

where each virtual machine can run Linux, Oracle Solaris, or Microsoft WindowsTM. Each Oracle VM Server also runs an Oracle VM

Agent.

Oracle VM Servers can be grouped together into a cluster called server pools, where multiple server pools can be configured and

managed within an Oracle VM environment. At least one server pool is required to be configured, even if it includes only one Oracle

VM Sever. Each server pool allows for high-availability and load balancing of virtual machines across the Oracle VM servers in the

sever pool.

ORACLE VM MANAGER

Oracle VM Manager provides the Oracle VM environment with a framework that includes a web-based GUI, a command line interface,

and an integrated database as a repository. Oracle VM Manager doesn’t necessarily require high-availability as an Oracle VM

environment can continue operations even if Oracle VM Manager is down.

Oracle VM Manager consists of Oracle VM Manager Server, and Oracle VM Manager Database. Other components are interfaced via

APIs to Oracle VM Manager.

6

ORACLE VM STORAGE CONNECT

Oracle VM Storage Connect is a program and a framework that contains a set of API for storage discovery and provisioning. Using

this framework and API, users can perform such storage discovery and provisioning operations directly from Oracle VM Manager.

This enhances business agility, simplifies storage management, and allows for more robust virtualized infrastructure.

The plugins for Oracle VM Storage Connect are written by each storage vendor, utilizing their specific and advanced storage

capabilities. By using these plugins the virtualization manager doesn’t need to know how to operate and manage each storage

system, yet benefit from correct operations based on the storage vendor’s implementation of their best practices via API. In this way

storage provisioning is performed as part of the overall management of a virtualized infrastructure.

Note: For more information about the Oracle Storage Connect framework and Oracle VM, visit:

Oracle Storage Connect (http://www.oracle.com/us/technologies/virtualization/storage-connect/overview/index.html)

Oracle VM for x86 (http://www.oracle.com/us/technologies/virtualization/oraclevm/overview/index.html)

ESI FOR ORACLE VM OVERVIEW

The ESI for Oracle VM is a plug-in that enables Oracle VM to discover and provision EMC storage arrays. The plug-in is built upon the

Oracle Storage Connect framework. The framework provides a set of storage discovery and provisioning Application Programming

Interfaces (APIs) that enhance the ability to manage and provision storage devices in an Oracle VM environment.

ESI for Oracle VM supports the following storage management and provisioning operations for EMC storage arrays:

LUN discovery and inquiry

Thin LUN creation and removal

Auto-provisioning (LUN masking)

Register and refresh arrays

Create and delete access groups

Clone device creation and removal

Snap device creation and removal

EMC storage information (total and pools capacities, free space, and more)

ESI for Oracle VM uses EMC SMI-S Provider to do the actual storage management work. SMI-S Provider supports the Storage

Networking Industry Association (SNIA) Storage Management Initiative (SMI), an American National Standards Institute (ANSI)

standard for storage management.

VMAX3 OVERVIEW1

The EMC VMAX3 family storage arrays are built on the strategy of simple, intelligent, modular storage, and incorporates a Virtual

Matrix interface that connects and shares resources across all VMAX3 engines, allowing the storage array to seamlessly grow from an

entry-level configuration into the world’s largest storage array. It provides the highest levels of performance and availability

featuring new hardware and software capabilities.

The newest additions to the EMC VMAX3 family, VMAX 100K, 200K and 400K, deliver the latest in Tier-1 scale-out multi-controller

architecture with consolidation and efficiency for the enterprise. With completely redesigned hardware and software, the new VMAX3

array provides unprecedented performance and scale. It offers dramatic increases in floor tile density with engines and high capacity

disk enclosures for both 2.5" and 3.5" drives consolidated in the same system bay.

1 At the time this paper is written, VMAX with Enginuity 5876 and SMI-S 5.7.6 is supported. VMAX3 with HYPERMAX OS 5977 and SMI-S 8.0 support is

planned to be added shortly.

7

In addition, VMAX3 arrays can be configured as either hybrid or all- flash arrays. All new VMAX3 models come pre-configured from

the factory to significantly shorten the time from installation to first I/O.

Figure 2 VMAX3 storage array

VMAX3 engines provide the complete foundation for high-availability storage arrays. Each engine contains two VMAX3 directors and

redundant interfaces to the new Dynamic Virtual Matrix dual InfiniBand® fabric interconnect. Each director consolidates front-end,

global memory, and back-end functions, enabling direct memory access to data for optimized I/O operations. Depending on the

array chosen, up to eight VMAX3 engines can be interconnected via a set of active fabrics that provide scalable performance and high

availability. Systems ports are no longer mapped directly to CPU resources. CPU resources are allocated as needed using pools

(front-end, back-end, or data services pool) of CPU cores which can service all activity in the VMAX3 array.

All storage in the VMAX3 array is virtually provisioned. All data pools are created in containers called Storage Resource Pools. The

array uses the dynamic and intelligent capabilities of Fully Automated Storage Tiering (FAST) to meet system or user-defined Service

Level Objectives throughout the lifecycle of each application. VMAX3 Service Level Objectives are tightly integrated with EMC FAST

software to optimize agility and array performance across all drive types in the system. EMC FAST technology improves application

performance while reducing cost by using Flash drives combined with cost-effective high-capacity drives.

1 – 8 redundant VMAX3 Engines

Up to 4 PB usable capacity

Up to 256 FC host ports

Up to 16 TB global memory (mirrored)

Up to 384 Cores, 2.7 GHz Intel Xeon E5-2697-v2

Up to 5,760 drives

SSD Flash drives 200/400/800 GB 2.5”/3.5”

300 GB – 1.2 TB 10K RPM SAS drives 2.5”/3.5”

300 GB 15K RPM SAS drives 2.5”/3.5”

2 TB/4 TB SAS 7.2K RPM 3.5”

8

For local replications, VMAX3 adds a new feature to TimeFinder software called SnapVX, which provides “hybrid cloud scale”

snapshots. Unlike previous VMAX snapshots, SnapVX snapshots do not require the use of dedicated target devices. SnapVX allows

for up to 256 snapshots per individual source. These snapshots can be linked to copy their data to new target devices, or re-linked to

update just the incremental data changes.

VMAX3 new features and capabilities provide Oracle VM and Oracle databases more flexibility, scalability, protection, and

performance, together with a dramatic increase in management simplification and ease of use.

INSTALLATION AND CONFIGURATION

TEST ENVIRONMENT AND INSTALLATION STEPS

Overview of Test Environment

The test environment was kept small for demonstration purposes. A larger scale environment can be easily set in a similar way with

larger server pools, more networks, and Oracle VM Servers.

Figure 3 Test environment

The Oracle VM servers were configured using two Dell PowerEdge R710. Oracle VM Manager was configured using a Dell R900. The

SMI-S Provider was installed on the same server as Oracle VM Manager for simplicity. A management public network was set

between all three servers and a private network was set just between the Oracle VM servers as they were clustered and belonged to

the same server pool. SAN connectivity was set between the VMAX 10K storage array and the Oracle VM Servers leveraging

redundant FC switches, host multipathing, and multiple ports on the array. The Storage Management host had connectivity to only a

few small devices for communication with the storage array.

9

List of hardware components can be seen in Table 1, and software component in Table 2.

Table 1 Hardware components

Item Type

Oracle VM Servers 2 x Dell R710

Oracle VM Manager and SMI-S Provider server 1 x Dell R900

Storage array 1 x VMAX 10K

SAN Switch 2 x EMC Connectrix DS-6510B

Host HBAs (per host) 2 x Emulex LPe12002

Table 2 Software components

Item Version

Oracle VM Server 3.3

Oracle VM Manager / Host operating system 3.3 / Oracle Linux 6.5

ESI for Oracle VM 3.3 V2.0

EMC SMI-S Provider for VMAX and VNX 7.6.2.9

Summary of Installation Steps

The following are the high-level steps for installing and configuring ESI for Oracle VM:

1. Install all Oracle VM Servers for x86

2. Install EMC VMAX SMI-S Provider

3. Install Oracle VM Manager

4. Install ESI for Oracle VM

The following sections explain each step in more details.

INSTALLATION OF ORACLE VM SERVER FOR X86

Oracle VM Servers are the physical compute nodes hosting the virtual machines. Oracle installation for Oracle VM Server is simple,

light weight, and contains only packages necessary for its task. For this reason it is not recommended to use the Oracle VM Server

for other purposes, such as Solutions Enabler array management software, or Linux package repositories for upgrades.

Note: Do not create a VMAX Masking View2 for the Oracle VM Servers. All storage provisioning operations between VMAX and the

Oracle VM Servers should be performed in Oracle VM Manager utilizing Storage Connect APIs. Device masking operations outside of

Oracle Storage Connect (such as by using Unisphere for VMAX or Solutions Enabler CLI) won’t be recognized by Oracle VM Manager.

An exception is device creation as Oracle VM Manager can discover and use existing VMAX devices.

2 VMAX Masking View is a component of VMAX Auto-provisioning Group. The masking view task is to make storage devices visible to hosts by creating a

relationship between a group of devices, a group of host HBA ports (initiators), and a group of storage array ports the devices are visible to.

10

INSTALLATION OF ORACLE VM SERVER

Oracle VM Server installation tasks

To install and configure Oracle VM Server, perform the following actions:

1. Navigate to http://www.oracle.com and sign in with your user account.

2. Navigate to Downloads > Linux and Oracle VM and download Oracle VM 3.3 Server for x86 installation media.

3. Follow the installation steps described in Oracle Installation and Upgrade Guide for Release 3.3. This will allow you to:

Install Oracle VM Server software

Configure the hostname

Configure network information

Define Oracle VM Agent and root user passwords

Note: During the installation the first password asked for is for the Oracle VM Agent. This password will be used later when Oracle

VM Manager discovers and connects with the Oracle VM Server.

Oracle VM Server post installation tasks

Post installation tasks may include the configuration of additional network interfaces and other tasks3. Oracle VM Servers are

discovered via the public network, however, Oracle recommends using dedicated networks or VLAN for communication between

Oracle VM Servers within a Server Pool. Additional networks can be configured for purposes such as Virtual Machine private network,

Live Migration, and Cluster Heart-bit. Refer to Oracle VM Concept Guide for release 3.3 for more details about Oracle VM networking.

Note: All hosts in the cluster should have the same subnet

For simplicity, this document describes the configuration of two Oracle VM Servers, with a dedicated private network, as shown in

Table 3.

Table 3 Oracle VM Server Networks

ORACLE VM SERVER 1 ORACLE VM SERVER 2 ORACLE VM MANAGER

SERVER NAME DSIB0062 DSIB0063 DSIB0064

PUBLIC IP/NIC 10.108.244.62/EM1 10.108.244.63/EM1 10.108.244.64

PRIVATE IP/NIC 192.168.1.62/EM4 192.168.1.63/EM4

During Oracle VM Server installation, the hostname and public network information are configured. To set additional network

interfaces perform the following actions on each of the Oracle VM Servers, making changes to the network information as

applicable.

3 Oracle VM 3.3 Administration Guide documents the steps to increase dom0 memory size to avoid performance issues, and to add SNMP monitoring

tools.

11

The example in this paper demonstrates the following activities:

Manual configuration of an additional network interface for private network

Update of /etc/hosts to include the private network

Test the new interface

Edit the appropriate network interface connected to the private network switch:

[root@dsib0062 ~]# cd /etc/sysconfig/network-scripts/ [root@dsib0062 network-scripts]# cp ifcfg-em4 sav_ifcfg-em4.sav [root@dsib0062 network-scripts]# vi ifcfg-em4

Update the file to include the private network information:

DEVICE="em4" BOOTPROTO="none" DHCP_HOSTNAME="dsib0062" HWADDR="A4:BA:DB:41:7B:6B" NM_CONTROLLED="yes" #orig: ONBOOT="no" TYPE="Ethernet" UUID="439b72dc-75d9-485c-9468-1118576e9eff" # Added ONBOOT="yes" IPADDR="192.168.1.62" NETMASK="255.255.255.0"

Restart network service for the changes to take effect:

[root@dsib0062 network-scripts]# service network restart

Update hosts file with the new host information (unless DNS is used):

[root@dsib0062 ~]# vi /etc/hosts

Include the relevant network information:

127.0.0.1 localhost 10.108.244.62 dsib0062 10.108.244.63 dsib0063 192.168.1.62 dsib0062-priv 192.168.1.63 dsib0063-priv

Ping the private network between the servers to test the connection:

[root@dsib0062 ~]# ping dsib0063-priv PING dsib0063-priv (192.168.1.63) 56(84) bytes of data. 64 bytes from dsib0063-priv (192.168.1.63): icmp_seq=1 ttl=64 time=0.190 ms 64 bytes from dsib0063-priv (192.168.1.63): icmp_seq=2 ttl=64 time=0.075 ms 64 bytes from dsib0063-priv (192.168.1.63): icmp_seq=3 ttl=64 time=0.072 ms 64 bytes from dsib0063-priv (192.168.1.63): icmp_seq=4 ttl=64 time=0.072 ms

12

INSTALLATION OF EMC VMAX SMI-S PROVIDER

Storage management host

EMC SMI-S Provider is the software that complies with SNIA SMI-S standards for storage management and works in conjunction with

Solutions Enabler4 software. Oracle VM Storage Connect for VMAX uses SMI-S for all storage monitoring and control operations.

Note: EMC SMI-S Provider and Solutions Enabler software should not be installed on the Oracle VM Servers. Instead, they should be

installed on a host referred to as the Storage Management host. In some deployments, the user may prefer to combine the Storage

Management host with Oracle VM Manager host though it is not required.

Note: Solutions Enabler requires a number of very small devices called Gatekeepers to be presented to the Storage Management

host for communication with the VMAX. There should be a minimum of six Gatekeepers visible to the Storage Management host

before Solutions Enabler and SMI-S are installed.

SMI-S Provider installation

In order to install the SMI-S Provider, perform the following actions on the Storage Management host.

Host Preparation

1. Engage the storage administrator and have 6 or more Gatekeepers provisioned and visible to the Storage Management host.

2. Update the Storage Management host path to include the path to Solutions Enabler and SMI-S binaries.

[root@dsib0064 ~]# vi ~.bashrc ... export PATH=$PATH:/usr/symcli/bin:/opt/emc/ECIM/ECOM/bin:.

3. Source the file to take effect.

[root@dsib0064 ~]# . ~/.bashrc

SMI-S Installation Steps

1. Navigate to http://www.emc.com.

2. Navigate to Support > Downloads and search for SMI-S Provider.

3. Download SMI-S Provider 4.6.2.9 for SMI-S for Linux (or a later release)5.

4. Copy the file to the Storage Management server and extract it.

4 Solutions Enabler software allows management of VMAX storage array using APIs, CLIs, and REST. It works in conjunction with other VMAX

management solutions such as Unisphere for VMAX GUI, EMC SMI-S Provider, EMC ViPR, and others. 5 SMI-S 4.6.2.9 release is considered a ‘customer hot-fix’ for SMI-S. If SMI-S 4.6.2.9 (or a later release) is not available on EMC SMI-S download page

request it from your EMC account representative as a minimum software requirement.

13

5. Follow the installation steps described in EMC SMI-S Provider Version 4.6.2 Release Notes. The main steps are described below.

Extract the installation software and run it:

[root@dsib0064 SE]# tar xvfz se7629-Linux-x86_64-SMI.tar.gz [root@dsib0064 SE]# ./se7629_install.sh –install ... #----------------------------------------------------------------- # The following HAS BEEN INSTALLED in /opt/emc via the rpm utility. #----------------------------------------------------------------- ITEM PRODUCT VERSION 01 EMC Solutions Enabler V7.6.2.9 02 SMI KIT V4.6.2 #-----------------------------------------------------------------

Run the storage discovery command6:

[root@dsib0064 SE]# symcfg discover

Test the SMI-S provider installation and communication with the storage array. To do so run7 TestSmiProvider command, accept

all the defaults until the command screen shows, then use Display Version Info (dv) command to test connectivity

[root@dsib0064 ~]# TestSmiProvider ... Built with EMC SMI-S Provider: V4.6.2 Namespace: root/emc repeat count: 1 (localhost:5988) ? dv ++++ Display version information ++++ CIM ObjectManager Name: EMC:10.108.244.63 CIMOM Version: EMC CIM Server Version 2.7.3.3.0.8 SMI-S qualified version: 1.5.0 SMI-S Provider version: V4.6.2.3 SMI-S Provider Location: Proxy SMI-S Provider Server: Linux dsib0063 3.8.13-35.1.1.el6uek.x86_64 #2 SMP Fri Jun 6 10:52:33 PDT 2014 x86_64 (64bit Libraries) Solutions Enabler version: V7.6-1808 2.9 Firmware version information: (Local) Symmetrix Array 000198700067 (VMAX10K) : 5876.268.174 Retrieve and Display data - 1 Iteration(s) In 0.327698 Seconds Please press enter key to continue...

6 It is assumed that the root user PATH was updated with Solutions Enabler binaries location, which has a default location of: /usr/symcli/bin 7 It is assumed that the root user PATH was updated with the SMI-S Provider binaries location, which has a default location of: /opt/emc/ECIM/ECOM/bin

14

Restarting SMI-S

If SMI-S Provider is not running, or if Oracle VM shows messages such as “Connection to Admin host failed” restart SMI-S Provider

by using the following command:

[root@dsib0064 ~]# ECOM -d

INSTALLATION OF ORACLE VM MANAGER

Oracle VM Prerequisites

Oracle VM Manager is required to be installed on a separate host than Oracle VM Servers.

Note: The Storage Management host and Oracle VM Manager may reside on the same host, or separate hosts. In this paper the

same host has been used for both.

For more information on Oracle VM Manager installation requirements refer to Oracle VM Installation and Upgrade Guide for Release

3.3. The main tasks are described below.

Install Oracle VM Manager

Install NTP Server on the Oracle VM Manager host to provide time coordination for the VMs.

1. Install the NTP server.

[root@dsib0064 ~]# yum install ntp

2. Configure the NTP Daemon to start at boot.

[root@dsib0064 ~]# chkconfig ntpd on

3. Start the NTP service.

[root@dsib0064 ~]# service ntpd start

4. Mount the Oracle VM Manager media and run the installer.

[root@dsib0064 ~]# mount -o loop /tmp/ovmm-3.3.1-installer-OracleLinux-b1065.iso /mnt [root@dsib0064 ~]# cd /mnt [root@dsib0064 mnt]# ./runInstaller.sh

During the installation MySQL database is installed for Oracle VM Manager repository.

Oracle VM Manager Post Installation Tasks

Oracle offers Oracle VM Utilities to help with the management of Oracle VM tasks from a command line. Refer to Oracle VM

Administrator’s Guide for Release 3.3 for a description of Oracle VM Utilities and their installation procedures.

Oracle also provides a package that should be installed on Oracle VM Manager host in order to provide VNC access to the Oracle VMs.

Download the ovmcore-console package from Oracle VM Download page and install it. If Linux Firewall is configured you’ll need to

open the appropriate ports or disable it. Refer to Oracle VM Installation and Upgrade Guide for details on the installation tasks of VNC

and Serial Console for Oracle VM.

15

In this example the ovmcore package is installed on the Oracle VM Manager host.

[root@dsib0064 OVM]# rpm -ivh ovmcore-console-1.0-36.el6.noarch.rpm warning: ovmcore-console-1.0-36.el6.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY Preparing... ########################################### [100%] 1:ovmcore-console ########################################### [100%]

If you wish to disable Linux firewall:

[root@dsib0064 ~]# service iptables stop [root@dsib0064 ~]# chkconfig iptables off

If you wish to disable SELinux:

[root@dsib0064 ~]# vi /etc/selinux/config ... SELINUX=disabled

Networking

Networking in a virtualized environment is an important topic that requires adequate thought and planning. Refer to Oracle VM

Concept Guide for Release 3.3 for details on such considerations. In this paper a single private network is added, as shown in Figure

4. It will be used for VM’s communication and Live Migration.

1. To create an additional network, choose the Networking tab.

2. Click to create a new network.

Figure 4 Configure Oracle VM networks

16

INSTALLATION OF ESI FOR ORACLE VM FOR X86

ESI for Oracle VM version 3.3 is the software that utilizes Oracle Storage Connect plugins to monitor and manage the VMAX storage

array from Oracle VM. Refer to Table 4 for installation prerequisites.

Prerequisites checklist

Table 4 ESI for Oracle VM prerequisites

ESI FOR ORACLE VM PREREQUISITES COMPLETE

1. ORACLE VM SERVERS INSTALLED

2. ORACLE VM MANAGER INSTALLED

3. SMI-S PROVIDER INSTALLED

Installation steps

1. Navigate to http://www.emc.com.

2. Navigate to Support > Downloads and search for ESI for Oracle VM.

3. Download the ESI for Oracle VM plugin.

4. Copy the plugin to each of the Oracle VM Servers.

5. Install the rpm on each of the Oracle VM Servers.

[root@dsib0062 ESI]# tar xvf esi_plugin.2.0.0.5_OVM-3.3.tar emc-osc-isa-2.0.0.5-1.el6.x86_64.rpm [root@dsib0062 ESI]# rpm -ivh emc-osc-isa-2.0.0.5-1.el6.x86_64.rpm Preparing... ########################################### [100%] 1:emc-osc-isa ########################################### [100%]

17

ESI Configuration File

ESI for Oracle VM uses a configuration file to pass user parameters that affect its behavior with Storage Connect.

Note: It is highly recommended to read the following section and be familiar with ESI configuration file options.

ESI for Oracle VM Configuration File Parameters:

AccessGroupPrefix: this parameter allows modifying the default prefix for VMAX Auto-provisioning Group names8. While

this prefix is not visible in Oracle VM Manager screens, it will be visible to the Storage Admin monitoring the VMAX storage

array using tools such as Unisphere for VMAX or Solutions Enabler CLI.

AutoMetaEnabled9: True means that the VMAX Auto-meta feature is configured and enabled. False means that it isn’t (you

can check by running Solutions Enabler command: symcfg list -v | grep “Auto Meta”). If left unset, the plugin will

inquire with the storage via SMI-S to check whether AutoMeta is enabled and will take the appropriate value.

Note: It is recommended to not set this parameter as its correct value will be found automatically by ESI plugin.

VolumeGroupPrefix: Provides a way of limiting which VMAX Pools are visible to Oracle VM. In this way Oracle VM can view

and access only certain storage Pools. The parameter should be consistent across all the Oracle VM Servers accessing the

same VMAX storage. Modifying this parameter after the storage was discovered will require removal of the array in Oracle

VM Manager and rediscovery. Prior to ESI for Oracle VM version 2.0.0.5, the Prefix can only contain part of the Pool name,

but not the full name of the pool. Starting with version 2.0.0.5 the prefix can include partial or full Pool name.

LogLevel: provides a way to control the level of detail in the Oracle VM Server log file located at /var/log/ovs-agent.log.

Set to “debug” for detailed information. Other values allowed: “info”, and “error”.

The log referenced in isa.conf is located at: /var/log/ovs-agent.log on the Oracle VM Server.

Create a new configuration file, or copy an existing one from a previously installed Oracle VM Server with ESI:

[root@dsib0062 ESI]# vi /opt/storage-connect/plugins/emc/isa/isa.conf AccessGroupPrefix=osc_ AutoMetaEnabled=false VolumeGroupPrefix=FC,SATA LogLevel=debug

Note: If the ESI plugin is upgraded, perform rediscovery of the Oracle VM Servers by right clicking on each and choosing

Rediscover Server from the menu.

8 The default prefix is osc_ for Oracle Storage Connect. A different one, such as ovm_ can be easily used. 9 Meta volumes is a feature available in VMAX with Enginuity code level 5876 and below. It allows for storage devices aggregation as a striped or

concatenated group of devices, and at the same time allows the group (meta-volume) to grow beyond the 240GB limit of a single device. VMAX3 with

HYPERMAX OS 5977 and above no longer supports or requires meta-volumes as all devices are virtual and natively striped, and each device capacity

can reach multi-terabytes.

18

Registration of Oracle VM Servers containing ESI Plugin with Oracle VM Manager

Once the ESI for Oracle VM plugin has been installed on the Oracle VM Servers, the Oracle VM Servers and VMAX storage array will

both need to be registered (even if the servers were previously registered with Oracle VM Manager).

1. Open a browser and navigate to https://<hostname>:7002/ovm/console

Example: https://dsib0064:7002/ovm/console

2. Log in with username admin using the password that was set during the installation of Oracle VM Manager.

3. Navigate to the Servers and VMs tab.

4. Click on the Discover Servers icon .

5. Enter the Oracle VM Agent password and Oracle VM servers IP addresses/hostnames. Click OK, as shown in Figure 5.

Figure 5 Discover Oracle VM Servers

19

6. Once discovered, the servers will be displayed under Unassigned Servers, as shown in Figure 6.

Figure 6 Discovered Oracle VM Servers

Note: In case the discovery encounters a failure during storage refresh action validate that the Oracle VM Servers can each ping the

management host back (/etc/hosts or DNS are set correctly in each Oracle VM Server), and that SMI-S is running.

20

VIEW VMAX STORAGE INFORMATION

Discovery of VMAX Storage Array

In order to discover the VMAX storage array and perform cloning and provisioning operations, the first task is to register the SMI-S

provider that has access to the VMAX storage.

1. Navigate to the Storage tab and click on the Discover SAN Server icon , as shown in Figure 7.

Figure 7 Discover VMAX Storage Array

2. Update the following fields appropriately, as shown in Figure 8.

Name – User defined name for EMC storage array

Description – User defined description for EMC storage array

Storage Type – Select FibreChannel

Storage Plug-in – Select EMC FC SCSI Plugin

Plug-in Private Data – Provide the VMAX array ID (either full or partial ID number)

Note: This field is mandatory for ESI plugin. Partial ID is allowed only for the end of the ID (e.g. 198700067, or 067 are valid)

Admin Host – Provide the host name or IP address of the host where SMI-S is installed

Admin Username – Provide the SMI-S admin username. The default is admin

Admin Password – Provide the SMI-S admin user password. The default is #1Password

21

Figure 8 Enter Storage Discovery Information

3. Click Next to continue. The Oracle VM Servers appear under Available Server(s). Move them to the right side under Selected

Server(s) then click Finish.

Identifying VMAX storage information from Oracle VM Manager

After the discovery job completes, the storage will appear with basic information, as shown in Figure 9.

Figure 9 Storage Summary

22

More information about the storage array such as the Pools and the devices associated with each pool can be viewed by selecting the

storage from the list of SAN Servers. Additional VMAX device information, such as device ID (appears as “Name”, or “Extra

Information”) can be viewed by expanding any of the storage devices listed, as shown in Figure 10. Storage devices also show which

of the Oracle VM Servers they are visible to, Access Groups they belong to, etc.

Figure 10 Storage Devices Information

Identifying VMAX storage devices directly from Oracle VM Server

Oracle VM Administrator can view basic VMAX device information from the Oracle VM Servers by using Linux commands such as

multipath or scsi_id.

With EMC PowerPath, powermt display dev=all command can be used. If Linux dm-multipathing is used, it can list multipath

devices with their name, serial, and capacity.

[root@dsib0062 ~]# multipath -l 360000970000198700067533030313435 dm-1 EMC,SYMMETRIX size=100G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=0 status=active `- 3:0:0:1 sdc 8:32 active undef running 360000970000198700067533030323235 dm-10 EMC,SYMMETRIX size=100G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=0 status=active `- 3:0:0:10 sdl 8:176 active undef running 360000970000198700067533030323234 dm-9 EMC,SYMMETRIX size=100G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=0 status=active `- 3:0:0:9 sdk 8:160 active undef running ...

23

Linux scsi_id command can also list storage device information with the addition of the VMAX Device ID. The field ID_SCSI_SERIAL

contains the last 6 digits of the VMAX Storage ID followed by the 3 digits of the VMAX Device ID.

[root@dsib0062 ~]# for i in `ls /dev/dm-*`; do echo $i; scsi_id -g -x $i | grep ID | grep SERIAL= ; done /dev/dm-0 ID_SERIAL=360000970000198700067533030303134 ID_SCSI_SERIAL=700067014000 /dev/dm-1 ID_SERIAL=360000970000198700067533030313435 ID_SCSI_SERIAL=700067145000 /dev/dm-10 ID_SERIAL=360000970000198700067533030323235 ID_SCSI_SERIAL=700067225000 ...

USE CASES

The following section assumes that all the configuration steps have been completed and provide an introduction to some of the basic

Oracle VM operations that Storage Connect helps achieve.

Oracle VM Storage Connect with ESI provides the Oracle VM Administrator with a way of performing tasks such as storage creation,

provisioning, and cloning with ease and simplicity.

The first use case shows basic storage provisioning tasks. Following use cases demonstrate how to provision storage devices to

Oracle VM Servers, Virtual Machines, create storage, repositories, and how to perform cloning operations at the VM level.

USE CASE 1 – BASIC STORAGE MANAGEMENT OPERATIONS

Review available storage pools capacities

Before adding new storage devices, let’s inspect the available capacities of the storage pools.

1. Choose the Storage tab at the top of the Oracle VM Manager screen.

2. On the left, under SAN Servers, choose the appropriate storage array, as shown in Figure 11. Click on the small triangle to the

left of the Storage name to expand its details and the available storage pools will be listed. By using the Info perspective it is

easy to view how much capacity is available in each storage pool. In Figure 11 we can see that the FC_Pool has 3,611.39 GB of

available capacity.

Figure 11 View Storage Pool Available Capacity

24

Create New Storage Devices

To create a new storage device change to Physical Disks perspective (refer also to Figure 14).

1. Click on the Create Physical Disk icon , as shown in Figure 12.

Figure 12 Create Physical Disk

Volume Group: If a specific array Storage Pool was already selected it will be used as Volume Group. If the Array Name was

selected instead, a drop-down list of the available storage pools will be listed. Select the appropriate storage pool.

Name: Provide a descriptive name for the new device.

Extra Information: This box can be ignored. It can be used during creation of storage meta volumes when Auto-meta is not

configured in the array. For example, it could say: meta_member_size=200G. However, it is optional even when creating meta

volumes.

Description: Provide an appropriate description for the new device.

Shareable: Specifies for Oracle VM that the device can be shared between multiple Virtual Machines with simultaneous read/write

access.

Thin Provision: Only Thin devices are supported for Oracle VM Storage Connect with ESI 2.0

25

2. Click OK to create the storage device.

Once the device is created, it will show in the Physical Disk list for the storage array. By clicking on the small triangle to the left of

the device name more information can be viewed, as shown in Figure 13. The VMAX device ID is shown to the right of Extra

Information field. For example, Oracle ERP Disk 1 is using device ID 625.

Figure 13 Show Device Information

The Storage Admin can identify the Oracle VM devices either by their ID (for example: device ID 625), or by their name. To list

device names they can use Solutions Enabler command: symdev list -ident device_name.

[root@dsib0064 ~]# symdev list -ident device_name Symmetrix ID: 000198700067 Device ------------------------------------------------------------------------------- Sym Config Attr Device Name ---- --------------- ---- ----------------------------------------------------- 01F3 TDEV ORA_DATA1 0622 TDEV Template1 0623 TDEV Volume 00147_clone 0624 TDEV Volume 00148_clone 0625 TDEV Oracle ERP Disk 1 0626 TDEV Volume 00147_clone ...

26

Manage Storage Devices

By selecting a VMAX storage device, a number of operations can be executed on the device, as shown in Figure 14.

Figure 14 Manage Storage Device

Device operations:

Edit Device: Attributes such as Device Name, Description, and Shareable can be modified.

Note: It is not allowed to change device capacity.

Delete Device: allows the user to delete the device.

Clone Device: allows the user to clone a device.

In the Clone Target Type field select: Storage Array.

In the Name field provide an optional name for the clone.

o If a name wasn’t provided the cloned device will show as the original device name with the addition of a

bracket and a clone number. For example, ‘Oracle ERP Disk 1 (2)’ is a default name for a clone to a device

named ‘Oracle ERP Disk 1’.

o If the name provided ends with ‘_EMC_SNAP’, then rather than a full-sized clone, a space-efficient snapshot

will be created with copy-on-write functionality (only changes to either source or target devices consume

additional storage capacity)

Note: As there is no other way in Storage Connect to pass information to the array, whether to create a full

copy or space-efficient copy of the source device, use the clone name to do so. The suffix ‘_EMC_SNAP’ will

result in a space-efficient clone, aka snap.

27

Only Thin Clone value is allowed in the Clone Type field.

After the snap or clone was created, as before, by clicking on the small triangle to the left of the clone device name its

VMAX device ID can be viewed in the Extra Information field.

USE CASE 2 – PROVISION STORAGE TO ORACLE VM SERVERS

Provisioning storage devices to the Oracle VM Servers is done by creating Access Groups. Storage devices in Access Groups can later

be used as storage for the Oracle VM Servers (for example: Server Pool storage, or Repositories), or provisioned to the virtual

machines as physical disks.

Note: This section assumes that storage devices were already created for Oracle VM and can be simply added to the Access Group.

To learn about creating new storage devices refer to Use Case 1.

In order to create an Access Group:

1. Select the Storage tab.

2. Choose Access Group perspective from the drop-down menu.

3. Click on the icon that creates a new Access Group, as shown in Figure 15.

Figure 15 Create Access Group

4. In the window that opens provide a name and description for the Access Group. Click Next.

Note: Access Group name must not contain any spaces. It will be used to create a VMAX Storage Group name (together with a pre-

defined prefix that can be modified in the ESI configuration file).

28

5. In the Initiators (HBA ports) selection screen, on the left side you can see a list of the Oracle VM Servers initiators prefixed by

the server name and followed by the port WWN. You may notice some WWN’s showing only zeros. These can be ignored as they

are not showing storage connectivity. Only initiators that have been zoned correctly to the storage array will show their WWN as

a non-zero address and can be selected for the Access Group HBA ports connectivity.

6. Move the appropriate initiators to the right side of the screen as shown in Figure 16. An Access Group that includes initiators of

multiple Oracle VM Servers can provide shared storage to these servers. Click Next.

Figure 16 Access Group initiators selection

7. In the next screen select the storage devices that will be provisioned to the Oracle VM Servers, as shown in Figure 17. Click

Finish to complete the task.

Figure 17 Access Group storage selection

29

After the Access Group is created, by switching to the Physical Disk perspective, the Server column will reflect the Oracle VM

Servers in the Access Group for the appropriate devices, as shown in Figure 18.

Figure 18 Access Group servers listed

USE CASE 3 – CREATE A SERVER POOL

The Oracle VM Servers can be placed in a Server Pool to allow shared storage for repositories, high availability and easy migration of

virtual machines.

1. To create a Server Pool select the Servers and VMs tab and click on the icon that creates a Server Pool, as shown in Figure 19.

Figure 19 Create server pool

30

2. Provide a Server Pool Name and a Virtual IP to manage the pool. Note that Clustered Server Pool will be checked by default.

3. In the Storage for Server Pool field choose Physical Disk and click on the search icon to select storage, as shown in Figure

20.

Figure 20 Storage selection for server pool

Storage devices visible to the Oracle VM Servers will be listed. Select one of them for the Server Pool shared storage. An OCFS2

filesystem will be created on this device and mounted on the Oracle VM Servers selected for the Server Pool. Click OK and you’ll be

returned to the Create Server Pool screen where you can add a description. Click Next. At this time the Oracle VM Servers for the

Server Pool should be selected. Move the appropriate servers to the right side of the screen and click Finish to complete the task, or

Next to add Tags. The Sever Pool will be listed upon completion, as shown in Figure 21.

Figure 21 Server pool created

31

USE CASE 4 – CREATE A REPOSITRY FOR SERVER POOL

For Server Pool clustering, a shared storage is required. For SAN storage, a disk image is created and formatted using OCFS2 (Oracle

Clustered Filesystem).

1. Change to the Repositories tab and click on the icon to add a repository, as shown in Figure 22.

Figure 22 Create a storage repository

2. In the Create a Repository window enter the repository name.

3. Choose the Physical Disk radio button and the Server Pool the Repository will belong to.

4. Next, click on the icon to discover a Physical Disk for the repository storage allocation, as shown in Figure 23.

Figure 23 New Repository - Storage Selection

5. In the Select Physical Disk screen select the VMAX storage array and Pool. A list of shared storage devices will be available.

Note in the example that the first device (volume 00145) is no longer visible as it was selected previously for the Server Pool

storage. The next volume, 00146, is therefore selected in this example.

6. After the storage selection for the repository you’ll be returned to the Create Repository screen where you can add a

description. Click Next. Select the Oracle VM Servers for the repository by moving them to the right side of the screen and click

Finish.

32

The Repository, as shown in Figure 24, is created and divided into 5 categories as explained in the Oracle VM Concept Guide:

Assemblies, which are a preconfigured collection of virtual machines, typically created with Oracle Assembly Builder.

ISO files that can be mounted as a virtual CD/DVD.

VM Templates for virtual machine templates (templates can be downloaded from Oracle website, or created).

VM Files containing the configuration files of the virtual machines.

Virtual Disks that can be dedicated or shared between VMs.

Figure 24 Newly Created Repository

33

USE CASE 5 – CREATE VIRTUAL MACHINES FROM A SERVER POOL

Import an Oracle VM Assembly

An easy way to create virtual machines is from Assemblies or Templates. Assemblies and Templates can be downloaded from Oracle

Software Delivery Cloud under Oracle VM Templates. They are not installed directly into the Oracle VM Servers and rather imported

by Oracle VM Manager using http, https, or ftp.

In this example, vsftpd (secure ftp), is installed on the Oracle VM Manager host and the vsftpd service started. An OVF set is selected

from Oracle website: Oracle VM 3 Templates (OVF) for Oracle Linux 6 Media Pack v7 for x86_64 (64 bit). It opens a new window in

the browser with multiple OVFs. Download and copy to the Oracle VM Manager host the appropriate template. Once copied, unzip the

packages.

[root@dsib0064 OVM]# cd /var/ftp/OVM [root@dsib0064 OVM]# unzip OVF_OEL65_V45138-01.zip

Import the OVA as an Assembly first, and then turn it into a Template. To do so:

1. Go to the Repository tab and choose Assemblies under the previously created Repository (Repository 1).

2. Click on the Import icon, as seen in Figure 25 . In this example ftp protocol is used with the ftp user on the Oracle VM Manager

host. The OVM is a sub-directory under the ftp user home directory (/var/ftp/OVM) where the .ova file is stored. The following

link is inserted in the Assembly download location: ftp://ftp:ftp@dsib0064/OVM/OVM_OL6U5_X86_64_PVM.ova.

Figure 25 Import an Assembly

34

Create an Oracle VM Template from an Assembly

1. Once the Assembly is imported select it, and click the icon to create a VM Template, as shown in Figure 26.

2. In the window that opens enter the template name and click OK.

Figure 26 Create a VM Template

Create an Oracle VM from a Template

Under Servers and VM tab click on the icon to create a new virtual machine. For example, the previously imported template can be

used to create 3 VMs by selecting Clone from an existing VM Template, and providing the appropriate information, as shown in

Figure 27.

Figure 27 Create New Virtual Machines

35

3. Select a VM and click on the Edit icon, as shown in Figure 28.

4. Under the Network tab add the private network created earlier for VMs. Under Storage tab add Physical or Virtual disks as

necessary. Virtual Disks capacity is taken from the storage device assigned to the Server Pool Repository. Physical Disks can be

chosen from the Access Group storage associated with the Server Pool.

5. After the update is completed the VM can be started.

Figure 28 Update VM Information

36

USE CASE 6 – CLONE VIRTUAL MACHINES

A Virtual Machine can easily be cloned together with its assigned storage devices. To do so:

1. Select the VM from the list, right click and choose Clone or Move, or click on the icon: Clone or Move Virtual Machine.

2. Optionally, provide a Clone Name, Count, and Index, as shown in Figure 29. It is also possible to set an advanced cloning with

Clone Customizer (for example, to use storage from another Repository).

3. To finish the operation click OK.

Figure 29 Clone a Virtual Machine

37

In this following example the VM OL6U5_X86_64_PVM.1 is cloned. When the clone operation completes, a new VM appears with a .0

suffix to the originator VM name (as the Name Index in the Clone Creation screen was set to “0”). If the Cloned VM storage details

are reviewed, as shown in Figure 30, it is easy to see that the original Physical and Virtual Disks were cloned. The new Virtual Disk

capacity is again taken from the Repository storage, while the Physical disk capacity is created as a new VMAX storage device and

automatically added to the Server Pool (by automatically joining the Access Group).

Figure 30 Review Cloned VM Storage Details

Figure 31 Review Cloned Storage Device

To see the new device information move to the Storage Tab, and review the new device details. Under Extra Information it is easy to

see that it has the VMAX device ID 00626, as shown in Figure 31.

38

CONCLUSION

Oracle VM and Storage Connect APIs, together with ESI and VMAX provides a robust set of storage capabilities, offer the

virtualization administrator a complete set of tools to manage their Oracle VM environment while performing tasks such as storage

provisioning and VM cloning with ease. By utilizing this integration, many tasks that previously required intervention by multiple IT

groups, such as Storage, Network, and Virtualization, can now be executed with just a few clicks directly from Oracle VM Manager.

The overall result is increased business agility, simplicity of operations, and better ability to manage virtualization and cloud

operations under Oracle VM.

REFERENCES

EMS Storage Integrator for Oracle VM Version 3.3 Product Guide

EMS Storage Integrator for Oracle VM Version 3.3 Release Notes

Oracle VM for x86 (http://www.oracle.com/us/technologies/virtualization/oraclevm/overview/index.html)

Oracle product documentation library: http://www.oracle.com/technetwork/indexes/documentation/index.html#virtualization

EMC SMI-S Provider version 4.6.2 Release Notes