oracle vm 3 change the server pool file system & repositories

8/20/2019 Oracle VM 3 Change the Server Pool File System & Repositories

1/32

An Oracle White Paper

August 2014

Oracle VM 3:Change the Server Pool File System &Repositories (SAN)


2/32

White Paper – Change the Server Pool File System & Repositories (SAN)

ContentsIntroduction ...................................................................................... 1

Solution Overview ............................................................................ 2

Only supported with Oracle VM 3.2.8+ ................................................. 2

Important terms used throughout the document ................................... 2

Goal of the solution ............................................................................... 3

A note about destroying the server pool ............................................... 3

The overall solution at a glance ............................................................ 3

Key solution highlights explained .......................................................... 4

Maintain continuous replication of storage repositories ..................... 4

Use replicated disks to create a new pool ......................................... 5

The device special file name will change .......................................... 5

The storage repository names remain the same ............................... 5

The server pool name remains the same .......................................... 6

Process flow ......................................................................................... 6

Step 1: Preparation Prior to Implementation .................................... 7

Step 1.1: Write down the server pool VIP ............................................. 7

Step 1.2: Catalog current storage ......................................................... 7

Step 1.3: Establish continuous replication of storage ............................ 7

Step 1.4: Perform ad-hoc backup of Oracle VM servers ....................... 8

Step 2: Release Ownership of Repositories .................................... 9

Step 2.1: Perform ad-hoc backup of Oracle VM database .................... 9

Step 2.2: Stop Oracle VM guests running in the pool .......................... 11

Step 2.3: Release ownership of current repositories ........................... 11

Step 3: Destroy Current Server Pool ............................................. 13

Step 3.1: Remove Oracle VM servers from pool ................................. 14

Step 3.2: Delete the empty server pool ............................................... 15

Step 3.3: Perform final synchronization of storage .............................. 16

Step 4: Delete Source Disks .......................................................... 16

Step 4.1: Un-map source LUNs .......................................................... 17


3/32


Step 4.2: Delete the physical disks from DB ....................................... 17

Step 5: Discover Target Disks ....................................................... 18

Step 5.1: Map the target LUNs ........................................................... 19

Step 5.2: Discover the new storage platform ...................................... 19

Step 5.3: Refresh the Oracle VM database......................................... 19

Step 5.3.1: If you are using Fibre Channel ...................................... 19

Step 5.3.2: If you are using iSCSI ................................................... 20

Step 6: Rebuild the Server Pool .................................................... 21

Step 7: Take Ownership of Target Repositories ............................ 24

Step 7.1: Refresh the target disks ....................................................... 24

Step 7.2: Refresh the target repositories ............................................ 25

Step 8: Start Oracle VM Guests .................................................... 27


4/32


1

Introduction

This white paper explains how to change the device special file (DSF) being used as a server

pool file system. This solution can be applied to a variety of situations, but the most common

need for this solution is migrating the pool file system and storage repositories used by a

server pool from a legacy storage array to a new or different storage array. In this case,

system administrators are charged with migrating storage repositories as well as pool file

systems. It is the pool file system that poses the challenge.

Oracle VM 3 does not currently have a built-in tool or way to change the pool file system

without incurring downtime of all running Oracle VM guests. This is because the server pool

must be temporarily destroyed and rebuilt using the new pool file system. There are two major

ways to approach the migration from one pool file system to another.

The first approach presented in this white paper is straightforward with well defined stages that

make it easier for people to follow. This approach involves stopping all Oracle VM guests,

temporarily destroying the server pool and rebuilding the server pool again using a new pool

file system.

Other than this brief explanation, we will not be explaining how to accomplish the second

approach at all since it is a little more complex with slightly blurred stages and doesn’t really

provide any advantage over the first approach. This approach involves using one Oracle VM

server from the legacy server pool to build a new server pool using the new pool file system,

then migrating the remaining Oracle VM servers one-by-one until everything is up and running

in the new pool. You still incur an outage of all Oracle VM guests at various stages using this

method and you will need another server pool VIP, so there is really no advantage to this

process except in cases where you cannot tolerate an outage of all virtual machines during the

same maintenance window.


5/32


2

Solution Overview

This solution assumes all SAN storage related to a single server pool is being migrated to a new storage

platform including the server pool file system and all storage repositories. In addition, the process of

destroying and then creating a new server pool also changes the OCFS2 cluster ID. This means the

storage repositories will not be available to the new server pool unless these they are first released from

ownership by the Oracle VM Manager.

This document is divided into several parts representing each step in the entire process. The first part

of the document explains the process, process flow, concepts and terms used throughout the

document. The remaining parts of the document are dedicated to accomplishing the steps one-by-one.

Important!!

Please take the time to read and understand the solution before moving directly to the step-by-step process

articulated in this document.

Only supported with Oracle VM 3.2.8+

Your Oracle VM platform must be at release 3.2.8 or higher; you must upgrade your Oracle VM

platform to a minimum release level of Oracle VM 3.2.8 before you can successfully use this solution.

The following table shows supported and unsupported versions of Oracle VM:

ORACLE VM RELEASE SUPPORT STATUS REMOTE REPLICATION

Oracle VM 3.2.1 Not supported This release of Oracle VM does not allow Oracle VM guests to be migrated fromSAN repositories to Unassigned Virtual Machines. Therefore ownership ofrepositories cannot be released which is a critical step in the process.

Oracle VM 3.2.2 Not supported





Oracle VM 3.2.8 Supported Solution is fully tested and supported

Oracle VM 3.3.1 Supported Solution is supported

Table 1: Supported releases of Oracle VM 3

Important terms used throughout the document

For the purpose of this paper, we will use the following terms throughout the document:

Model: A common term used to everything contained in the entire Oracle VM managementdatabase; everything you can see and access in the Oracle VM Manager user interface.

PoolFS: Server pool file system

SAN: Any storage presented using block protocols such as Fibre Channel (FCP) or iSCSI

Source storage: This refers to the legacy, old or current PoolFS and storage repositories associated with a single server pool; the source contains the original data that is being migrated from the legacystorage platform to the new storage platform.

Storage platform: This refers to either a physical storage array or logical volumes on a physicalstorage array. Perhaps you are migrating existing storage from one physical array to another, orperhaps you are instead migrating storage from one logical volume to another logical volume thatboth reside on the same physical storage array.


6/32


3

Target storage: This refers to the new, cloned or replicated PoolFS and storage repositories that willbe used to create the new server pool.

Goal of the solution

The goal of this solution is to migrate all of the storage being used for a server pool from a legacy

storage array or volume to a new storage array or volume. This particular solution assumes you are

migrating the storage repositories and your PoolFS from your current storage platform to a new

storage platform. The process will require downtime during the final storage replication and migration

stages.

As stated previously, this process will require you to rebuild the server pool using the new PoolFS.

Despite the ominous sounding step of destroying the server pool, it does not impact the Oracle VM

Manager, Oracle VM servers, Oracle VM guests, networking or anything else related to your existing

Oracle VM platform. The actual rebuilding of the server pool should take a minimal about of time.

Basically, you remove the servers from the source pool, destroy the source pool, refresh the storage

and then build a new pool with the same servers. There is no need to reinstall or reconfigure anythingother than creating the target server pool.

A note about destroying the server pool

The process described in this document requires you to destroy the current server pool and then

rebuild the same server pool using the replicated disks from the new storage platform. This is a

requirement and there is no way around it.

However, destroying the server pool is not as dramatic as it sounds. Nothing about Oracle VM is really

impacted by this step other than the server pool won’t exist for a few minutes while we build the new

server pool. You will not need to reinstall or reconfigure anything on the Oracle VM servers; all of the

bonding, networking and anything else about the Oracle VM servers remain unchanged.

In fact, there is nothing destructive about any of the steps related to your Oracle VM guests; you still

have all the parts and pieces to rebuild the same server pool and your virtual machines are quite safe.

We explain how to back up your Oracle VM management database and the Oracle VM servers in steps

1 & 2 that make it very easy to recover in the unlikely case you need to back out of the solution.

Important!!

Destroying the server pool is not as dramatic as it sounds. Everything related to your Oracle VM guests is

still there for you to use and it is very easy to recover if you follow the instructions in steps 1 and 3.

The overall solution at a glance

Figure 1 below shows the state of the storage for a fictional server pool prior to beginning the

migration process while Figure 2 the state of the storage for the new server pool after completing the

migration process.


7/32


4

This is shown only to explain the differences between the beginning and end of the solution. The

numbered items in each of the illustrations highlight key points that are accompanied by detailed

explanations below.

Legacy Storage

New Storage

Oracle VM Servers

/poolfsmnt/0004fbxxxx6eb0 [Mypool2] /dev/mapper/360a98xxxx3056

/OVS/Repositories/0004fbxxxx20f8 [Mypool2 Repo1]

RepoFS Clone

RepoFS LUN

PoolFS Clone

/dev/mapper/360a98xxxx302d

/OVS/Repositories/0004fbxxxx01fa [Mypool2 Repo2]

RepoFS Clone

RepoFS LUN/dev/mapper/360a98xxxx3845

4 3

15

PoolFS LUN

2

Figure 1: Showing the overall state of the storage before beginning the process

Legacy Storage

New Storage

Oracle VM Servers

/poolfsmnt/0004fb00xxxx0ad6 [Mypool2] /dev/mapper/360a98xxxx82a0

/OVS/Repositories/0004fbxxxx6cc2 [Mypool2 Repo1] RepoFS LUN

PoolFS LUN

/dev/mapper/360a98xxxx4453

/OVS/Repositories/0004fbxxxx1b3c [Mypool2 Repo2] RepoFS LUN/dev/mapper/360a98xxxx4456

RepoFS Clone

PoolFS Clone

RepoFS Clone

5

4 3

12

Figure 2: Showing the final overall state of the storage after completing the process

Key solution highlights explained

So let’s break down some of the more important concepts shown in Figure 1 & Figure 2 above needed

to successfully complete the migration process.

Maintain continuous replication of storage repositories

To decrease the downtime associated with the migration process, ensure that you maintain acontinuous replication relationship between the source and target storage platforms until you are ready

to actually stop your Oracle VM guests and release ownership of the storage repositories (see Figure 1,

item 1). You should start the replication process well before you begin the actual migration

documented in this white paper.


8/32


5

The replication relationship should be maintained and periodically refreshed minimize the delta

between the source and replica storage; this just means periodic replication, not synchronous

replication. For example, you would set up continuous replication between the source and target

storage platforms with scheduled refreshes until the server pool is destroyed in Step 3 of this

document. Notice in Figure 2, item 1 that the replication relationship no longer exists between the

source and target storage repositories after the final refresh has been completed and the old server pool

is destroyed.

Use replicated disks to create a new pool

The process is completely predicated on replicating/cloning the current LUNs being used for the

current server pool and then completely rebuilding the server pool using the replicated disks. You will

use the current LUNs as shown in Figure 1, item 2 until Step 6 where you build an entirely new server

pool using the replicated LUNs from the new storage platform as shown in Figure 2, item 2.

The device special file name will change

Compare item 3 in Figure 1 & Figure 2; notice that the WWID (page83 ID) of the device special files

(DSF) for the repository LUNs are different. Our illustration shows a truncated device name showing

only the last four significant digits of the page83 ID – this is simply to help us fit the illustrations on

the page.

During the process you will stop all the Oracle VM guests and destroy the server pool. Destroying the

server pool is a requirement because the DSF name for the PoolFS will be different when you present

the new PoolFS from the new storage platform. Since Oracle VM has no built-in features to simply

change the device special file, we need to destroy the current server pool and create a new one using

the same servers but a different PoolFS. The new PoolFS DSF will have a different WWID.

The storage repository names remain the same

Notice in both Figure 1 & Figure 2, item 4 that the storage repository names will remain the same

throughout the entire process. The simple name of the repository is contained in a hidden file within

the top level directory named .ovsrepo which will be seen in the Repositories tab of Oracle VM

Manager. Each newly discovered repository will be assigned a random ID by the Oracle VM Manager

as it is added to the management database and will be mounted to /OVS/repositories on the Oracle

VM servers using the new repository ID.

Note

Keep in mind that the Oracle VM guests will not be seen in the new server pool until the target repositories

have been refreshed in Step 9 of this document.

Your Oracle VM guests will remain safe and entirely intact during the entire process. It is normal for

the Oracle VM guests to remain undiscovered in the new server pool until the repositories have been

discovered, presented to Oracle VM servers in the new pool and the repositories refreshed in Step 9 of

this document.


9/32


6

The server pool name remains the same

Figure 2, item 5 shows that you will use the same server pool simple name when you create the new

pool. Item 5 also shows that the newly discovered PoolFS will be assigned a random ID by the Oracle

VM Manager as it is added to the management database and will be mounted to /poolfsmnt on theOracle VM servers using the new PoolFS ID.

Process flow

The table of contents shows all the steps for the entire process. However, we will show the following

process flow diagram at the beginning of each step to help you keep track of where you are in the

overall process.

Migrating a Server Pool to a New Storage Platform

Step 1:

Preparation

Step 2:

Release

ownership of

repos

Step 3:

Destroy current

pool

Step 4:

Delete source

disks

Step 5:

Discover target

disks

Step 6:

Rebuild server

pool

Step 7:

Take ownership

of repos

Step 8:

Start Oracle VM

guests

Figure 3: Process flow


10/32


7

Step 1: Preparation Prior to Implementation


Step 1:Preparation

Step 2:Release

ownership of

repos

Step 3:Destroy current

pool

Step 4:Delete source

disks

Step 5:Discover target

disks

Step 6:Rebuild server

pool

Step 7:Take ownership

of repos

Step 8:Start Oracle VM

guests

There will be an outage associated with the migration project since you will be destroying and

rebuilding your target server pool You should plan ahead to ensure the outage window is as small as

possible.

The following steps should be performed weeks or days before the planned outage. The choice of

timing to begin the following steps is completely at your discretion since you know how much storage

you will be migrating and how long the preparatory steps might take in your data center given your

operational governance and standard operating procedures.

Step 1.1: Write down the server pool VIP

You will need to remember the server pool VIP since you will use the same one to create the new

server pool after deleting the current server pool.

Step 1.2: Catalog current storage

You will need to get a list of current OCFS2 formatted devices from one of your servers before you

start the process. This will provide a list of the PoolFS and storage repositories that are using OCFS2.

Step 1.3: Establish continuous replication of storage

Begin the storage replication of PoolFS and repositories well ahead of your scheduled outage. You willperform a final refresh/synchronization of the PoolFS and repositories as part of Step 3 when we

destroy the server pool. Perform periodic refreshes of the replicated storage in between the time you

first create the replication relationship and the final refresh. Performing periodic refreshes of the

replica storage will reduce the time needed for the outage window.

Important!!

Start your storage replication a day or two before your scheduled outage


11/32


8

The replication relationship between the source and target storage platform should remain in place

until you destroy the server pool. You will use software on your storage array to accomplish the

storage replication. The following table shows some examples of storage replication software from

various storage vendors.

STORAGE VENDOR LOCAL REPLICATION REMOTE REPLICATION

EMC TimeFinder SRDF

Hitachi Data Systems ShadowImage TrueCopy

Network Appliance Snapshot cloning SnapMirror

Oracle ZFS storage appliance Snapshot cloning Remote replication

Table 2: Examples of replication software

If you do not have enterprise class storage with robust replication software, then you can use whatever

standard Linux tools you are comfortable using. The following table shows a few Linux commands

that can be used if you don’t have enterprise class replication tools.

LINUX COMMAND COMMENTS

rsync Recommended: Faster, displays progress of copies and only copies changed blocks on

subsequent writes

cpio Not recommended: Slow, complex to use for remote copies and copies both changedblocks and unchanged blocks on subsequent writes

dd Not recommended: Slow, complex to use for remote copies and copies both changedblocks and unchanged blocks on subsequent writes

scp Not recommended: Slow and copies both changed blocks and unchanged blocks onsubsequent writes

Table 3: Examples of common Linux tools

The rsync command is probably the best Linux tool to use since it will keep track of changed blocks

between the two copies. The rsync command will only copy changed blocks of data on subsequent

copies of the same repository, so rsync will be much faster for the final synchronization of all storage

repositories during Step 3.

Step 1.4: Perform ad-hoc backup of Oracle VM servers

This step combined with the ad-hoc Oracle VM management database backup in step 2 will ensure an

easy way to back out of the destroying the server pool performed in Step 3. It is highly unlikely that

you will need to back out of the process after destroying the server pool, but it is safer to error on the

side of caution.

The Oracle VM related configuration information on the Oracle VM servers is almost static with very

few changes ever being written to the servers once they have been put into production. It is assumed

you will not be making any major changes to the servers prior to the scheduled outage. So, it is okay to

back the servers up weeks or days in advance of the planned outage.

We will simply make copies of a few key directories and files on the Oracle VM servers. But the copies we make on the Oracle VM servers combined with the Oracle VM management database backup we

perform in step 2 should allow you to completely recover from the destroyed server pool in less than

twenty or thirty minutes.


12/32


9

It should be fine to perform the Oracle VM server backup a day or two prior to your outage, but the

backup of the Oracle VM management database should be performed in Step 2 just before you begin

the outage.

Log onto each Oracle VM server in the target server pool and perform the three tasks show in thefollowing two screen shots. The first task is to save a copy of the local Berkeley DB on the server as

shown in

[root@myserver1 ~]# cd /etc/ovs-agent

[root@myserver1 ovs-agent]# cp -rp db db.save

[root@myserver1 ovs-agent]#

Figure 4: Save a copy of the local Berkeley DB on each server

Save the OCFS2 configuration as shown in Figure 5 below.

[root@myserver1 ~]# cd /etc

[root@myserver1 etc]# cp -rp ocfs2 ocfs2.save

[root@myserver1 etc]# cd /etc/sysconfig

[root@myserver1 sysconfig]# cp -rp o2cb o2cb.save

Figure 5: Save a copy of the current OCFS2 configuration

The local server side database and OCFS configuration files are the only things that are changed when

you delete or remove an Oracle VM server from a server pool. If you need to back out of the process,

the above files and directories can simply be restored along with the Oracle VM management databaseyou will back up in the next step.

Step 2: Release Ownership of Repositories


Step 1:

Preparation

Step 2:

Release

ownership of

repos

Step 3:

Destroy current

pool

Step 4:

Delete source

disks

Step 5:

Discover target

disks

Step 6:

Rebuild server

pool

Step 7:

Take ownership

of repos

Step 8:

Start Oracle VM

guests

You must release ownership of the storage repositories before you perform the final refresh or

synchronization of the replicated storage.

Step 2.1: Perform ad-hoc backup of Oracle VM database

You need to take an ad-hoc backup of your Oracle VM management database as a first step. This is

documented in the Oracle VM administration guide, but the following screen shot shows an example

of creating a backup of the database. This should only take a few minutes and will be worth your time.


13/32


10

Oracle VM 3.2 and 3.3 have slight variations in the name and location of the backup script. Use the

backup process show in Figure 6 for Oracle VM 3.2:

[root@mymanager ~]# cd /u01/app/oracle/ovm-manager-3/bin/

[root@mymanager bin]# ./createBackup.sh -u Admin –n before-deleting-mypool2

Backing up the Oracle VM Manager MySQL Database...

Please enter the Oracle VM manager user password:

INFO: Succesfully backed up database as before-deleteing-mypool2-20140821_073506

[root@mymanager bin]#

Figure 6: Oracle VM 3.2 - creating a very quick ad-hoc backup of the Oracle VM management database (MySQL)

Use the backup process show in Figure 7 for Oracle VM 3.3:

[root@mymanager ~]# cd /u01/app/oracle/ovm-manager-3/ovm_tools/bin[root@mymanager bin]# ./BackupDatabase -u admin -n before-deleting-mypool2

Enter your OVM Manager password:

INFO: Backup job starting with destination:

/ovmm-backups/before-deleting-mypool2-20140821_074327

Job Id = 'Start Backup to: before-deleting-mypool2(1408974206937) Uri: https://

localhost:7002/ovm/core/wsapi/rest/Job/1408974206937'

Job Name = 'Start Backup to: before-deleting-mypool2'

INFO: Backup job finished

[root@mymanager bin]#

Figure 7: Oracle VM 3.3 - creating a very quick ad-hoc backup of the Oracle VM management database (MySQL)

You can use this backup along with the saved files and directories from the Oracle VM servers taken inthe previous step to back out of this entire process.

Note

If you did not choose to use MySQL as the database engine during the install of Oracle VM Manager, then

you will need to contact your Oracle DBA and ask them to back up the database for you.


14/32


11

Step 2.2: Stop Oracle VM guests running in the pool

The screen shot in Figure 8, item 2 below shows that all Oracle VM guests running on the target server

pool must be stopped and migrated to the Unassigned Virtual Machines folder in the navigation pane.

2 3 4

1

Figure 8: Stop all Oracle VM guests running on the target server pool

Stopping the Oracle VM guests is accomplished in the Servers and VMs tab (see Figure 8, item 1).

Notice that Figure 8, items 3 & 4 show our fictional virtual machines stopped and are no longer

assigned to any servers (our example uses Oracle VM guests named myguest51 through myguest56 ).

Note

You do not need to stop any Oracle VM guests belonging to other server pools being managed by the same

Oracle VM Manager

Step 2.3: Release ownership of current repositories

You must now release the ownership of the storage repositories once all of the Oracle VM guests have

been stopped. It is very important that you release ownership of the storage repositories before you

complete the final refresh of data between the source and target storage platforms. The new server

pool will not be able to use the storage repositories if you do not release ownership before the final

refresh of data between the two storage platforms.

Stop!

Do not skip this step if SAN storage repositories are part of your migration project. Ownership must be

released BEFORE you begin the final refresh of repositories for later steps to succeed. You may not be able

to complete the migration to new storage if you skip this step.


15/32


12

The following three screen shots explain how to release ownership of the storage repositories. This

must be completed for all storage repositories that are part of the target server pool. Change to the

Repositories tab as shown in Figure 9 item 1 below, select the first storage repository as shown in

item2 and then choose the Edit Repository icon shown in item 3.

1

2

3

Figure 9: Showing ownership of storage repositories

Make sure you check the Release Ownership checkbox as shown in Figure 10 below and then choose

OK to complete.

Figure 10: Releasing ownership of storage repositories

Ensure the ownership has been released for all storage repositories before moving on to the next step.

If the operation was successful, then you will need to select the Show All Repositories radio button to


16/32


13

reveal the un-owned repositories (see Figure 11, item 1). Select each repository (Figure 11, item 2) and

ensure that Ownership: indicates Unowned (Figure 11, item 3).

2

1

3

Figure 11: Showing storage repository is no longer owned by the Oracle VM Manager

Step 3: Destroy Current Server Pool


Step 1:

Preparation

Step 2:

Release

ownership of

repos

Step 3:

Destroy current

pool

Step 4:

Delete source

disks

Step 5:

Discover target

disks

Step 6:

Rebuild server

pool

Step 7:

Take ownership

of repos

Step 8:

Start Oracle VM

guests

You will delete the server pool after releasing ownership of the storage repositories. This step is

tedious, but not as dramatic as it sounds. Nothing about Oracle VM is really impacted by this step

other than the server pool won’t exist for a few minutes while we build the new server pool. You will

not need to reinstall or reconfigure anything on the Oracle VM servers; all of the bonding, networking

and anything else about the Oracle VM servers remain unchanged.

In fact, once the server pool is destroyed you can still remount the old storage repositories and start

your VMs by hand without the Oracle VM servers even being part of a server pool if you ever needed

to recover. It is highly unlikely you would need to do such a thing and the only reason we mention this

is to let you know nothing is irreversible once the pool is gone – you still have all the parts and pieces

to rebuild the same server pool and your virtual machines are quite safe.

Note

Destroying the server pool is not as dramatic as it sounds. Everything is still there for you to use and it is

very easy to recover


17/32


14

Step 3.1: Remove Oracle VM servers from pool

Remove the servers rather than delete the servers. Deleting the servers does not cause a problem it just

adds the unnecessary step having to discover them again. Change to the Servers and VMs tab and

select the target server pool to remove the servers from the pool as shown in Figure 12 below.

2

1

Figure 12: Select the target server pool

Move all Oracle VM servers from the Selected Server(s) box on the right to the Available Server(s)

box on the left as shown in Figure 13 below.

Figure 13: Move all servers to the Available Server(s) box


18/32


15

All Oracle VM servers from the target server pool should now reside in the Unassigned Servers

folder in the navigation pane as shown in Figure 14 below.

Figure 14: All Oracle VM servers from target pool should be in Unassigned Servers folder

Step 3.2: Delete the empty server pool

Now delete the target server pool as shown in Figure 15 below. From the Servers and VMs tab,

select the target server pool (item 1) and then select the Delete Server Pool icon from the tool bar in

the navigation pane (item 2).

2

1

Figure 15: Delete the target server pool


19/32


16

As shown in Figure 16 below, all of your Oracle VM servers from the target pool should be contained

in the Unassigned Servers folder (item 1) and all remaining unaffected server pools should be the

only pools displayed in either the management pane (item 2) or navigation pane.

12

Figure 16: Only unaffected server pools should remain

Step 3.3: Perform final synchronization of storage You will need to perform a final refresh or synchronization of data between the source and target

storage platforms now that the Oracle VM guests have been stopped. All I/O activity should be fully

quiesced at this point so you should have complete data integrity when you perform the final refresh.

You should have already established a replication relationship between the source storage platform and

the target platform as part of step 1. The data you are replicating between the source and target storage

platforms should include the storage repositories and any other LUNs being presented as physical disks

to the Oracle VM guests.

Note

Oracle VM is not used in any way to accomplish cloning or replication of storage repositories. Therefore

storage replication is beyond the scope or purpose of this document. We assume you know how to

accomplish the cloning or replication of the storage repositories and physical disks that are part of your

migration project.

Step 4: Delete Source Disks


Step 1:

Preparation

Step 2:

Release

ownership of

repos

Step 3:

Destroy current

pool

Step 4:

Delete source

disks

Step 5:

Discover target

disks

Step 6:

Rebuild server

pool

Step 7:

Take ownership

of repos

Step 8:

Start Oracle VM

guests

Now we will delete the source disks (LUNs) from the Oracle VM management database (model).

Deleting physical disks in this case means deleting the database record for each storage object, not

actually deleting the LUNs that reside on the underlying storage platform. This is a non-destructive

process which means all the data on the LUNs will remain in place and unaffected.


20/32


17

Deleting the physical disks is an important step since it will help reduce confusion between which disks

are the old source disks and which are the new target disks. You can always rediscover the deleted

disks if you need them again to back out of this solution.

Step 4.1: Un-map source LUNs

You will have your storage admin un-map or un-present the old source LUNs from the Oracle VM

servers so they can no longer access or see them from the source storage platform/array. Oracle VM

is not used or involved at all for this step; it is all accomplished on the storage array.

You will not be able to properly delete the source physical disks from the model until the Oracle VM

servers can no longer access the old source LUNs.

Step 4.2: Delete the physical disks from DB

As shown in Figure 17 below, change to the Storage tab (item 1) and then select whatever SAN server

you are using for your source SAN LUNs (item 2). The screen shot shows an iSCSI SAN server

selected; you should select Unmanaged FibreChannel Storage Array if you are using Fibre Channel

instead.

Ensure you are looking at the physical disks by selecting that from the Perspective (item 3). Notice

that the old source disks show an Event Severity of Warning (item 4). This means the old source

disks are no longer accessible by the Oracle VM servers – this is a desired condition since it means we

are able to delete the physical disk records from the database.

The reason that Oracle VM does not automatically remove the disks from the database is simply

because it has no idea if this is a temporary condition due to the temporary loss of access, a human

error or if the disks are no longer being used. So, the record of each disk will remain in the database

until a human confirms that these particular physical disks are no longer being used and it is okay to

remove the disks.


21/32


18

To delete the disk, simply select all of the disks (item 4) and then choose the Delete Disks icon from

the tool bar above the management pane (item 5).

1

2

34

5

Figure 17: Cleaning up the Oracle VM database to reduce confusion

You should no longer see any of the source physical disks once you have deleted them from the model

as seen in Figure 18 below.

Our screen shots assume these are the only physical disks being served from the legacy storage

platform, so you may still see other disks on the legacy storage platform if you are using them for other

server pools that are not part of your migration project.

Figure 18: All of the source physical disks have been deleted

Step 5: Discover Target Disks


Step 1:

Preparation

Step 2:

Release

ownership of

repos

Step 3:

Destroy current

pool

Step 4:

Delete source

disks

Step 5:

Discover target

disks

Step 6:

Rebuild server

pool

Step 7:

Take ownership

of repos

Step 8:

Start Oracle VM

guests


22/32


19

We deleted the source physical disks from the model simply to reduce confusion over which disks are

which in later steps. We can now discover replicated physical disks from the new target storage

platform.

Step 5.1: Map the target LUNs

You will have your storage admin map or present the replicated target LUNs to the Oracle VM servers

so they can access or see them from the new target storage platform/array. Oracle VM is not used or

involved at all in this step; it is all accomplished on the storage array.

You will not be able to discover the physical until the Oracle VM servers can access the new target

LUNs.

Step 5.2: Discover the new storage platform

Skip this step if:

You are using Fibre Channel You are using iSCSI but the new SAN server already exists in your model

This paper assumes target SAN server containing the replicated LUNs has already been discovered

prior to this project. If you have not discovered the new iSCSI SAN server yet, then please run

through the SAN server discovery process. This process is documented in the latest Oracle VM user

guide for your particular version of Oracle VM.

Oracle VM is not really “discovering” the storage array, it is simply probing the SCSI bus o n the Oracle

VM servers designated as admin servers for the SAN server and then cataloguing the LUNs it finds

into the Oracle VM management database. The discovery process really just adds LUNs to the Oracle

VM management database for the first time.

Step 5.3: Refresh the Oracle VM database

Skip this step if you performed the previous step of discovering a new iSCSI SAN server.

You will need to refresh the target SAN server containing the replicated LUNs before you attempt to

use them. You should perform this process even if you can see the new replicated LUNs in the Oracle

VM Manager. This will ensure that no new LUNs are overlooked and that the Oracle VM

management database contains all of the latest replicated LUNs.

Step 5.3.1: If you are using Fibre Channel

If you are using Fibre Channel to access your LUNs, then you should simply refresh the Unmanaged

FibreChannel Storage Array to have the Oracle VM servers reprobe the SCSI bus for new LUNs.

This process is documented in the latest Oracle VM user guide for your particular version of Oracle

VM.

http://www.oracle.com/technetwork/documentation/vm-096300.htmlhttp://www.oracle.com/technetwork/documentation/vm-096300.htmlhttp://www.oracle.com/technetwork/documentation/vm-096300.htmlhttp://www.oracle.com/technetwork/documentation/vm-096300.htmlhttp://www.oracle.com/technetwork/documentation/vm-096300.htmlhttp://www.oracle.com/technetwork/documentation/vm-096300.htmlhttp://www.oracle.com/technetwork/documentation/vm-096300.htmlhttp://www.oracle.com/technetwork/documentation/vm-096300.htmlhttp://www.oracle.com/technetwork/documentation/vm-096300.htmlhttp://www.oracle.com/technetwork/documentation/vm-096300.html


23/32


20

Ensure you are viewing the Storage tab as shown in Figure 19 below (item 1). Select the Unmanaged

FibreChannel Storage Array in the navigation pane (item 2), then choose the Refresh icon from the

tool bar above the navigation pane (item 3).

13

2

Figure 19: Refreshing Fibre Channel

You should now see all the replicated physical disks being served from the new storage platform as

shown in Figure 20 below. The LUNs have been catalogued as physical disks in the Oracle VM

management database at this point.

Figure 20: New target disks have now been added to the Oracle VM management database

Note

All Oracle VM servers accessing Fibre Channel disks must be designated as admin servers for the

Unmanaged FibreChannel Storage rray in order to reprobe the SCSI bus on the Oracle VM servers without

a reboot.

Step 5.3.2: If you are using iSCSI

Ensure you are viewing the Storage tab as shown in Figure 21 below (item 1). Select the iSCSI SAN

server in the navigation pane (item 2), then choose the Refresh icon from the tool bar above the

navigation pane (item 3).


24/32


21

13

2

Figure 21: Refreshing iSCSI

You should now see all the replicated physical disks being served from the new storage platform as

shown in Figure 22 below. The LUNs have been catalogued as physical disks in the Oracle VM

management database at this point.

Figure 22: New target disks have now been added to the Oracle VM management database

Step 6: Rebuild the Server Pool


Step 1:

Preparation

Step 2:

Release

ownership of

repos

Step 3:

Destroy current

pool

Step 4:

Delete source

disks

Step 5:

Discover target

disks

Step 6:

Rebuild server

pool

Step 7:

Take ownership

of repos

Step 8:

Start Oracle VM

guests

We can now rebuild the same server pool using the new PoolFS being presented from the target

storage platform.


25/32


22

Go to the Servers and VMs tab as shown in Figure 23 below (item 1) and then choose the Create

Server Pool icon from the tool bar above the navigation pane (item 2).

1

2

Figure 23: Create the same server pool

Looking at Figure 24 below, give the new server pool the same name as the old server pool youdestroyed earlier (item 1), enter the same VIP from the destroyed server pool (item 2). Assign a

PoolFS by selecting the Physical Disk radio button (item 3) for Storage Location and then choose

the Search icon to select the new PoolFS (item 4).

1

2

3 4

Figure 24: Enter the server pool details


26/32


23

Make sure you chose the correct SAN server from the Select Physical Disk dialog shown in Figure

25 below. Here we show our fictional iSCSI server; you should choose whatever is appropriate for

your environment.

Figure 25: Select the correct SAN server

You should now see the Storage Location field populated with your choice for PoolFS as shown in

Figure 26 below.

Figure 26: The new pool file system is selected


27/32


24

Finally, add all the Oracle VM servers you removed from the old server pool as shown in Figure 27

below.

Figure 27: Add all of Oracle VM servers

Step 7: Take Ownership of Target Repositories


Step 1:

Preparation

Step 2:

Release

ownership of

repos

Step 3:

Destroy current

pool

Step 4:

Delete source

disks

Step 5:

Discover target

disks

Step 6:

Rebuild server

pool

Step 7:

Take ownership

of repos

Step 8:

Start Oracle VM

guests

You have recreated the same server pool at this point using the PoolFS residing on the new storage

platform. There are no additional steps needed at this point other than incorporating the new

replicated storage repositories into the server pool and starting your Oracle VM guests again.

This should be a very easy step as long as you followed the instructions for releasing ownership of the

source repositories before the final replication refresh/synchronization.

Step 7.1: Refresh the target disks

We need to refresh the individual disks representing the replicated storage repositories before we can

use them as repositories. This step is slightly different from when we refreshed the SAN server; the

SAN server refresh only updates the Oracle VM management database with physical disks that are new

or missing. Refreshing the individual disks forces the Oracle VM Manager to examine the physical

disks for file systems. If the Oracle VM Manager discovers a know file system type on the physical

disk, it adds additional attributes to the database record for the physical disk indicating if the file system

contains a PoolFS or storage repository.


28/32


25

We only need to select the physical disk we know to be storage repositories. Ensure you change to the

Storage tab (item 1) as shown in Figure 28 below. Select the appropriate iSCSI or Fibre SAN server

for your environment (item 2), select all the disks you know to contain replicated repositories (item 3)

and then choose the Refresh Disks icon from the management pane tool bar (item 4).

1

4

2

3

Figure 28: Refresh the replicated physical disks to update the Oracle VM management database

Step 7.2: Refresh the target repositories

The Oracle VM Manager will now recognize the physical disks containing storage repositories. Go to

the Repositories tab (item 1) as shown in Figure 29 below. The repositories are Unowned at this

point so you won’t be able see them until you change the context of view. This is expected since you

released the ownership of the old source repositories before the final replication

refresh/synchronization before destroying the old server pool which leaves them un-owned at this

point.

Using Figure 29 below as a guide you will need to change the context by selecting the Show AllRepositories radio button (item 2). Select the top level folder for Repositories in the navigation pane

(item 3) and notice that the repositories show up with their old names and that they are Unowned

(item 4).

1

3

4

2

Figure 29: View the newly discovered storage repositories


29/32


26

Select each replicated repository in the navigation pane and then choose the Edit Repository icon

(item 1) as shown in Figure 30 below. Use the first sub tab of the wizard to assign each repository to

the correct Server Pool (item 2) and then Take Ownership (item 3).

2

3

1

Figure 30: Take ownership of the repositories

Use the Present Repositories sub tab in the Edit Repository wizard as shown in Figure 31 below to

present each repository to the newly reconstituted server pool.

Figure 31: Present each repository to the newly rebuilt server pool

Notice that the replicated repositories are now Owned by You (item 1) and presented to all Oracle

VM servers in Figure 32 below. We can now rediscover the Oracle VM guests residing in the


30/32


27

replicated repositories by selecting the two repositories in the management pane (item 1) and then

choosing Refresh Repositories icon from the tool bar above the management pane (item 2).

1

2

Figure 32: All repositories are now owned by the Oracle VM Manager again

Step 8: Start Oracle VM Guests


Step 1:Preparation

Step 2:Release

ownership of

repos

Step 3:Destroy current

pool

Step 4:Delete source

disks

Step 5:Discover target

disks

Step 6:Rebuild server

pool

Step 7:Take ownership

of repos

Step 8:Start Oracle VM

guests

The final step is simply to assign the Oracle VM guests to servers and start them.

To start the Oracle VM guests, you will need to select the Unassigned Virtual Machines folder in

the navigation pane as shown in Figure 33 below (item 1). Select one or more Oracle VM guests in the

management pane (item 2) and then choose the Migrate VMs icon from the tool bar above the

management pane (item 3). After that you can start

2

3

1

Figure 33: Migrate Oracle VM guests to Oracle VM servers


31/32


28

You can drag and drop multiple Oracle VM guests to Oracle VM servers and start them in mass, or

you can migrate and start them one at a time. The point is that you should now be able to run the

same Oracle VM guests from the new replicated repositories.

Figure 34: The final step is to start the Oracle VM guests


32/32

Oracle VM 3:

Change the Server Pool File System &

Repositories (SAN)

August 2014

Document: SN05220 rev. 5

Author: Gregory King

Oracle Corporation

World Headquarters

500 Oracle Parkway

Redwood Shores, CA 94065

U.S.A.

Worldwide Inquiries:

Phone: +1.650.506.7000

Fax: +1.650.506.7200

l

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only and the

contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other

warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or

fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are

formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by a ny

means, electronic or mechanical, for any purpose, without our prior written permission.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and

are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are

trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0612

oracle vm 3 change the server pool file system & repositories

Documents