doubletake failover-failback instructions

Cyber Country Knowledge Base Article

DOUBLETAKE FAILOVER-FAILBACK INSTRUCTIONS Summary: Follow the below instructions for failovers and failbacks of yMaxxNet database servers. Article Terminology: This article uses generic terminology that should be used to specifically relate to your specific installation settings. Tfollowing is an explanation of this terminology and what it gerepresents.

• PRIMARY or primary refers to your primary database serve• BACKUP or backup refers to your backup database server. • YourReplicationSet refers to the replication set used to

replicate data between your primary and backup database Table of Contents: The following table of contents that can be used to perform sptasks inside of failing over and failing back servers using DoIf you are not completely familiar with DoubleTake, please stae first section First Steps. th

First Steps – Page 2 thru 4 (staMirror and Replicate Primary to Backup – Pages 5 thru 8 Failover - Pages 9 thru 13 Failback - Pages 14 thru 20 Begin Replication/Monitoring - Page 21 Appendix A – Stopping DB2 Services - Page 22 thru 23 Appendix B – Starting DB2 Services - Page 24 Appendix C – Starting a Monitor - Page 25-31 Appendix D – Creating a Replication Set - Page 32 thru 33

Pa

our

he nerally

r.

servers.

ecific ubleTake. rt with

rt here)

ge 1 of 34


First Steps: 1. Always make sure you have a recent MaxxNet backup before

performing failovers and failbacks. If mistakes are made during this process, as well as the occurrence of unforeseen circumstances, MaxxNet databases become corrupted. If this happens, a recent MaxxNet database backup is necessary to bring your system back up to date. To perform a manual MaxxNet backup, run the Management Workstation. Select Database > Backup/Restore > Backup.

2. Next move to the PRIMARY server. 3. Is the Double-Take Management Console running? To determine

this, follow these steps. a. Is the Double-Take Management Console maximized? If so,

you will see ‘Double-Take Management Console’ in the dialog’s title bar (see below). If you do see this, move to step 4 of this section. If you do not see this, move to the next step.

b. Is the Double-Take Management Console minimized? If so, you will see the Double-Take Management Console minimized on the task bar (see below). If you do see this, first click on it to maximize it, then move to step 4 of this section. If you do not see this, move to the next step.

c. You have determined that the Double-Take Management Console is not running. Move directly to the next section titled Mirror and Replicate Primary to Backup.

of 34


4. The Double-Take Management Console is running and is maximized. Now, lets determine if it has been replicating its data to the backup server. Follow these steps. a. Left click on primary (see below).

b. Do you see YourReplicationSet in the right window (see

below)? If you do, move to the next step. If you do not, move to the next section labeled Mirror and Replicate Primary to Backup.

c. Does YourReplicationSet have a green check next to it (see below)? If it does, move to the next step. If it does not, first right click on YourReplicationSet and select disconnect. Then, move to the next section labeled Mirror and Replicate Primary to Backup.

d. Is YourReplicationSet’s Mirror Status Idle (see below)? If it is, move to the next step. If it is not, first right click on YourReplicationSet and select disconnect. Then, move to the next section labeled Mirror and Replicate Primary to Backup.

of 34


e. Is YourReplicationSet’s Replication Status Ready (see below)? If it is, your backup server should be up to date with the most recent replication set’s data. Move to the section labeled Failover. If the Replication Status is not Ready, first right click on YourReplicationSet and select disconnect. Then, move to the next section labeled Mirror and Replicate Primary to Backup.

of 34


Mirror and Replicate Primary to Backup: These instructions are required to be followed only if the primary has not been replicating its data to the backup. We must insure that the backup has all the up-to-date information before we perform a fail-over.

1. If you are not already on the PRIMARY server, move to the PRIMARY server.

2. If the Double-Take Management Console is not running do one of the following below instructions to start the application. a. Double-click on the desktop icon labeled DTMgmtCon or b. Start > Double-Take > Management Console.

3. Logon to the primary Double-Take Server by a. Right-Clicking on primary and selecting logon (see below).

Use the Windows user id and password you use to log onto the server. If you know you are already logged on, you can skip this step.

4. Next, left click on primary (see below).

of 34


5. Next, open the primary tree (see below).

5. Do you see YourReplicationSet under primary (see below)? If yes, move on to the next step. If no, move to Appendix D – Creating a Replication Set.

6. Next, move to your BACKUP computer. 7. On your BACKUP computer, follow the instructions in Appendix A –

Stopping DB2 Services. After you have finished continue with the following instruction.

8. On your PRIMARY server, right click on the replication set and select ‘Connection Manager’

of 34


9. Using the screenshot and instructions below, fill the proper settings in.

a. Source Server = primary b. Replication set = YourReplicationSet. The replication set

you right-clicked on should already fill in this box. c. Target = backup d. Mappings = ONE TO ONE

i. Once selected, make sure Source and Target have identical paths. It is EXTREMELY IMPORTANT to have a ONE TO ONE mapping in order for MaxxNet and DB2 to work properly when a fail-over occurs.

e. Check Start Mirror on connection f. Check Start Replication on connection. g. Press the ‘Connect’ button.

10. The mirroring/replication will begin. You will see, first the mirror status being calculated

11. Then the mirror status mirroring, as the percentage grows in size.

12. If you have a failure during the mirror (the green check will turn to a red x), the PRIMARY is replicating to files that are currently in use. Make sure that all of BACKUP’s DB2 services are shut down, and that all MaxxNet applications on the BACKUP are not running.

of 34


13. Once the mirror finishes the mirror status will go to Idle (see below). Wait until the mirror status turns to Idle. This concludes the full mirror. A Replication Status of Ready means that the replication set is currently replicating.

of 34


Failover: 1. On the BACKUP, if the Failover Control Center (FCC) is not

already running, start the Failover Control Center (FCC). Follow the next instructions to determine if the FCC is running.

2. If the FCC is maximized, you will see a dialog with ‘Failover Control Center’ in the dialog title bar (see below). If the FCC is maximized, move to step 5.

3. If the FCC is minimized, you will see the FCC icon on the lower-right portion of the taskbar (see below). If you do see this, double-click on the icon to maximize it and move to step 5. If you do not see this, go to step 4.

4. Start the FCC by doing one of the following a. Double-clicking on the FCC shortcut or b. Start > Programs > Double-Take > Failover Control Center.

of 34


5. The Target Machine must be the BACKUP server. If it is not, select it in the list.

6. If the Login button is not grayed out, you must Login by pressing the Login button. The below screenshot shows the Login button as active and all other buttons grayed out. This means that you are not logged onto your backup machine.

of 34


7. Follow the below instructions to find out if the PRIMARY server

is being monitored.

a. Look in the list box under ‘Monitored Machines/Names’. i. Is PRIMARY in the list (see below). If it is, move

to step 8. If it is not, follow the instructions in Appendix C – Starting a Monitor.

8. We have determined that the FCC is running and monitoring the PRIMARY server. Next, it is time to perform the failover.

9. Disconnect all client applications that are database dependent (AWS, MWS, BOB, etc.) The ACC’s can be left alone. Computers that run MaxxNet applications and are configured in the script files will be notified when a failover occurs. They will be told to restart their MaxxNet applications. If the failover is pre-planned, it is best to let your operators know what is happening before it occurs as not to alarm them.

10. Unplug the PRIMARY server’s network connection.

of 34


11. The BACKUP’s Failover Control Center’s icons will turn from green to yellow (see below). This means that the PRIMARY server is no where to be found on the network.

12. Once the ‘Time to Fail’ reaches 0 seconds, the icons will turn from yellow to red. At this time, the backup is assuming the primary’s IP address. The status at the bottom lets the user know that the failover is currently working.

of 34


13. Once the status turns to Monitoring (see below), your BACKUP has finished assuming the identity of your PRIMARY server. Wait for the ‘Monitoring’ to show up (below) before moving to the next step.

14. Next, on your BACKUP server, start all your DB2 services by following the instructions in Appendix B – Starting DB2 Services.

15. You can resume all MaxxNet client applications that are needed. You applications are now running off the backup server.

IMPORTANT NOTE: Since the BACKUP server has assumed the PRIMARY server’s identity, it is EXREMELY important not to plug the PRIMARY’s network connection back in until it is said to be OK in the below instructions. If you have work to do on the PRIMARY server, now is the time to do it. When you are ready to have the PRIMARY server take its identity back, follow the instructions in the section labeled Failback.

of 34


Failback: When you are ready to failback to the PRIMARY server follow the below instructions.

1. Using the PRIMARY server, follow the instructions in Appendix A – Stopping DB2 Services. Once you have finished these instructions, continue with the below steps.

2. On the BACKUP server, go to the Failover Control Center (FCC). 3. Disconnect all client applications that are database dependent

(AWS, MWS, BOB, etc.) The ACC’s can be left alone. Computers that run MaxxNet applications and are configured in the script files will be notified when a failover occurs to restart their MaxxNet applications. Since the failback’s are pre-planned, it is best to let your operators know what is happening before it occurs as not to alarm them.

4. Press the ‘Failback’ button.

5. Multiple message boxes will appear informing users on the network that the system is failing back. These message boxes all start off saying ‘Message from BACKUP to BACKUP. You may click OK through all message boxes. DO NOT CLICK CONTINUE on the following message box yet.

of 34


6. Next, PLUG THE PRIMARY network connection back into the primary server.

7. Now you may press the ‘Continue’ button for the above message box.

8. As long as the PRIMARY server was plugged in correctly and can be seen on the network, the FCC on the BACKUP will start to monitor the primary again.

9. Next, on your BACKUP computer, follow the steps in Appendix A –

Stopping DB2 Services. Once you have finished stopping DB2 on the BACKUP, shut down the Services and Administrative Tools windows and continue with the below steps to restore your database.

of 34


10. On the PRIMARY server, if it is not already running and maximized, start up and maximize the Doubletake Management Console (DTMgmtCon).

11. Right click on primary and select Logon. Use your windows user id and password to log on. If you know you are already logged on, you can skip this step.

of 34


14. Next, left click on primary (see below).

12. Next, if the replication set appears in the right window, right click on it and select Disconnect. You may notice that the Target Status shows: Restore Required. The restoration from backup to primary will be achieved in the following steps.

13. If you disconnected YourReplicationSet above, you may see the following message box. If you do, select Yes. Otherwise, move to the next step.

of 34


14. If you disconnected YourReplicationSet you may see the following dialog may appear. If you do, Select Yes. Otherwise, move to the next step.

15. Next, go to Tools > Restoration Manager.

16. You may be asked if you wish your backup server to stop monitoring your primary server. If you are asked this, select Yes. Otherwise, move to the next step.

of 34


17. Once inside the Restoration Manager, select (see below graphic) a. Original Source = primary b. Restore from = backup

18. The replication set should appear under replication set and Restore To should be set to primary.

19. Next, make sure ‘Use Backup Replication Set’ is unchecked. 20. Next, make sure that the ‘Restore To’ and ‘Restore From’ paths

are identical. They should be, as long as the replication set was configured properly before.

21. Press the ‘Restore’ button.

of 34


22. The restoration should begin.

23. Once the <YourReplicationSet> disappears (in the right window above), the restoration is complete.

24. Once the restoration is complete, follow the instructions in Appendix B – Starting DB2 Services for the PRIMARY server.

25. You can resume all MaxxNet client applications that are needed. You applications are now running off the primary server.

NOTE: If you have anything to do to the BACKUP server, now is the time to do so. If the BACKUP machine is rebooted, follow the below instructions to begin replication and monitoring.

of 34


Begin Replication/Monitoring To begin replication/monitoring again, follow the below instructions. 1. On the BACKUP server, follow the instructions in Appendix A –

Stopping DB2 Services. Once you have finished, continue with the below instructions.

2. Perform the steps in the section labeled Mirror and Replicate Primary to Backup (above-pg 5) in order to begin replication from PRIMARY to BACKUP again.

3. If you stopped the FCC Monitor on the BACKUP, perform the steps in Appendix C – Starting a Monitor on the BACKUP server in order to begin monitoring again.

of 34


Appendix A – Stopping DB2 Services

1. On the desktop locate the shortcut labeled DBTake. 2. Open the application. 3. Locate the button labeled ‘DB2 Stop’ and press it.

4. Once the application has finished, close the application by

pressing the ‘Close’ button. 5. You will see a dialog that states ‘The program has been closed by

the operator’. Select ok. 6. Next, go to Start > Settings > Control Panel > Administrative

Tools > Services.

of 34


7. STOP all of the DB2 services that may be already running by right clicking on the services and selecting stop. In the below screenshot, the services circled in red are your DB2 services. If the ‘Status’ of one of these services is blank, this means the service is already stopped. NOTE: If DB2-DB2 is still running, try disconnecting any local database applications and performing steps 1 thru 5 (on the previous page) again. If you still have difficulties, reboot your machine.

8. Once all DB2 statuses are blank, you have stopped all your DB2 services. Return to your original section.

of 34


Appendix B – Starting DB2 Services 1. Go to Start > Settings > Control Panel > Administrative Tools >

Services. 2. START all of the DB2 services that are labeled Startup Type =

Automatic by right clicking on each service and selecting Start. The below screenshot has all of the DB2 services circled in the large red circle. The smaller red circle shows which services should be started once you complete this step.

3. Once all of the above circled DB2 services are Started, your DB2

services have been started. Return to your original section.

of 34


Appendix C – Starting a Monitor If you are in this section, you must already have the FCC open and be logged into your BACKUP target machine.

1. Perform these steps on you BACKUP server. 2. Since you are not currently monitoring the PRIMARY server, you

will see nothing inside the ‘Monitored Machines/Names’ list box (see below).

of 34


3. Next, to add a monitor press the ‘Add Monitor’ button (see below).

4. Type in the PRIMARY machine name to monitor (see below). You can also browse for the Machine Name by pressing the ‘Browse’ button.

of 34


5. Next check the check boxes under Names to Monitor so you are monitoring PRIMARY and its IP address (see below).

6. Next, press the ‘Scripts’ button.

of 34


7. Check the box titled ‘Delay failover until script completes’ (see below).

8. Using the … (see below) button on the right next to Post-Failover and browse for X:\Program Files\MaxxNet\DoubleTake\fail.exe (where X is the drive where MaxxNet doubletake scripts were installed).

of 34


9. Using the … (see below) button on the right next to Post-Failover

and browse for X:\Program Files\MaxxNet\DoubleTake\fail.exe (where X is the drive where MaxxNet doubletake scripts were installed).

10. Lastly, check the ‘Make these scripts defaults for future monitor sessions’ checkbox (see below).

of 34


11. Press OK (see below).

12. Press OK (see below) to exit the ‘Monitor Settings – BACKUP’ dialog.

of 34


13. Open PRIMARY’s tree in the ‘Monitored Machines/Names’ list

box. You should see the monitoring happening with the Time to Fail seconds counting down and resetting.

14. You have completed starting a monitor. Return to your original section.

of 34


Appendix D – Creating a Replication Set To create a replication set, you must have the Double-Take Management Console open on your PRIMARY server. You must also be logged into your primary server (logged into Double-Take > Primary).

1. Right-click on Right click on primary and select New > Replication Set (see below).

2. Change the name of the replication set from ‘New Replication Set’ to your desired replication set name (see below).

of 34


3. Next, open the replication set’s tree and check the following directories (see the below screenshot and next step for further explanation).

4. The following directories should be checked (NOTE: In the below letters, X stands for the directory where you installed DB2, where you database(s) reside, and where MaxxNet was installed): a. X:\Program Files\IBM (if DB2 V8 and above) b. X:\Program Files\SQLLIB (if DB2 V7 and below) c. X:\DB2 (this directory can exist on multiple drives) d. X:\Program Files\MaxxNet

i. If you are running ACCs on your servers, under the MaxxNet directory uncheck the ACC folder.

5. Notice that the check with a white background in the box means that all sub-directories under this directory are checked. Checks with gray backgrounds represent a folder that has at least one but not all of its subdirectories checked.

6. Once you have checked the proper folders, return to your original section.

of 34


Revision History:

• 4/22/2004 – Initial Revision The information in this article applies to:

• Doubletake and MaxxNet

of 34

doubletake failover-failback instructions

Documents