using the ibm tivoli storage manager backup-archive · pdf filebasic concepts of clustering...

28
Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux Skill Level: Intermediate Jason Basler ([email protected]) Senior software engineer IBM Neeta Garimella ([email protected]) Staff Software Engineer IBM Kevin Hoyt ([email protected]) Staff software engineer IBM Giang T Pham ([email protected]) Staff software engineer IBM Jim Smith ([email protected]) Backup-Archive client architect IBM 18 Jul 2007 Section 1. About this tutorial Introduction Software products that provide a high availability of resources are becoming Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux © Copyright IBM Corporation 1994, 2008. All rights reserved. Page 1 of 28

Upload: vunga

Post on 31-Jan-2018

244 views

Category:

Documents


5 download

TRANSCRIPT

Page 1: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

Using the IBM Tivoli Storage ManagerBackup-Archive client in cluster environments onUNIX and LinuxSkill Level: Intermediate

Jason Basler ([email protected])Senior software engineerIBM

Neeta Garimella ([email protected])Staff Software EngineerIBM

Kevin Hoyt ([email protected])Staff software engineerIBM

Giang T Pham ([email protected])Staff software engineerIBM

Jim Smith ([email protected])Backup-Archive client architectIBM

18 Jul 2007

Section 1. About this tutorial

Introduction

Software products that provide a high availability of resources are becoming

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 1 of 28

Page 2: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

commonplace in today's computing environments. As the complexity of managinglogical disks and their data increases in cluster environments, understanding anddeploying an effective backup strategy for complete data protection is a top priority.

Our goal is to introduce a methodology for backing up and restoring UNIX and Linuxdata that resides in a cluster environment. Basic concepts of clustering and howTivoli Storage Manager can be deployed in the cluster environment are introduced,and practical examples are given on how to configure the Tivoli Storage ManagerBackup-Archive client and backup schedules. The migration procedure for TivoliStorage Manager Backup-Archive client configurations that currently useclusternode option in AIX, HACMP environments is also described. The proceduresoutlined here depend on features introduced in Tivoli Storage Manager version5.4.0.

Prerequisites

The term cluster can mean many things to many different vendors and users. It canmean "highly available," "high performance," "load balancing," "grid computing," orsome combination thereof. There are currently several clustering products availablefor UNIX and Linux. Our intent is not to provide a backup solution for any specificclustering product, but to define those aspects of a clustering environment that needto exist in order for this backup methodology to work correctly.

The concept of cluster, in this paper, refers to a UNIX or a Linux environment whichexhibits the following characteristics:

• Disks are shared between physical machines, either in an exclusivefashion (only one host has access to the logical disk at any one time) or ina concurrent fashion.

• Disks appear as local disks to the host and not as network resources. Wesuggest that the file systems be mounted locally to the system, notthrough a LAN-based file share protocol such as network file system(NFS).

• Mount points of local disks are identical on each physical host in theenvironment (if file system /group1_disk1 fails from NodeA to NodeB, it ismounted on NodeB as /group1_disk1).

If the cluster environment exhibits all of the above characteristics, the methodologieswe outline are supported by version 5.4 and higher of Tivoli Storage ManagerBackup-Archive client. If the cluster environment only exhibits a subset (or none) ofthe characteristics, the methodologies outlined are not applicable and are notsupported by Tivoli Storage Manager.

It is assumed that the reader has a basic understanding of how the Tivoli Storage

developerWorks® ibm.com/developerWorks

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and LinuxPage 2 of 28 © Copyright IBM Corporation 1994, 2008. All rights reserved.

Page 3: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

Manager product functions, and that they have a basic understanding of how theircluster software functions. This paper does not address any cluster software relatedactivities such as the development of application start and stop scripts.

Section 2. Overview of cluster environments

Cluster environments can be set up in many different configurations. This sectiondescribes the most popular cluster configurations.

Active/active: Pool cluster resources

In an active/active configuration, each node is actively managing at least oneresource and is configured as a backup for one or more resources in the cluster.Active/active is the most common form of a cluster environment.

Figure 1. Active/active cluster environment: Normal operation

ibm.com/developerWorks developerWorks®

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 3 of 28

Page 4: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

The configuration in Figure 1 is operating normally; each node is operational andmanaging only one resource.

Figure 2. Active/active cluster environment: NodeB fails over to NodeA

In the configuration shown in Figure 2, Node B has experienced a fault. Resource 2has failed over to Node A.

Active/passive: Fault tolerant

In an active/passive configuration, one node actively manages the resource. Theother node is only used if the primary node experiences a fault and the resourceneeds to failover. An active/passive cluster is a sub-type of an active/active cluster.For the examples described in this paper, the setup requirements are similar.

Figure 3. Active/passive cluster environment: Normal operation

developerWorks® ibm.com/developerWorks

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and LinuxPage 4 of 28 © Copyright IBM Corporation 1994, 2008. All rights reserved.

Page 5: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

The configuration in Figure 3 is operating normally. NodeA is managing Resource 1.If NodeA experiences a fault, Resource 1 will failover to NodeB.

Figure 4. Active/passive cluster environment: NodeA failover to NodeB

In the configuration shown in Figure 4, NodeA has experienced a fault. Resource 1has failed over to NodeB.

Concurrent access

In a concurrent configuration, more than one node manages a resource. When afault occurs, the resource continues to be managed by the other nodes. This type ofcluster is not common and is beyond the scope of this paper.

Figure 5. Concurrent access cluster environment: Normal operation

ibm.com/developerWorks developerWorks®

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 5 of 28

Page 6: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

The configuration shown in Figure 5 is operating normally. Every node is managingevery resource. If any node experiences a fault, the other nodes will continue tomanage the resources.

Figure 6. Concurrent access cluster environment: NodeB failover to NodeA

developerWorks® ibm.com/developerWorks

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and LinuxPage 6 of 28 © Copyright IBM Corporation 1994, 2008. All rights reserved.

Page 7: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

In the configuration shown in Figure 6, NodeB has experienced a fault. NodeA andNodeB continue to manage Resources 1, 2, and 3.

The following sections describe how to configure Tivoli Storage Manager in anactive/active or an active/passive cluster environment. Tivoli Storage Manager canbe used in a concurrent access cluster, but that configuration is not described in theupcoming sections.

Section 3. Tivoli Storage Manager backup conceptsrelevant to cluster environments

How does Tivoli Storage Manager fit into the concept of acluster environment?

One of the biggest challenges when providing backup and recovery in a cluster

ibm.com/developerWorks developerWorks®

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 7 of 28

Page 8: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

environment is determining where backup operations should be performed in thecontext of the cluster's resources. Clusters are designed to offer high-availability toapplications and users. A user of a mail or database application does not need toknow which host physically owns a disk resource, but instead relies upon avirtualized connection to the cluster's resources. For example, any movement ofresources within a cluster between hosts due to failover or load balancing is hiddenfrom the end-user. A backup product can be placed within or external to the cluster'sresource groups.

If the backup product is placed outside of the context of the cluster and its resourcegroups (for example, on a host that is not part of the cluster), it can map or mountthe file systems and perform the backups relative to the mapped or mounted drives.The advantage to this model has already been demonstrated: As the disk resourcesmove between the physical cluster hosts, the backup product retains the mapping ofthe file system. The big disadvantage of this method is that backup of data overnetwork protocols such as Network File System or Common Internet File System isslower than backup of local file systems. This becomes a factor as data volumesgrow and backup windows shrink. Another disadvantage is that you have noprotection if the host you have chosen for backups fails!

The Tivoli Storage Manager Backup-Archive client is designed to manage thebackup of cluster drives by placing the backup-archive client within the context of thecluster's resource groups. This gives the advantage of backing up data from localresources (as opposed to accessing the data across the network) to maximize theperformance of the backup operation and to manage the backup data relative to theresource group. Therefore, the backup-archive client can always back up data oncluster resources, as if the data were local data, and maximize backup performance.This ensures that critical data is getting backed up across system failures.

Cluster aware vs. highly available

The Tivoli Storage Manager Backup-Archive Client is not cluster aware, in that itdoes not use any API to determine the context under which it is running or provideany explicit mechanisms for failover or high-availability. The Tivoli Storage ManagerBackup-Archive client does offer fault-tolerant scheduling capabilities which can beexploited to provide protection in cluster environments; this makes the Tivoli StorageManager backup scheduling service highly available. For example, a schedule canbe defined that performs an incremental backup of the /group1_disk1 file system. Ifthe /group1_disk1 file system fails to another host in the cluster while the backup isbeing performed, the scheduled incremental operation is restarted from the otherhost, if the configuration steps outlined below are followed correctly. The progressiveincremental backup methodology employed by Tivoli Storage Manager determines iffiles are already backed-up, and effectively continues processing the incrementalbackup from the point where the failover occurred.

developerWorks® ibm.com/developerWorks

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and LinuxPage 8 of 28 © Copyright IBM Corporation 1994, 2008. All rights reserved.

Page 9: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

Section 4. Configuring the Tivoli Storage ManagerBackup-Archive client in a cluster environment

Figure 7. Active/Active cluster environment: Normal operation

The sample configuration in Figure 7 shows an active/active cluster environment thathas three physical hosts in the cluster named NodeA, NodeB, and NodeC. Thenodes have the following qualities:

• NodeA owns the cluster resource with file systems /A1 and /A2

• NodeB owns the cluster resources with file systems /B1 and /B2

• NodeC owns the cluster resources with file systems /C1 and /C2

Note: NodeA can also have two non-clustered volumes, /fs1 and /fs2, that must bebacked up (not shown in the figure).

ibm.com/developerWorks developerWorks®

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 9 of 28

Page 10: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

For best backup performance, you might want all nodes in the cluster to perform thebackups of the shared file systems that they own. When a node failover occurs, thebackup tasks of the failed node shift to the node to which the failover occurred. Forexample, when NodeA fails over to NodeB, the backup of /A1 and /A2 moves toNodeB.

The example below presumes that the Tivoli Storage Manager Backup-Archive clientis installed and configured on all nodes in the cluster environment. Follow the stepsbelow to configure the Tivoli Storage Manager Backup-Archive client to back upcluster and non-cluster volumes:

Setup prerequisites for a clustered environment

The following setup requirements must be observed when using the backup-archiveclient in a clustered environment:

• A separate backup-archive client scheduler process must be run for eachresource group being protected. In normal conditions, each node has twoscheduler processes: one for the cluster resources, and one for the localfile systems. After a failure, additional scheduler processes are started ona node in order to protect the resources that have moved over fromanother node.

• The backup-archive client password files must be stored on cluster disksso that after a failure, the generated backup-archive client password isavailable to the takeover node.

• The file systems to be protected as part of a resource group are definedusing the backup-archive client domain option. The domain option isspecified in the dsm.sys file, which should also be stored on a cluster diskso that it can be accessed by the takeover node.

If the Tivoli Storage Manager Web client access is desired during a failovercondition, the Tivoli Storage Manager Web client acceptor daemon (CAD) must alsobe configured for each cluster resource. In this case, we suggest that you use TivoliStorage Manager CAD to manage the scheduler process to simplify theadministration and configuration. See the "Enabling Tivoli Storage Manager Webclient access in a cluster environment" section of this document for moreinformation.

Perform the following steps to configure the Tivoli Storage Manager backup-archiveclient in a cluster environment:

Step 1: Register backup-archive client node definitions on the

developerWorks® ibm.com/developerWorks

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and LinuxPage 10 of 28 © Copyright IBM Corporation 1994, 2008. All rights reserved.

Page 11: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

Tivoli Storage Manager Server

All nodes in the cluster must be defined on the Tivoli Storage Manager Server. If youare defining multiple cluster resources in a cluster environment to failoverindependently, then unique node names must be defined per resource group.

For the three-way, active-active cluster configuration sample above, define threenodes (one per resource), as shown below:

tsm: CLINTON>register node nodeA nodeApw domain=standardtsm: CLINTON>register node nodeB nodeBpw domain=standardtsm: CLINTON>register node nodeC nodeCpw domain=standard

Step 2: Configure the backup-archive client system options file

Each node in the cluster must have separate server stanzas for each clusterresource group in order to be backed up in each respective dsm.sys file. You mustensure that the server stanzas are identical in the system option files on each node.Alternatively, you can place the dsm.sys file on a shared cluster location. The serverstanzas defined to back up clustered volumes must have the following specialcharacteristics:

• The nodename option must refer to the client node name registered onthe Tivoli Storage Manager Server. If the client node name is not defined,the nodename will default to the hostname of the node, which mightconflict with other nodenames used for the same client system. Wesuggest that you use the nodename option to explicitly define the clientnode.

• The tcpclientaddress option must refer to the service IP address of thecluster node.

• The passworddir option must refer to a directory on the shared volumesthat are part of the cluster resource group.

• The errorlogname and schedlogname options must refer to files on theshared volumes that are part of the cluster resource group to maintain asingle continuous log file.

• All include exclude statements must refer to files on the shared volumesthat are part of the cluster resource group.

• If you use the inclexcl option, it must refer to a file path on the sharedvolumes that are part of the cluster group.

ibm.com/developerWorks developerWorks®

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 11 of 28

Page 12: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

• The stanza names identified with the servername option must be identicalon all systems.

Other backup-archive client options can be set as desired.

In the example, all three nodes, NodeA, NodeB, and NodeC, must have thefollowing three server stanzas in their dsm.sys file:

Servername clinton_nodeAnodename NodeAcommmethod tcpiptcpport 1500tcpserveraddress clinton.ibm.comtcpclientaddres nodeA.ibm.compasswordaccess generatepassworddir /A1/tsm/pwdmanagedservices scheduleschedlogname /A1/tsm/dsmsched.logerrorlogname /A1/tsm/errorlog.log

Servername clinton_nodeBnodename NodeBcommmethod tcpiptcpport 1500tcpserveraddress clinton.ibm.comtcpclientaddres nodeB.ibm.compasswordaccess generatepassworddir /B1/tsm/pwdmanagedservices scheduleschedlogname /B1/tsm/dsmsched.logerrorlogname /B1/tsm/errorlog.log

Servername clinton_nodeCnodename NodeCcommmethod tcpiptcpport 1500tcpserveraddress clinton.ibm.comtcpclientaddres nodeC.ibm.compasswordaccess generatepassworddir /C1/tsm/pwdmanagedservices scheduleschedlogname /C1/tsm/dsmsched.logerrorlogname /C1/tsm/errorlog.log

Step 3: Configure the backup-archive client user options file

The backup-archive client user options file (dsm.opt) must reside on the sharedvolumes in the cluster resource group. Define the DSM_CONFIG environmentvariable to refer to this file. Ensure that the dsm.opt file contains the followingsettings:

• The value of the servername option must be the server stanza in thedsm.sys file which defines parameters for backing up clustered volumes.

developerWorks® ibm.com/developerWorks

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and LinuxPage 12 of 28 © Copyright IBM Corporation 1994, 2008. All rights reserved.

Page 13: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

• Define the clustered file systems to be backed up with the domain option.

Note: Ensure that you define the domain option in the dsm.opt file orspecify the option in the schedule or on the Tivoli Storage Managercommand-line client. This is to restrict clustered operations to clusterresources and non-clustered operations to non-clustered resources.

In the example, nodes NodeA, NodeB, and NodeC set up their correspondingdsm.opt file and DSM_CONFIG environment variable as follows:

NodeA

1. Set up the /A1/tsm/dsm.opt file:

servername clinton_nodeAdomain /A1 /A2

2. Issue the following command or include it in your user's profile:

export DSM_CONFIG=/A1/tsm/dsm.opt

NodeB

1. Set up the /B1/tsm/dsm.opt file:

servername clinton_nodeBdomain /B1 /B2

2. Issue the following command or include it in your user's profile:

export DSM_CONFIG=/B1/tsm/dsm.opt

NodeC

1. Set up the /C1/tsm/dsm.opt file:

ibm.com/developerWorks developerWorks®

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 13 of 28

Page 14: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

servername clinton_nodeCdomain /C1 /C2

2. Issue the following command or include it in your user's profile:

export DSM_CONFIG=/C1/tsm/dsm.opt

Step 4: Set up the schedule definitions for each clusterresource group

After the basic setup is complete, define the automated schedules to back up clusterresources to meet the backup requirements. The procedure described belowillustrates the schedule setup by using the built-in Tivoli Storage Manager scheduler.If you are using a third-party scheduler, refer to the documentation provided by thevendor of the scheduler .

• Define a schedule in the policy domain where cluster nodes are defined.Ensure that the schedule's startup window is large enough to restart theschedule on the failover node in case of a failure and fallback event. Thismeans that the schedule's duration must be set to longer than the time ittakes to complete the backup of the cluster data for that node, undernormal conditions.

If the reconnection occurs within the start window for that event, thescheduled command is restarted. This scheduled incremental backupreexamines files sent to the server before the failover. The backup willthen "catch up" to where it terminated before the failover situation.

In our example, the clus_backup schedule is defined in the standarddomain to start the backup at 12:30 a.m. every day with the duration setto two hours (which is the normal backup time for each node's data). Seebelow:

tsm: CLINTON>define schedule standard clus_backup action=incr starttime=00:30startdate=TODAY Duration=2

• Associate the schedule with all the backup-archive client nodes defined tobackup cluster resources.

developerWorks® ibm.com/developerWorks

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and LinuxPage 14 of 28 © Copyright IBM Corporation 1994, 2008. All rights reserved.

Page 15: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

tsm: CLINTON>define association standard clus_backup nodeAtsm: CLINTON>define association standard clus_backup nodeBtsm: CLINTON>define association standard clus_backup nodeC

Step 5: Setup the scheduler service for backup

On each client node, a scheduler service must be configured for each resource thatthe node is responsible for backing up, under normal conditions.

The DSM_CONFIG environment variable for each resource scheduler service mustbe set to refer to the corresponding dsm.opt file for that resource. For the sampleconfiguration, the following three shell scripts must be created to allow dsmcadprocesses to be started, as needed, from any node in the cluster.

NodeA: /A1/tsm/startsched#!/bin/kshexport DSM_CONFIG=/A1/tsm/dsm.optdsmcadNodeB: /B1/tsm/startsched#!/bin/kshexport DSM_CONFIG=/B1/tsm/dsm.optdsmcadNodeC: /C1/tsm/startsched#!/bin/kshexport DSM_CONFIG=/C1/tsm/dsm.optdsmcad

For more information, refer to the BM Tivoli Storage Manager for UNIX and LinuxBackup-Archive Clients Installation and User's Guide Chapter 7, "Automating tasks"(see the Resources section for a link).

Step 6: Define the Tivoli Storage Manager Backup-Archiveclient to the cluster application

To continue the backup of the failed resource after a failover condition, the TivoliStorage Manager scheduler service (for each cluster client node) must be defined asa resource to the cluster application in order to participate in the failover processing.This is required in order to continue the backup of the failed resources from the nodethat takes over the resource. Failure to do so would result in the incomplete backupof the failed resource.

The sample scripts in Step 5 can be associated with the cluster resources to ensurethat they are started on nodes in the cluster while the disk resources being protectedmove from one node to another.

ibm.com/developerWorks developerWorks®

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 15 of 28

Page 16: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

The actual steps required to set up the scheduler service as a cluster resource arespecific to the cluster software. Refer to your cluster application documentation foradditional information.

Step 7: Validate the setup

To validate the setup, perform the following test scenarios:

• Ensure each node's password is generated and cached correctly in thelocation specified using the passworddir option. This can be validated byperforming the following two steps.

1. Validate that each node can connect to the Tivoli Storage ManagerServer without the password prompt. You can do this by runningthe backup-archive command-line client and issuing the followingcommand on each node:

#dsmc query session

If you are prompted to submit your password, enter the passwordto run the command successfully and re-run the command. Thesecond time, the command should run without the prompt for thepassword. If you get prompted for the password, check yourconfiguration starting at step 1.

2. Validate that the other nodes in the cluster can start sessions to theTivoli Storage Manager Server for the failed-over node. This canbe done by running the same commands, as described in the stepabove, on the backup nodes.

For example, to validate if NodeB and NodeC can start a sessionas NodeA in the failover event without prompting for the password,perform the following.

On NodeB and NodeC:

#export DSM_CONFIG=/A1/tsm/dsm.opt#dsmc query session

The prompt for the password might appear at this time, but this isunlikely. If you are prompted, the password was not stored in the

developerWorks® ibm.com/developerWorks

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and LinuxPage 16 of 28 © Copyright IBM Corporation 1994, 2008. All rights reserved.

Page 17: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

shared location correctly. Check the passworddir option settingused for NodeA and follow the configuration steps again.

• Ensure that the schedules are run correctly by each node. You can triggera schedule by setting the schedule's starttime to now. Remember to resetthe starttime after testing is complete.

tsm: CLINTON>update sched standard clus_backup starttime=now

• Failover and fallback between nodeA and nodeB, while nodeA is in themiddle of the backup and the schedule's start window, is still valid. Verifythat the incremental backup will continue to run and finish successfullyafter failover and fallback.

• Issue the command below to cause a node's (nodeA) password to expire.Ensure that backup continues normally under normal cluster operations,as well as failover and fallback.

tsm: CLINTON>update node nodeA forcep=yes

Step 8: Configure the backup-archive client to backup localresources

• Define client nodes on the Tivoli Storage Manager Server. Localresources should never be backed up or archived using node namesdefined to back up cluster data. If local volumes that are not defined ascluster resources will be backed up, separate node names (and separateclient instances) must be used for both non-clustered and clusteredvolumes.

In the example, assume that only NodeA has local filesystems /fs1 and/fs2 to be backed up. In order to manage the local resources, register anode, NodeA_local, on the Tivoli Storage Manager server.

tsm: CLINTON>register node nodeA_local nodeA_localpw domain=standard

• Add a separate stanza in each node's system options file dsm.sys thatmust back up local resources with the following special characteristics:

ibm.com/developerWorks developerWorks®

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 17 of 28

Page 18: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

• The value of the tcpclientaddress option must be the local host nameor IP address. This is the IP address used for primary traffic to andfrom the node.

• If the client will back up and restore non-clustered volumes withoutbeing connected to the cluster, the value of the tcpclientaddressoption must be the boot IP address. This is the IP address used tostart the machine (node) before it rejoins the cluster.

Example stanza for NodeA_local:

Servername clinton_nodeA_localnodename nodeA_localcommmethod tcpiptcpport 1500tcpserveraddress clinton.ibm.comtcpclientaddres nodeA_host.ibm.compasswordaccess generatemanagedservices schedule

• Define the user options file dsm.opt in a path that is on a non-clusteredresource.

• The value of the servername option must be the server stanza in thedsm.sys file which defines parameters for backing up non-clusteredvolumes.

• Use the domain option to define the non-clustered file systems to bebacked up.

Note: Ensure that you define the domain option in the dsm.opt file orspecify the option in the schedule or on the Tivoli Storage Managercommand-line client, in order to restrict the backup-archive operations tonon-clustered volumes.

In the example, NodeA uses the following /home/admin/dsm.opt file andsets up the DSM_CONFIG environment to refer to/home/admin/A1.dsm.opt.

Contents of /home/admin/A1.dsm.opt:

servername clinton_nodeA_localdomain /fs1 /fs2

export DSM_CONFIG=/home/admin/A1.dsm.opt

developerWorks® ibm.com/developerWorks

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and LinuxPage 18 of 28 © Copyright IBM Corporation 1994, 2008. All rights reserved.

Page 19: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

• Define and set up a schedule to perform the incremental backup fornon-clustered file systems as described in Step 4.

tsm: CLINTON>define schedule standard local_backup action=incr starttime=00:30startdate=TODAY Duration=2

Associate the schedule with all of the backup-archive client nodes that aredefined to backup non-clustered resources.

tsm: CLINTON>define association standard nodeA_local

Step 9: Restore cluster file system data

All volumes in a cluster resource are backed up under the target node defined forthat cluster resource. If you need to restore the data that resides on a clustervolume, it can be restored from the client node that owns the cluster resource at thetime of the restore. The backup-archive client must use the same user options file(dsm.opt) that was used during the backup to restore the data. There are noadditional setup requirements necessary to restore data on cluster volumes. Refer tothe IBM Tivoli Storage Manager for UNIX and Linux Backup-Archive ClientsInstallation and User's Guide Chapter 5: "Restoring your data" (see Resources for alink).

Step 10: Restore local file system data

The non-clustered volumes are backed up under the separate node name setup fornon -clustered operations. In order to restore this data, the Tivoli Storage ManagerBackup-Archive client must use the same user options file dsm.opt that was usedduring the backup.

In the example, set environment variable DSM_CONFIG to refer to/home/admin/A1.dsm.opt prior to performing a TSM client restore for the local node,nodeA_local.

Refer to the IBM Tivoli Storage Manager for UNIX and Linux Backup-Archive ClientsInstallation and User's Guide Chapter 5: "Restoring your data" (see Resources for alink).

ibm.com/developerWorks developerWorks®

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 19 of 28

Page 20: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

Section 5. Enabling the Tivoli Storage Manager StorageAgent to perform LAN-free data movement

The enhancements in version 5.4 of Tivoli Storage Manager Backup-Archive clientto remove the programming dependencies between the Backup-Archive clientsoftware and cluster software, such as AIX HACMP, do not hinder theBackup-Archive client from moving data over the storage area network (SAN). Theyalso do not affect the failover of the Tivoli Storage Manager Storage Agent or TivoliStorage Manager Server in a cluster environment.

While this paper does not offer guidance in setting up the Storage Agent and Serverfor LAN-free data movement, some general concepts on setting up the StorageAgent and Server for LAN-free data movement in cluster environment should beconsidered if you intend to move data over the SAN.

The simplest way to deploy Storage Agents in a cluster environment is to install asingle Storage Agent on each physical host in the cluster. If a Backup-Archive clientinstance fails from one physical host (hostA) to another physical host (hostB), it isserviced by the Storage Agent on hostB. Remember the following concepts whendeploying this type of configuration:

• The Storage Agents should have unique names on each host, forexample staA on hostA and staB on hostB.

• The Storage Agents should have the same network address andcommunication protocol, for example, each Storage Agent can beconfigured to use the TCP/IP loopback address (127.0.0.1) and TCP/IPport 1500.

To enable LAN-free data movement in the example used by the previous section,add the following options to each stanza in dsm.sys files, which were described inprevious sections. For example, to modify the dsm.sys file for NodeA, the followingLAN-free options are added:

Servername clinton_NodeAnodename NodeAcommmethod tcpip...enablelanfree yeslanfreecommmethod tcpiplanfreetcpport 1500lanfreetcpserveraddress 127.0.0.1

Other considerations must be taken into account to correctly manage how tape

developerWorks® ibm.com/developerWorks

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and LinuxPage 20 of 28 © Copyright IBM Corporation 1994, 2008. All rights reserved.

Page 21: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

mounts are used during a failover of an instance of the Backup-Archive Client in aclustered environment. For example, an instance of the Backup-Archive Clientcommunicating with Storage Agent staA on hostA has one or more tapes mountedto satisfy the backup request, and the Backup-Archive client fails over to hostB andstarts communication with Storage Agent staB. How the tapes that were mounted onbehalf of Storage Agent staA are managed depends on the configuration of the TSMserver. You must consider the following:

• The Tivoli Storage Manager Server for Windows® and AIX® has support,which correctly identifies that Storage Agent staA is no longercommunicating with the Tivoli Storage Manager Server and after a shortperiod of time, releases the relevant drives. In order to enable thissupport, you must ensure that the shared=yes and resetdrives=yesparameters are set for the library on the Tivoli Storage Manager Server.

• Although the Tivoli Storage Manager Server on other platforms do nothave this support for the Library Manager, they can still be used as thetarget of LAN-free data movement in a clustered environment. In thesecases, using the example above, the drives will be orphaned and nolonger available for use until a Tivoli Storage Manager administratormanually releases them. This may be satisfactory if there are sufficientdrives, as it would allow the backup to finish normally.

• Note that the Tivoli Storage Manager Backup-Archive client must havesufficient mount points to handle failover cases, which might orphandrives for any amount of time. This includes not only having physicalmount points available but also ensuring that there are enough mountpoints configured to the client node. For example, the Tivoli StorageManager Server maximum number of mount points parametermaxnummp must be set in accordance for this scenario.

For more detailed information and examples on configuration of the Tivoli StorageManager Storage Agents and Servers in a cluster environment refer to the IBMRedbooks® title "IBM Tivoli Storage Manager in a Clustered Environment" (seeResources for a link).

Section 6. Enabling Tivoli Storage Manager Web clientaccess in a cluster environment

ibm.com/developerWorks developerWorks®

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 21 of 28

Page 22: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

If the Tivoli Storage Manager Web client access is desired during a failovercondition, you must configure the Tivoli Storage Manager Web client acceptordaemon (CAD) associated with the cluster to failover along with the cluster resource.

After you complete the configuration steps described in the "Configuring the TivoliStorage Manager Backup-Archive Client in a cluster environment" section of thisdocument, perform the additional steps described below to complete the Web clientaccess setup.

Step 1: Set up the CAD to manage the Web client andscheduler

Tivoli Storage Manager CAD should be set up to manage schedulers as well as Webclient access. This reduces the number of daemons that need to be configured ascluster applications and thus simplifies the configuration and administration. When afailover occurs, the Tivoli Storage Manager CAD starts on the node that is managingthe takeover.

Update the managedservices option in the system options file dsm.sys on eachnode for each server stanza, as shown below for NodeA:

Servername clinton_NodeAnodename NodeAcommmethod tcpiptcpp 1500tcps clinton.sanjose.ibm.comtcpclientaddres nodeA.sanjose.ibm.compasswordaccess generatepassworddir /A1/tsm/pwdschedlogn /A1/tsm/dsmsched.logerrorlogname /A1/tsm/errorlog.logmanagedservices webclient schedule

Step 2: Set up the CAD to use a known HTTP port

By default, the CAD uses http port 1581, when available, for the Web client access.If this port is not available, the CAD finds the first available port, starting with 1581.In a failover condition of an active-active cluster configuration, a failover cluster hostsystem will likely be running multiple instances of the CAD. If default settings areused for the HTTP port, the failover node will use any available port for the CADbeing failed over, since the default port will likely be in use by the failover host'scurrent CAD processes. This causes problems for the Web client associated with theCAD that failed over, as the new HTTP port will not be known to the Web clientusers.

developerWorks® ibm.com/developerWorks

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and LinuxPage 22 of 28 © Copyright IBM Corporation 1994, 2008. All rights reserved.

Page 23: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

You can use the httpport option to specify the specific ports for the Web clientaccess for each resource. This allows you to always use the same port whenconnecting from a Web browser, independent of the node serving the clusterresource.

Add the httpport option in the system options file (dsm.sys) on each node for eachserver stanza as follows, making sure that each stanza uses a unique value:

Servername clinton_NodeAnodename NodeAcommmethod tcpiptcpp 1500tcps clinton.sanjose.ibm.comtcpclientaddres nodeA.sanjose.ibm.compasswordaccess generatepassworddir /A1/tsm/pwdmanagedservices webclient scheduleschedlogn /A1/tsm/dsmsched.logerrorlogname /A1/tsm/errorlog.loghttpport 1510

Servername clinton_NodeBnodename NodeBcommmethod tcpiptcpp 1500tcps clinton.sanjose.ibm.comtcpclientaddres nodeB.sanjose.ibm.compasswordaccess generatepassworddir /B1/tsm/pwdmanagedservices webclient scheduleschedlogn /B1/tsm/dsmsched.logerrorlogname /B1/tsm/errorlog.loghttpport 1511

Servername clinton_NodeCnodename NodeCcommmethod tcpiptcpp 1500tcps clinton.sanjose.ibm.comtcpclientaddres nodeC.sanjose.ibm.compasswordaccess generatepassworddir /C1/tsm/pwdmanagedservices webclient scheduleschedlogn /C1/tsm/dsmsched.logerrorlogname /C1/tsm/errorlog.loghttpport 1512

Section 7. Migrating legacy AIX HACMP setups

ibm.com/developerWorks developerWorks®

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 23 of 28

Page 24: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

The Tivoli Storage Manager Backup-Archive client integrates with AIX HACMP usingthe backup-archive client clusternode option. When the clusternode option is set toyes, it allows the backup-archive client to obtain the cluster name using the HACMPAPI. The cluster name is used internally by the backup-archive client to providefailover management capabilities. The cluster name is also used as the default clientnode name if the nodename option is not specified explicitly in the system optionsfile.

The clusternode option is no longer required in version 5.4 the Tivoli StorageManager Backup-Archive client to provide failover management and correctencryption for stored password. This option will be phased out in a future release infavor of the generalized approach outlined in this paper. This generalized approachremoves the need for certification of the HACMP version by Tivoli Storage Managerand removes compatibility requirements between HACMP and Tivoli StorageManager, reducing the risk of unexpected or unwanted behavior between the twoproducts.

If you are currently using the Tivoli Storage Manager Backup-Archive cClient in anHACMP environment using the clusternode option, we suggest that you updatecurrent configurations to the one described in this paper by following the procedureoutlined in the following section.

Step 1: Update the backup-archive client system options file

As with the clusternode option, each node in the cluster must continue to haveseparate server stanzas for each cluster resource group to be backed up in eachrespective dsm.sys file.

The existing dsm.sys file for NodeA might appear as follows:

Servername clinton_nodeAcommmethod tcpiptcpp 1500tcps clinton.sanjose.ibm.comtcpclientaddres nodeA.sanjose.ibm.compasswordaccess generatepassworddir /A1clusternode yesmanagedservices scheduleschedlogn /A1/dsmsched.logerrorlogname /A1/errorlog.log

Notice that no nodename option is used in this sample.

Make the following changes to the existing dsm.sys file for NodeA.

developerWorks® ibm.com/developerWorks

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and LinuxPage 24 of 28 © Copyright IBM Corporation 1994, 2008. All rights reserved.

Page 25: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

• Remove the clusternode option.

• Specify a nodename option if you do not have one already specified. Inorder to avoid backing up the entire node's data again, use the existingcluster node name registered on the TSM server as the value for thenodename option.

The new dsm.sys file for NodeA should appear as follows:

Servername clinton_nodeAcommmethod tcpipnodename myclustcpp 1500tcps clinton.sanjose.ibm.comtcpclientaddres nodeA.sanjose.ibm.compasswordaccess generatepassworddir /A1managedservices scheduleschedlogn /A1/dsmsched.logerrorlogname /A1/errorlog.log

Note that myclus is the existing cluster name.

Step 2: Register backup-archive client nodes on the TivoliStorage Manager Server

If new backup-archive client nodes are added in the first step to replace the currentdefault value of the cluster node name, register those nodes on the Tivoli StorageManager Server.

Step 3: Update schedule definitions

If new backup-archive client nodes are added in Step 2, ensure that the backupschedule definitions used earlier to back up this node's data are now associated withthe new client node names.

Step 4: Validate the setup

To validate the setup, follow Step 6 described in the "Configuring the Tivoli StorageManager Backup-Archive client in a cluster environment" section.

ibm.com/developerWorks developerWorks®

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 25 of 28

Page 26: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

Section 8. Conclusion

Enhancements in version 5.4 of the Tivoli Storage Manager Backup-Archive clienthave helped remove the programming dependencies between the backup-archiveclient software and cluster software, such as AIX HACMP. These enhancementsenable the backup-archive client to be deployed into cluster environments on UNIXand Linux, independent of the cluster software. If you are currently using the TivoliStorage Manager Backup-Archive Client in an HACMP AIX cluster environment,update your configuration as described by this document.

developerWorks® ibm.com/developerWorks

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and LinuxPage 26 of 28 © Copyright IBM Corporation 1994, 2008. All rights reserved.

Page 27: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

Resources

• IBM Tivoli Storage Manager for UNIX and Linux Backup-Archive ClientsInstallation and User's Guide .

• IBM Redbooks: IBM Tivoli Storage Manager in a Clustered Environment .

• High Availability Cluster Multi-Processing for AIX - Planning and InstallationGuide

About the authors

Jason BaslerJason Basler is a Senior Software Engineer in the IBM Tivoli Software organization.He has spent the past twelve years testing various components and releases of theTivoli Storage Manager product family.

Neeta GarimellaNeeta Garimella is a Senior Engineer and has been part of the TSM developmentorganization for over six years. She has been a key contributor to the TSMBackup-Archive client and TSM for Data Protection products including operationswithin highly available environments. She was the lead developer for Tivoli WorkloadScheduler. Before joining IBM, she worked at BEA Systems as a professionalservices consultant where she helped customers build and deploy solutions usingBEA products.

Kevin HoytKevin Hoyt is a Staff Software Engineer in the IBM Tivoli Software organization. Hehas been involved with Tivoli Storage Manager for over 8 years and is the leaddeveloper for the Mac OS X client.

Giang T PhamGiang Pham is a Staff Software Engineer in the IBM Tivoli Software organization. He

ibm.com/developerWorks developerWorks®

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and Linux© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 27 of 28

Page 28: Using the IBM Tivoli Storage Manager Backup-Archive · PDF fileBasic concepts of clustering and how Tivoli Storage Manager ... of how the Tivoli Storage developerWorks® ibm ... the

is currently working in System Test Team for Tivoli Storage Manager product.

Jim SmithJim Smith has been involved with Tivoli Storage Manager for over ten years. He hasbeen a part of the support and development teams and is currently serving as thearchitect for the Tivoli Storage Manager Backup-Archive clients. He has beeninvolved in many Tivoli Storage Manager projects involving clustering solutions suchas Microsoft Windows Cluster Services, High Availability Cluster Multi-Processing,Novell Cluster Services and VERITAS Cluster Services.

developerWorks® ibm.com/developerWorks

Using the IBM Tivoli Storage Manager Backup-Archive client in cluster environments on UNIX and LinuxPage 28 of 28 © Copyright IBM Corporation 1994, 2008. All rights reserved.