SAP on RAC Update

Kurt Broeg Jan Klokkers Principal Member of Senior DirectorTechnical Staff

SAP DevelopmentServer Technologies

Oracle Corporation

Topics

� Introduction� Certification Status� Mandatory SAP Requirements for RAC� SAP Certified Technology Stack for RAC 9.2� Necessary Technology Stack for SAP with RAC 10.2� Introduction of Net Services� SAPCTL

Introduction

� More than 18 years of SAP – Oracle cooperation� More than 12,000 SAP customers use Oracle� 2 out of 3 SAP installations run on Oracle Database� Joint SAP & Oracle development/platform team in

Walldorf� Joint support teams in Rot (Germany) & US� SAP & Oracle Partner/Presales teams in Walldorf

Certification Status –Supported Platforms of SAP for Oracle

� IBM AIX5L� HP-UX PA-RISC� HP-UX IA64� HP Tru64� Linux x86� Linux x64� Linux IA64� Linux Power

� Solaris Sparc-64� Solaris x64 (AMD64)� Windows x86� Windows x64� Windows IA64

Certification Status – Supported Oracle Versions

� Oracle 9.2.x– Single Instance and RAC– SAP versions 4.6C, 4.7, ERP2004

� Oracle 10.1.0.x (x>=4)– Only Single Instance– Only for SAP ERP2005

� Oracle 10.2.0.x (x>=2)– Single Instance available– RAC 10.2 availability planned for Q4CY2006– SAP versions 4.6C, 4.7, ERP2004 and ERP2005

Mandatory SAP Requirements for RAC

� Shared Oracle Home– No support for local Oracle Homes

� Database on Shared Filesystem– No support for Raw Devices

– No support for ASM (planned for Oracle11)

� Shared Archive Destination on Shared Filesystem� Shared SAP Directories on Shared Filesystem

– saparch, sapbackup, sapreorg, saptrace, sapcheck

Certified RAC 9.2 Configurations for SAP

Oracle Clusterware + OCFS 9.2.0.7/8 2003 Enterprise ServerWindows

Veritas Foundation Suite 4.0/5 or SUN Cluster 3.1+ QFS

8, 9, 10Solaris (SUN)

Veritas Foundation Suite 4.0/5 or PrimeCluster + Net App Filer

8, 9, 10Solaris (Fujitsu Siemens)

Oracle Clusterware andOCFS2 1.2.x or NetApp NFS

SLES9, RHEL4Linux

Oracle Clusterware and

OCFS 1.0.14 or NetApp NFS

SLES8, RHEL3Linux

HACMP 4.5/5.1 + GPFS 2.2/2.35.2/3IBM AIX

TruCluster Server 5.15.1bHP Tru64

ServiceGuard with SGCFS 11.23HP-UX

Cluster Version and CFS TypeOS Version

Required RAC 10.2 Configurations SAP

Oracle Clusterware + OCFS 10.22003 Enterprise ServerWindows

Veritas Foundation Suite 4.0/5 or SUN Cluster 3.1+ QFS or

Oracle Clusterware + NetApp NFS

9, 10Solaris (SUN)

Veritas Foundation Suite 4.0/5 or PrimeCluster + Net App Filer orOracle Clusterware + NetApp NFS

9, 10Solaris (Fujitsu Siemens)

Oracle Clusterware and OCFS2 1.2.x or NetApp NFS

SLES9, RHEL4Linux

HACMP 5L + GPFS 2.3 or

Oracle Clusterware + GPFS 2.3

5.2/3IBM AIX

TruCluster Server 5.15.1bHP Tru64

ServiceGuard with SGCFS 11.23HP-UX

Cluster Version and CFS TypeOS Version

Q U E S T I O N SQ U E S T I O N SA N S W E R SA N S W E R S

Providing High Availability for SAP Resources

Abstract

Oracle Clusterware is the component that’s responsib le for restarting the Oracle Real Application Cluster Inst ance and Listeners on process failure & relocating Database Virtual IP’s on Node failure. A key new feature for Oracle Database 10g Release 2 – Oracle Clusterware is now made availa ble for customers and partners to protect 3rd Party Applica tions.

Especially for SAP, Oracle provides a tool sapctl which uses the capabilities of Oracle Clusterware to support th e SAP Replicated Enqueue feature.

This presentation will cover the implementation and use of sapctl in SAP HA - systems.

Concept of SAP Enqueue Replication

The SAP Enqueue Service provides distributed lock m anagement forSAP R/3 instances. The SAP Enqueue Service must be available whenever an SAP installation uses more than one SAP R/3 instance.

SAP uses an active/passive approach to enable high availability for the Enqueue Service. There is exactly one Enque ue Service running in the SAP system. The Replication Service run on the node on which the Enqueue Service runs in the event of a failure or planned outage.

The Replication Service maintains a copy of the Enq ueue Service ´s lock table; this copy is kept in a shared memory se gment during runtime.

The node that supports the Enqueue Service also host s a virtual IP address (VIP) for communication. Whenever the active node fails or is shut down, the VIP and the Enqueue Service fail ove r to the next node.

Control flow within SAPCTL to support SAP Enqueue Replication

Enqueue

Server

Lock

Table

Replication Server

Lock Replica Table

TCP Connection

Lock State

Transfer

0. Normal Operation, CRS checks healthiness of resources on all hosts in the cluster

VIP

Host 1 Host 2

Control flow within SAPCTL to support SAP Enqueue Replication

Enqueue

Server

Lock

Table

Replication Server

Lock Replica Table

1. CRS detects a failure of the node or in the Enqueue Server

VIP

Host 1 Host 2

Crash

Control flow within SAPCTL to support SAP Enqueue Replication

Replication Server

Lock Replica Table

2. CRS switches the VIP to the node where Replication Server is running

Host 1 Host 2

VIP

Control flow within SAPCTL to support SAP Enqueue Replication

Replication Server

Lock Replica Table

3. CRS starts a new Enqueue Server on the node supporting the VIP

Host 1 Host 2

VIP

Enqueue

Server

Lock

Table

Control flow within SAPCTL to support SAP Enqueue Replication

Replication Server

Lock Replica Table

4. Enqueue Server reads data from Lock Replica Table

Host 1 Host 2

VIP

Enqueue

Server

Lock

Table

fill Enq. Table with

replica Data

Control flow within SAPCTL to support SAP Enqueue Replication

5. CRS stops Replication Server after Lock Table is rebuild from Lock Replica Table

Host 1 Host 2

VIP

Enqueue

Server

Lock

Table

Control flow within SAPCTL to support SAP Enqueue Replication

6. CRS starts Replication Server on another available node in the cluster

Host 1 Host 2

VIP

Enqueue

Server

Lock

Table

Host N

Replication Server

Lock Replica Table

Control flow within SAPCTL to support SAP Enqueue Replication

7. Back to normal operation, CRS checks healthiness of resources on all hosts in the cluster

Host 1 Host 2

VIP

Enqueue

Server

Lock

Table

Host N

Replication Server

Lock Replica Table

Lock State

Transfer

Connect

Possible Configuration

� Oracle Clusterware HA Agent for SAP is protecting SAP Enqueue Server and SAP Enqueue Replication Server

� It is also providing resources to the RAC database running on nodes 1 & 2

listener

SAP Enqueue Server

ORA_CRS_HOME

ORACLE_HOME

listener

SAP Enqueue Repl.

cluster database

ORA_CRS_HOME

ORACLE_HOME

ORA_CRS_HOME

listener

database

ORA_CRS_HOME

ORACLE_HOME

SAP Enqueue Repl.

cluster database

The Oracle Clusterware Framework

� Calls SAP HA - agent� agent will be invoked by Oracle Clusterware

with one of three commands– start– stop– check

� Oracle Cluster Registry– Stores information relating to how Oracle

Clusterware interacts and manages the application

Oracle ClusterwareFramework Architecture

AgentFramework

Please Start

How are you ?

Please Stop

‘check’

‘start’

‘stop’

How the HA Agent for SAP is called by the Framework

The ‘start’ commandWould invoke the sapstart command

startsap r3 ASCSstartsap r3 ENR

The ‘check’ commandThere are a number of things that could be checked

Uses ensmon to obtain the status of the replicationBut: Does not check the message server

The ‘stop’ commandWould invoke the stopsap command

stopsap r3 ASCSstopsap r3 ENR

‘start’

‘stop’

‘check’

� sapctl create - registers resources � sapctl config - shows configuration� sapctl status - shows status of resources� sapctl start - starts resources� sapctl stop - stops resources� sapctl update - modfy configuration � sapctl remove - delete configuration

Overview: SAPCTL commands for use with SAP R/3 EnqueueReplication

� sapctl create -sapsid C11 <- name of the SAP System -if eth0 <- network interface-nm 255.255.254.0 <- netmask-ip 140.86.242.63 <- IP address VIP-u c11adm <- SAP admin-n “node1 node2” <- list of nodes-s /saphome/c11adm <- $HOME of SAP admin-e ASCS01 <- Central Instance-r ENR01 <- Enqueue replication

SAPCTL command create

� sapctl configshows list of protected SAP systems

� sapctl config enqueue –sapsid C11shows configuration of Central Instance

� sapctl config replication –sapsid C11shows configuration of Replication Server

� sapctl config all –sapsid C11shows complete configuration for SAP system

SAPCTL command config

� sapctl status enqueue –sapsid C11shows status of Central Instance

� sapctl status replication –sapsid C11shows status of Replication Server

� sapctl status all –sapsid C11shows status of all SAP resources

SAPCTL command status

� sapctl start enqueue –sapsid C11starts SAP Central Instance

� sapctl start replication –sapsid C11starts SAP Replication Server

� sapctl start all –sapsid C11starts all SAP resources

SAPCTL command start

� sapctl stop enqueue –sapsid C11stops SAP Central Instance

� sapctl stop replication –sapsid C11stops SAP Replication Server

� sapctl stop all –sapsid C11stops all SAP resources

SAPCTL command stop

� sapctl update -sapsid C11 <- name of the SAP System

-ip 140.86.242.63 <- IP address VIP

-a “node1 node2” <- list of nodes to add-d “node1 node2” <- list of nodes to delete

SAPCTL command update

� sapctl remove all –sapsid C11removal of all SAP resources

- only remove of all SAP resources possible- `root` permission required

SAPCTL command remove

Commonly used scripts startsap and stopsap trigger the start / stop

of CRS resources as well. Consider this as either a very convenient

feature or a big advantage, as other tools or scripts relying on thesecommands still work.

startsap r3 ASCS01 - starts Enqueue and Message Server

stopsap r3 ASCS01 - stops Enqueue and Message Server

startsap r3 ENR01 - starts Replication Server stopsap r3 ENR01 - stops Replication Server

Restrictions apply: User must use this command on the right node!

SAP start/stop commands still usable

startsap script:

• add instance type ENR / ASCS / SCS• add service type ER • call wrapper to notify Oracle Clusterware in function

start_instance if service is “CS” or “ER” :

sap2crs start \$SAPSYSTEMNAME \$SERVICE \$INSTANCE

SAP changes required to use SAPCTL

stopsap script:

• add instance type ENR / ASCS / SCS• add service type ER • call wrapper to notify Oracle Clusterware in function

stop_instance if service is “CS” or “ER” :

sap2crs stop \$SAPSYSTEMNAME \$SERVICE \$INSTANCE

SAP changes required to use SAPCTL

• Instance profile Standalone Enqueue Server (ENQ + MSG)

• Instance profile Enqueue Replication Server (ENR)enque/process_location=LOCALenque/server/replication=trueenque/server/internal_replication= true

• Start profile Standalone Enqueue Server• Start profile Enqueue replication Server

As recommended by SAP HA documentation

SAP changes required to use SAP HA

• All scripts exist multiple times: START_ASCS01_node-1, C11_ASCS01_node-1START_ASCS01_node-2, C11_ASCS01_node-2

…..START_ASCS01_node-n, C11_ASCS01_node-n

The hostname is not virtualized by Oracle Clusterware!

SAP changes required to use SAP HA

Demonstration

Further Information available:

� Oracle Whitepapers:

1. Technical White Paper: Providing High Availability for SAP Resources

2. SAP NetWeaver with Oracle 10gR2 RAC (UNIX & Linux)

Oracle Location:http://www.oracle.com/sap -> Oracle on RAC section

SAP Location:http://service.sap.com/dbaora ->Oracle Database

Administration -> Media Library -> Oracle RAC

Q U E S T I O N SQ U E S T I O N SA N S W E R SA N S W E R S