cics transaction gateway for z/os version 6vi cics transaction gateway for z/os version 6.1 5.2...

ibm.com/redbooks

CICS Transaction Gateway for z/OS Version 6.1

Nigel WilliamsColin Alcock

Steven DayTommy Joergensen

Rob JonesPhil Wakelin

Install and configure the Gateway on z/OS

Connect from WebSphere on Windows and z/OS

Enable global transaction support and SSL security

Front cover

http://www.redbooks.ibm.com/



May 2006

International Technical Support Organization

SG24-7161-00

© Copyright International Business Machines Corporation 2006. All rights reserved.Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADPSchedule Contract with IBM Corp.

First Edition (May 2006)

This edition applies to CICS Transaction Gateway for z/OS V6.1.

Note: Before using this information and the product it supports, read the information in “Notices” on page xi.

Contents

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiTrademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiThe team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xivBecome a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviComments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi

Part 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Chapter 1. CICS Transaction Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.1 What is CICS Transaction Gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.1.1 CICS TG products. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.1.2 CICS TG application programming interfaces. . . . . . . . . . . . . . . . . . . 8

1.2 CICS TG for z/OS modes of operation . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.2.1 Remote mode of operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.2.2 Local mode of operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.3 JCA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.3.1 Overview of JCA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.3.2 CICS resource adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.4 Using the JCA with different CICS TG topologies . . . . . . . . . . . . . . . . . . . 151.4.1 WebSphere Application Server and CICS TG on distributed . . . . . . 161.4.2 WebSphere Application Server on distributed, CICS TG on z/OS . . 181.4.3 WebSphere Application Server and CICS TG on zSeries . . . . . . . . 191.4.4 Mixed version support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Chapter 2. Installing and configuring the CICS TG . . . . . . . . . . . . . . . . . . 252.1 Preparing for the installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.1.1 Software checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262.1.2 Definitions checklist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.2 Installing CICS TG. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.2.1 Running the ctginstall script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.2.2 Basic security considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.3 Configuring CICS TG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.3.1 Defining an HFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.3.2 Setting permissions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352.3.3 Creating a Gateway daemon configuration file . . . . . . . . . . . . . . . . . 362.3.4 Creating an STDENV file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372.3.5 Creating the started task JCL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

© Copyright IBM Corp. 2006. All rights reserved. iii

2.4 Configuring CICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432.4.1 Creating a CONNECTION definition . . . . . . . . . . . . . . . . . . . . . . . . . 432.4.2 Creating a SESSIONS definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . 442.4.3 Installing the definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452.4.4 Configuring CICS for RRMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452.4.5 Compiling and installing the sample programs . . . . . . . . . . . . . . . . . 462.4.6 Editing and assembling DFHCNV . . . . . . . . . . . . . . . . . . . . . . . . . . . 462.4.7 Opening interregion communication . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.5 Testing the configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462.6 Operating CICS TG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

2.6.1 Starting the Gateway daemon. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 512.6.2 Stopping the Gateway daemon. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522.6.3 Automatic Restart Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

2.7 Configuring for multiple Gateway daemons . . . . . . . . . . . . . . . . . . . . . . . 552.8 Migrating to CICS TG V6.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

2.8.1 Migrating from ctgenvvar. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 582.8.2 Other migration and configuration considerations. . . . . . . . . . . . . . . 58

2.9 Problem determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 602.9.1 Common problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 602.9.2 Tips and utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 622.9.3 Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Part 2. Connection Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Chapter 3. Gateway daemon connections . . . . . . . . . . . . . . . . . . . . . . . . . 773.1 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

3.1.1 Software checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 793.1.2 Definitions checklist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

3.2 Configuring WebSphere Application Server . . . . . . . . . . . . . . . . . . . . . . . 803.2.1 Installing the resource adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 803.2.2 Creating connection factories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 823.2.3 Deploying the sample J2EE application . . . . . . . . . . . . . . . . . . . . . . 89

3.3 Configuring the Gateway daemon. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 913.3.1 TCP/IP connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 913.3.2 Gateway threads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

3.4 Configuring EXCI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 963.4.1 EXCI pipe limits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 973.4.2 Our EXCI settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 973.4.3 EXCI user-replaceable module: DFHXCURM. . . . . . . . . . . . . . . . . 1003.4.4 Transactional EXCI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

3.5 Configuring CICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1003.5.1 CICS CONNECTION and SESSION values . . . . . . . . . . . . . . . . . . 1013.5.2 Adding a private mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

iv CICS Transaction Gateway for z/OS Version 6.1

3.5.3 Transaction timeouts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1023.6 Configuring for high scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

3.6.1 TCP/IP load balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1033.6.2 Dynamic program link . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

3.7 Testing the configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1063.7.1 Single Gateway daemon connectivity tests. . . . . . . . . . . . . . . . . . . 1073.7.2 Cloned Gateway daemon connectivity tests . . . . . . . . . . . . . . . . . . 114

3.8 Problem determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1163.8.1 Common connection problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1163.8.2 Tips and utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1173.8.3 Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

Chapter 4. Connecting from WebSphere Application Server for z/OS . . 1214.1 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

4.1.1 Software checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1234.1.2 Definitions checklist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

4.2 WebSphere Application Server configuration . . . . . . . . . . . . . . . . . . . . . 1244.2.1 J2EE server configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1244.2.2 Defining workload profile. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1254.2.3 Installing the CICS ECI resource adapter . . . . . . . . . . . . . . . . . . . . 1264.2.4 Creating a connection factory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1284.2.5 Configuring the J2EE server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1334.2.6 Deploying our sample J2EE application . . . . . . . . . . . . . . . . . . . . . 134

4.3 Configuring EXCI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1374.3.1 EXCI pipe limits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1374.3.2 CTG_PIPE_REUSE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1374.3.3 Defining the EXCI library to your J2EE Server . . . . . . . . . . . . . . . . 138

4.4 Configuring CICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1394.4.1 Adding a private mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1404.4.2 Transaction timeouts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

4.5 Testing the configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1424.6 Configuring for high scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

4.6.1 TCP/IP load balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1494.6.2 Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1494.6.3 Dynamic program link . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1504.6.4 Other configuration considerations . . . . . . . . . . . . . . . . . . . . . . . . . 151

4.7 Problem determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1514.7.1 Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

Part 3. Transaction Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

Chapter 5. Gateway daemon transactions . . . . . . . . . . . . . . . . . . . . . . . . 1575.1 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

5.1.1 Software checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

Contents v

5.2 Transactions - what are they . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1595.2.1 CICS transactions, tasks, and syncpoints. . . . . . . . . . . . . . . . . . . . 1595.2.2 Distributed transactions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

5.3 Resource Recovery Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1635.3.1 How RRS works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1635.3.2 CICS TS and RRS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

5.4 Transactional support in the CICS TG . . . . . . . . . . . . . . . . . . . . . . . . . . 1665.4.1 Extended logical units of work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1665.4.2 Transactional support in the CICS ECI resource adapters . . . . . . . 167

5.5 Transactions in WebSphere Application Server . . . . . . . . . . . . . . . . . . . 1715.6 Configuring for extended LUWs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

5.6.1 Definitions checklist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1745.6.2 Configuring CICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1755.6.3 Configuring CICS TG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1755.6.4 Configuring WebSphere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1765.6.5 Testing the extended LUW configuration . . . . . . . . . . . . . . . . . . . . 182

5.7 Configuring for XA transactions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1935.7.1 Definitions checklist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1945.7.2 Configuring CICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1955.7.3 Configuring CICS TG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1955.7.4 Configuring WebSphere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1985.7.5 Testing the XA transaction configuration . . . . . . . . . . . . . . . . . . . . 201

5.8 Operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2195.8.1 Gateway daemon shutdown and restart . . . . . . . . . . . . . . . . . . . . . 2195.8.2 Gateway daemon restart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2205.8.3 The ctgasi utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

5.9 Problem determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2215.9.1 Common transaction problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2215.9.2 Tips and utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

Chapter 6. Gateway daemon transactions with TCP/IP load balancing . 2256.1 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2266.2 Software checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2276.3 Transaction affinity considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

6.3.1 Extended LUWs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2276.3.2 XA transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2286.3.3 XA-enabled CICS TG group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

6.4 Configuring an XA-enabled CICS TG group . . . . . . . . . . . . . . . . . . . . . . 2306.4.1 CICS TG master process configuration . . . . . . . . . . . . . . . . . . . . . 2306.4.2 Gateway daemon configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

6.5 Testing the CICS TG group configuration . . . . . . . . . . . . . . . . . . . . . . . . 2346.5.1 Definitions checklist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2366.5.2 Test scenarios. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

vi CICS Transaction Gateway for z/OS Version 6.1

6.6 Operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2486.6.1 CICS TG group startup and shutdown . . . . . . . . . . . . . . . . . . . . . . 2486.6.2 CICS TG group commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

6.7 Problem determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2516.7.1 Tracing with ctgmaster trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

Chapter 7. Transactions with WebSphere Application Server for z/OS . 2537.1 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

7.1.1 Software checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2557.2 Resource Recovery Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2557.3 Transactional support with WebSphere on z/OS . . . . . . . . . . . . . . . . . . 255

7.3.1 Transactional support in the CICS ECI resource adapters . . . . . . . 2567.4 Configuring for global transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

7.4.1 Definitions checklist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2587.4.2 Configuring CICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2587.4.3 Configuring WebSphere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2597.4.4 Testing the configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263


Part 4. Security Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

Chapter 8. Gateway daemon security . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2818.1 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282


8.2 Introduction to CICS and CICS TG security . . . . . . . . . . . . . . . . . . . . . . 2838.3 Configuring CICS and CICS TG security . . . . . . . . . . . . . . . . . . . . . . . . 2868.4 Configuring JCA security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293

8.4.1 J2C authentication data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2948.4.2 Setting user ID and password on the ECIConnectionSpec. . . . . . . 296

8.5 Testing the configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2968.5.1 Testing security with Java program EciB1 . . . . . . . . . . . . . . . . . . . 2968.5.2 Testing security with J2EE application CTGTesterCCI. . . . . . . . . . 299

8.6 Problem determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3068.6.1 Common security problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3068.6.2 Tips and utilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308

Chapter 9. Enabling SSL with the Gateway daemon . . . . . . . . . . . . . . . . 3099.1 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310


9.2 Secure Sockets Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3119.2.1 JSSE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312

Contents vii

9.2.2 Cipher suites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3139.2.3 Hardware cryptography. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3149.2.4 The hwkeytool command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3149.2.5 Java configuration for using hardware cryptography . . . . . . . . . . . 315

9.3 Configuring server authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3169.3.1 Creating a keystore on z/OS using keytool . . . . . . . . . . . . . . . . . . . 3169.3.2 Creating a key ring and certificates on z/OS using RACF . . . . . . . 3199.3.3 Configuring CICS TG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3239.3.4 Testing server authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325

9.4 Configuring client authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3299.4.1 Creating the client certificate on a Windows client machine . . . . . . 3299.4.2 Extracting a client personal certificate. . . . . . . . . . . . . . . . . . . . . . . 3309.4.3 Configuring CICS TG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3319.4.4 Testing client authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332

9.5 Migration considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3359.5.1 SystemSSL to JSSE migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335


Chapter 10. Security with WebSphere Application Server for z/OS . . . . 33910.1 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340

10.1.1 Software checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34110.1.2 Definitions checklist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341

10.2 Configuring CICS and CICS TG security . . . . . . . . . . . . . . . . . . . . . . . 34210.3 Configuring JCA security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347

10.3.1 Thread identity support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34710.4 Testing the configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

10.4.1 Testing thread identity support . . . . . . . . . . . . . . . . . . . . . . . . . . . 35310.5 Problem determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357

Part 5. Appendixes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359

Appendix A. Sample applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361The CTGTesterCCI application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362

Application overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363The CTGTesterCCIXA application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372

Application overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372

Appendix B. DFHCNV and CICS data conversion . . . . . . . . . . . . . . . . . . 379Data conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380

DFHCNV and the mirror program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380Code page aware Java programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381Code page aware Java programs without DFHCNV. . . . . . . . . . . . . . . . . 382

viii CICS Transaction Gateway for z/OS Version 6.1

Appendix C. EXCI Trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385EXCI trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386

Appendix D. Additional material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393Locating the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393Using the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394

System requirements for downloading the Web material . . . . . . . . . . . . . 394How to use the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394

Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399

Contents ix

x CICS Transaction Gateway for z/OS Version 6.1

Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental.

COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrates programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and distribute these sample programs in any form without payment to IBM for the purposes of developing, using, marketing, or distributing application programs conforming to IBM's application programming interfaces.

© Copyright IBM Corp. 2006. All rights reserved. xi

TrademarksThe following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:

Redbooks (logo) ™ibm.com®z/OS®z/VM®zSeries®AIX®CICS®CICSPlex®DB2®

HiperSockets™Iterations®IBM®IMS™Language Environment®MVS™OS/390®POWER™Rational®

Redbooks™RACF®SupportPac™Tivoli®TXSeries®VisualAge®WebSphere®

The following terms are trademarks of other companies:

Enterprise JavaBeans, EJB, Java, Java Naming and Directory Interface, JavaBeans, JavaServer, JDBC, JDK, JSP, JVM, J2EE, Solaris, Sun, and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.

Intel, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

Other company, product, or service names may be trademarks or service marks of others.

xii CICS Transaction Gateway for z/OS Version 6.1

Preface

The CICS® Transaction Gateway (CICS TG) provides the IBM® implementation of the J2EE™ Connector Architecture (JCA) for access to CICS from WebSphere® Application Server.

CICS TG V6 currently supports the following platforms: z/OS®, Linux® on zSeries®, AIX® , Linux on Intel®, Microsoft® Windows®, HP-UX, and Sun™ Solaris™ operating environments.The qualities of service of the CICS TG are highest when deploying on the z/OS platform. This configuration provides significantly improved performance and better management of connections, security, and transactions.

In this IBM Redbook, we introduce the new facilities of the CICS TG for z/OS V6.0 and V6.1, which provide significant improvements in the areas of transactional integration, systems management, performance, security and ease of use. Included are a set of configuration scenarios for a variety of different z/OS deployment topologies for integrating J2EE applications on WebSphere Application Server with CICS TS for z/OS.

The topologies illustrate how to use the new CICS ECI XA resource adapter to provide two-phase commit transactional integration between WebSphere Application Server and CICS TS. Also included are details on how to:

� Dynamically administer the Gateway daemon from the z/OS console

� Use the new JES logging capabilities

� Configure remote connections from WebSphere Application Server running on a Windows platform

� Configure local connections from WebSphere Application Server for z/OS

� Enable secure interoperability between WebSphere Application Server and CICS

� Exploit a RACF® key ring when configuring JSSE SSL support

© Copyright IBM Corp. 2006. All rights reserved. xiii

The team that wrote this redbookThis International Technical Support Organization (ITSO) redbook was produced by a team of specialists from around the world working at the Product Solutions and Support Centre in IBM Montpellier, France.

Figure 1 The team posing in the IBM Montpellier vineyard

Colin Alcock is an IT Software Support Specialist based in IBM Farnborough, UK. He has worked in IBM providing customer support for CICS products since 1989 and currently supports the CICS TS, TXSeries® CICS and CICS Transaction Gateway products. Colin has worked on two other ITSO residencies, and previously worked in the CICS TS change team.

Steven Day works for the CICS TG Level 3 Service Team at IBM Hursley. He has 20 years experience in mainframe systems programming.

Tommy Joergensen is a Senior IT Specialist working for IBM Global Services in IBM Denmark. He has more than 25 years of experience working in CICS technical support, including three years at IBM Hursley. In recent years he has delivered services at large accounts in Denmark for both the CICS and WebSphere products. Tommy is the IBM representative in the CICS working group of the Nordic Share Guide organization.

xiv CICS Transaction Gateway for z/OS Version 6.1

Rob Jones is a Software Engineer working for IBM at the Hursley software development laboratory in the United Kingdom. He has ten years of experience in the CICS Transaction Processing area, including development experience with CICS Transaction Server for z/OS and the CICS Transaction Gateway. He holds a degree in Computer Science and Mathematics from Durham University. He specializes in the z/OS platform aspects of the CICS TG product, most recently working on CTGBATCH and ctgmaster components.

Phil Wakelin is the CICS Transaction Gateway Technical Planner, responsible for the product's future development. Based at IBM Hursley, he's an IBM Certified Solutions Expert in CICS Web Enablement and the author of many papers, IBM Redbooks™, and CICS SupportPacs.

Nigel Williams is a Certified IT Specialist working in the IBM Design Centre for On Demand Business in Montpellier. He specializes in core business transformation, connectors, and service-oriented architectures. He is the author of several papers and IBM Redbooks and he speaks frequently on CICS and WebSphere topics. Previously, Nigel worked at the Hursley software lab as a software developer, in systems test and as customer support for the CICS Early Support Program. He holds a degree in Mathematics and Economics from Surrey University.

Thanks to the following people for their contributions to this project:

Phil Hanson and Andy Bates of IBM Hursley, and Chris Rayns of the ITSO Poughkeepsie, for their support with this project.

Gary Flood and Pete Masters of the IBM Hursley Lab, for explaining the workings of the CICS Transaction Gateway V6.1 XA transaction support.

Patrick Kappeler of IBM Montpellier, and Andrew Smithson of IBM Hursley for their help with the SSL test scenarios.

Philippe Richard of IBM Montpellier for providing excellent systems support.

Phil Wakelin, John Joro, Kevin Kinney and David Seager, who were the authors of the IBM Redbook CICS Transaction Gateway V5 The WebSphere Connector for CICS, SG24-6133.

The following people for the significant amount of time that they have spent reviewing and for their detailed review comments:

� Mike Cox of the Washington System Center� Mitch Johnson of IBM zSeries Services� Sylvie Lemariey of IBM Montpellier� Daniel McGinnes of IBM Hursley� Alan Roessle of IBM Montpellier� Steve Wall of IBM Montpelier� Adrian Warman of IBM Hursley

Preface xv

Become a published authorJoin us for a two- to six-week residency program! Help write an IBM Redbook dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. You'll team with IBM technical professionals, Business Partners, or customers.

Your efforts will help increase product acceptance and customer satisfaction. As a bonus, you'll develop a network of contacts in IBM development labs, and increase your productivity and marketability.

To learn more about the residency program, browse the residency index, and apply online at:

ibm.com/redbooks/residencies.html

Comments welcomeYour comments are important to us!

We want our Redbooks to be as helpful as possible. Send us your comments about this or other Redbooks in one of the following ways:

� Use the online Contact us review redbook form found at:

ibm.com/redbooks

� Send your comments in an e-mail to:

[email protected]

� Mail your comments to:

IBM Corporation, International Technical Support OrganizationDept. HYTD Mail Station P0992455 South RoadPoughkeepsie, NY 12601-5400

xvi CICS Transaction Gateway for z/OS Version 6.1

http://www.redbooks.ibm.com/residencies.html

http://www.redbooks.ibm.com/residencies.html



http://www.redbooks.ibm.com/contacts.html

Part 1 Introduction

In this part, we give a broad overview of the CICS Transaction Gateway. We outline the major CICS TG components and describe the different topologies in which you can use the CICS TG. We also describe how we installed and configured the CICS TG on our system and how we tested the installation.

Part 1

© Copyright IBM Corp. 2006. All rights reserved. 1

2 CICS Transaction Gateway for z/OS Version 6.1

Chapter 1. CICS Transaction Gateway

The CICS Transaction Gateway (CICS TG) is a market-leading connector that provides a high performing, secure, scalable, and tightly integrated method of e-business access to CICS. It enables CICS customers to expand the value of their investment in classic CICS applications and usually requires no changes to existing CICS applications. It is easy to install and has flexible configuration options that require minimal changes to CICS. It provides a range of networking options and provides a choice of Java™ and non-Java client programming interfaces.

In this chapter, we introduce the CICS TG components, the application programming interfaces (APIs) that it provides. We also discuss the J2EE Connector Architecture (JCA), the CICS JCA resource adapters, and the topologies in which you can deploy the CICS TG.

1


1.1 What is CICS Transaction GatewayThe CICS TG is a set of client and server software components that allow a remote client application to invoke services in a CICS region. The client application can be either a Java application or a non-Java application using either C, C++, COBOL or COM interfaces (depending on the platform used).

When a Java application is used, then the application can be any type of client (such as an applet, a servlet, or an enterprise bean). However, in the J2EE environment, the application is typically a servlet or enterprise bean that is deployed into a J2EE application server such as WebSphere Application Server.

1.1.1 CICS TG productsWith CICS TG V6 and later there are now the following two distinct orderable CICS TG products:

� CICS TG for Multiplatforms� CICS TG on z/OS

CICS TG for Multiplatforms V6.0CICS TG for Multiplatforms V6.0 is supported on the following range of operating systems and platforms and is designed to support connectivity to all in-service CICS servers:

� Linux on zSeries � Linux on Intel� Linux on POWER™� AIX� HP-UX (on PA-RISC)� Sun Solaris (on SPARC)� Windows XP, Windows 2000, and Windows 2003

This version was announced on 30 November 2004 in a U.S. announcement letter 204-2284, which is available at:

http://www-306.ibm.com/fcgi-bin/common/ssi/ssialias?infotype=an&subtype=ca&supplier=897&letternum=ENUS204-284

CICS TG for Multiplatforms is comprised of the following main runtime components:

� The Gateway daemon, which listens for incoming work and manages the threads and connections necessary to ensure good performance.

� The Client daemon, which provides the communication to CICS servers and the non-Java APIs.



� A Java class library or JCA resource adapter, which is deployed into the client runtime environment. When used in a JCA environment the resource adapter is deployed into the J2EE application server.

A Java client program can connect to a remote Gateway daemon using the TCP or SSL protocols. The Client daemon then provides the transport drivers to connect to the CICS server, as shown in Figure 1-1. Note that because the non-Java APIs are provided by the Client daemon, there is no remote connectivity support for non-Java clients.

Figure 1-1 Components of CICS TG for Multiplatforms

CICS TG for Multiplatforms V6.0 provides the following improvements:

� Runtime performance:

– The runtime performance of the Client daemon has been enhanced by improvements in the internal tracing mechanism.

– Support for the Native Posix Thread Library (NPTL) threading model is provided for the RHEL V3 and SLES 9 Linux distributions, providing for improved performance and scalability for multi-threaded applications.

Note: The CICS Universal Client V6, which provides the Client daemon functions within CICS TG, is available as a separate product on the AIX, Linux, and Windows platforms. CICS Universal Client provides the transport protocols and non-Java programming interfaces for single user access to CICS.

CICS TG

Client daemon

Gateway daemon

CICSServer

Protocol handler APPC

TCP62 TCP/IP

TCPor SSL

z/OS, OS/390VSE, TxSeries

Distributed platform

ctgjni.dll

JNI module

Java

Clientctgclient.jarcicseci.rarcicseciXA.rarcicsepi.rar

ctgserver.jar

Chapter 1. CICS Transaction Gateway 5

� Systems management:

– A normal shutdown mechanism has been introduced allowing the Gateway daemon to be terminated in either a controlled or immediate manner.

– A command-based systems administration tool (ctgadmin) provides improved usability and the added capability of invoking shutdown processing.

– Support for a system daemon process and a new daemon process (ctgd) in UNIX® and Linux environments.

– APPC network connections to CICS servers are now supported for Linux on zSeries using the facilities of IBM Communications Server.

– Management of Gateway daemon log files is now possible, allowing a maximum file size and number of log archives to be specified.

� Security

A new configuration option for SSL connections allows the negotiated SSL cipher suite to be controlled, providing for increased control of security levels when using SSL connections.

� Installation

Installation and migration is simplified using Install Shield Multiplatform (ISMP) on all platforms.

� Java

J2EE Connector Architecture (JCA) 1.5 supports provides for enhancements in the management of connections, security, and transactions in supported J2EE V1.4 application servers.

CICS TG for z/OS V6.1CICS TG for z/OS V6.1 is the latest version of the z/OS product. It is supported on z/OS V1R4 and later and supports connectivity to CICS TS for z/OS V1.3, V2.2, V2.3, and V3.1.

The product was announced on 04 October 2005 in a U.S. announcement letter 205-248, which is available at:





CICS TG for z/OS uses the external communication interface (EXCI) interface provided by CICS TS to communicate with CICS (Figure 1-2). It does not include the Client daemon and does not provide any support for non-Java based applications because this support is provided through the CICS EXCI interface.

Figure 1-2 Components of CICS TG for z/OS

CICS TG for z/OS V6.1 has many improvements principally in the area of systems management, usability, and performance as follows:

� XA transaction support enables CICS Transaction Server (CICS TS) for z/OS to participate in a global two-phase commit transaction that is initiated in a distributed J2EE V1.4 application server, such as WebSphere Application Server V6.0.

� Management of the Gateway daemon from SDSF provides better system administration capabilities, including a new normal shutdown mechanism.

� Direction of output to JES provides improved management for runtime messages.

� An option to limit the number of EXCI pipes to one per thread improves availability and scalability.

� Security Enhancements include the following:

– Storage of SSL private keys and certificates within the RACF database.

– Improved support for hardware cryptography cards when using SSL.

– A new external configuration option for SSL encryption allowing the SSL cipher suite to be specified.

CICS TG

EXCI

JavaClient

Gateway daemon

CICS TSProtocol handler

MRO

TCPor SSL

z/OS

libctgjni.so

JNI module

IRC

Java

Clientctgserver.jar

ctgclient.jarcicseci.rarcicseciXA.rar


� Connection enhancements:

– A load-balancing feature for XA transactions that enables a group of Gateways to act as one virtual endpoint for incoming TCP/IP requests, helping to maximize scalability and availability.

– An automatic TCP/IP subsystem reconnect feature that allows new TCP/IP connections to be made to the Gateway daemon after a restart of the TCP/IP subsystem. This leads to improved availability and less operator intervention.

– An IP address binding feature that allows administrators to control which network interfaces can be used by the Gateway daemon, providing improved control of security and ease of network management.

– A socket connection timeout feature that provides the option to specify a timeout duration on pending socket connection requests from the J2EE application to a remote Gateway daemon, giving improved recovery in the TCP/IP network.

� SMP/E packaging and installation, which simplifies the process of installing, migrating, and applying corrective maintenance to the CICS TG.

1.1.2 CICS TG application programming interfacesCICS TG for Multiplatforms provides the following programming interfaces for accessing CICS applications:

� External Call Interface (ECI)� External Presentation Interface (EPI)� External Security Interface (ESI)

External Call InterfaceThe ECI is used for calling COMMAREA-based CICS programs. The COMMAREA is the buffer that is used for passing the data between the client and the CICS server. CICS sees the client request as a distributed program link (DPL) request.

The ECI enables a user application to call a CICS program synchronously or asynchronously. It enables the design of new applications to be optimized for client-server operation, with the business logic on the server and the presentation logic on the client.

Note: The CICS TG on z/OS supports only the ECI, because of its reliance on the CICS EXCI function that provides the call interface and connectivity between the Gateway daemon and the CICS TS region.


An ECI request can be invoked from a Java application using a variety of different interfaces:

� The ECIRequest class that is provided by the CICS TG base classes.

This interface provides a simple procedural type interface to the ECI. It is supported in any Java environment (such as an stand-alone application) and provides similar capabilities to the JCA. However, it does not provide the same qualities of service (such as XA transaction support).

� The Common Client Interface (CCI) that is provided by the CICS ECI resource adapters (cicseci.rar or cicseciXA.rar).

These classes define a standard architecture for connecting the Java 2 Platform Enterprise Edition (J2EE) platform to a heterogeneous EIS such as CICS. Java applications interact with resource adapters using the Common Client Interface (CCI), which is a common framework of classes extended by each resource adapter to allow communication with a specific EIS.

External Presentation InterfaceThe EPI is used for invoking 3270-based transactions. A terminal is installed in CICS, and CICS sees the request as running on a remote terminal controlled by the CICS TG.

An EPI request can be invoked from a Java application using one of three different interfaces:

� The EPIRequest class provided by the CICS TG base classes.

This class provides a Java interface to the EPI, and is used for invoking 3270-based transactions. Due to its low-level nature, using it for developing EPI applications requires a strong knowledge of CICS and 3270 data streams.

CCF: Before the existence of the JCA, IBM recognized a need for a common way to connect to EIS systems and introduced the Common Connector Framework (CCF) through the IBM VisualAge® for Java product. CCF was similar in concept to JCA. However, CCF was not an open specification and did not provide the same support for system contracts and qualities of service (such as transaction support and connection pooling).

CICS TG V6.1 no longer provides support for the CCF. However, a SupportPac™ (CE52) is available that provides runtime support for CCF using a CICS TG V6 Java client. It is available at:

http://www.ibm.com/software/htp/cics/ctg/support



� The EPI support classes, which provide high-level constructs for handling 3270 data streams.

A wide range of classes is provided including AID, FieldData, Screen, Terminal, Map and MapData. These are used to represent the interface to a CICS 3270 terminal, and the resulting 3270 response.

� The Common Client Interface (CCI) provided by the CICS EPI resource adapter (cicsepi.rar).

EPI is typically used when it is not possible to separate the presentation logic from the business logic of an application, and allows the reformatting of the user interface, without modification of the CICS application. For a more detailed discussion on the benefits of separating business logic from presentation logic refer to Revealed! Architecting e-business Access to CICS, SG24-5466.

External Security InterfaceThe ESI is used for verifying and changing the user ID and password information held in the CICS external security manager (ESM), such as RACF. It is based on the CICS Password Expiration Management (PEM) function.

ESI calls to CICS can only be made using the ESIRequest class that is provided by the CICS TG base classes. This class provides a Java interface to the ESI, and provides two simple PEM requester functions:

Verify Password Allows a client application to verify that a password matches the password for a given user ID that is stored by the CICS ESM.

Change Password Allows a client application to change the password that is held by the CICS ESM for a given user ID.

There is no other interface available for the ESI, although both the EPI and ECI allow user IDs and passwords to be flowed within the actual requests. In this case the user ID and password is authenticated either within CICS, if using a Multiplatform CICS TG, or within the CICS TG, if using the CICS TG for z/OS.

1.2 CICS TG for z/OS modes of operationThis redbook focuses on installing and configuring the CICS TG for z/OS for two different modes of operation: remote and local.

1.2.1 Remote mode of operationThe remote mode of operation uses the Gateway daemon as a long running task which listens on specified ports for incoming ECI requests and then forwards


them to the CICS server via the EXCI protocol. The Gateway deamon runs in its own MVS™ address space and provides connection and thread management as shown in Figure 1-1 on page 5.

1.2.2 Local mode of operationIf the CICS TG is to be used on the same machine as WebSphere Application Server, it is more efficient to use the CICS TG classes within WebSphere Application Server to provide the gateway functionality (Figure 1-3). This mode of operation allows WebSphere Application server to manage the connections and threads and reduces the communications overhead. This configuration is known as the local mode of operation.

Figure 1-3 CICS TG in local mode with WebSphere Application Server

There is further discussion of the local and remote modes of operation in “Using the JCA with different CICS TG topologies” on page 15.

1.3 JCAThe J2EE connector architecture (JCA) defines a standard for connecting the Java 2 Platform, Enterprise Edition (J2EE) to heterogeneous Enterprise Information Systems (EIS) such as CICS. The architecture defines a set of scalable, secure, and transactional mechanisms that enable the integration of EISs with application servers and enterprise applications. A proprietary J2EE application server, such as IBM WebSphere Application Server, can be extended to support the resource adapter architecture and is then assured of a seamless

WebSphere ApplicationServer

HTTPJava

Client

EXCI or Client daemon

CICS TG

Resource adapter

Z/OS orDistributed

ServletJSP

EJB

CCI

CICSServer


connectivity to multiple EISs. Likewise, an EIS vendor provides one standard resource adapter with the capability to plug into any application server that supports the connector architecture.

1.3.1 Overview of JCAExample 1-4 shows the key parts of the JCA: the deployable components (resource adapters), the programming interface they implement (CCI), and the qualities of service that they implement (system contracts).

Figure 1-4 J2EE Connector Architecture (JCA)

Resource adaptersA resource adapter is a system-level software driver that a Java application uses to connect to an EIS. A resource adapter is installed into an application server and provides connectivity between the EIS, the application server, and the enterprise application.

CCIThe Common Client Interface (CCI) defines a standard client API for application components to access multiple resource adapters. This API can be used directly, or enterprise application integration frameworks can be used to generate EIS access code for the developer. The CCI is designed to be an EIS independent API, such that an enterprise application development tool can produce code for any J2EE compliant resource adapter that implements the CCI interface.

J2EE Server(e.g WebSphere Application Server)

Application Component (e.g EJB)

Resource Adapter(e.g CICS ECI resource adapter)

Enterprise Information System (e.g CICS)

System Contracts

Container-ComponentContract

ƒ Connection Managementƒ TransactionManagementƒ SecurityManagement

Common Client Interface (CCI)

EIS SpecificInterface

ConnectionFactory cf =<JNDI lookup>

Connection connection =cf.getConnection();

Interaction interaction

=connection.createInteraction();

interaction.execute(<input and output data>);

interaction.close();

connection.close();


The CCI has the following additional characteristics:

� It forms a base level API for EIS access on which higher level functionality specific to an EIS can be built.

� It provides an API that is consistent with other APIs in the J2EE platform, such as JDBC™.

� It is targeted primarily towards application development tools and enterprise application integration frameworks, rather than Java developers using the CCI API directly.

� It is an optional part of the JCA specification.

System contractsThe Connector architecture defines a standard set of system-level contracts between a J2EE server and a resource adapter.The standard contracts are as follows:

� A connection management contract that lets a J2EE server pool connections to an underlying EIS, and lets application components connect to an EIS. This leads to a scalable application environment that can support a large number of clients requiring access to EIS systems.

� A transaction management contract between the transaction manager and an EIS that supports transactional access to EIS resource managers. This contract lets a J2EE server use a transaction manager to manage transactions across multiple resource managers. This contract also supports transactions that are managed internal to an EIS resource manager without the necessity of involving an external transaction manager.

� A security contract that enables secure access to an EIS. This contract provides support for a secure application environment, which reduces security threats to the EIS and protects valuable information resources managed by the EIS.

These system contracts are transparent to the application developer, meaning they do not have to implement these services themselves.

JCA Version 1.5In addition to the standard connection, transaction and security system contracts, JCA Version 1.5 provides four additional system contracts:

� Life cycle management, allowing an application server to control the startup and termination of a resource adapter. This is not supported by the CICS TG or by WebSphere Application Server.


� Work management, allowing a resource adapter to submit work instances to an application server for execution. This is not supported by the CICS TG or by WebSphere Application Server.

� Message inflow, allowing a resource adapter to asynchronously deliver messages to message endpoints residing in the application server

� Transaction inflow, allowing a resource adapter to import a transaction context from the EIS system into an application server.

JCA 1.5 also provides two optimizations to the transaction management and connection management contracts, which are exploited in the JCA 1.5 resource adapters provided by CICS TG V6.0 and V6.1:

� Lazy connection association

This contract provides for potential improved connection pooling in J2EE application servers. It allows the resource adapter to disassociate a cached connection handle from the managed connection once the connection handle is no longer being used by the J2EE component. This allows J2EE applications to use the get-use-cache model for connection handles, and for the underlying connections to still be efficiently pooled by the Connection Pool manager in WebSphere Application Server.

� Lazy transaction enlistment

This contract optimizes transaction enlistment calls so that the resource adapter only participates in a transaction if it is actually invoked, rather than being enlisted whenever a reference exists within the J2EE component. This provides for potential better performance if transactional JCA applications are used in an J2EE application server.

1.3.2 CICS resource adapters The following resource adapters are provided for use with the CICS TG for Multiplatforms. The resource adapter archives (RAR files) are supplied in the <install_path>/deployable directory.

cicseci.rar The CICS ECI resource adapter, which provides one-phase transactional support.

cicsepi.rar The CICS EPI resource adapter, which is non-transactional.

Note: The CICS resource adapters do not exploit the new JCA V1.5 system contracts currently.


The following resource adapters are provided for use with the CICS TG for z/OS. The resource adapter archives (RAR files) are supplied in the <install_path>/deployable directory.

cicseci.rar The CICS ECI resource adapter, which provides one-phase transaction support when installed into WebSphere Application Server on a distributed platform, and two-phase transactional support when deployed into WebSphere Application Server for z/OS.

cicseciXA.rar The CICS ECI XA resource adapter, which provides two-phase transactional support when deployed into WebSphere Application Server on any supported platform.

1.4 Using the JCA with different CICS TG topologiesThe qualities of service provided by the JCA vary, depending on the topology in use. The most common topologies in use today are shown in Figure 1-5 and are as follows:

� Topology 1. WebSphere Application Server and the CICS Transaction Gateway are both deployed on a distributed (non-zSeries) platform.

� Topology 2. WebSphere Application Server is deployed on a distributed platform and CICS Transaction Gateway is deployed on a z/OS system.

� Topology 3. Both WebSphere Application Server and CICS Transaction Gateway are deployed on zSeries servers.


Figure 1-5 JCA deployment topologies

1.4.1 WebSphere Application Server and CICS TG on distributed In this topology both WebSphere Application Server and CICS TG are deployed on one of the distributed platforms such as AIX or Linux (Figure 1-6).

Figure 1-6 WebSphere Application Server and CICS TG on distributed platform

The Gateway daemon is not required in this configuration because the CICS TG is used in local mode to invoke the Client daemon native libraries directly from the J2EE enterprise bean. The Client daemon then flows the ECI request to the CICS server using either TCP/IP, a Systems Network Architecture (SNA) connection or a TCP62 connection. The management of connections,

CICSNetwork

HTML

WebSphereand

CICS TG on zSeries

WebSphereServer

andCICS TG

WebSphereApplication

Server

CICS TGz/OS

Topology 1

Topology 3

Topology 2 zSeries

SNA orTCP62 or

TCP/IP

CICS TS

COBOL application

COMMAREA

HTML

WebSphereApplication Server V6

ServletJSP

EJB

CICS TG V6.0

CICS ECI resource adapterCCI

Distributed platform z/OS

Clientdaemon


transaction and security is controlled within WebSphere Application Server through a combination of both configuration parameters and application deployment descriptors.

The specific qualities of service (in terms of the JCA system contracts) that apply to this topology are as follows.

� Connection pooling

Pooling of connections is provided seamlessly by the pool manager component of WebSphere Application Server allowing the reuse of connections to the resource adapter between multiple J2EE components.

� Transaction management

The CICS ECI resource adapter (cicseci.rar) only supports the LocalTransaction interface. As a result, the scope of the transaction is limited to the Resource Manager (that is, the associated connection factory and the specified CICS server). Such Resource Manager LocalTransactions only support one-phase commit processing.

However, if the J2EE application server supports the JCA option of last-resource commit optimization, an ECI interaction can participate in a global transaction, provided that it is the only one-phase-commit resource in the global transaction. This function is provided by the last-participant support in WebSphere Application Server V6.0 and WebSphere Application Server Enterprise Edition V5.0.

� Security management

Security credentials (user ID and password) propagated through to CICS from WebSphere Application Server can be determined by the application (component-managed) or by the Web or EJB™ container (container-managed). Container-managed sign-on is recommended because it is good practice to separate the business logic of an application from qualities of service, such as security and transactions. In this topology, however, the principal means of enabling container-managed authentication is by specifying the user ID and password in a JCA authentication entry (also

Note: The TCP62 protocol allows SNA flows to be encapsulated in TCP/IP packets and works in conjunction with the Anynet LU6.2/IP support in VTAM.

Note: The TCP/IP or SNA network connections from the Client daemon into the CICS region are managed and re-used by the Client daemon component of the CICS TG and are not subject to the JCA connection pooling system contract.


known as an alias) and associating the alias with the resource reference when the J2EE application is deployed.

1.4.2 WebSphere Application Server on distributed, CICS TG on z/OSWhen WebSphere Application Server is deployed on one of the distributed platforms, it is possible to access CICS through a Gateway daemon running on z/OS (Figure 1-7).

Figure 1-7 WebSphere Application Server on distributed and CICS TG on z/OS

The protocol specified in the connection settings of the connection factory is one of the remote protocols (TCP or SSL). The communication from the Gateway daemon on z/OS to the CICS region uses the EXCI.

This topology enables integration with z/OS internet protocol (IP) workload-management functions, including Sysplex Distributor and TCP/IP port sharing. Use of these technologies allows an individual Gateway daemon to be removed as a single point of failure and enables incoming work to be efficiently balanced across multiple Gateway daemons in multiple z/OS logical partitions (LPARs).

The specific JCA qualities of service that apply to this topology are as follows:


The connection pool represents physical network connections between WebSphere Application Server and the Gateway daemon on z/OS. In such a configuration, it is essential to have an efficient connection-pooling

Distributed

COBOLApplication

CICS TS V3.1

z/OS

EXCI

JNI

GatewayDaemon

TCPSSL

CICS TG V6.1HTML


ServletJSP

EJBCICS ECI or CICS ECI XAresource adapter

CCI

CICS TG V6.1


mechanism because otherwise, a significant proportion of the time from making the connection to receiving the result from CICS Transaction Server and closing the connection can be in the creation and destruction of the connection itself. The JCA connection-pooling mechanism mitigates this overhead by allowing connections to be pooled by the WebSphere Application Server pool manager.

For information about configuring connections with this topology see Chapter 3, “Gateway daemon connections” on page 77.


In this topology, two-phase commit global transactions are now supported, using the CICS ECI XA resource adapter. Also, one-phase commit transactions using the CICS ECI resource adapter are still supported. Note that in either case, if the enterprise bean is deployed with a transactional deployment descriptor (for example, a value of REQUIRED), the resulting ECI request to the CICS region uses transactional EXCI. In this case, both the Gateway daemon and the CICS region need to be configured to register with Resource Recovery Services (RRS).

For information about configuring transactions with this topology, see Chapter 5, “Gateway daemon transactions” on page 157 and Chapter 6, “Gateway daemon transactions with TCP/IP load balancing” on page 225.


In this configuration, the Gateway daemon is the entry point to the zSeries system in which your CICS system is running, so it is normal for the Gateway daemon to perform an authentication check for incoming ECI requests from clients. However, after the user has authenticated to WebSphere Application Server, a password might not be available to send to the Gateway daemon. In this case, therefore, you need to devise a way to establish a trust relationship between WebSphere Application Server and the Gateway daemon so that WebSphere Application Server can be trusted to flow only the user ID on the request through to CICS Transaction Gateway. Solutions such as SSL client authentication and virtual private networks (VPNs) can be used to establish such a trust relationship.

For information about configuring security with this topology, see Chapter 8, “Gateway daemon security” on page 281 and Chapter 9, “Enabling SSL with the Gateway daemon” on page 309.

1.4.3 WebSphere Application Server and CICS TG on zSeriesIn a zSeries topology, WebSphere Application Server can be deployed on either a z/OS system or on a Linux operating system. The qualities of service differ between these two topologies and are therefore discussed separately.


WebSphere Application Server and CICS TG on z/OSIn this topology (Figure 1-8), only the CICS ECI resource adapters are supported. The most common z/OS configuration uses the local mode of operation. This results in a direct cross-memory EXCI connection between WebSphere Application Server and CICS. Figure 1-8 shows an application deployed to WebSphere Application Server for z/OS.

Figure 1-8 WebSphere Application Server and CICS TG on z/OS

CICS TG V5.01 introduced remote connection support for this topology, however, the highest qualities of service can be achieved when a local connection is used.

The specific JCA qualities of service that apply to this topology are as follows:


The connection pool is a set of connection objects managed by WebSphere Application Server, which is not directly associated with the EXCI pipes used for communication to CICS.

For information about configuring connections with this topology, see Chapter 4, “Connecting from WebSphere Application Server for z/OS” on page 121.


A two-phase commit capability is provided through RRS, an integral part of z/OS. RRS acts as the external transaction coordinator for z/OS in managing the transaction scope between WebSphere Application Server, CICS and other z/OS subsystems, including IMS™, DB2®, and WebSphere MQ.

For information about configuring transactions with this topology, see Chapter 7, “Transactions with WebSphere Application Server for z/OS” on page 253.

CICS TS V3.1

COBOL application

HTML


ServletJSP

EJB

COMMAREA

CICS TG V6.1

CICS ECI or CICS ECI XAresource adapter

CCI

z/OS

EXCI



Both container-managed and component-managed sign-on are supported. In this topology, the ECI resource adapters flow only the user ID to CICS with the ECI request - it is assumed that the user is already authenticated by WebSphere Application Server. When using container-managed sign-on, a z/OS specific functionality known as thread identity support is provided by WebSphere Application Server for z/OS.

For information about configuring security with this topology, see Chapter 10, “Security with WebSphere Application Server for z/OS” on page 339.

CICS TG for Linux on zSeriesWebSphere Application Server and CICS TG deployed within Linux on zSeries (Figure 1-9) provides a flexible and scalable environment based on the virtualization capabilities of IBM z/VM® and Linux systems.

Figure 1-9 WebSphere Application Server and CICS TG for Linux on zSeries

The JCA qualities of service for this topology are almost identical to those described in 1.4.1, “WebSphere Application Server and CICS TG on distributed” on page 16 because Linux on zSeries (within a JCA and CICS TG scenario) can be treated as a distributed platform. A significant exception to this generalization is that the HiperSockets™ function can be used to provide a highly efficient cross-memory transport for TCP/IP based communication into CICS (using the ECI over TCP/IP function of CICS TS V2.2 and later releases). Alternatively, an APPC connection to CICS TS can be provided by IBM Communications Server for Linux on zSeries.

APPC,TCP62 or

TCP/IP

CICS TS

COBOL application

COMMAREA

HTML


ServletJSP

EJBCICS TG V6.0

CICS ECIresource adapterCCI

Linux on zSeries z/OS

Clientdaemon

zSeries

Comms Server or

HiperSockets


1.4.4 Mixed version support The client-server nature of the CICS TG means that different levels of the Gateway daemon and the Java client (including the resource adapter) can interoperate. This provides a degree of flexibility in deployment, allowing software components to be updated at different times. However, because of the defined internal flow levels and of the JNI interface, you should follow these rules should to ensure interoperability:

1. Remote Java clients connecting to a Gateway daemon, must use a CICS resource adapter of the same (or lower) level CICS TG version as the Gateway daemon.

2. If the CICS TG is used in local mode within WebSphere Application Server, then the Java client and the CICS TG JNI library must be at the same CICS TG level.

Full details of the latest supported software combinations are provided on the ibm.com® support pages for the CICS TG available at:


Here are some examples of supported configurations:

� Using a JCA 1.5 resource adapter (from CICS TG V6.1 or CICS TG V6.0) deployed in WebSphere Application Server V6 to connect to a CICS TG V6.1 Gateway daemon is supported.

� Using a JCA 1.0 resource adapter (from CICS TG v5.1) deployed in WebSphere Application Server v5.1 to connect to CICS TG V6.1 Gateway daemon is supported.

� Using a JCA 1.0 resource adapter (from CICS TG v5.1) deployed in WebSphere Application Server V6 to connect to a CICS TG V5.1 Gateway daemon is supported.



Here are some examples of non-supported configurations:

� Using a JCA 1.5 resource adapter (from CICS TG V6.1 or CICS TG V6.0) deployed in WebSphere Application Server v5 is not supported.

� Using a JCA 1.5 resource adapter (from CICS TG V6.1 or CICS TG V6.0) deployed in WebSphere Application Server V6 to connect to a CICS TG V5.1 daemon is not supported.

� Using a CICS TG V5.1 JNI library for use in local mode with a JCA 1.5 resource adapter (from CICS TG V6.0) with WebSphere Application Server V6.0 is not supported.

Note: Do not mix the levels of CICS resource adapters that are deployed on a WebSphere Application Server node. For example, do not install a resource adapter from CICS TG V6.1 to a node that has a V6.0 ECI or EPI resource adapter installed. The resource adapter version is shown in the ra.xml file in the resource adapter.


Chapter 2. Installing and configuring the CICS TG

In this chapter, we describe how to install and configure the CICS Transaction Gateway for z/OS V6.1 and how to test the installation with one of the sample Java applications that is supplied with the CICS TG.

2


2.1 Preparing for the installationThe following sections detail the software levels that we use in our configuration and include instructions on how to install the CICS TG for z/OS.

Figure 2-1 Software components: CICS TG for z/OS

The CICS TG for z/OS runs in the z/OS UNIX System Services (USS) environment. You need a mix of traditional MVS skills and UNIX skills in order to install and to configure the CICS TG.

2.1.1 Software checklistWe use the following software levels:

� z/OS V1R6� CICS Transaction Gateway for z/OS V6.1� IBM SDK for z/OS, Java 2 Technology Edition, V1.4.2 SR2� CICS Transaction Server for z/OS V3.1

The minimum levels that are supported with CICS TG V6.1 are:

� z/OS V1R4� IBM SDK for z/OS, Java 2 Technology Edition, V1.4.2 SR2� CICS Transaction Server for z/OS V1.3

CICS TS region

EC01Gatewaydaemon

z/OS

EXCI

tx1.mop.ibm.com

CICS TG

UNIX System Services

Java application

EciB1


2.1.2 Definitions checklistTable 2-1 summarizes the definitions that we use in this scenario. Before you configure the products, we recommend that you acquire definitions for each value that is listed in this table.

Table 2-1 Definitions checklist

2.2 Installing CICS TGDuring the SMP/E installation of CICS TG, the HFS data set OMVS.CTG61.HFS that contains the CICS TG product files was created and mounted at the mount point /usr/lpp/cicstg/ctg610 (known as the <install_path>). For more information about the SMP/E installation, see the Program Directory for CICS Transaction Gateway for z/OS, GI10-2587.

To install the product, you need to perform the following steps:

1. Run the ctginstall script.

2. Consider basic security setup.

Value Gateway daemon CICS TS

hostname tx1.mop.ibm.com -

IP address 9.100.193.122 -

TCP/IP port 13101 -

Job name CITGR1G CITGR1CI

APPLID - A6POTGR1

NETNAME CITGR1G -

CONNECTION - GR1G

Chapter 2. Installing and configuring the CICS TG 27

2.2.1 Running the ctginstall scriptAfter the SMP/E installation, you must run the ctginstall script by executing the ctginstall command in OMVS (ctginstall 40 can be used to limit the screen width if required). When you run the script, you must agree to the licensing agreement, as shown in Example 2-1.

Example 2-1 Partial listing of the CICS TG license agreement

CITGTJ: /usr/lpp/cicstg/ctg610>ctginstall To install CICS Transaction Gateway you must accept the following license agreement. If the license file does not display correctly, try restarting theinstallation using the command: '/usr/lpp/cicstg/ctg610/bin/ctgsetup 40'. Enter 1 to view the license, or 2 to quit.

1 >>> Beginning of the License File >>> International Program License Agreement Part 1 - General Terms BY DOWNLOADING, INSTALLING, COPYING, ACCESSING, OR USING THE PROGRAM YOU AGREE TO THE TERMS OF THIS AGREEMENT. IF YOU ARE ACCEPTING THESE TERMS ON BEHALF OF ANOTHER PERSON OR A COMPANY OR OTHER LEGAL ENTITY, YOU REPRESENT AND WARRANT... _______________________________________________________________________________ Enter 1 to page down, 2 to page up, 3 to accept or 4 to decline.

3 You have accepted the license agreement. Wait while the installation continues. The installation is complete. License files can be viewed using the command: '/usr/lpp/cicstg/ctg610/bin/ctgbrowse' CITGTJ: /usr/lpp/cicstg/ctg610>

Tip: When running the ctginstall script, you might see INPUT in the lower right-hand corner of your screen which means that OMVS has put your terminal to sleep. Press PF10 to refresh the screen.


Figure 2-2 shows the resulting directory structure that follows the installation of CICS TG.

Figure 2-2 CICS TG z/OS directory structure

The /usr/lpp/cicstg/ctg610/bin directory contains the following files which are referred to later in this chapter:

ctgsamp.ini Sample CICS TG configuration file.ctgenvvarsamp Sampled ctgenvvar script.ctgstart Shell script to start the Gateway daemon.

You can browse files using TSO OBROWSE or ISHELL.

Note: All the directories also contain an IBM directory that contains files with names in the form CTGnnnnn, where nnnnn are digits. These directories and files are part of the SMP/E installation. You should not alter or remove them.

bin

/usr/lpp/cicstg/ctg610

resource

classes

docs

Installation directory

executables

ctgcfg resources

CTG Java class library

CTG documentation

ccf 2. j arci csj 2ee. j ar conf t ool . j ar connect or . j ar ct gcl i ent . j ar ct gsampl es. j arct gser ver . j ar mask. j ar psk. j ar t gui de. j ar

samples

java

com

ibm

ctg

eci

j2ee

Security exit samples

ECI samples

serverCICS COBOL source code

samples

security

J2EE samples

ctgzoscfg non-z/OS config tool

ct gar m ct gasi ct gbr owsect gcf g ct gconvenv ct genvvar samp ct gl og. hl p ct gmast er ct gmsg. hl p ct gmsgs

ct gsamp. i ni ct gset up ct gset up_I BM- 930ct gset up_I BM- 935ct gset up_I BM- 939ct gst ar t l i bct gj ni . so l i bct gj ni _g. so l i bct gmvs. so t est z os

deployable J2EE RAR files


The /usr/lpp/cicstg/classes directory contains the following JAR files:

� ctgclient.jar

Java class library

� ctgserver.jar

Classes for use by Gateway daemon

� ctgsamples.jar

Samples

� cicsj2ee.jar, ccf2.jar, connector.jar

J2EE classes

� mask.jar, psk.jar, guide.jar

Classes used by the configuration tool

2.2.2 Basic security considerationsYou must perform the following basic RACF security tasks before starting the Gateway daemon:

1. Set up a user ID for the Gateway daemon started task.

2. Allow the Gateway daemon user ID read access to program-controlled libraries.

3. Define programs and libraries as program-controlled.

These tasks enable basic RACF security permissions. For information about configuring a more secure environment, see Chapter 8, “Gateway daemon security” on page 281.

Setting up a user ID for the Gateway daemon started taskThe user ID under which the Gateway daemon started task runs should:

� Have an OMVS segment defined.

� Be in a group that has an OMVS segment.

� Be defined without a password.

� Have READ access to the RACF profile that protects the TCPIP.STANDARD.TCPXLBIN data set.

Tip: The contents of a JAR or zipped file can be listed with the USS command in OMVS:

jar -tvf <file>


We ran our Gateway daemon as a started task under user ID CITGR1G.

There are two methods of defining a default user ID to a started task:

� Use the RACF exit - ICHRIN03.

� Use the STARTED class in RACF.

The following example shows the user ID CITGR1G in the CICS group being assigned as the user ID for the CITGR1G task:

RDEF STARTED CITGR1G* STDATA(USER(CITGR1G) PRIVILEGED(NO) TRUSTED(NO))

Allow Gateway daemon access to program-controlled MVS librariesIf the z/OS system has the BPX.SERVER FACILITY class profile defined within RACF, then the user ID under which the Gateway daemon runs must be permitted to this profile. The following example shows the PERMIT command that we issue for our CITGR1G Gateway daemon user ID:

PERMIT BPX.SERVER CLASS(FACILITY) ID(CITGR1G) ACCESS(READ)PERMIT BPX.FILEATTR.PROGCTL CLASS(FACILITY) ID(CITGR1G) ACCESS(READ)

If the BPX.SERVER FACILITY class is not defined, the Gateway daemon user ID must be defined with a UID of 0 (that is, be a root user).

Defining programs and libraries as program controlledUSS program control allows RACF to secure USS executables as though they were MVS programs. If a nonprogram controlled program is loaded into the Gateway daemon, the address space becomes dirty, and program control is lost, which results in security failures in the Gateway daemon.

After the SMP/E installation of CICS TG, we noted that the HFS files which need to be program controlled had been marked with the extended attribute +p.

Tip: The ls -E command from an OMVS screen shows the extended attributes of HFS file, including the program control attribute p.


Example 2-2 Results of the ls -E command showing the extended attributes of ctgstart

CITGCA: /usr/lpp/cicstg/ctg610/bin>ls -E total 6200 drwxrwxr-x 2 CITGT2G CTGV61 8192 Sep 17 13:05 IBM .-rwxrwxr-x --s- 2 CITGT2G CTGV61 29217 Sep 17 13:05 ctgsetup_IBM-930 -rwxrwxr-x --s- 2 CITGT2G CTGV61 29217 Sep 17 13:05 ctgsetup_IBM-935 -rwxrwxr-x --s- 2 CITGT2G CTGV61 29217 Sep 17 13:05 ctgsetup_IBM-939 -rwxrwxr-x -ps- 2 CITGT2G CTGV61 37522 Sep 17 13:05 ctgstart -rwxrwxr-x -ps- 2 CITGT2G CTGV61 856064 Sep 15 13:13 libctgjni.so -rwxrwxr-x -ps- 2 CITGT2G CTGV61 856064 Sep 15 13:13 libctgjni_g.so -rwxrwxr-x -ps- 2 CITGT2G CTGV61 172032 Sep 17 13:05 libctgmvs.so drwxrwxr-x 12 CITGT2G CTGV61 8192 Sep 17 13:05 resource -rwxrwxr-x --s- 2 CITGT2G CTGV61 8192 Sep 17 13:05 testzos

If the files are subsequently modified, they lose their program controlled status and might need to be reset, as follows:

extattr +p <install_path>/bin/lib*.so extattr +ps <install_path>/bin/ctgstart

The Java SDK must also be program controlled. This option is set by default when the Java SDK is installed.

The Gateway daemon loads required files from MVS libraries. Of these libraries, SCTGLOAD, SDFHEXCI, SDFHLINK and SCEERUN2 must be program controlled. To give these libraries PROGRAM CONTROL status, issue the following RACF commands:

RALTER PROGRAM * ADDMEM(’CTGV61.SCTGLOAD’//NOPADCHK)RALTER PROGRAM * ADDMEM(‘CICSTS31.CICS.SDFHEXCI’//NOPADCHK)RALTER PROGRAM * ADDMEM(’SYS1.CICSTS31.CICS.SDFHLINK’//NOPADCHK)RALTER PROGRAM * ADDMEM(’CEE.SCEERUN2’//NOPADCHK)

Then, make the program control settings active with the following command:

SETROPTS WHEN(PROGRAM) REFRESH

You can find more information about the RACF commands in the RACF Systems Programmer’s Guide, SC28-1913.

Tip: The ctgstart script is installed with the shared bit s set (Example 2-2) and should be left with this setting. This setting causes the _BPX_SHAREAS environment variable to be used, which allows the required sharing of the address space with CTGBATCH and non-sharing of the address space when using ctgstart from USS.


Advanced Program Function (APF) AuthorizationIf you plan to use the new XA transaction support in CICS TG V6.1, CTGINIT (supplied in the CICS TG SCTGLINK library) must be in the LNKLST and in an APF authorized library. For more details see “AFP Authorization of CTGINIT” on page 197.

2.3 Configuring CICS TGIn this section, we describe how to configure CICS TG. To configure CICS TG, you need to:

1. Define an HFS.2. Set permissions.3. Create a Gateway daemon configuration file.4. Create an STDENV file.5. Create the started task JCL.

2.3.1 Defining an HFS In this section, we show how we created and mounted HFS data sets to hold trace and configuration data.

Creating a data setIn our text environment, we did not want the configuration and trace files to be written to the same HFS as the CICS TG product executables, which we mounted with read only access. We needed the Gateway daemon to be able to write traces to HFS. Also, we needed to be able to alter the configuration files. So, we created two new HFS data sets:

HFS Mount pointOMVS.CITGTRC.HFS /cicstg/ctg610/traceOMVS.CITGCFG.HFS /cicstg/ctg610/config

An HFS can be allocated using ISPF option 3.2, an IEFBR14 batch job, or through the ISHELL menus.

Tip: zSeries File System (zFS) has improved caching functionality compared with HFS. This improved caching can provide significant performance benefits for log and trace files in a zFS compared with equivalent logs on an HFS. This redbook does not discuss zFS. For information about zFS, see z/OS Distributed File Service zSeries File System Implementation z/OS V1R7, SG24-6580.


Mounting the HFSIn OMVS, change to the root directory and use the OMVS mkdir command to create the /cicstg/ctg610/trace and /cicstg/ctg610/config directories (Figure 2-3).

Example 2-3 Creating a mount point directory

CITGCA: /CITG/CA>cd / CITGCA: />mkdir cicstg CITGCA: />mkdir cicstg/ctg610 CITGCA: />mkdir cicstg/ctg610/trace CITGCA: />mkdir cicstg/ctg610/configCITGCA: />

Then, mount each of the HFS data sets using the ISHELL File_systems menu (see Figure 2-3).

Figure 2-3 Using ISHELL to mount OMVS.CITGTRC.HFS


In our scenario, we wanted our HFS file systems to be available when our z/OS system was idle. So, we added MOUNT statements for /cicstg/ctg610/trace and /cicstg/ctg610/config to our BPXPRMxx member in SYS1.PARMLIB (Example 2-4).

Example 2-4 MOUNT statements for the CICS TG config and trace HFSs

MOUNT FILESYSTEM('OMVS.CITGTRC.HFS') TYPE(HFS) MODE(RDWR) MOUNTPOINT('/cicstg/ctg610/trace') MOUNT FILESYSTEM('OMVS.CITGCFG.HFS') TYPE(HFS) MODE(RDWR)

MOUNTPOINT('/cicstg/ctg610/config')

2.3.2 Setting permissionsTo be able to read the configuration file, the Gateway daemon user ID needs to have:

� Execute permission on the directories (to traverse the directories)� Read permission on the configuration file.

The Gateway daemon also needs write access the traces directory.

In our testing, we planned to have a number of Gateway daemons, all running under their own user IDs. In this environment, each daemon reads its configuration file from the configuration directory and writes trace output to the trace directory. To enable this, we connected all of the Gateway daemon user IDs to the CTGV61 group (group id 7100), which was made the owning group of the following directories:

� /cicstg� /cicstg/ctg610� /cicstg/ctg610/config� /cicstg/ctg610/trace

We use the ISHELL, specifying the relevant directory and using the Directory → Attributes → Edit → Owning Group menu selection (as shown in Figure 2-4).


Figure 2-4 Changing the GID with ISHELL

We then changed the permissions for the /cicstg and /cicstg/ctg610 directories to 750 and gave the configuration file CITGR1G.ini permissions of 660. For the /cicstg/ctg610/config and /cicstg/ctg610/trace directories we gave permissions 770.

2.3.3 Creating a Gateway daemon configuration fileCICS TG uses a Gateway daemon configuration file for its configuration parameters. The default name for this configuration file is CTG.INI. A sample configuration file is supplied in the <install_path>/bin/ctgsamp.ini file.

In our testing, we decided to used the job name of our Gateway daemon for our configuration file (for example, CITGR1G.ini) so that we could differentiate between configuration files for different Gateway daemons in the same directory. To create our configuration file we:

� Copied the sample configuration file /usr/lpp/cicstg/ctg610/bin/ctgsamp.ini to /cicstg/ctg610/config/CITGR1G.ini.


� Edited CITGR1G.ini as shown in (Example 2-5).

Example 2-5 CITGR1G.ini changes

SECTION GATEWAY connectionlogging=on nonames=on

[email protected][email protected][email protected]=port=13101;\ connecttimeout=2000;\ idletimeout=600000;\ pingfrequency=60000

We set the port parameter to 13101, and we also set connectionlogging to on (as an aid to debugging any connection problems that we might encounter). The connectionlogging parameter logs client connections and disconnections. For more detail about other connection related configuration file settings, see 3.3.2, “Gateway threads” on page 92.

2.3.4 Creating an STDENV fileBefore the Gateway daemon can be started, you need to set some environment variables in an STDENV file, which can be an MVS sequential file, a PDS member, or an HFS file. A sample STDENV is supplied as a member of SCTGSAMP.

You need to set the following environment variables:

� CICSCLI� CLASSPATH� PATH� DFHJVPIPE� DFHJVSYSTEM� STEPLIB

Tip: We recommend that you use a PDS or PDSE member for your STDENV. We found that an STDENV allocated as a sequential data set cannot be updated while the Gateway daemon is running.


We copied the supplied STDENV member to CITG.CTG610.STDENV(CITGR1G) and then did the following:

� Changed the environment variables� Removed the comments� Ordered the environment variables alphabetically

Example 2-6 shows the environment variable settings that we used.

Example 2-6 CITG.CTG610.STDENV(CITGR1G)

_BPX_SHAREAS=YES CICSCLI=/cicstg/ctg610/config/CITGR1G.ini CLASSPATH=/usr/lpp/cicstg/ctg610/classes/ctgsamples.jarCOLUMNS=80 CTG_RRMNAME=CTG.RRM.CITGR1G.IBM.UA DFHJVPIPE=CITGR1G DFHJVSYSTEM_00=A6POTGR1-Primary server PATH=/bin:/usr/lpp/java142/J1.4/bin TMPDIR=/cicstg/ctg610/config TZ=GMT-2

Description of the environment variablesThe following list provides a description of the main environment variables and the values that we set:

� _BPX_SHAREAS

The Gateway daemon can be started in its own address space or share the address space of the process that started it. If you are using CTGBATCH to start the Gateway daemon (see 2.6.1, “Starting the Gateway daemon” on page 51), ensure that _BPX_SHAREAS=YES is set in the STDENV DD statement. If starting the Gateway daemon from a USS shell prompt (such as OMVS), set _BPX_SHAREAS=NO in the ctgenvvar script to force the use of a clean address space.

We specified _BPX_SHAREAS=YES because we started the Gateway daemon using CTGBATCH and used one address space.

� CICSCLI

You can specify a different path to the configuration file by setting the location of the configuration file. This variable is important when running multiple Gateway daemon started tasks.

We set CICSCLI to /cicstg/ctg610/config/CITGR1G.ini to allow each Gateway daemon started task to have its own configuration file.


� CLASSPATH

Includes a concatenation of fully qualified jar files or HFS directories that contain class files. If running the Gateway daemon with the sample security exits, include <install_path>/classes/ctgsamples.jar in the CLASSPATH. The ctgstart script appends the main product jar files to the classpath.

We set CLASSPATH to /usr/lpp/cicstg/ctg610/classes/ctgsamples.jar.

� CTG_RRMNAME

This is the name with which the Gateway daemon registers with Resource Recovery Services (RRS) for transactional EXCI requests to CICS. The CTG_RRMNAME must conform to the naming rules for RRS groups.

The name can consist of the following printable characters:

– Alphanumeric characters: A-Z and 0-9– National characters: $ (X'5B'), # (X'7B'), @ (X'7C')– The period (.)– The underscore (_)

RRM name restrictions include:

– The name cannot start with a blank or contain embedded blanks.

– Lowercase characters are converted to uppercase characters.

– To avoid naming conflicts, use A through C or G through I as the first character.

– The length of CTG_RRMNAME must not exceed 32 characters and must end with the characters IBM.UA.

Note that different rules apply if XA transactions are enabled, for more details see 5.6.3, “Configuring CICS TG” on page 175.

If you do not specify a CTG_RRMNAME, a unique CTG_RRMNAME is generated by the CICS TG.

We set CTG_RRMNAME to CTG.RRM.CITGR1G.IBM.UA.

� COLUMNS

Specifies the output width of the screen for CICS TG messages. We left this at the COLUMNS=80 supplied in the sample STDENV.

� DFHJVPIPE

The value specified here must match the NETNAME in the CICS connection definition to be used for an EXCI connection using a specific pipe. The default connection name in the IBM-supplied group DFH$EXCI is EXCS, which has a NETNAME of BATCHCLI.

We created our own connection definition, and used a NETNAME of CITGR1G.


� DFHJVSYSTEM

The syntax of this variable is DFHJVSYSTEM_nn=aaaaaaaa literal, where:

– nn = 0 to 99– aaaaaaaa = the APPLID of the CICS region– literal = a description of the CICS region

Note the setting of this variable is optional, and only affects the result of the ECIRequest.listSystems() method and does not affect which CICS regions you can connect to.

We set DFHJVSYSTEM_00 to A6POTGR1 (our CICS APPLID).

� PATH

The PATH statement includes the binary directory where Java is located.

We set PATH to /usr/lpp/java142/J1.4/bin.

� STEPLIB

This variable identifies the libraries containing EXCI options. It also identifies the EXCI load libraries. If a modified EXCI options table DFHXCOPT is to be included, it must appear before the EXCI load library SDFHEXCI in order to override the default DFHXCOPT table.

At this stage, we did not specify a value for the STEPLIB variable, because we initially used the default DFHXCOPT options table provided in the SDFHEXCI library. See “TIMEOUT settings in DFHXCOPT” on page 98 for information about how we subsequently changed the EXCI options.

� TMPDIR

TMPDIR is a USS variable defining a temporary directory which is used whilst executing OMVS commands.

The default location for this directory is /tmp. We kept the TMPDIR within the CICS TG HFS in directory /cicstg/ctg610/traces.

� TZ

TZ is a USS environment variable which allows the mapping of the system time (stored in GMT) to a local time zone. For full details on TZ see z/OS V1R6 UNIX System Services Command Reference, SA22-7802.

We set TZ=GMT-2 so that the Gateway daemon messages were reported with the correct local time.

For a full description of all the CICS TG environment variables see CICS Transaction Gateway for z/OS Administration, SC34-6672.


Order of precedence of CICS TG environment variablesEnvironment variables passed by CTGBATCH to the Gateway daemon can be set in one of the following ways:

� Using the ctgenvvar script file� Within the JCL using a data set referenced by the STDENV DD card

The ctgenvvar script is best used when you run the Gateway daemon from the OMVS command line using the ctgstart command (Figure 2-5).

The search order for environment variables used by CTGBATCH is as follows:

1. If STDENV DD statement is defined in the startup JCL, environment variables specified in the reference file are used.

2. If the CTGENVVAR environment variable is set, the variables are taken from the referenced file and these override environment variables set by STDENV.

3. If CTGENVVAR is not set, or the ctgenvvar file which the CTGENVVAR variable specifies does not exist, the Gateway daemon looks in the <install_path>/bin directory for a ctgenvvar file. If it finds one, any environment variables which it contains override those already set by the STDENV referenced file.

Figure 2-5 Concatenation sequence for CICS TG variables

STDENV

specified?

ctgenvvar in <install_path>/bin?

CTGENVVARspecified and present ?

Use the variables specified, overiding any previous

definition

Yes

No

No

Use the variables specified

Yes

Use the variables specified, overiding any previous definition

Yes

No

Start Gateway daemon


For ease of maintenance, we placed our environment variables in an STDENV member. To ensure that these variables were not overridden, we set the CTGENVVAR variable and did not have a ctgenvvar file in our <install_path>/bin directory.

2.3.5 Creating the started task JCLIn 2.6.1, “Starting the Gateway daemon” on page 51, we discuss the different ways in which the Gateway daemon can be started. We recommend that you start the Gateway deamon with a started task job. A sample is supplied in member CTGPROC of SCTGSAMP that you can use as a base for the Gateway daemon started task JCL.

In our scenario, we copied CTGPROC, removed the comments, and specified values for CTGUSR, CTGHLQ, REGION, and the ctgstart path as shown in Example 2-7.

Example 2-7 JCL for Gateway daemon started task

//CITGR1G PROC // SET CTGHLQ='CTGV61' // SET CTGUSR='CITG.CTG610.STDENV(CITGR1G)' // SET LEOPTS='/' //CTG EXEC PGM=CTGBATCH,REGION=250M, // PARM='&LEOPTS./usr/lpp/cicstg/ctg610/bin/ctgstart -noinput '//STEPLIB DD DSN=&CTGHLQ..SCTGLOAD,DISP=SHR //STDOUT DD SYSOUT=* //STDERR DD SYSOUT=* //CTGDBG DD DUMMY //STDENV DD DSN=&CTGUSR.,DISP=SHR //*

We recommend that you specify a specific REGION size and do not specify REGION=0M. While REGION=0M requests that the job receives as large a region as it requires, the actual allocation can be limited by the z/OS IEFUSI exit, resulting in a region size that is much smaller than expected.

The IEFUSI exit is used in a z/OS system to limit the REGION size that is allowed to jobs and can override the allocation with a much lower system default value. This limitation can result in a Gateway daemon running with an unexpectedly small storage allocation, suffering storage and performance problems.

If a specific value is set for REGION and the requested storage is not available, the job will fail, allowing early problem determination. We specified a REGION size of 250M, which we considered reasonable for our testing purposes.


2.4 Configuring CICSIn this section, we deal with the following CICS configuration tasks:

1. Creating a CONNECTION definition.2. Creating a SESSIONS definition.3. Installing the definitions.4. Configuring CICS for RRMS.5. Compiling and installing the sample program.6. Editing and assembling DFHCNV.7. Opening interregion communication.

2.4.1 Creating a CONNECTION definitionAn EXCI connection for the CICS TG must be defined in CICS. You can copy the supplied EXCS CONNECTION definition from the DFH$EXCI group and update it to reflect the required definitions.

You can install either generic or specific EXCI connections in your CICS regions to allow them to communicate with the Gateway daemon. For a specific EXCI connection, the value specified to the Gateway daemon environment variable DFHJVPIPE must be the same as the NETNAME option on the connection definition. The netname itself is an arbitrary value. A generic connection is defined in much the same way, except that the NETNAME option is left blank.

We copied the EXCS connection to our own CITGR1 group as GR1G and set NETNAME to CITGR1G (Figure 2-6).

Note: We recommend that you use specific EXCI connections because this gives you more control over which Gateway daemons can access which CICS regions and also makes problem determination easier.


Figure 2-6 GR1G connection definition

2.4.2 Creating a SESSIONS definitionThe EXCS SESSIONS definition can also be copied from group DFH$EXCI and the connection attribute of the session definition altered to refer to the previously defined CONNECTION. It is also possible to set the RECEIVECOUNT and RECEIVEPFX parameters. It is desirable to do this because:

� RECEIVECOUNT defines the maximum number of EXCI pipes to be used for the defined connection.

� RECEIVEPFX allows the definition of a single or double character prefix so that the usage of each session can be easily identified. If a single character RECEIVEPFX is specified the RECEIVECOUNT can be a maximum of 999. If a two character RECEIVEPFX is specified the RECEIVECOUNT can be a maximum of 99.

We defined session GR1G, setting the RECEIVEPFX to 1 and the RECEIVECOUNT to 100 (Figure 2-7). See 3.5, “Configuring CICS” on page 100 for more details about the definition of EXCI CONNNECTION and SESSIONS.

OBJECT CHARACTERISTICS CICS RELEASE = 0640 CEDA View CONnection( GR1G ) CONnection : GR1G Group : CITGR1 DEscription : SAMPLE EXCI SPECIFIC CONNECTION CONNECTION IDENTIFIERS Netname : CITGR1G INDsys : REMOTE ATTRIBUTES REMOTESYSTem : REMOTEName : REMOTESYSNet : CONNECTION PROPERTIES ACcessmethod : IRc Vtam | IRc | INdirect | Xm PRotocol : Exci Appc | Lu61 | Exci Conntype : Specific Generic | Specific SInglesess : No No | Yes DAtastream : User User | 3270 | SCs | STrfield | Lms + RECordformat : U U | Vb SYSID=TGR1 APPLID=A6POTGR1 PF 1 HELP 2 COM 3 END 6 CRSR 7 SBH 8 SFH 9 MSG 10 SB 11 SF 12 CNCL


Figure 2-7 GR1G SESSIONS definition

2.4.3 Installing the definitionsWe activated the CICS definitions by installing the CITGR1 group using the CEDA INSTALL GROUP(CITGR1G) command.

2.4.4 Configuring CICS for RRMSBecause we intend to test CICS TG transactional support in our scenario, we configure CICS to register with Resource Recovery Management Services (RRMS).

It is a requirement that RRMS support is functioning and CICS is registered with RRMS if CICS is to handle extended logical units of work, because RRMS is the resource manager for these extended transactions. In our environment, we enabled CICS registration with RRMS by setting the CICS system initialization parameter RRMS=YES.

OBJECT CHARACTERISTICS CICS RELEASE = 0640 CEDA View Sessions( GR1G ) Sessions : GR1G Group : CITGR1 DEscription : Sample EXCI Specific sessions definition SESSION IDENTIFIERS Connection : GR1G SESSName : NETnameq : MOdename : SESSION PROPERTIES Protocol : Exci Appc | Lu61 | Exci MAximum : 000 , 000 0-999 RECEIVEPfx : 1 RECEIVECount : 100 1-999 SENDPfx : SENDCount : 1-999 SENDSize : 04096 1-30720 + RECEIVESize : 04096 1-30720 SYSID=TGR1 APPLID=A6POTGR1 PF 1 HELP 2 COM 3 END 6 CRSR 7 SBH 8 SFH 9 MSG 10 SB 11 SF 12 CNCL

Tip: It is possible to verify that RRMS is open using the CEMT INQUIRE RRMS command.


2.4.5 Compiling and installing the sample programsThe sample program ec01.ccp is provided with the CICS TG in the /usr/lpp/cicstg/ctg610/samples/server directory.

For our testing, we copied the program to our CICS source library, compiled the program, and added the load module to our program load library.

2.4.6 Editing and assembling DFHCNVYou can use the EciB1 sample application that is provided with the CICS TG to test the Gateway daemon configuration. EciB1 is a Java program that runs in the USS environment, a UNICODE environment. The Java program receives data that is produced by the EC01 COBOL program running in CICS, an EBCDIC environment. The data is passed in a COMMAREA. To ensure that this COMMAREA is converted correctly between UNICODE and EBCDIC, we created a DFHCNV table entry for program EC01 with SRVERCP=037,CLINTCP=850. For further details about using DFHCNV, refer to Appendix B, “DFHCNV and CICS data conversion” on page 379.

2.4.7 Opening interregion communicationIt is possible to check the status of CICS interregion communication using the CEMT INQURE IRC command (Example 2-8). If IRC is closed, you can issue the CEMT SET IRC OPEN command to open CICS IRC.

Example 2-8 CEMT INQUIRE IRC command

INQUIRE IRC STATUS: RESULTS - OVERTYPE TO MODIFY Irc Ope

2.5 Testing the configurationThis section explains how to verify that your Gateway daemon is functioning correctly using:

� The CTGTESTR sample JCL� CICS TG STDOUT output� The ping command� The netstat command


To test our configuration we used the CTGTESTR sample job supplied in the SCTGSAMP data set. This job runs the ctgtest script from the <install_path>/samples directory. The ctgtest script is designed to be run without user input so that it can be executed from a batch job. The script uses the EciB1 sample program.

We updated the CICSTESTR job (Example 2-9) with the following:

� URL and PORT used by our Gateway daemon

� PATH variable set to the HFS directory of the Java binary directory

� CLASSPATH variable set to the HFS directory of ctgclient.jar

Example 2-9 CTGTESTR job

//CITGCAT1 JOB (0),MSGCLASS=X,CLASS=A,NOTIFY=&SYSUID,REGION=250M // SET CTGHLQ='CTGV61' // SET LEOPTS='/' //TEST EXEC PGM=CTGBATCH, // PARM='&LEOPTS./usr/lpp/cicstg/ctg610/samples/ctgtest USERID// PASSWORD tx1.mop.ibm.com 13101' //STEPLIB DD DSN=&CTGHLQ..SCTGLOAD,DISP=SHR //STDOUT DD SYSOUT=* //STDERR DD SYSOUT=* //STDENV DD * PATH=/bin:/usr/lpp/java/J1.4/bin CLASSPATH=/usr/lpp/cicstg/ctg610/classes/ctgclient.jar /* //


EciB1 flows an ECI request to a connected CICS region through a Gateway daemon (Figure 2-8).

Figure 2-8 CTGTESTR test scenario

The output of this application is the date and time as formatted by the CICS COBOL program EC01.

We verified that our configuration was set up correctly as follows:

1. We started the Gateway daemon as a started task from SDSF using the /S CITGR1G command, as shown in Example 2-10.

Example 2-10 Results of /S CITGR1G written to the SDSF log

S CITGR1G $HASP100 CITGR1G ON STCINRDR IEF695I START CITGR1G WITH JOBNAME CITGR1G IS ASSIGNED TO USER CITGR1G , GROUP CITG $HASP373 CITGR1G STARTED

2. When the Gateway daemon had started, we checked the STDOUT to ensure that the Gateway daemon had started correctly (Example 2-11). Message CTG6524I shows that the TCP protocol handler has started successfully so

EXCI

CICS TG

z/OS

JNI

CICS

EC01Gatewaydaemon

tx1.mop.ibm.com

ECI request

UNIX System Services

JVM

ctgclient.jar

CICSApplication

13101

CTGBATCH CTGTESTR JCL

EciB1

A6POTGR1

ctgserver.jar

ctgstart

CITGR1G


that the Gateway daemon can accept incoming work. Message CTG5612I shows that the Gateway daemon has successfully initialized.

Example 2-11 Successful start of Gateway daemon as seen in STDOUT

CTG6400I CICS Transaction Gateway is starting CTG8400I Using configuration file /cicstg/ctg610/config/CITGR1G.ini. CTG6577I Java version is 1.4.2 CTG6502I Initial ConnectionManagers = 1, Maximum ConnectionManagers = 100 CTG6526I Initial Workers = 1, Maximum Workers = 100 CTG6574I Connection logging has been disabled CTG6738I XA transaction support is not enabledCTG6898I 250 EXCI pipes are available for use by the CICS Transaction Gateway.CTG6981I Successfully initialized JNI library CTG6505I Successfully created initial ConnectionManager and Worker threads CTG6524I Successfully started handler for the tcp: protocol on port 13101 CTG6512I CICS Transaction Gateway initialization complete CTG6898I 250 EXCI pipes are available for use by the CICS Transaction Gateway.

3. We checked the state of the TCP/IP sockets on our USS TCP/IP stack using the netstat command from OMVS. Example 2-12 shows the Gateway daemon listening on port 13101.

Example 2-12 OMVS netstat

CITGCA: /CITG/CA>netstat MVS TCP/IP NETSTAT CS V1R6 TCPIP Name: TCPIP1 16:45:22 User Id Conn Local Socket Foreign Socket State ------- ---- ------------ -------------- ----- CITGR1G 00003378 0.0.0.0..13101 0.0.0.0..0 Listen

4. We checked basic IP connectivity from our workstation to z/OS using the ping command from a Windows 2000 command prompt (Example 2-13).

Example 2-13 Ping to z/OS verifying hosts or DNS setup

C:\>ping tx1.mop.ibm.com

Pinging tx1.mop.ibm.com [9.100.193.122] with 32 bytes of data:

Reply from 9.100.193.122: bytes=32 time<10ms TTL=63Reply from 9.100.193.122: bytes=32 time<10ms TTL=63Reply from 9.100.193.122: bytes=32 time<10ms TTL=63Reply from 9.100.193.122: bytes=32 time<10ms TTL=63

Ping statistics for 9.100.193.122: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 0ms, Average = 0ms


5. We performed our basic test by submitting the CTGTESTR job. The job finished with a completion code of 0. Checking the job output we could see the date and time returned by the CICS program EC01 (Example 2-14).

Example 2-14 OMVS CTGTESTR test output

ctgtest: Executing EciB1 with userid = USERID and parameters: tx1.mop.ibm.com 13101 CICS Transaction Gateway Basic ECI Sample 1 Usage: java com.ibm.ctg.samples.eci.EciB1 [Gateway URL] [Gateway Port Number] [SSL Keyring] [SSL Password] To enable client tracing, run the sample with the following Java option: -Dgateway.T.trace=on The address of the Gateway has been set to tx1.mop.ibm.com Port:13101 CICS Servers Defined: . 1. A6POTGR1 -Primary server Choose Server to connect to, or q to quit: Program EC01 returned with data:- .Hex: 31392f30392f30352031363a30303a30330 .ASCII text: 19/09/05 16:00:03.

Note that the completion code indicates whether the CTGTESTR job successfully completed or not, rather than the result of the ECI call itself. The completion code is still 0 even if the CICS region is not active, however, the job output shows an ECI_ERR_NO_CICS error (Example 2-15).

Example 2-15 Results of CTGTEST when CICS is not available

ctgtest: Executing EciB1 with userid = USERID and parameters: tx1.mop.ibm.com 13101CICS Transaction Gateway Basic ECI Sample 1 Usage: java com.ibm.ctg.samples.eci.EciB1 [Gateway URL] [Gateway Port Number] [SSL Keyring] [SSL Password] To enable client tracing, run the sample with the following Java option: -Dgateway.T.trace=on The address of the Gateway has been set to tx1.mop.ibm.com Port:13101 CICS Servers Defined: 1. A6POTGR1 -Primary server Choose Server to connect to, or q to quit: ECI returned: ECI_ERR_NO_CICS Abend code was null


2.6 Operating CICS TGIn this section, we discuss the operation of CICS TG, including how to do the following:

1. Starting the Gateway daemon.2. Stopping the Gateway daemon.3. Automatic Restart Management (ARM).

2.6.1 Starting the Gateway daemonYou can start the Gateway daemon either from a USS command line or by a JCL batch job. We do not discuss the use of the ctgstart script from USS because the use of a batch job has many advantages, including more flexibility in the routing of Gateway daemon messages and the option to use ARM.

There are two alternatives for starting the Gateway daemon from JCL:

� BPXBATCH� CTGBATCH

BPXBATCHPrior to Version 6 of CICS TG, starting the Gateway daemon was dependent on the z/OS supplied utility BPXBATCH. Also, in previous releases, all log messages were written to the USS stdout and stderr destinations, and BPXBATCH can only map the USS stdout and stderr destinations to HFS files.

Starting the Gateway daemon using BPXBATCH is still supported with CICS TG V6.1 and is described in CICS Transaction Gateway for z/OS Administration, SC34-6672. However, the recommended way to start the Gateway daemon from JCL is to use CTGBATCH.

CTGBATCHCTGBATCH is a Language Environment® batch mode program which is supplied with CICS TG for z/OS V6 and later. It is used to start the Gateway daemon via JCL started task and to pass the required environment variables to the USS ctgstart script.

CTGBATCH has advantages over BPXBATCH in that it can route Gateway daemon messages to the following destinations:

� The JES log � An MVS sequential data set � An HFS file


If you start the Gateway daemon from JCL using CTGBATCH (or BPXBATCH), it can register with the z/OS Automatic Restart Manager component (see 2.6.3, “Automatic Restart Manager” on page 53 for more information). You can start CTGBATCH from a JCL batch job or as a started task procedure. We recommend that you start the Gateway daemon via CTGBATCH from a started task procedure.

We started the Gateway daemon using a procedure called CITGR1G in SYS1.PROCLIB (see Example 2-7 on page 42).

2.6.2 Stopping the Gateway daemonIn CICS TG V6, you can stop the Gateway daemon in a controlled manner, rather than using the SDSF CANCEL command.

Normal shutdownA normal shutdown quiesces the Gateway daemon so that all the work that is in progress completes but no new work is started. After all the work in progress has completed, the Gateway daemon then shuts down.

A normal shutdown is requested with the following SDSF command:

/F JOB_NAME,APPL=SHUT|SHUTDOWN

Example 2-16 Normal shutdown using /P CITGR1G

P CITGR1G BPXM023I (CITGR1G) 856 CTG8239I Response received from CICS Transaction Gateway CTG8235I Normal shutdown was requested

Immediate shutdownAn immediate shutdown terminates all work in progress and breaks any active connections. The Gateway daemon does not accept any new connections and shuts down immediately.

An immediate shutdown is requested with the following SDSF command:

/F JOB_NAME,APPL=SHUT|SHUTDOWN,IMM|IMMEDIATE

Tip: The SDSF stop command /P JOB_NAME that specifies the Gateway daemon job name has exactly the same effect as /F JOB_NAME,APPL=SHUT (as shown in Example 2-16).


Example 2-17 Immediate shutdown using /F CITGR1G,APPL=SHUT,IMM

F CITGR1G,APPL=SHUT,IMM BPXM023I (CITGR1G) 864 CTG8239I Response received from CICS Transaction Gateway CTG8234I Immediate shutdown was requested $HASP395 CITGR1G ENDED

For shutdown considerations when using extended logical units of work or XA transactions, see 5.8, “Operations” on page 219.

2.6.3 Automatic Restart ManagerThe CICS TG is enabled to use Automatic Restart Manager (ARM), which is a z/OS facility that allows the automated restart of an MVS subsystem. MVS automatic restart management is a sysplex-wide integrated automatic restart mechanism that can restart:

� An MVS subsystem in place if it abends (or if a monitor program notifies ARM of a stall condition).

� A failed MVS image.

The CTGARM utility is supplied in the CTGARM member of SCTGAUTH and can be used to register and deregister the CICS TG with ARM.

Example 2-18 shows how we removed the comments from the CTGARM-related code in the supplied CTGPROC and wrapped it around our existing JCL that we created originally in Example 2-7 on page 42. We then started the Gateway daemon with the /S CITGR1G,CTGID=CITGR1G command. We specified ARM_TYPE as SYLVL2, which is the default.

Example 2-18 CICS TG started task JCL with ARM registration

//CITGR1G PROC // SET CTGHLQ='CTGV61' // SET CTGUSR='CITG.CTG610.STDENV(CITGR1G)' // SET LEOPTS='/' //*

Note: If you have requested a normal shutdown and the Gateway daemon is taking too long to shutdown, it is possible to request an immediate shutdown subsequently.

Tip: To use CTGARM the SCTGAUTH load library must be defined as APF-authorized.


//* Optional step: register with ARM //* //REG EXEC PGM=CTGARM, // PARM='REGISTER CTG&CTGID. SYSLVL2' //STEPLIB DD DSN=&CTGHLQ..SCTGAUTH,DISP=SHR //SYSPRINT DD SYSOUT=* //* //*Main step: run CTG //* Only runs if successfully registered with ARM //* // IF RC <5 THEN //CTG EXEC PGM=CTGBATCH,REGION=250M, // PARM='&LEOPTS./usr/lpp/cicstg/ctg610/bin/ctgstart -noinput '//STEPLIB DD DSN=&CTGHLQ..SCTGLOAD,DISP=SHR // DD DSN=CITG.CITGR1G.USERLOAD,DISP=SHR // DD DSN=CICSTS31.CICS.SDFHEXCI,DISP=SHR //STDOUT DD SYSOUT=* //STDERR DD SYSOUT=* //CTGDBG DD DUMMY //STDENV DD DSN=&CTGUSR.,DISP=SHR //* //* Optional step: deregister with ARM //* Only runs if CTG terminated cleanly //* //DEREG EXEC PGM=CTGARM, // PARM='DEREGISTER' //STEPLIB DD DSN=&CTGHLQ..SCTGAUTH,DISP=SHR //SYSPRINT DD SYSOUT=* // ENDIF // //PEND //*

The syntax of the CTGARM registration is ’R[egister] ARM_ID [ARM_TYPE]’.

The restart order is governed by the ARM_TYPE where a job registered with SYSLVL0 will be restarted before one registered with SYSLVL1, and so on. For more details on Automatic Restart Management, see z/OSV1R2.0 MVS Setting up a Sysplex, SA22–7625.

The startup job expects the CTGID to be passed on the SDSF start command. If the CTGID is not passed on the start command the ARM registration fails and a the following message is written to the SYSPRINT DD destination:

CTG6022E Registration failed - ARM_ID is invalid

Return code 8 is returned to the CTGBATCH job, and the Gateway daemon does not start (there is a conditional statement requiring a return code less than five).


The ARM registration results in a number of ARM related CTG messages which are written to the SYSPRINT location (Figure 2-19). Similar messages are also written when the Gateway daemon is shut down.

Example 2-19 ARM registration messages sent to SYSPRINT

CTG6000I CTGARM - CTG Automatic Restart Manager Batch Utility CTG6002I Parameters processed: CTG6003I Command: Register CTG6004I ARM_ID: CTGCITGR1G CTG6005I ARM_TYPE: SYSLVL2 CTG6034I Registering with ARM CTG6006I Results are: CTG6007I Return code: 00000000 CTG6008I Reason code: 00000000 CTG6031I This is first time register with ARM CTG6035I Telling ARM we are ready CTG6006I Results are: CTG6007I Return code: 00000000 CTG6008I Reason code: 00000000 CTG6037I Successfully registered with ARM CTG6001I End of CTGARM - CTG Automatic Restart Manager Batch Utility

2.7 Configuring for multiple Gateway daemonsAfter we had tested that our single Gateway daemon could communicate with a CICS TS region successfully, we decided to set up a second Gateway daemon (see Table 2-2). You might run multiple Gateway daemons if:

� You need test and production Gateway daemon regions.� You are upgrading to a new version.� You require a second Gateway Daemon for a different e-business application.

Note: The USS ctgarm utility that is supplied in <install_path>/bin is provided for migration purposes only.


In 3.7.2, “Cloned Gateway daemon connectivity tests” on page 114, we demonstrate a similar scenario in which we clone Gateway daemons. You would use a cloned configuration if:

� You need more than 250 simultaneous pipes to CICS regions.� You need to workload manage across multiple cloned Gateway daemons.

Table 2-2 Definitions checklist with second Gateway daemon and CICS regions

Consider the following areas when setting up a second Gateway daemon:

� Set RACF permissions on the Gateway daemon started task user ID.

� Create started task JCL.

� Create new directories in USS.

� Create a new configuration file.

– Change the Gateway daemon TCP/IP port

� Create a new STDENV member.

– Set the name and location of the configuration file– Set DFHJVPIPE name– Set the DFHJVSYSTEM_00 name– Set the CLASSPATH– Set the RRMNAME

� Create CICS definitions.

– Connection– Sessions

Value Gateway daemon

Second Gateway daemon

CICS TS Region 1

CICS TS Region 2

hostname tx1.mop.ibm.com tx1.mop.ibm.com - -

IP address 9.100.193.122 9.100.193.122 - -

TCP/IP port 13101 11101 - -

Job name CITGR1G CITGS1G CITGR1CI CITGS1CI

APPLID - - A6POTGR1 A6POTGS1

NETNAME CITGR1G CITGS1G - -

CONNECTION - - GR1G GS1R


We list each step in this process. However, because we assume that you have performed these tasks previously to set up the first Gateway daemon, we do not go into each step in detail.

To set up a second Gateway daemon:

1. Set RACF permissions on the Gateway daemon started task user ID.

We set up a new started task user ID, CITGS1G and set the required permissions.

2. Create started task JCL in the Proclib.

We copied our CITGR1G started task job as CITGS1G and changed it to reference a new STDENV member CITG.CTG610.STDENV(CITGS1G).

3. Create new directories in USS.

We used the same cicstg/ctg610/config directory for the Gateway configuration file and /cicstg/ctg610/trace directory for trace files for both Gateway daemons.

You might want to use a separate HFS for the configuration files and traces of each Gateway daemon, in which case you would need to create the directories and set the required permissions (as shown in 2.3.2, “Setting permissions” on page 35).

4. Create a new configuration file.

We copied /cicstg/ctg610/config/CITGR1G.ini to /cicstg/ctg610/config/CITGS1G.ini and changed the port from 13101 to 11101.

5. Create a new STDENV member.

We copied CITG.CTG610.STDENV(CITGR1G) to CITG.CTG610.STDENV(CITGS1G) and made the following changes:

a. We set CICSCLI to /cicstg/ctg610/config/CITGS1G.ini.

b. We changed DFHJVPIPE to CITGS1G to match the NETNAME which we specified on our CONNECTION definition in CICS region CITGS1CI.

c. We changed CTG_RRMNAME to CTG.RRM.CITGS1G.IBM.UA.

6. Create CICS definitions.

In our CITGS1CI CICS region, we defined the following resources:

d. CONNECTION

We created a GS1G CONNECTION definition and set NETNAME to CITGS1G.

e. SESSIONS

We created a GS1G SESSION definition and set it the CONNECTION it referenced to GS1G and the RECEIVEPFX to 2.


2.8 Migrating to CICS TG V6.1Direct migration to CICS TG V6.1 from earlier versions of CICS TG for z/OS is not supported. It is necessary to install CICS TG for z/OS V6.1 and then migrate your settings to the new installation.

In this section, we discuss:

1. Migrating from ctgenvvar.2. Other migration and configuration considerations.

2.8.1 Migrating from ctgenvvarThe sample ctgconvenv script is supplied in the <install_path>/bin directory. You can use this script to migrate ctgenvvar settings from a pre-CICS TG V6 system. The script converts a ctgenvvar script into the STDENV style settings which can be used by CTGBATCH. You can invoke the ctgconvenv script from:

� A USS command line with the following command:

ctgconenv [wrap width] [source file name]

� A batch job. A sample job to invoke ctgconvenv is supplied in SCTGSAMP(CTGCONV).

Note that ctgconvenv is not tolerant of environment variables that it does not recognize. If it finds a variable that it cannot identify, it stops parsing the ctgenvvar.

2.8.2 Other migration and configuration considerationsIn this section, we list other migration and configuration considerations.

Default Configuration file nameThe default configuration file name has changed from CTG.INI in CICS TG V5 to ctg.ini. If both exist in the <install_path>/bin directory the ctg.ini file will be used.

You should specify the configuration file in the CICSCLI environment variable to ensure that the Gateway daemon starts with the required settings.

Tip: If the ctgenvvar script that is specified in the ctgconvenv command does not have the execute permissions set for the user ID running the ctgconvenv, you will receive the following message:

CTG6124W Source file filename not found or not executable.


Configuration toolThere are two variations of the CICS TG configuration tool that is shipped with CICS TG V6.1:

� The ctgcfg tool is supplied in the <install_path>/bin directory and runs on z/OS and is accessed via X-windows.

� The ctgfzoscfg tool is supplied in the <install_path>/ctgzoscfg directory and can be transferred to a distributed platform using FTP.

Both tools generate a configuration file and a ctgenvvar script. The advantage of using these tools is that they perform validation checking on the values specified.

We did not use a configuration tool.

Features removedIn CICS TG V6 the following features are removed:

� Support for HTTP and HTTPS protocols.� Disabling the validation of units of work.� Allowing Java Client applications to obtain generic replies.� The Handler wakeup timeout.� The resource adapter specific for WebSphere Application Server for z/OS

(cicsecirrs.rar) is no longer supplied because the CICS ECI resource adapters used on all platforms are now identical.

SystemSSLSystemSSL is not supported with CICS TG V6. For details on migration from SystemSSL to JSSE (Java Secure Sockets Extension), see 9.5.1, “SystemSSL to JSSE migration” on page 335.

WebSphere Application ServerThe CICS ECI resource adapters that are supplied with the CICS TG V6.1 for z/OS are supported with the following J2EE application servers:

� WebSphere Application Server V6.0.1 for z/OS� WebSphere Application Server V6.0.

Only the 32-bit application servers on the following platforms are supported:

– AIX– Windows – Linux on Intel – Linux on POWER– Linux on zSeries – HP-UX PA-RISC– Solaris SPARC


In remote mode, earlier versions of the CICS ECI resource adapters can be used to connect to a CICS TG V6.1 daemon. To do this you must use a version of the resource adapter that is supported with the specific WebSphere Application Server in question.

When installing the CICS TG V6.1 resource adapters, it is not possible to connect to Gateway daemons of a lower version or release level. If it is necessary to interoperate with Gateway daemons of a lower version or release, then you must maintain separate application servers, each with the appropriate level of resource adapter.

In local mode, the resource adapters and CICS TG must be at the same level.

For detailed information about connecting from WebSphere Application Server, see Part 2, “Connection Management” on page 75.

TCP/IP workload management If you are currently using a TCP/IP workload management capability, such as Sysplex Distributor, to workload manage TCP/IP requests across a set of Gateway daemons, there are additional migration considerations if you intend to use the new XA transaction support available with CICS TG V6.1. For more information, see Chapter 6, “Gateway daemon transactions with TCP/IP load balancing” on page 225.

2.9 Problem determinationIn this section, we present information that we learned while installing the CICS TG and general information about problem determination and tracing.

2.9.1 Common problemsIn this section, we list some common problems that you might experience when installing and testing the CICS TG.

ECI_ERR_NO_CICS This error is caused by a failure in an attempt to connect to a CICS region. Check STDOUT for the following message:

CCL6822E Function Call = 3, Response = 8, Reason = 203, Subreason field-1 = xx, subreason field-2 = 0, Cics_Rc = -3.


Subreason field-1 identifies the cause of the problem:

� Subreason field-1 = 92 = X’005C’ means IRERRNLG “System not logged on”

IRC is not open. Check using the CICS command CEMT INQ IRC.

Check that the CICS region user ID has READ access to the RACF FACILITY class profile DFHAPPL.<CICS_applid>.

� Subreason field-1 = 104 = X’0068’ means IRERRNSS “Secondary system not in primary LCB”

This error occurs because the value specified for the CICS TG DFHJVPIPE environment variable does not correspond with the NETNAME value of any installed CICS CONNECTION definition.

Check that DFHJVPIPE has been set correctly and that a CONNECTION definition with a NETNAME value the same as DFHJVPIPE has been installed on the CICS region.

Abend S806The abend S806 indicates that there has been a failure to load a module.

The Gateway daemon abends with abend code S806 because it is unable to find a module which it requires. You should:

� Check the syslog for message:

CSV003I REQUESTED MODULE DFHXCPRX NOT FOUND

� Ensure that the STEPLIB is set correctly in the STDENV file, using a statement such as:

EXCI_LOADLIB="CICSTS31.CICS.SDFHEXCI"

� Check that the profile of the user ID which runs the Gateway daemon does not set the STEPLIB to point to a different library that can contain conflicting modules.

Note: In CICS TG V6, JNI code is loaded during initialization. Thus, this error occurs at Gateway daemon startup, rather than on the first ECI call (as was the case with CICS TG V5).


CTG6114EThe CICS TG STDERR message shown in Example 2-20 indicates that the CICS TG has not been correctly installed.

Example 2-20 Starting Gateway daemon when ctginstall has not been run

CTG6114E CICS Transaction Gateway has not been fully installed. Run the command '/usr/lpp/cicstg/ctg610/bin/ctgsetup' to complete the installation. 09/17/05 11:09:35:231 [0] CTG0827W CTGBATCH Child completed with a non-zero return code 256

You should run the ctginstall script that is supplied in /usr/lpp/cicstg/ctg610/ (which executes the ctgsetup script in the <install_path>/bin directory) to complete installation of the CICS TG.

2.9.2 Tips and utilitiesThis section includes useful commands and utilities for debugging problems with your configuration.

z/OS TCP/IP commandsWe found the following TCP/IP commands useful when analyzing network problems with our CICS TG for z/OS:

� HOMETEST

Issue this as a TSO command. If there is a valid TCP/IP address or host name, this command tells you what it is, along with other configuration information.

� NETSTAT

Issue this command to obtain information about the TCP/IP sockets that are in use (TSO and other platforms). The USS version of this command can be invoked using the onetstat command.

� NSLOOKUP <IP address>

Issue this as a TSO command. A valid TCP/IP address tells you the host name and vice versa.

� PING

Use this to determine if an address is active and to resolve host names to IP addresses (Example 2-21).


Example 2-21 Pinging the z/OS host

tso ping wtsc66oe.itso.ibm.com CS V1R6: Pinging host TX1.MOP.IBM.COM (9.100.193.122) Ping #1 response took 0.000 seconds.

Java version To display the level of the Java Development Kit (JDK™) installed on your system, use the java -fullversion command as follows:

>java -fullversion >java full version "J2RE 1.4.2 IBM z/OS Persistent Reusable VM build cm142-20050623"

To determine where the JDK is installed on your system, issue the following command to search for the location of the Java executable:

>type java>java is /usr/lpp/java142/J1.4/bin/java

If this reports that java is not found, then you need to add the JDK directory to your PATH environment variable. This is best added in the USS /etc/profile so that all users can use Java.

BPXPRM parametersThe SYS1.PARMCLIB member BPXPRMxx contains parameters which are used when USS is initialized. The parameters that are currently in effect can be displayed by the command /D OMVS,OPTIONS (Example 2-22).

Example 2-22 SYS1.PARMLIB(BPXPRMxx) displayed by /D OMVS,OPTIONS

D OMVS,OPTIONS BPXO043I 14.25.36 DISPLAY OMVS 343 OMVS 000D ACTIVE OMVS=(00,V8) CURRENT UNIX CONFIGURATION SETTINGS: MAXPROCSYS = 900 MAXPROCUSER = 200 MAXFILEPROC = 10000 MAXFILESIZE = NOLIMIT MAXCPUTIME = 2147483647 MAXUIDS = 200 MAXPTYS = 800 MAXMMAPAREA = 40960 MAXASSIZE = 2147483647 MAXTHREADS = 15000 MAXTHREADTASKS = 5000 MAXCORESIZE = 2147483647 MAXSHAREPAGES = 131072 IPCMSGQBYTES = 2147483647 IPCMSGQMNUM = 10000 IPCMSGNIDS = 500 IPCSEMNIDS = 500 IPCSEMNOPS = 25 IPCSEMNSEMS = 1000 IPCSHMMPAGES = 524287 IPCSHMNIDS = 500 IPCSHMNSEGS = 500 IPCSHMSPAGES = 262144 SUPERUSER = BPXROOT FORKCOPY = COW


STEPLIBLIST = USERIDALIASTABLE= /etc/ualiastable PRIORITYPG VALUES: NONE PRIORITYGOAL VALUES: NONE MAXQUEUEDSIGS = 1000 SHRLIBRGNSIZE = 67108864 SHRLIBMAXPAGES = 4096 VERSION = / SYSCALL COUNTS = NO TTYGROUP = TTY SYSPLEX = NO BRLM SERVER = N/A LIMMSG = NONE AUTOCVT = OFF RESOLVER PROC = RESOLV1 AUTHPGMLIST = NONE SWA = BELOW

You can also make dynamic changes by using the SETOMVS command as follows:

SETOMVS MAXPROCUSER=5000

BPXPRMxx also contains the mount commands for your HFS structure. To display which HFS directories are mounted (available), you can use the /D OMVS, F command as shown in Example 2-23.

Example 2-23 Partial listing of HFSs mounted to USS

D OMVS,F BPXO045I 14.31.40 DISPLAY OMVS 353 OMVS 000D ACTIVE OMVS=(00,V8) TYPENAME DEVICE ----------STATUS----------- MODEAUTOMNT 23 ACTIVE RDWR NAME=*AMD/u PATH=/u HFS 124 ACTIVE RDWR NAME=OMVS.CITGCFG.HFS PATH=/cicstg/ctg610/config HFS 122 ACTIVE RDWR NAME=OMVS.CITGTRC.HFS PATH=/cicstg/ctg610/trace HFS 110 ACTIVE RDWR NAME=OMVS.CTG6127.HFS PATH=/usr/lpp/cicstg/ctg610

For more information about these settings, refer to z/OS V1R6 UNIX System Services Planning, GA22-7800.

Note: To make these parameter changes permanent, you must edit the BPXPRMxx member of your SYS1.PARMLIB library. The changes take effect after the next IPL.


CTGDBGIncluding a dummy DD statement in the Gateway daemon started task JCL results in extra messages being sent to STDOUT and STDERR (Example 2-24).

Example 2-24 Started task JCL specifying //CTGDBG DD DUMMY

//CITGR1G PROC // SET CTGHLQ='CTGV61' // SET CTGUSR='CITG.CTG610.STDENV(CITGR1G)' // SET LEOPTS='/' //CTG EXEC PGM=CTGBATCH,REGION=250M, // PARM='&LEOPTS./usr/lpp/cicstg/ctg610/bin/ctgstart -noinput ' //STEPLIB DD DSN=&CTGHLQ..SCTGLOAD,DISP=SHR // DD DSN=CITG.CITGR1G.USERLOAD,DISP=SHR //* DD DSN=CICSTS31.CICS.SDFHEXCI,DISP=SHR //STDOUT DD SYSOUT=* //STDERR DD SYSOUT=* //CTGDBG DD DUMMY //STDENV DD DSN=&CTGUSR.,DISP=SHR //*

We found it useful to have the details of the environment, especially the environment variables, which are routed to STDOUT at startup (Example 2-25)

Example 2-25 STDOUT with //CTGDBG DD DUMMY specified

CTG0826I CTGBATCH Parsed STDENV entry _BPX_SHAREAS=YES CTG0826I CTGBATCH Parsed STDENV entry CICSCLI=/cicstg/ctg610/config/CITGR1G.ini CTG0826I CTGBATCH Parsed STDENV entry CLASSPATH=/usr/lpp/cicstg/ctg610/classes/ctgsamples.jarCTG0826I CTGBATCH Parsed STDENV entry COLUMNS=80 CTG0826I CTGBATCH Parsed STDENV entry CTG_RRMNAME=CTG.RRM.CITGR1G.IBM.UA CTG0826I CTGBATCH Parsed STDENV entry DFHJVPIPE=CITGR1G CTG0826I CTGBATCH Parsed STDENV entry DFHJVSYSTEM_00=A6POTGR1-Primary serverCTG0826I CTGBATCH Parsed STDENV entry PATH=/bin:/usr/lpp/java142/J1.4/bin CTG0826I CTGBATCH Parsed STDENV entry TMPDIR=/cicstg/ctg610/config CTG0826I CTGBATCH Parsed STDENV entry TZ=GMT-2 CTG0813I CTGBATCH RLIMIT_AS reports current=258M, system max=2047M CTG0815I CTGBATCH CWD=/CITG/R1 CTG0817I CTGBATCH PID=83886125 CTG0818I CTGBATCH LOCALE=C CTG0819I CTGBATCH POSIX=1 USS Version=5 CTG0811I CTGBATCH Runtime env PATH=/bin:/usr/lpp/java142/J1.4/bin CTG0811I CTGBATCH Runtime env CICSCLI=/cicstg/ctg610/config/CITGR1G.ini CTG0811I CTGBATCH Runtime env TMPDIR=/cicstg/ctg610/config CTG0811I CTGBATCH Runtime env TZ=GMT-2 CTG0811I CTGBATCH Runtime env _BPX_SHAREAS=YES CTG0820I CTGBATCH Userid=CITGR1G UID=13 GID=7100. CTG0821I CTGBATCH Initial dir=/CITG/R1 Initial program=/bin/sh.


CTG0825I CTGBATCH Spawned child PID=67109054 CTG6400I CICS Transaction Gateway is starting CTG8400I Using configuration file /cicstg/ctg610/config/CITGR1G.ini. CTG6577I Java version is 1.4.2 CTG6502I Initial ConnectionManagers = 1, Maximum ConnectionManagers = 100 CTG6526I Initial Workers = 1, Maximum Workers = 100 CTG6738I XA transaction support is not enabled CTG6898I 250 EXCI pipes are available for use by the CICS Transaction Gateway. CTG6981I Successfully initialized JNI library CTG6505I Successfully created initial ConnectionManager and Worker threads CTG6524I Successfully started handler for the tcp: protocol on port 13101 CTG6512I CICS Transaction Gateway initialization complete

LEOPTSThe LEOPTS parameter on CTGBATCH allows Language Environment override options to be passed to LE. If // SET LEOPTS='/' in Example 2-18 on page 53 is replaced with the following, then the LE runtime options and storage report is routed to STDERR:

// SET LEOPTS='RPTOPTS(ON),RPTSTG(ON)/'

The df . commandIssuing the command df . on a USS command line gives information about the mount table entry for the current HFS.

Example 2-26 Use of the df . command to show filesystem mount information

CITGCA: /cicstg/ctg610/config>df . Mounted on Filesystem Avail/Total Files Status /cicstg/ctg610/config (OMVS.CITGCFG.HFS) 1208/1440 4294967289 Available CITGCA: /cicstg/ctg610/config>

Changing the code page for CICS TG messages We installed the CICS TG using the IBM-1047 code page. It is possible to convert the installation to another code page by running the following command:

<install_path>/bin/ctgmsgs <language code> <code set>

Tip: The PARM string is limited to 100 bytes. If you find that you need to code very long property strings, you can use the CTGSTART_OPTS environment variable in STDENV.


The user ID which runs this command needs write permission to the <install_path>/bin directory. Converting the installation to another code page will not affect messages used by the CTGBATCH program.

To display the supported language and code set combinations, issue the command (Example 2-27):

<install_path>/bin/ctgmsgs -?

Example 2-27 Output of ctgmsgs -?

This utility is used to change the language of user messages.------------------------------------------------------------- Language Locale Code Set --------------------- -------------- ---------- en US English En_US IBM-1047 ja Japanese Ja_JP IBM-939 Ja_JP.IBM-930 IBM-930 zh Simplified Chinese Zh_CN IBM-935 The syntax of this command is: ctgmsgs <Language> <Code Set>

Error with bad permissions on the directoryWe tried to start the Gateway daemon when we had not set the correct permissions to allow it to traverse the /cicstg/ctg610/config directory. The result was that the Gateway daemon failed to start and messages shown in Example 2-28 were written to the Gateway daemon job log.

Example 2-28 Starting Gateway daemon with incorrect permissions

IEF695I START CITGR1G WITH JOBNAME CITGR1G IS ASSIGNED TO USER CITGR1G , GROUP CITG $HASP373 CITGR1G STARTED $HASP100 BPXAS ON STCINRDR $HASP373 BPXAS STARTED ICH408I USER(CITGR1G ) GROUP(CITG ) NAME(CTG GATEWAY DAEMON U) 324 /cicstg/ctg610/config/CITGR1G.ini CL(DIRSRCH ) FID(01D6C5F0F0F0F100010400002DFB0000) INSUFFICIENT AUTHORITY TO OPEN ACCESS INTENT(--X) ACCESS ALLOWED(OTHER ---) EFFECTIVE UID(0000000013) EFFECTIVE GID(0000007100) $HASP395 CITGR1G ENDED

We needed to set execute permission on the directories for our group using the ISHELL command, specifying the relevant directory, and using menu selection Directory → Attributes → Edit → Owning Group as shown in Figure 2-4 on page 36.


Listing libraries in the LINKLISTWe found it helpful to check the libraries in the LINKLIST using the MVS command /D PROG,LNKLST, which gave the results in the log as shown in Example 2-29.

Example 2-29 Displaying libraries in the LINKLIST

D PROG,LNKLST CSV470I 16.36.01 LNKLST DISPLAY 141 LNKLST SET LNKLSTLA LNKAUTH=LNKLSTENTRY APF VOLUME DSNAME 56 A CICS31 SYS1.CICSTS31.CICS.SDFHLINK 57 A CICS31 CICSTS31.CICS.SDFHEXCI

2.9.3 TracingCICS TG on z/OS provides three types of trace:

� Gateway trace� JNI trace� EXCI trace

The Gateway daemon and JNI traces can be started statically (where the trace parameters are specified at startup) or dynamically (where the trace parameters are specified during the normal operation of the Gateway daemon).

The EXCI trace can be formatted from a dump of the Gateway daemon address space, or routed to the MVS Generalized Trace Facility (GTF).

Gateway trace The Gateway trace captures Gateway daemon requests and responses.

Static trace (Gateway daemon)You can specify the following parameters on the ctgstart command when starting the Gateway daemon:

� trace=on (enables trace)� tfile=<pathname> (specifies the destination of the Gateway trace)

The Gateway daemon must be stopped and restarted in order to change these settings. Setting the trace options statically is useful on occasions when you need to trace Gateway daemon initialization.


Dynamic trace (Gateway daemon)With CICS TG V6, it is also possible to administer Gateway tracing dynamically from the SDSF console using the following commands:

/F <job_name>,APPL=TRACE,TLEVEL=0|1|2|3|40 No trace information is output.1 Exception tracing. Only exceptions are traced. 2 Trace exceptions, and entry and exit of methods. 3 Trace exceptions, some internals, and entry and exit of methods.4 Full debug tracing.

/F <job_name>,APPL=TRACE,TFILESIZE= filesize

Specifies the maximum size, in kilobytes, of the Gateway trace output file, for example 50000.

/F <job_name>,APPL=TRACE,DUMPOFSET

Specifies the offset from which displays of any data blocks start, for example 512. If the offset is greater than the total length of data to be displayed, an offset of 0 is used.

You cannot use this together with the fulldatadump option.

/F <job_name>,APPL=TRACE,TRUNCATIONSIZE=

Specifies the byte at which to stop the hex dump, for example 2000. It defines the endpoint, not the number of bytes to display. So if on a dump of size 40 you set the dumpoffset to 11, and the truncationsize to 25, you will see 15 bytes (from 11 to 25).

You cannot use this together with the fulldatadump option.

/F <job_name>,APPL=TRACE,FULLDATADUMP

Sets the dumpoffset to 0 and ignores any value specified in truncationsize.

The current trace settings can be displayed by issuing the following command (Example 2-30).

/D <job_name>,APPL=TRACE


Example 2-30 Results of the command /D CITGR1G,APPL=TRACE

BPXM023I (CITGR1G) CTG8239I Response received from CICS Transaction GatewayGateway Daemon trace settings: tlevel=0 truncationsize=80 dumpoffset=0 tfile=/cicstg/ctg610/trace/CITGR1G.gwytrc tfilesize=0 JNI trace settings: jnilevel=0 jnifile=/cicstg/ctg610/trace/CITGR1G.jnitrc

To demonstrate the Gateway trace we issued the command /F CITGR1G,APPL=TRACE,TLEVEL=2 (Example 2-31) and used the supplied sample EciB1 to send a request to the gateway daemon.

Example 2-31 Activating Gateway trace with /D CITGR1G,APPL=TRACE, TLEVEL=2

BPXM023I (CITGR1G) CTG8239I Response received from CICS Transaction Gateway Gateway Daemon trace settings successfully changed: tlevel=2

The Gateway trace is written to the destination specified in the tfile configuration parameter, in our case /cicstg/ctg610/trace/CITGR1G.gwytrc.

Example 2-32 shows sample Gateway trace output of an ECI request for CICS APPLID A6POTGR1 and an error response Rc=-3 (ECI_ERR_NO_CICS).

Example 2-32 Edited version of our Gateway trace

12:38:33:038 Thread-2: .MBeanServerImpl:<- [setAttributes] = [tracelevel=2] 12:38:36:335 ConnectionManager-9: .GatewayRequest:-> [readPaddedString] (java.io.DataInputStream@48feeec0, 3) 12:38:36:335 ConnectionManager-9: .GatewayRequest:<- [readPaddedString] = ECI 12:38:36:335 ConnectionManager-9: S-C: -> [ClientMessages: getMessage()] (2, null, ECI, 500010, 1, 0, 96) 12:38:36:335 ConnectionManager-9: S-C: -> [ClientMessages: getMsg()] (null, 2, 0) 12:38:36:335 ConnectionManager-9: S-C: <- [ClientMessages: validateResourceBundle()] = com.ibm.ctg.client.ResourceWrapper@6966eec112:38:36:336 ConnectionManager-9: S-C: <- [ClientMessages: getMsg()] = CTG6602I GatewayRequest type = ECI, flow version = 500010, flow type = 1, Gateway return code = 0, length of data following the header = 9612:38:36:336 ConnectionManager-9: S-C: CTG6602I GatewayRequest type = ECI, flow version = 500010, flow type = 1, Gateway return code = 0, length of data following the header = 96.


12:38:36:393 Worker-9: S-C: CTG6695I Before JNI CcicsECI(1) : Server=A6POTGR1, Program=EC01, Commarea.Length=18, ExtendMode=0, LUW=0.12:38:36:451 Worker-9: S-C: CTG6698I After JNI CcicsECI(1) : Rc=-3, LUW=0, Null stripped commmarea len=0 . 12:38:36:452 Worker-9: .ServerECIRequest:<- [getCicsRcString()] = ECI_ERR_NO_CICS 12:38:36:453 Worker-9: S-C: CTG6721I ECIRequest: Call_Type = ECI_SYNC, Cics_Rc = ECI_ERR_NO_CICS, Abend_Code = , Luw_Token = 0, Message_Qualifier = 0.. 12:38:36:455 Worker-9: S-C: CTG6515I Finished work request. [Worker-9]

JNI traceThe JNI trace captures information about ECI requests and the EXCI flows which the Gateway daemon initiates.

Static trace (JNI)The following environment variables can be specified in STDENV to enable JNI tracing at startup. This is useful if you need to investigate JNI problems which occur during Gateway daemon startup.

CTG_JNI_TRACE=<pathname>CTG_JNI_TRACE_ON=YES

Dynamic trace (JNI)The JNI trace can also be started and stopped dynamically using the SDSF MODIFY command:

/F <job_name>,APPL=TRACE,JNILEVEL=0|1

0 No trace information is output.

1 On.

Tips: The Gateway and JNI trace destinations cannot be set dynamically. If you do not define specific destinations for the Gateway and JNI traces the output is written to STDERR. If both traces are written to STDERR, trace entries are interleaved making it easier to diagnose problems.

More than one modify command can be issued at one time, for example, the command /F CITGR1G,APPL=TRACE,JNILEVEL=1,TLEVEL=2 sets both JNI and Gateway tracing on.


Example 2-33 shows sample JNI trace output of an ECI request for CICS APPLID A6POTGR1 and an EXCI response 8, EXCI reason 203 (NO_CICS).

Example 2-33 Edited version of our JNI trace

CTG6865I ECI Server List Entry Parameters. Number of Systems = 20. CTG6866I ECI Server List Exit Parameters. Cics_RC = 0. CTG6800I ECI parameters on entry. Call_Type = 1, Extend_Mode = 0, Luw_Token = 0, Commarea_Length = 18, Cics_Rc = 0, Flags = 0, outbound Commarea length = 18.CTG6801I Performed parameter conversion. parameter name = jstrServer, parameter value = A6POTGR1. CTG6801I Performed parameter conversion. parameter name = jstrProgram, parameter value = EC01 . CTG6802I Input commarea information. commarea length = 18, commarea address = fac5588. CTG8602I JNI Method zos_SetContext entry CTG8604I JNI Method zos_SetContext exit (rc = 0). CTG6822E EXCI function error. Function Call = 3, Response = 8, Reason = 203, Subreason field-1 = 0x5c, subreason field-2 = 0x00, Cics_Rc = -3.CTG6870I EXCI Open pipe gave a Retryable Response. Allocate, open will be retried a further 5 times. CTG6822E EXCI function error. Function Call = 3, Response = 8, Reason = 203, Subreason field-1 = 0x5c, subreason field-2 = 0x00, Cics_Rc = -3.CTG6822E EXCI function error. Function Call = 3, Response = 8, Reason = 203, Subreason field-1 = 0x5c, subreason field-2 = 0x00, Cics_Rc = -3.CTG6822E EXCI function error. Function Call = 3, Response = 8, Reason = 203, Subreason field-1 = 0x5c, subreason field-2 = 0x00, Cics_Rc = -3.CTG6822E EXCI function error. Function Call = 3, Response = 8, Reason = 203, Subreason field-1 = 0x5c, subreason field-2 = 0x00, Cics_Rc = -3.CTG6822E EXCI function error. Function Call = 3, Response = 8, Reason = 203, Subreason field-1 = 0x5c, subreason field-2 = 0x00, Cics_Rc = -3.CTG6805I ECI parameters on exit. Call_Type = 1, Extend_Mode = 0, Luw_Token = 0, Commarea_Length = 18, Cics_Rc = -3, AV = 0. CTG8602I JNI Method zos_ResetContext entry CTG8632I No XID present on ECI Request. CTG8604I JNI Method zos_ResetContext exit (rc = 0).

We see that the Gateway daemon attempted to call program EC01 but received an error. Because this is a retryable error (Response code = 8) the request is retried a further five times before the response Rc=-3 is passed back to the application.

Note: Details of the EXCI response and reason codes can be found in the CICS External Interfaces Guide. Subreason codes are also documented in hex in the DFHIRSDS member of CICSTS31.CICS.SDFHMAC.


EXCI traceThe CICS TG writes trace entries to the EXCI trace. The trace entries are in the CICS trace EXCI format, so the trace entries in a dump can be printed using standard z/OS utilities.

For detailed information about using EXCI trace, see Appendix C, “EXCI Trace” on page 385.


Part 2 Connection Management

In this part, we provide detailed information about how we connected WebSphere Application Server to CICS using the CICS TG. We cover two different topologies:

� Running WebSphere Application Server on Windows� Running WebSphere Application Server on z/OS

We discuss the configuration of connections within WebSphere Application Server, the Gateway daemon, and CICS.

Part 2


Chapter 3. Gateway daemon connections

This chapter provides details on how we configured our test environment to allow our J2EE application running on WebSphere Application Server on Windows to connect through the CICS Transaction Gateway on z/OS to an application running in CICS Transaction Server on z/OS.

First, we document how we prepared the system and the settings that we used. Then, we provide details on how we set up and modified the connections between each of the following tiers in our test environment:

� Configuring WebSphere Application Server� Configuring the Gateway daemon� Configuring EXCI� Configuring CICS

Finally, in 3.7, “Testing the configuration” on page 106, we explain how we tested the environment.

3


3.1 PreparationIn our testing, we deployed a sample J2EE application on WebSphere Application Server for Windows and connected to CICS using the CICS ECI resource adapter provided by CICS TG for z/OS.

This chapter concentrates on how to configure the connections between each tier (Figure 3-1). It does not provide details on how to install all of the software components, and it assumes a working knowledge of CICS or WebSphere Application Server where appropriate.

Figure 3-1 Software components: Connection management scenario

Full details about how to install the CICS TG for z/OS and run the IVP are provided in Chapter 2, “Installing and configuring the CICS TG” on page 25.

Windows

COBOLApplication

CICS TS V3.1

z/OS

EXCI

JNI

GatewayDaemon

TCP

CICS TG V6.1HTML


ServletJSP

EJBCICS ECIresourceadapter

CCI

CICS TG V6.1


3.1.1 Software checklistTable 3-1 shows the levels of software that we used for this configuration.

Table 3-1 Software checklist

The CICS ECI resource adapters that are supplied with the CICS TG V6.1 for z/OS are supported with WebSphere Application Server V6.0 (32-bit) or later versions.

You can use earlier versions of the CICS ECI resource adapter to connect to a CICS TG V6.1 daemon. You must use a version of the resource adapter that is supported with the specific version of WebSphere Application Server that you are using.

3.1.2 Definitions checklistTable 3-2 lists the definitions that we used to configure the scenario. These values are the same that we used in Chapter 2, “Installing and configuring the CICS TG” on page 25.


Windows z/OS

Internet Explorer V6.0 z/OS V1R6

Windows 2000 SP4 CICS Transaction Gateway for z/OS V6.1

IBM WebSphere Application Server Network Deployment V6.0.2

IBM SDK for z/OS, Java 2 Technology Edition, V1.4.2 SR2.

CICS Transaction Server for z/OS V3.1

Note: Although we installed WebSphere Application Server Network Deployment, we did not use any of the Network Deployment clustering functions and only configured a base application server environment.

Value WebSphere Gateway daemon CICS TS

hostname cam21-pc11 tx1.mop.ibm.com -

IP address (dhcp) 9.100.193.122 -

TCP/IP port - 13101 -

Job name - CITGR1G CITGR1CI

APPLID - - A6POTGR1

NETNAME - CITGR1G -

CONNECTION - - GR1G

Chapter 3. Gateway daemon connections 79

3.2 Configuring WebSphere Application ServerIn this section, we discuss the configuration of our WebSphere Application Server on Windows 2000. We detail the following steps:

� Installing the resource adapter� Creating connection factories� Deploying the sample J2EE application

3.2.1 Installing the resource adapterThe CICS TG V6.1 now provides two JCA resource adapters, cicseci.rar and cicseciXA.rar. In our scenario, we used the cicseci.rar, which provides the same level of local transaction support as provided previously in CICS TG V6.0 and earlier versions. The cicseciXA.rar provides additional XA transactional support and is covered in Chapter 5, “Gateway daemon transactions” on page 157.

Transferring cicseci.rar to the workstationIn order to install the resource adapter into WebSphere, we first needed to transfer the cicseci.rar file from z/OS to our Windows workstation. We used the Windows ftp command line utility to connect to the z/OS FTP daemon and download the file from the HFS directory /usr/lpp/ctg/ctg610/deployable to our local c:\temp directory (Example 3-1).

Example 3-1 FTP of cicseci.rar to workstation

C:\temp>ftp tx1.mop.ibm.comConnected to tx1.mop.ibm.com.220-FTPD11 IBM FTP CS V1R6 at TX1.MOP.IBM.COM, 16:27:14 on 2005-09-16.220 Connection will close if idle for more than 5 minutes.User (tx1.mop.ibm.com:(none)): citgt2g331 Send password please.Password:230 CITGT2G is logged on. Working directory is "CITGT2G.".ftp> cd /usr/lpp/cicstg/ctg610/deployable250 HFS directory /usr/lpp/cicstg/ctg610/deployable is the current working directory.ftp> bin200 Representation type is Imageftp> get cicseci.rar200 Port request OK.125 Sending data set /usr/lpp/cicstg/ctg610/deployable/cicseci.rar250 Transfer completed successfully.ftp: 750023 bytes received in 0.16Seconds 4777.22Kbytes/sec.ftp>

Note: Because both resource adapters provide the same CCI application interface and connection management capability, our sample J2EE application CTGTesterCCI can be used with either resource adapter.


To install the resource adapter into WebSphere, do the following:

1. Start the Application Server and log into the Administrative Console using the following URL:

http://cam21-pc11:9060/ibm/console/

2. In the Administrative Console Welcome panel, click Resources and then Resource adapters. The panel in Figure 3-2 displays.

Figure 3-2 Administrative Console - installing RAR 1

3. Click Install RAR. In the Install RAR file panel, click Local Path and then Browse to locate the resource adapter cicseci.rar on the Windows workstation (Figure 3-3).



4. Click Next and then OK when the Resource Adapters panel displays. The ECIResourceAdapter is now added to the list of installed adapters (Figure 3-4).


5. Add the changes to the master repository by clicking Save and then clicking Save again.

3.2.2 Creating connection factoriesEach installed resource adapter should have one or more connection factories associated with it. The connection factory is used to define the connection to the target CICS system. As such, it must contain the following information:

� The CICS region that will be the target of requests via this connection factory.� The connection details of the Gateway daemon that will be used.

In addition, the connection factory can contain other optional information, such as:

� The security credentials that will be used for communication.� The mirror transaction name that should be used.

Note: When installing the CICS TG V6.1 resource adapters, it is possible to connect to Gateway daemons of a lower version or release level. If it is necessary to interoperate with Gateway daemons of a lower version or release, then you must maintain separate application servers, each with the appropriate level of resource adapter.

Conversely, application servers with resource adapters from a lower CICS TG version or release can connect to Gateway daemons of a higher version or release but will not be able to use all of the features available at the version or release.


Creating a connection factoryThis section explains how to create and configure a connection factory for the CICS ECI resource adapter. This connection factory is used in the next sections to test the connectivity to the CICS region.

To create a connection factory:

1. From the Resource Adapters panel (Figure 3-4 on page 82), click ECIResourceAdapter and the Configuration panel (Figure 3-5) displays. This panel lists the General Properties and Additional Properties of the resource adapter. There is usually no need to change the general properties of the resource adapter.

Figure 3-5 Administrative Console - Creating a connection factory 1

Tip: At least one connection factory is required to use a resource adapter. However, it is possible to create multiple connection factories each with different options (such as differing CICS region names) to allow different applications to use connections with different properties or to different CICS regions.


2. Click J2C connection factories and then click New, which displays the General Properties panel of the connection factory to be created (Figure 3-6).


There are many properties displayed in this window. However, the only important property at this stage is the Name field.

We entered the value CITGR1CI-tx1-13101, which is a concatenation of the name of the target CICS region, the TCP/IP host name and the port the Gateway daemon is using.

3. Enter a description of this connection factory and click Apply.


4. After applying these settings, the Additional Properties links are enabled. Click Custom Properties, which takes you to the CICS ECI resource adapter custom properties panel (Figure 3-7).


5. The custom properties are the parameters that are used by the CICS ECI resource adapter to communicate with a Gateway daemon and a target CICS region. Set the values as shown in Figure 3-7 and then save the configuration.

In our scenario, we entered the following settings:

– ConnectionURL

This is the URL that defines the TCP/IP host name and protocol that are used to communicate with the Gateway daemon. We used the value tcp://tx1.mop.ibm.com.

– PortNumber

This is the TCP/IP port number on which the Gateway is listening. It defaults to 2006. We used the value 13101.

– ServerName

This is the APPLID of the CICS region to which requests are sent. We used the value A6POTGR1. If it is not set, then the value from the DFHJVSYSTEM_00 variable is used by the Gateway daemon.


– TPNName

This is the CICS TPN name and is used as the name of the attached mirror transaction that requests run under in CICS. It overrides TranName and defaults to CSMI when using the CICS TG on z/OS. We used the value ECIT, which we then had to define as a private mirror transaction in our target CICS region (see 3.5.2, “Adding a private mirror” on page 101).

– SocketConnectTimeout

This is the timeout in milliseconds that a socket connection request will wait for when an attempt is made to connect to a remote Gateway daemon. We used the value 10.

We accepted the defaults for the following parameters:

� UserName

This is the user ID to be flowed to the Gateway and onto CICS.

� Password

This is the password to be flowed to the Gateway.

� TranName

This is the name CICS uses for the EIBTRNID of the mirror transaction. It does not affect the name of the attached mirror transaction and is overridden by TPNName.

� ClientSecurity

This is the name of the CICS TG client security exit. If specified the exit must be added to the system classpath.

� ServerSecurity

This is the name of the CICS TG server security exit. If specified the exit must be added to the system classpath.

� KeyRingClass

This is the SSL keyring name for the JSSE keystore.

� KeyRingPassword

This is the password used to access the SSL keyring.

Tip: In some cases, for example if you plan to use a large number of different CICS private mirror names, it might be necessary for the application itself to set the TPNName or TranName dynamically. This can be done using the ECIInteractionSpec methods setTPNName() or setTranName().


� TraceLevel

This is the JCA trace level, and can take one of the following values:

0 Disable all tracing1 Output exception trace stacks2 Output method entry and exit stack traces3 Output debug trace entries

Connection pool settingsAfter you have defined the connection factory, click Connection pool properties on the resource adapter connection factory panel and enter values to tune the connection pool (Figure 3-6).

Figure 3-8 Administrative Console - connection pool settings


In our scenario, we entered the following settings:

� Connection timeout

Specifies how long, in seconds, a getconnection() request by the J2EE application can wait if there are no connections available in the free pool and no new connections can be created. If a timeout occurs, a ConnectionWaitTimeoutException is thrown. We set this to 10 seconds in order to queue requests but to timeout requests early if no connection becomes available. By default, the request waits for 180 seconds.

� Maximum connections

This is the maximum number of connections that can be created within the connection pool. We set this to 55 in order to demonstrate how to limit the number of simultaneous connections between WebSphere and a Gateway daemon. We also set the maximum number of Gateway connection manager threads to 55 (see 3.3.2, “Gateway threads” on page 92).

� Minimum connections

This is the minimum number of connections that will be maintained in the connection pool after the pool manager maintenance thread has run. We set this to 10, the same as the initial number of Gateway connection manager threads.

� Reap time

This is the interval at which the pool manager maintenance thread will run. We let this default to three minutes.

� Unused timeout

This is the amount of time that connections will be allowed to remain idle before being eligible for closure by the pool manager maintenance thread. We set this to 600 so that connections would be closed down after 10 minutes of inactivity.

� Aged timeout

This is the amount of time that connections will be allowed to age before being eligible for closure by the pool manager maintenance thread. We set this to zero (0) because we did not want old connections to be closed.

� PurgePolicy

This specifies how to purge connections when a fatal connection error is detected by the pool manager. We set this to Failing Connection Only, which means a connection is purged only if it is used and found to be bad. The alternative is EntirePool, and in this case the connection pool is flushed if a fatal connection error occurs.


After you set these values, click OK and then save the settings to the master repository.

3.2.3 Deploying the sample J2EE applicationIn this section, we describe how to deploy the test application CTGTesterCCI. For further details about our sample, refer to Appendix A, “Sample applications” on page 361.

To deploy the sample application, do the following:

1. In the WebSphere Application Server Administrative Console Welcome panel, click Applications → Install New Application. The panel in Figure 3-9 displays.

Figure 3-9 Administrative Console - installing the EAR

2. Enter the location of the EAR file (CTGTesterCCI.ear) and click Next twice and then click Continue when prompted with the Application Security Warnings page. You are then prompted with the details of the eight step process for installing new applications.

Tip: We noted that either terminating the Gateway or the CICS region would lead to a fatal connection error.


3. Click Next through each step until you reach Step 5 Map Resource References to Resources, where you perform the following actions:

a. Select the JNDI name eis/CITGR1CI-tx1-13101 for the existing Connection Factory that you just created.

b. Scroll down the page and select the EJB module CTGTesterCCIEJB in the J2EE application.

c. Scroll back to the top of the page and click Apply.

Figure 3-10 shows the final result. In the bottom panel, the resource reference ECI in the EJB CTGTesterEJBCCI is now mapped to the JNDI name eis/CITGR1CI-tx1-13101, which represents our connection factory. The resource reference ECI in the J2EE application now uses the connection factory CITGR1CI-tx1-13101 and the properties that are contained within it.

Figure 3-10 Administrative Console - mapping resource references


4. Click Next three times and then click Finish to exit the process. Confirm that the deployment was successful and save your changes to the WebSphere repository.

3.3 Configuring the Gateway daemonIn order to configure the CICS TG environment, you might need to make changes to both the Gateway daemon settings and the TCP/IP environment.

3.3.1 TCP/IP connections When configuring the Gateway daemon, consider the following items regarding the usage of TCP/IP.

Choosing a portThe Gateway daemon requires use of a TCP/IP port. This defaults to 2006, but we used 13101 for our CITGR1G Gateway. The Gateway fails to start if the protocol handler cannot bind to the port at startup. So, it is important that you coordinate which jobs can access which ports. You can achieve this coordination through the use of the TCP/IP PORT reservation statement, which limits the applications that can use a specific port. We added the following value to our TCP/IP profile data set OMVS1.TCPIP.TCPPARMS(PROFILE) to ensure that only the job named CITGR1G could bind to port 13101:

13101 TCP CITGR1G ; Production Gateway daemon

We then varied our changes to TCP/IP online using the MVS TCP/IP command:

/V TCPIP,,O,OMVS1.TCPIP.TCPPARMS(PROFILE)

IP address bindingBy default the CICS TG protocol handlers bind to all available IP addresses (this is referred to as INADDR_ANY) and to all available TCP/IP stacks. To allow you to control access to your z/OS system, it is likely you will want to control this behavior if you are using multiple stacks or IP addresses (including VIPA addresses).

Tip: You can use the USS netstat -o command to display the port reservation list for restricted ports.


On our system, we had two IP addresses (9.100.193.121 and 9.100.193.122) configured. We decided to restrict our Gateway daemon to using only the 9.100.193.122 address. So, we added a bind parameter to our Gateway configuration file CITGR1G.ini (Example 3-2).

Example 3-2 IP address binding in Gateway configuration file

[email protected][email protected]=port=13101;\ connecttimeout=2000;\ idletimeout=600000;\ pingfrequency=60000;\ bind=9.100.193.122;

Multiple IP stacksWe were only using a single TCP/IP stack. So, we did not need to make any modification to the TCP/IP bind behavior. However, if required, you can set the environment variable _BPXK_SETIBMOPT_TRANSPORT in STDENV to specify the name of an individual TCP/IP stack to which you want the Gateway daemon to bind.

For more details on TCP/IP subsystem management refer to Communications Server for z/OS V1R2 TCP/IP Implementation Guide Volume 1: Base and TN3270 Configuration, SG24-5227.

3.3.2 Gateway threadsThe Gateway daemon receives network requests from remote Java clients, and is itself a sophisticated multi-threaded Java application that runs in the UNIX System Services environment. It can handle multiple requests simultaneously and has a set of properties configured in the Gateway configuration file. Two pools of threads can be configured: the connection manager threads and the worker threads.

Tip: You can use the USS netstat -d command to list the configured IP adapters on you system.

Note: The IP address binding function that is provided by the bind parameter in the Gateway configuration file is new function in CICS TG V6.1. However, if you are running a previous version of the Gateway, you can achieve similar functionality through the use of the BIND control for INADDR_ANY function in TCP/IP.


For each connected client (such as a connection in the WebSphere connection pool), one connection manager thread is used in the Gateway daemon, and is held until the client closes the connection or the Gateway times out the connection. In order for an ECI call to be performed via an allocated connection manager thread, a thread must be allocated from the worker thread pool for the duration of the ECI request. This relationship is summarized in Figure 3-11.

Figure 3-11 CICS TG threading model

The connection manager threads limit the maximum number of connected Java clients, and the worker threads limit the number of concurrent ECI calls that can be issued by these attached clients. The initial and maximum numbers of these connection manager and worker threads are set in the Gateway configuration file using the maxconnect and maxworker parameters. Requests from connection manager threads for worker threads can be queued internally, but will be timed out according to the workertimeout parameter.

Gateway daemon

JavaGateway.open() Connect

ConnectionManagers Workers

JavaGateway.flow()

Wait

ECI request

ECI reply

Java client

JavaGateway.close()

ECI

Disconnect


Gateway configuration parametersTo improve the operational characteristics of our Gateway daemon, we made changes to our Gateway configuration file as shown in Example 3-3.

Example 3-3 Modified Gateway configuration file settings

closetimeout=10000initconnect=10initworker=10maxconnect=55maxworker=55nonames=onworkertimeout=0protocol@[email protected]=port=13101;\ connecttimeout=0;\ idletimeout=600000;\ pingfrequency=30000;\ bind=9.100.193.122;

Here is a description of the parameters that we changed:

� closetimeout=10000

This parameter controls the amount of time a connection manager waits for in progress requests to complete if a Java client disconnects before a worker thread has completed its work. We set this to 10,000 ms (10 seconds) to limit any delays in cleanup processing.

� initconnect=10

This is the initial number of connection manager threads that are allocated when the Gateway daemon starts. Having pre-allocated connection manager threads can improve the response time of the initial ECI requests. We set it to the same value as the minimum number of connections in the connection factory (see 3.2.2, “Creating connection factories” on page 82).

� initworker=10

This is the initial number of worker threads that are allocated when the Gateway daemon starts. Having pre-allocated worker threads can improve the response time of the initial ECI requests. We set it to the same value as initconnect.

� maxconnect=55

This is the maximum number of connection manager threads that can be used and corresponds to the maximum number of socket connections that the Gateway daemon accepts. We set this to 55 which is the same as the value that we specified for the maximum number of connections in the connection factory (see 3.2.2, “Creating connection factories” on page 82).


� maxworker=55

This is the maximum number of worker threads that can be used. We set this to 55 which is the same as the upper limit of transactions that can be active and queued in our CICS TRANCLASS (see Figure 3-13 on page 102).

� nonames=on

This parameter means that TCP/IP host names in messages will not be resolved to IP addresses. We set nonames=on because we did not want the Gateway daemon to use Domain Name Services (DNS) to convert IP names to addresses because this can cause a significant performance overhead.

� workertimeout=0

This parameter sets the amount of time that a connection manager thread can be suspended waiting for a free worker thread. We set this value to 0 as we did not wish to queue requests internally in our Gateway daemon.

� connecttimeout=0

This parameter sets the amount of time the TCP/IP protocol handler waits for a connection manager thread to become available after a connection has been established. We set this to zero (0) because we did not want to queue request internally in our Gateway daemon.

� idletimeout=600000

This parameter sets the amount of time the Gateway daemon keeps an unused connection alive. We set this to 60,000 ms (10 minutes), which is the same as the unused timeout setting in the connection factory.

� pingfrequency=30000

This parameter sets the amount of time between ping requests that are sent by the Gateway daemon to the resource adapter. We set this to 30,000 ms (30 seconds) because we wanted to decrease the amount of time that it took for the Gateway to clean-up after a network failure.


3.4 Configuring EXCI The EXCI is used by the CICS TG for z/OS to send requests to the CICS region specified in the ECI request. To make an ECI call, the CICS TG for z/OS uses the EXCI CALL interface. The EXCI CALL interface provides a low-level API for calling a CICS program using a COMMAREA, and consists of the following six commands which control the usage of the CICS sessions (which equate to EXCI pipes):

1. Initialize_User2. Allocate_Pipe3. Open_Pipe4. DPL_Request5. Close_Pipe6. Deallocate_Pipe

The Gateway daemon by default does not perform a Deallocate_Pipe after it flows the initial ECI request from any given worker thread (Figure 3-12).

Figure 3-12 Default Gateway daemon EXCI pipe usage

An EXCI pipe stays allocated for the lifetime of the thread and is only deallocated if one of the following events occurs:

� An error is received by the Gateway worker thread on an Open_Pipe call (such as when a CICS connection has been closed).

� The Gateway daemon terminates.

� The worker thread allocates another pipe to a different CICS applid, and the CTG_PIPE_REUSE environment variable is set to ONE (see “CTG_PIPE_REUSE value” on page 98 for a description of this variable).

Worker threads

CICS

• Allocate• Open• DPL request• Close

• Open• DPL request• Close

• Deallocate

EXCI1st request

Nextrequest

GatewayTermination

Gateway Daemon


For all connected Gateway daemons on z/OS the CICS region requires both a CONNECTION and SESSIONS definition. The CONNECTION definition identifies the remote system, and the SESSIONS definition associated with this CONNECTION determines the properties of the sessions. In the following sections, we provide further information about the system limits that apply to EXCI pipes and the values that we used in our configuration. You can find further information about using EXCI pipe in the CICS External Interfaces Guide, SC33-1944.

3.4.1 EXCI pipe limitsEvery address space which uses EXCI pipes (such as a Gateway daemon) is limited to a maximum number of pipes that it can allocate to all attached CICS regions. The EXCI pipe limit rules are as follows:

� In CICS TS V1.3, the limit is 100 EXCI pipes in total per calling address space.

� In CICS TS V2.2 or higher the PTF for APAR PQ92943 allows a MVS system wide LOGONLIM to be set, which controls the pipe limit. The upper limit is 250 and the default is 100.

� The maximum pipe usage in a CICS region is limited by the RECEIVECOUNT parameter defined in the CICS MRO SESSIONS definition, this can be up to 999 (depending on the number of characters in the RECEIVEPREFIX).

3.4.2 Our EXCI settingsIn our environment we made changes to the following systems setting to provide for increased throughput and more predictable response times when using the EXCI:

� EXCI LOGON limit� CTG_PIPE_REUSE value� TIMEOUT settings in DFHXCOPT

EXCI LOGON limitIn order to exploit an increased worker thread count in any of our Gateway daemons, we increased the EXCI pipe limit on our z/OS LPAR from the default of 100 to the upper limit of 250. We added the following line to our SYS1.PARMLIB1(DFHSSI00) member.

LOGONLIM=250

Tip: In CICS TG V6 additional diagnostic messages are written to STDERR by the CICS TG if pipe usage reaches 90% or 100% of the EXCI LOGONLIM.


We then had to IPL our LPAR in order to activate this change. For more detail about how to configure the LOGONLIM, refer to the CICS Transaction Server for z/OS Installation Guide V3.1, GC34-6326.

CTG_PIPE_REUSE valueBy default if a Gateway is used to connect to multiple CICS regions, then worker threads can allocate EXCI pipes potentially to multiple regions. If this occurs, it can cause unpredictable pipe shortage issues. In CICS TG V6.0, a new environment variable called CTG_PIPE_REUSE was introduced. If this environment variable is set to ONE, then a worker thread only allocates one pipe and deallocates this pipe if it needs to reallocate the pipe to a different CICS region. This provides a more predictable usage model but with a small cost in performance. The exact performance cost depends on the frequency of pipe deallocations.

The default setting for CTG_PIPE_REUSE is ALL, which gives the same pipe allocation behavior as in CICS TG V5 (that is, worker threads can allocate multiple pipes to multiple CICS regions).

We set the value CTG_PIPE_REUSE=ONE in the STDENV for our Gateway CITGR1G. Because we only had one CICS region in our setup, this setting had no immediate effect. However, setting it is good practice if you anticipate that the Gateway daemon will connect to multiple CICS regions at some time in the future, and your aim is to ensure a predictable operating environment.

TIMEOUT settings in DFHXCOPTIn our testing, we decided that we required an EXCI timeout setting to ensure that our Gateway would not wait indefinitely for transactions that might have stalled in CICS. This setting can be achieved by setting a TIMEOUT in the EXCI options table DFHXCOPT. This setting is then used by the EXCI to terminate any requests that have been waiting too long for a DPL_Request command to complete.

Attention: Because the EXCI call Deallocate_Pipe has a cost associated to it, setting CTG_PIPE_REUSE=ONE can have a detrimental affect on performance if your Gateway daemon is servicing EXCI requests for many different CICS regions. This cost has been mitigated to some extent in CICS TS V2.3, so we recommend that is the minimum level if using this function. Note, however, that the benefit of its use is that it gives a predictable usage model that allows you to predict accurately absolute pipe usage, irrespective of what connections are made.


We created a DFHXCOPT table (Example 3-4) with a timeout value of 90 seconds (which is equivalent to 9000 hundreths).

Example 3-4 EXCI options table, DFHXCOPT

*********************************************************************** DFHXCO TYPE=CSECT, * TIMEOUT=9000, 90 seconds * TRACE=OFF, Only Exception trace entries * TRACESZE=16, 16K trace table * DURETRY=30, Retry SDUMPS for 30 seconds * TRAP=OFF, DFHXCTRA - OFF * GTF=OFF, GTF - OFF * MSGCASE=MIXED, Mixed case messages * CICSSVC=0, EXCI will obtain CICS SVC number * CONFDATA=SHOW, Show user commarea data in trace * SURROGCHK=YES Perform surrogate-user check END DFHXCOPT **************************** Bottom of Data ****************************

In our testing, we assembled the table and link-edited this into our CICS TG LOADLIB CITG.CITGR1G.USERLOAD, which we added to our STEPLIB DD statement ahead of the CICS SDFHEXCI library, as shown in Example 3-5. We also needed to define the CITG.CITGR1G.USERLOAD library as program controlled.

Example 3-5 Updated STEPLIB

//CITGR1G PROC // SET CTGHLQ='CTGV61' // SET CTGUSR='CITG.CTG610.STDENV(CITGR1G)' //CTG EXEC PGM=CTGBATCH,REGION=0M, // PARM='&LEOPTS./usr/lpp/cicstg/ctg610/bin/ctgstart -noinput ' //STEPLIB DD DSN=&CTGHLQ..SCTGLOAD,DISP=SHR // DD DSN=CITG.CITGR1G.USERLOAD,DISP=SHR

Note: Although the EXCI terminates the TCB running the request (and thus the Gateway worker thread), the task in CICS will not abend until it tries to return data back to the Gateway. Thus, in a CICS stall condition, it is quite possible for the CICS application to abend after the Gateway worker thread has returned the error back to the application.


3.4.3 EXCI user-replaceable module: DFHXCURMThe user-replaceable module DFHXCURM is invoked on every EXCI Allocate_Pipe request and also after detection of any EXCI retryable errors. This occurs under one of three circumstances:

� The target CICS region is not available.� There are no pipes available on the target CICS region.� IRC has not been active since the last IPL.

DFHXCURM can be used to remove the affinity of an EXCI request to a given CICS region by dynamically modifying the APPLID in the EXCI request. It can also be used to provide limited workload balancing of EXCI requests across multiple CICS regions from the CICS TG for z/OS.

3.4.4 Transactional EXCITransactional EXCI works together with Recovery Resource Services (RRS), a component of z/OS, in allowing multiple EXCI calls to become part of one logical unit of work. This means that the CICS TG for z/OS can be used by other transactional systems (such as WebSphere Application Server) to make transaction requests to CICS, which can be coordinated using a two-phase commit mechanism between the two systems. For full details on enabling transactions with the CICS TG see Part 3, “Transaction Management” on page 155.

3.5 Configuring CICS In this section, we provide details on the modifications that we made to our CICS region to provide a more production-like environment. We cover the following settings.

� CICS CONNECTION and SESSION values� Adding a private mirror � Transaction timeouts

Note: We allowed the value of CICSSVC in the DFHXCOPT table to default to zero (0), which means that EXCI gets the CICS SVC number from z/OS. Specify 0 only when you are sure that at least one CICS region has logged on to DFHIRP during the life of the z/OS IPL. If you use the default and the EXCI requests the SVC from z/OS, the request fails if no CICS region has logged on to DFHIRP. Alternatively, you can specify the same SVC number that is used by the CICS MRO regions that reside in the z/OS image where the Gateway daemon is running.


For further details about the basic CICS configuration changes that we made to enable communication from our CICS TG, refer to Chapter 2, “Installing and configuring the CICS TG” on page 25.

3.5.1 CICS CONNECTION and SESSION valuesThe process of creating CICS CONNECTION and SESSIONS definitions is described in Figure 2-6 on page 44 and Figure 2-7 on page 45. However, for the purpose of exploiting our increased pipe limit, we updated the EXCI session count in our SESSIONS definition by setting the value RECEIVECOUNT=999. We then activated this change by closing IRC, discarding the existing connection, installing the new definitions, and re-opening IRC.

3.5.2 Adding a private mirrorRather than running our ECI request under the default mirror transaction (CSMI) in our testing environment, we decided to define our own private mirror. This decision has the following advantages:

� It allows individual CICS monitoring and statistics information to be recorded for specific applications.

� It allows specific security attributes to be set for the transaction.

� It allows a specific TRANCLASS to be specified, which can control the queuing and scheduling priorities of the CICS application relative to the other transactions executing in the CICS region.

To define our private mirror, we performed the following steps:

1. We defined a mirror transaction ECIT, by copying the default mirror transaction definition CSMI to a new RDO group using the following command:

CEDA COPY GR(DFHISC) TRAN(CSMI) AS(ECIT) GROUP(ECIPROG)

2. We updated this definition to specify a TRANCLASS of TCLASS1 using the following command:

CEDA ALTER TRANS(ECIT) GROUP(ECIPROG) TRANCLASS(TCLASS1).

Note: Under very fast moving workloads, it is possible that closed EXCI pipes are not freed up quickly enough to be re-used by incoming open requests. So, if you set the RECEIVECOUNT to something less than the maximum 999, you should make sure that you set it slightly greater than the maximum potential number of EXCI pipes that will be allocated to your CICS region.


3. We defined a new TRANCLASS resource definition to control the attributes of the ECIT transaction with a MAXACTIVE value of 50 and a PURGETHRESHOLD of 6 (Figure 3-13). This TRANCLASS limits our CICS region to running 50 simultaneous ECIT transactions and also queues a further five requests before it refuses to queue further transactions. This is significantly lower than the CICS region MAXTASKS value of 100, thus ensuring that the CICS region is not paralyzed with requests for the ECIT transaction.

Figure 3-13 CICS TRANCLASS definition

4. We then installed the ECIT transaction and the TRANCLASS using the following command and added the ECIPROG group to our CICS startup list TGR1:

CEDA INSTALL GROUP(ECIPROG)

3.5.3 Transaction timeoutsWe did not set any timeout parameters in our CICS environment (such as RTIMOUT or DTIMOUT), because neither of these were applicable to our scenario. However, we recommend that you do consider setting a timeout on the file manager (such as DB2), because it is generally perceived as best practice to time out suspended tasks as close as possible to the file operation likely to cause delays. For further details about the end-to-end timeout settings that we used in our testing, refer to “End-to-end settings” on page 107.

OVERTYPE TO MODIFY CICS RELEASE = 0640 CEDA ALter TRANClass( TCLASS1 ) TRANClass : TCLASS1 Group : ECIPROG Description ==> CLASS LIMITS Maxactive ==> 050 0-999 Purgethresh ==> 0000006 No | 1-1000000 SYSID=TGR1 APPLID=A6POTGR1 PF 1 HELP 2 COM 3 END 6 CRSR 7 SBH 8 SFH 9 MSG 10 SB 11 SF 12 CNCL


3.6 Configuring for high scalabilityAfter you have successfully configured and tested your CICS TG configuration, you should consider how you can clone the Gateway daemon in order to improve scalability and availability. The two principal areas for consideration are:

� How to load balance TCP/IP requests across multiple Gateway daemons

� How to load balance CICS program link requests dynamically across multiple CICS AORs

3.6.1 TCP/IP load balancingThe CICS TG on z/OS is designed for use with the z/OS TCP/IP load balancing topologies, due to its usage of multiple socket connections. You can use the following z/OS TCP/IP load balancing technologies:

� TCP/IP port sharing� Sysplex Distributor

TCP/IP Port SharingTCP/IP port sharing is a feature provided by z/OS Communications Server which allows multiple address spaces (such as Gateway daemons) to listen for TCP/IP requests on the same port. It functions by allowing incoming socket open requests to be distributed among the listening address spaces according to the socket backlog and current usage. It is a very simple feature to implement, and provides effective load balancing across multiple Gateway daemons listening on the same TCP/IP stack in the same LPAR.

For details about how we enabled TCP/IP port sharing, see 3.7.2, “Cloned Gateway daemon connectivity tests” on page 114.

Sysplex DistributorSysplex Distributor, an integral part of z/OS Communications Manager, offers the ability to load balance incoming socket open requests across different address spaces running on different TCP/IP stacks (usually on different LPARs). The routing decision is based on real-time socket status and MVS Quality of Service (QoS) criteria. This provides the benefit of balancing work across different MVS images, providing enhanced scalability and failover in a z/OS Parallel Sysplex.


The beauty of Sysplex distributor is that the client view of the parallel sysplex is of a single machine; all the complexity of where and how the work is done, is hidden from the client, but with all the benefits that a parallel sysplex environment brings to TCP/IP networking, including:

� Virtual IP Addressing (VIPA)

Virtual IP addressing (VIPA) allows TCP/IP on z/OS to create IP addresses which do not correspond to a physical connection to the network, but to which requests can be routed, over all the physical connections to the network known to that z/OS. This gives the big benefit that if client requests use a Virtual IP address when connecting to that z/OS machine, and one or more of the physical connections to the network is down, the request can be routed over one of the other physical connections. VIPA also allows z/OS applications to be associated with specific IP addresses.

� Dynamic VIPA

Dynamic VIPA allows the members of a parallel sysplex to share a single VIPA address. One member of the parallel sysplex is configured as the owner of the VIPA and another member is defined as the backup. If the member that owns the dynamic VIPA fails, the parallel sysplex automatically starts routing requests to the backup member.

� z/OS Workload Manager

z/OS Workload Manager and TCP/IP Quality of Service policies (specified using Policy Agent) are used to make intelligent decisions about which is the best instance of the Gateway daemon in the parallel sysplex to process a new request.

Note: One important aspect of Sysplex distributor is that the instances of the Gateway daemon must be identical clones, and must be able to handle any affinities which can arise during execution of a client request.


Figure 3-14 shows the use of both TCP/IP port sharing and Sysplex Distributor.

Figure 3-14 High scalability and availability configuration with the Gateway daemon on z/OS

3.6.2 Dynamic program linkFigure 3-14 shows the recommended high availability configuration with CICS TG on z/OS, which is to have a given Gateway daemon connecting to a CICS region on the same LPAR. Such a CICS region is referred to as a router region. This region does not run the CICS application itself but instead routes the program link request to a CICS AOR. CICSPLex SM provides a dynamic routing program which supports the dynamic routing of LINK commands. This provides the ability for applications invoked by ECI requests to be routed dynamically across a CICSplex.

The performance and transactional characteristics of an XCF EXCI connection are different from an XM EXCI connection:

� Certain limits apply to XCF group membership and these limits can be exceeded if many XCF EXCI sessions are allocated.

� If an ECI request is part of an extended logical unit of work or an XA transaction, the EXCI connection from the Gateway daemon must be an XM EXCI connection.

EXCI LPAR - 3

COBOLapplication

TCP/IP

Gatewaydaemon

TCP/IP Port Sharing

TCP/IP Port Sharing

CICS router region 1

LPAR -1

LPAR -2

SysplexDistributor

EXCI

z/OS sysplex

Gatewaydaemon

CICS AORs

DPL

CICS router region 2

DPL

Tip: We recommend that a Gateway daemon connects to CICS regions on the same LPAR.


It is possible to use the EXCI user exit DFHXCURM (see 3.4.3, “EXCI user-replaceable module: DFHXCURM” on page 100) to re-route EXCI requests to an alternative (or backup) CICS router region in case of a failure of the target CICS router region.

When using techniques such as Sysplex Distributor with the Gateway daemon, it is important to be aware of a potential affinity-related problem which might be created. For example, in Figure 3-14, if a request for CICS router region1 is routed by Sysplex Distributor to LPAR-1 then a cross-memory (XM) EXCI connection is used between the Gateway daemon and the CICS router. However, if the request is instead routed to LPAR-2 then a cross-system coupling facility (XCF) EXCI connection would be used instead (something that we do not recommend).

It is possible to use the EXCI user exit DFHXCURM to resolve this affinity problem by making sure that an EXCI XM connection is always used in preference to an EXCI XCF connection. DFHXCURM is able to dynamically change the APPLID of the target CICS region such that all requests which are routed to the Gateway daemons running on LPAR-1 are passed to CICS router 1 and all requests which are routed to the Gateway daemons on LPAR-2 are passed to CICS router 2.

3.7 Testing the configurationAfter we installed and configured the software components, we tested our configuration in two scenarios using:

� A single Gateway daemon� Two Gateway daemons configured to use TCP/IP port sharing

Restriction: If an ECI request is part of an extended logical unit of work or an XA transaction, the EXCI connection from the Gateway daemon must be to a CICS region running on the same LPAR. However, further XCF links to CICS regions across the CICSPlex® are supported, as shown in Figure 3-14.


3.7.1 Single Gateway daemon connectivity testsIn the following section, we detail how we tested our connectivity environment using our sample J2EE application CTGTesterCCI. Figure 3-15 shows the basic outline of our environment.

Figure 3-15 CTGTesterCCI remote connection to Gateway daemon

End-to-end settingsTable 3-3 summarizes the settings that we used in our testing across the three-tiers (WebSphere, the Gateway daemon, and CICS).

Table 3-3 End-to-end connection settings

Tier Setting Value

WebSphere Application Server

EJB container thread pool 100

Connection factory: Minimum connections, maximum connections

10, 55

Connection factory: Unused timeout 10 minutes

Connection factory: SocketConnectTimeout 10 milliseconds

Connection factory Wait timeout 10 seconds

HTML


ServletJSP

CTGTesterCCICICS ECIresourceadapter

CCI

Windows

ECIPROG

CITGR1CI

z/OS

EXCI

JNI

Gatewaydaemon

CICS TG V6.1

TCP

CITGR1G

Cam21-pc11

tx1.mop.ibm.com

13101

CICS


The reasoning we used in setting these values can be summarized by four rules:

1. Queue early, this saves resources as each request consumes system resources (we were queuing at the connection factory level).

2. Timeout at the source of the file I/O, because this is where the delays are likely to originate from.

3. Do not define infinite queues, because they can lead to infinite delays (such as a purgethreshold of NO in the CICS TRANCLASS).

4. Set resources that are potential bottlenecks (for example, the maximum number of EXCI pipes) to the highest values unless they are to be configured to act as points of queuing.

How you tune your own system is dependent on a wide number of different factors. However, from our experience, we suggest that you apply the following specific rules to a WebSphere, JCA, CICS TG scenario such as ours:

� Set the maximum number of connection manager threads (maxconnect) in the Gateway daemon equal to the sum of the maximum number of connections in all the connection factories that connect to the Gateway.

� Set the initial number of connection manager threads (initconnect) equal to the minimum number of managed connections in the connection pool in all the connection factories that connect to the Gateway.

� Set the maximum number of worker threads to:

– Less than or equal to the maximum number of connection manager threads.

– Less than or equal to the EXCI LOGONLIM.

Gateway idletimeout 10 minutes

minconnect, maxconnect 10, 55

minworker, maxworker 10,55

idletimeout 10 minutes

CICS EXCI LOGONLIM 250

SESSION RECEIVECOUNT 999

Mirror TRANCLASS LIMIT (active, queued) 50, 5

MAXTASKS 100

Tier Setting Value


� Set the initial number of worker threads equal to the initial number of connection manager threads.

� Set the RECEIVECOUNT for the CICS SESSIONS definition:

– At least as big as the EXCI LOGONLIM.

– Larger than or equal to the sum of all the worker threads in the Gateway daemons that can connect through this connection (if multiple Gateways share a connection).

Testing the configurationTo test our configuration, we used three different Web browser instances, each of which invoked our test application CTGTesterCCI (Figure 3-16), using the following URL:

http://cam21-pc11:9080/CTGTesterCCIWeb/index.jsp

Figure 3-16 CTGTesterCCI application


http://cam21-pc11:9080/CTGTesterCCIWeb/index.jsp

We specified as input a COMMAREA of D (meaning the CICS program ECIPROG would delay for one minute), and 3 iterations. For further details about our sample application refer to Appendix A, “Sample applications” on page 361.

This test results in each invocation from the Web browser driving three calls to CICS and lasting approximately three minutes in total. During this period, we monitored the connection state in each tier using the following utilities:

WebSphere Tivoli® Performance ViewerTCP/IP netstatCICS TG stdout logCICS CEMT command

The next sections discuss each of these tools.

Tivoli Performance ViewerTivoli Performance Viewer is provided with WebSphere Application Server and provides runtime systems monitoring for a wide variety of resources in WebSphere.

First, we enabled the PMI function in the Administrative Console as follows:

� We selected Monitoring and Tuning → Performance Monitoring Infrastructure (PMI) in the Welcome panel, and then selected server1. We then selected Enable Performance Monitoring Infrastructure (PMI) and Extended. Finally, we selected OK to apply these changes and then saved them to the repository and restarted the Application Server.

� After the restart, in the Welcome panel, we selected Monitoring and Tuning → Performance Viewer → Current Activity → server1. In the left-side panel of the screen, we selected server1 → Performance Modules → JCA Connection Pools → ECIResourceAdapter → eis/CITGR1CI-tx1-13101 to enable collection of the extended performance metrics for our connection factory (Figure 3-17).


Figure 3-17 Tivoli Performance Viewer - Current Activity

� We then selected View Table to view the information in tabular form and selected the following statistics to be displayed:

CreateCount This is the total number of managed connections that have been created within the connection pool.

CloseCount This is the total number of managed connection that have been closed within the connection pool.

Poolsize This is the current number of managed connections in the pool.

FreePoolSize This is the number of managed connections that are not allocated to a J2EE component.


The resulting table then displayed in our browser (Figure 3-18).

Figure 3-18 Tivoli Performance Viewer - connection factory metrics

From this information, we can see that three connections were created at 15:49:48, one was recorded as free at 15:50:48, and then three were recorded as free at 15:51:48. After this, all three connections in the pool remain free for use by other applications.

Monitoring sockets with netstatAt the same time as we monitored our connection factory, we also monitored the socket status. We used the USS command netstat -a I<IP address> to show the status of sockets connected from z/OS to our Windows workstation (Example 3-6).

Example 3-6 TCP/IP socket status

CITGPW: /SYSTEM/tmp>netstat -a -I9.100.199.239MVS TCP/IP NETSTAT CS V1R6 TCPIP Name: TCPIP1 15:41:30User Id Conn Local Socket Foreign Socket State------- ---- ------------ -------------- -----CITGR1G 000052AE 9.100.193.122..13101 9.100.199.239..2047 EstablshCITGR1G 000052B0 9.100.193.122..13101 9.100.199.239..2050 EstablshCITGR1G 000052B3 9.100.193.122..13101 9.100.199.239..2051 Establsh

This example shows that three sockets were established from z/OS to WebSphere on our Windows machine. These sockets remained active after the EJB requests had completed due to the connection pooling of WebSphere Application Server.


Gateway monitoringTo monitor the Gateway daemon connection usage we viewed the STDOUT log, which enables us to view the status of connections (Example 3-7).

Example 3-7 Gateway STDOUT log

09/21/05 15:49:30:280 [0] CTG6506I Client connected. [ConnectionManager-9] - tcp@Socket[addr=9.100.199.239,port=2047,localport=13101]09/21/05 15:49:32:399 [0] CTG6506I Client connected. [ConnectionManager-8] - tcp@Socket[addr=9.100.199.239,port=2050,localport=13101]09/21/05 15:49:35:877 [0] CTG6506I Client connected. [ConnectionManager-7] - tcp@Socket[addr=9.100.199.239,port=2051,localport=13101]

These messages show that three connection manager threads were allocated one for each connection and that these connections remained active after the EJB requests had completed.

CICS monitoring While our three Web browser sessions were running, we used CEMT to monitor the status of our mirror transaction ECIT in CICS. We used the CEMT INQ EXCI and CEMT I TRANCLASS commands. Figure 3-19 shows that there are three EXCI requests active from the job CITGR1G running on system X1 (our LPAR).

Figure 3-19 CEMT INQUIRE EXCI

I EXCI STATUS: RESULTS Exc(CITGR1G.CITGR1G. - X1 ) Tas(0001417) Exc(CITGR1G.CITGR1G. - X1 ) Tas(0001418) Exc(CITGR1G.CITGR1G. - X1 ) Tas(0001419)

SYSID=TGR1 APPLID=A6POTGR1 RESPONSE: NORMAL TIME: 15.49.50 DATE: 09.21.05 PF 1 HELP 3 END 5 VAR 7 SBH 8 SFH 9 MSG 10 SB 11 SF


The CEMT INQ TRANCLASS command (Figure 3-20) shows that three tasks were active in our TRANCLASS TCLASS1.

Figure 3-20 CEMT INQUIRE TRANCLASS

3.7.2 Cloned Gateway daemon connectivity testsIn this section, we show how we cloned our Gateway daemon CITGR1G and tested a TCP/IP port sharing configuration. Cloning a Gateway daemon provides for improved failover and increased throughput. Figure 3-21 shows the outline of our configuration.

Figure 3-21 CTGTesterCCI remote connection to cloned Gateway daemons

I TRANCLASS(TCLASS1) STATUS: RESULTS - OVERTYPE TO MODIFY Tcl(TCLASS1 ) Max( 050 ) Act(003) Pur( 0000006 ) Que(000000)


13101

HTML


ServletJSP

CTGTesterCCICICS ECIresourceadapter

CCI

Windows

ECIPROG

CITGR1CI

z/OS

EXCI

JNI

Gatewaydaemon

CICS TG V6.1

TCP

CITGR1G/CITGR2G

Cam21-pc11

tx1.mop.ibm.com


To set up our cloned Gateway daemons, we performed the following tasks:

� Created another Gateway daemon start task CITGR2G.

� Created a new STDENV input member for CITGR2G, although note that we shared the same Gateway configuration file as used by CITGR1G (thus we inherited all the same Gateway settings).

� Defined a new DFHJVPIPE name of CITGR2G in the STDENV member for our Gateway daemon CITGR2G.

� Defined new CICS CONNECTION and SESSSIONS definitions in our CICS region CITGR1CI, specifying the NETNAME of CITGR2G, and a unique session prefix.

� Created a new user ID CITGR2G (as described in “Setting up a user ID for the Gateway daemon started task” on page 30) and used the STARTED class in RACF to associate the new user ID with the CITGR2G job.

� Modified the MRO bind security settings for CICS, by giving user ID CITGR2G READ access to the RACF FACILITY class DFHAPPL.A6POTGR1.

� Modified our TCP/IP PORT statement to allow CITGR2G to bind to and share the port 13101 as follows:

13101 TCP CITGR1G SHAREPORT ; Gateway daemon port shared 13101 TCP CITGR2G SHAREPORT ; Gateway daemon port shared

We then restarted TCP/IP to update the port allocations.

Monitoring cloned Gateway usage After we had setup our cloned Gateway daemons, we started each one in turn and ensured that they both started and that they had bound to port 13101 on IP address 9.100.193.122. We then used two Web browser instances to send two requests to the CTGTesterCCI application, each of which had a COMMAREA input of D. This caused the two requests to run in parallel.

During the test we monitored the STDOUT log of each Gateway daemon to view the status of connections (Example 3-8 and Example 3-9).

Example 3-8 CITGR1G Gateway STDOUT log

09/22/05 17:06:53:193 [0] CTG6485I Successfully started handler for the tcp: protocol on port 13101, bound to address /9.100.193.12209/22/05 17:06:53:601 [0] CTG6512I CICS Transaction Gateway initialization complete 09/22/05 17:07:14:287 [0] CTG6506I Client connected. [ConnectionManager-9] - tcp@Socket[addr=9.100.199.239,port=2560,localport=13101]


Example 3-9 CITGR2G Gateway STDOUT log

09/22/05 17:06:55:922 [0] CTG6485I Successfully started handler for the tcp: protocol on port 13101, bound to address /9.100.193.12209/22/05 17:06:56:311 [0] CTG6512I CICS Transaction Gateway initialization complete 09/22/05 17:07:13:534 [0] CTG6506I Client connected. [ConnectionManager-9] - tcp@Socket[addr=9.100.199.239,port=2559,localport=13101]

From the STDOUT logs we can see that a socket connection was established from the WebSphere connection factory to each of the cloned Gateway daemons, thus proving that the TCP/IP port sharing component is load balancing the socket connections across our two Gateway daemons.

3.8 Problem determinationIn this section, we document common connection problems and information that we learned while configuring our connection test scenarios. For general problem determination guidance, see 2.9, “Problem determination” on page 60.

3.8.1 Common connection problemsIn this section, we list some common connectivity problems that you might experience.

ECI_ERR_RESOURCE_SHORTAGEIf the Gateway daemon tries to allocate more pipes than the available number of CICS sessions, the EXCI Open_Pipe call fails with a RETRYABLE response, and reason code 202 is logged in the CICS TG STDOUT log. The ECI application receives a return code -16 (ECI_ERR_RESOURCE_SHORTAGE).

This situation can occur even if the client application program has allocated fewer pipes than the number of receive sessions defined on the target CICS connection. This is because CICS can be in the process of cleaning up a pipe from a Close_Pipe request. For this reason, we recommend that you specify a larger RECEIVECOUNT value than is theoretically necessary when defining the SESSIONS resource definition to CICS.

ECI_ERR_SYSTEM_ERRORIf the number of EXCI pipes in use exceeds the CICS EXCI pipe limit, the next EXCI Allocate_Pipe call fails with a SYSTEM_ERROR response and reason code 608. The ECI application receives return code -9 (ECI_ERR_SYSTEM_ERROR).


Because this means a limitation in the sending address space has been reached, you should consider one of the following options:

� Increasing the MVS LOGONLIM if using CICS TS V2.2 or higher.

� Decreasing the worker thread count and cloning your Gateway regions, to provide increased capacity if required.

� Setting the CTG_PIPE_REUSE=ONE variable, to provide a more predictable pipe usage

If you define the DFHJVPIPE environment variable incorrectly as blanks, the EXCI Initialize_User fails with a USER_ERROR response and reason code 403 (INVALID_APPL_NAME).

You should specify a non-blank value for DFHJVPIPE in STDENV.

3.8.2 Tips and utilitiesThis section includes additional tips and utilities for problem determination of connection-related problems. For general tips and information about standard utilities, such as the various Gateway daemon traces, see 2.9.2, “Tips and utilities” on page 62.

Testing a Connection pool failureTo simulate a connection failure we performed an immediate shutdown on our Gateway daemon by cancelling the Gateway. We then resubmitted a request from our test J2EE application and monitored the state of the sockets and the connection pool.

After the Gateway was cancelled, we noticed the following results:

� Our test application returned the following exception:

javax.resource.spi.CommException: CTG9630E IOException occurred in communication with CICS

� The netstat socket status showed all the sockets in a TimeWait state.

� The CloseCount in Tivoli Performance Viewer went up to 1 and the PoolSize decreased to 2. This was triggered by the IOException driving a fatal connection error on the connection that it tried to use. Figure 3-22 shows the results from our Tivoli Performance Viewer.


Figure 3-22 Tivoli Performance Viewer - connection factory counters

3.8.3 TracingIn this section, we outline the main traces that you can enable within the WebSphere environment in order to diagnose problems with a connection from WebSphere Application Server on a distributed platform to the Gateway daemon running on z/OS.

Java client traceYou can set CICS TG Java client tracing programmatically or as a system property.

CICS TG Java client tracing can be set programmatically by adding the following statements to the Java application:

com.ibm.ctg.client.T.setDebugOn(true);com.ibm.ctg.client.T.setTimingOn(true);

Both of these traces go to the stderr output stream, so that they appear in the standard error log file of the WebSphere Application Server. Example 3-10 shows the stderr log on our system.

Example 3-10 WebSphere stderr log location

C:\Program Files\IBM\WebSphere\AppServer/profiles/AppSrv01\logs\server1\SystemErr


In our CTGTesterCCI application the Trace option on the index.jsp welcome page controls this tracing.

You can also set Java client trace using the WebSphere Administrative Console. The following system properties control CICS TG Java client trace within the application server:

Dgateway.T=onDgateway.T.setTFile=<filename>

You can set these properties from the WebSphere Administrative Console as follows:

1. Click Servers → Application Servers → server1.

2. In the Server Infrastructure configuration settings, expand Java and Process Management and click Process Definition → Java Virtual Machine.

3. Enter the Java client trace system properties as Generic JVM™ Arguments.

J2EE resource adapter traceResource adapter tracing is used to control tracing of the flow of execution through the CCI classes. When using a managed environment, J2EE resource adapter tracing is enabled by setting the TraceLevel parameter in the connection factory. You can set the tracelevel to one of the following values:

0 Disable all tracing1 Output exception trace stacks2 Output method entry and exit stack traces3 Output debug trace entries

Note: Enabling CICS TG Java client tracing in an application enables it for all applications running in the application server, as a consequence of the static nature of the T class.

Note: If you do not explicitly set a file name, the trace entries are written to stderr.


Chapter 4. Connecting from WebSphere Application Server for z/OS

This chapter provides details about how we configured our test environment to allow our J2EE application running on WebSphere Application Server for z/OS to connect to an application running in CICS Transaction Server on z/OS using the CICS Transaction Gateway for z/OS.

4


4.1 PreparationIn our testing, we deployed our sample J2EE application on WebSphere Application Server for z/OS and connected to CICS using the CICS ECI resource adapter that is provided by CICS TG for z/OS.

This chapter concentrates on how to configure the connections between each tier (as shown in Figure 4-1). It does not provide details about how to install all of the software components, and it assumes a working knowledge of CICS or WebSphere Application Server where appropriate.

Figure 4-1 Software components: CICS TG and WebSphere Application Server for z/OS

Note: In this chapter we configure a local connection from WebSphere Application Server to a CICS region running in the same LPAR. The CICS TG also supports remote connections from WebSphere Application Server for z/OS. See Chapter 7, “Transactions with WebSphere Application Server for z/OS” on page 253 for an example of how to use a remote connection from WebSphere on z/OS.

CICS TS V3.1

COBOL application

HTML


ServletJSP

EJB

COMMAREA

CICS TG V6.1

CICS ECIresourceadapter

CCI

z/OS

EXCI




4.1.2 Definitions checklistTable 4-2 lists the definitions that we used to configure the scenario.

Table 4-2 Definitions checklist: WebSphere Application Server for z/OS

Windows 2000 z/OS

Internet Explorer V6.0 z/OS V1.6

Windows 2000 Service Pack 4 WebSphere Application Server V6.0.1 a

CICS Transaction Server V3.1

CICS Transaction Gateway for z/OS V6.1

IBM SDK for z/OS Java 2 Technology Edition,V1.4.2 SR2

a. WebSphere Application Server for z/OS V6.01 is the minimum WebSphere level supported with the CICS TG V6.1 ECI resource adapters.

Value WebSphere Application Server for z/OS

CICS TS

hostname tx1.mop.ibm.com

TCP/IP port 13880

Job names Daemon region: CITGRDMNControl region: CITGRS1Servant region: CITGRS1S

CITGR1CI

APPLID A6POTGR1

NETNAME CITGRS1

CONNECTION GRS1

Private mirror ECIW

Chapter 4. Connecting from WebSphere Application Server for z/OS 123

4.2 WebSphere Application Server configurationIn this section, we outline the configuration of our WebSphere Application Server for z/OS. We discuss the following:

� J2EE server configuration� Defining the workload profile� Installing the resource adapter� Creating connection factories� Deploying our sample J2EE application

4.2.1 J2EE server configurationOur WebSphere Application Server for z/OS configuration is a base Application Server node (Figure 4-2). It consists of:

� One application server, which in turn consists of:– One control region– One servant region (where the J2EE applications execute)

� One daemon (the location agent, responsible for resolving IIOP requests)� One node (a logical grouping of managed servers)� One cell (a logical grouping of one or more nodes)

Figure 4-2 WebSphere Application Server for z/OS - base configuration

For details about setting up such an environment, refer to WebSphere Application Server for z/OS V6.0.1, Installing your application serving environment, GA22-7957.

Controlregion

(CITGRS1)

Servantregion

(CITGRS1S)

J2EE Server(CITGR_server1)

Node (CITGR_node1)

Cell (CITGR_cell1)

Daemon (CITGRDMN)

z/OS


4.2.2 Defining workload profileThe WebSphere Application Server Workload profile controls the number of threads used in a servant region. The workload profile can have one of four values, ISOLATE, IOBOUND (the default), CPUBOUND, or LONGWAIT. We changed the value to LONGWAIT, which is the recommended value for applications that make frequent calls to other subsystems such as CICS. When LONGWAIT is specified each servant region is able to use a maximum of 40 threads.

To change the setting, we use the WebSphere Administrative Console as follows:

1. Click Servers, and then Application servers.

2. Select our server CITGR_server1 and on the presented panel, click Container Services, and then ORB Service.

3. Click z/OS additional settings.

4. In the next panel, change the default IOBOUND to LONGWAIT (Figure 4-3).

5. Click OK, save the configuration, and restart the J2EE server.

Note: The WebSphere Application Server configuration that we used in our tests is a very simple one. For further considerations when connecting to CICS from a Network Deployment configuration, see 4.6, “Configuring for high scalability” on page 149.

Note: The number of threads in a J2EE servant region is controlled by the server workload profile, which can have the following values:

� IOBOUND - 3 * the number of CPUs (max of 3*32=96)� CPUBOUND - the number of CPUs on line (max 32)� ISOLATE - 1 thread� LONGWAIT - 40 threads


Figure 4-3 Administrative Console - Workload profile

4.2.3 Installing the CICS ECI resource adapterCICS TG V6.1 provides two ECI resource adapters, cicseci.rar and cicseciXA.rar. For this scenario, we used the cicseci.rar adapter.

To install the resource adapter:

1. We started our Application Server and logged into the Administrative Console using the following URL in our Web browser:

http://tx1.mop.ibm.com:13880/ibm/console/

2. We were presented with the login page and entered CITGRADM (the WebSphere administration user ID) in the User ID field and clicked login.

Note: WebSphere Application Server for z/OS provides two-phase commit support for local connections when using either of the CICS ECI resource adapters. However, only the cicseciXA.rar provides two-phase commit support when connecting to CICS using a remote connection via a z/OS Gateway daemon. See Chapter 7, “Transactions with WebSphere Application Server for z/OS” on page 253 for more information about transactional support.


http://tx1.mop.ibm.com:13880/ibm/console/

3. In the Administrative Console Welcome panel, we clicked Resources and then clicked Resource Adapters. We selected the Node level view, clicked Apply, and clicked Install RAR.

4. On the presented panel (Figure 4-4), we selected Server path, and filled in the path to the cicseci.rar file, which in our environment was /usr/lpp/cicstg/ctg610/deployable/cicseci.rar. We then clicked next.

Figure 4-4 Administrative Console - specifying the path of the RAR

5. On the following panel (Figure 4-5), for the Native Path property, we entered the full path to the CICS TG bin directory where the .so files were stored. The directory on our system was /usr/lpp/cicstg/ctg610/bin.


Figure 4-5 Administrative Console - specifying the native path

6. We accepted the defaults for the other properties on this panel and scrolled to the bottom of the panel and clicked OK. We clicked Save twice to save the modified configuration.

4.2.4 Creating a connection factory After installing the resource adapter, you need to create a connection factory which targets the CICS region. To create a connection factory, we did the following:

1. From the Administrative Console, we clicked Resources, Resource Adapters and then clicked ECIResourceAdapter, which is the resource adapter that we just installed.

2. On the presented panel, we clicked J2C Connection Factories and then we clicked New.


3. We were then presented with the panel shown in Figure 4-6. We filled in the Name field. The name we gave the connection factory is CITGR1CI-tx1-local. In this name, CITGR1CI is the CICS region to which we connect, tx1 is the z/OS LPAR, and local designates that we are using a local connection. We allowed the security settings to default and clicked OK.

Figure 4-6 CICS connection without security or XA

4. We let the rest of the properties default, clicked Apply, and then clicked Save to save the configuration.

5. After these settings were applied and the connection factory was selected, the Additional Properties links were then enabled. We clicked Custom Properties, which took us to the CICS ECI resource adapter properties panel (Figure 4-7).


6. The custom properties are the parameters that are used by the CICS ECI resource adapter to communicate with the target CICS region. We set the following parameters by selecting each parameter, filling in the value for the parameter, and clicking OK.

– TPNName

This is the CICS TPN name and is used as the name of the attached mirror transaction that requests will run in CICS. We set this property to ECIW. If the property is not set, the CICS default mirror transaction CSMI is used.

– ConnectionURL

This is the URL that defines the TCP/IP host name and protocol that are used to communicate with the Gateway daemon. Because we are using a local connection, we set the connection URL to local.

– PortNumber

This is normally the port number that the Gateway daemon listens on. Because we are running in local mode, and there is no Gateway daemon, we set the port number to zero (0).

– ServerName

This is the APPLID of the CICS region to which requests are sent. We specified the APPLID of our CICS region A6POTGR1.

We then saved our configuration.

Tip: Make sure that you include the colon (:) in local.

Restriction: For local connections, the CICS region must run on the same MVS image as the WebSphere Application Server.


Figure 4-7 Administrative Console - custom properties

For a full explanation of all the properties, refer to 4.2.4, “Creating a connection factory” on page 128.

7. After we defined the connection factory custom properties, we clicked Connection pool properties on the resource adapter connection factory page and entered values to tune our connection pool.

8. This presented us with the general connection pool properties panel (Figure 4-8) where we entered the following values.

Note: For a local connection, the JCA connection pool managed by WebSphere Application Server is a set of local connection objects. The connection objects do not map onto network connections, nor do they map onto EXCI pipes because EXCI pipes are allocated directly by the JVM threads within the J2EE servant region.


– Connection timeout

Specifies how long, in seconds, a getconnection() request by the J2EE application can wait if there are no connections available in the free pool, and no new connections can be created. If a timeout occurs a ConnectionWaitTimeoutException is thrown.

We set this to 10 seconds in order to timeout in the event that a connection request cannot be satisfied. Note, however, that such event should not happen in practice because we are not restricting the size of the connection pool.

– Maximum connections

We set this to a value of zero (0).

If maximum connections is set to zero (0), the connection pool is allowed to grow infinitively. In that case the number of threads defined for the WebSphere Application Server servant regions, in our case 40, limits the maximum number of connections in the pool.

– Minimum connections

We set the minimum number of connections to a value of zero (0).

– Reap time

This is the interval at which the pool manager maintenance thread will run. With with connection timeout and aged timeout both set to 0, we also set this to zero (0), which stops the pool manager maintenance thread from running without any work to do.

– Unused timeout

Because we set the reap time to zero (0), we also set the unused timeout to zero (0).

– Aged timeout

Again, because we set the reap time to zero (0), we also set the aged timeout to zero (0).

– PurgePolicy

This specifies how connections are being purged when a fatal error is detected. We set this to FailingConnectionOnly. The alternative is EntirePool, which means that all threads are purged.

Note: For a detailed description of the custom pool properties, refer to Figure 3-8 on page 87.


Figure 4-8 Administrative Console - defining connection pool properties

4.2.5 Configuring the J2EE server In our testing, we used a specific EXCI connection when connecting to CICS (see “EXCI connections” on page 139). Therefore, we need to specify the environment variable DFHJVPIPE. Because we are running in local mode, CICS TG environment variables are specified using the WebSphere Administration Console.


To configure the J2ee server, we logged in to the Administrative Console, clicked Environment → WebSphere Variables → New. On the presented panel (Figure 4-9), we entered DFHJVPIPE in the name field and CITGRS1 in the Value field. Then we clicked OK.

Figure 4-9 WebSphere environment variable - DFHJVPIPE

4.2.6 Deploying our sample J2EE applicationIn this section, we describe how we deployed our test application CTGTesterCCI into WebSphere Application Server for z/OS. For details about the application, refer to Appendix A, “Sample applications” on page 361.

Deploying CTGTesterCCI To deploy the test application, we did the following:

1. In the Administrative Console Welcome panel, we clicked Applications and Install New Application. We then selected Remote file system and specified the path of the application ear file /CITG/TJ/CTGTesterCCI.ear


2. We clicked Next (Figure 4-10).

Figure 4-10 Administrative Console - deploying J2EE application

3. We clicked Next on the Preparing for the application installation screens and clicked Continue on the Application Security Warnings screen.

4. The Administrative Console now lays out steps in the installation of the application. These steps might differ from one application to another, depending on what the Administrative Console application sees in the deployment descriptors of the EAR file being installed. For example, when the deployment descriptor contains a <resource-ref> tag, the Administrative Console application presents a step to map that resource reference to a resource.

5. We accepted the defaults for all steps, except the Map resource references to resources, as shown in Figure 4-11.

6. We mapped the ECI resource reference to the JNDI name of the Connection Factory that we defined earlier.

All of the JNDI names of defined Connection Factories appeared in the pull-down window. We chose the JNDI name that matched the Connection Factory we created previously: eis/CITGR1CI-tx1-local.

We selected the EJB module that we wanted to map (CTGTesterCCIEJB) and choose the default authentication method.

We then clicked Apply to insert the JNDI name into the EJB module’s JNDI Name field.


Figure 4-11 Installing CTGTesterCCI - mapping resources

7. We clicked Next three times and then Finish (noted the ADMA5013I: Application CTGTesterCCI installed successfully message) then we clicked Save to master configuration, followed by Save.

8. In the Administrative Console, we clicked Applications and Enterprise Applications, selected the CTGTesterCCI application, and clicked Start.


4.3 Configuring EXCIWhen using local connections with WebSphere Application Serer on z/OS, the CICS ECI resource adapters use the EXCI to communicate directly with CICS. The following sections discuss the key points that we considered for our EXCI configuration.

4.3.1 EXCI pipe limitsEach WebSphere servant region is limited to a maximum number of pipes that it can allocate to all attached CICS regions. The EXCI pipe limit rules are as follows:

� In CICS TS V1.3, the limit is 100 EXCI pipes in total per calling address space.

� In CICS TS V2.2 or higher the PTF for APAR PQ92943 allows a MVS system wide LOGONLIM to be set, which controls the pipe limit. The upper limit is 250 and the default is 100.

The maximum pipe usage in a CICS region is limited by the RECEIVECOUNT parameter defined in the CICS MRO SESSIONS definition. This can be a maximum of 999 pipes, depending on the number of characters specified in the RECEIVEPFX.

The default EXCI pipe allocation initialization parameter defined in the DFHSSIN routine is LOGONLIM=100. See 4.3.1, “EXCI pipe limits” on page 137 for instructions on how we changed the default setting.

4.3.2 CTG_PIPE_REUSEBy default if a J2EE server is connected to multiple CICS regions, a servant region thread can potentially allocate multiple EXCI pipes. If this occurs it can cause unpredictable pipe shortage issues. In CICS TG V6.0 a new environment variable called CTG_PIPE_REUSE was introduced. If this is set to ONE then a thread will only allocate one pipe, and will deallocate the pipe if it needs to reallocate a pipe to a different CICS region. This provides a more predictable usage model, but with a small cost in performance. The exact performance cost depends on the frequency of pipe deallocations.

We set the value CTG_PIPE_REUSE=ONE using the WebSphere Administrative Console. We clicked Environment and WebSphere variables and we then added the variable by clicking New (Figure 4-12) and then saved the changes.


Because we only have one CICS region in our setup, this setting had no immediate effect. However, setting it is good practice if you anticipate that the WebSphere Application Server will connect to multiple CICS regions at some time in the future and your aim is to ensure a predictable operating environment.

Figure 4-12 CTG_PIPE_REUSE=ONE setting

4.3.3 Defining the EXCI library to your J2EE ServerThe J2EE Server that uses the CICS ECI resource adapter needs to load the CICS module DFHXCSTB from the SDFHEXCI library supplied with CICS. This can be enabled in one of the following ways:

� Place module DFHXCSTB in the link pack area (LPA).

� Add the SDFHEXCI library to either the system link list concatenation, or to the STEPLIB concatenation of the J2EE Server.

We added our SDFHEXCI data set (CICSTS31.CICS.SDFHEXCI) to the MVS system link list.


4.4 Configuring CICSFor the WebSphere Application Server to be able to connect to the CICS TS region, we performed the following tasks:

� Set CICS system initialization table (SIT) parameters� Configured an EXCI connection in CICS� Added a private mirror transaction� Planned for transaction timeouts

SIT parametersBecause the CICS TG and WebSphere Application Server use EXCI for communications and MVS resource recovery services (RRS) for transactional support, we enabled this support in CICS using the following SIT parameters:

� RRMS=YES� IRCSTRT=YES� ISC=YES

We restarted the CICS region to put these parameters into effect.

EXCI connectionsWe created the CONNECTION and SESSIONS definitions in CICS. Then, from a CICS screen, we entered the following:

CEDA DEFINE CONNECTION(GRS1) GROUP(GRS1)

We defined the GRS1 CONNECTION as shown in Figure 4-13. We recommend using specific EXCI pipes, which provides for better monitoring and accounting within CICS. We set Conntype to Specific and Netname to CITGRS1 (the DFHJVPIPE environment variable name that we defined previously using the WebSphere Administrative Console).

Figure 4-13 CICS definitions - CEDA Define CONNECTION

DEFINE GROUP(GRS1) CONNECTION(GRS1) OVERTYPE TO MODIFY CEDA DEFine CONnection( GRS1 ) CONnection : GRS1 Group : GRS1 DEscription ==> CONNECTION IDENTIFIERS Netname ==> CITGRS1 INDsys ==> CONNECTION PROPERTIES ACcessmethod ==> IRc Vtam | IRc | INdirect | Xm PRotocol ==> Exci Appc | Lu61 | Exci Conntype ==> Specific Generic | Specific SInglesess ==> No No | Yes


We then defined the SESSIONS:

CEDA DEFINE SESSIONS(GRS1) GROUP(GRS1) CONNECTION(GRS1)

We defined the GRS1 SESSIONS as shown in Figure 4-13. We set the RECEIVEPFX to 3, which means that the session names for this connection all start with the character 3, and we set the RECEIVECOUNT to 999.

Figure 4-14 CICS definitions - CEDA Define SESSIONS

Next, we installed our definitions using the following command:

CEDA INSTALL GROUP(GRS1)

4.4.1 Adding a private mirrorRather than running our ECI request under the default mirror transaction (CSMI) we defined our own private mirror, which has the following advantages:

� It allows individual CICS monitoring and statistics information to be recorded for the given application.

� It allows specific security attributes to be set for the transaction.

� It allows a specific TRANCLASS to be specified, which can control the queuing and scheduling priorities of the CICS application relative to the other transactions executing in the CICS region.

DEF SESSIONS(GRS1) GR(GRS1) CONNECTION(GRS1) OVERTYPE TO MODIFY CICS RELEASE = 0640 CEDA DEFine Sessions(GRS1 ) Sessions : GRS1 Group : GRS1 DEscription ==> SESSION IDENTIFIERS Connection ==> GRS1

SESSION PROPERTIES Protocol ==> Exci Appc | Lu61 | Exci MAximum ==> 000 , 000 0-999 RECEIVEPfx ==> 3 RECEIVECount ==> 999 1-999

SYSID=TGR1 APPLID=A6POTGR1


To define our private mirror we performed the following steps:

1. We defined a mirror transaction ECIW by copying the default mirror transaction definition CSMI to a new RDO group using the following command:

CEDA COPY GR(DFHISC) TRAN(CSMI) AS(ECIW) GROUP(ECIPROG)

2. We updated this definition to specify a TRANCLASS of TCLASS2 using the following command:

CEDA ALTER TRANS(ECIW) GROUP(ECIPROG) TRANCLASS(TCLASS2)

3. We defined a new TRANCLASS resource definition, in group ECIPROG, to control the attributes of the ECIW transaction with a MAXACTIVE value of 50 and a PURGETHRESHOLD of 5 (Figure 4-15). This TRANCLASS limits our CICS region to running 50 simultaneous ECIW transactions and also queues a further five requests before it refuses to queue further transactions.

Note that the TRANCLASS setting is larger than the number of threads defined in the WebSphere Application Server, see 4.2.2, “Defining workload profile” on page 125. However, if we were to connect to our CICS region at a later time from multiple WebSphere servant regions, the TRANCLASS would ensure that the CICS region would not be paralyzed with requests for the ECIW transaction.

Figure 4-15 CICS TRANCLASS definition

4. We installed the TRANCLASS using the CEDA INSTALL GROUP(ECIPROG) command and added the ECIPROG group to our CICS startup list TGR1.

OVERTYPE TO MODIFY CICS RELEASE = 0640 CEDA ALter TRANClass( TCLASS2 ) TRANClass : TCLASS2 Group : ECIPROG Description ==> CLASS LIMITS Maxactive ==> 050 0-999 Purgethresh ==> 0000005 No | 1-1000000

SYSID=TGR1 APPLID=A6POTGR1


4.4.2 Transaction timeoutsWe did not set any timeout parameters in our CICS environment (such as RTIMOUT or DTIMOUT), because neither of these were applicable to our scenario. However, we recommend that you consider setting a timeout on the file manager (such as DB2), because it is generally perceived as best practice to time out suspended tasks as close as possible to the file operation that is likely to cause delays.

For further details about the end-to-end timeout settings that we used, refer to “End-to-end settings” on page 143.

4.5 Testing the configurationIn this section, we detail how we tested our environment, using our sample J2EE application CTGTesterCCI. Figure 4-16 shows the basic outline of our environment.

Figure 4-16 CTGTesterCCI connection from WebSphere Application Server for z/OS

Tip: We found that a suspended task in CICS, for example, caused by someone leaving CEDF running inadvertently, caused a timeout in the WebSphere Application Server which resulted in a servant region failure and an automatic restart. For this reason, we recommend that you attempt to timeout CICS tasks using CICS facilities such as RTIMOUT and DTIMOUT.

CICS TS V3.1

ECIPROG

HTML


ServletJSP

CTGTesterCCI

COMMAREA

CTG V6.1

cicseci.rarCCI

z/OS

EXCI GRS1

tx1.mop.ibm.com

CITGR1CICITGR_server1


End-to-end settingsTable 4-3 summarizes the settings that we used throughout the chapter.

Table 4-3 End-to-end settings

To test our configuration, we used a Web browser to invoke the test application CTGTesterCCI. We used the following URL to start the application:

http://tx1.mop.ibm.com:13880/CTGTesterCCIWeb/index.jsp

Tier Setting Value

WebSphere Application Server for z/OS

Servant region threads (workload profile LONGWAIT)

40

Connection factory: Minimum connections, maximum connections

0,0

Connection factory: Connection timeout 10

Connection factory: Unused timeout 0

Connection factory: Aged timeout 0

CICS EXCI LOGONLIM 250

SESSION RECEIVECOUNT 999

SESSION RECEIVEPFX 3

Mirror TRANCLASSLIMIT(active,queued) 50,5

MAXTASKS 100



We were then presented with the screen that is shown in Figure 4-17. We specified a COMMAREA of D as input, which caused the target CICS program to enter a delay of one minute. Then we selected Submit. We repeated these steps on the two other browsers.

Figure 4-17 CTGTesterCCI - input screen


We were presented with the results as shown in Figure 4-18. The information in the COMMAREA is as follows:

� A6POTGR1 is the applid of the CICS region to which we connected.

� 01/12/05 13:11:57 is the transaction timestamp

� CITGR1D (the default CICS user ID) is the user ID the transaction is run under

� DS is the transaction start code

Figure 4-18 CTGTesterCCI - result screen

MonitoringDuring the test period, we monitored the connection state in WebSphere Application Server and CICS, using the following utilities:

WebSphere Tivoli Performance ViewerCICS CEMT command


Tivoli Performance ViewerTivoli Performance Viewer is provided with WebSphere Application Server and provides runtime systems monitoring for a wide variety of resources in WebSphere. First, we enabled the PMI function in the Administrative Console as follows:

1. We selected Monitoring and Tuning → Performance Monitoring Infrastructure (PMI) in the Welcome panel, and then selected server1. We selected Enable Performance Monitoring Infrastructure (PMI) → Extended. We selected OK to apply these changes and then saved them to the repository and restarted the Application Server.

2. After the restart, in the Welcome panel we selected Monitoring and Tuning → Performance Viewer → Current Activity → CITGR_server1. In the left-side panel of the screen, we selected CITGR_server1 → Performance Modules → JCA Connection Pools → ECIResourceAdapter → eis/CITGR1CI-tx1-local to enable the collection of the extended performance metrics for our connection factory (Figure 4-19).

Figure 4-19 Tivoli Performance Viewer - current activity


3. We clicked View Table to view the information in tabular form and selected the following statistics to display:

CreateCount The total number of managed connections that have been created within the connection pool.

CloseCount The total number of managed connection that have been closed within the connection pool.

Poolsize The current number of managed connections in the pool. Used and unused.

FreePoolSize The number of free managed connections currently in the pool.

Figure 4-20 Tivoli performance viewer - connection factory metrics

From the information, we can see that between 11:10:24 and 11:10:51 there were no connections in the pool. At 11:11:21, we see that two connections were created. In the same interval, we see the pool size is two, with one of the connections not being used, and the FreePoolSize is one. At 11:12:23, an additional connection is created, bringing the PoolSize to a total of three.

CICS monitoringWhile our Web browser sessions were running, we used CEMT to monitor the status of our mirror transaction ECIW in CICS. We used the CEMT INQUIRE EXCI and CEMT INQUIRE TASK commands.


Figure 4-21 shows that there are three EXCI requests active from the job CITGRS1S (the WebSphere servant region) running on system X1 (our LPAR).

Figure 4-21 CEMT INQUIRE EXCI

From the CEMT INQUIRE TASK command (Figure 4-22), we can see the private mirror transaction ECIW running on facilities with the character 3 in the first position of the name, as defined in the CICS SESSIONS RECEIVEPFX (see Figure 4-14 on page 140).

Figure 4-22 CEMT INQUIRE TASK

I EXCI STATUS: RESULTS Exc(CITGRS1S.CITGRS1S. - X1 ) Tas(0000803) Exc(CITGRS1S.CITGRS1S. - X1 ) Tas(0000806) Exc(CITGRS1S.CITGRS1S. - X1 ) Tas(0000807) SYSID=TGR1 APPLID=A6POTGR1 RESPONSE: NORMAL TIME: 13.12.08 DATE: 12.01.05 PF 1 HELP 3 END 5 VAR 7 SBH 8 SFH 9 MSG 10 SB 11 SF

I TASK STATUS: RESULTS - OVERTYPE TO MODIFY Tas(0000794) Tra(CEMT) Fac(N197) Run Ter Pri( 255 ) Sta(TO) Use(CITGR1D ) Uow(BDFE9EF50170E00B) Tas(0000796) Tra(CEMT) Fac(N196) Sus Ter Pri( 255 ) Sta(TO) Use(CITGR1D ) Uow(BDFE9F042F04DE09) Hty(ZCIOWAIT) Tas(0000803) Tra(ECIW) Fac(31 ) Sus Ter Pri( 001 ) Sta(DS) Use(CITGR1D ) Uow(BDFEACB372242247) Hty(ICWAIT ) Tas(0000806) Tra(ECIW) Fac(310 ) Sus Ter Pri( 001 ) Sta(DS) Use(CITGR1D ) Uow(BDFEACE043EEFE8D) Hty(ICWAIT ) Tas(0000807) Tra(ECIW) Fac(3100) Sus Ter Pri( 001 ) Sta(DS) Use(CITGR1D ) Uow(BDFEACE16CC1EF20) Hty(ICWAIT ) SYSID=TGR1 APPLID=A6POTGR1 RESPONSE: NORMAL TIME: 13.12.03 DATE: 12.01.05 PF 1 HELP 3 END 5 VAR 7 SBH 8 SFH 9 MSG 10 SB 11 SF


4.6 Configuring for high scalabilityAfter you have successfully configured and tested your WebSphere z/OS configuration, you should consider how you can clone the J2EE server in order to improve scalability and availability. The principal areas for consideration are:

� How to load balance TCP/IP requests across multiple J2EE servers

� Clustering the J2EE server

� How to load balance CICS program links dynamically across multiple CICS AORs

� Other configuration considerations

4.6.1 TCP/IP load balancingWebSphere Application Server for z/OS is designed to work with Sysplex Distributor. Sysplex Distributor is an integral part of z/OS Communications Manager, which offers the ability to load balance incoming socket open requests across different address spaces running on different IP stacks (usually on different LPARs). The routing decision is based on real-time socket status and MVS Quality of Service (QoS) criteria, which provides the benefit of balancing work across different MVS images and enhances scalability and failover in a z/OS Parallel Sysplex.

The benefits of Sysplex Distributor described in “Sysplex Distributor” on page 103 apply well to a WebSphere z/OS configuration.

4.6.2 ClusteringClustering is a technique in WebSphere Application Server that allows individual application servers to be combined into a logical whole. When two or more application servers are combined into a cluster, the Deployment Manager administrative interface sees that cluster as an entity into which applications are installed. In other words, the servers inside the cluster are no longer viewed as individual, but rather part of a group (or cluster). An application installed into the cluster is actually installed into all the servers in the cluster.

Clusters must stay within a cell, but a cell can span different systems within a parallel sysplex, thus taking advantage of the scalability and availability capabilities offered by the parallel sysplex.


Figure 4-23 shows the use Sysplex Distributor and a J2EE server cluster. Note that TCP/IP port sharing is not normally required with this configuration.

Figure 4-23 High scalability and availability configuration with WebSphere on z/OS

4.6.3 Dynamic program linkFigure 4-23 shows the recommended high availability configuration when using the CICS TG with WebSphere Application Server for z/OS. A given WebSphere servant region connects to a single CICS region. Such a CICS region is referred to as a router region. This region does not run the CICS application itself but instead routes the program link request dynamically to a CICS AOR.

It is possible to use the EXCI user exit DFHXCURM (see 3.4.3, “EXCI user-replaceable module: DFHXCURM” on page 100) to re-route EXCI requests to an alternative (or backup) CICS router region in case of a failure of the target CICS router region.

CICSPLex SM provides a dynamic routing program which supports the dynamic routing of LINK commands. This program provides the ability for applications invoked by ECI requests to be routed dynamically across a CICSplex.

LPAR - 3

CICSprogram

TCP/IP SysplexDistributor

z/OS sysplex

CICS AORs

DPL

DPL

EXCIServant regions

CICS router

region 1

LPAR -1

DaemonController

region

EXCIServant regions

CICS router

region 2

LPAR -2

Daemon

Controllerregion

J2EE server cluster

node 1

node 2

Restriction: For local connections, the CICS region must run on the same MVS image as the WebSphere servant region. However, you can define the target CICS program to be remote so that the program link is routed to another CICS region that is running on a different MVS image.


4.6.4 Other configuration considerationsWhen creating a configuration similar to that shown in Figure 4-23, there are configuration considerations in addition to those described in 4.2, “WebSphere Application Server configuration” on page 124, including:

� Installing the resource adapter

You must install the CICS ECI resource adapter on each node in the cell to ensure that the implementation code (JAR files and binaries) of the resource adapter is available to the WebSphere servant regions running on each node.

� Creating connection factories

You should create a connection factory for the LPAR specific CICS router region in each of the two WebSphere nodes (node 1 and node 2 in Figure 4-23). Each connection factory is identical except for the ServerName, which determines the APPLID of the CICS region to which requests are sent. On node 1, you need to specify the APPLID of CICS router 1 as the ServerName. On node 2, you need to specify the APPLID of CICS router 2 as the ServerName. The connection factories should be given the same JNDI name, for example, eis/CICS Router.

� Deploying the J2EE application

When you deploy the application, you deploy to the J2EE server cluster rather than to a specific J2EE server. When you map resource references to resources (as shown in Figure 4-11 on page 136), you should map the resource reference to the connection factory with JNDI name eis/CICS Router.

4.7 Problem determinationIn this section, we document common connection problems and information that we learned while configuring our connection test scenario using WebSphere Application Server for z/OS. For general problem determination guidance see 2.9, “Problem determination” on page 60.

CTG9630EYou might get the following message for an ECI request after installing the CICS ECI resource adapter:

CTG9630E IOException occurred in communication with CICS

Check the WebSphere servant region’s job log for other errors. Example 4-1 shows message CTG9630E and CTG6686E.


Example 4-1 Edited WebSphere servant region job log showing CTG9630E and CTG6686E

javax.resource.spi.CommException: CTG9630E IOException occurred in communication with CICS at com.ibm.connector2.cics.ECIManagedConnection.call(ECIManagedConnection.java:1295) at com.ibm.connector2.cics.ECIConnection.call(ECIConnection.java:122) .Caused by: java.io.IOException: CTG6686E Unable to initialize JNI library. [java.lang.UnsatisfiedLinkErrorCan't find library ctgjni (libctgjni.so) in sun.boot.library.path or java.library.path sun.boot.library.path=/CITG/CellCITGS/AppServer/java/bin

This error is caused by the failure to specify correctly the path to the JNI files (/usr/lpp/cicstg/ctg610/bin) in the WebSphere Application Server Resource Adapters → ECIResourceAdapter Native Path field.

4.7.1 TracingIn this section, we outline the main traces that you can enable within the WebSphere environment to diagnose problems with connections from WebSphere Application Server on z/OS.

Java client traceCICS TG Java client tracing can be set programmatically. See “Tracing” on page 118 for information about enabling Java client trace.

J2EE resource adapter traceResource adapter tracing is used to control tracing of the flow of execution through the CCI classes. See 3.8.3, “Tracing” on page 118 for information about enabling J2EE resource adapter trace.

JNI traceIn order to enable JNI trace, specify the CTG_JNI_TRACE and CTG_JNI_TRACE_ON environment variables using the WebSphere Administrative Console. To specify these variables, do the following:

1. Log in to the Administrative Console and click Environment → WebSphere Variables.

2. Click New.

3. Specify the trace environment variable in the name field and the setting in the Value field.

Figure 4-24 shows how we enabled JNI trace for WebSphere Application Server for z/OS.


Figure 4-24 Administrative Console - enabling CICS TG JNI trace

4. We need to connect user ID CITGRS1S to the CTGV61 group (group ID 7100) that we created in 2.3.2, “Setting permissions” on page 35 so that the WebSphere servant region can create the JNI log file.

Example 4-2 shows the initial JNI trace entries that are written as the result of the execution of the CTGTesterCCI application.

Example 4-2 JNI trace

CTG6813I Running under WebSphere z/OS. CTG6880I Specific pipe used. NETNAME=CITGRS1 . CTG6895I IEANTRT successful. Returned token value 0xfa. CTG6894I GetMaxLogon returned 0, maxlogon was set to 250. CTG6893I EXCI pipe reuse set to ONE per TCB.

If CTG_JNI_TRACE_ON=YES is defined as a WebSphere environment variable but CTG_JNI_TRACE is not set, then JNI messages are written to STDERR. Because STDERR for our servant region was defined as SYSOUT=*, this combination of settings means that JNI messages were written to SDSF in our scenario.

Note: We found that it was necessary to restart the WebSphere application server in order to pick up changes to environment variables.


Part 3 Transaction Management

In this part, we describe the transactional capabilities of the CICS Transaction Gateway for z/OS. We provide detailed information about how we configured transactions between WebSphere Application Server and CICS.

We cover two different topologies:


We discuss the configuration of transactions within WebSphere Application Server, the Gateway daemon, and CICS.

Part 3


Chapter 5. Gateway daemon transactions

In this chapter, we describe the transactional support that is provided by the Gateway daemon on z/OS.

We start with an overview of basic transactional concepts and standards such as two-phase commit and global transactions. Then, we describe the new transactional support that is provided with the CICS TG for z/OS, including support for global transactions (also referred to as XA transactions).

We document how we prepared the system and the values that we used in our testing environment. Then, we give details about how we set up and modified the different tiers in our test environment to support transactions. We also explain how we tested the environment.

For information about how to enable the Gateway daemon to support XA transactions in a TCP/IP workload management configuration, see Chapter 6, “Gateway daemon transactions with TCP/IP load balancing” on page 225.

5


5.1 PreparationIn our testing, we used WebSphere Application Server for Windows. For information about transactional support with WebSphere Application Server on z/OS, see Chapter 7, “Transactions with WebSphere Application Server for z/OS” on page 253.

First, we tested extended LUWs with our sample CTGTesterCCI application using the cicseci.rar. Then we tested the new XA transaction support using our second sample application CTGTesterCCIXA with the cicseciXA.rar (Figure 5-1).

Figure 5-1 Software components: Gateway daemon transaction scenarios

Figure 5-1 shows a global transaction spanning updates to multiple resource managers, a CICS region and a DB2 database on Windows. The global transaction is coordinated by the transaction manager, WebSphere Application Server. RRS is used to coordinate the resource updates on z/OS.

Note: In our global transaction configuration, we document the setup steps for transactions, which update recoverable resources in two CICS regions. However, we do not document the setup or testing of DB2.

CICS A

z/OS

TCP

CICS ECI XA

resource adapter

JCA


servlet

EJB

Windows

DB2

JDBC

Datasource

Global transaction

TransactionManager

ResourceManager

RRS

ResourceManager

Gateway daemon

ServletJSP

EXCI




5.2 Transactions - what are theyIn this section, we provide a brief overview of the following fundamental transactional concepts and standards:

� CICS transactions, tasks, and syncpoints� Distributed transactions, including the two-phase commit process

5.2.1 CICS transactions, tasks, and syncpointsWe refer to a CICS transaction as the work that is initiated in a CICS region and that runs as a CICS task under a four character transaction ID (tranid). At task initiation CICS implicitly starts a unit-of-work (UOW) for all CICS transactions. This is usually the initial boundary of the transactional work to be undertaken. All updates to recoverable resources or requests to other transactional systems are now part of this unit-of-work, until either a synchronization point (syncpoint) is reached within the CICS program or the CICS transaction finishes and the task terminates (Figure 5-2).

Figure 5-2 CICS synchronization points

Windows 2000 z/OS

Internet Explorer V6.0 z/OS 1.6





EXEC CICSSYNCPOINT End of task

Unit-of-work 1 Unit-of-work 2

Transaction initiation

Load initial program

Start of task

Chapter 5. Gateway daemon transactions 159

In certain circumstances, such as though an inter-system distributed program link (DPL) request is used, the CICS transaction that is linked to can be co-coordinated by a remote CICS system. In this instance, the mirror task remains suspended until the end of the transaction, which is referred to as a long-running mirror task (Figure 5-3).

Figure 5-3 Link with long-running mirror task

Additionally, the converse situation is also possible, where the invoked CICS transaction runs in a separate transactional context to that of the invoking application. This situation is referred to as running with sync-on-return, which refers to the fact that the mirror transaction in CICS issues a syncpoint on returning control to the calling application (Figure 5-4). The use of a sync-on-return type link also allows the called CICS program to issue EXEC CICS syncpoint commands, because it is not sub-ordinate to another transaction manager.

Figure 5-4 Link with SYNCONRETURN

Start of mirror task

Mirrortransaction

extended unit-of-work

CICS region

1st ECI request

2nd ECI request

Mirror task suspends

Commit

Mirror task issuesSYNCPOINT

Commitresponse

termination ofmirror task

Start of mirror task

Mirrortransaction

unit-of-work

CICS region

LINK with SYNCONRETURN

termination ofmirror task

Link to userprogram

Return to mirrortransaction

EXEC CICSSYNCPOINT


5.2.2 Distributed transactionsWithin a distributed transactional system each distributed system is either referred to as a resource manager or a transaction manager. The transaction manager controls the outcome of the transaction and is responsible for the recovery of it’s resources; it has to implement a logging mechanism in order to be able to coordinate multiple resource managers. The resource managers control access to recoverable resources, and as such have to implement the necessary network flows and logging procedures to provide transactional coordination.

In order to enable distributed systems to send and receive transactional requests the partners must send and receive the requests using a common mechanism. This mechanism is known as the two-phase commit process.

Two-phase commitThis is an architected set of flows that transaction managers use to ensure all resource managers in a transaction can be reliably coordinated, irrespective of any failure. It is implemented by all transactional protocols and the fundamental concepts are essentially the same.

Figure 5-5 summarizes the flows according to the XA specification. Other protocols, such as CICS or LU6.2, might use different terminology and variants on the flows. (For further details about CICS syncpoint flows, refer to the CICS Intercommunication Guide, SC34-6448.)

Figure 5-5 Two-phase commit

In the first phase (or Stage 1), the transaction manager asks all the resource managers to prepare to commit recoverable resources. Each resource manager can vote either positively (prepared) or negatively (failed to prepare). If a

GlobalTransaction

Stage 2 - Commit

ResourceManager

ResourceManager

Commit

Commit

TransactionManager

GlobalTransaction

Prepared

Prepare

Stage 1 - Prepare

Prepare

Prepared

TransactionManager

ResourceManager

ResourceManager

Committe

d

Committed

1

2

3

4


resource manager replies positively, it is then obliged to follow the eventual outcome of the transaction as determined at the next stage. The resource manager is now described as in-doubt, because it has delegated eventual transaction control to the transaction manager. If a resource manager replies negatively, the transaction manager assumes a backout is needed and the transaction will be backed out by all resource managers.

In Stage 2, providing all the resource managers voted positively, the transaction manager replies to each resource manager with a commit flow. Upon receipt of the commit flow, the resource manager finalizes updates to recoverable resources, and releases any locks held on the resources. The resource manager then responds with a final committed flow, which indicates to the transaction manager that it is no longer in-doubt.

Although the two-phase commit process is usually a prerequisite to distributed transactional support, there are certain instances where a single-phase commit process can be sufficient. This is referred to as last resource optimization, and is implemented by a variety of transaction managers. It essentially allows the commit decision to be delegated to the one-phase commit resource, allowing the one-phase commit to participate in a global transaction with any number of two-phase commit capable resources (Figure 5-6).

Figure 5-6 Last resource optimization

At transaction commit, the transaction manager first prepares the two-phase commit resource managers and, if this is successful, the one-phase commit-resource is then called to commit. The two-phase commit resources are

GlobalTransaction

Stage 2 - Commit

ResourceManager

ResourceManager

Commit

GlobalTransaction

Prepare

Stage 1 - Prepare

Prepare

TransactionManager

ResourceManager

ResourceManager

ResourceManager

(one phase)

ResourceManager

(one phase)

TransactionManager

1

2

Com

mit

Commit

3

5

4


then committed or rolled back depending on the response of the one-phase commit resource, effectively delegating transaction coordination to the one-phase commit resource.

Unlike for a two-phase commit resource, there is no recovery from a communication failure with a one-phase commit resource. Such a communication failure during commit of the one-phase commit resource introduces the risk of a mixed outcome to the transaction. The two-phase commit resources are by default rolled back, but the outcome of the one-phase commit resource is unknown; it could have committed or rolled back. Applications must therefore be configured to accept the additional risk of such heuristic outcomes.

Last resource optimization is implemented within WebSphere Application Server V6 as last participant support, and within CICS and APPC flows as last agent optimization. However, applications that are deployed to exploit last participant support are subject to an increased risk of a mixed outcome in a global transaction if the one-phase commit resource fails during the commit processing.

5.3 Resource Recovery ServicesResource Recovery Services (RRS), a component of z/OS, provides a global syncpoint manager that any resource manager on z/OS can exploit. It enables transactions to update recoverable resources managed by many resource managers.

5.3.1 How RRS worksRRS is an exit manager and does not itself perform tasks such as commit changes to a DB2 database, or backout changes made by a CICS program, but rather it provides a framework for resource managers such as DB2, CICS or IMS to be coordinated by an independent Transaction Manager, such as WebSphere Application Server (see Figure 5-1 on page 158).

Technically, by RRS we really mean Recoverable Resource Management Services (RRMS). RRMS consists of three services:

� Registration services� Context Services� Resource Recovery Services (RRS)

Note: You specify whether an application accepts the possibility of a mixed outcome occurring in a global transaction that contains a one-phase resource, when you deploy the application in WebSphere Application Server.


These three services are provided by the RRS MVS address space.

Registration servicesThe first thing a resource manager must do in order to start using RRS is to identify itself to RRS and supply RRS with its exit routines, for example, prepare, commit and backout routines. To do so, a resource manager uses RRMS registration services.

Context servicesContext services provide an exit manager allowing resource managers to track the changes made by a given unit of work. A context represents the environment that the application is running in and its work request.

A work manager (the gateway daemon, for example) can create a context and have an application run under it. A work manager can create more than one context, but at any point in time there can only be one active context for a single task. If the work manager needs to change the context, it has to explicitly switch to a different context.

RRSA resource manager registers various exit routines with RRS, which are driven at key points in the lifetime of a transaction, after the resource manager has indicated an interest in that particular transaction. An application can request RRS to commit a Unit of Recovery (UR), for which RRS will then invoke the commit exits of all resource managers involved in the UR.

RRS provides a set of ISPF operator panels for displaying and updating RRS information such as units of recovery (URs). Figure 5-7 shows the RRS primary options panel.

Figure 5-7 ISPF - RRS primary options panel

We show examples of how to use the RRS ISPF panels in 5.6.5, “Testing the extended LUW configuration” on page 182 and 5.7.5, “Testing the XA transaction configuration” on page 201.

RRSSelect an option and press ENTER: 1 Browse an RRS log stream 2 Display/Update RRS related Resource Manager information 3 Display/Update RRS Unit of Recovery information 4 Display/Update RRS related Work Manager information 5 Display/Update RRS UR selection criteria profiles 6 Display RRS-related system information


For general information about RRMS and RRS, refer to Systems Programmer's Guide to Resource Recovery Services (RRS), SG24-6980.

5.3.2 CICS TS and RRSDuring initialization, CICS TS registers exits with RRS as a resource manager and provides RRS with details of CICS exit routines to be driven for the range of transaction life cycle events (prepare, commit, backout, and so forth).

Transactional EXCIThe Transactional EXCI support that is provided with CICS TS uses RRS to allow multiple EXCI calls to be part of one extended unit-of-work. An EXCI client program can send several DPL requests to the same CICS region (or to different CICS regions within the same MVS image), as well as make calls to other resource managers and have all requests syncpointed by RRS.

A Transactional EXCI client program must first establish the transaction context which will tie the DPL requests together. Each DPL request includes a token representing this context, allowing CICS to execute the target program of the DPL request under the correct context.

After completing the last DPL request of the sequence, the application calls RRS to drive either commit or rollback of all updates made under the transaction context.

ABENDBKOUT option in DFHXCOPTBy default, CICS does not tell RRS to rollback updates to recoverable resources following an unhandled abend in a Transactional EXCI program (the

Note: Because different parts of a transaction are processed by different tiers of the infrastructure, the transaction is identified in different ways. See “Identifying the transaction” on page 221 for a list of the different transaction identifiers that are used by the different software components that we used in our tests.

Note: Transactional EXCI is a pre-requisite for CICS TG transactional support, for both extended LUWs and XA transactions.

Restriction: Transactional EXCI that is provided by CICS TS is restricted to operating within a single MVS image (or LPAR). A Transactional EXCI client cannot link to CICS regions on other MVS images or LPARs within the same sysplex.


long-running mirror task). This means that a transaction abend, unlike a CICS failure or a failure of the EXCI client, does not trigger an automatic rollback of the RRS UR.

The EXCI options table DFHXCOPT has been amended recently to include the new option ABENDBKOUT={NO|YES}, which you can use to control transactional behavior following a CICS transaction abend. This option specifies whether a CICS transaction abend should trigger an automatic rollback of the RRS UR and backout all updates made by the long-running mirror.

5.4 Transactional support in the CICS TGThis section outlines the transactional support that is provided by the CICS TG for z/OS V6.1.

CICS TG for z/OS operates as a long-running transactional EXCI client. In versions of CICS TG for z/OS prior to CICS TG V6.1, the Gateway daemon has a passive work manager role in relation to RRS using only the registration and context services from RRMS. This limits the Gateway daemon to providing only one-phase commit support.

CICS TG 6.1 for z/OS implements the two-phase commit protocol, additionally using Resource Recovery Services (RRS) and taking on a more active role as a syncpoint coordinator. This implementation allows ECI requests through the Gateway daemon on z/OS to fully participate in a global transaction (or XA transaction). Inclusion of support for XA transactions is configurable and is disabled by default in order to ease migration from earlier releases.

5.4.1 Extended logical units of workThe ECI and Transactional EXCI communications protocols used by the CICS TG extend the CICS DPL concept (see 5.2.1, “CICS transactions, tasks, and syncpoints” on page 159) by allowing the invoking application to initiate and co-ordinate the CICS unit-of-work.

Tip: Specify ABENDBKOUT=YES in DFHXCOPT if you want to back out updates made by a long-running mirror following a transaction abend. This option is introduced as APAR PK17426 in CICS TS V3.1 and APAR PK17427 in CICS TS V2.2 and V2.3.


Extended LUW support is provided with the base CICS TG ECIRequest class. ECI request calls can be:

Nonextended The called CICS program, not the client Java application, controls whether recoverable resources are committed or backed out. Each program link call corresponds to one CICS task and one or more CICS UOWs.

Extended The client Java application controls whether recoverable resources are committed or rolled back. Multiple calls are possible, allowing a LUW to be extended across successive ECI requests to the same CICS region as shown in Figure 5-3 on page 160. An extended LUW causes the CICS mirror task to be long running so that it does not terminate on return from the initial call. Multiple program link calls correspond to one CICS task and one CICS UOW.

5.4.2 Transactional support in the CICS ECI resource adaptersThe transactional capabilities of the CICS ECI resource adapters is built upon a combination of the ability of the CICS TG to extend the unit-of-work and the transaction management system contract which exists between the J2EE server and the resource adapter.

JCA and transactionsThe JCA specifies the system contract for transaction management (as well as other system contracts such as connection management and security management) that exists between the J2EE server (in our case WebSphere Application Server) and the enterprise information system (in our case CICS).

Important: The single MVS image restriction that is imposed by transactional EXCI means that extended ECI requests that are received by a specific Gateway daemon can only link to CICS regions running on the same MVS image as the Gateway daemon.


A resource adapter is required to implement one of the following transaction management contracts, as defined in the resource adapter's deployment descriptor (ra.xml):

� XATransaction

A resource adapter that can participate fully in a two-phase commit process and which can influence the outcome of the global transaction.

� LocalTransaction

A resource adapter that can participate in transactions that are local to the resource manager, but cannot participate in two-phase commit transactions (other than as an only agent or a last participant).

� NoTransaction

A resource adapter with no transactional properties, this can participate in a transactional context but is not influenced by, and has no effect upon, the outcome of the transaction.

The CICS TG for z/OS V6.1 provides two CICS ECI resource adapters:

cicseci.rar The CICS ECI resource adapter has the same transactional behavior as the cicseci.rar that is provided with CICS TG V6. It supports the LocalTransaction interface.

cicseciXA.rar The new CICS ECI XA resource adapter provided with CICS TG V6.1 provides full support for global transactions by implementing XAResource interface.

CICS ECI resource adapter (cicseci.rar)The WebSphere transaction manager enables a single one-phase capable resource manager such as the CICS ECI resource adapter to be used within a global transaction in the absence of any other transactional resource managers.

When you use the CICS ECI resource adapter, therefore, you can commit multiple JCA requests to the same CICS region as one unit-of-work if the only recoverable resources updated within the transaction are accessed via a single CICS ECI connection factory (see 3.2.2, “Creating connection factories” on page 82 for full details on creating CICS ECI connection factories). You can commit multiple JCA requests in this way by setting the EJB transaction attribute to specify the requests as being part of the same transaction (see “EJB transaction attribute” on page 172).

To commit a JCA request to a CICS region as part of a global transaction with other two-phase commit resource managers, it is necessary to use the Last Participant Support function of WebSphere Application Server V6. Using the


Last Participant Support function is only possible if the global transaction involves a single one-phase capable resource, and involves JCA requests to a single CICS ECI connection factory.

When using the CICS ECI resource adapter with WebSphere Application Server on a distributed platform it is not possible to commit multiple JCA requests to different CICS ECI connection factories as part of the same global transaction with other two-phase commit resource managers.

CICS ECI XA resource adapter (cicseciXA.rar)The WebSphere Application Server transaction manager can provide coordination, within a transaction, for any number of two-phase capable resource managers (for example, the CICS ECI XA resource adapter and a JDBC resource).

The XA transaction support that is provided with CICS TG for z/OS V6.1 enables CICS TS programs to participate fully (with other two-phase commit capable resource managers) in a global transaction that is initiated in a distributed J2EE V1.4 application server, such as WebSphere Application Server V6. Thus, your application can commit JCA requests to multiple CICS regions that are accessed via different connection factories as part of a global transaction with other two-phase commit resource managers such as DB2 and IMS.

Which resource adapter to useBoth the ECI and ECI XA resource adapters can be deployed into the same WebSphere Application Server, provided that they originate from the same release of the CICS TG. Thus, you can install both resource adapters, define connection factories for each one, and then map application resource references

Note: When installed into WebSphere Application Server for z/OS, cicseci.rar supports global transactions (see 7.3.1, “Transactional support in the CICS ECI resource adapters” on page 256).

Important: The single MVS image restriction that is imposed by transactional EXCI means that XA transaction requests that are received by a specific Gateway daemon can only link to CICS regions running on the same MVS image as the Gateway daemon.

However, a J2EE application can link to CICS regions on separate MVS images (or sysplexes) within a single global transaction by using multiple connection factories that define connections to multiple Gateway daemons running on different MVS images.


to these connection factories based on whether XA transaction support is required.

The following list gives guidelines on which resource adapter to use.

� If your J2EE application does not use transactions — for example it is an EJB deployed with the transaction attribute NotSupported or Never (see Example 5-1 on page 172) — then you can use either of the CICS ECI resource adapters.

� If your application needs to commit multiple JCA requests to the same CICS region as one unit-of-work, you should use the CICS ECI resource adapter (cicseci.rar) in order to avoid the overhead of unnecessary XA flows between the WebSphere Application Server and CICS TG.

See 5.6, “Configuring for extended LUWs” on page 174 for information about how to configure your environment for extended LUW support.

� If your application needs to commit JCA requests to multiple CICS regions, accessed via different connection factories, as part of a global transaction with other two-phase commit resource managers, you must use the CICS ECI XA resource adapter (cicseciXA.rar).

See 5.7, “Configuring for XA transactions” on page 193 for information about how to configure your environment for XA transaction support.

� If your application requires to commit JCA requests to a single CICS region as part of a global transaction with other two-phase commit resource managers:

– You should use the CICS ECI XA resource adapter (cicseciXA.rar) if data integrity your primary concern, because last participant support gives an increased risk of a mixed outcome in a global transaction if CICS fails during the commit processing.

Important: Running mixed levels of CICS TG resource adapters in the same application server is unsupported and can lead to unpredictable results. For example, you should not install the CICS TG V6.1 ECI XA resource adapter into a WebSphere Application Server along side either of the CICS TG V5.1 or CICS TG V6.0 ECI resource adapters.

Important: You should be aware that due to the processing overhead associated with XA transactions, network, and CPU utilization is significantly higher when you use the XA transactional support of the CICS ECI XA resource adapter.


– You could use the last participant support function of WebSphere Application Server V6 and the CICS ECI resource adapter (cicseci.rar) if performance is your primary concern. This avoids the overhead of XA flows between the WebSphere Application Server and CICS TG.

5.5 Transactions in WebSphere Application ServerThe way that WebSphere applications use transactions depends on the type of application component, as follows:

� In the EJB container:

– A session bean can either use container-managed transactions (where the bean delegates management of transactions to the container) or bean-managed transactions (where the bean manages transactions itself).

– Entity beans use container-managed transactions.

� In the Web container:

The Web container has limited transactional support, as its principal function is for servlet and JSP™ components which are primarily concerned with presentation logic. Web components (for example, servlets) can only use bean-managed transactions.

Transactions in the EJB ContainerThe EJB container in WebSphere Application Server is suited ideally to the deployment of transactional components and provides support for both container managed transactions and bean managed transactions. Session beans and message-driven beans can employ either type. Entity beans are restricted to container-managed transactions only.

Beans using bean-managed transactions are responsible for transaction demarcation and must begin and end a transaction explicitly.

Container-managed transactions are the preferred mechanism because this delegates transactional control to the Application Server, which allows the application developer to concentrate on developing the business logic while allowing the transactional properties of the application to be decided upon deployment. The key to transactional control with container-managed transactions is the EJB transaction attribute.

Note: We do not discuss Web container transactions in our transaction test scenarios, because it is best practice to use EJBs if you have a need for transactional integration between WebSphere and CICS.


EJB transaction attributeThe transaction attribute is set in the assembly descriptor section of the EJB deployment descriptor (that is, the file ejb-jar.xml). This attribute is used by the EJB container to control under which circumstances a global transaction is started when a bean method is invoked.

The transaction attribute appears in the <container-transaction> section and is specified with the <trans-attribute> tag. For example, the following XML specifies that the execute() method on the CTGTesterCCI bean has the transaction attribute of NotSupported.

Example 5-1 EJB Assembly descriptor transaction attribute

<assembly-descriptor><container-transaction>

<method><ejb-name>CTGTesterCCI</ejb-name><method-intf>Remote</method-intf><method-name>execute</method-name>

</method><trans-attribute>NotSupported</trans-attribute>

</container-transaction></assembly-descriptor>

Table 5-2 lists the possible values for the transaction attribute.

Table 5-2 EJB transaction attribute meaning

Note: We do not cover bean-managed transactions in our transaction test scenarios because we recommend that you leave transaction management to the EJB container.

Transaction Attribute

Meaning

NotSupported Bean method cannot execute within context of an global transaction

Required Bean method must execute within context of an global transaction.

RequiresNew Bean method must execute within context of a new global transaction.

Supports Bean method can execute within the context of a global transaction, depending on whether the caller supplies a transaction context, or not.


Impact of EJB transaction attribute Table 5-3 shows the resulting type of JCA request behavior, and resulting type of CICS mirror task, for each of the different transaction attributes and depending on which CICS ECI resource adapter is used.

Table 5-3 Impact of EJB attribute

Mandatory Bean method must execute within the context of the client's global transaction. If the caller does not supply a context (in other words there is no global transaction alive), then the execute fails with an exception.

Never Bean method must not be invoked within the context of a global transaction.

Note: The default EJB transaction attribute is Required.

Transaction Attribute

Meaning

Transaction Attribute Resulting JCA request CICS mirror task

cicseci.rar cicseciXA.rar

NotSupported Non-extended LUW SYNCONRETURN

Required Extended LUW XA transaction Long running

RequiresNew Extended LUW XA transaction Long running

Supports Non-extended or extended LUW

Non-extended or XA transaction

SYNCONRETURN or long running

Mandatory Extended LUW or Exception thrown

XA transaction or Exception thrown

Long running

Never Non-extended LUW SYNCONRETURN


5.6 Configuring for extended LUWsIn this section, we show how we configured our setup to support extended LUWs using the CICS ECI resource adapter (cicseci.rar). We needed to consider the following configuration and setup tasks:

� Configuring CICS

Checking that our CICS TS region was enabled to support Transactional EXCI.

� CICS TG for z/OS

Checking that our Gateway daemon was configured correctly to register with RRS.

� WebSphere Application Server

– Updating the transaction attribute for our sample application CTGTesterCCI

– Re-deploying the application and mapping the resource reference to a CICS ECI connection factory

– Configuring WebSphere transaction logging

We do not discuss installation of the CICS TG ECI resource adapter here or give information about defining connection factories, because these topics are covered in detail in 3.2, “Configuring WebSphere Application Server” on page 80.

5.6.1 Definitions checklistTable 5-4 lists the definitions that we used to configure this test scenario.



hostname tx1.mop.ibm.com -

IP address 9.100.193.122 -

TCP/IP port 13101 -

Job name CITGR1G CITGR1CI

APPLID A6POTGR1

NETNAME CITGR1G -

CONNECTION GR1G

RRM Name CTG.RRM.CITGR1G.IBM.UA


5.6.2 Configuring CICS First, we checked that the RRS address space was actively running on the LPAR. We then verified that our CICS region was registered correctly with RRS (the CICS system initialization parameter RRMS=YES must be specified). During CICS initialization, we see the CICS log messages shown in Example 5-2, which indicate that CICS has registered with RRS successfully.

Example 5-2 CICS TS RRS initialization log messages

DFHRX0100I A6POTGR1 RX domain initialization has started. DFHRX0104I A6POTGR1 The Resource Recovery Services (RRS) exit manager ATR.EXITMGR.IBM is now available.DFHRX0101I A6POTGR1 RX domain initialization has ended.

From a CICS terminal, we check that RRS is available for use by issuing the CEMT INQUIRE RRMS command. The response should indicate a status of open, as shown in Figure 5-8.

Figure 5-8 CEMT INQUIRE RRMS

5.6.3 Configuring CICS TGIn this test scenario, we use the CITGR1G Gateway daemon as configured in 3.3, “Configuring the Gateway daemon” on page 91, a single Gateway daemon with xasupport=off specified.

RRM Name considerationsThe environment variable CTG_RRMNAME is used to specify the name that the Gateway daemon uses when registering with RRS. The rules that apply when

I RRMS STATUS: RESULTS Rrm Ope SYSID=RGT1 APPLID=A6POTGR1 RESPONSE: NORMAL TIME: 17.00.02 DATE: 09.30.05 PF 1 HELP 3 END 5 VAR 7 SBH 8 SFH 9 MSG 10 SB 11 SF

Note: Running the CICS TG V6.1 with xasupport=off in the Gateway daemon configuration file provides the same transactional capabilities and behavior as CICS TG V6.0.


specifying this name are described in 2.3.4, “Creating an STDENV file” on page 37.

When the Gateway daemon is running with the default setting of xasupport=off, it uses only a subset of the RRMS functionality, which does not require it to be authorized. In this configuration, the Gateway daemon does not run as an authorized application and so must use the .UA (unauthorized) suffix when identifying itself via the Registration Services.

For this test scenario, we use the same CTG_RRMNAME setting that we defined when we first customized the Gateway daemon (CTG.RRM.CITGR1G.IBM.UA).

If the CTG_RRMNAME environment variable is found to be undefined during initialization, the Gateway daemon generates a Sysplex-unique name to use for RRS registration.

5.6.4 Configuring WebSphere In 3.2.3, “Deploying the sample J2EE application” on page 89, we deployed the CTGTesterCCI application into WebSphere Application Server to use container-managed transactions but with the transaction attribute of NotSupported. This deployment gives us the behavior of a non-extended mode ECI request and a SYNCONRETURN DPL in CICS. CICS is acting as both resource manager and transaction manager.

We now change from using CICS TS as the transaction manager to using WebSphere Application Server as the transaction manager by updating the transaction attribute on the EJB from NotSupported to Required. The combination of the Required transaction attribute in the EJB deployment descriptor and the use of the ECI resource adapter results in a local transaction in WebSphere Application Server, an extended mode ECI request, and a long-running mirror transaction in CICS.

The assembly descriptor can be updated using Rational® Application Developer or the WebSphere Application Server Toolkit.

Updating the EJB deployment descriptorWithin the application EAR file, the EJB archive CTGTesterCCIEJB.jar contains file META-INF/ejb-jar.xml. This deployment descriptor file contains a section called the assembly descriptor, which defines the transactional attributes for methods provided by the EJB.

Important: The value used for CTG_RRMNAME must be a Sysplex-unique value of no more than 32 characters.


We changed the transaction attribute for the CTGTesterCCI EJB using our development tool Rational Application Developer (Figure 5-9).

Figure 5-9 RAD - Updating EJB trans-attribute for CTGTesterCCI

After having used RAD to modify the transaction attribute, we exported the application to create an updated EAR file CTGTesterCCI.ear.

Example 5-3 is an extract from the CTGTesterCCI EJB deployment descriptor source, showing again the remote EJB method execute updated to use a trans-attribute of Required.

Example 5-3 Updated trans-attribute from CTGTesterCCI

<assembly-descriptor><container-transaction>

<method><ejb-name>CTGTesterCCI</ejb-name><method-intf>Remote</method-intf><method-name>execute</method-name>

</method><trans-attribute>Required</trans-attribute>

</container-transaction></assembly-descriptor>


Dealing with CICS transaction abendsCICS cannot tell RRS to roll back updates to recoverable resources following an abend of the long-running mirror task. This means that, by default, if the first JCA request of an extended LUW succeeds but a subsequent request abends (perhaps because of a file I/O failure), then the updates made by the first JCA request will be committed together with any updates made by the second request, even if some work did not complete successfully.

There are two ways of avoiding such an inconsistency resulting from a CICS transaction abend:

� The J2EE application can abandon the current local transaction by calling setRollbackOnly() on the current context.

In the event of a transaction abend, a CICSTxnAbendException is thrown by the CICS ECI resource adapter and the decision to rollback or commit resources updated by the local transaction can be taken by the J2EE application.

� The option ABENDBKOUT=YES can be set in the DFHXCOPT table (see “ABENDBKOUT option in DFHXCOPT” on page 165).

In the event of a transaction abend, the local transaction is rolled back automatically at the end of the EJB method call.

We modified the CTGTesterCCI application to issue a setRollbackOnly() on the EJB session context in the catch block for the ECIInteraction.execute() call (see Example A-1 on page 368). This ensures that the CICS abend is propagated onto WebSphere Application Server as the transaction manager and the local transaction (if one exists) is rolled back.

Re-deploying the applicationUsing the WebSphere Application Server Administrative Console, we navigated to the Enterprise Applications page, selected CTGTesterCCI and then clicked Update. The next panel requested that we specify the type of update to perform (Figure 5-10).

For simplicity, we updated CTGTesterCCI as a complete replacement by selecting Full application and clicked Browse to select the new EAR file.

Note: At the time of writing this book, we were unable to test the ABENDBKOUT option. We, therefore, chose to include transaction abend rollback logic in our sample application.


Figure 5-10 Updating the Enterprise Application

The subsequent steps for completing the deployment of the EAR file are common to those that are described in 3.2.3, “Deploying the sample J2EE application” on page 89. On the Map resource references to resources panel, we were careful to make sure that the Reference Binding ECI was bound to the JNDI name for our CICS ECI connection factory.

After completing the sequence of deployment panels, WebSphere Application Server stopped the CTGTesterCCI application, installed the new version, and then restarted the application.

Verifying the installed Application transaction attributeTo verify that we had updated the application to be transactional, we looked at the newly installed EJB Deployment Descriptor. Using the WebSphere Administrative Console, we navigated to Enterprise Applications → CTGTesterCCI → EJB Modules → CTGTesterCCIEJB.jar → Deployment Descriptor and found the trans-attribute for the CTGTesterCCI EJB Remote execute method had been updated correctly from NotSupported to Required (Figure 5-11).


Figure 5-11 Transaction attribute of Required

Configuring WebSphere transaction logging WebSphere Application Server stores transaction information in a transaction log. The default location for the log is in the WebSphere install directory <install_root>/tranlog/cell_name/node_name/server_name.

The WebSphere transaction log settings can be configured using the WebSphere Administrative Console by following the links Application servers → server1 → Container Service → Transaction Service.


In the configuration panel, you can set the transaction log directory using the following format:

directory_name;file_size

Where:

� directory_name is the name of the transaction log directory.

� file_size is the size specified in bytes. The nK or nM suffix can be used to indicate kilobytes or megabytes. If you do not specify a file size value, the default value of 1M is used.

We did not specify a transaction log directory. By default, WebSphere Application Server used the directory as shown in Example 5-4.

Example 5-4 WebSphere transaction log location

C:\Program Files\IBM\WebSphere\AppServer/profiles/AppSrv01\tranlog\Cam21-Pc11Node01Cell\Cam21-Pc11Node01\server1\transaction

In a high transaction load, excessive transaction logging can slow down performance of the application server due to its dependency on the operating system and the underlying storage systems. To achieve better performance, you can designate a new directory for the log files on a separate, physically larger storage system.

In the properties page for the Transaction Service, you can optionally change other transaction-related configuration properties. We changed the value for Total transaction lifetime timeout (default 120 seconds). This value sets the number of seconds a transaction can remain inactive before it is ended by the Transaction Service. A value of zero (0) indicates that there is no timeout limit. Because some of our test cases sometimes involve requests that suspend in CICS, we increased this value to 1200 seconds.

Note: You can disable transaction logging by defining a log size of zero (0). Specify “;0” as the transaction log name.


5.6.5 Testing the extended LUW configurationIn this section, we detail how we tested an extended LUW using our sample J2EE application CTGTesterCCI. Figure 5-12 shows the basic outline of our environment. We used this configuration for two scenarios:

� Single Extended LUW with commit� Extended LUW workload with commit and rollback

The Single Extended LUW test repeats the scenario that is described in 3.7.1, “Single Gateway daemon connectivity tests” on page 107 but highlights the differences, in transactional terms, that are attributed to the modified trans-attribute in the EJB assembly descriptor.

The Extended LUW workload builds upon the previous scenario, demonstrating a sequence of requests that lead to both committed and backed-out updates of a recoverable resource. For this test, we show how the status of WebSphere transactions can be monitored using the Tivoli Performance Viewer.

Figure 5-12 Extended LUW configuration

HTML


ServletJSP

CTGTesterCCI CICS ECIresourceadapterCCI

Windows

Long RunningMirror ECIT

CITGR1CI

z/OS

EXCI

JNI

Gatewaydaemon

CICS TG V6.1

CITGR1G

Cam21-pc11

tx1.mop.ibm.com

13101

CICS

TransactionRequired

ECI_Extend

RRS

Registrationand Context Services


To test our configuration we used our test application CTGTesterCCI via its Web browser interface (Figure 5-13).

Figure 5-13 CTGTesterCCI multiple iterations in extended LUW

During these tests, we specified DW or AW as the COMMAREA input to the ECIPROG program, meaning:

DW Write to a recoverable temporary storage queue RECIPROG and then delay for one minute.

AW Write to a recoverable temporary storage queue RECIPROG and then abend with abend code ECIP.

We used multiple iterations allowing us to demonstrate that multiple invocations of ECIPROG can be made within a local transaction. For further details about our sample application, refer to Appendix A, “Sample applications” on page 361.


During the course of these scenarios, we monitored the transaction using each of the following :

CICS CEMT commandCICS TG JNI trace outputRRS ISPF panels RRS related Work Manager informationWebSphere Tivoli Performance Viewer

Single Extended LUWThis test scenario used the CTGTesterCCI to perform two iterations, each of which called the same CICS program ECIPROG specifying a COMMAREA of DW. The Web browser appears to be busy for two minutes before receiving the results page, showing the COMMREA returned by the second iteration. At the end of the test, the recoverable TS Queue RECIPROG contained two new entries.

CICS CEMT commandWhile the first ECI request was in-flight, we issued the CEMT INQUIRE TASK command. Figure 5-14 shows the ECIT private mirror task running under Task 604, suspended in ICWAIT state. ECIPROG has issued an EXEC CICS DELAY call for 60 seconds.

Figure 5-14 CEMT INQUIRE TASK showing the mirror an UOWID

INQUIRE TASK STATUS: RESULTS - OVERTYPE TO MODIFY Tas(0000539) Tra(CEMT) Fac(N117) Sus Ter Pri( 255 ) Sta(TO) Use(CITGR1D ) Uow(BDB459523DDD7F0A) Hty(ZCIOWAIT) Tas(0000596) Tra(CEMT) Fac(N185) Sus Ter Pri( 255 ) Sta(TO) Use(CITGR1D ) Uow(BDB5D7858FFC578B) Hty(ZCIOWAIT) Tas(0000604) Tra(ECIT) Fac(11 ) Sus Ter Pri( 001 ) Sta(D ) Use(CITGR1D ) Uow(BDB5DAA6F51C928D) Hty(ICWAIT ) Tas(0000605) Tra(CEMT) Fac(N186) Run Ter Pri( 255 ) Sta(TO) Use(CITGR1D ) Uow(BDB5DABD1425AC4D) SYSID=TGR1 APPLID=A6POTGR1 RESPONSE: NORMAL TIME: 15.08.25 DATE: 10.04.05 PF 1 HELP 3 END 5 VAR 7 SBH 8 SFH 9 MSG 10 SB 11 SF


We then issued the CEMT INQUIRE EXCI command. Figure 5-15 shows CICS Task 604 again and the associated RRS UR identifier BDB5DAA67E5540000000042E01020000.

Figure 5-15 CEMT INQUIRE EXCI showing a UR identifier

We see later that the UR identifier shown on CEMT INQUIRE EXCI panel ties up with the active and archived data accessible by the RRS panels is ISPF. The INQUIRE EXCI command provides the information to correlate an active long-running mirror task with the RRS UR identifier associated with the wider transaction. The Exc field highlights the job name and connection name with which the UR identifier is associated.

RRS ISPF panelsRRS provides a set of ISPF operator panels for displaying and updating RRS information such as Units of Recovery (URs). We selected option 3 from the RRS primary menu panel and the RRS Unit of Recovery Selection panel was displayed. We pressed enter to see a list of URs and we then selected to view the UR with UR identifier BDB5DAA67E5540000000042E01020000 (Figure 5-16).

INQUIRE EXCI STATUS: RESULTS Exc(CITGR1G.CITGR1G. - X1 ) Tas(0000604) Uri(BDB5DAA67E5540000000042E01020000) SYSID=TGR1 APPLID=A6POTGR1 RESPONSE: NORMAL TIME: 15.08.10 DATE: 10.04.05 PF 1 HELP 3 END 5 VAR 7 SBH 8 SFH 9 MSG 10 SB 11 SF


Figure 5-16 RRS UR identifier details

Figure 5-16 shows:

� That the UR is in-flight

� That the RRMS work manager is the Gateway daemon with RM name CTG.RRM.CITGR1G.IBM.UA

� That the CICS region with RM name DFHRXDM.A6POTGR1.IBM is a participant in the UR

For most processing, URs are fleeting and you will normally have difficulty in displaying them. However, RRS provides an archive log for all URs so that you can investigate URs after a transaction has completed (see “RRS Archive logs” on page 187).

CICS TG JNI traceBy default, details of specific transactional requests are not externalized by the CICS TG. It is essentially a passive component between WebSphere Application Server and CICS. Therefore, to uncover evidence of transactions in the Gateway daemon, we turned on JNI trace.

Example 5-5 shows the JNI trace for our extended LUW test scenario. The trace has been edited and annotated for clarity purposes and only selected trace points are shown (although the correct sequence has been preserved).

RRS Unit of Recovery Details Row 1 to 1 of 1 Commands r-Remove Interest v-View URI Details UR identifier : BDB5DAA67E5540000000042E01020000 Create time : 2005/10/04 13:08:01.680410 Comments : UR state : InFlight UR type : Prot System : X1 Logging Group : X1 SURID : N/A Work Manager Name : CTG.RRM.CITGR1G.IBM.UA Display Work IDs Display IDs formatted Luwid . : Present Eid . . : Not Present Xid . . : Present Expressions of Interest: S RM Name Type Role DFHRXDM.A6POTGR1.IBM Prot Participant ******************************* Bottom of data ******************************** Command ===> Scroll ===> PAGE F1=Help F2=Split F3=Exit F7=Up F8=Down F9=Swap F12=Cancel


Example 5-5 JNI trace of an Extended LUW

1st ECI request - EXTENDED MODE DPL with LUW-token=0----------------------------------------------------15:08:01.669 CTG6800I ECI parameters on entry. Call_Type = 12, Extend_Mode = 1, Luw_Token = 0, Commarea_Length = 38, Cics_Rc = 0, Flags = 0, outbound Commarea length = 2.15:08:01.680 CTG6886I About to issue EXCI DPL call. Pipe token =0xf6c8140 , eci system name =A6POTGR1, program name =ECIPROG .15:09:01.758 CTG6804I Returned commarea information. Commarea length = 38, commarea address = f909dc0, inbound commarea data length =38.15:09:01.759 CTG6805I ECI parameters on exit. Call_Type = 12, Extend_Mode = 1, Luw_Token = 16777217, Commarea_Length = 38, Cics_Rc = 0, AV = 0.

2nd ECI request - EXTENDED MODE DPL with LU-token=16777217----------------------------------------------------------15:09:01.764 CTG6800I ECI parameters on entry. Call_Type = 12, Extend_Mode = 1, Luw_Token = 16777217, Commarea_Length = 38, Cics_Rc = 0, Flags = 0, outbound Commarea length = 2.15:09:01.766 CTG6886I About to issue EXCI DPL call. Pipe token =0xf6c8140 , eci system name =A6POTGR1, program name =ECIPROG .15:10:01.789 CTG6804I Returned commarea information. Commarea length = 38, commarea address = f907b50, inbound commarea data length =38.15:10:01.790 CTG6805I ECI parameters on exit. Call_Type = 12, Extend_Mode = 1, Luw_Token = 16777217, Commarea_Length = 38, Cics_Rc = 0, AV = 0.

3rd ECI request - EXTENDED MODE COMMIT with LU-token=16777217-------------------------------------------------------------15:10:01.795 CTG6800I ECI parameters on entry. Call_Type = 1, Extend_Mode = 2, Luw_Token = 16777217, Commarea_Length = 0, Cics_Rc = 0, Flags = 0, outbound Commarea length = 0.15:10:01.796 CTG6804I Returned commarea information. Commarea length = 0, commarea address = 0, inbound commarea data length =0.15:10:01.803 CTG6805I ECI parameters on exit. Call_Type = 1, Extend_Mode = 2, Luw_Token = 0, Commarea_Length = 0, Cics_Rc = 0, AV = 0.

The sequence of trace entries is as follows:

1. The first call to program ECIPROG initiates a new transaction. We see Extend_Mode=1 (ECI_EXTENDED) and LUW_Token=0. When this first call completes successfully, the COMMAREA data has been received and an Luw_Token 16777217 that represents the Extended LUW is assigned.

2. The second call to ECIPROG continues within the same context: Extend_Mode=1 and the same Luw_Token. The call ends normally.

3. The third call is for Extend_Mode=2 (ECI_COMMIT) with the same Luw_Token. This call requests the long-running mirror task in CICS to perform a syncpoint, committing all updates to recoverable resources performed during the extended LUW.

RRS Archive logsThe RRS archive log shows information relating to all RRS URs, including completed URs. We selected option 1 from the RRS primary menu panel and the


Browse an RRS log stream menu panel was displayed. We then chose to view the summary information of the RRS archive log.

We searched the RRS Archive log for CTG.RRM.CITGR1G.IBM.UA (the value of CTG_RRMNAME for our Gateway daemon). After we located the UR entries that were associated with our Gateway, we were able to check that the transaction was indeed Committed (Figure 5-17). The URID corresponds to the UR that we observed during the time when the transaction was in-flight.

Figure 5-17 RRS Archive summary log entry for committed transaction

Extended LUW workloadThis test scenario used 15 Web browsers, each running the CTGTesterCCI application and specifying a single interaction per request. The COMMAREA data sent used for the request was either DW (write to recoverable TS Queue

Attention: If the write activity to the RRS ARCHIVE logs is very high, this might impact the performance throughput of ALL RRS transactions if this log stream is defined and actively in use by RRS. This log stream is optional and only needed for any type of post transaction history type of investigation. We use the RRS archive log in order to illustrate the behavior of transactions. However, in a production system we recommend that you disable RRS archive logging.

Menu Utilities Compilers Help -------------------------------------------------------------------------------

BROWSE CITGRJ.ATR.REPORT Line 00062429 Col 001 080 X1 2005/10/04 15:10:01.802427 BLOCKID=00000000001ADC68 URID=BDB5DAA67E5540000000042E01020000 JOBNAME=CITGR1G USERID=CITGR1G PARENT URID=00000000000000000000000000000000 SURID=N/A WORK MANAGER NAME=CTG.RRM.CITGR1G.IBM.UA SYNCPOINT=Commit RETURN CODE=00000000 START=2005/10/04 13:10:01.796392 COMPLETE=2005/10/04 13:10:01.801917 EXITFLAGS=00800000 LUWID=PSSCX9.A6POTGR1 B5DAA6F51481 0001 TID= GTID= FORMATID=003654931682 (decimal) D9D9D4E2 (hexadecimal) GTRID= F0F0F2F0F6F4F1F1F6C9C2D4F5F1F0F0F0F0F0F0F0F6F5F5F2F9BDB5DAA6F51A1E0D BQUAL= D9D9D4E24BBDB5DAA6F51A236DF0F0F2F0F6F4F1F1F6C9C2D4F5F1F0F0F0F0F0F0F0F6F5F5F2F9 Command ===> Scroll ===> PAGE F1=Help F2=Split F3=Exit F5=Rfind F7=Up F8=Down F9=Swap F10=Left F11=Right F12=Cancel


and then delay) or AW (write to recoverable TS Queue and then abend). The workload consisted of the following:

� Five requests with DW � Five requests with AW � Five requests with DW

Each of the first five requests started a separate local transaction, the first of which spanned only one minute but the fifth of which spanned five minutes because it needed to wait for the lock on the TS Queue.

Requests 6, 7, 8, 9, and 10 each made their TS Queue updates but then abended with abend code ECIP. Each failed request caused an ResourceException to be thrown by the ECIInteraction.execute() call from the session EJB.

Upon catching this exception, the application issues setRollbackOnly() on the transaction context. At the end of the execute() method, the WebSphere Application Server EJB container finds that a rollback has been requested and drives a rollback of the transaction.

Requests 11 through 15 then followed the same pattern as requests1 through 5 and made their updates to the TS Queue over the last five minutes of the test. At the end of the test, the recoverable TS Queue RECIPROG contained ten new entries.

We used this test scenario to demonstrate the monitoring of WebSphere transactions using Tivoli Performance Viewer and to show a JNI trace of an extended LUW abnormal termination.

Tivoli Performance Viewer - Transaction Manager moduleConfiguring Tivoli Performance Viewer is covered in “Tivoli Performance Viewer” on page 110. For the purpose of transaction monitoring, we modified the settings using the WebSphere Administrative Console as follows:

1. We selected Monitoring and Tuning → Performance Monitoring Infrastructure (PMI) in the Welcome panel and then selected server1. In the Currently monitored statistic set panel, we switched from the Extended set to Basic. Because we are only using local transactions in these scenarios, there is no need to distinguish between local and global transactions.

2. We then selected OK to apply these changes, saved them to the repository, and restarted the application server.

3. After the restart, in the Welcome panel we selected Monitoring and Tuning → Performance Viewer → Current Activity → server1. In the left-side panel of the screen, we selected server1 → Performance Modules. We disabled the Connection Pool monitoring that we enabled previously by


deselecting JCA Connection Pools → ECIResourceAdapter → eis/CITGR1CI-tx1-13101, and we select Transaction Manager (as shown in Figure 5-18).

Figure 5-18 Tivoli Performance Viewer - Current Transaction Activity

Figure 5-19 shows a snapshot of the Transaction Manager statistics that are provided by the Basic statistics set. The snapshot was taken after requests 1 through 13 have completed but while requests 14 and 15 are still in-flight.

The (concurrent) active transactions peaked at 15 after all 15 requests have been submitted and gradually fall as each transaction waiting for write access to the TS Q is dispatched and the following one minute delay expires.

After the first five requests complete, the graph shows that five transaction rollback events appear to occur at an instant. The requests submitted with COMMAREA data AW did not contain the one minute delay, and so all five transaction rollbacks occurred within one monitoring interval. After this point, the number of active transactions was down to five. Each of the remaining transactions completes, bumping the transaction commit count up by one at the end of each one minute delay.

At the end of the monitoring period snapshot, the graph ends with the last two transactions still in-flight.


Figure 5-19 The Tivoli Performance Viewer Transaction Manager during the Extended LUW scenario

CICS CEBR commandWe used the CICS-supplied transaction CEBR to monitor the state of the recoverable TS Queue RECIPROG. The data in each entry reflects the APPLID, date and time, CICS user ID and task start code written at the start of each invocation (before the delay).


Figure 5-20 shows that the first five requests started at approximately one second intervals. There is then a gap in the time stamps, during which five requests result in task abends and the TS Queue updates are backed out. Finally, before the last five requests end normally and the TS Queue updates are made.

Figure 5-20 CEBR transaction showing committed TS Queue updates

CICS TG JNI traceExample 5-6 shows the JNI trace for one extended LUW which ends abnormally during the workload test scenario. The trace has been edited and annotated for clarity purposes, and only selected trace points are shown (although the correct sequence has been preserved).

Example 5-6 JNI trace an Extended LUW abnormal termination

ECI request - EXTENDED MODE DPL with LUW-token=0------------------------------------------------11:55:05.362 CTG6800I ECI parameters on entry. Call_Type = 12, Extend_Mode = 1, Luw_Token = 0, Commarea_Length = 38, Cics_Rc = 0, Flags = 0, outbound Commarea length = 2.11:55:05.365 CTG6886I About to issue EXCI DPL call. Pipe token =0x17f98180 , eci system name =A6POTGR1, program name =ECIPROG .

Error response - Cics_Rc=-7 means transaction abended -----------------------------------------------------11:59:50.055 CTG6887I EXCI DPL call returned.11:59:50.056 CTG6822E EXCI function error. Function Call = 6, Response = 12, Reason = 422, Subreason field-1 = 0x00, subreason field-2 = 0x00, Cics_Rc = -7.11:59:50.056 CTG6823E EXCI DPL_REQUEST specific error. RESP value = 0x00, RESP2 value = 0x00, Abend Code = ECIP, Cics_Rc = -7.

CEBR TSQ RECIPROG SYSID TGR1 REC 1 OF 10 COL 1 OF 38 ENTER COMMAND ===> ************************** TOP OF QUEUE ******************************* 00001 A6POTGR1 05/10/05 11:54:49 CITGR1D D 00002 A6POTGR1 05/10/05 11:54:50 CITGR1D D 00003 A6POTGR1 05/10/05 11:54:51 CITGR1D D 00004 A6POTGR1 05/10/05 11:54:51 CITGR1D D 00005 A6POTGR1 05/10/05 11:54:52 CITGR1D D 00006 A6POTGR1 05/10/05 11:55:16 CITGR1D D 00007 A6POTGR1 05/10/05 11:55:17 CITGR1D D 00008 A6POTGR1 05/10/05 11:55:18 CITGR1D D 00009 A6POTGR1 05/10/05 11:55:19 CITGR1D D 00010 A6POTGR1 05/10/05 11:55:19 CITGR1D D ************************* BOTTOM OF QUEUE ***************************** PF1 : HELP PF2 : SWITCH HEX/CHAR PF3 : TERMINATE BROWSE PF4 : VIEW TOP PF5 : VIEW BOTTOM PF6 : REPEAT LAST FIND PF7 : SCROLL BACK HALF PF8 : SCROLL FORWARD HALF PF9 : UNDEFINED PF10: SCROLL BACK FULL PF11: SCROLL FORWARD FULL PF12: UNDEFINED


11:59:50.056 CTG6805I ECI parameters on exit. Call_Type = 12, Extend_Mode = 1, Luw_Token = 16842753, Commarea_Length = 38, Cics_Rc = -7, AV = 0.

EXTEND_MODE 4 means ECI_BACKOUT (WebSphere Application Server has requested the Gateway daemon to rollback)-------------------------------11:59:50.068 CTG6800I ECI parameters on entry. Call_Type = 1, Extend_Mode = 4, Luw_Token = 16842753, Commarea_Length = 0, Cics_Rc = 0, Flags = 0, outbound Commarea length = 0.11:59:50.069 CTG6804I Returned commarea information. Commarea length = 0, commarea address = 0, inbound commarea data length =0.11:59:50.072 CTG6805I ECI parameters on exit. Call_Type = 1, Extend_Mode = 4, Luw_Token = 0, Commarea_Length = 0, Cics_Rc = 0, AV = 0.


1. The first trace entry shows the initiation of an Extended LUW (Extend_Mode=1, Luw_Token=0) for the invocation of ECIPROG. However, the supplied COMMAREA data in this case was AW, instructing the target program to issue an abend (code ECIP) after writing to the TS Queue.

2. The second trace entry shows that the abended task resulted in Cics_Rc=-7 (ECI_ERR_TRANSACTION_ABEND). It is this error that percolates back to the Session EJB as a ResourceException, thrown by the ECIInteraction.execute() method.

3. After the failure of the ECI request, the CTGTesterCCI application catches the Resource exception and calls the setRollbackOnly() method on the current context. Therefore, when the container managed transaction reaches syncpoint, WebSphere Application Server rolls back the local transaction. We see the Extend_Mode=4 (ECI_BACKOUT) request in the JNI trace (together with the Luw_Token 16842753 which was returned on the initial ECI request). The backout call to CICS then succeeds with Cics_Rc=0.

Note that we would also see evidence of this extended LUW failure in the RRS archive log.

5.7 Configuring for XA transactionsIn this section, we show how we configured our setup to support XA transactions using the CICS ECI XA resource adapter (cicseciXA.rar). We needed to consider the following configuration and setup tasks:

� Configuring CICS

Checking that our CICS TS region was enabled to support Transactional EXCI.


� CICS TG for z/OS

– Enabling CTGRRMS services– Configuring the Gateway daemons to register with RRS – Configuring the Gateway daemons with XA support enabled

� WebSphere Application Server

– Updating the J2EE application logic– Installing the CICS TG XA resource adapter– Defining connection factories based on the XA resource adapter– Configuring WebSphere transaction logging– Deploying our sample application CTGTesterCCIXA



Value Gateway daemon 1 Gateway daemon 2 CICS region 1

CICS region 2

hostname tx1.mop.ibm.com tx1b.mop.ibm.com -

IP address 9.100.193.122 9.100.193.121 -

TCP/IP port 13101 12101 -

Job name CITGR1G CITGT1G CITGR1CI CITGT1CI

APPLID A6POTGR1 A6POTGT1

NETNAME CITGR1G CITGT1G -

CONNECTION GR1G GT1G

RRM Name CTG.RRM.CITGR1G.IBM.UA

CTG.RRM.CITGT1G.IBM.UA


5.7.2 Configuring CICS No additional CICS setup is required for XA transaction support over and above the setup documented for Extended LUW support in 5.6.2, “Configuring CICS” on page 175.

5.7.3 Configuring CICS TG We needed to consider the following CICS TG configuration steps in order to enable XA transaction support:

� Enabling CTGRRMS services� Naming considerations for RRM� Enabling XA support in the Gateway daemon

Enabling CTGRRMS servicesTo provide support for XA transactions, the Gateway daemon makes use of RRMS facilities which require the calling address space to be registered as an authorized resource manager. A new address space called CTGRRMS acts as an agent for the Gateway daemon, providing access to the RRS capabilities required to support XA transactions.

CTGRRMS is a non-terminating address space, which will usually remain active until the next IPL of the MVS image or LPAR. CTGRRMS does not execute code itself beyond it’s own initialization, but does provide services used by any Gateway daemon in the same LPAR which supports XA transactions.

During initialization, a Gateway daemon with XA transaction support enabled checks the availability of the CTGRRMS address space. If CTGRRMS is not available, then the Gateway daemon requests the CTGRRMS Address Space Initiator (ctgasi) to start up CTGRRMS. Gateway daemon initialization fails if the CTGRRMS address space is unavailable and cannot be started.

Example 5-7 shows the messages which are written to syslog for a successful initialization of CTGRRMS.

Example 5-7 Syslog - CTGRRMS initialization messages

CTG6241I Initializing CTGRRMS Services IEF170I 1 CTGRRMS CTG6241I Initializing CTGRRMS Services CTG6242I CTGRRMS Services open for business IEF170I 1 CTGRRMS CTG6242I CTGRRMS Services open for business

The ctgasi utility also has a command interface which can be used in the event that the CTGRRMS address space needs to be shut down for maintenance


purposes, see 5.8, “Operations” on page 219 for more information about using ctgasi commands.

Usage or initiation of the CTGRRMS address space is restricted to users with UPDATE authority to the RACF FACILITY class profile CTG.RRMS.SERVICE. We gave the user IDs that are used for the Gateway daemon started tasks access to this profile as follows:

REDEFINE CTG.RRMS.SERVICE CLASS(FACILITY) UACC(NONE)PERMIT CTG.RRMS.SERVICE CLASS(FACILITY) ID(CITGR1G) ACCESS(UPDATE)PERMIT CTG.RRMS.SERVICE CLASS(FACILITY) ID(CITGT1G) ACCESS(UPDATE)

In order to initiate the CTGRRMS address space, the hlq.SCTGLINK PDS member CTGINIT must be placed into the Linked List Area (LLA). We added CTGV61.SCTGLINK to the LLA and then refreshed the LLA using the MVS command:

F LLA,REFRESH

Naming considerations for RRMThe environment variable CTG_RRMNAME is used to specify the name that the Gateway daemon will use when registering with RRS. The rules that apply when specifying this name are described in 2.3.4, “Creating an STDENV file” on page 37.

The same guidelines for defining the CTG_RRMNAME environment variable apply to a Gateway daemon with XA support enabled, except that the requirement for the .UA (unauthorized) suffix does not apply. The name must be Sysplex-unique and a maximum of 32 characters.

We decided to keep the same CTG_RRMNAME names for our Gateway daemons so that the names used for RRS registration were CTG.RRM.CITGR1G.IBM.UA and CTG.RRM.CITGT1G.IBM.UA.

Enabling XA support in the Gateway daemonCICS TG V6.1 has a single configuration parameter (xasupport=on) which determines whether XA transactions are supported and, thus, whether the additional XA transaction infrastructure, configuration, and setup is required.

Tip: We were tempted to change the setting of CTG_RRMNAME to not have the .UA suffix. However, in Chapter 6, “Gateway daemon transactions with TCP/IP load balancing” on page 225, we show how these Gateway daemons can be used in a CICS TG group and the CTG_RRMNAME of Gateway daemons in a CICS TG group must be suffixed with .UA. We, therefore, decided to keep the same name throughout the sequence of tests.


To ease migration from earlier versions of CICS TG you might find it convenient to run the Gateway daemon with the default setting of xasupport=off, until such time as XA transaction support is required. Here, we explore the configuration steps required when xasupport=on is specified.

We edited the Gateway daemon configuration files CITGR1G.ini and CITGT1G.ini to specify xasupport=on.

The CTG_XA_MAX_TRAN environment variable specifies the maximum number of concurrent XA transactions that the Gateway daemon can support. We used the default setting of 1000.

After starting the Gateway daemon we noted the following message during initialization:

CTG6737I XA transaction support is enabled

AFP Authorization of CTGINITSome MVS system functions, such as functions provided by RRS, are sensitive and access to these functions is restricted to only authorized programs, operating in a privileged mode know as supervisor state. to avoid compromising the security and integrity of the system. MVS provides a mechanism called the Authorized Program Facility (APF) to restrict the access to these sensitive system functions or programs to avoid integrity exposures. This allows you to specify which libraries contain those special functions or programs and prevent unauthorized programs from using them. Those libraries are then called APF libraries.

By default the libraries in the LNKLST concatenation are authorized (unless LNKAUTH=APFTAB is specified in the IEASYSxx parameter list). However, if the system accesses the libraries in the LNKLST concatenation via JOBLIB or STEPLIB DD statements, the system does not consider those libraries authorized unless you specified the library name in the APF list by using either the IEAAPFxx or the PROGxx parmlib member. If a JCL DD statement concatenates an authorized library in any order with an unauthorized library, the entire set of concatenated libraries becomes unauthorized.

In order for a Gateway daemon to use XA support, CTGINIT (supplied in the SCTGLINK library) must be in the LNKLST and in an APF authorized library.

Note: If the Gateway daemon is part of a TCP/IP load balancing group, CTG_XA_MAX_TRAN is specified by the CICS TG master process responsible for the group (see Chapter 6, “Gateway daemon transactions with TCP/IP load balancing” on page 225).


For more details about LNKIST and authorized libraries, refer to ABCs of z/OS System Programming Volume 2, SG24-6982.

5.7.4 Configuring WebSphere To test our XA transaction configuration we wrote a new application CTGTesterCCIXA which is an extension of the CTGTesterCCI application. CTGTesterCCIXA contains two ECI resource references and can be used to call programs running in two CICS regions via different connection factories. For further details about CTGTesterCCIXA refer to Appendix A, “Sample applications” on page 361.

Dealing with CICS transaction abendsCICS cannot tell RRS to rollback updates to recoverable resources following an unhandled abend of the long-running mirror task. Thus, by default, if the first JCA request of an XA transaction succeeds but the second JCA request fails (because of a CICS transaction abend), the updates made by the first request are committed but the updates made by the second request cannot be committed.

There are two ways of avoiding such an inconsistency resulting from a CICS transaction abend:

� The J2EE application can abandon the current global transaction by calling setRollbackOnly() on the current context.

In the event of a transaction abend, a CICSTxnAbendException is thrown by the CICS ECI XA resource adapter and the decision to rollback or to commit resources that are updated by the global transaction can be taken by the J2EE application.

� The option ABENDBKOUT=YES can be set in the DFHXCOPT table (see “ABENDBKOUT option in DFHXCOPT” on page 165).

In the event of a transaction abend, the global transaction is rolled back automatically at the end of the EJB method call.

Tip: You can use the D PROG,APF,ALL console command to display the APF authorized library list.

Note: At the time of writing this book , we were unable to test the ABENDBKOUT option. We, therefore, chose to include transaction abend rollback logic in the sample application CTGTesterCCIXA.


We coded the CTGTesterCCIXA application to issue a setRollbackOnly() on the EJB session context in the catch block for the ECIInteraction.execute() call. This ensures that the CICS abend is propagated onto WebSphere Application Server as the transaction manager and that the global transaction (if one exists) is rolled back. We used the same code as described in Example A-1 on page 368.

Installing the cicseciXA resource adapter The process of installing the CICS ECI XA resource adapter (cicseciXA.rar) is identical to that described for the ECI resource adapter (see 3.2.1, “Installing the resource adapter” on page 80).

Both resource adapters can be installed at the same time. Figure 5-21 shows the two resource adapters installed into our WebSphere Application Server for Windows.

Figure 5-21 Administrative Console - CICS ECI resource adapters


Creating connection factoriesThe process of creating connection factories using the CICS ECI XA resource adapter is the same as described in 3.2.2, “Creating connection factories” on page 82. We created two connection factories CITGR1CI-tx1-13101-XA and CITGR1CI-tx1b-12101-XA based upon the CICS ECI XA resource adapter . Figure 5-22 shows our connection factories in the WebSphere Administrative Console.

Figure 5-22 Administrative Console - CICS ECI XA connection factories

The two connection factories represent two connections to two different CICS regions which are accessed via two different Gateway daemons. The Gateway daemons each bind to different IP addresses.

Configuring WebSphere transaction logging We used the same WebSphere transaction logging configuration that we describe in “Configuring WebSphere transaction logging” on page 180.

Deploying our sample applicationTo deploy the CTGTesterCCIXA application we followed the same procedure as is described in 3.2.3, “Deploying the sample J2EE application” on page 89. This time we needed to map two resource references using the WebSphere Administrative Console, as shown in Figure 5-23.

Tip: Before using the XA transaction support, you should ensure that your WebSphere Application Server level contains the fix for APAR PQ99940. This APAR fixes a connection pooling performance problem following a connection failure, for example, as a result of a CICS region or Gateway daemon failure. The fix is contained in V6.0.2 of WebSphere Application Server for multi-platforms.


Figure 5-23 Administrative Console - mapping XA resource references

After CTGTesterCCIXA was successfully deployed, we started the application.

5.7.5 Testing the XA transaction configurationIn this section, we detail how we tested the XA transaction configuration using our sample J2EE application CTGTesterCCIXA. Figure 5-24 shows the basic outline of our environment. We used this configuration for two scenarios:

� Normal termination of an XA transaction

An update and commit of recoverable resources in two CICS regions within one XA transaction, via two Gateway daemons (see “XA transaction normal termination” on page 204).

� Abnormal termination of an XA transaction

A CICS subsystem failure during an in-flight XA transaction (see “XA transaction abnormal termination” on page 214).


Figure 5-24 XA transaction scenario

To test our configuration, we used our test application CTGTesterCCIXA via its Web browser interface (Figure 5-25).

Note: On our system the Gateway daemons bind to different TCP/IP addresses but are actually running on the same z/OS image. In practice, they could be running on two completely different systems.

z/OStx1.mop.ibm.com

HTML


JSP/Servlet

CTGTesterCCIXA

CICS ECI XA

Resource Adapter

CCI

Windows

CICS TG V6.1

Cam21-pc11

TransactionRequired

ECI Requestwithin XA txn

RRS

CITGR1G


CITGR1CI (applid A6POTGR1)

CICS

EXCI

JNI

13101

Gatewaydaemon

CTGRRMS

CITGT1G


CICS

EXCI

JNI

12101

Gatewaydaemon

tx1b.mop.ibm.com


CITGT1CI (applid A6POTGT1)


Figure 5-25 CTGTesterCCIXA Start Web browser interface

During these tests we specified DW as the input to the COMMAREA, which means that the ECIPROG program writes to a recoverable TS Queue and then delays for one minute.


The iteration count was set to 1 for transaction leg 1 and 5 for transaction leg 2. Thus, clicking Submit results in a total of six ECI calls: 1 request to CICS region CITGR1CI (applid A6POTGR1) and 5 requests to CICS region CITGT1CI (applid A6POTGT1). This iteration count was done to give us six minutes during which to monitor the transaction state across the various components of the system.

During the course of these scenarios, we monitored the transaction using each of the following :

CICS CEMT commandCICS TG JNI trace outputRRS ISPF panels RRS related Work Manager information

We discuss each of these tools in the next section:

XA transaction normal termination This test demonstrates successful update and commit of resources on two CICS regions accessed via different connection factories within the scope of a single global transaction.

The Web browser appeared to be busy for six minutes before displaying the results page that contained the COMMAREAs returned from the two legs of the global transaction. At the end of the test, the recoverable TS Queue RECIPROG on CICS region CITGR1CI (applid A6POTGR1) contained one new entry, and the recoverable TS Queue RECIPROG on CICS region CITGR1CI (applid A6POTGT1) contained five new entries.

CICS CEMT commandWhile the second ECI request was in-flight on A6POTGT1, we issued the CEMT INQUIRE TASK command on both A6POTGR1 and A6POTGT1. Figure 5-26 shows the ECIT mirror task running on A6POTGR1 as Task 182 suspended in RRMSEXIT state, which means that CICS is waiting for a commit or backout flow from RRS.

Program ECIPROG on A6POTGR1 had completed at this stage but was waiting for syncpoint of the global transaction from the transaction manager (WebSphere Application Server) to be communicated via the Gateway daemon CITGR1G and RRS. Also at this point in time, the mirror transaction on A6POTGT1 was suspended in ICWAIT due to the active EXEC CICS DELAY in ECIPROG.

Note: This scenario is for testing purposes only. Normal transactions would not be expected to delay in this way.


Figure 5-26 CEMT INQUIRE TASK on A6POTGR1 with the mirror in RRMSEXIT state

On A6POTGT1, we also issued the CEMT INQUIRE EXCI command. Figure 5-27 shows the EXCI information for the long-running mirror task ECIT (task 67) including the associated RRS UR identifier.

Figure 5-27 CEMT INQUIRE EXCI on A6POTGT1 showing a UR identifier

Later, we show that the UR identifier shown on CEMT INQUIRE EXCI panel matched the active and archived data accessible by the RRS ISPF panels (see Figure 5-28 on page 209). The INQUIRE EXCI command provides the information to correlate an active long-running mirror task in a CICS region, with the RRS UR id associated with a global transaction. The Exc field highlights the job name and connection name with which the UR identifier is associated.

CICS TG JNI traceBy default, details of specific transactional requests are not externalized by the CICS TG. It is essentially a passive component between WebSphere Application Server and CICS. Therefore, in order to uncover evidence of transactions in the Gateway daemon, we turned on JNI trace.

INQUIRE TASK STATUS: RESULTS - OVERTYPE TO MODIFY Tas(0000181) Tra(CEMT) Fac(N197) Run Ter Pri( 255 ) Sta(TO) Use(CITGR1D ) Uow(BDBD86A2A138664D) Tas(0000182) Tra(ECIT) Sus Tas Pri( 001 ) Sta(TO) Use(CITGR1D ) Uow(BDBD87EC2BB8A361) Hty(RRMSEXIT) SYSID=TGR1 APPLID=A6POTGR1 RESPONSE: NORMAL TIME: 17.41.40 DATE: 10.10.05 PF 1 HELP 3 END 5 VAR 7 SBH 8 SFH 9 MSG 10 SB 11 SF

INQUIRE EXCI STATUS: RESULTS Exc(CITGT1G.CITGT1G. - X1 ) Tas(0000067) Uri(BDBD88267E554A5C0000004801020000) SYSID=TGT1 APPLID=A6POTGT1 RESPONSE: NORMAL TIME: 17.42.09 DATE: 10.10.05 PF 1 HELP 3 END 5 VAR 7 SBH 8 SFH 9 MSG 10 SB 11 SF


In the previous Extended LUW scenario, we concentrated on the individual ECI requests and the use of LUW tokens. In this case, we highlight the start, prepare, and commit flows that are associated with XA transactions. We also highlight the role of the X/Open identifier (XID).

We found that the Gateway daemon trace reveals the presence of XID information for an ECI request, but does not trace out the actual value. JNI trace writes out the full formatted XID information that we could use to correlate with information available via the RRS ISPF panels.

Example 5-8 shows the JNI trace for the ECI requests flowed to A6POTGT1 through the Gateway daemon CITGT1G. The trace has been edited and annotated for clarity purposes, and only selected trace points are shown (although the correct sequence has been preserved).

Example 5-8 Annotated summary of JNI trace from CITGT1G for leg 2 of the global transaction

Initialization of XA support----------------------------17:39:03.076 CTG8660I Two phase commit support enabled

Start of an XA transaction--------------------------17:41:30.747 CTG8602I JNI Method Java_com_ibm_ctg_server_ServerXARequest_CcicsXAStart entry17:41:30.748 CTG9205I Before call to Begin_Context17:41:30.749 CTG9206I After successful call to Begin_Context17:41:30.761 CTG9206I After successful call to Express_UR_InterestUR ID : BDBD88267E554A5C0000004801020000UR Context : BDBD88262CAE980D0200000002A3E450UR Interest : 7E34C1000000001E0055000155555555XID.formatID : 1463898948XID.gtrid : 00-0F 00000106 D5D837F0 00000001 00000039 10-1F 056726FA 5740F82D ADB83B28 0239F181 20-23 BC7F5B5CXID.bqual : 00-0F 00000106 D5D837F0 00000001 00000039 10-1F 056726FA 5740F82D ADB83B28 0239F181 20-2F BC7F5B5C 00000001 00000000 00000000 30-35 00000000 000217:41:30.765 CTG8604I JNI Method Java_com_ibm_ctg_server_ServerXARequest_CcicsXAStart exit (rc = 0).

Gateway process sequence of five ECI Requests as part of XA transaction----------------------------------------------------------------------

Note: An X/Open identifier (XID) acts as the identifier for a distributed unit of work. An XID consists of a Global transaction identifier (GTRID) and a Branch qualifier (BQUAL).


17:41:30.835 CTG6800I ECI parameters on entry. Call_Type = 12, Extend_Mode = 0, Luw_Token = 0, Commarea_Length = 38, Cics_Rc = 0, Flags = 0, outbound Commarea length = 2.17:41:30.837 CTG8635I XID present on ECI Request.17:41:30.862 CTG6886I About to issue EXCI DPL call. Pipe token=0xf6c6510 , eci system name=A6POTGT1, program name=ECIPROG .17:42:31.122 CTG6887I EXCI DPL call returned.17:42:31.123 CTG6805I ECI parameters on exit. Call_Type = 12, Extend_Mode = 0, Luw_Token = 0, Commarea_Length = 38, Cics_Rc = 0, AV = 0.*Above DPL sequence repeated five times*

Gateway receives prepare from WebSphere Application Server----------------------------------------------------------17:46:32.576 CTG8602I JNI Method Java_com_ibm_ctg_server_ServerXARequest_CcicsXAPrepare entry17:46:32.587 CTG8604I JNI Method Java_com_ibm_ctg_server_ServerXARequest_CcicsXAPrepare exit (rc = 0).

Gateway receives Commit from WebSphere Application Server---------------------------------------------------------17:46:32.794 CTG8602I JNI Method Java_com_ibm_ctg_server_ServerXARequest_CcicsXACommit entry17:46:32.806 CTG8604I JNI Method Java_com_ibm_ctg_server_ServerXARequest_CcicsXACommit exit (rc = 0).


1. The first trace entry that we highlight shows that the Gateway daemon has initialized with XA support enabled.

2. The second set of trace entries show the start of the XA transaction and the Global transaction identifier (GTRID) and Branch qualifier (BQUAL). In the next section we explain how we obtained and subsequently used the GTRID to monitor the in-flight transaction using the RRS ISPF panels.

3. The third set of entries show the ECI requests.

4. The fourth set of entries show the prepare flow of the two-phase commit process.

5. The fifth set of entries show the commit flow of the two-phase commit process.

RRS panels from ISPFThe RRS panels in ISPF can be used to monitor progress of in-flight transactions down to a very low level of detail. We navigated through the menus, drilling down from the Gateway RRMNAME to the status of individual RM exit routines for a particular transaction.

Note: Extend_Mode=0 and Luw_token=0 for these ECI requests, but it is the presence of an Xid which indicate that the requests are part of a global transaction.


We show here, for illustrative purposes, a worked example of how the RRS panels can be navigated in order to uncover information about an in-flight UOW. For most processing, URs are fleeting and you will normally have difficulty in displaying them. However, RRS provides archive logs for all URs so that you can investigate URs after a transaction has completed (for example, see “RRS Unit of Recovery log” on page 212).

1. We selected option 2 (Display/Update RRS related Resource Manager information) from the RRS primary menu panel.

2. On the RRS Resource Manager selection panel, we entered the pattern CTG.RRM.CITG?1G.IBM.UA. The question mark (?) is used here as a wildcard.

3. The next panel (RRS Resource Manager List) showed both CITGR1G and CITGT1G resource managers in Run state. Because the first leg of our transaction runs on CITGR1G, we entered u-View URs against that Resource Manager entry (during the first minute of the test, there is no UR active in CITGT1G).

4. The RRS Unit of Recovery List panel showed a single in-flight UR associated with our Resource Manager CITGR1G. We selected v-View Details, which gave us the RRS Unit of Recovery Details panel (Figure 5-28).

Figure 5-28 shows:

– That the UR BDBD87EC7E5546E80000007C01020000 is in-flight

– That the RRMS work manager is the Gateway daemon with RM name CTG.RRM.CITGR1G.IBM.UA

– That the Gateway daemon is a syncpoint coordinator or SDSRM (Server Distributed syncpoint Resource Manager)

– That the CICS region with RM name DFHRXDM.A6POTGR1.IBM is a participant in the UR

Tip: In order to specify selections in this way on the RRS ISPF panels, it is important to have a structured naming convention for CTG_RRMNAME.


Figure 5-28 RRS Unit of Recovery information for CTG.RRM.CITGR1G.IBM.UA

By selecting the Display Work IDs and the Display IDs formatted options, we are able to see the formatted XID information that is associated with the global transaction from which we took a copy of the hexadecimal GTRID (160 bytes). We are able to tie up this XID with the details written out to the JNI trace.

5. We navigate back to the RRS primary menu panel and switch to option 3 (Display/Update RRS Unit of Recovery information). Paging down on the RRS Unit of Recovery Selection panel, we paste the GTRID directly into the input field (Figure 5-29) and press Enter.

RRS Unit of Recovery Details Row 1 to 2 of 2 Commands r-Remove Interest v-View URI Details UR identifier : BDBD87EC7E5546E80000007C01020000 Create time : 2005/10/10 15:40:29.803693 Comments : UR state : InFlight UR type : Prot System : X1 Logging Group : X1 SURID : N/A Work Manager Name : CTG.RRM.CITGR1G.IBM.UA / Display Work IDs / Display IDs formatted Luwid . : Present Eid . . : Not Present Xid . . : Present Expressions of Interest: S RM Name Type Role CTG.RRM.CITGR1G.IBM.UA Prot SDSRM DFHRXDM.A6POTGR1.IBM Prot Participant ******************************* Bottom of data ********************************


Figure 5-29 RRS Unit of Recovery selection using GTRID

This query displays a list of all URs that have an interest in the global transaction that is represented by the GTRID (Figure 5-30). We found that RRS knew about two URs that are associated with our GTRID. By now, the first transaction leg through the Gateway daemon CITGR1G has completed and the second leg processed by Gateway daemon CITGT1G has begun.

Figure 5-30 In-flight URs associated with global transaction

RRS Unit of Recovery Selection Commands: save-Save Profile get-Get Profile ENTER-Query Profile Name . . Format ID (from 1 to 4294967295 in decimal) GTRID Pattern 00-0F 00000106 D5D837F0 00000001 00000039 10-1F 056726FA 5740F82D ADB83B28 0239F181 20-2F BC7F5B5C 30-3F 40-4F 50-5F 60-6F 70-7F BQUAL Pattern 00-0F 10-1F 20-2F Command ===> F1=Help F2=Split F3=Exit F5=ListErr F7=Up F8=Down F9=Swap F12=Cancel

RRS Unit of Recovery List Row 1 to 2 of 2 Commands: v-View Details c-Commit b-Backout r-Remove Interest f-View UR Family S UR Identifier System Logging Group State Type Comments BDBD87EC7E5546E80000007C01020000 X1 X1 InFlight Prot BDBD88267E554A5C0000004801020000 X1 X1 InFlight Prot ******************************* Bottom of data ********************************


6. We drill down into one of these URs, by entering v-View Details against the second entry to show the resource managers involved in the UR BDBD88267E554A5C0000004801020000.

Example 5-9 shows that CTG.RRM.CITGT1G.IBM.UA (Gateway daemon) and DFHRXDM.A6POTGT1.IBM (CICS region) are involved in the second UR. Note that for this UR, the Gateway daemon role is also as a syncpoint coordinator or SDSRM.

Example 5-9 Drilling down into the UR detail - RMs with Expressions of Interest

RRS Unit of Recovery Details Row 1 to 2 of 2 Commands r-Remove Interest v-View URI Details UR identifier : BDBD88267E554A5C0000004801020000 Create time : 2005/10/10 15:41:30.755128 Comments : UR state : InFlight UR type : Prot System : X1 Logging Group : X1 SURID : N/A Work Manager Name : CTG.RRM.CITGT1G.IBM.UA Display Work IDs / Display IDs formatted Luwid . : Present Eid . . : Not Present Xid . . : Present Expressions of Interest: S RM Name Type Role CTG.RRM.CITGT1G.IBM.UA Prot SDSRM DFHRXDM.A6POTGT1.IBM Prot Participant ***************************** Bottom of data *****************************

7. RRS offers still further detail on the status of each Resource Manager with an interest in a UR. We issued v-View URI details against the CITGT1G to see the summary of exit activity (Example 5-10). Because the transaction was still in-flight, the exit status list is largely Uncalled, but we observed how this could be useful detail when analyzing a hung transaction.

Example 5-10 Drilling down into the UR detail - state of CITGT1G’s UR Interest

RRS URI Details

UR identifier : BDBD88267E554A5C0000004801020000 URI token . . : 7E34C1000000001E0055000155555555 RM name . . . : CTG.RRM.CITGT1G.IBM.UA Type . . . . : Prot Status . : ACTIVE Role . . . . : SDSRM State . : InFlight SURID : N/A

Exit/State Status Duration


BACKOUT . . . : Uncalled COMPLETION . . : Uncalled COMMIT . . . . : Uncalled DSE/IN_DOUBT . : N/A End_UR . . . . : Uncalled EXIT_FAILED . : Uncalled ONLY_AGENT . . : Uncalled PREPARE . . . : Uncalled STATE_CHECK . : Uncalled

CICS CEBR commandWe used the CICS-supplied terminal transaction CEBR to monitor the state of our recoverable TS Queue RECIPROG. The data in each entry reflects the APPLID, data and time, CICS user ID and transaction start code written at the start of each invocation (before the delay).

Example 5-11 shows the contents of the RECIPROG TS Queue at the end of the global transaction.

Example 5-11 CEBR output showing committed TS Queue updates

CEBR TSQ RECIPROG SYSID TGR1 REC 1 OF 1 COL 1 OF 38 ENTER COMMAND ===> ************************** TOP OF QUEUE ******************************* 00001 A6POTGR1 10/10/05 17:40:30 CITGR1D D ************************* BOTTOM OF QUEUE *****************************

CEBR TSQ RECIPROG SYSID TGT1 REC 1 OF 5 COL 1 OF 38 ENTER COMMAND ===> ************************** TOP OF QUEUE ******************************* 00001 A6POTGT1 10/10/05 17:41:31 CITGT1D D 00002 A6POTGT1 10/10/05 17:42:31 CITGT1D D 00003 A6POTGT1 10/10/05 17:43:32 CITGT1D D 00004 A6POTGT1 10/10/05 17:44:32 CITGT1D D 00005 A6POTGT1 10/10/05 17:45:32 CITGT1D D ************************* BOTTOM OF QUEUE *****************************

RRS Unit of Recovery logWhen the transaction was complete, we returned to the RRS primary menu panel and selected option 1 (Browse an RRS log stream). We then chose option 2 (RRS Unit of Recovery State) logs to see a summary of completed URs. The


progress of the global transaction is shown in examples Example 5-12 and Example 5-13.

Example 5-12 RRS Unit of Recovery State log - CITGT1G prepare and in-doubt entries

READING ATR.X1.DELAYED.UR LOG STREAM

X1 2005/10/10 17:46:32.578661 BLOCKID=000000000013BBB1 URID=BDBD88267E554A5C0000004801020000 LOGSTREAM=ATR.X1.DELAYED.UR PARENT URID=00000000000000000000000000000000 SURID=N/A WORK MANAGER NAME=CTG.RRM.CITGT1G.IBM.UA STATE=InPrepare EXITFLAGS=00000000 FLAGS=04000000 LUWID=PSSCX9.A6POTGT1 BD882649B2E9 0001 TID= GTID=

FORMATID=001463898948 (decimal) 57415344 (hexadecimal) GTRID=00000106D5D837F00000000100000039056726FA5740F82DADB83B280239F181BC7F5B5C BQUAL=00000106D5D837F00000000100000039056726FA5740F82DADB83B280239F181BC7F5B5C000000010000000000000000000000000002

X1 2005/10/10 17:46:32.583921 BLOCKID=000000000013BE0D URID=BDBD88267E554A5C0000004801020000 LOGSTREAM=ATR.X1.DELAYED.UR PARENT URID=00000000000000000000000000000000 SURID=N/A WORK MANAGER NAME=CTG.RRM.CITGT1G.IBM.UA STATE=InDoubt EXITFLAGS=00000000 FLAGS=04000000 LUWID=PSSCX9.A6POTGT1 BD882649B2E9 0001 TID= GTID=


Example 5-12 shows the prepare and in-doubt entries for Gateway daemon CITGT1G. Similar entries were present in the log for the CITGR1G leg of the global transaction. Example 5-13 shows the commit entries for both legs of the transaction.

Example 5-13 RRS Unit of Recovery log - CITGR1G and CITGT1G commit entries

X1 2005/10/10 17:46:32.772517 BLOCKID=000000000013C5B1 URID=BDBD87EC7E5546E80000007C01020000 LOGSTREAM=ATR.X1.DELAYED.UR PARENT URID=00000000000000000000000000000000 SURID=N/A WORK MANAGER NAME=CTG.RRM.CITGR1G.IBM.UA


STATE=InCommit EXITFLAGS=00800000 FLAGS=06000000 LUWID=PSSCX9.A6POTGR1 BD87EC2BB0A8 0001 TID= GTID=


X1 2005/10/10 17:46:32.797277 BLOCKID=000000000013C855 URID=BDBD88267E554A5C0000004801020000 LOGSTREAM=ATR.X1.DELAYED.UR PARENT URID=00000000000000000000000000000000 SURID=N/A WORK MANAGER NAME=CTG.RRM.CITGT1G.IBM.UA STATE=InCommit EXITFLAGS=00800000 FLAGS=06000000 LUWID=PSSCX9.A6POTGT1 BD882649B2E9 0001 TID= GTID=


XA transaction abnormal terminationThis test demonstrates a failure scenario, using the CTGTesterCCIXA application. The configuration and application parameters are identical to those used in the scenario “XA transaction normal termination” on page 204. In this scenario, however, during the second leg of the transaction the CICS region A6POTGR1 is cancelled.

The second leg of the transaction continues, making the five updates to TS Queue RECIPROG on CICS region A6POTGT1. However, at the end of the final ECI request, the prepare call to the Gateway daemon CITGR1G (an SDSRM) for transaction leg 1 fails. This leads to a complete rollback of all updated resources.


CICS CEBR commandAt just over three minutes into the test scenario, we looked at the contents of the TS Queue RECIPROG on both A6POTGR1 and A6POTGT1. We could see that both queues contained updates (Example 5-14).

Example 5-14 CEBR output on A6POTGR1 and A6POTGT1 after three minutes


CEBR TSQ RECIPROG SYSID TGT1 REC 1 OF 3 COL 1 OF 38 ENTER COMMAND ===> ************************** TOP OF QUEUE ******************************* 00001 A6POTGT1 11/10/05 14:53:28 CITGT1D D 00002 A6POTGT1 11/10/05 14:54:28 CITGT1D D 00003 A6POTGT1 11/10/05 14:55:28 CITGT1D D ************************* BOTTOM OF QUEUE *****************************

At this point, we cancelled CICS region A6POTGR1 with the MVS command /CANCEL CITGR1CI.

RRS Archive LogThe RRS archive log shows information relating to all RRS URs, including completed URs. We selected option 1 from the RRS primary menu panel and the Browse an RRS log stream menu panel was displayed. We then chose to view the detailed information of the RRS archive log.

Example 5-15 shows the log entries for the XA transaction abnormal termination scenario.

Example 5-15 RRS Archive log - XA transaction abnormal termination

RRS/MVS LOG STREAM BROWSE DETAIL REPORT

READING ATR.X1.ARCHIVE LOG STREAM

X1 2005/10/11 14:58:29.007861 BLOCKID=00000000001BF925 URID=BDBEA43A7E5546E80000008201020000 JOBNAME=CITGR1G USERID=CITGR1G PARENT URID=00000000000000000000000000000000 SURID=N/A


WORK MANAGER NAME=CTG.RRM.CITGR1G.IBM.UA SYNCPOINT=AtraPrp RETURN CODE=0000012D START=2005/10/11 12:58:29.007325 COMPLETE=2005/10/11 12:58:29.007448 EXITFLAGS=00000000 LUWID=PSSCX9.A6POTGR1 BEA43A704F3A 0001 TID= GTID=

FORMATID=001463898948 (decimal) 57415344 (hexadecimal) GTRID=00000106DA64BED6000000010000003D056726FA5740F82DADB83B280239F181BC7F5B5C BQUAL=00000106DA64BED6000000010000003D056726FA5740F82DADB83B280239F181BC7F5B5C000000010000000000000000000000000001 RMNAME=CTG.RRM.CITGR1G.IBM.UA ROLE=SDSRM FLAGS=10020000 PROTOCOL=PresumeNothing StateCheck EXIT RC=Uncalled Prepare EXIT RC=Uncalled DistSp EXIT RC=Uncalled Commit EXIT RC=Uncalled Backout EXIT RC=00000000 EndUr EXIT RC=Uncalled ExitFailed EXIT RC=Uncalled Completion EXIT RC=Uncalled OnlyAgent EXIT RC=Uncalled RMNAME=DFHRXDM.A6POTGR1.IBM.UA ROLE=Participant FLAGS=04010000 PROTOCOL=PresumeAbort StateCheck EXIT RC=Uncalled Prepare EXIT RC=Uncalled DistSp EXIT RC=Uncalled Commit EXIT RC=Uncalled Backout EXIT RC=Uncalled EndUr EXIT RC=Uncalled ExitFailed EXIT RC=Uncalled Completion EXIT RC=Uncalled OnlyAgent EXIT RC=Uncalled

X1 2005/10/11 14:58:30.181361 BLOCKID=00000000001BFB8F URID=BDBEA4737E554A5C0000004E01020000 JOBNAME=CITGT1G USERID=CITGT1G PARENT URID=00000000000000000000000000000000 SURID=N/A WORK MANAGER NAME=CTG.RRM.CITGT1G.IBM.UA SYNCPOINT=AtraBak RETURN CODE=00000000 START=2005/10/11 12:58:28.982962 COMPLETE=2005/10/11 12:58:30.180898 EXITFLAGS=00000000 LUWID=PSSCX9.A6POTGT1 BEA474198443 0001 TID= GTID=

FORMATID=001463898948 (decimal) 57415344 (hexadecimal) GTRID=00000106DA64BED6000000010000003D056726FA5740F82DADB83B280239F181BC7F5B5C BQUAL=


00000106DA64BED6000000010000003D056726FA5740F82DADB83B280239F181BC7F5B5C000000010000000000000000000000000002 RMNAME=CTG.RRM.CITGT1G.IBM.UA ROLE=SDSRM FLAGS=10021000 PROTOCOL=PresumeNothing StateCheck EXIT RC=Uncalled Prepare EXIT RC=00000014 DistSp EXIT RC=Uncalled Commit EXIT RC=Uncalled Backout EXIT RC=00000000 EndUr EXIT RC=Uncalled ExitFailed EXIT RC=Uncalled Completion EXIT RC=Uncalled OnlyAgent EXIT RC=Uncalled RMNAME=DFHRXDM.A6POTGT1.IBM.UA ROLE=Participant FLAGS=10000000 PROTOCOL=PresumeAbort StateCheck EXIT RC=Uncalled Prepare EXIT RC=00000000 DistSp EXIT RC=Uncalled Commit EXIT RC=Uncalled Backout EXIT RC=00000010 EndUr EXIT RC=Uncalled ExitFailed EXIT RC=00000010 Completion EXIT RC=Uncalled OnlyAgent EXIT RC=Uncalled

The archive log entries show the following:

� There are two RRS URs involved in the global transaction with the same GTRID:

00000106DA64BED6000000010000003D056726FA5740F82DADB83B280239F181BC7F5B5C

� The RRS work manager for the first UR (URID=BDBEA43A7E5546E80000008201020000) is the Gateway daemon CITGR1G with CTG_RRMNAME CTG.RRM.CITGR1G.IBM.UA.

� The RRS work manager for the second UR URID=BDBEA4737E554A5C0000004E01020000) is the Gateway daemon CITGT1G with CTG_RRMNAME CTG.RRM.CITGT1G.IBM.UA.

� The Gateway daemons take a syncpoint coordinator or SDSRM role for their respective URs.

� The CICS regions have a Participant role within the URs.


� The log entry for the first UR shows a prepare call (AtraPrp) to CITGR1G returns with code 0x12D. The RRS manual SA22-7616 explains the return code as follows:

ATR_BACKED_OUT_OUTCOME_PENDING (0x12D)Meaning: The commit operation failed. The RRS decision was to return to the previous consistent state. However, the state of one or more of the protected resources is not known. Action: Continue normal processing for a backed out unit of recovery. The UR state is now Forgotten.

This prepare failure occurs because the CICS region A6POR1CI has been cancelled. Note that CICS region A6POTGR1 will backout the ECIT mirror transaction updates during CICS emergency restart.

� The log entry for the second UR shows a backout call (AtraBak) to CITGT1G returns with code 0. This backout call is made because the transaction manager, WebSphere Application Server, has rolled back the global transaction following the failure of the prepare call to CITGR1G.

CICS message logInspection of the CICS message log for A6POTGT1 shows the backout of the TS Queue updates on A6POTGT1 (Example 5-16).

Example 5-16 CICS log message from A6POTGT1 indicating transaction backout

DFHAC2250 10/11/2005 14:58:30 A6POTGT1 The coordinator system has indicated that the current unit of work is to be backed out. Transaction ECIT running program DFHMIRS term ???? has been abnormally terminated with abend ASP3.

After the restart of CICS region A6POTGR1 was complete, the CICS log showed message DFHRM0201 (Example 5-17).


DFHRM0201 10/11/2005 14:57:17 A6POTGR1 0 backout-failed and 1 commit-failed UOWs were reconstructed

Finally, we used CEBR to check that the TS Queue updates were backed out on both CICS regions.


5.8 OperationsThe enablement of XA transaction support has an impact on operational procedures for the Gateway daemon.

5.8.1 Gateway daemon shutdown and restartBefore enabling XA transaction support, you should review your Gateway daemon shutdown procedures.

Gateway daemon normal shutdownWhen the Gateway daemon is requested to perform a normal shutdown:

� All in-flight extended LUWs and XA transactions are allowed to complete.

� No new transaction requests are accepted.

� No new XA recover requests are accepted.

XA recovery is required if an error occurs in the syncpoint process, caused by a transaction manager failure. In this case, it is necessary for the transaction manager (in our case WebSphere Application Server) to reconnect to the Gateway daemon at a later time to tell the daemon whether an XA transaction should be committed or backed out.

A CTG6591I message is issued (Example 5-18) for attempts to shutdown a Gateway daemon which is currently processing in-flight extended LUWs or XA transactions.

Example 5-18 Gateway daemon shutdown with outstanding XA transactions

10/12/05 13:26:42:339 [0] CTG6490I Normal shutdown of Gateway daemon started by z/OS Operator 10/12/05 13:26:42:371 [0] CTG6491I Number of active clients is 1

Gateway daemon immediate shutdownAn immediate shutdown is not recommended because it terminates all work in progress and breaks any active connections. When the Gateway daemon is requested to perform an immediate shutdown:

� Active requests are terminated abnormally.

� In-flight extended LUWs are rolled back.

� In-flight XA transactions are rolled back.

� XA transactions that are in post-prepare state are not rolled back. It is not possible for the transaction manager (WebSphere Application Server) to complete the XA transaction until the Gateway daemon is restarted.


5.8.2 Gateway daemon restartDuring restart of an XA enabled Gateway daemon, the Gateway daemon rebuilds information that is related to outstanding URs by sending a request to RRS for information related to its work manager name (CTG_RRMNAME).

5.8.3 The ctgasi utilityThe CICS TG Address Space Initiation (ctgasi) utility is provided in case you need to start, stop, or refresh the CTGRRMS address space manually. You should only be required to start, stop, or refresh the address space manually if there is a need to apply maintenance (PTFs) that affect the load module hlq.SCTGLINK(CTGINIT).

The post-maintenance apply procedure is :

� Refresh the LLA (/F LLA,REFRESH)� Shutdown all Gateway daemons� Shutdown all ctgmaster address spaces� Issue the command ctgasi -refresh from OMVS/USS shell

The operator user ID issuing the ctgasi -refresh command must have CONTROL access to the CICSTG.RRMS.SERVICE RACF FACILITY class profile. If this ID does not have this access, ctgasi returns error message CTG6209E (Example 5-19).

Example 5-19 ctgasi error for insufficient authority

CTG6209E ctgasi - user ID not authorized, SAF RC = 00000008 00000008 00000000

CTGRRMS keeps track of registered/de-registered address spaces (Gateway daemons or ctgmaster) and ctgasi refuses to shutdown or refresh CTGRRMS if it believes that there are active address spaces. However, if address spaces have terminated abnormally, then they might not have completed the de-registration step with CTGRRMS. In such a case, the -force option can be used to refresh or shutdown CTGRRMS even if active instances of the Gateway daemon are detected.

Important: You should not restart an XA-enabled Gateway daemon on a different LPAR if there is outstanding XA recovery to perform.


5.9 Problem determinationIn this section, we document common transaction problems and information we learned while configuring our transaction test scenarios. For general problem determination guidance see 2.9, “Problem determination” on page 60.

5.9.1 Common transaction problemsIf a Gateway daemon is started, but another Gateway daemon is running with the same CTG_RRMNAME, the Gateway daemon fails to initialize. We noted the following message

CTG9202E A RRS application is active using the same resource manager name('CTG.RRM.CITGS1G.IBM.UA’)

If an ECI request within an XA transaction is sent to a Gateway daemon which does not have xasupport=on specified as an initialization parameter, the request fails and the global transaction rolls back. We noted the following message in the WebSphere Application Server stderr log:

javax.transaction.SystemException: XAResource start association error:XAER_RMERR

5.9.2 Tips and utilitiesIn this section, we discuss additional tips and utilities for problem determination of transaction related problems. For general tips and information about standard utilities, such as the various Gateway daemon traces, see 2.9.2, “Tips and utilities” on page 62.

Identifying the transactionAs different parts of a transaction are processed by different tiers of the infrastructure, the transaction is identified in different ways. Table 5-6 and the definitions that follow show the different transaction identifiers that are used by the different software components.

Table 5-6 Transaction references

Software Component Transaction Identifier

WebSphere Application Server XID

CICS Transaction Gateway XID (for XA transactionsLuw_token (for extended LUWs)

RRS XID UR identifier (URID)


XID An X/Open identifier (XID) acts as the identifier for a distributed unit of work. An XID consists of a Global transaction identifier (GTRID) and a Branch qualifier (BQUAL).

Luw_token A token used by the CICS TG to tie together multiple DPL calls into an extended LUW.

UR identifier The URID uniquely identifies the RRS unit of recovery (UR). Multiple URIDs can be involved in a single XID.

UOW The CICS unit of work identifier. For transactional EXCI calls, the CICS UW references the RRS URID.

Netuowid The CICS network UOW identifier which is identical for all CICS UOWs which are connected within a single CICS distributed unit of work.

CICS UOW commandsWhen investigating transaction problems, in addition to using the RRS ISPF panels as shown in our test scenarios, you might also find it useful to use the CICS CEMT INQUIRE command to inquire on CICS units of work:

� CEMT INQUIRE UOW

This command returns information about a named unit of work, or about all the UOWs currently in the system. It displays the state of the UOW (for example, INDOUBT) and whether it is active, waiting, or shunted.

� CEMT INQUIRE UOWENQ

Retrieves information about enqueues held or waited on by a UOW, or about UOWs holding or waiting on a specified enqueue.

� CEMT INQUIRE UOWLINK

Retrieves information about connections involved in units of work. A connection can be to a remote system, or to a task-related user exit.

CICS TS UOWNetuowid

Note: A shunted UOW is one awaiting resolution of an in-doubt failure, a commit failure, or a backout failure.

Software Component Transaction Identifier


For example, Figure 5-31 shows the results of a CEMT INQUIRE UOWLINK command which shows a CICS UOW with two links: a link to RRS and a link to Gateway daemon CITGT1G.

Figure 5-31 CEMT INQUIRE TASK on A6POTGR1 with the mirror in RRSEXIT state

See 6.5.2, “Test scenarios” on page 237 for an example scenario in which these commands are used to analyze an in-doubt failure.

I UOWLINK STATUS: RESULTS - OVERTYPE TO MODIFY Uowl(01000017) Uow(BDBEBFBF1BCA0D66) Con Lin(RRS ) Unk Rrms Ok Net(..PSSCX9.A6POTGT1....C_....) Uowl(01010047) Uow(BDBEBFBF1BCA0D66) Con Lin(CITGT1G ) Unk Irc Sys(GT1G) Net(..PSSCX9.A6POTGT1....C_....)

SYSID=TGT1 APPLID=A6POTGT1 RESPONSE: NORMAL TIME: 16.56.04 DATE: 10.11.05 PF 1 HELP 3 END 5 VAR 7 SBH 8 SFH 9 MSG 10 SB 11 SF


Chapter 6. Gateway daemon transactions with TCP/IP load balancing

In this chapter, we outline the additional configuration steps that are required to support XA transactions when operating a Gateway daemon within a TCP/IP load balanced configuration. We introduce the concept of a CICS TG group that consists of a CICS TG master process and a set of cloned Gateway daemon instances, all listening on the same TCP/IP port. The role of the CICS TG master is to coordinate transactions for the CICS TG group. We also document how we configured the CICS TG group to support transactions and explain how we tested the environment.

6


6.1 PreparationFor this chapter, we used WebSphere Application Server for Windows and a cloned group of Gateway daemons running on a single z/OS LPAR. For information about transactional support with a stand-alone Gateway daemon, see Chapter 5, “Gateway daemon transactions” on page 157.

Figure 6-1 Software components: Transaction scenario with CICS TG group

Figure 6-1 shows a global transaction spanning updates to multiple resource managers, a CICS region, and a DB2 database on Windows. The global transaction is coordinated by the transaction manager, WebSphere Application Server.

The CICS region is accessed via one of the Gateway daemon instances which are part of a CICS TG group. RRS is used to coordinate the resource updates on z/OS. Access to RRS, for each of the Gateway daemon instances, is controlled by the CICS TG master process.

Note: In our global transaction configuration we document the setup steps for transactions which update recoverable resources in two CICS regions, but we do not document setup or testing of DB2.

CICS A

z/OS

TCP

CICS ECI XA

resource adapter

JCA


servlet

EJB

Windows

DB2

JDBC

Datasource

Global transaction

TransactionManager

ResourceManager

RRS

ResourceManager

ServletJSP

Port

EXCI

Gateway daemon

CICS TG Master


6.2 Software checklistTable 6-1 shows the levels of software that we used for our configuration.


6.3 Transaction affinity considerationsThe Gateway daemon can be used with different TCP/IP load balancing technologies, such as TCP/IP port sharing and Sysplex Distributor. An example high availability configuration is shown in Figure 3-14 on page 105.

In this section, we consider what restrictions apply when using TCP/IP load balancing with either extended LUWs or XA transactions.

6.3.1 Extended LUWs The first request of an Extended LUW sequence returns an LUW token to the ECI resource adapter. The LUW token is generated by the Gateway daemon which processes the request. The LUW token is Gateway daemon specific, such that any subsequent request from the resource adapter including the LUW token must be serviced by the same Gateway daemon.

This creates an affinity between the resource adapter and a specific Gateway daemon instance for the duration of the extended LUW. Figure 6-2 shows the flows that occur between the CICS ECI resource adapter and a Gateway daemon during the processing of an extended LUW. The complete sequence of flows, including the ECI requests and the commit flow must be processed by the same Gateway daemon.

Windows 2000 z/OS

Internet Explorer V6.0 z/OS 1.6





Chapter 6. Gateway daemon transactions with TCP/IP load balancing 227

Figure 6-2 Extended LUW transaction affinity

If the connection from the WebSphere Application Server to the Gateway daemon is lost, for example, the Gateway daemon suffers a failure, then the next request (another ECI request or a commit flow) fails. The transaction manager by default rolls back the transaction. Unlike for a two-phase commit resource, there is no recovery from a communication failure with a one-phase commit resource.

When using extended LUWs within a TCP/IP load balancing configuration, the complete set of flows for each extended LUW must be managed by a single Gateway daemon instance.

6.3.2 XA transactions The first request of an XA transaction sequence for a specific connection factory includes an Xid (X/Open identifier) that is generated by the WebSphere Application Server. Subsequent ECI requests include this Xid.

There is an affinity between the resource adapter and a specific Gateway daemon instance for all ECI requests during the post-prepare phase of an XA transaction. If a subsequent ECI request for an in-flight XA transaction is received by a different Gateway daemon (for example, after a failure of the Gateway daemon instance that processed the previous ECI requests), then the request will fail with a CICS abend AITD. However, the two-phase commit syncpoint flows (prepare and commit) can be serviced by another Gateway daemon instance in the same CICS TG group.

GatewayDaemonInstance

CICS ECI resource Adapter


Server

EJB

TransactionManager

ECI request 1

ECI response 1 (LUW-token)

ECI request 2 (LUW-token)

ECI response 2 (LUW-token)

Commit (LUW-token)

Commit response

z/OSWindows

ConnFactory


Figure 6-3 shows the flows that occur between the CICS ECI XA resource adapter and a Gateway daemon during the processing of an XA transaction. The CICS TG master process maintains XA transaction state information, including a table of all Xids in which the CICS TG group has an interest. The complete sequence of ECI requests must be serviced by a specific Gateway daemon. The prepare and commit flows can be serviced by other Gateway daemon instances within the same CICS TG group.

Figure 6-3 XA transaction affinity

Because the CICS ECI XA resource adapter supports two-phase commit processing, failures in the syncpoint process can produce in-doubt situations. This occurs if the Gateway daemon has responded positively to the prepare flow from the WebSphere Application Server but because of an application server failure, the Gateway daemon does not receive a commit flow. In this case, the WebSphere Application Server sends a recover flow when it is restarted to tell the Gateway daemon whether it should commit or backout. Transaction recovery flows can be serviced by any Gateway daemon instance in the same CICS TG group.

GatewayDaemonInstance

CICS ECIXA

resource Adapter


Server

EJB

TransactionManager

ECI request 1 (Xid)

ECI response 1

ECI request 2 (Xid)

ECI response 2

Prepare (Xid)

Prepare response

z/OSWindows

ConnFactory

CIC

S TG m

aster

XidTable

Commit (Xid)


6.3.3 XA-enabled CICS TG groupAn XA-enabled CICS TG group consists of:

� A CICS TG master process (ctgmaster address space)

� Two or more XA-enabled cloned Gateway daemons running on the same MVS image

In this configuration, when processing XA transaction requests, the ctgmaster accesses RRS on behalf of each of the Gateway daemons in the group.

As with a stand-alone XA-enabled Gateway daemon, the RRS registration by ctgmaster must be authorized (see “Enabling CTGRRMS services” on page 195). Therefore, during initialization, ctgmaster ensures that the CTGRRMS address space is started and available for use. If CTGRRMS is not already active on the LPAR when ctgmaster is started, then ctgmaster attempts to start it.

6.4 Configuring an XA-enabled CICS TG group In 3.7.2, “Cloned Gateway daemon connectivity tests” on page 114, we showed how to create cloned Gateway daemons in a TCP/IP port sharing configuration. In this section, we describe the additional configuration changes that we made in order to XA-enable the cloned gateway daemons.

We needed to consider the following CICS TG configuration steps:

� Configuring and starting the CICS TG master process� Configuring each Gateway daemon instance

6.4.1 CICS TG master process configurationThe CICS TG master process is started by executing the USS executable ctgmaster, which can be started from a USS shell or in batch mode (via CTGBATCH). Running ctgmaster within a USS shell session is only suitable for usage within a test environment. Batch mode is the recommended runtime environment for a production system.

Restriction: The Gateway daemon instances within an XA-enabled CICS TG group must be started on the same MVS image. This means that you cannot distribute ECI requests across Gateway daemons on different LPARs if the ECI request is part of an XA transaction.


The CICS TG master process is configured using environment variables or startup switches. Switches allow the environment variables to be overridden. However, they are more suited to usage within the USS shell because the PARM string on the JCL EXEC command is limited to 100 bytes.

There is no equivalent to the Gateway daemon environment variable CTGSTART_OPTS for the CICS TG master process.

Table 6-2 shows the mapping between CICS TG master process environment variables and command line switches and overrides.

Table 6-2 ctgmaster configuration environment variables and override switches

We now consider the key configuration settings for the CICS TG master process in more detail.

CTG_MASTER_RRMNAMEThe environment variable CTG_MASTER_RRMNAME is used to specify the name that ctgmaster uses when registering with RRS. It must match the name specified for CTG_MASTER_RRMNAME for each of the Gateway daemon instances within the CICS TG group. The name must be unique within the Sysplex.

We set CTG_MASTER_RRMNAME to CTG.RRM.CITGR0G for our CICS TG group. The default value is CICSTG.DEFAULT.MASTERRRM.

Note: You can also use the z/OS supplied BPXBATCH utility to run ctgmaster in batch mode, but we recommend the CICS TG supplied utility CTGBATCH, which provides the JES logging capability.

Environment variable Override switch Value options

CTG_MASTER_RRMNAME -rrmname= <CICS TG group RRM name>

CTG_MASTER_INPUT -input= STDIN|CONSOLE

CTG_MASTER_TRACE_ON -trace_on YES|NO

CTG_MASTER_TRACE -trace= <HFS filename>

CTG_MASTER_NATLANG -natlang= EN|JA|ZH

CTG_XA_MAX_TRAN -xa_max_tran= 1-8192


CTG_MASTER_INPUTDuring initialization, the CICS TG master process starts listening for commands entered either at the z/OS console, or from stdin.

The default behavior is to listen on stdin. Because we started ctgmaster from CTGBATCH, we set CTG_MASTER_INPUT to CONSOLE.

See 6.6, “Operations” on page 248 for a list of commands that can be used to operate ctgmaster.

CTG_XA_MAX_TRANThe Gateway daemon environment variable CTG_XA_MAX_TRAN is described in “Enabling XA support in the Gateway daemon” on page 196. When running within a CICS TG group, the Gateway daemon environment variable CTG_XA_MAX_TRAN is ignored, because it is the CICS TG master process which maintains XA transaction state for the group of Gateway daemons.

We set the CTG_XA_MAX_TRAN for the CICS TG master process to the default value 8192. This allows for a collective total of 8192 concurrent XA transactions, spread across all Gateway daemons in the CICS TG group.

Starting the CICS TG master process Example 6-1 shows how we customized the sample JCL in hlq.SCTGSAMP(CTGMASTR) to create the start procedure for our CICS TG master process CITGR0G.

Example 6-1 CICS TG master process PROCLIB member CITGR0G

//CITGR0G PROC // SET CTGHLQ='CTGV61' // SET CTGUSR='CITG.CTG610.STDENV(CITGR0G)' // SET LEOPTS='/' //MASTER EXEC PGM=CTGBATCH, // PARM='&LEOPTS./usr/lpp/cicstg/ctg610/bin/ctgmaster'//STEPLIB DD DSN=&CTGHLQ..SCTGLOAD,DISP=SHR //STDOUT DD SYSOUT=* //STDERR DD SYSOUT=* //STDENV DD DSN=&CTGUSR.,DISP=SHR //*

Note: Starting ctgmaster in batch mode without defining the input type to CONSOLE causes initialization to fail. Message CTG8997E reports that the stdin console is not available.


Example 6-2 shows how we customized the sample STDENV file hlq.SCTGSAMP(CTGMENV) to create a STDENV member for our CICS TG master process.

Example 6-2 CICS TG master process STDENV member CITGR0G

_BPX_SHAREAS=YES CTG_MASTER_RRMNAME=CTG.RRM.CITGR0G CTG_XA_MAX_TRAN=8192 TMPDIR=/cicstg/ctg610/config TZ=GMT-2 CTG_MASTER_INPUT=CONSOLE CTG_MASTER_TRACE=/cicstg/ctg610/trace/CITGR0G.trcCTG_MASTER_TRACE_ON=YES

6.4.2 Gateway daemon configuration In 5.6.3, “Configuring CICS TG” on page 175, we showed how we enabled XA transaction support for the Gateway daemon CITGR1G by:

� Enabling CTGRRMS services� Specifying a unique RRM name� Specifying the initialization parameter xasupport=on

We followed the same steps to enable XA transaction support for the Gateway daemon CITGR2G (the second Gateway daemon in our CICS TG R group).

CTG_MASTER_RRMNAMEThe environment variable CTG_MASTER_RRMNAME is used to specify the name that ctgmaster uses when registering with RRS. CTG_MASTER_RRMNAME must be the same for ctgmaster and for each of the Gateway daemon instances within the CICS TG group.

During Gateway daemon initialization, if the environment variable CTG_MASTER_RRMNAME is defined, then this signifies that the Gateway daemon is part of a group. Gateway daemon initialization fails if a CICS TG master process with matching CTG_MASTER_RRMNAME is not active.


We added entries for CTG_MASTER_RRMNAME to the STDENV files for the CITGR1G and CITGR2G Gateway daemons. Example 6-3 shows the environment variables for Gateway daemon CITGR1G.


_BPX_SHAREAS=YES PATH=/bin:/usr/lpp/java142/J1.4/bin CTG_MASTER_RRMNAME=CTG.RRM.CITGR0G CTG_RRMNAME=CTG.RRM.CITGR1G.IBM.UA #CTG_XA_MAX_TRAN=1000 DFHJVPIPE=CITGR1G CICSCLI=/cicstg/ctg610/config/CITGR1G.ini .....

Example 6-4 shows the environment variables for Gateway daemon CITGR2G.


_BPX_SHAREAS=YES PATH=/bin:/usr/lpp/java142/J1.4/bin CTG_MASTER_RRMNAME=CTG.RRM.CITGR0G CTG_RRMNAME=CTG.RRM.CITGR2G.IBM.UA #CTG_XA_MAX_TRAN=1000 DFHJVPIPE=CITGR2G CICSCLI=/cicstg/ctg610/config/CITGR1G.ini ....

Because a Gateway daemon which is part of a CICS TG group can process extended LUWs in addition to XA transaction requests, the environment variable CTG_RRMNAME is used for RRS registration of the Gateway daemon instance.

6.5 Testing the CICS TG group configurationWe started the CICS TG group by starting the ctgmaster and Gateway daemons in the following order:

1. ctgmaster 2. Gateway daemon CITGR1G3. Gateway daemon CITGR2G

We show how we tested the CICS TG group configuration using our sample J2EE application CTGTesterCCIXA. Figure 6-4 shows the basic outline of our environment.


Figure 6-4 CICS TG group scenario

The CTGTesterCCIXA J2EE application contains two ECI resource references and can be used to call programs running in two CICS regions via different connection factories. The two requests are part of the same global transaction. For further details about CTGTesterCCIXA, refer to Appendix A, “Sample applications” on page 361.

In this test scenario, the first connection factory is defined with port 13101 and IP name tx1.mop.ibm.com. Because the Gateway daemons CITGR1G and CITGR2G are both configured to listen on port 13101, socket connections for this connection factory will be balanced across the two Gateway daemons.

CITGR1G and CITGR2G are part of the same CICS TG group , therefore, the CICS TG group master process accesses RRS on behalf of the Gateway daemons.

The second connection factory is defined with port 12101 and IP address tx1b.mop.ibm.com, so that socket connections for this connection factory are processed by the stand-alone Gateway daemon CITGT1G.

z/OStx1.mop.ibm.com


JSP/Servlet

CTGTesterCCIXA

CICS ECI XA

Resource Adapter

Windows

CICS TG V6.1

Cam21-pc11

TransactionRequired


RRS

CITGR1G/CITGR2G


CITGR1CI (applid A6POTGR1)

CICS

EXCIJNI

13101

Gatewaydaemon

CTGRRMS

CITGT1G


CICS

EXCIJNI

12101

Gatewaydaemon

tx1b.mop.ibm.com



ctgmasterCCI


We used this configuration for two test scenarios:

� Normal termination of an XA transaction

An update and commit of recoverable resources in two CICS regions within one XA transaction. The first leg of the XA transaction is processed by the CICS TG group. The second leg of the transaction is processed by CITGT1G, a stand-alone XA-enabled Gateway daemon.

� In-doubt failure of an XA transaction

A WebSphere Application Server failure during the syncpoint processing which resulted in an in-doubt situation. We show how the in-doubt situation is resolved after the application server is restarted, and that the recover flow can be processed by either member of the CICS TG group.

6.5.1 Definitions checklistTable 6-3 and Table 6-4 list the definitions that we used to configure the scenario.

Table 6-3 Definitions checklist for CICS TG group R

Value CICS TG master process

Port sharing Gateway daemons

CICS TS

hostname - tx1.mop.ibm.com -

IP address - 9.100.193.122 -

TCP/IP port - 13101 -

Job name CITGR0G CITGR1G CITGR2G

CITGR1CI

APPLID - - A6POTGR1

NETNAME (DFHJVPIPE) - CITGR1GCITGR2G

-

CONNECTION - GR1G

CTG_MASTER_RRMNAME CTG.RRM.CITGR0G CTG.RRM.CITGR0G

CTG_RRMNAME CTG.RRM.CITGR1G.IBM.UA CTG.RRM.CITGR2G.IBM.UA


Table 6-4 Definitions checklist for CICS TG T1

6.5.2 Test scenariosWe now show the results of our two test scenarios:

� Normal termination of an XA transaction� In-doubt failure of an XA transaction

Normal termination of XA transaction We started two Web browser sessions and ran the CTGTesterCCIXA application on each Web browser. We specified WW as the input to the COMMAREA which means that the ECIPROG program writes to a recoverable TS Queue.

Each Web browser session, therefore, initiates an XA transaction consisting of two ECI calls: one request to CICS region A6POTGR1 via the CICS TG group R and one request to CICS region A6POTGT1 via stand-alone Gateway daemon CITGT1G.

We wanted to show in this basic test that a different Gateway daemon in the CICS TG group participates in each of the XA transactions.

We checked the Gateway daemon logs, which showed that each Gateway daemon received a connection request (Example 6-5).


hostname tx1b.mop.ibm.com -

IP address 9.100.193.121 -

TCP/IP port 12101 -

Job name CITGT1G CITGT1CI

APPLID A6POTGT1

NETNAME (DFHJVPIPE) CITGT1G -

CONNECTION GT1G

Note: On our system the CITGT1G Gateway daemon binds to a different TCP/IP address than the Gateway daemons within the CICS TG group, but all the Gateway daemons are actually running on the same z/OS image. In practice the CITGT1G Gateway daemon could be running on a completely different system. The CICS TG group Gateway daemon instances, however, must run on the same MVS image.


Example 6-5 Load-balancing concurrent connections to tx1.mop.ibm.com:13101

From CITGR1G stdout log:10/21/05 16:36:17:303 [0] CTG6506I Client connected. [ConnectionManager-9] - tcp@Socket[addr=9.100.199.134,port=1412,localport=13101]

From CITGR2G stdout log:10/21/05 16:36:17:081 [0] CTG6506I Client connected. [ConnectionManager-9] - tcp@Socket[addr=9.100.199.134,port=1411,localport=13101]

We then checked the state of the TS Queues on A6POTGR1 and A6POTGT1 to confirm that each request resulted in an update to the TS Queue RECIPROG.

In-doubt failure of an XA transactionIn this scenario, we simulated a WebSphere Application Server failure during the syncpoint process of the XA transaction. This resulted in both the UR in RRS, and the UOW in CICS being in an in-doubt state.

While the XA transaction was in-doubt, we used the following tools to analyze the state of the transaction:

� ctgmaster QUERY XIDS command to query the ctgmaster process

� CICS CEBR transaction to query the state of the TS queues

� CICS CEMT INQUIRE TASK, CEMT INQUIRE UOW and CEMT INQUIRE UOWLINK commands to query the state of CICS tasks and UOWs

� RRS ISPF panels to query the state of RRS URs

Note: The addr=9.100.199.134,port=1412 and addr=9.100.199.134,port=1411 values represent the protocol characteristics of specific connections in the WebSphere Application Server connection pool for connection factory CITGR1CI-tx1-13101-XA. See “Creating connection factories” on page 200 for information about how this connection factory was defined.

Note: This scenario is for illustrative purposes only. In-doubt failures are very rare in practice.


ctgmaster QUERY XIDS commandThe QUERY XIDS command is used to retrieve the list of Xids in which the CICS TG group has an interest. We executed the following MVS command:

/F CITGR0G,APPL=QUERY,XIDS

Example 6-6 shows the output for the QUERY XIDS command.

Example 6-6 ctgmaster QUERY XIDS output

CTG8998I ctgmaster command accepted. CTG8950I ctgmaster query xids result - entry 000001UR Interest : 7E2D80800000009D0055000155555555 XID.formatID : 1463898948 XID.gtrid : 00-0F 00000107 03167364 00000001 00000001 10-1F 652EE808 ECBAEA9A B3C52EA4 110E2BA2 20-23 FCD740E7 XID.bqual : 00-0F 00000107 03167364 00000001 00000001 10-1F 652EE808 ECBAEA9A B3C52EA4 110E2BA2 20-2F FCD740E7 00000001 00000000 00000000 30-35 00000000 0001 CTG8951I ctgmaster query xids found 1 entries.

This example shows that the CICS TG group has an interest in RRS UR 7E2D80800000009D0055000155555555 which has a GTRID of:

00000107031673640000000100000001652EE808ECBAEA9AB3C52EA4110E2BA2FCD740E7

CEBR We used the CICS-supplied terminal transaction CEBR to query the state of the recoverable TS Queue RECIPROG on each CICS region. Example 6-7 shows the contents of the RECIPROG TS Queue on both CICS regions.

Example 6-7 CEBR output from A6POTGR1 and A6POTGT1 during in-doubt window


CEBR TSQ RECIPROG SYSID TGT1 REC 1 OF 2 COL 1 OF 38 ENTER COMMAND ===> ************************** TOP OF QUEUE ******************************* 00001 A6POTGT1 19/10/05 12:31:26 CITGT1D D ************************* BOTTOM OF QUEUE *****************************


This shows that the mirror tasks have updated the TS Queue RECIPROG on both CICS regions.

CICS tasks and UOWsWe used the CEMT INQUIRE TASK to display the mirror task ECIT on each CICS region. Figure 6-5 shows that the mirror task on CICS region A6POTGT1 has ended.

Figure 6-5 CEMT INQUIRE TASK for CICS region A6POTGT1

Figure 6-6 shows that the mirror task on CICS region A6POTGR1 (Task 150) is suspended in RRMSEXIT state, which means that CICS is waiting for a commit or backout flow from RRS.

Figure 6-6 CEMT INQUIRE TASK for CICS region A6POTGR1

I TASK STATUS: RESULTS - OVERTYPE TO MODIFY Tas(0000485) Tra(CEBR) Fac(N124) Sus Ter Pri( 001 ) Sta(TO) Use(CITGT1D ) Uow(BDC72F3041FDE708) Hty(ZCIOWAIT) Tas(0000582) Tra(CEMT) Fac(N107) Run Ter Pri( 255 ) Sta(TO) Use(CITGT1D ) Uow(BDC8945686506548) SYSID=TGT1 APPLID=A6POTGT1 RESPONSE: NORMAL TIME: 12.34.37 DATE: 10.19.05 PF 1 HELP 3 END 5 VAR 7 SBH 8 SFH 9 MSG 10 SB 11 SF

I TASK STATUS: RESULTS - OVERTYPE TO MODIFY Tas(0000150) Tra(ECIT) Sus Tas Pri( 001 ) Sta(TO) Use(CITGR1D ) Uow(BDC8939F7B6C790F) Hty(RRMSEXIT) Tas(0000151) Tra(CEMT) Fac(N105) Run Ter Pri( 255 ) Sta(TO) Use(CITGR1D ) Uow(BDC8943C20924C20) SYSID=TGR1 APPLID=A6POTGR1 RESPONSE: NORMAL TIME: 12.34.12 DATE: 10.19.05 PF 1 HELP 3 END 5 VAR 7 SBH 8 SFH 9 MSG 10 SB 11 SF


We then looked at the state of the CICS UOW BDC8939F7B6C790F on CICS region A6POTGR1 using the CEMT INQUIRE UOW command (Figure 6-7).

Figure 6-7 CEMT INQUIRE UOW for CICS region A6POTGR1

Example 6-8 shows the expanded view of the UOW.

Example 6-8 CEMT INQUIRE UOW INDOUBT - detailed view

INQUIRE UOW(BDC8*) RESULT - OVERTYPE TO MODIFY Uow(BDC8939F7B6C790F) Uowstate( Indoubt ) Waitstate(Active) Transid(ECIT) Taskid(0000150) Age(00000260) Termid(21) Netname(CITGR2G) Userid(CITGR1D) Waitcause() Link() Sysid() Netuowid(..PSSCX9.A6POTGR1Hl.#......)

The expanded view of the UOW reveals that the UOW is in-doubt and that the Netname that is associated with the UOW is CITGR2G (the jobname of the Gateway daemon that initiated the CICS UOW).

Important: When a UOW is in-doubt, CICS retains locks on the recoverable resources that the UOW has updated. This prevents further tasks from updating the resource. The locks are maintained until the in-doubt situation is resolved.

INQUIRE UOW(BDC8*) STATUS: RESULTS - OVERTYPE TO MODIFY Uow(BDC8939F7B6C790F) Ind Act Tra(ECIT) Tas(0000150) Age(00000246) Ter(21 ) Netn(CITGR2G ) Use(CITGR1D ) SYSID=TGR1 APPLID=A6POTGR1 RESPONSE: NORMAL TIME: 12.35.35 DATE: 10.19.05 PF 1 HELP 3 END 5 VAR 7 SBH 8 SFH 9 MSG 10 SB 11 SF


From our analysis so far, we can establish that the XA transaction has failed in-doubt. The first leg of the XA transaction (processed by the CICS TG group) has resulted in a suspended task and an in-doubt UOW in CICS region A6POTGR1. Whereas the second leg of the transaction (processed by the stand-alone Gateway daemon) has completed normally.

We then used the CEMT INQUIRE UOWLINK command on CICS region A6POTGR1 to investigate further, in particular, to determine whether any other systems have links to this in-doubt UOW (Figure 6-8).

Figure 6-8 CEMT INQUIRE UOWLINK

Example 6-9 shows the expanded view of the UOWLINK.

Example 6-9 CEMT INQUIRE UOWLINK - detailed view

INQUIRE UOWLINK RESULT - OVERTYPE TO MODIFY Uowlink(010400BC) Uow(BDC8939F7B6C790F) Type(Connection) Link(RRS) Action( ) Role(Unknown) Protocol(Rrms) Resyncstatus(Ok) Sysid() Rmiqfy() Netuowid(..PSSCX9.A6POTGR1Hl.#......) Urid(BDC8939F7E517A5C000000BA01030000) Host()

The CEMT INQUIRE UOWLINK command retrieves information about connections involved in units of work. A connection can be to a remote system, or to a task-related user exit. We can see that our in-doubt CICS UOW has a link to RRS and that the RRS URID is BDC8939F7E517A5C000000BA01030000.

We use this information to query the status of the UR in the RRS ISPF panels.

INQUIRE UOWLINK STATUS: RESULTS - OVERTYPE TO MODIFY Uowl(010400BC) Uow(BDC8939F7B6C790F) Con Lin(RRS ) Unk Rrms Ok Net(..PSSCX9.A6POTGR1Hl.#......) SYSID=TGR1 APPLID=A6POTGR1 RESPONSE: NORMAL TIME: 12.36.05 DATE: 10.19.05 PF 1 HELP 3 END 5 VAR 7 SBH 8 SFH 9 MSG 10 SB 11 SF


RRS ISPF panelsWe explain how the RRS ISPF panels can be used to retrieve information about RRS URs in “RRS ISPF panels” on page 185.

The UR details for URID BDC8939F7E517A5C000000BA01030000 are shown in Figure 6-9. We see that:

� The UR state is in-doubt

� The RRS work manager for the UR is the CICS TG master process with CTG_MASTER_RRMNAME CTG.RRM.CITGR0G

� The section of the panel showing Expressions of Interest confirms that the CICS TG master process has a syncpoint coordinator or SDSRM (Server Distributed Syncpoint Resource Manager) role in the UR

� The CICS region A6POTGR1 has a Participant role in the UR

Figure 6-9 RRS UR details for in-doubt UR

RRS Unit of Recovery Details Row 1 to 2 of 2 Commands r-Remove Interest v-View URI Details UR identifier : BDC8939F7E517A5C000000BA01030000 Create time : 2005/10/19 10:31:24.853151 Comments : X UR state : InDoubt UR type : Prot System : X1 Logging Group : X1 SURID : N/A Work Manager Name : CTG.RRM.CITGR0G / Display Work IDs / Display IDs formatted Luwid . : Present Eid . . : Not Present Xid . . : Present Expressions of Interest: S RM Name Type Role CTG.RRM.CITGR0G Prot SDSRM DFHRXDM.A6POTGR1.IBM Prot Participant ******************************* Bottom of data ******************************** Command ===> Scroll ===> PAGE F1=Help F2=Split F3=Exit F7=Up F8=Down F9=Swap F12=Cancel


Finally, we show the RRS Unit of Recovery Work IDs panel for our in-doubt UR (Figure 6-10).

Figure 6-10 RRS Unit of Recovery Work IDs for in-doubt UR

You can check that the GTRID for the UR matches that displayed by the ctgmaster QUERY XIDS command in Example 6-6 on page 239.

Resolving the in-doubt transactionFrom our analysis, we have seen that the suspended ECIT task and in-doubt UOW in CICS region A6POTGR1 are awaiting resolution. This resolution occurs when the WebSphere Application Server is restarted.

RRS Unit of Recovery Work IDs UR identifier : BDC8939F7E517A5C000000BA01030000 Logical Unit of Work Identifier (LUWID): PSSCX9.A6POTGR1 C8939F7B6448 0001 NetID.LuName : PSSCX9.A6POTGR1 TP Instance : C8939F7B6448 SeqNum . . . : 0001 Enterprise Identifier (EID) TID : (decimal) GTID : X/Open Transactions Identifier (XID) Format ID : 001463898948 (decimal) 57415344 (hexadecimal) GTRID : 00-0F 00000107 03167364 00000001 00000001 |................| 10-1F 652EE808 ECBAEA9A B3C52EA4 110E2BA2 |..Y......E.u...s| 20-23 FCD740E7 |.P X | BQUAL : 00-0F 00000107 03167364 00000001 00000001 |................| 10-1F 652EE808 ECBAEA9A B3C52EA4 110E2BA2 |..Y......E.u...s| 20-2F FCD740E7 00000001 00000000 00000000 |.P X............| 30-35 00000000 0001 |...... | F1=Help F2=Split F3=Exit F5=ListErr F7=Up F8=Down F9=Swap F12=Cancel


We have also established that the ECIT task was initiated by an ECI request processed by Gateway daemon CITGR2G. In order to test the recovery capabilities of the CICS TG group, we shut down the Gateway daemon CITGR2G before restarting the WebSphere Application Server.

After restarting the WebSphere Application Server, we used the following tools to analyze the state of the XA transaction:

� The ctgmaster QUERY XIDS command

� CICS CEBR transaction to query the state of the TS queues

� CICS CEMT INQUIRE UOW INDOUBT command to verify that the in-doubt UOW in CICS is resolved

� RRS archive log to view the completed global transaction from an RRS perspective

ctgmaster QUERY XIDS commandExample 6-10 shows the output of the QUERY XIDS command. We can see that the CICS TG group has no interest in any Xid.

Example 6-10 ctgmaster QUERY XIDS output after in-doubt resolution

CTG8998I ctgmaster command accepted. CTG8951I ctgmaster query xids found 0 entries.

Note: After using RRS to list in-doubt URs, it is possible to commit or backout an in-doubt UR manually. Before doing this you should attempt to resolve the in-doubt situation by restarting the WebSphere Application Server and all resource managers which have an interest in the global transaction.

In order to force an in-doubt UR to commit or to backout manually, the operator user ID must have ALTER access to the FACILITY class MVSADMIN.RRS.COMMANDS.


CEBR Example 6-11 shows the contents of the RECIPROG TS Queue on both CICS regions. We see the TS queue updates on both CICS regions.

Example 6-11 CEBR output from A6POTGR1 and A6POTGT1 after in-doubt resolved


CEBR TSQ RECIPROG SYSID TGT1 REC 1 OF 2 COL 1 OF 38 ENTER COMMAND ===> ************************** TOP OF QUEUE ******************************* 00001 A6POTGT1 19/10/05 12:31:26 CITGT1D D ************************* BOTTOM OF QUEUE *****************************

CEMT INQUIRE UOW INDOUBTThe output of the CEMT INQUIRE UOW INDOUBT command on CICS region A6POTGR1 (Example 6-11) confirms that there are no in-doubt UOWs remaining.

Figure 6-11 CEMT INQUIRE UOW INDOUBT after in-doubt resolution

RRS Archive LogThe RRS archive log shows information relating to all RRS URs, including completed URs. See “RRS Archive logs” on page 187 for more information about browsing RRS archive logs.

INQUIRE UOW INDOUBT STATUS: RESULTS - OVERTYPE TO MODIFY Uow(* ) Ind NOT FOUND SYSID=TGR1 APPLID=A6POTGR1 RESPONSE: 1 ERROR TIME: 12.42.17 DATE: 10.19.05 PF 1 HELP 3 END 5 VAR 7 SBH 8 SFH 9 MSG 10 SB 11 SF


Example 6-12 shows the RRS archive log entries for our resolved in-doubt transaction.

Example 6-12 RRS Archive log - XA transaction in-doubt resolution

X1 2005/10/19 12:32:15.788069 BLOCKID=000000000021B58B URID=BDC893A07E517DD00000006801030000 JOBNAME=CITGT1G USERID=CITGT1G PARENT URID=00000000000000000000000000000000 SURID=N/A WORK MANAGER NAME=CTG.RRM.CITGT1G.IBM.UA SYNCPOINT=AtraCmt RETURN CODE=00000000 START=2005/10/19 10:31:25.899029 COMPLETE=2005/10/19 10:32:15.787771 EXITFLAGS=00800000 LUWID=PSSCX9.A6POTGT1 C893A04847C0 0001 TID= GTID= FORMATID=001463898948 (decimal) 57415344 (hexadecimal) GTRID= 00000107031673640000000100000001652EE808ECBAEA9AB3C52EA4110E2BA2FCD740E7 BQUAL= 00000107031673640000000100000001652EE808ECBAEA9AB3C52EA4110E2BA2FCD740E7000000010000000000000000000000000002

X1 2005/10/19 12:42:06.597390 BLOCKID=000000000021B7F5 URID=BDC8939F7E517A5C000000BA01030000 JOBNAME=CITGR1G USERID=CITGR1G PARENT URID=00000000000000000000000000000000 SURID=N/A WORK MANAGER NAME=CTG.RRM.CITGR0G SYNCPOINT=AtraCmt RETURN CODE=00000000 START=2005/10/19 10:31:28.361576 COMPLETE=2005/10/19 10:42:06.597245 EXITFLAGS=00800000 LUWID=PSSCX9.A6POTGR1 C8939F7B6448 0001 TID= GTID= FORMATID=001463898948 (decimal) 57415344 (hexadecimal) GTRID= 00000107031673640000000100000001652EE808ECBAEA9AB3C52EA4110E2BA2FCD740E7 BQUAL= 00000107031673640000000100000001652EE808ECBAEA9AB3C52EA4110E2BA2FCD740E7000000010000000000000000000000000001

Looking at the archive log, we are able to confirm that:

� The two RRS URs involved in the global transaction have the same GTRID:

00000107031673640000000100000001652EE808ECBAEA9AB3C52EA4110E2BA2FCD740E7

� The RRS work manager for the first UR (URID=BDC893A07E517DD00000006801030000) is the stand-alone Gateway daemon CITGT1G with CTG_RRMNAME CTG.RRM.CITGT1G.IBM.UA.


� This UR was committed (SYNCPOINT=AtraCmt RETURN CODE=00000000) by the Gateway daemon CITGT1G at 12:32:15.

� The RRS work manager for the second UR (URID=BDC8939F7E517A5C000000BA01030000 ) is the ctgmaster for CICS TG group R with CTG_MASTER_RRMNAME CTG.RRM.CITGR0G.

� This UR was committed (SYNCPOINT=AtraCmt RETURN CODE=00000000) by the Gateway daemon CITGR1G 10 minutes later at 12:42:06. We know that during the time between the two commit flows, the UR BDC8939F7E517A5C000000BA01030000 was in-doubt.

6.6 OperationsThe creation of an XA-enabled CICS TG group has an impact on the operational procedures for each Gateway daemon instance within the group.

6.6.1 CICS TG group startup and shutdownWhen starting up a CICS TG group, the master process must be started before the first Gateway daemon. If any Gateway daemon is started while the CICS TG master process is initializing, it suspends until the CICS TG master process initialization is complete.

CICS TG master process restartDuring initialization, the CICS TG master process retrieves information about any URs associated with its CTG_MASTER_RRMNAME which are in-doubt. After the CICS TG master process is started, Gateway daemons which are part of the CICS TG group can start concurrently.

CICS TG group normal shutdown sequenceThe CICS TG master process keeps track of the number of currently active Gateway daemons in the CICS TG group and, on receipt of a shutdown request, waits until all associated Gateway daemons have been shut down. If there are active Gateway daemons in the group at the time of a ctgmaster shutdown

Important: This example clearly shows that the commit flow which resolved the in-doubt UR was processed by Gateway daemon CITGR1G, while the original ECI request was processed by Gateway daemon CITGR2G. A Gateway daemon which is an instance of a CICS TG group can process syncpoint flows on behalf of other Gateway daemons within the same group.


request, message CTG8942I is written indicating the number of active Gateway daemons (Example 6-13).

Example 6-13 ctgmaster pending normal shutdown message

/F CITGR0G,APPL=SHUT

CTG8998I ctgmaster command accepted.10/19/2005 13:15:52.584 [0] CTG8932I ctgmaster is being shutdown10/19/2005 13:15:52.600 [0] CTG8942I ctgmaster is waiting on 1 associated Gateway daemons to complete shutdown

When all Gateway daemons in the CICS TG group have completed shutdown, then the CICS TG master process shutdown completes and RRS de-registration takes place.

While the CICS TG master process is pending shutdown, no new Gateway daemons within the CICS TG group can start.

CICS TG master process immediate shutdownIf the CICS TG master process receives the shutdown immediate command, it will terminate itself regardless of the number of associated active Gateway daemons. However, a log message is written indicating the number of associated Gateway daemons known to be active at the time (Example 6-14).

Example 6-14 ctgmaster immediate shutdown message

/F CITGR0G,APPL=SHUT,IMM

CTG8998I ctgmaster command accepted.10/19/2005 13:16:57.635 [0] CTG8943I ctgmaster immediate shutdown requested. Closing with 1 associated Gateway daemons10/19/2005 13:16:57.694 [0] CTG8933I ctgmaster shutdown is complete

Following an immediate shutdown of the CICS TG master process, the remaining active Gateway daemons continue to operate but cannot process ECI requests which are part of XA transactions.

As with a normal shutdown, no new Gateway daemon instances can be started until the CICS TG master process has itself been re-started.


6.6.2 CICS TG group commandsTable 6-5 summarizes the set of ctgmaster commands.

Table 6-5 ctgmaster commands

QUERY XIDSThe QUERY XIDS command is used to retrieve the list of Xids in which the CICS TG group has an interest. Example 6-6 on page 239 shows sample output for the QUERY XIDS command.

TRACEThe TRACE command is used to start or stop tracing activity in the CICS TG master process dynamically. The trace destination depends on the value of CTG_MASTER_TRACE environment variable or the -trace override switch. Example 6-15 shows sample output.

Example 6-15 ctgmaster TRACE command output

/F CITGR0G,APPL=TRACE,ON

CTG8998I ctgmaster command accepted.CTG8934I ctgmaster trace setting has been updated (state=1)

/F CITGR0G,APPL=TRACE,OFF

CTG8998I ctgmaster command accepted.CTG8934I ctgmaster trace setting has been updated (state=0)

USS command (input=stdin) MVS command (input=console)/F <jobname>,APPL=…..

query xids QUERY,XIDS

trace on|off TRACE,ON|OFF

shut|shutdown imm|immediate SHUT|SHUTDOWN[,IMM|IMMEDIATE]

dump DUMP

help HELP

Tip: The ctgmaster command accepts commas or spaces between verbs in the APPL= command string.


SHUTDOWNThe SHUTDOWN command shuts down ctgmaster. See 6.6.1, “CICS TG group startup and shutdown” on page 248 for guidance on shutting down ctgmaster.

DUMPInvoking the ctgmaster DUMP command does not produce an SDUMP but writes a message to stdout that contains the ctgmaster internal state data. It is intended for IBM technical support purposes.

HELPThe HELP command writes a message to stdout describing each of the available systems management commands.

6.7 Problem determinationIn this section, we describe additional tips and utilities for problem determination CICS TG group related problems. For other problem determination guidance about transaction related problems see 5.9, “Problem determination” on page 221.

CTGRRMS authorizationThe ctgmaster user ID (CITGR0G) needs to be authorized to use CTGRRMS services. It needs UPDATE access to the CICSTG.RRMS.SERVICE RACF FACILITY class profile. You can use the following TSO command to check that the ctgmaster user ID has the appropriate access:

RLIST FACILITY CTG.RRMS.SERVICE AUTHUSER

Example 6-16 shows the output of this command on our system.

Example 6-16 RLIST FACILITY CTG.RRMS.SERVICE AUTHUSER command output

USER ACCESS ACCESS COUNT---- ------ ------ -----CITGTJ ALTER 000000 CITGC1G UPDATE 000000 CITGR1G UPDATE 000000 CITGC2G UPDATE 000000 CITGR2G UPDATE 000000 CITGR0G UPDATE 000000

Tip: The current tracing status can be queried by omitting the ON|OFF parameter on the trace command.


CITGS0G UPDATE 000000 CITGS1G UPDATE 000000 CITGS2G UPDATE 000000 CITGT0G UPDATE 000000 CITGT1G UPDATE 000000 CITGT2G UPDATE 000000

CICS in-doubt failure messagesIn the event of an in-doubt failure, the Recovery Manager component of CICS issues a DFHRMxxxx message. The message contains detailed information, including:

� The time and date of the original failure � The transaction identifier and task number � The netname of the remote system � The operator identifier � The operator terminal identifier � The network-wide unit of work identifier � The local unit of work identifier

For examples of how to resolve in-doubt UOWs in CICS, see the CICS Intercommunication Guide, SC34-6448.

6.7.1 Tracing with ctgmaster traceTracing with ctgmaster trace is controlled by the environment variable CTG_MASTER_TRACE. It is used to set the HFS destination for ctgmaster trace data. If undefined, trace data is written to stderr and is mapped to the STDERR DD card when running in batch mode. This trace is distinct from the Gateway daemon and JNI trace described in 2.9.3, “Tracing” on page 68.

Having tracing active for the CICS TG master process does not have any impact on the performance of the Gateway daemons and does not produce a large amounts of data.


Chapter 7. Transactions with WebSphere Application Server for z/OS

In this chapter, we describe the transactional capabilities when the CICS Transaction Gateway for z/OS is used with WebSphere Application Server on z/OS. We document how we prepared the system and the values that we used. We also provide details on how we set up and modified the different tiers in our test environment to support transactions. In addition, we explain how we tested a global transaction involving both local and remote CICS connections from WebSphere Application Server for z/OS.

7


7.1 PreparationIn our testing, we used WebSphere Application Server for z/OS. For information about how to enable transactions when using WebSphere Application Server on Windows see Chapter 5, “Gateway daemon transactions” on page 157. We tested a global transaction with our sample CTGTesterCCIXA application using the cicseciXA.rar.

Figure 7-1 shows how a global transaction managed by WebSphere Application Server for z/OS can span different resource managers.

Figure 7-1 Software components: WebSphere z/OS transaction scenarios

Figure 7-1 shows a global transaction spanning updates to different resource managers, a CICS region, and a DB2 database. The global transaction is coordinated by the transaction manager, which is WebSphere Application Server for z/OS. RRS is used to coordinate the resource updates.

For an overview of basic transactional concepts and standards, see 5.2, “Transactions - what are they” on page 159.

Note: In our global transaction configuration, we document the setup steps for transactions which update recoverable resources in two CICS regions, but we do not document setup or testing of DB2.

CICS A

z/OS

EXCI

CICS ECIXA

resource adapter

JCA


servlet

CTGTesterCCIXA

DB2

JDBC

Datasource

Global transaction

TransactionManager

ResourceManager

RRS

ResourceManager

Java Servlet

JSP




7.2 Resource Recovery ServicesResource Recovery Services (RRS), a component of z/OS, provides a global syncpoint manager that any resource manager on z/OS can exploit. It enables transactions to update recoverable resources managed by many resource managers.

For information about how RRS is used by CICS and the CICS TG, see 5.3, “Resource Recovery Services” on page 163.

7.3 Transactional support with WebSphere on z/OSUnlike the more traditional transaction managers such as CICS or IMS, WebSphere Application Server for z/OS uses RRS services as a part of its base implementation to provide two-phase commit support. WebSphere Application Server for z/OS does not start without RRS being available, and it abends in the event of an RRS failure.

In addition to support for XA-capable resource managers, WebSphere Application Server for z/OS also supports resource managers that provide an RRS-enabled resource adapter. Such a resource adapter implements a specific type of transaction contract with WebSphere Application Server. This means that, in addition to the three types of transaction support defined by the JCA (see “JCA and transactions” on page 167), WebSphere Application Server for z/OS supports a fourth type, RRSTransactional. RRSTransactional is a z/OS only extension to the JCA.

Windows 2000 z/OS


Windows 2000 Service Pack 4 WebSphere Application Server V6.0.1




Chapter 7. Transactions with WebSphere Application Server for z/OS 255

A global transaction in WebSphere Application Server can use multiple resource managers that use either RRS-enabled or XA resource adapters, and WebSphere Application Server coordinates the updates across all resource managers. To do this, WebSphere Application Server generally takes an RRS SDSRM (Server Distributed Syncpoint Resource Manager) role so that it becomes the syncpoint coordinator and RRS drives the commit or backout processing for those resource managers that support RRS.

Connectors for z/OS that are RRS-enabled include:

� CICS ECI resource adapters � IMS Connect � IMS JDBC � DB2 for z/OS local JDBC driver � WebSphere MQ JMS

Some of these resource adapters support both XA and RRS modes of operation, depending on how they are configured. The RRS mode of operation is usually restricted to instances where both WebSphere Application Server and the resource manager reside on the same z/OS image.

For a complete list of RRS-capable connectors, refer to Systems Programmer's Guide to Resource Recovery Services (RRS), SG24-6980.

7.3.1 Transactional support in the CICS ECI resource adaptersThe two CICS ECI resource adapters provided with CICS TG for z/OS V6.1 can be used with WebSphere Application Server on z/OS:

cicseci.rar The CICS ECI resource adapter has the same transactional behavior as the CICS ECI resource adapter that is provided with CICS TG V6. When installed into WebSphere Application Server on z/OS, it provides two-phase commit support using RRSTransactional for local connections and single-phase commit support for remote connections.

cicseciXA.rar When installed into WebSphere Application Server on z/OS, the CICS ECI XA resource adapter provides two-phase commit support by using RRSTransactional for local connections and by supporting the XAResource interface for remote connections.

Restriction: For both CICS ECI resource adapters, when a local connection is used, the CICS region must reside on the same z/OS image as the WebSphere Application Server.


Which resource adapter to useIn normal circumstances, you will use a local connection from WebSphere Application Server for z/OS because a direct cross-memory connection to CICS is simpler and performs better than a remote TCP/IP connection via a Gateway daemon.

When local connections are used, the transactional behavior of the ECI resource adapters is the same so either can be used. However, you might chose to use the cicseciXA resource adapter if you have plans to use remote connections because you then have greater flexibility in enabling XA transaction support for these connections in the future.

There are circumstances in which you might chose to use a remote connection, for example:

� You do not want to install a CICS region on the same z/OS image as WebSphere Application Server.

� WebSphere application Server is running on z/OS.e1. Because it is not possible to install a CICS region on a z/OS.e image, you must use a remote connection.

� You want to connect to a CICS region on another system and, rather than routing the request via a CICS region installed on the same z/OS image and using a SNA based CICS ISC (Intersystem communication) connection, you prefer to use a TCP/IP connection.

When a remote connection is used, and if you want XA transaction support for the remote connection, you must install the cicseciXA resource adapter (cicseciXA.rar).

Both the ECI and ECI XA resource adapters can be deployed into WebSphere Application Server for z/OS at the same time, provided that they are both from the same release of CICS TG.

Important: Using a local connection provides two-phase commit support without the additional processing overhead associated with XA transactions.

1 z/OS.e is a specially priced offering of z/OS that provides select z/OS function for the zSeries 800 system. Traditional workloads, such as CICS TS, are not licensed for z/OS.e.


7.4 Configuring for global transactionsIn this section, we show how we configured WebSphere Application Server for z/OS to enable a global transaction that involves two ECI connection factories: a local connection to CICS region and a remote connection to CICS region (via the Gateway daemon CITGT1G).



7.4.2 Configuring CICS No additional CICS setup is required for global transaction support with WebSphere Application Server for z/OS over and above the transactional EXCI setup that is documented for Extended LUW support in 5.6.2, “Configuring CICS” on page 175.

Value WebSphere Application Server

CICS region 1 Gateway daemon CICS region 2

hostname tx1.mop.ibm.com - tx1b.mop.ibm.com -

IP address 9.100.193.122 - 9.100.193.121 -

TCP/IP port 13880 - 12101 -

Job name Daemon region: CITGRDMNControl region: CITGRS1Servant region: CITGRS1S

CITGR1CI CITGT1G CITGT1CI

APPLID A6POTGR1 A6POTGT1

NETNAME CITGRS1 - CITGT1G -

RRM Name BBO.CITGRC1.CITGRCTNCITGRS1.IBM

- CTG.RRM.CITGT1G.IBM.UA

-


7.4.3 Configuring WebSphere To test our global transaction, we used the CTGTesterCCIXA application that contains two ECI resource references and that can be used to call programs running in two CICS regions via different connection factories. For further details about CTGTesterCCIXA, refer to Appendix A, “Sample applications” on page 361.

Dealing with CICS transaction abendsCICS cannot tell RRS to rollback updates to recoverable resources following an unhandled abend of the long-running mirror task. This means that, by default, if the first JCA request of a global transaction succeeds but the second JCA request fails (because of a CICS transaction abend) the updates made by the first request are committed, while the updates made by the second request cannot be committed. See “Dealing with CICS transaction abends” on page 198 for information about ways to avoid such inconsistencies, including details on the new ABENDBKOUT=YES option of the DFHXCOPT table.

The CTGTesterCCIXA application is coded to issue a setRollbackOnly() on the EJB session context in the catch block for the ECIInteraction.execute() call (see Example A-1 on page 368). This ensures that the CICS abend is propagated onto WebSphere Application Server as the transaction manager and the global transaction (if one exists) is rolled back.

Installing the CICS ECI XA resource adapterBecause we are using a remote connection within our global transaction, we need to install the cicseciXA.rar. The process of installing the CICS ECI XA resource adapter (cicseciXA.rar) in WebSphere Application Server for z/OS is identical to that described for the ECI resource adapter (see 4.2.3, “Installing the CICS ECI resource adapter” on page 126).


Both resource adapters can be installed at the same time. Figure 7-2 shows the two resource adapters installed into our application server.

Figure 7-2 Administrative Console - CICS ECI resource adapters

Creating connection factoriesThe process of creating connection factories using the CICS ECI XA resource adapter in WebSphere Application Server for z/OS is the same as described in 4.2.4, “Creating a connection factory” on page 128.

We created two connection factories CITGR1CI-tx1-local and CITGR1CI-tx1b-12101-XA based upon the CICS ECI XA resource adapter. Figure 7-3 shows our connection factories in the WebSphere Administrative Console.


Figure 7-3 Administrative Console - CICS ECI connection factories

The two connection factories represent two connections to two different CICS regions; CICS region CITGR1CI is accessed with a local connection and CICS region CITGT1CI is accessed via the Gateway daemon which binds to the tx1b IP address using port 12101.

Deploying the applicationTo deploy the CTGTesterCCIXA application we followed the same procedure as is described in 4.2.6, “Deploying our sample J2EE application” on page 134. This time, we mapped two resource references using the WebSphere Administrative Console as shown in Figure 7-4.

Figure 7-4 Administrative Console - mapping XA resource references

Tip: Before using the XA transaction support, you should ensure that your WebSphere Application Server level contains the fix for APAR PQ99940. This APAR fixes a connection pooling performance problem following a connection failure, for example, as a result of a CICS region or Gateway daemon failure. The fix is contained in V6.0.1 of WebSphere Application Server for z/OS.


After the application was deployed, we started the application.

Configure WebSphere transaction logging Prior to WebSphere Application Server V5.1, WebSphere Application Server for z/OS had no logging system of its own. It used RRS to hold information about all transactions, and on startup it retrieves this information from RRS.

WebSphere Application Server V5.1 introduced an XA partner log, which holds information about XA resources accessed. When an application accesses XA resources, WebSphere Application Server stores information about the resource updates in the transaction log in order to enable XA transaction recovery.

On the WebSphere Administrative Console, follow the links Application servers → server1 → Container Services → Transaction Service to a page that contains tabs for both startup configuration and runtime settings.

By default, WebSphere Application Server uses the transaction log directory derived from the following pattern:

install_root/tranlog/cell_name/node_name/server_name/transaction/partnerlog

On z/OS, you can configure the transaction logs to be either in the HFS or in MVS logstreams. If the logs are placed in logstreams, we recommend that they be in the Coupling Facility instead of DASD logstreams. The logs (and structures, if applicable) are created in job BBOLOGSA in the WebSphere Application Server installation dialog. To specify the transaction log as a logstream, you need to specify the following for the transaction log directory:

logstream://HLQ

HLQ is a user-defined value from 1 to 8 characters which is specified in the WebSphere Application Server installation dialog. In our case, it is CITGRXA.

In the Configuration panel, we did not specify a transaction log directory. Example 7-1 shows the default path used by WebSphere Application Server on our test system.

Example 7-1 Default directory location of transaction log

/CITG/CellCITGR/AppServer/profiles/default/tranlog/CITGR_cell1/CITGR_node1/CITGR_server1/transaction/partnerlog

Transaction TimeoutIn the properties page for the Transaction Service, you can optionally change other transaction-related configuration properties. We changed the value for Total transaction lifetime timeout (default 120 seconds). This sets the number of seconds a transaction can remain inactive before it is ended by the


Transaction Service. A value of zero (0) indicates that there is no timeout limit. Because some of our test cases sometimes involve requests which suspend in CICS, we increased this value to 1200 seconds.

Configure Tivoli Performance Viewer Setting up of the Tivoli Performance Viewer for monitoring transactions is covered in section “Tivoli Performance Viewer - Transaction Manager module” on page 189. For this test scenario, we did not monitor transactions using Tivoli Performance Viewer.

7.4.4 Testing the configurationIn this section, we detail how we tested the WebSphere Application Server for z/OS global transaction configuration using our sample J2EE application CTGTesterCCIXA.

Figure 7-5 shows the basic outline of our environment. We used this configuration for two scenarios:

� Normal termination of a global transaction

An update and commit of recoverable resources in two CICS regions within one global transaction (see “Global transaction normal termination” on page 266).

� Abnormal termination of a global transaction

A CICS subsystem failure during an in-flight global transaction (see “Global transaction abnormal termination” on page 273).


Figure 7-5 Global transaction scenario with WebSphere Application Server for z/OS

Note: On our system the Gateway daemon binds to a different TCP/IP address than the WebSphere Application Server, but they are actually running on the same z/OS image. In practice, they could be running on two completely different systems.

HTML


ServletJSP

CTGTesterCCIXA

z/OS

tx1.mop.ibm.com

CITGR1CI(applid A6POTGR1)

CITGR_server1

CITGT1G


CICS

EXCIJNI

12101

Gatewaydaemon

tx1b.mop.ibm.com



CICSCICS TG V6.1

cicseciXA.rar EXCI Long RunningMirror ECIW

RRS

JNI

ECI RequestRRSTransactional


In test our configuration, we invoked our test application CTGTesterCCIXA (Figure 7-6).

Figure 7-6 CTGTesterCCIXA Start global transaction


During these tests, we specified DW as the input to the COMMAREA, which means that the ECIPROG program writes to a recoverable TS Queue and then delays for one minute.

The iteration count was set to 1 for transaction leg 1 and 3 for transaction leg 2. Clicking Submit results in a total of four ECI calls: one request to CICS region CITGR1CI (applid A6POTGR1) and three requests to CICS region CITGT1CI (applid A6POTGT1). This iteration give us four minutes during which to monitor the transaction state across the various components of the system.

During the course of these scenarios, we monitored the transaction using each of the following:

CICS CEMT commandRRS ISPF panels RRS archive log

Global transaction normal termination This test demonstrates successful update and commit of resources on two CICS regions accessed via different connection factories, within the scope of a single global transaction.

The Web browser appeared to be busy for four minutes before displaying the results page that contains the COMMAREAs that are returned from the two legs of the global transaction.

CICS CEMT commandWhile the second ECI request was in-flight on A6POTGT1, we issued the CEMT INQUIRE TASK command on both CICS regions, A6POTGR1 and A6POTGT1. Figure 7-7 shows the ECIW mirror task running on A6POTGR1 as Task 519 suspended in RRMSEXIT state, which means that CICS is waiting for a commit or backout flow from RRS.

Program ECIPROG on A6POTGR1 has completed at this stage but CICS is waiting for a commit flow from RRS, which occurs when the transaction manager (WebSphere Application Server) syncpoints the global transaction.

Note: This scenario is for testing purposes only. Normal transactions would not be expected to delay in this way.


Figure 7-7 CEMT INQUIRE TASK on A6POTGR1 with the mirror in RRMSEXIT state

Figure 7-8 shows the ECIT mirror task running on A6POTGT1 as Task 448, suspended in ICWAIT state, due to the active EXEC CICS DELAY in ECIPROG.

Figure 7-8 CEMT INQUIRE TASK on A6POTGT1 with the mirror in ICWAIT state

We also issued the CEMT INQUIRE EXCI command on both CICS regions. Figure 7-9 shows the EXCI information for the long-running mirror task ECIW (task 519) running on A6POTGR1, including the associated RRS UR identifier for the first leg of the transaction.

The INQUIRE EXCI command provides the information to correlate an active long-running mirror task in a CICS region, with the RRS UR identifier. We see that the Exc field highlights the jobname, stepname, and the mvsid associated to the EXCI client application. In this case, the EXCI client application is job CITGRS1S (the jobname of the WebSphere Application Server servant region) which is running on the TX1 system (mvsid X1).

INQUIRE TASK STATUS: RESULTS - OVERTYPE TO MODIFY Tas(0000519) Tra(ECIW) Sus Tas Pri( 001 ) Sta(TO) Use(CITGR1D ) Uow(BDC64F4348908621) Hty(RRMSEXIT) Tas(0000520) Tra(CEMT) Fac(N123) Run Ter Pri( 255 ) Sta(TO) Use(CITGR1D ) Uow(BDC64F49EA5BE4AE)


INQUIRE TASK STATUS: RESULTS - OVERTYPE TO MODIFY Tas(0000448) Tra(ECIT) Fac(11 ) Sus Ter Pri( 001 ) Sta(D ) Use(CITGT1D ) Uow(BDC64F7CD273C869) Hty(ICWAIT ) Tas(0000450) Tra(CEMT) Fac(N124) Run Ter Pri( 255 ) Sta(TO) Use(CITGT1D ) Uow(BDC64F931F317EA7)



Figure 7-9 CEMT INQUIRE EXCI on A6POTGR1 showing a UR identifier

Figure 7-10 shows the EXCI information for the long-running mirror task ECIT (task 448) running on A6POTGT1, including the associated RRS UR identifier for the second leg of the transaction. The EXCI client application is job CITGT1G (the jobname of the Gateway daemon).

Figure 7-10 CEMT INQUIRE EXCI on A6POTGT1 showing a UR identifier

Notice that the two RRS UR identifiers are different for the two legs of the global transaction. We see in the RRS archive log (see “RRS Archive Log” on page 270) how these two different URs are tied together in a global transaction with a Global transaction identifier (GTRID) .

INQUIRE EXCI STATUS: RESULTS Exc(CITGRS1S.CITGRS1S. - X1 ) Tas(0000519) Uri(BDC64F437E517374000000D701030000)


INQUIRE EXCI STATUS: RESULTS Exc(CITGT1G.CITGT1G. - X1 ) Tas(0000448) Uri(BDC64F7C7E5176E80000009001030000)



Figure 7-11 shows the results of the test displayed on the Web browser.

Figure 7-11 CTGTesterCCIXA global transaction normal termination


At the end of the test, the recoverable TS Queue RECIPROG on CICS region CITGR1CI (applid A6POTGR1) contained one new entry, and the recoverable TS Queue RECIPROG on CICS region CITGT1CI (applid A6POTGT1) contained three new entries (Example 7-2).



CEBR TSQ RECIPROG SYSID TGT1 REC 1 OF 3 COL 1 OF 38 ENTER COMMAND ===> ************************** TOP OF QUEUE ******************************* 00001 A6POTGT1 17/10/05 17:15:56 CITGT1D D 00002 A6POTGT1 17/10/05 17:16:56 CITGT1D D 00003 A6POTGT1 17/10/05 17:17:56 CITGT1D D ************************* BOTTOM OF QUEUE *****************************

RRS Archive LogThe RRS archive log shows information relating to all RRS URs, including completed URs. We selected option 1 from the RRS primary menu panel and the Browse an RRS log stream menu panel was displayed. We then chose to view the detailed information of the RRS archive log.

Example 7-3 shows the log entries for the global transaction normal termination scenario.

Example 7-3 RRS Archive log - XA transaction normal termination

RRS/MVS LOG STREAM BROWSE DETAIL REPORT

READING ATR.X1.ARCHIVE LOG STREAM

X1 2005/10/17 17:18:56.407057 BLOCKID=000000000020407E URID=BDC64F7C7E5176E80000009001030000 JOBNAME=CITGT1G USERID=CITGT1G PARENT URID=00000000000000000000000000000000 SURID=N/A WORK MANAGER NAME=CTG.RRM.CITGT1G.IBM.UA SYNCPOINT=AtraCmt RETURN CODE=00000000 START=2005/10/17 15:18:56.371335 COMPLETE=2005/10/17 15:18:56.406601 EXITFLAGS=00800000 LUWID=PSSCX9.A6POTGT1 C64F7CD26D28 0001 TID= GTID=

FORMATID=003284271494 (decimal) C3C20186 (hexadecimal)


GTRID=BDC64F4342ADA7450000002400000007BD915E37E48D782D0000012C000000010964C17A BQUAL=BDC64F4342ADA7450000002400000007BD915E37E48D782D0000012C000000010964C17A00000001000001C800000002000000000001 RMNAME=CTG.RRM.CITGT1G.IBM.UA ROLE=SDSRM FLAGS=10021000 PROTOCOL=PresumeNothing StateCheck EXIT RC=Uncalled Prepare EXIT RC=00000014 DistSp EXIT RC=Uncalled Commit EXIT RC=00000000 Backout EXIT RC=Uncalled EndUr EXIT RC=Uncalled ExitFailed EXIT RC=Uncalled Completion EXIT RC=Uncalled OnlyAgent EXIT RC=Uncalled RMNAME=DFHRXDM.A6POTGT1.IBM ROLE=Participant FLAGS=10001000 PROTOCOL=PresumeAbort StateCheck EXIT RC=Uncalled Prepare EXIT RC=00000000 DistSp EXIT RC=Uncalled Commit EXIT RC=00000010 Backout EXIT RC=Uncalled EndUr EXIT RC=Uncalled ExitFailed EXIT RC=00000010 Completion EXIT RC=Uncalled OnlyAgent EXIT RC=Uncalled

X1 2005/10/17 17:18:56.416139 BLOCKID=00000000002042E8 URID=BDC64F437E517374000000D701030000 JOBNAME=CITGRS1 USERID=CITGRCRU PARENT URID=00000000000000000000000000000000 SURID=N/A WORK MANAGER NAME=BBO.CITGRC1.CITGRCTNCITGRS1.IBM SYNCPOINT=AtraPrp RETURN CODE=00000000 START=2005/10/17 15:18:56.383242 COMPLETE=2005/10/17 15:18:56.416012 EXITFLAGS=00840000 LUWID=PSSCX9.A6POTGR1 C64F43488898 0001 TID= GTID=

FORMATID=003284271494 (decimal) C3C20186 (hexadecimal) GTRID=BDC64F4342ADA7450000002400000007BD915E37E48D782D0000012C000000010964C17A BQUAL=BDC64F4342ADA7450000002400000007BD915E37E48D782D0000012C000000010964C17A00000001000001C800000002000000000002RMNAME=DFHRXDM.A6POTGR1.IBM ROLE=Participant FLAGS=10001000 PROTOCOL=PresumeAbort StateCheck EXIT RC=Uncalled Prepare EXIT RC=00000000 DistSp EXIT RC=Uncalled


Commit EXIT RC=00000010 Backout EXIT RC=Uncalled EndUr EXIT RC=Uncalled ExitFailed EXIT RC=00000010 Completion EXIT RC=Uncalled OnlyAgent EXIT RC=Uncalled RMNAME=BBO.CITGRC1.CITGRCTNCITGRS1.IBM ROLE=SDSRM FLAGS=10041000 PROTOCOL=PresumeAbort StateCheck EXIT RC=Uncalled Prepare EXIT RC=00000000 DistSp EXIT RC=Uncalled Commit EXIT RC=00000000 Backout EXIT RC=Uncalled EndUr EXIT RC=00000000 ExitFailed EXIT RC=00000000 Completion EXIT RC=Uncalled OnlyAgent EXIT RC=Uncalled


� There are two different RRS URs involved in the global transaction, but they have the same GTRID:

BDC64F4342ADA7450000002400000007BD915E37E48D782D0000012C000000010964C17A

� The RRS work manager for the first UR (URID=BDC64F7C7E5176E80000009001030000) is the Gateway daemon CITGT1G with Work Manager Name CTG.RRM.CITGT1G.IBM.UA. The Gateway daemon has a syncpoint coordinator or SDSRM (Server Distributed Syncpoint Resource Manager) role for this UR.

� The RRS work manager for the second UR (URID=BDC64F437E517374000000D701030000) is the WebSphere Application Server CITGRS1 with Work Manager Name BBO.CITGRC1.CITGRCTNCITGRS1.IBM. WebSphere Application Server has the SDSRM role for this UR.

� The CICS regions DFHRXDM.A6POTGT1.IBM and DFHRXDM.A6POTGR1.IBM have a Participant role within the URs.

Note: An X/Open identifier (XID) acts as the identifier for a distributed unit of work. An XID consists of a Global transaction Identifier (GTRID) and a Branch qualifier (BQUAL). RRS supports XIDs and WebSphere Application Server for z/OS uses the XID to link the two URs. If you check the GTRID and BQUAL for both URs, you will see that they are the same.


� The RRS archive log also highlights the two types of 2PC protocol used by RRS:

Presumed nothing When the UR is in an in-prepare state, RRS logs information about the resource manager. If the resource manager fails, RRS presumes nothing and during resource manager restart RRS is able to return log information when the resource manager calls the Retrieve_UR_Interest service. RRS tells the resource manager that the UR is to be backed out. This protocol is used between RRS and the Gateway daemon.

Presumed abort When the UR is in an in-prepare state, RRS does not log any information about the resource manager. If the resource manager fails, RRS presumes that it’s resources are backed out. This protocol is used between RRS and CICS.

In this example, we see clearly that both an RRSTransactional resource manager (CICS ECI local connection) and an XA resource manager (CICS ECI XA remote connection) can be involved in a global transaction. WebSphere Application Server provides the overall transaction management and uses a combination of RRSTransactional and XA protocols to manage the global transaction.

Global transaction abnormal termination This test scenario demonstrates a failure scenario, using the CTGTesterCCIXA application. The configuration and application parameters are identical to those used in the scenario “Global transaction normal termination” on page 266. In this scenario, however, during the second leg of the transaction the CICS region A6POTGR1 is cancelled. The second leg of the transaction continues, making the three updates to TS Queue RECIPROG on CICS region A6POTGT1. However, at the end of the final ECI request, the syncpoint prepare call to CICS region A6POTGR1 for transaction leg 1 fails. This leads to a complete rollback of all updated resources.

Note: The RRS exit call return codes are not always easy to interpret. For example, in this case the CICS regions return x’10’ from the commit exit, meaning ATRX_FORGET. CICS is telling RRS to forget its interest in this UR. For more detailed information about RRS exit return codes, refer to z/OS MVS Programming: Resource Recovery, SA22-7616.


CICS CEBR commandAt just over three minutes into the test scenario, we looked at the contents of the TS Queue RECIPROG on both A6POTGR1 and A6POTGT1. We could see that both queues contained updates (Example 7-4).

Example 7-4 CEBR output on A6POTGR1 and A6POTGT1 after three minutes


CEBR TSQ RECIPROG SYSID TGT1 REC 1 OF 1 COL 1 OF 38 ENTER COMMAND ===> ************************** TOP OF QUEUE *******************************00001 A6POTGT1 18/10/05 09:55:50 CITGT1D D 00002 A6POTGT1 18/10/05 09:56:50 CITGT1D D

************************* BOTTOM OF QUEUE *****************************

At this point, we cancelled CICS region A6POTGR1 with the MVS command /CANCEL CITGR1CI.

After the ECIPROG program on A6POTGT1 ended, WebSphere Application Server attempts to commit the global transaction. At this time, the application server is unable to communicate with the failed CICS region A6POTGR1 so the global transaction must be rolled back. Example 7-12 shows how this transaction rollback is reported on the Web browser.


Figure 7-12 CTGTesterCCIXA global transaction abnormal termination

RRS Archive LogThe RRS archive log shows us information that is related to the failed transaction. We selected option 1 from the RRS primary menu panel and the Browse an RRS log stream menu panel was displayed. We then chose to view the detailed information of the RRS archive log.


Example 7-5 shows the log entries for the XA transaction abnormal termination scenario.

Example 7-5 RRS Archive log - XA transaction abnormal termination

RRS/MVS LOG STREAM BROWSE DETAIL REPORT READING ATR.X1.ARCHIVE LOG STREAM X1 2005/10/18 09:58:50.705010 BLOCKID=00000000002084EC URID=BDC72EFB7E517A5C0000002201030000 JOBNAME=CITGT1G USERID=CITGT1G PARENT URID=00000000000000000000000000000000 SURID=N/A WORK MANAGER NAME=CTG.RRM.CITGT1G.IBM.UA SYNCPOINT=AtraBak RETURN CODE=00000000 START=2005/10/18 07:58:50.663694 COMPLETE=2005/10/18 07:58:50.704442 EXITFLAGS=00000000 LUWID=PSSCX9.A6POTGT1 C72EFB37FEBB 0001 TID= GTID= FORMATID=003284271494 (decimal) C3C20186 (hexadecimal) GTRID= BDC72EC1458FC440000000240000000ABD915E37E48D782D0000012C000000010964C17A BQUAL= BDC72EC1458FC440000000240000000ABD915E37E48D782D0000012C000000010964C17A00000001000001C800000002000000000001 RMNAME=CTG.RRM.CITGT1G.IBM.UA ROLE=SDSRM FLAGS=10021000 PROTOCOL=PresumeNothing StateCheck EXIT RC=Uncalled Prepare EXIT RC=00000014 DistSp EXIT RC=Uncalled Commit EXIT RC=Uncalled Backout EXIT RC=00000000 EndUr EXIT RC=Uncalled ExitFailed EXIT RC=Uncalled Completion EXIT RC=Uncalled OnlyAgent EXIT RC=Uncalled RMNAME=DFHRXDM.A6POTGT1.IBM ROLE=Participant

FLAGS=10000000 PROTOCOL=PresumeAbort StateCheck EXIT RC=Uncalled Prepare EXIT RC=00000000 DistSp EXIT RC=Uncalled Commit EXIT RC=Uncalled Backout EXIT RC=00000010 EndUr EXIT RC=Uncalled ExitFailed EXIT RC=00000010 Completion EXIT RC=Uncalled OnlyAgent EXIT RC=Uncalled



� A single entry for just one RRS UR:

BDC72EFB7E517A5C0000002201030000

� The RRS work manager for the UR is the Gateway daemon CITGT1G with CTG_RRMNAME CTG.RRM.CITGT1G.IBM.UA.

� The Gateway daemon has an SDSRM role and the CICS region A6POPTGT1 has a Participant role within the UR.

� The log entry shows that the backout call (AtraBak) to CITGT1G returns with code 0. This backout call is made because the transaction manager, WebSphere Application Server for z/OS, has rolled back the global transaction following the failure to prepare the CICS region CITGR1G.

Note that there is no log entry for the prepare failure because the two-phase commit protocol used for a local connection between WebSphere Application Server for z/OS and the CICS region A6POTGR1 is the Presumed Abort protocol. RRS does not log any information about CICS; if CICS fails, RRS presumes that it’s resources are backed out.

After the restart of CICS region A6POTGR1 was complete, the CICS log showed message DFHRM0201 (Example 7-6).


DFHRM0201 10/18/2005 12:10:34 A6POTGR1 0 backout-failed and 1 commit-failed UOWs were reconstructed.

Finally, we used CEBR to check that the TS Queue updates on both CICS regions were backed out. The TS Queue updates made by the ECIT mirror transaction on A6POTGT1 were backed out when the transaction was rolled back. The TS Queue updates made by the ECIW mirror transaction are backed out during the emergency restart of A6POTGR1.

7.5 Problem determinationIn this section, we document common transaction problems and information that we learned while configuring our transaction test scenario with WebSphere Application Server for z/OS. For general problem determination guidance see 2.9, “Problem determination” on page 60.


Abend ARXCIf a Transactional EXCI is received by a CICS region which has the system initialization parameter RRMS=NO specified, the request fails and the global transaction rolls back. We noted the following message in the CICS message log:

A6POTGR1 Transaction ECIW abend ARXC in program DFHXMAB term 31. Updates to local recoverable resources will be backed out. EXCI job = CITGRS1S.CITGRS1S. - X1.

7.5.1 TracingIn order to enable JNI trace to analyze transaction problems, you need to specify the CTG_JNI_TRACE and CTG_JNI_TRACE_ON environment variables using the WebSphere Administrative Console. See 4.7.1, “Tracing” on page 152 for information about enabling JNI trace with WebSphere Application Server for z/OS.


Part 4 Security Management

In this part, we describe the different security capabilities of the CICS Transaction Gateway for z/OS. We provide detailed information about how we configured secure interoperation between WebSphere Application Server and CICS.

We cover two different topologies:


We discuss the configuration of security within WebSphere Application Server, the Gateway daemon, and CICS.

Part 4


Chapter 8. Gateway daemon security

In this chapter, we outline the main CICS and RACF security functions that can be used with the CICS Transaction Gateway for z/OS. We start with an introduction to CICS security and then we discuss how the basic CICS security mechanisms can be used to secure CICS programs which are called via the Gateway daemon on z/OS. We document how we prepared the system and the settings that we used. We explain how we tested the environment using the CICS TG sample ECI application EciB1 and our own sample J2EE application CTGTesterCCI. We also give details of some of the common security related problems you might see when using the CICS TG for z/OS.

8


8.1 PreparationIn our testing, we connected from Java client applications running on WebSphere Application Server for Windows to CICS via the Gateway daemon on z/OS. In this chapter, we concentrate on how to configure security within and between each tier (Figure 8-1).

Figure 8-1 Software components: Gateway daemon security

For information about how to enable SSL connections to the Gateway daemon, see Chapter 9, “Enabling SSL with the Gateway daemon” on page 309.



Before starting the security tests with our CTGTesterCCI J2EE application, we enabled security for our WebSphere Application Server. Our security configuration was as follows:

z/OS

CICS TS V3.1

CICS program

Windows

Java application

EciB1

EXCI

Gatewaydaemon

CICS TG V6.1

MRO/IRC

RACF

user ID

(user ID +password)

authentication authorization

EJBCTGTesterCCI

CICS ECI resource adapter

CICS TG V6.1

WebSphere Application Server V6

HTML

CCI

TCP/IP

Windows 2000 z/OS



IBM WebSphere Application Server Network Deployment V6.0.2 with security enabled




Global security EnabledJava 2 security Not enabledUser registry Local OS

For details on enabling security in WebSphere, see the WebSphere Application Server V6 information center (section on Setting Up and Enabling Security).

8.1.2 Definitions checklistTable 8-2 lists the definitions that we used to configure the security scenarios.


Table 8-3 lists the user IDs that we used in our configuration.

Table 8-3 User ID checklist

8.2 Introduction to CICS and CICS TG security CICS uses the z/OS System Authorization Facility (SAF) to route authorization requests to an external security manager (ESM) to perform all its security



IP address (dhcp) 9.100.193.122 -

TCP/IP port 9080 11101 -

Job name - CITGS1G CITGS1CI

APPLID - - A6POTGS1

NETNAME - CITGS1G -

CONNECTION - - GS1G


CICS region user ID - CITGS1CI

Gateway daemon user ID CITGS1G -

Flowed user ID to which we want to permit access

- CITGSD

Flowed user ID to which we want to deny access

- CITGNW

Chapter 8. Gateway daemon security 283

checks. You could use any suitable ESM. However, because the RACF product from IBM is the most commonly used, we refer to it for the remainder of this book. For complete information about CICS security, refer to the CICS RACF Security Guide, SC33-1701.

Every CICS region requires certain special user IDs to be established and also uses certain user IDs when receiving inbound requests from other systems. These user IDs are as follows:

Region user ID This is the user ID under which the CICS started task runs.

Default user ID This is used when users do not explicitly sign on, and should be given very low authorization. It is specified in the SIT parameter DFLTUSER.

Flowed user ID This is any user ID that is flowed in an ISC or MRO request, and includes user IDs flowed in ECI and EPI requests from Java applications.

Link user ID This is a user ID associated with the connecting system. It can be defined in the SESSIONS definition. It is used in link security and to determine if connected systems are equivalent.

Authentication of CICS users and access control is normally determined by RACF. Once authenticated, the user passes through transaction and resource security checks. If the request has been forwarded from another system, then surrogate and intercommunication security checks are also performed.

The different types of CICS security are explained briefly in the sections that follow.

Transaction securityCICS uses transaction security to control a user’s permission to start a transaction. CICS performs a transaction security check even if no user has signed on. Users who do not sign on can use only those transactions that are authorized to the CICS default user ID. Usually, the transactions to which this user ID has access are very limited.

Resource securityCICS provides a further (optional) level of security by controlling access to individual resources, which include programs, files, and other resources. There are no special implications for resource security with the CICS TG and so this subject is not addressed any further in this chapter.


Surrogate user securityCICS performs surrogate user security checking to ensure that one user is authorized to act for another user. For example, a surrogate check is performed to ensure that a Gateway daemon is authorized to initiate work on behalf of a user ID flowed in an ECI request.

Intercommunication securityIntercommunication security in CICS is concerned with incoming requests for access to CICS resources. There are three fundamentally different intercommunication security checks that are performed when a Gateway daemon forwards a request to a CICS region:

� Bind security

This verifies that the system wishing to connect (bind) to CICS is authorized to do so. The Gateway daemon is given authority to bind to a CICS region.

� User security

This causes a check to be made against the flowed user ID when an inbound request attaches the transaction in CICS. This behavior is defined in the ATTACHSEC parameter on the CONNECTION definition. For EXCI requests from the Gateway daemon, this should always be IDENTIFY, meaning that the flowed user ID is identified but not verified (a password check is not done by CICS).

� Link security

This is an additional level of authorization checking that can apply to the attached transaction. A specific user ID (the link user ID) can be thought of as the user ID associated with the server which is passing the request to CICS. In our case, this is the user ID of the Gateway daemon started task. If link security is enabled, the link user ID, like the flowed user ID, must be authorized to access all transactions and resources invoked as a result of the request.

In this chapter, we discuss how these security checks are configured when using the Gateway daemon on z/OS.

For information about how these apply when securing connections from WebSphere Application Server for z/OS see Chapter 10, “Security with WebSphere Application Server for z/OS” on page 339.


8.3 Configuring CICS and CICS TG securityThe commands listed in the following sections are in addition to the basic configuration necessary for normal functioning of the CICS TG for z/OS. Before you implement any security in your environment, we recommend that you set up and test a non-secure environment. You can information about the following in Chapter 2, “Installing and configuring the CICS TG” on page 25:

� Setting up a user ID for the Gateway daemon started task� Defining programs and libraries as program controlled� Allowing the Gateway daemon access to program controlled libraries

The following steps are necessary to secure access to our CICS programs:

� Configuring CICS security� Configuring MRO bind time security� Enabling CICS TG password checking� Configuring security for CICS CONNECTION and SESSIONS definitions� Configuring link security� Configuring the flowed user ID� Permitting access to the mirror transaction� Configuring surrogate security

Configuring CICS securityOur CICS region CITGS1CI (with APPLID A6POTGS1) was configured with security prefixing and transaction security active using the following parameters:

� IRCSTRT=YES� SECPRFX=YES� SEC=YES� XDCT=NO � XFCT=NO � XPCT=NO� XPPT=NO� XTRAN=YES� XCMD=NO

Note: We used security prefixing (SECPRFX=YES) in our CICS region, which prevents our RACF security profiles from affecting other CICS regions. This parameter can be quite useful in a production environment, because it means all security profiles are unique to an individual region. However, conversely it can mean more work for the security administrator because more profiles must be defined.


Configuring MRO bind security MRO bind security prevents unauthorized attached MRO regions from starting transactions in a CICS region. It is implemented using two different DFHAPPL FACILITY class profiles. To connect to a CICS region, the Gateway daemon user ID requires the following permissions:

� Update access to the RACF FACILITY class profile DFHAPPL.<DFHJVPIPE>, where <DFHJVPIPE> is the EXCI pipe name as defined in the CICS TG environment variable DFHJVPIPE and in the NETNAME parameter in the CICS CONNECTION definition. If a generic EXCI connection is used, there is no pipe name and this does not apply.

� Read access to the RACF FACILITY class profile DFHAPPL.<APPLID>, where <APPLID> is the APPLID of the CICS region in question.

We activated MRO bind security for our configuration as follows:

1. We gave update access to the RACF FACILITY class profile DFHAPPL.<DFHJVPIPE> using the following RACF command:

RDEFINE FACILITY (DFHAPPL.CITGS1G) UACC(NONE)PERMIT DFHAPPL.CITGS1G CLASS(FACILITY) ID(CITGS1G) ACCESS(UPDATE)SETROPTS RACLIST(FACILITY) REFRESH

In our configuration, CITGS1G is the user ID of the Gateway daemon started task and also the name that we used for the DFHJVPIPE environment variable.

2. We gave read access to the RACF FACILITY class profile DFHAPPL.<APPLID> with the following RACF command:

RDEFINE FACILITY (DFHAPPL.A6POTGS1) UACC(NONE)PERMIT DFHAPPL.A6POTGS1 CLASS(FACILITY) ID(CITGS1G) ACCESS(READ)SETROPTS RACLIST(FACILITY) REFRESH

In our configuration, A6POTGS1 is the APPLID of the CICS region.

Note: Use of MRO bind-time security is optional and if these DFHAPPL profiles are not defined, then any MRO connected system will be able to connect to your CICS region.

Note: We defined all of our security profiles with a UACC(NONE), meaning a universal access of none. Thus, by default all users were denied access, and permissions were then granted for each user ID as required.

Tip: The SETROPTS command refreshes the FACILITY class. It must be run each time you modify the DFHAPPL FACILITY class profiles.


Enabling CICS TG password checking To enable the CICS TG to authenticate each user ID and password flowed on an ECI request, the environment variable AUTH_USERID_PASSWORD must be set in the CICS TG environment variables. We set this variable in our CITG.CTG610.STDENV(CITGS1G) PDS member (Example 8-1), which we supplied as input to our Gateway daemon using the STDENV DD card.

Example 8-1 CICS TG environment variables

_BPX_SHAREAS=YES CICSCLI=/cicstg/ctg610/config/CITGS1G.ini COLUMNS=80 CTG_JNI_TRACE=/cicstg/ctg610/trace/CITGS1G.jnitrcCTG_JNI_TRACE_ON=YES CTG_RRMNAME=CTG.RRM.CITGS1G.IBM.UA DFHJVPIPE=CITGS1G DFHJVSYSTEM_00=A6POTGS1-Primary server DFHJVSYSTEM_01=A6POTGS2-Secondary server PATH=/bin:/usr/lpp/java142/J1.4/bin TMPDIR=/cicstg/ctg610/trace TZ=GMT-2AUTH_USERID_PASSWORD=YESJAVA_PROPAGATE=NO

See 8.5.1, “Testing security with Java program EciB1” on page 296 for how we tested CICS TG password checking.

Notes:

� If you set AUTH_USERID_PASSWORD to YES you must also set _BPX_SHAREAS to YES and JAVA_PROPAGATE to NO.

� If you include AUTH_USERID_PASSWORD with any value, YES will be assumed. For any value other than YES, you get the following warning message:

CTG6134W Environment variable ‘AUTH_USERID_PASSWORD’ is set, but not to the value ‘YES’; user authentication will be enabled


CONNECTION and SESSIONS definitions In order for our CICS region to use the user ID flowed in the EXCI call from the Gateway daemon, we set the parameter ATTACHSEC=IDENTIFY in the EXCI connection definition in our CICS region, as shown in Figure 8-2.

Figure 8-2 CICS CONNECTION definition

IDENTIFY means that CICS uses the flowed user ID in the EXCI request, but does not expect a password to be flowed with the request because this should be checked by the Gateway daemon itself. The default is ATTACHSEC=LOCAL, which means that the flowed user ID is not supplied by the connecting server.

Configuring link securityThe link user ID is the user ID associated with the Gateway daemon which is passing the request to CICS. If link security is enabled, the link user ID, like the flowed user ID, must be authorized to access all transactions and resources invoked as a result of the request.

The link user ID can be preset on the EXCI SESSIONS definition using the USERID parameter. For EXCI requests, link security works as follows:

1. If the link user ID is the same as the CICS region user ID, then the systems are deemed equivalent and no link security authorization is performed.

OVERTYPE TO MODIFY CICS RELEASE = 0640 CEDA ALter CONnection( GS1G ) CONnection : GS1G Group : GS1G DEscription ==> CONNECTION IDENTIFIERS Netname ==> CITGS1G INDsys ==> CONNECTION PROPERTIES ACcessmethod ==> IRc Vtam | IRc | INdirect | Xm PRotocol ==> Exci Appc | Lu61 | Exci Conntype ==> Specific Generic | Specific SECURITY SEcurityname ==> ATtachsec ==> Identify Local | Identify | Verify | Persistent | Mixidpe BINDPassword : PASSWORD NOT SPECIFIED BINDSecurity ==> No No | Yes Usedfltuser ==> No No | Yes SYSID=TGS1 APPLID=A6POTGS1


2. If the link user ID is preset as something other than the CICS region user ID, then this user ID must be authorized to access all transactions and resources invoked as a result of the request.

3. If the link user ID is not preset, then the user ID of the connected region (that is, the user ID of the Gateway daemon started task) is the link user ID and this user ID must be authorized to access all transactions and resources invoked as a result of the request.

We decided to use preset link security. This has the advantage that it avoids the overhead of EXCI sign-on for each request that is passed to CICS. When running any application that connects to CICS through the EXCI (such as the Gateway daemon), CICS writes one DFHSN1400/DFHSN1500 message pair to the CICS joblog for each request. By default, this message pair is written for each ECI request. When the link user ID is preset, however, EXCI sign-on is done when the CICS CONNECTION is installed but not for each ECI request.

It is also possible to suppress EXCI sign-on messages by using the CICS user exit XMEOUT. An assembler sample of this program can be found in SDFHSAMP(DFH$SXP1).

To enable preset link security, we set the USERID parameter in the SESSIONS definition to be CITGS1G, which is the CICS TG started task user ID. Our SESSIONS definition is shown in Figure 8-3.

Figure 8-3 CICS SESSIONS definition

OVERTYPE TO MODIFY CICS RELEASE = 0640 CEDA ALter Sessions( GS1G ) Sessions : GS1G Group : GS1G DEscription ==> SESSION IDENTIFIERS Connection ==> GS1G SESSName ==> NETnameq ==> MOdename ==> SESSION PROPERTIES Protocol ==> Exci Appc | Lu61 | Exci MAximum ==> 000 , 000 0-999 RECEIVEPfx ==> 1 1-999 RECEIVECount ==> 260 PRESET SECURITY USERId ==> CITGS1G SYSID=TGS1 APPLID=A6POTGS1


We recommend that you specify ATTACHSEC=IDENTIFY and use non-equivalent systems so that security checks are also performed against a a link user ID.

Table 8-4 lists the different settings for link security and ATTACHSEC and how they interoperate.

Table 8-4 Attach security settings with an EXCI connection from the Gateway daemon

In our configuration, the link user ID CITGS1G is not equivalent to the CICS region user ID CITGS1CI and ATTACHSEC IDENTIFY is defined. The security checks that apply are therefore those listed in column 4 of Table 8-4.

Configuring the flowed user IDThe flowed user ID is the identity that is passed on the ECI request. It often represents the identity of the end user of the application. When using the base CICS TG classes, the user ID and password can be specified as parameters when constructing an ECIRequest object (see 8.5.1, “Testing security with Java program EciB1” on page 296). When using the JCA, the user ID and password can be specified in a variety of different ways (see 8.5.2, “Testing security with J2EE application CTGTesterCCI” on page 299). In our tests we flow the user ID CITGSD.

Because the CICS TG runs as a shell process under UNIX System Services, any user ID that it tries to authenticate with RACF, such as a user ID flowed with an ECI request, must have an OMVS segment defined in its RACF profile. See OS/390® UNIX System Services Planning, GA22-7800 for more information.

Permitting access to the mirror transactionsAn EXCI request received by CICS from the Gateway daemon attaches a mirror transaction. Therefore, we needed to authorize our flowed user ID CITGSD and our link user ID CITGS1G to our private mirror transaction ECIT. We issued the following RACF commands:

REDFINE TCICSTRN CITGS1CI.ECIT UACC(NONE)PERMIT CITGS1CI.ECIT CL(TCICSTRN) ID(CITGS1G) ACCESS(READ)PERMIT CITGS1CI.ECIT CL(TCICSTRN) ID(CITGSD) ACCESS(READ)SETROPTS RACLIST(TCICSTRN) REFRESH

Equivalent systems? Link user ID = CICS region user ID

Link user ID not = CICS region user ID

ATTACHSEC LOCAL IDENTIFY LOCAL IDENTIFY

Link user ID check NO NO YES YES

Flowed user ID check NO YES NO YES

User ID mirror transaction runs under in CICS

CICS default user ID

Flowed user ID Link user ID Flowed user ID


Configuring surrogate security In order for the EXCI to be able to switch the security context of the EXCI request to the flowed user ID, the correct SURROGAT security profiles must be defined. The user ID of the EXCI client (in our case the Gateway daemon) requires read access to the SURROGAT class profile <USERID>.DFHEXCI, where <USERID> is the flowed user ID in the EXCI request.

We issued the following commands to authorize our gateway daemon user ID CITGS1G to switch to the user ID CITGSD which is flowed in the ECI request:

RDEFINE SURROGAT CITGSD.DFHEXCI UACC(NONE) OWNER(CITGSD) PERMIT CITGSD.DFHEXCI CLASS(SURROGAT) ID(CITGS1G) ACCESS(READ)

It is possible to disable surrogate security by either reassembling the EXCI options table DFHXCOPT, with SURROGAT=NO, or by using a RACF surrogate profile with universal READ access such as:

RDEFINE SURROGAT *.DFHEXCI UACC(READ) OWNER(CITGSD)

Tip: For our examples, we permitted access to single users (CITGSD and CITGS1G). In a production environment, you might create a group of users who require common access. When you build a group, you permit access to a group so that user access can be controlled by the group to which a user belongs, rather than by individual permissions. This method simplifies the security definitions that are required.

Note: If you assemble DFHXCOPT into a new library you should ensure that it is defined in the environment variable EXCI_OPTIONS and also that this library is program controlled. If it is not program controlled you will get the following error message:

ICH420I PROGRAM DFHXCOPT FROM LIBRARY CITG.CITGS1G.USERLOAD CAUSED THE ENVIRONMENT TO BECOME UNCONTROLLED


8.4 Configuring JCA securityThe JCA has specific support for enabling secure access from a J2EE application to an Enterprise Information System (EIS) such as CICS. Both container-managed sign-on and component-managed sign-on are supported. When deploying a J2EE component such as an EJB, the deployer must set the res-auth element in the deployment descriptor for each resource reference in order to indicate which authentication method is being used.

� Container-managed security

If you use container-managed security, the J2EE application server is responsible for flowing security context to the EIS. You must set the res-auth deployment descriptor element to Container. The application deployer is then able to set up the authentication information (user ID and password) when he deploys the application.

In WebSphere Application Server V5, authentication data can be specified in the connection factory by using a JAAS authentication alias. In WebSphere Application Server V6, this should be done using a J2C authentication data entry mapped to specific application resources (see 8.4.1, “J2C authentication data” on page 294).

� Component-managed security

For component-managed security, the application is responsible for flowing security context to the EIS. You must set the res-auth deployment descriptor element to Application.

Note: JAAS authentication aliases are deprecated in WebSphere Application Server V6 and are supported for compatibility reasons.

Note We do not cover component-managed security in our JCA security test scenarios as we recommend that you leave security management to the container.


Within our application EAR file, the EJB archive CTGTesterCCIEJB.jar contains file META-INF/ejb-jar.xml. This deployment descriptor file contains a resource-ref for the ECI reference. We set the res-auth for this resource to Container (Example 8-2).

Example 8-2 EJB deployment descriptor res-auth setting

<resource-ref id="ResourceRef_1"><description>CICS ECI Resource adapter</description><res-ref-name>ECI</res-ref-name><res-type>javax.resource.cci.ConnectionFactory</res-type><res-auth>Container</res-auth>

</resource-ref>

8.4.1 J2C authentication dataJ2C authentication data entries are used by resource adapters such as JDBC data sources and the CICS ECI resource adapters. An authentication data entry includes the following information:

Alias The name of the authentication data entry. When configuring the resource adapter, the administrator can specify which authentication data to choose using the corresponding alias.

User ID A user identity known in the target user registry (in our case a RACF user ID).

Password The password of the user ID.

We created a J2C authentication data entry as follows:

1. In the Administrative Console Welcome panel, we clicked Global security. In the right-side panel of the screen, we selected JAAS configuration → J2C authentication data.

Note: The password is encoded in the WebSphere configuration repository.


2. We then clicked New and defined our authentication alias as shown in Figure 8-4.

Figure 8-4 J2C authentication alias

3. We clicked OK and saved the changes.

In 8.5.2, “Testing security with J2EE application CTGTesterCCI” on page 299, we show how we modified the resource authentication method for the ECI resource reference in our CTGTesterCCI application in order to use this authentication data entry.

Important: The authentication of user ID and password in CICS TG for z/OS is optional (based on the setting of the environment variable AUTH_USERID_PASSWORD). So even if you define a J2C authentication alias the password will only be verified by the Gateway daemon if AUTH_USERID_PASSWORD is set.


If your application requires a more dynamic authentication data mapping function, you can develop your own J2C mapping module. For example, the mapping module could use the WSSubject.getRunAsSubject method to retrieve the subject that represents the identity of the current running WebSphere thread (the identity of the current running thread is known as the RunAs identity). The mapping module is then able to specify that this user ID should be used for the EIS connection.

8.4.2 Setting user ID and password on the ECIConnectionSpec It is also possible for the application code itself to specify the user ID and password. We designed the CTGTesterCCI application in such a way that if user ID or password information is specified on the input screen it uses this information in an ECIConnectionSpec when it gets a connection from the connection pool. This is shown in Example 8-3.

Example 8-3 Setting user ID and password on ECIConnectionSpec

ECIConnectionSpec connSpec = new ECIConnectionSpec();connSpec.setUserName("CITGSD");connSpec.setPassword("PASSWRD");Connection conn = connfac.getConnection(connSpec);

8.5 Testing the configurationAfter we configured the different software components we tested our configuration with two different Windows application clients:

� The sample Java test application EciB1 supplied with the CICS TG� Our sample J2EE application CTGTesterCCI

8.5.1 Testing security with Java program EciB1The aim of this basic security scenario is to verify that CICS TG user ID password authentication is working correctly. Figure 8-5 shows the outline of our environment. The platform of the Java client in our scenario is not important for

Note: If container managed sign-on is being used and a J2C authentication alias has been defined, the user ID and password that is set in the J2C authentication alias overrides those that are set on the ECIConnectionSpec.


our test purposes and could have equally been deployed on any other platform (such as UNIX System Services or Linux).

Figure 8-5 CICS TG for z/OS security test environment - EciB1

In order to test our secure CICS TG for z/OS environment, we chose to use the EciB1 synchronous ECI sample. The runtime class for this is provided by the CICS TG in the ctgsamples.jar and the source is also provided in the \samples\java\com\ibm\ctg\samples\eci directory.

To test from our Windows system, it was necessary to have a Java Development Kit (JDK) installed on the system. We then copied the ctgclient.jar and ctgsamples.jar files to the client machine and set the classpath to contain these two JAR files.

Example 8-4 shows the results of testing EciB1 from a Windows workstation.

Example 8-4 Testing z/OS EXCI security with EciB1 sample

C:\>java com.ibm.ctg.samples.eci.EciB1 tcp://tx1.mop.ibm.com 11101

CICS TG Basic ECI Sample 1Usage: java com.ibm.ctg.samples.eci.EciB1 [Gateway Url] [Gateway Port Number]

To enable client tracing, run the sample with the following Java option: -Dgateway.T.trace=onThe address of the Gateway has been set to TCP://tx1.mop.ibm.com Port:11101

CICS Servers Defined: 1. A6POTGS1 -Primary server 2. A6POTGS2 -Secondary server

Choose Server to connect to, or q to quit:

Windows

Java application

EciB1 TCP/IP

z/OS

CICS TS V3.1

EC01

CITGS1CI

EXCI

Gatewaydaemon

CITGS1G

MRO/IRC

RACF

user ID

(user ID +password)

authenticationauthorization

11101

9.100.193.122


1

Enter your CICS User ID:CITGSD

Enter your CICS Password:elfr1ede

Program EC01 returned with data:- Hex: 32332f30392f30352031323a31333a31310 ASCII text: 23/09/05 12:13:11

EciB1 is written so that it makes a call to the specified CICS region initially without a user ID and password, and then if it receives a security error, it then prompts you for a user ID and password. It also allows trace to be activated by specifying -Dgateway.T.trace=on. This traces the CICS TG calls made in the Java application and is particularly useful in showing the user ID and COMMAREA flowed to and from the Gateway daemon.

When we ran EciB1 with trace on, we saw the input on the first ECI call to the Gateway daemon specifying A6POTGS1 as the CICS region. As seen in Example 8-5, the user ID at offset 57 is not set (low values), because we have not been prompted for it as yet.

Example 8-5 EciB1 object/COMMAREA on initial call (created with -Dgateway.T.trace=on)

S-C: CCL6603I: # 00000: 47 61 74 65 00 60 00 00 00 00 00 01 00 00 00 00 [email protected]: CCL6603I: # 00016: 00 00 00 00 00 00 00 03 45 43 49 00 00 00 00 60 ........ECI....`S-C: CCL6603I: # 00032: 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 ................S-C: CCL6603I: # 00048: 00 53 43 53 43 50 4A 41 34 00 00 00 00 00 00 00 .A6POTGS1.......S-C: CCL6603I: # 00064: 00 00 00 00 00 00 00 00 00 2A 2A 2A 2A 2A 2A 2A .........*******

In Example 8-6, we see the second ECI request to the Gateway daemon, after we had been prompted for a user ID and password. This time the user ID (CITGSD) is contained in the ECI request, and the password is masked (********) for security.

Example 8-6 EciB1 object/COMMAREA after ID and password input (created with -Dgateway.T.trace=on)

S-C: CCL6603I: # 00000: 47 61 74 65 00 40 00 00 00 00 00 01 00 00 00 00 [email protected]: CCL6603I: # 00016: 00 00 00 00 00 00 00 03 45 43 49 00 00 00 00 60 ........ECI....`S-C: CCL6603I: # 00032: 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 ................S-C: CCL6603I: # 00048: 00 53 43 53 43 50 4A 41 34 63 69 63 73 72 73 32 .A6POTGS1CITGSD.S-C: CCL6603I: # 00064: 00 00 00 00 00 00 00 00 00 2A 2A 2A 2A 2A 2A 2A .........*******


8.5.2 Testing security with J2EE application CTGTesterCCIThe aim of this security scenario is to flow a user ID and password from WebSphere Application Server for Windows to a CICS region via the Gateway daemon running on z/OS. Figure 8-6 shows the basic outline of our environment. For details of our WebSphere Application Server configuration see 8.4, “Configuring JCA security” on page 293. We test the setting of user ID and password information in the CTGTesterCCI application itself and we also show how we configured a J2C authentication data entry.

Figure 8-6 CICS TG for z/OS security test environment - CTGTesterCCI

We performed the following sequence of tests:

1. For the initial test of our configuration, we invoked the sample CTGTesterCCI J2EE application (Figure 8-7) to call the ECIPROG COBOL program in CICS. We specified a valid user ID password combination. ECIPROG returns a COMMAREA which contains the user ID under which the CICS task runs.

For further details about our sample application refer to Appendix A, “Sample applications” on page 361.

If user ID or password information is specified on the input screen, the CTGTesterCCI application uses this information in an ECIConnectionSpec when it gets a connection from the connection pool.

z/OS CICS TS V3.1

ECIPROG

CITGS1CI

EXCI

Gatewaydaemon

CITGS1G

MRO/IRC

RACF

user ID

TCP/IP

(user ID +password)

authentication authorization

Windows

EJBCTGTesterCCI


CICS TG V6.1


HTML

CCI


Figure 8-7 CTGTesterCCI input with user ID and password specified

Figure 8-8 shows the results of our test. The output from ECIPROG shows that the request ran successfully in our secure CICS region CITGS1CI, and that the CICS task ran under the user ID CITGSD.


Figure 8-8 CTGTesterCCI results with user ID and password specified

2. We then repeated the test but this time we specified an invalid password on the CTGTesterCCI input screen. This test failed, as shown in Example 8-7, because our Gateway daemon is enabled for user ID password authentication.

Example 8-7 STDERR error for invalid user ID and password from CTGTesterCCI

CTG6808E Authorize userid/password with RACF. User ID = CITGSD, Return code = -1, errno = 111, errno2 = 0X90C0000


3. For our final test, we wanted to use the J2C Authentication alias that we defined in Figure 8-4 on page 295.

We needed to modify the resource authentication method for the ECI resource reference in the CTGTesterCCI application. In the Administrative Console Welcome panel, we clicked Applications → Enterprise Applications, then CTGTesterCCI. In the right-side panel of the screen, we selected Map resource references to resources where we performed the following actions:

a. We selected the JNDI name eis/CITGS1CI-tx1-11101 for our existing connection factory.

b. We then scrolled down the page and selected the EJB module CTGTesterCCIEJB in our J2EE application.

c. We scrolled back up to the top of the page and clicked Apply.

d. We then checked Use default method as the authentication method and we selected our authentication alias CICS-alias as the authentication data entry to be used for this resource reference, and we clicked Apply.

The final result is shown in Figure 8-9. In the bottom panel, the resource reference ECI in the EJB CTGTesterEJBCCI is mapped to the JNDI name eis/CITGS1CI-tx1-11101, and that Cam21-Pc3Node01/CICS-alias is defined as the authentication data for this resource. Thus, any use of the resource reference ECI in our J2EE application now results in the user ID CITGSD and its defined password being passed to our Gateway daemon.


Figure 8-9 Administrative Console - mapping resource reference to authentication alias


We repeated our test, but this time we did not specify user ID and password on the CTGTesterCCI input screen (Figure 8-10).

Figure 8-10 CTGTesterCCI with no user ID and password specified


Figure 8-11 shows the results of this test. The CICS program again runs successfully with the user ID CITGSD although this time the authentication information has been taken from the J2C authentication alias.

Figure 8-11 CTGTesterCCI with no user ID and password specified

Note: If user ID password checking is not enabled for the Gateway daemon, it is normally necessary to establish a trust relationship between the WebSphere Application Server and the Gateway daemon so that the application server can be trusted to flow an already authenticated user ID.

Solutions such as SSL client authentication and virtual private networks (VPN) can be used to establish such a trust relationship. We show how we enabled SSL client authentication between WebSphere Application Server and the Gateway daemon in 9.4, “Configuring client authentication” on page 329.


8.6 Problem determinationIn this section, we document common security problems and information we learned while configuring our Gateway daemon security scenarios. For general problem determination guidance see 2.9, “Problem determination” on page 60.

8.6.1 Common security problemsSecurity errors are reported normally to the Java client with the return code -27(ECI_ERR_SECURITY_ERROR). These errors can be caused by several different situations, including those described in this section.

Password verification errors If Gateway daemon user ID authentication fails because the password is not valid, the Gateway daemon writes a message to STDERR (Example 8-8).

Example 8-8 STDERR error for user ID password verification failure

CTG6808E Authorize userid/password with RACF. User ID = CITGNW, Return code = -1, errno = 111, errno2 = 0X90C0000

If user ID authentication fails because no user ID is specified, the Gateway daemon writes a similar message to STDERR (Example 8-9).

Example 8-9 STDERR error for user ID not specified failure

CTG6808E Authorize userid/password with RACF. User ID = , Return code = -2, errno = 0, errno2 = 0

Surrogate security errors If the Gateway daemon does not have the required access to flow a specific user ID (CITGNW) to CICS, you will receive the RACF error shown in Figure 8-10.

Example 8-10 RACF message of Surrogate check failure for Gateway daemon

ICH408I USER(CITGS1G) GROUP(CITG) NAME(CITG CTG GATEWAY DAEMON ) CITGNW.DFHEXCI CL(SURROGAT) INSUFFICIENT ACCESS AUTHORITY ACCESS INTENT(READ ) ACCESS ALLOWED(NONE )

The request has failed because the Gateway daemon user ID CITGS1G does not have access to the SURROGAT class profile CITGNW.DFHEXCI.


Bind security errors If the Gateway daemon does not have the required access to bind to the CICS connection, you will see the RACF error shown in Figure 8-11 in the syslog.

Example 8-11 RACF message of MRO bind security failure - DFHAPPL.<DFHJVPIPE>

ICH408I USER(CITGS1G ) GROUP(CITG ) NAME(CITG CTG GATEWAY DAEMON) DFHAPPL.CITGS1G CL(FACILITY) INSUFFICIENT ACCESS AUTHORITY ACCESS INTENT(UPDATE ) ACCESS ALLOWED(NONE )

If the Gateway daemon does not have the required access to bind to the CICS region, you will see the RACF error shown in Figure 8-12.

Example 8-12 RACF message of MRO bind security failure - DFHAPPL.<APPLID>

ICH408I USER(CITGS1G ) GROUP(CITG ) NAME(CITG CTG GATEWAY DAEMON) DFHAPPL.A6POTGS1 CL(FACILITY) INSUFFICIENT ACCESS AUTHORITY ACCESS INTENT(READ ) ACCESS ALLOWED(NONE )

If you do not see RACF messages but you still suspect an MRO bind security problem, you should enable JNI trace. A JNI trace for an MRO bind security check is shown in Example 8-13.

Example 8-13 JNI trace of MRO bind security failure - DFHAPPL.<APPLID>

EXCI function error. Function Call = 3, Response = 16, Reason = 609, Subreason field-1 = 0x10800b0, subreason field-2 = 0x00, Cics_Rc = -9

Security attach failures for the mirror transactionIf the user ID that is flowed in the ECI request does not have READ access to the TCICSTRN class profile that controls access to the specified mirror transaction, then return code -27 (ECI_ERR_SECURITY_ERROR) is reported to the Java client. If this is the case, then error message ICH408I is also written to the CICS job log. For details on configuring the required profiles, refer to “Permitting access to the mirror transactions” on page 291.


8.6.2 Tips and utilitiesIn this section, we discuss additional tips and utilities for problem determination of security related problems. For general tips and information about standard utilities, such as the various Gateway daemon traces, see 2.9.2, “Tips and utilities” on page 62.

Because there are several software tiers involved in this configuration, you will find security error messages in different locations:

CICS MSGUSR DD This is where to look for CICS security errors, for example, if the link or flowed user ID does not have authority to execute the mirror transaction.

JES SYSLOG General RACF security errors are written here. Search for the message ICH408.

Gateway daemon trace This provides an indication of the errors the Gateway daemon itself encounters, for example, password verification errors.

JNI trace This provides tracing of the EXCI calls made by the Gateway daemon JNI module libctgjni.so, and can be very useful in problem determination of security problems.


Chapter 9. Enabling SSL with the Gateway daemon

In this chapter, we outline the support that is provided by the CICS TG for z/OS for encrypted Secure Sockets Layer (SSL) connections. We show how we implemented SSL connections, using the Java Secure Sockets Extension (JSSE) SSL protocol handler, from remote Java client applications.

We tested the configuration using a RACF generated key ring and certificates on the z/OS server machine and JSSE generated keystores and certificates on the client workstation.

We provide details about:

� The creation of key rings and certificates using RACF� Cipher suites� Configuring server authentication� Configuring client authentication

Finally, we explain how we tested the environment using the CICS TG sample ECI application EciB1 and our own sample J2EE application CTGTesterCCI.

9


9.1 PreparationIn our testing, we connected from Java client applications running on WebSphere Application Server for Windows to CICS via the Gateway daemon on z/OS. We concentrate on how to configure SSL connections between the Java client applications and the Gateway daemon (Figure 9-1).

Figure 9-1 Software components: Gateway daemon SSL scenarios

For more information about how to enable non-SSL connections to the Gateway daemon, see Chapter 3, “Gateway daemon connections” on page 77. For information about other Gateway daemon security features, see Chapter 8, “Gateway daemon security” on page 281.



z/OS

CICS TS V3.1

CICS program

Windows

Java application

EciB1

EXCIGatewaydaemon

CICS TG V6.1

MRO/IRC

RACF

SSL

SSL server authentication

EJBCTGTesterCCI


CICS TG V6.1


HTML

CCI

SSL client authentication

Server certificate Server CA's

certificateClient certificates

Windows 2000 z/OS





JSSE V1.42 JSSE V1.42

SSL V3 Protocol





9.2 Secure Sockets LayerSecure Sockets Layer (SSL) is a protocol designed to create a secure connection to a server using public key encryption, and to protect the data integrity and privacy as the data is transferred over the connection. CICS TG supports the Java Secure Socket Extension (JSSE) implementation of SSL which provides 128 bit and 256 bit encryption. JSSE is supplied as part of the IBM SDK for z/OS, Java 2 Technology Edition, V1.4.2 SR2.

With previous versions of CICS TG you were able to use SystemSSL which is specific to z/OS and the Java-based SSLight. JSSE is the only supported option for providing SSL with CICS TG V6 and later releases. If migrating from V5 to V6 see 9.5.1, “SystemSSL to JSSE migration” on page 335.

The SSL Handshake Protocol consists of two phases:

� Server authentication

In the first phase, the server responds to a client’s request by sending its certificate and cipher preferences. The client then generates a master key which it encrypts with the server’s public key and then transmits the encrypted master key to the server. The server authenticates itself to the client by returning a message authenticated with keys derived from the master key.



IP address (dhcp) 9.100.193.122 -

SSL port 8059 -

Job name CITGS1G CITGS1CI

APPLID A6POTGS1

NETNAME CITGS1G -

CONNECTION GS1G

Important: SystemSSL and SSLight are not supported by CICS TG V6 and later releases.

Chapter 9. Enabling SSL with the Gateway daemon 311

Subsequent data is encrypted and authenticated with keys derived from the master key.

� Client authentication

In the second optional phase, the server requests that a client identifies itself during the SSL handshake by providing its client certificate. Client authentication can only be requested by the server.

When setting up an SSL environment you have a choice between self-signed certificates and CA signed certificates:

� Self-signed certificates

A self-signed certificate is an identity certificate that is signed by its creator, so the creator is verifying that the certificate is valid.

� Certificating authority (CA) signed certificates

CA signed certificates are created by a user organization and sent to a CA to be signed. Before signing a certificate the CA will verify that the organization requesting the certificate is actually who they claim to be, so the CA is verifying that the certificate is valid.

In the examples in this chapter, we use self-signed certificates as they are more easily and quickly generated. In a production environment, it is expected that CA signed certificates are used because they provide a more secure solution.

9.2.1 JSSEJava Secure Sockets Extension (JSSE) is a Java implementation of SSL common across all platforms. JSSE implements SSL 3.0 and TLS 1.0 (Transport Layer Security) algorithm types as Java 2 standard extensions. It provides the socket factories that allow for ease of programming, if the provided defaults are taken.

JSSE is now the strategic SSL implementation for CICS TG and supports the following:

1. Cryptographic handshakes on z/OS using an IBMJCE4758 Peripheral Component Interconnect (PCI) card.

2. Management tools to generate keystores and manage certificates:

keytool a command line interface for maintaining keystoreshwkeytool a command line interface for maintaining hardware keysikeyman a GUI interface for maintaining keystores

Note: CICS TG supports both server and client authentication.


3. The ability to store digital certificates in a RACF database.

9.2.2 Cipher suitesIBM SDK 1.4.2 JSSE supports a wide range of cipher suites. Example 9-1 provides an explanation of the cipher suite naming convention.

Example 9-1 Cipher suite components

SSL_RSA_WITH_RC4_128_SHA

RSA is a type of key exchange cipherRC4_128 is the cipher for data encryptionSHA is the hashing algorithm

The key exchange cipher is used for the initial handshake. The most widely used key exchange cipher is RSA. This type of cipher is referred to as asymmetric as it uses public and private keys.

The cipher for data encryption (also known as the data encryption cipher) is used to secure the data which is passed over the SSL connection after the handshake has completed. Possible data encryption ciphers are AES, DES, Triple DES, RC2 and RC4 (which have 40,128 and 256-bit variants). These ciphers are referred to as symmetric as they use a single key for encryption and decryption. Symmetric key or secret key algorithms can be further classified as either block ciphers or stream ciphers. Block ciphers (such as DES) operate on the data in blocks while stream ciphers (such as RC4) operate on the data one bit at a time.

The hashing algorithm is a one way method of producing a hash number from a message. A slight change in the message produces a completely different hash number (a 128 bit fingerprint) and is used to ensure a message has not been tampered with. The client and the server both generate a hash number from the data transmitted using the hashing algorithm. If the hash numbers do not match the integrity of the message has been compromised. Data integrity is provided by one of the following hashing algorithms (also known as message digest algorithms); SHA1, SHA2, SHA3, SHA5, MD2 and MD5. Cipher suite names ending in _SHA indicate that the SHA1 hashing algorithm is being used.

SSL can use signature algorithms SHA1 with RSA, SHA1 with DSA, MD5 with RSA and MD2 with RSA.

Restriction: At this time CICS TG does not support Transport Layer Security (TLS).


9.2.3 Hardware cryptographyYou can use hardware cryptography with the CICS TG to offload the SSL handshake to a cryptographic device. This provides extra security and improved performance.

9.2.4 The hwkeytool commandYou can use the hwkeytool command that is provided as part of the IBM Java SDK in much the same way that we used the keytool command to create a keystore and manage certificates (see 9.3.1, “Creating a keystore on z/OS using keytool” on page 316). Extra parameters are available to specify how the key is stored on the cryptographic device and how it is to be used.

Example 9-2 shows an example of hwkeytool.

Example 9-2 hwkeytool

hwkeytool -genkey -alias server1 -keyalg RSA -storetype JCE4785KS -dname “cn=tx1.mop.ibm.com,o=IBM,c=FR” -keystore ctgv61.server.HWkeystore -keypass antigone -storepass antigone -hardwaretype CLEAR -hardwareusage KEYMANAGEMENT

Tips: CICS TG V6 supports a range of different cipher suites and allows you to specify which ciphers to use. When deciding this you should consider the following performance criteria.

1. You can significantly reduce the number of SSL handshakes when connecting from WebSphere Application Server by using JCA connection pooling.

2. RSA encryption and decryption processing during the SSL handshake can be off-loaded using crypto engines (PCICA, PCIXCC or CEX2C).

3. Larger keys (512/1024/2048) for key exchange ciphers have a higher handshake cost.

4. Choose appropriate bulk data ciphers. It is generally accepted that:

– MD5 is faster than SHA but is less secure– RC4 normally has a lower CPU cost than DES in software – RC4 stronger ciphers (128-bit versus 40-bit) do not require more CPU

cycles– SHA, DES and AES can often be implemented efficiently in hardware

Note: At present JSSE supports hardware cryptography only for handshakes and not for data encryption.


The additional parameters used with hwkeytool are:

-hardwaretype The type of key pair that will be generated, this is either CLEAR (the default), PKDS (stored in the Public Key Data set and encrypted with the co-processor master key) or RETAINED (not recommended for z/OS).

-hardwareusage This parameter sets the usage of the key pair being generated. Possible options are KEYMANAGEMENT (default for RSA) or SIGNATURE (default for DSA).

-hardwarekey Specifies that the key pair is to be deleted from the hardware storage as well as from the keystore. The default is to delete from the keystore only.

9.2.5 Java configuration for using hardware cryptographyJava Cryptography Extension (JCE) 1.2.1 provides a framework and implementations for encryption, key generation, key agreement and Message Authentication Code (MAC) algorithms. The IBMJCE4758 implementation extends JCE to add hardware cryptography functions using the IBM Common Cryptographic Architecture (CCA) interfaces.

Tips:

� A keystore created by hwkeytool must contain a personal certificate and the CA certificate used to sign it. If the personal certificate is self-signed (generated with the -selfcert parameter) first export the certificate and then import it into the same keystore with a different alias. If you are warned when you are importing the certificate back into the keystore that it already exists, type Y to confirm that you want to import it.

� Secure key processing (where the key is stored in the PKDS) offers greater security over clear key processing by hiding sensitive key data but can require additional hardware and result in a processing overhead.

� For further information about hardware cryptography, we recommend the following:

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/PRS1535http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100647


http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/PRS1535

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100647

To use the hardware cryptographic functions of IBMJCE4758 and hwkeytool, take the following steps:

1. Update the java.security file located in the /<java-home>/lib/security directory so that it contains the following lines:

security.provider.1=com.ibm.crypto.hdwrCCA.provider.IBMJCE4758security.provider.2=com.ibm.crypto.provider.IBMJCE

2. Copy the unrestricted policy files from /<java-home>/demo/jce/policy-files/unrestricted to /<java-home>/lib/security.

9.3 Configuring server authenticationIn this section, we describe how we:

1. Created a keystore on z/OS using the Unix System Services (USS) shell keytool command.

2. Created a key ring and certificates on z/OS using RACF and used the ikeyman tool on Windows to create a keystore and to import a server certificate.

3. Configured CICS TG to support SSL.

4. Tested our configuration.

9.3.1 Creating a keystore on z/OS using keytoolIn this section, we describe how we created a keystore and the certificates required for server authentication using the keytool command.

Note: After you have configured your JVM to be able to use hardware cryptography, you will not be able to use keytool. If you wish to revert back to using keytool, you need to remove the line that references IBMJCE4758 from the java.security file.

Note: We show how we created a keystore on z/OS using the keytool command for completeness. However, we recommend the use of RACF for storing keys and certificates (see 9.3.2, “Creating a key ring and certificates on z/OS using RACF” on page 319).


A keystore can be created and a self-signed certificate generated on z/OS using the USS shell keytool command as shown in Example 9-3.

Example 9-3 Keytool command to create a keystore and generate a certificate

keytool -genkey -alias server -keysize 1024 -dname “cn=tx1.mop.ibm.com,o=IBM,c=FR” -keystore ctgv61.server.jks -keypass antigone -storepass antigone -keyalg RSA

The options shown with the keytool command are as follows:

-genkey Generates a key pair and puts the public key into a self-signed certificate.

-alias Specifies the unique name that is used to refer to the key pair and certificate within the keystore.

-dname Specifies the X.500 distinguished name to be associated with the alias. This is used as the issuer and subject fields of the self-signed certificate. The distinguished name consists of a number of fields separated by commas.

-keystore The keystore location.

-keypass The password used to protect the private key.

-storepass The password used to protect the keystore.

-keyalg Specifies the algorithm to be used to generate the key pair.

We used the keytool command with the -list and -v parameters to check the contents of the certificate (Example 9-4).

Example 9-4 Keytool -list command

keytool -list -v -keystore ctgv61.server.jks -storepass antigone

Example 9-5 shows the results of the keytool -list command.

Example 9-5 Output from keytool -list -v command

Keystore type: jks Keystore provider: IBMJCE Your keystore contains 1 entry Alias name: server Creation date: Sep 20, 2005

Note: We found that -keypass and -storepass must be the same in order for the keystore to be used successfully by the Gateway daemon.


Entry type: keyEntry Certificate chain length: 1 Certificate[1]: Owner: CN=tx1.mop.ibm.com, O=IBM, C=FR Issuer: CN=tx1.mop.ibm.com, O=IBM, C=FR Serial number: 43303dec Valid from: 9/20/05 6:50 PM until: 12/19/05 6:50 PM Certificate fingerprints: MD5: 19:2E:2C:78:E8:31:D9:BE:EC:21:6F:BC:FC:8F:80:2F SHA1: 5A:E0:04:14:ED:69:1A:12:F6:D0:19:CA:B7:CA:7B:65:E6:E1:D8:E1

The next step was to export the self-signed certificate from the server keystore to a binary file named servercert.der. Example 9-6 shows the keytool -export command was used to do this.

Example 9-6 Keytool -export command

keytool -export -alias server -keystore ctgv61.server.jks -storepass antigone -keypass antigone -file servercert.der

After successfully processing this command, keytool responds with the following message:

Certificate stored in file <servercert.der>

Configuring WindowsThe exported certificate was transferred to the client workstation using FTP (in binary mode). We then imported the certificate servercert.der into a client keystore ctgv61.client2.jks using the command shown in Example 9-7.

Example 9-7 Import of a server certificate using keytool on Windows

keytool -import -alias server -keystore ctgv61.client2.jks -storepass antigone -keypass antigone -file servercert.der

Example 9-8 shows the output from the keytool command.

Example 9-8 Output from a keytool import command on Windows

Owner: CN=tx1.mop.ibm.com, O=IBM, C=FRIssuer: CN=tx1.mop.ibm.com, O=IBM, C=FRSerial number: 43303decValid from: 9/20/05 6:50 PM until: 12/19/05 5:50 PMCertificate fingerprints: MD5: 19:2E:2C:78:E8:31:D9:BE:EC:21:6F:BC:FC:8F:80:2F SHA1: 5A:E0:04:14:ED:69:1A:12:F6:D0:19:CA:B7:CA:7B:65:E6:E1:D8:E1Trust this certificate? [no]: yCertificate was added to keystore


9.3.2 Creating a key ring and certificates on z/OS using RACFIn this section, we describe how we created a key ring and the certificates that are required for server authentication using RACF:

1. To create a self-signed Certificate Authority (CA) certificate, we used the following RACDCERT command:

RACDCERT CERTAUTH GENCERT SUBJECTSDN(OU(‘CTG V61 Local CA’) O(‘IBM’) C’FR’)) KEYUSAGE(CERTSIGN) WITHLABEL(‘CTG V61 CA’)

2. We then created a personal certificate signed by the CA certificate generated in the previous step:

RACDCERT ID(CITGS1G) GENCERT SUBJECTSDN(OU(‘CTG V61 Personal’) O(‘IBM’) C(‘FR’)) WITHLABEL(‘CTG V61 Personal’) SIGNWITH(CERTAUTH LABEL(‘CTG V61 CA’))

3. We created a key ring (CTGV61ring) using the command:

RACDCERT ADDRING(CTGV61ring) ID(CITGS1G)

4. We used the following commands to put the CA and personal certificates in the key ring:

RACDCERT ID(CITGS1G) CONNECT(CERTAUTH LABEL(‘CTG V61 CA’) RING(CTGV61ring) USAGE(CERTAUTH))

RACDCERT ID(CITGS1G) CONNECT(LABEL(‘CTG V61 Personal’) RING(CTGV61ring) DEFAULT USAGE(PERSONAL)

5. To complete this part of the configuration, we exported the CA certificate in Base64 X.509 format using the following RACDCERT command:

RACDCERT CERTAUTH EXPORT(LABEL(‘CTG V61 CA’)) DSN(CITGSD.CERTAUTH.X509) FORMAT(CERTB64)

Attention: When creating your key ring ensure that the ID parameter specified on the RACDCERT ADDRING command is the user ID used for the Gateway daemon started task (CITGS1G in our case).

Note: The RACF RACDCERT command can also be used for generating a key ring and certificates to be used by a cryptographic device. For more information, see The z/OS Security Server (RACF) Command Reference, SA22-7687.


Configuring WindowsWe used FTP as shown in Example 9-9 to send the exported certificate (in ASCII format) to our client machine.

Example 9-9 How to use FTP to send an exported CA certificate to a client workstation

C:\SDA_CTG_V61>ftp tx1.mop.ibm.comConnected to tx1.mop.ibm.com.220-FTPD11 IBM FTP CS V1R6 at TX1.MOP.IBM.COM, 07:49:41 on 2005-09-27.220 Connection will close if idle for more than 5 minutes.User (tx1.mop.ibm.com:(none)): citgsd331 Send password please.Password:230 CITGSD is logged on. Working directory is "CITGSD.".ftp> ascii200 Representation type is Ascii NonPrintftp> get "CERTAUTH.X509"200 Port request OK.125 Sending data set CITGSD.CERTAUTH.X509250 Transfer completed successfully.ftp: 898 bytes received in 0.00Seconds 898000.00Kbytes/sec.ftp> quit221 Quit command received. Goodbye.

After completing the FTP transfer we continued the SSL configuration on our Windows workstation. This can be done either with keytool or with ikeyman (the IBM key management tool). Both of these tools are provided with the IBM Java SDK. On our workstation, these tools were found in the directory:

C:\Program Files\IBM\Java142\jre\bin

After opening a command prompt window we set our PATH variable and typed ikeyman to invoke the iKeyMan GUI:

set PATH=C:\Program Files\IBM\Java142\jre\bin\;%PATH%ikeyman


We used iKeyMan to create a keystore called ctgv61.client.jks (with password antigone), and we then added the CERTAUTH.X509 file to the keystore as shown in Figure 9-2.

Figure 9-2 iKeyMan - Add a CA certificate to the keystore from a X509 file


We clicked OK to proceed, and we were then prompted to enter a label for the certificate. Figure 9-3 shows our CA certificate in the client workstation keystore.

Figure 9-3 iKeyMan - After adding CA certificate


See 9.3.4, “Testing server authentication” on page 325 for information about how we tested SSL server authentication from a java client using the ctgv61.client.jks keystore.

9.3.3 Configuring CICS TGIn order to use the RACF key ring that we created in 9.3.2, “Creating a key ring and certificates on z/OS using RACF” on page 319, we needed to configure our Gateway daemon to use SSL. We specified the keyring=<RACF keyring name> and esmkeyring parameters in the SSL protocol parameter list used by our Gateway daemon. Example 9-10 shows the SSL parameter list in the Gateway configuration file (CITGS1G.ini).

Example 9-10 SSL protocol parameters for using a RACF key ring

[email protected][email protected]=port=8059;\ connecttimeout=2000;\ idletimeout=600000;\ pingfrequency=60000;\ clientauth=off;\ keyring=CTGV61ring;\ esmkeyring;

Limiting available cipher suitesThe cipher suites which are available for use by CICS TG can be limited by using the ciphersuites parameter. The protocol definition to limit SSL to one cipher suite (SSL_RSA_WITH_RC4_128_SHA) is shown in Example 9-11.

Example 9-11 Ciphersuites parameter of SSL protocol

[email protected]=com.ibm.ctg.server.SslHandler [email protected]=port=8059;\ connecttimeout=2000;\ idletimeout=600000;\ pingfrequency=60000;\ clientauth=off;\ ciphersuites=SSL_RSA_WITH_RC4_128_SHA;\ keyring=CTGV61ring;\ esmkeyring;

Tip: We found that the esmkeyring parameter must be specified as the last parameter. If this was not the case, SSL initialization failed with the following error message:

CTG6525E Unable to start handler for the ssl: protocol. [java.lang.IllegalArgumentException: keyring=]


During startup, the Gateway daemon writes a CTG8401I message listing which ciphers are enabled. Cipher suite negotiation then takes place during the SSL handshake phase. At this point both sides of the connection agree on a cipher suite that will be used. The Gateway daemon writes a connection message showing which cipher suite is being used (Example 9-12).

Example 9-12 Gateway daemon message confirming cipher suite usage

19:12:46:516 ssl: S-C: Client socket[addr=9.100.199.140/9.100.199.140,port=1099,localport=8059] connected using cipher suite SSL_RSA_WITH_RC4_128_SHA

If you do not use the ciphersuites parameter to limit the number of cipher suites the content of the CTG8401I message written at gateway daemon startup will be as shown in Example 9-13.

Example 9-13 CTG8401I message when the ciphersuites parameter is not used

CTG8401I The following ciphers are enabled: SSL_RSA_WITH_RC4_128_MD5 SSL_RSA_WITH_RC4_128_SHA SSL_RSA_WITH_AES_128_CBC_SHA SSL_RSA_WITH_AES_256_CBC_SHA SSL_RSA_WITH_DES_CBC_SHA SSL_RSA_FIPS_WITH_DES_CBC_SHA SSL_RSA_WITH_3DES_EDE_CBC_SHA SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA SSL_DHE_RSA_WITH_AES_128_CBC_SHA SSL_DHE_RSA_WITH_AES_256_CBC_SHA SSL_DHE_RSA_WITH_DES_CBC_SHA SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA SSL_DHE_DSS_WITH_AES_128_CBC_SHA SSL_DHE_DSS_WITH_AES_256_CBC_SHA SSL_DHE_DSS_WITH_RC4_128_SHA SSL_DHE_DSS_WITH_DES_CBC_SHA SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA SSL_RSA_EXPORT_WITH_RC4_40_MD5 SSL_RSA_EXPORT_WITH_DES40_CBC_SHA SSL_RSA_EXPORT_WITH_RC2_CBC_40_MD5 SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA SSL_RSA_WITH_NULL_MD5

Note: The parameter ciphersuites=128 bitonly is obsolete. If this parameter is coded, it is replaced by a ciphersuites setting of SSL_RSA_WITH_RC4_128_MD5,SSL_RSA_WITH_RC4_128_SHA,SSL_DHE_DSS_WITH_RC4_128_SHA.


SSL_RSA_WITH_NULL_SHA SSL_DH_anon_WITH_AES_128_CBC_SHA SSL_DH_anon_WITH_AES_256_CBC_SHA SSL_DH_anon_WITH_RC4_128_MD5 SSL_DH_anon_WITH_DES_CBC_SHA SSL_DH_anon_WITH_3DES_EDE_CBC_SHA SSL_DH_anon_EXPORT_WITH_RC4_40_MD5 SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA

Of the SSL cipher suites that are listed in message CTG8401I, the following considerations apply:

� The SSL_DHE_xxx cipher suites require the use of a DSA encrypted key so key rings generated with RSA will not work. To use DSA to generate the key use the keytool command with the -keyalg DSA option.

� By default the Java Cryptography Extension (JCE) is shipped with limited strength ciphers. To use 256-bit Advanced Encryption Standard (AES) encryption algorithms you must apply unlimited jurisdiction policy files. These are placed in the <java_home>/lib/security directory.

� Although the CTG8401I message lists anonymous ciphers as being enabled, the SSL_DH_anon_xxx cipher suites will not work because the JSSE trust manager does not support anonymous connections. If you try to use them you will get a handshake failure.

9.3.4 Testing server authenticationAfter we created the RACF key ring and configured SSL support for our Gateway daemon we tested our configuration with two different Windows application clients:

� The sample Java test application EciB1 supplied with the CICS TG

� Our sample J2EE application CTGTesterCCI deployed on WebSphere Application Server (see “Testing SSL server authentication with CTGTesterCCI” on page 327)

Note: Details on upgrading the security policy file for the IBM SDK can be found at the following Web site:

http://www.ibm.com/developerworks/java/jdk/security/142/


http://www.ibm.com/developerworks/java/jdk/security/142/

Testing SSL server authentication with Java program EciB1To test our SSL setup we used the Java client application program EciB1 which is contained in the jar file ctgsamples.jar. We ran EciB1 with the SSL Keyring and SSL Password options, the command and output are shown in Example 9-14.

Example 9-14 Command to execute EciB1 and the output produced

C:\SDA_CTG_V61>java com.ibm.ctg.samples.eci.EciB1 ssl://tx1.mop.ibm.com 8059 ctgv61.client.jks antigone

CICS Transaction Gateway Basic ECI Sample 1

Usage: java com.ibm.ctg.samples.eci.EciB1 [Gateway URL] [Gateway Port Number] [SSL Keyring] [SSL Password]

To enable client tracing, run the sample with the following Java option: -Dgateway.T.trace=on

The address of the Gateway has been set to ssl://tx1.mop.ibm.com Port:8059

CICS Servers Defined:

1. A6POTGS1 -Primary server 2. A6POTGS2 -Secondary server

Choose Server to connect to, or q to quit:1

Enter your CICS User ID:citgsd

Enter your CICS Password:elfr1ede

Program EC01 returned with data:-

Hex: 30342f31302f30352031323a33363a35330 ASCII text: 04/10/05 12:36:53

Because we had specified connectionlogging=on for the Gateway daemon, we found confirmation that SSL was being used in the Gateway daemon STDOUT


log (Example 9-15). Note that the connection is to the Gateway daemon SSL port 8059.

Example 9-15 CICS TG log showing a connection to the Gateway daemon port 8059

10/11/05 12:32:09:714 [0] CTG6506I Client connected. [ConnectionManager-0] - ssl@Socket[addr=9.100.199.140/9.100.199.140,port=3041,localport=8059]

Testing SSL server authentication with CTGTesterCCITo demonstrate the use of SSL between WebSphere Application Server for Windows and the Gateway daemon we repeated our test scenario with the CTGTesterCCI J2EE application.

JCA connection pooling with SSL connectionsWith SSL security enabled there is an additional performance overhead incurred by the security processing, in particular, during the initial handshake negotiation. If JCA connection pooling is used this overhead can be significantly reduced (see 3.2.2, “Creating connection factories” on page 82 for details on connection pooling).

Because an SSL connection takes longer to set up than a non-SSL connection, the connection pool timeout settings should be reviewed at the time that SSL is being implemented. Ideally the connection pool should configured such that a minimum number of connections are terminated and re-established during normal operation. With SSL connections it might be preferable to have longer timeout values resulting in unused connections persisting longer rather than incurring the cost of starting new connections.

Enabling an SSL connection from WebSphere Application ServerWe needed to make the CERTAUTH.X509 certificate (created in 9.3.2, “Creating a key ring and certificates on z/OS using RACF” on page 319) available to our WebSphere Application Server. To do this we:

1. Transferred the CERTAUTH.X509 server certificate to the WebSphere Application Server machine in ASCII mode using FTP.

2. Imported the CERTAUTH.X509 certificate to the WebSphere Application Server DummyServerKeyFile.jks using iKeyMan. We used the same process that we describe in “Configuring Windows” on page 320.

Important: JCA connection pooling significantly reduces the performance overhead of SSL connections.


In the WebSphere Administration Console in Resource Adapters → ECIResourceAdapter → J2C connection factories → CITGS1CI-tx1-11101 → Custom properties, we updated the following:

� ConnectionURL to specify the SSL protocol� KeyRingClass to point to the DummyServerKeyFile.jks� KeyRingPassword to be WebAS� PortNumber to be the Gateway daemon’s SSL port, 8059

The connection factory custom properties are displayed in Figure 9-4.

Figure 9-4 Administrative Console - SSL connection factory custom properties

We stopped and restarted WebSphere Application Server to pick up the changes to the connection factory and tested with the CTGTesterCCI application as shown in 8.5.2, “Testing security with J2EE application CTGTesterCCI” on page 299.

Because we had specified connectionlogging=on for the Gateway daemon, we were able to confirm that SSL was being used by checking the connection message in the Gateway daemon STDOUT log (see Example 9-15 on page 327).

Important: We used the WebSphere Application Server default keystore for our tests. However, never use this dummy file in a production environment or any time that sensitive data is being transmitted.


9.4 Configuring client authenticationIn client authentication the server sends a request to the client to identify itself. The client authenticates itself by returning the client’s digital signature and its public-key certificate. In this section we describe:

1. How we created a self-signed client certificate2. How we exported the client personal certificate3. CICS TG configuration for client authentication4. How we tested our configuration

9.4.1 Creating the client certificate on a Windows client machineThe first step to set up client authentication is to create a client certificate. We did this using the iKeyMan option Create New Self-Signed Certificate as shown in Figure 9-5.

Figure 9-5 iKeyMan - Create a self-signed client certificate


9.4.2 Extracting a client personal certificateAfter creating the new self-signed client certificate we used the iKeyMan extract option to extract the certificate to a Base64 ASCII file (Figure 9-6).

Figure 9-6 iKeyMan - extract client certificate to a file

After extracting the client certificate to file clientcr.arm, we transferred it by FTP (in ASCII format) to our z/OS machine (Example 9-16).

Example 9-16 FTP of a client certificate

C:\SDA_CTG_V61>ftp tx1.mop.ibm.comConnected to tx1.mop.ibm.com.220-FTPD11 IBM FTP CS V1R6 at TX1.MOP.IBM.COM, 15:01:26 on 2005-10-11220 Connection will close if idle for more than 5 minutes.User (tx1.mop.ibm.com:(none)): citgsd331 Send password please.Password:230 CITGSD is logged on. Working directory is "CITGSD.".


ftp> ascii200 Representation type is Ascii NonPrintftp> put "clientcr.arm"200 Port request OK.125 Storing data set CITGSD.CLIENTCR.ARM250 Transfer completed successfully.ftp: 706 bytes sent in 0.00Seconds 706000.00Kbytes/sec.ftp> quit221 Quit command received. Goodbye.

To complete our setup for client authentication we used the RACDCERT command to add the client certificate to the RACF digital key ring class and to connect it to our RACF key ring. Example 9-17 shows the commands to do this.

Example 9-17 RACDCERT commands to add and connect a client certificate

RACDCERT CERTAUTH ADD(‘CITGSD.CLIENTCR.ARM’) TRUST WITHLABEL(‘CTG V61 Client Certificate’)RACDCERT ID(CITGS1G) CONNECT(CERTAUTH LABEL(‘CTG V61 Client Certificate’) RING(CTGV61ring) USAGE(CERTAUTH))

9.4.3 Configuring CICS TGTo activate client authentication in the Gateway daemon we set the parameter clientauth=on in the SSL protocol parameter list (Example 9-18).

Example 9-18 SSL protocol parameters

[email protected][email protected]=port=8059;\ connecttimeout=2000;\ idletimeout=600000;\ pingfrequency=60000;\ clientauth=on;\

ciphersuites=SSL_RSA_WITH_RC4_128_SHA;\ keyring=CTGV61ring;\ esmkeyring;

If tracing is active, the Gateway daemon writes the following message during initialization:

SslHandler: + Client Authentication enabled for ssl: protocol


Mapping client certificates to RACF user IDsOptionally with SSL client authentication, client certificates can be mapped to RACF user IDs. To map a certificate to a RACF user ID the certificate must first be associated with a RACF user ID, using one of the following methods:

� The RACF command RACDCERT

This method stores client certificates in the RACF database, and associates a user ID with them. This is an excellent way to create a one-to-one mapping between client certificates and user IDs, but it does not scale well in the case where, for example, large numbers of certificates need to be mapped to a small number of user IDs.

� RACF certificate name filtering

This uses rules, to allow many certificates to be assigned a single user ID with one profile.

The com.ibm.ctg.util.RACFUserid class can be used to map an ECI request to a RACF user ID, based on the distinguished name in the SSL client certificate. The class has the following methods:

� setCertificate(byte[] clientCertificateData)� getRACFUserid()

Create a RACFUserid as follows:

RACFUserid myUseridObject = new RACFUserid(myCertificate);

This creates an object and automatically populates it with certificate data. When the object has been created, the getRACFUserid() method can be used to make a native RACF call to determine the user ID associated with the certificate data. If successful, it returns a string containing the user ID.

The SSLServerCompression.java class in the <install_path>/samples/java/com/ibm/ctg/samples/security subdirectory shows an example of how to use the RACFUserid class. For more information about the com.ibm.ctg.util.RACFUserid class, refer to CICS Transaction Gateway: Programming Guide.

9.4.4 Testing client authenticationAfter we configured the different software components, we tested our client authentication configuration with two different Windows application clients:

� The sample Java test application EciB1 supplied with the CICS TG� Our sample J2EE application CTGTesterCCI


Testing SSL client authentication with Java program EciB1We ran EciB1 with the SSL Keyring and SSL Password options, as shown in Example 9-14 on page 326. In this case, the Java client is requested to send the client certificate that we created in the keystore ctgv61.client.jks.

Testing SSL client authentication with CTGTesterCCITo demonstrate the use of SSL client authentication between WebSphere Application Server for Windows and the Gateway daemon, we repeated our test scenario with the CTGTesterCCI J2EE application.

To demonstrate that we had correctly configured client authentication for the Gateway daemon we first tried to run CTGTesterCCI before adding a client certificate to the keystore used by the application server. Figure 9-7 shows the error that we received when attempting to execute the CTGTesterCCI application.

Figure 9-7 CTGTesterCCI Error with clientauth=on but no Client Certificate

We also noted the messages shown in Example 9-19 in STDERR.

Example 9-19 STDERR - SSL client authentication error messages

ssl: S-C: Client Socket[addr=9.100.199.140/9.100.199.140,port=1154,localport=8059] failed to connect, handshake failuressl: .SslHandler:javax.net.ssl.SSLHandshakeException: handshake failure ssl: .SslHandler: at com.ibm.jsse.bv.a(Unknown Source) ssl: .SslHandler: at com.ibm.jsse.bv.startHandshake(Unknown Source) ssl: .SslHandler: at com.ibm.ctg.server.JSSEServerSocket.accept(JSSEServerSocket.java:109) ssl: .SslHandler: at com.ibm.ctg.server.SocketHandler.run(SocketHandler.java:666) ssl: .SslHandler: at java.lang.Thread.run(Thread.java:568)

To enable client authentication we:

1. Created a self-signed Client certificate on the WebSphere Application Server machine in the DummyServerKeyFile.jks using iKeyMan, using the same process as in 9.4.1, “Creating the client certificate on a Windows client machine” on page 329.

2. Extracted the self-signed client certificate as clientws.arm using iKeyMan and transferred to z/OS in ASCII mode using FTP as was done in 9.4.2, “Extracting a client personal certificate” on page 330.

3. Imported to our server keystore on z/OS using commands similar to those in Example 9-17 on page 331.


We ran the CTGTesterCCI application and it completed successfully.

Asserted IdentityIt is possible to use SSL client authentication as a way of establishing a trust relationship between the WebSphere Application Server and the Gateway daemon so that the application server can be trusted to flow an asserted user ID as a security credential. In this scenario user ID password checking would not be enabled in the Gateway daemon.

We repeated the test shown in 8.5.2, “Testing security with J2EE application CTGTesterCCI” on page 299 passing user ID CITGSD but this time we did not specify a password. We found that if a password is not specified on the ECIInteractionSpec the specified user ID is not flowed and the target CICS program runs with the default CICS user ID (Figure 9-8).

Figure 9-8 CTGTesterCCI - User ID with no password

To get the result that we expected we specified the user ID CITGSD and non-null password (any password is allowed in this instance because the Gateway daemon was not configured to verify passwords). The CICS program then ran with the specified user ID (CITGSD) (Figure 9-9).

Figure 9-9 CTGTesterCCI - User ID with non-null password

Note: APAR PK17700 (CICS TG for Multiplatforms) and PK19503 (CICS TG for z/OS) have been raised to change the behavior of the CICS ECI resource adapters to allow a user ID to be specified on the ECIInteractionSpec without specifying a password.


9.5 Migration considerationsJSSE is the only supported implementation of the SSL protocol with CICS TG V6 and above. If a previous version of the CICS TG was configured to use SSLight or SystemSSL, you must take steps to migrate the associated resources to a JSSE configuration.

9.5.1 SystemSSL to JSSE migrationIn this section, we give a brief outline of the steps required to migrate SystemSSL certificates to JSSE.

To migrate a certificate with private keys perform the following steps:

� Use the SystemSSL tool gskkyman to export the certificate in Base64 PKCS#12 format

� Use the iconv command to convert the file from EBCDIC to ISO8859 format

� Use keytool to import the certificate and private key from a PKCS#12 file to a keystore

To migrate a certificate without private keys:

� Use gskkyman to export the certificate to a binary ASN.1 DER encoded file

� Use keytool to import the certificate and private key from a ASN.1 DER file to a keystore

See the CICS Transaction Gateway z/OS Administration Version 6.1 guide for detailed information about migrating to JSSE.


9.6 Problem determinationIn this section, we document common SSL problems and information that we learned while configuring our SSL test scenarios:

� The example shown in Example 9-20 shows the error messages sent to the workstation during client authentication when the client certificate had not been connected to the RACF key ring:

Example 9-20 Handshake failure, missing client certificate in RACF key ring

java.io.IOException: CTG6651E Unable to connect to the Gateway daemon. [address= tx1.mop.ibm.com, port = 8059] [javax.net.ssl.SSLHandshakeException: handshakefailure] at com.ibm.ctg.client.SslJavaGateway.open(SslJavaGateway.java:211) at com.ibm.ctg.client.JavaGateway.open(JavaGateway.java:526) at com.ibm.ctg.client.JavaGateway.<init>(JavaGateway.java:205) at com.ibm.ctg.samples.eci.EciB1.main(EciB1.java:132)Caused by: javax.net.ssl.SSLHandshakeException: handshake failure

� If we omitted a KeyRingClass definition in the connection factory after specifying the SSL protocol in “Testing SSL server authentication with CTGTesterCCI” on page 327 we received the CTG9674E error message shown in Figure 9-10.

Figure 9-10 Server Authentication, no keyfile in connection factory

� After we had defined the KeyRingClass if we failed to import the CERTAUTH.X509 certificate into the ServerKeyFile.jks, we received the CTG9630E error message shown in Figure 9-11.

Figure 9-11 Server Authentication, no certificate in keyfile


� Apart from the obvious case of when the key ring does not exist the message shown in Example 9-21 can occur if the user ID associated with the key ring by the RACDCERT ID(userid) ADDRING(ringname) command does not match the region userid of the CICS TG.

Example 9-21 Example 9-21Key ring not found

CTG6525E Unable to start handler for the ssl: protocol. [java.lang.Exception: java.io.IOException: R_datalib (IRRSDL00) error: profile for ring not found (8, 8, 84)]

9.6.1 TracingWhen diagnosing SSL security problems, we used a java security trace. We set this using the CTGSTART_OPTS environment variable as shown in Example 9-22.

Example 9-22 Java security trace

CTGSTART_OPTS=-j-Djava.security.auth.debug=all


Chapter 10. Security with WebSphere Application Server for z/OS

In this chapter, we outline the main CICS and RACF security functions that can be implemented when the CICS TG is used with WebSphere Application Server for z/OS.

We start with a look at how the security mechanisms that we introduced in Chapter 8, “Gateway daemon security” on page 281 can be used to secure connections from WebSphere Application Server for z/OS. We document how we prepared the system and the settings that we used. We also explain how we tested the environment using our sample J2EE application CTGTesterCCI. We provide details of some of the common security related problems that you might see when using the CICS TG with WebSphere Application Server for z/OS.

10


10.1 PreparationIn our testing, our sample J2EE application is deployed on WebSphere Application Server for z/OS and connects to CICS using the CICS ECI resource adapter. In this chapter, we concentrate on how to configure security within and between each tier when a local connection is made between the WebSphere application server and the CICS region (Figure 10-1).

Figure 10-1 Software components: WebSphere security on z/OS

For general information about configuring local connections from WebSphere Application Server on z/OS, see Chapter 4, “Connecting from WebSphere Application Server for z/OS” on page 121.

It is also possible to connect from WebSphere Application Server on z/OS in remote mode to a Gateway daemon. In this case the security implications are similar to those discussed in Chapter 8, “Gateway daemon security” on page 281.

z/OS

CICS TS

COBOL application

WebSphereApplication Server, Version 6

ServletJSP

CTGTesterCCI

COMMAREA

CICS TG V6.1

CICS ECIresourceadapterCCI

EXCI

HTTP(S)

User ID Password

orClient

Certificate

RACF

Authorizationchecks

Authentication and authorization checks





Table 10-2 Definitions checklist: WebSphere Application Server for z/OS

Windows 2000 z/OS


Windows 2000 Service Pack 4 WebSphere Application Server V6.0.1




Value WebSphere Application Server for z/OS

CICS TS

hostname tx1.mop.ibm.com

TCP/IP port 11880

Job names Daemon region: CITGSDMNControl region: CITGSS1Servant region: CITGSS1S

CITGS1CI

APPLID A6POTGS1

NETNAME CITGSS1

CONNECTION GSS1

Private mirror ECIW

Chapter 10. Security with WebSphere Application Server for z/OS 341

Table 10-3 lists the user IDs that we used in our configuration.

Table 10-3 User ID checklist

10.2 Configuring CICS and CICS TG security The basic CICS security checks were introduced in Chapter 8, “Gateway daemon security” on page 281. Here we discuss how these security checks apply to a WebSphere z/OS configuration.

We describe the following tasks:

� Configuring CICS security� Configuring MRO bind time security� Enabling CICS TG password checking� Configuring security for CICS CONNECTION and SESSIONS definitions� Configuring link security� Permitting access to the mirror transaction� Configuring surrogate security

Configuring CICS securityOur CICS region, CITGS1CI (with APPLID A6POTGS1), was configured with security prefixing and transaction security as described in “Configuring CICS security” on page 286.

Configuring MRO bind securityMRO bind security prevents unauthorized attached MRO regions from starting transactions in a CICS region, and as such applies to a WebSphere servant region. It is implemented using two different DFHAPPL FACILITY class profiles.


CICS region user ID CITGS1CI

WebSphere default user ID CITGSSRU

Flowed user ID to which we wish to permit access

CITGSD

Flowed user ID to which we wish to deny access

CITGNW


To connect to a CICS region, the WebSphere servant user ID requires the following permissions:

� Update access to the RACF FACILITY class profile DFHAPPL.<DFHJVPIPE>, where <DFHJVPIPE> is the EXCI pipe name as defined in the CICS TG environment variable DFHJVPIPE and in the NETNAME parameter in the CICS CONNECTION definition. If a generic EXCI connection is used, there is no pipe name and this does not apply.

� Read access to the RACF FACILITY class profile DFHAPPL.<APPLID>, where <APPLID> is the APPLID of the CICS region in question.

We enabled MRO bind security with the following commands:

1. We gave update access to the RACF FACILITY class profile DFHAPPL.<DFHJVPIPE> to our WebSphere servant region user ID CITGSSRU, using the following RACF commands:

RDEFINE FACILITY (DFHAPPL.CITGSS1) UACC(NONE)PERMIT DFHAPPL.CITGSS1 CLASS(FACILITY) ID(CITGSSRU) ACCESS(UPDATE)SETROPTS RACLIST(FACILITY) REFRESH

In our configuration, CITGSSRU is the user ID of the WebSphere servant region, and CITGSS1 is the name that we used for the DFHJVPIPE environment variable.

2. We gave read access to the RACF FACILITY class profile DFHAPPL.<APPLID> to our WebSphere servant region user ID CITGSSRU, using the following RACF command:

RDEFINE FACILITY (DFHAPPL.A6POTGS1) UACC(NONE)PERMIT DFHAPPL.A6POTGS1 CLASS(FACILITY) ID(CITGSSRU) ACCESS(READ)SETROPTS RACLIST(FACILITY) REFRESH

In our configuration, A6POTGS1 is the APPLID of the CICS region.

Enabling CICS TG password checking It is possible to enable the CICS TG password checking by setting the environment variable AUTH_USERID_PASSWORD as a WebSphere variable with the WebSphere Administration Console. However, for a local mode connection it is likely that the user ID has already been authenticated by the WebSphere Application Server and CICS TG password checking is not normally required.

Note: Use of MRO bind-time security is optional and if these DFHAPPL profiles are not defined, then any MRO connected system will be able to connect to your CICS region.


With WebSphere Application Server installed on z/OS, it is possible to configure a traditional z/OS authentication model. Authentication takes place at the entry point to z/OS (which for a J2EE application will be normally in an HTTP Server or in the WebSphere Application Server ) and after authentication the security identity (normally a RACF user ID) flows with the work request as it runs in different parts of the system.

We did not set the AUTH_USERID_PASSWORD environment variable.

CONNECTION and SESSIONS definitionsIn order for our CICS region to use the user ID flowed in the EXCI call from the WebSphere servant region, we set the parameter ATTACHSEC=IDENTIFY in the EXCI connection definition in our CICS region, as shown in Figure 10-2.

Figure 10-2 CICS CONNECTION definition

Link securityThis is an additional level of authorization checking that can apply to the attached transaction. A specific user ID (the link user ID) can be thought of as the user ID associated with the server which is passing the request to CICS. In our case, this is the user ID of the WebSphere servant region. If link security is enabled, the link

OVERTYPE TO MODIFY CICS RELEASE = 0640 CEDA ALter CONnection( GSS1 ) CONnection : GSS1 Group : GSS1 DEscription ==> CONNECTION IDENTIFIERS Netname ==> CITGSS1 INDsys ==> CONNECTION PROPERTIES ACcessmethod ==> IRc Vtam | IRc | INdirect | Xm PRotocol ==> Exci Appc | Lu61 | Exci Conntype ==> Specific Generic | Specific SECURITY SEcurityname ==> ATtachsec ==> Identify Local | Identify | Verify | Persistent | Mixidpe BINDPassword : PASSWORD NOT SPECIFIED BINDSecurity ==> No No | Yes Usedfltuser ==> No No | Yes SYSID=TGS1 APPLID=A6POTGS1


user ID, like the flowed user ID, must be authorized to access all transactions and resources invoked as a result of the request.

The link user ID can be preset on the EXCI SESSIONS definition using the USERID parameter. For EXCI requests, link security works as follows:

1. If the link user ID is the same as the CICS region user ID, then the systems are deemed equivalent and no link security authorization is performed.

2. If the link user ID is preset as something other than the CICS region user ID, then this user ID must be authorized to access all transactions and resources invoked as a result of the request.

3. If the link user ID is not preset, then the user ID of the connected region (that is, the user ID of the WebSphere servant region) is the link user ID and this user ID must be authorized to access all transactions and resources invoked as a result of the request.

We used preset link security. This has the advantage that it avoids the overhead of EXCI sign-on for each request that is passed to CICS.

To enable preset link security, we set the USERID parameter in the SESSIONS definition to be CITGSSRU, which is the WebSphere servant region user ID. Our SESSIONS definition is shown in Figure 10-3.

Figure 10-3 CICS SESSIONS definition

OVERTYPE TO MODIFY CICS RELEASE = 0640 CEDA ALter Sessions( GSS1) Sessions : GSS1 Group : GSS1 DEscription ==> SESSION IDENTIFIERS Connection ==> GSS1 SESSName ==> NETnameq ==> MOdename ==> SESSION PROPERTIES Protocol ==> Exci Appc | Lu61 | Exci MAximum ==> 000 , 000 0-999 RECEIVEPfx ==> 1 1-999 RECEIVECount ==> 260 PRESET SECURITY USERId ==> CITGSSRU SYSID=TGS1 APPLID=A6POTGS1


Table 8-4 on page 291 lists the different settings for link security and ATTACHSEC and how they interoperate.

Configuring the flowed user IDThe flowed user ID is the identity that is passed on the ECI request. It often represents the identity of the end user of the application. When using the base CICS TG classes, the user ID and password can be specified as parameters when constructing an ECIRequest object. When using the JCA with WebSphere Application Server for z/OS, it is possible for an authenticated user ID to be automatically passed to CICS (see 10.3, “Configuring JCA security” on page 347). In our tests we flow the user ID CITGSD.

Permitting access to the mirror transactionsAn EXCI request received by CICS from the WebSphere Application Server for z/OS attaches a mirror transaction. Therefore, we needed to authorize our flowed user ID CITGSD and our link user ID CITGSSRU to the private mirror transaction ECIW that we are using. We issued the following RACF commands:

RDEFINE TCICSTRN CITGS1CI.ECIW UACC(NONE)PERMIT CITGS1CI.ECIW CL(TCICSTRN) ID(CITGSSRU) ACCESS(READ)PERMIT CITGS1CI.ECIW CL(TCICSTRN) ID(CITGSD) ACCESS(READ)SETROPTS RACLIST(TCICSTRN) REFRESH

Configuring surrogate security In order for the EXCI to be able to switch the security context of the EXCI request to the flowed user ID, the correct SURROGAT security profiles must be defined. The user ID of the EXCI client (in our case the WebSphere servant region) requires read access to the <USERID>.DFHEXCI SURROGAT class profile, where <USERID> is the flowed user ID in the EXCI request.

We issued the following commands to permit our WebSphere servant region user ID CITGSSRU to switch to the user ID CITGSD which is flowed in the ECI request:

RDEFINE SURROGAT CITGSD.DFHEXCI UACC(NONE) OWNER(CITGSD) PERMIT CITGSD.DFHEXCI CLASS(SURROGAT) ID(CITGSSRU) ACCESS(READ)

Tip: For our examples, we permit access to single users (CITGSD and CITGSSRU). In a production environment, you will probably create a group of users requiring common access. After a group is built, you will permit access to a group. This permits user’s access to be controlled by the group to which they belong, rather than by individual permissions. This simplifies the security definitions required.


It is possible to disable surrogate security by either reassembling the EXCI options table DFHXCOPT, with SURROGAT=NO, or by using a RACF surrogate profile with universal READ access such as:

RDEFINE SURROGAT *.DFHEXCI UACC(READ) OWNER(CITGSD)

10.3 Configuring JCA securityThe JCA has specific support for enabling secure access from a J2EE application to an Enterprise Information System (EIS) such as CICS. Both container-managed sign-on and component-managed sign-on are supported, as described in 8.4, “Configuring JCA security” on page 293.

When using container-managed sign-on, the application deployer is able to set up the authentication information (user ID and password) when he deploys the application. In WebSphere Application Server V6, this can be done using a J2C authentication data entry (see 8.4.1, “J2C authentication data” on page 294). When using WebSphere Application Server for z/OS, the container can also derive the identity from the currently executing Java thread. This is known as thread identity support.

10.3.1 Thread identity supportThread identity support allows WebSphere Application Server for z/OS to automatically pass the user ID of the Java thread to CICS when using an ECI resource adapter. The setting of the thread user ID is dependent on the RunAs policy for the J2EE application. If RunAs is set to Caller, and the user of the Web application has authenticated with the WebSphere Application Server, then thread identity support enables the caller’s identity to be propagated to CICS automatically.

Thread identity support enables WebSphere Application Server to behave in a way that traditional z/OS address spaces behave. Once the user ID has been authenticated, that user ID flows with any work done within the z/OS system. This is similar to the way that MRO requests in CICS are managed, for example.

Thread identity support is enabled when:

� WebSphere Global security is enabled and the active user registry is defined as Local OS (normally this means RACF).

� A local connection is being used between WebSphere Application Server and CICS.

� Container-managed security is being used (the res-auth deployment descriptor is set to Container).


� The resource reference is not mapped to a J2C authentication data entry and the connection factory does not specify a JAAS Authentication Alias.

In order to enable thread identity support, we needed to consider the following configuration and setup tasks:

� Enabling WebSphere Global Security

� Creating a connection factory

� Updating the deployment descriptors for the sample application CTGTesterCCI

� Defining a security role

� Deploying the sample application

Enabling WebSphere Global SecurityTo enable Global Security for our WebSphere Application Server, we first started the server using the command:

S CITGSCR,JOBNAME=CITGSS1,ENV=CITGSC1.CITGSN1.CITGSS1

We entered the following URL in the Web browser to access the WebSphere Admin console and logged on with our user ID CITGSADM:

http://tx1.mop.ibm.com:11880/admin

To configure WebSphere Application Server Global Security, we did the following:

1. Clicked Security → Global Security → Authentication Mechanisms → LTPA.

2. Filled in our password and confirmed it by entering it again. We then clicked Apply and saved our configuration change.

3. Clicked Security → Global Security. Selected Enable global security and deselected Enforce Java 2 Security. Set Active Protocol as CSI and SAS and Active Authentication Mechanism as LTPA.

Note: JAAS authentication aliases are deprecated in WebSphere Application Server V6 and are supported for compatibility reasons.

Note: Before making changes to the WebSphere Application Server security settings, we took a copy of the security.xml file in /usr/lpp/zWebSphere/V6R0/config/cells/BaseApplicationServerCell. This allows us to back out the changes if we are unable to connect to the WebSphere Application Server Administration Console.


http://tx1.mop.ibm.com:11880/admin

4. Set Active User Registry as Local OS.

Figure 10-4 shows the Global Security definitions.

Figure 10-4 Administrative Console: define Global security

5. We clicked Apply, Save, and then restarted the J2EE server.

6. We connected to the Administrative Console again and noted that the server redirected us to the console URL accessed via the SSL port:

https://tx1.mop.ibm.com:11843/ibm/console/logon.jsp

7. We received a certificate warning and clicked Yes to proceed. We were then required to specify a password for the WebSphere administration user ID CITGSADM.


https://tx1.mop.ibm.com:11843/ibm/console/logon.jsp

Creating a connection factoryWe defined a connection factory called CITGS1CI-tx1-local using the same procedure as described in 4.2.4, “Creating a connection factory” on page 128. Figure 10-5 shows the custom properties for the connection factory.

Figure 10-5 Administrative Console - connection factory custom properties

Updating the application deployment descriptorsWe used Rational Application Developer to update the following deployment descriptors for the CTGTesterCCI sample application:

� Defining a security constraint� Setting the Res-auth for the resource reference to Container� Setting RunAs for the EJB methods to Caller

Defining a security constraintWe added a security constraint to the CTGTesterCCI application. If Global Security is enabled, and a security constraint is set for a particular resource, the resource is secured and users of the application must authenticate themselves.


Within the application EAR file, the Web application archive CTGTesterCCIWeb.jar contains file WEB-INF/web.xml. We added the security constraint as shown in Example 10-1.

Example 10-1 Web application deployment descriptor security constraint

<security-constraint><web-resource-collection>

<web-resource-name>CTGTesterCCI Web resource </web-resource-name><description></description><url-pattern>/*</url-pattern><url-pattern>/index.jsp</url-pattern>

</web-resource-collection><auth-constraint>

<description>Role required to run CTGTesterCCI</description><role-name>CTGTEST</role-name>

</auth-constraint><user-data-constraint>

<transport-guarantee>NONE</transport-guarantee></user-data-constraint>

</security-constraint><login-config>

<auth-method>BASIC</auth-method><realm-name>WASWebContainer</realm-name>

</login-config><security-role>

<description>Security role required in order to run CTGTesterCCI application</description>

<role-name>CTGTEST</role-name></security-role>

This security constraint specifies that the Web application user must:

� Login to the J2EE server using basic authentication (<auth-method>BASIC</auth-method>)

� Be authorized to role CTGTEST (<role-name>CTGTEST</role-name>)


Setting the Res-authWithin the application EAR file, the EJB archive CTGTesterCCIEJB.jar contains file META-INF/ejb-jar.xml. This deployment descriptor file contains a resource-ref for the ECI reference. We set the res-auth for this resource to Container (Example 10-2).

Example 10-2 EJB deployment descriptor res-auth setting

<resource-ref id="ResourceRef_1"><description>CICS ECI Resource adapter</description><res-ref-name>ECI</res-ref-name><res-type>javax.resource.cci.ConnectionFactory</res-type><res-auth>Container</res-auth>

</resource-ref>

Setting RunAs A RunAs setting on a servlet or EJB affects the Java thread identity of methods invoked on a subsequent call. WebSphere Application Server RunAs policy allows three choices in assigning the Java thread identity for the current request:

� Caller uses the caller’s identity for the method selected and to propagate it to any subsequent methods invoked or J2EE resources accessed. This is the default behavior.

� Server indicates that the method should assume the identity of the WebSphere servant region.

� Role the application assembler selects the name of a security role (we look at roles in a few moments) that is defined in the application. Authorization is performed by checking that the principal name has been assigned to one of the required security roles.

In our tests, we used the default RunAs setting (Caller) for all methods of the sample J2EE application CTGTesterCCI.

Defining a security roleSecurity roles are used to control access to J2EE resources such as servlets and EJBs. The name of the roles are defined in the application deployment descriptor, for example, in Example 10-1 on page 351 we defined a role-name CTGTEST.

There are two basic ways to control role access with WebSphere Application Server for z/OS:

� Using SAF EJBROLE profiles� Using WebSphere bindings


We defined the EJBROLE CITGS.CTGTEST and authorized the user ID CITGSD to the EJBROLE.

We used RACF to create the EJBROLE class and authorize the CITGSD user ID to the role with the following commands:

RDEFINE EJBROLE CITGS.CTGTEST UACC(NONE)PERMIT CITGS.CTGTEST CLASS(EJBROLE) ID(CITGSD) ACCESS(READ)SETROPTS RACLIST(EJBROLE) REFRESH

Deploying the sample applicationAfter having used RAD to modify the deployment descriptors of the application, we exported the application and created a new EAR called CTGTesterCCI_Sec.ear. We then followed the same process as described in 4.2.6, “Deploying our sample J2EE application” on page 134 to deploy the application to the WebSphere Application Server. During the deployment process, we mapped the ECI resource reference to the JNDI name of the CITGS1CI-tx1-local connection factory.

10.4 Testing the configurationWe wanted to test that thread identity support could be used to automatically flow an authenticated user ID from WebSphere Application Server to CICS.

10.4.1 Testing thread identity supportWe closed all the Web browser sessions to ensure that no basic authentication headers were cached, and we invoked the test application CTGTesterCCI using the following URL:


Note: CITGS is the security domain prefix for our WebSphere Application Server. The specification of a security domain prefix affects the specific EJBROLE profiles used by WebSphere Application Server for z/OS. This enables you to deploy the same application on different cells, but have different user to role mappings if desired.



The Web browser basic authentication pop-up window was displayed (Figure 10-6). We specified user ID CITGSD and a valid password. We noted that if we specified an invalid password the basic authentication login was re-displayed.

Figure 10-6 CTGTesterCCI basic authentication login

We clicked OK, and the CTGTesterCCI JSP was displayed. We did not specify a user ID and password on the CTGTesterCCI input screen (Figure 10-7).

Figure 10-7 CTGTesterCCI input parameters - no user ID and password specified


The CTGTesterCCI application ran successfully. ECIPROG returns a COMMAREA which contains the user ID under which the CICS task runs. The output from ECIPROG shows that the request ran successfully in our secure CICS region CITGS1CI, and that the CICS task ran under the user ID CITGSD (Figure 10-8). CITGSD is the user ID that we used to login to the WebSphere Application Server. It is automatically passed to CICS by the application server as a result of the unique thread identity support that is provided by WebSphere Application Server for z/OS.

Figure 10-8 CTGTesterCCI results with thread identity support enabled

In order to test that an unauthorized user is not able to access the CTGTesterCCI application, we repeated the test but this time we logged in with an unauthorized user ID.

We closed all the Web browsers on the machine and ran the CTGTesterCCI application again, specifying user ID CITGNW and a valid password when prompted. Figure 10-9 shows that the user is not authorized to run the application.


Figure 10-9 Web browser CTGTesterCCI authorization failure

We also noted the following RACF EJBROLE authorization message in the syslog (Example 10-3).

Example 10-3 SYSLOG - EJBROLE access not allowed

ICH408I USER(CITGNW ) GROUP(CITG ) CITGS.CTGTEST CL(EJBROLE ) INSUFFICIENT ACCESS AUTHORITY ACCESS INTENT(READ ) ACCESS ALLOWED(NONE )

In this case, the user CITGNW has authenticated successfully with the WebSphere Application Server but the EJBROLE check has failed because CITGNW does not have READ access to the EJBROLE CITGS.CTGTEST.


10.5 Problem determinationIn this section, we document common security problems and information we learned while configuring our WebSphere z/OS security scenarios. For additional tips and utilities for problem determination of WebSphere z/OS connection related problems, see 2.9.2, “Tips and utilities” on page 62.

Surrogate security errors If the WebSphere servant region does not have the required access to flow a specific user ID (CITGNW) to CICS, you will receive the RACF error shown in Example 10-4.

Example 10-4 Surrogate check failure for WebSphere servant region user ID

ICH408I USER(CITGSSRU) GROUP(CITGSCFG) NAME(WAS APPSVR SR ) CITGSD.DFHEXCI CL(SURROGAT) INSUFFICIENT ACCESS AUTHORITY ACCESS INTENT(READ ) ACCESS ALLOWED(NONE )

The request has failed because the WebSphere servant region user ID CITGSSRU does not have access to the SURROGAT class profile CITGSD.DFHEXCI.

Bind security errorsIf the WebSphere servant region does not have the required access to bind to the CICS connection, you will receive the RACF error shown in Figure 10-5.

Example 10-5 RACF message of MRO bind security failure - DFHAPPL.DFHJVPIPE

ICH408I USER(CITGSSRU) GROUP(CITGSCFG) NAME(WAS APPSVR SR ) DFHAPPL.CITGSS1 CL(FACILITY)

INSUFFICIENT ACCESS AUTHORITY ACCESS INTENT(UPDATE ) ACCESS ALLOWED(NONE )

If the WebSphere servant region does not have the required access to bind to the CICS region, you will receive the RACF error shown in Figure 10-6.

Example 10-6 RACF message of MRO bind security failure - DFHAPPL.APPLID

ICH408I USER(CITGSSRU) GROUP(CITGSCFG) NAME(WAS APPSVR SR ) DFHAPPL.A6POTGS1 CL(FACILITY)

INSUFFICIENT ACCESS AUTHORITY ACCESS INTENT(READ ) ACCESS ALLOWED(NONE )


Part 5 Appendixes

In this part, we describe the sample applications that are provided with this book. We also provide further details on data conversion and EXCI tracing.

Part 5


Appendix A. Sample applications

In this appendix, we give details of our two sample applications:

� CTGTesterCCI

CTGTesterCCI uses the J2EE CCI classes and the CICS ECI resource adapter to flow an ECI request to CICS. It uses a JSP/servlet/EJB design and the EJB session bean has a single JCA resource reference.

� CTGTesterCCIXA

The CTGTesterCCIXA application is based on CTGTesterCCI but it is intended to be used with the CICS ECI XA resource adapter. It contains two JCA resource references and can be used to flow two ECI requests to different CICS regions, such that the two calls can be committed within the same global transaction.

Both of these applications are designed for the testing of connectivity from the application server to CICS, either with a local connection or through the Gateway daemon. As such, all the CICS dependencies, such as COMMAREA size, input, encoding and length, are externalized to enable you to test different configurations. The applications are not designed for production use, because such externals are unlikely to be exposed in real-life customer applications.

A


The CTGTesterCCI applicationThe CTGTesterCCI application contains a Java servlet and a session bean that we used in various test scenarios. It is intended to provide a simple enterprise application that can be used out of the box to test ECI connectivity from any CICS Transaction Gateway to any configured CICS region using the CICS ECI resource adapter. It can be used to test a local connection from WebSphere Application Server to CICS or a remote connection via a Gateway daemon. This sequence of events is illustrated in Figure A-1.

Figure A-1 Architecture of CTGTesterCCI

All input parameters can be specified as HTTP query strings or HTML form parameters, and all output is through text-based HTML.

It has the following features:

� Setting of the CICS program name� Setting of the COMMAREA input data, length and codepage� Optional setting of user ID, password and mirror transaction name� Iteration count for repeated execution of the CICS program � Optional setting of application CICS TG Java client trace

CTGTesterCCI contains a single JCA resource reference. When deploying the application (see 3.2.3, “Deploying the sample J2EE application” on page 89), the resource reference must be mapped to a connection factory. The connection factory contains the CICS APPLID and the connection details of the Gateway daemon that will be used. In addition, the connection factory can contain other optional information, such as security credentials that is used for the connection.

CCI


CTGTesterCCIServlet

CTGTesterCCI

JSP

Web browser

HTML

CICS TS

CICS program

servlet

session bean CICS ECIresource adapter

Gatewaydaemon

CCI


Application overviewThe following is the sequence of events that occur when the end user interacts with the CTGTesterCCI application through a Web browser, illustrated in Figure 3-16 on page 109:

1. The user opens the Web application using the following URL and enters the input parameters for the JCA request:

http://localhost:9080/CTGTesterCCIWeb/index.jsp

2. The user then clicks a button on a Web page that submits a form to the servlet.

3. The servlet receives the request for action and calls a method on the remote interface of the CTGTesterCCI session bean.

4. The session bean uses the CICS ECI resource adapter to call the CICS program.

5. The CICS ECI resource adapter returns data from the CICS program to the session bean.

6. The session bean returns the output data from the CICS program back to the servlet.

7. The servlet forwards to a JSP, which displays the response to the user.

This sequence of events is illustrated in Figure A-2.

Figure A-2 Flow of CTGTesterCCI


CTGTesterCCI

welcomeJSP

Web browser

CTGTesterCCIServlet

servlet

resultsJSP

success

exceptionJSP

exception

CTGTesterCCIsession bean

1 2 3

45

67

Appendix A. Sample applications 363

http://localhost:9080/CTGTesterCCIWeb/index.jsp

Static HTML FormThe HTML form index.jsp is stored in the WebContent folder of the CTGTesterCCIWeb package and can be viewed using Rational Application Developer. The HTML form consists of a number of fields where data for the application is entered. Table A-1 shows the fields on the form.

Table A-1 Fields in the HTML form, CTGTesterCCI

The action of the HTML form is to call the servlet CTGTesterCCIServlet, using the POST method to send the parameters. Then the parameters are sent in the HTTP request body. Because an HTTP POST request is intended to affect some sort of change on the Web server, a Web browser prompts to resubmit the data if the user clicks Refresh on the results page. Conversely, the GET method could be used, which sends the parameters on the HTTP request URL. Because HTTP GETs are not intended to alter anything on the Web server, the results page of such a request can be refreshed without a prompt being shown.

Field Name Purpose

Mandatory details

CICS program name funcName Program on CICS to call

COMMAREA input commareaInput COMMAREA data

COMMAREA length commareaLength Length of the COMMAREA

Encoding encoding Encoding to convert data to before sending and after receiving

Optional details

User ID username Username to flow on the ECI request

Password password Password to flow on the ECI request

Mirror transaction mirror Transaction to use on the CICS server

Iterations® iterations The number of times to run the CICS program

Application trace appTrace Whether to enable CICS TG Java client trace


CTGTesterCCIServletThe Servlet converts the HTML FORM user data into a format which the Session EJB can use. The Session EJB facade masks how or where the request is actually processed, providing separation between presentation and business logic. Figure A-3 summarizes the CTGTesterCCIServlet code.

Figure A-3 CTGTesterCCIServlet logic overview

The following gives a brief description of the main methods of CTGTesterCCIServlet.

processRequest() - 1The CTGTesterCCIServlet doGet() and doPost() methods both pass on the HTML form data for the request (whichever convention was used) to the processRequest() method. The processRequest() method then creates an ECIRequestDetails object called eciReq1 (an encapsulation of the user input data) and calls MapRequest2Details() to populate it.

MapRequest2Details()The MapRequest2Details() method takes the HttpServerletRequest object (effectively the input) and an ECIRequestDetails object (effectively the output) together with an optional suffix string. The field names, shown in the central column of Figure A-1 are combined with the suffix input string to identify name/value pairs contained in the HttpServerletRequest data. The value for each field is copied into the corresponding EciRequestDetails class variable, performing any type conversions from the user input that are required.

CTGTesterCCIServlet

JNDINamespace

HTML FORMS DATA

processRequest()Map2Details()eciReq1Lookup ejbHome

ejbHome.create()

ejb.execute(eciReq1)

results.jsp/exception.jsp

ejb.execute()


processRequest() - 2Once the request data as been setup in eciReq1, processRequest() obtains a reference to CTGTesterCCIHome via a lookup on the initial context for the EJB reference ejb/CTGTesterCCI. The CTGTesterCCIHome.create() method is used to create an instance of the CTGTesterCCI EJB remote interface. The local CTGTesterCCI instance is called tester. The ejb reference is defined in the Web Deployment Descriptor (Figure A-4).

Figure A-4 CTGTesterCCI EJB resource reference definition

Having gathered together all of the details for the ECI request, and created an instance of the Session EJB CTGTesterCCI remote interface, processRequest() finally invokes the execute() method on the CTGTesterCCI bean instance, passing in the EciRequestDetails object, eciReq1.

Note: CTGTesterCCIServlet only uses a single ECI request, and so the suffix field is set to an empty string. The usage of the suffix string is clearer in the context of the CTGTesterCCIXAServlet code (see “CTGTesterCCIXAServlet” on page 374).

<ejb-ref id="EjbRef_1"><description></description><ejb-ref-name>ejb/CTGTesterCCI</ejb-ref-name><ejb-ref-type>Session</ejb-ref-type><home>itso.cics.eci.j2ee.testercci.CTGTesterCCIHome</home><remote>itso.cics.eci.j2ee.testercci.CTGTesterCCI</remote>

</ejb-ref>


Session EJB CTGTesterCCIFigure A-5 shows the interactions between the calling servlet and session EJB CTGTesterCCI.

Figure A-5 CTGTesterCCI EJB logic overview

The following gives a brief description of the main methods of CTGTesterCCI.

ejbCreate()The EJB life cycle method ejbCreate() is used to create a conection factory. We create an InitialContext object and use it to look up a connection factory using the JNDI name java:comp/env/ECI. This is a local JNDI name that will be mapped to the actual JNDI name of the connection factory at deploy time. To indicate that we want such a mapping, we create a resource reference in the EJB deployment descriptor with the name ECI.

execute()The only method available on the remote interface of the CTGTesterCCI EJB is the execute() method. This method and supporting logic is implemented by class CTGTesterCCIBean. The EciRequestDetails and JavaStringRecord classes also provide supporting function.

CTGTesterCCI

ejbCreate()lookup ("java:comp/env/ECI")create connection factory

execute(eciReq1)flowRequest(eciReq1)get CCI connectionget CCI interactioncopy eciReq1 to EciInteractionSpecloop # iterations

interaction.execute()return COMMAREA or throw exception

CTGTesterCCIServlet

CTGTesterCCIHome.create()

CTGTesterCCI.execute()

Invoke appropriate JSP


flowEciRequest()The flowEciRequest() method creates a connection by calling the getConnection() method. The getConnection()method creates an EciConnectionSpec and sets security details if specified on the JSP input page.

Method flowEciRequest() then creates an interaction by calling getInteraction() and sets up the ECIInteractionSpec object (eSpec) based on the other input parameters entered on the JSP input page (CICS program, COMMAREA length and mirror transaction name). Finally, it enters a loop, based on the iteration limit entered on the JSP input page, issuing the execute() method on the interaction (eciInt).

The execute() method takes the JavaRecordString object jsr as a parameter twice, once for COMMAREA input and once for COMMAREA output.

The iteration loop continues for as long as each request completes normally and until the iteration limit is reached, each time overwriting the output COMMAREA of the previous iteration.

Ultimately, the invocation of flowEciRequest() either returns a JavaStringRecord, representing the COMMAREA returned by the last iteration (if successful) or re-throws a ResourceException caught on the EciInteraction.execute() call.

Catching and handling a CICS transaction abendThe execute() method might throw a ResourceException, which has several sub-classes. For example, in the event of a CICS transaction abend a CICSTxnAbendException is thrown by the CICS ECI resource adapter. If this exception occurs, we issue a setRollbackOnly() on the EJB session context in the catch block for the execute() call. This ensures that the CICS abend will be propagated onto WebSphere Application Server as the transaction manager and the local transaction (if one exists) will be rolled back. Example A-1 shows this logic.

Example: A-1 Catching and handling a transaction abend

public class CTGTesterCCIBean implements SessionBean {...private JavaStringRecord flowEciRequest(ECIInteractionSpec eSpec,

Note: The transaction attribute is set in the assembly descriptor section of the EJB deployment descriptor for the execute() method to control the transactional behavior of the EJB and associated JCA request (see “Updating the EJB deployment descriptor” on page 176). In the version of CTGTesterCCI provided with the Redbook, the transaction attribute is set to NotSupported.


EciRequestDetails eciRD, String cfRef) throws Exception {

...// make the calltry {

eciInt.execute(eSpec, jsr, jsr);} catch (ResourceException re) {

// output the stacktrace and re-throw it back to the clientre.printStackTrace();// Depending upon the kind of exception thrown, we may decide // to abandon the current transaction by calling setRollbckOnly()// on the current context. If this request is not part of a local// or global transaction, there will be a second exception to // catch (IllegalStateException). If so, we will log the exception // details, but otherwise absorb it.

// In the case of a CICSTxnAbendException, we will write a log // message and rollback any current transaction.if (re instanceof com.ibm.connector2.cics.CICSTxnAbendException ){

System.err.println("CTGTesterCCIBean.flowEciRequest: CICS ABEND detected."+

" Connection Factory="+targetCF.toString());try {

mySessionCtx.setRollbackOnly();} catch(IllegalStateException ise) {

ise.printStackTrace();}

} else {// Write generic failure log messageSystem.err.println("CTGTesterCCIBean.flowEciRequest:

ResourceException detected."+" Connection Factory="+targetCF.toString());

}

// clean up interaction and connection referencesdropConnection(eciConn, eciInt);eciInt = null;eciConn = null;// throw exception back to callerResourceException re2 = new ResourceException(re.getMessage()+

" ConnectionFactory="+targetCF.toString());re2.initCause(re);throw re2;

}}

Note: The setRollbackOnly command shown in Example A-1 is not necessary if the option ABENDBKOUT=YES is set in the DFHXCOPT table (see “ABENDBKOUT option in DFHXCOPT” on page 165).


JSPsThe application uses two JSPs to format and display the results. One of two scenarios can occur; the request completes successfully, or an exception occurs inside the CICS ECI resource adapter or the application. The JSPs are stored in the CTGTesterCCIWeb WebContent folder and can be viewed using Rational Application Developer.

results.jspThe results.jsp processes successful completion of a request. The JSP defines the title of the HTML page and the style sheet used to format the HTML in the header. The JSP defines the JavaBeans™ components that the page uses to format and display the results (Figure A-6).

Figure A-6 JavaBeans components used by results.jsp

These components are all in the request scope and are all String objects. Their IDs correspond to the names of the attributes we set in the HttpRequest in our servlet. They contain the result of the CICS request made by the servlet.

The page then includes our common JSP header, which displays the information specified in the initial form, such as trace level and number of iterations. The rest of the page then outputs the values of the request attributes using the scriptlet format <%= JavaBean id %> (Figure A-7). In our application, the JavaBean component encoding contains the encoding used to convert the input COMMAREA into bytes, and is inserted into the HTML output with the JSP scriplet <%= encoding %>.

<jsp:useBean class="java.lang.String" id="results" scope="request"/><jsp:useBean class="java.lang.String" id="commareaInput" scope="request"/><jsp:useBean class="java.lang.String" id="encoding" scope="request"/><jsp:useBean class="java.lang.String" id="defaultResults" scope="request"/><jsp:useBean class="java.lang.String" id="hexResults" scope="request"/><jsp:useBean class="java.lang.String" id="requestTime" scope="request"/>


Figure A-7 Outputting the results into the HTML

The results page outputs the COMMAREA data converted to a String using the JVM default encoding, the encoding specified in the form and in a hexadecimal representation.

error.jspThe JSP error.jsp is used when an error occurs. It does not display the output COMMAREA. Instead, it displays the contents of the exception that occurred and any errors found in the processing of the form parameters by the servlet.

<TABLE BGCOLOR=white CELLPADDING=0 width="624"><TBODY>

<TR ALIGN=LEFT><TH width="121">Results</TH><TH width="497">COMMAREA</TH>

</TR><TR>

<TD width="121">Input:</TD><TD BGCOLOR=yellow width="497"><%= commareaInput %></TD>

</TR>

<TR><TD width="121">Output using <%= encoding %>:</TD><TD BGCOLOR=yellow width="497"><%= results %></TD>

</TR>

<TR><TD width="121">Output in HEX:</TD><TD BGCOLOR=yellow width="497"><%= hexResults %></TD>

</TR>

<TR><TD width="121">Request time (ms):</TD><TD BGCOLOR=yellow width="497"><%= requestTime %></TD>

</TR>

</TBODY></TABLE>

<TABLE BGCOLOR=white><TBODY><TR><TH>Request succeeded.</TH></TR></TBODY></TABLE>


The CTGTesterCCIXA applicationThe CTGTesterCCIXA application is based on CTGTesterCCI and provides a simple enterprise application that can be used out of the box to test the XA capabilities of the CICS TG V6.1 ECI XA resource adapter. It can be used to test connections to two different CICS regions (Figure A-8).

Figure A-8 Architecture of CTGTesterCCIXA

The pair of JCA requests can form two legs of an XA transaction.

Application overviewThe CTGTesterCCIXA application is an extension of CTGTesterCCI. The main differences to CTGTesterCCI are that the CICS ECI XA resource adapter must be used and two JCA calls are made rather than the single call made by CTGTesterCCI.

CTGTesterCCIXA contains two JCA resource references. When deploying the application (see “Deploying our sample application” on page 200), the resource references must be mapped to two connection factories.

The following is the sequence of events that occur when the end user interacts with the CTGTesterCCIXA application through a Web browser, illustrated in Figure 5-25 on page 203.

1. The user opens the Web application using the URL and enters the input parameters for the two JCA requests:

http://localhost:9080/CTGTesterCCIXAWeb/index.jsp

CCI


CTGTesterCCIXAServlet

CTGTesterCCIXA

JSP

Web browser

HTML servlet

session bean CICS ECI XA

resource adapterCCI

CICS TS

CICS program

Gatewaydaemon

CICS TG V6.1

CICS TS

CICS program

Gatewaydaemon

CICS TG V6.1


http://localhost:9080/CTGTesterCCIXAWeb/index.jsp

2. The user then clicks a button on a Web page that submits a form to the servlet.

3. The servlet receives the request for action and calls a method on the remote interface of the CTGTesterCCIXA session bean.

4. The session bean uses the CICS ECI XA resource adapter to call two CICS programs, possibly via two different Gateway daemons.

5. The CICS ECI XA resource adapter returns data from the CICS programs to the session bean.

6. The session bean returns the output data from the CICS programs back to the servlet.

7. The servlet forwards to a JSP, which displays the response to the user.

Static HTML FormThe HTML form index.jsp is stored in the WebContent folder of the CTGTesterCCIXAWeb package and can be viewed using Rational Application Developer. The HTML form contains the same fields used in CTGTesterCCIWeb (see Table A-1 on page 364), but includes a numeric suffix to distinguish the sets of parameters. Table A-2 shows the fields on the form.

Table A-2 Fields in the HTML form, CTGTesterCCI

Field Name Purpose

Mandatory details

CICS program name funcName{1+2} Program on CICS to call

COMMAREA input commareaInput{1+2}

COMMAREA data

COMMAREA length commareaLength{1+2}

Length of the COMMAREA

Encoding encoding{1+2} Encoding to convert data to before sending and after receiving

Optional details

User ID username{1+2} Username to flow on the ECI request

Password password{1+2} Password to flow on the ECI request

Mirror transaction mirror{1+2} Transaction to use on the CICS server

Iterations iterations{1+2} The number of times to run the CICS program

Application trace appTrace(common field)

Whether to enable CICS TG Java client trace


The action of the HTML form is to call the servlet CTGTesterCCIXAServlet, using the POST method to send the parameters.

CTGTesterCCIXAServletThe Servlet converts the HTML FORM user data into a format which the Session EJB can use. The Session EJB facade masks how or where the request is actually processed, providing separation between presentation and business logic. Figure A-9 summarizes the CTGTesterCCIXAServlet code.

Figure A-9 CTGTesterCCIXAServlet logic overview

The following gives a brief description of the main methods of CTGTesterCCIXAServlet:

� processRequest() - 1

The CTGTesterCCIXAServlet doGet() and doPost() methods both pass on the HTML form data for the request (whichever convention was used) to the processRequest()method. processRequest() creates a pair of ECIRequestDetails objects eciReq1 and eciReq2, and calls MapRequest2Details() twice to populate them, using the suffix parameter to distinguish the sets of name/value pairs.

� MapRequest2Details()

The MapRequest2Details() method takes a HttpServletRequest object (effectively the input), an ECIRequestDetails object (effectively the output) together with an optional suffix string. The field names, shown in the central column of Table A-2 are combined with the suffix input string to identify name/value pairs contained in the HttpServletRequest data. The value for

JNDINamespace

HTML FORMS DATA

processRequest()MapRequest2Details()eciReq1MapRequest2Details()eciReq2lookup ejbHome

ejbHome.create()

ejb.execute(eciReq1,eciEeq2)

results.jsp/exception.jsp

ejb.execute()



each field is copied into the corresponding EciRequestDetails class variable, performing any type conversions from the user input that are required.

MapRequest2Details() is invoked with the suffix 1 to populate eciReq1, and then again with suffix 2 to populate eciReq2.

� processRequest() - 2

When the request data as been setup in objects eciReq1 and eciReq2, processRequest() obtains a reference to CTGTesterCCIHome via a lookup on the initial context for EJB reference ejb/CTGTesterCCIXA. The CTGTesterCCIXAHome.create() method is used to create an instance of the CTGTesterCCIXA EJB remote interface. The local CTGTesterCCIXA instance is called tester. The ejb reference is defined in the Web Deployment Descriptor.

Having gathered together all of the details for the pair of JCA requests, and created an instance of the Session EJB CTGTesterCCIXA remote interface, processRequest() finally invokes the execute() method on the CTGTesterCCIXA bean instance, passing in both EciRequestDetails objects, eciReq1 and eciReq2.

Session EJB CTGTesterCCIXAFigure A-10 shows the interactions between the calling servlet and session EJB CTGTesterCCIXA.

Figure A-10 CTGTesterCCIXA EJB logic overview

CTGTesterCCIXA EJB

ejbCreate()lookup ("java:comp/env/ECIAXA")create connection factory1lookup ("java:comp/env/ECIAXA")create connection factory1

execute(eciReq1,eciReq2)flowRequest(eciReq1)get CCI connection1get CCI interaction1copy eciReq1 to eSpec1loop # iterations1

interaction1.execute()flowRequest(eciReq2)get CCI connection2get CCI interaction2copy eciReq2 to eSpec2loop # iterations2

interaction2.execute()

return commareaPair or throw exception


CTGTesterCCIHome.create()

CTGTesterCCI.execute()

Invoke appropriate JSP


We give a brief description of the main methods of CTGTesterCCIXA in the following sections.

ejbCreate()The EJB life cycle method ejbCreate() is used to create two conection factories. We create an InitialContext object and use it to look up the two connection factories using the JNDI names java:comp/env/ECIAXA and java:comp/env/ECIAXB. These are the local JNDI names that will be mapped to the actual JNDI name of the connection factories at deploy time. To indicate we want such a mapping, we create two resource references in the EJB deployment descriptor with the names ECIAXA and ECIBXA (Figure A-11).

Figure A-11 CTGTesterCCIXA EJB Deployment Descriptor

execute()The only method available on the remote interface of the CTGTesterCCIXA EJB is the execute() method. This method and supporting logic is implemented by class CTGTesterCCIXABean. The EciRequestDetails, commareaPair, and JavaStringRecord classes also provide supporting function.

<resource-ref id="ResourceRef_1126776812792"><description>CICS XA ECI Resource adapter to CICS region A</description><res-ref-name>ECIAXA</res-ref-name><res-type>javax.resource.cci.ConnectionFactory</res-type><res-auth>Container</res-auth><res-sharing-scope>Shareable</res-sharing-scope>

</resource-ref><resource-ref id="ResourceRef_1126872415486">

<res-ref-name>ECIBXA</res-ref-name><res-type>javax.resource.cci.ConnectionFactory</res-type><res-auth>Container</res-auth><res-sharing-scope>Shareable</res-sharing-scope>

</resource-ref>

Note: The transaction attribute is set in the assembly descriptor section of the EJB deployment descriptor for the execute() method to control the transactional behavior of the EJB and associated JCA requests. In the version of CTGTesterCCIXA provided with this book, the transaction attribute is set to Required.


The execute() method calls flowEciRequest() twice:

� The first request processes eciReq1, using ECIInteractionSpec eSpec1 and the resource reference ECIAXA

� The second request processes eciReq2, using ECIInteractionSpec eSpec2 and the resource reference ECIBXA

The final difference to the CTGTesterCCI EJB execute() method is the return object. The CTGTesterCCIXA.execute() method returns the two COMMAREAS in a commareaPair object which consists of a pair of String objects.

flowEciRequest()The flowEciRequest() method is unchanged from CTGTesterCCI. Refer to “flowEciRequest()” on page 377.

Catching and handling a CICS transaction abendThe handling of CICS transaction abends is unchanged from CTGTesterCCI. Refer to “Catching and handling a CICS transaction abend” on page 368.

JSPsThe application uses two JSPs to format and display the results. One of two scenarios can occur; the request completes successfully, or an exception occurs inside the CICS ECI XA resource adapter or the application. The JSPs are stored in the CTGTesterCCIXAWeb WebContent folder and can be viewed using Rational Application Developer.

The only difference to the CTGTesterCCIWeb version of results.jsp is the inclusion of an additional COMMAREA for the second JCA request.

Note: The first request completes all of its iterations before the second request starts its first iteration.


Appendix B. DFHCNV and CICS data conversion

Because WebSphere Application Server runs on ASCII based platforms and CICS runs on an EBCDIC based platform, the need arises for code page conversion to be performed on data passed between J2EE applications and CICS programs.

In this appendix, we explain how the data that is passed to a CICS program on an ECI call can be converted correctly between ASCII and EBCDIC.

B


Data conversionWe discuss the following data conversion considerations:

� DFHCNV � Code page aware Java programs with DFHCNV� Code page aware Java programs without DFHCNV

DFHCNV and the mirror programECI applications use the facilities of the CICS mirror program to link to the specified user program, passing the COMMAREA for input and output. The CICS mirror program (DFHMIRS) invokes the services of the data conversion program (DFHCCNV) to perform the necessary code page conversion of the COMMAREA (Figure B-1).

Figure B-1 ECI data conversion

Target application

program

COMMAREA

CICS TS V3.1

CICS TG V6

z/OS

ECI

DFHCCNV

Mirror program

DFHMIRS

Distributed platformor z/OS


If DFHCCNV finds a conversion template in the DFHCNV table whose resource name (RNAME) matches the target program name, it performs code page conversion for the COMMAREA associated with the ECI request. Example B-1 shows the DFHCNV table entry that we used for the program EC01 in our IVP test.

Example: B-1 DFHCNV entry

DFHCNV TYPE=ENTRY,RTYPE=PC,RNAME=EC01,USREXIT=NO, SRVERCP=037,CLINTCP=850 DFHCNV TYPE=SELECT,OPTION=DEFAULT DFHCNV TYPE=FIELD,OFFSET=0,DATATYP=CHARACTER,DATALEN=32767, LAST=YES

The SRVERCP represents the EBCDIC code page in which the data is stored within CICS. The CLINTCP is the default code page for client requests. The DATALEN is the maximum length of the COMMAREA you require to be translated.

Code page aware Java programsAll Java strings are stored in Unicode, a double byte character set which is similar to ASCII. The trailing byte maps to the ASCII code point for the common ASCII characters, for example, the character “A” represented by the ASCII code point X ‘41’ is represented in Unicode by X ‘0041’. The COMMAREA flowed to CICS in an ECI request must be a byte array (composed of single byte characters).

Typically, within Java, the getBytes() method on the String object is used to convert a String to a byte array, and similarly, the default String constructor is used to convert a byte array into a String. Unless an encoding is specified on these calls, the conversion from Unicode to the byte array will be performed using the default platform encoding. On Intel or UNIX platforms, this encoding will usually default to U.S. ASCII (ISO 8859-1); however, within the z/OS JVM, it could also be 1047 (an extended 037 EBCDIC code page).

Note: Since the arrival of WebSphere Application Server V5 on z/OS, the default JVM encoding has been U.S. ASCII (ISO 8859-1). This allows EJB components that rely on the JVM behavior instead of explicit specification of encoding to execute unchanged when ported to WebSphere Application Server for z/OS.

Appendix B. DFHCNV and CICS data conversion 381

Because Unicode and ASCII share the first 256 code points, String to ASCII byte array conversion performs well, as it just involves removal of the high-order byte. Example B-2 shows an example of this technique using the ECI Sample JavaStringRecord class provided by the CICS TG to perform the data conversion from Unicode to an ASCII byte array.

Example: B-2 Code page-aware Java application, ASCII COMMAREA

Context ic = new InitialContext();cxfn = (ConnectionFactory) ic.lookup("java:comp/env/eis/ECICICS");Connection cxn= cxnf.getConnection();Interaction ixn= cxn.createInteraction();ECIInteractionSpec ixnSpec= new ECIInteractionSpec(SYNC_SEND_RECEIVE,"ECIPROG");JavaStringRecord jsr = new JavaStringRecord("8859_1");jsr.setText("DATA1");ixn.execute(ixnSpec, jsr, jsr);ixn.close();cxn.close();

This technique means that the COMMAREA character data always flows to CICS in ASCII, and so a DFHCNV table entry is required in CICS to convert the COMMAREA from ASCII to EBCDIC.

Numeric dataIf you wish to flow numeric data to CICS from a Java application then it will be necessary to convert all integer values into a byte array before they can be passed to CICS. The description of how to do this is beyond the scope of this book, but further details are provided in Appendix B of Java Connectors for CICS: Featuring the J2EE Connector Architecture, SG24-6401.

Code page aware Java programs without DFHCNV.An alternative to passing an ASCII COMMAREA to CICS and having the ASCII data converted to EBCDIC is to create an EBCDIC byte array, using the code page IBM037 as the encoding. This removes the need for the DFHCNV usage in CICS but can increase the cost within the Java code, because conversion from Unicode to EBCDIC in Java requires a table lookup rather than simply removing the high-order byte.


Example B-3 shows a Java code example.

Example: B-3 Code page-aware Java application — EBCDIC COMMAREA

Context ic = new InitialContext();cxfn = (ConnectionFactory) ic.lookup("java:comp/env/eis/ECICICS");Connection cxn= cxnf.getConnection();Interaction ixn= cxn.createInteraction();ECIInteractionSpec ixnSpec= new ECIInteractionSpec();ixnSpec.setInteractionVerb(ixnSpec.SYNC_SEND_RECEIVE);ixnSpec.setFunctionName("ECIPROG");JavaStringRecord jsr = new JavaStringRecord("IBM037");jsr.setText("DATA1");ixn.execute(ixnSpec, jsr, jsr);ixn.close();cxn.close();

Note: The CTGTesterCCI application allows you to specify different code page encodings.

Appendix B. DFHCNV and CICS data conversion 383

Appendix C. EXCI Trace

This appendix provides information about using EXCI trace to diagnose CICS TG problems.

C


EXCI traceThe EXCI trace captures details of all EXCI calls which the Gateway daemon makes to CICS. Exception trace entries are always written to a trace table in the Gateway daemon address space. A more detailed trace can be obtained by coding TRACE=2 in DFHXCOPT as shown in Example C-1. It can be viewed by dumping the address space and formatting using the IPCS VERBEXIT supplied with CICS. The process for viewing an EXCI trace is outlined in the manual CICS Transaction Gateway for z/OS Administration, SC34-6672. It is repeated here for clarity.

To obtain the EXCI trace:

1. Use the following command to display all OMVS tasks:

/D OMVS,A=ALL

Example: C-1 CICS TG ASID displayed using /D OMVS,A=ALL

CITGR1G CITGR1G 006D 16777742 1 1F---- 09.58.25 10.028 LATCHWAITPID= 0 CMD=CTGBATCH

2. Find the Gateway address space, and note the address space ID (ASID).

3. Enter the DUMP command with a suitable comment. For example:

/DUMP COMM=(CICS TG DEMO DUMP)

4. Reply to the message with the ASID as follows:

/R 86,ASID=6D,END

where 86 is the message number for the reply and 6D is the ASID.

5. The system dumps as shown in Example C-2. Note the dump name created. In our example, the dump name was SYS1.DUMP.D050923.T141658.X1.S00007.

Example: C-2 Dumping the Gateway address space in SDSF

IEE600I REPLY TO 86 IS;ASID=6D,END IEA794I SVC DUMP HAS CAPTURED: 506 DUMPID=008 REQUESTED BY JOB (*MASTER*) DUMP TITLE=CICS TG DEMO DUMP IEF196I IGD100I 6614 ALLOCATED TO DDNAME SYS00023 DATACLAS ( ) IEF196I IEF285I SYS1.DUMP.D050927.T110610.X1.S00008 CATALOGEDIEF196I IEF285I VOL SER NOS= MOXWK0. IEA611I COMPLETE DUMP ON SYS1.DUMP.D050927.T110610.X1.S00008 510

Tip: The /D OMVS,A=<asid> command displays details of the process running in the address space.


6. Go into IPCS and select option 0. Enter the dump name as shown in Figure C-1.

Figure C-1 Selecting the dump data set in IPCS

7. Format the dump using the CICS TS V3.1 trace formatter by going to option 6 within IPCS and entering VERBX DFHPD640 ‘TR=1’ (the last parameter requests an abbreviated trace). Pressing Enter on this screen formats the dump. IPCS asks whether you want to use summary dump data. Reply Y as shown in Figure C-2.

------------------------- IPCS Default Values ---------------------------------Command ===> You may change any of the defaults listed below. The defaults shown before any changes are LOCAL. Change scope to GLOBAL to display global defaults. Scope ==> BOTH (LOCAL, GLOBAL, or BOTH) If you change the Source default, IPCS will display the current default Address Space for the new source and will ignore any data entered in the Address Space field. Source ==> DSNAME('SYS1.DUMP.D050927.T110610.X1.S00008') Address Space ==> Message Routing ==> NOPRINT TERMINAL Message Control ==> CONFIRM VERIFY FLAG(WARNING) Display Content ==> NOMACHINE REMARK REQUEST NOSTORAGE SYMBOL Press ENTER to update defaults. Use the END command to exit without an update.

Appendix C. EXCI Trace 387

Figure C-2 3270 ISPF screen capture

8. On the resulting IPCS formatted dump output, you now need to find the EXCI trace table. To do this, enter the command FIND ‘TRACE TABLE’. Example C-3 shows the output from our job. Note the EXIT ECI_PARAMETERS_OUTPUT line. The return code of -3 equates to an ECI_ERR_NO_CICS.

Example: C-3 IPCS dump trace table

INTERNAL TRACE TABLE XCI EXCI EX 03ED ECI ENTRY ECI_SERVER_LIST_ENTRY_PARAMETERS 20 XCI EXCI EX 03EE ECI EXIT ECI_SERVER_LIST_EXIT_PARAMETERS 0 XCI EXCI EX 8000 JVDLL ENTRY ECI_PARAMETERS_PASSED Worker-9,1 XCI EXCI EX 8003 JVDLL DATA CONVERTED_PARAMETER Worker-9,jstrServer,A6POTGR1 XCI EXCI EX 8003 JVDLL DATA CONVERTED_PARAMETER Worker-9,jstrProgram,EC01 XCI EXCI EX 8004 JVDLL DATA INBOUND_COMMAREA Worker-9,18,0F9440D8 XCI EXCI EX 0439 ????? ????? **_POINT_ID_NOT_RECOGNISED_** XCI EXCI EX 043B ????? ????? **_POINT_ID_NOT_RECOGNISED_** XCI EXCI EX 8021 JVDLL DATA FIRST_REQUEST_ON_TCB_ADDRESS 008CAE88 XCI EXCI EX 8010 JVDLL *EXC* ERROR_RESPONSE_RECEIVED 3,-3

The CICS TG trace entries are documented in Chapter 11, Problem Determination, of the CICS Transaction Gateway for z/OS Administration, SC34-6672.


EXCI trace and GTFIt is possible to use GTF to analyze both CICS and EXCI traces. This tracing is well suited to in-depth error analysis because it provides you can interleave both EXCI and CICS trace entries. It does require, however, that you start and initialize the GTF process in order to collect the data.

Example C-4 gives a brief explanation of how we debugged a simple problem with the CICS TG for z/OS using GTF tracing. In this error scenario, the call using EciB1 failed with ECI_ERR_TRANSACTION_ABEND (-7) and AEI0 due to the disabling of the invoked program EC01 within CICS.

The steps in this scenario are as follows:

� You must enable both EXCI and CICS trace entries to be written to GTF. The DFHXCOPT table controls EXCI tracing and activates an EXCI trace. You specify TRACE=2 and GTF=ON in the DFHXCOPT table (Example C-4).

Example: C-4 EXCI options table, DFHXCOPT

DFHXCO TYPE=CSECT, TIMEOUT=0, No Timeout TRACE=2, Trace Entries TRACESZE=4096, Trace table size DURETRY=30, Retry SDUMPS for 30 seconds TRAP=OFF, DFHXCTRA - OFF GTF=ON, GTF - ON MSGCASE=MIXED, Mixed case messages CICSSVC=0, EXCI will obtain CICS SVC number CONFDATA=SHOW, Show user commarea data in trace SURROGCHK=NO, Perform e-user check END DFHXCOPT

� The customized DFHXCOPT must be in the STEPLIB. This can be ensured by specifying the library containing the DFHXCOPT table in the EXCI_OPTIONS environment variable in STDENV.

� Restart the Gateway daemon to pick up the changed DFHXCOPT.

Note: The principal advantage of GTF tracing is that the EXCI and CICS trace entries can be combined in the same output, which can help considerably in problem determination. However, the Gateway daemon and JNI trace cannot be written to GTF and must be analyzed separately.


� CICS GTF tracing can be activated using the CETR transaction (Figure C-3) by setting the GTF Trace Status to STARTED. We also reduced the CICS trace entries to IS=1-2 and PC=1 to capture only MRO and program control trace entries. This can be achieved using PF4 on the CETR screen.

Figure C-3 Turning on CICS GTF tracing using CETR

� To activate GTF you need to start your GTF started task. The JCL for our started task is shown in Example C-5. We used the command /S GTF2.EXTRC,ID=CITGCA to pass the token EXTRC and user ID CITGCA.

CETR CICS Trace Control Facility TGR1 A6POTGR1 Type in your choices. Item Choice Possible choices Internal Trace Status ===> STARTED STArted, STOpped Internal Trace Table Size ===> 1024 K 16K - 1048576K Auxiliary Trace Status ===> STOPPED STArted, STOpped, Paused Auxiliary Trace Dataset ===> A A, B Auxiliary Switch Status ===> NO NO, NExt, All GTF Trace Status ===> STARTED STArted, STOpped Master System Trace Flag ===> ON ON, OFf Master User Trace Flag ===> ON ON, OFf When finished, press ENTER. PF1=Help 3=Quit 4=Components 5=Ter/Trn 6=JVM 9=Error List


Example: C-5 GTF2 proc

//GTF2 PROC MEMBER=GTFPARM,ID=DEFAULT //* // EXEC PGM=IEFBR14 //DSN DD DISP=(MOD,DELETE),DSN=&ID..GTF.TRACE, // UNIT=SYSDA,SPACE=(TRK,(1)) //* //GTF EXEC PGM=AHLGTF,PARM='MODE=EXT,DEBUG=NO,TIME=YES,BLOK=4M', // REGION=40M //IEFRDER DD DSN=&ID..GTF.TRACE, // DISP=(NEW,CATLG),UNIT=SYSDA,SPACE=(CYL,(250)), // DCB=(LRECL=4092,BLKSIZE=4096,RECFM=VB) , //SYSPRINT DD SYSOUT=H //SYSLIB DD DSN=SYS1.PARMLIB1(&MEMBER),DISP=SHR

� The SYS1.PARMLIB1(GTF) member referenced in Example C-5 defines the following options for GTF:

TRACE=SYSM,USR,TRC,DSP,PCI,SRM,RNIO

� When GTF has initialized, you need to reply U to the initialization message (Example C-6).

Example: C-6 Reply to GTF commands

R 99,U IEE600I REPLY TO 99 IS;U U AHL906I THE OUTPUT BLOCK SIZE OF 4096 WILL BE USED FOR OUTPUT 838 DATA SETS: CITGCA.GTF.TRACE AHL080I GTF STORAGE USED FOR GTF DATA: 839 GTFBLOCK STORAGE 4096K BYTES (BLOK= 4096K) PRIVATE STORAGE 1024K BYTES (SIZE= 1024K) SADMP HISTORY 40K BYTES (SADMP= 40K) SDUMP HISTORY 40K BYTES (SDUMP= 40K) ABEND DUMP DATA 0K BYTES (ABDUMP= 0K) AHL031I GTF INITIALIZATION COMPLETE

� GTF tracing is now active and trace entries from the Gateway daemon and the CICS region will be captured by GTF.

� Once the problem you wish to trace has been re-created, terminate the GTF started task using the token specified on initialization, for example, we issued the command /P EXTRC.

� Go to IPCS option 0 and set the name of the GTF trace data which has been created by GTF. In our case the data set was called CITGCA.GTF.TRACE.


� Select PF3 to return to the IPCS main menu, select option 6 and enter the command:

GTF USR(F6C)

This produces the formatted trace output for the EXCI and CICS trace entries. An abbreviated output from the formatted trace (Example C-7), showing the ECI return code -7 caused by the our PGMIDERR condition.

Example: C-7 Formatted GTF EXCI and CICS trace entries

EX 8003 JVDLL DATA - CONVERTED_PARAMETER JAVA_TID(Worker-0) NAME(jstrServer) VALUE(A6POTGR1) EX 8003 JVDLL DATA - CONVERTED_PARAMETER JAVA_TID(Worker-0) NAME(jstrProgram) VALUE(EC01 )EX 1000 XCPRH ENTRY - OPEN_PIPE TOKEN(0F6C8140) TO CICS(A6POTGR1) BY USER(CITGR1G ) EX 2001 XCPRH EVENT IRP_CONNECT TO CICS(A6POTGR1) BY USER(CITGR1G )EX 1000 XCPRH ENTRY - OPEN_PIPE TOKEN(0F6C8140) TO CICS(A6POTGR1) BY USER(CITGR1G ) EX 1000 XCPRH ENTRY - DPL TO PROGRAM(EC01 ) ON CICS(A6POTGR1) USING PIPE(0F6C8140) BY USER(CITGR1G ) EX 2004 XCPRH EVENT SWITCH_REQUEST SENT TO CICS(A6POTGR1) BY USER(CITGR1G )EX 2005 XCPRH DATA DATA SENT ON PIPE(0F6C8140) BY USER(CITGR1G ) AP DD18 CRNP EVENT - DEQUEUE WORK ELEMENT TYPE (NEW CONNECT WITH INPUT RECEIVED) TIMESTAMP(BDAE180BD69185A7) SCCB AT 7F4A2E80 SYSTEM CITGR1G EX 0802 XCPRH *EXC* - PGMIDERR_DETECTED - PROGRAM(EC01 ) ON CICS(A6POTGR1) BY USER(CITGR1G )EX 8011 JVDLL *EXC* - DPL_REQUEST_ERROR ECI_RC(-7) EX 1000 XCPRH ENTRY - CLOSE_PIPE TOKEN(0F6C8140) TO CICS(A6POTGR1) BY USER(CITGR1G )EX 2002 XCPRH EVENT IRP_DISCONNECT FROM CICS(A6POTGR1) BY USER(CITGR1G )


Appendix D. Additional material

This appendix describes the additional material to which this redbook refers that you can download from the Internet.

Locating the Web materialThe Web material associated with this redbook is available in softcopy format on the Internet from the redbooks Web server.

Point your Web browser to:

ftp://www.redbooks.ibm.com/redbooks/SG247161

Alternatively, you can go to the redbooks Web site at:

http://www.redbooks.ibm.com

Select Additional Materials and open the file that corresponds with the redbook form number SG247161.

D


ftp://www.redbooks.ibm.com/redbooks/SG247161

http://www.redbooks.ibm.com

Using the Web materialThe additional Web material that accompanies this redbook includes the following files:

File name DescriptionSG247161.zip Zipped Code Samples

The SG247161.zip file includes the three EAR files which we used in our test scenarios. Table D-1 provides a short description of each EAR file.

Table D-1 EAR files

For further details about the sample J2EE applications see Appendix A, “Sample applications” on page 361.

System requirements for downloading the Web materialThere are no specific system requirements for your workstation to download the supplied materials. However, to be able to use the supplied samples, you need to adhere to product specifications as described throughout this book.

How to use the Web materialCreate a subdirectory (folder) on your workstation, and unzip the contents of the Web material zipped file into this folder.

EAR files Description

CTGTesterCCI.ear Includes the CTGTesterCCI application code that we used in our connectivity tests in chapters 3 and 4 and in our security tests in chapters 8 and 9.

CTGTesterCCIXA.ear Includes the CTGTesterCCIXA application code that we used in our transaction tests in chapters 5, 6, and 7.

CTGTesterCCI_Sec.ear Includes the CTGTesterCCI application code with deployment descriptors that define a security constraint. We used this EAR in our Thread Identity Support test scenario in Chapter 10.


acronyms

AOR Application-Owning Region

API Application Programming Interface

APPC Advanced Program-to-Program Communications

ASCII American Standard Code for Information Interchange

CCI Common Client Interface

CICS Customer Information Control System

CICS TG CICS Transaction Gateway

DPL Distributed Program Link

EAR Enterprise Application Archive

EBCDIC Extended Binary Coded Decimal Interchange Code

ECI External Call Interface

EIS Enterprise Information Systems

EJB Enterprise JavaBeans™

EPI External Presentation Interface

ESI External Security Interface

ESM External Security Manager

EXCI External CICS Interface

FTP File Transfer Protocol

GTF Generalized Trace Facility

GTF Generalized Trace Facility

HFS Hierarchical File System

HTML Hypertext Transfer Protocol

HTTP Hypertext Markup Language

IBM International Business Machines

Abbreviations and

© Copyright IBM Corp. 2006. All rights reserved.

IDE Integrated development environment

IRC inter-region communication

ISC Inter-System Communication

ITSO International Technical Support Organization

J2EE Java 2 Enterprise Edition

JAR Java Archive

JCA J2EE Connector Architecture

JDBC Java Database Connectivity

JDK Java Developer’s Kit

JNDI Java Naming and Directory Interface™

JNI Java Native Interface

JSP JavaServer™ Page

JSSE Java Secure Sockets Extension

JVM Java Virtual Machine

LPAR Logical Partition

LUW Logical Unit of Work

MRO Multi Region Operation

PEM Password Expiration Management

RAR Resource Adapter Archive

RRMS Recoverable Resource ManagementServices

RRS Resource Recovery Services

SAF System Authorization Facility

SNA Systems Network Architecture

SSL Secure Sockets Layer

UOW Unit of Work

UR Unit of Recovery

395

USS Unix System Services

VIPA Virtual IP Addressing

XCF Cross Coupling Facility

XID X/Open identifier


Related publications

The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this redbook.

IBM RedbooksFor information about ordering these publications, see “How to get IBM Redbooks” on page 398. Note that some of the documents referenced here might be available in softcopy only.

� CICS Transaction Gateway V5 The WebSphere Connector for CICS, SG24-6133

� Revealed! Architecting e-business Access to CICS, SG24-5466

� WebSphere for z/OS V6 Connectivity Handbook, SG24-7064

� Java Connectors for CICS: Featuring the J2EE Connector Architecture, SG24-6401

Other publicationsThese publications are also relevant as further information sources:

� CICS External Interfaces Guide, SC33-1944

� CICS Intercommunication Guide, SC34-6448

� CICS Transaction Gateway for z/OS Administration, SC34-6672

� CICS Transaction Server for z/OS Installation Guide V3.1, GC34-6326

� Program Directory for CICS Transaction Gateway for z/OS, GI10-2587

� The z/OS Security Server (RACF) Command Reference, SA22-7687

� WebSphere Application Server for z/OS V6.0.1, Installing your application serving environment, GA22-7957

� z/OS V1R6 UNIX System Services Command Reference, SA22-7802

� z/OSV1R2.0 MVS Setting up a Sysplex, SA22-7625

� z/OS V1R6 UNIX System Services Planning, GA22-7800

� z/OS MVS Programming: Resource Recovery, SA22-7616


Online resourcesThese Web sites and URLs are also relevant as further information sources:

� CICS and CICS Transaction Gateway library

http://www.ibm.com/software/htp/cics/library/

� WebSphere Application Server library

http://www.ibm.com/software/webservers/appserv/was/library/index.html

� IBM Technical Sales Support Site

http://www.ibm.com/support/techdocs/atsmastr.nsf/Web/TechDocs

How to get IBM RedbooksYou can search for, view, or download Redbooks, Redpapers, Hints and Tips, draft publications, and Additional materials, as well as order hardcopy Redbooks or CD-ROMs at this Web site:

ibm.com/redbooks

Help from IBMIBM Support and downloads

ibm.com/support

IBM Global Services

ibm.com/services


http://www.ibm.com/software/htp/cics/library/



http://www.ibm.com/support/

http://www.ibm.com/support/

http://www.ibm.com/services/

http://www.ibm.com/services/

http://www.ibm.com/software/webservers/appserv/was/library/index.html

http://www.ibm.com/support/techdocs/atsmastr.nsf/Web/TechDocs

Index

Symbols_BPX_SHAREAS environment variable 38

AAbend ARXC 278abend code

ECIP 183S806 61

ABENDBKOUT, EXCI option 166, 198, 259, 369address space 31, 97, 137, 149, 164, 195, 386

balance incoming socket open requests 103required sharing 32

Address Space Initiation utility (ctgasi) 220Advanced Encryption Standard (AES) 313Advanced Program Function (APF) 33AIX 4Anynet LU6.2/IP 17APARS

PK17426 166PK17427 166PK17700 334PQ92943 97, 137PQ99940 200, 261

APPC connection 21Application Server

Network Deployment V6.0.2 79, 310V6.0 59

ASCII text 50, 298, 326AUTH_USERID_PASSWORD 288, 295, 343–344Authorized Program Facility (APF) 197Automatic Restart Management (ARM) 51

Bbasic authentication 351, 353–354bind security 285, 287, 307, 342–343, 357

CCA certificate 315ccf2.jar 30CEDA ALTER TRANS 101, 141CEMT INQUIRE EXCI

command 185, 267

© Copyright IBM Corp. 2006. All rights reserved.

CEMT INQUIRE UOWcommand 241INDOUBT 245

changing a password 10checklist

software 26, 78–79, 123, 159, 227, 255, 282, 310, 341

checklist, definitions 27, 79, 123, 174, 236, 258, 283, 311, 341CICS ECI resource adapter

connection factory 83transactional capabilities 167

CICS ECI resource adapter (cicseci.rar) 59, 78, 122, 167, 227, 256, 294, 334, 340, 361CICS ECI XA resource adapter (cicseciXA.rar) 169–170, 193, 199CICS EPI resource adapter (cicsepi.rar) 10CICS TG

configuring 33configuring security 286, 342ctginstall 27installing

running ctginstall 27master process 197, 225master process CITGR0G 232master process configuration 230master process environment variable 231master process immediate shutdown 249master process initialization 248master process shutdown 249multiple CICS regions 100overview 4password checking 286, 288, 342previous versions 311product executables 33product file 27security tasks 30SMP/E installation 27STDERR message 62STDOUT log 116STDOUT output 46transactional support 45, 165–166user ID password authentication 296XA resource adapter 194

399

CICS TG for Multiplatforms 4–5CICS TG for z/OS V6.1 6CICS Transaction Gateway. See CICS TG.CICS Transaction Server. See CICS TS.CICS TS 7, 26–27, 77, 79, 121, 123, 159, 165, 236, 257, 282–283, 310–311, 341CICS Universal Client V6 5CICSCLI environment variable 38cicseci.rar 9, 14–15cicseciXA.rar 9, 15cicsepi.rar 14cicsj2ee.jar 30CLASSPATH environment variable 39Client authentication 309

CICS TG configuration 329client certificate 312

one-to-one mapping 332COBOL 4COLUMNS environment variable 39command

CEMT INQUIRE EXCI 185, 267CEMT INQUIRE UOW 241RACDCERT 319

COMMAREA 8, 46, 96, 144, 183, 237, 266, 298, 355, 361, 380COMMAREA length 72, 187, 364commit optimization 17Common Client Interface (CCI) 9, 12Common Connector Framework (CCF) 9Common Cryptographic Architecture (CCA) 315Communications Server for Linux 21configuration file 29, 33, 36

default name 36configuration tool 59configuring

CICS TG 33CONNECTION definition 43connection factory 82, 124, 128, 169, 228, 235, 258–259, 293, 328, 348, 362

actual JNDI name 367CICS regions 198connection pool 108custom properties 350General Properties panel 84JNDI name 135KeyRingClass definition 336socket connections 235TraceLevel parameter 119

connection management contract 13

connection managerthread 92–93

connection pool 87–88, 132, 296, 327property 87, 131

connection pooling 17–18, 20connector.jar 30contract

connection management 13security 13transaction management 13

contracts, system-level 13creating a Gateway daemon configuration file 36creating an HFS data set 33creating an STDENV file 37creating the started task JCL 42credentials 17CTG.INI 36, 58CTG_JNI_TRACE 152–153, 278CTG_RRMNAME 175, 188, 196, 208, 217, 220–221CTG_RRMNAME environment variable 39ctgadmin 6ctgasi utility 195ctgclient.jar 30ctgd 6ctgenvvar 29ctgenvvarsamp 29ctginstall

running 28CTGRRMS 194, 230ctgsamp.ini 29ctgsamples.jar 30ctgserver.jar 30ctgstart 29CTGTesterCCI application 115, 136, 158, 295–296, 328, 333, 350, 362, 383

end user interacts 363password information 299

CTGTesterCCI inputscreen 301, 354

CTGTesterCCIXA application 199–200, 237, 254, 361, 372

Ddata conversion 46, 380definitions checklist 27, 79, 123, 174, 236, 258, 283, 311, 341DFHJVPIPE environment variable 39, 61, 287, 343


DFHJVSYSTEM environment variable 40DFHMIRS, CICS mirror program 380DFHXCURM 100, 106diagnosing problems 385distributed program link (DPL) 8, 160Domain Name Services (DNS) 95DSA 313dumping the Gateway address space 386dynamic trace 69, 71

EEC01, COBOL example 46, 50ECI

password, verify 10ECI request 9, 48, 93, 140, 166, 227, 285, 332, 346, 361, 381

complete sequence 229Gateway process sequence 206JNI trace 206specific Gateway daemon instance 228XID information 206

ECI sample 297ECI_ERR_NO_CICS 50, 60, 70, 388ECI_ERR_RESOURCE_SHORTAGE 116ECI_ERR_SECURITY_ERROR 306–307ECI_ERR_SYSTEM_ERROR 116EJB container 171Enterprise Information System (EIS) 11, 167, 293, 347environment variable 37, 42, 92, 137, 231

_BPX_SHAREAS 38CICSCLI 38CLASSPATH 39COLUMNS 39CTG_RRMNAME 39DFHJVPIPE 39, 61, 287, 343DFHJVSYSTEM 40PATH 40search order 41STEPLIB 40TMPDIR 40TZ 40

EPI 9EPI support classes 10EPIRequest 9ESI 10

password, change 10EXCI connection 43

EXCI interface 7EXCI options table 98, 166, 292, 347EXCI pipe 44, 96, 131

maximum number 108maximum potential number 101

EXCI request 98, 148, 285, 345flowed user ID 289limited workload balancing 100security context 292

EXCI trace 73EXCI_LOADLIB 61EXCI_OPTIONS 292, 389extended LUW 178, 227

JNI trace 189extended LUWs

TCP/IP load balancing 227External Call Interface. See ECI.External Presentation Interface.See EPI.External Security Interface. See ESI.external security manager (ESM) 10, 283

FFACILITY class, RACF 31, 115, 196, 220, 245, 251, 287, 342–343files

ccf2.jar 30cicseci.rar 9, 14–15cicseciXA.rar 9, 15cicsepi.rar 14cicsj2ee 30connector.jar 30ctgclient.jar 30ctgsamp.ini 29ctgsamples.jar 30ctgserver.jar 30guide.jar 30psk.jar 30STDENV 37

flowed user ID 283–286, 289, 291–292, 308, 342, 345–346formatting traces, IPCS 392

GGateway daemon 5, 18, 35, 56, 79, 82, 84, 126, 166–167, 169, 225, 227, 257, 261, 285, 309, 319, 361, 373, 386

cloned group 226configuration files 36

Index 401

connection details 362dummy DD statement 65ECI requests 230EXCI connection 105RACF permissions 56worker threads 109

Gateway daemon configuration file 36Gateway trace 68Generalized Trace Facility (GTF) 68get-use-cache model 14global transaction 157, 226, 245, 253, 258, 361

Configuring 258full support 168

group configuration 234group Gateway daemon instances 237group master process access 235group normal shutdown sequence 248group RRM name 231guide.jar 30

Hhardware cryptography 314HFS data set, creating 33HFS data set, mounting 34HFS directory 39, 80high availability configuration 105HiperSockets 21HP-UX 4hwkeytool 312, 314–316

IIBM SDK 26, 79, 123, 159, 227, 255, 282, 310, 341

security policy file 325ikeyman 312, 316, 320IMS JDBC 256in-doubt 162, 213, 222–223, 229, 236–239, 241–246, 248, 252initconnect, Gateway daemon parameter 94, 108initworker, Gateway daemon parameter 94Install Shield Multiplatform 6Intersystem communication (ISC) 257IPCS, formatting traces 392

JJ2EE Connector Architecture. See JCA.Java 2 Platform Enterprise Edition 9, 77, 90, 121, 379

RunAs policy 347secure access 293

Java client trace 362Java Cryptography Extension 315Java Development Kit 63, 297Java Secure Sockets Extension (JSSE) 309JCA 6

authentication entry 17Connection Pool 131, 190overview 11security 296Version 1.5 13with different topologies 15

JDBC 13JNDI name 135, 179, 353, 367JNI trace 68, 71, 152–153, 184, 186, 206, 278, 307

Kkey ring 309

personal certificates 319keystore 314, 316–317keytool 316–317

LLanguage Environment (LE) 66last-participant support 17last-resource commit optimization 17lazy connection association 14lazy transaction enlistment 14link pack area (LPA) 138Link user ID 284–285, 344Linked List Area (LLA) 196Linux 4Linux on zSeries 21Linux, RHEL V3 5local connection 122, 126, 256, 340, 361local mode of operation 11local transaction 80, 176, 178, 183, 189, 193, 368LocalTransaction interface 168LOGONLIM 97–98, 108–109, 117, 137LPAR 97, 165, 226Luw_Token 187

Mmaxconnect, Gateway daemon parameter 93–94, 108maxworker, Gateway daemon parameter 93–95,


108Message Authentication Code (MAC) 315mounting an HFS data set 34MRO bind security 115, 287, 307, 342–343

NNETNAME, CONNECTION definition 27, 56, 79, 123, 174, 194, 236–237, 258, 283, 311, 341netstat, TCP/IP test tool 49, 62nonames, Gateway daemon parameter 94–95

OOMVS 63, 386overview of CICS TG 4

Pparallel sysplex 103–104, 149password, changing 10password, verity 10PATH environment variable 40Performance Monitoring Infrastructure (PMI) 146Peripheral Component Interconnect (PCI) 312private mirror 86, 123protocol handler 91psk.jar 30

QQuality of Service (QOS) 103, 149

RRACDCERT command 319RACF

BPX.FILEATTR.PROGCTL FACILITY class 31BPX.SERVER FACILITY class 31TCICSTRN class 291, 307, 346

RACF error 306, 357RACF security tasks 30RACF user ID 294, 332, 344Rational Application Developer (RAD) 177, 350, 364, 370, 373, 377RECEIVECOUNT 44, 97, 137recoverable resource 158, 182, 226, 254

committed and backed-out updates 182region userid 337remote connection 122, 256, 362

XA transaction support 257XAResource interface 256

remote mode of operation 10resource adapter 12, 14, 22, 60, 79, 82–83, 127, 167, 174, 227, 255, 294, 328resource manager

logs information 273resource manager (RM) 45, 158, 161, 226, 254–255Resource Recovery Management Service (RRMS) 45Resource Recovery Service (RRS) 39, 163, 255resource reference 135, 174, 200, 259, 293, 302, 348, 361RRMS, SIT parameter 45, 139, 175RRMSEXIT state 204, 240, 266RRS archive log 186, 245, 266

Ssample application 110, 158, 178, 299, 348, 361sample ECI application EciB1 281, 309sample J2EE application 78, 122, 124, 182, 263, 339

CTGTesterCCI 80, 107, 142, 281, 309CTGTesterCCIXA 201, 234

scriptsctgenvvarsamp 29ctginstall 28ctgstart 29

SCTGLINK library 33SDSF log 48Secure Sockets Layer (SSL) 309security 6, 283, 286, 342

configuring CICS TG 286, 342considerations 30tasks 30

security contract 13security credentials 17security management 17, 19, 21self-signed certificate 312

public key 317subject fields 317

Servant region 123, 258, 341SESSIONS definition 43–45, 57, 97, 101, 109, 116, 139, 284, 286, 342

USERID parameter 290SETROPTS RACLIST 287, 343setting permissions 35SLES 9, Linux 5SNA network connections 17

Index 403

software checklist 26, 78–79, 123, 159, 227, 255, 282, 310, 341SSL

client authentication 305, 332client authentication error message 333client certificate 332connection 6, 282, 309–310, 399

JCA connection pooling 327performance overhead 327

keyring 50, 86, 326private keys 7protocol 5

handler 309parameter list 323

static trace 68, 71STDENV DD

card 41, 288statement 38

STDENV file 37STEPLIB environment variable 40subreason field-1 60, 192, 307subreason field-2 60, 192, 307surrogate security 292, 306, 346–347, 357syncpoint coordinator 166, 243, 256Sysplex Distributor 60, 103, 149, 227System Authorization Facility (SAF) 283system initialization table (SIT) 139system-level contracts 13systems administration tool 6systems management 6Systems Network Architecture (SNA) 16SystemSSL 59, 311, 335

TTCP

protocol 5TCP/IP 16TCP/IP connection 91, 257TCP/IP port 27, 79, 123, 174, 225, 258, 283, 341

cloned Gateway daemons 230sharing 103, 150, 227

TCP62 protocol 17Tivoli Performance Viewer 145, 189, 263TMPDIR environment variable 40topologies 15tracing

creating a data set 33dynamic trace 69, 71

EXCI trace 68, 73Gateway trace 68JNI trace 68, 71static trace 68, 71

TRANCLASS 101, 140transaction attribute 168, 173, 368

possible values 172transaction management 17, 19–20transaction management contract 13transaction manager 158, 160, 226, 254, 368

global transaction 204Transactional EXCI 100, 165, 258Transport Layer Security (TLS) 312TS Queue

RECIPROG 238two-phase commit 19–20, 100, 126, 157, 159, 161–163, 168–169, 228–229, 255–257, 277TZ environment variable 40

UUnit of Recovery (UR) 164, 238unit-of-work (UOW) 159, 238user ID 30, 112, 126, 191, 284–285, 319, 343, 362

CTGTesterCCI input 300USS program control 31

Vverifying a password 10virtual private networks 19VPN 19VTAM 17

WWeb browser 126, 183, 237, 266, 348, 363, 393

CTGTesterCCIXA application 237WebSphere Application Server 11, 13, 16, 18, 77, 100, 121, 163WebSphere servant region

EXCI call 344user ID 343

Windows 2000 4Windows 2003 4Windows XP 4work manager 164, 243, 270worker thread 49, 92–93

connection manager threads 93initial number 94


Java client disconnects 94maximum number 95

XX/Open identifier (XID) 206, 228, 272XA transaction 7, 39, 105, 157–158, 225, 257, 372

ECI request 221in-doubt failure 236Normal termination 237post-prepare phase 228

ZzSeries File System (ZFS) 33

Index 405

(0.5” spine)0.475”<

->0.875”

250 <->

459 pages


®

SG24-7161-00 ISBN 0738496227

INTERNATIONAL TECHNICALSUPPORTORGANIZATION

BUILDING TECHNICALINFORMATION BASED ONPRACTICAL EXPERIENCE

IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on realistic scenarios. Specific recommendations are provided to help you implement IT solutions more effectively in your environment.

For more information:ibm.com/redbooks


Install and configure the Gateway on z/OS

Connect from WebSphere on Windows and z/OS

Enable global transaction support and SSL security

The CICS Transaction Gateway (CICS TG) is used widely to provide access to CICS programs from Java environments. It provides the IBM implementation of the J2EE Connector Architecture (JCA).

This IBM Redbook introduces the new facilities of the CICS TG for z/OS V6.0 and V6.1, which provide improvements in the areas of transactional integration, systems management, performance, security, and ease of use.

Included in this book are a set of configuration scenarios for a variety of different deployment topologies for integrating J2EE applications on WebSphere Application Server with CICS TS for z/OS.

This book provides step-by-step explanations about how to configure connections between each tier of the test environment. We show you how to enable support for global transactions which span resources on different systems. We also cover the configuration of CICS TG security, including the use of JSSE for establishing secure SSL connections to the CICS TG.

Back cover




cics transaction gateway for z/os version 6vi cics transaction gateway for z/os version 6.1 5.2...

Documents