apm_9.5--apm for ibm cics transaction gateway guide

for IBM CICS Transaction Gateway Guide Release 9.5

CA Application Performance Management

This Documentation, which includes embedded help systems and electronically distributed materials, (hereinafter referred to as the “Documentation”) is for your informational purposes only and is subject to change or withdrawal by CA at any time.

This Documentation may not be copied, transferred, reproduced, disclosed, modified or duplicated, in whole or in part, without the prior written consent of CA. This Documentation is confidential and proprietary information of CA and may not be disclosed by you or used for any purpose other than as may be permitted in (i) a separate agreement between you and CA governing your use of the CA software to which the Documentation relates; or (ii) a separate confidentiality agreement between you and CA.

Notwithstanding the foregoing, if you are a licensed user of the software product(s) addressed in the Documentation, you may print or otherwise make available a reasonable number of copies of the Documentation for internal use by you and your employees in connection with that software, provided that all CA copyright notices and legends are affixed to each reproduced copy.

The right to print or otherwise make available copies of the Documentation is limited to the period during which the applicable license for such software remains in full force and effect. Should the license terminate for any reason, it is your responsibility to certify in writing to CA that all copies and partial copies of the Documentation have been returned to CA or destroyed.

TO THE EXTENT PERMITTED BY APPLICABLE LAW, CA PROVIDES THIS DOCUMENTATION “AS IS” WITHOUT WARRANTY OF ANY KIND, INCLUDING WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. IN NO EVENT WILL CA BE LIABLE TO YOU OR ANY THIRD PARTY FOR ANY LOSS OR DAMAGE, DIRECT OR INDIRECT, FROM THE USE OF THIS DOCUMENTATION, INCLUDING WITHOUT LIMITATION, LOST PROFITS, LOST INVESTMENT, BUSINESS INTERRUPTION, GOODWILL, OR LOST DATA, EVEN IF CA IS EXPRESSLY ADVISED IN ADVANCE OF THE POSSIBILITY OF SUCH LOSS OR DAMAGE.

The use of any software product referenced in the Documentation is governed by the applicable license agreement and such license agreement is not modified in any way by the terms of this notice.

The manufacturer of this Documentation is CA.

Provided with “Restricted Rights.” Use, duplication or disclosure by the United States Government is subject to the restrictions set forth in FAR Sections 12.212, 52.227-14, and 52.227-19(c)(1) - (2) and DFARS Section 252.227-7014(b)(3), as applicable, or their successors.

Copyright © 2013 CA. All rights reserved. All trademarks, trade names, service marks, and logos referenced herein belong to their respective companies.

CA Technologies Product References

This document references the following CA Technologies products and features:

■ CA Application Performance Management (CA APM)

■ CA Application Performance Management ChangeDetector (CA APM ChangeDetector)

■ CA Application Performance Management ErrorDetector (CA APM ErrorDetector)

■ CA Application Performance Management for CA Database Performance (CA APM for CA Database Performance)

■ CA Application Performance Management for CA SiteMinder® (CA APM for CA SiteMinder®)

■ CA Application Performance Management for CA SiteMinder® Application Server Agents (CA APM for CA SiteMinder® ASA)

■ CA Application Performance Management for IBM CICS Transaction Gateway (CA APM for IBM CICS Transaction Gateway)

■ CA Application Performance Management for IBM WebSphere Application Server (CA APM for IBM WebSphere Application Server)

■ CA Application Performance Management for IBM WebSphere Distributed Environments (CA APM for IBM WebSphere Distributed Environments)

■ CA Application Performance Management for IBM WebSphere MQ (CA APM for IBM WebSphere MQ)

■ CA Application Performance Management for IBM WebSphere Portal (CA APM for IBM WebSphere Portal)

■ CA Application Performance Management for IBM WebSphere Process Server (CA APM for IBM WebSphere Process Server)

■ CA Application Performance Management for IBM z/OS® (CA APM for IBM z/OS®)

■ CA Application Performance Management for Microsoft SharePoint (CA APM for Microsoft SharePoint)

■ CA Application Performance Management for Oracle Databases (CA APM for Oracle Databases)

■ CA Application Performance Management for Oracle Service Bus (CA APM for Oracle Service Bus)

■ CA Application Performance Management for Oracle WebLogic Portal (CA APM for Oracle WebLogic Portal)

■ CA Application Performance Management for Oracle WebLogic Server (CA APM for Oracle WebLogic Server)

■ CA Application Performance Management for SOA (CA APM for SOA)

■ CA Application Performance Management for TIBCO BusinessWorks (CA APM for TIBCO BusinessWorks)

■ CA Application Performance Management for TIBCO Enterprise Message Service (CA APM for TIBCO Enterprise Message Service)

■ CA Application Performance Management for Web Servers (CA APM for Web Servers)

■ CA Application Performance Management for webMethods Broker (CA APM for webMethods Broker)

■ CA Application Performance Management for webMethods Integration Server (CA APM for webMethods Integration Server)

■ CA Application Performance Management Integration for CA CMDB (CA APM Integration for CA CMDB)

■ CA Application Performance Management Integration for CA NSM (CA APM Integration for CA NSM)

■ CA Application Performance Management LeakHunter (CA APM LeakHunter)

■ CA Application Performance Management Transaction Generator (CA APM TG)

■ CA Cross-Enterprise Application Performance Management

■ CA Customer Experience Manager (CA CEM)

■ CA Embedded Entitlements Manager (CA EEM)

■ CA eHealth® Performance Manager (CA eHealth)

■ CA Insight™ Database Performance Monitor for DB2 for z/OS®

■ CA Introscope®

■ CA SiteMinder®

■ CA Spectrum® Infrastructure Manager (CA Spectrum)

■ CA SYSVIEW® Performance Management (CA SYSVIEW)

Contact CA Technologies

Contact CA Support

For your convenience, CA Technologies provides one site where you can access the information that you need for your Home Office, Small Business, and Enterprise CA Technologies products. At http://ca.com/support, you can access the following resources:

■ Online and telephone contact information for technical assistance and customer services

■ Information about user communities and forums

■ Product and documentation downloads

■ CA Support policies and guidelines

■ Other helpful resources appropriate for your product

Providing Feedback About Product Documentation

If you have comments or questions about CA Technologies product documentation, you can send a message to [email protected].

To provide feedback about CA Technologies product documentation, complete our short customer survey which is available on the CA Support website at http://ca.com/docs.

http://www.ca.com/support

mailto:[email protected]

http://www.ca.com/docs

http://www.ca.com/docs

Contents 7

Contents

Chapter 1: Overview of Extension for CTG 11

About Extension for CTG ............................................................................................................................................ 11

Extension for CTG system requirements ............................................................................................................. 12

Extension for CTG components ........................................................................................................................... 12

Extension for CTG and your CA Introscope® environment ........................................................................................ 13

ProbeBuilder Directives Deployment .................................................................................................................. 14

Request Exit overview ......................................................................................................................................... 17

Mixed distributed and z/OS CTG ................................................................................................................................ 19

Chapter 2: Install Extension for CTG 21

Prepare for the installation ........................................................................................................................................ 21

Extract the installation archive................................................................................................................................... 22

Install Management Modules and Typeviews for PBDs ............................................................................................. 23

Install Management Modules and Typeviews for the Request Exit ........................................................................... 23

Install Introscope agent files to monitor the CTG Server ........................................................................................... 24

Enable the CTG code using AutoProbe....................................................................................................................... 24

Enable the CTG code using Request Exit .................................................................................................................... 24

Enable the CTG code using Manual ProbeBuilder on distributed CTG platforms ...................................................... 25

Preparing to enable the CTG Server code ........................................................................................................... 25

Specifying CTG Server RequestExit Instrumentation for Windows ..................................................................... 26

Entering Manual ProbeBuilding options for the CTG Server ............................................................................... 26

Using the ProbeBuilder Wizard for the CTG Server ............................................................................................ 27

Using the command-line ProbeBuilder for the CTG Server ................................................................................ 27

Enable the CTG Client Code on WebSphere ............................................................................................................... 28

Modify the z/OS CTG startup script............................................................................................................................ 28

Create an instrumented CTG startup script for distributed systems ......................................................................... 29

Run instrumented code .............................................................................................................................................. 30

Revert to the original code ......................................................................................................................................... 31

Enable ChangeDetector on the distributed CTG platforms ........................................................................................ 31

Configure CTG Server RequestExit and Global Statistics Probing (Optional) ............................................................. 33

Configure CTG Server RequestExit and Global Statistics Probing on z/OS in Config file ctg.ini (Optional) ......... 34

Configure on Windows Platform to Obtain RequestExit Detail and Global Statistics Separately (Optional) ............................................................................................................................................................ 34

Configure CTG Client support ..................................................................................................................................... 35

Configure standalone CTG Client applications .................................................................................................... 36

Configure Channel and Container Metrics .......................................................................................................... 36

Configure Client applications under WebSphere ................................................................................................ 36

8 for IBM CICS Transaction Gateway Guide

Upgrade the Extension for CTG ........................................................................................................................... 37

Verifying the installation and configuration ............................................................................................................... 37

Chapter 3: Using Extension for CTG 39

Viewing Extension for CTG data in Introscope ........................................................................................................... 39

Viewing metric data in the tree view ......................................................................................................................... 40

Frontends ............................................................................................................................................................ 40

Backends ............................................................................................................................................................. 41

Analyzing historical data for a specific metric ..................................................................................................... 41

Server metrics ............................................................................................................................................................ 41

Backends | CTG_Global_Statistics ...................................................................................................................... 42

Backends | CTG_to_CICS_ECI_IPIC node ............................................................................................................ 43

Backends | CTG_to_CICS_EPI node .................................................................................................................... 44

Backends | JSSE to CTG node .............................................................................................................................. 44

Frontends | Client_to_CTG_Aggregates node .................................................................................................... 45

Frontends | Client_to_CTG_Details node ........................................................................................................... 47

Frontends | Client_to_CTG_Details transaction view node ............................................................................... 48

Frontends | Client_to_CTG_JSSE Session node .................................................................................................. 49

Using Introscope Investigator tab views .................................................................................................................... 49

Viewing Extension for CTG data in dashboards ......................................................................................................... 50

CTG Server - Overview dashboard ...................................................................................................................... 51

CTG Summary - Server ECI Activity dashboard ................................................................................................... 52

CTG Server - Global Statistics dashboard ............................................................................................................ 53

CTG Server - ECI/IPIC Request dashboard ........................................................................................................... 53

CTG Server - EPI Request dashboard ................................................................................................................... 54

CTG Server - Connection Manager & Workers dashboard ................................................................................. 54

CTG Server - SSL dashboard ................................................................................................................................ 55

CTG Client - Overview dashboard ....................................................................................................................... 55

CTG Client - JavaGateways and SSL Sessions dashboard .................................................................................... 56

CTG Client - EPI Request dashboard .................................................................................................................... 56

CTG Client - EPI Terminal Requests dashboard ................................................................................................... 57

Introscope ChangeDetector dashboard .............................................................................................................. 57

Modifying Introscope caution/danger alert thresholds ............................................................................................. 58

CTG Transaction Tracer .............................................................................................................................................. 58

Handling of Special Characters by PP CTG Transaction Tracer ........................................................................... 58

Detailed List of CTG Transaction Tracer Properties ............................................................................................ 59

Appendix A: CTG Performance Metrics 61

Frontend metrics ........................................................................................................................................................ 61

Frontend Client to CTG aggregates ..................................................................................................................... 63

Frontend Client to CTG details ............................................................................................................................ 64

Contents 9

Backend metrics ......................................................................................................................................................... 64

Backend CTG_to_CICS_ECI_IPIC aggregates ....................................................................................................... 65

Backend CTG Global Statistics ............................................................................................................................. 66

Backend CTG_to_CICS_EPI aggregate metrics .................................................................................................... 66

Backend CTG_to_CICS threads ............................................................................................................................ 66

CTG dashboard metrics .............................................................................................................................................. 67

CTGClient - Overview dashboard ........................................................................................................................ 67

CTGClient - EPI dashboard .................................................................................................................................. 67

CTGClient - JavaGateways & SSL dashboard ....................................................................................................... 68

CTGClient - Terminal & TerminalRequest dashboard ......................................................................................... 68

CTGServer - Overview dashboard ....................................................................................................................... 69

CTGServer - Connection Manager & Workers dashboard .................................................................................. 69

CTGServer - ECI Request dashboard ................................................................................................................... 70

CTGServer - EPI Request dashboard ................................................................................................................... 70

CTGServer - Global Statistics dashboard ............................................................................................................. 71

CTGServer - SSL dashboard ................................................................................................................................. 72

Request Exit metrics ................................................................................................................................................... 72

Backends | CTG_to_CTG_ECI_IPIC_RequestExit Metrics .................................................................................... 72

CTG_Global_Statistics_RequestExit Metrics ....................................................................................................... 73


Chapter 1: Overview of Extension for CTG

The extension for IBM® CICS® Transaction Gateway allows CA Introscope® administrators to monitor the CTG Client interfaces. These interfaces are used by applications in Java and J2EE and CTG Server.

This section contains the following topics:

About Extension for CTG (see page 11) Extension for CTG and your CA Introscope® environment (see page 13) Mixed distributed and z/OS CTG (see page 19)

About Extension for CTG

The CTG extension provides real-time monitoring of the CICS Transaction Gateway Server (CTG) product and the CTG Client Java. The CTG extension also monitors WebSphere CTG Client applications that invoke the Customer Information Control System (CICS) through the CTG. The extension monitors both the CTG Server itself and the instrumented clients that invoke the CTG Server. The CTG Server is an intermediary between Java and J2EE front-ends and CICS Transaction Server back-ends. Monitoring for clients includes monitoring of both JCA and Base API client applications.

Note: CTG Server is also known as the CTG daemon. This guide refers to it as CTG Server, because it is more easily understood as a server. However, in other technical documentation it is referred to as CTG daemon.

The extension for CTG can help Introscope users correlate and isolate the bottlenecks that can occur in their Java applications, WebSphere application servers, and the CTG itself. The extension for CTG monitors transactions and provides detailed metrics that let you determine if a bottleneck exists in the Java or J2EE front end, in the CTG Server middleware, or the CICS backend.

With the extension for CTG installed, you can view:

■ Graphs of performance and activities of applications that use the CTG. These items are available regardless of whether the applications invoke the CTG using the CTG Client base classes, the CCF interface, or the JCA interfaces.

■ A high-level overview of the health of the CTG, through out-of-the-box dashboards.

■ Hierarchical and historical views of performance.

About Extension for CTG


Extension for CTG system requirements

For a complete list of the CTG system requirements, see the Application Performance Management Compatibility Guide on the CA APM bookshelf. The product compatibility matrix provides a list of all supported operating environments.

Note: The Japanese version of the extension for CTG works only with Introscope 9.0.5.

Extension for CTG components

These extensions for CTG components enable an interaction with Introscope:

Component Description

PPCTGServer_ ManagementModule.jar

PPCTGClient_ ManagementModule.jar

Extension for CTG Management Module defines monitoring and reporting logic that is applied to metrics. These metrics include the Dashboards presented in the Workstation and Alerts that define metric thresholds.

PPCTGRequestExit_ ManagementModule.jar

Extension for CTG uses this module for Request Exit instrumentation.

PPCTGClient-full.pbd

PPCTGClient-typical.pbd

ProbeBuilder Directive file that controls the metrics that Introscope-enabled CTG components report to the agent.

When you use the PPCTGClient-full.pbd file, Introscope displays all metrics. If you do not want to incur the overhead of displaying all metrics, use the PPCTGClient-typical.pbd file.

PPCTGServer-full.pbd

PPCTGServer-minimal.pbd

PPCTGServer-typical.pbd

ProbeBuilder Directive file that controls the metrics that Introscope-enabled CTG components report to the agent.

When you use the PPCTGServer-full.pbd file, Introscope displays all metrics. If you do not want to incur the overhead of displaying all metrics, use the PPCTGServer-minimal.pbd or PPCTGServer-typical.pbd file.

PPCTGTranTrace.pbd The PPCTGTranTrace.pbd directive file provides the ability to trace CTG transaction and correlate the transaction between the frontend and the backend.

PPCTGAgent.jar Extension that enables the agent to report the CTG Server metrics to the Enterprise Manager. Valid for both CTG versions 6.x and 7.x.

PPCTGRequestExit.jar

PPCTGbmonitor.jar

Extension that enables the agent to report the CTG 8.x server metrics to the Enterprise Manager. It needs the PPrunGlobalStats batch job to be started. Valid for CTG versions 7.x and 8.x.

Extension for CTG and your CA Introscope® environment


ctg.typeviewers.xml Enterprise Manager extension that defines the CTG-specific views that are tab-selectable in the Investigator when the CTG components are selected in the tree.

ctg.requestexit.typerviewers.xml Enterprise Manager extension that defines the CTG-Request Extension-specific views. These views are tab-selectable in the Investigator when the CTG Request Exist components are selected in the tree.

ChangeDetector-config.xml XML file containing default ChangeDetector monitoring configuration entries for the CTG.

CTG_Tran_Trace_Template.profile This file contains CTG Transaction Trace properties that can be included in IntroscopeAgent.profile to collect CTG Trace specific metrics. For more information see CTG Transaction Tracer (see page 58).

PPrunGlobalStats.bat

PPrunGlobalStats.sh

This requirement is for Windows platforms. You can specify the CTG_CLASSES and ISCOPE_AGENT variables within the PPrunGlobalStats.bat file and executes the Global Statistics monitor embedded in the PPCTGbmonitor.jar. This is required on Windows because CTG runs as a Service under Windows. Running under Windows requires that a separate process is used to extract Global Statistics information. An equivalent file exists for Unix platforms as well.

The PPrunGlobalStatus.sh is the Unix/Linux version of the above .bat file.


Deploying the extension for CTG can involve one or more computers, depending on your environment. Deployment is done by using PBDs or the Request Exit. The CA Introscope® components ProbeBuilder Directives that are involved in the deployment include:

■ ProbeBuilder Directives used to enable the CTG Client libraries. These are deployed both to the computer that hosts the application and the computer that hosts the CTG Server.

■ Extension for CTG agent extensions and ProbeBuilder Directives used to enable CTG Server libraries. These are deployed both to the computer that hosts the CTG Server and to the computer that hosts the application.

■ Management Modules. These are deployed to the Enterprise Manager (typically on a different computer).

Important! CA Technologies does not recommend that you enable CTG with both ProbeBuilder and Request Exit. Doing so can cause overlapping metrics and higher CPU usage.



ProbeBuilder Directives Deployment

The following illustration shows the overall interactions when ProbeBuilder Directives are used for a deployment. The extension for CTG operates in both a z/OS environment and in a distributed (Windows/Linux/UNIX) environment.



CTG Components on z/OS

The following illustration shows the interactions among the components for CTG on z/OS:

C T G fo r z / O S

G a te w a y

d a e m o n

D is tr ib u te d

Ja v a c lie n ts

z /O S C IC S

S e rv e r

E n te rp r is e

M a n a g e r

JN I

m o d u le s

In tro s co p e

A g e n t

T C P o r S S L

M R O

W e b S p h e re fo r z /O S

Ja v a a p p lic a t io n

C T G JC A /C C F R e so u rc e A d a p te r

In tro s co p e

A g e n t

In tro s co p e

A g e n t

The CTG for z/OS is composed of two main runtime components:

■ The CTG Gateway daemon which listens to incoming works and uses the IBM EXCI facility to forward that work to the local CICS backend server.

■ A JCA (or CCF) resource adapter that is deployed in a WebSphere runtime environment.

The extension for CTG monitors both of these components. In the Introscope Investigator, all client metrics appear under the Frontends|Client_to_CTG_Aggregates and Frontends|Client_to_CTG_Details nodes. All server metrics appear under Backends|CTG_to_CICS_xxx and Backends|JSSE_to_CTG nodes.



CTG Components on distributed systems

The following illustration shows the interactions among the components for CTG on distributed systems:

E n te rp r is e

M a n a g e r

N e tw o rk

S ta n d a lo n e Ja v a

C lie n t A p p lic a t io n

In tro s c o p e A g e n t

C T G G a te w a y

d a e m o n

JN I

E C I E P I E S I

C T G C lie n t

d a e m o n

T ra n sp o r t d r iv e rs



z /O S C IC S

S e rv e r

T C P o r E E

(E n te rp r is e

E x te n d e r )

D is tr ib u te d C I C S T ra n s a c t io n G a te w a y

W e b S p h e re

Ja v a C lie n t

A p p lic a t io n




Request Exit overview

IBM supplies a request exit point that allows third-party plug-ins to extract statistics for requests that processed by CTG. The exit supports only ECI and IPIC transactions. EPI is not supported.

A Request Exit deployment uses the following components and interactions:



Request Exits with CTG components

The following illustration shows the interactions among the components for the CTG Request Exit on z/OS:

Mixed distributed and z/OS CTG


Request Exit Components on Distributed Systems

The following illustration shows the interactions among the components for the CTG Request Exit on distributed systems:

Mixed distributed and z/OS CTG

The extension for CTG can also be deployed in a mixed distributed and z/OS environment. For example, a distributed WebSphere instance can be running with a CTG Resource Adapter that communicates with a CTG Server running on z/OS.

The extension is able to monitor both environments. Hence, a single tool (Introscope) and a single extension suite (CTG) can integrate measurement data as needed across multiple environments.

Note: WebSphere does not interact with CICS directly. All communication is done through the CTG.


Chapter 2: Install Extension for CTG

This sections describe how to install and configure the extension for CTG and define CTG properties in the Introscope agent profile file.


Prepare for the installation (see page 21) Extract the installation archive (see page 22) Install Management Modules and Typeviews for PBDs (see page 23) Install Management Modules and Typeviews for the Request Exit (see page 23) Install Introscope agent files to monitor the CTG Server (see page 24) Enable the CTG code using AutoProbe (see page 24) Enable the CTG code using Request Exit (see page 24) Enable the CTG code using Manual ProbeBuilder on distributed CTG platforms (see page 25) Enable the CTG Client Code on WebSphere (see page 28) Modify the z/OS CTG startup script (see page 28) Create an instrumented CTG startup script for distributed systems (see page 29) Run instrumented code (see page 30) Revert to the original code (see page 31) Enable ChangeDetector on the distributed CTG platforms (see page 31) Configure CTG Server RequestExit and Global Statistics Probing (Optional) (see page 33) Configure CTG Client support (see page 35) Verifying the installation and configuration (see page 37)

Prepare for the installation

To prepare for you installation, make sure the following steps have been completed:

Follow these steps:

1. Ensure that your system meets the requirements (see page 12).

Note: If you have not yet installed Introscope, follow the instructions in the CA APM Installation and Upgrade Guide.

2. Identify the following directory locations in your Introscope environment:

■ Introscope Enterprise Manager directory (<EM_Home>) — The installation directory for your Enterprise Manager.

■ Introscope agent home directory (<Agent_Home>) — The installation directory for the Introscope agents that monitor the CTG Server. Identify this directory for each agent that collects CTG Server data.

Extract the installation archive


■ Introscope agent profile directory — The directory where IntroscopeAgent.profile is found on each agent where you plan to install the extension for CTG.

Note: Identify the system property directory of com.wily.introscope.agentProfile. The agent profile is typically in the wily\core\config directory of the agent installation.

3. Stop the Enterprise Manager where you plan to install Extension for CTG.

If you have a clustered environment:

1. Stop the Enterprise Manager serving as the Manager of Managers.

2. Stop each collector Enterprise Manager that is connected to Extension for CTG-enabled agents.

4. Stop all of the components where you plan to install the Extension for CTG. These may include:

■ Java applications

■ WebSphere application server

■ CTG Server software

Note: You do not need to stop more than one component at a time to install the extension for CTG. Stop a component and install before going on to another component. For example, stop the Enterprise Manager and install the extension for CTG, and then stop each Introscope agent and install. If you have a clustered environment, stop the Manager of Managers and install the extension for CTG, then stop each collector Enterprise Manager and install; then stop each agent and install.

Extract the installation archive

Extract the extension for CTG archive appropriate to your system.

The extension for CTG Management modules and server extensions extend the Enterprise Manager, enabling Introscope to monitor the CTG use and performance.

Important! If you are using the z/OS platform, be sure to transfer the entire archive using FTP binary mode; ASCII conversion will corrupt the files.

Install Management Modules and Typeviews for PBDs


Install Management Modules and Typeviews for PBDs

The following procedure applies to installing the management modules and typeviewers for PBD instrumentation.

Follow these steps:

1. Stop the Enterprise Manager on which you plan to install the extension for CTG.


1. Stop the Enterprise Manager serving as the Manager of Managers.

2. Stop each attached collector Enterprise Manager.

2. Copy the following files into <EM_Home>/config/modules:

■ PPCTGClient_ManagementModule.jar

■ PPCTGServer_ManagementModule.jar

If you are installing the Extension for CTG in a clustered Introscope environment, install it on the Manager of Managers and on all the connected Enterprise Managers.

3. Copy ctg.typeviewers.xml into <EM_Home>/ext/xmltv.

If you are installing extension for CTG in a clustered Introscope environment, install the extension for CTG server extension on the Enterprise Manager serving as the Manager of Managers and on all collector Enterprise Managers that are connected to the Manager of Managers.

4. Install the extensions by copying the extracted installation archive file ctg.typeviewers.xml to <Introscope_Home>/ext/xmltv.

Install Management Modules and Typeviews for the Request Exit

The following procedure applies to installing the management module typeviewers for Request Exit instrumentation:

Follow these steps:

1. Stop the Enterprise Manager on which you plan to install the extension for CTG.


1. Stop the Enterprise Manager that serves as the Manager of Managers.

2. Stop each attached collector Enterprise Manager.

Install Introscope agent files to monitor the CTG Server


2. Copy PPCTGRequestExit_ManagementModule.jar into <EM_Home>/config/modules.

If you are installing the Extension for CTG in a clustered Introscope environment, install it on the Manager of Managers and on all Enterprise Managers connected toit.

3. Copy ctg.requestexit.typeviewers.xml into <EM_Home>/ext/xmltv.

Install Introscope agent files to monitor the CTG Server

The Extension for CTG enables you to monitor the use and performance of the CTG Server. The following describes how to install both an agent or a Request Exit to monitor the CTG Server.

To install an agent to monitor the CTG Server, perform the following steps:

Follow these steps:

1. Stop the CTG Server where you plan to install the extension.

2. Copy PPCTGAgent.jar from <Agent_Home>/examples/PowerPackForIBMCTG/ext to <Agent_Home>/wily/core/ext.

To install the Request Exit to monitor the CTG Server, perform the following steps:

Follow these steps:

1. Stop the CTG Server where you plan to install the extension.

2. Copy PPCTGRequestExit.jar from <Agent_Home>/examples/PowerPackForIBMCTG/ext to the CTG classes directory.

3. Ensure that PPCTGRequestExit.jar is in the CTG's CLASSPATH list.

4. Configure the name of your CTG Request Exit in the CTG Config menu.

Enable the CTG code using AutoProbe

For PBD deployments, AutoProbe will automatically create instrumentation probes at startup.

Enable the CTG code using Request Exit

You can dynamically probe the CTG Server with the Request Exit. For CTG client code, you must use JVM AutoProbe instead.

Enable the CTG code using Manual ProbeBuilder on distributed CTG platforms



Manual ProbeBuilder is a non-dynamic method of Introscope-enabling the CTG Gateway Server classes. When you run ProbeBuilder manually, it Introscope-enables the CTG Java classes on a disk before the CTG Server is run.

The preferred method of CTG instrumentation is with JVM autoprobing or the use of the Request Exit. However, if you prefer to probe your CTG applications manually, follow the steps below.

Preparing to enable the CTG Server code

The instructions in this section assume that you have performed the following installation and configuration tasks:

■ Installed the Introscope agent.

■ Completed the following ProbeBuilder Directives Instrumentation:

■ Copied PPCTGClient-full.pbd and PPCTGServer-full.pbd into custompbd under the EM folder.

■ Copied PPCTGAgent.jar from examples into the Agent extension folder.

■ Include the PBD files. For further information, see Enabling CTG Client Code in WebSphere (see page ).

■ Configured the agent Name.

■ Configured any needed ProbeBuilder Options.

■ Copied errors.pbd into custompbd if you are using ErrorDetector.

■ Completed the Request Exit Instrumentation:

■ Installed PPCTGRequestExit.jar for Request Exit instrumentation.

■ Copied the PPCTGAgent.jar from examples into the Agent extension folder.

■ Included the PBD files.

■ Configured the agent name.

■ Copied PPCTGRequestExit.jar into the CTG classes folder.

■ Ensured PPCTGRequestExit.jar files are in the CTG class path list.

■ Configured the name of the CTG Request Exit in the ctg.ini file or in the CTG Config Menu for Windows.



Specifying CTG Server RequestExit Instrumentation for Windows

The Windows versions of CTG run only as a Windows service and require the use of the ctgservice command. The command is used to configure the Introscope properties that are passed into the CTG Server when the service is started. A typical invocation of this command is:

ctgservice -R

-A-j-Dcom.wily.introscope.agentProfile=C:\<Agent_HOME>\wily\core\config\Introscop

eAgent.profile

-A-j-javaagent:C:\<Agent_Home>\wily\Agent.jar

Entering Manual ProbeBuilding options for the CTG Server

Before Probe-Building the CTG Server files, do the following to prepare for the manual Probe building.

Follow these steps:

1. Save the current CTG classes/*.jar files into a backup folder.

2. Run ProbeBuilder against the following jar files in the CTG classes directory:

■ ctgclient.jar

■ ctgserver.jar

■ cicsj2ee.jar

This action creates a set of *.isc.jar files in the CTG’s \classes directory, for example, ctgclient.isc.jar, ctgserver.isc.jar, and cicsj2ee.isc.jar.

3. Rename the following *.isc.jar files:

■ Windows:

rename ctgserver.isc.jar ctgserver.jar

rename ctgclient.isc.jar ctgclient.jar

rename cicsj2ee.isc.jar cicsj2ee.jar

■ UNIX:

mv ctgserver.isc.jar ctgserver.jar

mv ctgclient.isc.jar ctgclient.jar

mv cicsj2ee.isc.jar cicsj2ee.jar



4. If the .isc versions of the probed jar files were created in a different directory, copy them directly into the CTG classes directory.

5. Instrument the jar files manually. There are two ways to do this:

■ ProbeBuilder Wizard—Provides a GUI dialog for running ProbeBuilder.

■ Command-line ProbeBuilder—A command-line interface to ProbeBuilder for environments without a windowing system.

Note: For more information about instrumenting bytecode, see the CA APM Java Agent Implementation Guide.

Using the ProbeBuilder Wizard for the CTG Server

If your computer has a windows environment, you can use the GUI-based ProbeBuilder Wizard.

Follow these steps:

1. Ensure that the ProbeBuilder Wizard has access to the CTG Extension .pbd files, which should be installed in your <Agent_Home> directory.

2. Run the ProbeBuilder Wizard against ctgserver.jar, ctgclient.jar, and cicsj2ee.jar.

3. Before you start, copy the original CTG jar files to a backup area for safe keeping.

Using the command-line ProbeBuilder for the CTG Server

If your computer does not have a windowing environment, you can use the command-line ProbeBuilder to probe manually the CTG jar files.

Follow these steps:

1. Before running ProbeBuilder, copy the original CTG jar files to a backup area for safe keeping.

2. Use the Probe builder command and directives.

3. If you are probing the distributed CTG files in the CTG classes directory, be sure to probe manually the following jar files:

■ ctgserver.jar

■ ctgclient.jar

■ cicsj2ee.jar

Note: For more information about the Probe builder command and directives, see the CA APM Java Agent Implementation Guide.

Enable the CTG Client Code on WebSphere


Enable the CTG Client Code on WebSphere

On WebSphere on z/OS, you can use JVM AutoProbe to automatically monitor the CTG Client and Server.

Follow these steps:

■ Edit the file IntroscopeAgent.profile and modify the property introscope.autoprobe.directivesFile to include the following files as part of its definition:

■ PPCTGClient-full.pbd

■ PPCTGServer-full.pbd

■ required.pbd

For example:

introscope.autoprobe.directivesFile=<list of existing PBDs>,

PPCTGClient-full.pbd,PPCTGServer-full.pbd,required.pbd

Note: The entire definition of introscope.autoprobe.directivesFile must be contained on a single line.

Modify the z/OS CTG startup script

If you are installing in a z/OS environment, modify the CTG’s ctgstart script to pass the Introscope-enabled JVM AutoProbe parameters into the CTG JVM at startup.

Follow these steps:

1. Create a backup copy of the original ctgstart script.

2. Open the CTG’s ctgstart script for editing. The ctgstart script is located in the CTG’s bin directory.

3. Search for the following statement block, which is near the bottom of the ctgstart script:

jvmoptions=$(echo $alloptions | $awk '{

for (i = 1; i<=NF; i++) {

if (substr($i,1,2) == "-j") jvmoptions = jvmoptions " "

substr($i,3)

}

print jvmoptions

}')

Create an instrumented CTG startup script for distributed systems


4. To enable Auto-Probe using the AutoProbeConnector, add the following lines immediately after the statement in step 2:

wilyoptions="-Xbootclasspath/p:./wily/connectors/AutoProbeConnector.jar:./wil

y/Agent.jar -Dcom.wily.introscope.agentProfile=./wily/IntroscopeAgent.profile

-Xverify:none"

jvmoptions="$jvmoptions $wilyoptions""

5. To enable Auto-Probe using the javaagent option, add the following lines immediately after the statement in step 2:

wilyoptions="-javaagent:./wily/Agent.jar

-Dcom.wily.introscope.agentProfile=./wily/IntroscopeAgent.profile"

jvmoptions="$jvmoptions $wilyoptions"

Create an instrumented CTG startup script for distributed systems

Under Windows and UNIX, the distributed CTG Startup is a binary file. This file must be invoked with a startup script to pass in the needed Introscope parameters.

Follow these steps:

1. After using Manual ProbeBuilder to add probes to the CTG classes, create a startup script for the distributed-CTG Server. In Windows, the script is a .bat file; in UNIX, it is a .sh shell file.

2. In the startup script, specify the location of the Introscope classes and the agent profile.

The following is a sample startup script (.bat file) for Windows. (This assumes CTG and WebSphere are the location of the installed folder for those IBM products.)

set

CLASSPATH=c:\CTG<ccc>\wily\Agent.jar;c:\CTG<ccc>\classes\cicsj2ee.jar;c:\CTG<

ccc>\classes\ctgserver.jar;c:\CTG<ccc>\classes\ctgclient.jar;

c:\CTG<ccc>\classes\ccf2.jar;c:\CTG<ccc>\classes\ctgsamples.jar;

C:\WebSphere<www>\AppServer\java\jre\bin;%CLASSPATH%

set JAVA_HOME=C:\IBM_JVM<jjj>\java\jre

set PATH=C:\WebSphere<www>\AppServer\java\jre\bin;.\bin;\%PATH%

ctgstart

-j-Dcom.wily.introscope.agentProfile=C:\CTG<ccc>\wily\core\config\IntroscopeA

gent.profile

where

<ccc> = CTG version number

<jjj> = Java version number

<www> = WebSphere version number

Run instrumented code


3. Edit the classpath of the application startup script to include locations of the directories containing the instrumented code that is created with the ProbeBuilder.

Note: Ensure that these entries proceed the original entries in the classpath. See Running instrumented code (see page 30).

4. Edit the classpath in the application startup script to include the path to:

<Agent_Home>/ Agent.jar.

For example, you can edit the following classpath:

java -classpath

/<your-applicationpath>/classes:/<yourapplicationpath>/lib/app.jar MainClass

to look like the following:

java -classpath

/<your-applicationpath>.isc/classes:/<yourapplicationpath>.isc/lib/app.jar:<A

gent_Home>/Agent.jar MainClass

5. After you complete the manual instrumentation, update the classpath of the CTG Server startup script to reflect the locations of the instrumented code and the agent.

6. Start your application with the new startup script.

Run instrumented code

There are three ways to point to Introscope-enabled code instead of your original code:

■ In classpaths, replace original class paths with instrumented code paths. The instructions in this chapter directed you to perform this process when you instrument your application for the first time.

Note: In your installation on Windows or UNIX, if it has not been done, it can be necessary to update your classpath to use the Java\bin folder. This folder is supplied from the WebSphere application server Java\bin directory, rather than Java supplied from the Java community. This classpath update is necessary for JAVA-CTG compatibility.

■ Prepend paths to classpaths. If only part of the application’s code is instrumented, in the classpath, place the instrumented code paths before the original paths.

Note: If you do this, instrumented code loads and reports performance data. Code that is not instrumented still loads and works normally, but does not report performance data.

Revert to the original code


■ Place instrumented code in original classpath.

Use this method when classpaths are set in many places, or to evaluate the system. Be careful of using this method in a production environment. You can easily forget whether you are using the original or the instrumented code.

■ Move the original code to a new location. Leave the classpaths unchanged, then move the instrumented code to the original location.

■ On a UNIX machine, you can also create a symbolic link pointing to the instrumented code from the original path location.

Revert to the original code

To revert to the original un-instrumented code, undo the instrumentation as follows:

■ If you have put the paths to your instrumented code into the Java classpaths, then replace paths to the instrumented code with the original values.

■ If you have added paths to the instrumented code in front of the paths to the original code, then remove the prepended portion of the classpath so that only the original classpath remains.

■ If you have removed the original code and put the instrumented code in the original classpath, then remove the instrumented code from the original path. Place the original code in the original classpath.

Note: If you used symbolic links on a UNIX system, point the symbolic link to the original directory or remove the link and move the code into the original classpath.

Enable ChangeDetector on the distributed CTG platforms

ChangeDetector is supported on distributed (non-z/OS) systems such as Windows, Linux, and UNIX. ChangeDetector is designed to monitor both the CTG configuration files (such as ctg.ini and ctgenvvar), as well as the associated JAR files, to detect if any changes have been made to the system. This design helps track down any configuration changes that can cause outages or problems with the CTG environment.

The extension for CTG includes a default ChangeDetector configuration file that you must customize in order to properly enable ChangeDetector to monitor CTG installations. In addition, you can add other files for monitoring by ChangeDetector.

Note: For more information about adding files for monitoring, see the CA APM ChangeDetector User Guide.

Enable ChangeDetector on the distributed CTG platforms


This section focuses explicitly on the CTG-related monitoring of ChangeDetector. The ChangeDetector-config.xml file contains a default set of ChangeDetector configuration settings for monitoring CTG and the associated Introscope agent file entities. To properly use this configuration, update two entries in the ChangeDetector-config.xml file.

Note: If you are using ChangeDetector on a distributed CTG system, be aware that ChangeDetector cannot detect changes in Java classes using the Java Class Monitor. This restriction is because the CTG Agent has been manually probed and cannot detect dynamic changes in Java classes. (ChangeDetector can monitor and detect changes in the file system and configuration properties file, as expected.)

Follow these steps:

1. To update the ChangeDetector-config.xml file, under the CTG directory comment block:







<scan-directory recursive="true" name="your CTG directory" fileset="default"

enabled="true" />

Change the name= parameter to point to your specific CTG directory.

For example, if your CTG installation was under the directory ctg<ccc> /usr/lpp/ctg/ctg<ccc>, you would set up the name parameter in the scan-directory entry as follows:

<scan-directory recursive="true" name="/usr/lpp/ctg/ctg<ccc>"

fileset="default" enabled="true" />

2. Under the CA APM directory comment block:







<scan-directory recursive="true" name="your CA APM directory" fileset="default"

enabled="true" />

change the name= parameter to point to your specific CA APM installation directory.

For example, if your CA APM installation was under the directory /usr/vendor/ca apm

then change the name parameter in the scan-directory entry as follows:

<scan-directory recursive="true" name="/usr/vendor/ca apm" fileset="default"

enabled="true" />

3. Once these modifications have been completed, place the updated ChangeDetector-config.xml file into the Introscope agent directory.

4. Ensure that the ChangeDetectorAgent.jar file is installed in the agent’s .ext directory.

Configure CTG Server RequestExit and Global Statistics Probing (Optional)


5. Modify the agent profile’s introscope.changeDetector.profile=config entry to specify the path to the ChangeDetector configuration file.

6. Ensure that the agent profile’s introscope.changeDetector.agentID= config entry contains the name that you want to use for the ChangeDetector agent.

For example, if your CTG installation was under the directory ctg<ccc> /usr/lpp/ctg/ctg<ccc> you would set up the name parameter in the scan-directory entry as follows:

<scan-directory recursive="true" name="/usr/lpp/ctg/ctg<ccc>"

fileset="default" enabled="true"/>

After you have performed the steps above, you can monitor the CTG Server using ChangeDetector.

For information about viewing and interpreting ChangeDetector data, see the CA APM ChangeDetector Guide.


To configure the CTG Server to obtain IBM RequestExit metrics, do the following.

Follow these steps:

1. Place PPCTGRequestExit.jar into CTG's classes directory.

For example:

/u/usr/lpp/cicstg/ctg800/classes/PPCTGRequestExit.jar

2. Add PPCTGRequestExit.jar to the CTG's CLASSPATH list.

For example:

export CLASSPATH=${CTG_CLASSES}/PPCTGRequestExit.jar:${CLASSPATH}

3. Customize the following values in IntroscopeAgent.profile

ppctg.statistics.host=localhost

ppctg.statistics.port=2980

ppctg.statistics.sleep=30

ppctg.statistics.enable=true

Note: The overview typeview under CTG_Global_Statistics_StatsExit is intentionally left blank and contains no metrics data under this tab. If you do not see data in the Type View for CTG_Global_Statistics_StatsExit, that indicates there are no metrics data contained in the CTG_Global_Statistics_StatsExit node.



Configure CTG Server RequestExit and Global Statistics Probing on z/OS in Config file ctg.ini (Optional)

To configure on a z/OS platform to obtain RequestExit detail and global status probing on z/OS in Config file ctg.ini, do the following:

Follow these steps:

1. Add the name of the APM/Wily PPCTG Request Exit into the ctg.ini.

2. Add other parameters depending on the CTG installation environment.

For example:

requestexits = com.ibm.ctg.server.APM_RequestExit_Monitor

[email protected]=com.ibm.ctg.server.RestrictedTCPHandler

[email protected]=port=2980;bind=;connecttimeout=2000;maxconn=5;

Configure on Windows Platform to Obtain RequestExit Detail and Global Statistics Separately (Optional)

To obtain RequestExit detail and global statistics separately on a Windows platform, do the following:

Follow these steps:

1. Run the ctgservice command to set classpath for Request Exit. After copying PPCTGRequestExit.jar from Agent\wily\examples\ext folder to IBM CTG\Classes folder.

For example:

ctgservice -R -A-classpath=C:\<IBM CTG Home>\classes\PPCTGRequestExit.jar

-A-j-Dcom.wily.introscope.agentProfile=C:\<Agent-Home>\wily\core\config\Intro

scopeAgent.profile

-A-j-javaagent:C:\<Agent-Home>\wily\Agent.jar

2. Configure the RequestExit_Monitor name into the CTG program by invoking the IBM CTG Configuration Tool and selecting the Gateway daemon node.

3. Select the Monitoring tab, tab to the 'Use These Request Monitors' field, and fill in the box with the following APM exit value:

com.ibm.ctg.server.APM_RequestExit_Monitor

4. Select Add to add the above entry.

Configure CTG Client support


To configure parameters for RequestExit Detailed status, do the following:

Follow these steps:

1. Expand and select the node CICS Transaction Gateway, Gateway Daemon, Statistics API Options.

2. Select the Enable Protocol Handler check box.

3. (Optional) To override the defaults, change the TCP Port number, timeouts, and other options.

4. If you are leaving the field BinAddress blank, by default, CTG uses localhost.

The previous step enables you to collect IBM Request Exit detailed metrics.

To configure parameters for RequestExit Global Statistics, do the following:

Follow these steps:

Note: Due to restrictions in IBM's support, the Request Exit Global Statistics cannot be automatically enabled.

1. To invoke global statistics collection by the PP, execute the PPrunGlobalStats.bat file and to customize the following entries.

2. To point to CTG classes and Introscope Agent directory:

set CTG_CLASSES= <point to the CTG class files>

set ISCOPE_AGENT= <point to Agent extension folder>


The extension for CTG provides metrics for both the CTG Server and CTG Client applications. This section describes how CTG Client applications are defined as applications that issue requests to the CTG Server. Additionally, the CTG Server is requesting that a specific CICS ECI or EPI application be run. The results are then returned to the client.

Client-side metrics, provided by Extension for CTG, extend the measurement reach to the initiating clients. This reach enables the comprehensive management of response times, triage, and so forth.

The extension for CTG can be configured to work with either standalone CTG Client applications or with client applications that run under WebSphere.



Configure standalone CTG Client applications

For standalone CTG Client applications, Introscope support is configured in the same manner as any other standalone Java application that is being measured using Introscope. In addition, you must add the PPCTGClient-full.pbd and required.pbd files to the IntroscopeAgent.profile's introscope.autoprobe.directivesFile= <parameter>. If you want to monitor local-mode (EXCI) interactions, you must also add PPCTGServer-full.pbd to the list.

Note: Ensure the PPCTGAgent.jar file for the CTG extension is located in the agent ext directory.

Configure Channel and Container Metrics

To minimize metric gathering and CPU overhead with channels and containers, an option is provided in the PBD files to toggle Channel and Container metrics. You do so by commenting or un-commenting directives in PPCTGServer-full.pbd and PPCTGServer-typical.pbd files.

# To disable IPIC Channel|Container metrics use this directive

# Turnoff: IPICChannelContainerTracing

# To enable use this directive

# Turnon: IPICChannelContainerTracing

You must restart the Agent for the changes to take effect.

Configure Client applications under WebSphere

For client applications that run under WebSphere, there are two ways of configuring CTG Client support.

Follow these steps:

■ Add the CTG's ctgclient.jar, cicsj2ee.jar, ccf2.jar, etc files directly in the WebSphere JVM's Classpath configuration menu.

Note: From CTG 8.0 forward, cicsj2ee.jar has been renamed cicsjee.jar.

■ Install the CTG jar files via the Resource Adapter screen under WebSphere, and install the associated cicseci.rar and/or cicsepi.rar files.

Verifying the installation and configuration


After CTG Client support has been added to WebSphere, the PPCTGClient-full.pbd and required.pbd files must be added to the IntroscopeAgent.profile introscope.autoprobe.directivesFile=<parameter>. If you want to monitor local-mode (EXCI) interactions, add PPCTGServer-full.pbd to the list.

Note: Ensure the PPCTGAgent.jar file for the CTG extension is located in the agent ext directory. For a list of files that are included, see Extension for CTG components (see page 12).

Upgrade the Extension for CTG

You cannot upgrade from an earlier version of extension for CTG. Instead, you must uninstall the earlier version and then install the current version.

Verifying the installation and configuration

It is important to verify that your extension for CTG installation installed and configured correctly.

Follow these steps:

1. Restart your monitored WebSphere application server, Java application, CTG Server software, and Introscope Enterprise Manager.

2. After these components have successfully restarted, the agents monitoring the CTG Server and WebSphere application server should start reporting data to the Enterprise Manager.

3. Launch a Workstation and connect to the Enterprise Manager.

4. Verify that you can view data from the CTG Server (see page 39) and application server in the Investigator.


Chapter 3: Using Extension for CTG

This section discusses the following:

■ How to use Introscope to monitor a CTG environment

■ An overview of the different kinds of data made available by Extension for CTG

■ How you can view that data in the Workstation and console

The instructions in this section assume that you have completed the installation steps and that your Introscope-enabled CTG application is up and running and reporting to the Enterprise Manager.


Viewing Extension for CTG data in Introscope (see page 39) Viewing metric data in the tree view (see page 40) Server metrics (see page 41) Using Introscope Investigator tab views (see page 49) Viewing Extension for CTG data in dashboards (see page 50) Modifying Introscope caution/danger alert thresholds (see page 58) CTG Transaction Tracer (see page 58)

Viewing Extension for CTG data in Introscope

There are several ways to view data from the extension for CTG in Introscope:

■ Raw metrics in the Investigator — Provide technical users a detailed view of the underlying performance of all resources and components of the CTG.

■ Tab views in the Investigator — Provide technical users aggregate views of performance and resources and components of the CTG system.

■ Dashboards in the Console — Provide an easy-to-use interface for users that lack familiarity with details of the CTG architecture.

■ Alerts in the Investigator — Show alerts such as those generated by the extension for CTG dashboards as well as alerts you create.

Viewing metric data in the tree view


Viewing metric data in the tree view

To view extension for CTG metrics within the Investigator, follow these steps:

Follow these steps:

1. Start your managed application.

2. Launch Enterprise Manager.

3. Launch and log into the Workstation.

4. Open an Investigator window.

All the Extensions for CTG-specific metrics appear under several nodes in the tree.

Note: The metrics available depend on the CTG and WebSphere resources that your applications use. Only those metrics that are used by your managed Java applications appear.

Frontends

The following Frontends CTG metrics are available:

■ Apps — Metrics for individual applications appear under this node, by application name

■ Client_to_CTG_Aggregates — This node has a Client Aggregates Graphical Summary tab. Under the node in the tree are several aggregate metrics as well as sub-nodes for the following:

■ BASE_ECI_EPI — This may show up when using older versions of CTG with non Request Exit instrumentation and not with CTG 8.x.

■ JCA_ECI

■ JCA_EPI — This may show up when using older versions of CTG with non Request Exit instrumentation and not with CTG 8.x.

■ Screen

■ Terminal

■ Client_to_CTG_Details — This node has a Client Aggregates Graphical Summary tab. Under the node in the tree are sub-nodes representing individual clients.

■ Client_to_CTG_JSSE_Session — This node has a JSSE to CTG Gateway tab. Under the node in the tree are several aggregate metrics.

Server metrics


Backends

The following Backends CTG metrics are available:

■ CTG_Global_Statistics — This node displays statistics:

■ CICS Aggregates

■ Connection Manager

■ Connection Manager Threads

■ Gateway Daemon

■ HTTPRequest

■ Incoming Connection Requests From Clients

■ Individual Servers

■ Session

■ Worker Threads

■ CTG_to_CICS_ECI_IPIC — This node has an ECI All Gateways Graphical Summary tab. Under the node in the tree are sub-nodes with metrics for each client.

■ CTG_to_CICS_EPI — This node has an EPI All Gateways tab. Under the node in the tree are sub-nodes with metrics for each client.

■ JSSE_to_CTG — This node has a JSSE Graphical Summary tab. Under the node in the tree is a sub-node that displays an Incoming JSSE Sessions Summary tab, which displays two metrics:

■ Aggregate Incoming SSL Handshakes

■ Incoming SSL handshakes Per Interval

Note: When an IPIC transaction is executed, the HTTPRequest node now displays.

Analyzing historical data for a specific metric

To analyze historical performance data for a specific metric, set up a Persistent Collection.

Note: For information about setting up a Persistent Collection, see the CA APM Installation and Upgrade Guide.

Server metrics

This section includes information on the back-end and front-end metrics available in the extension for CTG.

Server metrics


Backends | CTG_Global_Statistics

When you select the CTG_Global_Statistics node, Introscope Investigator displays the Global Stats Graphical Summary tab, showing several metrics:

■ Aggregate Client Request Invocations per Interval

■ Process Client Request Response Time

■ Send Client Reply Response Time

■ Connection Manager Threads In Use

■ Worker Threads In Use

In general, the connection thread pool is used to handle requests between the client application and CTG Server. And, the worker thread pool is used to handle requests between CTG Server and CICS on the mainframe.

The CTG_Global_Statistics node also contains a Global Stats Tabular Summary tab. This tab provides a set of running aggregate statistics, showing how much work has flowed through the CTG Server because it was started up. The page shows the following statistics:

■ Total Packets Received

■ Total Packets Sent

■ Connection Manager Threads In Use

■ Worker Threads In Use

The Introscope Investigator tree under the CTG_Global_Statistics node provides an entire suite of global statistics about the work flowing through the CTG Server. The Investigator tree is arranged hierarchically, with each sub-node in the hierarchy showing key statistics about a specific set of functions within the CTG Server.

The CICS Aggregates sub-node displays key statistics about the traffic flowing between the CTG Server and the upstream CICS systems. This sub-node includes statistics such as:

■ Read packets per interval and response time (SNA or TCP packets received)

■ Write packets per interval and response time (SNA or TCP packets sent)

Note: For JCA, the CommArea Aggregate Request Data metrics displays with varying values of data. For Base, metrics are not displayed.

The CICS WLM sub-node shows how the internal WLM is scheduling requests to be processed, and if any user written exits are being driven.

The ConnectionManager sub-node shows statistics regarding the processing of traffic between the CTG Server and the requesting CTG Clients.

Server metrics


The ConnectionManager Threads sub-node shows statistics regarding the use of the thread pool that is used to process ConnectionManager requests/replies to the client. These statistics indicate if any threading bottlenecks with the clients are occurring.

The Gateway Daemon sub-node shows the overall total number of requests flowing and the related total number of CICS transaction commits or rollbacks (SYNCONRETURN, etc) that are being performed.

The Incoming Connection Requests from Clients sub-node shows the per interval arrival rate of requests from the clients into the CTG Server.

The Worker Threads sub-node shows statistics regarding the use of the thread pool that is used to dispatch requests of work to CICS. This statistic can indicate if any threading bottlenecks going upstream to CICS are occurring.

Backends | CTG_to_CICS_ECI_IPIC node

When you select the CTG_to_CICS_ECI_IPIC node, the Introscope Investigator displays the ECI All Gateways Graphical Summary tab, showing several metrics:

■ Aggregate Number of Programs Invoked

■ Program Invocations Per Interval

■ Program Average Response Time (ms)

■ Aggregate Program Errors

CTG to CICS ECI Host ID sub-node

Each of the CICS ECI hosts has its own sub-node under the parent CTG_to_CICS_ECI_IPIC node. When you select the CTG to CICS ECI Host ID sub-node, the Introscope Investigator viewer pane displays the following metrics in the ECI Gateway Tabular Summary tab:

■ Aggregate Program Count

■ Average Response Time (ms)

■ Responses/Interval

■ Errors/Interval

■ Stalls

For each host, the tree displays a further sub-node corresponding to each CICS program running on the host.

Server metrics


Backends | CTG_to_CICS_EPI node

When you select the CTG_to_CICS_EPI node, Introscope Investigator displays the EPI All Gateways tab, showing several metrics:

■ Aggregate Number of Transactions Invoked

■ Transaction Average Response Time (ms)

■ Transaction Invocations Per Interval

■ Aggregate Transaction Errors

■ Aggregate Number of Service Requests Invoked

■ Aggregate Service Request Errors

CTG to CICS EPI Host ID sub-node

Each of the CICS EPI hosts has its own sub-node under the parent CTG_to_CICS_EPI node. When you select the CTG to CICS EPI Host ID sub-node, the Introscope Investigator viewer pane displays the following metrics in the EPI Aggregate Summary tab:


■ Transaction Invocations Per Second



For each host, the tree displays a further sub-node corresponding to each CICS program running on the host.

Backends | JSSE to CTG node

When you select the JSSE_to_CTG node, Introscope Investigator displays the JSSE Graphical Summary tab, with two metrics:

■ Aggregate Incoming SSL Handshakes

■ Incoming SSL Handshakes Per Interval

Under the JSSE to CTG node, there is a sub-node for the JSSE ServerSocket.

Server metrics


JSSE ServerSocket sub-node

The JSSE ServerSocket sub-node displays the Incoming JSSE Sessions Summary tab, with two metrics:

■ Aggregate Number of Incoming JSSE SSL Handshakes

■ Incoming JSSE SSL Handshake Invocations Per Interval

Frontends | Client_to_CTG_Aggregates node

When you select the Client_to_CTG_Aggregates node, Introscope Investigator displays the Client Aggregates Graphical Summary tab, with the following metrics:

■ TCP Aggregate Opens

■ TCP flow Aggregate Count

■ TCP flow Response Time (ms)

■ SSL Aggregate Opens

■ SSL flow Aggregate Count

■ SSL flow Response Time (ms)

Under the Client_to_CTG_Aggregates node of the Introscope Investigator tree are the following individual metrics (some are displayed in the tab view), as well as several sub-nodes:

■ Aggregate EPI Errors


■ CICS Resource Exceptions Per Interval

■ CICS Txn Abend Exceptions Per Interval

■ EPI Gateway Exceptions Per Interval

■ EPI Request Exceptions Per Interval

■ Resource Exceptions Per Interval

■ SSL Aggregate Flows

■ SSL flow Aggregate Count

■ SSL flow Response Time (ms)

■ TCP Aggregate Opens

Server metrics


■ TCP flow Aggregate Count

■ TCP flow Response Time (ms)

The following sections discuss the sub-nodes under the Client_to_CTG_Aggregates node.

Frontends | Client_to_CTG_Aggregates | BASE_ECI_EPI sub-node

When you select this node, Introscope Investigator displays the Client Aggregate Summary tab. This tab displays charts for the following metrics:




■ Aggregate Connection Opens

Frontends | Client_to_CTG_Aggregates | CCF_ECI sub-node






The tab is similar to the one shown for the BASE_ECI_EPI node.

Frontends | Client_to_CTG_Aggregates | CCF_EPI sub-node






Server metrics


Frontends | Client_to_CTG_Aggregates | JCA_ECI sub-node


■ Aggregate Number of CICS Programs Invoked




Frontends | Client_to_CTG_Aggregates | JCA_EPI sub-node






Frontends | Client_to_CTG_Details node

The Client_to_CTG_Details node provides the end-to-end aggregate response time across all gateways. When you choose this node, Introscope Investigator displays the All Client Gateways Tabular Summary tab. This tab displays the following columns:

■ Server

■ Program Name

■ Aggregate Count



■ Errors/Interval

■ Stalls

Server metrics


ECI host details sub-nodes

Under the Client_to_CTG_Details node are sub-nodes corresponding to servers hosting CICS programs. When you select one of these server sub-nodes, the Introscope Investigator viewer pane displays the Client Gateway Tabular Summary tab, with a table display of the following metrics:

■ Aggregate Count



■ Errors/Interval

■ Stalls

ECI program details sub-nodes

When you expand one of the ECI host sub-nodes, the tree displays a further sub-node for each of the programs on the server. Selecting these program nodes displays the Program Details View tab. The tab displays charts for the following metrics:

■ Aggregate Request Count

■ Invocations Per Interval


■ Aggregate Errors

Frontends | Client_to_CTG_Details transaction view node

Under the Frontends | Client_to_CTG_Details node of the Introscope Investigator tree, each sub-node corresponds to an EPI host.

When you select a host ID sub-node, the Introscope Investigator viewer pane displays the Client Gateway Tabular Summary tab. This tab displays the following metrics for each of the programs running on the host:

■ Aggregate Count



■ Errors/Interval

■ Stalls

Using Introscope Investigator tab views


Under each of the host sub-nodes is a further sub-node for each program running on a host. When you select one of these program sub-nodes, the Introscope Investigator viewer pane displays the Program Details View tab, with charts for these metrics:

■ Aggregate Request Count

■ Invocations Per Interval


■ Aggregate Errors

Frontends | Client_to_CTG_JSSE Session node

The Client_to_CTG_JSSE Session node provides an end-to-end aggregate response time across all gateways. When you choose this node, Introscope Investigator displays the JSSE to CTG Gateway tab. This tab displays charts for the following metrics:

■ Aggregate Number of SSL Sessions Created

■ SSL Handshake Average Response Time (ms)

■ SSL Session Invocations Per Interval

■ SSL Encrypted Data Requests Per Interval

In the Introscope Investigator tree, the following metrics are shown under the Client_to_CTG_JSSE Session node:

■ Aggregate SSL Data Flows

■ Aggregate SSL Handshakes

■ Aggregate SSL Opens

■ SSL Data Flow Response Time (ms)

■ SSL Data Flows Per Interval

■ SSL Handshake Allocate Response Time (ms)

■ Stall Count

Using Introscope Investigator tab views

You can change the view that is displayed in the Introscope Investigator viewer pane using the tabs at the top of the pane. Many of the CTG-specific nodes on the tree have special out-of-the-box summary views.

Viewing Extension for CTG data in dashboards


The following tabs are standard and available in the Introscope Workstation, regardless of the extensions you use.

■ General — The General tab is the default tab when any item below an agent in the Introscope Investigator tree is selected. When a metric is selected, the General tab is a visualization of the metric — either live data or for a selected historical period. For nodes in the tree, the General tab shows the path to that node object in the Introscope Investigator hierarchy.

■ Overview — The Overview tab, available when an agent is selected in the Introscope Investigator tree, enables application monitoring. This information presents high-level health indicators and a log of related events and historical metric information.

■ Search — The Search tab, available when a node in the Introscope Investigator tree that contains metrics is selected, allows you to find metrics quickly.

■ Trace — The Trace tab, available when a resource or component is selected in the Introscope Investigator tree, is similar to the Trace Viewer. This information lists the Transaction Traces in which the currently selected resource or component participated.

■ Error — The Error tab, available when a resource or component is selected in the Introscope Investigator tree, lists errors and error detail for the selected item.

Note: For more information about standard Introscope tabs, see the CA APM Workstation User Guide.


Extension for CTG provides a number of pre-configured Introscope dashboards and alerts.

All dashboards for the CTG Client or server begin with CTGClient or CTGServer to distinguish them from other Management Modules installed in Introscope.

To view dashboards, launch the Introscope Workstation and open a console. The extension for CTG also comes with default caution/danger alert thresholds for many metrics displayed in these dashboards.

Note: Some of the panels and dashboards included with the extension for CTG can remain blank due to the unique pattern of your CTG applications.



CTG Server - Overview dashboard

The CTGServer - Overview dashboard shows a summary of three major CTG Server areas being monitored by Introscope. This dashboard also provides links to dashboards that provide more details for these areas, as well as to the following dashboards:

■ CTG Client - Java Gateways & SSL

■ CTGServer - Connection Manager & Workers

Traffic lights visually indicate whether any of the underlying metrics have exceeded their defined thresholds.

The Thread Usage section shows the current usage of connection and worker threads by the CTG. In general, a connection thread is assigned when a client connects to the CTG and released when the client disconnects from the CTG. For each active request flowing between the CTG and CICS, a worker thread is assigned.



The metrics displayed in the CTGServer - Overview dashboard are:

■ ECI

■ ECI Request Execute Response Time

■ ECI Request Count

■ EPI

■ EPI Request Execute Response Time

■ EPI Request Count

The Currently in Use metric reflects the number of threads that are in the middle of processing a request (or reply) by the CTG Server code. The Pool Count metric reflects the number of threads that are allocated for that entity (Connection manager or Worker). The pool count is initially established based on the initconnect parameter (for Connection Manager threads) and the initworker parameter (for Worker threads) as defined in your ctg.ini file. As the workload increases, the respective pool count grows, up to the maximum specified by the ctg.ini’s maxconnect and maxworker parameters. You can use the peak value recorded in the Pool Count to fine-tune the associated maxconnect and maxworker parameters in the ctg.ini file.

CTG Summary - Server ECI Activity dashboard

The CTG Summary - Server ECI Activity dashboard provides a brief snapshot of executions and objects that may be causing a problem.

The metrics displayed on the CTG Summary-Server ECI Activity dashboard are:

■ ECI All Gateways-Aggregate Summary

■ Number of Program Invoked

■ Program Invocations per Interval

■ Program Average Response Time

■ Network Aggregates

■ TPC Socket New Accepts per Interval

■ JSSE/SSL Invocations per Interval

■ TCP Socket Input Bandwidth

■ TCP Socket Output Bandwidth

■ JSSE/SSL New Aggregate Count

Note: To enable displays of the TCP Socket Input Bandwidth and TCP Socket Output Bandwidth, set the property introscope.agent.sockets.reportRateMetrics=true in the IntroscopeAgent.profile.



CTG Server - Global Statistics dashboard

The CTGServer - Global Statistics dashboard provides a brief snapshot of overall CTG statistics from the perspective of the CTG Server. A link in the upper corner allows you to navigate to the CTGServer - Overview dashboard.

The metrics displayed on the CTG Request Exit - Global Statistics dashboard are:

■ I/O Requests Processed (Aggregate)

■ I/O Requests Processed Per Interval

■ Incoming Connection Requests from Clients (Per Interval)

■ Total ECI/IPIC/EPI Requests to CICS (Per Interval)

■ Connection Manager Threads Currently In Use

■ Connection Manager Threads Pool Count

■ Worker Threads Currently In Use

■ Worker Threads Pool Count

CTG Server - ECI/IPIC Request dashboard

The CTGServer - ECI Request dashboard provides a brief snapshot of executions and objects that may be causing a problem. A link in the upper corner allows you to navigate to the CTGServer - Overview dashboard.

The metrics displayed on this dashboard are:

■ Execution Performance

■ Request Execute Time

■ Execute List Time

■ Request Execute Count

■ Execute List Count

■ Request Initialize/Terminate and Write/Read Objects

■ Write/Read Object Time

■ Write/Read Object Count

Write/Read Objects is the IBM CTG terminology that is used to indicate service messages (packets) that are exchanged between the CTG Client and the CTG Server. This group of measurements shows the total number of such messages sent, and their average response time. If the CTG Server becomes overloaded (such as too few worker or connection threads), these messages become queued waiting for an available thread, and response times increase.



CTG Server - EPI Request dashboard

The CTG Server - EPI Request dashboard provides a performance summary of the server requests. A link in the upper corner allows you to navigate to the CTG Server - Overview dashboard.

The metrics displayed on the CTG Server - EPI Request dashboard are:

■ Execution Performance

■ Transaction Execution Time

■ Aggregate Transaction Starts Count

■ Read/Write Objects

■ Write/Read Objects Overhead Time

■ Write/Read Objects Overhead Count

■ CICS CP Requests

■ CP Request Execute Time

■ CP Request Execute Count

If the CTG is unable to establish an EPI session to CICS, then the following metrics are not populated:

■ Transaction Execution Time

■ Write/Read Overhead Time

■ Aggregate Transaction Starts Count

■ Write/Read Objects Overhead Count

Instead, an error message (found under the Errors tab) is logged, indicating the error causing the connection failure.

CTG Server - Connection Manager & Workers dashboard

The CTG Server - Connection Manager & Workers dashboard provides a brief snapshot of the performance of connection managers and worker that may be causing a problem. A link in the upper corner allows you to navigate to the CTG Server - Overview dashboard.

The dashboard displays these metrics:

■ Connection Manager Performance

■ Dispatch Response Time

■ Send Reply Response Time



■ Dispatch Count

■ Send Reply Count

■ Worker Threads Performance

■ Dispatch Response Time

■ Run/Close Response Time

■ Worker Dispatch Count

■ Run/Close Count

CTG Server - SSL dashboard

The CTG Server - SSL dashboard provides a brief snapshot of the performance of SystemSSL and JSSE SSL, which may be causing a problem. A link in the upper corner allows you to navigate to the CTG Server - Overview dashboard.

The metrics displayed on the CTG Server - SSL dashboard are:

■ JSSE/SSL Server Sockets

■ Socket Accept/Close Response Time

■ Socket Accept/Close Count

■ SSL Sockets (Legacy)

■ Socket Close Response Time

■ Socket Close Count

■ JSSE Socket

■ JSSE Socket Close Response Time

■ JSSE Socket Close Count

CTG Client - Overview dashboard

The CTG Client - Overview dashboard shows a summary of three major CTG Server areas that are monitored by Introscope. This dashboard also provides links to dashboards that provide more details for these three areas. Traffic lights visually indicate whether any of the underlying metrics have exceeded their defined thresholds.

The metrics displayed on the CTG Client - Overview dashboard are:

■ Java Gateways & SSL Sessions

■ Gateways Flow Response Time

■ JSSE SSL Session Allocate Response Time



■ EPI

■ EPI Connection Ops Response Time

■ EPI Terminal Transaction Ops Response Time

■ Terminal & Terminal Request

■ Terminal & Terminal Request Send Response Time

■ Terminal Connect & Disconnection Response Time

CTG Client - JavaGateways and SSL Sessions dashboard

The CTG Client - JavaGateways & SSL Sessions dashboard provides a brief snapshot of gateways and sessions that may be causing a problem. A link in the upper corner allows you to navigate to the CTG Client - Overview dashboard.

The metrics displayed on the CTG Client - JavaGateways & SSL Sessions dashboard are:

■ JavaGateways Performance

■ Flow Response Time

■ Currently Active Sessions Count

■ TCP Aggregate Sessions Invoked Count

■ Open Response Time

■ Close Response Time

■ JSSE SSL Sessions Performance

■ JSSE Aggregate SSL Session Count

■ JSSE SSL Session Allocate Time

■ JSSE SSL Session Invocations Per Interval

CTG Client - EPI Request dashboard

The CTGClient - EPI Request dashboard displays a snapshot of gateway connections.

The dashboard displays the following metrics:

■ Gateway response time metrics:

■ EPIGateway Connection Ops Response Time

■ EPIGateway Aggregate Connection Count



■ Terminal and basic screen handler metrics:

■ EPITerminal Connection Ops Response Time

■ EPITerminal Transaction Ops Response Time

■ EPI Basic Screen Handler Connection Count

CTG Client - EPI Terminal Requests dashboard

The CTG Client - EPI Terminal Request dashboard provides a brief snapshot of terminal and terminal request activity that may be causing a problem. A link in the upper corner allows you to navigate to the CTG Client - EPI Terminal Request dashboard.

The metrics displayed on this dashboard are:

■ Terminal Activities & Performance

■ Send Response Time

■ Connection & Disconnection Time

■ Send Count

■ Connection Count

Introscope ChangeDetector dashboard

The Introscope ChangeDetector dashboard provides a summary of the changes to the CTG configuration files and .JAR files that may be causing a problem.

The metrics displayed on the Introscope ChangeDetector dashboard are:

■ Changes Detected Per Interval

■ Most Changes Detected Per Interval

■ Total Changes Since Last Agent Restart

■ Completed Scans Since Last Agent Restart

Modifying Introscope caution/danger alert thresholds


Modifying Introscope caution/danger alert thresholds

Many extensions for CTG dashboards include alerts.

To modify the Introscope caution/danger alert thresholds:

1. Within the Introscope Investigator, go to Management Modules > CTGCLIENT > Alerts, select the alert.

The alert definition in the bottom pane.

2. In the alert definition pane, modify the Period field to specify how often you want the performance metrics compared against the Caution Threshold and Danger Threshold.

3. Specify the Caution Threshold and Danger Threshold that are appropriate for your organization's service levels.

4. Click Apply to apply the changes.

CTG Transaction Tracer

The CTG Transaction Tracer provides a capability for tracing by Server Name and Program Name. This capability is primarily for doing Transaction Tracing between a CTG Client (e.g. Websphere) and the CTG Server. It is not required to tie into the CICS mainframe cross-product TranTracer SYSVIEW capability. The tracer also supports the ability to extract and include a customer specified User Correlation field in the CommArea, perform Response Time thresh-holding, and trace a specific program name. This feature enables you to customize using the Agent properties entries and create value-added CTG tracing based on response-time thresholds, by program name, or by writing your own custom extensions on top.

Handling of Special Characters by PP CTG Transaction Tracer

The Introscope Transaction Trace correlation logic, that runs on the Introscope Workstation, severely restricts the use of special characters in user generated Correlation ids. Any special characters that are not allowed by the Introscope Workstation correlation support are automatically converted to underscores.

The following special characters are always converted to underscores:

: - + ( ) [ ] * " ~ ^ ?

The following additional set of special characters is also by default converted to underscores:

\ / { } | , . ; = ' < >



However, if you want to pass these special characters through (and allow all special characters and traces), add the following property to the IntroscopeAgent.profile:

introscope.ctg.trantracer.corrid.specialcharstrip=false

The default for this value is 'true', which suppresses all special characters to underscores and correlates frontend and backend traces.

If you want to remove duplicate underscores (reducing several underscore down to a single underscore) in the correlation id, the following property must be set in the IntroscopeAgent.profile:

introscope.ctg.trantracer.corrid.underscorestrip=true

Detailed List of CTG Transaction Tracer Properties

A detailed list of the CTG Transaction Tracer properties that can be configured in the IntroscopeAgent.profile file is shown in the table:

Property Usage

introscope.ctg.trantracer.publish.metrics Boolean that denotes if any simple metrics (Response Time and Invocation Count) should be enabled. Note this is meant to be a 'light weight' tracer.

Default = true

introscope.ctg.trantracer.publish. trantrace

Boolean that denotes if Tran Tracing should be enabled.

Default = true

introscope.ctg.trantracer.corridscan. offset

Integer indicating at what offset from the beginning of the CommArea the User Correlation Id starts.

Default = 147

introscope.ctg.trantracer.corridscan. length

Integer indicating the length of the User Correlation Id to be extracted.

Default = 100

introscope.ctg.trantracer.corrid. format.id

Boolean indicating if User Correlation Id should be scanned for illegal chars that are not allowed in EM TranTracer string entries [for example, ':' (colon) and '-' (dash)].

introscope.ctg.trantracer.corridscan. starttrace

Boolean denoting if the User Corr Id should be extracted at Start Trace instead of an Finish Trace.

Default = false



Property Usage

introscope.ctg.trantracer.rspthreshhold. value

Integer indicating what the max allowed Response Time threshold is for this tracer. Response times exceeding this value will be Tran Traced to the EM. Values below that value will not be traced.

Default = 0 = do not do RSP Thresh Hold checking

introscope.ctg.trantracer.program. name

String indicating the full or partial CICS Program name to be matched. Values that match will be Tran Traced to the EM. Values not matching will not be traced.

Default=null = no program matching

introscope.ctg.trantracer.program.match. criteria

Integer defining what kind of Program name matching should be performed.

1 = full name match 2 = program name starts with above Name string 3 = program name Ends with above Name string 0 = no matching.

Default = 0 (no matching).

introscope.ctg.trantracer.metric. HLQname

String indicating what Investigator Tree high level name should be.

Default is: Backends|CTG_to_CICS_ECI_IPIC_Trace and Frontends|Client_to_CTG_Trace

introscope.ctg.trantracer.debug Boolean denoting if we should trace debug messages to the Agent log.

Default = false


Appendix A: CTG Performance Metrics

This appendix describes the CTG classes and methods for which the extension for CTG collects performance metrics and which appear in the Introscope Investigator.


Frontend metrics (see page 61) Backend metrics (see page 64) CTG dashboard metrics (see page 67) Request Exit metrics (see page 72)

Frontend metrics

Frontend metrics are captured by CTG Client instrumentation. The data is captured from within an application Server (such as WebSphere) or within an instrumented user-written CTG Client application that is running standalone (outside of WebSphere). The frontend metrics are used to capture and measure the type of CTG application being run: JCA, CCF, or Base classes for either ECI or EPI.

Frontend metrics


Metrics are captured at an aggregate level (for example, all ECI programs being invoked by the Web sphere application server). Detailed metrics are captured at an individual program or transaction level (for example, the statistics for an individual ECI program that was invoked). Many of the metrics (especially detailed metrics) use BlamePoint tracers, so a common format is provided:

Metric Definition

Aggregate Request Count

Total number of times that a request for a program or transaction was initiated. If the Open connection fails (see below), this metric may not appear.

Note: See the Note in the Aggregate Service Count.

Aggregate Service Count EPI uses several setup and teardown flows (AddTerminal, PurgeTerminal, DeleteTerminal) that do not execute any actual CICS transactions. They are only service requests to set up an active 3270 connection to CICS. This metric keeps an aggregate count of all of these service type requests.

Note: The Aggregate Request Count records the actual 3270 transaction invocation and response (that is, the sending up of the 3270 buffer with the EPI request and the associated 3270 reply buffer from the CICS transaction).

Aggregate Errors Count Total number of times that a request for an Open connection or for a program or transaction initiation failed due to an error. This metric only appears if an actual error occurred.

Average Response Time Average response time for the request to be processed, measured from the client end (that is, total round-trip time).

Concurrent Invocations How many concurrent invocations for a program or transaction request were in flight.

Errors per Interval Total number of errors that were reported during the latest measurement interval (typically 7-15 seconds).

Responses per Interval Total number of replies from CTG that were reported during the latest measurement interval.

Stall Count Indicates if any stalls were detected when a request for a program was initiated, but not responded to by CICS within the stall detect time limit. The stall detect limit is set by the: introscope.agent.stalls.thresholdseconds=nn property in the IntroscopeAgent.profile. The nn value specifies the stall detect time in seconds.

Frontend metrics


For Client_to_CTG_Aggregates, the following additional metrics are tracked:

Metric Definition

Aggregate Opens

Total number of TCP or SSL connection opens that were issued from the CTG Client to the CTG Server. A CTG request cannot be initiated until a CTG open connection is performed.

Open Response Time Average response time for the connection open request to be processed and measured from the client end. That is, the total round-trip time it takes for the CTG Client to CTG Server connection to be opened.

Frontend Client to CTG aggregates

These metrics provide an aggregate count of all the ECI and EPI calls issued from the CTG Client to the CTG Server. For convenience, the calls are categorized by CTG API type. All the metrics listed in the following table utilize BlamePoint metrics.

If any of the Open connection requests fail, more details can be found under the Open Errors tab that appears under the Client_to_CTG_Aggregates tree.

Metric Definition

BASE_ECI_EPI All ECI calls that use the Base ECI class API, plus all EPI calls that use the Base EPI class API.

JCA_ECI All ECI calls that use the newer Java Connector Architecture (JCA) ECI API.

JCA_EPI all EPI calls that use the newer Java Connector Architecture (JCA) EPI API.

Stall Count Indicates if any stalls were detected when a request for a program was initiated, but not responded to by CICS within the stall detect time limit. The stall detect limit is set by the: introscope.agent.stalls.thresholdseconds=nn property in the IntroscopeAgent.profile. The nn value specifies the stall detect time in seconds.

Backend metrics


Frontend Client to CTG details

These metrics provide detailed counts for each program or transaction that is invoked by the CTG Client. For convenience, the CLI (Gateway) server categorizes the calls. Underneath each Gateway Server is a list of each individual ECI or EPI program that was invoked, along with the detailed metrics for the program.

Backend metrics

The backend metrics are captured within a CTG Server instrumented by Introscope. The backend metrics are used to capture and measure the flow of ECI and EPI requests between a client and the CICS systems.

Metrics are captured at an aggregate level (for example, all ECI programs that the clients invoke). The detailed metrics are captured at an individual program level (for example, the individual ECI program statistics that were invoked).

Underneath each Gateway Server is a list of each individual ECI or EPI program that was invoked, with the detailed metrics for the program. The detailed metrics use BlamePoint tracers in a common format:

Metric Definition

Aggregate Count

Aggregate Request Count

Total number of times that a request for a program or transaction was initiated. If the Open connection fails (see below), this metric may not appear.

Aggregate Errors Count Total number of times that a request for an Open connection, or for a program or transaction initiation, failed due to an error. This metric only appears if an actual error occurred.

Average Response Time Average total round-trip time for the request to be processed.

Errors per Interval The total number of errors that were reported during the latest measurement interval.

Backend metrics


Metric Definition

Responses per Interval The total number of replies from CTG that were reported during the latest measurement interval.

Stall Count The total number of stalls detected. A stall occurs when a request for a program was initiated but not responded to by CICS within the stall detect time limit. The stall detect limit is set by the introscope.agent.stalls.thresholdseconds=nn property in the IntroscopeAgent.profile, where nn is the stall detect time in seconds.

Backend CTG_to_CICS_ECI_IPIC aggregates

These metrics provide an aggregate count of all ECI/IPIC calls issued between the CTG Clients and the corresponding CICS Servers. Below each Gateway Server is a list of each individual ECI/IPIC program that is invoked, using BlamePoint tracers metrics, as described in the table. The following aggregate metrics are provided on a Gateway server by Gateway server basis:

Metric Definition

Program Aggregate Count

Total aggregate number of times that a request for an ECI/IPIC program was initiated.

Program Average Response Time Average aggregate response time for an ECI/IPIC request to be processed and measured from the CTG Server to CICS. That is, the total round-trip time between the CTG Server and CICS.

Program Invocations per Interval Total number of invocations for an ECI/IPIC program that were reported during the latest measurement interval.

Backend metrics


Backend CTG Global Statistics

Metric Definition

Process Client Request Response Time

The overall time required to read in the client request and forward it to CICS.

Send Client Reply Response Time The overall time required to read in the reply from CICS and forward it back to the requesting client.

Connection Manager Threads In Use

The number of threads currently being used to process client connections.

Worker Threads In Use The number of threads currently being used to process CICS requests.

Backend CTG_to_CICS_EPI aggregate metrics

CTG_to_CICS_EPI metrics are available only on distributed (non-z/OS) versions of CTG. These metrics provide an aggregate count of the EPI calls issued between CTG clients and the corresponding CICS servers.

Metric Definition

Transaction Aggregate Count

Total aggregate number of times that a request for an EPI Transaction was initiated.

Transaction Average Response Time

Average aggregate response time for an EPI request to be processed and measured from the CTG Server to CICS. That is, the total round-trip time between the CTG Server and CICS.

Transaction Invocations per Interval

Total number of invocations for an EPI transaction that were reported during the latest measurement interval.

Note: Underneath each Gateway Server is a list of each individual EPI Transaction that was invoked, using BlamePoint tracers metrics, as described in the table.

Backend CTG_to_CICS threads

These metrics show the current usage of connection and worker threads by the CTG Server.

CTG dashboard metrics



The following tables show the correspondence between labels that appear in extension dashboards and the specific CTG performance metrics.

The right columns describe the metrics in terms of their location in the Introscope Investigator.

CTGClient - Overview dashboard

Label CTG performance metric

Gateways flow Response Time

CTGCLIENT|{.+}JavaGateway:open Response Time (ms)

JSSE SSL Session Allocate Response Time (ms)

Client_to_CTG|JSSE Session:SSL Handshake Allocate Response Time

EPI Connection Ops Response Time

CTGCLIENT|EPI|Gateway: open Response Time (ms)

CTGCLIENT|EPI|Terminal:{connect|disconnect|process

Connect|processDisconnect|terminate} Response Time (ms)

EPI Terminal Transaction Response Time

CTGCLIENT|EPI|Terminal:{send|setTransactionData| startTran|terminate} Response Time (ms)

Terminal & Terminal Request Send Response Time

CTGCLIENT|{TerminalTerminalRequest}:send Response Time (ms)

Terminal Connect & Disconnect Response Time

CTGCLIENT|Terminal{connect|disconnect} Response Time (ms)

CTGClient - EPI dashboard


EPIGateway Connection Ops Response Time

CTGCLIENT|EPI|Gateway:{.*} Response Time (ms)

EPIGateway Aggregate Connection Count

CTGCLIENT|EPI|Gateway:{.*} Count

EPI Terminal Connection Ops Response Time

CTGCLIENT|EPITerminal:{connect|diswconnect|processConnect| processDisconnect|terminate} Response Time (ms)




EPI Terminal Transaction Ops Response Time

CTGCLIENT|EPITerminal{send|setTransactionData| startTran|terminate} Response Time (ms)

EPIMonitor & Terminal Connection Count

CTGCLIENT|EPITerminal:Count

CTGCLIENT|EPIMonitor:terminalConnectedCount

EPI BasicScreenHandler Connection Count

CTGCLIENT|EPIBasicScreenHandler: terminalConnectedCount

CTGClient - JavaGateways & SSL dashboard


JavaGateways: Flow Response Time

CTGCLIENT|{.+}JavaGateway:flow Response Time (ms)

JavaGateways: Total Active Sessions Count

CTGCLIENT|{.+}JavaGateway:Count

JavaGateways: Open Response Time

CTGCLIENT|{.+}JavaGateway:open Response Time (ms)

JavaGateways:Close Response Time

CTGCLIENT|{.+}JavaGateway:close Response Time (ms)

JSSE Aggregate SSL Session Count

Client_to_CTG|JSSE Session:Aggregate SSL Opens

JSSE SSL Session Allocate Time(ms)

Client_to_CTG|JSSE Session:SSL Handshake Allocate Response Time

JSSE SSL Session Invocations Per Interval

Client_to_CTG|JSSE Session:SSL Opens Per Interval

CTGClient - Terminal & TerminalRequest dashboard


Terminal Send Count CTGCLIENT|EPIGateway:{.*}Response Time (ms)

Terminal Connect & Disconnect Count

CTGCLIENT|EPIGateway:{.*} Count




TerminalRequest Send Response Time

CTGCLIENT|TerminalRequest:send Response Time (ms)

Connect and Disconnect Response Time

CTGCLIENT|TerminalRequest:{release|allocate} Response time (ms)

CTGServer - Overview dashboard

Label CTGServer performance metric

ECI Request Execute Response Time

CTGSERVER|ServerECIRequest|execute Response Time (ms)

ECI Request Count CTGSERVER|ServerECIRequest|execute Count

EPI Request Execute Response Time

CTGSERVER|ServerEPIRequest|execute Response Time (ms)

EPI Request Count CTGSERVER|ServerEPIRequest|execute Count

CTGServer - Connection Manager & Workers dashboard

Label WebSphere performance metric

ConnectionManager: Dispatch Response Time

CTGSERVER|ConnectionManager:kick Response Time (ms)

ConnectionManager: Dispatch Count

CTGSERVER|ConnectionManager:kick Count

ConnectionManager: Send Reply Response Time

CTGSERVER|ConnectionManager:sendReply Response Time (ms)

ConnectionManager: Send Reply Count

CTGSERVER|ConnectionManager|sendReply Count

Workers: Dispatch Response time

CTGSERVER|Worker:kick Response Time (ms)

Workers: Dispatch Count

CTGSERVER|Worker:kick Count




Workers: Run/Close Response time

CTGSERVER|Worker:{close|run} Response Time (ms)

Workers: Run/Close Count

CTGSERVER|Worker:{close|run} Count

CTGServer - ECI Request dashboard


Request Execute time CTGSERVER|ServerECIRequest:execute Response Time (ms)

Execute List time CTGSERVER|ServerECIRequest:executeList Response Time (ms)

Request Execute Count CTGSERVER|ServerECIRequest:execute Count

Execute List Count CTGSERVER|ServerECIRequest:executeList Count

Request Initialize/Terminate time

CTGSERVER|ServerECIRequest:{initialize|terminate} Response Time (ms)

Write/Read Object time CTGSERVER|ServerECIRequest:{read|write}Object Response Time (ms)

Request Initialize/Terminate Count

CTGSERVER|ServerECIRequest:{initialize|terminate} Count

Write/Read Object Count CTGSERVER|ServerECIRequest:{read|write}Object Count

CTGServer - EPI Request dashboard


Request Execute time CTGSERVER|ServerEPIRequest:execute Response Time (ms)

Execute List time CTGSERVER|ServerEPIRequest:executeList Response Time (ms)

Request Execute Count CTGSERVER|ServerEPIRequest:execute Count

Execute List Count CTGSERVER|ServerEPIRequest:executeList Count




Request Initialize/Terminate time

CTGSERVER|ServerEPIRequest:{initialize|terminate} Response Time (ms)

Write/Read Object time CTGSERVER|ServerEPIRequest:{read|write}Object Response Time (ms)

CicsCp Request Execute Response Time

CTGSERVER|ServerCicsCpRequest|execute Response Time (ms)

CicsCp Request Count CTGSERVER|ServerCicsCpRequest|execute Count

Request Initialize/Terminate Count

CTGSERVER|ServerEPIRequest:{initialize|terminate} Count

Write/Read Object Count

CTGSERVER|ServerEPIRequest:{read|write}Object Count

CTGServer - Global Statistics dashboard

Label CTGServer performance metric

I/O Requests Processed (Aggregate)

The total number of TCP I/O requests that have been processed.

I/O Requests Processed Per Interval

The number of TCP I/O requests that have been processed in the last 15 second interval.

Incoming Connection Requests from Clients (Per Interval)

The number of new requests received from clients that have occurred in the last 15 second interval.

Total ECI/IPIC/EPI Requests to CICS (Per Interval)

The total number of requests (ECI, IPIC, and EPI) that have been processed in the last 15 second interval.

Connection Manager Threads Currently In Use

The number of threads currently being used to process client connections.

Connection Manager Threads Pool Count

The current number of threads that are allocated to process client connections.

Worker Threads Currently In Use

The number of threads currently being used to process CICS requests.

Worker Threads Pool Count

The current number of threads that are allocated to process CICS requests.

Request Exit metrics


CTGServer - SSL dashboard


JSSE SSL Aggregate Incoming SSL Handshakes

Total number of incoming SSL Handshakes

JSSE SSL Incoming SSL Handshakes Per Interval

Number of SSL handshakes per interval


The following tables show the metrics for the Request Exit feature.

Backends | CTG_to_CTG_ECI_IPIC_RequestExit Metrics

This table details the metrics for the Backends | CTG_to_CICS_ECI_IPIC_RequestExit that appear underneath the Investigator node.

Detailed Metric Name Description

Program Average Response Time (usec)

Average response time for all programs

Program Invocations Per Interval

Total invocations per interval for all programs

Program Aggregate Count ECI_IPIC

Aggregate number of times the program has been invoked

Program Aggregate Errors Aggregate errors for all programs

{server_name|program_name}:Average Response Time (usec)

Average response time for the program

{server_name|program_name}:Responses Per Interval

Responses (Invocations) per interval for the program

{server_name|program_name}:Concurrent Invocations

Number of concurrent invocations of the program within a given (15 second) interval

{server_name|program_name}:Stall Count

Number of stalls/hangs recorded for the program within a given (15 second) interval

{server_name|program_name}:Errors Per Interval

Number of errors recorded for the program within a given (15 second) interval




{server_name|program_name}:Aggregate Program Count ECI_IPIC

Total aggregate number of times the program has been invoked

{server_name|program_name}:Aggregate Errors

Total aggregate number of times an error was reported when the program was invoked

{server_name|program_name}:CommArea Aggregate Request Data

Total amount of COMMAREA data bytes that were sent up to CICS for this program

{server_name|program_name}:CommArea

Total amount of COMMAREA data bytes that were received from CICS for this program

CTG_Global_Statistics_RequestExit Metrics

This table details the metrics for the CTG_Global_Statistics_RequestExit. The detailed names of the metrics correspond to the names that IBM assigns to the metric. This table helps to minimize the confusion between what Introscope calls the metric and metric definitions from IBM.


CICS Server:Amount of CICS request data

Amount of request data (in bytes) sent to connected CICS servers. This amount includes both application and CICS protocol data.

CICS Server:Amount of CICS response data

Amount of response data (in bytes) received from connected CICS servers. This amount includes both application and CICS protocol data.

CICS Server:Current number of installed terminals

Current number of established and installed (EPI) Terminal sessions.

CICS Server:Current number of Orphaned CICS requests

Current number of requests that are waiting for a response from CICS for which the owning application timed out or ended.

CICS Server:Maximum number of active requests

The defined maximum number of active requests defined in the configuration file.

CICS Server:Number of CICS servers

Number of CICS servers to which the Gateway daemon has attempted to send a request.

CICS Server:Number of IPIC session failures

Number of failures on IPIC sessions to CICS servers.

CICS Server:Number of IPIC session in use

Number of IPIC sessions in use with CICS servers.






CICS Server:Number of active requests

Current number of active requests running in the client daemon

CICS Server:Number of connect failures

Number of times an attempt to connect to a CICS server has failed.

CICS Server:Number of defined CICS servers

Number of CICS servers defined in the configuration file.

CICS Server:Number of lost connections

Number of times an established connection with a CICS server has been lost.

CICS Server:Number of negotiated IPIC sessions

Number of IPIC sessions negotiated with CICS servers.

CICS Server:Number of orphaned CICS requests

Number of CICS server requests that were waiting for a response, timed out, or ended.

CICS Server:Number of requests processed

Number of CICS server requests that have been processed.

CICS Server:Number of requests waiting on a response

Number of requests currently waiting for a response from a CICS server.

CICS Server:Number of terminal install requests

Number of terminal install requests sent to CICS servers.

CICS Server:Number of terminal uninstall events

Number of terminal uninstall events processed by the Gateway.

CICS Server:Number of timed out connections

Number of times a connection to a CICS server has timed out.

CICS Daemon:Client daemon running Time

Length of time in seconds since the client daemon was successfully initialized.

CICS Daemon:Number of connected Client applications

Current number of client application processes connected to the client daemon.

CICS Daemon:Number of requests Processed

Number of API call requeststhat have been processed.

Connection Manager:Pool Count

Peak number of connection manager threads concurrently allocated to clients.

Connection Manager:Thread dispatch count

Number of allocations for connection manager threads representing the number of connections that have been established from clients.






Connection Manager:Threads allocated to clients

Current number of connection manager threads allocated to clients.

Gateway Daemon:Amount of client request data

Amount of request data, in bytes, received from the client applications.

Gateway Daemon:Amount of client response data

Amount of response data, in bytes, sent to the client applications.

Gateway Daemon:Average Gateway daemon response time

Average time taken (in milliseconds) for the Gateway daemon to respond to an API request from a client.

Gateway Daemon:End of interval time HHMMSS

The local time of the next scheduled interval statistics reset event. At this point, all interval statistics will be reset to zero.

Gateway Daemon:Extended LUW transactions committed

Number of extended LUW based 1 phase transactions that were committed.

Gateway Daemon:Extended LUW transactions rolled back

Number of extended LUW based 1 phase transactions that were rolled back.

Gateway Daemon:Failed SYNCONRETURN Transactions

Number of SYNCONRETURN transactions that have failed in the current interval.

Gateway Daemon:Gateway daemon running time

Length of time in seconds since the Gateway daemon was successfully initialized.

Gateway Daemon:Interval running Time

Length of time in seconds since the last interval reset event (the age of the current Interval).

Gateway Daemon:Length of the statistics interval HHMMSS

The duration of the statistics interval being used by a Gateway daemon.

Gateway Daemon:Logical End of Day time HHMMSS

The local time designated as the logical end of day by the Gateway daemon. At that point, all interval statistics are reset to zero.

Gateway Daemon:Number of CICS request exit calls

The number of Request Exit calls invoked by the Gateway Daemon.






Gateway Daemon:Number of Extended LUW transactions

The number of extended LUW transactions processed in the latest interval.

Gateway Daemon:Number of SYNCONRETURN transactions

The number of SYNCONRETURN transactions processed in the latest interval.

Gateway Daemon:Number of requests processed

The number of requests processed in the last interval.

Gateway Daemon:Successful SYNCONRETURN transactions

The number of SYNCONRETURN transactions that have succeeded during the Gateway daemon process.

Gateway Daemon:Total Requests Processed/Second

The aggregate number of API calls that have been processed.

Protocol Handler:SSL protocol handler port number

The TCP port being used for SSL traffic.

Protocol Handler:TCP protocol handler port number

The TCP port being used for non-SSL traffic.

System Environment:JVM GC count

The number of garbage collection events.

System Environment:JVM GC time

Milliseconds taken by the JVM for garbage collection.

System Environment:JVM heap size after GC

Size of the JVM heap size after the latest garbage collection.

System Environment:JVM initial heap size

Initial size of the JVM heap.

System Environment:JVM maximum heap size

Maximum size of the JVM heap.

Worker Threads:Current number of worker threads

Current number of worker threads that have been created.

Worker Threads:Currently allocated worker threads

Current number of worker threads that are being used by connection managers.






Worker Threads:Initial number of worker threads

The configured initial number of worker threads created by the Gateway daemon.

Worker Threads:Maximum number of worker threads

The maximum configured number of parallel worker threads that the Gateway can process.

Worker Threads:Number of times worker timeout reached

Number of times the Gateway daemon failed to allocate a worker thread to a connection manager within the configured timeout time.

Worker Threads:Peak number of allocated worker threads

Peak number of worker threads that were concurrently allocated to connection manager threads.

apm_9.5--apm for ibm cics transaction gateway guide

Documents

ca software

ca database performance

ca copyright notices

proprietary information

applicable license agreement

software products

available copies

internal use