ims problem analysis via log records - ims ug april 2012 victoria

47
1 © 2012 IBM Corporation IMS Performance Analyzer IMS Tools – Technical Sales Support Dennis Eichelberger [email protected]

Upload: ibm-ims

Post on 20-May-2015

1.275 views

Category:

Technology


2 download

TRANSCRIPT

Page 1: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

1© 2012 IBM Corporation

IMS Performance Analyzer

IMS Tools – Technical Sales SupportDennis [email protected]

Page 2: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

2 © 2012 IBM Corporation

� Overview

� Input Specification

� ISPF Dialog

� Reporting

� Summary

IMS Performance Analyzer

Page 3: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

3 © 2012 IBM Corporation

IMS Performance Analyzer Highlights

IMS Performance Analyzer processes IMS Log, Monitor, IMS Connect event data, and OMEGAMON TRF data to provide comprehensive reports for use by IMS specialists to tune their IMS systems, and managers to verify service levels and predict trends.

IMS PA provides detailed information about transit times (actual system performance time), and IMS resource usage and availability. IMS PA can process logs from a

Page 4: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

4 © 2012 IBM Corporation

IMS Performance Analyzer Highlights

• Supports all versions of IMS

• Accepts input from the IMS Logs, DC Monitor, IMS Connect Journals, andOMEGAMON TRF and ATF data.

• Provides comprehensive performance analysis and tuning assistance for IMS.

• Delivers end-to-end transit analysis for all types of transaction workloads, including shared-queues, by merging sysplex log

files.• Delivers end-to-end IMS Connect and IMS Log reporting; providing a

complete picture of the life-cycle of transactions as they pass through Connect and into IMS.

• Offers DBRC Log selection for quick and easy log report requests.• Comprehensive IMS monitor reporting including Fast Path.• Analyze transaction response time.• Report Forms allow you to design your own reports.

Page 5: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

5 © 2012 IBM Corporation

IMS Performance Analyzer Highlights

• Measure usage and availability of important resources, including databases, programs, regions, buffers (including database) and queues (message and other internal queues).

• Plan for IMS operational management, including scheduling database reorganizations, monitoring service level adherence, charge-back

accounting and capacity planning. • Monitor significant system events that can adversely affect system

performance and availability.• Boost system and application programmer productivity.• Report critical performance information -- from high-level management

summaries to detailed traces for in-depth analysis.• Analyze the impact of IMS Connect on transaction performance.• Provides an ISPF dialog and batch commands to manage reporting

requirements across your entire IMS enterprise.

Page 6: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IMS Performance Analyzer

Page 7: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

Performance Analyzer Inputs

Page 8: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IMS Logs

• Transaction Transit Times

• Transaction Traffic

• Resource Usage

• Resource Availability

• Fast Path Transaction Transit

• Fast Path Transaction Resource Usage

• Traces

Page 9: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IMS Connect Journals

• Transaction Transit – Internal and External

• Resource Availability

• Session Errors

IMS DC Monitor

• System Options

• Region Activity

• System Analysis

• Program Activity

• Database Activity

• Monitor Trace

Page 10: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

Performance Analyzer ISPF Dialog

Page 11: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IMS Performance Analyzer

– Ex 'ipi.sipiexec(ipiexec)'

| | File Options Help | ----------------------------------------------------------------------------| IMS Performance Analyzer 4.2 - Primary Option Menu || Option ===> ________________________________________________________________ || | 0 IMS PA Profile Customize your IMS PA dialog profile| 1 System Definitions Specify IMS and Connect systems and OMEGAMON files || 2 Groups Specify Groups of IMS and Connect systems (Sysplexes) || 3 Report Sets Request and submit reports and extracts || 4 Expectation Sets Define Expectation Sets (Log exception reporting) || 5 Averages Edit Averages data sets (Log exception reporting) || 6 Object Lists Define Object Lists | 7 Distributions Define Distributions | 8 Graphing & Export Graph or export Log Extract by Interval data || 9 IMS Connect Submit IMS Connect report requests | 10 Report Forms Define Report Forms | X Exit Terminate IMS PA | |__________________________________________________________________________________|

Page 12: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IMS Performance Analyzer

Page 13: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IMS Performance Analyzer

Page 14: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IMS Performance Analyzer

Page 15: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

Performance Analyzer Reports

Page 16: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

Performance Analyzer Reports

• Log Reports• Monitor Reports• IMS Connect Reports• Omegamon TRF Reports• Omegamon ATF Reports

IMS PA provides an extensive suite of reports to help administrators monitor resources utilization and transaction performance of IMS TM and DB systems. There are standard reports or form based reports may be tailored to individual needs.

Page 17: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

Performance Analyzer Management Reports

The Management reports provide a highlevel overview of the IMS systemhighlighting potential items for furtherreporting.• Dashboard� Transaction Transit Graphic

Summary� Transaction Transit extract by

Interval• Transaction Exception

• Management Exception

The Monitor reports provide analysts and programmers complete information for programanalysis and problem resolution.

• Region Activity Summary� Region Activity Analysis• System Analysis• Resource Usage

• Fast Path Analysis

• Monitor Data AnalysisThe Detail reports provide analysts andprogrammers complete information forprogram analysis and problem resolution.

Transaction Transit Log– Transit activity of each message

� Region Histogram� Transaction scheduling patterns

• DC Queue Manager Trace– Time based list of each TM event

• Database Trace

– Before and after versions of updates by program, time

The Analysis reports provide IMS systemadministrators with summary informationregarding the operating IMS system.

• Transaction Transit Statistics� Transaction Transit Analysis• Transaction Resource Usage

• Resource Availability• CPU Usage

• Deadlocks

• Database Update Activity

• Checkpointing

Page 18: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IMS Performance Analyzer

Transaction Resource Usage

Data from 05.30.49 07Nov2006 to 05.45.18 07Nov2006 Page 1

Avg Avg Avg Avg <=1.0 1.0-2.0 >2.0 Avg Avg

Tran InputQ Process OutputQ Total Total Total Total CPU DB Call

Trancode Count Time Time Time IMS Time IMS Time IMS Time IMS Time Time Count

DSN8CS 7 0.011024 0.930524 0.000000 0.941549 71.43% 0.00% 28.57% 0.024617 0

DSN8PP 7 0.009017 0.627191 0.000000 0.636208 85.71% 14.29% 0.00% 0.018776 0

DSN8PS 21 0.011219 0.427845 0.007012 0.446075 100.00% 0.00% 0.00% 0.017876 0

DSN8PT 6 0.010506 0.373590 0.035327 0.419423 100.00% 0.00% 0.00% 0.018377 0

Total 41 0.010705 0.539763 0.008761 0.559229 92.68% 2.44% 4.88% 0.019254 0

Daily report flags that 28% of DSN8CS transaction were above the 2 second SLA

IMS PA daily performance summary report

Page 19: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

Forms: database updates and ESAF activity by transa ction

IMS Tran CPU InputQ Process OutputQ TotalStart Trancode PST Time Time Time Time IMS Time . . . 09.49.26.679840 MQATREQ1 1 0.026658 0.000145 1.724738 0.000000 1.72488310.37.00.753450 MQATREQ2 2 0.015126 0.000221 0.491174 0.000000 0.49139510.37.41.829685 MQATREQ3 3 0.015126 0.000150 0.386636 0.000000 0.38678610.38.09.060759 MQATREQ4 4 0.015126 0.000265 0.404939 0.000000 0.405204

Total IO DB IO VSAMRead VSAMWrit ESAFcallTrancode . . . Count Time Count Count CountMQATREQ1 7 0.000384 2 3 267MQATREQ2 12 0.001034 4 7 57 MQATREQ3 16 0.001287 6 11 62 MQATREQ4 18 0.001564 9 16 71

Bad response time! What caused it?

Perhaps it was an external subsystem,but which one and what happened?

Page 20: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

20 © 2012 IBM Corporation

IMS Performance Analyzer

• IMS Performance Analyzer is the premier tool for:– Service Levels – Capacity planning– Response time analysis

• End-to-end transit analysis for all database and transaction workloads– Provides a complete picture of the life cycle of IMS transactions

– 100’s of fields can be summarized, analyzed and reported

• IMS PA can use its forms-based reporting to identify specific processing deficiencies

• ISPF Dialog interface

Page 21: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

21© 2012 IBM Corporation

IMS Problem Investigator

IMS Tools – Technical Sales SupportDennis Eichelberger

Page 22: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

22 © 2012 IBM Corporation

IMS Problem Investigator

� Overview

� Input Specification

� ISPF Dialog

� Reporting

� Benefits

Page 23: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

© 2011 IBM Corporation and ICBC

IBM IMS Problem Investigator is a problem analysis tool for Information Management System Database (IMS DB) and Transaction Manager (IMS TM). It enables IMS administrators and programmers to interactively explore formatted, interpreted, and easily customizable views of log records. Users may identify and analyze problems quickly, without requiring an expert understanding of log data structures and the relationships between log records.

IMS Problem Investigator connects log records from across various sources to create a composite view of the entire transaction. This allows a user to walk through events in the lifecycle of the transaction, even as it spans multiple IMS systems, IMS Connect, Websphere MQ, and DB2. As a result you can rapidly isolate problems in even the most complex transactions to a specific area and then solve them in significantly less time.

IMS Problem Investigator

Page 24: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

© 2011 IBM Corporation and ICBC

IMS Problem Investigator supports input from the following types of recorded data

• IMS log records• IMS Transaction Index created by IMS Performance Analyzer

• IMS DC monitor and DB monitor files

• Common Queue Server (CQS) log stream

• IMS Connect event data collected by IMS Connect Extensions

• OMEGAMON® Transaction Reporting Facility (TRF) log

• DB2 log

• WebSphere® MQ log extract

• SMF – IRLM Long Lock records

IMS Problem Investigator

Page 25: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

© 2011 IBM Corporation and ICBC

• Modern IMS transactions affect multiple subsystems

• Information about these transactions is recorded in even more and disparate sources

• IMS PI provides a unified and logical view of transactions spanning all these systems

IMS log

IMS transaction

IMS Problem Investigator

Data sources

Transaction activity merged into a single view

IMS-related subsystems

Transaction activity logged in multiple locations

IMS monitor

TRF Extractordata set

Extracts

CQS logDB2 log

DB2 CQSIMS DB/TMIMS ConnectWebSphere

MQ

WebSphere MQlog extract

IMS ConnectExtensions

journal

SMF

ISPFdialog

Reports

OMEGAMONTRF records

IRLM long-lockrecords

User records

IMS ConnectExtensions

IMS PATransaction

Index

Page 26: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

© 2011 IBM Corporation and ICBC

Analysis of data can by performed through an ISPF dialog, batch reports, and REXX™ programming services. Filtered extracts can be created for more efficient problem investigation. Through the IMS Problem Investigator ISPF dialog you can:

• View formatted logs with detailed field descriptions.

• Navigate to an exact point in time within a log file.

• Investigate specific problem areas. For example, transaction, database, security, or checkpoint processing.

• Merge log files to combine different aspects of IMS processing into a single view.

• Track and show all events from a single transaction across multiple subsystems or an entire IMS PLEX.

• Determine response times and latencies.

IMS Problem Investigator

Page 27: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

Reduce time and costs associated with IMS problem determination

Reduce time and costs associated with IMS problem determination

IMS Problem Investigator (IMS PI)

• IMS PI allows you to interactively browse and analyze log records

• Instantaneous view of the logs

• Navigate by the time of day

• Select records to drill down right to the values of individual flag bits

• Gain an end-to-end picture of transactions in the Sysplex (a complex and time intensive activity)

• Determine transaction times and event latencies

Page 28: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IMS Problem Investigator

Start of Connect transaction

Start of IMStransaction

Connectresumes

Connect transaction ends after 0.278 seconds

Page 29: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IMS PI

Sudden jump in elapsed time indicates DB2 insert took over 2 seconds! Select the event to view details and begin analysis.

Page 30: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

© 2011 IBM Corporation and ICBC

• Time slicing: Process large log files with the responsiveness and convenience of a small extract fi le

• Transaction index: Extend the insights of transaction performance recorded by IMS PA to help analyze transactions in IMS PI

• Unit of work tracking: Isolate a single unit of rec overy (TU) within a transaction (TX) with a long program switch sequence

• Support for the DB2 and MQ logs

• Extract to CSV: Import the required fields from selected log record types into a DB2 database or PC spreadsheet

• OMEGAMON TRF: transaction accounting, DLI and DB2 calls

• OMEGAMON ATF: transaction call trace

IMS Problem Investigator Methods

Page 31: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

© 2011 IBM Corporation and ICBC

Track and merge all types of logs We have reached end of the transaction we are tracking.Observe all the syncpoint events in the IMS and DB2 logs, as well as the OMEGAMON ATF call trace, and end of transaction events:

BROWSE JCH.INDEX Record 00427560 More: < >Command ===> Scroll ===> CSR

Forwards / Backwards . . HH.MM.SS.THMIJU Time of Day . . HH.MM.SS.THMIJUCode Description Date 2009-07-10 Friday Time (Relative)

/ ---- ------------------------------------------------------ ----------------05 DLI ISRT TranCode=MQATREQ1 Region=0001 +0.5908315600 Commit Prepare starting Region=0001 SSID=CSQ6 +0.59177887 Generic ESS TranCode=MQATREQ1 Region=0001 SSID=CSQ6 +0.59388287 Generic ESS TranCode=MQATREQ1 Region=0001 SSID=DB3A +0.5939570020 DB2 Unit of Recovery Control - End Commit Phase 1 +0.59413503 Output Message Response LTerm=FUNTRM07 +0.59690735 Output Message Enqueue LTerm=FUNTRM07 Region=0001 +0.59693937 Syncpoint Region=0001 +0.59697637 Syncpoint message transfer Region=0001 +0.59701333 Free Message +0.59705331 Communications GU LTerm=FUNTRM07 +0.59726987 Generic ESS TranCode=MQATREQ1 Region=0001 SSID=CSQ6 +0.59907536 Output Message Dequeue LTerm=FUNTRM07 +0.60191633 Free Message +0.6019805600 Commit Continue completed Region=0001 SSID=CSQ6 +0.60262387 Generic ESS TranCode=MQATREQ1 Region=0001 SSID=DB3A +0.60262605 DLI Detail TranCode=MQATREQ1 Region=0001 Func=ASRT +0.5917480020 DB2 Unit of Recovery Control - Begin Commit Phase 2 +0.6027110020 DB2 Unit of Recovery Control - End Commit Phase 2 +0.6044075600 Commit Continue completed Region=0001 SSID=DB3A +0.6052225612 End of Phase 2 Syncpoint Program=MQATPGM Region=0001 +0.60523407 Application Terminate TranCode=MQATREQ1 Region=0001 +0.605362F1 Transaction End TranCode=MQATREQ1 Region=0001 +0.605699 04 Summary Completion TranCode=MQATREQ1 Region=0001 +0.605699

******************************** Bottom of Data *******************************

Page 32: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

32 © 2012 IBM Corporation

IMS Problem Investigator Benefits

� Efficiently identify and resolve IMS transaction problems.

� Interactively track potential problem transactions across events in several components.

� Quickly access the data needed at the identified time without scanning the entire log stream.

� Merge data from disparate systems in time sequence for a single point of view.

� Increase productivity by allowing programmers to view the data without spending excessive time extracting and correlating data.

� Complements IMS Performance Analyzer

Page 33: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

33© 2012 IBM Corporation

IMS Performance AnalyzerIMS Problem Investigator

IMS Tools – Technical Sales SupportDennis Eichelberger

Page 34: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IBM Software Group

34© 2010 IBM Corporation and ICBC

IMS Performance Analyzer and IMS Problem Investigator

� IMS Performance Analyzer (IMS PA) is tool that uses the IMS and related logs to create performance and throughput reports.

� IMS Problem Investigator (IMS PI) is a log analysis tool that allows users to interactively browse IMS and related logs.

� These tools offer significant value to the customer independently of each other. But the greatest benefit is obtained when they are used together.

Page 35: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IBM Software Group

35© 2010 IBM Corporation and ICBC

Usage scenarios

� In an enterprise environment, when a performance problem is identified, it could be caused by any of these subsystems:�IMS Connect

�OTMA

�Shared queues

�IMS TM

�IMS DB

�DB2

�MQSeries

�RACF

�Etc…

Page 36: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IBM Software Group

36© 2010 IBM Corporation and ICBC

Usage scenario (inspired by a real use case)

� To analyze the problem we need to use IMS Connect Extensions, IMS Performance Analyzer, and IMS Problem Investigator

� The combination of these tools will help isolate a performance problem to a particular system and often identify the underlyingcause of the problem

� All three tools are needed because:�Without IMS Connect Extensions the customer will not be able to

identify if OTMA or IMS Connect is causing the problem

�Without IMS Problem Investigator:

� the customer will not be able to drill-down and see exactly what events are associated with the problem

� the customer will not be able to correlate the transaction records with data in MQSeries and DB2 logs

Page 37: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IBM Software Group

37© 2010 IBM Corporation and ICBC

Scenario 1: TCP/IP reported problem

Page 38: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IBM Software Group

38© 2010 IBM Corporation and ICBC

IMS Performance Analyzer reportIMS Performance Analyzer 4.1

combined tran list

________________________________

0LIST0001 Printed at 19:33:38 12Dec2007 Data from 13.57.52 12Dec2007

CON Tran CON Resp PreOTMA OTMAproc IMS Tran InputQ Process Total PostOTMA

Start Trancode OTMA Time Time Time Start Time Time IMS Time Time

13.57.52.714 IMSTRANS CONNECT 1.810 0.000 1.803 13.57.54.517 0.000 0.001 0.001 0.006

13.57.52.964 IMSTRANS CONNECT 1.575 0.000 1.574 13.57.54.538 0.000 0.001 0.001 0.000

13.57.52.972 IMSTRANS CONNECT 1.588 0.000 1.588 13.57.54.548 0.009 0.002 0.011 0.000

13.57.53.091 IMSTRANS CONNECT 1.716 0.002 1.714 13.57.54.806 0.000 0.001 0.001 0.000

13.57.53.567 IMSTRANS CONNECT 1.839 0.000 1.839 13.57.55.403 0.000 0.002 0.002 0.000

13.57.54.044 IMSTRANS CONNECT 1.800 0.000 1.799 13.57.55.836 0.006 0.001 0.007 0.001

13.57.53.800 IMSTRANS CONNECT 1.879 0.000 1.878 13.57.55.677 0.000 0.001 0.001 0.000

13.57.54.120 IMSTRANS CONNECT 1.851 0.000 1.850 13.57.55.903 0.006 0.001 0.007 0.000

13.57.54.213 IMSTRANS CONNECT 1.904 0.000 1.903 13.57.56.116 0.000 0.001 0.001 0.000

13.57.54.251 IMSTRANS CONNECT 1.931 0.000 1.930 13.57.56.180 0.000 0.001 0.001 0.000

13.57.54.713 IMSTRANS CONNECT 2.007 0.001 2.005 13.57.56.718 0.000 0.001 0.001 0.000

13.57.55.461 IMSTRANS CONNECT 2.207 0.000 2.206 13.57.57.665 0.000 0.002 0.002 0.000

13.57.55.632 IMSTRANS CONNECT 2.070 0.001 2.069 13.57.57.700 0.000 0.001 0.001 0.001

13.57.55.890 IMSTRANS CONNECT 2.061 0.002 2.055 13.57.57.946 0.000 0.001 0.001 0.003

13.57.56.147 IMSTRANS CONNECT 2.171 0.002 2.169 13.57.58.314 0.000 0.003 0.003 0.000

13.57.56.190 IMSTRANS CONNECT 2.158 0.001 2.157 13.57.58.347 0.000 0.001 0.001 0.000

13.57.56.559 IMSTRANS CONNECT 2.222 0.000 2.222 13.57.58.780 0.000 0.001 0.001 0.000

13.57.56.909 IMSTRANS CONNECT 2.048 0.002 2.045 13.57.58.955 0.000 0.002 0.002 0.000

13.57.56.934 IMSTRANS CONNECT 2.033 0.001 2.031 13.57.58.966 0.000 0.001 0.001 0.000

Without IMS Connect Extensions, IMS Connect and OTMA performance cannot be obtained

IMS is showing millisecond response times while OTMA is taking seconds

Page 39: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IBM Software Group

39© 2010 IBM Corporation and ICBC

IMS PI View of the problem

A03C Prepare READ Socket 13.57.58.037571A049 READ Socket +0.000048A03D Message Exit called for READ +0.000063A03E Message Exit returned from READ TranCode=XXXXXXXXX +0.000082A03F Begin SAF call +0.000097A040 End SAF call +0.000539A03F Begin SAF call +0.000548A040 End SAF call +0.000658A041 Message sent to OTMA Datastore=MMMMMM +0.00070701 Input Message TranCode=XXXXXXXXX Source=Connect +2.48562135 Input Message Enqueue TranCode=XXXXXXXXX +2.48563608 Application Start TranCode=XXXXXXXXX Region=0184 +2.4857095607 Start of UOR Program=XXXXXXXX Region=0184 +2.48571031 DLI GU TranCode=XXXXXXXXX Region=0184 +2.48573603 Output Message Response LTerm=9999 Source=Connect +2.48871631 DLI GU TranCode=XXXXXXXXX Region=0184 +2.48872633 Free Message +2.4887745610 Start Phase 1 Syncpoint Region=0184 +2.48878437 Syncpoint Region=0184 +2.48880733 Free Message +2.4888495600 Commit found no work to do Region=0184 SSID=AAAA +2.4888595612 End of Phase 2 Syncpoint Program=YYYYYYY +2.488864A042 Message received from OTMA Datastore=MMMMMM +2.488917A042 Message received from OTMA Datastore=MMMMMM +2.489037A03D Message Exit called for XMIT +2.489926A03E Message Exit returned from XMIT +2.489943A04A WRITE Socket +2.489999A048 Trigger Event +2.490032

IMS Connect sends to IMS OTMA

OTMA finally logs the 01 record

Page 40: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IBM Software Group

40© 2010 IBM Corporation and ICBC

Scenarion 2: DB2 problem

Page 41: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IBM Software Group

41© 2010 IBM Corporation and ICBC

IMS PA daily performance summary report

IMS Performance Analyzer

Transaction Resource Usage

Data from 05.30.49 07Nov2006 to 05.45.18 07Nov2006 Page 1

Avg Avg Avg Avg <=1.0 1.0-2.0 >2.0 Avg Avg

Tran InputQ Process OutputQ Total Total Total Total CPU DB Call

Trancode Count Time Time Time IMS Time IMS Time IMS Time IMS Time Time Count

DSN8CS 7 0.011024 0.930524 0.000000 0.941549 71.43% 0.00% 28.57% 0.024617 0

DSN8PP 7 0.009017 0.627191 0.000000 0.636208 85.71% 14.29% 0.00% 0.018776 0

DSN8PS 21 0.011219 0.427845 0.007012 0.446075 100.00% 0.00% 0.00% 0.017876 0

DSN8PT 6 0.010506 0.373590 0.035327 0.419423 100.00% 0.00% 0.00% 0.018377 0

Total 41 0.010705 0.539763 0.008761 0.559229 92.68% 2.44% 4.88% 0.019254 0

Daily report flags that 28% of DSN8CS transaction were above the 2 second SLA

Page 42: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IBM Software Group

42© 2010 IBM Corporation and ICBC

IMS PA Forms-based list report

IMS Performance Analyzer

Transaction Transit Log

Data from 05.30.49 07Nov2006 Page 1

Org DB Call IMS Tran CPU InputQ Process OutputQ Total

LTERM Trancode Count Start Time Time Time Time IMS Time

. . .

FUNTRM69 DSN8PT 0 05.39.50.738060 0.023359 0.007564 0.199647 0.121131 0.328342

FUNTRM69 DSN8PS 0 05.43.38.400003 0.014557 0.020709 0.287724 0.000000 0.308433

FUNTRM69 DSN8CS 0 05.44.52.678102 0.046589 0.013049 2.037216 0.000000 2.050265

. . .

We can see an offending transaction from the approximate time of day

This form lists all the transactions that ran on th e day

Page 43: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IBM Software Group

43© 2010 IBM Corporation and ICBC

Gather the data required for the analysis

� Transaction DSN8CS uses DB2, which may be the cause of the problem.

� We need to gather the IMS log, DB2 log and SMF file for time period around when the problem occurred.

� IMS PI will merge all the log records and allow us to track the transaction across all these log files…

Page 44: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IBM Software Group

44© 2010 IBM Corporation and ICBC

Forwards / Backwards . . 00.00.00.000100 Time of Day . . 05.44.52.678102Code Description Date 2006-11-07 Tuesday Time (Elapsed)

/ ---- ------------------------------------------------------ ----------------01 Input Message TranCode=DSN8CS 05.44.52.67810235 Input Message Enqueue TranCode=DSN8CS 0.00791908 Application Start TranCode=DSN8CS Region=0001 0.0008075607 Start of UOR Program=DSN8IC0 Region=0001 0.00000031 DLI GU TranCode=DSN8CS Region=0001 0.0043155616 Start of protected UOW Region=0001 0.0002625600 Sign-on to ESAF Region=0001 SSID=DB2P 0.1657985600 Thread created for ESAF SSID=DB2P 0.0000410020 DB2 Unit of Recovery Control - Begin UR 1.6518710020 DB2 Exclusive Lock on Pageset Partition or DBD 0.0000160020 DB2 Insert into a Data Page 0.0000000020 DB2 Update Area Map/Spacemap Bit Changes or Whole Byte 0.0000320020 DB2 Type 2 Index Update 0.0019525600 Commit Prepare starting Region=0001 SSID=DB2P 0.0098630020 DB2 Unit of Recovery Control - End Commit Phase 1 0.01420003 Output Message Response LTerm=FUNTRM69 0.00406735 Output Message Enqueue LTerm=FUNTRM69 Region=0001 0.00002237 Syncpoint Region=0001 0.00002437 Syncpoint message transfer Region=0001 0.00093133 Free Message 0.00393603 Output Message Response LTerm=FUNTRM69 0.00759235 Output Message Enqueue LTerm=FUNTRM69 0.00003731 Communications GU LTerm=FUNTRM69 0.00005036 Output Message Dequeue LTerm=FUNTRM69 0.03422733 Free Message 0.001001101 DB2 Accounting SSID=DB2P SYSID=FTS1 0.13292533 Free Message 0.0022560020 DB2 Unit of Recovery Control - Begin Commit Phase 2 0.0001270020 DB2 Unit of Recovery Control - End Commit Phase 2 0.0020165600 Commit Continue completed Region=0001 SSID=DB2P 0.0039505612 End of Phase 2 Syncpoint Program=DSN8IC0 0.00000807 Application Terminate TranCode=DSN8CS Region=0001 0.000529

******************************** Bottom of Data *******************************

Page 45: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IBM Software Group

45© 2010 IBM Corporation and ICBC

Forwards / Backwards . . 00.00.00.000100 Time of Day . . 05.44.52.678102Code Description Date 2006-11-07 Tuesday Time (Elapsed)

/ ---- ------------------------------------------------------ ----------------E 01 Input Message TranCode=DSN8CS 05.44.52.678102

35 Input Message Enqueue TranCode=DSN8CS 0.00791908 Application Start TranCode=DSN8CS Region=0001 0.0008075607 Start of UOR Program=DSN8IC0 Region=0001 0.00000031 DLI GU TranCode=DSN8CS Region=0001 0.0043155616 Start of protected UOW Region=0001 0.0002625600 Sign-on to ESAF Region=0001 SSID=DB2P 0.1657985600 Thread created for ESAF SSID=DB2P 0.000041----------------------------------------------------------------------------0020 DB2 Unit of Recovery Control - Begin UR 1.651871

Userid=MKR IMSID=I9DE LUW=FTS1 /DB2PLU /BFAAB47AF91B/0001URID=0003EC74CC34

----------------------------------------------------------------------------0020 DB2 Exclusive Lock on Pageset Partition or DBD 0.0000160020 DB2 Insert into a Data Page 0.0000000020 DB2 Update Area Map/Spacemap Bit Changes or Whole Byte 0.0000320020 DB2 Type 2 Index Update 0.0019525600 Commit Prepare starting Region=0001 SSID=DB2P 0.0098630020 DB2 Unit of Recovery Control - End Commit Phase 1 0.01420003 Output Message Response LTerm=FUNTRM69 0.004067

…36 Output Message Dequeue LTerm=FUNTRM69 0.03422733 Free Message 0.001001----------------------------------------------------------------------------101 DB2 Accounting 0.132925

RecToken=I9DE/0000001900000000CPU1=00.030586 CPU2=00.000000 I/O3=00.000000 SSID=DB2P SYSID=FTS1GtPgRq=152 SyPgUp=11 Suspnd=0 DeadLk=0 TimOut=0 MxPgLk=6Des=0 Pre=0 Ope=1 Fet=12 Clo=1

----------------------------------------------------------------------------33 Free Message 0.0022560020 DB2 Unit of Recovery Control - Begin Commit Phase 2 0.0001270020 DB2 Unit of Recovery Control - End Commit Phase 2 0.0020165600 Commit Continue completed Region=0001 SSID=DB2P 0.0039505612 End of Phase 2 Syncpoint Program=DSN8IC0 0.000008----------------------------------------------------------------------------07 Application Terminate 0.000529

UTC=05.44.54.728881 TranCode=DSN8CS Program=DSN8IC0 Region=0001RecToken=I9DE/0000001900000000RegTyp=MPP MCNT=1 DBDLI=0 DCDLI=2 CPU=00.046589

----------------------------------------------------------------------------

This is a DB2 start of UOR: uniquely identifying the transaction so that we can use DB2 analytical tools should they be required

This is an SMF accounting record. It shows high page fetch and updated activity, as well as high CPU usage

The IMS log termination event shows that most of the CPU time was spent in DB2

Page 46: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IBM Software Group

46© 2010 IBM Corporation and ICBC

� :

IMS PerformanceAnalyzer

IMS ProblemInvestigator

IMS ConnectExtensions

Navigate IMS log online

Reports from IMS log

Navigate IMS Connectdata online

Investigate IMS Connectproblems, too

Get performance reportsfor IMS Connect, too

EnhancedIMS Connect reports

Page 47: IMS Problem Analysis via Log Records - IMS UG April 2012 Victoria

IBM Software Group

47© 2010 IBM Corporation and ICBC

Combined Benefits

� Isolate problems in complex interrelated components in hours instead of weeks, and so reduce down-time and lower staffing costs.

� Dissect IMS resource utilization to optimize hardware usage, identify unnecessary overhead and plan for future requirements.

� Enable less experienced staff to perform advanced analysis.

� Map the life-cycle of individual transactions to gain a better understanding of the IMS environment and to preemptively eliminate potential problems.