Download - 3452 - Managing your applications
Who is this presentation aimed at?
• Application developers who want to understand how their application is
really interacting with MQ
– Does the reality match the design and expectation?
• MQ administrators who want to understand how applications are
interacting with their queue manager
– Is the application behaving as described?
– Has the application been coded to required standards?
– Is the application exhibiting best practice?
What are accounting messages?
• Information about the MQI operations performed by an application
• PCF messages
– Delivered to SYSTEM.ADMIN.ACCOUNTING.QUEUE
• When the application disconnects and…
• When the specified interval has been reached (ACCTINT)
– Not persistent
What are accounting messages?
• Two types
– MQI accounting
• MQI accounting messages contain information relating to the number of MQI calls
made using a connection to a queue manager.
– Queue accounting
• Queue accounting messages contain information relating to the number of MQI calls
made using connections to a queue manager, grouped by queue.
How do I enable accounting messages?
• Queue manager attributes
– ACCTMQI
– ACCTQ
• Queue attributes
– ACCTQ
• MQCONNX connection options
– MQCNO_ACCOUNTING_MQI_ENABLED
– MQCNO_ACCOUNTING_MQI_DISABLED
– MQCNO_ACCOUNTING_Q_ENABLED
– MQCNO_ACCOUNTING_Q_DISABLED
How do I read accounting messages?
• Read PCF messages from SYSTEM.ADMIN.ACCOUNTING.QUEUE
• amqsmon sample
– Some output is in cryptic list form, example:
• OpenCount: [0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
• This means 2 objects of type MQOT_Q (queue) were opened
• Demystify via documentation:
– Search ‘MQI accounting message data’
• Vendors
• Roll your own
amqsmon example output (MQI accounting)
MonitoringType: MQIAccounting
QueueManager: 'MQ80'
IntervalStartDate: '2016-02-16'
IntervalStartTime: '18.16.56'
IntervalEndDate: '2016-02-16'
IntervalEndTime: '18.17.00'
CommandLevel: 800
ConnectionId: x'41…02e01'
SeqNumber: 0
ApplicationName: 'C:\WMQ\8.0\bin64\amqsput.exe'
ApplicationPid: 35388
ApplicationTid: 1
UserId: 'timmccor'
ConnDate: '2016-02-16'
ConnTime: '18.16.56'
DiscDate: '2016-02-16'
DiscTime: '18.17.00'
DiscType: Normal
OpenCount: [0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
OpenFailCount: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
CloseCount: [0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
CloseFailCount: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
PutCount: [2, 0]
– Much more data is logged than appears
on this slide
– Interval start/end matches connection
life, means no intermediates generated
– See useful information about the
application connected
– Most interesting fields are
[-,Q,-,-,-,QMGR,-,-,Topic,-,-,-,-,-]
We opened 2 queues.
– PutCount: [Nonpersistent,Persistent].
We put 2 non-persistent messages
amqsmon example output (queue accounting)
MonitoringType: QueueAccounting
QueueManager: 'MQ80'
ApplicationName: 'C:\WMQ\8.0\bin64\amqsput.exe'
UserId: 'timmccor'
QueueAccounting: 0
QueueName: 'Q1'
CreateDate: '2015-02-05'
CreateTime: '13.45.58‘
QueueDefinitionType: Local
OpenCount: 1
OpenDate: '2016-02-18'
OpenTime: '14.04.20'
CloseCount: 1
CloseDate: '2016-02-18'
CloseTime: '14.04.22'
PutCount: [1, 0]
PutFailCount: 0
Put1Count: [0, 0]
Put1FailCount: 0
PutBytes: [4, 0]
PutMinBytes: [4, 0]
PutMaxBytes: [4, 0]
– Much more data is logged than appears
on this slide
– See useful information about the
application connected and when it
opened and closed the queue
– See useful information about how the
queue was defined
– PutCount: [Nonpersistent,Persistent].
We put 1 non-persistent messages
When should I use accounting messages?
• Low impact on system
• Scoped to an individual connection or queue
– Connection level information, won’t provide much detail on the objects accessed
– Queue level information may be too broad
What is API trace?
• API trace is a part of the diagnostic trace IBM support will request in
PMRs
– Cut down to show only the API calls and less of the internal working of the system
– Intended for IBM support, but can be useful for customers
How do I enable API trace?
• strmqtrc –t api –p myapplicationname
– Start API trace for applications with named ‘myapplicationname’
– Don’t forget the .exe on Windows
• endmqtrc -a
– Stop trace
How do I read API trace?
• MQ_DATA_DIR/trace/AMQPID.0.TRC
– Where PID is the process ID
• Trace is output in a binary non-readable format
– Run dspmqtrc to format the trace into human readable format
– Use dspmqtrc from the same installation which generated the trace
– Except on Windows where trace is output in human readable format
API trace example (header)
• Header at top of every file
• A 64-bit version of amqsput generated
this file
• Application was run on a Windows 7
system named MYHOST
• Application was at MQ version 8.0.0.4
• MQ install and data paths
• This install is MQ80, there are 3 others
on the system
Process : C:\WMQ\8.0\bin64\amqsput.exe (64-bit)
Arguments :
Host : MYHOST
Operating System : Windows 7 Professional x64 Edition, Build
7601: SP1
Product Long Name : WebSphere MQ for Windows (x64 platform)
Version : 8.0.0.4 Level : p800-004-151017
O/S Registered : 1
Data Path : C:\WMQ\data
Installation Path : C:\WMQ\8.0
Installation Name : MQ80 (3)
License Type : Production
UTC Date : 2016/02/18: Time : 11:52:44.759
LCL Date : 2016/02/18: Time : 11:52:44.759 GMT Standard Time
API trace example (trace records)
• The counter is a hexadecimal field which
increments by 1 for every trace record
• The timestamp is the time in the
timezone from the header
• PID.TID – Process ID and Thread ID
logging the trace record
• Ident provides additional identity
information depending on context
• Data is the ‘interesting’ data which was
traced
Counter TimeStamp PID.TID Ident Data
==================================================
000001DE 12:09:21.687697 25528.1 : !! - __________
000001DF 12:09:21.687713 25528.1 : !! - MQCONN >>
000001E9 12:09:21.702901 25528.1 CONN:525958 !! - MQCONN <<
API trace example (request)
• Only trace data shown on this
slide
• Requests in the form
MQIVERB >>
• Bytes displayed at their
memory address, ASCII
conversion to the right
• MQ compresses duplicate trace
lines
• Name: MQ80
4D = M 51 = Q 38 = 8 30 = 0
Padded by spaces (20)
!! - MQCONN >>
!! - Name:
Data:-
0x00000000 002AEAB0 4D 51 38 30 20 20 20 20 20 20 20 20 20 20 20 20 : MQ80
0x00000000 002AEAC0 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 :
0x00000000 002AEAD0 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 :
!! - ConnectOpts:
Data:-
0x00000000 002AF6E0 43 4E 4F 20 01 00 00 00 00 00 00 00 : CNO ........
!! - Hconn : Output Parm
!! - Compcode : Output Parm
!! - Reason : Output Parm
Data:-
0x00000000 002AF010 4C 43 4E 4F 04 00 00 00 00 40 00 00 00 00 00 00 : LCNO.....@......
0x00000000 002AF020 FF FF FF FF 00 00 00 00 00 00 00 00 00 00 00 00 : ................
0x00000000 002AF030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
7 lines suppressed, same as above
0x00000000 002AF0B0 00 00 00 00 00 00 00 00 FF FF FF FF 00 00 00 00 : ................
0x00000000 002AF0C0 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 : ................
0x00000000 002AF0D0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
API trace example (response)
• Only trace data shown on this
slide
• Responses in the form
MQIVERB <<
• Little endian platforms need to
be byte swapped
• MQOPEN returned Hobj 2
02 00 00 00 byte swaps to
00 00 00 02 = 2
• MQOPEN was a queue
• API call successful
!! - MQOPEN <<
!! - Hconn : Input Parm
!! - Objdesc:
Data:-
0x00000000 002AEDD0 4F 44 20 20 01 00 00 00 01 00 00 00 51 31 00 00 : OD ........Q1..
0x00000000 002AEDE0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
4 lines suppressed, same as above
0x00000000 002AEE30 00 00 00 00 00 00 00 00 00 00 00 00 41 4D 51 2E : ............AMQ.
0x00000000 002AEE40 2A 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : *...............
0x00000000 002AEE50 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
0x00000000 002AEE60 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : ................
0x00000000 002AEE70 00 00 00 00 00 00 00 00 : ........
!! - Options : Input Parm
!! - Hobj:
Data:-
0x00000000 002AF7B0 02 00 00 00 : ....
!! - ObjHdl:2 ObjType:QUEUE ObjName:Q1 ResObjName:Q1
!! - Compcode:
Data:-
0x00000000 002AF6D4 00 00 00 00 : ....
!! - Reason:
Data:-
0x00000000 002BFD60 00 00 00 00 : ....
!! - MQI:MQOPEN HConn=03400006 HObj=00000002 rc=00000000 ObjType=00000001
ObjName=Q1
When should I use API trace?
• Easy to enable
• Impact on disk I/O
• Difficult to read
– Intended as diagnostic information for IBM support teams
• Scoped to process
– All Java applications will be seen as ‘java’ as the JVM is the process
What is application activity trace?
• Similar to accounting messages, but…
– Much more detail on what the application was doing
– Can be controlled at the application level
– Could be used to audit application behavior
• PCF messages
– Delivered to SYSTEM.ADMIN.TRACE.ACTIVITY.QUEUE when…
– Application disconnects, or…
– ActivityCount or ActivityInterval thresholds are reached, or…
– When the size of the message which will be written reaches the maximum message length for the
queue
– Non-persistent messages
How do I enable application activity trace?
• Queue manager attribute
– ACTVTRC
• MQCONNX connection option
– ACTVCONO queue manager attribute must be enabled
– MQCNO_ACTIVITY_TRACE_DISABLED
– MQCNO_ACTIVITY_TRACE_ENABLED
• mqat.ini configuration file
– Found in the queue manager data directory
– Does not require queue manager restart, reconnect or
alter qmgr object in any way
– Can specify individual settings by application name Overrides
Application activity trace on the MQ Appliance
• Subscribe to application activity trace system topic
– $SYS/MQ/INFO/QMGR/my_qmgr/ActivityTrace/…
• mqat.ini configuration file
– Only global settings available on the MQ Appliance via setmqini
mqat.ini example, global settings
AllActivityTrace:
ActivityInterval=1
ActivityCount=100
TraceLevel=MEDIUM
TraceMessageData=0
StopOnGetTraceMsg=ON
– Application activity trace won’t be logged without at least one ApplicationTrace stanza, or the queue
manager attribute ACTVTRC set to on
– Log an application activity trace message every one second, or one hundred MQI or XA operations
– Log a medium amount of detail, documentation lists which operations are logged for each level
– Do not trace any message data for get or put operations
– Don’t generate application activity trace when handling application activity trace messages (to avoid
looping)
mqat.ini example, application settings
ApplicationTrace:
ApplClass=ALL
ApplName=amqs*
Trace=ON
TraceLevel=HIGH
– You need at least one ApplicationTrace stanza to log application activity trace, or ACTVTRC set to on
– User applications only or include the MCA application?
– Application name, can be wildcarded. By default the application name is the process name. From MQ
7.5 JMS clients can specify application name via the connection factory.
– Turn application activity trace on
– Override global settings as needed
How do I read application activity trace?
• Read PCF messages from
SYSTEM.ADMIN.TRACE.ACTIVITY.QUEUE
• amqsact
• MS0P
• Other vendors
• Roll your own
amqsact exampleMonitoringType: MQI Activity Trace
QueueManager: 'MQ80'
Host Name: ‘MYHOST'
IntervalStartDate: '2016-02-17'
IntervalStartTime: '11:59:20'
IntervalEndDate: '2016-02-17'
IntervalEndTime: '11:59:22'
CommandLevel: 800
SeqNumber: 0
ApplicationName: 'C:\WMQ\8.0\bin64\amqsput.exe'
UserId: 'timmccor'
Trace Detail Level: 2
Platform: MQPL_WINDOWS_NT==========================================================Tid Date Time Operation CompCode MQRC HObj (ObjName)
001 2016-02-17 11:59:20 MQXF_CONNX MQCC_OK 0000 -
001 2016-02-17 11:59:20 MQXF_OPEN MQCC_OK 0000 2 (Q1)
001 2016-02-17 11:59:22 MQXF_PUT MQCC_OK 0000 2 (Q1)
001 2016-02-17 11:59:22 MQXF_CLOSE MQCC_OK 0000 2 (Q1)
001 2016-02-17 11:59:22 MQXF_DISC MQCC_OK 0000 -
==========================================================
– amqsput
– Running on Windows
– Connected to MQ80
– On MYHOST
– With user ID timmccor
– This activity trace was logged with
medium detail
• 1 = Low, 2 = Medium, 3 = High
– Summary of API calls made
amqsact verbose example (MQOPEN)
– 2nd MQI call
– MQOPEN
– Of a queue
– Named Q1
– Which resolved to a queue named Q1
on MQ80
– Opened on object handle 2
– With options 8208
• MQOO_FAIL_IF_QUIESCING &
MQOO_OUTPUT
– Successfully
– And the API call took 41ms to complete
on the queue manager
MQI Operation: 1
Operation Id: MQXF_OPEN
ApplicationTid: 1
OperationDate: '2016-02-17'
OperationTime: '16:31:42'
Object_type: MQOT_Q
Object_name: 'Q1'
Object_Q_mgr_name: ''
Hobj: 2
QMgr Operation Duration: 41
Completion Code: MQCC_OK
Reason Code: 0
Open_options: 8208
Resolved_Q_Name: 'Q1'
Resolved_Q_mgr: 'MQ80'
Resolved_local_Q_name: 'Q1'
Resolved_local_Q_mgr: 'MQ80'
Resolved_type: MQOT_Q
Dynamic_Q_name: 'AMQ.*'
amqsact verbose example (MQPUT extract)
– Much more data than can be displayed on this slide.
– 3rd MQI call
– MQPUT
– In addition to usual timings, you get the epoch value when the
operation started
– The object was Q1 which resolved to a local queue named Q1.
– The message was put under object handle 2
– With options 8260
• MQPMO_NO_SYNCPOINT & MQPMO_NEW_MSG_ID &
MQPMO_FAIL_IF_QUIESCING
– Successfully
– And the API call took 115ms to complete on the queue manager
– Did the application use best practices?
MQI Operation: 2
Operation Id: MQXF_PUT
High Res Time: 1455726703762959
QMgr Operation Duration: 115
Reason Code: 0
Hobj: 2
Put Options: 8260
Object_name: 'Q1‘
Resolved_Q_Name: 'Q1'
Resolved_local_Q_name: 'Q1‘
Resolved_type: MQOT_Q
Report Options: 0
Expiry: -1
Format_name: 'MQSTR'
Priority: -1
Persistence: 2
Msg_id: …
Correl_id: …
Reply_to_Q : ''
Reply_to_Q_Mgr: ''
Coded_char_set_id: 0
Encoding: 546
Put_date: '20160217'
Put_time: '16314375'
amqsact verbose example (MQGET extract)
– Much more data than can be displayed on this slide.
– 3rd MQI call
– MQGET
– In addition to usual timings, you get the epoch value when the
operation started
– The object was Q1 which resolved to a local queue named Q1.
– The MQGET was under object handle 2
– With options 16400
• MQGMO_BROWSE_FIRST & MQGMO_CONVERT
– But it failed, MQRC_NO_MSG_AVAILABLE
– And the API call took 17ms to complete on the queue manager
MQI Operation: 2
Operation Id: MQXF_GET
ApplicationTid: 1
OperationDate: '2016-02-18'
OperationTime: '11:18:50'
QMgr Operation Duration: 17
Completion Code: MQCC_FAILED
Reason Code: 2033
Hobj: 2
Get Options: 16400
Msg length: 0
High Res Time: 1455794330745687
Object_type: MQOT_Q
Object_name: ‘Q1'
Object_Q_mgr_name: ''
Resolved_Q_Name: ‘Q1'
Resolved_Q_mgr: 'MQ80'
Resolved_local_Q_name: ‘Q1'
Resolved_local_Q_mgr: 'MQ80'
Resolved_type: MQOT_Q
Buffer Length: 102400
Performance Impact
• Only trace the applications you’re interested in
• Set the TraceLevel at the lowest level which provides the information
you require
• Tune ActivityInterval and ActivityCount, experiment
– Less frequent messages will use more memory
– More frequent messages will use more IO
• Frequently consume messages from the queue to avoid queue full
When should I use application activity trace?
• Scoped to application
• Application audit trail
• Performance analysis
– How long are my API calls taking to complete on the queue manager?
• Resource usage
– Which queues or topics are actually being used
• Problem Determination
– Which Queue / Queue Manager is the application
actually opening
• Application Coding Standards
– Does everyone use the MQI in the recommended way
• And more …
Summary
Accounting
messages
API Trace Application Activity
Trace
Admin can enable X X X
Developer can enable X X
Documented format X X
Useful for performance X X
Scoped to applications X X
Notices and Disclaimers
37
Copyright © 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission
from IBM.
U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.
Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial
publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED
"AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS
INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and
services are warranted according to the terms and conditions of the agreements under which they are provided.
Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.
Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers
have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.
References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in
which IBM operates or does business.
Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and
discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their
specific situation.
It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and
interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such
laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law
Notices and Disclaimers Con’t.
38
Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not
tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products.
Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the
ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT
NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
The provision of the information contained h erein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual
property right.
IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®,
FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG,
Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®,
PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®,
StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business
Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM
trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.
Thank YouYour Feedback is Important!
Access the InterConnect 2016 Conference Attendee
Portal to complete your session surveys from your
smartphone,
laptop or conference kiosk.
Monday
10:30-11:30 3592 New MQ features
3452 Managing applications
12:00-13:00 2835 MQ on z/OS and Distributed
15:00-16:00 3470 Latest MQ z/OS features
2833 Where is my message?
3544 MQ Light in an MQ infrastructure
16:30-17:30 3573 Hybrid cloud messaging
2941 MQ Advanced
Tuesday
08:30-09:30 3540 The MQ Light API
12:00-13:00 3456 The IBM MQ Appliance
13:15-14:15 3499 Introducing Message Hub
3458 MQ Appliance administration
14:30-15:30 6432 MQ updates and futures (InnerCircle)
2849 Messaging feedback roundtable
16:00-17:00 3544 MQ Light in an MQ infrastructure
3513 MQ hands on lab
Wednesday
08:30-09:30 3602 Managing your MQ environment
12:00-13:00 3613 Designing MQ self service
6408 Hybrid messaging roadmap (InnerCircle)
13:15-14:00 3416 HA and DR with MQ
3433 Why secure your messaging?
15:45-16:30 3429 Securing MQ
2847 Meet the messaging experts
16:00-17:00 3508 MQ Light hands on lab
16:45-17:30 2275 Migrating to the IBM MQ Appliance
Thursday
08:30-09:15 3420 MQ Clustering
2931 Business agility with self service MQ
09:30-10:15 3479 MQ z/OS clusters and shared queue
3450 Optimising MQ applications
2849 Messaging feedback roundtable
10:30-11:15 3465 MQ Appliance high availability
3481 MQ z/OS messaging connectivity
11:30-12:15 3474 Active-active messaging
3537 Monitoring and managing MQ
3425 MQ publish/subscribe
Find us at the EXPO:
Hybrid Integration peds 65-68
Check out the Hybrid Messaging sub topic under the
Hybrid Integration topic for further customer and business
partner sessions
Hybrid Messaging from the IBM experts at InterConnect 2016
Sunday
14:30-15:30 6408 Hybrid messaging roadmap (InnerCircle)