march 25, 2003ny/nj mq family users’ grouptoo deep & too wide - 1 too deep & too wide...
TRANSCRIPT
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 1
Too Deep & Too WideUsing Soft Limits
To Avoid Hard Crashes
NY/NJ Websphere MQ Family User GroupIBM 590 Madison Ave, New York City
Tuesday March 25, 2003
T.Robert Wyatt, Vice PresidentBank Of America Differentiated Service
Websphere MQ Support
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 2
Objectives
• Demonstrate why the default queue depth and message length values pose a potential risk
• Discuss one approach to mitigating that risk
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 3
Scenario For Disaster
Unrealistic requirements:“Need to be able to queue up anentire day’s worth of large XMLmessages”
– Increase MAXDEPTH– Increase MAXMSGL
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 4
Whoops!
One day the channel goes downand the system performs exactlyas designed:
– Queues begin to fill up– Underlying filesystem fills– MQ begins to cut FDC files– MQ and possibly the OS are halted
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 5
Recovery
• Free up disk space any way possible– Move any diagnostic files– Delete non-essential files– Move older MQ logs if linear
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 6
Recovery
When (or if) QMgr becomes responsive again:
• Stop all channels• RCDMQIMG if linear • Shut down the QMgr
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 7
Recovery
Restarting the QMgr:Delete files or add space
– Clean up the file system– Remount the filesystem to larger disk
and copy files
Results in extended outage, highprobability of client impact
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 8
What Happened?
5,000 x 4mb messages = 20GB5k x 4mb persistent msgs > 40GBDef DeadQ 640,000 x 4mb = 2.5TB!
When the capacity specified bysoft limits exceeds the physicallimits, then the soft limits do notexist.
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 9
Hard Limits
Application constrained only bylimits of physical disk space
– Failure is sudden– Failure may be catastrophic
• Affects OS & co-resident apps
– Slow manual recovery– High client impact
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 10
Soft Limits
Application constrained by MQ– System degrades gracefully– Failure isolated to specific apps within
MQ– Recovery is automated – normal
operation of MQ restores service– Minimal client impact
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 11
Tuning Discipline
Insure soft limits < physical limits
– Reduce MAXDEPTH, MAXMSGLor both
– Reduce settings on SYSTEM.DEF*as well as application queues
– Reduce DeadQ Settings!
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 12
Tuning Discipline
Disk Space Available*
MAXMSGL MAXDEPTH
DSA = MAXMSGL x MAXDEPTHMAXDEPTH = DSA / MAXMSGLMAXMSGL = DSA / MAXDEPTH
*Use ½ DSA for persistent msgs
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 13
Tuning Discipline
Know your messages!– Soft limits must be appropriate to the
business to be enforceable– Standardize, preferably automate– Verify on each machine– No exceptions
Too shallow? Too narrow?Buy more disk!
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 14
Apply System Wide
System-wide tuning parameters– Channel MAXMSGL– QMgr MAXMSGL– Max uncommitted messages– Log file size and extents– Use separate, preferably dedicated,
filesystems for /var/mqm/qmgrs and /var/mqm/logs
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 15
Resources
IBM Websphere MQ Manualshttp://www-3.ibm.com/software/ts/mqseries/library/manualsa/
– Quick Beginnings Guide– System Administration Manual
Support Pacshttp://www-3.ibm.com/software/integration/support/supportpacs/
– Capacity planning pacs– Performance reports
March 25, 2003 NY/NJ MQ Family Users’ Group Too Deep & Too Wide - 16
Resources
MQSeries.net: http://www.mqseries.net
Vienna List Server:http://www-3.ibm.com/software/ts/mqseries/support/usergroups/listserver.html
Download presentation:http://www.t-rob.net/Downloads/2Deep2Wide.ppt
T.Rob: [email protected]