mainframe batch optimization

Whi

te P

aper

- Girish M. Totloor

Mainframe Batch OptimizationA Five Step Approach

www.infosys.com

Abstract

Batch work is one of the significant workloads that still run on Mainframes and hence it remains a necessity to have an optimized batch cycle which helps in reducing costs in terms of the resources needed to monitor and support them. The recurring license costs are associated with usage, optimizing Mainframe batch work load will help reduce license costs. Batch optimization calls for a holistic approach in analyzing the batch contribution to overall system performance. This paper talks about the ways in which we can look at batch tuning or optimization.

2 | Infosys – White Paper

In the current world of 24X7 requirements for online systems which are being used across multiple time zones of the globe, there continues to be a sizable Mainframe workload that runs as batch jobs for a large number of clients. Batch work is one of the significant workloads that still runs on Mainframes and hence it remains a necessity to have an optimized batch cycle which helps in reducing costs in terms of the resources needed to monitor and support them.

In the next few paragraphs we examine a few of the ways to look at batch tuning or optimization. There are various ways to optimize a batch workload. Each batch specialist has his or her own way of tuning the batch workload. This article proposes a few basic practices and processes that can be used in optimizing a batch workload.

Batch Processing is used in various mission critical business environments spanning from banking systems that process interest calculations and other accounting related tasks to retail and supply chain environments where the inventory processing and re-order values need to be updated, as well as an environment for business intelligence where the overnight batch stream replicates key business data to create a production copy for use in data marts for business analysis and in support of business intelligence.

Infosys – White Paper | 3

1. System ConfigurationAs a first step, look at the complete holistic view of the systems health by examining parameters such as CPU usage and paging/swapping. System capacity trends should be analyzed with a focus on the batch workload and taking into the account the contribution to peak MSU/MIPS utilization for the batch window. Based on spare capacity availability, the provisioning of initiators can also improve the performance. The analysis of the CPU usage with batch contention for a batch workload can provide pointers to spare system capacity. This allows you to rebalance the workload by taking advantage of this spare capacity. The best way

to start analyzing the batch performance is to look at the SMF/RMF summary reports for the time period of your batch window.

You also need to take a look at the basic parameter settings at the system level. If you are not a system programmer, you can work with your system programming team to look at the configuration of the system

Depicted below is a illustrative System/CPU utilization report for a particular LPAR for a given period of time. This report shows peak system utilization for a rolling 4 hour average. This represents the peak for each of the workloads such as batch, online, and database etc., along with their individual MSU/MIPS consumption.

Steps in Batch

Optimization5

0

100

200

300

400

500

600

700

800

MIPS

Month

Utilization per Service Group - PROD1Period Jun10 to Jun 11

Monitors

Started Tasks

MQ

IDMS

CICS

DDF

TSO

Batch


Once we have the data, we slice the batch trend further down in our analysis and we start with the batch jobs that are running at the time that the batch workload peaks. Studying each of these components will further help us to determine what actions to take: either in shifting the specific batch workload to a different time zone to reduce its contribution to overall MSU Peak, or by taking up each of these batch components for further optimization.

Another important feature that should be reviewed in the system configuration is the appropriate set up of Work Load Manager (WLM). The Work Load Manager can be used to define the policies based on business goals, including batch processing. Based on the needs and usage, a System Programmer can determine if the initiators should be managed by WLM or by JES.

In case of a WLM-managed environment, the initiators are started/stopped by WLM and are also assigned to service classes. WLM

does this based on several criteria such as the need to achieve the goals for the service classes, jobs waiting in the queue, etc.

For more details on this topic you can refer to the IBM Redbook “System Programmer’s Guide to:

Workload Manager”, SG24-6472.

2. Analysis of top CPU Consuming ApplicationsBatch tuning as a subset of application tuning is primarily focusing on batch throughput, scheduling, timing and resource consumption and improving the batch window to reduce costs. Consider leveraging the existing CPU consumption reports to perform a cumulative trend analysis for all batch loads, and investigate the peaks and the applications contributing to those peaks.

The table below shows a few basic parameters to be collected:

JOBNAME Date of Run START Time END Time CPU/S Elapsed M/S Number of Job Steps Remarks

XYZ123 4/1/2010 00.01.01 00.02.22 23.11 1.34 2Step20 to be investigated

XYZ124 7/1/2010 00.01.04 00.02.42 24.46 1.62 5Step30 : SQL in PORG XYZ to be tuned

ABC125 10/1/2010 02.36.11 02.37.41 24.36 1.5 4

ABC126 1/3/2011 00.01.02 00.02.09 23.21 1.11 3

There could be few jobs that will top this chart due to intensive processing that is required by those jobs, in such cases we can look to see if the job can be split into multiple smaller jobs, or if applicable, we can move some of the eligible workloads to specialty engines such as zIIPs or zAAPs.

3. Batch Scheduling The setting up of the batch streams effectively and efficient managing of batch windows is as much of an art as a science. This is especially applicable to the legacy systems that have been running mainframe batch jobs for decades. The management of a batch window either in production or test environments will have an impact on the processing delay and on larger batch windows. Though the batch scheduling is done using scheduling tools such as IBM TWS/OPC, CA7 or ASG products, a few of the following best practices should be considered in scheduling job streams for efficient batch window management.

Best Practices for Batch Scheduling

• Drawandanalyze the internalandexternaldependencychartsbefore actually setting up in the tool

• Identitythecriticalpathforaparticularsequenceofthejobflowand try to minimize or avoid the redundant dependencies

• Createalogicalgroupofjobsinonestreambasedonapplicationand/or business need

• Exploreparallelrunningofadditionalstreams


• Consider the allocation of special resources for a set of jobs accessing common tables/objects

• Every six months carry out an assessment of obsolete jobs and periodically clean up the system for both production and test environments

• For installations using the Tivoli Workload Scheduler (TWS), TWS can be integrated with WLM. This helps improve JCL management and monitoring. For additional details and for the required APARs and features you can refer to the IBM Redbook “System Programmer’s Guide to : Work Load Manager” SG24-6472.

4. Handling of Frequently Failing jobsWhile the steps previously mentioned will directly help in batch optimization, keep in mind that reducing the frequently failing jobs is a hygienic activity to maintain a stable batch environment. Though the frequently failing jobs will cause a lot of resource consumption in the form of CPU cycles, etc., they will also require human resources in restarting the jobs and frequent investigation of exceptions that take up valuable support staff time.

There are also failures that prevent the batch stream from completing due to dependencies on successor jobs. By resolving these issues you can have a healthy and more predictable batch environment.

The following steps can help reduce these frequently failing jobs:

1. Record (by support staff ) all the errors/incidents, along with the number of occurrences for the defined period within a month or quarter.

2. Track detailed information such as the cause of the job failure with information as the step the job failed in and any work around solution adopted.

3. Review of these jobs with the applications’ Subject Matter Experts (SME)/owners to develop a permanent solution for them.

4. Analyze the dependencies of the frequently failing jobs and determine if the culprit job can be moved out of the current stream, which can reduce the risk of a job holding up a successor job.

5. Publish the CPU savings achieved through reduction of the failures which will motivate the support team to develop the practice of identifying potential batch candidates for fixing on a regular basis.

5. Implementing Data In MemoryOne of the IBM recommendations from the Redbook “Batch Processing in a Parallel Sysplex”, SG24-5329, is to use the data in memory concept to reduce the repeated usage of I/O calls to the same data. According to IBM, Data-In-Memory enables individual jobs to run faster by reducing the I/O component of elapsed time for the job. This reduction is achieved by reading the data from (or in some cases writing to) a buffer in processor storage instead of tape or DASD.

There are two main benefits of these techniques:

• I/O operations are reduced, and therefore their component of elapsed time is reduced in proportion to the reduction in I/Os.

• The remaining I/Os are done to a less busy I/O subsystem, which gives the remaining I/Os a faster response.


Mainframe batch optimization can be approached in various ways. What is presented in this paper is just one of the various ways to optimize mainframe batch processing. The main theme here is to look at the various options which can help optimize the mainframe batch environment which can affect the MSU charges for the enterprise. One observation is that while we look at optimizing the batch stream, we should also consider all available options however insignificant or small they may appear. Once you have a laundry list of potential activities, you can decide which suits your specific environment. Success in optimization of a batch workload is achieved more through disciplined and consistent application of the methods adopted by you. The key feature is to link the savings generated in terms of CPU to MSU/MIPS and articulate to the business the potential cost savings generated by the optimization of batch workloads over a period of time.

Conclusion


About the AuthorGirish Totloor is an Industry Principal with Infrastructure Consulting Services at Infosys Ltd. For more than 10 years he is been working on Mainframe Systems, Databases and applications. His areas of interest include Databases, Optimization, MSU/MIPS improvement and consulting solutions in the Mainframe Infrastructure environment.

REFERENCES

Batch Processing in a Parallel Sysplex: http://www.redbooks.ibm.com/redbooks/pdfs/sg245329.pdf

System Programmer’s Guide to: Work Load Manager http://www.redbooks.ibm.com/abstracts/sg246472.html

© 2012 Infosys Limited, Bangalore, India. Infosys believes the information in this publication is accurate as of its publication date; suchinformation is subject to change without notice. Infosys acknowledges the proprietary rights of the trademarks and product names of other companies mentioned in this document.

About Infosys

Many of the world's most successful organizations rely on Infosys to deliver measurable business value. Infosys provides business consulting, technology, engineering and outsourcing services to help clients in over 30 countries build tomorrow's enterprise.

For more information, contact [email protected] www.infosys.com

mainframe batch optimization

Documents